Friday, 29 June 2012

The Externalizable Interface

Before you start exploring Externalization in Java, I would recommend you to gather some knowledge of Serialization in Java because Externalizable is an alternative to Serializable interface.


Externalizable interface allows you to customize how serialization is done. By implementing Externalizable you are controlling what gets serialized and what does not get serialized. Externalizable interface  extends Serializable interface and defines two methods to be overridden by the implementing class. 
  1. public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException
  2. public void writeExternal(ObjectOutput out) throws IOException
Java class implementing Externalizable interface


package learn.java.serialization;


import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import java.util.Date;


public class ExternalizableObject implements Externalizable{
   private String name;
   private  Date dob;
   private  int age;

public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Date getDob() {
return dob;
}
public void setDob(Date dob) {
this.dob = dob;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
@Override
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
name = (String) in.readObject();
age = in.readInt();
}
  @Override
public void writeExternal(ObjectOutput out) throws IOException {
out.writeObject(name);
out.writeInt(age);
}
}



As you have noticed we are not serializing/de-serializing dob variable in the overriding methods. Now, in the following code snippet we will try to serialize/deserialize the above class instance.

public static void main(String[] args) {
   ExternalizableObject externalizableObject = new ExternalizableObject();
   externalizableObject.setAge(26);
   externalizableObject.setDob(new Date("20/09/1985"));
   externalizableObject.setName("dev");
   externalizeObject(externalizableObject);

   ExternalizableObject deserializedObj = deserializeExternalizeObject();
   System.out.println("Name: "+ deserializedObj .getName());
   System.out.println("DOB: "+ deserializedObj.getDob());
   System.out.println("Age: "+ deserializedObj.getAge());
}


private static void externalizeObject(ExternalizableObject obj){
   try {
      FileOutputStream fos = new FileOutputStream("C:/temp/test.txt");
      ObjectOutputStream out = new ObjectOutputStream(fos);
      out.writeObject( obj ); //writeExternal() of ExternalizableObject is invoked
      out.close();
   } catch (IOException e) {
      e.printStackTrace();
   }
}

private static ExternalizableObject deserializeExternalizeObject(){
   ExternalizableObject obj = null;


   try {
      FileInputStream fis = new FileInputStream(new File("C:/temp/test.txt"));
      ObjectInputStream in = new ObjectInputStream(fis);
      obj = (ExternalizableObject) in.readObject(); //readExternal() of ExternalizableObject is invoked
   } catch (Exception e) {
      e.printStackTrace();
   } 
   return externalizableObject;
}


During the Serialization process JVM first checks if the object implements Externizable interface, it invokes writeExternal method to serialize the object otherwise the object is serialized using ObjectOutputStream.


While reconstructing the serialized object, an instance is created first using the public no-arg constructor, then the readExternal method is called. If the class only have parameterized constructor(s) defined,  than java.io.InvalidClassException: no valid constructor Exception is thrown.


If a class inherits some class implementing the Externalizable interface, it must override writeExternal() and readExternal() methods to serialize or deserialize this class in order to correctly save and restore the object.

Wednesday, 27 June 2012

How to scale image using Java

In this post we will see how can we scale an image to a desired dimension. 


In the following code snippet the scaleImage method is responsible for scaling the source image to the given height & width. Instead of overwriting the original file the code creates a new file for the scaled image version.



public static String  scaleImage(String imgSrc, int width, int height, String dest){
 try{
File f = new File(imgSrc);

//Reads the Image from the given URL
BufferedImage img = ImageIO.read(f);

//Scales the BufferedImage to the desired dimensions
   Image scaledImg = img.getScaledInstance(width, height, Image.SCALE_AREA_AVERAGING);
   
   //'coz the Image object cannot be written to disk directly 
   //We have to recreate a new BufferedImage instance from the scaled Image instance
   BufferedImage biScaledImg = toBufferedImage(scaledImg, BufferedImage.TYPE_INT_RGB);
     
   //Writes the file on to the disk at the given destination
   ImageIO.write(biScaledImg, "jpeg", new File(dest));
}
catch(IOException e){
return "Exception occurred while scaling the Image. Exception: "+e.getMessage();
}

return dest;
}


/**
* Method to create BufferedImage Instance from given Image Object
*/
private static BufferedImage toBufferedImage(Image image, int type) {
   int w = image.getWidth(null);
   int h = image.getHeight(null);
   BufferedImage result = new BufferedImage(w, h, type);
   Graphics2D g = result.createGraphics();
   g.drawImage(image, 0, 0, null);
   g.dispose();
   return result;
}



You can download the complete source code from here


It should be noted that if the new height & width is greater than the source image height & width then the resultant image will be a pixelated image.

Friday, 22 June 2012

How to delete Solr Index

To delete all documents from my Solr index, we have to issue an update request with the  stream.body parameter set to <delete><query>*:*</query></delete>, as with the following URL: 


http://<host>:<port>/solr/update?stream.body=<delete><query>*:*</query></delete>&commit=true


Just replace the placeholder with the correct details. If executed successfully the it will return a response with status = 0 and query execution time.

<response>

<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">3693</int>
</lst>
</response>



If you want to remove indexes of a particular field then specify the field name in the delete query
<delete><query>desc:*</query></delete>


You can also delete a wrongly indexed value from a particular field, just by adding the value in the delete query
<delete><query>desc:wrold</query></delete>

Using Solr Spellchecker from Java

Continuing from my prevoius post Implementing Spellchecker in Solr , we'll now see how to use SorlJ API to fetch spellcheck suggestions through a Java program.


public static Map<String, List<String>> getSuggestions(String queryStr){
if (queryStr != null &&  !queryStr.isEmpty()) {

try {

SolrServer server = new HttpSolrServer("http://localhost:8983/solr");


   ModifiableSolrParams params = new ModifiableSolrParams();
   params.set("qt", "/spell");
   params.set("q", queryStr);
   params.set("spellcheck", "true");
   params.set("spellcheck.collate", "true");


   QueryResponse response = server.query(params);
   System.out.println("response = " + response);

   if(response != null){
      SpellCheckResponse scr = response.getSpellCheckResponse();
    if(scr != null){
    List<Suggestion> suggLst = scr.getSuggestions();
    for(Suggestion sugg : suggLst){
    System.out.println("Suggestion for token"+sugg.getToken()+" are: ");
System.out.println(sugg.getAlternatives());
}
             System.out.println("Collated Result: "+scr.getAlternatives());
    }
   }
}
catch(Exception e){
e.printStackTrace();
}
}
return suggestionsMap;
}


You can download a copy of complete Java Program from here.


To run the above program you will need the following jar files

  • apache-solr-solrj-3.6.0.jar (I am using Solr 3.6.0)
  • httpclient-4.jar
  • httpcore-4.jar
  • httpmime-4.1.2.jar
  • slf4j-log4j12-1.0.jar
  • commons-logging.jar
  • log4j.jar

Wednesday, 20 June 2012

Implementing Spellchecker in Solr

A common need in search applications is suggesting correct word/ phrase for a misspelled word / phrase. These suggestions may come from a dictionary that is based upon some field or upon any other arbitrary dictionary. 

Implementing a spell-check suggester in Apace Solr is a piece of cake. All you need is to follow the given steps and you are done.

Step 1:
Open <SOLR_HOME>/example/solr/conf/solrconfig.xml and search for 
<lst name="spellchecker"> 
under this tag make changes to 
<str name="field">name</str> 
in place of  name define your field which will be referred for spell checking

Now we have to update the spell requestHandler settings, to do so search for 
<requestHandler name="/spell" class="solr.SearchHandler" startup="lazy">
and under this tag make changes to
<str name="df">text</str> 
in place of   text  define your field which will be referred for spell checking, also increase the  <str name="spellcheck.count">1</str> to the desired suggestion count.

Step 2: 
Your configuration part is over now so (re)start your solr server to reflect the changes done.

Step 3:
Now you'll need to instruct the spellcheck component to build its dictionary. This can be done by issuing an empty query with the spellcheck.build parameter set to true, as with the following URL: 
http://<host>:<port>/solr/spell?q=&spellcheck.build=true

NOTE: spellcheck.build=true is needed only once to build the spellcheck index and should not be specified with each request

Step 4:
Now your dictionary is built and ready to give suggestions for misspelled words. Consider this misspelled request 

http://<host>:<port>/solr/spell?q=blogr wrld&spellcheck=true&spellcheck.collate=true

The above request returns following suggestions as response

<response>

<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">13</int>
</lst>
<result name="response" numFound="0" start="0"/>
<lst name="spellcheck">
<lst name="suggestions">
<lst name="blgge">
<int name="numFound">3</int>
<int name="startOffset">0</int>
<int name="endOffset">5</int>
<arr name="suggestion">
<str>blogger</str>
<str>bigge</str>
<str>blame</str>
</arr>
</lst>
<lst name="wrld">
<int name="numFound">3</int>
<int name="startOffset">6</int>
<int name="endOffset">10</int>
<arr name="suggestion">
<str>world</str>
<str>wild</str>
<str>rld</str>
</arr>
</lst>
<str name="collation">blogger world</str>
</lst>
</lst>

</response>

Tuesday, 19 June 2012

Importing / Indexing MySQL data into Solr

Apache Solr is a fast, full featured, open-source Java search server. It enables you to easily create search engines which index, search & analyze data from websites, databases or files.

Before we proceed further I presume that you have already configured Solr server without data or refer to Solr in 5 minutes tutorial to configure your Solr server first.

Configure your Solr server


Step 1:

To index your data firstly you need to define schema by editing file called schema.xml present at <SOLR_HOME>/example/solr/conf/
Schema.xml in solr serves the similar purpose that table does in mysql
In schema.xml file you have to define the
  • Fields with there field type
  • Field which should be used as unique key
  • Which fields are required, indexed
  • Default search field (deprecated in new versions)
  • Default search operator (AND | OR deprecated in new versions)
**indexed fields are fields which undergo an analysis phase, and are added to the index.
Search for <fields> tag and define your fields under this

<fields>
    <field name="blog_id" type="string" indexed="true" stored="true" omitNorms="true"/>
    <field name="tags" type="text_general" indexed="true" stored="true"/>
    <field name="desc" type="text_general" indexed="true" stored="true"/>
    <field name="created" type="tlong" indexed="true" stored="true" omitNorms="true"/>
    <field name="uid" type="int" indexed="true" stored="true" omitNorms="true"/>
</fields>

Search for  <uniqueKey> tag and define your primary key name
<uniqueKey>blog_id</uniqueKey>

Step 2:

Now you need to define how this schema is mapped to your database. For this purpose you have to create a new file data-config.xml under <SOLR_HOME>/example/solr/conf directory.

<?xml version="1.0" encoding="UTF-8"?>
<dataConfig>
    <dataSource type="JdbcDataSource"
                  driver="com.mysql.jdbc.Driver"
                  url="jdbc:mysql://localhost/myDatabase"
                  user="dbuser"
                  password="p@ssword"/>
    <document name="doc">
        <entity name="blog"
                query="select b.blog_id, b.tags, concat(b.title,' ' ,b.desc)  as desc, b.created, b.uid, from blogs b order by b.blog_id;">
           
              <field column="blog_id" name="blog_id" />
              <field column="tags" name="tags" />
              <field column="desc" name="desc" />
              <field column="created" name="created"/>
              <field column="uid" name="uid"/>
        </entity>
    </document>
</dataConfig>

Step 3:

Edit file solrconfig.xml which is located at <SOLR_HOME>/example/solr/conf directory. Add the following requestHandler entry if not already existing.

<requestHandler name="/dataimport">
     <lst name="defaults">
          <str name="config">data-config.xml</str>
     </lst>
</requestHandler>

Step 4:


Download MySQL jdbc driver jar and copy it into <SOLR_HOME>/lib directory


Step 5:


(Re)Start your webserver

Performing full or delta indexing

If everything works correctly, you can get solr to fully index the configured tables by accessing the following command via your browser.  
http://<host>:<port>/solr/dataimport?command=full-import


You can check the status of the command by accessing
http://<host>:<port>/solr/dataimport


To do an incremental or delta indexing of data since the last full or delta, issue the command 
http://<host>:<port>/solr/dataimport?command=delta-import


Similarly any other database can be configured all you need is 
  • jdbc jar of that database
  • make changes to data-config.xml
     <dataSource type="JdbcDataSource" driver="xxxx" url="xxxx" user="xxxx"                password="xxxx"/>

Thursday, 7 June 2012

serialVerionUID in Java Object Serialization

What is serialVersionUID?

The purpose of the "serialVersionUID"  is to keep track of different versions of a class in order to perform valid deserialization of objects. This SUID should be unique to a certain version of a class, and should be changed when there are any details change to the class, such as a new field, which would affect the structure of the serialized object.


The serialization process associates a serialVersionUID (default if not explicitly defined) with each serialized instance of a class. This serialVersionUID is validated at the time of deserialization from the loaded class serialVersionUID, and in case of a mismatch InvalidClassException is thrown.


Default serialVersionUID

If a serializable class does not define serialVersionUID explicitly then the serialization process automatically calculates a default serialVersionUID (by invoking computeDefaultSUID() method of java.io.ObjectStreamClass) value for that class viz usually the hash value of class details. 

Factors that may effect the default serialVersionUID value
  • Since default generation is dependent on the class details, it will change as soon as any detail of the class changes (for eg. added a new field in class)
  • Even if you do not make any changes to class details there is a possibility of having different serialVersionUID if generated using different JVM implementations. That's why it is always recommended to define serialVersionUID manually in a Serializable class.


Defining serialVersionUID

To define serialVersionUID, declare a field named "serialVersionUID" that must be static, final, and of type long in the Serializable Class defination
ANY-ACCESS-MODIFIER static final long serialVersionUID = 1L;

NOTE: serialVersionUID is case-sensitive
By defining serialiVersionUID in your Serializable class you can make sure that for different implementation of JVMs' serialVersionUID will be the same.

When should the serialVersionUID be changed?


Changes for which the guarantee of interoperability (serialization-deserialization) cannot be maintained, serialVersionUID of that class should be changed.

For details Types of changes affecting Serialization process click here

Using serialVer utility to generate serialVersionUID


You can use serialver utility provided by JDK to generate serialVersionUID of a class. It is present in the bin directory of your JDK.

On running following on command prompt

serialver learn.java.serialization.SerializableObject

will generates following output

learn.java.serialization.SerializableObject: static final long serialVersionUID = 2873835861766732915L;

Tuesday, 5 June 2012

Serialization in Java

Serialization is the process of converting an object's state into byte sequence (to persist it on to a file), and to rebuild (deserialize) those bytes into an object.

To make an object serializable, the object should implement java.io.Serializable interface. Lets take an example to understand Serialization in Java.

Firstly lets define a serializable object

###############################################################

package learn.java.serialization;

import java.io.Serializable;

public class SerializableObject implements Serializable{

private String message;
public SerializableObject(){
}

public String getMessage() {
return message;
}

public void setMessage(String message) {
this.message = message;
}
}

###############################################################

As you must have noticed the only special thing that we have done  to the class is implement Serializable. The java.io.Serializable is a markup or marker interface, it allows the serialization mechanism to check object's ability to be persisted.

Now as we have defined a serializable object lets try to persist it onto a file 

###############################################################

package learn.java.serialization;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

public class Serialize {
   public static void main(String[] args) {
      SerializableObject serializableObject = new SerializableObject();
  serializableObject.setMessage("This message will be serialized.");
  System.out.println("Trying to serialize object with message ### "+serializableObject.getMessage()+" ###");
  System.out.println(serializeObject(serializableObject));
  System.out.println("############################################");
  System.out.println("Trying to deserialize the serialized object");
   
      SerializableObject deserializableObject = deserializeObject();
  if(deserializableObject == null)
System.out.println("Some problem occured in de-serialization process.");
  else
System.out.println("Object de-serialized successfully with message ### "+deserializableObject.getMessage()+" ###");
   
   }
  private static String serializeObject(SerializableObject serializableObject){
  String msg = "";
  try {
  FileOutputStream fos = new FileOutputStream("C:/tmp/serializableTest.txt");
ObjectOutputStream out = new ObjectOutputStream(fos);
out.writeObject(serializableObject); // Initiates the Serialization process          
out.close();
msg = "Object serialized successfully, with message ### "+serializableObject.getMessage()+" ###";
  } catch (IOException e) {
e.printStackTrace();
msg = "Problem in serialization, problem ### "+e.getMessage()+" ###";
  }


  return msg;
  }
   
   private static SerializableObject deserializeObject(){
  SerializableObject serializableObject = null;

  try {
  FileInputStream fis = new FileInputStream(new File("C:/tmp/serializableTest.txt"));
ObjectInputStream in = new ObjectInputStream(fis);
// Initiates the (De)serialization  process
serializableObject = (SerializableObject) in.readObject(); 
  } catch (FileNotFoundException e) {
  e.printStackTrace();
  } catch (IOException e) {
e.printStackTrace();
  } catch (ClassNotFoundException e) {
  e.printStackTrace();
}
  return serializableObject;
  }
}
###############################################################

Here serializeObject method defines the logic of persisting an object to a file and deserializeObject method defines the logic of reading the serialized data from the file and recreates a new object into the memory.


Good to know about Serialization in Java

  • An object must implement the java.io.Serializable interface or inherit that implementation from its object hierarchy to make it persistable.
  • Java serialization mechanism writes the metadata about the object, which includes the class name, field names and types, and superclass along with the variables value. 
  • Mark a variable as transient to exclude it from serialization process
  • Since static variables belong to the class and not to an object they are not the part of the state of object so they are not saved during Java Serialization process.
  • A Serializable class must define all non-serializable variable as transient
  • It is recommended to define private static final serialVersionUID value which guarantees a consistent serialiVersionUID across different java implementations.
  • If a serializable class does not explicitly declare a serialVersionUID, then a default serialVersionUID value is calculated for that class based on various aspects of the class, as described in the Java Object Serialization Specification.
  • Serializable class can override following methods to specify custom serialization & deserialization process

   private void writeObject(ObjectOutputStream out) throws IOException
   private void readObject(ObjectInputStream in)  throws IOException,   ClassNotFoundException

  • Serialization process throws java.io.NotSerializableException  when you try to serialize a non serializable instance 
  • Implementing java.io.Externalizable is an alternative for java.io.Serializable, it allows the implementing class to define custom serialization process.