Load XML into Mysql Using Java

Load XML into MySQL by using Java DOM Parser.

“Never memorize something that you can look up.”
― Albert Einstein

1. Introduction

XML provides the ability to represent hierarchical structure with its parent-child relationships. This enables applications to store structured data in XML for export. Importing this XML data into a database is a bit involved as we shall see in this article. You need to write code to manage the database connection. In addition you need parse the XML and isolate the data that needs to be imported.

Continue reading “Load XML into Mysql Using Java”

Java ArrayList Examples

1. Introduction

The ArrayList in Java is one of the most used classes in the JDK. It is an implementation of a resizable array with a range of useful methods for manipulating the array. In this article, we present some of the usage patterns of the ArrayList.

The ArrayList is a concrete implementation of the List interface. As such, it is fine to refer to List instead of ArrayList when the implementation detail does not matter. The examples below use List instead of ArrayList when this difference is not relevant.

2. Array vs ArrayList

A regular array in java is a container which holds a fixed number of elements of a specific type. The following (stringArray) is a declaration of an array of String objects and is initialized to an array enough to store 20 strings.

String[] stringArray = new String[20];

An ArrayList for storing Strings is defined as shown below. The number 20 is an indication to the ArrayList class to reserve enough space for 20 strings.

ArrayList<String> stringList = new ArrayList<String>(20);

Some of the differences between the two:

  • Addition: The stringArray can store upto 20 strings. If you need to store more later on, you will have to re-initialize the array and copy the contents from the old array. With the ArrayList, you can add more than 20 strings without worrying about the capacity.
  • Removal: With the ArrayList it is easy to remove entries, either at a particular index or a range or a particular object. Not so easy with the array. You will need loops and other boiler-plate code to manage the removal in your code.

3. Creating an ArrayList

As already shown above you can create an ArrayList using the constructor. To create an empty ArrayList with a capacity for 10 Objects:

ArrayList<Object> objArray = new ArrayList<>();

To create an ArrayList from another Collection such as a Set or a List, you can use:

Set<String> stringSet = ..;
List<String> list = new ArrayList<String>(stringSet);

What if you need to create a List from a bunch of objects already lying around in your code? You can use Arrays.asList() as follows:

List<String> list = Arrays.asList("Apple", "Banana", "Orange");

The following selects lines from a text file using Java 8 streams.

List<String> lines = Files
    .lines(Paths.get(textFile))
    .filter(x -> x.length() > 50)
    .collect(Collectors.toList());

4. Convert Array to ArrayList

As presented above, it is simple to convert to a List if you have a plain array of objects.

List<String> a = Arrays.asList(args);

The above is suitable if you have an array of objects. What if you have an array of a primitive type, say an int? Here is how you can handle that case.

int[] array = { 2, 4, 5 };
List<Integer> list = Arrays
    .stream(array)
    .boxed()
    .collect(Collectors.toList());

5. Convert ArrayList to Array

The List interface provides a method toArray() which creates and returns a new array containing all the elements in the ArrayList.

List<String> strList = ...;
String[] strArray = strList.toArray();

6. Add to ArrayList

Use the method add() to add items to a List. This invocation adds elements at the end of the list.

List<String> strList = new ArrayList<>();
strList.add("Apple");
strList.add("Banana");
strList.add("Orange");
strList.stream().forEach(System.out::println);

// prints
Apple
Banana
Orange

To add all elements of a collection, use addAll().

List<String> modList = new ArrayList<>();
modList.addAll(strList);
modList.add("Melon");

7. UnsupportedOperationException

The following code results in an UnsupportedOperationException. The reason is than Arrays.asList() returns a fixed sized list backed by the input array. So you cannot add elements to the List returned by Arrays.asList().

A simple solution is to create a new ArrayList from the original list as follows:

List<String> strList = Arrays.asList("Apple", "Banana", "Orange");
List<String> modList = new ArrayList<>(strList);
modList.add("Melon");

Or you could use Java 8 streams:

List<String> modList = strList
	.stream()
	.collect(Collectors.toList());
modList.add("Melon");
modList.stream().forEach(System.out::println);

// prints:
Apple
Banana
Orange
Melon

8. Remove from ArrayList

Removal of items from a List is also easy. Remove an element at a particular index.

List<String> modList = new ArrayList<>();
modList.addAll(strList);
modList.add("Melon");
modList.remove(2);

// contains:
Apple
Banana
Melon

Remove a particular object as shown below. Note: The equals() method is used for checking whether to remove the object. It works as shown for String. For another class, equals() might be defined in terms of memory addresses, in which case the same object instance must be passed in to remove it.

List<String> modList = new ArrayList<>();
modList.addAll(strList);
modList.add("Melon");
modList.remove("Orange");

// contains
Apple
Banana
Melon

What if you want to remove an object with some specific characteristic. Use the removeIf() method with the predicate. The methods removes all elements that match the predicate.

List<String> modList = new ArrayList<>();
modList.addAll(strList);
modList.add("Melon");
modList.removeIf(x -> x.endsWith("e"));

// contains
Banana
Melon

Remove all objects from one List that are found in another List using removeAll().

List<String> modList = new ArrayList<>();
modList.addAll(strList);
modList.add("Melon");
modList.removeAll(Arrays.asList("Banana", "Kiwi"));

// contains
Apple
Orange
Melon

Summary

This article explained Java arrays and ArrayList from the perspective of a beginner. We showed creation and initialization of ArrayList. The ArrayList provides a bunch of methods to add and remove elements from the list. In some cases, the List created may not be modifiable in which case an exception is thrown when you try to modify it.

Java File Examples

1. Introduction

Java provides the File class as an abstraction of file and directory path names. A path is presented as a hierarchical operating system independent view of files.

The File class supports a bunch of operations on files and directories which work on both Unix-like and Windows systems. Let us look at some of these operations in more detail.

2. Creating a New File

To create a new file, use the method createNewFile(). The following creates a new file “joe” in the current directory.

File file = new File("joe");
if ( file.createNewFile() ) System.out.println("created");

If a file specified already exists, the method returns false. In this case, the existing file is not touched or modified.

2.1. Hierarchy Not Created

The method does not automatically create the directory hierarchy if any of the parent directories do not exist. The following throws an exception if directory “joe” does not exist in the current directory.

File file = new File("joe/jack");
if ( file.createNewFile() ) System.out.println("created.");

// throws: java.io.IOException: Not a directory

However, if you run the above code after creating the parent directory, it works normally.

2.2. Creating Absolute Path File

In addition to specifying relative paths as above, you can also specify an absolute path as shown below:

File file = new File("/tmp/joe");
if ( file.createNewFile() ) System.out.println("created.");

// prints "created."

2.3. Permission Denied

When attempting to create a file where you don’t have permission, you get an exception:

File file = new File("/joe");
if ( file.createNewFile() ) System.out.println("created.");

// throws java.io.IOException: Permission denied

3. Delete a File

The File class provides a delete() method to delete a file. It is used as follows:

File file = new File("joe");
if ( file.delete() ) System.out.println("deleted.");

The method does not throw an exception if the file does not exists. It just returns false.

4. List Files

When a File object is created with a directory, you can use the list() method to list the files and directories in that directory. Here is an example of listing the files in the current directory using Java 8 streams.

Arrays.stream(file.list()).forEach(System.out::println);

You can also specify a FilenameFilter to select the required files. FilenameFilter is a functional interface so let us use a lambda expression to select and print the “.class” in the directory.

File file = new File(dirPath);
Arrays
    .stream(file.list((d, f) -> f.endsWith(".class")))
    .forEach(System.out::println);

In the above example, we use the FilenameFilter for filtering files by name. However, FileFilter is another functional interface provided by java that can be used to select files by some file property. In the following example, we select empty files.

File file = new File(path);
File[] emptyFiles = Arrays
    .stream(file.listFiles(f -> f.length() == 0));

Or select hidden files for processing.

File file = new File(path);
List<File> hiddenFiles = Arrays
    .stream(file.listFiles(f -> f.isHidden()))
    .collect(Collectors.toList());

5. Creating a Directory

The File class provides the method mkdir() to create a directory. It returns false if the directory cannot be created which could be for a variety of reasons. In other words, exceptions are not thrown including for the following reasons:

  • File or directory already exists.
  • No permission.
  • Hierarchy does not exist.
File file = new File(path);
if ( file.mkdir() ) System.out.println("created.");

If you want to create all non-existent directories of the hierarchy, use mkdirs() instead.

6. Renaming a File

To rename a file, use renameTo(). You must pass in a target File object to rename the file to.

File source = new File(pathA);
File target = new File(pathB);
if ( source.renameTo(target) ) System.out.println("renamed.");

Note that this method comes with a bunch of caveats including:

  • Might not be able to move a file from one file system to another.
  • Might not work if the target path exists. (In other words, might not overwrite the target file.)
  • It might not be atomic. In other words, removing the old file and creating the new file are two separate operations in which one might fail!

With all these warnings, it is better to check the return value to be sure the operation succeeded.

Summary

The Java File class is an system independent abstraction of file and directory paths. It provides a bunch of operations to create a file, remove a file, create a directory including the hierarchy and renaming files.

Java NIO – Using ByteBuffer

1. Introduction

Java provides a class ByteBuffer which is an abstraction of a buffer storing bytes. While just a little bit more involved than using plain byte[] arrays, it is touted as more performant. in this article, let us examine some uses of ByteBuffer and friends.

Continue reading “Java NIO – Using ByteBuffer”

Apache POI Excel Example

Learn how to create an Excel spreadsheet from within Java using Apache POI, a library for working with Microsoft Documents.

1. Introduction

Apache POI is a Java library to read and write Microsoft Documents including Word and Excel. Java Excel API can read and write Excel 97-2003 XLS files and also Excel 2007+ XLSX files. In this article, we show how to get going using the Apache POI library to work with Excel files.

Continue reading “Apache POI Excel Example”

Jackson Tree Model

1. Introduction

Jackson represents the JSON object model as a tree of JsonNode objects. This is called the Tree Model since it comprises of a tree of JsonNodes, including ObjectNode and ArrayNode. The tree mirrors the structure of the JSON document. It can be created by parsing a JSON document or by creating an in-memory representation.

2. Parsing a JSON File

A JSON file can be parsed and a tree model representation can be created using the ObjectMapper.readTree().

ObjectMapper mapper = new ObjectMapper();
JsonNode root = mapper.readTree(new File(args[0]));

The resulting tree model can be converted back to a JSON document as follows:

System.out.println(mapper.writeValueAsString(root));

3. Convert a Java Object (POJO) to JsonNode

Let us learn how to build a tree model from a POJO (Plain Old Java Object). Consider these classes:

public class Address {
    private String address1;
    private String address2;
    private String city;
    private String state;
    private String zip;
}

public class User
{
    private String firstName;
    private String lastName;
    private Address address;
}

Instantiate the objects as required:

User user = new User();
user.setFirstName("Harrison");
user.setLastName("Ford");
Address address = new Address();
address.setAddress1("123 Main Street");
address.setCity("Hollywood");
address.setState("CA");
address.setZip("33023");
user.setAddress(address);

A single function call suffices to convert the POJO to a JsonNode.

JsonNode root = mapper.valueToTree(user);
System.out.println(mapper.writeValueAsString(root));

The output JSON is:

{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "address" : {
    "address1" : "123 Main Street",
    "address2" : null,
    "city" : "Hollywood",
    "state" : "CA",
    "zip" : "33023"
  }
}

4. Convert a JsonNode to POJO

Once you have assembled a tree model from the JsonNode instances, how can you convert it to a POJO? Quite easy and all the contained JsonNode instances are also converted properly.

User user = mapper.treeToValue(root, User.class);

5. Convert Generic Containers to JsonNode

You can build up an object representation from nothing more than Java’s generic containers such as Map and List. Is it possible to convert such a representation to a tree model (JsonNode)? Of course it is. Use ObjectMapper.valueToTree() as before.

Map<String, Object> user = new HashMap<>();
user.put("firstName", "Harrison");
user.put("lastName", "Ford");
user.put("emailAddress", Arrays.asList("harrison@example.com",
              "hford@actors.com"));

Map<String, Object> addr = new HashMap<>();
addr.put("address1", "123 Main Street");
addr.put("address2", null);
addr.put("city", "Hollywood");
addr.put("state", "CA");
addr.put("zip", "33023");
user.put("address", addr);

JsonNode root = mapper.valueToTree(user);
System.out.println(mapper.writeValueAsString(root));

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "emailAddress" : [ "harrison@example.com", "hford@actors.com" ],
  "address" : {
    "zip" : "33023",
    "address2" : null,
    "city" : "Hollywood",
    "address1" : "123 Main Street",
    "state" : "CA"
  }
}

And of course, once you have the JsonNode tree model, you can use Jackson data binding to create a Java object (POJO).

User userAgain = mapper.treeToValue(root, User.class);

6. Iterating over a JsonNode

Once you have a JsonNode, how can you find out what it contains? You can loop over its contents and extract the children.

Find the key names of a JsonNode (assuming it is an ObjectNode). And lookup the value JsonNode using JsonNode.get(String).

JsonNode parent = ...;
for (Iterator<String> it = parent.fieldNames() ; it.hasNext() ; ) {
    String field = it.next();
    System.out.println(field + " => " + parent.get(field));
}

Or iterate over the fields, again assuming it is an ObjectNode:

for (Iterator<Map.Entry<String,JsonNode>> it = parent.fields() ;
     it.hasNext() ; ) {
    Map.Entry<String,JsonNode> e = it.next();
    System.out.println(e.getKey() + " => " + e.getValue());
}

Fetch and iterate over all the elements. This one works with an ArrayNode too.

for (Iterator<JsonNode> it = parent.elements() ; it.hasNext() ; ) {
    JsonNode node = it.next();
    System.out.println(node);
}

7. Creating a Tree Model from Scratch

You can also completely generate a tree model from scratch. Start by creating an ObjectNode and populate it using JsonNode.with(). Here we create the above User instance with the associated Address as shown below.

ObjectNode root = mapper.createObjectNode();
root.put("firstName", "Harrison");
root.put("lastName", "Ford");
root.with("address").put("address1", "123 Main Street");
root.with("address").put("address2", null);
root.with("address").put("city", "Hollywood");
root.with("address").put("state", "CA");
root.with("address").put("zip", "33023");
System.out.println(mapper.writeValueAsString(root));

8. Adding Items to a List

For adding lists into the JSON tree model, you can start with the method withArray() and add items as shown:

root.withArray("Genre").add("Drama").add("Horror");

// results in:
{
 ..
 "Genre" : [ "Drama", "Horror" ],
 ..
}

9. Adding from a List

Here is an example of using Java 8 Streams to add nodes from a List:

Arrays.asList("Stephen King (novel)",
                 "Stanley Kubrick (screenplay)",
                 "Diane Johnson (screenplay)")
    .stream()
    .forEach(root.withArray("Writer")::add);

// looks like:
{
 ..
 "Writer" : [ "Stephen King (novel)", "Stanley Kubrick (screenplay)", "Diane Johnson (screenplay)" ]
 ..
}

10. Adding from a Map

And adding to a tree model from a Map is similarly easy:

map.put("cat", "meow");
map.put("dog", "bark");
map.put("cow", "moo");
map
    .entrySet()
    .stream()
    .forEach(e -> root.with("animals").put(e.getKey(), e.getValue()));

// output:
{
 "animals" : {
   "cat" : "meow",
   "cow" : "moo",
   "dog" : "bark"
 }
}

11. Using JsonPointer

JsonPointer is a proposed standard for addressing nodes within a JSON document (somewhat similar to XPath for XML documents). Jackson supports extraction of JsonNodes with a JsonPointer expression using the method JsonNode.at().

JsonNode root = mapper.readTree(new File(jsonFile));
String jsonPtr = ...;
System.out.println(mapper.writeValueAsString(root.at(jsonPtr)));

Let us see some examples of JsonPointer. Consider this JSON.

{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "emailAddress" : [ "harrison@example.com", "hford@actors.com" ],
  "address" : {
    "zip" : "33023",
    "address2" : null,
    "city" : "Hollywood",
    "address1" : "123 Main Street",
    "state" : "CA"
  }
}

Here are some examples of JsonPointer expression evaluation on this document.

/firstName: "Harrison"

/emailAddress: [ "harrison@example.com", "hford@actors.com" ]

/emailAddress/0: "harrison@example.com"

/emailAddress/1: "hford@actors.com"

/address: {
  "zip" : "33023",
  "address2" : null,
  "city" : "Hollywood",
  "address1" : "123 Main Street",
  "state" : "CA"
}

/address/zip: "33023"

Summary

This article demonstrated how to easily work with the tree model in JSON. We showed how to parse JSON into JsonNode and also how to extract information from the tree model.

Guide to Jackson Annotations

1. Introduction

Jackson provides a number of annotations which help tweak various facets of JSON serialization and deserialization. In this guide, we present some important and commonly used Jackson annotations.

2. @JsonProperty

The annotation @JsonProperty is used to change the JSON property name used in serialization and deserialization. For example,

public class User {
    @JsonProperty("FirstName")
    private String firstName;
}

Output before and after:

// before
...
  "firstName" : "Harrison",
...

// after
  "FirstName" : "Harrison",
...

2.1. Confusion with required

The annotation @JsonProperty defines an attribute required which does not quite work the way it is expected. Despite being specified, it does not mark the property as required either during serialization or deserialization. It applies only when using @JsonProperty with @JsonCreator. For example, the following does not require the presence of the the property “FirstName“:

public class User {
    @JsonProperty(value="LastName", required=true)
    private String lastName;
}

2.2. @JsonProperty and @JsonCreator

@JsonProperty can be used with @JsonCreator to indicate the constructor to use when deserializing JSON. Consider the following class which does not implement a no-arguments default constructor but has a constructor taking appropriate arguments.

public class MyClass {
    private String firstName;
    private String lastName;

    public MyClass(String firstName,String lastName)
    {
	this.firstName = firstName;
	this.lastName = lastName;
    }
}

Create an instance of this class and serialize it to JSON.

MyClass obj = new MyClass("Harrison", "Ford");
System.out.println(mapper.writeValueAsString(obj));

// serialized to:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
}

Deserializing this JSON results in a JsonMappingException. That is because Jackson cannot create an instance of the class in the absence of a default constructor.

Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of sample.sample11$MyClass: no suitable constructor found, can not deserialize from Object value (missing default constructor or creator, or perhaps need to add/enable type information?)

What if you do not want to add a default constructor due to design considerations? Jackson offers a solution: use @JsonCreator with @JsonProperty to properly mark the constructor. The constructor takes two arguments, both of which are marked required. With this change, Jackson is able to create an instance of MyClass with the proper arguments.

public class MyClass {
    private String firstName;
    private String lastName;
    private Address address;

    @JsonCreator
    public MyClass(@JsonProperty(value = "firstName",required = true) String firstName,
		   @JsonProperty(value = "lastName", required = true) String lastName)
    {
	this.firstName = firstName;
	this.lastName = lastName;
    }
}

3. @JsonIgnore

Property-level annotation used to ignore specified properties. These properties are ignored during both serialization and deserialization. For instance, the following property password is neither written to JSON nor read from JSON.

...
    @JsonIgnore
    private String password;
...

4. @JsonIgnoreProperties

Class level annotation used to ignore multiple properties.

@JsonIgnoreProperties({"bar", "baz"})
public class MyClass
{
    private String foo;
    private String bar;
    private Integer baz;
    ...
}

Additionally, you can use this annotation to ignore unknown properties during deserialization. If unknown properties are present in the JSON, you will get an UnrecognizedPropertyException.

@JsonIgnoreProperties(ignoreUnknown=true, {"bar", "baz"})
public class MyClass
{
    private String foo;
    private String bar;
    ...
}

Parsing the following JSON will not result in an exception due to specifying ignoreUnknown.

{
  "foo" : "value of foo",
  "bar" : "value of bar",
  "qux" : "value of qux"
}

5. @JsonIgnoreType

To ignore a particular class (or built-in type) during serialization and deserialization, use the @JsonIgnoreType annotation on the class. In the following, the MyClass.address property will not be serialized or deserialized.

@JsonIgnoreType
public class Address {
    private String address1;
    ...
}

public class MyClass {
    private Address address;
    private String firstName;
    private String lastName;
}

Note that MyClass.address property has bot been serialized. It will be ignored during deserialization.

{
  "firstName" : "Harrison",
  "lastName" : "Ford"
}

6. @JsonFormat

@JsonFormat is used to specify some extra details when serializing certain types especially dates.

The date field in the following class is serialized by default as a timestamp.

public class MyClass
{
    private Date createdDate = new Date();
}

// serialized to:
{
  "createdDate" : 1485913763410
}

Annotate the date field with @JsonFormat and use pattern to set the date format:

public class MyClass
{
    @JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd@HH:mm:ss.SSSZ")
    private Date createdDate;
}

Serialization now shows the date in the specified format:

{
  "createdDate" : "2017-02-01@01:58:28.130+0000"
}

Deserialization also works as expected. If the date is not specified in the same format as required by the pattern, you get the following exception:

Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidFormatException: Can not deserialize value of type java.util.Date from String "...": expected format "yyyy-MM-dd@HH:mm:ss.SSSZ" ...

7. @JsonUnwrapped

You can use @JsonUnwrapped to indicate to Jackson that a contained object should be unwrapped during serialization i.e. the contained object properties should be included among the properties of the parent. An example:

public class Address {
    private String address1;
    private String address2;
    private String city;
    private String state;
    private String zip;
}

public class MyClass {
    private String firstName;
    private String lastName;
    @JsonUnwrapped
    private Address address;
}

When an object of MyClass is serialized, the properties of address are included among the properties of MyClass:

{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "address1" : "123 Main Street",
  "address2" : null,
  "city" : "Hollywood",
  "state" : "CA",
  "zip" : "33023"
}

To prevent clashes between the parent and child object properties, a prefix or suffix can be specified to be added to the child properties:

public class MyClass {
    private String firstName;
    private String lastName;
    @JsonUnwrapped(prefix="addr.")
    private Address address;
}

The resulting JSON is now:

{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "addr.address1" : "123 Main Street",
  "addr.address2" : null,
  "addr.city" : "Hollywood",
  "addr.state" : "CA",
  "addr.zip" : "33023"
}

During deserialization, either of the forms shown are accepted without errors.

// normal form
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "address" : {
    "address1" : "123 Main Street",
    "address2" : null,
    "city" : "Hollywood",
    "state" : "CA",
    "zip" : "33023"
  }
}

// no prefix
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "address1" : "123 Main Street",
  "address2" : null,
  "city" : "Hollywood",
  "state" : "CA",
  "zip" : "33023"
}

// with prefix
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "addr.address1" : "123 Main Street",
  "addr.address2" : null,
  "addr.city" : "Hollywood",
  "addr.state" : "CA",
  "addr.zip" : "33023"
}

8. @JsonGetter

This annotation can be used on a no-arguments method returning non-void, with the value specified to indicate the name of a logical property. An example, serializing an object of the class below will include a property “joe” defined as the value returned from the method theDate().

public class MyClass {
    private String firstName;
    private String lastName;

    @JsonGetter("joe")
    public Date theDate()
    {
	return new Date();
    }
}

// serialized to:
{
 "firstName" : "Harrison",
 "lastName" : "Ford",
 "joe" : 1485931403261
}

9. @JsonAnyGetter and @JsonAnySetter

In some instances, you may have a situation where a class has dynamic properties stored in a Map and these properties need to be exposed to JSON. In this situation, the getter method or the Map field can be marked with the @JsonAnyGetter annotation. An example will illustrate the point.

public class MyClass {
    private String firstName;
    private String lastName;
    private Address address;
    private Map<String,Object> moreProps = new HashMap<>();
}

// set properties
MyClass obj = new MyClass();
...
obj.setAddress(address);
obj.getMoreProps().put("born", "Chicago, Illinois");
System.out.println(mapper.writeValueAsString(obj));

// serialized to:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "moreProps" : {
    "born" : "Chicago, Illinois"
  }
}

As expected, the Map field appears as a separate object within the container object. Using the @JsonAnyGetter merges the Map entries into the parent JSON.

public class MyClass {
    private String firstName;
    private String lastName;
    private Address address;
    private Map<String,Object> moreProps = new HashMap<>();

    @JsonAnyGetter
    public Map<String,Object> getMoreProps() {
        return moreProps;
    }
}

// create and set object properties here as before
MyClass obj = new MyClass();
...

// serialized to:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "born" : "Chicago, Illinois"
}

That takes care of serialization. How about deserialization? JSON serialized for the above case cannot directly be used for deserialization as it results in an UnrecognizedPropertyException.

The solution is to use @JsonAnySetter on a method which accepts a key and a value to set the property. See below.

public class MyClass {
    ... // as before

    @JsonAnySetter
    public void setMore(String key,Object value)
    {
	moreProps.put(key, value);
    }
}

With this method definition and annotation in place, deserialization is also a breeze.

10. @JsonValue

@JsonValue can be used to serialize the complete state of an object using a single method rather than the usual serialization using properties. An example will help clarify the picture.

The following class marks the toString() method with @JsonValue. When serializing this object, Jackson uses the representation returned from this method as the value of the object. Note that the value returned is not valid JSON.

public class Address {
    private String address1;
    private String address2;
    private String city;
    private String state;
    private String zip;

    @JsonValue
    public String toString()
    {
	StringBuilder sbuf = new StringBuilder();
	sbuf.append(address1).append("|")
	    .append(address2).append("|")
	    .append(city).append("|")
	    .append(state).append("|")
	    .append(zip).append("|");
	return sbuf.toString();
    }
}

Serializing an instance of the above class yields:

"123 Main Street|null|Hollywood|CA|33023|"

Let us see how this works when included as a part of another class.

public class MyClass {
    private String firstName;
    private String lastName;
    private Address address;
}

After suitable initialization and serialization, we have the following JSON output. Note that the method annotated with @JsonValue was used for converting the value of Address to its representation.

{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "address" : "123 Main Street|null|Hollywood|CA|33023|"
}

10.1. @JsonValue and @JsonCreator

Now, how do we deserialize this JSON into the proper object? We make use of the @JsonCreator annotation to mark a static factory method (or a constructor) which can create the object from its string representation. This method does the opposite of what the @JsonValue method does: it splits the string representation and sets the appropriate fields.

public class Address {
    private String address1;
    private String address2;
    ...

    @JsonCreator
    static public Address fromString(String value)
    {
	String[] values = value.split("\\|");
	Address obj = new Address();
	int index = 0;
	if ( index < values.length ) {
	    obj.setAddress1(values[index]);
	    index++;
	}
	if ( index < values.length ) {
	    obj.setAddress2(values[index]);
	    index++;
	}
	if ( index < values.length ) {
	    obj.setCity(values[index]);
	    index++;
	}
	if ( index < values.length ) {
	    obj.setState(values[index]);
	    index++;
	}
	if ( index < values.length ) {
	    obj.setZip(values[index]);
	    index++;
	}
	return obj;
    }
}

11. @JsonPropertyOrder

Sometimes you may need to specify the order in which the properties are serialized. By default properties are serialized in the order they are defined in the class. If this order is not satisfactory, you can use @JsonPropertyOrder to specify the order of appearance.

The following definition illustrates the point. Note the order in which the properties are serialized in JSON.

public class MyClass {
    private Address address;
    private String firstName;
    private String lastName;
}

// serialized by default as:
{
  "address" : {
    "address1" : "123 Main Street",
    "address2" : null,
    "city" : "Hollywood",
    "state" : "CA",
    "zip" : "33023"
  },
  "firstName" : "Harrison",
  "lastName" : "Ford"
}

Now applying the @JsonPropertyOrder annotation as shown rearranges the order to the required.

@JsonPropertyOrder(value = {"firstName", "lastName", "address"})
static public class MyClass {
    private Address address;
    ...
}

// serialized to:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "address" : {
    "address1" : "123 Main Street",
    "address2" : null,
    "city" : "Hollywood",
    "state" : "CA",
    "zip" : "33023"
  }
}

12. Mixin Annotations

Sometimes the class being used in JSON conversion may not be available for direct modification to add or remove annotations. Maybe the class is a part of a library and hence cannot be edited. How can we declare the annotations that we want on such classes? We use Mixins! Mixin classes can be declared with the desired annotations and added to the ObjectMapper using the addMixIn() method.

Example class:

public class MyClass
{
    private String foo;
    private String bar;
    private Integer baz;
    ...
}

Declare and add a mixin class like this. (The class can be a local class.)

@JsonIgnoreProperties(ignoreUnknown=true)
class Mixin {
    @JsonIgnore
    private Integer baz;
};
mapper.addMixIn(MyClass.class, Mixin.class);

Sample output from serialization. Note that property baz is ignored.

{
  "foo" : "value of foo",
  "bar" : "value of bar"
}

Deserializing this JSON does not cause an exception since ignoreUnknown is set in the mixin.

{
  "foo" : "value of foo",
  "bar" : "value of bar",
  "qux" : "value of qux"
}

Summary

In this article, we looked at some of the most common Jackson annotations for JSON serialization and deserialization.

Using Jackson for JSON Serialization and Deserialization

1. Introduction

Jackson is one of the most common Java libraries for processing JSON. It is used for reading and writing JSON among other tasks. Using Jackson, you can easily handle automatic conversion from Java objects to JSON and back. In this article, we delve into some common Jackson usage patterns.

2. Converting a POJO to JSON

Suppose we want to convert a sample POJO (Plain Old Java Object) to JSON. An example class is defined below.

public class User {
    private String firstName;
    private String lastName;
    private Date dateOfBirth;
    private List<String> emailAddrs;

    public User(String firstName,String lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    // standard getters and setters here
}

Converting such an object to JSON is just a couple of lines:

ObjectMapper mapper = new ObjectMapper();
User user = new User("Harrison", "Ford");
user.setEmailAddrs(Arrays.asList("harrison@example.com"));
mapper.writeValue(System.out, user);

// prints:
{"firstName":"Harrison","lastName":"Ford","dateOfBirth":null,"emailAddrs":["harrison@example.com"]}

3. Pretty Printing

Note that the default output is quite compact. Sometimes it is useful to be able to view indented output for debugging purposes. To produce properly indented output, enable the option SerializationFeature.INDENT_OUTPUT on the ObjectMapper instance before using it to serialize the object.

ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
User user = new User("Harrison", "Ford");
...

The output generated now is nicely indented:

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : null,
  "emailAddrs" : [ "harrison@example.com" ]
}

4. Ignore NULL fields

How do we tell Jackson not to serialize null values (as for “dateOfBirth” above)? There are a couple of ways. You can tell the ObjectMapper to skip all NULL fields, or you can use annotations to specify per class or per field.

4.1. Configuring ObjectMapper

...
mapper.setSerializationInclusion(Include.NON_NULL);
User user = new User("Harrison", "Ford");
...

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "emailAddrs" : [ "harrison@example.com" ]
}

4.2. With an Annotation

Use the annotation @JsonInclude(Include.NON_NULL) on the target object class to eliminate serialization of all NULL fields.

@JsonInclude(Include.NON_NULL)
public class User {
    private String firstName;
    private String lastName;
    ...
}

Or on a field (or property) to disable specific NULL fields from being serialized. In the code below, dateOfBirth will be ignored if null, but not heightInM.

...
    @JsonInclude(Include.NON_NULL)
    private Date dateOfBirth;

    private Float heightInM;
...

5. Ignore Empty Arrays

When you have a class with an empty array initializer, the value is serialized as an empty JSON array.

class User {
    ...
    private List<String> phoneNumbers = new ArrayList<>();
    ...
}

// serialized to:
{
  ...
  "phoneNumbers" : [ ],
  ...
}

When you want to skip empty array declarations being output, you can use the following.

mapper.setSerializationInclusion(Include.NON_EMPTY);

6. Generate JSON String

How can we generate a JSON string representation instead of writing it directly to a file or an OutputStream? Maybe you want to store the JSON string in a database. Use the writeValueAsString() method.

// set up user
String jsonStr = mapper.writeValueAsString(user);

7. Formatting Dates

By default, Jackson prints Date fields as numeric timestamps as shown below:

Calendar c = Calendar.getInstance();
c.clear(); c.set(1942, 6, 13);
user.setDateOfBirth(c.getTime());

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : -866957400000,
  "emailAddrs" : [ "harrison@example.com" ]
}

Turning off numeric timestamp results in serialization of a date in the ISO 8601 format as shown below:

...
mapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
...

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : "1942-07-12T18:30:00.000+0000",
  "emailAddrs" : [ "harrison@example.com" ]
}

Change the default date format by using SimpleDateFormat as shown:

...
DateFormat fmt = new SimpleDateFormat("dd-MMM-yyyy");
mapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
...

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : "13-Jul-1942",
  "emailAddrs" : [ "harrison@example.com" ]
}

8. Converting JSON to POJO

Reading and converting JSON to a Java object is just as easy with Jackson. Accomplished in just a couple of lines:

ObjectMapper mapper = new ObjectMapper();
User user = mapper.readValue(new File(jsonFile), User.class);

The target class (User in this case) needs a no-arguments default constructor defined – failing which you get this exception:

Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of sample.User: no suitable constructor found, can not deserialize from Object value (missing default constructor or creator, or perhaps need to add/enable type information?)

The situation is easily address by adding a no-arguments default constructor.

public class User {
    private String firstName;
    private String lastName;
    private Date dateOfBirth;
    private List<String> emailAddrs;

    public User() {}
    ...
}

9. Adjusting Date Format

As before, the date format needs to be adjusted unless it is in ISO 8601 format. Parsing the following JSON fails:

{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : "13-Jul-1942",
  "emailAddrs" : [ "harrison@example.com", null ],
}

The following exception is reported:

Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidFormatException: Can not deserialize value of type java.util.Date from String "13-Jul-1942": not a valid representation (error: Failed to parse Date value '13-Jul-1942': Can not parse date "13-Jul-1942": not compatible with any of standard forms ("yyyy-MM-dd'T'HH:mm:ss.SSSZ", "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", "EEE, dd MMM yyyy HH:mm:ss zzz", "yyyy-MM-dd"))

Specify the date format if you are using a non-standard format as shown:

DateFormat fmt = new SimpleDateFormat("dd-MMM-yyyy");
mapper.setDateFormat(fmt);

10. Ignoring Unknown Properties

Sometimes the JSON you are trying to read might include some properties not defined in the Java class. Maybe the JSON was updated but the change is not yet reflected in the Java class. In such cases, you might end up with an exception like this:

Exception in thread "main" com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "gender" (class sample.User), not marked as ignorable (6 known properties: ...)

You can tell Jackson to ignore such properties on a global level by disabling a deserialization feature as follows:

mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);

Alternatively, you can use the following annotation on the class to ignore unknown properties.

@JsonIgnoreProperties(ignoreUnknown=true)
public class User {
    ...
}

11. Serializing Array of Objects to JSON

Serializing an array of objects to JSON is straightforward.

ObjectMapper mapper = new ObjectMapper();
List<User> users = new ArrayList<>();
users.add(new User(...));
users.add(new User(...));
System.out.println(mapper.writeValueAsString(users));

The array is properly serialized as shown:

[ {
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : "13-Jul-1942",
  ...},
  {
  "firstName" : "Samuel",
  "lastName" : "Jackson",
  "dateOfBirth" : "21-Dec-1948",
  ...} ]

12. Deserialize Array of Objects from JSON

Several methods are available to deserialize a JSON array to a collection of Java objects. Use whatever method suits you depending on your requirements.

Deserializing a JSON array to a Java array of objects of a particular class is as simple as:

User[] users = mapper.readValue(new File(jsonFile), User[].class);
System.out.println(mapper.writeValueAsString(users));

Using a JavaType is useful when constructing collections or parametric types.

JavaType listType = mapper
    .getTypeFactory()
    .constructCollectionType(List.class, User.class);
List<User> users = mapper.readValue(new File(jsonFile), listType);
System.out.println(mapper.writeValueAsString(users));

A third method is to create and use a TypeReference.

TypeReference ref = new TypeReference<List<User>>(){};
List<User> users = mapper.readValue(new File(jsonFile), ref);
System.out.println(mapper.writeValueAsString(users));

Summary

This article showed you how to convert Java objects to JSON and back. We also covered a few common use cases.

Java Concurrency and Executors

1. Introduction

Java provides a package java.util.concurrent which includes many facilities to ease concurrent programming. In this article, we take a look at java threads and Executors – what they are and how they work, etc.

2. Runnable Task as a Lambda

Since the early days of Java 1.0, Java provides a Runnable interface which must be implemented to run a task in a separate thread. An sample implementation of a Runnable task is shown below. Note that rather than creating a class implementing the Runnable interface, the code creates a Runnable lambda. The task is then executed in the main thread as well as a second thread.

static private void example_1(String[] args) throws Exception
{
    Runnable task = () -> {
	String threadName = Thread.currentThread().getName();
	System.out.println("Hello " + threadName);
    };

    task.run();

    Thread thread = new Thread(task);
    thread.start();
    System.out.println("Done.");
}

The output is shown below. Note that the main thread completes before the second thread but waits for it to complete before the program terminates.

Hello main
Done.
Hello Thread-0

3. Creating an Executor

With the new Executor Framework in Java 8, java provides an Executor interface with a single method execute(Runnable) for executing a Runnable task. As opposed to creating a Thread with a Runnable (as shown above), you can create an Executor and run one or more tasks as follows:

Executor executor = ...;
executor.execute(task1);
executor.execute(task2);

The advantage of this approach is that the Executor implementation takes care of thread creation and management. To this end, java provides several implementations of this interface supporting various techniques such as executing tasks in a thread pool, executing tasks sequentially in a worker thread, etc.

In addition to the Executor interface, Java also provides an ExecutorService which is a sub-interface of Executor. This interface provides additional facilities over Executor including convenience methods to execute multiple tasks, submit tasks for execution, etc.

Executors is a convenience class providing factory methods to create various kinds of ExecutorService implementation objects. The method newSingleThreadExecutor() creates a thread and executes pending tasks sequentially. A task is submitted for execution using the submit() method.

static private void example_2(String[] args) throws Exception
{
    ExecutorService esvc = Executors.newSingleThreadExecutor();
    Runnable task = () -> {
	try {
	    String threadName = Thread.currentThread().getName();
	    System.out.println("Thread " + threadName + " started");
	    TimeUnit.SECONDS.sleep(2);
	    System.out.println("Thread " + threadName + " ended");
	} catch(InterruptedException ex) {
	    System.err.println("Task interrupted.");
	}
    };

    esvc.submit(task);
}

On running this example, the ExecutorService executes the submitted task and continues to wait for more tasks without exiting. This is the default for all ExecutorService objects. Hit Control-C to halt the program.

4. Serial Execution of Tasks

In addition to executing a Runnable, an ExecutorService can also run a Callable<?>. The following class implements Callable<String> which returns a Future<String> after the execution is completed. It can be scheduled for execution using an ExecutorService.

static private class DelayTask implements Callable<String>
{
    private String name;
    private int secsDelay;

    public DelayTask(String name,int secsDelay) {
	this.name = name;
	this.secsDelay = secsDelay;
    }

    @Override
	public String call() {
	System.out.println(name + " started");
	try { TimeUnit.SECONDS.sleep(secsDelay); }
	catch(InterruptedException ex) {
	    System.err.println(name + ": interrupted");
	}
	System.out.println(name + " ended");
	return name;
    }
}

Let us create a bunch of these tasks and execute them using an ExecutorService created from newSingleThreadExecutor(). This ExecutorService executes the tasks sequentially. At the end, we add another task to shutdown the ExecutorService. Due to Future<?>.get(), the code waits for all the tasks to terminate and then cleanly shuts down the ExecutorService which cleans up all the resources.

static private void example_3(String[] args) throws Exception
{
    ExecutorService esvc = Executors.newSingleThreadExecutor();
    List<Callable<String>> tasks =
	Arrays.asList(new DelayTask("task 1", 2),
		      new DelayTask("task 2", 3),
		      new DelayTask("task 3", 1),
		      () -> {
			  esvc.shutdown();
			  return "shutdown";
		      });
    esvc.invokeAll(tasks)
	.stream()
	.map(future -> {
		try { return future.get(); }
		catch(Exception ex) {
		    return "exception: " + ex.getMessage();
		}
	    })
	.forEach(System.out::println);
}

Here is the output from the code above. Notice that tasks are executed one after another followed by the shutdown task. This is because we used the newSingleThreadExecutor() method which creates an ExecutorService with that characteristic.

task 1 started
task 1 ended
task 2 started
task 2 ended
task 3 started
task 3 ended
task 1
task 2
task 3
shutdown

5. Executing Tasks with a Thread Pool

Let us now examine how a thread pool returned from newCachedThreadPool() behaves. The source code is shown below; it uses the DelayTask class defined above.

static private void example_4(String[] args) throws Exception
{
    ExecutorService esvc = Executors.newCachedThreadPool();
    List<Callable<String>> tasks =
	Arrays.asList(new DelayTask("task 1", 2),
		      new DelayTask("task 2", 3),
		      new DelayTask("task 3", 10),
		      () -> {
			  System.err.println("Requesting shutdown ..");
			  esvc.shutdown();
			  return "shutdown";
		      });
    esvc.invokeAll(tasks)
	.stream()
	.map(future -> {
		try { return future.get(); }
		catch(Exception ex) {
		    return "exception: " + ex.getMessage();
		}
	    })
	.forEach(System.out::println);
}

Here is the output from this code. Notice that a shutdown is requested of the ExecutorService after the tasks have been scheduled. The ExecutorService allows the tasks to complete execution before shutting down. It does not, however, allow any more tasks to be scheduled after shutdown() has been invoked.

Task task 1 started
Task task 2 started
Task task 3 started
Requesting shutdown ..
Task task 1 ended
Task task 2 ended
Task task 3 ended
task 1
task 2
task 3
shutdown

6. Shutdown Thread Pool Immediately

When it is necessary to shutdown the thread pool immediately as opposed to waiting for all tasks to complete, we can use the shutdownNow() method.

static private void example_5(String[] args) throws Exception
{
    ExecutorService esvc = Executors.newCachedThreadPool();
    List<Callable<String>> tasks =
	Arrays.asList(new DelayTask("task 1", 2),
		      new DelayTask("task 2", 3),
		      new DelayTask("task 3", 10),
		      () -> {
			  System.err.println("Requesting shutdown ..");
			  esvc.shutdownNow();
			  return "shutdown";
		      });
    esvc.invokeAll(tasks)
	.stream()
	.map(future -> {
		try { return future.get(); }
		catch(Exception ex) {
		    return "exception: " + ex.getMessage();
		}
	    })
	.forEach(System.out::println);
}

As the output below shows, the tasks that have not yet been completed are interrupted and the thread pool is terminated.

Task task 1 started
Task task 2 started
Task task 3 started
Requesting shutdown ..
task 3: interrupted
Task task 3 ended
task 1: interrupted
Task task 1 ended
task 2: interrupted
Task task 2 ended
task 1
task 2
task 3
shutdown

Summary

This article provided an introduction to threading in Java as well as the new Executor framework in Java 8. The Executor framework simplifies a lot of plumbing code related to thread creation and management. It also brings new capabilities to threading in Java including thread pools, worker threads which can schedule tasks sequentially. As the code above demonstrated, it is also possible to implement code to shutdown the thread pool cleanly after all the tasks have completed.