Sorting Lists in Java

Learn how to sort a List in java. Use Java 8 Lambdas to compose sort conditions.

“You can only be afraid of what you think you know.”
― Jiddu Krishnamurti

1. Introduction

Sorting a list is a very common requirement when programming in java. Out of the box, java provides for sorting using a performant algorithm so there is no need to roll your own algo for the purpose. In this article, we demonstrate how to sort a list in java.

Continue reading “Sorting Lists in Java”

Regular Expressions Examples

Learn how to build regular expression patterns for parsing phone numbers, emails, etc.

. Introduction

Regular Expressions provide a powerful method to handle parsing tasks while handling texts. Whether the problem is a simple search for a pattern, or splitting a string into components, regular expressions are widely used today.

Continue reading “Regular Expressions Examples”

How to Convert Large CSV to JSON

Learn how to use Jackson Streaming to convert a large CSV to JSON.

“We are a way for the cosmos to know itself.”
― Carl Sagan, Cosmos

1. Introduction

Let us today look into converting a large CSV to JSON without running into memory issues. This previous article showed how to parse CSV and output the data to JSON using Jackson. However, since that code loads the entire data into memory, it will run into issues loading large CSV files such as:

Continue reading “How to Convert Large CSV to JSON”

Using Timer Class to Schedule Tasks

Learn how to use the Timer and the TimerTask classes to implement simple task scheduling for your application.

“Humor is reason gone mad.”
― Groucho Marx

1. Introduction

Scheduling tasks to run is a need which sometimes arises in a java program. Maybe you want to run periodic cleanup of some resource. Or check on the status of some job. Or maybe fetch a URL which might not be available the first time.

Continue reading “Using Timer Class to Schedule Tasks”

Using Jackson for JSON Serialization and Deserialization

Learn how to use Jackson to serialize java objects.

1. Introduction

Jackson is one of the most common Java libraries for processing JSON. It is used for reading and writing JSON among other tasks. Using Jackson, you can easily handle automatic conversion from Java objects to JSON and back. In this article, we delve into some common Jackson usage patterns.

2. Converting a POJO to JSON

Suppose we want to convert a sample POJO (Plain Old Java Object) to JSON. An example class is defined below.

public class User {
    private String firstName;
    private String lastName;
    private Date dateOfBirth;
    private List<String> emailAddrs;

    public User(String firstName,String lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    // standard getters and setters here
}

Converting such an object to JSON is just a couple of lines:

ObjectMapper mapper = new ObjectMapper();
User user = new User("Harrison", "Ford");
user.setEmailAddrs(Arrays.asList("harrison@example.com"));
mapper.writeValue(System.out, user);

// prints:
{"firstName":"Harrison","lastName":"Ford","dateOfBirth":null,"emailAddrs":["harrison@example.com"]}

3. Pretty Printing

Note that the default output is quite compact. Sometimes it is useful to be able to view indented output for debugging purposes. To produce properly indented output, enable the option SerializationFeature.INDENT_OUTPUT on the ObjectMapper instance before using it to serialize the object.

ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
User user = new User("Harrison", "Ford");
...

The output generated now is nicely indented:

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : null,
  "emailAddrs" : [ "harrison@example.com" ]
}

4. Ignore NULL fields

How do we tell Jackson not to serialize null values (as for “dateOfBirth” above)? There are a couple of ways. You can tell the ObjectMapper to skip all NULL fields, or you can use annotations to specify per class or per field.

4.1. Configuring ObjectMapper

...
mapper.setSerializationInclusion(Include.NON_NULL);
User user = new User("Harrison", "Ford");
...

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "emailAddrs" : [ "harrison@example.com" ]
}

4.2. With an Annotation

Use the annotation @JsonInclude(Include.NON_NULL) on the target object class to eliminate serialization of all NULL fields.

@JsonInclude(Include.NON_NULL)
public class User {
    private String firstName;
    private String lastName;
    ...
}

Or on a field (or property) to disable specific NULL fields from being serialized. In the code below, dateOfBirth will be ignored if null, but not heightInM.

...
    @JsonInclude(Include.NON_NULL)
    private Date dateOfBirth;

    private Float heightInM;
...

5. Ignore Empty Arrays

When you have a class with an empty array initializer, the value is serialized as an empty JSON array.

class User {
    ...
    private List<String> phoneNumbers = new ArrayList<>();
    ...
}

// serialized to:
{
  ...
  "phoneNumbers" : [ ],
  ...
}

When you want to skip empty array declarations being output, you can use the following.

mapper.setSerializationInclusion(Include.NON_EMPTY);

6. Generate JSON String

How can we generate a JSON string representation instead of writing it directly to a file or an OutputStream? Maybe you want to store the JSON string in a database. Use the writeValueAsString() method.

// set up user
String jsonStr = mapper.writeValueAsString(user);

7. Formatting Dates

By default, Jackson prints Date fields as numeric timestamps as shown below:

Calendar c = Calendar.getInstance();
c.clear(); c.set(1942, 6, 13);
user.setDateOfBirth(c.getTime());

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : -866957400000,
  "emailAddrs" : [ "harrison@example.com" ]
}

Turning off numeric timestamp results in serialization of a date in the ISO 8601 format as shown below:

...
mapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
...

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : "1942-07-12T18:30:00.000+0000",
  "emailAddrs" : [ "harrison@example.com" ]
}

Change the default date format by using SimpleDateFormat as shown:

...
DateFormat fmt = new SimpleDateFormat("dd-MMM-yyyy");
mapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
...

// prints:
{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : "13-Jul-1942",
  "emailAddrs" : [ "harrison@example.com" ]
}

8. Converting JSON to POJO

Reading and converting JSON to a Java object is just as easy with Jackson. Accomplished in just a couple of lines:

ObjectMapper mapper = new ObjectMapper();
User user = mapper.readValue(new File(jsonFile), User.class);

The target class (User in this case) needs a no-arguments default constructor defined – failing which you get this exception:

Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of sample.User: no suitable constructor found, can not deserialize from Object value (missing default constructor or creator, or perhaps need to add/enable type information?)

The situation is easily address by adding a no-arguments default constructor.

public class User {
    private String firstName;
    private String lastName;
    private Date dateOfBirth;
    private List<String> emailAddrs;

    public User() {}
    ...
}

9. Adjusting Date Format

As before, the date format needs to be adjusted unless it is in ISO 8601 format. Parsing the following JSON fails:

{
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : "13-Jul-1942",
  "emailAddrs" : [ "harrison@example.com", null ],
}

The following exception is reported:

Exception in thread "main" com.fasterxml.jackson.databind.exc.InvalidFormatException: Can not deserialize value of type java.util.Date from String "13-Jul-1942": not a valid representation (error: Failed to parse Date value '13-Jul-1942': Can not parse date "13-Jul-1942": not compatible with any of standard forms ("yyyy-MM-dd'T'HH:mm:ss.SSSZ", "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", "EEE, dd MMM yyyy HH:mm:ss zzz", "yyyy-MM-dd"))

Specify the date format if you are using a non-standard format as shown:

DateFormat fmt = new SimpleDateFormat("dd-MMM-yyyy");
mapper.setDateFormat(fmt);

10. Ignoring Unknown Properties

Sometimes the JSON you are trying to read might include some properties not defined in the Java class. Maybe the JSON was updated but the change is not yet reflected in the Java class. In such cases, you might end up with an exception like this:

Exception in thread "main" com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "gender" (class sample.User), not marked as ignorable (6 known properties: ...)

You can tell Jackson to ignore such properties on a global level by disabling a deserialization feature as follows:

mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);

Alternatively, you can use the following annotation on the class to ignore unknown properties.

@JsonIgnoreProperties(ignoreUnknown=true)
public class User {
    ...
}

11. Serializing Array of Objects to JSON

Serializing an array of objects to JSON is straightforward.

ObjectMapper mapper = new ObjectMapper();
List<User> users = new ArrayList<>();
users.add(new User(...));
users.add(new User(...));
System.out.println(mapper.writeValueAsString(users));

The array is properly serialized as shown:

[ {
  "firstName" : "Harrison",
  "lastName" : "Ford",
  "dateOfBirth" : "13-Jul-1942",
  ...},
  {
  "firstName" : "Samuel",
  "lastName" : "Jackson",
  "dateOfBirth" : "21-Dec-1948",
  ...} ]

12. Deserialize Array of Objects from JSON

Several methods are available to deserialize a JSON array to a collection of Java objects. Use whatever method suits you depending on your requirements.

Deserializing a JSON array to a Java array of objects of a particular class is as simple as:

User[] users = mapper.readValue(new File(jsonFile), User[].class);
System.out.println(mapper.writeValueAsString(users));

Using a JavaType is useful when constructing collections or parametric types.

JavaType listType = mapper
    .getTypeFactory()
    .constructCollectionType(List.class, User.class);
List<User> users = mapper.readValue(new File(jsonFile), listType);
System.out.println(mapper.writeValueAsString(users));

A third method is to create and use a TypeReference.

TypeReference ref = new TypeReference<List<User>>(){};
List<User> users = mapper.readValue(new File(jsonFile), ref);
System.out.println(mapper.writeValueAsString(users));

Summary

This article showed you how to convert Java objects to JSON and back. We also covered a few common use cases.

How to Generate RSA Keys in Java

Learn how to generate RSA keys and digitally sign files in java.

1. Introduction

Let us learn the basics of generating and using RSA keys in Java.

Java provides classes for the generation of RSA public and private key pairs with the package java.security. You can use RSA keys pairs in public key cryptography.

Public key cryptography uses a pair of keys for encryption. Distribute the public key to whoever needs it but safely secure the private key.

Public key cryptography can be used in two modes:

Encryption: Only the private key can decrypt the data encrypted with the public key.

Authentication: Data encrypted with the private key can only be decrypted with the public key thus proving who the data came from.

2. Generating a Key Pair

First step in creating an RSA Key Pair is to create a KeyPairGenerator from a factory method by specifying the algorithm (“RSA” in this instance):

KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");

Initialize the KeyPairGenerator with the key size. Use a key size of 1024 or 2048. Currently recommended key size for SSL certificates used in e-commerce is 2048 so that is what we use here.

kpg.initialize(2048);
KeyPair kp = kpg.generateKeyPair();

From the KeyPair object, get the public key using getPublic() and the private key using getPrivate().

Key pub = kp.getPublic();
Key pvt = kp.getPrivate();

3. Saving the Keys in Binary Format

Save the keys to hard disk once they are obtained. This allows re-using the keys for encryption, decryption and authentication.

String outFile = ...;
out = new FileOutputStream(outFile + ".key");
out.write(pvt.getEncoded());
out.close();

out = new FileOutputStream(outFile + ".pub");
out.write(pvt.getEncoded());
out.close();

What is the format of the saved files? The key information is encoded in different formats for different types of keys. Here is how you can find what format the key was saved in. On my machine, the private key was saved in PKCS#8 format and the public key in X.509 format. We need this information below to load the keys.

System.err.println("Private key format: " + pvt.getFormat());
// prints "Private key format: PKCS#8" on my machine

System.err.println("Public key format: " + pub.getFormat());
// prints "Public key format: X.509" on my machine

3.1. Load Private Key from File

After saving the private key to a file (or a database), you might need to load it at a later time. You can do that using the following code. Note that you need to know what format the data was saved in: PKCS#8 in our case.

/* Read all bytes from the private key file */
Path path = Paths.get(keyFile);
byte[] bytes = Files.readAllBytes(path);

/* Generate private key. */
PKCS8EncodedKeySpec ks = new PKCS8EncodedKeySpec(bytes);
KeyFactory kf = KeyFactory.getInstance("RSA");
PrivateKey pvt = kf.generatePrivate(ks);

3.2 Load Public Key from File

Load the public key from a file as follows. The public key has been saved in X.509 format so we use the X509EncodedKeySpec class to convert it.

/* Read all the public key bytes */
Path path = Paths.get(keyFile);
byte[] bytes = Files.readAllBytes(path);

/* Generate public key. */
X509EncodedKeySpec ks = new X509EncodedKeySpec(bytes);
KeyFactory kf = KeyFactory.getInstance("RSA");
PublicKey pub = kf.generatePublic(ks);

4. Use Base64 for Saving Keys as Text

Save the keys in text format by encoding the data in Base64. Java 8 provides a Base64 class which can be used for the purpose. Save the private key with a comment as follows:

Base64.Encoder encoder = Base64.getEncoder();

String outFile = ...;
Writer out = new FileWriter(outFile + ".key");
out.write("-----BEGIN RSA PRIVATE KEY-----\n");
out.write(encoder.encodeToString(pvt.getEncoded()));
out.write("\n-----END RSA PRIVATE KEY-----\n");
out.close();

And the public key too (with a comment):

out = new FileWriter(outFile + ".pub");
out.write("-----BEGIN RSA PUBLIC KEY-----\n");
out.write(encoder.encodeToString(kp.getPublic()));
out.write("\n-----END RSA PUBLIC KEY-----\n");
out.close();

5. Generating a Digital Signature

As mentioned above, one of the purposes of public key cryptography is digital signature i.e. you generate a digital signature from a file contents, sign it with your private key and send the signature along with the file. The recipient can then use your public key to verify that the signature matches the file contents.

Here is how you can do it. Use the signature algorithm “SHA256withRSA” which is guaranteed to be supported on all JVMs. Use the private key (either generated or load from file as shown above) to initialize the Signature object for signing. It is then updated with contents from the data file and the signature is generated and written to the output file. This output file contains the digital signature and must be sent to the recipient for verification.

Signature sign = Signature.getInstance("SHA256withRSA");
sign.initSign(pvt);

InputStream in = null;
try {
    in = new FileInputStream(dataFile);
    byte[] buf = new byte[2048];
    int len;
    while ((len = in.read(buf)) != -1) {
    sign.update(buf, 0, len);
    }
} finally {
    if ( in != null ) in.close();
}

OutputStream out = null;
try {
    out = new FileOutputStream(signFile);
    byte[] signature = sign.sign();
    out.write(signature);
} finally {
    if ( out != null ) out.close();
}

6. Verifying the Digital Signature

The recipient uses the digital signature sent with a data file to verify that the data file has not been tampered with. It requires access to the sender’s public key and can be loaded from a file if necessary as presented above.

The code below updates the Signature object with data from the data file. It then loads the signature from file and uses Signature.verify() to check if the signature is valid.

Signature sign = Signature.getInstance("SHA256withRSA");
sign.initVerify(pub);

InputStream in = null;
try {
    in = new FileInputStream(dataFile);
    byte[] buf = new byte[2048];
    int len;
    while ((len = in.read(buf)) != -1) {
    sign.update(buf, 0, len);
    }
} finally {
    if ( in != null ) in.close();
}

/* Read the signature bytes from file */
path = Paths.get(signFile);
bytes = Files.readAllBytes(path);
System.out.println(dataFile + ": Signature " +
   (sign.verify(bytes) ? "OK" : "Not OK"));

And that in a nutshell is how you can use RSA public and private keys for digital signature and verification.

Source Code

Go here for the source code.