What is the Python Yield Statement?

Learn about the Python yield statement which is used to create generators.

“Count your age by friends, not years. Count your life by smiles, not tears.”
― John Lennon

1. Introduction

Python provides an yield statement which allows you to create generator functions. What is a generator function and how does the yield statement help with it? Let us find out in this article.

Continue reading “What is the Python Yield Statement?”

Java ThreadLocal – Proper Usage

Learn how to properly create and use ThreadLocal to share data between threads.

“A lie gets halfway around the world before the truth has a chance to get its pants on.”
― Winston S. Churchill

1. Introduction

Getting multi-threaded programming right is something of an art; it can be hard to foresee and avoid the many pitfalls. One such hazard comes from attempting to share a variable across multiple threads.

Continue reading “Java ThreadLocal – Proper Usage”

Using Timer Class to Schedule Tasks

Learn how to use the Timer and the TimerTask classes to implement simple task scheduling for your application.

“Humor is reason gone mad.”
― Groucho Marx

1. Introduction

Scheduling tasks to run is a need which sometimes arises in a java program. Maybe you want to run periodic cleanup of some resource. Or check on the status of some job. Or maybe fetch a URL which might not be available the first time.

Continue reading “Using Timer Class to Schedule Tasks”

Java Process Example Part 2

When reading and writing to a child process, it is necessary to execute the read and write blocks in separate threads to avoid deadlock.

1. Introduction

In the previous part of this Java Process Guide, we looked at how to read from and write to a child process. While executing the read and write blocks in a single thread works for simple cases, we should run these in separate threads to avoid deadlocks.

Continue reading “Java Process Example Part 2”

Java Process Example

Startup a native process from Java using Runtime. Read the output from the process and write some data to it. After you are done, terminate the process.

1. Introduction

Have you ever run into a situation where you need to execute an OS command from inside java and read its output? You can use the Process class to do so, but there are some caveats to consider.

Continue reading “Java Process Example”

Java Math.random Examples

Math.Random

1. Introduction

Let us learn how to generate some random numbers in Java. Random numbers are needed for various purposes; maybe you want to generate a password or a session identifier. Whatever the purpose may be, there are a number of issues to be aware of when generating a random number.

Continue reading “Java Math.random Examples”

Java Concurrency and Executors

1. Introduction

Java provides a package java.util.concurrent which includes many facilities to ease concurrent programming. In this article, we take a look at java threads and Executors – what they are and how they work, etc.

2. Runnable Task as a Lambda

Since the early days of Java 1.0, Java provides a Runnable interface which must be implemented to run a task in a separate thread. An sample implementation of a Runnable task is shown below. Note that rather than creating a class implementing the Runnable interface, the code creates a Runnable lambda. The task is then executed in the main thread as well as a second thread.

static private void example_1(String[] args) throws Exception
{
    Runnable task = () -> {
	String threadName = Thread.currentThread().getName();
	System.out.println("Hello " + threadName);
    };

    task.run();

    Thread thread = new Thread(task);
    thread.start();
    System.out.println("Done.");
}

The output is shown below. Note that the main thread completes before the second thread but waits for it to complete before the program terminates.

Hello main
Done.
Hello Thread-0

3. Creating an Executor

With the new Executor Framework in Java 8, java provides an Executor interface with a single method execute(Runnable) for executing a Runnable task. As opposed to creating a Thread with a Runnable (as shown above), you can create an Executor and run one or more tasks as follows:

Executor executor = ...;
executor.execute(task1);
executor.execute(task2);

The advantage of this approach is that the Executor implementation takes care of thread creation and management. To this end, java provides several implementations of this interface supporting various techniques such as executing tasks in a thread pool, executing tasks sequentially in a worker thread, etc.

In addition to the Executor interface, Java also provides an ExecutorService which is a sub-interface of Executor. This interface provides additional facilities over Executor including convenience methods to execute multiple tasks, submit tasks for execution, etc.

Executors is a convenience class providing factory methods to create various kinds of ExecutorService implementation objects. The method newSingleThreadExecutor() creates a thread and executes pending tasks sequentially. A task is submitted for execution using the submit() method.

static private void example_2(String[] args) throws Exception
{
    ExecutorService esvc = Executors.newSingleThreadExecutor();
    Runnable task = () -> {
	try {
	    String threadName = Thread.currentThread().getName();
	    System.out.println("Thread " + threadName + " started");
	    TimeUnit.SECONDS.sleep(2);
	    System.out.println("Thread " + threadName + " ended");
	} catch(InterruptedException ex) {
	    System.err.println("Task interrupted.");
	}
    };

    esvc.submit(task);
}

On running this example, the ExecutorService executes the submitted task and continues to wait for more tasks without exiting. This is the default for all ExecutorService objects. Hit Control-C to halt the program.

4. Serial Execution of Tasks

In addition to executing a Runnable, an ExecutorService can also run a Callable<?>. The following class implements Callable<String> which returns a Future<String> after the execution is completed. It can be scheduled for execution using an ExecutorService.

static private class DelayTask implements Callable<String>
{
    private String name;
    private int secsDelay;

    public DelayTask(String name,int secsDelay) {
	this.name = name;
	this.secsDelay = secsDelay;
    }

    @Override
	public String call() {
	System.out.println(name + " started");
	try { TimeUnit.SECONDS.sleep(secsDelay); }
	catch(InterruptedException ex) {
	    System.err.println(name + ": interrupted");
	}
	System.out.println(name + " ended");
	return name;
    }
}

Let us create a bunch of these tasks and execute them using an ExecutorService created from newSingleThreadExecutor(). This ExecutorService executes the tasks sequentially. At the end, we add another task to shutdown the ExecutorService. Due to Future<?>.get(), the code waits for all the tasks to terminate and then cleanly shuts down the ExecutorService which cleans up all the resources.

static private void example_3(String[] args) throws Exception
{
    ExecutorService esvc = Executors.newSingleThreadExecutor();
    List<Callable<String>> tasks =
	Arrays.asList(new DelayTask("task 1", 2),
		      new DelayTask("task 2", 3),
		      new DelayTask("task 3", 1),
		      () -> {
			  esvc.shutdown();
			  return "shutdown";
		      });
    esvc.invokeAll(tasks)
	.stream()
	.map(future -> {
		try { return future.get(); }
		catch(Exception ex) {
		    return "exception: " + ex.getMessage();
		}
	    })
	.forEach(System.out::println);
}

Here is the output from the code above. Notice that tasks are executed one after another followed by the shutdown task. This is because we used the newSingleThreadExecutor() method which creates an ExecutorService with that characteristic.

task 1 started
task 1 ended
task 2 started
task 2 ended
task 3 started
task 3 ended
task 1
task 2
task 3
shutdown

5. Executing Tasks with a Thread Pool

Let us now examine how a thread pool returned from newCachedThreadPool() behaves. The source code is shown below; it uses the DelayTask class defined above.

static private void example_4(String[] args) throws Exception
{
    ExecutorService esvc = Executors.newCachedThreadPool();
    List<Callable<String>> tasks =
	Arrays.asList(new DelayTask("task 1", 2),
		      new DelayTask("task 2", 3),
		      new DelayTask("task 3", 10),
		      () -> {
			  System.err.println("Requesting shutdown ..");
			  esvc.shutdown();
			  return "shutdown";
		      });
    esvc.invokeAll(tasks)
	.stream()
	.map(future -> {
		try { return future.get(); }
		catch(Exception ex) {
		    return "exception: " + ex.getMessage();
		}
	    })
	.forEach(System.out::println);
}

Here is the output from this code. Notice that a shutdown is requested of the ExecutorService after the tasks have been scheduled. The ExecutorService allows the tasks to complete execution before shutting down. It does not, however, allow any more tasks to be scheduled after shutdown() has been invoked.

Task task 1 started
Task task 2 started
Task task 3 started
Requesting shutdown ..
Task task 1 ended
Task task 2 ended
Task task 3 ended
task 1
task 2
task 3
shutdown

6. Shutdown Thread Pool Immediately

When it is necessary to shutdown the thread pool immediately as opposed to waiting for all tasks to complete, we can use the shutdownNow() method.

static private void example_5(String[] args) throws Exception
{
    ExecutorService esvc = Executors.newCachedThreadPool();
    List<Callable<String>> tasks =
	Arrays.asList(new DelayTask("task 1", 2),
		      new DelayTask("task 2", 3),
		      new DelayTask("task 3", 10),
		      () -> {
			  System.err.println("Requesting shutdown ..");
			  esvc.shutdownNow();
			  return "shutdown";
		      });
    esvc.invokeAll(tasks)
	.stream()
	.map(future -> {
		try { return future.get(); }
		catch(Exception ex) {
		    return "exception: " + ex.getMessage();
		}
	    })
	.forEach(System.out::println);
}

As the output below shows, the tasks that have not yet been completed are interrupted and the thread pool is terminated.

Task task 1 started
Task task 2 started
Task task 3 started
Requesting shutdown ..
task 3: interrupted
Task task 3 ended
task 1: interrupted
Task task 1 ended
task 2: interrupted
Task task 2 ended
task 1
task 2
task 3
shutdown

Summary

This article provided an introduction to threading in Java as well as the new Executor framework in Java 8. The Executor framework simplifies a lot of plumbing code related to thread creation and management. It also brings new capabilities to threading in Java including thread pools, worker threads which can schedule tasks sequentially. As the code above demonstrated, it is also possible to implement code to shutdown the thread pool cleanly after all the tasks have completed.

Java – Read File Line by Line Using Java 8 Streams

1. Introduction

Java 8 Streams provide a cool facility to apply functional-style operations on Collection-based classes such as List and Map. These functional-style operations are very expressive and allow elimination of much boiler-plate code for processing pipelines which operate on  elements in these collections.

2. A Streams Example

An example is shown below. Here we process a list of airport names as follows:

1. Select airports that start with “B”

2. Convert the airport name to uppercase

3. Sort the list

4. And print out the element

List<String> airports =
    Arrays.asList("Birmingham-Shuttlesworth International",
		  "Anchorage International",
		  "Deadhorse",
		  "Phoenix Sky Harbor International",
		  "Tucson International",
		  "Los Angeles International",
		  "San Francisco International",
		  "Burbank Bob Hope Airport",
		  "Long Beach Airport",
		  "Oakland International");

airports
    .stream()
    .filter(a -> a.startsWith("B"))
    .map(String::toUpperCase)
    .sorted()
    .forEach(System.out::println);

// prints the following
// BIRMINGHAM-SHUTTLESWORTH INTERNATIONAL
// BURBANK BOB HOPE AIRPORT

Wouldn’t it be nice to apply these processing pipelines to other sequences such as the lines of a file? In this article, we show how to do exactly that.

4. Reading a File Line By Line

To read a file line by line in Java, we use a BufferedReader instance and read in a loop till all the lines are exhausted.

try (BufferedReader in = new BufferedReader(new FileReader(textFile))) {
    String line;
    while ((line = in.readLine()) != null) {
        // process line here
    }
}

3. Implement a Spliterator Class

To turn a BufferedReader into a class capable of being used with the Java 8 Streams API, we need to provide an implementation of the Spliterator interface. Shown is the LineReaderSpliterator class which implements Spliterator<String> and turns a BufferedReader into a stream of lines.

public class LineReaderSpliterator implements Spliterator<String>
{
    private final BufferedReader reader;
    private java.io.IOException exception;

    public LineReaderSpliterator(BufferedReader reader) {
	this.reader = reader;
    }

    public java.io.IOException ioException() { return exception; }

    public int characteristics() {
	return DISTINCT | NONNULL | IMMUTABLE;
    }

    public long estimateSize() {
	return Long.MAX_VALUE;
    }

    public boolean tryAdvance(Consumer<? super String> action) {
	try {
	    String line = reader.readLine();
	    if ( line != null ) {
		action.accept(line);
		return true;
	    } else return false;
	} catch(java.io.IOException ex) {
	    this.exception = ex;
	    return false;
	}
    }

    public Spliterator<String> trySplit() { return null; }
}

3.1 Constructor

The LineReaderSpliterator is initialized with an instance of BufferedReader which serves as the input line source.

public LineReaderSpliterator(BufferedReader reader) {
    this.reader = reader;
}

3.2 Characteristics

The characteristics of the Spliterator must be indicated with the implementation of the characteristics() method. The result must be an OR-ed values from the following:

ORDERED: indicates that the order of elements is defined. The ordering is expected to be preserved in parallel computations.

DISTINCT: Each element is distinct from another element.

SORTED: The sequence is sorted. In our case, the lines may not be sorted so this bit is not set.

SIZED: This bit must be set to indicate that the estimate of size returned by estimateSize() is correct. For our case, we do not know the number of lines in a file so this bit is not set.

NONNULL: Elements are guaranteed to be non-null.

IMMUTABLE: Requires that element source should not be modified to add, replace or remove elements.

CONCURRENT: Indicates that element source can be modified concurrently with additions, replacements and removals from multiple threads.

SUBSIZED: If the spliterator can be split and child spliterators are SIZED and SUBSIZED.

In our case, the spliterator is specified as DISTINCT, NONNULL and IMMUTABLE.

3.3 Size estimation

Our spliterator does not know the size of the collection since the number of lines in the BufferedReader is not known. If the size is now known or is unbounded, the method must return Long.MAX_VALUE.

public long estimateSize() {
    return Long.MAX_VALUE;
}

3.4 Process Next Element

The method to process the next element is the tryAdvance() method which accepts a functional interface Consumer<? super T>. Our implementation attempts to read a line and if successful (EOF not reached) invokes action.accept(). If EOF is reached or an exception occurs, the method return false to indicate end-of-sequence.

public boolean tryAdvance(Consumer<? super String> action) {
    try {
	String line = reader.readLine();
	if ( line != null ) {
	    action.accept(line);
	    return true;
	} else return false;
    } catch(java.io.IOException ex) {
	this.exception = ex;
	return false;
    }
}

3.5 Can the Spliterator split?

If the spliterator can partitioned to return separate ranges of elements, a new Spliterator must be returned. We cannot partition the input into separate sequences so we return null.

public Spliterator<String> trySplit() { return null; }

 4. Using the Stream

We can now use the Spliterator implementation to convert a BufferedReader into a Java 8 stream as follows:

static private Stream<String> createStreamReader(BufferedReader reader)
{
    LineReaderSpliterator s = new LineReaderSpliterator(reader);
    return StreamSupport.stream(s, false);
}

The earlier streams example can now be written as shown.

BufferedReader reader = null;
try {
    reader = new BufferedReader(new FileReader(textFile));
    createStreamReader(reader)
	.filter(a -> a.startsWith("B"))
	.map(String::toUpperCase)
	.sorted()
	.forEach(System.out::println);
} finally {
    if ( reader != null ) reader.close();
}

Check the output shown below:

BALTIMORE-WASHINGTON INTERNATIONAL
BANGOR INTERNATIONAL
BIRMINGHAM-SHUTTLESWORTH INTERNATIONAL
BRADLEY INTERNATIONAL
BURBANK BOB HOPE AIRPORT
BURLINGTON INTERNATIONAL

The example does not store the names in a data structure. Rather, names are directly read from the file and the processing pipeline is applied to the sequence of elements.

Summary

We have demonstrated how to read lines from a file and process it using Java 8 streams. This requires implementation of a Spliterator class for delivering a “stream” view of any sequence. The advantage of such an approach is the ease of filtering and processing text files.

Difference Between HashMap and Hashtable in Java

1. Introduction

Java provides several ways of storing key-value maps (also known as dictionaries). The most common ones are java.util.HashMap and java.util.Hashtable. Let us explore the difference between these two classes.

2. Synchronization

Synchronization is a mechanism in Java for preventing multiple threads from interfering with each other and eliminating memory consistency errors.

When one variable (a resource) is visible to multiple threads at the same time, consistency issues arise when one thread attempts to modify the value while another thread is accessing it. To prevent these issues, some form of synchronization must be used.

While synchronization helps in eliminating consistency errors, it adds an overhead when used. In a single-threaded program (or when you can guarantee that a single thread will access the resource), you can use HashMap to eliminate this overhead. Create a HashMap as follows:

HashMap<String,Object> map = new HashMap<>();
map.put("currentTime", new Date());

However, when accessing or modifying a dictionary shared between multiple threads, you must use Hashtable. The following shows how to create a Hashtable:

Hashtable<String,Integer> tbl = new Hashtable();
tbl.put("count", 32);

3. Using Null Keys or Values

When your dictionary needs to contain null keys or values, you cannot use Hashtable since this is not allowed. You must use HashMap in this case.

If you need multiple threads reading or writing the HashMap, you can wrap the HashMap using Collections.synchronizedMap() as follows:

HashMap<String,Object> map = Collections.synchronizedMap(new HashMap<>());
map.put(...);

A HashMap can contain one null key and any number of nulls for values.

4. Predictable Iteration Order

A subclass of HashMap is LinkedHashMap which maintains a doubly-linked list of the entries in the Map. This allows traversal of the Map entries in a predictable order (in the order that the entries were inserted into the Map). If you need such a predictable ordering of the entries, then you can easily replace the HashMap with a LinkedHashMap as follows:

HashMap<String,Object> map = new LinkedHashMap<>();
map.put(...);

When using a Hashtable, such a predictable iteration order is not possible. If this is required, use a LinkedHashMap with a Collections.synchronizedMap() wrapper as above.

5. Iterating using Enumerator

While both HashMap and Hashtable support iteration over the entries using the entrySet(), Hashtable also provides an Enumeration of the entries using the Hashtable.elements() method. In addition, a Hashtable.keys() method also returns an Enumeration over the keys of the Hashtable.

Hashtable<String,Object> tbl = ...;
for(Enumeration<String> keys = tbl.keys() ; tbl.hasMoreElements() ; ) {
  System.out.println(keys.nextElement());
}

Conclusion

Here is how you can decide when to use HashMap or Hashtable:

  • For using as a shared resource between multiple threads in a single program, a Hashtable is preferred.
  • When the dictionary needs to contain null keys or values, a HashMap must be used.
  • A HashMap can be used in a multi-threaded environment by wrapping it with Collections.synchronizedMap().