Python HTTP Client using urllib2

More on downloading HTTP URLs using urllib2.

“I like the night. Without the dark, we’d never see the stars.” ― Stephenie Meyer, Twilight

1. Introduction

Python provides the well-regarded urllib2 module for opening URLs. Let us investigate some of the capabilities of this module, shall we? Note that most use cases are better served by using the higher level Requests module. However, you should know about the available options.

Continue reading “Python HTTP Client using urllib2”

Python String Formatting Examples

Learn about all the facilities available with python string formatting

“The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom.” ― Isaac Asimov

 

1. Introduction

String formatting in python is somewhat complicated by the fact that there are a lot of flags and options. Let us learn about them.

Continue reading “Python String Formatting Examples”

How to Check Whether a File Exists in Python?

A review of various methods in python to check for file existence.

“Knowledge is a treasure, but practice is the key to it.” ― Lao Tzu

1. Introduction

Python offers several alternative ways of checking whether a file exists. Each of these ways come with several quirks. Examine each and pick the one that suits you.

Continue reading “How to Check Whether a File Exists in Python?”

Java – Zip Folder Tutorial

Learn how to zip a folder in Java

“Quotation, n: The act of repeating erroneously the words of another.”
― Ambrose Bierce, The Unabridged Devil’s Dictionary

1. Introduction

Java provides good support for reading and writing zip files. In this article, let us learn how to create a zip file from the contents of a folder in java.

Continue reading “Java – Zip Folder Tutorial”

Java – Reading a Large File Efficiently

Efficiently read text and binary files in Java

“The world is a book and those who do not travel read only one page.”
― Augustine of Hippo

1. Introduction

What’s the most efficient and easiest way to read a large file in java? Well, one way is to read the whole file at once into memory. Let us examine some issues that arise when doing so.

Continue reading “Java – Reading a Large File Efficiently”

Java String Format Examples

1. Introduction

Have you tried to read and understand Java’s String format documentation? I have and found it hard to understand. While it does include all the information, the organization leaves something to be desired.

This guide is an attempt to bring some clarity and ease the usage of string formatting in java.

Learn about string formatting in python? Check out this article.

2. String Formatting

Most common way of formatting a string in java is using String.format(). If there were a “java sprintf”, this would be it.

String output = String.format("%s = %d", "joe", 35);

For formatted console output, you can use printf() or the format() method of System.out and System.err PrintStreams.

System.out.printf("My name is: %s%n", "joe");

Create a Formatter and link it to a StringBuilder. Output formatted using the format() method will be appended to the StringBuilder.

StringBuilder sbuf = new StringBuilder();
Formatter fmt = new Formatter(sbuf);
fmt.format("PI = %f%n", Math.PI);
System.out.print(sbuf.toString());
// you can continue to append data to sbuf here.

3. Format Specifiers

Here is a quick reference to all the conversion specifiers supported.

Specifier Applies to Output
%a floating point (except BigDecimal) Hex output of floating point number
%b Any type “true” if non-null, “false” if null
%c character Unicode character
%d integer (incl. byte, short, int, long, bigint) Decimal Integer
%e floating point decimal number in scientific notation
%f floating point decimal number
%g floating point decimal number, possibly in scientific notation depending on the precision and value.
%h any type Hex String of value from hashCode() method.
 %n none Platform-specific line separator.
%o integer (incl. byte, short, int, long, bigint) Octal number
%s any type String value
%t Date/Time (incl. long, Calendar, Date and TemporalAccessor) %t is the prefix for Date/Time conversions. More formatting flags are needed after this. See Date/Time conversion below.
%x integer (incl. byte, short, int, long, bigint) Hex string.

3.1. Date and Time Formatting

Note: Using the formatting characters with “%T” instead of “%t” in the table below makes the output uppercase.

 Flag Notes
 %tA Full name of the day of the week, e.g. “Sunday“, “Monday
 %ta Abbreviated name of the week day e.g. “Sun“, “Mon“, etc.
 %tB Full name of the month e.g. “January“, “February“, etc.
 %tb Abbreviated month name e.g. “Jan“, “Feb“, etc.
 %tC Century part of year formatted with two digits e.g. “00” through “99”.
 %tc Date and time formatted with “%ta %tb %td %tT %tZ %tY” e.g. “Fri Feb 17 07:45:42 PST 2017
 %tD Date formatted as “%tm/%td/%ty
 %td Day of the month formatted with two digits. e.g. “01” to “31“.
 %te Day of the month formatted without a leading 0 e.g. “1” to “31”.
%tF ISO 8601 formatted date with “%tY-%tm-%td“.
%tH Hour of the day for the 24-hour clock e.g. “00” to “23“.
%th Same as %tb.
%tI Hour of the day for the 12-hour clock e.g. “01” – “12“.
%tj Day of the year formatted with leading 0s e.g. “001” to “366“.
%tk Hour of the day for the 24 hour clock without a leading 0 e.g. “0” to “23“.
%tl Hour of the day for the 12-hour click without a leading 0 e.g. “1” to “12“.
%tM Minute within the hour formatted a leading 0 e.g. “00” to “59“.
%tm Month formatted with a leading 0 e.g. “01” to “12“.
%tN Nanosecond formatted with 9 digits and leading 0s e.g. “000000000” to “999999999”.
%tp Locale specific “am” or “pm” marker.
%tQ Milliseconds since epoch Jan 1 , 1970 00:00:00 UTC.
%tR Time formatted as 24-hours e.g. “%tH:%tM“.
%tr Time formatted as 12-hours e.g. “%tI:%tM:%tS %Tp“.
%tS Seconds within the minute formatted with 2 digits e.g. “00” to “60”. “60” is required to support leap seconds.
%ts Seconds since the epoch Jan 1, 1970 00:00:00 UTC.
%tT Time formatted as 24-hours e.g. “%tH:%tM:%tS“.
%tY Year formatted with 4 digits e.g. “0000” to “9999“.
%ty Year formatted with 2 digits e.g. “00” to “99“.
%tZ Time zone abbreviation. e.g. “UTC“, “PST“, etc.
%tz Time Zone Offset from GMT e.g. “-0800“.

4. Argument Index

An argument index is specified as a number ending with a “$” after the “%” and selects the specified argument in the argument list.

String.format("%2$s", 32, "Hello");
// prints: "Hello"

5. Formatting an Integer

With the %d format specifier, you can use an argument of all integral types including byte, short, int, long and BigInteger.

Default formatting:
String.format("%d", 93);
// prints 93
Specifying a width:
String.format("|%20d|", 93);
// prints: |                  93|
Left-justifying within the specified width:
String.format("|%-20d|", 93);
// prints: |93                  |
Pad with zeros:
String.format("|%020d|", 93);
// prints: |00000000000000000093|
Print positive numbers with a “+”:

(Negative numbers always have the “-” included):

String.format("|%+20d|', 93);
// prints: |                 +93|
A space before positive numbers.

A “-” is included for negative numbers as per normal.

String.format("|% d|", 93);
// prints: | 93|

String.format("|% d|", -36);
// prints: |-36|
Use locale-specific thousands separator.

For the US locale, it is “,”:

String.format("|%,d|", 10000000);
// prints: |10,000,000|
Enclose negative numbers within parantheses (“()”) and skip the “-“:
String.format("|%(d|", -36);
// prints: |(36)|
Octal Output
String.format("|%o|"), 93);
// prints: 135
Hex Output
String.format("|%x|", 93);
// prints: 5d
Alternate Representation for Octal and Hex Output

Prints octal numbers with a leading “0” and hex numbers with leading “0x“.

String.format("|%#o|", 93);
// prints: 0135

String.format("|%#x|", 93);
// prints: 0x5d

String.format("|%#X|", 93);
// prints: 0X5D

6. String and Character Conversion

Default formatting:

Prints the whole string.

String.format("|%s|", "Hello World");
// prints: "Hello World"
Specify Field Length
String.format("|%30s|", "Hello World");
// prints: |                   Hello World|
Left Justify Text
String.format("|%-30s|", "Hello World");
// prints: |Hello World                   |
Specify Maximum Number of Characters
String.format("|%.5s|", "Hello World");
// prints: |Hello|
Field Width and Maximum Number of Characters
String.format("|%30.5s|", "Hello World");
|                         Hello|

Summary

This guide explained String formatting in Java. We covered the supported format specifiers. Both numeric and string formatting support a variety of flags for alternative formats.

Java File Examples

1. Introduction

Java provides the File class as an abstraction of file and directory path names. A path is presented as a hierarchical operating system independent view of files.

The File class supports a bunch of operations on files and directories which work on both Unix-like and Windows systems. Let us look at some of these operations in more detail.

2. Creating a New File

To create a new file, use the method createNewFile(). The following creates a new file “joe” in the current directory.

File file = new File("joe");
if ( file.createNewFile() ) System.out.println("created");

If a file specified already exists, the method returns false. In this case, the existing file is not touched or modified.

2.1. Hierarchy Not Created

The method does not automatically create the directory hierarchy if any of the parent directories do not exist. The following throws an exception if directory “joe” does not exist in the current directory.

File file = new File("joe/jack");
if ( file.createNewFile() ) System.out.println("created.");

// throws: java.io.IOException: Not a directory

However, if you run the above code after creating the parent directory, it works normally.

2.2. Creating Absolute Path File

In addition to specifying relative paths as above, you can also specify an absolute path as shown below:

File file = new File("/tmp/joe");
if ( file.createNewFile() ) System.out.println("created.");

// prints "created."

2.3. Permission Denied

When attempting to create a file where you don’t have permission, you get an exception:

File file = new File("/joe");
if ( file.createNewFile() ) System.out.println("created.");

// throws java.io.IOException: Permission denied

3. Delete a File

The File class provides a delete() method to delete a file. It is used as follows:

File file = new File("joe");
if ( file.delete() ) System.out.println("deleted.");

The method does not throw an exception if the file does not exists. It just returns false.

4. List Files

When a File object is created with a directory, you can use the list() method to list the files and directories in that directory. Here is an example of listing the files in the current directory using Java 8 streams.

Arrays.stream(file.list()).forEach(System.out::println);

You can also specify a FilenameFilter to select the required files. FilenameFilter is a functional interface so let us use a lambda expression to select and print the “.class” in the directory.

File file = new File(dirPath);
Arrays
    .stream(file.list((d, f) -> f.endsWith(".class")))
    .forEach(System.out::println);

In the above example, we use the FilenameFilter for filtering files by name. However, FileFilter is another functional interface provided by java that can be used to select files by some file property. In the following example, we select empty files.

File file = new File(path);
File[] emptyFiles = Arrays
    .stream(file.listFiles(f -> f.length() == 0));

Or select hidden files for processing.

File file = new File(path);
List<File> hiddenFiles = Arrays
    .stream(file.listFiles(f -> f.isHidden()))
    .collect(Collectors.toList());

5. Creating a Directory

The File class provides the method mkdir() to create a directory. It returns false if the directory cannot be created which could be for a variety of reasons. In other words, exceptions are not thrown including for the following reasons:

  • File or directory already exists.
  • No permission.
  • Hierarchy does not exist.
File file = new File(path);
if ( file.mkdir() ) System.out.println("created.");

If you want to create all non-existent directories of the hierarchy, use mkdirs() instead.

6. Renaming a File

To rename a file, use renameTo(). You must pass in a target File object to rename the file to.

File source = new File(pathA);
File target = new File(pathB);
if ( source.renameTo(target) ) System.out.println("renamed.");

Note that this method comes with a bunch of caveats including:

  • Might not be able to move a file from one file system to another.
  • Might not work if the target path exists. (In other words, might not overwrite the target file.)
  • It might not be atomic. In other words, removing the old file and creating the new file are two separate operations in which one might fail!

With all these warnings, it is better to check the return value to be sure the operation succeeded.

Summary

The Java File class is an system independent abstraction of file and directory paths. It provides a bunch of operations to create a file, remove a file, create a directory including the hierarchy and renaming files.

Java NIO – Using ByteBuffer

1. Introduction

Java provides a class ByteBuffer which is an abstraction of a buffer storing bytes. While just a little bit more involved than using plain byte[] arrays, it is touted as more performant. in this article, let us examine some uses of ByteBuffer and friends.

Continue reading “Java NIO – Using ByteBuffer”