Author: Jay Sridhar

How to Read a File from Resources Folder in Java

1. Introduction

When you build a java project and pack it into a jar (or a war), the files under the resources folder are included into the jar. These files may include configuration files, scripts and other resources needed during run time. When the software is executed, it may need to load the contents of these files for some kind of processing — may be properties, sql statements, etc. In this article, we show you how to load these resources when the program is running.

2. Packaging Resources

Check out the directory hierarchy below:

Maven packs all the files and folders under main/resources into the jar file at the the root. You can access these files and folders from your java code as shown below.

3. Loading the Resources

The following code snippet shows how to load resources packed thus into the jar or war file:

String respath = "/poems/Frost.txt";
InputStream in = sample2.class.getResourceAsStream(respath);
if ( in == null )
    throw new Exception("resource not found: " + respath);

Using the method Class.getResourceAsStream(String), you can get an InputStream to read the resource. The method returns null if the resource cannot be found or loaded.

To read binary resources, you can use directly use the InputStream instance. For reading a text resource, you can convert it to a Reader instance, possibly specifying the character encoding:

InputStreamReader inr = new InputStreamReader(in, "UTF-8");
int len;
char cbuf[] = new char[2048];
while ((len = inr.read(cbuf, 0, cbuf.length)) != -1) {
    // do something with cbuf
}

4. Using Absolute Path of Resource

To load a resource whose full path from the root of the jar file is known, use the full path starting with a “/“.

InputStream in = sample1.class.getResourceAsStream("/poems/Frost.txt");

5. Loading from Relative Paths

When you use a relative path (not starting with a “/“) to load a resource, it loaded relative to the class from which getResourceAsStream() is invoked. For example, to load “app.properties” relative to the invoking class, do not start the path with a “/“.

InputStream in = sample1.class.getResourceAsStream("app.properties");

Conclusion

In this article, we demonstrated how to use the Class.getResourceAsStream() method to load resources from the jar file or war file. An absolute resource path is resolved from the root of the jar file while a relative paths is resolved with respect to the loading class.

How to Convert UTF-16 Text File to UTF-8 in Java?

1. Introduction

In this article, we show how to convert a text file from UTF-16 encoding to UTF-8. Such a conversion might be required because certain tools can only read UTF-8 text. Furthermore, the conversion procedure demonstrated here can be applied to convert a text file from any supported encoding to another.

UTF-8 is a character encoding that can represent all characters (or code points) defined by Unicode. It is designed to be backward compatible with legacy encodings such as ASCII.

UTF-16 is another character encoding that encodes characters in one or two 16-bit code units whereas UTF-8 encodes characters in a variable number of 8-bit code units.

2. Supported Character Sets

You can find the characters sets supported by the JVM using the class java.nio.charset.Charset as follows:

for (Map.Entry e : Charset.availableCharsets().entrySet()) {
     System.out.println(e.getKey());
}

// prints the following
Big5
Big5-HKSCS
CESU-8
IBM-Thai
...
US-ASCII
UTF-16
UTF-16BE
UTF-16LE
UTF-32
UTF-32BE
UTF-32LE
UTF-8
...

3. Conversion Using java.io Classes

Java provides java.io.InputStreamReader class as a bridge between byte streams to character streams. Open the file using this class to be able to read character buffers in the specified encoding:

Reader in = new InputStreamReader(new FileInputStream(infile), "UTF-16");

Analogously, the class java.io.OutputStreamWriter acts as a bridge between characters streams and bytes streams. Create a Writer with this class to be able to write bytes to the file:

Writer out = new OutputStreamWriter(new FileOutputStream(outfile), "UTF-8");

With the Reader and Writer in place, it is trivial to copy data from the input file to the output file:

char cbuf[] = new char[2048];
int len;
while ((len = in.read(cbuf, 0, cbuf.length)) != -1) {
    out.write(cbuf, 0, len);
}

And that’s it! You have successfully read and converted data from UTF-16 to UTF-8. You can use this code to perform the conversion between any two character sets supported by your JVM.

4. Using String for Converting Bytes

Sometimes, you may have a byte array which you need converted and output in a specific encoding. You can use the String class for these cases as shown below. First convert the byte array into a String:

String str = new String(bytes, 0, len, "UTF-16");

Next, obtain the bytes in the required encoding by using the String.getBytes(String) method:

byte[] outbytes = str.getBytes("UTF-8");

Write the byte array to an OutputStream:

OutputStream out = new FileOutputStream(outfile);
out.write(outbytes);
out.close();

Note that while you could use the String class as shown to convert bytes, you should prefer using Reader/Writer combination when possible to avoid problems with multi-byte characters. Specifically, the byte array you have read may contain an incomplete multi-byte character at the beginning or the end. This may lead to character encoding errors.

Conclusion

When you need to convert text from one character encoding to another in Java, you have several options:

  • Using InputStreamReader and OutputStreamWriter bridge classes for conversion.
  • Using the String class directly with specified encoding.
  • A more advanced option is to use CharsetEncoder and CharsetDecoder class (not presented in this article).

Difference Between HashMap and Hashtable in Java

1. Introduction

Java provides several ways of storing key-value maps (also known as dictionaries). The most common ones are java.util.HashMap and java.util.Hashtable. Let us explore the difference between these two classes.

2. Synchronization

Synchronization is a mechanism in Java for preventing multiple threads from interfering with each other and eliminating memory consistency errors.

When one variable (a resource) is visible to multiple threads at the same time, consistency issues arise when one thread attempts to modify the value while another thread is accessing it. To prevent these issues, some form of synchronization must be used.

While synchronization helps in eliminating consistency errors, it adds an overhead when used. In a single-threaded program (or when you can guarantee that a single thread will access the resource), you can use HashMap to eliminate this overhead. Create a HashMap as follows:

HashMap<String,Object> map = new HashMap<>();
map.put("currentTime", new Date());

However, when accessing or modifying a dictionary shared between multiple threads, you must use Hashtable. The following shows how to create a Hashtable:

Hashtable<String,Integer> tbl = new Hashtable();
tbl.put("count", 32);

3. Using Null Keys or Values

When your dictionary needs to contain null keys or values, you cannot use Hashtable since this is not allowed. You must use HashMap in this case.

If you need multiple threads reading or writing the HashMap, you can wrap the HashMap using Collections.synchronizedMap() as follows:

HashMap<String,Object> map = Collections.synchronizedMap(new HashMap<>());
map.put(...);

A HashMap can contain one null key and any number of nulls for values.

4. Predictable Iteration Order

A subclass of HashMap is LinkedHashMap which maintains a doubly-linked list of the entries in the Map. This allows traversal of the Map entries in a predictable order (in the order that the entries were inserted into the Map). If you need such a predictable ordering of the entries, then you can easily replace the HashMap with a LinkedHashMap as follows:

HashMap<String,Object> map = new LinkedHashMap<>();
map.put(...);

When using a Hashtable, such a predictable iteration order is not possible. If this is required, use a LinkedHashMap with a Collections.synchronizedMap() wrapper as above.

5. Iterating using Enumerator

While both HashMap and Hashtable support iteration over the entries using the entrySet(), Hashtable also provides an Enumeration of the entries using the Hashtable.elements() method. In addition, a Hashtable.keys() method also returns an Enumeration over the keys of the Hashtable.

Hashtable<String,Object> tbl = ...;
for(Enumeration<String> keys = tbl.keys() ; tbl.hasMoreElements() ; ) {
  System.out.println(keys.nextElement());
}

Conclusion

Here is how you can decide when to use HashMap or Hashtable:

  • For using as a shared resource between multiple threads in a single program, a Hashtable is preferred.
  • When the dictionary needs to contain null keys or values, a HashMap must be used.
  • A HashMap can be used in a multi-threaded environment by wrapping it with Collections.synchronizedMap().

Converting String to Int in Java

1. Introduction

There are several ways of converting a String to an integer in Java. We present a few of them here with a discussion of the pros and cons of each.

2. Integer.parseInt()

Integer.parseInt(String) can parse a string and return a plain int value. The string value is parsed assuming a radix of 10. The string can contain “+” and “-” characters at the start to indicate a positive or negative number.

int value = Integer.parseInt("25"); // returns 25

int value = Integer.parseInt("-43"); // return -43

int value = Integer.parseInt("+9061"); // returns 9061

Illegal characters within the string (including period “.“) result in a NumberFormatException. Additionally a string containing a number larger than Integer.MAX_VALUE (231 - 1) also result in a NumberFormatException.

// throws NumberFormatException -- contains "."
int value = Integer.parseInt("25.0");

// throws NumberFormatException -- contains text
int value = Integer.parseInt("93hello");

// throws NumberFormatException -- too large
int value = Integer.parseInt("2367423890");

To explicitly specify the radix, use Integer.parseInt(String,int) and pass the radix as the second argument.

// returns 443
int value = Integer.parseInt("673", 8);

// throws NumberFormatException -- contains character "9" not valid for radix 8
int value = Integer.parseInt("9061", 8);

// returns 70966758 -- "h" and "i" are valid characters for radix 20.
int value = Integer.parseInt("123aghi", 20);

3. Integer.valueOf()

The static method Integer.valueOf() works similar to Integer.parseInt() with the difference that the method returns an Integer object rather than an int value.

// returns 733
Integer value = Integer.valueOf("733");

Use this method when you need an Integer object rather than a bare int. This method invokes Integer.parseInt(String) and creates an Integer from the result.

4. Integer.decode()

For parsing an integer starting with these prefixes: “0” for octal, “0x”, “0X” and “#” for hex, you can use the method Integer.decode(String). An optional “+” or “-” sign can precede the number. Al the following formats are supported by this method:

Signopt DecimalNumeral
Signopt 0x HexDigits
Signopt 0X HexDigits
Signopt # HexDigits
Signopt 0 OctalDigits

As with Integer.valueOf(), this method returns an Integer object rather than a plain int. Some examples follow:

// returns 53
Integer value = Integer.decode("0x35");

// returns 1194684
Integer value = Integer.decode("#123abc');

// throws NumberFormatException -- value too large
Integer value = Integer.decode("#123abcdef");

// returns -231
Integer value = Integer.decode("-0347");

5. Convert Large Values into Long

When the value being parsed does not fit in an integer (231-1), a NumberFormatException is thrown. In these cases, you can use analogous methods of the Long class: Long.parseLong(String), Long.valueOf(String) and Long.decode(String). These work similar to their Integer counterparts but return a Long object (or a long in the case of Long.parseLong(String)). The limit of a long is Long.LONG_MAX (defined to be 263-1).

// returns 378943640350
long value = Long.parseLong("378943640350");

// returns 3935157603823
Long value = Long.decode("0x39439abcdef");

6. Use BigInteger for Larger Numbers

Java provides another numeric type: java.math.BigInteger with an arbitrary precision. A disadvantage of using BigInteger is that common operations like adding, subtracting, etc require method invocation and cannot be used with operators like “+“, “-“, etc.

To convert a String to a BigInteger, just use the constructor as follows:

BigInteger bi = new BigInteger("3489534895489358943");

To specify a radix different than 10, use a different constructor:

BigInteger bi = new BigInteger("324789045345498589", 12);

7. Parse for Numbers Within Text

To parse for numbers interspersed with arbitrary text, you can use the java.util.Scanner class as follows:

String str = "hello123: we have a 1000 worlds out there.";
Scanner scanner = new Scanner(str).useDemiliter("\\D+");
while (s.hasNextInt())
  System.out.printf("(%1$d) ", s.nextInt());
// prints "(123) (1000)"

This method offers a powerful way of parsing for numbers, although it comes with the expense of using a regular expression scanner.

Summary

To summarize, there are various methods of converting a string to an int in java.

  • Integer.parseInt() is the simplest and returns an int.
  • Integer.valueOf() is similar but returns an Integer object.
  • Integer.decode() can parse numbers starting with “0x” and “0” as hex and octal respectively.
  • For larger numbers, use the corresponding methods in the Long class.
  • And for arbitrary precision integers, use the BigInteger class.
  • Finally, to parse arbitrary text for numbers, we can use the java.util.Scanner class with a regular expression.

InputStream to String Conversion in Java

1. Introduction

There are several ways of converting an InputStream to a String in java. Maybe you want to read the data and write it to a log file or do further processing. Here we look at several ways of accomplishing this task.

2. With InputStreamReader

Here is a simple implementation which uses the InputStreamReader to convert from bytes to characters. The code uses the platform default charset to decode the bytes. It reads input in chunks and appends the converted string to a StringBuilder.

private static String inputStreamToString(InputStream in)
    throws java.io.IOException
{
    BufferedReader br = null;
    try {
	InputStreamReader isr = new InputStreamReader(in);
	br = new BufferedReader(isr);
	char cbuf[] = new char[2048];
	int len;
	StringBuilder sbuf = new StringBuilder();
	while ((len = br.read(cbuf, 0, cbuf.length)) != -1)
	    sbuf.append(cbuf, 0, len);
	return sbuf.toString();
    } finally {
	if ( br != null ) br.close();
    }
}

3. Character Set Conversion

When converting input from a character set that is different from the platform default, you must specify the character set as follows:

private static String inputStreamToString(InputStream in,String charsetName)
    throws java.io.IOException
{
    BufferedReader br = null;
    try {
	InputStreamReader isr = new InputStreamReader(in, charsetName);
	br = new BufferedReader(isr);
	char cbuf[] = new char[2048];
	int len;
	StringBuilder sbuf = new StringBuilder();
	while ((len = br.read(cbuf, 0, cbuf.length)) != -1)
	    sbuf.append(cbuf, 0, len);
	return sbuf.toString();
    } finally {
	if ( br != null ) br.close();
    }
}

4. Using try-with-resources

When using a JDK 1.7 or later, you can use the try-with-resources block to eliminate some boilerplate code for exception handling:

private static String inputStreamToString(InputStream in,String charsetName)
    throws java.io.IOException
{
    try (BufferedReader br = new BufferedReader(new InputStreamReader(in, charsetName))) {
	char cbuf[] = new char[2048];
	int len;
	StringBuilder sbuf = new StringBuilder();
	while ((len = br.read(cbuf, 0, cbuf.length)) != -1)
		sbuf.append(cbuf, 0, len);
	return sbuf.toString();
    }
}

The try-with-resources block is used to automatically close resources when the block exits (whether normally or due to an exception).

try (BufferedReader br =
      new BufferedReader(new InputStreamReader(in, charsetName))) {
    // use the resource br here
    }

5. With ByteArrayOutputStream

Another option for converting InputStream to String uses the ByteArrayOutputStream. Here you can accumulate the bytes read from the InputStream and perform the final conversion to the desired character set.

private static String inputStreamToString(InputStream in,String charsetName)
    throws java.io.IOException
{
    try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
	byte buf[] = new byte[2048];
	int len;
	while ((len = in.read(buf)) != -1) out.write(buf, 0, len);
	return out.toString(charsetName);
    }
}

6. Using Apache Commons IO

Converting an InputStream to String can be achieved in a single line by using Apache Commons IO:

private static String inputStreamToString(InputStream in,String charsetName)
    throws java.io.IOException
{
    return IOUtils.toString(in, charsetName);
}

If you are using Maven as your build system, you need the following dependency:

<dependencies>
  <dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.4</version>
  </dependency>
</dependencies>

Conclusion

In this article, you learned several ways of converting an InputStream to String. Depending on your circumstances, you can pick the most appropriate one for your needs.

How do I check if a checkbox is checked in jQuery?

1. Introduction

A frequent concern when using jQuery for web development is: how to check if a checkbox is checked? There are multiple ways of accomplishing this task which we illustrate with examples in this article.

2. Using the DOM property “checked”

A checkbox is of the type HTMLInputElement and has a property called “checked” whose value is “true” if it is checked.

In the following example, this refers to the checkbox DOM element whose property “checked” is being checked. The event handler is invoked after the element is clicked and the function outputs the current state of the checkbox:

$('#check').change(function(ev) {
   if ( this.checked ) console.log('checked');
   else console.log('not checked');
});

Since no jQuery specific features are being used in this example, you can also write this as follows (without using jQuery):

document.getElementById('check').addEventListener('change', function() {
   if ( this.checked ) console.log('checked');
   else console.log('not checked');
});

3. Using jQuery is(‘:checked’)

jQuery provides an .is(‘:checked’) method which can be used as follows:

$('#check').change(function(ev) {
    if ( $(this).is(':checked') ) console.log('checked');
    else console.log('not checked');
});

If multiple checkboxes match the jQuery selector, this method can be used to check if at least one is checked. Consider this HTML which sets up a bunch of checkboxes with the same class.

<label class="checkbox-inline">
<input id="checkboxRed" type="checkbox"
   class="colorCheck"
   value="red"> Red
</label>
<label class="checkbox-inline">
<input id="checkboxGreen" type="checkbox"
   class="colorCheck"
   value="green"> Green
</label>
<label class="checkbox-inline">
<input id="checkboxRed" type="checkbox"
   class="colorCheck"
   value="blue"> Blue
</label>

Checkbox collection

The following code prints true if at least one is checked:

$('.colorCheck').change(function(ev) {
    console.log('any checked? ' + $('.colorCheck').prop(':checked'));
});

4. Using jQuery .prop()

Another method of checking the state using jQuery is to use the .prop() method:

$('#check').change(function(ev) {
    if ( $(this).prop('checked') ) console.log('4. checked');
    else console.log('4. not checked');
});

Check if one or more checkboxes are checked using the same construct:

$('.colorCheck').change(function(ev) {
    console.log('any checked? ' + $('.colorCheck').prop('checked'));
});

5. Using jQuery “checked” Selector

In the case when you have multiple related checkboxes, you can get a list of the checked ones by doing:

var boxes = $('.colorCheck:checked');
boxes.each(function(i, chkbox) {
  // process each checked box here
});

// How many are checked
console.log('Checked #' + boxes.length);

Conclusion

In this article, we presented various ways of checking the status of a checkbox: whether using jQuery or without. Depending on circumstances, you may prefer one method over another.

What Characters Need to be Escaped in XML Documents?

1. Introduction

Some characters are treated specially when processing XML documents. These are the characters which are used to markup XML syntax; when they appear as a part of a document rather than for syntax markup, they need to be appropriately escaped. These characters are:

"	&quot;		Double quote
'	&apos;		Single quote
<	&lt;		Left angle bracket
>	&gt;		Right angle bracket
&	&amp;		Ampersand

2. Character Data

All text that is not markup constitutes character data of the document. Within character data, “&” and “<” must not appear except when used in markup. However “>”, “”” and “‘” can appear directly within character data without having to be encoded.

<?xml version="1.0"?>
<valid>"'></valid>
<?xml version="1.0"?>
<invalid>&</invalid>
<?xml version="1.0"?>
<invalid><</invalid>

However, “>” cannot appear in the form “]]>” unless it is a part of CDATA ending sections.

<?xml version="1.0"?>
<invalid>]]></invalid>

The following is valid as “>” has been encoded properly:

<?xml version="1.0"?>
<valid>]]&gt;</valid>

2. Attributes

Within attributes, “>” is valid.

<valid attrib=">"></valid>

However, ‘”‘ is not valid within double quotes; it must be encoded using “&quot;”.

<valid attrib="&quot;"></valid>

Similarly “‘” is not valid within single quotes. It must be encoded as “&apos;”:

<valid attrib='&apos;'></valid>

3. Comments

Comments can appear anywhere in a document outside of markup. Within comments, none of the 5 special characters must be escaped or encoded.

<valid><!-- '"<>& --></valid>

In addition, the string “–” must not appear within comments.

<invalid><!-- Hello -- there --></invalid>

A consequence of this rule is that a comment must not end with “—>” (three dashes followed by a right-angle-bracket). The following is invalid:

<invalid><!-- Hello there ---></invalid>

4. Processing Instructions

Processing Instructions (PI) are used to add instructions for XML processing applications. None of the 5 special characters must be encoded within PI statements.

<valid><?execute <>"'& ?></valid>

A further restriction in the case of processing instructions is that the instruction name must not be the string “xml”; this name is reserved for standardization of the XML specification itself.

<invalid><?xml hello ?></invalid>

CData

CDATA sections are used to escape blocks of text containing characters which would otherwise be recognized as markup. This section begin with the string “<![CDATA[” and end with the string “]]>”. Within a CData section, none of the 5 special characters must be encoded.

<valid><![CDATA[[<greeting>Hello world: <>&'" </greeting>]]></valid>

However, within CData sections, the string “]]>” must not appear except to end the section:

<invalid><![CDATA[[This string("]]>") must not appear here]]></invalid>

The same can be re-written with two CData sections as follows:

<valid><![CDATA[[This string("]]]]><![CDATA[[>") must not appear here]]></valid>

Conclusion

This article demonstrated what the predefined XML entities are and the various circumstances in which they can be used.

Parsing XML in Python

1. Introduction

XML can be parsed in python using the xml.etree.ElementTree library. This article shows you how to parse and extract elements, attributes and text from XML using this library.

While this library is easy to use, it loads the whole XML document into memory and hence may not be suitable for processing large XML files or where run-time memory is an issue.

2. Sample XML

Here is a short snippet of the sample XML we will be working with.

<?xml version="1.0"?>
<catalog>
  <book id="bk101">
    <author>Gambardella, Matthew</author>
    <title>XML Developer's Guide</title>
    <genre>Computer</genre>
    <price>44.95</price>
    <publish_date>2000-10-01</publish_date>
    <description>An in-depth look at creating applications 
    with XML.</description>
  </book>
  <book id="bk102">
    <author>Ralls, Kim</author>
    <title>Midnight Rain</title>
...

3. Getting Started

3.1 Parsing an XML File

It is very simple to parse an XML using ElementTree. The following returns an ElementTree object which contains all the XML artifacts.

import xml.etree.ElementTree as ET

tree = ET.parse('catalog.xml')

Once you have the ElementTree object, you can get the root Element object using the tree.getroot() method:

root = tree.getroot()

3.2 Parsing an XML String

We use the ElementTree.fromstring() method to parse an XML string. The method returns root Element directly: a subtle difference compared with the ElementTree.parse() method which returns an ElementTree object.

print ET.fromstring('<a><b>1</b>2</a>')
# prints "<Element 'a' at ...>"

4. Working with Elements

From the Element object, you can get the XML tag name using the tag property.

print root.tag    # outputs "catalog"

Get a list of child Elements from any Element object using element.findall(‘*’). You can find specific children by name by passing the name of the tag e.g. element.findall(‘book’).

For example, the following code recursively processes all the elements in the XML and prints the name of the tag.

def show(elem):
    print elem.tag
    for child in elem.findall('*'):
        show(child)

show(root)

The above code can be modified to show nicely indented output of the tag names:

def show(elem, indent = 0);
    print ' ' * indent + elem.tag
    for child in elem.findall('*'):
         show(elem, indent + 1)

show(root)

To find a single element by name, use elem.find(tagName):

print root.find('book').tag    # prints "book"

5. Working with Attributes

XML attributes can be extracted from an Element object using the element.items() method which returns a sequence of name, value pairs. (The name-value pairs are returned in random order, not in the order they appear in the XML.)

for attrName, attrValue in elem.items():
    print attrName + '=' + attrValue

To retrieve a single attribute value by name, use the elem.get(attrName) method:

print root.find('book').get('id')    # prints "bk101"

Get a list of all attribute names defined on the element using elem.keys(). Returns an empty list if no attributes are defined.

print root.find('book').keys()
# prints ['id']

To get all the attributes as a python dictionary, use the elem.attrib property:

print root.find('book').attrib
# prints {'id': 'bk101'}

6. Retrieving Element Text Content

You can retrieve an element’s text content using the elem.text property as follows:

print ET.fromstring('<a>Hello<b>1</b>2</a>').text
# prints "Hello"

print ET.fromstring('<a>Hello<b>1</b>2</a>').find('b').text
#prints "1"

Text appearing after the element’s end tag is retrieved using elem.tail property:

print ET.fromstring('<a>Hello<b>1</b>2</a>').find('b').tail
# prints "2"

7. Using Path to Extract Content

Some of the Element object methods support extracting content by using a syntax similar to XPath:

Retrieve a descendant element:

print root.find('book/author').text
# prints "Gambardella, Matthew"

To obtain the text of the first matching element, use the elem.findtext() method as follows:

print root.findtext('book/author')
#prints "Gambardella, Matthew"

Retrieve and process a list of matching elements using elem.findall():

for e in root.findall('book/author'):
    print e.text

# prints the following
Gambardella, Matthew
Ralls, Kim
Corets, Eva
Corets, Eva
Corets, Eva
Randall, Cynthia
Thurman, Paula
Knorr, Stefan
Kress, Peter
O'Brien, Tim
O'Brien, Tim
Galos, Mike

Find a specific element by position. (Position indexes start with 1).

print root.find('book[2]/author').text
# prints "Ralls, Kim"

Here is an example to find and concatenate all text content using a reduce operation:

reduce(lambda x, y: x + '|' + y.text, root.findall("book/author"), '')

# prints "|Gambardella, Matthew|Ralls, Kim|Corets, Eva|Corets, Eva|Corets, Eva|Randall, Cynthia|Thurman, Paula|Knorr, Stefan|Kress, Peter|O'Brien, Tim|O'Brien, Tim|Galos, Mike"

Conclusion

This article demonstrated some aspects of parsing XML with python. We showed how to parse an XML file or an XML string and extract elements, attributes and text content.

See Also

What does <![CDATA[]]> in XML mean?

Introduction

CDATA stands for Character Data. A CDATA section in XML is used to escape text containing characters which would otherwise be recognized as markup. It can appear anywhere character data can occur.

Markup

A CDATA section is marked up starting with “<![CDATA[” and ending with “]]>“. Any character data (other than “]]>“) can appear within the section without needing to be escaped. For example, angle brackets (<>) and ampersands (&) which indicate XML markup need not be escaped within a CDATA section.

For example, the following is a CDATA section. The angle brackets surrounding “greeting” and “/greeting” need not be escaped. When processing this XML, the parser receives the text “<greeting>Hello, world!</greeting>” as character data and not as markup.

<![CDATA[<greeting>Hello, world!</greeting>]]>

In addition, parameter entity references are recognized within CDATA sections. For example, assume the following parameter entity is defined:

<!ENTITY AnEntity "Sample entity data here">

Within the following CDATA section, the entity reference “%AnEntity” is recognized and the value is replaced within character data passed to the XML processor.

<![CDATA[My value is %AnEntity]]>

Nesting

A CDATA section may not be nested inside another because “]]>” may not appear directly except to end the CDATA section. The following is invalid:

<![CDATA[We need the ending "]]>" here.]]>

Instead the above can be written in two sections as follows:

<![CDATA[We need the ending "]]]]><![CDATA[>" here.]]>

Attribute Value

Within a Document-Type Definition (DTD), an attribute value may be declared to be of type CDATA as follows:

<!ATTLIST img
          src CDATA #REQUIRED>

This declaration states that an img element must have a src attribute whose value type is CDATA.

Where is it used?

CDATA sections are used when larger amounts of verbatim text need to appear within XML documents and processed verbatim. Smaller quantities of such text can be properly encoded to escape the XML characters, but for larger text, it helps to preserve the meaning of the text without having to do so.