“A lie gets halfway around the world before the truth has a chance to get its pants on.” ― Anonymous
Contents
1. Introduction
Python provides the itertools package which provides convenience functions for many common iterator operations. We have covered count(), cycle() and chain() in the first part of this series, and compress(), dropwhile(), and groupby() in the second part. In this article, we present a few examples of ifilter(), islice(), imap() and izip().
2. Filtering Items
First off, we have itertools.ifilter() which filters an iterable or a list for items for which a predicate function returns true.
Consider this simple example which selects all characters with a numeric value greater than ‘e’.
print list(itertools.ifilter(lambda c : c > 'e', 'abcdefg')) # prints ['f', 'g']
Select a few random even integers?
print 'random evens: ', list(itertools.ifilter(lambda n : n % 2 == 0, [random.randint(0, 20) for x in xrange(0, 20)])) # prints random evens: [2, 12, 14, 8, 4, 12, 2, 6, 0, 20, 14, 14]
Or some random numbers divisible by 3? You get the picture.
print 'randoms div 3: ', list(itertools.ifilter(lambda n : not n % 3, [random.randint(0, 20) for x in xrange(0, 20)])) # prints randoms div 3: [3, 15, 3, 12]
2.1. Difference between ifilter() and filter()
Python has a global function called filter() which also filters items in a list through a predicate function. So what is the difference between ifilter() and filter()?
Well, itertools.ifilter() returns an iterator while filter() returns a list. While this might not matter for small lists, it makes a difference when processing a large number of items. (Check the example under imap()).
3. Slices of Arrays
The next function in the collection is itertools.islice() which returns an iterator for looping over slices in some ordered fashion.
The following example extracts every third character from the alphabet.
print list(itertools.islice(string.ascii_lowercase, 0, None, 3)) # prints ['a', 'd', 'g', 'j', 'm', 'p', 's', 'v', 'y']
In the following example, islice() is used to select elements starting with fourth, skipping alternate items.
a = [random.randint(0, 20) for x in xrange(0, 20)] print a print list(itertools.islice(a, 4, None, 2)) # prints [8, 11, 9, 10, 15, 7, 7, 13, 10, 1, 11, 1, 4, 4, 0, 6, 6, 15, 11, 14] [15, 7, 10, 11, 4, 0, 6, 11]
3.1. Difference between Array Slice Syntax and islice()
What is the difference between the array slice syntax and islice()? Well, the slice syntax allows you to select a single slice, whereas islice() allows you to loop over multiple slices in an ordered fashion.
3.2. Fibonacci Generator
Another difference is that islice() accepts an iterable while the slice syntax requires an array. For example, suppose you have a generator and you want to obtain slices from it.
Consider the following fibonacci generator from which we obtain a few values using islice().
def fib():
x, y = 1, 1
while True:
yield x
x, y = y, x + y
print list(itertools.islice(fib(), 8))
# prints
[1, 1, 2, 3, 5, 8, 13, 21]
4. Map Functional Computation using imap
Like the python global function map(), itertools provides imap() which works with iterables.
Here is an example to generate a random string. First a random number in the correct range (65, 90) is generated and imapped to a character and finally joined.
print ''.join(itertools.imap(lambda x : chr(x), [random.randint(65, 90) for x in xrange(0, 10)])) # prints RONWPAELVY
You can use this to generate a random-looking id (maybe a filename or customer id or something). **DO NOT* use to generate a password as it lacks sufficient entropy.
You can pass multiple iterables to imap(). In the following example, we include characters from the lowercase set too.
print ''.join(itertools.imap(lambda x, y : chr(x) + chr(y), [random.randint(65, 90) for x in xrange(0, 10)], [random.randint(97, 122) for x in xrange(0, 5)])) # prints GaSzQkIrAs
When you specify None as the mapping function, imap() returns the items in the iterables as tuples. The mapping is as long as the shortest iterable. In other words, the function terminates as soon as one of the iterables terminates.
print list(itertools.imap(None, [random.randint(0, 20) for x in xrange(20)], xrange(10))) # prints [(7, 0), (2, 1), (8, 2), (6, 3), (12, 4), (6, 5), (4, 6), (8, 7), (20, 8), (0, 9)]
4.1. So where is imap useful?
As with the other itertools functions, you should consider imap() in preference to map() when you have a large number of elements that you need some computation on. Consider the following example of doubling and summing a billion numbers using only iterables. (This fails when using map(), at least on my machine).
arr = xrange(1000000000) #map(lambda x : x * 2, arr) <-- fails print sum(itertools.imap(lambda x : x * 2, arr)) # prints 999999999000000000
5. Zip Items from Iterable using izip
Similar to the zip() global function, itertools.izip() returns corresponding tuples from the iterables.
print list(itertools.izip([random.randint(0, 20) for x in xrange(20)], xrange(10))) # prints [(18, 0), (16, 1), (13, 2), (10, 3), (6, 4), (12, 5), (15, 6), (2, 7), (10, 8), (20, 9)]
Used this way, it returns the same result as using imap() with a None for the function as described above.
Conclusion
As a followup to part 1 and part 2 of this series on python itertools, we covered ifilter(), islice(), imap() and izip() functions.