Category: Python

How to Avoid Deadlock when Calling External Command from Python

Use pipes from multiple threads for bidirectional communication.

“The truth will set you free, but first it will piss you off.” ― Joe Klaas, Twelve Steps to Happiness

1. Introduction

Invoking an external process from python and interacting with it can be quite tricky. This is especially true if the interaction is duplex i.e. involving both reading and writing to it. Such an interaction can cause deadlocks since both processes can end up waiting for output from the other. One way to avoid such deadlocks is to separate the reader and writer parts to different threads. We demonstrate such an approach in this article.

Continue reading “How to Avoid Deadlock when Calling External Command from Python”

Run External Command in Python Using Popen

Use subprocess.Popen for more control over the child process

“The world is a book and those who do not travel read only one page.” ― Augustine of Hippo

1. Introduction

In a previous article, we looked at calling an external command in python using subprocess.call(). That method allows you to accomplish several tasks during the invocation:

Continue reading “Run External Command in Python Using Popen”

How to Call an External Command in Python

Learn about the various aspects of the subprocess module.

“I hate quotations. Tell me what you know.” ― Ralph Waldo Emerson, The Essays of Ralph Waldo Emerson

1. Introduction

Do you want to run an external command from inside python? Maybe you want to startup a daemon process (no input or output), or perhaps invoke a command and capture its output?

Continue reading “How to Call an External Command in Python”

Python itertools – compress, dropwhile, takewhile, groupby

Boost your python expertise with these examples from itertools.

“Never put off till tomorrow what may be done day after tomorrow just as well.” ― Mark Twain

1. Introduction

We are continuing with our series on python itertools. In this article, we cover compress, dropwhile, takewhile and groupby.

Continue reading “Python itertools – compress, dropwhile, takewhile, groupby”

How to Check Whether a File Exists in Python?

A review of various methods in python to check for file existence.

“Knowledge is a treasure, but practice is the key to it.” ― Lao Tzu

1. Introduction

Python offers several alternative ways of checking whether a file exists. Each of these ways come with several quirks. Examine each and pick the one that suits you.

Continue reading “How to Check Whether a File Exists in Python?”

Python itertools – count, cycle and chain

A gentle introduction to python itertools

“Laughter is poison to fear.” ― George R.R. Martin, A Game of Thrones

1. Introduction

Python provides a module called itertools which, as the name suggests, provides a bunch of conveniences for dealing with iterations and looping. While you could spend your entire python career without ever having to touch this module, trust me when I say your life will be enriched if you at least know about what is available in itertools.

Continue reading “Python itertools – count, cycle and chain”

Updating a File from Multiple Threads in Python

Use locking to properly share resources across multiple threads

“You’re not obligated to win. You’re obligated to keep trying. To the best you can do everyday.” ― Jason Mraz

1. Introduction

Resource management in multi-threaded programs is a tricky situation. Say, for example, you have a counter variable which is read and modified by multiple threads. If you do not exercise proper care, it is possible that inconsistent values are read and/or propagated. This leads to subtle and hard-to-debug errors.

Continue reading “Updating a File from Multiple Threads in Python”

Python Threading Tutorial

A beginner’s guide to threading in python

“The truth is like salt. Men want to taste a little, but too much makes everyone sick.” ― Joe Abercrombie, The Heroes

1. Introduction

Let us learn about python threading. Threading refers to executing a task in a separate thread of control, freeing up the main thread for other activities. It is very useful for executing time-consuming tasks without holding up the rest of the program. Typical applications include servers which handle each request in a thread and programs with a user-interface which execute complex tasks in a background thread.

Continue reading “Python Threading Tutorial”

Proper Cleanup of Resources in Python using the With Statement

The python with statement is damn useful! Learn the intricacies of the with statement.

“Accept who you are. Unless you’re a serial killer.” ― Ellen DeGeneres, Seriously… I’m Kidding

1. Introduction

Resource acquisition and cleanup is a very crucial aspect of programs. Resources need to be properly cleaned and released as soon as their usage is complete. Failing which scare resources might not be available to other programs running on the same machine. This is especially true for long-running programs such as servers and daemons.

Continue reading “Proper Cleanup of Resources in Python using the With Statement”

Python Class Methods – Implementing Ordered Dictionary

Use python class special methods to add idiomatic behavior to your class.

“It is the time you have wasted for your rose that makes your rose so important.” ― Antoine de Saint-Exupéry, The Little Prince

1. Introduction

The python class system supports a number of special methods which allows class implementations to mimic built-in python objects objects. These special methods are invoked by the python run-time system in a way similar to the built-in classes. For example, the method __len__() can be defined by a user class which will be invoked when the client code attempts to find the length of the object using len(obj).

Continue reading “Python Class Methods – Implementing Ordered Dictionary”