“I hate quotations. Tell me what you know.” ― Ralph Waldo Emerson, The Essays of Ralph Waldo Emerson
Contents
1. Introduction
Do you want to run an external command from inside python? Maybe you want to startup a daemon process (no input or output), or perhaps invoke a command and capture its output?
Python supports several methods for calling an external command. Choice of the method used depends upon whether you want the output from the command, or send it input, or have close control over its lifecycle.
2. Using subprocess.call()
The recommended in python to execute an external command using the facilities provided by the subprocess module. Below, we use the call() function to invoke a simple command.
subprocess.call(['ls', '-l']) # prints total 20 -rwxr--r-- 1 xxxxxxx xxxxxxx 36 Jan 26 11:11 readme.txt ...
3. Redirecting STDOUT to a File
How do you redirect the output from the command to a file? Open the file wrapped in a with statement and use the stdout parameter.
with open('joe.txt', 'w') as f:
subprocess.call(['date'], stdout=f)
# cat joe.txt
Fri Jan 26 06:17:05 UTC 2018
4. Passing Input from STDIN to Command
How can I pass input into STDIN of the child process? Again, using call() you can open the file and pass the handle using the stdin parameter.
with open('readme.txt', 'r') as f:
subprocess.call(['cat'], stdin=f)
# prints the text in readme.txt
hello world
this is the second line.
5. Executing a Shell Command
Another parameter that can be passed to the call() function is shell=True. This causes the command line to be parsed by the shell.
subprocess.call('cat readme.txt', shell=True)
# prints
hello world
this is the second line.
This, however, is a security risk if the command line comes from an untrusted source such as user input. For example:
cmd = raw_input('enter a command >> ')
# suppose user enters "rm -rf /"
subprocess.call(cmd, shell=True)
# Boom! significant damage to your file system!
6. Exit Code from the Command
What about the exit code from the command? It is returned as the value from the function call().
print 'returncode: ', subprocess.call(['cat', 'jack.txt']) # prints cat: jack.txt: No such file or directory returncode: 1
7. Get More Information Using check_call()
While subprocess.call() allows you to perform quick and easy external command execution, you should use subprocess.check_call() when you need more information in case of command execution errors.
try:
print subprocess.check_call(['cat', 'jack.txt'])
except subprocess.CalledProcessError as x:
print 'exception: returncode: {}, command: {}'.format(x.returncode, x.cmd)
# prints
cat: jack.txt: No such file or directory
exception: returncode: 1, command: ['cat', 'jack.txt']
8. Capture Command Output in a String
While we saw how to redirect stdout to a file, what if we want the output from the command as a string? Use subprocess.check_output().
try:
print subprocess.check_output(raw_input('enter command >> '), shell=True)
except subprocess.CalledProcessError as x:
print 'exception: returncode: {}, command: {}, output = {}'.format(x.returncode, x.cmd, x.output)
# user enters "date -u" at the prompt
# prints
Fri Jan 26 10:08:08 UTC 2018
Errors are also captured as shown below.
try:
print subprocess.check_output(raw_input('enter command >> '), shell=True)
except subprocess.CalledProcessError as x:
print 'exception: returncode: {}, command: {}, output = {}'.format(x.returncode, x.cmd, x.output)
# user enters "ls -l joe" at the prompt
# prints
ls: cannot access joe: No such file or directory
exception: returncode: 2, command: ls -l joe
Note that using this method, you can also obtain partial output (if any) from the command, even if it fails later on. The following example raises the exception, and returns the partial output in the exception.
try:
print subprocess.check_output(raw_input('enter command >> '), shell=True)
except subprocess.CalledProcessError as x:
print 'exception: returncode: {}, command: {}, output = {}'.format(x.returncode, x.cmd, x.output)
# user enters "date -u; exit 1" at the prompt
# prints
exception: returncode: 1, command: date -u; exit 1, output = Fri Jan 26 10:15:39 UTC 2018
Conclusion
We have covered various aspects of using functions in the subprocess module to call external programs from python. Most aspects of the invocation can be controlled including redirecting standard output to file or a string, or arrange for the process to read input from a file.