July 31, 2019
There are many scenarios where you want to write a small programs that are designed to automate simple tasks. For instance, you want to resize thousands of images in your computer to a smaller sizes, or send email to multiple user whenever a row of data added into a csv file.
This is what people usually called scripting. And Python is a powerful programming language for this kind of job. Because, there are a bunch of built-in modules and third-party libraries that helps you to do some cool automation stuff with Python.
One of the problems you might encounter when you want to automate stuff is running other software that written in other programming languages. It could be the system software that built-in to your operating system. Or, it could be a third-party software that you have installed from the internet.
In effect, Python offers a built-in module called subprocess. It's a Python Standard Library that used to spawn new processes (programs), read the input/output/error, and obtain their return codes.
One of the very basic usage of subprocess module is by using the
run method. This method allows you to run external program by passing the command as a string inside the first argument.
In this example, we're going to execute the
ls command, which is a command that available in Unix operating systems (MacOS, Linux, etc.) to print all files and folders inside current directory.
import subprocess subprocess.run('ls')
The output will be:
$ python demo.py demo.py filename.txt
If you are on windows, and you are trying to run commands like
dir, you will get an error. It's because that command is built-in to the shell. So, to fix it, you can add the
shell=True argument to the
run method like below:
shell=True argument also allows you to run your entire command as a string. Because, if you dont enable the
shell argument, you actually can't pass the entire command as a string like we did. Instead, we need to pass our command as a list of arguments.
If you pay attention to the console, you will see that the command result is automatically printed out to the console. Even if we don't use
subprocess.run method doesn't capture our command output by default.
To prove this, we can try to capture our command by putting it into a variable.
result = subprocess.run('ls') print(result)
You will see something like this:
demo.py filename.txt CompletedProcess(args='ls', returncode=0)
First of all, the interpreter execute the
subprocess.run method, run the
ls command, and print the output to the terminal. Then, we try to print the value inside the
result variable and get this weird
CompletedProcess object. What's going on here?
As I said, the
subprocess.run method doesn't return the process ouput. It just print the result to the console. One way to get the process output is by adding the
capture_output=True named argument, and access the
stdout property of the result.
result = subprocess.run('ls', capture_output=True) print(result.stdout)
Now, instead of printing out the output directly to the terminal, the
run method return the output to us, so we can grab it and print it out the the terminal by ourself.
There is still a problem though. If you check the type of our
result.stdout, you can see that the type is not string, it's bytes. So what's the difference?
result = subprocess.run('ls', capture_output=True) print(type(result.stdout))
Well, the big difference of bytes and string in Python is that bytes is immutable / cannot be modified. Bytes object are machine readable, in other hand, strings are human readable.
To convert bytes object into strings, we can use the
decode method from the bytes object.
result = subprocess.run('ls', capture_output=True) print(result.stdout.decode())
Or, we can add
text=True parameter to our
run method. This makes it automatically decode our output into a string.
result = subprocess.run('ls', capture_output=True, text=True) print(result.stdout)
To make sure that your command runs properly without any error, you can check the return code of your process. Usually, if the return code is 0, your code run successfully. Otherwise, if the return code is 1, something went wrong.
For example, if we want pass the a wrong argument to the program we want to run, it will throw an error like below.
# notexist folder is not exist! result = subprocess.run(['ls', 'notexist'])
ls: notexist: No such file or directory
To make sure if the program doesn't throws an error, we can check the return code like below.
if result.returncode == 0: # Do something else: print("Something went wrong!")
Also, the result comes with the
stderr property. Use it if you want to see the error message.
if result.returncode == 0: # Do something else: print(result.stderr)
I'm a software engineer specialized in iOS and full-stack web development. If you have a project in mind, feel free to contact me and start the conversation.