I was asked recently about how to test the argument parsing bit of an application that used argparse. argparse is a built in Python library for dealing with parsing command line arguments for command line interfaces, CLI’s.

You know, like git clone <repo address>.

  • git is the application.
  • <repo address> is a command line argument.
  • clone is a sub-command.

Well, that might be a bad example, as I’m not going to use subcommands in my example, but lots of this still applies, even if you are using subcommands. Anyway, loads of applications use command line arguments, also sometimes called flags and options.

Even small utility scripts that started out with no arguments can grow arguments over time. And getting the parsing of the options right can be tricky and weird, even if you’ve done it before, so testing that bit totally makes sense.

Smug Answer

My first answer is to use click or typer as they are great to work with and have built in CliRunner objects, explicitly to help with testing. argparse doesn’t have built in testing tools. But it’s got some features that help.

There are also sometimes good reasons to use argparse.

  • Maybe you don’t have any other dependencies, why add one just for argument parsing?
  • Especially single file scripts and applications that you’re not packaging, just sharing as a single file.
  • Maybe you just really like argparse. Honestly, after researching this topic, I might use argparse more. It’s got some cool features.

Design for Test

First off, let’s design the application so it’s ready for testing. This applies to all applications, even small scripts.

Here’s a few good things to do with all CLI applications.

  • Have a main() function called by a if __name__ == "__main__" block that either contains all of your logic, or calls other functions.
  • Have a parse_args() that contains all of your interactions with argparse.
  • Have all of the logic of the application or script in a function.
  • This way our test code can import the application, test the parse_args() and main() function, and any other function. Or just call main() if you want.
  • But this makes testing easier.

Now, in more detail:

  • Have a if __name__ == "__main__": block that calls a single function, main(), with no arguments.

    • When running from the command line, this gets called.
    • When testing, we can import the different pieces and test separately, if we want.

  • Define your main(arg_list: list[str] | None = None) function to take a list of strings, or None, and default to None. This is going to be our argument list when testing, and when actually running, the None will be fine, as the __name__ == "__main__" block just calls main() with no args.

    • Seems weird at first. Bear with me. It will make sense.

  • The first thing you do in the main() function is call parse_args(arg_list) with the arg_list passed in to main, and assign the return value to args or params or whatever you want to call your variable.

  • The parse_args(arg_list: list[str] | None) function also takes either a list of strings, or None. No default is necessary.

    • This function will set up create a parser, add arguments, and call parser.parse_args(arg_list) with the arg_list passed in.
    • argparse’s parse_args() function, if passed in a list, will parse it. If it’s passed in None, it will parse sys.argv instead.
    • It’s starting to make sense now, isn’t it.

Test code

Now for our test code, we can import our application, or script, or file, or whatever and pull out the main() and parse_args() and other bits to test as a whole or by themselves, and being able to unittest parse_args() is super cool. Even if you don’t have any other unit tests. It will help you understand how to use argparse with your little unit test playground.

Also, when testing from an imported script, your code runs in process, so mocking and such work, if you want to fake or stub out side effects.

Adding -d/--debug and ‘-p/--preview’

While we’re at it, There are a couple of flags I like to add to CLI applications.

  • -d/--debug for printing out debug print statements to follow the flow of the script/application. You can write a debug_print() function that checks args.debug or just check it yourself whenever you want to print some info that will help with debugging.
    • I don’t care about testing this part, so throw a # pragma: no cover on it.
  • '-p/--preview for doing all of the logic, but not actually doing any side effects.
    • If you end up calling an external API, just print what you would have called instead in preview mode.

Lots of tools you probably use every day have these features as part of their user accessible API.

  • If you want to expose these, take those # pragma: no cover lines out and test it.
  • If you don’t want to expose them, you can set help=argparse.SUPPRESS and those flags won’t show up in the help description.
    • Click and Typer call this “hidden”.

Testing stdout

One of the beautiful parts of -p/--preview is that now you don’t have to mock or stub stuff.

You can test with pytest and the builtin fixture capsys and just check to make sure the output is correct with various command line flags.

Using shlex.split()

When testing CLI’s, I’d like to think in terms of the command line I’m passing to an application, which is a space delimited string. You know, something like --goodbye Brian.

The problem is that argparse and it’s parse_args() function accept lists of strings, like ["--goodbye", "Brian"]. This makes sense because that’s what sys.argv looks like.

It’s not bad for short things, but it gets annoying fast, especially when testing lots of combinations.

Luckily, Python has a builtin function, shlex.split() that does this work for us. It takes a space delimited string and produces a list of strings. Perfect. I use this all the time when testing CLIs.

Using subprocess.run()

If you cannot split up your application or script and importing is not an option, well, then you’re stuck with calling it with subprocess.run().

There are downsides to this:

  • It’s gonna run, and do any side effects. So hopefully you have some flags to disable that bit for testing.
  • It’s in a separate process, so introspection and mocking are either not possible, or at the very least, I don’t know how to do it.

Then again, plenty of applications are fine to test this way.

Take “Hello, World!”, for example.

An example: “Hello, World!” with argparse

Let’s look at some code. So, I need an example. How about “Hello, World!”?

You totally don’t need argparse for “Hello, World!”.

But let’s say we add a few requirements:

  • If a name is passed in, we’d like it to replace “World” with the name.
    • ex: python hello.py Brian should print "Hello, Brian!"
  • If someone passes in -g or --goodbye, “Hello” should be replaced with “Goodbye”.
    • ex: python hello.py -g Brian should print "Goodbye, Brian!"

Stick with me here, of course this is silly, but I’m trying to come up with an easy example that uses argparse.

I’ve got the code here, but it’s also in a repo: github.com/okken/test-argparse-apps-hello-world.

hello.py

import argparse


def parse_args(arg_list: list[str] | None):
    parser = argparse.ArgumentParser()

    parser.add_argument('name', type=str, default='World',
                        nargs='?', help='any name')

    parser.add_argument('-g', '--goodbye', action='store_true',
                        help='say goodbye instead')

    parser.add_argument('-d', '--debug', action='store_true',
                        help=argparse.SUPPRESS)


    args = parser.parse_args(arg_list)

    if args.debug:  # pragma: no cover
        print('--- debug output ---')
        print(f'  {args=}')
        print(f'  {args.goodbye=}, {args.name=}')
        print('')
    return args


def main(arg_list: list[str] | None = None):
    args = parse_args(arg_list)

    if args.goodbye:
        print(f'Goodbye, {args.name}!')
    else:
        print(f'Hello, {args.name}!')


if __name__ == '__main__':
    main()

test_hello.py

import subprocess
import pytest
import shlex
from hello import parse_args, main

def test_shlex():
    # I want to write this
    command = '-d -g Brian'

    # command parsers want this
    as_list = ['-d', '-g', 'Brian']

    # shlex.split() does the work for me
    assert shlex.split(command) == as_list


test_cases = [
    ("", "Hello, World!"),  # no args
    ("Okken", "Hello, Okken!"),  # one arg
    ("-g", "Goodbye, World!"),  # the other arg
    ("--goodbye", "Goodbye, World!"),  # long form
    ("Okken -g", "Goodbye, Okken!"),  # both args
]

@pytest.mark.parametrize('command, expected_output', test_cases)
def test_main(capsys, command, expected_output):
    main(shlex.split(command))
    output = capsys.readouterr().out.rstrip()
    assert output == expected_output


@pytest.mark.parametrize('command, expected_output', test_cases)
def test_app(command, expected_output):
    full_command = ["python", "hello.py"] + shlex.split(command)
    result = subprocess.run(full_command,
                            capture_output=True, text=True)
    output = result.stdout.rstrip()
    assert output == expected_output


@pytest.mark.parametrize(
    'command, debug, goodbye, name',
    [
        # no params
        ("", False, False, 'World'),
        # each param
        ("-d", True, False, 'World'),
        ("-g", False, True, 'World'),
        ("Name", False, False, 'Name'),
        # all params
        ("-d -g Earth", True, True, 'Earth'),
        # long form
        ("--goodbye", False, True, 'World'),

    ])
def test_parse_args(command, debug, goodbye, name):
    args = parse_args(shlex.split(command))

    # combine test into in one assert
    assert ((args.debug, args.goodbye, args.name) ==
            (debug, goodbye, name))

    # or split them up, either works
    assert args.debug == debug
    assert args.goodbye == goodbye
    assert args.name == name