Mocks Are Weird
If this is the first time you’ve encountered test doubles like mocks, stubs, and spies, it’s gonna get real weird real fast. It’s fun though, and quite powerful.
The mock package is used to swap out pieces of the system to isolate bits of our code under test from the rest of the system. Mock objects are sometimes called test doubles, spies, fakes, or stubs. Between pytest’s own monkeypatch fixture (covered in Using monkeypatch) and mock, you should have all the test double functionality you need.
Mocks Are Weird | |
|---|---|
|
|
If this is the first time you’ve encountered test doubles like mocks, stubs, and spies, it’s gonna get real weird real fast. It’s fun though, and quite powerful. |
The mock package is shipped as part of the Python standard library as unittest.mock as of Python 3.3. In earlier versions, it’s available as a separate PyPI-installable package as a rolling backport. What that means is that you can use the PyPI version of mock with Python 2.6 through the latest Python version and get the same functionality as the latest Python mock. However, for use with pytest, a plugin called pytest-mock has some conveniences that make it my preferred interface to the mock system.
For the Tasks project, we’ll use mock to help us test the command-line interface. In Coverage.py: Determining How Much Code Is Tested, you saw that our cli.py file wasn’t being tested at all. We’ll start to fix that now. But let’s first talk about strategy.
An early decision in the Tasks project was to do most of the functionality testing through api.py. Therefore, it’s a reasonable decision that the command-line testing doesn’t have to be complete functionality testing. We can have a fair amount of confidence that the system will work through the CLI if we mock the API layer during CLI testing. It’s also a convenient decision, allowing us to look at mocks in this section.
The implementation of the Tasks CLI uses the Click third-party command-line interface package.[23] There are many alternatives for implementing a CLI, including Python’s builtin argparse module. One of the reasons I chose Click is because it includes a test runner to help us test Click applications. However, the code in cli.py, although hopefully typical of Click applications, is not obvious.
Let’s pause and install version 3 of Tasks:
| | $ cd /path/to/code/ |
| | $ pip install -e ch7/tasks_proj_v2 |
| | ... |
| | Successfully installed tasks |
In the rest of this section, you’ll develop some tests for the “list” functionality. Let’s see it in action to understand what we’re going to test:
| | $ tasks list |
| | ID owner done summary |
| | -- ----- ---- ------- |
| | $ tasks add 'do something great' |
| | $ tasks add "repeat" -o Brian |
| | $ tasks add "again and again" --owner Okken |
| | $ tasks list |
| | ID owner done summary |
| | -- ----- ---- ------- |
| | 1 False do something great |
| | 2 Brian False repeat |
| | 3 Okken False again and again |
| | $ tasks list -o Brian |
| | ID owner done summary |
| | -- ----- ---- ------- |
| | 2 Brian False repeat |
| | $ tasks list --owner Brian |
| | ID owner done summary |
| | -- ----- ---- ------- |
| | 2 Brian False repeat |
Looks pretty simple. The tasks list command lists all the tasks with a header. It prints the header even if the list is empty. It prints just the things from one owner if -o or --owner are used. How do we test it? Lots of ways are possible, but we’re going to use mocks.
Tests that use mocks are necessarily white-box tests, and we have to look into the code to decide what to mock and where. The main entry point is here:
| | if __name__ == '__main__': |
| | tasks_cli() |
That’s just a call to tasks_cli():
| | @click.group(context_settings={'help_option_names': ['-h', '--help']}) |
| | @click.version_option(version='0.1.1') |
| | def tasks_cli(): |
| | """Run the tasks application.""" |
| | pass |
Obvious? No. But hold on, it gets better (or worse, depending on your perspective). Here’s one of the commands—the list command:
| | @tasks_cli.command(name="list", help="list tasks") |
| | @click.option('-o', '--owner', default=None, |
| | help='list tasks with this owner') |
| | def list_tasks(owner): |
| | """ |
| | List tasks in db. |
| | |
| | If owner given, only list tasks with that owner. |
| | """ |
| | formatstr = "{: >4} {: >10} {: >5} {}" |
| | print(formatstr.format('ID', 'owner', 'done', 'summary')) |
| | print(formatstr.format('--', '-----', '----', '-------')) |
| | with _tasks_db(): |
| | for t in tasks.list_tasks(owner): |
| | done = 'True' if t.done else 'False' |
| | owner = '' if t.owner is None else t.owner |
| | print(formatstr.format( |
| | t.id, owner, done, t.summary)) |
Once you get used to writing Click code, it’s not that bad. I’m not going to explain all of this here, as developing command-line code isn’t the focus of the book; however, even though I’m pretty sure I have this code right, there’s lots of room for human error. That’s why a good set of automated tests to make sure this works correctly is important.
This list_tasks(owner) function depends on a couple of other functions: tasks_db(), which is a context manager, and tasks.list_tasks(owner), which is the API function. We’re going to use mock to put fake functions in place for tasks_db() and tasks.list_tasks(). Then we can call the list_tasks method through the command-line interface and make sure it calls the tasks.list_tasks() function correctly and deals with the return value correctly.
To stub tasks_db(), let’s look at the real implementation:
| | @contextmanager |
| | def _tasks_db(): |
| | config = tasks.config.get_config() |
| | tasks.start_tasks_db(config.db_path, config.db_type) |
| | yield |
| | tasks.stop_tasks_db() |
The tasks_db() function is a context manager that retrieves the configuration from tasks.config.get_config(), another external dependency, and uses the configuration to start a connection with the database. The yield releases control to the with block of list_tasks(), and after everything is done, the database connection is stopped.
For the purpose of just testing the CLI behavior up to the point of calling API functions, we don’t need a connection to an actual database. Therefore, we can replace the context manager with a simple stub:
| | @contextmanager |
| | def stub_tasks_db(): |
| | yield |
Because this is the first time we’ve looked at our test code for test_cli,py, let’s look at this with all of the import statements:
| | from click.testing import CliRunner |
| | from contextlib import contextmanager |
| | import pytest |
| | from tasks.api import Task |
| | import tasks.cli |
| | import tasks.config |
| | |
| | |
| | @contextmanager |
| | def stub_tasks_db(): |
| | yield |
Those imports are for the tests. The only import needed for the stub is from contextlib import contextmanager.
We’ll use mock to replace the real context manager with our stub. Actually, we’ll use mocker, which is a fixture provided by the pytest-mock plugin. Let’s look at an actual test. Here’s a test that calls tasks list:
| | def test_list_no_args(mocker): |
| | mocker.patch.object(tasks.cli, '_tasks_db', new=stub_tasks_db) |
| | mocker.patch.object(tasks.cli.tasks, 'list_tasks', return_value=[]) |
| | runner = CliRunner() |
| | runner.invoke(tasks.cli.tasks_cli, ['list']) |
| | tasks.cli.tasks.list_tasks.assert_called_once_with(None) |
The mocker fixture is provided by pytest-mock as a convenience interface to unittest.mock. The first line, mocker.patch.object(tasks.cli, ’tasks_db’, new=stub_tasks_db), replaces the tasks_db() context manager with our stub that does nothing.
The second line, mocker.patch.object(tasks.cli.tasks, ’list_tasks’, return_value=[]), replaces any calls to tasks.list_tasks() from within tasks.cli to a default MagicMock object with a return value of an empty list. We can use this object later to see if it was called correctly. The MagicMock class is a flexible subclass of unittest.Mock with reasonable default behavior and the ability to specify a return value, which is what we are using in this example. The Mock and MagicMock classes (and others) are used to mimic the interface of other code with introspection methods built in to allow you to ask them how they were called.
The third and fourth lines of test_list_no_args() use the Click CliRunner to do the same thing as calling tasks list on the command line.
The final line uses the mock object to make sure the API call was called correctly. The assert_called_once_with() method is part of unittest.mock.Mock objects, which are all listed in the Python documentation.[24]
Let’s look at an almost identical test function that checks the output:
| | @pytest.fixture() |
| | def no_db(mocker): |
| | mocker.patch.object(tasks.cli, '_tasks_db', new=stub_tasks_db) |
| | |
| | |
| | def test_list_print_empty(no_db, mocker): |
| | mocker.patch.object(tasks.cli.tasks, 'list_tasks', return_value=[]) |
| | runner = CliRunner() |
| | result = runner.invoke(tasks.cli.tasks_cli, ['list']) |
| | expected_output = (" ID owner done summary\n" |
| | " -- ----- ---- -------\n") |
| | assert result.output == expected_output |
This time we put the mock stubbing of tasks_db into a no_db fixture so we can reuse it more easily in future tests. The mocking of tasks.list_tasks() is the same as before. This time, however, we are also checking the output of the command-line action through result.output and asserting equality to expected_output.
This assert could have been put in the first test, test_list_no_args, and we could have eliminated the need for two tests. However, I have less faith in my ability to get CLI code correct than other code, so separating the questions of “Is the API getting called correctly?” and “Is the action printing the right thing?” into two tests seems appropriate.
The rest of the tests for the tasks list functionality don’t add any new concepts, but perhaps looking at several of these makes the code easier to understand:
| | def test_list_print_many_items(no_db, mocker): |
| | many_tasks = ( |
| | Task('write chapter', 'Brian', True, 1), |
| | Task('edit chapter', 'Katie', False, 2), |
| | Task('modify chapter', 'Brian', False, 3), |
| | Task('finalize chapter', 'Katie', False, 4), |
| | ) |
| | mocker.patch.object(tasks.cli.tasks, 'list_tasks', |
| | return_value=many_tasks) |
| | runner = CliRunner() |
| | result = runner.invoke(tasks.cli.tasks_cli, ['list']) |
| | expected_output = (" ID owner done summary\n" |
| | " -- ----- ---- -------\n" |
| | " 1 Brian True write chapter\n" |
| | " 2 Katie False edit chapter\n" |
| | " 3 Brian False modify chapter\n" |
| | " 4 Katie False finalize chapter\n") |
| | assert result.output == expected_output |
| | |
| | |
| | def test_list_dash_o(no_db, mocker): |
| | mocker.patch.object(tasks.cli.tasks, 'list_tasks') |
| | runner = CliRunner() |
| | runner.invoke(tasks.cli.tasks_cli, ['list', '-o', 'brian']) |
| | tasks.cli.tasks.list_tasks.assert_called_once_with('brian') |
| | |
| | |
| | def test_list_dash_dash_owner(no_db, mocker): |
| | mocker.patch.object(tasks.cli.tasks, 'list_tasks') |
| | runner = CliRunner() |
| | runner.invoke(tasks.cli.tasks_cli, ['list', '--owner', 'okken']) |
| | tasks.cli.tasks.list_tasks.assert_called_once_with('okken') |
Let’s make sure they all work:
| | $ cd /path/to/code/ch7/tasks_proj_v2 |
| | $ pytest -v tests/unit/test_cli.py |
| | =================== test session starts =================== |
| | plugins: mock-1.6.2, cov-2.5.1 |
| | collected 5 items |
| | |
| | tests/unit/test_cli.py::test_list_no_args PASSED |
| | tests/unit/test_cli.py::test_list_print_empty PASSED |
| | tests/unit/test_cli.py::test_list_print_many_items PASSED |
| | tests/unit/test_cli.py::test_list_dash_o PASSED |
| | tests/unit/test_cli.py::test_list_dash_dash_owner PASSED |
| | |
| | ================ 5 passed in 0.06 seconds ================= |
Yay! They pass.
This was an extremely fast fly-through of using test doubles and mocks. If you want to use mocks in your testing, I encourage you to read up on unittest.mock in the standard library documentation,[25] and about pytest-mock at http://pypi.python.org.[26]