What the mock? — A cheatsheet for mocking in PythonYeray DiazBlockedUnblockFollowFollowingMay 9, 2018It’s just a fact of life, as code grows eventually you will need to start adding mocks to your test suite.
What started as a cute little two class project is now talking to external services and you cannot test it comfortably anymore.
That’s why Python ships with unittest.
mock, a powerful part of the standard library for stubbing dependencies and mocking side effects.
However, unittest.
mock is not particularly intuitive.
I’ve found myself many times wondering why my go-to recipe does not work for a particular case, so I’ve put together this cheatsheet to help myself and others get mocks working quickly.
You can find the code examples in the article’s Github repository.
I’ll be using Python 3.
6, if you’re using 3.
2 or below you’ll need to use the mock PyPI package.
The examples are written using unittest.
TestCase classes for simplicity in executing them without dependencies, but you could write them as functions using pytest almost directly,unittest.
mock will work just fine.
If you are a pytest user though I encourage you to have a look at the excellent pytest-mock library.
The Mock class in a nutshellThe centerpoint of the unittest.
mock module is, of course, the Mock class.
The main characteristic of a Mock object is that it will return another Mock instance when:accessing one of its attributescalling the object itselfThis is the default behaviour, but it can be overridden in different ways.
For example you can assign a value to an attribute in the Mock by:Assign it directly, like you’d do with any Python object.
Use the configure_mock method on an instance.
Or pass keyword arguments to the Mock class on creation.
To override calls to the mock you’ll need to configure its return_value property, also available as a keyword argument in the Mock initializer.
The Mock will always return the same value on all calls, this, again, can also be configured by using the side_effect attribute:if you’d like to return different values on each call you can assign an iterable to side_effect.
If you’d like to raise an exception when calling the Mock you can simply assign the exception object to side_effect.
With all these tools we can now create stubs for essentially any Python object, which will work great for inputs to our system.
But what about outputs?If you’re making a CLI program to download the whole Internet, you probably don’t want to download the whole Internet on each test.
Instead it would be enough to assert that requests.
download_internet (not a real method) was called appropriately.
Mock gives you handy methods to do so.
Note in our example assert_called_once failed, this showcases another key aspect of Mock objects, they record all interactions with them and you can then inspect these interactions.
For example you can use call_count to retrieve the number of calls to the Mock, and use call_args or call_args_list to inspect the arguments to the last or all calls respectively.
If this is inconvenient at any point you can use the reset_mock method to clear the recorded interactions, note the configuration will not be reset, just the interactions.
Finally, let me introduce MagicMock, a subclass of Mock that implements default magic or dunder methods.
This makes MagicMock ideal to mock class behaviour, which is why it’s the default class when patching.
We are now ready to start mocking and isolating the unit under tests.
Here are a few recipes to keep in mind:Patch on importThe main way to use unittest.
mock is to patch imports in the module under test using the patch function.
patch will intercept import statements identified by a string (more on that later), and return a Mock instance you can preconfigure using the techniques we discussed above.
Imagine we want to test this very simple function:Note we’re importing os and calling getcwd to get the current working directory.
We don’t want to actually call it on our tests though since it’s not important to our code and the return value might differ between environments it run on.
As mentioned above we need to supply patch with a string representing our specific import.
We do not want to supply simply os.
getcwd since that would patch it for all modules, instead we want to supply the module under test’s import of os , i.
e.
work.
os .
When the module is imported patch will work its magic and return a Mock instead.
Alternatively, we can use the decorator version of patch , note this time the test has an extra parameter: mocked_os to which the Mock is injected into the test.
patch will forward keyword arguments to the Mock class, so to configure a return_value we simply add it as one:Mocking classesIt’s quite common to patch classes completely or partially.
Imagine the following simple module:Now there’s two things we need to test:Worker calls Helper with "db"Worker returns the expected path supplied by HelperIn order to test Worker in complete isolation we need to patch the whole Helper class:Note the double return_value in the example, simply using MockHelper.
get_path.
return_value would not work since in the code we call get_path on an instance, not the class itself.
The chaining syntax is slightly confusing but remember MagicMock returns another MagicMock on calls __init__.
Here we’re configuring any fake Helper instances created by MockHelper to return what we expect on calls to get_path which is the only method we care about in our test.
Class speccingA consequence of the flexibility of Mock is that once we’ve mocked a class Python will not raise AttributeError as it simply will return new instances of MagicMock for basically everything.
This is usually a good thing but can lead to some confusing behaviour and potentially bugs.
For instance writing the following test,will silently pass with no warning completely missing the typo in assrt_called_once .
Additionally, if we were to rename Helper.
get_path to Helper.
get_folder, but forget to update the call in Worker our tests will still pass:Luckily Mock comes with a tool to prevent these errors, speccing.
Put simply, it preconfigures mocks to only respond to methods that actually exist in the spec class.
There are several ways to define specs, but the easiest is to simply pass autospec=True to the patch call, which will configure the Mock to behave as the object being mocked, raising exceptions for missing attributes and incorrect signatures as required.
For example:Partial class mockingIf you’re less inclined to testing in complete isolation you can also partially patch a class using patch.
object:Here patch.
object is allowing us to configure a mocked version of get_path only, leaving the rest of the behaviour untouched.
Of course this means the test is no longer what you would strictly consider a unit test but you may be ok with that.
Mocking built-in functions and environment variablesIn the previous examples we neglected to test one particular aspect of our simple class, the print call.
If in the context of your application this is important, like a CLI command for instance, we need to make assertions against this call.
print is, of course a built-in function in Python, which means we do not need to import it in our module, which goes against what we discussed above about patching on import.
Still imagine we had this slightly more complicated version of our function:We can write a test like the following:Note a few things:we can mock print with no problem and asserting there was a call by following the “patch on import” rule.
This however was a change introduced in 3.
5, previously you needed to add create=True to the patch call to signal unittest.
mockto create a Mock even though no import matches the identifier.
we’re using patch.
dict to inject a temporary environment variable in os.
environ, this is extensible to any other dictionary we’d like to mock.
we’re nesting several patch context manager calls but only using as in the first one since it’s the one we need to use to call assert_called_once_with .
If you’re not fond of nesting context managers you can also write the patch calls in the decorator form:Note however the order of the arguments to the test matches the stacking order of the decorators, and also that patch.
dict does not inject an argument.
Mocking context managersContext managers are incredibly common and useful in Python, but their actual mechanics make them slightly awkward to mock, imagine this very common scenario:You could, of course, add a actual fixture file, but in real world cases it might not be an option, instead we can mock the context manager’s output to be a StringIO object:There’s nothing special here except the magic __enter__ method, we just have to remember the underlying mechanics of context managers and some clever use of our trusty MagicMock .
Mocking class attributesThere are many ways in which to achieve this but some are more fool-proof that others.
Say you’ve written the following code:You could test the code without any mocks in two ways:If the code under test accesses the attribute via self.
ATTRIBUTE, which is the case in this example, you can simply set the attribute directly in the instance.
This is fairly safe as the change is limited to this single instance.
Alternatively you can also set the attribute in the imported class in the test before creating an instance.
This however changes the class attribute you’ve imported in your test which could affect the following tests, so you’d need to remember to reset it.
The main drawback from this non-Mock approaches is that if you at any point rename the attribute your tests will fail and the error will not directly point out this naming mismatch.
To solve that we can make use of patch.
object on the imported class which will complain if the class does not have the specified attribute.
Here are the some tests using each technique:Mocking class helpersThe following example is the root of many problems regarding monkeypatching using Mock.
It usually shows up on more mature codebases that start making use of frameworks and helpers at class definition.
For example, imagine this hypothetical Field class helper:Its purpose is to wrap and enhance an attribute in another class, a fairly pattern you commonly might see in ORMs or form libraries.
Don’t concern yourself too much with the details of it just note that there is a type and default value passed in.
Now take this other sample of production code:This class makes use of the Field class by defining its country attribute as one whose type is str and a default as the first element of the COUNTRIES constant.
The logic under test is that depending on the country a discount is applied.
For which we might write the following test:But that would NOT pass.
Let’s walk through the test:First it patches the default countries in pricer to be a list with a single entry GB ,This should make the CountryPricer.
country attribute default to that entry since it’s definition includes default=COUNTRIES[0] ,It then instanciates the CountryPricer and asks for the discounted price for GB !So what’s going on?The root cause of this lies in Python’s behaviour during import, best described in Luciano Ramalho’s excellent Fluent Python on chapter 21:For classes, the story is different: at import time, the interpreter executes the body of every class, even the body of classes nested in other classes.
Execution of a class body means that the attributes and methods of the class are defined, and then the class object itself is built.
Applying this to our example:the country attribute’s Field instance is built before the test is ran at import time,Python reads the body of the class, passing the COUNTRIES[0] that is defined at that point to the Field instance,Our test code runs but it’s too late for us to patch COUNTRIES and get a proper assertion.
From the above description you might try delaying the importing of the module until inside the tests, something like:This, however, will still not pass as mock.
patch will import the module and then monkeypatch it, resulting in the same situation as before.
To work around this we need to embrace the state of the CountryPricer class at the time of the test and patch default on the already initialized Field instance:This solution is not ideal since it requires knowledge of the internals of Field which you may not have or want to use if you’re using an external library but it works on this admittedly simplified case.
This import time issue is one of the main problems I’ve encountered while using unittest.
mock.
You need to remember while using it that at import time code at the top-level of modules is executed, including class bodies.
If the logic you’re testing is dependent on any of this logic you may need to rethink how you are using patch accordingly.
Wrapping upThis introduction and cheatsheet should get you mocking with unittest.
mock, but I encourage you to read the documentation thoroughly, there’s plenty of more interesting tricks to learn.
It’s worth mentioning that there are alternatives to unittest.
mock, in particular Alex Gaynor’s pretend library in combination with pytest’s monkeypatch fixture.
As Alex points out, using these two libraries you can take a different approach to make mocking stricter yet more predictable.
Definitely an approach worth exploring but outside the scope of this article.
unittest.
mock is currently the standard for mocking in Python and you’ll find it in virtually every codebase.
Having a solid understanding of it will help you write better tests quicker.
????.Thanks for reading!.If you’ve enjoyed this article you may want to have a look at my series on Python’s new built-in concurrency library, AsyncIO:AsyncIO for the Working Python Developer,AsyncIO Coroutine Patterns: Beyond awaitand AsyncIO Coroutine Patterns: Errors and Cancellation.
Enjoy!.????.. More details