Don’t be afraid of Python’s type hintsZethBlockedUnblockFollowFollowingMay 19Python is a high level dynamic language where you can quickly write code and immediately get results.
One of the exciting new features of Python in recent years is type annotations, also called ‘type hints’ and often informally just called ‘typing’ (after the module that contains much of the helper types).
If your initial reaction is that the idea of documenting types will stifle your creativity, don’t worry, it won’t.
The approach that Python has taken is really easy going and approachable.
As we will see in this article, type hints are for documenting what you are doing, not prescribing any particular approach for how you should or shouldn’t write your code.
For example, in this article, we will see how you can add type hints to code that uses formal inheritance or code that uses an interfaces approach, it is all supported.
If you can write it, you can document it using type hints.
The type hints syntax has been pretty stable for a while now and in my opinion you should be using it, or at least have it on your radar to implement it when you can.
However, the area is still under highly active evolution, with new goodies and helpers coming with each Python release.
This article aims to be a friendly introduction to type annotations.
I am not going to attempt a comprehensive guide, other people have done that.
Instead I will walk through a simple example of a stub class and incrementally add type hints to it.
Let’s start at the beginning.
Old style docstring approachAs a freelancer/contractor working with Python for the last 13 years, I have found that several companies that I have worked with have had formal internal guidelines for docstrings that required you to define inputs and outputs of each method and function; often to be picked up by a documentation generator such as Sphinx, Doxygen or whatever internal tool that they had invented.
Let’s make an example to show what I mean.
Let’s have some stub classes Boiler, Dispenser and Drink:Everyone in England starts their career by making tea in some capacity, either directly for a customer or for a boss or for colleagues.
So let’s imagine a robot that makes tea:Having the arguments and return values is pretty basic documentation but in complicated real world situations, it might be the difference between understanding what the class is meant to be doing in a few seconds versus having to pull it apart and study it line by line.
If you understand what is going on above, then you have the technical knowledge required to understand everything else I will talk about, it doesn’t get harder, it just gets easier.
Type HintsType hints help document to yourself and other people what you want your code to do.
It takes no more work than the traditional docstring approach outlined above but has the advantage of being a standardised approach with far better tooling, with linters such as mypy and pytype, as well as growing editor/IDE support.
Let’s look at our TeaMaker class again, this time with type hints:This example still contains what arguments the method needs and what the method returns, but in a far more compact way.
A talented JavaScript developer in one of my clients always refers to docstrings as “lies” because they can date very easily then become confusing or misleading.
The advantage of type hints is that linters can make sure the type hints are up to date, especially if the linter is plumbed into CI pipelines or git hooks.
Another beneficial side effect is that the linter might find bugs that you might otherwise have missed, if you are using the wrong type of argument for a function or method, the linter will tell you.
Many hard to spot runtime bugs are of this kind, especially if you have a larger team of people changing things all over the place.
You might change a function then forget to change all of the uses of this function.
Python might not raise an exception, it may just carry on happily doing what you told it to, destroying your data or doing something else that it shouldn’t.
How to install type lintersYou can install mypy with:pip install mypyTo run the linter on the code above, download the gist and then:mypy teamaker2.
pyThis will check the type hints but will leave un-annotated code alone.
If you want to go all in and lint all the code, you can use strict mode:mypy –strict teamaker2.
pyThe best thing about open source is having choice, mypy already has an alternative called pytype.
You can also install pytype with:pip install pytypeMypy is from people in the PSF/Core developer community, pytype is from Google people.
Try both, it is all free.
Let’s push on.
Tangent: Data ClassesThis is a slight tangent from our main topic but dataclasses build on type annotations so it is worth looking at them.
Since the attributes and arguments of our TeaMaker class are the same, we can lose more boilerplate in Python 3.
7 by making it a dataclass.
A dataclass is defined using the dataclass decorator.
Instead of putting our class arguments in the __init__() method, we define each field of the dataclass using type annotations.
Python then automatically generates an __init__() method that stores the arguments as attributes.
What if you need your class to do more work post initialisation?.Simple, define a __post_init__() method.
To learn more about dataclasses, read the dataclasses documentation.
Nominal SubtypingOur TeaMaker class above is expecting an instance of Dispenser.
Now imagine we have some subclasses of Dispenser:If you give instances of Caddy or SpacePouch to the TeaMaker class instead of instances an instance of Dispenser, a type hints linter (mypy or pytype) will be absolutely fine with it because they are a subclass of the type that it was expecting.
This is a tidy class hierarchy like you will find in a first year Java programming textbook, we can call this “nominal subtyping”.
The real world is messier.
You cannot necessarily put all the classes you use in a nice little tree.
You may not have full control, for example, you may be using classes from other libraries that you installed from Pypi, you might be using classes from another team and they don’t want to produce a new version right now, you might be using mock objects inside your testing framework, and so on.
Sometimes all you have are interfaces that are similar or can be wrapped to become similar-enough.
Let’s move on to see how we deal with that.
Using UnionIf you remember, our TeaMaker class required a Boiler instance.
Let’s imagine we don’t have of of them but we have an instance of a functionally equivalent class.
Here are some new classes:They are not subclasses of Boiler but they have a heat_water method so our TeaMaker will not care which one you give it.
This is Python after all, we like Duck Typing.
However, our type linter will throw an error.
The quickest way to get the linter to shut up and leave you alone is to just set the type of boiler to Any.
Any is nice syntactic sugar for when you do not want to be bothered right now about the type or when you have a method which really can take any type at all, e.
g.
some kind of print method perhaps.
While this will work but in this case, it slightly makes a mockery of the whole process.
Let’s find a better way.
Slightly better is that if we know the classes that we want to support, we can use Union to list the valid types of boiler.
This is slightly more useful than the Any example, however, if later we have access to a new class that we want to use, then we have to add it to the list.
There is a better way which we will cover next.
Static Subtyping with ProtocolNew helper types appear first in typing_extensions module while waiting for a new Python release.
The Protocol type that I am about to use will (probably) appear in Python 3.
8, until then you can do this:pip install typing-extensionsNow to have a complete working example that you can try out, I am going to repeat all the existing bits.
Skip down to line 53 below to see the WaterHeater class, then underneath the code, I will explain what is going on.
The WaterHeater class is a Protocol class, it documents what the acceptable instance has to look like, i.
e.
it has to have a heat_water() method.
You can see that SaucePan and Microwave each now have an extra method, but the linter doesn’t mind about that.
As long as it has the heat_water() method, the protocol is met.
This allows an interfaces/duck typing approach to type hints.
Download the gist and run mypy on it.
Fiddle around with the classes and see what happens.
For more on this topic, read the guide to protocols and structural subtyping at the mypy homepage.
That’s enough of thatHopefully, that was an interesting and different look at type hints.
I didn’t bother to start with Python’s built in types (strings, ints, bools, lists, dicts, etc) but most of the time that is what you are using as arguments and return values, and annotating those is really simple.
When you have a complicated case, at that point you can dip into the documentation and find a relevant helper type.
Type hints are there to help you, not be another form of stress, if you don’t want to annotate a particular right now, then just don’t.
Also, for really embedded data types, e.
g.
where you have a list of dicts each containing more things, you can really go to town and up with with a type hint that looks as long as a Django model, I am not entirely sure it is always worth bothering going so deep into such objects, especially if they are not your fault!Type Hints are already supported in most editors to some degree to aid in code completion.
At least in Emacs and Vim via Jedi, PyCharm and VS Code.
Check your own editor’s documentation to find out more.
Further ReadingThe Mypy Documentation is full of examples, this is probably the most useful link.
The Typing Module documentation is full of helper types but don’t be overwhelmed with that page, as you don’t really need to understand or remember everything (or anything) in there in order to get started.
The Real Python guide to type checking has lots of information, I admit that I haven’t read it all yet.
.