Python tipsEldiiar EgemberdievBlockedUnblockFollowFollowingJun 16“I do not like to repeat successes, I like to go on to other things”.
Walt DisneyAt the beginning of my way as a Python developer, I found myself searching for new bits of knowledge of what Python is capable of as a programming language.
Digging too deep into details made me understand how things work “under the hood”.
The more I learned, the more I had to google things a knew.
It takes a while, so I started to make notes on the most interesting things.
After a while, since I started writing notes and scratches, I had hundreds of them.
In this article, I share the most commonly used.
Zen of PythonThe very first thing to know before you can go any further in learning Python is the way to reach the Zen of Python.
It is a collection of 19 software principles that influence the design of Python Programming Language and is included as an Easter egg in the Python interpreter, which can be displayed by entering import this.
>>> import thisThe Zen of Python, by Tim PetersBeautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
The list comprehension.
This is an elegant way of initializing and generating lists.
The official documentation describes the full set of possibilities.
Example:>>> squares = [x**2 for x in range(10)]>>> squares[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]>>> [(x, y) for x in [1,2,3] for y in [3,1,4] if x != y][(1, 3), (1, 4), (2, 3), (2, 1), (2, 4), (3, 1), (3, 4)]Merging dictionariesPython 3.
5+ provides an easy way of how to merge dictionaries.
The official documentation about dictionaries.
1.
Using {**kwargs}>>> x = {‘a’: 1, ‘b’: 2}>>> y = {‘b’: 3, ‘c’: 4}>>> z = {**x, **y}>>> z{‘a’: 1, ‘b’: 3, ‘c’: 4}2.
Calling update.
Example:>>> dict1 = {'a': 1, 'b': 2}>>> dict2 = {'d': 4, 'c': 4}>>> dict2.
update(dict1){'d': 3, 'c': 4, 'a': 1, 'b': 2}Sorting a dictionary by its valuesThe first way is to write your own way to get a value from a dictionary>>> xs = {'a': 4, 'b': 3, 'c': 2, 'd': 1}>>> sorted(xs.
items(), key=lambda x: x[1])[('d', 1), ('c', 2), ('b', 3), ('a', 4)]Another way is to use operator.
itemgetter>>> xs = {'a': 4, 'b': 3, 'c': 2, 'd': 1}>>> import operator>>> sorted(xs.
items(), key=operator.
itemgetter(1))[('d', 1), ('c', 2), ('b', 3), ('a', 4)]Elegant Switch/Case in PythonThe first way to do this is a simple broadly known if/else statementdef dispatch_if(operator, x, y): if operator == 'add': return x + y elif operator == 'sub': return x – y elif operator == 'mul': return x * y elif operator == 'div': return x / y else: return NoneAnother way to do the same is to use a dictionarydef dispatch_dict(operator, x, y): return { 'add': lambda: x + y, 'sub': lambda: x – y, 'mul': lambda: x * y, 'div': lambda: x / y, }.
get(operator, lambda: None)()Named tuplesDefault values for NamedtuplesInstantiating Namedtuples classesDunder/Magic methodsThis section will begin with samples.
Here I am going to dive straight in.
You may find some methods you already met.
dct.
__getitem__(key)dct.
__setitem__(key, value)dct.
__delitem__(key)ins.
__getattr__(attr)ins.
__setattr__(attr, value)seq.
__next__()As you may have noticed “__” (two underscores) is also known as “dunder”.
These methods are called magical because you don’t need to call them explicitly.
They are called via built-in operators or functions.
The samples of operators follow methods each is calling.
dct.
__getitem__(key)>>> x[key]dct.
__setitem__(key, value)>>> x[key]= valuedct.
__delitem__(key)>>> del x[key]seq.
__next__()>>> next(seq)ins.
__getattr__(attr: str)>>> ins.
attr # here the instance attribute is retrieved.
The 'attr' after a dot (".
")is not a variableins.
__setattr__(attr, value)>>> ins.
attr = valueBy default, those methods are predefined in most of the cases.
However, Python lets you override built-ins so you can write the code that fits your needs better.
For a good example of how those dunder methods are used, you can visit agithub.
IterablesNow you know what the dunder methods are.
One of the use cases for them is to make a class iterable.
This kind of class allows you to iterate over its values.
Imagine now how you would iterate over a “list items” just like:for item in listLet’s write our own iterable class with our own logic of iteration over it.
The class name would be IterableExercise.
It has a list of exercises.
We define __iter__ method.
This iterates over the exercises and yields the tuple of lists of muscles you train with a body top.
class IterableExercise: exercises = [ {'top': True, 'core': False, 'arms': ['biceps', 'triceps'], 'back': [], 'legs': []}, {'top': True, 'core': True, 'arms': [], 'back': ['trapeze'], 'legs': []}, {'top': False, 'core': True, 'arms': [], 'back': ['loin'], 'legs': ['quadriceps']}, ] def __iter__(self): for exercise in self.
exercises: if exercise['top']: yield exercise['arms'], exercise['back'], exercise['legs']# Here is how you use it>>>for arms, back, legs in IterableExercise(): arms_text = ' '.
join(str(e) for e in arms) back_text = ' '.
join(str(e) for e in back) legs_text = ' '.
join(str(e) for e in legs) print('Top body is working when you train %s %s %s' % (arms_text, back_text, legs_text))Top body is working when you train biceps tricepsTop body is working when you train trapezeStatic typingBeginning from Python 3.
5 you can use a static typing.
It is useful when you need to support a big project and it comes hard to keep the type of the arguments passed to the methods in mind.
Just remember that there is some kind of type helper out there.
You can use it like this:def my_add(a: int, b: int) -> int: return a + bDive into documentation for a better understanding of how it works.
RegistryThis section describes the use of the Registry Pattern (RP) in a sample of Extract Transform Load (ETL) process.
Most of the developers that worked with data processing and data manipulating may have faced the issue of custom modifications that are based on natural rules.
RP is aimed to help make such modifications easier.
The need for adding new modules will remain still, but yet, things do not have to be hard on every part of a project lifetime.
The RP provides global accessibility of registered services.
Simple inheritance in Python allows building a Registry class.
It can be a Metaclass or even a single base class beginning from Python 3.
6.
Here, the examples of how Registry class can be built are provided.
The way the RP is implemented in case of run-time registration allows new classes to be added to the project with no need to explicitly register them in multiple places.
That makes a codebase easier to extend for future modifications.
ProjectFor the sake of simplicity, the Django skeleton is used as a template for project structure.
The following command generates the project structure you will further need to modify according to your needs.
$ django-admin startproject — template=https://github.
com/Mischback/django-project-skeleton/archive/development.
zip [projectname]The project structure with our ETL module looks like this:[projectname]/ <- project root├── [projectname]/ <- Django root│ ├── __init__.
py│ ├── settings/…│ ├── urls.
py├── apps/│ └── __init__.
py│ └── app_one/…├── etl/│ └── __init__.
py│ └── extractors.
py│ └── transformers.
py│ └── loaders.
py│ └── services/│ │ └── __init__.
py│ │ └── service_one.
py│ │ └── service_two.
py│ │ └── service_three.
py├── manage.
py├── README.
md├── …The etl module has a standard set of python files each responsible for data processing.
Registry classMetaclass as RegistryRegistry of distributed classes in different filesRegistry holderThe following code registers all its subclasses in the registry class property.
It ignores base_classes while saving and warns if any subclass does not have an identifier_property.
import warningsclass RegistryHolder(type): registry = {} base_classes = ['BaseServiceOne', 'BaseServiceTwo', 'BaseServiceThree'] @classmethod def register_class(mcs, target_class): try: identifier_property = getattr(target_class, 'identifier_property') mcs.
_save(identifier_property, target_class) except AttributeError: mcs.
_warn(target_class.
__name__) def __new__(mcs, name, bases, class_dict): cls = type.
__new__(mcs, name, bases, class_dict) mcs.
register_class(cls) return cls @classmethod def _save(mcs, identifier_property, target_class): registered_class = mcs.
registry.
get(identifier_property) if registered_class is None: mcs.
registry[identifier_property] = registered_class else: warning_message = 'Error occurred while registering {class_name}.
identifier_property {identifier_property} is already registered for class {registered_class_name}' # noqa: E501 warnings.
warn(warning_message.
format( identifier_property=identifier_property, registered_class_name=registered_class.
__name__, class_name=target_class.
__name__), RuntimeWarning) @classmethod def _warn(mcs, class_name): if class_name not in mcs.
base_classes: warnings.
warn('{class_name} does not have attribute identifier_property'.
format(class_name=class_name), RuntimeWarning)For as Python is a scripting language, only classes that are imported during the code execution will appear in the registry.
This problem can be solved by auto-importing all the submodules in the parent package.
Do it by adding the following code in init file in parent package:def import_submodules(package, recursive=True): """ Import all submodules of a module, recursively, including subpackages :param recursive: a flag to import submodules recursively :param package: package (name or actual module) :type package: str | module :rtype: dict[str, types.
ModuleType] """ if isinstance(package, str): package = importlib.
import_module(package) results = {} for loader, name, is_pkg in pkgutil.
walk_packages(package.
__path__): full_name = package.
__name__ + '.
' + name results[full_name] = importlib.
import_module(full_name) if recursive and is_pkg: results.
update(import_submodules(full_name)) return resultsimport_submodules(__name__)ConclusionThere is still a lot to learn out there, but now you have a starting point of a direction to dig in.
Pick up a topic you may find useful and interesting, dive into the documentation and go into details.
Try your best to understand how things actually work.
I bet the result will bring you fulfillment and expand the knowledge that you already have!.Even if you start with something small, you are heading in the right direction.
.