Bite-Sized Python RecipesA collection of small useful functions in PythonEhsan KhodabandehBlockedUnblockFollowFollowingApr 22Photo by Jordane Mathieu on UnsplashDisclaimer: This is a collection of small useful functions I’ve found around the web, mainly on Stack Overflow or Python’s documentation page.
Some may look trivial and some may not.
But one way or another, I have used them all in my projects and I think they are worth sharing.
You can find all of them, with some additional comments, in this notebook which I try to keep up-to-date.
Unless necessary, I intend not to over-explain the functions.
So, let’s begin!Create a Dictionary From Two Lists:>>> prod_id = [1, 2, 3]>>> prod_name = ['foo', 'bar', 'baz']>>> prod_dict = dict(zip(prod_id, prod_name))>>> prod_dict{1: 'foo', 2: 'bar', 3: 'baz'}Remove Duplicates From a List and Keep the Order:>>> from collections import OrderedDict>>> nums = [1, 2, 4, 3, 0, 4, 1, 2, 5]>>> list(OrderedDict.
fromkeys(nums))[1, 2, 4, 3, 0, 5]# As of Python 3.
6 (for the CPython implementation) and# as of 3.
7 (across all implementations) dictionaries remember# the order of items inserted.
So, a better one is:>>> list(dict.
fromkeys(nums))[1, 2, 4, 3, 0, 5]Create a Multi-Level Nested Dictionary:Create a dictionary as a value in a dictionary.
Essentially, it’s a dictionary that goes multiple levels deep.
from collections import defaultdictdef multi_level_dict(): """ Constructor for creating multi-level nested dictionary.
""" return defaultdict(multi_level_dict)Example 1:>>> d = multi_level_dict()>>> d['a']['a']['y'] = 2>>> d['b']['c']['a'] = 5>>> d['x']['a'] = 6>>> d{'a': {'a': {'y': 2}}, 'b': {'c': {'a': 5}}, 'x': {'a': 6}}Example 2:A list of products is given, where each product needs to be delivered from its origin to its distribution center (DC), and then to its destination.
Given this list, create a dictionary for the list of products that are shipped through each DC, coming from each origin and going to each destination.
import randomrandom.
seed(20)# Just creating arbitrary attributes for each Product instanceclass Product: def __init__(self, id): self.
id = id self.
materials = random.
sample('ABCD', 3) self.
origin = random.
choice(('o1', 'o2')) self.
destination = random.
choice(('d1', 'd2', 'd3')) self.
dc = random.
choice(('dc1', 'dc2')) def __repr__(self): return f'P{str(self.
id)}'products = [Product(i) for i in range(20)]# create the multi-level dictionarydef get_dc_origin_destination_products_dict(products): dc_od_products_dict = multi_level_dict() for p in products: dc_od_products_dict[p.
dc][p.
origin].
setdefault(p.
destination, []).
append(p) return dc_od_products_dictdc_od_orders_dict = get_dc_origin_destination_products_dict(products)>>> dc_od_orders_dict{'dc1': {'o2': {'d3': [P0, P15], 'd1': [P2, P9, P14, P18], 'd2': [P3, P13]}, 'o1': {'d1': [P1, P16], 'd3': [P4, P6, P7, P11], 'd2': [P17, P19]}}, 'dc2': {'o1': {'d1': [P5, P12], 'd3': [P10]}, 'o2': {'d1': [P8]}}}Note, that when you run the above two examples, you should see defaultdict(<function __main__.
multi_level_dict()>.
) in the output.
But they were removed here for legibility of the result.
Return the Keys and Values From the Innermost Layer of a Nested Dict:from collections import abcdef nested_dict_iter(nested): """ Return a generator of the keys and values from the innermost layer of a nested dict.
""" for key, value in nested.
items(): # Check if value is a dictionary if isinstance(value, abc.
Mapping): yield from nested_dict_iter(value) else: yield key, valueA few things should be explained about this function:The nested_dict_iter function returns a generator.
In each loop, the dictionary value is checked recursively until the last layer is reached.
In the condition check, collections.
abc.
Mapping is used instead of dict for generality.
That way container objects such as dict, collections.
defaultdict,collections.
OrderedDict and collections.
Counter are checked.
Why yield from?.Short and incomplete answer: it’s designed for situations where invoking a generator from within a generator is needed.
I know a brief explanation cannot do it any justice, so check this SO thread to learn more about it.
Example 1:>>> d = {'a':{'a':{'y':2}},'b':{'c':{'a':5}},'x':{'a':6}}>>> list(nested_dict_iter(d))[('y', 2), ('a', 5), ('a', 6)]Example 2: let’s retrieve keys and values from our dc_od_orders_dict above.
>>> list(nested_dict_iter(dc_od_orders_dict))[('d3', [P0, P15]), ('d1', [P2, P9, P14, P18]), ('d2', [P3, P13]), ('d1', [P1, P16]), ('d3', [P4, P6, P7, P11]), ('d2', [P17, P19]), ('d1', [P5, P12]), ('d3', [P10]), ('d1', [P8])]The Intersection of Multiple Sets:def get_common_attr(attr, *args): """ intersection requires 'set' objects """ return set.
intersection(*[set(getattr(a, attr)) for a in args])Example: Find the common comprising materials, if any, among our first 5 products.
>>> get_common_attr('materials', *products[:5]){'B'}First Match:Find the first element, if any, from an iterable that matches a condition.
first_match = next(i for i in iterable if check_condition(i))# Example:>>> nums = [1, 2, 4, 0, 5]>>> next(i for i in nums if i > 3)4The above implementation throws a StopIteration exception if no match is found.
We can fix that by returning a default value.
Since we are here, let’s make it a function:def first_match(iterable, check_condition, default_value=None): return next((i for i in iterable if check_condition(i)), default_value)Example:>>> nums = [1, 2, 4, 0, 5]>>> first_match(nums, lambda x: x > 3)4>>> first_match(nums, lambda x: x > 9) # returns nothing>>> first_match(nums, lambda x: x > 9, 'no_match')'no_match'Powerset:The powerset of a set S is the set of all the subsets of S.
import itertools as itdef powerset(iterable): s = list(iterable) return it.
chain.
from_iterable(it.
combinations(s, r) for r in range(len(s) + 1))Example:>>> list(powerset([1,2,3]))[(), (1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)]Timer Decorator:Shows the runtime of each class/method/function.
from time import timefrom functools import wrapsdef timeit(func): """ :param func: Decorated function :return: Execution time for the decorated function """ @wraps(func) def wrapper(*args, **kwargs): start = time() result = func(*args, **kwargs) end = time() print(f'{func.
__name__} executed in {end – start:.
4f} seconds') return result return wrapperExample:import random# An arbitrary function@timeitdef sort_rnd_num(): numbers = [random.
randint(100, 200) for _ in range(100000)] numbers.
sort() return numbers>>> numbers = sort_rnd_num()sort_rnd_num executed in 0.
1880 secondsCalculate the Total Number of Lines in a File:def file_len(file_name, encoding='utf8'): with open(file_name, encoding=encoding) as f: i = -1 for i, line in enumerate(f): pass return i + 1Example: How many lines of codes are there in the python files of your current directory?>>> from pathlib import Path>>> p = Path()>>> path = p.
resolve() # similar to os.
path.
abspath()>>> print(sum(file_len(f) for f in path.
glob('*.
py')))745Just For Fun!.Creating Long Hashtags:>>> s = "#this is how I create very long hashtags">>> "".
join(s.
title().
split())'#ThisIsHowICreateVeryLongHashtags'The following are not bite-sized recipes, but don’t get bitten by these mistakes!Be careful not to mix up mutable and immutable objects!Example: Initialize a dictionary with empty lists as values>>> nums = [1, 2, 3, 4]# Create a dictionary with keys from the list.
# Let's implement the dictionary in two ways>>> d1 = {n: [] for n in nums}>>> d2 = dict.
fromkeys(nums, [])# d1 and d2 may look similar.
But list is mutable.
>>> d1[1].
append(5)>>> d2[1].
append(5)# Let's see if d1 and d2 are similar>>> print(f'd1 = {d1}.d2 = {d2}')d1 = {1: [5], 2: [], 3: [], 4: []} d2 = {1: [5], 2: [5], 3: [5], 4: [5]}Don’t modify a list while iterating over it!Example: Remove all numbers less than 5 from a list.
Wrong Implementation: Remove the elements while iterating!nums = [1, 2, 3, 5, 6, 7, 0, 1]for ind, n in enumerate(nums): if n < 5: del(nums[ind])# expected: nums = [5, 6, 7]>>> nums[2, 5, 6, 7, 1]Correct Implementation:Use list comprehension to create a new list containing only the elements you want:>>> id(nums) # before modification 2090656472968>>> nums = [n for n in nums if n >= 5]>>> nums[5, 6, 7]>>> id(nums) # after modification2090656444296You can see above that id(nums) is checked before and after to show that in fact, the two lists are different.
So, if the list is used in other places and it’s important to mutate the existing list, rather than creating a new one with the same name, assign it to the slice:>>> nums = [1, 2, 3, 5, 6, 7, 0, 1]>>> id(nums) # before modification 2090656472008>>> nums[:] = [n for n in nums if n >= 5]>>> id(nums) # after modification2090656472008That’s it for now.
If you also have some bite-sized functions that you use regularly, let me know.
I’ll try to keep the notebook up-to-date on GitHub and yours can end up there too!I can be reached on Twitter and LinkedIn.
.. More details