It’s slightly significantly shorter than our list implementation to top it off!Generator functions iterators with less boilerplate than classes with a normal logic flow.
Generator functions automagically “pause” execution and return the specified value with every call of next().
This means that no code is run until the first next() call.
This means the flow is like this:next() is called,Code is executed up to the next yield statement.
The value on the right of yield is returned.
Execution is paused.
1–5 repeat for every next() call until the last line of code is hit.
StopIteration is raised.
Generator functions also allow for you to use the yield from keyword which future next() calls to another iterable until said iterable has been exhausted.
def yielded_range(): yield from my_range(10)print(list(yielded_range())) # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]That wasn’t a particularly complex example.
But you can even do it recursively!def my_range_recursive(stop, current = 0): if current >= stop: return yield current yield from my_range_recursive(stop, current + 1)r = my_range_recursive(10)print(list(r)) # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]Generator ExpressionGenerator expressions allow us to create iterators as one-liners and are good when we don’t need to give it external functions.
Unfortunately, we can’t make another my_range using an expression, but we can work on iterables like our last my_range function.
my_doubled_range_10 = (x * 2 for x in my_range(10))print(list(my_doubled_range_10)) # 0, 2, 4, 6, 8, 10, 12, 14, 16, 18]The cool thing about this is that it does the following:The list asks my_doubled_range_10 for its next value.
my_doubled_range_10 asks my_range for its next value.
my_doubled_range_10 returns my_range’s value multiplied by 2.
The list appends the value to itself.
1–5 repeat until my_doubled_range_10 raises StopIteration which happens when my_range does.
The list is returned containing each value returned by my_doubled_range.
We can even do filtering using generator expressions!my_even_range_10 = (x for x in my_range(10) if x % 2 == 0)print(list(my_even_range_10)) # [0, 2, 4, 6, 8]This is very similar to the previous except my_even_range_10 only returns values that match the given condition, so only even values between in the range [0, 10).
Throughout all of this, we only create a list because we told it to.
The BenefitSourceBecause generators are iterators, iterators are iterables, and iterators lazily return values.
This means that using this knowledge we can create objects that will only give us objects when we ask for them and however many we like.
This means we can pass generators into functions that reduce each other.
print(sum(my_range(10))) # 45Calculating the sum in this way avoids creating a list when all we’re doing is adding them together and then discarding.
We can rewrite the very first example to be much better using a generator function!s = 'baacabcaab'p = 'a'def find_char(string, character): for index, str_char in enumerate(string): if str_char == character: yield indexprint(list(find_char(s, p))) # [1, 2, 4, 7, 8]Now immediately there might be no obvious benefit, but let’s go to my first question: “what if we only want the first result; will we need to make an entirely new function?”With a generator function we don’t need to rewrite as much logic.
print(next(find_char(s, p))) # 1Now we could retrieve the first value of the list that our original solution gave, but this way we only get the first match and stop iterating over the list.
The generator will be then discarded and nothing else is created; massively saving memory.
ConclusionIf you’re ever creating a function the accumulates values in a list like this.
def foo(bar): values = [] for x in bar: # some logic values.
append(x) return valuesConsider making it return an iterator with a class, generator function, or generator expression like so:def foo(bar): for x in bar: # some logic yield xResources and SourcesPEPsGeneratorsGenerator Expressions PEPYield From PEPArticles and ThreadsIteratorsIterable vs IteratorGenerator DocumentationIterators vs GeneratorsGenerator Expression vs FunctionRecrusive GeneratorsDefinitionsIterableIteratorGeneratorGenerator IteratorGenerator ExpressionOriginally published at https://blog.
dacio.
dev/2019/05/03/python-iterators-and-generators/.
.. More details