Beginning Python Programming — Part 12An introduction to iterators and generatorsBob RoeblingBlockedUnblockFollowFollowingJun 10Photo by Timothy Dykes on UnsplashIn our previous piece, we covered packages and virtual environments.
Beginning Python Programming — Part 11Working with modules and virtual environmentsmedium.
comToday, we will be jumping back to cover a few more built-in class methods we haven’t covered yet.
Before we get started, we should cover a few basic algorithms.
These are all very basic, but provide a good baseline for us to get started.
Let’s begin with while loops which we have previously covered:I will admit there are much better ways to do this, but this provides us with a simple example to prepare us for the rest of the article.
Forward looping with while loops is not new to us, however, we haven’t yet covered reverse loops.
While this may be trivial to some of you, I won’t assume all of my readers already understand how looping through lists works.
In a reverse loop, we first need to get the length of the list and set our index variable to length – 1.
We do this because lists are 0-based, meaning the last accessible index will be one less than the total number of items in our list.
Instead of comparing the value of index to the length of the list, as we do in forward loops, we need to ensure that the value of index is no less than 0 during our loop.
We can perform the work needed on our list then subtract 1 from index to get the previous entry in the list.
Once index is less than 0, the while loop exits.
Finally, I created a function that returns the last item in a given list.
It doesn’t matter what list we give it, it will always return the last item.
I added this because it is a concept we will need to be familiar with to continue.
IteratorsSometimes we need to create classes that contain a series of objects that we can loop through while performing some constant action against the data we loop over.
These objects could range from a series of numbers up to a series of classes.
Iterators are classes that provide this functionality.
There are two class methods that you need to become familiar with:__iter__(self) —returns an iterator object__next__(self) — used to return the next item in a sequenceIt is important to understand the difference between __iter__ and __init__.
__init__ is used when you want to create an instance of the class while providing data to the class; this is where we assign data to an iterator.
__iter__ is used when you want to create an instance of the class using a sequence (list, ordered_dict, etc.
) to seed the data the iterator will loop through.
__iter__ returns an object with a __next__ method, if you have a custom __next__ method inside of the class, you only need to return self from the __iter__ method.
Time for an example:Here we have a Doubled class with an __init__, __iter__, and __next__ method.
__init__ asks for a parameter data upon initialization of the class.
In our case, we’ve designed the class to double a list of numbers.
For this, we need to ensure that we pass a list of integers or float values to the class.
The __init__ method then gets the length of the data, so we know when we need to stop iterating over the list we passed in.
It saves the data to a class property named data and sets the current index position to 0.
We set the current index, so we always start at 0 when we create a new iterator.
Because we are defining our own __next__ method in this class, we only need to return self inside of the __iter__ method.
__next__ contains the logic we use to return the next value in the list we passed.
If we were to pass in a dictionary, it would return the value in the dictionary, although we would need to specify how to return the value of the next key.
In our __next__ method, we first check that our current index position is not equal to the length of the data.
If it is, we need to raise the StopIteration exception.
This is different from IndexError, which we receive when we hit the end of a list because we might use this iterator for a different data type.
StopIteration is specific to what error we received, if we need to iterate over the series again, then we either need to create another iterator object or think of a different way to iterate through our data.
Next, we perform the logic for this iterator.
Because we are doubling each item in the list, we save the value of the current index multiplied by 2 into a result variable.
We do this because we need to prepare our index for the next run, which we do on the next line.
Finally, we return the result if everything was successful.
Iterators exist to make our job easier, but also to ensure uniformity in our logic.
However, they aren’t the only way to loop through code.
Photo by Jason Blackeye on UnsplashGeneratorsGenerators are similar to iterators, except they are functions that work like iterators.
The neat thing about generators is that they allow your functions to pause until called again.
In a sense, they have their own __next__ and __iter__ method.
Unlike functions, they return data using the keyword yield.
Let’s look at a straightforward example:For the sake of error handling, I added a try/except to this.
First, we create a function named increment which takes one parameter: number.
In the body of the function, we have two yield statements; the first returns the number that was passed in and adds 1 to it and the second returns the number plus 2.
We then need to store the increment function in the variable incrementor, providing the initial value for the increment function above.
This will all be stored in the variable and treated like a class object.
From here we can use it as we did an iterator, by passing incrementor to the next() function, we receive each yield statement in order.
This means that we would see 5 and 6 printed to the console.
If we call next(incrementor) a third time, as we do above, we receive a StopIteration exception.
What makes generators so interesting is that we use infinite loops with them.
Since yield pauses the execution of the function, we only receive one result at a time.
This means the following code is valid and will not cause problems.
Here we have a new incrementor function which accepts a number.
The =0 means to use 0 as the default value if no number was provided.
We then create a local variable n and assign number to it.
Next, we have an infinite loop, and while these usually are not OK to do, here it is acceptable because yield will pause the execution each time it is called.
At the end of the function we have n = n + 1 which increments our local n variable.
Since the function paused when yield was passed, it resumes on the line that contains n = n + 1 the next time we pass the incrementor to the next() function.
Generators can also be written in a format close to list comprehensions.
The only difference is that we use parentheses around the expression instead of square brackets.
From here we can access each item in generator_object using a simple for loop, just as we would the resulting list in list_comp_object or we could use next(generator_object) to get the next value.
One significant difference between iterators and generators is that iterators calculate all of the values up front and store them in memory, generators only store what is necessary for each run in memory.
Generators run slower than list comprehensions so unless you are facing a scenario where you might run out of memory, you should use a list comprehension.
Hey, it’s easy to refactor if needed.
Generators can be used for recursive tasks such as searching files on a hard drive or digging through web pages in a web scraping utility.
SummaryIterators and generators are more of an intermediate topic but once you use them a few times you start looking for ways to use them everywhere.
Knowing the difference between iterators and generators is essential for the performance of your application; make sure that you know the difference before including them “just because.
”What’s Next?Async is up next, this topic is pretty advanced and provides many options to synchronize your code.
Not only can you configure the timing, but you can also run code at the same time.
Hang in there, until then, keep practicing!.