Transcript:
Hello everyone, how are you? In this Python video we are going to see the generators, what you could use them for and some advantages they have over the lists. In this example I have this function here called “square_numbers” (square numbers) and what it does is it takes a list of numbers and then we have this variable “result” to which an empty list is assigned and then we go through all the numbers on the list that we pass and we add the square of that number to the “result” list and once we have gone through all the numbers, we return the result. And you can see that here I have the variable “my nums” equal to “square_numbers” and passing it a list 1, 2, 3, 4, 5 … and then I print “my_nums” down here. If I run this code, You will see that the list 1, 2, 3, 4, 5 that we passed to you, our result is 1, 4, 9, 16, 25. Currently our “square_numbers” function is returning a list, How would we turn it into a generator? To do this, we will not need this variable result so we can remove it, we do not need the “return” statement and this “result.append” we can remove this and all we have to do is type this keyword “yield” and it will just produce this square number from here. This “yield” keyword is what makes it a generator. If I save and run this, you can see now that when I print “my_nums” I no longer get the list, If you look at this comment, it is what the result was before, I no longer get 1, 4, 9, 16, 25, squares 1 to 5. I no longer get that result, now I get this generator object here. The reason for this is that generators do not store the entire result in memory. Produce the results one by one. You are really waiting for us to ask you for the next result. You haven’t calculated anything yet. If I printed “next (my_nums)” which asks for the following result, you can see that it is 1 because we passed him our list of 1, 2, 3, 4, 5 and we are running through that list and 1 is the first value so the “i” in here equals 1 and then 1 * 1 was generated and gave us that result. If we now copy this line here, and we print “next (my_nums)” a few times, and we run this, you can see that every time I execute “next” gets the next produced value so now we have 1 squared, 2 squared which is 4, 3 squared which is 9, 16, 25 and so on. 25 is the last value of our result, What if I ran “next” one more time? if i do, you can see i got an error, and the exception it gave us is “StopIteration” and that means that the generator has been used up. “StopIteration” means it has no more values. Now instead of getting these values one at a time, we can still use a “for” loop in these generators, and this is how I use generators mostly. Let me comment on this line and uncomment this and save it. Now we are saying, for each “num” in “my_nums”, where “my_nums” is my generator, print “num”. I run this. You can see that we get all the values and the exception “StopIteration” does not appear because the “for” loop knows when to stop before that happens. An immediate advantage over the lists, is that I think this is much more readable. Instead of having an empty list assigned to the result, and then attach to the result and then return it. This is somewhat more readable. We are saying, okay I’m passing these numbers to you, for every number in this list of numbers produces the result. For those of you who are more familiar with Python, maybe you have noticed that this whole process, these lines of code, it would have been easier to write them as a list for comprehension. Let me comment on this, and if you don’t know what a list is by comprehension, don’t worry too much, I just want to teach the generator example like this too. This is a list for understanding, and it’s going to do exactly the same thing our “square_numbers” function did. What we are doing is creating a list and we are doing “x” times “x”, that is, the square of “x”, for each “x” in this list 1, 2, 3, 4, 5. If I save this and run the code. You can see that I still get the same results and I can also … print this list up here. You can create a generator in the same way, and it’s as easy as removing these brackets and put some parentheses. So if I remove the brackets and put parentheses. If I run this, you can see that when I printed “my_nums” here, I got this generator object, and when you ran the “for” loop, went through all the values and it gave me that result. What if you wanted to print all the values of the generator? Well, they are not all stored in memory, but you can turn it into a list, it’s as easy as putting “list” and wrap it. If I run this, you can see that he printed it as if it were a list. When you convert this generator to a list, you lose the advantages you had in terms of performance. I haven’t talked about performance yet, but I have an example to better show those advantages. A generator has better performance because it is not storing all the values in memory. Which is not very relevant when you have a small list like this, of 1, 2, 3, 4, 5, But let’s say you have tens of thousands or even millions of items to go through. Having all those elements in memory is sure to be noticed and that does not happen with generators. So when you transform a generator into a list, if this generator had many values, and we had to turn it into a list, then you lose that performance, because it would store all those values in memory. Let me show you a better example of this performance difference. I have a file here where … here are some things you shouldn’t worry about, in these lines here I simply print the memory. And these “names” and “majors”, they will only be used to create random values. I have two functions here, one of them will create a list and the other will be a generator. and they both return the same values. In this list I have my result here, and i’m looping through a number of people that I am going to happen to this function, and for each person I will make a dictionary giving it an “id”, a random name from the list above and a random race from the race list. And I will return the result. And for the generator it is the same. I’m going to loop through the number of people that I pass, and I will produce this dictionary of person which has the same values as the function with the list. To make them the same, I’m going to put “xrange” here so that they are exactly the same. Well, here … don’t worry about these lines here, this time.clock and this t2 all I do is check how long it takes to run this function that returns a list. I’m going to pass a million values to this function, so it should return a list with a million results. and here below I print the memory used and the time it took. If I run this. You can see here at the top of the code, this “before” here, This is before I did anything my base memory usage was around 15 Mb. And this “after” memory, it is after creating this list with a million records. You can see that it increased about 300 Mb and it takes 1.2 seconds. If you are working with a large amount of data … it is not strange to have a million records. Let’s see what happens if I use the generator example, I’m going to comment on the function that returned the list and uncomment the one returned by the generator and I’m going to pass him the same amount of values, I’m going to pass him a million values. If I save this and run it. You can see that here the memory is almost the same, because really the generator hasn’t done anything yet, It is not storing that million values in memory. It is waiting for me to take the next one or looping through them and he will give them to me one at a time. And the time it took … basically it took nothing, because as soon as it reaches the “yield” statement for. If this were put as a whole number, it would be practically zero seconds. When I said before that if you turn it into a list you lose performance, let me show you what I mean. I will convert the result to a list, and if i run this now … I got the same result as when I ran the function that returned the list. If I remove this again, and I do the generator, you see that we have that performance again. This is how you use a generator, I think it is a little more readable, and it also gives us a huge performance boost. Not only in runtime, but also in memory. And you can also use them with understandings, this expression from a generator here, you lose nothing in that regard. These are some reasons why you would use generators and some of the advantages that come with it. Hope this video was helpful, if you have any questions about this topic, ask below in the comment section. Subscribe for future Python videos and thanks for watching.