Python Run Multiple Functions At Once | Python Tutorial – How To Use Multiprocessing To Run Multiple Functions At The Same Time

Johan Godinho

Subscribe Here





Python Tutorial - How To Use Multiprocessing To Run Multiple Functions At The Same Time


[MUSIC] right, so hey, guys, and welcome back to another python tutorial. So today’s tutorial is a lot different to what we’ve been doing recently because in today’s tutorial we’re going to be talking about synchronous and asynchronous functions and how they work or the execution types and we’re also going to learn how to use multi-processing to pretty much run two functions at the same time, so synchronous basically just means that a program runs in the order of the instructions, So as usually python does. If you give it a couple of instructions, it runs the instructions in a row, so let me open up Visual studio really quickly to explain what I mean. So if I give Python a few instructions such as print Hello, and then if I do print, um, hello again and then if I run this quickly. This, um, execution of code right here would be an asynchron. Uh, would be a synchronous type of, um, execution because, basically, what happens is it runs the code line by line now? An asynchronous type of executions. What you can do is run functions or run specific code at any stage of where you want it to be run so technically, it’s it’s a lot harder to explain, but if I put it in a simple way, it just means that you can run a function or multiple functions at the same time or at specified times, instead of just following the flow of instructions from line 1 to whatever line it ends on cool. So since we kind of understand how that works what we’re going to be doing now is understanding how we’re going to be using multiprocessing to actually understand how to run two functions at the same time. So what I’m going to need to do is import time first, because we’re going to be using time to actually figure out how long the functions take to execute. Now when we run the functions in a synchronous method, it should take way longer than when we run it in an asynchronous method. Because when we run it in asynchronous, it pretty much runs at the same time, whereas in synchronous, it just runs one by one, so that’s how we’re going to be proving our point and we’re going to be needing multi processing, which is a standard library that’s installed on pretty much every python 3 version, so you don’t need to install anything cool. So what you want to do? First of all is create a function called. I’m just going to call it do something or let’s just call it function. I’m going to call this sleep for a bit a weird function name, but, hey, that’s what the function is going to be doing. It’s going to be sleeping for a bit, and then we need to. Um, give it a parameter of how many seconds we wanted to sleep, so I wanted to sleep for so and so amount of time. That’s the amount of time I’m going to enter as a parameter when I run this function, so what I’m going to do is I’m going to type in print and then pass in an f-string and then say sleeping, and then I’m going to use my second’s variable, which is going to be passed to me later on when this function is one, and then I’m going to type sleeping for so and so seconds and then second bracket S cool, and then once it’s done sleeping, we’re actually going to use the time Dot sleep to make it sleep so and then we’re going to make it sleep for the amount of seconds that’s specified by the user and Lastly, we’re going to say done sleeping cool so literally. All this function does. Is it prints out how many seconds the user wants it to sleep? It sleeps that amount of seconds and then when it’s done sleeping, it will say done sleeping cool, so in a normal scenario, if you just run if you wanted to run the this function at the same time, you can’t really do that without multi-processing or threading or any other module that does dissimilar pretty much has a similar functionality. So if I try to run these two at the same time? This is going to be my approach so technically, it’s going to run it once one at a time, I can prove that by typing and obviously I’m going to sleep for one second. Let’s leave for one second, and I’m going to run this two times, so this is going to run the synchronous method because it’s going to go line by line. It’s going to go from line 1 to pretty much line ten, which involves running this first and then this later. So if I run this now? This is a synchronous execution of it, so it says, don’t. I’m sleeping one second, then done sleeping. Then it’s sleeping one second and then done sleeping again, so it’s pretty much done a line by line execution of code, which is asynchronous type. So what we’re going to be doing is learning how to make this an asynchronous or just running it at the same time, so we’re going to be needing to first of all. Create an object using the multiprocessing library so create a variable to which we’re going to be assigning the object so p1 equals multiprocessing, and then we access the method or class so multiprocessingprocess and then in that you need to specify target, which for me is going to be. Um, sleep for a bit. That’s the name of the function. I want to run and then I need to pass in. Um, the args or argument attribute, which is going to store all the arguments that I want to pass into this function now. This needs to be passed in as a tuple, so I’m going to do a square bracket and a closing square bracket. And then I’m going to type in one because I want it to sleep for only one second. Now I’m also going to create another process, which is going to be called p2 so process one and process two digits abbreviations and then in here, nothing else changes, so it’s just p1 and p2 now. This is kind of like cached into these variables, so these variables know that this is like a process to them. Now, now, at this stage, nothing’s gonna happen. We actually need to go ahead and run these two processes at the same time and before that as a requirement as a standard requirement to using multi-processing, we need to use the if name equals main approach. So we’re going to do that, and then it’s just a requirement that, um, multi-processing has if you don’t use this, it’s not going to work, so we type in that. And then we type in p1 dot start. We’re just going to start our first process and then p2start we’re just going to start a second process now since we want to track track the time that this finishes on what we’re going to be doing is typing up a new variable called finish, which is going to be at the same indent as the rest of the code and we’re going to be assigning that to time dot perf counter, which is pretty much going to return the execution time of this whole program until where this variable is and then we’re going to print out finished, finished running after it’s usually going to take seconds for this program to run. So I’m going to say, finish running after, um, plus, Oops, plus, let’s just do comma and finish. Um, yeah, that should do the trick so after seconds, so finish running after seconds, and it’s going to say how long it took to run. So let’s run this now to see if it actually works. So let’s run it up and as you see right here. What happens is it says it took about 0.72 seconds to actually execute the whole thing now. You might be saying. Hey, these, um, there’s an actual sleep. Um, of one second in these, uh, functions right here. So why is the time only literally? Um, one third of the sleep time. So that’s because what what happens here Is that this, um, if I go back to my code. So when when the program goes through all of this code, what it does is, it’s going to skip these lines here and then actually go on to the next line and then print out the finish time before it actually has run the two functions, which is why we kind of get an inaccurate time, so for it to actually go in the right order and run this stuff first, and then this at the end, all we need to do is type in p1join and then p2join so this way it’s going to give us accurate results, so let’s run this up now, and if I can show you now, it literally says at the end finish running after 1.15 so-and-so seconds, so as proof, technically these run at the same time, because if they had ran at different times, they would have taken two seconds and a bit more of time because technically each function sleeps for a second so since they since the fact that they’ve actually run in one second, ignoring the decimals, obviously, they’ve technically run in one second, which means both of them will run at the same time. So that’s our asynchronous done cool now, just to prove my point about, actually, um, if we did it the other way of synchronous, it would take two seconds lets. Try doing it that way, So if I just commented this out and I did it the old approach where I do sleep for a bit one second sleep for a bit and another second, it should take two seconds and some decimal places to run this entire program. So let’s see, and as you see right here, It finished after two seconds and a few more decimal places after that, so the previous method that I showed you using multiprocessing pretty much, Does it in one second and a few decimals, which pretty much proves my point of running two functions at the same time, so the way that it does it is using the different cores that your CPU has. It’s pretty clever the way it does it, So I’m going to uncomment this now now. If you want to actually see the improvements that this does, you’d probably use this. Um, when you have to have a lot of functions pretty much running at the same time, so these are just two functions were running at the moment what we had about. Let’s say 10 functions of these and each function was sleeping for one second. Now imagine how long that would be that would take about 10 seconds for the whole program to execute, whereas if we done it all at the same time, it would take about a single second that would create a major performance improvement, then having to wait 10 seconds for the entire program to execute. So let’s actually do that. Let’s run 10 of these processes instead of running just one so for this instead of taking the dumb method and the longer method we’re actually going to write a loop and use an array. So I’m gonna get rid of this code right here. Um, so from here, let’s get rid of this, and I’m also going to get rid of this cool. So what I’m going to do now is I’m going to create a variable called processes, processes And then I’m going to assign that to an array now. This array right here is going to store all the processes that we’re going to be running. Um, together at one time, so we’re gonna need a for loop and we’re gonna say for X in range 10 which pretty much means that this loop is going to run 10 and all the instructions inside. It are going to be 110 times. What we want to do? Is we’re going to grab this line right here. That says p1 equals so and so and then we’re going to paste it in our loop. I’m going to take off one because it’s just a process now. So it’s p and then we’re going to put this in our loop now. Technically, this is going to run 10 times right so once. We’re done with that. What we want to do is actually type in if name underscore, underscore, equals underscore underscore main because it’s a requirement to pretty much run the multiprocessing module and start the processes that we have what we want to do. Is P Dot start, which is pretty much just starting the process that we have going now at this stage, you might say. Hey, are we done yet? And can we just do p dot? Join over here and my answer would be No. Because if I run this now, it’s pretty much going to take. It’s sweet time so as you see right here. It’s waiting for 1.5 seconds before it’s actually done anything else. It’s literally going to take about 10 seconds for the entire program to run. So that’s obviously not how we want it to work. We want it to finish in one second, because, um, technically a bit more than one second because I’m waiting for 1.5 seconds. Let me take that off and make it one second. So we want it to finish in one second because we want to take all the 10 processors. Start them at the same time now. If you want it to work like that, what you got to do Is you got to take each of your process that you’re starting and then append it to your process list that you created earlier, so process Dot append. And then you pass in the P. Which is the, um, process reference or the object reference. So once you’re done with that, what you want to do is go ahead and create another loop so for P in processes. What you want to do is type in p dot. Join so it’s going to literally grab all the processes that you’ve stored in your processes array right here. It’s going to grab all of them one by one and then join them individually. So when the when it does that it pretty much knows that it wants to wait for all of these. Um, processes to be run at the same time first. And then it’s going to run these functions right here or these lines of code right here. So if I run this now as you see right here. It took about 1.22 seconds, which is perfect because 22 seconds. We can ignore that because technically, the other things or the other instructions such as imports Take a bit of time too. So as I’ve proved my point, This would save you a lot of time if you had to do a lot of processing at the same time, um, such as running 10 functions or the same function 10 times at the same time, instead of running it at different times or synchronously, which would pretty much take forever, so this function synchronously would literally take about 10 seconds because it would sleep one second, then run again and then sleep one second and blah, blah, blah. Whereas in asynchronous, it can just run in one second because we’re using the cores of our CPU to pretty much run these 10 processes at the same time anyway, guys. This is the old way of how we do it using multi-processing in the next tutorial. I will introduce you guys to the most recent and new way of how this can be simplified and we can receive the same outcome. So hope you’re excited for the next one. If you guys would like to support the channel you can do so by signing up as a patreon using the Patreon link in the description, you can also buy a super chat emoji or a highlighted message in the chat. I’m not forcing anyone to do so, but if you’d like to do so, that really helps out too also consider following up my socials and my discord, which is going to be stated in the description for a load of fun and guys, I will see your beautiful faces in the next tutorial peace.

0.3.0 | Wor Build 0.3.0 Installation Guide

Transcript: [MUSIC] Okay, so in this video? I want to take a look at the new windows on Raspberry Pi build 0.3.0 and this is the latest version. It's just been released today and this version you have to build by yourself. You have to get your own whim, and then you...

read more