Transcript:
This video is sponsored by HBT app more on that later. Alright, democracy! There was most interest in the Pybind 11 option. So let’s talk about my favorite library for writing python wrappers for C code. The blurb on their homepage pretty much shows why I love it. It’s a header only library, that’s great. It is very similar to boostpython, which was super easy to use and led to really simple boilerplate and you can. If you must call Python code from C Plus, I use this library primarily to use existing C plus code, but it is also useful for drivers and generally for speeding up certain bits of code, though I find myself reaching for number to do that more often than not now, so let’s start off simple and then layer on complexity we’re going to write a very simple C plus function and go through the minimum work needed to call it from Python, So here’s the function clearly. Such complicated expressions cannot be evaluated in native python and we must turn to C plus. That was a joke! The very first thing we need to do is clone the pi bind 11 repository. There are other more elegant ways to use this library, my favorite being using it through the actual python package. But this is the simplest. Now we can write our cmake file, which is mostly just boilerplate, and then we add the PI by 11 directory and then called this cmake function that’s loaded from the directory to define the python module. Great, we’re halfway there, but not quite living on a prayer just yet. Let’s go back to our C plus. File include Pi Bind 11 and sometimes people redefine the name space for Pi Band 11 for easy addressing. And I’m going to do that here, too now to define the module, we can call this macro and give it the name of the module, as well as the name of a module handle both as inputs and inside the macro, we can start to define the module itself. Let’s start by defining a cute little doc string and you can do that by setting moduledock nothing fancy here. But the official example shows a better way to do this through a macro. So I recommend checking that out to expose the function We call moduledef give it the name that that function would have on the python side, which can be different from the C plus plus side and then pass in a reference to the actual function and that’s it we’re done, We just need to compile now, so let’s go to the folder with the project and set up an out of source build with a new folder. Run cmake! Run, make ha missing semicolon. Let’s fix that and we’re done. If we look at the files in this directory now, we’ll see this dot so file on windows or Mac. This might be a different extension, but there will be an equivalent file and this file has the module name and then a whole bunch of other crap. This file is your module. It behaves exactly the same as a Regularpi file. Let’s fire an ipython session and explore this module. We can import the module by just using its name and using the dire command, we can see that it has a number of usual symbols, including the dock that we had defined as well as the function that we have added and renamed lets. Check out the dock. It is as expected, we can also get help on the function and it’ll show the arguments as well as their types and also the return type, and if we call this function with the incorrect argument types, the function will fail that said we can still pass in ins and the wrapper does the necessary conversions because they’re obvious and returns a float as expected. Pi bind 11 will do these conversions for built-in primitives like ins and strings, and we’ll also do that for more complicated data structures like lists and dicks and tuples. As I will show later before we start layering on complexity, Let’s set up a simple file to do some testing. We’re going to do a star import here for convenience, and for now let’s just set up a simple function call and print the results and then set up a very simple she’ll command that will build and then run this test. I find that in development like this. Having at least some basic manual testing can go a long way. All right, let’s define a super simple class that exposes functions designed to multiply numbers and has one multiplier as its internal state to expose this class. All we need to do is call this Pi Colon codon class function templated with the class name. Then we pass in the module handle and give the python interface a name as well if we compile this now and call the dire function on the module, We’ll see that the class name shows up and we can print it, but not initialize it. Oops, we forgot a constructor. Let’s do it, let’s add a constructor and then go back and run the test again and darn, it’s still failing. This is because we never exposed the constructor itself in Pi Bind 11 We have to be very explicit about which functions and fields to expose and I personally like this added level of control and responsibility since I think it leads to cleaner Apis. Please leave a comment if you think otherwise, and let’s have a discussion. Exposing the constructor is pretty easy. We can just call def on the Pi Colon clone class expression and pass in this special symbol. Pi Colon Colon Init. This defines the constructor. The arguments for the constructor are given as template arguments to this function. Now we run the test and after some fixing, we can finally construct the class. We have a c class accessible straight from python with almost no extra work. How exciting is that now? We can’t call the multiply method of this class just yet, so let’s just expose it. Same as before, except now we have to use the class’s name as a namespace so now that we have a basic class and basic function exposed, let’s talk a bit about data types and automatic data type conversions here. We have a function that accepts a vector as an input and returns a new list as an output just multiplying each value of the list by the classe’s multiplier. Pi, bind 11 Lets you expose and call this function with almost no extra work. If we try to use this function right now, we get this really weird error, complaining about type mismatches, but also a really nice message below, saying that we probably need to import the stlh file. This is to make the type conversions for complex data structures optional since there’s some copying involved and more experienced developers might want to do their own potentially more efficient conversions, Though. For most things I find the automatic conversions are just fine after we import stlh, the function works perfectly and spits out a normal python list. Wonderful if we wanted to. We can also construct and return Python types from inside C plus. This can sometimes be very, very useful here. We have a function that’s returning a tuple. Using the special factory function that’s provided by pybind11. There are similar facilities for lists strings, hash tables, etc. And you can check out the pi bind 11 documentation to read up more about how to use them while this is very nice, I find that it is generally cleaner and more scalable to separate Pibin 11 code or interface code from the underlying library itself. Sometimes it’ll be downright necessary. If you don’t actually have access to the library’s code, we can achieve that here. By using the C plus Lambda syntax, the arguments to the Lambda are a reference to the class. Let’s just call that self just like Python and then the actual function arguments, then we can just put our return statement in the Lambda body and bam. We now have a much cleaner implementation of the same function as before, and we can run our test and confirm that this works correctly now while we’re talking about Python types, let’s talk about Numpy for a second Pi. Band 11 has first class support for Numpy arrays and we can access them by including numpyh. The arrays can be accessed through the PI Colon Colon Array symbol and we can accept those as function arguments and even manipulate arrays inside C plus for now, let’s look at a simple use case. We have this function that creates a matrix using vectors which we can treat as an image and then there’s an internal loop that sets part of that image as white. We can expose this function simply as we’ve seen before, and it works fine, but having large nested arrays in Python isn’t exactly a good idea, plus any image processing code that we might want to use here would be expecting Numpy erase, so let’s return those. Instead, this can be done automatically using the PI Colon colon cast function, so if you change the interface code to return the image in a lambda and do the cast there bam, we get an empire array and now using opencv, we can save it as a jpeg and view it nice now before we talk briefly about properties in the global interpreter lock. I would like to thank this video. Sponsor HBT app, which is a brand new habit tracking app for the iphone. Check it out at hbtappcom or on the app store. One of my favorite things about Python is properties and changing a function into a property with pybind11 is really straightforward for a read only property. We can change def to def underscore property. Underscore read only and we can do that to this image function and test, and now we can access this like a field rather than a function for a normal property, We need to define both a getter and a setter first, so let’s do that for the multiplier note that these can also be lambda’s, and now we can call the def property function and pass it the reference to both the getter and a setter. Let’s add some prints to these functions to make sure they’re being called. Do a quick test, okay. The final thing that I want to talk about is the global interpreter lock, also known as the Gill. This is something very specific to python’s threading model. I have a whole other video on it. If you want to learn more in Pi Band 11 we can control the lock on the C plus plus side, so let’s see how I’m defining a function here that takes a while to finish in this case. It’s just sleeping, but it could also be doing some heavy computation that takes a long time. Let’s also add some prints here to show when the function runs and when the function exits on the python side, let’s create a threadpool executor. Give it four workers and then call this function four times inside it. I’m also timing this. If we run this test, we’ll see that the four times This function runs happens sequentially and takes eight seconds to finish. This actually makes a lot of sense. Now, Let’s release the gill and see what happens to release the gill. You just need to define this variable that I’m showing here. And then the function executed forward from that point happens without the gill. If we do this, we’ll see that the four copies of this function now execute pretty much at the same time. They can’t be completely simultaneously and the total execution time is just barely more than two seconds. Nice were able to run these functions in parallel Now. Of course, this is dangerous territory here. I’m showing a situation where we create and manipulate a python list in the same function and doing this without reacquiring. The guild throws a SEC fault which, as it turns out, strikes terror in the hearts of a majority of the people that vote on my polls fixing. This is simple. We can just acquire the gill again before making the list, all right. I think this covers pretty much everything that you need to know to start. Using Pi bind 11 effectively, there are a number of other things to consider, including object, lifetimes shared and unique pointers and sharing of data without copying. But that’s a whole other video. I also haven’t gone over how to deploy a module like this correctly, which is also a much longer video for now. If you want to at least wrap all of this in a neat python package, you should check out the example on the PI by 11 site. And I will link them in the description, too. Thank you so much for watching this video and staying until the end like comment. Subscribe share with colleagues. And I’ll see you next time bye.