Transcript:
Hi, everybody! Welcome back to a new. Pi Torch Tutorial. This time. We implement logistic regression. If you’ve watched the previous tutorials, then this should be very easy now. Once again, we implement our typical. Pi Touch pipeline with those three steps so first we set up our model. We define the input and output size and the forward pass. Then we create the loss and the OPTIMIZER functions And then we do the actual training loop with the forward. Pass the backward pass and the weight updates. The code here should be very similar to the code in the last tutorial where we implemented linear regression. We only have to make slight adjustments for the model and the loss function, so we add one more layer to our model and we select a different loss function from Pi towards Built-in functions. So let’s start first of all that it let’s import some things that we need, so we import torch, of course, and we import torch Dot N N S and N So the Neural network module. Then we import Numpy S and P to make some data transformations, then from SK learn we import datasets to load a binary classification dataset, then from SK learn dot pre-processing. We want to import Standard Skaila because we want to scale our features and then from SK Learn Dot model selection, we import train test split because we want to have a separation of training and testing data and now let’s do our three steps so first we want to set up the model. Then we want to set up the loss and the OPTIMIZER and then in the third step, we do the actual training loop and as a step zero. We want to prepare the data, so let’s do this, so let’s load the pressed concert data set from. SK learn so we can say BC equals data sets, dot load breast cancer. This is a binary classification problem where we can predict cancer based on the input features. So let’s say X and Y equals BC, dot data and BC dot target. And then we want to say well. First of all, it’s gotten get the number of samples and the number of features by saying this is X dot shape, so let’s print this first, so print the number of samples and the number of features to see how our dataset looks like, and we see we have 569 samples and 30 different features, so a lot of features here and now let’s continue and let’s split our data when we say X strain and X test and next test and Y train and Y test equals here, we can use the Train test but function where we put in X and Y, and we want to be the test size to be 20% so this is point two and let’s also give this a random state equals. Let’s say 1 2 3 4 and there should be a small s and now let’s convert or first of all. Now we want to scale our features. Scale them here. We set up a standard scale ax. SC, equals standards gala, which will make our features to have zero mean and unit variance. This is always recommended to do and when we want to deal with a logistic regression, so now we scale our data, so we say. X train equals S C dot fit, transform and then as an input we put in X train, and then we want to do the same thing with our test data. So we say X test equals. SC, dot here. We only transform it and here we put in X test now. We scaled our data now. We want to convert it to torch tens us, so let’s say. X train equals torch dot, and then here we can use the function from Numpy, and then we put in X Train and cost this to a float32 data type. So we say X train DOT S type numpy dot float32 Because right now this is of type double, and then we would run into some errors later, so let’s cast this and convert this to eight tens. Oh, and now let’s do this with all the other array, So let’s say X test equals this and our y train and also our Y test tens on my test, and now as a last thing to prepare, our data is to reshape our Y tends us. So Y train equals y train dot view. This is a built-in function from Pi Torch that will reshape our tenza with the given size, so it gets the size why train that shape, zero and one, so right now our Y has only one row, and we want to make it a column vector, so we want to put each value in one row with only one column, so this will do exactly this, and also for our Y test, so Y test equals this Y test, and now we are think we are done with our data preparing. So now let’s set up our model and here. Our model is a linear combination of weights and a bias and then in the logistic regression case, we apply a sigmoid function at the end. So let’s do this and for this. We want to write our own class, so let’s call this model or we can also call this logistic regression. Just tick regression, and this must be derived from N + Dot module, and then this will get a in it, which has self, and then it gets the number of input features and here first we call the Super Ini’t, So let’s say super logistic regression and self dot in it, and then here we define our layer, so we only have one layer self dot linear equals and here we can use the built-in layer N n dot linear, and this gets the input size so and input features and the output size is just one, so we only want to have one value one class label at the end, and then we also have to implement the forward pass here, which has self and the data and our forward pass is first we apply the linear layer and then the sigmoid function. So here we say. Y predicted equals Torche dot sigmoid, so this is also a built-in function that we can use and here we apply our self that linear layer so linear layer with our data X, and then we return our Y predicted, so this is our model, and now let’s create this so model equals logistic regression of size, and here we put in the number of features that we have so now our layer is of size thirty by one and no sorry. Thirty input features and one output feature, and now we have our model and now we can continue with the loss and the OPTIMIZER so for a loss, the loss function now is different than in the linear regression case. So here we say criterion equals and N DOT BCE loss, so the binary cross-entropy loss here and our optimizer is the same, so this can be this is Torche Dot Optim dot s GD for a stochastic gradient descent, and this gets some parameters that we want to optimize so here, we just say model dot parameters, and it also needs a learning rate, so let’s say our learning rate equals point zero one, and then here we say L R equals learning rate. So now this is step two and now step three, so let’s define some number of epochs equals. Let’s say 100 iterations, and now we do our training loop, so now we do for epoch in range Num, a pox and then first we do the forward pass forward pass and the lost calculation. Then we do the backward pass, and then we do the updates, so let’s say. Y predict it equals here. We call our model and as theta, it gets extreme, and then we say loss equals criterion and this will get a the Y predicted and the actual Y training, so the training samples or training labels. And now we do the backward pass and calculate the gradients and again we simply have to call loss that dots backward and Pi Torch will do all the calculations for us. And now we update our weights so here we simply have to say Optimizer Dot Step and again. Pi Torch will do all the update calculations for us. And then we don’t, or we must not forget to empty our gradients again, so one to zero the gradients because the backward function here will always add up all the radiance into the dot grat attribute, so let’s empty them again before the next iteration, and we simply must say OPTIMIZER DOT 0 Grat and then let’s also print some information. If a POC plus 1 Modulo 10 equals equals 0 so every tenth step. We want to print some information. Let’s use an F string here. Let’s say epoch and here we can use epoch plus 1 and then we also want to see the loss, so the loss equals loss dot item and that’s format this to say to only print for decimal values. And, yeah, so now we are done. This is our logistic regression implementation, and now let’s evaluate our model. So the evaluation should not be part of our computational graph where we want to track the history, so we want to say with Torch Dot. No grat, and then do our evaluation here so here. I want to get the accuracy, so let’s get all the predicted classes from our test samples, So let’s say this is model and here we put in X test and then let’s convert this to class label so 0 or 1 so remember the sigmoid function here will return a value between 0 and 1 and if this is larger than 0.5 we say this is Class 1 and otherwise it’s class 0 so let’s say. Y predicted classes equals Y pretty that dot round so here we can use a built-in function again and this will do exactly this, and yeah, so if we do don’t use this statement, then this would be part of the computational graph and it would track the gradient calculations for us so here. We don’t want this. We don’t need this because we are done. So that’s why we use this with statement here, And now let’s calculate the accuracy by saying. AK equals y predicted classes and here we can call the equal function equals y test and then the sum, so we want to sum them up for every prediction that is correct. It will add plus one, and then we divide this by the number of samples of test samples, so Y tests dot shape 0 this will return the number of test samples and then let’s print our accuracy. Print accuracy equals ax dot point for F also only for decimal values. And now let’s run this and hope that everything is correct and Standards Gala has no attribute transfrom. So here I have a typo transform. Now, let’s run this again, Trance. Form one more try, and now we are done and we have a accuracy of point 89 so it’s okay, it’s good, but it’s not perfect so you might want to play around with, for example, the number of iterations and where do we have it the number of epochs or the learning rate, for example, or also maybe try out a different OPTIMIZER here, But basically, that’s how we implement logistic regression. I hope you liked it. If you liked it, please subscribe to the channel and see you next time bye.