Pytorch Datasets | How To Build Custom Datasets For Images In Pytorch

Aladdin Persson

Subscribe Here





How To Build Custom Datasets For Images In Pytorch


[MUSIC] In this video, we will implement a custom data set to load our data in all the videos I’ve done. It’s just been about creating a model or saving a model. Uh, you’re using transfer learning, etc. But a really big problem is loading the actual data and there are a few data sets that you can load very nicely from pytorch, but let’s say you have your own images or maybe some data set that you downloaded online. How do you actually load the data efficiently? So that’s the question we’re going to try to answer in this video first? I have some standard code here. That’s working, except we have no way to load the data. The data that we have is a data set that I’ve downloaded of cats and dogs and what I’ve stored is in a csv file. I’ve stored the image name. That’s in this specific folder. And also the second column here is the target, so one for dog and zero for cat. What we’re going to start with is we’re going to try to Def Create a class which is going to be used to to load our data set, so we’re going to do cats and dogs data set and we’re going to inherit from data set. Then we’re going to define an init and what we’re going to send in is a CSV file, so the CSV file. I just showed a root directory, which is going to be the root directory to the to the images and then we’re going to have transform, which is optional. If we want to have it, then we’re going to use selfannotations to be Panda Read CSV CSV file. So we’re going to use Panda to help us read the CSV file. We’re not going to do anything complicated. Then selfgroupdirectory will just be the directory, and selftransform will be transformed, so the really the functions that we’re going to have are Len so the length and then we’re going to have get item and what we want get item to do is return. Is a get item is going to return a specific example I a specific image and corresponding target to that image, so we’re going to do first. The length is pretty easy. We’re going to return Len of selfannotation. In our case. We have 25 25 000 images, uh, 50 of cats, 50 of logs, then so what we’re going to start with is we’re going to find image path. We’re going to do Ospathjoin selfroot directory and then we’re going to do selfannotationsi look and then index comma 0. So remember, we send in a particular index, That’s we don’t choose the index. That’s Python does for us, but we’re gonna do selfannotation. Which is the CSV file? We’re gonna, uh, this the row. I and the column zero, right, The first column was the name of the image, then we’re going to do image is ioimread image path. Then we’re going to do y label is so the Y label will just be at that same except the first column and all that we’re going to do is we’re going to do in’t convert to an integer and then we’re going to do Torch Dot Tensor and in between we’re going to do ifselftransform we’re going to do image is selftransform of image, and then that’s it, so we’ve loaded the image. We’ve done some transform, which is optional, Uh, if we send in transform, it’s going to do the transformation, and then what we want to do Is we want to return the image and the Y label? Okay, so this class we’ve created is for a specific. It just loads, one image and the corresponding target to that image. Now we can go back to our load of the data and we’re gonna first say see, we’re gonna do from custom data set, so that’s the let’s see, that’s the the python file. Let’s actually make this full screen. We’re going to import cats and dogs data set, then we’re going to say data set is copy that and then we’re going to send in the CSV file, which we called cats. Dogscsv root directory will be cats, dogs resized and transform the transform we will use is since we’re going to load it as an image. We need to convert it to a tensor, so we’re going to do transforms DOT 2 tensor, and now one thing we can do here, See valid syntax, be probably one less. Yeah, then one thing we can use also is we can use train set comma test set, so we can use, uh, for this we can we can use. Torchutilsdatarando’m split from the data set and let’s do 20 000 to the training, set 5000 to the test set and then we’re going to define our train loader to be data loader. So this is the standard way. All that really we’ve done That’s unique to our specific data set is we’re going to use that class that we created. Now it’s just pretty much the standard that we always do. So we do. Data set equals train. Set batch size equals batch size shuffle equals true, and then we just copy this for the test loader and test set. Yeah, and that should be it. So that’s how we load data if we have images in a folder and we have a cc file that describes the image name and also the target corresponding to that specific image, so lets. Uh, so for fun. What I’ve done here Is I’ve imported the Google Net. [MUSIC] Google net architecture with pre-training. And yeah. I didn’t cover this code, but it’s pretty standard code. We have the loss. We have the optimizer. We have some training here of the network, and then in the end, we have check accuracy on the training set and the test set. Yeah, so let’s try to train on this. Just for curiosity. On the 5000 images in the test set, we got about 96 97 accuracy, which is pretty cool, so hopefully you were able to follow the video on how to load data. If you’ve gathered your own data set, and if you got any question, then leave them in the comment below. Thank you so much for watching the video, and I hope to see you in another video.

0.3.0 | Wor Build 0.3.0 Installation Guide

Transcript: [MUSIC] Okay, so in this video? I want to take a look at the new windows on Raspberry Pi build 0.3.0 and this is the latest version. It's just been released today and this version you have to build by yourself. You have to get your own whim, and then you...

read more