Transcript:
Hi, this is a new set of videos all about… I know you can’t wait to hear what this is about. Loading images… in Processing. Looking at the pixels of those images. What if those images are… video images coming from a live camera, coming from a movie file. What are the kinds of things we can do to load, display, manipulate, process, analyze images and pixel data in Processing. So that’s what this set of videos will be about. There’s maybe going to be somewhere between five and nine of them and… What we’re going to do here, and this first one is just start with the total and complete… Basics. The very first thing we need to do in Processing to display an image on the screen is have some sort of variable that can store a reference to all the information associated with that image. The variable that we’re going to need. We’re going to need is going to be of type PImage. PImage PImage is a new data type. I mean you’ve probably seen it before maybe you haven’t it doesn’t really matter. The point is, there is a PImage class. Just like at one point or another we wrote a bubble class or a ball class or a frog class. We wrote all.. We were writing classes to encapsulate information and data to store stuff that we want to move around the screen. This is a case where we don’t need to write a class to store all the information about an image. This is a class that’s built into Processing for us and we can just make objects we can make instances of PImage objects. So we have a variable let’s just call it img. It’s of type PImage. Now… How are we going to fill that variable? We need an image. There are lots of possibilities here. We could make a blank image and set all the pixels of that image according to some algorithm. We could pull an image from the web, from an API, from Google image search, Flickr… From any sort of service that you could imagine. We could… or what I think we’ll do right now, which is the simplest thing is just load an image from the hard drive. Let’s just get an image that we have on our computer. We want to load that image up and put it into our Processing sketch. So how do we do that? This is something, a point that I want to make is, this is a case where the code we’re about to write to load this image should really happen in setup(). Pulling the image off of a hard drive loading it into memory so that the computer can draw it into the screen. This is a slow process. Sure, we might want to draw it continuously. That will happen in the draw function. But that process of getting it in the first place Is something we want to do at the beginning of the program. Obviously there’s lots of other circumstances where as the program’s running we want to retrieve some images and will at some point hopefully look at some other scenarios. But for right now for our purpose. We simply want to say load… Image. Then we need can; you see all this? I think you can. We need the file name to go in here. The file name is going to be a string. Which means the characters that make up the name of that file, inside quotes. Now, where’s this file coming from? Processing sketches if you’ve probably seen this before but just in case you haven’t. Processing sketches are this… this sketch that I have called image1. It lives somewhere on my computer. In fact I can go up to here to Sketch. Show Sketch Folder and now I have in the Finder. I have my sketch folder right here. And there’s the file that has all the code for my stuff. So I want to make a folder that goes in here called data. This data folder… This is where I’m going to put images. I want to put things assets, things that I want to load, it could… later we’re going to see data files, sound files, movie files, all sorts of things. Font files can go in there. Anything that I want to load and use in my Processing sketch. Data… The data folder is a good place to put it. Once we have an image in that data folder. We can load that image, and we’ve got a reference to it in the variable. Now… an image is an object. What is an object? It’s a thing that encapsulates, that brings together data and functionality. So let’s try to think what… is the data of an image. What are the things that make up an image? Well one thing are the pixels. This… You know, really if, you’re watching this right now you can really just turn this video off and skip to the next one because the next one… or a couple from now we’re going to look at how to manipulate the pixels of an image or make up the pixels of an image from scratch. That’s going to be a very powerful technique for coming up with new ways of drawing and making stuff happen on the screen. The other pieces of information that an image might have… It does have in fact. Is the width and height. This, these are crucial pieces of information. The… the width and height comes from the source image itself. What are is the pixel resolution of that image? Is it some low resolution image, we downloaded off the web. Did it come from some fancy high resolution camera. The pixel width and the pixel height this is a sort of key factor. One thing you want to remember, is we’ve got a little Processing window… You know even if it was fullscreen.. Maybe it’s at the most like 1920 by 1080. We don’t need Massive high resolution images to display on the screen and in fact those might bog our program down and make it run slow. So something you might think about if you’re loading a ton of images in your program is sampling them down in advance. So that you have lower resolution images to work with so that your program rus faster and smoother. Okay, what is some functionality that might come with an image? Well a piece of functionality that we really want is to draw that image to the screen. Processing if you think about the way, it works… Doesn’t have too many objects that draw themselves to the screen. This is something we’ve been doing ourselves, as we’ve been making examples. We make a bubble object that has a display function, but the Processing draw functions are just commands really that we’re executing. rectangle, ellipse, line. The function for drawing an image to the screen is actually basically identical to rectangle. Rectangle if you remember needs an x and a y. Which references the… which… Indicates the top left corner. Hope, I’m recording this, I am. The top left corner of that rectangle. and then it needs a width and a height and that rectangle will appear somewhere in our window. Once we call the rectangle function. The image function which would happen in the draw loop. Is exactly the same as rectangle, in that we can give it an x, a y, a width and height. This is where that image, that rectangle will go. With what width and what height the big difference here is. We…. we now have the opportunity to reference… an image that we’ve loaded from the hard drive… and that image itself will appear inside the rectangle. This means an image can be moved around the screen. Just like anything else is. Using variables and algorithms and all sorts of fancy things. It can be repeated in a loop and tiled. Anything you could do with shapes, you could do with images. The other thing that’s interesting to note here is I spent all this time saying… Oh, the image has a width and a height that comes from the raw source file itself. Yes that’s true, but somehow the image function… I can give it a width and height. Processing will dynamically resize your image. Stretch it, skew it, shrink it, grow it. Whatever you do, whatever you… want… based on however you specify that width and height. By the way these are optional arguments. If you just give the x y… You’ll get the image at it raw resolution. The other thing I should point out, which you would probably write before you draw the image that I’m drawing it down here. Is just as rect mode has a way of saying: “Hey actually I want the reference point for that rectangle to be the center.” There’s a function called image mode. Where you can now reference the image by… You can now draw the image relative to a different part of it. Like it’s center. Center. There we go. Okay. Let’s…. I don’t know we’ve just been going on for way too long… and I don’t know if you’re still watching but let’s go and actually implement some of this in a quick sketch. We’ll wrap this video up, and then we’ll go to the next one. Now I’m back over. I’m a little out of practice. I haven’t recorded one of these in a while, okay. All right, so let’s go and find an image. I really should have done this in advance. I can’t believe I’m doing this in video. Bear with me. I don’t know let’s think of a cute animal that we like. Like a hedgehog. I don’t know is that a cute animal? I don’t know why I thought of hedgehog. Hedgehog not a hedge fund… That… whoops no video cell Ok, so yeah, that’s kind of a cute animal, so let’s get this hedgehog here View image, this is a little bit ridiculous. I… I just have to keep going with this. I’m going to save this image, and I need to find my Processing sketch. Which I have in documents… Processing… I really should have done this advanced pixels… image1… data. All right and let’s just call this hog.jpg there we go. Now… I’m back into my sketch, and I’m going to say a PImage. image. image equals load image hog.jpg and then image, hog, zer,o zero, background, zero and let’s make our image Our, sorry, sketch like actually be a little bit bigger and let’s run this. Okay ah so what happened? What happened first of all, I wrote image hog but I called my variable name img. Now this actually brings up a little point. I don’t know, I don’t really love the variable name img because then it’s just gonna lead me to say img1, img2. Actually, even though it is an image I think it actually would be useful to name it Something about what it’s going to display, so I’m going to call it hog. I’m gonna save this and I’m going to run this and boom we have I can close the browser now. We have.. a Processing sketch that displays… the image yay, okay. This is pretty exciting. It’s just sort of the background of the sketch. I could start drawing things on top of it… You know… like if I wanted to try to figure out how to…. give it some green eyes. I could kind of like try to guesstimate Where its eyes are and kind of put a certain… hey wow! That was amazing! I got really really close towards it’s eye… I just by looking. All right that was… I swear I did not practice that in advance. Okay, let’s put this back over here.. Now… Let’s look at a few more slightly more interesting features… for example…. I didn’t actually include the width and height as I mentioned you could do… So I’m going to actually give the with mouseX and the height mouseY and we can start to see now I’m able to interactively skew the size of the image. Another thing that I didn’t mention a rectangle, right… You could set the color for a rectangle using fill. fill(), right here I made an ellipse that was green. Zero comma five comma zero. The equivalent for an image is actually the tint function. So the tint function… will color that image with a different tint, so to speak. So if I say to 255, 0, 0. We’re going to see a very red hedgehog right now. This hedgehog… Now… what’s important to realize here is… We didn’t actually affect the colors of this image. We are just affecting which part… which parts of the image’s existing color are we drawing. Every single pixel has an RGB color. I am now saying take for every single pixel ramp the green all the way down to zero, ramp the blue all the way down to zero. So… the what I’m getting is the kind of the red values of that image. And you can see here if I were to do something like put mouseY…. you know, as I go down the image. I start to get the green parts of the color and and if I put mouseX into blue… I can start to get… Now I’m actually seeing the actual image itself. It doesn’t mean it’s I’m adding blue or I’m adding green. I’m just bringing up the actual existing of pixel values of this image itself. This is what happens when you don’t like…. Practice it and then do the video. I’m just rambling. Um, so… I want to wrap this one up. I guess when I’ve done these in the past they usually kind of give a little thing that you might think about trying to do before you want to go on to the next one. What I would say to you right now is set this up for yourself. Find an image, download it, get in your data folder. See if you can load it. See if you can draw it and see if you can come up with a more creative or inventive way of affecting its position, its size and its color through tint. So think about the same thing you first did… I had a bouncing ball moving around the screen or a bubble floating and moving back and forth floating up. You could do that with an image now. Another tidbit that I’ll just mention if you load a png image that has a transparent layer some…. in the you can load jpegs pngs and probably some other formats. That tends to be what I use jpegs and pngs. If that image has transparency… Processing will retain that transparency, so your image doesn’t necessarily have to be…. It will always draw as a rectangle but we might not see it as so rectangular if there are trans…. There’s a kind of transparent edge to it. We’ll see how to add our own transparency to it just later. Okay. Thank you for bearing with me. This is a video that I’ve recorded and I will hit stop and then record the next one in about two seconds.