Transcript:
Hey, guys, welcome back to this class. Unsupervised machine learning hidden. Markov models in Python. In this lecture, we are going to go over the very important scan, function and piano. Why is this important think about how? Fianna works! You have to create variables and link them together functionally, but they don’t have values until you actually run the functions. So when you create your X matrix, you don’t give their shape. You just say that it’s a matrix. So Fionna knows it’s a two-dimensional object. What happens if you want to loop through all N elements of X? Well, if Yano doesn’t know what N is because you haven’t yet told it what value you’re going to pass in for X If you know that X is always going to be length N then you can do a for loop from 0 to N. Generally speaking, we cannot usually guarantee the length of our training sequences. So what happens if you want to do something like for? I in X range X shape 0 You can’t do this because X dot shape 0 doesn’t have a value yet. This is where the piano scan function comes into play. The scan function allows you to pass in the length of a fee, a no variable as the number of times it will loop now. Sometimes you can use a for loop, but you still may want to use the scan function. Instead, here are some of the advantages of scan over for loops. First, it allows the number of iterations of the loop to be part of Fianna Symbolic graph. Second is that it minimizes the number of GPU transfers if the GPU is involved, third is that it can compute gradients through sequential steps. The fourth is that it’s slightly faster than a Python for loop with a compiled piano function. The fifth is that it can lower the overall memory usage by detecting the amount of memory needed. So let’s look at the anatomy of a scan function. This is the scan function in its simplest form. The first argument is some function that it’s going to why to every element of the sequence that you pass in the second argument is the actual sequence to pass in so every individual element in the sequence will have the. FN function applied on it. N steps is the number of times to iterate. Usually that’s just the length of the sequence. The function returns two things, something called outputs and something called updates. Updates is a Fianna Object of type ordered updates. Usually we just ignore it. So I’m not going to talk about it anymore. Outputs is what we’re really interested in assuming some function returns, Something outputs will be all the M things it returned concatenated together. So for example, if I pass in one two three and my function is Square, the outputs will be one four nine, Of course, the scan function can do more complex things in this, so let’s incorporate another argument outputs info, which allows us to compute recurrence relations. The output’s info argument represents the initial values of your recurring variables notice. I put it in a list That’s because there can be more than one and Theano expects a list here, so you can imagine we could compute something like Fibonacci, which depends on the previous two values note that when we do this when your recurrence returns more than one thing, the output’s variable of the scan gives you both of these things in a list, so you’ll get the recurring variables there, too, In fact, however, many things you return from the recurrence. That’s what you’ll get in the outputs yet. Another argument we can add is the non sequence’s argument. This is for passing arguments into the recurrence that we don’t want to loop through. We just want the whole thing. Let’s now write these in code and see what happens. All right, so we’re just gonna write. Scan One Dot Pi. This is in the Github Repo. If you don’t want to code along, so we’re gonna import Numpy as MP import fee. Ah, no import fee. Uh, no tensor as tea and we’re gonna set. X to be a vector, which means every element in the loop will be a scalar. We’ll define a function called square. This is gonna return x times. X now we’re gonna call. Piano Scan puts updates equals the anode scan. So the FN is square sequences is X and steps is Xscape zero. Okay, so we still need to create a Fionna function, so that’s gonna be called Square off. I’m gonna make a piano Function. Inputs will be just X and the outputs will be outputs, so we’re output value is going to be square up and we’re gonna pass in an array of one two, three, four, five and then we’ll print output over. Oops over, so lets. Run this, you all right, So the results are as expected, all right. X, were gonna work on scan to type I so we’re gonna have all the same imports as before, and this time we’re gonna try to calculate Fibonacci and we’re gonna pass in and instead, okay, so we’ll define a scalar called N. We’ll define our recurrence. It’s gonna take an N F at 1 FN 2 we’re not actually gonna use in. We’re just gonna return FN 1 plus FN 2 and also FN 1 so the outputs and updates will be the same we call. Fianna Scan So the FN is recurrence. Of course, now the sequences are not in actual sequence that we have, but we’re gonna make one, so we’re gonna use the a range function. This is just like the Python range function before Theano pass in N steps equal to N and outputs info is the initial values. So I’m gonna pass in 1 and 1 right, Because the first two initial values of Fibonacci are 1 and 1 now I’m going to create the actual function, which is a piano function and it’s gonna take his inputs just N and the outputs will be outputs, so my output value will be Fibonacci. And I’m gonna do eight steps of Fibonacci, and I’m gonna print out the output at the end, so lets. Run this so you can see. The results are as expected. Notice how we have two things in the output, right, We have two 255 and then 1 234 which seems to be one behind the first array and that’s because in our recurrence were returning the newest value and the previous value. Now, let’s do the next example, all right, so now we’re gonna be working on scan Three Dot Pi. It’s gonna require these imports, So what we’re gonna do in this example, is a low-pass filter, so we’re gonna create a signal, which is gonna have a lot of noise and it’s gonna be length 300 An underlying trend will be a sine wave. All right, and we’ll plot it. So you know what it looks like with the noise now? I’m going to define a variable called decay and a vector called sequence. So that’s gonna be our noisy signal now. Our recurrence function is going to take an X. The last value of our cleaned up sequence and decay. We’re gonna return 1 minus decay times X plus decay times the last value. Now the output’s here. I’m just gonna ignore the updates. This time we’re gonna call scan so FN will be recurrence. Sequences will be sequence and steps will be sequence. Dot shape, 0 outputs info will be a float, just the value 0 and nan sequences will be, of course decay. Okay, we’re gonna create a piano function called LPF and it’s going to take in the inputs sequence and decay and the outputs will just be the outputs of the scan, So let’s set Y to be outputted. Lpf, we’re gonna pass in. X and 0.99 will be the decay apply. Y give it a title and show it, so lets. Run this and see what happens. Okay, so here’s. The original signal, pretty noisy and here is the cleaned up signal.