[music] All right, everyone! I am here with my friend. Robert Osazuwa nest and we’re going to spend a few minutes chatting about the interview and how it relates to his causal modeling and machine. Learning course, Robert. Welcome back! Hey, Sam, thanks for having me again. Hey, why don’t we just get started by having you just kind of riff on the interview and how it ties into the the themes that you’ve covered in the course we’ve been doing the kind of joint Alt Deep Dot Ai. That’s your education company and twiml collaboration for about a year and a half now. Uh, what maybe four or so cohorts so far? Yeah, it’s been a good collaboration. You and I and I enjoyed the interview with. Uh, Sean. Is somebody we we go ways back? I think I first met him. When I was running a meet-up on probably programming back in the Bay Area and he came and talked about profit, which was the, uh, forecasting stop where he built on top of, uh, probabilistic programming language called Stan Nice and so did any particular themes. Jump out at you from the interview. I think you prepared some clips that you wanted to. Uh, took some snippets here. You know, it’s funny. You know, I’ve had these. It’s actually tangential to some conversations I’ve had recently he talks about so one of the approaches that we focus on in our course is a is a generative modeling approach to causal modeling and and some of those things came up here and so, for example, he talks about heterogeneous treatment effects, heterogeneous treatment effect says. Hey, maybe that pill works better for some people than for others, and we like the machine learning model to give us some ident idea for whom it will have stronger treatment effects. And if you think about this as a labeled data problem, it doesn’t work because I would like to take a vector for you and predict an effect and I’ll never observe you getting the treatment and not getting the treatments. I can’t actually estimate a per per observation treatment effects. Yeah, so one interesting thing if you take a generative modeling approach to causal modeling, so what you’re doing is you’re you’re modeling the distribution of the causal query, so say, you want to know what the causal effect is of this treatment on that outcome, you’re looking at the distribution of that outcome under the intervention under under the treatment and and so from generative modeling approach since you’re just directly modeling, the distribution heterogeneity is built in right, it’s a distribution and so it’s spread quantifies uncertainty. It quantifies the diversity of the population. And so if you want to then take what that allows you to do is take modern probabilistic modeling tools. Uh, many of which are using cutting edge, deep learning auto differentiation architectures to do inference And just say like. Okay, why I have this distribution. What’s the probability distribution of this treatment on this outcome? What’s the probability distribution of this treatment on this outcome? If this person were over, six foot five and, um, likes to ride bicycles, right, and so that’s all just we can. We have modern tools and probabilistic modeling that we can just apply directly to those problems, and and and so that’s the approach that we take in the workshop because many people actually already have some skills in, uh, using some of those tools, for example, Pi Torch. But there’s also one of the things that popped out to me in this interview was kind of talking about the joy of modeling, and that’s something. That really resonates with me. You guys had a conversation about flow state. I talk to people about it all. The time I think getting into a flow State is really something that you should try to make your work facilitate, and so I really want to get back to doing stuff where I lose track of time and are able to make big progress on projects with a little bit of time and space. You know, I could think kind of back in my career. As a statistician data scientist machine learning engineer where I learned some technical skill and it dramatically improved, just my experience at work say, for example, I learned a little bit of functional programming. How to plan it, apply it to exploratory data analysis. I learned probabilistic programming and how you can compose smaller programs into bigger programs, using some ideas from category theory to model complex systems. And it was great. You can solve new problems And, uh, you know, more importantly, I think for me personally. More importantly, was that aside from the productivity gains and the ability to do new things. It was just funner, right like I was able to get into that flow state. Um, you know, there’s other sources of joy, right, Ecstatic Bliss Fiero. Uh, I don’t know, I don’t care about those like. I guess I really care about just getting into a deep work type of mindset and being able to do cool things with with my data with cool tech and, um, is the idea that causal modeling provided some of that for you? The approach that we’re working on in this workshop does right. So a lot of causal modeling. If you go to the textbooks, it’s just like, um, you know, let’s construct an estimate an estimator for this identifiable S demand, and there’s a whole bunch of theory and math, and and it doesn’t really, you know, it’s almost as though you have to learn a new degree just to just to be able to apply these methods when if you can take some of these concepts from generative modeling from probabilistic programming from, you know, in some cases, functional programming and and connect it to your intuition and understand how bits of a causal model can compose together like programs, It actually makes it, uh, well. Frankly, much funner than other ways of of going about. It also makes it easier to learn and it makes it easier to apply and so. That was one of the things I got. I got to thinking about when I listened to this interview with Sean. Nice, nice. Yeah, I’ve often found it interesting. When folks ask you. What are the prerequisites for the course and you reply? Well, some basic probability. But, uh, it’s not like you need to have. You know, some deep theoretical grounding in causal causal modeling or causality, or, uh, step statistics or anything like that. Yeah, and obviously those things would help you. If you wanted to apply it to a specific applied problem where you know, say, you’re working with some variable, that’s that has a lot of nuance, and, and, uh, the in terms of statistically modeling it. You need to understand a little bit about say. I don’t know, you know, measure theory. I that that’s possible with a specific class of problems, but it’s completely separate from the problem of applying a causal model to the problem and reasoning about it causally. Did you have any other clips that you wanted to share? No, no, here’s the spit off. You were talk, guys. We’re talking about leaky abstractions. I think I I. It relates to what we were just discussing in terms of reasoning about a causal model as a program that you can build on iteratively and, you know, apply some unit testing to, uh, you know, have an explainable model and kind of build a component of that model where one thing is working and then you can move on to another part of your causal model so that you can grow it into something that is, uh, building value for you and your organization over time and so I’m wondering like the interface. I guess the thing that I’m thinking of is like leaky abstractions. Like you know, how does that? How does that manifest in trying to use causal models in a real-world scenario like this? Yeah, I love that question. I’ve been telling my team for we’ve been working on this for about two years. Now is that we’re not building a model we’re building a system for building models that’s going to evolve over time to help us like, make agile adjustments to the way the business runs or to or as we have new information that resonates with me because two things number one a lot of tools in machine learning and statistics, they kind of encourage you to get really good at learning the tool they block they black box, most of the workflow and you just kind of learn some kind of art of hyper parameter optimization and with these types of causal models, you need domain knowledge, and and so you need to get you need to be thinking. Um, in very detailed ways about the data generating process and so if you can if you can take if you make clear composable abstractions of that process and focus on some small element of it to start with and then build it up over time and also the inference algorithm. If you can separate the model from the inference algorithm, you can perhaps use some cutting edge deep learning techniques for probabilistic modeling, say, Stochastic variational inference. And and you, you learn how to you know, make programmable inference and build that up over time. Not only do you have something that’s more robust because you can make sure that each component is working in isolation before you. You, uh, bring them together, but also something that builds IP over time for your company, and this is a lot different from how causal imprints is usually taught because it’s usually thought, as though, uh, the problem that you’re presented with at the beginning is the is the only problem that you’re ever going to face, and so if there’s a new problem, you have to start from scratch and build a new model and in a production setting, that’s that’s not a good approach. Well, your target participant, for the course, is someone with a someone who’s thinking about these problems from an engineering perspective, as opposed to a traditional statistical perspective, whether that’s you know in the the kind of sciences or social sciences, or what have you, which is a lot of where the where causality’s been, uh, been where folks are thinking about causal models and causality? Is that right. Yes, so you know, you guys had this blurb. And so we have to incorporate the effects of our previous decisions into the forecast. There’s really like a rich space of modeling problems just within forecasting, and it’s really never going to be as simple as just take this line and extrapolate it into the future and we have to think about a system rather than just like any just a particular model you think about incorporating in the potential decisions that you could make into your forecasts? How do you close that loop? Do you end up using simulation techniques or other types of techniques to do that? Yeah, it’s at the end. Our forecasting system is designed around causal models, right, and so that at least currently is the killer app for causal modeling in production in most settings. If you’re if you’re building some kind of predictive algorithm, it’s going to generate a prediction that you are then going to use to make a decision but often, in fact, more than often the this that decision impacts the outcome of that decision impacts the data that your algorithm is going to use to make future decisions that feedback loop can cause a significant amount of bias to the algorithm And so, yes, that is the main engineering use case that said we have lots of researchers in applied fields who are trying to get a global perspective on causal modeling. Um, many people from public health and many people from economics, for example, who maybe have learned a few specific causal model causal modeling or causal inference techniques that are popular in their domain, but don’t really have a global understanding for the theory. Got it got it, lets. Maybe switch gears and talk a little bit about the course itself. What’s the structure? For the course, it is a cohort based course we have online lectures and videos. We have assignments, programming assignments, and we have a weekly retrospective cohort meeting where we go online and talk about that week’s lectures, answer questions, maybe workshop, some people’s individual problems and then alongside we have some reading groups with previous students who are interested in talking about adjacent topics. Uh, and then we have a project-based element to the course such that If you’re trying to build a project, uh, you can you can where you can apply. These ideas we’ll provide support with that, and you can team up with other members of the of the course to come up with a good project outcome. Can you mention a couple of projects that students have taken on, uh, recently? One student used a deep learning technique called normalizing flows to implement a counter factual reasoning algorithm on images. Some other students used these students were fans of soccer football and they used. They built a college model. That would predict the outcome of a trade other students prior to covid. This was a really cool project. They used they downloaded. A bunch of Airbnb data downloaded a bunch of real estate data from Redfin and then composed them into a model. That would predict, uh. How much would basically if you wanted to buy a house with the goal of maximizing revenue from Airbnb? It showed you, you know, basically created a search engine for, you know, Airbnb optimal properties. Another then post covid. Some people did some really interesting things with, um, epidemiological models and adjusting them so that they were they were using causal reasoning. Nice, nice and all that might sound intimidating to some folks who would be perfect fits for this course in the past. You’ve, uh, made a point to make sure to let people know that they can kind of adjust their they kind of you. Get out of it! What you put into it and you can take it at different levels. Can you elaborate on that a little bit? Yeah, so if you you know, people are busy. A lot of people are full-time employed who are taking the course. And so you, um, we set it up so that you can move at your own pace. If you miss a week because of work, you can hop on to the cohort and get fairly caught up in the review section of the cohort. Um, you know, obviously, if you were going to go to. Uh, go deeper into the videos and homework. You’d get more out of it, but you’ll not it. It’s set up so that you’re never in a position. Where if you if life happens, you kind of get left behind the rest of the course, and you know, you won’t have the time to, uh, to continue. You’ll always be able to, um, catch up and and get and get as much out of the course as you can awesome awesome. Well, there’s a ton more that we can go into. Uh, one of the things that past students have raved about is the access they get to Robert and the one-on-one guidance that he makes himself available to to deliver. Um, but really, the next step is to check out some more information about the course, and you can do that at Twimlaicom causal and thanks for tuning in And thank you, Robert [Music] you!