Building Intelligent Systems An Introduction to Machine Learning Engineering

Geoff: Introducing Intelligent Systems. Hi. I’m Geoff Hulten. I’ve spent fifteen years managing applied machine learning practitioners at a big tech company. I’ve been involved in maybe a dozen internet scale machine learning systems that have hundreds of millions of interactions with users every day. My research has appeared in top journals and I won a SIGKDD Test of Time Award for Outstanding Contributions to the Data Mining Community that have stood the test of time. And I’m also Apress author. You can check out my book at www.buildingintelligencesystems.com. You can also check out my blog where I share thoughts and experiences, buildingintelligencesystems.com/blog. So, what is an Intelligent System? Let me start with a little bit more history. I got my PhD. in Machine Learning. Then, I went to work as a researcher in a product group in a big tech company. As I’ve said, this is fifteen years ago. And no one in the group really knew what it meant to put machine learning into a big product. Especially not one that was internet scale. And what I mean by that is it had hundreds of millions of users and could legitimately break the internet if it made a bad mistake. Well, Intelligent Systems is the name I’ve given to the pattern that I, and about a couple hundred collaborators developed over the years to do this. Let me give another little story. Maybe ten, twelve years ago, we started building an intelligent system to solve an important problem that was affecting millions of users across the world. When we started, we had maybe thirty people working on it. We needed to figure out how to use machine learning in a web browser, update it rapidly, combine it with knowledge we kept in the Cloud, deal with mistakes, get training data fast enough, make the things smart enough to solve the problem. Lot of personal years went into building that system. Fast forward ten years and the system is still running. It’s winning third party tests against major competitors as being the best at solving the problem. And the whole thing is being run by one good, but somewhat junior data scientist and a few vendors to deal with mistakes. Building Intelligent Systems in my mind is everything it takes to do this over and over. Solve an important problem using machine learning. And then run the system reliably, safely, and efficiently over time. So, why am I so excited about Intelligent Systems? Some of the biggest, most valuable companies have their core business built around answering really simple questions. Things like: what webpage should I display based on a short query? What product should I show to this shopper? What movie would this person enjoy right now? Which program should I block from running to keep a computer safe? These are simple things. But answering them very well at scale, has resulted in companies worth billions or hundreds of billions of dollars. And it’s done it by making a lot of people smarter, more productive, happier, and safer. But the reason I’m so excited is that, this is just the tip of the iceberg. There are tens of thousands of other questions we could try to answer. Even simple things like: When should my front door unlock? When should a light bulb turn on? What type of song should an artist write next? How long should I toast a piece of bread? I could go on and on. Some of these might seem small. A bit silly even, like toast. But at scale, they can affect many, many people. The better we get at reliably and efficiently creating systems to answer questions like these, the more potential we have to help people in so many ways. And one more bit of context before we get into detail, there are many skills that go into making working Intelligent Systems. As an analogy, in general software, you have base skills like programming languages, algorithms and data structures, networking and other specialized skills. But then, you have to take these skills and combine them to make working systems, and the ability to do this combination is a skill in its own right. Sometimes called software engineering. To be good at software engineering, you need to know about software architecture, software life cycles, management, program management. All different ways to organize parts of the system and the people building the system to achieve success. Software engineering skills are critical to moving beyond building small systems with a couple of people and to start to have big impact. When working with AI and machine learning, you have to add a bunch of things to these base skills. Including statistics, data science, machine learning algorithms and then maybe some specialized things like computer vision or natural language understanding. But then, you also need to integrate these new skills into your broader software engineering process. So that you can turn data into value at large scale. This presentation is about what you need to know to take these base data and learning skills and turn them into working systems. It’s not software engineering exactly, maybe machine learning engineering.