By: Krista Davidson
5 Aug, 2020
Training AI: Q&A with Chelsea Finn
August 5, 2020
Chelsea Finn is a leading AI researcher defining the new field of meta and multi-task reinforcement learning (RL). She is a CIFAR fellow in the Learning in Machines & Brains program, an assistant professor at Stanford University and a researcher at Google Brain. In addition to running her research lab, IRIS, Finn teaches undergraduate and graduate courses at Stanford University. She’s developed the course outline for the new subfield of machine learning, meta reinforcement learning. Finn is joining the Deep Learning + Reinforcement Learning Summer School’s prestigious lineup of speakers to share what inspires her about the field of AI research and training the next generation of talent, as well as how the COVID-19 pandemic has introduced new perspectives and possibilities in her research.
What is your current research area?
CF: My work is at the intersection of machine learning and robotics. I am developing core machine learning algorithms that are inspired by the challenges we see when we deploy systems into the real world. One general problem that I’m moving towards is developing machine learning systems that can multitask. Robots can do one thing well, but if you want a robot to be broadly useful and to handle real-world situations, the robotic agent must be able to adapt to different situations and carry out different tasks. General purpose robots require a broader understanding of the world in order to be practical. This introduces a huge challenge in machine learning because we need to build systems that can generalize and handle different kinds of environments. A lot of my work recently has been in studying how we can get a robot into a new environment and allow them to quickly learn new tasks with only a small amount of data.
What are some of the challenges in this area?
CF: One of the big challenges in RL in particular is the de facto way that these algorithms operate. Conceptually, algorithms are much better at learning from scratch, but do so in a very inefficient way. My research is focused on developing a method that improves how robots learn by leveraging their past experiences. If we can do that, robots may be able to learn something in a matter of minutes as opposed to days.
Is that the subfield of multitask reinforcement learning?
CF: Multi-task reinforcement learning is how a robot can learn many things by using shared structure. It can learn to perceive objects or how to slide objects slide across the table. It’s able to learn all of those tasks, more quickly than if you were to learn each of them individually from scratch. Meta reinforcement learning takes us one step further and is able to leverage that previous experience to learn a new task.
This is a pretty gargantuan task. What are some of the challenges?
CF: One is defining what the tasks are and what tasks should be leveraged in order to take on new tasks. For example, if we learned a set of 50 tasks previously, could that learning be applied to a new task? Or is that new task better learned from scratch? On the robotic side of things we often focus on object manipulation tasks like pouring liquid from one container to another, or pushing an object across a table. There’s so much diversity in the world with regards to the kinds of tasks you could train a robot to do. We recently released a benchmark of a set of 50 tasks that help researchers to test this problem in isolation.
Another challenge is what’s known as distribution shifts. If you’ve trained an agent on one set of environments and something about the environment changes, such as the lighting conditions, the agent can’t handle things that are very different from what they’ve before. It would be a huge breakthrough in AI and machine learning if we could understand some of the core elements that are necessary to solve this problem. It would help robots move beyond the factory floor and act intelligently in homes and other environments where humans work. The challenges for introducing robots to these settings are similar to the challenges faced in self-driving vehicles, where 99 per cent of the time, everything is kept fairly standard but there is always the one per cent of the time where you run into a situation that’s different. Maybe it’s a construction zone or the lines on a street are painted differently, or there’s a person holding a stop sign.
How did you become interested in machine learning as a career path?
CF: I was quite interested in computer science, engineering, and solving problems. One of the things I love about research is that you are trying to solve problems that no one’s solved before. I’ve always been drawn to AI because of the interesting fundamental challenges, and also with regards to how we as humans are intelligent. AI has tremendous potential to have a real positive impact in society. Back in 2010, before self-driving cars were being used commercially, I was rear-ended by a distracted driver on the highway. I remember writing in my college admissions essay that these kinds of accidents could be preventable with AI because self-driving cars wouldn’t be distracted the way humans are or subjected to feeling drowsy.
Q: Why is it important for researchers such as yourself to participate in AI training programs like the DLRL Summer School?
CF: I’m hoping to educate and inspire the next generation of researchers. Students at the beginning of their research career are the future leaders in the field. They’ll decide where we go with machine learning in the years to come. Having this opportunity to train researchers from all over the world, could have an impact on the way they go about their research and how they think about future problems.
We live in really interesting times right now, how has the COVID-19 pandemic affected your research?
CF: There’s been a lot of issues around accessing labs and buildings. It has made me really think, with more people working from home, and how we might place more robots in the home. My lab has purchased low cost robots that our students are working with and they’re operating in their homes. What I’d love to do is pull the data across these robots, so that the robot sees many different environments rather than just a single lab environment. A lot of machine learning algorithms pull datasets from the Internet, but robots operating in the real-world need data that is pertinent to the environment it’s operating in. It may help us understand what happens when we deploy robots into more diverse environments.
Is there anything that surprises you in the exciting world of AI?
CF: I know how hard it is to get robots to do certain tasks that may seem simple, but yet robotic agents have mastered games like Go and chess, which are extremely difficult for humans. We’ve made tremendous progress in AI and machine learning, but at the same time, it’s still quite limited in its scope. I’m still mystified by how challenging it is to train AI on basic skills that we see in young children, like grasping objects like cups. However, few humans can play Go on the same level that a machine can. Humans are really good at carrying out tasks that are intuitive to them and it’s exactly those tasks that are difficult to train an AI system to do.
Related Articles
-
Three 2024 Nobel Laureates among CIFAR’s acclaimed community of researchers
October 15, 2024
-
Indigenous perspectives in AI
June 18, 2024
-
Beyond privacy: it’s time for a rights-based approach to regulating AI for children
April 24, 2024
-
Celebrating 20 years of DLRL
January 10, 2024