By: Krista Davidson
19 Jul, 2021
Leading with generosity
July 19, 2021
A conversation with Kyunghyun Cho on his major breakthrough in machine translation, why he donated his awards, and why the unknown inspires him.
In the past decade, CIFAR Fellow Kyunghyun Cho has made significant contributions to the field of machine learning, particularly in how machines see and learn from the world. His research in neural machine translation and natural language processing have advanced our understanding of machine and human intelligence.
CIFAR: How did you become interested in machine learning, and more specifically, in neural machine translation?
Cho: My area of expertise is designing deep learning algorithms for structured data such as natural language and molecular biology. I was trained to design learning algorithms for deep generative models during my PhD years. Under the advice of Yoshua Bengio [Canada CIFAR AI Chair, Co-Director, CIFAR Learning in Machines & Brains, Mila, IVADO and Universite de Montreal], I focused my research on neural machine translation and natural language processing.
I’m fascinated by the possibilities of machine learning algorithms and deep learning algorithms to go beyond what humans are capable of. One of my research directions, for instance, uses deep neural networks to model biological sequences of DNA and RNA. It is an extreme simplification, but everything that we know about biology comes from blueprints, that is a sequence called DNA. That DNA is transcribed into often-shorter segments, called RNA. The RNA sequence is then translated into another blueprint that is used to form proteins. A lot of these biological sequences resemble what we see in natural language.
CIFAR: One of your most cited papers is about neural machine translation, which you published with Yoshua Bengio and Dzmitry Bahdanau (Canada CIFAR AI Chair, McGill University and ServiceNow Element AI). Your contributions to the area of neural machine translation is often cited as one of the greatest discoveries in machine learning. What did you learn from that experience?
Cho: I’m very fond of that paper. Not because it has been cited a lot or even that it has become one of the core algorithms that power a lot of recent advances in machine translation, but because we worked together with the goal of building a new system for machine translation.
Our goal was never to improve an existing system, we wanted to build an entirely new system. People have been working on machine translation for decades, so there was already an established way to build a machine translation system, but our team thought it could be solved using a different approach, using end-to-end learning. There had to be a way for a system to look at a lot of examples of translation, and to learn on its own how to translate sentences from the source language without further help. We tried to take a new approach to solving this problem that a lot of people in the machine translation community didn’t take seriously. This research changed how I viewed machine learning, and how I approach every problem, going forward. This experience has made me look for new approaches for challenging problems, even if there’s already an established solution.
CIFAR: What are some of the potential future applications of neural machine translation?
Cho: Aside from the fact that my mom wants to read my blog and it’s almost exclusively in English, I’m really excited to go beyond what humans are already proficient with. In the case of natural language and computer vision, how humans see and hear, that is how we perceive the world; it is how systems should see and hear the world. In those instances, humans are the ground truth. However, there are so many problems that we don’t know how to solve that nature or the Universe has somehow figured out how already. For example, how are DNA sequences designed or arranged so that they are transcribed into RNA sequences and are eventually translated into useful proteins? How do atoms come to form molecules? How did the Universe end up creating planets, stars, and even humans? We don’t know how to answer these questions. Our knowledge of the world is limited. I use machine learning to uncover this knowledge.
CIFAR: You recently established the Ho Am Prize & Scholarship for Macadamia at Aalto University, which you created with the funds from your recent awards. You also created a scholarship for women called the Lim Mi-Sook Scholarship at KAIST, which you named after your mom. What inspired your generosity?
Cho: So much of my success was being in the right place at the right time with the right people. The benefit of coming into the field at the start of the AI boom gave me, and a lot of my peers, room to relax and not feel anxious about the financial insecurities of research or pressure of publication. This is not easy for early career researchers nowadays. When I started as a professor at NYU, I had the opportunity to start looking into different corners of society.
I grew up in Seoul during an economic expansion. I went to a good university in Korea and continued my research career in Montreal. I was very privileged, without realizing it back then. Machine learning, however, benefits and has benefited only a small group of people, and it wasn’t until I was recognized for my own contributions that I realized the hurdles that other people have to overcome in their careers. I wanted to lower this barrier so that other people could contribute to and benefit from AI and thereby make a large, positive impact on society.
I created a scholarship in honor of my mother because I recognized the difficult choices she had to make as a woman and a mother. My guess is that many of the female students at KAIST who receive this scholarship will graduate and have successful careers, but I believe a lot of them will still have to make similar decisions that my mom faced and made earlier. I want these students to feel comfortable in making decisions so as to best support their futures including their career.
CIFAR: How did you get involved with the CIFAR Learning in Machines & Brains program and what are the benefits of being part of the program?
Cho: In 2013, I attended the CIFAR Deep Learning + Reinforcement Learning Summer School in Toronto, which consisted of only about 40 or 60 people. The lectures were amazing and provided an opportunity to have close interaction among the participants, and it was at that point in my career that I decided I wanted to continue to do research full time in the area of deep learning.
Now the machine learning community has really expanded and there are large conferences dedicated to AI. Being part of the CIFAR Learning in Machines & Brains program allows researchers to really focus on a narrower set of early ideas before they get to the mainstream research stage. It’s valuable to see which directions this community of international researchers are moving in. Right now, I notice that this group is becoming increasingly more interested in causality, which is an age-old concept. It asks whether an AI system can tell what causes the observations made by AI systems. If we can tap into this knowledge, we may be able to make AI systems more robust, and it will also help us enter this new era of sophistication in AI, in a sensible and responsible way.
Cho is a fellow in the CIFAR Learning in Machines & Brains program, an associate professor at New York University, jointly appointed to computer science and the Center for Data Science. He is a lecturer at the CIFAR 2021 Deep Learning + Reinforcement Learning Summer School.
Related Articles
-
Three 2024 Nobel Laureates among CIFAR’s acclaimed community of researchers
October 15, 2024
-
Twentieth edition of CIFAR’s DLRL Summer School brings the world’s top AI talent to Canada
July 24, 2024
-
Indigenous perspectives in AI
June 18, 2024
-
Beyond privacy: it’s time for a rights-based approach to regulating AI for children
April 24, 2024