By: Kathleen Sandusky
20 Oct, 2023
Canada’s foundational role in generative AI
October 20, 2023
Canada’s strong competitiveness in the generative AI field has its roots in longstanding Canadian support of responsible AI research.
This has been a signal year for the field of artificial intelligence, in particular driven by the public release of several generative AI (or foundation) models that can be used by people with no training in machine learning. As the potential power of generative AI tools becomes increasingly evident to all, it has quickly become the most-talked about technology on the planet, with millions of media hits and thousands of C-suite crash courses for a technology that has been available for less than a year. But how many of us know about its Canadian roots?
Today, one in five Canadians is using generative AI tools at work or school. Per capita, Canada is third among all nations in the G7 in total funding raised for generative AI companies, and fourth globally in the number of generative AI companies. This is no coincidence: it was Canadian-funded research over several decades—much of it supported by CIFAR—that led to key advances behind generative AI, positioning Canada for success.
Key advances that laid the foundation for generative AI were led by the scientific leaders of Canada’s three National AI Institutes under the Pan-Canadian AI Strategy:
Geoffrey Hinton, Chief Scientific Advisor, Vector Institute
The fundamental advances that led to today’s explosion of generative AI have their roots in Canadian research. Deep learning is the fundamental technical approach used in generative AI. For decades, it was a relatively obscure area in computer science. But deep learning surged in attention in 2012 upon the debut of the large neural network AlexNet from Geoffrey Hinton. Then Director of CIFAR’s Learning in Machines and Brains Program (LMB) and now the Chief Science Officer at the Vector Institute and advisor to LMB, Hinton worked with students at the University of Toronto Alex Krizhevsky and Ilya Sutskever (who went on to co-found OpenAI) to build the new model. It vastly outperformed all others in the 2012 ImageNet competition, exceeding the runner-up by an astonishing 41%. To this day this is Hinton’s highest-impact paper, having been cited more than 140,000 times as it forged the way for deep learning.
Richard Sutton, Chief Scientific Advisor, Amii
Reinforcement learning has been described by OpenAI’s Sam Altman as the “magic” behind ChatGPT. This advance is largely founded on the work of Richard Sutton, today the Chief Scientist at Amii, who literally wrote the primary textbook on the field. Sutton was recruited to Canada from the U.S. in 2003 by the University of Alberta, and has since advanced the training of thousands of Canadian researchers, in-person at Amii and online via a massive open online course that he developed with Amii Canada CIFAR AI Chairs Martha White and Adam White. Through reinforcement learning, AI agents are assigned a goal and learn through trial and error to maximize reward and minimize penalties, much as living organisms with brains do. But as powerful as reinforcement learning is for generative models, Sutton today sets his aim yet higher: understanding consciousness, including the very nature of intelligence.
Yoshua Bengio, Scientific Director, Mila
Consider the “T” in ChatGPT—the transformer. This breakthrough technology, which plays a key role in today’s faster large language models, has deep ties to Canada and CIFAR, having been advanced from the start by CIFAR-affiliated researchers. It was Yoshua Bengio, now Mila Scientific Director, Canada CIFAR AI Chair and Co-Director of the Learning in Machines & Brains (LMB) program, who drove much of this revolution. A seminal 2014 paper by Bengio with his then-students Dzmitry Bahdanau (now a Canada CIFAR Chair at Mila) and Kyunghyun Cho (who trained at Mila, attended DLRL early in his career, and is today a Fellow in CIFAR’s LMB program) proposed a method for improving the ability of neural networks to translate language by more flexibly recognizing the highest-value words in sentences of varying length. This led to subsequent work by AI researchers everywhere to advance transformers as a way to improve large language models, key to language-based generative AI tools such as ChatGPT.
Canada a leader in generative AI sector
Today, Canada is a leading player in both the latest research advances and commercial applications of generative AI. Canada proudly sits at fourth in the world for both the number of AI startup companies and the investment that they have generated. At the top of the list is Cohere, a Toronto-based company that’s giving Silicon Valley a run for its money. In all, more than 30 generative AI companies are helping enterprises across sectors deploy responsible generative AI tools within their business models.
At Canada’s National AI Institutes, our Canada CIFAR AI Chairs continue to advance the science and application of generative AI. For example, at Mila, Yoshua Bengio along with Canada CIFAR AI Chair Doina Precup authored the first GFlowNets paper in 2021, launching a subfield in generative AI research that is now accelerating discovery of new drug molecules and candidates for disease treatment. At the Vector Institute, Wenhu Chen is advancing novel generative models that can be used to improve forecasting across many domains, including predictions of solar plant energy output, electricity consumption, and transportation. And at Amii, Alona Fyshe is using generative AI to develop responsive learning platforms that are customized to students’ needs.
Canada’s role in AI safety today
Tightly woven into research advances in generative AI is the recognition by Canadian and international researchers that this technology has the potential for wide adoption. As a result, there is an even stronger imperative for responsible development and deployment to mitigate the associated risks. Additionally, there is a need to establish a shared understanding of how to manage those risks through regulation.
Here, CIFAR-affiliated researchers also play an important part in charting the global way forward, with Bengio, Hinton and Yann LeCun (a member of CIFAR’s LMB program, who shared the prestigious 2018 Turing Award with Hinton and Bengio) all taking on advisory roles for international governments.
In addition to advising the Canadian government on matters such as the Voluntary Code of Conduct on the Responsible Development and Management of Advanced Generative AI Systems, Bengio today is a Member of the United Nations Scientific Advisory Board for Independent Advice on Breakthroughs in Science and Technology, and the United Kingdom’s Frontier AI Taskforce. Both Bengio and LeCun have testified at the U.S. Senate Subcommittee on Privacy, Technology and the Law. Hinton has consulted with governments and companies around the world about the safety of AI technologies. All three have been quoted extensively in thousands of news interviews and reports about the potential risks of AI, and while they are not all in perfect alignment in their views, their debate is conducted scientifically and respectfully. All agree that some form of regulation is required.
“AI is a powerful tool that can be used for immense benefit for humanity, and it also carries with it significant risks if not conducted responsibly,” comments Elissa Strome, Executive Director of the Pan-Canadian AI Strategy. “CIFAR works closely with all our partners to advance the responsible use of AI, and we are glad to see the widespread attention being paid to the regulation of AI, as it is a technology that is already impacting the lives of every one of us, and will only grow in importance. Just as Canada led the way in innovating to develop this technology, so too will we help to guide the world in its responsible use.”
Image generated using Adobe Firefly with the prompt “Robot looking in a mirror at a Canadian flag, the robot is wearing a red knitted toque and scarf.” CIFAR has adopted principles for responsible use of generative AI, which includes labeling images with source and prompt. The Firefly model is trained on Adobe Stock images, openly licensed content, and public domain content where copyright has expired.
Related Articles
-
Ross Mitchell, Amii: AI note-taking for doctors
November 16, 2024
-
Data Communities for Inclusion: Recentering technology in the knowledge and well-being of the collective
November 14, 2024
-
CIFAR expands its global impact through strategic partnerships in Asia, Europe and beyond
November 14, 2024
-
Former CIFAR Fellow Paul Hoffman awarded the 2024 Kyoto Prize for Groundbreaking Work in Earth Sciences
November 12, 2024