Anyone who has played video games since the days of Pong has witnessed a dramatic shift in their physics. Some of today’s games continue to hilariously disregard Newtonian physics, like the satisfying “boing!” of Mario scaling walls in his 2D world. But players of many of today’s popular video games demand increasingly realistic simulations of real-world experiences, such as snow crunching underfoot or objects bouncing with realistic kinetic energy.
Derek Nowrouzezahrai is a Canada CIFAR AI Chair at Mila, and holds the Ubisoft–Mila research Chair, Scaling Game Worlds with Responsible AI. Through his academic work as well as through his applied research with the video game company Ubisoft, Nowrouzezahrai’s group is advancing two physics-based applications of AI in video game development that are key to creating realistic virtual worlds. The first focuses on AI-driven simulations that allow for realistic interactions within strict performance constraints—such as a 16-millisecond processing budget—while handling multiple tasks, including user input and network communications. The second aims to improve how AI models render light physics for realistic visuals. These models must understand the position of the virtual camera and accurately depict how light interacts with various surfaces, including glass, metal, fur and water.
By embedding physics-inspired rules into AI models, Nowrouzezahrai’s team trains the models to learn more efficiently. Instead of needing vast amounts of data to understand basic concepts like gravity, models are pre-informed, saving time and resources. This leads to faster training, as well as more accurate and resilient AI systems.
As game design AI models continue to advance, their ability to assist in content creation for increasingly complex game worlds will continue to grow. But what does this mean for human game designers? Like many industries today, workers in the game industry are concerned about the job impacts of generative AI. That’s another focus for Nowrouzezahrai, who is passionate about not only the technical considerations of generative AI, but also the societal impacts.
“Generative AI can streamline parts of the game design process, augmenting artists’ workflows without replacing their creative input,” says Nowrouzezahrai. “It’s important to understand that the artistic input here is paramount to the success of these game products and experiences. There’s a spectrum at play, between fully automating the generation of all this content, and manually authoring it. The way we see it, there’s an interesting place in the middle, because we understand that creative input is by definition unique. It’s characterized and driven by artistic needs, and while they can be augmented and supported by AI techniques, they cannot be replaced wholesale. Instead of asking how will our roles disappear, I think the question is, how will our roles evolve? As I see it, this is an example of the highly collaborative experience that can come about between technology and humans.”
In the year since his appointment as a Canada CIFAR AI Chair at Mila, Nowrouzezahrai has come to see the role as invaluable for fostering a strong, collaborative community for the exchange of novel ideas. “What I’m struck by is how excited people are to see how their work or their interests connect to other fields and other problems that otherwise we wouldn’t have thought about, like taking novel techniques for computer vision and applying them to find better ways to simulate lighting in a virtual world,” he says.
“I happen to sit in two fields, computer graphics and machine learning, and it’s been fascinating to see how the synergies have grown between the two fields and how their impact has spread out even further. Canada has been a fantastic place to do this work. I’m very grateful. I consider myself blessed, and so does my team.”
Derek Nowrouzezahrai, Canada CIFAR AI Chair, Mila