About
Marlos’s research focuses on designing algorithms that discover spatial and temporal abstractions in order to empower reinforcement learning (RL) agents to tackle the problems of credit-assignment, exploration, and generalization. For space abstractions, he has shown the benefits of incorporating the sequential structure of decision making problems into the representation learning process, as well as the benefits of augmenting agents’ representations with predictions and uncertainty estimates. For temporal abstractions, Marlos has introduced the idea of using learned representations to discover options (i.e., courses of actions) for temporally extended exploration. Several of these ideas were incorporated in the design of a deep RL algorithm for controlling balloons in the stratosphere; one of the first deployments of deep RL in the real-world. Marlos is also passionate about reproducibility and proper experimentation in machine learning; he was responsible for introducing stochasticity and modes to the widely popular Arcade Learning Environment.
Relevant Publications
Agarwal, R., Machado, M. C., Castro, P., Bellemare, M. (2021) Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning. International Conference on Learning Representations (ICLR).
Ghosh, D., Machado, M.C., Le Roux, N. (2020). An Operator View of Policy Gradient Methods. Neural Information Processing Systems (NeurIPS).
Bellemare, M., Candido, S., Castro, P., Gong, J., Machado, M. C., Moitra, S., Ponda, S., & Wang, Z. (2020). Autonomous Navigation of Stratospheric Balloons using Reinforcement Learning. Nature; 588:77‑82.
Machado, M. C., Rosenbaum, C., Guo, X., Liu, M., Tesauro, G., Campbell, M. (2018). Eigenoption Discovery through the Deep Successor Representation. International Conference on Learning Representations (ICLR).
Machado, M. C., Bellemare, M., Talvitie, E., Veness, J., Hausknecht, M., Bowling, M. (2018). Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents. Journal of Artificial Intelligence Research 61: 523‑562.
Support Us
CIFAR is a registered charitable organization supported by the governments of Canada, Alberta and Quebec, as well as foundations, individuals, corporations and Canadian and international partner organizations.