Multimodal Deep Reinforcement Learning

Type Start End
Internal Sep 2016 Dec 2021
Responsible URL
Xavier Giro-i-Nieto

Description

Most recent breakthroughs in artificial intelligence are based on deep learning techniques trained over huge annotated datasets. These models are usually trained in a supervised manner and, while very effective for perceptual sensing tasks, their performance is upper-bounded by the annotator’s knowledge and involve high annotation costs. Many research efforts address more realistic scenarios based on reinforcement learning (RL) paradigm. In RL, an agent executes a sequence of actions in a responsive environment whose feedback is used as a guiding signal for learning. Analogously to perceptual sensing tasks, reinforcement learning agents have greatly benefited from the recent advances on deep learning. In this case, the necessary training datasets are often generated through computation, in particular, by running virtual environments. The Marenostrum supercomputer of the Barcelona Supercomputing Center (BSC-CNS) offers a unique infrastructure of thousands of CPUs that allow accelerating the process of collecting data and training the agents. This project will consider methods that are capable of taking advantage of the vast computational resources available.

The main goal of this research project is developing RL agents that can learn new tasks thanks to visually grounded language, mimicking the learning process of babies. We will aim at those skills which can only be learned thanks to a structured exploration of the environment following human feedback which, during the training phase, will be simulated by virtual environments such as Google DeepMind Lab or MIT’s VirtualHome. Results will impact in a more natural communication between humans and robots, allowing the later to quickly adapt the interaction skills learned on virtual environments to the real world.

 

Publications

Nieto JJosé, Creus R, Giró-i-Nieto X. Unsupervised Skill-Discovery and Skill-Learning in Minecraft. In: ICML 2021 Workshop on Unsupervised Reinforcement Learning (URL). ICML 2021 Workshop on Unsupervised Reinforcement Learning (URL). ; 2021. (5.67 MB)
Nieto JJosé, Creus R, Giró-i-Nieto X. PiCoEDL: Discovery and Learning of Minecraft Navigation Goals from Pixels and Coordinates. In: CVPR 2021 Embodied AI Workshop. CVPR 2021 Embodied AI Workshop. ; 2021. (847.54 KB)
Creus R, Nieto JJosé, Giró-i-Nieto X. PixelEDL: Unsupervised Skill Discovery and Learning from Pixels. In: CVPR 2021 Embodied AI Workshop. CVPR 2021 Embodied AI Workshop. ; 2021. (1.55 MB)
Creus R. Unsupervised skill learning from pixels Nieto JJosé, Giró-i-Nieto X. 2021 . (19.61 MB)
Nieto JJosé. Discovery and Learning of Navigation Goals from Pixels in Minecraft Campos V, Giró-i-Nieto X. 2021 . (16.15 MB)
Campos V, Trott A, Xiong C, Socher R, Giró-i-Nieto X, Torres J. Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skills. In: International Conference on Machine Learning (ICML) 2020. International Conference on Machine Learning (ICML) 2020. ; 2020. (6.89 MB)
Campos V, Giró-i-Nieto X, Torres J. Importance Weighted Evolution Strategies. In: NeurIPS 2018 Deep Reinforcement Learning Workshop . NeurIPS 2018 Deep Reinforcement Learning Workshop . Montreal, Quebec; 2018. (362.25 KB)
Bellver M, Giró-i-Nieto X, Marqués F, Torres J. Hierarchical Object Detection with Deep Reinforcement Learning. In: Deep Learning for Image Processing Applications. Vol. 31. Deep Learning for Image Processing Applications. Amsterdam, The Netherlands: IOS Press; 2017.
Bellver M, Giró-i-Nieto X, Marqués F. Efficient search of objects in images using deep reinforcement learning. NIPS Women in Machine Learning Workshop. 2016 .

Collaborators