Interactive Grounded Language Understanding in a Collaborative Environment
We are excited to announce winners of NLP and RL track here.
Want to know how data was collected for the competition? Check out our paper, Collecting Interactive Multi-modal Datasets for Grounded Language Understanding that will be presented at NeurIPS InterNLP workshop on Dec 3, 2022!
The current baseline for the IGLU RL task is explained in our preprint paper: "Learning to Solve Voxel Building Embodied Tasks from Pixels and Natural Language Instructions". Check it out!
Join our Slack workspace for discussions and asking questions!
IGLU has been accepted for the second year in NeurIPSConf competitions! This year we are serving a new NLP task as well as an RL task (NeurIPS2022 proposal).
Humans have the remarkable ability to adapt to new tasks and environments quickly. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions. Studies in developmental psychology have shown evidence of natural language communication being an effective method for transmission of generic knowledge between individuals as young as infants. This form of learning can even accelerate the acquisition of new skills by avoiding trial-and-error when learning only from observations.
Inspired by this, the AI research community attempts to develop grounded interactive embodied agents that are capable of engaging in natural back-and-forth dialog with humans to assist them in completing real-world tasks. Notably, the agent needs to understand when to initiate feedback requests if communication fails or instructions are not clear and requires learning new domain-specific vocabulary.
Despite all these efforts, the task is far from solved.
For that reason, we propose the IGLU competition, which stands for Interactive Grounded Language Understanding (IGLU) in a collaborative environment.
Specifically, the goal of our competition is to approach the following scientific challenge:
How to build interactive embodied agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment?
By "interactive agent" we mean that the agent can: (1) follow the instructions correctly, (2) ask for clarification when needed, and (3) quickly adapt newly acquired skills. The IGLU challenge is naturally related to two fields of study that are highly relevant to the NeurIPS community: Natural Language Understanding and Generation (NLU / NLG) and Reinforcement Learning (RL).
Please consider IGLU NeurIPS 2022 proposal for a more detailed description of the task and application scenario.