See, Hear, Explore: We propose a formulation of curiosity that encourages the agent to explore novel associations between modalities, such as audio and vision. In Habitat, the photorealistic navigation simulator shown above, our method allows for more efficient exploration than baselines. Many successes in reinforcement learning (RL) have come from agents maximizing a provided extrinsic reward such as a game score. However, in real-world settings such as robotics or autonomous driving, reward functions are hard to formulate and require significant…
