Why and how to study the impact of perception on language emergence in artificial agents

Abstract

The study of emergent languages in deep multi-agent simulations has become an important research field. While targeting different objectives, most studies focus on analyzing properties of the emergent language—often in relation to the agents' inputs—ignoring the influence of the agents' perceptual processes. In this work, we use communication games to investigate how differences in perception affect emergent language. Using a conventional setup, we train two deep reinforcement learning agents, a sender and a receiver, on a reference game. However, we systematically manipulate the agents' perception by enforcing similar representations for objects with specific shared features. We find that perceptual biases of both sender and receiver influence which object features the agents' messages are grounded in. When uniformly enforcing the similarity of all features that are relevant for the reference game, agents perform better and the emergent protocol better captures conceptual input properties. © Cognitive Science Society: Comparative Cognition: Animal Minds, CogSci 2021.All rights reserved.