Evaluating object permanence in embodied agents using the animal-AI environment

Abstract

Object permanence, the understanding and belief that objects continue to exist even when they are not directly observable, is important for any agent interacting with the world. Psychologists have been studying object permanence in animals for at least 50 years, and in humans for almost 50 more. In this paper, we apply the methodologies from psychology and cognitive science to present a novel testbed for evaluating whether artificial agents have object permanence. Built in the Animal-AI environment, Object-Permanence In Animal-Ai: GEneralisable Test Suites (O-PIAAGETS) improves on other benchmarks for assessing object permanence in terms of both size and validity. We discuss the layout of O-PIAAGETS and how it can be used to robustly evaluate OP in embodied agents.