Google DeepMind Unveils SIMA: AI Agent Masters 3D Games Using Natural Language Instructions, India

New Delhi, March 14: Google DeepMind has made significant progress in the field of artificial intelligence with the introduction of SIMA, the Scalable Instructable Multiworld Agent. This AI agent is designed to understand and execute tasks in 3D games using natural language instructions, an achievement that represents a first in the field of AI capabilities. The SIMA AI agent is capable of moving through various gaming environments without the need for game source codes or APIs. Google DeepMind’s breakthrough with SIMA is expected to demonstrate the potential for AI agents to interact naturally with human users. Google DeepMind has unveiled the Scalable Instructable Multiworld Agent (SIMA), which showcases a new level of proficiency in performing diverse tasks across different 3D games. The SIMA AI Agent has been trained on nine distinct games, each offering a unique set of challenges and skills to be learned, such as navigation, item crafting, resource mining, and spaceship piloting. The performance of SIMA is evaluated based on its execution of hundreds of basic skills within short timeframes. These skills span several categories, including navigation commands like object interactions such as pick up mushrooms, turn right, and menu-based tasks, for instance, opening a map or crafting an item. Google DeepMind has also used four different research environments, one of which is named as the Construction Lab, developed using Unity. DeepMind acknowledges that this research is still in its early stages. The current focus is on enhancing SIMA’s performance, with the vision for future versions of the AI agent potentially including improving its understanding and capability to execute increasingly complex tasks.