StarCraft as a Testbed for Engineering Complex Distributed Systems Using Cognitive Agent Technology

It has been argued that the evaluation of cognitive agent systems requires richer benchmark problems. We think that real-time strategy (RTS) games can offer such a testbed, as AI for RTS requires the design of complicated strategies for coordinating hundreds of units that need to solve a range of challenges. Therefore, in this paper, we report on the design and development of the first multi-agent connector that provides full access to StarCraft (Brood War). We provide a new interface that is dedicated to a multi-agent approach by connecting each unit in the game to a cognitive agent. Two main challenges are addressed in this work. First, we decide on the right level of abstraction for unit control by means of agents, designing for instance the percepts that are available to units. Second, a sufficient level of performance needs to be ensured in order to allow a large variety of multi-agent implementations to be successful at tackling challenges of RTS AI. The resulting open-source connector readily supports the hundreds of agents that can come and go during the game. Based on the development of the connector and its initial use by over 200 students, we gained valuable insights.