QuickSet: A Multimodal Interface for Distributed Interactive Simulation

We demonstrate QuickSet, a wireless, handheld, collaborative, system that can be used to control distributed interactive simulations based on the ModSAF simulator [3] and a 3-D terrain visualization system called CommandVu. Together with the CommandTalk spoken interaction component from SRI International [4] these form the LeatherNet system [1]. With QuickSet, users can formulate a military scenario by creating, positioning, and editing units, supplying them with behavior, specifying fortifications, objectives and other points, etc. In contrast to the original ModSAF GUI, users of QuickSet can employ multiple input modalities, including speech, gesture, and direct manipulation, to suit the task and situation at hand. QuickSet operates on a 3-lb. handheld 100MHz 486 PC (Fujitsu Stylistic 1000), as well as desktop PC's, employing wireless LAN communications, color screen, microphone, pen stylus, onboard speech recognition, gesture recognition, natural language processing, and multimodal integration. These components communicate with one and the other components of LeatherNet via the Open Agent Architecture [2]. Figure 1 shows collaborative use of the system for creating units, areas, lines, and minefields. QuickSet can run standalone on the handheld, with a "stub" component emulating ModSAF, and can under user control, upload its scenario to ModSAF to be simulated. Other users can do likewise, with the net effect being an accumulation of entitites and behaviors. For example, this capability can be used for one party playing "blue" and the another "red." When connected online, the handhelds track the ongoing simulation, and therefore are collaborative, in that each can see and effect the course of the simulation. Users can also couple their handheld systems, so that panning and zooming of one will cause identical activity on the others.