Verbal control of mathematical tools for simulation and virtual environments

This paper describes a method for verbal control of mathematical tools such as Computer Algebra applications for use in Simulation. This method is applied to the Maple 12 user interface, the SAGE notebook interface and the MuPad interface. The method is a general method for verbal control of mathematical tools for simulation based upon human factors design principles. The method illustrates the design principles for verbal control of User Interfaces. The method can utilize discrete speech or continuous speech recognition for command and control of GUI's. The design method has a "discovery" scheme that allows users to determine how to script and use voice to control commercial applications such as Maple, MuPad and the SAGE notebook. This paper focuses on the Macintosh OSX operating system since the control features of OSX simplify voice and scripting control of tools for simulation. The discovery scheme can be applied to other applications with an Objective-C based GUI to enable automation and voice control. Many mathematical tools are applicable to distributed simulations using a Task Characteristics human factors approach to introduce tool results into simulations within virtual environments. The voice control scheme described in this paper provides a novel and superior methodology for voice control of mathematical tools for use in simulation and virtual environments.