FasTensor Programming Model

In the previous chapter, we have introduced the motivation for a big data analysis system and its essential components: data model and programming model. We have also clearly stated the reasons why FasTensor chooses the multi-dimensional array as its data model. In this chapter, we continue to provide more details on the FasTensor’s programming model using the multi-dimensional array data model. The crucial constructs of a programming model for the data analysis include an abstract data type and a set of generic operators. The abstract data type allows users to define input and output data structures that their data analysis functions use. The abstract data type is defined on the top of the array data model. The set of generic operators should allow users to formulate a workflow with a wide range of data analysis functions. Through the abstract data type and these generic operators, a standard protocol between users and a data analysis system is established. On one hand, users can format their data with the abstract data type and express their data analysis with these generic operators. On the other hand, based on the abstract data type and these generic operators, the data analysis system can easily build its functions for generic data management functions, parallelization, and other tasks.