Non-Intrusive Data Collection for Load Balancing of Parallel Applications

Abstract: It is highly desirable to have a tool to support dynamic load balancing for parallel computation in a large network of heterogeneous computers. Many schedulers and mid-wares developed to date can only support dynamic load balancing in homogeneous computation environments. Although our previously developed scheduler and load balancer can support heterogeneous environment, they are only applicable to domain decomposition based parallel applications. The main problem of the slow progress in the development of general load balancing tool is due to the difficulty of obtaining the needed information of the parallel application for load balancing. This paper describes new application execution information gathering methods that use an MPI profiler or use an available system tool, PROC. PROC is supported by some versions of UNIX operating systems and Linux. These two new methods are different from our previously used time stamp method that inserts time measuring library functions into the application code. These new execution gathering approach can support dynamic load balancing with general parallel applications and support network of heterogeneous computers. This new approach also makes a lot convenient for user to take the advantage of dynamic load balancing.