A modern approach on map reduce cluster using dynamic slot allocation optimization framework

Emerging technology and effective mechanism oriented approach makes Map Reduce a popular as well as important computing paradigm which dealing with big data process in cloud computing. The mechanism of un-optimized resource allocation is the severe drawback for slot based Map Reduce System. To overcome the failures of the optimize resource allocation we identify three aspect in this process in order to improve the resource allocation in an effective manner. The initial one is pre-configuring the map slots by reduce the slots which are under-utilized and fungible. The key behind these methods is it makes the map slots fully utilized which reduces the empty slots and vice-versa. We deal this process with a proposed advanced technique known as Dynamic Hadoop Slot Allocation. It can be achieved by implementing the slot based model in this proposed mechanism. It makes every slot reallocated either on map or minimizing the task based on the requirements. The next things are handling the stagger problem. It can be involved in making every single job highly impact without affecting the efficiency of the clusters. Our proposed Speculative Execution Performance Balancing is developed with the motto of building a tradeoff between the single jobs with the batch of jobs. Th third one in our proposed approach is dealing with delay scheduling that makes the data locality in a better way without raising the cost. Based on the above mentioned methodology a combined approach enables a step-by-step slot allocation by the proven name of DynamicMR that shows the massive impact in the working of Map Reduce subsequently dealing with their workloads. The performance of DynamicMR shows high impact on Hadoop MRv1, by examining the outputs achieving the accuracy of 46% ∼ 115% gradually on single jobs as well as 49% ∼ 112% effectiveness on multiple jobs. To justify the result in technology aspect our proposed work is compared with YARN experiments that's shows the DynamicMR performance 2% ∼9% on multiple jobs as the betterment of ratio control mechanism based on the current map reduce approach.