Optimized Common Parameter Set Extraction by Benchmarking Applications on a Big Data Platform

This research proposes the methodology to extract common configuration parameter set by applying multiple benchmark applications including TeraSort., TestDFSIO, and MrBench on the Hadoop Distributed File System. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple tasks. In this way, we have found several parameter value sets that can reduce the execution time by 27% of the values provided by Hadoop default.