Enhancing hadoop performance in homogeneous big data environment assuming configuration of dynamic slots in map-reduce pattern - M. Ali Enfais, Abolgasem; Amril Jaharadak, Adam; Abdelkarim El Marghani, Amad; Saleh Alsusaa, Abdulbasat (International Journal of Engineering and Technology, 2019)
Hadoop is a Java-based programming framework that supports the storing and processing of large data sets in a distributed computing environment and it is very much appropriate for high volume of data. It uses HDFS for data storing and uses MapReduce for processing that data. MapReduce is a popular programming model to support data-intensive applications using shared-nothing clusters. The main objective of MapReduce programming model is to parallelize the job execution across multiple nodes for execution. Nowadays, all focus of the researchers and companies toward to Hadoop. Due to this, many scheduling algorithms have been proposed in the past decades. There are three important scheduling issues in MapReduce such as locality, synchronization and fairness. The most common objective of scheduling algorithms is to minimize the completion time of a parallel application and also achieve to these issues. Performance issues are introduced for Hadoop schedulers, and comparative performance analysis between different cases of jobs submission. These jobs are processed in different homogenous data environment and, under fixed or reconfigurable slot between map and reduce tasks for Hadoop MapReduce java programming clustering model. The results showed that when assigning tunable knob between map and reduce tasks under certain scheduler like FIFO algorithm, the performance enhanced 16.66% in inverted index, 55.55% in word count and 11.76% in classification process.