A hybrid and optimized resource scheduling technique using map reduce for larger instruction sets

  • Abstract
  • Keywords
  • References
  • PDF
  • Abstract

    MapReduce is a structural form to address larger-applications for handling tremendous data generated in parallel. These larger tasks is car-ried out by master and salve node architecture, where the master node judges all the available resources and manages the distributed applica-tions and the slave node is responsible to maintain the resources usability and conveys the information to the master node but the problem encountered is the varying of their resource which should be optimized. In today’s business applications resources optimization can either be solved as FIFO queues or using priority scheduler algorithms, thus supports both FIFO and priority algorithm by a concept known as PRISM i.e., Phase and Resources Information Aware Scheduler for map reduce. This incorporates implementation functionalities for both FIFO & priority algorithms named as hybrid algorithm where it optimizes the resources based on scenario of evaluation and parameters such as resource time, time to live and resource demand is considered. The importance of phase level scheduler is that it shows the resources usage variability with a particular time of a task. As a result the phase level scheduling algorithm will improve the execution parallelism and resources utilizations such that it ensures the data is not being lost or tampered



  • Keywords

    Map reduce; Optimization; Scheduling Algorithm; Resource Allocation

  • References

      [1] GridMix benchmark for Hadoop clusters [Online]. Available: http://hadoop.apache.org/docs/mapreduce/current/gridmix. Html, 2018.

      [2] Qi Zhang, Mohammed Faten Zhani, et.al, “PRISM: Fine Grained Resource-Aware Scheduling for MapReduce” IEEE Transitions on Cloud Computing, Vol 3, No.2, June 2015. Pp 182-194.

      [3] R. Boutaba, L. Cheng, and Q. Zhang, “On cloud computational models and the heterogeneity challenge,” J. Internet Serv. Appl., vol. 3, no. 1, pp. 1–10, 2012.

      [4] J. Dean and S. Ghemawat, “Mapreduce: Simplified data processing on large clusters,” Commun. ACM, vol. 51, no. 1, pp. 107–113, 2008.

      [5] J. Polo, C. Castillo, D. Carrera, Y. Becerra, I. Whalley, M. Steinder, J. Torres, and E. Ayguad_e, “Resource-aware adaptive scheduling for MapReduce clusters,” in Proc. ACM/IFIP/USENIX Int. Conf. Middleware, 2011, pp. 187–207.

      [6] A. Verma, L. Cherkasova, and R. Campbell, “Resource provisioning framework for MapReduce jobs with performance goals,” in Proc. ACM/IFIP/USENIX Int. Conf. Middleware, 2011, pp. 165–186.

      [7] Syed Thouheed Ahmed, Divya P Bhadrapur, Bibhu Prasad Mohanty, Lakshmi Bai G, Thanuja K, ”Tweeter Data Analysis Using Real time optimized sentimental machine learning algorithm using Hadoop” in International Journal of Pure and Applied Mathematics, Vol 118, No. 19, 2018, pp-2397-2406.

      [8] M. Zaharia, A. Konwinski, A. D. Joseph, R. H. Katz, and I. Stoica, “Improving MapReduce performance in heterogeneous environments,” in Proc. USENIX Symp. Oper. Syst. Des. Implementation, 2008, vol. 8, pp. 29–42.

      [9] Syed Thouheed Ahmed “A study on multi objective clustering technique for medical datasets” in Proc. IEEE International Conference of Intelligent Computing and Control Systems, 2017, pp 174-177.




Article ID: 15513
DOI: 10.14419/ijet.v7i2.33.15513

Copyright © 2012-2015 Science Publishing Corporation Inc. All rights reserved.