Loading...
A classification of hadoop job schedulers based on performance optimization approaches
Ghazali, R ; Sharif University of Technology | 2021
216
Viewed
- Type of Document: Article
- DOI: 10.1007/s10586-021-03339-8
- Publisher: Springer , 2021
- Abstract:
- Job scheduling in MapReduce plays a vital role in Hadoop performance. In recent years, many researchers have presented job scheduler algorithms to improve Hadoop performance. Designing a job scheduler that minimizes job execution time with maximum resource utilization is not a straightforward task. The primary purpose of this paper is to investigate agents affecting job scheduler efficiency and present a novel classification for job schedulers based on these factors. We provide a comprehensive overview of existing job schedulers in each group, evaluating their approaches, their effects on Hadoop performance, and comparing their advantages and disadvantages. Finally, we provide recommendations on choosing a preferred job scheduler in different environments for improving Hadoop performance. © 2021, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature
- Keywords:
- Computer networks ; Software engineering ; Job execution ; Job scheduler ; Job scheduling ; Map-reduce ; Performance optimizations ; Resource utilizations ; Scheduling
- Source: Cluster Computing ; Volume 24, Issue 4 , 2021 , Pages 3381-3403 ; 13867857 (ISSN)
- URL: https://link.springer.com/article/10.1007/s10586-021-03339-8