Analysis of Information Management and Scheduling Technology in Hadoop
Ma Weihua, Zhang Hong, Li Qianmu, Xia Bin
Journal of Digital Information Management, Vol. 12, No. 2, 2014.
DOI: 
Link: http://dline.info/fpaper/jdim/v12i2/9.pdf
Abstract:
Development of big data computing has brought many changes to society and social life is constantly digitized. How to handle vast amounts of data" has become a more and more fashionable topic. Hadoop is a distributed computing software framework, which includes HDFS and MapReduce distributed computing method, make distributed processing huge amounts of data possible. Then job scheduler determines the efficiency of Hadoop clusters and user experience. Under the premise of familiar with the mechanism of Hadoop's running tasks, make a full analysis of the existing Hadoop task scheduling algorithm, such as FIFO-Scheduler, Capacity Scheduler, FairShare-Scheduler and LATE-Scheduler, found that the existing scheduling algorithms do not perceive the performance of the computing node, so that it cannot assign different tasks depending on the machine performance in heterogeneous Hadoop cluster.