Map reduce technique for parallel-automata analysis of large scale rainfall data
Abstract
Vast scale rainfall information assumes an imperative part in farming field thus early expectation of rainfall is important for the better finan-cial development of a nation. Rainfall expectation is an expert among the most troublesome issue far and wide in a year back. This data is generally secured in the unstructured course of action. Along these, tremendous measure of data has been accumulated and archived. Thus, storage and handling of such tremendous information for accurate rainfall forecast are a major test. Big Data innovation like Hadoop have developed to fathom the difficulties and issues of huge information utilizing distributed computing. Till date few examinations have been accounted for on the preparing of vast scale rainfall information utilizing MapReduce. In this paper, the huge scale rainfall information is anticipated by utilizing MapReduce system which plays out the capacities which are required and diminishes the task to get proficient ar-rangements through taking the information and isolating into smaller tasks. At that point, the three Regression Automata (RA) algorithms such as Linear Regression automata, Support Vector Regression Automata and Logistic Regression Automata. are utilized to forecast the future esteem of large scale rainfall data. The proposed framework serves as a tool that takes in the rainfall information from diminished information as input and predicts the future rainfall. The outcomes obviously demonstrate that the all the three RA models can anticipate the rainfall productively in different terms, such as, error rate, coefficients and mean square error.
About this article
Keywords
- Big Data
- Hadoop
- Map Reduce
- Rainfall Forecast
- Regression Automata.
References
[1] N. Sethi, and K. Garg, “Exploiting data mining technique for rainfall prediction”, International Journal of Computer Science and Information Technologies, Vol.5, No.3, pp.3982-3984, 2014.
[2] P.S. Dutta, and H. Tahbilder, “Prediction of rainfall using data mining technique over Assam”, Indian Journal of Computer Science and Engineering (IJCSE), Vol.5, No.2, pp.85-90, 2014.
[3] M. Kannan, S. Prabhakaran, and P. Ramachandran, “Rainfall forecasting using data mining technique”, International Journal of Engineering and Technology, Vol.2, No.6, pp.397-401, 2010.
[4] A. Gautam, and P. Bedi, “MR-VSM: Map Reduce based vector Space Model for user profiling-an empirical study on News data”, In International Conference on Advances in Computing, Communications and Informatics (ICACCI), Kochi, pp. 355-360, 2015. https://doi.org/10.1109/ICACCI.2015.7275635.
[5] A. Gautam, R. Dhingra, and P. Bedi, “Use of NoSQL database for handling semi structured data: an empirical study of news RSS feeds”, Emerging Research in Computing, Information, Communication and Applications, New Delhi, pp.253-263, 2015.
[6] S. Navadia, P. Yadav, J. Thomas, and S. Shaikh, “Weather prediction: A novel approach for measuring and analyzing weather data,” In International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC), pp.414-417, 2017.
[7] D. V. Sahasrabuddhe, and P. Jamsandekar. "Data structure for representation of big data of weather forecasting: a review." International Journal of Computer Science Trends and Technology (IJCST) vol, 3, no. 6, pp. 48-56, 2015.
[8] V. Dagade, M. Lagali, S. Avadhani, P. Kalekar, “Big Data Weather Analytics Using Hadoop”, International Journal of Emerging Technology in Computer Science & Electronics (IJETCSE), Vol.14 No.2, 2015.
[9] L. Li, F. Noorian, D.J. Moss, and P.H. Leong, “Rolling window time series prediction using MapReduce,” In 15th International Conference on Information Reuse and Integration (IRI), pp.757-764, 2014.
[10] Y. Chen, Z. Wu, Z. Li, and Y. Zhang, “Research on time series forecasting model based on moore automata”, In Proc. of International Conf. On Advanced Data Mining and Applications, pp.98-105, Springer, Berlin, Heidelberg, 2010. https://doi.org/10.1007/978-3-642-17316-5_9.
[11] S. Mehrmolaei, and M.R. Keyvanpour, “Time series forecasting using improved ARIMA”, In Proc. of International Conf. On Artificial Intelligence and Robotics (IRANOPEN), pp.92-97 2016.
[12] R.M. Nabilah, Z. Othman, and B. A. Azuraliza, “Approaches of Handling Uncertain Time Series Data towards Prediction”, International Journal of Future Computer and Communication, Vol.5, No.6, pp.233, 2016. https://doi.org/10.18178/ijfcc.2016.5.6.477.
[13] N.F.M. Radzuan, Z. Othman, and A.A. Bakar, “Uncertain time series in weather prediction”, Procedia Technology, Vol.11, pp.557-564, 2013. https://doi.org/10.1016/j.protcy.2013.12.228.
[14] M. Joshi, S. Shaikh, P. Waghmode, and P. Mali, “Farmer Buddy-Weather Prediction and Crop Suggestion using Artificial Neural Network on Map-Reduce Framework”, International Journal of Computer Applications, Vol.159, No.7, 2017.
[15] K.A. Ismail, M.A. Majid, J.M. Zain, and N.A.A. Bakar, “Big Data prediction framework for weather Temperature based on MapReduce algorithm”, In Open Systems (ICOS), 2016 IEEE Conference, pp.13-17, 2016.
[16] K. Namitha, A. Jayapriya, and G. Santhosh Kumar, “Rainfall Prediction using Artificial Neural Network on Map-Reduce Framework,” Proceedings of the Third International Symposium on Women in Computing and Informatics. ACM, 2015.
[17] C. P. Shabariram, K. E. Kannammal, and T. Manojpraphakar, "Rainfall analysis and rainstorm prediction using MapReduce Framework." Computer Communication and Informatics (ICCCI), 2016 International Conference on. IEEE, 2016.
[18] A. Nair, Gurjeet Singh, and U. C. Mohanty. "Prediction of Monthly Summer Monsoon Rainfall Using Global Climate Models Through Artificial Neural Network Technique." Pure and Applied Geophysics, vol. 175, no. 1, pp. 403-419, 2018. https://doi.org/10.1007/s00024-017-1652-5.
[19] S. Shajitha Banu, S. Manjula, S. Swathi Priya, V. Yamuna Devi, M. Thangamani, “Predictive Analysis of Rainfall Data to Help the Farmers”, International Journal of Advanced Research in Computer Science and Software Engineering, Vol.6, No.3, 2016.
[20] A. Kaur, “Big Data: A Review of Challenges, Tools and Techniques”, International journal of scientific research in science, engineering and technology, Vol.2, No.2, pp. 1090-1093, 2016.
[21] K. Morton, M. Balazinska and D. Grossman, “Paratimer: a progress indicator for MapReduce DAGs”, In Proceedings of the 2010 international conference on Management of data, pp.507-518, 2010.
[22] W. Lu, Y. Shen, S. Chen, and B.C. Ooi, “Efficient processing of k nearest neighbor joins using mapreduce”, Proceedings of the VLDB Endowment, Vol.5, No.10, pp.1016-1027, 2012. https://doi.org/10.14778/2336664.2336674.
[23] P. Riyaz, and S.M. Varghese, “Leveraging map reduce with hadoop for weather data analytics”, IOSR Journal of Computer Engineering, Vol.17, No.3, 2015.
[24] B. Anurag, M. Prakash, V. Kanna, P. Choudhary, “Weather Forecasting using Map-Reduce”, International Journal of Innovative Research in Computer and Communication Engineering, Vol.5, No.9, 2017.
[25] G.J. Martínez, J.C. Seck-Tuoh-Mora, and H. Zenil, “Wolfram’s classification and computation in cellular automata Classes III and IV”, In Proc. of International Conf. On Irreducibility and Computational Equivalence, Berlin, Heidelberg, pp.237-259, 2013. https://doi.org/10.1007/978-3-642-35482-3_17.
[26] A. Kavousi-Fard, H. Samet, and F. Marzbani, “A new hybrid modified firefly algorithm and support vector regression model for accurate short term load forecasting”, Expert systems with applications, Vol.41, No.13, pp.6047-6056, 2014. https://doi.org/10.1016/j.eswa.2014.03.053.
[27] A. Belayneh, J. Adamowski, B. Khalil, and B. Ozga-Zielinski, “Long-term SPI drought forecasting in the Awash River Basin in Ethiopia using wavelet neural network and wavelet support vector regression models”, Journal of Hydrology, Vol.508, pp.418-429, 2014. https://doi.org/10.1016/j.jhydrol.2013.10.052.
[28] A.R. Imon, M.C. Roy, and S.K. Bhattacharjee, “Prediction of rainfall using logistic regression”, Pakistan Journal of Statistics and Operation Research, Vol.8, No.3, pp.655-667, 2012. https://doi.org/10.18187/pjsor.v8i3.535.
[29] Geetha, A. and Nasira, G.M., 2016. Time-series modelling and forecasting modelling of rainfall prediction using ARIMA model. International Journal of Society Systems Science, 8(4), pp.361-372. https://doi.org/10.1504/IJSSS.2016.081411.