Individual Classifiers Accuracy Based Online Boosting Algorithm
DOI:
https://doi.org/10.14419/vq3n9107Published
28-05-2026Keywords:
Concept Drift; Hellinger Distance; Data Stream; Online Learning; Hoeffding BoundAbstract
Boosting is a method for transforming weak learners into a powerful ensemble. To model this technique, classifiers are trained on several datasets. This article proposes a boosting-like online learning algorithm, a modified form of Oza and Russell's online boosting, aimed at maintaining high and consistent accuracy in environments where concepts change continuously. We use each classifier's accuracy to trans-form weak learners into strong learners. Our algorithm was tested against other state-of-the-art methods using several real and artificial da-tasets. Accuracy improved drastically in most tested situations.
References
I. Baidari and N. Honnikoll, “Bhattacharyya distance based concept drift detection method for evolving data stream,” Expert Syst. Appl., vol. 183, p. 115303, 2021, https://doi.org/10.1016/j.eswa.2021.115303.
R. S. M. Barros, D. R. L. Cabral, P. M. Gonçalves, and S. G. T. C. Santos, “RDDM: Reactive drift detection method,” Expert Syst. Appl., 2017, https://doi.org/10.1016/j.eswa.2017.08.023.
J. Gama, P. Medas, G. Castillo, and P. P. Rodrigues, “Learning with drift detection,” in Advances in Artificial Intelligence – SBIA 2004, vol. 3171, Berlin, Heidelberg: Springer Berlin Heidelberg, 2004, pp. 286–295. doi: 10.1007/978-3-540-28645-5_29.
L. Hu, Y. Lu, and Y. Feng, “Concept Drift Detection Based on Deep Neural Networks and Autoencoders,” Applied Sciences, vol. 15, no. 6, 2025, https://doi.org/10.3390/app15063056.
R. S. M. de Barros and S. G. T. de C. Santos, “An overview and comprehensive comparison of ensembles for concept drift,” Information Fusion, 2019, https://doi.org/10.1016/j.inffus.2019.03.006.
Y. Freund and R. E. Schapire, “Experiments with a New Boosting Algorithm,” Proceedings of the 13th International Conference on Machine Learn-ing, pp. 148–156, 1996.
N. C. Oza and S. J. Russell, “Online bagging and boosting,” in International workshop on artificial intelligence and statistics, 2001, pp. 229–236.
S. T. Chen, H. T. Lin, and C. J. Lu, “An online boosting algorithm with theoretical justifications,” in Proceedings of the 29th International Confer-ence on Machine Learning, ICML 2012, 2012, pp. 1007–1014.
A. Bifet, G. Holmes, B. Pfahringer, R. Kirkby, and R. Gavaldà, “New ensemble methods for evolving data streams,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2009. https://doi.org/10.1145/1557019.1557041.
A. Bifet, G. Holmes, and B. Pfahringer, “Leveraging bagging for evolving data streams,” in Machine Learning and Knowledge Discovery in Data-bases, vol. 6321, Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 135–150. https://doi.org/10.1007/978-3-642-15880-3_15.
G. J. Ross, N. M. Adams, D. K. Tasoulis, and D. J. Hand, “Exponentially weighted moving average charts for detecting concept drift,” Pattern Recognit. Lett., vol. 33, no. 2, pp. 191–198, Jan. 2012, https://doi.org/10.1016/j.patrec.2011.08.019.
J. Z. Kolter and M. A. Maloof, “Dynamic weighted majority: An ensemble method for drifting concepts,” Journal of Machine Learning Research, vol. 8, pp. 2755–2790, 2007.
L. L. Minku and X. Yao, “DDD: A new ensemble approach for dealing with concept drift,” IEEE Trans. Knowl. Data Eng., vol. 24, no. 4, pp. 619–633, 2012, https://doi.org/10.1109/TKDE.2011.58.
M. Baena-García, J. del Campo-Ávila, R. Fidalgo, A. Bifet, R. Gavaldà, and R. Morales-Bueno, “Early drift detection method,” in Fourth interna-tional workshop on knowledge discovery from data streams, 2006, pp. 77–86.
D. Brzezinski and J. Stefanowski, “Reacting to different types of concept drift: The accuracy updated ensemble algorithm,” IEEE Trans. Neural Netw. Learn. Syst., vol. 25, no. 1, pp. 81–94, 2014, https://doi.org/10.1109/TNNLS.2013.2251352.
S. G. T. D. C. Santos, P. M. Gonçalves, G. D. D. S. Silva, and R. S. M. De Barros, “Speeding up recovery from concept drifts,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2014, pp. 179–194. https://doi.org/10.1007/978-3-662-44845-8_12.
R. S. M. De Barros, S. G. T. De Carvalho Santos, and P. M. G. Junior, “A Boosting-like Online Learning Ensemble,” in Proceedings of the Interna-tional Joint Conference on Neural Networks, 2016, pp. 1871–1878. https://doi.org/10.1109/IJCNN.2016.7727427.
S. G. T. de Carvalho Santos and R. S. M. de Barros, “Online adaboost-based methods for multiclass problems,” Artif. Intell. Rev., vol. 53, no. 2, pp. 1293–1322, 2020. https://doi.org/10.1007/s10462-019-09696-6.
I. Fr’ias-Blanco, J. del Campo-Ávila, G. Ramos-Jiménez, R. Morales-Bueno, A. Ortiz-D’iaz, and Y. Caballero-Mota, “Online and non-parametric drift detection methods based on Hoeffding’s bounds,” IEEE Trans. Knowl. Data Eng., vol. 27, no. 3, pp. 810–823, 2015. https://doi.org/10.1109/TKDE.2014.2345382.
A. L. Freire, G. A. Barreto, M. Veloso, and A. T. Varela, “Short-term memory mechanisms in neural network learning of robot navigation tasks: A case study,” in 2009 6th Latin American Robotics Symposium, LARS 2009, 2009. https://doi.org/10.1109/LARS.2009.5418323
S. Zhong, W. Tang, and T. M. Khoshgoftaar, “Boosted noise filters for identifying mislabeled data,” Department of Computer Science and engineer-ing, Florida Atlantic University, 2005.
R. D. King, C. Feng, and A. Sutherland, “Statlog: Comparison of classification algorithms on large real-world problems,” Applied Artificial Intelli-gence, 1995, https://doi.org/10.1080/08839519508945477.
A. Vergara, S. Vembu, T. Ayhan, M. A. Ryan, M. L. Homer, and R. Huerta, “Chemical gas sensor drift compensation using classifier ensembles,” Sens. Actuators B Chem., vol. 166–167, no. 3, pp. 320–329, May 2012, https://doi.org/10.1016/j.snb.2012.01.074.
J. Demšar, “Statistical comparisons of classifiers over multiple data sets,” Journal of Machine Learning Research, 2006.