Concept Drift Detection Method Based on Shifting Classifier's Mean Error Rate
About this article
Keywords:
Concept Drift; Hellinger Distance; Data Stream; Online Learning; Hoeffding BoundAbstract
Concept drift detection is an important technique for handling issues in real-time online learning. Learning in a real-time environment is challenging because data distribution is dynamic. This dynamic nature of data distribution affects the performance of the learning algorithm. Therefore, drift detection algorithms are used to handle such issues. In this article, we introduce a simple algorithm, the Shifting Classifier's Mean Error Rate (SCMER), which identifies the drift position using the classifier's mean error rate, computed from a range of instances (e.g., from 1 to n). The algorithm identifies the drift position when the deviation between two cumulative mean error rates is greater than a threshold value, ϵ. We proposed a Hellinger distance-inspired metric for measuring the deviation between two cumulative mean error rates. For hypothesis testing, we adapted the Hoeffding Bound. Experimental analysis shows that our algorithm detects drifts with shorter detection delays, fewer false positives, and fewer false negatives when compared to state-of-the-art methods.
References
[1] I. Khamassi, M. Sayed-Mouchaweh, M. Hammami, and K. Ghédira, “Discussion and review on evolving data streams and concept drift adapting,” Evolving Systems, 2018, https://doi.org/10.1007/s12530-016-9168-2.
[2] Y. K. Lee, L. Wang, and K. H. Ryu, “A system architecture for monitoring sensor data stream,” in 7th IEEE International Conference on Computer and Information Technology (CIT 2007), Aizu-Wakamatsu, Fukushima, Japan: IEEE, Oct. 2007, pp. 1026–1031. https://doi.org/10.1109/CIT.2007.178.
[3] I. Katakis, G. Tsoumakas, and I. Vlahavas, “Tracking recurring contexts using ensemble classifiers: An application to email filtering,” Knowl. Inf. Syst., vol. 22, no. 3, 2010, https://doi.org/10.1007/s10115-009-0206-2.
[4] J. Smailović, M. Grčar, N. Lavrač, and M. Žnidaršič, “Stream-based active learning for sentiment analysis in the financial domain,” Inf. Sci. (N. Y)., vol. 285, pp. 181–203, Nov. 2014, https://doi.org/10.1016/j.ins.2014.04.034.
[5] I. Žliobaitė, M. Pechenizkiy, and J. Gama, “An overview of concept drift applications,” in Big Data Analysis: New Algorithms for a New Society, vol. 16, Cham: Springer International Publishing, 2016, pp. 91–114. https://doi.org/10.1007/978-3-319-26989-4_4.
View more references (32)
[6] J. Gama, Knowledge discovery from data streams. CRC Press, 2010. https://doi.org/10.1201/EBK1439826119.
[7] A. Bifet, G. Holmes, R. Kirkby, and B. Pfahringer, “MOA: Massive Online Analysis,” Journal of Machine Learning Research, vol. 11, pp. 1601–1604, 2010, [Online]. Available: http://portal.acm.org/citation.cfm?id=1859903.
[8] J. Gama, P. Medas, G. Castillo, and P. P. Rodrigues, “Learning with drift detection,” in Advances in Artificial Intelligence – SBIA 2004, vol. 3171, Berlin, Heidelberg: Springer Berlin Heidelberg, 2004, pp. 286–295. https://doi.org/10.1007/978-3-540-28645-5_29.
[9] M. Baena-García, J. del Campo-Ávila, R. Fidalgo, A. Bifet, R. Gavaldà, and R. Morales-Bueno, “Early drift detection method,” in Fourth interna-tional workshop on knowledge discovery from data streams, 2006, pp. 77–86.
[10] A. Bifet and R. Gavaldà, “Learning from time-changing data with adaptive windowing,” in Proceedings of the 2007 SIAM International Conference on Data Mining, Society for Industrial and Applied Mathematics, Apr. 2007, pp. 443–448. https://doi.org/10.1137/1.9781611972771.42.
[11] K. Nishida and K. Yamauchi, “Detecting concept drift using statistical testing,” in International conference on discovery science, 2007, pp. 264–269. https://doi.org/10.1007/978-3-540-75488-6_27.
[12] E. S. Page, “Continuous inspection schemes,” Biometrika, vol. 41, no. 1/2, p. 100, Jun. 1954, https://doi.org/10.2307/2333009.
[13] S. H. Bach and M. A. Maloof, “Paired learners for concept drift,” in 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy: IEEE, Dec. 2008, pp. 23–32. https://doi.org/10.1109/ICDM.2008.119.
[14] G. J. Ross, N. M. Adams, D. K. Tasoulis, and D. J. Hand, “Exponentially weighted moving average charts for detecting concept drift,” Pattern Recognit. Lett., vol. 33, no. 2, pp. 191–198, Jan. 2012, https://doi.org/10.1016/j.patrec.2011.08.019.
[15] I. Fr’ias-Blanco, J. del Campo-Ávila, G. Ramos-Jiménez, R. Morales-Bueno, A. Ortiz-D’iaz, and Y. Caballero-Mota, “Online and non-parametric drift detection methods based on Hoeffding’s bounds,” IEEE Trans. Knowl. Data Eng., vol. 27, no. 3, pp. 810–823, 2015. https://doi.org/10.1109/TKDE.2014.2345382.
[16] A. Pesaranghader and H. L. Viktor, “Fast hoeffding drift detection method for evolving data streams,” in Lecture Notes in Computer Science (includ-ing subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016.
[17] R. S. M. Barros, D. R. L. Cabral, P. M. Gonçalves, and S. G. T. C. Santos, “RDDM: Reactive drift detection method,” Expert Syst. Appl., 2017, https://doi.org/10.1016/j.eswa.2017.08.023.
[18] M. Han, F. Meng, and C. Li, “Variance Feedback Drift Detection Method for Evolving Data Streams Mining,” Applied Sciences, vol. 14, no. 16, 2024, https://doi.org/10.3390/app14167157.
[19] T. M. Mitchell and others, “Machine learning. 1997,” Burr Ridge, IL: McGraw Hill, vol. 45, no. 37, pp. 870–877, 1997.
[20] J. S. Hunter, “The exponentially weighted moving average,” Journal of Quality Technology, vol. 18, no. 4, pp. 203–210, Oct. 1986, https://doi.org/10.1080/00224065.1986.11979014.
[21] D. R. de L. Cabral and R. S. M. de Barros, “Concept drift detection based on Fisher’s Exact test,” Inf. Sci. (N. Y)., vol. 442–443, pp. 220–234, 2018, https://doi.org/10.1016/j.ins.2018.02.054.
[22] R. S. M. de Barros, J. I. G. Hidalgo, and D. R. de Lima Cabral, “Wilcoxon Rank Sum Test Drift Detector,” Neurocomputing, vol. 275, pp. 1954–1963, 2018, https://doi.org/10.1016/j.neucom.2017.10.051.
[23] X. Yu, L. Ruan, J. S. Evans, and E. Wong, “Adaptive windowing-based concept drift detection and adaptation framework for human-to-machine ap-plications over future communication networks,” J. Opt. Commun. Netw., vol. 17, no. 4, pp. 338–351, Apr. 2025, https://doi.org/10.1364/JOCN.538964.
[24] H. Li and T. Zhao, “A dynamic similarity weighted evolving fuzzy system for concept drift of data streams,” Inf. Sci. (N. Y)., vol. 659, p. 120062, 2024, https://doi.org/10.1016/j.ins.2023.120062.
[25] L. Hu, Y. Lu, and Y. Feng, “Concept Drift Detection Based on Deep Neural Networks and Autoencoders,” Applied Sciences, vol. 15, no. 6, 2025, https://doi.org/10.3390/app15063056.
[26] B. I. F. Maciel, S. G. T. C. Santos, and R. S. M. Barros, “A Lightweight Concept Drift Detection Ensemble,” in 2015 IEEE 27th International Con-ference on Tools with Artificial Intelligence (ICTAI), Vietri sul Mare, Italy: IEEE, Nov. 2015, pp. 1061–1068. https://doi.org/10.1109/ICTAI.2015.151.
[27] F. Enembreck, B. C. Ávila, E. E. Scalabrin, and J.-P. Barthès, “Learning drifting negotiations,” Applied Artificial Intelligence, vol. 21, no. 9, pp. 861–881, 2007. https://doi.org/10.1080/08839510701526954.
[28] J. Gama, R. Sebastião, and P. P. Rodrigues, “On evaluating stream learning algorithms,” Mach. Learn., vol. 90, no. 3, pp. 317–346, Mar. 2013, https://doi.org/10.1007/s10994-012-5320-9.
[29] A. Bifet, J. Read, I. Zliobaite, B. Pfahringer, and G. Holmes, “Pitfalls in benchmarking data stream classification and how to avoid them,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2013, pp. 465–479. https://doi.org/10.1007/978-3-642-40988-2_30.
[30] A. Bifet, G. Holmes, B. Pfahringer, R. Kirkby, and R. Gavaldà, “New ensemble methods for evolving data streams,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2009. https://doi.org/10.1145/1557019.1557041.
[31] C. O. Sakar, S. O. Polat, M. Katircioglu, and Y. Kastro, “Real-time prediction of online shoppers’ purchasing intention using multilayer perceptron and LSTM recurrent neural networks,” Neural Comput. Appl., vol. 31, no. 10, pp. 6893–6908, 2019. https://doi.org/10.1007/s00521-018-3523-0.
[32] A. V Uzilov, J. M. Keegan, and D. H. Mathews, “Detection of non-coding RNAs on the basis of predicted secondary structure formation free energy change,” BMC Bioinformatics, vol. 7, no. 1, p. 173, 2006, https://doi.org/10.1186/1471-2105-7-173.
[33] B. Strack et al., “Impact of HbA1c measurement on hospital readmission rates: Analysis of 70,000 clinical database patient records,” Biomed Res. Int., vol. 2014, 2014, https://doi.org/10.1155/2014/781670.
[34] R. Kohavi, “Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid,” Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, vol. 7, no. Utgo 1988, 1996.
[35] J. Demšar, “Statistical comparisons of classifiers over multiple data sets,” Journal of Machine learning research, vol. 7, no. Jan, pp. 1–30, 2006.
[36] T. Fawcett, “An introduction to ROC analysis,” Pattern Recognit. Lett., vol. 27, no. 8, 2006, https://doi.org/10.1016/j.patrec.2005.10.010.
[37] B. W. Matthews, “Comparison of the predicted and observed secondary structure of T4 phage lysozyme,” BBA - Protein Structure, vol. 405, no. 2, 1975, https://doi.org/10.1016/0005-2795(75)90109-9.