Concept drift detection method for noisy big data streams based on least squares line fitting Online publication date: Fri, 12-Sep-2025
by Zhihong Qin; Zhanlei Shang; Fuhong Geng
International Journal of Internet Protocol Technology (IJIPT), Vol. 18, No. 3, 2025
Abstract: In order to improve the accuracy of concept drift detection in big data streams, a noisy big data streams concept drift detection method based on least squares line fitting is proposed. Firstly, the Kalman Filter model is used to identify and remove noise components from the original data stream, and then the wavelet thresholding algorithm is used to filter out the remaining noise. Secondly, by fitting the denoised data stream with a least squares line, linear trend features are extracted to simulate the trend of concept drift. Finally, the Tr-OEM algorithm is used to identify concept drift in big data streams, and an efficient and accurate concept drift detection is achieved through a machine learning model that dynamically adapts to data changes. The experimental results show that the detection accuracy of our method remains between 96.89% and 97.45%, and the maximum average change distance does not exceed 1.
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Internet Protocol Technology (IJIPT):
Login with your Inderscience username and password:
Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.
If you still need assistance, please email subs@inderscience.com
Our Blog 
Follow us on Twitter 
Visit us on Facebook