Removal of outliers in geomagnetic field time series using the Hampel filter

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription or Fee Access

Abstract

The paper presents a methodology for removing outliers in geomagnetic field time series using the Hampel filter. Quality metrics for binary data classification based on the confusion matrix demonstrated the effectiveness of the method and are comparable to those for similar algorithms used for outlier removal in 1-second magnetograms of the international INTERMAGNET network. Outliers identified using the developed methodology for the period 2019-2022 at the Ak-Suu base station exhibit a seasonal distribution pattern that correlates well with thunderstorm activity. The method enhances the quality of preliminary data processing for the geomagnetic monitoring network of the Research Station of the Russian Academy of Sciences, specifically by automating the procedure of magnetograms outliers filtering.

About the authors

S. A. Imashev

Research Station of the Russian Academy of Sciences in Bishkek

Author for correspondence.
Email: sanzhar.imashev@gmail.com

Ph.D., Leading Researcher

Kyrgyzstan, Bishkek

E. A. Lazareva

Research Station of the Russian Academy of Sciences in Bishkek

Email: ekaterina.lazareva88@gmail.com

Junior Researcher

Kyrgyzstan, Bishkek

References

  1. Mukhamadeeva V. А. Vorontsova E. V., Lazareva E. A. Experience of geomagnetic observations at the Geodynamic Test Ground in Bishkek, Herald of KRSU, 2015, vol. 15, no. 3, pp. 130—133 (in Russian).
  2. Bogoutdinov Sh.R., Gvishiani A. D., Agayan S. M., Solovyev A. A., Kin E. Recognition of disturbances with specified morphology in time series. Part 1: Spikes on magnetograms of the worldwide INTERMAGNET network, Izvestiya, Physics of the Solid Earth, 2010, vol. 46, no. 11, pp. 1004—1016.
  3. Solovyev A. A., Agayan S. M., Gvishiani A. D., Bogoutdinov Sh. R., Chulliat A. Recognition of disturbances with specified morphology in time series: Part 2. Spikes on 1-s magnetograms, Izvestiya, Physics of the Solid Earth, 2012, vol. 48, no. 5, pp. 395—409.
  4. Imashev S. Extended isolation forest application to outlier detection in geomagnetic data, IOP Conference Series: Earth and Environmental Science, 2021, vol. 929, no. 012022, pp. 1—6.
  5. Chandola V., Banerjee A., Kumar V. Anomaly detection: A survey, ACM Comput. Surv, 2009, vol. 41, no. 3, pp. 1—58, doi: 10.1145/1541880.1541882.
  6. Hampel F. R. The influence curve and its role in robust estimation, Journal of the American Statistical Association, 1974, vol. 69, pp. 382—393.
  7. Liu H., Sirish S., Wei J. On-line outlier detection and data cleaning, Computers and Chemical Engineering, 2004, vol. 28, pp. 1635—1647, doi: 10.1016/j.compchemeng.2004.01.009.
  8. Pearson R. K. Outliers in process modeling and identification, IEEE Transactions on Control Systems Technology, 2002, vol. 10, no. 1, pp. 55—63, doi: 10.1109/87.974338.
  9. Leys C., Ley C., Klein O., Bernard P., Licata L. Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median, Journal of Experimental Social Psychology, 2013, vol. 49, iss. 4, pp. 764—766, doi: 10.1016/j.jesp.2013.03.013.
  10. Imashev S. A., Lazareva E. A. Spatial distribution of the main geomagnetic field components based on IGRF—13 model for Kyrgyzstan territory, Herald of KRSU, 2022, vol. 22, no 4, pp. 192—198 (in Russian).
  11. Imashev S. A., Lazareva E. A. Program for outlier removal in geomagnetic field variation time series using the Hampel filter: MagHam- pelOutlierCut, Federal Service for Intellectual Property (Rospatent). Certificate of Registration of Computer Program no. 2022684573, 15.12.2022, available at: https://www.elibrary.ru/item.asp?id=49979520.
  12. Podrezova Yu. A. The annual course of the recurrence of thunderstorms in Kyrgyzstan, Herald of KRSU, 2010, vol. 10, no. 4, pp. 150—153.
  13. Campbell W. H. Introduction to Geomagnetic Fields, Cambridge University Press, 2003, 337 p.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Copyright (c) 2025 Informacionnye Tehnologii


СМИ зарегистрировано Федеральной службой по надзору в сфере связи, информационных технологий и массовых коммуникаций (Роскомнадзор).
Регистрационный номер и дата принятия решения о регистрации СМИ: серия ПИ № 77 - 15565 от 02 июня 2003 г.