A new method to quickly identify outliers in air quality monitoring data

The PM2.5 monitoring instruments at State Key Laboratory of Atmospheric Boundary Layer Physics and Atmospheric Chemistry (LAPC), Institute of Atmospheric Physics, Chinese Academy of Sciences. Image by TANG Xiao

In practice, manual inspection is often applied to identify these outliers. However, as the amount of data grows rapidly, this method becomes increasingly cumbersome.

To deal with the problem, Dr. WU Huangjian and Associate Professor TANG Xiao from the Institute of Atmospheric Physics, Chinese Academy of Sciences, propose a fully automatic outlier detection method based on the probability of residuals.

The method adopts multiple regression methods, and the regression residuals are used to discriminate outliers. Based on the standard deviations of the residuals, probabilities of the residuals can be calculated, and the observations with small probabilities are tagged as outliers and removed by a computer program. Their findings are published in Advances in Atmospheric Sciences.

“By introducing the probabilities of residuals, multiple rules can be used for identifying outliers on the same framework,” says Dr. Wu.

“For example, by assuming that the residuals of spatial regression and temporal regression obey a bivariate normal distribution, spatial and temporal consistencies can be simultaneously evaluated for better identification of outliers”.

The method can flag potentially erroneous data in the hourly observations from 1436 stations of the China National Environmental Monitoring Center (CNEMC) within a minute. Indeed, it has been used in CNEMC's air quality forecasting system, and is going to be integrated into the data management system. The hope is that outliers in the system's real-time air quality data will be removed in the near future.

The method is published in Advances in Atmopheric Sciences.

Media Contact

Zheng Lin
jennylin@mail.iap.ac.cn
86-108-299-5053

 @aasjournal

http://english.iap.cas.cn/ 

Alle Nachrichten aus der Kategorie: Earth Sciences

Earth Sciences (also referred to as Geosciences), which deals with basic issues surrounding our planet, plays a vital role in the area of energy and raw materials supply.

Earth Sciences comprises subjects such as geology, geography, geological informatics, paleontology, mineralogy, petrography, crystallography, geophysics, geodesy, glaciology, cartography, photogrammetry, meteorology and seismology, early-warning systems, earthquake research and polar research.

Zurück zur Startseite

Kommentare (0)

Schreib Kommentar

Neueste Beiträge

Cyanobacteria: Small Candidates …

… as Great Hopes for Medicine and Biotechnology In the coming years, scientists at the Chair of Technical Biochemistry at TU Dresden will work on the genomic investigation of previously…

Do the twist: Making two-dimensional quantum materials using curved surfaces

Scientists at the University of Wisconsin-Madison have discovered a way to control the growth of twisting, microscopic spirals of materials just one atom thick. The continuously twisting stacks of two-dimensional…

Big-hearted corvids

Social life as a driving factor of birds’ generosity. Ravens, crows, magpies and their relatives are known for their exceptional intelligence, which allows them to solve complex problems, use tools…

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close