Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

A new method to quickly identify outliers in air quality monitoring data

30.10.2018

Ambient air quality monitoring data are the most important source for public awareness regarding air quality and are widely used in many research fields, such as improving air quality forecasting and the analysis of haze episodes. However, there are outliers among such monitoring data, due to instrument malfunctions, the influence of harsh environments, and the limitation of measuring methods.

In practice, manual inspection is often applied to identify these outliers. However, as the amount of data grows rapidly, this method becomes increasingly cumbersome.


The PM2.5 monitoring instruments at State Key Laboratory of Atmospheric Boundary Layer Physics and Atmospheric Chemistry (LAPC), Institute of Atmospheric Physics, Chinese Academy of Sciences.

Image by TANG Xiao

To deal with the problem, Dr. WU Huangjian and Associate Professor TANG Xiao from the Institute of Atmospheric Physics, Chinese Academy of Sciences, propose a fully automatic outlier detection method based on the probability of residuals.

The method adopts multiple regression methods, and the regression residuals are used to discriminate outliers. Based on the standard deviations of the residuals, probabilities of the residuals can be calculated, and the observations with small probabilities are tagged as outliers and removed by a computer program. Their findings are published in Advances in Atmospheric Sciences.

"By introducing the probabilities of residuals, multiple rules can be used for identifying outliers on the same framework," says Dr. Wu.

"For example, by assuming that the residuals of spatial regression and temporal regression obey a bivariate normal distribution, spatial and temporal consistencies can be simultaneously evaluated for better identification of outliers".

The method can flag potentially erroneous data in the hourly observations from 1436 stations of the China National Environmental Monitoring Center (CNEMC) within a minute. Indeed, it has been used in CNEMC's air quality forecasting system, and is going to be integrated into the data management system. The hope is that outliers in the system's real-time air quality data will be removed in the near future.

The method is published in Advances in Atmopheric Sciences.

Media Contact

Zheng Lin
jennylin@mail.iap.ac.cn
86-108-299-5053

 @aasjournal

http://english.iap.cas.cn/ 

Zheng Lin | EurekAlert!
Further information:
http://english.iap.cas.cn/RE/201810/t20181026_200518.html
http://dx.doi.org/10.1007/s00376-018-8067-9

More articles from Earth Sciences:

nachricht Geomagnetic jerks finally reproduced and explained
23.04.2019 | CNRS

nachricht "Flight recorder" of rocks within the Earth’s crust
16.04.2019 | Universität Bern

All articles from Earth Sciences >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Energy-saving new LED phosphor

The human eye is particularly sensitive to green, but less sensitive to blue and red. Chemists led by Hubert Huppertz at the University of Innsbruck have now developed a new red phosphor whose light is well perceived by the eye. This increases the light yield of white LEDs by around one sixth, which can significantly improve the energy efficiency of lighting systems.

Light emitting diodes or LEDs are only able to produce light of a certain colour. However, white light can be created using different colour mixing processes.

Im Focus: Quantum gas turns supersolid

Researchers led by Francesca Ferlaino from the University of Innsbruck and the Austrian Academy of Sciences report in Physical Review X on the observation of supersolid behavior in dipolar quantum gases of erbium and dysprosium. In the dysprosium gas these properties are unprecedentedly long-lived. This sets the stage for future investigations into the nature of this exotic phase of matter.

Supersolidity is a paradoxical state where the matter is both crystallized and superfluid. Predicted 50 years ago, such a counter-intuitive phase, featuring...

Im Focus: Explosion on Jupiter-sized star 10 times more powerful than ever seen on our sun

A stellar flare 10 times more powerful than anything seen on our sun has burst from an ultracool star almost the same size as Jupiter

  • Coolest and smallest star to produce a superflare found
  • Star is a tenth of the radius of our Sun
  • Researchers led by University of Warwick could only see...

Im Focus: Quantum simulation more stable than expected

A localization phenomenon boosts the accuracy of solving quantum many-body problems with quantum computers which are otherwise challenging for conventional computers. This brings such digital quantum simulation within reach on quantum devices available today.

Quantum computers promise to solve certain computational problems exponentially faster than any classical machine. “A particularly promising application is the...

Im Focus: Largest, fastest array of microscopic 'traffic cops' for optical communications

The technology could revolutionize how information travels through data centers and artificial intelligence networks

Engineers at the University of California, Berkeley have built a new photonic switch that can control the direction of light passing through optical fibers...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

Revered mathematicians and computer scientists converge with 200 young researchers in Heidelberg!

17.04.2019 | Event News

First dust conference in the Central Asian part of the earth’s dust belt

15.04.2019 | Event News

Fraunhofer FHR at the IEEE Radar Conference 2019 in Boston, USA

09.04.2019 | Event News

 
Latest News

Proteins stand up to nerve cell regression

24.04.2019 | Life Sciences

New sensor detects rare metals used in smartphones

24.04.2019 | Life Sciences

Controlling instabilities gives closer look at chemistry from hypersonic vehicles

24.04.2019 | Life Sciences

VideoLinks
Science & Research
Overview of more VideoLinks >>>