Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

A new method to quickly identify outliers in air quality monitoring data

30.10.2018

Ambient air quality monitoring data are the most important source for public awareness regarding air quality and are widely used in many research fields, such as improving air quality forecasting and the analysis of haze episodes. However, there are outliers among such monitoring data, due to instrument malfunctions, the influence of harsh environments, and the limitation of measuring methods.

In practice, manual inspection is often applied to identify these outliers. However, as the amount of data grows rapidly, this method becomes increasingly cumbersome.


The PM2.5 monitoring instruments at State Key Laboratory of Atmospheric Boundary Layer Physics and Atmospheric Chemistry (LAPC), Institute of Atmospheric Physics, Chinese Academy of Sciences.

Image by TANG Xiao

To deal with the problem, Dr. WU Huangjian and Associate Professor TANG Xiao from the Institute of Atmospheric Physics, Chinese Academy of Sciences, propose a fully automatic outlier detection method based on the probability of residuals.

The method adopts multiple regression methods, and the regression residuals are used to discriminate outliers. Based on the standard deviations of the residuals, probabilities of the residuals can be calculated, and the observations with small probabilities are tagged as outliers and removed by a computer program. Their findings are published in Advances in Atmospheric Sciences.

"By introducing the probabilities of residuals, multiple rules can be used for identifying outliers on the same framework," says Dr. Wu.

"For example, by assuming that the residuals of spatial regression and temporal regression obey a bivariate normal distribution, spatial and temporal consistencies can be simultaneously evaluated for better identification of outliers".

The method can flag potentially erroneous data in the hourly observations from 1436 stations of the China National Environmental Monitoring Center (CNEMC) within a minute. Indeed, it has been used in CNEMC's air quality forecasting system, and is going to be integrated into the data management system. The hope is that outliers in the system's real-time air quality data will be removed in the near future.

The method is published in Advances in Atmopheric Sciences.

Media Contact

Zheng Lin
jennylin@mail.iap.ac.cn
86-108-299-5053

 @aasjournal

http://english.iap.cas.cn/ 

Zheng Lin | EurekAlert!
Further information:
http://english.iap.cas.cn/RE/201810/t20181026_200518.html
http://dx.doi.org/10.1007/s00376-018-8067-9

More articles from Earth Sciences:

nachricht Massive impact crater from a kilometer-wide iron meteorite discovered in Greenland
15.11.2018 | Faculty of Science - University of Copenhagen

nachricht The unintended consequences of dams and reservoirs
14.11.2018 | Uppsala University

All articles from Earth Sciences >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: UNH scientists help provide first-ever views of elusive energy explosion

Researchers at the University of New Hampshire have captured a difficult-to-view singular event involving "magnetic reconnection"--the process by which sparse particles and energy around Earth collide producing a quick but mighty explosion--in the Earth's magnetotail, the magnetic environment that trails behind the planet.

Magnetic reconnection has remained a bit of a mystery to scientists. They know it exists and have documented the effects that the energy explosions can...

Im Focus: A Chip with Blood Vessels

Biochips have been developed at TU Wien (Vienna), on which tissue can be produced and examined. This allows supplying the tissue with different substances in a very controlled way.

Cultivating human cells in the Petri dish is not a big challenge today. Producing artificial tissue, however, permeated by fine blood vessels, is a much more...

Im Focus: A Leap Into Quantum Technology

Faster and secure data communication: This is the goal of a new joint project involving physicists from the University of Würzburg. The German Federal Ministry of Education and Research funds the project with 14.8 million euro.

In our digital world data security and secure communication are becoming more and more important. Quantum communication is a promising approach to achieve...

Im Focus: Research icebreaker Polarstern begins the Antarctic season

What does it look like below the ice shelf of the calved massive iceberg A68?

On Saturday, 10 November 2018, the research icebreaker Polarstern will leave its homeport of Bremerhaven, bound for Cape Town, South Africa.

Im Focus: Penn engineers develop ultrathin, ultralight 'nanocardboard'

When choosing materials to make something, trade-offs need to be made between a host of properties, such as thickness, stiffness and weight. Depending on the application in question, finding just the right balance is the difference between success and failure

Now, a team of Penn Engineers has demonstrated a new material they call "nanocardboard," an ultrathin equivalent of corrugated paper cardboard. A square...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

Optical Coherence Tomography: German-Japanese Research Alliance hosted Medical Imaging Conference

19.11.2018 | Event News

“3rd Conference on Laser Polishing – LaP 2018” Attracts International Experts and Users

09.11.2018 | Event News

On the brain’s ability to find the right direction

06.11.2018 | Event News

 
Latest News

Controlling organ growth with light

19.11.2018 | Life Sciences

New way to look at cell membranes could change the way we study disease

19.11.2018 | Life Sciences

Optical Coherence Tomography: German-Japanese Research Alliance hosted Medical Imaging Conference

19.11.2018 | Event News

VideoLinks
Science & Research
Overview of more VideoLinks >>>