Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

New study: Tools that assess bias in standardized tests are flawed

30.07.2010
Overturning more than 40 years of accepted practice, new research proves that the tools used to check tests of "general mental ability" for bias are themselves flawed.

This key finding from the Indiana University Kelley School of Business challenges reliance on such exams to make objective decisions for employment or academic admissions even in the face of well-documented gaps between mean scores of white and minority populations.

The study, published in the July issue of the Journal of Applied Psychology, investigated an amalgam of scores representing a vast sample of commonly used tests, including civil service or other pre-employment exams and university entrance exams.

"Test bias" means that two people with different ethnicity or gender, for example, who have the same test score are predicted to have different "scores" on the outcome (e.g., job performance); thus a biased test might benefit certain groups over others. Decades of earlier research consistently found no evidence of test bias against ethnic minorities, but the current study challenges this established belief.

"For generations, important decisions have been made about life-changing opportunities in employment and education based on results of these tests -- but we can no longer say with certainty they are unbiased," said Herman Aguinis, professor of organizational behavior and human resources and director of the Kelley School's new Institute for Global Organizational Effectiveness.

He led the study, which was co-authored by Steven A. Culpepper at the University of Colorado Denver and Charles A. Pierce at the University of Memphis.

"Our findings are significant because we proved that bias can be present but not be detected by even the top experts in the field, which could result in inaccurate prediction of outcomes such as job and academic performance for hundreds of thousands, if not millions, of individuals," Aguinis said.

To reach these conclusions, Aguinis and his co-authors created the largest simulation of its kind -- using nearly 16 million individual samples to yield more than eight trillion pairs of individual test/outcome scores. They built bias into most samples to resemble real-world results and used newly available super computing technology and power to check tens of billions of scores. They found the procedures in use today overwhelmingly and repeatedly missed the bias inserted in the data.

Few topics in human resource management have generated more public attention than bias in pre-employment and academic-admissions exams.

"The belief in the fairness of the tests and the accuracy of the gauges to check them has been so deeply engrained that to challenge them would be akin to questioning the sun as center of the solar system," said Aguinis, a nationally recognized expert who was also a co-author of an amicus brief in the landmark Ricci v. DeStefano Supreme Court case regarding employment testing.

"The irony is that for 40 years we have been trying to assess potential test bias with a biased procedure, and we now see that countless people may have been denied or given opportunities unfairly," he added. "From an ethical standpoint it may be argued that even if only one individual is affected this way, that is one too many. The problem is obviously magnified when we are dealing with hundreds of thousands, if not millions, of individuals taking standardized tests every year."

Prelude to a New Era?

Given the weight placed on such testing and the polarizing nature of the underlying racial/ethnic achievement gap, the authors expect the study will spur considerable controversy among the public and the academic, legal and policy communities, all of which will question the long-held belief that tests are unbiased.

They also anticipate a significant impact on the multi-billion dollar testing industry but made clear that they are not saying that any organization is deliberately using biased tests. However, as a preliminary step while more research is conducted, it is likely that many organizations will examine their existing tests and perhaps create new ones.

"While the academic community has demonstrated repeatedly that different racial or ethnic groups' cultural frames of reference and identity may play a role in affecting test scores, we have not used that knowledge to sufficiently advance testing processes," he said. "We sincerely hope that this research opens doors to thoughtful and important analysis that will allow us to legitimately assign scores that predict a job well done."

George Vlahakis | EurekAlert!
Further information:
http://www.indiana.edu

More articles from Studies and Analyses:

nachricht Innovative genetic tests for children with developmental disorders and epilepsy
11.07.2018 | Christian-Albrechts-Universität zu Kiel

nachricht Oxygen loss in the coastal Baltic Sea is “unprecedentedly severe”
05.07.2018 | European Geosciences Union

All articles from Studies and Analyses >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: First evidence on the source of extragalactic particles

For the first time ever, scientists have determined the cosmic origin of highest-energy neutrinos. A research group led by IceCube scientist Elisa Resconi, spokesperson of the Collaborative Research Center SFB1258 at the Technical University of Munich (TUM), provides an important piece of evidence that the particles detected by the IceCube neutrino telescope at the South Pole originate from a galaxy four billion light-years away from Earth.

To rule out other origins with certainty, the team led by neutrino physicist Elisa Resconi from the Technical University of Munich and multi-wavelength...

Im Focus: Magnetic vortices: Two independent magnetic skyrmion phases discovered in a single material

For the first time a team of researchers have discovered two different phases of magnetic skyrmions in a single material. Physicists of the Technical Universities of Munich and Dresden and the University of Cologne can now better study and understand the properties of these magnetic structures, which are important for both basic research and applications.

Whirlpools are an everyday experience in a bath tub: When the water is drained a circular vortex is formed. Typically, such whirls are rather stable. Similar...

Im Focus: Breaking the bond: To take part or not?

Physicists working with Roland Wester at the University of Innsbruck have investigated if and how chemical reactions can be influenced by targeted vibrational excitation of the reactants. They were able to demonstrate that excitation with a laser beam does not affect the efficiency of a chemical exchange reaction and that the excited molecular group acts only as a spectator in the reaction.

A frequently used reaction in organic chemistry is nucleophilic substitution. It plays, for example, an important role in in the synthesis of new chemical...

Im Focus: New 2D Spectroscopy Methods

Optical spectroscopy allows investigating the energy structure and dynamic properties of complex quantum systems. Researchers from the University of Würzburg present two new approaches of coherent two-dimensional spectroscopy.

"Put an excitation into the system and observe how it evolves." According to physicist Professor Tobias Brixner, this is the credo of optical spectroscopy....

Im Focus: Chemical reactions in the light of ultrashort X-ray pulses from free-electron lasers

Ultra-short, high-intensity X-ray flashes open the door to the foundations of chemical reactions. Free-electron lasers generate these kinds of pulses, but there is a catch: the pulses vary in duration and energy. An international research team has now presented a solution: Using a ring of 16 detectors and a circularly polarized laser beam, they can determine both factors with attosecond accuracy.

Free-electron lasers (FELs) generate extremely short and intense X-ray flashes. Researchers can use these flashes to resolve structures with diameters on the...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

Leading experts in Diabetes, Metabolism and Biomedical Engineering discuss Precision Medicine

13.07.2018 | Event News

Conference on Laser Polishing – LaP: Fine Tuning for Surfaces

12.07.2018 | Event News

11th European Wood-based Panel Symposium 2018: Meeting point for the wood-based materials industry

03.07.2018 | Event News

 
Latest News

Subaru Telescope helps pinpoint origin of ultra-high energy neutrino

16.07.2018 | Physics and Astronomy

Barium ruthenate: A high-yield, easy-to-handle perovskite catalyst for the oxidation of sulfides

16.07.2018 | Life Sciences

New research calculates capacity of North American forests to sequester carbon

16.07.2018 | Earth Sciences

VideoLinks
Science & Research
Overview of more VideoLinks >>>