Forum for Science, Industry and Business

Sponsored by:     3M 
Search our Site:

 

Don’t Trust Experts’ Forecasts, O.R. Study Says

29.08.2007
Embarrassment at Predicting Conflict Outcomes: Gurus Score 32%, Barely Outperforming Newbies, and Chance at 28%

A study about predicting the outcome of actual conflicts found that the forecasts of experts who use their unaided judgment are little better than those of novices, according to a new study in a publication of the Institute for Operations Research and the Management Sciences (INFORMS®).

When presented with actual crises, such as a disguised version of a 1970s border dispute between Iraq and Syria and an unfolding dispute between football players and management, experts were able to forecast the decisions the parties made in only 32% of the cases, little better than the 29% scored by undergraduate students. Chance guesses at the outcomes would be right 28% of the time.

The study, “The Ombudsman: Value of Expertise for Forecasting Decisions in Conflicts,” is by Kesten C. Green of Monash University in Australia and J. Scott Armstrong of the Wharton School at the University of Pennsylvania. It appears in the INFORMS journal Interfaces, Volume 37. no. 3.

The research can have serious consequences for foreign policy and business. Green says, “Political leaders in the West are pondering how best to deal with the threat of the Iranian government’s nuclear ambitions. Forecasting problems such as this are the stuff of not only international relations but also of takeover battles, commercial competition, and labor-management disputes. In most cases, experts use their judgment to predict what will happen. How good are their forecasts?

“The short answer is that they are of little value in terms of accuracy. In addition, they lead people into false confidence.”

In the study, the authors question experts’ ability to forecast without proven structured methods.

“Accurate prediction is difficult because conflicts tend to be too complex for people to think through in ways that realistically represent their actual progress,” the authors write. “Parties in conflict often act and react many times, and change because of their interactions.”

The authors designed an experiment to test their hypothesis. They wrote descriptions of eight diverse conflicts and presented them to conflict experts, domain experts, and forecasting experts. The conflicts were assigned to experts based on their specialties.

The case studies were diverse: they included a hostile takeover attempt, nations preparing for war, a controversial investment proposal, a nurses’ strike, an action by football players for a larger share of the gate, an employee resisting the downgrading of her job, artists demanding taxpayer funding, and a new distribution arrangement that a manufacturer proposed to retailers.

The authors provided both the expert and novice participants with a set of between three and six potential decisions. The authors received 106 expert responses. The authors presented the same material to undergraduate students and received 169 responses.

Analysis of additional data produced similar results. In one instance, the authors attempted to determine if veteran experts would be more likely to make accurate forecasts than less experienced experts. “Common sense expectations did not prove to be correct,” they write. “The 57 forecasts of experts with less than five years experience were more accurate (36%) than the 48 forecasts of experts with more experience (29%).”

The authors also asked experts about their previous experience with similar conflicts and looked at the relationship with the accuracy of their forecasts. Again, the expected conclusion did not prevail: those who considered themselves as having little experience with similar conflicts produced forecasts that were equally as accurate as those who were long-time veterans in the field.

The authors examined the confidence that the experts had in their forecasts by asking them how likely it was that they would have changed their forecasts had they spent more time on the task. Another surprise: 68 high-confidence forecasts were less accurate (28%) than the 35 low-confidence forecasts (41%).

Based on this study and earlier research, the authors conclude that there are no good grounds for decision makers to rely on experts’ unaided judgments for forecasting decisions in conflicts. Such reliance discourages experts and decision makers from investigating alternative approaches.

Instead, they recommend that experts use reliable decision-support tools. They cite two examples of decision aids that can improve forecasts. In an earlier study, Green reported that simulated interaction, a type of role playing for forecasting behavior in conflicts, reduced error by 47%.

Using another technique, structured analogies, the authors found favorable results. In that study, they asked experts to recall and analyze information on similar situations. When experts were able to think of at least two analogies, forecast error was reduced by 39%. This structured technique requires experts, and those with more expertise were able to contribute much more to making accurate forecasts.

About INFORMS
The Institute for Operations Research and the Management Sciences (INFORMS®) is an international scientific society with 10,000 members, including Nobel Prize laureates, dedicated to applying scientific methods to help improve decision-making, management, and operations. Members of INFORMS work in business, government, and academia. They are represented in fields as diverse as airlines, health care, law enforcement, the military, financial engineering, and telecommunications. The INFORMS website is www.informs.org. More information about operations research is at www.scienceofbetter.org.

INFORMS journals are strongly cited in Journal Citation Reports, an industry source. The special MBA issue published by Business Week includes Operations Research and two other INFORMS journals in its list of 20 top academic journals that are used to evaluate business school programs. Financial Times includes five INFORMS journals in its list of academic journals used to evaluate MBA programs.

Barry List | EurekAlert!
Further information:
http://www.informs.org

More articles from Business and Finance:

nachricht Microtechnology industry is hiring – positive developments of past years continue
09.04.2018 | IVAM Fachverband für Mikrotechnik

nachricht RWI/ISL-Container Throughput Index with minor decline on a high overall level
20.03.2018 | RWI – Leibniz-Institut für Wirtschaftsforschung

All articles from Business and Finance >>>

The most recent press releases about innovation >>>

Die letzten 5 Focus-News des innovations-reports im Überblick:

Im Focus: Temperature-controlled fiber-optic light source with liquid core

In a recent publication in the renowned journal Optica, scientists of Leibniz-Institute of Photonic Technology (Leibniz IPHT) in Jena showed that they can accurately control the optical properties of liquid-core fiber lasers and therefore their spectral band width by temperature and pressure tuning.

Already last year, the researchers provided experimental proof of a new dynamic of hybrid solitons– temporally and spectrally stationary light waves resulting...

Im Focus: Overdosing on Calcium

Nano crystals impact stem cell fate during bone formation

Scientists from the University of Freiburg and the University of Basel identified a master regulator for bone regeneration. Prasad Shastri, Professor of...

Im Focus: AchemAsia 2019 will take place in Shanghai

Moving into its fourth decade, AchemAsia is setting out for new horizons: The International Expo and Innovation Forum for Sustainable Chemical Production will take place from 21-23 May 2019 in Shanghai, China. With an updated event profile, the eleventh edition focusses on topics that are especially relevant for the Chinese process industry, putting a strong emphasis on sustainability and innovation.

Founded in 1989 as a spin-off of ACHEMA to cater to the needs of China’s then developing industry, AchemAsia has since grown into a platform where the latest...

Im Focus: First real-time test of Li-Fi utilization for the industrial Internet of Things

The BMBF-funded OWICELLS project was successfully completed with a final presentation at the BMW plant in Munich. The presentation demonstrated a Li-Fi communication with a mobile robot, while the robot carried out usual production processes (welding, moving and testing parts) in a 5x5m² production cell. The robust, optical wireless transmission is based on spatial diversity; in other words, data is sent and received simultaneously by several LEDs and several photodiodes. The system can transmit data at more than 100 Mbit/s and five milliseconds latency.

Modern production technologies in the automobile industry must become more flexible in order to fulfil individual customer requirements.

Im Focus: Sharp images with flexible fibers

An international team of scientists has discovered a new way to transfer image information through multimodal fibers with almost no distortion - even if the fiber is bent. The results of the study, to which scientist from the Leibniz-Institute of Photonic Technology Jena (Leibniz IPHT) contributed, were published on 6thJune in the highly-cited journal Physical Review Letters.

Endoscopes allow doctors to see into a patient’s body like through a keyhole. Typically, the images are transmitted via a bundle of several hundreds of optical...

All Focus news of the innovation-report >>>

Anzeige

Anzeige

VideoLinks
Industry & Economy
Event News

Munich conference on asteroid detection, tracking and defense

13.06.2018 | Event News

2nd International Baltic Earth Conference in Denmark: “The Baltic Sea region in Transition”

08.06.2018 | Event News

ISEKI_Food 2018: Conference with Holistic View of Food Production

05.06.2018 | Event News

 
Latest News

Creating a new composite fuel for new-generation fast reactors

20.06.2018 | Materials Sciences

Game-changing finding pushes 3D-printing to the molecular limit

20.06.2018 | Materials Sciences

Could this material enable autonomous vehicles to come to market sooner?

20.06.2018 | Materials Sciences

VideoLinks
Science & Research
Overview of more VideoLinks >>>