Network scientists at Indiana University have developed a new computational method that can leverage any body of knowledge to aid in the complex human task of fact-checking.
In the first use of this method, IU scientists created a simple computational fact-checker that assigns "truth scores" to statements concerning history, geography and entertainment, as well as random statements drawn from the text of Wikipedia, the well-known online encyclopedia.
In multiple experiments, the automated system consistently matched the assessment of human fact-checkers in terms of their certitude about the accuracy of these statements.
The results of the study, "Computational Fact Checking From Knowledge Networks," are reported in today's issue of PLOS ONE.
"These results are encouraging and exciting," said Giovanni Luca Ciampaglia, a postdoctoral fellow at the Center for Complex Networks and Systems Research in the IU Bloomington School of Informatics and Computing, who led the study. "We live in an age of information overload, including abundant misinformation, unsubstantiated rumors and conspiracy theories whose volume threatens to overwhelm journalists and the public.
"Our experiments point to methods to abstract the vital and complex human task of fact-checking into a network analysis problem, which is easy to solve computationally."
The team selected Wikipedia as the information source for their experiment due to its breadth and open nature. Although Wikipedia is not 100 percent accurate, previous studies estimate the online encyclopedia is nearly as reliable as traditional encyclopedias, but also covers many more subjects.
Using factual information from infoboxes on the site, IU scientists built a "knowledge graph" with 3 million concepts and 23 million links between them. A link between two concepts in the graph can be read as a simple factual statement, such as "Socrates is a person" or "Paris is the capital of France."
In what the IU scientists describe as an "automatic game of trivia," the team applied their algorithm to answer simple questions related to geography, history and entertainment, including statements that matched states or nations with their capitals, presidents with their spouses and Oscar-winning film directors with the movie for which they won the Best Picture awards, with the majority of tests returning highly accurate truth scores.
Lastly, the scientists used the algorithm to fact-check excerpts from the main text of Wikipedia, which were previously labeled by human fact-checkers as true or false, and found a positive correlation between the truth scores produced by the algorithm and the answers provided by the fact-checkers.
Significantly, the IU team found their computational method could even assess the truthfulness of statements about information not directly contained in the infoboxes. For example, the fact that Steve Tesich --- the Serbian-American screenwriter of the classic Hoosier film "Breaking Away" -- graduated from IU, despite the information not being specifically addressed in the infobox about him.
"The measurement of the truthfulness of statements appears to rely strongly on indirect connections, or 'paths,' between concepts," Ciampaglia said. "If we prevented our fact-checker from traversing multiple nodes on the graph, it performed poorly since it could not discover relevant indirect connections. But because it's free to explore beyond the information provided in one infobox, our method leverages the power of the full knowledge graph."
Although the experiments were conducted using Wikipedia, the IU team's method does not assume any particular source of knowledge. The scientists aim to conduct additional experiments using knowledge graphs built from other sources of human knowledge, such as Freebase, the open-knowledge base built by Google, and note that multiple information sources could be used together to account for different belief systems.
"Misinformation endangers the public debate on a broad range of global societal issues," said Filippo Menczer, director of the Center for Complex Networks and Systems Research and a professor in the IU School of Informatics and Computing, who is a co-author on the study. "With increasing reliance on the Internet as a source of information, we need tools to deal with the misinformation that reaches us every day. Computational fact-checkers could become part of the solution to this problem."
The team added a significant amount of natural language processing research, and other work remains before these methods could be made available to the public as a software tool.
Additional co-authors on the study are professor Luis M. Rocha, associate professors Johan Bollen and Alessandro Flammini, and doctoral student Prashant Shiralkar, all in the School of Informatics and Computing. This work was supported in part by the Swiss National Science Foundation, the Lilly Endowment, the James S. McDonnell Foundation, the National Science Foundation and the Department of Defense.
Kevin Fryling | EurekAlert!
Shaping nanoparticles for improved quantum information technology
15.10.2019 | DOE/Argonne National Laboratory
Controlling superconducting regions within an exotic metal
11.10.2019 | Ecole Polytechnique Fédérale de Lausanne
A very special kind of light is emitted by tungsten diselenide layers. The reason for this has been unclear. Now an explanation has been found at TU Wien (Vienna)
It is an exotic phenomenon that nobody was able to explain for years: when energy is supplied to a thin layer of the material tungsten diselenide, it begins to...
Researchers at Ludwig-Maximilians-Universitaet (LMU) in Munich have explored the initial consequences of the interaction of light with molecules on the surface of nanoscopic aerosols.
The nanocosmos is constantly in motion. All natural processes are ultimately determined by the interplay between radiation and matter. Light strikes particles...
Particles that are mere nanometers in size are at the forefront of scientific research today. They come in many different shapes: rods, spheres, cubes, vesicles, S-shaped worms and even donut-like rings. What makes them worthy of scientific study is that, being so tiny, they exhibit quantum mechanical properties not possible with larger objects.
Researchers at the Center for Nanoscale Materials (CNM), a U.S. Department of Energy (DOE) Office of Science User Facility located at DOE's Argonne National...
A new research project at the TH Mittelhessen focusses on the development of a novel light weight design concept for leisure boats and yachts. Professor Stephan Marzi from the THM Institute of Mechanics and Materials collaborates with Krake Catamarane, which is a shipyard located in Apolda, Thuringia.
The project is set up in an international cooperation with Professor Anders Biel from Karlstad University in Sweden and the Swedish company Lamera from...
Superconductivity has fascinated scientists for many years since it offers the potential to revolutionize current technologies. Materials only become superconductors - meaning that electrons can travel in them with no resistance - at very low temperatures. These days, this unique zero resistance superconductivity is commonly found in a number of technologies, such as magnetic resonance imaging (MRI).
Future technologies, however, will harness the total synchrony of electronic behavior in superconductors - a property called the phase. There is currently a...
02.10.2019 | Event News
02.10.2019 | Event News
19.09.2019 | Event News
18.10.2019 | Power and Electrical Engineering
18.10.2019 | Medical Engineering
18.10.2019 | Physics and Astronomy