Network scientists at Indiana University have developed a new computational method that can leverage any body of knowledge to aid in the complex human task of fact-checking.
In the first use of this method, IU scientists created a simple computational fact-checker that assigns "truth scores" to statements concerning history, geography and entertainment, as well as random statements drawn from the text of Wikipedia, the well-known online encyclopedia.
In multiple experiments, the automated system consistently matched the assessment of human fact-checkers in terms of their certitude about the accuracy of these statements.
The results of the study, "Computational Fact Checking From Knowledge Networks," are reported in today's issue of PLOS ONE.
"These results are encouraging and exciting," said Giovanni Luca Ciampaglia, a postdoctoral fellow at the Center for Complex Networks and Systems Research in the IU Bloomington School of Informatics and Computing, who led the study. "We live in an age of information overload, including abundant misinformation, unsubstantiated rumors and conspiracy theories whose volume threatens to overwhelm journalists and the public.
"Our experiments point to methods to abstract the vital and complex human task of fact-checking into a network analysis problem, which is easy to solve computationally."
The team selected Wikipedia as the information source for their experiment due to its breadth and open nature. Although Wikipedia is not 100 percent accurate, previous studies estimate the online encyclopedia is nearly as reliable as traditional encyclopedias, but also covers many more subjects.
Using factual information from infoboxes on the site, IU scientists built a "knowledge graph" with 3 million concepts and 23 million links between them. A link between two concepts in the graph can be read as a simple factual statement, such as "Socrates is a person" or "Paris is the capital of France."
In what the IU scientists describe as an "automatic game of trivia," the team applied their algorithm to answer simple questions related to geography, history and entertainment, including statements that matched states or nations with their capitals, presidents with their spouses and Oscar-winning film directors with the movie for which they won the Best Picture awards, with the majority of tests returning highly accurate truth scores.
Lastly, the scientists used the algorithm to fact-check excerpts from the main text of Wikipedia, which were previously labeled by human fact-checkers as true or false, and found a positive correlation between the truth scores produced by the algorithm and the answers provided by the fact-checkers.
Significantly, the IU team found their computational method could even assess the truthfulness of statements about information not directly contained in the infoboxes. For example, the fact that Steve Tesich --- the Serbian-American screenwriter of the classic Hoosier film "Breaking Away" -- graduated from IU, despite the information not being specifically addressed in the infobox about him.
"The measurement of the truthfulness of statements appears to rely strongly on indirect connections, or 'paths,' between concepts," Ciampaglia said. "If we prevented our fact-checker from traversing multiple nodes on the graph, it performed poorly since it could not discover relevant indirect connections. But because it's free to explore beyond the information provided in one infobox, our method leverages the power of the full knowledge graph."
Although the experiments were conducted using Wikipedia, the IU team's method does not assume any particular source of knowledge. The scientists aim to conduct additional experiments using knowledge graphs built from other sources of human knowledge, such as Freebase, the open-knowledge base built by Google, and note that multiple information sources could be used together to account for different belief systems.
"Misinformation endangers the public debate on a broad range of global societal issues," said Filippo Menczer, director of the Center for Complex Networks and Systems Research and a professor in the IU School of Informatics and Computing, who is a co-author on the study. "With increasing reliance on the Internet as a source of information, we need tools to deal with the misinformation that reaches us every day. Computational fact-checkers could become part of the solution to this problem."
The team added a significant amount of natural language processing research, and other work remains before these methods could be made available to the public as a software tool.
Additional co-authors on the study are professor Luis M. Rocha, associate professors Johan Bollen and Alessandro Flammini, and doctoral student Prashant Shiralkar, all in the School of Informatics and Computing. This work was supported in part by the Swiss National Science Foundation, the Lilly Endowment, the James S. McDonnell Foundation, the National Science Foundation and the Department of Defense.
Kevin Fryling | EurekAlert!
Next Generation Cryptography
20.03.2018 | Fraunhofer-Institut für Sichere Informationstechnologie SIT
TIB’s Visual Analytics Research Group to develop methods for person detection and visualisation
19.03.2018 | Technische Informationsbibliothek (TIB)
An international team of researchers has discovered a new anti-cancer protein. The protein, called LHPP, prevents the uncontrolled proliferation of cancer cells in the liver. The researchers led by Prof. Michael N. Hall from the Biozentrum, University of Basel, report in “Nature” that LHPP can also serve as a biomarker for the diagnosis and prognosis of liver cancer.
The incidence of liver cancer, also known as hepatocellular carcinoma, is steadily increasing. In the last twenty years, the number of cases has almost doubled...
In just a few weeks from now, the Chinese space station Tiangong-1 will re-enter the Earth's atmosphere where it will to a large extent burn up. It is possible that some debris will reach the Earth's surface. Tiangong-1 is orbiting the Earth uncontrolled at a speed of approx. 29,000 km/h.Currently the prognosis relating to the time of impact currently lies within a window of several days. The scientists at Fraunhofer FHR have already been monitoring Tiangong-1 for a number of weeks with their TIRA system, one of the most powerful space observation radars in the world, with a view to supporting the German Space Situational Awareness Center and the ESA with their re-entry forecasts.
Following the loss of radio contact with Tiangong-1 in 2016 and due to the low orbital height, it is now inevitable that the Chinese space station will...
Fraunhofer Institute for Organic Electronics, Electron Beam and Plasma Technology FEP, provider of research and development services for OLED lighting solutions, announces the founding of the “OLED Licht Forum” and presents latest OLED design and lighting solutions during light+building, from March 18th – 23rd, 2018 in Frankfurt a.M./Germany, at booth no. F91 in Hall 4.0.
They are united in their passion for OLED (organic light emitting diodes) lighting with all of its unique facets and application possibilities. Thus experts in...
A new scenario seeking to explain how Mars' putative oceans came and went over the last 4 billion years implies that the oceans formed several hundred million...
For the first time, an interdisciplinary team from the University of Basel has succeeded in integrating artificial organelles into the cells of live zebrafish embryos. This innovative approach using artificial organelles as cellular implants offers new potential in treating a range of diseases, as the authors report in an article published in Nature Communications.
In the cells of higher organisms, organelles such as the nucleus or mitochondria perform a range of complex functions necessary for life. In the networks of...
19.03.2018 | Event News
16.03.2018 | Event News
13.03.2018 | Event News
22.03.2018 | Trade Fair News
22.03.2018 | Earth Sciences
22.03.2018 | Earth Sciences