In an article published today in EMBO reports two experts support the use of citation indicators that are based on percentiles, a statistical parameter that allows for comparisons with a carefully defined group of reference data. Journal impact factors and h-index alone do not make the grade.
“We argue that new citation impact indicators are needed and that these indicators should allow the comparison of the observed impact for a given publication set with a reference set of similar publications,” said Lutz Bornmann, a sociologist of science at the Division for Science and Innovation Studies at the Max Planck Society in Munich, Germany. “This is a better way to make meaningful assessments of scientific work.” He added: “Indicators must also take into account that the distribution of citations across papers is often skewed. The use of percentiles described in our paper provides a solution.”
A scientific paper and its citation in other papers reveal how science makes progress and can help to establish the credit for new discoveries. Citation indicators are used extensively to evaluate the impact of the research of individual scientists, institutes and university departments, and sometimes even countries. The outcomes of such analyses can have a profound influence on the careers of scientists or, for example, the future funding of research institutes.
The authors of the study emphasize that journal impact factors and h-index, two frequently used parameters to evaluate the quality and quantity of science, do not readily allow for comparisons between the work of different scientists or publications in journals from unrelated research fields. The problem is similar to comparing the performance of individuals who score goals in sports as different as football and handball. The number of goals in football is often around 2 but in handball can reach 20 or more. Is someone who scores an average of five goals in handball a better athlete than a football player who scores one goal a game?
“Experts of bibliometrics avoid using journal impact factors and the h-index because these indicators do not provide normalized values. For many years, they have been using reference sets to normalize the number of citations which is indispensable for fair assessment,” commented Werner Marx, co-author of the paper and head of the Information Retrieval Services at the Max Planck Institute for Solid State Research in Stuttgart, Germany. “The use of this type of advanced analysis is growing. However, we have an opportunity to do more and this could have a big impact on the way we answer important questions such as how good research really is.”
How good is research really? Measuring the citation impact of publications with percentiles increases correct assessments and fair comparisons
Lutz Bornmann, Werner Marx
Read the paper:
Further information on EMBO reports is available at http://www.nature.com/embor
Head | Public Relations and Communications
Tel: +49 6221 8891 108/111
EMBO is an organization of more than 1500 leading researchers that promotes excellence in the life sciences. The major goals of the organization are to support talented researchers at all stages of their careers, stimulate the exchange of scientific information, and help build a European research environment where scientists can achieve their best work.
EMBO helps young scientists to advance their research, promote their international reputations and ensure their mobility. Courses, workshops, conferences and scientific journals disseminate the latest research and offer training in techniques to maintain high standards of excellence in research practice. EMBO helps to shape science and research policy by seeking input and feedback from our community and by following closely the trends in science in Europe.
Yvonne Kaul | Source: EMBO
Further information: www.embo.org
More articles from Science Education:
With board games, it's how children count that counts
18.11.2013 | Boston College
Investing in our future: The evidence base on preschool education
16.10.2013 | Society for Research in Child Development
The molecular architecture of three key proteins and their complexes reveals how plants fine-tune their immune response to pathogens
Plants rarely get sick in their natural environment. When the threat of infection arises, a quick decision is made about the necessary countermeasures. The course is set by a protein which forms complexes with its partner proteins for this purpose.
Jane Parker from the Max Planck Institute for Plant Breeding ...
Researchers studying speciation of butterfly orchids on the Azores have been startled to discover that the answer to a long-debated question "Do the islands support one species or two species?" is actually "three species".
Hochstetter's Butterfly-orchid, newly recognized following application of a battery of scientific techniques and reveling in a complex taxonomic history worthy of Sherlock Holmes, is arguably Europe's rarest orchid species. Under threat in its mountain-top retreat, the orchid urgently requires conservation recognition.
A lavishly illustrated publication, titled "Systematic revision of Platanthera in ...
Researchers from Brown University and the University of Hawaii have found some mineralogical surprises in the Moon's largest impact crater.
Data from the Moon Mineralogy Mapper that flew aboard India's Chandrayaan-1 lunar orbiter shows a diverse mineralogy in the subsurface of the giant South Pole Aitken basin.
The differing mineral signatures could be reflective of the minerals dredged up at the time of the giant impact 4 billion years ago, ...
In power electronics systems bonded connections create the central electrical connections between adjoining surfaces.
The quality of these bonded connections is one of the main factors that determines the reliability and availability of drive systems in electric vehicles, and hence constitutes a major design challenge for German auto manufacturers aiming to electrify their vehicles.
Now the partners participating in the RoBE (Robust Bonds in ...
International team of scientists develops new feedback method for optimizing the laser pulse shapes used in the control of chemical reactions
In many ways, traditional chemical synthesis is similar to cooking. To alter the final product, you can change the ingredients or their ratio, change the method of mixing ingredients, or change the temperature or pressure of the environment of the ingredients.
Like an accomplished chef, chemists have become very skilled ...
11.12.2013 | Information Technology
11.12.2013 | Life Sciences
11.12.2013 | Agricultural and Forestry Science
11.12.2013 | Event News
10.12.2013 | Event News
05.12.2013 | Event News