Past events

Centre for Transformative Innovation & Swinburne Law School: Research Seminar (11 August)

You are viewing a past event.

Date: Friday 11 August 2017
Venue: Moot Court, TD120, Swinburne University, Hawthorn Campus

Add to Calendar

Bias against Novelty in Science: A Cautionary Tale for Users of Bibliometric Indicators

by Professor Paula Stephan, professor of economics, Georgia State University and a research associate, National Bureau of Economic Research


Research which explores unchartered waters has a high potential for major impact but also carries a higher uncertainty of having impact. Such explorative research is often described as taking a novel approach.

This study examines the complex relationship between pursuing a novel approach and impact. Viewing scientific research as a combinatorial process, we measure novelty in science by examining whether a published paper makes first time ever combinations of referenced journals, taking into account the difficulty of making such combinations. We apply this newly developed measure of novelty to all Web of Science research articles published in 2001 across all scientific disciplines.

We find that highly novel papers, defined to be those that make more (distant) new combinations, deliver high gains to science: they are more likely to be a top 1% highly cited paper in the long run, to inspire follow on highly cited research, and to be cited in a broader set of disciplines and in disciplines that are more distant from their “home” field. At the same time, novel research is also more risky, reflected by a higher variance in its citation performance.

We also find strong evidence of delayed recognition of novel papers as novel papers are less likely to be top cited when using short time-windows. In addition, we find that novel research is significantly more highly cited in “foreign” fields but not in their “home” field.

Finally, novel papers are published in journals with a lower Impact Factor, compared with non-novel papers, ceteris paribus. These findings suggest that science policy, in particular funding decisions which rely on bibliometric indicators based on short-term citation counts and Journal Impact Factors, may be biased against “high risk/high gain” novel research. The findings also caution against a mono-disciplinary approach in peer review to assess the true value of novel research.

About the speaker

Paula Stephan is professor of economics, Georgia State University and a research associate, National Bureau of Economic Research. She is a Fellow of the American Association for the Advancement of Science and a member of the Board of Reviewing Editors, Science. Science Careers named Stephan their first “Person of the Year” in December of 2012 “honoring an individual who, during the past 12 months, has made an especially significant and sustained contribution to the welfare of early-career scientists.”

Stephan has published numerous articles in such journals as The American Economic Review, The Journal of Economic Literature, Management Science, Nature, Organization Science, Research Policy and Science. Her book How Economics Shapes Science was published by Harvard University Press, 2012 and has been translated into Chinese and Korean. Her research has been supported by the Alfred P. Sloan Foundation, the Andrew W. Mellon Foundation, and the National Science Foundation.

Stephan currently serves on the National Academies Committee on the Next Generation of Researchers Initiative and the Research Council of The State University of New York (SUNY) System. In the recent past she served on the National Research Council’s Board on Higher Education and Workforce and the Committee to Review the State of the Postdoctoral Experience for Scientists and Engineers. She served on the National Advisory General Medical Sciences Council, National Institutes of Health 2005-2009 and served on the Advisory Committee of the Social, Behavioral, and Economics Program, National Science Foundation, 2001-2008 (CEOSE, 2001-2003). She has held visiting positions at the Max Planck Institute, Munich, Germany, KU Leuven, Leuven, Belgium, Harvard University, International Center for Economic Research, Turin, Italy, and the Wizzenschaftszentrum für Social Forschung, Berlin, Germany. Stephan received her undergraduate degree in economics from Grinnell College and her PhD from the University of Michigan.

This research seminar is brought to you by Swinburne Law School and the Centre for Transformative Innovation. Join our monthly seminar series - check the Faculty of Business and Law events page for future seminars.

Contact Information: Jungho Kim
Email: Tel: +61 3 9214 8475