From climate change to vaccines to behavioral research, science has been under an intense spotlight as part of public discourse in recent years. The pandemic in particular highlighted new opportunities to accelerate research, as well as indicating some potential pitfalls in communication.
To understand the changing landscape around confidence in research, Elsevier has launched a global Confidence in Research collaboration. The initiative will include a landmark global survey of 3,000 researchers conducted by Economist Impact — experts in identifying actionable insights — and propose a set interventions to support researchers in their efforts to advance knowledge that benefits society.
Among the most important figures in the research landscape are librarians. While the role of a librarian varies from country to country, and institution to institution, librarians are often central to delivering trustworthy information to researchers and helping them navigate that information.
In recent years there has been a rise in the use of preprints, including the Elsevier-owned preprint server SSRN. Preprints enable researchers to share early-stage research widely before it goes through peer review. It’s an opportunity to speed up the scientific process and for researchers to provide insights into their areas of focus.
But how should librarians approach them if their responsibility is to source and provide reliable information?
Prof Michael Seadle, Director of the Berlin School of Library and Information Science and Professor of Digital Libraries at Humboldt-Universität zu Berlin, shared his thoughts in a recent interview with Elsevier Connect.
By way of example, Michael pointed to a recent prize-winning mathematics paper that was published in SSRN with the author declining the opportunity to publish in a peer reviewed paper:
Many of the library customers – if I can use that terminology – don’t exactly care where the paper is – they just want to get the content. One of the things we tell students who are going to work in libraries is that you want to make all forms of information that are reliable available. The reliability factor matters.
Michael pointed out that some librarians make the case that traditional journals have an additional peer review process, which helps provide that “reliability” factor, whereas preprints do not. Michael acknowledged that this could be the case in certain fields, but that the overall picture was more nuanced. He continued:
That peer review element certainly matters in social sciences and in medical sciences. But one of the arguments I’ve heard regarding the lack of review in archives and preprints is that if something is sufficiently mathematical, it’s kind of obvious if it’s wrong. If the logic doesn’t work, you can see it yourself.
He also noted that some journals, especially in economics, treat the feedback preprint articles receive as a form of review:
I’m told that some economics journals won’t publish your work unless it has had some exposure on SSRN, so that they have had that feedback. So in some ways, preprints are part of an ongoing discussion about the whole review process and what that looks like.
Indeed, while the process of reviewing research is integral to the concept of confidence in research, as Michael pointed out, peer-reviewed journals are not exempt from problems:
Peer review is being questioned right now partly because of what happened during COVID with medical journals, including some of [Elsevier’s journals], where papers were published based on data that had been purchased from unreliable sources. You have a pressure to get things done quickly, because of course this is very important, and time is of the essence, but it makes it harder to do a thorough investigation of the data. Peer reviewed journals are not foolproof.
Michael agreed that in addition to ensuring their researchers have access to reliable research, librarians also have a role to play in helping researchers evaluate that reliability:
Journals have an important role to play in quality assurance. But you have a number of new publishers who will just publish anything as long as a fee is paid, or you’ll have a service that indexes any research that has one positive review.
In either of those circumstances, such articles are indexed by Google as peer reviewed research and surfaced for anyone to read.
Is it easy to tell whether that’s reliable research?” Michael asked. “Is it a form of preprint or is it predatory publishing?
It’s surely preferable to have clarity about whether something is a preprint, so you can understand where it is in the research process, or whether it’s a published article that’s been through peer review. Librarians can help provide a serious evaluation of that.
Revisiting reward and recognition
Another topic that has emerged from Elsevier’s Confidence in Research collaboration is the idea that the research ecosystem needs to transform its mechanisms of reward and recognition. Many in the world of academia will be familiar with the aphorism “publish or perish,” referring to the pressure to publish research papers in order to succeed in your career. Michael agreed that this was something that needed to change.
“Interestingly, it’s not so much the case here in Germany, but certainly the emphasis on publishing is part of the evaluation process for academics in the UK and the US,” he said. “There are questions being raised now as to whether or not that’s a good thing. An Impact Factor might be more of an indication as to whether a subject is popular rather than whether a paper or a journal is high quality.”
Can librarians educate researchers about the reliability of AI?
An issue that often surfaces when evaluation and peer review are discussed is the role of artificial intelligence. When Elsevier conducted its Research Futures 2.0 report in early 2022, most researchers surveyed continued to object to AI peer review, with nearly two in three unwilling to read such articles (58 percent). That stance is reflected in Michael’s own response:
If people don’t know that they’re using AI tools, or what the limits are, they’re going to be dealing with things they don’t understand. And that is very dangerous for evaluation.
With that in mind, there is the question of how much of a role librarians should play in educating researchers about the reliability of such technology.
That’s an interesting one,” Michael said. “They can definitely play a role in communicating the challenges of certain technologies in the broadest sense, but we have to realize what the limits are.
Those limits, Michael said, may be the extent to which professors or doctoral students prioritize the advice of librarians, and that varies from country to country. In places where the librarian’s advice is acknowledged, there is an opportunity to drive understanding of what constitutes reliable research.
Librarians in some institutions have really deep experience in particular areas,” he said. “If people realize they can talk to their librarian rather than treating the library as a warehouse for information, they find – especially in the prestigious institutions – someone with real depth of knowledge. So we could be teaching people in more depth about how to deal with information and evaluate it.
