x
Sage Policy Profiles scans a database of 10 million documents to show researchers where their papers have been cited.
Research cited in policy reports can help scientists demonstrate impact in promotions.Credit: fermate/Getty
Ever wondered whether your research has influenced policy? Now, there’s a free tool that allows you to check.
The aim of Sage Policy Profiles, a web-based tool launched by Sage Publishing, is to empower researchers to demonstrate their impact or influence on policy when completing grant applications or being interviewed for promotion and tenure, its creators say.
Launched on 7 December, the tool allows any researcher to register and search their name or unique ORCID identifier. It then sifts through a database of around 10.2 million policy documents — the world’s largest such index — compiled by the UK start-up firm Overton, The tool returns a summary of policy documents that cite that researcher’s papers or mention their name, and a map showing where the citations originate and when they were made. Users can export that data in a spreadsheet or as a PowerPoint file.
“It’s a good first version of the tool,” says Robin Haunschild, a bibliometrician at the Max Planck Institute for Solid State Research in Stuttgart, Germany, who tried Sage Policy Profiles at Nature’s request. “It’s a nice service from Sage to the community. I think they should improve the tool over time to make it more helpful to users.”
A spokesperson for Sage, based in Thousand Oaks, California, which funded the tool as part of a partnership with Overton, says that Sage has no plans to charge for the tool. The spokesperson declined to reveal the amount Sage paid to Overton, but said the publisher was interested in understanding the degree to which social-science research is cited in policy documents.
Sage hopes to continue investing in the tool if researchers find it useful, the spokesperson says. “What we wanted to do here is help shift the research-impact conversation beyond scholarly citations.”
Biegzat Murat, a science-policy scholar at Leiden University in the Netherlands who has written about how to interpret citations in policy documents1, also tried out Sage Policy Profiles. “I really like it,” she says. Murat notes, however, that the tool allows users to access only their own citations and mentions, rather than search for the citations of certain papers or those of other academics.
Another shortcoming is that the tool often points users to a document’s list of references rather than the page on which their research is cited, Haunschild notes. “This is not particularly helpful,” adds Haunschild, who has analysed2 the extent to which policy documents mention papers indexed by leading databases.
Euan Adie, Overton’s founder and director, based in London, acknowledges that the firm’s database mainly contains documents released in the past decade, relies on papers freely accessible online and has a Western bias.
That means Overton probably has gaps in the global south, particularly nations where policy documents are often published in local languages. Around 60% of the policy reports in Overton’s database are in English, Adie says.
A 2021 study3 found that just under 6% of studies indexed in the Scopus database between 2008 and 2016 are cited by policy documents in Overton’s database, but the proportion differs drastically between disciplines. Adie notes that just because a study is cited, it does not mean it has influenced policy — it could be mentioned to offer background information.
Overton also has a broad definition of what constitutes a policy document. This include reports, guidelines and working papers written for or by policymakers and published by sources such as governments, non-governmental organizations, intergovernmental organizations and think tanks.
“A lot of people will vehemently disagree that policy citations are the only way that you should be assessing policy impact, and they’re completely right,” Adie says. So the tool also scans for mentions of researchers’ names alongside their affiliation, he adds, which can be relevant for academics who were part of an expert panel or those who provided evidence before a parliamentary committee.
Haunschild thinks it will be tricky to use Sage’s output alone to demonstrate impact in interviews or grant proposals. “It might be helpful if Sage would include some kind of context to these numbers: for example, how many policy citations are occurring on average for researchers of the same field?”
