Last month, European research funders collectively called for research publications to be made free, fully and immediately; so far, 14 funders have signed up. Before that, at least 50 funders and 700 research institutions worldwide had already mandated some form of open access for the work they support. Federally funded agencies and institutions argue that taxpayers should be able to read publicly funded research, and that broader accessibility will allow researchers whose institutions do not subscribe to a particular journal to build on existing research.

However, few empirical analyses have examined whether work supported by funding agencies with such mandates actually is open access1–4. Here, we report the first large-scale analysis of compliance, focusing on 12 selected funding agencies. Bibliometric data are fraught with idiosyncrasies (see ‘Analysis methods’), but the trends are clear.

Of the more than 1.3 million papers we identified as subject to the selected funders’ open-access mandates, we found that some two-thirds were indeed freely available to read. Rates varied greatly, from around 90% for work funded by the US National Institutes of Health (NIH) and UK biomedical funder the Wellcome Trust, to 23% for work supported by the Social Sciences and Humanities Research Council of Canada (see ‘Mandates matter’).

Data sources: Unpaywall/Clarivate Analytics Web of Science/ROARMAP. Analysis: V.L. & C.R.S.

Our findings have policy implications. They highlight the importance to open access of enforcement, timeliness and infrastructure. And they underline the need to establish sustainable and equitable systems as the financial burdens for science publishing shift from research libraries to authors’ research funds.

Free for all

Funders with open-access mandates have varying incentives, opt-out mechanisms, copyright protections, deposit guidelines and other associated infrastructures and requirements. These affect when, how and how much work is made open. Our analysis did not assess licensing and instead counted articles found to be freely available to read. We assessed whether access was gold (available on a journal website) or green (available in a repository, such as PubMed Central, the preprint server arXiv or elsewhere, sometimes with a delay or ‘embargo’ of up to a year after publication). About half (47.5%) of open papers are both green and gold (see ‘Green and gold’).

Data sources: Unpaywall/Clarivate Analytics Web of Science/ROARMAP. Analysis: V.L. & C.R.S.

Both the NIH and the Wellcome Trust state that they will withhold or suspend payments if articles are not made open access, although it is unclear whether they have done so. These agencies provide convenient repositories for depositing articles: PubMedCentral for NIH-funded work, and Europe PubMed Central in the case of Wellcome. Their policies encourage compliance and allow authors to publish in journals that do not permit articles to be available immediately without a subscription. Although articles must be in a repository at the time of publication, free access can occur later. For example, a paper with a 12-month embargo published in the March 2016 issue of a journal would become freely available in the repository in March 2017.

Funders that allow authors to deposit papers after publication see lower rates of compliance, presumably because authors lose track of this obligation. The Canadian Institutes of Health Research (CIHR) mandated deposit on publication from 2008 to 2015, but dropped this requirement when the three main Canadian research councils adopted a joint, harmonized policy. Compliance for CIHR-funded studies went from around 60% in 2014 to around 40% in 2017.

Other funders that have lower rates of compliance than the NIH and the Wellcome Trust provide less enforcement and infrastructure. For example, the US National Science Foundation (NSF) called for “voluntary compliance” with open-access mandates until early 2016 — its rate was around 47% in 2016. Its repository, which uses infrastructure developed by the US Department of Energy, has less visibility and fewer articles than PubMedCentral does. However, that might soon change, because deposition in this public repository is now mandatory for papers arising from NSF funding awarded after January 2016. Compliance at the CIHR is hampered by similar barriers. Unlike PubMed Central in the United States, PubMed Central Canada was never the dedicated infrastructure for Canadian medical papers. The Canadian repository faltered and then closed in February this year, and no strong environment of enforcement has arisen. Factors include lower funding in Canada compared to the United States, which makes it harder for authors to allocate funds for article-processing fees.

The United Kingdom has seen a steep rise in open-access compliance across all agencies (see ‘Very varied access’). Rates at all four of the UK research councils studied went up by at least 20 percentage points between 2009 and 2016; the Engineering and Physical Sciences Research Council went up by 50 percentage points. This follows the publication of the Finch report in 2012 (see go.nature.com/2yojrkc) by a working group of academics, funders and publishers that was established in 2011 by David Willetts, then the UK science minister. It strongly recommended increasing access to research through article-processing charges and gold open access rather than by archiving papers in repositories. For the next assessment of research institutions in England in 2021, major UK funders have now decided to consider only open-access publications.

Field culture

We find variations by discipline, with nearly full compliance in biomedicine, clinical medicine and health research. The social sciences, chemistry and engineering all show lower rates (see ‘Funder effect’). Within the same discipline, compliance varies drastically by funding agency. For example, in chemistry research, 81% of work funded by the NIH is publicly available, whereas that is true of only around one-quarter of chemistry studies supported by the NSF and CIHR. Different funders support different types of work, but the variations we found also remain consistent within sub-disciplines (see Supplementary Information, Figure S5). Although researchers cite norms and needs within disciplines as a reason not to comply with open-access mandates, we believe that the funding agency is a stronger driver of open access than is the culture of any particular discipline.

Analysis methods

We first identified the funding sources of papers using the published acknowledgements (mandated by most funders). These have been indexed by the Web of Science (WoS) since 2008 for science and medicine, and since 2015 for social-sciences articles. There is no uniform format, so we looked for variations of agency names (such as ‘NSF’ and ‘National Science Foundation’) and aggregated these.

Next, we used Unpaywall, a platform that helps researchers to find open-access articles. It identifies the population of scholarly papers using the list of unique digital object identifiers (DOIs) registered by Crossref, a non-profit indexing organization. Unpaywall mines all journal websites listed in the Directory of Open Access Journals, along with databases such as PubMed Central and 50,000 other journal websites and repositories. It intentionally excludes papers that are available on social-networking sites (such as ResearchGate) or illegally (such as on Sci-Hub). As of April 2018, Unpaywall provided the open-access status of nearly 96 million scholarly documents.

Of the 12,495,074 journal articles we matched with Unpaywall using DOIs, 1,352,918 acknowledged funding from 1 of the 12 funders we identified.

To determine rates of compliance, we matched Unpaywall data to our set of WoS articles and analysed them by funder and discipline. WoS includes papers published in about 12,500 journals annually, so some funded work is in journals not covered by our analysis, especially in the social sciences and humanities. Our ability to assign funders to papers is imperfect, given the various ways in which funder names appear and because authors do not always provide funding information. Rates of estimated compliance are likely to be conservative; there might be funded papers that are freely available online but which could not be found by Unpaywall.

Next steps

If funding agencies have their own data on compliance, the information should be openly published so that it can be used in assessments of the march of open access, such as ours. That would also allow comparisons to be drawn. Future research on compliance with open-access mandates should evaluate the utility of other data sources, such as Scopus, 1findr, Kopernio and Dimensions (run by Digital Science, a firm operated by the Holtzbrinck Publishing Group, which has a share in Nature’s publisher. Nature is editorially independent of its publisher). We must also create stronger reporting systems so that these data are more readily available for analysis. This involves collaboration between funders, publishers and indexers. Reporting should allow for analyses at the level of funded projects, which would provide information on the time between funding and open dissemination. On a broader level, more research is needed to understand what makes scientists comply with funder mandates and why.

Ultimately, open access needs a sustainable financing model. Libraries and other organizations have historically borne the cost of publishing through subscription fees. Gold open access displaces those costs on to authors (who often need to allocate funds from their research budgets to cover publishing), even as libraries continue to shell out for subscription fees. The cost of publishing in open-access journals ranges from less than US$100 to more than $5,000 per article, with dominant publishers such as Elsevier averaging $2,612 per paper in article-processing charges and Springer Nature (which publishes Nature) averaging $1,913 (see go.nature.com/2cn3zuy). The system as a whole risks charging multiple actors for the same product, and could price some places and people out of publishing.

Advocacy must be balanced with evidence in the open-access debate. Our research demonstrates that funders can clearly shape compliance through their mandates, and that this compliance needs to be monitored. Real barriers — such as infrastructure and funding — must be overcome to make mandates efficient. However, the rhetoric surrounding disciplinary barriers might be more a myth than a reality: when the proper structure and incentives are in place, researchers comply.

To move the conversation forward, we need a greater sense of the implications of open access on the scientific system’s financial structure. We must study how certain publishing models will put pressure on some parts of the system while alleviating it from other areas, or even enriching them. We need to ensure that the mandates are sensitive to financial inequity across countries, disciplines, institutions and researchers.

Universities, industry and funding agencies should think collectively about robust and scalable models. Cooperation and foresight are the only ways to ensure that everyone has open access to research — both for readers who want to consume it, and for authors who wish to publish it.