Research Nexus on Crossref

Schema 5.5 now available: adding CRediT, new record types for blogs and posters, and more

Patricia Feeney — Thu, 09 Jul 2026 00:00:00 +0000

Research is rarely limited to a single contributor performing a single role. Behind every research output are people contributing in various ways: software development, data analyses, methodology design, and much more. Often, the same person contributes in several of these ways. Until now, Crossref metadata could only capture part of that picture, but this is changing with Schema 5.5.

Crossref Schema 5.5 includes several improvements across different content types, but its most significant enhancement is the expanded support for contributor roles through the introduction of multiple roles per contributor, option to specify the corresponding author, and compatibility with the CRediT (Contributor Roles Taxonomy): a community-owned taxonomy of 14 contributor roles, which has been adopted and made available in multiple languages.

These enhancements allow members to describe research contributions in much greater detail, creating richer metadata that better reflects how research is actually produced, and supporting greater accountability and more comprehensive research assessment.

If your workflow already distinguishes between different kinds of contributions, Schema 5.5 gives you a way to record that detail more accurately using the CRediT taxonomy values. CRediT can be adopted gradually, where it fits your editorial or production workflow.

Figure 1: Until now, contributors could be assigned a single contributor role using Crossref’s existing contributor role vocabulary. In Schema 5.5, members can indicate that the same contributor was responsible for different roles, such as corresponding author; writing: reviewing and editing; and data curation.

Existing deposits remain fully supported, and members can continue using the current contributor role attribute while planning implementation of the new repeatable role type element. For our members, who have been using CRediT in their workflows already, as ever – we encourage updating your metadata when practicable.

Why this update is kind of a big deal

This update gives more accurate credit to all of the people behind research outputs. Crossref vocabulary includes roles that aren’t recognised in CRediT, and vice versa. Capturing richer contributor metadata recognises contributions that may not be visible in a single author line and improves transparency around how research is produced, thereby enabling downstream systems to interpret that information more reliably. The update also offers better interoperability with CRediT, which is well recognised across the scholarly ecosystem.

Figure 2: Schema 5.5 is an expansion of Crossref contributor metadata. Members can describe contributors using Crossref’s existing contributor role vocabulary, as well as the internationally recognised CRediT taxonomy.

In turn, this strengthens metadata reuse across repositories, discovery services, funders, institutions and other infrastructure providers; and supports evaluation, reporting and discovery workflows. Better contributor metadata strengthens the connections that make up the Research Nexus.

What else is included in Schema 5.5?

Beyond the expanded contributor support, Schema 5.5 includes several additional enhancements across the metadata schema.

1. Updates to report series metadata

Support has been added for metadata elements that were previously missing from report series records, including Crossmark, funding, and licence information.

2. Posted content improvements: now including blogs and posters

Posted content includes preprints, eprints, and other types of content that have been posted to a stewarded host platform. We’re all about persistence, so it’s vital that everything registered with us be maintained. Note that accepted manuscripts are not considered posted content. Schema 5.5 refreshes posted content sub-types by introducing blog and poster.

At the same time, we are “retiring” working paper, dissertation, and report from posted-content sub-types. Over time, these have been developed into separate record types that benefit from richer, dedicated schemas.

Finally, archive locations can now also be included for posted content records.

3. Expanded archive support

A new archive location, CINES, has been added to the list of supported archive providers.

4. Clinical trial metadata across more record types

Clinical trial information is no longer limited to journal articles and conference papers. Schema 5.5 extends support across additional content types, including books, datasets, dissertations, reports, posted content, standards, and pending publications.

Schema adoption

Taken together, the updates in our latest schema support more holistic recognition of contributions to the research and its communication, as well as greater accountability and integrity in related processes.

To support gradual adoption, Schema 5.5 maintains backwards compatibility with existing deposits. Members can continue using the current contributor_role attribute while preparing to implement the new repeatable role element. We have prepared a migration guide to help members transition to Schema 5.5.

As you prepare to adopt Schema 5.5, we encourage members to include contributor roles whenever they are available from editorial workflows and to use recognised vocabularies consistently, including CRediT roles where appropriate.

Building better connections: the story of Crossref's metadata development

Patricia Feeney — Thu, 18 Jun 2026 00:00:00 +0000

Three years ago, we asked our members what they needed from Crossref’s metadata. We received confirmation that we were going in the right direction, as well as some new ideas to explore. This helped set the course for our metadata development work since then, and continues to guide where we’re headed next.

Every metadata update we make is driven by the same set of priorities: supporting metadata that reflects our organizational truths, focusing on what metadata our members can actually provide, and aligning with best practices, vocabularies, and standards that our wider scholarly community has established. More recently our Metadata Advisory Group has helped us explore both the minutia of working with metadata as well as larger ideas around the value and impact of the metadata we support.

What We’ve Accomplished

Our schema 5.4 update included several new or expanded types of metadata. First, citation metadata can now be labelled with a publication type. This means when a work cites an article, a preprint, a dataset, or software, that distinction is clear, helping make citations without an accompanying DOI metadata record easier to identify. Second, version information is now supported across all record types, giving the scholarly record a more precise handle on exactly which version of a work is being described.

We’ve also made two meaningful improvements to how funding relationships are captured. ROR IDs are now supported as funder identifiers in both our standard metadata schema and our grants-specific schema. Also, Grant DOIs can now be explicitly identified within funding metadata, making it possible to draw clearer lines between research outputs and the grants that supported them.

What’s happening now

A significant update is nearly here. Schema 5.5 will expand contributor metadata to support multiple roles per contributor, and will introduce support for CRediT — the ANSI/NISO taxonomy for contributor roles. This means that an individual’s complete contribution to a research output can finally be described in our metadata, rather than flattened into a single role or omitted entirely. The schema isn’t released yet, but the final version of the XML schema is available in our GitLab repository for those who want to get a head start.

We’ll next begin implementation work for a new Grants schema (0.3.0). This update will remodel investigator names to include a new role (beneficiary) as well as an organizational grant recipient, making it possible to include recipient info for grants given to organizations. Grant records include project metadata, so this update will also include support for RAiD, a persistent identifier for projects. The XML schema for this update is also available in a GitLab repository.

What’s up next

Our next planned major update will build substantially on the contributor work in version 5.5. In the next version (6.0) we will remodel names to expand our current limited structure to support a variety of name types as well as alternate names. We’ll also expand the contributor identifiers we collect to include ISNI and Wikidata identifiers, better supporting contributors for whom an ORCID is not possible. Our organizational contributor will be remodelled as well to include organization-level identifiers like ROR.

We’ll also introduce statements to Crossref metadata. Statements will allow members to include free-text statements including funding acknowledgements, ethics declarations, AI usage disclosures, and other important contextual information that doesn’t fit neatly into structured fields.

Other updates include expanding our support for abstracts encoding beyond JATS to include ONIX, BITS, and a generic markup option, and implementing better in-schema validation to avoid surprises at the time of deposit.

Progress means letting go of the past. We’re planning to deprecate all schemas prior to version 5.3.1 by the end of 2027, to be carried out in phases as outlined in our deprecation blog post. This is a necessary step to keep our infrastructure sustainable and to ensure members are working with schemas that reflect current capabilities and standards.

Looking further ahead

Beyond 6.0, we’re exploring further support for provenance in metadata (to establish who is doing what to a metadata record), a rethinking of how we handle dates so that they better capture the lifecycle of a research object, better support for research objects we don’t yet fully support, and making our metadata inputs more consistent. The Metadata Development roadmap has full details on what’s being explored and prioritized.

Each of these updates contributes to Crossref’s research nexus vision: strengthening connections between funders and research, more accurately capturing and recognizing contributor roles in the scholarly record, and collecting free-text content to fill in the gaps that structured metadata alone can’t address. Better metadata means better research integrity and more trustworthy infrastructure for everyone who depends on it.

Two billion citation links in Crossref help research travel further

Kornelia Korzec — Tue, 26 May 2026 00:00:00 +0000

We’ve recently reached an important milestone for the research nexus: the works in our metadata corpus are now connected with over 2 billion citation links! This is a great opportunity to share a dedicated dataset and discuss why these are important for science.

The reference metadata is a lifeline of discoverability. Scholars use citations to critique and build on existing research. They acknowledge the contributions of others through references. Our members can then deposit those references as part of metadata with Crossref, and we use those to link the cited and citing objects. This results in complex thematic networks that can be explored by interested researchers. Many tools for research discovery use the linked reference metadata in Crossref to support searches of related content.

The citation links are derived from bibliographic references in the metadata of one work that include DOIs of materials it cites (scholarly works, data, code, etc.). It’s always best if the members can deposit these relationships in full. In a recent post, we shared that nearly half of these links are asserted by our members through metadata deposits, and the other half are created thanks to our automated matching. This form of metadata enrichment happens when members include some information about the references but without the DOI of the cited work, and it’s enough to automatically find and add that DOI. The enrichment supports making data more useful for the community.

The most important impact of citation links is the increased discoverability of connected works. Reference metadata is an important tool for improving visibility and readership of our members’ content. These links are also the foundation of our Cited-by service, which enables implementing members to display citation counts of the work they published on their landing pages.

The chart below shows the cumulative count of citations over time, by the created date of the citing DOI’s record. These include records linked by DOI either through member-submitted metadata or matched by Crossref, as well as records that are unmatched. Unmatched records can include records that we were unable to match with the information we have, but also records that truly have no DOI to link to. You can explore the full citation dataset of all 2 billion citation links between Crossref DOIs available now as a (somewhat hefty) download.

Cumulative count of references deposited to Crossref by created date of citing DOI

The push for open citation data is something that has unfolded over the last few decades, making more and more of these relationships public. Notably, the growth in citation links reflects not just the output of new scholarship, but also a sustained effort to extend coverage of the historical scholarly record. We can see evidence of this playing out over time by looking at our historical data—periodic snapshots of Crossref’s metadata going back to 2019. When comparing successive snapshots and examining the publication dates of citing and cited works, we can classify each newly appearing citation as either a new paper citation, or a retrospective one. A new citation is where the citing work was published since the previous snapshot, representing real growth in the scholarly record. A retrospective citation is where both papers already existed but the link between them had not yet been captured by Crossref, and these represent indexing catchup rather than new publishing activity.

The chart below shows the cumulative count of citations added in each category since 2019. In the early years of our data, retrospective backfill was the dominant source: the blue line climbs steeply from 2019 to 2021 as a large volume of previously uncaptured historical citation relationships entered the corpus. Over time, however, that rate of backfilling has levelled off. New paper citations, meanwhile, have grown steadily throughout the period, and by 2025 they surpassed the cumulative retrospective total. The open citation ecosystem continues recovering historical links, but the citation network’s growth is now increasingly driven by the natural momentum of scholarly publishing itself.

Cumulative citations added to Crossref by type, 2019–2026. Retrospective citations (blue) represent links to and from works that existed before the previous snapshot; new paper citations (green) come from works published since the last snapshot.

Combined with other metadata for more context, reference metadata supports bibliographic and meta-research on different aspects of the scholarly process, and can support judgements about research integrity and conflicts of interest.

Stereotypically, when talking about references, we consider links to published works (whether preprints, journal articles, or books). However all types of records in Crossref can be cited. Thanks to the changes in our latest schema, members can now signal the types of content that is being referenced. And with our new Data citations endpoint, the community can explore specifically links from Crossref-registered records to research data, including citation links to works within Crossref, as well as DataCite’s corpus.

Close to half of all records registered with Crossref still have none or not enough reference information to make such connections. We invite members to regular Metadata health-check webinars to support them in improving completeness of their records for increased transparency and visibility.

From 1990 to today: connecting HFSP's grant history to the research nexus

Rocío Gaudioso Pedraza — Thu, 14 May 2026 00:00:00 +0000

For a funder with over thirty years of funding history, making all of their funding metadata openly available is no small undertaking. In this conversation, I chat with Guntram Bauer, Chief Scientific Officer at the Human Frontiers Science Program (HFSP), about how the organisation is working to register decades of grant data with Crossref, the challenges of linking historical awards to published research outputs, and what open, structured funding metadata means for accountability to member countries and the wider scientific community.

RGP: Why did HFSP join Crossref and decide to share its funding metadata openly?

GB: Accountability starts with openness and transparency. If we are not open as a global funding organisation, we can’t convey that message to our constituents, our grantees, and our community. Before we could share our metadata openly through Crossref, we would share our funding activities through annual reports with a simple list of awardees and projects. In the digital era there are new possibilities to do this better and demonstrate to our member countries what we do with the funds. For us, it is a very helpful way to fulfil our obligations and the due diligence that is expected of us.

We started registering decade-old grant metadata, something we had always wanted to do but that before the current waiver would have meant a big financial undertaking.

– Guntram Bauer, Chief Scientific Officer, Human Frontiers Science Program

RGP: How did you find integrating the Grant Linking System (GLS) within your existing workflows?

GB: It was very straightforward. We work with Proposal Central, through Altum, which also acts as our Crossref sponsor, to help HFSP handle grant operations and related metadata, which makes it all very easy. One thing we did think carefully about was what the grant landing page would look like: what information people would see when they clicked on a grant DOI. Before Altum, we couldn’t even add an ORCID to our workflows, let alone a Crossref grant DOI! Having structured metadata to support transparent reporting to our own supporting member countries has been very valuable.

This step was independent from our funding policies. Registering funding metadata and assigning Crossref grant DOIs was paralleled with informative campaigns and direct information to our awardees to inform them about the new way of acknowledging funding.

RGP: Can you tell us about HFSP’s experience registering historical grant data?

GB: When HFSP started working with Crossref to benefit from the Grant Linking System (GLS) and make our funding metadata openly available, we began by registering data about our fellowships. More recently, in the context of the newly introduced waiver for historical grant data, we started registering decade-old grant metadata, something we had always wanted to do but that before the current waiver would have meant a big financial undertaking.

RGP: Why are you so interested in registering historical grants?

GB: HFSP has been registering funding with Crossref for a while, and the recent announcement of the two-year waiver made us decide to make all our historical data available. That’s going back all the way to 1990! We have been doing it little by little. We are keen to introduce as much funding metadata as possible into the system, to improve our transparency, but also to add to the research nexus and be able to link our funding to outputs, even the decades-old ones.

RGP: Are there any challenges specific to registering historical grants?

GB: There are a couple. When registering a historical grant, from 2005 for example, the publications that came out of that funding couldn’t be linked to the Crossref grant DOI since it didn’t exist when it was published, so the connection is missing. This is one of the key added values to the GLS, which in our view is the ability to track the impact of funding. Additionally, grantees who were awarded their funding many years ago and are only now receiving a Crossref grant DOI may not be as primed as current grantees to use that DOI when acknowledging their outputs.

Encouraging recent grantees of their new Crossref grant DOI so they can start acknowledging their outputs is one thing. However, persuading earlier grantees to go back and inform publishers of a new grant DOI involves a lot of actors and many potential points of failure. How can those connections between grant DOIs and related outputs be established, beyond direct communication with grantees?

RGP: That is indeed a challenge that we are well aware of! What excites us going forward is matching those records back to historical outputs with automated strategies. These will increasingly allow us to match, connect, and insert relationships between published outputs and Crossref grant DOIs. This is only possible when both the output and the grant are registered with Crossref and there is enough funding metadata in the output record to make the match. The more metadata available about an award (such as award number and funder name) in the output’s metadata, the more matches we can make, and the more confident the community can be in them.

RGP: Is the GLS changing how you approach reporting or shifting the culture within the organisation?

GB: For us, it’s an evolving situation. There is definitely a demand from our member countries to be more open and transparent, for the accountability reasons we mentioned. As the only truly globally operating funder supporting basic rather than applied research, demonstrating our impact across such a diverse membership is not always straightforward. If anything, the GLS and Crossref grant DOIs can help us in making our reporting evidence-based and follow the impact of the funding.

We are very grateful to Guntram Bauer for his perspectives and insights.

Voices from Crossref Metadata Sprint in São Paulo

Luis Montilla — Tue, 31 Mar 2026 00:00:00 +0000

This year, we placed a spotlight on the Latin American community, hosting the second Crossref Metadata Sprint in São Paulo, Brazil from 4 - 6 March 2026. In our first tri-lingual event, we brought together 31 participants from Argentina, Brazil, Colombia, Ecuador, and Mexico. Our goal was to foster community co-creation using the open scholarly metadata. The Sprint was an opportunity to pose questions, share ideas, collaborate on research, and propose innovative solutions that enhance the use of metadata in scholarly communication and beyond.

Read on for more details about the content of the Sprint, and the resulting projects. You can also register to join our Sprint Showcase call on 22nd April to hear directly from the team about their creations.

We were excited to receive more than 100 expressions of interest for this sprint. We’re excited to see the growing enthusiasm in this space and will find new ways to channel it (watch the space!). We sought to balance that interest with ensuring a productive size of the group for the sprint. Our participants included librarians, researchers, grad students, developers, journal editors, and scholarly communications professionals. They engaged in rich, multilingual conversations about the different editorial practices across the region - dynamically jumping between Spanish, Portuguese, and English.

We coordinated our activities and ideas with the SciELO Brazil team, who also participated in the Sprint. SciELO is one of the most recognized scholarly organizations in the region and an important source of open scholarly content and metadata; articles, books, preprints, and datasets published in different languages.

Some of the participants of the 2nd Crossref Metadata Sprint, and Crossref Staff

Susan Collins, Luis Montilla, Isaac Farley, Jason Portenoy, and Leandro Contreras from Crossref acted as facilitators, providing general support, answering questions, and clarifying the ins and outs of Crossref tools and interfaces.

Using the key learning from our first Metadata Sprint, we created opportunities for the group to interact and get acquainted remotely, before the event, to help them hit the ground running and maximise the value of the time we spent together. It really paid off in terms of quality and progress of the sprint projects.

Projects summary

The projects developed also have their own mini stories. When we opened the call for expressions of interest, we invited individual pitches for projects that make creative use of Crossref metadata to answer questions and support solutions to problems faced by our communities locally. During the preparations to the event, participants refined their proposals and coalesced into project teams. This is the list of the projects they progressed during our Sprint:

Crossref Metadata Refiner: A tool that queries the Crossref API for any publisher prefix or ISSN and computes a Crossref Health Score for each DOI, crossing citation count with metadata completeness. The result is a prioritized action list for the member: fix the most-cited papers with the worst metadata first.
Improving Error Messages in Crossref Submission Details: A proposal for the redesign of the presentation of error messages in the Crossref Submission Details interface. Instead of showing only raw XML diagnostics, display structured and human-readable messages.
Scholarly Retractions and Corrections Tool: The development of a user-friendly form for Crossref members to fill in correction and retraction metadata.
Best practices for metadata journals in OJS, according Crossref schema: A guide that shows the correct completion of metadata fields in order to avoid errors already seen in everyday work, and facilitate the correct registration of metadata with the necessary quality to avoid errors in XML export.
Metrics for publishers using the Crossref API: A Power BI dashboard that summarises members’ metrics and indicators based on metadata retrieved from the REST API.
Metadata Integrity Check: A proposal for the implementation of a second stage of XML integrity verification after using the XML parser. This additional step would allow preliminary verification of the presence of mandatory metadata elements and encourage the inclusion of recommended metadata, such as institutional affiliations and references.
Integration OJS / OPS - Crossmark: This project advanced the development of a plugin for the current versions of OJS (3.4 and 3.5), so that when an article receives a correction, update, or new version after publication, each published version of the same work receives its own DOI, while maintaining structured relationships between these versions.
Automated Detection of Reference Rot in Scholarly Web Citations: An automated evaluation pipeline to retrieve citations containing URLs, then traces redirect behavior, and records HTTP status codes and final page locations. It also extracts evidence from both the citation data and the webpage itself, including titles, creators, identifiers, and other metadata.

We will hold a community call where the teams will showcase their projects. If anything here caught your eye and you would like to learn more – join us on 22nd of April (or register to receive the recording). In the spirit of the São Paulo Sprint, we will hold the call in three languages again! Register now to join our Sprint Showcase call.

I am still in awe at what our community achieved in such a short time in the Sprint: dashboards, clever API integrations, best practices manuals… all of these projects are sure to make a mark. The Metadata Sprint in São Paulo is a testament to my belief that great ideas are everywhere, and I’m happy to have taken part in amplifying them!

Said Leandro Contreras, User Experience Researcher

Like many in attendance, this was my first Crossref Metadata Sprint. I went into the event hoping to make connections, help where I could, and learn from those in attendance. Thanks to Susan and Luis for all their planning and thoughtful leadership and the collaborative spirit of everyone participating in São Paulo, the sprint was three days full of rich co-creating, thoughtful shared problem solving, and lots of fun. It exceeded my expectations! I learned a lot and am eager for the next steps with the projects completed there. I hope to be able to participate in another Crossref sprint again very soon (and, encourage everyone reading to get to one in the future)!

Said Isaac Farley, Head of Support

Voices from the Sprint

During and after the Sprint, participants were actively sharing their experiences online. Here are a few highlights from across the community:

Being in this collaborative space allows us to deepen knowledge, exchange experiences with professionals from different countries and, above all, improve the quality of the metadata of our articles. This technical work, often invisible to the reader, is essential to ensure greater visibility, traceability and scientific impact to RBC’s publications.

Carina Munhoz de Lima - Librarian at Revista Brasileira de Cancerologia, on Linkedin, 5 March 2026.

The session was a valuable opportunity to exchange knowledge, strengthen professional ties and reflect on the current challenges around metadata management and scholarly communication.

Sandra Gisela Martín - Library System Director at the Catholic University of Córdoba, on Linkedin, 10 March 2026.

The best experience of the Sprint was recognizing the potential of combining open infrastructure, reusable data and collaboration, with the accompaniment of Crossref experts.

Sofia E. Calle Pesántez - Research Impact and Scholarly Publishing Consultant, On Linkedin, 10 March 2026.

I am grateful to see Crossref’s concern in personally listening to the demands of the Latin American scientific community!

Alex Mendonça - Client Solutions Manager at ScholarOne, on Linkedin, 5 March 2026.

Its been an absolute privilege representing Galoá at the Crossref metadata Sprint in Brazil… our specific focus during this Sprint has been clear: rolling up our sleeves to elevate the quality of metadata for Brazilian and Latin American scientific publications.

Fabiano Sant’Ana - Founder at Galoá, on Linkedin, 6 March 2026

It was wonderful to connect with so many people working to strengthen scholarly communication in the region, and to meet the Crossref staff who organised such a thoughtful and engaging program.

Zach Coble - Graduate Research Assistant, University of Missouri-Columbia, on Linkedin, 7 March 2026

We developed a free tool for registering retractions in the Crossref metadata Sprint… the motivation for the project was to develop a solution that would make the Crossmark registration process easier and more user-friendly.

Eugênio Telles, Genius Design blog, also on Crossref’s community forum. Browse Eugênio’s gallery of photos on Instagram, 8 March 2026

Participants

None of this would’ve been possible without our enthusiastic participants. Huge thanks to everyone! Here is the full list of those who attended our second Metadata Sprint:

Name	Affiliation
Julia Bottesini	Data-PASS Journal Editors Discussion Interface
Pedro Cacique	Public Knowledge Project
Sofia Calle	Red Iberoamericana de estudios científicos de la edición, evaluación y circulación del libro
Zach Coble	University of Missouri
Edilson Damasio	Universidade Estadual de Maringá
Ana Claudia Ribeiro	Editora E-papers
Eugênio Fernandes Telles	GeniusDesign
Jhonathan de Seixas Miranda	Lepidus Tecnologia
Juan Felipe Vargas Martinez	Journals & Authors
Mariana Garroux Montezuma	Sabesp
Nicolas Mejia Torres	Universidad de La Sabana
Juliana Aaparecida Bolzan	Universidade Federal de Santa Catarina
Deise Katiuscia Xavier Kaisa Oliveira	Centro Universitário de Mineiros
Glicélia Pereira Silva	Centro Universitário de Mineiros
Carolina Tanigushi	SciELO
Amanda de Souza Ramalho	SciELO
Karolayne Costa Rodrigues de Lima	Universidade Federal do Paraná
Sara Jaqueline Santos da Silva	Universidade Federal de Uberlândia
Diego Abadan M. Melgarejo	Lepidus Tecnologia
Natalino Perovano Filho	Universidade Estadual do Sudoeste da Bahia
Sandra Gisela Martin	Universidad Católica de Córdoba
Roberta Takenaka	SciELO
Amélia Galdino Monteiro Amâncio	Instituto de Física Gleb Wataghin
Quele Pinheiro Valença	Universidade Estadual de Santa Cruz
Rafael Dias Da Silva Campos	Universidade Federal do Triângulo Mineiro
Igor Moura Danieleviz e Silva	Lepidus Tecnologia
Heytor Diniz Teixeira	Universidade de São Paulo
María Eduarda Dos Santos Puga	Universidade Federal de São Paulo
Fabiano Sant’Ana	Galoá Science
Flaviane Cristina Rocha Cesar	Centro Universitário de Mineiros
David Antonio Da Costa	Universidade Federal de São Paulo

The second Crossref Metadata Sprint in a nutshell

Reduction of Grant DOI registration fees: a boost for the Research Nexus

Ginny Hendricks — Thu, 08 Jan 2026 00:00:00 +0000

We are pleased to announce that—effective 1st January 2026—we have made two changes to grant record registration fees that aim to accelerate adoption of Crossref’s Grant Linking System (GLS) and provide a two-year window of opportunity to increase the number and availability of open persistent grant identifiers and boost the matching of relationships with research objects.

Launched in 2019 with close input from several funders and other infrastructure organisations, the GLS primarily offers the ability to create and steward Crossref Grant DOIs, along with several benefits such as dedicated grant/award metadata like funding type, value, contributors, and projects, as well as hosted landing pages, tools to create and update metadata, and of course both member-asserted and Crossref-automatic matching of relationships within the global corpus of 180 million other research objects. Essentially, we need to identify what research objects are produced as a result of the award, and these objects could be articles, preprints, data, code, blogs, posters, and more.

This connected network is what we call the Research Nexus, essential for exploring research activity in general, as well as evaluating reach and return on funding and other support like use of facilities/equipment.

A fee reduction and a two-year fee waiver pilot

Following a review by our Membership & Fees Committee, the Board met in December and passed two related motions:

Current-Year (CY) grant registration fee has been cut in half to match other record types: The board approved the adjustment of the Current-Year (CY) grant registration fee down from $2.00 to $1.00 USD, effective 1st January 2026.
Back-Year (BY) grant registration fee is waived through 2027: The board approved a time-limited fee waiver as a pilot for Back-Year (BY) grant registration fees, bringing that per-record fee down from $0.30 to $0.00 for 2026 and 2027.

We aim to boost registration of Back-Year (BY) records and accelerate the growth of the Research Nexus with millions more grant<->output matches. During the course of the two-year pilot, the Membership & Fees Committee and our fee project work that started in 2023 and also brought in other fee reductions, will consider more adjustments across BY registration fees for the benefit of members beyond just funders and beyond just grants.

All Board motions are publicly available and we encourage questions from the community about our governance processes and the decisions on our members’ behalf; email us via feedback@crossref.org anytime, or post on the forum.

Supercharging the Grant Linking System

Leading up to the GLS launch in 2019, we worked with a group of funders and metadata experts to inform the design and implementation of the new service, including a funder governance and fees working group. That was seven years ago, and our Funder Advisory Group now includes nearly 100 funding community representatives the GLS has grown to almost 50 funder members that have registered more than 185,000 open grant metadata records. But they are mostly research councils and agencies or charities from Europe and North America, and we know that for a truly comprehensive and interconnected Research Nexus, more needs to be done to include organisations from all parts of the world. The other key driver is simply to boost more metadata connections; the more grant metadata we gather, the better we can match it to all kinds of research outputs, and this metadata directly feeds thousands of services available in our community, from Dimensions and Scopus, to OA.Report and OpenAlex, as well as funders’ own analytics tools. See our recent report about the latest dataset and of course use api.crossref.org directly.

Relatedly, we just added a new Grant DOI field to our schema for all record types, to give our members a precise and accurate way of capturing funding metadata for all research outputs. With the new lower CY registration fee and a pilot waiver of BY fees for grant records, we hope to boost the creation of more Grant DOIs by more funders from more parts of the world—so that others also see and can build on the momentum and reuse the data in their own tools and services. All actors need to play their role, and Crossref’s part is in running the global linking infrastructure at scale, connecting research objects and making them openly available while ensuring that the barriers for the registration, use and reuse of metadata remain as low as possible.

We feel we’re at a tipping point that only needs a small nudge to truly scale the Grant Linking System.

By waiving BY fees entirely for two years, we’re hoping to see members fill in historical data and create more comprehensive grant<->outcome connections. There is often a long period of time between funding being awarded, and the resulting research objects being generated and communicated. That is why historical grant metadata is so important; we think that there will be many funding outcome relationships and insights just waiting to be uncovered!

Why give funders a fee break and not others?

We’re not ruling out this kind of fee incentive in future for other members and other object types, but that needs more analysis (which we plan to do) and right now, the relatively small number of grant records, combined with a growing need for this kind of metadata, means the changes are small enough to have almost no impact on Crossref’s healthy financial position.

This decision is consistent with the goals of our Resourcing Crossref for Future Sustainability (RCFS) to review our fees to make sure they are equitable and clear, while ensuring Crossref retains a sustainable business model. Our fees can encourage or discourage the community to participate in Crossref. The RCFS project has also resulted in the creation of a lower membership fee tier for the very lowest-resourced members, and the tidying up of things like outlier volume discounts.

The BY fee waiver is positioned as a pilot to allow us to measure its impact over the next two years and feed into the Membership & Fees Committee and RCFS project. We will evaluate the pilot results (i.e. does it indeed supercharge funding metadata connections and adoption?) and consider additional adjustments to other BY registration fees and whether such fee incentives might be extended to other members.

We encourage all funders to take advantage of these reduced rates to contribute to the Research Nexus and help us build a more complete picture of the relationship between research funding and outcomes.

Take a look at the recent case studies from early GLS adopters FWF (Austria), NWO (The Netherlands), FCCN|FCT (Portugal), and Wellcome/EuropePMC, reach out to them or us with any questions, or peruse the GLS community forum!

Highlights of a very busy year: our 2025 annual report

Ginny Hendricks — Thu, 18 Dec 2025 00:00:00 +0000

As we finish celebrating our 25th anniversary, we can look back on a truly transformational year, defined by the successful delivery of several long-planned, foundational projects—as well as updates to our teams, services, and fees—that position Crossref for success over the next quarter century as essential open scholarly infrastructure. In our update at the end of 2024, we highlighted that we had restructured our leadership team and paused some projects. The changes made in 2024 positioned us for a year of getting things done in 2025. We launched cross-functional programs, modernised our systems, strengthened connections with our growing global community, and streamlined a bunch of technical and business operations while continuing to grow our staff, members, content, relationships, and community connections.

Read on for the highlights of a very busy year, grouped around our four strategic themes.

Strategic theme 1: Contribute to an environment where the community identifies and co-creates solutions for broad benefit

Enhanced tools and services

In October, we released an enhanced Participation Reports dashboard that shows metadata coverage across all 180 million records and provides individual member organisations with actionable gap reports to guide them to improve metadata completeness. The new tool provides more complete coverage of all members and resource types, now including funders and grants, with up to 11 best-practice metadata elements publicly tracked.

We launched support for journal articles in the New Metadata Manager record registration form (initially only for grants), which includes built-in reference and relationships deposit capabilities. In the New Metadata Manager, it’s now also possible to search for previously registered DOIs to edit your metadata records. In the coming years, we are planning to expand the new Metadata Manager to support all the many different content types that you can register with Crossref DOIs.

After a long break between regular updates, we have fixed our process for and just released v.1.63 of the Open Funder registry. With the updated process, we’re now able to resume more frequent updates to the registry (while of course still working towards the transition to ROR for funders).

Throughout 2025, we conducted a website information architecture review to improve the information we provide to our members and the wider community. Based on the recommendations from this review, we will be renewing our website and documentation in 2026.

Deprecations and modernisation

‘Old’ Metadata Manager is to be retired at the end of 2025, with users transitioning to the ‘New’ version or to our other helper tools for registering and updating DOIs. All users have been contacted during 2025 and received training on how to use the New Metadata Manager.

We also announced the deprecation of Co-access, which will end in 2026, bringing an end to the service that allowed duplicate DOIs for book content. Users of co-access have been informed and are in the process of transitioning to multiple resolution.

Together with Turnitin and our members, we are working to transition all subscribers to our Similarity Check service to a new version of iThenticate 2.0. We are happy to report that all platforms with integrations with us transitioned to 2.0 during 2025, and we will continue working with our members to get everyone transitioned during 2026.

Eating our own DOI dogfood

In June this year, we were particularly pleased to finally support the registration of DOIs for our own content, this very blog, through partnering with Rogue Scholar. Blogs are a growing format for scholarly discourse and our own blog is no different as it’s the main way that we share guidelines and best practices, as well as news and stories from the scholarly community. With a Crossref DOI for all blogs going back to 2006, we’re setting ourselves up to ensure better future preservation of the discussion and information about Crossref.

Community connections

We delivered 29 metadata health-check webinars over the course of the year, in French, Indonesian, Spanish, and English, reaching 2,166 participants with practical advice on identifying gaps in journal metadata using Participation Reports.

Crossref Accra took place in March as our first in-person event in a GEM country. We also held similar events in Ecuador and Türkiye with Crossref Quito in September and Crossref Ankara in November. At these three events, we welcomed key figures from each country’s library, government, publishing, and academic communities and we learned so much about the thriving communities there, and also that even more dedicated workshops on the specifics of metadata quality improvements would be appreciated.

Our metadata sprint in Madrid in April brought together community members to tackle specific problems collaboratively, with teams exploring coding, documentation, translation, and research using our open metadata. We’re already planning our next sprint in São Paulo for March 2026, and it will be held in three languages: Portuguese, Spanish, and English.

A strategic goal for Crossref is to grow research funders’ adoption of the Grant Linking System, and we produced the first in a series of interviews with funder members this year to highlight how and why Crossref DOIs are fulfilling goals to assess the reach and return of their research support for FWF (Austria), NWO (Netherlands), FCCN|FCT (Portugal), and Wellcome. This year, we welcomed more funders including Fonds de recherche du Québec (Canada) and Independent Research Fund Denmark as part of their national research platform NORA; we look forward to reporting on their experiences and outcomes next year and others as they work towards Crossref Grant DOI adoption.

We continued working closely with PKP and renewed our partnership to help drive better experience for OJS users registering metadata with Crossref. We also delivered a proportion of the metadata health-checks together to maximise the learning opportunities for our members using OJS; and we joined PKP’s Sprint in Oslo to help make improvements to OJS and OMP.

Crossref staff members serve on almost 50 committees, boards, and other community bodies alongside our own direct work. These include in the areas of research integrity, metascience, metadata and PID standards, open science policy or monitoring, development of new models (such as Diamond OA), editorial production, library and institutional publishing, and citation and other metadata analyses. We also work with other DOI Registration Agencies and support the sustainability of the DOI Foundation with an additional annual subsidy. Many DOI RAs are also Crossref Sponsors so that their members can access our unique reference matching service. While we often might advise, we also learn a huge amount from collaborating with the numerous systems and initiatives that make up the wider research community.

Our involvement with developing the Barcelona Declaration on Open Research Information led us to become the fiscal host and to participate in most of the working groups on open metadata. Of particular note this year was the Funding Metadata Working Group round table about moving forward the state of funding metadata, which we co-hosted with Barcelona Declaration colleagues, and three funding bodies, NWO (Netherlands), FWF (Austria), and ANR (France) as we heard from publishers and their vendors about challenges and how to overcome them to increase the quantity and quality of available open funding metadata.

All our community engagement activities have been enthusiastically supported and enriched by our indispensable Ambassadors and our group of now 130 Sponsors, organisations that help thousands of Crossref members with local language and technical support and lower cost access to our membership.

Strategic theme 2: A sustainable source of complete, open, and global scholarly metadata and relationships

Schema developments

The grant schema version 0.2.0 was released in January, adding support for ROR identifiers to identify funders and new funding types for in our taxonomy, including APC, BPC, and infrastructure. All of these funding types can be specified in the metadata of our grant-giving members alongside the existing types such as use of facilities or salary/training awards, etc.

Version 5.4 of our publications schema was released in March, marking our first update in many years and a great opportunity to learn how to do this and make the process more efficient. This release introduced typed references to denote the type of object referenced (dataset, blog, software, etc.), preprint status indicators, and version numbering.

Just last week, we also added a dedicated field for grant DOIs to our publications schema. This means it’s now possible to indicate in an article’s metadata which grant(s) funded the research using the persistent identifier. This is an essential step toward better alignment between grant funding and research, enriching the Research Nexus.

We also launched our new Metadata Advisory Group and they have already devised sub-working groups in three focus topic areas:

Multilingual metadata
Subjects and keywords
Relationships

Public data file

We released the 2025 public data file in March, containing metadata for (at the time) over 165 million research outputs from more than 22,000 organisations.

Inaugural Metadata Awards

In May, we launched the first-ever Metadata Awards to recognise members demonstrating excellence in metadata completeness and enrichment. Winners included Noyam Publishers (Ghana), GigaScience Press (Hong Kong), eLife (UK), American Society for Microbiology (USA), Universidad La Salle Arequipa (Peru), and Instituto Geologico y Minero de España (Spain). The awards will be held biennially going forward.

Metadata Matching project

In April, we launched the metadata matching project with the aim of building a more complete picture of the research nexus over time by automatically identifying missing relationships between entities across the scholarly record. The project’s goal is to modernise Crossref’s enrichment workflows by rebuilding them using modern software development and data science practices.

We are in the throws of developing a consolidated matching workflow that will eventually replace all existing production matching processes, with results exposed through the REST API. All new matching strategies will be rigorously evaluated, and the resulting data will be accompanied by clear provenance information. This project covers six matching tasks:

bibliographic reference matching
funder name matching
preprint matching
affiliation matching
grant matching
title matching

In the meantime, while work continues on integrating matching results into the REST API, we’ve been releasing standalone matching datasets for separate download and analysis. These include relationships between preprints and journal articles, relationships involving research organisations, and relationships between grants and research outputs.

Data infrastructure and Research Nexus participation dashboard

Staying on the data science front, we’ve established an internal data environment that combines all relevant data sources (scholarly metadata, logs and usage data, and external datasets) in their raw forms into a single place. This environment is supported by a suite of modern tools and data processing techniques, enabling data science experiments and analytics pipelines to run effectively at scale.

Building on this foundation, we plan to develop a series of dashboards to monitor the state of the scholarly record over time. These dashboards will feature both work-level and member-level statistics (for example, how many works of a given type have been registered, or how many members are registering grant IDs) as well as more detailed insights at the relationship level (for example, how many bibliographic references have been automatically matched, or how many times ROR IDs are included in funder assertions). Some of these statistics are already available in a public spreadsheet for now, pending the dashboard.

Retraction Watch integration

In 2023, Crossref acquired the Retraction Watch database to make it open data. Initially, this was done through sharing simple CSV files, but this year we have set up a pipeline to feed this information into our REST API, which means that Retraction Watch data is now fully available through the REST API, integrated with Crossref member-supplied retraction and correction metadata. This is the first example of Crossref integrating third-party metadata, and we’re learning a lot about how to best incorporate other datasets in future.

Metadata API and services improvements

From 1 December 2025, we revised rate limits for the REST API to ensure system stability whilst maintaining free access to metadata for everyone. Changes were made to the rate limits for our ‘public’ and ‘polite’ APIs, while the limits for our Metadata Plus users stayed the same. We continue to make all metadata openly available to the whole community.

We also improved how information from our content system feeds into the REST API. A tool we call ‘pusher’—because it pushes information from the content system to the REST API—was rebuilt so that we now have a more reliable transfer of information between our two systems.

While adding to technical improvements, we’ve also worked to better understand the use of and streamline the service offering for paid options. We’ll share more about this year’s Metadata Plus consultation soon. And based on feedback, we have already retired the ‘Query Affiliate’ service, where a handful of organisations still paid us a fee to access our XML API, whereas no credentials have been required for some time.

Strategic theme 3: Manage Crossref openly and sustainably, modernising and making transparent all operations so that we are accountable to the communities that govern us

Infrastructure modernisation

One of our biggest projects of 2025—if not the biggest—was the move from our data centre into the cloud (AWS). For 25 years, Crossref had been running a physical data centre in Massachusetts, USA, but as part of modernising our systems, it was high time to move everything into the cloud. The move to AWS took several months, but we successfully completed this move to the cloud in July this year. We’re spending these last weeks of 2025 fully decommissioning our data centre, which means that we are removing all the equipment we had there and locking the door for the last time.

A part of the move to AWS included moving onto an open-source database solution, PostgreSQL. This reduced our reliance on closed, costly licensed solutions, while also aligning with our POSI commitment to open-source. Running our entire system in AWS provides a more stable, modern approach to our infrastructure, but it also is expensive. We expect to spend about 2 million USD on AWS fees next year, with the majority of this cost coming from REST API usage. Some of the improvements described above will help us manage those costs and better observe traffic patterns.

Our new cloud infrastructure is a bittersweet milestone: while we are happy to not have to rely on a physical presence to support a 24/7 global infrastructure, we also say a sad farewell to our much-loved and long-suffering Sys Admin, Tim Pickard, who has been with Crossref since 2002, and has contributed significantly and unwaveringly to keeping our system up and running in the data centre. Tim will be leaving Crossref at the end of the year; we’re grateful to Tim for all his years of dedication, and we will greatly miss his impressive Hawaiian shirt game on our all-staff calls.

After 25 years, it was also time to get serious about modernising our core content system, because even though it serves our community well, an older system with legacy code is a constant risk and frustration. We’ve therefore embarked on a multi-year modernisation project where we are replacing our old code piece by piece. We no longer want to have one big content system (a monolith), but are planning to identify different pieces of functionality and rebuild these as separate services (a modular, flexible, and robust approach). This year, we already managed to reconstruct some smaller pieces (for example, the ‘pusher’ mentioned above), and next year we will tackle larger projects, such as Metadata Matching and Authentication.

We continue to prioritise open, timely communication for planned or unplanned service interruptions and encourage everyone to monitor our status page at status.crossref.org. We’ll further hone our incident response processes in 2026, including openly posting incident reviews, and we’ll also centre system maintenance and documentation clarity in everything we do.

RCFS Projects

The Resourcing Crossref for Future Sustainability projects (RCFS) and the work of our Membership & Fees Committee resulted in deciding not to change some things (such as the basis for annual membership fees), but to change three things about our fees, as reported in July:

A new lower membership fee tier of 200 USD for members with annual revenues/expenses of under 1000 USD - so far, this includes around 3000 members. See below for more info.
A removal of volume discounts to reduce complexity in our billing code; they were little used, and those who did use these were fine with the loss of the discount.
A removal of the rule that only publishers of a title could register peer review reports (including comments and annotations) at the lower 0.25 USD fee for the first review; this lower fee is now available to any member to register any reviews of any other members’ works.

A new late-breaking addition to these fee decisions is the reduction of fees for members registering grants. As of January 1st 2026, there will be no fee for back-year (BY) grant registration, to encourage the faster adoption of older grants, which are more likely to have research outputs to be matched. This will be a two-year pilot to trial how a reduced fee incentivises adoption and boosts metadata connections, and could be extended to other record types as we monitor its success and sustainability. In addition, the 2 USD fee per current-year (CY) grant record is being reduced to 1 USD in line with the next-nearest fee, this is a permanent change for the foreseeable future.

Membership growth, efficiencies, and accessibility

In March, the board voted to update membership terms and bylaws to clarify processes for suspending and revoking membership, and to be more explicit about member practices that preserve the integrity of the scholarly record. A short-term Member Practices Working Group will be meeting in the first half of 2026 to draft these.

Crossref now serves 23,600 members across 164 countries, with continued growth particularly in Asia and Latin America. We’ve continued our ongoing member onboarding activities to support new members joining the community. We see around 230 new members join each month, and have welcomed 2,700 this year so far. We recently reported on how the shape of membership has evolved over our 25 years of operation.

From January 2026, we’re introducing a new lower membership fee tier of 200 USD for organisations with annual revenue or expenses of 1,000 USD or less, making membership more accessible to low-resourced organisations. Already, over 3000 members have been eligible to move into or join under that fee, and the idea is to monitor how this affects Crossref’s financial sustainability and potentially adjust the 200 USD annual fee down again in future years.

From 1 January 2026, the GEM program, which offers fee-free membership and content registration for all members from certain countries, will expand to include 18 additional countries, further reducing financial barriers to participation in the scholarly record, so we expect several hundred further members to join the existing 600 organisations in this category. More information about the GEM program expansion here.

As our membership base continues to grow, the Membership and Finance teams are constantly exploring ways to make shared processes more efficient. A key component in this work has been the efforts to automate several tasks within both teams to help us manage the additional work caused by our growth and allow our teams to focus more on providing the best quality service we can.

Our membership team continues to support our members, sponsors, service providers, metadata users and the wider community by email and through our community forum. The membership team includes staff members who focus on member support, and staff members who focus on technical support. During 2025 so far, we’ve received 36.8k member enquiries through our support system, a 17% increase from last year. This includes 22.6k inquiries related to general membership and 13k technical support enquiries. We’ve received 3.8k membership applications, and welcomed 2.7k new members.

Growth by the numbers

Crossref continues its steady revenue growth in 2025 due to the expansion of our membership base. With the addition of new members and the general growth of Crossref, comes an increase in the transaction-based tasks our Finance team handles.

So far in 2025 we have issued 14,833 invoices, which is a 9% increase since last year. We’ve seen an 11% increase in the number of payments received and applied, and a 12% increase in the amount of credit and debit memos applied over the same time last year. We have also seen a 42% increase in the number of billing-related tickets, totalling 20,723. A large segment of these tickets are related to fee updates associated with the new $200 membership tier.

Not all transactional work in Finance has increased as steadily, with increased revenue of 8% we have also seen a 14% increase in operating expenses. Through the strategic consolidation of vendors and use of financial tools, we have only seen a 1% increase in Accounts Payable invoices processed.

Organisational sustainability

Finance-wise, we’re doing well. We’re projecting to finish this year with revenue of 14,200,000 USD and expect revenue next year of 14,500,000 USD. We’re budgeting 2% growth in overall revenue, accounting for some of the changes to fees that will reduce our earnings on membership dues, but anticipating continued growth of content registration revenue.

Revenue and expenses trends

About 67% of our expenses come from personnel costs, and the other 33% include non-personnel costs like AWS, travel, legal fees, etc. As we continue to build out the team, we have ten new positions planned for the next year (recruitment for many of these is already underway or done). With additional staff roles and AWS expenses, we’re expecting expense growth of 16%. We post our financial statements and Form 990 filings on the financials page on our website.

Revenue per member size (by tier)

As the chart above shows, we still see ’the long tail’ of smaller members in the lowest fee category (275 USD) contributing more revenue than those in the largest category (50,000 USD) at 5.8 million USD versus 5 million USD.

Another aspect of sustainability is our impact on the world around us. And this year we were able to publish a second report on Crossref’s carbon footprint, having monitored and controlled for several carbon-heavy activities, primarily staff travel. Our reported emissions went up 40% from 2023 to 2024, due to more travel given our growth in staff and members, better recording our emissions (for example, with hotel stays), and including travel that we support for our partners, ambassadors and board members. In terms of travel spending, we are still well below 2019 when we were smaller, demonstrating that we are following through on not going back to the pre-pandemic norm.

We were one of the first open infrastructure organisations to adopt the POSI Principles and now have a few years’ experience in trying to meet them. Together with other adopters, we proposed updates and additions to the principles, based on real-world practice, and gathered a lot of community comment, resulting in the group publishing POSI v2 in October. We conduct a self-assessment every other year and we’ll be involving all our staff in the next self-assessment, due later in 2026.

Open governance through board election and annual meeting

We continued our commitment to being member-led and community-driven. This year’s anniversary Annual Meeting in October brought together members to discuss strategy, metadata developments, and hear the results of their voting in our board election. It comprised two half-days of online conferencing and several in-person satellite meetings spread across five continents, gathering close to 500 members of our community. It was a platform to reflect together on the past quarter of the century of building community infrastructure and connections underpinning the progress of scholarship, and to share plans for the future.

Each member has one vote, and together they elected the following organisations to serve a three-year term alongside the rest of the board:

Tier 1 candidates (electing one seat):

Rebecca Wambua, Distance, Open and e-Learning Practitioners’ Association of Kenya

Tier 2 candidates (electing four seats):

Damian Bird, CABI
Rose L’Huillier, Elsevier*
Anjalie Nawaratne, Springer Nature*
Nick Lindsay, The MIT Press*

*returning board member

Congratulations to the remaining and incoming board members as we start their new term in January 2026. Have a look at all the outputs from our Annual Meeting.

Strategic theme 4: Foster a strong team—because reliable infrastructure needs committed people who contribute to and realise the vision, and thrive doing it

Team structure

We reorganised the team heading into 2025 because we had ambitious goals that required a more structured, collaborative approach. We reorganised the work around three strategic, mission-driven areas of focus described above. This was our first full year with the cross-functional program groups in place, and the activities reported here make it evident that our team members, both existing and new, are firing on all cylinders.

New staff and new roles

We welcomed eight new team members in 2025. In February, we welcomed our new Director of Programs & Services, Helena Cousijn, and a new member of the Technical Support team, Arley Soto. In March, we welcomed our new Community Manager for funders, Rocío Gaudioso Pedraza. In April, we launched our new Data Science team by welcoming Jason Portenoy and Alex Bédard-Vallée. In November, we welcomed our new DevOps Engineer, Thelma Laryea, and our new Program Technical Lead for the OSO program, Bharath Govindarajan. In December, we welcomed another member of the Technical Support team, Natali Giorgobiani.

We also had team members step up into new roles. Dominika Tkaczyk completed the new leadership team by taking on the Director of Technology role, Paul Davis has started his new role as Product Manager, and Michelle Cancel has taken on the Head of Human Resources role. And there’s more to come! As next year begins, two team members will step into Program Technical Lead roles: Carlos del Ojo Elias for the CRN program and Patrick Vale for the CCT program. Together with the Program Technical Lead for the OSO program and the Head of Infrastructure Services, these roles will complete the new structure of the technology team. This structure is more closely aligned with how our work is organised and will enable stronger coordination both within and across cross-functional programs.

Supporting a thriving global culture

As our team grows in different aspects within our new org structure to meet the needs of the community, we remain committed to supporting a thriving culture through training, conducting regular temperature checks, and organising our annual staff retreat. This year, we continued our work on psychological safety and introduced workshops on giving and receiving feedback and on consensus building. We were able to put some of this training into practice at our in-person all-staff event in Split, Croatia, where we all came together to build our roadmap.

We are ending the year with 51 staff in 14 countries and look forward to diversifying and evolving even further as a team in 2026—we’re currently hiring in UX, Communications, and Membership—and keep an eye on our jobs page for forthcoming opportunities in Software, DevOps, Metadata, and Operations!

Thank you to our community of members, partners, board, ambassadors, sponsors, metadata users, service providers, integrators—and of course our team—for making 2025 such a productive year. Together, we’re building a richer, more connected research ecosystem for the benefit of society. We can’t wait to continue the work together in 2026.

Wellcome and Europe PMC: supporting Open Research through open metadata

Rocío Gaudioso Pedraza — Mon, 15 Dec 2025 00:00:00 +0000

In my latest conversations with research funders, I talked with Hannah Hope, Open Research Lead at Wellcome, and Melissa Harrison, Team Leader of Literature Services at Europe PMC. Wellcome and Europe PMC are working together to realise the potential of funding metadata and the Crossref Grant Linking System for, among other things, programmatic grantee reporting. In this blog, we explore how this partnership works and how the Crossref Grant Linking System is supporting Wellcome in realising their Open Science vision.

What motivated you to join Crossref?

Hannah: The motivation for Crossref Grant IDs is to be able to disaggregate research outputs between funders. Funders’ grant identifiers come in a range of formats, funders might change them over time, and there are also similarities between funders’ names, which is a challenge. Permanent identifiers, in this case, Crossref Grant IDs, are an opportunity to avoid some of the confusion if we were able to implement them throughout the research ecosystem.

This is also being discussed in different contexts, for example, within the Barcelona Declaration working groups, funders and other stakeholders are exploring the diverse motivations that exist to implement changes into our workflows, as well as the challenges that funding metadata and persistent grant IDs can help solve.

The way Wellcome implemented the Grant Linking System is a bit unique, given that it partnered with Europe PMC for the technical implementation and metadata registration with Crossref. Can you tell us more about how it works?

Hannah: The collaboration between Wellcome and Europe PMC in the implementation of Crossref’s Grant Linking System started because they already had the grants landing page feature ready and available to us.

There was an initial hope that other funders of Europe PMC, which also have these grant landing pages, could then leverage that same system to make Crossref grant IDs more broadly available to the research community, but I am not sure if that has materialised yet.

Melissa: Currently we are supporting Wellcome’s implementation of Crossref grant IDs, but the infrastructure remains available to other Europe PMC funders should they decide to take advantage of it. We already have funding metadata for Europe PMC funders because it is a requirement for grantees to select their grant identifier when submitting their accepted manuscripts for indexing and archiving. As we already have that metadata, naturally we can pull it together and send it to Crossref, along with the link to the Europe PMC grant landing pages!

An additional benefit of partnering with Europe PMC is the comprehensive metadata we deliver to Crossref with the grant IDs. For example, we have invested in supplementing affiliation data with ROR iDs and we deliver to Crossref all the data we have that matches their schema for grant data.

How is Wellcome leveraging the funding metadata and Crossref grants IDs that are being shared and registered with Crossref?

Hannah: We are discussing internally how we can better socialise the Crossref grant DOIs among the grantees, either via our grant management system or through Europe PMC. One place where the Crossref grant DOIs are being used and shared is through our publishing platform, Wellcome Open Research. The Crossref grant DOI is included in the publication metadata, ensuring that the research output is linked to the funding via the open metadata registered.

However, as we use Europe PMC as our repository for funded written research outputs, these outputs are aggregated alongside the grant records which includes the Crossref grant DOI, facilitated by Europe PMC APIs. So we have the means to link the two things together.

Melissa: There are some UX and technical blockers to fully integrate Crossref grant IDs within the Europe PMC grant system currently that are detrimental to the utility of these IDs, for example, you can’t search for a specific grant in Europe PMC grant finder using a Crossref grant ID. We are partnering with Crossref to solve these challenges and offer users more functionality in this space next year.

Hannah: Beyond eLife and Wellcome Open Research, I am not sure which publishers use Crossref grants DOIs in their workflows.

Rocio: That’s an interesting question, as we aren’t seeing a massive flow of Crossref grant IDs in the works metadata records just yet. We are exploring with publishers and their service providers how to make this business-as-usual, and in the meantime, we are running a series of matching projects to ensure that, when possible, we make those connections ourselves to enrich the metadata with funding information. We already insert reciprocal relationships where one record asserts a link with another (in this case, where either a grant Finances a work or a work isFinancedBy a grant record, Crossref adds in the reverse). Improving and enriching these relationships directly in the metadata makes sure that metadata provided by funders can make their way to the research outputs that originate from the grant.

Wellcome is streamlining the way of asking grantees to report on their publications, facilitated by Europe PMC. Can you tell us a bit more about how this will work and what role metadata will play?

Hannah: We will stop asking researchers to report their publications directly to us as part of progress and end-of-grant reporting. We believe there is sufficient open metadata with high-quality tagging in the ecosystem for us to collect written research outputs programmatically from this public data. Under our new system, we will be directing researchers to look at their grant record within Europe PMC and make sure that their written research outputs are properly linked there; otherwise, we won’t see them. We are trying to leverage open data, existing infrastructure, and a route that enables us to improve the completeness of open metadata.

There aren’t many mechanisms that enable our researchers to add assertions to funding and research output records retrospectively, and Europe PMC offers us that opportunity, and that is really critical for us. Rather than collecting information in our own system, we can contribute to enhancing the global corpus of knowledge and the quality of open metadata more broadly. Since correcting metadata at source isn’t easy, Europe PMC presents us with an opportunity to contribute to that system.

Melissa: We are thinking broadly about this problem; many institutions curate their research information in spreadsheets or closed CRIS systems and struggle to make it publicly available. We are thinking about how Europe PMC can be leveraged to be a public home for that data. EMBL-EBI hosts Europe PMC and utilises it as the institutional repository, so we have started a pilot project to add ROR IDs for affiliations to EMBL-authored publications within Europe PMC. This is manually curated, high-quality metadata that would otherwise be lost from the public ecosystem.

If you look into the future, what would your hopes be for the GLS and greater transparency in funding metadata in general? What do you think that we could achieve collectively as a community?

Melissa: It would be amazing (!) if everybody, from funders to publishers, to institutions and authors, would coalesce around the Crossref Grant Linking System, and add to metadata exchange workflows – you would potentially have a very clean and clear picture of where the money is going, what the outputs are, and how they relate.

Currently, even with the Open Funder Registry, there is ambiguity around funder names - for example, different geographical national funders sharing the same exact name as their counterpart in another country - so even with the best will in the world, funder institutions could be misidentified in systems and assigned the wrong identifier. The Crossref Grant Linking System facilitates complete disambiguation because grants are associated with the issuing funder’s correct identifier, ensuring traceability of outputs and funding and enabling more precise, cleaner metadata.

Hannah: I think that is a bit of the Holy Grail and in reality, its a bit messy, there isn’t just one system! We need to be able to move past the chicken and egg discussion, where we talk about the use of different identifiers, with sometimes competing priorities. For me, the real challenge for the metadata community is how do we enrich metadata, correct errors, and develop greater interoperability between PID systems. So that multiple parties can contribute towards the creation of a greater whole record, rather than relying on a single owner of the record to provide all the information. If we could all, funders included, connect information from individual partners to create a unified record at the end of it, we could have better records and probably save time by distributing the workload.

What would you say to colleagues in other funders about investing in open metadata?

We all need information from other partners in the ecosystem and investing in our own internal system will not give us the same return as collectively investing in opening up that information wherever possible.

——-

We are very grateful to Hannah Hope and Melissa Harrison for their perspectives on open funding metadata and the role of the community in ensuring a complete and comprehensive Research Nexus.

New tool to report on completeness of open research information globally

Kornelia Korzec — Tue, 21 Oct 2025 00:00:00 +0000

Wednesday 22nd October 2025—Crossref, the open scholarly infrastructure nonprofit, today releases an enhanced dashboard showing metadata coverage and individual organisations’ contributions to documenting the process and outputs of scientific research in the open. The tool helps research-performing, funding, and publishing organisations identify gaps in open research information, and provides supporting evidence for movements like the Barcelona Declaration for Open Research Information, which encourages more substantial commitment to stewarding and enriching the scholarly record through open metadata.

Crossref’s Participation Reports now offer expanded features and provide full coverage of all members and all resource types registered with Crossref DOIs (Digital Object Identifiers)—over 175 million records representing a significant share of global research production from organisations in 164 countries. Each of Crossref’s 23,000 members has a dashboard to visualise their metadata contributions, display coverage of key information for scholarly works, and get actionable feedback via a gap report that specifies records that need enrichment, all helping to make more transparent the work that goes into creating and curating the scholarly record.

For any Crossref member—whether journal publisher, research funder, university, or museum—coverage of up to 11 key elements is public and visible to everyone, including: references, abstracts, ORCID iDs, affiliation strings, ROR IDs, Open Funder Registry IDs, funding award numbers, text-mining URLs, licence URLs, Similarity Check URLs (for text-based plagiarism checking) and the presence of a Crossmark policy, indicating the organisation’s commitment to declare corrections and retractions. These metadata elements provide greater context and visibility for research objects such as journal articles and preprints, grants and awards, books and book chapters, standards, datasets, conference papers and various ‘other’ content such as scholarly blogs, images, and even physical museum artefacts.

Mochammad Tanzil Multazam, Library Director of Universitas Muhammadiyah Sidoarjo, and Secretary of the Supervisory Board of Relawan Jurnals, says, “As a sponsoring organisation for several thousand small publishers across Indonesia, we support Crossref members to register complete metadata for their works. Despite time and resource constraints, this new actionable open report on key metadata elements will help drive improvements in the information they share for their publications. This has wide-reaching implications for the visibility of that research and trust among the community, and therefore has the potential to support Indonesian scholarship in the global context.”

Lena Stoll, Program Lead at Crossref, explains, “We are happy to have extended participation reports to cover more diverse record types, including grants, datasets, dissertations, and more, and to make it easier for our members to act on their ongoing improvements to enrich their records and build towards the vision of an open and more complete Research Nexus.”

Ludo Waltman, Scientific Director and Professor of Quantitative Science Studies at the Centre for Science and Technology Studies (CWTS) at Leiden University, comments, “As a representative of the researcher and metascience communities, this data is of great importance for us to analyse the trends and effects of global research activity. Crossref is one of the main driving forces in open infrastructure, and its commitment to supporting metadata completeness through this open reporting dashboard is a significant step for the open research information movement.”

Access Crossref Participation Reports and search for any Crossref member organisation.

Participation report for a typical Crossref member, Universidad La Salle Arequipa in Peru

About Crossref

Crossref runs an open infrastructure to link research objects, entities, and actions, creating a lasting and reusable scholarly record that underpins open science. Together with their 23,000 members in 164 countries, Crossref drives metadata exchange and supports nearly 2 billion monthly API queries, facilitating global research communication, for the benefit of society.

Enhancing repository integration with Crossref

Johanssen Obanda — Mon, 13 Oct 2025 00:00:00 +0000

Repositories are home to a wide range of scholarly content; they often archive theses, dissertations, preprints, datasets, and other valuable outputs. These records are an important part of the research ecosystem and should be connected to the broader scholarly record. But to truly serve their purpose, repository records need to be connected to each other, to the broader research ecosystem, and to the people behind the research. Metadata is what makes that possible. Enhancing metadata is a way to tell a fuller, more accurate story of research. It helps surface relationships between works, people, funders, and institutions, and allows us as a community to build and use a more connected, more useful network of knowledge - what Crossref calls the ‘Research Nexus’.

The challenge many repositories face is that metadata can be incomplete, inconsistent, or disconnected. Think of references without DOIs, authors without ORCID iDs, or research outputs that aren’t linked to funding. To address this, Crossref provides a range of services that repositories can use to improve the quality and interoperability of their metadata. Our REST API, which is openly and publicly accessible, allows repositories to retrieve structured metadata, such as DOIs, references, abstracts, contributors, ORCID iDs, and funder information, that can be used to enrich and update their local records. For repository members, with the Cited-by service and reference linking, repositories can also show how works are being cited and interconnect related content. The Grant Linking System (GLS) enables the clear indication of which research outputs are linked to specific grants, and funding bodies themselves are connected using Open Funder Registry and ROR, adding another layer of context. With Crossmark, repositories can flag updates, corrections, or retractions to ensure transparency and trust in the scholarly content they host.

Enriching repository metadata using Crossref is a practical and empowering step toward making your records more discoverable, complete, and connected. The process is simple, and you don’t need to be a developer to get started. Repositories can query the Crossref REST API using a DOI or basic metadata like a title or author name, and receive structured, reliable information. This can include full author lists, ORCID iDs, reference lists, funding data, and licensing terms. You can then match and merge this data into your repository records. Adding Crossref DOIs to your metadata enables persistent linking, helping users trace research outputs back to their stewards. It also helps create rich relationships between articles, datasets, software, grants, and other research objects. All of this supports the FAIR principles and contributes to a more connected and reusable scholarly record. And because Crossref’s infrastructure is open, any repository can access and use this metadata to improve the quality, visibility, and long-term value of their collections.

Steps to enrich repository metadata with Crossref:

Query the REST API using DOIs or basic metadata (visit our API learning hub to learn how to use the Crossref API)
Retrieve structured metadata like authors, ORCID iDs, funders, affiliations, ROR IDs, licenses, grants, and references
Map and merge with your local records
Display persistent links to all kinds of research objects using Crossref DOIs
Support FAIR by including open, structured, and complete metadata

Across the repository community, several institutions are already integrating Crossref metadata in meaningful ways to enrich their records and improve discoverability. DSpace users can enrich their deposits by using the platform’s “Live Import” feature, which allows them to pull in Crossref metadata, such as titles, authors, and DOIs, directly into items during the submission process. A deeper integration between DSpace and Crossref is currently in development. HAL in France uses the Crossref API to complete and standardise references, making its content more consistent and connected (hal.archives-ouvertes.fr). SciELO, a key open access platform in Latin America, leverages Crossref DOI links and citation metadata to strengthen the visibility of its journals (scielo.org). In Canada, the University of Saskatchewan’s eCommons repository queries the Crossref API to enhance metadata accuracy and link records to the broader scholarly graph (ecommons.usask.ca). The Apollo repository at the University of Cambridge uses Crossref to connect theses and articles to their published versions, creating a clearer picture of research outcomes (repository.cam.ac.uk). Zenodo, hosted by CERN, draws on Crossref metadata to link deposited datasets and software with related publications, supporting transparency and reuse (zenodo.org).

These examples show how even modest integrations with Crossref can lead to substantial gains in metadata quality, interoperability, and global discoverability. Altogether, these activities and organisations are enhancing the Research Nexus, enriching a scholarly graph for the benefit of all.

Want to learn more? You can explore the presentation slides (PDF) from Open Repositories 2025, which cover the Crossref API and its capabilities, how repositories can use it to query and enrich metadata, the benefits for repository managers, researchers, and funders, as well as recent updates to our metadata schema.

Sprinting to Progress: Behind the scenes of our first metadata sprint

Luis Montilla — Mon, 23 Jun 2025 00:00:00 +0000

If you take a peek at our blog, you’ll notice that metadata and community are the most frequently used categories. This is not a coincidence – community is central to everything we do at Crossref. Our first-ever Metadata Sprint was a natural step in strengthening both. Cue fanfare!. And what better way of celebrating 25 years of Crossref?

We designed the Crossref Metadata Sprint as a relatively short event where people can form teams and tackle short problems. What kind of problems? While we expected many to involve coding, teams also explored documenting, translating, researching—anything that taps into our open, member-curated metadata. Our motivation behind this format was to create a space for networking, collaboration, and feedback, centered on co-creation using the scholarly metadata from our REST API, the Public Data File, and other sources.

What have we learned in planning

The journey towards the event was filled with valuable lessons and learnings from our community. Our initial call received submissions from 71 people, which was exciting but presented the first challenge: we felt our event would work better with a relatively smaller group. An additional challenge we faced was the enthusiasm from people from different regions of the world who were eager to join, but needed support to attend in person. It reminded us how global our community is, and how important it is to think about different ways of making participation possible, especially in future events.

We also wanted to make sure that participation wasn’t limited by technical background. The selection process included a preliminary review by several members of our team to bring in a mix of perspectives and reduce bias. The event welcomed participants from all kinds of expertise levels, including colleagues who had never worked with APIs before. We sought to provide common ground for all with several group calls, where we presented introductions to our tools and used the opportunity to collect requests about tools, specific data, and questions from the participants that could enhance their preparation during the sprint.

At the Crossref Metadata Sprint

I’ve recently stumbled upon the following quote from a recognized data scientist:

Numbers have an important story to tell. They rely on you to give them a clear and convincing voice. (Stephen Few) ¹

It made me think that we can replace numbers for metadata and the idea still holds. Surrounded by the paleontological collections of the National Museum of Natural History, on 8th of April in Madrid, 21 participants and 5 Crossref staff came together to work on twelve different projects. These ranged from improvements to our Public Data file formats and exploring metadata completeness, to tackling multilingual metadata challenges, understanding citation impact for retracted works, and connecting Retraction Watch metadata with other knowledge graphs metadata.

The different teams that participated in the first Crossref Metadata Sprint.
The initial hours were the most energetic (but not chaotic!) as most of the participants had the chance to interact in person for the first time, ideas were exchanged, and pre-formed groups became more stable (however, one of the advantages of the format is that teams don't have to be rigid). Twelve coffee- and tea-powered projects started taking shape, a few of which are part of larger ideas under development. By the end of the second day, we saw:

Author changes between preprints and published articles.
Coverage of funding information by publisher.
Enriching citations with Crossref metadata.
Funding metadata completeness.
Improvement to the Public Data File.
Interoperability between Crossref DOIs and hash-based identifiers.
University of Tetova’s metadata coverage.
Retraction Watch data mash-up.
Perspective about AI-driven multilingual metadata.
Public Data File in Google Big Query.
Visibility of retractions across citations.
Visualising Crossref geographic member data.

Our team worked as part of some of these projects, providing valuable insights and feedback to the participants. We ended the first session with a group dinner and re-energised for the second day, which started with everybody fully immersed in their tasks. As we approached the conclusion, the groups started preparing some quick slides for a short presentation (that you can find here).

Our team and the participants left excited and looking forward to the next opportunity to collaborate. We certainly see the potential of recreating these spaces, and we’ll work on future editions in a different location. All of the project summaries and notes will remain stored in our metadata sprint Gitlab repo. Would you like to know more about any of these ideas? Let us know in the comments.

The first Crossref Metadata Sprint in a nutshell

Participants

None of this would’ve been possible without our enthusiastic participants. Huge thanks to everyone! Here is the full list of those who attended our inaugural Sprint:

Name
Blessing Abumere
Ana Bermejo
Robert Bianchi
Adam Buttrick
María de la Paz
Nicoleta Roxana Dinu
Jack Ekinsmyth
Castedo Ellerman
Álvaro Hontanar
Bianca Kramer
Anne L’Hôte
Cyril Labbe
Alexandra Malaga
Agon Memeti
Kaitlin Newson
Yağmur Öztürk
Dietrich Rordorf
Mohamed Selim
Sajad Sepehri
Ramazan Turgut
Iñaki Úcar

https://www.forbes.com/sites/brentdykes/2016/03/31/data-storytelling-the-essential-data-science-skill-everyone-needs/ ↩︎

Looking ahead: the Research Nexus and the state of metadata in 2050

Ed Pentz — Tue, 04 Mar 2025 00:00:00 +0000

Looking Ahead: The Research Nexus and the State of Metadata in 2050

Published in Science Editor (Vol. 48, No. 1, March 2025), this feature article uses 25 years of scholarly metadata history as a lens to look forward. The authors trace the journey from Crossref’s first DOIs in 2000 — when records carried little more than journal title, author, volume, issue, and page number — to today’s Research Nexus of 30+ output types and rich relationships between works, people, organisations, and funders. They then sketch what scholarly metadata might look like in 2050.

Strategists

Understand where scholarly metadata is heading. A long-view perspective on how the Research Nexus reshapes what gets counted, assessed, and valued in scholarship over the next 25 years.

Decision-makers

Plan for richer assessment beyond publication counts. The authors anticipate research evaluation shifting toward practices, broader impact, and the wider set of activities around research — not just papers.

Practitioners

Prepare for new output types and tighter relationships. Computational notebooks, AI-assisted research, and immersive scholarly outputs will need persistent identifiers and structured relationships if they are to participate in the record.

What this article covers

Looking back to look forward — how Crossref metadata grew from a handful of bibliographic fields in 2000 to over 30 research output types and rich relationship data today
The Research Nexus concept — the “complex, evolving network of objects, along with descriptions of how they relate”, and why this framing now underpins scholarly infrastructure
PIDs as the backbone — how DOIs, ORCID iDs, ROR IDs, and other persistent identifiers make the Research Nexus tractable at scale
Research Nexus in 2050 — emerging output types (computational notebooks, virtual and augmented reality experiences, AI-assisted research) and what they require from metadata infrastructure
Beyond publication counts — a future where assessment recognises research practices, broader impact, and activities alongside research, not just published papers
Challenges ahead — cultural resistance to open-data defaults, the financial sustainability of open infrastructure, and the importance of globally inclusive systems that resist regionalisation and fragmentation

Read the article

Read on Science Editor

Metadata beyond discoverability

Ginny Hendricks — Tue, 03 Dec 2024 00:00:00 +0000

Metadata is one of the most important tools needed to communicate with each other about science and scholarship. It tells the story of research that travels throughout systems and subjects and even to future generations. We have metadata for organising and describing content, metadata for provenance and ownership information, and metadata is increasingly used as signals of trust.

Following our panel discussion on the same subject at the ALPSP University Press Redux conference in May 2024, in this post we explore the idea that metadata, once considered important mostly for discoverability, is now a vital element used for evidence and the integrity of the scholarly record. We share our experiences and views on the metadata significance and workflows from the perspective of academic and university presses – thus we primarily concentrate on the context of books and journal articles.

The communication of knowledge is facilitated by tiny elements of metadata flitting around between thousands of systems telling minuscule parts of the story about a research work. And it isn’t just titles and authors and abstracts – what we think of as metadata has really evolved as more nuance is needed in the assessment and absorption of information. Who paid for this research and how much, how exactly did everyone contribute, what data was produced and is it available for me to reuse it, as well as, increasingly, things like post-publication comments, assertions from “readers like me”, who has reproduced this research or refuted these conclusions.

Different types of published works are described by different types of metadata – journal articles, book chapters, preprints, dissertations. And those metadata elements can be of varying importance for different users. In this article, we will talk about metadata from the perspectives of four personas highlighted by the Metadata 20/20:

Metadata Creators, who provide descriptive information (metadata) about research and scholarly outputs.
Metadata Curators, who classify, normalise and standardise this descriptive information to increase its value as a resource.
Metadata Custodians, who store and maintain this descriptive information and make it available for consumers.
Metadata Consumers, who knowingly or unknowingly use the descriptive information to find, discover, connect, and cite research and scholarly outputs.

Our approach delineates the metadata lifecycle, from authorship, through production, discovery and through continuous curation. Though some of the metadata is generated outside of that linear process, and much happens before the authorship step, we see it as a clear and useful breakdown of how metadata contributes to a new piece of content.

Authorship

The first stage in the metadata lifecycle, authorship, is just the beginning of a dynamic process with many collaborators. A formative piece of the puzzle, authorship involves the authors or contributors, the editorial team and/or the marketing team and this is when the shape of the project and its metadata takes form. During this stage, the book or journal’s metadata exists only between the originators and the publisher, allowing the most opportunity for creativity and enhancement. Once the metadata reaches the next checkpoint along the lifecycle and is sent out externally, it’s more difficult and riskier to make major changes to the key metadata elements. In scholarly monograph publishing especially, we have the advantage of longer production lead times during which to amend and manipulate metadata during this stage.

At this stage, authors may have ideas of titles, subtitles and descriptions and it is up to the editors and other team members at the publisher to think strategically about how this can be optimised. The marketing and sales teams may be thinking about how the abstracts, keywords, and classifications can be best optimised for the web, leading to increased sales. Discoverability and interoperability of metadata for a book or journal, especially the use of persistent identifiers, is beneficial both for the author – in that their book is easily discovered, used, and cited – and for the publisher – increased visibility, sales, and usage.

Current challenges at the authorship stage include changing goalposts for metadata standards and accessibility requirements, which also have knock-on effects in subsequent stages in the metadata lifecycle. One of the key challenges with these is that they require buy-in from multiple players to keep up with and amend, and publishers must think closely about how these changes may affect metadata workflows for books at different stages of publication.

Production

As a book or journal article comes into production, it’s time to update and release the metadata to retailers, libraries, data aggregators and distributors. The metadata should be updated and checked to make sure that it’s still a good reflection of the product or the content that it describes and complete enough to release, including a final cover image in the case of books. This is still very much a collaborative effort with multiple roles involved. Technical details, such as spine width, page extents, and weight, are added, capturing the final specification. The editorial team may update metadata entered into systems earlier in the process. For example reviewing the prices, updating subject classification codes or amending the chapter order. If any of the content is to be published open access, appropriate licensing and access metadata need to be included, so that users of the content are clear about what they can (and can’t!) do with it. Metadata that’s not yet captured upstream can be added or enhanced. For example, vendors already involved in the production process can verify that persistent identifiers (PIDs) are present and correct in funding metadata.

More and more metadata elements are being requested by supply chain partners. For example, new requirements being introduced to provide commodity codes, spine width, carton quantities, gratis copy value and country of manufacture. There may be differences in metadata depending on the methods of production. For example, country of manufacture will be supplied differently when using traditional print methods where the whole print run is carried out at a location, or where a title is manufactured print on demand and the location of printing is determined by the delivery address.

In an XML-first workflow, metadata can be captured with the content files to aid with discovery. This usually requires multiple systems, both internal and external. These systems need to be able to work together to ensure that only up-to-date metadata is used. Metadata will change throughout the production process, whether it’s the publication of an accepted manuscript through to the final version of record, or pre-order information to the published version, so updates need to feed out regularly.

The right metadata needs to go to the right recipient. Some is not useful or cannot be processed by certain recipients. For example, a printer, retailer, librarian or data aggregator each have their own needs and use cases and may receive and process metadata in different formats or require different fields.

Discovery

Discovery is the series of actions taken by an end user to retrieve and access relevant content they do not know about. Discovery can happen everywhere: Google (a search engine), a library catalog, a publisher platform, etc. However, Discovery is associated with using Discovery systems in the academic sector.

The technological landscape of libraries has developed in the last 15 years. Discovery systems are tools libraries subscribe to in order to allow their end users to have one search experience within their library holdings. It is paramount for librarians that library collections are used; hence, it is very important for them that the discovery system of their choice contains all the relevant metadata. Libraries expect their discovery service to include their content coverage as comprehensively as possible. Content items not represented or misrepresented in a discovery system create challenges to libraries in how they might otherwise ensure that these materials are discovered and accessed.

Libraries’ adoption and usage of discovery systems are surrounded by the belief that the great benefits of this technology are the one search box and the configuration flexibility, which are the most important benefits. Libraries invest a significant amount of money in discovery services. The increase in usage is the success indicator of this adoption and a positive return on investment.

The backbone of discovery systems is formed by three crucial elements: a user interface, a metadata index, and a link resolver or Knowledge Base. These elements, along with a back-end control panel for librarian configuration, are the key components that enable the discovery process.

The discovery index, a database storing descriptive data from various content providers, data sets, and content types, is a testament to the collaborative efforts of content providers and discovery systems vendors. Their work under the Discovery Metadata Sharing partnership agreements, which establish the format, scope, frequency, and support of the collaboration, is instrumental in meeting librarians’ expectations.

Format

The discovery metadata integration processes have settled down for most cases in these two metadata delivery workflows.

Metadata for the index of discovery: Discovery systems have traditionally made efforts to work with various metadata formats like MARC, proprietary templates, etc., but the preferred format is XML. This metadata could include all the bibliographic information data, including index terms and full text at the article and chapter level.

Metadata for link resolvers and Knowledge bases: Knowledge bases are tools that contain information about what is included in a product, packages, and/or databases. KBART is the preferred format in this area. It includes a set of basic bibliographic descriptions at the publication level and linking information for direct and OpenURL syntaxes.

Frequency

The delivery channels vary, and the frequency could vary daily to yearly, depending on the publication schedule.

Scope

Library collections include various content types, including archival materials, open access, and multimedia alongside the more traditional books and periodicals. Different content types will require different metadata elements to make a comprehensive discovery-friendly description, and the metadata elements will impact the formats in use.

Discovery services will receive this data and prioritise uploading. They will select and manipulate the required metadata elements according to their system requirements. These metadata tweaks and selections are not always communicated to the content providers and/or libraries. Ultimately, librarians decide which metadata will be visible on their discovery tool and the linking methods of their choice.

As described, Discovery is a complex area where the activities of its main stakeholders are interconnected. The success of the end users’ discovery journey from search to access depends on the successful integration, implementation, and maintenance of the discovery systems. This necessitates a combined effort from the three discovery stakeholders: content providers, discovery system providers, and libraries. Their collaborative work is not just crucial, but integral to supporting discovery and fulfilment in the most efficient manner possible. Your active involvement in this process is what makes it successful.

How do we ensure discoverability?

Electronic resources do not exist in isolation but are assessed and used depending on their level of integration in the discovery landscape where libraries and patrons are active. From a content provider’s perspective, discoverability is about the number and efficiency of entry points to our products created in third-party discovery products.

The level of discovery integration has a direct impact on sales and upsell opportunities. Products that are not discoverable are difficult to work with, and the opposite is true for products that are considered discoverable. Your role in ensuring discoverability directly influences the user experience and sales, making your work crucial and impactful. The term ‘Discoverability’ is critical in discovery library systems. It refers to the extent to which eResources are searchable in a discovery system, and it directly influences the ease with which users can find the information they need, thereby enhancing their overall experience. In practical terms, the degree of discoverability will be impacted by the quality of the metadata supplied, the transformations the metadata suffers in the integration process to discovery systems, and the configuration’s maintenance.

The general principles of metadata quality also apply in this area: accuracy, completeness, and timely delivery. Your attention to these principles is crucial to contributing to the effectiveness of the discovery process. Metadata enrichment practices like identifiers and standards are also applicable. Your meticulous attention to detail in maintaining metadata quality ensures the effectiveness of the discovery process.

Discovery as a mindset in the publishing process will increase discoverability, as it will be influenced by product designs (whether the content is linkable) and which metadata outputs are possible. For example, author-generated index terms will be more effective for meeting research search terms, and detailed article titles will probably be more discoverable than general titles. Finally, all the integration, descriptive metadata, configurations, etc., leave much room for errors. The flow is complex; on occasion, the products and content are more complicated to describe than tools can handle, and there are millions of holdings per library to manage. Constant maintenance and troubleshooting are crucial elements to maintaining and increasing discoverability.

Metadata beyond publication

In the lead-up to publication, finalising rich complete metadata can seem like establishing a fixed set of information. Post-publication, however, the metadata workflow should be dynamic, able to evolve to keep pace with new demands and opportunities. Think of metadata as a journey rather than a one-time destination, and look at ways to futureproof your metadata by actively adapting to some of the following types of change.

Changing Publisher Goals and Product Needs

Metadata should align with changing priorities for a publisher. Developing new formats, shifts in commissioning focus or building new distribution partnerships may require metadata updates. For instance, re-releasing content in audiobook form or digitising a backlist title warrants a metadata review to ensure current and prospective readers find accurate, relevant information.

Changing Technology and Metadata Standards

Advances in technology, from artificial intelligence to emerging metadata standards, offer enhanced possibilities for capturing and updating metadata. AI, for example, can help enrich metadata with more precise subject tagging, while new metadata formats may offer greater compatibility across platforms and discovery services. Staying current with these tools can help publishers manage metadata more efficiently and enhance discoverability.

Changing Societal Values

As society evolves, so do expectations for inclusive and socially responsible metadata. Utilising new categorisation codes, such as those for the United Nations Sustainable Development Goals, can align metadata with emerging social goals. Similarly, publishers may need to revisit keywords and category codes to reflect language changes, balancing the integrity of historic records with the need for current, appropriate terminology.

Changing Industry Priorities

Commitments to accessibility and sustainability have prompted developments in metadata. Increasingly, publishers need to be able to use metadata to build a record of sustainable production methods, such as paper sources, printing methods or ink types. New metadata fields for accessibility specifications will also support more inclusive reader experiences going forward. Metadata will play an increasingly vital role in meeting industry standards for accessibility, EUDR and EAA compliance, and environmental transparency.

Changing Customer and Librarian Expectations

Finally, as the metadata expectations of customers grow and the nature of roles and responsibilities in library and collection management professions develops, teamwork and making good use of available resources are essential. Publishers don’t have to tackle this alone. Working with organisations such as Crossref or Book Industry Communication (BIC), signing up to newsletters and webinars, and forming an in-house discovery group are all great ideas for sharing ideas and best practice, and ensuring your metadata workflow is adaptable and responsive. Be part of the conversation now rather than struggling to keep up down the line!

What are some challenges and opportunities with metadata?

JM: Metadata that establishes permanence is a real opportunity in a digital landscape where content can move or be taken down, links can rot, website certificates can expire. Persistent identifiers like ORCiDs for people and DOIs for content are key examples of metadata that establish enduring routes to, and provenance of, published digital content.

KM: Metadata creation, maintenance and change has long been seen as a manual process. AI tools offer a real opportunity for metadata creation and review, especially for keywords and classification codes, at a scale and speed that has the potential to transform metadata workflows. Especially for backlist transformation, AI could offer real opportunities in this area. A challenge we face for monograph metadata more specifically is that much of the scholarly metadata infrastructure is built around the journal article, and it can be difficult to fit longer form content into these systems of discovery.

MT: Metadata is crucial. Good metadata (complete, accurate, and timely) is the base for smooth integrations and easy discovery interactions with eResources. Bad metadata (inaccurate, incomplete, late) will be the main reason for undiscovered content. At this point, the eResources industry is still based on different versions of the same metadata, which is the leading cause of problems. It is probably time to start considering a unique record approach. This unique record, which will be complete and accurate, could be used by different systems for different purposes. I know there are many details to define here, but if you think about it, it is not impossible and could solve the many known issues.

How do you ensure the quality and completeness of your metadata? Do you have ways of auditing it?

SP: Validation of data is really important, so choosing or building a system that’s set up to do this is an important foundation. It’s straightforward to check for completeness of fields and I run daily checks on our book metadata to make sure there’s nothing missing in the files feeding out. Quality can be more challenging to monitor. Feedback from data recipients is key, and accreditation schemes such as the BIC Metadata Excellence Award are a great way to benchmark progress. Good training and clear documentation help to make sure that everyone involved in creating and updating metadata understands exactly what they need to do and the standards they need to meet.

KM: Earlier this year we completed a year-long data cleansing project as part of our move to a new title management database. This gave us the time to address gaps in backlist metadata as well as to identify any inconsistencies across records for the same book, and enrich key metadata fields like classification codes, keywords and PIDs. For frontlist titles, each person owns a number of fields to ensure they are complete before a book’s metadata is distributed – some of these have validation tools which will prevent a book’s metadata from being sent out unless it is complete.

MT: Strict and consistent internal processes are essential to ensure quality and completeness. Following the different standards and industry recommendations helps to keep the quality at high standards. Random manual checks and system-based checks help to ensure everything is good. We carry out projects where we work with specific aspects of the metadata. This building-blocks approach ensures the different data layers are as good as possible. As with any project, metadata projects should have specific goals, outcomes, resources, and documentation.

How do you know if (and how much) metadata helps achieve your goals?

JM: Take any available opportunities to find out what people think of your metadata – via library conferences, institutional customer feedback, and by working with the library team at our home institution, we’ve had some really useful and interesting conversations about MUP’s metadata and where we can improve it to make it as relevant as possible for different stakeholder needs.

MT: Customers and Discovery partners will inform us if something is incorrect. Usage data is also a good indicator of how healthy our metadata is. Following industry standards is another good reference point for assessing the metadata. Finally, the metadata is only good when we know what we want to use it for. So, always considering what we are trying to achieve helps us understand how effective the metadata is.

KM: As the others have noted here, and we represent a range of different types and sizes of publishers, measuring the direct impact of metadata is an ongoing challenge. We think about the different end users who might encounter our metadata further down the supply chain – retail customers searching on Amazon, librarians filtering results on purchasing platforms, researchers finding our books and journals through citations on popular online search engines – and consider what elements of our metadata might help reach those people in the right ways.

JM: Ideally, you’ll see an uplift in sales or usage for every metadata element that you add, review or expand, although it can be challenging to quantify and prove a direct correlation between richer metadata and higher revenue or discoverability, as there are will be other factors involved. For my Operations team, what is certain is that richer, more comprehensive metadata means fewer errors are thrown up by the distribution systems and feeds we use, which means colleagues save time and gain productivity by not having to resolve and rerun failed jobs, chase missing information from other teams, or manually send information to third parties. My job is also made easier because things like size and weight of every printed product are recorded in our bibliographic database as standard, easy to report on and analyse, which helps with forecasting costs for inventory storage or shipping. Metadata can be powerful.

Solving your technical support questions in a snap!

Isaac Farley — Thu, 25 Jan 2024 00:00:00 +0000

My name is Isaac Farley, Crossref Technical Support Manager. We’ve got a collective post here from our technical support team - staff members and contractors - since we all have what I think will be a helpful perspective to the question: ‘What’s that one thing that you wish you could snap your fingers and make clearer and easier for our members?’ Within, you’ll find us referencing our Community Forum, the open support platform where you can get answers from all of us and other Crossref members and users. We invite you to join us there; how about asking your next question of us there? Or, simply let us know how we did with this post. We’d love to hear from you!

A little about us and what drives the team

I’m fortunate to manage a great team - Evans, Kathleen, Paul, Poppy, and Shayn - who enjoy and are hardwired to guide. We have different strengths and interests, but the thing that unites us is that we are energized when we can unpick tricky problems for all of you, our members and users. In 2023, the technical support team answered around 11,000 questions from all of you. We do that with one-to-one requests sent to us via email and within our support center (using a closed-source software called Zendesk). And, we’ve been providing more and more support in our Community Forum, where we’re aiming for open interactions, so we can all learn from the rich exchanges with all of you (the Forum has an integration with Zendesk, so posts made in the Forum are delivered to us there, so our team won’t miss any of your questions).

We established in the previous paragraph that we have a great technical support team who all pride themselves on helping you. But we’re also human; the reality is that many of those ~11,000 technical support questions asked of us in 2023 were repetitive, and there are always trends in the questions asked. That’s another important reason why we’re hoping to have more and more of these questions asked and answered within our Community Forum; again, so we can all learn from one another. We know certain parts of content registration, metadata retrieval, and everything in between are, well, complicated. The Crossref learning curve can be steep for all of us. Collectively, our technical support team has more than 25 years of Crossref experience, and we’re continuously learning new things about the Research Nexus and the scholarly ecosystem from one another and all of you.

Learning through this complexity is one of the most enriching parts of our days. Our daily stand-up, modeled off of different software development methodologies, where together we troubleshoot tangly questions from all of you, share ideas, and just keep up-to-date on the latest from across the organisation leads to a lot of knowledge exchange. So, years ago, we decided to transform the issues we discuss in those stand-ups into public-facing posts in our Community Forum. It gave us the opportunity to share much-needed examples in a new community space; and, we knew, since these were the issues we were all discussing and learning from ourselves, that many of you would also benefit from us surfacing the topics openly. We call these posts tickets of the month, since the majority of topics we discuss have originated from tickets in our support center.

Examples of some of the most popular topics in the last two-plus years have been:

Snapping our fingers

Like I said, these posts originated from real-life questions of us from our community members. In most cases, we’ve been asked these questions by many of you. These Community Forum posts are our attempts to unlock understanding of our services, rich metadata, or the larger Research Nexus. Said another way: we all see value in putting in the effort to post one more example or answer that nuanced question. Perhaps one of our posts will include an example that really resonates with you and/or your work.

In that spirit, I asked Evans, Kathleen, Paul, Poppy, and Shayn to answer this question below (yes, I’m going to weigh in, too):

What’s that one thing that you wish you could snap your fingers and make clearer and easier for our members?

Evans, Technical Support Specialist

As a publisher and a Crossref member, at one point or another, you might have made a mistake in the metadata deposited for a given DOI. I’m sure after the slight ‘shock’, the next question you had in mind was, ‘How can I correct this mistake?’ Well, here is a simplified guide on how to do that correction/update!

Can I modify/ update the metadata of a registered DOI? As indicated by my colleague Shayn below in this blog post, Crossref DOIs are designed to be persistent (and cannot be changed/deleted once registered). And YES, you can update the metadata associated with any of your registered DOIs whenever necessary, at no additional fee.

How can I perform a standard metadata update? To add, change, or remove any metadata element from your existing records, you generally just need to resubmit your complete metadata record with the correct/new changes included. How you choose to update a DOI metadata record is highly dependent on the content registration tool/platform you are using/comfortable working with, as described below:

OJS: Navigate to the article record you wish to update, add in your new metadata/delete relevant metadata fields, and deposit it again using the Crossref import/export plugin. You must be running at least OJS 3.1.2 and have the Crossref import/export plugin enabled.
Web deposit form: Open the web deposit form, and re-enter all the metadata, including the new changes - leave the relevant field blank to delete it, or add in your new metadata to update it - and resubmit the form (note: there are a handful of exceptions to this for the web deposit form).
Depositing XML files with Crossref: Make changes to the relevant XML file and resubmit it to Crossref via the admin tool. When making an update, you must supply all the bibliographic metadata for the record being updated, not just the fields that need to be changed. During the update process, we overwrite the existing metadata with the new information you submit, and insert null values for any fields not supplied in the update. This means, for example, that if you’ve supplied an online publication date in your initial deposit, you’ll need to include that date in subsequent deposits if you wish to retain it. Note that the value included in the element must be incremented each time a DOI is updated.

If you’re looking for real-life examples of other members who have updated their metadata, the Community Forum is a great starting point. If you have follow-up questions on any of the existing threads, I invite you to post a message today.

Kathleen, Technical Support Specialist

One of my favorite types of queries to tackle are those regarding content registration problems. I love a good mystery and getting to the bottom of why that pesky submission just didn’t succeed. Sometimes members come to us with an error message and specific questions about what has gone awry. But, in fact, two of the most common questions we receive are: 1) I deposited something; did it work? and 2) I deposited something; why isn’t it showing up?!

To address the first question of whether your submission went through or not, I wrote a forum post back last June talking about how to use the admin tool to see whether your registration was successful or not. We know there are also email alerts and perhaps status messages within your own registration platform, but using the admin tool is a great way to concretely check where your submission has ended up. If it’s not there, we didn’t get it!

Using the admin tool is also a great way to get more details about the submission and more information in case the submission happened to fail. You may have had the experience in which you contacted us with a question about a failed deposit, and we asked you for the submission ID. You can find that info in the admin tool! And we ask for that, because that helps us get to the bottom of those error message mysteries.

And, as for the second question of when will your DOI be active, my colleague, Paul, wrote a fantastic post on the forum (with an excellent flowchart and all!), explaining when you can expect to see your DOI up and running. Often members will submit a deposit and expect the DOI to resolve immediately. When that doesn’t happen, many think that something has gone wrong or perhaps there is an error, but, in fact, our systems may still be updating and processing the metadata.

I recommend giving these two posts a read if you’re at all concerned about whether you’re depositing your content correctly or not. Hopefully, they’ll help ease your content-registration worries.

Isaac, Technical Support Manager

Oh, thanks for asking! Many of our members, after receiving one of our reports, will respond to us in support with a message similar to: ‘What did I do wrong? Please help me fix this. I don’t want to be out of compliance!’

The receipt of one of our reports does not necessarily mean that you’ve done anything wrong. In truth, the reports we send to our official member contacts are produced using very simple logic. It’s true that they may signal larger, more complicated problems, but we really need your help to determine next steps (and, in some cases, no action is needed because there is no issue for members to fix (e.g., many failed resolutions within the resolution reports)).

Let’s look at the conflict and resolution reports since those are the reports we get the most questions about:

Conflict reports are the most complicated of our reports to navigate. But, the reports are generated using simple logic: if you register two or more DOIs with matching bibliographic metadata, we’ll flag those DOIs as being in conflict, which will generate a warning message at the time of registration and a subsequent conflict report. When members receive this report, we often get the sense that members simply want us, the technical support team, to tell them how to fix it. The problem is we don’t know your content, so we don’t know if the two DOIs do represent a duplicate, or if both DOIs, while having very similar bibliographic metadata, are legitimate and will be maintained going forward (e.g., for errata). Paul wrote a great post in our community forum about what conflicts are and how to resolve them.

Resolution reports, like conflict reports, are generated using simple logic: a resolution is the result of a click on that DOI. If a DOI has been registered, that click results in a successful resolution. If that DOI has not been registered, that click results in a failed resolution. Our monthly report is a count of those resolutions - successful and failed. Failures can represent content registration errors in a member’s workflow. Or, they can signal that an end user has made a mistake when attempting to click the DOI in question. So, for example, an end user perhaps added an extra period onto their DOI link. Instead of trying to resolve https://doi-org.turing.library.northwestern.edu/10.5555/cupnfcm2wj, a legitimate DOI, they added a period to the end and tried to resolve https://doi-org.turing.library.northwestern.edu/10.5555/cupnfcm2wj. instead. That extra period at the end of the DOI has made it a completely different DOI that is not registered with us, thus they get a failed resolution. This is pretty common. For members with content being regularly clicked, there will be user errors in the logs appearing as failed resolutions. The first question members should ask themselves when reviewing the failed .csv report within the resolution report is: ‘are any of these DOIs legitimate DOIs that I thought we had registered?’ We have more on the basics of resolution reports also over in our Community Forum.

Paul, Technical Support Specialist & R&D Support Analyst

I know we were asked to name “one thing” but I have two that are closely related. May I snap my fingers twice and fix two issues? [Of course, Paul! Take it away!]

Paul’s first snap

One of the most asked questions we get in support is “why is my DOI not working?” 90% of the time it is down to a failed submission. A good proportion of those failures are a result of title mismatches between the deposited container title and the one we have stored on the system here. There are other error messages that occur, too, which I wrote about back in 2020.

So, “why do we fail submissions because of title differences?” You might ask.

Well, the title and ISSN/ISBN and/or the title level DOIs act like locks to the title record, which need the right keys to unlock the title so that you can add or update the records against it. So if you don’t match what was in the original submission, you get a failure. Without that stringent check, we would have way too many iterations of titles and matching to those would be a nightmare. Not to mention sorting those DOIs into one container in the REST API.

Isaac wrote a great forum post about these title-level issues as well.

If a title update is required due to an error with an original title deposit, then these need to be made by the support team, so get in touch with us on the Community Forum.

And, a second

Permissions against titles and DOIs: Lots of our members don’t realise that each DOI has its own permissions against the prefix that currently ‘owns’ or is associated with that DOI in the background.

It would be fair to assume you can tell just by looking at a DOI who the current publisher is, based on the prefix at the start —but that’s not always the case. Things can (and often do) change. Individual journals get purchased by other publishers, and whole organisations get bought and sold.

What you can tell from looking at a DOI prefix is who originally registered it, but not necessarily who it currently belongs to. That’s because if a journal (or whole organisation) is acquired, DOIs don’t get deleted and re-registered to the new owner. The update will of course be reflected in the relevant metadata, but the prefix on the DOI will stay the same. It never changes—and that’s the whole point, that’s what makes the DOI persistent. Isaac also wrote this in much more detail and explains the internal Crossref processes in his blog “What can often change, but always stays the same?“

These permissions are very important to understand when it comes to title transfers and working with updating your metadata against transferred DOIs to prevent duplicate DOIs for the same work.

Poppy, Technical Support Contractor

As a researcher myself, I’d like to talk about references in a journal article, book, conference paper, etc. (I’ll just use ‘article’ going forward for simplicity). These are the references included in an article by the author. References in one article result in citations for another article. It’s the thing every author dreams of and accruing citations can be a big deal for authors, journals, and publishers.

For readers, articles with no references can be less discoverable using systems that use citation links for relevance, and that discoverability is of critical importance for our members who decide to register references with us. We all want your content to be shared, cited, linked, and used far and wide.

We receive many questions from authors asking why citations don’t show up; it’s usually due to metadata deposits with no references included. There may be an assumption that our process is like Google Scholar, which crawls full text and websites. This misunderstanding has a big impact on references and citation counts. However, as we do not store a copy of the paper, our intake system does not extract references from the article, regardless if they have a DOI. This is one of the main reasons that Crossref citation counts are lower than services that use extraction methods. We only store the data that a publisher registers and maintains with us. On deposit of a metadata record that includes references, our system performs a matching process - if there is a match, a cited-by connection is applied to the metadata. With deposits with no references, however, there is no data to match to other articles (and, therefore limitations on the discoverability and no cited-by count increase).

An article with no references has big impacts for the authors, the journal, the publisher, researchers, and ultimately, the readers. This can mean decreased distribution of the content itself, reduced citation counts for cited articles, lower impact metrics for journals, and can ultimately affect value for publishers. For example, researchers just don’t include articles without references for scientometric analysis.

Our documentation on references includes the elements for both structured and unstructured data. Including the DOI in the structured data is best practice as it provides a precise location with rich data for matching. If the matcher does not see a link between the deposited DOI and the cited DOI at the time of deposit, then the references are stored to be crawled with other matching algorithms later. So, we’re always working to create those rich cited-by linkages between works (raising the content’s profile and overall discoverability), no matter when you register reference metadata. You can also see how your publisher is doing on depositing references by viewing their Participation Report. If you are an author, you can check if your DOIs that were registered contained any references by using our REST API. Don’t see them? You can always contact the editor of the journal or the publisher that published your paper and ask them to add them. Didn’t hear back? Just drop us a line in the Community Forum, we’re happy to help.

Shayn, Technical Support Specialist

Let’s ‘zoom out’ to the big picture. What are DOIs for? What makes them useful? What are we all doing here anyway?!

There are a lot of different answers to those questions. It’s a complex picture. But, way back in the late ‘90s, the DOI system was designed in order to allow for the creation of unique and persistent identifiers. Crossref members use these identifiers to represent their research outputs and publications. This allows for reliable linking to those items, and the ability to identify and communicate the relationships between them, notably (but not exclusively!) citation relationships.

So, what do we mean when we say that Crossref DOIs should be unique and persistent? In basic terms, unique means that there is only a single Crossref DOI registered for a given citable research output. And, persistent meaning that the DOI associated with a given research output today will continue to be associated with, and link to, that same research output indefinitely into the future.

Yes, there are some grey areas, and we know that everything doesn’t always work 100% perfectly all the time. But, the more deviations from persistence and uniqueness, the harder it becomes for end-users, publishers, Crossref, and other services which make use of our metadata to reliably find research outputs and reliably relate them to one another. It weakens the value and utility of DOIs for everyone.

So, what does this mean in practice?

Be certain that every item you register with Crossref is something you can maintain in the long-term.
- Have an arrangement with an archive that can take responsibility for your content if your organisation stops hosting it or ceases to exist.
- Don’t register things that you know will only exist for a short time.
When you’re about to register new content, be absolutely sure that it hasn’t been registered already, either by your organisation or any other organisation.
- If you acquire a new journal from another publisher, have a process in place to check what content has already been registered and adopt the use of the DOIs registered by the prior publisher for that content. We can always provide a list of the existing DOIs for a journal.
- If you publish books, and have a co-publishing agreement with another publisher, distributor, or hosting platform, be aware that one of those other parties may have already registered DOIs for your books. Adopt the use of those DOIs rather than assigning and registering new ones. And, if you don’t want them to do that going forward, communicate that to your co-publishing partners.
When mistakes happen, inadvertently resulting in duplicate DOIs for a single item, identify them quickly. Alias the new duplicate DOI to the long-standing original DOI, and remove all instances of the new DOI from your website or platform.
Ensure that your publishing software, platform, or journal management system can accommodate DOIs with various prefixes for the same publication. You should be able to use (display, link, update metadata and URLs for) the DOIs registered for older content by any prior publishers as easily as you use the DOIs that you registered yourself for more recent content.

Things like persistence and uniqueness can sound like theoretical abstractions, but they actually play an important role in the day-to-day grind of your publishing operations. Their impact on linking, citing, discovery, and analysis of your content is concrete and important. Thus, it’s not surprising that we often hear from members and others in the research community who share this commitment to persistence, uniqueness, and overall rich, accurate metadata. You’ll see that play out in the Community Forum where members and users get involved to troubleshoot issues, compare notes, and share ideas with us and one another. We appreciate the commitment to the Research Nexus and the overall spirit to serve in this growing community. Like we said at the top, we’re all wired to contribute in this way, so building an open, welcoming space that moves us forward excites us.

Again, we invite you to join the discussion on this and many other Crossref-related topics over in our Community Forum.

What was the talk of #Crossref2023?

Kornelia Korzec — Tue, 21 Nov 2023 00:00:00 +0000

Have you attended any of our annual meeting sessions this year? Ah, yes – there were many in this conference-style event. I, as many of my colleagues, attended them all because it is so great to connect with our global community, and hear your thoughts on the developments at Crossref, and the stories you share.

Let me offer some highlights from the event and a reflection on some emergent themes of the day. You can browse the recordings and slides archived on our Annual Meeting page.

Ginny Hendricks opened the meeting by reminding everyone about the research nexus vision, and the work that’s underway to bring us closer to it. Ginny went on to highlight progress in metadata and relationships being registered by our members, and mentioned members that have particularly rich metadata records – with the special joint recognition for learned societies of South Korea. Participation statistics can be reviewed in our Labs Member Metadata Metrics Tables.

Since 2018 we’ve seen a 512% increase in the number of abstracts included in the metadata; with Wiley’s recent addition of millions of abstracts to their records largely contributing to this change. On the relationships side, in the same period, we’ve noted a staggering 3004% growth in preprint-to-article links, and we’re pleased to report a growing number of funding relationships being made available thanks to more and more funders registering Crossref DOIs for grants.

For those who couldn’t join us at such an early hour, Ed Penz included some of these highlights in his own strategic update later in the day. However, he focused on our activity and plans towards fulfilling our four strategic goals:

To contribute to an environment where the community identifies and co-creates solutions for broad benefit
To be a sustainable source of complete, open, and global scholarly metadata and relationships
To be publicly accountable to the Principles of Open Scholarly Infrastructure (POSI) practices of sustainability, insurance, and governance
To foster a strong team—because reliable infrastructure needs committed people who contribute to and realise the vision, and thrive doing it

Speakers from across our global community shared their initiatives too. Most of these talks have been accompanied by posters or abstracts shared on our Community Forum and still available for preview and discussion:

Making data citations available at scale: The Global Open Data Citation Corpus by Iratxe Puebla;
“Who Cares?” Defining Citation Style in Scholarly Journals by Vincas Grigas and Pavla Vizváry;
DOI registration for scholarly blogs by Martin Fenner;
Enhancing Research Connections through Metadata: A Case Study with AGU and CHORUS by Tara Packer, Kristina Vrouwenvelder, Shelley Stall;
Index Crossref, Integrity, Professional And Institutional Development by Engjellushe Zenelaj;
Brazilian retractions in the Retraction Watch Database - RWDB by Edilson Damasio; and
Now that you’ve published, what do you do with Metadata? - by Joann Fogleson.

In addition to these updates, we’ve heard from:

Izabela Szyprowska (OP, European Commission), Nikolaos Mitrakis (RTD, European Commission), and Paola Mazzucchi (mEDRA) talked about the process and rationale of implementing Crossref DOIs for grants at the European Commission; and
Amanda French from ROR/Crossref about the new ‘ROR / Open Funder Registry overlap’ tool.

We also assembled a diverse panel and invited the community to discuss “What we still need to build a robust Research Nexus?” The discussion ranged from how different parts of our community currently use existing metadata, to how we can come together to make improvements, especially in the area of standards and equitability, and touched on metadata priorities. I’ll highlight some of the threads below, but it’s certainly worth engaging with the full recording of the discussion, and offering your own perspective on the Community Forum, commenting below.

Having participated in the whole day of talks, I found that a few themes emerged as popular in the community: data citations, making it easier to register metadata, making better use of metadata, retractions, and equity of participation in the research nexus.

Data citations

With the advances in the Crossref API relationships endpoint, Martyn Rittman demonstrated how we’re now providing more comprehensive support for data citations. You can follow his demonstration in the Collab Notebook he used for the demo and shared for your perusal. He also mentioned that the developments in this feature of our API will soon replace the current service provided via the Events API. Feel free to connect with Martin on the community forum and comment with questions and suggestions.

As mentioned above, DataCite’s Iratxe Puebla mentioned the Make Data Count initiative and the leaky pipeline of data citations we’ve got at the moment in the scholarly literature, obscuring the true picture of data reuse. This prevents the community from recognising and incentivising data creation and reuse appropriately. One way of addressing this is the Global Open Data Citation Corpus. Crossref and DataCite collaborate closely in connecting and making that data available.

Linking datasets, as well as software, was reported as part of the AGU and CHORUS initiative in Enhancing Research Connections through Metadata.

Data sharing and citing is as much a culture as a technology problem. As Iratxe Puebla admitted, there are many norms and processes for capturing and sharing that information,and DataCite is interested to hear about different use cases. As highlighting data’s relationship with works is a growing interest for our community, hopefully more understanding and perhaps even commonality can be built soon.

Making it easier to register metadata

As part of the Demonstrations session, we’ve seen two developments to support members with registering their metadata more easily.

Crossref’s Lena Stoll shared plans for the new version of the Crossref Registration Form, the helper tool for manual registration of metadata, which translates the submission into XML, for inclusion in the Crossref database. At the moment, the form only accepts grant registrations, but it will be bolstered before the end of the year to include journal articles then other record types in time.

Erik Hanson from PKP demonstrated the latest OJS version, commenting on specific changes made in the new version in response to the key pain points reported by users of the previous release.

In addition, we’ve heard of two independent projects by Martin Fenner and Esha Data to enable metadata registration and Crossref DOIs for scholarly blogs.

Making better use of metadata

Supported by the beginner’s demo of our REST API by Luis Montilla, there were many voices about opportunities for making good use of Crossref’s open metadata. Nikolaos Mitrakis of the European Commission talked about the implementation of Crossref IDs for grants as a step towards tracing and connecting the grants with not just academic but also societal outcomes of the awards, and the plans for using those in the evaluation and steering of their funding programmes.

Joann Fogleson of the American Society of Civil Engineers gave a buzzy metaphor of publishers’ role in their work with metadata being comparable with that of a pollinator – collecting the metadata at one end, then registering, displaying and making it available to different services, in order to enable a reacher scholarly environment for discovery.

Many of the major themes have found their way to the discussion of what is still needed to build a robust network of connections between scholarly objects, institutions and individuals. One of the ways Ludo Waltman of CWTS, Leiden University, intends to use our open metadata is as part of the upcoming open-source version of the Laiden rankings and he invited the community to contribute and help optimise this project to provide an alternative to closed and selective databases.

Panellists also spoke of new opportunities in the light of data mining and machine learning. Ran Dang, Atlantis Press, as a publisher shared a concern about the standard of metadata across cultures and disciplines, and the need to digitise past publications – which can then help better leverage multi-lingual scholarship. Matt Buys of DataCite, pointed out to the Global Data Citation Corpus they are developing, which leverages a SciBERT model to pull out data citations, which is brought together with Crossref/DataCite citation metadata.

Opening the data is essential to enabling its wider use, and here Ludo gave the example of the fantastic outcome for references metadata, which has been made open by default for the entire corpus of Crossref-registred works. He hopes that this can inspire us to make similar progress in other areas.

A little on a tangent with regards to metadata use, yet speaking of excellent examples of the community making progress together, Ginny pointed out ROR, how this is becoming a new standard for solving a longstanding problem of standardising affiliations metadata.

Retractions

Perhaps not entirely surprising, given the recent acquisition of the Retraction Watch database by Crossref and making the data openly available, retractions featured in a few different talks at the meeting. First, Lena Stoll and Martin Eve from Crossref, shared how that data can be accessed – that is as the csv file from https://api-labs-crossref-org.turing.library.northwestern.edu/data/retractionwatch?[your-email@here](add your email as indicated), and the Crossref Labs API also displays information about retractions in the /works/ route when metadata is available. There are plans for incorporating this information with our REST API in the future.

Ed and Ginny have shown stats for increases in retraction metadata registered in Crossmark but commented on limited participation in Crossmark overall. Recording retraction information in this way is still important, alongside the Retraction Watch data, this allows for multiple assertions of that information, and increases confidence in its accuracy. We’re preparing to consult with the community at large about the future direction of the Crossmark service, to make it easier to implement and more useful for the readers.

Finally, Edilson Damasio from State University of Maringá-UEM, Brazil, and a long-time Crossref Ambassador, presented the analysis of Brazilian records in the Retraction Watch data, and he promises further analysis to come, comparing the situation across geographies.

Equity of participation in the research nexus

Amanda Bartell opened the research nexus discussion with a reminder of what that vision entails and pointing out commonality of goals in the community – “Like others, Crossref has a vision of a rich and reusable open network of relationships connecting research organisations, people, things, and actions; a scholarly record that the global community can build on forever, for the benefit of society. We call this interconnected network the Research Nexus, but others in the community have different names for it, such as knowledge graph or PID graph.”

The richness of this network depends upon the participation of all those who produce and publish scholarship, so naturally the topic of equality emerged in that discussion. In addition to Ran Dang’s concern for multilingualism and digitisation of past publications from all parts of the world, Mercury Shitindo of St Paul’s University, Kenya talked of the need for more education, training and accessible resources for her community, to be able to participate more effectively in this ecosystem. She can see that affiliations and citations are of priority there, as these enable transparency and facilitate collaborations. Matt Buys of DataCite echoed her point, talking about the importance of the role of contributors “It’s important not to lose sight of people and places – to recognise the importance of contributor roles in the PID-graph”.

Earlier in the day, we mentioned the launch of our Global Equitable Membership, or GEM programme. Since January, 110 new organisations from eligible countries have joined Crossref fee-free. Ginny was quick to admit that the need for a fee-waiver programme like this stems from the regular fees schedule not being in tune with our global membership, and she mentioned the upcoming fees review.

Financial barriers are often what get attention, yet reducing barriers to participation with technology is equally important for building a robust research nexus. With the planned changes to our registration form, we’ll make it easier to register works for those who don’t regularly use XML.

Johanssen Obanda took time to show the examples of community activity and events organised by our global network of Ambassadors, and to thank all our advocates and partners for their tireless work. They are also helping tackle barriers, supporting our members to actively participate in the research nexus with their metadata, and help enable the community to make good use of the network of relationships that data denotes.

Showcasing our “One member one vote” truth, the Board election was the focal point of the annual meeting, as always. We closed the ballot and announced the results, with seven members selected to join the Board in 2024.

The event went very smoothly overall. Talks were delivered efficiently, the panellists shared diverse perspectives and we elected our new Board members. Huge thanks to Rosa Clark, our Communications and Events Manager, who orchestrated the event and has been a constant behind-the-scenes presence supervising the entire show. I’m grateful to all colleagues at Crossref, who helped make it an enjoyable experience and an informative event for our community. Finally – it wouldn’t be a real meeting without the active participation of the speakers and panellists, who shared their metadata stories, and even joined us for some relaxed unplugged chats.

News: Crossref and Retraction Watch

Ginny Hendricks — Tue, 12 Sep 2023 00:00:00 +0000

Crossref acquires Retraction Watch data and opens it for the scientific community

Agreement to combine and publicly distribute data about tens of thousands of retracted research papers, and grow the service together

12th September 2023 —– The Center for Scientific Integrity, the organisation behind the Retraction Watch blog and database, and Crossref, the global infrastructure underpinning research communications, both not-for-profits, announced today that the Retraction Watch database has been acquired by Crossref and made a public resource. An agreement between the two organisations will allow Retraction Watch to keep the data populated on an ongoing basis and always open, alongside publishers registering their retraction notices directly with Crossref.

Both organisations have a shared mission to make it easier to assess the trustworthiness of scholarly outputs. Retractions are an important part of science and scholarship regulating themselves and are a sign that academic publishing is doing its job. But there are more journals and papers than ever, so identifying and tracking retracted papers has become much harder for publishers and readers. That, in turn, makes it difficult for readers and authors to know whether they are reading or citing work that has been retracted. Combining efforts to create the largest single open-source database of retractions reduces duplication, making it more efficient, transparent, and accessible for all.

Product Director Rachael Lammey says, “Crossref is focused on documenting and clarifying the scholarly record in an open and scalable form. For a decade, our members have been recording corrections and retractions through our infrastructure, and incorporating the Crossmark button to alert readers. Collaborating with Retraction Watch augments publisher efforts by filling in critical gaps in our coverage, helps the downstream services that rely on high-quality, open data about retractions, and ultimately directly benefits the research community.”

The Center for Scientific Integrity and the Retraction Watch blog will remain separate from Crossref and will continue their journalistic work investigating retractions and related issues; the agreement with Crossref is confined to the database only and Crossref itself remains a neutral facilitator in efforts to assess the quality of scientific works. Both organisations consider publishers to be the primary stewards of the scholarly record and they are encouraged to continue to add retractions to their Crossref metadata as a priority.

“Retraction Watch has always worked to make our highly comprehensive and accurate retraction data available to as many people as possible. We are deeply grateful to the foundations, individuals, and members of the publishing services industry who have supported our efforts and laid the groundwork for this development,” said Ivan Oransky, executive director of the Center for Scientific Integrity and co-founder of Retraction Watch. “This agreement means that the Retraction Watch Database has sustainable funding to allow its work to continue and improve.”

Please join Crossref and Retraction Watch leadership, among other special guests, for a community call on 27th September at 1 p.m. UTC to discuss this new development in the pursuit of research integrity.

Supporting details

Crossref retractions number 14k, and the Retraction Watch database currently numbers 43k. There is some overlap, making a total of around 50k retractions.
The full dataset has been released through Crossref’s Labs API, initially as a .csv file to download directly: https://api-labs-crossref-org.turing.library.northwestern.edu/data/retractionwatch?name@email.org (add your ‘mailto’). Edit: 2024-10-10: The full dataset is available in a git repository at https://gitlab.com/crossref/retraction-watch-data.
The Crossref Labs API also displays information about retractions in the /works/ route when metadata is available, such as https://api-labs-crossref-org.turing.library.northwestern.edu/works/10.2147/CMAR.S324920?name@email.org (add your ‘mailto’). If you don’t have a .json viewer, please see below for screenshot.
Crossref is paying an initial acquisition fee of USD $175,000 and will pay Retraction Watch USD $120,000 each year, increasing by 5% each year.
The initial term of the contract is five years. ~~The full text of the contract will be made public in the coming fortnight.~~ EDIT 2023-09-26: Here is the signed agreement.
There will be a community call on 27th September at 1 p.m. UTC (your time zone here). Please register.
An open FAQ document is available to collect questions to be answered at the webinar.
This announcement will always be accessible via Crossref DOI https://doi-org.turing.library.northwestern.edu/10.13003/c23rw1d9; please use this persistent link for sharing.

About Retraction Watch and The Center for Scientific Integrity

The Center for Scientific Integrity is a U.S. 501(c)3 non-profit whose mission is to promote transparency and integrity in science and scientific publishing, and to disseminate best practices and increase efficiency in science. In addition to maintaining and curating the Retraction Watch Database, the Center is the home of Retraction Watch, a blog founded in 2010 that reports on scholarly retractions and related issues in research integrity.

About Crossref

Crossref is a global community infrastructure that makes all kinds of research objects easy to find, assess, and reuse through a number of services critical to research communications, including an open metadata API that sees over 2 billion queries every month. Crossref’s >19,000 members come from 151 countries and are predominantly university-based. Their ~150 million DOI records contribute to the collective vision of a rich and reusable open network of relationships connecting research organisations, people, things, and actions; a scholarly record that the global community can build on forever, for the benefit of society.

Enquiries

For Retraction Watch/Center for Scientific Integrity: Ivan Oransky, ivan@retractionwatch.com
For Crossref: Ginny Hendricks, ginny@crossref.org

A screenshot of an example Labs API metadata record with a Retraction Watch-asserted retraction

Start citing data now. Not later

Geoffrey Bilder — Thu, 23 Mar 2023 00:00:00 +0000

Recording data citations supports data reuse and aids research integrity and reproducibility. Crossref makes it easy for our members to submit data citations to support the scholarly record.

TL;DR

Citations are essential/core metadata that all members should submit for all articles, conference proceedings, preprints, and books. Submitting data citations to Crossref has long been possible. And it’s easy, you just need to:

Include data citations in the references section as you would for any other citation
Include a DOI or other persistent identifier for the data if it is available - just as you would for any other citation
Submit the references to Crossref through the content registration process as you would for any other record

And your data citations will flow through all the normal processes that Crossref applies to citations. And it will be distributed openly to the community (including DataCite!) via Crossref’s services and APIs. All data citations deposited with Crossref will be exposed in the (soon-to-be launched) Data Citation Corpus.

And then, you can sit back and congratulate yourself for making your publication more useful to researchers who want to be able to reuse the data underlying your publications.

Background

You might ask, “So if submitting Data Citations to Crossref has long been possible, why do you have to write this?”

Historically, authors did not cite data in the way they cited publications. Instead, they would often refer to the data in the main text of the article. This has made it hard to determine what data lay behind the research and/or access the data.

But the research community has increasingly recognized that data is a first-class research output and that we should treat it as such. In short, we should formally cite data.

But because citing data is a comparatively new practice, it has been subject to a lot of new analysis. And unsurprisingly, people analyzing data citation have discovered that there is a lot of nuance to citation of any kind.

There are lots of reasons for citing something. There are lots of internalized conventions for citing things. And there are different conventions for citation for different research objects. And SSH citation practice differs from STEM. And legal citation practices are different from scholarly citation practices. And citation practices even vary by subdiscipline and by journal.

Those who have been looking at what it means to “cite data” have naturally stumbled into a thicket of divergent practices - some of which are historical holdovers, some of which are stylistic preferences, and some of which are clearly adaptations to deal with the specific needs of certain research objects/containers or different disciplines.

The temptation has been to try and rationalize this before extending the practice of citation to data.

“Maybe because data is a distinct record type, we should include the fact that it is a data citation in the citation itself?”

“Maybe because people cite data for different reasons, we should include a typology of citation types in all data citations?”

And so you may hear some people say, “hold off on data citation - we don’t have an optimal way to do it yet, and it can be very complicated.”

But guess what?

We currently don’t label citations to monographs as “citation to monograph.”

And we don’t currently include the reason for citation when we are citing a journal article.

It would be very cool if we did. And it would likely make citations even more useful if we did.

But citations are already useful even without these features. And so, to delay citing data indefinitely because we have an opportunity to improve the act of citation is just perverse. Our community has always opted for progress over perfection.

For one thing - the efforts are not mutually exclusive. We can start citing data with the current limitations of citation practices and simultaneously propose mechanisms for making citation more useful in the future, including new guidelines to deal with the unique issues that citing data poses.

But in the meantime, we will be doing researchers a giant favour if we at least include our imperfect and ambiguous, and unconventional references to data in the references section of an article so that they can be accessed and processed along with all the other imperfect, ambiguous and variant citations that we find so useful.

Some of our members are already doing this. They have been for a long time. And they haven’t found it any more complicated than managing non-data references in the past.

Join them and make your metadata more useful.

Cite data now. Don’t put it off.

And Crossref will continue to work with DataCite and the rest of the community to make the distribution even easier and more useful.

So who is already citing data?

Top 10 members depositing data citations from November-May 2022

(broken down by DOI prefix, which is why you see some publishers listed twice):

Prefix	Member name	Data citations deposited
10.1038	Springer Science and Business Media LLC	7174
10.1016	Elsevier BV	6527
10.1007	Springer Science and Business Media LLC	4748
10.5194	Copernicus GmbH	3017
10.1080	Informa UK Limited	2346
10.1177	SAGE Publications	2082
10.1002	Wiley	2048
10.1111	Wiley	1888
10.1108	Emerald	1876
10.3390	MDPI AG	1827

Top 10 data citations per deposited work

(again, broken down by prefix)

Member name	Prefix	Data citations deposited	Data citations per work
Consortium Erudit	10.7202	580	1.149
SLACK, Inc.	10.3928	462	0.646
S. Karger AG	10.1159	1653	0.532
Proceedings of the National Academy of Sciences	10.1073	973	0.502
American Academy of Pediatrics (AAP)	10.1542	486	0.397
F1000 Research Ltd	10.12688	552	0.341
American Association for the Advancement of Science (AAAS)	10.1126	952	0.317
Springer Science and Business Media LLC	10.1038	7174	0.231
JMIR Publications Inc.	10.2196	864	0.187
American Geophysical Union (AGU)	10.1029	692	0.166

These are for the prefixes with the most data citations deposited (>500 in 6 months) so there might be smaller members doing better than this.

Summaries are great, but I want to see some actual examples!

Here are some examples showing how data is cited by our members:

This eLife article: https://doi-org.turing.library.northwestern.edu/10.7554/eLife.26410 cites this dataset in Dryad https://doi-org.turing.library.northwestern.edu/10.5061/dryad.854j2.
This Copernicus article: https://doi-org.turing.library.northwestern.edu/10.5194/acp-22-7105-2022 cite to this dataset https://doi-org.turing.library.northwestern.edu/10.24381/cds.bd0915c6
This Sciendo article: https://doi-org.turing.library.northwestern.edu/10.2478/plc-2021-0008 cites this APA-hosted language competence test https://doi-org.turing.library.northwestern.edu/10.1037/t15159-000
This De Gruyter article: https://doi-org.turing.library.northwestern.edu/10.1515/opth-2020-0160 cites this bibliography at Oxford Bibliographies: https://doi-org.turing.library.northwestern.edu/10.1093/OBO/9780195396584-0012

And here are some example API requests for discovering more metadata citations. You can use these API requests as examples and adapt to your own needs.

Shooting for the stars – ASM’s journey towards complete metadata

Kornelia Korzec — Tue, 14 Mar 2023 00:00:00 +0000

At Crossref, we care a lot about the completeness and quality of metadata. Gathering robust metadata from across the global network of scholarly communication is essential for effective co-creation of the research nexus and making the inner workings of academia traceable and transparent. We invest time in community initiatives such as Metadata 20/20 and Better Together webinars. We encourage members to take time to look up their participation reports, and our team can support you if you’re looking to understand and improve any aspects of metadata coverage of your content.

In 2022, we have observed with delight the growth of one of our members from basic coverage of their publications to over 90% in most areas, and no less than 70% of the corpus is covered by all key types of metadata Crossref enables (see their own participation report for details). Here, Deborah Plavin and David Haber share the story of ASM’s success and lessons learnt along the way.

Could you introduce your organisation?

The American Society for Microbiology publishes 16 peer-reviewed journals advancing the microbial sciences, from food microbiology, to genomics and the microbiome, comprising 14% of all microbiology articles. Six of those are open-access journals, and 56% of ASM’s published papers are open access. Together, our journals contribute 25% of all microbiology citations.

Would you tell us a little more about yourselves?

DH: David Haber, Publishing Operations Director at the American Society for Microbiology. I live in a century-old house that is in a perpetual state of renovation due to my inability to stop starting new projects before I complete old ones.

DP: Deborah Plavin, Digital Publishing Manager at the American Society for Microbiology. Following David’s example, my apartment in Washington D.C. is just up the block from one of the homes Duke Ellington lived in https://www.hmdb.org/m.asp?m=142334.

What value do society publishers in general see in metadata in your view?

DP: In my view, robust metadata allows publishers to look at changes over time, do comparative analysis within and across research areas, more easily identify trends, and plan for future analysis (e.g., if we deposit data citation information and we change our processes to make it more straightforward, do we see any change in the percentage of articles that include that information, etc.).

DH: To echo Deborah’s point, to be able to name something distinctly and clearly identify its specific attributes is vital to understanding past research and planning for future possibilities. One of our fundamental roles as a publisher for a non-profit society is to properly lay this metadata foundation so that we can provide services and new venues for our members, authors, and readers that match their needs and track with the trends in research. Without good and robust metadata, it is impossible to truly understand the direction in which our community is pointing us.

Metadata for your own research outputs in the last year has grown rapidly. Why such focus on metadata in 2022?

DP: This is something that ASM has been chipping away at over time. Years ago we found that it wasn’t always easy to take advantage of deposits that included new kinds of metadata. That was either because we needed to work out how and where to capture it in the process or because platform providers weren’t always ready — coming up with ways to process the XML that publishers supply in many different ways takes time. These back-end processes that feed the infrastructure aren’t usually of great interest to stakeholders, and so it allowed us to play around, flounder, fail, refine, and try again.

We looked at having 3rd parties deposit metadata for us, and while that helped expand the kind of metadata we were delivering, it created workflow challenges of its own. What turned out to be most effective was budgeting for content cleanup projects and depositing updated and more robust metadata to Crossref.

We also benefited from a platform migration, which allowed us to take advantage of additional resources during that process.

DH: Coming from a production background, I have always been fascinated with the when and how of capturing key metadata during the publishing process. When are those data good and valuable, and when should they be tossed or cleaned up for downstream deliveries? Because Deborah and ASM directors saw a more complete Crossref metadata set for our corpus as a truly valuable target, we were able to really think hard about what kind of data we were capturing and when, how those requirements may have influenced our various policies and copyediting requirements over the years, and how best to re-engineer our processes with the goal of good metadata capture throughout our publishing workflows. From our perspective, Crossref gave us a target, a “this-is-cool-bit-of-info" that Crossref can collect in a deposit; therefore, how can we capture that during our processes while driving further efficiencies? ASM journals had been so driven by legacy print workflows that such a change in perspective (toward metadata as a publishing object) really allowed us to re-imagine almost everything we do as a publisher.

Has the OSTP memo influenced your effort?

DP: I think that the Nelson memo hasn’t changed our focus; instead, I think it’s been another data point supporting our efforts and work in this area.

DH: Deborah is exactly right. The release of this memo only re-affirmed our commitment to creating complete and rich metadata. The Nelson memo points to many possible paths forward, in terms of both Open Access and Open Science, but we feel our work on improving our metadata outputs positions us well to pick a path that best suits our goals as a non-profit society publisher.

How big was this effort? Could you draw us a picture of how many colleagues or parts of the organisation were involved? Did you involve any external stakeholders, such as authors, editors, or others?

DH: It was simple. Took five minutes… In all seriousness, the key is having the support of the organisation as a whole. To do this properly, it is vitally important to know the end from the beginning, so to speak. It is one thing to say let’s start capturing ORCID IDs and deliver them to Crossref, but it is completely another to create a cohesive process in which those IDs are authenticated and validated throughout the workflow. So something as simple as a statement “ORCID IDs seem cool, let’s try to capture them” could affect how researchers submit files, how reviewers log into various systems (i.e., ORCID as SSO), how data are passed to production vendors, what copyeditors and XML QC people need to be focused on, and what integrations authors may expect at the time of publication. Being part of an organisation that embraced such change allowed us to proceed with care with each improvement to the metadata we made.

But that is more about incremental improvement. The beginning of this process started when we were making upgrades to our online publishing platform, and we were trying to figure out how best to get DOIs registered for our older content. When we started looking at this, we soon realized that, sure, we could do the bare minimum and just assign DOIs to this older content outside the source XML/SGML, but did that make sense? Wouldn’t it make more sense, especially since we were updating the corpus to a new DTD, to populate the source content with these newly assigned DOIs? Once we decided that we were going to revise the older content with DOIs, it made sense for us to create a custom XSL transform routine to generate Crossref deposits that would capture as much metadata as possible. So, working with a vendor to clean and update our content for one project (an online platform update) allowed us also to make massive improvements to our Crossref metadata as a side benefit.

Of course, I do have to apologize to the STM community for the Crossref outages in late 2019. That was just me depositing thousands of records in batches one sleepless night.

What were the key challenges you encountered in this project, and how did you overcome them?

DH: Resources and time are always an issue. Much of the work was done in-house in spare moments captured here and there. But there are great resources in github and at Crossref to help focus on defining what is important and what is possible in such a project. And, honestly, defining what was important and weighing that against the effort to find said important bit in the corpus of articles we have was the most challenging part of this process. In other words, limiting the focus. Once one decides to start looking at the inconsistencies in older content, it is hard not to say: “Oh, look. That semi-important footnote was treated as a generic author note rather than a conflict-of-interest statement; let’s fix that.” Once you start down that path, you can spend years fiddling with stuff. For me, a key mantra was: “We now have access to the content. We can always do another Crossref metadata update if things change or shift over time.”

Have there been any important milestones along the way you were able to celebrate? Or any set-backs you had to resolve in the process?

DP: For as long as I can remember, the importance of good metadata has been among the loudest messages of best practice in the industry. I don’t think that I have been able to really quantify/ demonstrate the value of that work. Looking at the consistent increases in the Crossref monthly resolution reports that we saw between 2015 and 2022 and looking at our participation reports has helped provide some measure of progress. For example, the number of average monthly successful resolutions in that Crossref report in 2015 was ~390,000. The last time I checked, the 2022 numbers were ~ 3.7 million. In 2023, I hope that we will be able to leverage Event Data for this as well.

The setbacks have fallen into two categories: timing and process. Our internal resourcing to get this done within our preferred time frame, to have the content loaded and delivered, and triage problems—it’s a battle between the calendar and competing priorities.

DH: When Deborah first shared those stats with me, I was floored. I don’t think either of us suspected such an increase was possible. For me, the biggest setback was mistakenly sending about ~50,000 DOI records to queue and watching them all fail because I grabbed the wrong batch. Ooops. I never made that mistake again, though.

Was any specific type of metadata or any part of the schema particularly easy or particularly difficult to get right in ASM’s production process?

DH: For us, the most difficult piece of metadata revolves around data availability and how we capture linked data resources (outside of data citation resources). Because of our current editorial style (which had been print-centric for years), we did not do a good job of identifying whether there are data associated with published content in a consistent machine-readable way. We did some experiments with one of our journals to capture this outside of our normal Crossref deposit routine, but that was not as accurate or sustainable as we would have liked. But, in that experiment, we learned a few things about how we treat these data throughout our publishing process and we have plans to create a sustainable integrated workflow for this to capture resource/data linkages in our Crossref deposits.

What were your thoughts on last year’s move to open references metadata? Has that impacted on your project in any way?

DP: We were really excited about this; based on the rather limited approach to sorting out impact at the moment, the more metadata we push out into the ecosystem, the more it appears to be used. In my view, that is at the core of what society publishers want to do—ensure that research is accessible and discoverable wherever our users expect to find it.

DH: 100% agree.

How did you keep motivated and on-course throughout?

DP: These kinds of things are never done; for example, we have placeholders for CRediT roles, and getting ready for that work as part of a DTD migration will be the next big thing. The motivation for that is really meeting our commitment to the community, seeing the impact of the author metadata versus article metadata, and seeing what we can learn.

DH: Metadata at its core is one of the pillars of our service as a publisher. To provide the best service, we need to provide the best metadata possible. Just remembering that this can be incremental, allows us to celebrate the large moments and the small. And whether one is partying with a massive 7 layer cake or a smaller cake pop, both are sweet and motivating.

Now that the project is completed, are you seeing the benefits you were hoping to achieve?

DP: This is a hard one to answer as we are using limited measurements at this time. At a high level, I am pleased. While I am eager to leverage event data in the coming year, it would be really helpful to get feedback from the community on how we can improve as well as other ways to evaluate impact.

DH: I want to take up this idea of metadata as a service once more. I don’t mean in terms of discoverability or searchability, either. Let’s take ORCID deposited into Crossref as an example. When done properly (with the proper authentication and validation occurring in the background), we are able to integrate citation data directly to an author’s ORCID profile. We have found that this small service is really appreciated.

Is there any metadata that you’d like to be able to include with your publishing records in the future that isn’t possible currently? What would it be and why?

DP: CRediT roles would be great because it could give greater insight into collaboration within and across disciplines, it could allow for some automation and integration opportunities in the peer review process, and maybe it would visualize aspects of authors’ careers.

DH: I second capturing CRediT roles. What would be really interesting is also creating a standard that quantifies the accessibility conformance/rating of content and passing that into Crossref.

What was the key lesson you learned from this project?

DP: Incremental change can be just as challenging as a massive overhaul, and so it’s important to reevaluate your goals along the way—things always change. There have been cases where we were able to do things that we hadn’t initially thought were feasible.

DH: Always keep the larger goal in mind and remember that any project can birth a new project. Everything does not happen at once.

What’s your next big challenge for 2023?

DP: There is a lot to contend with in the industry right now, and in addition to that we are going through some serious infrastructure changes in our program. With all that madness comes many opportunities. For that reason, when I take a step back from the tactical implications of all that and what we are interested in doing, I think our biggest challenge in 2023 will be identifying what has made an impact and why.

DH: In the short-term, it is making sure that none of our production process changes has negatively affected the past metadata work we spent so much time honing. Once that settles down, it will be determining the best way forward from a publishing perspective in handling true versioning and capturing accurate event data.

Based on your experience, what would be your advice for colleagues from other scholarly publishing organisations?

DP: It can seem daunting, but the small wins can create momentum and do not have to be expensive. Remembering that your publishing program benefits as much as everyone else’s when you deposit more metadata can help refine your short-term and long-term priorities.

DH: Don’t be afraid of making a mess of things. Messes are okay. They aren’t risky. They just reveal the clutter. And clutter gives one reason to clean things up.

THANK YOU for the interview!

About the American Society for Microbiology

The American Society for Microbiology is one of the largest professional societies dedicated to the life sciences and is composed of 30,000 scientists and health practitioners. ASM’s mission is to promote and advance the microbial sciences.

ASM advances the microbial sciences through conferences, publications, certifications and educational opportunities. It enhances laboratory capacity around the globe through training and resources. It provides a network for scientists in academia, industry and clinical settings. Additionally, ASM promotes a deeper understanding of the microbial sciences to diverse audiences. For more information about ASM visit asm.org.

How funding agencies can meet OSTP (and Open Science) guidance using existing open infrastructure

Ed Pentz — Thu, 17 Nov 2022 00:00:00 +0000

In August 2022, the United States Office of Science and Technology Policy (OSTP) issued a memo (PDF) on ensuring free, immediate, and equitable access to federally funded research (a.k.a. the “Nelson memo”). Crossref is particularly interested in and relevant for the areas of this guidance that cover metadata and persistent identifiers—and the infrastructure and services that make them useful.

Funding bodies worldwide are increasingly involved in research infrastructure for dissemination and discovery. While this post does respond to the OSTP guidelines point-by-point, the information here applies to all funding bodies in all countries. It will be equally useful for publishers and other systems that operate in the scholarly research ecosystem.

In response to calls from our community for more specifics, this post:

Provides an overview of the specific ways that Crossref (along with organisations and initiatives like DataCite, ORCID, and ROR) helps U.S. federal agencies—and indeed any other funder—meet critical aspects of the recommendations.
Restates our intent to collaborate with all stakeholders in the scholarly research ecosystem, including the OSTP, the US federal agencies, our existing funder, publisher, and university members, to support the recommendation as plans develop.
References the work and adoption of Crossref Grant DOIs, including analyses of existing metadata matching funding to outputs.
Highlights that what’s outlined in the memo aligns with our longstanding mission to capture and maintain the scholarly record and our vision of the Research Nexus, as we describe in our current blog series, regarding our role in preserving the integrity of the scholarly record (ISR).

Infrastructure already exists to support funder goals; it just needs more adoption

Ensuring free, immediate, and equitable access to metadata that captures the scholarly record is an essential part of meeting the aims of the memo but also supporting Open Science globally.

In September, Crossref ORCID, DataCite, and ROR participated in the 2022 Forum on Global Grants Management run by Altum and the summary provides a good example of the importance of open infrastructure and open metadata to the goals of Open Science:

Open Science begins with open infrastructure: Attendees agreed that Open Science relies on many other ‘opens’ – most notably, open metadata, open infrastructure, and open governance. Metadata and DOIs (digital object identifiers) for publications, grants, and research outputs, are essential to illuminate the connections that exist between funding and outcomes. That metadata runs on infrastructure powered by organisations such as Crossref, ORCID, ROR, and DataCite.

As a foundational scholarly infrastructure committed to meeting the Principles of Open Scholarly Infrastructure (POSI) of governance, insurance, and sustainability, Crossref plays an essential role in implementing and supporting key aspects of the guidance. For many years, we have been focused on the integrity of the scholarly record (ISR), and the shared vision to collectively achieve what we call the Research Nexus, which is described as

A rich and reusable open network of relationships connecting research organisations, people, things, and actions; a scholarly record that the global community can build on forever, for the benefit of society.

Metadata—including persistent identifiers and relationships between different research objects—is the foundation of the Research Nexus and is critical to openly and sustainably fulfilling the OSTP memo’s recommendations.

This topic of open metadata and identifiers isn’t just an issue for research resulting from US federal funding. We are working to implement open scholarly infrastructure globally, bringing significant benefits to the whole scholarly research ecosystem.

The current situation brings to mind the William Gibson quote, “The future is already here - it’s just not evenly distributed yet”. Much of the open infrastructure to support the identifier, metadata and reporting requirements of the OSTP memo already exists, but it is unevenly implemented. Increased collaboration and effort will be needed to bring this all to fruition.

We set out below some steps that all stakeholders can take to meet not just the OSTP guidelines, but Open Science goals more broadly, and globally.

What does ‘adoption’ look like? How exactly do funders and other stakeholders work with this infrastructure?

The OSTP memo calls for specific actions concerning metadata and identifiers where, fortunately, open and global solutions already exist.

For example, item 4 a) says, “Collect and make publicly available appropriate metadata associated with scholarly publications and data resulting from federally funded research.” Crossref and DataCite make metadata, including persistent identifiers (DOIs to be specific), openly available for a broad range of research objects from publications to data. Item 4 b) reads, “Assign unique digital persistent identifiers to all scientific research and development awards and intramural research protocols”. Again, federal agencies and other funders are already joining to register awards and grants and distribute these records openly through Crossref. However, this is an example of uneven adoption as registering awards and grants with DOIs is only being done by a few funders so far, which needs to increase.

Here is an ideal workflow that funders and publishers can already follow

Funders join Crossref to register grants and awards (or indeed any other object such as reports). They apply on our website, accept our terms, and provide key information such as contact details. An annual membership fee ranges from $200-$1200 USD.
Funders and publishers collect ROR IDs and authenticated ORCID iDs for all authors/awardees and their affiliations.
Funders register a Crossref DOI for the award/grant, including awardees’ ORCID iDs and ROR IDs. They send us XML information about the grant (note that we will imminently release an online form to make it easier for the less technical funders). Many funder members register the metadata through a third party, such as Altum (if they use ProposalCentral) or Europe PMC.
At the same time, funders update the awardees’ ORCID record directly with the Crossref Grant DOI and metadata.
Grantees produce research objects and outputs such as data, protocols, code, preprints, articles, conference papers, book chapters, etc.
These objects are registered with Crossref or DataCite, and DOIs are created by the publisher or repository members who include ORCID iDs, Crossref Grant DOIs (gathered from the author), ROR IDs for affiliations for all contributors, and other key metadata such as licensing information, and in the case of publications - references and abstracts. Note that the publisher works its magic (actually, publishers do a lot of editorial and production work, such as including data citations in the references using DataCite DOIs for the data in data repositories).
On the Crossref side, we do a bunch of processing and matching and are planning to refine this and do more. Sometimes relationships are notified and added, such as data citation, preprints related to articles or funding acknowledgements converted from free text to Open Funder Registry IDs and names.
Grant records with Crossref DOIs are now part of the scholarly record. All stakeholders may retrieve the open metadata and relationships through our public APIs. Crossref and DataCite will always provide open metadata, as safeguarded by our respective commitments to POSI.

Anyone can use the open metadata registered with Crossref, DataCite and ORCID as connections have been established between (ideally all) research objects and entities through open metadata and identifiers. This means that:

Funding agencies can monitor compliance with their policies
Publishers can identify the funder and meet their requirements
Funding agencies can assess and report on the reach and return of their funding programs
The provenance and integrity of the scholarly record is preserved and discoverable, benefitting all stakeholders.

Suggestions for meeting OSTP and Open Science guidance, point by point

OSTP Recommendation	Publishers should…	Funding agencies should…
4 a) Collect and make publicly available appropriate metadata associated with scholarly publications and data resulting from federally funded research	For scholarly publications: register comprehensive metadata & DOIs with Crossref. For scholarly data: register comprehensive metadata and DOIs with DataCite.	Use Crossref’s API to retrieve publication and other metadata. Use DataCite’s API to retrieve data/repository metadata.
i) all author and co-author names, affiliations, and sources of funding, referencing digital persistent identifiers, as appropriate;	Collect and validate the following from authors at manuscript submission: ROR & ORCiD IDs, Crossref Grant DOIs. Include data citations in reference lists, preferably with DataCite DOIs.	Register awards and grants with Crossref and create DOI records for them. Use ORCID’s API to retrieve validated contributor metadata. Update contributors’ ORCID records with Crossref Grant DOIs and metadata. Use ROR API to retrieve and verify affiliation metadata. Recommend data citations be included in published outputs.
ii) the date of publication; and,	Include acceptance and publication dates in Crossref metadata.	Use Crossref’s API to retrieve publication dates.
iii) a unique digital persistent identifier for the research output;	For scholarly publications and research outputs: register full metadata & DOIs with Crossref. For scholarly data: register full metadata and DOIs with DataCite.	Use Crossref and DataCite APIs to retrieve DOIs for research outputs.
4 b) Instruct federally funded researchers to obtain a digital persistent identifier that meets the common/core standards of a digital persistent identifier service defined in the NSPM-33 Implementation Guidance, include it in published research outputs when available, and provide federal agencies with the metadata associated with all published research outputs they produce, consistent with the law, privacy, and security considerations.	Collect ORCID iDs on manuscript submission for all authors. Register Crossref and DataCite DOIs and metadata for research outputs, including data.	Recommend that researchers applying for funding obtain an ORCID iD and collect them upon grant application for all applicants. Prepopulate grant applications with CV and publication information from applicants’ ORCID records. ORCID iDs should be included in the grants registered by the agencies with Crossref. Agencies can use our open APIs to retrieve the metadata on publications and data rather than ask researchers to do it, saving time and effort.
4 c) Assign unique digital persistent identifiers to all scientific research and development awards and intramural research protocols that have appropriate metadata linking the funding agency and their awardees through their digital persistent identifiers.		Join Crossref to register Crossref Grant DOIs, including ROR IDs and ORCID iDs Ensure grant proposal and assessment systems integrate with Crossref, ROR for affiliations and with ORCID for applicants/awardees.
5 a) coordinate between federal science agencies to enhance efficiency and reduce redundancy in public access plans and policies, including as it relates to digital repository access;	Work with agencies to ensure a smooth, automated workflow.	Using and supporting existing open scholarly infrastructure and using open identifiers will avoid duplication of effort and make the overall ecosystem more efficient .
5 b) improve awareness of federally funded research results by all potential users and communities;	Collect Crossref Grant DOIs from authors and use them to link from publications to grant information.	Communicate your Crossref Grant DOIs and open grant metadata widely via human and machine interfaces. Inclusion in the Crossref API will enhance dissemination and discoverability Update contributors’ ORCID records with Crossref Grant DOIs and metadata
5 c) consider measures to reduce inequities in the publishing of, and access to, federally funded research and data, especially among individuals from underserved backgrounds and those who are early in their careers;		Registering grants and sharing metadata through Crossref means it’s part of the world’s largest open community-governed metadata exchange and makes it available to the entire world without restriction.
5 d) develop procedures and practices to reduce the burden on federally funded researchers in complying with public access requirements;	Ensure your systems and those you work with make it as easy as possible for authors to provide the necessary metadata and persistent identifiers - work towards as much automation as possible and pulling from other systems rather than asking for data to be re-keyed.	Ensure the platforms you work with, such as grant proposal or assessment systems, retrieve and prepopulate ROR IDs, ORCID iDs, and Crossref and DataCite DOIs and associated metadata whenever possible so that the researchers don’t have to manually rekey or reformat data.
5 e) recommend standard consistent benchmarks and metrics to monitor and assess implementation and iterative improvement of public access policies over time;		Ensure that platforms and systems integrate with ROR, ORCID, Crossref, and DataCite so that this open metadata can lead to the creation of benchmarks and metrics.
5 f) improve monitoring and encourage compliance with public access policies and plans;	Use open infrastructure to help authors easily comply with public access and funder/institution policies. Automate systems as much as possible.	Using the open infrastructure, metadata, and identifiers outlined in this post will make monitoring more straightforward and compliance easier for all stakeholders. The community can build services on open infrastructure and metadata.
5 g) coordinate engagement with stakeholders, including but not limited to publishers, libraries, museums, professional societies, researchers, and other interested non-governmental parties on federal agency public access efforts;	Work with the global open infrastructure organisations (Crossref, DataCite and ORCID) whose members include funding agencies, societies, publishers, universities, libraries, repositories, museums, NGOs, and many other stakeholders - all looking to improve the efficiency of the research ecosystem.	Work with the global open infrastructure organisations (Crossref, DataCite and ORCID) whose members include funding agencies, societies, publishers, universities, libraries, repositories, museums, NGOs, and many other stakeholders - all looking to improve the efficiency of the research ecosystem.
5 h) develop guidance on desirable characteristics of—and best practices for sharing in—online digital publication repositories;	Support automated systems that use metadata and identifiers to populate repositories automatically.	Collaborate with publishers, Crossref and others to develop automated systems to populate repositories.
5 j) develop strategies to make federally funded publications, data, and other such research outputs and their metadata are findable, accessible, interoperable, and re-useable, to the American public and the scientific community in an equitable and secure manner.	Provide and support a range of discovery services based on open infrastructure.	Encourage discovery services - and develop services - that use the open infrastructure, metadata and persistent identifiers to enable.

Everybody needs to play their part

A lot of the work on making the above happen is already underway, and there is widespread adoption of open identifiers and metadata, but as noted above, funders are still early in the adoption journey, and implementation among all stakeholders is patchy.

Critical parts of the infrastructure rely on third-party platforms that supply tools and systems to authors, funders, and publishers - so coordinating the support for the appropriate metadata and identifiers in these systems and tools is very important.

We are emphasising how our existing open scholarly infrastructure systems are helping. But we also know that it’s not all perfect yet. Infrastructure is always evolving, metadata is never complete, refactoring workflows and systems can be costly, and integration can always be smoother. But our existing open infrastructure has already delivered significant benefits, and broader adoption will bring additional benefits to the whole scholarly research and communications ecosystem and help achieve the promise of Open Science in advancing human knowledge.

While working on this coordination and integration, we all try to remember that it should minimise work for researchers, and processes should be as automated as possible.

Collaboration is key to making this all work.

We already work with many funders through our Advisory Group, our 30 funder members, 25 of whom have so far collectively registered around 40,000 Crossref Grant DOIs, retrievable from our open API. Some grants are even matched to resulting outputs already, and some funders have recently dug into Crossref metadata to analyse outcomes from their investments, such as the Dutch Research Council (NWO) which presents findings and makes a case for greater emphasis on Crossref funding metadata.

We also work closely with partners Europe PMC and Altum, and we engage in community research and discussion, for example, through the Open Research Funders Group.

Alongside our fellow infrastructures and open identifier registries ORCID, DataCite, and ROR, we integrate with and support each other operationally and out in the community.

We will continue focusing our resources and efforts on engaging with funders, including US federal agencies responding by the OSTP guidelines, and all stakeholders to support the entire global scholarly research ecosystem.

Everyone has a part to play, and we must all pull together to prioritize this work.

Who’s in?

Please get in touch with Ed, Ginny, or Jennifer (or indeed DataCite or ORCID or ROR) if you’d like to have a discussion about the workflows described here, or just to make sure you’re up to date on the latest developments and opportunities we describe. We look forward to working with all funding agencies to support them as they develop their plans.

The Research Nexus: a vision for a more connected scholarly community

Jennifer Kemp — Tue, 01 Nov 2022 00:00:00 +0000

The Research Nexus: A vision for a more connected scholarly community

Published in Information Services and Use (2022), this paper sets out Crossref’s vision for a Research Nexus — a comprehensive, interconnected graph linking scholarly entities through persistent identifiers and open metadata. Based on a presentation at NISO Plus in February 2022.

Strategists

Understand why connections between scholarly entities matter as much as the entities themselves. How a richer graph of relationships — between works, people, organisations, and outputs — can transform discovery, assessment, and trust in the scholarly record.

Decision-makers

See how the Research Nexus vision maps to your organisation’s metadata strategy. Why investing in persistent identifiers and open metadata is foundational to a more connected, more trustworthy research ecosystem.

Practitioners

Learn how Crossref’s metadata infrastructure supports the Research Nexus. The role of DOIs, ORCIDs, ROR IDs, funder IDs, and relationships between records in building a comprehensive scholarly graph.

What this paper covers

The Research Nexus concept — a vision for a rich, machine-readable graph connecting all entities in the scholarly ecosystem: works, people, organisations, funders, datasets, and more
Why connections matter — how relationships between entities enable discovery, research assessment, integrity checking, and open science workflows that individual records cannot support alone
The role of persistent identifiers — DOIs, ORCIDs, ROR IDs, and funder IDs as the building blocks of a connected scholarly record
Crossref’s contribution — how Crossref metadata links works to people, institutions, funders, and related outputs, and how members can enrich those connections
Gaps and opportunities — where the scholarly graph is incomplete, and what it would take to fill those gaps through community collaboration
A call to action — how publishers, institutions, funders, and infrastructure providers can work together to realise the Research Nexus vision

Read the full paper

Loading…

Flies in your metadata (ointment)

Isaac Farley — Mon, 25 Jul 2022 00:00:00 +0000

Quality metadata is foundational to the research nexus and all Crossref services. When inaccuracies creep in, these create problems that get compounded down the line. No wonder that reports of metadata errors from authors, members, and other metadata users are some of the most common messages we receive into the technical support team (we encourage you to continue to report these metadata errors).

We make members’ metadata openly available via our APIs, which means people and machines can incorporate it into their research tools and services - thus, we all want it to be accurate. Manuscript tracking services, search services, bibliographic management software, library systems, author profiling tools, specialist subject databases, scholarly sharing networks - all of these (and more) incorporate scholarly metadata into their software and services. They use our APIs to help them get the most complete, up-to-date set of metadata from all of our publisher members. And of course, members themselves are able to use our free APIs too (and often do; our members account for the vast majority of overall metadata usage).

We know many organisations use Crossref metadata. We highlighted several different examples in our API case study blog series and user stories. Now, consider how errors could be (and often are) amplified throughout the whole research ecosystem.

While many inaccuracies in the metadata have clear consequences (e.g., if an author’s name is misspelled or their ORCID iD is registered with a typo, the ability to credit the author with their work can be compromised), there are others, like this example of typos in the publication date, that may seem subtle, but also have repercussions. When we receive reports of metadata quality inaccuracies, we review the claims and work to connect metadata users with our members to investigate and then correct those inaccuracies.

Thus, while Crossref does not update, edit, or correct publisher-provided metadata directly, we do work to enrich and improve the scholarly record, a goal we’re always striving for. Let’s look at a few common examples and how to avoid them.

Pagination faux pas

First page marked as 1

In the XML registered

<pages>
<first_page>1</first_page>
<last_page>1</last_page>
</pages>

https://api-crossref-org.turing.library.northwestern.edu/works?filter=type:journal-article&select=DOI,title,issue,page&sample=100

Other pagination errors

In the XML registered

<item_number item_number_type="article-number">1</item_number>

In the XML registered

<pages>
<first_page>121-123</first_page>
<last_page>129</last_page>
</pages>

Author naming lapses

Examples: Titles (Dr., Prof. etc.) in the given_name field; Suffixes (Jr., III, etc.) in the surname field; superscript number, asterisk, or dagger after author names (usually carried over from website formatting that references affiliations); full name in surname field

In the XML registered

<contributors>
<person_name sequence="first" contributor_role="author">
<given_name>DOCTOR KATHRYN</given_name>
<surname>RAILLY</surname>
</person_name>
<person_name sequence="additional" contributor_role="author">
<given_name>DOCTOR JOSIAH S.</given_name>
<surname>CARBERRY</surname>
</person_name>
</contributors>

<contributors>
<person_name contributor_role="author" sequence="first">
<surname>Mahmoud Rizk</surname>
</person_name>
<person_name contributor_role="author" sequence="additional">
<surname>Asta L Andersen(</surname>
</person_name>
</contributors>

organisations as authors slip-ups

Examples: The contributor role for person names is for persons, not organisational contributors, but we see this violated from time to time. Unfortunately, no persons are being credited with contributing to content that have these errors present in the metadata record.

In the XML registered

<contributors>
<person_name sequence="first" contributor_role="author">
<surname>Society</surname>
</person_name>
</contributors>

<person_name contributor_role="author" sequence="first">
<given_name>University of Melbourne</given_name>
<surname>University of Melbourne</surname>
</person_name>
</contributors>

Null no-nos

Examples: Too many times we see “N/A”, “null”, “none” in various fields (pages, authors, volume/issue numbers, titles, etc.). If you don’t have or know the metadata, it’s better to omit it for optional metadata elements than to include inaccuracies in the metadata record.

In the XML registered

<journal_volume>
<volume>null</volume>

<pages>
<first_page>null</first_page>
<last_page>null</last_page>
</pages>

<person_name sequence="first" contributor_role="author">
<given_name>Not Available</given_name>
<surname>Not Available</surname>
</person_name>
<person_name sequence="additional" contributor_role="author">
<given_name>Not Available</given_name>
<surname>Not Available</surname>
</person_name>

Where to go from here?

One thing we’ve said throughout this blog that we’ll reiterate here is: accurate metadata is important. It’s important in itself, and the metadata registered with us is heavily used by many systems and services, so think Crossref and beyond. In addition to that expanding perspective, there are practical steps members and metadata users can take to help us:

As a member registering metadata with us:

make sure we have a current metadata quality contact for your account and update us if there’s a change
if you receive an email request from us to investigate a potential metadata error, help us
if you do not know what to enter into a metadata element or helper tool field, please leave it blank; perhaps some of the examples of errors within this blog were placeholders that the responsible members intended to come back to - to correct in time; that’s also a practice to avoid
if you find a record in need of an update, update it - updates to existing records are always free (we do this to encourage updates and the resulting accurate, rich metadata, so take advantage of it).

As a metadata user:

if you spot a metadata record that doesn’t seem right, let us know with an email to support@crossref.org and/or report it to the member responsible for maintaining the metadata record (if you have a good contact there)
if you’re eager to confirm the last update of a metadata record, our REST API is a great resource; here’s a handy query to use as a starting point: this one returns records on our Crossref prefix 10.5555 that have been updated in 2022: https://api-crossref-org.turing.library.northwestern.edu/prefixes/10.5555/works?rows=500&filter=from-update-date:2022-01-01,until-pub-date:2022-12-31&mailto=support@crossref.org

Making connections between research objects is critical, and inaccurate metadata complicates that process. We’re continually working to better understand this, too. That’s why we’re currently researching the reach and effects of metadata. Our technical support team is always eager to assist in correcting errors. We’re also keen on avoiding those mistakes altogether, so if you are uncertain about a metadata element or have questions about anything included in this blog post, please do contact us at support@crossref.org. Or, better yet, post your question in the community forum so all members and users can benefit from the exchange. If you have a question, chances are others do as well.

How I think about ROR as infrastructure

Amanda French — Fri, 08 Jul 2022 00:00:00 +0000

The other day I was out and about and got into a conversation with someone who asked me about my doctoral work in English literature. I’ve had the same conversation many times: I tell someone (only if they ask!) that my dissertation was a history of the villanelle, and then they cheerfully admit that they don’t know what a villanelle is, and then I ask them if they’re familiar with Dylan Thomas’s poem “Do not go gentle into that good night.” So far, everyone has heard of it – it’s a very well-known poem indeed. I then explain that “Do not go gentle into that good night” is a villanelle, and that a villanelle is a poetic form something like a sonnet. So far, everyone also knows what a sonnet is, which is why I use that as a comparison, even though a villanelle isn’t all that much like a sonnet, in my opinion. They’re both poetic forms, however, with a particular standard number of lines and a particular standard rhyme scheme, so in that sense they certainly are alike.

Oddly enough, I think my early background in the study of poetic form is very much of a piece with my new role here at Crossref as Technical Community Manager for ROR, the Research Organization Registry. Both poetic form and metadata are invisible to most people, but both are valuable infrastructure. Both poetic form and metadata involve generally-accepted practices and standards that differ between different groups of people and change over time. Both writing formal poetry and creating rich metadata can seem burdensome and rigid to some people, but to my mind, both are generative. A solid underlying foundation allows for all kinds of creativity to flourish on the surface.

That might be part of why as soon as I heard about ROR I understood its tremendous potential. As someone who’s worked in digital humanities and scholarly communication for over fifteen years, I’ve long appreciated the value of clean, standard, comprehensive metadata in general. For instance, I explained the origin and value of the Dublin Core metadata standard to many a history scholar in the Omeka workshops I often taught at THATCamp. Later, while overseeing the institutional repository at Virginia Tech University Libraries, I learned even more about both the importance and the difficulty of creating, acquiring, and providing good metadata. When the pandemic began in 2020, I learned more than I ever wanted to know about messy data as Community Lead for The COVID Tracking Project at The Atlantic.

Data and metadata are, let’s admit it, very hard to keep clean and consistent as they travel through multiple systems, and that’s why it’s important to regularize as much as we can through automatic means such as APIs that use agreed-upon standards. Scholarship is a network of networks, and common identifiers like DOIs and ORCIDs enable the interchange of information in those networks about scholarly outputs and scholars, and thus they enable scholarship itself. What could be more important than that?

But the organisations that employ, fund, and publish scholarly researchers have had a hard time keeping track of everything “their” researchers have given to the world. That’s the problem that ROR, “a community-led registry of open, sustainable, usable, and unique identifiers for every research organisation in the world,” can help solve. In an ideal world, universities might use ROR IDs to track the research their faculty have produced, certainly, but they might also discover which universities their faculty’s co-authors most often come from. Funders might use ROR IDs to identify the research outputs that have benefited from their funds, certainly, but they might also analyze whether they are funding enough researchers from institutions in rural areas. Publishers might use ROR IDs to offer affiliation searching in their own public interfaces, certainly, but they might also create internal reports on compliance with institution-level transformative Open Access agreements. Once something like ROR is widely adopted, the vision of the Research Nexus becomes closer to reality: “A rich and reusable open network of relationships connecting research organisations, people, things, and actions; a scholarly record that the global community can build on forever, for the benefit of society.” ROR is all about the “organisations” part of that alluring vision.

If you’re curious about ROR and want to learn more (hey, that rhymes!), you might want to watch the highly informative presentation from September 2021 “Working with ROR as a Crossref Member”, in which you’ll learn several interesting things, including the following:

ROR itself is not an organisation, but an initiative supported jointly by Crossref, DataCite, and the California Digital Library;
Crossref members cited institutional affiliation identifiers as one of their top priorities in 2019, second only to abstracts;
The specifics of how one recent ROR integrator, the open access journal publisher Hindawi, used the ROR API to create a typeahead widget in its manuscript submission system that replaces user-supplied free text with a standard institution name and a ROR ID behind the scenes, helping them to generate useful internal reports about institutional payments; and
Crossref supports the submission of ROR IDs in its XML content registration process and makes ROR IDs available in its API.

I’m also enthusiastically inviting you to get in touch with me if you’d like to learn more about ROR or if you’d like to tell me about your previous experience with ROR. And if you don’t get in touch with me, please be aware that I might well reach out to you – I’m eager to hear what you hope for from ROR, but also what you’re skeptical about. For, after all, I learn by going where I have to go – don’t we all?

Seeing your place in the Research Nexus

Kornelia Korzec — Wed, 22 Jun 2022 00:00:00 +0000

Having joined the Crossref team merely a week previously, the mid-year community update on June 14th was a fantastic opportunity to learn about the Research Nexus vision. We explored its building blocks and practical implementation steps within our reach, and within our imagination of the future.

Read on (or watch the recording) for a whistlestop tour of everything – from what on Earth is Research Nexus, through to how it’s taking shape at Crossref, to how you are involved, and finally – to what concerns the community surrounding the vision and how we’re going to address that.

Summary of presentations

Click on image above to access the presentation.

The idea is simple in principle: scholarly records ought to be transparent – available to examine and learn from for all. Much of scientific production and communication these days has a heavy digital footprint so the Nexus is nothing but simply connecting the loose strands, right? Yet, as the scholarly record is a reflection of the continuous progress made by multiple actors within the context of scientific structures and processes, bringing the Nexus to life is a little short of simple.

“What we think of as metadata is expanding, and the notion of ‘record types’ is changing” – said Ginny Hendricks. A great majority of scholarly ‘objects’, whether they are data sets, research articles, monographs, or others, undergo many processes (including review, publication, licensing, correction, derivation) and influence knowledge and practice over time.

Making that progress visible and discoverable will allow for tracing the development of ideas and changes in our thinking over time. Transparency of the complete scholarly records will help to understand the impact of science funding and changing policies. It can support a more robust and comprehensive assessment of research, and contribute to improving integrity within as well as public trust in sciences.

The Research Nexus concept was first introduced by Jennifer Lin in 2017 as “Better research through better metadata”. Important adaptations to the model were needed to break it out of the content-specific schema. Ginny also pointed out that the concept is shared among the scholarly infrastructure community, citing a report from 2015 by OCLC Research on conscious coordination for stewardship of the evolving scholarly record.

Patricia Feeney has given us reasons for optimism in building a robust Nexus. She’s shown areas of greatest growth in metadata reported to Crossref and shared a public roadmap of types of information we’re asked to enable in the future. We’re seeing a true boom of datasets and peer review reports registrations, and the relationship metadata for our records is improving too. At the dawn of defaulting to open references, 44% of records we hold have associated references and that is growing. Provision of the newly enabled affiliation information (ROR IDs) is on the rise, as is the funder information. Some conversations and questions followed highlighting the need for further guidance in these areas.

To make a case for enriching metadata records, Martyn Rittman demonstrated examples of traceability of research influence on realities outside academia. He captured recent examples of data citations and other references present not just between scholarly papers, but also in policy documents and popular media. These allow for greater discoverability of literature – but also show the public influence and impact of the research and the work’s context in our wider society.

While Martyn shared our blue-skies aspiration to streamline Crossref’s APIs to offer insight to all these relationships with a single service, Joe Wass grounded those ambitions in the reality of technical work underway. His team’s attention is divided between three main areas. They continue to maintain and de-bug our existing infrastructure. They are developing self-service solutions for members. Finally, they are mapping and planning improved infrastructure, evaluating technology against the Research Nexus vision.

Bringing it back to the source (of metadata), Rachael Lammey offered a very practical guide to key activities enabling Research Nexus that all members can take on now. She highlighted the benefits of collecting and registering data citations, ROR IDs, and grant funding information. She went on to talk about challenges of subject classification (at a journal level) that our research and development efforts are focusing on at the moment.

Summary of discussions

Publishing has changed dramatically and our members recognise increasing opportunities for transparency of the scholarly record. Breaking the distant vision of Research Nexus down into actionable chunks made it more relatable for call participants. Many reflected on seeing their place in it properly for the first time. Yet, challenges remain and many were brought to the fore in the discussions.

The reliability and usability of the technology for registering metadata with Crossref needs to improve. We need to do better in supporting multi-language and multi-alphabet information. Not just developing systems anew, but also streamline the way content is registered and annotated, and continue to disambiguate the competing identifiers. Different record types, chiefly books, present specific challenges in this regard. Finally, making all that metadata accessible and usable is key to enabling insights from the rich data we collectively make available.

Technology is important, but won’t overcome the barriers that exist in the mindsets. Siloed thinking means that publishers may not be sensitive to benefits that improved relationship metadata could have for colleagues working on assessment, even within the same institutions. Greater guidance or best practices for new identifiers, such as ORCID, ROR, grants, would allow more publishers to get on board with the changes. Researchers often don’t help the cause either – many don’t realise the role and benefits of metadata for their work and are reluctant to provide rich information related to it, perceiving it as a bureaucratic burden.

In a nutshell, I learnt that – while the concept of Research Nexus is pretty complex – we’re all already participating in making it a reality. I’m grateful to the call participants for sharing their challenges and ideas so generously. It means we can work to address those. I’ll be sure to follow-up on requests for support and clearer guidelines about citing data, recording ROR IDs and grants information in the metadata, and we’ll engage our community on complex topics of record updates (corrections, retractions and versions). Be sure to keep in touch with the conversations on the Community Forum. I’ll see you there!

A ROR-some update to our API

Rachael Lammey — Wed, 19 Jan 2022 00:00:00 +0000

Earlier this year, Ginny posted an exciting update on Crossref’s progress with adopting ROR, the Research Organization Registry for affiliations, announcing that we’d started the collection of ROR identifiers in our metadata input schema. 🦁

The capacity to accept ROR IDs to help reliably identify institutions is really important but the real value comes from their open availability alongside the other metadata registered with us, such as for publications like journal articles, book chapters, preprints, and for other objects such as grants. So today’s news is that ROR IDs are now connected in Crossref metadata and openly available via our APIs. 🎉

This means ROR can be used by and within all the tools services that integrate with Crossref APIs to analyse, search, recommend, or evaluate research. It’s an important element of the Research Nexus, our vision of a fully connected open research ecosystem, and helps identify, share, and link the affiliations of those producing and publishing different types of research or receiving grants.

Now that this metadata is available, it helps confer the downstream benefits of ROR for different (and interconnected) groups:

It makes it easier for institutions to find and measure their research output by the articles their researchers have published, or perhaps make it easier to track the grants they’ve received.
Funders need to be able to discover and track the research and researchers they have supported.
Academic librarians need to easily find all of the publications associated with their campus.
Journals need to know where authors are affiliated so they can determine eligibility for institutionally sponsored publishing agreements.
Editors can use more accurate information on author and reviewer institutions during the peer review process, which can help avoid potential conflicts of interest.

Those are just a handful of use cases, which is why disseminating ROR affiliation identifiers via our APIs is so important; it lets others choose to do what they need to with the information, without restriction.

The story so far

A growing number of our members have started to include ROR in the metadata they register with us, so we’re excited to be able to see this via simple API queries.

At the time of writing we can see nearly 4,000 RORs being registered by these 21 members (we’ve removed test accounts). Note that many of these are being baked into metadata being registered for grant records, also recently released and now findable through the REST API:

"Wellcome": 2821,
"Natural Resources Canada/CMSS/Information Management": 277,
"University of Szeged": 139,
"RTI Press": 104,
"American Cancer Society": 103,
"University of Missouri Libraries": 77,
"Keldysh Institute of Applied Mathematics": 52,
"Boise State University, Albertsons Library": 52,
"Australian Research Data Commons (ARDC)": 52,
"The Neurofibromatosis Therapeutic Acceleration Program": 49,
"Boise State University": 12,
"The ALS Association": 11,
"Children's Tumor Foundation": 9,
"Episteme Health Inc": 3,
"The University of the Witwatersrand": 2,
"Office of Scientific and Technical Information": 2,
"AGH University of Science and Technology Press": 2,
"York University Libraries": 1,
"SZTEPress": 1,
"Masaryk University Press": 1,
"Institut für Germanistik der Universität Szeged": 1,

Our grants schema accommodated ROR first, so it’s the funder members and grant records that dominate the adoption of ROR… so far! But there are a few articles and reports there too already. These record types include ROR in their records:

"Grant": 3047,
"Report": 382,
"Dissertation": 164,
"Journal Article": 140,
"Conference Paper": 22,
"Posted Content": 12,
"Dataset": 7,
"Monograph": 6,
"Book": 3,
"Chapter": 2,
"Proceedings Series": 1,
"Peer Review": 1,
"Journal Issue": 1,
"Book Set": 1,
"Book Series": 1

We can currently see 205 different ROR IDs in Crossref metadata, with the most frequently provided ROR ID being: https://ror.org/02jx3x895, or University College London as it’s also known as.

If you’re a Crossref member keen to assert affiliation identification in your content, our recent webinar, Working with ROR as a Crossref member: what you need to know, covers all the detail.

Interested in using the information? Dig into our REST API documentation and into the API itself, use the polite pool if you can (i.e. identify yourself). There’s also a wealth of information on the ROR support site or being shared among integrators in the growing ROR community.

Join us in doing more with ROR!

Doing more with relationships - via Event Data

Martyn Rittman — Fri, 14 May 2021 00:00:00 +0000

Crossref aims to link research together, making related items more findable, increasing transparency, and showing how ideas spread and develop. There are a number of moving parts in this effort: some related to capturing and storing linking information, others to making it available.

By including relationship metadata in Event Data, we are taking a big step to improve the visibility of a large number of links between metadata. We know this is long-promised and we’re pleased that making this valuable metadata available supports a number of important initiatives. We will also be backfilling, so all previously deposited relationships will eventually become available as events. The first step will be to add relationships between items that have DOIs, such as between a research article and a related review report or dataset.

What are relationships?

When members register metadata with us, they have the possibility to identify other works, items, and websites that they know are related. This might be supplementary material or previous versions of a work (especially for preprints and working papers). Equally, identifiers for a protein, gene, or organism used in the research can be included. These are recorded as ‘relationships’ and can be accessed in the same way as the rest of the metadata we hold about registered content.

Some examples

Relationships in the metadata show links to the published article from this bioRxiv preprint. In the Crossref Rest API:

"relation": {
 "is-preprint-of": [
 {
 "id-type": "doi",
 "id": "10.1038/s41467-020-17892-0",
 "asserted-by": "subject"
 }
 ],
 "cites": []
},

And now in Event Data:

"subj": {
 "pid": "https://doi-org.turing.library.northwestern.edu/10.1101/2020.05.21.109546",
 "url": "https://doi-org.turing.library.northwestern.edu/10.1101/2020.05.21.109546",
 "work_type_id": "posted-content"
},
"obj": {
 "pid": "https://doi-org.turing.library.northwestern.edu/10.1038/s41467-020-17892-0",
 "url": "https://doi-org.turing.library.northwestern.edu/10.1038/s41467-020-17892-0",
 "method": "doi-literal",
 "verification": "literal",
 "work-type-id": "journal-article"
},

Linking to a dataset in the Dryad Digital Repository by a recent eLife article. In the Crossref metadata:

"relation": {
 "is-supplemented-by": [
 {
 "id-type": "doi",
 "id": "10.5061/dryad.s58qh",
 "asserted-by": "subject"
 }
 ],
 "references": [
 {
 "id-type": "doi",
 "id": "10.5061/dryad.s58qh",
 "asserted-by": "subject"
 }
 ],
 "cites": []
},

And now in Event Data:

"subj": {
 "pid": "https://doi-org.turing.library.northwestern.edu/10.7554/elife.19920",
 "url": "https://doi-org.turing.library.northwestern.edu/10.7554/elife.19920",
 "work_type_id": "journal-article"
},
"obj": {
 "pid": "https://doi-org.turing.library.northwestern.edu/10.5061/dryad.s58qh",
 "url": "https://doi-org.turing.library.northwestern.edu/10.5061/dryad.s58qh",
 "method": "doi-literal",
 "verification": "literal",
 "work-type-id": "Dataset"
},

If you are interested in relationships for a single DOI, we still recommend checking the metadata of that record, however Event Data is a great option for looking across multiple records. For example, to check for relationships across a prefix, in a given time period, or for a specific type of relationship.

Data citation

Data citations can be included in data deposits in relationship metadata, usually using the ‘is-supplemented-by’ relationship. By creating an event from each relationship, the links between journal articles and books, and the data they rely on are more visible. This makes the data much easier to locate.

Many datasets have DOIs which are usually recorded with DataCite, meaning you are unlikely to find them via searches of Crossref metadata. Making data citation relationship metadata available in Event Data means it will be available in the same format as citations from datasets to articles (which DataCite sends to Event Data) and citations from articles to datasets from Crossref reference metadata (more to come on this later this year). It also means we will convert this information into Scholix format so that it can be harvested and combined with other sets of Scholix-compliant article/data links. Data citations will therefore be available for the community to identify, share, link and recognise research data. We’re working with initiatives like Make Data Count and STM’s research data program to support the growing uptake of good data citation practices. This is a big step forward in making data citation happen for the community; we have more to do, but Crossref is committed to completing this work as a strategic priority.

What’s next?

In this first stage we are adding relationships that link two objects with a DOI, and later this year we will bring in relationships using other identifiers such as accession numbers and URIs. That will make it more straightforward to ask questions of Event Data such as which organisms have relationships to which works with a DOI.

More info and staying in touch

Find out more about Event Data in our support documentation or check out tickets in the GitLab repo.
Keep informed and ask us anything via our community forum for Event Data discussion

Fast, citable feedback: Peer reviews for preprints and other record types

Martyn Rittman — Wed, 09 Dec 2020 00:00:00 +0000

Crossref has supported depositing metadata for preprints since 2016 and peer reviews since 2018. Now we are putting the two together, in fact we will permit peer reviews to be registered for any record type.

Currently, peer reviews can be registered for journal articles, but that means that they can only be related to some of the content our members deposit. Preprints, books, chapters, working papers, dissertations, and a host of other works can also be registered with Crossref. A number of these frequently undergo some form of review and many of our members and voices in the community have called for us to widen the net on peer reviews, including journal publishers, book publishers, review platforms, and preprint servers. We’ve listened and taken action, and from now on Crossref members can add relationship metadata that links peer reviews to any record type. The metadata will also contain the type of review, stating whether it is a referee report, author response, or community comment, etc. This allows accurate reporting on whether the peer review is happening within a traditional editorial process or elsewhere.

Reviews for preprints

In the last decade there has been an increase in the number of disciplines using preprints. Since enabling registration of preprint metadata, it has become our fastest-growing record type. Preprints, working papers, and other forms of early publication help to accelerate dissemination of the latest research and discovery. They can also promote discussion on important topics, and help authors to improve papers before an editorial decision for journal publication. During the COVID-19 pandemic, preprints have become invaluable for speeding the publication of vital research and case studies.

On the other hand, preprints do not undergo formal review and editorial approval, leading to concerns about the dissemination of false information. While the issue of misinformation in preprints has been discussed for some time, the COVID-19 pandemic has brought it more sharply into focus. organisations that post preprints need to balance the benefits of rapid dissemination with promoting their responsible use.

To support the feedback process, preprint servers along with a growing number of other platforms and services offer scholars the opportunity to post public comments on preprints. By doing this, they give extra context for readers, provide suggestions for authors, and raise awareness of work that could be flawed or too preliminary.

Another growing trend is journal publishers adopting editorial processes that involve preprint-first options and open peer review. As Dr. Stephanie Dawson from ScienceOpen says:

“We have long believed in rewarding reviewers by assigning Crossref DOIs to their open reviews to make them citable objects and we were one of the first users of Crossref’s peer review schema. However, a large percentage of the articles reviewed on ScienceOpen are publicly available preprints. The UCL Open: Environment journal hosted on the platform, for example, is based on a workflow of open peer review of preprints. Our customers, editors, reviewers and authors are therefore extremely happy that these reviews can now also be assigned a Crossref peer review DOI for more accountability and transparency in scholarly publishing.”

At Crossref, we’re continually looking to support more record types and relations between them to build trust, support reproducibility and increase discoverability of content. This is another small step in building the research nexus and we look forward to working with members depositing peer reviews of preprints.

EASE Council Post: Rachael Lammey on the Research Nexus

Rachael Lammey — Mon, 12 Oct 2020 00:00:00 +0000

This blog was initially posted on the European Association of Science Editors (EASE) blog: “EASE Council Post: Rachael Lammey on the Research Nexus”. EASE President Duncan Nicholas accurately introduces it as a whole lot of information and insights about metadata and communication standards into one post…

I was given a wide brief to decide on the topic of my EASE blog, so I thought I’d write one that tries to encompass everything - I’ll explain what I mean by that.

In the past, Crossref has had the opportunity to talk to EASE members about the importance of registering content whose metadata contains important information related to the article. Richer metadata helps to connect the content to other key information such as who wrote it, who it was funded by, the relevant license, the research it cites, any updates to the work such as corrections and retractions, and the data that underpin the research. The use of open persistent identifiers like DOIs, funder IDs, ORCID iDs and ROR IDs are always recommended.

Such rich and connected metadata also helps discoverability of the published research in a different way than just direct access; if you can find something based on looking at the publications related to a particular funder, author, or institution, then there are more ways to come across what you’re looking for. Making links between objects underpinning the research also helps put the research in context and can help further research by making connections to other valuable information that may have been more difficult to make otherwise.

I’ve mentioned the Research Nexus in the title of this post. It’s achieved by declaring relationships between publications and other associated research objects, and from those objects to related publications. The metadata that reveals relationships between research objects can be as informative as the objects themselves. These relationships can assert certain facts that may not be otherwise obvious: this is our goal with the Research Nexus. These relationships and assertions need to exist not just on the web pages of the outputs, but also reflected in a standard way in the metadata so that the information is computer-readable and can be used at scale. As Jennifer Lin, who coined the term, explains:

“Researchers are adopting new tools that create consistency and shareability in their experimental methods. Increasingly, these are viewed as key components in driving reproducibility and replicability. They provide transparency in reporting key methodological and analytical information. They are also used for sharing the artefacts which make up a processing trail for the results: data, material, analytical code, and related software on which the conclusions of the paper rely. Where expert feedback was also shared, such reviews further enrich this record.”

In her Crossref blog, Jennifer goes on to give some examples, including:

Linking to an entire collection of methods and video protocols via Protocols.io
Linking to software and peer reviews in JOSS
Linking to preprint, data, code, source code, peer reviews in Gigascience

I’d include an additional example of linking research to the grant using the grant identifier and associated metadata from the funding section of this PLOS paper (read more about the example from EuroPMC who register grants with Crossref for Wellcome).

These links can be established by adding them into the Crossref relationship metadata schema. The information is then made available to anyone via our open APIs, so that they can easily see and use the information.

In all of these, publishers and other parties are linking to associated research outputs to support the reproducibility and discoverability of content.

The reproducibility point is worth reiterating; EASE has always supported projects to maintain high standards around the review of research, publication standards and ethics, and the reduction of research waste. And connecting articles to data, preprints, protocols, and peer reviews, and making the relationships open for analysis will help achieve this.

We also know that there are work and cost involved in establishing these links, and we’re working on ways to lower the barriers in doing so by:

Revisiting what we charge to encourage best practice. Starting in 2020, we have removed fees for registering vital information on corrections, retractions and other Crossmark metadata. This is timely in light of the updates to the EASE Standardised Retraction form.
We’re also working to remove fees for translations and versions that are linked together by the appropriate relationship metadata so that publishers posting translations or different versions of an article don’t have to pay multiple times for these. Our Membership & Fees Committee is currently reviewing other ways we can support publishers keen to make these connections.
Finding ways to make it easier for publishers to collect this information from authors e.g. submission systems integrations with data repositories to collect robust information on article/data links.
Allowing the registration of peer review metadata for content other than journal articles e.g. books, preprints (coming soon).
Making it easier for publishers to register this information with us at Crossref via the provision of simple to use tools, interfaces and reporting.

The outputs of the research process, such as journal articles, don’t exist in isolation - you only have to look at the interest in the corpus of COVID-19 publications, preprints and associated data to see this. This thinking is also supported by campaigns like Metadata 2020 advocating for “richer, connected, and reusable, open metadata will advance scholarly pursuits for the benefit of society.” The relationships revealed by the Research Nexus may one day help progress research to realise benefits that help us all, providing we all make efforts to effectively support them. More to come…

Why Data Citation matters to publishers and data repositories

Helena Cousijn — Thu, 08 Nov 2018 00:00:00 +0000

A couple of weeks ago we shared with you that data citation is here, and that you can start doing data citation today. But why would you want to? There are always so many priorities, why should this be at the top of the list?

I’m sure you heard this before—data sharing and data citation are important for scientific progress. The three key reasons for this are:

1) Transparency and reproducibility

Most scientific results that are shared today are just a summary of what researchers did and found. The underlying data are not available, making it difficult to verify and replicate results. If data would always be made available with publications, transparency of research would be greatly improved.

2) Reuse

The availability of raw data allows other researchers to reuse the data. Not just for replication purposes, but to answer new research questions.

3) Credit

When researchers cite the data they used, this forms the basis for a data credit system. Right now researchers are not really incentivized to share their data, because nobody is looking at data metrics and measuring their impact. Data citation is a first step towards changing that.

The benefits described above are all quite long-term, so why, as a publisher or data repository, should you put your resources towards implementing data citation workflows now? During our pre-conference workshop at FORCE2018 we asked repositories and publishers this question. Below you’ll find some of the answers.

Data repositories

For data repositories, data citation leads to increased visibility of both the repository and the datasets. The workshop revealed that many repositories do a lot of work to establish links between articles and datasets, thereby significantly contributing to transparency in research. Some of the repositories explained that they hire curators that text mine articles to find associations and manually curate datasets to ensure information about links is part of the metadata. This is reflected in Event Data, where 99% of links between articles and datasets comes from data repository metadata. This downstream enrichment of metadata is useful, but it would be more effective if all stakeholders strive to establish these links at a much earlier stage in the research communication process.

ICPSR, the Inter-university Consortium for Political and Social Research, shared:

ICPSR views data citation as vital. As a large social science data archive, ICPSR curates, preserves, and distributes data for the research community to re-use over time. Data citation makes data visible to the research community. Without it, data cannot be accessed for re-use or reproduced for transparency. Its use cannot be tracked and counted to reveal its impact and potential for new uses by investigators in new fields or in combination with new types of data. Data creators cannot receive adequate credit for their intellectual output. And the original investment by funders and scientists to create those data stops producing dividends. Therefore, data citation plays an essential role in the data sharing lifecycle.

Proper data citation, with a unique identifier, makes it much easier to measure impact. When data use is not cited or cited obliquely, it is rendered virtually invisible. Hence, much data use is still not easily detected. The ICPSR Bibliography of Data-related Literature represents ICPSR’s efforts to identify publications that analyze data distributed at ICPSR and link them directly to the data in the ICPSR catalog. As of 2018, ICPSR has a searchable database that contains nearly 80,000 citations of published and unpublished works resulting from analyses of data held in the archive. ICPSR also makes the case for data citation in its brief new video, “ICPSR 101: Why Should I Cite Data?”

GBIF, the Global Biodiversity Information Facility, explained:

The work required to collect, clean, compile and publish biodiversity datasets is significant and deserves recognition. Researchers publish studies based on data made available through GBIF.org at a rate of about 2 papers every single day. It is crucial for GBIF to link these scientific uses to the underlying data as one measure of demonstrating the value and impact of sharing free and open biodiversity data. At the moment, however, only about 10 percent of authors cite or acknowledge the datasets used in research papers properly. As a result, data publishers efforts often risk going unnoticed, and the true impact of sharing data remains invisible. GBIF will continue to work with publishers and researchers to provide guidance and input for how to best cite the use of GBIF-mediated data in scientific journals to ensure proper attribution and reproducible research and to demonstrate the true value of free and open access to biodiversity data.

Publishers

By ensuring data is cited in a consistent way, publishers help provide transparency and context for the content they publish. Depositing that information as part of the Crossref metadata helps that work go further by uncovering how data is being used across multiple publications and publishers This means patterns can be explored and researchers can gain more comprehensive recognition and credit for the work they have done.

Melissa Harrison, Head of Production Operations at eLife says:

eLife is committed to ensuring researchers get credit for all their outputs, and data is a major component of this. We’re working with Crossref and JATS4R to enable publishers to tag their JATS data content consistently and thus create an easy crosswalk to their Crossref deposits. The JATS4R guidance on Data Availability Statements, linked to and incorporating data citations, will be updated soon, please watch that space!

It will be really interesting to see how much re-use of previously published data is happening, look for patterns in re-use, and see links and hopefully building up of data by different research groups. Ultimately, this will incentivize researchers and publishers to ensure it is correctly accredited at source and in publications, improving the cycle further.’

Anita de Waard, VP of Research Collaborations at Elsevier, says:

One of the key recommendations of the Force11 Manifesto was to “3.3 Add data, software, and workflows into the publication as first-class research objects”, which will allow greater reproducibility and rigor to experimental research, and allow the reuse of all digital artefacts in the scholarly lifecycle. By following the data citation principles, we achieve two things: the author presents a richer representation of their work, and the data producer receives credit for the hard work of curating and publishing citable datasets.

Mendeley Data and Elsevier are active contributors to the Scholix framework that as a collaborative and open standard, allows the open mining of relationships between articles and datasets. We are also active participants in the new Enabling FAIR Data Project, and next to supporting the TOP Guidelines in all domains, require all authors in the earth and space sciences to deposit their data before publication.

Next week at Crossref LIVE18, Patricia Cruse from DataCite will talk about Data Citations and why they matter. If you’re in Toronto next week, do not hesitate to ask her or anyone from Crossref anything you want to know about data citation!

Data citation: let’s do this

Rachael Lammey — Thu, 04 Oct 2018 00:00:00 +0000

Data citation is seen as one of the most important ways to establish data as a first-class scientific output. At Crossref and DataCite we are seeing growth in journal articles and other record types citing data, and datasets making the link the other way. Our organisations are committed to working together to help realize the data citation community’s ambition, so we’re embarking on a dedicated effort to get things moving.

Efforts regarding data citation are not a new thing. One of the first large-scale initiatives to establish data citation as a standard academic practice was the FORCE11 Joint Declaration of Data Citation Principles (JDDCP) in 2014. This declaration was endorsed by over 100 organisations in the scholarly community as well as many individuals.

Following this agreement on how data citation should be done, many projects followed. Within FORCE11, the Data Citation Implementation Pilot brought together publishers and repositories to put data citation into practice and work on the implementation of the JDDCP. Within the context of the Research Data Alliance, a data-literature linking group started under the name of Scholix to establish a framework for exchanging information about the relationships between articles and datasets. The infrastructure building blocks now feed into projects such as Make Data Count and Enabling FAIR Data.

Projects aside, if datasets are cited consistently and in a standard way, it will make it much easier for the research community to see links between different research outputs and work with these outputs. It also makes it much easier to count these citations, so that researchers can get credit for their data and the sharing of that data.

The underlying work has been done to create an infrastructure that will effectively support and disseminate information on data citation. Data citation is here today!

Different organisations know how to handle data citations, and are starting to count these and make that information available in turn. This means that the only thing that’s needed is for people to actually cite data, and this information be captured and passed on. Some Crossref and DataCite members have already made great progress on this already (see Melissa Harrison’s blog on what eLife is doing).

The goals of all the data citation projects can only be realized if you start doing data citation, and we know you’ll have questions about it…

In the coming months, we’ll be posting several blogs and organizing sessions to tell you how you can start doing data citation - if you’re attending FORCE2018 you can catch our joint workshop there. So stay tuned and please get in touch if you can’t wait, we’d love to help you get started!

Leaving the house - where preprints go

Jennifer Lin — Tue, 21 Aug 2018 00:00:00 +0000

“Pre-prints” are sometimes neither Pre nor Print (c.f. https://doi-org.turing.library.northwestern.edu/10.12688/f1000research.11408.1, but they do go on and get published in journals. While researchers may have different motivations for posting a preprint, such as establishing a record of priority or seeking rapid feedback, the primary motivation appears to be timely sharing of results prior to journal publication.

So where in fact do preprints get published?

Although this is a simple question, we have not had an easy way to answer how this varies across disciplines, preprint repositories and journals. Until now. Crossref metadata provides not only an open and easy way to do so, but up-to-date data to get the latest results.

rOpenSci makin’ it sweet & easy

Crossref asks preprint repositories to update their metadata once a preprint has been published by adding the article link into its record via the “is-preprint-of” relation. As the record is processed, we make the link available going both directions, while preserving the provenance of the statement in the metadata output (“asserted-by”: “subject” or “asserted-by”: “object”). This results in bidirectional assertions in the Crossref REST API where search engines, analytics providers, indexes, etc. can get from the preprint to the article (“is-preprint-of”) as well as vice versa (“has-preprint”), making it easier to find, cite, link, assess, and reuse.

Using rOpenSci’s R library for the Crossref REST API (rcrossref), we pulled all articles connected to a previous preprint (https://api-crossref-org.turing.library.northwestern.edu/works?filter=relation.type:has-preprint&facet=publisher-name:&rows=0) and then aggregated them based on journal via their ISSNs (https://api-crossref-org.turing.library.northwestern.edu/works?filter=relation.type:has-preprint&facet=issn:), tallying the results in a tidy table with the journal name (ex: PLOS Biology (https://api-crossref-org.turing.library.northwestern.edu/journals/2167-8359)).

The big reveal

So without further delay, let’s look at the results of the 20 journals with the highest number of preprints associated with its articles (data from August 21, 2018):

Publisher	Journal	Count
PeerJ	PeerJ	1184
Springer Nature	Scientific Reports	394
eLife	eLife	375
PLOS	PLOS ONE	338
Proceedings of the National Academy of Sciences	PNAS	205
PLOS	PLOS Computational Biology	196
Springer Nature	Nature Communications	187
PLOS	PLOS Genetics	169
The Genetics Society of America	Genetics	168
Oxford University Press	Nucleic Acids Research	148
Oxford University Press	Bioinformatics	138
The Genetics Society of America	Genetics	120
The Genetics Society of America	G3: Genes, Genomes, Genetics	104
Cold Spring Harbor Laboratory	Genome Research	104
Oxford University Press	Molecular Biology and Evolution	100
MDPI AG	Energies	98
MDPI AG	Sensors	96
Springer Nature	BMC Genomics	92
MDPI AG	International Journal of Molecular Sciences	86
JMIR Publications	Journal of Medical Internet Research	83

This list has not been normalized or weighted based on the size of the journal. The following observations are informed speculations, as we can only infer so much from the raw data:

Disciplinary practice: This phenomenon where preprints are a part of disciplinary practice accounts for about half of the journals represented on the list. Certain communities such as genetics and computational fields have been early adopters of preprints. As such, we see higher rates of preprint-to-article publication in journals that publish their work.
Partnerships: Partnerships that facilitate submission from the preprint repository directly to a publisher or peer review service (ex: BioRxiv B2J program) make it easier for researchers to move from preprint-sharing seamlessly to submitting their journal article manuscript.
Tie-ins: A quarter of the journals on the list are run by publishers with a preprint service, and have been able to tie together both arms of publishing. This removes barriers to journal article submission in the same manner as integrations between repositories and publishers, but does so as a single party.
Publisher support and treatment: We also see that strong proponents and early partners of preprint repositories tend to have higher counts. Some publishers have been more outspoken in their welcome of preprints, such as PNAS. Sometimes this support also comes in the form of special treatment. In the process of crafting editorial policy on publishing results previously posted in a preprint, some journals have carved out particular affordances in their publication workflow and content delivery streams that may contribute to the higher counts of articles. For example, Nature Research displays the preprints of submitted articles under consideration: https://nature-research-under-consideration-nature-com.turing.library.northwestern.edu/.
Mega-journals: Mega-journals such as Scientific Reports and PLOS ONE have not discouraged preprints. As such, and due to the size of their publication output, they have easily found a place among the higher counts on the list.

Taking a closer look

One major consideration in these results, concerns what’s missing in the data. These fall into two camps: incomplete member data, and incomplete membership coverage.

We have been working with our members to deposit preprints using the proper record type, and to provide links to published articles in their metadata. However, not all have yet done so (ex: SSRN), leading to holes in our research nexus graph, which subsequently detracts from the completeness of the data.

We celebrate the preprint repositories who are required to update their metadata when an article is published from a preprint, thereby populating the map with critical bridges between preprints and articles. Crossref participation benefits not only the content owner, but the membership at large and all the systems across the research ecosystem powered by Crossref metadata.

Lastly, this data is dependent on the coverage of preprint repositories who register content with us. We are thrilled that Center for Open Science, our newest preprints addition who represents 21 community repositories, has recently filled in swaths of the map. But there remain dead zones in the research graph from repositories who are not Crossref members (ex: ArXiv). Their disciplines, as a result, are under represented in these results.

Everyone dive in!

As to the question of “where do preprints get published?”, anyone in fact can answer this question based on the metadata Crossref collects and provides to the community as an open infrastructure provider. We encourage the community to explore and analyze the data further with other available datasets to glean more insights on how scholarly communications is changing with the increasing growth of preprints. For example, the effective results across all journals represented can be weighted based on the number of articles published by each journal.

Crossref data is open for all to examine and reuse through our REST API. Please dive in and share your findings with us!

Peer review publications

Jennifer Lin — Sun, 12 Aug 2018 00:00:00 +0000

Peer review publications—not peer-reviewed publications, but peer reviews as publications

Our newest dedicated record type—peer review—has received a warm welcome from our members since rollout last November. We are pleased to formally integrate them into the scholarly record, giving the scholars who participated credit for their work, ensuring readers and systems dependably get from the reviews to the article (and vice versa), and making sure that links to these works persist over time.

Many of our members make the peer review history available to researchers in different ways. Their extra effort to post review materials alongside the article will now go further once they are registered with us and linked to the journal article. They spoke of publishing peer reviews as a standard part of their publishing operation. The scholarly contributions of their editors and referees are validated, stewarded, and published in the manner of the articles: as per general practice. To fully realize this, they are ensuring that these publications are discoverable, citable, and part of the formal scholarly record—for all the thousands of systems which draw on Crossref metadata.

Article metadata + peer review metadata = a fuller picture of the evolution of knowledge

The growing collection

As of August 12, 2018 three publishers have registered 12446 peer reviews in the dedicated resource type schema we rolled out last November. PeerJ (10.7287) with 12015 at time of writing and Stichting SciPost (10.21468) with 297 works. ScienceOpen (10.14293) has registered 126 reviews of papers on their post-publication platform.

The peer review metadata collected is partly similar, though otherwise unique to other content. In the former, general metadata that we accept for the articles, as well as the reviews, include an ORCID iD to identify the reviewer, editor, and/or author 0; license 0. This metadata is quite distinct from the article and is important to collect, not only as a discrete publication in its own right, but also to provide richer context for the actual results shared in the associated article. They are authored by different people than the paper’s contributors (author response/rebuttal excepting). They need not have the same license.

Currently, none of this data has been registered. (From the publishers we’ve talked to, this is largely due to factors related to limitations in their technology systems.) And like other record types, we link up scholarly materials in the metadata and fill in the research nexus graph through relations.

There’s no better way to understand peer review metadata than to look at real examples from our members:

PeerJ review (https://doi-org.turing.library.northwestern.edu/10.7287/peerj.2707v0.1/reviews/1) and its metadata (https://api-crossref-org.turing.library.northwestern.edu/works/10.7287/peerj.2707v0.1/reviews/1)
ScienceOpen review (https://doi-org.turing.library.northwestern.edu/10.14293/s2199-1006.1.sor-uncat.a5995373.v1.rhrmgu) and its metadata (https://api-crossref-org.turing.library.northwestern.edu/works/10.14293/s2199-1006.1.sor-uncat.a5995373.v1.rhrmgu)
SciPost review (https://doi-org.turing.library.northwestern.edu/10.21468/scipost.report.10) and its metadata (https://api-crossref-org.turing.library.northwestern.edu/works/10.21468/scipost.report.10)

Review-specific metadata is also critical to capturing the shape of the scholarly discussion. These include:

Review date (required)
Scholarly work reviewed (required)
Recommendation
Revision stage
Review round
Contributor name

PeerJ, SciPost, and ScienceOpen have registered this whole set where applicable (review round not applicable to post-publication reviews), with the exception of the recommendation.

Scholarly contributions captured in time

Published peer reviews uniquely highlight the nature of research ideas evolving over time, spotlighting the nature of this as a collective effort involving multiple individuals. The more metadata, the bolder the story. We have created a set of reference metadata (fictitious) to illustrate this phenomenon. Josiah Carberry submits a manuscript to the Journal of Psychoceramics, entitled “Dog: A Methodology for the Development of Simulated Annealing.” It undergoes two rounds of review with two referees each round. The article https://doi-org.turing.library.northwestern.edu/10.5555/12345681 is published and registered on May 6, 2012 along with the history of peer review materials on the same day:

First submission

Referee report 1 - https://doi-org.turing.library.northwestern.edu/10.5555/12345681.9879
Referee report 2 - https://doi-org.turing.library.northwestern.edu/10.5555/12345681.9880
Editor decision - https://doi-org.turing.library.northwestern.edu/10.5555/12345681.9881

Revision round 1

Author rebuttal - https://doi-org.turing.library.northwestern.edu/10.5555/12345681.9882
Referee report 1 - https://doi-org.turing.library.northwestern.edu/10.5555/12345681.9883
Referee report 2 - https://doi-org.turing.library.northwestern.edu/10.5555/12345681.9884
Editor decision - https://doi-org.turing.library.northwestern.edu/10.5555/12345681.9885

Published reviews can show peer feedback in progress; the progress of scholarly discussion unfolding, as expert ideas build upon each other. Many of us have traditionally located the article’s publication as the climactic event, but the story in fact doesn’t end there. Pre-publication becomes post-publication. Throughout this time, research is validated and sprouts into new ideas.

Peer review platform Publons is working on getting reviews authored on its platform registered with us. Doing so will mean that PeerJ article, “Transformative optimisation of agricultural land use to meet future food demands” by Lian Pin Koh, Thomas Koellner, and Jaboury Ghazoul https://doi-org.turing.library.northwestern.edu/10.7717/peerj.188 with three scholarly discussions published over the course of peer review, would also be accompanied by a fourth that occurred after publication from Gene A. Bunin https://publons.com/publon/3374/, not yet registered.

Research begets research

In my investigation of review publications registered, two examples cropped up, highlighting the richness of the research process not only as it shows a set of research results evolve through scholarly discussion, but as it is then folded into new research outputs.

A PeerJ article “Software citation principles” https://doi-org.turing.library.northwestern.edu/10.7717/peerj-cs.86 has had a very rich life: https://api-crossref-org.turing.library.northwestern.edu/works/10.7717/peerj-cs.86. It was originally submitted as a preprint and underwent multiple iterations of improvement (https://doi-org.turing.library.northwestern.edu/10.7287/peerj.preprints.2169, https://doi-org.turing.library.northwestern.edu/10.7287/peerj.preprints.2169v1, https://doi-org.turing.library.northwestern.edu/10.7287/peerj.preprints.2169v2, etc.). It then was subjected to peer review. And three referee reports are published alongside the final publication:

We glimpse a view of time unfolding here:

NB: in the review metadata, all the dates provided reference September 19, 2016 when they were published with the accompanying research article. To really make the metadata useful, we recommend providing the date the review was received, rather than published (for publishers who are publishing pre-publication review materials).

The reviews were then cited in three versions of the F1000Research article, “A multi-disciplinary perspective on emergent and future innovations in peer review” (https://doi-org.turing.library.northwestern.edu/10.12688/f1000research.12037.1, https://doi-org.turing.library.northwestern.edu/10.12688/f1000research.12037.2, and https://doi-org.turing.library.northwestern.edu/10.12688/f1000research.12037.3). These three all link up on the Crossref metadata map. The visualization below is only an entrypoint into this picture of research dissemination and the spread of ideas.

András Láng served as a reviewer for a paper by Danilo Garcia and Fernando R. González Moraga published as “The Dark Cube: dark character profiles and OCEAN” (https://doi-org.turing.library.northwestern.edu/10.7717/peerj.3845). As of the blog release date, this paper has been cited by two sources:

Source: https://doi-org.turing.library.northwestern.edu/10.7717/peerj.3845, CC-BY 4.0

What this view of the paper does not reveal is that Láng’s review (https://doi-org.turing.library.northwestern.edu/10.7287/peerj.3845v0.1/reviews/2) provided such insight to the original researchers that the first author (Garcia) incorporates the discussion in his subquent work. This evidence is documented in the citation list of that new publication, “Encyclopedia of Personality and Individual Differences” https://doi-org.turing.library.northwestern.edu/10.1007/978-3-319-28099-8_2302-1. What a wonderful illustration of the ways in which peer reviews can operate like other publications, and how far is it from being unique. But up to now, we have not yet programmatically captured them in a formal way as we do now with these materials registered properly as a review.

The evolution of Crossref’s piece

In the same spirit of ever evolving knowledge, we also continue to update our schemas based upon community feedback. Are references important? Tell us! What new metadata on peer reviews are important to answer your questions or help you do what you need? Members, if you are interested in registering your peer review content with us, please get in touch.

How good is your metadata?

Kirsty Meddings — Thu, 26 Apr 2018 00:00:00 +0000

Exciting news! We are getting very close to the beta release of a new tool to publicly show metadata coverage. As members register their content with us they also add additional information which gives context for other members and for services that help e.g. discovery or analytics.

Richer metadata makes content useful. Participation reports will give—for the first time—a clear picture for anyone to see the metadata Crossref has. This is data that’s long been available via our Public REST API, now visualized.

Who are participation reports for? Everyone!

It’s an opportunity to evaluate and educate. See for yourself where the gaps are, and what our members could improve upon. Understand best practice through seeing what others are doing, and learn how to level-up.

Monitor what metadata is being registered, even if this work is done by a third party or another department. And see what other organisations in scholarly communications see when they use Crossref metadata in their research, tools, and services.

The beta release—expected after acceptance testing some time late May—will let anyone look up any of our 15,000+ members and see whether they are registering ten key elements that add context and richness to the basic required bibliographic metadata.

What do we mean by ‘richer metadata’?

The ten checks for Beta, will be:

References
~~Open references~~ [EDIT 6th June 2022 - all references are now open by default].
ORCID iDs
Funder IDs
Funding award numbers
Crossmark metadata
License information
Full text links
Similarity Check URLs
Abstracts

Each of these additional metadata elements helps increase discovery and wider and more varied use—and usefulness—of research outputs.

Why are we doing this and what do we mean by ‘participation’?

Over the years when we’ve talked with our members about their metadata, we learned that many just can’t be certain exactly how they’re performing. It could be that they’ve outsourced Content Registration to another service provider or larger publisher, or it could be they just weren’t previously aware they could collect and share authors’ ORCID iDs, Funder IDs, and so on. So our primary aim is to give our members the information they need in order to make a case for improving their metadata records. Each check will come with information about why it is important and guidance on how to improve. Additionally, with the growing use of Crossref as a central source of metadata for the research community, it’s in everyone’s interest to be as transparent as possible about what metadata we have - and encourage greater understanding of what’s possible.

Member ‘participation’ is an important concept. Crossref distinguishes itself from other DOI registration agencies by providing this richer infrastructure which allows for things like funding information, license information, links between data and preprints, and so on—all contributing to the research nexus for everyone’s benefit.

Membership of Crossref is not just about getting a persistent identifier for your content, it’s about placing your content in context by providing as much metadata as possible and looking after it long-term.

Here’s a sneak preview of what the report will look like:

So whether you’re a member who wants to run a “health check” on your own metadata, or a consumer of metadata interested in what’s available and from whom, watch this space for Participation Reports!

Would you like a heads-up on your report, pre-beta?

Beta will be released some time in May or June this year, following acceptance testing with members and others. Then we’re looking for about 20 members to have a half-hour phone call with a walk-through ‘health check’. Please contact Anna if you’d like to schedule one.

Metadata and integrity: the unlikely bedfellows of scholarly research

Damian Pattinson — Thu, 14 Dec 2017 00:00:00 +0000

I was invited recently to present parliamentary evidence to the House of Commons Science and Technology Select Committee on the subject of Research Integrity. For those not familiar with the arcane workings of the British Parliamentary system, a Select Committee is essentially the place where governments, and government bodies, are held to account. So it was refreshing to be invited to a hearing that wasn’t about Brexit.

The interest of the British Parliament in the integrity of scientific research confirms just how far science’s ongoing “reproducibility crisis” has reached. The fact that a large proportion of the published literature cannot be reproduced is clearly problematic, and this call to action from MPs is very welcome. And why would the government not be interested? At stake is the process of how new knowledge is created, and how reliable that purported knowledge is.

The other issue driving this overview of research practices are the cases of deliberate fraud and wrongdoing that have recently created headlines (e.g., the STAP papers concerning the reprogramming of stem cells). While these cases are clearly dramatic outliers, they nevertheless serve to diminish public confidence in scholarly research and the findings that come out of this enterprise.

As with most inquiries, the question quickly boiled down to: who is to blame? As Bill Grant MP asked me directly, “Where does the responsibility lie?”

My answer was lifted from an article by Ginny Barbour and colleagues in F1000Research this November (https://doi-org.turing.library.northwestern.edu/10.12688/f1000research.13060.1): publishers are responsible for the integrity of the published literature, while institutions and employers are ultimately responsible for the conduct of their staff. Misconduct entails intent, usually to deceive the reader into believing a conclusion that the researcher wishes them to believe. But journal editors can never know, and are not in a position to investigate, whether a researcher has deliberately falsified their data.

However, there are things that publishers can do to ensure high standards of integrity. Much of this involves making a study’s authors publish as much information about what they have done as possible - the more the reader can see of how data were generated, the more that reader can trust the findings communicated in the published article.

Article metadata directly supports this function. It provides structure and transparency to information pertaining to ethics and integrity. And because metadata is independent of the main article, it can be readable even if the article itself is locked behind a paywall.

Crossref already provides metadata that can demonstrate the integrity of published articles. The metadata collected on 91+ million scholarly works across publishers and disciplines is open and freely accessible to all. Bibliographic information, for example, allows readers to see who the authors of the article are, where they are from, and what else they have published. Similarly, funding data allows readers to identify potential conflicts of interest, for example if the funder has commercial or political affiliations. Even if the reader cannot see the conflict of interest statement (or if the journal has not provided one), they can use the funding statement to surface potential conflicts.

And if they wanted, publishers could provide additional metadata to add still more transparency to the research process. Ethical approval by institutional review boards, for example, could be captured, and any protocol numbers traced back to the original ethics committee approval. At present the process of ethical approval varies from country to country, and from institution to institution. Encouraging authors and journals to deposit information on the approval process would both demonstrate the high ethical standards the author is working to, and also improve the standards themselves, since institutions would have to encode their approval processes in a way that is understandable to others. This could pave the way to significantly higher international ethical standards, all through a simple addition to the indexed metadata underlying the scholarly literature.

One key recommendation that I and many others made to the Committee was, in short, “show your work”. As a researcher, that means showing your data. As a publisher, that means showing what checks you have done. In both cases, metadata can help.

A major issue that publishers and researchers can – and should – address is the provision of actual scientific data. Most papers, today, present only the end results of the authors’ (often quite extensive) analyses. The case for sharing data is an obvious one - many recent cases of misconduct could have been identified earlier, or even avoided altogether, if editors and readers had had access to underlying datasets.

With images, a requirement to submit raw images alongside the edited figures would dramatically reduce the cases of manipulation that are rife in the literature (studies suggest up to 20% of papers have some kind of inappropriate figure manipulation, with around 1 in 40 papers showing manipulation beyond that which can be expected to be a result of error). Similarly, providing the numbers that a paper’s analyses are based upon would allow readers to fully assess if datasets are distributed as would be expected through random sampling, and, if they choose, to determine if the data are sufficient to support the statistical inferences made in the paper. The Crossref schema – by providing unique identifiers to data citations - makes this link between data and paper possible. (See the recent blog post on the Research Nexus for more information.)

For publishers, showing your work also means being transparent to your readers about the editorial checks that a manuscript has undergone. Crossref has a tool that enables this editorial transparency: it’s called Crossmark. Crossmark allows readers to see the most up-to-date information about an article, even on downloaded PDFs. In most cases it is used to show whether the version of an article is most recent one, or whether any corrigenda or retractions have been subsequently added. But it can also be used to provide whatever information a publisher wishes to share about the paper. Some journals have experimented with using Crossmark to ‘thread’ publications together, for example, by linking all the outputs generated from a single clinical trial registration number (blog post here). But publishers could go further and display metadata pertaining to the editorial checks they have performed on a paper. So Crossmark could tell readers that the paper has been checked for plagiarism, or figure manipulation, or reporting standards such as CONSORT or ARRIVE guidelines. Here at Research Square we have been addressing this with a series of Badges that researchers can apply to their papers to demonstrate what checks have been performed.

Together, these implementations would provide value to the reader, who can see exactly what has been checked, and to the publisher, who can show how rigorous their editorial processes are. It would also serve to highlight the integrity of the authors who have passed all of these checks.

Research integrity is not something that can be easily measured but, unlike wit or charm, it is something that people generally know that they have.* This means that they just need to be transparent in their output to demonstrate this to the world. Metadata provides a simple way of doing this, so researchers and publishers should make sure they provide it as openly as they can.

*with apologies to Laurie Lee for the mangled quote

The research nexus - better research through better metadata

Jennifer Lin — Tue, 14 Nov 2017 00:00:00 +0000

Researchers are adopting new tools that create consistency and shareability in their experimental methods. Increasingly, these are viewed as key components in driving reproducibility and replicability. They provide transparency in reporting key methodological and analytical information. They are also used for sharing the artifacts which make up a processing trail for the results: data, material, analytical code, and related software on which the conclusions of the paper rely. Where expert feedback was also shared, such reviews further enrich this record. We capture these ideas and build on the notion of the “article nexus” blogpost with a new variation: “the research nexus.”

Some of Crossref’s publishing community are encouraging the scholarly communication practices surrounding these tools in a variety of ways: incorporating them into the publishing workflow, integrations between the tools and publishing systems, as well as linking and exposing the artifacts in the publications for readers to access. A special set of publishers have gone all the way and included these links into their Crossref metadata record. They insert them directly into the metadata deposit when they register the content (technical documentation). Doing so, these connections reach further than the publisher platform and propagate to systems across the research ecosystem including places like indexers, research information management systems, sharing platforms (oh, the list goes on!). We highlight a small set of examples to illustrate how these outstanding publishing practices are supporting good research.

1. Linking to an entire collection of methods

Crossref member, Protocols.io, is supporting transparency and methods reproducibility with their open access repository of science methods. Leitão-Goncalves R, Carvalho-Santos Z, Francisco AP, et al. investigated the concerted action of the commensal bacteria Acetobacter pomorum and Lactobacilli in Drosophila melanogaster, demonstrating how the interaction of specific nutrients within the microbiome can shape behavioral decisions and life history traits. Findings were published in PLOS Biology earlier this year: https://doi-org.turing.library.northwestern.edu/10.1371/journal.pbio.2000862. Authors deposited detailed methods and protocols used in the project (Drosophila rearing, media preparations, and microbial manipulations) as a collection in Protocols.io: https://doi-org.turing.library.northwestern.edu/10.17504/protocols.io.hdtb26n. So Protocols.io registered their content with us, linking the protocol to the paper. This creates the crosswalk between both so that users can get from one to the other through the metadata. The full metadata record can be found here.

2. Linking to video protocol

If a picture is worth a thousand words, the truism might apply to moving pictures many times over. Fasel B, Spörri J, Schütz P, et al. proposed a set of calibration movements optimized for alpine skiing and validated the 3D joint angles of the knee, hip, and trunk during alpine skiing in a PLOS ONE paper: https://doi-org.turing.library.northwestern.edu/10.1371/journal.pone.0181446. These movements consisted of squats, trunk rotations, hip ad/abductions, and upright standing. The specific team responsible for designing them (Fasel B, Spörri J, Kröll J, and Aminian K) described the set of calibration movements performed but found videos to be a far more effective way to communicate the technical movements used in their study. They made the visuals available too: https://doi-org.turing.library.northwestern.edu/10.17504/protocols.io.itrcem6. So Protocols.io deposited the link between video protocol and paper to the Crossref metadata record (full metadata record).

3. Linking to software and peer reviews

The Journal of Open Source Software (JOSS) is an academic journal about high quality research software across broadly diverse disciplines. Sara Mahar works on the effectiveness of organisations funded by the US Department of Housing and Urban Development to combat homelessness. She collaborated with computational physicist Matthew Bellis to create a python tool for researchers to visualize and analyze data from the Homeless Management Information System:https://doi-org.turing.library.northwestern.edu/10.21105/joss.00384. The software was archived in Zenodo: https://doi-org.turing.library.northwestern.edu/10.5281/zenodo.13750 and the peer review artifacts were also published. JOSS deposited all these links in the metadata record (found here).

4. Linking to preprint, data, code, source code, peer reviews

Gigascience, published by Oxford University Press, is experimenting with a number of new tools in their mission to promote reproducibility of analyses and data dissemination, organisation, understanding, and use. In a recent paper Luo R, Schatz M, and Salzberg S shared the results of the firstly publicly available implementation of variant calling using a 16-genotype probabilistic model for germline variant detection: https://doi-org.turing.library.northwestern.edu/10.1093/gigascience/gix045. Prior to formal peer review, the group posted the preprint in bioRxiv: https://doi-org.turing.library.northwestern.edu/10.1101/111393. When the paper was published, the authors made the supporting data available, including snapshots of the test and result data, in a public repository: https://doi-org.turing.library.northwestern.edu/10.5524/100316. OUP included this data citation in their Crossref metadata record via the routes recommended in our previous blog post about depositing data citations. The researchers made the code available in Github, and the algorithm is ready for researchers to run on Code Ocean, a cloud-based computational reproducibility platform that allows researchers to wrap and encapsulate the data, code, and computation environment linked to an article: https://doi-org.turing.library.northwestern.edu/10.24433/CO.0a812d9b-0ff3-4eb7-825f-76d3cd049a43. For further transparency, expert reviews of the manuscript from the peer review history were published in Publons: https://doi-org.turing.library.northwestern.edu/10.5524/review.100737 and https://doi-org.turing.library.northwestern.edu/10.5524/review.100738. (As of last month, publishers can register peer reviews at Crossref). The full metadata record contains links to the entire set of materials listed above.

5. Linking to preprint, Code, Docker hub, video, reviews

Narechania A, Baker R, DeSalle R, et al. used bird flocking behavior to design an algorithm, Clusterflock, for optimizing distance-based clusters in orthologous gene families that share an evolutionary history. Their paper was published in Gigascience last year: https://doi-org.turing.library.northwestern.edu/10.1186/s13742-016-0152-3. Supporting data, code snapshots and video were published in GigaDB: https://doi-org.turing.library.northwestern.edu/10.5524/100247. Code was maintained in GitHub. And authors also created a Docker application for Clusterflock, a lightweight, stand-alone, executable package of the software which includes everything needed to run it: code, runtime, system tools, system libraries, settings (Docker Hub link here). They created a video demo of the algorithm. Publons reviews were published https://doi-org.turing.library.northwestern.edu/10.5524/review.100507 and https://doi-org.turing.library.northwestern.edu/10.5524/review.100508. Gigascience shared all these assets in their publication, including the link to the original bioRxiv preprint: https://www.biorxiv.org/content/early/2016/03/25/045773). The full metadata record containing these links can be found here.

The Research Nexus: better research through better metadata

These five are just a few exemplary cases showing how publishers are declaring the relationships between their publications and other associated artifacts to support reproducibility and discoverability of their content. We welcome you to check out our overview of relationships between DOIs and other materials for more information. Members who are enriching your publishing pipeline in similar ways, please register these links to make your reach go further. We also welcome everyone to retrieve these relations in our REST API (technical documentation).

Data citations and the eLife story so far

Melissa Harrison — Thu, 18 May 2017 00:00:00 +0000

When we set up the eLife journal in 2012, we knew datasets were an important component of research content and decided to give them prominence in a section entitled ‘Major datasets’ (see images below). Within this section, major previously published and generated datasets are listed. We also strongly encourage data citations in the reference list.

Major Datasets for “Structural basis of protein translocation by the Vps4-Vta1 AAA ATPase” by N. Monroe, H. Han, P. Shen, et. al.

Almost five years on and I feel we have still not cracked it! We have signed up to the Force11 data citation principles, which were published three years back; we have been actively involved in working groups of Force11 and others, for example the Data Citation Roadmap for Scientific Publishers and the JATS XML data citation recommendation of JATS4R. I am also currently working with other publishers to come up with recommended JATS XML tagging for data availability statements, which is easier said than done considering the nuances of dataset uses and also how different publishers approach this.

Added to this, there is still significant push-back from authors about putting all dataset citations in the reference list (for example, authors are concerned about self-citing by citing a dataset created as part of the research article; “dataset citations” that are in effect a link to a search results page on a database; and the necessitation of hundreds of reference entries if an author has used a large base for the research).

While eLife is very active in this space, and aims to arrange and mark up the datasets and citations produced by our authors in line with recommendations, the recommendations still have some gaps and the complete picture is not yet clear.

In late 2014, we brought in-house the process of depositing Crossref metadata (previously our online host did this for us). It gave us control of our processes and, at the time, we sent all the information we could to Crossref and have ensured our references are open and available in the Crossref public API. The code for this conversion process is all open-source and available for reuse. It can be found on GitHub. Since then, besides small improvements to the code and troubleshooting problems, we’ve not updated the code. I have been keeping a list of Crossref features and new deposit metadata we can add to our deposits, and now is the time for us to start working on this again.

One of the items we’ll be addressing is data citations.

The Crossref reference schema does not cater well for non-book or -journal content, and if an item does not have a DOI, the “reference” is not very useful because of the few tags available in the Crossref schema.

However, Crossref have introduced the relationship type to their schema, so data references can be well linked and mineable. As I see Crossref as a potential broker between publishers and data repositories in the future, using the relationship-type deposit for all datasets will assist this and also allow these data points to more easily be seen within the article Nexus framework (see the recent blog post, How do you deposit data citations?).

At eLife, we already distinguish between Dataset generated as part of research results (relationship type in the Crossref schema: “isSupplementedBy”) and Dataset produced by a different set of researchers or previously published (relationship type: “references”). Therefore, it will not be hard for us to convert all the information about data referencing that is within the dataset section into a relationship-type deposit in the conversion to Crossref XML.

We have also recently gone through an exercise of defining a set of rules for all our references and, of the 12 allowed types, one is data. The rules for Schematron (a rule-based validation language for making assertions about the presence or absence of patterns in XML trees; see also this useful article about Schematron on the JATS4R learning centre) have been written for the eLife ‘business’ rules. Subject to final testing, these will be integrated into our workflow (the Schematron is open source and available for reuse on GitHub, and we will also build an API for people to use the Schematron direct). This will allow us to easily identify all data references and convert them into relationship types in the XML delivered to Crossref. This way, they will not be lost in the references section of our deposits, but properly identified.

However, we do appreciate this will become harder for us as authors become more familiar with datasets as references, because we will not be able to identify the difference between generated and analysed datasets so easily.

The code developed and used to complete these conversions will, again, be on Github and open source, and we actively encourage the reuse of this.

While the industry is still working on the best way to deal with data and ensuring it is given the prominence it requires, we feel this is the best approach we can take. Nothing is forever and we can still change what we do in the future. The beauty of open-source code also means that if there is an alternative approach now or in the future, the code we wrote at eLife can be developed by someone else in the future and we can all benefit.

If you have any questions, please do not hesitate to contact us.

How do you deposit data citations?

Jennifer Lin — Thu, 02 Mar 2017 00:00:00 +0000

Please visit Crossref’s official Data & Software Citations Deposit Guide for deposit details.

Very carefully, one at a time? However you wish.

Last year, we introduced linking publication metadata to associated data and software when registering publisher content with Crossref Linking Publications to Data and Software. This blog post follows the “whats” and “whys” with the all-important “how(s)” for depositing data and software citations. We have made the process simple and fairly straightforward: publishers deposit data & software links by adding them directly into the standard metadata deposit via relation type and/or references. This is part of the **existing Content Registration ** process and requires no new workflows.

Relationships

Data & software citations are a valuable part of the “research article nexus”, comprised of the publication linked to a variety of associated research objects, including data and software, supporting information, protocols, videos, published peer reviews, a preprint, conference papers, etc. For all of these resources, we use relation types in the metadata deposit to “anchor” the article in the article nexus and link to it.

For data & software, we ask for:

identifier of the dataset/software
identifier type: “DOI”, “Accession”, “PURL”, “ARK”, “URI”, “Other” *
relationship type: “isSupplementedBy” or “references”
description of dataset or software.
*Additional identifier types beyond those used for data or software are also accepted, including ARXIV, ECLI, Handle, ISSN, ISBN, PMID, PMCID, and UUID.

Crossref maintains an expansive set of relationship types to support the various resources linked in the research article nexus. For data and software, we recommend “isSupplementedBy” and “references” as relationship types in the metadata. Use the former if it was generated de novo as part of the research results. For those generated by another project and then reused, we recommend applying “references” in the relationship type. These were selected in consultation with DataCite and data working groups. They will provide the level of specificity requested by the community.

To illustrate how to represent the link within the metadata deposit, we offer two examples from two popular dataset identifiers, one for each of the relationship types.

Dataset	Snippet of deposit XML containing link
Dataset with DOI: Data from: Extreme genetic structure in a social bird species despite high dispersal capacity. Database: Dryad Digital Repository DOI: https://doi-org.turing.library.northwestern.edu/10.5061/dryad.684v0	`<program xmlns="http://www.crossref.org.turing.library.northwestern.edu/relations.xsd">` `<related_item>` `<description>Data from: Extreme genetic structure in a social bird species despite high dispersal capacity</description>` `<inter_work_relation relationship-type="isSupplementedBy" identifier-type="doi">10.5061/dryad.684v0</inter_work_relation>` `</related_item>` `</program>`
Dataset with accession number: NKX2-5 mutations causative for congenital heart disease retain functionality and are directed to hundreds of targets Database: Gene Expression Omnibus (GEO) Accession number: GSE44902 URL: https://www-ncbi-nlm-nih-gov.turing.library.northwestern.edu/geo/query/acc.cgi?acc=GSE44902	`<program xmlns="http://www.crossref.org.turing.library.northwestern.edu/relations.xsd">` `<related_item>` `<description>NKX2-5 mutations causative for congenital heart disease retain and are directed to hundreds of targets</description>` `<inter_work_relation relationship-type="references" identifier-type="Accession">GSE44902</inter_work_relation>` `</related_item>` `</program>`

In the examples above, the Dryad dataset was generated as part of the research published in an article. Hence, it contains the “isSupplementedBy” relationship type. The GEO dataset was reused by and referenced in a scholarly article published separate from the project that generated this dataset. Hence, it contains the “references” relationship type.

Both Crossref and DataCite employ this method of linking. Data repositories who register their content with DataCite follow the same process and apply the same metadata tags. This means that we achieve direct data interoperability with links in the reverse direction (data and software repositories to journal articles).

References

Another mechanism for depositing data and software citations is to insert it into the manuscript’s references. Publishers then deposit it as part of the article’s references. To do so, publishers follow the general process for depositing references. (Visit Crossref’s Support page for step-by-step instructions.)

Publishers can deposit the full data or software citation as a unstructured reference.
<citation key="ref=3">
<unstructured_citation>Morinha F, Dávila JA, Estela B, Cabral JA, Frías Ó, González JL, Travassos P, Carvalho D, Milá B, Blanco G (2017) Data from: Extreme genetic structure in a social bird species despite high dispersal capacity. Dryad Digital Repository. https://doi-org.turing.library.northwestern.edu/10.5061/dryad.684v0</unstructured_citation\>
</citation>
</citation_list>

Or they can employ any number of reference tags currently accepted by Crossref. Most do not readily suit datasets and software as the suite was originally established to match article and book references. This leaves out substantial metadata needed to identify and describe the dataset, however, if the resource does not have a DOI.
<citation key="ref2">
<doi>10.5061/dryad.684v0</doi>
<cYear>2017</cYear>
<author>Morinha F, Dávila JA, Estela B, Cabral JA, Frías Ó, González JL, Travassos P, Carvalho D, Milá B, Blanco G</author>
</citation>
We are exploring the JATS4R recommendations while we consider expanding the current collection. We welcome additional suggestions from the community.

Precise, accessible links

Crossref’s infrastructure is setup to facilitate the flow of information about scholarly works across the research network. We maintain a fair degree of flexibility both in the structure and completeness of metadata deposited. The aim, though, is to make the links rich in metadata, accurate in associating literature to corresponding resource, and available to both human and machine consumers as per Principle #5 and #7 in the Joint Declaration of Data Citation Principles.

As with the other associated resources in the article nexus, we recommend depositing data/software links in the publication metadata via relationships. Publishers are free to do this on top of or independent of references. Relationship metadata offer a high degree of precision. References are a hodgepodge of various resources cited by the publication, including articles, books, media, blogs, reference materials, etc. and data citations are hard to isolate. Furthermore, the unstructured, “spaghetti string” text is difficult for systems to parse and extract specific information.

With relationship metadata, data and software resources are expressly designated. We obtain a more accurate link that specifies identifier type and explicitly identifies data generated as part of the research shared in the paper or as reuse of existing data). The richer metadata contained here enables consumers to conduct powerful queries based on different attributes (identifier type, description, relationship), taking data discovery and mining to the next level.

Furthermore, relationships are important for achieving full accessibility of data and software citations. Access to references is based on publisher permission so not all data citations can be shared (excluding DataCite DOIs). In contrast, all links deposited via relationships are publicly available.

Publishers play an important role in supporting research validation and reproducibility. Data & software citation is a basic part of of this practice, and instrumental in enabling the reuse and verification of these research outputs, tracking their impact, and creating a scholarly structure that recognizes and rewards those involved in producing them. For the full scoop of how to deposit (i.e., technical details and more), we encourage you to reference the Crossref Data & Software Citations Deposit Guide and contact us (support@crossref.org) with questions or feedback.

Linking Publications to Data and Software

Jennifer Lin — Wed, 07 Sep 2016 00:00:00 +0000

TL;DR

Crossref and Datacite provide a service to link publications and data. The easiest way for Crossref members to participate in this is to cite data using DataCite DOIs and to include them in the references within the metadata deposit. These data citations are automatically detected. Alternatively and/or additionally, Crossref members can deposit data citations (regardless of identifier) as a relation type in the metadata. Data & software citations from both methods are freely propagated. This blog post also describes how to retrieve the links collected between publication and data & software.

Data & software citation is good research practice (DataCite-STM Joint Statement and FORCE11 Joint Declaration of Data Citation Principles) and is part of the scholarly ecosystem supporting research validation and reproducibility. Data & software citation is also instrumental in enabling the reuse and verification of these research outputs, tracking their impact, and creating a scholarly structure that recognises and rewards those involved in producing them.

Crossref supports the propagation of data & software citations alongside a publisher’s standard bibliographic metadata. members deposit the data citation link as part of the overall publication metadata when registering their content. Crossref partners with DataCite and together, we jointly provide a clearinghouse for the citations collected. These are all made freely available to the community as open data.

Citation practices are evolving across different communities of practice. Crossref’s offering is flexible and easily accommodates variations and changes, since it does not rely on a specific set of citation metadata elements, citation format, nor manner of credit and attribution. Publishers deposit data & software citations in their metadata deposit via a) references and/or b) relation type.

Method A: Bibliographic references

Crossref and DataCite have partnered to provide automatic linking between publications registered with Crossref and datasets bearing DataCite DOIs. This is the most efficient and effective way to ensure that data citations are fully integrated into the scholarly research information network with full and accurate metadata.

All data & software citations that include datasets bearing a DataCite DOI are eligible for auto-update linking with Crossref. In this method: authors cite the dataset or software containing the DataCite DOI per journal article submission guidelines and add it to the article citation list (c.f. FORCE11 citation placement, FORCE11 Software Citation Principles). Publishers then deposit references as part of their standard practice when registering content. Crossref checks every reference deposited for a DOI. If the DOI is identified as DataCite’s, we automatically link it to the article. With this method, no additional action is needed when publishers register their content with Crossref.

Data citation links to non-DataCite DOIs can only be exposed in the references if the publisher makes references openly available. Even in the event that the data citation is shared, it remains undifferentiated from other references. Method B described below offers another approach.

Method B: Relation type

Publishers can link their publication to a variety of associated research objects as part of the article metadata directly in the metadata deposited to Crossref, including data & software, protocols, videos, published peer reviews, preprints, conference papers, etc. Doing so not only groups digital objects together, but formally associates them with the publication. Each link is a relationship and the sum of all these relationships constitutes a ‘research article nexus.’ Data & software citations are a valuable part of this.

To tag the citation in the metadata deposit, we ask for:

description of dataset or software (optional)

dataset or software identifier

identifier type

relationship type.

Crossref can accommodate research outputs with any identifier, though we currently only validate DOI relationships during metadata processing. Technical details are documented in the [Data & Software Citations Deposit Guide][4].

Combining methods increases total available citations

The two methods are independent and can be used exclusively or jointly. Each caters to a different set of conditions and their practical considerations. See the comparison of benefits and limitations for each method in the deposit guide. We recommend that publishers use both methods where possible at this time for optimum specificity and coverage.

How to access data & software citations

Crossref and DataCite make the data & software citations deposited by Crossref members and DataCite data repositories openly available to a wide host of parties, including both Crossref and DataCite communities as well as the extended research ecosystem (funders, research organisations, technology and service providers, research data frameworks such as Scholix, etc.).

Data & software citations from references can be accessed via the Crossref Event Data API Citations included directly into the metadata by relation type can be accessed via Crossref’s APIs in a number of formats (REST, OAI-PMH, OpenURL). (A single channel containing data & software citations across interfaces is in development and will be released next year.)

Publishers, visit our detailed guide on how to deposit data and software citations. We welcome your questions and concerns at feedback@crossref.org.

Special thanks to the following who provided valuable feedback in developing the guide: Martin Fenner (DataCite), Amye Kenall (Springer Nature), Brooks Hanson (AGU), Shelley Stall (AGU), and the FORCE11 Data Citation Implementation Pilot publisher’s subgroup.

The article nexus: linking publications to associated research outputs

Jennifer Lin — Thu, 25 Aug 2016 00:00:00 +0000

Crossref began its service by linking publications to other publications via references. Today, this extends to relationships with associated entities. People (authors, reviewers, editors, other collaborators), funders, and research affiliations are important players in this story. Other metadata also figure prominently in it as well: references, licenses and access indicators, publication history (updates, revisions, corrections, retractions, publication dates), clinical trial and study information, etc. The list goes on.

What is lesser known (and utilized) is that Crossref is increasingly linking publications to associated scholarly artifacts. At the bottom of it all, these links can help researchers better understand, reproduce, and build off of the results in the paper. But associated research objects can enormously bolster the research enterprise in many ways (e.g., discovery, reporting, evaluation, etc.).

With all the relationships declared across all 80+ million Crossref metadata records, Crossref creates a global metadata graph across subject areas and disciplines that can be used by all.

Research article nexus

As research increasingly goes digital, more research artifacts associated with the formal publication are stored or shared online. We see a plethora of materials closely connected to publications, including: versions, peer reviews, datasets generated or analysed in the research, software packages used in the analysis, protocols and related materials, preprints, conference posters, language translations, comments, etc. Occasionally, these resources are linked from the publication. But very rarely are these relationships made available beyond the publisher platform.

Crossref will make these relationships available to the broader research ecosystem. When publishers register content for a publication, they can identify the associated scholarly artifacts directly in the article metadata. Doing so not only groups digital objects together, but formally associates with the publication. Each link is a relationship and the sum of all these relationships constitutes a “research article nexus.”

An assortment of connections already abound in the wild today. Examples include:

F1000Research article https://doi-org.turing.library.northwestern.edu/10.12688/f1000research.2-198.v3 connected to initial version https://doi-org.turing.library.northwestern.edu/10.12688/f1000research.2-198.v1

OECD publication https://doi-org.turing.library.northwestern.edu/10.1787/empl_outlook-2014-en and its German translation https://doi-org.turing.library.northwestern.edu/10.1787/empl_outlook-2014-de

PeerJ article https://doi-org.turing.library.northwestern.edu/10.7717/peerj.1135 and its peer review https://doi-org.turing.library.northwestern.edu/10.7287/peerj.1135v0.1/reviews/3

eLife article https://doi-org.turing.library.northwestern.edu/10.7554/eLife.09771 and its BioArXiv preprint https://doi-org.turing.library.northwestern.edu/10.1101/018317

PLOS ONE article https://doi-org.turing.library.northwestern.edu/10.1371/journal.pone.0161541 with underlying data in Dryad https://doi-org.turing.library.northwestern.edu/10.5061/dryad.d2vf8

Frontiers article https://doi-org.turing.library.northwestern.edu/10.3389/fevo.2015.00015 with a figshare https://doi-org.turing.library.northwestern.edu/10.6084/m9.figshare.1305089.v1 video

Journal of Chemical Theory and Computation article https://doi-org.turing.library.northwestern.edu/10.1021/ct400399x with software archived in Zenodo https://doi-org.turing.library.northwestern.edu/10.5281/zenodo.60678

Nature Biotech article https://doi-org.turing.library.northwestern.edu/10.1038/nbt.3481 with a Protocols.io protocol https://doi-org.turing.library.northwestern.edu/10.17504/protocols.io.dm649d

To date, almost all these relationships are not directly recorded in the article metadata (great job, PeerJ!). And as a result, they are more than likely “invisible” to the broader scholarly research ecosystem. Publishers can remedy these gaps by depositing associations when registering content with Crossref or updating the records after registration. That is how the article nexus is formed.

(Associated datasets can also be identified in the reference list as per Joint Declaration of Data Citation Principles as with the FORCE11 Software Citation Principles. Stay tuned next week for a follow up blog post on Crossref’s support for publisher data and software citations through its metadata.)

Forming the nexus

The mechanism of declaring these relationships is straightforward and a longstanding part of the standard deposit process. For each associated research object, simply provide the identifier and identifier type for the object, an optional description of it, as well as name the relationship into the metadata record. For the latter, Crossref and DataCite share a closed list of relationship types, which ensures interoperability between mappings. See Crossref technical documentation for more details.

We maintain a list of the recommended relation types for a host of associated research objects to promote standardization across publishers. If you have relationships not specified, please contact us at feedback@crossref.org to identify a suitable one considered best practice. Common adoption of relation types will make relationship metadata useful to tool builders and systems. For example, programmatic queries on supporting materials require proper tagging of their respective relationship types.

This approach is highly extensible and accommodates the introduction of new research object forms as they emerge. It also supports associated research objects regardless of identifier type. When an associated entity has a DOI, however, we can validate the relationship during metadata processing as well as provide a more reliable representation of the article nexus.

Article nexus: a far richer scholarly map

Bibliographic metadata is like a ship’s manifest that catalogs each item of cargo in a ship’s hold - crate, drum, sack, and barrel. It identifies the components that have an internal relation to the publication (contributor, funder, article update, license, etc.), each of which are well-understood points on the scholarly map. But when we integrate the article nexus into the graph, new territories become visible - not isolated islands, but places with highways connecting them to addresses already known.

When a publication has its relationships clearly identified, the connections both go out as well as lead back to it. The more connections, the more visibility on the scholarly map, as the Art of Cartography goes. Numerous systems tap into this map: publishing, funders, research institutions, research councils, indexers & repositories, indexers, research information systems, lab & diagnostics systems, reference management and literature discovery, other PID suppliers. So publishers, you can provide the fullest value to your own publishing operation, your authors, their research communities, and the overall research enterprise by ensuring that all publications are fully linked both inside and out.

Crossref to Auto-Update ORCID Records

Ginny Hendricks — Thu, 24 Sep 2015 00:00:00 +0000

In the next few weeks, authors with an ORCID iD will be able to have Crossref automatically push information about their published work to their ORCID record. It’s something that ORCID users have been asking for and we’re pleased to be the first to develop the integration. 230 publishers already include ORCID iDs in their metadata deposits with us, and currently there are 248,000 DOIs that include ORCID iDs.

What this means for researchers

More visibility for your work! Crossref represents over 5000 scholarly publishers and many of them ask authors for their ORCID iD and include it in the publication information they send us. Also it will mean less manual searching and adding; you’ve always been able to search crossref metadata for your name and/or publications and manually add them to your ORCID record, this auto-update simply means that when your publishers include the info we can update and add work(s) to your ORCID record automatically for you. You can still choose to hide/show whatever works you choose, and, of course, you’ll have the opportunity to authorize or switch off the integration completely (though future publications may trigger a new request). Overall, you’ll benefit from a more complete and up-to-date ORCID record to showcase your work.

What this means for publishers

If you’re one of the 230 Crossref publishers who already supply ORCID iDs along with the usual metadata submissions, then you’re all good. If you don’t offer this yet, you might want to think about starting - it’s beneficial for funders, publishers, other researchers, libraries, and universities to be able to integrate with complete researcher records. You can ask for ORCIDs upon manuscript submission or acceptance and tag it in your metadata deposits with Crossref. We’ll ensure the rest.

Various caveats and important details to be aware of

Apparently not all publishers are members of Crossref (we know, crazy), and in addition only a subset of Crossref publishers (230 in total) are asking authors for ORCID iDs and/or including them in their metadata deposits.
Some publishers may choose to opt out of passing through the details to ORCID using the Crossref auto-update (perhaps they plan to send this directly at some point) but if you’ve included your ORCID with your submission and it isn’t automatically updated, then check with your publisher.
We have a “backlog” of almost 250,000 DOIs that include ORCID iDs so that may mean we do some bulk updates at a later date where authors will receive an email with a long list of works to add. Even if the works have been listed before, it’s worth accepting as it will add the most up-to-date metadata to ensure the most accurate record.

Any questions can be directed to our support team.