Preprints on Crossref

Innovation in scientific publishing and its implications for Crossref DOI registration practices - MetaROR’s approach

Ludo Waltman — Tue, 03 Feb 2026 00:00:00 +0000

A couple of months ago, Ludo Waltman and André Brasil raised some questions about good practices for Crossref DOI registration, asking for input from the scholarly communication community. In this post, Ludo and André reflect on the input received and discuss the approach to DOI registration that the MetaROR publish-review-curate platform is going to take.

Practices for assigning DOIs and structuring the associated metadata are not merely technical details. They shape how scholarly outputs are discovered, cited, evaluated, indexed, and preserved over time. As new models of publishing emerge, especially those that decouple dissemination from evaluation, these infrastructural choices increasingly influence what counts as a scholarly object, as well as how credit and accountability mechanisms are organized.

As editors of MetaROR (MetaResearch Open Review), a platform launched in 2024 and operating under the publish-review-curate model, we are interested in good practices for Crossref DOI registration in the context of innovative new approaches to scientific publishing. In the earlier blog post, we invited members of the broader scholarly communication community to share their perspective on the following two questions:

For each article on the MetaROR platform, there is a corresponding article on a preprint server. Is it acceptable to have two Crossref DOIs, one registered by the preprint server and one registered by the MetaROR platform, for essentially the same article?
If Crossref DOIs are registered for articles on the MetaROR platform, should the articles be assigned the type ‘journal-article’ or the type ‘preprint’ in their Crossref metadata, or something else entirely?

We were pleasantly surprised by the level of interest in these two questions. We received about 15 responses from colleagues in the scholarly communication community. Some colleagues posted a reply at the bottom of our blog post. Others responded on social media (Bluesky, LinkedIn) or shared their perspective by email.

Below we reflect on the responses received and we outline the approach to Crossref DOI registration that MetaROR is going to take.

DOI registration for articles on the MetaROR platform

Colleagues offered mixed opinions on the question of whether articles on the MetaROR platform should have their own DOI, in addition to the DOI these articles have on the preprint server on which they were originally published. Some colleagues argued there is no good reason for registering DOIs for articles on the MetaROR platform and suggested this may cause confusion. One colleague reasoned that “if we want peer review to be something more ongoing and evolve beyond a single point in time judgment”, our approach should be to “better map the connections between events” rather than registering a new DOI each time an article has been peer-reviewed.

However, other colleagues expressed support for registering DOIs for articles on the MetaROR platform. One colleague pointed out that this “allows the user to reference the exact artefact they have consulted”. This colleague also reminded us that in the past “people were worried about having a different DOI for a preprint and another for a VoR (version of record)”, while nowadays this is a generally accepted practice. Another colleague emphasized the value of decentralization and suggested to “let a thousand DOIs bloom”. Authors of an article peer-reviewed by MetaROR argued in favor of “an overarching DOI for the full package (preprint, reviews, author response and link to updated preprint)”, which in their view would make MetaROR’s “process more coherent”.

Having considered the various arguments in favor of or against registering DOIs for articles on the MetaROR platform, we feel the arguments in favor are more compelling. Our perspective is that an article on the MetaROR platform differs in a meaningful way from the corresponding article on a preprint server, since the article on the MetaROR platform has been enriched with an evaluation by peer reviewers and editors. MetaROR provides a carefully curated package that includes not only the article itself, but also review reports and an editorial assessment. In our view, this justifies registering DOIs for articles on the MetaROR platform. We also see DOI registration for articles on the MetaROR platform as a way to promote appropriate recognition for authors of articles peer-reviewed by MetaROR, similar to the way authors get recognition for articles published in traditional peer-reviewed journals.

Of course, when an article has multiple versions, each with their own DOI, it is important to establish a link between the different DOIs, indicating that the DOIs are associated with the same work. This is important for articles published first on a preprint server and then on a platform such as MetaROR just like it is important for articles published first on a preprint server and then in a peer-reviewed journal. In practice, we establish these links by registering relationships between DOIs in the associated metadata. In this way, we ensure that indexing services, discovery systems, and research analytics tools are able to recognize that the DOIs refer to different manifestations of the same work rather than independent outputs.

Record type for articles on the MetaROR platform

Our second question is about the record type to be used when registering a Crossref DOI for an article on the MetaROR platform. Many colleagues who provided input on this question argued there is a need for a new Crossref record type for ‘reviewed preprints’.

We feel the idea of such a new record type is interesting and its pros and cons deserve further consideration. However, any solution that requires changes in Crossref’s metadata schema will take time to realize, while for MetaROR we need a solution in the short term. At the moment, the most obvious options for MetaROR therefore seem to be to use either the record type ‘journal-article’ or the record type ‘preprint’ (which is in fact a subtype of the record type ‘posted-content’).

The use of the record type ‘preprint’ seems somewhat problematic to us, because preprints are typically understood to be articles that have not yet been formally peer-reviewed. In a way, articles on the MetaROR platform are the opposite of this, since these articles have undergone formal peer review. An article on the MetaROR platform is part of a package that also includes review reports and an editorial assessment. Such a package provides readers with a more informed understanding of an article than what they get from reading only the article itself. For this reason, we do not consider the record type ‘preprint’ to be suitable for articles on the MetaROR platform.

Instead of the record type ‘preprint’, we have decided to use the record type ‘journal-article’ for articles on the MetaROR platform. The record type ‘journal-article’ is intended for articles published in journals. To be clear, MetaROR considers itself a ‘platform’, not a ‘journal’. However, the distinction between ‘platforms’ and ‘journals’ is not very well defined and the choice of terminology therefore involves a certain degree of arbitrariness. Moreover, articles on the MetaROR platform have been formally evaluated, and in that sense they resemble articles in traditional peer-reviewed journals. Although the nature of the evaluation is different (i.e., MetaROR provides a narrative assessment, while traditional journals provide a ‘stamp of approval’), we feel the resemblance justifies the use of the record type ‘journal-article’. We also hope that the use of this record type will help to ensure that articles evaluated by publish-review-curate (PRC) platforms are treated similarly to articles evaluated by traditional journals, advancing beyond more conservative ways of dealing with articles on PRC platforms.

There is a precedent for using the Crossref record type ‘journal-article’ for articles evaluated by PRC platforms. For over a decade, this approach has been used by platforms operated by F1000, such as F1000Research, Gates Open Research, Open Research Europe, and Wellcome Open Research. The approach we are taking at MetaROR is similar to the approach taken by these platforms. At the same time, our approach is different from the approach of eLife, another prominent PRC platform. eLife uses the record type ‘preprint’ for all versions of an article on its platform except for the version that the authors consider to be final and that they choose to designate as the ‘version of record’. This version has the record type ‘journal-article’.

Summary of MetaROR’s approach to Crossref DOI registration

Figure 1 summarizes MetaROR’s approach to Crossref DOI registration. The figure considers the situation in which an article went through two rounds of peer review by MetaROR. Both rounds of peer review involved two reviewers. After two rounds of peer review by MetaROR, the article was published in a journal. We emphasize that journal publication is optional in MetaROR’s PRC approach. It is included in Figure 1 for the sake of completeness.

Figure 1: MetaROR’s approach to Crossref DOI registration

Each element in Figure 1 represents an item that has its own Crossref DOI. The shape of an element indicates the Crossref record type of an item (‘preprint’, ‘journal-article’, ‘peer-review’). MetaROR is responsible for the blue elements in the figure. The gray elements are the responsibility of other actors, either a preprint server or a journal. Arrows represent relationships between items. These relationships are captured in the Crossref metadata of the various items.

Figure 1 shows how MetaROR treats articles, review reports, editorial assessments, and author responses as first-class research objects. Each object has its own DOI, while the objects are linked through structured metadata. Assigning DOIs to review reports, editorial assessments, and author responses is central to our commitment to transparency, recognition, and reuse of evaluative contributions.

We note that Figure 1 assumes each version of an article on a preprint server has its own DOI. This is indeed how DOI registration is handled by many preprint servers, such as the OSF servers (e.g., MetaArXiv, PsyArXiv, SocArXiv), ChemRxiv, Research Square, and Preprints.org. However, some preprint servers use a single DOI for all versions of an article. This is the case for bioRxiv and medRxiv and also for arXiv, which registers DOIs with DataCite rather than Crossref. In the future, we hope these preprint servers will also adopt versioned DOIs.

Outlook

Over the past 25 years, practices for registering DOIs and associated metadata have evolved along with broader developments in the scholarly communication landscape. Inevitably, DOI registration practices will always be lagging behind the most recent developments in scholarly communication. From this point of view, the lack of agreement on good practices for DOI registration in the context of PRC platforms is not surprising. This lack of agreement can in fact be seen as part of a larger discussion about the pros and cons of different infrastructural approaches for handling ‘preprint review metadata’, including for instance the COAR Notify approach and the DocMaps approach.

MetaROR’s approach to DOI registration demonstrates both the power and richness of Crossref’s metadata schema and its limitations. As discussed above, several colleagues who responded to our earlier blog post consider the lack of a record type for ‘reviewed preprints’ to be a significant limitation. With the growing interest in PRC models for scientific publishing, there appears to be a need to systematically evaluate possible improvements that can be made to Crossref’s metadata schema to offer better support for new approaches to scientific publishing.

We see this not only as a technical challenge but also as an issue of infrastructure governance. We therefore invite further dialogue between DOI registration agencies, other metadata infrastructures, preprint servers, PRC platforms, and indexing services to explore pathways for improving metadata standards, whether through new record types, extended relationship vocabularies, or shared best practices. We hope our experiences with MetaROR will contribute to the collective effort needed to ensure that emerging models of scholarly communication are represented accurately, transparently, and responsibly in the scholarly record.

Crossref note: This discussion chimes with related plans for extending our schemas: more granular vocabulary for items within journal articles, preprints, reviews, and others; clearer relationship types; and support for the forthcoming NISO JAV recommendations. Our Preprint Advisory Group will discuss the topic this year, and our Metadata Advisory Group has both ‘journal article type vocab’ and ‘relationships’ on its radar for 2026. We look forward to engaging further on this topic as we work towards more flexible schemas in support of the Research Nexus.

Innovation in scientific publishing and its implications for Crossref DOI registration practices - Request for input

Ludo Waltman — Thu, 25 Sep 2025 00:00:00 +0000

Lots of exciting innovations are being made in scientific publishing, often raising fundamental questions about established publishing practices. In this guest post, Ludo Waltman and André Brasil discuss the recently launched MetaROR publish-review-curate platform and the questions it raises about good practices for Crossref DOI registration in this emerging landscape.

Digital Object Identifiers (DOIs) are unique identifiers commonly assigned to research outputs such as journal articles, preprints, peer review reports, and datasets. The DOI of a research output allows the output to be identified online in a persistent way, even when the underlying publishing infrastructure changes (e.g., a journal moving from one publisher to another).

There are several DOI registration agencies. Most of the larger scientific publishers work with Crossref, and so do many preprint servers, and therefore our focus in this post is on Crossref. Crossref also keeps track of metadata associated with research outputs, such as the title, authors, and publication date of an output, and it makes this metadata openly available via APIs for all kinds of services to ingest and reuse. Because indexing, discovery, and evaluation tools rely heavily on this metadata, content registration practices and metadata design choices can have major effects on the visibility and findability of research outputs and on analytics used to monitor and assess research outputs and their contributors.

For the most common types of research outputs, such as journal articles and preprints, a broad consensus has emerged over the past decades on good practices for DOI registration. Such consensus means that articles are assigned the record type ‘article’ in their Crossref metadata. Likewise, many preprint servers register DOIs for preprints at Crossref, with the record type ‘preprint’ in the metadata. (The arXiv preprint server is an exception; it registers DOIs for preprints with DataCite rather than Crossref.)

For innovative new publication platforms, however, good practices for DOI registration are less clear. The approaches to scientific publishing offered by these platforms often do not fit neatly into established ways of working. For instance, for some of these platforms, the traditional distinction between peer-reviewed articles published in scientific journals and non-peer-reviewed articles posted on preprint servers is no longer applicable. This raises fundamental questions about suitable DOI registration practices for new approaches to scientific publishing.

MetaROR

The MetaROR (MetaResearch Open Review) platform, launched in November 2024 by the Research on Research Institute (RoRI) and the Association for Interdisciplinary Meta-Research and Open Science (AIMOS), offers an example of the challenge of developing appropriate DOI registration practices for new publishing models.

Inspired by similar initiatives such as eLife and others, MetaROR adopts the so-called publish-review-curate model. Authors first publish their article on a preprint server and then submit it to MetaROR. MetaROR then organizes an open peer review process for the article. Review reports are published on the MetaROR platform, along with a copy of the preprinted article and an editorial assessment. Rather than a simple binary decision (accept vs. reject), an editorial assessment is a short one-paragraph statement summarizing the strengths and weaknesses of an article. Each review report and each editorial assessment has its own DOI registered at Crossref. In this way, review reports are treated as first-class research outputs that can, for instance, be indexed in scientific literature databases and can be cited in other research outputs.

For an article submitted to MetaROR, the publication of the review reports, the editorial assessment, and a copy of the article itself concludes MetaROR’s publish-review-curate process. The authors of the article may revise their work in light of the feedback received, and MetaROR may review the revised article. However, there is no requirement that revisions must be made. The primary aim of the review reports and the editorial assessment published on the MetaROR platform is to offer context for readers of the article, helping readers understand the strengths and weaknesses of the article.

Crossref DOI registration

Registration of DOIs for open peer review reports is increasingly common. By registering Crossref DOIs for review reports and editorial assessments, MetaROR enables reviewers and editors to be recognized for their contributions. But what about recognition for authors?

A crucial element in MetaROR’s philosophy is that authors of articles peer-reviewed by MetaROR deserve to be recognized in a similar way as authors of articles published in traditional peer-reviewed journals. One way to promote appropriate recognition for authors of articles peer-reviewed by MetaROR is to ensure that articles on the MetaROR platform, just like articles in peer-reviewed journals, have their own DOI. While this may seem straightforward to arrange, it actually raises two non-trivial questions about good practices for Crossref DOI registration:

For each article on the MetaROR platform, there is a corresponding article on a preprint server. Is it acceptable to have two Crossref DOIs, one registered by the preprint server and one registered by the MetaROR platform, for essentially the same article?
If Crossref DOIs are registered for articles on the MetaROR platform, should the articles be assigned the type ‘article’ or the type ‘preprint’ in their Crossref metadata, or something else entirely?

On the first question, it could be argued that having two Crossref DOIs for the same article is problematic and that MetaROR, therefore, should not register DOIs for articles on its platform. Alternatively, one could argue that an article on the MetaROR platform differs in a meaningful way from the corresponding article on a preprint server, since the article on the MetaROR platform has been enriched with peer review reports and an editorial assessment, similar to the way an article in a peer-reviewed journal may be seen as an enriched version of the corresponding article on a preprint server. This line of reasoning would justify registering DOIs for articles on the MetaROR platform.

On the second question, the argument could be made that articles on the MetaROR platform should be assigned the type ‘preprint’ in their Crossref metadata, since the type ‘article’ is intended for articles in journals and MetaROR does not consider itself to be a journal (in fact, MetaROR works with partner journals to enable articles peer-reviewed by MetaROR to be published in journals) and does not certify articles in the way journals do (i.e., MetaROR does not make accept/reject decisions). On the other hand, one could argue that articles on the MetaROR platform should be assigned the type ‘article’, since the peer-reviewed nature of articles in journals is typically seen as the key factor distinguishing these articles from articles on preprint servers. Articles on the MetaROR platform have been peer-reviewed, and in that sense, they resemble articles in journals. A third line of reasoning could be that neither the ‘preprint’ nor the ‘article’ type is fully appropriate for articles on the MetaROR platform and, consequently, that there is a need for a new Crossref record type.

What is your take?

The MetaROR team, in consultation with Crossref, will need to decide how to deal with the two questions discussed in this blog post. After some preliminary conversations between the MetaROR team and Crossref, we decided to share these questions more widely to solicit input from the broader community. We invite you to share your thoughts on the two questions, either by posting a comment on this blog post or by reaching out to us on social media or by email. Community perspectives will help shape good practices not only for MetaROR but also for other publish-review-curate initiatives facing similar questions. We look forward to hearing from you!

Ludo Waltman and André Brasil are members of the editorial team of MetaROR. Ludo and André are grateful to Ginny Hendricks at Crossref for valuable discussions about the issues raised in this blog post.

Discovering relationships between preprints and journal articles

Dominika Tkaczyk — Thu, 07 Dec 2023 00:00:00 +0000

In the scholarly communications environment, the evolution of a journal article can be traced by the relationships it has with its preprints. Those preprint–journal article relationships are an important component of the research nexus. Some of those relationships are provided by Crossref members (including publishers, universities, research groups, funders, etc.) when they deposit metadata with Crossref, but we know that a significant number of them are missing. To fill this gap, we developed a new automated strategy for discovering relationships between preprints and journal articles and applied it to all the preprints in the Crossref database. We made the resulting dataset, containing both publisher-asserted and automatically discovered relationships, publicly available for anyone to analyse.

TL;DR

We have developed a new, heuristic-based strategy for matching journal articles to their preprints. It achieved the following results on the evaluation dataset: precision 0.99, recall 0.95, F0.5 0.98. The code is available here.
We applied the strategy to all the preprints in the Crossref database. It discovered 627K preprint–journal article relationships.
We gathered all preprint–journal article relationships deposited by Crossref members, merged them with those discovered by the new strategy, and made everything available as a dataset. There are 642K relationships in the dataset, including:
- 296K provided by the publisher and discovered by the strategy,
- 331K new relationships discovered by the strategy only,
- 15K provided by the publisher only.
In the future, we plan to replace our current matching strategy with the new one and make all discovered relationships available through the Crossref REST API.

Introduction

Relationships between preprints and journal articles link different versions of research outputs and allow one to follow the evolution of a publication over time. The Crossref deposit schema allows Crossref members to provide these relationships for new publications, either as a has-preprint relationship deposited with a journal article, or an is-preprint-of relationship deposited with a preprint.

To assist members who deposit preprints, we also try to connect deposited journal articles with preprints. The current method looks for an exact match between the title and first authors. We send possible matches as suggestions to the preprint server, which decides whether to update the metadata with the relationship.

At the time of writing, 137,837 journal articles in the Crossref database have a has-preprint relationship¹, and 562,225 works of type posted-content (preprints belong to this type) have an is-preprint-of relationship².

We suspected that many preprint–journal article relationships are missing, as some members inevitably fail to deposit them, even after suggestions from the current matching strategy. Another factor is that the current strategy is fairly conservative, and probably misses a significant number of relationships. For these reasons, we decided to investigate whether we could improve on the current process. Doing so would allow us to infer missing relationships on a large scale, similar to how we automatically match bibliographic references to DOIs.

This preprint matching task can be defined in two directions:

We start with a journal article and we want to find all its preprints.
We start with a preprint and we want to find a subsequently published journal article.

On the one hand, matching from journal articles to preprints would allow us to enrich the database continually with new relationships, either periodically or every time new content is added. Since journal articles tend to appear in the database later than their preprints, it makes sense for a new journal article to trigger the matching and not the other way round. This way we can expect the potential matches to be already in the database at the time of matching.

On the other hand, matching from preprints to journal articles can be useful in a situation where we want to add relationships in an existing database retrospectively. In our case, the database contains many more journal articles than preprints, so for performance reasons it is better to start with preprints.

In both cases we are dealing with structured matching, meaning that we match a metadata record of a work (preprint or journal article), rather than unstructured text.

As a result of matching a single preprint or a single journal article, we should expect zero or more matched journal articles/preprints. Multiple matches occur when:

there are multiple versions of the matched preprint and/or
matched works have duplicates.

The image shows the result of matching a journal article to two versions of a preprint:

Matching strategy

Our matching strategy uses the following workflow:

Gathering a short list of candidates using the Crossref REST API.
Scoring the similarity between the input item and each candidate.
A final decision about which candidates, if any, should be returned as matches.

Gathering candidates is done using the Crossref REST API’s query.bibliographic parameter. The query is a concatenation of the title and authors’ last names of the input item. We filter the candidates based on their type, to leave only preprints or only journal articles, depending on the direction of the matching. In the future, instead of getting the candidates from the REST API, we will be using a dedicated search engine, optimised for preprint matching.

Scoring candidates is heuristic-based. Similarities between titles, authors, and years are scored independently, and the final score is their average. Titles are compared in a fuzzy way using the rapidfuzz library. Authors are compared pairwise using the ORCID ID, or first/last names if ORCID ID is not available. The similarity score between issued years is 1 if the article was published no earlier than one year before the preprint and no later than three years after the preprint, or 0 otherwise.

The final decision is made based on two parameters: minimum score and maximum score difference, both chosen based on a validation dataset. The following diagram depicts the results of applying these two parameters in all possible scenarios. First, any candidate scoring below the minimum score is rejected (grey area in the diagram). Second, the scores of the remaining candidates are compared with the score of the top candidate. If the score of a candidate is close enough to the score of the top candidate, it is returned as a match (blue area).

This process can result in the following scenarios:

Scenario A: there is no candidate above the minimum score. This means nothing matches sufficiently, so nothing is returned.
Scenario B: there is only one candidate above the minimum score. This means it is the best match and we don’t have much of a choice, so it is returned.
Scenario C: there are multiple candidates above the minimum score, and they all have similar scores. This means they all are similarly good matches, so all are returned.
Scenario D: there are multiple candidates above the minimum score, but their scores differ a lot. In this case, we don’t want to return all of them, but only those that are close to the top match. Intuitively, we don’t want to return less-than-great matches if we have really great ones. This is when the maximum score difference comes into play: we return the candidates with the “score distance” to the top candidate lower than the maximum score difference.

We evaluated this strategy on a test set sampled from the Crossref metadata records. The test set contains 3,000 pairs (journal article, set of corresponding preprints). Half of the journal articles have known preprints and the other half don’t. The test set can be accessed here.

We used precision, recall, and F0.5 as evaluation metrics:

Precision measures the fraction of the matched relationships that are correct.
Recall measures the fraction of the true relationships that were matched.
F0.5 combines precision and recall in a way that favours precision.

The strategy achieved the following results: precision 0.9921, recall 0.9474, F0.5 0.9828. The average processing time was 0.96s.

We have made this strategy (journal article -> preprints) available through the (experimental) API: https://marple-research-crossref-org.turing.library.northwestern.edu/match?task=preprint-matching&strategy=preprint-sbmv&input=10.1109/access.2022.3213707. The input is the DOI of a journal article we want to match to preprints, and the output is a list of matches found, along with the score for each.

We have investigated other approaches to making decisions about which candidates to return as matches (step 3 above), including using machine learning. At present none have outperformed the heuristic approach described above. The heuristic method is also preferred because of its fast performance.

Preprint–journal article relationship dataset

We applied the strategy to the entire Crossref database:

We selected all preprints published until the end of August 2023. This included only works with type posted-content and subtype preprint, as reported by the REST API. There were 1,050,247 of them.
We ran the matching strategy (preprint -> journal article) on them. This resulted in 627,011 preprint–journal article relationships.
The resulting relationships were combined with the relationships deposited by the Crossref members. We included relationships of types has-preprint or is-preprint-of, where both sides of the relationship exist in our database, were published until the end of August 2023, and are of proper types and subtypes (type=journal-article for the journal article and type=posted-content, subtype=preprint for the preprint).

The resulting dataset is a single CSV file with the following fields:

preprint DOI (string)
journal article DOI (string)
whether the publisher of the journal article deposited this relationship (boolean)
whether the publisher of the preprint deposited this relationship (boolean)
the confidence score returned by the strategy (float, empty if the strategy did not discover this relationship)

The dataset contains:

641,950 relationships in total, including 580,532 preprints and 565,129 journal articles,
14,939 of them were deposited by the Crossref members, but not discovered by the strategy,
330,826 of them were discovered by the strategy, but not provided by any Crossref member,
296,185 of them were both deposited by a Crossref member and discovered by the strategy.

The dataset can be downloaded here.

Conclusions and what’s next

Overall, based on the number of existing and newly discovered preprint–journal article relationships, it seems that employing automated matching strategies would approximately double the number of these relationships in the Crossref database. In the future, we would like to match new journal articles on an ongoing basis. We also plan to make all discovered relationships available through the REST API.

In the meantime, we will be publishing the discovered relationships in the form of datasets, and we invite anyone interested to further analyse this data. And if you find out something interesting about preprints and their relationships, do let us know!

Better preprint metadata through community participation

Martyn Rittman — Wed, 09 Nov 2022 00:00:00 +0000

Preprints have become an important tool for rapidly communicating and iterating on research outputs. There is now a range of preprint servers, some subject-specific, some based on a particular geographical area, and others linked to publishers or individual journals in addition to generalist platforms. In 2016 the Crossref schema started to support preprints and since then the number of metadata records has grown to around 16,000 new preprint DOIs per month.

Preprints aren’t the same as journal articles, books, or conference papers. They have unique features, and how they are viewed and integrated into the publishing process has evolved over the past six years. For this reason, we have been revisiting the preprint metadata schema and decided that the best approach would be to form an advisory group (AG) of preprint practitioners and experts to help us.

The AG has identified a number of areas in which preprint metadata could be improved. Four of these were considered to have the highest priority:

Withdrawal and removal of preprints.
Preprints as an article type (not a subtype of posted content) in the schema.
Relationships between preprints and other outputs.
Versioning of preprints.

The members of the AG set to work with great enthusiasm, sharing perspectives and expertise. This led to a first tranche of recommendations shared for feedback earlier this year, and we’re grateful for engagement and feedback from the community over the last few months.

What did the community say?

Some of the points raised in the feedback were:

Could the origin of a withdrawal be included in the metadata, in particular whether it was requested by an author or another party?
Can the metadata represent when a preprint has been submitted to a journal and what stage it is in the editorial process?
Crossref is not alone in looking at preprint metadata, and several NISO groups are also engaged in related work.
Interoperability and the ability to create relationships with identifiers beyond DOIs is important to maintain an accurate and comprehensive record of research outputs.

These will form the basis for ongoing discussions.

What happens next?

There are three next steps that we will be taking.

The recommendations outline only the outcomes of discussions in a relatively brief format. We have been working on a more detailed paper to communicate more about what was discussed and provide some extra justification and alternatives.
The AG will continue to meet and discuss the points raised during consultation on the recommendations, along with topics that were considered a lower priority at an earlier stage.
We will draw up a set of proposals for specific changes to the metadata schema that will reflect the outcomes of the recommendations and discussions.

Although the initial period for feedback on preprint metadata has ended, we welcome feedback at any time. If you would like to get in touch, please contact me or any member of the advisory group.

An Advisory Group for Preprints

Martyn Rittman — Wed, 09 Jun 2021 00:00:00 +0000

We are delighted to announce the formation of a new Advisory Group to support us in improving preprint metadata. Preprints have grown in popularity over the last few years, with increasing focus brought by the need to rapidly disseminate knowledge in the midst of a global pandemic. We have supported metadata deposits for preprints under the record type ‘posted content’ since 2016, and members currently register a total of around 17,000 new preprints metadata records each month.

As preprints develop and different practices arise, we are keen to re-examine the metadata schema: to do this properly we need community input. We want to ensure that the schema is fit for purpose and supports the diversity of ways in which preprints are posted, linked with other objects, and used. Metadata schema need regular review, and this is just one example of a number of areas we are looking to update. Several topics we see as a high priority for preprints are better notification for when a preprint has been withdrawn or removed, accurate recording of versioning, and better indication of preprint server names.

We have invited a number of organisations we know to be active in this area, and are looking forward to some very positive discussions. Participants span five continents and include members who post preprints, indexing services, and others with significant experience in the area of preprints. The first meeting took place earlier this week and brought up a diverse range of themes that will be tackled in future meetings.

Using the Crossref REST API. Part 12 (with Europe PMC)

Christine Cormack Wood — Wed, 10 Oct 2018 00:00:00 +0000

As part of our blog series highlighting some of the tools and services that use our API, we asked Michael Parkin—Data Scientist at the European Bioinformatics Institute—a few questions about how Europe PMC uses our metadata where preprints are concerned.

Tell us a bit about Europe PMC

Europe PMC is a knowledgebase for life science research literature and a platform for innovation based on the content, such as text mining. It contains 34.6 million abstracts and 5 million full-text articles. At Europe PMC we support the research community by developing tools for knowledge discovery, linking publications with underlying research data, and building infrastructure to support text and data mining. Our goal is to create a supportive environment around open access content and data, to maximise its reuse.

What problem is your service trying to solve?

Recent years have seen a dramatic increase in the popularity of preprints within life sciences literature. Preprints have been supported by Crossref since November 2016. In response to the rise in popularity, we have started indexing preprints alongside traditional journal publishing within Europe PMC. We expect this will:

provide another means to access and discover this emergent form of scholarly content
help explore more transparently the role of preprints in the publishing ecosystem
support their inclusion in processes such as grant reporting and credit attribution systems

How do you use Crossref metadata?

Europe PMC operates an open citation network that uses reference lists from our full-text content, supplemented with metadata supplied by the Crossref OAI-PMH API. The number of citations we retrieve from Crossref increased significantly in 2017 thanks to the efforts of the Initiative for Open Citations (I4OC) in improving awareness about sharing citation data.

Our work to ingest preprints into Europe PMC, however, represents our first use of the Crossref REST API. We make a series of queries for each preprint provider, making use of the “posted-content”, “prefix” and (optionally) “has-abstract” filters. We intend to migrate to using the REST API for the majority of retrievals of Crossref content in due course.

What metadata values do you make use of?

Currently we make use of the following fields:

posted as a publication date
abstract
DOI
author for author given names and surnames
title as the preprint title
is-preprint-of to establish preprint –> article links

How often do you extract/query metadata?

We query the REST API daily making use of the from-index-date filter and cursor pagination to insert new or modify existing records. This means that preprints will be available in Europe PMC within 24 hours of the metadata being sent to Crossref. We store the full REST response in MongoDB, a document-based database. Here are some examples of Crossref API queries used to preprint provider PeerJ Preprints:

calling `https://api-crossref-org.turing.library.northwestern.edu/works?filter=type:posted-content,has-abstract:true,from-index-date:2018-07-29,prefix:10.7287&sort=updated&rows=1000&cursor=*`
calling `https://api-crossref-org.turing.library.northwestern.edu/works?filter=type:posted-content,has-abstract:true,from-index-date:2018-07-29,prefix:10.7287&sort=updated&rows=1000&cursor=AoN4ldf88uQCe6e1g%2FPkAj8SaHR0cDovL2R4LmRvaS5vcmcvMTAuNzI4Ny9wZWVyai5wcmVwcmludHMuMjcwNjJ2MQ%3D%3D`
Done importing PeerJ Preprints
modified: 2
inserted: 10

What do you do with the metadata?

From the database we parse out the relevant fields and pass them to our main relational database prior to indexing. This avails the preprint abstracts to all of the value-added services we offer for peer-reviewed abstracts, such as citations, grants, ORCID claiming, text mining, etc. We assign a unique persistent identifier comprising “PPR” followed by a number (1) to each preprint record.

This is displayed on the Europe PMC site as an abstract record, analogous to PubMed records, but with an obvious banner (2) indicating to readers the preprint designation; a tooltip provides further explanation of what a preprint is in comparison to a peer-reviewed article.

Once available on the Europe PMC platform, we then apply downstream processes including:

providing an Unpaywall link directly to the full-text (3);
adding a hyperlink to the final published version (if there is one that we can detect) (4);
incorporating the preprint into our citation network (5);
adding useful links to e.g. alternative metrics, scientific comments and peer reviews, underlying research data in life science databases (6);
providing text mined annotations via SciLite (7);
including funding information (8);
displaying ORCID claims in the author list (9).

What are the future plans for Europe PMC and preprints?

The inclusion of preprints within Europe PMC is of immediate benefit to researchers who want to explore the very latest research. Moreover we see this as an opportunity for both ourselves and the community to explore how preprints fit into the wider publishing ecosystem; for example to answer questions such as: How often will they be cited? How will they be linked to grant funding and other credit systems? How will they be reused?

What else would you like our API to do?

The REST API and rich metadata model provided by Crossref around preprints are both excellent, but the population of the metadata fields by preprint providers can be limited and/or heterogeneous. The key challenge we see is in encouraging providers to populate the Crossref metadata fields more fully and in a uniform manner.

Thanks to Michael.

If you’d like to share how you use our Metadata APIs please contact the Community team.

Leaving the house - where preprints go

Jennifer Lin — Tue, 21 Aug 2018 00:00:00 +0000

“Pre-prints” are sometimes neither Pre nor Print (c.f. https://doi-org.turing.library.northwestern.edu/10.12688/f1000research.11408.1, but they do go on and get published in journals. While researchers may have different motivations for posting a preprint, such as establishing a record of priority or seeking rapid feedback, the primary motivation appears to be timely sharing of results prior to journal publication.

So where in fact do preprints get published?

Although this is a simple question, we have not had an easy way to answer how this varies across disciplines, preprint repositories and journals. Until now. Crossref metadata provides not only an open and easy way to do so, but up-to-date data to get the latest results.

rOpenSci makin’ it sweet & easy

Crossref asks preprint repositories to update their metadata once a preprint has been published by adding the article link into its record via the “is-preprint-of” relation. As the record is processed, we make the link available going both directions, while preserving the provenance of the statement in the metadata output (“asserted-by”: “subject” or “asserted-by”: “object”). This results in bidirectional assertions in the Crossref REST API where search engines, analytics providers, indexes, etc. can get from the preprint to the article (“is-preprint-of”) as well as vice versa (“has-preprint”), making it easier to find, cite, link, assess, and reuse.

Using rOpenSci’s R library for the Crossref REST API (rcrossref), we pulled all articles connected to a previous preprint (https://api-crossref-org.turing.library.northwestern.edu/works?filter=relation.type:has-preprint&facet=publisher-name:&rows=0) and then aggregated them based on journal via their ISSNs (https://api-crossref-org.turing.library.northwestern.edu/works?filter=relation.type:has-preprint&facet=issn:), tallying the results in a tidy table with the journal name (ex: PLOS Biology (https://api-crossref-org.turing.library.northwestern.edu/journals/2167-8359)).

The big reveal

So without further delay, let’s look at the results of the 20 journals with the highest number of preprints associated with its articles (data from August 21, 2018):

Publisher	Journal	Count
PeerJ	PeerJ	1184
Springer Nature	Scientific Reports	394
eLife	eLife	375
PLOS	PLOS ONE	338
Proceedings of the National Academy of Sciences	PNAS	205
PLOS	PLOS Computational Biology	196
Springer Nature	Nature Communications	187
PLOS	PLOS Genetics	169
The Genetics Society of America	Genetics	168
Oxford University Press	Nucleic Acids Research	148
Oxford University Press	Bioinformatics	138
The Genetics Society of America	Genetics	120
The Genetics Society of America	G3: Genes, Genomes, Genetics	104
Cold Spring Harbor Laboratory	Genome Research	104
Oxford University Press	Molecular Biology and Evolution	100
MDPI AG	Energies	98
MDPI AG	Sensors	96
Springer Nature	BMC Genomics	92
MDPI AG	International Journal of Molecular Sciences	86
JMIR Publications	Journal of Medical Internet Research	83

This list has not been normalized or weighted based on the size of the journal. The following observations are informed speculations, as we can only infer so much from the raw data:

Disciplinary practice: This phenomenon where preprints are a part of disciplinary practice accounts for about half of the journals represented on the list. Certain communities such as genetics and computational fields have been early adopters of preprints. As such, we see higher rates of preprint-to-article publication in journals that publish their work.
Partnerships: Partnerships that facilitate submission from the preprint repository directly to a publisher or peer review service (ex: BioRxiv B2J program) make it easier for researchers to move from preprint-sharing seamlessly to submitting their journal article manuscript.
Tie-ins: A quarter of the journals on the list are run by publishers with a preprint service, and have been able to tie together both arms of publishing. This removes barriers to journal article submission in the same manner as integrations between repositories and publishers, but does so as a single party.
Publisher support and treatment: We also see that strong proponents and early partners of preprint repositories tend to have higher counts. Some publishers have been more outspoken in their welcome of preprints, such as PNAS. Sometimes this support also comes in the form of special treatment. In the process of crafting editorial policy on publishing results previously posted in a preprint, some journals have carved out particular affordances in their publication workflow and content delivery streams that may contribute to the higher counts of articles. For example, Nature Research displays the preprints of submitted articles under consideration: https://nature-research-under-consideration-nature-com.turing.library.northwestern.edu/.
Mega-journals: Mega-journals such as Scientific Reports and PLOS ONE have not discouraged preprints. As such, and due to the size of their publication output, they have easily found a place among the higher counts on the list.

Taking a closer look

One major consideration in these results, concerns what’s missing in the data. These fall into two camps: incomplete member data, and incomplete membership coverage.

We have been working with our members to deposit preprints using the proper record type, and to provide links to published articles in their metadata. However, not all have yet done so (ex: SSRN), leading to holes in our research nexus graph, which subsequently detracts from the completeness of the data.

We celebrate the preprint repositories who are required to update their metadata when an article is published from a preprint, thereby populating the map with critical bridges between preprints and articles. Crossref participation benefits not only the content owner, but the membership at large and all the systems across the research ecosystem powered by Crossref metadata.

Lastly, this data is dependent on the coverage of preprint repositories who register content with us. We are thrilled that Center for Open Science, our newest preprints addition who represents 21 community repositories, has recently filled in swaths of the map. But there remain dead zones in the research graph from repositories who are not Crossref members (ex: ArXiv). Their disciplines, as a result, are under represented in these results.

Everyone dive in!

As to the question of “where do preprints get published?”, anyone in fact can answer this question based on the metadata Crossref collects and provides to the community as an open infrastructure provider. We encourage the community to explore and analyze the data further with other available datasets to glean more insights on how scholarly communications is changing with the increasing growth of preprints. For example, the effective results across all journals represented can be weighted based on the number of articles published by each journal.

Crossref data is open for all to examine and reuse through our REST API. Please dive in and share your findings with us!

Preprints growth rate ten times higher than journal articles

Jennifer Lin — Thu, 31 May 2018 00:00:00 +0000

The Crossref graph of the research enterprise is growing at an impressive rate of 2.5 million records a month - scholarly communications of all stripes and sizes. Preprints are one of the fastest growing types of content. While preprints may not be new, the growth may well be: ~30% for the past 2 years (compared to article growth of 2-3% for the same period). We began supporting preprints in November 2016 at the behest of our members. When members register them, we ensure that: links to these publications persist over time; they are connected to the full history of the shared research results; and the citation record is clear and up-to-date.

Summary

As of May 24, 2018 we have 44,388 works (see API query https://api-crossref-org.turing.library.northwestern.edu/types/posted-content/works with a json viewer) registered as posted content. Today that number is over 150k. Preprints are part of this record type category, which is meant to house scholarly outputs that have been posted online and intended for publication in the future.

For a more granular view, see the monthly stats captured by Jordan Anaya in PrePubMed. This data is based on a slightly different set of preprint repositories, though both show the same trends.

The figure below shows the preprints registered with Crossref, broken down by repository.

We eagerly await our newest preprints member, Center for Open Science, who will soon be registering the preprints from their 18 community archives with us (~9k preprints total to date).

Metadata coverage

We accept a range of metadata for the preprints registered with us, including:

Repository name & hosting platform
Contributor names & ORCID iDs
Title
Dates (posted, accepted)
License
Funding
Abstract
Relations
References

As with all resource/record types, certain metadata is required, though others are optional. We encourage full coverage of metadata in the record where applicable and possible. So what are publishers including in their posted content records? The summary view is as follows:

License: 9926 (json), 22% (PeerJ Preprints, ChemRxiv)
Funder: 0 (json), 0%
ORCID: 19309 (json), 44% (bioRxiv, PeerJ Preprints, Preprints.org, ChemRxiv)
Abstracts: 35874 (json), 81% (bioRxiv, PeerJ Preprints, ChemRxiv)
References: 1921 (json):, 4% (JMIR)

Compared to all the published content registered with us over time, preprints have above average coverage of ORCID iDs deposited and show well above average with abstract metadata. However, they are significantly lagging behind with depositing references, license, and funding metadata. (See a summary of the full corpus stats taken two months ago in the blog post, A Lustrum over the Weekend).

Preprint-article pairs

Members registering preprints have an obligation to update the metadata record when a journal article is subsequently published, to clearly identify this work. This pairing is passed on to our metadata users: indexing platforms; recommendations engines; platforms; tools, etc. which pull from our APIs. (The preprint landing page also must link to the article.) As such, the preprint-article pairings are amassing as each week passes. We currently have a total of 12983 (json) preprints connected to articles. The figure below provides the counts based on repository.

Citations

We can see from preprint Cited-by counts that researchers are indeed citing preprints in their articles. This practice is an extension of the common citation behavior to provide evidence for and credit to previous work, a natural consequence of work shared with their peers. The most highly cited preprint papers (json) as of May 24, 2018 are as follows. In some cases, a subsequent paper was published from the results shared in the preprint. These have also accrued citations in their own right and these are also indicated in the table below.

No.	Cited-by	Preprint DOI	Preprint title	Date	Subsequent journal article	Citations of journal article
1	Cited-by 72	https://doi-org.turing.library.northwestern.edu/10.1101/005165	qqman: an R package for visualizing GWAS results using Q-Q and manhattan plots	May 14, 2014.	n/a	n/a
2	Cited-by 63	https://doi-org.turing.library.northwestern.edu/10.1101/002824	HTSeq - A Python framework to work with high-throughput sequencing data	August 19, 2014	Bioinformatics, https://doi-org.turing.library.northwestern.edu/10.1093/bioinformatics/btu638	2372
3	Cited-by 43	https://doi-org.turing.library.northwestern.edu/10.1101/030338	Analysis of protein-coding genetic variation in 60,706 humans	May 10, 2016	Nature, https://doi-org.turing.library.northwestern.edu/10.1038/nature19057	1598
4	Cited-by 38	https://doi-org.turing.library.northwestern.edu/10.1101/002832	Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2	November 17, 2014	Genome Biology, https://doi-org.turing.library.northwestern.edu/10.1186/s13059-014-0550-8	3284
5	Cited-by 32	https://doi-org.turing.library.northwestern.edu/10.1101/021592	Salmon provides accurate, fast, and bias-aware transcript expression estimates using dual-phase inference	August 30, 2016	Nature Methods, https://doi-org.turing.library.northwestern.edu/10.1038/nmeth.4197	112
6	Cited-by 22	https://doi-org.turing.library.northwestern.edu/10.1101/012401	DensiTree 2: Seeing Trees Through the Forest	December 8, 2014	n/a	n/a
7	Cited-by 21	https://doi-org.turing.library.northwestern.edu/10.1101/011650	FusionCatcher - a tool for finding somatic fusion genes in paired-end RNA-sequencing data	November 19, 2014	n/a	n/a
8	Cited-by 19	https://doi-org.turing.library.northwestern.edu/10.1101/048991	Analysis of shared heritability in common disorders of the brain	September 6, 2017	n/a	n/a
9	Cited-by 18	https://doi-org.turing.library.northwestern.edu/10.1101/006395	Error correction and assembly complexity of single molecule sequencing reads	June 18, 2014	n/a	n/a
10	Cited-by 18	https://doi-org.turing.library.northwestern.edu/10.1101/032839	Spread of the pandemic Zika virus lineage is associated with NS1 codon usage adaptation in humans	November 25, 2015	n/a	n/a

The relationship between preprints and the proceeding publication is an interesting area that is not yet well understood. We invite the community to analyze the Crossref metadata using the REST API in concert with other datasets. For example, the citation lifecycle for these two research products has been one of speculation so far without a systematic investigation into patterns and timeframes of preprint citations and those of its succeeding article across the corpus. Here, submission dates would be critical data to this research question as publication windows vary significantly by publisher and by paper.

More metadata for machines-citations, relations, and preprints arrive in the REST API

Kirsty Meddings — Mon, 11 Sep 2017 00:00:00 +0000

Over the past few months we have been adding to the metadata and functionality of our REST API, Crossref’s public machine interface for the metadata of all 90 million+ registered content items. Much of the work focused on a review and upgrade of the API’s code and architecture in order to better support its rapidly growing usage. But we have also extended the types of metadata that the API can deliver.

One of the biggest changes is that references are now available if the publisher has made them public (a simple email instruction to us). Currently 45% of all publications with deposited references are now accessible. For example:

This article studying fluid ejection from animals has 55 references and they are all in the metadata here. You can also see that the article has an is-referenced-by count of 6.
This article exploring whether people bitten by their cat are more likely to develop depression has 142 references and is referenced by 12.

We recently announced that we would be accepting preprints, and the metadata for 15,000 preprints registered to date is now in the API, labelled as posted-content. Over 4,000 have been subsequently published in a journal, and the Crossref metadata now links these preprints to their respective articles (and vice versa). For example this article in Biorxiv has since been published in a journal, and this relationship is recorded in its metadata as is-preprint-of.

Also new to the API:

Cited-by counts - the number of times each work has been referenced by other content registered with us. Look for is-referenced-by-count within a record.
This article from 1953 about a fairly notable discovery has been cited 4832 times, but the two most cited articles both have over 100,000 citations and thousands have been cited more than Watson and Crick.
Abstracts for over 1 million works.
Similarity Check URLs–the ones that Turnitin crawl to add content to the database–are now showing so that participating publishers can check that they are including them in their metadata deposits.
Subject categories have been added for an additional 7000 journal titles, taking the total number of classified titles to ~45,000.

Are you already using our Metadata APIs for your system or project? We’re always keen to hear new use cases and happy to answer any questions.

You may need to install a JSON viewer extension in your browser to render API queries in a human-friendly way.

Included, registered, available: let the preprint linking commence.

Rachael Lammey — Mon, 05 Dec 2016 00:00:00 +0000

We began accepting preprints as a new record type last month (in a category known as “posted content” in our XML schema). Over 1,000 records have already been registered in the first few weeks since we launched the service.

By extending our existing services to preprints, we want to help make sure that:

links to these publications persist over time
they are connected to the full history of the shared research
the citation record is clear and up-to-date.

It’s not just collecting the metadata however, it’s also making it available so that it can be as widely used as possible. Preprint metadata is no different. As with all record types, we make the metadata available for machine and human access, across multiple interfaces (e.g. REST API, OAI-PMH, Crossref Metadata Search)

For example, you can see information on the preprint https://doi-org.turing.library.northwestern.edu/10.20944/preprints201608.0191.v1 in a number of ways:

If you want to see all the preprint metadata deposited so far, try https://api-crossref-org.turing.library.northwestern.edu/v1/types/posted-content/works. Over 1,000 records have already been registered in the first few weeks since we launched the service.

Crossref members depositing preprints need to make sure they:

Register content using the posted content metadata schema.
Respond to our match notifications that a manuscript / version of record (AM/VOR) has been registered and link to that within seven days.
Label the manuscript as a preprint clearly, above the scroll on the preprint landing page, and ensure that any link to the AM/VOR is also prominently displayed above the scroll.

It’s important to clearly label the record type so we can ensure that the connections between preprints and the associated literature are clearly visible, to both humans and machines.

As with other record types, there is a registration fee to include content in the Crossref system. For preprints, it’s $0.25 fee for current preprint files and $0.15 for back-year records.

Are you an existing Crossref member who wants to assign preprint DOIs? Let’s talk about getting started or migrating any existing content over to the dedicated preprint deposit schema.

Interested in becoming a Crossref member to assign DOIs to your preprints? Contact our membership specialist so we can answer any questions and get you set up as a member.

Preprints are go at Crossref!

Rachael Lammey — Wed, 02 Nov 2016 00:00:00 +0000

We’re excited to say that we’ve finished the work on our infrastructure to allow members to register preprints. Want to know why we’re doing this? Jennifer Lin explains the rationale in detail in an earlier post, but in short we want to help make sure that:

links to these publications persist over time
they are connected to the full history of the shared research results
the citation record is clear and up-to-date

Doing so will help fully integrate preprint publications into the formal scholarly record.

What’s new?

We’ve had to do some work on our own infrastructure to facilitate the inclusion of preprints, enabling:

Crossref membership for preprint repositories by updating our membership criteria and creating a policies for preprints
The deposit of persistent identifiers for preprints to ensure successful links to the scholarly record over the course of time via the DOI resolver.
Content Registration for preprints with custom metadata that reflect researcher workflows from preprint to formal publication (this custom metadata will then be visible to anyone using the Crossref metadata).
Notification of links between preprints and formal publications that may follow (journal articles, monographs, etc.).
Auto-update of ORCID records to ensure that preprint contributors get credit for their work.
Preprint and funder registration to automatically report research contributions based on funder and grant identification.
It will also allow for the collection of “event data” that capture activities surrounding preprints (usage, social shares, mentions, discussions, recommendations, links to datasets and other research entities, etc.).

Now we’re ready to go!

Early adopters

We have been working with various preprint publishers who are launching (or planning to launch) their own preprint initiatives.

Preprints.org is the first to successfully make preprints deposits using the dedicated schema. For example, this preprint https://doi-org.turing.library.northwestern.edu/10.20944/preprints201608.0191.v1 is registered with Crossref. It is linked to a published journal article https://doi-org.turing.library.northwestern.edu/10.3390/data1030014 both in the online display as well the preprint’s Crossref metadata record. Others are getting ready to go - will your organisation be next? (Technical documentation available here.)

Martyn Rittman, from Preprints, operated by MDPI said: Preprints.org is delighted to be the very first to integrate the Crossref schema for preprints. We believe it is an important step in allowing working papers and preliminary results to be fully citable as soon as they are available. It also makes it easy to link to the final peer-reviewed version, regardless of where it is published. Thanks to the hard work of Crossref and clear documentation, the schema was very simple to implement and has been applied retrospectively to all preprints at Preprints.org.

Jessica Polka, Director, ASAPbio adds: ASAPbio is a scientist-driven community initiative to promote the productive use of preprints in the life sciences. We’re thrilled to see Crossref’s development of a service that enables preprints to better contribute to the scholarly record. This infrastructure lays a necessary foundation for increasing acceptance of preprints as a valuable form of scientific communication among biologists.

Questions?

Get in touch with any questions or comments, or join our upcoming webinar to talk about preprints, infrastructure and where we go from here.

Preprints and Crossref’s metadata services

Chuck Koscher — Mon, 29 Aug 2016 00:00:00 +0000

We’re putting the final touches on the changes that will allow preprint publishers to register their metadata with Crossref and assign DOIs. These changes support Crossref’s Cited-by linking between the preprint and other scholarly publications (journal articles, books, conference proceedings). Full preprint support will be released over the next few weeks.

I’d like to mention one change that will be immediately visible to Crossref members who use our OAI based service to retrieve Cited-by links to their content.

This API, show in an example here, is intended to retrieve large quantities of data detailing all the Cited-by links to a given publication. The example request shows pulling the data for an IEEE conference proceeding.

example:

http://oai.crossref.org.turing.library.northwestern.edu/OAIHAndler?verb=ListRecords&usr=*** pwd=****&set=B:10.1109:1070762&metadataPrefix=cr_Cited-by

With the new change, results will now identify the type of content that is doing the citing. The example results below shows that the DOI 10.1109/CSMR.2012.14 is cited by five other items and displays the DOIs of those items and their record type.

When preprint content that cites other scholarly work starts being registered with Crossref, members using this API will start seeing data like the following:

For many users of Crossref metadata the introduction of preprints will be transparent until preprint content starts being registered. However, a few changes like the one above have benefits not limited to just preprints.

Getting ready to run with preprints, any day now

Jennifer Lin — Tue, 16 Aug 2016 00:00:00 +0000

While preprints have been a formal part of scholarly communications for decades in certain communities, they have not been fully adopted to date across most disciplines or systems. That may be changing very soon and quite rapidly, as new initiatives come thick and fast from researchers, funders, and publishers alike. This flurry of activity points to the realization from these parties of preprints’ potential benefits:

Accelerating the sharing of results;

Catalyzing research discovery;

Establishing priority of discoveries and ideas;

Facilitating career advancement; and

Improving the culture of communication within the scholarly community.

To acknowledge them as a legitimate part of the research story, we need to fully build preprints into the broader research infrastructure. Preprints need infrastructure support just like journal articles, monographs, and other formal research outputs. Otherwise, we (continue to) have a two-tiered scholarly communications system, unlinked and operating independently.

Infrastructure for preprints

For this reason, the team at Crossref is extending its infrastructure services to allow members to register preprints. This new development is designed to provide custom support for preprints. It will ensure that: links to these publications persist over time; they are connected to the full history of the shared research results; and the citation record is clear and up-to-date. We established this preprints service to fully integrate preprint publications into the formal scholarly record with features such as:

Crossref membership for preprint repositories, joining the community of publishers who have made a commitment to maintain and connect scholarly publications.

Persistent identifiers for preprints to ensure successful links to the scholarly record over the course of time via the DOI resolver.

Content Registration for preprints with custom metadata that reflect researcher workflows from preprint to formal publication.

Notification of links between preprints and formal publications that may follow (journal articles, monographs, etc.).

Collection of “event data” that capture activities surrounding preprints (usage, social shares, mentions, discussions, recommendations, links to datasets and other research entities, etc.).

Reference linking for preprints, connecting up the scholarly record to associated literature

Auto-update of ORCID records to ensure that preprint contributors get credit for their work.

Preprint and funder registration to automatically report research contributions based on funder and grant identification.

Supporting utility & effectiveness of preprints for all

To build the service, we are listening to the research community tell us their vision of what preprints will do. We solicited use cases from the community and have built a registry of preprint user stories with researchers, publishers, funding agencies, tenure and promotion committees in academic institutions, and technology providers.

To realize the user stories, the research enterprise will no doubt need brand new tools and existing systems enhancements. Crossref’s preprints infrastructure will support the development of all needs currently registered. The community at large can focus on building effective solutions, instead of finding or securing access to data. All data are available without restriction to all so that participants as well the services and systems supporting them can access the data and reuse it for advancing early dissemination, literature discovery, research tracking, promotion and funding assessment, etc.

These are exciting days for scholarly communications. Over time, we envision an even more vibrant ecosystem of research outputs that include existing artefacts linked up to preprints. And Crossref is committed to providing infrastructure for the dynamic enterprise all along the way.

We plan to announce the availability of the preprints infrastructure and further technical details within the next few weeks. If you’re interested in learning more about how these will be supported, get in touch!

Hello preprints, what’s your story?

Jennifer Lin — Thu, 23 Jun 2016 00:00:00 +0000

The role of preprints

Crossref provides infrastructure services and therefore we support scholarly communications as it evolves over time. Today, preprints are increasingly discussed as a valuable part of the research story (beyond physics, math, and a small set of sub-disciplines). Preprints might play a positive role in catalyzing research discovery, establishing priority of discoveries and ideas, facilitating career advancement, and improving the culture of communication within the scholarly community.

As we shared in an earlier blog post last month, members will be able to register Crossref DOIs for preprints later this year. We will connect the full history of a research work, and ensure the citation record is clear and up-to-date. As we build out this new record/resource type, we’d love to hear how the research community envisions what preprints will do.

What’s your story, preprint?

So we can develop a service that supports the whole host of potential uses for all stakeholders, we ask the entire research community to contribute preprints user stories . User stories are concrete descriptions of a specific need, typically used in technology development: As a [x], I want to [y] to that I can [z]. User stories take the “end-user’s” perspective as they focus on a discrete result and its value. They are essential when implementing solutions that must meet a wide range of needs, across a diverse set of constituents. For example:

As an author, I want to share results before my paper is submitted to a journal so that I can get rapid feedback on it and make improvements before publication.

As a researcher who is part of a tenure and promotion committee or funder review panel, I want to know the reach of early results published from the candidate so that I can more quickly track the impact of results, rather than relying only on journal articles that take much longer to publish.

As a journal publisher, I want to know whether a preprint exists for a manuscript submitted to me so that I can decide whether I will accept the submission based on my editorial policy.

We aim to assemble a full catalog that cuts across research disciplines and stakeholder groups. We want to hear from you: researchers, publishers, funding agencies, scholarly societies, academic institutions, technology providers, other infrastructure providers , etc .

Tell us your story here

To ensure that your needs are included, please send us your user stories via this user story “deposit” form . They will be added to the full registry of contributions from the community, which we hope will serve as a key resource for all those developing preprints into a core part of scholarly communications (e.g., ASAPbio, etc.).

Members will soon be able to assign Crossref DOIs to preprints

Geoffrey Bilder — Thu, 05 May 2016 00:00:00 +0000

TL;DR

By August 2016, Crossref will enable its members to assign Crossref DOIs to preprints. Preprint DOIs will be assigned by the Crossref member responsible for the preprint and that DOI will be different from the DOI assigned by the publisher to the accepted manuscript and version of record. Crossref’s display guidelines, tools and APIs will be modified in order to enable researchers to easily identify and link to the best available version of a document (BAV). We are doing this in order to support the changing publishing models of our members and in order to clarify the scholarly citation record.

Background

Why is this news? Well, to understand that you need to know a little Crossref history.

(cue music and fade to sepia)

When Crossref was founded, one of its major goals was to clarify the scholarly record by uniquely identifying formally published scholarly content on the web so that it could be cited precisely. At the time, our members had two primary concerns:

That a Crossref DOI should point to one intellectually discrete scholarly document. That is, they did not want one Crossref DOI to be assigned to two documents that appeared largely similar, but which might vary in intellectually significant ways.
That two DOIs should not point to the same intellectually discrete document. They wanted it to be easy for all to tell when the same discrete intellectual content was cited.

As such, when Crossref was founded, we developed a complex set of rules that were colloquially known by our members as Crossref’s rules “prohibiting the assignment of DOIs to duplicative content.”

(cue music, show wavy lines, return to color)

Well… as we gained experience in assigning DOIs, many of these rules have been amended or discarded when it became apparent that they didn’t actually support common scholarly citation practice and/or otherwise muddied the scholarly citation record.

For example, sometimes a document will be re-published in a special issue or an anthology. Before the advent of the DOI, it was common citation practice to always cite a document in the context in which it was read. The context of the document could, after all, affect the interpretation or crediting of the work. But it would be impossible to support this common citation practice if we were to assign the same Crossref DOI to the article on both its original context and in its re-published form. Our current recommendation in these situations is to assign separate DOIs to content that is republished in another context.

Another example occurs when a particular copy of a two identical documents has been annotated. For example, though the Handbook to The birds of Australia By John Gould has its own Crossref DOI (https://doi-org.turing.library.northwestern.edu/10.5962/bhl.title.8367), another copy of the same book has been hand-annotated by Charles Darwin and also has its own, different Crossref DOI (https://doi-org.turing.library.northwestern.edu/10.5962/bhl.title.50403). Historians of science quite reasonably may want to refer and cite the particular annotated copy of this historic document.

__[So much for not assigning two separate Crossref DOIs to identical documents.]

Finally, we should note a far more common example practice in our industry. Our members often make content available online with a Crossref DOI before they consider it to be formally published. This practice goes by a number of names including “publish ahead of print,” “article in progress,” “article in press,” “online ahead of print,” “online first”, etc.

But in each case, the process is the same- the publisher is assigning a Crossref DOI to the document soon after it has been accepted for publication and this same Crossref DOI is carried over to the finally published article. Again, this practice just reflects that the “intellectual” content of the accepted manuscript should not change between the point of acceptance and the point of publication, so of the purposes of “citation” they are largely interchangeable.

[So much for not assigning one Crossref DOI to two versions of the same document.]

Now, in the above cases it also helps to clarify the scholarly record to also specify that the respective Crossref DOIs of the original and the “duplicative” work are related, and we encourage our members to make these connections explicit when they can. Nonetheless, it is paramount in both cases to allow the “duplicative works” to be cited precisely and independently.

Which brings us back to preprints.

The case for preprints

First we should define what was meant by preprints because even this commonly used term sometimes means different things to different communities. We have historically considered preprints to be any version of a manuscript that is intended for publication but that has not yet been submitted to a publisher for formal review. Note that this definition does not include “accepted manuscripts” which -as we noted above- often already have Crossref DOIs assigned to them soon after acceptance.

Crossref members originally worried that, by assigning DOIs to preprints, we would end up muddying the scholarly record. They worried that the very presence of a Crossref DOI would be interpreted to mean that the content to which it had been applied had gone through a formal publishing process. And unlike the case with “accepted manuscripts”, the difference between intellectual content of a preprint and the final published version can sometimes be substantial. At the time, it seemed that the scholarly record would be clarified by prohibiting the assignment of DOIs to preprints.

But again, changes in the scholarly communication landscape have led us to -as the youngsters say- pivot.

A Koan

When is a preprint a preprint?

Crossref has always been catholic in its definition of “publisher.” Many of our members do not consider “publishing” to be their primary mission. The OECD and World Bank are two obvious cases here. But our membership also includes government departments, universities and archives. In these latter cases they have traditionally assigned Crossref DOIs to things like internal reports, grey literature, working papers, etc. This activity was clearly within the original rules set out by Crossref. And this is where our koan comes into play- “when is a preprint a preprint?”

It is often difficult to predict when something might eventually be formally published. How do you a priori know that working paper will never be submitted for publication? After all, everything could potentially be submitted for publication (Sometimes it seems everything is.)

This is the dilemma that was faced by a few of our members. For example, Cold Spring Harbor Laboratory, which runs bioRxiv has been a Crossref member since 2000 and has assigned over 35,000 Crossref DOIs. They have been assiduous in trying to stick to Crossref’s rules about preprints. Furthermore, they have taken equal care to ensure that preprints in bioRxiv are labeled as such and linked to the final publication (via a Crossref journal DOI) when it is available. This takes a lot of work.

But often bioRxiv simply has no way of telling when the authors of a working paper or report might suddenly decide to submit their work for publication. So they have found themselves occasionally and inadvertently violating Crossref’s rules on preprints because they had no way of predicting when something would magically transform from being an innocuous working paper into a fraught preprint.

It is a testament to bioRxiv that they have persevered. We have other members who face the same problem. They have not given up. They have not gone elsewhere for their DOIs.

Which brings us to our next point.

Not All DOIs

Have you noticed how often we use the phrase “Crossref DOIs?” Were you wondering if this was an annoying affectation or an example of a marketing department gone mad? It’s neither. It is an essential distinction that we make because Crossref is just one of several DOI registration agencies. Although all DOIs are “compatible” in the minimal sense that you can “resolve” them to a location on the web, that does not mean that all DOIs work identically. Different DOI registration agencies have different constituencies, different services, different governance models and different rules covering what their members can assign their respective DOIs to.

This was not the case when Crossref was founded and our rules were first drafted. At the time, Crossref was the only registration agency and, as such, the rule which prohibited the assignment of Crossref DOIs to preprints kinda worked. But it was unworkable in the longer term.

Quite naturally, new DOI registration agencies have been established for different communities with different primary use-cases. While Crossref could have a rule prohibiting the assignment of Crossref DOIs to preprints, there was nothing stopping another registration agency from allowing (indeed, encouraging) its members to assign DOIs to preprints.

So the simple fact is that DOIs could be assigned to preprints regardless of Crossref’s old rules. By continuing to prohibit the practice at Crossref we were just making life for some of our existing members more difficult.

And it has become clear that the situation would only get worse as more of our members started to roll-out new publishing and business models.

Business model neutral

Crossref has always been business model neutral. We need to adapt and change to support our members’ business models, not the other way around.

A number of our members are starting to adopt publishing workflows that are more fluid and public than established publishing models. These new workflows make much of the submission and review process open, which, in turn often blurs the historically hard distinctions between a draft manuscript, a preprint, a revised proof, an accepted manuscript, the “final” published version, and subsequent corrections and updates. Where as in classic publishing models a document went through a series of discrete state-changes (some in public, many in private) new publishing workflows treat document versions as a continuum, most of which are made available publicly and which consequently may be used cited at almost any point in the publishing process.

In short, Crossref’s members increasingly need the flexibility to assign DOIs at different points in the publishing lifecycle. Rather than enforce rules that enshrined an existing publishing or business model, we need to work with our members to establish and adopt new DOI assignment practices which support evolving publishing models whilst maintaining a clear citation record and which lets researchers easily identify the best available version (BAV) of a document or research object.

So you see, not all of our motivations for this change in policy are opportunistic or prosaic. Underneath our gruff and flinty exterior is a soft, idealistic center. There are principles at work here as well.

What next

So this isn’t just matter of changing our rules and display guidelines. We also have to make some schema changes, and adjust our services and APIs to clearly distinguish between preprints and accepted manuscripts/versions of record. Additionally, we will be building tools to make it much easier for our members to link preprints to the final published article (and vice versa). Finally, we need to update our documentation to help our members take advantage of the new functionality. We expect that everything will be in place by the end of August, 2016, at which point you will see another announcement from us.

eprintweb.org

Ed Pentz — Fri, 02 Mar 2007 00:00:00 +0000

IOP has created an instance of the arXiv repository called eprintweb.org at https://web.archive.org/web/20130803071935/http://eprintweb.org/S/. What’s the difference from arXiv? From the eprinteweb.org site - “We have focused on your experience as a user, and have addressed issues of navigation, searching, personalization and presentation, in order to enhance that experience. We have also introduced reference linking across the entire content, and enhanced searching on all key fields, including institutional address.”

The site looks very good and it’s interesting to see a publisher developing a service directly engaging with a repository.

Some interesting points to note: There are DOI links to published articles - http://www.eprintweb.org/S/article/astro-ph/0603001 - which IOP gets from Crossref. References in the preprints are also linked - http://www.eprintweb.org/S/article/astro-ph/0603001/refs

Crossref will soon be making available an author/title only query for repositories to use to find DOIs for published papers when the preprint doesn’t have the full citation. Many authors don’t go back to their preprints to update the reference to the published version but the new Crossref query will enable the repositories to do this automatically.