Evidence of the impact of journal articles

Raw citation counts show how often other articles indexed in the same database cite an article. Citations to an article can best be seen as the starting point to uncover who is citing your research, and preferably the reasons why, as these (may) point to qualitative aspects of your work.

Further guidance:

• Citations & Citation databases – Who is citing?
• Citations & Citation databases – Citation context

Citations are most often associated with citations in scholarly publications (books and journal articles) tracked by citation databases such as the Web of Science, Scopus, Dimensions, Lens, and Google Scholar *. They hardly index any of the professional journals, or more popular magazines and no citation database is all-inclusive, meaning that you might have to use more than one citation database to find citing documents of your research output.

The search for citations in citation databases is straightforward:

• Do a title or DOI search for your article
• Look for ‘Times Cited’, ‘Cited by’, or ‘Citing Works’
• Link to the list of citing sources when you want to analyze them further to uncover who is citing your research or for what reason.

Raw citation counts for an article are also available through other platforms such as journal websites, Research Information, PlumX and Altmetric (Explorer), but use raw citation counts with caution as they are difficult to interpret when a benchmark is missing to which the counts could be compared. Benchmarks necessary to interpret raw citation counts in a meaningful way only exist for citations to articles, reviews, books, or proceedings papers indexed in the Web of Science or Scopus.

Further guidance:

• Responsible metrics – Pitfalls when looking for indicators – Using raw counts (citations, mentions, views etc.)

* OpenAlex, the successor of Microsoft Academic tracking citations as well, is not addressed because the web-interface is not (yet) available and information is only accessible through an API

The number of citations that a publication receives depends on its age (publication year), its subject area, and the document type (book, article, proceeding paper, review). To interpret this number in a meaningful way, you use a benchmark that considers these differences, called the expected or average number of citations typical for papers of the same age, of the same document type, and belonging to same subject area.

The normalized citation impact of a publication is not available in the Web of Science as such, but in InCites (build on top of the Web of Science) to which the UM subscribes. In InCites, the normalized citation impact indicator is called the Category Normalized Citation Impact (CNCI). In Scopus (and SciVal build on top of Scopus – to which Maastricht University does not subscribe, the normalized citation impact is called the Field-Weighted Citation Impact (FWCI). Although Maastricht University does not subscribe to SciVal, Scopus displays the FWCI for individual publications as well, including for books and chapters when indexed.

Another citation-based indicator that normalizes for the differences between publications is the percentile (in subject area).

For further guidance on how to retrieve normalized citation indicators and how to interpret these:

• Normalized citation indicators and percentile based indicators
• Selecting a publication set in InCites
• Acquire normalized citation indicators from InCites

Altmetrics for journal articles are available through platforms such as:

• Altmetric bookmarklet
• PlumX (via Pure or Scopus)
• Altmetric Explorer (via UL databases)
• Journal website
• Lens profile, and
• (some) other platforms where article is uploaded

Further guidance:

• Altmetrics (alternative metrics) subject guide
• Citations / mentions of special interest (book, chapters, articles and datasets) – Summary
• Responsible metrics – Pitfalls when looking for indicators – Using raw counts (citations, mentions, views etc.)
• Responsible metrics – Why not the Altmetric Attention Score or Research Interest score?

Article views and downloads are available through:

• Web of Science (views only)
• Scopus (views only)
• PlumX (via Pure or Scopus)
• journal websites (some), and
• other platforms (some) where article is uploaded.

Although views and downloads have major drawbacks as indicators, these usage indicators might be the only ones available for publications in the professional or popular press. As these might be important to address in your narrative – pointing out that your research is also picked-up by the professional or the public in general – you might check if articles are shared, viewed, or downloaded. Some platforms display this information.

However, be cautious when using usage data, and only use them when they are exceptionally high which is often difficult to judge.

To check if and what comments or reviews your article has received, check:

• PubPeer (The online Journal Club): search for your paper by its title or DOI on the platform, or install the browser plugin, which will show a box from PubPeer when you are on the article’s webpage and when there are comments on the specific paper.
• F1000Research (when paper is submitted) for reviews of others or comments on your work.
• Faculty Opinions (for biomedical research)