Research strategy
Bibliometrics and Research-Impact Analysis for Institutional Decisions
Make publication and citation evidence useful for the decision in front of you. Bibliometrics can help describe a research portfolio, examine collaboration patterns and investigate changes over time. Outlier Science can support a defined analysis with clear source choices, documented methods and an explanation of what the findings can reasonably tell you.
The service can stand alone or contribute to a wider institutional assessment. The question comes first: comparing a research theme, understanding a department or examining collaboration requires different data and interpretation.
On this page 7 sections
01
Define the question before selecting the indicator
A useful analysis starts with its purpose, population and time period. Leadership may want to understand where a research theme is developing, whether collaboration is broadening, or how a portfolio compares with an appropriate peer group. Those questions should determine which records and measures are relevant.
Publication volume can describe activity but does not explain the quality of each contribution. Citation patterns can inform discussion of scholarly attention, while broader impact may require evidence of use outside academic publishing. Keeping these questions separate prevents one available number from becoming an answer to everything.
The scope can include publications, citation distributions, collaboration, open-access status where reliable information exists, subject patterns and selected non-publication outputs. The analysis should explain both the inclusion criteria and the evidence that falls outside them. CoARA’s commitments support assessment that combines contextual judgement with responsible indicators.
02
Prepare records that can support a defensible comparison
Source selection needs to reflect the subject, output types and intended analysis. Web of Science and Scopus can be relevant sources where access and licensing permit. Public and institutional records may contribute additional evidence, but their fields and coverage should not be assumed to be interchangeable.
Preparation can address duplicate records, author identifiers, affiliation variants, organisational changes and missing fields. A documented inclusion rule is especially important when the institution has campuses, affiliated hospitals or researchers with more than one affiliation. Different rules can produce different totals without either total being an arithmetic error.
The analysis plan should record the retrieval date, publication window, citation window, document types and counting approach. If several sources are combined, the matching rules and unresolved records need to be visible. Outputs derived from incomplete or uncertain data should retain that qualification.
Where the principal problem is maintaining reliable records over time, research information management addresses the underlying data process.
03
Interpret differences instead of presenting a league table
A comparison needs peers that make sense for the question. The choice may depend on discipline, institution size, research mission or maturity of the portfolio. A convenient set of well-known universities is not automatically a relevant benchmark.
The report can distinguish totals from distributions and explain why an aggregate changes. A small number of frequently cited outputs may influence an average; recent publications may have had little time to receive citations; a change in subject mix may affect a comparison. The analysis should make such effects assessable rather than hiding them behind a single score.
Field-adjusted measures can be considered when suitable source data and expertise are available. Their method and limits still need explanation. The label “normalised” does not remove every difference between disciplines, time periods or research populations.
For wider impact, the scope can identify additional evidence such as documented use in policy, professional practice or research infrastructure. Claims about resulting change need appropriate support and should distinguish contribution from sole attribution. Detailed planning for that evidence belongs in research impact and dissemination support.
04
Illustrative deliverable: an analysis specification
The following example shows a reviewable specification for a collaboration analysis. It describes a method, not actual institutional results.
Specification item | Illustrative decision to document |
|---|---|
Question | How has collaboration within a defined research theme changed? |
Population | Agreed institutional units and included output types |
Period | A defined publication window and a fixed retrieval date |
Partner identification | Reviewed organisation names and affiliation-matching rules |
Counting | Stated treatment of outputs with multiple institutions |
Interpretation | Distinguish new links, continuing links and isolated co-authorship |
Limitations | Missing affiliations, source coverage and uncertain matches |
A corresponding report could include a methods note, an annotated partner table and a concise interpretation. An illustrative finding of more co-authored papers would not, by itself, establish a more durable institutional partnership. The next question would concern repeated collaboration, substantive roles and relevant project evidence.
This approach makes the work reproducible enough for a later review. If the question changes, the institution can see which parts of the method need to change with it.
05
Outputs, process and practical scope
The engagement can proceed through question definition, data feasibility, an agreed analysis specification, record preparation, analysis and interpretation. A review of preliminary findings allows the institution to identify incorrect affiliations or missing context before the report is finalised.
Outputs can include cleaned working tables within permitted usage, a data dictionary, selected charts, a methodology statement, limitations and a management summary. The exact formats and any analysis files should be specified in the agreement, including restrictions imposed by source licences.
Scope and cost depend on record volume, matching complexity, the number of comparison groups and the depth of interpretation. A defined question using prepared data requires different work from rebuilding institutional affiliations across several sources. Specialist indicators, network analysis or repeated reporting should be scoped explicitly.
The service does not guarantee higher citations, ranking movement or a favourable comparison. It also does not convert a journal-level metric into a valid quality judgement about every article or researcher. The purpose is to make evidence clearer and decisions better informed.
06
Questions about bibliometric analysis
Can you analyse our institution using its existing exports?
Yes, subject to the records containing the required fields and being shared under appropriate permissions. An initial feasibility review can identify what is possible and what additional information would be needed. An export is not assumed complete simply because it came from a recognised database.
Can Web of Science and Scopus totals be combined?
They can be reconciled for a defined purpose, but simple addition would risk double counting. A combined dataset needs matching rules, source labels and a clear explanation of the population. Some comparisons are clearer when the sources are reported separately.
Can the report identify potential research partners?
Collaboration patterns can help identify organisations for further investigation. A co-authorship record is evidence of a publication connection, not proof of availability, institutional commitment or suitability for a new project. Partner assessment needs additional work.
Do we need to buy a new analytics platform?
Not necessarily. The appropriate approach depends on the question, existing access and reporting frequency. A one-off analysis may be possible with authorised exports. Software selection should follow demonstrated requirements rather than become an automatic part of the service.
Can the analysis support individual assessment?
Only within an explicitly agreed and appropriately governed scope. Individual assessment requires particular care with attribution, discipline, career stage and purpose. Institutional planning evidence should not be repurposed as an automatic staff ranking.
07
Discuss your analysis question
Explain what decision the analysis needs to support, which institutional units are involved and which sources are available. The guide to research-performance audits can help determine whether a focused analysis or a broader diagnostic is the right fit.