What Does a Research-Performance Audit Include?

Write down the questions that leadership needs answered. Examples include whether a faculty needs more methodological support, which research themes have credible development potential, where project handovers break down, or whether publication reporting can be trusted. “Improve research performance” is too broad to determine an audit's scope.

State the unit of analysis. A university review may consider faculties and centres; a departmental diagnostic may need to understand individual project types and support arrangements. Define the historical period and the date to which records will be considered. If a new organisational structure replaced an old one, decide whether the audit will report historical departments, current departments or a documented mapping between them.

Agree what the audit will not decide. It may support research strategy without making individual promotion decisions. It may identify data-management gaps without reviewing every dataset. It may assess accreditation-readiness processes without conducting an accreditation assessment. These boundaries protect the usefulness of the work by preventing readers from treating a limited diagnostic as a complete evaluation of every activity.

An inventory should identify each available source, its owner, the period covered, the access route and known limitations. Relevant sources may include publication exports, project registers, funding records, policy documents, internal review forms, training records and interviews. The institution should confirm that it can lawfully provide the material and that any licence permits the intended use.

Separate “available” from “usable”. A publication spreadsheet may lack stable identifiers. A grant register may mix submitted proposals with expressions of interest. Workshop attendance may not show whether learning was applied. An audit should record these limitations early, because repairing them can be a meaningful part of the work rather than an invisible preliminary task.

Use a controlled request list. For each item, explain why it is needed and the minimum level of detail required. A process review may only require an anonymised example of a completed form. A management baseline may need departmental totals rather than individual personnel files. Agree a transfer route before requesting restricted records. The confidentiality page explains the questions to resolve when discussing access arrangements.

Before producing trends, inspect whether records refer to the intended institution. Similar names, former organisational structures, multiple campuses and joint appointments can complicate matching. Preserve the original data and maintain a record of corrections, exclusions and uncertain matches. An unresolved item should remain visible as an uncertainty rather than being silently assigned to the most convenient category.

Define counting rules. Will a publication co-authored by two departments count once at university level and once for each contributing department? Will each department receive a fraction? Either approach can answer a useful question, but combining the two without explanation makes totals difficult to interpret. Specify how duplicate records, early-online versions, document types and reporting years will be handled.

Distinguish collection coverage from institutional activity. A database export is a view of records within a selected source and search boundary. It is not automatically the complete body of the university's research. Use the Scopus and Web of Science planning guide when setting these boundaries, and document other relevant outputs separately where they matter to the institutional mission.

A useful profile combines evidence about research activity with the conditions in which it occurs. Consider the department's disciplines, staff composition, major facilities, doctoral activity, funding arrangements and development priorities. Compare like periods, and make any changes in definitions explicit. Small departments or recently formed units may need a narrative account alongside numerical summaries.

Keep the profile compact enough to discuss. A practical structure covers the department's research contribution, publication and funding patterns, collaborations, project pipeline, support needs and data limitations. Include a small number of questions requiring follow-up rather than adding every available metric. The profile should identify where expert judgement is needed, not imply that the chosen indicators settle all questions.

Journal-level measures require particular care. Clarivate states that the Journal Impact Factor is a journal measure and should not substitute for evaluating individual papers or researchers in its Journal Citation Reports guidance. An audit can describe the journals in which work appears while assessing the work's contribution through other relevant evidence.

Interview and document review should explain how research support actually operates. Trace a small number of representative projects from planning through relevant approvals, analysis, publication or reporting. Ask where decisions wait, what information is missing and who resolves disagreements. A policy can exist without being easy to find or consistently followed.

Select examples deliberately. Include different disciplines, project sizes, career stages and outcomes where the scope permits. A sample made only of successful projects may conceal recurring difficulties. A sample made only of complaints may exaggerate exceptional cases. Record how examples were selected and avoid presenting a small qualitative sample as a statistically representative picture of the entire institution.

Test important claims against another source when possible. If staff describe slow internal review, examine the relevant timestamps and definitions before deciding what “slow” means. If a form is said to be unnecessary, identify the decision or obligation it supports. Some obstacles require a simpler process; others require more capacity or a clear owner. Research-process standardisation addresses implementation once the actual gap has been understood.

Peer comparison can help leaders understand alternative structures or patterns, but a comparison group needs a rationale. Similar institutional names do not establish comparable missions, disciplinary mixes or resources. Document why each peer is included and which differences remain relevant. A narrowly selected comparison may be more informative than a large league table.

Use the same source boundary and counting method wherever possible. If peer data cover only selected publications while local data include a broader set of outputs, report the difference before interpreting the gap. Avoid attributing an observed difference to a single cause without evidence. Funding, staffing, publication practices and the age of a research area may all require examination.

When credible peer data are unavailable, the audit can still establish an internal baseline and identify improvement opportunities. It should say which questions cannot be answered with confidence. “No defensible comparison available” is more useful than an attractive chart assembled from inconsistent figures. The resulting recommendations should rely on the evidence that was actually obtained.

An audit finding needs a clear claim, supporting evidence, implications and a proposed response. Keep observations separate from interpretations. A missing field in a register is observable. The conclusion that nobody manages the underlying process requires further investigation. Assign a confidence description appropriate to the evidence rather than giving every finding equal authority.

The following is an illustrative extract, not evidence about any institution:

Each action needs an institutional owner, required resources, dependencies and a review point. Some actions can proceed immediately; others depend on a policy decision, specialist expertise or improved data. An action plan should expose those dependencies rather than place every recommendation in the first month.

The commissioning brief should name the outputs. A substantial audit may include an evidence inventory, documented data methods, departmental profiles, process findings, an action register and a leadership summary. A shorter diagnostic may cover a narrower set. Decide whether editable tables, a presentation and a working session are needed, and whether implementation support is included or separately scoped.

Agree how factual corrections will be handled. Departments should have an opportunity to identify misattributed outputs, outdated descriptions or missing context. That process should not allow an unfavourable finding to disappear without a documented reason. Separate correction of facts from disagreement about interpretation, and retain the final rationale for consequential decisions.

Acceptance should focus on the agreed work: traceable evidence, understandable methods, complete outputs and practical recommendations. It should not depend on whether the audit produces a flattering result or a predetermined ranking. Ask how the provider will handle unavailable data, material scope changes and limitations that emerge during the work. Clear change control is more useful than an apparently fixed scope built on untested assumptions.

Scope also determines the effort required. A clean, consistently defined export can support analysis quite differently from several disconnected departmental spreadsheets. Interviews across multiple campuses, specialist disciplines, historical reorganisations and restricted records can add work. When comparing quotations, check whether data repair, departmental validation and management discussion are included. A lower price may describe a smaller diagnostic rather than a more efficient version of the same assignment.

Ask who maintains the baseline after handover. The final package should identify refreshable fields, definitions that must remain stable, and circumstances that require a new comparison. Without this ownership, a carefully prepared baseline can become difficult to update after the original analyst leaves.

Assign a person or group to turn approved findings into implementation decisions. Choose the first actions according to their importance, feasibility and dependencies. Some institutions need a data-quality foundation before setting performance targets. Others already have good evidence but need clearer ownership or access to specialist support. The audit's value depends partly on recognising that difference.

Maintain an action register that distinguishes proposed, approved, in progress and completed work. Completion should refer to an observable output, such as a tested referral route or an approved reporting definition. Schedule a review that asks whether the change is operating and whether unexpected effects have appeared. Do not assume that delivering a report changes the underlying process.

To prepare a discussion with Outlier Science, describe the decisions at stake, the organisational scope, the evidence available and any important constraints. The research-performance audit service sets out a possible engagement, while the self-assessment checklist can help organise initial questions. A focused brief makes it easier to distinguish a useful diagnostic from a larger implementation programme.