How to Prepare Data for a Statistical Consultation

Write a short study brief covering the purpose, main question, design, population, recruitment or selection method, measurements and intended comparison or relationship. Identify the outcome that matters most and explain why. If the study is exploratory, say so. If a protocol or analysis plan already exists, include the relevant version and note any departures.

Describe the decisions you want the consultation to resolve. “Which model fits this design?” is more useful than “Please run all possible tests.” Other useful questions concern missing data, repeated observations, clustering, measurement construction or interpretation of uncertain findings. The scope should follow the scientific question rather than a preferred software command.

Tell the analyst whether data collection is planned, ongoing or complete. Advice before collection can influence measurement and sampling decisions that may be impossible to repair later. For an existing dataset, describe the original purpose and any limits on reuse. Research design support may be the appropriate starting point when the main uncertainty concerns the study itself.

Define the unit of observation in every file. A row might represent a person, visit, school, specimen, experimental run or transaction. Several rows can relate to one unit, and several files may describe different levels of the same study. The analysis needs those relationships; a large row count should not be mistaken for a large number of independent participants.

Identify the variables that connect records. Use project-specific identifiers where appropriate and document whether an identifier is unique within a file or only within a subgroup. Keep the linkage key to directly identifying information separately under the institution's approved arrangements. Removing names does not automatically make every dataset anonymous.

If data come from more than one source, explain how they were linked and which records failed to match. Describe timing differences and changes in definitions. A merged file can look complete while combining measurements collected under different conditions. The consultation package should preserve enough provenance for the analyst to decide which comparisons are defensible.

A dictionary should explain each variable, its type, units, permitted values, missing-value conventions and any derivation. The UK Data Service documentation guidance emphasises these variable-level meanings and relationships. Embedded labels can help, but review whether they actually explain the file to someone who did not collect the data.

The following dictionary is illustrative and uses no participant data:

Include the questionnaire, instrument manual or collection form where you have permission to share it. For a derived variable, give the formula or code and the original variables used. A label such as “final_score” is insufficient if nobody can reconstruct how it was calculated. If a definition remains uncertain, mark that uncertainty and identify who can investigate it.

Keep an unchanged copy of the original received or exported data. Create a working copy for preparation and record each transformation. A useful change record identifies the date, variable or records affected, original issue, action, reason and person responsible. Where practical, reproducible code is preferable to a sequence of unrecorded manual edits.

Check structure before changing values. Look for duplicated identifiers, mixed units, inconsistent date formats, text stored as numbers and category labels that differ only in spelling. These are questions to investigate rather than reasons to delete records automatically. A duplicate may be an accidental copy, a repeated observation or a legitimate second event.

Do not overwrite a surprising measurement because it looks inconvenient. Check it against the original source and document the outcome. If it is a confirmed entry error, correct it with a traceable record. If it is a valid extreme value, its analytical treatment requires a reasoned decision. If the source cannot resolve it, retain the uncertainty and discuss suitable handling with the analyst.

Prepare a small reconciliation note before handover. Record the number of files, the number of rows in each, the number of distinct study units where known, and any exclusions already made. Explain why totals differ from recruitment or collection records. For example, a participant may appear in the recruitment log but have no eligible measurements in the analysis file. This should be described rather than hidden by changing the recruitment count.

Check that labels and units in earlier tables agree with the current dictionary. If an outcome was converted from one unit to another, record the conversion and whether the manuscript already reflects it. These simple checks help the consultation begin with a coherent account of the material. They do not establish analytical validity, but they can expose version and transcription problems before more complex work begins.

Distinguish values that were not collected, not applicable, unavailable, invalid or withheld where the study records allow it. These categories can have different meanings. A blank cell, zero and a code such as 999 should not be treated as interchangeable. The dictionary should make each convention explicit so that software does not interpret a missing-value code as a genuine measurement.

Summarise missing information by variable and, where relevant, by group or time point. State known reasons and whether they were recorded systematically. Do not invent reasons from the pattern alone. The purpose of preparation is to make the pattern visible; the choice of analytical approach depends on design, assumptions and the question being addressed.

Avoid deleting all incomplete rows or filling gaps with averages before the consultation unless an agreed plan already specifies that approach. Such actions can change the analysis and may be inappropriate. Provide the original structure and any earlier prepared versions, explain what was done and let the consultation determine which version and assumptions should support the analysis.

For repeated measurements, identify the participant, occasion and relevant timing. For clustered studies, identify the school, clinic, team, site or other grouping that matters to the design. For experimental data, distinguish biological units, technical repeats and batches. The exact terminology depends on the study, but the relationships must be understandable.

Consider an illustrative study with 30 participants measured at three planned visits. A file containing 90 rows does not describe 90 different participants. If some visits are missing, the observed row count may be lower. The study brief should state the planned design, while the data record what was actually observed. These synthetic figures explain file structure and are not a sample-size recommendation.

Do not reshape the data merely to imitate an online example. A long format can be appropriate for one task, while a wide format may serve another. Provide a readable description and preserve the relationships. The analyst can recommend a structure compatible with the agreed analysis, along with code or instructions for reproducing the transformation where included in scope.

Include earlier analyses, code, output and drafts relevant to the question, even if the results were disappointing or inconsistent. Describe which analyses were planned and which were added after looking at the data. This helps avoid unintentional duplication and makes interpretation more transparent. Concealing earlier choices can make a new analysis appear more independent or confirmatory than it is.

State any existing interpretation you want reviewed, but do not make a desired result the acceptance condition. The American Statistical Association's statement on p-values explains that a p-value alone does not establish effect size or scientific importance. A consultation should address the research question, uncertainty and assumptions rather than seek a preferred significance threshold.

If a reviewer requested new analysis, supply the exact relevant comments, decision letter and submitted version. If an institution or funder requires a particular output, provide the actual requirement. The analyst should be able to distinguish a scientific need from a formatting request or an external expectation that may require clarification.

Before transferring files, confirm that you are authorised to share them for the proposed work and that the agreed route is appropriate. Begin an inquiry with a description of the project and the kinds of data involved. Do not place participant records, credentials or restricted material in an ordinary contact form. Confidentiality information identifies the arrangements to discuss before access is granted.

Prepare a file manifest: file name, version, purpose, format, approximate size and relationship to other files. Distinguish the raw export, working dataset, dictionary, protocol, change log and previous analysis. Remove unrelated personal information and unnecessary files. A small, coherent package is easier to review than an unlabelled archive containing several conflicting “final” versions.

If access must occur within an institutional environment, describe that constraint at scoping. Availability of software, permitted exports and review access can affect the work. Do not assume a consultant can use a particular platform or receive a complete dataset. The delivery arrangement should be agreed before the analysis plan depends on it.

Create a short cover note identifying the project, research question, package version and main contact for factual questions. List unresolved issues explicitly. For example: “The definition of two missing-value codes is being checked with the collection team; no recoding has been performed.” This is an illustrative note. It is more useful than describing the dataset as fully clean when an important uncertainty remains.

Identify the point at which a data version is accepted for a particular analysis. If collection continues or corrections arrive later, record how updates will be assessed and whether re-analysis is included. An analyst should not unknowingly receive a changed file with the same name. Changes affecting results need a traceable explanation across the code, tables and manuscript.

Request outputs that match the project's needs: an analysis plan, documented preparation, annotated results, reproducible code, figures or a methods description. Each requires a defined scope and suitable expertise. The statistical consulting service explains possible support. A useful first discussion begins with the question, the design and the state of the records, allowing the analytical work to be planned on an honest foundation.