Four questions determine the correct test in the overwhelming majority of studies. Work through them in order.
1. What type is your outcome variable?
- Continuous — height, blood pressure, score on a scale
- Categorical — yes/no, disease grade, treatment group
- Time-to-event — survival, time to relapse
- Count — number of admissions, number of events
2. How many groups are you comparing?
One, two, or more than two. This alone eliminates most candidates.
3. Are the groups independent or paired?
Independent means different people in each group. Paired means the same people measured twice, or matched pairs. Using an independent test on paired data throws away the pairing and loses power.
4. Are the assumptions met?
Normality where required, homogeneity of variance, independence of observations. Test them and report that you did — assumption checking is among the things reviewers most often find missing.
The common choices
Continuous outcome:
- Two independent groups → independent t-test, or Mann-Whitney U if non-normal
- Two paired measurements → paired t-test, or Wilcoxon signed-rank if non-normal
- Three or more independent groups → one-way ANOVA, or Kruskal-Wallis if non-normal
- Three or more paired measurements → repeated-measures ANOVA, or Friedman test
- Relationship between two continuous variables → Pearson correlation, or Spearman if non-normal
- Predicting a continuous outcome → linear regression
Categorical outcome:
- Two categorical variables → chi-square, or Fisher's exact where expected counts are small
- Paired categorical data → McNemar's test
- Predicting a binary outcome → logistic regression
Time-to-event: Kaplan-Meier with log-rank test for comparison; Cox proportional hazards for adjusted analysis.
Mistakes reviewers catch
- Multiple t-tests instead of ANOVA, inflating the false positive rate
- No correction for multiple comparisons where many tests were run
- Parametric tests on ordinal data such as Likert items treated as continuous without justification
- Ignoring clustering — patients within hospitals, students within schools
- Choosing the test after seeing which produced significance
Decide before you collect
The analysis plan should be written with the protocol. It protects you from the accusation of choosing an analysis to obtain a result, and it makes the methods section straightforward to write.
Our data analysis service includes design review before collection, which is when these decisions cost nothing to change.