17 Aug ESS IA Statistical Tests: Choose, Check and Interpret
Choose ESS IA analysis from the research question, variable types and sampling design. Start by plotting and summarising the observations. A formal statistical test can be useful, but no single test, significance threshold or advanced method is compulsory for every investigation.
Start with the question and independent sampling units
Are you comparing groups, investigating a relationship or examining categorical frequencies? Identify what each row represents and how observations were obtained. Repeated readings of one sample do not automatically create independent replicates.
For example, ten quadrats within one grazed field and ten within one ungrazed field are not ten independently replicated grazing treatments. Field-specific differences may explain the pattern. Analyse the evidence at the scale the design supports and acknowledge this limitation instead of treating a test result as proof of a grazing effect.
Identify the variables
- Nominal: Unordered categories, such as land-use type or presence/absence. Summarise counts or proportions.
- Ordinal: Ordered categories without assured equal gaps, such as one questionnaire rating item.
- Numerical: Measurements or counts. Consider their units, range, distribution and any bounds, such as percentages between 0 and 100.
Paired observations have a meaningful link, such as the same independently selected sites measured before and after an event. Keep those links in the analysis. Simply taking measurements in different places does not establish independence if sites are clustered or share the same treatment unit.
Check assumptions for the actual method
There is no rule that every sample below 30 must use a nonparametric test. Relevant considerations include design, distribution, extreme observations, sample size and what you want to estimate. Nonparametric does not mean assumption-free.
Inspect plots and consider the data-generation process. A normality-test p-value above 0.05 does not prove a normal distribution, and a small p-value alone does not determine which alternative method is suitable. For a paired t-test, normality concerns the within-pair differences; for a linear model, assess the relevant residual assumptions. Do not automatically replace Shapiro–Wilk with Kolmogorov–Smirnov at an arbitrary sample-size threshold.
Common methods to consider
| Purpose | Possible method | Important qualification |
|---|---|---|
| Compare two independent means | Welch’s t-test or an appropriate alternative | Requires independent units and suitable distributional behaviour; Welch’s method does not assume equal group variances. |
| Compare two related measurements | Paired t-test or a suitable paired alternative | Preserve pairing; check assumptions for the differences. |
| Compare several independent groups | ANOVA, a variance-robust method or a justified rank-based method | Account for the design and planned follow-up comparisons; one significant overall test does not identify every differing pair. |
| Compare ranked independent observations | Mann–Whitney or Kruskal–Wallis, when appropriate | These concern distributions/ranks. Interpreting a difference as purely a median shift requires additional shape assumptions. |
| Examine categorical association | Chi-square test, or a suitable exact method for sparse data | Use counts, independent units and appropriate expected-frequency conditions; percentages alone are not input counts. |
| Examine a relationship | Pearson or Spearman correlation, or a justified regression model | Pearson describes linear association; Spearman describes monotonic association. A curved relationship is not necessarily monotonic. |
Use a method you can explain and justify. Numerical variables do not automatically meet the assumptions of every numerical test. A transformation changes interpretation and needs justification; it is not a universal cure for skewed data or unsuitable sampling.
Use software transparently
A spreadsheet or statistical package can calculate results, but cannot decide whether the design makes them meaningful. Record software, variables, missing-value handling, test settings and any exclusions. Retain the original data and a clear processing trail.
Excel includes CHISQ.TEST, which returns a p-value from observed and expected counts. It is incorrect to say Excel has no native chi-square function. R and JASP offer other methods; use their current documentation rather than assuming a menu click validates the analysis.
Report the result and its meaning
- Explain why the method suits the question, variable types and sampling structure.
- Give relevant sample sizes, summaries and units, and clarify what counts as an independent observation.
- For a formal test, report its name, statistic, degrees of freedom where applicable and p-value with enough precision. Never report a rounded value as p = 0.
- Include an appropriate effect estimate and uncertainty where useful, and explain their environmental meaning.
- Answer the research question and acknowledge confounding, sampling limits and assumptions.
A reporting template is: “Using [method], the estimated difference/association was [value and units or appropriate scale], with [uncertainty]. The result is consistent with [careful interpretation], subject to [specific limitation].” Replace placeholders with your own verified output; do not copy invented numerical results.
Interpret p-values, effect size and multiple tests together
A p-value describes how incompatible a test statistic is with a specified null model and its assumptions. It is not the probability that the null hypothesis is true, that the result occurred “by chance,” or that an effect is environmentally important. Failure to reject a null hypothesis is not proof of no effect.
Effect estimates and their uncertainty help assess magnitude. Labels such as small, medium or large are conventions, not universal ecological thresholds, and estimated effect sizes still vary between samples. Use the actual scale and context rather than relying only on a standardised label.
Plan analyses where possible. Running many tests and reporting only the smallest p-value increases the risk of a misleading conclusion. Use an appropriate comparison strategy, report relevant tests transparently and label analyses suggested by the observed data as exploratory. More measurements do not repair a lack of independent treatment replication.
Keep the main reasoning in the report
Show the key data treatment, assumption considerations, results and interpretation in the assessed report. Supplementary output may be stored separately where permitted, but an appendix must not carry essential reasoning that is absent from the main text. A screenshot alone does not explain a method.
Discuss your analysis choices
Bring your research question, a small example of your own data and an explanation of the sampling units. Marija Maletz is an ESS teacher, examiner and tutor, with 13 years of teaching experience and more than 10 years as an IB ESS examiner. Guidance can help you understand choices; your calculations, interpretation and assessed writing remain your own.
Request a FREE 20-minute trial lesson to discuss needs and a suitable starting point. A complete analysis or ready-to-submit results paragraph is not promised in the introduction.
No Comments