Sample size & power calculations

Aim

To calculate sample size and/or power when applicable.

Requirements

Documentation of sample size and/or power and how these are calculated.

A reader has to be able to reproduce your calculations

Documentation

The following variables are required for calculating sample size:

  • Significance (α)
  • Power (b)
  • Relevant effect
  • Standard deviation (SD) of individual changes
  • Outcome of sample size calculation.
  • The assumptions made in your calculations.
Responsibilities

Executing researcher: 

  • To make sure sample size and/or power are calculated.

Project leaders: 

  • To calculate the sample size and/or power for the study, potentially in consultation with a statistician.

Research assistant: N.a.

Why

Sample size calculations are usually performed to determine the number of participants needed to detect a clinically relevant treatment effect. Pre-study calculation of the required sample size is warranted in the majority of quantitative studies. Also in objective research sample size calculations or power calculations can be of use, to determine how many cases or controls you need or how large the groups must be to detect meaningful differences.

Calculating the sample size in the design stage of the study is increasingly becoming a requirement for grant applications and when seeking ethical committee approval for a research project.

How To

Most importantly is to contact our biostatisticians for assistance. See ‘Inleiding in de toegepaste biostatistiek’ of Prof. dr. Jos Twisk for more information.

There are numerous formulas for calculating sample size and/or power [Cohen, 1998]. The differences correspond to differences in study design. From a statistical point of view, a lot of power calculations should come with critical notes. This is due to the fact that it is often difficult to provide true estimates of the parameters required for each formula. On the other hand, a sample size calculation is required for most grant applications to allow people to complete the necessary parameter values one way or the other. Here you can find an overview of calculation methods.

The calculation of sample size requires several components:

  • the type 1 error (alpha) / false positive results: the probability of falsely rejecting H0 and detecting a statistically significant difference when the groups in reality are not different.
  • the type 2 error (beta) / false negative results: the probability of falsely accepting H0 and not detecting a statistically significant difference when a specified difference between the groups in reality exists.
  • power (1-beta): the probability of correctly rejecting H0 and detecting a statistically significant difference when a specified difference between the groups in reality exists.
  • the smallest effect of interest: the minimal difference between the groups that is considered biologically plausible and clinically relevant.
  • the variability / variance: the variability of the outcome measure.

The null hypothesis (H0) hypothesizes that the groups that are being compared are not different. The alternative hypothesis (Ha or H1) hypothesizes that the groups are different.

It should be noted that the routine of null hypothesis significance testing is increasingly criticized. See De Boer et al for a gentle introduction.