Introduction to sample size calculation

You first collect data and process them before analysing numbers . The following steps are calculating descriptive statistics and if applicable also inferential statistics. Most research projects also involve inferential statistics.

If your project only includes descriptive statistics focus your sample size calculation on that. Examples of focusing on descriptive statistics: To confirm a 5% prevalence of a condition with a margin of error of 3% (2-8%) would require 377 observations.

However, if your project includes some inferential statistics focus your sample size calculation on that. In most projects this would have higher priority than doing a sample size calculation for descriptive statistics.

The statistical calculations (=inferential statistics) looks at your data and produces results such as p-values, odds ratios, hazard ratios, etc. Is the reason for not reaching statistical significance that there are no correlation / no difference between groups or is the reason that your sample size was too small. To avoid ending up with the latter problem it is recommended to use a special software and do the statistical calculation backwards using some assumptions. It has gradually become more common that ethics committees require a sample size estimation before approving a project.

Different approaches to sample size estimation

• Get a convenient sample and hope it is enough
• See how many observations other published projects included and imitate them
• Follow a rule of thumb
• Make a calculation based on your best assumptions.

Hope is good in many situations except this one. Imitate others is also not a good advice. What if the others did an underpowered study? Why replicate their mistake? There are some rules of thumb such as:

• For group comparisons of means (t-test) have at least 30 in each group.
• For group comparisons of proportions (chi-square) have at least 5 in each cell.
• For standard linegressions/correlations have at least 20 observations for each independent variable.
• For logistic regression have have at least 10 times more events / end points than independent variables .
• For Cox regression have at least 10 times more events / end points than independent variables . For example: you have four independent predictor variables in the model and the proportion of positive cases in the population is expected to be 0.30 (30%) the minimum number of cases required would be 133.

However, these rule of thumb are quite rudimentary because they do not consider the magnitude of the effect size or correlation you are looking for. They just give the bare minimum number you should have to avoid violating underlying mathematical assumptions but they do not consider your particular situation. The best approach to estimate the size of the sample is to do a proper sample size calculation considering the situation in your study. This is done by first making four important decisions:

1. Decide what statistical method is going to be used for the inferential statistics.
2. Decide what effect size / correlation you are looking for. It is best if this can be estimated using data from previous publications. You have to make a qualified guess if no prior publications exists.
3. Decide what would be an acceptable safety margin to avoid doing a type one error (claiming a statistical finding that is not true). This safety margin is labelled alpha or level of significance and is commonly set to 0.05. This means that you have a one in twenty chance of doing a type one error.
4. Decide what power your study should have. This is the same as the inverse of the risk of doing a type two error (not identifying an effect/correlation that is true). The power is often set to something between 0.80-0.95 which corresponds to a 5-20% chance of doing a type two error.

The rest is quite easy once we have made these four decisions. We put in our decisions in a software that does the statistical calculation backwards and states how large sample we need. Example of such software are G*Power and PASS. G*Power is free but PASS is quite expensive. G*Power can manage most situations except Cox regression.

Different scenarios

• Sample size for a single proportion (such as prevalence or incidence)
• Sample size for group comparisons
• Sample size for regression analysis

(This section is under construction)

Examples of sample size calculations

Click to expand and watch a video explaining further:

Examples using the software G*Power

Example 1 of sample size calculation for comparing two groups – T-test and Mann-Whitneys test

Example 2 of sample size calculation for comparing two groups – T-test and Mann-Whitneys test

Example of sample size calculation for unconditional binary Logistic regression when the independent variable is binary (such as gender)

Examples using the software PASS

Example of sample size calculation for Cox regression

Sample size calculation in multivariable regression

You may plan for a multivariable regression as your preferred final statistical analysis. There are a few approaches to this situation:

1. Make one sample size calculation for each independent variable as if you are going to do simple (unadjusted) regressions. You will get one sample size for each independent variable. Pick the one with the highest number as your preferred sample size (and perhaps add a margin of 20% extra). This is the most common strategy and the one used in the videos above.
2. In case you are only interested in one independent variable and want to add a few more only to adjust for them (as confounding variables) try to estimate the contribution from the covariates (R square other X in G*Power) and add it in G*Power together with the expected information around your main independent variable to calculate the sample size required. Finding the right value on the “R square other X” is tricky and might be impossible. Either make a reasonable guess or go with strategy 1 above.
3. There may be many independent variables in an exploratory study and none are initially more important than another. The simplest solutions is to use strategy 1 above. It may be difficult to sort out how the variables may relate in a multivariable model without making a lot of guesses.
4. Calculating sample size for interaction variables in a regression is tricky for two reasons. Firstly, it is often difficult to find support for the assumptions you need to make so you may be left with some wild guessing. Secondly, you would need more advanced software than G*Power and a statistician who has experience of this advanced calculation (not all statisticians would have that).

Level of significance (alpha) versus p-value

A low p-value says it is unlikely that we would get the observed observations if the effect / correlation we’re looking for in reality is zero. A low P value indicates that the null hypothesis can be rejected and the alternative hypothesis is the most likely. How low must the p-value be for us to believe that our alternative hypothesis is the most plausible? This should be determined from case to case. Read more about this on the page describing the level of significance or alpha.

Sample size estimation in clustered studies

Observations are often grouped (clustered). A typical example can be that observations are clustered in different primary health care centres (GP clinics) or different hospitals. These clusters will add a random variation between clusters that makes your vision slightly blurred. It means that you must increase your sample size to maintain your ability to find what you are looking for. It can be shown that it is better to have many clusters contributing with a few observations compared to having a few clusters contributing with many observations. To estimate this calculate the required sample size as if there was no cluster effect. After that use the calculator below to estimate the effect on required sample size different cluster designs will have.

The impact of clusters is measured with Intra Class Correlation (ICC). You need to find a suitable assumption for ICC to put in below. The ideal situation is if you find a publication with a study similar to yours stating the ICC. If that is the case use that. Otherwise make a reasonable guess to estimate ICC. In a hospital setting common values if ICC are 0.02-0.1 . In a primary care setting common estimates of ICC are 0.1-0.2 although estimates up to 0.3 may occasionally be seen .

Examples of how to write

Below are examples of how to write the sample size section in a study protocol. You need to click to expand and read.

Useful links

• ClinCalc.com
• Calculator for confidence interval for proportions
• Power and Sample Size Primer (a video)
• Sample size calculator for non-inferiority studies if the variable is binary
• Sample size calculator for non-inferiority studies if the variable is continuous
• Tips around using G*Power
• Calculating sample size for survival analysis (such as Cox regression)
• Power and sample size calculators from HyLown Consulting LLC
• PS: Power and Sample Size calculator from Vanderbilt University in the US

Referenser