Animal pilot studies should not be used to estimate sample size if effect size and population variance are unknown

Reducing the number of animals required for a given experiment is part of the 3Rs strategies for animal welfare. Sample size estimation is a critical step in efficient and ethical experimental design. It is generally believed that pilot studies can be used to estimate sample sizes, which could lead to an overall reduction in the number of animals used. As part of the standard approach to ensuring that a planned animal experiment has sufficient statistical power, estimates of effect size and population variance are required. Here we derive the distribution of the sample size estimator when both effect size and variance are unknown. We show that, in this case, it is not feasible to conduct a preliminary pilot study to estimate the required sample size. Our analysis indicates that the sample size of a useful pilot study will often be much larger than that of the main study itself when the effect size is unknown. Therefore, we conclude that performing pilot studies with the aim of estimating sample size will not help to minimize the overall number of animal experiments in basic or pre-clinical research. A practical example is given, and alternative approaches are proposed and discussed. When conducting an experiment, researchers attempt to achieve a high probability of getting conclusive data regarding a treatment effect, i.e., to have high statistical power. This power depends on, among other things, the unknown true effect size of a treatment, the unknown population variability, and the sample size. In animal experiments, ethics require using the smallest sample size necessary to produce adequate power. Estimating this sample size requires assumptions about the unknown true effect size and the population variability. We show mathematically that these assumptions are poorly constrained by data from preliminary pilot studies. We conclude that additional pilot studies should not be conducted as part of the estimation process as the data they provide cannot lead to a reduction in the total number of animal test subjects. eng

• Bird, A. D.
• Jedlicka, P.
• Wilhelm, J.