In order to increase the power of a test it may be possible to examine many similar studies and combine their findings into an overall analysis, thereby increasing the effective sample size. The advantages of this approach are obvious, in that existing published and peer-reviewed research from separate studies around the world can be combined to strengthen the analytical process. Special statistical techniques have been developed to support meta-analysis [BOR1], and graphics such as the forest plot (see below) have been designed to clarify both the similarities and differences between the separate studies. The problems associated with the approach are also self-evident: different studies will inevitably have been produced under differing circumstances, on different groups of subjects, with varying levels and types of control; and only published studies tend to be compared, so there is a (significant) risk that a bias towards studies that show results, rather than no-result studies, are combined (see Goldacre, [GOL1]) for a lengthy discussion of this issue); likewise there is a risk of selection bias, whereby only studies that confirm a hypothesis are selected, again resulting in potentially serious distortion of the results.

The following forest plot was produced using the R function forestplot() in the rmeta meta-analysis package using data from the Cochrane Collaboration (www.cochrane.org). The data comes from 7 randomized trials before 1980 of corticosteroid therapy in premature labor and its effect on neonatal death (corticosteroids are given to women in early labor help the babies' lungs to mature and so reduce the number of babies who die or suffer breathing problems at birth). The plot shows the odds ratio (OR) and 95% confidence intervals in each study, together with an estimated combined (summary) odds ratio computed using the Mantel-Haenszel OR fixed effects method [MAN1]. This is simply a form of weighted version of the standard odds ratio ad/bc applied to each of k tables, where the sample sizes are the ni values (the fixed effects assumption involves regarding each study as evaluating the same overall treatment effect):

The overall OR in this case is 0.53, and the width of the diamond shows the confidence intervals for this estimate, which are [0.39,0.73]. Where the odds ratio is less than 1 (the vertical line shown) it indicates that the treatment (in this case the use of steroids) reduces the chances that the baby will die - in this example, by approximately 50% as a best estimate. Each line on the forest plot relates to a separate trial, and smaller trials are shown with longer horizontal lines to indicate that they are less certain of their results (the width of possible OR values is greater). As can be seen, several trials have horizontal lines that cross the OR=1 (no effect) line, so taken individually the impression is given that use of these steroids may well not be effective. However, by combining all the trials the beneficial effect of treatment becomes clear.

An overall random effects model (DSL) and summary is also available in the rmeta package, which in this case produces the same OR estimate of 0.53 but a slightly wider confidence interval: [0.37,0.78]. The random effects model, due to DerSimonian and Laird, assumes that there is random variation in treatment effects and estimates the mean and variance of the effect. The scale illustrated is logarithmic with limits set at [0.1,2.5] and the arrows indicate that the confidence intervals extend beyond these limits, i.e. the limits have been clipped or trimmed (software packages vary on the methods used to compute confidence intervals for combined studies). The box size illustrated is based on the study precision.

Forest Plot – Seven Randomized Controlled Trials – Cochrane dataset – Fixed effect (MH) model

The latest version of this Cochrane Review, which combines the findings from 21 studies, can be obtained from the Cochrane website http://www.cochrane.org .

Forest plots can be used in a variety of ways - for example, in the 2004 meta-analysis of European case-control studies into the levels of radon in homes and the associated risk of death from lung cancer [DAR1], the authors displayed the results from 13 studies with stratification for age, sex, smoking habits and cancer histology, with the horizontal scale indicating the percentage increase in the risk of lung cancer per 100Bq/m2 of measured Radon (i.e. not an OR plot in this instance). A subset of their plot is shown below:

Radon and Lung Cancer - percentage increase in risk per 100Bq/m2 of measured Radon

References

[BOR1] Borenstein M, Hedges L V, Higgins J P T ,Rothstein H R (2009) Introduction to Meta Analysis. John Wiley & Sons Ltd, Chichester, UK

[DAR1] Darby S and others (2004) Radon in homes and risk of lung cancer: collaborative analysis of individual data from 13 European case-control studies. BMJ, 330, 223-237. Available, with additional material, from: http://www.ctsu.ox.ac.uk/radonmethods

[GOL1] Goldacre B (2012) Bad Pharma: How drug companies mislead doctors and harm patients. Fourth Estate.

[MAN1] Mantel N, Haenszel W. (1959) Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748