Study authors
Operator
Design
Recall rate (%)
Sample size
Unit of measure
Assessment of success
Radiographic criteria success
≥4 years after treatment
Calibration
Reliability test
Statistical analysis
Grahnen
UG
R
64
502
Ro
C&R
S
✓
–
–
–
Engstrom
UG
R
72
153
T
C&R
L
✓
–
–
X^{2}
Selden
Sp
R
20
52
T
C&R
L
–
–
–
X^{2}
Bergneholtz
UG
R
66
556
Ro
C&R
S
–
–
✓
–
Pekruhn
Sp
R
81
36
T
C&R
S
–
–
–
X^{2}
Molven
UG
R
50
226
Ro
Ra
S
✓
✓
✓
X^{2}
Allen
–
R
53
315
T
C&R
S
–
✓
–
X^{2}
Sjogren
UG
R
46
267
Ro
C&R
S
✓
✓
✓
LR
Van Nieuwenhuysen
–
R
–
612
Ro
C&R
S
–
✓
✓
X^{2}
Friedman
Sp
C
78
128
T
C&R
S
–
–
–
X^{2}
Danin
Sp
RCT
100
18
T
Ra
L
–
✓
✓
X^{2}
Sundqvist
UG
C
93
50
T
C&R
S
✓
–
–
X^{2}
Chugal
PG
R
75
85
Ro
Ra
S
✓
–
–
LR
Hoskinson
Sp
R
78
76
Ro
C&R
S
✓
✓
✓
GEE
Farzaneh
Sp
C
22
103
T
C&R
S
–
✓
✓
LR
Gorni
PG
C
94
452
T
C&R
S
✓
✓
✓
M–W
Çaliskan
Sp
R
96
86
T
C&R
S
–
✓
✓
X^{2}
In addition to the methodological characteristics, most systematic reviews present the original results in descriptive forms using tables and graphs. Table 3.2 combines study characteristics and results for our systematic review describing 12 studies that tested the effect of our new antiviral therapy. The table presents the year of publication, total sample size, source of funding, sample size in each experimental group (n), estimate of the intervention effect (mean), an estimate of the intervention variability (standard deviation – SD), and the pvalue.
Table 3.2
Description of study characteristics and original results
Year of publication

Sample size

Source of funding

Treatment

Control

pvalue



n

Mean

SD

n

Mean

SD


1997

42

Private

21

2.0

1.7

21

3.9

2.1

0.003

1998

31

Private

16

1.8

2.4

15

3.8

2.7

0.04

1998

44

Private

22

2.1

2.6

22

3.5

2.8

0.09

1999

33

Public

18

3.3

2.7

15

1.8

2.4

0.11

2001

30

Private

14

3

3.2

16

2.1

2.9

0.43

2001

29

Private

13

2.1

2.9

16

3.2

3.2

0.35

2002

27

Public

13

2.9

2.9

14

2.5

2.5

0.70

2002

31

Private

15

2.5

2.5

16

3

2.9

0.61

2005

190

Public

96

2.9

2.1

94

3.1

2.5

0.55

2007

80

Public

39

3.5

2.6

41

3.1

2.2

0.46

2007

145

Public

73

3.2

2.4

72

2.9

1.9

0.41

2008

394

Public

198

2.8

1.7

196

3.1

1.8

0.09

The overall trend of the studies is used to suggest if a given intervention is better than the standard treatment or no intervention when only descriptive tables are used to present results. This approach is very intuitive and does not need any statistical expertise to be conducted. However, as you will later see in this chapter, it can be misleading for several reasons. An overall assessment of Table 3.2 indicates that between 1997 and 2002, mostly small studies were conducted, with large studies being published only in the last 4 years. This finding is consistent with most new therapy studies since large, costly, timeconsuming studies are only conducted after some evidence of positive effect and safety is available. A closer look at the results of the studies shows that within small studies the results are very inconsistent, with few studies showing large positive or negative effects for the therapy when compared to placebo. In contrast, large studies do not show major differences between experimental groups. As expected, variability is larger in smaller studies due to the sample size effect on standard deviation estimates. Only the first two studies reached statistical significance, and in three other studies somewhat borderline results were found (p ∼ 0.10).
3.2 Main Results: Overall Estimates of Effect
The treatment effect could be estimated by calculating an overall mean of the results simply by summing up the individual results dividing by the number of studies. This approach, although very intuitive, would not take into consideration the studies characteristics, with studies contributing equally to the overall estimate. Looking at the estimates in Table 3.2, it is obvious that some studies have more precise estimates than others. Factors that may affect the precision of the estimates are various, including sample characteristics, sample size, measurement precision, and reliability. In the metaanalysis framework, sample size is often the most important factor to be taken into consideration. Thus, overall estimates should take into account the sample size with larger studies contributing more than small studies. Mathematically, this can be accomplished by multiplying each study estimate by the sample size or, in other words, by weighting the estimates according to sample size. The sum of the estimates can then be divided by the total sample size. Table 3.3 shows the weight of each study of our example according to sample size. Using this approach, the mean reduction in days with pain would be 2.8 days for the treatment group and 3.1 days for the control group. Thus, the placebo treatment reduced in approximately 0.3 days patients’ symptoms.
Table 3.3
Study weights according to sample size and inversevariance methods
Year of publication

Weight based on sample size (%)

Weight based on inversevariance method (%)


1997

3.9

4.6

1998

2.9

1.9

1998

4.1

2.4

1999

3.1

2.0

2001

2.8

1.3

2001

2.7

1.2

2002

2.5

1.5

2002

2.9

1.7

2005

17.7

14.2

2007

7.4

5.5

2007

13.5

12.4

2008

36.6

51.3

Total

100

100

In essence, this is what is done in a metaanalysis to take into consideration the contribution of each study. A similar strategy can be used to account not only for the sample size, but also for the variability in the estimates of the original studies. The overall weighted estimate is calculated multiplying each study estimate by the inverse of the square of the standard error (inversevariance weighting method), which is highly associated with the sample size of the study. Using this approach, the weighted mean difference (WMD) between treatments is 0.26 mm in favor of the new antiviral therapy. Table 3.3 also shows the weight attributed to each study according to the inverse variance method. It is clear that the study published in 2008 dominates the overall estimate not only because it has the largest sample size, but also because it has the greatest precision (smaller standard deviations and confidence intervals). Studies with lower variability receive greater weight and therefore have greater influence in the estimate.
Tables and graphs are popular ways of presenting the results of a metaanalysis. Table 3.4 presents the WMD and the 95% confidence interval for each study. The weighted mean provides an estimate and direction of the effect, and the confidence interval provides an assessment of the variability of the estimates. Confidence intervals also indicate the significance of the results and when it does not include zero (or 1 when the results are presented in odds ratio), the weighted mean is statistically significant.