Understanding and Interpreting Systematic Review and Meta-Analysis Results

Study authors
Operator
Design
Recall rate (%)
Sample size
Unit of measure
Assessment of success
Radiographic criteria success
≥4 years after treatment
Calibration
Reliability test
Statistical analysis
Grahnen
UG
R
64
502
Ro
C&R
S
Engstrom
UG
R
72
153
T
C&R
L
X2
Selden
Sp
R
20
52
T
C&R
L
X2
Bergneholtz
UG
R
66
556
Ro
C&R
S
Pekruhn
Sp
R
81
36
T
C&R
S
X2
Molven
UG
R
50
226
Ro
Ra
S
X2
Allen
R
53
315
T
C&R
S
X2
Sjogren
UG
R
46
267
Ro
C&R
S
LR
Van Nieuwenhuysen
R
612
Ro
C&R
S
X2
Friedman
Sp
C
78
128
T
C&R
S
X2
Danin
Sp
RCT
100
18
T
Ra
L
X2
Sundqvist
UG
C
93
50
T
C&R
S
X2
Chugal
PG
R
75
85
Ro
Ra
S
LR
Hoskinson
Sp
R
78
76
Ro
C&R
S
GEE
Farzaneh
Sp
C
22
103
T
C&R
S
LR
Gorni
PG
C
94
452
T
C&R
S
M–W
Çaliskan
Sp
R
96
86
T
C&R
S
X2
“–” missing information; UG undergraduate students; PG postgraduate students; Sp specialist endodontists; R retrospective study; C prospective cohort study; RCT randomized controlled trial; T teeth; Ro root; C&R combined clinical and radiographic examination; Ra radiographic examination only; S strict criteria; L loose criteria; LR single level logistic regression; GEE generalized estimating equations; X 2 chi-square test; M-W Mann-whitney U-test
In addition to the methodological characteristics, most systematic reviews present the original results in descriptive forms using tables and graphs. Table 3.2 combines study characteristics and results for our systematic review describing 12 studies that tested the effect of our new antiviral therapy. The table presents the year of publication, total sample size, source of funding, sample size in each experimental group (n), estimate of the intervention effect (mean), an estimate of the intervention variability (standard deviation – SD), and the p-value.

Table 3.2

Description of study characteristics and original results
Year of publication
Sample size
Source of funding
Treatment
Control
p-value
n
Mean
SD
n
Mean
SD
1997
42
Private
21
2.0
1.7
21
3.9
2.1
0.003
1998
31
Private
16
1.8
2.4
15
3.8
2.7
0.04
1998
44
Private
22
2.1
2.6
22
3.5
2.8
0.09
1999
33
Public
18
3.3
2.7
15
1.8
2.4
0.11
2001
30
Private
14
3
3.2
16
2.1
2.9
0.43
2001
29
Private
13
2.1
2.9
16
3.2
3.2
0.35
2002
27
Public
13
2.9
2.9
14
2.5
2.5
0.70
2002
31
Private
15
2.5
2.5
16
3
2.9
0.61
2005
190
Public
96
2.9
2.1
94
3.1
2.5
0.55
2007
80
Public
39
3.5
2.6
41
3.1
2.2
0.46
2007
145
Public
73
3.2
2.4
72
2.9
1.9
0.41
2008
394
Public
198
2.8
1.7
196
3.1
1.8
0.09
The overall trend of the studies is used to suggest if a given intervention is better than the standard treatment or no intervention when only descriptive tables are used to present results. This approach is very intuitive and does not need any statistical expertise to be conducted. However, as you will later see in this chapter, it can be misleading for several reasons. An overall assessment of Table 3.2 indicates that between 1997 and 2002, mostly small studies were conducted, with large studies being published only in the last 4 years. This finding is consistent with most new therapy studies since large, costly, time-consuming studies are only conducted after some evidence of positive effect and safety is available. A closer look at the results of the studies shows that within small studies the results are very inconsistent, with few studies showing large positive or negative effects for the therapy when compared to placebo. In contrast, large studies do not show major differences between experimental groups. As expected, variability is larger in smaller studies due to the sample size effect on standard deviation estimates. Only the first two studies reached statistical significance, and in three other studies somewhat borderline results were found (p ∼ 0.10).

3.2 Main Results: Overall Estimates of Effect

The treatment effect could be estimated by calculating an overall mean of the results simply by summing up the individual results dividing by the number of studies. This approach, although very intuitive, would not take into consideration the studies characteristics, with studies contributing equally to the overall estimate. Looking at the estimates in Table 3.2, it is obvious that some studies have more precise estimates than others. Factors that may affect the precision of the estimates are various, including sample characteristics, sample size, measurement precision, and reliability. In the meta-analysis framework, sample size is often the most important factor to be taken into consideration. Thus, overall estimates should take into account the sample size with larger studies contributing more than small studies. Mathematically, this can be accomplished by multiplying each study estimate by the sample size or, in other words, by weighting the estimates according to sample size. The sum of the estimates can then be divided by the total sample size. Table 3.3 shows the weight of each study of our example according to sample size. Using this approach, the mean reduction in days with pain would be 2.8 days for the treatment group and 3.1 days for the control group. Thus, the placebo treatment reduced in approximately 0.3 days patients’ symptoms.

Table 3.3

Study weights according to sample size and inverse-variance methods
Year of publication
Weight based on sample size (%)
Weight based on inverse-variance method (%)
1997
3.9
4.6
1998
2.9
1.9
1998
4.1
2.4
1999
3.1
2.0
2001
2.8
1.3
2001
2.7
1.2
2002
2.5
1.5
2002
2.9
1.7
2005
17.7
14.2
2007
7.4
5.5
2007
13.5
12.4
2008
36.6
51.3
Total
100
100
In essence, this is what is done in a meta-analysis to take into consideration the contribution of each study. A similar strategy can be used to account not only for the sample size, but also for the variability in the estimates of the original studies. The overall weighted estimate is calculated multiplying each study estimate by the inverse of the square of the standard error (inverse-variance weighting method), which is highly associated with the sample size of the study. Using this approach, the weighted mean difference (WMD) between treatments is 0.26 mm in favor of the new antiviral therapy. Table 3.3 also shows the weight attributed to each study according to the inverse- variance method. It is clear that the study published in 2008 dominates the overall estimate not only because it has the largest sample size, but also because it has the greatest precision (smaller standard deviations and confidence intervals). Studies with lower variability receive greater weight and therefore have greater influence in the estimate.
Tables and graphs are popular ways of presenting the results of a meta-analysis. Table 3.4 presents the WMD and the 95% confidence interval for each study. The weighted mean provides an estimate and direction of the effect, and the confidence interval provides an assessment of the variability of the estimates. Confidence intervals also indicate the significance of the results and when it does not include zero (or 1 when the results are presented in odds ratio), the weighted mean is statistically significant.

Table 3.4

Meta-analysis result using the inverse-variance method
Only gold members can continue reading. Log In or Register to continue

Year
Sample size
Weighted mean difference
95% CI
Weight (%)
Lower
Upper
1997
42
−1.9

Stay updated, free dental videos. Join our Telegram channel

Nov 16, 2015 | Posted by in General Dentistry | Comments Off on Understanding and Interpreting Systematic Review and Meta-Analysis Results

VIDEdental - Online dental courses

Get VIDEdental app for watching clinical videos