Missing participant outcome data (MOD) are ubiquitous in clinical trials and may threaten the validity of the results. The risk of bias associated with MOD is carried over to the meta-analysis by including clinical trials with participant losses. In this section, we discuss MOD, and we present common statistical methods to address MOD in a pairwise meta-analysis. Despite their conceptual and statistical shortcomings, these methods have prevailed in the published systematic reviews for their simplicity.
The missing outcome data mechanisms
Analysis of MOD involves untestable assumptions about the reason behind the participant losses (the mechanism of dropout). Little and Rubin classified the mechanisms of MOD in 3 categories: (1) missing completely at random (MCAR), when the probability of dropout is independent of observed and unobserved covariates; (2) missing at random (MAR), when the probability of dropouts depends on the fully observed covariates, but not on the unobserved ones; and (3) missing not at random (MNAR), when the probability of dropouts depends on some unobserved covariates and the outcome. For instance, if the outcome of a participant is missing because of a laboratory accident, the mechanism is MCAR. If elderly patients tend to have lower responses to a particular intervention than younger patients and leave the trial early, the mechanism is MAR because the probability of dropout can be explained by the observed covariate age. Suppose patients without health improvement tend to leave the trial early. In that scenario, the mechanism is MNAR because dropout depends on the outcome.
The MCAR assumption is equivalent to randomizing the patients to the interventions and then randomly deciding whom to observe. However, this assumption is rarely plausible in clinical trials. In contrast, the MNAR assumption considers MOD to be nonignorable. Hence, ignoring MOD from the analysis may lead to biased results. Typically, the reasons for premature discontinuation are unknown. Therefore, the analysts may consider the MAR assumption as the starting point in their analysis and then evaluate the sensitivity of the primary analysis results to different scenarios about the mechanism of MOD. Any inconsistency in the conclusions is a strong indication that the MOD may be MNAR. For practical reasons, many statistical techniques for MOD are based on the MAR assumption.
Handling missing outcome data in a meta-analysis
In the meta-analysis, the data are typically available in a summary form for each trial, such as mean responses and standard deviations or numbers of events for each intervention. Without access to individual participant data, there are limited options for addressing MOD in the meta-analysis. The majority of these methods have been developed mostly for binary outcomes. Below, we briefly present these methods along with their advantages and disadvantages.
Under the available case analysis (ACA), MOD are first excluded from the analysis, and the analyst proceeds with the synthesis of only the available but reduced randomized sample of the trials. This is the most common approach in the meta-analysis and is based on the MCAR assumption. However, excluding MOD from the analysis may yield imprecise and biased results—the latter, if the MCAR assumption does not hold.
Under the imputed case analysis (ICA), the analyst either imputes (fills in) the missing responses with an assumed outcome (number of events or nonevents) or with the risk of an event on the basis of the observed responses. Then, the analyst proceeds with the synthesis of the full randomized sample of the trials. If the assumptions are clinically plausible on average, the ICA may yield unbiased estimates. The most frequently applied assumptions are the following : (1) all MOD are nonevents in both arms (ICA-0): all missing participants in both arms have not experienced the event; (2) all MOD are events in both arms (ICA-1): all missing participants in both arms have experienced the event; (3) the best-case scenario for the experimental arm (ICA-b): all missing participants in the experimental arm have experienced the event, whereas all missing participants in the control arm have not experienced the event; (4) the worst-case scenario for the experimental arm (ICA-w): all missing participants in the experimental arm have not experienced the event, whereas all missing participants in the control arm have experienced the event; (5) the same risk as in the experimental arm (ICA-pE): missing participants in both arms have the same risk of an event as those completing the experimental arm; (6) the same risk as in the control intervention (ICA-pC): missing participants in both arms have the same risk of an event as those completing the control arm; (7) intervention-specific risk (ICA-p), when the risk of an event among those dropping from arm k is the same as the risk of an event among those remaining in arm k with k being the experimental or control arm. This approach corresponds to the MAR assumption.
Advantages and disadvantages of ICA methods
ICA-0 and ICA-1 are appropriate when there is a rational relationship between the missingness and the outcome. For example, in studies of patients with Class II malocclusion treated with either a Twin-block or an activator appliance, it is rational to assume that missing participants might not have cooperated with wearing appliance (ICA-0). In studies for controlling temporomandibular joint dysfunction with splint therapy vs physical, we can assume that missing participants might not have experienced remission in the symptoms (ICA-1). ICA-b and ICA-w reflect the most extreme assumptions, and thus, they yield the most extreme effect sizes in either direction (favoring the experimental or the control intervention), often resulting in conflicting inferences. However, because we do not know the exact reasons for premature discontinuation, there is high uncertainty in these scenarios. None of the ICA methods accounts for this uncertainty in the summary results. Consequently, the ICA methods tend to provide spuriously precise results for treating the imputed values as observed. The Supplementary Table illustrates how to apply the ACA and ICA approaches to a dataset with MOD before proceeding with the pooling of the trials.
Gamble and Hollis proposed an approach that incorporates the uncertainty about MOD into the results. Initially, in each trial, we estimate the effect size under the ACA approach. Then, in each trial, we implement the ICA-b and ICA-w approaches and use the most extreme lower and upper limits of the confidence intervals to form a so-called uncertainty interval. Consequently, the corresponding standard errors are inflated, leading to reduced weights; the higher the proportion of MOD, the smaller the weights. Then, we perform an inverse-variance meta-analysis.
Finally, the ICA methods and the Gamble-Hollis approach consider only a handful of scenarios whose clinical relevance is difficult to defend. In the next column article, we introduce a conceptually and statistically advantageous method that allows for more scenarios about the mechanism of dropouts and naturally accounts for the uncertainty induced by MOD.
All approaches can be perform using the new command metamiss2 in Stata, which is an extension of the metamiss command to handle aggregate MOD in meta-analysis.
Application to real data: binary outcome
We implement all methods mentioned above in an illustrative example for binary data. Again, we use the data of 5 studies ( Table ) included in a meta-analysis of a published systematic review that compares bond failures between plasma and halogen curing lights. The effect measure of interest is the risk ratio (RR). The original data have been manipulated to produce MOD in the range of 0.66%-8.65%. The primary analysis is the ACA. We will compare the results from the primary analysis with those from all ICA methods and the Gamble-Hollis approach. The Figure depicts the results of this sensitivity analysis. The summary RR under ICA-1 and the 2 extreme approaches (ICA-b and ICA-w) are not consistent with the summary RR under ACA. This indicates that the MOD might not be MAR; hence, the primary analysis under ACA might yield biased results.
