Both randomized and nonrandomized studies are integral to orthodontic research and practice because they permit evaluation of relationships between exposures and outcomes, allowing the efficacy, effectiveness, and safety of interventions to be assessed. These designs allow clinical decisions to be informed. Nonrandomized designs include nonrandomized clinical trials, cohort studies, case-control studies, cross-sectional studies, case series, and ecological studies. There is debate surrounding the optimal research design; however, both randomized and nonrandomized designs are important to build a broad, informative evidence base. The designs are therefore complementary, with unique advantages and limitations. The applicability of either approach hinges on the clinical question posed, the feasibility of studying it, and ethical considerations.
Clinical research involves investigation of the cause of a disease, evaluation of associations between cause and effect, and assessment of the preventive or therapeutic value by isolating and recording potential associations. It is imperative that research findings are interpreted and used optimally. Best conduct, use, and interpretation of research require an understanding of research methodology and study design complemented by adequate and transparent reporting.
Clinical research can be broadly classified into nonrandomized and randomized studies. The nonrandomized category, also known as observational studies, includes mainly nonexperimental studies: nonrandomized clinical trials, cohort studies, case-control studies, cross-sectional studies, case series, and ecological studies ( Fig 1 ). Randomized studies are usually known in biomedical research as randomized controlled trials (RCTs). The purpose of randomization is the creation of groups that differ only randomly at the time of allocation of the intervention. Thus, the key difference between randomized and nonrandomized studies is that in the former, the investigator allocates the interventions to participants randomly: eg, by throwing dice or coins, or by using computer software to generate an unpredictable sequence.
There has been considerable debate as to which design is more appropriate to answer clinically important questions. RCTs are considered the gold standard for assessing the efficacy and safety of the intervention of interest. However, it is not always possible or ethical to conduct an RCT. When RCTs are not feasible or unethical, nonrandomized studies, such as cohort or case-control studies, may be undertaken. In general, nonrandomized studies are more prone to systematic and confounding biases than are RCTs; consequently, it is also more difficult to make causal inferences concerning the effect of an intervention.
Observational studies are used extensively to describe the distribution of disease and exposure in populations. They are also useful for hypothesis generation and testing; however, hypotheses may be assessed more adequately, when feasible, with RCTs. On the other hand, RCTs, although highly controlled and often less biased because they are conducted in highly selected settings, may yield less generalizable results that lack relevance to other populations and settings. RCTs, therefore, tend to have high internal validity but low external validity. For example, if we consider the assessment of Class II correction, Class II malocclusion is a multifactorial problem expressed with many variants including maxillary protrusion, mandibular retrognathia, or a combination. A successful intervention for Class II correction demonstrated in an RCT with a highly selected group of participants may not apply to the wider Class II population. Consequently, we can say that the findings of RCTs are less generalizable. Examples of published randomized and nonrandomized (observational) studies are shown in Table I .
Study type | Treatment | Outcome | Source |
---|---|---|---|
Randomized | |||
Randomized clinical trial | Lingual retainer failures Chemical vs light cured bonding |
Failure | Pandis et al, 2013 |
Randomized clinical trial | Comparison of the Twin-block vs the Dynamax appliance | Class II correction | Thiruvenkatachari et al, 2010 |
Nonrandomized | |||
Cohort | Self-ligating vs conventional brackets Wire size and material |
Arch-wire ligation time | Turnbull and Birnie, 2007 |
Case control | Self-ligating vs conventional brackets | Treatment efficiency | Harradine, 2001 |
Nonrandomized studies
For the purposes of this review, the discussion will be limited to cohort and case-control designs, since these are most commonly used when an RCT is inappropriate.
Cohort studies
Cohort studies are also called follow-up, longitudinal, or incidence studies. The subjects are followed over time to monitor their health outcomes. Participants with different levels of exposures to risk factors or different characteristics at baseline are then compared to estimate differences in the rate of developing certain health outcomes later in life. These groups are defined as study cohorts. All participants must be at risk of developing the outcome. The participants are followed for a set period of observation (usually a long period), and all new cases of the outcome of interest are identified. Comparisons of outcome experiences are made within the study cohorts. Ideally, the exposure factor would constitute the only difference between the populations under comparison, although in reality people with different levels of exposures also differ in other characteristics.
Cohort studies that are based on information concerning the exposure and the outcome collected from preexisting sources in the past are called retrospective cohort studies. This design is common in orthodontics when information on exposures and outcomes from patient records is retrieved and associations explored. However, the usefulness of such a study depends on the thoroughness of the certification of the outcome in the records for the time period under consideration. Moreover, information on confounding factors may not be available because there was no planning for the study during completion of the files. Cohort designs have key advantages and disadvantages ( Table II ).
Type of study | Advantages | Disadvantages |
---|---|---|
Cohort study | Direct information on the sequence of events; therefore, the temporal relationship between exposure and disease can be more easily clarified | Prospective cohort studies can be expensive and time-consuming, whereas retrospective cohort studies require the availability of adequate records |
Multiple effects of an exposure can be examined | Losses to follow-up, incomplete information on exposures and outcomes, and confounding (retrospective designs) | |
Direct measurement of incidence rates or risks and their differences and ratios is possible | Associations between exposures and rare outcomes require a large cohort and a long period of observation | |
Case-control study | Less expensive and easier to conduct compared with cohort and randomized studies | Temporal relationship between exposure and disease may be difficult to establish |
The method of choice for investigating rare diseases | Assessment of rare exposures is usually difficult unless the study is large or the exposure is common among those with the disease | |
Multiple risk factors can be studied simultaneously | Incidence rates of disease in exposed and unexposed subjects cannot be estimated in most instances. | |
Efficient in terms of sample size | Prone to selection and recall biases | |
Randomized controlled trial | Intervention studies often provide the strongest support for a cause-effect relationship | Many research questions cannot be tested in trials |
Confounding is minimized, thus facilitating fair comparisons | Ethical, feasibility, and cost issues | |
External validity |
Case-control studies
This design aims to achieve the same goal as the cohort study more efficiently by using sampling. Simultaneously, cases and controls free from that particular condition are chosen as a representative sample of the population. Ideally, the control group represents the exposure distribution in the source population that produced the cases. Subsequently, information concerning exposure is collected for both the cases and the controls. Data are analyzed to determine whether exposure patterns differ between the cases and the controls.
The selection of subjects is based on disease status. Sources of cases can include private practices, clinics, health registries, or screening programs. Controls can be selected by using random sampling from the general population, practices, or clinics, or they might be relatives and friends of the cases. The specific advantages and disadvantages of case-control studies are outlined in Table II .
Bias and confounding in nonrandomized studies
It is often difficult to establish a causal relationship between the exposure and the outcome based on observational designs because of potential biases and confounding. Bias is the systematic error in the selection of participants, the collection of data, and the ascertainment of exposure and outcome variables, leading to incorrect associations. Methods of reducing potential bias should be carefully considered and planned at the study design stage, and the risk of bias should be accounted for when interpreting the results of such studies. Bias or systematic error differs from random error, which is related to the variability in the sampled population and is expressed by uncertainty (confidence intervals) around the estimate. Increasing the sample size increases the precision of the estimate by reducing the random error but not the systematic error. Selection and information bias are the 2 most prevalent types of systematic bias encountered in observational studies ( Table III ).
Systematic bias | Description |
---|---|
Selection bias | Comparison groups differ due to selection error in terms of important baseline characteristics that are important outcome predictors. |
Information bias | |
Recall bias | Recollection of information on exposure is not accurate and can be influenced by experiencing or not experiencing the outcome. |
Detection bias | Recording the exposure or the outcome is influenced by knowledge of the outcome or the exposure, respectively. |
Confounding bias caused by factors that causally affect the exposure and outcome (known as confounders) can blur the associations between exposures and outcomes ( Fig 2 ). Let us assume that we are interested in evaluating the association between oral hygiene status and dental caries in 2 groups of young children with good and less-good hygiene. If all other parameters were equal, as expected in a randomized design, we could be confident that any differences in dental caries between treatment groups would be the result of differences in the levels of hygiene. Now, imagine that another risk factor, dietary sugar, is associated with caries (the outcome). Those subjects with substandard hygiene consumed more sugar than those with good hygiene because the former like to eat sweets. If we accept that sugar consumption promotes caries and that the consumption of sugar is higher among those with substandard hygiene, it is likely that the effect of hygiene on caries may be exaggerated because of another factor that would be counteracting caries prevention in those with substandard hygiene. This factor is called a confounder because it complicates the association between exposure (brushing) and outcome (caries). Sugar consumption is considered a potential confounder, since it is associated with the exposure—hygiene level—and is an independent predictor of the outcome, and it is not an effect of the exposure (the third condition for a parameter to be considered a potential confounder).
Randomized studies
RCTs are investigations similar to cohort studies in which the researcher randomly assigns the exposure or intervention to the study participants. The aim of clinical trials is to investigate the effectiveness and safety of an intervention. RCTs are widely regarded as the optimal approach in assessing a new treatment. In orthodontics, studies exploring the efficacy of a device could be generally characterized as clinical trials. Clinical trials involving skill-dependent interventions such as devices (device trials) resemble clinical trials for drugs (drug trials); however, devices have less need for extensive developmental testing compared with drugs in humans, but not less rigorous testing overall. All RCTs, like device trials, share common core design features such as explicit inclusion and exclusion criteria, use of controls, randomization, masking where feasible, and the intention-to-treat principle. Some of those characteristics also apply to nonrandomized clinical studies. RCTs can be subcategorized in many ways, depending on their characteristics and purposes ( Table IV ). The key advantages and limitations of randomized studies are outlined in Table II , and the differences between randomized and nonrandomized studies are outlined in Table V .
Subcategory | Characteristics |
---|---|
Single site | Treatment provided by a single center |
Multisite | Treatment provided by multiple coordinated centers |
Preventive | Prophylactic agent is provided to prevent outcome occurrence |
Therapeutic | Therapy is provided to improve survival or a condition |
Parallel | Everyone in a group receives the same treatment that is different from the treatment given to other group |
Cross-over | Each treatment is given at different times to each subject. A variation is the split-mouth design. |
Factorial | Each group gets more than 1 treatment |
Noninferiority | To establish noninferiority of a new therapeutic modality that may be more cost-effective or easier to deliver |
Clustered | Interventions are assigned to clusters (schools, classrooms) and not to individuals |
Adaptive | Allows prespecifying prospective modification of at least 1 aspect of the trial based on the accumulating data |