Introduction
The clinical choice of diagnostic tests or treatment options is determined by the probability that the value of their execution (called the warrant for the test) exceeds their cost, and by their usefulness. The purpose of this study was to determine the warrant and usefulness of STOP-Bang, an obstructive sleep apnea screening questionnaire, and cone-beam computed tomography (CBCT) information about the minimal cross-sectional area for referring a mouth-breathing patient to a sleep specialist and for modifying planned orthodontic treatment.
Methods
A branching survey was used to identify the prominence of paths between the presenting situation, 2 diagnostic tests, and 2 referral and/or treatment options. A description was given of a hypothetical patient: an overweight, mouth-breathing female teenager. Path analysis was used as a method for quantifying diagnostic warrant and usefulness.
Results
There was a wide variation among the 125 orthodontists who responded to the survey. All paths were chosen. The use of tests altered the referral (χ 2 = 8.039; P = 0.03) and/or treatment decisions (χ 2 = 12.636; P = 0.005). Ownership of a CBCT system significantly influenced the use of this diagnostic test, with owning a CBCT system resulting in greater use in-office (χ 2 = 50.416; P <0.001) and greater use in the study (χ 2 = 22.959; P <0.001). The usefulness of the diagnostic tests could not be determined directly because common values were used for each test, but the variation in the use of this standard stimulus was very large, indicating personal differences in the interpretation of actual data.
Conclusions
Wide variation in the choice and interpretation of diagnostic tests for referral and orthodontic treatment modification relative to airway condition exists among orthodontists. Diagnostic path analysis is a potentially useful model for studying how practitioners make decisions independent of research evidence.
Highlights
- •
Mouth-breathing teenaged patients require appropriate diagnostic testing.
- •
Given the same diagnostic findings, orthodontists choose diverse referrals, treatment plans.
- •
Diagnostic path analysis helps explain how practitioners make decisions.
Clinical decisions are the pivot between what is known about a particular presenting patient and patients of that type and specific interventions undertaken to improve patient oral health. It is possible to entertain, with varying degrees of confidence, several plausible explanations for what might happen if one chooses to intervene or not. However, once an action is taken, it changes the situation. What orthodontists do usually changes the circumstances and alters the situation because this affects later decisions.
That is also the case for diagnostic decisions. Information gathering, including tests, is undertaken specifically to change what is known and useful for guiding more high-stakes interventions. Gathering information that is not used to guide treatment choice is wasteful and may expose patients to unnecessary risks or costs. Failing to use the tests with the highest likelihood of revealing what is needed to treat patients well is not the best practice.
Professional judgment is necessary in choosing which diagnostic information to gather or ignore, and in what order chosen tests should be undertaken. This is the warrant decision: the authority under which it is initiated. Professional judgment is also needed regarding how the test results inform action. This is the usefulness decision. Such basic diagnostic decision units can be chained to form complex personal decision protocols and professional guidelines. This model is illustrated in Figure 1 .
In this highly simplified model, the diagnostic test is warranted by prior information and useful for initiating future action. The term “screening” is generally used to refer to justifications that are population- rather than patient-specific. It is commonly recommended that all males over a certain age should be screened with the prostate-specific antigen test, and some practices use a policy that cone-beam computed tomography (CBCT) images should be taken on all new patients. By contrast, “testing” means warranting gathering of information on the basis of unique factors of the patient such as complaints about thumb habits, signs such as open bite, or risk factors such as parents with conspicuous retrognathic profiles.
This approach is known as a diagnostic path. Situations, including sets of information, are represented by ovals. Actions are represented by arrows connecting situations. The path through the information is sequential so that new actions are warranted by previously obtained information. For example, a practitioner may decide, on the basis of the results of a diagnostic test, that treatment A or treatment B is the best, or that the findings are negative and do not affect treatment decisions. Test outcomes may even warrant further tests, as when high prostate-specific antigen scores warrant biopsies. For simplicity, this model in Figure 1 does not show that presenting population or initial personal factors might lead directly to treatment or to doing nothing. Diagnostic decisions may be chained in simple or complex patterns. This is called the diagnostic protocol.
Path analysis is a formal statistical procedure that permits estimates of the relative weight of each arrow. Taken together, these represent the probability of various paths through the diagnostic process. An important feature of path analysis is that it is sequential. The probability of making a treatment decision with information is different from the probability of making the decision in the absence of that information. The probability of ordering a test depends on what prior information is known. Each practitioner has a personal diagnostic protocol. Given identical presenting patients, there is a variation in which diagnostic tests are chosen. Given identical diagnostic findings, there is variation in preferred treatment.
Path analysis has been used for decades in the social sciences and medicine, to understand, for example, residency performance, medical research priorities, fitness practices, drug use, and pro-health choices of adolescents.
This study aimed to demonstrate a model called path analysis for characterizing the range of diagnostic protocols that describe a specific presenting situation: an overweight, female teenaged patient with signs of mouth breathing. This research differs from the evidence-based or clinical guidelines approach by being entirely descriptive rather than normative. The goal was to characterize how orthodontists use selected information to make decisions regarding management patients with specific occlusal conditions and the potential for obstructive sleep apnea (OSA). It is not a study of how experts interpret the best evidence or a commentary on clinical guidelines.
It was hypothesized that there is wide individual variation among orthodontics in terms of the warrant of diagnostic tests and the usefulness assigned to outcomes when orthodontists are presented with a standardized case.
Material and methods
Survey instrument
A simple diagnostic model was constructed. Descriptions of presenting information and results of diagnostic tests were made available in a branching format, when requested, thus permitting respondents to create their personal diagnostic protocols. The initial information, common to all respondents, was a teenaged female patient who was described as a mouth breather. Two diagnostics tests were available: the STOP-Bang survey for OSA and CBCT images with reported minimal cross-sectional airway findings. Two action outcomes were also available: respondents could refer the patient to a medical doctor specializing in sleep disorders or not, and additionally, they could either modify the orthodontic treatment on the basis of the found information or not. There were 10 paths through this diagnosis model for OSA referral (various paths from presentation to nonreferral, including direct referral without diagnostic tests, presentation to STOP-Bang to CBCT to nonreferral, etc.). There were also 10 paths for the modification of orthodontic treatment. These paths are shown by the arrows in Figures 2 and 3 .
Initially, respondents were given the presenting information and asked to indicate their preferred course of action by assigning weights to the available options at that point and each subsequent decision node. When a referral and modification action was selected, the experiment ended. The STOP-Bang questionnaire is a widely used 8-item survey that can be completed as part of normal health provided by the patient or administered verbally. All questions are yes or no and include snoring, chronic tiredness, observed signs of stopped breathing or chocking, high blood pressure, body mass index above 35 kg/m 2 , older than 50, enlarged neck circumference, and being male. A score of 2 or less is considered “low risk,” 3 or 4 is “intermediate risk,” and above 4 is “high risk.” If the respondent requested this information, they were told that the patient’s score was 4, representing intermediate risk. Respondents could then choose among the remaining options (CBCT, referral and/or nonreferral, and modification and/or nonmodification). When respondents selected the CBCT diagnostic test, they were told that the minimal cross-sectional area (MCA) was 50 mm 2 and were asked what standard they would use for MCA to trigger some sort of response on their parts. At that point, respondents could select any test or action they had not used. In pilot testing, it was noticed that subjects tended to characterize the airway in terms of MCA rather than volume and that they expressed concern regarding the risk of OSA when MCA values approached 55 mm 2 or lower.
The survey was administered electronically on personal laptops or mobile phones, using the Qualtrics system (Qualtrics, Seattle, Wash). In field tests, the survey was completed at an average of 2.4 minutes. Demographic information was also collected. The study was approved by the institutional review board at the University of the Pacific (no. 18-111). The survey is shown in the Supplementary Text .
Sample
The survey was distributed via the American Association of Orthodontists Partners in Research program. In addition, email invitations to participate were sent to faculty and alumni (2005-2017) at the University of the Pacific, Arthur A. Dugoni School of Dentistry, members of the Northern California Angle Society, and members of the Schulman Study Group. A link to the survey was also posted in the Orthodontic Pearls Facebook group. The survey link was active from May 14, 2018, to July 13, 2018.
Statistical analysis
The primary quantitative analysis used for these data was path analysis, also known as causal modeling. The strength of each path was determined by the proportion of respondents selecting that path as an exit from the previous diagnostic test. Weights given to each path for individual respondents recorded in the survey were also used for analysis, but are not reported here because the averages very closely matched the proportion of respondents. All path values were determined by agreement between the 2 authors.
Demographic responses were analyzed by standard descriptive statistics, and chi-square was used to test hypotheses. Actual test statistics and P values were reported.
Results
Sample characteristics
A total of 125 usable responses were returned. Of these, 52% were from respondents who had been out of residency for at least 20 years, 21% for 11-20 years, 15% for 5-10 years, and 12% had been out of residency for fewer than 5 years. The number of years of practice experience was unrelated to any other variable. Years of experience with CBCT ranged from 0 to 10 years, with an overall mean of 3.9 ± 3.05 years. Nineteen percent reported no CBCT experience.
Path diagrams
Figures 2 and 3 show the path diagrams for the action options of refer and/or watch and modify orthodontic treatment and/or same, respectively. Because such path analyses contain a great deal of information and are unusual in the orthodontic literature, some interpretive comments will be made. The numbers on the path arrows exiting each diagnostic test represent percentages of respondents selecting that path as his or her most likely response given the cumulative data at that point. Because the paths are mutually exclusive and exhaustive, the total of the percentages exiting each diagnostic test always equals 100%. That is true for each test, regardless of its sequence in the decision path. Irrespective of the number of steps in the path, the probability of achieving an endpoint (refer or watch, for example) always equals the product of the probabilities along the path. So the probability of the initial examination directly producing a referral equals 0.22. The probability of a referral passing through the STOP-Bang questionnaire is 0.19 (0.34-0.57). The probability of referral with the aid of a CBCT image, alone or in connection with the questionnaire, is small. Because the paths are assumed to be independent, segments of different path diagrams can be pieced together without altering the retained path segments.
There were 2 checks on the validity of the experimental design built into this study. First, all paths for both action outcomes were well populated, meaning that the full model contains viable diagnostic options for orthodontics. If the model contained options that were not considered viable, they would have tiny warrants represented by small probability arrows leading into them. In addition, there was a null option included in the study. Respondents were given the option of fabricating an appliance. Only 6 respondents selected this option, none giving it more than 5 of 100 points for weight. This node in the diagnostic path was therefore dropped. Excluding it from the analysis did not alter any of the reported results. Fabricating such an appliance other than on the prescription of a specialist in this discipline is not within the scope of practice for orthodontists.
Table I compares the action choices of respondents with and without using intermediate diagnostic tests. It is more likely that a referral will be made and that treatment will be modified if either diagnostic test was used. With the initial examination only, the odds were in favor of not referring to a sleep specialist and in keeping the same treatment plan. Relative to the initial examination alone, getting any diagnostic test increased the likelihood of referring to a specialist (χ 2 = 8.039; P = 0.03) and the likelihood of modifying the treatment plan (χ 2 = 12.636; P = 0.005). Specifically, getting the results of the screening questionnaire drove referral to odds of a referral being 2.71. Similarly, getting the results of the CBCT MCA drove modification of the odds of treatment modification to 5.00.
