Introduction
The aim of this study was to evaluate the diagnostic correlation and reliability of Dolphin Imaging fully automated segmentation (Dolphin Imaging and Management Solutions, Chatsworth, Calif) for assessing adenoid hypertrophy. This was investigated through 3 modes: (1) intraobserver and interobserver agreement of repeated airway auto-segmentation procedures, (2) correlation between auto-segmentation measures of volume and minimal cross-sectional airway against nasopharyngoscopy, and (3) optimum diagnostic cutoff thresholds for volume and minimal cross-sectional airway identified and tested with sensitivity and specificity analyses.
Methods
Cone-beam computed tomography scans of 38 patients with suspected upper airway obstruction were analyzed. Two calibrated evaluators applied a previously validated method to quantify nasopharyngeal minimal cross-sectional airway and volume using Dolphin Imaging. Assessments were compared against grades of obstruction provided by otolaryngologists’ diagnoses.
Results
The reliability between the 2 assessments by the same evaluator on the Dolphin automatic segmentation function for volume (ICC, 0.97; 95% CI, 0.95, 0.98) and minimal cross-sectional airway (ICC, 0.84; 95% CI, 0.69, 0.91) was excellent. The interoperator reliability for volume was also excellent (ICC, 0.97; 95% CI, 0.95, 0.98), but only good (ICC, 0.701; 95% CI, 0.44, 0.85) for minimal cross-sectional airway. In contrast, the Spearman rank correlation test demonstrated weak associations between the values of the automatic measurements for both volume (4.9 %; ρ = −0.22) and minimal cross-sectional airway (3.7 %; ρ = 0.19). Assessments of accuracy via Receiver Operating characteristic analysis, sensitivity, specificity, negative predictive values, positive predictive values, and likelihood ratios demonstrated the poor clinical applicability of volume and minimal cross-sectional airway numbers provided by Dolphin Imaging.
Conclusions
The evaluators were reliable at manipulating the selected software, achieving consistent volume and minimal cross-sectional airway measurements, However, Dolphin Imaging volumetric and minimal cross-sectional airway measurements did not correlate well with the nasopharyngoscopy-supported reference standard for adenoid hypertrophy assessment. Under these study conditions, volume and minimal cross-sectional airway used to assess localized adenoid hypertrophy with cone-beam computed tomography imaging based on automated measurements may not yield high-quality clinically relevant information about upper airway constriction related to adenoid hypertrophy.
Highlights
- •
CBCT scans of patients with nasopharyngeal airway obstruction were analyzed with Dolphin Imaging.
- •
Volume and minimal cross-sectional area failed to properly represent airway anatomy in the adenoid region.
- •
Clinicians should gather additional diagnostic information from clinical history and symptoms before a definitive adenoid hypertrophydiagnosis is supported.
A common cause of an upper airway dysfunction in children and adolescents is adenoid or tonsil hypertrophy. Chronic adenoid hypertrophy can cause partial or total airway obstruction. In chronic cases, adenotonsillar hypertrophy is considered an important risk factor for obstructive sleep apnea development.
The diagnosis of obstructive dysfunction is primarily made on the basis of the medical history, as well as the physical findings. Chronic snoring, breathing interruption during sleep, diminished growth rate, tendency to fall asleep during the day, behavioral difficulties, and chronic runny nose are common symptoms. An indicative medical history with associated physical findings will support an imaging request to provide a definitive diagnosis. In such cases, nasopharyngoscopy is considered the reference standard of imaging; it is a noninvasive and radiation-free technique that facilitates the assessment of the adenoids without sedation.
Among dental professionals, 2-dimensional (2D) lateral skull radiographs have been the traditional tool to screen for adenoid hypertrophy. The advent of cone-beam computed tomography (CBCT) imaging has expanded the screening capability of the upper airway area by allowing simultaneous assessment of cross-sectional areas and volumetric portrayals. Three-dimensional (3D) imaging compensates for the downside of 2D views by refining the image definition, eliminating structural superimpositions, and improving the overall diagnostic accuracy. Simultaneously, CBCT is becoming the imaging tool of choice for diagnosis and treatment planning of complex orthodontic cases. However, the high percentage of CBCT incidental findings in the upper airway area (42.3%) has raised the attention of researchers.
Several software companies have developed specific applications to analyze and display the DICOM reconstructions of complex airway space and anatomy. Some of these craniofacial reconstruction softwares propose that their computer algorithms could efficiently replace the time-demanding manual segmentation of the upper airway space in 3D reconstructions. These software programs were designed to offer automatically or semiautomatically upper airway CBCT segmentation for limited or no operator control and knowledge of anatomy while transforming the data into a 3D volume.
A previous study questioned the accuracy of Dolphin Imaging automated software (Dolphin Imaging and Management Solutions, Chatsworth, Calif); the authors found that observers’ effects influenced the segmentation and volume when the method was not standardized and the numbers were compared with manual segmentation. To our knowledge, no study has evaluated the airway volume and minimal cross-sectional airway area calculated from this software against the reference standard assessment for adenoid hypertrophy by an otolaryngologist head and neck surgeon through nasopharyngoscopy. Therefore, the aims of this study were (1) to test the reliability of CBCT volumetric and cross-sectional area measurements of the nasopharynx using Dolphin Imaging and (2) to test the correlation of CBCT volumetric and cross-sectional area measurements of the nasopharynx with an otolaryngologist head and neck surgeon’s nasopharyngoscopy-supported adenoid hypertrophy diagnosis.
Material and methods
The Research Ethics Board at the University of Alberta in Canada (Pro00044649) approved this retrospective cross-sectional study protocol.
The study sample consisted of CBCT imaging from 39 subjects between 6.3 and 15.8 years of age (mean, 11.5 years ± 2.8). This prospective sample was selected from a pool of patients’ records from the Inter-disciplinary Upper Airway Clinic at the University of Alberta. Consecutively examined patients were previously selected as explained in a previous study. No CBCT images were taken specifically for this study, according to the ALARA principle and the standards of care of the American and European guidelines.
The inclusion criteria were (1) children from 6 to 16 years old and (2) referrals for significant upper airway concerns. Patients with (1) previous active treatment of upper airway dysfunction or sleep disorders, (2) previous orthodontic treatment, or (3) developmental craniofacial disorders such as syndromes and cleft palate were excluded.
All subjects had the CBCT and nasopharyngoscopy obtained during a 2-hour appointment. The same technician using the same scan parameters followed a CBCT imaging protocol. The images were taken using an ICat Classic (cone beam 3D dental imaging system; Imaging Sciences International, Hatfield, Pa) at 110 kV, 6.19 mAs, with a customized height field of view maximum of 12 in, 300-μm voxel, and 8.9-second scan time with the patient in an upright position with maximum intercuspation. Images were stored in DICOM format.
Subsequently, an otolaryngologist head and neck surgeon analyzed the nasopharyngoscopy of each patient, and adenoid size was classified on a 4-point scale based on its obstruction. The severity of adenoid hypertrophy was graded by this validated method: grade 1 (up to 25% obstruction), grade 2 (25%-50% obstruction), grade 3 (50%-75% obstruction), and grade 4 (>75% obstruction). The nasopharyngoscopy was performed following the American Academy of Allergy’s protocol.
A third-party reconstruction software, Dolphin Imaging 3D (version 11.7 Premium; Dolphin Imaging and Management Solutions), was used. Our choice of software was justified by the fact that Dolphin Imaging is the most commonly used reconstruction software for CBCT imaging in North America.
We applied a standardized protocol to reposition the images according to head position orientation to minimize errors while limiting the landmarks and calculating the volume areas. It had been confirmed that most programs in use have limitations in measuring curvilinear areas. The Dolphin Imaging orientation (head positioning) tool was used by the same researcher (C.P.P.), who oriented all images as follows.
- 1.
Frontal view, the axial plane crossed the right and left orbital points.
- 2.
Lateral view, the coronal plane was parallel to the Frankfort horizontal plane and coincident with the axial plane. The Frankfort horizontal plane was constructed from the right to left porions in the external auditory meatus and the right and left orbitale.
- 3.
Axial view was based on the posterior nasal spine point located on the frontal view in a vertical axis crossing the nasofrontozygomatic plane.
The segmentation method was validated, described, and tested for validity by Palomino-Gomez. For evaluation of the nasopharynx around the adenoid area, landmark points and limits for segmentation were located in the posterior nasal spine, vomer posterior, horizontal extension point vomer posterior, vertical extension point posterior nasal spine, uvula, and basion. Landmark location details for each point can be found in Table I .
| PNS | Posterior nasal spine: the most posterior point of the posterior palatal bone and the floor of the nose cavity, in the midsagittal median point |
| VP | Vomer posterior: the most posterior portion of the vomer bone |
| Ba | Basion: the most anterior-inferior portion in the margin of the foramen magnum, in the midsagittal plane |
| Uv | Uvula: the inferior limit of uvula in the coronal view |
The representation of the sagittal plane was amplified 4 times for better visualization of the area. The ideal slice to start the landmark identification was the one that clearly showed the distal cortical aspect of the posterior portion of the vomer. The reference points were identified in the sagittal plane.
The delimitation of the adenoid area was defined by the junction of posterior nasal spine, vomer posterior, basion, and a plane passing through the uvula point as localized in the coronal view. This plane crossed 2 perpendiculars, one originated at the posterior nasal spine anteriorly and the other posteriorly at the basion point. These 5 junction points form a pentagon. The pentagon area represents the nasopharynx; the posterior vomer point limits it superiorly and inferiorly by the uvula plane. The adenoid area, which is our region of interest, was visualized inside this pentagon area on the sagittal view of 3D reconstructions created by the third-party software ( Fig 1 , A ).
The uvula point was identified on the coronal view, and then the evaluators used a “seed point” specifically for this purpose. The seed point thereafter showed the uvula inferior limit when the operator changed the view. The uvula plane was based on the sagittal view and crossed the posterior vomer and posterior nasal spine perpendiculars.
To determine the area contrast sensibility, 2 reference points (“seed points,” specific tool of Dolphin Imaging 3D) were placed in the middle of the airway space inside the projected pentagon. Thereafter, the area to be measured was filled out (pink color). This delimitation represents the densities of the airway and should not invade the airway space and structures around it in any of the planes. The determination of the “slice airway sensibility” number on Dolphin followed the same criteria for all reconstructions. ( Fig 1 , B , shows the seed points of Dolphin Imaging 3D).
After previous calibration of raters (described in the next section), the airway volume was quantified in cubic millimeters. To delimit the airway’s minimal transversal area section (delimited on the airway space of the nasopharynx), Dolphin Imaging offered 2 limiting lines shown in Figure 2 . Using these 2 lines that limit the desired slices, the evaluator defined the upper and lower limits, and the automatic tool of the software produced a yellow cut representing the most restricted area and generated that area in square millimeters ( Fig 3 ).
The operators were oriented, trained, and calibrated to assess the adenoid and upper airway area using CBCT images not included in this study sample. An intraclass correlation coefficient (ICC) test was applied to evaluate the intraoperator calibration until a high level of consistency was achieved with an ICC of 0.969 (95% CI, 0.948, 0.983). The training took place until adequate agreement of 5 images (different from the study sample), measured in 2 consecutive trials within a 30-day interval, had intrarater reliability above an ICC of 0.90, representing very good agreement between raters.
The sample measurements followed a randomized order, generated by a statistician, for each trial. All measurements were completed by the same evaluators (C.P.P. and S.P.G.) and repeated twice under identical conditions, after a 2-week washout period. The evaluators did not have access to their previous measurements for the second trial. In addition, the raters were blinded to any other information from the patients’ records.
Statistical analysis
The Statistical Package for the Social Sciences (version 22; IBM, Armonk, NY) was used for data analysis. ICCs were applied to evaluate intraobserver and interobserver reliability. Reliability was ranked according to the ICC value; the level of agreement reflected by the ICC was as follows: 0 to 0.2, poor agreement; 0.3 to 0.4, fair agreement; 0.5 to 0.6, moderate agreement; 0.7 to 0.8, strong agreement; and more than 0.8, almost perfect agreement.
Association was evaluated in part using the Spearman rank correlation coefficient (Spearman rho, ρ), which measured the statistical dependence of the nonparametric variables, representing the association of the volume and minimum cross-sectional area with nasopharyngoscopy, the reference standard. The values of the correlation were interpreted according to the guidelines of Cohen et al to interpret effect size: a correlation coefficient of 0.10 to 0.30 represents a weak or small association; a correlation coefficient of 0.31 to 0.49 is a moderate correlation; and a correlation coefficient of 0.50 or greater is a strong or large correlation.
Correlation strength was indirectly further investigated using Receiver Operating Characteristics (ROC) analysis, establishing the optimum “diagnostic” cutoff points for volume and minimal cross-sectional airway area. From these numbers, sensitivity, specificity, positive predictive values, negative predictive values, and likelihood ratio will be discussed. Positive and negative predictive values are disease-prevalence dependent statistics. The average of prevalence in published studies with populations matching our age range was calculated as 31.4%.
P values below 0.05 indicated statistical significance and 95% confidence intervals.
Results
The final sample consisted of 38 subjects, 16 girls and 22 boys; 1 subject from the initial sample was excluded because a radiopaque object interferred in the measurement of landmarks. The age of these patients ranged from 6.3 to 15.8 years; the mean age was 12.3 years (SD, ± 2.8 years). Their distribution showed a heterogeneous sample composed of different levels of obstruction.
The consistency between the 2 trials in Dolphin Imaging 3D for airway volume (ICC, 0.97; 95% CI, 0.95, 0.98) and airway minimal cross-sectional area (ICC, 0.84; 95% CI, 0.69, 0.91) was excellent for the principal evaluator. The interoperator reliability for volume followed a constant pattern (ICC, 0.97; 95% CI, 0.95, 0.98) and was good (ICC, 0.70; 95% CI, 0.45, 0.85) for minimal cross-sectional area, with a large confidence interval. Table II presents the consistency and agreement between raters. Overall, the differences in evaluation when comparing the 2 trials and evaluators were considered insignificant.
| ICC | 95% CI | Degrees of freedom | P value | ||
|---|---|---|---|---|---|
| Lower bound | Upper bound | ||||
| Intraoperator volume | 0.999 | 0.995 | 0.999 | 37 | 0.000 |
| Intraoperator MCA | 0.840 | 0.693 | 0.917 | 37 | 0.000 |
| Interoperator volume | 0.975 | 0.952 | 0.987 | 37 | 0.000 |
| Interoperator MCA | 0.710 | 0.448 | 0.849 | 37 | 0.000 |
Since high operator agreement was found in all trials, we selected randomly only 1 trial from the first evaluator (C.P.P.) to be correlated with the reference standard, nasopharyngoscopy. The mean volume of the nasopharynx was 6990.91 ± 2845.49 mm 3 . The minimal area was 50.71 ± 30.42 mm 2 . Detailed numbers are presented in Table III .
| Dolphin 3D | Variation | SD | ||
|---|---|---|---|---|
| Measurements | Minimum | Maximum | ||
| Average volume | 6990.91 | 3316.20 | 13411.50 | 2845.49 |
| Average MCA | 50.07 | 19 | 177 | 30.42 |
Stay updated, free dental videos. Join our Telegram channel
VIDEdental - Online dental courses
