Introduction
Our objective was to assess the intraexaminer and interexaminer reliabilities of upper airway linear, area, and volumetric measurements in cone-beam computed tomography.
Methods
Cone-beam computed tomography scans of 12 subjects were randomly selected from a pool of 132 orthodontic patients. An undergraduate student, an orthodontist, and a dental radiologist independently made linear, area, and volumetric measurements. Linear anteroposterior and transversal measurements, cross-sectional area, sagittal area, minimum axial area, and volume measurements were made. The intraclass correlation coefficient (ICC) was used to assess intraexaminer and interexaminer reliabilities, and measurement errors were assessed. Agreement was further assessed with the Bland-Altman method and 95% limits of agreement.
Results
Overall, the ICC values indicated good reliability for the measurements assessed. The ICC values were greater than 0.9 (excellent) for 93% of intraexaminer and 73% of interexaminer assessments. Transversal width measurements and cross-sectional area at the level of the vallecula, however, had only moderate reliability (minimum ICC, 0.63), large 95% limits of agreement, and the greatest mean measurement errors (as high as 16% and 13% of the mean measurements, respectively). Linear anteroposterior measurements; cross-sectional areas at the levels of the palatal plane, soft palate, and tongue; and sagittal area and volume were reliable measurements, with a minimum ICC of 0.93 and more restricted limits of agreement.
Conclusions
Based on these results, airway assessments by examiners with different backgrounds might have reliable anteroposterior linear measurements; cross-sectional areas at the levels of the palatal plane, soft palate, and tongue; and sagittal area and volume. The unreliable measurements were linear width, cross-sectional area at the level of the vallecula, and minimum axial area.
An increasing interest in cone-beam computed tomography (CBCT) and the benefits it can bring to diagnosis and treatment planning in the oral and maxillofacial specialties can be seen in the literature in the last decade. Particularly, CBCT airway assessment has been studied by many authors, because this 3-dimensional (3D) tool has many advantages over the way we analyzed the airway in the past: ie, mainly through 2-dimensional cephalometric radiographs.
Two recent systematic reviews on airway analysis with CBCT images, one focusing on clinical indications, technical parameters, and accuracy and reliability of upper airway analysis, and the other on upper airway segmentation, show that some aspects are not yet completely elucidated because of few adequate studies. Some aspects regarding upper airway analysis that lack evidence and still need to be addressed are the validity and reliability of CBCT-generated 3D models ; the influence of respiratory phase, tongue position, and mandibular morphology on the CBCT images obtained; longitudinal and cross-sectional analysis; and determination of anatomic limits.
Accuracy and reliability of airway measurements in CBCT images have been tested. Lenza et al compared the linear, area, and volumetric measurements by 2 examiners and found no significant differences. However, they did not use statistical tests to assess the reliability of the measurements, since that was not the focus of the study. Luiz and Szklo suggested that the results of more than 1 statistical strategy should be considered when assessing the reliability of quantitative measurements.
The best approach in airway assessment is a thorough analysis, including linear measurements, area, and volume, since volume information alone might not necessarily represent or identify the locations of the relevant constrictions.
Nevertheless, airway measurements are not easy to make and demand much attention and concentration. Whether the examiner’s background influences the measurements when he or she has been properly calibrated remains unanswered.
The purpose of this study was to assess intraexaminer and interexaminer reliabilities of upper airway linear, area, and volumetric measurements in CBCT images among examiners with different backgrounds.
Material and methods
The sample size estimates recommended by Walter et al for reliability studies were followed, considering ρ 0 = 0.5 (minimum acceptable level of reliability), ρ 1 = 0.9 (expected level of reliability), α = 0.05, β = 0.2 (which implies a power test of 80%), n = 2 (intraexaminer), and n = 3 (interexaminer). We determined that a sample of CBCT images from 9 subjects would be sufficient.
CBCT images of 12 patients were randomly selected from an available pool of 132 images from patients in the orthodontics clinics of the postgraduate course in dentistry of the Universidade Federal do Rio de Janeiro in Brazil. The exclusion criteria were cleft, systemic condition, craniofacial syndrome, detectable airway pathology, or previous orthognathic or craniofacial surgery. The research was approved by the research ethics committee of the Institute of Collective Health Studies of the Universidade Federal do Rio de Janeiro, and informed consent was signed by all subjects. The methods and principles involved in the research complied with the Helsinki Declaration.
The CBCT scans were obtained by 1 operator using the same I-CAT tomograph (Imaging Sciences International, Hatfield, Pa). The subjects were positioned with the Frankfort horizontal plane parallel to the floor and instructed to maintain maximum intercuspation with the tongue touching the palate and to avoid swallowing during the scanning period. The imaging protocol used were 120 kV, 5 mA, 13 × 17 cm field of view, 0.25-mm voxel size, and a scanning time of 20 seconds. Images were saved in DICOM files, and these files were imported into Dolphin Imaging software (version 11.5; Dolphin Imaging and Management Systems, Chatsworth, Calif) to obtain the primary reconstructed images (sagittal, coronal, and axial) and the 3D reconstructions.
Head orientation was the same for each CBCT image performed by the same experienced operator. The horizontal reference plane was defined bilaterally by porion, and the head on the right side view was tilted clockwise or counterclockwise until the palatal plane—defined by the anterior nasal spine and the posterior nasal spine—became parallel to the horizontal reference plane. This condition was checked later on the midsagittal plane slice. The transporionic plane was oriented vertically, defined bilaterally by porion, and perpendicular to the horizontal reference plane. The midsagittal plane was oriented vertically, defined by nasion, and perpendicular to the previously defined planes.
A fourth-year dental student (L.A.P.), an orthodontist (T.C.S.M.), and a dental radiologist (C.V.C.) were oriented, trained, and calibrated as examiners for airway analysis using images not included in this study. The calibration protocol included an explanation of the 3D measurement tools in the Dolphin Imaging software and a demonstration of the measurements to be made for this research (see Video , available at www.ajodo.org ). The examiners were given 4 images for training at will. Each examiner was considered calibrated when an intraclass correlation coefficient (ICC) test between 2 time measurements and between his or her measurements and those from the trainer was above 0.9 for all variables. After calibration, each examiner independently made the linear, area, and volumetric measurements twice in the CBCT images selected for this study, with a 2-week interval between these periods. The images were randomly analyzed to allow a blinded assessment, and the examiners did not have access to their previous measurements at the second analysis.
Eight linear measurements, 6 area measurements, and 1 volumetric measurement were made for each image. Initially, each examiner measured the airway’s anteroposterior length in the midsagittal plane slice at 4 levels: palatal plane, soft palate, tongue, and vallecula. At the soft palate and tongue levels, the smaller anteroposterior length was determined by visual inspection and measured. In this same slice view, the coronal plane was moved to the airway region until it reached a position 1 mm posterior to the most posterior point in the soft palate, tongue, or anterior pharyngeal wall so that the coronal slice would fall into the airway region in every level that it would have to be measured. The airway’s transversal width was then measured in the coronal slice at the same levels of the measurements in the sagittal slice. The axial plane was then moved to each level of the previous measurements in the sagittal slice, and the airway area was then calculated by the software after its limits were determined by the examiner ( Fig 1 ). Finally, the airway/sinus tool from the software was accessed. Airway limits were defined with this tool according to the following: upper limit, palatal plane extended until it reached the posterior pharyngeal wall; lower limit, a plane parallel to the palatal plane passing through the vallecula; posterior limit, posterior pharyngeal wall; and anterior limit, soft palate, tongue, and anterior pharyngeal wall. The airway was scanned 3-dimensionally to ensure that every aspect was included. The software calculated the sagittal area, the minimum axial area, and the volume of the airway within the established limits ( Fig 2 ).
The data were checked carefully, and extreme outliers were removed. Intraexaminer reliability was calculated by ICC for the measurements obtained by each examiner at both times. ICC values were also used to assess interexaminer concordance by comparing their second measurements. ICC values were estimated using a 2-way mixed-effects model. Reliability was ranked according to the ICC value and considered excellent when it was above 0.9, good when it was between 0.75 and 0.9, moderate when it was between 0.5 and 0.75, and poor when it was below 0.5. Intraexaminer and interexaminer measurement errors (average of the absolute mean differences) were determined for all variables assessed. Measurement agreement was further assessed with the Bland-Altman method through means, standard deviations, and 95% limits of agreement, which were available as data and plots. Ninety comparisons were made in this method, including 45 intraexaminer comparisons (the 2 measurements for each of the 15 variables measured by the 3 examiners) and 45 interexaminer comparisons (15 variables measured by each pair of examiners: student and orthodontist, student and radiologist; and orthodontist and radiologist). Each comparison generated a plot, and additional plots were constructed by mixing data from more than 1 comparison by assigning different colors for the comparisons. Paired t tests were used to detect systematic intraexaminer and interexaminer errors in each comparison. P values below 0.01 were considered statistically significant.
Results
Intraexaminer and interexaminer reliabilities estimated by ICC values for each measurement are shown in Table I . These results indicate good reliability for both intraexaminer and interexaminer assessments. The ICC values were above 0.9 for 42 (93.3%) of the 45 intraexaminer assessments, and none had poor reliability (ICC <0.5). The ICC values were above 0.9 for 11 (73.3%) of the 15 interexaminer assessments, and none had poor reliability. The less reliable measurements in this research, with moderate reliability (0.5 <ICC <0.75), were the transversal widths at the level of the tongue (intraexaminer for 1 examiner) and at the level of the vallecula (interexaminer), and the cross-sectional area at the level of the vallecula (interexaminer).
| Intraexaminer ICC | Interexaminer ICC | |||
|---|---|---|---|---|
| Undergraduate student | Orthodontist | Radiologist | ||
| Anteroposterior length (mm) at the level of | ||||
| Palatal plane | 0.981 | 0.979 | 0.955 | 0.983 |
| Soft palate | 0.989 | 0.957 | 0.987 | 0.979 |
| Tongue | 0.991 | 0.995 | 0.996 | 0.986 |
| Vallecula | 0.999 | 0.995 | 0.989 | 0.996 |
| Width (mm) at the level of | ||||
| Palatal plane | 0.998 | 0.986 | 0.996 | 0.969 |
| Soft palate | 0.988 | 0.943 | 0.999 | 0.883 |
| Tongue | 0.942 | 0.637 | 0.976 | 0.853 |
| Vallecula | 0.987 | 0.855 | 0.989 | 0.664 |
| Cross-sectional area (axial slice) (mm 2 ) at the level of | ||||
| Palatal plane | 0.993 | 0.993 | 0.993 | 0.988 |
| Soft palate | 0.975 | 0.984 | 0.996 | 0.974 |
| Tongue | 0.935 | 0.974 | 0.987 | 0.960 |
| Vallecula | 0.993 | 0.984 | 0.989 | 0.696 |
| Sagittal area (mm 2 ) | 0.983 | 0.979 | 0.985 | 0.977 |
| Minimum axial area (mm 2 ) | 0.999 | 0.869 | 0.999 | 0.932 |
| Volume (mm 3 ) | 0.995 | 0.987 | 0.994 | 0.992 |
Table II shows measurement errors (average of the absolute mean differences) in perspective, considering their relationship to measurement means, and minimum and maximum values (made by the researcher [C.T.M.] who trained the examiners). The sample’s descriptive statistics show the variability of airway measurements among the images of the subjects in this study. As to the measurement errors, the greatest mean differences (>10% of the mean measurement value) in linear measurements were the transversal widths at the level of the soft palate (interexaminer, 11.5%), at the level of the tongue (interexaminer, 13.4%), and at the level of the vallecula (interexaminer, 16.1%). The greatest mean differences in area measurements were the cross-sectional area at the level of the vallecula (interexaminer, 13.3%). The volumetric mean differences were below 10% of the mean measurement value. The maximum measurement error was exceedingly high in relation to the maximum measurement of some variables: intraexaminer, widths at the levels of the soft palate (26.6%), tongue (42.4%), and vallecula (30.8%); and interexaminer, widths at the levels of the tongue (29.1%) and vallecula (26%); cross-sectional area at the level of the vallecula (31.1%); and minimum axial area (25.4%).
| Measurement | Intraexaminer | Interexaminer | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mean | SD | Minimum | Maximum | Mean difference | SD | Minimum | Maximum | Mean difference | SD | Minimum | Maximum | |
| Anteroposterior length (mm) at the level of | ||||||||||||
| Palatal plane | 20.70 | 2.91 | 17.06 | 26.61 | 0.48 | 0.45 | 0.00 | 1.88 | 0.49 | 0.39 | 0.03 | 1.39 |
| Soft palate | 10.35 | 1.88 | 8.38 | 13.76 | 0.25 | 0.26 | 0.00 | 1.15 | 0.30 | 0.19 | 0.00 | 0.75 |
| Tongue | 11.24 | 3.53 | 8.31 | 20.71 | 0.32 | 0.24 | 0.00 | 0.99 | 0.30 | 0.33 | 0.00 | 1.62 |
| Vallecula | 15.91 | 4.06 | 10.08 | 24.12 | 0.32 | 0.29 | 0.00 | 1.21 | 0.52 | 0.30 | 0.00 | 1.12 |
| Width (mm) at the level of | ||||||||||||
| Palatal plane | 26.83 | 7.49 | 15.23 | 37.40 | 0.48 | 0.60 | 0.00 | 3.26 | 1.16 | 1.23 | 0.06 | 4.28 |
| Soft palate | 21.90 | 8.08 | 10.03 | 35.38 | 1.13 | 1.89 | 0.00 | 9.43 | 2.52 | 2.75 | 0.11 | 8.35 |
| Tongue | 26.21 | 6.94 | 16.62 | 37.68 | 2.61 | 3.55 | 0.00 | 15.98 | 3.52 | 3.30 | 0.01 | 10.98 |
| Vallecula | 20.43 | 7.49 | 9.81 | 35.56 | 1.41 | 2.25 | 0.11 | 10.96 | 3.29 | 3.26 | 0.11 | 9.25 |
| Cross-sectional area (axial slice) (mm 2 ) at the level of | ||||||||||||
| Palatal plane | 485.67 | 154.65 | 263.35 | 825.44 | 14.99 | 11.58 | 0.05 | 38.78 | 19.94 | 14.20 | 0.18 | 52.86 |
| Soft palate | 224.45 | 83.85 | 107.62 | 405.44 | 10.25 | 11.74 | 0.18 | 48.66 | 18.75 | 17.96 | 1.23 | 78.33 |
| Tongue | 253.48 | 75.40 | 136.90 | 423.20 | 13.43 | 13.54 | 0.75 | 56.41 | 18.33 | 14.02 | 0.07 | 54.27 |
| Vallecula | 232.62 | 79.24 | 126.17 | 359.46 | 8.74 | 7.96 | 0.65 | 30.30 | 31.15 | 30.00 | 0.03 | 112.14 |
| Sagittal area (mm 2 ) | 701.01 | 147.80 | 498.50 | 886.40 | 18.28 | 13.96 | 0.30 | 60.90 | 31.31 | 30.29 | 0.50 | 142.00 |
| Minimum axial area (mm 2 ) | 198.41 | 61.52 | 103.10 | 291.20 | 8.33 | 14.92 | 0.00 | 53.40 | 12.79 | 23.28 | 0.00 | 74.20 |
| Volume (mm 3 ) | 17184.49 | 6238.42 | 7667.40 | 25512.40 | 464.68 | 423.78 | 11.30 | 1999.70 | 860.17 | 914.53 | 22.60 | 3492.20 |
Tables III and IV present Bland-Altman method statistics, including mean differences, standard deviations, 95% limits of agreement, and P value from paired t tests. These results show wider limits (proportionally to the measurements) for transversal width, cross-sectional area, and minimum axial area slices.
| Undergraduate student | t test P value | Orthodontist | t test P value | Radiologist | t test P value | ||||
|---|---|---|---|---|---|---|---|---|---|
| Mean (SD) | 95% LoA | Mean (SD) | 95% LoA | Mean (SD) | 95% LoA | ||||
| Anteroposterior length (mm) at the level of | |||||||||
| Palatal plane | 0.19 (0.57) | −0.93; 1.33 | 0.257 | 0.05 (0.57) | −1.06; 1.18 | 0.735 | −0.11 (0.82) | −1.72; 1.49 | 0.635 |
| Soft palate | −0.06 (0.25) | −0.56; 0.43 | 0.430 | −0.04 (0.51) | −1.05; 0.96 | 0.780 | −0.11 (0.29) | −0.69; 0.47 | 0.222 |
| Tongue | 0.16 (0.46) | −0.74; 1.06 | 0.249 | 0.20 (0.38) | −0.53; 0.95 | 0.100 | −0.10 (0.31) | −0.72; 0.51 | 0.274 |
| Vallecula | −0.09 (0.21) | −0.52; 0.32 | 0.148 | 0.13 (0.41) | −0.67; 0.94 | 0.273 | −0.12 (0.59) | −1.29; 1.05 | 0.488 |
| Width (mm) at the level of | |||||||||
| Palatal plane | −0.05 (0.39) | −0.83; 0.71 | 0.632 | −0.29 (1.13) | −2.50; 1.92 | 0.414 | 0.08 (0.65) | −1.20; 1.36 | 0.678 |
| Soft palate | 0.98 (1.40) | −1.77; 3.74 | 0.043 | −1.36 (3.12) | −7.48; 4.75 | 0.158 | 0.08 (0.41) | −0.73; 0.89 | 0.527 |
| Tongue | −0.44 (2.84) | −6.02; 5.14 | 0.616 | −2.45 (6.55) | −15.30; 10.40 | 0.222 | 0.20 (1.45) | −2.65; 3.06 | 0.653 |
| Vallecula | 0.37 (1.09) | −1.77; 2.52 | 0.261 | −0.34 (4.39) | −8.94; 8.26 | 0.793 | 0.18 (1.14) | −2.06; 2.43 | 0.597 |
| Cross-sectional area (axial slice) (mm 2 ) at the level of | |||||||||
| Palatal plane | −4.77 (19.14) | −42.29; 32.74 | 0.406 | −5.61 (19.01) | −42.87; 31.64 | 0.328 | −7.21 (17.85) | −42.20; 27.76 | 0.189 |
| Soft palate | 2.87 (20.32) | −36.96; 42.70 | 0.634 | 0.16 (17.59) | −34.32; 34.65 | 0.976 | −1.71 (7.36) | −16.13; 12.71 | 0.438 |
| Tongue | 2.42 (25.62) | −47.78; 52.64 | 0.749 | −1.77 (17.25) | −35.60; 32.05 | 0.740 | −6.73 (12.43) | −31.11; 17.64 | 0.087 |
| Vallecula | −3.65 (9.49) | −22.25; 14.94 | 0.209 | 1.66 (13.06) | −23.94; 27.27 | 0.681 | −4.67 (12.37) | −28.93; 19.57 | 0.217 |
| Sagittal area (mm 2 ) | −7.62 (26.51) | −59.58; 44.33 | 0.362 | −2.44 (29.54) | −60.35; 55.45 | 0.779 | −12.25 (25.55) | −62.34; 37.83 | 0.143 |
| Minimum axial area (mm 2 ) | 0.25 (0.71) | −1.14; 1.65 | 0.264 | 7.10 (29.66) | −51.03; 65.25 | 0.445 | −0.99 (2.24) | −5.39; 3.41 | 0.174 |
| Volume (mm 3 ) | −196.82 (648.61) | −1468.12; 1074.46 | 0.338 | −183.95 (949.42) | −2044.84; 1676.92 | 0.516 | −401.47 (672.43) | −1719.44; 916.49 | 0.076 |
Stay updated, free dental videos. Join our Telegram channel
VIDEdental - Online dental courses
