A study to evaluate the reliability of using two-dimensional photographs, three-dimensional images, and stereoscopic projected three-dimensional images for patient assessment


Clinicians are accustomed to viewing conventional two-dimensional (2D) photographs and assume that viewing three-dimensional (3D) images is similar. Facial images captured in 3D are not viewed in true 3D; this may alter clinical judgement. The aim of this study was to evaluate the reliability of using conventional photographs, 3D images, and stereoscopic projected 3D images to rate the severity of the deformity in pre-surgical class III patients. Forty adult patients were recruited. Eight raters assessed facial height, symmetry, and profile using the three different viewing media and a 100-mm visual analogue scale (VAS), and appraised the most informative viewing medium. Inter-rater consistency was above good for all three media. Intra-rater reliability was not significantly different for rating facial height using 2D ( P = 0.704), symmetry using 3D ( P = 0.056), and profile using projected 3D ( P = 0.749). Using projected 3D for rating profile and symmetry resulted in significantly lower median VAS scores than either 3D or 2D images (all P < 0.05). For 75% of the raters, stereoscopic 3D projection was the preferred method for rating. The reliability of assessing specific characteristics was dependent on the viewing medium. Clinicians should be aware that the visual information provided when viewing 3D images is not the same as when viewing 2D photographs, especially for facial depth, and this may change the clinical impression.

Following the clinical assessment of patients with dentofacial deformities, additional records including pre-treatment extraoral photographs, radiographs, and study models are recommended. These are used to document the initial presentation, for diagnosis and treatment planning, as well as providing evidence of treatment. The records also have a role in patient communication, education, audit, and research. Pre-treatment records should therefore be valid, accurate, complete, and contemporaneous. Given the longitudinal nature of orthognathic surgery, the pre-treatment records should enable the different members of the orthognathic team to re-identify the key features of the clinical assessment, in the absence of the patient. One of the assumptions of present clinical practice is that the extraoral photographs are valid in this context.

Using stereophotogrammetry or laser scanning it is possible to capture the photorealistic three-dimensional (3D) facial soft tissue appearance. Clinicians are accustomed to viewing conventional two-dimensional (2D) photographs and assume that working with and viewing 3D images will be similar. However, even though the individual’s facial images are captured in 3D, they are currently not viewed in ‘true 3D’. This may result in underutilization of the additional information acquired during 3D capture and may be clinically important.

Currently the 3D facial images are viewed after they have been reconstructed (rendered) onto a 2D plane, i.e. the screen of an LCD computer monitor. Image depth perception is created using monocular cues, such as the size differential generated by perspective projection and the texture gradient by lighting and shading. These perspective techniques are not able to provide a true perception of depth as experienced by the human visual system, i.e. binocular vision. This can only be re-created using ‘stereoscopic viewers’, such as active or passive 3D projection systems. Passive 3D projection systems project two different images with a spatial disparity onto a screen; each eye views each image at the same time using polarized glasses. The differences between these two images (known as stereo-pairs of images) are recovered by the human visual system as depth. In a medical context, stereoscopic displays offer significant clinical improvements, especially in the areas of diagnostic applications including ophthalmic imaging, mammography, vascular imaging, and orthopaedic imaging.

There appears to have been no study investigating the reliability of using 2D photographs and 3D images (both relying on monocular cues) or stereoscopic projected 3D images (binocular cues) to assess specific features of a facial deformity. Therefore the aim of this study was to evaluate the reliability of using these three different viewing media to rate specific facial characteristics in a group of pre-surgical class III patients: facial height, symmetry, and profile. The null hypothesis was that there are no differences in ratings of the facial height, symmetry, and profile (as assessed using a visual analogue scale (VAS) score) using the three different viewing media ( P < 0.05). In addition, the raters were asked supplementary questions to appraise which viewing medium provided the most information during the assessment.

Materials and methods


Ethical approval for the study was obtained from the Institutional Review Board (IRB) of Hong Kong University and Hospital Authority Hong Kong West Cluster. Forty pre-surgical orthognathic patients attending the Department of Orthodontics or the Department of Oral and Maxillofacial Surgery of Prince Philip Dental Hospital, Hong Kong, were recruited. All patients had previously been diagnosed with a skeletal class III deformity by the orthognathic team and required surgical correction. Patients with craniofacial syndromes or anomalies, including cleft lip and palate, were excluded. Written informed consent was obtained from all patients included in the study. The average age of the patients was 22.9 ± 3.2 years; 22 were female and 18 were male.

Image acquisition

Each patient was imaged using a conventional digital camera (2D images) and a 3D stereophotogrammetry system (3D images). To standardize the images, hair was kept away from the face, the lips were kept in repose, and the head was maintained in the natural head position (NHP).

2D images

Seven standardized 2D photographs of each patient (frontal view, right and left profiles, right and left three-quarter profiles, bird’s eye view, and worm’s eye view) were captured using a Canon EOS 700D camera (Canon, Tokyo, Japan) with Canon Macro Ring Lite MR-14EX II and Canon Macro Lens EF-S 60 mm 1:2.8 USM (Canon). All photographs were taken using the same camera settings, illumination, background, and distance. A vertical plumb line was used as the true vertical reference. Using Adobe Photoshop CS (Adobe Systems Inc., San Jose, CA, USA), each facial image was cropped to remove the headband, ears, neck, and shoulders, producing a standardized image. The photographs for each patient were organized onto one PowerPoint slide (Microsoft, Redmond, WA, USA) and saved ( Fig. 1 ). This was repeated for each patient using the same template PowerPoint slide.

Fig. 1
Seven two-dimensional images arranged in a PowerPoint slide for 2D image viewing.

3D images

Immediately following 2D photographs, 3D facial images were taken using the Di3D system (Dimensional Imaging Ltd, Glasgow, UK) based on the standardized capture protocol and saved as Wavefront files (OBJ). Using MeshLab software (STI-CNR, Rome, Italy; meshlab.sourceforge.net/ ), each facial image was cropped to remove the headband, ears, neck, and shoulders, producing a standardized image. The 3D facial image was then reoriented to NHP according to a previously published technique and saved in OBJ format ( Fig. 2 ).

Fig. 2
Three-dimensional image used for rating of 3D image.

Viewing method

2D images and 3D images

The 2D photographs were viewed as PowerPoint slides and the 3D facial images in NHP were viewed using Di3DView (Dimensional Imaging Ltd). Both were projected onto a smart board (SMART Technologies, Calgary, Canada) by a projection system (EB-480; Epson, Suwa, Nagano, Japan) during the rating sessions.

Stereoscopic projected 3D images

The 3D facial images in NHP were imported into 3D-Hub software (WSP av , Colchester, Essex, UK; Instant Effects, Santa Barbara, CA, USA), and projected onto a large projector screen (Da-Lite Versatol Tripod Screen 99 in. (70 × 70) Silver Lite 2.5; Da-Lite, Warsaw, IN, USA) using a passive 3D projection system (EB-W16SK; Epson) ( Fig. 3 ). The stereoscopic projected 3D facial images were viewed using a pair of polarized 3D glasses.

Fig. 3
(A) Stereoscopic projector setup used to display the 3D image. (B) Horizontal disparity used to create stereoscopic vision, as seen with no glasses on.


Eight raters (with a Membership in Orthodontics diploma (MOrth) or equivalent) from the Department of Orthodontics at Prince Philip Dental Hospital, Hong Kong, were asked to rate the facial images. All raters were involved in the management of dentofacial deformities and were acquainted with orthognathic surgery.

Rating scale

The raters assessed the severity of the facial deformity for 40 patients using a 100-mm VAS. The descriptor at the left end of the VAS (0 mm) was ‘too short’, ‘acceptable symmetry’, and ‘acceptable’ for facial height, facial symmetry, and profile, respectively, whilst on the right (100 mm) the descriptor was ‘too long’, ‘very asymmetric’, and ‘very severe’, respectively ( Fig. 4 ). Each rating sheet was completed by each rater for only one patient using each viewing medium.

Fig. 4
The visual analogue scale (VAS) for assessing facial height, facial symmetry, and profile.

Rating process

Verbal introduction

All image viewings were undertaken by all raters under the same conditions. Prior to commencing the main study, all raters were calibrated by explaining the rating scale. In addition, the raters were shown the 2D photographs of the 40 patients in a random order prior to each rating session to familiarize them with the range in severity of facial height, asymmetry, and class III profile, so they could use the fullest possible extent of the VAS scale.

Initial rating session (T1)

The raters assessed the 2D photographs, 3D images, and stereoscopic projected 3D images alternately, i.e. 10 2D photographs, 10 3D images, and finally 10 stereoscopic projected 3D images, under standardized viewing conditions. This was repeated until the images of all 40 patients had been rated using the three viewing media. The orders of 2D photographs, 3D images, and stereoscopic projected 3D images were chosen at random.

During the 3D assessments, the same operator (SZ) manually rotated the image in the same manner. During the rating session, raters were not allowed to go back to the previous rating sheet.

The raters were asked two additional questions: (1) Which viewing medium provided the most information during the assessment? (2) What percentage of the class III discrepancy is due to maxillary retrusion?

Re-rating session (T2)

The same eight raters reassessed the 2D photographs, 3D images, and stereoscopic projected 3D images of 20 randomly selected patients 1 week later. In total, 60 images (20 2D photographs, 20 3D images, and 20 stereoscopic projected 3D images) were assessed in series of 10, as in the initial rating session.

Statistical analysis

A Shapiro–Wilk test indicated that the data were not normally distributed. Intra-rater reliability was assessed using intra-class correlation (ICC). A Wilcoxon signed-rank test was used to detect any significant differences in the median VAS between the two rating sessions (T1 and T2).

Cronbach’s alpha was used to test the internal consistency of the VAS for inter-rater reliability. Comparison of the three viewing media to assess the severity of facial height, symmetry, and profile was evaluated by performing a Kruskal–Wallis test on the mean rater score (mean of scores at T1 and T2). The same test was used to determine whether there was a statistical difference in the percentage of the class III discrepancy due to maxillary retrusion between the three viewing media.


The inter-rater reliability was good to excellent (Cronbach’s alpha ≥0.8) for the three different viewing media when assessing the three components of the face, i.e. height, symmetry, and profile ( Table 1 ). Cronbach’s alpha ranged from 0.82 for assessing facial height using 3D images to 0.97 for assessing symmetry using projected 3D images.

Table 1
Inter-rater reliability for the three characteristics of class III severity (height, symmetry, and profile) in 2D, 3D, and stereoscopic projected 3D; Cronbach’s alpha.
Viewing medium Height Symmetry Profile
2D 0.87 0.86 0.88
3D 0.82 0.93 0.90
Projected 3D 0.88 0.97 0.89
2D, two-dimensional; 3D, three-dimensional.

All ICC values for assessing intra-rater reliability were between 0.48 and 0.76, representing moderate to good levels of reliability. There was a significant difference in median VAS when rating facial symmetry ( P = 0.001) and profile ( P = 0.001), but not facial height ( P = 0.704) using 2D photographs. For 3D imaging, there was no significant difference for rating symmetry ( P = 0.056), but there was for rating facial height ( P = 0.004) and profile ( P = 0.018). Finally, based on projected 3D images, there were significant differences in ratings for facial height ( P = 0.001) and symmetry ( P = 0.001), but not profile ( P = 0.749) ( Table 2 ).

Table 2
Intra-rater reliability for rating the height, symmetry, and profile using 2D, 3D, and stereoscopic projected 3D. The median differences in VAS scores between T1 and T2 are shown, together with the intra-class correlation coefficients and outcomes of the Wilcoxon signed-rank test.
ICC Median difference IQR P -value
Height 0.61 0.0 −7.1 to 6.1 0.704
Symmetry 0.57 −6.3 −18.0 to 4.5 0.001 a
Profile 0.73 −3.0 −12.6 to 3.1 0.001 a
Height 0.63 −2.5 −8.6 to 3.5 0.004 a
Symmetry 0.76 −2.8 −11.5 to 7.0 0.056
Profile 0.70 −3.0 −13.0 to 5.5 0.018 a
Projected 3D
Height 0.55 −3.5 −9.5 to −1.0 0.001 a
Symmetry 0.48 −4.0 −21.5 to 5.6 0.001 a
Profile 0.71 0.0 −10.1 to 9.6 0.749
Only gold members can continue reading. Log In or Register to continue

Dec 14, 2017 | Posted by in Oral and Maxillofacial Surgery | Comments Off on A study to evaluate the reliability of using two-dimensional photographs, three-dimensional images, and stereoscopic projected three-dimensional images for patient assessment
Premium Wordpress Themes by UFO Themes