16

Assessment of Facial Movement

Hashmat Popat, Stephen Richmond, David Marshall, Paul L. Rosin, and Lanthao Benedikt

INTRODUCTION

Traditional craniofacial imaging techniques used in clinical assessment, diagnosis, and treatment planning include lateral cephalometry, photography, and more recently laser scanning and cone beam computed tomography. These techniques provide a static image of the facial structure and as such give a snapshot of static information at a given point in time. However, in most everyday life scenarios, we engage in functional activity. Although static imaging techniques continue to be routine in clinical practice, the focus is beginning to switch to imaging techniques that can accurately record and quantify facial movement.

Typically, dynamic facial movement has for the most part, received particular attention from the computer gaming, film and animation industries. However, three – dimensional (3D) facial movement can be of value to many clinical specialties, for example orthodontists, craniofacial surgeons, psychologists, and speech and language therapists. Specific clinical applications of facial movement would include the evaluation of functional surgical outcomes (e.g., following cleft lip and palate surgery and orthognathic surgery), the assessment of patients with motor nerve defects (e.g., facial nerve palsies), and the evaluation of psychomotor function associated with depression or pain. Facial expressions of emotion are also increasingly being used in neuroscience as probes for functional imaging and as stimuli for studying hemispheric specialization for face- and emotion-processing.

The technique of quantifying facial movement can be broadly termed “facial motion analysis/capture” or “dynamic facial imaging” and encompasses a variety of modalities from simple grading scales through to high-specification 3D imaging systems. Some of these systems have been labelled as “4D,” and may record sound as well as image data over time. A classification of techniques to analyze facial movement is proposed in Table 16.1.

Table 16.1 Motion analysis systems

2D, 3D, 4D, two-, three, and four- dimensional; MRI, magnetic resonance imaging.

SUBJECTIVE GRADING SCALES

Attempts to describe facial movement were described as early as the nineteenth century but suffered problems due to the lack of precise techniques to measure the face. For example, photographing subjects during electrical stimulation of selected facial muscles allowed the changing appearance of the face on contraction of the facial muscles to be studied. ¹ The reports, however, were essentially verbal descriptions of facial expression and did not attempt to characterize the entire face as a whole or to quantify selected facial expressions.²

The House – Brackmann Facial Nerve Grading System is a widely used subjective grading scale for reporting facial nerve paralysis (Table 16.2).³ Although the reliability of this system has been found to be high, large interobserver disagreements have been revealed.⁴ The main problem in an evaluation such as this is the lack of an objective measure of the spatial and temporal aspects of facial function.⁵

OBJECTIVE GRADING SCALES

An objective component was added to the House – Brackmann scale by measuring the movement of the eyebrow and corner of the mouth on both the side affected by facial nerve palsy and the normal side.³ The results can be expressed as a percentage of function, but the system is subject to examiner bias as no reference point is defined and placement of the measuring instrument is dependent on the observer.

A further objective scale, described as the Facial Nerve Function Index, uses a single expression, the broad smile, to represent facial function.⁶ The distance between the lateral canthi and the lateral oral commissure at rest is measured for the normal side and the side affected by facial nerve palsy separately. The same distances are measured with the broadest possible smile. The “at rest” distance minus the smile distance for the affected side is divided by the same calculation on the normal side and multiplied by 100, yielding the Facial Nerve Function Index. The system is simple, straightforward, and objective but lacks any information about velocity, acceleration, or synkinesis. It also has built-in observer error as it lacks a fixed reference point.²

Table 16.2 House- Brackmann facial nerve grading system

TWO – DIMENSIONAL MOTION ANALYSIS SYSTEMS

Although objective components have been incorporated into grading systems, many are still fieldspecific (i.e., primarily for the assessment of facial nerve defects) and therefore not appropriate for broader applications. In the 1990s, the development of modified two-dimensional (2D) imaging systems based on either photographs or cine camera films began the move towards true objective analysis of facial function.

Photographing subjects at maximal animation and at rest can quantify facial movement by measuring the amplitude of facial landmark motions.⁷ Physical markers are worn on selected landmarks and an adhesive ruler must be incorporated to allow for magnification error and measurement calibration. Although allowing quantification of movement in all regions of the face simultaneously, the 2D nature of the system means there is no representation of the actual path of motion of the facial landmarks.⁸

Cine cameras overcame the limitation of photographs and have been used to quantify the 2D trajectories of various lip landmarks during natural smiling and to report both the amplitude and the direction of the landmark motions.⁹ Ultimately, the limitations of 2D imaging in representing a 3D structure are well documented, and similarly when measuring amplitudes, it has been reported that 2D measurements can significantly underestimate 3D amplitudes.¹⁰

THREE – DIMENSIONAL MOTION ANALYSIS SYSTEMS

Motion capture systems are an established technique used in the field of gait analysis. They provide kinematic (measurement of the movement of the body in space) and kinetic (on the forces involved in producing the movements) information. Landmarks are placed on anatomic points and “tracked” during movement using multiple cameras. As the subject moves (e.g., walks on a treadmill), trajectories of the markers are calculated in 3D, thereby providing 3D motion analysis. In gait analysis, this information usually concerns joint mobility. More recently, 3D facial motion capture systems have evolved by means of the same principle.

There are, however, different demands and approaches between facial motion and body motion capture. With body motion capture, the movements of a structured bony framework are analyzed, which primarily involve generic, fluid movements. Facial motion capture on the other hand has a wider range of movements and expressions, and in addition lifelike nuances and subtle variations. These movements are often less than a few millimeters and therefore require greater resolution and fidelity than are usually used in body motion capture.

Nonetheless, the main systems that have dominated the field of facial motion capture are the “marker based tracking systems,” which are similar to body motion capture systems. Using the concept of stereophotogrammetry, they consist of groups of one or more cameras that are connected to a computer and track multiple landmarks placed on the subject’ s face. Several of these systems have been described as in-house developments using either “passive-” or “active-”based markers.

Passive marker – based tracking systems

Passive marker – based systems use markers coated with a retroreflective material to reflect back light that is generated near the cameras lens. The camera’ s threshold can be adjusted so only the bright reflective markers will be sampled, ignoring skin and fabric. The facial markers can vary in diameter from between 2 and 6 mm, can number up to 30, and are placed on specific facial landmarks.

To obtain 3D coordinate data for a marker, two cameras must record the marker’ s position in space. As markers on the face may be carried outside the field of view of the two primary cameras, additional cameras can be used to ensure that data from at least two cameras are always recorded. Prior to recording facial movements, the cameras must be calibrated by way of an object with an array of markers whose positions in space are certified to a known accuracy.

For example, C3D (Glasgow Dental Hospital/Faraday Laboratory, University of Glasgow, UK) is a stereo-photogrammetric camera system¹¹ based on two pairs of video cameras placed either side of the subject with integral illumination. Preliminary investigations with the system using a dummy head indicated that it had an accuracy of 0.1 mm.¹² The main study recruited subjects who were highlighted with 20 landmarks on their face. Images were captured while performing a sequence of five facial expressions (rest, natural smile, maximal smile, lip purse, and maximal cheek puff) while maintaining natural head posture. The authors concluded that the extent of facial expression reproducibility was expressionspecific and that differences existed between males and females.

The Motion – Analysis system (Motion Analysis, Santa Rosa, CA, USA) uses analog video cameras with a focal length of 25 mm to capture data at a sampling rate of 60 frames per second (fps). Spherical retroreflective markers of between 2 and 4 mm diameter were attached to designated facial landmarks on subjects. Subjects performed a variety of maximal facial animations including smile, lip purse, cheek puff, eye closure, eye opening, and grimace. Among the findings of these particular studies has been the modeling of facial movement in patients with cleft lip and palate,¹³ the influence of gender and facial shape in the 3D analysis of facial movement in normal adults,¹⁴ and the dynamic analysis of differences caused by orthognathic surgery.¹⁵

Vicon (Vicon 250; Vicon Motion Systems, Oxford, UK) tracks the motion of 2 mm reflective markers placed on the face at 60 fps using five infrared video cameras, and has been used to quantify spontaneous facial motility in infancy.¹⁶ Four separate reference markers were placed on the forehead to correct for head movement that would otherwise be included in the facial movement signals. Image- processing software identified the marker locations in each 2D infrared camera image to compute its 3D location relative to a calibration plate that was positioned in the data collection room. The system has been documented for its usefulness in research applications,¹⁷ but has also been reported to be too complicated for daily clinical application.¹⁸

FACIAL CLIMA (Sports Training Technologies, S.L., San Sebastian, Spain) has been described most recently; this uses marker- based tracking with infrared cameras on healthy subjects and has been proposed in the assessment of the functional outcome of facial paralysis reanimation surgery.¹⁹

Active marker/>