Identification and appraisal of outcome measures used to evaluate hypodontia care: A systematic review


Identification and appraisal of the outcome measures that have been used to evaluate hypodontia treatment and deliver services are essential for improving care. A lack of alignment between outcomes and patient values can limit the scope for patient-centered care. Our objectives were to identify and appraise the outcomes selected to evaluate hypodontia care.


Data sources included 10 electronic databases and grey literature, searched using terms for hypodontia and its treatment methods. Study eligibility included mixed study designs to ensure comprehensive identification of outcomes, excluding case reports and case series with fewer than 10 participants and nonsystematic reviews. Participants and interventions involved people with hypodontia receiving any dental treatment to manage their hypodontia. Simulated treatment, purely laboratory-based interventions, and future treatments still in development were excluded. Research outcomes were identified and synthesised into 4 categories: clinical indicators, and patient-reported, clinician-reported, and lay-reported outcomes. No synthesis of efficacy data was planned, and consequently no methodologic quality appraisal of the studies was undertaken.


The search identified 497 abstracts, from which 106 eligible articles were retrieved in full. Fifty-six studies and 8 quality-improvement reports were included. Clinical indicators were reported in 49 studies (88%) including appearance, function, dental health, treatment longevity, treatment success and service delivery. Patient-reported outcomes were given in 22 studies (39%) including oral health-related quality of life, appearance, function, symptoms of temporomandibular dysfunction, and patient experience. Clinician-reported outcomes were limited to appearance. Variability was seen in the tools used for measuring outcomes.


There is a lack of rationale and consistency in the selection of outcome measures used to evaluate hypodontia care. Outcomes are largely clinician and researcher-driven with little evidence of their relevance to patients. There was a paucity of outcomes measuring access to care, quality of care, and cost. Evidence from hypodontia research is clinician-focused and likely to have limited value to support patients during decision making. Attempts to synthesise the evidence base for translation into practice will be challenging. There is a need for a core outcomes set with a patient-centric approach to drive improvements in health services.


  • Rationale and consistency are needed in outcome measures evaluating hypodontia care.

  • Research is clinician- and researcher-driven, with limited relevance to patients.

  • Health service research outcomes are scarce: eg, access, quality, and cost.

  • Involving patients in research design will lead to more appropriate outcomes.

  • Health service research outcomes should be considered in future studies.

Hypodontia affects the permanent dentition in 4% to 6% of the population, with substantial impacts on functional, emotional, and social well-being, and oral health-related quality of life. The variation in severity and presentation of hypodontia result in a range of care (treatment options and configuration of care pathways for service delivery) that is often complex, multidisciplinary, and delivered over a protracted time period. Adopting an evidence-based approach with patient-centered outcomes is important for shared decision-making, to allow patients to choose care that is consistent with their values. Inconsistency in outcome measures presents challenges for comparing treatment efficacies between studies, while outcomes that fail to consider what is important to patients will contribute to a clinician-directed process. The orthodontic community needs to encourage the use of consistent outcomes across all types of research and service evaluations and strive for outcomes that have the most value to patients, clinicians, and stakeholders. This will enable shared decision making, improve satisfaction and adherence to care, and direct effective use of health resources.

Our preliminary search of the literature identified 4 previous reviews relating to specific presentations of hypodontia or treatment, but no reviews that examined the outcomes used for evaluating hypodontia care. Despite an abundance of studies evaluating hypodontia care, there appeared to be little consistency in the outcomes chosen to measure effectiveness. This corresponds with a recent systematic review of orthodontic trials with children, in which significant limitations were found in the choice of research outcomes and a focus on physical changes caused by treatment. The authors highlighted the need to determine important and relevant outcomes from the perspective of all stakeholders, including patients and clinicians.

In this study, we aimed to identify and appraise which outcome measures have been used to evaluate hypodontia care. The objectives were to (1) identify studies that evaluate hypodontia care, where hypodontia care is defined as both treatment methods and service delivery; (2) classify the outcomes used in the studies; and (3) appraise the outcomes in terms of patient relevance and ability to improve clinical care.

Material and methods

This systematic review was conducted using best-practice principles and with guidance from previous reviews exploring research outcomes. A preliminary scope of the literature confirmed that no reviews were planned or published on this topic. Stakeholder opinion was sought from providers of hypodontia care and people with hypodontia in Leeds Dental Institute, Leeds, United Kingdom, during protocol development to maximize the relevance and applicability of this review. The protocol, published on PROSPERO in May 2015, was devised with a broader scope from which the identification of hypodontia care outcomes was 1 component.

The search strategy was deployed across 6 electronic databases using search terms related to hypodontia and treatment methods, plus 4 further electronic databases searched using terms for hypodontia only ( Table I ). No language restrictions were used. Additional grey literature searching was performed for unpublished or nonacademic literature. Hand searching of reference lists of relevant publications and reviews, citation searching for relevant papers identified in searches, and personal contacts via e-mail with the members of 2 dental specialty societies were also undertaken. Search terms were developed through piloting alternative search concepts based on population, intervention, and outcome to estimate sensitivity and specificity. Two concepts were used for searching based on MeSH terms and free headings for hypodontia and treatment methods ( Supplemental Table I ).

Table I
Search strategy for identifying empirical research reports
Search strategy
Electronic databases
(Search date, December 19, 2016)
  • 1.


  • 2.

    Medline via Ovid (1946 onward)

  • 3.

    EMBASE via Ovid (1947 onward)

  • 4.


  • 5.

    Web of Science

    • a.

      Science Citation Index and

    • b.

      Conference Indexes

  • 6.

    Dissertations and theses database

Search terms for hypodontia AND treatment methods
Search terms for hypodontia only
Grey literature searches
  • Hand searching reference lists of relevant publications and reviews

  • Additional citation searching for relevant papers identified in searches

  • Personal contact via e-mail to the members of 2 dental specialty societies (British Orthodontic Society, Restorative Dentistry UK) to identify unpublished academic work

  • Hand searching of the clinical effectiveness bulletins of dental specialty societies

We developed inclusion and exclusion criteria for study selection ( Table II ). Studies including participants with syndromic hypodontia and cleft lip and palate were excluded due to potential differences in treatment options and care provision. Mixed-method study designs were included to ensure comprehensive identification of outcomes relating to care evaluation. Quality improvement methods such as audit and service evaluations, although distinct from research, were included as potential sources of information about stakeholder priorities and expected care standards. These records were managed separately during data extraction and synthesis.

Table II
Inclusion and exclusion criteria for study selection
Included Excluded
Population People with hypodontia People with syndromic hypodontia
No restrictions on participants based on age, ethnicity, or severity of hypodontia Treatment for tooth loss with etiology other than hypodontia
Intervention Any type of dental treatment undertaken to manage hypodontia:

  • Management of retained deciduous teeth

  • Orthodontic treatment to redistribute space, alone or in combination with restorative procedures to aid/eliminate tooth replacement (fixed appliances, removable appliances)

  • Restorative treatment for tooth replacement (fixed prostheses, removable prostheses, implant-supported restoration) or tooth camouflage

  • Tooth autotransplantation

Any aspect of hypodontia service delivery

Simulated treatment such as treatment prediction using computer software and hypothetical treatments
Purely laboratory-based interventions
Future treatments still in development: eg, biologic methods for growing and replacing missing teeth
Comparator For comparative studies, any treatment listed above or no treatment
Noncomparative studies were eligible for inclusion
Outcomes Any evaluation of hypodontia care (treatment methods or service delivery) Etiology of hypodontia
Prevalence of hypodontia
Study design Systematic reviews with data synthesis that produced primary summary data (eg, meta-analysis) Nonsystematic reviews

  • Reviews without evidence synthesis

  • Expert opinion

Experimental designs

  • Randomized controlled trials

  • Quasi-experimental studies

Case series with less than 10 participants or single case reports
Observational studies

  • Comparative or noncomparative

  • Prospective or retrospective

Conference abstracts with inadequate information regarding methodology and results
Economic studies
Qualitative studies
Quality improvement reports

  • Audit

  • Service evaluation

Study characteristics Studies originating from any country Studies published before 1970
Non-English publications Studies not available in full text in English after reasonable attempts to obtain or translate

Titles identified through the electronic database searches were collected and imported into EndNote X4 (Clarivate Analytics, Philadelphia, PA). Programmed deduplication was undertaken and checked manually using a sample of studies to ensure that the automated process was reliable. References were screened initially by title by a reviewer (S.B.) with verification of excluded titles by a second reviewer (B.K.). No errors were identified. Abstracts for all relevant titles were retrieved and evaluated independently by 2 reviewers (S.B. and B.K.). Full-text articles were obtained for studies meeting the inclusion criteria or when it was not possible to make a decision based on the abstract. Potential articles found through grey literature searches were included at this stage. Two reviewers (S.B. and B.K.) reviewed the full-text articles independently, and disagreements were resolved by consensus. Excluded studies were indexed in Excel (version14.7.4; Microsoft, Redmond, Wash) with the reason for exclusion.

A data extraction form modified from the Cochrane Public Health Group template was used to systematically extract data from each study. Coding was piloted for suitability before the review and developed through an iterative process until the form was judged fit for purpose. The following data items were extracted: author, date, design, setting and country, participants, interventions, comparator, outcome, measurement tools, reference for tool, and follow-up period. Data extraction was completed by 1 reviewer (S.B.) and checked for accuracy and consistency by a second reviewer (B.K.). Corrections were recorded to check for areas of ambiguity that would indicate internal inconsistency.

Synthesis of the studies was undertaken using a novel categorization approach based on the perspective of the evaluator in the study. Four outcome categories were developed: 3 subjective outcomes grouped by assessor (patient-reported, clinician-reported, and lay-reported) and 1 objective outcome (clinical indicators). Within each outcome category, themes and subthemes were used to separate the outcomes. For example, the patient-reported outcome category included themes for quality of life, appearance, function, and service delivery. In the appearance theme, the outcomes were further divided into subthemes including dental appearance and smile attractiveness. This coding theme was adopted to emphasize the use of patient-centered outcomes and to aid future work in developing outcomes of relevance to patients, clinicians, and stakeholders. This differs from the method previously reported in reviews of research outcomes where the outcome domains concerned outcomes from treatment in terms of efficacy.

In line with a previously published review exploring research outcomes, no synthesis of efficacy data was planned. Consequently, a methodologic quality appraisal of studies was not undertaken because the use of outcomes might not be associated with the quality of the studies.


Electronic searches were initially completed on September 1, 2015, and were updated on December 19, 2016. The PRISMA flow diagram of study selection is shown in the Figure . Totals of 56 research records and 8 quality improvement records were included in the review ( Supplemental Tables II-IX ). The research reports included 1 randomized and 2 nonrandomized controlled trials and 53 observational studies , , , ( Table III ). Research activity in hypodontia has grown with a noticeable increase in publications from 2000 onward. The majority of studies (43 of 56 [78%]) originated from Europe. The research teams had dental backgrounds and were based in a university or hospital with a few based in private practices. All studies were published in dental journals. Details of the key characteristics of the studies are summarized in the Supplemental Tables, grouped by intervention. Studies evaluated a single treatment method (n = 41) or compared methods (n = 12). Only 3 studies evaluated service delivery rather than outcomes from specific treatment methods ( Table IV ). The 8 quality-improvement reports were clinical audits undertaken in National Health Service hospitals and published between 2002 and 2016. The purpose of the audits was evaluation of service delivery including treatment duration, clinical efficiency, treatment planning, record keeping, and access to care ( Table V ).

Flow diagram of the study selection process.

Table III
Summary of included studies by design
Study design Number of studies
Randomized controlled trial 1
Nonrandomized controlled trial 2
Comparative 16
Noncomparative 23
Comparative 8
Noncomparative 6
Quality improvement
Audit 8

Table IV
Synthesis of research studies by intervention with an indication of the stated purpose
Topic Studies (n) Purpose of the study
Management of deciduous teeth 11 Evaluate outcome from treatment

  • Longevity

  • Dental health

  • Success

Description of treatment technique

Orthodontic space closure 3 Evaluate outcome from treatment

  • Dental health

  • Success

Tooth-supported tooth replacement 5 Evaluate outcome from treatment

  • Survival

  • Success

  • Technical complications

  • Oral health-related quality of life

Implant-supported tooth replacement 19 Evaluate outcome from treatment

  • Survival

  • Success

  • Dental health

  • Appearance

  • Oral health-related quality of life

Evaluate outcome from preparatory treatment

  • Stability

  • Success for facilitating future implant treatment

Tooth autotransplantation 3 Evaluate outcome from treatment

  • Survival

  • Dental health

Comparison between treatments 12 Evaluate outcome from treatment

  • Appearance

  • Dental health

  • Success

  • Cost-effectiveness

Service delivery 3 Patient experience of treatment
Description of service delivery model

Table V
Summary of outcomes used in audit to evaluate hypodontia care (n = 7)
Outcome domain Subdomain Audits using outcome (n)
Service delivery Treatment duration 3
Clinical efficiency

  • Appropriate use of clinic slots

  • Clinic organization

  • Availability of clinical records for consultation

  • Time waiting to be seen for consultation/treatment

Treatment planning

  • Confirmation of plan

  • Confirmation of treatment provider

  • Adherence to treatment plan

Record keeping 2
Access to care 1

Clinical indicators

Clinical indicators were the most popular outcome, used in 49 (88%) research studies ( Table VI ) and in all the audits for evaluating service delivery. The experimental studies used clinical indicators alone or in conjunction with patient-reported evaluations of appearance to measure the success of treatment. Dental health, treatment longevity, and treatment success were measured exclusively with clinical indicators. Variability was found in the choice of measurement tools ( Table VII ). For some outcomes, such as technical and biologic complications, implant success, and subthemes relating to dental health, the precise measurement method varied across the studies due to inconsistency in the definition of the outcome. In comparison, easily defined outcomes—eg, treatment survival—were more uniform in the measurement tool used. Interestingly, service delivery was evaluated with clinical indicators in only 3 studies, and none reported the involvement of health-service researchers or stakeholders.

Table VI
Categorization of research studies (n = 56) based on outcome
(number of studies)
(number of studies)
(number of studies)
Patient-reported outcome (n = 22) Quality of life (n = 6) Oral health-related quality of life (n = 6)
Appearance (n = 5) Smile attractiveness (n = 3)
Dental appearance (n = 4)
Function (n = 4) Mastication (n = 3)
Functional disturbance (n = 1)
Harms (n = 4) Temporomandibular dysfunction or parafunction (n = 4)
Service delivery (n = 6) Satisfaction with hypodontia care (n = 6)
Patient experience (n = 3)
Clinician-reported outcomes (n = 5) Appearance (n = 5) Smile attractiveness (n = 3)
Dental appearance (n = 2)
Lay-reported outcomes (n = 5) Appearance (n = 5) Smile attractiveness (n = 3)
Dental appearance (n = 2)
Clinical indicators (n = 49) Appearance (n = 8) Esthetic parameters (n = 1)
Appearance of implant restoration (n = 5)
Soft tissue profile (n = 1)
Function (n = 1) Mastication (n = 1)
Dental health (n = 2) Hard tooth tissue health (n = 13)
Gingival health (n = 14)
Periodontium (n = 20)
Occlusion (n = 10)
Treatment longevity (n = 22) Survival of treatment (n = 22)
Treatment success (n = 29) Success of treatment (n = 8)
Complications (n = 14)
Proxy markers of treatment success (n = 11)
Service delivery (n = 3) Treatment duration (n = 1)
Cost effectiveness (n = 1)
Efficiency (n = 1)
Studies may use multiple outcomes and therefore be included more than once.

Table VII
Measurement tools used for evaluating outcomes in hypodontia care
Outcome theme Subtheme Measurement tools
Quality of life Oral health-related quality of life Qualitative interviews
OHIP-49 or OHIP-20
Appearance Smile attractiveness Rating photographs with visual analog scale
Modified Eastman Esthetic Index
Ranking photographs using 10-point numeric scale
Rating photographs using 5-point Likert scale
Choice between pairs of photographs
Dental appearance Interview questions
OHIP-49 (esthetic domains)
Visual analog scale with interview
Rating photographs with bipolar adjective scale
Rating parameters of appearance of restoration
Esthetic parameters
Rating own satisfaction with dental appearance using visual analog scale
Appearance of implant restoration Copenhagen Index Score
Pink Esthetic Score and White Esthetic Score
Gingival zenith score
Soft tissue profile Cephalometric measures
Function Mastication OHIP-49 (functional domains)
Mastication Index
Mandibular Function Impairment Questionnaire
Masticatory ability
Functional disturbance Functional disturbance
Harms Temporomandibular dysfunction or parafunction Modified HELMIKO Index
Questionnaire regarding TMD symptoms
TMD signs
Service delivery Satisfaction with hypodontia care Questionnaire
Qualitative interviews
Patient experience Interviews
Treatment duration Average treatment time
Cost effectiveness Cost of treatment and maintenance
Clinic efficiency Clinic running time
Agreement of treatment plan
Dental health Hard tooth tissue health Coronal pathology
Pulp pathology
Root pathology
Gingival health Gingival Index/gingival health
Irritant Index
Presence of plaque
Probing depth
Bleeding on probing
Papilla Index Score
Periodontium Marginal bone levels
Ankylosis or infraocclusion
Tooth mobility
Occlusion Tipping of adjacent teeth
Occlusal contacts
Parameters of occlusion
Parameters of functional occlusion
Treatment success Survival of treatment Survival of primary tooth
Transplant survival
Implant survival
Prosthesis survival
Success of treatment Space closure
Implant success
Root development
Complications Technical complications
Biologic complications
Proxy markers of treatment success Alveolar bone volume
Bone gain from augmentation
Root approximation
Skeletal and dental parameters

Patient-reported outcomes

Patient-reported outcomes were mentioned in 22 (39%) studies, most commonly with clinical indicators ( Table VI ). No audits gave patient-reported outcomes. Three studies used exclusively patient-reported outcomes. Oral health-related quality of life and patient satisfaction and experience were measured in relation to a specific treatment or to hypodontia care as a whole. The Oral Health Impact Profile was the most popular patient-reported measurement tool, used in full or limited to specific domains relating to function and appearance. On the other hand, the questionnaires used to measure patient satisfaction in 4 studies were each developed for that study with little uniformity between the questionnaires. Patient-reported outcomes relating to smile attractiveness, dental appearance, masticatory ability, and functional disturbance were used to determine patient perceptions of success after specific hypodontia treatments. Symptoms of temporomandibular dysfunction were used as a marker of harm from treatment. No studies indicated patient involvement during development of the outcomes or selection of the measurement tools.

Clinician-reported and lay-reported outcomes

Clinician-reported and lay-reported outcomes were limited to outcomes relating to appearance: specifically, smile attractiveness and dental appearance. A wide variety of measurement tools were used for clinician evaluation of appearance, with consistency only seen in studies by the same research teams.


Patient-centered delivery of care and shared decision-making in treatment planning require provision of information that is useful and relevant to patients, yet this review illustrates inadequacies in the selection of appropriate outcome measures. People with hypodontia and their families are often committing to long courses of complex treatment and to enable effective care delivery, patients need to understand likely treatment consequences based on evidence with patient-centered outcomes. This review is unique in categorizing the outcome measures that are currently being used to evaluate treatment methods and service delivery in hypodontia. Understanding the characterization of potential outcomes and providing a system for categorization will assist future development of outcomes that satisfy all stakeholders in future studies.

The various outcomes identified from existing research predominantly assessed the efficacy of specific treatments, yet few studies indicated how the selection of outcomes and subsequent findings were expected to translate into improvements in clinical care. The lack of consistency in outcome measures limits the scope for comparing and combining study findings. Currently, the choice of outcomes appears to be largely driven by dental professionals and researchers. No study reported patient or public involvement in research design and selection of outcomes, and few teams included health-services research methodologists. This observation has 3 key implications: (1) the information gained from research may have limited value for patients during decision making; (2) there will be challenges when attempting to synthesize the evidence base for translation into practice; and (3) the research findings are unlikely to drive improvements in health services.

Clinical indicators were dominant throughout the studies, particularly for evaluating dental health after treatment and success of treatment. Clinical indicators can provide information about the effectiveness of treatment, the findings may have limited relevance for patients if a high level of dental knowledge is required for interpretation. Outcomes relating to quality of life and appearance are likely to be relevant to patients, because both have been shown to be motivators for seeking treatment and to be a determinant of satisfaction with treatment. It is less clear whether patients consider mastication and temporomandibular dysfunction useful measures of function and harm from treatment. A number of potential outcomes that could provide useful patient information for decision making were absent, such as patient preferences for dental health, treatment success, and further outcomes associated with service delivery. Clinician-reported outcomes were surprisingly scarce, limited to judgments on appearance after treatment. The beliefs, preferences, and unconscious biases of treatment providers could influence patient decision making, and evidence about clinician perceptions would be useful.

Perhaps the most surprising result was the almost complete absence of outcomes from a service-delivery perspective. Health-service research outcomes, such as measures of access to care, acceptability of treatment, and cost-effectiveness are essential for driving change. Quality-improvement studies were included in the review, with the expectation that the reports would indicate some outcomes that are important to service providers. Although the audits did provide some additional outcomes, these were limited to clinical indicators for measuring service delivery. Inadequate evidence for evaluating service delivery is a barrier to improving services and wider health. The prominence of evidence-based dentistry and the development clinical guidelines have resulted in emphasis on randomized controlled trials to measure the efficacy of treatments. This may in part explain the reduced focus on service evaluation and the scarcity of health-service research values in dental research. Present-day outcomes are unlikely to be able to answer important service delivery questions, such as whether a discrepancy exists between different providers of dental treatment, the effect of the dental care system on the health of hypodontia patients, and how social factors and organization structures may impact care provision. To enable recommendations to be made for improving hypodontia care, a broader approach to selecting research outcomes is required to incorporate information about the organization, management, finances, and delivery of care.

Trends were noted in the study characteristics and outcomes that reflect changes in research philosophy and the paradigm shift toward patient-centered care. Research activity has increased over time with a move toward assessing patient-reported outcomes. Oral health-related quality of life was first used in 2009 but has since been used in 4 further studies. The use of qualitative methods to gain a deeper understanding of patient experience was only recently reported but is likely to continue to gain popularity. Clinical indicators have remained popular over time, but there are signs of progress toward using better-defined, standardized measurement methods. Use of a standardized and validated tool is a variable across all outcomes and although some outcomes, such as oral health-related quality of life, tend to consistently use a widely accepted measurement tool (Oral Health Impact Profile), the tools for other outcomes, such as appearance, remain inconsistent.

In all studies, the construct of the research question and subsequent study design will influence the suitability and selection of outcome measures. Although randomized controlled trials are preferable for determining the effectiveness of a treatment, other research questions may demand different study designs : eg, surveys using qualitative methods to understand patient experience. In some instances, retrospective studies may be the only feasible design—eg, for determining long-term survival of deciduous teeth—but outcomes in these studies will be limited by the availability of the information. Regardless of study design, coherence is needed between the purpose of the research, the underlying theoretical basis, and the research design. The suitability of the outcome is intrinsically linked to the setting, participants, intervention, comparator, and methods. The location of studies, composition of the research team, and health care system in which the results will be applied are likely to influence the choice of outcomes. Universities may be more driven by academic interests than practice-based research teams, which will often be motivated by a need for clinical information or incentives for change.

This review is strengthened by the rigorous methodology and originality in subject and approach. The review was conducted following best-practice guidelines, and the findings will inform future research regarding development of patient-centered evidence for shared decision making.

Only gold members can continue reading. Log In or Register to continue

Dec 12, 2018 | Posted by in Orthodontics | Comments Off on Identification and appraisal of outcome measures used to evaluate hypodontia care: A systematic review
Premium Wordpress Themes by UFO Themes