THE DILEMMA

The assessment of any health program can usually be conducted because assessors know what the program is like and how it should work. We use the term “assessment” rather than “evaluation” because the latter implies measurement, whereas assessment can accommodate both health status measurements and other qualitative outcomes that cannot be measured but nonetheless are essential components and outcomes of healthcare. In addition, the term “evaluation” is sometimes misunderstood to be restricted to summative measurements. Similarly, implementers of a program would be more confident about how to provide care when there is evidence of effectiveness gathered from an outcome assessment. However, in the case of oral healthcare for frail elders, care providers and assessors are neither sure how to provide care nor how to assess it. Without gold standard for care, the following questions arise:

• What should be the objectives of care?
• How does the healthcare program function?
• Is it working and meeting expected needs?
• How do we know if what we do is making a difference?

In this chapter, we will explain the concepts and challenges in developing outcome assessments for mouthcare programs in long-term care (LTC) facilities, and offer a comprehensive framework for assessing outcomes of mouthcare in the facilities.

ASSESSING ORAL HEALTHCARE PROGRAMS

There are methods for assessing the oral status of elderly people (Kayser-Jones et al., 1995; MacEntee and Wyatt, 1999; Bauer, 2001) and methods for assessing the psychosocial impact of oral impairment, disease, and disability (Atchison and Dolan 1990; Slade, 1997; Slade et al., 1998). However, the methods have been used mostly to identify the needs for care, but there has been remarkably little effort to assess the benefits of oral healthcare services in LTC, and there are no guidelines or best practices for assessing the effectiveness of the services (Dolan, 1993; MacEntee et al., 1999).

Caring for frail elders involves complicated biophysical, psychosocial, and organizational elements. Consequently, assessment of outcomes is far from straightforward. Mouthcare products and techniques can be efficacious––they can operate effectively––in the controlled context of clinical trials, but whether or not their benefits are sustainable within the much less controlled operations of LTC is doubtful (Chalmers et al., 2004). Similarly, the dental knowledge and skills acquired by nursing staff during in-service educational programs also appear to be unsustainable within the conflicting demands of most nursing facilities (MacEntee et al., 1999, 2007; Watt et al., 2006; Pruksapong and MacEntee, 2007).

The lack of demonstrable benefits from oral healthcare programs for frail elders is worrisome, particularly in an era where the very sustainability of healthcare in any context is under close scrutiny, and depends on accountability demonstrating practical effectiveness and value for money (Romanow, 2002).

KEY CONCEPTS

Basic concepts of outcome assessment

There are two main concepts of how healthcare programs should be assessed: one is “quality of care” and the other “program evaluation.” They both emerged separately from demands for improved quality and accountability. Initially, quality of care focused mostly on the performance of people, while evaluation attempted to measure the results or outcome of the program. However, they have many similarities, and, more recently, both have been expanded to consider a wide range of the components that are integral to the operations and success of a program.

According to the Institute of Medicine (2001) in the United States, quality of care typically identifies “the degree to which health services increase the likelihood of outcomes that are consistent with current professional knowledge.” Donabedian (2003) created a very useful framework with three levels for assessing the quality of care. On Level 1, he proposed an assessment of the structure of the human, financial, and other material resources within which care is delivered; on Level 2, he judged the process of care, including the technical and interpersonal activities; and on Level 3, he considered the outcomes from the perspectives of both the recipients and the providers of care. He recommended also that the results of the assessment at each level should lead to corrective actions when the quality is deemed to be inadequate.

Considerations of quality assurance go beyond the limits of a unique performance and outcome judgement to include a range of organizational variables. It provides a continuous rather than a unique assessment of care and improvement of quality. Some administrators of programs refer to this as a “continuous quality improvement” or “total quality management” (McLaughlin and Kaluzny, 2006).

Evaluation of a healthcare program, on the other hand, aims to judge merit, enhances effectiveness, and provides information for further improvements (Weiss, 1998; Patton, 2002). Formative evaluation is linked usually to the process of development, whereas summative evaluation is associated more with outcome measures or final tallies when the program functions successfully. It too, like quality of care, has expanded to include a range of information about the characteristics of a program, in addition to developments and outcomes.

Essentially, both quality assessment and program evaluation are systematic inspections of the structure, activity, and outcome of a service or organization. When information about the program comes from a comprehensive investigation based on a combination of quality assurance and program evaluation, there is the hope that both accountability and improvement of quality are addressed.

Oral healthcare

The success of oral healthcare as a part of LTC depends at least as much on the organizational culture, philosophy, and connectivity of everyone involved as it does on the dental treatment available for the residents (MacEntee et al., 1999). Indeed, oral healthcare as an integral part of the general care provided to residents is an important quality indicator of the overall care offered by a nursing facility. Unfortunately, the nursing and dental personnel all too frequently operate independently of one another with an “us and them” perspective. When asked what makes oral healthcare challenging, typically the nurses complain about the difficulty of getting dentists or dental hygienists to attend residents in the facility, about the high cost of dental treatment, and about the disinterest and different healthcare values among the residents and their families. On the other hand, families are apt to complain about nurses who do not provide the help and care needed. Overall, the conflicting values and expectations of all the participants make it exceedingly difficult to introduce new programs of any kind, and especially programs for which nurses and care-aides have very little training.

A MODEL FOR ASSESSING AN ORAL HEALTHCARE FOR FRAIL ELDERS

Assessing multiple components

Several key components of an oral healthcare program in LTC influence how it operates. The existing healthcare system and social structure are major influences, but they are shaped in various important ways by the expectations of the residents and by the competence and resources of the staff (Figure 16.1).

Each component can be assessed by various indicators, such as the philosophy and management of the administration, the financial support for different personnel, the comprehensiveness of operational and health records, and, of course, the ease of access to dental personnel (Figure 16.2).