Chapter 9. How to Appraise and Use an Article about Economic Analysis
Lusine Abrahamyan, M.D., M.P.H., Ph.D.; Petros Pechlivanoglou, Ph.D.; Murray Krahn, M.D., M.Sc.; Alonso Carrasco-Labra, D.D.S., M.Sc., Ph.D.; Romina Brignardello-Petersen, D.D.S., M.Sc., Ph.D.; Michael Glick, D.M.D.; Gordon H. Guyatt, M.D., M.Sc.; and Amir Azarpazhooh, D.D.S., M.Sc., Ph.D.
In This Chapter:
Why Economic Analysis in Dentistry?
Trial-Based versus Decision Model-Based Economic Analyses
Critically Appraising an Economic Analysis to Inform Clinical Decisions
• How Serious Is the Risk of Bias?
Introduction
In the previous eight chapters in this book, we introduced the process of evidence-based dentistry1 and explained how to search for evidence to inform clinical practice2 and how to use a research report to inform clinical decisions regarding questions of therapy,3 harm,4 diagnosis,5 systematic reviews,6 clinical practice guidelines,7 and qualitative research.8 In this chapter, we explain how to use an economic analysis to inform clinical and policy decision-making in dentistry. We introduce and describe the basic concepts needed to understand economic analysis, and we explain how to critically appraise such studies.
Box 9.1. Clinical Scenario
One of your patients, a first-year college student, came to ask for your opinion regarding his third molars, which have not erupted yet. He explained that a friend of his just had two of his mandibular third molars extracted and is planning to extract the remaining two because he was told that the early, prophylactic removal of third molars is less traumatic than the “inevitable late extraction of infected third molars,” and that it prevents future teeth crowding. Your patient does not have dental benefits, and he is concerned about his out-of-pocket expenses for extracting these teeth and whether such expenses would be worth the potential benefits and risks of the procedure. You realize that, to answer your patient’s question, you need to find an economic analysis whose authors considered both short-term and long-term risks, benefits, and costs for third-molar extraction. You decide to conduct a literature search and a critical appraisal to inform the decision.
Why Economic Analysis in Dentistry?
The economic burden of oral health care is significant, with a reported $111 billion spent on dental care in the United States and $11.7 billion in Canada in 2012.9,10 Public health agencies invest significant resources in oral health care programs that amounted to $9 billion in 2012 in the United States.9 Although most of the programs offered are assessed with respect to their effectiveness, whether they represent a good “value for the money” rarely is investigated.
Clinicians daily make treatment decisions not only on the basis of information about the benefits or harms but also on the basis of costs. With a patient’s best interest in mind, a clinician needs to assess whether the expected treatment benefits justify the resources used. For example, imagine that you want to buy more advanced three-dimensional (3-D) dental imaging equipment for your practice; does this possible purchase represent a good value for money spent? Or imagine yourself as a policy maker who must decide if the $2 million set aside for a public dental program should be directed toward an oral health prevention program for children or toward a program for adults who have low incomes and who are edentulous. Patients also need to invest their resources (for example, personal income, time off work) in interventions that will provide them with the best value for the money. Over time, such decisions are likely to get more, rather than less, difficult: the projected demographic changes in countries with high and low levels of income, our ever-increasing demand for better care, and increasingly costly health care innovations will continue to strain our already scarce health care resources. All these aspects illustrate the importance of investigating an intervention’s effectiveness and safety in conjunction with its efficiency, the balance of costs, and (positive and negative) health consequences.
There are different types of economic analysis that can evaluate the efficiency of a dental intervention. If the dentist is only interested in the overall cost of treating a particular condition, he or she can use a cost analysis, taking into account all resource utilization during and after treatment. This is, however, not a full economic analysis as it does not compare alternative treatments. If the dentist is interested in both the benefits and the costs of two or more treatments, a full economic analysis in the form of cost-effectiveness, cost-utility, or cost-benefit analyses would be a more appropriate source of evidence (Table 9.111–14). In all these types of economic analyses, treatment costs are measured in monetary units.
Cost-Effectiveness Analysis
In a cost-effectiveness analysis (CEA), treatment consequences (that is, benefits and harms) are measured in natural units, such as number of teeth extracted, gingival bleeding rates, or tooth survival. The main outcome of a CEA is the incremental cost-effectiveness ratio (ICER) (that is, the additional cost per additional unit of effect of a candidate intervention compared with an alternative). The results of a CEA can assist clinicians only in making decisions between treatments that share the same clinical effect.
Cost-Utility Analysis
In a cost-utility analysis (CUA), treatment consequences are measured in quality-adjusted life-years (QALYs), which is a combined measure of the duration and quality of life.15 The advantage of this type of analysis is its transferability, as it offers the means to make comparisons across different interventions and different diseases using a common measure (for example, cost per QALY for oral health prevention versus cost per QALY for hypertension prevention). Because of this advantage, CUA is the most common form of economic analysis.
Table 9.1. Types of Economic Analyses
* Source: Mohd-Dom and colleagues.11
‡ Source: Jacobson and colleagues.13
† Source: Zitzmann and colleagues.12
§ Source: Oscarson and colleagues.14
CUA also has limitations: the QALY can be insensitive to improvements in health-related quality of life achieved with dental interventions owing to the fact that few dental interventions are lifesaving or extend life. Furthermore, given that in most settings dental care is paid out of pocket or through private insurance, the need for prioritizing the allocation of resources across dental strategies (for example, investing in a caries prevention program for children or in an oral cancer awareness campaign) is limited. For these reasons, CUAs are rarely used in dentistry.
Cost-Benefit Analysis
In a cost-benefit analysis (CBA), the treatment consequences are evaluated in monetary terms, providing a direct estimate of whether consequences exceed costs.15 CBA is the least used form of economic analyses, with only few examples in dental literature.
Trial-Based versus Decision Model-Based Economic Analyses
Economic analyses can be conducted alongside clinical studies (trial-based) in which investigators collect patient-level data on health care resource use and costs, along with effectiveness outcomes.16 These clinical studies include randomized controlled trials (RCTs), observational studies, patient registries, and administrative databases.17 Constraints of a trial-based economic analysis include the facts that the duration for which costs and outcomes are assessed is limited to the actual study duration, information originating from other similar studies on the treatments of interest is ignored, and collecting data for economic analysis alongside a trial is often resource-intensive.15,17
Alternatively, decision models can be used to estimate the long-term (or lifetime) costs and consequences of health care interventions (see Figure 9.115,18 for a simplified example of a decision tree). A decision model is a statistical tool that allows clinicians to compare the costs and benefits of two or more alternative clinical decisions while considering the probability of events occurring over a selected period (that is, the time horizon).
Figure 9.1. Example of a Simplified Decision Model Comparing Surgical Removal or Retention of Third Molars
Decision models combine information from multiple sources (for example, randomized controlled trials, literature searches, administrative databases, and expert opinions) to reconstruct the clinical pathways for each alternative intervention under conditions of uncertainty over the time horizon.15 These models are more suited to evaluate long-term costs and consequences of treatments. This example decision tree considers only one adverse consequence (that is, anterior crowding). A decision tree in which adverse consequences are considered more extensively can be found in the study by Edwards and colleagues.18
Box 9.2. The Economic Analysis You Found
During your search, you found that the prophylactic extraction of disease-free, impacted third molars remains controversial. The American Association of Oral and Maxillofacial Surgeons, for example, supports the removal of “erupted and impacted third molar teeth even if the teeth are asymptomatic, if there is presence or reasonable potential that pathology may occur caused by or related to the third-molar teeth.”19 In contrast, the investigators of several systematic reviews did not find sufficient evidence to support removal over retention.20–22 You read that annually in the United States, approximately 10 million third molars are extracted from approximately five million people, with total costs exceeding $3 billion.23 You found an economic evaluation that compared removal versus retention of asymptomatic, disease-free mandibular third molars, using a decision model.18 In the abstract of the study, the investigators reported that the probability estimates for different clinical outcomes were obtained from a comprehensive literature review, and the treatment costs were obtained from the National Health Service hospitals in Wales, United Kingdom. The effect of each clinical outcome was assessed among 100 patients attending a single dental hospital. The authors concluded that mandibular third-molar retention was more cost-effective than removal.18 You obtain the article and conduct a critical review of the methods and results.
Critically Appraising an Economic Analysis to Inform Clinical Decisions
Economic analyses can be critically appraised using three steps: assessing the risk of bias, assessing the results, and assessing their applicability to your patients’ care.24 Below, we describe each of these steps.
How Serious Is the Risk of Bias?
The main research question of an economic analysis should define the patient population, the treatment alternatives, the perspective of evaluation, the type of analysis, and the time horizon for which costs and consequences are to be evaluated. Ideally, economic analyses should compare the new intervention with all standard treatment alternatives.15
For logistical reasons, however, this is not always feasible. Whatever treatments authors have chosen to compare, we suggest assessing three risk-of-bias criteria: consideration of subgroups, accurate measurement of consequences and costs, and consideration of timing. Table 9.212,25–28 presents examples of assessments of the risk of bias in economic analyses. In a critical appraisal process, it is important to evaluate if the new intervention has been compared with a relevant alternative, and if the time horizon of the study was sufficiently long to see the expected costs and consequences of treatments. Components of assessments of the risk of bias in economic analyses that could create a risk of bias are discussed in more details below.
Table 9.2. Critically Appraising the Risk of Bias in Studies of Economic Analysis
|
Questions |
Examples |
Explanations |
|
Are results reported separately for relevant patient subgroups? |
“The objectives of this study are to examine the utilization of dental sealants and its determinants, evaluate the incremental effectiveness and expenditure associated with sealant placement after correcting the potential selection issue, and explore the differences in sealant’s cost-effectiveness among subpopulations . . . . Children at relatively high caries risk, as well as children who visited dentists for preventive care more than once a year, had greater odds of receiving sealants.”* |
In this study, the authors specified subgroup analysis at the study planning phase and included it as part of the study aim. The authors further supported the subgroup analysis by conducting a literature review and by comparing the characteristics of children who visited dentists for preventive care and who either received or did not receive sealants. The cost-effectiveness was evaluated for the full sample and for the selected subgroups. The risk of bias is low on the basis of this criterion. |
|
Were consequences and costs measured accurately?† |
“For each patient, the costs of delivering treatment were recorded by a research nurse. . . . Laboratory costs were recorded as part of normal hospital policy. . . . All of the dental materials used were recorded and given a unit price . . . [and] the amount of time spent in the dental surgery for each appointment was measured using a stop watch. . . . The total number of clinical appointments was recorded, including unscheduled postoperative care, and the total clinical time calculated for each patient. The cost of professional time per patient was estimated using the highest point of the salary scale for the community dental service in Ireland (€85 185). Based on this salary, the hourly rate for a clinician providing care was €44.37 per hour for 240 8-hour working days per year.”‡ |
In this cost-effectiveness analysis, the authors compared the partial removable dental prosthesis and the shortened dental arch for older patients who were partially dentate in a randomized controlled trial that had 12 months of follow-up. The analysis was conducted from the “perspective of a publicly funded body.”‡ The authors described the cost components that were accounted for (that is, laboratory costs, dental materials, clinic visits, and time and cost of professional care) and only some of the sources for unit costs. For example, it is unclear how dental material costs were obtained. Moreover, the reporting of results was not transparent, as authors presented only the total costs per patient without information on frequency of use and unit costs.‡ These limitations entail high risk of bias for this criterion. |
|
Did investigators consider the timing of costs and consequences? |
“Costs were calculated in Euros and future costs discounted at 3% per annum. . . . No such discounting was performed for future effectiveness, since it remains unclear whether and how to discount years of tooth retention.”§ |
In this study, the authors used a decision model approach to evaluate the cost-effectiveness of one- and two-step incomplete and complete excavations for caries. They assessed the benefits (that is, tooth retention and vitality) and costs over the patient’s lifetime. The authors applied a 3% discounting rate to account for differential timing of costs. Effectiveness measures were discounted neither in their main analysis nor in sensitivity analyses. This limitation may indicate a high risk of bias for this criterion. |
† Authors’ note: The accuracy of measuring consequences has been covered in previous chapters in this book; here we discuss only costs.
‡ Source: McKenna and colleagues.26
§ Source: Schwendicke and colleagues.27
Are results reported separately for relevant patient subgroups?
Similar to clinical effectiveness studies, results of economic analyses can vary widely between different patient subgroups. Such variations can be explained by differences in treatment consequences or costs in these subgroups.29 For example, implant-supported dentures may be more cost-effective than conventional dentures in patients who are edentulous and younger than 60 years but not cost-effective for patients who are 85 years and older, and ignoring this difference can result in misleading interpretation of the results. The subgroups for economic analysis should be defined at the study planning stage and should be reported with the rationale for their selection (for example, to explore heterogeneity in results, to determine policy relevance, or on the basis of a literature review). Once defined, all results should be analyzed for selected subgroups separately.
Were consequences and costs measured accurately?
In an economic analysis, the evidence on consequences (that is, clinical effectiveness, safety) may come from a single RCT or an observational study, or, more appropriately, from evidence synthesis (that is, systematic review). The quality of outcomes of an economic analysis depends on the quality of the effectiveness evidence on which it relies. For that reason, systematic collection of the best, unbiased evidence on consequences is important. In previous chapters of this book, we have covered all major issues related to the risk of bias to establish treatment effectiveness,3 harm,4 and diagnostic accuracy.5 Here, we discuss issues pertaining to costs.
The cost components (that is, resources utilized) included in an economic analysis should reflect the perspective assumed. Hence, once you identify the perspective of the economic analysis in the reviewed article, you need to critically appraise whether all relevant cost components have been considered. For example, investigators of a study evaluating from a societal perspective the cost of establishing a community-based oral health promotion program by health educators who do not have an oral health background should consider not only the costs of training the educators (for example, hourly salary, space rental fees, costs of education materials)30 but also the productivity losses of the participants who attend the sessions. After identifying the cost components and the frequency of their use, unit costs are applied to obtain an estimate of the total costs associated with each patient.
Did investigators consider the timing of costs and consequences?
The consequences and costs of health care interventions can occur at different times. For example, although most of the costs for establishing an oral health education program in schools occur at the time of the program launch, the benefit of caries prevention may occur several years later. Investigators of a CEA comparing two alternative approaches for such a program should consider this differential timing of costs and benefits.
As a society and as individual people, we prefer to have resources available to us now, and not later, either because we can invest these resources and receive benefits over time or simply because we prefer good things now to good things later. Time preferences, therefore, play a significant role both in making individual decisions and in influencing public policy.31
To adjust for these differential time preferences, especially when the study’s time horizon is long, we devalue benefits and costs that accrue later, relative to those that occur earlier. This process of devaluing is called “discounting,” and economic analysts apply a discounting rate to costs and outcomes. Most economic evaluation guidelines recommend using either a 3% or a 5% per year discounting of future costs and outcomes to present values.15 It is, however, debatable if the costs and consequences should be discounted in the same way.31
Box 9.3. Your Assessment of the Risk of Bias of the Economic Analysis
You Identified The authors of the study you identified18 did not specify any subgroups, although they could have considered age and smoking status on the basis of the literature. Effectiveness was estimated by asking patients to rate different scenarios after tooth removal or retention, using a visual analog scale, which is the least preferred method to evaluate health preferences. Furthermore, the variability around the average effectiveness scores was not presented. Only aggregate costs by scenario and by health care resource use were presented, which limited your ability to see, for example, medication costs (see the Supplemental Table18 at the end of this chapter). The overall time horizon for costs and benefits was not specified, and discounting was not considered. Bearing in mind the identified limitations, you proceed to read the results.
Stay updated, free dental videos. Join our Telegram channel
VIDEdental - Online dental courses
