Abstract
Objectives
The primary aim of this study was to evaluate the quality of published dental clinical guidelines using the AGREE II instrument.
Methods
Online searching of a wide range of organisations (national and International) was undertaken to identify dental clinical practice guidelines published between 2000 and 2014. The quality of each included guideline was assessed in relation to the AGREE II instrument by four assessors independently. Inter-rater agreement was assessed. Descriptive statistics and both univariate and multivariate analyses were conducted.
Results
162 guidelines were identified. The overall mean quality score was 51.9% (SD 13.3). There was variation in the reporting quality of individual domains with both Applicability (20.4%) and Editorial Independence (34.25%) poorly reported. Variation between the overall quality scores for guidelines produced by different dental specialities was evident. The quality of guidelines improved per publication year (β = 0.76, 95% CI: 0.26, 1.26, p = 0.003). Guidelines based on formal evidence (β = 19.94, 95% CI: 15.25, 24.64, p = 0.001) achieved higher quality scores.
Conclusion
Overall, the quality of clinical dental practice guidelines is suboptimal. There is variation in the overall quality, reporting of individual items and domains of the AGREE II instrument between different dental speciality clinical practice guidelines. Guidelines based on formal evidence achieved higher quality scores.
Clinical significance
Clinicians should be aware of the variation in the quality of dental clinical guidelines in particular related to methodological rigour. The use of formal evidence may be a useful indicator of their quality prior to their implementation.
1
Introduction
Evidence based medicine has been defined as the integration of the best research evidence with clinical expertise and patient values . Clinical practice guidelines can be a means to bridge the gap between research and healthcare provision . The Institute of Medicine defines guidelines as systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances . Whilst clinical practice guidelines have numerous benefits they may also negatively influence patient care or be of questionable applicability in dental practice . It is of paramount importance that guidelines are of sufficient quality to allow the implementation of clear and effective recommendations. Whilst numerous instruments have been developed to assess the quality of guidelines , the AGREE II (Appraisal of Guidelines, Research and Evaluation II) is an internationally developed, validated, easy-to-use and transparent instrument . Health care organisations which have included and assess their guidelines with the AGREE instrument include National Institute for Clinical Excellence (NICE), National Federation of Cancer Centres (FNCLCC), The Agency for Quality in Medicine in Germany (ÄZQ), Scottish Intercollegiate Guidelines Network (SIGN) and World Health Organisation .
Previous quality assessments of clinical practice guidelines in dentistry have been undertaken . The majority of these studies have been limited to specific subspecialties such as cone beam computer tomography , orthodontics , paediatric dentistry , dental management of antithrombotic drug use and common clinical procedures . The results of these studies have identified that the reporting and quality of dental guidelines is lacking and inadequate in relation to the AGREE instrument.
To date, no assessment of the quality of clinical practice guidelines in dentistry using the AGREE II has been undertaken with four reviewers as recommended by the AGREE collaboration . In addition, characteristics that may influence quality have not been identified. The primary aim of this study was to evaluate the quality of published speciality dental clinical practice guidelines in relation to the AGREE II instrument. A secondary aim was to identify factors associated with improved guideline reporting.
2
Materials and methods
2.1
Information sources and search strategy
An electronic literature search was undertaken to identify guidelines related to dentistry published between 2000 and 2014. The search was restricted to guidelines published in English and only interventions at the individual/patient level were included. Conference abstracts, non-English guidelines, laboratory based guidelines and those aimed at non-dental healthcare workers were excluded. A MEDLINE (Ovid ® ) database search was carried out on the 13th February 2015 using the terms described in Table 1 . In addition, the TRIP (Turning Research Into Practice) database, National Institute For Health and Clinical Excellence (NICE) Evidence and US National Guideline Clearinghouse were searched using the search term (dent*) limiting results to 2000 to 2014. The TRIP database search was limited to “guidelines only”. The websites of national and international dental organisations were also searched to identify dental guidelines satisfying the inclusion criteria (Appendix I in Supplementary material). A single author (SM) initially screened all potentially relevant dental guidelines. All identified guidelines were then independently screened by two authors (SM and JS). Any disagreements were resolved by discussion.
| Search term | Number of papers |
|---|---|
| 1. guideline.mp | 72298 |
| 2. exp guideline/ | 27387 |
| 3. Clinical recommendation.mp | 126 |
| 4. Position statement.mp | 2090 |
| 5. Position paper.mp | 2003 |
| 6. 1 or 2 or 3 or 4 or 5 | 75380 |
| 7. Dent*.mp | 468408 |
| 8. 6 and 7 | 1577 |
| 9. Limit 8 to yr = 2000–2014 | 1097 |
2.2
AGREE II instrument
The quality of the process and reporting of clinical guideline development of each guideline was assessed using the AGREE II instrument which consists of a twenty-three item checklist categorised into six domains (Scope and Purpose, Stakeholder involvement, Rigour of Development, Clarity of Presentation, Applicability and Editorial Independence). Each domain aims to measure a different aspect of guideline quality and identify potential biases (Appendix II in Supplementary material). Each of the AGREE II items are rated on a seven-point Likert scale ranging from ‘Strongly Agree’ to ‘Strongly Disagree’. A score is assigned based upon the reporting of the item in relation to the full criteria or considerations, its level of completeness and quality of reporting.
2.3
Evaluation of guidelines
Four assessors evaluated the guidelines independently. Each assessor was calibrated in the use of the AGREE II instrument by completing the online training tool and by completing a pilot of 5 guidelines. Any discrepancies or clarifications were discussed until a consensus was obtained. In addition, each guideline was assessed by referring directly to the associated explanation of each item as stated in the user manual. Guideline demographic data collected included: the development process classification (expert opinion, consensus based or formal evidence based) , the dental sub-specialty of each guideline, number of authors, continent of publication, identification as single-or multi-centre guideline development and whether the guideline was an update.
2.4
Statistical analysis
Inter-assessor reliability was assessed using Intra-class correlation coefficient (ICC). Descriptive statistics for individual reporting items for each dental guideline were calculated and converted to a percentage scale with 100% indicating the maximum score for all applicable items. Linear regression modeling was implemented with univariate analysis to identify characteristics associated with mean score; multivariate modeling was used to determine the adjusted effect on reporting quality score. Significant predictors identified during the univariate analysis were entered individually in the multivariate model. The final model was derived by comparing candidate models using the likelihood ratio test. A two-tailed p-value of 0.05 was considered statistically significant. Statistical analyses were performed with STATA ® version 14.2 software (Stata Corporation, College Station, Texas, USA).
2
Materials and methods
2.1
Information sources and search strategy
An electronic literature search was undertaken to identify guidelines related to dentistry published between 2000 and 2014. The search was restricted to guidelines published in English and only interventions at the individual/patient level were included. Conference abstracts, non-English guidelines, laboratory based guidelines and those aimed at non-dental healthcare workers were excluded. A MEDLINE (Ovid ® ) database search was carried out on the 13th February 2015 using the terms described in Table 1 . In addition, the TRIP (Turning Research Into Practice) database, National Institute For Health and Clinical Excellence (NICE) Evidence and US National Guideline Clearinghouse were searched using the search term (dent*) limiting results to 2000 to 2014. The TRIP database search was limited to “guidelines only”. The websites of national and international dental organisations were also searched to identify dental guidelines satisfying the inclusion criteria (Appendix I in Supplementary material). A single author (SM) initially screened all potentially relevant dental guidelines. All identified guidelines were then independently screened by two authors (SM and JS). Any disagreements were resolved by discussion.
| Search term | Number of papers |
|---|---|
| 1. guideline.mp | 72298 |
| 2. exp guideline/ | 27387 |
| 3. Clinical recommendation.mp | 126 |
| 4. Position statement.mp | 2090 |
| 5. Position paper.mp | 2003 |
| 6. 1 or 2 or 3 or 4 or 5 | 75380 |
| 7. Dent*.mp | 468408 |
| 8. 6 and 7 | 1577 |
| 9. Limit 8 to yr = 2000–2014 | 1097 |
2.2
AGREE II instrument
The quality of the process and reporting of clinical guideline development of each guideline was assessed using the AGREE II instrument which consists of a twenty-three item checklist categorised into six domains (Scope and Purpose, Stakeholder involvement, Rigour of Development, Clarity of Presentation, Applicability and Editorial Independence). Each domain aims to measure a different aspect of guideline quality and identify potential biases (Appendix II in Supplementary material). Each of the AGREE II items are rated on a seven-point Likert scale ranging from ‘Strongly Agree’ to ‘Strongly Disagree’. A score is assigned based upon the reporting of the item in relation to the full criteria or considerations, its level of completeness and quality of reporting.
2.3
Evaluation of guidelines
Four assessors evaluated the guidelines independently. Each assessor was calibrated in the use of the AGREE II instrument by completing the online training tool and by completing a pilot of 5 guidelines. Any discrepancies or clarifications were discussed until a consensus was obtained. In addition, each guideline was assessed by referring directly to the associated explanation of each item as stated in the user manual. Guideline demographic data collected included: the development process classification (expert opinion, consensus based or formal evidence based) , the dental sub-specialty of each guideline, number of authors, continent of publication, identification as single-or multi-centre guideline development and whether the guideline was an update.
2.4
Statistical analysis
Inter-assessor reliability was assessed using Intra-class correlation coefficient (ICC). Descriptive statistics for individual reporting items for each dental guideline were calculated and converted to a percentage scale with 100% indicating the maximum score for all applicable items. Linear regression modeling was implemented with univariate analysis to identify characteristics associated with mean score; multivariate modeling was used to determine the adjusted effect on reporting quality score. Significant predictors identified during the univariate analysis were entered individually in the multivariate model. The final model was derived by comparing candidate models using the likelihood ratio test. A two-tailed p-value of 0.05 was considered statistically significant. Statistical analyses were performed with STATA ® version 14.2 software (Stata Corporation, College Station, Texas, USA).
3
Results
3.1
Inter-assessor reliability
The inter-assessor level of agreement (ICC) between the four assessors was high (0.87; 95% Cl: 0.78, 0.92).
3.2
Search results
A total of 162 dental guidelines were identified Fig. 1 .
