Introduction
Patients may express views about their orthodontic care by posting publicly available reviews on the Internet. This study analyzed online reviews of orthodontists with an emphasis on the types and frequency of complaints expressed in negative reviews.
Methods
A random sample of 10.6% of the American Association of Orthodontists membership was evaluated to identify members practicing in a North American office that is limited to orthodontics and has an online presence. Information regarding those orthodontists and their offices was collected. For offices with Google and/or Yelp reviews, all negative (1- or 2-star) reviews were saved for content analysis. If an office posted a response to a negative review, those responses were also collected for analysis.
Results
Of the 807 eligible orthodontists, 92.4% had reviews on Google and/or Yelp. Average ratings of orthodontists were very positive (ie, 4- and 5-star reviews constituted >97% of Google reviews and >88% of Yelp reviews), yielding an average orthodontist rating of 4.72 on Google and 4.42 on Yelp. However, approximately half of those orthodontists (50.9%) also had at least 1 negative review. The 3 most frequently mentioned categories of complaints were regarding quality of care and/or service, interpersonal interactions, and money and/or financial issues. An analysis of the specific kinds of complaints is described. Orthodontists posting responses to negative reviews had significantly better average ratings than those who did not, but this association does not demonstrate a causal relationship.
Conclusions
Understanding the complaints orthodontic patients express in online reviews may suggest strategies to improve patient satisfaction and an orthodontist’s online reputation.
Highlights
- •
Most orthodontists have online reviews; half of them have received a negative review.
- •
Negative reviews commonly cite the treatment and exam-records-consult phases.
- •
The top 3 complaint categories are quality of care, interpersonal issues, and money.
- •
The most common specific complaint refers to impolite, rude, or mean interaction.
- •
Most orthodontists do not post an online response to a negative review.
Since the advent of the consumer rights movement, patient satisfaction in health care has been increasingly emphasized with reports that higher satisfaction is associated with better quality health care, increased treatment compliance, better-perceived treatment results, and fewer medical malpractice claims. Improving patient satisfaction has been recommended as a practice-building strategy. , Consequently, patient satisfaction has received attention from medical and dental specialties, including orthodontics. In orthodontics, Bennet et al concluded that the treatment process, psychosocial benefits, and overall treatment outcome were the 3 most valid and reliable factors related to patient satisfaction. A systematic review from 2015 found that the factors most commonly associated with satisfaction in orthodontics were perceived esthetic outcomes, psychological benefits, and quality of care, whereas dissatisfaction was related to treatment duration, pain and/or discomfort, and retention appliances.
Social media provides a public platform to express opinions about one’s health care. Online patient reviews and ratings give prospective patients easily accessible information to consider when selecting health care practitioners. , , Survey results indicate that a majority of the US population is aware of online physician ratings and consider physician rating Web sites to be either “somewhat important” or “very important” when choosing a physician. Another survey found that 41% of consumers reported that information found on social media could affect their choice of a doctor. The number of subjects using online reviews to evaluate physicians is growing rapidly, as is the number of physicians receiving online ratings and the number of ratings per provider. , , Research suggests that prospective patients conducting online searches are more likely to select a doctor with more favorable reviews and avoid doctors with negative reviews. , , Indeed, a 2016 survey found that 48% of respondents would be willing to go out of their insurer’s provider network to avoid a doctor with worse reviews.
Although patient satisfaction research has relied mostly on patient interviews, surveys, or focus groups, publicly posted online reviews, ratings, or commentaries provide a new method to evaluate patient satisfaction. Research in the field of medicine suggests that patient satisfaction based on views expressed on social media can reflect the quality of health care. At the level of the practitioner, quality of care was better correlated with online review content than with a physician’s clinical experience, board certification, education, or lack of malpractice claims. At an institutional level, ratings posted on Yelp produced findings comparable with traditional hospital performance measures, such as the systematically collected Hospital Consumer Assessment of Healthcare Providers and Systems ratings.
Despite the growing body of medical research on patient satisfaction using social media, , , , , there has been limited research evaluating the concerns expressed online about orthodontics. , Although patient satisfaction in orthodontics is generally high, , there are still dissatisfied patients, and social media allows these subjects to share their negative opinions publicly. This descriptive study analyzed online reviews posted about a random sample of orthodontists with an emphasis on the content of negative online reviews as well as the content of office responses to negative reviews. Negative reviews are usually unsolicited and should provide insight into the causes of patient dissatisfaction. In addition, little is known about how orthodontic offices respond online to negative reviews. Understanding the causes of online patient complaints is a prerequisite to devising strategies that will reduce complaints with the ultimate goal of increasing patient satisfaction and improving an orthodontic office’s online reputation.
Material and methods
This study evaluates the content, prevalence, and distribution of online reviews of orthodontists obtained from 2 commonly used public Web sites that provide reviews of orthodontists, that is, Google and Yelp. The University of Washington’s Human Subjects Research Determination Guide indicated that this study did not require approval by the Institutional Review Board because the information is publicly available and not considered private.
The American Association of Orthodontics (AAO) provided its membership list (in alphabetical order by last name) from which a random sample of orthodontists was selected by placing the individual members in a random order based on a set of random numbers generated using the Rand function in Excel. A large sample of orthodontists (∼10.6% of the membership list) was selected to improve the accuracy of the descriptive statistical estimates being calculated. For example, based on the 10.6% sample of the total AAO membership, the margin of error for each of the estimated percentages in that sample is at most ±1.5%, based on the formula for the standard error of a sample proportion with adjustment for the finite population correction. Estimated percentages that are based on a subset of this sample will have larger margins of error that can be determined as described in Bondy and Zlot.
Methodologic details describing data collection, analysis, and scoring of online reviews, and the reliability of review scoring are provided online in Supplementary Appendix (available at www.ajodo.org ).
Statistical analysis
Descriptive statistics, including counts, percentages, means, standard deviations, and medians, were calculated for various variables. Several t tests were performed to make comparisons of interest. These t tests allowed for unequal variance produced a 95% confidence interval (CI) that accompanied each test and did not account for multiple comparisons.
Results
The study sample consisted of 1024 orthodontists randomly selected from the AAO’s list of 9627 members, as described in the flowchart ( Fig ). A subset of those selected orthodontists (n = 953; 93.1%) was identified on the AAO’s online membership directory during the approximate 13-month data collection period (from May 21, 2017 to June 24, 2018) after receipt of the membership list. Of those confirmed AAO members, approximately 84.7% (n = 807) were in private practice orthodontics settings with an average of 1.53 offices per orthodontist (median, 1; range, 1-8). Online reviews could be found for the great majority of these orthodontists (92.4%; 746 of 807), with approximately half of them (50.9%; 380 of 746) having received at least 1 negative Google and/or Yelp review. These 380 orthodontists yielded a total of 956 negative textual reviews for analysis.
Table I describes the proportion of eligible orthodontists and their orthodontic offices that have Google and/or Yelp reviews as well as the distribution of those reviews according to the overall number of stars assigned. An important observation is that most reviews are positive (ie, 5- and 4-star ratings constituted >97% of reviews on Google and >88% of reviews on Yelp). Negative (1- and 2-star) ratings constituted 2.1% of reviews on Google and 9.8% of reviews on Yelp. There were 134 negative 1- and 2-star ratings on Google that did not include a textual review.
| Categories | Reviews on Google | Reviews on Yelp | Reviews on Google or Yelp | No reviews on Google or Yelp |
|---|---|---|---|---|
| Orthodontists (n = 807) | ||||
| Percentage with 1-5 star reviews | 89.2 (720) | 62.1 (501) | 92.4 (746) | 7.6 (61) |
| Percentage with 1- and 2-star reviews | 36.7 (296) | 23.7 (191) | 47.1 (380) | |
| Orthodontic offices (n = 1234) | ||||
| Percentage with 1-5 star reviews | 80.9 (998) | 50.8 (627) | 85.3 (1052) | 18.2 (182) |
| Percentage with 1- and 2-star reviews | 27.6 (340) | 18.2 (224) | 37.0 (457) | |
| Distribution of all reviews | ||||
| Total number reviews, n | 29,133 | 5279 | ||
| 5-star reviews | 95.2 (27,733) | 84.8 (4476) | ||
| 4-star reviews | 2.4 (690) | 3.9 (205) | ||
| 3-star reviews | 0.4 (104) | 1.5 (79) | ||
| 2-star reviews | 0.4 (106) | 2.0 (108) | ||
| 1-star reviews | 1.7 (500) | 7.8 (411) | ||
| Mean stars | 4.89 | 4.56 |
The average online rating per orthodontist was 4.72 stars on Google, and 4.42 stars on Yelp ( Table II ). Each orthodontist’s rating is an average of that individual’s mean ratings from that practitioner’s office(s) (calculated separately for Google and Yelp). All of the orthodontists’ individual mean ratings were then averaged to yield the overall mean Google and Yelp ratings for orthodontists. When orthodontist ratings are compared based on whether they have negative online reviews or not, having negative reviews lowers the overall mean rating for orthodontist as would be anticipated. Specifically, orthodontists without any negative reviews have higher average ratings (mean orthodontist-averaged Google rating, 4.96; mean orthodontist-averaged Yelp rating, 4.92) compared with orthodontists with negative reviews (mean orthodontist-averaged Google rating, 4.39; mean orthodontist-averaged Yelp rating, 3.61) ( Table II ). The distribution of ratings among orthodontists with negative reviews differs from the distribution of ratings among those without any negative reviews. For Google reviews, orthodontists with negative reviews have an average of 60.6 reviews distributed as averages of 1.7 reviews for 1-star ratings, 0.4 for 2-star ratings, 0.2 for 3-star ratings, 1.6 for 4-star ratings, and 56.7 for 5-star ratings, whereas those with no negative reviews have an average of 26.4 reviews distributed as no 1- and 2-star ratings and averages of 0.1 reviews for 3-star ratings, 0.5 for 4-star ratings, and 25.8 for 5-star ratings. For Yelp reviews, orthodontists with negative reviews have an average of 19.2 reviews distributed as averages of 2.2 reviews for 1-star ratings, 0.6 for 2-star ratings, 0.3 for 3-star ratings, 0.8 for 4-star ratings, and 15.3 for 5-star ratings, whereas those with no negative reviews have an average of 5.2 reviews distributed as no 1- and 2-star ratings and averages of 0.1 reviews for 3-star ratings, 0.2 for 4-star ratings, and 5.0 for 5-star ratings.
| Categories | Years after orthodontics graduation ∗ | Google rating | Yelp rating |
|---|---|---|---|
| Orthodontists in AAO directory (n = 953) | 23.1 (12.8; 0.0-69.0) | NA | NA |
| Eligible orthodontists (n = 807) | |||
| Had online Google or Yelp reviews (n = 746) | 23.4 (12.1; 3.0-69.0) | 4.72 (720; 0.58) | 4.42 (501; 1.00) |
| No online Google or Yelp reviews (n = 61) | 27.6 (13.8; 4.0-60.0) | NA | NA |
| Orthodontists with online reviews (n = 746) | |||
| Had negative reviews on Google or Yelp (n = 380) | 24.0 (11.5; 3.0-67.0) | ||
| No negative reviews on either Google or Yelp (n = 366) | 22.7 (12.7; 3.0-69.0) | ||
| Had negative reviews on Google (n = 296) | 24.0 (11.5; 3.0-67.0) | 4.39 (296; 0.78) | |
| No negative reviews on Google (n = 424) | 22.8 (12.5; 3.0-69.0) | 4.96 (424; 0.13) | |
| Had negative reviews on Yelp (n = 191) | 24.2 (11.2; 4.0-61.0) | 3.61 (191; 1.20) | |
| No negative reviews on Yelp (n = 310) | 21.8 (12.0; 3.0-58.0) | 4.92 (310; 0.25) |
On average, orthodontists with no online reviews completed orthodontic training 4.2 years before those orthodontists with online reviews ( t test, 2.3; degrees of freedom (df), 67.7; P = 0.02; 95% CI difference range is 0.6 years after to 7.9 years before; Table II ). However, whether orthodontists with online reviews do or do not have any negative reviews was not significantly related to the number of years since completing orthodontics training. On average, orthodontists with no negative online reviews completed orthodontic training 1.3 years after orthodontists who had 1 or more negative reviews ( t test, −1.5; df, 730.2; P = 0.14; 95% CI difference range is 0.4 years before to 3.1 years after; Table II ). Table III also presents the average number of reviews and average orthodontist ratings (for Google and Yelp separately) according to the decade of graduation.
| Orthodontist with online reviews by graduation decade | Orthodontists (n) with Google reviews | Orthodontists (n) with Yelp reviews | Google rating, mean (mean no. of reviews) | Yelp rating, mean (mean no. of reviews) |
|---|---|---|---|---|
| 1950s | 3 | 1 | 3.31 (3.3) | 1.00 (4.0) |
| 1960s | 7 | 5 | 4.67 (24.0) | 4.80 (3.6) |
| 1970s | 68 | 46 | 4.46 (29.5) | 4.19 (4.9) |
| 1980s | 142 | 87 | 4.67 (24.5) | 4.16 (8.4) |
| 1990s | 194 | 146 | 4.67 (36.8) | 4.46 (10.0) |
| 2000s | 214 | 152 | 4.84 (55.5) | 4.47 (15.6) |
| 2010s | 92 | 64 | 4.89 (48.3) | 4.78 (7.3) |
Almost half of all negative online reviews (47.5%; 454 of 956) pertained to the period of “active orthodontic treatment.” The next most frequently cited periods of orthodontic care referred to the (1) “No specified time/in general” (17.7%; 169 of 956), (2) “Exam/Consult/Records” appointments (16.1%; 154 of 956), (3) “Retention Phase or Post-Treatment Retrospective Evaluation” (12.7%; 121 of 956), and (4) “Prior to the Initial Appointment” (6.5%; 62 of 956). Other possible phases of orthodontic care such as “Recall” exams, “Second Opinion” exams, and “Emergency Appointments” occurred in less than 1% of reviews.
Supplementary Table (see online, available at www.ajodo.org ) presents the detailed findings from the content analysis regarding the types of complaints expressed in at least 1% of the 956 negative online reviews. The 3 most common general categories or domains of complaints found in negative reviews were about (1) quality of care and/or service, which occurred in 53.2% of reviews; (2) interpersonal interactions, which occurred in 52.0% of reviews; and (3) money and/or financial issues, which occurred in 46.9% of reviews. These were followed by significant but less frequent types of complaints, shown in continued order of decreasing frequency of occurrence, that had to do with (4) time, which occurred in 26.8% of reviews, (5) communication, which occurred in 23.8% of reviews, and (6) unprofessional behavior, which occurred in 10.7% of reviews. The top 5 specific complaints across all categories were “Impolite/Rude/Mean” (23.5% of reviews), “Bad outcome/Complications/Not finish” (21.3% of reviews), “Errors/Poor skill, ability, office organization” (18.1% of reviews), “Money before patients/Greedy” (15.6% of reviews), and “No time for patient/Factory environment” (14.1% of reviews).
Certain types of complaints are usually attributed to specific personnel in the orthodontic office ( Supplementary Table ). For example, orthodontists were typically cited for problems about the excessive length of treatment, whereas the front office staff was cited for scheduling errors and/or changes. Complaints regarding money and/or financial issues typically referred to the orthodontist, although 82.1% of billing problems were attributed to the billing staff. Communication complaints were largely about the reception staff, interpersonal complaints were most frequently associated with the orthodontist and the reception staff, and quality of care and/or service was most frequently attributed to the orthodontist.
Frequently mentioned miscellaneous complaints were, (1) poor customer service (7.6% of reviews), (2) lost and/or broken and/or replace retainer (7.0% of reviews), and (3) refusal to treat (or continue to treat) (2.5% of reviews). Complaints of multiple providers and/or staff turnover occurred in 1.8% of reviews, whereas complaints of office layout and/or office design, office cleanliness and/or shabby, and office catered toward only children and/or adults each occurred in 1.2% of reviews. No identifiable complaint was found in 3.0% of all negative reviews.
Table IV describes the 219 responses posted in reply to negative online reviews. A minority of orthodontic offices (27.4%; 93 of 340) that received a negative Google review posted a response, and none of those negative reviews were updated after the office’s response. Similarly, a minority of orthodontic offices (31.3%; 70 of 224) that received a negative Yelp review posted a response. Out of those 70 offices, 17 (24.3%) had patients update their review after the office response. However, only 4 of those 17 offices had reviews that were updated to a more positive rating of 3-, 4-, or 5-stars, whereas the updated reviews remained negative (either a 1- or 2-star rating) for the other 13 offices. Orthodontists who responded to negative reviews had a greater number of total reviews (mean Google reviews, 72.0; mean Yelp reviews, 34.7) compared with those who did not post a response (mean Google reviews, 59.1; mean Yelp reviews, 16.1) (mean Google difference, 13.0; t test, 1.2; df, 174.7; P = 0.24; 95% CI difference, −9.0-34.9; mean Yelp difference, 18.5; t test, 3.0; df, 81.5; P <0.01, 95% CI difference, 6.2-30.9). In addition, orthodontists who responded to negative Google reviews also had a significantly higher average rating (4.54) when compared with orthodontists who did not respond to negative Google reviews (4.35) (difference, 0.19; t test, 2.1; df, 187.0; P = 0.03; 95% CI difference, 0.01-0.36; Table IV ). Similarly, orthodontists who responded to negative Yelp reviews had a significantly higher average rating (4.15) than orthodontists who did not post a response (3.41) (difference, 0.74; t test, 5.2; df, 183.6; P <0.01; 95% CI difference, 0.46-1.02; Table IV ).
