Artificial intelligence in oral and maxillofacial pathology is in relatively nascent stages but can be used to assist pathologists in detecting certain lesions (such as squamous cell carcinoma), classifying tissues, and prognosticating the behavior of tumors. Comparing performance of different artificial intelligence strategies for a given task is difficult due to the lack of robust publicly available datasets, which is important due to the large degree of preanalytical variance in the creation of digital pathology images.
Key points
-
•
Artificial intelligence (AI) detection of relatively straightforward diseases performs similarly to humans.
-
•
Prognostication of head and neck squamous cell carcinoma behavior may benefit from the use of robust AI models.
-
•
AI models may help in predicting outcomes for malignant transformation of oral premalignant disorders, though this is still uncertain.
-
•
Addressing preanalytical concerns that cause data variability is essential for bringing AI models in oral pathology from a research setting to real-life diagnostic settings.
Abbreviations
| AI | artificial intelligence |
| CNNs | convolutional neural networks |
| CPS | Combined Positive Score |
| FDA | Food and Drug Administration |
| H&E | hematoxylin and eosin |
| HNSCC | head and neck squamous cell carcinoma |
| IHC | immunohistochemical |
| ROI | regions of interest |
| WHO | World Health Organization |
Introduction
Oral and maxillofacial pathology is the science of identifying and managing disease of the oral and maxillofacial complex; oral pathologists use clinical, microscopic, radiologic, and serologic examination techniques to establish a diagnosis and provide treatment. Given the overlap between multiple dental specialties, this article will focus on artificial intelligence (AI) in the application of microscopic examination techniques.
Background of Conventional Laboratory Techniques
The method by which a piece of tissue becomes a diagnosis is not frequently known to those outside of the practice of pathology, and it is prudent to understand the details of how glass slides are made in order to understand the potential preanalytical variance and how that relates to training AI on pathology data; it can also give some insight as to how the practice of oral pathology is changing (and may change) with the application of AI.
Once a piece of tissue comes into the laboratory (often stored in a fixative like formalin, which irreversibly preserves the tissue and increases its firmness), it goes through multiple steps before it comes a glass slide ready for review ( Box 1 ). First, it is accessioned into the laboratory information system so it can be electronically tracked through every step of the process. The tissue is then “grossed,” where it is examined with the naked eye, described in the report, and cut into smaller pieces if it is a large piece of tissue. The tissue is placed into small boxes (“cassettes”), which are then filled with paraffin wax; these are called “blocks.” The blocks are then taken to a microtome (functionally a medical-grade, highly precise deli slicer) to cut the wax and tissue into thin (5–15 μm), nearly transparent slices, which are placed onto glass slides. These must be stained before review; almost all tissues relevant to oral pathology are stained with hematoxylin and eosin (H&E) by default. Following staining, a coverslip is placed over the tissue. This is then reviewed by a pathologist, who looks at the slide under a microscope and provides a diagnosis by writing a pathology report, usually by typing semistructured free text. The report is then released in a “final” state; further comments to a report may be provided in an “addendum,” or the report may be modified in an “amendment.” In more diagnostically complex cases, the pathologist may request additional stains to evaluate the presence of certain cellular staining patterns that may assist in providing a more specific diagnosis or provide predictive or prognostic variables relevant to the patient’s disease.
Box 1
Overview of a typical pathology workflow
Each step may have variability within a given laboratory and also among laboratories.
-
•
Accessioning
-
•
Grossing
-
•
Embedding
-
•
Microsectioning
-
•
Staining
-
•
Coverslipping
-
•
Digital scanning
-
•
Pathologist review
Implementation of Digital Histology
To enable the use of AI on histologic tissue sections, the slide must be digitized, ideally at high magnification. To do so, significant costs are incurred: an entire digital pathology workflow for clinical use requires purchasing slide scanners, digital storage space, a digital slide viewing software, high-resolution monitors for viewing the slides, and maintenance for all those components. These costs, unlike the transition from handwritten clinical notes to electronic health records and film radiology to digital radiology, are additive rather than a replacement for the old method—digitization of slides involve an additional step following the conventional workflow. Laboratory leadership must also consider modifying their existing workflow to optimize the slide scanning process (eg, at our institution, we chose to switch to film coverslips to avoid errors caused by misaligned glass coverslips and excess adhesive). Institutions that have adopted digital pathology workflows have done so to centralize processing while allowing pathologists to work remotely, pursue cost savings relating to storage of physical media, and to prepare for AI-enabled workflows.
The amount of data available depends on how much tissue is on the slide, the magnification used, and if multiple focal planes, or “z-stacks,” are scanned (in practice, z-stacking is not often used in oral pathology images). To provide an example, Table 1 contains metadata from 2 digital pathology images, 1 small biopsy and 1 large resection, that were originally scanned at 40× on the Philips Ultrafast Scanner (Koninklijke Philips N.V., Amsterdam, Netherlands) as ∗.isyntax files. Each was converted to TIFF files (with quality parameter set to 80, the default), exported once at 20× magnification and again at 40× magnification, and metadata was extracted using QuPath (an open-source digital pathology software). The TIFF file for the biopsy is significantly smaller than the resection, and at 20×, the files are smaller than at 40× because a digital slide scanned at 40× objective has 4 times the area and approximately 4 times as many pixels as a slide scanned with at 20×. While most oral pathology biopsy cases (in the author’s experience) consist of a single slide, a resection for head and neck cancer frequently consists of dozens of slides from multiple specimens: separately submitted margins, the resection itself, and lymph nodes. The specifics of the case can cause the amount of data generated to vary wildly; a single large resection case at our institution may use hundreds of gigabytes of storage.
Table 1
Digital pathology file size based on size of tissue and magnification
| Biopsy | Resection | |||
|---|---|---|---|---|
| 20× | 40× | 20× | 40× | |
| File size (GB) | 0.070 | 0.245 | 0.392 | 1.231 |
| Pixel dimensions | 28,160 × 17,920 | 56,320 × 35,328 | 77,824 × 43,080 | 155,648 × 92,672 |
| Total pixels (M) | 504.6 | 1989.7 | 3352.7 | 14,424.2 |
This is a comparison of metadata between a resection and biopsy slide, each at 20× magnification and 40× magnification. A 40× magnification increases data size by about 4×, and resection tissues tend to be much larger in area than biopsy tissues.
The quantity of data has implications for considering the cost of implementing a digital pathology workflow and also for how AI works in these cases. One consideration would be to use lower magnifications for AI tasks, which would reduce the amount of data that needs to be processed. However, depending on the AI task, the magnification could make a difference in performance. Some sample images from the biopsy and resection earlier are provided in Fig. 1 . While little visual difference can be appreciated for the files exported at 20× and 40× resolution at low power (4×), as magnification increases the 20× file becomes blurrier compared to the 40× file. A task that relies on detailed nuclear features might benefit from the higher resolution 40× (and possibly a higher TIFF quality). However, tasks that rely on low-power features (eg, identifying large metastatic foci in lymph nodes) may be able to function reasonably at 20× or lower magnification. Regardless, for AI tasks in digital pathology, the resolution of the image (pixels per micrometer) is important, and should be reported in publications or when considering the implementation of an AI tool in pathology practice.
Visual comparison of digitized pathology for a biopsy and a resection, scanned at 20× and 40× magnification ( top ). Machine learning tasks can take images from a tissue level specified ( left ); while minimal visual differences can be seen at lower magnification, at higher magnification tissue features begin to lose definition, which can impact downstream performance even if the differences seem minimal to the human eye.
General Considerations of the Digital Pathology Workflow on Artificial Intelligence
There are multiple implications that this workflow has on the use of AI in digital pathology, from a practical perspective and from a reporting/research perspective. Due to the size of the images, a single digital slide cannot be used in toto in an AI algorithm—most AI algorithms expect an image no larger than perhaps 1024 × 1024 pixels large, so researchers must have mechanisms to break the slide down into smaller components (regions of interest, ROI) that can be reviewed by AI. Combined with the highly variable amount of data available for each case, algorithms for research and for clinical use must be able to handle these data by either identifying areas of interest or performing some sort of aggregation technique to designate a single class for multiple regions and multiple slides. For example, a task where an AI is performing the pathologist’s primary task—providing a diagnosis on a single specimen—the AI must be able to review all ROIs on each slide, then consolidate the results from all slides for that specimen to provide a final diagnosis.
Another substantial issue that the overall pathology workflow imposes on the use of AI in digital pathology is that many steps in the slide preparation process are inconsistent among laboratories—sometimes, even within a single laboratory. Variability may be present in
-
•
Tissue sampling procedures (in resections especially, as sampling decisions are not necessarily consistent)
-
•
Artifactual changes, including changes from frozen sectioning before the slide is submitted for permanent/formalin-fixed sections
-
•
Quality of block cutting (variable tissue thickness and artifactual holes in the tissue)
-
•
H&E staining (particularly in the intensity of hematoxylin)
-
•
Immunohistochemical (IHC) staining (intensity of staining, background “blush”)
In addition to the prescanning differences, digital pathology may also be different among laboratories. There are multiple digital pathology vendors, and because there is not yet a consistent digital slide format, digital pathology scanners output a wide variety of different file types that each store image data differently. A further complication is that many of these file types are proprietary and full specifications are not always available for review. An AI model that is trained on one type of scanner may not translate well to a different scanner.
Due to the substantial preanalytical variance present in digital pathology data, extreme caution should be taken before using an “out-of-the-box” solution in a new setting. While there are mechanisms to address variability of input data (such as stain normalization ), AI models in digital pathology should undergo robust validation testing on real-world data in an institution before going “live,” and companies creating these models would ideally release some level of demographic information describing their dataset so health care professions can make educated decisions about the applicability of those models to their specific contexts.
Examples of artificial intelligence models in oral pathology
The 2 data modalities we consider in this section are digital slide data and pathology report text data. AI imaging models using digital slides in oral pathology generally fall into these categories: (1) detection tasks (“is cancer present in the slide?”), (2) quantification tasks (“what percentage of cells stain positively with an IHC stain like Ki67?”), (3) prognostication tasks (“given this tumor, what is the likelihood that the patient would respond with chemoradiation?”), and (4) prediction tasks (“with conventional treatment modalities, what is the likely outcome for a patient with this tumor?”).
Artificial Intelligence in Head and Neck Squamous Cell Carcinoma
Carcinoma detection
In a production setting, carcinoma detection algorithms could be run on digitally scanned slides to triage cancer cases for a faster turnaround time and provide quality assurance that cancers are not missed by pathologists. It could also assist pathologists in identifying which slides in a case have cancer, which can be especially helpful in histologic review of lymph node dissections (which can have dozens of slides); while not yet reported in head and neck squamous cell carcinoma (HNSCC), a similar concept in identifying metastatic breast cancer in lymph nodes has been reported. Segmentation of carcinomatous components of whole slide images can also be a useful preprocessing step before evaluating features specific to cancer.
Generally speaking, AI performs fairly well at using histology to identify cases of conventional squamous cell carcinoma, which (in the author’s opinion) generally looks histologically distinct from other disease entities. Convolutional neural networks (CNNs) assessing entire slides were able to identify HNSCC with an area under the receiver operating characteristic curve (AUC) of 0.944 in one study. Recently developed foundation models also serve well in learning broader pathology datasets that be implemented in downstream tasks like detecting squamous cell carcinoma. In oral cancers (which included non-HNSCC cancers), one group tested multiple foundation models on a large dataset of over 400,000 histologic whole slide images and found that the AUC for detecting oral cancers was well over 0.90 for all foundation models. In human papillomavirus (HPV)-associated oropharyngeal squamous cell carcinoma, the AUC for the downstream task of assessing if a squamous cell carcinoma was HPV-positive was 0.93. A detailed investigation into less common subtypes of squamous cell carcinoma (like barnaculate or cuniculate carcinoma) has not been reported.
Some investigators are interrogating the use of clinical photography to ensure that lesions suspicious for carcinoma are detected and are appropriately biopsied; some studies report a sensitivity of at least 90% in identifying squamous cell carcinoma. , However, caution should be taken to avoid overinterpreting studies using data of unknown quality. A recent review found that there are potential issues with bias and lack of appropriate reporting in many publications that use clinical photographs, notably in ground truth designation (ie, omitting how the final diagnosis was eventually determined).
Carcinoma prognostication
Some publications address the prognostication of squamous cell carcinoma behavior, some utilizing histologic images and others using only clinical data available prior to resection (such as size). For example, in one study using Surveillance, Epidemiology, and End Results Program, United States (SEER) data, investigators attempted to predict overall survival by using an ensemble technique (using results from multiple networks together), resulting in an AUC of 0.70 ; another group attempted a similar approach to predict overall survival by comparing single different machine learning methods on an institutional dataset that resulted in an AUC of 0.71 in their best model. In another study using the National Cancer Data Base, the authors achieved an AUC of 0.80 compared to Tumor, Node, Metastasis (TNM) staging’s AUC of 0.68, though the authors did not specify which TNM edition was used. Another group used a Cox proportional hazards deep neural network in a single-institution dataset resulting in a concordance index of 0.78.
Other studies also incorporate histologic imaging as the primary data modality or as a component in a larger dataset. Some studies have correlated tumor nuclear features with outcomes, while other studies suggest that there is prognostic value in quantifying T-lymphocyte infiltration , (notably a task that would be prohibitively time-consuming for a pathologist to fully quantify). Some studies incorporate several data modalities; for example, in one multimodal study using Cancer Genome Atlas (TCGA) clinical, histologic imaging, and genomic data, the investigators reached a concordance index of 0.834.
Overall, it is challenging to confidently determine which machine learning methods work best at prognostication due to differing reporting metrics, lack of validation on a shared multi-institutional dataset, and different goals in prognostication. However, the trend appears to be moving toward a systems approach with multimodal datasets consisting of histologic imaging, radiologic imaging, clinical features, and molecular/genetic data, with the assumption that each of these modalities captures a different piece of the prognostication puzzle.
Stay updated, free dental videos. Join our Telegram channel
VIDEdental - Online dental courses