Application direction:

Hyperspectral imaging technology has shown broad application prospects in the field of medical diagnosis. Hyperspectral imaging can simultaneously obtain the spatial structure and spectral information of tissues in a single image, revealing the microscopic optical characteristics of cells and tissues in different bands, and achieving non-invasive and quantitative identification of diseased tissues. Its application directions in the field of medical diagnosis mainly include: optical diagnosis of colorectal polyps and precancerous lesions, achieving real-time differentiation between adenomatous and non tumor lesions in endoscopic examination; Rapid tissue identification and margin localization during surgery to assist doctors in accurately removing lesions; Pathological section analysis and automatic classification, etc.

Background:

Colorectal cancer is the second leading cause of cancer death worldwide, and its prevention and control depend on early detection and removal of precancerous lesions - colorectal polyps. Currently, colonoscopy is considered the primary method for preventing advanced colorectal cancer and reducing mortality. However, in actual examinations, the majority of polyps found are small in size (diameter 6-9 mm) or extremely small (1-5 mm), with a very low proportion possessing advanced pathological features or potential for cancer. The traditional approach is to remove all polyps and perform pathological testing, but this method increases costs and workload. To this end, a "resection and discard" strategy based on optical diagnosis has been proposed, which utilizes high-precision endoscopic imaging technology to accurately determine the nature of polyps under endoscopy, thereby eliminating some pathological testing.

However, existing optical diagnostic techniques such as narrowband imaging NBI, blue light imaging BLI, etc. still have limitations in clinical practice: accuracy relies on experienced endoscopists,Poor consistency in diagnosis for beginners; Meanwhile, although artificial intelligence has been used to assist in polyp recognition, its promotion is still hindered due to issues such as insufficient specificity, poor model interpretability, and large heterogeneity of training data.

To overcome the above difficulties, this study introduced hyperspectral imaging technology. This technology combines digital imaging with spectral analysis, which can simultaneously capture the morphology and spectral characteristics of tissues at each pixel level, obtaining rich spectral information reflecting the biochemical composition and pathological changes of tissues. Compared with traditional RGB images, hyperspectral images can reveal tissue differences that are not visible to the naked eye, providing new basis for histological property recognition and pathological grading.

Author Information:Peng Cheng, Qilu Hospital, Shandong University

Journal source:Cancer Medicine

Research Content

The aim of this study is to develop and validate a colorectal polyp tissue classification algorithm based on hyperspectral imaging and deep learning (residual network, transfer learning), which can quickly distinguish between tumorous and non tumorous polyps during endoscopic examination. This will promote the implementation of AI assisted "resection and discard" strategies, improve diagnostic accuracy, and reduce medical costs.

Experimental Design

During the colonoscopy procedure, all polyps are first removed using a cold snare (CSP); For small lesions with a diameter of ≤ 3 mm and high difficulty in CSP technology, biopsy forceps are used to completely remove them. This study only included fresh mucosal specimens with polyps<1 cm; To enable the model to acquire higher-order pathological features, advanced tumor biopsies were also collected simultaneously. Once the polyp or lesion is removed/clamped, it should be immediately imaged using a hyperspectral (HS) camera according to standardized procedures. The brief process is shown in Figure 1A.

The hyperspectral imaging data was collected using the GaiaMicro-V10 E-HR device from Shuangli Hesheng, which includes the GaiaMicro series hyperspectral camera, microscope, calibration whiteboard, system holder, and microscope objective. The Gaia Micro series includes a built-in push scan hyperspectral camera that can push space to obtain hyperspectral data cubes. Collect images with a spatial size of 1101 rows to generate 960 × 1101 × 360 (number of rows × number of rows × number of bands).

All colorectal polyp samples were fixed in formalin, embedded in paraffin, sliced, and stained with H&E. Two experienced pathologists independently reviewed the images and provided the final diagnosis based on the World Health Organization's classification criteria for colorectal tumors. According to the consensus of the Multi Society Colorectal Cancer Working Group in the United States, the samples were divided into two categories: non tumor lesions and tumor lesions, with tumor lesions further subdivided into non progressive adenomas and progressive tumors.

research methodology

To balance the sample size of different histological labels, we implement data augmentation for each category by randomly selecting hyperspectral images from multiple fields of view for each sample. To address the challenges of high dimensionality and computational complexity in hyperspectral data, we extract spectral and spatial information from the original hyperspectral cube and reconstruct it into a two-dimensional (2D) image; Figure 1B visually illustrates the correspondence between one-dimensional spectra and 2D images. After dimensionality reduction, each HS image can generate 900 2D images.

This study is based on a two-dimensional convolutional neural network (2D CNN) trained model for automatic classification of colorectal lesion images. This network can combine both spectral and spatial information of samples to comprehensively analyze organizational features. In order to overcome the problems of gradient vanishing and network degradation that traditional convolutional neural networks are prone to when the network layers are deepened, and further improve model performance, the residual network (ResNet) structure was adopted in the study. It is a method of fine-tuning existing model parameters to adapt to new tasks or data features. In this study, the researchers adopted a transfer learning strategy, loading pre trained ResNet model parameters as initial weights, and then adding fully connected layers at the end of the model, and re establishing all layer structures of the network to enhance the model's recognition ability for different pathological types of colorectal polyps (as shown in Figure 1B).

Figure 1 Experimental design and workflow (A) Schematic diagram of in-situ hyperspectral imaging and spectral analysis of fresh polyp tissue during colonoscopy. (B) Spectral spatial characteristics of hyperspectral cubes, conversion of cubes to two-dimensional images, and schematic diagram of residual neural network model architecture. (C) Assessment of optical diagnostic ability of endoscopists after two weeks of white light endoscopic imaging and HSI-AI assisted diagnosis.

To evaluate the diagnostic ability of endoscopists for small and small colorectal polyps, the study invited 4 senior endoscopists (cumulative>2000 screening colonoscopes) and 6 junior endoscopists (cumulative 200-400 cases) to participate in the test. The test is conducted in the form of a questionnaire, based on white light endoscopic images of lesions in the test set for judgment. To evaluate the diagnostic efficiency of primary endoscopists and expert endoscopists, two re examinations were conducted on the images. Firstly, the endoscopists were independently tested, and then the predictive results of the HSI-AI model were communicated to the endoscopists before the final diagnosis was made, known as the HSI-AI assisted diagnostic test. The two tests were conducted with a two-week cleaning period between them, and the final results were compared.

Evaluate the diagnostic performance of the model through the confusion matrix. Accuracy refers to the proportion of correctly classified samples by a classification model among all observed samples. To comprehensively evaluate the performance of the model, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were also calculated, and the overall diagnostic ability of the model was reflected through the receiver operating characteristic curve (ROC curve) and its area under the curve (AUC). The secondary evaluation indicators include the diagnostic performance of primary and expert endoscopists, as well as the auxiliary efficiency of hyperspectral artificial intelligence (HSI-AI) models in their diagnosis. All diagnostic indicators of endoscopists are expressed as median and interquartile range (IQR). Using Mann Whitney U test to compare the differences between independent diagnosis and HSI-AI assisted diagnosis, bilateralpA value less than 0.05 is considered statistically significant.

result

A total of 94 tissue samples from 73 patients were included (64 from polypectomy and 30 from biopsy). The pathological evaluation results showed that 53 samples were low-grade dysplasia of tubular adenomas (non progressive adenomas, NAA), 28 were non tumor associated tumors (NNP), and 13 were progressive tumors (AN), including 6 cases of high-grade dysplasia of tubular adenomas, 3 cases of villous adenomas, and 4 cases of adenocarcinoma. After data augmentation, a total of 144900 two-dimensional images were generated from 161 hyperspectral images, which were divided into training, testing, and validation sets in a ratio of 6:3:1.

By comparing the typical spectral curves of different tissue types, it was found that there were significant differences in the spectral shape and peak size of the three types of tissues (NNP, NAA, AN) in the wavelength range of 406.9-662.1 nm. The results showed that there was a highly statistically significant difference in reflectance among the three types of tissues at a wavelength of 575 nm (p<0.001), indicating that hyperspectral imaging can effectively distinguish different pathological types of tissues.

To verify the reliability of the data and explore the correlation of two-dimensional image data, two sets of model experiments were conducted. In the first set of models, the researchers randomly shuffled the two-dimensional images and trained them without considering individual differences between samples. The results showed that the model achieved high accuracy in all groups, with AUC values of 0.97 and 0.96 for the training and testing sets, respectively, indicating good correlation and consistency between spectral images from different regions of the same sample (Figure 2A).

Figure 2 shows the training, validation, and testing results of the residual neural network for automatic diagnosis based on hyperspectral imaging (A, B) in the three classification tasks of non tumor polyps/non advanced adenomas/advanced tumors. (C, D) Performance comparison of multiple neural network models in automatic classification tasks.

In the second set of models, the researchers validated the data on an independent patient basis, grouping the data into sample groups and scrambling them to generate two-dimensional images. If more than 80% of the two-dimensional images generated from the same sample are correctly classified, the sample is judged as correctly predicted. The model performed well in classifying three types of organizations (NNP, NAA, AN), with an overall accuracy of 95.8% and an average AUC of 0.97. Among them, the predictive sensitivity for tumor lesions (including NAA and AN) was 99.0%, and the specificity was 96.0%, indicating that the model has the highest diagnostic potential in identifying tumor and non tumor lesions.

The study evaluated the diagnostic performance of white light endoscopic images and compared the differences in distinguishing between tumorous and non tumorous lesions among endoscopists before and after AI assistance. In the independent diagnosis stage, experienced endoscopists have a median single image diagnostic accuracy of 91.1% for tumor lesions, while junior physicians have only 71.4%. In the test set, the HSI-AI model classified each tumor type, and only one case of non progressive adenoma (NAA) was misclassified as a progressive tumor (AN). With the assistance of the HSIAI model, the overall median accuracy of junior endoscopists has increased to 94.7%, approaching the level of senior physicians. Self comparison showed that there was a significant difference in diagnostic performance between the primary group and the AI assisted group (p<0.01), while there was no statistical difference in the senior group (p=0.63). In terms of negative predictive value (NPV), multiple junior physicians exceeded 90% after AI assistance, which meets the requirements of the PIVI "resection discard" strategy; The overall median NPV of the primary group increased from 50.0% to 88.2% (p=0.013). Other diagnostic indicators also showed complete improvement, as shown in Figure 3 for junior physicians.

Figure 3 Comparison of diagnostic performance of primary endoscopists for tumorous and non tumorous colorectal polyps under white light endoscopic images without (left) and with (right) HSI-AI assistance

Conclusion

This study confirms that hyperspectral imaging (HSI) can achieve high-precision differentiation between adenomas and non adenomas in fresh colon tissue for colorectal polyps, and further identify advanced tumors. This tool significantly improves the diagnostic level of junior endoscopists and demonstrates the potential of AI assisted diagnosis and real-time synchronization with colonoscopy.