Artificial intelligence and machine learning for optical coherence tomography-based diagnosis in central serous chorioretinopathy

Full Text

In ophthalmology in general and in retinology in particular, diagnostic approaches are based mainly on the technologies related to obtaining and analyzing images. However, large volumes of data requiring analysis and the complex nature of images analyzed can reduce the effectiveness of such diagnostic approaches. This is most noticeable in screening programs and primary diagnostics where the need for automation and increased accuracy of analysis is the highest.

Recently, with the exponential growth in automation, relatively quick analysis of a large amount of data has become possible; this formed the basis of artificial intelligence (AI) technologies. AI enables the use of a computer to solve problems without using a strict algorithm. An example of such a task is image recognition to identify the displayed objects, regardless of their variable non-specific characteristics and the environment in the image. In this case, a specific problem is solved by learning during the process of the preliminary solving of many similar problems, that is, machine learning. Thus, the use of AI and machine learning, as methods to solve problems with the analyses of diagnostic images, could revolutionize the diagnosis of ophthalmic diseases. An example of AI and machine learning clinical use is a screening for the presence of diabetic retinopathy using machine analysis of fundus images with high accuracy [1].

Optical coherence tomography (OCT) is a technology of non-invasive in vivo imaging of ocular structures with a resolution of up to 5 microns. OCT is suitable for the screening approach in retinology because it enables rapid and non-invasive diagnoses of all central retina main diseases (maculopathy).

OCT is used to diagnose central serous chorioretinopathy (CSС). With the advent of OCT, CSC diagnosis has become relatively easy; however, without the use of fluorescein angiography (FA), one of the methods of diagnosing CSC is by excluding other possible causes of subretinal fluid accumulation. In this case, the analysis of the retinal pigment epithelium (RPE) state is crucial, and it remains relatively intact in the acute form of this disease [2]. A well-known limitation of OCT use for CSC diagnosis is the inability to identify the leakage point that is necessary not only to verify the diagnosis, but also for treatment, if indicated. Nevertheless, several studies have found that the identification of a leakage point using OCT is possible in at least some CSC patients; however, this approach is difficult even for an experienced specialist [3–5].

Thus, improvement in the analysis of OCT image in CSC represents an important clinical problem that can be solved using machine learning. The potential aim of machine recognition of OCT data can be reduced to the tasks that a specialist solves in diagnosing CSC on OCT basis: 1) the assumption that this patient is suspected to have CSC (as per the presence of a detachment with respect to unchanged neuroepithelium, 2) ruling out of similar pathological conditions, primarily of wet age-related macular degeneration according to the degree of RPE changes, and 3) identification of the area that is supposed to be responsible for leakage. The possibilities to detect subretinal fluid using machine segmentation have already been shown [6–8]. However, a successful solution of all three problems not only enables to classify the OCT data with high degree of certainty as relevant to CSC, but also creates the basis for a treatment plan, if necessary.

This study aimed to investigate the possibility of using machine learning to detect the retinal neuroepithelium (RNE) isolated detachment and RPE changes as diagnostic criteria for CSC.

MATERIAL AND METHODS

Data collection

The study included patients with acute CSC with a duration of ≤6 months with a leakage point verified using FA. Exclusion criteria were ascertained recurrent episodes in the disease history; signs of chronic CSC (marked changes in the pigment epithelium, RNE atrophy, diffuse leakage in FA); any concomitant disease of the posterior segment; changes in the transparency of the optical media that impede adequate visualization of the posterior segment; and OCT scan quality less than 7/10.

All patients underwent standard ophthalmologic examination and OCT. OCT was performed using RTVueXR Avanti (Otovue, Fremont, CA) and included a 6-mm three-dimensional (3D) scan (400 repeated B-scans with 400 A-scans in each) using the Angio Retina HD protocol, centered on the center of the macula or the leakage point. In the case of the presence of several RNE detachments, which were not within one scan limits, each detachment was included in a separate scan

Data separation

Structural en face images in the Custome layer (between the segmentation line of the Bruch’s membrane at the position of 0 μm, and the segmentation line of the Bruch’s membrane at the position of 10 μm) were exported to visualize the detachment area and create masks for the spatial distribution of the RNE detachment areas, areas of RPE changes and, separately, areas of RPE changes, responsible for leakage. The area responsible for leakage was determined by superimposing the en face image on the FA image. Fluorescein angiography was performed as per the standard protocol; images taken at 20–35 s were used for analysis.

Creating a learning set

In accordance with the tasks of neural network learning, pathological changes were divided into following three categories: (1) the presence of subretinal fluid, (2) anomalies of RPE, and (3) leakage points (3). If the category was revealed on a 3D scan, the coordinates limiting its area were noted. Category 1 area was further refined using the two-dimensional (2D) mask of the spatial distribution of the RNE detachment areas. Thus, a 3D OCT image was compared with binary groundtruthmasks (by the number of categories) indicating the bounding coordinates of the area of interest. Original OCT scan files were read into numpy arrays, translated into a logarithmic scale, and normalized. The resulting arrays were converted to 2D images, similar to A- and B-scans, and compared with the corresponding section of the 3D mask. Ten out of 46 OCT images were randomly selected as test images and were not presented to the neural network during the learning. The learning set for a separate task included images that showed an area of interest. Therefore, the amount of data differed for each category, amounting to 4800/1400 learning/test images for class 1, 2000/802 for class 2, and 1504/408 for class 3. To increase the power of the learning set, the data were augmented; the image sizes could be enlarged/reduced by 0.4 from the original size and reflected horizontally or rotated by an angle of up to 30°. The size of the images transferred into the model was 256 × 256 pixels.

Statistical methods

The models were trained to solve the binary segmentation problems; therefore, in order to assess the quality of the algorithm, we used an error matrix (TP – true positive, TN – true negative, FP – false positive, FN – false negative) and the indicators based on it, namely sensitivity, specificity, accuracy, F1 measure.

Sensitivity = TP/P

Specificity = TN/N

Accuracy = TP/(TP + FP)

F1 measure = 2TP/(2TP + FP + FN)

The receiver operating characteristic (ROC) and area under curve metrics were not applicable due to significant class imbalance. The ROC curve indicates the quality of classification in the coordinates of sensitivity and specificity.

When analyzing the images, two main approaches were used, classification and segmentation, corresponding to the analysis of the array of scans and each individual image.

Neural network architecture

To solve the problem of lesion segmentation on OCT scans, we chose an architecture similar to Unet [9] that is widely used for medical data analysis. A neural network consists of four convolution layers (32–32–64–64 filters with 3 × 3 pixels in size), alternating with subsampling operations, and four subsequent convolution layers, alternating with upsampling operations (64–64–64–64 filters with 3 × 3 pixels in size). After the first operation, increasing the discretization, the information from the initial convolution layers is added to characteristics extracted in the previous layer. The process of segmentation is completed with two additional convolution layers (32–1 filters with a size of 3 × 3 pixels). ReLU, the output sigmoid layer, is used in the inner layers as an activation function. The total number of trained network parameters is 305 K.

RESULTS

The study included 40 patients (46 eyes), the average age of the patients was 44.8 ± 10.8 years, and there were 34 men and 6 women. A total of 46 individual 3D scans were used, each of which was considered as a separate case.

For each of the tasks assigned, a neural network was trained, highlighting pathological areas in images of A- and B-scans. The result of the network is a two-dimensional mask with a size similar to the input data (256 × 256 pixels). The mask is a map of the probabilities of assigning each individual pixel to the target class. To calculate the quality of the algorithm operation, the output mask was binarized by the value of 0.5, where all pixels with probabilities greater than the specified were taken as positively classified.

The quality of the model’s work was estimated by the scan as a whole, that is, B-scans were extracted from the OCT data that were individually transferred to the model for analysis; thereafter, the predictions obtained were assembled in a 3D volume and compared with a 3D mask. Table 1 represents the results averaged over the entire test set. It should be remembered that the marking was not pixel-by-pixel and the mask approximately limited the pathological area (bounding box marking). Moreover, the area of interest was approximated to a rectangular one that, in the process of learning and testing the models, led to errors in segmentation (Fig. 1).

 

Fig. 1. Example of detection of subretinal fluid within an individual B-scan by the taught-in neural network: а – a raw cross-sectional OCT scan; b – resultant image of detection of subretinal fluid accumulation area; c – distribution of a probabilistic characteristic of subretinal fluid presence

Рис. 1. Пример детекции обученной нейронной сетью субретинальной жидкости на индивидуальном В-скане: а — исходный В-скан; b — результат распознавания зоны скопления субретинальной жидкости; c — распределение вероятностной характеристики наличия субретинальной жидкости

 

Table 1 / Таблица 1

Statistical indices of neural network working efficacy upon B-scan segmentation

Статистические показатели эффективности работы нейронной сети по сегментации В-сканов

Indicators	Sensitivity	Specificity	Accuracy	F1-measure
Subretinal fluid	0.67	0.98	0.70	0.68
RPE abnormalities	0.51	0.99	0.13	0.21
Leakage points	0.60	0.99	0.05	0.09
Note. RPE – retinal pigment epithelium.

 

In addition, we evaluated the quality of image classification for the entire set of B-scans; each B-scan was assigned a positive class if at least 5 pixels that were classified as pathological were found in it. Categories obtained for the sections were compared with the initial data (B-scan was considered positive if the mask contained positively marked pixels). The results are presented in Table 2 (Fig. 2).

 

Fig. 2. Representative example of subretinal fluid detection within a stack of B-scans by the taught-in neural network: а – en face image demonstrating the subretinal fluid distribution; b – resultant image after detection and mapping of subretinal fluid from a stack of B-scans; c – distribution of a probabilistic characteristic of subretinal fluid presence on an individual B-scan. The dashed line represents a position of cross-sectional scan

Рис. 2. Репрезентативный пример детекции субретинальной жидкости на совокупности В-сканов обученной нейронной сетью: а — анфас-изображение, демонстрирующее распределение субретинальной жидкости; b — результат распознавания и картирования зоны распределения субретинальной жидкости в совокупности В-сканов; c — распределение вероятностной характеристики наличия субретинальной жидкости на индивидуальном В-скане. Положение скана отмечено пунктирной линией

 

Table 2 / Таблица 2

Statistical indices of neural network working efficacy upon B-scan set classification

Статистические показатели эффективности работы нейронной сети по классификации совокупности В-сканов

Indicators	Sensitivity	Specificity	Accuracy	F1-measure
Subretinal fluid	0.61	0.99	0.99	0.76
RPE abnormalities	0.14	0.95	0.94	0.24
Leakage points	0.06	1.0	1.0	0.12
Note. RPE – retinal pigment epithelium.

 

DISCUSSION

This study analyzes the capabilities of machine learning to classify retinal changes in acute uncomplicated CSC. The neural network used, and the learning algorithm enabled us to achieve high sensitivity and specificity in the diagnosis of RNE detachment both within each individual B-scan and in the set of B-scans. Thus, the problem of mapping the distribution of subretinal fluid can be considered successfully solved. In the indication of RPE anomalies, a more difficult task, the neural network showed slightly worse performance, primarily due to reduced sensitivity indicators when analyzing the set of scans, although the detection of noticeable RPE alteration (for example, high detachments) is performed efficiently and with high specificity. Despite the fact that the totality of these diagnostic algorithms can form the basis of the primary diagnostics of CSC using OCT, the current state of the model does not allow the use of all the algorithm elements with equal efficiency.

There are two main reasons for reduced system performance when analyzing RPE changes, including the difficulties of accurate marking and the limited representation of this class of changes in the learning sets. We believe that the latter is most important. With a high prevalence in the population, only few patients were suitable for training the neural network, as indicated in the inclusion and exclusion criteria. High requirements for uniformity and resolution of scans also limit the input flow of patients. In addition, some patients had to be excluded from the learning set for subsequent use in the test. Thus, only approximately 40 individual patients were included for learning.

The problem of large data arrays for learning is well known in the field of AI. In previously published works that have studied the use of machine learning technologies for analyzing OCT data, the learning sets include up to 100,000 individual scans. The total number of scans in our learning set was more than 16,000. However, the representation of the analyzed characteristics was uneven. If, according to rough estimates, 40%–50% of B-scans included RNE detachment, the RPE changes were observed in maximum 15% of the scans, and the leakage point was represented in no more than 5% of the scans. This reflects the results of the neural network in identifying these types of changes. Thus, 10 times more individual cases may be required for adequate training to enable accurate identification of the leakage point.

Another problem mentioned above is the accuracy of mapping the changes. It was not possible for a limited group of specialists to mark each individual B-scan from a large data array; therefore, the use of a coordinate system (bounding box marking) is most appropriate. However, this leads to the fact that significant representation in pathological areas belongs to normal areas or those non-belonging to this type of pathology. This becomes apparent when it is considered that the marking area is a rectangle, and any pathological area has an irregular shape. Although this problem cannot be solved directly, by improving the marking, increase in a learning set can overcome this limitation. Moreover, on a large set, this can give advantages in learning because the system will become “familiar” with “imperfect” data.

Despite the fact that the study used a small learning set, its results show the prospects of using AI in the field of OCT diagnostics in ophthalmology. Moreover, the logical diagnostic approaches used in our work demonstrate their applicability, at least within the framework of CSC and, possibly, similar diseases.

The algorithm described is based on the assessment of the entire data array of a 3D scan, not just the individual B-scan; this is important for the differential diagnoses and the diagnosis in asymptomatic patients. The absence of significant changes outside the detachment of the RNE is significant for the differential diagnosis because with other maculopathies, the RNE and RPE outside the detachment of RNE can show various pathological changes. Analysis of the “raw” file enables us to exclude the human factor input at the stage before data processing with neural network. This also enables us to rely on the identification of areas responsible for leakage because potentially, such an area can be localized in any site of the RNE detachment.

We know that 3D scanning is an option for OCT option, using which it is possible to form an idea on the state of the entire macula based on one scan. However, for a long time, when evaluating a large fundus area, there was a loss in image quality of the cross-sectional tissues, and ultrastructural analysis of the condition of the RNE and RPE on such scans could not be performed adequately. During a couple of years, two events in the process of OCT evolution changed the situation, namely a multiple increase in the scanning speed and the use of motion artifact correction technologies. All this led to an increase in the resolution of each individual cross-sectional scan within the 3D scan to values that ensure the identification of minimal changes in RPE, for example, characteristic of the leakage point in CSC. In addition, this enabled us to reduce the step between successive cross-sectional scans and minimize the likelihood that pathological changes will be localized between the scans. Thus, the analysis of the retina and RPE with high resolution within the three-dimensional scan is an additional, however not fully appreciated, option of OCT angiography, which fits well with the concept of machine learning.

The strength of this study is in the fact that, in addition to the direct task of improving the CSC diagnostics, it demonstrates the possibility of solving similar classification problems to identify the source of exudation in other maculopathies. For example, this approach can be implemented for planning laser interventions in case of diabetic macular edema associated with leakage from a microaneurysm. The work of the classifier for these cases will have a similar algorithm and will include 1) detection of a pathological area, 2) identification of suspected microaneurysms, and 3) identification of microaneurysms responsible for leakage and serving as targets for laser coagulation. In addition, in this study, learning and test sets were obtained in different patients; therefore, there was no duplication of images in the learning process and the experiment.

This study has certain limitations. First, we did not compare the diagnostic capabilities of the system for a mixed sample that included other diseases in addition to CSC. This issue is important for direct clinical use; however, this needs to be addressed in a separate study. Second, for CSC diagnosis, the state of the choroid that is assessed based on the morphological changes, internal structure, and thickness, is crucial. These data could be used in primary (and differential) diagnosis because an increase in the vascularity index and choroidal thickness enables us to rule out several other maculopathies. This increases the accuracy of the identification of the leakage point that is usually associated with local thickening of choroidal vessels. Although we do not know what role the analysis of the choroid played in our algorithm, a three-dimensional scan based on averaging of the four scans may not be sufficient to fully analyze some cases using spectral OCT data because this variant of OCT is limited by the depth of penetration of the scanning beam. Third, a relatively small number of patients were enrolled in the study. As in many similar studies, this drawback is partially offset by image fragmentation. However, this imposes restrictions on the extrapolation of the results to the entire spectrum of CSC that, in general (unlike the acute form), is a rather polymorphic pathology.

CONCLUSION

This study demonstrated high potential of AI technology and computer-aided learning in diagnosis of individual morphological characteristics of CSC based on the data of 3D OCT scanning. Among the main morphological changes, RNE detachments and RPE alteration are most accurately detected both within the individual B-scans and in the en face image. Detection of the leakage point by OCT data based on machine learning is possible; however, adequate operation of such an algorithm requires a large array of learning data.

Disclosure of conflict of interest

The authors declare no conflict of interest or material interest in this study.

Contribution of authors

D.S. Maltsev, E.Yu. Malakhova created the concept and design of the study.

D.S. Maltsev, E.Yu. Malakhova collected and processed the material.

D.S. Maltsev, E.Yu. Malakhova, and A.N. Kulikov were involved in writing the text.

A.N. Kulikov edited the text.

About the authors

Alexey N. Kulikov

S.M. Kirov Military Medical Academy

Email: alexey.kulikov@mail.ru
SPIN-code: 6440-7706

MD, PhD, DMedSc, Professor, Head of the Department

Russian Federation, Saint Petersburg

Ekaterina Yu. Malahova

Pavlov Institute of Physiology Russian Academy of Sciences

Email: katerina.malahova@gmail.com

Associate Researcher

Russian Federation, Saint Petersburg

Dmitrii S. Maltsev

S.M. Kirov Military Medical Academy

Author for correspondence.
Email: glaz.med@yandex.ru
ORCID iD: 0000-0001-6598-3982

MD, PhD, ophthalmologist of the Ophthalmology Department

Russian Federation, Saint Petersburg

References

Supplementary files