Diabetic Retinopathy (DR) is a type of disease in eyes as a result of a diabetic condition that ends up damaging the retina, leading to blindness or loss of vision. Morphological and physiological retinal variations involving slowdown of blood flow in the retina, elevation of leukocyte cohesion, basement membrane dystrophy, and decline of pericyte cells, develop. As DR in its initial stage has no symptoms, early detection and automated diagnosis can prevent further visual damage. In this research, using a Deep Neural Network (DNN), segmentation methods are proposed to detect the retinal defects such as exudates, hemorrhages, microaneurysms from digital fundus images and then the conditions are classified accurately to identify the grades as mild, moderate, severe, no PDR, PDR in DR. Initially, saliency detection is applied on color images to detect maximum salient foreground objects from the background. Next, structure tensor is applied powerfully to enhance the local patterns of edge elements and intensity changes that occur on edges of the object. Finally, active contours approximation is performed using gradient descent to segment the lesions from the images. Afterwards, the output images from the proposed segmentation process are subjected to evaluate the ratio between the total contour area and the total true contour arc length to label the classes as mild, moderate, severe, No PDR and PDR. Based on the computed ratio obtained from segmented images, the severity levels were identified. Meanwhile, statistical parameters like the mean and the standard deviation of pixel intensities, mean of hue, saturation and deviation clustering, are estimated through K-means, which are computed as features from the output images of the proposed segmentation process. Using these derived feature sets as input to the classifier, the classification of DR was performed. Finally, a VGG-19 deep neural network was trained and tested using the derived feature sets from the KAGGLE fundus image dataset containing 35,126 images in total. The VGG-19 is trained with features extracted from 20,000 images and tested with features extracted from 5,000 images to achieve a sensitivity of 82% and an accuracy of 96%. The proposed system was able to label and classify DR grades automatically.
Diabetic Retinopathy (DR) is a disease that occur in eyes that can ultimately damage the retina resulting in loss of vision. IDF (International Diabetes Federation) statistics had indicated that, the diabetic population worldwide was recorded as 500 million in 2018 which shall increase by 9.9% approximately in the next 5 years. India has a diabetic population of 38 million in 2018 which is expected to be 79 million by the year 2030. Type 2 diabetes largely affected the city community residents of India, but current researches evidently indicated its growth predominance in villages. Though several diabetic diagnosed patients may not have symptoms, there occurred morphological and physiological retinal variations. The initial clinical symptom was a mild diabetic retinopathy characterized by microaneurysms formation. When not properly diagnosed, it may lead to moderate diabetic retinopathy where variations in dimensions across veins and retinal blood vessels occur. The rigor and degree of such lacerations are indicated as severe diabetic retinopathy of non-proliferative type where the blood flow in retina gets rapidly increased. The restricted blood flow regions across the retina will provoke fresh growth in the microvascular region, which results in Proliferative Diabetic Retinopathy (PDR). Macular Edema (ME), appeared as retinal enlargement due to fluid leakage inside the macula. The setback is that, image interpretation was difficult due to the presence of image artifacts leading to poor image quality. Also, fundus photography cannot compute membrane thickness and detect the presence of edema. Left out anomalies related to diabetic retinopathy include neovascularization, venous beading, and intra-retinal microvascular growth. Moreover, detection of hard exudates near the optic disk was difficult that had the same intensity values. Extraction of optic disk from fundus image was quite challenging one. To enhance automatic diagnosis and classification of DR, the research approaches are
To propose an novel image preprocessing technique for detecting Region of Interest (ROI) of the anomalies.
To find the ratio of total contour length to true arc length of irregular blobs (white regions) as ROI in images, for automatic prediction of DR conditions.
Finally, classify the DR severity using extracted feature sets from segmented images using VGG-19.
Gargeya [
For classification of DR conditions, the model used sequence of operations as its fundamental blocks
Image augmentation
Image preprocessing
Deep convolutional neural network classification
Data augmentation is a key to collect the image data for experiments. As a part of analysis, a familiar KAGGLE competition dataset is utilized for the experiments, whose sample fundus image is shown in
Classes | DR condition | No. of images |
---|---|---|
0 | Normal | 25810 |
1 | Mild | 2443 |
2 | Moderate | 5292 |
3 | Severe | 873 |
4 | Proliferative | 708 |
Classes | DR condition | No. of images | No. of images for training | No. of images for testing |
---|---|---|---|---|
0 | Normal | 25810 | 14692 | 3673 |
1 | Mild | 2443 | 1388 | 347 |
2 | Moderate | 5292 | 3012 | 753 |
3 | Severe | 873 | 496 | 124 |
4 | Proliferative | 708 | 412 | 103 |
The KAGGLE dataset consists of substantial quantity of indefinable images because of the presence of artifacts and incorrect labeling. Therefore, image segmentation methods are used to detect the type and lesions present in order to annotate the labels, for detecting the DR severity levels in fundus images.
Amount of detected lesions from images | Corresponding DR severity grades |
---|---|
No lesions present | Normal |
Few microaneurysms and soft exudates | Mild NPDR |
More microaneurysms and soft exudates, less hemorrhages | Moderate NPDR |
More hemorrhages | Severe NPDR |
More hemorrhages and hard exudates | PDR |
Now, the main aim of image segmentation is to detect hard and soft exudates, hemorrhages and microaneurysms as white regions for the convenience of annotating the class.
During the process of image segmentation, saliency detection usually extracts visually salient regions from an image and generates the saliency map that is used as a beneficial tool for identification of visual details of low-level luminance from color images. This method was commonly preferred for automatic evaluation of objects and image regions without any prior knowledge. Thus, described as variations between pixels and its neighboring locality [
The image is comprised of color and brightness features. In order, to extract efficient color and brightness details, the saliency map is applied to the region of interest (ROI). Saliency map will be depicted as an image which displays distinct quality of every pixel. The purpose of saliency map [
Consider mr, mg and mb are the values of mean for red, green and blue color features, correspondingly. The pixel saliencies are evaluated, by squaring the difference between its feature and mean value. The saliency maps for r, g and b colors are denoted by Sr, Sg and Sb correspondingly and their sizes by M × N.
The final saliency color map for an image is obtained by summing Sr, Sg and Sb.
Next, gradient based structure tensors are used to improve the intensity changes on object edges and derive those edge elements by performing series of opening and closing, thresholding and erosion operations. The structure tensor was used to depict the edge or gradient details. It can powerfully describe the local patterns rather than directional derivatives along with its coherence estimation. Structure tensor can connect the shape and structure of objects in an image and determine the features of ROI for classification. This method the edge based active contours of smaller sizes were detected. In the regions near edges, the tensors will have higher values of magnitude and coherence, i.e., only one eigenvalue was larger compared with the rest, whereas corner areas has higher values of magnitude with less coherence.
Structure tensors have the ability to incorporate and retrieve magnitude information along two directions without canceling the gradients. Anatomical features like tissue boundaries or blood vessels appear as a collection of speckles that are correlated along a specific direction. These methods describe the local vicinity of a pixel better than traditional derivative approaches. Structure tensor [
In which,
Here, I is a grayscale image with scalar values
Moreover, orientation and structure magnitude details exist before the gradients, the structure tensor comprises of certain additional details. These details can be obtained using smoothing operation and estimates the orientation homogeneity within adjacent regions. Using structure tensor as a transformation about its principal axis
An original level contours have to be defined by inserting curves enclosing objects that needs to be identified. The gradient operator [
For continuous vector field v, the divergence operator can be defined using first order finite differences.
And its discrete form is represented by,
Optimized image is got by minimizing the cost function
In which,
The smoothed total variation gradient norm was described by
Concisely, in the proposed image segmentation process, saliency map was applied to the input fundus image after limiting its boundary. Structure tensor evaluates second order moment that was applied to both CLAHE enhanced green channel and gray scale image to evaluate the morphology gradient. The output is then subjected to optimize the contour area and perimeter by applying level set segmentation using gradient descent method for optimization.
From the output images obtained from proposed segmentation process, the statistical parameters like mean and the standard deviation of pixel intensities, mean of hue and saturation, deviation clustering from mean are found through K-means which are computed as features. In order to identify the size, location and region of DR involving hard and soft exudates, hemorrhages and microaneurysms, K-means CCA is applied. K-means clustering is applied to determine the feature sets in classification of the images according to severity of DR disease. It is observed that, this technique is quite beneficial to detect and differentiate the blob or object sizes, shapes and affected regions from segmented retinal images.
Differences | image005 | image007 | image013 |
---|---|---|---|
Standard deviation | 0.51 | 0.96 | 0.12 |
Intensity | 0.02 | 0.02 | 0.01 |
r-channel intensity | 0.74 | 0.34 | 0.42 |
g-channel | 0.49 | 0.25 | 0.21 |
deviation from mean of grayscale | 0.05 | 0.03 | 0.07 |
K-means distance | 0.96 | 0.97 | 0.99 |
CNN [
VGG-19 is so beneficial and it simply uses 3 × 3 convnet arranged as above to extend the depth. In order to decrease the size, max-pooling layers are applied as a handler. FCN layers are two in number to which have 4096 neurons applied. VGG is trained based on individual lesions and for testing all types of lesions were considered to reduce the number of false positives. Convolution layers perform convolution process over images at every pixel, allowing outcome to pass through the subsequent layer. Filters are used in convolution layer is of 3 × 3 dimension which are trained for feature extraction. Every stacked convolution layer is subsequently added with Rectified Linear Unit (ReLU) layer and max-pooling layer. ReLU is presently the best known non-linear activation function which allows only the positive portion of the input.
While, comparing the ReLU function with sigmoid function, ReLU is quite effective in computing to indicate the best convergence behavior that vanish the gradient issue. A down-sampling max-pooling layer is used after ReLU activation function. Generally, the filter of 2 × 2 dimension is considered of same step size. The output will be of maximal value in every sub-region. For dense layer, the activation function must be designed. The dropout layer was abandoned while random activation in the layer to make it zero value. The neurons are eliminated in random process during the training stage to reduce over-fitting issue. This dropout is applied during training.
This section discussed the experimental results obtained from classifier using proposed image preprocessing technique. For extraction and classification using features, 71% of images were used from KAGGLE dataset. In order to assess the performance of DR classifier through training and testing, the 80:20 ratio was applied on the dataset. KAGGLE, being the standard dataset it contains fundus images of five different DR classes namely, non-proliferative DR, mild DR, moderate DR, severe DR and proliferative DR with distribution as depicted in
As 71% of KAGGLE dataset images were used for training, their labeling method is crucial. The experts were asked to mark the areas related to the microaneurysms, hemorrhages, hard and soft exudates. The ground truth confidence levels mostly represented the decision certainty to mark if the findings were correct. The experts failed manually to investigate and assess the stage from symptomatic fundus images as reported in Krause et al. [
Proposed method | ||||
---|---|---|---|---|
Sl.No | Total contour area of white edges (A) | Total true arc length of white regions (B) | A/B | DR Conditions |
1 | 8433 | 1047 | 8.05 | Mild |
2 | 9000 | 328 | 27.44 | No PDR |
3 | 9113 | 275 | 33.14 | No PDR |
4 | 8336 | 4976 | 1.67 | PDR |
5 | 20212 | 12507 | 1.62 | PDR |
6 | 10072 | 1140 | 8.83 | Mild |
7 | 16272 | 7431 | 2.19 | Severe |
8 | 11618 | 3081 | 3.77 | Moderate |
9 | 10038 | 1877 | 5.35 | Moderate |
10 | 8949 | 322 | 27.79 | No PDR |
11 | 9641 | 1810 | 5.32 | Moderate |
12 | 8476 | 1122 | 7.55 | Mild |
13 | 14039 | 14546 | 0.96 | PDR |
14 | 13953 | 7858 | 1.77 | PDR |
15 | 18211 | 5412 | 3.36 | Severe |
16 | 15708 | 4930 | 3.19 | Severe |
17 | 11139 | 2588 | 4.30 | Moderate |
18 | 11664 | 1330 | 8.77 | Mild |
Label | DR conditions | Upper limit | Lower limit |
---|---|---|---|
0 | No PDR | >10 | |
1 | Mild | <10 | >7 |
2 | Moderate | <7 | >3 |
3 | Severe | <3 | >2 |
4 | PDR | <2 |
Parameters/ |
No |
Mild |
Moderate DR | Severe |
Proliferative DR |
---|---|---|---|---|---|
Accuracy | 98.28 | 98.72 | 98.4 | 99.6 | 99.6 |
Precision | 98.83 | 90.77 | 94.69 | 91.94 | 90.3 |
Specificity | 96.76 | 99.32 | 99.06 | 99.8 | 99.8 |
Sensitivity | 98.83 | 90.78 | 94.69 | 91.94 | 90.3 |
The experiment for testing is carried out using 5000 images from KAGGLE datasets. Various DR severity images are taken as per
The performance parameters like specificity, accuracy of evaluation was based on inference from confusion matrix. Here, diagonal values of confusion matrix depicts true values for analysis.
Here number of epochs is taken as 100. Here the network fits regularly on test data and provides maximum accuracy. The accuracy of training and validation is depicted in
The remarkable advantage of KAGGLE fundus image database for feature extraction and training the classifier is in that, it allows classifier to directly learn more lesion features than exploring on its own image features. The fundus image dataset labeled by experts, may lead to conflict among the diagnostics while labeling the disease. So, the ratio of total contour area and total true arc length was used to grade DR over segmented images. For training and testing the classifier performance, KAGGLE fundus image dataset was used. Practically, the images that are supplied to deep neural network for grading DR severity cannot precisely predict the grades without proper preprocessing. The DNN has no problem in identifying the healthy eye images, because quite large healthy eye images are available in the dataset. To classify the images of extreme cases, the need for extensive training required is substantially low. Our proposed segmentation method using VGG-19 architecture effectively identifies abnormality with accuracy of 96% on 5000 validation images. VGG-19 efficiently distinguishes various DR levels labeled based on contour area to true contour arc length ratio. Future research work will be focused to detect neovascularization abnormality using concept of transfer learning.