Paper Review: COVID-SEGNET

Diagnosis of Covid-19 Cases on Radiological Images using Mask R-CNN by A. Kundu, C. Mishra and S. Bilgaiyan.

I recently went in search for materials on the instance segmentation of CT scans for the detection of COVID-19 for a recent project I am undertaking and surprisingly met with a dearth of materials on this particular problem.

This is quite surprising, considering the popularity and importance of the subject matter, especially within data science communities. While there were lots of materials on machine learning approaches in classifying chest scans as COVID-19 affected or normal, this paper was the only one I found ( as at the time of writing) which was interested in the instance segmentation of objects that are particular to the affected patients.

Indeed the authors noted that

Most of the research works related to COVID-19 virus, were based on the classification of COVID-19 and normal patients. Till now, no study has been conducted regarding instance segmentation for detection of ground glass opacities, which forms a visible unique pattern on the CT images of the COVID-19 infected patients.

This article is my review of a unique work done by the authors in implementing instance segmentation of ground glass opacities (GGO) for detecting COVID-19 in CT images using Mask-RCNN which achieved an accuracy of 98.25%.

Instance Segmentation task

An instance segmentation task generally involves producing the pixel-wise masks for individual objects of interest in an image. The outputs of an instance segmentation model include the class label, probability score, bounding box coordinates and actual mask for individual objects of interest in an image. It is similar to object detection but also includes the prediction of actual masks for objects.

Comparison of common computer vision problems. Credits: Deep Learning For Computer Vision by Adrian Rosebrock

The Mask R-CNN Framework

Mask R-CNN is an efficient framework for the detection and instance segmentation of objects of interest in images. It is a member of the R-CNN family which adds a mask branch to Faster R-CNN. It achieves instance segmentation by feeding the outputs of successive Feature Pyramid Network (FPN), Regional Proposal Network (RPN) and Region of Interest (ROI) Align layers to mask-predicting convolutional layers to obtain a total output of the class label probabilities, bounding boxes and actual pixel-wise masks for each detected object in an image which are all parallelly executed.

Layers and Outputs of the Mask RCNN Framework. Credits: Deep Learning For Computer Vision by Adrian Rosebrock

Check out the Mask R-CNN paper to get more details on how the framework achieves instance segmentation by implementing these layers.

Aim

The authors aimed to build an accurate detector capable of automating the classification and instance segmentation of images as COVID-19 affected or normal. This could prove useful in aiding health professionals fasten the screening and validation of the initial assessment towards COVID-19 patients.

Approach

In this study, the authors have proposed a Mask R-CNN-based detector named COVID-SEGNET to achieve this aim.

Their approach was to detect ground glass opacities (GGOs) in chest CT images which are indicative of positive COVID-19 infections by utilizing the Mask R-CNN framework to train an accurate detector that is capable of identifying these objects (GGOs) when they are present in images.

Sample CT images of lungs pointing at the GGOs. Credits: COVID-SEGNET: Diagnosis of Covid-19 Cases on Radiological Images using Mask R-CNN.

The dataset used contains 1110 annotated binary pixel masks of chest CT scans affected by COVID-19.

Workflow

1. Conversion of the dataset from .nii files to .png images and .json files which stores the object coordinates and image data. These are the required input formats to train Mask R-CNN detectors.
2. Initialization of the Mask R-CNN framework with a ResNet-101 feature pyramid network backbone for the model.
3. Training the model for 30 epochs with a learning rate of 0.001, batch size of 2 and learning momentum of 0.9 on the images that were rescaled to 640x480.

Output generated from each module of Mask R-CNN. Credits: COVID-SEGNET: Diagnosis of Covid-19 Cases on Radiological Images using Mask R-CNN.

4. Loading the best weights saved during training and evaluating the model using Dice coefficient. The model achieved a Sørensen–Dice index score of 0.764 and 98.25% accuracy on the validation images.

Overall Loss of the model. Credits: COVID-SEGNET: Diagnosis of Covid-19 Cases on Radiological Images using Mask R-CNN

Results

The model was well trained and is able to identify and segment GGOs when present CT scans. This is proven by the impressive evaluation metrics and outputs on validation images.

Prediction of the GGOs found on CT of COVID-19 patient. COVID-SEGNET: Diagnosis of Covid-19 Cases on Radiological Images using Mask R-CNN.

Conclusion.

The authors did a great work in many respects to implement instance segmentation of objects (GGOs) that are indicative of COVID-19 infection on chest CT scans. I personally found the work helpful as there weren’t much materials on similar projects. The final results were even more impressive with metrics and outputs that shows that the model can clearly identify GGOs in chest scans. The author suggested that training the model on more datasets, initializing the model with weights of a model that is trained on chest CT images and using ResNet50 as the backbone of the Mask R-CNN could improve the efficiency of the model and limit overfitting.

Do well to take a read of the original paper.