GSoC week 2: developing a model for semantic segmentation

 This week I was developing models for medical images segmentation in VR.


Data

I was working with two datasets: binary semantic segmentation dataset and multi class semantic segmentation dataset.

The binary segmentation was done using Polyp Segmentation in Colonoscopy data. This dataset contains 300 images with binary masks. 

The multi class segmentation was done using Cholec8K dataset. This dataset contains 8080 laparoscopic cholecystectomy image frames  from 17 videos from Cholec80 dataset. 


Model and training

For both segmentation tasks I was using the DeepLabV3Plus model with ResNet50 encoder, pretrained on imaginet data set. 

For polyp segmentation I was using 80% of data for training and 20% for validation-testing. In case of Cholec8k I decided to make the train-test-split based on videos. 12 videos were taken for training and 5 for testing.

For training I was using the open source Catalyst framework and SMP. It reduced the time required for building the training pipeline.

Polyp segmentation

Results obtained by the original model


The original model performed very well. After 25 epoch of training with Dice, IoU and BCE losses, it has achieved the IoU of 0.97. It is a fantastic result!

Then I have added the distortion to the mask and image pre-processing.



Images after distortion



Model performance slightly degraded after applying distortion. It has dropped from 0.97 IoU down to 0.93.  Such a small change could be related to the simplicity of task. I assumed that for multi class segmentation task the relative change in performance will be greater.

P.S my experiment can be found here 

Cholec8k segmentation


Multi class segmentation is a more complicated problem. First of all, I had to transfer a 3-channel mask with class ids to separate masks. An easy solution is to simply use the to_categorical function, however, the class ids in the docs and ids on images did not match. That is why I had to reindex classes first.

Experiment has shown that after few epoch the original model can achieve F-score of 0.8 and IoU of 0.7. I believe that configuring the hyper parameters it is possible to achieve even higher results, but since the aim of this experiment is to check whether it is possible to adapt domains for VR, I think the shown result is ok.

The I had to add distortion to images. However, it was important to apply it AFTER splitting the image on classes, because adding distortion could change pixel values a bit. 

Resulting model has shown 7-10% degradation in IoU and F scores, however it simply was learning slower. I believe that given more time, the model could make the difference less.


Here is my experiment


Further work


My further work is to validate models again and try tuning them to improve performance

Comments

Post a Comment

Popular posts from this blog

Summing up my GSoC experience

Image
 Introduction Once again, I am Daniil Orel from Kazakhstan (Nur-Sultan / Astana city). This summer I was working with LibreHealth organisation for an amazing GSoC project. Few words about my host organisation LibreHealth is a  collaborative community, which supports free open source software projects in HealthTech. This organisation has multiple projects, but I was working towards the radiology project. My project Several medical procedures in surgery or interventional radiology are recorded as videos used for review, training, and quality monitoring. These videos have at least three interesting artifacts - anatomical structures such as organs, tumors, tissues, etc.; medical equipment, and medical information overlaid that describes the patient. It will be immensely helpful for review and search purposes if these can be identified and automatically labeled in the videos. My task was to develop ML models which can process videos and that are suitable for VR environment. As a result, I h
Read more

How can we "simulate" VR?

     Having the proper data is important when it comes to developing an ML/DL application. In some cases, the amount of data is limited due to some reasons. For example, if we are trying to detect a rare disease, collecting 1KK samples could be simply impossible. When collecting a proper dataset is too expensive, we cannot afford it as well. What shall we do in such a case?     The answer is simple: we have to generate data. Generative Adversarial Networks  are said to be one of the ways to synthetically generate the data. However, it requires at least some data from the domain of interest to generate new data. In this case augmentations could help.  Data augmentation is the artificial creation of training data for machine learning by transformations (c) It means, that we can change the original data to increase the training sample and / or apply some transformations to another dataset to get something in the domain of our interest. The second technique can be used, for example, to add
Read more

Surgical tools detection

Image
 This week I was training the Surgical Tools Detection model. The dataset was taken from here . To train the model and prepare it for inference I was using Roboflow. Roboflow  is a robust web-platform for collecting datasets, training models and running them into production. Training I have split the data on 69/21/10 for train, val and test sets. Graphs with training metrics is shown below Train / val metrics To be more precise, on test set the model has 93.8% precision with 90% recall and 96.4% mAP.  It is considered as a high performance and means that the model can be used in production. Inference Roboflow itself provides multiple ways for inference. I could use web-API method (requests are sent to the server and the model sends back predictions), but to make it more flexible, I prefer python-package method. It can be easily imported to the main project of LibreHealth and run.
Read more