Multi class semantic segmentation in VR

This week I was mostly improving my model for multi class semantic segmentation.


Dataset

For this task I was using the Cholec8k Segmentation dataset. This dataset is interesting, because it contains not only the segmentation of organs, but also the data needed for segmentation of surgical tools. 

The data in this dataset is split on videos. Totally there are 17 videos, so I decided to make the train-test split based on video ids. For this purpose, I was using 13 videos for training and 4 videos for validation and testing.


Model

Last week I was experimenting with different models and at the beginning of this week I was sure that DeepLavV3+ should be used. DeepLavV3+ uses a decoder to better segment boundaries of objects. It is an important part of medical images segmentation, since we want to know where the border between organs and surgical tools located. Actually, it is one more reason for me to use DeepLabV3+.


For training I was using 2 loss functions: CE and Dice Loss. CE loss is widely used for mutli class semantic segmentation. And, to be honest, the original model (with no distortion) performed well on the data. 
Example of segmentation results Original model CE

However, in case of distorted data the result is not that optimistic.  My models were struggling with segmentation of the picture's border (exactly what I predicted at the very beginning of the project).
"Distorted model"

To fight this issue I have just changed the loss function for training. I was using the Dice Loss function, which is closely correlated with IoU score (main metric in this work). The original model trained with this loss function performed even better than the CE model. If CE model had 0.65 IoU, the resulting model, trained with Dice Loss had IoU about 0.7
Original model, trained with Dice Loss

That is why I expected from the distorted model high performance. And it actually worked. The distorted model, trained with this loss function had IoU close to the one of original model. Also, since the videos do nit overlap, I can conclude that these results are not caused overfitting.



Distorted model, Dice Loss

Conclusion

From my work it can be concluded that VR can be simulated for training model with multi class segmentation tasks. Results are quite accurate and precise.

Comments

Post a Comment

Popular posts from this blog

Summing up my GSoC experience

Image
 Introduction Once again, I am Daniil Orel from Kazakhstan (Nur-Sultan / Astana city). This summer I was working with LibreHealth organisation for an amazing GSoC project. Few words about my host organisation LibreHealth is a  collaborative community, which supports free open source software projects in HealthTech. This organisation has multiple projects, but I was working towards the radiology project. My project Several medical procedures in surgery or interventional radiology are recorded as videos used for review, training, and quality monitoring. These videos have at least three interesting artifacts - anatomical structures such as organs, tumors, tissues, etc.; medical equipment, and medical information overlaid that describes the patient. It will be immensely helpful for review and search purposes if these can be identified and automatically labeled in the videos. My task was to develop ML models which can process videos and that are suitable for VR environment. As a result, I h
Read more

How can we "simulate" VR?

     Having the proper data is important when it comes to developing an ML/DL application. In some cases, the amount of data is limited due to some reasons. For example, if we are trying to detect a rare disease, collecting 1KK samples could be simply impossible. When collecting a proper dataset is too expensive, we cannot afford it as well. What shall we do in such a case?     The answer is simple: we have to generate data. Generative Adversarial Networks  are said to be one of the ways to synthetically generate the data. However, it requires at least some data from the domain of interest to generate new data. In this case augmentations could help.  Data augmentation is the artificial creation of training data for machine learning by transformations (c) It means, that we can change the original data to increase the training sample and / or apply some transformations to another dataset to get something in the domain of our interest. The second technique can be used, for example, to add
Read more

Surgical tools detection

Image
 This week I was training the Surgical Tools Detection model. The dataset was taken from here . To train the model and prepare it for inference I was using Roboflow. Roboflow  is a robust web-platform for collecting datasets, training models and running them into production. Training I have split the data on 69/21/10 for train, val and test sets. Graphs with training metrics is shown below Train / val metrics To be more precise, on test set the model has 93.8% precision with 90% recall and 96.4% mAP.  It is considered as a high performance and means that the model can be used in production. Inference Roboflow itself provides multiple ways for inference. I could use web-API method (requests are sent to the server and the model sends back predictions), but to make it more flexible, I prefer python-package method. It can be easily imported to the main project of LibreHealth and run.
Read more