Learning Multiple Conditional Random Fields for Semantic Segmentation with Convolutional Neural Network

Abstract

Image semantic segmentation is a task that assigns pixel-level classification in an image. Compared to object classification, much more information are needed to make detailed prediction. Many semantic segmentation algorithms have devised a way to store a lot of information, and as a result, algorithms require a lot of memory. Meanwhile, the ensemble between several branches in a feature map lead to improved results through complementary relationships and Conditional random field gives detailed prediction without requirement of huge memory. In this paper, we exploit ‘intra ensemble’ of complementary information from several branches in a feature map of DeepLab-ASPP which gets branches having complementary relationship with each other. Second, we introduce the CRF layer of CRF-RNN which combines feature map and CRF into end-to-end training to improve performance of each branch. The inference is implemented to exploit complementary and detailed branches. We demonstrated our method is better than previous work by getting higher score than our baseline in testing on Pascal VOC 2012 benchmark dataset and showing the great qualitative results.