Sparse Representations for Semantic Segmentation and Deblurring

Abstract

Sparse representation is an approximation of an input signal (e.g., audio, image, video, ...) with a small number of coefficients using a set of overcomplete elementary functions, i.e., dictionary. This kind of approximation has numerous applications in computer vision and image processing such as compressed sensing, denoising, super-resolution, etc. In this dissertation, sparse representation based semantic segmentation and deblurring algorithms are investigated.

First, we present a simple and effective approach to the labeling image regions (or semantic segmentation) problem. Inspired by coupled dictionary concept which is often used to convert an image in a source style into a target style, we convert the image parsing problem into a superpixel-wise sparse representation problem with coupled dictionaries related to features and likelihoods. This algorithm works by image-level classification with global image descriptors, followed by sparse representation based likelihood estimation with local features. Finally, Markov random field (MRF) optimization is applied to incorporate neighborhood context. Experimental results on the SIFTflow dataset and Camvid dataset support the use of our approach for solving the task of image parsing. The advantage of the proposed algorithm is that it can estimate likelihoods from a small set of bases (dictionary) whereas recent nonparametric scene parsing algorithms need features and labels of whole datasets to compute likelihoods. To our knowledge, this is the first approach that utilizes sparse representation to superpixel-based image parsing.

Second, we present a novel blind-deblurring framework with an application to noisy-blurry image deblurring. Under low-light conditions, traditional deblurring approaches based on sparsity priors often fail due to the presence of noise - even if a small amount of noise exists. We introduce image and blur kernel priors which help to obtain proper deblurred image and blur kernel. These priors are especially effective when we want to suppress the effect of noise in deblurring process. Specifically, we have pairs of dictionaries in which sharp ones are obtained from traditional sparse coding dictionary learning schemes and the others are the blurred version of sharp ones. We generate priors with these coupled dictionaries. The proposed priors are constructed with a convolutional sparse coding (CSC) algorithm which is suitable for considering the correlation between local neighborhood and convolution operator of blur kernel rather than traditional patch-wise sparse coding (SC) algorithms. Our method extends the commonly used image deblurring framework with the proposed coupled dictionary priors to be incorporated in the optimization process. With the proposed priors, we can also reduce the artifact that come from naive deconvolution process. Experimental results on synthetic and real images support the use of our approach compared with previous approaches.