Regularized Deep Networks For Inverse Problems In Image Processing And Vision

Recent computational advances such as those afforded by widespread usage of Graphic Computation Units (GPUs) and cloud computing has brought about an avalanche of learning methods that design deep neural networks. Deep learning methods have become popular because of their representation and modeling ability and subsequently demonstrated success in multiple vision tasks where unprecedented performance gains have been reported. Based on the literature, the performance gains are often obtained by carefully adding (designing) more layers to classical neural networks, i.e. going deeper (hence the name deep learning). The additional layers can deliver improvements invariably by exploiting an abundance of training data like Image Net and public video databases such as YouTube. Traditionally, analytical methods have been used to solve inverse problems in image processing and computer vision, such as image restoration, inpainting, and super-resolution (SR). Unlike analytical methods for which the problem is explicitly defined and domain-knowledge carefully engineered into the solution, in most existing literature, deep neural networks do not benefit from such prior knowledge and instead make use of large datasets to learn the unknown solution to the inverse problems. There exist however compelling practical scenarios that are constraining such practice: such as memory constraints in mobile devices, limited training data, and requirement of real-time operation, etc. These limitations dictate that designing a network such as a deep Convolution Neural Network (CNN) with a large number of parameters is not always desirable or even feasible. With these practical constraints in mind, this dissertation mainly focuses on developing deep learning-based methods that are more computationally efficient, less parameterized, require less training data, and having a smaller memory footprint. By marrying classical signal processing tools with the deep learning methods, multiple newly designed Regularized Deep Networks (RDNs) are proposed. These RDNs take advantage of the robustness and efficiency brought by exploiting a signal model or the knowledge of signal/image structure, while simultaneously retaining the generality brought by the deep learning methods. The design processes of these RDNs usually consist of incorporating domain knowledge and designing customized regularization. This dissertation focuses on designing RDNs for inverse problems in image processing and vision. A key representative image processing inverse/regression problem is Single Image Super-Resolution (SISR), solutions to which via RDNs are discussed in Chapter II. The proposed RDNs include transform domain processing, its integration into a deep network and constrained optimization of transform basis. Another key inverse problem in this domain is Single Image Dehazing (SIDH) which is addressed using two different RDNs detailed in Chapters III and IV. From an analytical standpoint, this involves both the development of novel network architectures as well as the…