Bayesian Compressive Sensing

  

Introduction

Bayesian Compressive Sensing (BCS) is a Bayesian framework for solving the inverse problem of compressive sensing (CS). The basic BCS algorithm adopts the relevance vector machine (RVM) [Tipping & Faul, 2003], and later it is extended by marginalizing the noise variance (see the multi-task CS paper below) with improved robustness. Besides providing a Bayesian solution, the Bayesian analysis of CS, more importantly, provides a new framework that allows one to address a variety of issues that previously have not been addressed. These issues include: (i) a stopping criterion for determining when a sufficient number of CS measurements have been performed, (ii) adaptive design of the projection matrix, and (iii) simultaneous inverse of multiple related CS measurements (i.e., multi-task CS or simultaneous sparse approximation).

In our most recent work, rather than assuming independence between coefficients as in the basic BCS, the tree-structured Bayesian compressive sensing (TS-BCS) exploits the statistical structure of the coefficients to reduce the number of CS measurements. Specifically, under the wavelet basis, if a parent node in a wavelet tree is zero or close to zero, with a very large probability its children nodes are also zero or close to zero. This tree structure can be readily extended to the block-DCT coefficients, so that the inferred coefficients are directly compatible with JPEG compression. The TS-BCS algorithms for wavelet and for block-DCT are implemented via a hierarchical Bayesian framework, with the tree structure incorporated naturally in the prior setting. Both MCMC-based inference and VB-based inference are implemented. The MCMC approach achieves a good reconstruction accuracy, while the VB approach requires much less computation time with a relatively small cost of reconstruction quality.

Manifold based CS theory has shown that if a signal lives in a low-dimensional manifold, then the signal can be reconstructed using only a few compressed measurements. However, till now there is no practical algorithm to implement CS on manifolds. Our recent work fills the gap by employing a nonparametric mixture of factor analyzers (MFA) to learn the manifold using training data, and then analytically reconstructing testing signals with compressed measurements. We also give bounds of the required number of measurements based on the concept of block-sparsity. The proposed methodology is validated on several synthetic and real datasets.

This page contains information about BCS, TS-BCS, MFA, research papers, and a code distribution that can be used for academic and/or research purposes.


Papers

Bayesian Compressive Sensing
Shihao Ji, Ya Xue, and Lawrence Carin, IEEE Trans. Signal Processing, vol. 56, no. 6, pp. 2346-2356, June 2008.

 

Multi-Task Compressive Sensing
Shihao Ji, David Dunson, and Lawrence Carin, IEEE Trans. Signal Processing, vol. 57, no. 1, pp. 92-106, Jan. 2009.

 

Multi-Task Compressive Sensing with Dirichlet Process Priors
Yuting Qi, Dehong Liu, David Dunson, and Lawrence Carin, in Proc. IEEE Int. Conf. Machine Learning (ICML), 2008.

 

Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing
Lihan He and Lawrence Carin, IEEE Trans. Signal Processing, vol. 57, no. 9, pp. 3488-3497, Sept. 2009.

Tree-Structured Compressive Sensing with Variational Bayesian Analysis
Lihan He, Haojun Chen, and Lawrence Carin, IEEE Signal Processing Letters, vol. 17, no. 3, pp. 233-236, 2010.

Non-parametric Bayesian Dictionary Learning for Sparse Image Representations
Mingyuan Zhou, Haojun Chen, John Paisley, Lu Ren, Guillermo Sapiro, and Lawrence Carin, in Proc. Neural and Information Processing Systems (NIPS), 2009.

Compressive Sensing on Manifolds Using a Nonparametric Mixture of Factor Analyzers: Algorithm and Performance Bounds
Minhua Chen, Jorge Silva, John Paisley, Chunping Wang, David Dunson, and Lawrence Carin, IEEE Trans. Signal Processing, pp. 6140-6155, Dec 2010.

Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images
Mingyuan Zhou, Haojun Chen, John Paisley, Lu Ren, Lingbo Li, Zhengming Xing, David Dunson, Guillermo Sapiro, and Lawrence Carin, to appear in IEEE Trans. Image Processing, 2011.

Bayesian Robust Principal Component Analysis
Xinghao Ding, Lihan He, and Lawrence Carin, to appear in IEEE Trans. Image Processing, 2011.


Related Work

Bayesian Inference and Optimal Design in the Sparse Linear Model
Finding Needles in Noisy Haystacks
Fast Bayesian Matching Pursuit
Model-Based Compressive Sensing


Duke Hosted CS Workshop

AFRL-Duke Workshop on Compressive Sensing


Code Distribution

This is a MatLab 7.0 implementation of BCS, VB-BCS (BCS implemented via a variational Bayesian (VB) approach), TS-BCS for wavelet and for block-DCT implemented via both MCMC approach and VB approach. These codes have been designed on a Windows machine, but they should run on any Unix or Linux architecture with MatLab installed without any problems.

Distribution and use of this code is subject to the following agreement:

This Program is provided by Duke University and the authors as a service to the research community. It is provided without cost or restrictions, except for the User's acknowledgement that the Program is provided on an "As Is" basis and User understands that Duke University and the authors make no express or implied warranty of any kind.  Duke University and the authors specifically disclaim any implied warranty or merchantability or fitness for a particular purpose, and make no representations or warranties that the Program will not infringe the intellectual property rights of others. The User agrees to indemnify and hold harmless Duke University and the authors from and against any and all liability arising out of User's use of the Program.

  • BCS: At the moment, the distribution includes the core BCS code and the spike examples for the adaptive CS and the multi-task CS. Read the README file in the main directory for more information.

Download: bcs_ver0.1.zip [1.04MB] (Last updated: Aug. 03, 2008)
(NB: A bug was fixed in MT_CS.m for the cases where signals are dramatic undersampled.)

  • VB-BCS: The basic BCS implemented via a variational Bayesian approach. The package includes the core VB-BCS code, one example of a 1-dimensional signal and two examples of 2-dimensional images.

Download: bcs_vb.zip [90KB] (Last updated: Mar. 03, 2009)
 

  • TS-BCS for wavelet via MCMC: The TS-BCS for wavelet implemented by an MCMC approach. The package includes the core TS-BCS code for wavelet coefficients with MCMC inference, one example of a 1-dimensional signal and two examples of 2-dimensional images.

Download: tswcs.zip [196KB] (Last updated: Mar. 10, 2009, allowing other wavelets besides 'db1' (Haar).)
 

  • TS-BCS for block-DCT via MCMC: The TS-BCS for block-DCT implemented by an MCMC approach. The package includes the core TS-BCS code for block-DCT coefficients with MCMC inference, and two examples of 2-dimensional images.

Download: tsdctcs.zip [94KB] (Last updated: Mar. 04, 2009)
 

  • TS-BCS via VB: The TS-BCS for both wavelet and block-DCT implemented by a VB approach. The package includes the core TS-BCS codes for wavelet coefficients and for block-DCT coefficients, respectively, with VB inference, and two examples of 2-dimensional images.

Download: tsbcs_vb.zip [100KB] (Last updated: Aug. 04, 2009)

  • BPFA image denoising and inpainting: The package includes the inference update equations and Matlab codes for image denoising and inpainting via the non-parametric Bayesian dictionary learning approach.

    Download: BPFA.zip [1.74MB] (Last updated: Oct. 30, 2009)
     
  • MFA-CS: This is an implementation of the nonparametric mixture of factor analyzers for manifold-based CS, as described in the paper "Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: algorithm and performance bounds". The code includes a manifold learning algorithm as well as an analytic CS reconstruction procedure.

Download: MFA-CS.zip [110MB] (Last updated: Nov. 12, 2009)

  • Bayesian robust PCA: The package includes the Matlab codes for Bayesian robust PCA, as described in the paper "Bayesian robust principal component analysis" listed above. Demos for toy examples and video examples are provided.

    Download: BRPCA.zip [3.62MB] (Last updated: Aug. 13, 2010)
     

Many other CS code can be found at Compressive Sensing Resources.


Feedback

Please contact the corresponding authors for questions/suggestions.