Bayesian Compressive Sensing
Sensing (BCS) is a Bayesian framework for solving the inverse problem of compressive sensing (CS). The basic BCS algorithm
adopts the relevance vector machine (RVM) [Tipping & Faul,
2003], and later it is extended by marginalizing the noise
variance (see the multi-task CS paper below) with improved robustness.
Besides providing a Bayesian solution, the Bayesian analysis of CS, more
importantly, provides a new framework that allows one to address a variety of
issues that previously have not been addressed. These issues include: (i) a stopping criterion for determining when a sufficient
number of CS measurements have been performed, (ii) adaptive design of the
projection matrix, and (iii) simultaneous inverse of multiple related CS
measurements (i.e., multi-task CS or simultaneous sparse approximation).
In our most recent
work, rather than assuming independence between coefficients as in the basic
BCS, the tree-structured Bayesian compressive sensing (TS-BCS) exploits the
statistical structure of the coefficients to reduce the number of CS
measurements. Specifically, under the wavelet basis, if a parent node in a
wavelet tree is zero or close to zero, with a very large probability its
children nodes are also zero or close to zero. This
tree structure can be readily extended to the block-DCT coefficients, so that
the inferred coefficients are directly compatible with JPEG compression. The
TS-BCS algorithms for wavelet and for block-DCT are implemented via a
hierarchical Bayesian framework, with the tree structure incorporated
naturally in the prior setting. Both MCMC-based inference and VB-based
inference are implemented. The MCMC approach achieves a
good reconstruction accuracy, while the VB approach requires much less
computation time with a relatively small cost of reconstruction quality.
Manifold based CS
theory has shown that if a signal lives in a low-dimensional manifold, then
the signal can be reconstructed using only a few compressed measurements.
However, till now there is no practical algorithm to implement CS on
manifolds. Our recent work fills the gap by employing a nonparametric mixture
of factor analyzers (MFA) to learn the manifold using training data, and then
analytically reconstructing testing signals with compressed measurements. We
also give bounds of the required number of measurements based on the concept
of block-sparsity. The proposed methodology is validated
on several synthetic and real datasets.
This page contains
information about BCS, TS-BCS, MFA, research papers, and a code distribution
that can be used for academic and/or research purposes.
Bayesian Compressive Sensing
Shihao Ji, Ya Xue, and Lawrence Carin, IEEE Trans. Signal Processing, vol.
56, no. 6, pp. 2346-2356, June 2008.
Multi-Task Compressive Sensing
Shihao Ji, David Dunson, and Lawrence Carin, IEEE Trans. Signal Processing, vol. 57, no. 1, pp.
92-106, Jan. 2009.
Multi-Task Compressive Sensing with Dirichlet
Yuting Qi, Dehong Liu,
David Dunson, and Lawrence Carin, in Proc. IEEE
Int. Conf. Machine Learning (ICML), 2008.
Exploiting Structure in Wavelet-Based Bayesian Compressive
Lihan He and Lawrence Carin, IEEE Trans. Signal
Processing, vol. 57, no. 9, pp. 3488-3497, Sept. 2009.
Tree-Structured Compressive Sensing with Variational Bayesian Analysis
Lihan He, Haojun Chen,
and Lawrence Carin, IEEE Signal Processing Letters, vol. 17, no. 3, pp.
Non-parametric Bayesian Dictionary Learning for Sparse
Mingyuan Zhou, Haojun
Chen, John Paisley, Lu Ren, Guillermo Sapiro, and Lawrence Carin, in Proc. Neural and
Information Processing Systems (NIPS), 2009.
Compressive Sensing on Manifolds Using a Nonparametric
Mixture of Factor Analyzers: Algorithm and Performance Bounds
Minhua Chen, Jorge Silva, John Paisley, Chunping Wang, David Dunson,
and Lawrence Carin, IEEE Trans. Signal Processing, pp. 6140-6155, Dec
Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images
Mingyuan Zhou, Haojun
Chen, John Paisley, Lu Ren, Lingbo
Li, Zhengming Xing, David Dunson,
Guillermo Sapiro, and Lawrence Carin, to appear in IEEE
Trans. Image Processing, 2011.
Principal Component Analysis
Xinghao Ding, Lihan He,
and Lawrence Carin, to appear in IEEE Trans. Image Processing, 2011.
Bayesian Inference and Optimal Design in the Sparse Linear
Finding Needles in Noisy Haystacks
Fast Bayesian Matching Pursuit
Model-Based Compressive Sensing
Duke Hosted CS Workshop
AFRL-Duke Workshop on Compressive Sensing
This is a MatLab 7.0 implementation of BCS, VB-BCS (BCS implemented
via a variational Bayesian (VB) approach), TS-BCS for wavelet and for block-DCT implemented via both
MCMC approach and VB approach. These codes have been designed on a Windows
machine, but they should run on any Unix or Linux architecture with MatLab installed without any problems.
Distribution and use
of this code is subject to the following agreement:
This Program is provided by Duke University and the
as a service to the research community. It is provided without cost or
restrictions, except for the User's acknowledgement that the Program is
provided on an "As Is" basis and User understands that Duke
University and the authors make no express or implied warranty of any
kind. Duke University and the authors specifically disclaim any implied
warranty or merchantability or fitness for a particular purpose, and make no
representations or warranties that the Program will not infringe the
intellectual property rights of others. The User agrees to indemnify and hold
harmless Duke University and the authors from and against any and all
liability arising out of User's use of the Program.
- BCS: At the
moment, the distribution includes the core BCS code and the spike
examples for the adaptive CS and the multi-task CS. Read the README file
in the main directory for more information.
[1.04MB] (Last updated: Aug. 03, 2008)
(NB: A bug was fixed in MT_CS.m for the cases where
signals are dramatic undersampled.)
- VB-BCS: The
basic BCS implemented via a variational
Bayesian approach. The package includes the core VB-BCS code, one
example of a 1-dimensional signal and two examples of 2-dimensional
[90KB] (Last updated: Mar. 03, 2009)
- TS-BCS for
wavelet via MCMC: The TS-BCS for wavelet implemented by an MCMC
approach. The package includes the core TS-BCS code for wavelet
coefficients with MCMC inference, one example of a 1-dimensional signal
and two examples of 2-dimensional images.
[196KB] (Last updated: Mar. 10, 2009, allowing other wavelets besides 'db1' (Haar).)
- TS-BCS for block-DCT
via MCMC: The TS-BCS for block-DCT implemented by an MCMC approach. The
package includes the core TS-BCS code for block-DCT coefficients with
MCMC inference, and two examples of 2-dimensional images.
[94KB] (Last updated: Mar. 04, 2009)
- TS-BCS via VB:
The TS-BCS for both wavelet and block-DCT implemented by a VB approach.
The package includes the core TS-BCS codes for wavelet coefficients and
for block-DCT coefficients, respectively, with VB inference, and two
examples of 2-dimensional images.
[100KB] (Last updated: Aug. 04, 2009)
- BPFA image denoising and inpainting:
The package includes the inference update equations and Matlab codes for image denoising
and inpainting via the non-parametric Bayesian
dictionary learning approach.
[1.74MB] (Last updated: Oct. 30, 2009)
- MFA-CS: This is an
implementation of the nonparametric mixture of factor analyzers for
manifold-based CS, as described in the paper "Compressive sensing
on manifolds using a nonparametric mixture of factor analyzers:
algorithm and performance bounds". The code includes a manifold
learning algorithm as well as an analytic CS reconstruction procedure.
[110MB] (Last updated: Nov. 12, 2009)
robust PCA: The package includes the Matlab
codes for Bayesian robust PCA, as described in the paper "Bayesian
robust principal component analysis" listed above. Demos for toy
examples and video examples are provided.
[3.62MB] (Last updated: Aug. 13, 2010)
Many other CS code
can be found at Compressive Sensing Resources.
contact the corresponding authors for questions/suggestions.