Bayesian Compressive Sensing
Introduction
Bayesian
Compressive Sensing (BCS) is a Bayesian framework for solving the inverse
problem of compressive
sensing (CS). The basic BCS algorithm adopts the relevance vector
machine (RVM) [Tipping & Faul, 2003], and later
it is extended by marginalizing the noise variance (see the multi-task CS
paper below) with improved robustness. Besides providing a Bayesian solution,
the Bayesian analysis of CS, more importantly, provides a new framework that
allows one to address a variety of issues that previously have not been
addressed. These issues include: (i) a stopping
criterion for determining when a sufficient number of CS measurements have
been performed, (ii) adaptive design of the projection matrix, and (iii)
simultaneous inverse of multiple related CS measurements (i.e., multi-task CS
or simultaneous sparse approximation).
In our
most recent work, rather than assuming independence between coefficients as
in the basic BCS, the tree-structured Bayesian compressive sensing (TS-BCS)
exploits the statistical structure of the coefficients to reduce the number
of CS measurements. Specifically, under the wavelet basis, if a parent node
in a wavelet tree is zero or close to zero, with a very large probability its
children nodes are also zero or close to zero. This tree structure can be
readily extended to the block-DCT coefficients, so that the inferred
coefficients are directly compatible with JPEG compression. The TS-BCS
algorithms for wavelet and for block-DCT are implemented via a hierarchical
Bayesian framework, with the tree structure incorporated naturally in the
prior setting. Both MCMC-based inference and VB-based inference are
implemented. The MCMC approach achieves a good
reconstruction accuracy, while the VB approach requires much less computation
time with a relatively small cost of reconstruction quality.
Manifold
based CS theory has shown that if a signal lives in a low-dimensional
manifold, then the signal can be reconstructed using only a few compressed
measurements. However, till now there is no practical algorithm to implement
CS on manifolds. Our recent work fills the gap by employing a nonparametric
mixture of factor analyzers (MFA) to learn the manifold using training data,
and then analytically reconstructing testing signals with compressed
measurements. We also give bounds of the required number of measurements
based on the concept of block-sparsity. The proposed
methodology is validated on several synthetic and real datasets.
This
page contains information about BCS, TS-BCS, MFA, research papers, and a code
distribution that can be used for academic and/or research purposes.
Papers
Bayesian
Compressive Sensing
Shihao Ji, Ya Xue, and
Lawrence Carin, IEEE Trans. Signal Processing, vol. 56, no. 6,
pp. 2346-2356, June 2008.
Multi-Task
Compressive Sensing
Shihao Ji, David Dunson,
and Lawrence Carin, IEEE
Trans. Signal Processing, vol. 57, no. 1, pp. 92-106, Jan.
2009.
Multi-Task
Compressive Sensing with Dirichlet Process Priors
Yuting Qi, Dehong Liu,
David Dunson, and Lawrence Carin, in Proc. IEEE Int. Conf. Machine
Learning (ICML), 2008.
Exploiting
Structure in Wavelet-Based Bayesian Compressive Sensing
Lihan He and Lawrence Carin, IEEE Trans. Signal
Processing, vol. 57, no. 9, pp. 3488-3497, Sept. 2009.
Tree-Structured
Compressive Sensing with Variational Bayesian
Analysis
Lihan He, Haojun Chen,
and Lawrence Carin, IEEE Signal Processing Letters, vol. 17, no. 3,
pp. 233-236, 2010.
Non-parametric Bayesian Dictionary Learning for Sparse
Image Representations
Mingyuan Zhou, Haojun Chen, John Paisley, Lu Ren,
Guillermo Sapiro, and Lawrence Carin, in Proc. Neural and Information
Processing Systems (NIPS), 2009.
Compressive
Sensing on Manifolds Using a Nonparametric Mixture of Factor Analyzers:
Algorithm and Performance Bounds
Minhua Chen, Jorge Silva, John Paisley, Chunping Wang, David Dunson, and Lawrence Carin, IEEE
Trans. Signal Processing, pp. 6140-6155, Dec 2010.
Nonparametric
Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images
Mingyuan Zhou, Haojun Chen, John Paisley, Lu Ren,
Lingbo Li, Zhengming Xing, David Dunson, Guillermo Sapiro, and Lawrence
Carin, IEEE Trans. Image Processing, 2011.
Bayesian Robust
Principal Component Analysis
Xinghao Ding, Lihan He,
and Lawrence Carin, IEEE Trans. Image Processing, 2011.
Hierarchical
Infinite Divisibility for Multiscale Shrinkage
X. Yuan, V. Rao, S. Han and L. Carin, IEEE Trans. Image Processing,
2014.
Video
Compressive Sensing Using Gaussian Mixture Models
Jianbo
Yang, Xin Yuan, Xuejun Liao, Patrick Llull, David
J. Brady, Guillermo Sapiro and Lawrence Carin, IEEE Trans. Image
Processing, 2014
Related
Work
Bayesian Inference and Optimal Design in the Sparse Linear
Model
Finding Needles in Noisy Haystacks
Fast Bayesian Matching Pursuit
Model-Based Compressive Sensing
Duke
Hosted CS Workshop
AFRL-Duke Workshop on Compressive Sensing
Code
Distribution
This
is a MatLab 7.0 implementation of BCS, VB-BCS (BCS
implemented via a variational Bayesian (VB)
approach), TS-BCS for wavelet and for block-DCT implemented via both MCMC
approach and VB approach. These codes have been designed on a Windows
machine, but they should run on any Unix or Linux architecture with MatLab installed without any problems.
Distribution
and use of this code is subject to the following agreement:
This
Program is provided by Duke University and the authors as a service to the
research community. It is provided without cost or restrictions, except for
the User's acknowledgement that the Program is provided on an "As
Is" basis and User understands that Duke University and the authors make
no express or implied warranty of any kind. Duke University and the
authors specifically disclaim any implied warranty or merchantability or
fitness for a particular purpose, and make no representations or warranties
that the Program will not infringe the intellectual property rights of
others. The User agrees to indemnify and hold harmless Duke University and
the authors from and against any and all liability arising out of User's use
of the Program.
- BCS: At the
moment, the distribution includes the core BCS code and the spike
examples for the adaptive CS and the multi-task CS. Read the README file
in the main directory for more information.
Download:
bcs_ver0.1.zip
[1.04MB] (Last updated: Aug. 03, 2008)
(NB: A bug was fixed in MT_CS.m for the cases where
signals are dramatic undersampled.)
- VB-BCS: The
basic BCS implemented via a variational
Bayesian approach. The package includes the core VB-BCS code, one
example of a 1-dimensional signal and two examples of 2-dimensional
images.
Download:
bcs_vb.zip [90KB]
(Last updated: Mar. 03, 2009)
- TS-BCS for
wavelet via MCMC: The TS-BCS for wavelet implemented by an MCMC
approach. The package includes the core TS-BCS code for wavelet
coefficients with MCMC inference, one example of a 1-dimensional signal
and two examples of 2-dimensional images.
Download:
tswcs.zip
[196KB] (Last updated: Mar. 10, 2009, allowing other wavelets besides 'db1' (Haar).)
- TS-BCS for
block-DCT via MCMC: The TS-BCS for block-DCT implemented by an MCMC
approach. The package includes the core TS-BCS code for block-DCT
coefficients with MCMC inference, and two examples of 2-dimensional
images.
Download:
tsdctcs.zip
[94KB] (Last updated: Mar. 04, 2009)
- TS-BCS via VB:
The TS-BCS for both wavelet and block-DCT implemented by a VB approach.
The package includes the core TS-BCS codes for wavelet coefficients and
for block-DCT coefficients, respectively, with VB inference, and two
examples of 2-dimensional images.
Download:
tsbcs_vb.zip
[100KB] (Last updated: Aug. 04, 2009)
- BPFA image denoising and inpainting:
The package includes the inference update equations and Matlab codes for image denoising
and inpainting via the non-parametric Bayesian
dictionary learning approach.
Download: BPFA.zip
[1.74MB] (Last updated: Oct. 30, 2009)
- MFA-CS: This
is an implementation of the nonparametric mixture of factor analyzers
for manifold-based CS, as described in the paper "Compressive
sensing on manifolds using a nonparametric mixture of factor analyzers:
algorithm and performance bounds". The code includes a manifold
learning algorithm as well as an analytic CS reconstruction procedure.
Download:
MFA-CS.zip
[110MB] (Last updated: Nov. 12, 2009)
- Bayesian robust
PCA: The package includes the Matlab codes for
Bayesian robust PCA, as described in the paper "Bayesian robust
principal component analysis" listed above. Demos for toy examples
and video examples are provided.
Download: BRPCA.zip
[3.62MB] (Last updated: Aug. 13, 2010)
- Tree-Based
Shrinkage: This Matlab code employs
multi-scaled shrinkage, and is applicable to wavelet or DCT-based signal
expansions. Details of the method are discussed in X. Yuan, V. Rao, S. Han and L. Carin,
“Hierarchical Infinite Divisibility for Multiscale
Shrinkage,” IEEE Trans. Signal
Processing, 2014
Download: V9_shrinkage.zip
(Last updated: August 2014)
·
Gaussian Mixture Model (GMM) Based
Inversion: This Matlab code is for the video CS
inversion based on GMMs, as described in “Video Compressive Sensing Using
Gaussian Mixture Models,” by Jianbo Yang, Xin Yuan, Xuejun Liao, Patrick Llull, David J. Brady, Guillermo Sapiro and Lawrence
Carin, IEEE Trans. Image Processing, 2014
Download:
GMM_TP_dist_v1.0.zip
(Last updated: November 2014)
Many
other CS code can be found at Compressive Sensing Resources.
Feedback
Please
contact the corresponding authors for questions/suggestions.
|