Statistical independence is one of the most desirable properties of
a coordinate system for representing and modeling images.
In reality, however, truly independent coordinates may not exist for
a given set of images, or it may be too difficult to compute them in practice.
Therefore, it makes sense to obtain the least statistically-dependent
coordinate system efficiently.
To achieve this goal, we use the best-basis algorithm with new criterion that
can rapidly select the least statistically-dependent basis (LSDB) from
a basis dictionary (e.g., the local cosine or wavelet packet
dictionaries) containing a huge number of orthonormal (or biorthogonal)
bases. Our new basis selection criterion is minimization of the mutual
information of the distributions of the basis coefficients as a measure of
statistical dependence, which in turn is equivalent to minimization of
the sum of the differential entropy of each coordinate in the
basis dictionary. We show that this criterion, combined with the
best-basis algorithm, can find the coordinates closest to the statistical
independence from all possible bases searchable in a basis dictionary with
O(n [log n]^p), where n is the dimensionality of the image
(the number of pixels in each image), and p=1 for the wavelet packet
dictionaries, and p=2 for the local cosine/sine dictionaries.
In this sense, we can view this LSDB algorithm as the best-basis
version of the Independent Component Analysis (ICA), which is increasingly
gaining popularity.
This criterion is different from that of the Joint Best Basis (JBB) proposed
by Wickerhauser, which can be viewed as the best-basis version of
the Karhunen-Loeve basis (KLB).
We demonstrate the application of the LSDB to image approximation and modeling
and compare its performance with that of KLB and JBB using a collection of real
geophysical acoustic waveforms and an image database of human faces.
For these datasets, the LSDB provides the best approximation in terms of
the average relative l^2 errors among various bases including the KLB,
JBB, DCT, and wavelet basis.
For image modeling, we propose two simple stochastic models for a given class
of signals or images based on the LSDB coordinates.
The first model is to assume the statistical independence among the LSDB
coordinates, which allows us to sample typical coefficients of each coordinate
separately using the empirical distribution estimated from the available
training coefficients of that coordinate, which in turn easily allows us to
simulate new images at our disposal. For the geophysical acoustic waveforms,
this first model turned out to be good enough.
The second model is based on the "second rotation" by the KLB computed
from the top m LSDB coordinates. This model gives us the decorrelated
coordinates built on top of the LSDB coordinates.
The simulation results on the human face database using the second model
suggest that this second rotation can further reduce the statistical dependency
among the coordinates, and allows better modeling for a class of
complicated images.
Get the full paper: PDF file.
Get the official version via doi:10.1016/S0031-3203(00)00116-3.
Please email
me if you have any comments or questions!
Go
back to Naoki's Publication Page