0.3 Approximation and processing in bases  (Page 3/5)

Nonlinear approximations operate in two stages. First, a linear operator approximates the analog signal $\overline{f}$ with N samples written $f\left[n\right]=\overline{f}☆{\overline{\phi }}_s\left(ns\right)$ . Then, a nonlinear approximation of $f\left[n\right]$ is computed to reduce the N coefficients $f\left[n\right]$ to $M\ll N$ coefficients in a sparse representation.

The discrete signal $f$ can be considered as a vector of ${\mathbb{C}}^N$ . Inner products and norms in ${\mathbb{C}}^N$ are written

To obtain a sparse representation with a nonlinear approximation, we choose a new orthonormal basis $\mathcal{B}={\left\{g_m\left[n\right]\right\}}_{m\in \Gamma }$ of ${\mathbb{C}}^N$ , which concentrates the signal energy as much as possibleover few coefficients. Signal coefficients ${\{⟨f,g_m⟩\}}_{m\in \Gamma }$ are computed from the N input sample values $f\left[n\right]$ with an orthogonal change of basis that takes N ² operations in nonstructured bases. In a wavelet or Fourier bases, fastalgorithms require, respectively, $O\left(N\right)$ and $O\left(N{log}_{2}N\right)$ operations.

Approximation by thresholding

For $M<N$ , an approximation $f_M$ is computed by selecting the “best” $M<N$ vectors within $\mathcal{B}$ . The orthogonal projectionof $f$ on the space V _λ generated by M vectors ${\left\{g_m\right\}}_{m\in \Lambda }$ in $\mathcal{B}$ is

Since $f={\sum }_{m\in \gamma }⟨f,g_m⟩g_m$ , the resulting error is

We write $\left|\lambda \right|$ the size of the set λ . The best $M=\left|\lambda \right|$ term approximation, which minimizes $\parallel f-f_{\lambda }{\parallel }^2$ ,  is thus obtained by selecting the M coefficients of largest amplitude. These coefficients are above a threshold T that depends on M :

This approximation is nonlinear because the approximation set λ _T changes with $f$ . The resulting approximation error is:

[link] (b) shows that the approximation support λ _T of an image in a wavelet orthonormal basis depends on the geometry of edges and textures.Keeping large wavelet coefficients is equivalent to constructing an adaptiveapproximation grid specified by the scale–space support λ _T . It increases the approximation resolutionwhere the signal is irregular. The geometry of λ _T gives the spatial distribution of sharp image transitions and edges, and their propagation across scales.Chapter 6 proves that wavelet coefficients give important information about singularities and local Lipschitz regularity.This example illustrates how approximation support provides “geometric”information on $f$ , relative to a dictionary, that is a wavelet basis in this example.

[link] (d) gives the nonlinear wavelet approximation $f_M$ recovered from the $M=N/16$ large-amplitude wavelet coefficients, with an error $\parallel f-f_M{\parallel }^2/{\parallel f\parallel }^2=5\times {10}^{-3}$ . This error is nearly three times smaller thanthe linear approximation error obtained with the same number of wavelet coefficients, and the imagequality is much better.

An analog signal can be recovered from the discrete nonlinear approxima-tion  $f_M$ :

Since all projections are orthogonal, the overall approximationerror on the original analog signal $\overline{f}\left(x\right)$ is the sum of the analog sampling error and the discrete nonlinear error:

In practice, N is imposed by the resolution of the signal-acquisition hardware, and M is typically adjusted so that ${ϵ}_n(M,f)\ge {ϵ}_l(N,f)$ .