0.3 Approximation and processing in bases  (Page 4/5)

Sparsity with regularity

Sparse representations are obtained in a basis that takes advantage of some form of regularity of the input signals,creating many small-amplitude coefficients. Since wavelets have localized support,functions with isolated singularities produce few large-amplitude wavelet coefficients in the neighborhoodof these singularities. Nonlinear wavelet approximation produces a small error over spaces of functions that do not have “too many”sharp transitions and singularities. Chapter 9 shows that functionshaving a bounded total variation norm are useful models for images with nonfractal (finite length) edges.

Edges often define regular geometric curves. Waveletsdetect the location of edges but their square support cannot take advantage of their potential geometric regularity.More sparse representations are defined in dictionaries of curvelets or bandlets, which have elongated support in multiple directions,that can be adapted to this geometrical regularity. In such dictionaries, the approximation support λ _T is smaller but provides explicit information about edges' local geometricalproperties such as their orientation. In this context, geometry does not just apply to multidimensional signals.Audio signals, such as musical recordings, also have a complex geometric regularity in time-frequency dictionaries.

Compression

Storage limitations and fast transmission through narrow bandwidth channels require compression ofsignals while minimizing degradation. Transform codes compress signals by coding a sparse representation.Chapter 10 introduces the information theory needed to understand these codes and to optimize their performance.

In a compression framework, the analog signal has already been discretized intoa signal $f\left[n\right]$ of size N . This discrete signal isdecomposed in an orthonormal basis $\mathcal{B}={\left\{g_m\right\}}_{m\in \Gamma }$ of ${\mathbb{C}}^N$ :

Coefficients $⟨f,g_m⟩$ are approximated by quantized values $Q(⟨f,g_m⟩)$ . If Q is auniform quantizer of step Δ , then $|x-Q(x\left)\right|\le \Delta /2$ ; and if $\left|x\right|<\Delta /2$ , then $Q\left(x\right)=0$ . The signal $\tilde{f}$ restored from quantized coefficients is

An entropy code records these coefficients with R bits. The goal is to minimize the signal-distortion rate $d(R,f)=\parallel \tilde{f}{-f\parallel }^2$ .

The coefficients not quantized to zero correspond to the set ${\lambda }_T=\{m\in \gamma :|⟨f,g_m⟩|\ge T\}$ with $T=\Delta /2$ . For sparse signals, Chapter 10shows that the bit budget R is dominated by the number of bitsto code λ _T in γ , which is nearly proportional to its size $|{\lambda }_T|$ . This means that the “information” about a sparse representationismostly geometric. Moreover, the distortion isdominated by the nonlinear approximation error $\parallel f-f_{{\Lambda }_T}{\parallel }^2$ , for $f_{{\Lambda }_T}={\sum }_{m\in {\lambda }_T}⟨f,g_m⟩g_m$ . Compression is thus a sparse approximation problem. For a given distortion $d(R,f)$ , minimizing R requires reducing $|{\lambda }_T|$ and thus optimizing the sparsity.

The number of bits to code Λ _T can take advantage of any prior information on the geometry. [link] (b) shows that large wavelet coefficients are not randomly distributed. They have a tendency to be aggregated towardlarger scales, and at fine scales they are regrouped along edge curves or in texture regions. Using suchprior geometric models is a source of gain in coders such as JPEG-2000.