0.6 Nonparametric regression with wavelets  (Page 3/5)

The choice of the primary resolution level in nonlinear wavelet estimation has the same importance as the choice of a particular kernel in local polynomial estimation, i.e., it is not the most important factor. It is common practice to take $j_0=2$ or $j_0=3$ , although a cross-validation determination is of course possible [link] .

The selection of a threshold value is much more crucial. If it is chosen too large, the thresholding operation will kill too many coefficients. Too few coefficients will then be included in the reconstruction, resulting in an oversmoothed estimator. Conversely, a small threshold value will allow many coefficients to be included in the reconstruction, giving a rough, or undersmoothed estimator. A proper choice of the threshold involves thus a careful balance between smoothness and closeness of fit.

In case of an orthogonal transform and i.i.d. white noise, the same threshold can be applied to all detail coefficients, since the errors in the wavelet domain are still i.i.d. white noise. However, if the errors are stationary but correlated, or if the transform is biorthogonal, a level-dependent threshold is necessary to obtain optimal results [link] , [link] . Finally, in the irregular setting, a level and location dependent threshold must be utilized.

Many efforts have been devoted to propose methods for selecting the threshold. We now review some of the procedures encountered in the literature.

Choice of the threshold

Universal threshold

The most simple method to find a threshold whose value is supported by some statistical arguments, is probably to use the so-called `universal threshold' [link] , [link]

where the only quantity to be estimated is ${\sigma }_d^2$ , which constitutes the variance of the empirical wavelet coefficients. In case of an orthogonal transform, ${\sigma }_d={\sigma }_{ϵ}/\sqrt{n}$ .

In a wavelet transform, the detail coefficients at fine scales are, with a small fraction of exception, essentially pure noise. This is the reason why Donoho and Johnstone proposed in [link] to estimate ${\sigma }_d$ in a robust way using the median absolute deviation from the median (MAD) of ${\widehat{d}}_{J-1}$ :

If the universal threshold is used in conjunction with soft thresholding, the resulting estimator possesses a noise-free property: with a high probability,an appropriate interpolation of $\left\{\widehat{m}\left(x_i\right)\right\}$ produces an estimator which is at least as smooth as the function $m$ , see Theorem 1.1 in [link] . Hence the reconstruction is of good visual quality, so that Donoho andJohnstone called the procedure `VisuShrink' [link] . Although simple, this estimator enjoys a near-minimax adaptivity property, see "Adaptivity of wavelet estimator" . However, this near-optimality is an asymptotic one: for small sample size $t_{\text{univ}}$ may be too large, leading to a poor mean square error.

Oracle inequality

Consider the soft-thresholded detail coefficients ${\widehat{d}}^t$ . Another approach to find an optimal threshold is to look at the $l_2-$ risk

and to relate this risk with the one of an ideal risk ${\mathcal{R}}_{\text{ideal}}$ . The ideal risk is the risk obtained if an oracle tells us exactly which coefficients to keep or to kill.