# 0.6 Nonparametric regression with wavelets

 Page 1 / 5

In this section, we consider only real-valued wavelet functions that form an orthogonal basis, hence $\varphi \equiv \stackrel{˜}{\varphi }$ and $\psi \equiv \stackrel{˜}{\psi }$ . We saw in Orthogonal Bases from Multiresolution analysis and wavelets how a given function belonging to ${L}_{2}\left(\mathbb{R}\right)$ could be represented as a wavelet series. Here, we explain how to use a wavelet basis to construct a nonparametric estimator for the regression function $m$ in the model

${Y}_{i}=m\left({x}_{i}\right)+{ϵ}_{i},\phantom{\rule{0.277778em}{0ex}}i=1,...,n,\phantom{\rule{0.277778em}{0ex}}n={2}^{J},\phantom{\rule{0.277778em}{0ex}}J\in \mathbb{N}\phantom{\rule{3.33333pt}{0ex}},$

where ${x}_{i}=\frac{i}{n}$ are equispaced design points and the errors are i.i.d. Gaussian, ${ϵ}_{i}\phantom{\rule{3.33333pt}{0ex}}\sim \phantom{\rule{3.33333pt}{0ex}}N\left(0,{\sigma }_{ϵ}^{2}\right)$ .

A wavelet estimator can be linear or nonlinear . The linear wavelet estimator proceeds by projecting the data onto a coarse level space. This estimator is of a kernel-type, see "Linear smoothing with wavelets" . Another possibility for estimating $m$ is to detect which detail coefficients convey the important information about the function $m$ and to put equal to zero all the other coefficients. This yields a nonlinear wavelet estimator as described in "Nonlinear smoothing with wavelets" .

## Linear smoothing with wavelets

Suppose we are given data ${\left({x}_{i},{Y}_{i}\right)}_{i=1}^{n}$ coming from the model [link] and an orthogonal wavelet basis generated by $\left\{\varphi ,\psi \right\}$ . The linear wavelet estimator proceeds by choosing a cutting level ${j}_{1}$ and represents an estimation of the projection of $m$ onto the space ${V}_{{j}_{1}}$ :

$\stackrel{^}{m}\left(x\right)=\sum _{k=0}^{{2}^{{j}_{0}}-1}{\stackrel{^}{s}}_{{j}_{0},k}{\varphi }_{{j}_{0},k}\left(x\right)+\sum _{j={j}_{0}}^{{j}_{1}-1}\sum _{k=0}^{{2}^{j}-1}{\stackrel{^}{d}}_{j,k}{\psi }_{j,k}\left(x\right)=\sum _{k}{\stackrel{^}{s}}_{{j}_{1},k}{\varphi }_{{j}_{1},k}\left(x\right),$

with ${j}_{0}$ the coarsest level in the decomposition, and where the so-called empirical coefficients are computed as

${\stackrel{^}{s}}_{j,k}=\frac{1}{n}\sum _{i=1}^{n}{Y}_{i}\phantom{\rule{4pt}{0ex}}{\varphi }_{jk}\left({x}_{i}\right)\phantom{\rule{1.em}{0ex}}\text{and}\phantom{\rule{1.em}{0ex}}{\stackrel{^}{d}}_{j,k}=\frac{1}{n}\sum _{i=1}^{n}{Y}_{i}\phantom{\rule{4pt}{0ex}}{\psi }_{jk}\left({x}_{i}\right)\phantom{\rule{3.33333pt}{0ex}}.$

The cutting level ${j}_{1}$ plays the role of a smoothing parameter: a small value of ${j}_{1}$ means that many detail coefficients are left out, and this may lead to oversmoothing. On the other hand, if ${j}_{1}$ is too large, too many coefficients will be kept, and some artificial bumps will probably remain in the estimation of $m\left(x\right)$ .

To see that the estimator [link] is of a kernel-type, consider first the projection of $m$ onto ${V}_{{j}_{1}}$ :

$\begin{array}{ccc}\hfill {\mathcal{P}}_{{V}_{{j}_{1}}}m\left(x\right)& =& \sum _{k}\left(\int \phantom{\rule{-0.166667em}{0ex}}m\left(y\right){\varphi }_{{j}_{1},k}\left(y\right)dy\right){\varphi }_{{j}_{1},k}\left(x\right)\hfill \\ & =& \int \phantom{\rule{-0.166667em}{0ex}}{K}_{{j}_{1}}\left(x,y\right)m\left(y\right)dy\phantom{\rule{3.33333pt}{0ex}},\hfill \end{array}$

where the (convolution) kernel ${K}_{{j}_{1}}\left(x,y\right)$ is given by

${K}_{{j}_{1}}\left(x,y\right)=\sum _{k}{\varphi }_{{j}_{1},k}\left(y\right){\varphi }_{{j}_{1},k}\left(x\right)\phantom{\rule{3.33333pt}{0ex}}.$

Härdle et al. [link] studied the approximation properties of this projection operator. In order to estimate [link] , Antoniadis et al. [link] proposed to take:

$\begin{array}{ccc}\hfill \stackrel{^}{{\mathcal{P}}_{{V}_{{j}_{1}}}}m\left(x\right)& =& \sum _{i=1}^{n}{Y}_{i}{\int }_{\left(i-1\right)/n}^{i/n}{K}_{{j}_{1}}\left(x,y\right)dy\hfill \\ & =& \sum _{k}\sum _{i=1}^{n}{Y}_{i}\left({\int }_{\left(i-1\right)/n}^{i/n}{\varphi }_{{j}_{1},k}\left(y\right)dy\right){\varphi }_{{j}_{1},k}\left(x\right)\phantom{\rule{3.33333pt}{0ex}}.\hfill \end{array}$

Approximating the last integral by $\frac{1}{n}{\varphi }_{{j}_{1},k}\left({x}_{i}\right)$ , we find back the estimator $\stackrel{^}{m}\left(x\right)$ in [link] .

By orthogonality of the wavelet transform and Parseval's equality, the ${L}_{2}-$ risk (or integrated mean square error IMSE) of a linear wavelet estimator is equal to the ${l}_{2}-$ risk of its wavelet coefficients:

$\begin{array}{ccc}\hfill \text{IMSE}=E{∥\stackrel{^}{m},-,m∥}_{{L}_{2}}^{2}& =& \sum _{k}E{\left[{\stackrel{^}{s}}_{{j}_{0},k}-{s}_{{j}_{0},k}^{\circ }\right]}^{2}+\sum _{j={j}_{0}}^{{j}_{1}-1}\sum _{k}E{\left[{\stackrel{^}{d}}_{jk}-{d}_{jk}^{\circ }\right]}^{2}\hfill \\ & +& \sum _{j={j}_{1}}^{\infty }\sum _{k}{d}_{jk}^{\circ \phantom{\rule{4pt}{0ex}}2}={S}_{1}+{S}_{2}+{S}_{3}\phantom{\rule{3.33333pt}{0ex}},\hfill \end{array}$

where

${s}_{jk}^{\circ }:=〈m,\phantom{\rule{0.166667em}{0ex}},,,{\varphi }_{jk}〉\phantom{\rule{1.em}{0ex}}\text{and}\phantom{\rule{1.em}{0ex}}{d}_{jk}^{\circ }=〈m,\phantom{\rule{0.166667em}{0ex}},,,{\psi }_{jk}〉$

are called `theoretical' coefficients in the regression context. The term ${S}_{1}+{S}_{2}$ in [link] constitutes the stochastic bias whereas ${S}_{3}$ is the deterministic bias. The optimal cutting level is such that these two bias are of the same order. If $m$ is $\beta -$ Hölder continuous, it is easy to see that the optimal cutting level is ${j}_{1}\left(n\right)=O\left({n}^{1/\left(1+2\beta \right)}\right)$ . The resulting optimal IMSE is of order ${n}^{-\frac{2\beta }{2\beta +1}}$ . In practice, cross-validation methods are often used to determine the optimal level ${j}_{1}$ [link] , [link] .

Define Intelligence.
what does BS Psychology do? or what jobs are available?
Hello guys good morning
gud mornin
ecstasy
afterboon
moshood
night
Hashirama
hii
Lakhwinder
Good evening, I hope everyone is having a great day! :)
Namira
Hy
fatima
Fine namira
fatima
What makes people with sickle cell not fat does the illness damage their body?
fatima
Normally, RBCs are shaped like discs, which gives them the flexibility to travel through even the smallest blood vessels. However, with this disease, the RBCs have an abnormal crescent shape resembling a sickle. This makes them sticky and rigid and prone to getting trapped in small vessels
this leads to blood not reaching much to parts, leading to anaemia. and hence them not being able to gain any weight
nice I see
Purim
I have a question regarding counselling.
Namira
A teenaged girl comes up to a therapist and speaks about her issues that she is having with her mother, for instance, she wants something from her mother ( attention, care and love) that is she is deprived of. How do we counsel her? what do we tell her in such a situation where she cannot directly
Namira
ok
Abdul
convey her needs to her mother.
Namira
natural law therapy
Abdul
first, like, try to get to know how is her conversational relationship with her mother. that if her mother blatantly ignores her, or doesn't pay much attention or what. also try to pry if their relationship is strained or okay okay or good.
your next course of action can only be decided by the answer to this
if her mother is like, not that good a parent, try to bring her in in therapy as well, if she's an okay okay one, have her and the girl sit down and talk in front of you. and if she's a good parent, all you have to do is encourage the girl to talk.
and also talk to the mother explaining​ that a healthy discussion or even a simple heart to heart with children is a must
Alright, thankyou so much. :)
Namira
What made psychology be studied under philosophy
..Cos everything begins with the love of Wisdom
Wayne
first ,explain what is psychology and philosophy .Then you state and explain the various reasons for which psychology was studied under philosophy.
Then you give a suitable conclusion.
please can anyone help me with different between Cognitive and Personality structure
Muhd
cognitive deals with mind and metal while personality deals with physical actions
Purim
cognitive is nothing but our way of processing the information....were as personality is totally different ....that it relates to our environment....that means how we behave according to our circumstances...
Divya
thanks
Muhd
now are u clear
Divya
okay
Purim
what is psychology
Psychology is the study of soul
fatima
scientific study of mind and behavior
Purim
a real life example of humanistic model
what is a linear relationship
directly related. like if one factor changes, so will the other
explain the process of visua sensation
Sensation is defined as the stimulation of sense organs Visual sensation is a physiological process which means that it is the same for everyone. We absorb energy such as electro magnetic energy (light) or sound waves by sensory organs such as eyes.
Happy
what is selective attention?
Aqeelah
Happy what did you say visual sensation is ?
Tanam
Tanam
you mean you experience sensation by what you see..
Ingrid
I thought it was by what your five senses interpreted
Fire
visual has to do with the eyes alone
Ayomide
sensory has to do with all the sense organs ranging from the eyes,nose, skin,tongue and ear
Ayomide
selective attention is focusing on a particular stimulus out of myriads of stimuli
Ayomide
visual sensation in simple terms is standing from far it seems to assume that the trees are emerged but when you get they totally separated
Tanam
seeing the trees been emerged is the visual sensation.That is my understanding, you can also find out
Tanam
a woman watch me all time why
because it is hard to test alot of things in it empiricaly
good Morning, I want someone to give me a lecture on levels of conscious
why is psyche represented as a butterfly?
Psyche's mythological imagery in ancient art is represented with butterfly wings, amply depicted in pottery as well. ... The metamorphosis of the butterfly inspired many to use butterflies as a symbol of the soul's exit from the body. Thus, the myth of Psyche concomitantly signifies soul and butterf
Sashini
ok thanks
Gabriella
On what basis do people argue that genes do not set limits on a person's potential?
The Nature-Nature Thingy
Wayne
Nature vs nurture
Eric
well, going by Polygenic Hypothesis formulated in a genome-wide association study (GWAS), each (even all) gene(s) affects every complex trait. from physical to even one's mental growth (though it's still debatable).
Like you know, the selfish gene, like genes passed on to generation after generation​; and if a person with a certain gene arrangement pattern meets someone remotely resembling even a part of it, he/she'll automatically behave selflessly towards the other person.
somewhat linking to the idea of soulmates, and Plato's idea of conjoint men and women who were split by God and hence strive to become one yet again. (I know I'm wandering away from psychology here)
But IQ is affected by both genes and environment.
"Other studies at the world-leading Minnesota Center for Twin and Family Research suggest that many of our traits are more than 50% inherited, including obedience to authority, vulnerability to stress, and risk-seeking. "
as for effect of the environment, study of epigenetics shows that how many inherited traits only get “switched on” in certain environments.
BUT, "Various options are pencilled in by our genes, and our life experiences determine which get inked." That is, what gets 'unlocked' depends on our experience and environment factor.
like determining if a person will be introvert or extrovert, by just observing his parents is not enough. though it does contribute a major part.
so, I'd like to go against your question (somewhat) and say, genes do set limitations, but what some genes only 'unlock' under certain circumstances and environment, so they don't limit it totally
hope it helped. @ me (CK) if any doubts or queries remain
great
Mateen
Hi, I have completed my Engineering with MBA in HR, and I want to pursue my career in field related to psychology. Is it possible to study psychology and pursue my dream.
You can but you'll have to start from FYBA again
Rama
Oh! hmm thank you.
Aparnaa
Look into getting a doctorate in industrial/organizational psychology. There are myriad fields within psychology; which one are you pursuing?
David
something like counselling!
Aparnaa
if you want to be a licensed professional counselor, you need to get a master of arts degree in counseling. It takes about two years to complete. you will have a practicum/internship as well within your coursework. Before you apply for a degree, find out your states requirements about licensure.
David
Hmm i will check the details and thank you so much!
Aparnaa
am a a student clinical officer
Got questions? Join the online conversation and get instant answers!