Where
${w}_{k}$ is the new weight vector,
${w}_{k-1}$ is the old weight vector, and
$\mu {x}_{k}{e}_{k}$ is a small step in the instantaneous error gradient
direction.
Interpretation in terms of weight error vector
Define
${v}_{k}={w}_{k}-{w}_{\mathrm{opt}}$
Where
${w}_{\mathrm{opt}}$ is the optimal weight vector and
Show that
${\Gamma}_{k}=({v}_{k}{v}_{k}^T)$ , the weight vector error covariance is bounded for
all
$k$ .
We
could have
$({v}_{k})\to 0$ , but
$\sigma({v}_{k})^2\to $∞ ; in which case the algorithm would not be
stable.
Recall that it is fairly straightforward to show that the
diagonal elements of the transformed covariance
${C}_{k}=U{\Gamma}_{k}U^T$ tend to zero if
$\mu < \frac{1}{{\lambda}_{\mathrm{max}}({R}_{\mathrm{xx}})}$ (
$U$ is the eigenvector matrix of
${R}_{\mathrm{xx}}$ ;
${R}_{\mathrm{xx}}=UDU^T$ ). The diagonal elements of
${C}_{k}$ were denoted by
${\gamma}_{k,i}\forall i, i=\{1, \dots , p\}$ .
We found a sufficient condition for
$\mu $ that guaranteed that the
steady-state
${\gamma}_{i}$ 's (and hence
$\sigma({v}_{k})^2$ ) are bounded:
$$\mu < \frac{\frac{2}{3}}{\sum_{i=1}^{p} {\lambda}_{i}}$$ Where
$\sum_{i=1}^{p} {\lambda}_{i}=\mathrm{tr}({R}_{\mathrm{xx}})$ is the input vector energy.
With this choice of
$\mu $ we have:
convergence in mean
bounded steady-state variance
This implies
$\lim_{B\to}B\to $∞vkB0
In other words, the LMS algorithm is stable about the optimumweight vector
${w}_{\mathrm{opt}}$ .
Learning curve
Recall that
${e}_{k}={y}_{k}-{x}_{k}^T{w}_{k-1}$
and
. These imply
${e}_{k}={\epsilon}_{k}-{x}_{k}^T{v}_{k-1}$
where
${v}_{k}={w}_{k}-{w}_{\mathrm{opt}}$ . So the MSE
$({e}_{k}^{2})={\sigma}_{\epsilon}^{2}+({v}_{k-1}^T{x}_{k}{x}_{k}^T{v}_{k-1})={\sigma}_{\epsilon}^{2}+(({x}_{n}{\epsilon}_{n}\forall n, n< k, {v}_{k-1}^T{x}_{k}{x}_{k}^T{v}_{k-1}))={\sigma}_{\epsilon}^{2}+({v}_{k-1}^T{R}_{\mathrm{xx}}{v}_{k-1})={\sigma}_{\epsilon}^{2}+(\mathrm{tr}({R}_{\mathrm{xx}}{v}_{k-1}{v}_{k-1}^T))={\sigma}_{\epsilon}^{2}+\mathrm{tr}({R}_{\mathrm{xx}}{\Gamma}_{k-1})$
Where
$(\mathrm{tr}({R}_{\mathrm{xx}}{\Gamma}_{k-1})\equiv {\alpha}_{k-1})\to (\alpha =\frac{c{\sigma}_{\epsilon}^{2}}{1-c})$ . So the limiting MSE is
${\epsilon}_{}$∞k∞ek2σε2cσε21cσε21c
Since
$0< c< 1$ was required for convergence,
${\epsilon}_{}$∞σε2 so that we see noisy adaptation leads to an MSE
larger than the optimal
To quantify the increase in the MSE, define the so-called
misadjustment :
$M=\frac{-{\epsilon}_{}}{}$∞σε2σε2ε∞σε21ασε2c1c
We would of course like to keep
$M$ as small as possible.
Learning speed and misadjustment trade-off
Fast adaptation and quick convergence require
that we take steps as large as possible. In other words,learning speed is proportional to
$\mu $ ; larger
$\mu $ means faster convergence. How
does
$\mu $ affect the
misadjustment?
To guarantee convergence/stability we require
$$\mu < \frac{\frac{2}{3}}{\sum_{i=1}^{p} {\lambda}_{i}({R}_{\mathrm{xx}})}$$ Let's assume that in fact
$\ll (\mu , \frac{1}{\sum_{i=1}^{p} {\lambda}_{i}})$ so that there is no problem with convergence. This condition
implies
$\ll (\mu , \frac{1}{{\lambda}_{i}})$ or
$\ll (\mu {\lambda}_{i}, 1)\forall i, i=\{1, \dots , p\}$ . From here we see that
This shows that larger step size
$\mu $ leads to larger
misadjustment.
Since we still have convergence in mean, this
essentially means that with a larger step size we "converge"faster but have a larger variance (rattling) about
${w}_{\mathrm{opt}}$ .
Yeah, it is a pain to say the least. You basically have to heat the substarte up to around 1000 degrees celcius then pass phosphene gas over top of it, which is explosive and toxic by the way, under very low pressure.
Harper
Do you know which machine is used to that process?
In this morden time nanotechnology used in many field .
1-Electronics-manufacturad IC ,RAM,MRAM,solar panel etc
2-Helth and Medical-Nanomedicine,Drug Dilivery for cancer treatment etc
3- Atomobile -MEMS, Coating on car etc.
and may other field for details you can check at Google
Azam
anybody can imagine what will be happen after 100 years from now in nano tech world
Prasenjit
after 100 year this will be not nanotechnology maybe this technology name will be change .
maybe aftet 100 year . we work on electron lable practically about its properties and behaviour by the different instruments
Azam
name doesn't matter , whatever it will be change... I'm taking about effect on circumstances of the microscopic world
Prasenjit
how hard could it be to apply nanotechnology against viral infections such HIV or Ebola?
Damian
silver nanoparticles could handle the job?
Damian
not now but maybe in future only AgNP maybe any other nanomaterials
Azam
Hello
Uday
I'm interested in Nanotube
Uday
this technology will not going on for the long time , so I'm thinking about femtotechnology 10^-15
Prasenjit
can nanotechnology change the direction of the face of the world