1.17 Matrix representation of the approximation problem

Suppose our inner product space $V={\mathbb{R}}^M$ or ${\mathbb{C}}^M$ with the standard inner product (which induces the ${\ell }_2$ -norm).

Re-examining what we have just derived, we can write our approximation $\widehat{x}=Px=Vc$ , where $V$ is an $M\times N$ matrix given by

and $c$ is an $N\times 1$ vector given by

Given $x\in {\mathbb{R}}^M$ (or ${\mathbb{C}}^M$ ), our search for the closest approximation can be written as

or as

Using $V$ , we can replace $G=V^{H}V$ and $b=V^{H}x$ . Thus, our solution can be written as

which yields the formula

The matrix $V^{†}={\left(V^{H}V\right)}^{-1}{V}^H$ is known as the “pseudo-inverse.” Why the name “pseudo-inverse”? Observe that

Note that $\widehat{x}=V{V}^{†}x$ . We can verify that $V{V}^{†}$ is a projection matrix since

Thus, given a set of $N$ linearly independent vectors in ${\mathbb{R}}^M$ or ${\mathbb{C}}^M$ ( $N<M$ ), we can use the pseudo-inverse to project any vector onto the subspacedefined by those vectors. This can be useful any time we have a problem of the form:

where $x$ denotes a set of known “observations”, $V$ is a set of known “expansion vectors”, $c$ are the unknown coefficients, and $e$ represents an unknown “noise” vector. In this case, the least-squares estimate is given by