<< Chapter < Page
  Signal theory   Page 1 / 1
Chapter >> Page >
Introduces the singular value decomposition and its application in principal component analysis.

Singular value decomposition (SVD) can be thought of as an extension to eigenvalue decomposition for non-symmetric matrices. Consider an m × n matrix X . The following two matrices are symmetric and so have eigenvalue decompositions

X X H = U Λ 1 U H and X H X = V Λ 2 V H ,

where X X H is an m × m matrix and X H X is an n × n matrix. It turns out that we can therefore decompose the matrix X as X = U Σ V H , where Σ is an m × n “diagonal” matrix: Σ i , i = σ i are the singular values of X , and Σ i , j = 0 for i j . The pseudoinverse of the matrix can then be written as X = V Σ U H , where Σ i , i = 1 / σ i and Σ i , j = 0 for i j .

Principal component analysis

Principal component analysis can be thought of as KLT for sampled data. Assume that { x 1 , x 2 , x L } R n is a zero-mean dataset, and collect it into a matrix X = [ x 1 x 2 x L ] R n x L . Next, compute the SVD X = U Σ V T with the corresponding eigenvalue decomposition X X T = U Λ U T . The matrix U is known as the principal component analysis (PCA) matrix of X ; its columns U 1 , U 2 , ... U n are known as principal components, and its PCA coefficients are given by Y = U T X = Σ V T . The matrix Y contains the “scores” of all data points in the columns of X against the principal components U i . One can show that the principal components in the matrix U follow the formulation

u j = arg max w R n 1 L i = 1 L | x i , u i | 2 subject to w , u i = 0 , i = 1 , ... , j - 1 .

In words, u j is the direction in which the projections of the data has the largest variance while being orthogonal to { u 1 , u 2 , ... u j - 1 } .

Get Jobilize Job Search Mobile App in your pocket Now!

Get it on Google Play Download on the App Store Now




Source:  OpenStax, Signal theory. OpenStax CNX. Oct 18, 2013 Download for free at http://legacy.cnx.org/content/col11542/1.3
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Signal theory' conversation and receive update notifications?

Ask