5.6 Linear algebra: matrices

The word matrix dates at least to the thirteenth century, when it was used to describe the rectangular copper tray, or matrix, that held individual leaden letters that were packed into the matrix to form a page of composed text. Each letter in the matrix, call it $a_{ij}$ , occupied a unique position $\left(ij\right)$ in the matrix. In modern day mathematical parlance, a matrix is a collection of numbers arranged in a two-dimensional array (a rectangle). We indicate a matrix with a boldfaced capital letter and the constituent elements with double subscripts for the row and column:

In this equation $A$ is an $mxn$ matrix, meaning that $A$ has $m$ horizontal rows and $n$ vertical columns. As an extension of the previously used notation, we write $A\in R^{mxn}$ to show that $A$ is a matrix of size $mxn$ with $a_{ij}\in R$ . The scalar element $a_{ij}$ is located in the matrix at the $i^{th}$ row and the $j^{th}$ column. For example, $a_{23}$ is located in the second row and the third column as illustrated in [link] .

The main diagonal of any matrix consists of the elements $a_{ii}$ . (The two subscripts are equal.) The main diagonal runs southeast from the top leftcorner of the matrix, but it does not end in the lower right corner unless the matrix is square $(\in R^{m\times m})$ .

The transpose of a matrix $A\in R^{m\times n}$ is another matrix $B$ whose element in row $j$ and column $i$ is $b_{ji}=a_{ij}$ for $1\le i\le m$ and $1\le j\le n$ . We write $B=A^T$ to indicate that $B$ is the transpose of $A$ . In MATLAB, transpose is denoted by. A more intuitive way of describing the transpose operation is to say that it flips the matrix about its main diagonal so thatrows become columns and columns become rows. A'

Matrix Addition and Scalar Multiplication. Two matrices of the same size (in both dimensions) may be added or subtracted in the same way as vectors, by adding or subtracting the corresponding elements. The equation $C=A\pm B$ means that for each i and $j,c_{ij}=a_{ij}\pm b_{ij}$ . Scalar multiplication of a matrix multiplies each element of the matrix by the scalar:

Matrix Multiplication. A vector can be considered a matrix with only one column. Thus we intentionally blur the distinction between $R^{n\times 1}$ and R ⁿ . Also a matrix can be viewed as a collection of vectors, each column of the matrix being a vector:

In the transpose operation, columns become rows and vice versa. The transpose of an $n\times 1$ matrix, a column vector , is a 1 $\times n$ matrix, a row vector :

Now we can define matrix-matrix multiplication in terms of inner products of vectors. Let's begin with matrices $A\in R^{m\times n}$ and $B\in R^{n\times p}$ . To find the product AB, first divide each matrix into column vectors and row vectors as follows:

Thus $a_i$ is the $i^{th}$ column of A and ${\alpha }_j^T$ is the $j^{th}$ row of A. For matrix multiplication to be defined, the width of the first matrix must match thelength of the second one so that all rows ${\alpha }_i^T$ and columns $b_i$ have the same number of elements $n$ . The matrix product, $C=$ AB, is an $m\times p$ matrix defined by its elements as $c_{ij}=({\alpha }_i,b_j)$ . In words, each element of the product matrix, $c_{ij}$ , is the inner product of the $i^{th}$ row of the first matrix and the $j^{th}$ column of the second matrix.

For n-vectors $x$ and $y$ , the matrix product $x^{T}y$ takes on a special significance. The product is, of course, a $1\times 1$ matrix (a scalar). The special significance is that $x^{T}y$ is the inner product of $x$ and $y$ :

Thus the notation $x^{T}y$ is often used in place of $(x,y)$ . Recall from Demo 1 from "Linear Algebra: Other Norms" that MATLAB usesto denote inner product. x'*y

Another special case of matrix multiplication is the outer product . Like the inner product, it involves two vectors, but this time the result is a matrix:

In the outer product, the inner products that define its elements are between one-dimensional row vectors of $x$ and one-dimensional column vectors of $y^T$ , meaning the $(i,j)$ element of $A$ is $x_i{y}_j$ .

There are several other equivalent ways to define matrix multiplication, and a careful study of the following discussion should improve your understanding of matrix multiplication. Consider A $\in R^{m\times n},B\in R^{n\times p}$ , and $C=AB$ so that $C\in R^{m\times p}$ . In pictures, we have

In our definition, we represent $C$ on an entry-by-entry basis as

In pictures,

You will prove in Exercise 3 that we can also represent $C$ on a column basis:

Finally, $C$ can be represented as a sum of matrices, each matrix being an outer product:

A numerical example should help clarify these three methods.

Let

Using the first method of matrix multiplication, on an entry-by-entry basis, we have

or

or

On a column basis,

$c_1=1231+2132+1221+3412;c_2=1232+2132+1223+3411;c_3=1231+2131+1222+3411.$

Collecting terms together, we have

On a matrix-by-matrix basis,

as we had in each of the other cases. Thus we see that the methods are equivalent-simply different ways of organizing the same computation!

Rotation

We know from the chapter on complex numbers that a complex number $z_1=x_1+j{y}_1$ may be rotated by angle $\theta$ in the complex plane by forming the product

When written out, the real and imaginary parts of $z_2$ are

If the real and imaginary parts of $z_1$ and $z_2$ are organized into vectors $z_1$ and $z_2$ as in the chapter on complex numbers , then rotation may be carried out with the matrix-vector multiply

We call the matrix $R\left(\theta \right)=cos\theta -sin\theta sin\theta cos\theta$ a rotation matrix .