Subsection 10.1.1 Definitions
Definition 10.1.1.
Let \(A\) be an \(n \times n\) matrix. A non-zero vector \(v \in \RR^n\) is an eigenvector for \(A\) with eigenvalue \(\lambda\) if \(Av = \lambda v\text{.}\) The set of all eigenvalues for the matrix \(A\) is called the spectrum of \(A\text{.}\)
Eigenvectors for a matrix are vectors which do not change direction. They may be dilated by a factor \(\lambda\) (including a flip if \(\lambda\) is negative), but they still point in the same direction. Projections are also included since \(\lambda = 0\) is allowed; such an eigenvector would represent a direction sent entirely to zero under the transformation. Note that if \(v\) is an eigenvector, then so is \(\alpha v\) for any \(\alpha \neq 0\text{,}\) \(\alpha
\in \RR\text{.}\)
At first glance, this definition may seem insignificant; interesting, perhaps, but not necessarily central. It turns out that eigenvectors and eigenvalues are one of the most useful and important definitions in linear algebra. Many problems in applied mathematics depend upon finding the eigenvalues of a particular matrix.
Before I show how to calculate eigenvectors and eigenvalues, let me go through some obvious examples.
Example 10.1.2.
The identity is the easiest: for any vector \(v\text{,}\) \(\Id v = v\text{.}\) Therefore, all vectors are eigenvectors of the identity with eigenvalue \(\lambda = 1\text{.}\)
Example 10.1.3.
The zero matrix is similarly easy: it sends all vectors to the origin. All vectors are eigenvectors with eigenvalue \(\lambda = 0\text{.}\)
Example 10.1.4.
Consider a dialation matrix in \(\RR^3\text{:}\)
\begin{align*}
\begin{pmatrix}
a \amp 0 \amp 0 \\
0 \amp b \amp 0 \\
0 \amp 0 \amp c
\end{pmatrix}
\begin{pmatrix}
1 \\ 0 \\ 0
\end{pmatrix} =
\begin{pmatrix}
a \\ 0 \\ 0
\end{pmatrix} = a
\begin{pmatrix}
1 \\ 0 \\ 0
\end{pmatrix}\\
\begin{pmatrix}
a \amp 0 \amp 0 \\
0 \amp b \amp 0 \\
0 \amp 0 \amp c
\end{pmatrix}
\begin{pmatrix}
0 \\ 1 \\ 0
\end{pmatrix} =
\begin{pmatrix}
0 \\ b \\ 0
\end{pmatrix} = b
\begin{pmatrix}
0 \\ 1 \\ 0
\end{pmatrix}\\
\begin{pmatrix}
a \amp 0 \amp 0 \\
0 \amp b \amp 0 \\
0 \amp 0 \amp c
\end{pmatrix}
\begin{pmatrix}
0 \\ 0 \\ 1
\end{pmatrix} =
\begin{pmatrix}
0 \\ 0 \\ c
\end{pmatrix} = c
\begin{pmatrix}
0 \\ 0 \\ 1
\end{pmatrix}
\end{align*}
Here, all three of the standard basis vectors are eigenvectors. \(e_1\) has eigenvalue \(a\text{,}\) \(e_2\) has eigenvalue \(b\) and \(e_3\) has eigenvalue \(c\text{.}\) If \(a\text{,}\) \(b\) and \(c\) are all distinct numbers, there are no other eigenvectors (up to scale).
Example 10.1.5.
Consider a rotation in \(\RR^2\text{.}\) Assuming the rotation is not trivial (\(\theta \neq 0\)), then there are no preserved directions. This rotation has no eigenvectors. However, for a rotation in \(\RR^3\text{,}\) I have to choose an axis. The axis direction is fixed, so a vector in that direction would be an eigenvector with eigenvalue one.
Example 10.1.6.
Consider a reflection in \(\RR^2\text{.}\) A vector along the line of reflection is unchanged, so it would be an eigenvector with eigenvalue one. A vector perpendicular to the line of reflection is flipped exactly, so it would be an eigenvector with eigenvalue \(-1\text{.}\) In \(\RR^3\text{,}\) reflection happens over a plane through the origin. Any vector in the plane of reflection is unchanged so that vector is an eigenvector with eigenvalue one. Any vector perpendicular to the plane of reflection is precisely flipped, so that vector is an eigenvector with eigenvalue \(-1\text{.}\)
Example 10.1.7.
Finally, consider projections in \(\RR^2\text{.}\) Projection onto the \(x\)-axis preserves the \(x\) axis, so any vector \(\begin{pmatrix} a \\ 0 \end{pmatrix}\) is an eigenvector with eigenvalue one. However, it sends the \(y\)-axis direction to the origin, so any vector \(\begin{pmatrix} 0 \\ b \end{pmatrix}\) is an eigenvector with eigenvalue zero.
Subsection 10.1.2 Calculation of Eigenvalues and Eigenvectors
I would like to demonstrate an algorithm for finding eigenvectors and eigenvalues. Remember the definition: \(v\) is an eigenvector of \(A\) if \(Av = \lambda v\) for some real number \(\lambda\text{.}\) I can write the scalar multiplication \(\lambda v\) as \(\lambda \Id v\text{,}\) since the matrix \(\lambda \Id\) multiplies each entry by \(\lambda\text{.}\) Therefore, the equation becomes \(Av = \lambda \Id v\) or \(Av - \lambda \Id v = 0\text{,}\) where the right-hand side is the zero vector. Since matrix action is linear, this is the same as \((A - \lambda \Id) v = 0\text{.}\)
The definition of an eigenvector states that \(v\) is non-zero. Therefore, the matrix \((A - \lambda \Id)\) sends a non-zero vector \(v\) to the zero vector. This implies \((A - \lambda \Id)\) has a non-trivial kernel, hence it must have a zero determinant. Therefore, a necessary condition for the existence of an eigenvector with eigenvalue \(\lambda\) is that \(\det (A - \lambda \Id) = 0\text{.}\) I start the algorithm by computing this determinant.
This determinant has terms that involve the entries of \(A\) and \(\lambda\text{.}\) Since \(A\) is an \(n \times n\) matrix, each term can be the product of \(n\) entries. That means that the determinant will be a degree \(n\) polynomial in the variable \(\lambda\text{.}\)
Definition 10.1.8.
Let \(A\) be an \(n \times n\) matrix. The polynomial \(\det (A - \lambda \Id)\) in the variable \(\lambda\) is called the characteristic polynomial of the matrix.
I need an important definition concerning the roots of polynomials. Recall that \(\alpha\) is a root of a polynomial \(p(\lambda)\) if and only if \((\lambda - \alpha)\) is a factor of the polynomial. However, it may be true that \((\lambda - \alpha)^m\) is a factor for some integer \(m>1\text{.}\)
Definition 10.1.9.
Let \(p(\lambda)\) be a polynomial in the variable \(\lambda\) and \(\alpha\) a root. Let \(m\) be the largest positive integer such that \((\lambda -
\alpha)^m\) is a factor of \(p(\lambda)\text{.}\) Then \(m\) is called the multiplicity of the root \(\alpha\text{.}\)
A degree \(n\) polynomial has at most \(n\) real roots, so I can find at most \(n\) values for \(\lambda\) where the determinant vanishes. These will be the possible eigenvalues. However, the polynomial also has at most \(n\) factors, so the sum of all the multiplicities of the roots is also at most \(n\text{.}\) That implies that if there are roots with higher multiplicities, the polynomial will have fewer than \(n\) roots.
Now I return to the eigenvalue/eigenvector algorithm. When there is an eigenvalue \(\lambda\text{,}\) I want to find the matching eigenvectors. These vectors will be in the kernel of \((A - \lambda \Id)\text{,}\) so I just have to calculate this kernel.
The second step will always find eigenvectors: the determinant of the matrix is zero, so it must have a non-trivial kernel. However, I don’t know how many I will find (I don’t know the dimension of the kernel). In the first step, I am looking for the roots of a polynomial of degree \(n\text{.}\) It may have as many as \(n\) distinct real roots, but it may have far fewer or even none. Recall the identity and the zero matrix, where all vectors in \(\RR^n\) are eigenvectors for the appropriate eigenvalues, and also the rotations in \(\RR^2\) where there were no eigenvectors for any eigenvalue. Also, note that solving for roots of polynomials, particularly in high degrees, is a very hard problem.
Here is a definition that helps keep track of this information.
Definition 10.1.10.
Let \(A\) be an \(n \times n\) matrix and let \(\lambda\) be an eigenvalue of \(A\text{.}\) Then the eigenspace of \(\lambda\text{,}\) written \(E(\lambda)\text{,}\) is the set of all eigenvectors for the eigenvalue \(\lambda\text{.}\) Equivalently, it is the kernel of the matrix \(A - \lambda \Id\text{.}\) The dimension of the eigenspace associated with \(\lambda\) is bounded by the multiplicity of \(\lambda\) as a root of the characteristic polynomial.
Not all matrices have the maximum number of eigenvalues or eigenvectors. However, there is a special class of matrices that always achieves this maximum number. I defined the transpose of a matrix in
Section 4.1. The transpose leads to a new definition in this chapter.
Definition 10.1.11.
An \(n \times n\) matrix \(A\) is symmetric if \(A = A^T\text{.}\)
Example 10.1.12.
Here are two symmetric matrices.
\begin{align*}
\begin{pmatrix}
5 \amp 1 \amp -3 \\
1 \amp 2 \amp 0 \\
-3 \amp 0 \amp 4
\end{pmatrix} \amp \amp
\begin{pmatrix}
1 \amp 4 \amp -2 \amp -9 \\
4 \amp -4 \amp 3 \amp 3 \\
-2 \amp 3 \amp 8 \amp -1 \\
-9 \amp 3 \amp -1 \amp 7
\end{pmatrix}
\end{align*}
I defined symmetric matrices in this section because they behave very well with respect to eigenvalues and eigenvectors.
Proposition 10.1.13.
A symmetric \(n \times n\) matrix always has \(n\) real eigenvalues, counted with multiplicity. Moreover, the eigenspaces all have the maximum dimension, which is the multiplicity of the eigenvalue.
I’ll end this section with some examples of this algorithm for calculating eigenvalues and eigenvectors.
Example 10.1.14.
\begin{equation*}
A = \begin{pmatrix}
3 \amp 1 \\
1 \amp 3
\end{pmatrix}
\implies A - \lambda \Id = \begin{pmatrix}
3-\lambda \amp 1 \\
1 \amp 3 - \lambda
\end{pmatrix}
\end{equation*}
The determinant of this is \(9 - 6\lambda + \lambda^2 - 1 =
8-6\lambda + \lambda^2 = (\lambda-4)(\lambda-2)\text{.}\) So the eigenvalues are \(\lambda = 4\) and \(\lambda = 2\text{.}\) I can calculate the kernel of \(A - \lambda \Id\) first for \(\lambda = 2\text{.}\)
\begin{equation*}
A - 2 \Id = \begin{pmatrix}
1 \amp 1 \\
1 \amp 1
\end{pmatrix}
\end{equation*}
The kernel of this matrix is \(\Span \left\{
\begin{pmatrix} 1 \\ -1 \end{pmatrix} \right\}\text{.}\) All multiples of \(\begin{pmatrix} 1 \\ -1 \end{pmatrix}\) are eigenvectors for \(\lambda = 2\text{.}\) Next, I move on to \(\lambda = 4\text{.}\)
\begin{equation*}
A - 4 \Id = \begin{pmatrix}
-1 \amp 1 \\
1 \amp - 1
\end{pmatrix}
\end{equation*}
The kernel of this matrix is \(\Span \left\{
\begin{pmatrix} 1 \\ 1 \end{pmatrix} \right\} \text{.}\) All multiples of \(\begin{pmatrix} 1 \\ 1 \end{pmatrix}\) are eigenvectors for \(\lambda = 4\text{.}\)
Example 10.1.15.
\begin{equation*}
A = \begin{pmatrix}
0 \amp -2 \\
2 \amp 0
\end{pmatrix}
\implies A - \lambda \Id = \begin{pmatrix}
-\lambda \amp -2 \\
2 \amp -\lambda
\end{pmatrix}
\end{equation*}
The characteristic polynomial is \(\lambda^2 + 4 = 0\text{,}\) which has no roots. Therefore, there are no eigenvalues.
Example 10.1.16.
Consider a diagonal matrix.
\begin{equation*}
A = \begin{pmatrix}
1 \amp 0 \amp 0 \amp 0 \\
0 \amp -3 \amp 0 \amp 0 \\
0 \amp 0 \amp 4 \amp 0 \\
0 \amp 0 \amp 0 \amp 0
\end{pmatrix} \implies A - \lambda \Id = \begin{pmatrix}
1 - \lambda \amp 0 \amp 0 \amp \\
0 \amp -3 - \lambda \amp 0 \amp 0 \\
0 \amp 0 \amp 4 - \lambda \amp 0 \\
0 \amp 0 \amp 0 \amp - \lambda
\end{pmatrix}
\end{equation*}
The characteristic polynomial is \((1-\lambda)(-3-\lambda)(4-\lambda)(-\lambda)\text{,}\) which clearly has solutions \(\lambda = 1\text{,}\) \(-3\text{,}\) \(4\) and \(0\text{.}\) The eigenvectors are just the axis vectors \(e_1\text{,}\) \(e_2\text{,}\) \(e_3\) and \(e_4\text{,}\) respectively. In general, all diagonal matrices have axis vectors as eigenvectors and diagonal entries as eigenvalues.
