Chapter 6 Eilgenvalues and Eigenvectors

6.1 Introduction to Eigencalues

This chapter enters a new part of linear algebra, based on Ax=AX.Ax = AX.

Certain exceptional vectors x are in the same direction as Ax. Those are the "eigenvectors". Multiply an eigenvector by A, and the vector Ax is a number A times the original x.

+ 선형변환 A에 의한 변환 결과가 자기 자신의 상수배가 되는 0이 아닌 벡터

The basic equation is Ax=AxAx = Ax. The number AA is an eigenvalue of AA.

  • 고유값 λ\lambda는 A의 고유값이며, 0이 될 수 있다.
  • Ax=0xAx=0x는 이 고유벡터 x가 nullspace 안에 있다는 것을 의미한다.
  • A가 identity matrix라면 모든 벡터는 Ax=x를 가지고 있음.
  • 모든 벡터들이 I의 고유벡터이고, 모든 고유값 lambda는 1임
ex1)

A=[.8.3.2.7]det[.8−λ.3.2.7−λ]=λ2−32λ+12=(λ−1)(λ−12)A= \begin{bmatrix}{.8}&{.3}\\{.2}&{.7} \end{bmatrix} det\begin{bmatrix}{.8-\lambda}&{.3}\\{.2}&{.7-\lambda} \end{bmatrix} = \lambda^{2}-\frac{3}{2} \lambda + \frac{1}{2} = (\lambda-1)(\lambda-\frac{1}{2})

nullspace에 있는

  • (A−I)x1=0,Ax1=x1(A-I)x_1=0, Ax_1=x_1, 고유벡터(.6,.4)→Ax1=[.8.3.2.7][.6.4]=x1(.6,.4) \rightarrow Ax_1=\begin{bmatrix}{.8}&{.3}\\{.2}&{.7}\end{bmatrix}\begin{bmatrix}{.6}\\{.4} \end{bmatrix}=x_1
  • (A−12I)x2=0,Ax2=x2(A-\frac{1}{2}I)x_2=0, Ax_2=x_2, 고유벡터(1,−1)→Ax2=[.8.31−1][.5−.5]=x2(1,-1) \rightarrow Ax_2=\begin{bmatrix}{.8}&{.3}\\{1}&{-1}\end{bmatrix}\begin{bmatrix}{.5}\\{-.5} \end{bmatrix}=x_2

When A is squared, the eigenvectors stay the same. The eigenvalues are squared.

  • 여기서 고유벡터들이 절대 섞이지 않고 자신의 방향에 머물기 때문에 패턴이 유지된다.

Other vectors do change direction. But all other vectors are combinations of the two eigenvectors.

  • 방향이 변하는 다른 벡터들의 경우.
  • A의 첫번째 열은 x1+(.2)x2x_1+(.2)x_2 임.
  • 예제에서는 [.8−.2]=x1+(.2)x2=[.6−.4]+[.2−.2]\begin{bmatrix}{.8}\\{-.2} \end{bmatrix}=x_1+(.2)x_2=\begin{bmatrix}{.6}\\{-.4} \end{bmatrix}+\begin{bmatrix}{.2}\\{-.2} \end{bmatrix}
  • A에 곱해질때 각 고유벡터는 고유값에 곱해짐
  • At every step x1x_1 is unchanged and x2x_2 is multiplied by (12\frac{1}{2}), so we have (12)99\frac{1}{2})^{99}
  • x99[.8−.2]isreally1+(.2)1299x2=[.6−.4]+[verysmallvector]x^{99}\begin{bmatrix}{.8}\\{-.2} \end{bmatrix} isreally_1+(.2)\frac{1}{2}^{99}x_2=\begin{bmatrix}{.6}\\{-.4} \end{bmatrix}+\begin{bmatrix}{very}\\{small}\\{vector} \end{bmatrix}
  • This is the first column of A 100. →\rightarrow 여기서(.8,.2)를 의미함

A is a Markov matrix(각 열 벡터들을 더하면 1이 되는 행렬).

  • Its eigenvector xl=(.6,.4)x_l = (.6,.4) is the steady state-which all columns of Ak will approach.
ex2)

The projection matrix P=[.5.5.5.5]P=\begin{bmatrix}{.5}&{.5}\\{.5}&{.5} \end{bmatrix} has eigencalues λ=1\lambda=1 and λ=0\lambda=0

위 예는 markov matrices, singular matrices, 그리고 symmetric matrices를 설명한다.

  • All have special A's and x 's:
    • P=[.5.5.5.5]P=\begin{bmatrix}{.5}&{.5}\\{.5}&{.5} \end{bmatrix}의 각 열은 1을 추가해서 람다=1이 고유값
    • P는 singular여서 람다=0이 고유값
    • P는 대칭이어서 고유벡터(1,1)과 (1,-1)이 직각
  • 영공간nullspace는 0에 투영되어있음
  • 열 공간은 그 스스로를 투영함
  • Project each aprt v=[1−122]v=\begin{bmatrix}{1}&{-1}\\{2}&{2} \end{bmatrix} projects onto Pv=[0022]P_v=\begin{bmatrix}{0}&{0}\\{2}&{2} \end{bmatrix}
  • Projections have λ=0\lambda = 0 and 11. Permutations have all ∣A∣=1|A| = 1.
ex3)

The reflection matrix R=[0110]R=\begin{bmatrix}{0}&{1}\\{1}&{0} \end{bmatrix} has eigenvalues 1 snd -1

  • 고유벡터(1,1)은 R에 의해 변하지 않음!
  • reflection=2(projection)−Ireflection = 2(projection)-I 때문에 R에서 고유벡터은 P에서와 같다.
  • R=2P−I,[0110]=2[.5.5.5.5]−[1001].…(2)R=2P-I, \begin{bmatrix}{0}&{1}\\{1}&{0} \end{bmatrix}=2\begin{bmatrix}{.5}&{.5}\\{.5}&{.5} \end{bmatrix}-\begin{bmatrix}{1}&{0}\\{0}&{1} \end{bmatrix}.\dots (2)
  • 2Px=2λx2Px=2\lambda x처럼 행렬이 두 배가 되면 고유값이 2배가 된다.
  • 그 결과가 (2p−i)X=(2λ−1)x(2p-i)X=(2\lambda-1)x!!
  • 행렬이 I에 의해 변하면 각 λ\lambda는 1에 의해 바뀐다. 고유벡터 내에서는 변화 없음!!

Key idea: The eigenvalues of Rand P are related exactly as the matrices are related:

  • The eigenvalues of R=2P−IR = 2P - I are 2(1) - 1 = 1 and 2(0) - 1 = -1.
  • The eigenvalues of R2R^2 are λ2\lambda^2. In this case R2=IR^2 = I. Check (I)2(I)^2 = 1 and (−1)2(-1)^2 = 1.

The Equation for the Eigenvalues

The Equation for the Eigenvalues: Px=x,Px=0,Rx=−xP x = x, P x = 0, Rx = -x is the key caculation almost every application starts by solving Ax=λxAx = \lambda x.

  • 일단 λx\lambda x를 왼쪽으로 넘기면 (A−λI)x=0(A-\lambda I)x=0을 쓸 수 있고
  • 행렬 (A−λI)(A-\lambda I)매를 한 고유벡터 x는 0 벡터임.
  • The eigenvectors make up the nullspace of A−λIA - \lambda I
  • 고유값 람다를 알때, (A−λI)x=0(A-\lambda I)x=0을 풀어서 고유벡터를 찾을 수 있음

If (A−λI)X=0(A - \lambda I)X = 0 has a nonzero solution, A−λIA - \lambda I is not invertible. The determinant of A−λIA -\lambda I must be zero.

  • 고유값 람다 알아내는 법
    • The number λ\lambda is an eignvalue of A if and only if A−λIA-\lambda I is singular:
    • Equtation for the eigenvalues →det(A−λI)=0\rightarrow det(A-\lambda I)=0
      • det(A−λI)=0det(A-\lambda I)=0 여기에는 x가 아닌 람다만 들어있음
      • A가 N ×\times N 행렬일때 det(A−λI)=0det(A-\lambda I)=0는 degree N을 가지게 됨

For each elgenvalue λ\lambda solve (A−λI)x=0(A - \lambda I)x = 0 or Ax=λxA_x=\lambda x to find an eigenvector x.

ex4)

A=[1224]A=\begin{bmatrix}{1}&{2}\\{2}&{4} \end{bmatrix} is already singular (zero determinant). Find its λ\lambda's and xx's

  • A가 singular일때 λ=0\lambda=0은 고유값들 중 하나이다.
  • Ax=0xAx=0x가 solution을 가짐! 이는 λ=0\lambda=0에 대한 고유 벡터임
  • 하지만 det(A−λI)=0det(A-\lambda I)=0은 모든 람다들과 x들을 찾는 방법임
  • Subtract λ\lambda from the diagonal to find a−λi=[1−λ224−λ]a-\lambda i =\begin{bmatrix}{1-\lambda}&{2}\\{2}&{4-\lambda} \end{bmatrix}

Take the determinant "ad - bc" of this 2 by 2 matrix.

  • det[1−λ224−λ]=(1−λ)(4−λ)−(2)(2)=λ2−5λdet\begin{bmatrix}{1-\lambda}&{2}\\{2}&{4-\lambda} \end{bmatrix}=(1-\lambda)(4-\lambda)-(2)(2)=\lambda^{2}-5\lambda
  • λ2−5λ\lambda^{2}-5\lambda를 0으로 놓으면 근은 람다가 0 또는 5
  • One solution is λ=0\lambda = 0 (as expected, since A is singular). Factoring into λ\lambda times λ−5\lambda - 5, the other root is λ=5\lambda = 5.:
  • det(A−λI)=λ2−5λ=0det(A-\lambda I)=\lambda^2-5\lambda =0 yields the eigenvalues λ1=0\lambda_1=0 and λ2=5\lambda_2=5
  • 고유벡터를 찾아보자
  • (A−0I)x=[1224][yz]=[00](A-0I)x=\begin{bmatrix}{1}&{2}\\{2}&{4} \end{bmatrix}\begin{bmatrix}{y}\\{z} \end{bmatrix}=\begin{bmatrix}{0}\\{0} \end{bmatrix} yields the eigenvector [yz]=[2−1]\begin{bmatrix}{y}\\{z} \end{bmatrix}=\begin{bmatrix}{2}\\{-1} \end{bmatrix} for λ1=0\lambda_1=0
  • (A−5I)x=[−422−1][yz]=[00](A-5I)x=\begin{bmatrix}{-4}&{2}\\{2}&{-1} \end{bmatrix}\begin{bmatrix}{y}\\{z} \end{bmatrix}=\begin{bmatrix}{0}\\{0} \end{bmatrix} yields the eigenvector [yz]=[12]\begin{bmatrix}{y}\\{z} \end{bmatrix}=\begin{bmatrix}{1}\\{2} \end{bmatrix} for λ2=5\lambda_2=5.
    • 0과 5가 고유값이기 때문에 A−0IA-0I, A−5IA-5I는 singular이다.
    • 고유벡터 (2,-1)과 (1,2)은 영공간nullspace에 있다.: (A−λI)x=0(A-\lambda I)x=0 is Ax=λxAx=\lambda x

Warning: There is nothing exceptional about A = O. Like every other number, zero might be an eigenvalue and it might not. If A is singular, it is. The eigenvectors fill the nullspace: Ax = Ox = O. If A is invertible, zero is not an eigenvalue. We shift A by a multiple of I to make it singular.

summary: n matrix에 의한 고유값 문제를 해결하기 위해 밟을 step!

  1. Compute the determinant of A−λIA-\lambda I.
  2. Find the roots of this polynomial
  3. solve (A−λI)x=0(A-\lambda I)x=0 to find an eigenvector xx

Warning: Some 2 by 2 matrices have only one line of eigenvectors. This can only happen when two eigenvalues are equal. Similarly some n by n matrices don’t have n independent eigenvectors. Without n eigenvectors, we don’t have a basis. We can’t write every v as a combination of eigenvectors.

Good News, Bad News

Bad news: A 행에 또다른 열을 추가하거나 행을 바꾼다면 고유값들은 항상 변할 것임!

  • Elimination does not preserve the A'S. The triangular U has its eigenvalues sitting along the diagonal-they are the pivots. But they are not the eigenvalues of A!

Good news: The product λ1\lambda_1 times λ2\lambda_2 and the sum λ1\lambda_1 + λ2\lambda_2 can be found quickly from the matrix.

The product of the n eigenvalues equals the determinant. The sum of the n eigenvalues equals the sum of the n diagonal entries.

  • The sum of the entries on the main diagonal is called the trace of A:
    • λ1+λ2+⋯+λn=trace=a11+a22+⋯+ann\lambda_1+\lambda_2+\dots+\lambda_n=trace=a_{11}+a_{22}+\dots+a_{nn}
    • 계산을 잘못했는지 알 수 있음! 잘못되었다면? go back to det(A−λI)=0det(A-\lambda I)=0
  • The determinant test makes the product of the A'S equal to the product of the pivots (assuming no row exchanges). But the sum of the A'S is not the sum of the pivots-as the example showed. The individual A's have almost nothing to do with the pivots. In this new part of linear algebra, the key equation is really nonlinear: A multiplies x.

Imaginary Eigenvalues

The eigenvalues might not be real numbers.?!?!

ex5)

The 90o90^o rotation Q=[0−110]Q=\begin{bmatrix}{0}&{-1}\\{1}&{0} \end{bmatrix} has no real eigenvectors. Its eigenvalues are λ=i\lambda=i and λ=−i\lambda=-i. Sum of λ\lambda's= trace=0, Product=determinany=1.

  • rotation 후 QxQ_x 어느 벡터도 x와 같은 방향에 있지 않음 x=0일때 빼고.
  • 그러면 고유벡터가 될 수 없는데 그래도 우리는 허수imaginary numbers로 보겠다.
  • −I-I인 Q2Q^2을 봐봐. 만약 Q가 90도 회전하면 Q2Q^2은 180도 회전하겠지? 그건 고유값들이 -1과 -1이라는 거고.(확실히 −IX=−1X-IX=-1X인 건 아니까.) Q2Q^2이니까 각 λ2\lambda ^2이 될거고. 우리는 λ2=−1\lambda ^2=-1을 얻게될거야. 그러면 근은 i,−ii,-i가 나오겠지. 우리는 허수 i가 고유벡터에 있다는 것을 알 수 있어.
    • complexeigenvectors\begin{matrix} complex \\ eigenvectors \end{matrix} [0−110][1i]=−i[1i]\begin{bmatrix}{0}&{-1}\\{1}&{0} \end{bmatrix}\begin{bmatrix}{1}\\{i} \end{bmatrix}=-i\begin{bmatrix}{1}\\{i} \end{bmatrix} and [0−110][i1]=i[i1]\begin{bmatrix}{0}&{-1}\\{1}&{0} \end{bmatrix}\begin{bmatrix}{i}\\{1} \end{bmatrix}=i\begin{bmatrix}{i}\\{1} \end{bmatrix}
    • 여기서 복소수벡터complex vector x1=(1,i),x2=(i,1)x_1=(1,i),x_2=(i,1)는 회전한 방향을 유지하지.
    • 이 예제들은 실제 행렬이 복소수 고유치들과 고유값들을 쉽게 가질 수 있는 모든 중요한 포인트를 만들지.
    • 특히 고유치 i와 -i는 Q의 two special properties을 말해.
    1. Q is an otrhogonal matrix so the absolute value of each λ\lambda is ∣λ∣=1|\lambda|=1
    2. Q is a skew-wymmetric matrix so each λ\lambda is pure imaginary.
    • A symmetric matrix (AT=A)(^AT=A) can be compared to a real number. A skew-symmetric matrix (AT=−A)(A^T = -A) can be compared to an imaginary number. An orthogonal matrix (ATA=1)(A^T A = 1) can be compared to a complex number with ∣λ∣=1|\lambda| = 1.
    • The eigenvectors for all these special matrices are perpendicular. Somehow (i, I) and (1, i) are perpendicular

Eigshow in MATLAB

The eigenvalue A is the length of AxAx , when the unit eigenvector x lines up. The built-in choices for A illustrate three possibilities: 0,1, or 2 directions where AxA_x crosses x.

  • 0: 실제 고유벡터가 없음. AxAx가 x의 뒤 혹은 앞에 있음. 이건 고유값과 고유벡터들이 rotation Q에서 복소수라는 것을 의미함.
  • 1: 고유벡터들의 오직 한 line만이 존재. Ax와 x는 움직이는 방향에서 만나긴 하는데 교차하진 않음.(2×\times2행렬에서)
  • 2: 두 독립적인 방향의 고유벡터가 있음. This is typica! Ax는 x와 첫번째 고유벡터 x1x_1에서 교차하고, 두번째 고유벡터 x2x_2에서 다시 돌아옴(cross back). 그러면 Ax와 x는 −x1-x_1과 −x2-x_2에서 다시 교차함.

REVIEW OF THE KEY IDEAS

  1. Ax=λxAx = \lambda x says that eigenvectors x keep the same direction when multiplied by A.
  2. Ax=λxAx = \lambda x also says that det(A−λI)=0det(A - \lambda I) = 0. This determines n eigenvalues.
  3. The eigenvalues of A2A^2 and A−1A^{-1} are λ2\lambda^2 and λ−1\lambda^{-1}, with the same eigenvectors.
  4. The sum of the A'S equals the sum down the main diagonal of A (the trace). The product of the A'S equals the determinant.
  5. Projections PP, reflections RR, 90o90^o rotations QQ have special eigenvalues 1,0,−1,i,−i1,0, -1, i, -i. Singular matrices have λ=O\lambda = O. Triangular matrices have A'S on their diagonal.

WORKED EXAMPLES

6.1.A 다 끝내고 시간 나면 할 것,,

6.2 Diagonalizing a Matrix

Does the matrix A turn into a diagonal matrix A when we use the eigenvectors properly?

Diagonalization: Suppose the n by n matrix A has n linearly independent eigervectors x1,…,xnx_1,\dots,x_n. Put them into the columns of an eigenvectors matrix S. Then S−1ASS^{-1}AS is tje eigenvalue matrix Λ\Lambda

  • Eigenvector matrix S, Eigenvalue matrix Λ\Lambda, S−1AS=Λ=[λ1⋱λn]S^{-1}AS=\Lambda=\begin{bmatrix}{\lambda_1}&{}&{}\\{}&{\ddots}&{}\\{}&{}&{\lambda_n}\end{bmatrix}
  • i.e. AS=SΛAS=S\Lambda is S−1AS=ΛS^{-1}AS=\Lambda or A=SAS−1A=SAS^{-1}
  • 행렬 S는 역을 취할 수 있기 때문에 그 열들은 선형적으로 독립이 될 수 있다. n개의 독립적인 고유벡터 없이 대각선화 할 수 없다.
  • AA와 Λ\Lambda는 같은 고유값 λ1,…λn\lambda_1,\dots\lambda_n을 가지고 있다.
  • 이 고유 벡터들은 다르다.
ex1)

$$\text{Eigenvectors }\begin{bmatrix}{1}\\{0}\end{bmatrix}\begin{bmatrix}{1}\\{1}\end{bmatrix} \begin{matrix}\begin{bmatrix}{1}&{-1}\\{0}&{1}\end{bmatrix}\\S^{-1}\end{matrix}\begin{matrix}\begin{bmatrix}{1}&{5}\\{0}&{6}\end{bmatrix}\\A\end{matrix}\begin{matrix}\begin{bmatrix}{1}&{1}\\{0}&{1}\end{bmatrix}\\S\end{matrix}$ = $\begin{matrix}\begin{bmatrix}{1}&{0}\\{0}&{6}\end{bmatrix}\\ \Lambda\end{matrix}$$

A=SΛS−1→A2=SΛS−1SΛS−1→ remove S−1S=I→SΛ2S−1A=S\Lambda S^{-1} \rightarrow A^2=S \Lambda S^{-1}S \Lambda S^{-1} \rightarrow \text{ remove } S^{-1}S=I \rightarrow S \Lambda^2S^{-1}

  • The kthk^{th} power will be Ak=SΛkS−1A^k = S \Lambda^k S^{-1}

Remark

  1. 고유값 λ1,…λn\lambda_1,\dots\lambda_n는 다 달라서 자동으로 고유벡터 x1,…,xnx_1,\dots,x_n도 독립적이다. 반복죄지 않는 고유값을 가진 어느 행렬도 대각성화 할 수 없다. 2.어느 nonzero constats를 고유벡터에 곱할 수 있다.Ax=λxAx=\lambda x는 true로 남아있다.
  2. S에서 고유벡터는 Λ\Lambda의 고유값에 따라 같은 정렬에 있다.(The eigenvectors in SS come in the same order as the eigenvalues in Λ\Lambda) 4.어떤 행렬이 너무 적은 고유벡터를 가지고 있으면 그 행렬은 대각선화할 수 없다.

Note: Remember that there is no connection between invertibility and diagonalizability; 1. Invertibility is concerened with the eigenvalues(λ=0 or λ≠0\lambda=0 \text{ or } \lambda \neq 0) 2. Diagonalizability is concerend with the eigenvectors(too few or enough for S)

Warning: Each eigenvalue has at least one eigenvector! A−λIA - \lambda I is singular. If (A−λI)x=0(A - \lambda I)x = 0 leads you to x=0x = 0, λ\lambda is not an eigenvalue. Look for a mistake in solving det(A−λI)=Odet(A - \lambda I) = O

n개의 람다들에 대해 고유벡터들은 독립적이다. 그러면 우리는 A를 대각화할 수 있다.

Independent xx from differentt λ\lambda

  • 구별되는 고유값들에 관련된 고유벡터 x1,…,xnx_1,\dots,x_n는 선형적으로 독립이다.
  • n개의 다른 고유값들을(반복되지 않은 람다들) 가진 n by n 행렬은 대각화되어야만 한다.
ex2)

Powers of A

  • A가 Markov matrix이면?
    • A=SΛS−1A=S\Lambda S^{-1}에서 구한 고유벡터들은 SS의 열 안에 있고, A2A^2의 고유벡터이기도 함.
  • A2A^2은 SS와 같고, A2A^2 고유벡터 행렬은 Λ2\Lambda^2임을 보여보자.
    • SAME SS for A2A^2
      • A2=SΛS−1SΛS−1=SΛ2S−1A^2 = S\Lambda S^{-1}S\Lambda S^{-1} = S\Lambda^2 S^{-1}
    • Powers of A
      • Ak=SΛkS−1A^k = S\Lambda^k S^{-1}
    • Limit k→∞k \rightarrow \infty
      • A100=SΛS−1A^{100}=S\Lambda S^{-1}

Note: Q: When does Ak→zeroA^k \rightarrow zero matrix? A: All ∣λ∣<1|\lambda|<1.

Fibonacci Numbers

Every new Fibonacci number is the sum of the two previous F's:

  • The sequence 0,1,1,2,3,5,8,13,... comes from Fk+2=Fk+1+FkF_{k+2}=F_{k+1}+F_{k}

step 1: 일단 matrix equation uk+1=Auku_{k+1}=Au_k으로 시작!

  • Let uk=[Fk+1Fk]u_k=\begin{bmatrix}{F_{k+1}}\\{F_k}\end{bmatrix} . The rule Fk+2=Fk+1+FkFk+1=Fk+1\begin{matrix}{F_{k+2}=F_{k+1}+F_k}\\{F_{k+1}=F_{k+1}}\end{matrix} is uk+1=[1110]uku_{k+1} = \begin{bmatrix}{1}&{1}\\{1}&{0}\end{bmatrix}u_k

Every step multiples by A=[1110]A= \begin{bmatrix}{1}&{1}\\{1}&{0}\end{bmatrix}.

  • 100번하면 u100=A100uku_{100}=A^{100}u_k가 된다.
  • Subtract λ\lambda from the diagonal of A:
    • A−λI=[1−λ11−λ]A-\lambda I = \begin{bmatrix}{1-\lambda}&{1}\\{1}&{-\lambda}\end{bmatrix} leads to det(A−λI)=λ2−λ−1det(A-\lambda I)=\lambda^2 - \lambda - 1
    • −b±sqrt(b2−4ac)/2a-b\pm sqrt(b^2-4ac)/2a 으로 해를 구할 수 있지
    • Eigenvalues λ1=1+52≈1.618\lambda_1=\frac{1+\sqrt{5}}{2} \approx 1.618 and λ2=1−52≈−.618\lambda_2=\frac{1-\sqrt{5}}{2} \approx -.618
    • Eigenvalues로 Eigenvectors를 구해보자. x1=(λ1,1)x_1=(\lambda_1,1) and x2=(λ2,1)x_2=(\lambda_2,1)

step 2: Find the combination of those eigenvectors that gives u0=(1,0)u_0=(1,0)

  • [10]=1λ1−λ2([λ11]−[λ21])\begin{bmatrix} {1}\\{0} \end{bmatrix} = \frac{1}{\lambda_1-\lambda_2} \bigg( \begin{bmatrix}{\lambda_1}\\{1} \end{bmatrix} - \begin{bmatrix}{\lambda_2}\\{1} \end{bmatrix} \bigg) or u0=x1−x2λ1−λ2u_0=\frac{x_1-x_2}{\lambda_1-\lambda_2}

step 3: multiplies u0u_0 by A100A^{100} to find u100u_{100}

  • 100 steps from u0u_0 u100=(λ1)100x1−(λ2)100x2λ1−λ2u_{100}=\frac{(\lambda_1)^{100}x_1-(\lambda_2)^{100}x_2}{\lambda_1-\lambda_2}
  • 우리는 F100= second componets of u100F_{100}= \text{ second componets of } u_{100} 를 원해
  • x1x_1과 x2x_2의 second component는 1이야
  • λ1−λ2{\lambda_1-\lambda_2}은 5\sqrt{5}
  • 결국 F100=15[(1+52)100−(1−52)100]≈3.54⋅1020F_{100}=\frac{1}{\sqrt{5}}\bigg[\bigg(\frac{1+\sqrt{5}}{2} \bigg)^{100} - \bigg(\frac{1-\sqrt{5}}{2} \bigg)^{100}\bigg] \approx 3.54\cdot10^{20}
  • The fractions and square roots must disappear, because Fibonacci's rule Fk+2=Fk+l+FkF_{k+2} = F_{k+l} + F_k stays with integers
  • k번째 피보나치 수=λ1k−λ2kλ1−λ2\frac{\lambda_{1}^{k}-\lambda_{2}^{k}}{\lambda_1-\lambda_2} = 가장 가까운 상수는 15(1+52)k\frac{1}{\sqrt{5}}\bigg(\frac{1+\sqrt{5}}{2} \bigg)^{k}

The ratio FlOd F100 must be very close to the limiting ratio (I + 0) /2. The Greeks called this number the "golden mean".

Matrix Powers AkA^k

Power of A: AKU0=(SΛS−1)…(SΛS−1)u0=SΛkS−1u0A^KU_0 = (S\Lambda S^{-1})\dots(S\Lambda S^{-1})u_0=S\Lambda^k S^{-1}u_0

  1. Write u0u_0 as a combination c1x1+⋯+cnxnc_1x_1+\dots+c_nx_n of the eigenvectors.
  2. Multuply eaxh eigenvector xix_i by (λi)k(\lambda_i)^k
  3. Add up the pieces ci(λi)kxic_i(\lambda_i)^kx_i to find the solution uk=Aku0u_k=A^ku_0
  • Solution for uk+1=Auku_{k+1}=Au_k uk=Aku0=c1(λ1)kx1+⋯+cn(λn)kxnu_k=A^ku_0=c_1(\lambda_1)^kx_1+\dots+c_n(\lambda_n)^kx_n

the eigenvectors in S lead to the c's in the combination u0=c1x1+⋯+cnxnu_0=c_1x_1+\dots+c_nx_n

step 1

  • u0=[x1…xn][c1⋮cn]u_0= \begin{bmatrix}{}\\ {x_1} & {\dots} & {x_n}\\{} \end{bmatrix}\begin{bmatrix}{c_1}\\{\vdots}\\{c_n} \end{bmatrix}. This says that u0=Scu_0=Sc
  • 상관계수는 c=S−1U0c=S^{-1}U_0

step 2

  • Multiply by Λk\Lambda^k

step 3

  • the final result uk=∑ci(λi)kxiu_k=\sum c_i(\lambda_i)^kx_iis the product of S and Λk\Lambda^k and S−1u0S^{-1}u_0
  • Aku0=SΛkS−1u0=SΛkc=[x1…xn][(λ1)k⋱(λn)k][c1⋮cn]A^ku_0 = S\Lambda^kS^{-1}u_0 = S\Lambda^kc = \begin{bmatrix}{}&{}&{}\\{x_1} & {\dots}&{x_n}\\{}&{}&{}\end{bmatrix} \begin{bmatrix}{(\lambda_1)^k}&{}&{}\\{}&{\ddots}&{}\\{}&{}&{(\lambda_n)^k}\end{bmatrix} \begin{bmatrix}{c_1}\\{\vdots}\\{c_n}\end{bmatrix}
ex 3)

고유벡터와 고유값을 가진 S와 람다집합일때, AKu0A^Ku_0를 계산해보자.

  • u0=(1,0)u_0=(1,0)
  • A=[1210] has λ1=2 and x1=[21]A=\begin{bmatrix}{1}&{2}\\{1}&{0}\end{bmatrix} \text{ has } \lambda_1=2 \text{ and } x_1=\begin{bmatrix}{2}\\{1}\end{bmatrix}
  • λ2=−1 and x2=[1−1]\lambda_2=-1 \text{ and } x_2=\begin{bmatrix}{1}\\{-1}\end{bmatrix}

Solution in three steps: u0=c1x1+c2x2u_0=c_1x_1+c_2x_2를 찾고 uk=c1(λ1)kx1+c2(λ2)kx2u_k=c_1(\lambda_1)^kx_1+c_2(\lambda_2)^kx_2도 찾자

  • step 1 u0=[10]=13[21]+13[1−1] so c1=c2=13u_0=\begin{bmatrix}{1}\\{0}\end{bmatrix} = \frac{1}{3}\begin{bmatrix}{2}\\{1}\end{bmatrix} + \frac{1}{3}\begin{bmatrix}{1}\\{-1}\end{bmatrix} \text{ so } c_1 = c_2=\frac{1}{3}
  • step 2 Multiply the two parts by (λ1)k=2k and (λ2)k=(−1)k(\lambda_1)^k=2^k \text{ and } (\lambda_2)^k=(-1)^k
  • step 3 Combine eigenvectors c1(λ1)kx1 and c2(λ2)kx2 into ukc_1(\lambda_1)^kx_1 \text{ and }c_2(\lambda_2)^kx_2 \text{ into } u_k
    • uk=Aku0u_k=A^ku_0 uk=132k[21]+13(−1)k[1−1]u_k=\frac{1}{3}2^k\begin{bmatrix}{2}\\{1}\end{bmatrix}+\frac{1}{3}(-1)^k\begin{bmatrix}{1}\\{-1}\end{bmatrix}

Nondiagonalizable Matrices (Optional)

람다가 A의 고유값일때 우리는 두가지를 알아냈지

  1. 기하학적으로 고유벡터; 0이 아닌nonzero 해결책은 Ax=λxAx=\lambda x
  2. 대수로 고유값; A−λIA-\lambda I 의 determinant는 0이야.

For exceptional matrices, an eigenvalue can be repeated. Then there are two different ways to count its multiplicity. Always OM < AM for each A: 고유값들이 반복되는 예외적인 경우에 중복도(?)를 세는 두가지 다른 방법이 있다.

  1. 기하학적 중복도 Geomatric Multiplicity=GM
    • λ\lambda에서 독립적인 고유벡터를 센다.이건 A−λIA-\lambda I의 nullspace의 차원이다.
  2. 대수적 중복도 Algebraic Multiplicity=AM
    • 고유값을 사용한 λ\lambda의 반복을 센다. det(A−λI)=0det(A-\lambda I)=0의 n개의 근을 본다.

λ=4,4,4\lambda=4,4,4이면 GM은 3이고 AM은 1 또는 2 또는 3임!!

Warning: AM=2, GM=1 일때, 고유값은 2개인데 고유벡터가 1개인 경우가 있다. 이는 A가 대각선화 할 수 없은AM평균 아래에 있을때 고유벡터의 단점이다.(?)When repeats = [1 1. .. 1] we know that the n eigenvalues are
all different and A is diagonalizable.
  • When repeats = [1 1. .. 1] we know that the n eigenvalues are all different and A is diagonalizable.
    • The sum of all components in "repeats" is always n, because every nth degree equation det(A−λI)=0det(A - \lambda I) = 0 has n roots (counting repetitions). The sum of all components in "repeats" is always nn, because every nthn^{th} degree equation det(A−λI)=0det(A - \lambda I) = 0 has nn roots (counting repetitions).
  • The total number of independent eigenvectors might be less than n. Then A is not diagonalizable.

Eigenvalues of A B and A + B

An eigenvalue λ\lambda of A times an eigenvalue β\beta of B usually does not give an eigenvalue of AB:

False proof: ABx=Aβx=βAx=βλxABx = A\beta x = \beta A x = \beta \lambda x.

  • 이건 베타를 곱한 람다가 고유값처럼 보인다.
  • x가 A,B의 고유벡터일때 이는 옳은 증명이다.
  • 하지만 항상 그렇지는 않지!
  • A의 고유벡터들은 일반적으로 B의 고유벡터들이 아니다.
  • 1이 AB의 고유값일때, A,B는 모두 0 고유값을 가질 수 있다.
  • 이 때문에 A+B의 고유값은 일반적으로 λ+β\lambda+\beta가 아니다.
  • 이 잘못된 증명은 위 사실을 보인다.
  • x가 진짜 A,B 모두의 고유벡터라면, ABx=λβxABx=\lambda \beta x와 BAx=βλxBAx= \beta \lambda x가 나와야 한다.
  • n개의 고유벡터가 공유된다면 고유값들을 곱할 수 있다.

Note: Commuting matrics share eigenvectpors: A와 B가 대각화할 수 있다고 가정할떄 AB=BAAB=BA일 경우에만 같은 고유벡터 행렬 S가 공유된다.

REVIEW OF THE KEY IDEAS

  1. If AA has nn independent eigenvectors x1…xnx_1 \dots x_n, they go into the columns of SS.
  • AA is diagonalized by SS S−1AS=ΛS^{-1}AS = \Lambda and A=SΛS−1A = S\Lambda S^{-1}.
  1. The powers of AA are Ak=SΛkS−1A^k = S\Lambda^kS^{-1}. The eigenvectors in SS are unchanged.
  2. The eigenvalues of AkA^k are (λ1)k,...,(λn)k(\lambda_1)^k, ... , (\lambda_n)^k in the matrix Λk\Lambda^k.
  3. The solution to uk+l=Auku_{k+l} = Au_k starting from u0u_0 is uk=Aku0=SΛkS−1u0u_k = A^ku_0 = S\Lambda^kS^{-1}u_0:
  • uk=c1(λ1)kx1+⋯+cn(λn)kxnu_k=c_1(\lambda_1)^kx_1+\dots+c_n(\lambda_n)^kx_n provided u0=c1x1+cnxnu_0 = c_1x_1 + c_nx_n
  • That shows Steps 1,2,3 (cc's from S−1uO,λkS^{-1} u_O , \lambda^k from Λk\Lambda^k, and xx's from SS)
  1. A is diagonalizable if every eigenvalue has enough eigenvectors (GM = AM).

WORKED EXAMPLES

Problem Set 6.2

6.3 Applications to Differential Equations

The whole point of the section is this: To convert constant-coefficient differential equations into linear algebra.

One equation dudt=λu\frac{du}{dt}=\lambda u has the solutions u(t)=Ceλtu(t) = Ce^{\lambda t}.

  • Linear algebra moves to n by n. The unknown is a vector u (now boldface).
  • 우리는 n exponentials eλtxe^{\lambda t}x in u(t)u(t)를 expect하지.

n equations dudt=Au\frac{du}{dt}=Au starting from the vector u(0)u(0) at t=0

  • These differential equations are linear. If u(t) and vet) are solutions, so is Cu(t)+Dv(t)C u(t) + Dv(t).
  • 우리가 가장 먼저 할 일: "pure exponential solutions" Ax=λxAx=\lambda x를 사용하여 u=eλtxu=e^{\lambda t}x 를찾는 것
  • Notice that A is a constant matrix.
  • dudt=Au\frac{du}{dt} = Au is "linear with constant coefficients"

The main point will be:Solve linear constant coefficient equations by exponentials eλtxe^{\lambda t}x, when Ax=λXAx = \lambda X.

Solution of du/dt = Au

Use u=eλtxu=e^{\lambda t}x when Ax=λxAx=\lambda x dudt=λeλtx\frac{du}{dt}=\lambda e^{\lambda t}x agrees with Au=AeλtxAu=Ae^{\lambda t}x

  • All components of this special solution u=eλtxu = e^{\lambda t}x share the same eλte^{\lambda t}. The solution grows when λ>O\lambda > O. It decays when λ<O\lambda < O. If λ\lambda is a complex number, its real part decides growth or decay. The imaginary part ww gives oscillation ei wt like a sine wave.
ex1)

Solve dudt=Au=[0110]u\frac{du}{dt}=Au=\begin{bmatrix}{0}&{1}\\{1}&{0}\end{bmatrix}u starting from u(0)=[42]u(0)=\begin{bmatrix}{4}\\{2}\end{bmatrix}

  • u에 대한 벡터방정식
  • y,z라는 스칼라를 포함

dudt=Au\frac{du}{dt}=Au, fracddt[yz]=[1110][yz]frac{d}{dt}\begin{bmatrix}{y}\\{z}\end{bmatrix} = \begin{bmatrix}{1}&{1}\\{1}&{0}\end{bmatrix} \begin{bmatrix}{y}\\{z}\end{bmatrix} means that dydt=z\frac{dy}{dt}=z and dzdt=y\frac{dz}{dt}=y

  • This matrix A has eigenvalues 1 and -1. The eigenvectors are (1, 1) and (1, -1).
  • u1(t)=eλ1tx1=et[11]u_1(t)=e^{\lambda_1 t}x_1=e^t \begin{bmatrix} {1}\\{1}\end{bmatrix} and u2(t)=eλ2tx2=e−t[1−1]u_2(t)=e^{\lambda_2 t}x_2=e^{-t} \begin{bmatrix} {1}\\{-1}\end{bmatrix}

Note: 위의 u는 고유벡터임!, et,e−te^t,e^{-t}들은 시간에 따라 변함!

To find all other solutions, multiply those special solutions by any C and D and add:

  • Complete solution
    • u(t)=Cet[11]+De−t[1−1]=[42]u(t) = Ce^t\begin{bmatrix}{1}\\{1}\end{bmatrix}+De^{-t}\begin{bmatrix}{1}\\{-1}\end{bmatrix} = \begin{bmatrix}{4}\\{2}\end{bmatrix} yields C=3C=3 and D=1D=1

The same three steps that solved Uk+l = AUk now solve dudt=Au:\frac{du}{dt} = Au:

  1. Write u(0)u (0) as a combination c1x1+...+cnxnc_1 x_1 + ... + c_n x_n of the eigenvectors of AA.
  2. Multiply each eigenvector xix_i by eλite^{\lambda_i t} .
  3. The solution is the combination of pure solutions eλtxe^{\lambda t}x:
    • u(t)=c1eλ1tx1+⋯+cneλntxnu(t)=c_1e^{\lambda_1 t}x_1+\dots+c_ne^{\lambda_n t}x_n

Warning: Not included: If two λ\lambda’S are equal, with only one eigenvector, another solution is needed. (It will be teλtxte^{\lambda t} x). Step 1 needs A=SΛS−1A = S\Lambda S^{-1} to be diagonalizable: a basis of eigenvectors.
ex2)

Solve dudt=Au\frac{du}{dt}=Au knowing tjhe eigenvalues λ=1,2,3\lambda=1,2,3 of A

dudt=[111022003]u starting from u(0)=[974]\frac{du}{dt}=\begin{bmatrix}{1}&{1}&{1}\\{0}&{2}&{2}\\{0}&{0}&{3} \end{bmatrix}u \text{ starting from } u(0)=\begin{bmatrix}{9}\\{7}\\{4} \end{bmatrix}

The eigenvectors are x1=(1,0,0)x_1 = (1,0,0) and x2=(1,1,0)x_2 = (1,1,0) and x3=(1,1,1)x_3 = (1,1,1).

  • Step 1 The vector u(0)=(9,7,4)u(0) = (9,7,4) is 2x1+3x2+4x32x_1 + 3x_2 + 4x_3. Thus (c1,c2,c3)=(2,3,4)c_1, c_2, c_3) = (2,3,4).
  • Step 2 The pure exponential solutions are etx1e^t x_1 and e2tx2e^{2t} x_2 and e3tx3e^{3t} x_3.
  • Step 3 The combination that starts from u(0)u(0) is u(t)=2etx1+3e2tx2+4e3tx3u(t) = 2e^{t}x_1 + 3e^{2t} x_2 + 4e^{3t} x_3.

The coefficients 2, 3, 4 came from solving the linear equation c1x1+c2x2+c3x3=u(0)c_1x_1 + c_2x_2 + c_3x_3 = u(0):

[x1x2x3][c1c2c3]=[111011001][234]=[974] which is Sc=u(0)\begin{bmatrix}{}&{}&{}\\{x_1}&{x_2}&{x_3}\\{}&{}&{} \end{bmatrix}\begin{bmatrix}{c_1}\\{c_2}\\{c_3} \end{bmatrix}=\begin{bmatrix}{1}&{1}&{1}\\{0}&{1}&{1}\\{0}&{0}{1} \end{bmatrix}\begin{bmatrix}{2}\\{3}\\{4} \end{bmatrix}=\begin{bmatrix}{9}\\{7}\\{4} \end{bmatrix} \text{ which is } Sc=u(0)

Second Order Equations

The most important equation in mechanics is my"+by′+ky=0my" +by' +ky = 0.

?

md2ydt2+bdydt+ky=0 becomes (mλ2+bλ+k)eλt=0m\frac{d^2 y}{dt^2}+b\frac{dy}{dt} + ky = 0 \text{ becomes } (m\lambda^2 + b\lambda + k) e^{\lambda t}=0

Everything depends on mλ2+bλ+k=0m\lambda^2 + b\lambda + k = 0. This equation for λ\lambda has two roots λ1\lambda_1 and λ2\lambda_2. Then the equation for yy has two pure solutions y1=eλ1ty_1 = e^{\lambda_1 t} and y2=eλ2ty_2 = e^{\lambda_2 t} . Their combinations c1y1+c2y2c_1 y_1 + c_2y_2 give the complete solution unless λ1=λ2\lambda_1 = \lambda_2.

u(t)=c1eλ1t[1λ1]+cweλ2t[1λ2]u(t)=c_1e^{\lambda_1 t}\begin{bmatrix}{1}\\{\lambda_1} \end{bmatrix}+c_we^{\lambda_2 t}\begin{bmatrix}{1}\\{\lambda_2} \end{bmatrix}

ex3)

Motion around a circle with y"+y=0y" + y = 0 and y=costy = cos t

Forward from n−1 Centeren at n Backward from n+1Yn+1−2Yn+Yn−1(Δt)2=−Yn−1−Yn−Yn+1\begin{matrix}{\text{Forward from }n-1}\\{\text{ Centeren at }n}\\{\text{ Backward from }n+1} \end{matrix}\frac{Y_{n+1}-2Y_n +Y_{n-1}}{(\Delta t)^2}=\begin{matrix}{-Y_{n-1}}\\{-Y_n}\\{-Y_{n+1}} \end{matrix}

  • 이 세가지 방법은 완전한 원을 구성하진 않아, Δt=2π32\Delta t = \frac{2\pi}{32} 길이의 32단계에서

Foward ∣λ∣>1 (spiral out) Centered∣λ∣=1(best) Backward ∣λ∣<1 (spiral in)\text{Foward }|\lambda|>1\text{ (spiral out) Centered} |\lambda|=1\text{(best) Backward } |\lambda|<1 \text{ (spiral in)}

Forward Yn+1=Yn+ΔtZnZn+1=Zn−ΔtYn becomes Un+1=[1Δt−Δt1][YnAn]=AUn\text{Forward }\begin{matrix}{Y_{n+1}=Y_n+\Delta tZ_n}\\{Z_{n+1}=Z_n-\Delta tY_n}\end{matrix}\text{ becomes } U_{n+1}=\begin{bmatrix}{1}&{\Delta t}\\{-\Delta t}&{1} \end{bmatrix}\begin{bmatrix}{Y_n}\\{A_n} \end{bmatrix}=AU_n

Eigenvalues of A λ=1±iΔt ∣λ∣>1 and (Yn,Zn) spirals out\text{Eigenvalues of A } \lambda = 1 \pm i\Delta t \text{ } |\lambda | >1 \text{ and } (Y_n, Z_n) \text{ spirals out}

$$\text{Backward }\begin{matrix} {Y_{n+1}=Y_n+\Delta t Z_{n+1}}\\{Z_{n+1}=Z_n-\Delta t T_{n+1}}\end{matrix} \text{ is } \begin{bmatrix}{1} &{-\Delta t}\\{\Delta}&{1}\end{bmatrix}\begin{bmatrix}{Y_{n+1}\\{Z_{n+1}}}\end{bmatrix}=\begin{bmatrix}{Y_n}\\{Z_n}\end{bmatrix}=U_n$$

Stability of 2 by 2 Matrices

  • complete solution u(t)u(t) 는 eλte^{\lambda t}로 만들어짐!
  • 만약 고유값 람다가 있다면, eλte^{\lambda t}가 0에 가까워지면 그땐 정확하게 람다가 음수여야 한다는 것을 알지.
  • 만약 고유값이 복소수 λ=r+is\lambda = r+is라면 실제 부분 r은 음수여야 하지.

eist=cos⁡st+isin⁡st has ∣eist∣2=cos⁡2st+sin⁡2st=1e^{ist} =\cos st + i \sin st \text{ has }|e^{ist}|^2=\cos ^2 st+\sin ^2 st =1

Which matrices have negative eigenvalues? More accurately, when are the real parts of the λ\lambda's all negative?

Stability; a is stable and u(t)→0 when all eigenvalues have negative real parts.\text{Stability; a is stable and } u(t)\rightarrow 0 \text{ when all eigenvalues have negative real parts.}

The 2 by 2 matrix A=[abcd]must pass two tests:\text{The 2 by 2 matrix } A=\begin{bmatrix}{a}&{b}\\{c}&{d}\end{bmatrix} \text{must pass two tests:}

λ1+λ2<0λ1λ2>0The trace T=a+dmust be negativeThe determinant D=ad−bc must be positive\begin{matrix}{\lambda_1+\lambda_2<0}\\{\lambda_1\lambda_2 >0}\end{matrix} \begin{matrix}{\text{The trace } T=a+d \text{must be negative}}\\{\text{The determinant } D=ad-bc \text{ must be positive}}\end{matrix}

  • 람다들이 음수로 존재하면 합은 음수일테니까, 그 합이 trace T이고. 그럼 곱은 양수가 되겠지. 이게 determinant D야.
  • 람다들이 복소수라면 r±isr\pm is로부터 얻을 수 있겠지, 그렇지 않으면 T,D는 얻을 수 없어

The Exponential of a Matrix

We want to write the solution u(t)u(t) in a new form eAtu(0)e^{At} u(0).

Matrix exponentil eAt eAt=IAt+12(At)2+16(At)3+…\text{Matrix exponentil }e^{At} \text{ } e^{At} = I At +\frac{1}{2}(At)^2 + \frac{1}{6}(At)^3+\dots

Its derivative is AeAt A+A2+12A3t2+⋯=AeAt\text{Its derivative is } Ae^{At} \text{ }A+A^2+\frac{1}{2}A^3t^2+\dots = Ae^{At}

Its eigenvalues are eλr (I+At+12(At)2)x=(1+λt+12(λt)2+… )x\text{Its eigenvalues are } e^{\lambda r}\text{ }(I+At +\frac{1}{2}(At)^2)x=(1+\lambda t +\frac{1}{2}(\lambda t)^2+\dots)x

This chapter emphasizes how to findu(t)=eAtu(0) u(t) = e^{At} u(0) by diagonalization. Assume A does have nn independent eigenvectors, so it is diagonalizable. Substitute A=SΛS−1A = S\Lambda S^{-1} into the series for eAte^{At}. Whenever ΛS−1SΛS−1\Lambda S^{-1} S\Lambda S^{-1} appears, cancel S−1SS^{-1} S in the middle:

Use the series eAt=I+SΛS−1t+12(SΛS−1t)(SΛS−1t)+…\text{Use the series } e^{At}=I+S\Lambda S^{-1}t +\frac{1}{2}(S\Lambda S^{-1}t)(S\Lambda S^{-1}t)+\dots

Factor out S and S−1 =S[I+Λt+12(Λt)2+… ]S−1\text{Factor out } S \text{ and } S^{-1} \text{ } = S[I+\Lambda t +\frac{1}{2}(\Lambda t)^2+\dots]S^{-1}

Diagonalize eAt =SeΛtS−1\text{Diagonalize } e^{At} \text{ } = Se^{\Lambda t}S^{-1}

eAtu(0)=SeΛtS−1u(0)=[x1…xn][eλ1t⋱eλ)nt][c1⋮cn]e^{At}u(0) = Se^{\Lambda t}S^{-1}u(0)=\begin{bmatrix}{}\\{x_1}&{\dots}&{x_n}\\{}\end{bmatrix}\begin{bmatrix}{e^{\lambda_1t}}&{}&{}\\{}&{\ddots}&{}\\{}&{}&{e^{\lambda)n t}}\end{bmatrix}\begin{bmatrix}{c_1}\\{\vdots}\\{c_n}\end{bmatrix}

eAtu(0)e^{At}u(0)의 solution

  1. u(0)=c1x1+⋯+cnxnu(0)=c_1x_1+\dots+c_nx_n을 쓰자, n개의 독립적인 고유벡터가 필요하다.
  2. eλite^{\lambda_i t}에 각 xix_i를 곱하자
  3. eAtu(0)e^{At}u(0)의 best form은 u(t)=c1eλ1tx1+⋯+cneλntxnu(t)=c_1e^{\lambda_1 t}x_1+\dots+c_ne^{\lambda_n t}x_n
ex4)

When you substitute y=eλty = e^{\lambda t} into y"−2y′+y=0y" - 2y' + y = 0, you get an equation with repeated roots: λ2−2λ+1=0=(λ−1)\lambda^2 - 2\lambda + 1 = 0 = (\lambda-1).

Short series eAt=eIteP−It=et[I+(A−I)t\text{Short series }e^{At}=e^{It}e^P-I{t=e^t[I+(A-I)t}

u(t)=et[I+[−11−11]t]u(0)u(t)=e^t\begin{bmatrix}I+\begin{bmatrix}{-1}&{1}\\{-1}&{1}\end{bmatrix} t \end{bmatrix}u(0)

y(t)=ety(0)−tety(0)+tety[0]y(t) = e^ty(0) -te^ty(0) +te^t y[0]

ex5)

Use the infinite series to find eAte^{At} for A=[01−11]A=\begin{bmatrix}{0}&{1}\\{-1}&{1}\end{bmatrix}. Notice that A4=IA^4=I

A=[01−11]A=\begin{bmatrix}{0}&{1}\\{-1}&{1}\end{bmatrix}

eAt=[cos⁡tsin⁡tsin⁡tcos⁡t]e^{At}=\begin{bmatrix}{\cos t}&{\sin t}\\{\sin t}&{\cos t}\end{bmatrix}

The eigenvalues of eAt are e it and e-it . Three rules:

  1. eAte^{At} always has the inverse e−Ae^{-A}t .
  2. The eigenvalues of eAte^{At} are always eλte^{\lambda t}.
  3. When A is skew-symmetric, eAte^{At} is orthogonal. Inverse = transpose = $e^{-At}.
ex6)

Solve

dudt=Au=[1102]u\frac{du}{dt}=Au=\begin{bmatrix}{1}&{1}\\{0}&{2}\end{bmatrix}u starting from u(0)=[21]u(0)=\begin{bmatrix}{2}\\{1}\end{bmatrix} at t=0

Solution

u(t)=et[10]+e2t[11]u(t)=e^t\begin{bmatrix}{1}\\{0}\end{bmatrix}+e^{2t}\begin{bmatrix}{1}\\{1}\end{bmatrix}

REVIEW OF THE KEY IDEAS

WORKED EXAMPLES

Problem Set 6.3

6.4 Symmetric Matrices

I have to write "can be chosen" because the two in the plane are not automatically perpendicular.

This section makes that best possible choice for symmetric matrices: The eigenvectors of P=PTP = P^T are perpendicular unit vectors.

What is special about Ax=λxA x = \lambda x when A is symmetric?

  • A와 trsnspose한 A가 같을 때 고유값 람다와 고유벡터 엑스의 특징을 볼거야

diagonalization A=SΛS−1A=S\Lambda S^{-1}는 A의 대칭을 반영할걸, AT=(S−1)TΛSTA^T=(S^{-1})^T \Lambda S^{T}에서 힌트를 얻어볼까!

  • A와 trsnpose A가 같다 했으니까.
  • S=S−1S = S^{-1} 겠네?
    • That makes each eigenvector in S orthogonal to the other eigenvectors.
    • S의 각 고유 벡터를 다르 고유벡터와 직교하게 만듦!

key fact

  1. Asymmetric matrix has only real eigenvalues.
  2. The eigenvectors can be chosen orthonormal.

Its eigenvector matrix SS becomes an orthogonal matrix QQ. Orthogonal matrices have Q−l=QTQ^{-l} = Q^T

Note: 직교한 고유벡터를 선택할 떄 S = Q 라 쓴다는 것을 기억하자

Why do we use the word "choose"? Because the eigenvectors do not have to be unit vectors.

Their lengths are at our disposal. We will choose unit vectors-eigenvectors of length one, which are orthnormal and not just orthogonal.

Then SΛS−1S\Lambda S^{-1} is in its special and particular form QΛQTQ\Lambda Q^T for symmetric matrices:

(Special Theorem) 모든 대칭 행렬은 람다 행렬의 고유값들과 S=QS=Q 인 직교 고유벡터와 같이 factorization A=QΛQTA=Q\Lambda Q^T을 가지고 있음

Symmetric diagonalixation A=QΛQ−1=QΛQT with Q−1=QT\text{Symmetric diagonalixation } A=Q\Lambda Q^{-1}=Q\Lambda Q^T \text{ with } Q^{-1} = Q^T

This is the "spectral theorem" in mathematics and the "principal axis theorem" in geometry and physics.

  1. By an example, showing real Λ\Lambda's in A and orthonormal xx's in QQ.
  2. By a proof of those facts when no eigenvalues are repeated.
  3. By a proof that allows repeated eigenvalues (at the end of this section).
ex1)

Find the λ\lambda's and xx's when A=[1224]A=\begin{bmatrix}{1}&{2}\\{2}&{4}\end{bmatrix} and A−λI=[1−λ224−λ]A-\lambda I = \begin{bmatrix}{1-\lambda}&{2}\\{2}&{4-\lambda}\end{bmatrix}

Solution

  • determinant of A−λIA-\lambda I인 λ2−5λ\lambda^2-5\lambda로 고유값 0,5를 찾을 수 있지
  • 두 고유 벡터는 (2,-1),(1,2)야.(orthogonal but not yet orthonormal.)
  • 람다가 0일때 고유벡터는 A의 nullspace안에 있어.
  • 람다가 5일때 고유벡터는 열공간column space 안에 있어.
  • why are the nullspace and column space perpendicular?
    • The Fundamental Theorem says that the nullspace is perpendicular to the row space-not the column space.
  • 하지만 우리가 보는 행령은 대칭이잖아!
  • 그래서 행과 열 공간은 같지.

These eigenvectors have length 5\sqrt{5}. Divide them by 5\sqrt{5} to get unit vectors.

Put those into the columns of SS (which is QQ). Then Q−1AQQ^{-1} AQ is Λ\Lambda and Q−1=QTQ^{-1} = Q^T:

Q−1AQ=15[2−112][1224]15[21−12]=[0005]=ΛQ^{-1}AQ = \frac{1}{\sqrt{5}}\begin{bmatrix}{2}&{-1}\\{1}&{2}\end{bmatrix} \begin{bmatrix} {1}&{2}\\{2}&{4} \end{bmatrix} \frac{1}{\sqrt{5}} \begin{bmatrix} {2}&{1}\\{-1}&{2} \end{bmatrix}=\begin{bmatrix}{0}&{0}\\{0}&{5}\end{bmatrix}=\Lambda

Now comes the nn by nn case. The λ\lambda's are real when A=ATA = A^T and Ax=λxAx = \lambda x .

Real Eigenvalues; All the eigen values of a real symmetric matrix are real

Orthogonal Eigenvectors; Eigenvectors of a real symmetic matrix (when they correspond to different $\lambda|'s)are always perpendeicular.

(λ1x)Ty=(Ax)Ty=xTATy=xTAy=xTλ2y(\lambda_1x)^Ty = (Ax)^Ty=x^TA^Ty = x^TAy=x^T\lambda_2y

ex2)

The eigenvectors of a 2 by 2 symmetric matrix have a special form:

Not widely known A=[abbc]A=\begin{bmatrix}{a}&{b}\\{b}&{c}\end{bmatrix} has x1=[bλ1−a]x_1=\begin{bmatrix}{b}\\{\lambda_1-a}\end{bmatrix} and x2=[λ2−cb]x_2=\begin{bmatrix}{\lambda_2-c}\\{b}\end{bmatrix}

x1Tx2=b(λ2−c)+(λ1−a)b=b(λ1+λ2−a−c)=0x_{1}^{T}x_2=b(\lambda_2-c)+(\lambda_1-a)b=b(\lambda_1+\lambda_2-a-c)=0

  • λ1+λ2\lambda_1+\lambda_2가 trace a+c와 같기 떄문에 0이다~
  • x1Tx2=0x_{1}^{T}x_2=0이 되겠지?
  • x1=x2=0x_1=x_2=0일때, a=c,b=0
    • 이 경우 A=I에서 고유값이 반복됨

This example shows the main goal of this section-to diagonalize symmetric matrices A by orthogonal eigenvector matrices S=QS = Q. Look again at the result: Symmetry A=SΛS−1 becomes A=QΛQT with QTQ=I\text{Symmetry } A=S\Lambda S^{-1} \text{ becomes } A=Q\Lambda Q^T \text{ with } Q^TQ=I

이는 모든 2 by 2 행렬이 이렇게 생겼다는 것을 의미 A=QΛQT=[x1x2][λ1λ2][x1Tx2T]A=Q\Lambda Q^T = \begin{bmatrix}{}\\{x_1}&{x_2}\\{}\end{bmatrix} \begin{bmatrix} {\lambda_1}&{}\\{}&{\lambda_2}\end{bmatrix} \begin{bmatrix}{}&{x^{T}_{1}}&{}\\{}&{x_{2}^{T}}&{} \end{bmatrix}

Sum of ranl-one matrics A=λ1x1x1T+λ2x2x2T\text{Sum of ranl-one matrics }A=\lambda_1x_1x_{1}^{T}+\lambda_2x_2x_{2}^{T}

  • xixiTx_ix_{i}^{T}를 n번 곱하면 projection matrices임.
  • Including the A's, the spectral theorem A = QAQT for symmetric matrices says that A is a combination of projection matrices: A=λ1P1+⋯+λnPn λi= eigenvalue, Pi=projection onto eigenspaceA=\lambda_1P_1+\dots+\lambda_nP_n \text{ } \lambda_i=\text{ eigenvalue, } P_i=\text{projection onto eigenspace}

Complex Eigenvalues of Real Matrices

For real matrices, complex λ\lambda's and xx's come in "conjugate pairs." If Ax=λx then Axˉ=λˉxˉ\text{If }Ax=\lambda x \text{ then } A\bar{x}=\bar{\lambda}\bar{x}

ex3)

A=[cos⁡θ−sin⁡θsin⁡θcos⁡θ] has λ1=cos⁡θ+isin⁡θ and λ2cos⁡θ−isin⁡θA=\begin{bmatrix}{\cos \theta} &{-\sin \theta}\\{\sin \theta} &{\cos \theta} \end{bmatrix} \text{ has } \lambda_1=\cos \theta + i \sin \theta \text{ and } \lambda_2 \cos \theta -i\sin \theta

This is λx Ax=[cos⁡θ−sin⁡θsin⁡θcos⁡θ][1−i]=(cos⁡θ+isin⁡θ)[1−i])\text{This is } \lambda x \text{ } Ax=\begin{bmatrix} {\cos \theta}&{-\sin \theta}\\{\sin \theta}&{\cos \theta}\end{bmatrix}\begin{bmatrix}{1}\\{-i}\end{bmatrix}=(\cos \theta + i \sin \theta) \begin{bmatrix}{1}\\{-i}\end{bmatrix})

This is λˉxˉ Axˉ=[cos⁡θ−sin⁡θsin⁡θcos⁡θ][1i]=(cos⁡θ−isin⁡θ)[1i])\text{This is } \bar{\lambda}\bar{x} \text{ } A\bar{x}=\begin{bmatrix} {\cos \theta}&{-\sin \theta}\\{\sin \theta}&{\cos \theta}\end{bmatrix}\begin{bmatrix}{1}\\{i}\end{bmatrix}=(\cos \theta - i \sin \theta) \begin{bmatrix}{1}\\{i}\end{bmatrix})

For this rotation matrix the absolute value is ∣λ∣=1|\lambda|=1, because ∣cos2θ+sin⁡2θ=1|cos^2 \theta + \sin^2 \theta = 1. This fact ∣λ∣=1|\lambda| = 1 holds for the eigenvalues of every orthogonal matrix.

Eigenvalues versus Pivots

productofpivots=determinant=productofeigenvalues.product of pivots = determinant = product of eigenvalues.

We are assuming a full set of pivots d 1, ... , dn. There are n real eigenvalues AI, ... , An. The d's and A'S are not the same, but they come from the same matrix. This paragraph is about a hidden relation. For symmetric matrices the pivots and the eigenvalues have the same signs:

  • The number of positive eigenvalues of A=ATA = A^T equals the number of positive pivots. Special case: AA has all λi>0\lambda_i > 0 if and only if all pivots are positive.
ex4)

The signs of the pivots match the signs of the eigenvalues, one plus and one minus. This could be false when the matrix is not symmetric.

  • 대칭 행렬에서 pivot이 둘 중 한 개만 양수라면 고유값도 둘 중 한 개만 양수
  • 비대칭행렬에서는 pivot이 둘 다 양수여도 고유값은 아닐 수 있으니 주의!
  • the pivots and eigenvalues have matching signs,
  • But to change sign, a real eigenvalue would have to cross zero.
  • The matrix would at that moment be singular

All Symmetric Matrices are Diagonalizable

  • 어느 A의 고유값도 반복되지 않을 때, 고유벡터는 독립이 확실하다.
  • 그러면 A는 대각화될 수 있다.
  • 하지만 반복된 고유값은 고유벡터의 단점을 가질 수 있음
  • 이건 비대칭 행렬에대해 일어나곤 함.
  • 대칭 행렬에서는 절대 안 일어나는 일
  • There are always enough eigenvectors to diagonalize A=ATA = A^T.

Here is one idea for a proof. Change A slightly by a diagonal matrix diag(c,2c,...,nc)diag( c , 2c, ... , n c). If c is very small, the new symmetric matrix will have no repeated eigenvalues. Then we know it has a full set of orthonormal eigenvectors. As c→0c \rightarrow 0 we obtain n orthonormal eigenvectors of the original A-even if some eigenvalues of that A are repeated.

Schur's Theorem

  • Every square matrixfactors into A=QTQ−1A=QTQ^{-1} where TT is upper triangular and QˉT=Q−l.\bar{Q}^{T} =Q-^{l.} If A has real eigenvalues then QQ and TT can be chosen real: QTQ=IQ^T Q = I
  • 임의의 복소수 정사각 행렬을 상삼각 행렬로 나타내는 행렬 분해
  • ?

REVIEW OF THE KEY IDEAS

WORKED EXAMPLES

Problem Set 6.4

Challenge Problems

6.5 Positive Definite Matrices

This section concentrates on symmetric matrices that have positive eigenvalues.

Symmetric matrices with positive eigenvalues are at the center of all kinds of applications. They are called positive definite.

Here are two goals of this section:

  • To find quick tests on a symmetric matrix that guarantee positive eigenvalues.
  • To explain important applications of positive definiteness.

Start with 2 by 2 When does A=[abbc] have λ1>0 and λ2>0?\text{Start with 2 by 2 When does } A=\begin{bmatrix}{a}&{b}\\{b}&{c}\end{bmatrix} \text{ have } \lambda_1>0 \text{ and } \lambda_2>0?

The eigenvalues of A are positive if and only if a>0 and ac−b2>0\text{The eigenvalues of A are positive if and only if } a>0 \text{ and } ac-b^2>0

Proof that the 2 by itest is passed when λl>0\lambda_l > 0 and ∣lambda2>O|lambda_2 > O. Their product λ1λ2\lambda_1 \lambda_2 is the determinant so ac−b2>Oac - b^2 > O. Their sum is the trace so a+c>Oa + c > O. Then aa and cc are both positive (if one of them is not positive, ac−b2>0ac - b^2 > 0 will fail).

The eigenvalues of A=AT are positive if and only if the pivots are positive: a>0 and ac−b2a>0\text{The eigenvalues of } A = A^T \text{ are positive if and only if the pivots are positive: } a>0 \text{ and } \frac{ac-b^2}{a}>0

  • a>0a > 0은 위 두 test에서 모두 요구하는 조건!!
  • 그래서 ac>b2ac>b^2도 요구됨!

[1bbc]The first pivot is a⟶The multiplier is b/a[ab0c−bab]The second pivot isc−b2a=ac−b2a\begin{bmatrix}{1}&{b}\\{b}&{c}\end{bmatrix} \begin{matrix}{\text{The first pivot is a}}\\{\longrightarrow}\\{\text{The multiplier is b/a}}\end{matrix}\begin{bmatrix}{a}&{b}\\{0}&{c-\frac{b}{a}b}\end{bmatrix}\begin{matrix}{\text{The second pivot is}}\\{c-\frac{b^2}{a}=\frac{ac-b^2}{a}}\end{matrix}

Positive eigenvalues mean positive pivots and vice versa. 양수인 고유겂은 양수인 피벗을 의미하고 그 반대도 마찬가지!

Energy-based Definition

The new idea is that xTAxx^T A x is positive for all nonzero vectors x, not just the eigenvectors. In many applications this number xTAx(or12xTAx)x^T Ax (or \frac{1}{2}x^TAx) is the energy in the system. The requirement of positive energy gives another definition of a positive definite matrix. I think this energy-based definition is the fundamental one.

Definition; A is positive definite if xTAx>0x^TAx>0 for every nonzero vector x xTAx=[xy][abbc][xy]=ax2+2bxy+cy2>0x^TAx = \begin{bmatrix}{x}&{y}\end{bmatrix}\begin{bmatrix}{a}&{b}\\{b}&{c}\end{bmatrix}\begin{bmatrix}{x}&{y}\end{bmatrix}=ax^2 + 2bxy + cy^2 >0

This energy-based definition leads to a basic fact: If A and B are symmetric positive definite, so is A + B

  • Reason: The pivots and eigenvalues are not easy to follow when matrices are added, but the energies just add

If the columns of R are independent, then A=RTRA = R^T R is positive definite.

  • 여기서 A=RTRA = R^T R은 square 이고 symmetric임

The number xTAxx^T Ax is the same as xTRTRxx^T R^T Rx. That is exactly (Rx)T(Rx)(Rx)^T(Rx)-another important proof by parenthesis! That vector RxRx is not zero when x≠0x \neq 0 (this is the meaning of independent columns). Then xTAxx^TAx is the positive number ∣∣Rx∣∣2||Rx||^2 and the matrix A is positive definite.

When a symmetric matrix has one of these five properties, it has them all:

  1. All n pivots are positive.
  2. All n upper left determinants are positive
    • 1 by 1, w by 2,... , n by n
  3. All n eigenvalues are positive.
  4. xTAxx^TAx is positive except at x = 0 This is the energy-based definition.
  5. A equals RTRR^TR for a matrix R with independent columns.
ex1)

A,B가 positive definiteness일때, A=[2−10−12−10−12]and,B=[2−1b−12−1b−12]A=\begin{bmatrix}{2}&{-1}&{0}\\{-1}&{2}&{-1}\\{0}&{-1}&{2}\end{bmatrix} and, B=\begin{bmatrix}{2}&{-1}&{b}\\{-1}&{2}&{-1}\\{b}&{-1}&{2}\end{bmatrix}

  • A의 pivot은 2, 3/2, 4/3 임! 모두 양수!
  • 이것의 upper left determinants는 2,3,4이고 모두 양수!
  • A 고유값은 2−2,2,2+22-\sqrt{2},2,2+\sqrt{2}로 모두 양수!
  • xTAxx^T Ax도 양수! xTAx=2(x12−x1x2+x22−x2x3+x32=2(x1−12x2)2+32(x2−23x3)2+43(x3)2x^T Ax=2(x_{1}^{2}-x_1x_2+x_{2}^{2}-x_2x_3+x_{3}^{2}=2(x_1-\frac{1}{2}x_2)^2+\frac{3}{2}(x_2-\frac{2}{3}x_3)^2+\frac{4}{3}(x_3)^2

  • I have two candidates to suggest for RR. Either one will show that A=RTA = R^T R is positive definite. RR can be a rectangular first difference matrix, 4 by 3, to produce those second differences -1,2, -1 in AA

    • 여기서 R의 세 개 열은 모두 독립이고, A는 positive definite이다.

  • Another RR comes from A=LDLTA = LDL^T (the symmetric version of A=LUA = LU). Elimination gives the pivots 2, 3/2, 4/3 in DD and the multipliers -1/2, 0, -2/3 in L. Just put (D)\sqrt(D) with LL. LDLT=(LD)(LD)2=RTRLDL^T=(L\sqrt{D})(L\sqrt{D})^2=R^TR

    • 여기서 R은 Cholesky factor!!
    • 콜레스키 분해는 MATLAB에서 R=chol(A)R=chol(A)로 계산된다.
    • 정사각 R은 어떻게 A웅 만드는 방법이고, 이 콜레스키 R은 우리가 분해하는 방법이다.In applications, the rectangular R is how we build A and this Cholesky R is how we break it apart.

고유값은 대칭이 R=QΛQTR=Q\sqrt{\Lambda}Q^T를 선택하게 한다. 또한 RTR=QΛQTR^TR=Q\Lambda Q^T도 성공적!

Now tum to B, where the (1,3) and (3,1) entries move away from 0 to b.

  • b는 너무 크면 안 된다.
  • The determinant test은 가장 쉽다.
  • 1 by 1 determinant 은 2이고, 2 by 2 determinant은 여전히 3이다. 3 by 3 determinant는 0을 포함한다.
  • detB=4+2b−2b2=(1+b)(4−2b)det B = 4+2b-2b^2 = (1 +b)(4-2b)must be positive.

Positive Semidefinite Matrices

  • Often we are at the edge of positive definiteness.
  • The determinant is zero.
  • The smallest eigenvalue is zero.
  • The energy in its eigenvector is xTAx=xTOX=Ox^T Ax = x^TOX = O.
  • These matrices on the edge are called positive semidefinite.
  • Positive semidefinite matrices have all λ>0\lambda > 0 and all xTAx>Ox^T Ax > O.
  • Those weak inequalities (> instead of > ) include positive definite matrices and the singular matrices at the edge.

First Application: The Ellipse ax2+2bxy+cy2=1ax^2 + 2bxy + cy^2 = 1

A=QΛQ−l=QΛQTA = Q\Lambda Q^{-l} = Q\Lambda Q^T:

  1. The tilted ellipse is associated with A. Its equation is xTAx=1x^T Ax = 1.
  2. The lined-up ellipse is associated with Λ\Lambda. Its equation is XTΛX=1X^T \Lambda X = 1.
  3. The rotation matrix that lines up the ellipse is the eigenvector matrix QQ.
ex2)

Find the axes of this tilted ellipse 5x2+8xy+5y2=15x^2 + 8xy + 5y^2 = 1.

Solution

  • Start with the positive definite matrix that matches this equation:
  • 방정식은 [xy][5445][xy]=1\begin{bmatrix} {x}&{y} \end{bmatrix} \begin{bmatrix} {5}&{4}\\{4}&{5} \end{bmatrix} \begin{bmatrix} {x}\\{y} \end{bmatrix} = 1
  • 행렬은 A=[5445]A=\begin{bmatrix}{5}&{4}\\{4}&{5}\end{bmatrix}
  • 고유값은 9과 1
  • 고유벡터는 [11]\begin{bmatrix}{1}\\{1}\end{bmatrix}랑 [1−1]\begin{bmatrix}{1}\\{-1}\end{bmatrix}
  • 단위 벡터에 대해 2\sqrt{2}로 나누면 A=QΛQTA=Q\Lambda Q^T
  • xTAx=5x2+8xy+5y2=9(x+y2)2+1(x−y2)2x^TAx = 5x^2 + 8xy + 5y^2 = 9 \big( \frac{x+y}{\sqrt{2}} \big)^2 + 1 \big( \frac{x-y}{\sqrt{2}} \big)^2
  • 상관계수는 9와 1임(pivot 5와 9/5가 아니라)

The axes of the tilted ellipse point along the eigenvectors.

  • This explains why A=QΛQTA = Q\Lambda Q^T is called the "principal axis theorem"-it displays the axes.
  • Not only the axis directions (from the eigenvectors) but also the axis lengths (from the eigenvalues).
  • To see it all, use capital letters for the new coordinates that line up the ellipse:
    • x+y2=X\frac{x+y}{\sqrt{2}}=X and x−y2=Y\frac{x-y}{\sqrt{2}}=Y and 9X2+Y2=19X^2+Y^2=1
  • The bigger eigenvalue λ1\lambda_1 gives the shorter axis, of half-length 1/λ11 /\sqrt{\lambda_1} = 1/3.
  • The smaller eigenvalue λ2=1\lambda_2 = 1 gives the greater length 1/λ1=1.1/\sqrt{\lambda_1} = 1.
  • In the xy system, the axes are along the eigenvectors of A.
  • In the XY system, the axes are along the eigenvectors of A-the coordinate axes
  • All comes from A=QΛQTA = Q\Lambda Q^T.

REVIEW OF THE KEY IDEAS

WORKED EXAMPLES

Problem Set 6.5

Challenge Problems

6.6 Similar Matrices

The key step in this chapter is to diagonalize a matrix by using its eigenvectors.

  • SS는 고유벡터 행렬
  • S−1ASS^{-1}AS는 Λ\Lambda(고유값 행렬)
  • 하지만 대각화는 모든 A에 대해 가능하진 않음
  • 어떤 행렬은 고유벡터가 너무 작게 가지고 있기도 함
  • 고유벡터 행렬 S는 위의 상황을 해결하기 위한 best choice이지만 지금 우리는 어느 역행렬 M이든 허용함(?) In this new section, the eigenvector matrix S remains the best choice when we can find it, but now we allow any invertible matrix M
  • 어떤 M을 선택하든지간에 고유값은 같음!

DEFINITION; Let M be all y invertible.matrlx. Then B=M−1AM is similar to A.\text{DEFINITION; Let M be all y invertible.matrlx. Then }B = M^{- 1} AM \text{ is similar to }A.

If B=M−1AMB = M^{ -1} A M then immediately A=MBM−1A = M B M^{ -1}. That means: If BB is similar to AA then AA is similar to BB.

대각화할 수 있는 행렬음 Λ\Lambda와 유사함. In that special case M is S.

We have A=SΛS−1A = S\Lambda ^{S-1} and Λ=S−1AS\Lambda = S^{-1} AS. They certainly have the same eigenvalues!

M−1AMM^{-1}AM은 differential equation에서 변수에 변화를 줄 떄 나타난다.

  • u=Mvu=Mv라고 하면,
  • dudt=Au\frac{du}{dt}=Au 는 Mdvdt=AMvM\frac{dv}{dt}=AMv가 되고, 그건 dvdy=M−1amV\frac{dv}{dy}=M^{-1}amV임
  • 원래 상관계수 행렬은 A였는데 새로운 상관계수 행렬은 M−1AMM^{-1}AM임
  • u가 v로 변하는 것은 유사한 행렬으로 이끄는 것과 같음
  • 할상 u로 돌아갈 수 있기 때문에,
  • 유사한 행렬은 같기 growth되거나 decay 해야한다. 더 자세하게 말하자면,A와 B의 고유값은 같다.

(No .change in Λ\Lambda's) Similar matrices A and M−1M^{-1} have the same eigenvalues. If x is an eigenvector of A, M−1xM^{-1}x is an eigenvector of =M−1AM=M^{-1}AM.

  • The proof is quick, since B=M−1AMB = M^{-1} AM gives A=MBM−1A = MBM^{- 1}. Suppose Ax=λXAx = \lambda X:
    • MBM−1x=λxMBM^{-1}x=\lambda x means that B(M−1x)=λ(M−1x)B(M^{-1}x)=\lambda(M^{-1}x)

The eigenvalue of BB is the same λ\lambda.

The eigenvector has changed to M−1xM^{-1}x.

Two matrices can have the same repeated λ\lambda, and fail to be similar-as we will see

ex1)

These matrices M−1AMM^{-1}AM all have the same eigenvalues 1 and 0. The projection A=[.5.5.5.5] is similar to Λ=S−1AS=[1000]\text{The projection } A = \begin{bmatrix}{.5}&{.5}\\{.5}&{.5}\end{bmatrix} \text{ is similar to }\Lambda = S^{-1}AS = \begin{bmatrix}{1}&{0}\\{0}&{0}\end{bmatrix} Now choose M=[1012]. The similar matrix M−1AM is [1100]\text{Now choose } M = \begin{bmatrix}{1}&{0}\\{1}&{2}\end{bmatrix} \text{. The similar matrix } M^{-1}AM \text{ is } \begin{bmatrix}{1}&{1}\\{0}&{0}\end{bmatrix} Also choose M=[0−110]. The similar matrix M−1AM is [.5−.5−.5.5]\text{Also choose } M = \begin{bmatrix}{0}&{-1}\\{1}&{0}\end{bmatrix} \text{. The similar matrix } M^{-1}AM \text{ is } \begin{bmatrix}{.5}&{-.5}\\{-.5}&{.5}\end{bmatrix}

  • 고유값 1,0을 가진 모든 2 * 2 행렬은 서로 유사하다.
  • 고유 벡터는 M에 의해 변하고, 고유값은 변하지 않는다.
ex2)

The zero matrix shares those eigenvalues, but it is similar only to itself: M−10M=0M^{-1}0M = 0.

A family of similar matrices with A = 0, 0 (repeated eigenvalue) A=[0−100] is similar to [1−11−1] and all B=[cdd2−c2−cd] except [0000]A=\begin{bmatrix}{0}&{-1}\\{0}&{0}\end{bmatrix} \text{ is similar to }\begin{bmatrix}{1}&{-1}\\{1}&{-1}\end{bmatrix}\text{ and all }B=\begin{bmatrix}{cd}&{d^2}\\{-c^2}&{-cd}\end{bmatrix} \text{ except } \begin{bmatrix}{0}&{0}\\{0}&{0}\end{bmatrix}

  • 이 매트릭스 B는 모두 0 determinant(A와 같은)를 가지고 있다.
  • 모두 A와 같이 rank one을 가지고 있다.
  • 한 고유값은 0이고, trace는 cd-dc=0이어서 다른 고유값도 0이어야 한다,

M=[abcd] with ad−bc=1,, and B=M−1AMM=\begin{bmatrix}{a}&{b}\\{c}&{d}\end{bmatrix}\text{ with } ad-bc=1,\text{, and }B=M^{-1}AM

  • 행렬 B는 대각화할 수 없다.
  • A는 가능한한 대각화에 가깝게 해야한다.
  • 이건 행렬 B의 family에 대한 조르단 형식Jordan form이다.
  • 오직 한 고유벡터만 있을때, Jordan form J=A는 이 행렬을 대각화할만큼 근접하다.
  • AA에서 B=M−1AMB=M^{-1}AM에 가는동안 어떤 것은 변하고 어떤 것은 변하지 않는다.
    • 변하는 것: Eigenvectors, Nullspace , Column space , Row space , Left nullspace , Singular values
    • 변하지 않는 것: Eigenvalues , Trace and determinant , Rank , Number of independent eigenvectors , Jordan form
  • 고유값과 고유벡터는 유사한 메트릭스에서 변하지 않는다.
  • trace는 변하지 않는 고유값들의 합이다.
  • determinanat는 같은 고유값들의 곱이다.
  • nullspace는 고유값이 0인게 하나라도 존재하는 고유벡터로 구성되어 있어서 변할 수 있다.
  • 이 차원은 n-r이고, 변하지 않는다!
  • 벡터들이 스스로 M−1M^{-1}에 곱해지는 동안 고유벡터는 각 고유값에서 같다.
  • singular 값은 ATAA^TA에 의존하고, 명확하게 변한다.

Examples of the Jordan Form

We lead up to it with one more example of similar matrices: triple eigenvalue, one eigenvector.

ex3)

This Jordan matrix JJ has λ=5,5,5\lambda = 5,5,5 on its diagonal. Its only eigenvectors are multiples of x=(1,0,0)x = (1,0,0). Algebraic mUltiplicity is 3, geometric multiplicity is 1: If J=[510051005] then J−5I=[010001000] has rank 2\text{If }J=\begin{bmatrix}{5}&{1}&{0}\\{0}&{5}&{1}\\{0}&{0}&{5}\end{bmatrix} \text{ then } J-5I=\begin{bmatrix}{0}&{1}&{0}\\{0}&{0}&{1}\\{0}&{0}&{0}\end{bmatrix} \text{ has rank 2}

  • 모든 유사한 행렬 B=M−1JMB=M^{-1}JM 은 같은 세 고유값 5,5,5를 가지고 있다.
  • 또한, B−5IB-5I는 같은 rank 2를 가져야만 한다.
  • nullspace는 차원1을 가지고 있다.
  • Jordan block인 J와 규사한 모든 B는 유일한 하나의 독립적인 고유벡터 M−1xM^{-1}x를 가지고 있다.

Jordan's theorem에서는 JtJ^t가 JJ와 유사하다고 말한다. 유사함을 제공하는 행렬 M은 reverse identity가 된다. JT=M−1JM is [500150015]=[111][510051005][111]J^T=M^{-1}JM\text{ is } \begin{bmatrix}{5}&{0}&{0}\\{1}&{5}&{0}\\{0}&{1}&{5}\end{bmatrix} = \begin{bmatrix}{}&{}&{1}\\{}&{1}&{}\\{1}&{}&{}\end{bmatrix} \begin{bmatrix}{5}&{1}&{0}\\{0}&{5}&{1}\\{0}&{0}&{5}\end{bmatrix} \begin{bmatrix}{}&{}&{1}\\{}&{1}&{}\\{1}&{}&{}\end{bmatrix}

  • 모든 빈 entries는 0이다.
  • JTJ^T의 고유백터는 M−1(1,0,0)=(0,0,1)M^{-1}(1,0,0)=(0,0,1)이다.
  • J에서 고유벡터 one line (x1,0,0)(x_1,0,0)과 JTJ^T에서 another line (0,0,x3)(0,0,x_3)이 있다.

The key fact is that this matrix JJ is similar to every matrix AA with eigenvalues 5,5,5 and one line of eigenvectors.

There is an MM with −1AM=J^{-1}AM = J.

ex4)

Since JJ is as close to diagonal as we can get, the equation dudt=Ju\frac{du}{dt} = J u cannot be simplified by changing variables. We must solve it as it stands: dudt=Ju=[510051005][xyz] is dx/dt=5x+ydy/dt=5y+zdz/dt=5z\frac{du}{dt}=Ju=\begin{bmatrix}{5}&{1}&{0}\\{0}&{5}&{1}\\{0}&{0}&{5}\end{bmatrix}\begin{bmatrix}{x}\\{y}\\{z}\end{bmatrix} \text{ is } \begin{matrix}{dx/dt=5x+y}\\{dy/dt=5y+z}\\{dz/dt=5z}\end{matrix}

  • system은 triangular이다. Last equation dzdt=5zyields z=z(0)e5t\text{Last equation }\frac{dz}{dt}=5z \text{yields } z=z(0)e^{5t} Notice te5t dydt=5y+zyields y=(y(0)+tz(0))e5t\text{Notice }te^{5t}\text{ }\frac{dy}{dt}=5y+z \text{yields } y=(y(0)+tz(0))e^{5t} Notice t2e5t dxdt=5x+yyields x=(x(0)+ty(0)+12t2z(0))e5t\text{Notice }t^2e^{5t}\text{ }\frac{dx}{dt}=5x+y \text{yields } x=(x(0)+ty(0)+\frac{1}{2}t^2z(0))e^{5t}
  • The two missing eigenvectors are responsible for the teSt and t 2eSt terms in y and x.
  • The factors t and t 2 enter because A = 5 is a triple eigenvalue with one eigenvector.

The Jordan Form

  • 모든 A에 대해 우리는 M을 선택해서 M−1AMM^{-1}AM이 가능한한 대각화에 근접하기를 원한다.
  • A가 n개 고유벡터들의 전체 집합일때, M의 열들이 고유벡터가 된다.(When A has a full set of n eigenvectQrs, they go. into. the cQlumns Qf M. )
  • 그러면 M=S가 된다.
  • 행렬 S−1ASS^{-1}AS는 대각화되어 있다.
  • 행렬 Λ\Lambda는 A의 조르단 형식이다.(A가 대각화될 수 있다면.)
  • 일반적으로 고유벡터는 결측이고, 고유값 집합은 얻을 수 없다(In the general case, eigenvectors are missing and A can't be reached. ).

(Jordan form) If A has s independent elegevectors, it is similar to a matrix JJ that has s Jordan blocks on its diagonal: Some matrix M plus A into Jordan form: Jordan form M−1AM=[J1⋱Js]=J\text{Jordan form }M^{-1}AM=\begin{bmatrix}{J_1}&{}&{}\\{}&{\ddots}&{}\\{}&{}&{J_s}\end{bmatrix}=J

  • J에서 각 block은 하나의 고유값 람다, 고유벡터, 그리고 위 대ㅏㄱ에서 1들을 가지고 있다. Jordan block Ji=[λi1⋅⋅⋅1λi]\text{Jordan block }J_i=\begin{bmatrix}{\lambda_i}&{1}{}&{}\\{}&{\cdot}&{\cdot}&{}\\{}&{}&{\cdot}&{1}\\{}&{}&{}&{\lambda_i}\end{bmatrix}
  • 만일 같은 jordan form J를 가지고 있다면, A는 B와 유사하다.
  • Jordam form J는 결측 고유벡터에 대해 off-diagonal 1을 가지고 있다.
  • 이는 행렬 유사성에 대한 big theotem이다.
  • 유사한 행렬의 모든 family에서 J라고 불리는 특정 member를 선택한다,
  • 그건 대각화와 근사하다.
  • J에 대해 du/dt=Ju를 ex4에서 풀 수 있었다.
  • family에서 모두 다른 행렬은 form A=MJM−1A=MJM^{-1}을 가지고 있다.
  • M을 통한 연결은 du/dt=Au를 해결한다.
  • MJM−1MJM−1=MJ2M−1MJM^{-1}MJM^{-1}=MJ^2M^{-1}

REVIEW OF THE KEY IDEAS

WORKED EXAMPLES

Problem Set 6.6

Challenge ProblemsS

6.7 Singular Value Decomposition (SVD)

A is any m by n matrix, square or rectangular. Its rank is r. We will diagonalize this AA, but not by S−1ASS^{-1} AS.

  • 특이값 분해
  • 고유벡터 s는 3가지 큰 문제가 있다.
    • 항상 직교가 아니며
    • 항상 충분한 고유벡터를 가지고 있지 않으며,
    • Ax=λxAx=\lambda x는 A가 squre이기를 원한다.

The uu's are eigenvectors of AATAA^T and the vv's are eigenvectors of ATAA^T A.

  • 행렬이 둘 다 대칭이기 때문에 이 고유벡터들은 직교로 선택될 수 있다.
  • In equation (13) below, the simple fact that AA times ATAA^T A is the same as AATAA^T will lead to a remarkable property of these u's and v's: "A is diagonalized" Av1=σ1u1 Av2=σ2u2 …Avr=σrur\text{"A is diagonalized" } Av_1=\sigma_1u_1\text{ } Av_2=\sigma_2u_2\text{ }\dots Av_r=\sigma_r u_r
  • The singular vectors v1,...,vrv_1, ... , v_r are in the row space of AA.
  • The outputs u1,...,uru_1 , ... , u_r are in the column space of AA.
  • The singular values σ1…σr\sigma_1\dots \sigma_r are all positice numbers
  • When the v'S and u's go into the columns of V and U, orthogonality gives VTV=IV^T V = I and UTU=IU^T U = I.
  • The σ\sigma's go into a diagonal matrix ∑\sum

Axi=λixiAx_i=\lambda_i x_i가 대각 AS=AΛAS=A\Lambda로 이끌기 떄문에, 방정식 Avi=σiuiAv_i=\sigma_i u_i는 AV=UΣAV=U \Sigma에서 column by column이라 말한다. (m by n)(n by r)equals(m by r)(r by r)A[v1,…vr]=[u1…ur][σ1⋱σr]\begin{matrix} { \text{(m by n)(n by r)}} \\{\text{equals}}\\{\text{(m by r)(r by r)}}\end{matrix} A\begin{bmatrix}{}\\{v_1,\dots v_r}\\{}\end{bmatrix}= \begin{bmatrix}{}\\{u_1\dots u_r}\\{}\end{bmatrix} \begin{bmatrix}{\sigma_1}&{}&{}\\{}&{\ddots}&{}\\{}&{}&{\sigma_r}\end{bmatrix}

  • v와 u는 A의 행과 열 공간을 차지하기도 함
  • nullspace N(A)와 왼쪽 nullspaceN(AT)N(A^T)로부터 더 많은 v를 위해 n-r이 필요하고, u를 위해 m-r이 필요하다.
    • 두 공간이 직교가 된다면 자동적으로 v들과 u들은 직교가 된다.
  • 그래서 행렬은 sqaure가 되고, AV=UΣAV=U\Sigma을 얻게 된다. (m by n)(n by n)equals(m by m)(m by n)A[v1,…vr…vn]=[u1…ur…um][σ1⋱σr]\begin{matrix} { \text{(m by n)(n by n)}} \\{\text{equals}}\\{\text{(m by m)(m by n)}}\end{matrix} A\begin{bmatrix}{}\\{v_1,\dots v_r \dots v_n}\\{}\end{bmatrix}= \begin{bmatrix}{}\\{u_1\dots u_r \dots u_m}\\{}\end{bmatrix} \begin{bmatrix}{\sigma_1}&{}&{}&{}\\{}&{\ddots}&{}&{}\\{}&{}&{\sigma_r}&{}\\{}&{}&{}&{}\end{bmatrix}
  • 새로운 Σ\Sigma은 m by n 행렬
  • m-r개의 새로운 0행과 n-r개의 새로운 0 행을 가진 기존의 r * r 행렬이다.
  • VTV=IV^TV=Idhk UTU=IU^TU=I는 유지되고,크기는 n과 m이다.
  • V는 이제 V−1=VTV^{-1}=V^T인 직각 직교 행렬이다.
  • AV=UΣAV=U\Sigma은 A=UΣVTA=U\Sigma V^T가 될 수 있다.

Singular Value Decomposition: SVD A=UΣVT=u1σ1v1T+⋯+urσrvrT\text{SVD } A=U\Sigma V^T = u_1\sigma_1 v^{T}_{1}+\dots+u_r\sigma_r v^{T}_{r}

This reduced SVD gives the same splitting of A into a sum of r matrices, each of rank one.

We will see that σi2\sigma^{2}_{i} is an eigenvalue of ATAA^T A and also AATAA^T. When we put the singular values in descending order, σ1>σ2>...σr>0\sigma_1 > \sigma_2 > ... \sigma_r > 0, the splitting in equation (4) gives the r rank -one pieces of A in order of importance.

ex1)

When is UΣVTU\Sigma V^T (singular values) the same as ΛAS−1\Lambda AS^{-1} (eigenvalues)?

solution

  • S=U에서 직교 고유벡터가 필요하다. Λ=Σ\Lambda= \Sigma에서 음수가 아닌 고유값들이 필요하다.
  • A는 positive semidefinite (or definite) symmetric matrix QΛQTQ\Lambda Q^T여야 한다.
ex2)

If A=xyTA = xy^T with unit vectors xx and yy, what is the SVD of A?

solution

  • 기존 Σ\Sigma있는 식에서 감소한 SVD는 rank=1을 가진 xyTxy^T이지.
  • 이건 u1=xu_1=x, v1=yv_1=y, σ1=1\sigma_1=1를 가져.
  • 전체 SVD에서 u1=xu_1=x를 u의 직교 basis에서 complete하고, v의 직교 basis에서 v1=yv_1=y로 coplete해.(For the full SVD, complete ul=xu_l = x to an orthonormal basis of u's, and complete v1=yv_1 = y to an orthonormal basis of v's. No new O"s. )

Image Compression

What is compression?

  • We want to replace those 217 matrix entries by a smaller number, without losing picture quality

try an SVD approach: Replace the 256 by 512 pixel matrix by a matrix of rank one: a column times a row. If this is successful, the storage requirement becomes 256 + 512 (add instead of multiply). The compression ratio (256)(512)/(256 + 512) is better than 170 to 1. This is more than we hope for. We may actually use five matrices of rank one (so a matrix approximation of rank 5). The compression is still 34 : 1 and the crucial question is the picture quality.

Where does the SVD come in? The best rank one approximation to A is the matrix σ1u1v1T\sigma_1 u_1 v_{1}^{T}. It uses the largest singular value σ1\sigma_1. The best rank 5 approximation includes also σ2u2v2T+...+σ5u5v5T\sigma_2 u_2 v_{2}^{T} + ... + \sigma_5 u_5 v_{5}^{T}, The SVD puts the pieces of A in descending order.

The Bases and the SVD

  • Start with a 2 by 2 matrix. Let its rank be r = 2, so A is invertible.
  • We want v1v_1 and v2v_2 to be perpendicular unit vectors.
  • We also want Av1Av_1 and Av2Av_2 to be perpendicular.
  • Then the unit vectors Ul=Av1/∣∣Av1∣∣U_l = Av_1/||Av_1|| and u2=Av2/∣∣Av2∣∣u_2 = Av_2/||Av_2|| will be orthonormal. Unsysmmetric matrix A=[22−11]\text{Unsysmmetric matrix }A=\begin{bmatrix}{2}&{2}\\{-1}&{1}\end{bmatrix}
  • No orthogonal matrix QQ will make Q−lAQQ^{-l} AQ diagonal.
  • We need U−1AVU^{-1} AV.
  • The two bases will be different-one basis cannot do it.
  • The output is Av1=σ1u1Av_1 = \sigma_1 u_1 when the input is v1v_1.
  • The "singular values" σ1\sigma_1 and σ2\sigma_2 are the lengths ∣∣Av1∣∣||Av_1|| and ∣∣Av2∣∣||Av_2||. AV=UΣA=UΣVT A[v1v2]=[σ1u1σ2u2]=[u1u2][σ1σ2]\begin{matrix}{AV=U\Sigma}\\{A=U\Sigma V^T}\end{matrix}\text{ } A\begin{bmatrix}{v_1} &{v_2}\end{bmatrix}=\begin{bmatrix}{\sigma_1u_1}&{\sigma_2u_2}\end{bmatrix}=\begin{bmatrix}{u_1}&{u_2}\end{bmatrix}\begin{bmatrix}{\sigma_1}&{}\\{}&{\sigma_2}\end{bmatrix}

ATA=(UΣVT)T(UΣVT)=VΣTΣVTA^T A=(U\Sigma V^T)^T (U\Sigma V^T)=V\Sigma {}^T\Sigma {V}^T

  • UTUU^T U disappears because it equals I.
  • uT U disappears because it equals I.
  • Multiplying those diagonal ΣT\Sigma{}^T and Σ\Sigma gives σ12\sigma_{1}^{2} and σ22\sigma_{2}^{2}.
  • That leaves an ordinary diagonalization of the crucial symmetric matrix ATAA^T A, whose eigenvalues are σ12\sigma_{1}^{2} and σ22\sigma_{2}^{2}: Eigenvalues σ12,σ22Eigenvectors v1,v2 ATA=V[σ1200σ22]VT\begin{matrix}{\text{Eigenvalues }\sigma_{1}^{2}, \sigma_{2}^{2}}\\{\text{Eigenvectors }v_1, v_2}\end{matrix} \text{ }A^T A=V\begin{bmatrix}{\sigma_{1}^{2}}&{0}\\{0}&{\sigma_{2}^{2}}\end{bmatrix}V^T
  • 이건 A=QΛQTA=Q\Lambda Q^T와 같아
  • 하지만 대칭 행렬은 A itself가 아니지
  • 이제 대칭 행렬은 ATAA^T A야
  • 그리고 V의 열들은 ATAA^TA의 고유벡터이고. Last is UU...
ex3)

Find the singular value decomposition of that matrix A=[22−11]A=\begin{bmatrix}{2}&{2}\\{-1}&{1}\end{bmatrix}

Solution

  • ATAA^TA와 그 고유벡터를 계산하자
  • 그러면 unit vector가 나와 ATA=[5335] has unit eigenvectors v1=[1/21/2] and v2=[−1/21/2]A^TA=\begin{bmatrix}{5}&{3}\\{3}&{5}\end{bmatrix}\text{ has unit eigenvectors }v_1=\begin{bmatrix}{1/\sqrt{2}}\\{1/\sqrt{2}}\end{bmatrix} \text{ and } v_2=\begin{bmatrix}{-1/\sqrt{2}}\\{1/\sqrt{2}}\end{bmatrix}
  • ATAA^TA의 고유값들은 8하고 2임
  • v들은 직교야. 왜냐하면 모든 대칭 행렬의 고유 벡터는 직교이기 떄문이지!
  • 그리고 ATAA^TA는 자동적으로 대칭이 돼 Av1=[22−11][1/21/2]=[220]. The unit vector is u1=[10]Av_1=\begin{bmatrix}{2}&{2}\\{-1}&{1}\end{bmatrix}\begin{bmatrix}{1/\sqrt{2}}\\{1/\sqrt{2}}\end{bmatrix}=\begin{bmatrix}{2\sqrt{2}}\\{0}\end{bmatrix}\text{. The unit vector is } u_1=\begin{bmatrix}{1}\\{0}\end{bmatrix}
  • Av1Av_1은 22u12\sqrt{2} u_1과 같지
  • 처음 특이값singular value는 σ1=22\sigma_1 = 2\sqrt{2}임. 그러면 σ12=8\sigma_{1}^{2}=8이 돼. Av2=[22−11][−1/21/2]=[02]. The unit vector is u2=[01]Av_2=\begin{bmatrix}{2}&{2}\\{-1}&{1}\end{bmatrix}\begin{bmatrix}{-1/\sqrt{2}}\\{1/\sqrt{2}}\end{bmatrix}=\begin{bmatrix}{0}\\{\sqrt{2}}\end{bmatrix}\text{. The unit vector is } u_2=\begin{bmatrix}{0}\\{1}\end{bmatrix}
  • Av2Av_2는 2u2\sqrt{2}u_2이고 ]sigma2=2]sigma_2=\sqrt{2}임.
  • 그러면 σ22\sigma_{2}^{2}는 ATAA^T A의 다른 고유값 2에 agree 하지... A=UΣVT is [22−11]=[1001][222][1/21/2−1/21/2]A=U\Sigma V^T \text{ is }\begin{bmatrix}{2}&{2}\\{-1}&{1}\end{bmatrix}=\begin{bmatrix}{1}&{0}\\{0}&{1}\end{bmatrix}\begin{bmatrix}{2\sqrt{2}}&{}\\{}&{\sqrt{2}}\end{bmatrix}\begin{bmatrix}{1/\sqrt{2}}&{1/\sqrt{2}}\\{-1/\sqrt{2}}&{1/\sqrt{2}}\end{bmatrix}
  • 이 행렬과 모든 역을 취할 수 있는 2 * 2 행렬은 단위 원을 타원으로 변환한다!

We found the u's from the v's. Could we find the u's directly? Yes, by multiplying AATAA^T instead of ATAA^T A: Diagonal in this example AAT=[22−11][2−121]=[8002]\text{Diagonal in this example } A A^T = \begin{bmatrix}{2}&{2}\\{-1}&{1}\end{bmatrix}\begin{bmatrix}{2}&{-1}\\{2}&{1}\end{bmatrix}= \begin{bmatrix}{8}&{0}\\{0}&{2}\end{bmatrix}

  • 이 고유벡터 (1,0)과 (0,1)은 u1,u2u_1,u_2를 빨리 찾을 수 있게 하지!
  • 왜 처음 고유벡터가 (-1,0)이나(0,1) 대신에 (1,0)이 될까?
  • 우리는 Av1Av_1를 따라야 하기 때문이지!
  • AATAA^T는 ATAA^TA와 같은 고유값 8,2를 가져
  • 그 특이값은 8,2\sqrt{8},\sqrt{2}야
ex4)

Find the SVD of the singular matrix A=[2211]A = \begin{bmatrix}{2}&{2}\\{1}&{1}\end{bmatrix}. The rank is r = 1.

Solution

  • 행 공간은 오직 한 basis 벡터 v1=(1,1)/2v_1=(1,1)/\sqrt{2}를 가지고 있지.
  • 그 열 공간은 오직 한 basis 벡터 u1=(2,1)/5u_1=(2,1)/\sqrt{5}를 가지고 있고.
  • 그러면 Av1=(4,2)/2Av_1=(4,2)/\sqrt{2}는 σ1u1\sigma_1 u_1과 같아야 해.
  • 그러면 σ1=10\sigma_1=\sqrt{10}

행렬 U와 V는 모든 4개의 준공간에서 직교 기반을 포함한다.

  • V의 열에서 첫 r 은 A의 행 공간
  • V의 열에서 마지막 n-r은 A의 영공간
  • U의 열에서 첫 r은 A의 열공간
  • U의 열에서 마지막 n-r은 ATA^T의 영공간

The first columns v1,...,vrv_1, ... ,v_r and u1,...,uru_1, ... ,u_r are eigenvectors of ATAA^T A and AATAA^T. We now explain why AVi falls in the direction of uiu_i. The last v's and u's (in the nullspaces) are easier. As long as those are orthonormal, the SVD will be correct.

Eigshow (Part 2)

Section 6.1 described the MATLAB demo called eigshow.

Searching the Web

Search engines are link-based more than content-based.

REVIEW OF THE KEY IDEAS

WORKED EXAMPLES

Problem Set 6.7

Challenge Problems

Gilbert Strang 강의영상...!.