GT
1.
원점을 지나는 회귀모형은 다음과 같이 정의할 수 있다.
\[y_i = \beta x_i + \epsilon_i, \epsilon_i \sim_{i.i.d.} n(0,\sigma^2), i=1,\dots, n\]
(1)
\(\beta\)에 대한 최소제곱추정량(\(LSE\)) \(\hat{\beta}\)을 구하시오
answer
\(\hat{\beta} = argmin_{\beta \in R} S = \sum^n_{i=1} (y_i - \beta x_i)^2 \to \frac{\partial S}{\partial \beta}|_{\beta = \hat{\beta}} = 0\)
\(\frac{\partial S}{\partial \beta} = \sum^n_{i=1} (-2x_i)(y_i - \beta x_i) \to \sum^n_{i=1} x_i(y_i - \hat{\beta}x_i) = 0 \to \hat{\beta} = \frac{\sum^n_{i=1} x_iy_i}{\sum^n_{i=1} x^2_i}\)
(2)
\(E(\hat{\beta})\)을 구하시오
answer
\(\hat{\beta} = \frac{\sum^n_{i=1} x_i y_i}{\sum^n_{i=1} x^2_i} = \sum^n_{i=1} \frac{x_i}{\sum^n_{j=1} s^2_j} y_i\)
\(\to E(\hat{\beta})= \sum^n_{i=1} \frac{x_i}{\sum^n_{i=1} x^2_j} E(y_i) = \sum^n_{i=1}\frac{x_i}{\sum^n_{j=1} x^2_j} \beta x_i = \beta\frac{\sum^n_{i=1} x^2_{i=1} x^2_i}{\sum^n_{i=1} x^2_i} = \beta\)
(3)
\(Var(\hat{\beta})\)을 구하시오
answer
\(Var(\hat{\beta}) = \sum^n_{i=1} \frac{x^2_i}{(\sum^n_{j=1} x^2_j)^2} Var(y_i) = \sum^n_{i=1} \frac{x^2_i}{(\sum^n_{j=1} x^2_j)^2} \sigma^2 = \sigma^2 \frac{\sum^n_{i=1} x^2_i}{(\sum^n_{i=1} x^2_i)^2} = \frac{\sigma^2}{\sum^n_{i=1} x^2_i}\)
2.
중회귀모형 \(y = X\beta + \epsilon, \epsilon \sim N(0_n, I\sigma^2)\)에서 \(X\)가 \(n \times (p+1)\) 행렬이고, \(rank\)가 \(p+1\) 이라면 적합된 모형 \(\hat{y} = X \hat{\beta}\)에 대하여 다음을 증명하여라. 단, \(\mathbf{e} = \mathbf{y} - \mathbf{\hat{y}}\)
(1)
\(\sum^n_{j=1} Var(\hat{y}_j) = (p+1) \sigma^2\)
answer
\(\hat{\bf{y}}= X\hat{\beta} = X(X^\top X)^{-1} X^\top y = Hy\)
\(Var(X\hat{\beta}) = XVar(\hat{\beta})X^\top\)
\(Var(\hat{\beta}) = Var((X^\top X)^{-1} X^\top y) = (X^\top X)^{-1} X^\top Var(y)X(X^\top X)^{-1} = (X^\top X)^{-1} X^\top X(X^\top X)^{-1} \sigma^2 = (X^\top X )^{-1} \sigma^2\)
\(Var(\hat{y}) = Var(X\hat{\beta}) = X(X^\top X)^{-1} X^\top \sigma^2\)
\(\to Var(\hat{y}) = HVar(y) H^\top = HH\sigma^2 = H\sigma^2\)
\(\star H^\top = H, H^2 = H, Var(y) = I_n \sigma^2\)
\(\sum^{n}_{j=1} Var(\hat{y}_j) = tr(Var(\hat{y})) = tr(H\sigma^2) = \sigma^2 tr(H) = tr((X^\top X)^{-1} X^\top X)\sigma^2 = tr(I_{p+1})\sigma^2 = (p+1)\sigma^2\)
\(\star tr(X (X^\top X)^{-1} X^\top)\)
(2)
\(Cov(\mathbf{e,y}) = \sigma^2[I_n - X(X^\top X)^{-1} X^\top]\)
answer
\(\bf{e} = y - \hat{y} = y - Hy = (I-H)y\)
\(Cov(\bf{e},y) = Cov((I-H)y,y) = (I-H)Var(y) = (I-H) \sigma^2\)
\(\star Cov(Ax,y) = A Cov(X,Y)\)
\(\star Var(y) = \sigma^2\)
(3)
\(Cov(\mathbf{e,\hat{y}}) = O_n\)
answer
\(Cov(e(I - H)\bf{y},Hy) = (I-H) Cov(y,y)H^\top = (I-H)H\sigma^2 = \mathbb{O}_n \sigma^2\)
\(\star Cov(Ax,By) = ACov(X,Y)B^\top\)
\(\star H^\top= H\)
\(\star H^2 = H\)
(4)
\(Cov(\mathbf{e,\hat{\beta}}) = O_{n \times (p+1)}\)
answer
\(Cov((I-H)\bf{y},(X^\top X)^{-1} y) = (I-H) Cov(y,y) X (X^\top X)^{-1} = (I - X(X^\top X)^{-1} X^\top ) X (X^\top X)^{-1} \sigma^2 = \{ X(X^\top X)^{-1} - X(X^\top X)^{-1} \} \sigma^2 = \mathbb{O}_{n \times ([+1)}\)
(5)
\(Cov(\mathbf{\epsilon, \hat{\beta}}) = \sigma^2 X(X^\top X)^{-1}\)
answer
\(\bf{\hat{\beta}} = (X^\top X)^{-1} X^\top y = (X^\top X)^{-1} X^\top (X\beta + \epsilon) = \beta + (X^\top X)^{-1} X^\top \epsilon\)
\(Cov(\bf{\epsilon} , \beta + (X^\top X)^{-1} X^\top \epsilon ) = Cov(\epsilon, \beta) + Cov(\epsilon, \epsilon)X(X^\top X)^{-1} = \sigma^2 X(X^\top X)^{-1}\)
\(\star Cov(\epsilon, \beta) = 0\)
\(\star Cov(\epsilon, \epsilon) = I_n \sigma^2\)
(6)
\(\mathbf{e^\top y} = SSE\)
answer
\(\sum^{n}_{j=1} e_j y_j = \bf{e^\top y} = \{ (I - H)y\}^\top y = y^\top (I-H)y\)
\(\star \bf{e}^\top (e_1, \dots , e_n) = (y-\hat{y})^\top\)
\(\star \bf{y}^\top = (y_1 , \dots , y_n)\)
\(SSE = \sum^n_{j=1} (y_i - \hat{y}_j)^2 = (\bf{y} - \hat{y} ) ^\top ( y - \hat{y} ) = e^\top e = \{ (I - H)y \}^\top \{ (I - H)y \} = y^\top (I - H) (I-H)y = y^\top (I - H)y\)
\(\star I - H_H+H^2 = I-H, H^2 = H\)
\(\therefore \sum^{n}_{j=1} e_j y_j = SSE\)
(7)
\(\mathbf{e^\top \hat{y}}=0\)
answer
\(\sum^{n}_{j=1} \bf{e_j \hat{y}_j} = e^\top \hat{y} = y^\top (I-H) Hy = y^\top_{1\times n} \mathbb{O}_{n\times n} y_{n \times 1} = 0\)
\(\star H - H^2 = H - H = \mathbb{O}_{n \times n}\)
(8)
\(E(\frac{SSE}{n-p-1}) = \sigma^2\)
answer
정리 5.1
\(y \sim N(\mu,V)\) 이면
\(E(y^\top A y) = tr(AV) + \mu^\top A \mu, cov(y,y^\top A y) = 2VA\mu\)
\(\frac{SSE}{\sigma^2} = y^\top \frac{1}{\sigma^2}(I-H)y\)
\(B = \frac{1}{\sigma^2}(I-H)\)
\(y \sim N(X\beta, I \sigma^2)\)
\(BV = \frac{1}{\sigma^2}(I-H)I\sigma^2 = B, BV=B\)
\(B, BV\)는 멱등행렬
\(tr(BV) = tr(B) = tr(I-H) = tr(I) - tr(H) = n-p-1\)
\(\star\) \(X\)가 \(n \times (p+1)\) 행렬이고, \(rank\)가 \(p+1\), \(\therefore tr(H) = p+1\)
\(\mu = X B\)
\(B = \frac{1}{\sigma^2}(I-H)\)
\(B\mu = \frac{1}{\sigma^2}(I-H) X \beta = \frac{1}{\sigma^2}(X B - HXB) = 0\)
\(\star HX = X\)
\(\therefore \mu^\top B \mu = 0\)
\(E(\frac{SSE}{\sigma^2}) = n-p-1\)
\(E(SSE) = (n-p-1)\sigma^2\)
\(\therefore E(\frac{SSE}{(n-p-1)}) = \sigma^2\)
3.
(1)
적합결여검정을 위한 가설을 제시하라.
(2)
적합결여검정 수행을 위한 과정과 결과 추론을 구체적으로 제시하라(계산 X)