In a simple linear regression Y = X β + ε, residuals are given by ε ^ = M ε, where M = I n − P is the annihilator matrix, and P = X ( X T X ) − 1 X T is the projection matrix, and X is the design matrix. Assuing that the errors ε are iid normal with mean 0 and standard deviation σ, what is the joint (conditional on X) distribution of the residuals ε ^ ?

Question

In a simple linear regression   Y  =  X  β  +  ε, residuals are given by       ε    ^    =  M  ε, where   M  =      I    n    −  P is the annihilator matrix, and   P  =  X  (      X    T    X      )          −      1            X    T   is the projection matrix, and   X is the design matrix. Assuing that the errors   ε are iid normal with mean   0 and standard deviation  σ, what is the joint (conditional on   X) distribution of the residuals       ε    ^  ?

matracavade · Accepted Answer

Note that the residuals       ε    ^   are linear transformation of   Y, that are multivariate normal, hence, as       ε    ^    =  (  I  −  H  )  Y, thus       ε    ^   is multivariate normal with   μ  =  (  I  −  H  )  E  Y  =  X  β  −  X  β  =  0 and       Σ              ε      ^        =  (  I  −  H  )      Σ    Y    (  I  −  H  )  =      σ      Y    2    (  I  −  H  ).

Open question

Answer & Explanation

New Questions in High school statistics