LR - ANOVA Table

Total

Sum of Squares Total (TSS)
Sum of Squares Restricted
Sum of Squares Around Mean

• 𝑆𝑆_𝑇𝑂𝑇 = 𝛴_{1≤𝑖≤𝑛}[𝑦_𝑖 - 𝑦̅]²
• 𝑆𝑆_𝑇𝑂𝑇= 𝑆𝑆_𝑅𝐸𝐺 + 𝑆𝑆_𝐸𝑅𝑅

𝑑𝑓_𝑇𝑂𝑇 = 𝑛 - 1

Error

Sum of Squares Error (ESS)
Sum of Squares Residual (RSS)
Sum of Squares UnRestricted
Sum of Squares Around Model

• 𝑆𝑆_𝐸𝑅𝑅 = 𝛴_{1≤𝑖≤𝑛}[𝑦_𝑖 - 𝑦̂_𝑖]² = 𝛴_{1≤𝑖≤𝑛}[𝑒_𝑖]²

𝑑𝑓_𝐸𝑅𝑅= 𝑛 - # of model params including 𝜃₀
𝑑𝑓_𝐸𝑅𝑅= 𝑛 - (𝑘 + 1)
𝑑𝑓_𝐸𝑅𝑅= 𝑛 - 𝑘 - 1

𝑀𝑆_𝐸𝑅𝑅 = 𝑆𝑆_𝐸𝑅𝑅 / 𝑑𝑓_𝐸𝑅𝑅

Mean Square Error (MSE)
Regression Variance

𝑀𝑆_𝑅𝐸𝐺/ 𝑀𝑆_𝐸𝑅𝑅

this 𝐹 formula is used to test significance of the ENTIRE regression model

for other 𝐹 formulas used to test PARTIAL significance of regression model consult table below

Model

Sum of Squares Regression (RSS)
Sum of Squares Explained (ESS)

• 𝑆𝑆_𝑅𝐸𝐺 = 𝑆𝑆_𝑇𝑂𝑇 - 𝑆𝑆_𝐸𝑅𝑅 
• 𝑆𝑆_𝑅𝐸𝐺 = 𝛴_{1≤𝑖≤𝑛}[𝑦̂_𝑖 - 𝑦̅]²

𝑑𝑓_𝑅𝐸𝐺 = 𝑑𝑓_𝑇𝑂𝑇 - 𝑑𝑓_𝐸𝑅𝑅
𝑑𝑓_𝑅𝐸𝐺 = (𝑛 - 1) - (𝑛 - # of model params)
𝑑𝑓_𝑅𝐸𝐺 = (# of model params) - 1
𝑑𝑓_𝑅𝐸𝐺 = 𝑘 = number of predictor variables 𝜃_𝑖‘s excluding 𝜃₀

𝑀𝑆_𝑅𝐸𝐺 = 𝑆𝑆_𝑅𝐸𝐺 / 𝑑𝑓_𝑅𝐸𝐺

𝐹 statistic for testing the null hypothesis that ALL variables are insignificant (e.g. 𝐻₀: 𝜃₁= … 𝜃_𝑘 = 0)

𝐹 statistic for testing the null hypothesis that SOME variables are insignificant (e.g. 𝐻₀: 𝜃_𝑖= 0, ∀𝜃_𝑖∊𝑆 where 𝑆⊆{𝜃₁, … 𝜃_𝑘})

unrestricted model

• 𝑦̂_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑} = 𝜃₀+ 𝜃₁𝑥₁ + … + 𝜃_𝑘𝑥_𝑘

restricted model

• 𝑦̂_{𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑} = 𝜃₀+ 0·𝑥₁ + … + 0·𝑥_𝑘= 𝜃₀ = 𝑦̅

𝐹 sum of squares form

• 𝐹 = 𝑀𝑆_𝑅𝐸𝐺/ 𝑀𝑆_𝐸𝑅𝑅
• 𝐹 = [(𝑆𝑆_𝑇𝑂𝑇- 𝑆𝑆_𝐸𝑅𝑅)/((𝑛 - 1)-(𝑛 - 𝑘 - 1))] / [(𝑆𝑆_𝐸𝑅𝑅)/(𝑛 - 𝑘 - 1)]
• 𝐹 = [(𝑆𝑆_𝑇𝑂𝑇- 𝑆𝑆_𝐸𝑅𝑅)/(𝑘)] / [(𝑆𝑆_𝐸𝑅𝑅)/(𝑛 - 𝑘 - 1)]
• 𝐹 = [(𝑆𝑆_{𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑}- 𝑆𝑆_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑})/(𝑘)] / [(𝑆𝑆_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑})/(𝑛 - 𝑘 - 1)]

𝐹 𝑅² form

• 𝐹 = [𝑅²_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑}/𝑘] / [(1 - 𝑅²_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑})/(𝑛 - 𝑘 - 1)]

unrestricted model

• 𝑦̂_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑} = 𝜃₀+ 𝜃₁𝑥₁ + … + 𝜃_𝑘𝑥_𝑘

restricted model

• 𝑦̂_{𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑} = 𝜃₀ + (linear combination of 0·𝑥_𝑖 for 𝜃_𝑖∊𝑆) + (linear combination of 𝜃_𝑗𝑥_𝑗 for 𝜃_𝑗∉𝑆)
• 𝑦̂_{𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑} = 𝜃₀ + (linear combination of 𝜃_𝑗𝑥_𝑗 for 𝜃_𝑗∉𝑆)

𝐹 sum of squares form

• 𝐹 = [(𝑆𝑆_{𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑} - 𝑆𝑆_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑})/((𝑛 - (𝑘-|𝑆|) - 1)-(𝑛 - 𝑘 - 1))] / [(𝑆𝑆_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑})/(𝑛 - 𝑘 - 1)]
• 𝐹 = [(𝑆𝑆_{𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑} - 𝑆𝑆_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑})/(|𝑆|)] / [(𝑆𝑆_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑})/(𝑛 - 𝑘 - 1)]

𝐹 𝑅² form

• 𝐹 = [(𝑅²_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑}- 𝑅²_{𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑})/(|𝑆|)] / [(1 - 𝑅²_{𝑢𝑛𝑟𝑒𝑠𝑡𝑟𝑖𝑐𝑡𝑒𝑑})/(𝑛 - 𝑘 - 1)]

𝐹 has f-distribution with parameters (𝑘, (𝑛 - 𝑘 - 1))

𝐹 has f-distribution with parameters (|𝑆|, (𝑛 - 𝑘 - 1))