• is a type of continuous regression model whose function/estimator is linear with respect to the regression coefficients {𝜃₀, …, 𝜃_𝑝}:
• 𝑦̂ = 𝜃₀ + 𝜃₁𝑓₁(𝒙) + … + 𝜃_𝑝𝑓_𝑝(𝒙)
• models the relationship between:
  • 𝑌 - a single scalar response/dependent variable (for categorical use logistic regression)
  • {𝑋₁, …, 𝑋_𝑘} - one or more regressors or explanatory/predictor/covariate/independent variables. predictor variable types:
    • continuous/scalar/numerical predictor
    • categorical predictor - itself can be either nominal or ordinal
• models expected response as a function/conditional of regressors (where 𝑓_𝑖(..) are feature functions)
  • 𝐄[𝑌|𝑋₁=𝑥₁, …, 𝑋_𝑘=𝑥_𝑘] = ℎ(𝑥₁, …, 𝑥_𝑘) = 𝑦̂ = 𝜃₀+ 𝜃₁𝑓₁(𝑥₁, …, 𝑥_𝑘) + … + 𝜃_𝑝𝑓_𝑝(𝑥₁, …, 𝑥_𝑘)
  • coefficient 𝜃₀ represents the𝑦 intercept when all feature functions 𝑓_𝑖(..) equate to 0
  • coefficient 𝜃_𝑖 represents the mean change in the dependent variable 𝑦 given a 1 unit change in the independent feature function 𝑓_𝑖(𝑥₁, …, 𝑥_𝑘) # for 1≤𝑖≤𝑝
• is a type of level-level model(or even alevel-log model when 𝑓_𝑖(..) are log functions)
• the dependent variable 𝑦 is the combination of the regression model and error
  • 𝑦 = 𝑦̂ + 𝑒
  • dependent variable = (constant + independent variables) + error
  • dependent variable = deterministic + stochastic
  • deterministic component is the portion of the variation in the dependent variable that the independent variables explain. In other words, the mean of the dependent variable is a function of the independent variables. In a regression model, all of the explanatory power should reside here
  • error is the difference between the expected value 𝑦̂ and the observed value 𝑦. Let’s put these terms together—the gap between the expected and observed values must not be predictable. Or, no explanatory power should be in the error. If you can use the error to make predictions about the response, your model has a problem. This issue is where residual plots play a role.
  • the theory here is that the deterministic component of a regression model does such a great job of explaining the dependent variable that it leaves only the intrinsically inexplicable portion of your study area for the error. If you can identify non-randomness in the error term, your independent variables are not explaining everything that they can

Linear Regression Models - takes an input vector 𝑥∊ℝ^𝑛 as input and predicts the value of a scalar 𝑦∊ℝ as output (whose function/estimator is linear wrt the regression coefficients {𝜃₀, …, 𝜃_𝑝})