Continuous Regression Models - takes an input vector 𝑥∊ℝ^𝑛 as input and predicts the value of a scalar 𝑦∊ℝ as output

Linear Regression Models Linear Regression Models

Linear Regression Models - takes an input vector 𝑥∊ℝ^𝑛 as input and predicts the value of a scalar 𝑦∊ℝ as output (whose function/estimator is linear wrt the regression coefficients {𝜃₀, …, 𝜃_𝑝})

Linear Model Type	Description
Ordinary Least Squares Regression	has several weaknesses, including sensitivity to both outliers and multicollinearity and it is prone to overfitting
Stepwise Regression Best Subsets Regression	these automated methods can help identify candidate regressors early in the model specification process
Robust Regression	applicable in all cases where Ordinary Least Squares (OLS) Regression can be used applies re-weighting to reduce outlier influence
Ridge Regression	address multicollinearity allows you to analyze data even when severe multicollinearity is present and helps prevent overfitting. This type of model reduces the large, problematic variance that multicollinearity causes by introducing a slight bias in the estimates. The procedure trades away much of the variance in exchange for a little bias, which produces more useful coefficient estimates when multicollinearity is present
Lasso Regression (Least Absolute Shrinkage and Selection Operator)	performs variable selection that aims to increase prediction accuracy by identifying a simpler model. It is similar to Ridge regression but with variable selection
Elastic Net Regression	is a combination of regularizers Ridge regression and LASSO regression
Partial Least Squares (PLS) Regression	is useful when you have very few observations compared to the number of independent variables or when your independent variables are highly correlated. PLS decreases the independent variables down to a smaller number of uncorrelated components, similar to Principal Component Analysis. Then, the procedure performs linear regression on these components rather than the original data. PLS emphasizes developing predictive models and is not used for screening variables. Unlike OLS, you can include multiple continuous dependent variables. PLS uses the correlation structure to identify smaller effects and model multivariate patterns in the dependent variables
Beta Regression	models variables within (0, 1) range
Dirichlet Regression	models compositional data
Loess Regression	smoothing time series
Isotonic Regression	for approximation of data that can only increase (typically cumulative data)

Link to original

Non-Linear Regression Models Non-Linear Regression Models

Non-Linear Regression Models - takes an input vector 𝑥∊ℝ^𝑛 as input and predicts the value of a scalar 𝑦∊ℝ as output (whose function/estimator is linear wrt the coefficients {𝜃₀, …, 𝜃_𝑝})

Non-Linear Regression Type	Function Form Example
power	𝑦̂ = 𝜃1𝑥𝜃2
weibull growth	𝑦̂ = 𝜃1 + (𝜃2- 𝜃1)·𝑒𝑥𝑝(-𝜃3𝑥𝜃4)
fourier	𝑦̂ = 𝜃0 + 𝜃1𝑐𝑜𝑠(𝑥 + 𝜃4) + 𝜃2𝑐𝑜𝑠(2𝑥 + 𝜃4)

Link to original