given 𝒙 = {𝑥1, …, 𝑥𝑛}

Min-Max Normalization

  • 𝑥𝑖 ← [𝑥𝑖 - 𝑚𝑖𝑛(𝒙)] / [𝑚𝑎𝑥(𝒙) - 𝑚𝑖𝑛(𝒙)]

Mean Normalization

  • 𝑥𝑖 ← [𝑥𝑖 - 𝜇] / [𝑚𝑎𝑥(𝒙) - 𝑚𝑖𝑛(𝒙)]

Z-Score Normalization
Standardization

  • 𝑥𝑖 ← [𝑥𝑖 - 𝜇] / 𝜎
  • transform all variables to have zero-mean and same standard deviation = 1

where:

  • 𝜇 - mean
  • 𝜎 - standard deviation

Z-Score Normalization Advantages

  • standardized variables are centered at 0 so that we don’t need to compute the y-axis intercept for linear regression
  • in gradient descent learning algorithms learns the model coefficients “equally” in multiple linear regression
  • the slope is then exactly the same as the correlation coefficient, which saves another computational step