Univariate Normal Distribution Derivation

Normal Distribution = 𝑓(𝑋=𝑥) = (1/[𝜎*𝑠𝑞𝑟𝑡(2𝜋)])·(𝑒^{-(𝑥-𝜇)²/(2𝜎²)})

Basic Assumptions

1. deviates from the center become less likely moving away from it
2. deviates from the center along one direction (for example, horizontally) don’t condition ones along the perpendicular direction (vertically)
3. deviates do not depend on the orientation of the coordinate system

Determining the Shape of Distribution

The probability at point (𝑥,𝑦) is some joint probability distribution 𝐏(𝑥,𝑦)

By the independence assumption:

• 𝐏(𝑥,𝑦) = 𝐏(𝑥)𝐏(𝑦)

By the orientation-free assumption:

• 𝐏(𝑥)𝐏(𝑦) = 𝐆(𝑟), where 𝑟 is the distance from the center

By differentiating both sides w.r.t. 𝜃

• 𝐏(𝑥)·[𝑑𝐏(𝑦)/𝑑𝜃] + 𝐏(𝑦)·[𝑑𝐏(𝑥)/𝑑𝜃] = 𝑑𝐆(𝑟)/𝑑𝜃 = 0
• 𝐏(𝑥)·[𝑑𝐏(𝑦)/𝑑𝑦]·[𝑑𝑦/𝑑𝜃] + 𝐏(𝑦)·[𝑑𝐏(𝑥)/𝑑𝑥]·[𝑑𝑥/𝑑𝜃] = 0 𝑑𝑦/𝑑𝑦 = 1
• 𝐏(𝑥)·𝐏’(𝑦)·[𝑑𝑦/𝑑𝜃] + 𝐏(𝑦)·𝐏’(𝑥)·[𝑑𝑥/𝑑𝜃] = 0 simplified syntax 𝑑𝐏(𝑦)/𝑑𝑦 = 𝐏’(𝑦)

Using 𝑥 = 𝑟·𝑐𝑜𝑠(𝜃) and 𝑦 = 𝑟·𝑠𝑖𝑛(𝜃), we can rewrite the derivatives above:

• 𝐏(𝑥)𝐏’(𝑦)[𝑑𝑟·𝑠𝑖𝑛(𝜃)/𝑑𝜃] + 𝐏(𝑦)𝐏’(𝑥)[𝑑𝑟·𝑐𝑜𝑠(𝜃)/𝑑𝜃] = 0
• 𝐏(𝑥)𝐏’(𝑦)(𝑟·𝑐𝑜𝑠(𝜃)) + 𝐏(𝑦)𝐏’(𝑥)(-𝑟·𝑠𝑖𝑛(𝜃)) = 0
• 𝐏(𝑥)𝐏’(𝑦)𝑥 - 𝐏(𝑦)𝐏’(𝑥)𝑦 = 0
• 𝐏(𝑥)𝐏’(𝑦)𝑥 = 𝐏(𝑦)𝐏’(𝑥)𝑦
• 𝐏’(𝑥) / [𝐏(𝑥)𝑥] = 𝐏’(𝑦) / [𝐏(𝑦)𝑦]

This differential equation is true for any 𝑥 and 𝑦, and 𝑥 and 𝑦 are independent. This can only happen if the ratio defined is a constant:

• 𝐏’(𝑥) / [𝐏(𝑥)𝑥] = 𝐏’(𝑦) / [𝐏(𝑦)𝑦] = 𝐶

Solving:

• 𝐏’(𝑥) / [𝐏(𝑥)𝑥] = 𝐶
• 𝐏’(𝑥)/𝐏(𝑥) = 𝐶𝑥
• 𝐏’(𝑥)/𝐏(𝑥) = 𝐶𝑥
• ∫𝐏’(𝑥)/𝐏(𝑥)·𝑑𝑥 = ∫𝐶𝑥·𝑑𝑥
• 𝑙𝑛𝐏(𝑥) = (𝐶/2)𝑥² + 𝑐
• 𝐏(𝑥) = 𝑒^{(𝐶/2)𝑥² + 𝑐} + 𝑐
• 𝐏(𝑥) = 𝑒^𝑐𝑒^{(𝐶/2)𝑥²}
• 𝐏(𝑥) = 𝐴𝑒^{(𝐶/2)𝑥²}

By assumption, large errors are less likely than small errors:

• 𝐏(𝑥) = 𝐴𝑒^{(-𝑘/2)𝑥²}, with 𝑘 positive

Determining Coefficient 𝐴

From previous:

• 𝐏(𝑥) = 𝐴𝑒^{(-𝑘/2)𝑥²}, with 𝑘 positive

For 𝐏(𝑥) to be a probability distribution, the total area under the curve must equal 1. We need to determine the value of 𝐴 that fits this criterion

• _-∞∫^+∞𝐴𝑒^{(-𝑘/2)𝑥²}𝑑𝑥 = 1
• _-∞∫^+∞𝑒^{(-𝑘/2)𝑥²}𝑑𝑥 = 1/𝐴
• ₀∫^+∞𝑒^{(-𝑘/2)𝑥²}𝑑𝑥 = 1/(2𝐴) because the symmetry of function
• (₀∫^+∞𝑒^{(-𝑘/2)𝑥²}𝑑𝑥) · (₀∫^+∞𝑒^{(-𝑘/2)𝑦²}𝑑𝑦) = 1/(4𝐴²) because 𝑥 and 𝑦 are just dummy variables
• ₀∫^+∞₀∫^+∞𝑒^{(-𝑘/2)(𝑥²+𝑦²)}𝑑𝑥𝑑𝑦 = 1/(4𝐴²) because 𝑥 and 𝑦 are independent
• ₀∫^𝜋/2₀∫^+∞𝑒^{(-𝑘/2)𝑟²}𝑟 𝑑𝑟𝑑𝜃 = 1/(4𝐴²) in polar coordinates

To evaluate the polar form requires a u-substitution in an improper integral. Performing the integration with respect to 𝑟, we have

• ₀∫^𝜋/2₀∫^+∞𝑒^{(-𝑘/2)𝑟²}𝑟 𝑑𝑟𝑑𝜃 = 1/(4𝐴²)
  • 𝑢 = (-𝑘/2)𝑟²
  • 𝑑𝑢 = -𝑘𝑟 𝑑𝑟
• ₀∫^𝜋/2₀∫^+∞𝑒^{(-𝑘/2)𝑟²}𝑟(-𝑘/-𝑘) 𝑑𝑟𝑑𝜃 = 1/(4𝐴²)
• ₀∫^𝜋/2(1/-𝑘)₀∫^+∞𝑒^{(-𝑘/2)𝑟²}𝑟(-𝑘) 𝑑𝑟𝑑𝜃 = 1/(4𝐴²)
• ₀∫^𝜋/2(1/-𝑘)₀∫^-∞𝑒^𝑢𝑑𝑢 𝑑𝜃 = 1/(4𝐴²)
  • ₀∫^-∞𝑒^𝑢𝑑𝑢
  • 𝑒^𝑢₀|^-∞
  • 𝑒^-∞ - 𝑒⁰
  • 0 - 1
  • -1
• ₀∫^𝜋/2(1/-𝑘)(-1) 𝑑𝜃 = 1/(4𝐴²)
• ₀∫^𝜋/2(1/𝑘)𝑑𝜃 = 1/(4𝐴²)
• (1/𝑘)₀∫^𝜋/2𝑑𝜃 = 1/(4𝐴²)
• (1/𝑘) [𝜃]₀|^𝜋/2 = 1/(4𝐴²)
• (1/𝑘) [𝜋/2 - 0] = 1/(4𝐴²)
• 𝜋/(2𝑘) = 1/(4𝐴²)
• 𝜋/𝑘 = 1/(2𝐴²)
• (2𝐴²)𝜋 = 𝑘
• 𝐴² = 𝑘/(2𝜋)
• 𝐴 = √𝑘/(2𝜋)]

Thus

• 𝐏(𝑥) = √𝑘/(2𝜋)] · 𝑒^{(-𝑘/2)𝑥²}

Determining Coefficient 𝑘

From previous:

• 𝐏(𝑥) = √𝑘/(2𝜋)] · 𝑒^{(-𝑘/2)𝑥²}

A question often asked about probability distributions is “what are the mean and variance of the distribution?” Perhaps the value of 𝑘 has something to do with the answer to these questions.

• The mean, 𝜇, is defined to be the value of the integral _-∞∫^+∞𝑥𝐏(𝑥)𝑑𝑥
• The variance, 𝜎², is defined to be the value of the integral _-∞∫^+∞(𝑥-𝜇)²𝐏(𝑥)𝑑𝑥

Since 𝑥𝐏(𝑥) is an odd function, we know the mean is zero. The value of the variance needs further computation.

• _-∞∫^+∞𝑥²𝐏(𝑥)𝑑𝑥 = 𝜎²
• 2₀∫^+∞𝑥²𝐏(𝑥)𝑑𝑥 = 𝜎²
• 2√𝑘/(2𝜋)] ₀∫^+∞𝑥²𝑒^{(-𝑘/2)𝑥²}𝑑𝑥 = 𝜎²
  • Integration by parts states: ∫𝑢·𝑑𝑣 = 𝑢·𝑣 - ∫𝑣·𝑑𝑢
    • 𝑢 = 𝑥
    • 𝑑𝑢 = 1
    • 𝑣 = -(1/𝑘)𝑒^{(-𝑘/2)𝑥²}
    • 𝑑𝑣 = 𝑥𝑒^{(-𝑘/2)𝑥²}
  • Thus:
    • [-(𝑥/𝑘)𝑒^{(-𝑘/2)𝑥²}]₀|^+∞ + (1/𝑘)₀∫^+∞𝑒^{(-𝑘/2)𝑥²}𝑑𝑥
    • 0 + (1/𝑘)₀∫^+∞𝑒^{(-𝑘/2)𝑥²}𝑑𝑥
    • (1/𝑘)₀∫^+∞𝑒^{(-𝑘/2)𝑥²}𝑑𝑥
    • (1/𝑘) 1/(2𝐴)
    • (1/𝑘) 1/(2√𝑘/(2𝜋)])
• 2√𝑘/(2𝜋)] ₀∫^+∞𝑥²𝑒^{(-𝑘/2)𝑥²}𝑑𝑥 = 𝜎²
• 2√𝑘/(2𝜋)] (1/𝑘) 1/(2√𝑘/(2𝜋)]) = 𝜎²
• (1/𝑘) = 𝜎²
• 𝑘 = 1/𝜎²

Thus:

• 𝐏(𝑥) = √1/(2𝜋𝜎²)] · 𝑒^{-1/(2𝜎²)𝑥²}
• 𝐏(𝑥) = 1/[𝜎√2𝜋)] · 𝑒^{-(1/2)(𝑥/𝜎)²}

Resources

• normal-distribution-derivation.pdf
• https://math.stackexchange.com/questions/384893/how-was-the-normal-distribution-derived