／var／logmarcus chiu

Likelihood vs Probability

Created on · Last Modified on

Probability vs Likelihood

Probability

Likelihood

Discrete Example (e.g. Bernoulli Distribution)
TODO

Continuous Example (e.g. Normal Distribution)

𝐏(𝑑𝑎𝑡𝑎|𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛-𝑝𝑎𝑟𝑎𝑚𝑡𝑒𝑟-𝜃)

𝐋(𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛-𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟-𝜃|𝑑𝑎𝑡𝑎)

  • data value varies
  • 𝜃 value is constant
  • data value is constant
  • 𝜃 value varies
  • integrates to 1: given constant value 𝜃, the sum of all probability for each possible data value equals 1
  • does not integrate to 1: given constant value data, the sum of all likelihood values for each possible 𝜃 does not equal to 1

Probability vs Likelihood - Duality

the probability of data conditioned on the value(s) of distribution parameter(s) is equal to the likelihood of the distribution parameter value(s) given the same data 𝐏(𝑑𝑎𝑡𝑎|𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛-𝑝𝑎𝑟𝑎𝑚𝑡𝑒𝑟-𝜃) = 𝐋(𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛-𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟-𝜃|𝑑𝑎𝑡𝑎)

Discrete Example (Bernoulli Distribution)

Consider a coin flip where it falls heads with probability 𝜃:

  • 𝑑𝑎𝑡𝑎 = 0 when tails
  • 𝑑𝑎𝑡𝑎 = 1 when heads

Thus, 𝐏(𝑑𝑎𝑡𝑎=𝑘|𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛-𝑝𝑎𝑟𝑎𝑚𝑡𝑒𝑟-𝜃) = 𝜃𝑘(1 - 𝜃)1-𝑘

Now when:

  • 𝑑𝑎𝑡𝑎=1 ⇒ 𝐏(𝑑𝑎𝑡𝑎=1|𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛-𝑝𝑎𝑟𝑎𝑚𝑡𝑒𝑟-𝜃) = 𝜃 = 𝐋(𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛-𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟-𝜃|𝑑𝑎𝑡𝑎=1)
  • 𝑑𝑎𝑡𝑎=0 ⇒ 𝐏(𝑑𝑎𝑡𝑎=0|𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛-𝑝𝑎𝑟𝑎𝑚𝑡𝑒𝑟-𝜃) = (1 - 𝜃) = 𝐋(𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛-𝑝𝑎𝑟𝑎𝑚𝑒𝑡𝑒𝑟-𝜃|𝑑𝑎𝑡𝑎=0)
Continuous Example

Resources