Law of Total Probability

Law of Total Probability

• if {𝑌₁, 𝑌₂, …, 𝑌_𝑛} are mutually disjoint events in the sample space 𝛺 and ⋃_𝑖𝑌_𝑖 = 𝛺, then:
  • 𝐏(𝑋) = 𝛴_{1≤𝑖≤𝑛}𝐏(𝑋,𝑌_𝑖) = 𝛴_{1≤𝑖≤𝑛}𝐏(𝑋|𝑌_𝑖)𝐏(𝑌_𝑖)
• in other words: the marginal probability 𝐏(𝑋) is the weighted average of the conditional probabilities 𝐏(𝑋|𝑌_𝑖) weighted by 𝐏(𝑌_𝑖)

• proof

  Prove that 𝐏(𝑋) = 𝛴_{1≤𝑖≤𝑛}𝐏(𝑋,𝑌_𝑖):

  • 𝛴_{1≤𝑖≤𝑛}𝐏(𝑋,𝑌_𝑖) = 𝛴_{1≤𝑖≤𝑛}𝐏(𝑌_𝑖|𝑋)𝐏(𝑋) # by baye’s theorem
  • 𝛴_{1≤𝑖≤𝑛}𝐏(𝑋,𝑌_𝑖) = 𝐏(𝑋) 𝛴_{1≤𝑖≤𝑛}𝐏(𝑌_𝑖|𝑋) # multiplication distributes over addition
    • 𝛴_{1≤𝑖≤𝑛}𝐏(𝑌_𝑖|𝑋) = 𝐏(⋃_𝑖𝑌_𝑖|𝑋) # because {𝑌₁, 𝑌₂, …, 𝑌_𝑛} mutually disjoint events
    • 𝛴_{1≤𝑖≤𝑛}𝐏(𝑌_𝑖|𝑋) = 𝐏(𝛺|𝑋) # because ⋃_𝑖𝑌_𝑖 = 𝛺
    • 𝛴_{1≤𝑖≤𝑛}𝐏(𝑌_𝑖|𝑋) = 1 # the probability of 𝛺 occurring given 𝑋 is equal to 1
  • 𝛴_{1≤𝑖≤𝑛}𝐏(𝑋,𝑌_𝑖) = 𝐏(𝑋)

Together With Baye’s Theorem

• 𝐏(𝑍|𝑋) = 𝐏(𝑋|𝑍)𝐏(𝑍) / [𝛴_{1≤𝑖≤𝑛}𝐏(𝑋|𝑌_𝑖)𝐏(𝑌_𝑖)]