Conditional Probability Distribution (CPD)

CPD - Relation to Other Probability Distributions & Baye’s Theorem

see: Baye’s Theorem

• 𝐏(𝑋|𝑌=𝑦) = 𝐏(𝑋,𝑌=𝑦) / 𝐏(𝑌=𝑦)

where:

• 𝐏(𝑋|𝑌=𝑦) - conditional probability
• 𝐏(𝑋,𝑌=𝑦) - unconditional joint probility
• 𝐏(𝑌=𝑦) - unconditional marginal probability

CPD - Continuous vs Discrete

• a Continuous CPD is called a Conditional Probability Density Function which is a type of probability density function
• a Discrete CPD is called a Conditional Probability Mass Function which is a type of probability mass function

CPD - 1-Dimensional (Univariate) vs Multi-Dimensional (Multivariate/Joint)

1-Dimensional/Univariate Conditional Probability Distribution

• 𝐏(𝑋₁=𝑥₁ | 𝐸₁=𝑒₁, …, 𝐸_𝑛=𝑒_𝑛)

𝑛-Dimensional/Multivariate/Joint Conditional Probability Distribution

• 𝐏(𝑋₁=𝑥₁, …, 𝑋_𝑛=𝑥_𝑛 | 𝐸₁=𝑒₁, …, 𝐸_𝑛=𝑒_𝑛)

CPD - Relation to Independence

Random variables 𝑋 and 𝑌 are independent if and only if the conditional distribution of 𝑋 given 𝑌 is, for all possible realizations of 𝑌, equal to the unconditional distribution of 𝑋

for discrete random variables

• 𝐏(𝑋=𝑥|𝑌=𝑦) = 𝐏(𝑋=𝑥) # for all possible 𝑥 and 𝑦  with 𝑃(𝑌=𝑦) > 0

for continuous random variables

• 𝑓_𝑋(𝑥|𝑌=𝑦) = 𝑓_𝑋(𝑥) # for all possible 𝑥 and 𝑦 with 𝑓_𝑌(𝑦) > 0

CPD - Relation to Conditional Independence

if the random variables 𝑋 and 𝑌 are conditionally independent given 𝑍, then:

• 𝐏(𝑋|𝑌,𝑍) = 𝐏(𝑋|𝑍)

discrete vs continuous:

• for discrete random variables
  𝐏(𝑋=𝑥|𝑌=𝑦,𝑍=𝑧) = 𝐏(𝑋=𝑥|𝑍=𝑧) for all possible values: 𝑥, 𝑦, and 𝑧
• for continuous random variables
• 𝑓_𝑋(𝑥|𝑌=𝑦,𝑍=𝑧) = 𝑓_𝑋(𝑥|𝑍=𝑧) for all possible values: 𝑥, 𝑦 and 𝑧

CPD - Representations

• Conditional Probability Table (CPT)- multiple CPTs are used in Bayesian Networks
• Tree-CPD - can be used to represent a full CPT or simpler models
• Logistic-Based CPD
• Noisy-OR CPD
• Deterministic CPD
• Rule-Based CPD
• Nueral-Network CPD

other:

• Mixing Discrete & Continuous Variables - also used in Bayesian Networks