Empirical/Sample Distribution

Mathematical Definition

let:

• 𝐷 = [𝑥₁, …, 𝑥_𝑛] be a set of 𝑛 samples taken from some true population distribution 𝐏

the empirical distribution 𝐏ˆ_𝐷is defined as:

• 𝐏ˆ_𝐷(𝐴) = (1/|𝐷|) 𝛴_{𝑥𝑖∊𝐷}𝐼(𝑥_𝑖∊𝐴)

where:

• 𝐴 - an event
• 𝐼(𝑥_𝑖∊𝐴) - is an indicator function that is equal to 1 if 𝑥_𝑖is found in 𝐴 and 0 otherwise

note:

• 𝐏ˆ_𝐷is an estimate of true distribution 𝐏
• 𝐏ˆ_𝐷(𝐴) is the probability of the event 𝐴 is simply the fraction of training examples that satisfy 𝐴
• as the number of training examples grows, the empirical distribution approaches the true population distribution 𝐏
  • 𝑙𝑖𝑚_{𝑛→ ∞} 𝐏ˆ_𝐷(𝐴) = 𝐏(𝐴)