Covariance - Covariation

Covariance - Formula

Population Covariance Formula

• 𝐶𝑜𝑣(𝑋,𝑌) = 𝐄[(𝑥_𝑖 - 𝜇_𝑥)(𝑦_𝑖 - 𝜇_𝑦)]
• 𝐶𝑜𝑣(𝑋,𝑌) = 𝐄[𝑥_𝑖𝑦_𝑖] - 𝜇_𝑥𝜇_𝑦

• Click here to expand...

  • 𝐶𝑜𝑣(𝑋,𝑌) = 𝐄[𝑥_𝑖𝑦_𝑖 - 𝜇_𝑥𝑦_𝑖 - 𝜇_𝑦𝑥_𝑖 + 𝜇_𝑥𝜇_𝑦]
  • 𝐶𝑜𝑣(𝑋,𝑌) = 𝐄[𝑥_𝑖𝑦_𝑖] - 𝐄[𝜇_𝑥𝑦_𝑖] - 𝐄[𝜇_𝑦𝑥_𝑖] + 𝐄[𝜇_𝑥𝜇_𝑦]
  • 𝐶𝑜𝑣(𝑋,𝑌) = 𝐄[𝑥_𝑖𝑦_𝑖] - 𝜇_𝑥𝐄[𝑦_𝑖] - 𝜇_𝑦𝐄[𝑥_𝑖] + 𝐄[𝜇_𝑥𝜇_𝑦]
  • 𝐶𝑜𝑣(𝑋,𝑌) = 𝐄[𝑥_𝑖𝑦_𝑖] - 𝜇_𝑥𝜇_𝑦 - 𝜇_𝑦𝜇_𝑥 + 𝜇_𝑥𝜇_𝑦
  • 𝐶𝑜𝑣(𝑋,𝑌) = 𝐄[𝑥_𝑖𝑦_𝑖] - 𝜇_𝑥𝜇_𝑦

• 𝐶𝑜𝑣(𝑋,𝑌) = [𝛴_{1≤𝑖≤𝑛}(𝑥_𝑖 - 𝜇_𝑥)(𝑦_𝑖 - 𝜇_𝑦)] / (𝑛)
• 𝐶𝑜𝑣(𝑋,𝑌) = [𝛴_{1≤𝑖≤𝑛}(𝑥_𝑖𝑦_𝑖)]/(𝑛) - 𝜇_𝑥𝜇_𝑦

where:

• 𝜇_𝑥- true average (population mean) of all 𝑥 values
• 𝜇_𝑦- true average (population mean) of all 𝑦 values

Sample Covariance Formula

• 𝐶𝑜𝑣’(𝑋,𝑌) = 𝐄[(𝑥_𝑖 - 𝑥̅)(𝑦_𝑖 - 𝑦̅)]
• 𝐶𝑜𝑣’(𝑋,𝑌) = [𝛴_{1≤𝑖≤𝑛}(𝑥_𝑖 - 𝑥̅)(𝑦_𝑖 - 𝑦̅)] / (𝑛 - 1)
• 𝐶𝑜𝑣’(𝑋,𝑌) = [𝛴_{1≤𝑖≤𝑛}(𝑥_𝑖𝑦_𝑖)]/(𝑛 - 1) - 𝑥̅𝑦̅

where:

• 𝑥̅ - estimated average (sample mean) of all 𝑥 values
• 𝑦̅ - estimated average (sample mean) of all 𝑦 values

Covariance - With Itself is Variance

• 𝐶𝑜𝑣(𝑋,𝑋) = 𝐄[(𝑥_𝑖 - 𝜇_𝑥)(𝑥_𝑖 - 𝜇_𝑥)]
• 𝐶𝑜𝑣(𝑋,𝑋) = [𝛴_{1≤𝑖≤𝑛}(𝑥_𝑖 - 𝜇_𝑥)(𝑥_𝑖 - 𝜇_𝑥)] / (𝑛)
• 𝐶𝑜𝑣(𝑋,𝑋) = [𝛴_{1≤𝑖≤𝑛}(𝑥_𝑖 - 𝜇_𝑥)²] / (𝑛)
• 𝐶𝑜𝑣(𝑋,𝑋) = 𝑉𝑎𝑟(𝑋)

Covariance - Diagrams

Covariance - Properties

• 𝐶𝑜𝑣(𝑋,𝑌) = 𝐄[(𝑋-𝐄[𝑋])(𝑌-𝐄[𝑌])] = 𝐄[𝑋𝑌] - 𝐄[𝑋]𝐄[𝑌]
• 𝐶𝑜𝑣(𝑋,𝑌) = 𝐄[(𝑋-𝜇_𝑋)(𝑌-𝜇_𝑌)] = 𝐄[𝑋𝑌] - 𝜇_𝑋𝜇_𝑌
• 𝐶𝑜𝑣(𝑋,𝑌) = 𝐶𝑜𝑣(𝑌,𝑋)
• 𝐶𝑜𝑣(𝑎𝑋 + 𝑏, 𝑐𝑌 + 𝑑) = 𝑎𝑐𝐶𝑜𝑣(𝑋,𝑌)
• 𝐶𝑜𝑣(𝑎𝑋 + 𝑏𝑌, 𝑐𝑍 + 𝑑𝑊) = 𝑎𝑐𝐶𝑜𝑣(𝑋,𝑍) + 𝑎𝑑𝐶𝑜𝑣(𝑋,𝑊) + 𝑏𝑐𝐶𝑜𝑣(𝑌,𝑍) + 𝑏𝑑𝐶𝑜𝑣(𝑌,𝑊)
• 𝐶𝑜𝑣(𝑋,𝑌) = 0 # for independent 𝑋 and 𝑌

Link to original

Covariance - Other

• Partial Covariance