／var／logmarcus chiu

Cauchy-Schwarz Inequality

Created on Β· Last Modified on

Cauchy-Schwarz Inequality (For 2 Vectors)

For any 2 vectors 𝑒,π‘£βˆŠπ‘‰ of an inner product space (𝑉,𝐹,⟨·,·⟩):

where:

  • ⟨·,·⟩ is the inner product
  • ||Β·|| is the norm which is defined as:Β ||Β·||⟨·,·⟩ = √⟨·,·⟩

Proof

Cauchy-Schwarz Inequality (For 2 Random Variables)

For any 2 random variables 𝑋 and π‘Œ, if π‘Œ=𝛼𝑋 for any scalar constant π›ΌβˆŠβ„ then the following holds:

Proof

Define a random variable:

  • π‘Š = (π‘Œ - 𝛼𝑋)2

π‘Š is a non-negative random variable for any value π›ΌβˆŠβ„, thus:

  • 0 ≀ 𝐄[π‘Š]
  • 0 ≀ 𝐄[(π‘Œ - 𝛼𝑋)2]
  • 0 ≀ 𝐄[(π‘Œ - 𝛼𝑋)(π‘Œ - 𝛼𝑋)]
  • 0 ≀ 𝐄[π‘Œ2 - 2π›Όπ‘Œπ‘‹ + 𝛼2𝑋2]
  • 0 ≀ 𝐄[π‘Œ2] - 𝐄[2π›Όπ‘Œ??] + 𝐄[𝛼2𝑋2]

Let 𝑓(π‘₯) = 𝐄[π‘Œ2] - 𝐄[2π›Όπ‘Œπ‘‹] + 𝐄[𝛼2𝑋2],Β then we know that 𝑓(π‘₯) β‰₯ 0 for allΒ π›ΌβˆŠβ„.

Moreover, if 𝑓(π‘₯) = 0 for some 𝛼, then we have 𝐄[π‘Š] = 𝐄[(π‘Œ - 𝛼𝑋)2] = 0, which essentially means π‘Œ = 𝛼𝑋 with probability one

To prove the Cauchy-Schwarz Inequality, choose:

  • 𝛼 = 𝐄[π‘Œπ‘‹] / 𝐄[𝑋2]

we obtain:

  • 0 ≀ 𝐄[π‘Œ2] - 𝐄[2π›Όπ‘Œπ‘‹] + 𝐄[𝛼2𝑋2]
  • 0 ≀ 𝐄[??2] - 2𝛼𝐄[π‘Œπ‘‹] + 𝛼2𝐄[𝑋2]
  • 0 ≀ 𝐄[π‘Œ2] - 2(𝐄[π‘Œπ‘‹]/𝐄[𝑋2])𝐄[π‘Œπ‘‹] + [(𝐄[π‘Œπ‘‹])2/𝐄[𝑋2])2]𝐄[𝑋2]
  • 0 ≀ 𝐄[??2] - 2𝐄[π‘Œπ‘‹]2/𝐄[??2] + (𝐄[π‘Œπ‘‹])2/𝐄[𝑋2])
  • 0 ≀ 𝐄[??2] - 𝐄[π‘Œπ‘‹]2/𝐄[𝑋2]

thus

  • 𝐄[π‘Œπ‘‹]2/𝐄[𝑋2] ≀ 𝐄[π‘Œ2]
  • 𝐄[π‘Œπ‘‹]2≀ 𝐄[π‘Œ2]𝐄[𝑋2]
  • |𝐄[π‘Œπ‘‹]| ≀ π‘ π‘žπ‘Ÿπ‘‘(𝐄[π‘Œ2]𝐄[𝑋2])Β # hence Cauchy-Schwarz Inequality proved

also if |𝐄[π‘Œπ‘‹]| = π‘ π‘žπ‘Ÿπ‘‘(𝐄[π‘Œ2]𝐄[𝑋2]), we conclude that 𝑓(𝐄[π‘Œπ‘‹]/𝐄[𝑋2]) = 0, which implies π‘Œ = (𝐄[π‘Œπ‘‹]/𝐄[𝑋2])·𝑋 with probability one