Formula

The capacity 𝐶 of a discrete channel is given by:

• 𝐶 = 𝑙𝑖𝑚_𝑇→∞[ 𝑙𝑜𝑔𝑁(𝑇) / 𝑇 ]

where:

• 𝑁(𝑇) - is the number of allowed signals of duration 𝑇

Info

It can be shown that the limit will exist as a finite number in most cases
[!info]

𝐶 = 𝑙𝑖𝑚_𝑇→∞[ 𝑙𝑜𝑔₂𝑁(𝑇) / 𝑇 ] measures capacity in binary digits (bits)
𝐶 = 𝑙𝑖𝑚_𝑇→∞[ 𝑙𝑜𝑔₁₀𝑁(𝑇) / 𝑇 ] measures capacity in decimal digits

Solving 𝑁(𝑇) for Large 𝑇

Suppose all sequences of symbols {𝑆₁, 𝑆₂, …, 𝑆_𝑛} are allowed and these symbols have duration {𝑡₁, 𝑡₂, …, 𝑡_𝑛}. What is the channel capacity? If 𝑁(𝑡) represents the number of sequences of duration 𝑡 we have:

• 𝑁(𝑡) = 𝑁(𝑡₁) + 𝑁(𝑡₂) + … + 𝑁(𝑡_𝑛)

The total number is equal to the sum of the number of sequences ending in 𝑆₁, 𝑆₂, …, 𝑆_𝑛 and these are 𝑁(𝑡 - 𝑡₁), 𝑁(𝑡 - 𝑡₂), …, 𝑁(𝑡 - 𝑡_𝑛) respectively. According to a well-known result in finite differences, 𝑁(𝑡) is then asymptotic for large 𝑡 to (𝑢_𝑜)^𝑡 where 𝑢_𝑜 is the largest real solution of the characteristic equation:

• 𝑢^-𝑡₁+ 𝑢^-𝑡₂+ … + 𝑢^-𝑡ₙ= 1

Thus

• 𝐶 = 𝑙𝑜𝑔(𝑢_𝑜)

Calculating Channel Capacity Examples

teletype example

Assume in a communications system:

• there are 32 distinct symbols (thus, each symbol represents 5 bits of information)
• each symbol have the same duration of (1/𝑛), in other words, the system transmits 𝑛 symbols per unit-time
• a signal is allowed to have any permutation of symbols

Intuitively, the channel capacity is 5𝑛 bits per unit-time.

When solving it via the formula we should get the same result

𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑙𝑜𝑔2𝑁(𝑇) / 𝑇 ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑙𝑜𝑔2(number-of-symbols^(𝑇/symbol-duration)) / 𝑇 ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑙𝑜𝑔2(32^(𝑇𝑛)) / 𝑇 ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[ (𝑇𝑛) * 𝑙𝑜𝑔2(32) / 𝑇 ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[  𝑛 * 𝑙𝑜𝑔2(32) ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[  𝑛 * 𝑙𝑜𝑔2(32) ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[  𝑛5 ] 𝐶 = 𝑛5

𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑙𝑜𝑔2𝑁(𝑇) / 𝑇 ] 𝑁(𝑇) = 𝑁(𝑇 - 𝑡1) + 𝑁(𝑇 - 𝑡2) + … + 𝑁(𝑇 - 𝑡32) 𝑢-𝑡₁+ 𝑢-𝑡₂+ … + 𝑢-𝑡₃₂ = 1 𝑢-(1/𝑛)+ 𝑢-(1/𝑛)+ … + 𝑢-(1/𝑛) = 1 32𝑢-(1/𝑛)= 1 𝑢-(1/𝑛)= 1/32 𝑢(1/𝑛)= 32 𝑢= 32𝑛 thus, 𝑁(𝑇) = (32𝑛)𝑇 𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑙𝑜𝑔2[(32𝑛)𝑇 ] / 𝑇 ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑇𝑛 𝑙𝑜𝑔2[32] / 𝑇 ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑛5 ] 𝐶 = 𝑛5

Thus, channel capacity is 5𝑛 bits per unit-time

telegraphy example

Assume in a communications system:

• allowed symbols:
  • a dot - consisting of a unit-time closure followed by a unit-time open
  • a dash - consisting of 3 unit-time closure followed by a unit-time open
  • a letter space - consisting of 3 unit-time open
  • a word space - consisting of 6 unit-time open
• allowable sequences:
  • no spaces follow each other

When solving it via the formula

𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑙𝑜𝑔2𝑁(𝑇) / 𝑇 ] 𝑁(𝑡) = 𝑁(𝑡 - dot-duration) + 𝑁(𝑡 - dash-duration) + 𝑁(𝑡 - dot-letter-space-duration) + 𝑁(𝑡 - dash-letter-space-duration) + 𝑁(𝑡 - dot-word-space-duration) + 𝑁(𝑡 - dash-word-space-duration) 𝑁(𝑡) = 𝑁(𝑡 - 2) + 𝑁(𝑡 - 4) + 𝑁(𝑡 - 5) + 𝑁(𝑡 - 7) + 𝑁(𝑡 - 8) + 𝑁(𝑡 - 10) 𝑢-2+ 𝑢-4+ 𝑢-5+ 𝑢-7+ 𝑢-8 + 𝑢-10= 1 the roots of the characteristic equation above are: 𝑢 ≈ -1.15909 𝑢 ≈ 1.4529 we choose the largest root 𝑢 ≈ 1.4529 𝑁(𝑇) = (1.4529)𝑇 𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑙𝑜𝑔2[(1.4529)𝑇] / 𝑇 ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑇 𝑙𝑜𝑔2[1.4529] / 𝑇 ] 𝐶 = 𝑙𝑖𝑚𝑇→∞[ 𝑙𝑜𝑔2[1.4529] ] 𝐶 = 𝑙𝑜𝑔2[1.4529] 𝐶 = 0.53893541

Thus, channel capacity is 0.53893541 bits per unit-time