information is a subtle mixture of predictability and surprise

studies the quantification of information both in storage and communication
originally proposed by Claude Shannon in 1948 to find fundamental limits on signal processing and communication operations such as data compression, in a landmark paper titled A Mathematical Theory of Communication

Communication System

A communication system has 5 components:

information source - produces a message or sequence of messages to be communicated to the receiving terminal
transmitter - operates on the message in some way to produce a signal suitable for transmission over the channel
channel - the medium used to transmit the signal from transmitter to receiver
receiver - operates on the signal to produce the message
destination - is the entity for whom the message is intended for

Communication System - Categories/Classes

discrete message - the message is a sequence of discrete symbols (e.g. sequence of letters)
continuous message - the message is treated as a continuous function (e.g. voice)

discrete signal - the signal is a sequence of discrete symbols (e.g. bits in an IP network)
continuous signal - the signal are treated as continuous functions (e.g. electromagnetic waves, specifically radio waves)

discrete system - both the message and signal are discrete (e.g. telegraph; messages are sequences of letters, signals are sequences of dots, dashes, and spaces of dots, dashes, and spaces)
continuous system - both the message and signal are discrete (e.g. radio or television)
mixed system - message and signal is mixed (e.g. PCM transmission of speech)

noiseless channel - an idealistic channel in which no information is lost, corrupted, nor duplicated
noisy channel - a realistic channel in which information can be lost, corrupted, or duplicated

𝑆 is the size of possible symbols such as (letters, words, etc)