• the encoder tries to compute the probability distribution 𝑞_𝜙(𝑧|𝑥) where 𝜙 represents the weights of the encoder
• the decoder tries to compute the probability distribution 𝑝_𝜃(𝑥|𝑧) where 𝜃 represents the weights of the decoder

The loss function 𝐿 is defined as:

• 𝐿(𝜙, 𝜃, 𝑥) = (𝑟𝑒𝑐𝑜𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛 𝑙𝑜𝑠𝑠) + (𝑟𝑒𝑔𝑢𝑙𝑎𝑟𝑖𝑧𝑎𝑡𝑖𝑜𝑛 𝑡𝑒𝑟𝑚)

where:

• 𝑟𝑒𝑐𝑜𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛 𝑙𝑜𝑠𝑠 = 𝑙𝑜𝑔-𝑙𝑖𝑘𝑒𝑙𝑖ℎ𝑜𝑜𝑑 OR ||𝑥 - 𝑥̂||²
• 𝑟𝑒𝑔𝑢𝑙𝑎𝑟𝑖𝑧𝑎𝑡𝑖𝑜𝑛 𝑡𝑒𝑟𝑚 = 𝐷_𝐾𝐿( 𝑞_𝜙(𝑧|𝑥) || 𝑞(𝑧) )
  • 𝐷_𝐾𝐿( || ) - is the KL Divergence which measures the distance between 2 probability distributions
  • 𝑞_𝜙(𝑧|𝑥) - inferred latent distribution
  • 𝑞(𝑧) - fixed prior on latent distribution (usually a standard distribution)