Related: Gibbs Distribution

Probabilistic Graphical Models & Gibbs Distribution

A probability distribution 𝐏_𝐅(𝐗) is a Gibbs Distribution over a graphical model 𝒢 = ⟨𝐗, 𝐃, 𝐒, 𝐅, 𝐂⟩ if it can be written as

• 𝐏_𝐅(𝐗) = (1/𝘡) * 𝛱_{1≤𝑖≤𝑚}[ 𝐹_𝑖(𝑆_𝑖) ]

where:

• the set of variables (𝑆_𝑖) in each factor 𝐹_𝑖form a clique in the primal graph of 𝒢

• graphical model syntax

  A graphical model 𝒢 is a tuple 𝒢 = ⟨𝐗, 𝐃, 𝐒, 𝐅, 𝐂⟩ where:

  • 𝐗 = {𝑋₁, …, 𝑋_𝑛} set of ordered variables
  • 𝐃 = {𝐷₁, …, 𝐷_𝑛} set of corresponding domains of each variable 𝑋_𝑖 (e.g. if 𝑋₁is a boolean variable then 𝐷₁= {true, false}). The size of each 𝐷_𝑖corresponds to the cardinality of variable 𝑋_𝑖
  • 𝐒 = {𝑆₁, …, 𝑆_𝑚} set of variable scopes, where each variable scope 𝑆_𝑖is a subset of 𝐗 (i.e. 𝑆_𝑖 ⊆ 𝐗)
  • 𝐅 = {𝐹₁, …, 𝐹_𝑚} set of factors/functions, where each factor/function 𝐹_𝑖 is defined over its corresponding variable scope 𝑆_𝑖and maps any assignment over its scope to a real value
    • in context of Bayesian Networks, 𝐅 = set of conditional probability tables
    • in context of Markov Networks, 𝐅 = set of factors
    • global function - is a function whose scope includes all variables (i.e. 𝑆_𝑖 = 𝐗)
    • local functions - is a function whose scope is a proper subset of variables (i.e. 𝑆_𝑖⊂ 𝐗)
  • 𝐂 is a set of combination operators which defines how functions are combined. common combination operators are:
    • summation operator (𝛴)
    • multiplication operator (𝛱)
    • AND operator (∧) - for Boolean functions
    • relational join operator (⨝) - when the functions are relation
    • marginalization operator - for reasoning queries
    • max operator - e.g. = argmax_𝑦[ 𝐹_𝑖(𝑥,𝑦) ] = 𝐹_𝑗(𝑥) where 𝐹_𝑗is a new function with scope over variable 𝑥

  the set of local functions can be combined in a variety of ways (e.g. combination operators) to generate a new local function or even a global function

  Link to original

Theorem 1: Factorization Implies Conditional Independencies

Click here to expand...

If 𝐏_𝐅(𝐗) is a Gibbs Distribution for 𝒢, then the primal graph of 𝒢 is an I-Map for probability distribution 𝐏(𝐗):

• 𝐈(𝐆) ⊆ 𝐈(𝐏)

proof, suppose:

• 𝐴, 𝐵, and 𝐶 are disjoint sets of variables
• 𝐴 is connected to 𝐵
• 𝐶 is connected to 𝐵
• 𝐵 separates 𝐴 from 𝐶

then we can write:

• 𝐏(𝐴, 𝐵, 𝐶) = (1/𝘡) * 𝐹₁(𝐴,𝐵) * 𝐹₂(𝐵,𝐶)

Theorem 2: Conditional Independencies Implies Factorization

Click here to expand...

If 𝐏(𝐗) is a positive distribution and the primal graph of 𝒢 is an I-Map for 𝐏(𝐗), then 𝐏(𝐗) is a Gibbs Distribution that factorizes over graphical model 𝒢

Subpages

• MRF - Variants (Gibbs Distribution)

  List indent undo

  • PGM - Gibbs Distribution - Bayesian Network