Factor graphs

A factor graph draws how a joint probability density factorises into a product of local functions, the factors. ReactiveMP.jl runs all its inference on a factor graph.

Variables and factors

A factor graph has two kinds of node:

• A variable stands for one quantity of the model: a latent quantity you infer, an observation, or a constant.
• A factor node stands for one local function: a prior, a likelihood, a conditional distribution or a deterministic relation. Its edges are its interfaces.

An edge joins a factor node to a variable when the factor's function takes that variable as an argument. Take the model of Getting started, a latent x observed through normal noise:

\[p(x, y) = \underbrace{\mathcal{N}(x \mid 0, 10)}_{f(x)} \; \underbrace{\mathcal{N}(y \mid x, 1)}_{g(x, y)}.\]

It has two factors and two variables:

  [f] ──── (x) ──── [g] ──── (y)

Each factor is local: f takes only x, and g takes only x and y. A message along an edge is therefore computed at one node, from what arrives at that node alone (see Message passing).

The graph in code

In ReactiveMP.jl every argument of a factor is a variable, the known ones included: the mean $0$ and the variances $10$ and $1$ are constants on edges of their own. Both factors are the Gaussian node of the example node, a normal density with interfaces out, μ and v:

x = randomvar(label = :x)
y = datavar(label = :y)

f = factornode(Gaussian, [(:out, x), (:μ, constvar(0.0)), (:v, constvar(10.0))])
g = factornode(Gaussian, [(:out, y), (:μ, x), (:v, constvar(1.0))])

FactorNode Gaussian (stochastic)
  out ── y    data
  μ   ── x    random
  v   ── 1.0  constant
  clusters: (out, μ, v)

A randomvar is inferred, a datavar receives observations and a constvar holds a fixed value (see Variables). A factornode creates a FactorNode, connecting each interface to one variable. The variable x has two edges, one to each factor:

ReactiveMP.degree(x)

2

Stochastic and deterministic factors

A factor node is one of two kinds:

• A stochastic node, Stochastic, is a probability density over its interfaces, such as $\mathcal{N}(y \mid x, 1)$: a prior, a likelihood, or the relation between latent variables.
• A deterministic node, Deterministic, is a function, such as $z = x + c$: it enforces an exact relation and adds no uncertainty.

Gaussian is stochastic. A deterministic node is declared the same way, with type = Deterministic; this one computes out = in + c:

struct Shift end

@define_factor_node(node = Shift, type = Deterministic, interfaces = [:out, :in, :c])

z = randomvar(label = :z)
factornode(Shift, [(:out, z), (:in, x), (:c, constvar(1.0))])

FactorNode Shift (deterministic)
  out ── z    random
  in  ── x    random
  c   ── 1.0  constant
  clusters: (out) (in, c)

The kind decides how messages are computed and how the node enters the free energy. A deterministic node has no average energy. Its clusters are always its output and the joint over its inputs, (out) and (in, c) here. sdtype, isdeterministic and isstochastic read the kind back.

Where nodes come from

The engine defines no node. A node is declared with @define_factor_node from MessagePassingRulesBase, as Shift is above. The declaration names the node, its kind and its interfaces, the first being the output. It is data, which draws itself:

MessagePassingRulesBase.nodespec(Gaussian)

The declaration gives the node its structure only. Its message update rules, marginal rules and average energy are defined next to it with the same package's macros, such as @define_message_update_rule; the example node shows them for Gaussian. The standard nodes, for the common distributions, arithmetic, logic and mixtures, are in StandardMessagePassingRules, and the others in the packages of the ecosystem.

Next steps

• Variables: the three kinds of variable and how they work.
• Message passing: how information flows through the graph.
• Inference lifecycle: the phases of building and running inference.