Glossary

The terms the documentation of the ReactiveMP packages uses, in alphabetical order. Every site links here the first time a page uses a term.

Algorithm

The value that selects which rules of a node run, and carries their parameters, such as the number of cubature points of an approximation. Almost every node runs under DefaultAlgorithm, where the factorisation alone decides whether a rule is belief propagation or variational message passing. An algorithm is not an inference scheme. See Algorithms and dependencies.

Average energy

A node's term of the Bethe free energy: the expected negative log-density of the node under the marginals of its clusters, $U[q] = -\mathbb{E}_q[\log f]$. Defined with @define_average_energy.

Belief propagation

Message passing with exact messages, also called the sum-product algorithm. The message from a node towards one of its variables integrates the node's function against the messages on its other edges. It computes exact marginals on a graph without cycles. A rule is a belief propagation rule when it takes only messages.

Bethe free energy

The objective that message passing minimises: the sum of the nodes' average energies minus the entropies of the clusters and variables. At its minimum, the marginals are the posterior (exactly on a tree, approximately otherwise) and its value is an upper bound on $-\log p(\text{data})$.

Cluster

A set of a node's interfaces whose variables share one factor of the approximate posterior. A node's clusters come from the factorisation: under q(out, μ) q(v), a node with interfaces out, μ and v has the clusters (out, μ) and (v,). A rule takes messages from its target's own cluster and marginals of the other clusters.

Default scheme

What a rule receives under DefaultAlgorithm: the messages on the other interfaces of its target's cluster and the marginals of the other clusters. A node that declares nothing else follows it. See Algorithms and dependencies.

Dependencies

The inputs a node's rules take, target by target: m[:x] for a message, q[:x] for a marginal, q[:x, :y] for a joint marginal. The default scheme derives them from the factorisation; a node's own algorithm declares them with @define_dependencies.

Deterministic node

A node whose output is a function of its inputs, out = f(in...), such as +. Its clusters are always its output and the joint over its inputs. See Deterministic.

Expectation propagation

A message passing scheme that approximates each message by projecting the corresponding marginal onto a simpler family, usually by matching moments. A node with its own algorithm builds a node whose rule towards its input is an expectation propagation rule.

Factor graph

A graph of a probabilistic model's factorisation: factor nodes for the factors, variables for the random quantities, and an edge wherever a factor depends on a variable. Inference runs on it by passing messages along the edges.

Factor node

A factor of the model, such as a normal density or a sum. It is declared once with @define_factor_node, which names its interfaces, and it computes the messages towards its variables with its rules.

Factorisation

How the approximate posterior splits into independent factors, for example q(x, y, z) = q(x, y) q(z). It gives each node its clusters. Without a factorisation, every node has one cluster and inference is belief propagation; with every variable in a cluster of its own, it is mean-field variational message passing.

Group

An interface with any number of members, declared with a trailing ..., as in.... A sum's summands and a mixture's components are groups. A rule targets a member as (:in, k) and reads a group as a tuple in member order.

In-place rule

A rule that writes its result into a buffer the caller gives it, to save allocations. Declared with inplace = true.

Initial message

A message a node places on one of its edges before any rule has run, so that a rule reading that edge can start. Declared with initial_messages.

Interface

A named edge of a factor node, such as out, μ or v. The first interface is the output by convention. An interface may have aliases, other names a model can use for it.

Log scale

The logarithm of a message's normalising constant: a message is $\exp(\text{log scale}) \cdot p(x)$ with $p$ a normalised distribution. Summed over a graph, log scales give the model's evidence. Only a message with a finite integral has one: an improper message, which an exact message can be, has none. See Log scales.

Marginal

The current belief about a variable, or about the variables of a cluster jointly: the normalised product of the messages arriving at them. Written q(x), and q[:x] among a rule's inputs.

Mean field

The factorisation that puts every variable in a cluster of its own, q(x, y, z) = q(x) q(y) q(z). Every rule then takes only marginals.

Message

What a node tells one of its variables about it, summarising the rest of the graph on the node's side. Written μ(x), and m[:x] among a rule's inputs.

Point mass

A distribution that puts all its probability on one value: how an observation or a constant enters a rule. PointMass(2.0) comes from BayesBase.

Pushforward

The distribution of f(x) when x has a known distribution. A deterministic node's message towards its output is the pushforward of the messages on its inputs.

Resolution

Finding the rule that runs for a call: the node, the target, the algorithm and the types of the inputs select one rule. It is Julia's method dispatch over the methods the definition macros generate, so it needs no list of rules. When no rule matches, the result is a RuleNotFound. See Calling rules.

Rule

A function that computes a message, a joint marginal or an average energy of a node from its inputs. A rule is found by the node, its target, the algorithm and the types of its inputs. See Defining rules.

Scratch

Working memory a rule keeps between calls, created on its first call. Declared with scratch.

Service

A value a rule needs from whoever runs it, such as a random number generator (rng). A rule declares the services it reads with ctx = (:rng,) and reads them from its RuleContext. See The rule context.

Stochastic node

A node with a probability density over its interfaces, f(out | in...), such as a normal distribution. See Stochastic.

Structured variational message passing

Variational message passing under a factorisation that keeps some variables together in one cluster. Its rules take messages from their own cluster and marginals from the others.

Variational message passing

Message passing that minimises the Bethe free energy under a factorisation. A message from a node towards x is $\exp \mathbb{E}_q[\log f]$, the expectation taken under the marginals of the node's other clusters. It handles models where exact messages have no closed form.