Nodes implementation

In the message passing framework, one of the most important concepts is a factor node. A factor node represents a local function in a factorised representation of a generative model.

ReactiveMP.@node — Macro

@node(fformtype, sdtype, interfaces_list)

@node macro creates a node for a fformtype type object. To obtain a list of predefined nodes use ?is_predefined_node.

Arguments

fformtype: Either an existing type identifier, e.g. Normal or a function type identifier, e.g. typeof(+)
sdtype: Either Stochastic or Deterministic. Defines the type of the functional relationship
interfaces_list: Defines a fixed list of edges of a factor node, by convention the first element should be out. Example: [ out, mean, variance ]

Note: interfaces_list must not include names that contain _ symbol in them, as it is reserved to identify joint posteriors around the node object.

Examples


struct MyNormalDistribution
    mean :: Float64
    var  :: Float64
end

@node MyNormalDistribution Stochastic [ out, mean, var ]


@node typeof(+) Deterministic [ out, in1, in2 ]

List of available nodes

See ?is_predefined_node.

source

ReactiveMP.FactorNode — Type

GenericFactorNode(functionalform, interfaces)

Generic factor node object that represents a factor node with a given functionalform and interfaces.

source

ReactiveMP.FactorNodeLocalMarginal — Type

FactorNodeLocalMarginal

This object represents local marginals for some specific factor node. The local marginal can be joint in case of structured factorisation. Local to factor node marginal also can be shared with a corresponding marginal of some random variable.

source

Interfaces

Every edge of a factor node — a connection to one variable — is represented by a ReactiveMP.NodeInterface. When a FactorNode is constructed, one NodeInterface is created per edge. The constructor of NodeInterface immediately calls ReactiveMP.create_new_stream_of_inbound_messages! on the connected variable, which allocates a per-connection ReactiveMP.MessageObservable slot in the variable's input_messages and returns it. This observable is stored as m_out on the interface: it is the outbound message from the node's perspective (flowing toward the variable) and the inbound message from the variable's perspective.

At construction time all message streams are unconnected (lazy). The actual rule computations are wired up later during graph activation (see Activation).

For nodes with a variable-length list of same-named edges (e.g. the means of a Gaussian Mixture node), ReactiveMP.IndexedNodeInterface wraps a NodeInterface and adds a positional index. The ReactiveMP.ManyOf container collects the corresponding streams for use in @rule dispatch; see the Delta node documentation for usage examples.

ReactiveMP.NodeInterface — Type

ReactiveMP.NodeInterface

Represents a single directed connection between a factor node and an ReactiveMP.AbstractVariable.

Each interface owns one ReactiveMP.MessageObservable (m_out) — the outbound message stream from this node toward the connected variable. The constructor immediately calls ReactiveMP.create_new_stream_of_inbound_messages! on the variable, which allocates a per-connection slot in the variable's input_messages and returns the same observable together with its index. This means m_out for the interface is the inbound message stream from the variable's perspective.

After graph construction the streams are unconnected (lazy). ReactiveMP.activate! wires m_out to the result of the message update rule via ReactiveMP.set_stream_of_outbound_messages!.

source

ReactiveMP.IndexedNodeInterface — Type

ReactiveMP.IndexedNodeInterface

A thin wrapper around ReactiveMP.NodeInterface that adds a positional index, used for nodes with a variable-length list of same-named edges (e.g. the means or precisions of a Gaussian Mixture node). All stream and variable accessors delegate to the wrapped interface.

source

ReactiveMP.get_stream_of_inbound_messages — Function

get_stream_of_inbound_messages(interface)

Returns an inbound messages stream from the given interface.

source

ReactiveMP.get_stream_of_outbound_messages — Function

get_stream_of_outbound_messages(interface)

Returns an outbound messages stream from the given interface.

source

ReactiveMP.set_stream_of_outbound_messages! — Function

ReactiveMP.set_stream_of_outbound_messages!(interface, stream)

Connects stream to the outbound message observable of interface. See also ReactiveMP.get_stream_of_outbound_messages, ReactiveMP.get_stream_of_inbound_messages.

source

ReactiveMP.tag — Function

tag(interface)

Returns a tag of the interface in the form of Val{ name(interface) }. The major difference between tag and name is that it is possible to dispath on interface's tag in message computation rule.

source

ReactiveMP.name — Function

name(interface)

Returns a name of the interface.

source

ReactiveMP.interfaces — Function

interfaces(fform)

Returns a Val object with a tuple of interface names for a given factor node type. Returns nothing for unknown functional form.

source

ReactiveMP.getvariable — Function

getvariable(interface)

Returns a variable connected to the given interface.

source

ReactiveMP.inputinterfaces — Function

inputinterfaces(fform)

Similar to interfaces, but returns a Val object with a tuple of input interface names for a given factor node type. Returns nothing for unknown functional form.

source

ReactiveMP.alias_interface — Function

alias_interface(factor_type, index, name)

Converts the given name to a correct interface name for a given factor node type and index.

source

Activation

Graph activation is the step that connects all lazy ReactiveMP.MessageObservable and ReactiveMP.MarginalObservable streams into a live reactive network. For factor nodes this is done by calling ReactiveMP.activate! with a ReactiveMP.FactorNodeActivationOptions that bundles all inference-time configuration.

ReactiveMP.FactorNodeActivationOptions — Type

ReactiveMP.FactorNodeActivationOptions

Collects all configuration needed to activate a FactorNode. Passed to ReactiveMP.activate!(::FactorNode, ::FactorNodeActivationOptions).

Fields:

metadata — node-specific metadata forwarded to message update rules (see ReactiveMP.collect_meta)
dependencies — a ReactiveMP.FunctionalDependencies policy that determines which messages and marginals each outbound message computation depends on (default: ReactiveMP.DefaultFunctionalDependencies)
postprocessor — optional stream postprocessor applied to every created stream (see ReactiveMP.AbstractStreamPostprocessor)
annotations — optional annotation processors (see ReactiveMP.AbstractAnnotations)
rulefallback — optional fallback called when no @rule method matches
callbacks — optional callbacks invoked at key points in the message computation (see ReactiveMP.invoke_callback)

source

ReactiveMP.activate! — Method

ReactiveMP.activate!(factornode::FactorNode, options::FactorNodeActivationOptions)

Wires all reactive message-passing streams of a FactorNode into the factor graph.

Activation proceeds in three phases:

Collect functional dependencies — calls ReactiveMP.collect_functional_dependencies to determine, for each interface, which inbound messages and local marginals are needed to compute the outbound message. The policy is controlled by options.dependencies (default: ReactiveMP.DefaultFunctionalDependencies).
Initialize clusters — sets up a ReactiveMP.FactorNodeLocalMarginal stream for each cluster in the factorization. For mean-field each cluster contains one variable and its local marginal is shared directly with the variable's marginal stream. For structured factorizations a joint marginal stream is created from the cluster's message dependencies.
Wire outbound message streams — for every interface connected to a ReactiveMP.RandomVariable or ReactiveMP.DataVariable:
- combines the required inbound message and marginal streams with combineLatest
- wraps the rule application in a ReactiveMP.MessageMapping, producing a ReactiveMP.DeferredMessage on each upstream update
- applies any options.postprocessor transformations
- connects the result to the interface's ReactiveMP.MessageObservable via ReactiveMP.set_stream_of_outbound_messages!

Interfaces connected to ReactiveMP.ConstVariable are skipped: their message is fixed at graph construction.

source

Adding a custom node

ReactiveMP.jl exports the @node macro that allows for quick definition of a factor node with a fixed number of edges. The example application can be the following:

struct MyNewCustomNode end

@node MyNewCustomNode   Stochastic         [ x, y, (z, aliases = [ d ] ) ]
#     ^^^^^^^^^^^^^^^   ^^^^^^^^^^^^^      ^^^^^^^^^^^
#     Node's tag/name   Node's type        A fixed set of edges
#                       Another possible   The very first edge (in this example `x`) is considered
#                       value is           to be the output of the node
#                       `Deterministic`    - Edges can have aliases, e.g. `z` can be both `z` or `d`

This expression registers a new node that can be used with the inference engine. Note, however, that the @node macro does not generate any message passing update rules. These must be defined using the @rule macro.

Collecting node properties

ReactiveMP.collect_factorisation — Function

collect_factorisation(nodetype, factorisation)

This function converts given factorisation to a correct internal factorisation representation for a given node.

source

ReactiveMP.collect_meta — Function

collect_meta(nodetype, meta)

This function converts given meta object to a correct internal meta representation for a given node. Fallbacks to default_meta in case if meta is nothing.

Node types

We distinguish different types of factor nodes in order to have better control over Bethe Free Energy computation. Each factor node has either the Deterministic or Stochastic functional form type.

ReactiveMP.Deterministic — Type

Deterministic

Deterministic object used to parametrize factor node object with determinstic type of relationship between variables.

source

ReactiveMP.Stochastic — Type

Stochastic

Stochastic object used to parametrize factor node object with stochastic type of relationship between variables.

source

ReactiveMP.isdeterministic — Function

isdeterministic(node)

Function used to check if factor node object is deterministic or not. Returns true or false.

source

ReactiveMP.isstochastic — Function

isstochastic(node)

Function used to check if factor node object is stochastic or not. Returns true or false.

source

ReactiveMP.sdtype — Function

sdtype(object)

Returns either Deterministic or Stochastic for a given object (if defined).

source

For example the + node has the Deterministic type:

println("Is `+` node deterministic: ", isdeterministic(sdtype(+)))
println("Is `+` node stochastic: ", isstochastic(sdtype(+)))

Is `+` node deterministic: true
Is `+` node stochastic: false

On the other hand, the Bernoulli node has the Stochastic type:

println("Is `Bernoulli` node deterministic: ", isdeterministic(sdtype(Bernoulli)))
println("Is `Bernoulli` node stochastic: ", isstochastic(sdtype(Bernoulli)))

Is `Bernoulli` node deterministic: false
Is `Bernoulli` node stochastic: true

To get an actual instance of the type object we use sdtype function:

println("sdtype() of `+` node is ", sdtype(+))
println("sdtype() of `Bernoulli` node is ", sdtype(Bernoulli))

sdtype() of `+` node is Deterministic()
sdtype() of `Bernoulli` node is Stochastic()

Node functional dependencies

The generic implementation of factor nodes in ReactiveMP supports custom functional dependencies policies. Briefly, the functional dependencies define what dependencies are needed to compute a single message. As an example, consider the belief-propagation message update equation for a factor node $f$ with three edges: $x$, $y$ and $z$:

\[\mu(x) = \int \mu(y) \mu(z) f(x, y, z) \mathrm{d}y \mathrm{d}z\]

Here we see that in the standard setting for the belief-propagation message out of edge $x$, we need only messages from the edges $y$ and $z$. In contrast, consider the variational message update rule equation with mean-field assumption:

\[\mu(x) = \exp \int q(y) q(z) \log f(x, y, z) \mathrm{d}y \mathrm{d}z\]

We see that in this setting, we do not need messages $\mu(y)$ and $\mu(z)$, but only the marginals $q(y)$ and $q(z)$.

List of functional dependencies policies

The purpose of a functional dependencies policy is to determine functional dependencies (a set of messages or marginals) that are needed to compute a single message. By default, ReactiveMP.jl uses so-called DefaultFunctionalDependencies that correctly implements belief-propagation and variational message passing schemes (including both mean-field and structured factorisations). The full list of built-in policies is presented below:

ReactiveMP.DefaultFunctionalDependencies — Type

DefaultFunctionalDependencies

This functional dependencies translate directly to a regular variational message passing scheme. In order to compute a message out of some interface, this strategy requires messages from interfaces within the same cluster and marginals over other clusters.

source

ReactiveMP.RequireMessageFunctionalDependencies — Type

RequireMessageFunctionalDependencies(specifications::NamedTuple)
RequireMessageFunctionalDependencies(; specifications...)

The same as DefaultFunctionalDependencies, but in order to compute a message out of some edge also requires the inbound message on the this edge.

The specification parameter is a named tuple that contains the names of the edges and their initial messages. When a name is present in the named tuple, that indicates that the computation of the outbound message on the same edge must use the inbound message. If nothing is passed as a value in the named tuple, the initial message is not set. Note that the construction allows passing keyword argument to the constructor instead of using NamedTuple directly.

RequireMessageFunctionalDependencies(μ = vague(NormalMeanPrecision),     τ = nothing)
                                     # ^^^                               ^^^
                                     # request 'inbound' for 'x'         we may do the same for 'τ',
                                     # and initialise with `vague(...)`  but here we skip initialisation

source

ReactiveMP.RequireMarginalFunctionalDependencies — Type

RequireMarginalFunctionalDependencies(specifications::NamedTuple)
RequireMarginalFunctionalDependencies(; specifications...)

The same as DefaultFunctionalDependencies, but in order to compute a message out of some edge also requires the marginal on the this edge.

The specification parameter is a named tuple that contains the names of the edges and their initial marginals. When a name is present in the named tuple, that indicates that the computation of the outbound message on the same edge must use the marginal on this edge. If nothing is passed as a value in the named tuple, the initial marginal is not set. Note that the construction allows passing keyword argument to the constructor instead of using NamedTuple directly.

RequireMarginalFunctionalDependencies(μ = vague(NormalMeanPrecision),     τ = nothing)
                                     # ^^^                               ^^^
                                     # request 'marginal' for 'x'        we may do the same for 'τ',
                                     # and initialise with `vague(...)`  but here we skip initialisation

source

ReactiveMP.RequireEverythingFunctionalDependencies — Type

RequireEverythingFunctionalDependencies

This strategy specifies that in order to compute a message of some edge update rules request everything that is available locally. This includes all inbound messages (including on the same edge) and marginals over all local edge-clusters (this may or may not include marginals on single edges, depends on the local factorisation constraint).

source

Node traits

Each factor node has to define the ReactiveMP.is_predefined_node trait function and to specify a ReactiveMP.PredefinedNodeFunctionalForm singleton as a return object. By default ReactiveMP.is_predefined_node returns ReactiveMP.UndefinedNodeFunctionalForm. Objects that do not specify this property correctly cannot be used in model specification.

Note

@node macro does that automatically

ReactiveMP.PredefinedNodeFunctionalForm — Type

PredefinedNodeFunctionalForm

Trait specification for an object that has been marked as a valid factor node with the @node macro.

source

ReactiveMP.UndefinedNodeFunctionalForm — Type

UndefinedNodeFunctionalForm

Trait specification for an object that has not been marked as a factor node with the @node macro. Note that it does not necessarily mean that the object is not a valid factor node, but rather that it has not been marked as such. The ReactiveMP inference engine support arbitrary deterministic function as factor nodes, but they require manual specification of the approximation method.

source

ReactiveMP.is_predefined_node — Function

is_predefined_node(object)

Determines if the object has been marked as a factor node with the @node macro. Returns either PredefinedNodeFunctionalForm() or UndefinedNodeFunctionalForm().

source

Stream postprocessors

Stream postprocessors are composable transformations applied to the reactive observables produced during activation — outbound message streams, marginal streams, and score streams. They are attached to a node via ReactiveMP.FactorNodeActivationOptions and to a random variable via ReactiveMP.RandomVariableActivationOptions, and can be used for scheduling or custom instrumentation.

See the dedicated Stream postprocessors page for a full description and API reference.

List of predefined factor node

To quickly check the list of all predefined factor nodes, call ?ReactiveMP.is_predefined_node or Base.doc(ReactiveMP.is_predefined_node).

?ReactiveMP.is_predefined_node

is_predefined_node(object)

Determines if the object has been marked as a factor node with the @node macro. Returns either PredefinedNodeFunctionalForm() or UndefinedNodeFunctionalForm().

Type{<:Uninformative}          : Stochastic      : out

The Uninformative has been marked as a valid Stochastic factor node with the @node macro with [ out ] interfaces.

Type{<:Uniform}                : Stochastic      : out, a (or α, left), b (or β, right)

The Uniform has been marked as a valid Stochastic factor node with the @node macro with [ out, a (or α, left), b (or β, right) ] interfaces.

Type{<:NormalMeanVariance}     : Stochastic      : out, μ (or mean), v (or var)

The NormalMeanVariance has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), v (or var) ] interfaces.

Type{<:NormalMeanPrecision}    : Stochastic      : out, μ (or mean), τ (or invcov, precision)

The NormalMeanPrecision has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), τ (or invcov, precision) ] interfaces.

Type{<:MvNormalMeanCovariance} : Stochastic      : out, μ (or mean), Σ (or cov)

The MvNormalMeanCovariance has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), Σ (or cov) ] interfaces.

Type{<:MvNormalMeanPrecision}  : Stochastic      : out, μ (or mean), Λ (or invcov, precision)

The MvNormalMeanPrecision has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), Λ (or invcov, precision) ] interfaces.

Type{<:MvNormalMeanScalePrecision} : Stochastic      : out, μ (or mean), γ (or precision)

The MvNormalMeanScalePrecision has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), γ (or precision) ] interfaces.

Type{<:MvNormalWeightedMeanPrecision} : Stochastic      : out, ξ (or xi, weightedmean), Λ (or invcov, precision)

The MvNormalWeightedMeanPrecision has been marked as a valid Stochastic factor node with the @node macro with [ out, ξ (or xi, weightedmean), Λ (or invcov, precision) ] interfaces.

Type{<:MvNormalMeanScaleMatrixPrecision} : Stochastic      : out, μ (or mean), γ (or scale), G (or matrix)

The MvNormalMeanScaleMatrixPrecision has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), γ (or scale), G (or matrix) ] interfaces.

Type{<:MvNormalWishart}        : Stochastic      : out, μ (or mean), W (or scale), λ, ν

The MvNormalWishart has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), W (or scale), λ, ν ] interfaces.

Type{<:Gamma}                  : Stochastic      : out, α (or shape), θ (or scale)

The Gamma has been marked as a valid Stochastic factor node with the @node macro with [ out, α (or shape), θ (or scale) ] interfaces.

Type{<:GammaInverse}           : Stochastic      : out, α (or shape), θ (or scale)

The GammaInverse has been marked as a valid Stochastic factor node with the @node macro with [ out, α (or shape), θ (or scale) ] interfaces.

Type{<:GammaShapeRate}         : Stochastic      : out, α (or a, shape), β (or b, rate)

The GammaShapeRate has been marked as a valid Stochastic factor node with the @node macro with [ out, α (or a, shape), β (or b, rate) ] interfaces.

Type{<:Beta}                   : Stochastic      : out, a (or α), b (or β)

The Beta has been marked as a valid Stochastic factor node with the @node macro with [ out, a (or α), b (or β) ] interfaces.

Type{<:Categorical}            : Stochastic      : out, p

The Categorical has been marked as a valid Stochastic factor node with the @node macro with [ out, p ] interfaces.

Type{<:Dirichlet}              : Stochastic      : out, a

The Dirichlet has been marked as a valid Stochastic factor node with the @node macro with [ out, a ] interfaces.

Type{<:DirichletCollection}    : Stochastic      : out, a

The DirichletCollection has been marked as a valid Stochastic factor node with the @node macro with [ out, a ] interfaces.

Type{<:Bernoulli}              : Stochastic      : out, p (or θ)

The Bernoulli has been marked as a valid Stochastic factor node with the @node macro with [ out, p (or θ) ] interfaces.

Type{<:GCV}                    : Stochastic      : y, x, z, κ, ω

The GCV has been marked as a valid Stochastic factor node with the @node macro with [ y, x, z, κ, ω ] interfaces.

Type{<:Wishart}                : Stochastic      : out, ν (or df), S (or scale)

The Wishart has been marked as a valid Stochastic factor node with the @node macro with [ out, ν (or df), S (or scale) ] interfaces.

Type{<:InverseWishart}         : Stochastic      : out, ν (or df), S (or scale, Ψ)

The InverseWishart has been marked as a valid Stochastic factor node with the @node macro with [ out, ν (or df), S (or scale, Ψ) ] interfaces.

typeof(dot)                    : Deterministic   : out, in1, in2

The typeof(dot) has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

Type{<:softdot}                : Stochastic      : y, θ (or theta), x, γ (or gamma)

The softdot has been marked as a valid Stochastic factor node with the @node macro with [ y, θ (or theta), x, γ (or gamma) ] interfaces.

Type{<:AR}                     : Stochastic      : y (or out), x, θ, γ

The AR has been marked as a valid Stochastic factor node with the @node macro with [ y (or out), x, θ, γ ] interfaces.

Type{<:BIFM}                   : Deterministic   : out, in, zprev, znext

The BIFM has been marked as a valid Deterministic factor node with the @node macro with [ out, in, zprev, znext ] interfaces.

Type{<:BIFMHelper}             : Stochastic      : out, in

The BIFMHelper has been marked as a valid Stochastic factor node with the @node macro with [ out, in ] interfaces.

Type{<:Probit}                 : Stochastic      : out, in

The Probit has been marked as a valid Stochastic factor node with the @node macro with [ out, in ] interfaces.

Type{<:Poisson}                : Stochastic      : out, l (or λ)

The Poisson has been marked as a valid Stochastic factor node with the @node macro with [ out, l (or λ) ] interfaces.

Type{<:ContinuousTransition}   : Stochastic      : y, x, a, W

The ContinuousTransition has been marked as a valid Stochastic factor node with the @node macro with [ y, x, a, W ] interfaces.

Type{<:HalfNormal}             : Stochastic      : out, v (or var, σ²)

The HalfNormal has been marked as a valid Stochastic factor node with the @node macro with [ out, v (or var, σ²) ] interfaces.

Type{<:BinomialPolya}          : Stochastic      : y, x, n, β

The BinomialPolya has been marked as a valid Stochastic factor node with the @node macro with [ y, x, n, β ] interfaces.

Type{<:MultinomialPolya}       : Stochastic      : x, N, ψ

The MultinomialPolya has been marked as a valid Stochastic factor node with the @node macro with [ x, N, ψ ] interfaces.

Type{<:Flow}                   : Deterministic   : out, in

The Flow has been marked as a valid Deterministic factor node with the @node macro with [ out, in ] interfaces.

typeof(+)                      : Deterministic   : out, in1, in2

The typeof(+) has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

typeof(-)                      : Deterministic   : out, in1, in2

The typeof(-) has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

typeof(*)                      : Deterministic   : out, A, in

The typeof(*) has been marked as a valid Deterministic factor node with the @node macro with [ out, A, in ] interfaces.

Type{<:AND}                    : Deterministic   : out, in1, in2

The AND has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

Type{<:OR}                     : Deterministic   : out, in1, in2

The OR has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

Type{<:NOT}                    : Deterministic   : out, in

The NOT has been marked as a valid Deterministic factor node with the @node macro with [ out, in ] interfaces.

Type{<:IMPLY}                  : Deterministic   : out, in1, in2

The IMPLY has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.