GitHub

Nodes implementation

In the message passing framework, one of the most important concepts is a factor node. A factor node represents a local function in a factorised representation of a generative model.

ReactiveMP.@node — Macro
@node(fformtype, sdtype, interfaces_list)

@node macro creates a node for a fformtype type object. To obtain a list of predefined nodes use ?is_predefined_node.

Arguments

  • fformtype: Either an existing type identifier, e.g. Normal or a function type identifier, e.g. typeof(+)
  • sdtype: Either Stochastic or Deterministic. Defines the type of the functional relationship
  • interfaces_list: Defines a fixed list of edges of a factor node, by convention the first element should be out. Example: [ out, mean, variance ]

Note: interfaces_list must not include names that contain _ symbol in them, as it is reserved to identify joint posteriors around the node object.

Examples


struct MyNormalDistribution
    mean :: Float64
    var  :: Float64
end

@node MyNormalDistribution Stochastic [ out, mean, var ]

@node typeof(+) Deterministic [ out, in1, in2 ]

List of available nodes

See ?is_predefined_node.

source
ReactiveMP.FactorNode — Type
GenericFactorNode(functionalform, interfaces)

Generic factor node object that represents a factor node with a given functionalform and interfaces.

source
ReactiveMP.FactorNodeLocalMarginal — Type
FactorNodeLocalMarginal

This object represents local marginals for some specific factor node. The local marginal can be joint in case of structured factorisation. Local to factor node marginal also can be shared with a corresponding marginal of some random variable.

source
ReactiveMP.IndexedNodeInterface — Type
IndexedNodeInterface

IndexedNodeInterface object represents a repetative node-variable connection. Used in cases when a node may connect to a different number of random variables with the same name, e.g. means and precisions of a Gaussian Mixture node.

source
ReactiveMP.tag — Function
tag(interface)

Returns a tag of the interface in the form of Val{ name(interface) }. The major difference between tag and name is that it is possible to dispath on interface's tag in message computation rule.

source
ReactiveMP.interfaces — Function
interfaces(fform)

Returns a Val object with a tuple of interface names for a given factor node type. Returns nothing for unknown functional form.

source
ReactiveMP.inputinterfaces — Function
inputinterfaces(fform)

Similar to interfaces, but returns a Val object with a tuple of input interface names for a given factor node type. Returns nothing for unknown functional form.

source
ReactiveMP.alias_interface — Function
alias_interface(factor_type, index, name)

Converts the given name to a correct interface name for a given factor node type and index.

source
ReactiveMP.collect_factorisation — Function
collect_factorisation(nodetype, factorisation)

This function converts given factorisation to a correct internal factorisation representation for a given node.

source
ReactiveMP.collect_pipeline — Function
collect_pipeline(nodetype, pipeline)

This function converts given pipeline to a correct internal pipeline representation for a factor given node.

source
ReactiveMP.collect_meta — Function
collect_meta(nodetype, meta)

This function converts given meta object to a correct internal meta representation for a given node. Fallbacks to default_meta in case if meta is nothing.

See also: default_meta, FactorNode

source

Adding a custom node

ReactiveMP.jl exports the @node macro that allows for quick definition of a factor node with a fixed number of edges. The example application can be the following:

struct MyNewCustomNode end

@node MyNewCustomNode   Stochastic         [ x, y, (z, aliases = [ d ] ) ]
#     ^^^^^^^^^^^^^^^   ^^^^^^^^^^^^^      ^^^^^^^^^^^
#     Node's tag/name   Node's type        A fixed set of edges
#                       Another possible   The very first edge (in this example `x`) is considered
#                       value is           to be the output of the node
#                       `Deterministic`    - Edges can have aliases, e.g. `z` can be both `z` or `d`

This expression registers a new node that can be used with the inference engine. Note, however, that the @node macro does not generate any message passing update rules. These must be defined using the @rule macro.

Node types

We distinguish different types of factor nodes in order to have better control over Bethe Free Energy computation. Each factor node has either the Deterministic or Stochastic functional form type.

For example the + node has the Deterministic type:

println("Is `+` node deterministic: ", isdeterministic(sdtype(+)))
println("Is `+` node stochastic: ", isstochastic(sdtype(+)))
Is `+` node deterministic: true
Is `+` node stochastic: false

On the other hand, the Bernoulli node has the Stochastic type:

println("Is `Bernoulli` node deterministic: ", isdeterministic(sdtype(Bernoulli)))
println("Is `Bernoulli` node stochastic: ", isstochastic(sdtype(Bernoulli)))
Is `Bernoulli` node deterministic: false
Is `Bernoulli` node stochastic: true

To get an actual instance of the type object we use sdtype function:

println("sdtype() of `+` node is ", sdtype(+))
println("sdtype() of `Bernoulli` node is ", sdtype(Bernoulli))
sdtype() of `+` node is Deterministic()
sdtype() of `Bernoulli` node is Stochastic()

Node functional dependencies pipeline

The generic implementation of factor nodes in ReactiveMP supports custom functional dependency pipelines. Briefly, the functional dependencies pipeline defines what dependencies are need to compute a single message. As an example, consider the belief-propagation message update equation for a factor node $f$ with three edges: $x$, $y$ and $z$:

\[\mu(x) = \int \mu(y) \mu(z) f(x, y, z) \mathrm{d}y \mathrm{d}z\]

Here we see that in the standard setting for the belief-propagation message out of edge $x$, we need only messages from the edges $y$ and $z$. In contrast, consider the variational message update rule equation with mean-field assumption:

\[\mu(x) = \exp \int q(y) q(z) \log f(x, y, z) \mathrm{d}y \mathrm{d}z\]

We see that in this setting, we do not need messages $\mu(y)$ and $\mu(z)$, but only the marginals $q(y)$ and $q(z)$. The purpose of a functional dependencies pipeline is to determine functional dependencies (a set of messages or marginals) that are needed to compute a single message. By default, ReactiveMP.jl uses so-called DefaultFunctionalDependencies that correctly implements belief-propagation and variational message passing schemes (including both mean-field and structured factorisations). The full list of built-in pipelines is presented below:

ReactiveMP.DefaultFunctionalDependencies — Type
DefaultFunctionalDependencies

This functional dependencies translate directly to a regular variational message passing scheme. In order to compute a message out of some interface, this strategy requires messages from interfaces within the same cluster and marginals over other clusters.

source
ReactiveMP.RequireMessageFunctionalDependencies — Type
RequireMessageFunctionalDependencies(specifications::NamedTuple)
RequireMessageFunctionalDependencies(; specifications...)

The same as DefaultFunctionalDependencies, but in order to compute a message out of some edge also requires the inbound message on the this edge.

The specification parameter is a named tuple that contains the names of the edges and their initial messages. When a name is present in the named tuple, that indicates that the computation of the outbound message on the same edge must use the inbound message. If nothing is passed as a value in the named tuple, the initial message is not set. Note that the construction allows passing keyword argument to the constructor instead of using NamedTuple directly.

RequireMessageFunctionalDependencies(μ = vague(NormalMeanPrecision),     τ = nothing)
                                     # ^^^                               ^^^
                                     # request 'inbound' for 'x'         we may do the same for 'τ',
                                     # and initialise with `vague(...)`  but here we skip initialisation

See also: ReactiveMP.DefaultFunctionalDependencies, ReactiveMP.RequireMarginalFunctionalDependencies, ReactiveMP.RequireEverythingFunctionalDependencies

source
ReactiveMP.RequireMarginalFunctionalDependencies — Type
RequireMarginalFunctionalDependencies(specifications::NamedTuple)
RequireMarginalFunctionalDependencies(; specifications...)

The same as DefaultFunctionalDependencies, but in order to compute a message out of some edge also requires the marginal on the this edge.

The specification parameter is a named tuple that contains the names of the edges and their initial marginals. When a name is present in the named tuple, that indicates that the computation of the outbound message on the same edge must use the marginal on this edge. If nothing is passed as a value in the named tuple, the initial marginal is not set. Note that the construction allows passing keyword argument to the constructor instead of using NamedTuple directly.

RequireMarginalFunctionalDependencies(μ = vague(NormalMeanPrecision),     τ = nothing)
                                     # ^^^                               ^^^
                                     # request 'marginal' for 'x'        we may do the same for 'τ',
                                     # and initialise with `vague(...)`  but here we skip initialisation

See also: ReactiveMP.DefaultFunctionalDependencies, ReactiveMP.RequireMessageFunctionalDependencies, ReactiveMP.RequireEverythingFunctionalDependencies

source
ReactiveMP.RequireEverythingFunctionalDependencies — Type

RequireEverythingFunctionalDependencies

This pipeline specifies that in order to compute a message of some edge update rules request everything that is available locally. This includes all inbound messages (including on the same edge) and marginals over all local edge-clusters (this may or may not include marginals on single edges, depends on the local factorisation constraint).

See also: DefaultFunctionalDependencies, RequireMessageFunctionalDependencies, RequireMarginalFunctionalDependencies

source

Node traits

Each factor node has to define the ReactiveMP.is_predefined_node trait function and to specify a ReactiveMP.PredefinedNodeFunctionalForm singleton as a return object. By default ReactiveMP.is_predefined_node returns ReactiveMP.UndefinedNodeFunctionalForm. Objects that do not specify this property correctly cannot be used in model specification.

Note

@node macro does that automatically

ReactiveMP.UndefinedNodeFunctionalForm — Type
UndefinedNodeFunctionalForm

Trait specification for an object that has not been marked as a factor node with the @node macro. Note that it does not necessarily mean that the object is not a valid factor node, but rather that it has not been marked as such. The ReactiveMP inference engine support arbitrary deterministic function as factor nodes, but they require manual specification of the approximation method.

See also: ReactiveMP.is_predefined_node, ReactiveMP.PredefinedNodeFunctionalForm

source

Node pipelines

ReactiveMP.apply_pipeline_stage — Function
apply_pipeline_stage(stage, factornode, tag, stream)

Applies a given pipeline stage to the stream argument given factornode and tag of an edge.

source
ReactiveMP.LoggerPipelineStage — Type
LoggerPipelineStage <: AbstractPipelineStage

Logs all updates from stream into output

Arguments

  • output: (optional), an output stream used to print log statements
source
ReactiveMP.ScheduleOnPipelineStage — Type
ScheduleOnPipelineStage{S} <: AbstractPipelineStage

Applies the schedule_on() operator from Rocket.jl library to the given pipeline with a provided scheduler

Arguments

  • scheduler: scheduler to schedule updates on. Must be compatible with Rocket.jl library and schedule_on() operator.
source
ReactiveMP.schedule_updates — Function
schedule_updates(variables...; pipeline_stage = ScheduleOnPipelineStage(PendingScheduler()))

Schedules posterior marginal updates for given variables using stage. By default creates ScheduleOnPipelineStage with PendingScheduler() from Rocket.jl library. Returns a scheduler with release! method available to release all scheduled updates.

source

List of predefined factor node

To quickly check the list of all predefined factor nodes, call ?ReactiveMP.is_predefined_node or Base.doc(ReactiveMP.is_predefined_node).

?ReactiveMP.is_predefined_node
is_predefined_node(object)

Determines if the object has been marked as a factor node with the @node macro. Returns either PredefinedNodeFunctionalForm() or UndefinedNodeFunctionalForm().

See also: ReactiveMP.PredefinedNodeFunctionalForm, ReactiveMP.UndefinedNodeFunctionalForm

Type{<:Uninformative}          : Stochastic      : out

The Uninformative has been marked as a valid Stochastic factor node with the @node macro with [ out ] interfaces.

Type{<:Uniform}                : Stochastic      : out, a (or α, left), b (or β, right)

The Uniform has been marked as a valid Stochastic factor node with the @node macro with [ out, a (or α, left), b (or β, right) ] interfaces.

Type{<:NormalMeanVariance}     : Stochastic      : out, μ (or mean), v (or var)

The NormalMeanVariance has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), v (or var) ] interfaces.

Type{<:NormalMeanPrecision}    : Stochastic      : out, μ (or mean), τ (or invcov, precision)

The NormalMeanPrecision has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), τ (or invcov, precision) ] interfaces.

Type{<:MvNormalMeanCovariance} : Stochastic      : out, μ (or mean), Σ (or cov)

The MvNormalMeanCovariance has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), Σ (or cov) ] interfaces.

Type{<:MvNormalMeanPrecision}  : Stochastic      : out, μ (or mean), Λ (or invcov, precision)

The MvNormalMeanPrecision has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), Λ (or invcov, precision) ] interfaces.

Type{<:MvNormalMeanScalePrecision} : Stochastic      : out, μ (or mean), γ (or precision)

The MvNormalMeanScalePrecision has been marked as a valid Stochastic factor node with the @node macro with [ out, μ (or mean), γ (or precision) ] interfaces.

Type{<:MvNormalWeightedMeanPrecision} : Stochastic      : out, ξ (or xi, weightedmean), Λ (or invcov, precision)

The MvNormalWeightedMeanPrecision has been marked as a valid Stochastic factor node with the @node macro with [ out, ξ (or xi, weightedmean), Λ (or invcov, precision) ] interfaces.

Type{<:Gamma}                  : Stochastic      : out, α (or shape), θ (or scale)

The Gamma has been marked as a valid Stochastic factor node with the @node macro with [ out, α (or shape), θ (or scale) ] interfaces.

Type{<:GammaInverse}           : Stochastic      : out, α (or shape), θ (or scale)

The GammaInverse has been marked as a valid Stochastic factor node with the @node macro with [ out, α (or shape), θ (or scale) ] interfaces.

Type{<:GammaShapeRate}         : Stochastic      : out, α (or a, shape), β (or b, rate)

The GammaShapeRate has been marked as a valid Stochastic factor node with the @node macro with [ out, α (or a, shape), β (or b, rate) ] interfaces.

Type{<:Beta}                   : Stochastic      : out, a (or α), b (or β)

The Beta has been marked as a valid Stochastic factor node with the @node macro with [ out, a (or α), b (or β) ] interfaces.

Type{<:Categorical}            : Stochastic      : out, p

The Categorical has been marked as a valid Stochastic factor node with the @node macro with [ out, p ] interfaces.

Type{<:MatrixDirichlet}        : Stochastic      : out, a

The MatrixDirichlet has been marked as a valid Stochastic factor node with the @node macro with [ out, a ] interfaces.

Type{<:Dirichlet}              : Stochastic      : out, a

The Dirichlet has been marked as a valid Stochastic factor node with the @node macro with [ out, a ] interfaces.

Type{<:Bernoulli}              : Stochastic      : out, p (or θ)

The Bernoulli has been marked as a valid Stochastic factor node with the @node macro with [ out, p (or θ) ] interfaces.

Type{<:GCV}                    : Stochastic      : y, x, z, κ, ω

The GCV has been marked as a valid Stochastic factor node with the @node macro with [ y, x, z, κ, ω ] interfaces.

Type{<:Wishart}                : Stochastic      : out, ν (or df), S (or scale)

The Wishart has been marked as a valid Stochastic factor node with the @node macro with [ out, ν (or df), S (or scale) ] interfaces.

Type{<:InverseWishart}         : Stochastic      : out, ν (or df), S (or scale, Ψ)

The InverseWishart has been marked as a valid Stochastic factor node with the @node macro with [ out, ν (or df), S (or scale, Ψ) ] interfaces.

typeof(dot)                    : Deterministic   : out, in1, in2

The typeof(dot) has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

Type{<:softdot}                : Stochastic      : y, θ (or theta), x, γ (or gamma)

The softdot has been marked as a valid Stochastic factor node with the @node macro with [ y, θ (or theta), x, γ (or gamma) ] interfaces.

Type{<:Transition}             : Stochastic      : out, in, a

The Transition has been marked as a valid Stochastic factor node with the @node macro with [ out, in, a ] interfaces.

Type{<:AR}                     : Stochastic      : y (or out), x, θ, γ

The AR has been marked as a valid Stochastic factor node with the @node macro with [ y (or out), x, θ, γ ] interfaces.

Type{<:BIFM}                   : Deterministic   : out, in, zprev, znext

The BIFM has been marked as a valid Deterministic factor node with the @node macro with [ out, in, zprev, znext ] interfaces.

Type{<:BIFMHelper}             : Stochastic      : out, in

The BIFMHelper has been marked as a valid Stochastic factor node with the @node macro with [ out, in ] interfaces.

Type{<:Probit}                 : Stochastic      : out, in

The Probit has been marked as a valid Stochastic factor node with the @node macro with [ out, in ] interfaces.

Type{<:Poisson}                : Stochastic      : out, l (or λ)

The Poisson has been marked as a valid Stochastic factor node with the @node macro with [ out, l (or λ) ] interfaces.

Type{<:ContinuousTransition}   : Stochastic      : y, x, a, W

The ContinuousTransition has been marked as a valid Stochastic factor node with the @node macro with [ y, x, a, W ] interfaces.

Type{<:HalfNormal}             : Stochastic      : out, v (or var, σ²)

The HalfNormal has been marked as a valid Stochastic factor node with the @node macro with [ out, v (or var, σ²) ] interfaces.

Type{<:Flow}                   : Deterministic   : out, in

The Flow has been marked as a valid Deterministic factor node with the @node macro with [ out, in ] interfaces.

typeof(+)                      : Deterministic   : out, in1, in2

The typeof(+) has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

typeof(-)                      : Deterministic   : out, in1, in2

The typeof(-) has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

typeof(*)                      : Deterministic   : out, A, in

The typeof(*) has been marked as a valid Deterministic factor node with the @node macro with [ out, A, in ] interfaces.

Type{<:AND}                    : Deterministic   : out, in1, in2

The AND has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

Type{<:OR}                     : Deterministic   : out, in1, in2

The OR has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.

Type{<:NOT}                    : Deterministic   : out, in

The NOT has been marked as a valid Deterministic factor node with the @node macro with [ out, in ] interfaces.

Type{<:IMPLY}                  : Deterministic   : out, in1, in2

The IMPLY has been marked as a valid Deterministic factor node with the @node macro with [ out, in1, in2 ] interfaces.