Delta node

DeltaMeta(method = ..., [ inverse = ... ])

DeltaMeta structure specifies the approximation method for the outbound messages in the DeltaFn node.

Arguments

• method: required, the approximation method, currently supported methods are Linearization, Unscented and CVI.
• inverse: optional, if no inverse provided, the backward rule will be computed based on RTS (Petersen et al. 2018; On Approximate Delta Gaussian Message Passing on Factor Graphs)

Is is also possible to pass the AbstractApproximationMethod to the meta of the delta node directly. In this case inverse is set to nothing.

Some nodes use IndexedInterface, ManyOf structure reflects a collection of marginals from the collection of IndexedInterfaces. @rule macro also treats ManyOf specially.

The Linearization structure defines the approximation method of the Delta and Flow factor nodes. This method performs a local linearization of f around expansion point x.

The Linearization structure with default parameters can be constructed as Linearization().

The Linearization structure is used inside the DeltaMeta or FlowMeta structures and can be included as:

    y ~ f(x) where { meta = DeltaMeta(method = Linearization()) }
    # or
    y ~ Flow(x) where { meta = FlowMeta(flowmodel, Linearization()) }

local_linearization(g, x)

Returns linear components (a, b) for the function g at the point x.

RTS smoother update for inbound marginal; based on (Petersen et al. 2018; On Approximate Delta Gaussian Message Passing on Factor Graphs)

The Unscented structure defines the approximation method of the Delta and Flow factor nodes. More specifically, it contains the hyperparameters used for sigma points computation.

Arguments

• α: Spread parameter for unscented transform #1
• β: Algorithm parameter for incorporating prior information on the (non-Gaussian) distribution of Delta node input
• κ: Spread parameter for unscented transform #2
• e: Internal cache

The Unscented structure with default parameters can be constructed as Unscented().

The Unscented structure is used inside the DeltaMeta or FlowMeta structure and can be included as:

    y ~ f(x) where { meta = DeltaMeta(method = Unscented()) }
    # or
    y ~ Flow(x) where { meta = FlowMeta(flowmodel, Unscented()) }

Return the sigma points and weights for a Gaussian distribution

CVIApproximationDeltaFnRuleLayout

Custom rule layout for the Delta node in case of the CVI approximation method:

Layout

In order to compute:

• q_out: mirrors the posterior marginal on the out edge
• q_ins: uses inbound message on the out edge and all inbound messages on the ins edges
• m_out: uses the joint over the ins edges
• m_in_k: uses the inbound message on the in_k edge and q_ins

This function calculates the log of the Gauss-laguerre integral by making use of the log of the integrable function. ln ( ∫ exp(-x)f(x) dx ) ≈ ln ( ∑ wi * f(xi) ) = ln ( ∑ exp( ln(wi) + logf(xi) ) ) = ln ( ∑ exp( yi ) ) = max(yi) + ln ( ∑ exp( yi - max(yi) ) ) where we make use of the numerically stable log-sum-exp trick: https://en.wikipedia.org/wiki/LogSumExp

ForwardDiffGrad(chunk_size::Int)

The auto-differentiation backend for the CVI procedure. Uses the ForwardDiff library to compute gradients/derivatives. If chunk_size is not specified then uses the heuristic from ForwardDiff, which is type-unstable.

An alias for the Unscented approximation method.

An alias for the Unscented approximation method.

ProdCVI

The ProdCVI structure defines the approximation method hyperparameters of the prod(approximation::CVI, logp::F, dist). This method performs an approximation of the product of the dist and logp with Stochastic Variational message passing (SVMP-CVI) (See Probabilistic programming with stochastic variational message passing).

Arguments

• rng: random number generator
• n_samples: number of samples to use for statistics approximation
• n_iterations: number of iteration for the natural parameters gradient optimization
• opt: optimizer, which will be used to perform the natural parameters gradient optimization step
• grad: optional, defaults to ForwardDiffGrad(), structure to select how the gradient and the hessian will be computed
• n_gradpoints: optional, defaults to 1, number of points to estimate gradient of the likelihood (dist*logp)
• enforce_proper_messages: optional, defaults to true, ensures that a message, computed towards the inbound edges, is a proper distribution, must be of type Val(true)/Val(false)
• warn: optional, defaults to true, enables or disables warnings related to the optimization steps

Alias for the ProdCVI method. See help for ProdCVI

CVIProjection(; parameters...)

A structure representing the parameters for the Conjugate Variational Inference (CVI) projection method. This structure is a subtype of AbstractApproximationMethod and is used to configure the settings for CVI.

Parameters

• rng::R: The random number generator used for sampling. Default is Random.MersenneTwister(42).
• marginalsamples::S: The number of samples used for approximating marginal distributions. Default is 10.
• outsamples::S: The number of samples used for approximating output message distributions. Default is 100.
• prjparams::P: Parameters for the exponential family projection. Default is nothing, in which case it will use ExponentialFamilyProjection.DefaultProjectionParameters().

cvi_setup!(opt, λ)

Initialises the given optimiser for the CVI procedure given the structure of λ. Returns a tuple of the optimiser and the optimiser state.

cvi_update!(tuple_of_opt_and_state, new_λ, λ, ∇)

Uses the optimiser, its state and the gradient ∇ to change the trainable parameters in the λ. Modifies the optimiser state and and store the output in the newλ. Returns a tuple of the optimiser and the newλ.

DeltaFnDefaultRuleLayout

Default rule layout for the Delta node:

Layout

In order to compute:

• q_out: mirrors the posterior marginal on the out edge
• q_ins: uses inbound message on the out edge and all inbound messages on the ins edges
• m_out: uses all inbound messages on the ins edges
• m_in_k: uses the inbound message on the in_k edge and q_ins

See also: ReactiveMP.DeltaFnDefaultKnownInverseRuleLayout

DeltaFnDefaultKnownInverseRuleLayout

Default rule layout for the Delta node:

Layout

In order to compute:

• q_out: mirrors the posterior marginal on the out edge (same as the DeltaFnDefaultRuleLayout)
• q_ins: uses inbound message on the out edge and all inbound messages on the ins edges (same as the DeltaFnDefaultRuleLayout)
• m_out: uses all inbound messages on the ins edges (same as the DeltaFnDefaultRuleLayout)
• m_in_k: uses inbound message on the out edge and inbound messages on the ins edges except k