Delta node

DeltaMeta(method = ..., [ inverse = ... ])

DeltaMeta structure specifies the approximation method for the outbound messages in the DeltaFn node.

Arguments

• method: required, the approximation method, currently supported methods are Linearization, Unscented and CVI.
• inverse: optional, if no inverse provided, the backward rule will be computed based on RTS (Petersen et al. 2018; On Approximate Delta Gaussian Message Passing on Factor Graphs)

Is is also possible to pass the AbstractApproximationMethod to the meta of the delta node directly. In this case inverse is set to nothing.

Some nodes use IndexedInterface, ManyOf structure reflects a collection of marginals from the collection of IndexedInterfaces. @rule macro also treats ManyOf specially.

The Linearization structure defines the approximation method of the Delta and Flow factor nodes. This method performs a local linearization of f around expansion point x.

The Linearization structure with default parameters can be constructed as Linearization().

The Linearization structure is used inside the DeltaMeta or FlowMeta structures and can be included as:

    y ~ f(x) where { meta = DeltaMeta(method = Linearization()) }
    # or
    y ~ Flow(x) where { meta = FlowMeta(flowmodel, Linearization()) }

local_linearization(g, x)

Returns linear components (a, b) for the function g at the point x.

RTS smoother update for inbound marginal; based on (Petersen et al. 2018; On Approximate Delta Gaussian Message Passing on Factor Graphs)

The Unscented structure defines the approximation method of the Delta and Flow factor nodes. More specifically, it contains the hyperparameters used for sigma points computation.

Arguments

• α: Spread parameter for unscented transform #1
• β: Algorithm parameter for incorporating prior information on the (non-Gaussian) distribution of Delta node input
• κ: Spread parameter for unscented transform #2
• e: Internal cache

The Unscented structure with default parameters can be constructed as Unscented().

The Unscented structure is used inside the DeltaMeta or FlowMeta structure and can be included as:

    y ~ f(x) where { meta = DeltaMeta(method = Unscented()) }
    # or
    y ~ Flow(x) where { meta = FlowMeta(flowmodel, Unscented()) }

Return the sigma points and weights for a Gaussian distribution

CVIApproximationDeltaFnRuleLayout

Custom rule layout for the Delta node in case of the CVI approximation method:

Layout

In order to compute:

• q_out: mirrors the posterior marginal on the out edge
• q_ins: uses inbound message on the out edge and all inbound messages on the ins edges
• m_out: uses the joint over the ins edges
• m_in_k: uses the inbound message on the in_k edge and q_ins

This function calculates the log of the Gauss-laguerre integral by making use of the log of the integrable function. ln ( ∫ exp(-x)f(x) dx ) ≈ ln ( ∑ wi * f(xi) ) = ln ( ∑ exp( ln(wi) + logf(xi) ) ) = ln ( ∑ exp( yi ) ) = max(yi) + ln ( ∑ exp( yi - max(yi) ) ) where we make use of the numerically stable log-sum-exp trick: https://en.wikipedia.org/wiki/LogSumExp

ForwardDiffGrad(chunk_size::Int)

The auto-differentiation backend for the CVI procedure. Uses the ForwardDiff library to compute gradients/derivatives. If chunk_size is not specified then uses the heuristic from ForwardDiff, which is type-unstable.

An alias for the Unscented approximation method.

An alias for the Unscented approximation method.

ProdCVI

The ProdCVI structure defines the approximation method hyperparameters of the prod(approximation::CVI, logp::F, dist). This method performs an approximation of the product of the dist and logp with Stochastic Variational message passing (SVMP-CVI) (See Probabilistic programming with stochastic variational message passing).

Arguments

• rng: random number generator
• n_samples: number of samples to use for statistics approximation
• n_iterations: number of iteration for the natural parameters gradient optimization
• opt: optimizer, which will be used to perform the natural parameters gradient optimization step
• grad: optional, defaults to ForwardDiffGrad(), structure to select how the gradient and the hessian will be computed
• n_gradpoints: optional, defaults to 1, number of points to estimate gradient of the likelihood (dist*logp)
• enforce_proper_messages: optional, defaults to true, ensures that a message, computed towards the inbound edges, is a proper distribution, must be of type Val(true)/Val(false)
• warn: optional, defaults to true, enables or disables warnings related to the optimization steps

Alias for the ProdCVI method. See help for ProdCVI

CVIProjection(; parameters...)

A structure representing the parameters for the Conjugate Variational Inference (CVI) projection method. This structure is a subtype of AbstractApproximationMethod and is used to configure the settings for CVI.

CVI approximates the posterior distribution by projecting it onto a family of distributions with a conjugate form.

Requirements

The CVIProjection method requires the ExponentialFamilyProjection package to be installed and loaded in the current environment with using ExponentialFamilyProjection.

Parameters

• rng::R: The random number generator used for sampling. Default is Random.MersenneTwister(42).
• outsamples::S: The number of samples used for approximating output message distributions. Default is 100.
• out_prjparams::OF: The form parameter used to specify the target distribution family for the output message. If nothing (default), the form will be inferred from the marginal form.
• in_prjparams::IFS: A NamedTuple-like object that specifies the target distribution family for each input edge. Keys should be of the form :in_k where k is the input edge index. If nothing (default), the forms will be inferred from the incoming messages.
• proposal_distribution::PD: The proposal distribution used for generating samples. If not provided or set to nothing, it will be inferred from incoming messages and automatically updated during iterations.
• sampling_strategy::SS: The strategy for approximating the logpdf:
  • FullSampling(n): Uses n samples drawn from distributions (default: n=10). Provides more accurate approximation at the cost of increased computation time.
  • MeanBased(): Uses only the mean of each distribution as a single sample. Significantly faster but less accurate for non-linear nodes or complex distributions.

Examples

# Standard CVI projection with default settings
method = CVIProjection()

# Fast approximation using mean-based sampling
method = CVIProjection(sampling_strategy = MeanBased())

# Custom proposal with increased sample count
using Distributions
proposal = FactorizedJoint((NormalMeanVariance(0.0, 1.0), NormalMeanVariance(0.0, 1.0)))
method = CVIProjection(
    proposal_distribution = ProposalDistributionContainer(proposal),
    sampling_strategy = FullSampling(1000)
)

# Specify projection family for the output message
method = CVIProjection(out_prjparams = ProjectedTo(NormalMeanPrecision))

# Specify projection family for input edges
method = CVIProjection(in_prjparams = (in_1 = ProjectedTo(NormalMeanVariance), in_2 = ProjectedTo(GammaShapeRate)))

CVISamplingStrategy

An abstract type representing the sampling strategy for the CVI projection method. Concrete subtypes implement different approaches for generating samples used in approximating distributions.

FullSampling <: CVISamplingStrategy
FullSampling(samples::Int = 10)

A sampling strategy that uses multiple samples drawn from distributions.

Arguments

• samples::Int: The number of samples to draw from each distribution. Default is 10.

Example

# Use 100 samples for more accurate approximation
strategy = FullSampling(100)

MeanBased <: CVISamplingStrategy

A sampling strategy that uses only the mean of the proposal distribution as a single sample.

ProposalDistributionContainer{PD}

A mutable wrapper for proposal distributions used in the CVI projection method.

The container allows the proposal distribution to be updated during inference without recreating the entire approximation method structure.

Fields

• distribution::PD: The wrapped proposal distribution, can be of any compatible type.

cvi_setup!(opt, λ)

Initialises the given optimiser for the CVI procedure given the structure of λ. Returns a tuple of the optimiser and the optimiser state.

cvi_update!(tuple_of_opt_and_state, new_λ, λ, ∇)

Uses the optimiser, its state and the gradient ∇ to change the trainable parameters in the λ. Modifies the optimiser state and and store the output in the newλ. Returns a tuple of the optimiser and the newλ.

DeltaFnDefaultRuleLayout

Default rule layout for the Delta node:

Layout

In order to compute:

• q_out: mirrors the posterior marginal on the out edge
• q_ins: uses inbound message on the out edge and all inbound messages on the ins edges
• m_out: uses all inbound messages on the ins edges
• m_in_k: uses the inbound message on the in_k edge and q_ins

See also: ReactiveMP.DeltaFnDefaultKnownInverseRuleLayout

DeltaFnDefaultKnownInverseRuleLayout

Default rule layout for the Delta node:

Layout

In order to compute:

• q_out: mirrors the posterior marginal on the out edge (same as the DeltaFnDefaultRuleLayout)
• q_ins: uses inbound message on the out edge and all inbound messages on the ins edges (same as the DeltaFnDefaultRuleLayout)
• m_out: uses all inbound messages on the ins edges (same as the DeltaFnDefaultRuleLayout)
• m_in_k: uses inbound message on the out edge and inbound messages on the ins edges except k

SoftDot

The SoftDot node can be used as a substitute for the dot product operator delta node (the outgoing variable is the dot product of two others). It softens the delta constraint by adding a Gaussian noise as follows:

y ~ N(dot(θ, x), γ^(-1))

Interfaces:

1. y - result of the "soft" dot product,
2. θ - first variable to be multiplied,
3. x - second variable to be multiplied,
4. γ - precision of the Gaussian noise.

The advantage of using SoftDot is that it offers tractable and optimized closed-form variational messages for both Belief Propagation and Variational Message Passing.

See also: softdot

Alias for the SoftDot node.