Approximation methods

Approximation methods are used when exact message computation is intractable — most commonly inside Delta nodes, where the factor is a nonlinear or non-conjugate function y = f(x). Each method trades off accuracy, computational cost, and assumptions about f.

All approximation methods are passed through DeltaMeta or FlowMeta.

Choosing a method

Method	Best for	Dimensionality	Requires
`Linearization`	Smooth, nearly linear `f`	Any	ForwardDiff (auto)
`Unscented`	Smooth nonlinear `f`	Low–moderate	Nothing
`GaussHermiteCubature`	Univariate integrals with Gaussian inputs	Univariate	Point count `p`
`GaussLaguerreQuadrature`	Integrals over `[0, ∞)`	Univariate	Point count `n`
`srcubature` / `SphericalRadialCubature`	Multivariate Gaussian integrals	Multivariate	Nothing
`LaplaceApproximation`	Unimodal posteriors, differentiable `f`	Any	ForwardDiff + Optim
`CVI`	Black-box or non-differentiable `f`	Any	Optimizer + gradient
`CVIProjection`	CVI + exponential family projection	Any	ExponentialFamilyProjection.jl
`ImportanceSamplingApproximation`	General expectations via sampling	Any	Proposal distribution

Deterministic approximations

Linearization

Linearization approximates f by its first-order Taylor expansion around the current operating point. Jacobians are computed automatically using ForwardDiff.jl. This is the default method for FlowMeta and a common choice for DeltaMeta when f is smooth and not highly nonlinear.

Unscented transform

Unscented (also UT / UnscentedTransform) propagates a deterministic set of sigma points through f and fits a Gaussian to the outputs. It captures mean and covariance through nonlinearities more accurately than linearization, without requiring derivatives. The number of sigma points scales linearly with the input dimension.

Gauss-Hermite cubature

GaussHermiteCubature computes expectations of the form ∫ g(x) N(x; μ, σ²) dx using a fixed set of quadrature points and weights optimized for Gaussian measures. It is exact for polynomials up to a certain degree determined by the number of points p:

DeltaMeta(method = GaussHermiteCubature(21))  # 21-point rule

Gauss-Laguerre quadrature

GaussLaguerreQuadrature computes expectations over the half-line [0, ∞) — useful when the input has a Gamma distribution or similar semi-infinite support.

Spherical radial cubature

srcubature() constructs a spherical-radial cubature rule for multivariate Gaussian integrals, using 2d + 1 deterministic points (where d is the input dimension). It provides a good balance between accuracy and cost for moderate dimensions.

Laplace approximation

LaplaceApproximation finds the mode of the log-unnormalized posterior (using Optim.jl) and fits a Gaussian at that mode using the local curvature (via ForwardDiff). Best for unimodal, differentiable posteriors.

Stochastic approximations

CVI — Constrained Variational Inference

CVI and ProdCVI approximate messages using stochastic gradient optimization of a variational objective. A gradient estimator (default: ForwardDiffGrad) computes the gradient of the log-likelihood with respect to the natural parameters of the approximating distribution. An optimizer (default: Adam) applies gradient steps until convergence.

DeltaMeta(method = CVI(
    rng,          # random number generator
    n_samples,    # samples for gradient estimation
    n_iterations, # gradient steps per message update
    Adam(params), # optimizer
))

Note

Loading Optimisers.jl unlocks additional optimizers for use with CVI. See the Extensions page.

CVIProjection

CVIProjection extends CVI by projecting the result onto the nearest member of a chosen exponential family, using ExponentialFamilyProjection.jl. This guarantees that the output is a valid member of the target family.

Note

CVIProjection requires the ExponentialFamilyProjection package to be loaded. Without it, using CVIProjection in a delta node will throw an informative error.

Importance sampling

ImportanceSamplingApproximation estimates expectations by drawing samples from a proposal distribution and reweighting. It is the most flexible method but converges slowly in high dimensions.

API reference

ReactiveMP.Linearization — Type

The Linearization structure defines the approximation method of the Delta and Flow factor nodes. This method performs a local linearization of f around expansion point x.

Note

The Linearization works well for differentiable functions only. The results might not be accurate for non-differentiable functions.

The Linearization structure with default parameters can be constructed as Linearization().

The Linearization structure is used inside the DeltaMeta or FlowMeta structures and can be included as:

    y ~ f(x) where { meta = DeltaMeta(method = Linearization()) }
    # or
    y ~ Flow(x) where { meta = FlowMeta(flowmodel, Linearization()) }