GitHub

Verification against the node

A table's expected values are derived by hand. Verification checks a message rule against what it is derived from, the node's density. The message a rule computes must be, up to a constant, the node's factor integrated against its other inputs. Verification computes that integral numerically at a set of points.

With messages as inputs, the reference is belief propagation:

\[\log \mu(x) = \log \int f(x, y_1, \dots, y_n) \prod_j m_j(y_j) \, dy + \text{const}.\]

The constant is the rule's log scale: the message times the exponential of its log scale is the integral itself. With marginals, the reference is naive variational message passing, $\log \mu(x) = \mathrm{E}_{q}[\log f(x, y)] + \text{const}$.

Verify a rule

The node is the Gaussian of the first tutorial, $f(y, x, v) = \mathcal{N}(y \mid x, v)$. Verification reads its log-density from nodefunction, which the declaration defines when the node is callable as the distribution of its output: here Gaussian(μ, v) is a NormalMeanVariance.

using MessagePassingRulesBase, MessagePassingRulesTestUtils
using BayesBase, ExponentialFamily, Distributions, Test

struct Gaussian end
Gaussian(μ, v) = NormalMeanVariance(μ, v)

@define_factor_node(node = Gaussian, type = Stochastic, interfaces = [:out, :μ, :v])

@define_message_update_rule(
    node = Gaussian, target = :out,
    args = (m[:μ]::NormalMeanVariance, m[:v]::PointMass),
    logscale = 0,
    body = (args) -> NormalMeanVariance(mean(args.m[:μ]), var(args.m[:μ]) + mean(args.m[:v])),
)

@define_message_update_rule(
    node = Gaussian, target = :out,
    args = (q[:μ]::Any, q[:v]::PointMass),
    body = (args) -> NormalMeanVariance(mean(args.q[:μ]), mean(args.q[:v])),
)

MessagePassingRulesBase.nodefunction(Gaussian)(out = 1.0, μ = 0.0, v = 2.0)
-1.5155121234846454

The last line evaluates the log-density at $y = 1$, $x = 0$ and $v = 2$.

Verify the belief propagation rule towards out, with a normal message on μ, which verification integrates:

@verify_message_update_rule(node = Gaussian, target = :out, m = (μ = NormalMeanVariance(0.5, 1.5), v = PointMass(2.0)))
9-element Vector{Float64}:
 3.76081388253624e-11
 7.105427357601002e-13
 5.995204332975845e-14
 7.771561172376096e-15
 1.9984014443252818e-15
 7.771561172376096e-15
 5.995204332975845e-14
 7.105427357601002e-13
 3.76081388253624e-11

The result is the log-ratio of the message to the reference at nine quantiles of the message. It varies by less than atol, 1e-6 by default, so the shape is right. It is zero, which is minus the declared log scale, so the scale is right too.

Verify the variational rule towards out, with marginals:

@verify_message_update_rule(node = Gaussian, target = :out, q = (μ = NormalMeanVariance(0.5, 1.5), v = PointMass(2.0)))
9-element Vector{Float64}:
 0.37499996632001853
 0.37499996805684277
 0.37499996866824437
 0.37499996894385657
 0.37499996902556143
 0.3749999689438568
 0.3749999686682448
 0.37499996805684366
 0.37499996632001764

The log-ratio is again constant, so the shape is right. It is not zero, and nothing checks it: a variational message has no log scale, and this rule declares none.

Each check is a Test assertion. Inside a @testset, the summary counts them:

@testset "Gaussian: verification" begin
    @verify_message_update_rule(node = Gaussian, target = :out, m = (μ = NormalMeanVariance(0.5, 1.5), v = PointMass(2.0)))
    @verify_message_update_rule(node = Gaussian, target = :out, q = (μ = NormalMeanVariance(0.5, 1.5), v = PointMass(2.0)))
end
Test Summary:          | Pass  Total  Time
Gaussian: verification |    3      3  0.0s

When a check fails, the report gives the log-ratio at every point, or the log scale the node's density implies. Reading a failure shows both.

Limitations

  • The node must be stochastic and without groups: the reference is its nodefunction.
  • The inputs are point masses, which are substituted, finite discrete distributions, which are enumerated, and at most two continuous univariate distributions, which are integrated.
  • The message is univariate. It is compared at its quantiles, or over its support when it is discrete.
  • A case takes messages or marginals, not both. A rule that mixes them is checked by its table only.

Verify any implementation

verify_message_update runs the same check on any function that returns a message and its log scale, for an implementation that is not a rule of the base package. It takes the node's log-density, its interface names and the target:

message(m, q) = (NormalMeanVariance(mean(m.μ), var(m.μ) + mean(m.v)), 0.0)
logdensity(; out, μ, v) = logpdf(NormalMeanVariance(μ, v), out)

verify_message_update(message, logdensity, (:out, :μ, :v), :out; m = (μ = NormalMeanVariance(0.5, 1.5), v = PointMass(2.0)))
9-element Vector{Float64}:
 3.76081388253624e-11
 7.105427357601002e-13
 5.995204332975845e-14
 7.771561172376096e-15
 1.9984014443252818e-15
 7.771561172376096e-15
 5.995204332975845e-14
 7.105427357601002e-13
 3.76081388253624e-11

API

MessagePassingRulesTestUtils.@verify_message_update_rule — Macro
@verify_message_update_rule(node = ..., target = ..., m = (...), q = (...), ...)

Check the message rule of a node towards one of its interfaces against the node's definition, integrated numerically. It takes the arguments and keywords of verify_message_update_rule, where the checks and their limits are described, all by name, and returns what it returns.

The macro takes keyword arguments only, written name = value, and passes them to verify_message_update_rule; node and target are required, and a positional argument or a missing required keyword is an error when the macro expands. Every check is reported against the line of the macro call, so a failure points at the call that made it.

Examples

@verify_message_update_rule(node = NormalMeanVariance, target = :out, m = (μ = NormalMeanVariance(0.5, 1.5), v = PointMass(2.0)))
source
MessagePassingRulesTestUtils.verify_message_update_rule — Function
verify_message_update_rule(node, target::Symbol; m = (;), q = (;), algorithm = default_algorithm(node),
    points = nothing, atol = 1e-6) -> Vector{Float64}

Check the message rule of node towards target against the node's definition: the message it returns, and the log scale it declares, against its nodefunction, integrated numerically. @verify_message_update_rule is the same check written with keywords, and reports failures at its own line. The rule is recorded as selected for check_rule_coverage.

The reference depends on the kind of inputs, which is one or the other:

  • with messages, m, belief propagation: $\log \int f(x, y) \prod_j m_j(y_j) \, dy$, the node's density f integrated against the normalised inbound messages;
  • with marginals, q, naive variational message passing: $\mathrm{E}_{q}[\log f(x, y)]$.

Each input is treated by its kind: a PointMass is substituted; a discrete univariate distribution with fewer than 10 000 values in its support is enumerated; a continuous univariate distribution is integrated by HCubature between its 1e-9 and 1 - 1e-9 quantiles, for at most two such inputs per call.

Two checks are made, each a Test assertion:

  • shape: logpdf(message, x) - reference(x) is the same at every test point, within atol, so the message is the reference up to a constant;
  • scale, for belief propagation when the message comes with a log scale: that constant is -logscale, within atol, so the message times exp(logscale) is the reference itself. It is checked only when the shape holds, since otherwise there is no single constant to compare.

The rule's log scale is checked when it declares one; an undefined log scale leaves the scale unchecked.

Limitations: the node must be stochastic and without groups, since only such a node has a nodefunction; target is a single interface, not a group member; the message must be a univariate distribution; the rule runs with an empty RuleContext, so a rule that reads services gets nothing for them; and a rule that takes messages and marginals together cannot be verified, since the inputs are one kind or the other. Such rules are checked by their tables.

Arguments

  • node: the node, as declared with @define_factor_node.
  • target: the interface the message goes to, a Symbol.

Keywords

  • m: the inbound messages, a NamedTuple keyed by interface name, with one entry for every interface but target. Default: none.
  • q: the marginals, keyed the same way. Default: none. Give m or q, not both.
  • points: the values of the target at which to compare. Default: nothing, which is the support of a discrete message, or 9 quantiles of the message, from 0.05 to 0.95.
  • atol: the absolute tolerance on the log-ratio's spread and on the log scale. Default: 1e-6.
  • source: the line checks are reported against. Default: LineNumberNode(0, :unknown); the macro form passes its own line.
  • algorithm: the algorithm value to run under. Default: default_algorithm(node).

Returns

The log-ratio logpdf(message, x) - reference(x) at each test point, in order; empty when no rule takes the inputs or the rule throws, each recorded as a failure, rule_found or rule_runs.

Throws

  • ArgumentError when both m and q are given, when an interface other than target has no input, when an input is neither a point mass nor a univariate distribution, when a discrete input's support has 10 000 values or more, or when more than two inputs need integrating.
  • MethodError when node has no nodefunction.

Examples

@verify_message_update_rule(node = NormalMeanVariance, target = :out, m = (μ = NormalMeanVariance(0.5, 1.5), v = PointMass(2.0)))
@verify_message_update_rule(node = NormalMeanVariance, target = :μ, q = (out = NormalMeanVariance(1.0, 2.0), v = InverseGamma(3.0, 4.0)))
source
MessagePassingRulesTestUtils.verify_message_update — Function
verify_message_update(message, logdensity, interfaces, target::Symbol; m = (;), q = (;),
    points = nothing, atol = 1e-6) -> Vector{Float64}

Check a message against the node definition it comes from, for any implementation of a rule: the message must be, up to a constant, the reference computed numerically from the node's log-density. verify_message_update_rule does this for a rule defined with the base package.

The reference depends on the kind of inputs, which is one or the other:

  • with messages, m, belief propagation: $\log \int f(x, y) \prod_j m_j(y_j) \, dy$, the node's density f integrated against the normalised inbound messages;
  • with marginals, q, naive variational message passing: $\mathrm{E}_{q}[\log f(x, y)]$.

Each input is treated by its kind: a PointMass is substituted; a discrete univariate distribution with fewer than 10 000 values in its support is enumerated; a continuous univariate distribution is integrated by HCubature between its 1e-9 and 1 - 1e-9 quantiles, for at most two such inputs per call.

Two checks are made, each a Test assertion:

  • shape: logpdf(message, x) - reference(x) is the same at every test point, within atol, so the message is the reference up to a constant;
  • scale, for belief propagation when the message comes with a log scale: that constant is -logscale, within atol, so the message times exp(logscale) is the reference itself. It is checked only when the shape holds, since otherwise there is no single constant to compare.

Arguments

  • message: a function (m, q) -> (message, logscale), called once with the keywords m and q, returning the outbound message, a univariate distribution, and its log scale, a number, or nothing to leave the scale unchecked. If it throws, the check fails with the exception, as rule_found for a missing rule and rule_runs otherwise.
  • logdensity: the node's log-density, called with one keyword per interface: logdensity(; out, μ, σ).
  • interfaces: the node's interface names, target among them.
  • target: the interface the message goes to.

Keywords

  • m: the inbound messages, a NamedTuple keyed by interface name, with one entry for every interface but target. Default: none.
  • q: the marginals, keyed the same way. Default: none. Give m or q, not both.
  • points: the values of the target at which to compare. Default: nothing, which is the support of a discrete message, or 9 quantiles of the message, from 0.05 to 0.95.
  • atol: the absolute tolerance on the log-ratio's spread and on the log scale. Default: 1e-6.
  • source: the line checks are reported against. Default: LineNumberNode(0, :unknown); the macro form passes its own line.

Returns

The log-ratio logpdf(message, x) - reference(x) at each test point, in order; empty when the message could not be computed, which is recorded as a failure.

Throws

  • ArgumentError when both m and q are given, when an interface other than target has no input, when an input is neither a point mass nor a univariate distribution, when a discrete input's support has 10 000 values or more, or when more than two inputs need integrating.
source