Testing your first rule

In this tutorial you test the two rules of a node with tables. You write a case for each rule, learn what a case checks, and read the report of a case that fails.

You need to know what a rule is and how one is defined. MessagePassingRulesBase's Your first node derives the rules this page tests.

The node

The node is a normal distribution with a known variance, $f(y, x, v) = \mathcal{N}(y \mid x, v)$, where $y$ is the output out, $x$ the mean μ and $v$ the variance v:

using MessagePassingRulesBase, MessagePassingRulesTestUtils
using BayesBase, ExponentialFamily, Distributions, Test

struct Gaussian end   # out ~ Normal(μ, v)
Gaussian(μ, v) = NormalMeanVariance(μ, v)

@define_factor_node(node = Gaussian, type = Stochastic, interfaces = [:out, :μ, :v])

The line Gaussian(μ, v) = NormalMeanVariance(μ, v) makes the node callable as the distribution of its output. The next tutorial uses it to check the rules against the node's density.

The belief propagation message towards out integrates the factor against the message on μ. With a normal message of mean $m$ and variance $w$ on μ, and a known variance $v$, it is a normal of mean $m$ and variance $w + v$:

@define_message_update_rule(
    node = Gaussian, target = :out,
    args = (m[:μ]::NormalMeanVariance, m[:v]::PointMass),
    logscale = 0,
    body = (args) -> NormalMeanVariance(mean(args.m[:μ]), var(args.m[:μ]) + mean(args.m[:v])),
)

The integral is a normalised density, so the rule's log scale is 0.

A table

A table lists cases. A case is the inputs of one call to the rule and the result the rule must return:

@testset "Gaussian: out" begin
    @test_message_update_rule(
        node = Gaussian, target = :out,
        cases = [
            (m = (μ = NormalMeanVariance(1.0, 2.0), v = PointMass(0.5)),) => NormalMeanVariance(1.0, 2.5),
        ],
    )
end

Test Summary: | Pass  Total  Time
Gaussian: out |   11     11  0.2s

The inputs are a NamedTuple. Its entry m holds the inbound messages, keyed by the interfaces' names. The table selects the rule from the inputs' types, as an engine would.

The summary counts eleven checks for the one case:

The rule is found. One rule takes these inputs.
The result is the expected one. The types must match exactly, a NormalMeanVariance{Float64} for a NormalMeanVariance{Float64}. The values must agree within the tolerances. approximately_equal makes the comparison.
The result has the promoted type. The case runs nine more times, with its inputs converted to Float32, Float64 and BigFloat: both inputs at once, then each alone. With both inputs in Float32, the result must be a NormalMeanVariance{Float32}.

The log scale

A plain expected value leaves the log scale unchecked. Wrap it in ExpectedWithLogScale to check the log scale the rule declares:

@testset "Gaussian: out, with its log scale" begin
    @test_message_update_rule(
        node = Gaussian, target = :out,
        cases = [
            (m = (μ = NormalMeanVariance(1.0, 2.0), v = PointMass(0.5)),) => ExpectedWithLogScale(NormalMeanVariance(1.0, 2.5), 0),
        ],
    )
end

Test Summary:                     | Pass  Total  Time
Gaussian: out, with its log scale |   12     12  0.1s

The case makes one more check, the log scale's.

A variational rule

Under variational message passing, the rule towards out reads the marginal of μ, not its message. The message is a normal of the marginal's mean and the variance $v$:

@define_message_update_rule(
    node = Gaussian, target = :out,
    args = (q[:μ]::Any, q[:v]::PointMass),
    body = (args) -> NormalMeanVariance(mean(args.q[:μ]), mean(args.q[:v])),
)

A case for it gives marginals, in the entry q. One table holds the cases of both rules:

@testset "Gaussian: out, both rules" begin
    @test_message_update_rule(
        node = Gaussian, target = :out,
        cases = [
            (m = (μ = NormalMeanVariance(1.0, 2.0), v = PointMass(0.5)),) => ExpectedWithLogScale(NormalMeanVariance(1.0, 2.5), 0),
            (q = (μ = NormalMeanVariance(1.0, 2.0), v = PointMass(0.5)),) => NormalMeanVariance(1.0, 0.5),
        ],
    )
end

Test Summary:             | Pass  Total  Time
Gaussian: out, both rules |   23     23  0.8s

The first case selects the belief propagation rule and the second the variational one, by the inputs each gives.

A failing case

A rule can be right for Float64 inputs and still fail its table. This version of the variational rule converts the variance to Float64:

@define_message_update_rule(
    node = Gaussian, target = :out,
    args = (q[:μ]::Any, q[:v]::PointMass),
    body = (args) -> NormalMeanVariance(mean(args.q[:μ]), Float64(mean(args.q[:v]))),
)

Run its case again. The page collects the failures and prints them, as Reading a failure shows. In your suite, each prints the same way, at the line of the table:

ts = @testset Collected "Gaussian: out, variational" begin
    @test_message_update_rule(
        node = Gaussian, target = :out,
        cases = [(q = (μ = NormalMeanVariance(1.0, 2.0), v = PointMass(0.5)),) => NormalMeanVariance(1.0, 0.5)],
    )
end
failures(ts)

Test Failed
  Expression: rule_output ≈ expected
   Evaluated: case 1, message rule for Gaussian towards :out under DefaultAlgorithm(), inputs (q = (μ = NormalMeanVariance{Float64}(μ=1.0, v=2.0), v = PointMass{Float64}(0.5))), with input(s) [1, 2] converted to Float32: got NormalMeanVariance{Float64}(μ=1.0, v=0.5)::NormalMeanVariance{Float64}, expected NormalMeanVariance{Float32}(μ=1.0f0, v=0.5f0)::NormalMeanVariance{Float32}

Test Failed
  Expression: rule_output ≈ expected
   Evaluated: case 1, message rule for Gaussian towards :out under DefaultAlgorithm(), inputs (q = (μ = NormalMeanVariance{Float64}(μ=1.0, v=2.0), v = PointMass{Float64}(0.5))), with input(s) [2] converted to BigFloat: got NormalMeanVariance{Float64}(μ=1.0, v=0.5)::NormalMeanVariance{Float64}, expected NormalMeanVariance{BigFloat}(μ=1.0, v=0.5)::NormalMeanVariance{BigFloat}

Read a report from the top:

Expression names the check: rule_output ≈ expected compares the result with the expected one.
Evaluated says which run failed: the case, the rule, the inputs, and which inputs were converted to which float type. Then it gives both values, each with its type.

The value is right in both reports, and the type is wrong. Both runs convert v: with every input in Float32, and with v alone in BigFloat. The result must follow v's type, and the rule's Float64(…) holds it at Float64. Remove the conversion, and the case passes:

@define_message_update_rule(
    node = Gaussian, target = :out,
    args = (q[:μ]::Any, q[:v]::PointMass),
    body = (args) -> NormalMeanVariance(mean(args.q[:μ]), mean(args.q[:v])),
)

@testset "Gaussian: out, variational" begin
    @test_message_update_rule(
        node = Gaussian, target = :out,
        cases = [(q = (μ = NormalMeanVariance(1.0, 2.0), v = PointMass(0.5)),) => NormalMeanVariance(1.0, 0.5)],
    )
end

Test Summary:              | Pass  Total  Time
Gaussian: out, variational |   11     11  0.2s

The expected values in these tables are worked out by hand, so they can repeat a mistake of the rule's derivation. Testing a rule package checks the rules against the node's density instead, adds derivative checks, and makes the suite fail when a rule has no test. Table tests lists every keyword of a table.