GitHub

Constraint Specification

GraphPPL represents your probabilistic model and as a Bethe Free Energy (BFE), which means that users can define constraints on the variational posterior that influence the inference procedure. The BFE is chosen as the objective function because it is a generalization of many well-known inference algorithms. In this section we will explain how to specify constraints on the variational posterior. There are two major types of constraints we can apply: We can apply factorization constraints to factor nodes, which specify how the variational posterior factorizes around a factor node. We can also apply functional form constraints to variable nodes, which specify the functional form of the variational posterior that a variable takes. We can specify all constraints using the @constraints macro.

The constraints macro

The constraints macro accepts a high-level constraint specification and converts this to a structure that can be interpreted by GraphPPL models. For example, suppose we have the following toy model, that defines a Gaussian distribution over x with mean y and variance z:

using GraphPPL
using Distributions
import GraphPPL: @model

@model function toy_model(x, y, z)
    x ~ Normal(y, z)
end

Suppose we want to apply the following constraints over the variational posterior q:

\[q(x, y, z) = q(x, y)q(z) \\ q(x) \sim Normal\]

We can write this in the constraints macro using the following code:

@constraints begin
    q(x, y, z) = q(x, y)q(z)
    q(x) :: Normal
end
Constraints: 
  q(x, y, z) = q(x, y)q(z)
  q(x) :: Distributions.Normal

We can reference variables in the constraints macro with their corresponding name in the model specification. Naturally, this raises the question on how we can specify constraints over variables in submodels, as these variables are not available in the scope of the model specification. To this extent, we can nest our constraints in the same way in which we have nested our models, and use the for q in submodel block to specify constraints over submodels. For example, suppose we have the following model:

@model function toy_model(x, y, z)
    x ~ Normal(y, z)
    y ~ Normal(0, 1)
end

@model function outer_toy_model(a, b, c)
    a ~ toy_model(y = b, z = c)
end

We can specify constraints over the toy_model submodel using the following code:

@constraints begin
    for q in toy_model
        q(x, y, z) = q(x, y)q(z)
        q(x) :: Normal
    end
end
Constraints: 
  q(toy_model) = 
    q(x, y, z) = q(x, y)q(z)
    q(x) :: Distributions.Normal

The submodel constraint specification applies to all submodels with the same name. However, as a user you might want to specify constraints over a specific submodel. To this extent, we can use the for q in (submodel, index) syntax. This will only apply the constraints to the submodel with the corresponding index. For example, suppose we have the following model:

@model function toy_model(x, y, z)
    x ~ Normal(y, z)
    y ~ Normal(0, 1)
end

@model function outer_toy_model(a, b, c)
    a ~ toy_model(y = b, z = c)
    a ~ toy_model(y = b, z = c)
end

We can specify constraints over the first toy_model submodel using the following code:

@constraints begin
    for q in (toy_model, 1)
        q(x, y, z) = q(x, y)q(z)
        q(x) :: Normal
    end
end
Constraints: 
  q((toy_model, 1)) = 
    q(x, y, z) = q(x, y)q(z)
    q(x) :: Distributions.Normal

Plugin's internals

GraphPPL.Constraints — Type
Constraints

An instance of Constraints represents a set of constraints to be applied to a variational posterior in a factor graph model.

source
GraphPPL.GeneralSubModelConstraints — Type
GeneralSubModelConstraints

A GeneralSubModelConstraints represents a set of constraints to be applied to a set of submodels. The submodels are specified by the fform field, which contains the identifier of the submodel. The constraints field contains the constraints to be applied to all instances of this submodel on this level in the model hierarchy.

See also: GraphPPL.SpecificSubModelConstraints

source
GraphPPL.FactorizationConstraint — Type
FactorizationConstraint{V, F}

A FactorizationConstraint represents a single factorization constraint in a variational posterior constraint specification. We use type parametrization to dispatch on different types of constraints, for example q(x, y) = MeanField() is treated different from q(x, y) = q(x)q(y).

The FactorizationConstraint constructor checks for obvious errors, such as duplicate variables in the constraint specification and checks if the left hand side and right hand side contain the same variables.

See also: [`GraphPPL.FactorizationConstraintEntry`](@ref)
source
GraphPPL.MeanField — Type
MeanField

Generic factorisation constraint used to specify a mean-field factorisation for recognition distribution q. This constraint ignores default_constraints from submodels and forces everything to be factorized.

See also: BetheFactorization

source
GraphPPL.BetheFactorization — Function
BetheFactorization

Generic factorisation constraint used to specify the Bethe factorisation for recognition distribution q. An alias to UnspecifiedConstraints.

See also: MeanField

source
GraphPPL.MarginalFormConstraint — Type

A MarginalFormConstraint represents a single functional form constraint in a variational marginal constraint specification. We use type parametrization to dispatch on different types of constraints, for example q(x, y) :: MvNormal should be treated different from q(x) :: Normal.

source
GraphPPL.MessageFormConstraint — Type

A MessageConstraint represents a single constraint on the messages in a message passing schema. These constraints closely resemble the MarginalFormConstraint but are used to specify constraints on the messages in a message passing schema.

source
GraphPPL.materialize_constraints! — Function
materialize_constraints!(model::Model, node_label::NodeLabel, node_data::NodeData)

Materializes the factorization constraint in node_data in model at node_label. This function converts the BitSet representation of a constraint in node_data to the tuple representation containing all interface names.

source
GraphPPL.factorization_split — Function
factorization_split(left, right)

Creates a new FactorizationConstraintEntry that contains a SplittedRange splitting left and right. This function is used to convert two FactorizationConstraintEntrys (for example q(x[begin])..q(x[end])) into a single FactorizationConstraintEntry containing the SplittedRange.

See also: [`GraphPPL.SplittedRange`](@ref)
source
GraphPPL.SplittedRange — Type
SplittedRange{L, R}

SplittedRange represents a range of splitted variable in factorization specification language. Such variables specified to be not in the same factorization cluster.

See also: GraphPPL.CombinedRange

source