# Constraint Specification

`GraphPPL`

represents your probabilistic model and as a Bethe Free Energy (BFE), which means that users can define constraints on the variational posterior that influence the inference procedure. The BFE is chosen as the objective function because it is a generalization of many well-known inference algorithms. In this section we will explain how to specify constraints on the variational posterior. There are two major types of constraints we can apply: We can apply factorization constraints to factor nodes, which specify how the variational posterior factorizes around a factor node. We can also apply functional form constraints to variable nodes, which specify the functional form of the variational posterior that a variable takes. We can specify all constraints using the `@constraints`

macro.

## The constraints macro

The constraints macro accepts a high-level constraint specification and converts this to a structure that can be interpreted by `GraphPPL`

models. For example, suppose we have the following toy model, that defines a Gaussian distribution over `x`

with mean `y`

and variance `z`

:

```
using GraphPPL
using Distributions
import GraphPPL: @model
@model function toy_model(x, y, z)
x ~ Normal(y, z)
end
```

Suppose we want to apply the following constraints over the variational posterior `q`

:

\[q(x, y, z) = q(x, y)q(z) \\ q(x) \sim Normal\]

We can write this in the constraints macro using the following code:

```
@constraints begin
q(x, y, z) = q(x, y)q(z)
q(x) :: Normal
end
```

```
Constraints:
q(x, y, z) = q(x, y)q(z)
q(x) :: Distributions.Normal
```

We can reference variables in the constraints macro with their corresponding name in the model specification. Naturally, this raises the question on how we can specify constraints over variables in submodels, as these variables are not available in the scope of the model specification. To this extent, we can nest our constraints in the same way in which we have nested our models, and use the `for q in submodel`

block to specify constraints over submodels. For example, suppose we have the following model:

```
@model function toy_model(x, y, z)
x ~ Normal(y, z)
y ~ Normal(0, 1)
end
@model function outer_toy_model(a, b, c)
a ~ toy_model(y = b, z = c)
end
```

We can specify constraints over the `toy_model`

submodel using the following code:

```
@constraints begin
for q in toy_model
q(x, y, z) = q(x, y)q(z)
q(x) :: Normal
end
end
```

```
Constraints:
q(toy_model) =
q(x, y, z) = q(x, y)q(z)
q(x) :: Distributions.Normal
```

The submodel constraint specification applies to all submodels with the same name. However, as a user you might want to specify constraints over a specific submodel. To this extent, we can use the `for q in (submodel, index)`

syntax. This will only apply the constraints to the submodel with the corresponding index. For example, suppose we have the following model:

```
@model function toy_model(x, y, z)
x ~ Normal(y, z)
y ~ Normal(0, 1)
end
@model function outer_toy_model(a, b, c)
a ~ toy_model(y = b, z = c)
a ~ toy_model(y = b, z = c)
end
```

We can specify constraints over the first `toy_model`

submodel using the following code:

```
@constraints begin
for q in (toy_model, 1)
q(x, y, z) = q(x, y)q(z)
q(x) :: Normal
end
end
```

```
Constraints:
q((toy_model, 1)) =
q(x, y, z) = q(x, y)q(z)
q(x) :: Distributions.Normal
```

## Stacked functional form constraints

In the constraints macro, we can specify multiple functional form constraints over the same variable. For example, suppose we have the following model:

```
@constraints begin
q(x) :: Normal :: Beta
end
```

```
Constraints:
q(x) :: (Distributions.Normal, Distributions.Beta)
```

In this constraint the posterior over `x`

will first be constrained to be a normal distribution, and then the result with be constrained to be a beta distribution. This might be useful to create a chain of constraints that are applied in order. The resulting constraint is a tuple of constraints.

The inference backend must support stacked constraints for this feature to work. Some combinations of stacked constraints might not be supported or theoretically sound.

## Default constraints

While we can specify constraints over all instances of a submodel at a specific layer of the hierarchy, we're not guaranteed to have all instances of a submodel at a specific layer of the hierarchy. To this extent, we can specify default constraints that apply to all instances of a specific submodel. For example, we can define the following model, where we have a `recursive_model`

instance at every layer of the hierarchy:

```
@model function recursive_model(n, x, y)
z ~ Gamma(1, 1)
if n > 0
y ~ Normal(recursive_model(n = n - 1, x = x), z)
else
y ~ Normal(0, z)
end
end
```

We can specify default constraints over the `recursive_model`

submodel using the following code:

```
GraphPPL.default_constraints(::typeof(recursive_model)) = @constraints begin
q(x, y, z) = q(x)q(y)q(z)
end
```

When a model of type `recursive_model`

is now created, the default constraints will be applied to all instances of the `recursive_model`

submodel. Note that default constraints are overwritten by constraints passed to the top-level model, if they concern the same instance of a submodel.

## Prespecified constraints

`GraphPPL`

provides a set of prespecified constraints that can be used to specify constraints over the variational posterior. These constraint sets are aliases for their corresponding equivalent constriant sets, and can be used for convenience. The following prespecified constraints are available:

`GraphPPL.MeanField`

— Type`MeanField`

Generic factorisation constraint used to specify a mean-field factorisation for recognition distribution `q`

. This constraint ignores `default_constraints`

from submodels and forces everything to be factorized.

See also: `BetheFactorization`

`GraphPPL.BetheFactorization`

— Function`BetheFactorization`

Generic factorisation constraint used to specify the Bethe factorisation for recognition distribution `q`

. An alias to `UnspecifiedConstraints`

.

See also: `MeanField`

This means that we can write the following:

```
@constraints begin
q(x, y, z) = MeanField() # Equivalent to q(x, y, z) = q(x)q(y)q(z)
q(a, b, c) = BetheFactorization() # Equivalent to q(a, b, c) = q(a, b, c), can be used to overwrite default constraints.
end
```

```
Constraints:
q(x, y, z) = MeanField()
q(a, b, c) = Constraints:
```

## Plugin's internals

`GraphPPL.VariationalConstraintsPlugin`

— Type`VariationalConstraintsPlugin(constraints)`

A plugin that adds a VI related properties to the factor node for the variational inference procedure.

`GraphPPL.Constraints`

— Type`Constraints`

An instance of `Constraints`

represents a set of constraints to be applied to a variational posterior in a factor graph model.

`GraphPPL.SpecificSubModelConstraints`

— Type`SpecificSubModelConstraints`

A `SpecificSubModelConstraints`

represents a set of constraints to be applied to a specific submodel. The submodel is specified by the `tag`

field, which contains the identifier of the submodel.

See also: `GraphPPL.GeneralSubModelConstraints`

`GraphPPL.GeneralSubModelConstraints`

— Type`GeneralSubModelConstraints`

A `GeneralSubModelConstraints`

represents a set of constraints to be applied to a set of submodels. The submodels are specified by the `fform`

field, which contains the identifier of the submodel. The `constraints`

field contains the constraints to be applied to all instances of this submodel on this level in the model hierarchy.

See also: `GraphPPL.SpecificSubModelConstraints`

`GraphPPL.FactorizationConstraint`

— Type`FactorizationConstraint{V, F}`

A `FactorizationConstraint`

represents a single factorization constraint in a variational posterior constraint specification. We use type parametrization to dispatch on different types of constraints, for example `q(x, y) = MeanField()`

is treated different from `q(x, y) = q(x)q(y)`

.

The `FactorizationConstraint`

constructor checks for obvious errors, such as duplicate variables in the constraint specification and checks if the left hand side and right hand side contain the same variables.

`See also: [`GraphPPL.FactorizationConstraintEntry`](@ref)`

`GraphPPL.FactorizationConstraintEntry`

— Type`FactorizationConstraintEntry`

A `FactorizationConstraintEntry`

is a group of variables (represented as a `Vector`

of `IndexedVariable`

objects) that represents a factor group in a factorization constraint.

See also: `GraphPPL.FactorizationConstraint`

`GraphPPL.MarginalFormConstraint`

— TypeA `MarginalFormConstraint`

represents a single functional form constraint in a variational marginal constraint specification. We use type parametrization to dispatch on different types of constraints, for example `q(x, y) :: MvNormal`

should be treated different from `q(x) :: Normal`

.

`GraphPPL.MessageFormConstraint`

— TypeA `MessageConstraint`

represents a single constraint on the messages in a message passing schema. These constraints closely resemble the `MarginalFormConstraint`

but are used to specify constraints on the messages in a message passing schema.

`GraphPPL.materialize_constraints!`

— Function`materialize_constraints!(model::Model, node_label::NodeLabel, node_data::NodeData)`

Materializes the factorization constraint in `node_data`

in `model`

at `node_label`

. This function converts the BitSet representation of a constraint in `node_data`

to the tuple representation containing all interface names.

`GraphPPL.factorization_split`

— Function`factorization_split(left, right)`

Creates a new `FactorizationConstraintEntry`

that contains a `SplittedRange`

splitting `left`

and `right`

. This function is used to convert two `FactorizationConstraintEntry`

s (for example `q(x[begin])..q(x[end])`

) into a single `FactorizationConstraintEntry`

containing the `SplittedRange`

.

`See also: [`GraphPPL.SplittedRange`](@ref)`

`GraphPPL.SplittedRange`

— Type`SplittedRange{L, R}`

`SplittedRange`

represents a range of splitted variable in factorization specification language. Such variables specified to be **not** in the same factorization cluster.

See also: `GraphPPL.CombinedRange`

`GraphPPL.CombinedRange`

— Type`CombinedRange{L, R}`

`CombinedRange`

represents a range of combined variable in factorization specification language. Such variables specified to be in the same factorization cluster.

See also: `GraphPPL.SplittedRange`