# Built-in Functional Forms

This section describes built-in functional forms that can be used for posterior marginal and/or messages form constraints specification. Read more information about constraints specification syntax in the Constraints Specification section.

## Background

In message passing framework, in order to compute a posterior over some latent state $q(x)$, it is necessary to compute a normalized product of two messages $q(x) = \frac{\mu_1(x) \mu_2(x)}{\int \mu_1(x) \mu_2(x) \mathrm{d}x}.$ In some situations, when functional forms of $\mu_1(x)$ and $\mu_2(x)$ are know in advance, it is possible to compute the normalized product efficiently and analytically. It is, however, not always the case, since the messages can have arbitrary functional form and it is not always easy to compute the normalization factor.

Functional forms help to circumvent this. They implement a custom callback on the product of two messages, which cannot be computed analytically. Essentially, a functional form constraint defines a functional `F`

, such that $q(x) = F[\mu_1, \mu_2] \approx \frac{\mu_1(x) \mu_2(x)}{\int \mu_1(x) \mu_2(x) \mathrm{d}x}.$

See also Bethe Free Energy section for more information on variational inference and posterior computation.

## UnspecifiedFormConstraint

Unspecified functional form constraint is used by default and uses only analytical update rules for computing posterior marginals. Throws an error if a product of two colliding messages cannot be computed analytically.

```
@constraints begin
# This is the default setting for all latent variables
q(x) :: UnspecifiedFormConstraint()
end
```

## PointMassFormConstraint

The most basic form of posterior marginal approximation is the `PointMass`

function. In a few words `PointMass`

represents the delta function. In the context of functional form constraints `PointMass`

approximation corresponds to the MAP estimate. For a given distribution `d`

- `PointMass`

functional form simply finds the `argmax`

of the `logpdf`

of `q(x)`

, thus $q(x) = F[\mu_1, \mu_2] = \delta(x - \arg\min_{x} \mu_1(x) \mu_2(x))$. This is especially useful when exact functional form of `q(x)`

is not available or cannot be parametrized efficiently.

```
@constraints begin
q(x) :: PointMassFormConstraint()
end
@constraints begin
q(x) :: PointMassFormConstraint(starting_point = (args...) -> 1.0)
end
```

`RxInfer.PointMassFormConstraint`

— Type`PointMassFormConstraint`

One of the form constraint objects. Constraint a message to be in a form of dirac's delta point mass. By default uses `Optim.jl`

package to find argmin of `-logpdf(x)`

. Accepts custom `optimizer`

callback which might be used to customise optimisation procedure with different packages or different arguments for `Optim.jl`

package.

**Keyword arguments**

`optimizer`

: specifies a callback function for logpdf optimisation. See also:`RxInfer.default_point_mass_form_constraint_optimizer`

`starting_point`

: specifies a callback function for initial optimisation point: See also:`RxInfer.default_point_mass_form_constraint_starting_point`

`boundaries`

: specifies a callback function for determining optimisation boundaries: See also:`RxInfer.default_point_mass_form_constraint_boundaries`

**Custom optimizer callback interface**

```
# This is an example of the `custom_optimizer` interface
function custom_optimizer(::Type{ Univariate }, ::Type{ Continuous }, constraint::PointMassFormConstraint, distribution)
# should return argmin of the -logpdf(distribution)
end
```

**Custom starting point callback interface**

```
# This is an example of the `custom_starting_point` interface
function custom_starting_point(::Type{ Univariate }, ::Type{ Continuous }, constraint::PointMassFormConstraint, distribution)
# built-in optimizer expects an array, even for a univariate distribution
return [ 0.0 ]
end
```

**Custom boundaries callback interface**

```
# This is an example of the `custom_boundaries` interface
function custom_boundaries(::Type{ Univariate }, ::Type{ Continuous }, constraint::PointMassFormConstraint, distribution)
# returns a tuple of `lower` and `upper` boundaries
return (-Inf, Inf)
end
```

`RxInfer.default_point_mass_form_constraint_optimizer`

— Function`default_point_mass_form_constraint_optimizer(::Type{<:VariateType}, ::Type{<:ValueSupport}, constraint::PointMassFormConstraint, distribution)`

Defines a default optimisation procedure for the `PointMassFormConstraint`

. By default uses `Optim.jl`

package to find argmin of `-logpdf(x)`

. Uses the `starting_point`

and `boundaries`

callbacks to determine the starting point and boundaries for the optimisation procedure.

`RxInfer.default_point_mass_form_constraint_starting_point`

— Function`default_point_mass_form_constraint_starting_point(::Type{<:VariateType}, ::Type{<:ValueSupport}, constraint::PointMassFormConstraint, distribution)`

Defines a default starting point for the `PointMassFormConstraint`

. By default uses the support of the distribution. If support is unbounded returns a zero point. Otherwise throws an error.

`RxInfer.default_point_mass_form_constraint_boundaries`

— Function`default_point_mass_form_constraint_boundaries(::Type{<:VariateType}, ::Type{<:ValueSupport}, constraint::PointMassFormConstraint, distribution)`

Defines a default boundaries for the `PointMassFormConstraint`

. By default simply uses the support of the distribution.

## SampleListFormConstraint

`SampleListFormConstraints`

approximates the resulting posterior marginal (product of two colliding messages) as a list of weighted samples. Hence, it requires one of the arguments to be a proper distribution (or at least the inference backend should be able to sample from it). This setting is controlled with `LeftProposal()`

, `RightProposal()`

or `AutoProposal()`

objects. It also accepts an optional `method`

object, but the only one available sampling method currently is the `BayesBase.BootstrapImportanceSampling`

.

```
@constraints begin
q(x) :: SampleListFormConstraint(1000)
# or
q(y) :: SampleListFormConstraint(1000, LeftProposal())
end
```

`RxInfer.SampleListFormConstraint`

— Type`SampleListFormConstraint(rng, strategy, method)`

One of the form constraint objects. Approximates `DistProduct`

with a SampleList object.

`RxInfer.AutoProposal`

— TypeTries to determine the proposal distribution in the SampleList approximation automatically.

`RxInfer.LeftProposal`

— TypeUses the left argument in the `prod`

call as the proposal distribution in the SampleList approximation.

`RxInfer.RightProposal`

— TypeUses the right argument in the `prod`

call as the proposal distribution in the SampleList approximation.

## FixedMarginalFormConstraint

Fixed marginal form constraint replaces the resulting posterior marginal obtained during the inference procedure with the prespecified one. Worth to note that the inference backend still tries to compute real posterior marginal and may fail during this process. Might be useful for debugging purposes. If `nothing`

is passed then the computed posterior marginal is returned (see also `UnspecifiedFormConstraint`

).

```
@constraints function block_updates(x_posterior = nothing)
# `nothing` returns the computed posterior marginal
q(x) :: FixedMarginalFormConstraint(x_posterior)
end
```

`RxInfer.FixedMarginalFormConstraint`

— Type`FixedMarginalFormConstraint`

One of the form constraint objects. Provides a constraint on the marginal distribution such that it remains fixed during inference. Can be viewed as blocking of updates of a specific edge associated with the marginal. If `nothing`

is passed then the computed posterior marginal is returned. Use `.fixed_value`

field to update the value of the constraint.

It is also possible to control the constraint manually, e.g:

```
form_constraint = FixedMarginalFormConstraint(nothing)
constraints_specification = @constraints function manual_block_updates(form_constraint)
q(x) :: form_constraint
end
# later on ...
form_constraint.fixed_value = Gamma(1.0, 1.0)
```

`Gamma{Float64}(α=1.0, θ=1.0)`

## CompositeFormConstraint

It is possible to create a composite functional form by stacking operators, e.g:

```
@constraints begin
q(x) :: SampleListFormConstraint(1000) :: PointMassFormConstraint()
end
```

```
Constraints:
q(x) :: (SampleListFormConstraint(Random._GLOBAL_RNG(), AutoProposal(), BayesBase.BootstrapImportanceSampling()), PointMassFormConstraint())
```

## Custom functional forms

See the ReactiveMP.jl library documentation for more information about defining novel custom functional forms that are compatible with `ReactiveMP`

inference backend.