ExponentialFamilyProjection

ProjectionParameters(; kwargs...)

A type to hold the parameters for the projection procedure. The following parameters are available:

• strategy = ExponentialFamilyProjection.DefaultStrategy(): The strategy to use to compute the gradients.
• niterations = 100: The number of iterations for the optimization procedure.
• tolerance = 1e-6: The tolerance for the norm of the gradient.
• stepsize = ConstantStepsize(0.1): The stepsize for the optimization procedure. Accepts stepsizes from Manopt.jl.
• seed: Optional; Seed for the rng
• rng: Optional; Random number generator
• direction = BoundedNormUpdateRule(static(1.0): Direction update rule. Accepts Manopt.DirectionUpdateRule from Manopt.jl.

DefaultProjectionParameters()

Return the default parameters for the projection procedure.

getinitialpoint(strategy, M::AbstractManifold, parameters::ProjectionParameters)

Returns an initial point to start optimization from. By default returns a rand point from M, but different strategies may implement their own methods.

ProjectedTo(::Type{T}, dims...; conditioner = nothing, parameters = DefaultProjectionParameters)

A specification of a projection to an exponential family distribution.

The following arguments are required:

• Type{T}: a type of an exponential family member to project to, e.g. Beta
• dims...: dimensions of the distribution, e.g. 2 for MvNormal

The following arguments are optional:

• conditioner = nothing: a conditioner to use for the projection, not all exponential family members require a conditioner, but some do, e.g. Laplace
• parameters = DefaultProjectionParameters: parameters for the projection procedure
• kwargs = nothing: Additional arguments passed to Manopt.gradient_descent! (optional). For details on gradient_descent! parameters, see the Manopt.jl documentation. Note, that kwargs passed to project_to take precedence over kwargs specified in the parameters.

julia> using ExponentialFamily

julia> projected_to = ProjectedTo(Beta)
ProjectedTo(Beta)

julia> projected_to = ProjectedTo(Beta, parameters = ProjectionParameters(niterations = 10))
ProjectedTo(Beta)

julia> projected_to = ProjectedTo(MvNormalMeanCovariance, 2)
ProjectedTo(MvNormalMeanCovariance, dims = 2)

julia> projected_to = ProjectedTo(Laplace, conditioner = 2.0)
ProjectedTo(Laplace, conditioner = 2.0)

project_to(to::ProjectedTo, argument::F, supplementary..., initialpoint, kwargs...)

Finds the closest projection of argument onto the exponential family distribution specified by to.

Arguments

• to::ProjectedTo: Configuration for the projection. Refer to ProjectedTo for detailed information.
• argument::F: An (un-normalized) function representing the log-PDF of an arbitrary distribution or a list of samples.
• supplementary...: Additional distributions to project the product of argument and these distributions (optional).
• initialpoint: Starting point for the optimization process (optional).
• kwargs...: Additional arguments passed to Manopt.gradient_descent! (optional). For details on gradient_descent! parameters, see the Manopt.jl documentation.

Supplementary

The supplementary distributions must match the type and conditioner of the target distribution specified in to. Including supplementary distributions is equivalent to modified argument function as follows:

f_modified = (x) -> argument(x) + logpdf(supplementary[1], x) + logpdf(supplementary[2], x) + ...

julia> using ExponentialFamily, BayesBase

julia> f = (x) -> logpdf(Beta(30.14, 2.71), x);

julia> prj = ProjectedTo(Beta; parameters = ProjectionParameters(niterations = 500))
ProjectedTo(Beta)

julia> project_to(prj, f) isa ExponentialFamily.Beta
true

julia> using ExponentialFamily, BayesBase, StableRNGs

julia> samples = rand(StableRNG(42), Beta(30.14, 2.71), 1_000);

julia> prj = ProjectedTo(Beta; parameters = ProjectionParameters(tolerance = 1e-2))
ProjectedTo(Beta)

julia> project_to(prj, samples) isa ExponentialFamily.Beta
true

DefaultStrategy

The DefaultStrategy selects the optimal projection strategy based on the type of the second argument provided to the project_to function.

Rules:

• If the second argument is an AbstractArray, use MLEStrategy.
• For all other types, use ControlVariateStrategy.

ControlVariateStrategy(; kwargs...)

A strategy for gradient descent optimization and gradients computations that resembles the REINFORCE gradient estimator.

The following parameters are available:

• nsamples = 2000: The number of samples to use for estimates
• buffer = StaticTools.MallocSlabBuffer(): Advanced option; A buffer for temporary computations

MLEStrategy()

A strategy for gradient descent optimization and gradients computations that resembles MLE estimation.

preprocess_strategy_argument(strategy, argument)

Checks the compatibility of strategy with argument and returns a modified strategy and argument if needed.

create_state!(
    strategy,
    M::AbstractManifold,
    parameters::ProjectionParameters,
    projection_argument,
    initial_ef,
    supplementary_η,
)

Creates, initializes and returns a state for the strategy with the given parameters.

prepare_state!(
    strategy,
    state,
    M::AbstractManifold,
    parameters::ProjectionParameters,
    projection_argument,
    distribution,
    supplementary_η,
)

Prepares an existing state of the strategy for the new optimization iteration for use by setting or updating its internal parameters.

compute_cost(
    M::AbstractManifold,
    strategy,
    state,
    η,
    logpartition,
    gradlogpartition,
    inv_fisher,
)

Compute the cost using the provided strategy.

Arguments

• M::AbstractManifold: The manifold on which the computations are performed.
• strategy: The strategy used for computation of the cost value.
• state: The current state for the strategy.
• η: Parameter vector.
• logpartition: The log partition of the current point (η).
• gradlogpartition: The gradient of the log partition of the current point (η).
• inv_fisher: The inverse Fisher information matrix of the current point (η).

Returns

• cost: The computed cost value.

compute_gradient!(
    M::AbstractManifold,
    strategy,
    state,
    X,
    η,
    logpartition,
    gradlogpartition,
    inv_fisher,
)

Updates the gradient X in-place using the provided strategy.

Arguments

• M::AbstractManifold: The manifold on which the computations are performed.
• strategy: The strategy used for computation of the gradient value.
• state: The current state of the control variate strategy.
• X: The storage for the gradient.
• η: Parameter vector.
• logpartition: The log partition of the current point (η).
• gradlogpartition: The gradient of the log partition of the current point (η).
• inv_fisher: The inverse Fisher information matrix of the current point (η).

Returns

• X: The computed gradient (updated in-place)

ProjectionCostGradientObjective

This structure provides an interface for Manopt to compute the cost and gradients required for the optimization procedure based on manifold projection. The actual computation of costs and gradients is defined by the strategy argument.

Arguments

• projection_parameters: The parameters for projection, must be of type ProjectionParameters
• projection_argument: The second argument of the project_to function.
• current_η: Current optimization point.
• supplementary_η: A tuple of additional natural parameters subtracted from the current point in each optimization iteration.
• strategy: Specifies the method for computing costs and gradients, which may support different projection_argument values.
• strategy_state: The state for the strategy, usually created with create_state!

BoundedNormUpdateRule(limit; direction = IdentityUpdateRule())

A DirectionUpdateRule is a direction rule that constrains the norm of the direction to a specified limit.

This rule operates in two steps:

• Initial direction computation: It first applies the specified direction update rule to compute an initial direction.
• Norm check and scaling: The norm of the resulting direction vector is checked using Manopt.norm(M, p, d), where:
  • M` is the manifold on which the optimization is running,
  • p is the point at which the direction was computed,
  • d is the computed direction.
  • If this norm exceeds the specified limit, the direction vector is scaled down so that its new norm exactly equals the limit. This scaling preserves the direction of the gradient while controlling its magnitude.

Read more about Manopt.DirectionUpdateRule in the Manopt.jl documentation.