Layer instantiator

首先檢視現有的 CachingOptimizer、AbstractBridgeOptimizer 和 MOI.ModelLike 介面，然後將它們與表中的 layer 案例進行比較，包括 Dualization、ParametricOptInterface、DiffOpt 和 MultiObjectiveAlgorithms。確定了 layer API 設計和支援規則後，該 issue 即完成，其中包括增量介面、索引共用、負索引以及自動快取行為。

It would be useful to have some kind of layer API so that the user could just do:

model = Model(MOI.layers(POI.Optimizer, HiGHS.Optimizer)

The challenge is the need to automatically add CachingOptimizer when needed. We could also automatically detected the needed coefficient type but I think for now, we can require the user to explicitly use {Float32} at each layer if he chooses to not use Float64.

It is challenging to find the right interface for this because it's quite complicated. But for the same reason, combining the layers is very difficult for our users and the errors are quite cryptic. So I think we should make the effort to find something that works and make the layers plug&play

Current issues

Bridge layers

Bridge layers don't create index maps for efficiency reason. What they do is use negative indices for the constraint that are bridged. This means that you cannot stack two bridge layers without a CachingOptimizer in between. I remember @chriscoey and @lkapelevich being hit by this bug when stacking SingleBridgeOptimizer layers on top of Hypatia. @GiovanniKarra was also hit by this issue with SingleBridgeOptimizer. I also got hit by this recently in https://github.com/blegat/ComplementOpt.jl/pull/29 because ComplementOpt.Optimizer is a subtype of AbstractBridgeOptimizer and MOI.instantiate(() -> ComplementOpt.Optimizer(MOI.instantiate(Ipopt.Optimizer)) was creating a ComplementOpt.Optimizer layer with a LazyBridgeOptimizer layer directly following it so I had to explicitly create a cache in between like so: https://github.com/blegat/ComplementOpt.jl/blob/c852c0fd7016528e352a3a0b7158d611b51f0aad/test/runtests.jl#L322-L328

Need for incremental interface

POI needs an incremental interface and bridge layers do too. So a caching optimizer should automatically be added if needed.

Solution

The implementation would be something like

# This works for all layers of the table below
MOI.requires_incremental_interface(::Type{<:MOI.ModelLike}) = true

MOI.layers(optimizer_constructor; kws...) = MOI.instantiate(optimizer_constructor; kws...)
function MOI.layers(layer_type::Type{<:MOI.ModelLike}, args...; kws...)
    model = MOI.layers(args...; kws...)
    if (MOI.requires_incremental_interface(layer_type) && !MOI.supports_incremental_interface(model)) ||
        (layer_type <: MOI.AbstractBridgeOptimizer && model isa MOI.AbstractBridgeOptimizer)
         model = MOI.CachingOptimizer(# add a caching optimizer on top of `model`
    end
    return layer_type(model)
end

This does not completely resolves the issue with bridges. You could also have that model is not a bridge layer but its inner layer is a bridge optimizer and model does not map indices.

For a complete solution, we could have something like this that would be useful for https://github.com/jump-dev/JuMP.jl/issues/4014

abstract type Layer <: MOI.AbstractOptimizer end # Make `CachingOptimizer` and `AbstractBridgeOptimizer` be subtype of that
MOI.inner_optimizer(model::MOI.Layer) = model.inner # Name inner by convention ?
MOI.inner_optimizer(model::MOI.Bridges.AbstractBridgeOptimizer) = model.model
MOI.inner_optimizer(model::MOI.Utilities.CachingOptimizer) = model.Optimizer

Then, we add the following that should be implemented for any subtype of MOI.Layer (in addition to MOI.requires_incremental_interface)

MOI.share_indices_with_inner_optimizer(model::MOI.Layer) = MOI.supports_incremental_interface(model)
# Only exception to the above default implemention according to table below
MOI.share_indices_with_inner_optimizer(model::MOI.Utilities.CachingOptimizer) = false

and then the following one that shouldn't be implemented for layers, it correspond to the "index map" column in the above table

MOI.may_have_negative_indices(model::MOI.ModelLike) = false
MOI.may_have_negative_indices(model::MOI.AbstractBridgeOptimizer) = true
MOI.may_have_negative_indices(model::MOI.Layer) = MOI.share_indices_with_inner_optimizer(model) && MOI.may_have_negative_indices(MOI.inner_optimizer(model))

Then, we can do

function MOI.layers(layer_type::Type{<:MOI.Layer}, args...; kws...)
    model = MOI.layers(args...; kws...)
    if (MOI.requires_incremental_interface(layer_type) && !MOI.supports_incremental_interface(model)) ||
        (layer_type <: MOI.AbstractBridgeOptimizer && MOI.may_have_negative_indices(model))
         model = MOI.CachingOptimizer(# add a caching optimizer on top of `model`
    end
    return layer_type(model)
end

Table

Let's use this issue to collect the list of layers we want to support and their particularities before we commit with a specific design.

Layer	supports_incremental_interface	requires_incremental_interface	share_indices_with_inner_optimizer
CachingOptimizer	✓	✓	✘
AbstractBridgeOptimizer	✓	✓	✓
Dualization	✘	✓	✘
ParametricOptInterface	✓	✓	✓
DiffOpt	✓	✓	✓
MultiObjectiveAlgorithms	✓	✓	✓

Given that all layers require the incremental interface anyway, maybe we can remove requires_incremental_interface and only add it once we have a solver that needs it so the implementation is simply

function MOI.layers(layer_type::Type{<:MOI.Layer}, args...; kws...)
    model = MOI.layers(args...; kws...)
    if !MOI.supports_incremental_interface(model) ||
        (layer_type <: MOI.AbstractBridgeOptimizer && MOI.may_have_negative_indices(model))
         model = MOI.CachingOptimizer(# add a caching optimizer on top of `model`
    end
    return layer_type(model)
end