Type instability of `add_foo_container`

[luke-kiernan]

If you run performance_test.jl, you'll see that the first run of build! take almost 2 minutes, and the second run takes merely 2 seconds. Much of that extra time is spent doing type inference, which says that our code has bad type stability issues. I think I've identified the culprit: it's the _add_foo_container! functions. Namely, we do:

function _add_variable_container!(
    container::OptimizationContainer,
    ...;
    sparse::Bool,
) where {T <: VariableType, U <: Union{PSY.Component, PSY.System}}
    if sparse
        var_container = sparse_container_spec(JuMP.VariableRef, axs...)
    else
        var_container = container_spec(JuMP.VariableRef, axs...)
    end
    _assign_container!(container.variables, var_key, var_container)
end

function _add_param_container!(
    container::OptimizationContainer,
    ....
)
    if built_for_recurrent_solves(container) && !get_rebuild_model(get_settings(container))
        param_type = JuMP.VariableRef
    else
        param_type = Float64
    end
    # add param container with that type
end

Types of core containers depend on runtime values: that's bad type instability. Steps that would help:

1. Make rebuild_model a type parameter of Settings. Similarly, OptimizationContainer should be parameterized on built_for_recurrent_solves and on rebuild_model, so that the if branch can be deduced at compile time. DecisionModel passes its built_for_recurrent_solves field along to OptimizationContainer, so that one would likely need to become a type parameter too.
2. Change built_for_recurrent_solves and sparse to positional arguments. Julia does not specialize or dispatch on keyword arguments. Looking through the code, it looks like we always know the value of sparse at compile time (it's never provided by the user), so we could even go one step further and use Val{true} and Val{False}.

The first item above should be pretty doable, as long as we're ok with always providing built_for_recurrent_solves upon initialization and its value remaining the same over the lifetime of the object. The second item breaks backwards compatibility--it'd change the public interface of DecisionModel--so that's more drastic.

[luke-kiernan]

In the branch lk/add_sparse_container, I split off the sparse case into separate functions named add_sparse_foo_container!. However, the compilation improvements are pretty marginal, maybe 10% improvement at best: running @snoop_inference, _assign_container! is still among the top functions for inference time.

Perhaps this type instability is not as big of an issue as I initially thought, and most of that inference time is coming from something other than these container types. Will continue looking into it.