ENH: Add a POMDPs.jl interface for DiscreteDP (extension or companion package)

[oyamad]

Proposal

Add a bridge between DiscreteDP and POMDPs.jl, the standard interface of the JuliaPOMDP ecosystem — tentatively as a package extension ext/DiscreteDPPOMDPsExt.jl (naming open; see the placement question at the end). DiscreteDP is a finite MDP in POMDPs.jl's terms, and it fits the ecosystem's "standard case" (explicit, enumerable spaces; see their offline-solver example) exactly, so unlike the continuous case (QuantEcon/ContinuousDPs.jl#89, the model-direction-only sibling of this proposal) both directions of the bridge work:

• Model direction: wrap a DiscreteDP as a POMDPs.MDP, so that every discrete solver in the ecosystem can consume models built with QuantEcon.jl.
• Solver direction: expose our solvers as a POMDPs.Solver, so that any finite explicit POMDPs.jl model can be solved with QuantEcon's policy iteration and modified policy iteration.

Rough design

Model direction (DiscreteDP as POMDPs.MDP{Int,Int}): states(m) = 1:num_states, state-dependent actions(m, s) from the state-action-pairs formulation, transition(m, s, a) as a SparseCat over the corresponding row of Q, reward(m, s, a) from R, discount(m) = beta, and trivial stateindex/actionindex.

Solver direction:

struct DiscreteDPSolver{A<:DDPAlgorithm} <: POMDPs.Solver
    algo::A  # VFI / PFI / MPFI
    # max_iter, epsilon, ...
end

function POMDPs.solve(sol::DiscreteDPSolver, m::POMDPs.MDP)
    # enumerate ordered_states(m) / actions(m, s);
    # materialize s_indices, a_indices,
    #   R[k] = E over transition(m, s, a) of reward(m, s, a, sp),
    #   Q rows from support/pdf of transition(m, s, a), via stateindex;
    # encode terminal states as zero-reward self-loops;
    # solve(DiscreteDP(R, Q, discount(m), s_indices, a_indices), sol.algo);
    # return a POMDPs.Policy wrapping the result (action/value via stateindex)
end

Main design decisions: terminal-state encoding (zero-reward self-loop is the natural one), using the expected form reward(m, s, a, sp) for widest model compatibility (as in the POMDPs.jl offline-solver example), dense-vs-sparse Q construction, and which solver options to surface.

Motivation: a concrete prototype for QuantEcon/QuantEcon.py#228

This bridge would serve, first and foremost, as a concrete working prototype for the interface design that QuantEcon/QuantEcon.py#228 needs.

QuantEcon/QuantEcon.py#228 (2016) asked for a simpler interface for specifying DiscreteDP models, and the discussion there started from dolo-style modeling languages. The durable core of that request is arguably narrower: a modeling language represents the model symbolically (model-as-data), which buys symbolic services DiscreteDP never uses — the solvers only ever evaluate R and Q entries. For construction, evaluation access is the entire requirement, so passing plain functions (model-as-code, with parameters captured in closures) suffices; the language layer belongs to dolo, which can compile down to this interface.

POMDPs.jl is a decade-tested instance of exactly this design: QuickPOMDPs defines models by passing functions as keyword arguments, TabularPOMDP is the arrays form (a DiscreteDP sibling), and the explicit-vs-generative split (transition vs gen) marks the same boundary as the materialize-vs-implicit backends discussed for QuantEcon/QuantEcon.py#228. Notably, no modeling language emerged in that ecosystem — functions proved sufficient.

The connection is concrete, not just analogical: the solver-direction solve above — enumerate states and feasible actions, query transition/reward, materialize R/Q, run our solvers — is the "construct a DiscreteDP from function definitions" builder, with the contract already designed and battle-tested by POMDPs.jl (actions(m, s) as the feasibility function, distribution objects as the transition contract, stateindex as the integer mapping). Implementing this bridge — or even just settling its design — therefore serves as a working prototype for QuantEcon/QuantEcon.py#228, and should inform that API before anything is frozen on the Python side.

Placement: extension inside QuantEcon.jl, or an independent package?

DiscreteDP is one component among many in QuantEcon.jl, which raises the question of whether QuantEcon.jl itself should carry a POMDPs-facing extension.

• ext/ in QuantEcon.jl: auto-loads when both packages are present; no new repository or registration. But QuantEcon.jl takes on the POMDPs.jl compat and test surface, and an extension serving a single component sets a precedent (would we add one per component per ecosystem?).
• Independent glue package (e.g. QuantEconPOMDPs.jl): matches the JuliaPOMDP convention, where every solver is its own registered package listed in their solver table — the solver direction in particular would be a first-class ecosystem citizen there; own release cadence and compat bounds; zero burden on QuantEcon.jl (everything the bridge needs — DiscreteDP, solve, the algorithm types — is public API, so no changes here are required at all). Costs: a separate repository and registration, tracking breaking releases of two upstreams, and explicit using instead of auto-loading.

🤖 Generated with Claude Code (Claude Fable 5)