Repository navigation
Description
Written by Claude (Claude Code) at the request of ocots, from a design discussion with ocots.
Objective
CTDirect writes each transcription once, without knowing the NLP modeler underneath. Adding a modeler (JuMP, for instance) then means implementing two contracts next to the modeler, in CTSolvers, with no change in CTModels, CTDirect or CTParser.
This must be done before §3 (transcription recipe) and §5 (own the Exa transcription) of the CTDirect redesign (#610, discussion #609): it fixes the NLP emission axis that the recipe is written against.
Problem: the transcription exists three times
| Modeler | Where the transcription lives | Problem |
|---|---|---|
| ADNLP | CTDirect (src/ode/*.jl) |
calls CTModels through a single convention, the in-place f!(r, t, x, u, v) |
| Exa | CTParser, code generated at @def time (onepass.jl, if scheme == … chains) |
4 schemes, no custom grid, no path/boundary multipliers, only for @def problems; the result is stored in Model.build_examodel, so CTModels holds an already discretized object |
| JuMP | OptimalControlProblems, by hand (control-toolbox/OptimalControlProblems.jl#383) | a third copy |
Each new modeler needs a new transcription, and each new scheme must be written in every copy. The scheme conventions already drift between copies (final control, aliases: control-toolbox/CTParser.jl#355, time grid: #638).
Principle
One transcription, in CTDirect. CTDirect holds the modeler and passes it to every API call; that argument drives the dispatch. CTDirect uses two APIs, each taking the modeler:
- Evaluate the OCP: get the dynamics, costs and constraints in the form the modeler consumes. API of CTModels.
- Build the NLP: declare variables, families of constraints and objective terms. API of CTSolvers, where the modelers live.
Roles
CTModels: describe the problem, select the representation
- Each component (dynamics, Lagrange, Mayer, path and boundary constraints) stores several redundant representations, consistent by construction when CTParser fills them from one expression:
struct Dynamics{F,G,E,S} inplace::F # f!(r, t, x, u, v): always present traceable::G # generic out-of-place form, per component, or nothing expression::E # Expr per component, or nothing sparsity::S # dependencies of each component on (t, x, u, v), or nothing end
- Traits tell what is available (
has_traceable,has_expression,has_sparsity). - Forms and selection: CTModels defines the forms (
InPlaceForm,TraceableForm) and the generic functionrequired_form. The API takes the modeler, for dispatch and for its options (Strategies.options(m), e.g.base_typeon GPU):dynamics(ocp, modeler) = dynamics(ocp, required_form(modeler), modeler)
- The API stays continuous
(t, x, u, v): no grid, no scheme. - A hand-written model only gives
inplace: it works with ADNLP, and Exa or JuMP throw a clear error unless a traceable form is declared. - The expression is not used for evaluation (that would need
eval): it serves analysis (true sparsity, linear or box constraints), display, and as a fallback for code generation.
CTSolvers: the modelers (they stay there)
A modeler is an NLP concept (it already serves non-OCP problems, e.g. test/problems/rosenbrock.jl) and goes with the solvers (MadNLPGPU needs Exa on GPU). Thanks to required_form, CTModels does not need to know the modelers, and the dependency graph CTBase ← CTModels ← CTSolvers ← CTDirect does not change.
- Each modeler declares the form it consumes:
CTModels.required_form(::Modelers.Exa) = CTModels.TraceableForm(). - Each modeler implements the generic NLP-building API, modeler as argument:
Exa:
X = add_variables!(nlp, modeler, (n, N+1); lvar, uvar, start) # opaque handle c = add_constraints!(nlp, modeler, body, indices; lcon, ucon) # a family add_objective!(nlp, modeler, body, indices) model = build(nlp, modeler)
add_var/add_con/add_objon generators, traced once at build time (the tree is evaluated, also on GPU). ADNLP: families accumulated, thenfandc!loop over them. JuMP:@variable/@constraintby operator overloading. - The handles are used to read back the primal values and the multipliers the same way for every modeler.
CTDirect: the recipe, written once
Schemes (Butcher tableaux), layout, grids, families, quadrature. Optimized schemes (midpoint, FSAL trapeze) stay specializations by dispatch (#609 §3.3).
f = CTModels.dynamics(ocp, modeler)
X = add_variables!(nlp, modeler, (n, N+1); …)
U = add_variables!(nlp, modeler, (m, N+1); …)
defect(j) = X[j+1] - X[j] - h/2 * (f(t[j], X[j], U[j], v) + f(t[j+1], X[j+1], U[j+1], v))
add_constraints!(nlp, modeler, defect, 0:N-1; lcon=0, ucon=0)CTDirect would then no longer depend on ADNLPModels, ExaModels or CTParser.
CTParser: from expressions to the model
Translates the @def expressions into a Model with every representation of each component. No discretization any more: no grid, no scheme, no build_examodel.
Where each thing goes
| Concept | Package |
|---|---|
| OCP description, multi-representation components, traits | CTModels |
Forms, required_form (generic), dynamics(ocp, modeler) and the selection |
CTModels |
Modelers (strategies and options) and their required_form method |
CTSolvers.Modelers |
| NLP-building API, handles, reading the multipliers | CTSolvers.Modelers (contract), one extension per modeler |
Linear algebra on Exa nodes (today CTParserExaModels, control-toolbox/CTParser.jl#365) |
CTSolversExaModels, or upstream in ExaModels |
AbstractDiscretizer contract |
CTSolvers.DOCP (unchanged) |
| Schemes, layout, grids, recipe, solution rebuild | CTDirect |
| Expressions to every representation | CTParser |
build_examodel |
removed |
Open technical point: one calling convention
To write defect(j) once, the callable CTDirect gets from dynamics(ocp, modeler) must have the same calling convention for every modeler. Proposal: CTParser generates an out-of-place form returning a tuple or an SVector (scalar in 1-D, DIM1): no allocation, ForwardDiff-compatible, traceable, so one function could serve both ADNLP and Exa. For a hand-written model with only f!, CTModels adapts it. Benchmarks on midpoint and trapeze against the current in-place code must show no loss.
Work items
- Prototype: one transcription for ADNLP and Exa (trace-based Exa, out-of-place ADNLP) #674 Prototype: trace-based Exa, out-of-place ADNLP (gates the APIs below)
- Components with several representations, selected by the modeler (required_form) CTModels.jl#516 CTModels: multi-representation components, forms,
required_form, selection; deprecatebuild_examodel - Generic NLP-building API dispatched on the modeler, and required_form for ADNLP and Exa CTSolvers.jl#319 CTSolvers:
required_formfor ADNLP and Exa, generic NLP-building API with handles - Generate every representation of each component; remove the discretization from the :exa backend CTParser.jl#415 CTParser: generate every representation; remove the discretization from
:exa - CTDirect: write the recipe against both APIs ([Roadmap] Consolidate CTDirect following ct-conventions #610 §3 and §5)
- JuMP modeler (after the generic NLP-building API) CTSolvers.jl#320 CTSolvers: JuMP modeler (later)
Order: #674 → control-toolbox/CTModels.jl#516 → control-toolbox/CTSolvers.jl#319 and control-toolbox/CTParser.jl#415 → CTDirect recipe → control-toolbox/CTSolvers.jl#320.
Related
- CTDirect: roadmap [Roadmap] Consolidate CTDirect following ct-conventions #610, discussion Roadmap #609 (§3 emission axis, §5 Exa repatriation, §6.2 true sparsity),
:examodeler with an explicittime_grid: crash with the defaultgrid_size, silently uniform otherwise #638 (:exawithtime_grid) - CTParser: [Dev] Exa refactor CTParser.jl#209 (Exa refactor), [Dev] Add non uniform grid for exa CTParser.jl#65 (non-uniform grid for Exa), [Dev] Exa retrieve multipliers for boundary and path constraints CTParser.jl#134 and Keep the :exa constraint handles by label and return the path/boundary multipliers by label CTParser.jl#414 (Exa multipliers),
:exarejects the scheme aliases that:adnlpaccepts (:euler_forward,:euler_explicit,:euler_backward) CTParser.jl#355 (scheme aliases), Run Aqua onCTParserExaModels— 112 documented pirated methods, 82 ambiguities CTParser.jl#365 (CTParserExaModelspiracy) - OptimalControlProblems: feat: JuMP models for Euler, implicit Euler and midpoint, as Butcher tableaux (#369) OptimalControlProblems.jl#383 (JuMP models as Butcher tableaux; they stay an independent oracle)
- Roadmap: https://github.com/orgs/control-toolbox/discussions/68
Activity
ocots commentedon Nov 6, 2026
Written by Claude (Claude Code) at the request of ocots.
The JuMP models of OptimalControlProblems are now a small prototype of this design, on the JuMP side. Notes for #674, control-toolbox/CTModels.jl#516 and control-toolbox/CTSolvers.jl#319.
What exists there.
ProblemData(ext/JuMPModels/schemes/data.jl) describes a problem by continuous functions(x, u, v, t): dynamics, Lagrange, Mayer, path and boundary constraints, plus bounds and start values. JuMP traces them by operator overloading. This is the "traceable" form of Components with several representations, selected by the modeler (required_form) CTModels.jl#516.schemes/build.jlis the transcription, written once for 22 problems and 4 schemes (feat: JuMP models for Euler, implicit Euler and midpoint, as Butcher tableaux (#369) OptimalControlProblems.jl#383). Schemes are Butcher tableaux, and FSAL selects the evaluation of the dynamics by dispatch (Roadmap #609 §3.3).Points for the design.
Calling convention. The functions of
ProblemDatareturn aVector. That is fine for JuMP, but it allocates; [Tracking] Modeler-agnostic transcription: one recipe in CTDirect, any NLP modeler #673 proposes a tuple or anSVector, and Prototype: one transcription for ADNLP and Exa (trace-based Exa, out-of-place ADNLP) #674 should settle it.Constraint order is not portable. MOI passes the constraints to the solver grouped by type (function × set). The order of creation is kept only within a type. On 21 problems, grouping the constraints by kind changed the order seen by the solver on one problem only; the order of the variables is what changes. The NLP-building API should promise an order of the variables, not of the constraints across families.
Handles.
buildstoresmodel[:constraints], the constraints in CTDirect's order whatever the layout, and the dynamics constraints by component (model[:∂x₁]) for the costate. This is the role of the handles of Generic NLP-building API dispatched on the modeler, and required_form for ADNLP and Exa CTSolvers.jl#319 for reading the multipliers.Scaled models. (Updated.)
space_shuttleused to write the JuMP dynamics on the true altitude and velocity, throughProblemData.state_expressions, so its rows were scaled by 1e5 and 1e4 with respect to CTDirect's. It now writes them on the scaled state, as the OptimalControl model does, andstate_expressionsis gone ([Tests] NLP equivalence tests between JuMP, ADNLP and Exa OptimalControlProblems.jl#371). Nothing to represent here, then.Oracle. The hand-written JuMP models stay an independent oracle until JuMP modeler (after the generic NLP-building API) CTSolvers.jl#320 exists. Afterwards, comparing them with JuMP-through-CTDirect will test CTDirect's transcription itself.
Initial point of the Exa model. CTParser declares
N+1controls for every scheme, and it initialises the control at the last node with the value of nodeN-1, for the trapezoidal rule too (chain:-2, 0, 2, 4, 4against CTDirect's-2, 0, 2, 4, 6). The initial point therefore differs from CTDirect's, even when the equations are the same. The new transcription should take the initial point from the scheme, as CTDirect does. [Tests] NLP equivalence tests between JuMP, ADNLP and Exa OptimalControlProblems.jl#371 reports this by computation, without failing.OptimalControlProblems tracks its side in a theme issue, one source of truth per problem, with this issue as the target: