fix(#816): Expression shape/len/iter protocol + concatenate/stack

[jkitchin]

Summary

Closes #816.

Two behaviors on modeling Expression objects.

Bug (the urgent part). Variable supported __getitem__ but not __len__/__iter__, and out-of-range indexing returned an expression instead of raising. Python then fell back to the legacy sequence-iteration protocol (x[0], x[1], … until IndexError), which never terminated — list(x) / sum(x) / np.array(x) wedged the process on any accidental iteration.

• Expression gains __len__ (leading-axis length) and __iter__ (yields expr[0], expr[1], …); both raise TypeError for scalar or shape-unknown expressions instead of hanging. __iter__ is what actually kills the hang (Python no longer uses the sequence protocol).
• Out-of-range integer indexing via the [] operator on a shaped (ndim ≥ 1) expression now raises IndexError (numpy semantics) — for a plain int or a full-arity all-int tuple that is out of range (x[99], X[i, j]). The guard lives on Expression.__getitem__; direct IndexExpression(base, idx) construction stays conservative (never raises), because the expression-graph internals (GAMS import, NL export resolver, signomial/sign-domain matching) build out-of-range / non-integer / too-many-index nodes as lazy forms resolved downstream (test_sign_domain_checks: IndexExpression(v, 10) "must be conservative, not raise"). Scalar-base indexing (x[0] on shape ()), slices, wrong-arity tuples, and non-integer keys stay lenient.

Enhancement. .shape now propagates through indexing/slicing, element-wise BinaryOp/UnaryOp, and element-wise FunctionCall (sqrt, exp, …), exposed as a read-only Expression.shape property that raises AttributeError when the shape is not statically known — so getattr(expr, "shape", default) / hasattr keep their prior behaviour for shape-unknown nodes (reductions, matmul, custom). dm.concatenate / dm.stack assemble a vector of scalar expressions from pieces (the method-of-lines "boundary, interior, boundary" pattern) as a numpy object ndarray — no new DAG node type, each element is an ordinary scalar expression the solver already handles.

The issue's shared momentum_residual now reuses verbatim across np.sqrt and dm.sqrt (a regression test pins this). Request #3 (np.sqrt(expr) via __array_ufunc__) is intentionally not in scope: it would require replacing the load-bearing __array_ufunc__ = None (which defers A @ x to __rmatmul__) with a handler that re-owns all ufunc dispatch including matmul — a correctness-sensitive marshaling change that belongs in its own bound-neutral issue.

Second commit (chore(types), unrelated to #816, requested in review) clears 31 pre-existing mypy errors in the solver stack (TypeAlias on VectorLike; -np.inf → float("-inf") for -> float returns; int()/bool() wraps). mypy python/discopt/ now reports 0 errors.

Correctness

No solver-math change: this is modeling-DSL metadata and Python-side iteration. The out-of-bounds guard only makes a provably invalid user-facing index (x[99]) fail loudly instead of building a silently-invalid node; it never rejects a valid model, and internal lazy IndexExpression construction is untouched (stays conservative). .shape inference is exact (numpy broadcast rules / the probe mirrors the compiler's base[index]) and never raises, so the M8 build-time shape guard fires only on genuinely incompatible shapes. getattr(con.body, "shape", ()) and similar consumers are unaffected (AttributeError preserves the default). End-to-end solves with shaped/sliced/sqrt expressions verified (objective correct).

• Cannot produce a false certificate (N/A — no solver-math change)
• No validation, fallback, or safety guard was weakened to make a check pass

Tests run (state the result — numbers, not adjectives)

• pytest -m smoke → 817 passed, 15 skipped, 2 xpassed
• Full Python fast CI lane green on ef889ae (an earlier revision failed 7 tests — GAMS/export/signomial/sign-domain — because the guard first sat in IndexExpression.__init__ and broke internal lazy construction; moving it to __getitem__ fixed all 7)
• pytest -m slow python/tests/test_adversarial_recent_fixes.py → 9 passed, 1 flaky (test_large_dense_jacobian_no_crash trips its wall-clock deadline under load; passes standalone in 27s, scalar-only, unrelated to this change)
• cargo test -p discopt-core → N/A (no Rust touched)
• ruff check / ruff format --check clean; mypy python/discopt/ → 0 errors

Regression test

python/tests/test_expression_shape_iter_816.py (17 cases): OOB IndexError 1-D/2-D via [], scalar-indexing leniency, len/iter, no-hang builtins (list/sum/np.array), .shape propagation, concatenate/stack, and the issue's shared momentum-residual reused verbatim across numpy and discopt backends. Each fails before the change (hang / silent no-op / AttributeError) and passes after.

• Added a regression test that fails before / passes after

---

Generated by Claude Code