migrate AutoML forecasting model loading to skops

[HarshaVardhanBabu (babu-namburi)]

Status

Ready for team review.

The producer-side integration is available in AzureMlCli PR 2234915. It directly serializes fitted forecasting models to outputs/mlflow-model/model.skops while retaining existing pickle/PyTorch artifacts for older consumers.

Summary

• replace pickle-based forecasting model loading with fail-closed .skops loading
• reject malformed archives, disguised pickle payloads, symlinked artifacts, unsupported extensions, and ambiguous multiple-model inputs
• allow one safe .skops artifact to coexist with legacy artifacts for cross-version compatibility
• install and validate skops==0.14.0 against the existing NumPy 1.23.5 AutoML stack
• enable feature-gated safe forecasting export in the AutoML GPU environment
• retain the latest environment vulnerability-remediation updates from main

Backward compatibility

• existing AutoML runtime behavior is unchanged unless AUTOML_ENABLE_SAFE_FORECASTING_MODEL_EXPORT=true
• opted-in training environments publish both .skops and unchanged legacy artifacts
• older component versions continue consuming legacy artifacts
• this hardened component consumes .skops and intentionally rejects legacy-only pickle/PyTorch models

Existing models must be retrained or re-exported before using this component version.

Security improvements

• pickle files are not discovered or deserialized
• renamed pickle payloads are rejected without executing payload code
• artifact-provided trusted-type lists are never accepted
• unsupported PyTorch/TCN full-object deserialization is rejected
• malformed paths, symlinks, unsafe-only artifacts, and multiple .skops artifacts fail closed

Dependency resolution

The current AutoML runtime requires NumPy <=1.23.5, while skops 0.14 declares NumPy >=1.25. The environment installs skops==0.14.0 with --no-deps and runs a build-time compatibility test without changing the supported AutoML NumPy stack. This override remains outside skops' declared support matrix and should be removed when AzureML supports NumPy 1.25 or later.

Validation

• 10 focused model-loader tests
• realistic sklearn regression pipeline round-trip
• malicious disguised-pickle rejection
• invalid archive, path, symlink, ambiguous artifact, and unsupported extension coverage
• build-time skops/NumPy compatibility test
• repository documentation-style and diff checks
• conflict resolution against current main

Remaining merge requirements

• derive and security-review the exact trusted-type allowlist required by a genuine ForecastingPipelineWrapper
• validate producer-to-consumer round-trip and forecast equivalence in the internal AutoML environment
• keep TCN unsupported until a fixed architecture can be reconstructed from a weights-only state dictionary

Broad azureml.* trust and artifact-supplied trust lists are explicitly out of scope because they would weaken the deserialization boundary.

[Copilot]

Pull request overview

Hardens AutoML forecasting inference by replacing pickle/PyTorch deserialization with fail-closed skops loading and updating the supporting GPU environment.

Changes:

• Adds secure .skops discovery and loading.
• Installs and validates skops against the existing NumPy stack.
• Updates component metadata and loader tests.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
environments/automl-gpu/environment.yaml	Includes the compatibility test in the build context.
environments/automl-gpu/context/skops_compatibility_test.py	Smoke-tests skops serialization compatibility.
environments/automl-gpu/context/Dockerfile	Installs skops and enables safe forecasting export.
environments/automl-gpu/context/conda_dependencies.yaml	Adds the PrettyTable dependency.
automl-single-model-inference/tests/test_model_loader.py	Tests safe model discovery and loading.
automl-single-model-inference/src/model_loader.py	Implements fail-closed skops loading.
automl-single-model-inference/src/inference.py	Delegates model handling to the secure loader.
automl-single-model-inference/components/inference/spec.yaml	Versions and documents the hardened component.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

This pull request has been marked as stale because it has been inactive for 14 days.