diff --git a/crates/cli/src/server/mod.rs b/crates/cli/src/server/mod.rs index 7172e50b8..502d8c90f 100644 --- a/crates/cli/src/server/mod.rs +++ b/crates/cli/src/server/mod.rs @@ -899,7 +899,7 @@ enum ServerPluginActivation { Dynamic(PluginActivation), } -const REMOVED_SWITCHYARD_MESSAGE: &str = "the built-in Switchyard service integration was removed in NeMo Relay >=0.8.0; remove this `[[components]]` entry and follow the NeMo Relay Switchyard migration guide for the Switchyard-owned dynamic plugin: https://docs.nvidia.com/nemo/relay/reference/migration-guides#migrate-to-the-switchyard-owned-dynamic-plugin"; +const REMOVED_SWITCHYARD_MESSAGE: &str = "the built-in Switchyard service integration was removed in NeMo Relay >=0.8.0; remove this `[[components]]` entry and refer to the NeMo Relay migration guides for current Switchyard migration information: https://docs.nvidia.com/nemo/relay/reference/migration-guides"; impl ServerPluginActivation { fn clear(self) -> Result<(), CliError> { diff --git a/docs/about-nemo-relay/release-notes/index.mdx b/docs/about-nemo-relay/release-notes/index.mdx index cb178f982..45eab73bd 100644 --- a/docs/about-nemo-relay/release-notes/index.mdx +++ b/docs/about-nemo-relay/release-notes/index.mdx @@ -1,7 +1,7 @@ --- title: "Release Notes for NVIDIA NeMo Relay" sidebar-title: "Release Notes" -description: "Review highlights, compatibility updates, fixed known issues, and current known issues for NVIDIA NeMo Relay 0.8." +description: "Review highlights, compatibility updates, fixed known issues, and current known issues for NVIDIA NeMo Relay 0.9." template-library-version: "1.0.0" position: 6 --- @@ -25,271 +25,17 @@ This is the Release Notes template. Document one release version per page and us This page contains the release notes for [NVIDIA NeMo Relay](/about-nemo-relay/overview). -## Release 0.8.3 +## Release 0.9 -Python release wheels now synchronize the selected release version between the -package metadata and the compiled native extension. This prevents consumers -that inspect the native extension version, including native-plugin compatibility -checks, from observing a stale Relay version. +NeMo Relay 0.9 is under development. This page will track its user-visible +changes, compatibility updates, and fixed known issues. -### Managed Coding-Agent Reliability Fixes - -- `nemo-relay install`, `nemo-relay uninstall`, and - `nemo-relay integrations refresh` now recover when an orphaned persisted - plugin-state file remains after an interrupted uninstall or manual cleanup. - A normal install still refuses to overwrite that state without `--force`. -- `nemo-relay install codex --force` now resolves the Relay executable the same - way when it writes and validates generated hooks. This prevents a - self-consistency failure when a different Relay binary appears earlier on - `PATH`, such as during local development. -- Persistent managed gateways that stop answering health checks are now safely - terminated and restarted when their recorded Relay process identity matches. - Relay continues to leave unverified listeners untouched, and operational logs - now classify gateway port conflicts and recovery failures with remediation. - -## Release 0.8.2 - -### Managed Gateway and Coding-Agent Lifecycle - -- The CLI now provides explicit `nemo-relay gateway start` and - `nemo-relay gateway stop` commands. Stopping verifies the Relay process at - the selected local loopback address before requesting shutdown; Unix callers - can use `--force` when graceful shutdown is not possible. -- Persistent MCP integrations now send a configurable heartbeat. Set - `NEMO_RELAY_PLUGIN_HEARTBEAT_INTERVAL_SECS` to a positive value shorter than - the gateway idle timeout when the default three-second interval is unsuitable. -- After upgrading Relay, run `nemo-relay integrations refresh` to replace all - Relay-managed Codex and Claude Code integrations and their authenticated - sidecars. A newly started managed MCP client can also replace a verified - Relay-owned sidecar from a different Relay version; it never replaces a - foreign, invalid, or same-version gateway. -- `nemo-relay uninstall --force` can complete Relay-owned coding-agent - cleanup when normal safety checks fail. - -### Observability and Logging Fixes - -- ATIF exports retain the correct request metadata when LLM end events are - interleaved. -- Set `stderr_enabled = false` under `[logging]` to disable the stderr sink - while retaining configured file sinks. See [Operational - Logging](/reference/operational-logging). - -## Release 0.8.1 - -### Plugin and Configuration Improvements - -- The interactive plugin editor now handles structured maps, nested sections, - lists, and discriminator-selected configuration variants. -- Built-in PII redaction supports `target_path_globs`. A complete `*` JSON - Pointer segment matches one object key or array index; existing - `target_paths` remain exact selectors. See [PII Redaction - Configuration](/configure-plugins/pii-redaction/configuration#path-semantics). -- Native and worker plugins can register callback-based conditional middleware - guardrails. The additions preserve the existing native ABI v4 prefix and - `grpc-v1` constant-reason fields, so existing plugins remain compatible. -- Built-in and configured static plugins now activate before dynamic plugins, - allowing dynamic plugin registrations to discover the runtime state required - during startup. - -### Observability and Security - -- OpenTelemetry can promote selected root Scope-start metadata into OTLP - resource attributes with `promote_resource_metadata_prefixes`. Each distinct - effective resource keeps its own exporter pipeline for the subscriber - lifetime. See [Root Resource Metadata - Promotion](/configure-plugins/observability/opentelemetry#root-resource-metadata-promotion). -- Trajectory redaction preserves supported log-severity metadata, so sanitized - semantic logs remain exportable. -- Gateway, permission, dynamic-plugin, observability, and response-cache - boundaries were hardened without changing public APIs, ABIs, configuration - schemas, or event formats. - -## Release 0.8.0 - -NeMo Relay 0.8 strengthens plugin extensibility, managed tool execution, and -observability while introducing source-breaking migration steps for native Rust -plugins, Node.js streaming intercepts, and tool-result callbacks. - -### Highlights - -The following updates expand native Rust plugin capabilities: - -- Typed native Rust middleware can await Tokio timers, I/O, codecs, unary - continuations, and downstream LLM streams on an SDK-owned executor. Native - ABI v4 adds completion-scoped codec operations, pull-based LLM stream - continuations, and `emit_mark_v2` support for typed data schemas and log - severity. Existing raw ABI v2 and v3 binaries remain loadable without a - rebuild for host-table compatibility; rebuild only when the plugin also needs - the changed native API 1 tool-result contract. -- Tool callbacks and execution-intercept continuations now share the canonical - `ToolExecutionResult` contract across Rust, Python, Node.js, Go, the public C - surface, native plugins, and `grpc-v1` workers. The application-owned - `result` can travel with an optional opaque `annotation` for observability. -- ATOF, ATIF, and the `full` and `openinference` OpenTelemetry projections - preserve sanitized tool result annotations without flattening their - application-defined schemas. -- Managed tool execution accepts an optional provider- or harness-supplied - tool-call ID and preserves it on the matching start and end events across - success, error, and cancellation paths. - -### Runtime Controls and Caching - -- Conditional middleware guardrails let applications and plugins temporarily - exclude matching global registrations, such as an exporter during an outage - or an expensive intercept while a dependency is unhealthy. Gates are - fail-open, so a failed control callback does not silently remove runtime - behavior. See [Conditional Middleware - Guardrails](/about-nemo-relay/concepts/conditional-middleware-guardrails). -- Event metadata injectors can enrich events from application code and plugins - across Rust, Python, Node.js, Go, and the C FFI. Selected metadata keys can - also be promoted into OpenTelemetry attributes, making useful application - context available to telemetry backends without changing event schemas. -- The Adaptive response cache can now cache explicitly classified, read-only - tool results. Tool caching is disabled by default because a cache hit skips - the live tool call; configure a stable TTL and tool policy only when that is - safe for the tool's behavior. See [Response - Cache](/configure-plugins/adaptive/response-cache). - -### Provider and Framework Support - -- Built-in codecs now support OCI Generative AI chat payloads and Gemini - `generateContent` payloads, including streaming and normalized lifecycle - data. Use these codecs directly when a framework sends either provider's - request shape. See [Provider - Codecs](/integrate-into-frameworks/provider-codecs). -- Managed LLM requests now carry a runtime-owned W3C `traceparent` header, - allowing downstream provider work to remain connected to the Relay trace. -- The Deep Agents integration now models the orchestrator and in-process - subagents as nested semantic Agent scopes. This makes parent-child activity - easier to understand without turning internal LangGraph nodes into extra - agent spans. - -### OpenTelemetry Logs and Metrics - -- Independent OTLP log and metric pipelines are available alongside trace - export. Observability configuration version 4 derives log and metric - destinations from a trace endpoint when their endpoint lists are omitted. -- Rust, Python, Node.js, and the experimental C FFI expose typed mark - schema and severity options plus metric helpers. Direct subscribers and - native and worker plugin SDKs expose bounded runtime diagnostics snapshots. -- OpenTelemetry keeps valid endpoints active when a peer endpoint is invalid, - retains completed trace context for late events for a configurable period, - and reports signal-specific delivery failures without exposing sensitive - endpoint details. - -### Support Matrix and Compatibility Updates - -The [Support Matrix](/reference/support-matrix) is the canonical reference for -supported platforms and architectures, worker runtimes, coding agents, and -integrations. It also records current limitations, including platform-specific -worker requirements. - -Migration guidance for upgrading from 0.7 to 0.8 is available in the -[Migration Guides](/reference/migration-guides). - -### Breaking Changes - -- Typed native Rust middleware callbacks now return futures and receive owned - arguments. Native subscribers and raw synchronous ABI entry points are - unchanged. Rebuild typed native Rust middleware plugins with - `nemo-relay-plugin` 0.8.0 and declare `compat.relay = ">=0.8.0,<1.0"`; see - the [native Rust migration - guide](/reference/migration-guides#upgrade-native-rust-plugins-to-nemo-relay-08). -- Node.js LLM streaming execution intercepts now receive and return lazy async - iterables. Existing intercepts that collect `next(request)` into an array or - return a scalar or array must use an async generator instead. See the [Node.js - stream-intercept migration - guide](/reference/migration-guides#stream-nodejs-llm-execution-intercepts-lazily). -- Relay-created plugin-component effective names now use a versioned format - that includes a one-based component ordinal, including ordinal `1` for a - singleton component. Effective names are an internal implementation detail: - discover them from the active runtime instead of constructing or persisting - them. -- Managed tool callbacks, execution-intercept continuations, execute-helper - returns, and manual tool end APIs now use `ToolExecutionResult` instead of a - raw JSON result. Applications that need the original business payload must - read its `result` field. Refer to the - [tool result migration guide](/reference/migration-guides#return-canonical-tool-execution-results). -- Relay 0.8 resets native API 1 and the `grpc-v1` tool-result contract to the - canonical `ToolExecutionResult` shape. Rebuild native API 1 and worker - plugins that use tool callbacks, execution intercepts, or manual tool-end - APIs, and declare a `compat.relay` range that excludes Relay versions before - 0.8. Frozen raw ABI v2 and v3 binaries that do not use this changed contract - remain loadable without a rebuild. The native ABI v4 table remains unchanged. - The worker package and RPC method names remain `nemo.relay.worker.v1`, but - the `ToolNext` response and tool-execution outcome field now use structured - protobuf messages. Regenerate custom worker bindings before rebuilding. -- Repository-local `.nemo-relay/config.toml`, `plugins.toml`, and - `.dynamic-plugins.json` files are no longer discovered or activated. Runtime - resolution is explicit-or-user followed by higher-precedence system policy. -- Project setup scopes and the `--project` and `config --reset --scope` - interfaces have been removed. Setup and reset now target XDG user - configuration; `--user`, `--global`, `--config`, and - `--plugin-config-path` remain supported. -- `nemo-relay doctor` reports ignored ancestor project configuration as - warning-only migration diagnostics. Relay does not automatically move, - rewrite, or delete those legacy files. -- Removed Hermes-specific support from the NeMo Relay CLI, including the - `hermes` shortcut, `run --agent hermes`, and Hermes-specific install, - uninstall, doctor, configuration, MCP selection, and `/hooks/hermes` paths. - NeMo Relay is built into Hermes Agent, and Hermes Agent understands NeMo Relay - plugin configurations. No separate observability plugin or Relay CLI setup is - required. -- Removed the experimental, service-backed Switchyard integration, including - the `nemo-relay-switchyard` crate, CLI `switchyard` feature, built-in - `switchyard` component, and service examples. Switchyard 0.3.0 will distribute - and document the Switchyard-owned dynamic plugin. Refer to the - [Switchyard migration guide](/reference/migration-guides#migrate-to-the-switchyard-owned-dynamic-plugin). - -Refer to [Migration Guides](/reference/migration-guides#move-project-configuration-to-a-supported-location) -for destination paths and explicit-file alternatives. - -### Fixed Known Issues in 0.8 - -- Codex image-generation requests now pass through the local Relay gateway to - the configured OpenAI upstream. Persistent Codex installations must refresh - the `nemo-relay-openai` provider configuration; see [Migration - Guides](/reference/migration-guides#refresh-the-installed-codex-provider-configuration). -- Restored a trailing `/` on an OTLP/HTTP trace endpoint as an explicit - root-path destination. A bare HTTP authority still defaults to - `/v1/traces`; when version 4 implicitly derives log or metric destinations - from either form, it uses `/v1/logs` or `/v1/metrics` respectively. Explicit - trace, log, and metric endpoint paths remain unchanged. -- OpenTelemetry activation now isolates invalid endpoints so they do not - disable valid trace, log, or metric destinations. Repeated shutdown is also - safe, and blocked trace flushes no longer delay event delivery to other - subscribers. -- OTLP diagnostics now retain distinct causes, identify the affected signal, - report direct log queue drops and trace or metric export failures, and redact - sensitive endpoint details from trace-export failures. -- Successful force flushes for direct OTLP trace and log subscribers now expose - all queue drops observed so far in runtime diagnostics. The reported count - remains cumulative through later flushes and shutdown. -- Deferred marks can retain their completed trace lineage for a configurable - time instead of becoming orphaned after a fixed number of completed scopes. -- ATIF exports now correlate OpenAI Responses function-call results with the - semantic invocation `call_id`, so the exported tool observation remains - connected to the matching tool call instead of being orphaned by the response - item's separate identifier. - -### Other Improvements - -- Manually observed LLM responses can now receive configured model-pricing - estimates when they include sufficient normalized model and usage data; - provider-reported costs remain authoritative. -- `nemo-relay run --dry-run` now warns when the forwarded command repeats the - explicitly selected coding-agent executable, helping catch a common launch - mistake without changing live-launch behavior. -- Set `NEMO_RELAY_PLUGIN_SNAPSHOT_DIR` to choose the parent directory for - plugin activation snapshots when system temporary paths are unsuitable or - too long. - -## Known Issues in 0.8 +## Known Issues in 0.9 - OTLP collectors can return a successful response while rejecting individual - spans, log records, or metric data points. Relay 0.8 does not report these - partial successes in runtime diagnostics, and flush or shutdown can still - succeed. Monitor collector-side rejection metrics and logs; see + spans, log records, or metric data points. Relay does not report these partial + successes in runtime diagnostics, and flush or shutdown can still succeed. + Monitor collector-side rejection metrics and logs; see [OpenTelemetry](/configure-plugins/observability/opentelemetry#trace-batch-processor-configuration). - Go and the raw C FFI remain experimental and source-first. Generated API pages focus on Rust, Python, and Node.js. @@ -308,13 +54,12 @@ for destination paths and explicit-file alternatives. - OpenClaw has public hook-backed telemetry. Its security and optimization coverage is partial because it does not own a managed execution path. - The built-in `nemo_guardrails` plugin is deprecated and scheduled for removal - in NeMo Relay 0.9. It remains available in 0.8: the remote backend inherits - its configured service's availability, latency, and policy behavior, and the - local backend requires Python 3.11 or later and `nemoguardrails==0.22.0`. A - replacement is not included in 0.8 and will target 0.9 or later. Removal will - include the built-in component kind, the public - `nemo_relay::plugins::nemo_guardrails` Rust module, its CLI editor entry, and - the `guardrails-remote` Cargo feature. + in NeMo Relay 0.9. No replacement is included. Until removal, the remote + backend inherits its configured service's availability, latency, and policy + behavior, and the local backend requires Python 3.11 or later and + `nemoguardrails==0.22.0`. Removal will include the built-in component kind, + the public `nemo_relay::plugins::nemo_guardrails` Rust module, its CLI editor + entry, and the `guardrails-remote` Cargo feature. - The PII redaction plugin currently supports its deterministic local backend; local-model backend configuration is reserved for future work. - Pricing and optimization estimates depend on model names, token data, pricing diff --git a/docs/reference/migration-guides.mdx b/docs/reference/migration-guides.mdx index df0062c9e..be52a3865 100644 --- a/docs/reference/migration-guides.mdx +++ b/docs/reference/migration-guides.mdx @@ -1,316 +1,17 @@ --- title: "Migration Guides" -description: "Plan an upgrade from NeMo Relay 0.7 to 0.8." +description: "Plan an upgrade to NeMo Relay 0.9." position: 6 --- {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */} -Use this page to plan an upgrade from NeMo Relay 0.7 to 0.8. It groups the -actions from the release notes by the surface you operate. If you skip one or -more releases, review the migration guides and release notes for every -intervening release in sequence. +Use this page to plan an upgrade to NeMo Relay 0.9. It will collect required +upgrade actions as they are identified during the 0.9 development cycle. -## Upgrade to NeMo Relay 0.8 +## Upgrade to NeMo Relay 0.9 -### Refresh Relay-Managed Coding-Agent Integrations - -After upgrading to 0.8.2 or later, refresh every Relay-managed Codex and Claude -Code integration before starting a new session: - -```bash -nemo-relay integrations refresh -``` - -The command retires old MCP generations and replaces Relay-owned sidecars with -the current Relay binary. It does not modify manually configured MCP servers. -For a focused repair of one managed Codex installation, use -`nemo-relay install codex --force`. This refresh also replaces the older -uninstall-and-install procedure for the persistent Codex provider configuration -needed by the initial 0.8 release. - -### Update Exhaustive Cache Configuration Literals - -Rust code that constructs `ResponseCacheConfig` with an exhaustive struct -literal must add `tools: None`. Prefer `..ResponseCacheConfig::default()` when -the literal should remain compatible with new optional cache surfaces. - -### Stream Node.js LLM Execution Intercepts Lazily - - -NeMo Relay 0.8 changes Node.js LLM streaming execution intercepts from buffered -JSON values to lazy async iterables. `next(request)` now resolves to an -`AsyncIterable`, and the intercept callback must return an async iterable -or a promise of one. Scalar and array return values are rejected. - - -This change applies to global, scope-local, and plugin-owned registrations made -with `registerLlmStreamExecutionIntercept`, -`scopeRegisterLlmStreamExecutionIntercept`, and -`PluginContext.registerLlmStreamExecutionIntercept`. - -Replace array transforms such as this pre-0.8 callback: - -```js -async (request, next) => - (await next(request)).map((chunk) => ({ ...chunk, intercepted: true })); -``` - -Use an async generator to transform downstream chunks incrementally: - -```js -async function* transformStream(request, next) { - for await (const chunk of await next(request)) { - yield { ...chunk, intercepted: true }; - } -} -``` - -Return `next(request)` directly when the intercept only forwards the stream. To -short-circuit the downstream chain, return an async generator that yields the -replacement chunks. Do not collect the iterable into an array unless the -application intentionally needs to buffer the complete response. - -### Move Hermes Agent to Its Native Relay Integration - -NeMo Relay 0.8 removes Hermes Agent from the Relay CLI. The `nemo-relay hermes` -shortcut, `nemo-relay run --agent hermes`, Hermes install, uninstall, doctor, -configuration, MCP selection, hook forwarding, and `/hooks/hermes` endpoint are -no longer supported. Relay configuration under `[agents.hermes]` is also -unsupported. - -Remove the `[agents.hermes]` section from every Relay configuration file before -upgrading. If the section remains, Relay rejects the configuration and prevents -configuration-resolving commands, including Claude Code and Codex runs, from -starting. - -NeMo Relay is built into Hermes Agent. Do not install Relay separately or -enable an observability plugin. Hermes Agent understands NeMo Relay plugin -configurations. - -### Migrate to the Switchyard-Owned Dynamic Plugin - -Support for the Switchyard integration now shifts to the -[Switchyard project](https://github.com/NVIDIA-NeMo/Switchyard). Switchyard -0.3.0 will provide the dynamic plugin, release artifacts, and configuration -documentation. - -NeMo Relay >=0.8.0 no longer ships the experimental `nemo-relay-switchyard` -crate, the CLI `switchyard` feature, or the service-backed `switchyard` -component. Configurations that contain `[[components]]` entries with -`kind = "switchyard"` are rejected with migration guidance. - -To migrate an existing configuration: - -1. Remove the CLI `switchyard` Cargo feature and any dependency on - `nemo-relay-switchyard`. -2. Remove the legacy `[[components]]` entry with `kind = "switchyard"`. -3. Remove Decision API settings and ATOF HTTP sinks used only by the former - service integration. Keep any ATOF sink that another Relay consumer uses. -4. Install and configure the Switchyard-owned dynamic plugin by following the - documentation published with Switchyard 0.3.0. - -Do not copy service-era component fields into a dynamic-plugin entry. The -Switchyard release documentation is the source of truth for supported -algorithms, target configuration, and migration details. For Relay's manifest, -trust, policy, and activation model, refer to -[Configure Discoverable Plugins](/configure-plugins/discoverable-plugins). - -### Return Canonical Tool Execution Results - - -NeMo Relay 0.8 replaces raw managed tool results with -`ToolExecutionResult`. Update managed tool callbacks, execution-intercept -continuations and outcomes, execute-helper consumers, and manual tool end calls -as one cutover. - - -The canonical shape has a required application-owned `result` and an optional -opaque JSON `annotation`. Relay transports the annotation alongside the result -without interpreting it: - -```json -{ - "result": {"hits": 2}, - "annotation": {"source": "cache"} -} -``` - -Update Python callbacks and read `.result` where the application needs the -business payload: - -```python -async def search(args): - return nemo_relay.ToolExecutionResult( - {"hits": 2}, - {"source": "cache"}, - ) - -execution_result = await nemo_relay.tools.execute("search", {"query": "weather"}, search) -hits = execution_result.result["hits"] -``` - -Update Node.js callbacks to return the canonical object: - -```js -const executionResult = await toolCallExecute( - "search", - { query: "weather" }, - async () => ({ - result: { hits: 2 }, - annotation: { source: "cache" }, - }), -); -const hits = executionResult.result.hits; -``` - -Update Rust callbacks to return `ToolExecutionResult`: - -```rust -use nemo_relay::api::tool::ToolExecutionResult; - -.func(Arc::new(|_args| { - Box::pin(async { - Ok(ToolExecutionResult::annotated( - json!({"hits": 2}), - json!({"source": "cache"}), - )) - }) -})) -``` - -An execution intercept's `next(args)` now returns `ToolExecutionResult`. -Forwarding intercepts must preserve both fields. Rust can use -`ToolExecutionInterceptOutcome::from(result)`. Python, Node.js, Go, and public -C callbacks return their binding-specific canonical outcome with `result` and -`annotation`. Legacy raw callback and intercept returns are rejected. - -Relay 0.8 establishes this result contract as the native API 1 and `grpc-v1` -baseline. Set `compat.native_api = "1"` or `compat.worker_protocol = "grpc-v1"`, -rebuild every dynamic plugin, and declare a `compat.relay` range that excludes -Relay versions before 0.8. Use `>=0.8.0,<1.0` unless you deliberately need a -narrower or open-ended 0.8-or-newer range. The manifest is the plugin author's -compatibility assertion, not proof that an artifact was rebuilt. - -The native ABI remains v4. Workers continue to use `grpc-v1` and the -`nemo.relay.worker.v1` protobuf package, but this baseline changes the -`ToolNext` response from `JsonResult` to `ToolExecutionResultResponse` and -changes `ToolExecutionInterceptResult.outcome` from `JsonEnvelope` to -`ToolExecutionInterceptOutcome`. Regenerate worker protobuf bindings during -the required rebuild. Future incompatible contract changes must increment the -corresponding native API or worker protocol version. - -Manual tool end helpers also require the canonical wrapper. For example, pass -`nemo_relay.ToolExecutionResult(payload)` to Python `tools.call_end`, -`{ result: payload }` to Node.js `toolCallEnd`, and -`ToolExecutionResult::new(payload)` to the Rust -`ToolCallEndParams.execution_result` builder field. - -Tool sanitize-response guardrails continue to receive only the business -`result`. A successful tool end event stores a non-null annotation under -`category_profile.tool_result_annotation`, after which a scope-end event -sanitizer can replace or remove it. Refer to -[Tool Execution Intercept Outcomes](/reference/tool-execution-intercept-outcomes) -for the complete continuation, lifecycle, exporter, and binding contract. - -### Upgrade Native Rust Plugins to NeMo Relay 0.8 - - -NeMo Relay 0.8 is not source-compatible with typed native Rust plugins built -against NeMo Relay 0.7. Update every typed guardrail, sanitizer, and intercept -callback to the asynchronous contract, rebuild the plugin with -`nemo-relay-plugin` 0.8.0, and update its Relay compatibility range before -deployment. - - -NeMo Relay 0.8 changes every typed native Rust guardrail, sanitizer, and -intercept callback to return a future. This is a source-breaking SDK change. -Subscribers, validation, registration, scope and mark helpers, raw synchronous -ABI callbacks, Go, and worker subscriber contracts remain synchronous. - -Use owned callback arguments and return `Result` from an async block: - -```rust -context.register_tool_request_intercept("policy", 10, false, |_name, args| { - async move { - // Await policy I/O here. - Ok(args) - } -})?; - -context.register_tool_execution_intercept("policy", 10, |_name, args, next| { - async move { - let result = next.call(args).await?; - Ok(ToolExecutionInterceptOutcome::from(result)) - } -})?; -``` - -Event sanitizers now receive `Arc`. `ToolNext`, `LlmNext`, and -`LlmStreamNext` are owned, cloneable values with async `call` methods. Stream -interceptors return a future containing `LlmJsonAsyncStream`. LLM sanitizer -codec facades remain available before and after an await for the lifetime of -the middleware completion. - -The SDK owns one multi-thread Tokio runtime for each configured plugin component, -starts it lazily, and uses two workers by default. Override -`NativePlugin::executor_config` to select a different nonzero worker count. -Use async I/O or `spawn_blocking`; do not block executor workers. Root -middleware futures preserve Relay scope context across awaits and worker-thread -migration. Child tasks created with `tokio::spawn` do not inherit that context -automatically. - -Set new typed async native plugin manifests to: - -```toml -[compat] -relay = ">=0.8.0,<1.0" -native_api = "1" -``` - -Relay 0.8's current native ABI is v4, with fallback to frozen v3 and v2 tables -for old binaries. ABI v4 includes typed asynchronous middleware operations such -as completion-scoped codecs and pull-based downstream LLM streams, then appends -`emit_mark_v2` for typed data-schema and log-severity options. This does not -change the manifest `native_api` value. `native_api = "1"` uses the canonical -`ToolExecutionResult` contract without changing the ABI v4 table layout. Rebuild -all native plugin artifacts; support for frozen table layouts does not preserve -the pre-0.8 JSON semantics. - -### Move Project Configuration to a Supported Location - -NeMo Relay 0.8 no longer discovers repository-local configuration. Files named -`.nemo-relay/config.toml`, `.nemo-relay/plugins.toml`, and -`.nemo-relay/.dynamic-plugins.json` do not affect normal commands, plugin -activation, or dynamic-plugin lifecycle state. - -Move settings that should apply to your account into these files: - -- `$XDG_CONFIG_HOME/nemo-relay/config.toml` and `plugins.toml` -- `~/.config/nemo-relay/config.toml` and `plugins.toml` when - `XDG_CONFIG_HOME` is not set - -Use `/etc/nemo-relay/config.toml` and `/etc/nemo-relay/plugins.toml` on Unix, -or `%ProgramData%\nemo-relay\config.toml` and -`%ProgramData%\nemo-relay\plugins.toml` on Windows, for system policy. System -configuration has higher precedence than the selected user or explicit -configuration. - -For a deliberately selected file in any location, pass `--config` or -`--plugin-config-path`. A `plugins.toml` beside an explicit `config.toml` is -still selected automatically. Relay does not move, rewrite, or delete legacy -project files. Run `nemo-relay doctor` to find ignored files in the current -directory's ancestors. - -Remove `--project` from config, plugin, and model-pricing commands. The -interactive `nemo-relay config` workflow now writes user configuration only, -and `config --reset` resets that user file. The former setup project/both -choices and `config --reset --scope` option have been removed. `--user` and -`--global` remain available. - -Local output directories such as `.nemo-relay/atof`, `.nemo-relay/atif`, and -logs remain supported. Only configuration discovery and dynamic-plugin -lifecycle state lose project semantics. +No migration actions have been recorded for NeMo Relay 0.9 yet. ## Related Release Information diff --git a/docs/reference/tool-execution-intercept-outcomes.mdx b/docs/reference/tool-execution-intercept-outcomes.mdx index d73aca76b..1a1c58130 100644 --- a/docs/reference/tool-execution-intercept-outcomes.mdx +++ b/docs/reference/tool-execution-intercept-outcomes.mdx @@ -167,8 +167,8 @@ return `{ result: payload }` or `{ result: downstream.result, annotation: downstream.annotation }`. Rebuild native plugins and workers against the same NeMo Relay release that -hosts them. For binding-specific examples, refer to the -[Migration Guides](/reference/migration-guides#return-canonical-tool-execution-results). +hosts them. For release-specific upgrade actions, refer to the +[Migration Guides](/reference/migration-guides). LLM request intercepts use a different outcome because they must preserve an annotated request and codec authority. That contract is described in [LLM diff --git a/fern/docs.yml b/fern/docs.yml index a5dc1e8df..a5b33af96 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -22,10 +22,6 @@ redirects: - source: /nemo/relay/about-nemo-relay/concepts/framework-integrations destination: /nemo/relay/integrate-into-frameworks/about -# Migration relocations -- source: /nemo/relay/configure-plugins/switchyard/about - destination: /nemo/relay/reference/migration-guides#migrate-to-the-switchyard-owned-dynamic-plugin - # Release notes - source: /nemo/relay/about-nemo-relay/release-notes/related-topics destination: /nemo/relay/about-nemo-relay/release-notes