Execution runtime
NVIDIA NeMo Relay
NVIDIA NeMo Relay is an agent execution-runtime layer for observing and controlling LLM, tool, subagent and framework work through scopes, middleware, plugins and lifecycle events. Stable 0.8.3 integrates with several agent-harness surfaces, but it is not UHP and is not a HarnessRouter backend.
Status first
Section titled “Status first”NVIDIA published NeMo Relay 0.8.3 on 2 September 2026. The project describes Relay as a shared runtime for scopes, policy, plugins and lifecycle events around agent work. It can sit beside a coding-agent CLI, run inside an application or framework integration, or be embedded directly by a host that owns the real model and tool callbacks.
Stable 0.8.3 is the maturity boundary used for the support claims on this page. Current upstream main has already moved beyond that release: pull request #977, merged as 30b684db on 2 September, adds a post-release Pi routing path for Relay-launched sessions. That source-level work is described separately below and is not attributed to stable 0.8.3.
A fresh integration signal also landed in Hermes on 3 September 2026. Hermes PR #96633, merge 48c0c3a8, upgrades its existing native Relay integration from the 0.7 line to nemo-relay>=0.8.3,<0.9, adapting Relay’s typed tool-result and trace-propagation contracts. Hermes still owns its session, model, tool and streaming callbacks; Relay owns explicitly selected plugin activation.
What Relay controls
Section titled “What Relay controls”Relay’s runtime model is built around execution activity rather than UHP task resources:
- Scopes model parent/child relationships for runs, turns, LLM calls, tool calls and subagents and provide lifecycle cleanup and request isolation.
- Lifecycle events and subscribers expose the activity observed by Relay to exporters and other consumers.
- Marks represent point-in-time events such as compaction or skill loading without inventing a start/end pair.
- Managed LLM and tool calls let Relay apply middleware around callbacks it is actually allowed to invoke.
- Middleware can block, sanitize, transform, route, retry or replace managed execution.
- Plugins package reusable policy, observability, guardrail and adaptive behavior.
The control strength depends on the integration boundary. If the real LLM or tool call is routed through Relay, middleware can participate in managed execution. If a framework exposes only before/after hooks, Relay can observe that lifecycle but does not gain authority it does not actually own. If provider traffic is routed through the Relay gateway, Relay becomes part of that production request path.
Stable 0.8.3 integration surface
Section titled “Stable 0.8.3 integration surface”The tagged 0.8.3 support matrix distinguishes several integration styles.
| Integration path | Stable 0.8.3 evidence | Important boundary |
|---|---|---|
| Claude Code CLI | Minimum 2.1.121; persistent install, transparent run, lifecycle hooks, local gateway routing and pre-tool security | Claude desktop/web/application sessions are outside this claim unless they expose the same local controls. |
| Codex CLI | Minimum 0.143.0; persistent install, transparent run, ten supported plugin hooks, local gateway routing and pre-tool security | Cloud/remote work that bypasses the local machine has partial or no LLM capture; encrypted Codex multi-agent v2 payloads cannot be reliably linked. |
| Hermes Agent | Native in-process integration; no separate Relay CLI or observability plugin is required | Hermes owns the actual session/model/tool/streaming loop; Relay supplies the selected runtime/plugin layer. |
| LangChain / LangGraph / Deep Agents | Maintained integrations with observability, security and optimization support | Framework integration is not a new external agent protocol. |
| OpenClaw | Maintained integration with observability and partial security support | Public hooks do not expose the managed-execution rewrite authority required for Relay optimization. |
Relay also supports native and local gRPC worker plugins. The project explicitly warns that the worker process boundary isolates crashes and dependencies but is not a security sandbox.
Trajectory and telemetry boundary
Section titled “Trajectory and telemetry boundary”Relay records its canonical lifecycle stream as Agent Trajectory Observability Format (ATOF) events. Exporters can project completed runs into Agent Trajectory Interchange Format (ATIF) trajectories or translate runtime activity to OpenTelemetry/OpenInference output.
That makes Relay relevant to the guide’s execution-evidence coverage, but the layers are different:
- ATOF is Relay’s runtime event model.
- ATIF is a portable completed-run trajectory format.
- OpenTelemetry/OpenInference are observability projections.
- UHP is the live external execution contract for selecting a harness and running/continuing/cancelling work through server task/session semantics.
An ATIF trajectory exported by Relay is therefore evidence about an agent run; it is not a UHP task object or a UHP conformance result.
What changed in the 0.8 line
Section titled “What changed in the 0.8 line”Release 0.8.0 established several contracts that matter when Relay is embedded into a harness:
- tool callbacks and execution intercepts use the canonical
ToolExecutionResultcontract across primary bindings and plugin/worker surfaces; - a provider- or harness-supplied tool-call ID can be preserved on matching lifecycle events;
- managed LLM requests carry a runtime-owned W3C
traceparentso downstream provider work can remain attached to the Relay trace; - Deep Agents can represent orchestrator/subagent activity as nested semantic Agent scopes;
- conditional middleware and event-metadata injection give policy/observability code explicit runtime hooks without changing the agent framework’s own session model.
Patches 0.8.1 and 0.8.2 harden plugin/configuration boundaries, gateway lifecycle, managed integration refresh and observability behavior. 0.8.3 then fixes release-version identity between the Python package and compiled native extension and adds managed coding-agent reliability fixes, including recovery from stale managed plugin state and safe restart of an unresponsive gateway only when Relay can verify that it owns the recorded process.
Hermes 0.8.3 adoption: ownership stays explicit
Section titled “Hermes 0.8.3 adoption: ownership stays explicit”Hermes PR #96633 is useful architectural evidence because it shows a harness embedding Relay without giving Relay ownership of the harness session itself.
The merged integration updates Hermes to Relay 0.8.3, returns ToolExecutionResult from managed callbacks, forwards Hermes’ provider tool-call ID into Relay correlation, and propagates Relay’s traceparent through SDK/custom request paths that already accept extra headers. Strict native transports do not receive an unsupported SDK-only keyword.
Upstream reports 270 focused Relay tests when isolated by file, a native Anthropic streaming end-to-end test, and a deterministic Hermes CLI smoke covering two managed model calls, streaming, a real file-tool call, shared metrics and export validation. Those are upstream verification results, not a UHP conformance statement.
The reusable boundary is runtime interposition without session-ownership confusion: an execution runtime may observe or govern selected call boundaries while the harness remains authoritative for session state, model/tool invocation semantics and streaming lifecycle.
Post-release current main: Pi named-upstream routing
Section titled “Post-release current main: Pi named-upstream routing”NeMo Relay current main at this verification cutoff is 30b684db. PR #977 extends Relay-launched Pi sessions so the selected provider can name its original upstream endpoint even when that destination is not statically configured on the gateway.
The trust boundary is narrow:
- the client-supplied destination header is accepted only from a Relay-launched process tree authenticated with its per-invocation proxy credential;
- Relay validates the destination and strips the routing header before forwarding;
- a client-named destination receives only credentials supplied by that caller — Relay does not inject configured or environment provider credentials into an endpoint selected by the client;
- standalone gateways keep static routing because they do not issue the invocation credential.
This is meaningful post-release runtime/security behavior, but it is not part of tagged 0.8.3 and does not establish a Pi UHP implementation or a HarnessRouter integration.
Relay vs UHP
Section titled “Relay vs UHP”| Boundary | NVIDIA NeMo Relay | UHP / HarnessRouter |
|---|---|---|
| Primary role | Execution-runtime interposition, policy, lifecycle events and observability | Client↔server contract for selecting and running a complete harness |
| Main objects | Scopes, lifecycle events, marks, managed calls, middleware/plugins | Harness/model catalogues, tasks, responses, sessions and server capabilities |
| Typical placement | In-process SDK, framework integration, coding-agent sidecar/hooks, or provider gateway | External HTTP/SSE boundary in front of a selected harness |
| Control authority | Depends on whether Relay owns the managed call, only observes hooks, or routes provider traffic | Defined by the UHP server/API contract and the selected harness adapter |
| Evidence output | ATOF events; ATIF/OpenTelemetry/OpenInference projections | UHP response/task/session behavior and conformance checks |
| Adoption evidence | Native Hermes integration; stable local CLI support for Codex/Claude Code; maintained framework integrations | HarnessRouter separately exposes eight released harness backends through UHP |
| UHP relationship | No standardized binding or native UHP adoption verified | NeMo Relay is not a released HarnessRouter backend |
A system could theoretically place Relay inside a harness that is itself exposed through UHP, but those layers would remain separate. The UHP client should not be assumed to see Relay scopes, plugins, trace state or middleware decisions unless an implementation explicitly maps them into a documented UHP-visible surface.
Why it matters
Section titled “Why it matters”NeMo Relay is a useful missing layer in the agent-harness architecture map. It is neither just an observability exporter nor another client↔agent protocol: when integrated at the managed-call or gateway boundary, it can participate in policy and execution control while preserving the host harness as the owner of the agent session.
That distinction matters for interoperability analysis. A UHP adapter can standardize access to a complete harness; Relay can independently instrument or govern parts of what happens inside that harness. Treating those as the same layer would overstate both UHP adoption and Relay’s authority.
Related pages
Section titled “Related pages”Read harness composition for nested and cross-harness execution topologies, UHP vs ATIF for portable trajectory evidence, UHP and Hermes, UHP and Codex, UHP and Claude Code, and UHP and Pi for the upstream harness boundaries that Relay can touch through separate integrations.