IpcPayload::RawJson payloads always fail deserialization on the guest — to_guest_bytes strips the tag the #[capsule] macro requires

[jvsteiner]

SDK bug — IpcPayload::RawJson can never round-trip to a capsule handler

Title

IpcPayload::RawJson payloads always fail deserialization on the guest — to_guest_bytes strips the tag the #[capsule] macro requires

Environment

• astrid-types = "0.7.0"
• astrid-sdk = "0.7.1"
• astrid-sdk-macros = "0.7.1"
• Astrid daemon 0.9.4
• Target: wasm32-unknown-unknown (Component Model), Rust 1.95.0

Reproduced on macOS with a purpose-built driver capsule publishing to another capsule's interceptor over the bus. Observable in the daemon log as:

WARN astrid_capsule::dispatcher: Interceptor: Deny
    capsule_id=<subscriber>
    action=<handler>
    topic=<topic>
    reason=failed to parse arguments: missing field `type` at line 1 column N

The bug in one paragraph

IpcPayload uses #[serde(tag = "type", rename_all = "snake_case")], so on-the-wire every variant carries a discriminator like "type": "raw_json". But IpcPayload::to_guest_bytes() has a hard-coded special case that unwraps RawJson(data) and Custom { data } to the bare inner Value, dropping the tag. The #[capsule] macro, meanwhile, generates guest-side dispatch that does a strict serde_json::from_slice::<UserHandlerArgType>(&payload) — where UserHandlerArgType is often IpcPayload. When the daemon forwards a RawJson message to a handler that takes IpcPayload, the tag has already been stripped and the strict from_slice returns "missing field type". Every such message is silently rejected at dispatch. The Custom variant has the same asymmetry.

Concrete evidence

Producer side — the tag is stripped for RawJson and Custom:

astrid-types-0.7.0/src/ipc.rs, IpcPayload::to_guest_bytes:

pub fn to_guest_bytes(&self) -> Result<Vec<u8>, serde_json::Error> {
    match self {
        Self::Custom { data } | Self::RawJson(data) => serde_json::to_vec(data),
        other => serde_json::to_vec(other),
    }
}

The doc comment even flags it:

[Custom] and [RawJson] payloads return the inner data value directly (no type wrapper). Structured variants return the full tagged serialization.

Consumer side — strict from_slice on the handler's arg type:

astrid-sdk-macros-0.7.1/src/lib.rs, the #[capsule] macro's dispatch generator:

quote! {
    {
        let args = ::serde_json::from_slice(&payload)
            .map_err(|e| format!("failed to parse arguments: {}", e))?;
        instance.#method_name(args).map_err(|e| e.to_string())?
    }
}

args gets typed to whatever the user's handler declares. For a handler with signature fn handle(&self, req: IpcPayload), the generated code is effectively:

let args: IpcPayload = serde_json::from_slice(&payload)
    .map_err(|e| format!("failed to parse arguments: {}", e))?;

which is exactly the strict-parse path that requires the "type" tag.

The fallback exists but the macro doesn't use it:

astrid-types-0.7.0/src/ipc.rs, IpcPayload::from_json_value:

pub fn from_json_value(data: Value) -> Self {
    let is_known = data
        .get("type")
        .and_then(|v| v.as_str())
        .is_some_and(Self::is_known_tag);
    if is_known {
        serde_json::from_value::<Self>(data.clone()).unwrap_or(Self::Custom { data })
    } else {
        Self::Custom { data }
    }
}

from_json_value is exactly the tag-tolerant inverse of to_guest_bytes, but the generated dispatch code doesn't call it. If the macro used from_json_value (going through serde_json::Value first), unwrapped RawJson bytes would round-trip cleanly.

Repro

Two capsules, both on astrid-sdk = "0.7":

Publisher (#[astrid::run] loop):

let envelope = serde_json::json!({
    "type": "raw_json",
    "value": { "foo": "bar" },
});
ipc::publish_json("my.v1.topic", &envelope)?;

Subscriber (#[astrid::interceptor("handle")]):

#[astrid::interceptor("handle")]
pub fn handle(&self, req: IpcPayload) -> Result<(), SysError> {
    log::info(format!("got: {req:?}"));
    Ok(())
}

With Capsule.toml:

[publish]
"my.v1.topic" = { wit = "opaque" }

[subscribe]
"my.v1.topic" = { wit = "opaque", handler = "handle" }

Actual result: daemon logs Interceptor: Deny ... reason=failed to parse arguments: missing field 'type'. Handler never fires.

Expected result: handler fires with req = IpcPayload::RawJson(Value) containing {"foo": "bar"}.

Same failure for IpcPayload::Custom { data } publishes — they hit the same to_guest_bytes branch.

Impact

Any capsule-to-capsule messaging that wants to carry a free-form JSON payload — the exact use case RawJson and Custom are documented to serve — cannot use a handler signature of req: IpcPayload. Practical consequences we hit:

1. astrid-emit uses IpcPayload::RawJson(envelope) for its six-field sage envelope. That specific path works only because sage strips + re-emits before delivery — the intermediate RawJson never has to survive the guest deserialization boundary. Anything not going through sage's mediator can't work the same way.
2. Direct peer-to-peer bus messaging between capsules is currently forced through structured variants (ToolExecuteRequest, LlmRequest, etc.) or requires the handler to declare String / bare Value / serde_json::Value as its arg type and re-parse manually.
3. Any third party building a capsule that wants to receive arbitrary JSON — the natural framing for "a capsule that exposes a bus API" — first has to discover this asymmetry the hard way (silent Interceptor: Deny in a log the caller can't see from their own capsule).

Suggested fix directions

There are at least three ways this could be closed. Ranked by size:

1. Guest dispatch goes through from_json_value. Change astrid-sdk-macros's generated dispatch to deserialize the raw payload bytes into serde_json::Value first, then call IpcPayload::from_json_value(v), then coerce/downcast to UserHandlerArgType. This keeps to_guest_bytes's current wire format and makes the deserialize side tag-tolerant. Smallest change but requires the macro to know about IpcPayload specifically.

2. to_guest_bytes keeps the tag. Drop the special case for RawJson and Custom; emit them the same way serde does natively (as {"type":"raw_json", ...}). Symmetric with from_slice. Breaks anyone who currently reads the unwrapped inner value with serde_json::from_slice::<Value>(payload) — worth checking for callers of that shape.

3. Handler-arg type match on unwrapped variants. When the macro sees the arg type is IpcPayload, generate branchier code that first tries from_slice::<IpcPayload>, then on tag-error tries from_slice::<Value> and wraps in IpcPayload::RawJson.

Happy to send a PR against whichever direction is preferred — this bit us on a real capsule (astrid-capsule-srouter) and the workaround (changing our handler to take String and parse manually) is easy, but the ergonomics + the doc-comment/behavior mismatch felt like something worth surfacing.

Related evidence

Test in the same file demonstrates the wire behavior is intentional:

astrid-types-0.7.0/src/ipc.rs, to_guest_bytes_raw_json_unwraps:

#[test]
fn to_guest_bytes_raw_json_unwraps() {
    let inner = serde_json::json!({"key": "value"});
    let payload = IpcPayload::RawJson(inner.clone());
    let bytes = payload.to_guest_bytes().unwrap();
    let roundtrip: Value = serde_json::from_slice(&bytes).unwrap();
    assert_eq!(roundtrip, inner);
    assert!(roundtrip.get("type").is_none());
}

The test asserts the tag is absent. There's no corresponding round-trip test that goes IpcPayload -> to_guest_bytes -> from_slice::<IpcPayload>, which is exactly the path the macro takes.