Redesign OneOf unions as first-class sum types

[raphael]

Redesigning OneOf Unions as First-Class Sum Types

Overview

This change overhauls how Goa represents and transports OneOf unions.
Instead of interface-based unions with helper transport objects, Goa now
generates a first-class sum-type struct per union with a discriminator
field, strongly-typed branches, and custom JSON marshaling.

At a high level this delivers:

• Cleaner Go APIs: unions are concrete, inspectable values with a
  well-defined API (Kind, NewXxx, AsXxx, Validate).
• Predictable JSON contracts: every union uses the same canonical
  { "type": "...", "value": <native JSON> } shape.
• Better alignment with standards: OpenAPI oneOf and protobuf
  oneof now map more directly to the generated Go types.
• Simpler code generation: the old web of helper types and
  transform templates is removed in favor of a single, explicit model.

This document focuses on why this redesign was made and the
benefits it unlocks, independent of any particular application.

---

Background: The Old Interface-Based Unions

Historically, Goa modeled unions as interfaces with marker methods
plus helper transport objects:

• Each union U became an interface implemented by per-branch wrapper
  types (e.g. type U interface{ uVal() }, type UString struct{...}).
• HTTP/JSON used a separate {Type,Value string} helper object, where
  Value held JSON-encoded data as a string.
• GoTransform had dedicated templates to shuttle between the interface
  and this helper object for HTTP and JSON-RPC.

This worked, but it had clear drawbacks:

• The Go API did not look like a first-class sum type.
• JSON contained JSON-in-a-string, which was hard to read and debug.
• Transforms needed union-specific special cases and helper graphs.

---

New Model: Closed Sum-Type Structs

Unions are now generated as closed sum types:

• Each union U becomes a concrete struct:
  • a kind field of enum-like type UKind.
  • one field per branch (A, B, String, Int, etc.).
• Each branch has:
  • a constructor NewUA(...) U.
  • an accessor AsA() (T, bool).
• The union type implements:
  • Validate() error to check the discriminant.
  • custom MarshalJSON / UnmarshalJSON using a canonical
    {type,value} representation.

Conceptually, this moves unions much closer to:

• algebraic data types / sum types in typed functional languages,
• protobuf oneof,
• OpenAPI oneOf with an explicit discriminator.

The type is closed: only the generated branches are valid, and
the discriminator is always one of the known tags (or empty).

---

Why This Is Better (Conceptually and Practically)

1. First-Class Sum Types in Go

The Go representation is now explicitly a sum type:

• There is a single, concrete struct that owns:
  • the active branch discriminator (kind), and
  • the values for each branch.
• Branch operations are explicit:
  • NewUnionFoo(v) to construct a value in the Foo branch.
  • AsFoo() to safely read the Foo branch.

Benefits:

• Discoverability: IDEs show the complete union API in one place.
• Type safety: you cannot accidentally introduce a branch that
  the type does not know about.
• Refactorability: renaming a branch or changing its type
  affects one struct instead of an interface plus multiple wrappers.

2. Canonical, Human-Friendly JSON

All unions use the same clear JSON shape:

{ "type": "foo", "value": <native JSON> }

where:

• type is the discriminator string for the chosen branch.
• value is the native JSON representation of the branch value
  (number, string, object, array, etc.).

Benefits:

• No JSON-in-string: tools, logs, and humans see real JSON
  payloads, not nested escape sequences.
• Predictability: every union looks the same on the wire,
  independent of where it appears (request payload, response,
  nested object).
• Interoperability: non-Go clients can easily implement this
  contract without knowing about internal helper types or special
  transform rules.

3. Alignment With OpenAPI oneOf and Protobuf oneof

The new model aligns Goa internal representation and transports
with existing standards:

• OpenAPI:
  • oneOf can be expressed as an object with type (discriminator)
    and value (payload) and a clear mapping from union tags to
    schemas.
• gRPC / protobuf:
  • oneof branches naturally map to the union branch fields.
  • Conversions between protobuf messages and the sum-type struct
    are explicit and straightforward.

Benefits:

• Less impedance mismatch: API descriptions and generated Go
  types reflect the same conceptual model.
• Easier tooling: code generators, schema validators, and client
  libraries can rely on well-known patterns (oneOf/oneof + discriminator)
  instead of Goa-specific conventions.

4. Simpler, More Robust Code Generation

By centralizing the union logic into a single sum-type struct with
custom JSON methods, several layers of complexity vanish:

• No more {Type,Value} helper graphs.
• No more special transform templates to shuttle between unions and
  helper objects.
• GoTransform only needs to handle union-to-union transforms;
  other cases fail clearly instead of relying on hidden helpers.

Benefits:

• Less surface area: fewer templates, fewer special cases, and
  fewer cross-cutting hacks.
• Easier to evolve: adding features such as additional validation,
  richer discriminators, or better OpenAPI annotations becomes more
  straightforward.
• Stronger contracts: unexpected union shapes now fail fast in
  codegen rather than being massaged by transport-specific helpers.

5. Clearer Transport Semantics

The redesign deliberately constrains where unions can appear:

• Unions are supported in JSON bodies (HTTP and JSON-RPC).
• Unions are rejected by design in query parameters, headers,
  cookies, path parameters, and form data.

This is a principled choice:

• Non-body transports generally lack a natural way to encode a rich
  sum type with discriminated variants and nested JSON.
• Forcing such usage tends to produce ad-hoc conventions and fragile
  consumers.

Benefits:

• Fail-fast designs: if a union is placed where it cannot be
  represented cleanly, the DSL validation fails early with a clear
  message.
• Better APIs: unions appear where they make semantic sense
  (JSON bodies), and other transports lean on simpler types that are
  easier to consume.

---

Migration

This redesign is intentionally breaking in a few places. This section
describes the main cases and how to migrate.

1. Go code using the old union interfaces and wrapper types

Previously, unions were exposed as interfaces with per-branch wrapper
types and marker methods. You are affected if you:

• Declare fields, parameters, or results using generated union
  interfaces (e.g. Values as an interface).
• Type-assert on concrete wrapper types (ValuesString, ValuesInt,
  etc.) or call marker methods (valuesVal, myUnionVal, …).

How to migrate:

• Change those interface-typed values to the generated sum-type struct.
• Use the new API:
  • constructors: NewUnionFoo(v) to build a union in a given branch.
  • accessors: AsFoo() to read a branch, combined with Kind() if
    you want a switch on the discriminator.
• Replace references to wrapper types with the corresponding branch
  field type of the sum struct.

2. JSON / HTTP / JSON-RPC clients using {Type,Value string}

Clients and tools that previously sent or expected:

{ "Type": "foo", "Value": "{\"x\": 1}" }

must now speak the canonical shape:

{ "type": "foo", "value": { "x": 1 } }

How to migrate:

• Update non-Goa clients to:
  • use lowercase type / value field names, and
  • send value as native JSON (numbers, strings, objects, arrays),
    not as a JSON string.
• For persisted data, either:
  • migrate stored documents to the new {type,value} shape, or
  • introduce a one-off compatibility reader that accepts the old
    {Type,Value string} form and rewrites it into the sum-type
    representation before handing it to generated code.

3. Unions used outside JSON bodies

Unions are now only supported in JSON bodies (HTTP and JSON-RPC).
If a union appears in headers, query parameters, path parameters,
cookies, or form data, the DSL now rejects the design.

How to migrate:

• Replace such unions with:
  • a concrete struct modeling the possible values explicitly, or
  • multiple endpoints or methods, one per variant, or
  • a simpler enum plus additional fields if that better matches the
    domain.
• Keep OneOf only on attributes that map to JSON bodies.

4. Custom templates or tooling relying on union helper transforms

Custom codegen that called GoTransform with a union on one side and a
{Type,Value} helper object on the other is no longer supported.
GoTransform now expects union-to-union transforms; other cases
fail with a descriptive error.

How to migrate:

• For transports, rely on the union MarshalJSON / UnmarshalJSON
  instead of custom transforms to helper objects.
• For internal projections, use union-to-union transforms (same number
  and order of branches) or write explicit code in terms of the sum-type
  struct (Kind, AsXxx, NewXxx).

---

What Users Gain

From a user perspective, this redesign yields:

• More natural Go types:
  • A single struct with methods is easier to read, debug, and use
    than an interface plus multiple hidden wrappers.
• Cleaner JSON APIs:
  • { "type": "...", "value": ... } is self-explanatory and works
    across languages and platforms.
• Stronger guarantees:
  • The union must be in one of a finite set of states; invalid
    discriminants are caught and reported clearly.
• More predictable generation:
  • The mapping from DSL OneOf to Go, HTTP, OpenAPI, and gRPC is
    uniform and documented, instead of split across several helper
    layers.

Overall, this change trades some short-term migration work for a
long-term model that is simpler, more principled, and easier to
build on for all services using Goa unions.

[xeger]

This seems like a substantial improvement to the status quo. I worry about effectively migrating, thinking about persisted data (yikes!) and other corner cases - but the upsides (e.g. real OpenAPI discriminator declarations!) outweigh the downsides for future net-new services.

Would we enqueue this for a Goa major version bump?

[raphael]

Yes this is a breaking change that can have deep ramifications. We could bump the major version but that would mean changing the import path of Goa which would have even more severe implications (and might cause a lot of people to get "stuck") so my inclination would be not to do that even though you're right in theory that would be the correct approach.

[RedMarcher]

Yes please. Even though this is a breaking change, it will make the code so much cleaner.

I do wonder if this will solve this issue at the same time?
#3869

[akhilmhdh]

Hi @raphael

I was checking out Goa and impressed by it's design choices.
This was the only one felt a weird to pass JSON. Great to see that it's getting changed to be more aligned with openapi.

One small question in this PR is would you be open to support customizing the key props. Like instead of hard coding it as type and value, user can add there own. Like instead of value some APIs would be payload. This would help in adoption to an existing architectures as well.

I have already took a fork of this branch and added support to it. Of-course did it with LLM, so i need to verify it thoroughly before raising the PR. I was thinking it would be optional customization like with Meta("union:type:key", "paymentType")