refactor: cardinal

[rmrt1n]

TL;DR

Not a rollup anymore, just a regular game server with snapshots.

API changes

Below are changes to the user-facing APIs.

Systems no longer return errors

Previously, systems that return errors will cause the world to shutdown. This means systems must always be correct, so returned error is practically useless. A void function is more accurate here.

// Before:
func CreatePlayerSystem(state *CreatePlayerSystemState) error {
    // ...
}

// After:
func CreatePlayerSystem(state *CreatePlayerSystemState) {
    // ...
}

Removed BaseCommand and BaseEvent

These were originally for extending user-defined commands by embedding an interface that we control, but in pratice they're unused and user-defined methods on the command/event type can conflict with the embedded interface.

// Before:
type CreatePlayerCommand struct {
    cardinal.BaseCommand
    Nickname string `json:"nickname"`
}

// After:
type CreatePlayerCommand struct {
    Nickname string `json:"nickname"`
}

Command Persona() and Payload() methods are now struct fields

We're exposing them as public fields instead of getter methods, since the methods don't add anything useful. Removes the extra call in case Go doesn't optimize getters.

// Before:
command := cmd.Payload()

// After:
command := cmd.Payload

Changes to cardinal.WorldOption

More details on this in later sections. EpochFrequency is removed and the SnapshotFrequency is renamed to SnapshotRate. Snapshot types are now imported from pkg/cardinal/snapshot instead of pkg/micro/.

// Before:
import "github.com/argus-labs/world-engine/pkg/micro"

world, err := cardinal.NewWorld(cardinal.WorldOptions{
    EpochFrequency:      10,
    SnapshotStorageType: micro.SnapshotStorageJetStream,
    SnapshotFrequency:   50,
})

// After:
import "github.com/argus-labs/world-engine/pkg/cardinal/snapshot"

world, err := cardinal.NewWorld(cardinal.WorldOptions{
    SnapshotStorageType: snapshot.SnapshotStorageJetStream,
    SnapshotRate:   50,
})

Otherworld.Send is now Otherworld.SendCommand

Just to make it consistent with the client SDKs method names.

// Before:
OtherShard.Send(SomeCommand{})

// After:
OtherShard.SendCommand(SomeCommand{})

Added Remove method to component refs

Should've been here since the original refactor but I just realized I forgot to include it. Note, after a component has been removed, you can't Get/Set it anymore as the entity has been moved to another archetype. Existing refs will be broken. Use with caution. The component methods (Get/Set/Remove) will panic if used on a broken ref. These are programmer errors, so we panic instead of making the methods return error. You should write correct code.

for entity, player := range state.Players.Iter() {
   player.Debuff.Remove() // Removes the component from this entity
}

Environment variable changes

Added

• CARDINAL_DEBUG, boolean, default: false

Renamed

• SHARD_SNAPSHOT_STORAGE_TYPE -> CARDINAL_SNAPSHOT_STORAGE_TYPE
• SHARD_SNAPSHOT_FREQUENCY -> CARDINAL_SNAPSHOT_RATE
• SNAPSHOT_SNAPSHOT_STORAGE_MAX_BYTES -> CARDINAL_SNAPSHOT_STORAGE_MAX_BYTES

Removed

• SHARD_MODE
• SHARD_DISABLE_PERSONA
• SHARD_EPOCH_STREAM_MAX_BYTES

Architecture changes

This section explains the changes done in this PR.

Cardinal is no longer a rollup framework

We're removing all of the rollup-related functionaly (e.g. epochs/transactions, signed commands, follower mode/transaction replay) to simplify the architecture and improve the experience for us and for the users.

Data persistence is now the responsibility of snapshots. Nothing much has changed here.

Note

For reviewers:
The files pkg/micro/shard_* are removed. Snapshots are moved to pkg/cardinal/snapshot/.

Consolidate command and event handling

Previously cardinal and ecs each have their own structures for collecting and processing commands and events. This PR removes the ECS layer in favor of doing all command/event handling in Cardinal. This also simplifies the system registration code, which is the ugliest part of the code by far.

Note

For reviewers:
All command-related code are moved to: pkg/cardinal/internal/command/.
All event-related code are moved to pkg/cardinal/internal/event/.

Refactor system registration

Previously, system registration is done in the ecs package, and cardinal extends these with callback functions, e.g. to register the NATS command handlers when commands are registered.

Now, the control is inverted and split, with cardinal calling ecs to register certain fields. ECS only understands the concepts "components" and "system events", but systems can still use "commands" and "events" injected by Cardinal.

Note

For reviewers:
Relevant files: pkg/cardinal/system.go and pkg/cardinal/internal/ecs/system.go.

New debug module

Basically a place to put all debug-related utilities. It embeds a connectrpc server that clients (e.g. World CLI, Cardinal editor) can connect to directly instead of going through NATS or a gateway. ATM it only has the introspect endpoint for getting the schema of the registered commands, events, and components.

Note

For reviewers:
Relevant files: pkg/cardina/debug.go. It's in the same package as World so it can access the world fields directly.

Refactor the cardinal/service module

The service module will be merged into cardinal. It's the wrong abstraction layer as it requires a lot of fields from World, so just like the debug module, I moved it to the same layer as World.

Note

For reviewers:
Relevant files: pkg/cardinal/service/* moved to pkg/cardinal/service_*.

Out of scope

Several things came up while refactoring, these are basically the next steps that I ignored in this PR so it doesn't get too large. These will be done in another PR:

• The Snapshot interface leaks abstraction layers., nvm, I was overthinking.
• Changed component serialization format from JSON to msgpack (feat: replace json with messagepack #867).
• Query all entities and query limits (feat: query with empty filter #863).

[rmrt1n]

• refactor: cardinal #865 : 3 dependent PRs (#863 , #867 , #868 ) 👈 (View in Graphite)
• main

---

How to use the Graphite Merge Queue

Add the label graphite/merge to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codecov]

❌ 4 Tests Failed:

Tests completed	Failed	Passed	Skipped
119	4	115	0

View the top 3 failed test(s) by shortest run time

::TestMain

Stack Traces | 0s run time

FAIL	github..../bare-bone/shards/game [build failed]

::TestMain

Stack Traces | 0s run time

FAIL	github..../basic/shards/game [build failed]

::TestMain

Stack Traces | 0s run time

FAIL	github..../multi-shard/shards/game [build failed]

::TestMain

Stack Traces | 0s run time

FAIL	github..../multi-shard/shards/chat [build failed]

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}