Skip to content

[Feature Request] distribute tpl commit templates like create-from-image templates #499

@zyl1121

Description

@zyl1121

Problem / Motivation

tpl commit currently creates an object with kind=template, but the committed template appears to have a ready replica only on the origin node where the source sandbox was running.

This differs from templates created through create-from-image, which are distributed to healthy target nodes and can then be scheduled cluster-wide.

In a multi-node cluster, this makes the scheduling semantics of tpl commit templates unclear:

  • a template created by tpl commit can only create new sandboxes on the origin node
  • other healthy nodes do not appear to receive the committed rootfs data
  • the final object is still kind=template, so it looks like an ordinary reusable template, but does not behave like templates created through create-from-image

I would like to confirm whether this single-origin behavior is intentional, or whether templates created by tpl commit are expected to be distributed like other templates.

Current Behavior

In a multi-node cluster:

  1. A sandbox is running on one node.
  2. tpl commit is used to commit that sandbox into a template.
  3. The resulting object is created as kind=template.
  4. The committed rootfs remains available only on the origin node.
  5. New sandboxes based on that template cannot be scheduled normally across other healthy nodes unless the rootfs is replicated manually outside CubeSandbox.

Expected Behavior / Question

Should a template created by tpl commit behave like an ordinary distributed template?

In other words, after the sandbox is committed on the origin node, should the committed rootfs be distributed to healthy target nodes using the normal template replica/status lifecycle?

Proposed Solution

My current implementation direction is:

  1. Keep tpl commit producing kind=template.
  2. After runTemplateCommitJob commits the source sandbox on the origin node, export the committed rootfs as a standard ext4 rootfs artifact.
  3. Register that ext4 file as a RootfsArtifact.
  4. Reuse the existing distributeRootfsArtifact flow used by create-from-image.
  5. Reuse the normal template replica/status lifecycle so the committed template is distributed to healthy target nodes instead of staying single-node.

This would keep the committed object on the ordinary template path:

  • normal template status and replica reporting
  • normal delete and reconcile behavior
  • failed-node retries through the existing redo flow

The change I am considering is intentionally scoped to tpl commit only. It does not try to change snapshot semantics or redesign create-from-image build placement in the same PR.

Alternatives Considered

  1. Keep tpl commit as a single-node template.

This preserves the current behavior, but it makes kind=template semantics inconsistent with the rest of the template system and makes committed templates much less useful in multi-node clusters.

  1. Treat tpl commit output more like a snapshot than a template.

This would align better with single-node locality, but it would be a larger semantic change because the current API already creates kind=template. It would likely need broader lifecycle and UX discussion.

  1. Add a new direct node-to-node or origin-node download path.

This seems less attractive than reusing RootfsArtifact, because CubeSandbox already has a standard artifact distribution flow with status tracking and retry behavior.

Additional Context

Relevant code paths:

  • CubeMaster/pkg/templatecenter/template_commit.go
  • CubeMaster/pkg/templatecenter/template_image.go

Maintainer Feedback Requested

Should tpl commit templates be distributed to healthy target nodes like create-from-image templates, or is the current single-origin behavior intentional?

If distributed behavior is preferred, I plan to continue with the artifact-based approach above and keep the change scoped to tpl commit template distribution only.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/CubeMasterImpacts the CubeMatser (control plane)enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions