Skip to content

feat: add bitmap surface protocol and renderer - #133

Open
ata-sesli wants to merge 7 commits into
orhun:mainfrom
ata-sesli:feature/bitmap-surface
Open

feat: add bitmap surface protocol and renderer#133
ata-sesli wants to merge 7 commits into
orhun:mainfrom
ata-sesli:feature/bitmap-surface

Conversation

@ata-sesli

Copy link
Copy Markdown

Summary

This introduces a Ratty-native bitmap surface protocol and renderer, initially motivated by RChat's inline image previews.

  • add the ratty;i APC protocol for registering, placing, updating, and deleting bitmap surfaces
  • support bounded PNG registration and RGBA8 live-frame updates
  • add cropping, fit modes, filtering, opacity, and stable surface placement
  • synchronize bitmap assets and instances with the Bevy renderer
  • advertise Ratty session and terminal cell-size information to child applications
  • answer the terminal capability queries needed by existing Kitty-aware applications
  • add placement, pan/zoom, and live-frame examples
  • document the protocol and integration model

Motivation

Ratty already supports Kitty graphics, but applications such as RChat also need reusable GPU-backed surfaces with explicit identifiers and lifecycle management.

This is deliberately a separate Ratty protocol rather than a replacement for Kitty graphics or RGP. It gives Ratty-native applications a stable path for image registration, repeated placement, and efficient live-frame updates.

This follows our earlier discussion about supporting a first version of RChat in Ratty.

Review guide

The commits are organized as a review path:

  1. protocol specification
  2. parsing and surface lifecycle
  3. rendering and synchronization
  4. terminal capability advertisement
  5. examples
  6. user-facing documentation

The production changes are localized around the new bitmap surface state, protocol parser, renderer integration, and terminal capability handling. There are no new dependencies or unrelated configuration changes.

Safety and lifecycle

  • protocol headers and payloads are bounded
  • chunked uploads are assembled transactionally
  • malformed and stale transfers are cleaned up
  • surface generations prevent stale GPU assets from being reused
  • renderer assets and instances have explicit cleanup paths
  • terminal compatibility is advertised without pretending that Ratty is xterm-kitty

Compatibility

  • existing RGP behavior remains unchanged
  • existing Kitty graphics support remains available
  • the new commands use the separate ratty;i namespace
  • existing terminal applications can query window and cell dimensions
  • Kitty-aware applications can detect the relevant supported capabilities

Verification

  • cargo fmt --all -- --check
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo nextest run — 90 tests passed
  • cargo build
  • manually exercised RChat inline image previews in Ratty

Demo

Screenshot 2026-07-16 at 18 29 31 Screenshot 2026-07-16 at 18 45 50

Draft status

Opening this as a draft so we can confirm the protocol shape and renderer boundaries before requesting final review.

@orhun

orhun commented Jul 29, 2026

Copy link
Copy Markdown
Owner

Needs rebasing 😇

@ata-sesli
ata-sesli force-pushed the feature/bitmap-surface branch from 484cab7 to 69c3589 Compare July 30, 2026 10:16
@ata-sesli

Copy link
Copy Markdown
Author

Rebased onto the latest main.

@orhun

orhun commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Thoughts on this? @gold-silver-copper

@orhun
orhun requested review from gold-silver-copper and orhun and removed request for gold-silver-copper August 4, 2026 23:20
@gold-silver-copper

Copy link
Copy Markdown
Collaborator

Slightly redundant since we already support kitty, but the direct gpu access for a terminal protocol is pretty interesting, and since ratty is already experimental I think it's fine.

Nitpicks: No limits on decompressd images. There is a 64mb limit on compressed ones, but none for decompressed, and no limits on image dimensions. A malicious actor could compress a 10 gigabyte image in less than 10 megabytes, effectively zip bombing ratty.

Suggested fix: pass explicit image::Limits with max_image_width/max_image_height before decoding, add a configurable total-bytes budget across bitmaps evicting or rejecting past it, and cap concurrent pending transfers.

@ata-sesli

Copy link
Copy Markdown
Author

Slightly redundant since we already support kitty, but the direct gpu access for a terminal protocol is pretty interesting, and since ratty is already experimental I think it's fine.

Nitpicks: No limits on decompressd images. There is a 64mb limit on compressed ones, but none for decompressed, and no limits on image dimensions. A malicious actor could compress a 10 gigabyte image in less than 10 megabytes, effectively zip bombing ratty.

Suggested fix: pass explicit image::Limits with max_image_width/max_image_height before decoding, add a configurable total-bytes budget across bitmaps evicting or rejecting past it, and cap concurrent pending transfers.

Thanks, fixed! PNG decoding now includes clear limits on image dimensions and memory allocation, along with configurable budgets for total bitmap memory, pending bytes, and concurrent transfers. I also added targeted tests for oversized images, budget exhaustion, and recovery after deletion or transfer completion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants