Skip to content

Prototype: ML Assisted labeling using superpixels#860

Draft
IgorSusmelj wants to merge 4 commits into
mainfrom
ml-assisted-labeling-prototype-is
Draft

Prototype: ML Assisted labeling using superpixels#860
IgorSusmelj wants to merge 4 commits into
mainfrom
ml-assisted-labeling-prototype-is

Conversation

@IgorSusmelj

Copy link
Copy Markdown
Contributor

What has changed and why?

This is a prototype for ML assisted labeling using superpixels (SLIC algorithm).

  • We add a new toolbar button for the labeling feature using ML assistance
  • The moment the user clicks on it we compute super pixels for the whole image
  • To speed things up we compute super pixels on a reduced resolution (we resize the image to be contained in max 512 pixels width/height)
  • We compute superpixels on a fine grained level and then merge cells in two stages to make them bigger. We end up with three levels of details (coarse, medium, fine)
  • The user can either click on a cell or paint over the cells to "toggle" the current class. The output annotation is always an instance segmentation
  • Users can also edit existing instance segmentations using the ML assisted labeling feature
  • The backend remains unchanged (we don't have to change anything as we simply add a "smart" brush in the frontend
superpixel-labeling.mp4

How has it been tested?

  • New tests and manual testing

Did you update CHANGELOG.md?

  • Yes
  • Not needed (internal change)

@coderabbitai

coderabbitai Bot commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 63b5491f-f68b-4363-83ac-cdd94d28f071

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch ml-assisted-labeling-prototype-is

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

IgorSusmelj and others added 4 commits July 5, 2026 22:12
- Use useSegmentationMaskBrush (renamed from useInstanceSegmentationBrush)
  and keep main's requestLabel class-picker flow instead of the removed
  DEFAULT-label auto-creation
- Map SLIC tool onto the unified segmentation_mask annotation type
  (instance/semantic split was removed on main)
- Wire deleteAnnotation and SelectClassDialog into SampleSlicRect
- Update thunk-based useAnnotation/useAnnotationLabels usage
- Port finishBrush options (deferDrawingReset, skipImageRefetch,
  refreshAnnotations) and return-value tests to useSegmentationMaskBrush.test.ts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extract the SLIC algorithms (clustering, boundaries, hierarchical
merging, mask helpers) into the @lightly-ai/slic package (Rust/WASM
with automatic pure-JS fallback, ~1.5-1.8x faster than the previous
in-repo implementation). The app keeps only the DOM-side adapter:
image loading/downscaling, data-URL rendering, dev URL rewriting,
level presets, and per-image caching.

Consumed as a local file: dependency on ../slic-package for now;
switch to a published npm version once the package stabilizes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SlicResult now carries the package's CSR pixel-index buffers and only
materializes labelPixelIndexes lazily on first access, so mask building
in the SLIC tool iterates typed arrays instead of nested JS arrays.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@IgorSusmelj IgorSusmelj force-pushed the ml-assisted-labeling-prototype-is branch from f79d74a to a75240b Compare July 6, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant