Handling multiple images annotations with unique context

[AndrzejMachnikDacDigital]

I know that OpenSeadragon let's you turn on the sequenceMode, but that annotates every images at once.
I want to annotate multiple images, but have separate context for them.

Is there some out-of-the-box solution for that, or do I have to somehow initialize multiple instances of the annotator and viewer and - at the end - collect all its instances data into one, keeping the particular image filename?
Seems like it may be troublesome to do so, but I didnt think of a better idea, yet

Maybe someone already have a solution for something similar and is up to share?

[rsimon]

Hi,

by separate context, you mean you'd want to store which image the annotation belongs to, correct? I don't have anything ready right now. But it's a scenario that has come up a few times.

FWIW, Annotorious v2 did have a very simple sequence mode plugin that simply kept a record of the annotations per page. I'm also facing a use case myself right now, where I'd have multiple overlay versions of the same image, stacked on top of each other. And then annotation would either be on a specific layer, of the stack as a whole. (Contract not signed yet, so not sure if this is coming.)

In any case: both of these solutions would use a single OpenSeadragon viewer and, accordingly, a single Annotator instance.

If, instead, you want to spawn multipe OpenSeadragon viewers, then you'd need one dedicated Annotator instance for each viewer. There's actually a package that simplifies handling of multiple Annotators together. I'm using it in this multi-image app. Let me know if that's more what you're looking for and I can explain more.

[AndrzejMachnikDacDigital]

Hello, this seems like it hits home pretty close, with the difference that i'd like to have one image viewable at a time, but that doesnt seem to fall that far off and shouldnt be much of an issue.

The biggest problem I'd see in my case would be how to get all the instances data at once at some point, since the hook wouldnt yield all the results and all these instances would have it abstracted from the parent. Of course I could use some global context managment like Zustand (just thought about it and it seems to be a good way around I guess), but maybe thats whats the package is solving (or you, in this case)

What does the package allow/simplifies, exactly? Prior to asking the question I'd most likely just start doing something like in the package's App.tsx, so just having multiple instances at once, but I guess I'd stumble upon some issue without that?

[rsimon]

with the difference that i'd like to have one image viewable at a time

That actually sounds like all you need to do is swap out the OSD tile source. There's a bit of state management you'd have to take care of yourself–clear the annotations from the annotator, and load the annotations for the new image. That should be fairly straight forward. But, yes, in a sense it would be a kind of state management outside of Annotorious. In the most trivial case it could be a simple map. Something like

imageId: string -> annotations: ImageAnnotation[].

But something like zustand is certainly a good choice once things start getting more involved.

As for the @annotorious/react-manifold package: what it does is provide a few small helpers if you are working with multiple annotators at once. You'd have to do some state management yourself. But manifold gives you stuff like:

• A getAnnotation(id) method that automatically finds the annotator which has that annotation
• on(event) methods so you don't have to manually register separate event listers on all annotator instances
• a bit of coordination between annotators, so that e.g. annotations in all other annotators get de-selected when you select an annotation in another annotator.

[AndrzejMachnikDacDigital]

Oh I see, so the package doesnt seem to be the fit for my case, thank you for the explaination, though!

When you change the tile source it keeps the annotations (not showing them) until you draw new.

You'd suggest creating a solution that could work on listeners (create/update/destroy) and just catch any annotation done on a particular image? I'd have to load them back into the annotator when the image is back into rotation, thats what you meant?

Wouldnt be the easiest, but definitely seems doable

[rsimon]

Yes, that's right. Changing the tilesource doesn't affect the annotations. (It might not actually be desirable - see e.g. use case of multiple wavelength photograph layers for the same object.) So you'd have to manage the switch yourself. Either by keeping track of the annotations via events, as you say - or by simply leaving it up to Annotorious + calling .getAnnotations/clearAnnotations before changing the image, and then setAnnotations with the annotations for the next image after you changed the image.

[AndrzejMachnikDacDigital]

I think I got all there's to know about the topic - thank you @rsimon!

[OsamuBlack]

Hi,

Regarding this setup, I have an app that needs to label images in sequence. i.e. loading image one after another and I am basically swapping the Open Sea Dragon Tile source to load new image.

However, loading next image using external state triggers a rerender in the react app and new image loading which causes a bit of jitter.

There are events in the OSD previous and next buttons in its sequence mode. I wish to listen to press events and modify the state externally (controlled sequence if you may). They images are replaced more seamlessly using this method.

I also need to create a custom reference strip.

If we could get a reference to the OSD viewer via the useViewer. Can't seem to set it up.

[OsamuBlack]

Somehow got this to work:

import { OpenSeadragonViewerProps, useViewer } from "@annotorious/react";
import { Button } from "@nextui-org/react";
import { useEffect } from "react";

interface ViewerControlsProps {
  handleSelectImage: (index: number) => void;
}

export default function ViewerControls({
  handleSelectImage,
}: ViewerControlsProps) {
  const viewer = useViewer();

  useEffect(() => {
    if (!viewer) return;

    viewer?.addHandler("page", (props) => {
      handleSelectImage(props.page);
    });

    return () => {
      if (viewer) {
        viewer.removeAllHandlers("page");
      }
    };
  }, [viewer]);

  return null;
}

     <OpenSeadragonAnnotator
          tool={tool === "select" ? "rectangle" : tool}
          drawingEnabled={tool === "select" ? false : true}
          drawingMode="drag"
        >
          <OpenSeadragonViewer
            className="h-[calc(100vh_-_4rem)]"
            options={
              tileSources: images.map((image) => ({
                type: "image",
                url: `${process.env.NEXT_PUBLIC_API_URL}/image/serve${image.path}`,
              })),
              preserveViewport: true,
              sequenceMode: true,
              showReferenceStrip: true,
              referenceStripPosition: "BOTTOM_LEFT",
              gestureSettingsMouse: {
                dblClickToZoom: true,
                clickToZoom: false,
              },
              autoHideControls: false,
              gestureSettingsTouch: {
                pinchToZoom: true,
                scrollToZoom: true,
                clickToZoom: false,
                dragToPan: true,
              },
            }}
          />
          <ViewerControls handleSelectImage={handleDirectImage} />
          </OpenSeadragonAnnotator>

Might be nice for a tutorial.

OSD Documentation Reference