Sending images to IJ2 Swing UI fails

[gselzer]

Steps to reproduce:

• Open napari, launch napari-imagej
• Open the IJ2 Swing UI
• Open data (e.g. bricks sample image) in napari
• Transfer the data to ImageJ
  This yields the following error:

[INFO] Found 10 JHotDraw adapters.
Exception in thread "AWT-EventQueue-0" java.lang.ArrayIndexOutOfBoundsException: -1
	at java.util.ArrayList.elementData(ArrayList.java:424)
	at java.util.ArrayList.get(ArrayList.java:437)
	at net.imagej.space.CombinedSpace.axis(CombinedSpace.java:87)
	at net.imagej.interval.CombinedRealInterval.realMin(CombinedRealInterval.java:94)
	at net.imagej.display.DefaultImageDisplay.realMin(DefaultImageDisplay.java:370)
	at net.imagej.display.DefaultImageDisplay.getPlaneExtents(DefaultImageDisplay.java:216)
	at net.imagej.ui.swing.viewer.image.JHotDrawImageCanvas$1.getPreferredSize(JHotDrawImageCanvas.java:166)
	at javax.swing.ViewportLayout.preferredLayoutSize(ViewportLayout.java:96)
	at java.awt.Container.preferredSize(Container.java:1799)
	at java.awt.Container.getPreferredSize(Container.java:1783)
	at javax.swing.JComponent.getPreferredSize(JComponent.java:1664)
	at javax.swing.ScrollPaneLayout.preferredLayoutSize(ScrollPaneLayout.java:492)
	at java.awt.Container.preferredSize(Container.java:1799)
	at java.awt.Container.getPreferredSize(Container.java:1783)
	at javax.swing.JComponent.getPreferredSize(JComponent.java:1664)
	at java.awt.BorderLayout.preferredLayoutSize(BorderLayout.java:719)
	at java.awt.Container.preferredSize(Container.java:1799)
	at java.awt.Container.getPreferredSize(Container.java:1783)
	at javax.swing.JComponent.getPreferredSize(JComponent.java:1664)
	at net.miginfocom.swing.SwingComponentWrapper.getLayoutHashCode(Unknown Source)
	at net.miginfocom.swing.MigLayout.checkCache(Unknown Source)
	at net.miginfocom.swing.MigLayout.getSizeImpl(Unknown Source)
	at net.miginfocom.swing.MigLayout.preferredLayoutSize(Unknown Source)
	at java.awt.Container.preferredSize(Container.java:1799)
	at java.awt.Container.getPreferredSize(Container.java:1783)
	at javax.swing.JComponent.getPreferredSize(JComponent.java:1664)
	at java.awt.BorderLayout.preferredLayoutSize(BorderLayout.java:719)
	at java.awt.Container.preferredSize(Container.java:1799)
	at java.awt.Container.getPreferredSize(Container.java:1783)
	at javax.swing.JComponent.getPreferredSize(JComponent.java:1664)
	at javax.swing.JRootPane$RootLayout.preferredLayoutSize(JRootPane.java:920)
	at java.awt.Container.preferredSize(Container.java:1799)
	at java.awt.Container.getPreferredSize(Container.java:1783)
	at javax.swing.JComponent.getPreferredSize(JComponent.java:1664)
	at java.awt.BorderLayout.preferredLayoutSize(BorderLayout.java:719)
	at java.awt.Container.preferredSize(Container.java:1799)
	at java.awt.Container.getPreferredSize(Container.java:1783)
	at java.awt.Window.pack(Window.java:809)
	at org.scijava.ui.swing.viewer.SwingDisplayWindow.showDisplay(SwingDisplayWindow.java:68)
	at org.scijava.ui.viewer.DisplayViewer.view(DisplayViewer.java:84)
	at org.scijava.ui.AbstractUserInterface$1.run(AbstractUserInterface.java:138)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$1(DefaultThreadService.java:211)
	at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

[gselzer]

Thanks to @elevans for helping me make progress here.

The issue is that the dimensions of the Dataset are not assigned to X and Y; DefaultImageDisplay expects them to be set (here) when the Swing UI displays that Dataset.

The easiest path forward here is to wait until a pyimagej release with the fixes provided in imagej/pyimagej#238. With the fixes there, we can pass dim_order=['row', 'col'] to ij().py.to_dataset here, and things will magically work (I tested it with a local installation of @elevans' branch on my machine).

My only concern then would be what happens with non-two-dimensional images. Does anyone else (@ctrueden?) know how this plan should be altered for images of arbitrary dimensionality?

[ctrueden]

ImageJ2 Dataset supports images with any dimension order. Except that as you discovered, not having XY as the first two dimensions does not actually work in practice, due to planar assumptions in a few places. Ideally we would change this on the ImageJ2/Java side, but failing that, changes to napari-imagej to ensure that X and Y are the first two dimensions should avoid the issue. You shouldn't have to shuffle around anything else; just pull X and Y to the front.

[gselzer]

@ctrueden thanks. The other question, though, is whether there is dim_order support for saying "make sure the first axis is rows, the second axis is columns, and then I don't care about the rest". @elevans can you do this?

[elevans]

@gselzer I can add a special flag to the direct image conversions (maybe something like enforce='napari') that will force row/ col or X/Y dimension assignment on the first 2 axes. Would that be helpful?

[gselzer]

@elevans I think that the current kwargs are fine, and I wouldn't want to change them for this.

If I'm passing a 5-D image, I want dim_order=['row', 'col'] to work even though I don't have 5. Does that work now? If not, are there obstacles to making that work?

[elevans]

That does not work now, but I can write a little helper method that will detect the mismatch in dim_order length and the shape of the array and have it fill the remaining dimensions with dim_n or unknown.

[elevans]

@gselzer I've added support (imagej/pyimagej@b2beb69) for dim_order lengths that are smaller than the number of dimensions the source data has. In these cases the unknown dimensions are populated with dim_n where n is the incrementing number of unknown dimensions. This follows xarray's convention. If you pass a dim_order that is larger than the number of dimensions the source data has we raise a ValueError.

Now I just need to get it merged 😛