Regression: sar_slc_preprocessing results can no longer get trough load_stac

[EmileSonneveld]

url = "https://s3.waw4-1.cloudferro.com/calrissian/pvc-91dd9cc8-952b-4f4a-abf9-77a9e32beb58/j-260511103149443793-cal-cwl-fbeee30d/qnu6gdwm/collection.json"
datacube_load = get_connection().load_stac(url)
job = datacube_load.create_job(title=auto_title + " datacube_load")
job.start_and_wait()
job.get_results().download_files(tmp_dir)

java.lang.IllegalArgumentException: requirement failed: Range bounds must be ordered, but 4611686018427387579 > 1012
	at scala.Predef$.require(Predef.scala:337)
	at org.locationtech.sfcurve.zorder.ZRange.<init>(ZRange.scala:22)
	at org.locationtech.sfcurve.zorder.ZRange$.apply(ZRange.scala:39)
	at org.openeo.geotrelliscommon.package$ConfigurableSpatialPartitioner.indexRanges(package.scala:187)
	at geotrellis.spark.partition.SpacePartitioner.<init>(SpacePartitioner.scala:34)
	at org.openeo.geotrellis.layers.FileLayerProvider.readKeysToRasterSources(FileLayerProvider.scala:827)
	at org.openeo.geotrellis.layers.FileLayerProvider.prepareRasterRegions(FileLayerProvider.scala:1049)
	at org.openeo.geotrellis.layers.FileLayerProvider.readMultibandTileLayer(FileLayerProvider.scala:1013)
	at org.openeo.geotrellis.file.PyramidFactory.datacube(PyramidFactory.scala:175)
	at org.openeo.geotrellis.file.PyramidFactory.datacube_seq(PyramidFactory.scala:132)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

[EmileSonneveld]

@clausmichele When loading and saving the CWL stac, the output has a different resolution and is flipped among the Y-axis in Q-gis. Not sure if the flipping has an influence on the geocoded result.

[clausmichele]

It shouldn't be an issue since for geocoding it's important that the associated lat/lon pairs are correctly assigned to the pixels.

Anyway, geocoding returns an error for me, here is the code I used to test it:

import numpy as np
import openeo

connection = openeo.connect("https://openeo.dataspace.copernicus.eu").authenticate_oidc()
slc_datacube = connection.load_stac("https://s3.waw4-1.cloudferro.com/calrissian/pvc-85434649-b4e1-4b9f-a44e-65533ed2e71f/j-2605130921384eaf90-cal-cwl-93c664f2/7k4jnpna/collection.json")

# Boxcar filter
range_dim   = 21
azimuth_dim = 5
# Create kernel
kernel = np.ones((azimuth_dim,range_dim))

# Pad with zeros to get a square kernel
pad = int((range_dim - azimuth_dim) / 2)
kernel = np.pad(kernel,((pad,pad),(0,0)))
vh_boxcar = slc_datacube.filter_temporal(["2018-01-27","2018-01-29"]).filter_bands(["i_VH","q_VH"]).apply_kernel(kernel=kernel,
                                                factor=1/(range_dim*azimuth_dim),
                                                border=0)

amplitude = (vh_boxcar.band("i_VH")**2 + vh_boxcar.band("q_VH")**2)**0.5

amplitude_db = amplitude.apply(lambda x: x.log(base=10)) * 10

coordinate_bands = slc_datacube.filter_temporal(["2018-01-27","2018-01-29"]).filter_bands(["grid_lat","grid_lon"]).drop_dimension("t").rename_labels(dimension="bands", target=["latitude","longitude"])
amplitude_db.metadata = None  # clear to avoid temporal dimension issues
amplitude_db = amplitude_db.add_dimension(name="bands",type="bands",label="VH_amp").drop_dimension("t")\
    .merge_cubes(coordinate_bands).add_dimension(name="t", type="temporal", label="2018-01-28T00:00:00Z")

amplitude_db = amplitude_db.resample_spatial(projection="EPSG:32632",resolution=20.0,method="geocode").save_result(format="GTiff")

job_options = {
    "openeo-jar-path": "https://artifactory.vgt.vito.be/artifactory/libs-release-public/org/openeo/geotrellis-extensions/geocode/geotrellis-extensions-geocode.jar"
}

job = amplitude_db.create_job(title="ClouDInSAR_amplitude_from_preprocessing_geocoded_new", job_options=job_options)
job.start()

and the error:

OpenEO batch job failed: OpenEOApiException(status_code=500, code='Internal', message="Unexpected error during 'load_stac' (node id 'loadstac1'): Py4JError('An error occurred while calling o2128.setResolveTileOverlap. Trace:\\npy4j.Py4JException: Method setResolveTileOverlap([class java.lang.Boolean]) does not exist\\n\\tat py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:321)\\n\\tat py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:329)\\n\\tat py4j.Gateway.invoke(Gateway.java:274)\\n\\tat py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)\\n\\tat py4j.commands.CallCommand.execute(CallCommand.java:79)\\n\\tat py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:184)\\n\\tat py4j.ClientServerConnection.run(ClientServerConnection.java:108)\\n\\tat java.base/java.lang.Thread.run(Thread.java:1583)\\n\\n'). The process had these arguments: {'url': 'https://s3.waw4-1.cloudferro.com/calrissian/pvc-85434649-b4e1-4b9f-a44e-65533ed2e71f/j-2605130921384eaf90-cal-cwl-93c664f2/7k4jnpna/collection.json'} ", id='no-request')

@jdries

[EmileSonneveld]

The setResolveTileOverlap overlap error is due to a method call that got added in Python and Scala at the same time. I'll try to merge the latest scala code in a new geocode branch.

EDIT: The updated branch fails on maven build:
[main] ERROR org.apache.maven.cli.MavenCli - Failed to execute goal on project geotrellis-extensions: Could not resolve dependencies for project org.openeo:geotrellis-extensions:jar:geocode_20260505: Could not find artifact org.openeo:geotrellis-sentinelhub:jar:geocode_20260505 in repo (http://artifactory.vgt.vito.be/artifactory/repo)
EDIT2: Retrying without including sentinelhub code in the build
EDIT3: geocode_20260505 did compile. It might be a fix
EDIT4: A job with geocode came trough, but was empty: j-2605131520454736ba5a3d8a43f40979
EDIT5: Updated the original geocode branch too, but also gave empty result: j-26051321233047529be25461dd447571

[EmileSonneveld]

@clausmichele Error is fixed, but no idea about the empty results yet