Skip to content

Mark tests that require network or geotiff dependency. #698

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sebastic wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Mark tests that require network or geotiff dependency. #698

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions pytest.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,6 @@
addopts = --import-mode=importlib
pythonpath = tests
testpaths = tests
markers =
online: marks tests requiring network
geotiff: marks tests requiring geotiff module
1 change: 1 addition & 0 deletions tests/test_execute.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,6 +235,7 @@ def get_output(doc):
class ExecuteTest(TestBase):
"""Test for Exeucte request KVP request"""

@pytest.mark.online
@pytest.mark.xfail(reason="test.opendap.org is offline")
def test_dods(self):
if not WITH_NC4:
Expand Down
4 changes: 4 additions & 0 deletions tests/test_inout.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
import json
from pywps import inout
import base64
import pytest

from pywps import Format, FORMATS
from pywps.app.Common import Metadata
Expand Down Expand Up @@ -128,6 +129,7 @@ def test_file(self):
with self.assertRaises(TypeError):
self.iohandler[0].data = '5'

@pytest.mark.online
def test_url(self):
if not service_ok('https://demo.mapserver.org'):
self.skipTest("mapserver is unreachable")
Expand Down Expand Up @@ -552,6 +554,7 @@ def test_base64(self):
b = self.complex_out.base64
self.assertEqual(base64.b64decode(b).decode(), self.data)

@pytest.mark.online
def test_url_handler(self):
wfsResource = 'http://demo.mapserver.org/cgi-bin/wfs?' \
'service=WFS&version=1.1.0&' \
Expand Down Expand Up @@ -798,6 +801,7 @@ def test_json(self):
)


@pytest.mark.online
class TestMetaLink(TestBase):

def setUp(self) -> None:
Expand Down
3 changes: 3 additions & 0 deletions tests/test_ows.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from pywps import get_ElementMakerForVersion
import pywps.configuration as config
from pywps.tests import client_for, assert_response_success, service_ok
import pytest

wfsResource = 'https://demo.mapserver.org/cgi-bin/wfs?service=WFS&version=1.1.0&request=GetFeature&typename=continents&maxfeatures=10' # noqa
wcsResource = 'https://demo.mapserver.org/cgi-bin/wcs?service=WCS&version=1.0.0&request=GetCoverage&coverage=ndvi&crs=EPSG:4326&bbox=-92,42,-85,45&format=image/tiff&width=400&height=300' # noqa
Expand Down Expand Up @@ -91,6 +92,7 @@ def sum_one(request, response):
supported_formats=[get_format('GEOTIFF')])],
grass_location='epsg:4326')

@pytest.mark.online
def test_wfs(self):
if not service_ok('https://demo.mapserver.org'):
self.skipTest("mapserver is unreachable")
Expand All @@ -117,6 +119,7 @@ def test_wfs(self):
# . the inclusion of output
# . the type of output

@pytest.mark.online
def test_wcs(self):
if not config.CONFIG.get('grass', 'gisbase'):
self.skipTest('GRASS lib not found')
Expand Down
4 changes: 4 additions & 0 deletions tests/validator/test_complexvalidators.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ class data_format(object):

return fake_input

@pytest.mark.online
def test_gml_validator(self):
"""Test GML validator
"""
Expand All @@ -79,6 +80,7 @@ def test_gml_validator(self):
# self.assertTrue(validategml(gml_input, MODE.VERYSTRICT), 'VERYSTRICT validation')
gml_input.stream.close()

@pytest.mark.online
@pytest.mark.xfail(reason="gml verystrict validation fails")
def test_gml_validator_verystrict(self):
"""Test GML validator
Expand Down Expand Up @@ -117,6 +119,7 @@ def test_shapefile_validator(self):
self.assertTrue(validateshapefile(shapefile_input, MODE.STRICT), 'STRICT validation')
shapefile_input.stream.close()

@pytest.mark.geotiff
Copy link
Collaborator

@Zeitsperre Zeitsperre Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not simply add the geotiff library to the test requirements?

Copy link
Contributor Author

@sebastic sebastic Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not simply add the geotiff library to the test requirements?

Because the library is not packaged, as mentioned in the OP:

The geotiff module is also not available.

Copy link
Collaborator

@Zeitsperre Zeitsperre Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean.
PyPI: https://pypi.org/project/geotiff/
Conda: https://github.com/conda-forge/python-geotiff-feedstock

geotiff is a core dependency as well:

pywps/requirements.txt

Line 4 in 04f734b

geotiff

Copy link
Contributor Author

@sebastic sebastic Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean.

There is no Debian package for the geotiff Python library:

$ apt-file search dist-packages/geotiff ; echo $?
1

See also: https://lists.debian.org/debian-gis/2025/12/msg00011.html

geotiff is a core dependency as well:

It is not imported unconditionally:

https://github.com/geopython/pywps/blob/main/pywps/validator/complexvalidator.py#L332-L338

Copy link
Collaborator

@Zeitsperre Zeitsperre Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this is specifically a Debian issue! My bad.

If we want to have this level of change, I would suggest that we remove geotiff from the core dependencies and add it to an extras for tracking optional dependencies.

Copy link
Collaborator

@cehbrecht cehbrecht Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sebastic @Zeitsperre @ldesousa can we limit support to conda-forge? I'm not sure if anyone is using direct installation on debian based dists.

Copy link
Member

@fmigneault fmigneault Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If core dependencies are adjusted to remove format-specific libraries, that will help reduce the package, which is great since not every deployment employs all combinations of data formats for their processing.

However, if this change is done, it should be marked as a higher release than a simple patch, since deployments relying on them could break.

Another thing to clarify in the release notes would be that these libraries are actually required only if using ">=STRICT" validation mode (they are imported under them in https://github.com/geopython/pywps/blob/main/pywps/validator/complexvalidator.py), so the formats could still be employed in I/O definitions with lower validation if the libraries are not installed.

Copy link
Contributor Author

@sebastic sebastic Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sebastic @Zeitsperre @ldesousa can we limit support to conda-forge? I'm not sure if anyone is using direct installation on debian based dists.

OSGeo-Live is using the Debian package currently:

https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_pywps.sh#L53

That can of course be changed, I can then stop spending time on the pywps package in Debian.

CC: @kalxas

Copy link
Collaborator

@Zeitsperre Zeitsperre Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

zarr v3 support has been notoriously difficult to adopt due to some very breaking changes. Most projects I work on are pinned below that version. geotiff would need to address this issue before anything.

Who is "us"?

I've worked on geotiff as well and could try my hand at packaging it for Debian (for personal learning purposes). There are other packages (non-Python) I work on that I'd love to see in Debian/Ubuntu.

Yes, please. Treat fiona, geotiff, and netCDF4 in the same way.

I can justify putting some work towards this for this week. Fun end of year project, maybe. @sebastic Would you be fine with me cherry-picking your commit here?

Copy link
Contributor Author

@sebastic sebastic Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who is "us"?

I've worked on geotiff as well and could try my hand at packaging it for Debian (for personal learning purposes). There are other packages (non-Python) I work on that I'd love to see in Debian/Ubuntu.

You're very welcome to join the Debian GIS team:

https://debian-gis-team.pages.debian.net/policy/index.html#membership

Yes, please. Treat fiona, geotiff, and netCDF4 in the same way.

I can justify putting some work towards this for this week. Fun end of year project, maybe. @sebastic Would you be fine with me cherry-picking your commit here?

Sure, go ahead.

def test_geotiff_validator(self):
"""Test GeoTIFF validator
"""
Expand All @@ -141,6 +144,7 @@ def test_netcdf_validator(self):
else:
self.assertFalse(validatenetcdf(netcdf_input, MODE.STRICT), 'STRICT validation')

@pytest.mark.online
@pytest.mark.xfail(reason="test.opendap.org is offline")
def test_dods_validator(self):
opendap_input = ComplexInput('dods', 'opendap test', [FORMATS.DODS,])
Expand Down