Skip to content

add sensorthings support #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

add sensorthings support #34

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3d10459
add sensorthings support
jutoth Jan 6, 2025
7528706
Update helpers.py
jutoth Jan 7, 2025
dceaaf4
Update helpers.py
jutoth Jan 7, 2025
6838066
Update filters.py
jutoth Jan 7, 2025
93f57bc
added start/stop flag to sensorthings filter | convert to single geom…
jutoth Jan 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion controller.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from typing import Iterable, Optional, List

from qgis.PyQt.QtCore import pyqtSignal, QObject
from qgis.core import Qgis, QgsProject, QgsMapLayer, QgsMapLayerType, QgsWkbTypes, QgsGeometry
from qgis.core import QgsProject, QgsMapLayer, QgsMapLayerType, QgsWkbTypes, QgsGeometry
from qgis.gui import QgsRubberBand
from qgis.utils import iface

Expand Down Expand Up @@ -125,6 +125,7 @@ def stopSketchingTool(self):
self.mapTool.deactivate()

def onSketchFinished(self, geometry: QgsGeometry):
geometry.convertToSingleType() # SensorThings Filter only supports single geometry type
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other providers are fine with multi-geometries and iirc it works fine if one selects an existing multipolygon as filter geometry. Being able to use multi-geometries is also a wish from the community and something that will probably be added some day.

Any provider-specific limitations should be handled only for that provider, and trigger a warning visible to the user.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, than I will convert it to single Geometry in a later step. For example here:

filters.py -> filterString()

if layer.storageType() == SENSORTHINGS_STORAGE_TYPE:
     ### convert wkt back to QgsGeometry
     ### convert to singleType

self.stopSketchingTool()
if not geometry.isGeosValid():
iface.messageBar().pushWarning(LOCALIZED_PLUGIN_NAME, self.tr("Geometry is not valid"))
Expand Down
64 changes: 53 additions & 11 deletions filters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,45 @@
from typing import List

from qgis.PyQt.QtWidgets import QMessageBox
from qgis.core import QgsVectorLayer, QgsGeometry, QgsCoordinateReferenceSystem

from qgis.core import (
QgsVectorLayer,
QgsGeometry,
QgsCoordinateReferenceSystem,
Qgis
)

if Qgis.QGIS_VERSION_INT > 33600:
from qgis.core import QgsSensorThingsUtils

from qgis.utils import iface

from .settings import FILTER_COMMENT_START, FILTER_COMMENT_STOP, LOCALIZED_PLUGIN_NAME
from .helpers import tr, saveSettingsValue, readSettingsValue, allSettingsValues, removeSettingsValue, \
getLayerGeomName, matchFormatString
from .settings import (
FILTER_COMMENT_START,
FILTER_COMMENT_STOP,
LOCALIZED_PLUGIN_NAME,
SENSORTHINGS_STORAGE_TYPE
)

from .helpers import (
tr,
saveSettingsValue,
readSettingsValue,
allSettingsValues,
removeSettingsValue,
getLayerGeomName,
matchFormatString,
getEntityTypeFromSensorThingsLayer,
reproject_wkt_geometry
)


FILTERSTRING_TEMPLATE = "{spatial_predicate}({geom_name}, ST_TRANSFORM(ST_GeomFromText('{wkt}', {srid}), {layer_srid}))"

# sensorthings filter does not support reprojection (st_transform)
# reprojection happens in helpers.py -> addFilterToLayer
FILTERSTRING_TEMPLATE_SENSORTHINGS = "{spatial_predicate}({geom_name}, geography'{wkt}')"


class Predicate(IntEnum):
INTERSECTS = 1
Expand Down Expand Up @@ -51,25 +80,38 @@ def filterString(self, layer: QgsVectorLayer) -> str:
Returns:
str: A layer filter string
"""
# ST_DISJOINT does not use spatial indexes, but we can use its opposite "NOT ST_INTERSECTS" which does
spatial_predicate = f"ST_{Predicate(self.predicate).name}"
wkt = self.wkt if not self.bbox else self.boxGeometry.asWkt()
srid=self.crs.postgisSrid()
layer_srid=layer.crs().postgisSrid()

if layer.storageType() == SENSORTHINGS_STORAGE_TYPE:
reprojected_wkt = reproject_wkt_geometry(wkt, srid, layer_srid)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A comment why client-side reprojection is necessary (no ST_Transform available for SensorThings I presume?) would be nice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To explain this I added a comment on line 41, where i define the filter template. Of course, i can add it here, too :)

# sensorthings filter does not support reprojection (st_transform)
# reprojection happens in helpers.py -> addFilterToLayer
FILTERSTRING_TEMPLATE_SENSORTHINGS = "{spatial_predicate}({geom_name}, geography'{wkt}')"

spatial_predicate = spatial_predicate.lower() # sensorthings specification uses lower case
entity_str = getEntityTypeFromSensorThingsLayer(layer)
entity_type = QgsSensorThingsUtils.stringToEntity(entity_str)
geom_field = QgsSensorThingsUtils.geometryFieldForEntityType(entity_type)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer the existing getLayerGeomName method to be extended if possible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could add a condition in getLayerGeomName like this, what do you think?:

def getLayerGeomName(layer: QgsVectorLayer):
    if layer.storageType() == SENSORTHINGS_STORAGE_TYPE:
        entity_str = layer.dataProvider().uri().param('entity')
        entity_type = QgsSensorThingsUtils.stringToEntity(entity_str)
        geom_field = QgsSensorThingsUtils.geometryFieldForEntityType(entity_type)
        return geom_field
    return layer.dataProvider().uri().geometryColumn() or getLayerGeomNameOgr(layer)


return FILTERSTRING_TEMPLATE_SENSORTHINGS.format(
spatial_predicate=spatial_predicate,
geom_name=geom_field,
wkt=reprojected_wkt
)
# ST_DISJOINT does not use spatial indexes, but we can use its opposite "NOT ST_INTERSECTS" which does
if self.predicate == Predicate.DISJOINT:
spatial_predicate = "NOT ST_INTERSECTS"

wkt = self.wkt
if self.bbox:
wkt = self.boxGeometry.asWkt()

geom_name = getLayerGeomName(layer)

return FILTERSTRING_TEMPLATE.format(
spatial_predicate=spatial_predicate,
geom_name=geom_name,
wkt=wkt,
srid=self.crs.postgisSrid(),
layer_srid=layer.crs().postgisSrid()
srid=srid,
layer_srid=layer_srid
)


@staticmethod
def fromFilterString(subsetString: str) -> 'FilterDefinition':
start_index = subsetString.find(FILTER_COMMENT_START) + len(FILTER_COMMENT_START)
Expand Down
93 changes: 76 additions & 17 deletions helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,37 @@

from osgeo import ogr
from qgis.PyQt.QtCore import QCoreApplication
from qgis.core import Qgis, QgsExpressionContextUtils, QgsSettings, QgsMapLayer, QgsMapLayerType, QgsVectorLayer,\
QgsWkbTypes
from qgis.core import (
Qgis,
QgsExpressionContextUtils,
QgsSettings,
QgsMapLayer,
QgsMapLayerType,
QgsVectorLayer,
QgsWkbTypes,
QgsProviderRegistry,
QgsGeometry,
QgsCoordinateReferenceSystem,
QgsCoordinateTransform,
QgsProject
)

from qgis.utils import iface

from .settings import SUPPORTED_STORAGE_TYPES, GROUP, FILTER_COMMENT_START, FILTER_COMMENT_STOP, \
LAYER_EXCEPTION_VARIABLE, LOCALIZED_PLUGIN_NAME
from .settings import (
SUPPORTED_STORAGE_TYPES,
GROUP,
FILTER_COMMENT_START,
FILTER_COMMENT_STOP,
LAYER_EXCEPTION_VARIABLE,
LOCALIZED_PLUGIN_NAME,
SENSORTHINGS_STORAGE_TYPE
)

from typing import TYPE_CHECKING

if TYPE_CHECKING:
from .filters import FilterDefinition

def tr(message):
return QCoreApplication.translate('@default', message)
Expand Down Expand Up @@ -73,23 +97,53 @@ def isLayerSupported(layer: QgsMapLayer):


def removeFilterFromLayer(layer: QgsVectorLayer):
currentFilter = layer.subsetString()
if FILTER_COMMENT_START not in currentFilter:
return
start_index = currentFilter.find(FILTER_COMMENT_START)
stop_index = currentFilter.find(FILTER_COMMENT_STOP) + len(FILTER_COMMENT_STOP)
newFilter = currentFilter[:start_index] + currentFilter[stop_index:]
layer.setSubsetString(newFilter)
if layer.storageType() == SENSORTHINGS_STORAGE_TYPE:
layer.setSubsetString('') # sensorthings filter does not support comments (FILTER_COMMENT_START)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to use a different magic string for SensorThings, maybe some string comparison that always evaluates true and gets added with AND? Something like

'{FILTER_COMMENT_START}' == '{FILTER_COMMENT_START}' AND ... filter ... AND '{FILTER_COMMENT_STOP}' == '{FILTER_COMMENT_STOP}'

Otherwise it would not be possible to support additional filters set by the user, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems like a nice work-around. Good idea :)

Unfortunaly comments are not allowed

else:
currentFilter = layer.subsetString()
if FILTER_COMMENT_START not in currentFilter:
return
start_index = currentFilter.find(FILTER_COMMENT_START)
stop_index = currentFilter.find(FILTER_COMMENT_STOP) + len(FILTER_COMMENT_STOP)
newFilter = currentFilter[:start_index] + currentFilter[stop_index:]
layer.setSubsetString(newFilter)


def addFilterToLayer(layer: QgsVectorLayer, filterDef: 'FilterDefinition'):
currentFilter = layer.subsetString()
if FILTER_COMMENT_START in currentFilter:
removeFilterFromLayer(layer)
currentFilter = layer.subsetString()
connect = " AND " if currentFilter else ""
newFilter = f'{currentFilter}{FILTER_COMMENT_START}{connect}{filterDef.filterString(layer)}{FILTER_COMMENT_STOP}'
layer.setSubsetString(newFilter)
if layer.storageType() == SENSORTHINGS_STORAGE_TYPE:
newFilter = filterDef.filterString(layer)
layer.setSubsetString(newFilter)
else:
if FILTER_COMMENT_START in currentFilter:
removeFilterFromLayer(layer)
currentFilter = layer.subsetString()
connect = " AND " if currentFilter else ""
newFilter = f'{currentFilter}{FILTER_COMMENT_START}{connect}{filterDef.filterString(layer)}{FILTER_COMMENT_STOP}'
layer.setSubsetString(newFilter)




def reproject_wkt_geometry(wkt: str, source_crs_epsg: int, target_crs_epsg: int) -> str:
"""
Reproject a WKT geometry from a source CRS to a target CRS.

Args:
wkt (str): The WKT geometry to reproject.
source_crs_epsg (int): The EPSG code of the source CRS.
target_crs_epsg (int): The EPSG code of the target CRS.

Returns:
str: The reprojected WKT geometry.
"""
source_crs = QgsCoordinateReferenceSystem(source_crs_epsg)
target_crs = QgsCoordinateReferenceSystem(target_crs_epsg)
Comment on lines +154 to +155
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe have a quick if source_crs == target_crs: return wkt here. Probably not really that much of a performance improvement overall, but wouldn't hurt :D

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, this is more elegant 👍

transform = QgsCoordinateTransform(source_crs, target_crs, QgsProject.instance())
geometry = QgsGeometry.fromWkt(wkt)
geometry.transform(transform)
return geometry.asWkt()



def getLayerGeomName(layer: QgsVectorLayer):
Expand Down Expand Up @@ -118,6 +172,11 @@ def getLayerGeomNameOgr(layer: QgsVectorLayer):
return columnName


def getEntityTypeFromSensorThingsLayer(layer: QgsVectorLayer):
decoded_url = QgsProviderRegistry.instance().decodeUri(layer.providerType(), layer.dataProvider().dataSourceUri())
return decoded_url.get('entity')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you could directly access layer.dataProvider().uri().param('typename') instead? I am not sure if that would be the same information. :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, this is much shorter:
layer.dataProvider().uri().param('entity')



def hasLayerException(layer: QgsVectorLayer) -> bool:
return QgsExpressionContextUtils.layerScope(layer).variable(LAYER_EXCEPTION_VARIABLE) == 'true'

Expand Down
6 changes: 6 additions & 0 deletions settings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from qgis.PyQt.QtCore import QCoreApplication
from qgis.core import Qgis

def tr(message):
return QCoreApplication.translate('@default', message)
Expand All @@ -19,3 +20,8 @@ def tr(message):

# The QGIS Storage Types that can be filtered by the plugin
SUPPORTED_STORAGE_TYPES = ['POSTGRESQL DATABASE WITH POSTGIS EXTENSION', 'GPKG', 'SQLITE']

SENSORTHINGS_STORAGE_TYPE = 'OGC SensorThings API'

if Qgis.QGIS_VERSION_INT > 33600:
SUPPORTED_STORAGE_TYPES.append(SENSORTHINGS_STORAGE_TYPE.upper())