Skip to content

grass.experimental: Add object to access tools as functions #2923

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 56 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
d905882
grass.experimental: Add object to access modules as functions
wenzeslaus Apr 18, 2023
aaef183
Support verbosity, overwrite and region freezing
wenzeslaus Apr 21, 2023
54db575
Raise exception instead of calling handle_errors
wenzeslaus Apr 22, 2023
82f5894
Allow to specify stdin and use a new instance of Tools itself to exec…
wenzeslaus Apr 22, 2023
0f1e210
Add ignore errors, r_mapcalc example, draft tests
wenzeslaus Apr 22, 2023
f4e3fed
Add test for exceptions
wenzeslaus Apr 24, 2023
04087e8
Add tests and Makefile
wenzeslaus May 4, 2023
6ab8e40
Convert values to ints and floats in keyval
wenzeslaus May 4, 2023
744cfac
Do not overwrite by default to follow default behavior in GRASS GIS
wenzeslaus May 4, 2023
24c27e6
Add doc, remove old code and todos
wenzeslaus Jun 3, 2023
ff187a6
Add to top Makefile
wenzeslaus Jun 3, 2023
22773c8
Add docs for tests
wenzeslaus Jun 3, 2023
2911065
Allow test to fail because of the missing seed parameter (so results …
wenzeslaus Jun 4, 2023
3ac46c3
Merge branch 'main' into add-session-tools-object
echoix Nov 11, 2024
437d46e
Allow for optional output capture (error handling and printing still …
wenzeslaus Apr 23, 2025
cb8f483
Merge branch 'main' into add-session-tools-object
wenzeslaus Apr 23, 2025
a958142
Merge remote-tracking branch 'upstream/main' into add-session-tools-o…
wenzeslaus Apr 25, 2025
61972d4
Access JSON as dict directly without an attribute using getitem. Sugg…
wenzeslaus Apr 25, 2025
c86d8ff
Fix whitespace and regexp
wenzeslaus Apr 25, 2025
3b995c9
Represent not captured stdout as None, not empty string.
wenzeslaus Apr 25, 2025
d8c354d
Merge remote-tracking branch 'upstream/main' into add-session-tools-o…
wenzeslaus Apr 29, 2025
4cc5a32
Add run subcommand to have a CLI use case for the tools. It runs one …
wenzeslaus Apr 29, 2025
459b2ad
Update function name
wenzeslaus Apr 30, 2025
513c9f8
Add prototype code for numpy support
wenzeslaus Jun 2, 2025
24ef6b9
Merge main branch
wenzeslaus Jun 2, 2025
4a1e374
Make the special features standalone objects used by composition
wenzeslaus Jun 11, 2025
651df11
Merge remote-tracking branch 'upstream/main' into add-session-tools-o…
wenzeslaus Jun 11, 2025
15aa936
Remove NumPy and pack file code
wenzeslaus Jun 11, 2025
5e9b50d
Capture the name of a subcommand to fix the tests
wenzeslaus Jun 11, 2025
2acdde0
Don't worry about specific r.univar output value for mean, just test …
wenzeslaus Jun 11, 2025
8f80eb9
Use the special grass.script sauce to run subprocess to have it worki…
wenzeslaus Jun 11, 2025
a422356
Use low level run functions internally, align result with subprocess.…
wenzeslaus Jun 12, 2025
8f1e8ff
Use tools API also for help in the CLI, support other special flags, …
wenzeslaus Jun 12, 2025
2085b94
Provide dir(tools) functionality
wenzeslaus Jun 12, 2025
d341183
Support prefixes in attribute access and error messages. Also improve…
wenzeslaus Jun 13, 2025
4f86be4
Merge remote-tracking branch 'upstream/main' into add-session-tools-o…
wenzeslaus Jun 13, 2025
d8f6c40
Improve error message without tool suggestions (still include tool na…
wenzeslaus Jun 13, 2025
ac98322
Remove prefixes
wenzeslaus Jun 13, 2025
3abc34b
Clean up error handling, remove naive run_command family wrappers
wenzeslaus Jun 16, 2025
816b215
Add test with corresponding run_command family examples except feed, …
wenzeslaus Jun 17, 2025
e95bb51
Merge remote-tracking branch 'upstream/main' into add-session-tools-o…
wenzeslaus Jun 17, 2025
3c99498
Use the no-copy approach for env taked also elsewhere like init which…
wenzeslaus Jun 17, 2025
14be050
Remove the feed_input_to method and stdin from the constructor. Inste…
wenzeslaus Jun 17, 2025
f9f7c0a
Add support for len on the result
wenzeslaus Jun 17, 2025
3a45c00
Make the file layout what it can be if not experimental, and make it …
wenzeslaus Jun 18, 2025
5658d3e
Disable Pylint warning because Pylint does not see the dynamic attrib…
wenzeslaus Jun 20, 2025
549cf1a
Merge remote-tracking branch 'upstream/main' into add-session-tools-o…
wenzeslaus Jun 20, 2025
bbd80a5
Use create_project
wenzeslaus Jun 20, 2025
ff17612
Merge to update handle_errors
wenzeslaus Jun 24, 2025
7754592
Add doc for the main run function
wenzeslaus Jun 25, 2025
0587621
Merge remote-tracking branch 'upstream/main' into add-session-tools-o…
wenzeslaus Jun 25, 2025
1345f79
Use input for stdin aligning with subprocess.run and Popen.communicat…
wenzeslaus Jun 25, 2025
11d7bb0
Use also command in result if kwargs are not available. Do not use se…
wenzeslaus Jun 25, 2025
35a785b
More migration examples. More examples with context managers.
wenzeslaus Jun 25, 2025
2176395
Merge remote-tracking branch 'upstream/main' into add-session-tools-o…
wenzeslaus Jun 25, 2025
acff8ce
Add g.list examples
wenzeslaus Jun 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 58 additions & 2 deletions python/grass/app/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,47 @@
import tempfile
import os
import sys
import subprocess
from pathlib import Path


import grass.script as gs
from grass.app.data import lock_mapset, unlock_mapset, MapsetLockingException
from grass.experimental.tools import Tools

# Special flags supported besides help and --json which does not need special handling:
SPECIAL_FLAGS = [
"--interface-description",
"--md-description",
"--wps-process-description",
"--script",
]
# To make this list shorter, we don't support outdated special flags:
# --help-text --html-description --rst-description


def subcommand_run_tool(args, tool_args: list, help: bool):
command = [args.tool, *tool_args]
with tempfile.TemporaryDirectory() as tmp_dir_name:
project_name = "project"
project_path = Path(tmp_dir_name) / project_name
gs.create_project(project_path)
with gs.setup.init(project_path) as session:
tools = Tools(session=session, capture_output=False)
try:
if help:
# We consumed the help flag, so we need to add it explicitly.
tools.no_nonsense_run_from_list([*command, "--help"])
elif any(item in command for item in SPECIAL_FLAGS):
# This is here basically because of how --json behaves,
# two JSON flags are accepted, but --json currently overridden by
# other special flags, so later use of --json in tools will fail
# with the other flags active.
tools.no_nonsense_run_from_list(command)
else:
tools.run_from_list(command)
except subprocess.CalledProcessError as error:
return error.returncode


def subcommand_lock_mapset(args):
Expand Down Expand Up @@ -73,10 +110,22 @@ def main(args=None, program=None):
description="Experimental low-level CLI interface to GRASS. Consult developers before using it.",
prog=program,
)
subparsers = parser.add_subparsers(title="subcommands", required=True)
subparsers = parser.add_subparsers(
title="subcommands", dest="subcommand", required=True
)

# Subcommand parsers

run_subparser = subparsers.add_parser(
"run",
help="run a tool",
add_help=False,
epilog="Tool name is followed by it parameters.",
)
run_subparser.add_argument("tool", type=str, nargs="?", help="name of a tool")
run_subparser.add_argument("--help", action="store_true")
run_subparser.set_defaults(func=subcommand_run_tool)

subparser = subparsers.add_parser("lock", help="lock a mapset")
subparser.add_argument("mapset_path", type=str)
subparser.add_argument(
Expand Down Expand Up @@ -120,5 +169,12 @@ def main(args=None, program=None):
subparser.add_argument("page", type=str)
subparser.set_defaults(func=subcommand_show_man)

parsed_args = parser.parse_args(args)
# Parsing
parsed_args, other_args = parser.parse_known_args(args)
# Standard help already exited, but we need to handle tools separately.
if parsed_args.subcommand == "run":
if parsed_args.tool is None and parsed_args.help:
run_subparser.print_help()
return 0
return parsed_args.func(parsed_args, other_args, help=parsed_args.help)
return parsed_args.func(parsed_args)
7 changes: 5 additions & 2 deletions python/grass/experimental/Makefile
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
MODULE_TOPDIR = ../../..

include $(MODULE_TOPDIR)/include/Make/Other.make
include $(MODULE_TOPDIR)/include/Make/Python.make
include $(MODULE_TOPDIR)/include/Make/Dir.make

DSTDIR = $(ETC)/python/grass/experimental

SUBDIRS = \
tools

MODULES = \
create \
mapset

PYFILES := $(patsubst %,$(DSTDIR)/%.py,$(MODULES) __init__)
PYCFILES := $(patsubst %,$(DSTDIR)/%.pyc,$(MODULES) __init__)

default: $(PYFILES) $(PYCFILES)
default: subdirs $(PYFILES) $(PYCFILES)

$(DSTDIR):
$(MKDIR) $@
Expand Down
40 changes: 39 additions & 1 deletion python/grass/experimental/tests/conftest.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Fixtures for grass.script"""
"""Fixtures for grass.experimental"""

import uuid
import os
Expand Down Expand Up @@ -69,3 +69,41 @@ def xy_mapset_non_permament(xy_session): # pylint: disable=redefined-outer-name
"test1", create=True, env=xy_session.env
) as session:
yield session


@pytest.fixture
def rows_raster_file3x3(tmp_path):
project = tmp_path / "xy_test3x3"
gs.create_project(project)
with gs.setup.init(project, env=os.environ.copy()) as session:
gs.run_command("g.region", rows=3, cols=3, env=session.env)
gs.mapcalc("rows = row()", env=session.env)
output_file = tmp_path / "rows3x3.grass_raster"
gs.run_command(
"r.pack",
input="rows",
output=output_file,
flags="c",
superquiet=True,
env=session.env,
)
return output_file


@pytest.fixture
def rows_raster_file4x5(tmp_path):
project = tmp_path / "xy_test4x5"
gs.create_project(project)
with gs.setup.init(project, env=os.environ.copy()) as session:
gs.run_command("g.region", rows=4, cols=5, env=session.env)
gs.mapcalc("rows = row()", env=session.env)
output_file = tmp_path / "rows4x5.grass_raster"
gs.run_command(
"r.pack",
input="rows",
output=output_file,
flags="c",
superquiet=True,
env=session.env,
)
return output_file
21 changes: 21 additions & 0 deletions python/grass/experimental/tools/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MODULE_TOPDIR = ../../../..

include $(MODULE_TOPDIR)/include/Make/Other.make
include $(MODULE_TOPDIR)/include/Make/Python.make

DSTDIR = $(ETC)/python/grass/experimental/tools

MODULES = \
session_tools \
support

PYFILES := $(patsubst %,$(DSTDIR)/%.py,$(MODULES) __init__)
PYCFILES := $(patsubst %,$(DSTDIR)/%.pyc,$(MODULES) __init__)

default: $(PYFILES) $(PYCFILES)

$(DSTDIR):
$(MKDIR) $@

$(DSTDIR)/%: % | $(DSTDIR)
$(INSTALL_DATA) $< $@
10 changes: 10 additions & 0 deletions python/grass/experimental/tools/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
def __getattr__(name):
if name == "Tools":
from .session_tools import Tools

return Tools
msg = f"module {__name__} has no attribute {name}"
raise AttributeError(msg)


__all__ = ["Tools"] # pylint: disable=undefined-all-variable
Loading
Loading