feat: data-prep script improvements BM-1571 by ccbblin · Pull Request #98 · linz/topographic-system

ccbblin · 2026-03-23T19:55:18Z

Motivation

script improvements

Modifications

use a proper argument parser
split the parquet writer into its own folder as all of these should have similar arguments
plit the gpd.overlay into N threads as its currently just running on a single cpu

Verification

https://argo.linzaccess.com/workflows/argo/test-topographic-data-preparation-qdhc5?tab=workflow&uid=2f402e35-ae9e-4ab7-8771-ab0aba1d86d8

github-actions · 2026-03-23T19:59:33Z

Changes Summary

Total Features Changed: 0
Datasets Affected: 0
Git Diff Lines: 1

Kart Diff

Kart Diff (0 features)

Git Diff

Git Diff (1 lines)

packages/data-prep/src/data_prep/contour_with_landcover.py

ccbblin temporarily deployed to nonprod March 23, 2026 19:55 — with GitHub Actions Inactive

ccbblin changed the title ~~feat: data-prep script improvements~~ feat: data-prep script improvements BM-1571 Mar 23, 2026

ccbblin temporarily deployed to nonprod March 23, 2026 19:58 — with GitHub Actions Inactive

ccbblin had a problem deploying to nonprod March 23, 2026 19:58 — with GitHub Actions Failure

ccbblin temporarily deployed to nonprod March 24, 2026 19:46 — with GitHub Actions Inactive

ccbblin temporarily deployed to nonprod March 24, 2026 19:49 — with GitHub Actions Inactive

ccbblin temporarily deployed to nonprod March 24, 2026 19:52 — with GitHub Actions Inactive

ccbblin had a problem deploying to nonprod March 24, 2026 19:52 — with GitHub Actions Failure

ccbblin added the container build pull request container for testing label Mar 24, 2026

ccbblin temporarily deployed to nonprod March 24, 2026 20:31 — with GitHub Actions Inactive

ccbblin temporarily deployed to nonprod March 24, 2026 20:34 — with GitHub Actions Inactive

ccbblin had a problem deploying to nonprod March 24, 2026 20:34 — with GitHub Actions Failure

ccbblin temporarily deployed to nonprod March 26, 2026 19:55 — with GitHub Actions Inactive

ccbblin added container build pull request container for testing and removed container build pull request container for testing labels Mar 26, 2026

ccbblin had a problem deploying to nonprod March 26, 2026 19:59 — with GitHub Actions Failure

ccbblin temporarily deployed to nonprod March 26, 2026 19:59 — with GitHub Actions Inactive

ccbblin temporarily deployed to nonprod March 26, 2026 20:07 — with GitHub Actions Inactive

ccbblin added container build pull request container for testing and removed container build pull request container for testing labels Mar 26, 2026

ccbblin had a problem deploying to nonprod March 26, 2026 20:10 — with GitHub Actions Failure

ccbblin temporarily deployed to nonprod March 26, 2026 20:10 — with GitHub Actions Inactive

feat: data-prep script improvements

3cb810a

ccbblin temporarily deployed to nonprod March 29, 2026 22:08 — with GitHub Actions Inactive

ccbblin added container build pull request container for testing and removed container build pull request container for testing labels Mar 29, 2026

ccbblin had a problem deploying to nonprod March 29, 2026 22:11 — with GitHub Actions Failure

ccbblin temporarily deployed to nonprod March 29, 2026 22:11 — with GitHub Actions Inactive

tawera-manaena previously approved these changes Mar 29, 2026

View reviewed changes

schmidtnz reviewed Mar 29, 2026

View reviewed changes

packages/data-prep/src/data_prep/contour_with_landcover.py Outdated Show resolved Hide resolved

make sure contour uses geometry instead of geom

4a61170

ccbblin dismissed tawera-manaena’s stale review via 4a61170 March 30, 2026 00:31

ccbblin temporarily deployed to nonprod March 30, 2026 00:31 — with GitHub Actions Inactive

ccbblin had a problem deploying to nonprod March 30, 2026 00:34 — with GitHub Actions Failure

ccbblin temporarily deployed to nonprod March 30, 2026 00:34 — with GitHub Actions Inactive

fix test

bc153a4

ccbblin temporarily deployed to nonprod March 30, 2026 00:40 — with GitHub Actions Inactive

ccbblin temporarily deployed to nonprod March 30, 2026 00:43 — with GitHub Actions Inactive

ccbblin had a problem deploying to nonprod March 30, 2026 00:43 — with GitHub Actions Failure

format

815ccbe

ccbblin temporarily deployed to nonprod March 30, 2026 00:45 — with GitHub Actions Inactive

ccbblin had a problem deploying to nonprod March 30, 2026 00:48 — with GitHub Actions Failure

ccbblin temporarily deployed to nonprod March 30, 2026 00:48 — with GitHub Actions Inactive

ccbblin added container build pull request container for testing and removed container build pull request container for testing labels Mar 30, 2026

ccbblin requested a review from schmidtnz March 30, 2026 20:14

schmidtnz approved these changes Mar 30, 2026

View reviewed changes

ccbblin added this pull request to the merge queue Mar 30, 2026

linz-li-bot mentioned this pull request Mar 30, 2026

chore: release master #103

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: data-prep script improvements BM-1571#98

feat: data-prep script improvements BM-1571#98
ccbblin merged 11 commits intomasterfrom
feat/data-prep-script-improvements

ccbblin commented Mar 23, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

Conversation

ccbblin commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Verification

Uh oh!

github-actions bot commented Mar 23, 2026

Changes Summary

Kart Diff

Git Diff

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

ccbblin commented Mar 23, 2026 •

edited

Loading