Add offline map tile prefetch tooling#18
Open
brothercorvo wants to merge 2 commits intotorlando-tech:mainfrom
Open
Add offline map tile prefetch tooling#18brothercorvo wants to merge 2 commits intotorlando-tech:mainfrom
brothercorvo wants to merge 2 commits intotorlando-tech:mainfrom
Conversation
Contributor
Greptile SummaryAdds a Python script (
Confidence Score: 2/5
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Parse CLI args] --> B[Compute bounding box from center + radius]
B --> C[For each zoom level z8..z14]
C --> D[Enumerate tiles intersecting radius]
D --> E{Dry run?}
E -- Yes --> F[Print tile counts and exit]
E -- No --> G[For each tile x,y]
G --> H{File exists locally?}
H -- Yes --> I[Skip]
H -- No --> J[Build URL from template]
J --> K[Download tile with retries]
K --> L{Success?}
L -- Yes --> M[Write to tiles/z/x/y.png]
L -- No --> N[Log failure to stderr]
M --> O[Sleep delay]
N --> O
O --> G
I --> G
Prompt To Fix All With AIThis is a comment left during a code review.
Path: tools/prefetch_map_tiles.py
Line: 36
Comment:
**Violates OpenStreetMap tile usage policy**
The default `--url-template` points to `tile.openstreetmap.org`, which [explicitly prohibits](https://operations.osmfoundation.org/policies/tiles/) bulk downloading and offline prefetching:
> **You must not:** Bulk download ("scrape") tiles or offer prefetch features.
>
> **Offline use is not permitted** on `tile.openstreetmap.org`. Features such as "Download city/country for offline use" [...] Note: If you require offline maps, use self-hosted tiles or a provider that explicitly allows offline/prefetching.
At the default settings (zoom 8–14, 200 km radius), this script downloads ~55,000 tiles in a single run, which will very likely get the user's IP blocked.
Consider changing the default URL template to a placeholder or a self-hosted tile server, and add a prominent warning in both the script's `--help` output and the README that users must use a tile provider that permits bulk/offline downloads (e.g., a self-hosted tile server, Thunderforest with an appropriate plan, or MapTiler).
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: prefetch_map_tiles.stderr.log
Line: 1-16
Comment:
**Log files should not be committed**
These log files (`prefetch_map_tiles.stderr.log` and `prefetch_map_tiles.stdout.log`) are runtime output artifacts from a test run and should not be tracked in the repository. Consider removing them from the commit and adding `*.log` or these specific filenames to `.gitignore`.
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: tools/prefetch_map_tiles.py
Line: 187-202
Comment:
**Antimeridian wrapping not handled**
If the bounding box wraps around the antimeridian (e.g., center near lon=179° with a large radius), `bounding_box()` returns `west > east` after normalization (e.g., `west=170`, `east=-170`). In that case, `latlon_to_tile(north, west)` produces a high `x0` and `latlon_to_tile(south, east)` produces a low `x1`. The subsequent `min(x0, x1)` / `max(x0, x1)` logic would then scan nearly the entire x-range of the globe rather than the two small strips near the antimeridian.
This isn't a problem for the current default coordinates (eastern Canada), but it will produce incorrect results for any user who specifies a center point near the dateline. If this is a known limitation, it's worth documenting; otherwise, the function needs to detect the wrap and split the x-range into two segments.
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: tools/README.md
Line: 53-54
Comment:
**Missing OSM tile usage policy warning**
The README should note that the default tile server (`tile.openstreetmap.org`) prohibits bulk downloading and offline prefetching per the [OSM Tile Usage Policy](https://operations.osmfoundation.org/policies/tiles/). Users should be directed to use a self-hosted tile server or a provider that explicitly permits offline/bulk downloads.
How can I resolve this? If you propose a fix, please make it concise.Last reviewed commit: 3e336a4 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| DEFAULT_RADIUS_KM = 200.0 | ||
| DEFAULT_MIN_ZOOM = 8 | ||
| DEFAULT_MAX_ZOOM = 14 | ||
| DEFAULT_URL_TEMPLATE = "https://tile.openstreetmap.org/{z}/{x}/{y}.png" |
Contributor
There was a problem hiding this comment.
Violates OpenStreetMap tile usage policy
The default --url-template points to tile.openstreetmap.org, which explicitly prohibits bulk downloading and offline prefetching:
You must not: Bulk download ("scrape") tiles or offer prefetch features.
Offline use is not permitted on
tile.openstreetmap.org. Features such as "Download city/country for offline use" [...] Note: If you require offline maps, use self-hosted tiles or a provider that explicitly allows offline/prefetching.
At the default settings (zoom 8–14, 200 km radius), this script downloads ~55,000 tiles in a single run, which will very likely get the user's IP blocked.
Consider changing the default URL template to a placeholder or a self-hosted tile server, and add a prominent warning in both the script's --help output and the README that users must use a tile provider that permits bulk/offline downloads (e.g., a self-hosted tile server, Thunderforest with an appropriate plan, or MapTiler).
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/prefetch_map_tiles.py
Line: 36
Comment:
**Violates OpenStreetMap tile usage policy**
The default `--url-template` points to `tile.openstreetmap.org`, which [explicitly prohibits](https://operations.osmfoundation.org/policies/tiles/) bulk downloading and offline prefetching:
> **You must not:** Bulk download ("scrape") tiles or offer prefetch features.
>
> **Offline use is not permitted** on `tile.openstreetmap.org`. Features such as "Download city/country for offline use" [...] Note: If you require offline maps, use self-hosted tiles or a provider that explicitly allows offline/prefetching.
At the default settings (zoom 8–14, 200 km radius), this script downloads ~55,000 tiles in a single run, which will very likely get the user's IP blocked.
Consider changing the default URL template to a placeholder or a self-hosted tile server, and add a prominent warning in both the script's `--help` output and the README that users must use a tile provider that permits bulk/offline downloads (e.g., a self-hosted tile server, Thunderforest with an appropriate plan, or MapTiler).
How can I resolve this? If you propose a fix, please make it concise.
Comment on lines
+1
to
+16
| FAIL https://tile.openstreetmap.org/13/2552/2984.png -> tiles\13\2552\2984.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2552/2985.png -> tiles\13\2552\2985.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2552/2986.png -> tiles\13\2552\2986.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2552/2987.png -> tiles\13\2552\2987.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2552/2988.png -> tiles\13\2552\2988.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2552/2989.png -> tiles\13\2552\2989.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2552/2990.png -> tiles\13\2552\2990.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2552/2991.png -> tiles\13\2552\2991.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2553/2984.png -> tiles\13\2553\2984.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2553/2985.png -> tiles\13\2553\2985.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2553/2986.png -> tiles\13\2553\2986.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2553/2987.png -> tiles\13\2553\2987.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2553/2988.png -> tiles\13\2553\2988.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2553/2989.png -> tiles\13\2553\2989.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2553/2990.png -> tiles\13\2553\2990.png (HTTP Error 404: Not Found) | ||
| FAIL https://tile.openstreetmap.org/13/2553/2991.png -> tiles\13\2553\2991.png (HTTP Error 404: Not Found) |
Contributor
There was a problem hiding this comment.
Log files should not be committed
These log files (prefetch_map_tiles.stderr.log and prefetch_map_tiles.stdout.log) are runtime output artifacts from a test run and should not be tracked in the repository. Consider removing them from the commit and adding *.log or these specific filenames to .gitignore.
Prompt To Fix With AI
This is a comment left during a code review.
Path: prefetch_map_tiles.stderr.log
Line: 1-16
Comment:
**Log files should not be committed**
These log files (`prefetch_map_tiles.stderr.log` and `prefetch_map_tiles.stdout.log`) are runtime output artifacts from a test run and should not be tracked in the repository. Consider removing them from the commit and adding `*.log` or these specific filenames to `.gitignore`.
How can I resolve this? If you propose a fix, please make it concise.
Comment on lines
+187
to
+202
| def enumerate_tiles(center_lat: float, center_lon: float, radius_km: float, zoom: int) -> list[tuple[int, int]]: | ||
| south, west, north, east = bounding_box(center_lat, center_lon, radius_km) | ||
| x0, y1 = latlon_to_tile(north, west, zoom) | ||
| x1, y0 = latlon_to_tile(south, east, zoom) | ||
| max_index = (1 << zoom) - 1 | ||
| x_start = max(0, min(x0, x1)) | ||
| x_end = min(max_index, max(x0, x1)) | ||
| y_start = max(0, min(y0, y1)) | ||
| y_end = min(max_index, max(y0, y1)) | ||
|
|
||
| tiles: list[tuple[int, int]] = [] | ||
| for tile_x in range(x_start, x_end + 1): | ||
| for tile_y in range(y_start, y_end + 1): | ||
| if tile_intersects_radius(center_lat, center_lon, radius_km, tile_x, tile_y, zoom): | ||
| tiles.append((tile_x, tile_y)) | ||
| return tiles |
Contributor
There was a problem hiding this comment.
Antimeridian wrapping not handled
If the bounding box wraps around the antimeridian (e.g., center near lon=179° with a large radius), bounding_box() returns west > east after normalization (e.g., west=170, east=-170). In that case, latlon_to_tile(north, west) produces a high x0 and latlon_to_tile(south, east) produces a low x1. The subsequent min(x0, x1) / max(x0, x1) logic would then scan nearly the entire x-range of the globe rather than the two small strips near the antimeridian.
This isn't a problem for the current default coordinates (eastern Canada), but it will produce incorrect results for any user who specifies a center point near the dateline. If this is a known limitation, it's worth documenting; otherwise, the function needs to detect the wrap and split the x-range into two segments.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/prefetch_map_tiles.py
Line: 187-202
Comment:
**Antimeridian wrapping not handled**
If the bounding box wraps around the antimeridian (e.g., center near lon=179° with a large radius), `bounding_box()` returns `west > east` after normalization (e.g., `west=170`, `east=-170`). In that case, `latlon_to_tile(north, west)` produces a high `x0` and `latlon_to_tile(south, east)` produces a low `x1`. The subsequent `min(x0, x1)` / `max(x0, x1)` logic would then scan nearly the entire x-range of the globe rather than the two small strips near the antimeridian.
This isn't a problem for the current default coordinates (eastern Canada), but it will produce incorrect results for any user who specifies a center point near the dateline. If this is a known limitation, it's worth documenting; otherwise, the function needs to detect the wrap and split the x-range into two segments.
How can I resolve this? If you propose a fix, please make it concise.
Comment on lines
+53
to
+54
| - The script defaults to the same OpenStreetMap tile URL currently used by the firmware. | ||
| - If you use a different tile provider, pass `--url-template`. |
Contributor
There was a problem hiding this comment.
Missing OSM tile usage policy warning
The README should note that the default tile server (tile.openstreetmap.org) prohibits bulk downloading and offline prefetching per the OSM Tile Usage Policy. Users should be directed to use a self-hosted tile server or a provider that explicitly permits offline/bulk downloads.
Prompt To Fix With AI
This is a comment left during a code review.
Path: tools/README.md
Line: 53-54
Comment:
**Missing OSM tile usage policy warning**
The README should note that the default tile server (`tile.openstreetmap.org`) prohibits bulk downloading and offline prefetching per the [OSM Tile Usage Policy](https://operations.osmfoundation.org/policies/tiles/). Users should be directed to use a self-hosted tile server or a provider that explicitly permits offline/bulk downloads.
How can I resolve this? If you propose a fix, please make it concise.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Notes
Testing