Feature/#451 customizable registry url #781
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… when pulling from registry cache)
This builds on Manfred Martin's <[email protected]> original implementation from PR getwud#454 (which was closed without merging). His work introduced the `wud.registry.lookup.url` label to help users bypass Docker registry caches when checking for updates. This commit merges the original feature branch with version 8.1.1 to bring it up to date with the latest stable release, ensuring compatibility with recent changes. However, while testing the original URL-based approach with my Harbor registry setup, I discovered a fundamental issue: it doesn't work with registries that use project prefixes or nested paths. The problem: The original implementation accepted registry URLs like "https://registry-1.docker.io" and tried to extract the image name from them. This breaks with enterprise registries like Harbor, where images look like: harbor.example.com/dockerhub-proxy/traefik:v3.5.3 The "dockerhub-proxy/" part is a Harbor project name, not part of the actual image name. The URL-based approach would incorrectly include it as part of the image path, causing registry matching to fail. This same problem affects other registries that use hierarchical project structures. The solution: Instead of accepting registry URLs and trying to parse them, accept full image paths directly. While this requires typing the full image path (which may be slightly longer than just a URL), it's more explicit and generic, works with any registry structure, and lets users specify exactly what upstream image WUD should check for updates. What changed: - Renamed label from `wud.registry.lookup.url` to `wud.registry.lookup.image` - Now accepts full image paths (e.g., "library/traefik" or "ghcr.io/user/image") - Automatically defaults to Docker Hub for short-form images without a registry - Fixed registry normalization to use the lookup image instead of the original - Updated documentation - Added Prometheus metric support for the lookup image field - Removed tests specific to URL-based parsing (no longer needed with generic image path approach) Example usage: image: harbor.example.com/dockerhub-proxy/traefik:v3.5.3 labels: - wud.watch=true - wud.registry.lookup.image=library/traefik This allows WUD to check Docker Hub for traefik updates while the container continues pulling from the Harbor proxy cache, solving the original issue where proxy registries don't have up-to-date tag information. Tested with: - Harbor proxy cache (Docker Hub, GHCR, Codeberg) - All 428 unit tests passing - Live deployment on my homelab
|
Can I download this feature as a Docker image somewhere to test it with my local Docker Registry (behind a reverse proxy)? |
Contributor
Author
|
Hi @EdwinKM , It seems like there are no automated workflows in this repository to build Docker images for feature branches, so you would either need to wait until
|
|
Does not seem to work. Probably a mistake on my side. Steps below: |
Contributor
Author
|
The issue is that this PR introduces a breaking change (only a breaking change incase you have used PR #454 before) in the label name: Old: wud.registry.lookup.url You'll need to update your compose files to use the new label name. Sorry for the breaking change! Since PR #454 was never merged into a release, I figured no one in production would be affected. I should probably consider just renaming everything back into "wud.registry.lookup.url" to not confuse anyone. |
|
no, i was expected the breaking change. |
|
validated even with inspect: only "wud.registry.lookup.image" is present. |
Contributor
Author
|
I think i might have just found the issue and am trying to fix it :) Your store contains containers with the old image.registry.lookupUrl field from the previous version of PR #454, but the new schema only allows lookupImage. When the app starts and tries to validate containers from the store, it fails as there is no migration logic that handled the schema change. It slipped past testing as i did not previously have containers with the old image.registry.lookupUrl field. The tests did not cover that. Sorry for the inconvenience :) I will push a new commit soon that just renames wud.registry.lookup.image back into wud.registry.lookup.url to avoid breaking changes while migration from the old feature branch to the new one. |
Changed the label from wud.registry.lookup.image back to wud.registry.lookup.url to maintain backward compatibility with users already using the previous PR getwud#454 implementation. This prevents a breaking change for existing users like Edwin who were using the old label name. Changes: - app/model/container.js: Updated schema to use lookupUrl instead of lookupImage - app/watchers/providers/docker/Docker.js: Updated all references to use lookupUrl - app/watchers/providers/docker/label.js: Changed constant from wudRegistryLookupImage to wudRegistryLookupUrl - ui/src/components/ContainerImage.vue: Updated UI to display lookupUrl - docs/configuration/watchers/README.md: Updated documentation and examples
|
i am the only one running that feature. I would not suggest to use illogical label names for this one case. |
This reverts commit 6506dae.
|
i moved wud.json out of the way. It now starts, get some weird logging. But not sure if new anyway. |
|
please revert to "image". It is the logical name |
Contributor
Author
|
@EdwinKM hmm you are probably right. I just already pushed an commit that would fix any issues that users experience while migrating from PR #454 (It is an issue only specific to that scenario). I believe that you have a good point and i therefore reverted my latest commit back to : wud.registry.lookup.image is more logical than wud.registry.lookup.url because:
If i had more time today i could maybe implement automatic migration in the WUD store so that there are no issues in that specific scenario, but i need to study for an exam so here is the solution in the meantime:
|
Contributor
Author
|
oops you were faster than i could write my comment :) |
Contributor
Author
|
Your ""No Registry Provider found" errors" in the logs seem to be fine as they should just be telling you that you did not add the wud.registry.lookup.image label to tell WUD where to check for updates yet and WUD is still looking at your local registry cache. The "DEBUG whats-up-docker/watcher.docker.local: Unsupported Registry unknown" logs belong to that too. I should probably remove those before the PR is merged since they're just for debugging. |
Contributor
Author
|
Please also keep in mind that the PR #454 you were using is quite old. This PR merges the original feature branch with version 8.1.1 to bring it up to date with the latest stable release. |
Contributor
Author
concerning the other logs: The "Unsupported Registry unknown" errors for syncthing, share-gitea, gitea, and lms-en-media are happening because these containers are pulled from your local registry cache and WUD doesn't know where to check for updates so this is exactly what we can solve :) . You just need to add the wud.registry.lookup.image label to tell WUD the upstream image. |
Contributor
Author
|
@EdwinKM sry for the debugging inconvenience :) Does it seem to work now after just letting WUD recreate a new wud.json store file? |
i think so, thanks for all your work! I am toying a bit with WUD again. Did not touch it for a long time. Back to your studies! Will add my results in the next days... |
|
Most containers converter. Docker images working fine. 1 ghcr.io image validated. |
|
I think it is ready to merge. User @Stitch10925 hoped for a easier solution. But i do not mind, for WUD we usually already need to add lines to the compose file anyway. |
Contributor
Author
|
@EdwinKM @Stitch10925 That's true, but I couldn't think of a better solution yet 😅 I do not know what still needs to be done until this can be merged. I guess that we just need to wait until @fmartinou reviews the PR? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improve registry lookup feature with full image path support
This builds on @manfred-martin 's original implementation from PR #454 (which was closed without merging).
His work introduced the
wud.registry.lookup.urllabel to help users bypass Docker registry caches when checking for updates.Closes #451
Background
Since PR #454 was created, the codebase has moved forward a few versions. This PR merges the original feature branch with version 8.1.1 to bring it up to date with the latest stable release.
However, while testing the original URL-based approach with my Harbor registry setup, I discovered a fundamental issue for my usecase: it doesn't work with registries that use project prefixes or nested paths.
The Problem
The original implementation accepted registry URLs like
https://registry-1.docker.ioand tried to extract the image name from them. This breaks with registries like Harbor, where images look like:The
dockerhub-proxy/part is a Harbor project name, not part of the actual image name.The URL-based approach would incorrectly include it as part of the image path, causing registry matching to fail. This same problem affects other registries that use hierarchical project structures.
My Solution and Changes
Instead of accepting registry URLs and trying to parse them, accept full image paths directly. While this requires typing the full image path (which may be slightly longer than just a URL), it's more explicit and generic, works with any registry structure, and lets users specify exactly what upstream image WUD should check for updates.
What Changed
wud.registry.lookup.urltowud.registry.lookup.imagelibrary/traefikorghcr.io/user/image)Example Usage
This allows WUD to check registries like Docker Hub for traefik updates while the container continues pulling from the Harbor proxy cache, solving the original issue where proxy registries don't have up-to-date tag information.
Testing
Breaking Changes
This changes the label name from
wud.registry.lookup.urltowud.registry.lookup.image. However, since PR #454 was never merged into a release, this shouldn't affect any users that are not using the feature/#451_customizable_registry_url branch.