feat(asset): move download_link property to an asset #2029#2091
Draft
pdavid-cssopra wants to merge 1 commit intodevelopfrom
Draft
feat(asset): move download_link property to an asset #2029#2091pdavid-cssopra wants to merge 1 commit intodevelopfrom
pdavid-cssopra wants to merge 1 commit intodevelopfrom
Conversation
pdavid-cssopra
force-pushed
the
feat-move-downloadlink-property-to-an-asset-2069
branch
from
86d88b2 to
5c91ba2
Compare
Contributor
Test Results 2 files - 2 2 suites - 2 35s ⏱️ - 2m 38s For more details on these failures, see this check. Results for commit 1a81d2b. ± Comparison against base commit 798a34a. ♻️ This comment has been updated with latest results. |
Collaborator
|
Code Coverage (Ubuntu)DetailsDiff against developResults for commit: 1a81d2b Minimum allowed coverage is ♻️ This comment has been updated with latest results |
pdavid-cssopra
force-pushed
the
feat-move-downloadlink-property-to-an-asset-2069
branch
from
5c91ba2 to
6061929
Compare
+ update/complete asset mapping behaviour + update provider configuration + update specific providers + centralize/deduplicate source
pdavid-cssopra
force-pushed
the
feat-move-downloadlink-property-to-an-asset-2069
branch
from
6061929 to
1a81d2b
Compare
sbrunato
requested changes
Apr 9, 2026
| self, collection: Optional[str] = None | ||
| ) -> dict[str, Union[str, list[str]]]: | ||
| """Get the plugin metadata mapping configuration (collection specific if exists) | ||
| :param collection: the desired collection |
Collaborator
There was a problem hiding this comment.
Suggested change
| :param collection: the desired collection | |
| :param collection: the desired collection |
| """ | ||
| metadata_mapping = getattr(self.config, "metadata_mapping", {}) | ||
| if isinstance(collection, str): | ||
| # Special overfload for collection config |
Collaborator
There was a problem hiding this comment.
Suggested change
| # Special overfload for collection config | |
| # Special overload for collection config |
| watchdog -= 1 | ||
| # Check if collection has a specific configuration | ||
| collection_config = self.config.products.get(target_collection, {}) | ||
| # Overload imported asset mapping configuration by current one |
Collaborator
There was a problem hiding this comment.
Suggested change
| # Overload imported asset mapping configuration by current one | |
| # Overload imported metadata mapping configuration by current one |
| collection_metadata_mapping = buffer | ||
| # This collection configuration can refer to another collection configuration | ||
| target_collection = collection_config.get( | ||
| "metadata_mapping_from_product", None |
Collaborator
There was a problem hiding this comment.
Suggested change
| "metadata_mapping_from_product", None | |
| "metadata_mapping_from_product" |
| :returns: The collection metadata assets-mapping | ||
| """ | ||
| metadata_mapping = getattr(self.config, "metadata_mapping", {}) | ||
| if isinstance(collection, str): |
Collaborator
There was a problem hiding this comment.
Suggested change
| if isinstance(collection, str): | |
| if collection is not None: |
avoids having to wonder if collection supports different types and if it needs to be handled
| assets_mapping = getattr(self.config, "assets_mapping", {}) | ||
| if isinstance(collection, str): | ||
| # Special overfload for collection config | ||
| # "metadata_mapping_from_product" overloaded by "current" |
Collaborator
There was a problem hiding this comment.
as metadata_mapping and assets_mapping are already distinct config entries, i'd use a new assets_mapping_from_product here
| ) -> dict[str, Union[str, list[str]]]: | ||
| """Get the plugin metadata mapping configuration (collection specific if exists) | ||
| :param collection: the desired collection | ||
| :returns: The collection metadata assets-mapping |
Collaborator
There was a problem hiding this comment.
Suggested change
| :returns: The collection metadata assets-mapping | |
| :returns: The collection specific metadata-mapping |
| ) | ||
| metadata_mapping.update(collection_metadata_mapping) | ||
| if watchdog == 0: | ||
| logger.warning("get_assets_mapping watchdog triggered") |
Collaborator
There was a problem hiding this comment.
Suggested change
| logger.warning("get_assets_mapping watchdog triggered") | |
| logger.warning("get_metadata_mapping watchdog triggered") |
| return queryables | ||
|
|
||
| def get_assets_from_mapping(self, provider_item: dict[str, Any]) -> dict[str, Any]: | ||
| def get_assets_from_mapping( |
Collaborator
There was a problem hiding this comment.
name is confusing with get_assets_mapping, please rename to build_assets_from_mapping or similar
| properties[field_name] = collection_properties[field_name] | ||
|
|
||
| # Gather asset mapping | ||
| assets_mapping = self.get_assets_mapping(collection) |
Collaborator
There was a problem hiding this comment.
do not rewrite a new metadata-mapping parser, use existing mechanisms:
assets_mapping = self.get_assets_mapping(collection)
assets = AssetsDict(product)
if not assets_mapping:
return assets
for key, mapping in assets_mapping.items():
asset_querypaths = mtd_cfg_as_conversion_and_querypath(mapping)
assets |= {key: properties_from_json(provider_item, asset_querypaths)}You will need to update roles in providers.yml:
from
roles:
- 'archive'
- 'data'to
roles: '["archive", "data"]'
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
From #1929
Task
move
eodag:download_linkproperty to an asset when it is downloadable (do not keep it has a fake-value and when product download was performed from other assets). If the download_link is not required, remove it and do not add it to assets.download_linkas keyarchive, dataas rolesWhen downloading a whole product:
archivein rolesarchivein roles, download all assetsNote:
hrefororder_linkWork on