Ensure package_create resources trigger XLoader submission for datastore and views#265
Ensure package_create resources trigger XLoader submission for datastore and views#265TomeCirun wants to merge 3 commits intockan:masterfrom
Conversation
| "resource did not pass validation yet.", resource_dict.get('id')) | ||
| return | ||
| # Only submit if URL has changed | ||
| if not getattr(entity, 'url_changed', False): |
There was a problem hiding this comment.
What about changing the Data Dictionary? Or the resource format? Or removing a schema?
There was a problem hiding this comment.
Hi @ThrawnCA , thanks for taking the time to review this PR — I really appreciate it. Could you please clarify what the expected output should be? Just to note, this line was copied from the previous logic, so it remains as originally implemented.
There was a problem hiding this comment.
Hmm. You're right that the logic did include this before. But are you also going to replace after_resource_update, which doesn't include that restriction and applies XLoader to any update?
There was a problem hiding this comment.
I forgot about that — sorry, and thanks for pointing it out! I’ll update the PR soon.
There was a problem hiding this comment.
Hi @ThrawnCA
I moved the logic into a separate function sync_datastore_flag and we now call it inside this check:
if not getattr(entity, 'url_changed', False):This reduces redundant calls. If the URL has changed, _submit_to_xloader will set the flag on the resource. I also removed the after_resource_update hook.
Please let me know if there’s anything else I should address.
There was a problem hiding this comment.
There are other fields that should potentially trigger XLoader on change, such as the resource format.
When we harvest a dataset, we call the
create_or_update_packagefunction or usepackage_createwith resources. All resources are included inside package_dict, so they aren’t created separately —resource_createis never called. As a result, the xloader hook isn’t triggered, resources aren’t pushed to the datastore, and their default views aren’t created.To resolve this, I removed the after_resource_create hook and now handle everything within notify.