Skip to content

Conversation

@shuchenliu
Copy link
Contributor

This PR resolves the Elasticsearch portion of #64.

1. Adapt retriever to use metadata stored in Elasticsearch

As metadata is now extracted by the upstream pipeline and indexed alongside the data mappings, the tier1 driver now includes logic to fetch and cache this metadata directly from Elasticsearch.

2. Generate operations from per-dataset metadata

Previously, tier1 relied on tier0’s unified metadata to derive the available operations. Now, metadata is retrieved and parsed per datasource, and the final operation set is constructed by combining these individual definitions.

@shuchenliu shuchenliu requested a review from tokebe December 12, 2025 05:57
@sentry
Copy link

sentry bot commented Dec 12, 2025

Codecov Report

❌ Patch coverage is 28.65497% with 122 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
.../retriever/data_tiers/tier_1/elasticsearch/meta.py 22.31% 94 Missing ⚠️
...etriever/data_tiers/tier_1/elasticsearch/driver.py 39.13% 14 Missing ⚠️
src/retriever/data_tiers/utils.py 26.31% 14 Missing ⚠️
Files with missing lines Coverage Δ
.../elasticsearch/constraints/attributes/attribute.py 52.17% <100.00%> (ø)
.../elasticsearch/constraints/attributes/meta_info.py 100.00% <100.00%> (ø)
...ch/constraints/attributes/ops/handle_comparison.py 75.00% <100.00%> (ø)
...csearch/constraints/attributes/ops/handle_match.py 87.80% <100.00%> (ø)
.../elasticsearch/constraints/qualifiers/qualifier.py 93.75% <100.00%> (ø)
...elasticsearch/constraints/types/attribute_types.py 100.00% <ø> (ø)
...elasticsearch/constraints/types/qualifier_types.py 100.00% <ø> (ø)
...ever/data_tiers/tier_1/elasticsearch/transpiler.py 77.16% <100.00%> (ø)
...retriever/data_tiers/tier_1/elasticsearch/types.py 94.20% <100.00%> (ø)
...etriever/data_tiers/tier_1/elasticsearch/driver.py 45.83% <39.13%> (-2.61%) ⬇️
... and 2 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@everaldorodrigo everaldorodrigo merged commit 7df39e2 into main Dec 19, 2025
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants