Metadetect aliasing - #491
Conversation
|
I'm not sure we want absolutely all the columns - there are millions of them! I thought the idea was just to make sure that all the columns we did import get an alias to their old name? |
|
So I was unsure, it was suggested in PO that it was better to do the other way around. I.e. get all columns, and then add our relevant aliases. |
joezuntz
left a comment
There was a problem hiding this comment.
Looks really good, just minor changes!
There was a problem hiding this comment.
Pull request overview
This PR updates Rubin MetaDetect ingestion to optionally retain the full input column set and then add TXPipe-friendly column aliases into the output shear catalog. It also refactors MetaDetect column selection logic and tweaks diagnostics plotting.
Changes:
- Add an
all_columnsstage option and pass it through toprocess_metadetect_datato control whether all input columns are written. - Add a post-write aliasing step to link TXPipe-standard column names to their original MetaDetect column names.
- Adjust structured-array sanitization allocation and make small diagnostic plot scale tweaks.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| txpipe/ingest/metaDetect.py | Adds all_columns config, changes Butler get path, and adds an aliasing post-pass on the output HDF5. |
| txpipe/ingest/lsst.py | Refactors process_metadetect_data to emit either full columns or a reduced set, and adjusts sanitize structured-array allocation. |
| txpipe/ingest/dp1_info.py | Introduces the alias mapping dictionary for TXPipe column names. |
| txpipe/diagnostics.py | Minor plotting scale/comment tweaks for diagnostic plots. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This is an update to the ingestion of MetaDetect, to include the full list of columns, and then providing an aliasing list that then adds the aliases to the TXPipe needed columns