Skip to content

Gb edf metadata adapter#2

Merged
Wiebke merged 8 commits intomainfrom
gb_edf_metadata_adapter
Mar 12, 2025
Merged

Gb edf metadata adapter#2
Wiebke merged 8 commits intomainfrom
gb_edf_metadata_adapter

Conversation

@rajasriramoju
Copy link
Contributor

This branch deals with the adapter for .gb files. Additional functionality has been added for .gb file ingestion. The goal is to take metadata associated with the .edf images associated to the respective sfloat .gb image: the hi and lo .edf images. The helper function create new indices if the same key exists with different values in the hi and low images' metadata, in order to not lose information.

The current version works in ingesting .gb files when there are .edf files with the associated .txt files. But there is a bug when ingesting a folder with just .gb files. This adapter needs debugging. The current terminal output looks like this:

Screenshot 2024-12-04 at 8 01 41 PM

with a final error statement of: AttributeError: 'NoneType' object has no attribute 'new'

@rajasriramoju rajasriramoju requested a review from Wiebke December 4, 2024 16:02
…t existing .edf files in the directory (user copied only relevant files)
Copy link
Member

@Wiebke Wiebke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks great.

I left a few very minor comments, the one regarding date comparison is likely of more relevance for the similar if a simple sort mechanism on strings is in use there currently.


# Compare two dates and select the later one
if hi_date is not None and lo_date is not None:
gb_dictionary["Date"] = hi_date if hi_date > lo_date else lo_date
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this point, both hi_date and lo_date should be of type string. Given that the format in the header is of form e.g. Sat Apr 1 11:35:48 2023 this may be lead to incorrect ordering if high and low scan happen to involve a date switch.

This is unlikely to matter much here, but would be important to take into account in e.g. sorting according to date.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latest commit now addresses this, and compares the two dates as datetime objects 👍🏼

Comment on lines +127 to +128
lo_header = lo_file.header
lo_date = lo_header.get("Date")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any particular reason why the remainder of the header information is not added to the metadata?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The header information will now be added to the metadata dictionary. Previously, only the date was required from the header, but the header data will not be added to avoid loss of any information.

Comment on lines +120 to +121
# File does not exist, set the date as None - empty metadata dictionary is
# returned in the parse_txt_accompanying_edf method
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be a slightly outdated comment.

Copy link
Contributor Author

@rajasriramoju rajasriramoju Jan 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment isn't outdated, but it will be written in a better way to explain why an empty dictionary is needed.

@rajasriramoju rajasriramoju requested a review from Wiebke January 23, 2025 08:03
Copy link
Member

@Wiebke Wiebke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

One aspect that remains to be investigated in regards to the adapters is why gb adapter is called multiple times: If a corresponding .edf does not exist for a .gb file, the warning regarding this gets logged three times.

Testing and attempting to debug #10, indicates that this is not related to this adapter in particular, but would be good to understand overall.

@Wiebke Wiebke merged commit 72f0e1c into main Mar 12, 2025
2 checks passed
@Wiebke Wiebke deleted the gb_edf_metadata_adapter branch March 20, 2025 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants