Dictionary Handling

[mgt16-LANL]

I'm having trouble with h5io_browser saving/reading dictionaries. I'm not sure if this was a recent change because this would work previously.

Version = '0.1.8'

This works:

import h5io_browser

td = {'example':'thing','thing2':{'0':1,'1':2}}
h5io_browser.write_dict_to_hdf(file_name='example.h5',data_dict=td)

This doesn't:

import h5io_browser

td = {'example':'thing','thing2':{0:1,1:2}}
h5io_browser.write_dict_to_hdf(file_name='example.h5',data_dict=td)

The keys in the dictionary have to be strings?

This is creating issues when trying to load/save pandas dataframes in pyiron output data hdf5s.

-Michael.

[jan-janssen]

For pandas dataframes pyiron typically uses the internal to_hdf() function from pandas, for example in the case of the pyiron tables: https://github.com/pyiron/pyiron_base/blob/main/pyiron_base/jobs/datamining.py#L768

Still I agree that it would be a nice extension to handle this directly in h5io_browser as at least h5io already provides support for the storage of pandas DataFrames.

[jan-janssen]

I opened two pull requests to address this issue, by storing the pandas data frame directly:

• Fix group reloading #119
• Fix writing pandas DataFrame to open HDF5 file h5io#146

Example:

td = {'example':'thing','thing2':pandas.DataFrame({0:[1],1:[2]})}
h5io_browser.write_dict_to_hdf(file_name='example.h5',data_dict=td)

[mgt16-LANL]

@jan-janssen Thanks for hopping on this so quick! I'm ~90% sure this error was breaking the functionality you described here: For pandas dataframes pyiron typically uses the internal to_hdf() function from pandas. I could share the stack trace (lmk if you'd like to see), but whenever trying to use .to_hdf() from pandas to a pyiron job h5 file it would give errors. I wanted to mention in the ~10% case this was not related.

[jan-janssen]

As a temporary solution you could try to use:

import pandas
import h5py
from pyiron_atomistics import Project

pr = Project("test")
job = pr.create.job.Lammps("lmp")
job.structure = pr.create.structure.ase.bulk("Al")
job.run()

# save additional data to job
df = pandas.DataFrame({0:[1],1:[2]})
h5_path = "/" + job.job_name + "/mydata"
df.to_hdf(job.project_hdf5.file_name, key=h5_path)
with h5py.File(job.project_hdf5.file_name, "a") as hdf:
    hdf[h5_path].attrs["TITLE"] = "pd_dataframe"

# read data
print(job["mydata"])