I triggered the writing of ncdiags off x0054 and saved a whole lot of ncdiag files along the experiment.
Two days ago, I tried to start converting those data to IODA. The IODA converted failed - reported here:
GEOS-ESM/swell#755
as at first I thought this was a issue w/ SWELL/JEDI. After some digging I have come to realize that the issue is w/ the radiance files - specifically the QC_Flag parameter. What used to show up as zeros, now show up as NaNs. QC_Flag is written as float and converted to integer in the the converted to integer in the python program doing the IODA conversion; python doesn't like to have NaNs converted to integers - there are tricks to do that, but the really issue is why is this happening.
So I am now looking more closely at the ana.log files from when I triggered the NCDIAG file writing in GSI and I am seen a zillion messages like this:
** WARNING: Amount of data written in Weighted_Lapse_Rate (1124)
differs from variable Channel_Index (562)!
** WARNING: Amount of data written in Weighted_Lapse_Rate (1686)
I tried checking whether 5.43 (x0053RPY) has the same issue and unfortunately it does.
As some of you recall, when I first released GEOS-JEDI I generated IODA files from prePP without any issues. This gives me a starting point to a healthy version of the code:
version I know ncdiags were good: v5.42.3
version I know ncdiags (rad) are fishy: v5.42.9
it's possible that a few versions pass v5.42.3 are still ok, but I don't know it for a fact.
I am now comparing the in-between versions to see what might be causing the problem.
I triggered the writing of ncdiags off x0054 and saved a whole lot of ncdiag files along the experiment.
Two days ago, I tried to start converting those data to IODA. The IODA converted failed - reported here:
GEOS-ESM/swell#755
as at first I thought this was a issue w/ SWELL/JEDI. After some digging I have come to realize that the issue is w/ the radiance files - specifically the QC_Flag parameter. What used to show up as zeros, now show up as NaNs. QC_Flag is written as float and converted to integer in the the converted to integer in the python program doing the IODA conversion; python doesn't like to have NaNs converted to integers - there are tricks to do that, but the really issue is why is this happening.
So I am now looking more closely at the ana.log files from when I triggered the NCDIAG file writing in GSI and I am seen a zillion messages like this:
** WARNING: Amount of data written in Weighted_Lapse_Rate (1124)
differs from variable Channel_Index (562)!
** WARNING: Amount of data written in Weighted_Lapse_Rate (1686)
I tried checking whether 5.43 (x0053RPY) has the same issue and unfortunately it does.
As some of you recall, when I first released GEOS-JEDI I generated IODA files from prePP without any issues. This gives me a starting point to a healthy version of the code:
version I know ncdiags were good: v5.42.3
version I know ncdiags (rad) are fishy: v5.42.9
it's possible that a few versions pass v5.42.3 are still ok, but I don't know it for a fact.
I am now comparing the in-between versions to see what might be causing the problem.