Particles not moving: CROCO and xarray #1665

Activity

1. knutfrode commented on Aug 20, 2025

   

   knutfrode

   on Aug 20, 2025

   Collaborator

   More actions

   Hi,
   The first problem seems to be that h is found in both ROMS-file and the gridfile, and then Xarray don't know which to use.
   I added now compat=override to the merge command, as was suggested:
   230d6d8

   Thus you can update and see if it makes a difference (or make your netCDF-files abailable).

   The second problem seems to be that mask_u is missing, but this is perhaps found in the grd-file that was not merged?

   Older versions of the ROMS reader probably worked because they neglected the mask altogether.
2. AndresSepulveda commented on Aug 20, 2025

   

   AndresSepulveda

   on Aug 20, 2025

   ContributorAuthor

   More actions

   Thanks Knut,

   New error now. A quick solution would be to use an older version of OpenDrift. This is an open ocean domain so there is not much need of mask variables. How do I install an older version using conda?

   The error message is

   File "/home/asepulveda/miniconda3/envs/pnud/lib/python3.13/site-packages/xarray/structure/merge.py", line 1024, in merge
   merge_result = merge_core(
   dict_like_objects,
   ...<3 lines>...
   fill_value=fill_value,
   )
   File "/home/asepulveda/miniconda3/envs/pnud/lib/python3.13/site-packages/xarray/structure/merge.py", line 766, in merge_core
   raise MergeError(
   ...<2 lines>...
   )
   xarray.structure.merge.MergeError: unable to determine if these variables should be coordinates or not in the merged result: {'lat_v', 'lon_v', 'lon_u', 'lat_rho', 'lon_rho', 'lat_u'}

   Regards,

   Andres
3. knutfrode commented on Aug 22, 2025

   

   knutfrode

   on Aug 22, 2025

   Collaborator

   More actions

   The ROMS reader has had a number of "fixes" for CROCO data over the years, as CROCO files seem to be very variable, and also not very stringent.
   Are these specific output files, including the grd-file, raw output from a plain and standardised CROCO setup?
   Or are they customized locally by the user (e.g. yourself), i.e. different from what other users might have?
   In the latter case, it is impossible to make the ROMS reader "digest" all possible forms of data- and grd-files that end-users might have (e.g. as here where variable h is duplicated in both data-file and grd-file, which does not sound like a good idea).

   If this output format is identical to other CROCO setups elsewhere, it is better to adapt the ROMS reader to this format, instead of using older versions of OpenDrift.
   E.g. older versions of the ROMS reader do not support providing an Xarray dataset directly, only filenames, with possible wildcards.
   If output files are non-standard or manipulated, opening and merging them locally with Xarray (including merging in the missing variables from grd-file) might be a flexible solution.

   If these files are standardized (i.e. not post-processed by anyone), it would be useful if you could provide a minimalistic sample file with two time steps only, and the corresponding grd-file (ideally < 0.5 MB).
   Then I could include this in the test-suit, so ensure the ROMS reader will handle CROCO data also in the future.
4. AndresSepulveda commented on Aug 22, 2025

   

   AndresSepulveda

   on Aug 22, 2025

   ContributorAuthor

   More actions

   Thanks Knut,

   There is not much liberty in what non-prognostic variables are in a CROCO output file, but yeah, they do repeat the h variable in the GRD and in the HIS/AVG output. It seems the HIS/AVG are meant to be kind of stand alone for reading, but they are not completely stand alone.

   I will provide you with a minimalistic output of the default test case for CROCO (Benguela configuration) in the latest CROCO stable version, 2.1.0.
5. knutfrode commented on Aug 22, 2025

   

   knutfrode

   on Aug 22, 2025

   Collaborator

   More actions

   Good.
   Maybe you could make 3 timesteps, instead of 2, but in a single file.
   But a small region is preferrable to minimise size.
6. AndresSepulveda commented on Aug 25, 2025

   

   AndresSepulveda

   on Aug 25, 2025

   ContributorAuthor

   More actions

   Hi Knut,

   Here are the files for the default case in the latest CROCO.
   Domain size is 41x42 (dx=30km)

   croco_grd.nc.gz
   croco_his.nc.gz
7. knutfrode commented on Aug 27, 2025

   

   knutfrode

   on Aug 27, 2025

   Collaborator

   More actions

   Hi,

   One (of several) problem seem to be that the dimensions of e.g. current components are not the same as the corresponding masks (from grd-file).
   E.g. the horizontal dimensions of u are (eta_rho, xi_u) which I assume must be a bug, and should instead be (eta_u, xi_u) as for the corresponding mask?
   I made now a fix for this in the ROMS-reader, which now also handles the second problem of duplicate variables in his- and grd-files:
   58cfaa5#diff-e8bb6de9d5f842a302a347821fd8734bbae0109243a85aea2d3cf02b8b5bdd6d

   After this, your sample files work well, and I added also a reduced version (removed unnecessary variables and lower layers) in the repository and a corresponding test.

   A third problem with the Croco-output, which btw is not critical in this specific case, is the use of standard_name for elements which are not found in the official standard_name table: https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html
   Examples are ocean_s_coordinate_g2_at_w_location and longitude_at_u_location
   This is bad practice, and undermines the very purpose of the standard_name table. If anything, long_name should be used instead of standard_name here.

   If you have contact with the croco-developers, you are welcome to forward/repost these concerns - as it is better to fix these problems at the origin, instead of independent softwares (such as OpenDrift) needing to make ugly, hardcoded workarounds.

   I have not yet made a new conda version (1.14.3) including this fix, but will come within a day or two.
8. knutfrode commented on Aug 28, 2025

   

   knutfrode

   on Aug 28, 2025

   Collaborator

   More actions

   OpenDrift 1.14.3 (with mentioned fix for Croco) is now available on conda-forge: https://anaconda.org/conda-forge/opendrift