Skip to content

Ingesting NetCDF or GRIB files #1905

@pflashgary

Description

@pflashgary

Hi rfecher, I infer from this line netcdf format is supported. What I would like to try is ingesting a folder containing NetCDF files. Due to the differences between NetCDF/GRIB files and GeoTiff files I have a few questions:

  • Does the folder have to contain one NetCDF(GRIB) file for each time step? For example if we have one year worth of daily air temperature, do we need to have 365 NetCDF files in the folder? or does the geowave ingestion plugin support both spatial and temporal ingesting from one file?

  • How about different variables in a GRIB or NetCDF file? Imagine we have air temperature as well wind speed in the same file.

I have downloaded an ERA5 netcdf file just to test this and these are the steps I have taken:

source /tmp/geowave-env.sh
geowave store add nc --gwNamespace geowave.nc -t dynamodb --region US_EAST_2
geowave index add nc ncspatial_temporal -t spatial_temporal --partitionStrategy round_robin --numPartitions $NUM_PARTITIONS
geowave ingest localtogw adaptor.mars.internal-<some-file>.nc nc ncspatial_temporal

Nothing failed only some SLF4J warnings.

These are the tables:

aws dynamodb list-tables --region us-east-2
{
    "TableNames": [
        "geowave.nc_INDEX_GEOWAVE_METADATA",
        "geowave.nc_STATISTICS_GEOWAVE_METADATA",
        "geowave.nc_STORE_PROPERTIES_GEOWAVE_METADATA"
    ]
}

However, these tables contain no items (after running the describe table command on each table). That means ingesting didn't happen.

aws dynamodb describe-table --table-name geowave.nc_STORE_PROPERTIES_GEOWAVE_METADATA --region us-east-2

I am wondering if an example of ingesting from NetCDF /GRIB files can be included?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions