UofMD-landState-3-0: land use data set

[slm7826]

CMIP6Plus data set, v20241002: Time units
First off, I'd like to thank the authors of this data set for providing the data for examination and review.

One thing that bothers me a bit is the units of the time axis: In this data set it is set to “days since 0850-01-01 00:00:00”, in NOLEAP calendar. If taken on its face value, it would mean that the models need to apply land use every 365 days. GIven that the historical and future simulations are typically done using calendars other than "NOLEAP", it would result in significant shift in application dates with respect to the year boundaries.

I think CMIP6 data set used "years since ...", and this seems to be more logical choice in this case, clearly indicating that the data are annual. I know that CF conventions does not recommend [although does not prohibit] using "years" as time units, but perhaps in this case it is justified.

[slm7826]

CMIP6Plus data set, v20241002: Comparison of transition rates with CMIP6

Comparison of globally-integrated transition rates among different land use types showed some significant differences from CMIP6 data set, in particular fairly strange spiky character of the CMIP6Plus data. To illustrate, the figure below shows the transition to pasture (primf_to_pastr + primn_to_pastr + secdf_to_pastr + secdn_to_pastr + urban_to_pastr + c3ann_to_pastr + c4ann_to_pastr + c3per_to_pastr + c4per_to_pastr + c3nfx_to_pastr + range_to_pastr + pltns_to_pastr)

As evident, these transition rates -- and their time variability -- are much higher in CMIP6plus compared to CMIP6, starting around 1990. There are also significant strange spikes at the boundaries of the 1950 decade. The next figure shows transitions from pasture (pastr_to_secdf + pastr_to_secdn + pastr_to_urban + pastr_to_c3ann + pastr_to_c4ann + pastr_to_c3per + pastr_to_c4per + pastr_to_c3nfx + pastr_to_range + pastr_to_pltns) that shows the same spiky behavior, and significant increase of transition rates in later decades.

I would be grateful for comments on this change from earlier data set; in particular, the spikes at the beginning and the end of 1950 decade. Also, should we expect all transition rates to increase in the later decades in CMIP6plus data set compared to CMIP6?

[znicholls]

@lchini if you have thoughts here too I am sure they would be appreciated

[lchini]

Thanks for pointing out this issue about the LUH transitions. Apologies for the delay in my response - I’ve been looking into this issue and wanted to wait until I had some analysis and possible solutions before responding.

So, it appears that this issue comes from the processed HYDE data, one of the inputs into LUH (rather than in the other transitions calculated by LUH). The HYDE dataset provides the gridded areas of grazing, cropland, and urban land that is used by LUH. We process this data for use in LUH by aggregating the data from 5min to 0.25 degree resolution, converting the data from areas to fractions of grid-cells, linearly interpolating the century and decadal timescale data to annual values, and also ensuring that fractions of all processed HYDE land-use variables do not exceed 1. The new HYDE 3.4 data is provided every century from 800 to 1700, then every decade from 1700 to 1950, and annually from 1950 onwards. An earlier version of HYDE used in the LUH2 provided the data by century from 800 to 1700, decadally from 1700 to 2000, and annually from 2000 onwards. So the new version of HYDE now has annual data from 1950 to 2000 that was previously provided decadally. These new annual values are responsible for the longer period of annually changing transitions, when compared with LUH2. These annual transitions also have higher values than those in LUH2 for the same time period and show several large spikes in transitions that are anomalous and it is not clear whether they are based on underlying remote sensing or FAO data, or are a numerical artifact.

The attached figures show the globally aggregated transitions to and from the managed pasture data (in red) in the processed HYDE data (before it is run through LUH). These transitions are responsible for the corresponding transitions coming out of LUH2. For comparison I have also included the transitions from the older LUH2 data (in blue), which shows fewer large spikes and lower levels of transitions in general. These figures are very similar to the ones provided in the comment/question above, indicating that this input dataset is largely responsible for these features in the LUH dataset.

To remove these spikes in transitions we will probably need to re-process the HYDE data in some way, and we have been actively exploring different ways to do this. There are a couple of options for re-processing the HYDE data and I would like to get some feedback from the ESM groups who will be using the data on which would be the preferred approach.

Option 1: resample the HYDE data so that from 1700 onwards we linearly interpolate between decadal time points (instead of switching to annual values after 1950). This will remove all the annual variations in the data. The data will more closely resemble the temporal resolution of the LUH2 dataset. Results from a test of this approach are shown in the attached figures (in green). As you can see, the large spikes in transitions are removed and the overall level of transitions is lower from 1990 onwards than those using the current version of HYDE 3.4. The resampled data is identical to the current version of HYDE3.4 for all years prior to 1950.
Pros of Option 1: spikes and level of transitions is improved and comparable to LUH2, temporal resolution of data is consistent throughout the time-series
Cons of Option 1: we are not including any recent annual data in the land-use dataset

Option 2: reprocess the linearly interpolated HYDE data to use a 5-year average for each datapoint for all years in the time series.
Pros of Option 2: This will smooth out any sudden changes/spikes in the data. We will be using the recent annual data (although somewhat modified).
Cons of Option 2: this is more untested and un-developed which poses some risks. We will be introducing new features to the data which are not present in the underlying inputs, which could be harder to justify and explain when analyzing the land-use change results. The underlying temporal resolution of inputs is not consistent throughout the time-series which can make land-use change appear to become more frequent in recent years.

[durack1]

@lchini thanks for this, it's great that we're digging in to fix issues before the data is used! I wonder, with the year to year step changes in the red (HYDE 3.4) timeseries oscillating by up to 80000 km^2 (about 1/3 the size of the UK!) one year to the next, this has to be a data artifact, no? It seems the magnitude of the oscillations extend well into the satellite-era, and I really have a hard time believing that such large changes could occur in reality

[vnaik60]

ping @slm7826 any thoughts on Louise's suugestions on dealing with the spikes?

[nkianggiss]

It looks like this was resolved in 3-1-1.
However, does one of the 3-1-1 files have the time series of the land mask?
There is an icwtr array for an ice/water mask in the static file, but that is only one time point. Thanks to anyone who can point me to where the time series is.

[znichollscr]

Thanks for the feedback @slm7826

Re the time units:

The reason we changed this was that, with the "years since...", the data would not load cleanly with xarray. This is, arguably, a bug in xarray although it's not completely clearly because, as you say, "years since..." is not a cf-compliant unit. That said, writing a dataset that can't be read with such a standard tool seemed a terrible idea, so we switched the units.

If taken on its face value, it would mean that the models need to apply land use every 365 days

I think this is a misreading of the point of the calendars. The time bounds of the dataset make clear that the transitions happen between Jan 1 of year X and Jan 1 of year X + 1. The calendar just tells the user what calendar the data is in, and changing to another calendar is a trivial exercise given this state of the data (I don't think any calendar translator would do anything other than put this data on 1 Jan of every year, although I can see it is perhaps not totally unambiguous).

However, given your confusion, this is probably something we should include once we get our user guidance notes going (x-ref #118 (comment)). A line like the following may be enough to clarify, but please suggest any adjustments you would like, "The data is annual. We use a noleap calendar for simplicity and for cf-compliance. However, don't let that confuse you, in all calendars, the transitions should be applied between 1 Jan of year X and 1 Jan of year X + 1."

[slm7826]

Absolutely, and in general this would be true. However, for this instance, the change is trivial, just divide the time calendar by 365 and you're back to years so I don't think we need to spend any more brain power on this.

I would not trivialize the code changes changes necessary for such translations for many modeling centers without intimate knowledge of their code, nor the potential compatibility issues with the older data sets.

[znichollscr]

Just to check I understand, you're saying that writing a quick script to alter the time axis back to years isn't trivial? You can do that in an easy language (i.e. a language other than C/C++/Fortran) if you need, i.e. effectively create your own version of the file that is back in the previous format, then just use it and your code won't even know that the file on the ESGF was different.

[slm7826]

I was interpreting your suggestions as changing the models' codes to read the data with new time units directly.

If you propose to change time units in pre-processing to match the expectations of the existing models' code then I agree, that should be easy. On the other hand, why do it and force many CMIP7 participants to do their pre-processing if the time units in this data set can be simply made the same as in CMIP6?

Besides, I still think that "years since..." is a more logical choice for annual data of this nature.

[znichollscr]

I get all that. I can only repeat what I said previously to explain why we changed from CMIP6 (#141 (comment)) so I think we've reached the end of our circle.

[durack1]

Just stepping back a little. @slm7826 and @znichollscr wouldn't the aspirational goal be to have CF compliant, well oiled and formatted datasets that are as consistent with one another as possible? That would then mean that clunky once off code (calendar, time axis, grids etc) disappears.. And potentially code to could written and shared across groups..

I also feel uncomfortable about hacky time axis being defined with observed data, which is collected on a gregorian calendar, so why impose a non-reality calendar "noleap" when the data was recorded against a gregorian, including leap days..

There's a balance to usability, vs never-ending iteration for little gain, but just thought this perspective is useful, particularly as we are aspiring to "operational" and "sustainable" dataset delivery..

[durack1]

@lchini ping!

[znicholls]

@lchini a question from Roland Seferian: do you know what 'plnts' land-use type means ?

I guess, also additionally, are there docs on what the various variables in the netCDF files mean somewhere?

[znichollscr]

Creating a global attribute with mappings, e.g., primf = primary vegetation on LUH2 potential forest land, harv = wood harvest, primn = primary vegetation on LUH2 potential non-forest land, etc etc would be a useful tweak

My dissenting two cents on this: I feel this is just another thing to keep track of (and potentially get wrong with typos, forgetting about it when we add a new data type etc.). The current descriptions are pretty clear, it's trivial to make a summary if that's what someone wants and future documentation will be a better place for full details.

[durack1]

My dissenting two cents on this

I would dissent with your dissent. The whole point of netcdf is that data and it's identifying metadata is in the same file, which is a major advantage of the format over GRIB as a single example. If you want to go and read Hurtt et al., 2020, nowhere is plnts, primf, primn defined, and I would far prefer if this info was in the file that contains these matrices.. rather than an appendix of a paper that will not be linked to in a global attribute (as it will be published subsequent to data release).

Ultimately up to @lchini to make a decision how much effort she wants to go to explaining what is what..

[znichollscr]

The whole point of netcdf is that data and it's identifying metadata is in the same file, which is a major advantage of the format over GRIB as a single example

I agree with this

I would far prefer if this info was in the file that contains these matrices

I agree with this too

I think where we differ is in how many times this should be in the file. The information is already there (in the long names). You're suggesting it should be in the file twice (once in long names, once in global attributes in a summarised form). I would stick with having it there just once. (For what it's worth, duplicating info is why I also don't like having source ID, source version and source in the files, nor frequency, start time and end time. Source ID, source version and source are all subtly different forms of the same thing, which makes it hard to know what you're looking at and a disaster if they contradict each other. Frequency, start time and end time provide no extra information in addition to the file's time axis, so the only thing you can do is provide contradictory information.)

[durack1]

Just to be specific, what I am suggesting is we take the existing metadata/long_names, e.g.,

primf_harv:long_name = "wood harvest area from primary vegetation on LUH2 potential forest land" ;
primn_harv:long_name = "wood harvest area from primary vegetation on LUH2 potential non-forest land" ;
secmf_harv:long_name = "wood harvest area from secondary mature vegetation on LUH2 potential forest land" ;
secyf_harv:long_name = "wood harvest area from secondary young vegetation on LUH2 potential forest land " ;
secnf_harv:long_name = "wood harvest area from secondary vegetation on LUH2 potential non-forest land" ;
primf_bioh:long_name = "wood harvest biomass carbon from primary vegetation on LUH2 potential forest land" ;
primn_bioh:long_name = "wood harvest biomass carbon from primary vegetation on LUH2 potential non-forest land" ;
secmf_bioh:long_name = "wood harvest biomass carbon from secondary mature vegetation on LUH2 potential forest land" ;
secyf_bioh:long_name = "wood harvest biomass carbon from secondary young vegetation on LUH2 potential forest land" ;
secnf_bioh:long_name = "wood harvest biomass carbon from secondary vegetation on LUH2 potential non-forest land" ;
pltns_bioh:long_name = "wood harvest biomass carbon from plantation forest vegetation" ;
pltns_harv:long_name = "wood harvest area from plantation forest vegetation" ;
primf_to_pltns:long_name = "transition from primary forest to plantation forest" ;
primn_to_pltns:long_name = "transition from primary non-forest to plantation forest" ;
pltns_to_secdf:long_name = "transition from plantation forest to secondary forest" ;
secdf_to_pltns:long_name = "transition from secondary forest to plantation forest" ;
pltns_to_secdn:long_name = "transition from plantation forest to secondary non-forest" ;
secdn_to_pltns:long_name = "transition from secondary non-forest to plantation forest" ;
pltns_to_c3ann:long_name = "transition from plantation forest to C3 annual crops" ;
c3ann_to_pltns:long_name = "transition from C3 annual crops to plantation forest" ;
pltns_to_c4ann:long_name = "transition from plantation forest to c4ann" ;
c4ann_to_pltns:long_name = "transition from c4ann to plantation forest" ;
pltns_to_c3per:long_name = "transition from plantation forest to c3per" ;
c3per_to_pltns:long_name = "transition from c3per to plantation forest" ;
pltns_to_c4per:long_name = "transition from plantation forest to c4per" ;
c4per_to_pltns:long_name = "transition from c4per to plantation forest" ;
pltns_to_c3nfx:long_name = "transition from plantation forest to c3nfx" ;
c3nfx_to_pltns:long_name = "transition from c3nfx to plantation forest" ;
pltns_to_pastr:long_name = "transition from plantation forest to pastr" ;
pastr_to_pltns:long_name = "transition from pastr to plantation forest" ;
pltns_to_range:long_name = "transition from plantation forest to range" ;
range_to_pltns:long_name = "transition from range to plantation forest" ;
pltns_to_urban:long_name = "transition from plantation forest to urban" ;
urban_to_pltns:long_name = "transition from urban to plantation forest" ;

And make sure that there are full expansions for any identifying acronyms/variable names in the files. Some obvious candidates are c3ann,c4ann, c3per, c4per, c3nfx, range etc, which I can attempt to infer from names C3 vs C4 photosynthesis, per = perennial ann = annual (and presumably a biennial will appear at some stage), but that is because I have a vague interest in gardening, which may not map across all ncdumpers who come across the files.

And we agree, no duplicate info should be added (so keep info with the variables, rather than duplicating in global_atts), we could capture long info in a variable :comment entry, keeping the long_names pithy and short, as they already are

[vnaik60]

I think you are asking for more self-explanatory long names. For e.g.

pltns_to_c4ann:long_name = "transition from plantation forest to c4ann"

could be

pltns_to_c4ann:long_name = "transition from plantation forest to c4 annual crops"

Let's minimize additional work especially given the Feb 28th deadline. Making the longnames a bit more informative would be the easiest, I think. Those who use this data, are probably aware of what each of the terms mean. And from past experience, highly likely that an update to https://luh.umd.edu/ is in the works.

[lchini]

The land mask in the land-use dataset is constant/static. You can use the icwtr data layer for all years.On Aug 8, 2025, at 4:01 PM, Nancy Y. Kiang ***@***.***> wrote: It looks like this was resolved in 3-1-1. However, does one of the 3-1-1 files have the time series of the land mask? There is an icwtr array for an ice/water mask in the static file, but that is only one time point. Thanks to anyone who can point me to where the time series is. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[NortonAlex]

@lchini, not sure if I should start a new thread for this but I have a general question about the land use dataset (version 3-1-1). Following what is described in Hurtt et al. (2020, https://doi.org/10.5194/gmd-13-5425-2020), am I correct in assuming that all wood harvested biomass and wood harvested area post-1920 is associated with transitions other than conversion to agricultural/urban lands (e.g. primf to secdf, or secmf to secyf, or primn to secdn)? In other words, the biomass or area associated with conversion to agricultural lands (c3ann, c3per, pastr, etc.) is not accounted for in the bioh or harv variables post-1920. Any guidance would be hugely appreciated!