Cube attributes with numbers are decorated with `np.int` and `np.float` instead of plain number

[mo-garethsjones]

🐛 Bug Report

For iris 3.13.0, loaded climate data into "cubes" have attributes which are shown as "np.int...(...)" or "np.float...(...)" instead of just numbers as is done for earlier versions of iris.
e.g., https://scitools-iris.readthedocs.io/en/v3.13.0/further_topics/ugrid/operations.html#plotting in the example code the "print(sample_mesh_cube)" prints the cube, with the very last attribute = "nco_openmp_thread_number np.int32(1)"
This should be (as in earlier versions of iris) "nco_openmp_thread_number 1"
I spotted this while examining CMIP5 and CMIP6 netcdf data.

How To Reproduce

Steps to reproduce the behaviour:

>>> import iris
>>> filename = iris.sample_data_path('mesh_C4_synthetic_float.nc')
>>> co = iris.load(filename)
>>> co[0].attributes['nco_openmp_thread_number']
np.int32(1)

Expected behaviour

>>> import iris
>>> filename = iris.sample_data_path('mesh_C4_synthetic_float.nc')
>>> co = iris.load(filename)
>>> co[0].attributes['nco_openmp_thread_number']
1

Environment

• Iris Version: 3.13.0

[rcomer]

This appears to be a numpy v2 change to the __repr__ method on the numpy.int32 object.

import numpy as np

print(np.__version__)
print(repr(np.int32(4)))

Older environment:

1.26.4
4

Newer environment:

2.2.6
np.int32(4)

[ukmo-ccbunney]

From @SciTools/peloton - this is expected behaviour with the change to Numpy v2.
See the notes in the Whats New for Iris 3.11: https://scitools-iris.readthedocs.io/en/stable/whatsnew/3.11.html#dependencies

[mo-garethsjones]

As this is "expected behaviour" does that mean the bug will never be fixed in iris?

[trexfeathers]

As this is "expected behaviour" does that mean the bug will never be fixed in iris?

This is not Iris behaviour: if you were to use Iris with NumPy v1.26 or earlier you would not see it.

This is not a bug: this is NumPy correcting poor design behaviour in their previous versions. While I cannot speak for their development team, I believe it was thought to be misleading that a NumPy object appeared to be a plain number.

---

In the past we have chosen to 'suppress' changes in NumPy behaviour, in the interest of maximising stability for our users, but this had negative consequences:

• Took more-and-more resource to maintain as NumPy continued to move on while we attempted to preserve its behaviour from years before.
• It was difficult for experienced Python users to use Iris, as it behaved differently to the rest of the ecosystem (i.e. showed behaviour that hadn't been seen elsewhere for several years).

Ultimately this approach proved unsustainable. The best we can do is to use the Iris documentation to advertise changes in NumPy behaviour when we spot them, to reduce the surprise to our user base. Similar with other core dependencies.

[mo-garethsjones]

It is a bug as the data is no longer being presented in the wanted way,
e.g, If I use the ncdump command to list the attributes of the example netcdf file I used in the initial issue
$ ncdump -h /data/apps/sss/environments/production-os47-2/lib/python3.12/site-packages/iris_sample_data/sample_data/mesh_C4_synthetic_float.nc
It produces at the end of global attributes
"nco_openmp_thread_number = 1 "
This is consistent with how Iris used to present the attribute in the loaded cube.

If numpy is causing the problem, then iris should have a fix to convert the "numpy object" (I don't know what that is in this context, I naively assumed it was a number) into a number, or a string at the very least.
"np.int32(1)" means nothing to a user unless they are experienced, and they have imported numpy as np (which I had not in the above example).

This may seem like a trivial issue, but CMIP model data have lots of attributes with normal numbers in them. Having them presented as "numpy objects" is confusing at best, and can be code breaking at worst.

[trexfeathers]

@mo-garethsjones thanks for the insight, that's very helpful. I just did a quick test and it looks like ANY numeric attributes in NetCDF files are loaded as NumPy scalars like this; I do not know whether there is an important reason, and there is a decent chance we have just been running with the default behaviour, not noticed/important before this printing change.

I will re-open this under a modified name as I would be interested in continuing this conversation with Iris developers and other interested parties.

[rcomer]

Iris has loaded these scalar attributes as numpy types since at least v3.0.3. I would say that changing the type of the object is a bigger breaking change than changing its __repr__. What if someone's code relies on a type check?

Is the main problem what you get when you do print(cube)? If so, perhaps that could be addressed explicitly. It looks like currently any object with a dtype attribute is assumed to be an array. Scalar numpy objects could potentially use str(value) instead (which would make them consistent with anything else that is not a string or an array).

iris/lib/iris/_representation/cube_summary.py

Lines 72 to 91 in 2c998a5

	def array_repr(arr):
	"""Produce a single-line printable repr of an array."""
	# First take whatever numpy produces..
	text = repr(arr)
	# ..then reduce any multiple spaces and newlines.
	text = re.sub("[ \t\n]+", " ", text)
	text = string_repr(text, quote_strings=False, clip_strings=True)
	return text
	def value_repr(value, quote_strings=False, clip_strings=False):
	"""Produce a single-line printable version of an attribute or scalar value."""
	if hasattr(value, "dtype"):
	value = array_repr(value)
	elif isinstance(value, str):
	value = string_repr(
	value, quote_strings=quote_strings, clip_strings=clip_strings
	)
	value = str(value)
	return value

[bjlittle]

After group discussion at the @SciTools/peloton:

1. We agree that the cube summary could be clearer with regards to this issue, so we will replace the np.int32(1) with 1 (as you expected) for clarity
2. We can't use builtin types to replace the native numpy encoding as there is a danger of losing the original type definition of the data
3. We will continue to expose the repr/str of any underlying object rather than pre-processing it for users; this is a philosophy that we adopted since the inception of iris to avoid an ongoing maintenance burden
4. Please never rely on processing the output of the cube summary, it isn't guaranteed to be stable between releases