[Question / possible bug] When `Saturate = false`, underflow gives `nan` but overflow gives `inf`; is this standard? What's the use?

[SwuduSusuwu]

Examples of underflow (or subnormal) = nan

ml_dtypes/ml_dtypes/include/float8.h

Lines 1423 to 1431 in 1527272

	if (from_bits == 0 && kFromHasZero) {
	if constexpr (kToHasZero) {
	// Keep the sign, if `To` supports it.
	return from_sign_bit && kToIsSigned ? -To{} : To{};
	} else {
	return kSaturate ? std::numeric_limits<To>::denorm_min()
	: Eigen::NumTraits<To>::quiet_NaN();
	}
	}

ml_dtypes/ml_dtypes/include/float8.h

Lines 1433 to 1438 in 1527272

	if constexpr (!kToIsSigned && kFromIsSigned) {
	if (from_sign_bit) {
	return kSaturate ? std::numeric_limits<To>::lowest()
	: Eigen::NumTraits<To>::quiet_NaN();
	}
	}

Examples of overflow = inf

ml_dtypes/ml_dtypes/include/float8.h

Lines 1562 to 1572 in 1527272

	// `From` supports larger values than `To`, we may overflow.
	if constexpr (std::make_pair(std::numeric_limits<To>::max_exponent,
	std::numeric_limits<To>::digits) <
	std::make_pair(std::numeric_limits<From>::max_exponent,
	std::numeric_limits<From>::digits)) {
	if (rounded_from_bits > aligned_highest) {
	// Overflowed values map to highest or infinity depending on kSaturate.
	to = kSaturate ? Eigen::NumTraits<To>::highest()
	: Eigen::NumTraits<To>::infinity();
	}
	}

Expected results

Both nan, or both inf.

nan > inf

ml_dtypes/ml_dtypes/include/float8.h

Lines 1103 to 1105 in 1527272

	constexpr inline bool(isnan)(const float8_e3m4& a) {
	return abs(a).rep() > std::numeric_limits<float8_e3m4>::infinity().rep();
	}

ml_dtypes/ml_dtypes/include/float8.h

Lines 1111 to 1113 in 1527272

	constexpr inline bool(isnan)(const float8_e4m3& a) {
	return abs(a).rep() > std::numeric_limits<float8_e4m3>::infinity().rep();
	}

So it is confusing to use nan for lower values.

[jakevdp]

My understanding is that NaN is returned in the first block when we are asking for a representation of 0 in a dtype that cannot represent the value 0. I don't think inf would be a reasonable output in that case.

For overflowing values, I think returning infinity is reasonable, because we are asking for the best representation of a value which is larger than the representable range of values. Returning inf for this case is typical of floating point conversions.

I'm not sure what your nan > inf block is meaning to propose or ask – could you clarify?

[SwuduSusuwu]

My understanding is that NaN is returned in the first block when we are asking for a representation of 0 in a dtype that cannot represent the value 0. I don't think inf would be a reasonable output in that case.

For overflowing values, I think returning infinity is reasonable.

True, should have asked "Why not both nan" (opposed to both inf).

I'm not sure what your nan > inf block is meaning to propose or ask – could you clarify?

Just confused that nan is used for underflow but inf is used for overflow, since the binary representation of nan is > inf and floats are supposed to allow binary sort formulas.

But your response summarizes the purpose of how it is now; to allow programs to use those return values to distinguish underflows from overflows. Did not choose "Close with comment" in case this discussion has future use (don't know if all attributes of relevant formulas were covered so far).

[jakevdp]

Just confused that nan is used for underflow but inf is used for overflow, since the binary representation of nan is > inf and floats are supposed to allow binary sort formulas.

The relative ordering of the binary representation of a floating point values has no direct relation to the relative ordering of the values they represent. NaN is not considered to be larger than inf: in fact, it is neither larger nor smaller:

>>> np.nan > np.inf
False
>>> np.nan < np.inf
False

The result of comparisons involving NaN is dictated by the IEEE 754 standard.