Skip to content

Fix/cummax cummin pyarrow float#66283

Open
genrichez wants to merge 2 commits into
pandas-dev:mainfrom
genrichez:fix/cummax-cummin-pyarrow-float
Open

Fix/cummax cummin pyarrow float#66283
genrichez wants to merge 2 commits into
pandas-dev:mainfrom
genrichez:fix/cummax-cummin-pyarrow-float

Conversation

@genrichez

Copy link
Copy Markdown

BUG: fix cummax/cummin for pyarrow float dtypes

Closes #66257

What's wrong

Series.cummax() on pyarrow float columns returns garbage when the running max is negative. For example:

s = pd.Series([-4.0, -3.0, -2.0, -1.0], dtype="float64[pyarrow]")
s.cummax()
# returns [2.2e-308, 2.2e-308, 2.2e-308, 2.2e-308] instead of [-4, -3, -2, -1]

Same idea for cummin when the running min is +inf.

Why it happens

pyarrow.compute.cumulative_max defaults its start parameter to the smallest positive normal float (~2.2e-308 for float64), not -inf. So any negative value loses to that default. Similarly cumulative_min defaults to the largest finite float, not +inf.

The fix

Pass start=-inf for cummax and start=+inf for cummin when the array dtype is floating. This matches numpy behavior exactly. Integer and temporal types are unaffected (pyarrow already uses correct defaults for those).

if pa.types.is_floating(pa_dtype) and name in ("cummax", "cummin"):
    start_value = float("-inf") if name == "cummax" else float("inf")
    kwargs.setdefault("start", pa.scalar(start_value, type=pa_dtype))

Tests

Added regression tests covering:

  • All negative values (the reported bug)
  • +inf in cummin
  • Mixed positive/negative matching numpy output
  • NA handling with skipna True/False
  • Both float32 and float64

genrichez added 2 commits July 10, 2026 10:28
… extension arrays

When `df.loc[:, bool_mask] = value` is called with a boolean mask
that selects no columns on a single-column DataFrame with a nullable
extension dtype (Float64, Int64, string, boolean, etc.), the values
are incorrectly overwritten.

Root cause: In ExtensionBlock._unwrap_setitem_indexer, the condition
`is_list_like(indexer[1]) and indexer[1][0] == 0` incorrectly matches
boolean arrays because `False == 0` evaluates to True in Python.
This causes the column indexer to be discarded and the row indexer
(slice(None)) to be used alone, selecting all rows for assignment.

Fix: Add an early return in BlockManager.setitem() that detects an
all-False boolean column indexer and returns without modification.
This mirrors the existing early-return pattern already present in the
Copy-on-Write branch (line 571-572).

Closes pandas-dev#66255
pyarrow's cumulative_max/min use the smallest positive normal float
(~2.2e-308) and largest finite float as default start values. This
gives wrong results when the running max is negative or the running
min is +inf.

Pass start=-inf for cummax and start=+inf for cummin explicitly
when the dtype is floating, matching numpy behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: cummax / cummin incorrect for pyarrow floating dtypes

1 participant