fix: support melting empty DataFrames without crashing#2509
Conversation
product-auto-label
bot
added
size: m
shuoweil
changed the title
| assert table.labels | ||
| assert table.labels["test"] == "labels" | ||
|
|
||
|
|
There was a problem hiding this comment.
nit: move test_dataframe_melt_multiindex to tests/system/small/test_multiindex.py file, which has tests related to multi index.
bigframes/core/blocks.py
Outdated
| rows.append(row) | ||
|
|
||
| return core.ArrayValue.from_pyarrow(pa.Table.from_pylist(rows), session=session) | ||
| import pyarrow as pa |
There was a problem hiding this comment.
IIUC, this import can be removed because the header of this file has the pyarrow importing?
bigframes/core/blocks.py
Outdated
| import pyarrow as pa | ||
|
|
||
| if not rows: | ||
| from bigframes.dtypes import bigframes_dtype_to_arrow_dtype |
There was a problem hiding this comment.
we have bigframes.dtypes import in the head of this file.
There was a problem hiding this comment.
This testcase has been cleaned up.
tests/system/small/test_dataframe.py
Outdated
| assert table.labels["test"] == "labels" | ||
|
|
||
|
|
||
| def test_dataframe_melt_multiindex(session): |
There was a problem hiding this comment.
It looks like I cannot reproduce the issue on the main branch. See screenshot/AXRSuFmXPTt5gzj
There was a problem hiding this comment.
Good catch! test_dataframe_melt_multiindex didn't catch the syntax error because the DataFrame wasn't actually empty. The core issue only happens when we try to unpivot an empty selection, causing an untyped NULL compilation.
To properly cover this edge case, I added test_count_empty_multiindex_columns in test_multiindex.py. Note that the test explicitly asserts the generated .to_frame().sql locally. If we run .to_pandas() directly, it trips over an unrelated Ibis compiler bug with empty [STRUCT()] layouts on the BigQuery backend, which I am fixing in a separate upcoming PR. This test safely completely isolates the unpivot fix in this branch..
shuoweil
force-pushed
the
shuowei-fix-unpivot-empty
branch
from
32ce90c to
f03758d
Compare
shuoweil
force-pushed
the
shuowei-fix-unpivot-empty
branch
from
77dff89 to
2d1c6d4
Compare
shuoweil
force-pushed
the
shuowei-fix-unpivot-empty
branch
from
2d1c6d4 to
f8241d7
Compare
PR created by the Librarian CLI to initialize a release. Merging this PR will auto trigger a release. Librarian Version: v0.7.0 Language Image: us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:160860d189ff1c2f7515638478823712fa5b243e27ccc33a2728669fa1e2ed0c <details><summary>bigframes: 2.39.0</summary> ## [2.39.0](v2.38.0...v2.39.0) (2026-03-31) ### Features * add `df.bigquery.ai.forecast` method to pandas dataframe accessor (#2518) ([1126cec](1126cec9)) * support full round-trip persistence for multimodal reference cols (#2511) ([494a0a1](494a0a11)) * expose DataFrame.bigquery in both pandas and bigframes DataFrames (#2533) ([69fe317](69fe3176)) ### Bug Fixes * to_gbq may swap data columns when replace table (#2532) ([17ecc65](17ecc65e)) * handle aggregate operations on empty selections (#2510) ([34fb5da](34fb5daa)) * Localize BigQuery log suppression for gbq.py (#2541) ([af49ca2](af49ca29)) * Respect remote function config changes even if logic unchanged (#2512) ([b952428](b9524284)) * support melting empty DataFrames without crashing (#2509) ([e8c4603](e8c46032)) ### Performance Improvements * Make executor data uploads async internally (#2529) ([96597f0](96597f0b)) ### Documentation * gemini retouch of the index page for seo (#2514) ([2e5311e](2e5311e2)) * Rename Blob column references to ObjectRef column (#2535) ([44e0ffd](44e0ffd9)) </details>
2 participants
Allows alignment melts over zero-row offset layouts
Fixes #<452681068> 🦕