Skip to content

nanodbc: handle SQL_NO_TOTAL when retrieving binary data in chunks - #1025

Open
jeroen wants to merge 1 commit into
mainfrom
nanodbc-binary-sql-no-total
Open

jeroen wants to merge 1 commit into
mainfrom
nanodbc-binary-sql-no-total

Conversation

@jeroen

@jeroen jeroen commented Aug 21, 2026

Copy link
Copy Markdown
Member

Fixes #1024.

Problem

When retrieving unbound SQL_C_BINARY columns, nanodbc reads the value in 1024-byte chunks via repeated SQLGetData calls. The loop in get_ref_impl<std::vector<std::uint8_t>> only appended a chunk when the length indicator was positive:

if (ValueLenOrInd > 0) { ... out.insert(...) ... }

Per the ODBC spec, a driver may instead set the indicator to SQL_NO_TOTAL (−4) on intermediate SQL_SUCCESS_WITH_INFO calls when it does not know the remaining data length — which is what the Databricks/Simba Spark driver does. Since −4 fails the > 0 test, every full 1024-byte chunk was silently discarded, and only the final chunk (returned with SQL_SUCCESS and a real length) was kept. That yields exactly the behavior reported in #1024: n mod 1024 bytes returned, with exact multiples of 1024 surviving intact.

(One note on the issue's analysis: the returned bytes are the last chunk, not a prefix — the reporter's prefix check used a period-256 payload with a length that is a multiple of 1024, where the two are byte-identical.)

Fix

Handle SQL_NO_TOTAL by appending the full buffer, exactly mirroring the existing SQL_C_CHAR and SQL_C_WCHAR blob paths a few lines above (which already have this branch). On SQL_SUCCESS_WITH_INFO/01004 with SQL_C_BINARY the driver fills the entire buffer, so appending buffer_size is correct.

Testing

There is no regression test because reproducing this requires a driver that reports SQL_NO_TOTAL for binary chunks (the Simba Spark driver does; the drivers in CI report actual remaining lengths, which take the already-correct ValueLenOrInd > 0 path). Verification against a Databricks warehouse per the repro in #1024 would be welcome.

Upstream nanodbc main has the identical gap in the binary path; I'd suggest upstreaming this patch as well.

🤖 Generated with Claude Code

When retrieving unbound SQL_C_BINARY data, drivers may set the length
indicator to SQL_NO_TOTAL (-4) on intermediate SQL_SUCCESS_WITH_INFO
calls when the remaining data length is unknown. The retrieval loop in
get_ref_impl<std::vector<std::uint8_t>> only appended chunks when the
indicator was positive, so every full 1024-byte chunk was silently
dropped and only the final chunk was kept, returning n mod 1024 bytes.

Handle SQL_NO_TOTAL by appending the full buffer, mirroring the
existing SQL_C_CHAR and SQL_C_WCHAR blob paths.

Fixes #1024.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@jeroen

jeroen commented Aug 21, 2026

Copy link
Copy Markdown
Member Author

Upstreamed to nanodbc as nanodbc/nanodbc#475.

@jeroen
jeroen requested a review from simonpcouch August 21, 2026 12:47
@andrie

andrie commented Aug 22, 2026

Copy link
Copy Markdown

Removing the comment. I meant to post this in #1024

@simonpcouch simonpcouch left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, looks good. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BINARY columns silently truncated to n mod 1024 bytes (Databricks/Spark ODBC; pyodbc unaffected)

3 participants