Skip to content

Fix : Unable to poll results in async API #515

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jprakash-db merged 5 commits into from
Feb 28, 2025
Merged

Fix : Unable to poll results in async API #515

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ed0f73b
Fixed the async issue
jprakash-db Feb 25, 2025
3b51b4e
Added unit tests
jprakash-db Feb 27, 2025
d84c15b
Minor change
jprakash-db Feb 27, 2025
0c0ea0c
Changed to 5
jprakash-db Feb 27, 2025
f2a01b1
Increased time
jprakash-db Feb 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/databricks/sql/thrift_backend.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -898,8 +898,10 @@ def execute_command(
sessionHandle=session_handle,
statement=operation,
runAsync=True,
# For async operation we don't want the direct results
getDirectResults=ttypes.TSparkGetDirectResults(
maxRows=max_rows, maxBytes=max_bytes
maxRows=0 if async_op else max_rows,
Comment thread
jprakash-db marked this conversation as resolved.
Outdated

@jayantsing-db jayantsing-db Feb 27, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of setting maxRows as 0, I would recommend not specifying the direct result object in the request altogether. This is to make the intention clear of not expecting the direct results.

@jprakash-db jprakash-db Feb 27, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have made the object None

maxBytes=0 if async_op else max_bytes,
),
Comment on lines +902 to 907

@jayantsing-db jayantsing-db Feb 27, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have a linter for python driver yet? if not, could you please open an issue to community or internally?

@jprakash-db jprakash-db Feb 27, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jayantsing-db I can raise a new PR for that

canReadArrowResult=True if pyarrow else False,
canDecompressLZ4Result=lz4_compression,
Expand Down