-
Notifications
You must be signed in to change notification settings - Fork 11
Adds Databricks backend #325
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
isabelizimm
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a few methods that don't need to be staticmethod (anything that uses self won't need a staticmethod decorator), but apart from that this looks good!
FYI that the failing tests are unrelated-- I have a few pieces of maintenance that I'll need to merge before this can go in.
| # putting imports inside rsconnect particulars for now | ||
|
|
||
|
|
||
| def skip_if_dbc(func): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Due to some boring housekeeping reasons, our test boards are all read-only right now for Databricks. That means things we test a few things like pin_read and pin_list but do NOT test pin_write and many other functionalities.
I've added this decorator + lots of TODOs to note all the tests we are skipping on dbc 😬 We'll release this as-is for now, but plan to change to make this fully tested in the future.
Test results:
Overview:
Following the same pattern for Posit Connect, it adds a "stub"
fsspecspec to create the integration. It usesdatabricks-sdkto authenticate and interact with Databricks.Details:
board_databricks()function that follows the same pattern of the otherboardsAbstractFileSystem(fsspec) namedDatabricksFs, which is called by the board constructor, just like it does with Posit Connect boardsDatabricksFsoverridesls,exists,open,get,mkdir,get,putandrmBoardBuilderfor testing.DbcBoardBuilderoverrides__init__,create_tmp_board,teardown_board, andteardown