Update Switch documentation for informatica and custom_etl#2310
Update Switch documentation for informatica and custom_etl#2310
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2310 +/- ##
=======================================
Coverage 66.44% 66.44%
=======================================
Files 99 99
Lines 9090 9090
Branches 974 974
=======================================
Hits 6040 6040
Misses 2874 2874
Partials 176 176 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
✅ 143/143 passed, 7 flaky, 5 skipped, 31m45s total Flaky tests:
Running from acceptance #3936 |
| sdp_language: "python" | ||
| ``` | ||
|
|
||
| Next, edit `custom_etl.yml` (located in `.lakebridge/switch/resources/builtin_prompts/` in your workspace root) to include your custom instructions. You can use other prompt yaml files as examples. |
There was a problem hiding this comment.
I'm assuming users would replace the placeholders ({ADD_YOUR_CUSTOM_DEFINITION_HERE}, {ADD_YOUR_CUSTOM_INSTRUCTIONS_HERE}, {ADD_YOUR_CUSTOM_INPUT_HERE}, {ADD_YOUR_CUSTOM_OUTPUT_HERE}) with their own content? If so, it'd be good to mention that explicitly. The current wording just says to edit custom_etl.yml but doesn't call out which placeholders to replace.
| ### Example 5: Informatica ETL to Databricks SDP in SQL | ||
|
|
||
| Convert PySpark ETL workload to Databricks Spark Declarative Pipeline in SQL: | ||
| Convert Informatica ETL workload to Databricks Spark Declarative Pipeline in SQL: |
There was a problem hiding this comment.
Just to confirm: Could you share the reasoning behind replacing the PySpark ETL example with Informatica? I'm guessing it was to keep the number of examples manageable?
What does this PR do?
Update switch documentation for https://github.com/databrickslabs/switch/pull/68
Relevant implementation details
Caveats/things to watch out for when reviewing:
Linked issues
Resolves #..
Functionality
databricks labs lakebridge ...Tests