Recommended way to load DBT assets

[chleech]

Hey all, what is the recommended way to load DBT assets into Dagster?

The tutorial uses the dbt.cli function like

@dbt_assets(manifest=dbt_manifest_path)
def jaffle_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource):
    yield from dbt.cli(["build"], context=context).stream()

but there’s also this method load_assets_from_dbt_project.

How do I decide one over the other?

[rexledesma]

load_assets_from_dbt_project was the API for loading your dbt project as software-defined assets in dagster-dbt<0.20.0. It was kept around for backwards compatability.

In dagster-dbt>=0.20.0, the @dbt_assets decorator and DbtCliResource was released. These two APIs give more flexibility when interacting with dbt and Dagster. One example of this flexibility is that you can modify the compute function of your @dbt_assets decorated function.

Explicitly:

@dbt_assets(manifest=dbt_manifest_path)
def jaffle_dbt_assets(context: AssetExecutionContext, dbt: DbtCliResource):
    print("I can run custom code before my dbt invocation")
    
    yield from dbt.cli(["build"], context=context).stream()
    
    print("I can run custom code after my dbt invocation")

With load_assets_from_dbt_project, you could not accomplish such behavior.

We highly recommend that you use the dagster-dbt>=0.20.0 APIs (e.g. @dbt_assets, DbtCliResource, DagsterDbtTranslator) since the tutorial and docs are tailored towards them.

Resources:

• Tutorial: https://docs.dagster.io/integrations/dbt/using-dbt-with-dagster
• Reference: https://docs.dagster.io/integrations/dbt/reference
• API Docs: https://docs.dagster.io/_apidocs/libraries/dagster-dbt