Skip to content

[Stream 1049] Feature fsc evm livequery #107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jensenity wants to merge 16 commits into
base: main
Choose a base branch
from
Open

[Stream 1049] Feature fsc evm livequery #107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
397c698
fsc-evm udtfs
jensenity Feb 9, 2025
35e16e5
update changes from shah's branch
jensenity Feb 9, 2025
d8f4af4
update row_count variable
jensenity Feb 10, 2025
2a7308b
add row_count as signature
jensenity Feb 10, 2025
5584885
add ez_native_transfers udtf
jensenity Feb 10, 2025
0f31a43
update
jensenity Feb 11, 2025
1069980
update temp
jensenity Feb 11, 2025
8e29a5a
update tf transactions
jensenity Feb 11, 2025
dce1513
update decoded event logs
jensenity Feb 11, 2025
41579c9
update decoded log bronze logic
jensenity Feb 12, 2025
a6c441d
add evm ez token transfers
jensenity Feb 12, 2025
57ba812
add stored procedure macro
jensenity Feb 13, 2025
f48d5a3
temp, fix token transfers
jensenity Feb 24, 2025
54c94c2
uncomment ez decoded event logs
jensenity Apr 4, 2025
63ef741
Merge branch 'main' into STREAM-1049/feature-fsc-evm-livequery
jensenity Apr 4, 2025
7e48c89
fix
jensenity Apr 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -462,20 +462,20 @@ livequery:

To control the creation of UDF or SP macros with dbt run:

* UPDATE_UDFS_AND_SPS
* LQ_UPDATE_UDFS_AND_SPS
When True, executes all macros included in the on-run-start hooks within dbt_project.yml on model run as normal
When False, none of the on-run-start macros are executed on model run

Default values are False

* Usage:
`dbt run --vars '{"UPDATE_UDFS_AND_SPS":True}' -m ...`
`dbt run --var '{"LQ_UPDATE_UDFS_AND_SPS":True}' -m ...`

Dropping and creating udfs can also be done without running a model:

```sh
dbt run-operation create_udfs --vars '{"UPDATE_UDFS_AND_SPS":True}' --args '{"drop_":false}'
dbt run-operation create_udfs --vars '{"UPDATE_UDFS_AND_SPS":True}' --args '{"drop_":true}'
dbt run-operation create_udfs --vars '{"LQ_UPDATE_UDFS_AND_SPS":True}' --args '{"drop_":false}'
dbt run-operation create_udfs --vars '{"LQ_UPDATE_UDFS_AND_SPS":True}' --args '{"drop_":true}'
```

## Resources
Expand Down
44 changes: 41 additions & 3 deletions dbt_project.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,43 @@ models:
+tags: evm
marketplace:
+tags: marketplace
overrides:
+materialized: ephemeral
fsc_evm:
+enabled: false # disable package by default
+materialized: ephemeral
decoder_package:
decoded_logs:
gold:
core__ez_decoded_event_logs:
+enabled: true
silver:
silver__decoded_logs:
+enabled: true
main_package:
core:
gold:
core__fact_blocks:
+enabled: true
core__fact_transactions:
+enabled: true
core__fact_event_logs:
+enabled: true
core__fact_traces:
+enabled: true
core__ez_native_transfers:
+enabled: true
core__ez_token_transfers:
+enabled: true
silver:
silver__blocks:
+enabled: true
silver__transactions:
+enabled: true
silver__receipts:
+enabled: true
silver__traces:
+enabled: true

tests:
+store_failures: true # all tests
Expand All @@ -40,7 +77,7 @@ tests:
marketplace:
blockpour:
# TODO: enable tests for blockpour once we get an API key
+enabled: false
+enabled: false

on-run-start:
- "{{ create_sps() }}"
Expand All @@ -57,7 +94,7 @@ on-run-end:

vars:
"dbt_date:time_zone": GMT
UPDATE_UDFS_AND_SPS: false
LQ_UPDATE_UDFS_AND_SPS: false
DROP_UDFS_AND_SPS: false
UPDATE_SNOWFLAKE_TAGS: true
STREAMLINE_INVOKE_STREAMS: False
Expand All @@ -67,9 +104,10 @@ vars:
API_INTEGRATION: '{{ var("config")[target.name]["API_INTEGRATION"] }}'
EXTERNAL_FUNCTION_URI: '{{ var("config")[target.name]["EXTERNAL_FUNCTION_URI"] }}'
ROLES: '{{ var("config")[target.name]["ROLES"] }}'
GLOBAL_PROD_DB_NAME: 'LIVEQUERY_DEV'

config:
# The keys correspond to dbt profiles and are case sensitive
# The keys correspond to dbt profiles and are case sensitive
dev:
API_INTEGRATION: AWS_LIVE_QUERY_STG
EXTERNAL_FUNCTION_URI: u5z0tu43sc.execute-api.us-east-1.amazonaws.com/stg/
Expand Down
2 changes: 1 addition & 1 deletion macros/create_sps.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% macro create_sps() %}
{% if var("UPDATE_UDFS_AND_SPS") %}
{% if var("LQ_UPDATE_UDFS_AND_SPS") %}
{% if target.database == 'LIVEQUERY' %}
CREATE schema IF NOT EXISTS _internal;
{{ sp_create_prod_clone('_internal') }};
Expand Down
167 changes: 79 additions & 88 deletions macros/evm/evm.yaml.sql
View file
Open in desktop

Large diffs are not rendered by default.

Loading