🐛 [firestore-bigquery-export] Streaming is not working properly

[aya-z-arketa]

Describe your configuration

• Extension name: firestore-bigquery-export-service-cat
• Extension version: firebase/firestore-bigquery-export@0.2.3
• Configuration values (redact info where appropriate):
  • Firestore Project ID: sutra-prod
  • BigQuery Project ID: sutra-prod
  • Firestore Instance Location: nam5
  • Collection path: users/{userId}/serviceCategories
  • Enable Wildcard Column field with Parent Firestore Document IDs (Optional): true
  • Dataset ID: firestore_export_service_categories
  • Table ID: service_categories

everything else was kept as default values

Describe the problem

Steps to reproduce:

I created a new document in one of the collections and it was not streamed into bigquery. I took this loom video to show the problem. Apologies for the background noise.

Expected result

The document should appear in my bigquery export within seconds.

Actual result

The document does not get sent to bigquery.

[aya-z-arketa]

@cabljac let me know if I can provide any additional info! I downgraded to 0.1.58 and it works great.

[cabljac]

Thanks! I'll investigate further and keep you updated!

[cabljac]

Hi, if you go into the logs of the cloud function managed by the extension, do you see any errors?

I've found with the new v2 functions that it is difficult to actually find these logs, i think another issue with the platform.

if you go to the cloud run page for the function in GCP console, something like this:

https://console.cloud.google.com/run/detail/<FUNCTION_LOCATION>/ext-<EXTENSION_INSTANCE_ID>-fsexportbigquery/logs

(make sure you're in the correct GCP project)

Or you can click on view logs here for fsbigqueryexport

and then copy and paste the function id in the search