Add SQL Server Profiler by goodwillpunning · Pull Request #2151 · databrickslabs/lakebridge

goodwillpunning · 2025-11-17T16:20:27Z

Changes

What does this PR do?

This PR adds a new profiler for mssql.

Relevant implementation details

This profiler closely follows the implementation of the Azure Synapse profiler. However, this implementation provides a last_execution_time param for all mssql server queries, allowing for a future scheduler component to hook in an repeat queries.

Caveats/things to watch out for when reviewing:

This profiler will not support on-prem mssql servers.

Linked issues

N/A

Functionality

added relevant user documentation
added new CLI command
modified existing command: databricks labs lakebridge ...
added new profiler component
... +add your own

Tests

manually tested
added unit tests
added integration tests

codecov · 2025-11-17T16:23:15Z

Codecov Report

❌ Patch coverage is 0% with 171 lines in your changes missing coverage. Please review.
✅ Project coverage is 13.47%. Comparing base (65ce810) to head (42d0028).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
...bridge/resources/assessments/mssql/info_extract.py	0.00%	71 Missing ⚠️
...ge/resources/assessments/mssql/activity_extract.py	0.00%	46 Missing ⚠️
...idge/resources/assessments/mssql/common/queries.py	0.00%	44 Missing ⚠️
...ge/resources/assessments/mssql/common/connector.py	0.00%	5 Missing ⚠️
...ge/resources/assessments/mssql/common/functions.py	0.00%	4 Missing ⚠️
...tabricks/labs/lakebridge/assessments/_constants.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2151       +/-   ##
===========================================
- Coverage   65.23%   13.47%   -51.77%     
===========================================
  Files         100      105        +5     
  Lines        8504     8673      +169     
  Branches      875      875               
===========================================
- Hits         5548     1169     -4379     
- Misses       2769     7445     +4676     
+ Partials      187       59      -128

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-11-17T16:26:05Z

❌ 129/130 passed, 8 flaky, 1 failed, 5 skipped, 30m28s total

❌ test_recon_databricks_job_succeeds: TimeoutError: timed out after 0:20:00: (20m47.393s)

TimeoutError: timed out after 0:20:00:
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer catalog: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment=None, connection_name=None, created_at=1770742838553, created_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', effective_predictive_optimization_flag=EffectivePredictiveOptimizationFlag(value=<EnablePredictiveOptimization.DISABLE: 'DISABLE'>, inherited_from_name='primary', inherited_from_type=None), enable_predictive_optimization=<EnablePredictiveOptimization.INHERIT: 'INHERIT'>, full_name='dummy_c7y70emer', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='dummy_c7y70emer', options=None, owner='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', properties={'RemoveAfter': '2026021019'}, provider_name=None, provisioning_info=None, securable_type=<SecurableType.CATALOG: 'CATALOG'>, share_name=None, storage_location=None, storage_root=None, updated_at=1770742838553, updated_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e')
17:00 INFO [tests.integration.reconcile.conftest] Created catalog dummy_c7y70emer for recon tests
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer.dummy_sdnkweryn schema: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added schema fixture: SchemaInfo(browse_only=None, catalog_name='dummy_c7y70emer', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='dummy_c7y70emer.dummy_sdnkweryn', metastore_id=None, name='dummy_sdnkweryn', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
17:00 INFO [tests.integration.reconcile.conftest] Created schema dummy_sdnkweryn in catalog dummy_c7y70emer for recon tests
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_sdnkweryn volume: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn/dummy_sdnkweryn
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added volume fixture: VolumeInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', comment=None, created_at=1770742848084, created_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_sdnkweryn', metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='dummy_sdnkweryn', owner='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', schema_name='dummy_sdnkweryn', storage_location='abfss://labs-CLOUD_ENV-TEST_CATALOG-container@databrickslabsstorage.dfs.core.windows.net/8952c1e3-b265-4adf-98c3-6f755e2e1453/volumes/69f8c236-2ce7-46f4-8455-e2ad2cf173ab', updated_at=1770742848084, updated_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', volume_id='69f8c236-2ce7-46f4-8455-e2ad2cf173ab', volume_type=<VolumeType.MANAGED: 'MANAGED'>)
17:00 INFO [tests.integration.reconcile.test_recon_databricks] Using recon job overrides: ReconcileJobConfig(existing_cluster_id='DATABRICKS_CLUSTER_ID', tags={'RemoveAfter': '2026021019'})
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer.dummy_sdnkweryn.dummy_tjlvpkjwm schema: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn/dummy_tjlvpkjwm
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_tjlvpkjwm', metastore_id=None, name='dummy_tjlvpkjwm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2026021019'}, row_filter=None, schema_name='dummy_sdnkweryn', securable_kind_manifest=None, sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/dummy_sdnkweryn/dummy_tjlvpkjwm', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer.dummy_sdnkweryn.dummy_tq6jxx7gs schema: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn/dummy_tq6jxx7gs
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_tq6jxx7gs', metastore_id=None, name='dummy_tq6jxx7gs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2026021019'}, row_filter=None, schema_name='dummy_sdnkweryn', securable_kind_manifest=None, sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/dummy_sdnkweryn/dummy_tq6jxx7gs', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
17:00 INFO [tests.integration.reconcile.conftest] Created recon tables dummy_tjlvpkjwm, dummy_tq6jxx7gs in schema dummy_sdnkweryn
17:00 INFO [tests.integration.reconcile.conftest] Inserted data into table dummy_tjlvpkjwm and got response StatementStatus(error=None, state=<StatementState.SUCCEEDED: 'SUCCEEDED'>)
17:00 INFO [tests.integration.reconcile.conftest] Inserted data into table dummy_tq6jxx7gs and got response StatementStatus(error=None, state=<StatementState.SUCCEEDED: 'SUCCEEDED'>)
17:00 INFO [tests.integration.reconcile.test_recon_databricks] Setting up application context for recon tests
17:00 INFO [tests.integration.reconcile.test_recon_databricks] Installing app and recon configuration into workspace
17:00 DEBUG [databricks.labs.lakebridge.install] No existing version found in workspace; assuming fresh installation.
17:01 INFO [databricks.labs.lakebridge.install] Installing Lakebridge reconcile Metadata components.
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Installing reconcile components.
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Deploying reconciliation metadata tables.
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table main in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table metric in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table detai in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table aggregate_metric in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table aggregate_detai in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table aggregate_rule in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Deploying reconciliation dashboards.
17:01 INFO [databricks.labs.lakebridge.deployment.dashboard] Deploying dashboards from base folder /home/runner/work/lakebridge/lakebridge/src/databricks/labs/lakebridge/resources/reconcile/dashboards
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 INFO [databricks.labs.lakebridge.deployment.dashboard] Dashboard deployed with URL: https://DATABRICKS_HOST/sql/dashboardsv3/01f106a21a4310cc9b155656ebd1e9e1
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 INFO [databricks.labs.lakebridge.deployment.dashboard] Dashboard deployed with URL: https://DATABRICKS_HOST/sql/dashboardsv3/01f106a21b861a7cb084e1ebca350cc4
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Deploying reconciliation jobs.
17:01 INFO [databricks.labs.lakebridge.deployment.job] Deploying reconciliation job.
17:01 DEBUG [databricks.labs.lakebridge.deployment.job] Applying deployment overrides: ReconcileJobConfig(existing_cluster_id='DATABRICKS_CLUSTER_ID', tags={'RemoveAfter': '2026021019'})
17:01 DEBUG [databricks.labs.lakebridge.deployment.job] Reconciliation job task cluster: existing: DATABRICKS_CLUSTER_ID or name: None
17:01 INFO [databricks.labs.lakebridge.deployment.job] Creating new job configuration for job `Reconciliation Runner`
17:01 INFO [databricks.labs.lakebridge.deployment.job] Reconciliation job deployed with job_id=766135390220606
17:01 INFO [databricks.labs.lakebridge.deployment.job] Job URL: https://DATABRICKS_HOST#job/766135390220606
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Installation of reconcile components completed successfully.
17:01 INFO [tests.integration.reconcile.test_recon_databricks] Application context setup complete for recon tests
[gw7] linux -- Python 3.10.19 /home/runner/work/lakebridge/lakebridge/.venv/bin/python
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer catalog: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment=None, connection_name=None, created_at=1770742838553, created_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', effective_predictive_optimization_flag=EffectivePredictiveOptimizationFlag(value=<EnablePredictiveOptimization.DISABLE: 'DISABLE'>, inherited_from_name='primary', inherited_from_type=None), enable_predictive_optimization=<EnablePredictiveOptimization.INHERIT: 'INHERIT'>, full_name='dummy_c7y70emer', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='dummy_c7y70emer', options=None, owner='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', properties={'RemoveAfter': '2026021019'}, provider_name=None, provisioning_info=None, securable_type=<SecurableType.CATALOG: 'CATALOG'>, share_name=None, storage_location=None, storage_root=None, updated_at=1770742838553, updated_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e')
17:00 INFO [tests.integration.reconcile.conftest] Created catalog dummy_c7y70emer for recon tests
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer.dummy_sdnkweryn schema: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added schema fixture: SchemaInfo(browse_only=None, catalog_name='dummy_c7y70emer', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='dummy_c7y70emer.dummy_sdnkweryn', metastore_id=None, name='dummy_sdnkweryn', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
17:00 INFO [tests.integration.reconcile.conftest] Created schema dummy_sdnkweryn in catalog dummy_c7y70emer for recon tests
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_sdnkweryn volume: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn/dummy_sdnkweryn
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added volume fixture: VolumeInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', comment=None, created_at=1770742848084, created_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_sdnkweryn', metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='dummy_sdnkweryn', owner='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', schema_name='dummy_sdnkweryn', storage_location='abfss://labs-CLOUD_ENV-TEST_CATALOG-container@databrickslabsstorage.dfs.core.windows.net/8952c1e3-b265-4adf-98c3-6f755e2e1453/volumes/69f8c236-2ce7-46f4-8455-e2ad2cf173ab', updated_at=1770742848084, updated_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', volume_id='69f8c236-2ce7-46f4-8455-e2ad2cf173ab', volume_type=<VolumeType.MANAGED: 'MANAGED'>)
17:00 INFO [tests.integration.reconcile.test_recon_databricks] Using recon job overrides: ReconcileJobConfig(existing_cluster_id='DATABRICKS_CLUSTER_ID', tags={'RemoveAfter': '2026021019'})
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer.dummy_sdnkweryn.dummy_tjlvpkjwm schema: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn/dummy_tjlvpkjwm
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_tjlvpkjwm', metastore_id=None, name='dummy_tjlvpkjwm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2026021019'}, row_filter=None, schema_name='dummy_sdnkweryn', securable_kind_manifest=None, sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/dummy_sdnkweryn/dummy_tjlvpkjwm', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer.dummy_sdnkweryn.dummy_tq6jxx7gs schema: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn/dummy_tq6jxx7gs
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_tq6jxx7gs', metastore_id=None, name='dummy_tq6jxx7gs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2026021019'}, row_filter=None, schema_name='dummy_sdnkweryn', securable_kind_manifest=None, sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/dummy_sdnkweryn/dummy_tq6jxx7gs', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
17:00 INFO [tests.integration.reconcile.conftest] Created recon tables dummy_tjlvpkjwm, dummy_tq6jxx7gs in schema dummy_sdnkweryn
17:00 INFO [tests.integration.reconcile.conftest] Inserted data into table dummy_tjlvpkjwm and got response StatementStatus(error=None, state=<StatementState.SUCCEEDED: 'SUCCEEDED'>)
17:00 INFO [tests.integration.reconcile.conftest] Inserted data into table dummy_tq6jxx7gs and got response StatementStatus(error=None, state=<StatementState.SUCCEEDED: 'SUCCEEDED'>)
17:00 INFO [tests.integration.reconcile.test_recon_databricks] Setting up application context for recon tests
17:00 INFO [tests.integration.reconcile.test_recon_databricks] Installing app and recon configuration into workspace
17:00 DEBUG [databricks.labs.lakebridge.install] No existing version found in workspace; assuming fresh installation.
17:01 INFO [databricks.labs.lakebridge.install] Installing Lakebridge reconcile Metadata components.
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Installing reconcile components.
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Deploying reconciliation metadata tables.
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table main in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table metric in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table detai in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table aggregate_metric in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table aggregate_detai in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table aggregate_rule in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Deploying reconciliation dashboards.
17:01 INFO [databricks.labs.lakebridge.deployment.dashboard] Deploying dashboards from base folder /home/runner/work/lakebridge/lakebridge/src/databricks/labs/lakebridge/resources/reconcile/dashboards
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 INFO [databricks.labs.lakebridge.deployment.dashboard] Dashboard deployed with URL: https://DATABRICKS_HOST/sql/dashboardsv3/01f106a21a4310cc9b155656ebd1e9e1
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 INFO [databricks.labs.lakebridge.deployment.dashboard] Dashboard deployed with URL: https://DATABRICKS_HOST/sql/dashboardsv3/01f106a21b861a7cb084e1ebca350cc4
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Deploying reconciliation jobs.
17:01 INFO [databricks.labs.lakebridge.deployment.job] Deploying reconciliation job.
17:01 DEBUG [databricks.labs.lakebridge.deployment.job] Applying deployment overrides: ReconcileJobConfig(existing_cluster_id='DATABRICKS_CLUSTER_ID', tags={'RemoveAfter': '2026021019'})
17:01 DEBUG [databricks.labs.lakebridge.deployment.job] Reconciliation job task cluster: existing: DATABRICKS_CLUSTER_ID or name: None
17:01 INFO [databricks.labs.lakebridge.deployment.job] Creating new job configuration for job `Reconciliation Runner`
17:01 INFO [databricks.labs.lakebridge.deployment.job] Reconciliation job deployed with job_id=766135390220606
17:01 INFO [databricks.labs.lakebridge.deployment.job] Job URL: https://DATABRICKS_HOST#job/766135390220606
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Installation of reconcile components completed successfully.
17:01 INFO [tests.integration.reconcile.test_recon_databricks] Application context setup complete for recon tests
17:01 DEBUG [databricks.labs.lakebridge.reconcile.runner] Reconcile job id found in the install state.
17:01 INFO [databricks.labs.lakebridge.reconcile.runner] Triggering the reconcile job with job_id: `766135390220606`
17:01 INFO [databricks.labs.lakebridge.reconcile.runner] 'RECONCILE' job started. Please check the job_url `https://DATABRICKS_HOST/jobs/766135390220606/runs/76236922761755` for the current status.
17:21 INFO [tests.integration.reconcile.test_recon_databricks] Reconcile job run had 1 tasks
17:21 INFO [tests.integration.reconcile.test_recon_databricks] Task run_reconciliation has error message: No output is available until the task begins.
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer catalog: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment=None, connection_name=None, created_at=1770742838553, created_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', effective_predictive_optimization_flag=EffectivePredictiveOptimizationFlag(value=<EnablePredictiveOptimization.DISABLE: 'DISABLE'>, inherited_from_name='primary', inherited_from_type=None), enable_predictive_optimization=<EnablePredictiveOptimization.INHERIT: 'INHERIT'>, full_name='dummy_c7y70emer', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='dummy_c7y70emer', options=None, owner='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', properties={'RemoveAfter': '2026021019'}, provider_name=None, provisioning_info=None, securable_type=<SecurableType.CATALOG: 'CATALOG'>, share_name=None, storage_location=None, storage_root=None, updated_at=1770742838553, updated_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e')
17:00 INFO [tests.integration.reconcile.conftest] Created catalog dummy_c7y70emer for recon tests
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer.dummy_sdnkweryn schema: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added schema fixture: SchemaInfo(browse_only=None, catalog_name='dummy_c7y70emer', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='dummy_c7y70emer.dummy_sdnkweryn', metastore_id=None, name='dummy_sdnkweryn', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
17:00 INFO [tests.integration.reconcile.conftest] Created schema dummy_sdnkweryn in catalog dummy_c7y70emer for recon tests
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_sdnkweryn volume: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn/dummy_sdnkweryn
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added volume fixture: VolumeInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', comment=None, created_at=1770742848084, created_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_sdnkweryn', metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='dummy_sdnkweryn', owner='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', schema_name='dummy_sdnkweryn', storage_location='abfss://labs-CLOUD_ENV-TEST_CATALOG-container@databrickslabsstorage.dfs.core.windows.net/8952c1e3-b265-4adf-98c3-6f755e2e1453/volumes/69f8c236-2ce7-46f4-8455-e2ad2cf173ab', updated_at=1770742848084, updated_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', volume_id='69f8c236-2ce7-46f4-8455-e2ad2cf173ab', volume_type=<VolumeType.MANAGED: 'MANAGED'>)
17:00 INFO [tests.integration.reconcile.test_recon_databricks] Using recon job overrides: ReconcileJobConfig(existing_cluster_id='DATABRICKS_CLUSTER_ID', tags={'RemoveAfter': '2026021019'})
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer.dummy_sdnkweryn.dummy_tjlvpkjwm schema: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn/dummy_tjlvpkjwm
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_tjlvpkjwm', metastore_id=None, name='dummy_tjlvpkjwm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2026021019'}, row_filter=None, schema_name='dummy_sdnkweryn', securable_kind_manifest=None, sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/dummy_sdnkweryn/dummy_tjlvpkjwm', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
17:00 INFO [databricks.labs.pytester.fixtures.baseline] Created dummy_c7y70emer.dummy_sdnkweryn.dummy_tq6jxx7gs schema: https://DATABRICKS_HOST/#explore/data/dummy_c7y70emer/dummy_sdnkweryn/dummy_tq6jxx7gs
17:00 DEBUG [databricks.labs.pytester.fixtures.baseline] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_tq6jxx7gs', metastore_id=None, name='dummy_tq6jxx7gs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2026021019'}, row_filter=None, schema_name='dummy_sdnkweryn', securable_kind_manifest=None, sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/dummy_sdnkweryn/dummy_tq6jxx7gs', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
17:00 INFO [tests.integration.reconcile.conftest] Created recon tables dummy_tjlvpkjwm, dummy_tq6jxx7gs in schema dummy_sdnkweryn
17:00 INFO [tests.integration.reconcile.conftest] Inserted data into table dummy_tjlvpkjwm and got response StatementStatus(error=None, state=<StatementState.SUCCEEDED: 'SUCCEEDED'>)
17:00 INFO [tests.integration.reconcile.conftest] Inserted data into table dummy_tq6jxx7gs and got response StatementStatus(error=None, state=<StatementState.SUCCEEDED: 'SUCCEEDED'>)
17:00 INFO [tests.integration.reconcile.test_recon_databricks] Setting up application context for recon tests
17:00 INFO [tests.integration.reconcile.test_recon_databricks] Installing app and recon configuration into workspace
17:00 DEBUG [databricks.labs.lakebridge.install] No existing version found in workspace; assuming fresh installation.
17:01 INFO [databricks.labs.lakebridge.install] Installing Lakebridge reconcile Metadata components.
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Installing reconcile components.
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Deploying reconciliation metadata tables.
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table main in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table metric in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table detai in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table aggregate_metric in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table aggregate_detai in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.table] Deploying table aggregate_rule in dummy_c7y70emer.dummy_sdnkweryn
17:01 INFO [databricks.labs.lakebridge.deployment.table] SQL Backend used for deploying table: StatementExecutionBackend
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Deploying reconciliation dashboards.
17:01 INFO [databricks.labs.lakebridge.deployment.dashboard] Deploying dashboards from base folder /home/runner/work/lakebridge/lakebridge/src/databricks/labs/lakebridge/resources/reconcile/dashboards
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 INFO [databricks.labs.lakebridge.deployment.dashboard] Dashboard deployed with URL: https://DATABRICKS_HOST/sql/dashboardsv3/01f106a21a4310cc9b155656ebd1e9e1
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
17:01 INFO [databricks.labs.lakebridge.deployment.dashboard] Dashboard deployed with URL: https://DATABRICKS_HOST/sql/dashboardsv3/01f106a21b861a7cb084e1ebca350cc4
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Deploying reconciliation jobs.
17:01 INFO [databricks.labs.lakebridge.deployment.job] Deploying reconciliation job.
17:01 DEBUG [databricks.labs.lakebridge.deployment.job] Applying deployment overrides: ReconcileJobConfig(existing_cluster_id='DATABRICKS_CLUSTER_ID', tags={'RemoveAfter': '2026021019'})
17:01 DEBUG [databricks.labs.lakebridge.deployment.job] Reconciliation job task cluster: existing: DATABRICKS_CLUSTER_ID or name: None
17:01 INFO [databricks.labs.lakebridge.deployment.job] Creating new job configuration for job `Reconciliation Runner`
17:01 INFO [databricks.labs.lakebridge.deployment.job] Reconciliation job deployed with job_id=766135390220606
17:01 INFO [databricks.labs.lakebridge.deployment.job] Job URL: https://DATABRICKS_HOST#job/766135390220606
17:01 INFO [databricks.labs.lakebridge.deployment.recon] Installation of reconcile components completed successfully.
17:01 INFO [tests.integration.reconcile.test_recon_databricks] Application context setup complete for recon tests
17:01 DEBUG [databricks.labs.lakebridge.reconcile.runner] Reconcile job id found in the install state.
17:01 INFO [databricks.labs.lakebridge.reconcile.runner] Triggering the reconcile job with job_id: `766135390220606`
17:01 INFO [databricks.labs.lakebridge.reconcile.runner] 'RECONCILE' job started. Please check the job_url `https://DATABRICKS_HOST/jobs/766135390220606/runs/76236922761755` for the current status.
17:21 INFO [tests.integration.reconcile.test_recon_databricks] Reconcile job run had 1 tasks
17:21 INFO [tests.integration.reconcile.test_recon_databricks] Task run_reconciliation has error message: No output is available until the task begins.
17:21 INFO [tests.integration.reconcile.test_recon_databricks] Tearing down application context for recon tests
17:21 INFO [databricks.labs.lakebridge.install] Uninstalling Lakebridge from https://DATABRICKS_HOST.
17:21 INFO [databricks.labs.lakebridge.deployment.recon] Uninstalling reconcile components.
17:21 INFO [databricks.labs.lakebridge.deployment.recon] Removing reconciliation dashboards.
17:21 INFO [databricks.labs.lakebridge.deployment.recon] Removing dashboard with id=01f106a21a4310cc9b155656ebd1e9e1.
17:21 INFO [databricks.labs.lakebridge.deployment.recon] Removing dashboard with id=01f106a21b861a7cb084e1ebca350cc4.
17:21 INFO [databricks.labs.lakebridge.deployment.recon] Removing Reconciliation Jobs.
17:21 INFO [databricks.labs.lakebridge.deployment.recon] Removing job Reconciliation Runner with job_id=766135390220606.
17:21 INFO [databricks.labs.lakebridge.install] Uninstallation completed successfully.
17:21 INFO [tests.integration.reconcile.test_recon_databricks] Application context teardown complete for recon tests
17:21 DEBUG [databricks.labs.pytester.fixtures.baseline] clearing 2 table fixtures
17:21 DEBUG [databricks.labs.pytester.fixtures.baseline] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_tjlvpkjwm', metastore_id=None, name='dummy_tjlvpkjwm', owner=None, pipeline_id=None, properties={'RemoveAfter': '2026021019'}, row_filter=None, schema_name='dummy_sdnkweryn', securable_kind_manifest=None, sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/dummy_sdnkweryn/dummy_tjlvpkjwm', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
17:21 DEBUG [databricks.labs.pytester.fixtures.baseline] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_tq6jxx7gs', metastore_id=None, name='dummy_tq6jxx7gs', owner=None, pipeline_id=None, properties={'RemoveAfter': '2026021019'}, row_filter=None, schema_name='dummy_sdnkweryn', securable_kind_manifest=None, sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/dummy_sdnkweryn/dummy_tq6jxx7gs', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
17:21 DEBUG [databricks.labs.pytester.fixtures.baseline] clearing 1 volume fixtures
17:21 DEBUG [databricks.labs.pytester.fixtures.baseline] removing volume fixture: VolumeInfo(access_point=None, browse_only=None, catalog_name='dummy_c7y70emer', comment=None, created_at=1770742848084, created_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', encryption_details=None, full_name='dummy_c7y70emer.dummy_sdnkweryn.dummy_sdnkweryn', metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='dummy_sdnkweryn', owner='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', schema_name='dummy_sdnkweryn', storage_location='abfss://labs-CLOUD_ENV-TEST_CATALOG-container@databrickslabsstorage.dfs.core.windows.net/8952c1e3-b265-4adf-98c3-6f755e2e1453/volumes/69f8c236-2ce7-46f4-8455-e2ad2cf173ab', updated_at=1770742848084, updated_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', volume_id='69f8c236-2ce7-46f4-8455-e2ad2cf173ab', volume_type=<VolumeType.MANAGED: 'MANAGED'>)
17:21 DEBUG [databricks.labs.pytester.fixtures.baseline] clearing 1 schema fixtures
17:21 DEBUG [databricks.labs.pytester.fixtures.baseline] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='dummy_c7y70emer', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='dummy_c7y70emer.dummy_sdnkweryn', metastore_id=None, name='dummy_sdnkweryn', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
17:21 DEBUG [databricks.labs.pytester.fixtures.baseline] clearing 1 catalog fixtures
17:21 DEBUG [databricks.labs.pytester.fixtures.baseline] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment=None, connection_name=None, created_at=1770742838553, created_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', effective_predictive_optimization_flag=EffectivePredictiveOptimizationFlag(value=<EnablePredictiveOptimization.DISABLE: 'DISABLE'>, inherited_from_name='primary', inherited_from_type=None), enable_predictive_optimization=<EnablePredictiveOptimization.INHERIT: 'INHERIT'>, full_name='dummy_c7y70emer', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='dummy_c7y70emer', options=None, owner='3fe685a1-96cc-4fec-8cdb-6944f5c9787e', properties={'RemoveAfter': '2026021019'}, provider_name=None, provisioning_info=None, securable_type=<SecurableType.CATALOG: 'CATALOG'>, share_name=None, storage_location=None, storage_root=None, updated_at=1770742838553, updated_by='3fe685a1-96cc-4fec-8cdb-6944f5c9787e')
[gw7] linux -- Python 3.10.19 /home/runner/work/lakebridge/lakebridge/.venv/bin/python

Flaky tests:

🤪 test_installs_and_runs_local_bladebridge (19.962s)
🤪 test_installs_and_runs_pypi_bladebridge (28.055s)
🤪 test_transpiles_informatica_to_sparksql_non_interactive[True] (14.515s)
🤪 test_transpiles_informatica_to_sparksql_non_interactive[False] (14.567s)
🤪 test_transpiles_informatica_to_sparksql (14.22s)
🤪 test_transpile_teradata_sql_non_interactive[True] (16.08s)
🤪 test_transpile_teradata_sql (5.682s)
🤪 test_transpile_teradata_sql_non_interactive[False] (13.101s)

_{Running from acceptance #3666}

sundarshankar89 · 2025-12-05T11:52:21Z

src/databricks/labs/lakebridge/resources/assessments/mssql/common/connector.py

+from databricks.labs.lakebridge.connections.database_manager import DatabaseManager
+
+
+def get_sqlserver_reader(


Database manager should already have the MSSQLConnector we should be using that.

sundarshankar89 · 2025-12-05T11:53:37Z

src/databricks/labs/lakebridge/resources/assessments/mssql/pipeline_config.yml

+steps:
+  - name: activity_extract
+    type: python
+    extract_source: src/databricks/labs/lakebridge/resources/assessments/mssql/activity_extract.py


Can't we just run sql queries directly? do we need python wrapper for SQL Server profilers?

goodwillpunning added 2 commits November 17, 2025 10:24

Add initial MSSQL profiler files

5e4ff3d

Add activity extract tables.

bd40c8a

goodwillpunning requested a review from a team as a code owner November 17, 2025 16:20

goodwillpunning had a problem deploying to tool November 17, 2025 16:20 — with GitHub Actions Error

goodwillpunning added the feat/profiler Issues related to profilers label Nov 17, 2025

Merge branch 'main' into feature/add_mssql_profiler

4ba54c3

goodwillpunning temporarily deployed to tool November 17, 2025 16:20 — with GitHub Actions Inactive

Add info extract queries.

46f5e75

goodwillpunning temporarily deployed to tool November 17, 2025 21:58 — with GitHub Actions Inactive

Add info extraction step.

866bfc7

goodwillpunning temporarily deployed to tool November 18, 2025 13:29 — with GitHub Actions Inactive

Update MSSQL configuration

6db273d

goodwillpunning temporarily deployed to tool November 19, 2025 22:35 — with GitHub Actions Inactive

Update common DuckDB helpers with MSSQL schemas

60c38d8

goodwillpunning temporarily deployed to tool November 25, 2025 21:28 — with GitHub Actions Inactive

Add missing pipeline dependency

adbaf7a

goodwillpunning temporarily deployed to tool November 25, 2025 21:39 — with GitHub Actions Inactive

Merge branch 'main' into feature/add_mssql_profiler

5a8185f

goodwillpunning temporarily deployed to tool November 26, 2025 15:49 — with GitHub Actions Inactive

goodwillpunning added 2 commits November 26, 2025 14:05

Update MSSQL prompt.

6a1c150

Fix arg parsing.

777e9d1

goodwillpunning temporarily deployed to tool November 26, 2025 19:06 — with GitHub Actions Inactive

Remove loop through all SQL servers in subscription.

1d39231

goodwillpunning temporarily deployed to tool November 26, 2025 19:20 — with GitHub Actions Inactive

Refactor info extract script.

60a1f05

goodwillpunning temporarily deployed to tool December 1, 2025 17:17 — with GitHub Actions Inactive

Simplify config prompts.

42d0028

goodwillpunning temporarily deployed to tool December 1, 2025 17:34 — with GitHub Actions Inactive

sundarshankar89 reviewed Dec 5, 2025

View reviewed changes

Merge branch 'main' into feature/add_mssql_profiler

06241d5

goodwillpunning temporarily deployed to tool January 12, 2026 17:02 — with GitHub Actions Inactive

Merge branch 'main' into feature/add_mssql_profiler

8bc9cfb

goodwillpunning temporarily deployed to tool January 30, 2026 17:32 — with GitHub Actions Inactive

Add SQL task queries.

cd230e1

goodwillpunning temporarily deployed to tool January 30, 2026 19:45 — with GitHub Actions Inactive

Update mssql pipeline with SQL tasks.

726fdec

goodwillpunning had a problem deploying to tool February 2, 2026 16:26 — with GitHub Actions Failure

asnare assigned sundarshankar89 Feb 4, 2026

sundarshankar89 assigned goodwillpunning Feb 6, 2026

Merge branch 'main' into feature/add_mssql_profiler

a816961

goodwillpunning had a problem deploying to tool February 10, 2026 16:57 — with GitHub Actions Failure

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add SQL Server Profiler#2151

Add SQL Server Profiler#2151
goodwillpunning wants to merge 19 commits intomainfrom
feature/add_mssql_profiler

goodwillpunning commented Nov 17, 2025

Uh oh!

codecov bot commented Nov 17, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 17, 2025 •

edited

Loading

Uh oh!

sundarshankar89 Dec 5, 2025

Uh oh!

sundarshankar89 Dec 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		from databricks.labs.lakebridge.connections.database_manager import DatabaseManager


		def get_sqlserver_reader(

Conversation

goodwillpunning commented Nov 17, 2025

Changes

What does this PR do?

Relevant implementation details

Caveats/things to watch out for when reviewing:

Linked issues

Functionality

Tests

Uh oh!

codecov bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sundarshankar89 Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

sundarshankar89 Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Nov 17, 2025 •

edited

Loading

github-actions bot commented Nov 17, 2025 •

edited

Loading