Test failure: `test_all_grant_types`

[github-actions]

❌ test_all_grant_types: AssertionError: assert {('ANY FILE',...y_tfby8h950')} == {('ANONYMOUS ...lp2sis'), ...} (18m55.997s)

AssertionError: assert {('ANY FILE',...y_tfby8h950')} == {('ANONYMOUS ...lp2sis'), ...}
  
  Extra items in the right set:
  ('ANONYMOUS FUNCTION', None)
  
  Full diff:
    {
  -     (
  -         'ANONYMOUS FUNCTION',
  -         None,
  -     ),
        (
            'ANY FILE',
            None,
        ),
        (
            'CATALOG',
            'hive_metastore',
        ),
        (
            'DATABASE',
            'hive_metastore.dummy_svu9pz6jt',
        ),
        (
            'TABLE',
            'hive_metastore.dummy_svu9pz6jt.dummy_tlc5hugty',
        ),
        (
            'UDF',
            'hive_metastore.dummy_svu9pz6jt.dummy_fbblp2sis',
        ),
        (
            'VIEW',
            'hive_metastore.dummy_svu9pz6jt.dummy_tfby8h950',
        ),
    }
05:30 INFO [databricks.labs.ucx.install] Creating ucx schemas...
[gw5] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python
05:30 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.grants] fetching grants inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.grants] crawling new set of snapshot data for grants
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.tables] fetching tables inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.tables] crawling new set of snapshot data for tables
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_svu9pz6jt] listing tables and views
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_svu9pz6jt.dummy_tfby8h950] fetching table metadata
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_svu9pz6jt.dummy_tlc5hugty] fetching table metadata
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.tables] found 2 new records for tables
05:38 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.udfs] fetching udfs inventory
05:38 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.udfs] crawling new set of snapshot data for udfs
05:38 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_svu9pz6jt] listing udfs
05:38 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_svu9pz6jt.dummy_fbblp2sis] fetching udf metadata
05:38 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.udfs] found 1 new records for udfs
05:49 ERROR [databricks.labs.ucx.hive_metastore.grants] Couldn't fetch grants for object ANONYMOUS FUNCTION : TEMPORARILY_UNAVAILABLE: The service at /api/2.0/sql-acl/get-permissions is taking too long to process your request. Please try again later or try a faster operation. [TraceId: 00-ec8b5d21fdaa097cf79ae91f54f14578-96b8f02b5a43deea-00]
05:49 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.grants] found 10 new records for grants
05:30 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.grants] fetching grants inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.grants] crawling new set of snapshot data for grants
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.tables] fetching tables inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.tables] crawling new set of snapshot data for tables
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_svu9pz6jt] listing tables and views
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_svu9pz6jt.dummy_tfby8h950] fetching table metadata
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_svu9pz6jt.dummy_tlc5hugty] fetching table metadata
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.tables] found 2 new records for tables
05:38 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.udfs] fetching udfs inventory
05:38 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.udfs] crawling new set of snapshot data for udfs
05:38 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_svu9pz6jt] listing udfs
05:38 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_svu9pz6jt.dummy_fbblp2sis] fetching udf metadata
05:38 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.udfs] found 1 new records for udfs
05:49 ERROR [databricks.labs.ucx.hive_metastore.grants] Couldn't fetch grants for object ANONYMOUS FUNCTION : TEMPORARILY_UNAVAILABLE: The service at /api/2.0/sql-acl/get-permissions is taking too long to process your request. Please try again later or try a faster operation. [TraceId: 00-ec8b5d21fdaa097cf79ae91f54f14578-96b8f02b5a43deea-00]
05:49 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sg570kjhx.grants] found 10 new records for grants
[gw5] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #775}

[github-actions]

❌ test_all_grant_types: AssertionError: assert {('ANY FILE',...y_tu8ryy5yl')} == {('ANONYMOUS ...4qm0vv'), ...} (21m16.193s)

AssertionError: assert {('ANY FILE',...y_tu8ryy5yl')} == {('ANONYMOUS ...4qm0vv'), ...}
  
  Extra items in the right set:
  ('ANONYMOUS FUNCTION', None)
  
  Full diff:
    {
  -     (
  -         'ANONYMOUS FUNCTION',
  -         None,
  -     ),
        (
            'ANY FILE',
            None,
        ),
        (
            'CATALOG',
            'hive_metastore',
        ),
        (
            'DATABASE',
            'hive_metastore.dummy_sb9mlgcti',
        ),
        (
            'TABLE',
            'hive_metastore.dummy_sb9mlgcti.dummy_tcvvywbpm',
        ),
        (
            'UDF',
            'hive_metastore.dummy_sb9mlgcti.dummy_fjn4qm0vv',
        ),
        (
            'VIEW',
            'hive_metastore.dummy_sb9mlgcti.dummy_tu8ryy5yl',
        ),
    }
05:36 INFO [databricks.labs.ucx.install] Creating ucx schemas...
[gw7] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python
05:36 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.grants] fetching grants inventory
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.grants] crawling new set of snapshot data for grants
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.tables] fetching tables inventory
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.tables] crawling new set of snapshot data for tables
05:46 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sb9mlgcti] listing tables and views
05:46 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sb9mlgcti.dummy_tcvvywbpm] fetching table metadata
05:46 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sb9mlgcti.dummy_tu8ryy5yl] fetching table metadata
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.tables] found 2 new records for tables
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.udfs] fetching udfs inventory
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.udfs] crawling new set of snapshot data for udfs
05:46 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_sb9mlgcti] listing udfs
05:46 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_sb9mlgcti.dummy_fjn4qm0vv] fetching udf metadata
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.udfs] found 1 new records for udfs
05:57 ERROR [databricks.labs.ucx.hive_metastore.grants] Couldn't fetch grants for object ANONYMOUS FUNCTION : TEMPORARILY_UNAVAILABLE: The service at /api/2.0/sql-acl/get-permissions is taking too long to process your request. Please try again later or try a faster operation. [TraceId: 00-476110260f12717ccd1ef01b3572c066-48c8631451e962d2-00]
05:57 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.grants] found 14 new records for grants
05:36 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.grants] fetching grants inventory
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.grants] crawling new set of snapshot data for grants
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.tables] fetching tables inventory
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.tables] crawling new set of snapshot data for tables
05:46 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sb9mlgcti] listing tables and views
05:46 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sb9mlgcti.dummy_tcvvywbpm] fetching table metadata
05:46 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_sb9mlgcti.dummy_tu8ryy5yl] fetching table metadata
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.tables] found 2 new records for tables
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.udfs] fetching udfs inventory
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.udfs] crawling new set of snapshot data for udfs
05:46 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_sb9mlgcti] listing udfs
05:46 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_sb9mlgcti.dummy_fjn4qm0vv] fetching udf metadata
05:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.udfs] found 1 new records for udfs
05:57 ERROR [databricks.labs.ucx.hive_metastore.grants] Couldn't fetch grants for object ANONYMOUS FUNCTION : TEMPORARILY_UNAVAILABLE: The service at /api/2.0/sql-acl/get-permissions is taking too long to process your request. Please try again later or try a faster operation. [TraceId: 00-476110260f12717ccd1ef01b3572c066-48c8631451e962d2-00]
05:57 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sugvitjkv.grants] found 14 new records for grants
[gw7] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #779}

[github-actions]

❌ test_all_grant_types: AssertionError: assert {('ANY FILE',...y_tpwekmszg')} == {('ANONYMOUS ...pedv3h'), ...} (20m16.24s)

AssertionError: assert {('ANY FILE',...y_tpwekmszg')} == {('ANONYMOUS ...pedv3h'), ...}
  
  Extra items in the right set:
  ('ANONYMOUS FUNCTION', None)
  
  Full diff:
    {
  -     (
  -         'ANONYMOUS FUNCTION',
  -         None,
  -     ),
        (
            'ANY FILE',
            None,
        ),
        (
            'CATALOG',
            'hive_metastore',
        ),
        (
            'DATABASE',
            'hive_metastore.dummy_s2afghuqz',
        ),
        (
            'TABLE',
            'hive_metastore.dummy_s2afghuqz.dummy_tsu0ktg5g',
        ),
        (
            'UDF',
            'hive_metastore.dummy_s2afghuqz.dummy_fm1pedv3h',
        ),
        (
            'VIEW',
            'hive_metastore.dummy_s2afghuqz.dummy_tpwekmszg',
        ),
    }
05:28 INFO [databricks.labs.ucx.install] Creating ucx schemas...
[gw6] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python
05:28 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.grants] fetching grants inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.grants] crawling new set of snapshot data for grants
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.tables] fetching tables inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.tables] crawling new set of snapshot data for tables
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_s2afghuqz] listing tables and views
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_s2afghuqz.dummy_tpwekmszg] fetching table metadata
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_s2afghuqz.dummy_tsu0ktg5g] fetching table metadata
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.tables] found 2 new records for tables
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.udfs] fetching udfs inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.udfs] crawling new set of snapshot data for udfs
05:37 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_s2afghuqz] listing udfs
05:37 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_s2afghuqz.dummy_fm1pedv3h] fetching udf metadata
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.udfs] found 1 new records for udfs
05:48 ERROR [databricks.labs.ucx.hive_metastore.grants] Couldn't fetch grants for object ANONYMOUS FUNCTION : TEMPORARILY_UNAVAILABLE: The service at /api/2.0/sql-acl/get-permissions is taking too long to process your request. Please try again later or try a faster operation. [TraceId: 00-41ee322823b2af994d2c36c4fd343933-ae442cd70b8bbb99-01]
05:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.grants] found 34 new records for grants
05:28 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.grants] fetching grants inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.grants] crawling new set of snapshot data for grants
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.tables] fetching tables inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.tables] crawling new set of snapshot data for tables
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_s2afghuqz] listing tables and views
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_s2afghuqz.dummy_tpwekmszg] fetching table metadata
05:37 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.dummy_s2afghuqz.dummy_tsu0ktg5g] fetching table metadata
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.tables] found 2 new records for tables
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.udfs] fetching udfs inventory
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.udfs] crawling new set of snapshot data for udfs
05:37 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_s2afghuqz] listing udfs
05:37 DEBUG [databricks.labs.ucx.hive_metastore.udfs] [hive_metastore.dummy_s2afghuqz.dummy_fm1pedv3h] fetching udf metadata
05:37 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.udfs] found 1 new records for udfs
05:48 ERROR [databricks.labs.ucx.hive_metastore.grants] Couldn't fetch grants for object ANONYMOUS FUNCTION : TEMPORARILY_UNAVAILABLE: The service at /api/2.0/sql-acl/get-permissions is taking too long to process your request. Please try again later or try a faster operation. [TraceId: 00-41ee322823b2af994d2c36c4fd343933-ae442cd70b8bbb99-01]
05:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.dummy_sina6lp4f.grants] found 34 new records for grants
[gw6] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #791}