Paginate through Account Groups list request

[pritishpai]

Changes

Customer facing issues where an account group is not matching. This is due to the list account groups not paginating through the API calls. Since it is a bigger change and an account api client is not always available at workspace level, we replicate the sdk pagination here.

Tests

• manually tested
• added unit tests
• added integration tests
• verified on staging environment (screenshot attached)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.80%. Comparing base (d7a1c55) to head (8cbeb6f).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4730      +/-   ##
==========================================
- Coverage   87.95%   87.80%   -0.15%     
==========================================
  Files         123      123              
  Lines       17570    17583      +13     
  Branches     3714     3717       +3     
==========================================
- Hits        15453    15439      -14     
- Misses       1425     1456      +31     
+ Partials      692      688       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

❌ 58/59 passed, 1 failed, 5 skipped, 3h46m33s total

❌ test_permission_for_files_anonymous_func_migration_api: TimeoutError: Timed out after 0:02:00 (11m15.883s)

TimeoutError: Timed out after 0:02:00
[gw3] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups.
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-whZRtOaj-ra78c2a421 (workspace) -> sdk-Mn0BBpZ0-ra78c2a421 (account) starting
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-whZRtOaj-ra78c2a421 (workspace) -> sdk-Mn0BBpZ0-ra78c2a421 (account) progress=2(+2)
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-whZRtOaj-ra78c2a421 (workspace) -> sdk-Mn0BBpZ0-ra78c2a421 (account) finished
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 2 permissions: sdk-whZRtOaj-ra78c2a421 (workspace) -> sdk-Mn0BBpZ0-ra78c2a421 (account)
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 2 permissions for 1/1 groups successfully.
20:03 ERROR [databricks.labs.ucx.hive_metastore.grants] Couldn't fetch grants for object ANY FILE : TEMPORARILY_UNAVAILABLE: The service at /api/2.0/sql-acl/get-permissions is taking too long to process your request. Please try again later or try a faster operation. [TraceId: 00-69f8d22882a1b6d9ba6ed6fe4b3642ef-76c77db035725346-01]
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions for 1 account groups.
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-whZRtOaj-ra78c2a421 (workspace) -> sdk-Mn0BBpZ0-ra78c2a421 (account) starting
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-whZRtOaj-ra78c2a421 (workspace) -> sdk-Mn0BBpZ0-ra78c2a421 (account) progress=2(+2)
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrating permissions: sdk-whZRtOaj-ra78c2a421 (workspace) -> sdk-Mn0BBpZ0-ra78c2a421 (account) finished
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 2 permissions: sdk-whZRtOaj-ra78c2a421 (workspace) -> sdk-Mn0BBpZ0-ra78c2a421 (account)
19:52 INFO [databricks.labs.ucx.workspace_access.groups] Migrated 2 permissions for 1/1 groups successfully.
20:03 ERROR [databricks.labs.ucx.hive_metastore.grants] Couldn't fetch grants for object ANY FILE : TEMPORARILY_UNAVAILABLE: The service at /api/2.0/sql-acl/get-permissions is taking too long to process your request. Please try again later or try a faster operation. [TraceId: 00-69f8d22882a1b6d9ba6ed6fe4b3642ef-76c77db035725346-01]
[gw3] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from acceptance #8968}