Migrate to generated API client libraries

[aaemnnosttv]

Feature Description

Much work has been done to explore a transition to generated API client libraries, and several PRs have been prepared to make reviewing much easier and safer. This issue is to review and merge all the PRs.

---

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

• All Google API client libraries should be migrated to local packages generated from the official client services tooling
• google/apiclient-services should no longer be needed or installed
• All partially generated clients and automated patching should be removed and refactored to use the now-present respective client (e.g. analyticsadmin-v1alpha)

Implementation Brief

Generate client libraries on-demand

Create Dockerized image for generator

• Add new generator/Dockerfile
  • This is a minimal Python based image to contain a specific version of the official PHP api client code generator
• Add generator/apis/{apiId}/discoverydoc.json
  • e.g. generator/apis/analyticsadmin:v1beta/discoverydoc.json which is source from https://analyticsadmin.googleapis.com/$discovery/rest?version=v1beta
  • adding these to the repo ensures the resulting client doesn't change unexpectedly
• Store the sha1 hash of the commit to pin the code generator to (either in composer.json in extra or perhaps generator/config.json)
  • This can be passed as a build arg to the generator's Dockerfile
• Create a workflow to automatically build and push the image when the sha or Dockerfile changes
  • Use .github/workflows/publish-vrt-docker-image.yml as an example
  • The image tag should be based on the sha which is the main version it contains

Automate generation into Composer lifecycle

• Add a script to Composer pre-install-cmd and pre-update-cmd, e.g. generator/sync-packages.sh
  • This script should run for each api in generator/apis/*/
    • generate a composite hash of (discoverydoc.json + generator sha)
    • compare this to a file in the generated packages/apiclient-serivces-{api} files, potentially the composer.json .version
    • if they don't match, run the generator via docker run ... --pull ghcr.io/google/site-kit-wp/codegen:{sha}
      • in addition to generating the source, it must also add the minimal composer.json in the root (only needs name + version)
  • Each generated client should be excluded from git via gitignore

Exceptions

Note the subscribewithgoogle API is not public and will remain as-is, but upgraded with the latest version.

PRs

Service	PR to add package (against https://github.com/google/site-kit-wp/tree/feature/php-client-src)	PR to migrate (against PR to add package)
Subscribe with Google	#11923	#11925

Finalizing transition

The resulting changes that are needed in the plugin will be the same as the original IB, only the migration PRs need to be rebased or recreated to exclude the added client src code since those PRs were created from their earlier branches that added it. These can probably all be combined into a single PR as most of the changes are very simple with the exception of GA4 APIs.

Original IB (committed code)

The main branch for this transition is https://github.com/google/site-kit-wp/tree/feature/php-client-src

Each service has two PRs against this feature branch:

Service	PR to add package (against https://github.com/google/site-kit-wp/tree/feature/php-client-src)	PR to migrate (against PR to add package)
AdSense API v2	#11916	#11915
Analytics Admin API v1 alpha	#11977 (both)	#11978 (both)
Analytics Admin API v1 beta	#11977 (both)	#11978 (both)
Analytics Data API v1 beta	#11919	#11979
PageSpeed	#11920	#11926
People API v1	#11921	#11927
Search Console v1	#11922	#11928
Site Verification API	#11980	#11981
Subscribe with Google	#11923	#11925
Tag Manager API v2	#11924	#11929

[!NOTE]
Note: the changes to the Analytics Admin API are the most substantial and the PRs above include both v1alpha and v1beta together

The PR to add package is against the https://github.com/google/site-kit-wp/tree/feature/php-client-src branch and contains the client package src ONLY.
The PR to migrate to the new package is against the previous (PR to add) – this way the changes in the plugin are easier to see and review whereas otherwise things can be missed in the large number of files that some of these have.

There are different ways this could be approached but I suggest (for each service):

1. Review and merge client library addition PR
2. Update migration PR to target feature branch (CI will run)
3. Review & merge migration PR
4. Finally merge last PRs below

Once all services have been migrated, there is one last PR to remove the now-unused apiclient-services package, which is required implicitly as a dependency of google/apiclient

• LAST: Remove unused google/apiclient-services #11982

Once everything is ready to land, there is a final PR to merge the feature branch

• Migrate to generated API client libraries #11984

Final notes

Some merge conflicts may arise while merging these changes, so keep the below in mind regarding the final state:

In the end, the `composer.json` should have these changes

diff --git a/composer.json b/composer.json
index 2f0c0fafe4..cb4398c58b 100644
--- a/composer.json
+++ b/composer.json
@@ -29,6 +29,7 @@
     "symfony/polyfill-intl-idn": "^1.29.0"
   },
   "replace": {
+    "google/apiclient-services": "*",
     "paragonie/random_compat": ">=2"
   },
   "autoload": {
@@ -49,35 +50,12 @@
     "allow-plugins": {
       "dealerdirect/phpcodesniffer-composer-installer": true,
       "roots/wordpress-core-installer": true,
-      "cweagans/composer-patches": true
     }
   },
   "extra": {
-    "google/apiclient-services": [
-      "Adsense",
-      "AnalyticsData",
-      "GoogleAnalyticsAdmin",
-      "PagespeedInsights",
-      "PeopleService",
-      "SearchConsole",
-      "SiteVerification",
-      "TagManager"
-    ],
     "wordpress-install-dir": "vendor/roots/wordpress",
-    "patches": {
-      "google/apiclient-services": {
-        "Fixes for empty filter": "patches-composer/google-api-client-services-v0.355.0.patch"
-      }
-    }
   },
   "repositories": [
-    {
-      "type": "path",
-      "url": "packages/apiclient-services-adsenselinks",
-      "options": {
-        "symlink": false
-      }
-    },
     {
       "type": "path",
       "url": "packages/apiclient-services-subscribewithgoogle",

Note

The Composer lock file may need updating one last time once all changes have been merged to pass strict validation. Run composer update --lock to update the hash in the lock file if needed.

Test Coverage

• No changes; this is a refactoring

QA Brief

• All services should be tested to ensure communication with APIs continues to work as expected (this is essentially smoke testing of all setup flows and basic reporting)

Changelog entry

• Migrate to generated Google API client libraries

[eugene-manuilov]

@aaemnnosttv, why do we commit all those generated files? They should be ignored, should they? So that everyone will need to generate them on their machines to make it work locally. The same how it is done with webpack bundles.