diff --git a/.agents/skills/java-development/SKILL.md b/.agents/skills/java-development/SKILL.md deleted file mode 100644 index 0b929dec6cd0..000000000000 --- a/.agents/skills/java-development/SKILL.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -name: java-development -description: General guidance on Java development practices, building, testing, and style in the monorepo. Use this skill when working on Java code across the repository. ---- - -# Java Development Guide - -This skill provides general guidelines for Java development inside the monorepo. It covers building, formatting, testing, and style conventions to ensure consistency across modules. - -## Workflow - -### 1. Building the Project - -The repository uses Maven as its primary build system. - -* **Build All Modules**: To build all modules from the root of the repository, run: - ```bash - mvn install -T 1C -P quick-build - ``` - > [!TIP] - > Use `-T 1C` to build modules in parallel (one thread per CPU core) and `-P quick-build` to skip unnecessary plugins for faster builds. -* **Build a Specific Module**: You can also run Maven commands within a specific module directory (e.g., `java-bigquery`) to build only that module. - -### 2. Code Formatting - -Code formatting is enforced using the `fmt-maven-plugin`. - -* **Check Formatting**: To check for formatting issues without modifying files, run: - ```bash - mvn fmt:check -T 1C - ``` -* **Apply Formatting**: To automatically format the code according to the project style, run: - ```bash - mvn fmt:format -T 1C - ``` - > [!TIP] - > To save time, run `mvn fmt:format` within the specific module directory you are working on, rather than at the root. - > [!NOTE] - > Always run `mvn fmt:format` before committing changes to avoid build failures due to formatting. - -### 3. Testing Strategy - -* **Unit Tests**: Traditional unit tests should be added for individual classes and methods. Run them using: - ```bash - mvn test -T 1C - ``` -* **Integration Tests**: Many modules have integration tests that run against live services or emulators. These may require specific profiles or environment variables. Refer to the specific module's README for details. - -### 4. Style Guide - -Follow these general rules to maintain code quality and consistency: - -1. **Minimize Visibility**: Default to the most restrictive access level possible. Avoid using `public` unless the class or method is intended to be part of the public API. -2. Avoid Fully Qualified Names: Use imports to keep class names short and readable, rather than using fully qualified names in the code. -3. **Avoid Obsolete APIs**: Do not call methods marked with `@ObsoleteApi` or `@Deprecated` unless there are no viable alternatives. -4. **Clean Diffs**: Avoid unnecessary formatting changes or whitespace modifications to keep diffs clean and easy to review. - -### 5. Dependency Management - -* **Version Bumps**: Try not to bump any external dependency versions unless there is a known security vulnerability (CVE) or a critical bug fix. -* **New Dependencies**: Avoid introducing new external dependencies. If a new dependency is required, provide a strong justification in the pull request. -* **Standard Library First**: Prefer to use features from the Java standard library, followed by existing dependencies in the project (preferably Google-managed dependencies). - -### 6. Contribution Guidelines - -* **Commit Messages**: Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification. Include the module as the scope (e.g., `feat(spanner): ...`, `fix(bigquery): ...`). -* **Pull Requests**: All code changes must be submitted via a pull request and require review. Ensure you pull the latest changes from `main` and resolve any conflicts before submitting. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000000..a3b8b988d09f --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,61 @@ +# Agents Guide for google-cloud-java + +## 1. Overview +This repository, `google-cloud-java`, contains the Java client libraries for Google Cloud Platform services, as well as core components in `sdk-platform-java`. + +## 2. Project Structure +The repository is a monorepo containing both generated and handwritten libraries, as well as core platform components and parent POMs. + +### Core Modules +- **`sdk-platform-java/`**: Contains foundational components for building client libraries. + - **Note**: This directory has its own `GEMINI.md` file with detailed instructions specific to core development (GAPIC generator, GAX). + - Includes `gapic-generator-java` (the generator) and `gax-java` (Google API Extensions). +- **`google-auth-library-java/`**: The Google Auth Library for Java. This is a **handwritten** library used for authentication and credential management across all Google Cloud clients. It is a critical dependency for all client libraries. + +### Parent Modules and BOMs +- **`google-cloud-pom-parent/`**: The top-level parent POM for all modules in the repository. It manages plugin versions and common configuration. +- **`google-cloud-jar-parent/`**: The parent POM for all client library JAR modules in the repository. It inherits from `google-cloud-pom-parent` and manages shared dependencies. +- **`gapic-libraries-bom/`**: The Bill of Materials (BOM) that manages versions of all client libraries to ensure compatibility when used together. +- **`java-shared-dependencies/`** (inside `sdk-platform-java`): Manages shared Maven dependencies for all Google Cloud Java client libraries to ensure consistency and avoid conflicts. + +### Client Libraries (`java-/`) +Directories starting with `java-` are client libraries for specific Google Cloud services. +- **Generated Clients**: The majority of these are automatically generated from service definitions (protos) using the GAPIC generator in `sdk-platform-java`. +- **Handwritten & Split Repositories**: Some major libraries are either entirely handwritten or are maintained as "split repos" (they have their own standalone repositories in the `googleapis` GitHub organization but are also managed here). When working on these, be aware that changes may need to be synchronized with their respective split repos. Key examples include: + - **BigQuery**: [java-bigquery](java-bigquery) + - **BigQuery Storage**: [java-bigquerystorage](java-bigquerystorage) + - **Spanner**: [java-spanner](java-spanner) + - **Spanner JDBC**: [java-spanner-jdbc](java-spanner-jdbc) + - **Storage**: [java-storage](java-storage) + - **Storage NIO**: [java-storage-nio](java-storage-nio) + - **Datastore**: [java-datastore](java-datastore) + - **Logging**: [java-logging](java-logging) + - **Logging Logback**: [java-logging-logback](java-logging-logback) + +## 3. Getting Started +This is a standard Maven project. +- **Build all**: `mvn install -T 1C -P quick-build` from root (recommended). +- **Scoped build**: Run `mvn` commands within specific module directories. +- **Prerequisites**: Java 11+, Maven 3.0+, Bazelisk (for core integration tests). + +## 4. Style Guide +1. Minimize visibility scopes. Default to most restrictive access level. +2. Use short names over fully qualified names. +3. Avoid calling `@ObsoleteApi` or `@Deprecated` methods and classes. +4. Avoid unnecessary formatting changes to keep diffs clean. +5. Use `mvn` for everything other than the `test/integration` folder. + +## 5. Dependency Management +- Do not bump external dependency versions unless for CVE or critical bug fix. +- Avoid introducing new external dependencies if possible. Prefer Java standard library, then opt for existing dependencies. + +## 6. Contribution Guidelines +- **Commits**: Conventional Commits `(): `. +- **Pull Requests**: Submitted via PR, require review. Pull latest from main and resolve conflicts. +- **Testing**: All new logic should be accompanied by tests. + +## 7. Specialized Development Guides +For development on core components, refer to the following guides in `sdk-platform-java`: +- **GAPIC Generator**: [sdk-platform-java/gapic-generator-java/DEVELOPMENT.md](sdk-platform-java/gapic-generator-java/DEVELOPMENT.md) +- **GAX**: [sdk-platform-java/gax-java/DEVELOPMENT.md](sdk-platform-java/gax-java/DEVELOPMENT.md) +- **Hermetic Build**: [sdk-platform-java/hermetic_build/DEVELOPMENT.md](sdk-platform-java/hermetic_build/DEVELOPMENT.md) diff --git a/sdk-platform-java/AGENTS.md b/sdk-platform-java/AGENTS.md new file mode 100644 index 000000000000..e45efd77bb97 --- /dev/null +++ b/sdk-platform-java/AGENTS.md @@ -0,0 +1,100 @@ +# Gemini CLI Context for sdk-platform-java + +## 1. Overview + +This directory, `sdk-platform-java`, contains the foundational components for building Java client libraries for Google Cloud Platform services. It includes the GAPIC (Generated API Client) generator for Java, the GAX (Google API Extensions) runtime library, and other shared modules. + +For general project rules, style guides, dependency management, and contribution guidelines, please refer to the root [AGENTS.md](../AGENTS.md). + +## 2. Project Structure + +The repository is structured into several key modules: + +* **`gapic-generator-java`**: The core component for creating new client libraries. It is a Protobuf compiler plugin that generates Java client libraries from API definition files. The `DEVELOPMENT.md` file in this module provides detailed information on how to work with the generator. + +* **`gax-java`**: The Google API Extensions for Java (GAX) library. It provides runtime support for the generated client libraries, including features like pagination, request batching, and polling of long-running operations. It has the following submodules: + * `gax`: Transport-independent part of GAX for Java. + * `gax-grpc`: gRPC-specific logic for GAX. + * `gax-httpjson`: REST-specific logic for GAX. + +* **`api-common-java`**: Contains foundational types and utilities related to Google APIs. It includes the following packages: + * `core`: Core library containing API stability annotations and wrappers around Guava types. + * `pathtemplate`: Path Template library for manipulating strings that are formatted as Google API resource names. + * `resourcenames`: Resource Name library used by generated resource name types. + +* **`java-common-protos`**: Provides Protobuf-generated classes for common Google services that are used across multiple APIs. The code is in this module is auto-generated and should not be modified. + +* **`java-iam`**: Contains Protobuf-generated classes for Google's Identity and Access Management (IAM) service, used for managing policies. The code is in this module is auto-generated and should not be modified. + +* **`java-showcase`**: A demonstration client library for the "Showcase" API, which is a fake API used for integration testing of the GAPIC generator and GAX library. + +* **`java-shared-dependencies`**: Manages shared Maven dependencies for all Google Cloud Java client libraries. This ensures consistency and helps avoid dependency conflicts. + +## 3. Testing + +For detailed testing strategy and heuristics on which tests to run based on code changes, please refer to [DEVELOPMENT.md](DEVELOPMENT.md). + +### 3.1. Running Unit Tests + +Unit tests can be tested via this command: + +```sh +mvn test +``` + +### 3.2. Running Golden Integration Tests + +Golden integration tests are run using Bazel. To run all golden integration tests, use the following command from the root of the repository: + +```sh +bazelisk test //... +``` + +### 3.3. Running Showcase Integration Tests + +```sh +bazelisk test //test/integration:redis +``` + +### 3.4. Updating Golden Files + +If you make changes that affect the generated code, you will need to update the golden files. This can be done using the following command: + +```sh +bazelisk run //test/integration:update_asset && bazelisk run //test/integration:update_credentials && bazelisk run //test/integration:update_iam && bazelisk run //test/integration:update_kms && bazelisk run //test/integration:update_pubsub && bazelisk run //test/integration:update_logging && bazelisk run //test/integration:update_redis && bazelisk run //test/integration:update_storage && bazelisk run //test/integration:update_library && bazelisk run //test/integration:update_compute && bazelisk run //test/integration:update_bigtable && bazelisk run //test/integration:update_apigeeconnect +``` + +### 3.5. Running Showcase Integration Tests + +Showcase integration tests are run against a local server that implements the Showcase API. The `java-showcase/README.md` file provides detailed instructions on how to run these tests. The general steps are: + +1. **Install the Showcase server:** + + ```sh + go install github.com/googleapis/gapic-showcase/cmd/gapic-showcase@latest + ``` + +2. **Run the Showcase server:** + + ```sh + gapic-showcase run + ``` + +3. **Run the integration tests:** + + ```sh + cd java-showcase + mvn verify -P enable-integration-tests + ``` + +## 4. File Exclusions & Loading Constraints + +To maintain a highly focused context window and prevent memory overload: + +* **Permanent Exclusions (Always Ignore):** + * Do **NOT** load, read, or search through files under the directories `java-showcase-3.21.0/` and `java-showcase-3.25.8/`. Treat these folders as completely out of scope. +* **Conditional Exclusions (Ignore by Default):** + * By default, do **NOT** load, read, or search through files in the `test/` folder (which contains massive auto-generated integration golden files). + * **Exceptions:** You are only authorized to load or modify files under `test/` when: + 1. The user explicitly asks you to *"update the golden files"* or *"inspect test files"*. + 2. You are actively executing the Golden Integration Update commands (e.g. `bazelisk run //test/integration:update_...`). diff --git a/sdk-platform-java/GEMINI.md b/sdk-platform-java/GEMINI.md deleted file mode 100644 index 22b330ec1ba4..000000000000 --- a/sdk-platform-java/GEMINI.md +++ /dev/null @@ -1,169 +0,0 @@ -# Gemini CLI Context for sdk-platform-java - -## 1. Overview - -This directory, sdk-platform-java, contains the foundational components for building Java client libraries for Google Cloud Platform services. It includes the GAPIC (Generated API Client) generator for Java, the GAX (Google API Extensions) runtime library, and other shared modules. - -## 2. Project Structure - -The repository is structured into several key modules: - -* **`gapic-generator-java`**: The core component for creating new client libraries. It is a Protobuf compiler plugin that generates Java client libraries from API definition files. The `DEVELOPMENT.md` file in this module provides detailed information on how to work with the generator. - -* **`gax-java`**: The Google API Extensions for Java (GAX) library. It provides runtime support for the generated client libraries, including features like pagination, request batching, and polling of long-running operations. It has the following submodules: - * `gax`: Transport-independent part of GAX for Java. - * `gax-grpc`: gRPC-specific logic for GAX. - * `gax-httpjson`: REST-specific logic for GAX. - -* **`api-common-java`**: Contains foundational types and utilities related to Google APIs. It includes the following packages: - * `core`: Core library containing API stability annotations and wrappers around Guava types. - * `pathtemplate`: Path Template library for manipulating strings that are formatted as Google API resource names. - * `resourcenames`: Resource Name library used by generated resource name types. - -* **`java-common-protos`**: Provides Protobuf-generated classes for common Google services that are used across multiple APIs. The code is in this module is auto-generated and should not be modified. - -* **`java-iam`**: Contains Protobuf-generated classes for Google's Identity and Access Management (IAM) service, used for managing policies. The code is in this module is auto-generated and should not be modified. - -* **`java-showcase`**: A demonstration client library for the "Showcase" API, which is a fake API used for integration testing of the GAPIC generator and GAX library. - -* **`java-shared-dependencies`**: Manages shared Maven dependencies for all Google Cloud Java client libraries. This ensures consistency and helps avoid dependency conflicts. - -## 3. Getting Started - -### 3.1. Prerequisites - -To build and work with this project, you will need to install: - -* Java 11+ -* Maven - -### 3.2. Building the Project - -The project uses Maven for its primary build system. To build all modules, run the following command from the root of the repository: - -```sh -mvn install -``` - -### 3.3. Code Formatting - -Code formatting is enforced using the `fmt-maven-plugin`. To check for formatting issues, run: - -```sh -mvn fmt:check -``` - -To format the code, run: - -```sh -mvn fmt:format -``` - -## 4. Testing - -### 4.1. Testing Strategy - -The repository employs a multi-layered testing strategy to ensure the quality and correctness of the generated code: - -* **Unit Tests:** Traditional unit tests for individual classes and methods. See [unit testing best practices](https://engdoc.corp.google.com/eng/doc/devguide/testing/unit/best-practices.md?cl=head&polyglot=java). -* **Golden Unit Tests:** These tests generate code from test protos and compare the output to "golden" files, which are pre-approved versions of the generated code. These test cases exist inside the `gapic-generator-java` module. See [GrpcServiceStubClassComposerTest](https://github.com/googleapis/google-cloud-java/blob/main/sdk-platform-java/gapic-generator-java/src/test/java/com/google/api/generator/gapic/composer/grpc/GrpcServiceStubClassComposerTest.java) for an example of golden unit tests. The steps for golden unit tests include creating a test proto, loading the test proto, generating from the test proto, and comparing the result with a golden file. -* **Showcase Integration Tests:** These tests run the generated Showcase client against a local Showcase server to verify end-to-end functionality. This is the preferred way of testing integration tests. -* **Golden Integration Tests:** These tests generate full client libraries for real Google Cloud APIs and compare them to golden versions. This is an older test strategy and showcase testing is preferred. - -Based on where the code changes are, we should add different tests, in general - -* If the changes are in `gax` or `api-common` only, you _must_ add traditional unit tests, you _may_ add showcase integration tests if you are modifying a public setting. -* If the changes are in `gapic-generator-java` only, showcase integration tests are most likely not needed - * If they are in the `composer` module, you _must_ add golden unit tests, you _may_ add traditional unit tests for better coverage and easier testing. - * If they are in the `parser` module, you _should_ add traditional unit tests with a test proto if possible, you _may_ add golden unit tests to easily see the changes. For example, [routing_rule_parser_testing](https://github.com/googleapis/google-cloud-java/blob/main/sdk-platform-java/gapic-generator-java/src/test/proto/routing_rule_parser_testing.proto) that is only used for testing [RoutingRuleParser](https://github.com/googleapis/google-cloud-java/blob/main/sdk-platform-java/gapic-generator-java/src/main/java/com/google/api/generator/gapic/protoparser/RoutingRuleParser.java). - * If they are in `other modules(ast, model, writer etc.)`, you _must_ add traditional unit tests, you _may_ add golden unit tests to easily see the changes. -- If the changes are in both `gax` and `gapic-generator-java`, you _must_ add all test layers, including traditional unit tests, golden unit tests and showcase integration tests. - -### 4.2. Running Unit Tests - -Unit tests can be tested via this command: - -```sh -mvn test -``` - -### 4.3. Running Golden Integration Tests - -Golden integration tests are run using Bazel. To run all golden integration tests, use the following command from the root of the repository: - -```sh -bazelisk test //... -``` - -Specific integration tests can be run by specifying the target, for example: - -```sh -bazelisk test //test/integration:redis -``` - -### 4.4. Updating Golden Files - -If you make changes that affect the generated code, you will need to update the golden files. This can be done using the following command: - -```sh -bazelisk run //test/integration:update_asset && bazelisk run //test/integration:update_credentials && bazelisk run //test/integration:update_iam && bazelisk run //test/integration:update_kms && bazelisk run //test/integration:update_pubsub && bazelisk run //test/integration:update_logging && bazelisk run //test/integration:update_redis && bazelisk run //test/integration:update_storage && bazelisk run //test/integration:update_library && bazelisk run //test/integration:update_compute && bazelisk run //test/integration:update_bigtable && bazelisk run //test/integration:update_apigeeconnect -``` - -### 4.5. Running Showcase Integration Tests - -Showcase integration tests are run against a local server that implements the Showcase API. The `java-showcase/README.md` file provides detailed instructions on how to run these tests. The general steps are: - -1. **Install the Showcase server:** - - ```sh - go install github.com/googleapis/gapic-showcase/cmd/gapic-showcase@latest - ``` - -2. **Run the Showcase server:** - - ```sh - gapic-showcase run - ``` - -3. **Run the integration tests:** - - ```sh - cd java-showcase - mvn verify -P enable-integration-tests - ``` - -## 5. Style Guide - -1. Minimize visibility scopes by defaulting to the most restrictive access level, avoiding the `public` modifier unless required. -2. Use short names over fully qualified names. -3. Avoid calling `@ObsoleteApi` or `@Deprecated` methods unless there are no alternatives. -4. Avoid unnecessary formatting changes to keep diffs clean. -5. Use `mvn` for everything other than the `test/integration` folder. - -## 6. Dependency Management - -- Try not to bump any external dependency version unless there is a known CVE (security or vulnerability issue) or a critical bug fix. -- Try to avoid introducing new external dependencies. If a new dependency is required, please state the reason. -- Prefer to use features from the Java standard library, then try to use features from any existing dependencies (preferably from Google managed dependencies). - -## 7. Contribution Guidelines - -- **Commits:** Commit messages should follow the [Conventional Commits](https://www.conventionalcommits.org/) - specification. The format is `: `. The type should be one of the following: fix, feat, - build, chore, docs, test, or refactor. -- **Issues:** All significant changes should start with a GitHub issue. -- **Pull Requests:** All code changes must be submitted via a pull request and require review. Before creating a PR, always pull latest from main, merge main to local branch and resolve any conflicts. -- **Testing:** All new logic should be accompanied by tests. -- For more details, see `CONTRIBUTING.md`. - -## 8. File Exclusions & Loading Constraints - -To maintain a highly focused context window and prevent memory overload: - -* **Permanent Exclusions (Always Ignore):** - * Do **NOT** load, read, or search through files under the directories `java-showcase-3.21.0/` and `java-showcase-3.25.8/`. Treat these folders as completely out of scope. -* **Conditional Exclusions (Ignore by Default):** - * By default, do **NOT** load, read, or search through files in the `test/` folder (which contains massive auto-generated integration golden files). - * **Exceptions:** You are only authorized to load or modify files under `test/` when: - 1. The user explicitly asks you to *"update the golden files"* or *"inspect test files"*. - 2. You are actively executing the Golden Integration Update commands (e.g. `bazelisk run //test/integration:update_...`). \ No newline at end of file