Skip to content

Conversation

@exceptionfactory
Copy link
Contributor

Summary

NIFI-15286 Adds manual connection verification to Kafka Connection Services.

The implementation builds on the initial proposal in PR #10596 and provides the following approach:

  1. Moves all verification methods to the new KafkaConnectionVerifier class with one reusable instance of the class for each instance of the Kafka Connection Service
  2. Moves Topic Listing to the final verification step
  3. Verifies address formatting before verifying socket connections
  4. Verifies socket connections before Kafka Client connections

The implementation includes updated integration tests against Kafka running in a container to evaluate common failure and success paths.

Successful verification includes the following steps:

  1. Broker Addresses displaying number of addresses validated
  2. Node Connection displaying host and port for each Node with socket connection
  3. Cluster Description displaying number of Nodes found
  4. Topic Listing displaying the number of Topics found

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using ./mvnw clean install -P contrib-check
    • JDK 21
    • JDK 25

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

Copy link
Contributor

@pvillard31 pvillard31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @exceptionfactory, it is working as expected. Left two comments, happy to merge after that.

Image Image

}

private List<InetSocketAddress> getConnectedAddresses(final ComponentLog verificationLogger, final List<ConfigVerificationResult> results, final List<InetSocketAddress> bootstrapAddresses) {
final List<InetSocketAddress> connectedAddresses = new ArrayList<>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use a thread-safe list here? Given that we have virtual threads that execute in parallel, we could have a ConcurrentModificationException, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, yes, I will wrap this in a synchronized List.

new ConfigVerificationResult.Builder()
.verificationStepName(ADDRESSES_STEP)
.outcome(SUCCESSFUL)
.explanation("Addresses Validated [%d]".formatted(validatedAddresses.size()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.explanation("Addresses Validated [%d]".formatted(validatedAddresses.size()))
.explanation("Addresses validated [%d]".formatted(validatedAddresses.size()))

To be consistent with, for example, "Topics found"

@exceptionfactory
Copy link
Contributor Author

Thanks for the review @pvillard31, I pushed an update to synchronize the List and adjust the message capitalization.

Copy link
Contributor

@pvillard31 pvillard31 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, +1, will merge once checks are green (but likely tomorrow now - you can self merge before if you want)

@exceptionfactory exceptionfactory merged commit f90a921 into apache:main Dec 24, 2025
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants