Add support for altering and partially resetting connector offsets - #405
Open
AfrinKhan02 wants to merge 5 commits into
Open
Add support for altering and partially resetting connector offsets#405AfrinKhan02 wants to merge 5 commits into
AfrinKhan02 wants to merge 5 commits into
Conversation
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Context
The first feature of the connector offset epic introduced support for fully resetting connector offsets in michelin/kafkactl#380.
This change extends that flow so users can alter or partially reset source connector offsets directly from
kafkactl. It allows users to target specific source partitions and either assign a connector-specific offset or reset the selected partition by setting its offset tonull.The corresponding Ns4Kafka API support is implemented in michelin/ns4kafka#826.
Proposed solution
I introduced a dedicated CLI flow for connector offset alteration through:
kafkactl connector alter-offsets <connector-name> -f <offsets-file>The command accepts a YAML payload using the Kafka Connect alter-offsets format. This keeps the operation aligned with the existing connector command structure while supporting both targeted offset alteration and partial reset.
Example payload:
Implementation
I added a connector offsets request model representing the Kafka Connect
offsetspayload.I then added client-side support for the Ns4Kafka endpoint:
The new endpoint is wired through the resource service, including standard response, not-found, and HTTP error handling.
On the CLI side, I added and registered the
connector alter-offsetssubcommand. The command:ConnectorResetOffsetsResponseresultI also updated the README with command usage and examples for altering or partially resetting connector offsets.
Tests
I added command-level tests covering:
alter-offsetssubcommand help without parent parametersI added service-level tests covering:
The full test suite and Spotless checks pass.
Remark
This change complements the full connector offset reset introduced in michelin/kafkactl#380. Together, the two commands support both complete reset and targeted alteration or partial reset of connector offsets.
The server-side implementation used by this command is available in michelin/ns4kafka#826.