Balance Topic partitions evenly on the cluster

[cj-gediminas-simkus]

Hi,

Is it possible to configure cruise-control so it would distribute topic partitions evenly across the cluster instead of mixing all topic partitions and then distributing.
Example:
I have a topic of 12 partitions and a cluster of 6 brokers, I would like to configure cruise control to distribute 2 partitions on each node of the cluster.
Currently it decides and distributes 2 partitions from topic1, 5 partitions from topic2 and 8 partitions of topic3. Overall number of partitions per broker looks evenly spread, but it's not what I'm looking at.

Thanks

[kyguy]

Hi @cj-gediminas-simkus,

Just to clarify, are you asking if it's possible to configure Cruise Control to evenly distribute the leader partitions of a topic across all brokers?

If so, yes, this can be accomplished using the MinTopicLeadersPerBrokerGoal. [1]

Check out the min.topic.leaders.per.broker configuration [2]:

The minimum required number of leaders per broker for topics that must have leader on brokers that are not excluded for replica move. When set to 0, the minimum number of leaders per broker is computed dynamically for each topic as no-of-topic-leaders / no-of-brokers

So when we set min.topic.leaders.per.broker to 0, Cruise Control computes the value dynamically as:

no-of-topic-leaders / no-of-brokers

For example:

12 / 6 = 2 partition leaders per broker

Let me know if that helps!

[1] https://github.com/linkedin/cruise-control/blob/main/cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/analyzer/goals/MinTopicLeadersPerBrokerGoal.java
[2] https://github.com/linkedin/cruise-control/wiki/Configurations