IGNITE-25534 Update Ignite versions validation for cluster #12301

chesnokoff · 2025-08-27T19:50:49Z

Thank you for submitting the pull request to the Apache Ignite.

In order to streamline the review of the contribution
we ask you to ensure the following steps have been taken:

The Contribution Checklist

There is a single JIRA ticket related to the pull request.
The web-link to the pull request is attached to the JIRA ticket.
The JIRA ticket has the Patch Available state.
The pull request body describes changes that have been made.
The description explains WHAT and WHY was made instead of HOW.
The pull request title is treated as the final commit message.
The following pattern must be used: IGNITE-XXXX Change summary where XXXX - number of JIRA issue.
A reviewer has been mentioned through the JIRA comments
(see the Maintainers list)
The pull request has been checked by the Teamcity Bot and
the green visa attached to the JIRA ticket (see TC.Bot: Check PR)

Notes

If you need any help, please email [email protected] or ask anу advice on http://asf.slack.com #ignite channel.

chesnokoff · 2025-09-05T07:16:28Z

...org/apache/ignite/internal/processors/nodevalidation/OsDiscoveryNodeValidationProcessor.java

+        Set<Byte> versions = ctx.discovery().allNodes().stream()
+            .map(node -> IgniteProductVersion.fromString(node.attribute(ATTR_BUILD_VER)).minor())
+            .collect(Collectors.toSet());


Previous patch used field to store available version only first time. But there are two problems:

If we change coordinator to node with higher version, it may accept +2 version so there is a probability to have three versions of nodes in cluster

If +1 version node was accepted, after that it disconnected and we want to add -1 version it will be rejected

So we have to scan available versions in discovery().allNodes() every time

chesnokoff · 2025-09-05T07:19:17Z

modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/ServerImpl.java

            NavigableMap<Long, Collection<ClusterNode>> hist = updateTopologyHistory(topVer, top);

-            lsnr.onDiscovery(
+            IgniteFuture<?> fut = lsnr.onDiscovery(


The future is responsible for updating discovery().allNodes() that is used in allowedMinorVersions so we need to wait it. Otherwise, there is race codition, when we can accept +1 and -1 version

I have not implemented test for this because I think usually there will no be race condition with discovery().allNodes(). It can be reproduced only for high load systems. For instance when ten nodes with +1 and -1 versions are trying to connect to slow master at the same time.

I think long TCP communication will usually save us from the race. So a reproducer would be flaky

timoninmaxim · 2025-09-05T10:27:55Z

modules/core/src/test/java/org/apache/ignite/internal/GridReleaseTypeSelfTest.java

+
+        String stackTrace = errors.toString();
+
+        assert stackTrace.contains("Remote node rejected due to incompatible version for cluster join");


Can be replaced with X.hasCause

timoninmaxim · 2025-09-05T10:34:55Z

modules/core/src/test/java/org/apache/ignite/internal/GridReleaseTypeSelfTest.java

+
+        assertTrue(waitForCondition(() -> Ignition.allGrids().size() == 2, getTestTimeout()));
+
+        startGrid(3, "2.19.0", false);


For RU you should reuse node ids. grid(0) is restarted with new version

timoninmaxim · 2025-09-05T10:36:44Z

modules/core/src/test/java/org/apache/ignite/internal/GridReleaseTypeSelfTest.java

+
+    /** */
+    @Test
+    public void testRollingUpgrade0() throws Exception {


Let's add more tests:

For downgrade version of cluster

Upgrade and then downgrade back

Start upgrade not from coordinator node

timoninmaxim · 2025-09-05T10:39:22Z

...org/apache/ignite/internal/processors/nodevalidation/OsDiscoveryNodeValidationProcessor.java

+                + "Allowed versions for joining:\n"
+                + "  - " + locVer.major() + '.' + locVer.minor() + ".X\n"
+                + "  - " + locVer.major() + '.' + (locVer.minor() + 1) + ".X\n"
+                + "  - " + locVer.major() + '.' + (locVer.minor() - 1) + ".X";


In case cluster has 2.18 (locVer) and 2.19 versions, then we can't accept 2.17 while this log promises it.

timoninmaxim · 2025-09-05T10:41:41Z

modules/core/src/test/java/org/apache/ignite/internal/GridReleaseTypeSelfTest.java

+        boolean withClients
+    ) throws Exception {
+        startGrid(0, acceptedVer1, false);
+        startGrid(1, acceptedVer2, withClients);


Let's have a cluster with multiple server nodes. In this case different client nodes can be connected to different server nodes. We should check that all client nodes versions are participating in the check.

wernerdv · 2025-09-11T15:04:07Z

modules/core/src/test/java/org/apache/ignite/internal/GridReleaseTypeSelfTest.java

- * Test grids starting with non compatible release types.
+ * Test Rolling Upgrade release types.
 */
+@WithSystemProperty(key = "IGNITE.ROLLING.UPGRADE.VERSION.CHECK", value = "true")


Let's use OsDiscoveryNodeValidationProcessor#IGNITE_ROLLING_UPGRADE_VERSION_CHECK as a key.

wernerdv · 2025-09-11T15:23:32Z

modules/core/src/test/java/org/apache/ignite/internal/GridReleaseTypeSelfTest.java

+    }
+
+    /** Checks that the third grid is not compatible. */
+    private void testConflictVersions(String acceptedVer1, String acceptedVer2, String rejVer, boolean withClients) {


withClients -> isClient

wernerdv · 2025-09-11T15:24:20Z

modules/core/src/test/java/org/apache/ignite/internal/GridReleaseTypeSelfTest.java

+    }
+
+    /** Tests that starting a node with rejected version fails with remote rejection. */
+    private void testConflictVersions(String acceptedVer, String rejVer, boolean withClient) {


withClient -> isClient

sergey-chugunov-1985 · 2025-09-18T09:36:48Z

...org/apache/ignite/internal/processors/nodevalidation/OsDiscoveryNodeValidationProcessor.java

 public class OsDiscoveryNodeValidationProcessor extends GridProcessorAdapter implements DiscoveryNodeValidationProcessor {
+    /** Enables version check for rolling upgrade. */
+    @SystemProperty(value = "Enables version check for rolling upgrade.")
+    public static final String IGNITE_ROLLING_UPGRADE_VERSION_CHECK = "IGNITE.ROLLING.UPGRADE.VERSION.CHECK";


Underscore is used as a separator in system properties accross ignite codebase. Please fix this in string literal: IGNITE_ROLLING_UPGRADE_VERSION_CHECK instead of IGNITE.ROLLING.UPGRADE.VERSION.CHECK

Initially, I found LocalDeploymentSpi#IGNITE_DEPLOYMENT_ADDITIONAL_CHECK:

/** Enables additional check for resource name on resources removal. */ @SystemProperty(value = "Enables an additional check of a resource name on resources removal") public static final String IGNITE_DEPLOYMENT_ADDITIONAL_CHECK = "IGNITE.DEPLOYMENT.ADDITIONAL.CHECK";

That's why I used dots. But now I see that other properties use underscores. Resolved

sonarqubecloud · 2025-09-18T09:45:13Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

…Ignite versions

…lowedRef field

…e.contains with X#hasCause

…static field

…ment to isClient

…perty: replace dots with underscores

This reverts commit f80c523.

This reverts commit f640732.

This reverts commit 1918293.

This reverts commit c944ae4

chesnokoff commented Sep 5, 2025

View reviewed changes

timoninmaxim requested changes Sep 5, 2025

View reviewed changes

chesnokoff force-pushed the ignite-25534 branch from 23dc391 to 48a6be0 Compare September 11, 2025 14:58

wernerdv reviewed Sep 11, 2025

View reviewed changes

chesnokoff force-pushed the ignite-25534 branch from 6c98a2f to e503a05 Compare September 17, 2025 06:51

wernerdv approved these changes Sep 18, 2025

View reviewed changes

sergey-chugunov-1985 reviewed Sep 18, 2025

View reviewed changes

maksaska and others added 19 commits September 28, 2025 21:38

IGNITE-25534 Validate that cluster might contain at most 2 different …

12b5e6c

…Ignite versions

IGNITE-25534 Validate that cluster might contain at most 2 different …

b7b6e5a

…Ignite versions

IGNITE-25534 Rename tests

540d4ec

IGNITE-25534 Add IGNITE.ROLLING.UPGRADE.VERSION.CHECK system property

9d0f3cd

IGNITE-25534 Enable system property for GridReleaseTypeSelfTest

ac260f3

IGNITE-25534 Update OnDiscoveryNodeValidationProcess: remove rmtVerAl…

b11b472

…lowedRef field

IGNITE-25534 Add testCoordinatorChange

80a0f2c

IGNITE-25534 Fix race condition

3543f27

IGNITE-25534 Fix checkstyle

27ffc6a

IGNITE-25534 Update fix for race condition

a913a15

IGNITE-25534 Remove asserts

507f832

IGNITE-25534 Update isRollingUpgradeEligible method

6588353

IGNITE-25534 Update testCoordinatorChange: add assertThrows

e2f43f2

IGNITE-25534 Fixup assert for IgniteProductVersion#fromString

1b55ea2

IGNITE-25534 Fixup assert for IgniteProductVersion#fromString

3dfc9df

IGNITE-25534 Update errMsg for OsDiscoveryNodeValidationProcessor

f447957

IGNITE-25534 Update GridReleaseTypeSelfTest: replace assert stackTrac…

5ee547c

…e.contains with X#hasCause

IGNITE-25534 Update tests for GridReleaseTypeSelfTest

f744a30

IGNITE-25534 Add default comment for testDifferentServersAndClients

f5cde7f

chesnokoff added 15 commits September 28, 2025 21:38

IGNITE-25534 Update testDifferentServersAndClients

5781d06

IGNITE-25534 Remove nl

c650ec0

IGNITE-25534 Add empty doc

5203ff6

IGNITE-25534 Add docs

6591f50

IGNITE-25534 Small refactoring

582623f

IGNITE-25534 Fix checkstyle

6138c9a

IGNITE-25534 Fix javadoc

ddec3c1

IGNITE-25534 Remove MAX_MINOR_DIFF

48d4572

IGNITE-25534 Update GridReleaseTypeSelfTest: replace string key with …

6db16e8

…static field

IGNITE-25534 Refactor GridReleaseTypeSelfTest: rename withClient argu…

03bf71f

…ment to isClient

IGNITE-25534 Refactor IGNITE_ROLLING_UPGRADE_VERSION_CHECK system pro…

8539410

…perty: replace dots with underscores

Revert "IGNITE-25534 Fixup assert for IgniteProductVersion#fromString"

d0eee50

This reverts commit f80c523.

Revert "IGNITE-25534 Fixup assert for IgniteProductVersion#fromString"

41db07d

This reverts commit f640732.

Revert "IGNITE-25534 Update fix for race condition"

c31967a

This reverts commit 1918293.

Revert "IGNITE-25534 Fix race condition"

c4fef5e

This reverts commit c944ae4

chesnokoff force-pushed the ignite-25534 branch 2 times, most recently from c4fef5e to 8539410 Compare October 1, 2025 12:09

chesnokoff added 2 commits October 2, 2025 01:41

IGNITE-25534 Add ROLL_UP_VERSION_CHECK

d5961ba

IGNITE-25534 Add ROLL_UP_VERSION_CHECK

667e226

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

IGNITE-25534 Update Ignite versions validation for cluster #12301

IGNITE-25534 Update Ignite versions validation for cluster #12301

Uh oh!

chesnokoff commented Aug 27, 2025

Uh oh!

chesnokoff Sep 5, 2025 •

edited

Loading

Uh oh!

chesnokoff Sep 5, 2025 •

edited

Loading

Uh oh!

chesnokoff Sep 18, 2025 •

edited

Loading

Uh oh!

timoninmaxim Sep 5, 2025

Uh oh!

timoninmaxim Sep 5, 2025

Uh oh!

timoninmaxim Sep 5, 2025

Uh oh!

timoninmaxim Sep 5, 2025

Uh oh!

timoninmaxim Sep 5, 2025

Uh oh!

wernerdv Sep 11, 2025

Uh oh!

wernerdv Sep 11, 2025

Uh oh!

wernerdv Sep 11, 2025 •

edited

Loading

Uh oh!

sergey-chugunov-1985 Sep 18, 2025

Uh oh!

chesnokoff Sep 19, 2025 •

edited

Loading

Uh oh!

sonarqubecloud bot commented Sep 18, 2025

Uh oh!

Uh oh!


		String stackTrace = errors.toString();

		assert stackTrace.contains("Remote node rejected due to incompatible version for cluster join");


		assertTrue(waitForCondition(() -> Ignition.allGrids().size() == 2, getTestTimeout()));

		startGrid(3, "2.19.0", false);

IGNITE-25534 Update Ignite versions validation for cluster #12301

Are you sure you want to change the base?

IGNITE-25534 Update Ignite versions validation for cluster #12301

Uh oh!

Conversation

chesnokoff commented Aug 27, 2025

The Contribution Checklist

Notes

Uh oh!

chesnokoff Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chesnokoff Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chesnokoff Sep 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wernerdv Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chesnokoff Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Sep 18, 2025

Quality Gate passed

Uh oh!

Uh oh!

chesnokoff Sep 5, 2025 •

edited

Loading

chesnokoff Sep 5, 2025 •

edited

Loading

chesnokoff Sep 18, 2025 •

edited

Loading

wernerdv Sep 11, 2025 •

edited

Loading

chesnokoff Sep 19, 2025 •

edited

Loading