Bump jruby 10.0 logstash 8.19

[andsel]

Release notes

TODO

What does this PR do?

This PR backports ecadb4a to 8.19 with some adaptations, that make it non clean:

• in 8.19 we don't have batch metrics
• BufferedTokenizerExt is not a wrapped around the pure Java BufferedTokenizer but uses JRuby arrays
• CompiledPipeline doesn't have the chunker

Description of commits:

• first commit is the cherry-pick with some adaptations related to deprecated JRuby methods.
• second commit fix remaining deprecations introduced by JRuby 10.0.
• third commit renames lockfile from Ruby 3.1 to 3.4
• fourth commit pin the Bundler version shipped with JRuby 10.0 (2.7.2 instead of 2.6.3).

Why is it important/What is the impact to the user?

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files (and/or docker env variables)
• I have added tests that prove my fix is effective or that my feature works

Author's Checklist

• TODO update the gemfile lock
• TODO regenerate NOTICE.txt
• TODO verify licences

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• run exhaustive tests : Run the exhaustive tests Buildkite pipeline.

[cla-checker-service]

💚 CLA has been signed

[mergify]

This pull request does not have a backport label. Could you fix it @andsel? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit.
• If no backport is necessary, please add the backport-skip label

[infra-vault-gh-plugin-prod]

💔 Build Failed

• Buildkite Build
• Commit: 7460073

Failed CI Steps

• :rspec: Ruby unit tests
• :rspec: Ruby unit tests
• :rspec: Ruby unit tests
• :rspec: Ruby unit tests
• :rspec: Ruby unit tests - FIPS mode
• :rspec: Ruby unit tests - FIPS mode
• :rspec: Ruby unit tests - FIPS mode
• :rspec: Ruby unit tests - FIPS mode
• Observability SRE container build
• Observability SRE container build
• Observability SRE container build
• Observability SRE container build
• 🥼 Integration Tests - FIPS mode / part 1-of-6
• 🥼 Integration Tests - FIPS mode / part 1-of-6
• 🥼 Integration Tests - FIPS mode / part 1-of-6
• 🥼 Integration Tests - FIPS mode / part 1-of-6
• 🥼 Integration Tests - FIPS mode / part 2-of-6
• 🥼 Integration Tests - FIPS mode / part 2-of-6
• 🥼 Integration Tests - FIPS mode / part 2-of-6
• 🥼 Integration Tests - FIPS mode / part 2-of-6
• 🥼 Integration Tests - FIPS mode / part 3-of-6
• 🥼 Integration Tests - FIPS mode / part 3-of-6
• 🥼 Integration Tests - FIPS mode / part 3-of-6
• 🥼 Integration Tests - FIPS mode / part 3-of-6
• 🥼 Integration Tests - FIPS mode / part 4-of-6
• 🥼 Integration Tests - FIPS mode / part 4-of-6
• 🥼 Integration Tests - FIPS mode / part 4-of-6
• 🥼 Integration Tests - FIPS mode / part 4-of-6
• 🥼 Integration Tests - FIPS mode / part 5-of-6
• 🥼 Integration Tests - FIPS mode / part 5-of-6
• 🥼 Integration Tests - FIPS mode / part 5-of-6
• 🥼 Integration Tests - FIPS mode / part 5-of-6
• 🥼 Integration Tests - FIPS mode / part 6-of-6
• 🥼 Integration Tests - FIPS mode / part 6-of-6
• 🥼 Integration Tests - FIPS mode / part 6-of-6
• 🥼 Integration Tests - FIPS mode / part 6-of-6
• 🥼 Integration Tests / part 1-of-6
• 🥼 Integration Tests / part 1-of-6
• 🥼 Integration Tests / part 1-of-6
• 🥼 Integration Tests / part 1-of-6
• 🥼 Integration Tests / part 2-of-6
• 🥼 Integration Tests / part 2-of-6
• 🥼 Integration Tests / part 2-of-6
• 🥼 Integration Tests / part 2-of-6
• 🥼 Integration Tests / part 3-of-6
• 🥼 Integration Tests / part 3-of-6
• 🥼 Integration Tests / part 3-of-6
• 🥼 Integration Tests / part 3-of-6
• 🥼 Integration Tests / part 4-of-6
• 🥼 Integration Tests / part 4-of-6
• 🥼 Integration Tests / part 4-of-6
• 🥼 Integration Tests / part 4-of-6
• 🥼 Integration Tests / part 5-of-6
• 🥼 Integration Tests / part 5-of-6
• 🥼 Integration Tests / part 5-of-6
• 🥼 Integration Tests / part 5-of-6
• 🥼 Integration Tests / part 6-of-6
• 🥼 Integration Tests / part 6-of-6
• 🥼 Integration Tests / part 6-of-6
• 🥼 Integration Tests / part 6-of-6
• 🥼 IT Persistent Queues / part 1-of-6
• 🥼 IT Persistent Queues / part 1-of-6
• 🥼 IT Persistent Queues / part 1-of-6
• 🥼 IT Persistent Queues / part 1-of-6
• 🥼 IT Persistent Queues / part 2-of-6
• 🥼 IT Persistent Queues / part 2-of-6
• 🥼 IT Persistent Queues / part 2-of-6
• 🥼 IT Persistent Queues / part 2-of-6
• 🥼 IT Persistent Queues / part 3-of-6
• 🥼 IT Persistent Queues / part 3-of-6
• 🥼 IT Persistent Queues / part 3-of-6
• 🥼 IT Persistent Queues / part 3-of-6
• 🥼 IT Persistent Queues / part 4-of-6
• 🥼 IT Persistent Queues / part 4-of-6
• 🥼 IT Persistent Queues / part 4-of-6
• 🥼 IT Persistent Queues / part 4-of-6
• 🥼 IT Persistent Queues / part 5-of-6
• 🥼 IT Persistent Queues / part 5-of-6
• 🥼 IT Persistent Queues / part 5-of-6
• 🥼 IT Persistent Queues / part 5-of-6
• 🥼 IT Persistent Queues / part 6-of-6
• 🥼 IT Persistent Queues / part 6-of-6
• 🥼 IT Persistent Queues / part 6-of-6
• 🥼 IT Persistent Queues / part 6-of-6
• 🥼 x-pack unit tests

History

• 💔 Build #5050 failed 14e2234
• 💔 Build #5049 failed db61dbf

cc @andsel

[github-actions]

TL;DR

Build 5051 is failing primarily during dependency bootstrapping, not in the integration scenarios. All 18 integration jobs fail before running examples because multi_json 1.21.1 attempts to load options_cache/concurrent_store, while the observability SRE image fails because JRuby 10 activates cgi 0.4.2 but the lockfile requires cgi 0.3.7.

Remediation

• Pin multi_json below 1.20 (or explicitly to the compatible 1.19.x release) in logstash-core/logstash-core.gemspec; the current ~> 1.19.1 constraint permits 1.21.1, despite the adjacent comment saying the dependency is pinned.
• Regenerate Gemfile.jruby-3.4.lock.release with JRuby 10.0.3.0/Bundler 2.7.2 so the lockfile is consistent with JRuby 10’s default cgi 0.4.2 (or remove the stale explicit cgi dependency if it is not required).
• Re-run the integration matrix and the SRE container build after the dependency changes; the current logs show no integration examples executed.

Investigation details

Root Cause

The integration failures are identical across normal, FIPS, and persistent-queue shards: RSpec aborts while loading logstash-core/lib/logstash/elasticsearch_client.rb, before examples run, with cannot load such file -- .../multi_json-1.21.1/lib/multi_json/options_cache/concurrent_store. The failure is reported in the representative log at lines 23-43 and the summary is 0 examples, 0 failures, 4 errors at lines 63-65.

The gemspec currently declares multi_json, "~> 1.19.1" at logstash-core/logstash-core.gemspec:81; that range allows later 1.x releases. The PR’s lockfile rename only changes the BUNDLED WITH entry from 2.6.3 to 2.7.2 (Gemfile.jruby-3.1.lock.release -> Gemfile.jruby-3.4.lock.release, diff lines 61-73), so the QA integration bundle can resolve the incompatible newer release.

The SRE container log reports at lines 43-45: You have already activated cgi 0.4.2, but your Gemfile requires cgi 0.3.7. This is a lockfile/default-gem mismatch introduced by moving to JRuby 10; the build fails in Bundler setup before plugin packaging (BUILD FAILED at line 131).

The normal Ruby unit job also has one failed test (79 tests, 76 successes, 1 failures, 2 skipped, lines 91-103), but the retained log contains no test name or assertion, so it cannot be attributed confidently to a source change. The FIPS unit log likewise retains only the Gradle failure tail. These should be re-evaluated after dependency bootstrap is fixed.

Evidence

• Build: https://buildkite.com/elastic/logstash-pull-request-pipeline/builds/5051
• Jobs: 18 integration shards, the SRE container build, and the Ruby unit jobs failed.
• Key integration log: logstash-pull-request-pipeline-lab_coat-integration-tests-part-1-of-6.txt:23-43
• Key container log: logstash-pull-request-pipeline-observability-sre-container-build.txt:43-45

Verification

• Inspected all 21 failed-job summaries and representative normal/FIPS/persistent-queue logs.
• Confirmed no flaky-test issue matches the multi_json/concurrent_store failure.
• No code changes or local test reruns were performed in this read-only investigation.

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.