Skip to content

feat: new launcher using Apache Mina sshd library #570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 24 commits into
base: main
Choose a base branch
from

Conversation

kuisathaverat
Copy link
Contributor

@kuisathaverat kuisathaverat commented Apr 6, 2025

This pull request introduces several enhancements and new features related to SSH connection management. It implements a brand new launcher base in Apache Mina SSH library.
This new launcher is in the early stages of development and is not yet fully functional, but it lays the groundwork for future improvements.

Screenshot 2025-06-28 at 17 52 18

SSH Connection Management:

  • Added Connection interface to define methods for managing SSH connections, including executing commands, copying files, and setting various connection parameters. (src/main/java/io/jenkins/plugins/sshbuildagents/ssh/Connection.java)
  • Implemented ConnectionImpl class using Apache Mina SSHD library to provide concrete functionality for the Connection interface, including authentication, session management, and file transfer via SCP. (src/main/java/io/jenkins/plugins/sshbuildagents/ssh/mina/ConnectionImpl.java)
  • Added ShellChannel interface for managing non-interactive SSH sessions, facilitating command execution and I/O stream handling. (src/main/java/io/jenkins/plugins/sshbuildagents/ssh/ShellChannel.java)

Dependency Management:

  • Updated pom.xml to include new dependencies (eddsa-api, mina-sshd-api-common, mina-sshd-api-core, mina-sshd-api-scp) and added exclusions for trilead-api in specific dependencies to improve compatibility and functionality. (pom.xml)

Code Style and Build Configuration:

  • Modified .editorconfig to include Java-specific settings such as indent_style, indent_size, and charset. (.editorconfig)
  • Updated Jenkinsfile to use JDK 21 for Windows builds, ensuring consistency across platforms. (Jenkinsfile)

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests - that demonstrates feature works or fixes the issue

JENKINS-64106

@kuisathaverat kuisathaverat force-pushed the apache_mina branch 2 times, most recently from db78481 to beb4353 Compare April 6, 2025 17:35
@kuisathaverat kuisathaverat self-assigned this Apr 6, 2025
@kuisathaverat
Copy link
Contributor Author

kuisathaverat commented Apr 7, 2025

Not sure about the CI errors, 106 test run, and there is no failures. It fails when it try to make the Surefire report

[2025-04-07T07:35:51.011Z] [INFO] 
[2025-04-07T07:35:51.011Z] [INFO] Results:
[2025-04-07T07:35:51.012Z] [INFO] 
[2025-04-07T07:35:51.012Z] [WARNING] Tests run: 106, Failures: 0, Errors: 0, Skipped: 11
[2025-04-07T07:35:51.012Z] [INFO] 
[2025-04-07T07:35:51.012Z] [INFO] ------------------------------------------------------------------------
[2025-04-07T07:35:51.012Z] [INFO] BUILD FAILURE
[2025-04-07T07:35:51.012Z] [INFO] ------------------------------------------------------------------------
[2025-04-07T07:35:51.012Z] [INFO] Total time:  40:50 min
[2025-04-07T07:35:51.012Z] [INFO] Finished at: 2025-04-07T07:35:50Z
[2025-04-07T07:35:51.013Z] [INFO] ------------------------------------------------------------------------
[2025-04-07T07:35:51.013Z] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.2:test (default-test) on project ssh-slaves: 
[2025-04-07T07:35:51.013Z] [ERROR] 
[2025-04-07T07:35:51.013Z] [ERROR] See /home/jenkins/agent/workspace/Plugins_ssh-agents-plugin_PR-570/target/surefire-reports for the individual test results.
[2025-04-07T07:35:51.013Z] [ERROR] See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[2025-04-07T07:35:51.013Z] [ERROR] ExecutionException There was an error in the forked process
[2025-04-07T07:35:51.013Z] [ERROR] [SUREFIRE] std/in stream corrupted
[2025-04-07T07:35:51.014Z] [ERROR] java.io.IOException: Stream closed
[2025-04-07T07:35:51.014Z] [ERROR] 	at java.base/java.io.BufferedInputStream.ensureOpen(BufferedInputStream.java:206)
[2025-04-07T07:35:51.014Z] [ERROR] 	at java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:411)
[2025-04-07T07:35:51.014Z] [ERROR] 	at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
[2025-04-07T07:35:51.014Z] [ERROR] 	at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
[2025-04-07T07:35:51.014Z] [ERROR] 	at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:347)
[2025-04-07T07:35:51.015Z] [ERROR] 	at java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420)
[2025-04-07T07:35:51.015Z] [ERROR] 	at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399)
[2025-04-07T07:35:51.015Z] [ERROR] 	at org.apache.maven.surefire.api.util.internal.Channels$3.readImpl(Channels.java:169)
[2025-04-07T07:35:51.015Z] [ERROR] 	at org.apache.maven.surefire.api.util.internal.AbstractNoninterruptibleReadableChannel.read(AbstractNoninterruptibleReadableChannel.java:50)
[2025-04-07T07:35:51.015Z] [ERROR] 	at org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:430)
[2025-04-07T07:35:51.015Z] [ERROR] 	at org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:419)
[2025-04-07T07:35:51.015Z] [ERROR] 	at org.apache.maven.surefire.api.stream.AbstractStreamDecoder.readMessageType(AbstractStreamDecoder.java:116)
[2025-04-07T07:35:51.016Z] [ERROR] 	at org.apache.maven.surefire.booter.stream.CommandDecoder.decode(CommandDecoder.java:77)
[2025-04-07T07:35:51.016Z] [ERROR] 	at org.apache.maven.surefire.booter.spi.CommandChannelDecoder.decode(CommandChannelDecoder.java:60)
[2025-04-07T07:35:51.016Z] [ERROR] 	at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:290)
[2025-04-07T07:35:51.016Z] [ERROR] 	at java.base/java.lang.Thread.run(Thread.java:1583)

@jglick
Copy link
Member

jglick commented Apr 9, 2025

Are any tests attempting to write to stdout/stderr (not using System.out / System.err)? I recall that for example https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/ProcessBuilder.html#inheritIO() can crash Surefire.

@kuisathaverat
Copy link
Contributor Author

I do not remember any, but I will review it.

@kuisathaverat
Copy link
Contributor Author

kuisathaverat commented Apr 14, 2025

Jesse was right; the cause was a bunch of outputs to System.err and System.out. Now, I have to resolve a few SpotBugs warnings.

@kuisathaverat kuisathaverat force-pushed the apache_mina branch 11 times, most recently from 0b2fcc4 to df049bb Compare June 9, 2025 17:48
@kuisathaverat kuisathaverat marked this pull request as ready for review June 10, 2025 05:42
@kuisathaverat kuisathaverat requested a review from a team as a code owner June 10, 2025 05:42
@kuisathaverat
Copy link
Contributor Author

kuisathaverat commented Jun 10, 2025

This is the initial implementation; it lacks host key verifications and other features the current launcher has. These features will be added in follow-up PRs.

@jglick
Copy link
Member

jglick commented Jun 20, 2025

Did you mean to update the PR title?

@kuisathaverat kuisathaverat changed the title test: implement launcher using Apache Mina sshd library feat: new launcher using Apache Mina sshd library Jun 22, 2025
@kuisathaverat kuisathaverat force-pushed the apache_mina branch 2 times, most recently from 71669ee to 3568ca0 Compare June 22, 2025 13:25
@olamy
Copy link
Member

olamy commented Jun 29, 2025

I will address some issues and then merge them as is. This PR is a WIP new launcher, and this is the first iteration. It is not perfect, and it should not be. I do not want to make this PR an open issue that remains unmerged. Also, it helps to work on different improvements in parallel and track the changes better

If you do so, you'll need to mark and let users know about the experimental aspect of your PR when they use the new launcher because many features are not working now (maybe some warning as well when using this using the new launcher)

Yes, it will be marked as experimental, not production-ready in the UI before I merge.

Maybe changing the PR title as well?

I completely understand the rationale behind adding an abstraction layer on top of the SSH connection. In theory, it makes sense and can offer flexibility. That said, I'm wondering if it's really necessary in the specific context of the Jenkins SSH Agents plugin. Do we foresee switching from Apache Mina to another implementation anytime soon (while it took already so long having some changes out of trilead :))? From what I can see, both the SSH protocol and the Apache Mina library are quite stable and well-maintained, especially when it comes to bugs and security.
Introducing a new abstraction would mean committing to a public API that we'll need to support in the long term. Given that this is currently scoped just to the Jenkins SSH Agents plugin. I'm not sure the added complexity is justified.

I have a different opinion that the abstraction layer makes it easy to adapt the Apache Mina SSH library to the terms of Jenkins, making it easy to troubleshoot and locate errors in the implementation. Makes the Launcher more modular, making it easy to understand what each part does. Each part functions as a black box, performing a single task. To have an abstraction layer does not affect security or updates from the Apache Mina SSH library. We will continue to update the library with new releases.

Not sure you got my point. I mean I'm not sure there is need for an interface Connection and ConnectionImpl and especially make this a public API because at the end of day there will be only a single implementation.

Why not simply make a single class MinaConnection package private so it will not be seen as a public API to maintain. same for other new interfaces.

Also, I'm a bit hesitant about re-implementing the authentication layer from scratch. The existing solution is widely used, battle-tested, and already aligns with security requirements. https://github.com/jenkinsci/mina-sshd-api-plugin/tree/main/mina-sshd-api-core/src/main/java/io/jenkins/plugins/mina_sshd_api/core/authenticators

At the time I started this launcher, that code did not exist, I would evaluate if I can reuse it, but not in this PR.

I wish to have you open to comments as definitely such changes need some discussion and acceptance by the broader community, as this has some large effect on the long term maintenance of the plugin

I am open to any suggestions you have. I have been part of the Jenkins community for over 10 years, maintaining six active plugins and having previously maintained four more. My plugins are all well-maintained and up-to-date, and I am always open to accepting contributions and suggestions. The Jenkins community is nothing new to me. I have always been an active member of the community.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants