[MSHADE-434] Strange side-effect on rat-plugin

[jira-importer]

Chesnay Schepler opened MSHADE-434 and commented

When upgrading the shade-plugin from 3.2.4 to 3.3.0, without changing anything else, the Apache Flink builds are getting failed by the rat-plugin, complaining about a number of files in the root target/ directory.

These files do exist, but are usually excluded by the plugin.
For some reason upgrading the shade-plugin is changing this behavior.

[WARNING] Files with unapproved licenses:
  /home/chesnay/dev/repos/flink/flink/target/rat.txt
  /home/chesnay/dev/repos/flink/flink/target/checkstyle-result.xml
  /home/chesnay/dev/repos/flink/flink/target/checkstyle-cachefile

This is reeeeally strange, and were wondering if you guys have any ideas for what could going on here.

Interestingly enough explicitly listing these files (e.g., **/rat.txt) results in them being excluded again.

This happened with Java 8 + Maven 3.2.5 and Java 11 + Maven 3.8.6 and is 100% reproducible.

---

Affects: 3.3.0

Issue Links:

• MSHADE-321 Always respect 'createDependencyReducedPom' flag
  ("is caused by")
• FLINK-30083 Bump maven-shade-plugin to 3.4.1

[jira-importer]

Slawomir Jaranowski commented

After some of tests:

Old version 3.1.1

[DEBUG] Configuring mojo 'org.apache.rat:apache-rat-plugin:0.13:check' with basic configurator -->
...
[DEBUG]   (f) basedir = /Users/slawomir.jaranowski/repos/temp/flink
[DEBUG]   (f) project = MavenProject: org.apache.flink:flink-parent:1.17-SNAPSHOT @ /Users/slawomir.jaranowski/repos/temp/flink/pom.xml
...

 
New version:

DEBUG] Configuring mojo 'org.apache.rat:apache-rat-plugin:0.13:check' with basic configurator -->
...
[DEBUG]   (f) basedir = /Users/slawomir.jaranowski/repos/temp/flink/target
[DEBUG]   (f) project = MavenProject: org.apache.flink:flink-parent:1.17-SNAPSHOT @ /Users/slawomir.jaranowski/repos/temp/flink/target/dependency-reduced-pom.xml
...

Looks like new reduced pom is now used as project pom ...

Flink build looks a little complicated.

Why you check licenses by RAT at the one of last phase - verify.
Change phase of RAT to standard validate will help you.

[jira-importer]

Chesnay Schepler commented

??Why you check licenses by RAT at the one of last phase - verify.??
I don't quite remember why we run it in the verify phase.

??Change phase of RAT to standard validate will help you.??
We can certainly work around this particular issue; that's not really the problem at hand.

What I worry about is that this may also affect other plugins in some way that isn't as obvious.

[jira-importer]

Chesnay Schepler commented

??Looks like new reduced pom is now used as project pom ...??

Why would this affect the execution of another plugin though? AFAICT the content of the reduced pom still contains original rat-plugin configuration without any changes.

[jira-importer]

Chesnay Schepler commented

Could this be due to MSHADE-321?

If I read the code correctly than before this change no dependency reduced pom was created when the module creates no artifact, which is the case for modules with the pom packaging. Now it is always created, independent of what the packaging is.

This would at least explain the changes that sjaranowski  noticed about the basedir/project. And this could maybe break exclusions because they are relative paths after all.

[jira-importer]

Chesnay Schepler commented

After changing the rat-plugin to run in the validate phase our checkstyle-plugin is now complaining in submodules that it can't find the suppression file in the root directory anymore :/

[jira-importer]

Christofer Dutz commented

I can confirm the same behaviour in the Apache PLC4X build. Here we are stuck at 
maven-shade-plugin as updating to anything above 3.2.4 causes the rat plugin to complain about stuff in the target directory as soon as the shade plugin is being used.

[jira-importer]

Christofer Dutz commented

Thanks to your hint, I moved the rat execution to the test phase and with that I was able to go to the latest shade plugin version.

[jira-importer]

Tamas Cservenak commented

Why is RAT plugin using pom directory as "basedir"? Theoretically POM can be anywhere, RAT should ask for MavenProject basedir instead, or we have some bug around?

[elharo]

per gemini:

This is an issue that should be fixed in the maven-shade-plugin, not closed. The Maven team has correctly classified it as a bug with priority:major.

The core issue stems from an unintended side effect of MSHADE-321 (Always respect 'createDependencyReducedPom' flag), introduced in version 3.3.0 of the plugin.

Here is why this is a legitimate bug in the shade plugin:

1. Context Mutation: When the shade plugin runs, it generates a dependency-reduced-pom.xml file. The bug occurs because the plugin tells the core Maven execution context that the current project's POM is now located at target/dependency-reduced-pom.xml instead of the original pom.xml in the base directory.
2. Cascading Failures (Breaking the Build Environment): Because Maven's basedir property is inferred from the location of the active pom.xml file, moving the active POM into the target/ directory causes the project's basedir to shift from the root directory to the target/ directory for any plugin that executes after the shade plugin.
3. Widespread Plugin Impact: This shifted basedir breaks plugins that rely on relative paths from the project root. In the reported issues, it caused the apache-rat-plugin (license checker) to fail because it suddenly started scanning from target/ and its relative exclusions broke. It also broke the maven-checkstyle-plugin because it could no longer find its suppression file using a path relative to the root directory.

While users can work around this by changing the phase of their other plugins (e.g., running rat in validate or test before shade runs in package), this is a breaking change in behavior. A plugin should not silently mutate the build's base directory context for downstream plugins.