Skip to content

[Maven4] Migrate from Maven Archiver to standard jar tool - #508

Open
desruisseaux wants to merge 11 commits into
apache:masterfrom
Geomatys:feat/from-archiver-to-jar-tool
Open

[Maven4] Migrate from Maven Archiver to standard jar tool#508
desruisseaux wants to merge 11 commits into
apache:masterfrom
Geomatys:feat/from-archiver-to-jar-tool

Conversation

@desruisseaux

Copy link
Copy Markdown
Contributor

This is a major refactoring of the Maven JAR Plugin for using the standard jar tool instead of Maven Archiver. The standard tool is available through the java.util.spi.ToolProvider interface, which was introduced in Java 9. Since Maven 4 upgraded its requirement from Java 8 to Java 17, the use of that interface is no longer problematic.

Rational

In early Java days, the jar tool was equivalent to a zip command with a different syntax and a little bit of special processing for the META-INF/MANIFEST.MF file. Because of this quasi-equivalence, it was not difficult to create JAR files ourselves using any library capable to write ZIP files. This is what Maven Archiver does, together with supporting other archive formats. But today, the jar tool became more sophisticated. It now includes options for verifying the consistency of multi-release JAR files, options for updating module-info.class, provides security features specific to Java, etc.. This evolution can be seen in the "Modular JAR files" section of the Maven Plugin documentation, which states that the plugin uses the jar tool for updating the JAR file in a way that the Maven Archiver can't do easily. Therefore, since Maven 4 requires Java 17 and since that Java version gives us an easy access to the jar tool through the java.util.spi.ToolProvider interface, it may be time to abandon our manual creation of a JAR file and rely fully on the jar tool instead.

Benefits

Safer multi-release JAR files

In a multi-release JAR file, blindingly storing the content of META-INF/versions/ directories as if they were ordinary resources is not equivalent to using the jar --release option. The difference is that in the latter case, the jar tool performs some consistency checks. This issue was independently reported by user in #484, which is fixed by this pull request. Note that if this verification is not desired, Maven users can disable it by setting the detectMultiReleaseJar plugin option to false.

Main class managed by the jar tool

In a modular JAR file, it is no longer sufficient to declare the main class in the Main-Class entry of the MANIFEST.MF file. The main class needs to be specified by the --main-class option of the jar tool, which will update module-info.class. For compatibility reason the Maven JAR plugin gets the option value from that manifest entry, but internally the use of the jar tool is mandatory.

Options not yet supported by the plugin

This approach allows to add a <jarArgs> configuration option, similar to the <compilerArgs> in the Maven Compiler Plugin. Such option would allow developers to use new tool arguments before they are supported by the plugin.

Examples of options not yet supported by the plugin are --module-version and --hash-modules. Explicit support for those options could be added in a future plugin version, especially since security is becoming more and more a concern.

Easier debugging and toolchain

This approach makes easy to generate a target/jar.args file when the build fails or when Maven is run in verbose mode. This is similar to the compiler plugin generating a target/javac.args file. This file allows the user to test easily on the command-line, which makes debugging faster. Likewise, the options can also be passed to another tool, which makes easier to resolve #439 as well.

Behavioral changes

The plugin behaviour after the proposed refactoring is different than version 3 in the following aspects:

Removal of default **/package.html excludes

The current plugin version uses an undocumented **/package.html default excludes. This default seems to exist since the initial revision in March 2004, but I saw no explanation for this oddity. This default is not mentioned in the documentation. The removal of this oddity is necessary for allowing the proposed new plugin implementation to specify only some root directories to the jar tool, since the tool can traverse the directory tree itself.

Automatic use of MANIFEST.MF

After this refactor, the plugin automatically uses the META-INF/MANIFEST.MF file found in the classes directory. Before this refactor, the plugin used that file only if explicitly specified in the <manifestFile> archive configuration. The previous policy was discussed in #255. The new policy is a natural consequence of the way that the JAR plugin is reimplemented, and also more useful in the context of multi-module (in Java module sense) projects since each module could contain its own MANIFEST.MF file.

Multi-module projects

The refactoral support multi-module projects, including projects that are both multi-module and multi-release, as discussed in apache/maven-compiler-plugin#998.

Dependencies

This pull request depends on a Maven release which include the following pull requests:

@desruisseaux desruisseaux self-assigned this Dec 13, 2025
@desruisseaux desruisseaux added enhancement New feature or request java Pull requests that update Java code labels Dec 13, 2025
ascheman added a commit to support-and-care/java9-jigsaw-examples that referenced this pull request Dec 19, 2025
Update all m4/pom.xml to use maven-jar-plugin 4.0.0-beta-2-PR508-SNAPSHOT
which provides automatic JAR-per-module creation.

Additional changes:
- Convert example_addExports_manifest/m4/src/modmain from symlink to real
  directory to support MANIFEST.MF with Add-Exports and Main-Class
- Add Maven 4 Migration Notes to example_addExports_manifest/README.adoc
- Document resource handling in README.adoc

Dependencies:
- apache/maven-jar-plugin#508: Automatic JAR-per-module creation
- apache/maven#11505: Module-aware resource copying

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

Introduced in the course of support-and-care/maven-support-and-care#137
@desruisseaux
desruisseaux force-pushed the feat/from-archiver-to-jar-tool branch from f564cdc to 21db1f0 Compare July 20, 2026 16:50
@desruisseaux
desruisseaux force-pushed the feat/from-archiver-to-jar-tool branch 2 times, most recently from 5a235c8 to 5886fef Compare August 4, 2026 12:23
@desruisseaux
desruisseaux marked this pull request as ready for review August 4, 2026 12:24
@desruisseaux
desruisseaux requested a review from ascheman August 4, 2026 13:46
@ascheman

Copy link
Copy Markdown

Thanks for this migration, @desruisseaux — moving JAR creation to the jar ToolProvider with module-source-hierarchy + multi-release support (and the derived per-module POMs) is a big, elegant step.

Reviewing it, I dug into the CI failures, and they share a root theme: the multi-release/module handling in FileCollector/Archive depends on the unspecified Files.walkFileTree directory-iteration order and on jar-tool-vendor tolerances. So it passes on macOS/Temurin and fails on Linux/Zulu — i.e. on CI. I found six distinct manifestations and prepared an atomic fix + tests for each.

They're on aschemaven:feat/from-archiver-to-jar-tool (off your 5886fef) — compare. The full matrix (ubuntu/macos/windows × JDK 17/21 × rc-6) is green with all six, plus deterministic unit tests (ArchiveTest) and a records-jar-validate IT. Each is its own commit, so you can cherry-pick à la carte:

  1. jar --validate crashes on records (JDK 17/18)ToolExecutor runs --validate after create; the JDK 17/18 jar tool throws This feature requires ASM8 on any record (JDK-8282446, fixed in 19). Guard: skip validate when Runtime.version().feature() < 19. — 27cce4a
  2. Base-release dir → NPE — the base FileSet is seeded from whichever dir first creates the Archive; on some walk orders that's a versions-modular/<n>/<module> dir (no module-info), so ModuleFinder finds nothing and PomDerivation NPEs. Rebind the base to the version-less dir. — c8a3f66
  3. Main-Class on the wrong modulesetMainClass removes Main-Class from a manifest shared across modules, so the first module processed consumes it. Per-module manifest copy. — 18581f7
  4. Absolute jar entry namesFileSet.add relativizes only the first file; later files stay absolute and the jar tool records absolute .class entries ("names do not match"). Relativize all + repeat -C per file. — 63fe387
  5. Empty -C "" for version releases — a versions/<n> dir is added to its own FileSet → relativize(dir,dir)=""; Zulu rejects -C dir "", Temurin tolerates it. Emit .. — e71b7ad
  6. Base files leak into a version FileSet — a whole version dir is SKIP_SUBTREE'd, so postVisitDirectory (which resets to the base release) never runs; base files walked afterward land in the version FileSet as ../../../<pkg>. Reset inline. — 0ee85a8

Because these are order/vendor-dependent they only surface on CI (Linux+Zulu); the ArchiveTest unit tests pin the behaviour deterministically so it can't silently regress on any walk order. Happy to open a PR against your branch if that's easier — just say the word. Thanks again, this is really nice work.

…ard `java.util.spi.ToolProvider` API (requires Java 9+).

Files in the `classes` directories are dispatched to the `--manifest` and `--release` options, which allow additional verifications by the `jar` tool.
Derive a POM for each individual JAR file as the intersection of the project model and the module-info of the JAR file.

Side effects:
* Remove the default `**/package.html` exclude.
* Automatic use of `META-INF/MANIFEST.MF` file found in `classes` directory.
* Automatic Multi-Release always enabled, unless `detectMultiReleaseJar` is set to `false`.
* In a multi-module project, use Java module names as artifact names.
* Whether the `--date` option is supported depends on the Java version.
* Re-run the `jar` tool with the --validate operation mode if validation was not implicit.
@desruisseaux
desruisseaux force-pushed the feat/from-archiver-to-jar-tool branch from 5886fef to 046fbb2 Compare August 12, 2026 16:25
@desruisseaux

desruisseaux commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Thanks! I cherry-picked the commits with modifications:

  1. Rearranged the if and else blocks for avoiding duplicated tests and calls of Runtime.version().feature(). Omitted the part of the log message saying that the JAR was created successfully on JDK 17/18 for consistency with the JDK 19+ branch which emitted no message. Edited the comments and commit message with minor clarifications (the validation issue was not only in the context of module source hierarchy projects) and omission of details about JDK internal in commit (kept in commit message), replaced by links to JDK issue tracker.
  2. The commit repairs the problem after it happened (directory handled as the base version when actually it was another version). I think that the root cause of the problem is that the Archive constructor had no version argument. I tried a correction of that root cause instead. We will see with GitHub action if it works.
  3. Changes only in documentation and minor adjustments of the tests.
  4. Repeating the -C option with the path to the directory in front of every files to include in the JAR file seems a little bit extreme. The report generated by Claude does not explain why the iteration order is an issue. My hypothesis is that the first path must be the shortest one, so that next paths have a common prefix. I'm trying that and we will see what are the GitHub actions result.
  5. Same commit but adapted for the changes done in point 4 above.
  6. The root cause of the issue can be made clearer, but refactoring in a separated method the part of the postVisitDirectory(…) method that needs to be invoked when a directory is skipped.
  7. This integration test for step 1 has been squashed with commit 1.

@desruisseaux
desruisseaux force-pushed the feat/from-archiver-to-jar-tool branch 2 times, most recently from 3ea1283 to bb93fec Compare August 13, 2026 14:09
ascheman and others added 3 commits August 13, 2026 17:07
The plugin runs `jar --validate` after creating each archive.
With the `jar` tool bundled in JDK 17 and 18, that validation crashes on
any class compiled as a record (JDK-8282446, "This feature requires ASM8").
The issue was fixed in JDK 19 via the ASM 9.2 upgrade (JDK-8282508),
but the fix was not backported to JDK 17u/18u.
Since Maven 4 runs on JDK 17+, packaging a module that contains a record
may fail in project with default configuration if using a JDK older than 19.

Guard the validation pass on Runtime.version().feature() >= 19
and log an INFO line naming the skipped archive.
The archive was already created by the `--create` pass.
Behaviour on JDK 19+ is unchanged.

Add an IT which tests a modular JAR containing a record and expects the build to succeed,
so it turns the jdk-17 CI leg red until the validation pass is guarded for JDK 17/18.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Martin Desruisseaux <[email protected]>
`Archive` seeded its base (version-less) `FileSet` from whichever directory first created the `Archive` via
`computeIfAbsent(…)` in `FileCollector.enterModuleDirectory(…)`. Directory iteration order is unspecified,
so when the walk enters "META-INF/versions-modular/<n>/<module>" before the base "<module>" dir, the base
`FileSet` was bound to a version-specific directory with no module-info.class. The problem was not really
the version-specific directory, but its association to the `null` key in `filesetForRelease`, which makes
it the first map entry. Consequently `baseRelease().directory` pointed there, `ModuleFinder.of(...)` found
no module, and `PomDerivation` dereferenced a null `ModuleReference` -> NPE. The "green on JDK 19" was
accidental walk order, not a version gate.

Add a `version` argument to the `Archive` constructor, which keep the `null` key free for the real base
version when the walk will reach that version.

Bug description partially from Claude Opus 4.8 <[email protected]>
Co-Authored-By: Martin Desruisseaux <[email protected]>
`setMainClass(…)` unconditionally removes the `Main-Class` attribute from the plugin manifest
(to either pass it via `--main-class` instead or to ignore it), but that manifest is shared
across every module of a module hierarchy. The first module processed consumed the attribute,
so when a non-owning module was processed first (directory iteration order is unspecified),
the owning module never received its main class. Give each module a copy of the shared manifest
in `ToolExecutor.writeSingleJAR` so processing order no longer decides the outcome.

Add `ArchiveTest` with deterministic unit tests for this and for the base-release binding
(both previously exercised only by ITs whose result depends on filesystem walk order).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Martin Desruisseaux <[email protected]>
@desruisseaux
desruisseaux force-pushed the feat/from-archiver-to-jar-tool branch from bb93fec to 0238e7d Compare August 13, 2026 15:09
`FileSet.add(…)` relativized only the first file added and stored the rest as absolute paths.
This heuristic rule is based on the observation that is seems to be a `jar` tool requirement.
However, which file is first depends on the unspecified directory iteration order.
On some platforms, the tests fail with "names do not match" error message.
This make the first file deterministic by choosing the shortest one.

Issue identified by: Claude Opus 4.8 <[email protected]>
Co-Authored-By: Martin Desruisseaux <[email protected]>
@laeubi

laeubi commented Aug 19, 2026

Copy link
Copy Markdown

If that proves to be true and we can't find a workaround, that's a blocker. Reproducible builds are the more important feature.

I disagree that it is the most important feature. I think that they real feature you are asking for is a way to verify the integrity of a JAR (no malicious code). This is semantic reproducible builds, and something that we could provide.

Tycho has such comparison (and can even e.g. detect different order in manifest files)

but it is more used to compare if a version increment is needed to prevent deploying an artifact that (semantically) only differs in its promoted version.

@ascheman

Copy link
Copy Markdown

Thanks for creating the OpenJDK documentation issue about jar -C usage, @jaikiran. We should perhaps mention it in the comments about the respective workarounds, @desruisseaux.

@hboutemy

hboutemy commented Aug 19, 2026

Copy link
Copy Markdown
Member

it seems I'll have to revisit Reproducible Builds for the jar plugin: after so many years of success, this looks like a regression that is so tiring to have to work back on :/ ... but I suppose better Java Module support deserves such efforts

in the past, we did some JDK detection to patch when using JDK earlier than 18 https://issues.apache.org/jira/browse/MJAR-275 requiring plexus-archiver tweak codehaus-plexus/plexus-archiver#205
perhaps we'll have to do some equivalent tweaks now

@seregamorph

Copy link
Copy Markdown

If that proves to be true and we can't find a workaround, that's a blocker. Reproducible builds are the more important feature.

💯 , this is very critical to have proper reproducible builds

@seregamorph

Copy link
Copy Markdown

Regarding the target directory creation. The target directory should be only created if the JAR is to be produced. Now the pom assemblies create redundant empty target directories

@seregamorph

Copy link
Copy Markdown

I'd duplicate my message from the mailing list there, for history.
The current implementation of the plugin does not include the directory entries in the zip archive of JAR file.
As a result, this breaks spring applications component scan which relies on it.
The behavior should be preserved without any customization of the plugin configuration.
This is a critical blocker.

@seregamorph seregamorph left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested:

  1. target should be only created when JAR is to be created
  2. missing directory entries

@pzygielo

Copy link
Copy Markdown

As a result, this breaks spring applications component scan which relies on it.

Please link spring issue for reference. Directory entries are not required in a JAR/ZIP file.

@desruisseaux

desruisseaux commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

@hboutemy, @elharo: about the issue of adding the missing directories while preserving deterministic file order, a workaround may be to add in the pull request a post-generation step where the JAR file is read and rewritten with directory entries added using java.util.zip. That step would be executed only when files were enumerated (as opposed to specifying the directory to the jar tool). I will try to do that during the weekend. I will also check for the other issue raised by @seregamorph (too many target directories created).

Above proposal should preserve strict reproducible builds in the JAR plugin. For longer term, I still believe that bit-for-bit reproducible builds is too much and that we need semantically reproducible builds, but this is another story for maybe a separated plugin. Thanks @laeubi for the link to Tycho artifact comparator.

Thanks @jaikiran for stepping in. The issue is that for creating a reproducible build with the jar tool, we need to enumerate explicitly all the *.class files and resources in order to enforce an order that does not depend on the file system. But the problems with enumerating files are:

  • We have to repeat the -C option for every file (not a blocking issue).
  • When specifying foo/bar/X.class, the jar tool does not generate foo/ and foo/bar/ entries. It generates them if we specify the whole directory, but then the entry order is back to unspecified even if we are still enumerating the files in addition to the directories. While the absence of directory entries is legal according @pzygielo (thanks for the tip), it seems to be an issue with Spring.

A possible jar tool evolution could be to specify a new option in replacement of -C. It could be, for example, --change-dir. The new option would behave in the same way as -C with two differences:

  • All files after --change-dir would be relative to the specified directory instead of only the first file.
  • Adding foo/bar/X.class implies adding foo/ and foo/bar/.

@ascheman

Copy link
Copy Markdown

the jar tool does not seem to be designed for an explicit enumeration of files to include. It seems to be designed for working on directories only, in which case the entry order may depend on the tool or the platform.

@desruisseaux, I dug into this, and I think the directory-based form is the way out here rather than a dead end — the tool's directory order is not platform-dependent. Since JDK-8276764 ("enable deterministic file content ordering for jar and jmod", in JDK 17), the jar tool sorts entries by name when handed a directory, independent of the filesystem's iteration order or file timestamps.

Verified empirically — files created in a deliberately non-sorted order, then jar -C dir .:

  • macOS Temurin 17 & 21 and Linux Temurin 17 → byte-identical, name-sorted output every time.

So handing a directory to the tool gives us both reproducible entry order and the intermediate directory entries (com/, com/acme/, …) — the ones @seregamorph reported missing, which break Spring Boot component scanning. Per-file enumeration is exactly what drops those directory entries; the directory form keeps them and lets the tool sort for us. This also sidesteps a post-generation rewrite of the JAR — the directory entries come straight from the tool, so the archive is correct as produced (no second read/rewrite pass, and nothing extra to keep reproducible).

To make it concrete, I put together a branch on top of this PR — CI green on JDK 17 + 21 (full -P run-its, 3 OSes):

aschemaven/maven-jar-pluginbugfix/508-preserve-directory-entries

  • Plain JARs — hand the whole directory to the tool when unfiltered (base, non-modular, no physical MANIFEST.MF) instead of enumerating files. Fixes the missing-directory-entries regression. IT: directory-entries.
  • Non-modular multi-release — declare resource directories to the tool even in reproducible mode (the JDK 19+ tool sorts each, JDK-8276764), so base and versioned areas keep their directory entries. IT: directory-entries-mr.

Include/exclude filters and modular archives still fall back to per-file enumeration (they need it); that's documented in the code.

It's meant as a starting point — cherry-pick or adapt as you see fit; I didn't want to push commits onto your PR. Happy to open it as a PR against this branch if that's easier to review. And if any genuine jar-tool reproducibility gaps remain, @jaikiran's offer to take specifics to core-libs-dev is the right venue — but for entry ordering the tool already does what we need.

@ascheman

ascheman commented Aug 19, 2026

Copy link
Copy Markdown

@seregamorph — since you have the exact Spring Boot 4.1.0 reproduction, would you be willing to try the branch against your real sources? Commit 1 alone should already cover your case (a plain, single-classes-directory app) — that's the one that restores the missing directory entries.

To test:

git clone --branch bugfix/508-preserve-directory-entries https://github.com/aschemaven/maven-jar-plugin.git
cd maven-jar-plugin
mvn -DskipTests install        # builds & installs 4.0.0-beta-2-SNAPSHOT into your local repo (JDK 17+)

Then rebuild your project (it already resolves 4.0.0-beta-2-SNAPSHOT) and check that:

  • the Spring Boot application starts, and
  • jar -tf <your-app>.jar shows the com/… directory entries again.

It would be great to confirm it resolves the blocker on real sources. The full -P run-its suite is green on JDK 17 + 21 with this change, but your application is the case that actually matters here.

@laeubi

laeubi commented Aug 19, 2026

Copy link
Copy Markdown

I just wanted to note that directory entries are optional, so if spring relies on them to be present it would be better fixed there to work without it.

@seregamorph

seregamorph commented Aug 19, 2026

Copy link
Copy Markdown

I just wanted to note that directory entries are optional, so if spring relies on them to be present it would be better fixed there to work without it.

Let me give some clarity here. While I actually don't argue regarding this reasoning, absolutely most of spring boot projects as for today are based on Spring Boot 2 and 3. And Spring Boot 2 and 3 will never receive public updates anymore because of EoL OSS (there are only private for paid extended subscriptions), see https://spring.io/projects/spring-boot#support

@desruisseaux

Copy link
Copy Markdown
Contributor Author

@jaikiran : just for information, I was not aware of JDK-8276764. It makes --change-dir proposal less needed. Maybe there is no change needed in jar after all. We are exploring the options.

@seregamorph

Copy link
Copy Markdown
  1. @ascheman thanks, I've rechecked your PR and I'd like to confirm - it fixes the problem with directories, now the spring application is starting 👍
  2. the good news is that JARs are reproducible, at least on my machine/os/jvm version, and it's binary reproducible.
    There was a comment regarding "semantically reproducible" instead of "binary reproducible" - that's not acceptable, at least because it breaks Docker layered file system optimization. It can be lenient for the timestamp of files, but if the JAR file content is different (like different order or timestamp), it will not work. At scale that's a significant factor.
  3. what I've noticed is that MANIFEST.MF are now having two entries
Manifest-Version: 1.0
Created-By: 25.0.3 (Amazon.com Inc.)

while the original MANIFEST had only Manifest-Version: 1.0 @desruisseaux , please recheck this regression.
The configuration is

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <configuration>
        <archive>
            <addMavenDescriptor>false</addMavenDescriptor>
            <manifest>
                <addDefaultEntries>false</addDefaultEntries>
            </manifest>
        </archive>
    </configuration>
</plugin>

ascheman added a commit to aschemaven/maven-jar-plugin that referenced this pull request Aug 19, 2026
With addDefaultEntries=false, Maven archiver writes no Created-By and the
jar tool then stamps its own "<version> (<vendor>)" value. That is JDK-
and vendor-specific (Temurin and Corretto differ), so it breaks binary
reproducibility across JDK distributions and ignores addDefaultEntries=false.

Ensure a stable, JDK-independent Created-By (the value MavenArchiver
already supplies via setCreatedBy) is present before handing the manifest
to the jar tool. This also replaces the previous "let the tool declare
itself" branch, so non-reproducible builds no longer get the JDK-specific
value either.

Add the manifest-created-by integration test.

Reported by Sergey Chernov on apache#508.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@ascheman

Copy link
Copy Markdown

@seregamorph @desruisseaux @hboutemy

what I've noticed is that MANIFEST.MF are now having two entries Manifest-Version: 1.0 / Created-By: 25.0.3 (Amazon.com Inc.) while the original MANIFEST had only Manifest-Version: 1.0

Confirmed, and it is worth pinning down precisely because it is a reproducibility regression, not just a cosmetic one.

Root cause. The JDK jar tool stamps its own Created-By: <java.version> (<java.vendor>) into any manifest that does not already contain one — there is no way to suppress it short of -M/--no-manifest. Reproduced in isolation:

JDK jar --create --manifest=<minimal> yields
Temurin 17.0.18 Created-By: 17.0.18 (Eclipse Adoptium)
Corretto 21.0.10 Created-By: 21.0.10 (Amazon.com Inc.)

So the same sources produce different manifest bytes on different JDK distributions — Corretto's (Amazon.com Inc.) matches exactly what you see. With addDefaultEntries=true this never surfaces, because Maven archiver supplies a stable Created-By (Maven JAR Plugin <version>) and the jar tool keeps it. It only bites with addDefaultEntries=false, where Maven archiver deliberately omits it and the jar tool fills the gap with its JDK-specific value.

The decision (and its alternative). There are two ways to fix it, and they differ in what addDefaultEntries=false should mean under the jar tool:

  • (a) Always write a stable, JDK-independent Created-By. Even with addDefaultEntries=false, have the plugin set Created-By: Maven JAR Plugin <version> (the same value archiver uses with default entries) before handing the manifest to the jar tool, so the tool keeps ours. This keeps the artifact reproducible across JDK distributions. Trade-off: the manifest is not literally the Manifest-Version-only one you had — it carries a (JDK-independent) Created-By.
  • (b) Truly minimal manifest. Reproduce the exact pre-4.x output (no Created-By at all). Since the jar tool cannot be told to omit it, this needs either -M/--no-manifest with the manifest supplied as a regular entry (I verified this yields byte-identical manifests across Temurin and Corretto), or a post-generation manifest rewrite.

I went with (a) as a candidate on my branch bugfix/508-preserve-directory-entries — a small change in ToolExecutor plus a manifest-created-by integration test that fails if the final Created-By is the JDK-specific value. @desruisseaux, cherry-pick, adapt, or drop as you see fit — you may prefer (b) to honor addDefaultEntries=false literally.

One nuance for the record: strictly, reproducible builds are defined against a pinned build environment (the JDK included), so cross-JDK divergence is not a definitional violation. But Maven's own convention is deliberately JDK-independent manifests (archiver uses Apache Maven / Maven JAR Plugin, never the JDK string), and there are real practical costs to re-introducing the coupling — e.g. Docker layer dedup across CI runners on different JDK patch/vendor. That is why keeping Created-By JDK-independent seems worth it either way.

@desruisseaux

Copy link
Copy Markdown
Contributor Author

This is an example of why I hate binary reproducible builds. I think that Docker and other environments should adapt to semantically equivalent builds, but I realize that it will not happen soon.

I will need time for incorporating the changes and will try to do that Saturday. Some changes compared to the current commits were discussed on Slack. In particular, a more systematic approach may be desirable for the issue of missing directory entries.

@rmannibucau

Copy link
Copy Markdown
Contributor

a post-generation step where the JAR file is read and rewritten with directory entries added using java.util.zip

if this true? if so let's drop toolprovider and get back maven archiver, this is cleaner than a workaround which can be insanelt slower (twice at least) on big jars and try to push upstream the needed features, there is no point to move to a new tool which is twice worse for only a validation most people will not need

@desruisseaux

Copy link
Copy Markdown
Contributor Author

if this true?

Not any more. When I wrote this comment, I was not aware of JDK-8276764. The proposal to apply a post-processing step is obsolete now.

@hboutemy

Copy link
Copy Markdown
Member

I just tested building a few simple projects with misc JDKs and this PR:

mvn -V -e -Dversion.maven-jar-plugin=4.0.0-beta-2-SNAPSHOT -DskipTests clean package artifact:3.6.1:describe-build-output ; unzip -p target/*-SNAPSHOT.jar META-INF/MANIFEST.MF

first, I can see the the META-INF/MANIFEST.MF file remains as usual: great

from my test, with JDK 19+, result is locally reproducible: I always get the same output on the same laptop
I did not yet try to compare builds on different machines, I'll check later

with JDK 17 and 18, not reproducible: no problem with order of files, but with timestamp
and also a timestamp is injected in pom.properties

To me, avoid the issue with pom.properties should be easy
and postprocessing jar file to set timestamp should not be hard: like moditect for example https://github.com/moditect/moditect/blob/41784cb65f9a564012212ba94307ad98455d789d/core/src/main/java/org/moditect/commands/AddModuleInfo.java#L115

@laeubi

laeubi commented Aug 20, 2026

Copy link
Copy Markdown

So the same sources produce different manifest bytes on different JDK distributions

I wanted to note here that (binary) reproducible builds are not guaranteeing anyways between different JDK versions and vendors as they could provide different byte codes.

Apart from that, supplying the manifest directly would be the safest choice (also for the future) and thats why I think generation of files should be independent from packaging a jar:

Apart from that, having the validation of MR-jars baked into the jar tool is from my opinion a design flaw and actually what we aim here if I understand correctly, so if the JarOutputStream would have this support this would not only benefit maven but also others as this verification can then be reused see for reference here:

so if anyone is in contact with the JDK team it would be great to bring this to their attention as it would avoid people doing duplicate work an/or workarounds where a programmatic solution would be superior.

@hboutemy

Copy link
Copy Markdown
Member

continuing my base test (no MR or modular build yet, just basic classic jars)

rebuilding on a Mac in parallel of a Linux box with JDK 21 gave me same output: great, we have the output stability that I expect and got until now

So definitively, the JDK 17 and 18 reproducibility issue is what has to be solved for now

We'll see in a second step about modular or MR jars: if someone can point me to a simple project that I can build mself easily, that would be nice, please

one issue I get is twice attach:

[INFO] --- jar:4.0.0-beta-2-SNAPSHOT:jar (default-jar) @ maven-dependency-analyzer ---
[INFO] Building JAR: "target/maven-dependency-analyzer-1.17.2-SNAPSHOT.jar".
[WARNING] artifact 'org.apache.maven.shared:maven-dependency-analyzer:pom:consumer:1.17.2-SNAPSHOT' already attached, replacing previous instance

that is also visible in the artifact:3.6.1:describe-build-output too: and the message that says "replacing" is apparently not replacing but adding another attach
but perhaps it's more a Maven 4.0..0-rc-6 bug: if I drop the artifact:3.6.1:describe-build-output goal to the mvn command, this issue disappears
I'll open a Maven issue for that

@desruisseaux

Copy link
Copy Markdown
Contributor Author

I wanted to note here that (binary) reproducible builds are not guaranteeing anyways between different JDK versions and vendors as they could provide different byte codes.

I agree, and this is one reason why I disagree with the importance given to strict (as opposed to semantically) reproducible builds. I think that it can even deserve the real goal, which is security. If some Java tools produce vulnerable code (e.g., Javadoc HTML frame injection vulnerability in 2013), it is difficult to know if a JAR file has been produced by patched tools if we don't have a useful Created-By entry in the MANIFEST.MF file. But we are throwing away this information in the name of strict reproducible builds.

Regarding the split of metadata generation (MANIFEST.MF and Maven files) and packaging in two different phases, I agree with that. But I propose to go step by step:

  1. Fix the issue raised in above comments (missing directory entries, Created-By attribute).
  2. Get this pull request merged.
  3. In a separated pull request, copy (in modified form) in this plugin the Maven Archiver code which is still used. This is the code that generates the MANIFEST.MF attributes.
  4. Refactor in two separated goals.

Regarding the JAR file validation, this is one of the reasons for this migration to the jar tool but not the only one. There is also additional options such as --hash-modules. The support of this option is not included in this pull request because it will be a new feature, but it would be the topic of a future pull request.

@desruisseaux

Copy link
Copy Markdown
Contributor Author

postprocessing jar file to set timestamp should not be hard

But is is worth the effort? It would be a hack needed for JDK 17 and 18 only. And since the file ordering issue (JDK-8276764) was fixed in JDK 18 only, even if we fix the timestamp the build would still non-reproducible on JDK 17. Therefore, setting the timestamp would be useful for only one specific JDK version: 18.

@jaikiran

jaikiran commented Aug 20, 2026

Copy link
Copy Markdown
Member

Hello Martin,

And since the file ordering issue (JDK-8276764) was fixed in JDK 18 only, even if we fix the timestamp the build would still non-reproducible on JDK 17.

Bug fixes/enhancements in the JDK are sometimes backported. Those backports are tracked/linked against the original issue. For this specific issue https://bugs.openjdk.org/browse/JDK-8276764 , under the issue description, there's a backports section which shows that this enhancement was backported to JDK 17 (the Resolved/Fixed status on those backports is a sign that the work is complete).

Having said that, my comment is mostly informational and isn't meant as an input to whether or not some post processing should be done for the JAR files.

@seregamorph

Copy link
Copy Markdown

IMO it's critical to support binary reproducibility for all JDKs since minimal 17, otherwise it'll be a regression.

@desruisseaux

Copy link
Copy Markdown
Contributor Author

Thanks @jaikiran for the clarification. Then indeed, editing timestamps in a post-processing would work on both JDK 17 and 18.

Whether we should do this post-processing is still uncertain to me. Not doing this post-processing would be a regression only for users wanting binary reproducible builds with the tools of JDK 17 or 18. Users who do not upgrade to a more recent Java version at least for building (even if they target a lower release with --release) may be conservative regarding their Maven upgrade too, in which case the 3.x plugins continue to work as today.

@rmannibucau

Copy link
Copy Markdown
Contributor

IMO it's critical to support binary reproducibility for all JDKs since minimal 17, otherwise it'll be a regression.

stating that for maven 4 java 17 bytecode is supported using a java 21 or more (to speak "LTS") doesn't sound crazy to me
also agree it is reproducible only for the same JVM baseline (not even minor matching) so on my side it stays not critical since covered technically if well documented

like moditect for example

ZipFileSystem#sync can be worth benching on a big jar (some thousands of classes), ideally it is very small compared to producing the zip (jar) itself but worth evaluating before thinking this is an option IMHO - since we know we can always do it right directly if needed and JDK version is a blocker (once again not for me)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request java Pull requests that update Java code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[MJAR-289] Support toolchain

10 participants