Skip to content

storm-dist: fail early when sibling build outputs are missing from the binary package - #9281

Open
linzch3 wants to merge 1 commit into
apache:masterfrom
linzch3:storm-dist-require-files
Open

linzch3 wants to merge 1 commit into
apache:masterfrom
linzch3:storm-dist-require-files

Conversation

@linzch3

@linzch3 linzch3 commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Closes #7724 (STORM-3942).

What is the purpose of the change

The binary distribution in storm-dist/binary/final-package copies files that other modules build: the lib-* directories from the *-bin siblings, and the UI files from storm-webapp/target/classes/WEB-INF. maven-assembly-plugin silently skips a fileSet whose directory doesn't exist, so if one of those wasn't built you still get BUILD SUCCESS and a tarball that is missing pieces.

For example, on master, running mvn package in storm-dist/binary/final-package alone (the *-bin modules not built) succeeds, and the tarball has no lib-webapp, lib-common, lib-autocreds, submit-tools or kafka-monitor jars.

I added a maven-enforcer-plugin requireFilesExist execution to final-package, bound to prepare-package (right before those files are used). It checks the five sibling lib-* directories plus storm-webapp/target/classes/WEB-INF/index.html and WEB-INF/dist, and fails with the list of missing paths and a short hint on how to build them.

Two notes:

  • The paths start from ${maven.multiModuleProjectDirectory} instead of ${project.basedir}/../.., because requireFilesExist compares each path with its canonical form and treats anything with .. as missing. The root pom already uses that property for the rewrite and checkstyle scripts.
  • I didn't add anything to storm-dist/source. Its assembly only reads the source tree (the repo root and two READMEs under external/), nothing that a build produces, so I couldn't find a file there that can be missing in a normal checkout. Happy to add checks there too if you had something specific in mind.

Happy to adjust if you'd prefer a different approach (for example checking more of the files in common.xml, or a different phase).

How was the change tested

Locally with JDK 25 and Maven 3.9.11, after installing storm-core, storm-webapp, storm-submit-tools, storm-autocreds and storm-kafka-monitor:

  • mvn package -f storm-dist/binary/pom.xml -Dgpg.skip -DskipTests (same as the nightlies job): the rule passes and both tarballs are built
  • with storm-webapp/target/classes/WEB-INF/dist moved away, mvn package in storm-dist/binary fails and names that path
  • after mvn clean on storm-dist/binary, mvn package in final-package alone fails and lists the five missing lib-* directories (on master the same command succeeds with the incomplete tarball described above)

I didn't run the full -Pdist release build.

An AI assistant helped draft this change; I reviewed it and take responsibility for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[STORM-3942] mvn package in storm-dist relies on files being available in sibling directories

1 participant