Maven Multi-module Builds with Centralized Version Management

Categories: Java

The Problem

Given a multi-module Maven project where all modules should have the same version-number, how can the version-numbers best be managed? In particular, how can the released artifacts be assigned a non-snapshot version, and how can developers continue to work on the right code following a release?

One solution is to use literal version strings in each pom-file and use the versions plugin to update these literal strings; running mvn versions:set ... updates all poms in a directory-tree consistently. The primary problem with this solution is that the modified files need to be committed to version-control - and potentially once in the “release branch” and once in the “development branch”. When a project consists of many modules, the commits can be large and ugly.

This article describes how versioning actually works in Maven, and describes a solution that requires no pom changes at all to do a release while still producing valid artifacts in the repository. Note that there is nothing actually new here; this is just a (in my opinion) more readable presentation of the Maven CI Friendly page and the flatten maven plugin documentation.

But if you don’t care about all the background reasons, and just want a clean multi-module build, follow the Maven CI Friendly page.

This information applies to Maven 3.6.3 (current version as at 2020-06).

Parent Poms and Multi-Module Builds

The concepts of a “parent pom” and the concept of a “module pom” are two quite different things in Maven.

Any pom-file (aka Maven module) can inherit configuration from a parent pom by defining a <parent> clause.

A pom-file with packaging=pom can include a <modules> clause which contains a list of directories that include pom-files. Running a command on this pom-file causes all referenced pom-files to be loaded into memory, ordered by inter-module dependencies, and then the requested command is run on each module in turn. This is known as a “reactor build”.

Often these two concepts are combined together, ie the poms that a multi-module pom points to use that calling pom as their parent. The following discussion generally assumes that this is the case, but will work even when the multi-module pom is not the parent pom of the modules it builds.

In-Reactor Dependency vs Repository Dependency

While reading this, it is also important to remember that Maven behaves differently when a pom depends (as parent or normal dependency) on:

  • an artifact that is part of the same reactor build, vs
  • an artifact that it looks up from the local maven repository (which perhaps downloads it from a remote repository)

As described later, due to the fact that a reactor loads all poms into memory first, expression evaluation (ie variable expansion) works differently.

Element project.version

Each pom has a <version> element which defines the value of runtime expression ${project.version}. This element may be a literal string, or may be an expression (eg ${some.variable}).

When pom.version is not specified, it defaults to ${project.parent.version} which is the version of the parent pom (the real one, not the literal value of element parent.version). A pom which has no parent must provide an explicit version element (ie cannot use the default) and cannot use expression ${project.parent.version}.

The expression may reference properties defined in the same pom-file, in an ancestor pom file, via Maven standard variables such as ${project.version}, or any of the other standard mechanisms (eg commandline-options or file maven.config).

Note however that any expression in a <version> element which uses a variable other than the following “magic names” will result in a warning message being emitted:

  • revision
  • changelist
  • sha1

These variables are normal variables, ie are not pre-defined via Maven, and are set in the usual way (via a properties.property element, parent-pom, commandline options, etc.); the only thing special about them is that they disable the warning message normally emitted. Presumably this is done in order to force standardisation on the variable-names used for defining versioning.

Dependency-declarations within a pom can use ${project.version} to reference other artifacts with the same version as itself.

Using an expression in the version tag has a specific advantage: the version of an artifact can be changed by setting an appropriate commandline option without changing the pom. This is commonly used to make non-snapshot releases of projects without changing code. In particular, it is common for a pom-file to declare its version like <version>${revision}${changelist}</version> where the pom’s properties define:

  • revision={current development version}
  • changelist="-SNAPSHOT"

Making a release build can then be done simply by setting changelist to an empty string on the Maven commandline, and moving developers to the next version (post-release) is just a matter of updating the value of property revision in the pom-file. For a project consisting of a single artifact (pom-file) this isn’t a big advance - but is important when dealing with large multi-module projects (see later).

Going one step further, the pom can specify revision={some fixed version} and the post-release process does not update the pom at all - ie as far as developer local environments are concerned, they work on the same “revision” permanently. Having a fixed version-number does not increase the risk for developers who switch between branches of the same source-code; even when the version-number is incremented in version-control after each release, developers who switch branches still need to recompile everything to be sure that the code they are working on is consistent (stale artifacts are not being included when launching a local instance of the product).

However there is an important problem to note about using expressions in poms: they aren’t expanded when the pom is deployed to the local repository, or when the pom is embedded into the META-INF/maven directory of a jarfile. When a pom contains <version>${revision}</version> and defines revision=7 (via properties.property or other mechanism) then when mvn install is run, in the local repository (typically ~/.m2/repository):

  • a directory {groupId}/{artifactId}/{expanded-version}/{expanded-version} is created
  • a file {artifactId}-{expanded-version}.pom is created in this directory
  • but the contents of the file contains a literal copy of the original pom, ie without expressions expanded

In most cases, unexpanded expressions are ok; the values are provided by Maven when the pom is used. However having unexpanded expressions in the <parent> clause of a pom means the pom-file is actually broken and unusable. When an artifact is deployed to a remote repository, the same thing happens - the “metadata” for the deployed artifact holds the expanded values, but the pom-file itself holds the original unexpanded expressions.

When running a “reactor build”, ie executing mvn using the “root pomfile” of a multi-module build, these poms are ignored as the reactor builds its own representation of the pom entities in memory directly from the source-code version in its directory-tree; the broken pom-files in the local repository are therefore not a problem.

However broken poms (those which use expressions in the parent element) are a problem in other circumstances; see later for more discussion of this and a solution.

Element project.parent.version

A pom which has a <parent> element must specify nested elements groupId, artifactId and version; these have no default value and must always be explicitly specified. There is also a nested element relativePath which defaults to ../pom.xml.

When evaluating a Maven module during a “reactor build” (ie multi-module build), the relativePath entry is evaluated before looking in the artifact repository. If there is a pom-file at the relative-path then its groupId/artifactId/version are compared to the values in the child; if they are literally identical then the parent is accepted else an error is reported. Note that the version field is treated as a string at this point (no expressions are expanded); this means that:

  • when the parent defines its version with a literal, the child must reference its parent’s version with that same literal
  • when the parent defines its version with an expression, the child must reference its parent’s version with exactly the same string - which looks like an expression in the child, but actually is treated as a literal string during the parent-resolution phase.

Once the parent has been located, its attributes define variables ${project.parent.groupId}, ${project.parent.artifactId} and ${project.parent.version} - but here the real (expanded) value of any expressions is used - in particular ${project.parent.version} is the expression result and not just its literal value.

As noted above, an artifact by default inherits the version of its parent (ie its version defaults to ${project.parent.version}).

Referencing an Artifact from a Repository

When running a reactor-build, Maven creates an in-memory structure for all poms. Any reference between two in-memory poms (parent or dependency) uses the in-memory data-structure, which has expressions expanded. It is therefore irrelevant whether the poms in the local repository have unexpanded expressions or not.

However when building a pom which references another pom that is not in-memory, the pom-file from the local or remote repository is used. If that pom includes a <parent> clause which contains unexpanded expressions, then the parent will not be found and the build will fail. This is the broken pom problem described earlier.

This problem does not occur when a pom from the repository uses expressions in dependency declarations or similar; those can be expanded after loading the pom from disk into memory. However expressions in the parent cannot, as resolving the expressions requires the parent to be found first.

It is perhaps a bit strange that Maven doesn’t expand expressions in the <parent> element of a pom when doing “mvn install”; maybe it will do so some day, or maybe there is a reason why it doesn’t do that.

To summarize: the fact that poms without a fixed parent/version value are broken doesn’t matter sometimes. In particular, if

  • you only build the project from the top-level directory (for both local dev and continuous-integration), and
  • the point of the project is to build a combined artifact such as a fat-jar or a war-file

then broken poms in the local maven repository (ie expressions in the parent clause) don’t matter; nobody is using them. When building a complete project in an IDE (ie developing in an IDE project created by importing a Maven multi-module project), the IDE will usually behave like a Maven reactor build, ie broken artifacts in the repo also don’t matter.

However if the multi-module project is building libraries that other projects consume, or developers want to be able to build at the command-line from subdirs of the project, then the broken poms do matter; poms loaded from the repository will fail to find their parent, get an invalid version, and then fail to resolve other dependencies using ${project.version}.

Fixing the Parent Problem

The flatten maven plugin provides a solution to the broken-parent-version problem - by simply ensuring that each artifact which is deployed to a local or remote maven repository has no parent element at all. It does this by inlining all data inherited from the artifact ancestors. This does make the pom.xml associated with an artifact larger, but avoids the “parent version as expression” problem - and also probably speeds up processing.

The flatten plugin also has a less intrusive mode where it just expands expressions useful in the parent version tag before uploading a pom to a repository; this is specified by setting flattenMode=resolveCiFriendliesOnly, as shown in the Maven CI Friendly page.

Note that at the current time, the documentation for the flatten-maven-plugin includes examples that don’t work. The multi-module example uses an arbitrary version-number for the “leaf poms” but that is not supported by modern Maven releases (see error message described above).

Note also that when the output artifact is a Java jarfile (packaging=jar) then a copy of the pom is also written into the jar’s META-INF/maven directory. The same “broken parent version” value problem occurs there unless the flatten plugin is used (and its flatten operation is bound to the “copy-resources” phase).

The flatten project’s central version documentation suggests using a fixed version “dev” to reference the parent, rather than an expression. However that does not allow a release-build to override the version using a commandline-variable, and it uses expressions in a way that triggers warnings during the Maven build; it therefore appears that this documentation is completely out-of-date, and the Maven CI Friendly documentation should be followed instead.