What would a Maven build system be like?

What would a Maven build system be like?

[Ricardo Wurmus]

Hi Guix,

Roel and I discussed this off-list already and we thought it would be a
good idea to bring this discussion to the list.  (This email recycles
sentences both Roel and I wrote in our off-list discussion; mistakes are
all mine.)

In order to make packaging of Java software easier we should have a
maven-build-system.  By default Maven downloads binary artifacts from a
remote Maven repository.  The required binary artifacts are listed in
“pom.xml” files and identified by a combination of “groupId”,
“artifactId”, and “version”[1].

According to the documentation[2] it is possible to force Maven to build
stuff offline with:

    mvn -o package

They also mention “internal repositories” that can be accessed via 
“file://” URLs and that can be managed locally by manually downloading
artifacts:

    “Such an internal repository can be downloaded from using HTTP or
     the file system (using a file:// URL), and uploaded to using SCP,
     FTP, or a file copy.
     [...]
     To set up an internal repository just requires that you have a
     place to put it, and then start copying required artifacts there
     using the same layout as in a remote repository such as
     repo.maven.apache.org.”

We could probably generate such a fake repository as part of the
maven-build-system and populate it with only the jars that have been
specified as inputs.

A problem we now face is to recreate a local subset of the Maven
repository in a build phase using nothing but the package recipes and
static data encoded in our build system.  I don’t know if there is a
generic way to do this or if we need to store additional properties like
“maven-artifact-id” and “maven-group-id”.

For example, there’s a library called “hamcrest-core”.  Currently, I’m
building it with the upcoming “ant-build-system”, which installs a jar
archive to “$out/share/java/hamcrest-core-1.3.jar” — an ad-hoc location
because the location of jars really isn’t important as long as they are
on the classpath.  “hamcrest-core-1.3.jar” is the name that is generated
at compile time; I only had a hand in setting the prefix as
“$out/share/java/”.

Now, if I wanted to use a “maven-build-system” to build something that
depended on “hamcrest-core” it would not be enough for me to ensure that
the jar is on the classpath.  Instead I would have to generate a
directory structure mimicking the central Maven repository and make the
jar available at the expected location.

For “hamcrest-core” this happens to be

   (string-append repo-root "/"
                  (string-join (string-split maven-group-id #\.) "/") "/"
                  maven-artifact-id "/"
                  version "/"
                  maven-artifact-id "-" version ".jar")

Here’s a path on a public Maven repository for “hamcrest-core”:

    http://repo.maven.apache.org/maven2/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar

The first task of such a Maven build system would be to create a
temporary tree in which all of these jars are linked to the expected
location (instead of just hanging around at their own “$out/share/java/”
directory).

Since not every application uses Maven (or even the same version of
Maven), and I cannot yet be certain that the directory layout remains
the same across different versions of Maven, I think it would be best to
generate this dynamically rather than change the prefix for Java
libraries to “$out/maven-repo/” and then place the jars at the leaves of
this structure.

(Still, the question remains where best to put jars in their own output
directory.)

According to the “Apache Maven 3 cookbook” there are conventional build
steps for the Maven build system:

> mvn validate:         validates that all project information is
>                       available and is correct 
> mvn compile:          compiles the source code
> mvn test:             runs unit tests within a suitable framework
> mvn package:          packages the compiled code in its distribution
>                       format 
> mvn integration-test: processes the package in the integration-test
>                       environment 
> mvn verify:           runs checks to verify that the package is valid
> mvn install:          installs the package in the local repository

Since we must keep outputs separate, we cannot just install the jars
with “mvn install” into the fake ad-hoc Maven repository we generate
from store items at build time.  We would need a final step to move the
installed jars from the local Maven repository to their final location
in the store.

So, to summarise:

* do we need to change the default target location of all jar archives
  to accomodate the needs of Maven or can we generate a temporary local
  Maven repository as part of the maven-build-system?

* is there an alternative to storing “maven-artifact-id” and
  “maven-group-id” with Guix package expressions for Java libraries that
  can be required by a package using the maven-build-system?

* is this a sane way to move forward?

Comments are very welcome!

~~ Ricardo


[1]: https://maven.apache.org/general.html#importing-jars
[2]: https://maven.apache.org/guides/introduction/introduction-to-repositories.html

Re: What would a Maven build system be like?

[BCG]

On 02/29/2016 11:14 AM, Ricardo Wurmus wrote:
>
> Since not every application uses Maven (or even the same version of
> Maven), and I cannot yet be certain that the directory layout remains
> the same across different versions of Maven, I think it would be best to
> generate this dynamically rather than change the prefix for Java
> libraries to “$out/maven-repo/” and then place the jars at the leaves of
> this structure.
>
>
FWIW - I've been using Maven for 10 years and I can't recall the 
repository layout ever changing since Maven 2 was released, although I 
could be wrong about that.

One thing you may want to look at is the "maven-ant-tasks" Ant plugin here:

https://maven.apache.org/ant-tasks/

This allows you to take artifacts built outside of Maven and install 
them into a Maven repository layout.  The Maven repository layout is the 
defacto standard these days for all major Java build systems these days, 
so create a folder structure with that layout for Guix might be a great 
choice.

Other Java dependency managers support Maven repositories as well, 
including Apache Ivy (http://ant.apache.org/ivy/) and Gradle 
(http://gradle.org/why/robust-dependency-management/).

As a Java developer, my gut tells me that it is worth exploring putting 
Guix's Java packages get put into a Maven repository layout by 
default... Maven is notoriously difficult to bend to your will otherwise :)

-- Ben

Re: What would a Maven build system be like?

[Ludovic Courtès]

Hello!

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> Roel and I discussed this off-list already and we thought it would be a
> good idea to bring this discussion to the list.  (This email recycles
> sentences both Roel and I wrote in our off-list discussion; mistakes are
> all mine.)

I’m afraid I don’t have anything useful to contribute to the discussion,
but I’m interested in seeing progress on this front.  What’s the status?

I think that if Roel and you find a solution that sounds good to you,
then we should take this as a starting point.  People can always
complain anytime later.  :-)

Ludo’.