From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ricardo Wurmus Subject: What would a Maven build system be like? Date: Mon, 29 Feb 2016 17:14:14 +0100 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45990) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aaQSq-0004d2-SF for guix-devel@gnu.org; Mon, 29 Feb 2016 11:14:30 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aaQSn-0001WM-IG for guix-devel@gnu.org; Mon, 29 Feb 2016 11:14:28 -0500 Received: from sinope.bbbm.mdc-berlin.de ([141.80.25.23]:36088) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aaQSn-0001Vu-5E for guix-devel@gnu.org; Mon, 29 Feb 2016 11:14:25 -0500 Received: from localhost (localhost [127.0.0.1]) by sinope.bbbm.mdc-berlin.de (Postfix) with ESMTP id D7E2D2808CC for ; Mon, 29 Feb 2016 17:14:21 +0100 (CET) Received: from sinope.bbbm.mdc-berlin.de ([127.0.0.1]) by localhost (sinope.bbbm.mdc-berlin.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id M5v1HCweOKCF for ; Mon, 29 Feb 2016 17:14:15 +0100 (CET) Received: from HTCAONE.mdc-berlin.net (puck.citx.mdc-berlin.de [141.80.36.101]) by sinope.bbbm.mdc-berlin.de (Postfix) with ESMTP for ; Mon, 29 Feb 2016 17:14:14 +0100 (CET) List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: guix-devel Hi Guix, Roel and I discussed this off-list already and we thought it would be a good idea to bring this discussion to the list. (This email recycles sentences both Roel and I wrote in our off-list discussion; mistakes are all mine.) In order to make packaging of Java software easier we should have a maven-build-system. By default Maven downloads binary artifacts from a remote Maven repository. The required binary artifacts are listed in =E2=80=9Cpom.xml=E2=80=9D files and identified by a combination of =E2=80= =9CgroupId=E2=80=9D, =E2=80=9CartifactId=E2=80=9D, and =E2=80=9Cversion=E2=80=9D[1]. According to the documentation[2] it is possible to force Maven to build stuff offline with: mvn -o package They also mention =E2=80=9Cinternal repositories=E2=80=9D that can be acc= essed via=20 =E2=80=9Cfile://=E2=80=9D URLs and that can be managed locally by manuall= y downloading artifacts: =E2=80=9CSuch an internal repository can be downloaded from using HTT= P or the file system (using a file:// URL), and uploaded to using SCP, FTP, or a file copy. [...] To set up an internal repository just requires that you have a place to put it, and then start copying required artifacts there using the same layout as in a remote repository such as repo.maven.apache.org.=E2=80=9D We could probably generate such a fake repository as part of the maven-build-system and populate it with only the jars that have been specified as inputs. A problem we now face is to recreate a local subset of the Maven repository in a build phase using nothing but the package recipes and static data encoded in our build system. I don=E2=80=99t know if there i= s a generic way to do this or if we need to store additional properties like =E2=80=9Cmaven-artifact-id=E2=80=9D and =E2=80=9Cmaven-group-id=E2=80=9D. For example, there=E2=80=99s a library called =E2=80=9Chamcrest-core=E2=80= =9D. Currently, I=E2=80=99m building it with the upcoming =E2=80=9Cant-build-system=E2=80=9D, which i= nstalls a jar archive to =E2=80=9C$out/share/java/hamcrest-core-1.3.jar=E2=80=9D =E2=80= =94 an ad-hoc location because the location of jars really isn=E2=80=99t important as long as th= ey are on the classpath. =E2=80=9Chamcrest-core-1.3.jar=E2=80=9D is the name th= at is generated at compile time; I only had a hand in setting the prefix as =E2=80=9C$out/share/java/=E2=80=9D. Now, if I wanted to use a =E2=80=9Cmaven-build-system=E2=80=9D to build s= omething that depended on =E2=80=9Chamcrest-core=E2=80=9D it would not be enough for me= to ensure that the jar is on the classpath. Instead I would have to generate a directory structure mimicking the central Maven repository and make the jar available at the expected location. For =E2=80=9Chamcrest-core=E2=80=9D this happens to be (string-append repo-root "/" (string-join (string-split maven-group-id #\.) "/") "/" maven-artifact-id "/" version "/" maven-artifact-id "-" version ".jar") Here=E2=80=99s a path on a public Maven repository for =E2=80=9Chamcrest-= core=E2=80=9D: http://repo.maven.apache.org/maven2/org/hamcrest/hamcrest-core/1.3/ha= mcrest-core-1.3.jar The first task of such a Maven build system would be to create a temporary tree in which all of these jars are linked to the expected location (instead of just hanging around at their own =E2=80=9C$out/share= /java/=E2=80=9D directory). Since not every application uses Maven (or even the same version of Maven), and I cannot yet be certain that the directory layout remains the same across different versions of Maven, I think it would be best to generate this dynamically rather than change the prefix for Java libraries to =E2=80=9C$out/maven-repo/=E2=80=9D and then place the jars a= t the leaves of this structure. (Still, the question remains where best to put jars in their own output directory.) According to the =E2=80=9CApache Maven 3 cookbook=E2=80=9D there are conv= entional build steps for the Maven build system: > mvn validate: validates that all project information is > available and is correct=20 > mvn compile: compiles the source code > mvn test: runs unit tests within a suitable framework > mvn package: packages the compiled code in its distribution > format=20 > mvn integration-test: processes the package in the integration-test > environment=20 > mvn verify: runs checks to verify that the package is valid > mvn install: installs the package in the local repository Since we must keep outputs separate, we cannot just install the jars with =E2=80=9Cmvn install=E2=80=9D into the fake ad-hoc Maven repository = we generate from store items at build time. We would need a final step to move the installed jars from the local Maven repository to their final location in the store. So, to summarise: * do we need to change the default target location of all jar archives to accomodate the needs of Maven or can we generate a temporary local Maven repository as part of the maven-build-system? * is there an alternative to storing =E2=80=9Cmaven-artifact-id=E2=80=9D = and =E2=80=9Cmaven-group-id=E2=80=9D with Guix package expressions for Java= libraries that can be required by a package using the maven-build-system? * is this a sane way to move forward? Comments are very welcome! ~~ Ricardo [1]: https://maven.apache.org/general.html#importing-jars [2]: https://maven.apache.org/guides/introduction/introduction-to-reposit= ories.html