Help with a GraalVM package?

Help with a GraalVM package?

[Gary Johnson]

Hi Guix,

Has anyone had any luck in building a package definition for GraalVM?

https://www.graalvm.org/

I'm a Clojure programmer in my day job, and one of the areas I work in
is (open source) high performance wildland fire modeling. GraalVM could
really help me speed up my existing OpenJDK-based models as well as help
me build other GraalVM-dependent Guix packages like the Clojure linter,
clj-kondo.

Any help with this task (or a pointer to a channel with a working
package definition) would be much appreciated.

Happy hacking!
  Gary

-- 
GPG Key ID: 7BC158ED
Use `gpg --search-keys lambdatronic' to find me
Protect yourself from surveillance: https://emailselfdefense.fsf.org
=======================================================================
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments

Why is HTML email a security nightmare? See https://useplaintext.email/

Please avoid sending me MS-Office attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html

Re: Help with a GraalVM package?

[Ekaitz Zarraga]

Hi Gary,

> Hi Guix,
>
> Has anyone had any luck in building a package definition for GraalVM?
>
> Https://www.graalvm.org/

Probably not, but it can be you, or maybe me in the near future.

> I'm a Clojure programmer in my day job, and one of the areas I work in
> Is (open source) high performance wildland fire modeling. GraalVM could
> Really help me speed up my existing OpenJDK-based models as well as help
> Me build other GraalVM-dependent Guix packages like the Clojure linter,
> Clj-kondo.
>
> Any help with this task (or a pointer to a channel with a working
> Package definition) would be much appreciated.
>

The guix documentation has a good explanation about how to make packages, but
this one might be complex.

I searched a little bit and I didn't find any guide on how to build GraalVM
itself, if you can find me one I can try to guide you in making the package
or at least we could evaluate how hard is it.

Cheers,
Ekaitz

Re: Help with a GraalVM package?

[Julien Lepiller]

Le 27 octobre 2021 06:15:27 GMT-04:00, Ekaitz Zarraga <ekaitz@elenq.tech> a écrit :
>Hi Gary,
>
>> Hi Guix,
>>
>> Has anyone had any luck in building a package definition for GraalVM?
>>
>> Https://www.graalvm.org/
>
>Probably not, but it can be you, or maybe me in the near future.
>
>> I'm a Clojure programmer in my day job, and one of the areas I work in
>> Is (open source) high performance wildland fire modeling. GraalVM could
>> Really help me speed up my existing OpenJDK-based models as well as help
>> Me build other GraalVM-dependent Guix packages like the Clojure linter,
>> Clj-kondo.
>>
>> Any help with this task (or a pointer to a channel with a working
>> Package definition) would be much appreciated.
>>
>
>The guix documentation has a good explanation about how to make packages, but
>this one might be complex.
>
>I searched a little bit and I didn't find any guide on how to build GraalVM
>itself, if you can find me one I can try to guide you in making the package
>or at least we could evaluate how hard is it.
>
>Cheers,
>Ekaitz
>

Hi,

This is what I found: https://github.com/oracle/graal/blob/master/vm/README.md

That info is not easy to find. Why hide it in a subdirectory?… Also, I have no idea what this mx thing is.

Re: Help with a GraalVM package?

[Gary Johnson]

Ekaitz said:
>>I searched a little bit and I didn't find any guide on how to build
>>GraalVM itself, if you can find me one I can try to guide you in
>>making the package or at least we could evaluate how hard is it.

Julien said:
> This is what I found:
> https://github.com/oracle/graal/blob/master/vm/README.md
>
> That info is not easy to find. Why hide it in a subdirectory?… Also, I
> have no idea what this mx thing is.

Thanks, Ekaitz and Julien.

I did some further digging (why do they make this so hard?), and I
finally found the `mx` command:

  https://github.com/graalvm/mx

It looks like it's their custom build tool written in Python, so we
would need a package for `mx` first to bootstrap the process.

Once again for reference, here are the build instructions that Julien found:

  https://github.com/oracle/graal/blob/master/vm/README.md

In this file, the author says the following:

> In our CI, we build it using:
> 
> the latest JVMCI-enabled JDK8 (pre-built archives; build instructions). The JAVA_HOME environment variable must point to it.
> gcc: 4.9.2
> make: 3.83
> binutils: 2.23.2
> cmake: 3.15.2

I think the closest input packages in the latest guix are probably:

- gcc-toolchain@4.9.4 (which includes binutils@2.34 as a dependency)
- binutils@2.34 (do we need to include this as an input if we already have gcc-toolchain?)
- make@4.2.1
- cmake@3.21.1

The build docs reference a JVMCI-enabled JDK8. Of course the repository
pointed to by the "build instructions" link does not actually contain
any build instructions.

  https://github.com/graalvm/openjdk8-jvmci-builder

I dug a bit further into the issues list on that repository and found a
rather involved exchange with various programmers voicing their
frustrations with not being able to easily build a JVMCI-enabled JDK8
based on this repository's sources due to some proprietary components
that the GraalVM developers seemed to have added without any warning.
Eventually, it looked like at least one person on the issue thread
managed to get it to build, but the latest shared instructions were from
Dec 4th, 2020.

  https://github.com/graalvm/openjdk8-jvmci-builder/issues/11

However...I then backed up for a second to wonder what even is this
JVMCI thing and why do we need to add it to JDK8? A quick web search
turned up "JEP 243: Java-Level JVM Compiler Interface", which is a
feature that looks like it was shipped in JDK9.

  https://openjdk.java.net/jeps/243

Okay, so why not just use a newer JDK, right?

Heading back to that issues thread from earlier, I saw that one of the
GraalVM developers did eventually share a link to building a version of
JDK11 that should work as a base for building GraalVM.

  https://github.com/graalvm/labs-openjdk-11/blob/master/README.md

So this got me wondering whether they have done something similar for
newer versions of OpenJDK. Running a quick repository filtering search
under the graalvm Github organization led me here:

  https://github.com/orgs/graalvm/repositories?q=labs

It looks like they have worked on extending four OpenJDK versions to be
usable as the base for building GraalVM:

- labs-openjdk-11
- labs-openjdk-15
- labs-openjdk-16
- labs-openjdk-17

The Github commit graphs indicate that labs-openjdk-11 and
labs-openjdk-17 have seen the most activity in the past few months.
However, labs-openjdk-16 looked like it was getting some attention up
through July 2021.

Which one to try then? I remembered a warning about JDK17-based GraalVM
distributions being "experimental with several known limitations", so
maybe not that version.

  https://www.graalvm.org/docs/introduction/#available-distributions

How about the OpenJDK16-based version then? Perhaps that's a reasonable
path forward, and not too far off base since we already have an
openjdk@16.0.1 package in Guix. Perhaps we could modify it for this
purpose?

Continuing with this line of thinking, perhaps we could also try using
the newer versions of gcc-toolchain, binutils, make, and cmake as well?

...

Whew! So that was a lot of information in one email. I think the path
forward is first to make a package for the `mx` build tool, then to make
a package for `labs-openjdk-16`, and finally(?) to make a package for
`graalvm` that uses the previous two as inputs.

1. mx              https://github.com/graalvm/mx
2. labs-openjdk-16 https://github.com/graalvm/labs-openjdk-16
3. graalvm         https://github.com/oracle/graal/blob/master/vm/README.md

I have built a number of rather simple Guix package definitions for my
machine, but the scope of this one is a bit beyond my current
capabilities IMHO. Any and all help from you packaging wizards would be
much appreciated!

Cheers,
  Gary

-- 
GPG Key ID: 7BC158ED
Use `gpg --search-keys lambdatronic' to find me
Protect yourself from surveillance: https://emailselfdefense.fsf.org
=======================================================================
()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments

Why is HTML email a security nightmare? See https://useplaintext.email/

Please avoid sending me MS-Office attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html