Julia and Flux

Julia and Flux

[Cayetano Santos via Guix-Science]

Hello,

  I noticed that the following fails

    guix shell -C --network --emulate-fhs --link-profile julia nss-certs -- julia -e 'using Pkg; Pkg.add("Flux"); using Flux'

  Any idea why ?

Thanks,

Cayetano Santos

Re: Julia and Flux

[Hugo Buddelmeijer]

Seems to work fine for me. At least I get no errors; but I'm not a
julia user, so I don't know what to expect.

You would probably get a more useful reply if you would include what
you expect to happen, and what actually happens, when you type in that
command. This is the (eventual) output that I get:

Precompiling project...
  97 dependencies successfully precompiled in 54 seconds. 4 already precompiled.





On Tue, 26 Mar 2024 at 11:08, Cayetano Santos via Guix-Science
<guix-science@gnu.org> wrote:
>
>
> Hello,
>
>   I noticed that the following fails
>
>     guix shell -C --network --emulate-fhs --link-profile julia nss-certs -- julia -e 'using Pkg; Pkg.add("Flux"); using Flux'
>
>   Any idea why ?
>
> Thanks,
>
> Cayetano Santos
>

Re: Julia and Flux

[Cayetano Santos via Guix-Science]

(reply to whole list)

>ven. 29 mars 2024 at 15:17, Hugo Buddelmeijer <hugo@buddelmeijer.nl> wrote:

> Seems to work fine for me. At least I get no errors; but I'm not a
> julia user, so I don't know what to expect.
>
> You would probably get a more useful reply if you would include what
> you expect to happen, and what actually happens, when you type in that
> command. This is the (eventual) output that I get:

This is what I get

    https://paste.sr.ht/blob/65c1328a9d63d56ca39180163bd9088750d75e51

> Precompiling project...
>   97 dependencies successfully precompiled in 54 seconds. 4 already
>   precompiled

Hum, strange, ...  the point on using guix is not to get same results,
regardless of the machine ?

My guix --version is

  guix (GNU Guix) 04be2cd15a712224f6c88085296c3eecb7fdacb0

more exactly,

    guix describe --format=channels

gives

(list (channel
        (name 'guix-science)
        (url "https://github.com/guix-science/guix-science.git")
        (branch "master")
        (commit
          "7c6ca7e1c19c92bfe5e6add688662e08b07db2bf")
        (introduction
          (make-channel-introduction
            "b1fe5aaff3ab48e798a4cce02f0212bc91f423dc"
            (openpgp-fingerprint
              "CA4F 8CF4 37D7 478F DA05  5FD4 4213 7701 1A37 8446"))))
      (channel
        (name 'guix-hpc)
        (url "https://gitlab.inria.fr/guix-hpc/guix-hpc.git")
        (branch "master")
        (commit
          "3fb254c01590872068bf22ca62591270437761ba"))
      (channel
        (name 'nonguix)
        (url "https://gitlab.com/nonguix/nonguix")
        (branch "master")
        (commit
          "a4356defe7d4ee9b0ac6789d67f94d00714a9409")
        (introduction
          (make-channel-introduction
            "897c1a470da759236cc11798f4e0a5f7d4d59fbc"
            (openpgp-fingerprint
              "2A39 3FFF 68F4 EF7A 3D29  12AF 6F51 20A0 22FB B2D5"))))
      (channel
        (name 'guix)
        (url "https://git.savannah.gnu.org/git/guix.git")
        (branch "master")
        (commit
          "04be2cd15a712224f6c88085296c3eecb7fdacb0")
        (introduction
          (make-channel-introduction
            "9edb3f66fd807b096b48283debdcddccfea34bad"
            (openpgp-fingerprint
              "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"))))).

Re: Julia and Flux

[Cayetano Santos via Guix-Science]

(reply to full mail list)

>dim. 31 mars 2024 at 15:48, Hugo Buddelmeijer <hugo@buddelmeijer.nl> wrote:

> Hi Cayetano,
>
> Disclaimer, I'm still learning Guix as well, and I'm not (yet) a Julia
> user. So here are my 2 cents, which might not be that useful.
>
> Thanks for the channel description. It is indeed possible to reproduce
> your error with those channels. It seems to be the Julia 1.9.3 in the
> guix-hpc channel that is the cause of the differences. Removing that
> channel will give you Julia 1.8 from the main guix channel, where Flux
> does seem to import.

Sure, there is no issue with main guix channel. The issue arises only with
guix-hpc channel.

> The main error message seems to be this:
>
>> Illegal inttoptr
>>  %magicptr = ptrtoint {} addrspace(10)* %242 to i64, !dbg !26
>
> And there are various issues (e.g. [1]) on github saying that this
> problem occurs because an incorrect version of LLVM is used, and that
> a specific "patched" version of LLVM is needed. This is about how far
> my investigation went.
>
> However, I've learned that using both Guix and a language's own
> package manager, can lead to problems. It might be better to create
> Guix packages for Flux and its dependencies. At least that is my plan,
> for my (Python) projects.

I see. Julia 1.8 uses llvm-julia, whereas julia 1.9 uses llvm-14-julia
in guix-hpc.

A. Out of a guix shell, when I compile something, I’m using my system
llvm, whatever it is, and this probably produces  some mess.

If you’re right, to fix the problem, it would be necessary to create a different
guix package to all my julia dependencies (and also to its
dependencies), using julia 1.9 instead of 1.8, as they currently do in
main channel.

At this point, I think it might be easier for all if julia 1.9 (and 1.10 at some
point), was merged into main guix repo, given that this problem will
arise with other packages too.

That being said, if you’re right, one will need to install a julia
package not in the guix repositories soon or later.

B.  Coming back to my original issue, I’m doing

    guix shell -C --network --emulate-fhs --link-profile julia nss-certs -- julia -e 'using Pkg; Pkg.add("Flux"); using Flux'

meaning that I’m using only guix provided packages and environment. This
implies julia itself, along with llvm-14-julia. The problem remains the
same, regardless of the system I’m using outside of the container environment.

> There is already a julia-zygote package in Guix itself. So perhaps you
> could already achieve what you need by using the julia from the main
> Guix channel, instead of the one from guix-hpc, because then you can
> use the packaged zygote.
>
> [1] "Zygote and Flux Illegal inttoptr"
> https://github.com/JuliaLang/julia/issues/50153
>
> Hope this was useful Cayetano; I doubt I can contribute more.
>
> Hugo

I’d prefer to use recent version  of julia, so main channel is not an
option for me.

Thanks anyway,

C.