Using julia in guix

Using julia in guix

[Théo Maxime Tyburn]

Hello guix people and julia users!

I started using julia in one of my university projects and I am not sure
how this should be done the guix way. Has someone experience with this ?

For know I just used the julia package manager and the packages are in
~/.julia. I was not able to use the guix packages for julia
however. Should something be done manually to tell julia where to find
its packages ?

I encountered two problems. The first with the package "GR" the
second with the package "GLFW". I encountered these bugs while trying
the packages "Plots" and the "GLMakie" backend of "Makie"


The GR bug:
> julia> using GR
> julia> histogram(randn(10000))
> env: ‘/home/teddd/.julia/artifacts/b5fea2b0d5c84336f26b139587a84ee7e8ca2e12/bin/gksqt’: No such file or directory
> connect: Connection refused
> GKS: can't connect to GKS socket application
it then exits because of a segmentation fault.

I tried running the above inside this shell
> guix shell gr-framework
but the result is the same.


The GLFW bug:
> julia> using GLFW
> julia> GLFW.CreateWindow(300,300, "Test")
> ERROR: GLFWError (API_UNAVAILABLE): GLX: No GLXFBConfigs returned

I also tried running this into
> guix shell mesa glfw mesa-utils libglvnd
with exact same result.

Also I was able to compile and run a minimal GLFW C code so I think my
glfw and mesa installs are fine.


I am not sure these two errors are related but I have the
impression they both come from the fact that guix doesn’t allow the
access to some libraries or binaries the julia packages expect to
find. Please correct me if I am wrong.

Has someone an idea how I could investigate what is going wrong ?


Kind Regards,

Théo

Re: Using julia in guix

[zimoun]

Hi,

On Wed, 3 Nov 2021 at 00:30, Théo Maxime Tyburn <theo.tyburn@gmail.com> wrote:

> I started using julia in one of my university projects and I am not sure
> how this should be done the guix way. Has someone experience with this ?

Cool!


> For know I just used the julia package manager and the packages are in
> ~/.julia. I was not able to use the guix packages for julia
> however. Should something be done manually to tell julia where to find
> its packages ?

For now, it is either packages from Pkg.jl or either packages julia-*
from Guix.  A patch [1] is still pending for being able to mix both.
Aside that packages from Pkg.jl do not always work out-of-the-box,
sadly.

1: <http://issues.guix.gnu.org/51319>


> I encountered two problems. The first with the package "GR" the
> second with the package "GLFW". I encountered these bugs while trying
> the packages "Plots" and the "GLMakie" backend of "Makie"

IIUC, you installed all the packages using Pkg.jl, right?

Because then, I get:

--8<---------------cut here---------------start------------->8---
ERROR: could not download
https://pkg.julialang.org/registry/23338594-aafe-5451-b93e-139f81909106/302047ed63e49eace1abd20f300ac6e22768690b
--8<---------------cut here---------------end--------------->8---

and I do not know if Pkg.jl works correctly with the current julia
from Guix.  For instance, "guix shell julia -CN -- julia" then 'using
Pkg; Pkg.add("Plots")' reports failures.  One is reference to build
directory "/tmp/guix-build-julia-1.6.2.drv-0", see [2].

2: <http://issues.guix.gnu.org/issue/48238>



> The GR bug:
> > julia> using GR
> > julia> histogram(randn(10000))
> > env: ‘/home/teddd/.julia/artifacts/b5fea2b0d5c84336f26b139587a84ee7e8ca2e12/bin/gksqt’: No such file or directory
> > connect: Connection refused
> > GKS: can't connect to GKS socket application[
> it then exits because of a segmentation fault.

Well, I do not know where does the issue come from.  I suspect
something related to [2] above.

For now, I do not have a solution and I do not have time to work on
it.  Sorry.  I would be happy if someone fixes the issue; probably
patch#51319 helps here. :-)

Cheers,
simon

Re: Using julia in guix

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2751 bytes --]

Apologies for top-posting. If you test the patch and find that it works
then I'll be happy to apply it.

On Wed, Nov 03, 2021 at 11:06:25AM +0100, zimoun wrote:
> Hi,
> 
> On Wed, 3 Nov 2021 at 00:30, Théo Maxime Tyburn <theo.tyburn@gmail.com> wrote:
> 
> > I started using julia in one of my university projects and I am not sure
> > how this should be done the guix way. Has someone experience with this ?
> 
> Cool!
> 
> 
> > For know I just used the julia package manager and the packages are in
> > ~/.julia. I was not able to use the guix packages for julia
> > however. Should something be done manually to tell julia where to find
> > its packages ?
> 
> For now, it is either packages from Pkg.jl or either packages julia-*
> from Guix.  A patch [1] is still pending for being able to mix both.
> Aside that packages from Pkg.jl do not always work out-of-the-box,
> sadly.
> 
> 1: <http://issues.guix.gnu.org/51319>
> 
> 
> > I encountered two problems. The first with the package "GR" the
> > second with the package "GLFW". I encountered these bugs while trying
> > the packages "Plots" and the "GLMakie" backend of "Makie"
> 
> IIUC, you installed all the packages using Pkg.jl, right?
> 
> Because then, I get:
> 
> --8<---------------cut here---------------start------------->8---
> ERROR: could not download
> https://pkg.julialang.org/registry/23338594-aafe-5451-b93e-139f81909106/302047ed63e49eace1abd20f300ac6e22768690b
> --8<---------------cut here---------------end--------------->8---
> 
> and I do not know if Pkg.jl works correctly with the current julia
> from Guix.  For instance, "guix shell julia -CN -- julia" then 'using
> Pkg; Pkg.add("Plots")' reports failures.  One is reference to build
> directory "/tmp/guix-build-julia-1.6.2.drv-0", see [2].
> 
> 2: <http://issues.guix.gnu.org/issue/48238>
> 
> 
> 
> > The GR bug:
> > > julia> using GR
> > > julia> histogram(randn(10000))
> > > env: ‘/home/teddd/.julia/artifacts/b5fea2b0d5c84336f26b139587a84ee7e8ca2e12/bin/gksqt’: No such file or directory
> > > connect: Connection refused
> > > GKS: can't connect to GKS socket application[
> > it then exits because of a segmentation fault.
> 
> Well, I do not know where does the issue come from.  I suspect
> something related to [2] above.
> 
> For now, I do not have a solution and I do not have time to work on
> it.  Sorry.  I would be happy if someone fixes the issue; probably
> patch#51319 helps here. :-)
> 
> Cheers,
> simon
> 

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Using julia in guix

[Théo Maxime Tyburn]

Hi,

Thanks for the quick reply!

> For now, it is either packages from Pkg.jl or either packages julia-*
> from Guix.  A patch [1] is still pending for being able to mix both.
> Aside that packages from Pkg.jl do not always work out-of-the-box,
> sadly.
>
> 1: <http://issues.guix.gnu.org/51319>

Ok I see so either I use guix julia-* packages and then I need julia to
ignore packages in ~/.julia and avoid using Pkg, or I use Pkg to
install packages but it will break in some cases and also ignore already
install julia packages from guix.

What about using JLLs ? I have no precise understanding of how a jll work
and how to use it but it seems they contain binaries and there are
guix packages for them. For example julia-glfw-jll. Could I use GLFW in
julia with this guix package installed ?

>> I encountered two problems. The first with the package "GR" the
>> second with the package "GLFW". I encountered these bugs while trying
>> the packages "Plots" and the "GLMakie" backend of "Makie"
>
> IIUC, you installed all the packages using Pkg.jl, right?

Yes until my first message I only installed stuff with Pkg. Now I tried
the guix packages and they work fine. Especially the GR package works so
I can at least plot things now, thanks ! (Makie with Cairo backend is
also working from Pkg install though)

> and I do not know if Pkg.jl works correctly with the current julia
> from Guix.  For instance, "guix shell julia -CN -- julia" then 'using
> Pkg; Pkg.add("Plots")' reports failures.  One is reference to build
> directory "/tmp/guix-build-julia-1.6.2.drv-0", see [2].
>
> 2: <http://issues.guix.gnu.org/issue/48238>

This seems nasty. For me however everything I tried to install and
precompile with Pkg worked. So "pkg>add Plots" and "using Plots" works
fine. Problems arise when I call a function, say "plot(1:3,1:3)".

> For now, I do not have a solution and I do not have time to work on
> it.  Sorry.  I would be happy if someone fixes the issue; probably
> patch#51319 helps here. :-)

I’ll try that out, it seems promising. Combining packages from Pkg and
guix while prioritizing guix packages and avoid reinstalling them seems
like the right way to get the problematic packages under control for now
(by that I mean creating a guix package for them).

I never tested a patch though. How do I do this ? Is it on a branch of
the git repo ?

Kind regards,

Théo

Re: Using julia in guix

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 3333 bytes --]

On Wed, Nov 03, 2021 at 10:56:51PM +0100, Théo Maxime Tyburn wrote:
> Hi,
> 
> Thanks for the quick reply!
> 
> > For now, it is either packages from Pkg.jl or either packages julia-*
> > from Guix.  A patch [1] is still pending for being able to mix both.
> > Aside that packages from Pkg.jl do not always work out-of-the-box,
> > sadly.
> >
> > 1: <http://issues.guix.gnu.org/51319>
> 
> Ok I see so either I use guix julia-* packages and then I need julia to
> ignore packages in ~/.julia and avoid using Pkg, or I use Pkg to
> install packages but it will break in some cases and also ignore already
> install julia packages from guix.
> 
> What about using JLLs ? I have no precise understanding of how a jll work
> and how to use it but it seems they contain binaries and there are
> guix packages for them. For example julia-glfw-jll. Could I use GLFW in
> julia with this guix package installed ?
> 
> >> I encountered two problems. The first with the package "GR" the
> >> second with the package "GLFW". I encountered these bugs while trying
> >> the packages "Plots" and the "GLMakie" backend of "Makie"
> >
> > IIUC, you installed all the packages using Pkg.jl, right?
> 
> Yes until my first message I only installed stuff with Pkg. Now I tried
> the guix packages and they work fine. Especially the GR package works so
> I can at least plot things now, thanks ! (Makie with Cairo backend is
> also working from Pkg install though)
> 
> > and I do not know if Pkg.jl works correctly with the current julia
> > from Guix.  For instance, "guix shell julia -CN -- julia" then 'using
> > Pkg; Pkg.add("Plots")' reports failures.  One is reference to build
> > directory "/tmp/guix-build-julia-1.6.2.drv-0", see [2].
> >
> > 2: <http://issues.guix.gnu.org/issue/48238>
> 
> This seems nasty. For me however everything I tried to install and
> precompile with Pkg worked. So "pkg>add Plots" and "using Plots" works
> fine. Problems arise when I call a function, say "plot(1:3,1:3)".
> 
> > For now, I do not have a solution and I do not have time to work on
> > it.  Sorry.  I would be happy if someone fixes the issue; probably
> > patch#51319 helps here. :-)
> 
> I’ll try that out, it seems promising. Combining packages from Pkg and
> guix while prioritizing guix packages and avoid reinstalling them seems
> like the right way to get the problematic packages under control for now
> (by that I mean creating a guix package for them).
> 
> I never tested a patch though. How do I do this ? Is it on a branch of
> the git repo ?

The way I do it is I read my emails with mutt and I pipe the patch file
(either the e-mail or a part of it, depending on how it came through) to
'cd workspace/guix; git am' and then test it out from there.

There is also CBaines's patch tracker. If you add
https://git.guix-patches.cbaines.net/git/guix-patches as another remote
(git remote add guix-patches https://git.guix-patches.cbaines.net/git/guix-patches)
then you can cherry-pick c6e882a52b5e736d0ac11872bc2dc6e26100115a and
test out the patch that way.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Using julia in guix

[Théo Maxime Tyburn]

Efraim Flashner <efraim@flashner.co.il> writes:

> The way I do it is I read my emails with mutt and I pipe the patch file
> (either the e-mail or a part of it, depending on how it came through) to
> 'cd workspace/guix; git am' and then test it out from there.
>
> There is also CBaines's patch tracker. If you add
> https://git.guix-patches.cbaines.net/git/guix-patches as another remote
> (git remote add guix-patches https://git.guix-patches.cbaines.net/git/guix-patches)
> then you can cherry-pick c6e882a52b5e736d0ac11872bc2dc6e26100115a and
> test out the patch that way.

Thanks, CBaines’ patch tracker made it very easy to get the code.

So I run "./pre-inst-env guix shell julia". But apparently julia wasn’t
rebuilt as I did that. So when I then run "./pre-inst-env julia" I am
probabilly using the unpatched julia version, right ?
If I install say "julia-docstringextensions" with guix and then in julia
add it with Pkg it still gets installed the usual way.
How can I use the julia package as defined in the patch ?

Kind Regards,

Théo

Re: Using julia in guix

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1748 bytes --]

On Fri, Nov 05, 2021 at 10:52:54PM +0100, Théo Maxime Tyburn wrote:
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> 
> > The way I do it is I read my emails with mutt and I pipe the patch file
> > (either the e-mail or a part of it, depending on how it came through) to
> > 'cd workspace/guix; git am' and then test it out from there.
> >
> > There is also CBaines's patch tracker. If you add
> > https://git.guix-patches.cbaines.net/git/guix-patches as another remote
> > (git remote add guix-patches https://git.guix-patches.cbaines.net/git/guix-patches)
> > then you can cherry-pick c6e882a52b5e736d0ac11872bc2dc6e26100115a and
> > test out the patch that way.
> 
> Thanks, CBaines’ patch tracker made it very easy to get the code.
> 
> So I run "./pre-inst-env guix shell julia". But apparently julia wasn’t
> rebuilt as I did that. So when I then run "./pre-inst-env julia" I am
> probabilly using the unpatched julia version, right ?
> If I install say "julia-docstringextensions" with guix and then in julia
> add it with Pkg it still gets installed the usual way.
> How can I use the julia package as defined in the patch ?

I looked at the patch again. It doesn't actually affect the julia
package itself but packages built using the julia-build-system. Right
now I'm trying to test it with './pre-inst-env guix shell julia
julia-pyplot julia-staticarrays julia-statsbase' and then using the
julia from that shell to try importing other packages to finish loading
a different project.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Using julia in guix

[Théo Maxime Tyburn]

Efraim Flashner <efraim@flashner.co.il> writes:
>> So I run "./pre-inst-env guix shell julia". But apparently julia wasn’t
>> rebuilt as I did that. So when I then run "./pre-inst-env julia" I am
>> probabilly using the unpatched julia version, right ?
>> If I install say "julia-docstringextensions" with guix and then in julia
>> add it with Pkg it still gets installed the usual way.
>> How can I use the julia package as defined in the patch ?
>
> I looked at the patch again. It doesn't actually affect the julia
> package itself but packages built using the julia-build-system. Right
> now I'm trying to test it with './pre-inst-env guix shell julia
> julia-pyplot julia-staticarrays julia-statsbase' and then using the
> julia from that shell to try importing other packages to finish loading
> a different project.

Ok it seems I am doing it right then. When I run "./pre-inst-env guix
shell julia julia-gr" then "julia" and finally "pkg> add Plots" then Pkg
just reinstalls GR as a dependency of Plots. If I run "pkg> add GR" the
GR package also gets reinstalled. With the patch it should use GR from
the guix package instead, right ? Maybe this has to do with newer
version of GR being available ?

Re: Using julia in guix

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1794 bytes --]

On Sun, Nov 07, 2021 at 05:58:28PM +0100, Théo Maxime Tyburn wrote:
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> >> So I run "./pre-inst-env guix shell julia". But apparently julia wasn’t
> >> rebuilt as I did that. So when I then run "./pre-inst-env julia" I am
> >> probabilly using the unpatched julia version, right ?
> >> If I install say "julia-docstringextensions" with guix and then in julia
> >> add it with Pkg it still gets installed the usual way.
> >> How can I use the julia package as defined in the patch ?
> >
> > I looked at the patch again. It doesn't actually affect the julia
> > package itself but packages built using the julia-build-system. Right
> > now I'm trying to test it with './pre-inst-env guix shell julia
> > julia-pyplot julia-staticarrays julia-statsbase' and then using the
> > julia from that shell to try importing other packages to finish loading
> > a different project.
> 
> Ok it seems I am doing it right then. When I run "./pre-inst-env guix
> shell julia julia-gr" then "julia" and finally "pkg> add Plots" then Pkg
> just reinstalls GR as a dependency of Plots. If I run "pkg> add GR" the
> GR package also gets reinstalled. With the patch it should use GR from
> the guix package instead, right ? Maybe this has to do with newer
> version of GR being available ?

I suppose it could be. I was having trouble building newer versions of
gr so Guix ended up with the version it has.

For some reason I'm having trouble downloading the registry so I can't
test the patch as well as I'd like to.

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Using julia in guix

[zimoun]

Hi,

On Sun, 07 Nov 2021 at 17:58, Théo Maxime Tyburn <theo.tyburn@gmail.com> wrote:

> Ok it seems I am doing it right then. When I run "./pre-inst-env guix
> shell julia julia-gr" then "julia" and finally "pkg> add Plots" then Pkg
> just reinstalls GR as a dependency of Plots. If I run "pkg> add GR" the
> GR package also gets reinstalled. With the patch it should use GR from
> the guix package instead, right ? Maybe this has to do with newer
> version of GR being available ?

Thanks.  Indeed, the patch should not work this way.  Well, my
computational resources are currently busy so I cannot try myself.  I
will give a look later this week.


Cheers,
simon