bug#61121: Cannot import IJulia in Julia

bug#61121: Cannot import IJulia in Julia

[Theodore Ehrenborg]

[-- Attachment #1: Type: text/plain, Size: 1044 bytes --]

Hi Guix,

I would like to run a Jupyter notebook using Julia, so I need to install
the IJulia backend:

guix install julia
julia # Enter julia REPL
] # To go into the julia pkg REPL
add IJulia
# Now type backspace to go to julia REPL
using IJulia

This produces the error:

[ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
ERROR: LoadError: InitError: SystemError: opening file
"/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
No such file or directory

The full error message is here: https://pastebin.com/qC8yyHXT

I saw a very similar bug on Gentoo:

Without this file (which can be a symbolic link to
`/etc/ssl/certs/ca-certificates.crt`) many Julia 1.8.3 packages, e.g.
`HTTP`, do not work.
This is what happens:

julia> import HTTP
[ Info: Precompiling HTTP [cd3eb016-35fb-5094-929b-558a96fad6f3]
ERROR: LoadError: InitError: SystemError: opening file
"/usr/share/julia/cert.pem":

(https://bugs.gentoo.org/888978)

Any help would be greatly appreciated.

Best regards,
Theodore Ehrenborg

[-- Attachment #2: Type: text/html, Size: 1518 bytes --]

bug#61121: Cannot import IJulia in Julia

[Simon Tournier]

Hi,

I confirm this bug.

On sam., 28 janv. 2023 at 13:45, Theodore Ehrenborg <theodore.ehrenborg@gmail.com> wrote:

> [ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
> ERROR: LoadError: InitError: SystemError: opening file
> "/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
> No such file or directory

[...]

> I saw a very similar bug on Gentoo:

[...]

> (https://bugs.gentoo.org/888978)

Well, that’s because Julia upstream does not take care about packagers;
as explicitly mentioned in this comment:

    https://github.com/JuliaLang/MbedTLS.jl/pull/261#issuecomment-1346886879

The Guixer Cayetano Santos fixed upstream the issue for one package.
But as you are noticing it is not done for all.

I do not know what is the best solution because the issue is coming from
Julia itself.

Efraim, any suggestion?

Cheers,
simon

bug#61121: Cannot import IJulia in Julia

[Theodore Ehrenborg]

[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]

Hi,

Thanks for getting back with me.

Gentoo appears to have fixed this bug by linking julia/cert.pem to the
system's ca-certificates.crt.
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168

Is there a way I could rebuild my own slightly modified Julia with a link
like that?

I understand that there's probably a good reason that Guix's Julia doesn't
by default have cert.pem, but I would be pleased with a hacky custom
solution if it made Jupyter notebooks work.

Thanks,
Theodore

Den mån 30 jan. 2023 kl 12:47 skrev Simon Tournier <zimon.toutoune@gmail.com
>:

> Hi,
>
> I confirm this bug.
>
> On sam., 28 janv. 2023 at 13:45, Theodore Ehrenborg <
> theodore.ehrenborg@gmail.com> wrote:
>
> > [ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
> > ERROR: LoadError: InitError: SystemError: opening file
> >
> "/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
> > No such file or directory
>
> [...]
>
> > I saw a very similar bug on Gentoo:
>
> [...]
>
> > (https://bugs.gentoo.org/888978)
>
> Well, that’s because Julia upstream does not take care about packagers;
> as explicitly mentioned in this comment:
>
>
> https://github.com/JuliaLang/MbedTLS.jl/pull/261#issuecomment-1346886879
>
> The Guixer Cayetano Santos fixed upstream the issue for one package.
> But as you are noticing it is not done for all.
>
> I do not know what is the best solution because the issue is coming from
> Julia itself.
>
> Efraim, any suggestion?
>
> Cheers,
> simon
>

[-- Attachment #2: Type: text/html, Size: 2523 bytes --]

bug#61121: Cannot import IJulia in Julia

[Simon Tournier]

Hi,

On Mon, 30 Jan 2023 at 21:55, Theodore Ehrenborg <theodore.ehrenborg@gmail.com> wrote:

> Gentoo appears to have fixed this bug by linking julia/cert.pem to the
> system's ca-certificates.crt.
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168

This trick is not possible, IIUC.

> Is there a way I could rebuild my own slightly modified Julia with a link
> like that?

Maybe, by adding the package nss-certs as propagated-inputs in the
definition of julia.

> I understand that there's probably a good reason that Guix's Julia doesn't
> by default have cert.pem, but I would be pleased with a hacky custom
> solution if it made Jupyter notebooks work.

The reason is security. ;-)  It’s Julia that does poorly here.

As pointed with the upstream package MbedTLS.jl, the fix should come
from Julia itself; therefore, it could be worth to open an issue, if it
is not already the case. ;-)

From my understanding, the culprit is this [1]:

--8<---------------cut here---------------start------------->8---
function __init__()
    global artifact_dir = dirname(Sys.BINDIR)
    global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
end
--8<---------------cut here---------------end--------------->8---

And it is not clear for me if NetworkOptions.jl [2] provides the option
of not, and I am missing why Julia itself does not depend on it.

1: https://github.com/JuliaLang/julia/blob/master/stdlib/MozillaCACerts_jll/src/MozillaCACerts_jll.jl#L20
2: https://github.com/JuliaLang/NetworkOptions.jl


Efraim, do you think it would be possible to patch Julia to point to
some certificates via bundled_ca_roots or ca_roots_path?

Well, somehow turn back these tests:

--8<---------------cut here---------------start------------->8---
             ;; julia embeds a certificate, we are not doing that
             (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
               (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
                "@test_broken isfile(MozillaCACerts_jll.cacert)"))
             ;; since certificate is not present some tests are failing in network option
             (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
               (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
                "@test_broken isfile(bundled_ca_roots())")
               (("@test ispath\\(ca_roots_path\\(\\)\\)")
                "@test_broken ispath(ca_roots_path())")
               (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
                "@test_broken ca_roots_path() != bundled_ca_roots()"))
--8<---------------cut here---------------end--------------->8---


Cheers,
simon

bug#61121: Cannot import IJulia in Julia

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 4198 bytes --]

On Tue, Jan 31, 2023 at 12:34:16PM +0100, Simon Tournier wrote:
> Hi,
> 
> On Mon, 30 Jan 2023 at 21:55, Theodore Ehrenborg <theodore.ehrenborg@gmail.com> wrote:
> 
> > Gentoo appears to have fixed this bug by linking julia/cert.pem to the
> > system's ca-certificates.crt.
> > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168
> 
> This trick is not possible, IIUC.
> 
> > Is there a way I could rebuild my own slightly modified Julia with a link
> > like that?
> 
> Maybe, by adding the package nss-certs as propagated-inputs in the
> definition of julia.

By itself I don't think this would do anything.

> > I understand that there's probably a good reason that Guix's Julia doesn't
> > by default have cert.pem, but I would be pleased with a hacky custom
> > solution if it made Jupyter notebooks work.
> 
> The reason is security. ;-)  It’s Julia that does poorly here.
> 
> As pointed with the upstream package MbedTLS.jl, the fix should come
> from Julia itself; therefore, it could be worth to open an issue, if it
> is not already the case. ;-)
> 
> From my understanding, the culprit is this [1]:
> 
> --8<---------------cut here---------------start------------->8---
> function __init__()
>     global artifact_dir = dirname(Sys.BINDIR)
>     global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
> end
> --8<---------------cut here---------------end--------------->8---
> 
> And it is not clear for me if NetworkOptions.jl [2] provides the option
> of not, and I am missing why Julia itself does not depend on it.
> 
> 1: https://github.com/JuliaLang/julia/blob/master/stdlib/MozillaCACerts_jll/src/MozillaCACerts_jll.jl#L20
> 2: https://github.com/JuliaLang/NetworkOptions.jl
> 
> 
> Efraim, do you think it would be possible to patch Julia to point to
> some certificates via bundled_ca_roots or ca_roots_path?

In the initial patch for julia-1.8.1 I think there was a substitution to
hardcode /etc/ssl/something instead for 'global cacert' but I took that
out since we don't like hardcoding that.

GIT_SSL_CAINFO=/home/efraim/.guix-home/profile/etc/ssl/certs/ca-certificates.crt
SSL_CERT_DIR=/run/current-system/profile/etc/ssl/certs
CURL_CA_BUNDLE=/home/efraim/.guix-home/profile/etc/ssl/certs/ca-certificates.crt
SSL_CERT_FILE=/run/current-system/profile/etc/ssl/certs/ca-certificates.crt

I think it would be fine to tell Julia to look at SSL_CERT_FILE as the
cacert so it can be overridden as desired, and then we can add a
(native-?)search-path to Julia for SSL_CERT_FILE.

Does anyone know offhand how to get the environment variable? If not
I'll grep the sources and then look online.

> Well, somehow turn back these tests:
> 
> --8<---------------cut here---------------start------------->8---
>              ;; julia embeds a certificate, we are not doing that
>              (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
>                (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
>                 "@test_broken isfile(MozillaCACerts_jll.cacert)"))
>              ;; since certificate is not present some tests are failing in network option
>              (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
>                (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
>                 "@test_broken isfile(bundled_ca_roots())")
>                (("@test ispath\\(ca_roots_path\\(\\)\\)")
>                 "@test_broken ispath(ca_roots_path())")
>                (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
>                 "@test_broken ca_roots_path() != bundled_ca_roots()"))
> --8<---------------cut here---------------end--------------->8---

That one might be a little harder, I'd rather not add nss-certs to the
build just for the test suite, but I'll see how it goes. Or at least
update the comment afterward.

> 
> Cheers,
> simon

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#61121: Cannot import IJulia in Julia

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1873 bytes --]

On Tue, Jan 31, 2023 at 12:34:16PM +0100, Simon Tournier wrote:
> 
> --8<---------------cut here---------------start------------->8---
> function __init__()
>     global artifact_dir = dirname(Sys.BINDIR)
>     global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
> end
> --8<---------------cut here---------------end--------------->8---

I've changed this line to:

global cacert = get(ENV, \"SSL_CERT_FILE\", "\"/etc/ssl/certs/ca-certificates.crt\")

and then tested it with the example at the beginning of the bug report.

> Well, somehow turn back these tests:
> 
> --8<---------------cut here---------------start------------->8---
>              ;; julia embeds a certificate, we are not doing that
>              (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
>                (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
>                 "@test_broken isfile(MozillaCACerts_jll.cacert)"))
>              ;; since certificate is not present some tests are failing in network option
>              (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
>                (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
>                 "@test_broken isfile(bundled_ca_roots())")
>                (("@test ispath\\(ca_roots_path\\(\\)\\)")
>                 "@test_broken ispath(ca_roots_path())")
>                (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
>                 "@test_broken ca_roots_path() != bundled_ca_roots()"))
> --8<---------------cut here---------------end--------------->8---

I wasn't able to turn these tests back on though.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]