Re: 01/02: gnu: libressl: Update to 2.5.3.

Re: 01/02: gnu: libressl: Update to 2.5.3.

[Mark H Weaver]

leo@famulari.name (Leo Famulari) writes:

> lfam pushed a commit to branch master
> in repository guix.
>
> commit 69121e95cd5568238a0f207dfab708501ec4a753
> Author: Leo Famulari <leo@famulari.name>
> Date:   Tue Apr 11 13:09:03 2017 -0400
>
>     gnu: libressl: Update to 2.5.3.
>     
>     * gnu/packages/tls.scm (libressl): Update to 2.5.3.

This failed to build on Hydra on both x86_64 and i686 (the only builds
that have been attempted so far):

  https://hydra.gnu.org/build/1976679  (x86_64)
  https://hydra.gnu.org/build/1976657  (i686)

In both cases 26 out of 60 tests failed, and from a quick glance it
seems to be the same set of failures in both cases, so this doesn't
appear to be a non-deterministic problem.

Could you take a look?

     Mark

Re: 01/02: gnu: libressl: Update to 2.5.3.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1949 bytes --]

On Wed, Apr 12, 2017 at 05:15:32AM -0400, Mark H Weaver wrote:
> leo@famulari.name (Leo Famulari) writes:
> >     gnu: libressl: Update to 2.5.3.
> >     
> >     * gnu/packages/tls.scm (libressl): Update to 2.5.3.
> 
> This failed to build on Hydra on both x86_64 and i686 (the only builds
> that have been attempted so far):
> 
>   https://hydra.gnu.org/build/1976679  (x86_64)
>   https://hydra.gnu.org/build/1976657  (i686)

The failures look like this:

[...]
FAIL: arc4randomforktest.sh
PASS: asn1test
PASS: base64test
../test-driver: line 107:  6370 Killed                  "$@" > $log_file 2>&1
FAIL: bntest
PASS: bftest
PASS: asn1time
PASS: bytestringtest
PASS: chachatest
../test-driver: line 107:  6397 Killed                  "$@" > $log_file 2>&1
FAIL: cipherstest
../test-driver: line 107:  6402 Killed                  "$@" > $log_file 2>&1
FAIL: cipher_list
../test-driver: line 107:  6410 Killed                  "$@" > $log_file 2>&1
[...]

This is libressl-portable bug #290:

https://github.com/libressl-portable/portable/issues/290

There is a problem with using getentropy() or getrandom() from
glibc-2.25 with Linux < 3.17, when these syscalls where introduced.
Basically, glibc will return ENOSYS, which applications are not handling
properly.

I expect the build to succeed on armhf, where I believe the builders
have kernels > 3.17.

In the case of libressl, the developers have closed as WONTFIX, although
perhaps they could be persuaded to make libressl handle ENOSYS somehow.

Cpython hit the same problem, and they worked around it. This means that
the Python interpreters Hydra builds for x86_64 and i686 not use the new
getentropy() / getrandom() syscalls, even though many Guix users and
probably all GuixSD users have more recent kernels:

https://bugs.python.org/issue29157

Can we disable the build on Hydra without marking the package as
non-substitutable?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: 01/02: gnu: libressl: Update to 2.5.3.

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1402 bytes --]

Leo Famulari <leo@famulari.name> skribis:

> This is libressl-portable bug #290:
>
> https://github.com/libressl-portable/portable/issues/290
>
> There is a problem with using getentropy() or getrandom() from
> glibc-2.25 with Linux < 3.17, when these syscalls where introduced.
> Basically, glibc will return ENOSYS, which applications are not handling
> properly.
>
> I expect the build to succeed on armhf, where I believe the builders
> have kernels > 3.17.
>
> In the case of libressl, the developers have closed as WONTFIX, although
> perhaps they could be persuaded to make libressl handle ENOSYS somehow.
>
> Cpython hit the same problem, and they worked around it. This means that
> the Python interpreters Hydra builds for x86_64 and i686 not use the new
> getentropy() / getrandom() syscalls, even though many Guix users and
> probably all GuixSD users have more recent kernels:
>
> https://bugs.python.org/issue29157
>
> Can we disable the build on Hydra without marking the package as
> non-substitutable?

A simple approach is to force LibreSSL to always use its non-getentropy
code, and lift this restriction once we clearly require newer kernels¹.
The attached patch does that.

Thoughts?

Thanks,
Ludo’.

¹ We currently build libc with "--enable-kernel=2.6.32", and this is
  pretty much what defines our minimal kernel version requirement.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 711 bytes --]

diff --git a/gnu/packages/tls.scm b/gnu/packages/tls.scm
index 1c99a3ad7..5fe35a535 100644
--- a/gnu/packages/tls.scm
+++ b/gnu/packages/tls.scm
@@ -442,6 +442,10 @@ required structures.")
        (base32
         "0c4awq45cl757fv7f7f75i5i0ibc6v7ns13n7xvfak7chv2lrqql"))))
     (build-system gnu-build-system)
+    (arguments
+     ;; Do as if 'getentropy' was missing since older Linux kernels lack it
+     ;; and libc would return ENOSYS, which is not properly handled.
+     '(#:configure-flags '("ac_cv_func_getentropy=no")))
     (native-search-paths
       ;; FIXME: These two variables must designate a single file or directory
       ;; and are not actually "search paths."  In practice it works OK in

Re: 01/02: gnu: libressl: Update to 2.5.3.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1303 bytes --]

On Thu, Apr 13, 2017 at 05:08:29PM +0200, Ludovic Courtès wrote:
> A simple approach is to force LibreSSL to always use its non-getentropy
> code, and lift this restriction once we clearly require newer kernels¹.
> The attached patch does that.
> 
> Thoughts?

> +     ;; Do as if 'getentropy' was missing since older Linux kernels lack it
> +     ;; and libc would return ENOSYS, which is not properly handled.
> +     '(#:configure-flags '("ac_cv_func_getentropy=no")))

If we are committed to building glibc with the 2.6 kernel headers, and
to providing substitutes for libressl and it's dependent packages, then
I think this patch is a good option.

But, it's a bit of a shame to leave this ~2.5 year old feature behind,
especially when the 2.6 Linux series is not even part of the Linux
long-term-support project. [0] These kernels *will* live for a long time
through support from RHEL; their most recent kernel on RHEL7 is 3.10.

However, I don't fully understand the impact of building glibc with a
newer set of headers, so my objection is a weak one :)

Personally, I don't think it's paramount to offer substitutes for the
packages in question. But I know this is an unpopular position, in
general :)

[0]
https://ltsi.linuxfoundation.org/
https://www.kernel.org/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: 01/02: gnu: libressl: Update to 2.5.3.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1328 bytes --]

On Thu, Apr 13, 2017 at 02:59:21PM -0400, Leo Famulari wrote:
> On Thu, Apr 13, 2017 at 05:08:29PM +0200, Ludovic Courtès wrote:
> > A simple approach is to force LibreSSL to always use its non-getentropy
> > code, and lift this restriction once we clearly require newer kernels¹.
> > The attached patch does that.
> > 
> > Thoughts?
> 
> > +     ;; Do as if 'getentropy' was missing since older Linux kernels lack it
> > +     ;; and libc would return ENOSYS, which is not properly handled.
> > +     '(#:configure-flags '("ac_cv_func_getentropy=no")))
> 
> If we are committed to building glibc with the 2.6 kernel headers, and
> to providing substitutes for libressl and it's dependent packages, then
> I think this patch is a good option.
> 
> But, it's a bit of a shame to leave this ~2.5 year old feature behind,
> especially when the 2.6 Linux series is not even part of the Linux
> long-term-support project. [0] These kernels *will* live for a long time
> through support from RHEL; their most recent kernel on RHEL7 is 3.10.
> 
> However, I don't fully understand the impact of building glibc with a
> newer set of headers, so my objection is a weak one :)

After reading more, I realized that my knowledge on this topic was even
weaker than I thought. So please disregard what I wrote here.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: 01/02: gnu: libressl: Update to 2.5.3.

[Ludovic Courtès]

Leo Famulari <leo@famulari.name> skribis:

> On Thu, Apr 13, 2017 at 05:08:29PM +0200, Ludovic Courtès wrote:
>> A simple approach is to force LibreSSL to always use its non-getentropy
>> code, and lift this restriction once we clearly require newer kernels¹.
>> The attached patch does that.
>> 
>> Thoughts?
>
>> +     ;; Do as if 'getentropy' was missing since older Linux kernels lack it
>> +     ;; and libc would return ENOSYS, which is not properly handled.
>> +     '(#:configure-flags '("ac_cv_func_getentropy=no")))
>
> If we are committed to building glibc with the 2.6 kernel headers, and
> to providing substitutes for libressl and it's dependent packages, then
> I think this patch is a good option.
>
> But, it's a bit of a shame to leave this ~2.5 year old feature behind,
> especially when the 2.6 Linux series is not even part of the Linux
> long-term-support project. [0] These kernels *will* live for a long time
> through support from RHEL; their most recent kernel on RHEL7 is 3.10.
>
> However, I don't fully understand the impact of building glibc with a
> newer set of headers, so my objection is a weak one :)

I would suggest bumping the kernel requirement in glibc on the next
core-updates cycle (well, the current one!) and also making sure all our
build machines run the right thing.

In the meantime, I’d apply the above hack, and hopefully we can remove
it in a couple of months.

How does that sound?

Ludo’.

Re: 01/02: gnu: libressl: Update to 2.5.3.

[Leo Famulari]

On Fri, Apr 14, 2017 at 02:43:34PM +0200, Ludovic Courtès wrote:
> I would suggest bumping the kernel requirement in glibc on the next
> core-updates cycle (well, the current one!) and also making sure all our
> build machines run the right thing.
> 
> In the meantime, I’d apply the above hack, and hopefully we can remove
> it in a couple of months.
> 
> How does that sound?

Sounds good to me! Hopefully the libressl-portable developers choose to
handle ENOSYS in a future release.

Re: 01/02: gnu: libressl: Update to 2.5.3.

[Mark H Weaver]

Leo Famulari <leo@famulari.name> writes:

> On Thu, Apr 13, 2017 at 05:08:29PM +0200, Ludovic Courtès wrote:
>> A simple approach is to force LibreSSL to always use its non-getentropy
>> code, and lift this restriction once we clearly require newer kernels¹.
>> The attached patch does that.
>> 
>> Thoughts?
>
>> +     ;; Do as if 'getentropy' was missing since older Linux kernels lack it
>> +     ;; and libc would return ENOSYS, which is not properly handled.
>> +     '(#:configure-flags '("ac_cv_func_getentropy=no")))

Ludo's approach looks good to me.

> Personally, I don't think it's paramount to offer substitutes for the
> packages in question. But I know this is an unpopular position, in
> general :)

It's not just about not providing substitutes.  At present, our libressl
simply won't work properly on systems with older kernels, including
hydra.gnu.org and our x86 build slaves.

One issue is that there's a longstanding hope for us to switch to using
libressl for most or all packages that currently use openssl.  We would
be blocked from doing that if we accept that our libressl won't work on
older kernels.

Thoughts?

       Mark

Re: 01/02: gnu: libressl: Update to 2.5.3.

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> Leo Famulari <leo@famulari.name> writes:
>
>> On Thu, Apr 13, 2017 at 05:08:29PM +0200, Ludovic Courtès wrote:
>>> A simple approach is to force LibreSSL to always use its non-getentropy
>>> code, and lift this restriction once we clearly require newer kernels¹.
>>> The attached patch does that.
>>> 
>>> Thoughts?
>>
>>> +     ;; Do as if 'getentropy' was missing since older Linux kernels lack it
>>> +     ;; and libc would return ENOSYS, which is not properly handled.
>>> +     '(#:configure-flags '("ac_cv_func_getentropy=no")))
>
> Ludo's approach looks good to me.

Thanks for checking.  Committed.

>> Personally, I don't think it's paramount to offer substitutes for the
>> packages in question. But I know this is an unpopular position, in
>> general :)
>
> It's not just about not providing substitutes.  At present, our libressl
> simply won't work properly on systems with older kernels, including
> hydra.gnu.org and our x86 build slaves.
>
> One issue is that there's a longstanding hope for us to switch to using
> libressl for most or all packages that currently use openssl.  We would
> be blocked from doing that if we accept that our libressl won't work on
> older kernels.

Yeah, I think we have to be consistent about what our kernel requirement
is.  With libc unconditionally providing syscall wrappers like this one,
it’s easy to let incompatibilities slip through.

Ludo’.