why is linux-libre-headers behind linux-libre?

why is linux-libre-headers behind linux-libre?

[Dave Love]

Why is linux-libre-headers a long way behind linux-libre (currently at
version 4.4.47, compared with 4.13.10 for linux-libre)?

I ask because current OmniPath fabric support ("psm2") won't build
against it, so I wonder if it can be updated.  Maybe other recent
hardware support will be similar.  (I realize the relevant driver, hfi1,
is deblobbed in linux-libre.)

Re: why is linux-libre-headers behind linux-libre?

[Vincent Legoll]

Hello,

On Tue, Oct 31, 2017 at 1:35 PM, Dave Love <fx@gnu.org> wrote:
> Why is linux-libre-headers a long way behind linux-libre (currently at
> version 4.4.47, compared with 4.13.10 for linux-libre)?

I suspect this is due to massive rebuilding that would occur when
updating linux-libre-headers

-- 
Vincent Legoll

Re: why is linux-libre-headers behind linux-libre?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 762 bytes --]

On Tue, Oct 31, 2017 at 02:00:35PM +0100, Vincent Legoll wrote:
> Hello,
> 
> On Tue, Oct 31, 2017 at 1:35 PM, Dave Love <fx@gnu.org> wrote:
> > Why is linux-libre-headers a long way behind linux-libre (currently at
> > version 4.4.47, compared with 4.13.10 for linux-libre)?
> 
> I suspect this is due to massive rebuilding that would occur when
> updating linux-libre-headers
> 

This is typically updated in the core-updates branch, but it hasn't been
updated yet. Based on the LTS versions, we should upgrade it to the 4.9
branch.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: why is linux-libre-headers behind linux-libre?

[Dave Love]

Efraim Flashner <efraim@flashner.co.il> writes:

> On Tue, Oct 31, 2017 at 02:00:35PM +0100, Vincent Legoll wrote:
>> Hello,
>> 
>> On Tue, Oct 31, 2017 at 1:35 PM, Dave Love <fx@gnu.org> wrote:
>> > Why is linux-libre-headers a long way behind linux-libre (currently at
>> > version 4.4.47, compared with 4.13.10 for linux-libre)?
>> 
>> I suspect this is due to massive rebuilding that would occur when
>> updating linux-libre-headers
>> 
>
> This is typically updated in the core-updates branch, but it hasn't been
> updated yet. Based on the LTS versions, we should upgrade it to the 4.9
> branch.

Thanks for the explanations.  I checked that 4.9 would support the
Omnipath library, at least.

Re: why is linux-libre-headers behind linux-libre?

[Ludovic Courtès]

Hello,

Dave Love <fx@gnu.org> skribis:

> Efraim Flashner <efraim@flashner.co.il> writes:
>
>> On Tue, Oct 31, 2017 at 02:00:35PM +0100, Vincent Legoll wrote:
>>> Hello,
>>> 
>>> On Tue, Oct 31, 2017 at 1:35 PM, Dave Love <fx@gnu.org> wrote:
>>> > Why is linux-libre-headers a long way behind linux-libre (currently at
>>> > version 4.4.47, compared with 4.13.10 for linux-libre)?
>>> 
>>> I suspect this is due to massive rebuilding that would occur when
>>> updating linux-libre-headers

That and also because glibc targets (can target) older kernels, which is
something we rely on.

>> This is typically updated in the core-updates branch, but it hasn't been
>> updated yet. Based on the LTS versions, we should upgrade it to the 4.9
>> branch.
>
> Thanks for the explanations.  I checked that 4.9 would support the
> Omnipath library, at least.

The Omnipath library relies on Linux (not libc) headers, and a specific
version thereof?

I suppose we could also introduce a more recent version of
‘linux-libre-headers’ specifically for this purpose, with the
understanding that the resulting binaries rely on a specific kernel
version.

Thanks,
Ludo’.

Re: why is linux-libre-headers behind linux-libre?

[Mark H Weaver]

Hi,

ludo@gnu.org (Ludovic Courtès) writes:

> Dave Love <fx@gnu.org> skribis:
>
>> Efraim Flashner <efraim@flashner.co.il> writes:
>>
>>> On Tue, Oct 31, 2017 at 02:00:35PM +0100, Vincent Legoll wrote:
>>>> Hello,
>>>> 
>>>> On Tue, Oct 31, 2017 at 1:35 PM, Dave Love <fx@gnu.org> wrote:
>>>> > Why is linux-libre-headers a long way behind linux-libre (currently at
>>>> > version 4.4.47, compared with 4.13.10 for linux-libre)?
>>>> 
>>>> I suspect this is due to massive rebuilding that would occur when
>>>> updating linux-libre-headers
>
> That and also because glibc targets (can target) older kernels, which is
> something we rely on.
>
>>> This is typically updated in the core-updates branch, but it hasn't been
>>> updated yet. Based on the LTS versions, we should upgrade it to the 4.9
>>> branch.
>>
>> Thanks for the explanations.  I checked that 4.9 would support the
>> Omnipath library, at least.
>
> The Omnipath library relies on Linux (not libc) headers, and a specific
> version thereof?
>
> I suppose we could also introduce a more recent version of
> ‘linux-libre-headers’ specifically for this purpose, with the
> understanding that the resulting binaries rely on a specific kernel
> version.

Are you sure about this?  My impression was that binaries compiled with
newer linux-libre-headers can be run on older kernels.  If you were
correct, then the binaries we've been building throughout 2017 could be
reliably run only on linux-libre-4.4 or newer.

In fact, we've been successfully running these Guix binaries on
hydra.gnu.org with its old 2.6.x kernel, and on build slaves running
kernels older than 4.4.  Furthermore, I strongly suspect that many of
our users (e.g. Trisquel users) have been running Guix on older kernels
as well, and yet I don't recall seeing any bug reports related to this.

My recommendation would be to update linux-libre-headers to the latest
LTS kernel (currently 4.9.x) in every core-updates cycle.

What do you think?

      Mark

Re: why is linux-libre-headers behind linux-libre?

[Pjotr Prins]

On Mon, Nov 06, 2017 at 03:44:41AM -0500, Mark H Weaver wrote:
> Are you sure about this?  My impression was that binaries compiled with
> newer linux-libre-headers can be run on older kernels.  If you were
> correct, then the binaries we've been building throughout 2017 could be
> reliably run only on linux-libre-4.4 or newer.
> 
> In fact, we've been successfully running these Guix binaries on
> hydra.gnu.org with its old 2.6.x kernel, and on build slaves running
> kernels older than 4.4.  Furthermore, I strongly suspect that many of
> our users (e.g. Trisquel users) have been running Guix on older kernels
> as well, and yet I don't recall seeing any bug reports related to this.
> 
> My recommendation would be to update linux-libre-headers to the latest
> LTS kernel (currently 4.9.x) in every core-updates cycle.
> 
> What do you think?

The Linux kernel API is remarkably stable. It is indeed the reason we
*can* run GNU Guix on older kernels in the first place - the kernel is
our single dependency. The headers do change, but mostly due to kernel
internals which are not used from glibc etc.

I remember for Nix there was a breaking kernel API change at
some point.  But it is extremely rare. Must have been 10 years ago.

Pj.
-- 

Re: why is linux-libre-headers behind linux-libre?

[Dave Love]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> The Linux kernel API is remarkably stable.

Actually, I don't think that's the case, speaking as someone who's had
to deal with out-of-tree drivers for various reasons.  For instance, the
OrangeFS module typically broke when I tried to rebuild the package on a
new version of Fedora.  Even RHEL can break such things with minor
releases.

> It is indeed the reason we
> *can* run GNU Guix on older kernels in the first place - the kernel is
> our single dependency. The headers do change, but mostly due to kernel
> internals which are not used from glibc etc.

It's a hardware driver library, not anything at the libc level.

Re: why is linux-libre-headers behind linux-libre?

[Mark H Weaver]

Dave Love <fx@gnu.org> writes:

> Pjotr Prins <pjotr.public12@thebird.nl> writes:
>
>> The Linux kernel API is remarkably stable.
>
> Actually, I don't think that's the case, speaking as someone who's had
> to deal with out-of-tree drivers for various reasons.  For instance, the
> OrangeFS module typically broke when I tried to rebuild the package on a
> new version of Fedora.  Even RHEL can break such things with minor
> releases.

When Pjotr said that the Linux API is remarkably stable, I assume he was
talking about the syscall API, i.e. the API available to programs run in
user space.  The Linux kernel project has a strict policy to avoid
changing its API in a way that would break user programs.

The internal API available to kernel modules and drivers is another
matter entirely.  The Linux kernel project reserves the right to change
that internal API freely, and does so regularly.

      Mark

Re: why is linux-libre-headers behind linux-libre?

[Pjotr Prins]

On Mon, Nov 06, 2017 at 02:22:50PM -0500, Mark H Weaver wrote:
> When Pjotr said that the Linux API is remarkably stable, I assume he was
> talking about the syscall API, i.e. the API available to programs run in
> user space.  The Linux kernel project has a strict policy to avoid
> changing its API in a way that would break user programs.
> 
> The internal API available to kernel modules and drivers is another
> matter entirely.  The Linux kernel project reserves the right to change
> that internal API freely, and does so regularly.

Right, and libre-headers should follow that kernel version. In other
words, for the generic case any version of libre-headers will work. I
am not talking about device drivers etc.

Pj.
-- 

Re: why is linux-libre-headers behind linux-libre?

[Ludovic Courtès]

Mark H Weaver <mhw@netris.org> skribis:

> Hi,
>
> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Dave Love <fx@gnu.org> skribis:
>>
>>> Efraim Flashner <efraim@flashner.co.il> writes:
>>>
>>>> On Tue, Oct 31, 2017 at 02:00:35PM +0100, Vincent Legoll wrote:
>>>>> Hello,
>>>>> 
>>>>> On Tue, Oct 31, 2017 at 1:35 PM, Dave Love <fx@gnu.org> wrote:
>>>>> > Why is linux-libre-headers a long way behind linux-libre (currently at
>>>>> > version 4.4.47, compared with 4.13.10 for linux-libre)?
>>>>> 
>>>>> I suspect this is due to massive rebuilding that would occur when
>>>>> updating linux-libre-headers
>>
>> That and also because glibc targets (can target) older kernels, which is
>> something we rely on.
>>
>>>> This is typically updated in the core-updates branch, but it hasn't been
>>>> updated yet. Based on the LTS versions, we should upgrade it to the 4.9
>>>> branch.
>>>
>>> Thanks for the explanations.  I checked that 4.9 would support the
>>> Omnipath library, at least.
>>
>> The Omnipath library relies on Linux (not libc) headers, and a specific
>> version thereof?
>>
>> I suppose we could also introduce a more recent version of
>> ‘linux-libre-headers’ specifically for this purpose, with the
>> understanding that the resulting binaries rely on a specific kernel
>> version.
>
> Are you sure about this?  My impression was that binaries compiled with
> newer linux-libre-headers can be run on older kernels.  If you were
> correct, then the binaries we've been building throughout 2017 could be
> reliably run only on linux-libre-4.4 or newer.

You’re right, but my guess was that if the Omnipath library requires
specific kernel headers, then it may be using functionality (and
syscalls) only implemented by newer kernels.  I haven’t checked though.

Ludo’.

Re: why is linux-libre-headers behind linux-libre?

[Dave Love]

Ludovic Courtès <ludo@gnu.org> writes:

> The Omnipath library relies on Linux (not libc) headers, and a specific
> version thereof?

The current version of the library relies on a recent enough version of
the module/headers as it's fairly new stuff.  It's interacting with the
driver at a fairly low level.  I assume it's similar for the Infiniband
equivalents, but I don't know what the Mellanox ones require.

I don't know the details of what support is required, or if there's
useful out-of-tree support for different versions, but OPA is probably
most commonly used on RHEL 7-ish kernels.