bug#54010: Linux-libre > 5.15.12 leads to unbootable system

bug#54010: Linux-libre > 5.15.12 leads to unbootable system

[Ludovic Courtès]

2022 has left me without a working Linux-libre kernel.

Breakage occurred sometime between:

  • 92faad0adb93b8349bfd7c67911d3d95f0505eb2
    (Jan. 3rd; Linux-libre 5.15.12)

  • 43dd34c7777a212c99a97da7a2c237158faa9a1b
    (Jan. 31st; Linux-libre 5.15.17)

The symptoms are that, on my x86_64 laptop, the kernel boots, I can
enter the passphrase for my root file system (‘cryptsetup’ is invoked
from the initrd), things get printed, the console font changes, and then
things get stuck here.  The last messages are from my sound card driver
(Intel HDA).  It hangs before syslogd is started, and I think before
shepherd is even started (I can’t be 100% sure because I cannot scroll
back at that point, but I did add print statements in ‘shepherd.conf’
and didn’t see them.)

The same config works in ‘guix system vm’.

I can work around the problem by reconfiguring from current ‘master’ and
doing this:

--8<---------------cut here---------------start------------->8---
(define inferior-for-kernel
  (inferior-for-channels
   (list (channel
          (name 'guix)
          (url "https://git.savannah.gnu.org/git/guix.git")
          (commit
           "92faad0adb93b8349bfd7c67911d3d95f0505eb2")))))

(operating-system
  ;; …
  (kernel
    (first (lookup-inferior-packages inferior-for-kernel "linux-libre"))))
--8<---------------cut here---------------end--------------->8---

This suggests a kernel issue.

Ideas and suggestions on how to debug this most welcome!

Ludo’.

bug#54010: Linux-libre > 5.15.12 leads to unbootable system

[Squirrel via Bug reports for GNU Guix]

Have you tried blacklisting the the iwlwifi kernel module on boot by 
adding (kernel-arguments '("modprobe.blacklist=iwlwifi")) to 
operating-system declaration? The problem may be a bug from upstream 
linux. Please see 
https://lore.kernel.org/all/20220203161959.3edf1d6e@valencia/

For your reference, there is a similar thread (bug#53712: Guix System 
hangs after boot with linux-libre 5.15.17) in bug-guix at 
https://lists.gnu.org/archive/html/bug-guix/2022-02/msg00012.html

The patch for it has just been applied in the newest upstream kernel 
releases. (https://cdn.kernel.org/pub/linux/kernel/v5.x/ChangeLog-5.15.23)

bug#54010: Linux-libre > 5.15.12 leads to unbootable system

[Leo Famulari]

On Wed, Feb 16, 2022 at 12:39:46AM +0800, Squirrel via Bug reports for GNU Guix wrote:
> The patch for it has just been applied in the newest upstream kernel
> releases. (https://cdn.kernel.org/pub/linux/kernel/v5.x/ChangeLog-5.15.23)

Are you sure the bug is fixed in 5.15.23?

I don't see anything relevant in the changelog and I still have to apply
the workaround in order to boot 5.15.23 on affected hardware.

bug#54010: Linux-libre > 5.15.12 leads to unbootable system

[Leo Famulari]

On Tue, Feb 15, 2022 at 03:56:27PM +0100, Ludovic Courtès wrote:
> 2022 has left me without a working Linux-libre kernel.
> 
> Breakage occurred sometime between:
> 
>   • 92faad0adb93b8349bfd7c67911d3d95f0505eb2
>     (Jan. 3rd; Linux-libre 5.15.12)
> 
>   • 43dd34c7777a212c99a97da7a2c237158faa9a1b
>     (Jan. 31st; Linux-libre 5.15.17)

Isn't this #53712?

I recommend trying the workaround described there, which is to blacklist
the iwlwifi kernel module, if you are not using it but have iwlwifi
hardware:

https://issues.guix.gnu.org/53712#13

That worked for me and is consistent with the upstream discussion of
this bug:

https://lore.kernel.org/stable/164448100914.10463.9523338503936670263.kvalo@kernel.org/

bug#54010: bug#53712: Guix System hangs after boot with linux-libre 5.15.17

[Ludovic Courtès]

Hi,

Leo Famulari <leo@famulari.name> skribis:

> On Tue, Feb 15, 2022 at 03:56:27PM +0100, Ludovic Courtès wrote:
>> 2022 has left me without a working Linux-libre kernel.
>> 
>> Breakage occurred sometime between:
>> 
>>   • 92faad0adb93b8349bfd7c67911d3d95f0505eb2
>>     (Jan. 3rd; Linux-libre 5.15.12)
>> 
>>   • 43dd34c7777a212c99a97da7a2c237158faa9a1b
>>     (Jan. 31st; Linux-libre 5.15.17)
>
> Isn't this #53712?

It is!

> I recommend trying the workaround described there, which is to blacklist
> the iwlwifi kernel module, if you are not using it but have iwlwifi
> hardware:
>
> https://issues.guix.gnu.org/53712#13

I confirm that 5.15.23-gnu works for me with:

  (kernel-arguments
   '(; https://issues.guix.gnu.org/53712#12
     "modprobe.blacklist=usbmouse,usbkbd,iwlwifi"
     "quiet"))

Thank you!

Ludo’.

bug#54010: Linux-libre > 5.15.12 leads to unbootable system

[Squirrel via Bug reports for GNU Guix]

On Tue, 15 Feb 2022 16:15:51 -0500, Leo Famulari <leo@famulari.name> wrote:
> Are you sure the bug is fixed in 5.15.23?
>
> I don't see anything relevant in the changelog and I still have to apply
> the workaround in order to boot 5.15.23 on affected hardware.

I'm not sure...I haven't tested. Sorry for that!

The iwlwifi patch is related to use-after-free, but not the one in 
5.15.23 changelog. It's this one: 
https://lore.kernel.org/stable/164448100914.10463.9523338503936670263.kvalo@kernel.org/