bug#43143: Building iwd on armhf triggers kernel panic

bug#43143: Building iwd on armhf triggers kernel panic

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 594 bytes --]

I've been getting kernel panics when i try to build iwd, a dependency of
network-manager.

It crashes hard, so there's no recorded log, but I managed to scrape
this from an ssh session:

$ guix build iwd
...
PASS: unit/test-p2p
PASS: unit/test-client
./build-aux/test-driver: line 107:  6837 Aborted                 "$@" >
$log_file 2>&1
FAIL: unit/test-arc4
PASS: unit/test-sa


When the test-arc4 fails, soon after a kernel panic results and the
system freezes. Repeated this multiple times.

Going to remove iwd support for network manager locally and then try
again.


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

bug#43143: Building iwd on armhf triggers kernel panic

[Mark H Weaver]

Vagrant Cascadian <vagrant@debian.org> writes:
> I've been getting kernel panics when i try to build iwd, a dependency of
> network-manager.

Which kernel package variant(s) and version(s) have you seen panic when
building iwd?  What hardware?

     Thanks,
       Mark

bug#43143: Building iwd on armhf triggers kernel panic

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1078 bytes --]

On 2020-08-31, Mark H. Weaver wrote:
> Vagrant Cascadian <vagrant@debian.org> writes:
>> I've been getting kernel panics when i try to build iwd, a dependency of
>> network-manager.
>
> Which kernel package variant(s) and version(s) have you seen panic when
> building iwd?  What hardware?

The hardware is veryron-speedy (a.k.a. asus chromebook c201p), with a
rockchip rk3288 CPU and 4GB of ram; I might be able to test on
additional hardware at some point (I've got a pretty old guix install on
a novena, but don't recall if it was in good working order last time it
booted).

linux-libre-arm-generic@5.4 5.4.57, and presumably older versions as
well, as I had commits in my local guix from mid June disabling iwd on
network-manager.

I've had issues booting linux-libre 5.6+ on this hardware, and struggled
to get the right initrd modules to use the regular "linux-libre"
variant, though in theory that should also be possible, but might
require additional kernel config options, and I'd like to postpone
exploring those rabbit holes for the moment...


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

bug#43143: Building iwd on armhf triggers kernel panic

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

On 2020-08-31, Vagrant Cascadian wrote:
> On 2020-08-31, Mark H. Weaver wrote:
>> Vagrant Cascadian <vagrant@debian.org> writes:
>>> I've been getting kernel panics when i try to build iwd, a dependency of
>>> network-manager.
>>
>> Which kernel package variant(s) and version(s) have you seen panic when
>> building iwd?  What hardware?
>
> The hardware is veryron-speedy (a.k.a. asus chromebook c201p), with a
> rockchip rk3288 CPU and 4GB of ram; I might be able to test on
> additional hardware at some point (I've got a pretty old guix install on
> a novena, but don't recall if it was in good working order last time it
> booted).

Ok, managed to build iwd just fine on a novena imx6 platform, running
armhf on "linux-libre" 5.4.25 (haven't yet finished upgrading the
system).

So it appears to potentially be hardware-specific? The imx6 vs. rk3288
are pretty different platforms; I'm not remembering off the top of my
head what the major differences are, but if the build is detecting some
optional cpu feature, and the test is assuming one way or the
other... though still, triggering a kernel panic is a bit exciting way
to fail!


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

bug#43143: Building iwd on armhf triggers kernel panic

[Mark H Weaver]

Hi Vagrant,

Vagrant Cascadian <vagrant@debian.org> writes:

> On 2020-08-31, Vagrant Cascadian wrote:
>> On 2020-08-31, Mark H. Weaver wrote:
>>> Vagrant Cascadian <vagrant@debian.org> writes:
>>>> I've been getting kernel panics when i try to build iwd, a dependency of
>>>> network-manager.
>>>
>>> Which kernel package variant(s) and version(s) have you seen panic when
>>> building iwd?  What hardware?
>>
>> The hardware is veryron-speedy (a.k.a. asus chromebook c201p), with a
>> rockchip rk3288 CPU and 4GB of ram; I might be able to test on
>> additional hardware at some point (I've got a pretty old guix install on
>> a novena, but don't recall if it was in good working order last time it
>> booted).

That message went on to say the following, although you omitted it here:

>> linux-libre-arm-generic@5.4 5.4.57, and presumably older versions as
>> well, as I had commits in my local guix from mid June disabling iwd on
>> network-manager.

> Ok, managed to build iwd just fine on a novena imx6 platform, running
> armhf on "linux-libre" 5.4.25 (haven't yet finished upgrading the
> system).
>
> So it appears to potentially be hardware-specific? The imx6 vs. rk3288
> are pretty different platforms;

Based on what you wrote, it sounds like it could either be a
hardware-specific problem, or else a kernel-configuration-specific
problem, since you used radically different kernel configurations in
these two cases.

If I understand correctly, 'linux-libre-arm-generic@5.4' is the variant
that panicked on an rk3288, whereas the 'linux-libre@5.4' variant with
our hand-crafted default configuration worked on an imx6.

I don't know whether it's possible to use precisely the same kernel
configuration on these two systems.  Anyway, it would be helpful to
retry the 'iwd' build on these two devices with kernel versions and
configurations that are identical except for a minimal set of required
differences.  That would help us narrow down the set of possible causes
for this kernel panic.

What do you think?

      Thanks,
        Mark

bug#43143: Building iwd on armhf triggers kernel panic

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2775 bytes --]

On 2020-09-02, Mark H Weaver wrote:
> Vagrant Cascadian <vagrant@debian.org> writes:
>> On 2020-08-31, Vagrant Cascadian wrote:
>>> On 2020-08-31, Mark H. Weaver wrote:
>>>> Vagrant Cascadian <vagrant@debian.org> writes:
>>>>> I've been getting kernel panics when i try to build iwd, a dependency of
>>>>> network-manager.
>>>>
>>>> Which kernel package variant(s) and version(s) have you seen panic when
>>>> building iwd?  What hardware?
>>>
>>> The hardware is veryron-speedy (a.k.a. asus chromebook c201p), with a
>>> rockchip rk3288 CPU and 4GB of ram; I might be able to test on
>>> additional hardware at some point (I've got a pretty old guix install on
>>> a novena, but don't recall if it was in good working order last time it
>>> booted).
>
> That message went on to say the following, although you omitted it here:
>
>>> linux-libre-arm-generic@5.4 5.4.57, and presumably older versions as
>>> well, as I had commits in my local guix from mid June disabling iwd on
>>> network-manager.
>
>> Ok, managed to build iwd just fine on a novena imx6 platform, running
>> armhf on "linux-libre" 5.4.25 (haven't yet finished upgrading the
>> system).
>>
>> So it appears to potentially be hardware-specific? The imx6 vs. rk3288
>> are pretty different platforms;
>
> Based on what you wrote, it sounds like it could either be a
> hardware-specific problem, or else a kernel-configuration-specific
> problem, since you used radically different kernel configurations in
> these two cases.
>
> If I understand correctly, 'linux-libre-arm-generic@5.4' is the variant
> that panicked on an rk3288, whereas the 'linux-libre@5.4' variant with
> our hand-crafted default configuration worked on an imx6.
>
> I don't know whether it's possible to use precisely the same kernel
> configuration on these two systems.  Anyway, it would be helpful to
> retry the 'iwd' build on these two devices with kernel versions and
> configurations that are identical except for a minimal set of required
> differences.  That would help us narrow down the set of possible causes
> for this kernel panic.

Sorry for the late follow-up...

Using a more recent version of linux-libre-arm-generic@5.4, it failed to
build on both the veyron-speedy (rk3288) and the novena (imx6). It
doesn't necessarily *always* trigger a kernel panic; I haven't
identified which conditions trigger the kernel panic, but so far I've
only seen them on the veyron-speedy. I haven't seen it successfully
build with any recent linux-libre-arm-generic kernel, though recent
linux-libre@5.4 and @5.8 worked.

So it seems something in iwd needs specific kernel features in order to
build at all...

Not sure when I'll get a chance to debug further, but figured I'd update
about what I've tried so far.

live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

bug#43143: Building iwd on armhf triggers kernel panic

[Maxim Cournoyer]

Hello,

[...]

> Using a more recent version of linux-libre-arm-generic@5.4, it failed to
> build on both the veyron-speedy (rk3288) and the novena (imx6). It
> doesn't necessarily *always* trigger a kernel panic; I haven't
> identified which conditions trigger the kernel panic, but so far I've
> only seen them on the veyron-speedy. I haven't seen it successfully
> build with any recent linux-libre-arm-generic kernel, though recent
> linux-libre@5.4 and @5.8 worked.
>
> So it seems something in iwd needs specific kernel features in order to
> build at all...
>
> Not sure when I'll get a chance to debug further, but figured I'd update
> about what I've tried so far.

Bordeaux managed to build it, according to 'guix weather
--system=armhf-linux iwd', hence I'm closing this old forgotten issue.
Please reopen if you still experience it on a standard Guix System
kernel.

Thanks,

Maxim