bug#35660: guix weather runaway memory consumption

bug#35660: guix weather runaway memory consumption

[Danny Milosavljevic]

[-- Attachment #1.1: Type: text/plain, Size: 346 bytes --]

With guix master a0dc97a517cbc4c82640e30cacb2a564d128bbe9 guix weather consumes
8 GB of memory on a machine with 8 GB of memory and 1 GB of swap, then is killed
by Linux OOM killer.

Reproducible every time.

Reduced problem to GUIX_PACKAGE_PATH being set.

If I unset it, it works.

Attached contents of $GUIX_PACKAGE_PATH directory.

[-- Attachment #1.2: wip2.tar.gz --]
[-- Type: application/gzip, Size: 942080 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

bug#35660: guix weather runaway memory consumption

[Ludovic Courtès]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> With guix master a0dc97a517cbc4c82640e30cacb2a564d128bbe9 guix weather consumes
> 8 GB of memory on a machine with 8 GB of memory and 1 GB of swap, then is killed
> by Linux OOM killer.
>
> Reproducible every time.
>
> Reduced problem to GUIX_PACKAGE_PATH being set.
>
> If I unset it, it works.

I don’t think this is the place to debug personal package collections.
:-)

I’d love to give a hand but I feel we’re already overwhelmed with bugs
related to things happening on ‘master’.

WDYT?

Thanks,
Ludo’.

bug#35660: guix weather runaway memory consumption

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 1467 bytes --]

Hi Ludo,

On Mon, 13 May 2019 22:46:22 +0200
Ludovic Courtès <ludo@gnu.org> wrote:

> I don’t think this is the place to debug personal package collections.
> :-)

I didn't even remember the personal package collection at first.
This here is a bug in guix that happens to manifest with that personal package
collection.

The failure mode here is very very bad.  Guix will consume all available
memory and then start on the swap, at which point the computer will
become unresponsive to any input and the user can't save any open
documents and has to kill the power to the computer.

After the user did that and restarted the computer, it will happen all
over again on the next "guix weather" (100% of the time).

> I’d love to give a hand but I feel we’re already overwhelmed with bugs
> related to things happening on ‘master’.
> 
> WDYT?

Oh, it's not time-critical at all for me.

I reported this bug for documentation--I already know how to work
around it.

Though there is a bug somewhere--it's not going to magically vanish.

But a user is going to encounter the same problem eventually, then not
reduce it to his GUIX_PACKAGE_PATH and give up.  

Furthermore, if he wanted to find out where exactly the problem is, he'd have
to delete files in $GUIX_PACKAGE_PATH one by one and invoke guix weather
in-between, each time crashing his entire computer after like 10 min per try.
That's not a good user experience.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

bug#35660: guix weather runaway memory consumption

[Ludovic Courtès]

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> The failure mode here is very very bad.  Guix will consume all available
> memory and then start on the swap, at which point the computer will
> become unresponsive to any input and the user can't save any open
> documents and has to kill the power to the computer.

I agree that the failure mode is terrible.  It’s very likely a cycle in
the package graph, given the symptoms you describe.  So ‘guix build
OFFENDING-PACKAGE’ would probably give you the same result.

Chris Baines proposed a patch a while back to detect and report cycles,
but we never got around to polishing and integrating it.  That would
probably help a lot in these cases.

We can open a new bug for that (if there’s not already one), but I think
it should be framed in terms of cycle detection in the package graph and
error reporting.

Thanks,
Ludo’.

bug#35660: guix weather runaway memory consumption

[swedebugia]

On 2019-05-14 22:52, Ludovic Courtès wrote:
> Hi Danny,
> 
> Danny Milosavljevic <dannym@scratchpost.org> skribis:
> 
>> The failure mode here is very very bad.  Guix will consume all available
>> memory and then start on the swap, at which point the computer will
>> become unresponsive to any input and the user can't save any open
>> documents and has to kill the power to the computer.
> 
> I agree that the failure mode is terrible.  It’s very likely a cycle in
> the package graph, given the symptoms you describe.  So ‘guix build
> OFFENDING-PACKAGE’ would probably give you the same result.
> 
> Chris Baines proposed a patch a while back to detect and report cycles,
> but we never got around to polishing and integrating it.  That would
> probably help a lot in these cases.
> 
> We can open a new bug for that (if there’s not already one), but I think
> it should be framed in terms of cycle detection in the package graph and
> error reporting.

+1

When working on the NPM importer earlier this was very very common 
happening with "guix build NPM-PACKAGE" because of cycles.

My strategy was to manually kill the build after a minute or two if no 
output was produced.

-- 
Cheers Swedebugia

bug#35660: guix weather runaway memory consumption

[Maxim Cournoyer]

retitle 35660 Implement cycle detection with proper reporting
thanks

swedebugia <swedebugia@riseup.net> writes:

> On 2019-05-14 22:52, Ludovic Courtès wrote:
>> Hi Danny,
>> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>> 
>>> The failure mode here is very very bad.  Guix will consume all available
>>> memory and then start on the swap, at which point the computer will
>>> become unresponsive to any input and the user can't save any open
>>> documents and has to kill the power to the computer.
>> I agree that the failure mode is terrible.  It’s very likely a cycle
>> in
>> the package graph, given the symptoms you describe.  So ‘guix build
>> OFFENDING-PACKAGE’ would probably give you the same result.
>> Chris Baines proposed a patch a while back to detect and report
>> cycles,
>> but we never got around to polishing and integrating it.  That would
>> probably help a lot in these cases.
>> We can open a new bug for that (if there’s not already one), but I
>> think
>> it should be framed in terms of cycle detection in the package graph and
>> error reporting.
>
> +1

This would indeed be very useful.  I'm retitling this bug so that it is
more actionable.  Could someone link to the patch Chris Baines had
written going in this direction?

Thanks,

Maxim