bug#66997: nar-herder uses 10 GiB of resident memory, 100% CPU on hydra-guix-129

bug#66997: nar-herder uses 10 GiB of resident memory, 100% CPU on hydra-guix-129

[Maxim Cournoyer]

Hi,

I was looking at top on the hydra-guix-129 node, which runs nar-herder,
and saw this:

--8<---------------cut here---------------start------------->8---
4772 nar-her+  20   0   28.9g  11.5g 100.0   6.1     55,55 S .nar-herder-rea
--8<---------------cut here---------------end--------------->8---

11.5 GiB of memory seems a bit excessive, no?  Its cumulated processing
time is also at the top of the table, and it seems stuck at 100% of CPU
usage.

Is this expected of the nar-herder, or has it gone awry?

-- 
Thanks,
Maxim

bug#66997: nar-herder uses 10 GiB of resident memory, 100% CPU on hydra-guix-129

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 727 bytes --]


Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> I was looking at top on the hydra-guix-129 node, which runs nar-herder,
> and saw this:
>
> 4772 nar-her+  20   0   28.9g  11.5g 100.0   6.1     55,55 S .nar-herder-rea
>
> 11.5 GiB of memory seems a bit excessive, no?  Its cumulated processing
> time is also at the top of the table, and it seems stuck at 100% of CPU
> usage.
>
> Is this expected of the nar-herder, or has it gone awry?

I think hydra-guix-129 was running an older version of the nar-herder,
so I've updated it and I think that's improved the situation.

I've just reconfigured the machine to use a specific commit at the
moment, but it would be good to bump the commit that the Guix package is
using.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

bug#66997: nar-herder uses 10 GiB of resident memory, 100% CPU on hydra-guix-129

[Maxim Cournoyer]

Hi,

Christopher Baines <mail@cbaines.net> writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>> I was looking at top on the hydra-guix-129 node, which runs nar-herder,
>> and saw this:
>>
>> 4772 nar-her+  20   0   28.9g  11.5g 100.0   6.1     55,55 S .nar-herder-rea
>>
>> 11.5 GiB of memory seems a bit excessive, no?  Its cumulated processing
>> time is also at the top of the table, and it seems stuck at 100% of CPU
>> usage.
>>
>> Is this expected of the nar-herder, or has it gone awry?
>
> I think hydra-guix-129 was running an older version of the nar-herder,
> so I've updated it and I think that's improved the situation.
>
> I've just reconfigured the machine to use a specific commit at the
> moment, but it would be good to bump the commit that the Guix package is
> using.

The situation appears unresolved; c.f. `top' output on hydra-guix-129:

--8<---------------cut here---------------start------------->8---
62775 nar-her+  20   0  903.7g  93.9g   0.0  49.8     22,44 S  `- fibers 0 
--8<---------------cut here---------------end--------------->8---

It's gobbled up half of the memory of the machine so far, and will
probably keep growing slowly.

Perhaps the machine should be reconfigured & rebooted, as its generation
appears to be from Nov 23.  It's been up for 41 days.

-- 
Thanks,
Maxim