hydra queue length

hydra queue length

[Troy Sankey]

[-- Attachment #1: Type: text/plain, Size: 983 bytes --]

I use Guix on an old thinkpad, and I've noticed that every time I run
`guix pull` followed by `guix package -u`, I need to compile many of the
packages.  That whole process could take about an hour on my laptop
(compiling the package definitions, and many of the package upgrades in
my profile).  I usually also have to close all my Firefox processes to
free up memory.  On a weekly basis, I basically need to give up my
laptop to guix for an hour.

My assumption is that this is due to the massive Hydra queue length.
Hydra is so over-worked that many of the jobs get delayed for multiple
days.  As of this writing, 2 of the 7 jobs currently running on Hydra
were queued 2 and 3 days ago.

Is that a correct assumption?  If so, should I just be waiting 2 days
between `guix pull` and `guix package -u`?  Is there some development
work lined up to reduce the queue length, or is the Guix project just in
need of more money to add machines to the build farm?

Troy

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEE0zLJ6STd4Cp+CgbIgs677ofYV8IFAlmoPZQACgkQgs677ofY
V8IDlA//R81NLbHTuLrw0ju7EkER2B2+5GgHmOJXF7mv27m1w/xkHQ0nmQdaufog
BU+LuZp1p1q/9BVA9Z3yynJiH+eqO/ORBvn0gn5WEefkvnWVAYIsjJY3Ege0CzA1
7/StV858zZZJ6Wc+/cREy5Ew0z/65m4q3n6AvjHNxC5cVOxE6psyqa5TbN95SUN2
r7WxGJTWddTcvSrFBtmwoewzkjzSkPgL7vlBTP2sGqQbpgZ2qWbeg4g1UV68W5Xe
OaCXOzQJ0ri7LNfblIksjbS8kM9IWj2sOdEu9r2QGTn0ABdH6eJYi2FvZOlupEfd
WyCloKLPURbWMPFvDw+/Uw6yh2yK8qO2mpQDgiKeLq3r9u2J38CwanSKpJKXc7pf
Pkrm8VpRvOl7J7eklUV6VH03+SVxnGnCzX2MGH43gSR/BGE/HJK0+q1ddSBF7gGK
0fYLh8EB4taucujMzcjm6btcGtJChAn2jHz5ISr0ebYpfsq2Q+/7ArPbw0ccoaLT
U68FFAhzxRpkCBbW7D3Aaj52BA5oMJnnJB5EhECDHCdLhc4cFF6V0gZEvVSFz+Dx
+2q7KC1dLShabgfTrv8ln0IJishiYYQhaAWsEtq+cqGtW2bn1PMR2Y0s/LiJwSNt
sb8tCa0eSzYGToHSBH6TV8YuAkre7mS8AIh8ZpHqje5CFukml+U=
=14AX
-----END PGP SIGNATURE-----

Re: hydra queue length

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1996 bytes --]

On Thu, 31 Aug 2017 12:47:21 -0400
Troy Sankey <sankeytms@gmail.com> wrote:

> I use Guix on an old thinkpad, and I've noticed that every time I run
> `guix pull` followed by `guix package -u`, I need to compile many of
> the packages.  That whole process could take about an hour on my
> laptop (compiling the package definitions, and many of the package
> upgrades in my profile).  I usually also have to close all my Firefox
> processes to free up memory.  On a weekly basis, I basically need to
> give up my laptop to guix for an hour.
> 
> My assumption is that this is due to the massive Hydra queue length.
> Hydra is so over-worked that many of the jobs get delayed for multiple
> days.  As of this writing, 2 of the 7 jobs currently running on Hydra
> were queued 2 and 3 days ago.
> 
> Is that a correct assumption?  If so, should I just be waiting 2 days
> between `guix pull` and `guix package -u`?  Is there some development
> work lined up to reduce the queue length, or is the Guix project just
> in need of more money to add machines to the build farm?

I think there is lots of room for improvement in this general area,
but, initially, you might want to try a few things, and see if they
improve your experience.

Firstly, try using `guix package --dry-run -u` or possibly `guix
weather` [1] if you have a manifest, as I think both of these commands
give some information about how much building guix package -u would
do, if you were to run it.

1: https://lists.gnu.org/archive/html/guix-devel/2017-07/msg00288.html

Secondly, there are 3 main substitute servers that I know of: hydra,
bayfront, and berlin. You might want to check which ones you are using,
as adding more should increase the chance that you get substitutes
rather than building from source. There is some information about the
berlin substitute servers here [2].

2: https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00128.html

Let us know how you get on :)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 963 bytes --]