unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: Mark H Weaver <mhw@netris.org>
To: Julien Lepiller <julien@lepiller.eu>
Cc: guix-devel@gnu.org
Subject: Re: Merging staging?
Date: Thu, 20 Dec 2018 20:17:07 -0500	[thread overview]
Message-ID: <8736qradap.fsf@netris.org> (raw)
In-Reply-To: <6AD6EA9A-7634-4A8E-8A59-F3D35BC0F82A@lepiller.eu> (Julien Lepiller's message of "Thu, 20 Dec 2018 23:45:00 +0100")

Hi Julien,

I've rearranged your reply from "top-posting" style to "bottom-posting"
style.  Please consider using bottom-posting in the future.

I wrote:

> Julien Lepiller <julien@lepiller.eu> writes:
>
>> I'd like to get staging merged soon, as it wasn't for quite some
>> time. Here are some stats about the current state of substitutes for
>> staging:
>>
>> According to guix weather, we have:
>>
>> | architecture | berlin | hydra |
>> +--------------+--------+-------+
>> | x86_64       | 36.5%  | 81.7% |
>> | i686         | 23.8%  | 71.0% |
>> | aarch64      | 22.2%  | 00.0% |
>> | armhf        | 17.0%  | 45.6% |
>>
>> What should the next step be?
>
> I think we should wait until the coverage on armhf and aarch64 have
> become larger, for the sake of users on those systems.
>
> Also, I've seen some commits that make me wonder if hydra is still
> being configured as an authorized substitute server on new Guix
> installations.
> Do you know?
>
> If 'berlin' is the only substitute server by default, then we certainly
> need to wait for those numbers to get higher, no?
>
> What do you think?

Julien Lepiller <julien@lepiller.eu> responded:

> I agree, but I wonder if there is a reason for these to be so low?

It's a good question.  I have several hypotheses:

* Unfortunately, it is fairly common for builds for important core
  packages to spuriously fail, often due to unreliable test suites, and
  to cause thousands of other important dependent packages to fail.
  When this happens on Hydra, I can see what's going on, and restart the
  build and all of its dependents.

  I wouldn't be surprised if some important core packages spuriously
  failed to build on Berlin, but we have no effective way to see what
  happened there.  If that's the case, the 'guix weather' numbers above
  might never get much higher no matter how long we wait.

* Berlin's build slots may have been occupied for long periods of time
  by 'test.*' jobs stuck in an endless "waiting for udevd..." loop, as
  described in <https://bugs.gnu.org/33362>.

  Hydra's web interface allows me to monitor active jobs and manually
  kill those stuck jobs when I find them.  I don't know how to do that
  on Berlin.

* Especially on armhf and aarch64, where Berlin has very little build
  capacity, and new builds are being added to Berlin's build queue must
  faster than they can be built, it is quite possible that Berlin is
  spending most of its effort on long-outdated builds.

  On Hydra, I can see when this is happening, and often intervene by
  cancelling large numbers of outdated builds on armhf, so that it
  remains focused on the most popular and up-to-date packages.

* On WIP branches like 'core-updates' and 'staging', when a new
  evaluation is done, I cancel all outdated Hydra jobs on those
  branches.  I don't know if anything similar is done on Berlin.

In summary, there are several things that I regularly do to make
efficient use of Hydra's limited build capacity.  I periodically look at
Berlin's web interface to see how it has progressed, but it is currently
mostly a black box to me.  I see no effective way to focus its limited
resources on the most important builds, or to see when build slots are
stuck.

     Regards,
       Mark

  reply	other threads:[~2018-12-21  1:18 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-12-20 14:58 Merging staging? Julien Lepiller
2018-12-20 18:36 ` Mark H Weaver
2018-12-20 22:45   ` Julien Lepiller
2018-12-21  1:17     ` Mark H Weaver [this message]
2018-12-21  9:06       ` Gábor Boskovits
2018-12-21 23:06       ` Ludovic Courtès
2018-12-23 14:00 ` Efraim Flashner
2019-01-09 16:27   ` Efraim Flashner
2019-01-09 19:15     ` Mark H Weaver
2019-01-09 19:27       ` Efraim Flashner
2019-01-10  2:27       ` Mark H Weaver
2019-01-10 22:04         ` Ludovic Courtès
2019-01-09 21:51   ` Ludovic Courtès
  -- strict thread matches above, loose matches on Subject: below --
2022-06-01  8:35 Release v1.4? zimoun
2022-06-03 16:41 ` Ludovic Courtès
2022-06-06 21:17   ` Merging ‘staging’? Ludovic Courtès
2022-06-08 11:50     ` Efraim Flashner
2022-06-08 21:24       ` Ludovic Courtès
2022-06-10  7:57         ` Efraim Flashner
2022-06-11  9:53           ` Ludovic Courtès
2022-06-11 10:49             ` Tom Fitzhenry
2022-06-12  3:58             ` Efraim Flashner
2022-06-12 21:08               ` Ludovic Courtès
2022-06-13  7:03                 ` Ludovic Courtès
2022-06-14  4:01                   ` Thiago Jung Bauermann
2022-06-09 17:19     ` pelzflorian (Florian Pelz)
2022-06-09 17:41       ` Efraim Flashner
2022-06-09 19:02         ` pelzflorian (Florian Pelz)
2022-06-10  6:07           ` pelzflorian (Florian Pelz)
2022-06-11  7:35           ` pelzflorian (Florian Pelz)
2022-06-12  4:54     ` Thiago Jung Bauermann
2022-06-12 21:06       ` Ludovic Courtès

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://guix.gnu.org/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=8736qradap.fsf@netris.org \
    --to=mhw@netris.org \
    --cc=guix-devel@gnu.org \
    --cc=julien@lepiller.eu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).