We started building the wip-ungrafting branch a week ago: https://ci.guix.gnu.org/jobset/ungrafting Some progress has been made, but not enough. This is partially due to <https://bugs.gnu.org/48574>, and also due to inefficient building of expensive bootstrap paths (namely, every Rust package derivation tried to build the entire Rust dependency graph concurrently). I just rebased the branch and pushed again.
On Tue, May 25, 2021 at 04:04:22PM -0400, Leo Famulari wrote: > I just rebased the branch and pushed again. The latest iteration failed en masse due to network timeouts between build farm nodes. Now that Cuirass is aware of dependencies, we have useful info as shown here: https://ci.guix.gnu.org/build/515005/details It lets us see that the Audacity build is not attempted due to the failure of wxwidgets. And in turn, the failure of webkitgtk and libsoup. However, there is no information (logs, duration) available about those failed builds. I successfully built the Audacity package on the wip-ungrafting branch "by hand" on the build farm. What can we do to clear all these "spurious" build failures and re-try building the branch? I already tried using the "restart all builds" button, but there were still a lot of spurious failures.
Hello Leo, I monitored a previous evaluation of the ungrafting Cuirass specification that was more successful, with more than 17000 builds performed in less than 24 hours, a new record! The recent evaluation is sadly less victorious. > The latest iteration failed en masse due to network timeouts between > build farm nodes. Yes, I described this issue here: https://issues.guix.gnu.org/48468 and hopefully fixed it with https://issues.guix.gnu.org/48556. I need to perform a reconfigure / deploy but I'm waiting to have a few days ahead of me to fix potential issues. > It lets us see that the Audacity build is not attempted due to the > failure of wxwidgets. And in turn, the failure of webkitgtk and libsoup. > However, there is no information (logs, duration) available about those > failed builds. When a build is cancelled ("Cancel pending builds" button), all its dependents are automatically updated to "Failed dependency". It looks like this is the case for https://ci.guix.gnu.org/build/515005/details. > What can we do to clear all these "spurious" build failures and re-try > building the branch? I already tried using the "restart all builds" > button, but there were still a lot of spurious failures. I think the easiest way to recover from this situation is to remove the ungrafting specification and recreate it. It ensures that no build will fail because of previously failing/cancelled build dependencies. Note that when a build B is restarted in Cuirass ("Restart" button), any other build B' depending from B, will automatically be restarted if B becomes successful and all the other dependencies of B' are already successful. I used this mechanism to restart a few builds failing on the master branch because of substitutes timeout. I hope that when the keep-alive patchset will solve this situation. Thanks, Mathieu
On Wed, Jun 02, 2021 at 07:53:35PM +0200, Mathieu Othacehe wrote: > I monitored a previous evaluation of the ungrafting Cuirass > specification that was more successful, with more than 17000 builds > performed in less than 24 hours, a new record! The recent evaluation is > sadly less victorious. Yeah, it was great for a while! > I need to perform a reconfigure / deploy but I'm waiting to have a few > days ahead of me to fix potential issues. Okay. Can you let me know when you do? I'll delay the ungrafting builds until then.
Hey Leo,
> Okay. Can you let me know when you do? I'll delay the ungrafting builds
> until then.
I deployed the upgrade yesterday. I've been monitoring it for 12 hours
without any sign of substitutes timeout. I think you can proceed whenever
you want.
Thanks,
Mathieu