* Re: 01/01: gnu: curl: Update to 7.45.0. [not found] ` <E1ZoZal-0001Fl-M5@vcs.savannah.gnu.org> @ 2015-10-20 18:17 ` Mark H Weaver 2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès 0 siblings, 1 reply; 11+ messages in thread From: Mark H Weaver @ 2015-10-20 18:17 UTC (permalink / raw To: Andreas Enge; +Cc: guix-devel Andreas Enge <andreas@enge.fr> writes: > andreas pushed a commit to branch master > in repository guix. > > commit 075c3ebd2dc3d8223e23025ceb5026810dfaa98d > Author: Andreas Enge <andreas@enge.fr> > Date: Sun Oct 18 12:20:02 2015 +0200 > > gnu: curl: Update to 7.45.0. To avoid unnecessary rebuilds on hydra, I reverted this commit on master, and instead pushed it to the 'dbus-update' branch, which will hopefully be merged soon. Thanks, Mark ^ permalink raw reply [flat|nested] 11+ messages in thread
* Dealing with mass rebuilds 2015-10-20 18:17 ` 01/01: gnu: curl: Update to 7.45.0 Mark H Weaver @ 2015-10-20 19:45 ` Ludovic Courtès 2015-10-20 19:56 ` Efraim Flashner ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: Ludovic Courtès @ 2015-10-20 19:45 UTC (permalink / raw To: Mark H Weaver; +Cc: guix-devel Mark H Weaver <mhw@netris.org> skribis: > Andreas Enge <andreas@enge.fr> writes: > >> andreas pushed a commit to branch master >> in repository guix. >> >> commit 075c3ebd2dc3d8223e23025ceb5026810dfaa98d >> Author: Andreas Enge <andreas@enge.fr> >> Date: Sun Oct 18 12:20:02 2015 +0200 >> >> gnu: curl: Update to 7.45.0. > > To avoid unnecessary rebuilds on hydra, I reverted this commit on > master, and instead pushed it to the 'dbus-update' branch, which will > hopefully be merged soon. We’re talking of ~150 dependent packages, which to me sounded OK¹. If we delay obvious changes like this too much, the risk is to end up with upgrade-everything branches that take ages to get merged (what is happening with ‘dbus-update’.) The Nixpkgs folks have a ‘staging’ branch for changes that cause mass rebuilds but are well tested, such that they can be merged into ‘master’ anytime². Perhaps something we should do as well? Ludo’. ¹ https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00653.html ² http://comments.gmane.org/gmane.linux.distributions.nixos/13447 ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds 2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès @ 2015-10-20 19:56 ` Efraim Flashner 2015-10-20 21:47 ` Andreas Enge 2015-10-22 18:20 ` Mark H Weaver 2 siblings, 0 replies; 11+ messages in thread From: Efraim Flashner @ 2015-10-20 19:56 UTC (permalink / raw To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1683 bytes --] On Tue, 20 Oct 2015 21:45:54 +0200 ludo@gnu.org (Ludovic Courtès) wrote: > Mark H Weaver <mhw@netris.org> skribis: > > > Andreas Enge <andreas@enge.fr> writes: > > > >> andreas pushed a commit to branch master > >> in repository guix. > >> > >> commit 075c3ebd2dc3d8223e23025ceb5026810dfaa98d > >> Author: Andreas Enge <andreas@enge.fr> > >> Date: Sun Oct 18 12:20:02 2015 +0200 > >> > >> gnu: curl: Update to 7.45.0. > > > > To avoid unnecessary rebuilds on hydra, I reverted this commit on > > master, and instead pushed it to the 'dbus-update' branch, which will > > hopefully be merged soon. > > We’re talking of ~150 dependent packages, which to me sounded OK¹. > > If we delay obvious changes like this too much, the risk is to end up > with upgrade-everything branches that take ages to get merged (what is > happening with ‘dbus-update’.) > > The Nixpkgs folks have a ‘staging’ branch for changes that cause mass > rebuilds but are well tested, such that they can be merged into ‘master’ > anytime². Perhaps something we should do as well? > > Ludo’. > > ¹ https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00653.html > ² http://comments.gmane.org/gmane.linux.distributions.nixos/13447 > It would make an obvious place to push an sqlite update and allow the dbus-update update branch to focus just on the dbus-update, or a core-update branch to focus just on core updates. -- Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds 2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès 2015-10-20 19:56 ` Efraim Flashner @ 2015-10-20 21:47 ` Andreas Enge 2015-10-21 7:27 ` Efraim Flashner 2015-10-21 16:45 ` Ludovic Courtès 2015-10-22 18:20 ` Mark H Weaver 2 siblings, 2 replies; 11+ messages in thread From: Andreas Enge @ 2015-10-20 21:47 UTC (permalink / raw To: Ludovic Courtès; +Cc: guix-devel On Tue, Oct 20, 2015 at 09:45:54PM +0200, Ludovic Courtès wrote: > The Nixpkgs folks have a ‘staging’ branch for changes that cause mass > rebuilds but are well tested, such that they can be merged into ‘master’ > anytime². Perhaps something we should do as well? Would that be different from your suggestion that if the curl update caused a rebuild of too many packages (whatever that would mean, which is a separate discussion) it should not be done on core-updates, but on its own branch, for instance, curl-update? One practical advantage I see in a staging branch, if I understand your suggestion correctly, is that one would not need to modify the set of jobs on hydra to now also build curl-update, and then maybe giflib-update, and then xyz-update, but it would always be called "staging". Now the "well tested" assumption is dubious. I actually do not know whether the curl update will go through. I tried to build one of the packages shown by "guix refresh -l curl" that looked simple (mpd), but it turned out it was not simple at all and needed a lot of rebuilds (including texlive). Admittedly, I could have spent more time searching a direct dependency, probably by using one of your recent graph drawing commands. So in case a problem turns up, would we revert a curl update on the staging branch and do more local work before proposing it again? And if Efraim wished to do a gitlib update after my curl update, what would be the protocol? Would he be expected to wait until the hydra build of the curl update has finished? Or at least until it has finished on x86_64? Otherwise, we would again face the risk of the staging branch becoming an update-the-world branch with unforeseen ramifications. Andreas ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds 2015-10-20 21:47 ` Andreas Enge @ 2015-10-21 7:27 ` Efraim Flashner 2015-10-21 16:41 ` Ludovic Courtès 2015-10-21 16:45 ` Ludovic Courtès 1 sibling, 1 reply; 11+ messages in thread From: Efraim Flashner @ 2015-10-21 7:27 UTC (permalink / raw To: Andreas Enge; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 2483 bytes --] On Tue, 20 Oct 2015 23:47:16 +0200 Andreas Enge <andreas@enge.fr> wrote: > On Tue, Oct 20, 2015 at 09:45:54PM +0200, Ludovic Courtès wrote: > > The Nixpkgs folks have a ‘staging’ branch for changes that cause mass > > rebuilds but are well tested, such that they can be merged into ‘master’ > > anytime². Perhaps something we should do as well? > > Would that be different from your suggestion that if the curl update caused > a rebuild of too many packages (whatever that would mean, which is a separate > discussion) it should not be done on core-updates, but on its own branch, > for instance, curl-update? One practical advantage I see in a staging > branch, if I understand your suggestion correctly, is that one would not need > to modify the set of jobs on hydra to now also build curl-update, and then > maybe giflib-update, and then xyz-update, but it would always be called > "staging". > > Now the "well tested" assumption is dubious. I actually do not know whether > the curl update will go through. I tried to build one of the packages shown > by "guix refresh -l curl" that looked simple (mpd), but it turned out it > was not simple at all and needed a lot of rebuilds (including texlive). > Admittedly, I could have spent more time searching a direct dependency, > probably by using one of your recent graph drawing commands. So in case > a problem turns up, would we revert a curl update on the staging branch > and do more local work before proposing it again? > > And if Efraim wished to do a gitlib update after my curl update, what would > be the protocol? Would he be expected to wait until the hydra build of the > curl update has finished? Or at least until it has finished on x86_64? > Otherwise, we would again face the risk of the staging branch becoming > an update-the-world branch with unforeseen ramifications. > > Andreas > > How much build infrastructure does nix have? (do they build all of staging all the time?) Would it be possible to identify the subset of packages that are immediately dependant on the updated package, and build only those as a test on hydra? That would be faster, but if something broke it'd be harder to track down, a la 白い熊's conkeror issue. -- Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds 2015-10-21 7:27 ` Efraim Flashner @ 2015-10-21 16:41 ` Ludovic Courtès 0 siblings, 0 replies; 11+ messages in thread From: Ludovic Courtès @ 2015-10-21 16:41 UTC (permalink / raw To: Efraim Flashner; +Cc: guix-devel Efraim Flashner <efraim@flashner.co.il> skribis: > How much build infrastructure does nix have? (do they build all of staging > all the time?) Would it be possible to identify the subset of packages that > are immediately dependant on the updated package, and build only those as a > test on hydra? ‘guix refresh -l’ provides that information. Hydra itself always tries to build everything though. > That would be faster, but if something broke it'd be harder to > track down, a la 白い熊's conkeror issue. The Conkeror issue is a run-time issue though, one that is not revealed by its test suite apparently. Ludo’. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds 2015-10-20 21:47 ` Andreas Enge 2015-10-21 7:27 ` Efraim Flashner @ 2015-10-21 16:45 ` Ludovic Courtès 2015-10-22 20:11 ` Paul van der Walt 1 sibling, 1 reply; 11+ messages in thread From: Ludovic Courtès @ 2015-10-21 16:45 UTC (permalink / raw To: Andreas Enge; +Cc: guix-devel Andreas Enge <andreas@enge.fr> skribis: > On Tue, Oct 20, 2015 at 09:45:54PM +0200, Ludovic Courtès wrote: >> The Nixpkgs folks have a ‘staging’ branch for changes that cause mass >> rebuilds but are well tested, such that they can be merged into ‘master’ >> anytime². Perhaps something we should do as well? > > Would that be different from your suggestion that if the curl update caused > a rebuild of too many packages (whatever that would mean, which is a separate > discussion) it should not be done on core-updates, but on its own branch, > for instance, curl-update? One practical advantage I see in a staging > branch, if I understand your suggestion correctly, is that one would not need > to modify the set of jobs on hydra to now also build curl-update, and then > maybe giflib-update, and then xyz-update, but it would always be called > "staging". Yes. > Now the "well tested" assumption is dubious. I actually do not know whether > the curl update will go through. I tried to build one of the packages shown > by "guix refresh -l curl" that looked simple (mpd), but it turned out it > was not simple at all and needed a lot of rebuilds (including texlive). > Admittedly, I could have spent more time searching a direct dependency, > probably by using one of your recent graph drawing commands. So in case > a problem turns up, would we revert a curl update on the staging branch > and do more local work before proposing it again? Ideally, we’d test as much as possible locally, or, for important library updates, in a dedicated branch. The TeX Live issue is orthogonal: We seriously need a ‘texlive-minimal’ variant, and the full-blown ‘texlive’ must not be depended on by any other package. > And if Efraim wished to do a gitlib update after my curl update, what would > be the protocol? Would he be expected to wait until the hydra build of the > curl update has finished? Or at least until it has finished on x86_64? > Otherwise, we would again face the risk of the staging branch becoming > an update-the-world branch with unforeseen ramifications. Yeah, for sure. I have to admit that we may be unable to do much better until we can finally change the hydra.gnu.org front-end, which is the main bottleneck here (at least for i686/x86_64.) Thanks, Ludo’. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds 2015-10-21 16:45 ` Ludovic Courtès @ 2015-10-22 20:11 ` Paul van der Walt 2015-10-25 21:34 ` Ludovic Courtès 0 siblings, 1 reply; 11+ messages in thread From: Paul van der Walt @ 2015-10-22 20:11 UTC (permalink / raw To: Ludovic Courtès; +Cc: guix-devel On 2015-10-21 at 18:45, quoth Ludovic Courtès: > The TeX Live issue is orthogonal: We seriously need a ‘texlive-minimal’ > variant, and the full-blown ‘texlive’ must not be depended on by any > other package. Even more orthogonal: can packages be marked as leaves (or roots, depending on your point of view) that may not be used as inputs? If not that sounds like a potentially useful bit of package metadata. Cheers, p. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds 2015-10-22 20:11 ` Paul van der Walt @ 2015-10-25 21:34 ` Ludovic Courtès 0 siblings, 0 replies; 11+ messages in thread From: Ludovic Courtès @ 2015-10-25 21:34 UTC (permalink / raw To: Paul van der Walt; +Cc: guix-devel Paul van der Walt <paul@denknerd.org> skribis: > On 2015-10-21 at 18:45, quoth Ludovic Courtès: >> The TeX Live issue is orthogonal: We seriously need a ‘texlive-minimal’ >> variant, and the full-blown ‘texlive’ must not be depended on by any >> other package. > > Even more orthogonal: can packages be marked as leaves (or roots, > depending on your point of view) that may not be used as inputs? If not > that sounds like a potentially useful bit of package metadata. We could do that, but I’m not sure it’s useful, apart from this very particular case. Do you have other uses in mind? Ludo’. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds 2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès 2015-10-20 19:56 ` Efraim Flashner 2015-10-20 21:47 ` Andreas Enge @ 2015-10-22 18:20 ` Mark H Weaver 2015-10-22 18:32 ` Andreas Enge 2 siblings, 1 reply; 11+ messages in thread From: Mark H Weaver @ 2015-10-22 18:20 UTC (permalink / raw To: Ludovic Courtès; +Cc: guix-devel ludo@gnu.org (Ludovic Courtès) writes: > Mark H Weaver <mhw@netris.org> skribis: > >> Andreas Enge <andreas@enge.fr> writes: >> >>> andreas pushed a commit to branch master >>> in repository guix. >>> >>> commit 075c3ebd2dc3d8223e23025ceb5026810dfaa98d >>> Author: Andreas Enge <andreas@enge.fr> >>> Date: Sun Oct 18 12:20:02 2015 +0200 >>> >>> gnu: curl: Update to 7.45.0. >> >> To avoid unnecessary rebuilds on hydra, I reverted this commit on >> master, and instead pushed it to the 'dbus-update' branch, which will >> hopefully be merged soon. > > We’re talking of ~150 dependent packages, which to me sounded OK¹. After applying the 'giflib' and 'curl' updates to the 'dbus-update' branch (and also merging master), the number of queued jobs on that jobset went from about 1000 to over 2200. Based on this, and also based on my past memories of curl security updates, I suspect that this is another case where "guix refresh -l" drastically underestimated the number of rebuilds. Mark ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds 2015-10-22 18:20 ` Mark H Weaver @ 2015-10-22 18:32 ` Andreas Enge 0 siblings, 0 replies; 11+ messages in thread From: Andreas Enge @ 2015-10-22 18:32 UTC (permalink / raw To: Mark H Weaver; +Cc: guix-devel On Thu, Oct 22, 2015 at 02:20:08PM -0400, Mark H Weaver wrote: > After applying the 'giflib' and 'curl' updates to the 'dbus-update' > branch (and also merging master), the number of queued jobs on that > jobset went from about 1000 to over 2200. Based on this, and also based This is actually coherent with the numbers given by "guix refresh -l": 143 for curl, 131 for giflib (there may be some overlap, but probably not too much); and then we need to multiply it by 4 now for our multiple architectures! Andreas ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-10-25 21:34 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- [not found] <20151020161651.4759.8347@vcs.savannah.gnu.org> [not found] ` <E1ZoZal-0001Fl-M5@vcs.savannah.gnu.org> 2015-10-20 18:17 ` 01/01: gnu: curl: Update to 7.45.0 Mark H Weaver 2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès 2015-10-20 19:56 ` Efraim Flashner 2015-10-20 21:47 ` Andreas Enge 2015-10-21 7:27 ` Efraim Flashner 2015-10-21 16:41 ` Ludovic Courtès 2015-10-21 16:45 ` Ludovic Courtès 2015-10-22 20:11 ` Paul van der Walt 2015-10-25 21:34 ` Ludovic Courtès 2015-10-22 18:20 ` Mark H Weaver 2015-10-22 18:32 ` Andreas Enge
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.