* Re: 01/01: gnu: curl: Update to 7.45.0.
[not found] ` <E1ZoZal-0001Fl-M5@vcs.savannah.gnu.org>
@ 2015-10-20 18:17 ` Mark H Weaver
2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès
0 siblings, 1 reply; 11+ messages in thread
From: Mark H Weaver @ 2015-10-20 18:17 UTC (permalink / raw
To: Andreas Enge; +Cc: guix-devel
Andreas Enge <andreas@enge.fr> writes:
> andreas pushed a commit to branch master
> in repository guix.
>
> commit 075c3ebd2dc3d8223e23025ceb5026810dfaa98d
> Author: Andreas Enge <andreas@enge.fr>
> Date: Sun Oct 18 12:20:02 2015 +0200
>
> gnu: curl: Update to 7.45.0.
To avoid unnecessary rebuilds on hydra, I reverted this commit on
master, and instead pushed it to the 'dbus-update' branch, which will
hopefully be merged soon.
Thanks,
Mark
^ permalink raw reply [flat|nested] 11+ messages in thread
* Dealing with mass rebuilds
2015-10-20 18:17 ` 01/01: gnu: curl: Update to 7.45.0 Mark H Weaver
@ 2015-10-20 19:45 ` Ludovic Courtès
2015-10-20 19:56 ` Efraim Flashner
` (2 more replies)
0 siblings, 3 replies; 11+ messages in thread
From: Ludovic Courtès @ 2015-10-20 19:45 UTC (permalink / raw
To: Mark H Weaver; +Cc: guix-devel
Mark H Weaver <mhw@netris.org> skribis:
> Andreas Enge <andreas@enge.fr> writes:
>
>> andreas pushed a commit to branch master
>> in repository guix.
>>
>> commit 075c3ebd2dc3d8223e23025ceb5026810dfaa98d
>> Author: Andreas Enge <andreas@enge.fr>
>> Date: Sun Oct 18 12:20:02 2015 +0200
>>
>> gnu: curl: Update to 7.45.0.
>
> To avoid unnecessary rebuilds on hydra, I reverted this commit on
> master, and instead pushed it to the 'dbus-update' branch, which will
> hopefully be merged soon.
We’re talking of ~150 dependent packages, which to me sounded OK¹.
If we delay obvious changes like this too much, the risk is to end up
with upgrade-everything branches that take ages to get merged (what is
happening with ‘dbus-update’.)
The Nixpkgs folks have a ‘staging’ branch for changes that cause mass
rebuilds but are well tested, such that they can be merged into ‘master’
anytime². Perhaps something we should do as well?
Ludo’.
¹ https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00653.html
² http://comments.gmane.org/gmane.linux.distributions.nixos/13447
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds
2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès
@ 2015-10-20 19:56 ` Efraim Flashner
2015-10-20 21:47 ` Andreas Enge
2015-10-22 18:20 ` Mark H Weaver
2 siblings, 0 replies; 11+ messages in thread
From: Efraim Flashner @ 2015-10-20 19:56 UTC (permalink / raw
To: Ludovic Courtès; +Cc: guix-devel
[-- Attachment #1: Type: text/plain, Size: 1683 bytes --]
On Tue, 20 Oct 2015 21:45:54 +0200
ludo@gnu.org (Ludovic Courtès) wrote:
> Mark H Weaver <mhw@netris.org> skribis:
>
> > Andreas Enge <andreas@enge.fr> writes:
> >
> >> andreas pushed a commit to branch master
> >> in repository guix.
> >>
> >> commit 075c3ebd2dc3d8223e23025ceb5026810dfaa98d
> >> Author: Andreas Enge <andreas@enge.fr>
> >> Date: Sun Oct 18 12:20:02 2015 +0200
> >>
> >> gnu: curl: Update to 7.45.0.
> >
> > To avoid unnecessary rebuilds on hydra, I reverted this commit on
> > master, and instead pushed it to the 'dbus-update' branch, which will
> > hopefully be merged soon.
>
> We’re talking of ~150 dependent packages, which to me sounded OK¹.
>
> If we delay obvious changes like this too much, the risk is to end up
> with upgrade-everything branches that take ages to get merged (what is
> happening with ‘dbus-update’.)
>
> The Nixpkgs folks have a ‘staging’ branch for changes that cause mass
> rebuilds but are well tested, such that they can be merged into ‘master’
> anytime². Perhaps something we should do as well?
>
> Ludo’.
>
> ¹ https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00653.html
> ² http://comments.gmane.org/gmane.linux.distributions.nixos/13447
>
It would make an obvious place to push an sqlite update and allow the
dbus-update update branch to focus just on the dbus-update, or a core-update
branch to focus just on core updates.
--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds
2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès
2015-10-20 19:56 ` Efraim Flashner
@ 2015-10-20 21:47 ` Andreas Enge
2015-10-21 7:27 ` Efraim Flashner
2015-10-21 16:45 ` Ludovic Courtès
2015-10-22 18:20 ` Mark H Weaver
2 siblings, 2 replies; 11+ messages in thread
From: Andreas Enge @ 2015-10-20 21:47 UTC (permalink / raw
To: Ludovic Courtès; +Cc: guix-devel
On Tue, Oct 20, 2015 at 09:45:54PM +0200, Ludovic Courtès wrote:
> The Nixpkgs folks have a ‘staging’ branch for changes that cause mass
> rebuilds but are well tested, such that they can be merged into ‘master’
> anytime². Perhaps something we should do as well?
Would that be different from your suggestion that if the curl update caused
a rebuild of too many packages (whatever that would mean, which is a separate
discussion) it should not be done on core-updates, but on its own branch,
for instance, curl-update? One practical advantage I see in a staging
branch, if I understand your suggestion correctly, is that one would not need
to modify the set of jobs on hydra to now also build curl-update, and then
maybe giflib-update, and then xyz-update, but it would always be called
"staging".
Now the "well tested" assumption is dubious. I actually do not know whether
the curl update will go through. I tried to build one of the packages shown
by "guix refresh -l curl" that looked simple (mpd), but it turned out it
was not simple at all and needed a lot of rebuilds (including texlive).
Admittedly, I could have spent more time searching a direct dependency,
probably by using one of your recent graph drawing commands. So in case
a problem turns up, would we revert a curl update on the staging branch
and do more local work before proposing it again?
And if Efraim wished to do a gitlib update after my curl update, what would
be the protocol? Would he be expected to wait until the hydra build of the
curl update has finished? Or at least until it has finished on x86_64?
Otherwise, we would again face the risk of the staging branch becoming
an update-the-world branch with unforeseen ramifications.
Andreas
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds
2015-10-20 21:47 ` Andreas Enge
@ 2015-10-21 7:27 ` Efraim Flashner
2015-10-21 16:41 ` Ludovic Courtès
2015-10-21 16:45 ` Ludovic Courtès
1 sibling, 1 reply; 11+ messages in thread
From: Efraim Flashner @ 2015-10-21 7:27 UTC (permalink / raw
To: Andreas Enge; +Cc: guix-devel
[-- Attachment #1: Type: text/plain, Size: 2483 bytes --]
On Tue, 20 Oct 2015 23:47:16 +0200
Andreas Enge <andreas@enge.fr> wrote:
> On Tue, Oct 20, 2015 at 09:45:54PM +0200, Ludovic Courtès wrote:
> > The Nixpkgs folks have a ‘staging’ branch for changes that cause mass
> > rebuilds but are well tested, such that they can be merged into ‘master’
> > anytime². Perhaps something we should do as well?
>
> Would that be different from your suggestion that if the curl update caused
> a rebuild of too many packages (whatever that would mean, which is a separate
> discussion) it should not be done on core-updates, but on its own branch,
> for instance, curl-update? One practical advantage I see in a staging
> branch, if I understand your suggestion correctly, is that one would not need
> to modify the set of jobs on hydra to now also build curl-update, and then
> maybe giflib-update, and then xyz-update, but it would always be called
> "staging".
>
> Now the "well tested" assumption is dubious. I actually do not know whether
> the curl update will go through. I tried to build one of the packages shown
> by "guix refresh -l curl" that looked simple (mpd), but it turned out it
> was not simple at all and needed a lot of rebuilds (including texlive).
> Admittedly, I could have spent more time searching a direct dependency,
> probably by using one of your recent graph drawing commands. So in case
> a problem turns up, would we revert a curl update on the staging branch
> and do more local work before proposing it again?
>
> And if Efraim wished to do a gitlib update after my curl update, what would
> be the protocol? Would he be expected to wait until the hydra build of the
> curl update has finished? Or at least until it has finished on x86_64?
> Otherwise, we would again face the risk of the staging branch becoming
> an update-the-world branch with unforeseen ramifications.
>
> Andreas
>
>
How much build infrastructure does nix have? (do they build all of staging
all the time?) Would it be possible to identify the subset of packages that
are immediately dependant on the updated package, and build only those as a
test on hydra? That would be faster, but if something broke it'd be harder to
track down, a la 白い熊's conkeror issue.
--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds
2015-10-21 7:27 ` Efraim Flashner
@ 2015-10-21 16:41 ` Ludovic Courtès
0 siblings, 0 replies; 11+ messages in thread
From: Ludovic Courtès @ 2015-10-21 16:41 UTC (permalink / raw
To: Efraim Flashner; +Cc: guix-devel
Efraim Flashner <efraim@flashner.co.il> skribis:
> How much build infrastructure does nix have? (do they build all of staging
> all the time?) Would it be possible to identify the subset of packages that
> are immediately dependant on the updated package, and build only those as a
> test on hydra?
‘guix refresh -l’ provides that information. Hydra itself always tries
to build everything though.
> That would be faster, but if something broke it'd be harder to
> track down, a la 白い熊's conkeror issue.
The Conkeror issue is a run-time issue though, one that is not revealed
by its test suite apparently.
Ludo’.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds
2015-10-20 21:47 ` Andreas Enge
2015-10-21 7:27 ` Efraim Flashner
@ 2015-10-21 16:45 ` Ludovic Courtès
2015-10-22 20:11 ` Paul van der Walt
1 sibling, 1 reply; 11+ messages in thread
From: Ludovic Courtès @ 2015-10-21 16:45 UTC (permalink / raw
To: Andreas Enge; +Cc: guix-devel
Andreas Enge <andreas@enge.fr> skribis:
> On Tue, Oct 20, 2015 at 09:45:54PM +0200, Ludovic Courtès wrote:
>> The Nixpkgs folks have a ‘staging’ branch for changes that cause mass
>> rebuilds but are well tested, such that they can be merged into ‘master’
>> anytime². Perhaps something we should do as well?
>
> Would that be different from your suggestion that if the curl update caused
> a rebuild of too many packages (whatever that would mean, which is a separate
> discussion) it should not be done on core-updates, but on its own branch,
> for instance, curl-update? One practical advantage I see in a staging
> branch, if I understand your suggestion correctly, is that one would not need
> to modify the set of jobs on hydra to now also build curl-update, and then
> maybe giflib-update, and then xyz-update, but it would always be called
> "staging".
Yes.
> Now the "well tested" assumption is dubious. I actually do not know whether
> the curl update will go through. I tried to build one of the packages shown
> by "guix refresh -l curl" that looked simple (mpd), but it turned out it
> was not simple at all and needed a lot of rebuilds (including texlive).
> Admittedly, I could have spent more time searching a direct dependency,
> probably by using one of your recent graph drawing commands. So in case
> a problem turns up, would we revert a curl update on the staging branch
> and do more local work before proposing it again?
Ideally, we’d test as much as possible locally, or, for important
library updates, in a dedicated branch.
The TeX Live issue is orthogonal: We seriously need a ‘texlive-minimal’
variant, and the full-blown ‘texlive’ must not be depended on by any
other package.
> And if Efraim wished to do a gitlib update after my curl update, what would
> be the protocol? Would he be expected to wait until the hydra build of the
> curl update has finished? Or at least until it has finished on x86_64?
> Otherwise, we would again face the risk of the staging branch becoming
> an update-the-world branch with unforeseen ramifications.
Yeah, for sure. I have to admit that we may be unable to do much better
until we can finally change the hydra.gnu.org front-end, which is the
main bottleneck here (at least for i686/x86_64.)
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds
2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès
2015-10-20 19:56 ` Efraim Flashner
2015-10-20 21:47 ` Andreas Enge
@ 2015-10-22 18:20 ` Mark H Weaver
2015-10-22 18:32 ` Andreas Enge
2 siblings, 1 reply; 11+ messages in thread
From: Mark H Weaver @ 2015-10-22 18:20 UTC (permalink / raw
To: Ludovic Courtès; +Cc: guix-devel
ludo@gnu.org (Ludovic Courtès) writes:
> Mark H Weaver <mhw@netris.org> skribis:
>
>> Andreas Enge <andreas@enge.fr> writes:
>>
>>> andreas pushed a commit to branch master
>>> in repository guix.
>>>
>>> commit 075c3ebd2dc3d8223e23025ceb5026810dfaa98d
>>> Author: Andreas Enge <andreas@enge.fr>
>>> Date: Sun Oct 18 12:20:02 2015 +0200
>>>
>>> gnu: curl: Update to 7.45.0.
>>
>> To avoid unnecessary rebuilds on hydra, I reverted this commit on
>> master, and instead pushed it to the 'dbus-update' branch, which will
>> hopefully be merged soon.
>
> We’re talking of ~150 dependent packages, which to me sounded OK¹.
After applying the 'giflib' and 'curl' updates to the 'dbus-update'
branch (and also merging master), the number of queued jobs on that
jobset went from about 1000 to over 2200. Based on this, and also based
on my past memories of curl security updates, I suspect that this is
another case where "guix refresh -l" drastically underestimated the
number of rebuilds.
Mark
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds
2015-10-22 18:20 ` Mark H Weaver
@ 2015-10-22 18:32 ` Andreas Enge
0 siblings, 0 replies; 11+ messages in thread
From: Andreas Enge @ 2015-10-22 18:32 UTC (permalink / raw
To: Mark H Weaver; +Cc: guix-devel
On Thu, Oct 22, 2015 at 02:20:08PM -0400, Mark H Weaver wrote:
> After applying the 'giflib' and 'curl' updates to the 'dbus-update'
> branch (and also merging master), the number of queued jobs on that
> jobset went from about 1000 to over 2200. Based on this, and also based
This is actually coherent with the numbers given by "guix refresh -l":
143 for curl, 131 for giflib (there may be some overlap, but probably not
too much); and then we need to multiply it by 4 now for our multiple
architectures!
Andreas
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds
2015-10-21 16:45 ` Ludovic Courtès
@ 2015-10-22 20:11 ` Paul van der Walt
2015-10-25 21:34 ` Ludovic Courtès
0 siblings, 1 reply; 11+ messages in thread
From: Paul van der Walt @ 2015-10-22 20:11 UTC (permalink / raw
To: Ludovic Courtès; +Cc: guix-devel
On 2015-10-21 at 18:45, quoth Ludovic Courtès:
> The TeX Live issue is orthogonal: We seriously need a ‘texlive-minimal’
> variant, and the full-blown ‘texlive’ must not be depended on by any
> other package.
Even more orthogonal: can packages be marked as leaves (or roots,
depending on your point of view) that may not be used as inputs? If not
that sounds like a potentially useful bit of package metadata.
Cheers,
p.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Dealing with mass rebuilds
2015-10-22 20:11 ` Paul van der Walt
@ 2015-10-25 21:34 ` Ludovic Courtès
0 siblings, 0 replies; 11+ messages in thread
From: Ludovic Courtès @ 2015-10-25 21:34 UTC (permalink / raw
To: Paul van der Walt; +Cc: guix-devel
Paul van der Walt <paul@denknerd.org> skribis:
> On 2015-10-21 at 18:45, quoth Ludovic Courtès:
>> The TeX Live issue is orthogonal: We seriously need a ‘texlive-minimal’
>> variant, and the full-blown ‘texlive’ must not be depended on by any
>> other package.
>
> Even more orthogonal: can packages be marked as leaves (or roots,
> depending on your point of view) that may not be used as inputs? If not
> that sounds like a potentially useful bit of package metadata.
We could do that, but I’m not sure it’s useful, apart from this very
particular case. Do you have other uses in mind?
Ludo’.
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-10-25 21:34 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20151020161651.4759.8347@vcs.savannah.gnu.org>
[not found] ` <E1ZoZal-0001Fl-M5@vcs.savannah.gnu.org>
2015-10-20 18:17 ` 01/01: gnu: curl: Update to 7.45.0 Mark H Weaver
2015-10-20 19:45 ` Dealing with mass rebuilds Ludovic Courtès
2015-10-20 19:56 ` Efraim Flashner
2015-10-20 21:47 ` Andreas Enge
2015-10-21 7:27 ` Efraim Flashner
2015-10-21 16:41 ` Ludovic Courtès
2015-10-21 16:45 ` Ludovic Courtès
2015-10-22 20:11 ` Paul van der Walt
2015-10-25 21:34 ` Ludovic Courtès
2015-10-22 18:20 ` Mark H Weaver
2015-10-22 18:32 ` Andreas Enge
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.