* substitute derivation: also substitute grafts? @ 2022-09-15 12:58 Ricardo Wurmus 2022-09-15 14:46 ` Csepp 0 siblings, 1 reply; 19+ messages in thread From: Ricardo Wurmus @ 2022-09-15 12:58 UTC (permalink / raw) To: Guix Devel Hi Guix, I really like the fact that Guix can substitute derivations. It’s a great way to transfer a profile from one powerful machine to a weak machine without having to set up SSH for “guix copy” first. I just build a profile on machine A, and then on the weak machine B I do guix build --substitute-urls="http://A:8000" /gnu/store/…-profile.drv and all items for that profile are copied to B. Did I say *all items*? Well, … grafts are not included, because graft derivations are marked as not substitutable. Can we change that conditionally? I would really like to avoid having to build grafts on B when they have already been built on A. -- Ricardo PS: please keep me in Cc as I’m not subscribed to guix-devel at the moment. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-09-15 12:58 substitute derivation: also substitute grafts? Ricardo Wurmus @ 2022-09-15 14:46 ` Csepp 2022-09-15 15:06 ` Maxime Devos 0 siblings, 1 reply; 19+ messages in thread From: Csepp @ 2022-09-15 14:46 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: guix-devel Ricardo Wurmus <rekado@elephly.net> writes: > Hi Guix, > > I really like the fact that Guix can substitute derivations. It’s a > great way to transfer a profile from one powerful machine to a weak > machine without having to set up SSH for “guix copy” first. > > I just build a profile on machine A, and then on the weak machine B I > do > > guix build --substitute-urls="http://A:8000" > /gnu/store/…-profile.drv > > and all items for that profile are copied to B. > > Did I say *all items*? Well, … grafts are not included, because graft > derivations are marked as not substitutable. > > Can we change that conditionally? I would really like to avoid having > to build grafts on B when they have already been built on A. I would love this too, because IO can be incredibly slow on HDDs and large packages. My netbook would be thankful. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-09-15 14:46 ` Csepp @ 2022-09-15 15:06 ` Maxime Devos 2022-09-15 17:43 ` Csepp 2022-09-19 22:00 ` Ricardo Wurmus 0 siblings, 2 replies; 19+ messages in thread From: Maxime Devos @ 2022-09-15 15:06 UTC (permalink / raw) To: Csepp, Ricardo Wurmus; +Cc: guix-devel [-- Attachment #1.1.1: Type: text/plain, Size: 1524 bytes --] On 15-09-2022 16:46, Csepp wrote: > > Ricardo Wurmus <rekado@elephly.net> writes: > >> [...] >> Did I say *all items*? Well, … grafts are not included, because graft >> derivations are marked as not substitutable. >> >> Can we change that conditionally? I would really like to avoid having >> to build grafts on B when they have already been built on A. > > I would love this too, because IO can be incredibly slow on HDDs and > large packages. My netbook would be thankful. > There are some some opportunities for optimizations in the grafting code before substituting more -- for example, to avoid seek times, it would be possible to rewrite multiple files concurrently (maybe using 'par-for-each' to process each file in a directory in parallel). This is already done in (guix build grafts), except that: * 'find-files' itself is not parallelised, even though parallelising it could potentially reduce the time spent seeking (*) * it uses (parallel-job-count), which is (IIUC) 1 by default because --cores=1 by default. As grafts are more IO intensive than CPU intensive, maybe it would be reasonable to impose a _minimum_ amount of parallelism, I don't, know, 2 or 4 or so? (I'm assuming the main problem here is seek times.) Also combinable with the proposal of having substitutes for grafts. Greetings, Maxime. (*) IIUC, the time for N parallel seeks should, in theory, ≃ 1 seek, for small values of N, because of elevator algorithms. [-- Attachment #1.1.2: OpenPGP public key --] [-- Type: application/pgp-keys, Size: 929 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 236 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-09-15 15:06 ` Maxime Devos @ 2022-09-15 17:43 ` Csepp 2022-09-15 17:51 ` Maxime Devos 2022-09-19 22:00 ` Ricardo Wurmus 1 sibling, 1 reply; 19+ messages in thread From: Csepp @ 2022-09-15 17:43 UTC (permalink / raw) To: Maxime Devos; +Cc: Csepp, Ricardo Wurmus, guix-devel Maxime Devos <maximedevos@telenet.be> writes: > [[PGP Signed Part:Undecided]] > > > On 15-09-2022 16:46, Csepp wrote: >> Ricardo Wurmus <rekado@elephly.net> writes: >> >>> [...] >>> Did I say *all items*? Well, … grafts are not included, because >>> graft >>> derivations are marked as not substitutable. >>> >>> Can we change that conditionally? I would really like to avoid >>> having >>> to build grafts on B when they have already been built on A. >> I would love this too, because IO can be incredibly slow on HDDs and >> large packages. My netbook would be thankful. >> > > There are some some opportunities for optimizations in the grafting > code before substituting more -- for example, to avoid seek times, it > would be possible to rewrite multiple files concurrently (maybe using > 'par-for-each' to process each file in a directory in parallel). > > This is already done in (guix build grafts), except that: > > * 'find-files' itself is not parallelised, even though parallelising it > could potentially reduce the time spent seeking (*) > * it uses (parallel-job-count), which is (IIUC) 1 by default because > --cores=1 by default. As grafts are more IO intensive than CPU > intensive, maybe it would be reasonable to impose a _minimum_ amount > of parallelism, I don't, know, 2 or 4 or so? > > (I'm assuming the main problem here is seek times.) > > Also combinable with the proposal of having substitutes for grafts. > > Greetings, > Maxime. > > (*) IIUC, the time for N parallel seeks should, in theory, ≃ 1 seek, > for small values of N, because of elevator algorithms. > > [2. OpenPGP public key --- application/pgp-keys; OpenPGP_0x49E3EE22191725EE.asc]... > > [[End of PGP Signed Part]] Could we store the offsets of references somewhere at build time? ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-09-15 17:43 ` Csepp @ 2022-09-15 17:51 ` Maxime Devos 2022-09-19 16:26 ` Josselin Poiret 0 siblings, 1 reply; 19+ messages in thread From: Maxime Devos @ 2022-09-15 17:51 UTC (permalink / raw) To: Csepp; +Cc: Ricardo Wurmus, guix-devel [-- Attachment #1.1.1: Type: text/plain, Size: 749 bytes --] On 15-09-2022 19:43, Csepp wrote: > Could we store the offsets of references somewhere at build time? I now remember that idea, I forgot about that one. My answer: I don't see why not, maybe by adding a phase to %standard-phases (at the very end, to avoid it becoming invalid) that saves it in, say, OUTPUT/.graft-offsets? That would avoid grafting files that don't even have any references, where a copy-file (maybe using copy_file_range or such when available) would suffice. Fallbacks might be necessary (not every store item is constructed from a package), but it all sounds doable and efficient. Also the union could needs to be modified to ignore the .graft-offsets of the union'ed things. Greetings, Maxime. [-- Attachment #1.1.2: OpenPGP public key --] [-- Type: application/pgp-keys, Size: 929 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 236 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-09-15 17:51 ` Maxime Devos @ 2022-09-19 16:26 ` Josselin Poiret 2022-09-19 16:57 ` Maxime Devos 0 siblings, 1 reply; 19+ messages in thread From: Josselin Poiret @ 2022-09-19 16:26 UTC (permalink / raw) To: Maxime Devos, Csepp; +Cc: Ricardo Wurmus, guix-devel Hi everyone, Maxime Devos <maximedevos@telenet.be> writes: > Fallbacks might be necessary (not every store item is constructed from a > package), but it all sounds doable and efficient. Also the union could > needs to be modified to ignore the .graft-offsets of the union'ed things. If I understand the whole thing properly, the daemon already scans for references and could register their location in the database, avoiding the need for a potentially brittle in-store format as you highlighted, although that would require a non-trivial change to both the daemon and the database format. In any case, I don't think it's necessary to treat packages in a special way: any derivation can hold derivations and thus be grafted. Best, -- Josselin Poiret ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-09-19 16:26 ` Josselin Poiret @ 2022-09-19 16:57 ` Maxime Devos 0 siblings, 0 replies; 19+ messages in thread From: Maxime Devos @ 2022-09-19 16:57 UTC (permalink / raw) To: Josselin Poiret, Csepp; +Cc: Ricardo Wurmus, guix-devel [-- Attachment #1.1.1: Type: text/plain, Size: 2429 bytes --] On 19-09-2022 18:26, Josselin Poiret wrote: > Hi everyone, > > Maxime Devos <maximedevos@telenet.be> writes: >> Fallbacks might be necessary (not every store item is constructed from a >> package), but it all sounds doable and efficient. Also the union could >> needs to be modified to ignore the .graft-offsets of the union'ed things. > > If I understand the whole thing properly, the daemon already scans for > references and could register their location in the database, avoiding > the need for a potentially brittle in-store format as you highlighted, > although that would require a non-trivial change to both the daemon and > the database format. If you do this, fallbacks are still required for old daemons. Daemon database changes sound much more brittle to me than the simple .graft-offsets -- the only brittleness I see is the unioning code, but that's easy to address. OTOH, in the ".graft-offsets" proposal, the build process has to do reference scanning (even though the daemon will do so as ell later), which adds some inefficiency -- so perhaps performance-wise, the additional testing for the fallbacks and brittleness might be worth it. > In any case, I don't think it's necessary to treat packages in a special > way: any derivation can hold references and thus be grafted. (Corrected a likely typo derivations->references.) They indeed don't need to be treated specially -- in my proposal, the grafting code wouldn't care whether a store item is of a package or something else, the only thing it cares about, is the presence of a .graft-offsets file. However (unless we go for daemon changes), this .graft-offsets still needs to be made somewhere -- my proposal was to do so this, in case of packages, in an additional phase (which would call a 'make-graft-offsets' procedure '(make-graft-offsets #$output ...)'. Then, I suppose this could be extended to non-packages as well (e.g. modify gexp->derivation and sexp->derivation to do the make-graft-offsets automatically). However, packages should cover almost everything (profiles are mostly a bunch of symlinks and thus don't benefit from grafting optimisations much IIUC), and then gexp->derivation would insert an additional file without it being asked for, which would sound rather surprising to me, so I think that covering packages would be sufficient. Greetings, Maxime. [-- Attachment #1.1.2: OpenPGP public key --] [-- Type: application/pgp-keys, Size: 929 bytes --] [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 236 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-09-15 15:06 ` Maxime Devos 2022-09-15 17:43 ` Csepp @ 2022-09-19 22:00 ` Ricardo Wurmus 2022-10-01 16:43 ` Ludovic Courtès 1 sibling, 1 reply; 19+ messages in thread From: Ricardo Wurmus @ 2022-09-19 22:00 UTC (permalink / raw) To: Maxime Devos; +Cc: Csepp, guix-devel Maxime Devos <maximedevos@telenet.be> writes: > On 15-09-2022 16:46, Csepp wrote: >> Ricardo Wurmus <rekado@elephly.net> writes: >> >>> [...] >>> Did I say *all items*? Well, … grafts are not included, because graft >>> derivations are marked as not substitutable. >>> >>> Can we change that conditionally? I would really like to avoid having >>> to build grafts on B when they have already been built on A. >> I would love this too, because IO can be incredibly slow on HDDs and >> large packages. My netbook would be thankful. >> > > There are some some opportunities for optimizations in the grafting > code before substituting more -- for example, to avoid seek times, it > would be possible to rewrite multiple files concurrently (maybe using > 'par-for-each' to process each file in a directory in parallel). While this is also an interesting development for grafts, my question is much narrowed. I’d be very happy if we could have a little switch that made my daemon(s) ignore the “substitutable?” property and just substitute everything. I’m not looking for performance improvements in grafting, but in an option to totally avoid doing the work twice when substituting a derivation from a server in the same network. -- Ricardo ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-09-19 22:00 ` Ricardo Wurmus @ 2022-10-01 16:43 ` Ludovic Courtès 2022-10-01 17:29 ` Ricardo Wurmus 0 siblings, 1 reply; 19+ messages in thread From: Ludovic Courtès @ 2022-10-01 16:43 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: Maxime Devos, Csepp, guix-devel Hi! Ricardo Wurmus <rekado@elephly.net> skribis: > While this is also an interesting development for grafts, my question is > much narrowed. I’d be very happy if we could have a little switch that > made my daemon(s) ignore the “substitutable?” property and just > substitute everything. > > I’m not looking for performance improvements in grafting, but in an > option to totally avoid doing the work twice when substituting a > derivation from a server in the same network. The premise was that computing a graft is usually quicker than transferring it, which is why they have #:substitutable? #f. I don’t think it would be reasonable to have a switch to toggle #:substitutable? because that would make .drv depend on external factors. However, when copying things around, you can still do: ‘guix copy /gnu/store/…-xyz’ and that’ll work fine, whether or not it’s a graft. But maybe that’s not a satisfying answer for your situation? Ludo’. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-01 16:43 ` Ludovic Courtès @ 2022-10-01 17:29 ` Ricardo Wurmus 2022-10-01 18:00 ` Tobias Geerinckx-Rice 0 siblings, 1 reply; 19+ messages in thread From: Ricardo Wurmus @ 2022-10-01 17:29 UTC (permalink / raw) To: Ludovic Courtès; +Cc: Maxime Devos, Csepp, guix-devel Ludovic Courtès <ludo@gnu.org> writes: > The premise was that computing a graft is usually quicker than > transferring it, which is why they have #:substitutable? #f. > > I don’t think it would be reasonable to have a switch to toggle > #:substitutable? because that would make .drv depend on external > factors. If substitutable was merely a hint that a daemon could be free to ignore it wouldn’t be any worse than downloading a substitute vs building something locally. > However, when copying things around, you can still do: ‘guix copy > /gnu/store/…-xyz’ and that’ll work fine, whether or not it’s a graft. > But maybe that’s not a satisfying answer for your situation? “guix copy” only works over SSH, which requires more configuration (sshd, key exchange, etc) and firewall exceptions. The beauty of derivation substitution is that it just works without any prior configuration. -- Ricardo ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-01 17:29 ` Ricardo Wurmus @ 2022-10-01 18:00 ` Tobias Geerinckx-Rice 2022-10-01 18:27 ` Ricardo Wurmus 0 siblings, 1 reply; 19+ messages in thread From: Tobias Geerinckx-Rice @ 2022-10-01 18:00 UTC (permalink / raw) To: guix-devel, Ricardo Wurmus, Ludovic Courtès; +Cc: Maxime Devos, Csepp Quick note: On 1 October 2022 17:29:03 UTC, Ricardo Wurmus <rekado@elephly.net> wrote: >If substitutable was merely a hint that a daemon could be free to ignore >it wouldn’t be any worse than downloading a substitute vs building >something locally. Substitutability has legal implications in Guix, whether or not it shouldn't. Any 'suggestion' would have to be a third state. Just something to factor into the decision. Kind regards, T G-R Sent on the go. Excuse or enjoy my brevity. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-01 18:00 ` Tobias Geerinckx-Rice @ 2022-10-01 18:27 ` Ricardo Wurmus 2022-10-05 10:03 ` Ludovic Courtès 0 siblings, 1 reply; 19+ messages in thread From: Ricardo Wurmus @ 2022-10-01 18:27 UTC (permalink / raw) To: Tobias Geerinckx-Rice Cc: guix-devel, Ludovic Courtès, Maxime Devos, Csepp Tobias Geerinckx-Rice <me@tobias.gr> writes: > Quick note: > > On 1 October 2022 17:29:03 UTC, Ricardo Wurmus <rekado@elephly.net> wrote: >>If substitutable was merely a hint that a daemon could be free to ignore >>it wouldn’t be any worse than downloading a substitute vs building >>something locally. > > Substitutability has legal implications in Guix, whether or not it > shouldn't. Any 'suggestion' would have to be a third state. Just > something to factor into the decision. Yes, I was about to mention that. In my scenario I have two machines, one building stuff the other only substituting. The serving machine would be the one deciding whether to enforce substitutability or not. -- Ricardo ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-01 18:27 ` Ricardo Wurmus @ 2022-10-05 10:03 ` Ludovic Courtès 2022-10-05 11:41 ` zimoun 2022-10-05 12:53 ` Ricardo Wurmus 0 siblings, 2 replies; 19+ messages in thread From: Ludovic Courtès @ 2022-10-05 10:03 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: Tobias Geerinckx-Rice, guix-devel, Maxime Devos, Csepp Hi, Ricardo Wurmus <rekado@elephly.net> skribis: > In my scenario I have two machines, one building stuff the other only > substituting. The serving machine would be the one deciding whether to > enforce substitutability or not. An is it too expensive for that machine to build grafts locally? Ludo’. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-05 10:03 ` Ludovic Courtès @ 2022-10-05 11:41 ` zimoun 2022-10-07 22:21 ` Mark H Weaver 2022-10-10 15:32 ` Ludovic Courtès 2022-10-05 12:53 ` Ricardo Wurmus 1 sibling, 2 replies; 19+ messages in thread From: zimoun @ 2022-10-05 11:41 UTC (permalink / raw) To: Ludovic Courtès, Ricardo Wurmus Cc: Tobias Geerinckx-Rice, guix-devel, Maxime Devos, Csepp Hi, On Wed, 05 Oct 2022 at 12:03, Ludovic Courtès <ludo@gnu.org> wrote: > Ricardo Wurmus <rekado@elephly.net> skribis: > >> In my scenario I have two machines, one building stuff the other only >> substituting. The serving machine would be the one deciding whether to >> enforce substitutability or not. > > An is it too expensive for that machine to build grafts locally? Using my old desktop from work, yes the experience is really poor; especially for some packages where the number of grafts is sometimes something, e.g., see [1] where grafting were longer than building. 1: <https://yhetil.org/guix/86wnud6db3.fsf@gmail.com> Cheers, simon ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-05 11:41 ` zimoun @ 2022-10-07 22:21 ` Mark H Weaver 2022-10-10 15:32 ` Ludovic Courtès 1 sibling, 0 replies; 19+ messages in thread From: Mark H Weaver @ 2022-10-07 22:21 UTC (permalink / raw) To: zimoun, Ludovic Courtès, Ricardo Wurmus Cc: Tobias Geerinckx-Rice, guix-devel, Maxime Devos, Csepp zimoun <zimon.toutoune@gmail.com> writes: > On Wed, 05 Oct 2022 at 12:03, Ludovic Courtès <ludo@gnu.org> wrote: >> Ricardo Wurmus <rekado@elephly.net> skribis: >> >>> In my scenario I have two machines, one building stuff the other only >>> substituting. The serving machine would be the one deciding whether to >>> enforce substitutability or not. >> >> An is it too expensive for that machine to build grafts locally? > > Using my old desktop from work, yes the experience is really poor; > especially for some packages where the number of grafts is sometimes > something, e.g., see [1] where grafting were longer than building. > > 1: <https://yhetil.org/guix/86wnud6db3.fsf@gmail.com> I wonder how fast grafting could be if it were rewritten in C. Mark -- Disinformation flourishes because many people care deeply about injustice but very few check the facts. Ask me about <https://stallmansupport.org>. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-05 11:41 ` zimoun 2022-10-07 22:21 ` Mark H Weaver @ 2022-10-10 15:32 ` Ludovic Courtès 2022-10-10 15:55 ` zimoun 1 sibling, 1 reply; 19+ messages in thread From: Ludovic Courtès @ 2022-10-10 15:32 UTC (permalink / raw) To: zimoun Cc: Ricardo Wurmus, Tobias Geerinckx-Rice, guix-devel, Maxime Devos, Csepp Hi, zimoun <zimon.toutoune@gmail.com> skribis: > On Wed, 05 Oct 2022 at 12:03, Ludovic Courtès <ludo@gnu.org> wrote: [...] >> An is it too expensive for that machine to build grafts locally? > > Using my old desktop from work, yes the experience is really poor; > especially for some packages where the number of grafts is sometimes > something, e.g., see [1] where grafting were longer than building. > > 1: <https://yhetil.org/guix/86wnud6db3.fsf@gmail.com> This has to be compared with the cost of a rebuild/redownload of the same set of packages. Even in the worst case, grafts are faster than that. Now, the difference is that those grafts need to be recomputed regularly, where the rebuild/redownload would be relatively rare. It’s a tradeoff. We could have machinery to help determine when to ungraft and perhaps even automate ungrafting. Mark H Weaver <mhw@netris.org> skribis: > I wonder how fast grafting could be if it were rewritten in C. I’d expect it to be I/O-bound, so maybe not that much? Thanks, Ludo’. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-10 15:32 ` Ludovic Courtès @ 2022-10-10 15:55 ` zimoun 2022-10-10 17:50 ` Ricardo Wurmus 0 siblings, 1 reply; 19+ messages in thread From: zimoun @ 2022-10-10 15:55 UTC (permalink / raw) To: Ludovic Courtès Cc: Ricardo Wurmus, Tobias Geerinckx-Rice, guix-devel, Maxime Devos, Csepp Hi Ludo, On Mon, 10 Oct 2022 at 17:32, Ludovic Courtès <ludo@gnu.org> wrote: > This has to be compared with the cost of a rebuild/redownload of the > same set of packages. Even in the worst case, grafts are faster than > that. Now, the difference is that those grafts need to be recomputed > regularly, where the rebuild/redownload would be relatively rare. I have not done some stats, indeed. :-) The build/download or rebuild/redownload is really faster (at least for some R packages) than the graft part for some machines with spinning disks. Well, I have been enough annoyed for some packages on that machine to end by systematically run the '--no-grafts' option for all packages. :-/ Cheers, simon ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-10 15:55 ` zimoun @ 2022-10-10 17:50 ` Ricardo Wurmus 0 siblings, 0 replies; 19+ messages in thread From: Ricardo Wurmus @ 2022-10-10 17:50 UTC (permalink / raw) To: zimoun Cc: Ludovic Courtès, Tobias Geerinckx-Rice, guix-devel, Maxime Devos, Csepp zimoun <zimon.toutoune@gmail.com> writes: > Hi Ludo, > > On Mon, 10 Oct 2022 at 17:32, Ludovic Courtès <ludo@gnu.org> wrote: > >> This has to be compared with the cost of a rebuild/redownload of the >> same set of packages. Even in the worst case, grafts are faster than >> that. Now, the difference is that those grafts need to be recomputed >> regularly, where the rebuild/redownload would be relatively rare. > > I have not done some stats, indeed. :-) > > The build/download or rebuild/redownload is really faster (at least > for some R packages) than the graft part for some machines with > spinning disks. Well, I have been enough annoyed for some packages on > that machine to end by systematically run the '--no-grafts' option for > all packages. :-/ I’m not worried about grafts as such; I only think it’s a pity to have grafts recomputed on machines on the very same network. When I have a direct link from one machine to another it is very fast to just copy things over instead of performing the graft locally. This is not something that can be decided generally. When substituting from a remote server it’s probably the right thing to compute the graft locally, but there are a number of cases where that’s not a good idea (e.g. machines on the same network, underpowered aarch64 machines downloading substitutes from a faster machine, etc). -- Ricardo ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: substitute derivation: also substitute grafts? 2022-10-05 10:03 ` Ludovic Courtès 2022-10-05 11:41 ` zimoun @ 2022-10-05 12:53 ` Ricardo Wurmus 1 sibling, 0 replies; 19+ messages in thread From: Ricardo Wurmus @ 2022-10-05 12:53 UTC (permalink / raw) To: Ludovic Courtès Cc: Tobias Geerinckx-Rice, guix-devel, Maxime Devos, Csepp Ludovic Courtès <ludo@gnu.org> writes: > Ricardo Wurmus <rekado@elephly.net> skribis: > >> In my scenario I have two machines, one building stuff the other only >> substituting. The serving machine would be the one deciding whether to >> enforce substitutability or not. > > An is it too expensive for that machine to build grafts locally? It’s a matter of time. Having prepared the environment previously on another fast machine I’d like to avoid repeating the same work again on a slow machine. We needn’t restrict ourselves to substitutions — if “guix copy” could work without first having to set up SSH that would be an equally valid solution to this problem. -- Ricardo ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2022-10-10 18:59 UTC | newest] Thread overview: 19+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-09-15 12:58 substitute derivation: also substitute grafts? Ricardo Wurmus 2022-09-15 14:46 ` Csepp 2022-09-15 15:06 ` Maxime Devos 2022-09-15 17:43 ` Csepp 2022-09-15 17:51 ` Maxime Devos 2022-09-19 16:26 ` Josselin Poiret 2022-09-19 16:57 ` Maxime Devos 2022-09-19 22:00 ` Ricardo Wurmus 2022-10-01 16:43 ` Ludovic Courtès 2022-10-01 17:29 ` Ricardo Wurmus 2022-10-01 18:00 ` Tobias Geerinckx-Rice 2022-10-01 18:27 ` Ricardo Wurmus 2022-10-05 10:03 ` Ludovic Courtès 2022-10-05 11:41 ` zimoun 2022-10-07 22:21 ` Mark H Weaver 2022-10-10 15:32 ` Ludovic Courtès 2022-10-10 15:55 ` zimoun 2022-10-10 17:50 ` Ricardo Wurmus 2022-10-05 12:53 ` Ricardo Wurmus
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).