Re: substitute derivation: also substitute grafts?

[Maxime Devos <maximedevos@telenet.be>]

[-- Attachment #1.1.1: Type: text/plain, Size: 1524 bytes --]



On 15-09-2022 16:46, Csepp wrote:
> 
> Ricardo Wurmus <rekado@elephly.net> writes:
> 
>> [...]
>> Did I say *all items*? Well, … grafts are not included, because graft
>> derivations are marked as not substitutable.
>>
>> Can we change that conditionally? I would really like to avoid having
>> to build grafts on B when they have already been built on A.
> 
> I would love this too, because IO can be incredibly slow on HDDs and
> large packages.  My netbook would be thankful.
>

There are some some opportunities for optimizations in the grafting code 
before substituting more -- for example, to avoid seek times, it would 
be possible to rewrite multiple files concurrently (maybe using 
'par-for-each' to process each file in a directory in parallel).

This is already done in (guix build grafts), except that:

  * 'find-files' itself is not parallelised, even though parallelising it
     could potentially reduce the time spent seeking (*)
  * it uses (parallel-job-count), which is (IIUC) 1 by default because
    --cores=1 by default.  As grafts are more IO intensive than CPU
    intensive, maybe it would be reasonable to impose a _minimum_ amount
    of parallelism, I don't, know, 2 or 4 or so?

(I'm assuming the main problem here is seek times.)

Also combinable with the proposal of having substitutes for grafts.

Greetings,
Maxime.

(*) IIUC, the time for N parallel seeks should, in theory, ≃ 1 seek, for 
small values of N, because of elevator algorithms.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]