Compressing compressed files

Compressing compressed files

[Andreas Enge]

Hello,

I am seeing this funny command being executed on one of the build machines:
   bash -c guix archive --export /gnu/store/06g54r7am7ka96d7jydgmayfyqgx1s1l-cfe-3.5.2.src.tar.xz | xz -c

Would it be difficult to prevent re-compressing of already compressed
source when offloading? Especially since xz is rather expensive, and more
or less occupies one core of the arm machines.

Andreas

Re: Compressing compressed files

[Ludovic Courtès]

Hi!

Andreas Enge <andreas@enge.fr> skribis:

> I am seeing this funny command being executed on one of the build machines:
>    bash -c guix archive --export /gnu/store/06g54r7am7ka96d7jydgmayfyqgx1s1l-cfe-3.5.2.src.tar.xz | xz -c
>
> Would it be difficult to prevent re-compressing of already compressed
> source when offloading? Especially since xz is rather expensive, and more
> or less occupies one core of the arm machines.

This all takes place in (guix scripts offload) and should be relatively
easy to modify (see commit 5ef9d7deeab348e0f714f91a3c02ce4b72bbe3c1 for
an example).

[Time passes…]

I thought it would be quicker to fix it but was proved wrong, because
‘guix substitute’ expects to be able to pass non-file ports to
‘decompressed-ports’, which means we cannot just use ‘duplicate-port’
there.  Anyway, I’ve pushed ‘wip-offload-compression’ and we’ll look at
it again after the release.

Thanks,
Ludo’.

Re: Compressing compressed files

[Andreas Enge]

On Mon, Aug 01, 2016 at 06:02:54PM +0200, Ludovic Courtès wrote:
> [Time passes…]
> 
> I thought it would be quicker to fix it but was proved wrong, because
> ‘guix substitute’ expects to be able to pass non-file ports to
> ‘decompressed-ports’, which means we cannot just use ‘duplicate-port’
> there.  Anyway, I’ve pushed ‘wip-offload-compression’ and we’ll look at
> it again after the release.

Apologies, my mail was not meant as an immediate call for action! I would
also have been happy about a reply stating that it is too difficult compared
to the limited benefit.

Andreas

Re: Compressing compressed files

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> On Mon, Aug 01, 2016 at 06:02:54PM +0200, Ludovic Courtès wrote:
>> [Time passes…]
>> 
>> I thought it would be quicker to fix it but was proved wrong, because
>> ‘guix substitute’ expects to be able to pass non-file ports to
>> ‘decompressed-ports’, which means we cannot just use ‘duplicate-port’
>> there.  Anyway, I’ve pushed ‘wip-offload-compression’ and we’ll look at
>> it again after the release.
>
> Apologies, my mail was not meant as an immediate call for action! I would
> also have been happy about a reply stating that it is too difficult compared
> to the limited benefit.

No problem!  I thought it’d be so easy that I should do it right away,
all my fault.  :-)

Ludo’.