Re: Dropping gzip-compressed substitutes

[Maxim Cournoyer <maxim.cournoyer@gmail.com>]

Hi Mathieu!

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hey Maxim,
>
> Sound like a fine plan.
>
>> 1.  Promptly set up both a blog post and a NEWS entry announcing the
>> support for gzip substitutes is about to be phased out from the build
>> farm (1 month notice), urging users to upgrade their daemon to a version
>>>= 1.1.0.
>
> I addition, we could warn users with old daemons as proposed by Ludo &
> Ricardo. There's an attached patch here.

Awesome, thank you!  It LGTM.  Could you apply it?

>> -                   ;; TODO: Eventually, disable gzip, as discussed at
>> -                   ;; <https://lists.gnu.org/archive/html/guix-devel/2021-03/msg00333.html>.
>> -                   (compression '(("gzip" 9) ("lzip" 9) ("zstd" 19)))
>> +                   (compression '(("lzip" 9) ("zstd" 19)))
>>                     (cache-bypass-threshold cache-bypass-threshold)
>>                     (workers publish-workers)))
>
> Nice.
>
>> 3. Come up with a Guile script that is able to
>>
>>   a) Strip gzip-related metadata from the .narinfo guix-publish metadata
>>   files
>>   b) recompute and update their 'Signature' field.
>>
>> 4. Finally, 'rm -r /var/guix/publish/gzip' and free about 6.5 TiB of data.
>
> I hope the script will be able to complete within a reasonable amount of
> time, but we cannot know before trying it out :).

The script will run on the cache data kept on the SSD array (which we
can snapshot before experimenting with). It'll probably take a while
still, so care should be taken to ensure it takes as little time as
possible (e.g. spawning many threads or processes to spread the load
across multiple CPUs).

I'll work on the blog post and news entry, and then we'll have to look
at the script.

Thank you!

Maxim