bug#44723: Latest binary tarball & Hurd qcow2 image unavailable (502)

bug#44723: Latest binary tarball & Hurd qcow2 image unavailable (502)

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

Guix,

The binary tarball and Hurd qcow2 links on 
<http://guix.gnu.org/en/download/latest/>, which resolve to 
<https://ci.guix.gnu.org/download/1385> and 
<https://ci.guix.gnu.org/download/1450> respectively, return 502.

Here's why:

  2020-11-18T17:31:24 GET /download/1450
  Uncaught exception in fiber ##f:
  In ice-9/boot-9.scm:
    1669:16  0 (raise-exception _ #:continuable? _)
  ice-9/boot-9.scm:1669:16: In procedure raise-exception:
  In procedure open-file: No such file or directory:
  "/gnu/store/vd5y7hmvgqc1hwmqyiwqa171r0iz1wib-hurd-barebones.qcow2"

The file really is gone.  It seems that Cuirass is no longer 
creating sufficient GC roots.  I vaguely remember some changes to 
image GC'ing being discussed but not what (if anything) changed.

Thanks to zimoun for pointing out the broken links.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#44723: Latest binary tarball & Hurd qcow2 image unavailable (502)

[Ludovic Courtès]

Hi,

Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org>
skribis:

> The binary tarball and Hurd qcow2 links on
> <http://guix.gnu.org/en/download/latest/>, which resolve to 
> <https://ci.guix.gnu.org/download/1385> and
> <https://ci.guix.gnu.org/download/1450> respectively, return 502.
>
> Here's why:
>
>  2020-11-18T17:31:24 GET /download/1450
>  Uncaught exception in fiber ##f:
>  In ice-9/boot-9.scm:
>    1669:16  0 (raise-exception _ #:continuable? _)
>  ice-9/boot-9.scm:1669:16: In procedure raise-exception:
>  In procedure open-file: No such file or directory:
>  "/gnu/store/vd5y7hmvgqc1hwmqyiwqa171r0iz1wib-hurd-barebones.qcow2"
>
> The file really is gone.  It seems that Cuirass is no longer creating
> sufficient GC roots.  I vaguely remember some changes to image GC'ing
> being discussed but not what (if anything) changed.

QCOW2 images get GC’d rather quickly, because they take up a lot of
space; see in particular:

  https://git.savannah.gnu.org/cgit/guix/maintenance.git/commit/?id=4a720a03f9d190f7b1abc883f49d51a95a5f58a9
  https://git.savannah.gnu.org/cgit/guix/maintenance.git/commit/?id=0bf8202de811364ebd79e390fcd039f7936b6ea7

I guess it’s OK for Guix System ISO images because a fresh one gets
built at each evaluation, roughly, but perhaps the Hurd images are not
rebuilt frequently enough and the changes that they’re GC’d are higher.

I don’t have a good solution for that, we’re consuming storage space too
quickly with all these images currently.  :-/

Thanks,
Ludo’.

bug#44723: Latest binary tarball & Hurd qcow2 image unavailable (502)

[zimoun]

Hi,

On Wed, 18 Nov 2020 at 18:46, Ludovic Courtès <ludo@gnu.org> wrote:

> I guess it’s OK for Guix System ISO images because a fresh one gets
> built at each evaluation, roughly, but perhaps the Hurd images are not
> rebuilt frequently enough and the changes that they’re GC’d are higher.
>
> I don’t have a good solution for that, we’re consuming storage space too
> quickly with all these images currently.  :-/

Does it make sense to only keep the latest?  And GC all the older one
except the stable one (pinned).

Cheers,
simon

bug#44723: Latest binary tarball & Hurd qcow2 image unavailable (502)

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 336 bytes --]

Simon,

zimoun 写道：
> Does it make sense to only keep the latest?  And GC all the 
> older one
> except the stable one (pinned).

It absolutely makes sense.  But knowing how much of a baroque 
spaghetti painting some of this stuff is behind the scenes, it 
might not be as trivial as it sounds.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#44723: Latest binary tarball & Hurd qcow2 image unavailable (502)

[Bengt Richter]

Hi,

On +2020-11-18 18:50:03 +0100, zimoun wrote:
> Hi,
> 
> On Wed, 18 Nov 2020 at 18:46, Ludovic Courtès <ludo@gnu.org> wrote:
> 
> > I guess it’s OK for Guix System ISO images because a fresh one gets
> > built at each evaluation, roughly, but perhaps the Hurd images are not
> > rebuilt frequently enough and the changes that they’re GC’d are higher.
> >
> > I don’t have a good solution for that, we’re consuming storage space too
> > quickly with all these images currently.  :-/
>
Eric Blake from Red Hat is IIUC working on qemu features to share clean parts
of images without duplicating [1] and LWN has an article re his work [2]
(I hope posting this [2] subsriberlink here is Ok with LWN -- I would think so)

Perhaps their incremental chaining technology can help consolidate "all these images" ?

> Does it make sense to only keep the latest?  And GC all the older one
> except the stable one (pinned).
> 
> Cheers,
> simon
> 
> 
> 
[1] https://static.sched.com/hosted_files/kvmforum2020/92/kvmforum_2020_Bitmaps_and_NBD.pdf
[2] https://lwn.net/SubscriberLink/837053/06072931fc4162bf/
    ([2] should be available freely to all as [3] in less than a week)
[3] https://lwn.net/Articles/837053/ 
-- 
Regards,
Bengt Richter

bug#44723: Latest binary tarball & Hurd qcow2 image unavailable (502)

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> On Wed, 18 Nov 2020 at 18:46, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> I guess it’s OK for Guix System ISO images because a fresh one gets
>> built at each evaluation, roughly, but perhaps the Hurd images are not
>> rebuilt frequently enough and the changes that they’re GC’d are higher.
>>
>> I don’t have a good solution for that, we’re consuming storage space too
>> quickly with all these images currently.  :-/
>
> Does it make sense to only keep the latest?  And GC all the older one
> except the stable one (pinned).

We could change ‘cleanup-cuirass-roots’ (the mcron job I showed) to keep
at least one ‘hurd-barebones.qcow2’ root.  Help welcome!

Ludo’.

bug#44723: Latest binary tarball & Hurd qcow2 image unavailable (502)

[zimoun]

Hi,
On Thu, 19 Nov 2020 at 09:05, Ludovic Courtès <ludo@gnu.org> wrote:

>> Does it make sense to only keep the latest?  And GC all the older one
>> except the stable one (pinned).
>
> We could change ‘cleanup-cuirass-roots’ (the mcron job I showed) to keep
> at least one ‘hurd-barebones.qcow2’ root.  Help welcome!

Where is the configuration?  Repo maintenance?  Elsewhere?

All the beest,
simon

bug#44723: Latest binary tarball & Hurd qcow2 image unavailable (502)

[Mathieu Othacehe]

Hey zimoun,

> Where is the configuration?  Repo maintenance?  Elsewhere?

Yes it's here: 
https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/modules/sysadmin/services.scm

Thanks,

Mathieu