From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christopher Baines Subject: Re: Avoiding the garbage collector for the Guix Data Service Date: Wed, 12 Feb 2020 23:54:36 +0000 Message-ID: <877e0r1hvn.fsf@cbaines.net> References: <87tv4tp0c8.fsf@cbaines.net> <87r1yzucir.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:32989) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j21q7-0004Wf-Ol for help-guix@gnu.org; Wed, 12 Feb 2020 18:54:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j21q6-0008HE-Ja for help-guix@gnu.org; Wed, 12 Feb 2020 18:54:43 -0500 In-reply-to: <87r1yzucir.fsf@gnu.org> List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+gcggh-help-guix=m.gmane-mx.org@gnu.org Sender: "Help-Guix" To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: help-guix@gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s writes: > Hi Christopher, > > Christopher Baines skribis: > >> The Guix Data Service has Guix compute lots of derivations, and then it >> loads the data in to a database. I'm having some issues with the garbage >> collector though, as sometimes it removes data from the disk before I've >> had a chance to read it (at least I think this is what's happening, I >> think this job failed for this reason [1]). >> >> 1: http://data.guix.gnu.org/job/10638 >> >> I'm not quite sure what the best strategy is to combat this? I had a >> look, and I couldn't see how to acquire a lock to prevent the garbage >> collector from running. I thought about registering thousands of gc >> roots, but that seems to just move the race a bit, as there's still a >> period between generating the derivation and registering the root where >> it can be deleted. >> >> Any ideas? > > At the Guix Days we discussed using =E2=80=98add-temp-root=E2=80=99 and s= imilar to > ensure that .drv files are at least GC-protected for the duration of the > session (connection to the daemon). > > Does that address your problem? I think so, it at least seems to greatly reduce the chance the derivations get removed. I made the Guix Data Service change here [1], and I haven't seen the issue since. 1: https://git.savannah.gnu.org/cgit/guix/data-service.git/commit/?id=3Df59= 354ed23a9c50e43b7aaaeaddb5365feda4e29 Thanks, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAl5EkDxfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE 9Xdtfw//bt2ZYo/+qM7vhmAeZ7zsewv0/vk8KbpWFCZb0K/BV/gGcnAn1AOC8LfM d1NWP1WeK7NEYuvlNaSqp/GnBxLldSgeFli3E9jNFC1eD4lmOd1dbB3pRKDEhfMn ZtdN680qKcRef6R1CFF1x2+11GK5KU+d+LtaBKVgkgP2QUGi5revfuZvjPDGCpbg oZw178W9rNpHS0zO7HTtUekC89VCqijQt692PGgDojdhNa+4nx/yuuMmH376mrfP ptt35POmXwPf2zAZJ/UKc+fuk21dESNenwk2o4Fnjeh7Dj1eaOEhe6WYZrIebPWe IL1tRMmYs5f800kNKfdwKiCcg1Fl+TeFKxGugGrpuaCGEdF7D23i93f0kJBPuDiF SSt/wBHTHdhYPKji7jzQbbxHImAAxZYt1mK9h+UiCCvpvusAm3YRJBKoLUfoQup5 nCBy8DmEoyNJ4rVbwwVMoCZC1R4KuSiNIPIH0KMCLdu7WublgboCv14844cw0M8j 0zGzaVyvLBhJXCr6fDRucGFq3xSWUUufHyiDccxqsk/6IIHfs6iEP8e+SnhpumZw HhY5BmFDT+3hyokttmxHvqnJFqavlBq0oH6V/ca22X5KJMLptSDcOyhbbCrXXI3u D0mNJHBasY1NpMOplOq0ONqVS2A93/gCqQBgJW+kvyfymaxHdjA= =e3UE -----END PGP SIGNATURE----- --=-=-=--