From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id gJt6Mjsl/WF2WQEAgWs5BA (envelope-from ) for ; Fri, 04 Feb 2022 14:08:11 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id 4DvDLzsl/WGbmwAA9RJhRA (envelope-from ) for ; Fri, 04 Feb 2022 14:08:11 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7C5C925534 for ; Fri, 4 Feb 2022 14:08:11 +0100 (CET) Received: from localhost ([::1]:36462 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nFyJu-00022c-KM for larch@yhetil.org; Fri, 04 Feb 2022 08:08:10 -0500 Received: from eggs.gnu.org ([209.51.188.92]:46316) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nFxzW-0004vC-Fu for guix-devel@gnu.org; Fri, 04 Feb 2022 07:47:07 -0500 Received: from [2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27] (port=47697 helo=mira.cbaines.net) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nFxzR-0007JN-SY for guix-devel@gnu.org; Fri, 04 Feb 2022 07:47:05 -0500 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 010B727BBE9; Fri, 4 Feb 2022 12:46:14 +0000 (GMT) Received: from capella (localhost [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id d473d251; Fri, 4 Feb 2022 12:46:14 +0000 (UTC) References: <874k79zs29.fsf@cbaines.net> <87bl1bdj7b.fsf@gnu.org> <86r1a6sxc6.fsf@gmail.com> User-agent: mu4e 1.6.10; emacs 27.2 From: Christopher Baines To: zimoun Subject: Re: extend =?utf-8?Q?=E2=80=99guix_archive=E2=80=99=3F?= Date: Fri, 04 Feb 2022 12:36:52 +0000 In-reply-to: <86r1a6sxc6.fsf@gmail.com> Message-ID: <87k0eag5pl.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27 (failed) Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, RDNS_NONE=0.793, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1643980091; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=tFWQ8cCOJeqarC87v1ySozd/kwaAwDCoaMKV7+idINU=; b=AWKQhA0BRF68s0zPBfRraGuX1GhO+AmpyAzsAFDBu3Y3cVLHvnl9B4l3DiBaHZEH7sDpE/ OVvl/hsvvJsHxq6vl1ifjp5WjB/Drf2xcpPl+fF/q7mh0JozjiRWOU7OsOqNBPQY4CAjW6 2le5uuQh5yg8wTvNoU61HevoWEb4HYVFjJyXNia2HKPA2Lhxgc6kEY02hqQmFqZ2m77e+o kweT94ahGoFj56usJmzkyFYEI2I/Ve3u0CCG/Lam/Ju0d4pUIN7NxH7XcxnQY+Cx80lIbC d8MKVoGG54BACCugvuBn4ZLevS5TW+S6eSJXNxEfIxiSwH5SwH5uUt4ktf2SUw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1643980091; a=rsa-sha256; cv=none; b=NWKqbcnPhDZvoWCsF+8dyefiQz9yTEMR7NQdOIk0AT2IB44nQ8NmI7fTr16xT8As6oUEfR d8w0W7cpDjswcbdkin+hlRE5atdWkRR2J+nq9qcFDKoEOV0mCJ1E7Vb2SuX+cU43gaZihK FeAlKSnzcoH3Z3E/ZomXv2P2U+klVYGJ2Q4gU3wErdarlBhl7IqLx8cMnvl6h4ivfaOYhJ MkBH7Xi1PVU1ruZV0M/+3u5Zxerv/KSNKUoUuvCP0/Fe/MHwV3rLXO5fPKC4NeXHCoqG4W ypHNbx+HNpp9/qxOzTAAOxdkDRvKq+VDKCm8lHqfHLb13rDiYmWjt1IsW542fg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.23 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 7C5C925534 X-Spam-Score: -4.23 X-Migadu-Scanner: scn0.migadu.com X-TUID: DkTkF0Nv7FkB --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable zimoun writes: > On Mon, 20 Dec 2021 at 23:07, Ludovic Court=C3=A8s wrote: > >> Regarding nar-herder, I think it=E2=80=99d be nice to have a solution to >> mirroring in Guix proper, developed similarly to other components, >> because it could be a fairly central tool. >> >> =E2=80=98guix publish=E2=80=99 is probably not extensible enough to supp= ort that, but we >> could make it a new =E2=80=98guix mirror=E2=80=99 or =E2=80=98guix sync= =E2=80=99 or whatever command. > > Why not extend =E2=80=9Cguix archive=E2=80=9D? > > However, without all the details so my remark is totally naive, I miss > what =E2=80=9Cnar-herder=E2=80=9D is doing that =E2=80=9Cguix archive=E2= =80=9D+rsync+=E2=80=9Cguix publish=E2=80=9D is > not doing =E2=80=93 other said I miss why another SQL database is require= d to > serve stuff from one place to another. I have read README but I did not > get the point. Apologies, I missed replying to this at the time. Using an SQL database (sqlite) isn't required in my opinion, but I do like that aspect of the design. You could for example keep the narinfo's as files on the disk, and then use rsync say to copy them between machines to setup mirrors. I think this would perform worse compared to storing the narinfos in a database when initially setting up a mirror. With a database, you only have to download one large file, whereas with individual narinfo files, you have to download lots of individual small files. Storing all the narinfo files individually also generally takes up more space than storing them in a database, since there's an overhead associated with each file on the filesystem. Additionally, the database is used to do extra things on top of just storing the narinfos. The references from the narinfos are stored separately to facilitate doing GC like removal of the nars. Additionally, there's a way to tag nars, something which I was thinking of using to allow selectively removing some nars which only need to be stored for a short time. https://git.cbaines.net/guix/nar-herder/tree/nar-herder/database.scm#n74 I hope that makes some sense, Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmH9IBZfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XftNxAArZQrrqrUijZC0fwps43ETF3JzhgErrKe 2k+6ECmcBOKfpxsyuyRS9LY0Q23zyXFRnVGQRkzsxsQ7XkV1c+aHkinQ9MIVnsTo sXVL2qX+H3ciBVdwfzmc0GmbAMfleGLh9pymgDOFb87KJINEgERy5P6m+uv5uY0y 0Fa0Z97NKxcyMJNX/fMf3RTaVptm++J3A74sGIeK9xG/NN647Ckz9w1TuOg2U52x 2JFgzRyzwzMHd7PPJZSL0EZ/NZzDhurnn9YrffK5U62qLKg+KE40wyxEzGOCyiiN 9tgDtDi34rszm1qURh72470TvntvMaXBOCMf7ZqGFhCBmZnRJQUliVVWQ+0CHTFh 1ZkQjMQNUgWBKTh0r1cxXJgqA9A+JZYzML3ZHnEE9CJJF+SkIPEzd222dvDZZJ1A 9znL5Bjwbk//LFhnGc+0tHW0u6NnsTEmuKoQf7z1BCuYHAMHnrnY1SmAM87aERQu FsGcIMDKtv6C19/i9qyY8h081rt7rDcVayAxNrKlAmwzlgdet7sBohBL/iSxnZlR 3tclzAAP/FQiSEHehaYNd/RzXTJaL0t+v4rx5a1rWKq7LeBD3T8qyRf7irgc1818 fnif7AfNaw8xOroIymjoIeDTUlmadd0BNxSKNk3erI+/v0aKtaoVHGV+k3iqCBTl dvVm0AMdyWs= =kPcj -----END PGP SIGNATURE----- --=-=-=--