From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id QOpQLwSixmKbFAEAbAwnHQ (envelope-from ) for ; Thu, 07 Jul 2022 11:06:12 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id mEkcLwSixmLmewAAauVa8A (envelope-from ) for ; Thu, 07 Jul 2022 11:06:12 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6B8202692A for ; Thu, 7 Jul 2022 11:06:12 +0200 (CEST) Received: from localhost ([::1]:38746 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o9NSc-0001sk-Qs for larch@yhetil.org; Thu, 07 Jul 2022 05:06:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41214) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o9NOP-0008BT-IO for guix-devel@gnu.org; Thu, 07 Jul 2022 05:01:49 -0400 Received: from ns13.heimat.it ([46.4.214.66]:39552) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o9NOK-0007DW-21; Thu, 07 Jul 2022 05:01:49 -0400 Received: from localhost (ip6-localhost [127.0.0.1]) by ns13.heimat.it (Postfix) with ESMTP id D246530022C; Thu, 7 Jul 2022 09:01:38 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at ns13.heimat.it Received: from ns13.heimat.it ([127.0.0.1]) by localhost (ns13.heimat.it [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zQSV2kJvbXN7; Thu, 7 Jul 2022 09:01:37 +0000 (UTC) Received: from bourrache.mug.xelera.it (unknown [93.56.171.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by ns13.heimat.it (Postfix) with ESMTPSA id 09C2C3001D5; Thu, 7 Jul 2022 09:01:37 +0000 (UTC) Received: from roquette.mug.biscuolo.net (roquette [10.38.2.14]) by bourrache.mug.xelera.it (Postfix) with SMTP id 74E381C47641; Thu, 7 Jul 2022 11:01:36 +0200 (CEST) Received: (nullmailer pid 6724 invoked by uid 1000); Thu, 07 Jul 2022 09:01:35 -0000 From: Giovanni Biscuolo To: Ludovic =?utf-8?Q?Court=C3=A8s?= , zimoun Cc: Akib Azmain Turja , guix-devel@gnu.org Subject: Re: First guix pull is too costly In-Reply-To: <87fsjl3sgm.fsf@gnu.org> Organization: Xelera.eu References: <87wnd1kycb.fsf@disroot.org> <86zghxdobv.fsf@gmail.com> <87fsjl3sgm.fsf@gnu.org> Date: Thu, 07 Jul 2022 11:01:34 +0200 Message-ID: <87tu7ts3dd.fsf@xelera.eu> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=46.4.214.66; envelope-from=g@xelera.eu; helo=ns13.heimat.it X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1657184772; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=956dpt2v7DzoNsM+GqkLb17nNl8bRMyvZ7rEPDdlbZY=; b=O5F0otbZaQq5iCvjY1cWkj+zAzNUUZzODCe8UhY49HjXfYzI5IZ9uV+cFEIZoIM/lOCSwO RKJ1iKR3VjhRQ6HT9PcmG5wEO9fQqR+mOSvovRQO2M1K/lsS8GSrm+QxXBXppyVlslTU4c /tGjtugJ35lXSXPuI9/wj5o9K6F6VsmUT8OS7k1HBGGcbZ79eJCc2KfmkOfvaiW6MRiERU sFERgpJ99JFwc2z5yddTy0n5kMU8zVpnxlMi/lOF0TRAXKCJsyu2Z45LuSNfJ9Xszw61Xr YMASgZkfS8AiJGKHnSF4QcaAL0d9uDx98pY2KaoQYPm+9l2Cz1kCiqR6BTpbAw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1657184772; a=rsa-sha256; cv=none; b=thlhYP20GSYwOCYx6UzceQTu+1S7hQ+pt8ppNrC+OZDlnfyGPAyAbZrnxvPRmI/8gDCVog 9rUCEaK/hV/Ki+Q4hj9WQlU2pFfQjC/DnjrV4LrjyZ9zvkiTzjOaxcZDtInyuAxDAyOPfq jtr/FGFagJq9UAdj+uA4OaXa4MqCJ97GiHI+I3p7ibc2nyQ0+JjFQk7R2MpmbrKcYTX/Nu glmsuJtFgSlQ4IopkAMBbHSsTVuAiyp9EbwypGSOT/JYhheTXW+9zwr0yMnLLSlIv+Jx84 vMiH3u0zRJ0NVA2gbaThPjC4yUtRUGIjoeKOBMAq/ESIpZf92hXX8Pi4xIMH9Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.15 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 6B8202692A X-Spam-Score: -4.15 X-Migadu-Scanner: scn0.migadu.com X-TUID: ncK4dwDaviQ9 --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello Ludo=E2=80=99 and Simon, Ludovic Court=C3=A8s writes: [...] > Does =E2=80=98git clone=E2=80=99 do a better job than libgit2/Guile-Git (= which is what > =E2=80=98guix pull=E2=80=99 uses) in case it gets disconnected while fetc= hing the repo? > Is it able to resume when libgit2 isn=E2=80=99t? > > (I=E2=80=99d be surprised, but I don=E2=80=99t know.) AFAIU `git clone` is still not able to resume after a failed connection, there is a SoC2008 proposal [1] but it seems it was never implemented. Thanks to this stackoverflow thread: https://stackoverflow.com/questions/3954852/how-to-complete-a-git-clone-for= -a-big-project-on-an-unstable-connection it seems there are two available workarounds to this problem: 1. Use shallow clone: git clone --depth=3D1 git fetch --unshallow if fetch fails unshallowing due to connection problems, AFAIU the next "unshallow" will continue fetching more objects until it's done 2. git-bundle [2] some tagged release The bundle itself is an ordinary file, which you can download any way, via HTTP/FTP with resume support, via BitTorrent, via rsync, etc. You can create clone from a bundle, fix configuration [3], and do further fetches from a proper git repository. A Guix repo git bundle for every new release, starting from 1.3.0 (1.4?), could be useful for people with slow or unreliable internet connection. Could `guix pull` be extended to automatically do this for the user? I mean: for each channell, `guix pull` could test if the "channel-bundle" (see below) is available and download it with a resumable method, then use the git-bundle file to create the clone and finally fix the remote url in the git configuration. To define the "channel-bundle" we could add a "bundle" sexp (gexp?) to the channels.scm file, like this: =2D-8<---------------cut here---------------start------------->8--- (list (channel (name 'guix) (bundle "https://git.savannah.gnu.org/guix-1.3.0.bundle") (url "https://git.savannah.gnu.org/git/guix.git") (commit "a3d86b341d361530127c36fbfbf77d638df2c1de") (introduction (make-channel-introduction "9edb3f66fd807b096b48283debdcddccfea34bad" (openpgp-fingerprint "BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA"))))) =2D-8<---------------cut here---------------end--------------->8--- WDYT? Ciao, Gio' P.S.: I'm sorry I still cannot help with proper patches to `guix pull`, I know it's too easy to just describe ideas than to implement it in code! [1] https://git.wiki.kernel.org/index.php/SoC2008Ideas#Restartable_Clone [2] https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-bundle.ht= ml [3] that means fix the url in the [remote "origin"]; we can easily document it in our manual =2D-=20 Giovanni Biscuolo Xelera IT Infrastructures --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJABAEBCgAqFiEERcxjuFJYydVfNLI5030Op87MORIFAmLGoO8MHGdAeGVsZXJh LmV1AAoJENN9DqfOzDkSkegQAKFc3jkjZ9qVL1JL7qfqBla7JA6k1tFbgky5KPL1 cVCNm93djjl0IzENGmE/ZsLgPXMoJPdNHcrcO37jtZliTTR8BrOnmy/94qZ1h7bf KJdqEaOG9FBteoI4trqeQ+1a1d8VQrnhMWLUZxl/E7P8AZnczRUD8Z1sfddW7qc6 T+D3zU0WGSyqpw95vwIFlp/Ar8dctb4btwJg8LTe3Hj73I1tLAA0ETlV9j9rf3KH PSrKpRKYGr0RvwIM/eBdAIDnAx8cLCOr6qd9q/phxgKO3hdMgq53+kJGm/+Biowz NSOKlnVc9XVsvnG15obIjgHq+UZQp2wyYMGlUtx4Jn2BMeyBcbn/zcZdMeqS0vgX 1ovWaraZqMPG6Q6il43koP6kk+DAK3FGcCtf5slCrMwCSDUfTDU29a15wyCSNq70 gXszEjQQpHTkl3oW2Ix6alKmVZV6SQCmpzCv9YWxBplPT+rnen1DMYWnEmynNgsM TIXqN6Ebt83z8efYvH5a/FYOi0iNVe+16MIOllsSk//N7glWAXcF+O94PHfTKDiG soqSP6KdEIj79JW3+OmPbwY4taWVbiq6ae/tG7UwmT9r7ChBJwvcb1RZEB0aTVn8 /4fqJExtfnuqtfAL36ccqLSmgzNxFY4m9F9wBp6JnJPNxEB25iiC4KNsqGUaVn4n 5+nB =pma1 -----END PGP SIGNATURE----- --=-=-=--