From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id UANaIJVLsWW4CwAA62LTzQ:P1 (envelope-from ) for ; Wed, 24 Jan 2024 18:40:37 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id UANaIJVLsWW4CwAA62LTzQ (envelope-from ) for ; Wed, 24 Jan 2024 18:40:37 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1706118037; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=9qJNKyaVvziA3XjW4lwrI+Fkt7+7te3qvCMq1Mo+DBU=; b=P3N0b8c8AzZy9yDkgurgtAMJLcfZeJhVZebX4IdJA6krXOt9nnGrIxrnGjju+hki/ar+qZ cmeHy1qFI0wtN5WxfPcLmSitjg7Ur9zXxwlI7LPRbaJMeTMdcr+3+3xPKu3C2cQ1M3Aqo5 COSQgB/zb90xxzW8oco5yFVMObAu/7nabn8O5uxpmmXSODwdFVBLn8c/8aMKQfEfsXmHny rwbj6/HBDagifdaPoZdylSOj5ncJcbHuk/2YHvKZ22x+pQV/YpOYWTsbnMa83bm/7k2RpS /Ie9BZPHsvdNYrIOTOusN02Z0+1OkQvbp/m5fKwKpV5TlQt/N9GJfD4OYC9JVQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Seal: i=1; s=key1; d=yhetil.org; t=1706118037; a=rsa-sha256; cv=none; b=WyEbjTrFbIorZm8tLcoeITVhAA20DZ5VCUraJbzyCOR8/dgiZTuZx0J1JiEl3/it1JJyPX xXKZhO0W9G93EPzzbLHearObGOTY+dZIk0CeVF/aKs+GEmIroJl2quE0rk9nFr1rsr+Jfa KNO8z8qdEa2ozXYgZIyzq0NKRKDv5jb8nlW/seu/P/PySzlc7v7lpcW0whPU2eABjYgbUT xDTejjhwGlQjnfn4fFJXxvo9TmlpGSR0Zk+Xh1vvS1EfK0UKTiMfHyRfs8r9uVE0BdmpQT zcKBBhutY7l4w25gqIsT23+CareQWLTHsp/LKxL0sf0uqcOgdWQjqd++2mWkUQ== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 37A1E6A509 for ; Wed, 24 Jan 2024 18:40:37 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rShDm-0002OZ-F4; Wed, 24 Jan 2024 12:39:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rShDe-0002F0-LY; Wed, 24 Jan 2024 12:39:24 -0500 Received: from ns13.heimat.it ([46.4.214.66]) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rShDb-0002BV-2a; Wed, 24 Jan 2024 12:39:21 -0500 Received: from localhost (ip6-localhost [127.0.0.1]) by ns13.heimat.it (Postfix) with ESMTP id CC23C3007E0; Wed, 24 Jan 2024 17:39:14 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at ns13.heimat.it Received: from ns13.heimat.it ([127.0.0.1]) by localhost (ns13.heimat.it [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r2vtPgqWARUZ; Wed, 24 Jan 2024 17:39:12 +0000 (UTC) Received: from bourrache.mug.xelera.it (unknown [93.56.171.217]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by ns13.heimat.it (Postfix) with ESMTPSA id 5C12530022F; Wed, 24 Jan 2024 17:39:12 +0000 (UTC) Received: from roquette.mug.biscuolo.net (roquette [10.38.2.14]) by bourrache.mug.xelera.it (Postfix) with SMTP id D2A3A2DD2215; Wed, 24 Jan 2024 18:39:11 +0100 (CET) Received: (nullmailer pid 13954 invoked by uid 1000); Wed, 24 Jan 2024 17:39:10 -0000 From: Giovanni Biscuolo To: Ludovic =?utf-8?Q?Court=C3=A8s?= , guix-sysadmin Cc: guix-devel@gnu.org Subject: Re: Git-LFS or Git Annex? In-Reply-To: <87mssuu57m.fsf@inria.fr> Organization: Xelera.eu References: <87mssuu57m.fsf@inria.fr> Date: Wed, 24 Jan 2024 18:39:09 +0100 Message-ID: <87il3i8wci.fsf@xelera.eu> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=46.4.214.66; envelope-from=g@xelera.eu; helo=ns13.heimat.it X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -7.76 X-Migadu-Queue-Id: 37A1E6A509 X-Spam-Score: -7.76 X-Migadu-Scanner: mx11.migadu.com X-TUID: xzEzWl2Q6lYn --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo=E2=80=99 Ludovic Court=C3=A8s writes: [...] > The question boils down to: Git-LFS or Git Annex? > > From a quick look (I haven=E2=80=99t used them), Git-LFS seems to assume a > rather centralized model where there=E2=80=99s an LFS server sitting next= to the > Git server=C2=B9. Git Annex looks more decentralized, allowing you to ha= ve > several =E2=80=9Cremotes=E2=80=9D, to check the status of each one, to sy= nc them, etc.=C2=B2 > Because of this, Git Annex seems to be a better fit. I've never used Git-LFS for my media repository (and will never use it, never). AFAIK this two advantages of git-annex vs Git-LFS are still valid today: =2D-8<---------------cut here---------------start------------->8--- A major advantage of git annex is that you can choose which file you want to download. You still know which files are available thanks to the symlinks. For example suppose that you have a directory full of ISO files. You can list the files, then decide which one you want to download by typing: git annex get my_file. Another advantage is that the files are not duplicated in your checkout. With LFS, lfs files are present as git objects both in .git/lfs/objects and in your working repository. So If you have 20 GB of LFS files, you need 40 GB on your disk. While with git annex, files are symlinked so in this case only 20 GB is required. =2D-8<---------------cut here---------------end--------------->8--- (https://stackoverflow.com/a/43277071, 2018-10-23) So, AFAIU, with Git-LFS you can have all your media or no media, you cannot selectively choose what media to get. Another important limitation of Git-LFS is that you cannot delete (remotely stored) objects [1], with git-annex is very easy. > Data point: guix.gnu.org source is hosted on Savannah, which doesn=E2=80= =99t > support Git-LFS; to host a Git-LFS service a Git-LFS server implementation (one that reply to GIT_LFS API) is needed: https://github.com/git-lfs/git-lfs/wiki/Implementations AFAIU we dont't have one packaged (I'd save some precious time trying to package one of them). AFAIK Savannah do not support git-annex also, so we need to set up a Guix git-annex server [3], I suggest using gitolite [4]: I can help with this task if needed! > the two other web sites above are hosted on GitLab instances, which I > think do support Git-LFS. Yes, Git-LFS is supported on GitLab.com and included in the Community Edition [2] since late 2015. git-annex repository support was available on GitLab.com in 2015/16 but was removed in 2017 [5] > What=E2=80=99s your experience? What would you suggest? I've no experience with Git-LFS (and will never have) but from what I read I definitely suggest git-annex: it's more efficient, it's more flexible, can be hosted everywhere with a little bit of effort... can be hosted on a Guix System host! :-) As a bonus, git-annex have a plenty of super cool features that will make us very happy, i.e.: =2D special remotes: https://git-annex.branchable.com/special_remotes/ (including rclone https://git-annex.branchable.com/special_remotes/rclone/) =2D location tracking (https://git-annex.branchable.com/location_tracking/) =2D manage metadata of annexed files HTH! Gio' > Thanks, > Ludo=E2=80=99. > > =E2=81=B0 https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hyd= ra/berlin.scm#n193 > =C2=B9 https://github.com/git-lfs/git-lfs/wiki/Tutorial > =C2=B2 https://git-annex.branchable.com/walkthrough/ [1] https://github.com/git-lfs/git-lfs/wiki/Limitations [2] GitLab Community Edition [3] https://git-annex.branchable.com/tips/centralized_git_repository_tutorial/o= n_your_own_server/ [4] https://git-annex.branchable.com/tips/using_gitolite_with_git-annex/ [5] https://about.gitlab.com/blog/2015/02/17/gitlab-annex-solves-the-proble= m-of-versioning-large-binaries-with-git/ =2D-=20 Giovanni Biscuolo Xelera IT Infrastructures --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJABAEBCgAqFiEERcxjuFJYydVfNLI5030Op87MORIFAmWxSz0MHGdAeGVsZXJh LmV1AAoJENN9DqfOzDkSk0cQALsrdyL8iLlY/Dg1IbkeGhCix4WBN8/ltGMB+wNE oAcp8wDKbDAJG/XJumvpOxPGMGJNFudLQ9sckO8EwdH8pTaefwV0SP16035BSBLu GuSgxjsXLxJc6d1uleMcqks/4JCCBKhRDPEWxofcVn1VUmPqGaN/+fAvsG3rSv0u TcC8vP1CKmpJRmousInB37jGHt0slzH8Ukt/1ADAiO2P6T3kmQHxlGOrskloWi07 oAZxlbxQYTLPOEkVX7BsqXmQdSDjYKzIJw0DIejfJwSFud09nx6WD8Kvc8J87o6Z uCmYXkF1LKAWB1TxacjNBnP9p7xITmXDJuJ/6nYrZvuk5u7+JC2gfKBazCGKbNzP ZIKt9lXBNku3fNG9yi8oImijz6YZ1fq2eFwuyS01K83Oh2B5suGPvicwrvuItH0V 59FV/FQyAxZRmN6mn7ZZum3KwGVO49fyI3lW4vH/Jv6h6toOVZqgai35Kcpm8xnJ p/UxsKD+5azuBVDQeeqs4oa/PhVvjmYJFW/Um7s0sGCHqE3bS7P0Hck4T13hSCOz NnFWkDtzkfetX7rnIutdKWB1Q0MDTfybVZSSLeut3CCAs8fAroQC6qCYQ04iQPUk tTBb5VN6VFa5dBQBOhnSpZ6CNM5nkIlLa52zU5adQy53cHqmIQuQ3FpL17jbO2fk /oFm =WCSx -----END PGP SIGNATURE----- --=-=-=--