From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id 9983Ei9nsWUVaQEA62LTzQ:P1 (envelope-from ) for ; Wed, 24 Jan 2024 20:38:23 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id 9983Ei9nsWUVaQEA62LTzQ (envelope-from ) for ; Wed, 24 Jan 2024 20:38:23 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=iP9npT4p; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1706125103; a=rsa-sha256; cv=none; b=iHpyplyscRghIQtBO5c8HN55i0SoOBSfRHOPC/7b80YhxGK5j3h5ixFnAebrA3Mjc3finG DLUh3har5f2RKPEnEd9bjMOuh9WMFq8GVT0AuFWup946WGkmZJc4jsYBEM9bFHKmTVGB3x 3bPoWNn2aacj67dBJIA/VS6qLO5ZelrOef/1vnBQFYZ1mPVupsLbWVn6nL8T1zWIrEYZBV ZARMsM1QKKMSilfAyHOLj4VMHXATWMg0h09MnYLBGDZPb17f3YvIfGBmWg3fn4CGGlTSEl R8c1Z7GOAW3PiCLcnzwAUlLzSOKgZX5YVVYdcNLlw9K4bzO14llcF8yyIjoOwg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=iP9npT4p; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1706125103; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=3daHlr6GRzrfzPvo6seJThLfbCZQx2TUk9T1N9B+mAk=; b=gNXO9qEf4+IWtZOvzKh1ZOW9XV8brZbRiw0VYeudL8cnv0FkZ42kcBAxMyK7sJv+epgbuY kEQG2Eipnc/xUtTSOXqVI3H8R4DA1I6EdaIrDSaugc1uJqTs10x/y7h2odsHe/iN55mM/h 5WYOWrIkH610QxYAjg2XFE982QfudGnNSoXDavepe9f/n+4I7di0Va6UXIMWsc8mlj59SB 8sf8SflYmtZTLcmcZYEiMycgewlm7xGjmn44szPiOcnhviA3/1ATPJ82LiTgv6nrGnlpZm XWonpYjyTPT1mqw0j31RO0KWuKtztBcezqgBLjL7Qx6qF3NRx4qMsBR71okp7Q== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 937D5BF66 for ; Wed, 24 Jan 2024 20:38:22 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rSj44-0005t9-Nm; Wed, 24 Jan 2024 14:37:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rSj42-0005sr-TT for guix-devel@gnu.org; Wed, 24 Jan 2024 14:37:34 -0500 Received: from mout01.posteo.de ([185.67.36.65]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rSj40-0000PA-EP for guix-devel@gnu.org; Wed, 24 Jan 2024 14:37:34 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id 0ED0724002A for ; Wed, 24 Jan 2024 20:37:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1706125049; bh=PhmesjIYveJpW43ai4EX2nHHaSVznaTa6tJ9nvu6E3E=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:Content-Type: Content-Transfer-Encoding:From; b=iP9npT4pGgR8FhofzdgU2PGmME3DGIdMV09JI6hnkRfG5oLSqZauGKxGG/PLWBCCa XgWVUuy1PYLAsGFV6KDXCRT79QwDuLvbXfmzFoV+pY5CnBDY4xAHQ14fLuI39Lh0qP T6jGfaWKlxcoLrPA9Jg3TrXsJuSmHS3fX34pdFvFlr32jhXW5nD+MIrkESx5GSnORo Eh5v+tA1jKJa2BmbvUIPwJO3JRD8HtczJfn5LwVAxE8R4SOmBWOxvx/Fx9lXGGAk31 vkxu0d2GXrXPjfNS+ySgxgaNm7QdBCdDD/BBYl4+Mgcfb3cDYuMDfJ0shI+mx7csJ4 HNxBtBjmBP7GA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4TKvP33vCwz6tw7; Wed, 24 Jan 2024 20:37:27 +0100 (CET) References: <87mssuu57m.fsf@inria.fr> From: pukkamustard To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-devel@gnu.org, guix-sysadmin@gnu.org Subject: Re: Git-LFS or Git Annex? Date: Wed, 24 Jan 2024 18:41:37 +0000 In-reply-to: <87mssuu57m.fsf@inria.fr> Message-ID: <86jznyy13e.fsf@posteo.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=185.67.36.65; envelope-from=pukkamustard@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Scanner: mx10.migadu.com X-Spam-Score: -9.39 X-Migadu-Queue-Id: 937D5BF66 X-Migadu-Spam-Score: -9.39 X-TUID: 374w6MqUcX89 Hi! Ludovic Court=C3=A8s writes: [..] > From a quick look (I haven=E2=80=99t used them), Git-LFS seems to assume a > rather centralized model where there=E2=80=99s an LFS server sitting next= to the > Git server=C2=B9. Git Annex looks more decentralized, allowing you to ha= ve > several =E2=80=9Cremotes=E2=80=9D, to check the status of each one, to sy= nc them, etc.=C2=B2 > Because of this, Git Annex seems to be a better fit. I agree that Git Annex seems to be a better fit for the reasons you list. > What=E2=80=99s your experience? What would you suggest? I've used Git Annex for managing many large files (~100s of GiBs) and it worked. However, I found Git Annex to be quite complex and to do things automatically without me fully realizing. The use case was to use Git Annex to distribute large test vectors. This involved many Git checkouts and worktrees on quite a few different hosts - some of them ephermal. When running `git annex sync` Git Annex tries to synchronize the current view of the state to all Git remotes (which file is available where) with a lot of git pushing and pulling. It ended up sharing remotes that are no longer existant or not-accessible and somehow it was hard/impossible to remove reference to those remotes (afaiu Git Annex remotes can only be marked as "dead" and not removed - https://git-annex.branchable.com/git-annex-dead/). As the number of such remotes increased, I became more and more confused. But this is maybe a special use-case which is not relevant for video sharing as you describe and probably reflects my inability to understand how Git Annex works more than anything else. Still, I would recommend to NOT store the videos in a remote Git repository but a publicly accessible rsync server as a Git Annex special remote (https://git-annex.branchable.com/special_remotes/). Cheers, pukkamustard