From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <guix-devel-bounces+larch=yhetil.org@gnu.org>
Received: from mp2.migadu.com ([2001:41d0:303:e16b::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by ms8.migadu.com with LMTPS
	id E7AaKEuHtGVwCAEAe85BDQ:P1
	(envelope-from <guix-devel-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Sat, 27 Jan 2024 05:32:11 +0100
Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	by mp2.migadu.com with LMTPS
	id E7AaKEuHtGVwCAEAe85BDQ
	(envelope-from <guix-devel-bounces+larch=yhetil.org@gnu.org>)
	for <larch@yhetil.org>; Sat, 27 Jan 2024 05:32:11 +0100
X-Envelope-To: larch@yhetil.org
Authentication-Results: aspmx1.migadu.com;
	dkim=pass header.d=philipmcgrath.com header.s=fm3 header.b=nLEj0C0x;
	dkim=pass header.d=messagingengine.com header.s=fm3 header.b="e e+R6zE";
	spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org";
	dmarc=none
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org;
	s=key1; t=1706329931;
	h=from:from:sender:sender:reply-to:subject:subject:date:date:
	 message-id:message-id:to:to:cc:cc:mime-version:mime-version:
	 content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding:
	 in-reply-to:in-reply-to:references:references:list-id:list-help:
	list-unsubscribe:list-subscribe:list-post:dkim-signature:autocrypt:autocrypt;
	bh=hUKkivB5bU0Zbw2kYQnvfl5UJSw+70GV78ey1XDbf8M=;
	b=Mhq7BnJIY20Xyq0p0W9EiULs3b/iRSUtLGhfd123XWzHl5IdPf2Yki7smFEJi3mtqdE8Qq
	lqWZYsHQSz8rsKGJMwNygomOpTQ2NW1pOFcjZQklYA5ywzi7hr1aaaTLLji7At5SBcpj03
	nVRyE0uokp9NHzKX58KVoMSOraXqAsvYBxVsG82bIzfyav5GiShHhz+dVyy/cpEeVGlYK6
	QIzU/NQZm1MnzeUeh2XTMboFjqMNuGFe8uy6viidsBzYrQHYSQSPKDDF8R+7bH/UZTfJbQ
	bN61q+O95d+/b8yKM711Dg+WqDhQtvU2sCoXiHGAiAh/jJic+UHfzpE0/5ilEw==
ARC-Authentication-Results: i=1;
	aspmx1.migadu.com;
	dkim=pass header.d=philipmcgrath.com header.s=fm3 header.b=nLEj0C0x;
	dkim=pass header.d=messagingengine.com header.s=fm3 header.b="e e+R6zE";
	spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org";
	dmarc=none
ARC-Seal: i=1; s=key1; d=yhetil.org; t=1706329931; a=rsa-sha256; cv=none;
	b=V009f56cnHFh6trrZtH9wG/6hcpIgvzcS3HZXQ8J8tTh1xv0LJVAUr1patuhajjPLQyM4h
	ps4fa47r6aoSrnIjMQSqOKOD/BoYKoeoxHC10gobd0ZWbL9i3LTwzL+eNARfFYcv3tFabj
	UvjxxMEkpsKmc4dN9fGm3bXw7zhVpxz54/wqlD/bdg1/LOqBwTONNsyaphlU8KWSBgDJ3p
	NiPaWo+VTn0Qb7I63MaS+z8XyTai07DXZcE71PPx7BrP+FqSCPcYaFgJIrhh2G5iHWJzsw
	2d2OW1vkzvRj8ytTYvSYUmwGngB1R+QXCdJdzLCHik33LjFUn8mpokebzjZdhA==
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by aspmx1.migadu.com (Postfix) with ESMTPS id 71FF65342F
	for <larch@yhetil.org>; Sat, 27 Jan 2024 05:32:11 +0100 (CET)
Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <guix-devel-bounces@gnu.org>)
	id 1rTaLv-0004qZ-Ed; Fri, 26 Jan 2024 23:31:35 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <philip@philipmcgrath.com>)
 id 1rTaLt-0004oh-2K; Fri, 26 Jan 2024 23:31:33 -0500
Received: from out5-smtp.messagingengine.com ([66.111.4.29])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <philip@philipmcgrath.com>)
 id 1rTaLq-0006Rv-U7; Fri, 26 Jan 2024 23:31:32 -0500
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
 by mailout.nyi.internal (Postfix) with ESMTP id 1B3DE5C00D0;
 Fri, 26 Jan 2024 23:31:27 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163])
 by compute5.internal (MEProxy); Fri, 26 Jan 2024 23:31:27 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 philipmcgrath.com; h=cc:cc:content-transfer-encoding
 :content-type:content-type:date:date:from:from:in-reply-to
 :in-reply-to:message-id:mime-version:references:reply-to:subject
 :subject:to:to; s=fm3; t=1706329887; x=1706416287; bh=hUKkivB5bU
 0Zbw2kYQnvfl5UJSw+70GV78ey1XDbf8M=; b=nLEj0C0xCWvHIHBbc7TsSgo1vz
 6Rk7z0Czu9qkoaheZP3x3QtELl4npzhuimChscz35Y8qjdinl70GYUKodnE4X4hj
 Uo+Yd7noTeLg1jkTyMl6kVm9Ia8u9w2HqIsZw6Q6N2e4dB2RzUaq4TZ8pHCBaNtL
 /gXmiXI16OCi6v30A3sALesjSoeR6KhQ5sCqcxKOa1mI8aaK1WxGSbWmPC5uhOBc
 nEEncL+e8IjeSTTw/XjuWKl81WTuTn0XQuRvMBRdD5EsRtWDpw9bv0UO77FdQXNY
 SS7BCgHWP0ei37h+6DZODqGCxgQxU7CLO4ltgAqudgdiEAz1NXTbjARC/75A==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:cc:content-transfer-encoding
 :content-type:content-type:date:date:feedback-id:feedback-id
 :from:from:in-reply-to:in-reply-to:message-id:mime-version
 :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy
 :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1706329887; x=
 1706416287; bh=hUKkivB5bU0Zbw2kYQnvfl5UJSw+70GV78ey1XDbf8M=; b=e
 e+R6zEjvXDQS4Nw4ME+zS+Oa+vuxeQeTzNXsWo0Hh5sv0Xikqml3bxWdQWYw8KXE
 MK0iuSigKnaoKWLqA5RRiQF/xU7gtyXTmpso7qQ7v+KVzALK3NkXUZrCtqXI/HU8
 XI+IG475fkx5zRZfHm7UmfU7F5XzGaNfiA8BInFh9xndruzuBYxNI42XQYGdDBT8
 d1VeCAb1cUQhaHoAzmzl4BaTYUCSaxJvstrcWuJWz5VWj1xAA1eQUtDJMOVNHVeB
 RmejYj0uHAIMu8cKYGJnkRC5mA+NJbdCu8brDgNUbE+oNzvxU7id9ViM3K8zrNLD
 id0AcdW8JdTrQx6zA/cIg==
X-ME-Sender: <xms:Hoe0ZXyE1uhMuynQf57qVsNv9dLuvJbu8KLjRTYeC_qvKoseTmVhpg>
 <xme:Hoe0ZfTso9BwkVwZv8jLwSdFUbDuHBW-Z87pEhOcHIhXyDRv3eTHZy-1IAzcF8k8X
 yW0_dF9JRN275qjK44>
X-ME-Received: <xmr:Hoe0ZRV0_xxuU0dxJ_w4A-JPUMJL090g4MT7t_v4JiP-uRtG8m7mVKzG8TDXiDJL__TfUyyRKqL_zXy2vF8-8g2QPo0QZGNFmes9Nw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdelkedgjeduucetufdoteggodetrfdotf
 fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
 uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
 cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtvdejnecuhfhrohhmpefrhhhi
 lhhiphcuofgtifhrrghthhcuoehphhhilhhiphesphhhihhlihhpmhgtghhrrghthhdrtg
 homheqnecuggftrfgrthhtvghrnhepheelieekvefhgeejgfejhffgveffjeehffdtheeg
 tdejjedvgeehfeehheffieejnecuffhomhgrihhnpehgihhtqdhstghmrdgtohhmnecuve
 hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhhihhlihhp
 sehphhhilhhiphhmtghgrhgrthhhrdgtohhm
X-ME-Proxy: <xmx:Hoe0ZRi3TlnL1dVUrMh8bk88H2tquLaV8JpNny3VrtWgttqR1yAqBQ>
 <xmx:Hoe0ZZBJZ5WitRz7uy5CaZh12zSiqlkPXGtcaOAx9QxvW3QMNv-jEg>
 <xmx:Hoe0ZaJOuJp06iV0zru3xFNIVaC-gdH1dqxOHVg4TK9cpywcM7dFUA>
 <xmx:H4e0Zb0AHv_X5Nk3E15Wj5dy1DgB54Q3R3W7pMA1BtomC8-KaycmhA>
Feedback-ID: i2b1146f3:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri,
 26 Jan 2024 23:31:26 -0500 (EST)
Message-ID: <0452e9d6-7954-42e0-836f-83dcc9487b73@philipmcgrath.com>
Date: Fri, 26 Jan 2024 23:31:24 -0500
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Git-LFS or Git Annex?
To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= <ludo@gnu.org>,
 guix-sysadmin <guix-sysadmin@gnu.org>
Cc: guix-devel@gnu.org, morganlemmerwebber@gmail.com
References: <87mssuu57m.fsf@inria.fr>
Content-Language: en-US
From: Philip McGrath <philip@philipmcgrath.com>
Autocrypt: addr=philip@philipmcgrath.com; keydata=
 xsFNBFxfDXMBEAC50KOX7FVfbH5kzv1IWAu58hz77HfgH46izt8+3X51/AzD0QsrVnzz3fS0
 37e5wnB4hpT/a6iGPbaah02nOhrCWjllWQrnp7rb0q9w8rnA41W5Aeajsob3/mPlSTkgX77/
 nfXHFSaIPi+6DseAQO6EMbQrA/Ue4Yal/LWaAcpmGeAXUQabvH1I7LY5f6RI/pMii5cVmqm/
 ea/oVroVeT4lBLh/1jEKv+tCVn1h9ks5L5AcbumaenEjgtJZ22/WUNowLD21uug7WCHlVdf+
 3PKSg/Hd7jotxYrfbhIEH2494KnenslxX4psavsH/AAWWTouvdiY5Q2d7eDH6P6ggWa1fXgs
 LlxFgtTTOmNTKViBKYk/g6gE2DpTaNG2pPqblxVlgy751O9y5UTBET0zj75BY55dKzoKg541
 hBwKuwhNj0fA3xKesYzs2VdrzredEHegPeucQfTBSQ0G8lNdofElyv1e92rJtkAJXpodwNJf
 gmfJpdP33PPd1a7WQwWbo96JAkTkYnisEQyvJbQ8AvJw7V9xY79/pd2dXJkz88HZP486UiSK
 RmO9kItVSjCYz2Tqv75aZocpC661fv66GuLZfOCR1I0nbXNFxNs6K3/otJ3ignDFJCD4wk6j
 TlSODDeeXxDJtyBBrnYIzgb86KI3RJxoKrfbos0oavfWA2NGpwARAQABzSlQaGlsaXAgTWNH
 cmF0aCA8cGhpbGlwQHBoaWxpcG1jZ3JhdGguY29tPsLBlAQTAQgAPgIbAwULCQgHAgYVCgkI
 CwIEFgIDAQIeAQIXgBYhBPRlq6zWN66sFBVVz8oDY436Pxx6BQJkZmCHBQkJlXgUAAoJEMoD
 Y436Pxx6JvoQAKufvt1nAdezaIj+LccaKMOCNnVFxXBIDyaU9+Gmqh1ANQGLpdljfFmecpWm
 1SA3sqDZapF2zg/hMHWbDGvdHJnPbtfnfS7ZZtOpdgRtMpE4zVeSi4t2NcIV0wQiPBXZhR0O
 Qp2PCMsExunJ0DV1pPMEt5r825haNPXZErSM3ectFgJwxjxEeOZ7khmrVrWZe0OddSsEEQsn
 9jxpftLRgE/6N31wDuM/BDIrz+6AOc98zKFIoMmv8z9p1EEOvoFs7WunJklpWq6W4FggIPT0
 UxlycFQBxzpVj2ga6MZVZ8Sp2fq0jgKn3uubYySyNi8SE6PfSre+QIG54OA+j9NSiVi36um7
 Z4nqRC6JPoBe2VTwaZLskeO4grYHKFZHKVyOk3jcvDY5R8uyh+hPPj5kvYS34YFESZf+5LTA
 HoYmaOY8YGEqH66p9ukYbXaAoH9lmQig0zVcyyE9o8OuKhkvksiJ+2pEYtWXOkVqvj18xlfQ
 mLfhlWYo/o42x975zACjrZQe5+ZdQS9sz7ietWrHbZ9+hl79iSfeVkNvMikdZ5CZ81YvF42z
 kpDr6vWc2ImxIpgCTRp5HAUtKVsUJE/+yJop+AeVMonzzGaFd5tG/aT5BOz3w6ua3WfXqA9b
 NpeO4ANi2IQrhhWNq26JWx3Yx/QgxrNm44skIzuEEyL3b6Q7zsFNBFxfDXMBEADq/Vlop4lx
 W9Bi7cYaxjnhNMbDxR3aVV3GMy7zkuV3fGjvTAuO7raDoqN91y2bvm+vkM54Mw+Fu3ThBHCc
 niYkdgyIlpLNfFhQOOBtlj1jNmCGTOp7JBJ8mjQucOGy3A0uUaqvkMncO2trsfEsRD25nTZV
 dji4cHePP86Uc/w98OBt5cPLyM5/WU+GTpPBATFc1u4DocMSqbCZcNLDr1O7P6Knl8vsCtRj
 r4X4xp3d3I+aZ1h5l/Hu20u+/7fg8PO/WDOCMwqN64xB2OHLv7bQm0piwVN+Gkyjhk9EJINO
 m05ieGxvsyAaQDMkgvm/Hp1G2LJEWkxaTalnBawaVYM4TgpE5hU9gda3Ld/pdT2lkNmVq47v
 d//NBYWa1/aJGqBP2XlW0JMywmDEVH2fFwVqMQ3ZzE+N4HnSR6Uslv4DpIeYLGevheHLqegO
 o5VySo/P+lGQDO0vSHbBZgRJ/ROwNvkJVEQ9pI34Edt2zCfUYqjUU+3tcd2xgR8/JP2dnjYp
 NDq4CK/8ywk+sl8WY8MQpKccJdWp0CVgp2y88YALn6x9wxsPGQ8Lk7sB4ADwwPX87pFdJqe2
 2oKgG9HklBTSxvdr1fN+/n4AD5XPBhMpi9UZHdaVuWzVfXU6EKMHlsJfJ7sF1atKVPcQDIN/
 SxwTlxmsHVCMxifikklhQwmD/wARAQABwsF8BBgBCAAmAhsMFiEE9GWrrNY3rqwUFVXPygNj
 jfo/HHoFAmNsLkkFCQmC/B0ACgkQygNjjfo/HHrGqxAAgdV2GRCEIJwbLtRTv4qpfANsfX/I
 S0KZBDbwtZuCDceQTfsS8lF5WGJOPxF7MDkVGRhBPYpVamO56V+Y/o6drX0JEeWqLwB3nc8b
 PFpFcGTFOxp6Q1Ps4n/L/+hhTsI4vO79jsf97s7GPhSLpnzakb2Enh25G/MoToj6qQvuuqFW
 lDX1hxBC9Jf6pnuX1XGUik9LZVpuqf9suVgNa4Wf6ZverzphPWObXifpjVREk7jJ0g2OBGZz
 CATDfgEfu4OtV2JEeSrPZeFoHPQzsFfO/n99ZxBWVLbiXJgNTnU/vwbPkMpPjED3WSwTesW+
 SgXfZf4KHFigC3Z34uhXfxn4I1XEW7UGgbtsrFqJI+WVzP2mUeOzNoz980N1tfowzoNShfKD
 rcDhtpyZZcaA+0O1oVSKs3LUT4QNiYl/lcg4l0qnBvj+ilG3ViXikhKn4RMJUppbXCx09ZBK
 Pz1PAvig0mnpCWCjizdbtuFzkZsLmfYjIbxkXBLMmQdqMgqUnsUGvUh0u+v5zQlc3IQnpLBt
 vkQuitsVuqkCKSILz8p2DyA6PWRI988fGPakjvV13iGwAIfFPsNvOoWsVLz3GT6f9chF4fAY
 JuOem41lA82RZB9fPty0S8YVavucU7qzTfHRj6LwvrWzLk6wRA7ThPHWVAZsyH1iYO+nUOsU
 FFu7icw=
In-Reply-To: <87mssuu57m.fsf@inria.fr>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=66.111.4.29;
 envelope-from=philip@philipmcgrath.com; helo=out5-smtp.messagingengine.com
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001,
 SPF_HELO_PASS=-0.001, SPF_PASS=-0.001,
 T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: guix-devel@gnu.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Development of GNU Guix and the GNU System distribution."
 <guix-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/guix-devel>,
 <mailto:guix-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/guix-devel>
List-Post: <mailto:guix-devel@gnu.org>
List-Help: <mailto:guix-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/guix-devel>,
 <mailto:guix-devel-request@gnu.org?subject=subscribe>
Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org
Sender: guix-devel-bounces+larch=yhetil.org@gnu.org
X-Migadu-Flow: FLOW_IN
X-Migadu-Country: US
X-Migadu-Spam-Score: -8.74
X-Migadu-Scanner: mx13.migadu.com
X-Spam-Score: -8.74
X-Migadu-Queue-Id: 71FF65342F
X-TUID: 6o7fF7sxGQVi

Hi,

On 1/24/24 10:22, Ludovic Courtès wrote:
> 
> The question boils down to: Git-LFS or Git Annex?
> 
> [...]
> 
> What’s your experience?  What would you suggest?
> 

I have a few times had a problem for which I thought Git LFS might be a 
solution, and each time I have ended up ripping out Git LFS in 
frustration before long.

I have not used Git Annex. I have looked into it a few times, but each 
time I decided it was too complex or not quite suitable for my use-case 
in some way. On the other hand, I have heard good things about it from 
people who have used it: in particular, I believe Morgan Lemmer-Webber 
(CC'ed) used it to manage a large set of art history images.

The main thing in this context that still isn't clear to me from by 
reading so far is how sharing lists of remotes works with Git Annex. In 
plain Git, remotes are part of the local state of a particular clone, 
not distributed as part of the repository. For the objectives here, 
though, a lot of the benefit would seem to be having many copies in 
synchronized, possibly "special" remotes so that anyone trying to get 
the videos would have plenty of ways to get them. I'm not sure to what 
extent Git Annex does that out of the box.

I did see that Git Annex can use Git LFS as a "special remote".

There are also two other approaches I think would be worth at least 
considering:

1. Just use Git

While the limitations of Git for storing large media files are well 
known, I have found it to be good enough for several use-cases, and it 
has the strong advantage of not requiring additional tools. My 
impression is that a significant factor in people using Git LFS, in 
particular, is the limit on repository size imposed by the popular 
hosting providers. There are strategies within Git to avoid having to 
download unwanted artifacts, including creating branches with unrelated 
histories, shallow clones (e.g. --depth=1 --single-branch), partial 
clones [1][2][3] (e.g. --filter=blob:none), and sparse checkouts [4][5], 
with the later two being fairly new features.

[1]: https://git-scm.com/docs/partial-clone
[2]: 
https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---filterltfilter-specgt
[3]: 
https://git-scm.com/docs/git-rev-list#Documentation/git-rev-list.txt---filterltfilter-specgt
[4]: https://git-scm.com/docs/git-sparse-checkout
[5]: https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---sparse

2. Mirror URLs

Another approach would be just to make each video available at a few 
URLs and have Guix origins with the list. If one of the available URLs 
were the Internet Archive, it would have a high degree of assurance of 
long-term preservation. I think the biggest downside is that this might 
not help much with managing the collection of videos.

Philip