From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id E7AaKEuHtGVwCAEAe85BDQ:P1 (envelope-from ) for ; Sat, 27 Jan 2024 05:32:11 +0100 Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id E7AaKEuHtGVwCAEAe85BDQ (envelope-from ) for ; Sat, 27 Jan 2024 05:32:11 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=philipmcgrath.com header.s=fm3 header.b=nLEj0C0x; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="e e+R6zE"; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1706329931; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature:autocrypt:autocrypt; bh=hUKkivB5bU0Zbw2kYQnvfl5UJSw+70GV78ey1XDbf8M=; b=Mhq7BnJIY20Xyq0p0W9EiULs3b/iRSUtLGhfd123XWzHl5IdPf2Yki7smFEJi3mtqdE8Qq lqWZYsHQSz8rsKGJMwNygomOpTQ2NW1pOFcjZQklYA5ywzi7hr1aaaTLLji7At5SBcpj03 nVRyE0uokp9NHzKX58KVoMSOraXqAsvYBxVsG82bIzfyav5GiShHhz+dVyy/cpEeVGlYK6 QIzU/NQZm1MnzeUeh2XTMboFjqMNuGFe8uy6viidsBzYrQHYSQSPKDDF8R+7bH/UZTfJbQ bN61q+O95d+/b8yKM711Dg+WqDhQtvU2sCoXiHGAiAh/jJic+UHfzpE0/5ilEw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=philipmcgrath.com header.s=fm3 header.b=nLEj0C0x; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="e e+R6zE"; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Seal: i=1; s=key1; d=yhetil.org; t=1706329931; a=rsa-sha256; cv=none; b=V009f56cnHFh6trrZtH9wG/6hcpIgvzcS3HZXQ8J8tTh1xv0LJVAUr1patuhajjPLQyM4h ps4fa47r6aoSrnIjMQSqOKOD/BoYKoeoxHC10gobd0ZWbL9i3LTwzL+eNARfFYcv3tFabj UvjxxMEkpsKmc4dN9fGm3bXw7zhVpxz54/wqlD/bdg1/LOqBwTONNsyaphlU8KWSBgDJ3p NiPaWo+VTn0Qb7I63MaS+z8XyTai07DXZcE71PPx7BrP+FqSCPcYaFgJIrhh2G5iHWJzsw 2d2OW1vkzvRj8ytTYvSYUmwGngB1R+QXCdJdzLCHik33LjFUn8mpokebzjZdhA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 71FF65342F for ; Sat, 27 Jan 2024 05:32:11 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rTaLv-0004qZ-Ed; Fri, 26 Jan 2024 23:31:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rTaLt-0004oh-2K; Fri, 26 Jan 2024 23:31:33 -0500 Received: from out5-smtp.messagingengine.com ([66.111.4.29]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rTaLq-0006Rv-U7; Fri, 26 Jan 2024 23:31:32 -0500 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 1B3DE5C00D0; Fri, 26 Jan 2024 23:31:27 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Fri, 26 Jan 2024 23:31:27 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= philipmcgrath.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1706329887; x=1706416287; bh=hUKkivB5bU 0Zbw2kYQnvfl5UJSw+70GV78ey1XDbf8M=; b=nLEj0C0xCWvHIHBbc7TsSgo1vz 6Rk7z0Czu9qkoaheZP3x3QtELl4npzhuimChscz35Y8qjdinl70GYUKodnE4X4hj Uo+Yd7noTeLg1jkTyMl6kVm9Ia8u9w2HqIsZw6Q6N2e4dB2RzUaq4TZ8pHCBaNtL /gXmiXI16OCi6v30A3sALesjSoeR6KhQ5sCqcxKOa1mI8aaK1WxGSbWmPC5uhOBc nEEncL+e8IjeSTTw/XjuWKl81WTuTn0XQuRvMBRdD5EsRtWDpw9bv0UO77FdQXNY SS7BCgHWP0ei37h+6DZODqGCxgQxU7CLO4ltgAqudgdiEAz1NXTbjARC/75A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1706329887; x= 1706416287; bh=hUKkivB5bU0Zbw2kYQnvfl5UJSw+70GV78ey1XDbf8M=; b=e e+R6zEjvXDQS4Nw4ME+zS+Oa+vuxeQeTzNXsWo0Hh5sv0Xikqml3bxWdQWYw8KXE MK0iuSigKnaoKWLqA5RRiQF/xU7gtyXTmpso7qQ7v+KVzALK3NkXUZrCtqXI/HU8 XI+IG475fkx5zRZfHm7UmfU7F5XzGaNfiA8BInFh9xndruzuBYxNI42XQYGdDBT8 d1VeCAb1cUQhaHoAzmzl4BaTYUCSaxJvstrcWuJWz5VWj1xAA1eQUtDJMOVNHVeB RmejYj0uHAIMu8cKYGJnkRC5mA+NJbdCu8brDgNUbE+oNzvxU7id9ViM3K8zrNLD id0AcdW8JdTrQx6zA/cIg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdelkedgjeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtvdejnecuhfhrohhmpefrhhhi lhhiphcuofgtifhrrghthhcuoehphhhilhhiphesphhhihhlihhpmhgtghhrrghthhdrtg homheqnecuggftrfgrthhtvghrnhepheelieekvefhgeejgfejhffgveffjeehffdtheeg tdejjedvgeehfeehheffieejnecuffhomhgrihhnpehgihhtqdhstghmrdgtohhmnecuve hluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepphhhihhlihhp sehphhhilhhiphhmtghgrhgrthhhrdgtohhm X-ME-Proxy: Feedback-ID: i2b1146f3:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 26 Jan 2024 23:31:26 -0500 (EST) Message-ID: <0452e9d6-7954-42e0-836f-83dcc9487b73@philipmcgrath.com> Date: Fri, 26 Jan 2024 23:31:24 -0500 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Git-LFS or Git Annex? To: =?UTF-8?Q?Ludovic_Court=C3=A8s?= , guix-sysadmin Cc: guix-devel@gnu.org, morganlemmerwebber@gmail.com References: <87mssuu57m.fsf@inria.fr> Content-Language: en-US From: Philip McGrath Autocrypt: addr=philip@philipmcgrath.com; keydata= xsFNBFxfDXMBEAC50KOX7FVfbH5kzv1IWAu58hz77HfgH46izt8+3X51/AzD0QsrVnzz3fS0 37e5wnB4hpT/a6iGPbaah02nOhrCWjllWQrnp7rb0q9w8rnA41W5Aeajsob3/mPlSTkgX77/ nfXHFSaIPi+6DseAQO6EMbQrA/Ue4Yal/LWaAcpmGeAXUQabvH1I7LY5f6RI/pMii5cVmqm/ ea/oVroVeT4lBLh/1jEKv+tCVn1h9ks5L5AcbumaenEjgtJZ22/WUNowLD21uug7WCHlVdf+ 3PKSg/Hd7jotxYrfbhIEH2494KnenslxX4psavsH/AAWWTouvdiY5Q2d7eDH6P6ggWa1fXgs LlxFgtTTOmNTKViBKYk/g6gE2DpTaNG2pPqblxVlgy751O9y5UTBET0zj75BY55dKzoKg541 hBwKuwhNj0fA3xKesYzs2VdrzredEHegPeucQfTBSQ0G8lNdofElyv1e92rJtkAJXpodwNJf gmfJpdP33PPd1a7WQwWbo96JAkTkYnisEQyvJbQ8AvJw7V9xY79/pd2dXJkz88HZP486UiSK RmO9kItVSjCYz2Tqv75aZocpC661fv66GuLZfOCR1I0nbXNFxNs6K3/otJ3ignDFJCD4wk6j TlSODDeeXxDJtyBBrnYIzgb86KI3RJxoKrfbos0oavfWA2NGpwARAQABzSlQaGlsaXAgTWNH cmF0aCA8cGhpbGlwQHBoaWxpcG1jZ3JhdGguY29tPsLBlAQTAQgAPgIbAwULCQgHAgYVCgkI CwIEFgIDAQIeAQIXgBYhBPRlq6zWN66sFBVVz8oDY436Pxx6BQJkZmCHBQkJlXgUAAoJEMoD Y436Pxx6JvoQAKufvt1nAdezaIj+LccaKMOCNnVFxXBIDyaU9+Gmqh1ANQGLpdljfFmecpWm 1SA3sqDZapF2zg/hMHWbDGvdHJnPbtfnfS7ZZtOpdgRtMpE4zVeSi4t2NcIV0wQiPBXZhR0O Qp2PCMsExunJ0DV1pPMEt5r825haNPXZErSM3ectFgJwxjxEeOZ7khmrVrWZe0OddSsEEQsn 9jxpftLRgE/6N31wDuM/BDIrz+6AOc98zKFIoMmv8z9p1EEOvoFs7WunJklpWq6W4FggIPT0 UxlycFQBxzpVj2ga6MZVZ8Sp2fq0jgKn3uubYySyNi8SE6PfSre+QIG54OA+j9NSiVi36um7 Z4nqRC6JPoBe2VTwaZLskeO4grYHKFZHKVyOk3jcvDY5R8uyh+hPPj5kvYS34YFESZf+5LTA HoYmaOY8YGEqH66p9ukYbXaAoH9lmQig0zVcyyE9o8OuKhkvksiJ+2pEYtWXOkVqvj18xlfQ mLfhlWYo/o42x975zACjrZQe5+ZdQS9sz7ietWrHbZ9+hl79iSfeVkNvMikdZ5CZ81YvF42z kpDr6vWc2ImxIpgCTRp5HAUtKVsUJE/+yJop+AeVMonzzGaFd5tG/aT5BOz3w6ua3WfXqA9b NpeO4ANi2IQrhhWNq26JWx3Yx/QgxrNm44skIzuEEyL3b6Q7zsFNBFxfDXMBEADq/Vlop4lx W9Bi7cYaxjnhNMbDxR3aVV3GMy7zkuV3fGjvTAuO7raDoqN91y2bvm+vkM54Mw+Fu3ThBHCc niYkdgyIlpLNfFhQOOBtlj1jNmCGTOp7JBJ8mjQucOGy3A0uUaqvkMncO2trsfEsRD25nTZV dji4cHePP86Uc/w98OBt5cPLyM5/WU+GTpPBATFc1u4DocMSqbCZcNLDr1O7P6Knl8vsCtRj r4X4xp3d3I+aZ1h5l/Hu20u+/7fg8PO/WDOCMwqN64xB2OHLv7bQm0piwVN+Gkyjhk9EJINO m05ieGxvsyAaQDMkgvm/Hp1G2LJEWkxaTalnBawaVYM4TgpE5hU9gda3Ld/pdT2lkNmVq47v d//NBYWa1/aJGqBP2XlW0JMywmDEVH2fFwVqMQ3ZzE+N4HnSR6Uslv4DpIeYLGevheHLqegO o5VySo/P+lGQDO0vSHbBZgRJ/ROwNvkJVEQ9pI34Edt2zCfUYqjUU+3tcd2xgR8/JP2dnjYp NDq4CK/8ywk+sl8WY8MQpKccJdWp0CVgp2y88YALn6x9wxsPGQ8Lk7sB4ADwwPX87pFdJqe2 2oKgG9HklBTSxvdr1fN+/n4AD5XPBhMpi9UZHdaVuWzVfXU6EKMHlsJfJ7sF1atKVPcQDIN/ SxwTlxmsHVCMxifikklhQwmD/wARAQABwsF8BBgBCAAmAhsMFiEE9GWrrNY3rqwUFVXPygNj jfo/HHoFAmNsLkkFCQmC/B0ACgkQygNjjfo/HHrGqxAAgdV2GRCEIJwbLtRTv4qpfANsfX/I S0KZBDbwtZuCDceQTfsS8lF5WGJOPxF7MDkVGRhBPYpVamO56V+Y/o6drX0JEeWqLwB3nc8b PFpFcGTFOxp6Q1Ps4n/L/+hhTsI4vO79jsf97s7GPhSLpnzakb2Enh25G/MoToj6qQvuuqFW lDX1hxBC9Jf6pnuX1XGUik9LZVpuqf9suVgNa4Wf6ZverzphPWObXifpjVREk7jJ0g2OBGZz CATDfgEfu4OtV2JEeSrPZeFoHPQzsFfO/n99ZxBWVLbiXJgNTnU/vwbPkMpPjED3WSwTesW+ SgXfZf4KHFigC3Z34uhXfxn4I1XEW7UGgbtsrFqJI+WVzP2mUeOzNoz980N1tfowzoNShfKD rcDhtpyZZcaA+0O1oVSKs3LUT4QNiYl/lcg4l0qnBvj+ilG3ViXikhKn4RMJUppbXCx09ZBK Pz1PAvig0mnpCWCjizdbtuFzkZsLmfYjIbxkXBLMmQdqMgqUnsUGvUh0u+v5zQlc3IQnpLBt vkQuitsVuqkCKSILz8p2DyA6PWRI988fGPakjvV13iGwAIfFPsNvOoWsVLz3GT6f9chF4fAY JuOem41lA82RZB9fPty0S8YVavucU7qzTfHRj6LwvrWzLk6wRA7ThPHWVAZsyH1iYO+nUOsU FFu7icw= In-Reply-To: <87mssuu57m.fsf@inria.fr> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=66.111.4.29; envelope-from=philip@philipmcgrath.com; helo=out5-smtp.messagingengine.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -8.74 X-Migadu-Scanner: mx13.migadu.com X-Spam-Score: -8.74 X-Migadu-Queue-Id: 71FF65342F X-TUID: 6o7fF7sxGQVi Hi, On 1/24/24 10:22, Ludovic Courtès wrote: > > The question boils down to: Git-LFS or Git Annex? > > [...] > > What’s your experience? What would you suggest? > I have a few times had a problem for which I thought Git LFS might be a solution, and each time I have ended up ripping out Git LFS in frustration before long. I have not used Git Annex. I have looked into it a few times, but each time I decided it was too complex or not quite suitable for my use-case in some way. On the other hand, I have heard good things about it from people who have used it: in particular, I believe Morgan Lemmer-Webber (CC'ed) used it to manage a large set of art history images. The main thing in this context that still isn't clear to me from by reading so far is how sharing lists of remotes works with Git Annex. In plain Git, remotes are part of the local state of a particular clone, not distributed as part of the repository. For the objectives here, though, a lot of the benefit would seem to be having many copies in synchronized, possibly "special" remotes so that anyone trying to get the videos would have plenty of ways to get them. I'm not sure to what extent Git Annex does that out of the box. I did see that Git Annex can use Git LFS as a "special remote". There are also two other approaches I think would be worth at least considering: 1. Just use Git While the limitations of Git for storing large media files are well known, I have found it to be good enough for several use-cases, and it has the strong advantage of not requiring additional tools. My impression is that a significant factor in people using Git LFS, in particular, is the limit on repository size imposed by the popular hosting providers. There are strategies within Git to avoid having to download unwanted artifacts, including creating branches with unrelated histories, shallow clones (e.g. --depth=1 --single-branch), partial clones [1][2][3] (e.g. --filter=blob:none), and sparse checkouts [4][5], with the later two being fairly new features. [1]: https://git-scm.com/docs/partial-clone [2]: https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---filterltfilter-specgt [3]: https://git-scm.com/docs/git-rev-list#Documentation/git-rev-list.txt---filterltfilter-specgt [4]: https://git-scm.com/docs/git-sparse-checkout [5]: https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---sparse 2. Mirror URLs Another approach would be just to make each video available at a few URLs and have Guix origins with the list. If one of the available URLs were the Internet Archive, it would have a high degree of assurance of long-term preservation. I think the biggest downside is that this might not help much with managing the collection of videos. Philip