Re: Git-LFS or Git Annex?

[Nicolas Graves via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org>]

I've left git-annex for git-lfs, I'll just add a few points about
git-lfs.


On 2024-01-24 18:39, Giovanni Biscuolo wrote:

> Hi Ludo’
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>> The question boils down to: Git-LFS or Git Annex?
>>
>> From a quick look (I haven’t used them), Git-LFS seems to assume a
>> rather centralized model where there’s an LFS server sitting next to the
>> Git server¹.  Git Annex looks more decentralized, allowing you to have
>> several “remotes”, to check the status of each one, to sync them, etc.²
>> Because of this, Git Annex seems to be a better fit.

This is not always true. Git-LFS also has the concept of Custom Transfer
Agents, which in some cases do not need a running server. One example is
lfs-folderstore, which can simply use a remote directory as a LFS remote.

>
> I've never used Git-LFS for my media repository (and will never use it,
> never).
>
> AFAIK this two advantages of git-annex vs Git-LFS are still valid today:
>
> --8<---------------cut here---------------start------------->8---
>
> A major advantage of git annex is that you can choose which file you
> want to download.
>
> You still know which files are available thanks to the symlinks.
>
> For example suppose that you have a directory full of ISO files. You can
> list the files, then decide which one you want to download by typing:
> git annex get my_file.

This is true, but
1) you can still adapt your filters to ignore certain files, although
more inconvenient, it's not impossible
2) in practice, I think most uses don't need to. I just now that all .lz
files in a directory are to LFS, no questions asked.

>
> Another advantage is that the files are not duplicated in your
> checkout. With LFS, lfs files are present as git objects both in
> .git/lfs/objects and in your working repository. So If you have 20 GB of
> LFS files, you need 40 GB on your disk. While with git annex, files are
> symlinked so in this case only 20 GB is required.

True.
>
> --8<---------------cut here---------------end--------------->8---
> (https://stackoverflow.com/a/43277071, 2018-10-23)
>
> So, AFAIU, with Git-LFS you can have all your media or no media, you
> cannot selectively choose what media to get.
>
> Another important limitation of Git-LFS is that you cannot delete
> (remotely stored) objects [1], with git-annex is very easy.

Probably true, haven't encountered the use-case yet.
>
>> Data point: guix.gnu.org source is hosted on Savannah, which doesn’t
>> support Git-LFS;
>
> to host a Git-LFS service a Git-LFS server implementation (one that
> reply to GIT_LFS API) is needed:
> https://github.com/git-lfs/git-lfs/wiki/Implementations

See my point on custom transfer agents.
>
> AFAIU we dont't have one packaged (I'd save some precious time trying to
> package one of them).
>
> AFAIK Savannah do not support git-annex also, so we need to set up a
> Guix git-annex server [3], I suggest using gitolite [4]: I can help with
> this task if needed!
>
>> the two other web sites above are hosted on GitLab instances, which I
>> think do support Git-LFS.
>
> Yes, Git-LFS is supported on GitLab.com and included in the Community
> Edition [2] since late 2015.
>
> git-annex repository support was available on GitLab.com in 2015/16 but
> was removed in 2017 [5]
>
>> What’s your experience?  What would you suggest?
>
> I've no experience with Git-LFS (and will never have) but from what I
> read I definitely suggest git-annex: it's more efficient, it's more
> flexible, can be hosted everywhere with a little bit of effort... can be
> hosted on a Guix System host! :-)
>
> As a bonus, git-annex have a plenty of super cool features that will
> make us very happy, i.e.:
>
> - special remotes: https://git-annex.branchable.com/special_remotes/
>   (including rclone
>   https://git-annex.branchable.com/special_remotes/rclone/)
>
> - location tracking
>   (https://git-annex.branchable.com/location_tracking/)
>
> - manage metadata of annexed files
>
> HTH! Gio'

Just a note on upsides of Git-LFS :
- integration with git is better. A special magit extension to use
git-lfs is not needed, whereas it is with git-annex.
- less operations: once I know which files will be my media files, I
have less headaches (basically the exact git experience, you don't have
to think about where I should `git add` or `git annex add` a file).

It's indeed less copyleft though. Simpler, but also maybe less adapted
to this use-case.

>
>> Thanks,
>> Ludo’.
>>
>> ⁰ https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/berlin.scm#n193
>> ¹ https://github.com/git-lfs/git-lfs/wiki/Tutorial
>> ² https://git-annex.branchable.com/walkthrough/
>
>
> [1] https://github.com/git-lfs/git-lfs/wiki/Limitations
>
> [2] GitLab Community Edition
>
> [3]
> https://git-annex.branchable.com/tips/centralized_git_repository_tutorial/on_your_own_server/
>
> [4] https://git-annex.branchable.com/tips/using_gitolite_with_git-annex/
>
> [5] https://about.gitlab.com/blog/2015/02/17/gitlab-annex-solves-the-problem-of-versioning-large-binaries-with-git/

-- 
Best regards,
Nicolas Graves