Re: best practise between git-fetch vs url-fetch?

[zimoun <zimon.toutoune@gmail.com>]

Dear all,

On Thu, 14 May 2020 at 18:16, Jack Hill <jackhill@jackhill.us> wrote:

> Heh, I'll take the bait. I also really appreciate how easy we make it for
> people to exercise their software freedom and run modified software. How
> best to make that possible and balance it with other usability constraints
> (e.g. mirror:// urls should be more robust) may vary by package, so I
> agree with Leo that some discretion is needed. However, that's not to say
> that I wouldn't appreciate some guidance as to our default preference when
> there is no package-specific reason to prefer one way over the other.

I second these words. :-)


On Wed, 13 May 2020 at 19:13, Leo Famulari <leo@famulari.name> wrote:

> Do we need a consensus? Sometimes it's enough for the reviewer to 1)
> bite their tongue or 2) fix before pushing. Often it's a matter of taste
> and it is beneficial to not second-guess the patch author too much, to
> not hurt their confidence.

From my point of view, it should not be a matter of taste.  To me, it
should ideally be uniform per file.  For example, bioconductor.scm
uses mainly 'url-fetch' and emacs-xyz.scm mainly 'git-fetch'.
However, some are mixed which does no appear to me right; for example
bioinformatics.scm.

--8<---------------cut here---------------start------------->8---
for f in $(ls -1 gnu/packages/*.scm);
do
    echo $f
    grep "\-fetch" $f \
        | sed -e 's/^[[:space:]]*//' \
        | sort | uniq -c | sort -n
    echo ""
done | less
--8<---------------cut here---------------end--------------->8---


> I think we should try to avoid a situation where we have to bootstrap
> Git. We do have git-minimal, which works for now. Libgit2 recently
> started releasing tarballs, so that could be an option, too.

But, is Git bootstrapped anyway?  Somehow, Guix bootstraps Git, isn't it?
Your point is remove 'git' to the implicit dependency graph, right?


> Another point for url-fetch is that Git's transition from SHA1 to SHA256
> identifiers may be easy for us, or it may not be. We don't know yet.

Tempus narrabo. :-)


On Wed, 13 May 2020 at 10:24, Brice Waegeneire <brice@waegenei.re> wrote:

> An other argument in recommending the 'git-fetch' method is that it
> facilitate the use of “guix build --with-commit”:

+1 for some packages! :-)
-1 for some other ones; for example the ones coming from big
collection as BioConductor released like a whole.


> It does, it avoid having to discuss it with each new contributor
> and avoid noise in patches about changing the source's method based
> on each developer preference (I'm personally guilty of that).

Especially when there is no committer attached to some specific set of packages.


On Wed, 13 May 2020 at 20:07, Tobias Geerinckx-Rice <me@tobias.gr> wrote:

> That's only tautologically true if we limit ourselves to using raw
> commit hashes instead of the more common (because far more
> practical) named tags.  I obviously don't think we should do that.

Why?  If there is a good motivation.
We could imagine a transition: recommanding and encouraging
'git-fetch' with raw commit for any addition/update.
However, it is not clear that there is a strong value for such transition. :-)


> For the sake of argument¹, though, so is --with-source=<actually
> released and supported upstream tarball dot tar>.

Well, in my workflow I never used '--with-source' because there is an
extra step that '--with-commit' does not have.  But it is a minor
detail.  The real issue is what Jack described with the autotools (or
any other pre-built X generated files).

For a concrete example, let consider 'htop':

--8<---------------cut here---------------start------------->8---
git clone https://github.com/hishamhm/htop /tmp/htop
git -C /tmp/htop checkout 2.2.0
guix build htop --with-source=/tmp/htop
[...]
running './autogen.sh'
patch-shebang: ./autogen.sh: changing `/bin/sh' to
`/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/sh'
./autogen.sh: line 3: autoreconf: command not found
--8<---------------cut here---------------end--------------->8---

In my scientific workflow, I find easier to do, e.g.,

    guix build bamm --with-commit=bamm=XXXXX

to test new features or reproduce something based on a paper using one
specific commit.  Well, it is easier to hack on some packages and it
saves time by collectively maintain the burden.


What annoys me is that I cannot predict* for which package it works.
That's why a recommendation per file seems right to me.

*obviously, we could add the method of origin in "guix show"; even it
would help, it is not really the point, IMHO.


> Somehow erasing that hard distinction is the real winning move.

To me, today 'git-fetch' provides more flexibility / hackability than
'url-fetch'.


On Sun, 24 May 2020 at 22:30, Ludovic Courtès <ludo@gnu.org> wrote:

> Probably we should aim towards not using pre-built Autotools generated
> files and, by extension, probably not using pre-built tarballs when VCS
> checkouts are available.

This would help to reduce the "hard distinction".  Well, what could be
a working plan?


> It’s kinda happening on leaf packages, often upstream developers people
> don’t bother running “make dist”.  It’ll take some time before tarballs
> disappear and needs some thought in particular from a bootstrapping
> standpoint.

Well, from a reproducibility point of view, we should not use the
pre-built Autotools generated files, IMHO.



All the best,
simon