wishlist: Guix tagging/logging

wishlist: Guix tagging/logging

[zimoun]

Hi Guix,

Some years ago, Ludo proposed [1] pinning Guix versions which is now
implemented (see Inferiors, "guix pull --commit=", manifest.scm, "guix
time-machine", etc.). However, in the bug report, Ludo also proposed
tagging.

--8<---------------cut here---------------start------------->8---
I think ‘guix pull’ would be a good place to add support for tagging
Guixes and similar, but it would be good if the naive approach above
would work just as well.
--8<---------------cut here---------------start------------->8---

[1] http://issues.guix.gnu.org/issue/22017


Do you think it is a feature that is missing? Something in the flavour
of "git tag" to ease the navigation between all the version.

What I have in mind is about the versions of interpreters and/or compilers.

Now, from an end-user point of view, it is hard to find the commit
introducing say Python 3.7 or if we have in our history Python 3.6.
(Replace Python by your favorite language other than Guile. ;-))

A concrete example is described in this thread [2].

[2] https://lists.gnu.org/archive/html/help-guix/2019-06/msg00098.html


Well, the idea is to:

 - expose to the end-user via "guix <command>" some Git history; ease
the search for a particular version.
 - locally tag


What do you think?


All the best,
simon

Re: wishlist: Guix tagging/logging

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 163 bytes --]

I think it's a great idea!

How do we tag locally though?  Some trivial <commit, tag> database?
An s-exp file?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: wishlist: Guix tagging/logging

[zimoun]

Hi Pierre,

On Tue, 11 Feb 2020 at 09:47, Pierre Neidhardt <mail@ambrevar.xyz> wrote:
>
> I think it's a great idea!
>
> How do we tag locally though?  Some trivial <commit, tag> database?
> An s-exp file?

Using "git tag".

For example, currently the "guix time-machine" only accepts
'--commit='<commit>' and because only the released versions are
tagged, it is not easy to navigate.
Another example is, let consider that I publish a channel with tags,
then my collaborators could easily obtain the list of all the tags via
"guix tag --list" (or whatever CLI)and then pull to one particular tag
and/or time-machine to it etc.
Yet another example, today when I play with issues from the mailing
list and/or the bug tracker, I often use "guix time-machine" or "guix
pull -p /tmp/test" but often I am doing that between "real" day job
and often on several days. Then it is boring to search in my shell
history which commit or reopen the message or collect in an org file,
so I often create a branch and/or tags in my local Guix checkout and I
retrieve my information using Git. This extra step using Git is not
natural and somehow the Git information is already there hidden under
some Guix layer.

We could imagine that Guix (optionally) checkouts the complete repo
and this Git repo is the database. (Note that Guix already checkouts
some more or less large parts or the repo.)
And the tags could be exported to a s-exp file, go under version
control and shared; with an importer tags.


Moreover, aside tagging, we could improve the inferior creation by
cutting some "updating time" using e.g., worktree.


All the best,
simon

Re: wishlist: Guix tagging/logging

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 826 bytes --]

zimoun <zimon.toutoune@gmail.com> writes:

> We could imagine that Guix (optionally) checkouts the complete repo
> and this Git repo is the database. (Note that Guix already checkouts
> some more or less large parts or the repo.)

A drawback is that the Git repo is huge and expensive to clone.
Also it would be nice if the user could create virtual tags, that is to
say, without having their own clone of the Guix repo (because it's
expensive).

It's lightweight to transfer an s-exp mapping of commits-tags to another
machine.  It's costsly to transfer the full Guix repo.

> And the tags could be exported to a s-exp file, go under version
> control and shared; with an importer tags.

This could fit the ~/.config/guix/config.scm we've been waiting for :D

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: wishlist: Guix tagging/logging

[zimoun]

On Wed, 4 Mar 2020 at 09:16, Pierre Neidhardt <mail@ambrevar.xyz> wrote:
>
> zimoun <zimon.toutoune@gmail.com> writes:
>
> > We could imagine that Guix (optionally) checkouts the complete repo
> > and this Git repo is the database. (Note that Guix already checkouts
> > some more or less large parts or the repo.)
>
> A drawback is that the Git repo is huge and expensive to clone.
> Also it would be nice if the user could create virtual tags, that is to
> say, without having their own clone of the Guix repo (because it's
> expensive).

It is already "huge and expensive". Give a look at:

--8<---------------cut here---------------start------------->8---
du -sh ~/.cache/guix/*
1.2G    /home/simon/.cache/guix/checkouts
9.7M    /home/simon/.cache/guix/http
8.0K    /home/simon/.cache/guix/inferiors
329M    /home/simon/.cache/guix/pull
222M    /home/simon/.cache/guix/substitute
--8<---------------cut here---------------end--------------->8---

And if you use regularly "guix time-machine" with different commits,
you have already some checkouts.

Last, note that the "huge and expensive" part is not the downloading
part, but the storage of the .scm files.
For example,

--8<---------------cut here---------------start------------->8---
du -sh ~/.cache/guix/checkouts/*
328M    ~/.cache/guix/checkouts/2zjgqq3yxnpiyamyqsqcurbyj3gb2suroq5jympr4ej3mqfzq4oa
209M    ~/.cache/guix/checkouts/5cm2wsegguxwkzg5hjgitjtqrkt3rkdbqfadxr734umsv4mh66ya
395M    ~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq
1.4M    ~/.cache/guix/checkouts/qcl7k7z35uutwy6b5fjkzqupfxeokcwqmlexkwmqlmoklosybyka
1.7M    ~/.cache/guix/checkouts/qzs5gpwlvfiigfcdqtmcqdrzdl5wdk7hsal5nyetzdjolcvxcvya
214M    ~/.cache/guix/checkouts/utzfc5hcbrwkfzpmyvucrvoz2klqxig3rstebsf7ehtkdm77xnsq
1.2M    ~/.cache/guix/checkouts/v4suojkdt3rogvagkns44b4muehmwxqyg37cvnnau43fk777d2ha
53M     ~/.cache/guix/checkouts/zw2o4kxknhggvaztdo7jnxbmvkncthjbgkppwazw7qbkqlqyhgua
--8<---------------cut here---------------end--------------->8---


Then let's go to
'~/.cache/guix/checkouts/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq'

--8<---------------cut here---------------start------------->8---
git log | tail
    Split (guix) in (guix store) and (guix derivations).

    * guix.scm: Move contents to other files.  Just aggregate these.
    * guix/derivations.scm, guix/store.scm: New files.

commit 207cba8114d354737b231e510d6110ea2a42e07b
Author: Ludovic Courtès <ludo@gnu.org>
Date:   Wed Apr 18 23:21:11 2012 +0200

    Initial commit.
--8<---------------cut here---------------end--------------->8---

and

--8<---------------cut here---------------start------------->8---
git log | head
commit be54abc61d4193fe3bee4bd62cc33412a9314975
Author: Nicolas Goaziou <mail@nicolasgoaziou.fr>
Date:   Tue Mar 3 19:23:17 2020 +0100

    gnu: wine-staging: Update to 5.3.

    * gnu/packages/wine.scm (wine-staging-patchset-data): Update to 5.3.
    (wine-staging): Update hash.
--8<---------------cut here---------------end--------------->8---

And just for the record, "guix describe" returns ' guix be54abc'.



I do not know exactly the internals of Guix but the full Git repo
seems already cloned; somehow. Therefore, it appears to me easier to
add tags (or to expose more Git features) using this database instead
of creating a new one, IMHO.


(And because that, I do not know how the inferiors work but maybe
'worktree' could improve the efficiency; especially the "Updating"
part when using "guix time-machine".)


> It's lightweight to transfer an s-exp mapping of commits-tags to another
> machine.  It's costsly to transfer the full Guix repo.

I am not saying that I would like to transfer the full Guix repo. :-)
I am saying that the tags should be stored in the Git repo database
(already locally there, see above) and then some 'exporter/importer'
to s-exp file in order to ease the transfer (lightweight, easy to have
under version control, etc.).


> > And the tags could be exported to a s-exp file, go under version
> > control and shared; with an importer tags.
>
> This could fit the ~/.config/guix/config.scm we've been waiting for :D

Yes, I agree. :-)


Cheers,
simon

Re: wishlist: Guix tagging/logging

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 320 bytes --]

I didn't know that the git repos under .cache were writable.  Well, good
to know! :)

So we could do as you said, but then we need a way to propagate the tags
across all checkouts in .cache/guix/checkouts.

Otherwise, on the next guix pull  the tags would be lost.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: wishlist: Guix tagging/logging

[zimoun]

On Wed, 4 Mar 2020 at 16:52, Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> So we could do as you said, but then we need a way to propagate the tags
> across all checkouts in .cache/guix/checkouts.

I do not know the (inferior) internals.
And currently I do not have the time to give a look into the sources
to crunch concrete details. :-)
So, it is hard for me to discuss further some "implementation
details". That's maybe why I pointed to your attention this wishlist.
;-)


> Otherwise, on the next guix pull  the tags would be lost.

Maybe not, if "guix pull" is simply an overlay to "git pull".
Well, I cannot tell because I do not know the internal details. With
*if* my aunt becomes my uncle. ;-)


My point is: since all or part of Git repo information is somehow
already locally there, then all or part of this information should
somehow be exposed at the Guix CLI level. The 2 first features missing
me are "git tag" and "git log".
However, I do not have more clue on how to process then. :-D
I mean right now I have more use-cases in mind than concrete
implementation details. :-)


Cheers,
simon