Re: the upcoming Great Python2 Purge™

[zimoun <zimon.toutoune@gmail.com>]

Hi Konrad,


On Mon, 18 Feb 2019 at 17:31, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:
>
> Hi Ricardo,
>
> >> "My channel" doesn't exist (because I haven't yet found the time to
> >> figure out how to set up and manage a channel, although it's been on my
> >> to-do list for a while).
> >
> > I’d be happy to assist.
>
> Thanks! I might come back to that offer when I find at least enough time
> to figure out precisely what information I might be lacking
> (that's meta-lack-of-time).

From my understanding, it is simpler than expected ;-)

For example, instead of GUIX_PACKAGE_PATH=~/work/your-repo
you create the file ~/.config/guix/channels.scm containing:

(cons* (channel
        (name 'bimsb)
        (url "file:///home/simon/work/your-repo")
        (branch "master"))
      %default-channels)

Then:

guix pull
guix package -s <your-package>
guix package -i <your-other-package>

etc.


> >> But... how about splitting off *all* of Python 2 and everything that
> >> depends on it into a separate channel, which would then be maintained
> >> by a separate team?
> >
> > Currently this is not feasible, in my opinion, as a lot of packages in
> > Guix still depend on Python 2 for some reason or the other.  When Python
> > 2 reaches EOL, however, I think this would be a reasonable thing to do.
>
> Sounds good, then the only remaining issue is defining a transition
> protocol. What I'd like to avoid is that packages disappear one by one
> from Guix and then have to be dug out one by one from Git history for
> setting up a Python 2 channel.

First, the package will not disappear. Because now, they are in the
Guix tree, one can always reach them with `guix pull --commit' even if
they are not "visible" in gnu/packages.

Second, I am not sure it is that ugly. :-)
If one package "disappears", then you can quickly find the last commit
where it is defined with:

git rev-list --all | xargs git --no-pager grep 'public python2-stuff' | head -1

Then you can add the package to your channel (or a semi-official one),
with its dependencies if needed.

To me, this is the best transition protocol.
 a- the Guix tree is moving forward "reserving its freedom to break third party
channels"
 b- if no one moves the disappeared package, then it means the package
is really obsolete.


Last about scientific reproducibility, imagine you have your local
channel, containing your packages that rocks!
Today, you compute some stuff and so on.
You "just" have to store the output of `guix describe'. For example,
my current one is:

Generation 16   Feb 07 2019 15:47:37    (current)
  bimsb d1cba4a
    repository URL: file:///home/simon/work/guix/bimsb
    branch: master
    commit: d1cba4a2cba1eb6c6c33328fea511b75dfcffe39
  guix 89ea625
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 89ea6252b6849131ba35d141006e1bbf3a49594f

Months later, my "collaborators" want to reproduce my result. With
this information (the 2 commit hashes), they will be able to set again
all the dependency graph.
(however, substitutes should not be there so maybe they should need to
build a lot; another story)

The `guix pull --commit=' is not super friendly yet with multiple
channels. But, it is only UI gaps, isn't it? :-)

For example, it should be nice to run:
 guix pull -C bimsb --commit=d1cba4a -C guix --commit=89ea625
to restore the exact same tree.

Today, if I am correct, you can put this information in a manifest
file and then instantiate an environment for your computations. From
my point of view, it is the long-term practise to have the
"scientific" reproducibilty.


Please correct me if I am wrong. :-)


However, some whistle.
 1. Even if inferiors and other improve the travel in time, there is
no guarantee that nothing will be break in the future compared to the
past (if you see what I mean).
 2. Archivist issue: will the current material be still available?


> More generally, I think it would be useful to collect in some place all
> package definitions that are removed from Guix. For example, a list with
> package names plus the last Guix commit that had the package.

From my opinion, when we speak about reproducibility, remove a package
is similar to update it. And all this information is tracked by the
Git repo.
I agree that maybe it is not friendly to dive in and it is really
resource hungry.
For example, it is not "easy" to find when the package lispf4 have been removed.

And if we speak about scientific reproducibility, it appears to me
difficult to have the Q&A that you propose at the Guix commit level.
However, it should be easier to find from which commit the package is
installed. Somehow, improve the `guix package --list-installed' and/or
`guix package --list-generations' commands, i.e., also tracking the
channel and the commit.

As a user, if I run:
 guix package -i python-numpy
 guix pull # the tree is changed
 do other stuff (install remove pull again etc.)
 guix package -i python-scipy
and I forgot to store the commit hash of my first install.
Then, it requires some work to find it: I need  to correlate the `guix
pull -l' information to the `guix package -l' information.




All the best,
simon