Re: Finding versions of packages (was: [outreachy] Walk through the Git history)

[zimoun <zimon.toutoune@gmail.com>]

Hi,

Thanks for your inputs.


On Sat, 12 Dec 2020 at 22:08, Ryan Prior <ryanprior@hey.com> wrote:

> I propose a different approach: the "guix versions" subcommand provides
> direct answers to practical user questions.
> - What package versions are available?
> - How do I use them?
> - Which versions are known to be vulnerable?
> - Which have available substitutes?

Roughly speaking, it is already possible with the Data Service.  Missing
glue code to use JSONs. :-)

<https://data.guix.gnu.org/repository/1/branch/master/package/gmsh/output-history.json>

For the last Hacktathon about “time travel”, I tried to implement
something fetching these JSONs, collecting the hash, find intersection
about 2 commit ranges, etc.  But 1) I do not want to depend on external
services, especially when it starts late on 2019 and 2) custom channels
will be always missing.

<https://hpc.guix.info/blog/2020/07/reproducible-research-hackathon-experience-report>


(Aside the “bug” with the accuracy of in/out hash commits, see
<https://yhetil.org/guix/868sazm4tu.fsf@gmail.com>.)


> For example, the command "guix versions esbuild" can provide this
> output:
> name: esbuild version: 0.8.21 guix-hash: eee3af86c7 name: esbuild
> version: 0.8.19 guix-hash: 6374a25357 name: esbuild version: 0.8.16
> guix-hash: 8c3caf9c5d vulnerabilities: cve-2020-1337, cve-2019-1024
> ...and so on.

Your proposal is the final aim when I proposed “guix git log”.  It comes
from this old discussion:

<https://yhetil.org/guix/CAOOOohSzUezKvm=RO0bXRGH3m0eo2x0cOTvd--vARxWoqtceaQ@mail.gmail.com>

which clearly shows that something is missing; especially when all the
information is somehow already there.

In my views, “guix git log” is the first actionable step in the
direction of what you are proposing.  Your description is what I want
too at the end, so I proposed the “guix git log” starting point with
then potential incremental improvements.

Since ~/.cache/guix/checkouts/ provides all the information required to
implement this, what is missing is somehow “glue code” and all is
already there.  Even if it is not straightforward to get the accurate
commit range: does Gmsh at commit c5372cd488 is the same version as Gmsh
at commit 44e65a75, even if both are the code source from upstream
4.6.0?  These 2 commits do not have the same derivation because one of
the dependency changed.  Well, “guix versions” is more complicated than
it appears at first (parse the commit message).

Therefore, “guix git log” is only the first step with a concrete
outcome:

  guix git log | grep esbuild

This path will provide information about performance, what is missing on
the Guile-Git side and how to deal with multichannels, etc.  Then, yes
the next idea–out of the Outreachy scope–is to create a local
cache–probably substituable–with all the versions and the commit hash
in/out per version, and probably also add bit of graph theory to find
commit range with multi-packages multi-version; in order to reduce the
number of inferiors to create.

Since it is piece of work, the idea is to implement one step after the
other. :-)

> Then, to make the output directly actionable, we extend Guix to accept
> recfile manifests following the same structure, such that given the
> following file "packages.rec:"
> name: esbuild version: 0.8.19 guix-hash: 6374a25357 name: python-
> html5lib version: 1.1 guix-hash: 6374a25357
> …we can use those exact packages using "guix environment -m
> packages.rec", or find which of those packages have substitutes using
> "guix weather -m packages.rec"

Ah interesting.  Personally, I am taking the other direction: adding
other “formatters“ than ’package->recutils’, for instance
’package->bibtex’ or ’package->custom’ or ’package->name-it’, because it
should be possible to display the information as I want.  For example:

  guix search emacs --format=oneline
  emacs                 27.1            The extensible, customizable, self-documenting text editor  
  emacs-with-editor     2.9.4-1.c4768f5 Emacs library for using Emacsclient as EDITOR  
  emacs-restart-emacs   0.1.1-1.9aa90d3 Restart Emacs from within Emacs  
  emacs-spacemacs-theme 0-1.f79c40f     Light and dark theme for spacemacs that supports GUI and terminal 
  […]

Therefore, adding somehow a formatter compatible with the ’--manifest’
option.


All the best,
simon