Re: “What’s in a package”

[Katherine Cox-Buday <cox.katherine.e@gmail.com>]

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> I agree.  Like Konrad and you wrote, it’s good that we can all have our
> quick-and-dirty packages in personal channels, and it’s good that these
> are separate channels.

Definitely! I would also encourage Guix to adopt some of the tools that make creating quick, but bad, packages easier. E.g. builds that patch elfs with the library paths of their dependencies.

> I’m not sure how tooling could help in the way of making quick packages,
> but it’s worth exploring.  Examples that come to mind are: merging the
> npm importer that currently lives in a branch, and providing a generic
> “guix import upstream” importer that would figure out as much as
> possible so that one doesn’t have to start from a blank page.

That sounds like a great start. I tossed out some other ideas elsewhere in the thread. Most of them involve meta-inspection of the package, Guix ecosystem, runtime environment and logs. It would be nice in general to have a kind of "agent" that you could run repeatedly over the course of packaging that would suggest next steps on ~stderr~ and next logical packaging definition on ~stdout~. Kind of like pair-programming with Guix :)

It would perform different operations dependent on what stage in the life-cycle the package is at, i.e. ~import~ when no package definition exists, build when one does, and possibly running the result in a container when the package build succeeds.

E.g. your PyTorch example, starting from scratch (note: ~guix import~ may not always feel like the right command to invoke in this example. This may be some larger concept than import; also, the example always redirects to package.scm for brevity, but the user would probably want to look at it first):

#+begin_example
  $ guix import upstream pytorch

  stderr: This looks like it might be python package (heuristics.scm:123 - package name starts with py), try this instead:
  stdout: guix import upstream pypi pytorch

  $ guix import upstream pypi pytorch | tee package.scm

  $ guix import upstream package.scm | tee package.scm

  stderr: downloading...
  stderr: It looks like this fails to build because it's missing autoconf (heuristics.scm:133 - grepping build output found a missing autoconf error). Try adding it as a native-input.
  stdout: (package definition with imports defined and native-input modified)

  $ guix import upstream package.scm

  stderr: downloading...
  stderr: It looks like this package comes with binaries that are available as Guix packages (heuristics.scm:143 - unpacking source includes binary or object files, heuristics.scm:153 - bundled files match output of known packages). Try this package definition instead:
  stdout: (package definition with suggested inputs and overridden phases to remove the binaries from the download)

  $ guix import upstream package.scm | tee package.scm

  stderr: It looks like this package vendors libraries that are available as Guix packages (heuristics.scm:163 - unpacking source includes vendored libraries, heuristics.scm:153 - bundled files match output of known packages). Try this package definition instead:
  stdout: (package definition with suggested inputs and overridden phases to remove the vendored libraries from the download)

  $ guix import upstream package.scm | tee package.scm
  
  stderr: It looks like this package searches XDG_DATA_DIRS for some files (heuristics.scm:163 - grep an strace of a containerized run of the output). Try this package definition instead:
  stdout: (package definition with ~native-search-paths~ defined)
#+end_example

etc., etc. Typing that out, it feels dangerously close to Microsoft's Clippy, but hopefully more helpful :)

Heuristics, by definition, wouldn't be correct all the time, but this kind of thing could help new contributors (or experienced contributors with bad memories like me!), and in some cases actually do some of the programming.

And every time someone comes to the mailing list or IRC with a question, we can ask ourselves if this is a common question, and maybe create a new heuristic.

-- 
Katherine