Re: [RFC] A simple draft for channels

[Pjotr Prins <pjotr.public12@thebird.nl>]

On Fri, Jan 19, 2018 at 09:24:27AM +0100, Ricardo Wurmus wrote:
> Hi Guix,
> 
> I’d like to retire GUIX_PACKAGE_PATH as the main way to get third-party
> packages, because we can’t really keep track of packages that were added
> or redefined in this way.  I want to replace it with slightly more
> formal “channels”.
> 
> As a first implementation of channels I’d just like to have a channel
> description file that records at least the following things:
> 
> * the channel name (all lower case, no spaces)
> * a URL from where package definitions can be loaded (initially, this
>   can be restricted to git repositories)

Name and URL can be one. I would think the URL+branch or git tag is
unique for the channel.

A channel can still be named by a user, but it need not be specified.
Like git does with branches and remote repos. E.g.

  guix channel add bioinfo1 git-URL

So the user choses the name.

> Optional fields:
> 
> * a description of the channel
> 
> * a URL from where substitutes for the packages can be obtained (this
>   will be backed by “guix publish”)
> 
> * a mail address or URL to contact the maintainers of the channel, or to
>   view the status of the channel
> 
> * the Guix git commit that was used when this channel was last
>   updated.  This is useful when Guix upstream breaks the ABI or moves
>   packages between modules.

The last is critical. One thing we need to think through is that we
don't want to build the full guix repo since it slow and will take
more and more time. Would it be possible to package the binary .go
files and use those on the go since they are fixated anyway? I presume
they are system independent (but not arch independent - but I think
different arch will have different channels anyway).

The package definition can be hosted inside the git repo as a
guix-channel.scm file. When the user adds a channel we can fetch this
info.

> On the Guix side we’d need to add the “guix channel” command, which
> allows for adding, removing, updating, and downgrading channels.  Adding
> a channel means fetching the channel description from a URL and storing
> state in ~/.config/guix/channels/, and fetching the git repo it
> specifies (just like what guix pull does: it’s a git frontend).  It also
> authorizes the the substitute server’s public key.

That would be wonderful. Maybe make it an --auto-authorize switch for
those who choose to live dangerously.

> Internally, it’s just like GUIX_PACKAGE_PATH in that the repos are used
> to extend the modules that Guix uses.  Unlike GUIX_PACKAGE_PATH,
> however, we now have a way to record the complete state of Guix,
> including any extensions: the version of Guix and all active channels
> with their versions.  

Exactly.

> We would also have a way to fetch substitutes from
> channels without having to “globally” enable new substitute servers and
> authorize their keys.  (Is this safe?  Can we have per-user extensions
> to the set of public keys that are accepted?)
> 
> Downsides: Guix has no stable ABI, so channels that are not up-to-date
> will break with newer versions of Guix.  

Popular channels will move along, I have no doubt.

> Moving around packages to
> different modules might break channels.  That’s okay.  It’s still an
> improvement over plain GUIX_PACKAGE_PATH.
> 
> I don’t think it has to be more complicated than that.  What do you
> think?

I think it is spot on. Some devil in details, I am sure.

We can also build a web page where we can list channels, test-run
them, and their libre-state. In principle build-farm support could be
added for libre-channels. It would help GNU Guix move forward as a
generalized solution. You will not believe the pain I went through
with Fred just to install a huge GUIX_PACKAGE_PATH package. And we
both already use Guix and can write packages! 

In the next phase I can add support for relocatable packages attached
to a channel. We could get to the point that:

1. Install Guix with proot support
2. Fetch channel
3. Install binary package from channel (native compiled and optimized)
   into proot

So far will work for all software that runs in proot. Next

4. Relocate binary

And we have a local optimized software without ever getting root
because the relocated binary won't need proot.

It won't work for all software though. But for most computational
tools it will. Anyway, separate story. Even a native channel is a
huge win.

We can start hacking at FOSDEM. I am happy to participate.

Pj.