unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* "guix potluck", a moveable feast
@ 2017-03-31 14:44 Andy Wingo
  2017-04-01 14:50 ` Christopher Allan Webber
  2017-04-01 23:05 ` Ludovic Courtès
  0 siblings, 2 replies; 10+ messages in thread
From: Andy Wingo @ 2017-03-31 14:44 UTC (permalink / raw)
  To: guix-devel; +Cc: guile-devel

Hi!

Following on thoughts from earlier this month that Guix is the guildhall
that we always wanted, I think I have more of a plan.


s/guildhall/potluck/
--------------------

Firstly, a name change: the guidhall was a somewhat unified, curated
thing that was Guile-specific, whereas I think I think we need to take
this feast outdoors and move it around a bit.  I would like to use the
term "potluck" to refer to the offering of tasty home-hacked goods, the
composition of which is but loosely planned.  This is inspired by the
"potluck" dishes that we would hack up to celebrate the Guile 2.0
release anniversary.

I don't want to take the metaphor too far of course; I think there was a
forced aspect around the guildhall name that never really became
natural.

Also, the potluck facility is just one part of the system.  Firstly we
use Guix itself as the package manager.  Secondly we build on the
upcoming "guix channel" facility.  (In its most basic form, a channel is
just a git branch somewhere that contains package files.)  The job of
the potluck facility is just cultivating the git branch that is the
"potluck channel".

As an interlude, here is how a user would enter an environment that has
a potluck package "foo" using Guix (using a pack is also possible).  We
start with setup steps:

  (1) Install Guix as a user.  (This needs to be easier.)
  (2) guix channel add potluck https://gitlab.com/potluck/potluck master
  (3) guix channel enable potluck

Then:

  (4) guix environment --ad-hoc foo

Assuming "foo" is a package that's in the potluck channel.  Basically
the potluck channel augments the set of available packages.  This is
handled by the channel mechanism.


A packaging language for stability and security
-----------------------------------------------

So how do packages enter the potluck channel?  Good question, fictional
reader!  This is the tricky bit.  There are some concerns here:

  (1) The Guix API is not stable and has no plans to be stable.  This
      works great for now because all packages are in one atomic
      repository and people work on making the whole thing make sense
      together.  One of the goals of the potluck effort is to
      decentralize things a bit, so we have an impedance mismatch
      between potluck packages and Guix itself.

  (2) Potluck package definitions will live in many different git
      repositories across the internet, and anyone should be able to
      make a potluck package.  Some potluck package authors will be
      malicious.  They could:

       1. Damage the server that manages the potluck channel

       2. Damage the users that run Guix commands with the potluck
          channel enabled

       3. Damage the users that install potluck packages

      I think we need to forget about 3, for now at least.  (Flatpak
      solves this, more or less; Guix has ongoing work to do here I
      think.)

Both of these large issues point to the need for careful design of the
language that potluck packages are written in.  The language that Guix
packages are written in is inappropriate because of (1).  In particular
we should not depend on which module a package comes from, and what
identifier binds any given package.  For (2), packages are currently
written in full Scheme, staged between the Guix command itself and the
sandbox that runs inside guix-daemon.  Full Scheme might be OK in the
daemon but it's not OK in the Guix command itself.

Concretely I would propose that the language that potluck files are
written in is like this:

  (1) It's code, not inert data.

  (2) It's a subset of Scheme, like core Guix packages.

  (3) The general structure looks like this:

      (import-guix-packages ((guile "guile@2.0")
                             (glibc "glibc")))
      (import-potluck-packages ((foo "foo")))

      (define bar
        (package
          (name "guile-bar")
          (version "1.0.0")
          (build-system gnu-build-system)
          (inputs `(("guile" ,guile))
          ....)))

I.e. we have a special mechanism to import packages by name using
specification->package.  We can define packages using Guix's "package"
form, and a number of standard Guix bindings are available to the code
(license:gplv3+, gnu-build-system, etc).

There is a particular concern about staging: there is staged Scheme code
in these modules that runs inside build processes in guix-daemon.  I
don't have any nice solution here.  Simply sandboxing the host Scheme
will be fine enough for me, using (ice-9 sandbox) if that lands soon
(see
https://lists.gnu.org/archive/html/guile-devel/2017-03/msg00111.html).
This requires Guile 2.2.1 (when it is released).

I can see an argument to support more simple package definition,
specifically that it should be easier to build Guile-only projects from
git -- skipping autotools, skipping tarballs.  But we should be able to
support tarball builds containing C or anything else, so the full
package capabilities are needed.  I think a first version can use the
normal package facilities.


A potluck channel manager
-------------------------

The "guix potluck manage-channel" command manages a registry of sources
of potluck definitions and turns them into a git branch of package
files.  This is the web service.

The idea is that as a developer, you should be able to do:

  guix potluck add https://gitlab.com/wingo/foo master

This causes the client to make a request to some web service, say
running on potluck.guixsd.org, to register that git branch.

One question would be, who is able to register potluck sources?  Do we
need users and all that?  I think we might be able to avoid that -- it
could be that we just assume that any git branch that has a
"potluck.scm" in its root directory has implicitly granted permission to
be a part of the potluck repo.

potluck.guixsd.org needs to be isolated from other hosts because it will
load potluck.scm files from untrusted sources; we hope the sandbox works
but we need a bit of defense-in-depth.

As I mentioned, I think it would be nice to be able to install some
potluck packages directly from git, without requiring those packages to
make releases and update the potluck.scm.  But until then, we can make
it so that the source is fixed in the potluck.scm as it is with other
Guix packages, and therefore that any update to potluck.scm in the
source git branch registered with potluck.guixsd.org constitutes a new
release which replaces the old one.  A developer should signal
potluck.guixsd.org about the update via a re-invocation of "guix potluck
add".  Maybe "guix potluck add" could remember the branch, dunno.

Anyway!  The result of the "guix potluck channel-manager" is a stream of
guix modules as a continually updated git tree -- a guix channel.  I am
thinking that we need to rewrite these files to be more "normal" -- like
starting with a (define-module), but a #:pure module and an appropriate
set of imports to enforce the sandbox.  We should be able to compile
this module, to prevent the potluck channel from slowing things down.
So basically the channel-manager rewrites the potluck.scm files.


I think that's the essence of my proposal.  WDYT?

Andy

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2017-04-04 12:01 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-03-31 14:44 "guix potluck", a moveable feast Andy Wingo
2017-04-01 14:50 ` Christopher Allan Webber
2017-04-01 16:01   ` ng0
2017-04-01 23:05 ` Ludovic Courtès
2017-04-02  2:20   ` Chris Marusich
2017-04-02  9:24     ` Ludovic Courtès
2017-04-04  2:20       ` Chris Marusich
2017-04-02 10:52   ` Andy Wingo
2017-04-02 14:45     ` Christopher Allan Webber
2017-04-04 12:01     ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).