From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.devel,gmane.comp.gnu.guix.devel Subject: "guix potluck", a moveable feast Date: Fri, 31 Mar 2017 16:44:35 +0200 Message-ID: <87d1cxh5f0.fsf@igalia.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1490971537 8119 195.159.176.226 (31 Mar 2017 14:45:37 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 31 Mar 2017 14:45:37 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) Cc: guile-devel@gnu.org To: guix-devel@gnu.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Fri Mar 31 16:45:33 2017 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ctxnt-0001N2-KF for guile-devel@m.gmane.org; Fri, 31 Mar 2017 16:45:29 +0200 Original-Received: from localhost ([::1]:41407 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ctxnz-0001AD-G8 for guile-devel@m.gmane.org; Fri, 31 Mar 2017 10:45:35 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:38067) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ctxnp-00018B-FV for guile-devel@gnu.org; Fri, 31 Mar 2017 10:45:27 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ctxnl-0005m8-AH for guile-devel@gnu.org; Fri, 31 Mar 2017 10:45:25 -0400 Original-Received: from pb-sasl2.pobox.com ([64.147.108.67]:56728 helo=sasl.smtp.pobox.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ctxnl-0005Do-2p; Fri, 31 Mar 2017 10:45:21 -0400 Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl2.pobox.com (Postfix) with ESMTP id 042DD63636; Fri, 31 Mar 2017 10:44:43 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:date:message-id:mime-version:content-type; s=sasl; bh=8 0KD7WmruYwsjXDt/QaywiWb/Y4=; b=iVk2Fdc1PRgj9o1V+rSpneYJfdndB4a3c 4jBy8L9vnOHKj65pj2zpT9cfXnHt0n2RV8/zlLZJpmX2saPCNn0vUKLFBi3K/pcD C1Kh3skJ0/oqJdDI8K8la6CDfsG41iHRDBDU1fUUsdqwuwbQUu5KWAgM40cnV859 TAWo0y+QgY= Original-Received: from pb-sasl2.nyi.icgroup.com (unknown [127.0.0.1]) by pb-sasl2.pobox.com (Postfix) with ESMTP id F141C63635; Fri, 31 Mar 2017 10:44:42 -0400 (EDT) Original-Received: from rusty (unknown [88.160.190.192]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl2.pobox.com (Postfix) with ESMTPSA id F319863634; Fri, 31 Mar 2017 10:44:41 -0400 (EDT) X-Pobox-Relay-ID: 93200A6E-1620-11E7-ACB4-85AB91A0D1B0-02397024!pb-sasl2.pobox.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 64.147.108.67 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19079 gmane.comp.gnu.guix.devel:36698 Archived-At: Hi! Following on thoughts from earlier this month that Guix is the guildhall that we always wanted, I think I have more of a plan. s/guildhall/potluck/ -------------------- Firstly, a name change: the guidhall was a somewhat unified, curated thing that was Guile-specific, whereas I think I think we need to take this feast outdoors and move it around a bit. I would like to use the term "potluck" to refer to the offering of tasty home-hacked goods, the composition of which is but loosely planned. This is inspired by the "potluck" dishes that we would hack up to celebrate the Guile 2.0 release anniversary. I don't want to take the metaphor too far of course; I think there was a forced aspect around the guildhall name that never really became natural. Also, the potluck facility is just one part of the system. Firstly we use Guix itself as the package manager. Secondly we build on the upcoming "guix channel" facility. (In its most basic form, a channel is just a git branch somewhere that contains package files.) The job of the potluck facility is just cultivating the git branch that is the "potluck channel". As an interlude, here is how a user would enter an environment that has a potluck package "foo" using Guix (using a pack is also possible). We start with setup steps: (1) Install Guix as a user. (This needs to be easier.) (2) guix channel add potluck https://gitlab.com/potluck/potluck master (3) guix channel enable potluck Then: (4) guix environment --ad-hoc foo Assuming "foo" is a package that's in the potluck channel. Basically the potluck channel augments the set of available packages. This is handled by the channel mechanism. A packaging language for stability and security ----------------------------------------------- So how do packages enter the potluck channel? Good question, fictional reader! This is the tricky bit. There are some concerns here: (1) The Guix API is not stable and has no plans to be stable. This works great for now because all packages are in one atomic repository and people work on making the whole thing make sense together. One of the goals of the potluck effort is to decentralize things a bit, so we have an impedance mismatch between potluck packages and Guix itself. (2) Potluck package definitions will live in many different git repositories across the internet, and anyone should be able to make a potluck package. Some potluck package authors will be malicious. They could: 1. Damage the server that manages the potluck channel 2. Damage the users that run Guix commands with the potluck channel enabled 3. Damage the users that install potluck packages I think we need to forget about 3, for now at least. (Flatpak solves this, more or less; Guix has ongoing work to do here I think.) Both of these large issues point to the need for careful design of the language that potluck packages are written in. The language that Guix packages are written in is inappropriate because of (1). In particular we should not depend on which module a package comes from, and what identifier binds any given package. For (2), packages are currently written in full Scheme, staged between the Guix command itself and the sandbox that runs inside guix-daemon. Full Scheme might be OK in the daemon but it's not OK in the Guix command itself. Concretely I would propose that the language that potluck files are written in is like this: (1) It's code, not inert data. (2) It's a subset of Scheme, like core Guix packages. (3) The general structure looks like this: (import-guix-packages ((guile "guile@2.0") (glibc "glibc"))) (import-potluck-packages ((foo "foo"))) (define bar (package (name "guile-bar") (version "1.0.0") (build-system gnu-build-system) (inputs `(("guile" ,guile)) ....))) I.e. we have a special mechanism to import packages by name using specification->package. We can define packages using Guix's "package" form, and a number of standard Guix bindings are available to the code (license:gplv3+, gnu-build-system, etc). There is a particular concern about staging: there is staged Scheme code in these modules that runs inside build processes in guix-daemon. I don't have any nice solution here. Simply sandboxing the host Scheme will be fine enough for me, using (ice-9 sandbox) if that lands soon (see https://lists.gnu.org/archive/html/guile-devel/2017-03/msg00111.html). This requires Guile 2.2.1 (when it is released). I can see an argument to support more simple package definition, specifically that it should be easier to build Guile-only projects from git -- skipping autotools, skipping tarballs. But we should be able to support tarball builds containing C or anything else, so the full package capabilities are needed. I think a first version can use the normal package facilities. A potluck channel manager ------------------------- The "guix potluck manage-channel" command manages a registry of sources of potluck definitions and turns them into a git branch of package files. This is the web service. The idea is that as a developer, you should be able to do: guix potluck add https://gitlab.com/wingo/foo master This causes the client to make a request to some web service, say running on potluck.guixsd.org, to register that git branch. One question would be, who is able to register potluck sources? Do we need users and all that? I think we might be able to avoid that -- it could be that we just assume that any git branch that has a "potluck.scm" in its root directory has implicitly granted permission to be a part of the potluck repo. potluck.guixsd.org needs to be isolated from other hosts because it will load potluck.scm files from untrusted sources; we hope the sandbox works but we need a bit of defense-in-depth. As I mentioned, I think it would be nice to be able to install some potluck packages directly from git, without requiring those packages to make releases and update the potluck.scm. But until then, we can make it so that the source is fixed in the potluck.scm as it is with other Guix packages, and therefore that any update to potluck.scm in the source git branch registered with potluck.guixsd.org constitutes a new release which replaces the old one. A developer should signal potluck.guixsd.org about the update via a re-invocation of "guix potluck add". Maybe "guix potluck add" could remember the branch, dunno. Anyway! The result of the "guix potluck channel-manager" is a stream of guix modules as a continually updated git tree -- a guix channel. I am thinking that we need to rewrite these files to be more "normal" -- like starting with a (define-module), but a #:pure module and an appropriate set of imports to enforce the sandbox. We should be able to compile this module, to prevent the potluck channel from slowing things down. So basically the channel-manager rewrites the potluck.scm files. I think that's the essence of my proposal. WDYT? Andy