potluck status

potluck status

[Andy Wingo]

Hi!

To recall, "guix potluck" is a packaging facility that is a decoupled
from mainline Guix development.  The latest patches are in wip-potluck,
and a recent copy of the patches is here:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26645

Here is part of the documentation in that branch:

  5.15 Invoking ‘guix potluck’
  ============================

       Note: As of version 0.12.0, this tool is experimental.  The
       interface is subject to radical change in the future.

     Guix is developed as a unified project composed of both the package
  manager and the set of packages.  This allows Guix to evolve while
  remaining healthy and coherent.  If there is a change that needs to be
  done across Guix’s entire tree, Guix developers can make it happen.  One
  way in which this principle manifests itself is in the ‘package’ data
  type, where input packages are directly specified by value in the
  ‘inputs’, ‘native-inputs’, and ‘propagated-inputs’ fields, instead of
  being specified as some abstract package name and version constraint
  that Guix would have to solve for.  *Note package Reference::, for more
  on the ‘package’ data type.

     However it is sometimes desirable to develop a package or set of
  packages in a more decoupled way, for example when a package set is
  still incubating or when a package is inappropriate for sending upstream
  for some reason.  Such packages use Guix and extend Guix but are not a
  part of the Guix project, properly speaking.  As such, they need to be
  resilient to changes in upstream Guix.  It would be brittle if such a
  package definition had to reference a Guix package by value; the Scheme
  variable denoting the upstream Guix package might move to a different
  module, or be renamed, or changed in some unexpected way.

     Guix’s “potluck” facility fills this gap.  A “potluck package” is
  like a normal Guix package, except it expresses its inputs in the form
  of package specifications instead of direct references.  *Note
  potluck-package Reference::.  Potluck packages also have a simpler
  package structure with fewer fields; compared to normal Guix packages,
  they are less expressive but more isolated from details of upstream
  Guix.

     The user interface to potluck packages is concentrated in the ‘guix
  potluck’ command.  To begin, let’s say you are a developer of the ‘foo’
  project, and you’d like to package ‘foo’ for use in your Guix system and
  maybe also that of a friend.  You’re not sure if you want to support it
  yet so you don’t want to make a proper release, but there should be
  something in the middle between that and not packaging it at all.  You
  decide to give ‘guix potluck’ a go.

     So in that git checkout, you run ‘guix potluck init URL’, where URL
  is a publicly accessible git URL at which ‘foo’ is hosted.  ‘guix
  potluck init’ takes the following options:

  ‘--build-system=SYS’
  ‘--build-system=help’
  ‘--autotools’
       Indicate that the package uses the build system named SYS.  Pass
       ‘help’ as the build system to see available options.  ‘--autotools’
       is like the common ‘--build-system=gnu’, but additionally
       indicating that an ‘autoreconf’ step is needed before building.
  ‘--license=LICENSE’
  ‘--license=help’
       Specify the license of the project.

     Calling ‘guix potluck init’ will result in the creation of a
  ‘guix-potluck’ directory in your git checkout, containing a brief
  overview ‘README.md’ file as well as a ‘foo.scm’ potluck package
  definition.  *Note potluck-package Reference::.  Just fill in the
  synopsis and description and add the inputs and you have the beginnings
  of a potluck package.

     You can try building your new package by running ‘guix build -f
  guix-potluck/foo.scm’.  Once that works, you can share the file with
  your friends and they can build your package too.

     Of course, it would be nice if you could share that package with the
  world.  And it would be nice if your potluck package definitions could
  augment the set of available packages and versions.  And it would be
  nice if your potluck package could serve as a first draft of a proper
  Guix package definition.  We agree completely!

     Guix’s potluck facility also implements a kind of registry of potluck
  recipes, as if it were hosting an actual potluck.  This ‘host-channel’
  facility takes requests to add potluck packages and translates that into
  a git repository of all potluck packages, as well as a git repository of
  Guix packages compiled from those potluck packages.

     To inform a channel host of the presence of fresh tasty potluck
  dishes, run ‘guix potluck update URL BRANCH’.  URL should be the URL of
  a git repository containing a ‘guix-potluck’ directory, and BRANCH is a
  ref in that repository.  By default, the request is made to add the
  package to the default ‘guix-potluck.org’ host; pass ‘--host=HOST’ to
  specify an alternate registry.

     Running ‘guix potluck update’ will simply enqueue an update request
  on the server.  Visit ‘https://HOST/’ in your browser to see the state
  of the work queue, and to see whether your package update actually
  succeeded.  If it does succeed, you should be able to check out the git
  repository conventionally hosted at ‘https://HOST/git/main.git’, add
  that checkout to your ‘GUIX_PACKAGE_PATH’, and thereby have access to
  that package.  In the future this will be made easier with a ‘guix
  channel’ facility, but for now you have to use ‘GUIX_PACKAGE_PATH’ or
  similar.  *Note Defining Packages::, for more information.

     Finally, there is the code that actually runs the potluck service:
  ‘guix potluck host-channel’.  This tool will host a web server on
  localhost, listening on port 8080 by default.  It expects to be behind
  some kind of HTTPS terminator, like ‘nginx’.  It does the work of
  translating update requests to git repositories.  Guix includes an
  example operating system definition for a server combining HTTPS access
  to git repositories, an ‘nginx’ web front-end, and a ‘guix potluck
  host-channel’ instance.

With the release of Guile 2.2.2, I have re-enabled
https://guix-potluck.org.  I also refactored that server's configuration
to move some service definitions into Guix itself (the certbot service,
for example); see:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26684
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26685
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26686
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26687

I also updated the code for "guix potluck host-channel" to show recent
packages added, log entries, and so on.  Finally I found the source of
the HTTPS post error; it was in Guile:

  https://git.savannah.gnu.org/cgit/guile.git/commit/

With that I think I'd like to move to a "use" phase where I just sit
back and see how people use the thing :)  WDYT?  If you agree I can push
patches from the above bugs into master maybe this weekend or so.  We
can fix the "specification->package" interface in parallel as discussed
here:

  https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00401.html

when we add the "guix channel" facility, which I guess will depend on
guile-git.

Happy hacking,

Andy

Re: potluck status

[Hartmut Goebel]

Hi,

Am 28.04.2017 um 14:05 schrieb Andy Wingo:
>   5.15 Invoking ‘guix potluck’

Please think about an other name for this command. "potlouk"  may be
common to native speakers but I never heard this word. Thanks.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: potluck status

[Andy Wingo]

On Fri 28 Apr 2017 14:42, Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 28.04.2017 um 14:05 schrieb Andy Wingo:
>>   5.15 Invoking ‘guix potluck’
>
> Please think about an other name for this command. "potlouk"  may be
> common to native speakers but I never heard this word. Thanks.

I thought about many things :)  Do you have a suggestion?
https://guix-potluck.org/ does provide the definition.

Andy

Re: potluck status

[Christopher Allan Webber]

Andy Wingo writes:

> On Fri 28 Apr 2017 14:42, Hartmut Goebel <h.goebel@crazy-compilers.com> writes:
>
>> Am 28.04.2017 um 14:05 schrieb Andy Wingo:
>>>   5.15 Invoking ‘guix potluck’
>>
>> Please think about an other name for this command. "potlouk"  may be
>> common to native speakers but I never heard this word. Thanks.
>
> I thought about many things :)  Do you have a suggestion?
> https://guix-potluck.org/ does provide the definition.
>
> Andy

I personally like Potluck a lot; I know it might not be a particularly
common English word, but it may be a good one for people to learn if
they don't already know it.  It captures the spirit of things to me very
nicely.

Re: potluck status

[David Pirotte]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

Le Fri, 28 Apr 2017 11:46:43 -0500,
Christopher Allan Webber <cwebber@dustycloud.org> a écrit :

> Andy Wingo writes:
> 
> > On Fri 28 Apr 2017 14:42, Hartmut Goebel <h.goebel@crazy-compilers.com> writes:
> >  
> >> Am 28.04.2017 um 14:05 schrieb Andy Wingo:  
> >>>   5.15 Invoking ‘guix potluck’  
> >>
> >> Please think about an other name for this command. "potlouk"  may be
> >> common to native speakers but I never heard this word. Thanks.  
> >
> > I thought about many things :)  Do you have a suggestion?
> > https://guix-potluck.org/ does provide the definition.
> >
> > Andy  
> 
> I personally like Potluck a lot; I know it might not be a particularly
> common English word, but it may be a good one for people to learn if
> they don't already know it.  It captures the spirit of things to me very
> nicely.
 
1+
and this name is part of 'our' history

David

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: potluck status

[Ludovic Courtès]

Christopher Allan Webber <cwebber@dustycloud.org> skribis:

> Andy Wingo writes:
>
>> On Fri 28 Apr 2017 14:42, Hartmut Goebel <h.goebel@crazy-compilers.com> writes:
>>
>>> Am 28.04.2017 um 14:05 schrieb Andy Wingo:
>>>>   5.15 Invoking ‘guix potluck’
>>>
>>> Please think about an other name for this command. "potlouk"  may be
>>> common to native speakers but I never heard this word. Thanks.
>>
>> I thought about many things :)  Do you have a suggestion?
>> https://guix-potluck.org/ does provide the definition.
>>
>> Andy
>
> I personally like Potluck a lot; I know it might not be a particularly
> common English word, but it may be a good one for people to learn if
> they don't already know it.  It captures the spirit of things to me very
> nicely.

Perhaps we could add a short footnote in the manual to explain what a
potluck is and how this relate to all this?

Ludo’.

Re: potluck status

[Jan Nieuwenhuizen]

Hartmut Goebel writes:

> Please think about an other name for this command. "potlouk"  may be
> common to native speakers but I never heard this word. Thanks.

Potluck dish --even though it's English--- is a common word even in the
Netherlands...possibly this is a European thing.

I like the name, you are the one who wants another name; what would you
suggest?

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar®  http://AvatarAcademy.nl  

Re: potluck status

[Incoherent-lime]

[-- Attachment #1: Type: text/plain, Size: 840 bytes --]

In the UK,
We do potluck dinners, I also had one in Canada.

Though I agree it's quite colloquial term, does one have to understand the reference to understand how to use the command on the CLI?

-H

Sent from ProtonMail mobile

-------- Original Message --------
On 28 Apr 2017, 13:58, Jan Nieuwenhuizen wrote:
Hartmut Goebel writes:

> Please think about an other name for this command. "potlouk" may be
> common to native speakers but I never heard this word. Thanks.

Potluck dish --even though it's English--- is a common word even in the
Netherlands...possibly this is a European thing.

I like the name, you are the one who wants another name; what would you
suggest?

Greetings,
janneke

--
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.nl

[-- Attachment #2: Type: text/html, Size: 1139 bytes --]

Re: potluck status

[Katherine Cox-Buday]

I don't hold any strong opinions on the name, but in the interest of
ideas:

- guix quorum
- guix collab
- guix confer
- guix conjoin

That's all I have for now :)

-- 
Katherine

Re: potluck status

[Incoherent-lime]

[-- Attachment #1: Type: text/plain, Size: 378 bytes --]

I am personally happy with potluck, however collab has a similar group feeling to it.

-H

Sent from ProtonMail mobile -------- Original Message -------- On 28 Apr 2017, 14:46, Katherine Cox-Buday wrote: I don't hold any strong opinions on the name, but in the interest of ideas: - guix quorum - guix collab - guix confer - guix conjoin That's all I have for now :) -- Katherine

[-- Attachment #2: Type: text/html, Size: 420 bytes --]

Re: potluck status

[Katherine Cox-Buday]

Andy Wingo <wingo@igalia.com> writes:

> With that I think I'd like to move to a "use" phase where I just sit
> back and see how people use the thing :)  WDYT?

I am very excited about this functionality for all of the reasons you
enumerated. I don't have any time at the moment, but I was (and still
am) interested in getting Scala and sbt packaged, and I think this is
how I would begin. I could also see packaging a few private work-related
things as potluck packages for use by myself and others.

A couple of questions:

1. When Guix grows channel support, will it have a concept of
   dev->test->prod?
2. Would it make sense for the dev channel to be expressed in terms of
   potluck packages?

I'm not very active in the community at the moment, but I wanted to
chime in and say +1, and thanks for the effort.

-- 
Katherine

Re: potluck status

[Andy Wingo]

Hi :)

On Fri 28 Apr 2017 15:41, Katherine Cox-Buday <cox.katherine.e@gmail.com> writes:

> Andy Wingo <wingo@igalia.com> writes:
>
>> With that I think I'd like to move to a "use" phase where I just sit
>> back and see how people use the thing :)  WDYT?
>
> I am very excited about this functionality for all of the reasons you
> enumerated. I don't have any time at the moment, but I was (and still
> am) interested in getting Scala and sbt packaged, and I think this is
> how I would begin. I could also see packaging a few private work-related
> things as potluck packages for use by myself and others.
>
> A couple of questions:
>
> 1. When Guix grows channel support, will it have a concept of
>    dev->test->prod?
> 2. Would it make sense for the dev channel to be expressed in terms of
>    potluck packages?
>
> I'm not very active in the community at the moment, but I wanted to
> chime in and say +1, and thanks for the effort.

Great to hear the interest.  There are many many many unknowns at this
point.  The basic "channel" facility could be just like "git remote" --
you do "guix channel add testing https://..../testing.git".  "git
channel pull" or so could update that checkout.  "git channel enable"
makes a channel active for you by default.  I guess you would probably
also want to to specify also a set of active channels for a given guix
command; e.g. "guix package --channels=testing --install my-package".

Concretely you could "git channel add potluck
https://guix-potluck.org/git/target.git" to get the potluck channels.
Currently what you have to do is manually check out that repo and do
"guix package -L /path/to/checkout --install my-package".  So an initial
"guix channel" would just pave that guix-packages-in-a-git-repository
cowpath.

As for Scala and sbt and everything -- I think potluck packages are most
appropriate for "leaf" packages.  For packages that form
"infrastructure" like sbt and all, I think you will probably want to
integrate more closely in Guix.  But I don't know.

& as for dev/testing/prod/etc -- I have no idea :)  I'm not really an
ops person, so I can only speculate, and anyone can do that as well as I
can :)  I think with guix-potluck.org my main focus is to let people
share work-in-progress Guix packages immediately.  I can imagine many
ways this could relate to a sort of devopsy workflow but I can't pretend
to be an expert here :)

Cheers,

Andy

Re: potluck status

[Katherine Cox-Buday]

Andy Wingo <wingo@igalia.com> writes:

> I guess you would probably also want to to specify also a set of
> active channels for a given guix command; e.g. "guix package
> --channels=testing --install my-package".

Being able to work with packages at the level of granularity of channels
would be ah-mazing. So many possibilities there.

> As for Scala and sbt and everything -- I think potluck packages are most
> appropriate for "leaf" packages.  For packages that form
> "infrastructure" like sbt and all, I think you will probably want to
> integrate more closely in Guix.  But I don't know.

I agree! I brought up potluck because for Java-type things (scala and
sbt included) it is often very unclear how to tie it back to a
reproducible build (see Hartmut Goebel's herculean efforts with maven).
The idea that potluck makes it easy to get going and then provides an
easy way to share your in-progress work seemed to fit nicely.

> & as for dev/testing/prod/etc -- I have no idea :)  I'm not really an
> ops person, so I can only speculate, and anyone can do that as well as I
> can :)  I think with guix-potluck.org my main focus is to let people
> share work-in-progress Guix packages immediately.  I can imagine many
> ways this could relate to a sort of devopsy workflow but I can't pretend
> to be an expert here :)

Sorry, I didn't mean to be too dev-opsy. All I really meant is just like
we have a staging branch/channel now that eventually gets rolled into
master, we could have a dev branch/channel which is really unstable and
experimental, a staging branch/channel (which is test in my vernacular),
and master branch/channel (which is prod in my vernacular).

Potluck seemed appropriate for dev for the same reasons it seemed
appropriate for scala/sbt.

-- 
Katherine

Re: potluck status

[Andy Wingo]

Greets,

Just a quick yes-and note:

On Tue 02 May 2017 04:26, Katherine Cox-Buday <cox.katherine.e@gmail.com> writes:

> Andy Wingo <wingo@igalia.com> writes:
>> & as for dev/testing/prod/etc -- I have no idea :)  I'm not really an
>> ops person, so I can only speculate, and anyone can do that as well as I
>> can :)  I think with guix-potluck.org my main focus is to let people
>> share work-in-progress Guix packages immediately.  I can imagine many
>> ways this could relate to a sort of devopsy workflow but I can't pretend
>> to be an expert here :)
>
> Sorry, I didn't mean to be too dev-opsy. All I really meant is just like
> we have a staging branch/channel now that eventually gets rolled into
> master, we could have a dev branch/channel which is really unstable and
> experimental, a staging branch/channel (which is test in my vernacular),
> and master branch/channel (which is prod in my vernacular).
>
> Potluck seemed appropriate for dev for the same reasons it seemed
> appropriate for scala/sbt.

Oh I didn't mean to disparage this kind of workflow, rather the
opposite!  Apologies if I didn't communicate that right -- I think
there's lots of cool stuff to do in this area, and I look forward to
seeing what people come up with.

As far as having dev and stable channels of Guix itself -- you're right,
having a channel facility could be a step towards Guix itself having a
stable branch and a development branch and perhaps something in-between
too.  Really interesting to explore this space mentally.  In the past,
Guix developers have always said "sure but probably now isn't the right
time" when the topic of production vs development branches was brought
up -- and that could be the answer now too, but when channels are
implemented that certainly does seem like a good mechanism for it.

One thing about Guix packages is that currently every package should
have a unique "bindings" -- I mean, that you can't usefully have a
package bound to variable V in module M, and have different versions of
module M in A/M.scm and B/M.scm.  The potluck facility gives somewhat
unique names M to the modules, "solving" this problem there.  You can of
course have as many (package (name "foo")) values as you like, and Guix
will know about all of them and their versions.  But to be loadable they
need to have different bindings.  This makes channels inappropriate for
directly implementing Guix dev/testing/prod workflows, I think -- you
want to have the same module M, but three versions of it to choose
from.

So implementing a dev/testing/prod thing in Guix would require solving
this problem.  It's not so much a Guix problem as a Guile problem, in a
way, or a mismatch between how Guix wants to think of available packages
and how Guile wants to think about bindings.  Something to solve!

Andy

Re: potluck status

[Ludovic Courtès]

Hello!

Andy Wingo <wingo@igalia.com> skribis:

> Here is part of the documentation in that branch:
>
>   5.15 Invoking ‘guix potluck’
>   ============================

Excellent, nicely written as usual!

> With the release of Guile 2.2.2, I have re-enabled
> https://guix-potluck.org.  I also refactored that server's configuration
> to move some service definitions into Guix itself (the certbot service,
> for example); see:
>
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26684
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26685
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26686
>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26687

Awesome.

> I also updated the code for "guix potluck host-channel" to show recent
> packages added, log entries, and so on.  Finally I found the source of
> the HTTPS post error; it was in Guile:
>
>   https://git.savannah.gnu.org/cgit/guile.git/commit/
>
> With that I think I'd like to move to a "use" phase where I just sit
> back and see how people use the thing :)  WDYT?  If you agree I can push
> patches from the above bugs into master maybe this weekend or so.

The week-end has passed ;-), but I agree.  I’m taking a look at #26645
(the actual potluck patch) to see if I have anything to add, but yeah,
let’s push this real soon!

When we’re done, what about writing a blog post for the web site about
it?  I guess it could largely be a copy/paste of the documentation you
posted above.

> We can fix the "specification->package" interface in parallel as
> discussed here:
>
>   https://lists.gnu.org/archive/html/guix-devel/2017-04/msg00401.html
>
> when we add the "guix channel" facility, which I guess will depend on
> guile-git.

Yes, post 0.13.0, we’ll depend on Guile-Git and start doing fun stuff
with it I hope.

Ludo’.