new manifest format readers?

new manifest format readers?

[Ricardo Wurmus]

Hi Guix,

I received some feedback on the usability of Guix and one of the points
I got was about manifests.

For the simplest case, a user may just want to ask Guix to install
a fixed set of packages at specified versions.  Currently, doing this
involves some boilerplate code.

One can either write a manifest using variable names or convert a list
of spec strings to a manifest.  The latter case is used a lot at my
institute and it’s unfortunate that this requires so much code (with
emphasis on “code”, not on “so much”).

Here’s an example:

--8<---------------cut here---------------start------------->8---
(define packages
  (list "gfortran@5"
        "gcc-toolchain@5"
        "cdogs-sdl"))

;; Boilerplate code to turn the above list of packages into a manifest
(use-modules (gnu packages))
(packages->manifest (map (compose list specification->package+output) packages))
--8<---------------cut here---------------end--------------->8---

It’s not bad, but when a person uses specification strings they really
just care about the package names and versions.

Can we simplify this case by adding manifest readers for different
formats?  One simple format could be plain text:

--8<---------------cut here---------------start------------->8---
# this is for compilers
gfortran@5
gcc-toolchain@5

# this is for fun
cdogs-sdl
--8<---------------cut here---------------end--------------->8---

The manifest reader would just discard # comments and interpret each
non-empty line as a spec string.

While I love Scheme, I don’t think anything is to be gained from
exposing users who don’t want to use Scheme for manifests to opaque
blobs of code and syntax they don’t want to understand.

What do you think?

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: new manifest format readers?

[Mathieu Othacehe]

Hi Ricardo,

> It’s not bad, but when a person uses specification strings they really
> just care about the package names and versions.

I recently introduced Guix to some friends and writting a manifest is
one of the first task as a Guix user.

Explaining them "specification->package+output", "packages->manifest" and the
concept of multiple values in Guile was a bit tough.

So I would be in favor of adding guix package the ability to parse
manifest.json, manifest.yaml, or manifest.whatever format !

Mathieu

Re: new manifest format readers?

[Ludovic Courtès]

Hi,

Ricardo Wurmus <rekado@elephly.net> skribis:

> Can we simplify this case by adding manifest readers for different
> formats?  One simple format could be plain text:
>
> # this is for compilers
> gfortran@5
> gcc-toolchain@5
>
> # this is for fun
> cdogs-sdl
>
> The manifest reader would just discard # comments and interpret each
> non-empty line as a spec string.

Sounds reasonable.

I guess we’d need to add a new option to read this format, or should
‘guix package -m’ try to automatically dispatch based on the file name
extension with a new --format option to override the guess?

> While I love Scheme, I don’t think anything is to be gained from
> exposing users who don’t want to use Scheme for manifests to opaque
> blobs of code and syntax they don’t want to understand.

I thought everyone needed their dose of parentheses, but maybe you’re
right.  :-)

Ludo’.

Re: new manifest format readers?

[Jan Nieuwenhuizen]

Ricardo Wurmus writes:

> One can either write a manifest using variable names or convert a list
> of spec strings to a manifest.  The latter case is used a lot at my
> institute and it’s unfortunate that this requires so much code (with
> emphasis on “code”, not on “so much”).
>
> Here’s an example:
>
> (define packages
>   (list "gfortran@5"
>         "gcc-toolchain@5"
>         "cdogs-sdl"))

and don't forget things like

   (list icedtea-8 "jdk")

or

   `(,icedtea-8 "jdk")

> ;; Boilerplate code to turn the above list of packages into a manifest
> (use-modules (gnu packages))
> (packages->manifest (map (compose list specification->package+output) packages))

Yes, I finally found this and it is not very friendly/discoverable.

> Can we simplify this case by adding manifest readers for different
> formats?  One simple format could be plain text:

> # this is for compilers
> gfortran@5
> gcc-toolchain@5

Can I suggest sticking with s-expresions, and preferrably with something
that can be used verbatim in the packages section of (operating-system)?
New users will most probably have to maintain that too.

So something like

   (use-package-modules gcc)
   (packages
    ;; this is for compilers
    gfortran-5)

or go further even and add some sugar to remove use-package-modules and
do something like

   (package-strings
    ;; this is for compilers
    "gfortran@5"
    ;; this is for work
    "icedtea:jdk")

and have package-strings be usable in operating-system too?

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: new manifest format readers?

[Ricardo Wurmus]

Jan Nieuwenhuizen <janneke@gnu.org> writes:

> Ricardo Wurmus writes:
>
>> One can either write a manifest using variable names or convert a list
>> of spec strings to a manifest.  The latter case is used a lot at my
>> institute and it’s unfortunate that this requires so much code (with
>> emphasis on “code”, not on “so much”).
>>
>> Here’s an example:
>>
>> (define packages
>>   (list "gfortran@5"
>>         "gcc-toolchain@5"
>>         "cdogs-sdl"))
>
> and don't forget things like
>
>    (list icedtea-8 "jdk")
>
> or
>
>    `(,icedtea-8 "jdk")

You can use “icedtea:jdk” as a spec string.  But doing this with just
variable names does indeed require something awkward like (list
icedtea-8 "jdk").

> Can I suggest sticking with s-expresions, and preferrably with something
> that can be used verbatim in the packages section of (operating-system)?
> New users will most probably have to maintain that too.
>
> So something like
>
>    (use-package-modules gcc)
>    (packages
>     ;; this is for compilers
>     gfortran-5)
>
> or go further even and add some sugar to remove use-package-modules and
> do something like
>
>    (package-strings
>     ;; this is for compilers
>     "gfortran@5"
>     ;; this is for work
>     "icedtea:jdk")
>
> and have package-strings be usable in operating-system too?

I’m not sure about this, but I’ll think about it.  My initial response
is that there is probably relatively little to be gained from adding
more special syntax (“package-strings”) to S-expressions when we already
have regular Guile.  But maybe there’s something else we can do to
require less boilerplate when using package spec strings.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: new manifest format readers?

[Arun Isaac]

> or go further even and add some sugar to remove use-package-modules and
> do something like
>
>    (package-strings
>     ;; this is for compilers
>     "gfortran@5"
>     ;; this is for work
>     "icedtea:jdk")
>
> and have package-strings be usable in operating-system too?

I think this is a good idea.

At least as a new user, until I "got it", I found it rather
counter-intuitive that I had to duplicate the same package in two places
(once in the modules, and once in the packages section).

Once I realized I was importing variables from modules, I had to figure
out which module a package was in. This is not the user experience in
other distros. It may not be such a bad idea to spare users the trouble
of figuring this out.

Re: new manifest format readers?

[Ludovic Courtès]

Jan Nieuwenhuizen <janneke@gnu.org> skribis:

> Ricardo Wurmus writes:

[...]

>> ;; Boilerplate code to turn the above list of packages into a manifest
>> (use-modules (gnu packages))
>> (packages->manifest (map (compose list specification->package+output) packages))
>
> Yes, I finally found this and it is not very friendly/discoverable.

Agreed.  How about making these two changes:

  1. Include (gnu packages) in the default evaluation environment of
     manifests.

  2. (define (package-specifications->manifest specs)
       (packages->manifest (map (compose list specification->package+output)
                           specs)))

Thoughts?

(To be clear, my suggestion is to do this in addition to adding support
for a new format.)

Ludo’.

Re: new manifest format readers?

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>
>> Ricardo Wurmus writes:
>
> [...]
>
>>> ;; Boilerplate code to turn the above list of packages into a manifest
>>> (use-modules (gnu packages))
>>> (packages->manifest (map (compose list specification->package+output) packages))
>>
>> Yes, I finally found this and it is not very friendly/discoverable.
>
> Agreed.  How about making these two changes:
>
>   1. Include (gnu packages) in the default evaluation environment of
>      manifests.
>
>   2. (define (package-specifications->manifest specs)
>        (packages->manifest (map (compose list specification->package+output)
>                            specs)))
>
> Thoughts?

This sounds like a very useful improvement to me.

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: new manifest format readers?

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Jan Nieuwenhuizen <janneke@gnu.org> skribis:
>>
>>> Ricardo Wurmus writes:
>>
>> [...]
>>
>>>> ;; Boilerplate code to turn the above list of packages into a manifest
>>>> (use-modules (gnu packages))
>>>> (packages->manifest (map (compose list specification->package+output) packages))
>>>
>>> Yes, I finally found this and it is not very friendly/discoverable.
>>
>> Agreed.  How about making these two changes:
>>
>>   1. Include (gnu packages) in the default evaluation environment of
>>      manifests.
>>
>>   2. (define (package-specifications->manifest specs)
>>        (packages->manifest (map (compose list specification->package+output)
>>                            specs)))
>>
>> Thoughts?
>
> This sounds like a very useful improvement to me.

Done in c08ea55e7ec25261e4596bf6726a83c1ed056b94.

In fact #1 was already implemented.

Ludo’.

Re: new manifest format readers?

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

>>>   1. Include (gnu packages) in the default evaluation environment of
>>>      manifests.
>>>
>>>   2. (define (package-specifications->manifest specs)
>>>        (packages->manifest (map (compose list specification->package+output)
>>>                            specs)))
>>>
>>> Thoughts?
>>
>> This sounds like a very useful improvement to me.
>
> Done in c08ea55e7ec25261e4596bf6726a83c1ed056b94.

Beautifuli!
janneke

Re: new manifest format readers?

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Agreed.  How about making these two changes:
>
>   1. Include (gnu packages) in the default evaluation environment of
>      manifests.
>
>   2. (define (package-specifications->manifest specs)
>        (packages->manifest (map (compose list specification->package+output)
>                            specs)))

I just wanted to say that I think #2 is a fantastic idea.  Hide those
implementation details!  :-)

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: new manifest format readers?

[Björn Höfling]

On Fri, 09 Jun 2017 14:03:22 +0200
Ricardo Wurmus <rekado@elephly.net> wrote:

> Hi Guix,
> 
> I received some feedback on the usability of Guix and one of the
> points I got was about manifests.
> 
> For the simplest case, a user may just want to ask Guix to install
> a fixed set of packages at specified versions.  Currently, doing this
> involves some boilerplate code.
> 
> One can either write a manifest using variable names or convert a list
> of spec strings to a manifest.  The latter case is used a lot at my
> institute and it’s unfortunate that this requires so much code (with
> emphasis on “code”, not on “so much”).
> 
> Here’s an example:
> 
> --8<---------------cut here---------------start------------->8---
> (define packages
>   (list "gfortran@5"
>         "gcc-toolchain@5"
>         "cdogs-sdl"))
> 
> ;; Boilerplate code to turn the above list of packages into a manifest
> (use-modules (gnu packages))
> (packages->manifest (map (compose list specification->package+output)
> packages)) --8<---------------cut
> here---------------end--------------->8---
> 
> It’s not bad, but when a person uses specification strings they really
> just care about the package names and versions.
> 
> Can we simplify this case by adding manifest readers for different
> formats?  One simple format could be plain text:
> 
> --8<---------------cut here---------------start------------->8---
> # this is for compilers
> gfortran@5
> gcc-toolchain@5
> 
> # this is for fun
> cdogs-sdl
> --8<---------------cut here---------------end--------------->8---
> 
> The manifest reader would just discard # comments and interpret each
> non-empty line as a spec string.
> 
> While I love Scheme, I don’t think anything is to be gained from
> exposing users who don’t want to use Scheme for manifests to opaque
> blobs of code and syntax they don’t want to understand.
> 
> What do you think?


Cool :-)

I use Guix for a year now and haven't herd of the Manifests until
recently (contrary to what Mathieu stated), I just use the profile
generational or add my packages to the system-configuration.

I just stumbled over Manifests because I wanted to give Cuirass a try
and there I convert a list of package-strings into a manifest (as
explained in the Guix manual). I really thought how brainfucked this
is, if you are not used to Scheme and returning multiple values. Worse,
if you have one typo in your file, you are confronted with an obscure
stacktrace.

So, definitely a good idea!

And I would like to see it integrated into Cuirass: Currently I have to
prepare a complicated configuration, and a Scheme file that must contain
a function that takes a store (what is that? And why should _I_ care
about that?) and returns a data-structore of ... was it manifests or
derivations or ...

Instead, just give the git repository and a file with packages-list as
you proposed.

Björn



 

Re: new manifest format readers?

[Ricardo Wurmus]

Björn Höfling <bjoern.hoefling@bjoernhoefling.de> writes:

> And I would like to see it integrated into Cuirass: Currently I have to
> prepare a complicated configuration, and a Scheme file that must contain
> a function that takes a store (what is that? And why should _I_ care
> about that?) and returns a data-structore of ... was it manifests or
> derivations or ...
>
> Instead, just give the git repository and a file with packages-list as
> you proposed.

Cuirass is still in flux.  I agree that there are many things about it
that aren’t intuitive, and sadly the documentation is also lacking.

On a related note: the research group that I’m working with is open for
master’s students who would like to work on Guix + Cuirass for the
duration of a student project:

    http://bioinformatics.mdc-berlin.de/mastersprojects

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: new manifest format readers?

[myglc2]

On 06/09/2017 at 14:03 Ricardo Wurmus writes:

> Hi Guix,
>
> I received some feedback on the usability of Guix and one of the points
> I got was about manifests.
>
[...]
> What do you think?

I use manifests all the time.  In my experience, a more basic issue than
the syntax is that, when you make an error, you get smacked with an
uninformative trace back. It doesn't help that, depending on the
manifest form used, you must use specify a package in terms of a string
or a symbol.

So I wonder: When users complain about manifests, is it really about the
syntax or is it about the error handling?

ISTM that, before we create a new manifest syntax, we should add some
"package validation" that traps errors and produces more informative
messages.  I know nothing about guile error handling but that doesn't
stop me from imagining ...


... an enhanced 'package->manifest' function that does something like ...

          (use-package-modules guile emacs)
          (packages->manifest
           (list emacs
                 guile-2.0
                 foo
                 ;; Use a specific package output.
                 (list guile-2.0 "debug")))

Error: packages->manifest: 'foo' is unspecified.

Message: guix module foobar contains recipes for foo versions 3.0 &
         3.2. You can install package foo version 3.2 by adding
         '(use-package-modules foobar)' before '(packages->manifest
         ...'.


... an enhanced 'specification->package+output' function that does something like ...

(packages->manifest
           (map (compose list specification->package+output)
                '("emacs" "guile@2.0" "guile@2.0:debug" foo@3.4)))

Error: specification->package+output: You used 'foo@3.4' where a quoted
       string was expected.

Message: Guix contains recipes for foo versions 3.0 & 3.2. You can
         install foo version 3.2 by replacing 'foo' with '"foo"' or
         '"foo@3.2"'


... and an enhanced 'packages' function that does something like ...

       ;; Globally-installed packages.
       (packages (package-check tcpdump foo %base-packages))

Error: packages: 'foo' is unspecified.

Message: guix module foobar contains recipes for foo versions 3.0 &
         3.2. You can install package foo version 3.2 by adding
         '(use-package-modules foobar)' before '(packages ...'.

WDYT? - George