Return back original implementation for text-config serialization

Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 4628 bytes --]

During the upstreaming process of Guix Home commit
fee0bced7fec2f9950957976a28f033edd4f877c slipped into master.  It
introduces a different serialization approach for text-config from what
was orginally used:
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services/configuration.scm?h=2c451db39aabc69bdbf186f412ee6e0b4e180ccb#n386

This new approach is inconisistent with the ideas used in the number of
other home services (not only the ones using text-config):
https://git.sr.ht/~abcdw/rde/tree/47f6c65c82a4f6761fa1ff5b9405b363cfda6482/item/gnu/home-services

So I had to fork it back to avoid an inconsistency and renamed
text-config to gexp-text-config to avoid a confusion with upstreamed
version.
https://git.sr.ht/~abcdw/rde/tree/47f6c65c82a4f6761fa1ff5b9405b363cfda6482/item/gnu/home-services-utils.scm#L355

Having two serialization approaches stops me from upstreaming the rest
of home services from rde to Guix and also makes a split to rde home
services and guix home services, which I would like to avoid.

We need to decide, which approach should be used or we will end up with
two competitive incompatible implementations and unecessary split of
effort and lose of human force and time.

I wanted to solve this question as a part of
https://issues.guix.gnu.org/52698, but seems it doesn't get enough
attention (probably cause of the volume of information in the patch).

I'll provide a few technical details, which supports the original
appoarch.

Before fee0bc serialization for text-config worked this way:
--8<---------------cut here---------------start------------->8---
`("string here"
  ,#~"string valued gexp"
  "source \"
  ,(local-file "blabla.sh"))
--8<---------------cut here---------------end--------------->8---

would yeld something like:

--8<---------------cut here---------------start------------->8---
string here
string valued gexp
source \
/gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh
--8<---------------cut here---------------end--------------->8---


There is another less generalized example, which demonstrates how
serialization of sway configuration works right now.

--8<---------------cut here---------------start------------->8---
`((include ,(local-file "./sway/config"))
  (bindsym $mod+Ctrl+Shift+a exec
           ,(file-append emacs "/bin/emacsclient") -c --eval "'(eshell)'")
  (bindsym $mod+Ctrl+Shift+o "[class=\"IceCat\"]" kill)
  (input * ((xkb_layout us,ru)
            (xkb_variant dvorak,))))
--8<---------------cut here---------------end--------------->8---

would yield something like:

--8<---------------cut here---------------start------------->8---
include /gnu/store/408jwvh6wxxn1j85lj95fniih05gx5xj-config
bindsym $mod+Ctrl+Shift+a exec /gnu/store/...-emacsclient -c --eval '(eshell)'
bindsym $mod+Ctrl+Shift+o [class="IceCat"] kill
input * {
    xkb_layout us,ru
    xkb_variant dvorak,
}
--8<---------------cut here---------------end--------------->8---

The new text-config serialization implementation (after fee0bc commit)
doesn't support gexps and tries to insert the literal content of the
file in place where file-like object was used, which

1. Confuses the users.
People reporting that it's hard to implement something like

source \
/gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh

with the new approach (using file-like objects), and they switch to the
original implementation of home services for shells (the ones currently
living in rde repo), which allows to use gexps.

2. Looks strange implementation-wise.

If we want to insert the file path to file-like object for new-style
text-config internally we do something like

(mixed-text-file ...
 "source \" "\n"
 #~(read-everything-from-file
    #$(computed-file "unecessary-file-name"
       #~#$(local-file "blabla.sh"))))

when originally it was
(mixed-text-file
 "source \" "\n"
 (local-file "blabla.sh"))

3. Inconsistent with other home services.

I do not insist that one approach is superior to the other and there are
of course some pros and cons for both of them, but personally I find the
old one to be more user-friendly, simpler and cleaner.  Also, having the
new implementation stops me from upstreaming all other home services,
because almost all of them allows to use gexps inside configuration
field and resolves file-like object to the path in the store instead of
literal content.

Please, express arguments, opinions and rationales and let's decide:
Keep the new approach or return back to the original one.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Liliana Marie Prikler]

Hi Andrew,

Am Sonntag, dem 09.01.2022 um 12:12 +0300 schrieb Andrew Tropin:
> Before fee0bc serialization for text-config worked this way:
> --8<---------------cut here---------------start------------->8---
> `("string here"
>   ,#~"string valued gexp"
>   "source \"
>   ,(local-file "blabla.sh"))
> --8<---------------cut here---------------end--------------->8---
> 
> would yield something like:
> 
> --8<---------------cut here---------------start------------->8---
> string here
> string valued gexp
> source \
> /gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh
> --8<---------------cut here---------------end--------------->8---
> 
> [...]
> 
> The new text-config serialization implementation (after fee0bc
> commit) doesn't support gexps and tries to insert the literal content
> of the file in place where file-like object was used[.]
I agree that the old one looks nicer in this context, but wasn't the
new one introduced to solve the issue of (slurp-file-gexp) in the
importer?  Meaning whichever way we go, we need something that allows
us to insert literal file contents of another file at more or less G-
exp compile time.

Perhaps that issue could be solved, if instead the importer just reads
the file contents and declares it as a variable as in
(define %bashrc "
;; Original contents of bashrc
")
(define bashrc (plain-file %bashrc)).

WDYT?

> If we want to insert the file path to file-like object for new-style
> text-config internally we do something like
> 
> (mixed-text-file ...
>  "source \" "\n"
>  #~(read-everything-from-file
>     #$(computed-file "unecessary-file-name"
>        #~#$(local-file "blabla.sh"))))
> 
> when originally it was
> (mixed-text-file
>  "source \" "\n"
>  (local-file "blabla.sh"))
Is unnecessary-file-name ever created in this instance?  I think we
have something similar in other locations as well, where G-expressions
are merged -- public keys for substitutes spring to mind.  Perhaps all
it'd need to make use of new-style text configs is a better way of
quoting such things, e.g. a procedure which takes a file-like object
and produces a plain file with its name.

For the record, the use of (read-everything-from-file) in your example
-- a procedure which I'm unable to find in my local Guix checkout --
makes it somewhat difficult to come up with concrete solutions here, so
pardon me for being abstract.


Cheers

Re: Return back original implementation for text-config serialization

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1889 bytes --]

Andrew Tropin schreef op zo 09-01-2022 om 12:12 [+0300]:
> [...] There is another less generalized example, which demonstrates
> how
> serialization of sway configuration works right now.
> 
> --8<---------------cut here---------------start------------->8---
> `((include ,(local-file "./sway/config"))
>   (bindsym $mod+Ctrl+Shift+a exec
>            ,(file-append emacs "/bin/emacsclient") -c --eval
> "'(eshell)'")
>   (bindsym $mod+Ctrl+Shift+o "[class=\"IceCat\"]" kill)
>   (input * ((xkb_layout us,ru)
>             (xkb_variant dvorak,))))
> --8<---------------cut here---------------end--------------->8---
> 
> would yield something like:
> 
> --8<---------------cut here---------------start------------->8---
> include /gnu/store/408jwvh6wxxn1j85lj95fniih05gx5xj-config
> bindsym $mod+Ctrl+Shift+a exec /gnu/store/...-emacsclient -c --eval
> '(eshell)'
> bindsym $mod+Ctrl+Shift+o [class="IceCat"] kill
> input * {
>     xkb_layout us,ru
>     xkb_variant dvorak,
> }
> --8<---------------cut here---------------end--------------->8---
> 
> The new text-config serialization implementation (after fee0bc
> commit)
> doesn't support gexps and tries to insert the literal content of the
> file in place where file-like object was used, which
> 
> 1. Confuses the users. [...]
> 2. Looks strange implementation-wise. [...]
> (mixed-text-file ...
> "source \" "\n"
> #~(read-everything-from-file
>    #$(computed-file "unecessary-file-name"
>       #~#$(local-file "blabla.sh"))))
> when originally it was
> (mixed-text-file
>  "source \" "\n"
>  (local-file "blabla.sh"))

Can we have (mixed-text-file "source \" "\n" (local-file "blabla.sh"))
and the sway configuration you quoted?  That syntax seems reasonable
and easy to use to me.  But without reintroducing slurp-file-gexp.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Return back original implementation for text-config serialization

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1889 bytes --]

Andrew Tropin schreef op zo 09-01-2022 om 12:12 [+0300]:
> [...] There is another less generalized example, which demonstrates
> how
> serialization of sway configuration works right now.
> 
> --8<---------------cut here---------------start------------->8---
> `((include ,(local-file "./sway/config"))
>   (bindsym $mod+Ctrl+Shift+a exec
>            ,(file-append emacs "/bin/emacsclient") -c --eval
> "'(eshell)'")
>   (bindsym $mod+Ctrl+Shift+o "[class=\"IceCat\"]" kill)
>   (input * ((xkb_layout us,ru)
>             (xkb_variant dvorak,))))
> --8<---------------cut here---------------end--------------->8---
> 
> would yield something like:
> 
> --8<---------------cut here---------------start------------->8---
> include /gnu/store/408jwvh6wxxn1j85lj95fniih05gx5xj-config
> bindsym $mod+Ctrl+Shift+a exec /gnu/store/...-emacsclient -c --eval
> '(eshell)'
> bindsym $mod+Ctrl+Shift+o [class="IceCat"] kill
> input * {
>     xkb_layout us,ru
>     xkb_variant dvorak,
> }
> --8<---------------cut here---------------end--------------->8---
> 
> The new text-config serialization implementation (after fee0bc
> commit)
> doesn't support gexps and tries to insert the literal content of the
> file in place where file-like object was used, which
> 
> 1. Confuses the users. [...]
> 2. Looks strange implementation-wise. [...]
> (mixed-text-file ...
> "source \" "\n"
> #~(read-everything-from-file
>    #$(computed-file "unecessary-file-name"
>       #~#$(local-file "blabla.sh"))))
> when originally it was
> (mixed-text-file
>  "source \" "\n"
>  (local-file "blabla.sh"))

Can we have (mixed-text-file "source \" "\n" (local-file "blabla.sh"))
and the sway configuration you quoted?  That syntax seems reasonable
and easy to use to me.  But without reintroducing slurp-file-gexp.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 5854 bytes --]

On 2022-01-09 12:19, Liliana Marie Prikler wrote:

> Hi Andrew,
>
> Am Sonntag, dem 09.01.2022 um 12:12 +0300 schrieb Andrew Tropin:
>> Before fee0bc serialization for text-config worked this way:
>> --8<---------------cut here---------------start------------->8---
>> `("string here"
>>   ,#~"string valued gexp"
>>   "source \"
>>   ,(local-file "blabla.sh"))
>> --8<---------------cut here---------------end--------------->8---
>> 
>> would yield something like:
>> 
>> --8<---------------cut here---------------start------------->8---
>> string here
>> string valued gexp
>> source \
>> /gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh
>> --8<---------------cut here---------------end--------------->8---
>> 
>> [...]
>> 
>> The new text-config serialization implementation (after fee0bc
>> commit) doesn't support gexps and tries to insert the literal content
>> of the file in place where file-like object was used[.]
> I agree that the old one looks nicer in this context, but wasn't the
> new one introduced to solve the issue of (slurp-file-gexp) in the
> importer?  Meaning whichever way we go, we need something that allows
> us to insert literal file contents of another file at more or less G-
> exp compile time.
>

From my experience the usage of slurp-file-gexp is somewhat really rare,
so I didn't upstream it originally, keeping in mind that it is possible
to use the gexp mentioned below directly and that later we can add
slurp-file-gexp or alternative if necessary.  Just missed that importer
already uses it.

We could just do something like that instead of slurp-file-gexp:
--8<---------------cut here---------------start------------->8---
#~(call-with-input-file #$(local-file "./files/bashrc")
    (@ (ice-9 textual-ports) get-string-all))
--8<---------------cut here---------------end--------------->8---

Or just take the implementation
https://git.sr.ht/~abcdw/rde/tree/47f6c65c82a4f6761fa1ff5b9405b363cfda6482/gnu/home-services-utils.scm#L90
and rename it to read-file-content-gexp or somewhat more apropriate and
use it.

> 
> Perhaps that issue could be solved, if instead the importer just reads
> the file contents and declares it as a variable as in (define %bashrc
> " ;; Original contents of bashrc ") (define bashrc (plain-file
> %bashrc)).
>
> WDYT?
>

Another possible solution, but I would prefer to stick with the direct
usage of gexp with the code for reading a file, because importer is
intended for creation of an example configuration and user will need to
continue the work on it and copying the content of bashrc and other
configs to scheme files, escaping string can be a little tedious.

>> If we want to insert the file path to file-like object for new-style
>> text-config internally we do something like
>> 
>> (mixed-text-file ...
>>  "source \" "\n"
>>  #~(read-everything-from-file
>>     #$(computed-file "unecessary-file-name"
>>        #~#$(local-file "blabla.sh"))))
>> 
>> when originally it was
>> (mixed-text-file
>>  "source \" "\n"
>>  (local-file "blabla.sh"))
> Is unnecessary-file-name ever created in this instance?  I think we
> have something similar in other locations as well, where G-expressions
> are merged -- public keys for substitutes spring to mind.  Perhaps all
> it'd need to make use of new-style text configs is a better way of
> quoting such things, e.g. a procedure which takes a file-like object
> and produces a plain file with its name.
>
> For the record, the use of (read-everything-from-file) in your example
> -- a procedure which I'm unable to find in my local Guix checkout --
> makes it somewhat difficult to come up with concrete solutions here, so
> pardon me for being abstract.

read-everything-from-file is just a shorthand for

--8<---------------cut here---------------start------------->8---
#~(begin
    (use-modules (ice-9 rdelim))
    (with-fluids ((%default-port-encoding "UTF-8"))
      (with-input-from-file #$COMPUTED_FILE_CALL_HERE read-string)))
--8<---------------cut here---------------end--------------->8---

a gexp used in
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services/configuration.scm?h=2c451db39aabc69bdbf186f412ee6e0b4e180ccb#n386

The example was already verbose, so I decided to simplify it a little
bit.

Actually, it's very similar to what slurp-file-gexp does, but it's moved
inside serializer and hidden from user.  Why not to give it to the user
and eleminate two more layers of wrapping in gexps and file-likes.

To demonstrate the idea of those additional layers, I'll "rephrase" the
previous example in slightly different form.

Originally it was:
--8<---------------cut here---------------start------------->8---
(mixed-text-file
 ...
 ;; evaluated to path to the store
 file-like
 ;; evaluated to the string value ; 1
 gexp ; can be the result of slurp-file-gexp function if we
      ; need the file content
 ;; evaluated to itself
 "string")
--8<---------------cut here---------------end--------------->8---

It became:
--8<---------------cut here---------------start------------->8---
(mixed-text-file
 ...
 ;; evaluated to path to the store ; 2
 (read-everything-from-file (mixed-text-file "intermediate-name" file-like))
 ;; evaluated to the string value ; 3
 (read-everything-from-file (computed-file "intermediate-name" gexp))
 ;; evaluated to itself
 "string")
--8<---------------cut here---------------end--------------->8---

New style is not that bad, especially having some helper functions
(get-file-path file-like) and (get-value-of-gexp gexp) for 2 and 3, but
why?  Especially, if we can have only one slurp-file-gexp like helper
for 1, which is rarely used and maybe not even really needed.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 2745 bytes --]

On 2022-01-09 13:19, Maxime Devos wrote:

> Andrew Tropin schreef op zo 09-01-2022 om 12:12 [+0300]:
>> [...] There is another less generalized example, which demonstrates
>> how
>> serialization of sway configuration works right now.
>> 
>> --8<---------------cut here---------------start------------->8---
>> `((include ,(local-file "./sway/config"))
>>   (bindsym $mod+Ctrl+Shift+a exec
>>            ,(file-append emacs "/bin/emacsclient") -c --eval
>> "'(eshell)'")
>>   (bindsym $mod+Ctrl+Shift+o "[class=\"IceCat\"]" kill)
>>   (input * ((xkb_layout us,ru)
>>             (xkb_variant dvorak,))))
>> --8<---------------cut here---------------end--------------->8---
>> 
>> would yield something like:
>> 
>> --8<---------------cut here---------------start------------->8---
>> include /gnu/store/408jwvh6wxxn1j85lj95fniih05gx5xj-config
>> bindsym $mod+Ctrl+Shift+a exec /gnu/store/...-emacsclient -c --eval
>> '(eshell)'
>> bindsym $mod+Ctrl+Shift+o [class="IceCat"] kill
>> input * {
>>     xkb_layout us,ru
>>     xkb_variant dvorak,
>> }
>> --8<---------------cut here---------------end--------------->8---
>> 
>> The new text-config serialization implementation (after fee0bc
>> commit)
>> doesn't support gexps and tries to insert the literal content of the
>> file in place where file-like object was used, which
>> 
>> 1. Confuses the users. [...]
>> 2. Looks strange implementation-wise. [...]
>> (mixed-text-file ...
>> "source \" "\n"
>> #~(read-everything-from-file
>>    #$(computed-file "unecessary-file-name"
>>       #~#$(local-file "blabla.sh"))))
>> when originally it was
>> (mixed-text-file
>>  "source \" "\n"
>>  (local-file "blabla.sh"))
>
> Can we have (mixed-text-file "source \" "\n" (local-file "blabla.sh"))
> and the sway configuration you quoted?  That syntax seems reasonable
> and easy to use to me.  But without reintroducing slurp-file-gexp.
>
> Greetings,
> Maxime.

I think we can just rely on something like that:
--8<---------------cut here---------------start------------->8---
#~(call-with-input-file #$(local-file "./files/bashrc")
    (@ (ice-9 textual-ports) get-string-all))
--8<---------------cut here---------------end--------------->8---

As I mentioned in reply to Liliana, from my experience it's a rare case,
when we really need to slurp the content of the file, in most cases
include/source and other similar consturctions can do the trick, so it
maybe not necessary to have a helper for this case at all.

WDYT?

BTW, even after reading "Code Staging in GNU Guix" paper, I still not
sure what the problem with slurp-file-gexp (except maybe name :) ).

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1536 bytes --]

Andrew Tropin schreef op zo 09-01-2022 om 20:59 [+0300]:
> [...]
> I think we can just rely on something like that:
> --8<---------------cut here---------------start------------->8---
> #~(call-with-input-file #$(local-file "./files/bashrc")
>     (@ (ice-9 textual-ports) get-string-all))
> --8<---------------cut here---------------end--------------->8---
> 
> As I mentioned in reply to Liliana, from my experience it's a rare case,
> when we really need to slurp the content of the file, in most cases
> include/source and other similar consturctions can do the trick, so it
> maybe not necessary to have a helper for this case at all.
> 
> WDYT?
> 
> BTW, even after reading "Code Staging in GNU Guix" paper, I still not
> sure what the problem with slurp-file-gexp (except maybe name :) ).

Looking at the reasons at <https://issues.guix.gnu.org/52698>, the
issue seems mostly naming to me.  When I read 'slurp-file-gexp', what I
thought was happening, is that first the file-like object is lowered
and then the G-Exp of the builder is extracted from the derivation.

Maybe name it

(define (file-contents object)
  "Construct a G-expression that reads the file-like object OBJECT
and returns the contents as a string."
  ;; TODO: do we need to set the encoding to UTF-8 here?
  #~((@ (relevant guile module) call-with-input-file)
     #$object
     (@ (ice-9 textual-ports) get-string-all)))

That would avoid having to remember the (call-with-input-file ...)
trick.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Return back original implementation for text-config serialization

[Liliana Marie Prikler]

Hi,

Am Sonntag, dem 09.01.2022 um 20:48 +0300 schrieb Andrew Tropin:
> On 2022-01-09 12:19, Liliana Marie Prikler wrote:
> 
> > Hi Andrew,
> > 
> > Am Sonntag, dem 09.01.2022 um 12:12 +0300 schrieb Andrew Tropin:
> > 
> > > Before fee0bc serialization for text-config worked this way:
> > > --8<---------------cut here---------------start------------->8---
> > > `("string here"
> > >   ,#~"string valued gexp"
> > >   "source \"
> > >   ,(local-file "blabla.sh"))
> > > --8<---------------cut here---------------end--------------->8---
> > > 
> > > would yield something like:
> > > 
> > > --8<---------------cut here---------------start------------->8---
> > > string here
> > > string valued gexp
> > > source \
> > > /gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh
> > > --8<---------------cut here---------------end--------------->8---
> > > 
> > > [...]
> > > 
> > > The new text-config serialization implementation (after fee0bc
> > > commit) doesn't support gexps and tries to insert the literal
> > > content
> > > of the file in place where file-like object was used[.]
> > I agree that the old one looks nicer in this context, but wasn't
> > the new one introduced to solve the issue of (slurp-file-gexp) in
> > the importer?  Meaning whichever way we go, we need something that
> > allows us to insert literal file contents of another file at more
> > or less G- exp compile time.
> > 
> 
> From my experience the usage of slurp-file-gexp is somewhat really
> rare, so I didn't upstream it originally, keeping in mind that it is
> possible to use the gexp mentioned below directly and that later we
> can add slurp-file-gexp or alternative if necessary.  Just missed
> that importer already uses it.
> 
> We could just do something like that instead of slurp-file-gexp:
> --8<---------------cut here---------------start------------->8---
> #~(call-with-input-file #$(local-file "./files/bashrc")
>     (@ (ice-9 textual-ports) get-string-all))
> --8<---------------cut here---------------end--------------->8---
> 
> Or just take the implementation
> https://git.sr.ht/~abcdw/rde/tree/47f6c65c82a4f6761fa1ff5b9405b363cfda6482/gnu/home-services-utils.scm#L90
> and rename it to read-file-content-gexp or somewhat more apropriate
> and use it.
Using ill-defined slurp-patterns at all just adds ways of shooting
yourself in the foot.  You're turning Guix into an ouroboros that reads
itself inside-out.  Better restrict such patterns to where they cause
the least harm and make their effects very explicit.

> > Perhaps that issue could be solved, if instead the importer just
> > reads the file contents and declares it as a variable as in (define
> > %bashrc " ;; Original contents of bashrc ") (define bashrc (plain-
> > file %bashrc)).
> > 
> > WDYT?
> 
> Another possible solution, but I would prefer to stick with the
> direct usage of gexp with the code for reading a file, because
> importer is intended for creation of an example configuration and
> user will need to continue the work on it and copying the content of
> bashrc and other configs to scheme files, escaping string can be a
> little tedious.
There is certainly a tradeoff here, but I think explicitly showing
(plain-file CONTENT) is the right approach here.  Users are going to
figure out mixed-text-file exists soon enough.  As for proper string
escaping, that's not really an excuse imo -- pretty-print exists and
you can use it. 

> read-everything-from-file is just a shorthand for
> 
> --8<---------------cut here---------------start------------->8---
> #~(begin
>     (use-modules (ice-9 rdelim))
>     (with-fluids ((%default-port-encoding "UTF-8"))
>       (with-input-from-file #$COMPUTED_FILE_CALL_HERE read-string)))
> --8<---------------cut here---------------end--------------->8---
> 
> a gexp used in
> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services/configuration.scm?h=2c451db39aabc69bdbf186f412ee6e0b4e180ccb#n386
> 
> The example was already verbose, so I decided to simplify it a little
> bit.
> 
> Actually, it's very similar to what slurp-file-gexp does, but it's
> moved inside serializer and hidden from user.  Why not to give it to
> the user and eleminate two more layers of wrapping in gexps and file-
> likes.
That's like saying "memory management already exists, but it's hidden
from the user, why not let them play around with malloc?"  In
programming languages that aren't C/C++, abstractions ought to make it
harder to shoot yourself in the foot.  You (in my POV correctly)
pointed out that our current handling of text configs makes it very
easy to shoot yourself in the foot and is therefore badly abstracted. 
The point I'm making is that we shouldn't swap out one bad abstraction
for another, but pave the road towards good abstractions, e.g. G-
expressions in the way the rest of Guix typically uses them.

Now one thing I had not considered back in my previous mail was that we
want to be able to compose bashrc together from multiple sources, so
having the field be just one file-like object probably won't cut it. 
However, we can let it be a list of file like objects (with a field
sanitizer transparently turning a single file-like object into a list
either silently or loudly).  And within those file-like objects, we can
use the usual semantics.

I'm not sure how the current implementation of Guix Home handles
composition, but I believe the root of the issue probably stems from
there in some capacity.  Might want to check what our requirements are.


Cheers

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 9228 bytes --]

On 2022-01-09 20:44, Liliana Marie Prikler wrote:

> Hi,
>
> Am Sonntag, dem 09.01.2022 um 20:48 +0300 schrieb Andrew Tropin:
>> On 2022-01-09 12:19, Liliana Marie Prikler wrote:
>> 
>> > Hi Andrew,
>> > 
>> > Am Sonntag, dem 09.01.2022 um 12:12 +0300 schrieb Andrew Tropin:
>> > 
>> > > Before fee0bc serialization for text-config worked this way:
>> > > --8<---------------cut here---------------start------------->8---
>> > > `("string here"
>> > >   ,#~"string valued gexp"
>> > >   "source \"
>> > >   ,(local-file "blabla.sh"))
>> > > --8<---------------cut here---------------end--------------->8---
>> > > 
>> > > would yield something like:
>> > > 
>> > > --8<---------------cut here---------------start------------->8---
>> > > string here
>> > > string valued gexp
>> > > source \
>> > > /gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh
>> > > --8<---------------cut here---------------end--------------->8---
>> > > 
>> > > [...]
>> > > 
>> > > The new text-config serialization implementation (after fee0bc
>> > > commit) doesn't support gexps and tries to insert the literal
>> > > content
>> > > of the file in place where file-like object was used[.]
>> > I agree that the old one looks nicer in this context, but wasn't
>> > the new one introduced to solve the issue of (slurp-file-gexp) in
>> > the importer?  Meaning whichever way we go, we need something that
>> > allows us to insert literal file contents of another file at more
>> > or less G- exp compile time.
>> > 
>> 
>> From my experience the usage of slurp-file-gexp is somewhat really
>> rare, so I didn't upstream it originally, keeping in mind that it is
>> possible to use the gexp mentioned below directly and that later we
>> can add slurp-file-gexp or alternative if necessary.  Just missed
>> that importer already uses it.
>> 
>> We could just do something like that instead of slurp-file-gexp:
>> --8<---------------cut here---------------start------------->8---
>> #~(call-with-input-file #$(local-file "./files/bashrc")
>>     (@ (ice-9 textual-ports) get-string-all))
>> --8<---------------cut here---------------end--------------->8---
>> 
>> Or just take the implementation
>> https://git.sr.ht/~abcdw/rde/tree/47f6c65c82a4f6761fa1ff5b9405b363cfda6482/gnu/home-services-utils.scm#L90
>> and rename it to read-file-content-gexp or somewhat more apropriate
>> and use it.
> Using ill-defined slurp-patterns at all just adds ways of shooting
> yourself in the foot.  You're turning Guix into an ouroboros that reads
> itself inside-out.  Better restrict such patterns to where they cause
> the least harm and make their effects very explicit.

Not sure what you mean.

>> > Perhaps that issue could be solved, if instead the importer just
>> > reads the file contents and declares it as a variable as in (define
>> > %bashrc " ;; Original contents of bashrc ") (define bashrc (plain-
>> > file %bashrc)).
>> > 
>> > WDYT?
>> 
>> Another possible solution, but I would prefer to stick with the
>> direct usage of gexp with the code for reading a file, because
>> importer is intended for creation of an example configuration and
>> user will need to continue the work on it and copying the content of
>> bashrc and other configs to scheme files, escaping string can be a
>> little tedious.
> There is certainly a tradeoff here, but I think explicitly showing
> (plain-file CONTENT) is the right approach here.  Users are going to
> figure out mixed-text-file exists soon enough.  As for proper string
> escaping, that's not really an excuse imo -- pretty-print exists and
> you can use it. 

Yep, of course it possible, but I mean that the whole point of escape
hatch is to make it possible to reuse existing files directly whithout
any manipulation on them and importer should demonstrate how to do it.
If importer internally do some manipulation (escaping) with the content
of the file and places the result in the string in scheme file, user
won't be able to replicate such process easily for other services, which
not covered by the importer. If I understood correctly what you meant in
the first message.

>> read-everything-from-file is just a shorthand for
>> 
>> --8<---------------cut here---------------start------------->8---
>> #~(begin
>>     (use-modules (ice-9 rdelim))
>>     (with-fluids ((%default-port-encoding "UTF-8"))
>>       (with-input-from-file #$COMPUTED_FILE_CALL_HERE read-string)))
>> --8<---------------cut here---------------end--------------->8---
>> 
>> a gexp used in
>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services/configuration.scm?h=2c451db39aabc69bdbf186f412ee6e0b4e180ccb#n386
>> 
>> The example was already verbose, so I decided to simplify it a little
>> bit.
>> 
>> Actually, it's very similar to what slurp-file-gexp does, but it's
>> moved inside serializer and hidden from user.  Why not to give it to
>> the user and eleminate two more layers of wrapping in gexps and file-
>> likes.
> That's like saying "memory management already exists, but it's hidden
> from the user, why not let them play around with malloc?"  In
> programming languages that aren't C/C++, abstractions ought to make it
> harder to shoot yourself in the foot.  You (in my POV correctly)
> pointed out that our current handling of text configs makes it very
> easy to shoot yourself in the foot and is therefore badly abstracted.

Mostly my point was that it's more intuitive to expect that file-like
objects get serialized to the file path in the store and string-valued
gexps to the strings (as they does when we use them in mixed-text-file
for example), but new approach doesn't allow to use gexps directly and
serializes file-likes to it's content and requires much more hassle.

In both cases it's possible to use both gexps and file-likes, but in the
second one we have a few more levels of nestiness of gexps inside
file-likes inside gexps, which seems unecessary.

> The point I'm making is that we shouldn't swap out one bad abstraction
> for another, but pave the road towards good abstractions, e.g. G-
> expressions in the way the rest of Guix typically uses them.

Actually, for me, the original implementation looks consistent with how
the rest of Guix treats G-expressions (uses already known abstraction)
and only new one intoduces a new abstraction.

We can treat slurp-file-gexp as an abstraction here, but actually
1. this is more a tiny helper function, than an abstraction.

On 2022-01-09 20:00, Maxime Devos wrote:
> That would avoid having to remember the (call-with-input-file ...)
> trick.

2. this function is not really necessary.

>
> Now one thing I had not considered back in my previous mail was that we
> want to be able to compose bashrc together from multiple sources, so
> having the field be just one file-like object probably won't cut it. 
> However, we can let it be a list of file like objects (with a field
> sanitizer transparently turning a single file-like object into a list
> either silently or loudly).  And within those file-like objects, we can
> use the usual semantics.

I considered this option, but the problem is that we can't control the
place where the content will be inserted.  Let's assume it will go at
the end of the bashrc (we can demonstrate the similar if it will be
added to the beginning).

(home-bash-configuration
 (bashrc
  (list "call_function_from_lib_A()"))
 (bashrc-content
  (list (local-file "lib_A.sh"))))

which will make a call before lib_A loaded.  Also, it's already possible
to achieve the same with gexps/file-like in both new and old text-config
implementations.

In reality, it's rarely needed to insert the content directly, in most
cases it better to source/include/whatever file from the store.

With old implementation it will look like:

(home-bash-configuration
 (bashrc
  (list
   "source \\"
   (local-file "lib_A.sh")
   ;; Or an alternative way to do the same with gexp
   ;; #~(format #f "source ~a" #$(local-file "lib_A.sh"))
   "call_function_from_lib_A()")))

With new implementation:

(home-bash-configuration
 (bashrc
  (list
   "source \\"
   (mixed-text-file
    "unecessary-name"
    (local-file "lib_A.sh"))
   ;; Or an alternative way to do the same with gexp
   ;; (computed-file
   ;;  "unecessary-name"
   ;;  #~(format #f "source ~a" #$(local-file "lib_A.sh")))
   "call_function_from_lib_A()")))

which not that bad, but still looks more verbose, less intuitive and
consistent with the rest of the Guix (at least in my perception of it).

* All the code examples are in a pseudocode and maybe don't work and
  used mostly to demonstarte the idea, provide a feeling of how final
  configurations look.

>
> I'm not sure how the current implementation of Guix Home handles
> composition, but I believe the root of the issue probably stems from
> there in some capacity.  Might want to check what our requirements are.


It's offtopic, but when you will have time please take a look at
https://issues.guix.gnu.org/52388.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Liliana Marie Prikler]

Am Montag, dem 10.01.2022 um 12:49 +0300 schrieb Andrew Tropin:
> [T]he whole point of escape hatch is to make it possible to reuse
> existing files directly without any manipulation on them and importer
> should demonstrate how to do it.
That'd make more sense if the importer copied bashrc to some well-known
location (e.g. ~/.config/guix/data/.bashrc) and then used that rather
than the file it wishes to replace.  IIRC that was one suggested
behaviour of Guix Home in the past, but it didn't get approval because
users wouldn't typically ask for that copying to happen.  If you make
it so that plain-file is used normally, but add a switch to express
things in terms of local-file instead, that'd work.  

OTOH, I do think local-file is already well-documented on its own, so
perhaps it'd only take a cookbook entry to show it in combination with
Guix Home and an explanation as to why Guix Home doesn't do that
normally while explaining all the caveats.

> If importer internally do some manipulation (escaping) with the
> content of the file and places the result in the string in scheme
> file, user won't be able to replicate such process easily for other
> services, which not covered by the importer. If I understood
> correctly what you meant in the first message.
Yes, the user would have to manually quote every new line of code
they're adding, but they're free to use all other file-like objects,
including local-file.  Having (bashrc (local-file ".bashrc")), whether
implemented on top of slurp-file-gexp or not, is inherently dangerous,
though.

> > The point I'm making is that we shouldn't swap out one bad
> > abstraction for another, but pave the road towards good
> > abstractions, e.g. G-expressions in the way the rest of Guix
> > typically uses them.
> 
> Actually, for me, the original implementation looks consistent with
> how the rest of Guix treats G-expressions (uses already known
> abstraction) and only new one intoduces a new abstraction.
That's the point.  The old style works just like you'd expect it to, it
becomes a problem when you try to feed it stuff like slurp-file-gexp to
work around some limitations in a way I'm not convinced makes sense.

> [I]t's already possible to achieve the same [-- merging multiple
> bashrc snippets into a single file --] with gexps/file-like in both
> new and old text-config implementations.
I find the lack of services in your example concerning, but I'll take
your word for it.  In that case, using a gexp for bashrc in the typical
sense is probably the best idea, but we still need to do something with
the reliance on slurp-file-gexp.

Cheers

> 
PS:
> It's offtopic, but when you will have time please take a look at
> https://issues.guix.gnu.org/52388.
Hahaha, it's been a month, hasn't it?  There's some aesthetic
unpleasanties, so I'm not sure if I'll upstream it with slight
stylistic changes or give you some harsher feedback, but I'll try to
decide this weekend.

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 3973 bytes --]


On 2022-01-10 21:12, Liliana Marie Prikler wrote:

> Am Montag, dem 10.01.2022 um 12:49 +0300 schrieb Andrew Tropin:
>> [T]he whole point of escape hatch is to make it possible to reuse
>> existing files directly without any manipulation on them and importer
>> should demonstrate how to do it.
> That'd make more sense if the importer copied bashrc to some well-known
> location (e.g. ~/.config/guix/data/.bashrc) and then used that rather
> than the file it wishes to replace.  IIRC that was one suggested
> behaviour of Guix Home in the past, but it didn't get approval because
> users wouldn't typically ask for that copying to happen.  If you make
> it so that plain-file is used normally, but add a switch to express
> things in terms of local-file instead, that'd work.  
>
> OTOH, I do think local-file is already well-documented on its own, so
> perhaps it'd only take a cookbook entry to show it in combination with
> Guix Home and an explanation as to why Guix Home doesn't do that
> normally while explaining all the caveats.

It seems that current implementation of importer copy files to the
directory provided as command line argument and after that generates a
config file, which refers those files with local-file.

Overall, I think that the importer is not really well-defined both in
terms of purpose and implementation and maybe it will be better to hold
it until it polished.

>> If importer internally do some manipulation (escaping) with the
>> content of the file and places the result in the string in scheme
>> file, user won't be able to replicate such process easily for other
>> services, which not covered by the importer. If I understood
>> correctly what you meant in the first message.
> Yes, the user would have to manually quote every new line of code
> they're adding, but they're free to use all other file-like objects,
> including local-file.  Having (bashrc (local-file ".bashrc")), whether
> implemented on top of slurp-file-gexp or not, is inherently dangerous,
> though.
>

From this

>> > The point I'm making is that we shouldn't swap out one bad
>> > abstraction for another, but pave the road towards good
>> > abstractions, e.g. G-expressions in the way the rest of Guix
>> > typically uses them.
>> 
>> Actually, for me, the original implementation looks consistent with
>> how the rest of Guix treats G-expressions (uses already known
>> abstraction) and only new one intoduces a new abstraction.
> That's the point.  The old style works just like you'd expect it to, it
> becomes a problem when you try to feed it stuff like slurp-file-gexp to
> work around some limitations in a way I'm not convinced makes sense.
>

and this, it looks like you are concerned with local-file, which can
lead to the non-reproducible code, when for example the local-file
refers something outside of the git repo and on another machine the same
config will lead to an error?  And slurp-file-gexp additionally hides it
from the user by wrapping local-file.  Correct me if I'm wrong.

>> [I]t's already possible to achieve the same [-- merging multiple
>> bashrc snippets into a single file --] with gexps/file-like in both
>> new and old text-config implementations.
> I find the lack of services in your example concerning, but I'll take
> your word for it.  In that case, using a gexp for bashrc in the typical
> sense is probably the best idea, but we still need to do something with
> the reliance on slurp-file-gexp.
>
> Cheers
>
>> 
> PS:
>> It's offtopic, but when you will have time please take a look at
>> https://issues.guix.gnu.org/52388.
> Hahaha, it's been a month, hasn't it?  There's some aesthetic
> unpleasanties, so I'm not sure if I'll upstream it with slight
> stylistic changes or give you some harsher feedback, but I'll try to
> decide this weekend.
>

Yep :)  Thank you, will be waiting for it!)

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Ludovic Courtès]

Hi,

Andrew Tropin <andrew@trop.in> skribis:

> During the upstreaming process of Guix Home commit
> fee0bced7fec2f9950957976a28f033edd4f877c slipped into master.  It
> introduces a different serialization approach for text-config from what
> was orginally used:
> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services/configuration.scm?h=2c451db39aabc69bdbf186f412ee6e0b4e180ccb#n386
>
> This new approach is inconisistent with the ideas used in the number of
> other home services (not only the ones using text-config):
> https://git.sr.ht/~abcdw/rde/tree/47f6c65c82a4f6761fa1ff5b9405b363cfda6482/item/gnu/home-services
>
> So I had to fork it back to avoid an inconsistency and renamed
> text-config to gexp-text-config to avoid a confusion with upstreamed
> version.
> https://git.sr.ht/~abcdw/rde/tree/47f6c65c82a4f6761fa1ff5b9405b363cfda6482/item/gnu/home-services-utils.scm#L355
>
> Having two serialization approaches stops me from upstreaming the rest
> of home services from rde to Guix and also makes a split to rde home
> services and guix home services, which I would like to avoid.

I’d like to clarify how I think we should work.  Guix development
happens here, on Guix channels, using the project’s peer review
processes.

What rde does certainly is inspirational, and that’s what got us Home in
the first place; in my view, we may sometimes choose to take ideas from
rde in Guix and Guix Home, but rde development alone cannot be used to
justify changes.

> We need to decide, which approach should be used or we will end up with
> two competitive incompatible implementations and unecessary split of
> effort and lose of human force and time.

Making Guix Home part of Guix was and still is a commitment, in
particular the commitment that we’d all be working on one
implementation, that there are no “competitive incompatible
implementations”.  It’s a choice we made, not a phenomenon we’re
passively observing.

[...]

> The new text-config serialization implementation (after fee0bc commit)
> doesn't support gexps and tries to insert the literal content of the
> file in place where file-like object was used, which
>
> 1. Confuses the users.
> People reporting that it's hard to implement something like
>
> source \
> /gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh
>
> with the new approach (using file-like objects), and they switch to the
> original implementation of home services for shells (the ones currently
> living in rde repo), which allows to use gexps.

Are there Guix Home issues reporting this?

> 2. Looks strange implementation-wise.
>
> If we want to insert the file path to file-like object for new-style
> text-config internally we do something like
>
> (mixed-text-file ...
>  "source \" "\n"
>  #~(read-everything-from-file
>     #$(computed-file "unecessary-file-name"
>        #~#$(local-file "blabla.sh"))))

When would one write such a thing?

A couple of examples from Guix System: <cgit-configuration> has an
‘include’ field take accepts file-like objects, <tor-configuration> has
a ‘config-file’ field.

Guix System users probably never have to use complicated constructs like
the one above.  Why would it be different for Home services?


Are there any new arguments since the already lengthy discussions that
led to fee0bced7fec2f9950957976a28f033edd4f877c?  Is it really what’s
leading to Guix Home being stale, or is there something else?

Thanks,
Ludo’.

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 6272 bytes --]

On 2022-01-18 15:26, Ludovic Courtès wrote:

> Hi,
>
> Andrew Tropin <andrew@trop.in> skribis:
>
>> During the upstreaming process of Guix Home commit
>> fee0bced7fec2f9950957976a28f033edd4f877c slipped into master.  It
>> introduces a different serialization approach for text-config from what
>> was orginally used:
>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/services/configuration.scm?h=2c451db39aabc69bdbf186f412ee6e0b4e180ccb#n386
>>
>> This new approach is inconisistent with the ideas used in the number of
>> other home services (not only the ones using text-config):
>> https://git.sr.ht/~abcdw/rde/tree/47f6c65c82a4f6761fa1ff5b9405b363cfda6482/item/gnu/home-services
>>
>> So I had to fork it back to avoid an inconsistency and renamed
>> text-config to gexp-text-config to avoid a confusion with upstreamed
>> version.
>> https://git.sr.ht/~abcdw/rde/tree/47f6c65c82a4f6761fa1ff5b9405b363cfda6482/item/gnu/home-services-utils.scm#L355
>>
>> Having two serialization approaches stops me from upstreaming the rest
>> of home services from rde to Guix and also makes a split to rde home
>> services and guix home services, which I would like to avoid.
>
> I’d like to clarify how I think we should work.  Guix development
> happens here, on Guix channels, using the project’s peer review
> processes.
>
> What rde does certainly is inspirational, and that’s what got us Home in
> the first place; in my view, we may sometimes choose to take ideas from
> rde in Guix and Guix Home, but rde development alone cannot be used to
> justify changes.
>

Hi Ludovic,

It's not a justification for the changes it's a recap of the past events
to provide the context.  We were working on moving Guix Home and home
services from rde to guix, but in the middle of this process the change
to text-config type happend and now it's not possible to continue
upstreaming without rewriting the rest of home services in rde repo or
reverting back this change.

>
>> We need to decide, which approach should be used or we will end up with
>> two competitive incompatible implementations and unecessary split of
>> effort and lose of human force and time.
>
> Making Guix Home part of Guix was and still is a commitment, in
> particular the commitment that we’d all be working on one
> implementation, that there are no “competitive incompatible
> implementations”.  It’s a choice we made, not a phenomenon we’re
> passively observing.

This is exactly what I want to achieve: To be able to collectively work
on one consistent implementation, but fee0bc, which slipped to the
master unreviewed, splitted home service configuration approach into two
competitive implementations, a few essential home services in guix repo
and bigger rest of non-essential stuck in rde.

I already mentioned at least two possible ways to handle this
situtation:
1. Rewrite the rest of rde home services.
2. Rollback this change.

I'm ok with both options, but #1 requires much more human hours to
complete and I still not sure if fee0bc was introduced for strong
reasons or was added almost accidentially.  So I try to find a
justification for this change.

From what I understood from Liliana's and Maxime's replies: I'm not the
only one finding the original implementation to be more intuitive and
consistent with the rest of Guix API.  Please, correct me if I'm wrong.

>> The new text-config serialization implementation (after fee0bc commit)
>> doesn't support gexps and tries to insert the literal content of the
>> file in place where file-like object was used, which
>>
>> 1. Confuses the users.
>> People reporting that it's hard to implement something like
>>
>> source \
>> /gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh
>>
>> with the new approach (using file-like objects), and they switch to the
>> original implementation of home services for shells (the ones currently
>> living in rde repo), which allows to use gexps.
>
> Are there Guix Home issues reporting this?
>

Just a 3 cases I observed in Guix Russia telegram chat.

>> 2. Looks strange implementation-wise.
>>
>> If we want to insert the file path to file-like object for new-style
>> text-config internally we do something like
>>
>> (mixed-text-file ...
>>  "source \" "\n"
>>  #~(read-everything-from-file
>>     #$(computed-file "unecessary-file-name"
>>        #~#$(local-file "blabla.sh"))))
>
> When would one write such a thing?

I have very same question :)

It's what happens internally in current implementation, something like
that goes to home-files-service-type.

>
> A couple of examples from Guix System: <cgit-configuration> has an
> ‘include’ field take accepts file-like objects, <tor-configuration> has
> a ‘config-file’ field.

During the course of this year learned a lot of different service
configurations and approaches used in them and aware of this too.  We
can also mention some services, which has a special field, which accepts
a list of strings or string-valued G-exps, which will be added to the
end of the file like xorg-configuration or httpd-config-file.  Those
approaches have their own pros and cons and I already shared some
thoughts on this topic:

https://issues.guix.gnu.org/50967#65

Also, I already started work on 'Writing Service Configuration' section
in the manual, which is good platform for discussion and should help to
align our visions about service configurations and prevent some
unecessary discussion in the future:

https://issues.guix.gnu.org/52698

>
> Guix System users probably never have to use complicated constructs like
> the one above.  Why would it be different for Home services?

Users won't use this construction directly, it is just what hapenning
under the hood and looks a little too much excessive.

> Are there any new arguments since the already lengthy discussions that
> led to fee0bced7fec2f9950957976a28f033edd4f877c?  Is it really what’s
> leading to Guix Home being stale, or is there something else?

IMO, changes to text-config in fee0bc really makes it harder to continue
the work on many Guix Home related tasks.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2931 bytes --]

Andrew Tropin schreef op do 20-01-2022 om 16:20 [+0300]:
> [...]
> 
> From what I understood from Liliana's and Maxime's replies: I'm not the
> only one finding the original implementation to be more intuitive and
> consistent with the rest of Guix API.  Please, correct me if I'm wrong.

To be clear:

  * >> source \
    >> /gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh

    How about defining a procedure

    (define (source-file file-like)
      (mixed-text-file "source " file-like)),

    the 'concatenated-file' described below, and giving an example or
    two in the manual on how to use it?

    (concatenated-file ""
      (source-file (local-file "some-bash-functions.sh"))
      (mixed-text-file "" (file-append coreutils "/bin/echo")
                          "hello Guix Home!) "\n"
                       "invoke-some-function" "argument")) 

  * I don't like 'slurp-file-gexp' (what are G-exps doing there, and
    what's slurping?).  A better name would improve things though.
    Also, we already have 'mixed-text-file', so maybe we can create
    an ‘concatenated-file'?

    (appended-file name (plain-file "" "foo") (local-file "bar"))
    -->
    foo
    <contents of the file "bar">

    A slight downside is that the plain-file needs to be given a name,
    in this case "", as you have noted for 'mixed-text-file', but that
    can be avoided to a degree by giving it "" as name.

  * IIUC, the reason why 'slurp-file-gexp' or the like was necessary,
    was because the implementation doesn't use records for
    configuration, but rather some mixture of S-exps and ‘copy this
    and that file is the serialisation here and there’.

    I would prefer not using S-exps like

    (home-service barfoo-service-type
      (barfoo-configuration
        (config
          `((this-option "that")
            (foo (bar z)
                 (foobar (include ,(local-file ...)))))))

    and instead write these 'this-option', 'foo', 'bar' and 'foobar'
    in records, such that there's to some degree a type system and
    some discoverability.

    Yes, if there's a lot of options that can be configured,
    then initially Guix won't support all, but it should be easy
    to patch Guix to support more options on an as-needed basis.
    There can also be an 'extra-content' escape hatch.

    For software that doesn't support inclusion directives in
    configuration, we could:

      1. patch upstream to support it (it's free software and
         it's potentially useful outside Guix, so why not?)
      2. or do something like 'concatenated-file'

    with a preference for (1).

As such, I'm not exactly agreeing, since there appear to be better
options than 'slurp-file-gexp'.  Renaming 'slurp-file-gexp' to
something more descriptive would help, but there's more that could be
done.


Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Return back original implementation for text-config serialization

[Ludovic Courtès]

Hi Andrew,

Andrew Tropin <andrew@trop.in> skribis:

>> Making Guix Home part of Guix was and still is a commitment, in
>> particular the commitment that we’d all be working on one
>> implementation, that there are no “competitive incompatible
>> implementations”.  It’s a choice we made, not a phenomenon we’re
>> passively observing.
>
> This is exactly what I want to achieve: To be able to collectively work
> on one consistent implementation, but fee0bc, which slipped to the
> master unreviewed, splitted home service configuration approach into two
> competitive implementations, a few essential home services in guix repo
> and bigger rest of non-essential stuck in rde.

I love that rde is going much further than Guix, but I think it’s in
nobody’s interest to “compete”.

The change in question was discussed at length and reviewed at
<https://issues.guix.gnu.org/50967>.

> I already mentioned at least two possible ways to handle this
> situtation:
> 1. Rewrite the rest of rde home services.
> 2. Rollback this change.
>
> I'm ok with both options, but #1 requires much more human hours to
> complete and I still not sure if fee0bc was introduced for strong
> reasons or was added almost accidentially.  So I try to find a
> justification for this change.

I don’t want to cause troubles in rde for you and its users, but I also
don’t want Home decisions to be discussed there.

>> Are there Guix Home issues reporting this?
>>
>
> Just a 3 cases I observed in Guix Russia telegram chat.

OK.  I don’t see anything at <https://issues.guix.gnu.org> though, which
is where I’d expect bug reports for Guix Home.

[...]

>> Are there any new arguments since the already lengthy discussions that
>> led to fee0bced7fec2f9950957976a28f033edd4f877c?  Is it really what’s
>> leading to Guix Home being stale, or is there something else?
>
> IMO, changes to text-config in fee0bc really makes it harder to continue
> the work on many Guix Home related tasks.

It feels exaggerated to me, but that’s perhaps because I’m overlooking
important aspects.

I’d like us to move forward on this.  I think the best course of action
is to focus on concrete things rather than abstract design discussions.

Can we move, one by one, a few more services from rde to Home?

Earlier I mentioned the SSH client service, but there are more.  When we
move them, let’s see if problems arise related to this pattern or to
other changes made in Guix Home.  At that point perhaps it’ll be clearer
for everyone (or at least for me) what concrete problems this poses and
how we could address it.

How does that sound?

In the meantime I submitted a patch for my first Home service this
week-end.  :-)

Thanks,
Ludo’.

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 8443 bytes --]

On 2022-01-21 10:30, Maxime Devos wrote:

> Andrew Tropin schreef op do 20-01-2022 om 16:20 [+0300]:
>> [...]
>> 
>> From what I understood from Liliana's and Maxime's replies: I'm not the
>> only one finding the original implementation to be more intuitive and
>> consistent with the rest of Guix API.  Please, correct me if I'm wrong.
>
> To be clear:
>
>   * >> source \
>     >> /gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh
>
>     How about defining a procedure
>
>     (define (source-file file-like)
>       (mixed-text-file "source " file-like)),

Possible, but not really necessary I think it will be ok to just use:

(mixed-text-file "" "source " file-like)

It looks like a light missuse of file like objects because we have to
specify "" file name each time, nothing critical, but still feels a
little wierd.

>
> the 'concatenated-file' described below, and giving an example or
>     two in the manual on how to use it?
>
>     (concatenated-file ""
>       (source-file (local-file "some-bash-functions.sh"))
>       (mixed-text-file "" (file-append coreutils "/bin/echo")
>                           "hello Guix Home!) "\n"
>                        "invoke-some-function" "argument")) 

concatenated-file is not needed, we already can use source-file and
mixed-text-file directly in bash-home-configuration:
(bashrc
 (list
  (source-file (local-file "some-bash-functions.sh"))
  (mixed-text-file "" (file-append coreutils "/bin/echo")
                   " hello Guix Home!) \n"
                   "invoke-some-function" "argument"))
                   
>
> * I don't like 'slurp-file-gexp' (what are G-exps doing there, and
>     what's slurping?).  A better name would improve things though.

The G-expression, which slurps the file.  It's just a funny name I
borrowed from Clojure:
https://clojuredocs.org/clojure.core/slurp
It was just WIP name, I don't mind naming it differently.

>     Also, we already have 'mixed-text-file', so maybe we can create
>     an ‘concatenated-file'?

Yes, I was missing it a few times, when I started to use guix services a
year ago.

Current implementation of text-config do a similar thing, but it would
be cool to have a separate helper function independent from guix
services.

>
> (appended-file name (plain-file "" "foo") (local-file "bar"))
>     -->
>     foo
>     <contents of the file "bar">
>
>     A slight downside is that the plain-file needs to be given a name,
>     in this case "", as you have noted for 'mixed-text-file', but that
>     can be avoided to a degree by giving it "" as name.

Not a big deal I think.  Optionally, we can support strings, so it will
work as a mixed-text-file, but will be inserting the content of the file
instead of the path, however it can be a little confusing.

>
> * IIUC, the reason why 'slurp-file-gexp' or the like was necessary,
>     was because the implementation doesn't use records for
>     configuration, but rather some mixture of S-exps and ‘copy this
>     and that file is the serialisation here and there’.
>
>     I would prefer not using S-exps like
>
>     (home-service barfoo-service-type
>       (barfoo-configuration
>         (config
>           `((this-option "that")
>             (foo (bar z)
>                  (foobar (include ,(local-file ...)))))))
>
>     and instead write these 'this-option', 'foo', 'bar' and 'foobar'
>     in records, such that there's to some degree a type system and
>     some discoverability.

Very good you rised this question.  Discoverability is true, with
records it's a little easier to get tips from geiser, however, the
safety provided by this weak type system is almost illusory, also, the
same degree of type correctness can be achieved without records.

IIRC, I covered this tradeoff in Writing Service Configuration manual
section: https://issues.guix.gnu.org/52698

>
> Yes, if there's a lot of options that can be configured,
>     then initially Guix won't support all, but it should be easy
>     to patch Guix to support more options on an as-needed basis.
>     There can also be an 'extra-content' escape hatch.
>
>     For software that doesn't support inclusion directives in
>     configuration, we could:
>
>       1. patch upstream to support it (it's free software and
>          it's potentially useful outside Guix, so why not?)
>       2. or do something like 'concatenated-file'
>
>     with a preference for (1).
>
> As such, I'm not exactly agreeing, since there appear to be better
> options than 'slurp-file-gexp'.  Renaming 'slurp-file-gexp' to
> something more descriptive would help, but there's more that could be
> done.

Having extra-content basically says that we give up on implementing a
proper serialization and user have to use a mix of target configuration
format placed inside a string, which must be escaped of course +
lisp-flavored version of the same configuration.

I have an excerpt of very simple real-world nginx configuration, which
still demonstrate the idea:

--8<---------------cut here---------------start------------->8---
                      (nginx-configuration
                       (modules
                        (list
                         (file-append nginx-rtmp-module "\
/etc/nginx/modules/ngx_rtmp_module.so")))
                       (extra-content
                        (format #f "\
}
rtmp {
        server {
                listen 1935;
                chunk_size 4096;

                application live {
                        live on;
                        record off;
                        push rtmp://a.rtmp.youtube.com/live2/~a;
                        push rtmp://diode.zone:1935/live/~a;
                }
        }
" youtube-key peertube-key)) ;; WARNING: secrets goes to the store.

                       (server-blocks
                        (list (nginx-server-configuration
                               (server-name `(,ip))
                               (listen '("8088"))
                               (root "/var/www/"))))))
--8<---------------cut here---------------end--------------->8---

In addition to the problems I mentioned above:

1. Mixed usage of two configuration languages (nginx-conf and lisp).
2. Having a string, which should be properly escaped (luckily for this
example it's not a problem).

we also:

3. Have to implement our own templating engine (using format function in
this case) to share the values from guile with the config.
4.1. Don't know where extra-content goes. (It goes to http section not the
end of the file, so we have to start with "}" to get a correct
configuration).
4.2. Don't control where it must be placed. (Can be important in other
use cases, which I can share if needed).
5. Have inconsistent implementation of extra-config, extra-content, raw-lines
and other escape hatches (We need to learn it everytime we write a new
service configuration).

but it could be:

--8<---------------cut here---------------start------------->8---
(nginx-configuration
 (config
  `((load_module ,(file-append nginx-rtmp-module "\
/etc/nginx/modules/ngx_rtmp_module.so"))
    (http
     ((server
       ((listen 8088)
        (server_name ,ip)
        (root "/var/www")
        (index "index.html")))))
    (rtmp
     ((server
       ((listen 1935)
        (chunk_size 4096)
        (application live
         ((push ,(string-append "rtmp://a.rtmp.youtube.com/live2/" youtube-key))
          (push ,(string-append "rtmp://diode.zone:1935/live/" peertube-key))
          (live on)
          (record off))))))))))
--8<---------------cut here---------------end--------------->8---

Ludovic mentioned someday that nginx-configuration is problematic, but I
highlighted the generic problems, which are applicable for many other
guix service configurations.

I discussed some other pros and cons of record-based configurations in
https://issues.guix.gnu.org/52698

and I see the benifits of the records, but I'm not sure if they
outweight the weaknesses.

It maybe sound unrelated to this thread, but it's actually very ontopic,
because it lead to the design and implementation of home services
configuration approach in general and slurp-file-gexp and text-config in
particular.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 8443 bytes --]

On 2022-01-21 10:30, Maxime Devos wrote:

> Andrew Tropin schreef op do 20-01-2022 om 16:20 [+0300]:
>> [...]
>> 
>> From what I understood from Liliana's and Maxime's replies: I'm not the
>> only one finding the original implementation to be more intuitive and
>> consistent with the rest of Guix API.  Please, correct me if I'm wrong.
>
> To be clear:
>
>   * >> source \
>     >> /gnu/store/00fl96dj2aak4i1vqvdqzlhbmmskc7fx-blabla.sh
>
>     How about defining a procedure
>
>     (define (source-file file-like)
>       (mixed-text-file "source " file-like)),

Possible, but not really necessary I think it will be ok to just use:

(mixed-text-file "" "source " file-like)

It looks like a light missuse of file like objects because we have to
specify "" file name each time, nothing critical, but still feels a
little wierd.

>
> the 'concatenated-file' described below, and giving an example or
>     two in the manual on how to use it?
>
>     (concatenated-file ""
>       (source-file (local-file "some-bash-functions.sh"))
>       (mixed-text-file "" (file-append coreutils "/bin/echo")
>                           "hello Guix Home!) "\n"
>                        "invoke-some-function" "argument")) 

concatenated-file is not needed, we already can use source-file and
mixed-text-file directly in bash-home-configuration:
(bashrc
 (list
  (source-file (local-file "some-bash-functions.sh"))
  (mixed-text-file "" (file-append coreutils "/bin/echo")
                   " hello Guix Home!) \n"
                   "invoke-some-function" "argument"))
                   
>
> * I don't like 'slurp-file-gexp' (what are G-exps doing there, and
>     what's slurping?).  A better name would improve things though.

The G-expression, which slurps the file.  It's just a funny name I
borrowed from Clojure:
https://clojuredocs.org/clojure.core/slurp
It was just WIP name, I don't mind naming it differently.

>     Also, we already have 'mixed-text-file', so maybe we can create
>     an ‘concatenated-file'?

Yes, I was missing it a few times, when I started to use guix services a
year ago.

Current implementation of text-config do a similar thing, but it would
be cool to have a separate helper function independent from guix
services.

>
> (appended-file name (plain-file "" "foo") (local-file "bar"))
>     -->
>     foo
>     <contents of the file "bar">
>
>     A slight downside is that the plain-file needs to be given a name,
>     in this case "", as you have noted for 'mixed-text-file', but that
>     can be avoided to a degree by giving it "" as name.

Not a big deal I think.  Optionally, we can support strings, so it will
work as a mixed-text-file, but will be inserting the content of the file
instead of the path, however it can be a little confusing.

>
> * IIUC, the reason why 'slurp-file-gexp' or the like was necessary,
>     was because the implementation doesn't use records for
>     configuration, but rather some mixture of S-exps and ‘copy this
>     and that file is the serialisation here and there’.
>
>     I would prefer not using S-exps like
>
>     (home-service barfoo-service-type
>       (barfoo-configuration
>         (config
>           `((this-option "that")
>             (foo (bar z)
>                  (foobar (include ,(local-file ...)))))))
>
>     and instead write these 'this-option', 'foo', 'bar' and 'foobar'
>     in records, such that there's to some degree a type system and
>     some discoverability.

Very good you rised this question.  Discoverability is true, with
records it's a little easier to get tips from geiser, however, the
safety provided by this weak type system is almost illusory, also, the
same degree of type correctness can be achieved without records.

IIRC, I covered this tradeoff in Writing Service Configuration manual
section: https://issues.guix.gnu.org/52698

>
> Yes, if there's a lot of options that can be configured,
>     then initially Guix won't support all, but it should be easy
>     to patch Guix to support more options on an as-needed basis.
>     There can also be an 'extra-content' escape hatch.
>
>     For software that doesn't support inclusion directives in
>     configuration, we could:
>
>       1. patch upstream to support it (it's free software and
>          it's potentially useful outside Guix, so why not?)
>       2. or do something like 'concatenated-file'
>
>     with a preference for (1).
>
> As such, I'm not exactly agreeing, since there appear to be better
> options than 'slurp-file-gexp'.  Renaming 'slurp-file-gexp' to
> something more descriptive would help, but there's more that could be
> done.

Having extra-content basically says that we give up on implementing a
proper serialization and user have to use a mix of target configuration
format placed inside a string, which must be escaped of course +
lisp-flavored version of the same configuration.

I have an excerpt of very simple real-world nginx configuration, which
still demonstrate the idea:

--8<---------------cut here---------------start------------->8---
                      (nginx-configuration
                       (modules
                        (list
                         (file-append nginx-rtmp-module "\
/etc/nginx/modules/ngx_rtmp_module.so")))
                       (extra-content
                        (format #f "\
}
rtmp {
        server {
                listen 1935;
                chunk_size 4096;

                application live {
                        live on;
                        record off;
                        push rtmp://a.rtmp.youtube.com/live2/~a;
                        push rtmp://diode.zone:1935/live/~a;
                }
        }
" youtube-key peertube-key)) ;; WARNING: secrets goes to the store.

                       (server-blocks
                        (list (nginx-server-configuration
                               (server-name `(,ip))
                               (listen '("8088"))
                               (root "/var/www/"))))))
--8<---------------cut here---------------end--------------->8---

In addition to the problems I mentioned above:

1. Mixed usage of two configuration languages (nginx-conf and lisp).
2. Having a string, which should be properly escaped (luckily for this
example it's not a problem).

we also:

3. Have to implement our own templating engine (using format function in
this case) to share the values from guile with the config.
4.1. Don't know where extra-content goes. (It goes to http section not the
end of the file, so we have to start with "}" to get a correct
configuration).
4.2. Don't control where it must be placed. (Can be important in other
use cases, which I can share if needed).
5. Have inconsistent implementation of extra-config, extra-content, raw-lines
and other escape hatches (We need to learn it everytime we write a new
service configuration).

but it could be:

--8<---------------cut here---------------start------------->8---
(nginx-configuration
 (config
  `((load_module ,(file-append nginx-rtmp-module "\
/etc/nginx/modules/ngx_rtmp_module.so"))
    (http
     ((server
       ((listen 8088)
        (server_name ,ip)
        (root "/var/www")
        (index "index.html")))))
    (rtmp
     ((server
       ((listen 1935)
        (chunk_size 4096)
        (application live
         ((push ,(string-append "rtmp://a.rtmp.youtube.com/live2/" youtube-key))
          (push ,(string-append "rtmp://diode.zone:1935/live/" peertube-key))
          (live on)
          (record off))))))))))
--8<---------------cut here---------------end--------------->8---

Ludovic mentioned someday that nginx-configuration is problematic, but I
highlighted the generic problems, which are applicable for many other
guix service configurations.

I discussed some other pros and cons of record-based configurations in
https://issues.guix.gnu.org/52698

and I see the benifits of the records, but I'm not sure if they
outweight the weaknesses.

It maybe sound unrelated to this thread, but it's actually very ontopic,
because it lead to the design and implementation of home services
configuration approach in general and slurp-file-gexp and text-config in
particular.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 4606 bytes --]

On 2022-01-24 16:37, Ludovic Courtès wrote:

> Hi Andrew,
>
> Andrew Tropin <andrew@trop.in> skribis:
>
>>> Making Guix Home part of Guix was and still is a commitment, in
>>> particular the commitment that we’d all be working on one
>>> implementation, that there are no “competitive incompatible
>>> implementations”.  It’s a choice we made, not a phenomenon we’re
>>> passively observing.
>>
>> This is exactly what I want to achieve: To be able to collectively work
>> on one consistent implementation, but fee0bc, which slipped to the
>> master unreviewed, splitted home service configuration approach into two
>> competitive implementations, a few essential home services in guix repo
>> and bigger rest of non-essential stuck in rde.
>
> I love that rde is going much further than Guix, but I think it’s in
> nobody’s interest to “compete”.
>
> The change in question was discussed at length and reviewed at
> <https://issues.guix.gnu.org/50967>.
>

The intent of the patch series was to rename modules and the change
about text-config was added somewhere in between.  I asked to move it
out to the separate thread and do a separate review on that, but seems
the message was missed.

> I think this patch requires more discussion and better to keep it
> outside of this patch series.  Skimmed throught other patches, overall
> LGTM.

The proper review of this big patch series with a few unrelated changes
is hard and inefficient.  We can see that here:
https://issues.guix.gnu.org/50967#66

The semantically-incorrect change was applied, not mentioning that the
discussion about this whole patch #13 wasn't finished in my opinion.

I'll be more clear next time and will state the intent more precisely to
prevent such situations in the future.


Sorry for rising the same thread again and again, but it's really
improtant in my opinion and I would like to complete this discussion.

Seems Maxime rised good questions and proposed nice ideas and discussion
is going forward.

>> I already mentioned at least two possible ways to handle this
>> situtation:
>> 1. Rewrite the rest of rde home services.
>> 2. Rollback this change.
>>
>> I'm ok with both options, but #1 requires much more human hours to
>> complete and I still not sure if fee0bc was introduced for strong
>> reasons or was added almost accidentially.  So I try to find a
>> justification for this change.
>
> I don’t want to cause troubles in rde for you and its users, but I also
> don’t want Home decisions to be discussed there.
>
>>> Are there Guix Home issues reporting this?
>>>
>>
>> Just a 3 cases I observed in Guix Russia telegram chat.
>
> OK.  I don’t see anything at <https://issues.guix.gnu.org> though, which
> is where I’d expect bug reports for Guix Home.
>

Of course, I try to redirect people to guix mailing lists, but despite
this fact the discussions and question happens in other places too.

>>> Are there any new arguments since the already lengthy discussions that
>>> led to fee0bced7fec2f9950957976a28f033edd4f877c?  Is it really what’s
>>> leading to Guix Home being stale, or is there something else?
>>
>> IMO, changes to text-config in fee0bc really makes it harder to continue
>> the work on many Guix Home related tasks.
>
> It feels exaggerated to me, but that’s perhaps because I’m overlooking
> important aspects.
>
> I’d like us to move forward on this.  I think the best course of action
> is to focus on concrete things rather than abstract design discussions.
>
> Can we move, one by one, a few more services from rde to Home?
>
> Earlier I mentioned the SSH client service, but there are more.  When we
> move them, let’s see if problems arise related to this pattern or to
> other changes made in Guix Home.  At that point perhaps it’ll be clearer
> for everyone (or at least for me) what concrete problems this poses and
> how we could address it.
>
> How does that sound?
>

Ok, let's try, but please don't hurry, a few first services are
important, because they set the style and I would really want it to be
consistent and well-designed.

> In the meantime I submitted a patch for my first Home service this
> week-end.  :-)

Saw one of your week-end patches, will find the rest, test them and
share my thoughts in a few days.

I have some attention problems, so I don't follow all the patches on
guix-patches, please CC me for Guix Home related changes if it's
possible and not very burdening.  

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 408 bytes --]


> I have some attention problems, so I don't follow all the patches on
> guix-patches, please CC me for Guix Home related changes if it's
> possible and not very burdening.  

I'll make a special search query, which shows all the messages with
"home:" in subject line, so I should be fine on following Guix Home
related threads, still don't hesitate to CC me =)

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Maxim Cournoyer]

Hi Andrew,

Andrew Tropin <andrew@trop.in> writes:

[...]

> Ludovic mentioned someday that nginx-configuration is problematic, but I
> highlighted the generic problems, which are applicable for many other
> guix service configurations.
>
> I discussed some other pros and cons of record-based configurations in
> https://issues.guix.gnu.org/52698
>
> and I see the benifits of the records, but I'm not sure if they
> outweight the weaknesses.
>
> It maybe sound unrelated to this thread, but it's actually very ontopic,
> because it lead to the design and implementation of home services
> configuration approach in general and slurp-file-gexp and text-config in
> particular.

Pardon my ignorance about Guix Home, but couldn't we have configuration
as records by default, and the lower level, config stitching primitives
still available to hand craft strings that could be used as the value of
an 'extra-config' field, for example?

It seems to me the use of records for configurations in Guix Home would
be the natural choice, as Guix System users are already familiar with
them and it leans to better discoverability/documentation.

Thanks,

Maxim

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 2077 bytes --]

On 2022-01-27 00:04, Maxim Cournoyer wrote:

> Hi Andrew,
>
> Andrew Tropin <andrew@trop.in> writes:
>
> [...]
>
>> Ludovic mentioned someday that nginx-configuration is problematic, but I
>> highlighted the generic problems, which are applicable for many other
>> guix service configurations.
>>
>> I discussed some other pros and cons of record-based configurations in
>> https://issues.guix.gnu.org/52698
>>
>> and I see the benifits of the records, but I'm not sure if they
>> outweight the weaknesses.
>>
>> It maybe sound unrelated to this thread, but it's actually very ontopic,
>> because it lead to the design and implementation of home services
>> configuration approach in general and slurp-file-gexp and text-config in
>> particular.
>
> Pardon my ignorance about Guix Home, but couldn't we have configuration
> as records by default, and the lower level, config stitching primitives
> still available to hand craft strings that could be used as the value of
> an 'extra-config' field, for example?

extra-config/extra-content/file doesn't play well with the rest of
configuration record.

See the proposed patch, which introduces home service for redshift:
https://issues.guix.gnu.org/53466

I briefly mentioned it, but can elaborate if necessary.

To be clear the top-level SOMETHING-configuration still being the
record, fields, which introduces some complex changes like
shepherd-service? or something-else-quite-big still fields of this
record, but what get serialized to config file will be mostly a plain
datastructure in the approach I propose and recommend.

> It seems to me the use of records for configurations in Guix Home would
> be the natural choice, as Guix System users are already familiar with
> them and it leans to better discoverability/documentation.

The point about Guix System users is valid, however I don't see it as a
big paradigm shift or very different approach.

I made a few points about discoverability/documentation in issue about
redshift.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Ludovic Courtès]

Hi!

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Pardon my ignorance about Guix Home, but couldn't we have configuration
> as records by default, and the lower level, config stitching primitives
> still available to hand craft strings that could be used as the value of
> an 'extra-config' field, for example?
>
> It seems to me the use of records for configurations in Guix Home would
> be the natural choice, as Guix System users are already familiar with
> them and it leans to better discoverability/documentation.

Guix Home uses records for configuration and the same service as Guix
System, and I think it’s important that it remain this way for the
reasons you give and also from a maintenance viewpoint.

Ludo’.

Re: Return back original implementation for text-config serialization

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2508 bytes --]

Andrew Tropin schreef op wo 26-01-2022 om 11:36 [+0300]:
> In addition to the problems I mentioned above:
> 
> 1. Mixed usage of two configuration languages (nginx-conf and lisp).
> 2. Having a string, which should be properly escaped (luckily for
> this
> example it's not a problem).

Mixing two configuration languages can be avoided by supporting
everything with records.

> 
> we also:
> 
> 3. Have to implement our own templating engine (using format function
> in this case) to share the values from guile with the config.

This seems to be the same for this list based configuration system
and record based configuraiton system; for the nginx example you gave,
all these lists with parentheses need to turned into something with
brackets that nginx understands anyway.

> 4.1. Don't know where extra-content goes. (It goes to http section
> not the
> end of the file, so we have to start with "}" to get a correct
> configuration).

Can be solved by adding missing options to the Guix service definition
(and documentation) when the need arises.

> 4.2. Don't control where it must be placed. (Can be important in
> other
> use cases, which I can share if needed).

Likewise.

> 5. Have inconsistent implementation of extra-config, extra-content,
> raw-lines
> and other escape hatches (We need to learn it everytime we write a
> new
> service configuration).

Likewise.

Also, the mapping of upstream configuration files to lists in Guix
seems far from obvious to me: in https://issues.guix.gnu.org/52698,
how am I supposed to know that 'us,ru' must be a symbol, why isn't
it a string?  If one of the strings for some property includes a
special character from the configuration language (say, '$'),
should it be escaped in Guix ((bindsym ... "[class=\"$100 dollars\"]"
...) or (bindsym ... "[class=\"\\$100 dollars\""]""))?

Why (bindsym ... "[class=\"foo\"]") instead of
(bindsym ... (= class "foo"))?

Why (bindsym ... exec emacsclient ...) and not
(bindsym ... exec (file-append emacs "/bin/emacsclient) ...)?
How am I supposed to know whether emacs is in the path or not,
and if it is, is this merely an implementation detail?

How would  I know if it's (bindsym ... exec emacsclient -c --eval
"'(eshell)'") or (bindsym ... "exec emacsclient -c --eval
\"'(eshell)'\"")?  Since the idea is to keep as close to the
configuration language as possible, shouldn't it be the second?

Why lists and not vectors?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Return back original implementation for text-config serialization

[Xinglu Chen]

[-- Attachment #1: Type: text/plain, Size: 834 bytes --]

Hi,

Andrew schrieb am Donnerstag der 20. Januar 2022 um 16:20 +03:

>>> 2. Looks strange implementation-wise.
>>>
>>> If we want to insert the file path to file-like object for new-style
>>> text-config internally we do something like
>>>
>>> (mixed-text-file ...
>>>  "source \" "\n"
>>>  #~(read-everything-from-file
>>>     #$(computed-file "unecessary-file-name"
>>>        #~#$(local-file "blabla.sh"))))
>>
>> When would one write such a thing?
>
> I have very same question :)
>
> It's what happens internally in current implementation, something like
> that goes to home-files-service-type.

From what I can tell, ‘home-files-service-type’ doesn’t seem to have
anything like that.  I am not able to find any usages for
‘computed-file’ in Guix Home services either.  I am missing something?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 6383 bytes --]

On 2022-02-05 12:34, Maxime Devos wrote:

> Andrew Tropin schreef op wo 26-01-2022 om 11:36 [+0300]:
>> In addition to the problems I mentioned above:
>> 
>> 1. Mixed usage of two configuration languages (nginx-conf and lisp).
>> 2. Having a string, which should be properly escaped (luckily for
>> this
>> example it's not a problem).
>
> Mixing two configuration languages can be avoided by supporting
> everything with records.
>

I suppose it will lead to a huge maintenance burden, but it's just a
guess.

>> 
>> we also:
>> 
>> 3. Have to implement our own templating engine (using format function
>> in this case) to share the values from guile with the config.
>
> This seems to be the same for this list based configuration system
> and record based configuraiton system; for the nginx example you gave,
> all these lists with parentheses need to turned into something with
> brackets that nginx understands anyway.
>
>> 4.1. Don't know where extra-content goes. (It goes to http section
>> not the
>> end of the file, so we have to start with "}" to get a correct
>> configuration).
>
> Can be solved by adding missing options to the Guix service definition
> (and documentation) when the need arises.
>
>> 4.2. Don't control where it must be placed. (Can be important in
>> other
>> use cases, which I can share if needed).
>
> Likewise.
>
>> 5. Have inconsistent implementation of extra-config, extra-content,
>> raw-lines
>> and other escape hatches (We need to learn it everytime we write a
>> new
>> service configuration).
>
> Likewise.
>
> Also, the mapping of upstream configuration files to lists in Guix
> seems far from obvious to me: in https://issues.guix.gnu.org/52698,
> how am I supposed to know that 'us,ru' must be a symbol, why isn't
> it a string?

It's quite obvious.  ((layout us,ru)) will be translated to
`layout us,ru` and this is what expected by man 5 sway-input.

Strings and their purpose are covered below.

> If one of the strings for some property includes a special character
> from the configuration language (say, '$'), should it be escaped in
> Guix ((bindsym ... "[class=\"$100 dollars\"]" ...) or (bindsym
> ... "[class=\"\\$100 dollars\""]""))?

Not sure about this question.  If this character have to be escaped in
the target configuration, "[class=\"\\$100 dollars\"]" will produce what
you need: [class="\$100 dollars"].

According to string serialization: In first iteration I made a soft
escape hatch (all strings are serialized to its values), it made it
possible to express this CRITERIA (man 5 sway) statement you menshioned
above.

I added a proper gexp support a little later, but the example with
string already was in use.  Currently it should be done this way:

`((bindsym ... ,#~"[class=\"$100 dollars\"]" ...))

And probably strings must be serialized to quoted values now, if I
make home-sway v2 it will be done this way.

I didn't make a CRITERIA to be a part of a grammar because:

1. I needed a working prototype fast to move forward on Guix Home
itself.
2. I encountered a bug in guile compiler and already spend a lot of time
on home-sway service, after I finally localized it and it was fixed by
Andy.  I decided to postpone further improvements of sway service for
later times.

> Why (bindsym ... "[class=\"foo\"]") instead of
> (bindsym ... (= class "foo"))?

This one is good.  As you see I didn't made a CRITERIA a part of the
grammar, so there is no proper way to express it without escape hatch,
however home-sway v2 can be done slightly different, more on that in the
reply to lists and vectors question.

>
> Why (bindsym ... exec emacsclient ...) and not
> (bindsym ... exec (file-append emacs "/bin/emacsclient) ...)?
> How am I supposed to know whether emacs is in the path or not,
> and if it is, is this merely an implementation detail?

In most cases it should be
`((bindsym ... exec ,(file-append emacs "/bin/emacsclient") ...))

You are right.

>
> How would  I know if it's (bindsym ... exec emacsclient -c --eval
> "'(eshell)'") or (bindsym ... "exec emacsclient -c --eval
> \"'(eshell)'\"")?  Since the idea is to keep as close to the
> configuration language as possible, shouldn't it be the second?

exec is a part of configuration grammar and it should not be quoted.
Command itself doesn't have to be quoted too, but probably can be if you
want:

`((bindsym ... exec ,#~"'emacsclient -c --eval \"\\'(eshell)\\'\"'"))

>
> Why lists and not vectors?

This one is good as well, back in the day I was implmenting home-sway
service I didn't have much experience with guile vectors.  I tried not
to be any fancy and used lists for both sequential and associative data
structures.

Vectors can be a good match, also it will be easier to make a CRITERIA
to be a part of a grammar and be used without escape hatch.  In the
combination of proper string serialization it will look like:

--8<---------------cut here---------------start------------->8---
`#(#(bindsym $mod+o ((class . "foo")
                     (tiling)
                     (window_type . toolbar))
             kill)
   #(bindsym $mod+e exec ,(file-append emacs "/bin/emacsclient"))
   ,#~"# This is comment\n# Layout related settings:"
   #(input
     #(#(xkb_layout us,ru)
       #(xkb_variant dvp,))))
--8<---------------cut here---------------end--------------->8---


and the resulting config will be:

--8<---------------cut here---------------start------------->8---

bindsym $mod+o [class="foo" tiling window_type=toolbar] kill
bindsym $mod+e exec /gnu/store/2808l07ld4hzlmlslvbqjlqrprw7f1xz-emacs/bin/emacsclient
# This is comment
# Layout related settings:
input * {
    xkb_layout us,ru
    xkb_variant dvp,
}
--8<---------------cut here---------------end--------------->8---

Sharps are a little bit noisy, but such config is a little more complete
than original one, probably easier to parse, type check and serialize.
Also, I used vectors recently for serialization to a few different type
of configuration formats and quite satisfied with them.


Thank you very much for a fresh look, the thoughtful questions and
ideas!  Only the knowledge I got thanks to this discussion is worth
starting this thread :)

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]

Re: Return back original implementation for text-config serialization

[Andrew Tropin]

[-- Attachment #1: Type: text/plain, Size: 981 bytes --]

On 2022-02-05 12:09, Ludovic Courtès wrote:

> Hi!
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Pardon my ignorance about Guix Home, but couldn't we have configuration
>> as records by default, and the lower level, config stitching primitives
>> still available to hand craft strings that could be used as the value of
>> an 'extra-config' field, for example?
>>
>> It seems to me the use of records for configurations in Guix Home would
>> be the natural choice, as Guix System users are already familiar with
>> them and it leans to better discoverability/documentation.
>
> Guix Home uses records for configuration and the same service as Guix
> System, and I think it’s important that it remain this way for the
> reasons you give and also from a maintenance viewpoint.
>
> Ludo’.

I think I agree on that, probably home services and system services in
Guix proper should follow the same style.

-- 
Best regards,
Andrew Tropin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 853 bytes --]