“Guix Profiles in Practice”

“Guix Profiles in Practice”

[Ludovic Courtès]

Hello Guix!

Here’s a new blog post by Pierre Neidhardt entitled “Guix Profiles in
Practice”:

  https://guix.gnu.org/blog/2019/guix-profiles-in-practice/

A great discussion on the benefits of manifests and on ways to manage
several profiles!

Ludo’.

Re: “Guix Profiles in Practice”

[Thompson, David]

On Fri, Oct 25, 2019 at 6:57 AM Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hello Guix!
>
> Here’s a new blog post by Pierre Neidhardt entitled “Guix Profiles in
> Practice”:
>
>   https://guix.gnu.org/blog/2019/guix-profiles-in-practice/
>
> A great discussion on the benefits of manifests and on ways to manage
> several profiles!

Great article!

I spotted a typo in the first paragraph: "on a same system" should be
"on the same system."

The section below reminds me about some unfinished work from back when
I wrote 'guix environment':

    Eventually, we set to work on that project again, so we run guix
    environment -m manifest.scm. But now we have to wait for Guix to
    build and install stuff!

This is indeed a frustration.  It takes a bunch of extra switches to
have 'guix environment' create a profile and even I don't bother with
it, just opting to build a new profile if I've upgraded guix since the
last time I worked on the project.  This gets me into trouble when I
upgrade guix and decide that I want to work on the project while I'm
somewhere with no network connection or too slow of a connection to
download everything in a reasonable time.  It has happened to me
multiple times while commuting by train.

'guix environment' is a very composable command, and it's not clear to
me how I would add the stuff I want onto it and it's even less clear
if it's a good idea.  Probably not.  So, I wonder if maybe a new
subcommand, say 'guix develop', could address this common development
use-case while allowing 'guix environment' to continue being the swiss
army knife that it is.  Some simple naming conventions could make this
tool "just work."  Running 'guix develop' would check for a profile
symlink with some canonical name, for example '.guix-develop-profile'.
If it exists, it applies the environment variables and spawns a
subshell.  If it doesn't exist, it looks for a 'develop.scm' file
(canonical name TBD), builds the profile, symlinks it to
'.guix-develop-profile', and then does the prior steps.  The tool
would provide a mechanism to update, etc. and it could even be
expanded later to spawn services like databases.

- Dave

Re: “Guix Profiles in Practice”

[Pjotr Prins]

On Fri, Oct 25, 2019 at 12:36:01PM -0400, Thompson, David wrote:
> On Fri, Oct 25, 2019 at 6:57 AM Ludovic Courtès <ludo@gnu.org> wrote:
> >
> > Hello Guix!
> >
> > Here’s a new blog post by Pierre Neidhardt entitled “Guix Profiles in
> > Practice”:
> >
> >   https://guix.gnu.org/blog/2019/guix-profiles-in-practice/
> >
> > A great discussion on the benefits of manifests and on ways to manage
> > several profiles!
> 
> Great article!

I agree. One useful thing not mentioned in there is, that before
loading a profile, it may be worthwhile to empty the shell with

  env -i /bin/bash --login --noprofile --norc

  . ~/opt/my-tools-profile/etc/profile

so no existing shell variables may pollute your beautiful setup. 

Pj

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 2166 bytes --]

"Thompson, David" <dthompson2@worcester.edu> writes:

> Great article!

Thanks! :)

> I spotted a typo in the first paragraph: "on a same system" should be
> "on the same system."

I can fix it.  Out of curiosity, why is it wrong?  "All users on a
system..."  is correct, "All users on a given system" too, so I wonder
why "on a same system" would not be.  What I meant here is "the same
unspecified system".

> 'guix environment' is a very composable command, and it's not clear to
> me how I would add the stuff I want onto it and it's even less clear
> if it's a good idea.  Probably not.  So, I wonder if maybe a new
> subcommand, say 'guix develop', could address this common development
> use-case while allowing 'guix environment' to continue being the swiss
> army knife that it is.  Some simple naming conventions could make this
> tool "just work."  Running 'guix develop' would check for a profile
> symlink with some canonical name, for example '.guix-develop-profile'.
> If it exists, it applies the environment variables and spawns a
> subshell.  If it doesn't exist, it looks for a 'develop.scm' file
> (canonical name TBD), builds the profile, symlinks it to
> '.guix-develop-profile', and then does the prior steps.  The tool
> would provide a mechanism to update, etc. and it could even be
> expanded later to spawn services like databases.

Indeed, while I worked on the article I was frustrated with the CLI, so
I wrote this Emacs "hack":

https://gitlab.com/emacs-guix/emacs-guix/issues/13

This is essentially what you propose, but to have it at the CLI level
would provide several benefits:

- More universal.
- Possibly more optimized (e.g. caching).



At the end of the article, I also talk about reproducibility, which is
currently quite cumbersome to deploy because of the need to refer to a
"specific guix channel checkout".

Konrad talked about how we could possibly improve the CLI to deal with
that as well:

https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00511.html

Both issues could be addressed the same way.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1275 bytes --]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

> On Fri, Oct 25, 2019 at 12:36:01PM -0400, Thompson, David wrote:
>> On Fri, Oct 25, 2019 at 6:57 AM Ludovic Courtès <ludo@gnu.org> wrote:
>> >
>> > Hello Guix!
>> >
>> > Here’s a new blog post by Pierre Neidhardt entitled “Guix Profiles in
>> > Practice”:
>> >
>> >   https://guix.gnu.org/blog/2019/guix-profiles-in-practice/
>> >
>> > A great discussion on the benefits of manifests and on ways to manage
>> > several profiles!
>> 
>> Great article!
>
> I agree. One useful thing not mentioned in there is, that before
> loading a profile, it may be worthwhile to empty the shell with
>
>   env -i /bin/bash --login --noprofile --norc
>
>   . ~/opt/my-tools-profile/etc/profile
>
> so no existing shell variables may pollute your beautiful setup. 

Nice tip, Pjotr!

We can add it to the cookbook.  Shall I add it to the blog article now
that it's published?

In https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00565.html
we discussed the possibility to remove a specific profile from the
environment, which would be a more general version of what you propose.
It currently has some shortcomings, but maybe we can fix them.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 594 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Here’s a new blog post by Pierre Neidhardt entitled “Guix Profiles in
> Practice”:
>
>   https://guix.gnu.org/blog/2019/guix-profiles-in-practice/
>
> A great discussion on the benefits of manifests and on ways to manage
> several profiles!
>
> Ludo’.

I used some of these tricks just the other day!  Thanks, Pierre.  When I
find some time I want to also help figure out the issues which we
discovered, like Emacs not working well outside of the default profile.

Thank you very much for authoring the article!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: “Guix Profiles in Practice”

[Bengt Richter]

Hi Pierre,

On +2019-10-26 12:00:30 +0200, Pierre Neidhardt wrote:
> "Thompson, David" <dthompson2@worcester.edu> writes:
> 
> > Great article!
> 
> Thanks! :)
> 
> > I spotted a typo in the first paragraph: "on a same system" should be
> > "on the same system."
> 
> I can fix it.  Out of curiosity, why is it wrong?  "All users on a
> system..."  is correct, "All users on a given system" too, so I wonder
> why "on a same system" would not be.  What I meant here is "the same
> unspecified system".
> 
[...]

How would you say it in French? (I'm trying to recall high school French here, so be kind ;-)

I think english "a" is like French "un" or "une" -- so translating "on a same system" literally
(using my flawed French), does "en un même système" sound right to you?

It doesn't to me, unless même has a sense of pareil used that way.
Maybe "en utilisant le même système" ? (note le/the -- not un/a)
--
Regards,
Bengt Richter

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 565 bytes --]

Bengt Richter <bokr@bokr.com> writes:

> How would you say it in French? (I'm trying to recall high school French here, so be kind ;-)
>
> I think english "a" is like French "un" or "une" -- so translating "on a same system" literally
> (using my flawed French), does "en un même système" sound right to you?
>
> It doesn't to me, unless même has a sense of pareil used that way.
> Maybe "en utilisant le même système" ? (note le/the -- not un/a)

Yes, in French "sur un même système" is correct ;)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Pjotr Prins]

On Sat, Oct 26, 2019 at 12:02:51PM +0200, Pierre Neidhardt wrote:
> Pjotr Prins <pjotr.public12@thebird.nl> writes:
> 
> > On Fri, Oct 25, 2019 at 12:36:01PM -0400, Thompson, David wrote:
> >> On Fri, Oct 25, 2019 at 6:57 AM Ludovic Courtès <ludo@gnu.org> wrote:
> >> >
> >> > Hello Guix!
> >> >
> >> > Here’s a new blog post by Pierre Neidhardt entitled “Guix Profiles in
> >> > Practice”:
> >> >
> >> >   https://guix.gnu.org/blog/2019/guix-profiles-in-practice/
> >> >
> >> > A great discussion on the benefits of manifests and on ways to manage
> >> > several profiles!
> >> 
> >> Great article!
> >
> > I agree. One useful thing not mentioned in there is, that before
> > loading a profile, it may be worthwhile to empty the shell with
> >
> >   env -i /bin/bash --login --noprofile --norc
> >
> >   . ~/opt/my-tools-profile/etc/profile
> >
> > so no existing shell variables may pollute your beautiful setup. 
> 
> Nice tip, Pjotr!
> 
> We can add it to the cookbook.  Shall I add it to the blog article now
> that it's published?

Why not?

> In https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00565.html
> we discussed the possibility to remove a specific profile from the
> environment, which would be a more general version of what you propose.
> It currently has some shortcomings, but maybe we can fix them.

Sounds great.

Pierre, we have the minimalistic and emerging languages devroom at
FOSDEM this year. I am thinking we could do a talk about using Guix as
a development environment. I mean profiles are great for development -
I use them all the time, including guix environment -C. That could be
a talk.

Another talk could be about programming Guix from the REPL. Guix as a
DSL if you will. Anyone interested in doing a talk about that?

BTW I am also starting to use rash - the racket shell and we have
gash. Anyone want to talk about that? We should invite the authors.

Pj

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1773 bytes --]

Pjotr Prins <pjotr.public12@thebird.nl> writes:

>> We can add it to the cookbook.  Shall I add it to the blog article now
>> that it's published?
>
> Why not?

Depends how we look at the "Guix blog": shall we guarantee the content
of the articles to remain the same (beside typos) or shall we update
them as new ideas and other changes pop up?

The latter option has higher maintenance costs, plus we would need a way
to tell the reader when the article was first created, last edited, and
a link to the history of changes.

For the cookbook we've got all that with Git.

> Pierre, we have the minimalistic and emerging languages devroom at
> FOSDEM this year. I am thinking we could do a talk about using Guix as
> a development environment. I mean profiles are great for development -
> I use them all the time, including guix environment -C. That could be
> a talk.

You offering me to give a talk on this?  Why not, we can talk about it :)


> Another talk could be about programming Guix from the REPL. Guix as a
> DSL if you will. Anyone interested in doing a talk about that?

I would be very interested if someone could explain how they hack with
Geiser: I'm still frustrated with it on a daily basis :p I'm in
particular interested in debugging.  Regarding Guix-specific debugging,
I wonder how Guix developers debug staged code (i.e. when errors happen
for code executed by the daemon).

> BTW I am also starting to use rash - the racket shell and we have
> gash. Anyone want to talk about that? We should invite the authors.

Just watched the video: https://rash-lang.org/
It's amazing!
Now all we need is an Emacs (or https://github.com/soegaard/remacs?) integration...

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Konrad Hinsen]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> "Thompson, David" <dthompson2@worcester.edu> writes:
>
>> if it's a good idea.  Probably not.  So, I wonder if maybe a new
>> subcommand, say 'guix develop', could address this common development
>> use-case while allowing 'guix environment' to continue being the swiss
>> army knife that it is.  Some simple naming conventions could make this
...

> This is essentially what you propose, but to have it at the CLI level
> would provide several benefits:
...

> Both issues could be addressed the same way.

Perhaps. But then, "could" doesn't imply "should". I'd love to see all
that functionality easily accessible from the CLI, but I can't say I
know how this should best be organized.

Maybe we should start a Guix CLI nursery. A project/repository separate
from Guix itself that contains a copy of the "guix" script under a
different name ("guixx" for guix-extras?) and with the same interface
for scripting modules. We could then use this to play collectively with
ideas, and if something turns out to work well, migrate it to the
official Guix CLI.

Does this sound like a good idea? Would anyone else participate in such
an experiment?

Cheers,
  Konrad.

Re: “Guix Profiles in Practice”

[Jelle Licht]

Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

> Maybe we should start a Guix CLI nursery. A project/repository separate
> from Guix itself that contains a copy of the "guix" script under a
> different name ("guixx" for guix-extras?) and with the same interface
> for scripting modules. We could then use this to play collectively with
> ideas, and if something turns out to work well, migrate it to the
> official Guix CLI.
>
> Does this sound like a good idea? Would anyone else participate in such
> an experiment?

This could even work as a proper guix channel, once
http://issues.guix.info/issue/37399 is fixed!

Re: “Guix Profiles in Practice”

[Hartmut Goebel]

Am 25.10.19 um 18:36 schrieb Thompson, David:
> tool "just work."  Running 'guix develop' would check for a profile
> symlink with some canonical name, for example '.guix-develop-profile'.
> If it exists, it applies the environment variables and spawns a
> subshell.  If it doesn't exist, it looks for a 'develop.scm' file
> (canonical name TBD), builds the profile, symlinks it to
> '.guix-develop-profile', and then does the prior steps.  The tool

I like this idea :-)

I suggest naming the file like the directory: .guix-develop-profile.scm
(altough not sure whether the filenname should have the leading removed
or not)

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: “Guix Profiles in Practice”

[Hartmut Goebel]

Am 25.10.19 um 12:24 schrieb Ludovic Courtès:
> Here’s a new blog post by Pierre Neidhardt entitled “Guix Profiles in
> Practice”:

Great! I was not aware about the benefits of profiles and mostly use
environments.

Now I'm eager to learn how to mimic "guix envorinment some-package",
whihc oulls in all build dependencies of "some-package". Any hints?

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 395 bytes --]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Now I'm eager to learn how to mimic "guix envorinment some-package",
> whihc oulls in all build dependencies of "some-package". Any hints?

It does not seem possible at the moment, I started discussing this here:

https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00565.html

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1380 bytes --]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 25.10.19 um 18:36 schrieb Thompson, David:
>> tool "just work."  Running 'guix develop' would check for a profile
>> symlink with some canonical name, for example '.guix-develop-profile'.
>> If it exists, it applies the environment variables and spawns a
>> subshell.  If it doesn't exist, it looks for a 'develop.scm' file
>> (canonical name TBD), builds the profile, symlinks it to
>> '.guix-develop-profile', and then does the prior steps.  The tool
>
> I like this idea :-)
>
> I suggest naming the file like the directory: .guix-develop-profile.scm
> (altough not sure whether the filenname should have the leading removed
> or not)

I think it's useful to support multiple directories since it's likely
that users will work on multiple projects.

See what I've done for Emacs-Guix.  The (configurable) file structure
looks like this:

- ~/.guix-extra-profiles/my-project
- ~/.guix-extra-manifests/my-project
- ~/.guix-extra-channels/my-project

The mapping is obvious.

Then I can do "guix-install-manifest", Emacs prompts me for a manifest
name in ~/.guix-extra-manifests (with completion), and does all the
necessary to install it in ~/.guix-extra-profiles.  It dumps the channel
specification in ~/.guix-extra-channels/my-project.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Hartmut Goebel]

Am 27.10.19 um 20:06 schrieb Pierre Neidhardt:
> I think it's useful to support multiple directories since it's likely
> that users will work on multiple projects.
>
> See what I've done for Emacs-Guix.  The (configurable) file structure
> looks like this:
>
> - ~/.guix-extra-profiles/my-project
> - ~/.guix-extra-manifests/my-project
> - ~/.guix-extra-channels/my-project

AFAIU David (and this is what I meant, too), is to have the
"develop.scm" (or ".guix-develop-profile.scm") in the project working
tree. This would allow to easily maintain the profile together with the
source and a simple workflow like

cd /path/to/my-project
guix develop

or

guix develop /path/to/my-project

> Then I can do "guix-install-manifest", Emacs prompts me for a manifest
> name in ~/.guix-extra-manifests (with completion), and does all the
> necessary to install it in ~/.guix-extra-profiles.  It dumps the channel
> specification in ~/.guix-extra-channels/my-project.
IMHO this is a too emacs-centric approach. We should also have in mind
those not using guix-mode and not even using emacs.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1679 bytes --]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 27.10.19 um 20:06 schrieb Pierre Neidhardt:
>> I think it's useful to support multiple directories since it's likely
>> that users will work on multiple projects.
>>
>> See what I've done for Emacs-Guix.  The (configurable) file structure
>> looks like this:
>>
>> - ~/.guix-extra-profiles/my-project
>> - ~/.guix-extra-manifests/my-project
>> - ~/.guix-extra-channels/my-project
>
> AFAIU David (and this is what I meant, too), is to have the
> "develop.scm" (or ".guix-develop-profile.scm") in the project working
> tree. This would allow to easily maintain the profile together with the
> source and a simple workflow like
>
> cd /path/to/my-project
> guix develop
>
> or
>
> guix develop /path/to/my-project

Got it, we meant different things indeed.

>> Then I can do "guix-install-manifest", Emacs prompts me for a manifest
>> name in ~/.guix-extra-manifests (with completion), and does all the
>> necessary to install it in ~/.guix-extra-profiles.  It dumps the channel
>> specification in ~/.guix-extra-channels/my-project.
> IMHO this is a too emacs-centric approach. We should also have in mind
> those not using guix-mode and not even using emacs.

That was my point: it should not be Emacs-centric.

From the command line, we could do

  guix install-manifest my-project

and guix would lookup the user-define directories in
~/.config/guix/config.scm for instance.

Also I believe the Guix should store the channel specifications into the
profile when installed from a manifest.  It would make everything simpler.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 511 bytes --]

Chris Marusich <cmmarusich@gmail.com> writes:

> I used some of these tricks just the other day!  Thanks, Pierre.  When I
> find some time I want to also help figure out the issues which we
> discovered, like Emacs not working well outside of the default profile.

Chris, the Emacs issue is being discussed here:
https://lists.gnu.org/archive/html/help-guix/2019-10/msg00184.html
Maxim came up with a neat solution, we are testing it.  Feedback welcome!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 329 bytes --]

Pjotr, I've added your tip to the cookbook.

>   env -i /bin/bash --login --noprofile --norc
>
>   . ~/opt/my-tools-profile/etc/profile

Note that I've replaced /bin/bash with $(which bash) since /bin/bash
does not exist on Guix System ;)

Thanks for sharing!

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Hartmut Goebel]

Am 27.10.19 um 22:08 schrieb Pierre Neidhardt:
> From the command line, we could do
>
>   guix install-manifest my-project

IC. This is also a good way, esp. if one uses the same "my-project"
manifest for several locations.

But this way you can not make the manifest part of the project source.

> Also I believe the Guix should store the channel specifications into the
> profile when installed from a manifest.  It would make everything simpler.

ACK

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1021 bytes --]

Actually maybe we could generalize both features into one.

What's the difference between the two approaches?

- `guix develop' takes a path and guesses the manifest spec filename.
- `guix install-manifest' takes a manifest spec filename as argument.

What about supporting both?  It's easy, we simply need to check if the
argument is a folder or not.

David that the manifest be installed  to ~/.guix-develop-profile.  I
suggest instead that we all manifest have their own folder,
e.g. according to their path.

For example

- foo/bar/manifest.scm -> ~/.guix-extra-profiles/foo/bar
- foo2/bar/manifest.scm -> ~/.guix-extra-profiles/foo2/bar

This way the user needs not worry that the profiles get overwritten.

David suggested that the command automatically activated the profile.
To achieve this would could implement `guix environment --profile
/path/to/profile', and `guix develop` would run this automatically in
the background.

Thoughts?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Hartmut Goebel]

Am 31.10.19 um 12:25 schrieb Pierre Neidhardt:

> David that the manifest be installed  to ~/.guix-develop-profile.  I
> suggest instead that we all manifest have their own folder,
> e.g. according to their path.

Dave did not write about any path, but wrote:

> Running 'guix develop' would check for a profile
> symlink with some canonical name, for example '.guix-develop-profile'.
> If it exists, it applies the environment variables and spawns a
> subshell.  If it doesn't exist, it looks for a 'develop.scm' file
> (canonical name TBD), builds the profile, symlinks it to
> '.guix-develop-profile', and then does the prior steps.  The tool


This only makes sense, of the link and the .scm file are places in you
project's worktree (aka checkout). Otherwise one could have a single
"guix-develop-profile" only.

At least this is what I intend.

This would keep the definition and the links to the profiles in the
project. If I remove the worktree, the link to the profile will vanish,
too, and the garbage collector can collect it. Whereas if the profile is
stored in e.g. "~/.guix-extra-profiles/" this will end up in stale
profiles in this directory.


> I suggest instead that we all manifest have their own folder,
> e.g. according to their path.
>
> For example
>
> - foo/bar/manifest.scm -> ~/.guix-extra-profiles/foo/bar
> - foo2/bar/manifest.scm -> ~/.guix-extra-profiles/foo2/bar

What the base for "foo/bar" be in your examples? For me, worktrees may
have paths like

~/projects/customerA/project42/mixmas/
/mnt/server/outgroup/projects/customerA/project42/mixmas/

Also I strongly vote against using yet another "top-level" directory
like ~/-guix-extra-path. There is the XDG directory standard and I
suggest to stick with it. Thus IMHO this should be
$XDG_DATA_HOME/guix/profiles or $XDG_CONFIG_HOME/… - but I stell prefer
having this within the project's worktree.


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 297 bytes --]

Very good points, agreed with all of them.

I think we can still stick to a single guix command and use an option to
tell whether we want to drop  the profile in the current folder or in
the "default" folder (e.g. XDG_DATA_HOME/guix/profiles).

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Ludovic Courtès]

Hi,

"Thompson, David" <dthompson2@worcester.edu> skribis:

> 'guix environment' is a very composable command, and it's not clear to
> me how I would add the stuff I want onto it and it's even less clear
> if it's a good idea.  Probably not.  So, I wonder if maybe a new
> subcommand, say 'guix develop', could address this common development
> use-case while allowing 'guix environment' to continue being the swiss
> army knife that it is.  Some simple naming conventions could make this
> tool "just work."  Running 'guix develop' would check for a profile
> symlink with some canonical name, for example '.guix-develop-profile'.
> If it exists, it applies the environment variables and spawns a
> subshell.  If it doesn't exist, it looks for a 'develop.scm' file
> (canonical name TBD), builds the profile, symlinks it to
> '.guix-develop-profile', and then does the prior steps.  The tool
> would provide a mechanism to update, etc. and it could even be
> expanded later to spawn services like databases.

That sounds like a good idea.  ‘guix develop’ could keep the profiles it
manages under ~/.cache/guix.

Now, this would be very much stateful: you can’t tell in advance whether
you’re going to build a new profile based on the current Guix, or
whether you’re going to reuse a previously cached profile that could be
arbitrarily old.  That doesn’t sound great.

Perhaps we should focus on improving the performance of ‘guix
environment’ first and foremost?

Thanks,
Ludo’.

Re: “Guix Profiles in Practice”

[Jonathan Frederickson]

On Nov 3, 2019, at 9:24 AM, Ludovic Courtès <ludo@gnu.org> wrote
> Now, this would be very much stateful: you can’t tell in advance whether
> you’re going to build a new profile based on the current Guix, or
> whether you’re going to reuse a previously cached profile that could be
> arbitrarily old.  That doesn’t sound great.

What if something identifying the profile (directory name? config file?) included info about the Guix revision it was built from? That way a profile would be reused if it’s built from the same Guix revision, but if you switch revisions it’d be rebuilt.

I’m not sure how this would interact with a profile built from packages in multiple channels.

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1247 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> That sounds like a good idea.  ‘guix develop’ could keep the profiles it
> manages under ~/.cache/guix.
>
> Now, this would be very much stateful: you can’t tell in advance whether
> you’re going to build a new profile based on the current Guix, or
> whether you’re going to reuse a previously cached profile that could be
> arbitrarily old.  That doesn’t sound great.

This could be fixed, maybe the following way:

- Dump the channel specifications for every profile in
  etc/channel-spec.scm for instance.

- Merge Konrad's time-machine script so that we can easily reuse a
  channel specification to activate a profile.

> Perhaps we should focus on improving the performance of ‘guix
> environment’ first and foremost?

But isn't it orthogonal to this issue?

As I understand it, the current discussion is about "pinning"
profiles.
Should any dependency get garbage collected, an environment will need
Internet access to refetch the missing parts; I think this is what we
are trying to avoid, regardless of how fast `guix environment` is.


What about `guix environment --pin` instead of `guix develop`?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]

Jonathan Frederickson <jonathan@terracrypt.net> writes:

> What if something identifying the profile (directory name? config
> file?) included info about the Guix revision it was built from? That
> way a profile would be reused if it’s built from the same Guix
> revision, but if you switch revisions it’d be rebuilt.
>
> I’m not sure how this would interact with a profile built from packages in multiple channels.

Absolutely, I was suggesting something similar in another thread: dump
the output of `guix describe -f channels` into the profile,
e.g. etc/channel-spec.scm.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Ludovic Courtès]

Hello,

Jonathan Frederickson <jonathan@terracrypt.net> skribis:

> On Nov 3, 2019, at 9:24 AM, Ludovic Courtès <ludo@gnu.org> wrote
>> Now, this would be very much stateful: you can’t tell in advance whether
>> you’re going to build a new profile based on the current Guix, or
>> whether you’re going to reuse a previously cached profile that could be
>> arbitrarily old.  That doesn’t sound great.
>
> What if something identifying the profile (directory name? config file?) included info about the Guix revision it was built from? That way a profile would be reused if it’s built from the same Guix revision, but if you switch revisions it’d be rebuilt.

Then in that case you’re just looking for a form of caching, is that
correct?

If ‘guix environment’ were 10 times faster, it’d be just fine, right?

(It’s not getting 10 times faster any time soon I’m afraid :-), but just
to make sure I understand whether we’re talking about a UI issue or a
performance issue.)

Thanks,
Ludo’.

Re: “Guix Profiles in Practice”

[Carlo Zancanaro]

Hey Pierre,

On Mon, Nov 04 2019, Pierre Neidhardt wrote:
> As I understand it, the current discussion is about "pinning"
> profiles.
> Should any dependency get garbage collected, an environment will 
> need
> Internet access to refetch the missing parts; I think this is 
> what we
> are trying to avoid, regardless of how fast `guix environment` 
> is.
>
> What about `guix environment --pin` instead of `guix develop`?

Have you used `guix environment --root`?

In my git checkout of the Guix repository I once ran `guix 
environment guix --root=.environment`, and now whenever I want to 
hack on Guix I start a shell and `source 
.environment/etc/profile`. The .environment symlink is treated as 
a GC root by Guix, and because I'm not actually invoking Guix 
there's never a risk that it will attempt to fetch anything from 
the internet.

Upgrading the environment is quite easy, and must be done 
manually: I delete the existing symlink and run `guix environment 
guix --root=.environment` again. If I'm feeling paranoid I'll run 
`guix environment guix` once first to make sure it succeeds, 
before deleting the old symlink and adding the 
`--root=.environment` to the end.

Carlo

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 225 bytes --]

Oh my!!  I had no clue about `guix environment --root` :/
That's embarrassing... :p

This probably solves many issues.

I guess we ought to update the cookbook with this!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: “Guix Profiles in Practice”

[Ludovic Courtès]

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Oh my!!  I had no clue about `guix environment --root` :/
> That's embarrassing... :p

At least we learned something about discoverability of these options.
:-)

> This probably solves many issues.
>
> I guess we ought to update the cookbook with this!

That’d be great!

Ludo’.

Re: “Guix Profiles in Practice”

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 928 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Pierre Neidhardt <mail@ambrevar.xyz> skribis:
>
>> Oh my!!  I had no clue about `guix environment --root` :/
>> That's embarrassing... :p
>
> At least we learned something about discoverability of these options.
> :-)
>
>> This probably solves many issues.
>>
>> I guess we ought to update the cookbook with this!
>
> That’d be great!

Hmm, actually I don't really know where to put it.

The trick is that `guix environment --root` does no just use the
symlinked profile when it exists, it first builds the environment and
thus builds/downloads packages if necessary.

We don't  talk about guix environment in the cookbook chapter, so I'm
not sure if it's a good idea to introduce it there.

Maybe we first need to progress with regard to the command line
enhancements we are discussing in the other thread.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]