User-profile search paths should include system-profile directories

User-profile search paths should include system-profile directories

[Carlos Sánchez de La Lama]

Hi guys,

I have an interesting case here. I have guile installed in my system
profile, so that

/var/guix/profiles/system/profile/share/aclocal/guile.m4

is there. However, autoconf is installed in my *user* profile, so
ACLOCAL_PATH is augmented in ~/.guix-profile/etc/profile to include

~/.guix-profile/share/aclocal

But not the system-profile aclocal directory, which would be put into
ACLOCAL_PATH by /var/guix/profiles/system/profile/etc/profile if
autoconf was installed in the systme profile as well.

Is this the intended behaviour? I am wondering whether packages with
search paths should include both the user-profile directories and the
system-profile ones.

Thoughts?

BR

Carlos

-- 
'There is prodigious strength,' I answered him, 'in sorrow and despair.' 

Charles Dickens, "A Tale of Two Cities" (1859)

Re: User-profile search paths should include system-profile directories

[Ludovic Courtès]

Hello,

csanchezdll@gmail.com (Carlos Sánchez de La Lama) skribis:

> I have an interesting case here. I have guile installed in my system
> profile, so that
>
> /var/guix/profiles/system/profile/share/aclocal/guile.m4
>
> is there. However, autoconf is installed in my *user* profile, so
> ACLOCAL_PATH is augmented in ~/.guix-profile/etc/profile to include
>
> ~/.guix-profile/share/aclocal
>
> But not the system-profile aclocal directory, which would be put into
> ACLOCAL_PATH by /var/guix/profiles/system/profile/etc/profile if
> autoconf was installed in the systme profile as well.
>
> Is this the intended behaviour? I am wondering whether packages with
> search paths should include both the user-profile directories and the
> system-profile ones.

I think you’re right.  This was discussed at
<http://bugs.gnu.org/20255>, leading to a patch (for GuixSD).

However, we failed to build consensus around the approach of this patch,
so we did not apply it.  If you have ideas, please email
20255@debbugs.gnu.org.  :-)

One of the fruits of this discussion, though, is that you can do:

  guix package -p /run/current-system/profile -p ~/.guix-profile --search-paths

to get the combined search paths.

Ludo’.

Re: User-profile search paths should include system-profile directories

[Alex Kost]

Ludovic Courtès (2016-09-14 16:58 +0200) wrote:

> Hello,
>
> csanchezdll@gmail.com (Carlos Sánchez de La Lama) skribis:
>
>> I have an interesting case here. I have guile installed in my system
>> profile, so that
>>
>> /var/guix/profiles/system/profile/share/aclocal/guile.m4
>>
>> is there. However, autoconf is installed in my *user* profile, so
>> ACLOCAL_PATH is augmented in ~/.guix-profile/etc/profile to include
>>
>> ~/.guix-profile/share/aclocal
>>
>> But not the system-profile aclocal directory, which would be put into
>> ACLOCAL_PATH by /var/guix/profiles/system/profile/etc/profile if
>> autoconf was installed in the systme profile as well.
>>
>> Is this the intended behaviour? I am wondering whether packages with
>> search paths should include both the user-profile directories and the
>> system-profile ones.
>
> I think you’re right.  This was discussed at
> <http://bugs.gnu.org/20255>, leading to a patch (for GuixSD).
>
> However, we failed to build consensus around the approach of this patch,
> so we did not apply it.  If you have ideas, please email
> 20255@debbugs.gnu.org.  :-)

I think I was the one who prevents the consensus.  To make it clear, I'm
for the suggested solution, but only *after* giving a user a freedom to
avoid loading such a heavy command as "guix package --search-paths".  On
a "usual" GNU/Linux distro a user can edit /etc/profile, but on GuixSD
it is not possible currently.  That's why I think there should be
provided a possibility to override /etc/profile at first.

> One of the fruits of this discussion, though, is that you can do:
>
>   guix package -p /run/current-system/profile -p ~/.guix-profile --search-paths
>
> to get the combined search paths.

This is what I do to combine my profiles (I don't use ~/.guix-profile
but several other profiles instead).

-- 
Alex

Customizing /etc

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 2006 bytes --]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2016-09-14 16:58 +0200) wrote:
>
>> Hello,
>>
>> csanchezdll@gmail.com (Carlos Sánchez de La Lama) skribis:
>>
>>> I have an interesting case here. I have guile installed in my system
>>> profile, so that
>>>
>>> /var/guix/profiles/system/profile/share/aclocal/guile.m4
>>>
>>> is there. However, autoconf is installed in my *user* profile, so
>>> ACLOCAL_PATH is augmented in ~/.guix-profile/etc/profile to include
>>>
>>> ~/.guix-profile/share/aclocal
>>>
>>> But not the system-profile aclocal directory, which would be put into
>>> ACLOCAL_PATH by /var/guix/profiles/system/profile/etc/profile if
>>> autoconf was installed in the systme profile as well.
>>>
>>> Is this the intended behaviour? I am wondering whether packages with
>>> search paths should include both the user-profile directories and the
>>> system-profile ones.
>>
>> I think you’re right.  This was discussed at
>> <http://bugs.gnu.org/20255>, leading to a patch (for GuixSD).
>>
>> However, we failed to build consensus around the approach of this patch,
>> so we did not apply it.  If you have ideas, please email
>> 20255@debbugs.gnu.org.  :-)
>
> I think I was the one who prevents the consensus.  To make it clear, I'm
> for the suggested solution, but only *after* giving a user a freedom to
> avoid loading such a heavy command as "guix package --search-paths".  On
> a "usual" GNU/Linux distro a user can edit /etc/profile, but on GuixSD
> it is not possible currently.  That's why I think there should be
> provided a possibility to override /etc/profile at first.

Indeed, thanks for the reminder!

In fact, we have this through ‘etc-service-type’, except that currently
/etc/profile is systematically added.

So an idea that comes to mind is to allow ‘etc-service-type’ to be
extended with procedures that would be able to filter or otherwise
change the /etc entries (similar to what we do for PAM):


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1144 bytes --]

diff --git a/gnu/services.scm b/gnu/services.scm
index 7e322c5..9397232 100644
--- a/gnu/services.scm
+++ b/gnu/services.scm
@@ -426,9 +426,13 @@ directory."
                 (extensions
                  (list
                   (service-extension activation-service-type
-                                     (lambda (files)
-                                       (let ((etc
-                                              (files->etc-directory files)))
+                                     (lambda (files+procs)
+                                       (let* ((proc (apply compose
+                                                           (filter procedure?
+                                                                   files+procs)))
+                                              (files (filter pair? files+procs))
+                                              (etc
+                                               (files->etc-directory (proc files))))
                                          #~(activate-etc #$etc))))
                   (service-extension system-service-type etc-entry)))
                 (compose concatenate)

[-- Attachment #3: Type: text/plain, Size: 460 bytes --]


In your config, you could have something like:

  (services (cons (simple-service 'rm-/etc/profile etc-service-type
                                  (const (lambda (files)
                                           (assoc-delete "profile" files))))
                  %base-services))

WDYT?

In fact I think we would need to have a more generic mechanism to hook
into ‘fold-services’, but I’m not sure what it should look like.

Ludo’.

Re: Customizing /etc

[Alex Kost]

Ludovic Courtès (2016-09-19 23:08 +0900) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2016-09-14 16:58 +0200) wrote:
[...]
>>> However, we failed to build consensus around the approach of this patch,
>>> so we did not apply it.  If you have ideas, please email
>>> 20255@debbugs.gnu.org.  :-)
>>
>> I think I was the one who prevents the consensus.  To make it clear, I'm
>> for the suggested solution, but only *after* giving a user a freedom to
>> avoid loading such a heavy command as "guix package --search-paths".  On
>> a "usual" GNU/Linux distro a user can edit /etc/profile, but on GuixSD
>> it is not possible currently.  That's why I think there should be
>> provided a possibility to override /etc/profile at first.
>
> Indeed, thanks for the reminder!
>
> In fact, we have this through ‘etc-service-type’, except that currently
> /etc/profile is systematically added.
>
> So an idea that comes to mind is to allow ‘etc-service-type’ to be
> extended with procedures that would be able to filter or otherwise
> change the /etc entries (similar to what we do for PAM):
>
> diff --git a/gnu/services.scm b/gnu/services.scm
> index 7e322c5..9397232 100644
> --- a/gnu/services.scm
> +++ b/gnu/services.scm
> @@ -426,9 +426,13 @@ directory."
>                  (extensions
>                   (list
>                    (service-extension activation-service-type
> -                                     (lambda (files)
> -                                       (let ((etc
> -                                              (files->etc-directory files)))
> +                                     (lambda (files+procs)
> +                                       (let* ((proc (apply compose
> +                                                           (filter procedure?
> +                                                                   files+procs)))
> +                                              (files (filter pair? files+procs))
> +                                              (etc
> +                                               (files->etc-directory (proc files))))
>                                           #~(activate-etc #$etc))))
>                    (service-extension system-service-type etc-entry)))
>                  (compose concatenate)
>
>
> In your config, you could have something like:
>
>   (services (cons (simple-service 'rm-/etc/profile etc-service-type
>                                   (const (lambda (files)
>                                            (assoc-delete "profile" files))))
>                   %base-services))
>
> WDYT?

I think it would be great!  I didn't realize it can be implemented this
way.

> In fact I think we would need to have a more generic mechanism to hook
> into ‘fold-services’, but I’m not sure what it should look like.

Yeah, a more generic way would be better of course, but I think this
mixing of files and procedures is already good enough for now, as it
provides a freedom in customizing a system that we didn't have before.
Besides I will no longer object against the "search-paths" fix for the
bug 20255 :-)

-- 
Alex

Re: User-profile search paths should include system-profile directories

[Carlos Sánchez de La Lama]

Hi,

>> Is this the intended behaviour? I am wondering whether packages with
>> search paths should include both the user-profile directories and the
>> system-profile ones.
>
> I think you’re right.  This was discussed at
> <http://bugs.gnu.org/20255>, leading to a patch (for GuixSD).
>
> However, we failed to build consensus around the approach of this patch,
> so we did not apply it.  If you have ideas, please email
> 20255@debbugs.gnu.org.  :-)

what about something like:

- /etc/profile
  # [...]
  GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
  . /run/current-system/profile/etc/profile 
  # [...]
  . "$HOME/.guix-profile/etc/profile"
  # [...]

- /run/current-system/profile/etc/profile
  profiles="${GUIX_PROFILES:=/gnu/store/<hashA>-profile}"
  export PATH="${profiles//:/\/bin}/bin ${profiles//:/\/sbin}/sbin"
  # [...]

  
- $HOME/.guix-profile/etc/profile
  profiles="${GUIX_PROFILES:=/gnu/store/<hashB>-profile}"
  export ACLOCAL_PATH="${profiles//:/\/share/aclocal}/share/aclocal"
  # [...]

That is, each profile adds all its search paths to all the profile roots
passed in the colon separated variable GUIX_PROFILES. If GUIX_PROFILES
is empty, it adds the search paths inside its own directory only (as
until now).

No need to expand recursively (no ":$OLD_VALUE" at then end of each
assignment) as we are generating the profile, user additions will come
later. Should be fast enough, and a user can do the (slower) guix
--search-paths in its own profile if he wants to.

If you think it's fine, I will prepare a patch.

Thoughts?

Carlos

Re: User-profile search paths should include system-profile directories

[Alex Kost]

Carlos Sánchez de La Lama (2016-09-20 09:39 +0200) wrote:

> Hi,
>
>>> Is this the intended behaviour? I am wondering whether packages with
>>> search paths should include both the user-profile directories and the
>>> system-profile ones.
>>
>> I think you’re right.  This was discussed at
>> <http://bugs.gnu.org/20255>, leading to a patch (for GuixSD).
>>
>> However, we failed to build consensus around the approach of this patch,
>> so we did not apply it.  If you have ideas, please email
>> 20255@debbugs.gnu.org.  :-)
>
> what about something like:
>
> - /etc/profile
>   # [...]
>   GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
>   . /run/current-system/profile/etc/profile
>   # [...]
>   . "$HOME/.guix-profile/etc/profile"
>   # [...]
>
> - /run/current-system/profile/etc/profile
>   profiles="${GUIX_PROFILES:=/gnu/store/<hashA>-profile}"
>   export PATH="${profiles//:/\/bin}/bin ${profiles//:/\/sbin}/sbin"
>   # [...]
>
>
> - $HOME/.guix-profile/etc/profile
>   profiles="${GUIX_PROFILES:=/gnu/store/<hashB>-profile}"
>   export ACLOCAL_PATH="${profiles//:/\/share/aclocal}/share/aclocal"
>   # [...]
>
> That is, each profile adds all its search paths to all the profile roots
> passed in the colon separated variable GUIX_PROFILES. If GUIX_PROFILES
> is empty, it adds the search paths inside its own directory only (as
> until now).

I think it was mentioned somewhere in the bug discussion: this will not
work for some things.  For example, if a user has 'guile' in a system
profile, and several guile packages (but not 'guile' itself) in
~/.guix-profile, then GUILE_LOAD_PATH will include
"<system-profile>/share/guile/site/2.0" but not
"<user-profile>/share/guile/site/2.0".  That's why combining profiles
inside "guix package --search-paths" command looks like the only
solution.

-- 
Alex

Re: User-profile search paths should include system-profile directories

[Carlos Sánchez de La Lama]

Alex Kost <alezost@gmail.com> writes:

> Carlos Sánchez de La Lama (2016-09-20 09:39 +0200) wrote:
>
>> Hi,
>>
>>>> Is this the intended behaviour? I am wondering whether packages with
>>>> search paths should include both the user-profile directories and the
>>>> system-profile ones.
>>>
>>> I think you’re right.  This was discussed at
>>> <http://bugs.gnu.org/20255>, leading to a patch (for GuixSD).
>>>
>>> However, we failed to build consensus around the approach of this patch,
>>> so we did not apply it.  If you have ideas, please email
>>> 20255@debbugs.gnu.org.  :-)
>>
>> what about something like:
>>
>> - /etc/profile
>>   # [...]
>>   GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
>>   . /run/current-system/profile/etc/profile
>>   # [...]
>>   . "$HOME/.guix-profile/etc/profile"
>>   # [...]
>>
>> - /run/current-system/profile/etc/profile
>>   profiles="${GUIX_PROFILES:=/gnu/store/<hashA>-profile}"
>>   export PATH="${profiles//:/\/bin}/bin ${profiles//:/\/sbin}/sbin"
>>   # [...]
>>
>>
>> - $HOME/.guix-profile/etc/profile
>>   profiles="${GUIX_PROFILES:=/gnu/store/<hashB>-profile}"
>>   export ACLOCAL_PATH="${profiles//:/\/share/aclocal}/share/aclocal"
>>   # [...]
>>
>> That is, each profile adds all its search paths to all the profile roots
>> passed in the colon separated variable GUIX_PROFILES. If GUIX_PROFILES
>> is empty, it adds the search paths inside its own directory only (as
>> until now).
>
> I think it was mentioned somewhere in the bug discussion: this will not
> work for some things.  For example, if a user has 'guile' in a system
> profile, and several guile packages (but not 'guile' itself) in
> ~/.guix-profile, then GUILE_LOAD_PATH will include
> "<system-profile>/share/guile/site/2.0" but not
> "<user-profile>/share/guile/site/2.0".  That's why combining profiles
> inside "guix package --search-paths" command looks like the only
> solution.

It will actually solve it, that is the case I was trying to fix in
fact. The code I show will include /share/guile/site/2.0 prefixed
by every root in the colon-separated list GUIX_PROFILES (note the
difference with current code with has GUIX_PROFILE, singular). If
GUIX_PROFILES is (as in the example):

GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"

And guile is in the system profile, the line

export GUILE_LOAD_PATH="${profiles//://share/guile/site/2.0:}/share/guile/site/2.0"

will produce:

export GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/2.0:$HOME/.guix-profile/share/guile/site/2.0

As desired. Same result if guile is only in user profile, or if it is on
both. Try the following in bash:

profiles="/run/current-system/profile:$HOME/.guix-profile:/my/third/profile"
echo "${profiles//://share/guile/site/2.0:}/share/guile/site/2.0"

BR

Carlos

Re: User-profile search paths should include system-profile directories

[Ludovic Courtès]

Hi,

csanchezdll@gmail.com (Carlos Sánchez de La Lama) skribis:

> Alex Kost <alezost@gmail.com> writes:

[...]

>>> what about something like:
>>>
>>> - /etc/profile
>>>   # [...]
>>>   GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
>>>   . /run/current-system/profile/etc/profile
>>>   # [...]
>>>   . "$HOME/.guix-profile/etc/profile"
>>>   # [...]
>>>
>>> - /run/current-system/profile/etc/profile
>>>   profiles="${GUIX_PROFILES:=/gnu/store/<hashA>-profile}"
>>>   export PATH="${profiles//:/\/bin}/bin ${profiles//:/\/sbin}/sbin"
>>>   # [...]
>>>
>>>
>>> - $HOME/.guix-profile/etc/profile
>>>   profiles="${GUIX_PROFILES:=/gnu/store/<hashB>-profile}"
>>>   export ACLOCAL_PATH="${profiles//:/\/share/aclocal}/share/aclocal"
>>>   # [...]
>>>
>>> That is, each profile adds all its search paths to all the profile roots
>>> passed in the colon separated variable GUIX_PROFILES. If GUIX_PROFILES
>>> is empty, it adds the search paths inside its own directory only (as
>>> until now).
>>
>> I think it was mentioned somewhere in the bug discussion: this will not
>> work for some things.  For example, if a user has 'guile' in a system
>> profile, and several guile packages (but not 'guile' itself) in
>> ~/.guix-profile, then GUILE_LOAD_PATH will include
>> "<system-profile>/share/guile/site/2.0" but not
>> "<user-profile>/share/guile/site/2.0".  That's why combining profiles
>> inside "guix package --search-paths" command looks like the only
>> solution.
>
> It will actually solve it, that is the case I was trying to fix in
> fact. The code I show will include /share/guile/site/2.0 prefixed
> by every root in the colon-separated list GUIX_PROFILES (note the
> difference with current code with has GUIX_PROFILE, singular). If
> GUIX_PROFILES is (as in the example):
>
> GUIX_PROFILES="/run/current-system/profile:$HOME/.guix-profile"
>
> And guile is in the system profile, the line
>
> export GUILE_LOAD_PATH="${profiles//://share/guile/site/2.0:}/share/guile/site/2.0"
>
> will produce:
>
> export GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/2.0:$HOME/.guix-profile/share/guile/site/2.0

Unless I’m mistaken, I don’t think this would work.

In the example Alex gave, none of the two profiles specifies
GUILE_LOAD_PATH if taken individually.  This example is also in the
manual to illustrate the use of multiple -p flags with --search-paths
(info "(guix) Invoking guix package").

Thoughts?

Ludo’.

Re: User-profile search paths should include system-profile directories

[Carlos Sánchez de La Lama]

Hi Ludo, Alex,

> In the example Alex gave, none of the two profiles specifies
> GUILE_LOAD_PATH if taken individually.  This example is also in the
> manual to illustrate the use of multiple -p flags with --search-paths
> (info "(guix) Invoking guix package").

I think I understood now. I knew "guix --search-paths" only returned
those search paths which *do exist* in the current profile (thus the
need for multiple -p parameters), but I (wrongly) thought search paths
were added unconditionally to <profile_root>/etc/profile.

Making this work would need two things:
1) not stripping unexisting directories from search paths
2) constructing environment variables prepending every profile root to
   the search paths of each profile.

And my suggestion only solved point 2.

Now, my question is, why are unexisting search paths removed? This is
done in guix-search-paths.scm(evaluate-sarch-paths), by calling
search-path-as-list, which filters out unexisting directories. Why not
just add all of them? Profiles are immutable once created, so having a
search path pointing into a subexisting subdirectory should not do any
harm.

Am I missing something?

Carlos

Re: User-profile search paths should include system-profile directories

[Ludovic Courtès]

Hi Carlos,

csanchezdll@gmail.com (Carlos Sánchez de La Lama) skribis:

> Hi Ludo, Alex,
>
>> In the example Alex gave, none of the two profiles specifies
>> GUILE_LOAD_PATH if taken individually.  This example is also in the
>> manual to illustrate the use of multiple -p flags with --search-paths
>> (info "(guix) Invoking guix package").
>
> I think I understood now. I knew "guix --search-paths" only returned
> those search paths which *do exist* in the current profile (thus the
> need for multiple -p parameters), but I (wrongly) thought search paths
> were added unconditionally to <profile_root>/etc/profile.
>
> Making this work would need two things:
> 1) not stripping unexisting directories from search paths
> 2) constructing environment variables prepending every profile root to
>    the search paths of each profile.
>
> And my suggestion only solved point 2.
>
> Now, my question is, why are unexisting search paths removed?

I don’t think this is really the problem here.  The problem here is that
if a profile contains only, say, guile-json, there is no way Guix can
know about GUILE_LOAD_PATH because that variable is associated with
Guile itself, not with guile-json.

Does that make sense?

Thanks,
Ludo’.

Re: User-profile search paths should include system-profile directories

[Carlos Sánchez de La Lama]

Hi Ludo,

>> Making this work would need two things:
>> 1) not stripping unexisting directories from search paths
>> 2) constructing environment variables prepending every profile root to
>>    the search paths of each profile.
>>
>> Now, my question is, why are unexisting search paths removed?
>
> I don’t think this is really the problem here.  The problem here is that
> if a profile contains only, say, guile-json, there is no way Guix can
> know about GUILE_LOAD_PATH because that variable is associated with
> Guile itself, not with guile-json.

The idea is each profile environment unconditionally adds search paths
for all the profiles in use (which are passed when evaluating
<profile>/etc/profile in the GUIX_PROFILES variable). This was the
proposal in my first message (looks *similar* to an earlier post in the
bug discussion, but it is quite different).

In the case you suggest, the profile cointaining guile will define
GUILE_LOAD_PATH to point inside its own profile and *also* inside the
profile containing guile-json.

The profile that has guile-json will do nothing, as its installer
packages have no search paths.

This is the key:

export ACLOCAL_PATH="${profiles//:/\/share/aclocal:}/share/aclocal

So ACLOCAL_PATH will have an entry for <each profile>/share/aclocal,
regardless of that path existing or not. If path is not there, no harm
done. If it is (i.e. that profile has a package installing an aclocal
.m4 file) then aclocal will find it.

I will try to prepare a patch, so we can see (and might be there is some
problem I still fail to see).

BR

Carlos

Re: User-profile search paths should include system-profile directories

[Ludovic Courtès]

Howdy,

csanchezdll@gmail.com (Carlos Sánchez de La Lama) skribis:

>>> Making this work would need two things:
>>> 1) not stripping unexisting directories from search paths
>>> 2) constructing environment variables prepending every profile root to
>>>    the search paths of each profile.
>>>
>>> Now, my question is, why are unexisting search paths removed?
>>
>> I don’t think this is really the problem here.  The problem here is that
>> if a profile contains only, say, guile-json, there is no way Guix can
>> know about GUILE_LOAD_PATH because that variable is associated with
>> Guile itself, not with guile-json.
>
> The idea is each profile environment unconditionally adds search paths
> for all the profiles in use (which are passed when evaluating
> <profile>/etc/profile in the GUIX_PROFILES variable). This was the
> proposal in my first message (looks *similar* to an earlier post in the
> bug discussion, but it is quite different).
>
> In the case you suggest, the profile cointaining guile will define
> GUILE_LOAD_PATH to point inside its own profile and *also* inside the
> profile containing guile-json.

The problem is that, as things currently are, the profile that contains
only Guile has no reason to define GUILE_LOAD_PATH.  So I’m unsure if
this can work as is.  We’ll see!

Thanks for looking into it,
Ludo’.

Customizing /etc

[Alex Kost]

This is almost the same message as:

  http://lists.gnu.org/archive/html/guix-devel/2016-09/msg01570.html

I mean this is a reminder on the subject.

Ludovic, I think you suggested a good solution to a "Customizing
/etc/profile" problem.  Since I will no longer be a blocker of
<http://bugs.gnu.org/20255> after this fix, would you like to commit
that change, or are there any issues with that?

-- 
Thanks,
Alex

Re: Customizing /etc

[Ludovic Courtès]

Hi Alex,

Alex Kost <alezost@gmail.com> skribis:

> This is almost the same message as:
>
>   http://lists.gnu.org/archive/html/guix-devel/2016-09/msg01570.html
>
> I mean this is a reminder on the subject.

Yes, I haven’t forgotten about it, contrary to what one might think.
;-)

> Ludovic, I think you suggested a good solution to a "Customizing
> /etc/profile" problem.  Since I will no longer be a blocker of
> <http://bugs.gnu.org/20255> after this fix, would you like to commit
> that change, or are there any issues with that?

In the message you cite above, I proposed to hack etc-service-type in
the same way we did for PAM.

But then I think we’d better have the generic solution in place (where
any service could be extended “at the end”, i.e., after it’s ‘compose’
and ‘extend’ procedures have been called) before people start relying on
the hack.  I even started working on it long ago, and then moved on to
something else.

I’m sorry that this has not happened yet.  I hope to resume work on it
after the release, but if someone wants to give it a shot, I will
definitely not block it.

Thanks,
Ludo’.

Re: Customizing /etc

[Alex Kost]

Ludovic Courtès (2017-05-17 10:28 +0200) wrote:

> Hi Alex,
>
> Alex Kost <alezost@gmail.com> skribis:
>
>> This is almost the same message as:
>>
>>   http://lists.gnu.org/archive/html/guix-devel/2016-09/msg01570.html
>>
>> I mean this is a reminder on the subject.
>
> Yes, I haven’t forgotten about it, contrary to what one might think.
> ;-)

Oh, great!  (you're right, that's what I thought :-))

>> Ludovic, I think you suggested a good solution to a "Customizing
>> /etc/profile" problem.  Since I will no longer be a blocker of
>> <http://bugs.gnu.org/20255> after this fix, would you like to commit
>> that change, or are there any issues with that?
>
> In the message you cite above, I proposed to hack etc-service-type in
> the same way we did for PAM.
>
> But then I think we’d better have the generic solution in place (where
> any service could be extended “at the end”, i.e., after it’s ‘compose’
> and ‘extend’ procedures have been called) before people start relying on
> the hack.  I even started working on it long ago, and then moved on to
> something else.

Wow, a general solution for any service sounds really great!

> I’m sorry that this has not happened yet.  I hope to resume work on it
> after the release, but if someone wants to give it a shot, I will
> definitely not block it.

I see :-)  No hurry, I just was a bit concerned that your solution was
lost, but now I know that it's not, thanks!

-- 
Alex

Adding operating-system field for a custom /etc/profile.

[Alex Kost]

This is a continuation of the discussion beginning here:
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20255#44>.

To sum up: I would like to have a possibility to use my own /etc/profile
instead of the default one, but Ludovic doesn't want to provide me this
freedom :-(

Ludovic Courtès (2015-11-23 17:31 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2015-11-23 02:04 +0300) wrote:
>>
>>> Alex Kost <alezost@gmail.com> skribis:
[...]
>>>> … what I suggest now is just to give an option to avoid generating the
>>>> default /etc/profile.  What about making an 'operating-system' field for
>>>> this file (similar to 'sudoers-file' or 'hosts-file')?  So when such
>>>> 'profile-file' is specified, it will be used instead of the default one
>>>> (of course, it should be mentioned in the manual that it's only for
>>>> those users who are sure what they do).
>>>
>>> I think we could make an /etc/profile-service that receives snippets
>>> meant to be glued together into the final /etc/profile.  Users could
>>> specify the top or bottom of the file.
>>>
>>> There could be a combined-search-paths-service that implements the
>>> solution I proposed here.
>>>
>>> WDYT?
>>
>> I agree, the more ways to change a default behaviour, the better.
>> Although I will not use these things if there will be ‘profile-file’
>> field that allows to specify my own "/etc/profile".
>
> [...]
>
>> Great!  So is it OK to send a patch for adding ‘profile-file’ field?
>
> Hmm, I’m not sure if we want to give direct access to /etc/profile like
> this.

Oh, no!  If there is one person (me) who wants to have a full control on
his /etc/profile, there may be the others with the same wish.

> The problem is that several things in there are here to make the system
> work, and to to make it conform to the ‘operating-system’ declaration,
> such as:
>
>
> export LANG="en_US.utf8"
> export TZ="Europe/Paris"
> export TZDIR="/gnu/store/rwvf6xqgsyb8bmpi7rwk9fildnwvzrv5-tzdata-2015c/share/zoneinfo"
>
> # Tell 'modprobe' & co. where to look for modules.
> export LINUX_MODULE_DIRECTORY=/run/booted-system/kernel/lib/modules

Yes, that's why I suggest to add a note to the manual about a danger of
using this field.

> The risk I see with adding a raw ‘profile-file’ option is that newcomers
> may end up getting rid of such things without really noticing, and then
> getting a broken system.

But a newcomer will learn about this option only if (s)he reads the
manual with the warning I've mentioned.  For me, your phrase sounds
like: «We will not provide "rm" command, because a newcomer may
accidentally run "rm -rf ~"».  Please give me an opportunity to shoot
myself in the foot!

Besides will the system really be broken?  What do you mean?  Even if
/etc/profile is empty, the system will boot successfully and a user
could login, no?

> What about instead giving a way to populate the top and/or bottom of
> this file?  Controversial parts, if any, could still be turned on and
> off by adding or removing services that add these lines?

It is better than nothing, but it is not sufficient IMO.  Any part of
/etc/profile can be controversial (you'll never know what a user would
like to change), so I think providing an option to change this file
completely is essential.

But I agree that appending/prepending some lines may also be useful for
those who like to keep the default /etc/profile and who just want to add
something to it.

> I think we should open a separate bug report to discuss this.

I agree that it's not related to this bug, so I'm sending this message
to guix-devel list.

-- 
Alex

Re: Adding operating-system field for a custom /etc/profile.

[宋文武]

On 2015-11-24 04:07, Alex Kost wrote:
> This is a continuation of the discussion beginning here:
> <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20255#44>.
> 
> To sum up: I would like to have a possibility to use my own 
> /etc/profile
> instead of the default one, but Ludovic doesn't want to provide me this
> freedom :-(
Well, every comment in /etc/profile came with a hack which make
something work.  but it's becomming big and hard to understand every 
line.

> 
> Ludovic Courtès (2015-11-23 17:31 +0300) wrote:
> 
>> Alex Kost <alezost@gmail.com> skribis:
>> 
>>> Ludovic Courtès (2015-11-23 02:04 +0300) wrote:
>>> 
>>>> Alex Kost <alezost@gmail.com> skribis:
> [...]
>>>>> … what I suggest now is just to give an option to avoid generating 
>>>>> the
>>>>> default /etc/profile.  What about making an 'operating-system' 
>>>>> field for
>>>>> this file (similar to 'sudoers-file' or 'hosts-file')?  So when 
>>>>> such
>>>>> 'profile-file' is specified, it will be used instead of the default 
>>>>> one
>>>>> (of course, it should be mentioned in the manual that it's only for
>>>>> those users who are sure what they do).
>>>> 
>>>> I think we could make an /etc/profile-service that receives snippets
>>>> meant to be glued together into the final /etc/profile.  Users could
>>>> specify the top or bottom of the file.
>>>> 
>>>> There could be a combined-search-paths-service that implements the
>>>> solution I proposed here.
>>>> 
>>>> WDYT?
>>> 
>>> I agree, the more ways to change a default behaviour, the better.
>>> Although I will not use these things if there will be ‘profile-file’
>>> field that allows to specify my own "/etc/profile".
>> 
>> [...]
>> 
>>> Great!  So is it OK to send a patch for adding ‘profile-file’ field?
>> 
>> Hmm, I’m not sure if we want to give direct access to /etc/profile 
>> like
>> this.
> 
> Oh, no!  If there is one person (me) who wants to have a full control 
> on
> his /etc/profile, there may be the others with the same wish.
Sure, I think we all want (and should have) a full control.

> 
>> The problem is that several things in there are here to make the 
>> system
>> work, and to to make it conform to the ‘operating-system’ declaration,
>> such as:
>> 
>> 
>> export LANG="en_US.utf8"
>> export TZ="Europe/Paris"
>> export 
>> TZDIR="/gnu/store/rwvf6xqgsyb8bmpi7rwk9fildnwvzrv5-tzdata-2015c/share/zoneinfo"
>> 
>> # Tell 'modprobe' & co. where to look for modules.
>> export LINUX_MODULE_DIRECTORY=/run/booted-system/kernel/lib/modules
> 
> Yes, that's why I suggest to add a note to the manual about a danger of
> using this field.
> 
>> The risk I see with adding a raw ‘profile-file’ option is that 
>> newcomers
>> may end up getting rid of such things without really noticing, and 
>> then
>> getting a broken system.
> 
> But a newcomer will learn about this option only if (s)he reads the
> manual with the warning I've mentioned.  For me, your phrase sounds
> like: «We will not provide "rm" command, because a newcomer may
> accidentally run "rm -rf ~"».  Please give me an opportunity to shoot
> myself in the foot!
> 
> Besides will the system really be broken?  What do you mean?  Even if
> /etc/profile is empty, the system will boot successfully and a user
> could login, no?
Yes, login works, but then /run/current-system/profile/bin isn't in
PATH, and some system configurations (eg: locale, timezone) are ignored.

> 
>> What about instead giving a way to populate the top and/or bottom of
>> this file?  Controversial parts, if any, could still be turned on and
>> off by adding or removing services that add these lines?
> 
> It is better than nothing, but it is not sufficient IMO.  Any part of
> /etc/profile can be controversial (you'll never know what a user would
> like to change), so I think providing an option to change this file
> completely is essential.
To be clear, /etc/profile contains 3 parts:

  1. variables from configuration of the operating-system (LANG, TZ, 
etc.)
  2. environment setup for system and user profiles
     (source .guix-profile/etc/profile)
  3. hacks for making sensible defaults (LINUX_MODULE_DIRECTORY,
     ASPELL_CONF, etc).

And it's only effective for POSIX login shells (bash and zsh).

For 1, maybe the most important one, it's already managed, but doesn't
work for fish and rc.  We need to move these into /etc/environment,
which work for all shells (even emacs? :-)

I had recall my tries, but at that time only zsh was considered.
<https://lists.gnu.org/archive/html/guix-devel/2014-11/msg00674.html>


For 2, we had build a etc/profile file for each profile's search-paths,
here source both system and user to make most things work 
out-of-the-box.

I think this is the real purpose for our /etc/profile.
Technical, if we remove those, the result system will be the same as
guix on foreign distros.  So, it's ok to completely replace it.

(some variables (eg: MANPATH, INFOPATH, XDG_DATA_DIRS) can be set in
each profile, and mergerd well).


And 3, IMO is the controversial parts.

the one don't related to profiles can go into /etc/environment
(eg: LINUX_MODULE_DIRECTORY, SSL_CERT_DIR, DBUS_FATAL_WARNINGS),
these need to be addressing by adding services?

and others may go into profile (eg: ASPELL_CONF, GST_PLUGIN_PATH).


So, the plan is add /etc/environment and only use /etc/profile for 2.
then, a sh-profile file-like configuration can be added.  WDYT?

Re: Adding operating-system field for a custom /etc/profile.

[Alex Kost]

宋文武 (2015-11-24 18:22 +0300) wrote:

> On 2015-11-24 04:07, Alex Kost wrote:
>> This is a continuation of the discussion beginning here:
>> <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20255#44>.
>>
>> To sum up: I would like to have a possibility to use my own
>> /etc/profile
>> instead of the default one, but Ludovic doesn't want to provide me this
>> freedom :-(
> Well, every comment in /etc/profile came with a hack which make
> something work.  but it's becomming big and hard to understand every
> line.

Sorry, I don't understand what you want to say.  I'm able to make my own
/etc/profile based on the default one, and I just want to have an option
to do it.

>> Ludovic Courtès (2015-11-23 17:31 +0300) wrote:
>>
>>> Hmm, I’m not sure if we want to give direct access to /etc/profile
>>> like
>>> this.
>>
>> Oh, no!  If there is one person (me) who wants to have a full control
>> on
>> his /etc/profile, there may be the others with the same wish.
> Sure, I think we all want (and should have) a full control.

Yes, unluckily GuixSD does not provide such control currently.

[...]
>>> The risk I see with adding a raw ‘profile-file’ option is that
>>> newcomers
>>> may end up getting rid of such things without really noticing, and
>>> then
>>> getting a broken system.
>>
>> But a newcomer will learn about this option only if (s)he reads the
>> manual with the warning I've mentioned.  For me, your phrase sounds
>> like: «We will not provide "rm" command, because a newcomer may
>> accidentally run "rm -rf ~"».  Please give me an opportunity to shoot
>> myself in the foot!
>>
>> Besides will the system really be broken?  What do you mean?  Even if
>> /etc/profile is empty, the system will boot successfully and a user
>> could login, no?
> Yes, login works, but then /run/current-system/profile/bin isn't in
> PATH, and some system configurations (eg: locale, timezone) are ignored.

Yes, but we are talking about an optional thing, that should be
explicitly set by a user, so I don't really understand concerns about
the potential risk, as a user will learn about this option at first
before using it.

>>> What about instead giving a way to populate the top and/or bottom of
>>> this file?  Controversial parts, if any, could still be turned on and
>>> off by adding or removing services that add these lines?
>>
>> It is better than nothing, but it is not sufficient IMO.  Any part of
>> /etc/profile can be controversial (you'll never know what a user would
>> like to change), so I think providing an option to change this file
>> completely is essential.
> To be clear, /etc/profile contains 3 parts:
>
>  1. variables from configuration of the operating-system (LANG, TZ,
> etc.)
>  2. environment setup for system and user profiles
>     (source .guix-profile/etc/profile)
>  3. hacks for making sensible defaults (LINUX_MODULE_DIRECTORY,
>     ASPELL_CONF, etc).
>
> And it's only effective for POSIX login shells (bash and zsh).
>
> For 1, maybe the most important one, it's already managed, but doesn't
> work for fish and rc.  We need to move these into /etc/environment,
> which work for all shells (even emacs? :-)

I didn't know about /etc/environment.  So IIUC it is used for VAR=VALUE
pairs, right?  If so and if it is supported by all shells (I don't see a
mention of it in the bash manual though), I agree with you to move these
things, great idea!

> For 2, we had build a etc/profile file for each profile's search-paths,
> here source both system and user to make most things work
> out-of-the-box.
>
> I think this is the real purpose for our /etc/profile.
> Technical, if we remove those, the result system will be the same as
> guix on foreign distros.  So, it's ok to completely replace it.
>
> (some variables (eg: MANPATH, INFOPATH, XDG_DATA_DIRS) can be set in
> each profile, and mergerd well).

IIUC invoking "guix package --search-paths" on both system and user
profiles sets all required environment variables, so sourcing
/run/current-system/profile/etc/profile and ~/.guix-profile/etc/profile
is not needed, right?

> And 3, IMO is the controversial parts.
>
> the one don't related to profiles can go into /etc/environment
> (eg: LINUX_MODULE_DIRECTORY, SSL_CERT_DIR, DBUS_FATAL_WARNINGS),
> these need to be addressing by adding services?

I agree that it's better to put plain VAR=VAL to /etc/environment.

> and others may go into profile (eg: ASPELL_CONF, GST_PLUGIN_PATH).

Yes.  And this is another example of the thing I want to change: I don't
like to have any mention of "$HOME/.guix-profile" in /etc/profile, so I
would remove these things it if had a chance.

> So, the plan is add /etc/environment and only use /etc/profile for 2.
> then, a sh-profile file-like configuration can be added.  WDYT?

I like the idea of separating /etc/environment and /etc/profile, but my
main concern is to have a possibility to change /etc files the way I
want, as I explained in the reply to Ludovic.

-- 
Alex

Customizing /etc

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> 宋文武 (2015-11-24 18:22 +0300) wrote:

[...]

>> So, the plan is add /etc/environment and only use /etc/profile for 2.
>> then, a sh-profile file-like configuration can be added.  WDYT?
>
> I like the idea of separating /etc/environment and /etc/profile, but my
> main concern is to have a possibility to change /etc files the way I
> want, as I explained in the reply to Ludovic.

I agree that specifying what goes into /etc is something we want to
allow (though not directly related to the /etc/profile issue.)

What about exposing the name/file-like pairs that are passed to
‘etc-service’?  That way, one could write:

  (define os
    (operating-system
      ;; …
      (etc-files `(("hosts" ,(local-file "my-hosts-file"))
                   ("issue" ,(plain-file "Hello!\n"))
                   ("sudoers" ,(local-file "sudoers"))
                   ("profile" ,(local-file "myprofile"))
                   ,@(fold alist-delete
                           (default-etc-files os)
                           '("hosts" "issue" "sudoers" "profile"))))))

We may remove the ‘hosts-file’ and ‘sudoers-file’ fields, but keep
higher-level things like ‘name-service-switch’ because they’re more
convenient.

The difficulty is that some of the default files, such as /etc/hosts,
are generated as a function of the ‘operating-system’ declaration.  Thus
I think we need ‘default-etc-files’ to be a procedure as shown above,
and the ‘etc-files’ field must be thunked or delayed.  Hmm not fully
sure this is the right interface.

WDYT?

The bottom line is that /etc is not a great configuration interface
because it’s all flat and GuixSD has no idea of the meaning of those
files and their relationship.  So the preferred approach remains
configuration via services and high-level configuration objects.

Thanks,
Ludo’.

Re: Customizing /etc

[Alex Kost]

Ludovic Courtès (2015-11-27 17:58 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
[...]
>> I like the idea of separating /etc/environment and /etc/profile, but my
>> main concern is to have a possibility to change /etc files the way I
>> want, as I explained in the reply to Ludovic.
>
> I agree that specifying what goes into /etc is something we want to
> allow (though not directly related to the /etc/profile issue.)

Oof, that's a relief for me!  I had an impression that you are against
giving a user a full control over /etc files.

> What about exposing the name/file-like pairs that are passed to
> ‘etc-service’?  That way, one could write:
>
>   (define os
>     (operating-system
>       ;; …
>       (etc-files `(("hosts" ,(local-file "my-hosts-file"))
>                    ("issue" ,(plain-file "Hello!\n"))
>                    ("sudoers" ,(local-file "sudoers"))
>                    ("profile" ,(local-file "myprofile"))
>                    ,@(fold alist-delete
>                            (default-etc-files os)
>                            '("hosts" "issue" "sudoers" "profile"))))))

Yes, changing /etc files is exactly what I want!

> We may remove the ‘hosts-file’ and ‘sudoers-file’ fields, but keep
> higher-level things like ‘name-service-switch’ because they’re more
> convenient.

Yes, I agree; if this will be accepted, keeping '<foo>-file' fields (for
hosts, sudoers and future files) is probably not needed.

> The difficulty is that some of the default files, such as /etc/hosts,
> are generated as a function of the ‘operating-system’ declaration.  Thus
> I think we need ‘default-etc-files’ to be a procedure as shown above,
> and the ‘etc-files’ field must be thunked or delayed.  Hmm not fully
> sure this is the right interface.
>
> WDYT?

Yes, this will probably not be an easy-to-use interface, but to have it
is better than to have nothing.  So I am totally for it!

> The bottom line is that /etc is not a great configuration interface
> because it’s all flat and GuixSD has no idea of the meaning of those
> files and their relationship.  So the preferred approach remains
> configuration via services and high-level configuration objects.

Yes, I agree; but if a user is not satisfied by the result of these
high-level services, I think (s)he should have a fallback way to
change/override the resulting /etc file.

-- 
Alex