bug#56076: guix shell -P behavior is surprising with persistent home

bug#56076: guix shell -P behavior is surprising with persistent home

[Ethan Blanton via Bug reports for GNU Guix]

I am using guix shell to create an isolated container, but using a
persistent home directory to preserve configuration and state for the
program in the container.  Specifically, I am using (lightly
simplified; note that the user is elb and the manifest contains ONLY
syncthing):

CONTAINER_HOME=/path/to/persistent/home
MANIFEST=/path/to/manifest.scm

guix shell --container --network --no-cwd -P      \
           --share=$CONTAINER_HOME=/home/elb --   \
           /bin/sh -c 'SSL_CERT_DIR="$HOME/.guix-profile/etc/ssl/certs" syncthing'

Because syncthing requires a certificate store, the easiest way to
configure that seemed to be through the profile /etc dir, which meant
using -P.  The above command works, and works correctly, but only the
first time the profile is started.

On subsequent starts, guix complains that:

guix shell: error: cannot link profile: '/home/elb/.guix-profile' already exists within container

This is easily worked around by removing .guix-profile from the
--share-bound home directory before invoking guix shell.

It is not clear to me that this is a bug, but it was surprising.  It
was also surprising that there was not an obvious way to simply
declare a persistent home directory for a container, although now that
I understand the `guix shell` command better, I find this less
surprising than I did when I was first exploring.

Ethan

bug#56076: guix shell -P behavior is surprising with persistent home

[Ludovic Courtès]

Hi Ethan,

Ethan Blanton <elb@kb8ojh.net> skribis:

> I am using guix shell to create an isolated container, but using a
> persistent home directory to preserve configuration and state for the
> program in the container.  Specifically, I am using (lightly
> simplified; note that the user is elb and the manifest contains ONLY
> syncthing):
>
> CONTAINER_HOME=/path/to/persistent/home
> MANIFEST=/path/to/manifest.scm
>
> guix shell --container --network --no-cwd -P      \
>            --share=$CONTAINER_HOME=/home/elb --   \
>            /bin/sh -c 'SSL_CERT_DIR="$HOME/.guix-profile/etc/ssl/certs" syncthing'
>
> Because syncthing requires a certificate store, the easiest way to
> configure that seemed to be through the profile /etc dir, which meant
> using -P.  The above command works, and works correctly, but only the
> first time the profile is started.

I guess it’s confusing because ‘-P’ and ‘--share’ kind of step on each
other’s toes: they both want to control /home/elb.

> On subsequent starts, guix complains that:
>
> guix shell: error: cannot link profile: '/home/elb/.guix-profile' already exists within container
>
> This is easily worked around by removing .guix-profile from the
> --share-bound home directory before invoking guix shell.
>
> It is not clear to me that this is a bug, but it was surprising.  It
> was also surprising that there was not an obvious way to simply
> declare a persistent home directory for a container, although now that
> I understand the `guix shell` command better, I find this less
> surprising than I did when I was first exploring.

Yeah, I’m not sure how to better handle that; the two options are
conflicting.

That said, for this particular use case, you could do:

  guix shell syncthing nss-certs openssl -- syncthing

Adding ‘openssl’ to the mix is a trick to ensure that SSL_CERT_DIR is
defined, thanks to the search path mechanism:

  https://guix.gnu.org/manual/devel/en/html_node/Search-Paths.html

Thoughts?

Ludo’.

bug#56076: guix shell -P behavior is surprising with persistent home

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 567 bytes --]

Ludovic Courtès schreef op vr 24-06-2022 om 23:10 [+0200]:
> That said, for this particular use case, you could do:
> 
>   guix shell syncthing nss-certs openssl -- syncthing
> 
> Adding ‘openssl’ to the mix is a trick to ensure that SSL_CERT_DIR is
> defined, thanks to the search path mechanism:
> 
>   https://guix.gnu.org/manual/devel/en/html_node/Search-Paths.html
> 
> Thoughts?

Having to add openssl looks like a bug, so I'd go for something like
<https://issues.guix.gnu.org/55297> instead, which resolves that.

Greetings,
Maxime.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#56076: guix shell -P behavior is surprising with persistent home

[Ludovic Courtès]

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> Ludovic Courtès schreef op vr 24-06-2022 om 23:10 [+0200]:
>> That said, for this particular use case, you could do:
>> 
>>   guix shell syncthing nss-certs openssl -- syncthing
>> 
>> Adding ‘openssl’ to the mix is a trick to ensure that SSL_CERT_DIR is
>> defined, thanks to the search path mechanism:
>> 
>>   https://guix.gnu.org/manual/devel/en/html_node/Search-Paths.html
>> 
>> Thoughts?
>
> Having to add openssl looks like a bug,

It is, but it’s a reasonable solution for the problem at hand.

> so I'd go for something like <https://issues.guix.gnu.org/55297>
> instead, which resolves that.

That’s a bit of a hack to work around this venerable bug:

  https://issues.guix.gnu.org/22138

:-)

Ludo’.

bug#56076: guix shell -P behavior is surprising with persistent home

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2221 bytes --]

> > Having to add openssl looks like a bug,
> 
> It is, but it’s a reasonable solution for the problem at hand.

I don't think it is.  The point of a package manager or distribution is
to automatically install dependencies where required -- in case of
Guix, dependencies can be considered in a more general sense to include
search paths and not only store items.  Having to search #guix or the
ML for work-arounds defeats the point.

My patch <https://issues.guix.gnu.org/55297> could be considered a
work-around, but at least it's automatic for the users, users don't
need to remember random Guix bug facts.

Ludovic Courtès schreef op ma 27-06-2022 om 09:55 [+0200]:
> That’s a bit of a hack to work around this venerable bug:
> 
>   https://issues.guix.gnu.org/22138
> 
> :-)

Maybe yes, but there's no progress there, so work-arounds are needed,
and I consider my <https://issues.guix.gnu.org/55297> to be less of a
hack than ‘add random packages’ (at least 55297 is automatic for the
user, the user doesn't have to remember random packages).  And even if
we do 55297 now, it's still possible to do 22138 later if deemed
proper.

And the patch there is fragile (e.g., if you have gcc:lib in the
references, you automatically get $LIBRARY_PATH TOO, if you have a
cross-gcc:lib, you'd get a $CROSS_LIBRARY_PATH (untested), it's also
not really lexical (+) -- wrap-program boundaries aren't respected.  It
could also easily go wrong in case of static linking (because then
references disappear).  So I consider 22138 not be be a bug/working
patch but a source of more bugs and 55297 to be a proper fix (*).

(*) Except for $GUIX_LOCPATH, but that could be made magical in the
same sense of $PATH.

(+) E.g., suppose that pkg-config and a cross-pkg-config are somewhere
in the references for whatever reason, but only a single one is in the
(propagated) native-inputs.  They both ahave a $PKG_CONFIG_PATH search
path.  However, they are different search paths (one is native-inputs,
other is inputs), so we would get in trouble when "guix shell -D
package" is extended with a --target option or such to make a cross-
compilation environment.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#56076: guix shell -P behavior is surprising with persistent home

[Ludovic Courtès]

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

>> > Having to add openssl looks like a bug,
>> 
>> It is, but it’s a reasonable solution for the problem at hand.
>
> I don't think it is.

On another distro, you might have to set SSL_CERT_DIR manually.  It’s
not that big of a deal, or at least not really worse.

[...]

> My patch <https://issues.guix.gnu.org/55297> could be considered a
> work-around, but at least it's automatic for the users, users don't
> need to remember random Guix bug facts.

If we start adding such workarounds, where do we stop?  Someone will
want to have similar workarounds for TERMINFO_DIRS; do we add that to
each and every package that uses ncurses?  What about GST_PLUGIN_PATH?

So far, we’ve addressed those by special-casing a few key variables in
/etc/profile on Guix System and similarly in the file generated by the
install script on foreign distros.

Maybe we could do the same with SSL_CERT_DIR?

> Ludovic Courtès schreef op ma 27-06-2022 om 09:55 [+0200]:
>> That’s a bit of a hack to work around this venerable bug:
>> 
>>   https://issues.guix.gnu.org/22138
>> 
>> :-)
>
> Maybe yes, but there's no progress there

True, but that doesn’t mean it has to stay that way.

I understand your frustration (I really do!) and I agree that sometimes
we need workarounds when there’s no “proper” solution in sight.  Our
disagreement is about the kind of workaround in this case; I’m reluctant
to adding search path definitions to users of a library that defines it.

Thanks,
Ludo’.

bug#56076: guix shell -P behavior is surprising with persistent home

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1521 bytes --]

> On another distro, you might have to set SSL_CERT_DIR manually.  It’s
> not that big of a deal, or at least not really worse.

On other distros, that's sufficient because they only have global
installation, in /etc/ssl/certs. So I don't think it's a good
comparison, given that Guix is all about user-level and occassionally
ad-hoc "guix shell".

Ludovic Courtès schreef op di 28-06-2022 om 09:43 [+0200]:
> > My patch <https://issues.guix.gnu.org/55297> could be considered a
> > work-around, but at least it's automatic for the users, users don't
> > need to remember random Guix bug facts.
> 
> If we start adding such workarounds, where do we stop?  Someone will
> want to have similar workarounds for TERMINFO_DIRS; do we add that to
> each and every package that uses ncurses?  What about
> GST_PLUGIN_PATH?

Sounds good.

> So far, we’ve addressed those by special-casing

That seems equally workaround to me?  Though way less work so sounds
good, except for it being rejected in the past.

> a few key variables in /etc/profile on Guix System and similarly in
> the file generated by the install script on foreign distros.
> 
> Maybe we could do the same with SSL_CERT_DIR?

Shouldn't this be $GUIX_ENVIRONMENT/etc/profile instead of
/etc/profile?  Otherwise, it wouldn't have any effect on "guix shell".
Also, I believe I have proposed making some other variables behave like
$PATH as well, but you rejected it for being special-cased and magic.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#56076: guix shell -P behavior is surprising with persistent home

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 649 bytes --]

Ludovic Courtès schreef op di 28-06-2022 om 09:43 [+0200]:
> Hi,
> 
> Maxime Devos <maximedevos@telenet.be> skribis:
> 
> > > > Having to add openssl looks like a bug,
> > > 
> > > It is, but it’s a reasonable solution for the problem at hand.
> > 
> > I don't think it is.
> 
> On another distro, you might have to set SSL_CERT_DIR manually.  It’s
> not that big of a deal, or at least not really worse.

It's not a huge deal, but it could be even better: no deal at all,
automatically!  That's what computers are for.  Also, we don't have to
limit ourself to ‘Guix: not worse than other distros’.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]