bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Thompson, David]

[-- Attachment #1: Type: text/plain, Size: 951 bytes --]

Hi,

When 'guix shell' is run without arguments, there is some convenient
default logic applied to check for a manifest.scm or guix.scm file and do
the right thing with it. However, using -- to override the default command
like 'guix shell -- make' doesn't do the same thing. I expect that it would
still automagically apply manifest.scm or guix.scm but just run the
specified command instead of spawning a shell.  Instead, 'guix shell'
outputs this warning letting me know that something isn't right:

    guix shell: warning: no packages specified; creating an empty
environment

On one hand: Sure, I *did* pass arguments (though not flags.) On the other
hand: I think this is a bad user experience. I doubt I'm alone in expecting
the only difference between 'guix shell' and 'guix shell -- make' to be
that 'make' is run instead of a shell.  I can implement this if there's
some indication that such a patch would be acceptable.

Thoughts?

- Dave

[-- Attachment #2: Type: text/html, Size: 1181 bytes --]

bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Thompson, David]

[-- Attachment #1.1: Type: text/plain, Size: 279 bytes --]

Hi again,

I decided to just implement the fix and see what people think of it.
Simply removing a check for non-interactive invocation solves the issue and
now 'guix shell' and 'guix shell -- make' act exactly the same except for
which command they run.  Patch attached.

- Dave

[-- Attachment #1.2: Type: text/html, Size: 383 bytes --]

[-- Attachment #2: 0001-shell-Look-for-manifest.scm-guix.scm-in-non-interact.patch --]
[-- Type: text/x-patch, Size: 1346 bytes --]

From f2b8d4a9da5a9df0aef0e9da71a62fd9d285e994 Mon Sep 17 00:00:00 2001
From: David Thompson <dthompson2@worcester.edu>
Date: Sun, 28 Aug 2022 21:21:09 -0400
Subject: [PATCH] shell: Look for manifest.scm/guix.scm in non-interactive
 case, too.

Fixes <https://issues.guix.gnu.org/57467>.

Fixes a bug where a command like 'guix shell -- make' does not look for
guix.scm or manifest like 'guix shell' with no additional arguments does.

* guix/scripts/shell.scm (auto-detect-manifest): Remove check for
non-interactive invocation that was stopping implicit loading.
---
 guix/scripts/shell.scm | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm
index c115a00320..0e9ab0dd94 100644
--- a/guix/scripts/shell.scm
+++ b/guix/scripts/shell.scm
@@ -260,14 +260,10 @@ (define (options-contain-payload? opts)
       ((('expression . _) . _) #t)
       ((_ . rest) (options-contain-payload? rest))))
 
-  (define interactive?
-    (not (assoc-ref opts 'exec)))
-
   (define disallow-implicit-load?
     (assoc-ref opts 'explicit-loading?))
 
-  (if (or (not interactive?)
-          disallow-implicit-load?
+  (if (or disallow-implicit-load?
           (options-contain-payload? opts))
       opts
       (match (find-file-in-parent-directories '("manifest.scm" "guix.scm"))
-- 
2.37.2

bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 732 bytes --]


On 29-08-2022 03:28, Thompson, David wrote:
> Hi again,
>
> I decided to just implement the fix and see what people think of it.  
> Simply removing a check for non-interactive invocation solves the 
> issue and now 'guix shell' and 'guix shell -- make' act exactly the 
> same except for which command they run.  Patch attached.
>
The interactive check is a feature, not a bug:

> https://issues.guix.gnu.org/50960#69:
> [...]
> Agreed. The automatic reading of guix.scm/manifest.scm, if we keep it,
> should only happen in interactive use; I’ll double-check and make sure
> this is the case.
It might still be possible to solve 57467, but I don't think this patch 
is the solution.

Greetings,
Maxime.


[-- Attachment #1.1.1.2: Type: text/html, Size: 1495 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#57467: [EXT] Re: bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Thompson, David]

[-- Attachment #1: Type: text/plain, Size: 1071 bytes --]

Hi Maxime,

On Mon, Aug 29, 2022 at 6:29 AM Maxime Devos <maximedevos@telenet.be> wrote:

> On 29-08-2022 03:28, Thompson, David wrote:
>
> Hi again,
>
> I decided to just implement the fix and see what people think of it.
> Simply removing a check for non-interactive invocation solves the issue and
> now 'guix shell' and 'guix shell -- make' act exactly the same except for
> which command they run.  Patch attached.
>
> The interactive check is a feature, not a bug:
>
Could you please explain why it's a feature? I've provided an example that
shows how it is confusing and unexpected.

> https://issues.guix.gnu.org/50960#69:
> [...]
> Agreed. The automatic reading of guix.scm/manifest.scm, if we keep it,
> should only happen in interactive use; I’ll double-check and make sure
> this is the case.
>
> It might still be possible to solve 57467, but I don't think this patch is
> the solution.
>

Could you propose an alternate solution?  What are the next steps here?
Right now all I know is that you don't like my patch.

- Dave

[-- Attachment #2: Type: text/html, Size: 2074 bytes --]

bug#57467: [EXT] Re: bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1684 bytes --]


On 29-08-2022 14:48, Thompson, David wrote:
> Hi Maxime,
>
> On Mon, Aug 29, 2022 at 6:29 AM Maxime Devos <maximedevos@telenet.be> 
> wrote:
>
>     On 29-08-2022 03:28, Thompson, David wrote:
>>     Hi again,
>>
>>     I decided to just implement the fix and see what people think of
>>     it.  Simply removing a check for non-interactive invocation
>>     solves the issue and now 'guix shell' and 'guix shell -- make'
>>     act exactly the same except for which command they run.  Patch
>>     attached.
>>
>     The interactive check is a feature, not a bug:
>
> Could you please explain why it's a feature?
The quoted text was my explanation. Maybe that thread has more 
information, or failing that, maybe the person I quoted knows why.
> I've provided an example that shows how it is confusing and unexpected.
Your example was "guix shell -- ...", not interactive checks in general.
>
>>     https://issues.guix.gnu.org/50960#69:
>>     [...]
>>     Agreed. The automatic reading of guix.scm/manifest.scm, if we
>>     keep it,
>>     should only happen in interactive use; I’ll double-check and make
>>     sure
>>     this is the case.
>     It might still be possible to solve 57467, but I don't think this
>     patch is the solution.
>
>
> Could you propose an alternate solution?  What are the next steps 
> here?  Right now all I know is that you don't like my patch.

Possibly, but try proposing an alternate solution yourself first. And 
you know more than that, you know that the interactive check shouldn't 
be simply removed and have a link to a discussion that may have more 
information.

Greetings,
Maxime


[-- Attachment #1.1.1.2: Type: text/html, Size: 4466 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#57467: [EXT] Re: [EXT] Re: bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Thompson, David]

[-- Attachment #1: Type: text/plain, Size: 1965 bytes --]

On Tue, Aug 30, 2022 at 9:24 AM Maxime Devos <maximedevos@telenet.be> wrote:

> On 29-08-2022 14:48, Thompson, David wrote:
>
> Hi Maxime,
>
> On Mon, Aug 29, 2022 at 6:29 AM Maxime Devos <maximedevos@telenet.be>
> wrote:
>
>> On 29-08-2022 03:28, Thompson, David wrote:
>>
>> Hi again,
>>
>> I decided to just implement the fix and see what people think of it.
>> Simply removing a check for non-interactive invocation solves the issue and
>> now 'guix shell' and 'guix shell -- make' act exactly the same except for
>> which command they run.  Patch attached.
>>
>> The interactive check is a feature, not a bug:
>>
> Could you please explain why it's a feature?
>
> The quoted text was my explanation. Maybe that thread has more
> information, or failing that, maybe the person I quoted knows why.
>

Oops, I missed this. It looked like a quoted section of my original email
so I totally missed it and was confused.  My bad.

> I've provided an example that shows how it is confusing and unexpected.
>
> Your example was "guix shell -- ...", not interactive checks in general.
>
> https://issues.guix.gnu.org/50960#69:
>> [...]
>> Agreed. The automatic reading of guix.scm/manifest.scm, if we keep it,
>> should only happen in interactive use; I’ll double-check and make sure
>> this is the case.
>>
>> It might still be possible to solve 57467, but I don't think this patch
>> is the solution.
>>
>
> Could you propose an alternate solution?  What are the next steps here?
> Right now all I know is that you don't like my patch.
>
> Possibly, but try proposing an alternate solution yourself first. And you
> know more than that, you know that the interactive check shouldn't be
> simply removed and have a link to a discussion that may have more
> information.
>
 The hostility here and in the other issue where you are applying stop
energy to my work is less than appreciated.  Please stop it.

- Dave

[-- Attachment #2: Type: text/html, Size: 4613 bytes --]

bug#57467: [EXT] Re: [EXT] Re: bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 557 bytes --]

Hi David,

Thompson, David 写道：
> The hostility here and in the other issue where you are applying 
> stop energy to my work is less than appreciated.

Some healthy ‘stop energy’ was needed here, and in bug #56444.

Please spend that energy on fleshing out requirements and 
improving the patches if needed.  Maxime's review is a good start.

Shopping around for a ‘core dev’ to fast-track these patches 
disrespects the work Maxime has already put in, and is not how 
things are done.  It won't happen.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

bug#57467: [EXT] Re: bug#57467: [EXT] Re: [EXT] Re: bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Thompson, David]

[-- Attachment #1: Type: text/plain, Size: 1636 bytes --]

Hi Tobias,

On Tue, Aug 30, 2022 at 11:01 AM Tobias Geerinckx-Rice <me@tobias.gr> wrote:

> Hi David,
>
> Thompson, David 写道：
> > The hostility here and in the other issue where you are applying
> > stop energy to my work is less than appreciated.
>
> Some healthy ‘stop energy’ was needed here, and in bug #56444.
>

It made me feel like my effort was not appreciated or wanted, so I don't
think it was healthy.

In this case I made a mistake and I'm sorry.  I thought Maxime hadn't
provided any context for saying something was a feature and not a bug,
which is why I asked for clarification and direction so I wasn't at a dead
end. I thought I was being dismissed. I tried to find a justification in
the source code and couldn't find it, so I thought it couldn't hurt to just
submit a patch and figure out what the issues might be.  It was a
misunderstanding and I'd like to reset and return to talking through the
technical issues of this bug report.


> Please spend that energy on fleshing out requirements and
> improving the patches if needed.  Maxime's review is a good start.
>

Yes, I agree now that I have the context that I missed the first time.


> Shopping around for a ‘core dev’ to fast-track these patches
> disrespects the work Maxime has already put in, and is not how
> things are done.  It won't happen.
>

 I am not trying to fast track anything, and certainly not the patch here.
I am going to write a separate message to go over the competing desires for
'guix shell' that make my proposed behavior change potentially
controversial.

Thanks,

- Dave

[-- Attachment #2: Type: text/html, Size: 2422 bytes --]

bug#57467: [EXT] bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Thompson, David]

[-- Attachment #1: Type: text/plain, Size: 1564 bytes --]

So there are some competing expectations here. The status quo is that
non-interactive invocations of 'guix shell' will not load a guix.scm or
manifest.scm file unless explicitly told to via --file/--manifest following
the "explicit is better than implicit" philosophy.  People like myself who
almost exclusively invoke 'guix shell' without any arguments would expect
that 'guix shell -- make' would load their guix.scm file like they're used
to.  It is the latter expectation that is typically the status quo in other
language environments.  For example, in Ruby land, 'bundle exec foo' runs
the command 'foo' non-interactively in the context of a project and you
don't have to tell it where to find the conventional Gemfile because it's
the default.  Treating interactive and non-interactive invocation
differently in 'guix shell' was a surprise for me.  Of course, 'guix shell
-D -f guix.scm -- make' works, but it doesn't "just work."  Both viewpoints
have their merits and I'm wondering if there's a way to satisfy both
expectations.  Perhaps 'guix shell -- make' would load guix.scm, but 'guix
shell --pure -- make', or any invocation with any flags, would not?  Or,
less ideal, a new short flag that can be passed that would be the
equivalent of `-D -f guix.scm` (or the manifest.scm variant) to at least
make for less typing.  Personally, I think that defaulting to loading from
a conventional file when no packages are specified is much more user
friendly than generating an empty profile, regardless of interactive vs.
non-interactive.

Thoughts?

- Dave

[-- Attachment #2: Type: text/html, Size: 1778 bytes --]

bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Ludovic Courtès]

Hi David,

Thanks for your feedback on this.

"Thompson, David" <dthompson2@worcester.edu> skribis:

> So there are some competing expectations here. The status quo is that
> non-interactive invocations of 'guix shell' will not load a guix.scm or
> manifest.scm file unless explicitly told to via --file/--manifest following
> the "explicit is better than implicit" philosophy.  People like myself who
> almost exclusively invoke 'guix shell' without any arguments would expect
> that 'guix shell -- make' would load their guix.scm file like they're used
> to.  It is the latter expectation that is typically the status quo in other
> language environments.  For example, in Ruby land, 'bundle exec foo' runs
> the command 'foo' non-interactively in the context of a project and you
> don't have to tell it where to find the conventional Gemfile because it's
> the default.  Treating interactive and non-interactive invocation
> differently in 'guix shell' was a surprise for me.  Of course, 'guix shell
> -D -f guix.scm -- make' works, but it doesn't "just work."

Right.  As you note, there were different expectations and constraints
when we discussed this.  We ended up with a tradeoff that has its pros
and cons.  The whole discussion took place here:

  https://issues.guix.gnu.org/50960

As a side note, I think as a project we may want to set up an RFC kind
of process to have a documented way to make decisions like these.  This
would ensure every person who wants to chime in has an opportunity to do
so, even people who would not follow implementation discussions or
follow the patch stream on guix-patches.  Food for thought!

> Both viewpoints have their merits and I'm wondering if there's a way
> to satisfy both expectations.  Perhaps 'guix shell -- make' would load
> guix.scm, but 'guix shell --pure -- make', or any invocation with any
> flags, would not?  Or, less ideal, a new short flag that can be passed
> that would be the equivalent of `-D -f guix.scm` (or the manifest.scm
> variant) to at least make for less typing.  Personally, I think that
> defaulting to loading from a conventional file when no packages are
> specified is much more user friendly than generating an empty profile,
> regardless of interactive vs.  non-interactive.

The main reason to me for distinguishing non-interactive behavior is
ensuring that scripts behave in a deterministic fashion, as opposed to
having their behavior dependent on the presence of a file in $PWD.

FWIW, I’m rather satisfied with the current behavior, though I’m open to
other opinions (and indeed, feedback from users of other tools in this
area is much welcome).

The main difficulty here is that, should we eventually decide to change
behaviors, we’ll have to devise a migration timeline etc.  (As an
example, we chose to keep ‘guix environment’ until at least May 2023;
all this must take time if we want to avoid breaking user workflows.)

Thoughts?

Thanks,
Ludo’.

bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Thompson, David]

Hi Ludo,

On Mon, Sep 5, 2022 at 9:06 AM Ludovic Courtès <ludo@gnu.org> wrote:
>
> Hi David,
>
> Thanks for your feedback on this.
>
> "Thompson, David" <dthompson2@worcester.edu> skribis:
>
> > So there are some competing expectations here. The status quo is that
> > non-interactive invocations of 'guix shell' will not load a guix.scm or
> > manifest.scm file unless explicitly told to via --file/--manifest following
> > the "explicit is better than implicit" philosophy.  People like myself who
> > almost exclusively invoke 'guix shell' without any arguments would expect
> > that 'guix shell -- make' would load their guix.scm file like they're used
> > to.  It is the latter expectation that is typically the status quo in other
> > language environments.  For example, in Ruby land, 'bundle exec foo' runs
> > the command 'foo' non-interactively in the context of a project and you
> > don't have to tell it where to find the conventional Gemfile because it's
> > the default.  Treating interactive and non-interactive invocation
> > differently in 'guix shell' was a surprise for me.  Of course, 'guix shell
> > -D -f guix.scm -- make' works, but it doesn't "just work."
>
> Right.  As you note, there were different expectations and constraints
> when we discussed this.  We ended up with a tradeoff that has its pros
> and cons.  The whole discussion took place here:
>
>   https://issues.guix.gnu.org/50960
>
> As a side note, I think as a project we may want to set up an RFC kind
> of process to have a documented way to make decisions like these.  This
> would ensure every person who wants to chime in has an opportunity to do
> so, even people who would not follow implementation discussions or
> follow the patch stream on guix-patches.  Food for thought!

Yeah, that could be a good future improvement.  For what it's worth,
though, I do think you gave plenty of opportunity for commenting when
you posted that patch set.  It was just bad timing for me and I wasn't
able to participate in the discussion much.  It's all good, I think
'guix shell' is a worthy successor to 'guix environment' as it has
better UX and performance.  After all, it implements what I wanted
from 'guix environment' all those years ago where the --ad-hoc
behavior is the default.

> > Both viewpoints have their merits and I'm wondering if there's a way
> > to satisfy both expectations.  Perhaps 'guix shell -- make' would load
> > guix.scm, but 'guix shell --pure -- make', or any invocation with any
> > flags, would not?  Or, less ideal, a new short flag that can be passed
> > that would be the equivalent of `-D -f guix.scm` (or the manifest.scm
> > variant) to at least make for less typing.  Personally, I think that
> > defaulting to loading from a conventional file when no packages are
> > specified is much more user friendly than generating an empty profile,
> > regardless of interactive vs.  non-interactive.
>
> The main reason to me for distinguishing non-interactive behavior is
> ensuring that scripts behave in a deterministic fashion, as opposed to
> having their behavior dependent on the presence of a file in $PWD.
>
> FWIW, I’m rather satisfied with the current behavior, though I’m open to
> other opinions (and indeed, feedback from users of other tools in this
> area is much welcome).
>
> The main difficulty here is that, should we eventually decide to change
> behaviors, we’ll have to devise a migration timeline etc.  (As an
> example, we chose to keep ‘guix environment’ until at least May 2023;
> all this must take time if we want to avoid breaking user workflows.)

It seems that I'm all alone in wanting this and it would be far too
annoying to go through a deprecation process for a change this small,
anyway.  I'm going to close this bug.  Thanks for the additional
context.

- Dave

bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 797 bytes --]


On 05-09-2022 15:06, Ludovic Courtès wrote:
> The main difficulty here is that, should we eventually decide to change
> behaviors, we’ll have to devise a migration timeline etc.  (As an
> example, we chose to keep ‘guix environment’ until at least May 2023;
> all this must take time if we want to avoid breaking user workflows.)
>
> Thoughts?

"guix shell" is for making packages available in the environment. 
Currently, "guix shell -- foobar" does not make any packages available 
-- it's effectively a no-op except for setting GUIX_ENVIRONMENT. As 
such, I expect nobody is actually relying on "guix shell -- foobar" to 
not load guix.scm or manifest.scm and I think that if we go for this 
change, the migration timeline can be rather minimal.

Greetings,
Maxime.


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Ludovic Courtès]

Maxime Devos <maximedevos@telenet.be> skribis:

> On 05-09-2022 15:06, Ludovic Courtès wrote:
>> The main difficulty here is that, should we eventually decide to change
>> behaviors, we’ll have to devise a migration timeline etc.  (As an
>> example, we chose to keep ‘guix environment’ until at least May 2023;
>> all this must take time if we want to avoid breaking user workflows.)
>>
>> Thoughts?
>
> "guix shell" is for making packages available in the
> environment. Currently, "guix shell -- foobar" does not make any
> packages available -- it's effectively a no-op except for setting
> GUIX_ENVIRONMENT.

True, though you could always have scripts that read:

  guix shell $packages -- whatever

and that will suddenly behave differently if $packages expands to an
empty string.  Tricky!

Ludo’.

bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 530 bytes --]


On 06-09-2022 09:18, Ludovic Courtès wrote:
>> "guix shell" is for making packages available in the
>> environment. Currently, "guix shell -- foobar" does not make any
>> packages available -- it's effectively a no-op except for setting
>> GUIX_ENVIRONMENT.
> True, though you could always have scripts that read:
>
>    guix shell $packages -- whatever
>
> and that will suddenly behave differently if $packages expands to an
> empty string.  Tricky!

Right, I didn't think of such uses.

Greetings,
Maxime.


[-- Attachment #1.1.1.2: Type: text/html, Size: 1012 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#57467: 'guix shell' does not honor default behavior when given a specific command to run

[Thompson, David]