bug#40549: [usability] revert last generation

bug#40549: [usability] revert last generation

[Tom Zander via Bug reports for GNU Guix]

This could be nicer:

I just installed something and want to revert this last generation, as seen in 
`guix package -l`.

Doing a single call like `guix package -S 17 -d 18`   does not work, and I'd 
like it to work.

What it does is that it notices I'm currently on generation 18 and thus it 
can't delete that one. As such it just errors out and returns.

What I expect to happen (regardless of order of arguments) is that it switches 
to the 17th generation and then deletes generation 18.

-- 
Tom Zander

bug#40549: [usability] revert last generation

[Ludovic Courtès]

Hi,

Tom Zander <tomz@freedommail.ch> skribis:

> This could be nicer:
>
> I just installed something and want to revert this last generation, as seen in 
> `guix package -l`.
>
> Doing a single call like `guix package -S 17 -d 18`   does not work, and I'd 
> like it to work.

Oh indeed.  In practice I’d simply run “guix package --roll-back” in
such a case so I think the example above is a bit convoluted, but I
agree that it’d be nice if it behaved like you describe.

Thanks,
Ludo’.

bug#40549: More usability issues:

[Tom via Bug reports for GNU Guix]

To add detail here:

Doing `guix package -d 18 -S 17` actually works.

This gives me the impression that the order of arguments is relevant to the 
processing of them.
This sounds like a bad idea because that is quite unlike the normal gnu 
command line parsers behavior (and generally any command line parsers, unless 
in very rare cases).

The 'package' subcommand should likely always try to do things in-order. First 
Switch, reject any second Switch argument.
Then delete (which can be repeated).
Etc.

That would be much more predicable and UX friendly.


Another, similar, example is:

   guix package -l --profile=a

this doesn't work. But the most curious thing is that this does:

   guix package --list-installed --profile=a

and this too:

   guix package --profile=a -l

and this too;

   guix package -l --profile=a -l

The things that DONT work are confusingly similar to the things that do, and 
the things don't make sense (having -l twice) work just fine without errors.

As said, this is a UX gem to fix. Please take some time to make command line 
parting usable by humans!

bug#40549: More usability issues:

[zimoun]

On Thu, 23 Apr 2020 at 21:52, Tom via Bug reports for GNU Guix
<bug-guix@gnu.org> wrote:
>
> To add detail here:
>
> Doing `guix package -d 18 -S 17` actually works.
>
> This gives me the impression that the order of arguments is relevant to the
> processing of them.

It is known and cumbersome: a feature? ;-)

Other examples:

  guix package -I -A # does nothing
  guix package -A -I # list available


> Another, similar, example is:
>
>    guix package -l --profile=a
>
> this doesn't work. But the most curious thing is that this does:
>
>    guix package --list-installed --profile=a

It is worse than that.

--8<---------------cut here---------------start------------->8---
# OK
 guix package --list-generations -p /path/to/profile
 guix package --list-installed -p /path/to/profile

# KO
 guix package -l -p /path/to/profile
 guix package -I -p /path/to/profile

# OK
 guix package -p /path/to/profile -l
 guix package -p /path/to/profile -I

# KO
 guix package -l --profile=/path/to/profile

# Do nothing
 guix package -I --profile=/path/to/profile

# OK
 guix package -l --profile=/path/to/profile -l
 guix package -I --profile=/path/to/profile -I
--8<---------------cut here---------------end--------------->8---


Well, it seems there is a bug. :-)

Back to the first examples:

--8<---------------cut here---------------start------------->8---
# Do something
 guix package --list-installed -A

# Do nothing
 guix package -I -A
--8<---------------cut here---------------end--------------->8---


Thanks for reporting.

All the best,
simon

bug#40549: More usability issues:

[zimoun]

TLDR: there is no "real" bug. :-)
Just a choice to do and document it.


On Fri, 24 Apr 2020 at 10:28, zimoun <zimon.toutoune@gmail.com> wrote:

>   guix package -I -A # does nothing
>   guix package -A -I # list available

Expected.

First line, -I -A' means that '-A' is seen as an argument for '-I'.
Idem for the second line '-A -I', i.e., '-I' is seen as an argument for '-A'.

> --8<---------------cut here---------------start------------->8---> # OK
>  guix package --list-generations -p /path/to/profile
>  guix package --list-installed -p /path/to/profile
>
> # KO
>  guix package -l -p /path/to/profile
>  guix package -I -p /path/to/profile
>
> # OK
>  guix package -p /path/to/profile -l
>  guix package -p /path/to/profile -I
>
> # KO
>  guix package -l --profile=/path/to/profile
>
> # Do nothing
>  guix package -I --profile=/path/to/profile
>
> # OK
>  guix package -l --profile=/path/to/profile -l
>  guix package -I --profile=/path/to/profile -I
> --8<---------------cut here---------------end--------------->8---

All are expected too.
Same reason.  And the long option works because no argument is
provided by '=' so it fallback to the default one "".
Short options expect an argument so read the next characters as the
value or fallback to the default one "" when there is no next
character.

Fixing this will add complexity on parsing 'args' when building
'opts'.  Basically, "guix package -I -p /path/tp/profile" returns an
error because the short option '-I' expect only one argument, read
'-p' and then Guix cannot deals with the option '/path/to/profile' and
so raises an error. See the dance with 'handle-argument' and
'arg-handler'.  And "guix package -I '' -p /path/to/profile' works,
obviously.  Well, the extra quotes ('') is annoying but I am not
convince that better could be done for short options -- regardless the
order of CLI arguments.

One solution should be add short options as '-AA' or '-II' for such
cases.  But I am not convince that such "weird" combination deserves
such attention. :-)


Back to the initial report:

 (a) guix package -S 17 -d 18 # KO
 (b) guix package -d 18 -S 17 # OK

This is not the same issue than the one described previously.  Here
the culprit is 'process-actions'.  And composing "action" seems more
than legitimate (composing "query" is questionable).

Why (a) works and (b) not?  Because the command-line is transformed
into an alist.  And this alist is built reading the command-line from
right to left.  Therefore, if you are on the generation 18 and you try
to delete it, Guix raises an error which seems expected.  The second
one (b) works because first you switch and then you delete.

Well, that's said, IMHO, two options:

 1) the order of CLI does not matter;
 2) the order of CLI matters.

Well, the order of 'actions' necessary matters as it is seen with this
example: "switch and then delete" does not end in the same state than
"delete and then switch".  Welcome in the classical mess of imperative
package manager. ;-)
Therefore, I am not convinced that something should be fixed.  It
comes from the very nature of 'actions': actions is not always
commutative.  Otherwise the best is to forbid to provide several
actions with the same transaction; which seems a bad idea -- at least
for me.


However, main of us are used to read from left to right so it seems
more natural to write:

    guix package --action1 --action2  # (a)
than
    guix package --action2 --action1  # (b)

in other words, the fix should be to simply 'reverse opts' and the CLI
will read (a) instead of the current (b).  My only concern is about
backward compatibility.


My opinion based on backward compatibility argument is: let as it is
and document it in the manual.

WDYT?


All the best,
simon

bug#40549: More usability issues:

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

>> --8<---------------cut here---------------start------------->8---> # OK
>>  guix package --list-generations -p /path/to/profile
>>  guix package --list-installed -p /path/to/profile
>>
>> # KO
>>  guix package -l -p /path/to/profile
>>  guix package -I -p /path/to/profile
>>
>> # OK
>>  guix package -p /path/to/profile -l
>>  guix package -p /path/to/profile -I
>>
>> # KO
>>  guix package -l --profile=/path/to/profile
>>
>> # Do nothing
>>  guix package -I --profile=/path/to/profile
>>
>> # OK
>>  guix package -l --profile=/path/to/profile -l
>>  guix package -I --profile=/path/to/profile -I
>> --8<---------------cut here---------------end--------------->8---
>
> All are expected too.
> Same reason.  And the long option works because no argument is
> provided by '=' so it fallback to the default one "".
> Short options expect an argument so read the next characters as the
> value or fallback to the default one "" when there is no next
> character.
>
> Fixing this will add complexity on parsing 'args' when building
> 'opts'.  Basically, "guix package -I -p /path/tp/profile" returns an
> error because the short option '-I' expect only one argument, read
> '-p' and then Guix cannot deals with the option '/path/to/profile' and
> so raises an error. See the dance with 'handle-argument' and
> 'arg-handler'.  And "guix package -I '' -p /path/to/profile' works,
> obviously.  Well, the extra quotes ('') is annoying but I am not
> convince that better could be done for short options -- regardless the
> order of CLI arguments.

Nothing new here, and everything is properly documented.

I think there are option parsers that “correctly” deal with the
ambiguity that arises for instance with “-I -p foo” (is ‘-p’ the
argument to ‘-I’ or something else?).  Perhaps libc’s argp does it
right.

However (srfi srfi-37) does it as we see it now.  Fixing it would mean
implementing a different option parser.

> Why (a) works and (b) not?  Because the command-line is transformed
> into an alist.  And this alist is built reading the command-line from
> right to left.  Therefore, if you are on the generation 18 and you try
> to delete it, Guix raises an error which seems expected.  The second
> one (b) works because first you switch and then you delete.
>
> Well, that's said, IMHO, two options:
>
>  1) the order of CLI does not matter;
>  2) the order of CLI matters.
>
> Well, the order of 'actions' necessary matters as it is seen with this
> example: "switch and then delete" does not end in the same state than
> "delete and then switch".  Welcome in the classical mess of imperative
> package manager. ;-)
> Therefore, I am not convinced that something should be fixed.  It
> comes from the very nature of 'actions': actions is not always
> commutative.  Otherwise the best is to forbid to provide several
> actions with the same transaction; which seems a bad idea -- at least
> for me.

Right, but at least we could reverse the list returned by ‘args-fold’.

> However, main of us are used to read from left to right so it seems
> more natural to write:
>
>     guix package --action1 --action2  # (a)
> than
>     guix package --action2 --action1  # (b)
>
> in other words, the fix should be to simply 'reverse opts' and the CLI
> will read (a) instead of the current (b).  My only concern is about
> backward compatibility.

We’ll need to check exactly what will behave differently.  If the tests
don’t catch anything, I think we’re fine.  Most likely, we’re talking
about corner cases like ‘-S x -d y’, which probably very few people
tried.

Thanks,
Ludo’.

bug#40549: More usability issues:

[Tom Zander via Bug reports for GNU Guix]

On dinsdag 12 mei 2020 10:51:28 CEST Ludovic Courtès wrote:
> Nothing new here, and everything is properly documented.

The bugreport was not about a disconnect between documentation and the tool,

the bugreport is a usability bug which stems from the fact that the command 
line parser behaves differently from every single other commandline parser 
average people like me have ever used.

A near 100% of the command line tools on your Gnu/Linux box will behave 
differently than guix does now.

C apps using libc, python apps using their parser, even C++ apps using the Qt 
commandline classes, all are generally compatible with regards to behavior.

Only Guix is different.

> However (srfi srfi-37) does it as we see it now.  Fixing it would mean
> implementing a different option parser.

Then fix that parser. It is inconsistent with the rest of the world and as long 
as it is end-user-facing this inconsistency is a usability bug. A rather 
massive one, I might say as this is about as core to the user-interaction of 
the platform as it can get.

-- 
Tom Zander

bug#40549: More usability issues:

[zimoun]

Hi Ludo,

Sorry, I am not compliant and reorder your quotes to ease the
discussion -- from my point of view. :-)


On Tue, 12 May 2020 at 10:51, Ludovic Courtès <ludo@gnu.org> wrote:

> However (srfi srfi-37) does it as we see it now.  Fixing it would mean
> implementing a different option parser.

Yes or add a lot of complexity.
Both appears to me wrong.  Such corner cases do not deserve one or the other.


> I think there are option parsers that “correctly” deal with the
> ambiguity that arises for instance with “-I -p foo” (is ‘-p’ the
> argument to ‘-I’ or something else?).  Perhaps libc’s argp does it
> right.

I have never deeply dove into srfi-37 and 'option' but from my
understanding, it is not possible.  Somehow, the issue comes from
srfi-37 and srfi-37 should consider that if an argument starts with
dash, then it is not an argument and turn it into an option.


> Nothing new here, and everything is properly documented.

I am not sure.  The manual says, for example:

--8<---------------cut here---------------start------------->8---
‘--list-installed[=REGEXP]’
‘-I [REGEXP]’
     List the currently installed packages in the specified profile,
     with the most recently installed packages shown last.  When REGEXP
     is specified, list only installed packages whose name matches
     REGEXP.
--8<---------------cut here---------------end--------------->8---

which is somehow inaccurate.  The REGEXP is not optional for the short
option '-I'.  And that's true for all the short options with optional
argument, if I understand correctly.  For example, "guix package -d -p
/path/to/profile" fails.


Moreover, the distinction between 'action' and 'query' is already
stated so why not underline that composing actions make sense
(transaction) but composing query not?


> > However, main of us are used to read from left to right so it seems
> > more natural to write:
> >
> >     guix package --action1 --action2  # (a)
> > than
> >     guix package --action2 --action1  # (b)
> >
> > in other words, the fix should be to simply 'reverse opts' and the CLI
> > will read (a) instead of the current (b).  My only concern is about
> > backward compatibility.
>
> We’ll need to check exactly what will behave differently.  If the tests
> don’t catch anything, I think we’re fine.  Most likely, we’re talking
> about corner cases like ‘-S x -d y’, which probably very few people
> tried.

Ok, on this light, let first point the corner cases.


All the best,
simon

bug#40549: More usability issues:

[zimoun]

On Tue, 12 May 2020 at 11:55, Tom Zander via Bug reports for GNU Guix
<bug-guix@gnu.org> wrote:

> the bugreport is a usability bug which stems from the fact that the command
> line parser behaves differently from every single other commandline parser
> average people like me have ever used.
>
> A near 100% of the command line tools on your Gnu/Linux box will behave
> differently than guix does now.
>
> C apps using libc, python apps using their parser, even C++ apps using the Qt
> commandline classes, all are generally compatible with regards to behavior.
>
> Only Guix is different.

Could you provide concrete examples?  Other than "guix package"?
And other than short-option with optional argument?

bug#40549: proposal for 'process-actions'

[zimoun]

Hi again, :-)

On Tue, 12 May 2020 at 12:38, zimoun <zimon.toutoune@gmail.com> wrote:

> > We’ll need to check exactly what will behave differently.  If the tests
> > don’t catch anything, I think we’re fine.  Most likely, we’re talking
> > about corner cases like ‘-S x -d y’, which probably very few people
> > tried.
>
> Ok, on this light, let first point the corner cases.

The only corner cases are the '%actions' (roll-back,
delete-generation, switch-generation).  There are processed in
reversed order as they appear on the command-line -- because
'for-each' and 'assoc-ref'.

However, the transaction plan is always the same:
 step0 process %actions
 step1 remove
 step2 install
 step3 manifest

Therefore, I propose to split the 'for-each' on '%actions' into fixed
steps, such as the transaction always happens using this plan:

 1. roll-back
 2. switch-generation
 3. delete-generation
 4. remove
 5. install
 6 manifest

On one hand, it reduces the "power" of combining '-S', '-d' and
'--roll-back'.  On the other hand, it enforces commutativity which is
somehow what we want a transaction to be.

If yes, I can come up with a patch.


All the best,
simon

bug#40549: More usability issues:

[zimoun]

On Tue, 12 May 2020 at 10:51, Ludovic Courtès <ludo@gnu.org> wrote:

> Nothing new here, and everything is properly documented.

Using optional argument with short-option names is unusual, AFAIK.
And for sure, there is an ambiguity; as we are seeing here. :-)
However, the only mention of that is in the commentaries of srfi-37.

--8<---------------cut here---------------start------------->8---
;;; `required-arg?' and `optional-arg?' are mutually exclusive
;;; booleans and indicate whether an argument must be or may be
;;; provided.  Besides the obvious, this affects semantics of
;;; short-options, as short-options with a required or optional
;;; argument cannot be followed by other short options in the same
;;; program-arguments string, as they will be interpreted collectively
;;; as the option's argument.
--8<---------------cut here---------------end--------------->8---

http://git.savannah.gnu.org/cgit/guile.git/tree/module/srfi/srfi-37.scm#n51


Well, using short-option with optional-argument is not recommended by
POSIX, neither GNU (if I understand well)

https://pubs.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap12.html#tag_12_02
https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html


Therefore, it deserves to document it, IMHO.

bug#40549: More usability issues:

[zimoun]

Dear Tom,

On Tue, 12 May 2020 at 11:55, Tom Zander via Bug reports for GNU Guix
<bug-guix@gnu.org> wrote:

[...]

> C apps using libc, python apps using their parser, even C++ apps using the Qt
> commandline classes, all are generally compatible with regards to behavior.
>
> Only Guix is different.

Please could you indicate me command-line tools where short-option
with optional-argument is possible.
Because if there is one, I could have inspiration to know how it
resolves the ambiguity.


> > However (srfi srfi-37) does it as we see it now.  Fixing it would mean
> > implementing a different option parser.
>
> Then fix that parser. It is inconsistent with the rest of the world and as long
> as it is end-user-facing this inconsistency is a usability bug. A rather
> massive one, I might say as this is about as core to the user-interaction of
> the platform as it can get.

The parser is not inconsistent with the rest of the world.  Or please
indicate with concrete examples what is wrong.

The issue is that Guix uses a bad practise: option with
optional-argument with both short and long name.  It is a mistake to
provide the short-name for such case.


Thank you for the report.  Now all this is clearer for me and I do not
think it is a fixable bug.

All the best,
simon

bug#40549: More usability issues:

[Tom Zander via Bug reports for GNU Guix]

On dinsdag 12 mei 2020 13:35:04 CEST zimoun wrote:
> On Tue, 12 May 2020 at 11:55, Tom Zander via Bug reports for GNU Guix
> 
> <bug-guix@gnu.org> wrote:
> > the bugreport is a usability bug which stems from the fact that the
> > command
> > line parser behaves differently from every single other commandline parser
> > average people like me have ever used.
> > 
> > A near 100% of the command line tools on your Gnu/Linux box will behave
> > differently than guix does now.
> > 
> > C apps using libc, python apps using their parser, even C++ apps using the
> > Qt commandline classes, all are generally compatible with regards to
> > behavior.
> > 
> > Only Guix is different.
> 
> Could you provide concrete examples?  Other than "guix package"?
> And other than short-option with optional argument?

This report lists two items.

Indeed the short options are a good one. It is unheard of to have an alias not 
be an alias but behave differently.
It also can't be said to be documented as the --help output does not actually 
state this. I only learned this difference today :-)

The other is the ordering of arguments being parsed unpredictable.
The usecase given was the `-d 1 -S 2`  arguments  (see earlier emails for 
details).


> Please could you indicate me command-line tools where short-option
> with optional-argument is possible.
> Because if there is one, I could have inspiration to know how it
> resolves the ambiguity.

The design of the short options is that it is an alias. Identical to the 
software regardless of what the user typed.

So you get 'cut --field 1' or 'cut -f1' or 'cut -f 1' or 'cut -f=1'.
All identical.
The important part here is that each _option_ is written separately, with a 
leading dash.

When you talk about flags you can group them. `mv -vufi` is again identical to
`mv -v -u -f -i`.  But this is irrelevant to your question because, as stated, 
this is about _flags_, not option.

You asked for an example;  see `git commit -S`. From the manpage:

   -S[<keyid>], --gpg-sign[=<keyid>]


> The issue is that Guix uses a bad practise: option with optional-argument
>  with both short and long name.  It is a mistake to provide the short-name
>  for such case.

It looks like the parser could be improved by preferring to see any argument 
with leading dash as a option when it **might** be an argument.

So; if you type -`guix package -l --help` then your parser **first** finds all 
the items with leading dashes and second it tries to find out if there is an 
argument for the `-l`.
In this case I expect the help to be shown.


This is widely seen as a solution.
Users can still use items with leading dashes by using two commonly used 
tricks.
The -l=a type of construction allows the argument to be anything. Including it 
having a leading dash.

Second is the double-dash argument that stops words leading with dashes being 
parsed as options.
  For instance;   grep -- -v *
the  -v is parsed as an actual string and not an option because it follows the 
double dashes.

> Now all this is clearer for me and I do not think it is a fixable bug.

It is, just follow the suggestion from me and from zimoun: any command-line-
argument that starts with a dash should be preferred to be an option. Only in 
a second phase do you try to match anything to (optional) options.

As stated, the rest of the world does this, please check out the various 
examples I gave here to confirm that others have solved it and it may be 
possible to solve it for guix too.
-- 
Tom Zander

bug#40549: proposal for 'process-actions'

[Tom Zander via Bug reports for GNU Guix]

On dinsdag 12 mei 2020 15:03:14 CEST zimoun wrote:
> If yes, I can come up with a patch.

I think that would be much improved, yes.

-- 
Tom Zander

bug#40549: More usability issues:

[zimoun]

Dear Tom,

On Tue, 12 May 2020 at 18:23, Tom Zander <tomz@freedommail.ch> wrote:

> The other is the ordering of arguments being parsed unpredictable.
> The usecase given was the `-d 1 -S 2`  arguments  (see earlier emails for
> details).

Fix for that coming soon. :-)
Thank you for the report.


> > Please could you indicate me command-line tools where short-option
> > with optional-argument is possible.
> > Because if there is one, I could have inspiration to know how it
> > resolves the ambiguity.
>
> The design of the short options is that it is an alias. Identical to the
> software regardless of what the user typed.

Yes.  But AFAIU, it is hard -- not impossible -- to detect what is an
argument or what is another option in the case of optional argument in
the short-name form.  Because it leads to ambiguous parsing.


> So you get 'cut --field 1' or 'cut -f1' or 'cut -f 1' or 'cut -f=1'.
> All identical.
> The important part here is that each _option_ is written separately, with a
> leading dash.

And try "cut -f -d' '", it raises the error "cut: invalid field value ‘d ’".

All short-name and long-name are ``equivalent`` when they do not
require any argument -- for example with cut: -s, --only-delimited --
*_or_* they require one argument -- for example: -f, --fields=LIST.

But there is an ambiguity for optional argument.  How do you detect if
the argument is provided or not?
With the long-name, it is done with the character '='.
For short-name, it is ambiguous.  Imagine that "guix package" has in
addition to '-S' the option '-2' meaning verbosity to level 2
(--verbosity=2).  Then what is the meaning of:

  guix package -S -2

?

Is it equivalent to
  + --switch-generation=-2
or
  + --switch-generation --verbose=2
?


> You asked for an example;  see `git commit -S`. From the manpage:
>
>    -S[<keyid>], --gpg-sign[=<keyid>]

Thank you for the example.   Let me show you that it raises an issue
too because it is not so "simple". :-)

--8<---------------cut here---------------start------------->8---
mv ~/.gnupg ~/.gnupg.bak
gpg --gen-key
gpg --list-keys

mkdir -p /tmp/foo
touch /tmp/foo/bar
git -C /tmp/foo init
git -C /tmp/foo/ add  bar

git commit -S -m 'init'
error: gpg failed to sign the data
fatal: failed to write commit object

git -C /tmp/foo commit -S 4417B7AADBEFFBEBE4C201271A6DD2B6218BF4B3 -m 'init'
error: pathspec '4417B7AADBEFFBEBE4C201271A6DD2B6218BF4B3' did not
match any file(s) known to git

git -C /tmp/foo commit
--gpg-sig=4417B7AADBEFFBEBE4C201271A6DD2B6218BF4B3 -m 'init'
[master (root-commit) f4df0ff] init
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 bar

git config --global user.signingkey 4417B7AADBEFFBEBE4C201271A6DD2B6218BF4B3

echo ok > /tmp/foo/bar

git -C /tmp/foo commit -S -am 'bis'
[master 639c41e] bis
 1 file changed, 1 insertion(+)
--8<---------------cut here---------------end--------------->8---

Maybe Git manual is lying. ;-)


> > The issue is that Guix uses a bad practise: option with optional-argument
> >  with both short and long name.  It is a mistake to provide the short-name
> >  for such case.
>
> It looks like the parser could be improved by preferring to see any argument
> with leading dash as a option when it **might** be an argument.

It does not work for the general case as you describe.  It is not so simple. :-)
Because '-d -1' means '--delete-generation=-1' and not '--delete-generation -1'.

So considering the situation '-d -X', the parser needs to guess if
'-X' is the argument of the option '-d' or if it is another option.
Yes, it is possible but it is not so easy -- nor impossible -- because
what does it means '-S -2d'?
   + --switch-generation=-2d
or
  + --switch-generation --verbosity=2 --delete-generation

From my knowledge, all that is solved by the rule: no short option
with optional argument.


> So; if you type -`guix package -l --help` then your parser **first** finds all
> the items with leading dashes and second it tries to find out if there is an
> argument for the `-l`.
> In this case I expect the help to be shown.
>
> This is widely seen as a solution.
> Users can still use items with leading dashes by using two commonly used
> tricks.
> The -l=a type of construction allows the argument to be anything. Including it
> having a leading dash.
>
> Second is the double-dash argument that stops words leading with dashes being
> parsed as options.
>   For instance;   grep -- -v *
> the  -v is parsed as an actual string and not an option because it follows the
> double dashes.

You miss the point, I believe.
The issue of *any* parser is only for the "flag" with optional
argument in their short-name form.
Because, as I explained above, the syntax for such cases is ambiguous.

Otherwise, the parser really behaves as you expect!


> > Now all this is clearer for me and I do not think it is a fixable bug.
>
> It is, just follow the suggestion from me and from zimoun: any command-line-
> argument that starts with a dash should be preferred to be an option. Only in
> a second phase do you try to match anything to (optional) options.

You are referring these lines [1] from me, right?

--8<---------------cut here---------------start-------------->8---
srfi-37 should consider that if an argument starts with
dash, then it is not an argument and turn it into an option.
--8<---------------cut here---------------end--------------->8---

I thought that and then I tried to fix it in the Guile implementation
of SRFI-37 [2] and I learned [3] that it is not so simple, as I
explained to you above.

[1] http://issues.guix.gnu.org/40549#7
[2] http://git.savannah.gnu.org/cgit/guile.git/tree/module/srfi/srfi-37.scm
[3] http://issues.guix.gnu.org/40549#10


> As stated, the rest of the world does this, please check out the various
> examples I gave here to confirm that others have solved it and it may be
> possible to solve it for guix too.

Please consider that some people here are long standing GNU hackers. :-)
So they might be the same ones that implemented the "rest of the world" too. ;-)

All the best,
simon

bug#40549: More usability issues:

[Tom Zander via Bug reports for GNU Guix]

On dinsdag 12 mei 2020 20:08:32 CEST zimoun wrote:
> > The design of the short options is that it is an alias. Identical to the
> > software regardless of what the user typed.
> 
> Yes.  But AFAIU, it is hard -- not impossible -- to detect what is an
> argument or what is another option in the case of optional argument in
> the short-name form.  Because it leads to ambiguous parsing.

The solution here is to be consistent with the way that others do this.
For those cases that you highlight the consistent solution elsewhere is 
something you can apply here too. This follows the rule of the least surprise.
 
> And try "cut -f -d' '", it raises the error "cut: invalid field value ‘d ’".

that is because -f has a required argument. As such I agree with your example, 
but ignore it because it is not relevant to our discussion. It is not an 
optional short option.
 
> All short-name and long-name are ``equivalent`` when they do not
> require any argument -- for example with cut: -s, --only-delimited --
> *_or_* they require one argument -- for example: -f, --fields=LIST.

They are expected to always be equivalent. It would not be logical to have the 
short one as an alias if they are not equivalent.

> But there is an ambiguity for optional argument.  How do you detect if
> the argument is provided or not?
> With the long-name, it is done with the character '='.
> For short-name, it is ambiguous.  Imagine that "guix package" has in

The point here is that there exist rules that others use, to remove this 
ambiguity. I'm trying to explain them to you to allow guix to behave the same.
I agree its tricky, the good thing is that it has been solved before.

For instance you can provide an argument to short options without spaces too:
You can write `cut -f1 -d:`.

> addition to '-S' the option '-2' meaning verbosity to level 2
> (--verbosity=2).  Then what is the meaning of:
> 
>   guix package -S -2

If the user meant the "-2" argument to be an argument to the -S option, they 
would have to write (again, following the rules used by most software) as:

echo "test" > -bla
ls -I \-bla

or, to comment on your specific example. In git you don't use the dash to have 
a negative, they use a different char. The tilde. Or dotdotdot. (think: range)
   git show HEAD~
    git diff  ...master

> Is it equivalent to
>   + --switch-generation=-2
> or
>   + --switch-generation --verbose=2

In most software you would interpret any argument that starts with a dash as 
an option. For consistency sake you would thus interpret your example as the 
latter option.
The user can specify what they mean, but preferably you pick your syntax 
better to avoid the -2 in the example you gave :)


> > You asked for an example;  see `git commit -S`. From the manpage:
> >    -S[<keyid>], --gpg-sign[=<keyid>]
> 
> Thank you for the example.   Let me show you that it raises an issue
> too because it is not so "simple". :-)

Easier example then: from the 'ls(1)' manpage:
  -I, --ignore=PATTERN

seems git is trying to be smart.

 
> > It looks like the parser could be improved by preferring to see any
> > argument with leading dash as a option when it **might** be an argument.
> 
> It does not work for the general case as you describe.  It is not so simple.
> :-) Because '-d -1' means '--delete-generation=-1' and not
> '--delete-generation -1'.

The point is consistency.
The point is not to reinvent the wheel that have been invented so many 
times... A command line parser is a known thing that you can, and should, 
mirror how others do things.

So, sure, "-d -1" does not give you what you want. BUT it is consistent with
--delete -1.
This is the point the exercise. The basic premise is that the short version is 
an alias for the long version.
Yes, you have to be aware of how to format things, how the rules apply. But 
this is the entire point: people should be able to learn it once. And not 
again for guix because we thought something else worked better...


> From my knowledge, all that is solved by the rule: no short option
> with optional argument.

Sure, you can do that.
You'll probably annoy of a lot of people if you remove the thing they 
use a lot :)

Even 'ls' uses optional short arguments (--ignore). So I'm not sure I agree 
with your line of reasoning.


> > So; if you type -`guix package -l --help` then your parser **first** finds
> > all the items with leading dashes and second it tries to find out if
> > there is an argument for the `-l`.
> > In this case I expect the help to be shown.
> > 
> > This is widely seen as a solution.
> > Users can still use items with leading dashes by using two commonly used
> > tricks.
> > The -l=a type of construction allows the argument to be anything.
> > Including it having a leading dash.
> > 
> > Second is the double-dash argument that stops words leading with dashes
> > being parsed as options.
> > 
> >   For instance;   grep -- -v *
> > 
> > the  -v is parsed as an actual string and not an option because it follows
> > the double dashes.
> 
> You miss the point, I believe.
> The issue of *any* parser is only for the "flag" with optional
> argument in their short-name form.
> Because, as I explained above, the syntax for such cases is ambiguous.

I'm going to have to disagree with you: this is software. You can **decide** 
the rule and thus decide the parsing. Making the idea that it is ambiguous 
false. The rules I explained remove the ambiguity. It makes it extremely clear 
what happens.
And that -S -2 example has a very clear behavior.

The point of this bugreport is that my suggestion is that the behavior should 
be changed to follow what most other software does.

Doing;
  ls -I -v
clearly understands that '-v' is not to be used as an optional argument.
The reason, as far as I can tell, is that it does not fit as another argument.


The actual design of the arguments needs a bit of finesse, I agree. The '-d -2' 
example is really not the best UI design.
Avoidance is best.

Yes, usage of 'guix package [option] is stupid because it generates a lot of 
cornercases here.
Ideally it would be guix package list [options].

In general I do agree that your solution of dropping all optional short 
arguments would be the best solution.
They are extremely rare elsewhere.

It would be a bit of a bigger project to make a nice UI without them, though.
Please keep as the hard rule that the short version is an ALIAS to the long 
version.

-- 
Tom Zander

bug#40549: More usability issues:

[zimoun]

On Tue, 12 May 2020 at 22:19, Tom Zander <tomz@freedommail.ch> wrote:

> They are expected to always be equivalent. It would not be logical to have the
> short one as an alias if they are not equivalent.

I agree.
Note that you cannot have short-name with optional argument or you
have to break a rule; see below.


> > > You asked for an example;  see `git commit -S`. From the manpage:
> > >    -S[<keyid>], --gpg-sign[=<keyid>]
> >
> > Thank you for the example.   Let me show you that it raises an issue
> > too because it is not so "simple". :-)
>
> Easier example then: from the 'ls(1)' manpage:
>   -I, --ignore=PATTERN

No. `ls -I -l` is not doing the right behaviour, i.e., the flag '-l'
is not applied.
PATTERN is not optional.


> seems git is trying to be smart.

Git resolves the ambiguity by removing the form '-S keyid'.
Other said,
(1) '-Skeyid' uses keyid as argument
(2) '-S keyid' fails as I showed you.
(3) '-S' fallbacks to the default (see .gitconfig)

Back to Guix, using the same strategy means:

(1)  guix package -d8 -p /path/to/profile
(2)  guix package -d 8 -p /path/to/profile # fails
(3)  guix pacakge -d -p /path/to/profile # delete all the generations
except the current one

The three cases cannot all works.  You have to choose two cases.
Currently Guix uses (1) and (2); and (3) fails.   Git uses (1) and
(3); and (2) fails.

You have right by remarking that the "git-way" seems more consistent
when flipping the options.  However they are less consistent in regard
with option requiring one argument.

 git commit -S4417B7 -m 'init'

I do not have a strong opinion on the topic.  Even if I am often
annoyed by "guix package -I -p /path/to/profile".


> The point is not to reinvent the wheel that have been invented so many
> times... A command line parser is a known thing that you can, and should,
> mirror how others do things.

Are you aware of the wheel? :-)

https://srfi.schemers.org/srfi-37/srfi-37.html
https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html
https://pubs.opengroup.org/onlinepubs/007904975/basedefs/xbd_chap12.html#tag_12_02


> Even 'ls' uses optional short arguments (--ignore). So I'm not sure I agree
> with your line of reasoning.

No, the argument of '--ignore' is not optional.  At least with the GNU
version from coreutils 8.30.
Note the options with optional argument (--color and --hyperlink) do
not have a short-name form.


> Doing;
>   ls -I -v
> clearly understands that '-v' is not to be used as an optional argument.
> The reason, as far as I can tell, is that it does not fit as another argument.

You have wrong.  Compare

   ls -I -l
   ls -l -I
   ls -I '' -l


Because Guix uses SRFI-37 to parse command-line arguments and this
will not change, IMHO, the question asked on guile-devel is the one
explained above about the 3 cases.

All the best,
simon

bug#40549: More usability issues:

[Tom Zander via Bug reports for GNU Guix]

It looks like I was wrong on the ls example,

more importantly, it seems we all agree that short options with optional 
arguments are messy, at best.

I think the best course of action is to take a look at the guix command line 
design and find a way to move away from depending on them. Maybe some iterative 
change its approach.

bug#40549: More usability issues:

[Arne Babenhauserheide]

Tom Zander via Bug reports for GNU Guix <bug-guix@gnu.org> writes:
>> > You asked for an example;  see `git commit -S`. From the manpage:
>> >    -S[<keyid>], --gpg-sign[=<keyid>]
>> 
>> Thank you for the example.   Let me show you that it raises an issue
>> too because it is not so "simple". :-)
>
> Easier example then: from the 'ls(1)' manpage:
>   -I, --ignore=PATTERN
>
> seems git is trying to be smart.

Yes — and in that it often annoys me.

Anytime I use -S I worry about the interpretation of arguments, because
it already bit me a few times.

Git is a counter-example: It’s commandline is as unintuitive as it can
get without being an unfinished prototype.

And there really isn’t as much consistency in tools as you state.

Many tools interpret --foo bar as ((foo bar)), while others only accept
--foo=bar as ((foo bar)) and treat --foo bar as ((foo #t) bar).

Some tools only accept options before positional arguments, while others
allow you to intersperse both.

Some tools use named arguments without dash (dd).

GNU tools often differ from BSD tools by allowing options after
positional arguments and treating --foo bar as (foo bar).

And please don’t do radical changes on guix package. That would break
the workflow of every slightly longer term guix user.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

bug#40549: More usability issues:

[zimoun]

Dear Arne,

On Wed, 13 May 2020 at 18:32, Arne Babenhauserheide <arne_bab@web.de> wrote:

> And please don’t do radical changes on guix package. That would break
> the workflow of every slightly longer term guix user.

I do not want to do radical changes.  The parser of the command-line
is SRFI-37.  So it is documented and the command-line follows guide
lines that will not change.

The change I am asking for is only the short-name option with optional
argument.  In such case the optional argument is not optional.  Other
said, the issue is rooted in a bad practise and, AFAIK, Guix does not
follow what other GNU tools follow: no short-name with optional
argument.  And the Guile implementation of SRFI-37 says that the
semantic of short-name with optional argument leads to an issue.  But
now, it is here so what do we do?

In the end, there is annoyance.  And the goal is to discuss what could
be changed to reduce the annoyance, on all sides.

For example, I am always annoyed by:

   guix package -I -p /tmp/profile

It would like it works.  And to do so, I accept that "guix package -I
regexp -p /tmp/profile" does not anymore and would be replaced by
"guix package -Iregexp -p /tmp/profile" which already works (as
specified by SRFI-37).


Today, the Guix manual is lying because the optional argument for
short-name is *not* optional depending on its position.  And that
leads to surprise:

   guix package -I -p /tmp/profile # fails
   guix package -p /tmp/profile -I # works

And this is really really annoying!  It is hard to understand why such
different behaviour.


Instead of what I am proposing, what do you suggest?


Best regards,
simon

bug#40549: More usability issues:

[Arne Babenhauserheide]

Dear zimoun,

zimoun <zimon.toutoune@gmail.com> writes:
> It would like it works.  And to do so, I accept that "guix package -I
> regexp -p /tmp/profile" does not anymore and would be replaced by
> "guix package -Iregexp -p /tmp/profile" which already works (as
> specified by SRFI-37).

Wow, this surprised me. I expected

   guix package -Iregexp

to be equivalent to

   guix package -I -r -e -g -e -x -p

which is how getopt long works in the shell.

> Today, the Guix manual is lying because the optional argument for
> short-name is *not* optional depending on its position.  And that
> leads to surprise:
>
>    guix package -I -p /tmp/profile # fails
>    guix package -p /tmp/profile -I # works
>
> And this is really really annoying!  It is hard to understand why such
> different behaviour.

Yes, this seems annoying.

> Instead of what I am proposing, what do you suggest?

What I would suggets would break -Iregexp and as such deviate from
SRFI-37, so it wouldn’t be optimal.

It would first need another SRFI so it’s no shortterm fix :-)

The commandline handling I want is this:
https://www.draketo.de/english/free-software/shell-argument-parsing

But I don’t yet have a good way to do it in Guile.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

bug#40549: More usability issues:

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2167 bytes --]

On Fri, Apr 24, 2020 at 10:28:50AM +0200, zimoun wrote:
> On Thu, 23 Apr 2020 at 21:52, Tom via Bug reports for GNU Guix
> <bug-guix@gnu.org> wrote:
> >
> > To add detail here:
> >
> > Doing `guix package -d 18 -S 17` actually works.
> >
> > This gives me the impression that the order of arguments is relevant to the
> > processing of them.
> 
> It is known and cumbersome: a feature? ;-)
> 
> Other examples:
> 
>   guix package -I -A # does nothing
>   guix package -A -I # list available
> 
> 
> > Another, similar, example is:
> >
> >    guix package -l --profile=a
> >
> > this doesn't work. But the most curious thing is that this does:
> >
> >    guix package --list-installed --profile=a
> 
> It is worse than that.
> 
> --8<---------------cut here---------------start------------->8---
> # OK
>  guix package --list-generations -p /path/to/profile
>  guix package --list-installed -p /path/to/profile
> 
> # KO
>  guix package -l -p /path/to/profile
>  guix package -I -p /path/to/profile
> 
> # OK
>  guix package -p /path/to/profile -l
>  guix package -p /path/to/profile -I
> 
> # KO
>  guix package -l --profile=/path/to/profile
> 
> # Do nothing
>  guix package -I --profile=/path/to/profile
> 
> # OK
>  guix package -l --profile=/path/to/profile -l
>  guix package -I --profile=/path/to/profile -I
> --8<---------------cut here---------------end--------------->8---
> 
> 
> Well, it seems there is a bug. :-)
> 
> Back to the first examples:
> 
> --8<---------------cut here---------------start------------->8---
> # Do something
>  guix package --list-installed -A
> 
> # Do nothing
>  guix package -I -A
> --8<---------------cut here---------------end--------------->8---
> 
> 
> Thanks for reporting.
> 
> All the best,
> simon
> 

To add to this:

$ guix package -p profile1 -p profile2 -I | wc -l
91
$ guix package -p profile2 -p profile1 -I | wc -l
12

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#40549: More usability issues:

[zimoun]

Dear Arne,


On Wed, 13 May 2020 at 20:53, Arne Babenhauserheide <arne_bab@web.de> wrote:
> zimoun <zimon.toutoune@gmail.com> writes:

> > It would like it works.  And to do so, I accept that "guix package -I
> > regexp -p /tmp/profile" does not anymore and would be replaced by
> > "guix package -Iregexp -p /tmp/profile" which already works (as
> > specified by SRFI-37).
>
> Wow, this surprised me. I expected
>
>    guix package -Iregexp
>
> to be equivalent to
>
>    guix package -I -r -e -g -e -x -p
>
> which is how getopt long works in the shell.

I am not familiar with getopt but I think getopt does what the POSIX specifies.
Your expectation is legitimate when all the flags does not require a
mandatory argument.  For example,

   ls -artl

but it is not true when one of the flags requires one mandatory
argument, for example,

   ls -aIrtl

then 'rtl' is seen as the argument of the option '-I, --ignore='.
Well, the correct is:

   ls -aI '' -rtl

And the specification e.g., [1], says that "ls -aI'' -rtl" should too;
as for example "cut -f1 -d" and "cut -f 1 -d" both are allowed.

I should misread but all that perfectly works and is clearly specified
by SRFI-37.  What is missing is the corner case: how to deal with
short-name option with optional argument?  There is 2 incompatible
rules as I try to explain here [2].  The choice of the Guile
implementation leads to break the permutation rule for the short-name
with optional argument and maintain consistency with the space between
flag and argument.  Another choice is to break the consistency with
the space between flag and argument and so it does not break the
permutation rule; as Tom mentioned with the Git example of
'-S[<keyid>], --gpg-sign[=<keyid>]'.

The issue is that the semantic of short-name of optional argument is
ambiguous considering all the other rules.


[1] https://www.gnu.org/software/libc/manual/html_node/Argument-Syntax.html
[2] http://issues.guix.gnu.org/40549#16


> > Instead of what I am proposing, what do you suggest?
>
> What I would suggets would break -Iregexp and as such deviate from
> SRFI-37, so it wouldn’t be optimal.

Just to note that the Guile implementation of SRFI-37 is "aware" of
this corner case as it is mentioned in their commentary, already said
here [3]. :-)

http://git.savannah.gnu.org/cgit/guile.git/tree/module/srfi/srfi-37.scm#n51

[3] http://issues.guix.gnu.org/40549#10


> It would first need another SRFI so it’s no shortterm fix :-)

From my point of view, different fixes are possible depending on what
we accept to break (here for "guix package" and probably similar cases
with other subcommand exist):

 - nothing => improve the manual
 - remove the short-name -u, -l, -d, -I, -A
 - remove the argument for the short-name and keep it for the long-name only.
 - remove '-X value' and allow only -'Xvalue' for X in {u, l, d, I, A}
=> new / amend SRFI-37
 - other that my imagination is missing

Well, I have asked advices on guile-user [4].  I do not know if the
issue could be seen as a bug of SRFI-37.

[4] https://lists.gnu.org/archive/html/guile-user/2020-05/msg00034.html


> The commandline handling I want is this:
> https://www.draketo.de/english/free-software/shell-argument-parsing

Thank you for the link.  For my understanding -- even after reading
the stackoverflow link -- your handling does not address the
short-name with optional argument.  Does it mean you consider that
should not happen?


Best regards,
simon

bug#40549: More usability issues:

[zimoun]

Hi Efraim,

On Thu, 14 May 2020 at 10:15, Efraim Flashner <efraim@flashner.co.il> wrote:

> $ guix package -p profile1 -p profile2 -I | wc -l
> 91
> $ guix package -p profile2 -p profile1 -I | wc -l
> 12

I bet:

$ guix package -p profile1 -I | wc -l
91
$ guix package -p profile2 -I | wc -l
12

Well, thank you for the report.

Cheers,
simon

bug#40549: Fix -p profile -p profile -I

[zimoun]

On Thu, 14 May 2020 at 10:15, Efraim Flashner <efraim@flashner.co.il> wrote:

> $ guix package -p profile1 -p profile2 -I | wc -l
> 91
> $ guix package -p profile2 -p profile1 -I | wc -l
> 12

Just to let you know that a fix is proposed there [1].

[1] http://issues.guix.gnu.org/issue/41260


All the best,
simon

bug#40549: [usability] revert last generation

[zimoun]

Hi,

Thanks for the report.  Because you reported 2 issues:

  a) one about the order of ’%actions’ processing
  b) one about short-name with optional argument

I prefer to close this bug and track the issues into 2 separated
numbers, i.e.,

  a) see #50472 <http://issues.guix.gnu.org/issue/50472>
  b) see #50473 <http://issues.guix.gnu.org/issue/50473>

because it eases the actionable steps to fix them.


All the best,
simon