bug#27820: guix package -u: order of argument is significant

bug#27820: guix package -u: order of argument is significant

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 850 bytes --]

Hello,

I'm try upgrading from guix-0.12.0-10.ba2260d but the profile is not updated.

I used "guix pull" to get the latest version.

"guix package -u" is loading substitutes, fails with this and recommends
using --fallback.

"guix package -u --fallback" when run the first time did compile some
packages, but did not update the profile.

"guix package -u --fallback" when run another time does *nothing*. "guix
package -l" still show the old generation.

So I updated only guix "guix package -u guix", which gave me guix-0.13.0-4.f1ddfe4.


"guix package -u --fallback" when run another time again did nothing.

BUT:

"guix package --fallback -u" did upgrade the packages.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: 0xBF773B65.asc --]
[-- Type: application/pgp-keys, Size: 14855 bytes --]

bug#27820: guix package -u: order of argument is significant

[Mark H Weaver]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> I'm try upgrading from guix-0.12.0-10.ba2260d but the profile is not updated.
>
> I used "guix pull" to get the latest version.
>
> "guix package -u" is loading substitutes, fails with this and recommends
> using --fallback.
>
> "guix package -u --fallback" when run the first time did compile some
> packages, but did not update the profile.

That's because "--fallback" was treated as the argument to -u, i.e. the
regexp specifying which packages to upgrade.  The few compiled packages
were needed to run the profile hooks.

> "guix package -u --fallback" when run another time does *nothing*. "guix
> package -l" still show the old generation.

See above.  It did nothing this time because the packages needed to run
the profile hooks were already up-to-date.

> So I updated only guix "guix package -u guix", which gave me guix-0.13.0-4.f1ddfe4.
>
>
> "guix package -u --fallback" when run another time again did nothing.
>
> BUT:
>
> "guix package --fallback -u" did upgrade the packages.

In this case, since there's no argument after -u, it implicitly upgrades
all packages.

When I want to add more options after -u, I add "." as the argument
after -u.

I agree that this is quite confusing.  Perhaps we should issue a warning
if the regexp begins with "-".

Also, perhaps we should *always* require an argument after "-u", even if
"-u" is at the end of the command line, failing otherwise.  Users would
then learn to always pass an argument to "-u", and thus would be less
likely to fall into this trap when adding more options after the "-u".

Thoughts?

      Mark

bug#27820: guix package -u: order of argument is significant

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 489 bytes --]

Mark H Weaver <mhw@netris.org> skribis:

> I agree that this is quite confusing.  Perhaps we should issue a warning
> if the regexp begins with "-".
>
> Also, perhaps we should *always* require an argument after "-u", even if
> "-u" is at the end of the command line, failing otherwise.  Users would
> then learn to always pass an argument to "-u", and thus would be less
> likely to fall into this trap when adding more options after the "-u".

I’m in favor of the former:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 835 bytes --]

diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
index 8da7a3fd3..b6133b6af 100644
--- a/guix/scripts/package.scm
+++ b/guix/scripts/package.scm
@@ -486,6 +486,11 @@ Install, remove, or upgrade packages in a single transaction.\n"))
                              arg-handler))))
          (option '(#\u "upgrade") #f #t
                  (lambda (opt name arg result arg-handler)
+                   (when (string-prefix? "-" arg)
+                     (warning (G_ "upgrade regexp '~a' looks like a \
+command-line option~%")
+                              arg)
+                     (warning (G_ "is this intended?~%")))
                    (let arg-handler ((arg arg) (result result))
                      (values (alist-cons 'upgrade arg
                                          ;; Delete any prior "upgrade all"

[-- Attachment #3: Type: text/plain, Size: 25 bytes --]


Thoughts?

Ludo’.

bug#27820: guix package -u: order of argument is significant

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Mark H Weaver <mhw@netris.org> skribis:
>
>> I agree that this is quite confusing.  Perhaps we should issue a warning
>> if the regexp begins with "-".
>>
>> Also, perhaps we should *always* require an argument after "-u", even if
>> "-u" is at the end of the command line, failing otherwise.  Users would
>> then learn to always pass an argument to "-u", and thus would be less
>> likely to fall into this trap when adding more options after the "-u".
>
> I’m in favor of the former:
>
> diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
> index 8da7a3fd3..b6133b6af 100644
> --- a/guix/scripts/package.scm
> +++ b/guix/scripts/package.scm
> @@ -486,6 +486,11 @@ Install, remove, or upgrade packages in a single transaction.\n"))
>                               arg-handler))))
>           (option '(#\u "upgrade") #f #t
>                   (lambda (opt name arg result arg-handler)
> +                   (when (string-prefix? "-" arg)
> +                     (warning (G_ "upgrade regexp '~a' looks like a \
> +command-line option~%")
> +                              arg)
> +                     (warning (G_ "is this intended?~%")))
>                     (let arg-handler ((arg arg) (result result))
>                       (values (alist-cons 'upgrade arg
>                                           ;; Delete any prior "upgrade all"
>
> Thoughts?

This seems good to me.  I just wonder if there are legitimate cases
where a package regexp would look like a command line option.  If that’s
not the case could we just “unread” the argument and parse it as the
next option?

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#27820: guix package -u: order of argument is significant

[Ludovic Courtès]

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Mark H Weaver <mhw@netris.org> skribis:
>>
>>> I agree that this is quite confusing.  Perhaps we should issue a warning
>>> if the regexp begins with "-".
>>>
>>> Also, perhaps we should *always* require an argument after "-u", even if
>>> "-u" is at the end of the command line, failing otherwise.  Users would
>>> then learn to always pass an argument to "-u", and thus would be less
>>> likely to fall into this trap when adding more options after the "-u".
>>
>> I’m in favor of the former:
>>
>> diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
>> index 8da7a3fd3..b6133b6af 100644
>> --- a/guix/scripts/package.scm
>> +++ b/guix/scripts/package.scm
>> @@ -486,6 +486,11 @@ Install, remove, or upgrade packages in a single transaction.\n"))
>>                               arg-handler))))
>>           (option '(#\u "upgrade") #f #t
>>                   (lambda (opt name arg result arg-handler)
>> +                   (when (string-prefix? "-" arg)
>> +                     (warning (G_ "upgrade regexp '~a' looks like a \
>> +command-line option~%")
>> +                              arg)
>> +                     (warning (G_ "is this intended?~%")))
>>                     (let arg-handler ((arg arg) (result result))
>>                       (values (alist-cons 'upgrade arg
>>                                           ;; Delete any prior "upgrade all"
>>
>> Thoughts?
>
> This seems good to me.  I just wonder if there are legitimate cases
> where a package regexp would look like a command line option.  If that’s
> not the case could we just “unread” the argument and parse it as the
> next option?

I thought about it but in theory “-” is perfectly legitimate, so I
thought we’d rather not try to be smart.  Thoughts?

Ludo’.

bug#27820: guix package -u: order of argument is significant

[Hartmut Goebel]

Am 25.07.2017 um 21:40 schrieb Mark H Weaver:
> That's because "--fallback" was treated as the argument to -u, i.e. the
> regexp specifying which packages to upgrade.  The few compiled packages
> were needed to run the profile hooks.

I'm astound about this. Python's argparse library is handling this as
the user expects:

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-u', '--update', nargs='?', const=42)
parser.add_argument('--fallback', action="store_true")

print(parser.parse_args(['-u', '--fallback']))

print(parser.parse_args(['-u', 'xxx', '--fallback']))

prints

Namespace(fallback=True, update=None)
Namespace(fallback=True, update='xxx')

See https://docs.python.org/3/library/argparse.html#nargs

Isn't there a elaborated scheme/guile argparse library we can you? This
would/shouls/could also solve the issue that in guix we can not shortcut
options.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

bug#27820: guix package -u: order of argument is significant

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Mark H Weaver <mhw@netris.org> skribis:
>>>
>>>> I agree that this is quite confusing.  Perhaps we should issue a warning
>>>> if the regexp begins with "-".
>>>>
>>>> Also, perhaps we should *always* require an argument after "-u", even if
>>>> "-u" is at the end of the command line, failing otherwise.  Users would
>>>> then learn to always pass an argument to "-u", and thus would be less
>>>> likely to fall into this trap when adding more options after the "-u".
>>>
>>> I’m in favor of the former:
>>>
>>> diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
>>> index 8da7a3fd3..b6133b6af 100644
>>> --- a/guix/scripts/package.scm
>>> +++ b/guix/scripts/package.scm
>>> @@ -486,6 +486,11 @@ Install, remove, or upgrade packages in a single transaction.\n"))
>>>                               arg-handler))))
>>>           (option '(#\u "upgrade") #f #t
>>>                   (lambda (opt name arg result arg-handler)
>>> +                   (when (string-prefix? "-" arg)
>>> +                     (warning (G_ "upgrade regexp '~a' looks like a \
>>> +command-line option~%")
>>> +                              arg)
>>> +                     (warning (G_ "is this intended?~%")))
>>>                     (let arg-handler ((arg arg) (result result))
>>>                       (values (alist-cons 'upgrade arg
>>>                                           ;; Delete any prior "upgrade all"
>>>
>>> Thoughts?
>>
>> This seems good to me.  I just wonder if there are legitimate cases
>> where a package regexp would look like a command line option.  If that’s
>> not the case could we just “unread” the argument and parse it as the
>> next option?
>
> I thought about it but in theory “-” is perfectly legitimate, so I
> thought we’d rather not try to be smart.  Thoughts?

Is it really legitimate?  The regular expression is supposed to match on
package names and we have no packages starting with “-”.  And even if we
did (or the user has some oddly named packages in GUIX_PACKAGE_PATH),
they could write “^-”.  Or we could demand that the argument be quoted
(“'--foo'” or “"--foo"”) in that case.

It just seems like a really rare edge case to *want* it to behave as it
does now.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#27820: guix package -u: order of argument is significant

[Ricardo Wurmus]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 25.07.2017 um 21:40 schrieb Mark H Weaver:
>> That's because "--fallback" was treated as the argument to -u, i.e. the
>> regexp specifying which packages to upgrade.  The few compiled packages
>> were needed to run the profile hooks.
>
> I'm astound about this. Python's argparse library is handling this as
> the user expects: […]

> Isn't there a elaborated scheme/guile argparse library we can you? This
> would/shouls/could also solve the issue that in guix we can not shortcut
> options.

This is not a technical problem.  It behaves as intended; the question
is just whether the intention really makes sense.

What do you mean by “shortcut options”?

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#27820: guix package -u: order of argument is significant

[Mark H Weaver]

Ricardo Wurmus <rekado@elephly.net> writes:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Ricardo Wurmus <rekado@elephly.net> skribis:
>>
>>> Ludovic Courtès <ludo@gnu.org> writes:
>>>
>>>> Mark H Weaver <mhw@netris.org> skribis:
>>>>
>>>>> I agree that this is quite confusing.  Perhaps we should issue a warning
>>>>> if the regexp begins with "-".
>>>>>
>>>>> Also, perhaps we should *always* require an argument after "-u", even if
>>>>> "-u" is at the end of the command line, failing otherwise.  Users would
>>>>> then learn to always pass an argument to "-u", and thus would be less
>>>>> likely to fall into this trap when adding more options after the "-u".
>>>>
>>>> I’m in favor of the former:
>>>>
>>>> diff --git a/guix/scripts/package.scm b/guix/scripts/package.scm
>>>> index 8da7a3fd3..b6133b6af 100644
>>>> --- a/guix/scripts/package.scm
>>>> +++ b/guix/scripts/package.scm
>>>> @@ -486,6 +486,11 @@ Install, remove, or upgrade packages in a single transaction.\n"))
>>>>                               arg-handler))))
>>>>           (option '(#\u "upgrade") #f #t
>>>>                   (lambda (opt name arg result arg-handler)
>>>> +                   (when (string-prefix? "-" arg)
>>>> +                     (warning (G_ "upgrade regexp '~a' looks like a \
>>>> +command-line option~%")
>>>> +                              arg)
>>>> +                     (warning (G_ "is this intended?~%")))
>>>>                     (let arg-handler ((arg arg) (result result))
>>>>                       (values (alist-cons 'upgrade arg
>>>>                                           ;; Delete any prior "upgrade all"
>>>>
>>>> Thoughts?
>>>
>>> This seems good to me.  I just wonder if there are legitimate cases
>>> where a package regexp would look like a command line option.  If that’s
>>> not the case could we just “unread” the argument and parse it as the
>>> next option?
>>
>> I thought about it but in theory “-” is perfectly legitimate, so I
>> thought we’d rather not try to be smart.  Thoughts?
>
> Is it really legitimate?  The regular expression is supposed to match on
> package names and we have no packages starting with “-”.

The provided regexp is not necessarily intended to match at the
beginning of the package name.  For example, suppose that someone wants
to upgrade both variants of python-six, i.e. {python,python2}-six.  An
easy way to ask for this would be to write:

  guix package -u -six

For comparison, this is how the well-known "grep" program treats
arguments to its '-e' option:

  mhw@jojen ~$ echo "python2-six" | grep -e -six
  python2-six

> And even if we did (or the user has some oddly named packages in
> GUIX_PACKAGE_PATH), they could write “^-”.

And in this case, they could write:

  guix package -u "()-six"

to work around cleverness on our part.

> Or we could demand that the argument be quoted (“'--foo'” or
> “"--foo"”) in that case.

That wouldn't work, because in general quoting is handled by the shell,
not the program.  We never see those quotes, and have no way of knowing
whether or not they were present.

If we *do* see quotes, it means that the user wrote something like this:

  guix package -u '"-six"'

in which case an experienced POSIX user would expect the quotes to be
included in the regexp.  That's how POSIX programs behave in practice.

> It just seems like a really rare edge case to *want* it to behave as it
> does now.

I don't feel strongly about it, and so I'm glad to go along with
whatever you and Ludovic can agree on.  I can appreciate the arguments
on both sides.

     Thanks,
       Mark

bug#27820: guix package -u: order of argument is significant

[Alex Kost]

Ricardo Wurmus (2017-07-26 10:59 +0200) wrote:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Ricardo Wurmus <rekado@elephly.net> skribis:
[...]
>>> This seems good to me.  I just wonder if there are legitimate cases
>>> where a package regexp would look like a command line option.  If that’s
>>> not the case could we just “unread” the argument and parse it as the
>>> next option?
>>
>> I thought about it but in theory “-” is perfectly legitimate, so I
>> thought we’d rather not try to be smart.  Thoughts?
>
> Is it really legitimate?  The regular expression is supposed to match on
> package names and we have no packages starting with “-”.  And even if we
> did (or the user has some oddly named packages in GUIX_PACKAGE_PATH),
> they could write “^-”.  Or we could demand that the argument be quoted
> (“'--foo'” or “"--foo"”) in that case.
>
> It just seems like a really rare edge case to *want* it to behave as it
> does now.

I am on "not try to be smart" side.  Mark described why "-foo" is a
legitimate regexp, so I think it's better to allow users to be free in a
choice of regexps.

-- 
Alex

bug#27820: guix package -u: order of argument is significant

[Ludovic Courtès]

Hello,

I pushed the proposed patch as 6ddf97f81bb99a73f00e30ad5fc19577872b5b49.

Thanks,
Ludo’.