Unhelpful "--help" output

Unhelpful "--help" output

[Ricardo Wurmus]

Hi Guix,

I found the help output not to be very helpful whenever there are
sub-commands.  The output contains options that are not relevant to the
sub-command.  The options also don't appear to be ordered.

Here is an example.  I want to know more about "guix system
docker-image", so I run "guix system docker-image --help".  This is the
output I get:

--8<---------------cut here---------------start------------->8---
Usage: guix system [OPTION ...] ACTION [ARG ...] [FILE]
Build the operating system declared in FILE according to ACTION.
Some ACTIONS support additional ARGS.

The valid values for ACTION are:

   search           search for existing service types
   edit             edit the definition of an existing service type
   reconfigure      switch to a new operating system configuration
   roll-back        switch to the previous operating system configuration
   describe         describe the current system
   list-generations list the system generations
   switch-generation switch to an existing operating system configuration
   delete-generations delete old system generations
   build            build the operating system without installing anything
   container        build a container that shares the host's store
   vm               build a virtual machine image that shares the host's store
   image            build a Guix System image
   docker-image     build a Docker image
   init             initialize a root file system to run GNU
   extension-graph  emit the service extension graph in Dot format
   shepherd-graph   emit the graph of shepherd services in Dot format

  -L, --load-path=DIR    prepend DIR to the package module search path
  -K, --keep-failed      keep build tree of failed builds
  -k, --keep-going       keep going when some of the derivations fail
  -n, --dry-run          do not build the derivations
      --fallback         fall back to building when the substituter fails
      --no-substitutes   build instead of resorting to pre-built substitutes
      --substitute-urls=URLS
                         fetch substitute from URLS if they are authorized
      --no-grafts        do not graft packages
      --no-offload       do not attempt to offload builds
      --max-silent-time=SECONDS
                         mark the build as failed after SECONDS of silence
      --timeout=SECONDS  mark the build as failed after SECONDS of activity
      --rounds=N         build N times in a row to detect non-determinism
  -c, --cores=N          allow the use of up to N CPU cores for the build
  -M, --max-jobs=N       allow at most N build jobs
      --debug=LEVEL      produce debugging output at LEVEL
  -d, --derivation       return the derivation of the given system
  -e, --expression=EXPR  consider the operating-system EXPR evaluates to
                         instead of reading FILE, when applicable
      --allow-downgrades for 'reconfigure', allow downgrades to earlier
                         channel revisions
      --on-error=STRATEGY
                         apply STRATEGY (one of nothing-special, backtrace,
                         or debug) when an error occurs while reading FILE
      --list-image-types list available image types
  -t, --image-type=TYPE  for 'image', produce an image of TYPE
      --image-size=SIZE  for 'image', produce an image of SIZE
      --no-bootloader    for 'init', do not install a bootloader
      --volatile         for 'image', make the root file system volatile
      --persistent       for 'vm', make the root file system persistent
      --label=LABEL      for 'image', label disk image with LABEL
      --save-provenance  save provenance information
      --share=SPEC       for 'vm' and 'container', share host file system with
                         read/write access according to SPEC
      --expose=SPEC      for 'vm' and 'container', expose host file system
                         directory as read-only according to SPEC
  -N, --network          for 'container', allow containers to access the network
  -r, --root=FILE        for 'vm', 'image', 'container' and 'build',
                         make FILE a symlink to the result, and
                         register it as a garbage collector root
      --full-boot        for 'vm', make a full boot sequence
      --no-graphic       for 'vm', use the tty that we are started in for IO
      --skip-checks      skip file system and initrd module safety checks
  -v, --verbosity=LEVEL  use the given verbosity LEVEL

      --graph-backend=BACKEND
                         use BACKEND for 'extension-graph' and 'shepherd-graph'

  -I, --list-installed[=REGEXP]
                         for 'describe' and 'list-generations', list installed
                         packages matching REGEXP

      --list-targets     list available targets
      --target=TRIPLET   cross-build for TRIPLET--e.g., "aarch64-linux-gnu"

      --list-systems     list available systems
  -s, --system=SYSTEM    attempt to build for SYSTEM--e.g., "i686-linux"

      --help-docker-format list options specific to the docker image type.

  -h, --help             display this help and exit
  -V, --version          display version information and exit

Report bugs to: bug-guix@gnu.org.
GNU Guix home page: <https://guix.gnu.org>
General help using Guix and GNU software: <https://guix.gnu.org/en/help/>
--8<---------------cut here---------------end--------------->8---

Note that this is the same output as for "guix system --help".

I don't think it's great to see all these "for 'vm'" qualifiers on the
right-hand side.

Here is what I propose:

- In the case of commands that offer sub-commands, remove *all* the
options from the help output of the main command (e.g. "guix system
--help"), and only showing the list of available sub-commands.

- For "guix [command] [sub-command] --help" show only options to this
  sub-command.

- optionally, group all generic options (e.g. --system, --help,
  --version, etc) under a single heading.

What do you think?

-- 
Ricardo

Re: Unhelpful "--help" output

[Suhail Singh]

Ricardo Wurmus <rekado@elephly.net> writes:

> Here is what I propose:
>
> - In the case of commands that offer sub-commands, remove *all* the
> options from the help output of the main command (e.g. "guix system
> --help"), and only showing the list of available sub-commands.
>
> - For "guix [command] [sub-command] --help" show only options to this
>   sub-command.
>
> - optionally, group all generic options (e.g. --system, --help,
>   --version, etc) under a single heading.
>
> What do you think?

Yes, please!  As a user of noman.el (i.e., a frequent user of --help)
this would be a tremendous improvement over the current implementation.

-- 
Suhail

Re: Unhelpful "--help" output

[Luis Felipe]

[-- Attachment #1.1.1: Type: text/plain, Size: 6543 bytes --]

Hi Ricardo,

On 12/11/24 13:14, Ricardo Wurmus wrote:
> Hi Guix,
>
> I found the help output not to be very helpful whenever there are
> sub-commands.  The output contains options that are not relevant to the
> sub-command.  The options also don't appear to be ordered.
>
> Here is an example.  I want to know more about "guix system
> docker-image", so I run "guix system docker-image --help".  This is the
> output I get:
>
> --8<---------------cut here---------------start------------->8---
> Usage: guix system [OPTION ...] ACTION [ARG ...] [FILE]
> Build the operating system declared in FILE according to ACTION.
> Some ACTIONS support additional ARGS.
>
> The valid values for ACTION are:
>
>     search           search for existing service types
>     edit             edit the definition of an existing service type
>     reconfigure      switch to a new operating system configuration
>     roll-back        switch to the previous operating system configuration
>     describe         describe the current system
>     list-generations list the system generations
>     switch-generation switch to an existing operating system configuration
>     delete-generations delete old system generations
>     build            build the operating system without installing anything
>     container        build a container that shares the host's store
>     vm               build a virtual machine image that shares the host's store
>     image            build a Guix System image
>     docker-image     build a Docker image
>     init             initialize a root file system to run GNU
>     extension-graph  emit the service extension graph in Dot format
>     shepherd-graph   emit the graph of shepherd services in Dot format
>
>    -L, --load-path=DIR    prepend DIR to the package module search path
>    -K, --keep-failed      keep build tree of failed builds
>    -k, --keep-going       keep going when some of the derivations fail
>    -n, --dry-run          do not build the derivations
>        --fallback         fall back to building when the substituter fails
>        --no-substitutes   build instead of resorting to pre-built substitutes
>        --substitute-urls=URLS
>                           fetch substitute from URLS if they are authorized
>        --no-grafts        do not graft packages
>        --no-offload       do not attempt to offload builds
>        --max-silent-time=SECONDS
>                           mark the build as failed after SECONDS of silence
>        --timeout=SECONDS  mark the build as failed after SECONDS of activity
>        --rounds=N         build N times in a row to detect non-determinism
>    -c, --cores=N          allow the use of up to N CPU cores for the build
>    -M, --max-jobs=N       allow at most N build jobs
>        --debug=LEVEL      produce debugging output at LEVEL
>    -d, --derivation       return the derivation of the given system
>    -e, --expression=EXPR  consider the operating-system EXPR evaluates to
>                           instead of reading FILE, when applicable
>        --allow-downgrades for 'reconfigure', allow downgrades to earlier
>                           channel revisions
>        --on-error=STRATEGY
>                           apply STRATEGY (one of nothing-special, backtrace,
>                           or debug) when an error occurs while reading FILE
>        --list-image-types list available image types
>    -t, --image-type=TYPE  for 'image', produce an image of TYPE
>        --image-size=SIZE  for 'image', produce an image of SIZE
>        --no-bootloader    for 'init', do not install a bootloader
>        --volatile         for 'image', make the root file system volatile
>        --persistent       for 'vm', make the root file system persistent
>        --label=LABEL      for 'image', label disk image with LABEL
>        --save-provenance  save provenance information
>        --share=SPEC       for 'vm' and 'container', share host file system with
>                           read/write access according to SPEC
>        --expose=SPEC      for 'vm' and 'container', expose host file system
>                           directory as read-only according to SPEC
>    -N, --network          for 'container', allow containers to access the network
>    -r, --root=FILE        for 'vm', 'image', 'container' and 'build',
>                           make FILE a symlink to the result, and
>                           register it as a garbage collector root
>        --full-boot        for 'vm', make a full boot sequence
>        --no-graphic       for 'vm', use the tty that we are started in for IO
>        --skip-checks      skip file system and initrd module safety checks
>    -v, --verbosity=LEVEL  use the given verbosity LEVEL
>
>        --graph-backend=BACKEND
>                           use BACKEND for 'extension-graph' and 'shepherd-graph'
>
>    -I, --list-installed[=REGEXP]
>                           for 'describe' and 'list-generations', list installed
>                           packages matching REGEXP
>
>        --list-targets     list available targets
>        --target=TRIPLET   cross-build for TRIPLET--e.g., "aarch64-linux-gnu"
>
>        --list-systems     list available systems
>    -s, --system=SYSTEM    attempt to build for SYSTEM--e.g., "i686-linux"
>
>        --help-docker-format list options specific to the docker image type.
>
>    -h, --help             display this help and exit
>    -V, --version          display version information and exit
>
> Report bugs to: bug-guix@gnu.org.
> GNU Guix home page: <https://guix.gnu.org>
> General help using Guix and GNU software: <https://guix.gnu.org/en/help/>
> --8<---------------cut here---------------end--------------->8---
>
> Note that this is the same output as for "guix system --help".
>
> I don't think it's great to see all these "for 'vm'" qualifiers on the
> right-hand side.
>
> Here is what I propose:
>
> - In the case of commands that offer sub-commands, remove *all* the
> options from the help output of the main command (e.g. "guix system
> --help"), and only showing the list of available sub-commands.
>
> - For "guix [command] [sub-command] --help" show only options to this
>    sub-command.
>
> - optionally, group all generic options (e.g. --system, --help,
>    --version, etc) under a single heading.
>
> What do you think?

That's the functionality I would expect, so +1



[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 2881 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: Unhelpful "--help" output

[Christopher Howard]

Hi, if planning to make changes to --help output, could you please try to preserve the output formatting as much as possible? I'm trying to fix the bitrot in the emacs-guix "shell commands" code, which actually parses the --help output using regexs, and and generates magit-popup menus based on that. So little things like the number of spaces on the line before the command name actually matter.

-- 
Christopher Howard

Re: Unhelpful "--help" output

[indieterminacy]

Hello Christopher,

On 2024-11-12 16:23, Christopher Howard wrote:
> Hi, if planning to make changes to --help output, could you please try 
> to preserve the output formatting as much as possible? I'm trying to 
> fix the bitrot in the emacs-guix "shell commands" code, which actually 
> parses the --help output using regexs, and and generates magit-popup 
> menus based on that. So little things like the number of spaces on the 
> line before the command name actually matter.

Sounds interesting.

I guess you are being implicit regarding the spacing order?
It may be more resilient to consider counting the number of opening 
spaces/tabs and then compare the results.

In any case Id like to look at the uri to see how Transient is operating 
off your PEGs.

Thanks for working on this!

Kind regards,


Jonathan

Re: Unhelpful "--help" output

[Christopher Howard]

indieterminacy <indieterminacy@libre.brussels> writes:

> Sounds interesting.
>
> I guess you are being implicit regarding the spacing order?
> It may be more resilient to consider counting the number of opening
> spaces/tabs and then compare the results.
>
> In any case Id like to look at the uri to see how Transient is
> operating off your PEGs.
>

Hi, right now I'm just trying, during my lunchbreaks, to fix the code that is already written. But I keep having to use my lunch breaks for other things, like vehicle maintenance, so progress has been slow.

https://git.savannah.gnu.org/git/guix/emacs-guix.git/

One of the interfaces emacs-guix provides is a magit-popup interface to all the guix commands, which it generates by parsing the "guix --help" output, along with Improver code to improve handling of some of the options. That was broke (for years...?) because the regex expected three spaces before a command description, whereas now there is four spaces. I submitted a patch for that. What is the old saying? Solve a problem with a regular expression, and now you have two problems. :) But the regular expressions are written in that nice rx notation, so that helps a lot.

Emacs-guix uses magit-popup, the predecessor of Transient.

-- 
Christopher Howard

Re: Unhelpful "--help" output

[Ricardo Wurmus]

Christopher Howard <christopher@librehacker.com> writes:

> Hi, if planning to make changes to --help output, could you please try
> to preserve the output formatting as much as possible?

I don't think I can guarantee this.

> I'm trying to
> fix the bitrot in the emacs-guix "shell commands" code, which actually
> parses the --help output using regexs, and and generates magit-popup
> menus based on that. So little things like the number of spaces on the
> line before the command name actually matter.

Perhaps emacs-guix could instead fetch the list of items as an
S-expression.

-- 
Ricardo

Re: Unhelpful "--help" output

[Christopher Howard]

Ricardo Wurmus <rekado@elephly.net> writes:

>> I'm trying to
>> fix the bitrot in the emacs-guix "shell commands" code, which actually
>> parses the --help output using regexs, and and generates magit-popup
>> menus based on that. So little things like the number of spaces on the
>> line before the command name actually matter.
>
> Perhaps emacs-guix could instead fetch the list of items as an
> S-expression.

You mean through guix guile API, or...? As far as the help output, emacs-guix only knows if something is a "main command", as opposed to a description of the command grouping, or other help text, based on the number of spaces preceding it, which currently is 4 spaces. For options and switches, it can look for the "-" or "--".

The emacs-guix "shell command" functionality is essentially this engine that takes the perhaps dubious, but mostly effective, approach of automatically parsing all "help" and "--help" output to make it possible to manage all commands through an automatically generated magit-popup interface.

The switch and option regexs are not dependent an any exact amount of whitespace, so I think that part of it is resilient enough. But the regex for the main commands are dependent on the number of spaces before the start of the command name, in the command description line. So at least those number of spaces need to remain consistent, for the "shell command" functionality to work as it is currently coded.

-- 
Christopher Howard

Re: Unhelpful "--help" output

[Ricardo Wurmus]

Christopher Howard <christopher@librehacker.com> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>>> I'm trying to
>>> fix the bitrot in the emacs-guix "shell commands" code, which actually
>>> parses the --help output using regexs, and and generates magit-popup
>>> menus based on that. So little things like the number of spaces on the
>>> line before the command name actually matter.
>>
>> Perhaps emacs-guix could instead fetch the list of items as an
>> S-expression.
>
> You mean through guix guile API, or...?

Yes.  The (guix scripts *) modules all define %options, a list of
options.  They are not exported, but they could be.   (Alternatively, an
accessor could be exported that returns the options as plain data.)
This structural data could be used in emacs-guix instead of parsing
its textual representation for presentation in the shell.

-- 
Ricardo

Re: Unhelpful "--help" output

[Simon Tournier]

Hi Ricardo,

On Tue, 12 Nov 2024 at 14:14, Ricardo Wurmus <rekado@elephly.net> wrote:

> Note that this is the same output as for "guix system --help".

Yes, that’s because the command-line parser is poorly implemented. :-)

> - In the case of commands that offer sub-commands, remove *all* the
> options from the help output of the main command (e.g. "guix system
> --help"), and only showing the list of available sub-commands.

Yes.  Well, let start with “guix system”. :-)

> - For "guix [command] [sub-command] --help" show only options to this
>   sub-command.

Yes.

> - optionally, group all generic options (e.g. --system, --help,
>   --version, etc) under a single heading.

Yes.  It appears to me the easiest. ;-)

Cheers,
simon

PS: I think it might be an “easy” contribution for Google Summer of
Code candidate… But maybe it could be nice to improve before the next
round.