Ideas to improve the output of C-h m?

Ideas to improve the output of C-h m?

[Clément Pit-Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 4891 bytes --]

Hi all,

A while ago I tried to improve the discoverability of the keybindings of one of my packages by writing an enhanced version of the keybindings part of C-h m.  The idea was to list keybindings along with the first line of their docstrings, instead of just listing the name of the function.

I have pasted at the botom of this email the output of the current C-h m and the output of the code I wrote a while ago.  Here's how it looks for a single entry, to see the difference (note that unlike C-h m, it currently groups keybindings that map to the same command):

Current output:
  h               describe-mode
  …
  ?               describe-mode

In biblio.el:
  ‘h’, ‘?’ (describe-mode)
    Display documentation of current major mode and minor modes.

Is there interest in integrating something like this to the usual C-h m? I'm not thinking of replacing the existing code with the one I wrote; rather, I'm wondering if some of its ideas could be used to improve C-h m, by adjusting the layout slightly and inserting (maybe only when a user-customizable option is set?) the first line of each docstring (possibly with a button to expand the full docstring?).

Or maybe there's a better way to display this information? (one thing I did in that package was to use the order of bindings as defined in the keymap, while C-h m sorts the keybindings alphabetically; the idea was to give control to package developers over the order of the keybindings)

Cheers,
Clément.

    key             binding
    ---             -------

    C-c		Prefix Command
    ESC		Prefix Command

    C-c C-a		outline-show-all
    C-c C-b		outline-backward-same-level
    C-c C-c		outline-hide-entry
    C-c C-d		outline-hide-subtree
    C-c C-e		outline-show-entry
    C-c C-f		outline-forward-same-level
    C-c TAB		outline-show-children
    C-c C-k		outline-show-branches
    C-c C-l		outline-hide-leaves
    C-c RET		outline-insert-heading
    C-c C-n		outline-next-visible-heading
    C-c C-o		outline-hide-other
    C-c C-p		outline-previous-visible-heading
    C-c C-q		outline-hide-sublevels
    C-c C-s		outline-show-subtree
    C-c C-t		outline-hide-body
    C-c C-u		outline-up-heading
    C-c C-v		outline-move-subtree-down
    C-c C-^		outline-move-subtree-up
    C-c @		outline-mark-subtree
    C-c C-<		outline-promote
    C-c C->		outline-demote

After:

    Help with ‘outline-mode’

    ‘C-c @’ (outline-mark-subtree)
      Mark the current subtree in an outlined document.
    ‘C-c C-n’ (outline-next-visible-heading)
      Move to the next visible heading line.
    ‘C-c C-p’ (outline-previous-visible-heading)
      Move to the previous heading line.
    ‘C-c TAB’ (outline-show-children)
      Show all direct subheadings of this heading.
    ‘C-c C-s’ (outline-show-subtree)
      Show everything after this heading at deeper levels.
    ‘C-c C-d’ (outline-hide-subtree)
      Hide everything after this heading at deeper levels.
    ‘C-c C-u’ (outline-up-heading)
      Move to the visible heading line of which the present line is a subheading.
    ‘C-c C-f’ (outline-forward-same-level)
      Move forward to the ARG'th subheading at same level as this one.
    ‘C-c C-b’ (outline-backward-same-level)
      Move backward to the ARG'th subheading at same level as this one.
    ‘C-c C-t’ (outline-hide-body)
      Hide all body lines in buffer, leaving all headings visible.
    ‘C-c C-a’ (outline-show-all)
      Show all of the text in the buffer.
    ‘C-c C-c’ (outline-hide-entry)
      Hide the body directly following this heading.
    ‘C-c C-e’ (outline-show-entry)
      Show the body directly following this heading.
    ‘C-c C-l’ (outline-hide-leaves)
      Hide the body after this heading and at deeper levels.
    ‘C-c C-k’ (outline-show-branches)
      Show all subheadings of this heading, but not their bodies.
    ‘C-c C-q’ (outline-hide-sublevels)
      Hide everything but the top LEVELS levels of headers, in whole buffer.
    ‘C-c C-o’ (outline-hide-other)
      Hide everything except current body and parent and top-level headings.
    ‘C-c C-^’ (outline-move-subtree-up)
      Move the current subtree up past ARG headlines of the same level.
    ‘C-c C-v’ (outline-move-subtree-down)
      Move the current subtree down past ARG headlines of the same level.
    ‘C-c C-<’ (outline-promote)
      Promote headings higher up the tree.
    ‘C-c C->’ (outline-demote)
      Demote headings lower down the tree.
    ‘C-c RET’ (outline-insert-heading)
      Insert a new heading at same depth at point.

(in a real Emacs window, the function names and the keybindings are highlighted, and on graphical terminals there's a thin line between each entry).


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Ideas to improve the output of C-h m?

[Stefan Monnier]

> I have pasted at the botom of this email the output of the current C-h
> m and the output of the code I wrote a while ago.  Here's how it looks
> for a single entry, to see the difference (note that unlike C-h m, it
> currently groups keybindings that map to the same command):

I think grouping is a good idea, tho the current "columnar" output has
its advantages as well.

>     ‘C-c @’ (outline-mark-subtree)
>       Mark the current subtree in an outlined document.
>     ‘C-c C-n’ (outline-next-visible-heading)
>       Move to the next visible heading line.

More importantly, while I like the idea of showing the first line of the
docstring, I find the above much harder to read.  It makes the common
`C-c` and `outline-` prefixes much less obvious so you can't so easily
focus on the other parts to find what you're looking for.

IOW it's a definite improvement when you're looking for something akin
to a "quick overview" or "very short manual" that you will read from
top-to-bottom.  But if you're instead interested in looking up a table,
I think it's rather worse.

Not sure how/if we can reconcile those two kinds of uses.
Of course, we could simply do:

>     ‘C-c @’    (outline-mark-subtree)          Mark the current subtree in an outlined document.
>     ‘C-c C-n’  (outline-next-visible-heading)  Move to the next visible heading line.

and then play with the line-truncation keep it on a single line to get
the same benefit as the current text (but at the cost of pushing the
docstring to the right were it will only be visible in a very wide
window or after the user explicitly scrolls or changes the line-truncation).

> (in a real Emacs window, the function names and the keybindings are highlighted, and on graphical terminals there's a thin line between each entry).

Don't know about you, bu I'm looking at it in a real (and graphical)
Emacs window and I can't see any of those finer details ;-)


        Stefan

Re: Ideas to improve the output of C-h m?

[Clément Pit-Claudel]

[-- Attachment #1: Type: text/plain, Size: 1227 bytes --]

On 01/05/2020 11.55, Stefan Monnier wrote:
>> I have pasted at the botom of this email the output of the current C-h
>> m and the output of the code I wrote a while ago.  Here's how it looks
>> for a single entry, to see the difference (note that unlike C-h m, it
>> currently groups keybindings that map to the same command):
> 
> I think grouping is a good idea, tho the current "columnar" output has
> its advantages as well.
> 
>>     ‘C-c @’ (outline-mark-subtree)
>>       Mark the current subtree in an outlined document.
>>     ‘C-c C-n’ (outline-next-visible-heading)
>>       Move to the next visible heading line.
> 
> More importantly, while I like the idea of showing the first line of the
> docstring, I find the above much harder to read.  It makes the common
> `C-c` and `outline-` prefixes much less obvious so you can't so easily
> focus on the other parts to find what you're looking for.

That's fair, though with a bit of luck that was partly due to the text-only output.  WDYT of the attached screenshot? (of course the issue remains to some extent, but hopefully the font size, face, and spacing changes help?)
(I also removed the parentheses and the quotes, and re-added the vertical alignment)

[-- Attachment #2: light-theme.png --]
[-- Type: image/png, Size: 129812 bytes --]

[-- Attachment #3: dark-theme.png --]
[-- Type: image/png, Size: 141328 bytes --]

Re: Ideas to improve the output of C-h m?

[Stefan Monnier]

> That's fair, though with a bit of luck that was partly due to the text-only
> output.  WDYT of the attached screenshot?

Right on the money: the colors make a big difference here, thanks.

> (I also removed the parentheses and the quotes, and re-added the vertical alignment)

No opinion on the parentheses, but the alignment in indeed
very important.


        Stefan

Re: Ideas to improve the output of C-h m?

[Stefan Kangas]

Clément Pit-Claudel <cpitclaudel@gmail.com> writes:

> That's fair, though with a bit of luck that was partly due to the text-only output.  WDYT of the attached screenshot? (of course the issue remains to some extent, but hopefully the font size, face, and spacing changes help?)
> (I also removed the parentheses and the quotes, and re-added the vertical alignment)

Thank you for working on this.  I like the look of the screenshot, but
you didn't include a patch.  If you could provide one, I may be able
to give you more useful feedback on the basis of testing it.

One related idea comes to mind:

Some modes, e.g. Info-mode, currently implement their own doc string
and never bother with 'substitute-command-keys'
("\\{<foo>-mode-map}").[1]  One important motivation for doing that is
to group commands by category, something which AFAICT is not possible
with s-c-k.  (My impression is also that it sometimes is done
specifically to exclude certain key bindings from a listing, likely
because they are not perceived as integral to the functioning of the
mode.)

It would be useful, I think, if the mode author was able to customize
the listing of commands while also "automatically" using the best
standard for displaying them (with the added faces, indentation, etc.
as you suggest) rather than having to write a doc string manually to
achieve it.  For example, maybe we could come up with a way to group
commands by categories and then have them displayed under proper
headlines in key binding listings.

If you find the idea interesting, perhaps it could be a further
development of what you're proposing here.

Best regards,
Stefan Kangas

Footnotes:
1. As an aside, I have started converting 'substite-command-keys' from
C to Lisp some time ago, but I got stuck somewhere and didn't complete
it.  I hope I can find the energy and time to finish that job.

Re: Ideas to improve the output of C-h m?

[Clément Pit-Claudel]

[-- Attachment #1: Type: text/plain, Size: 2051 bytes --]

Hi Stefan,

On 01/05/2020 19.02, Stefan Kangas wrote:
> Thank you for working on this.  I like the look of the screenshot, but
> you didn't include a patch.  If you could provide one, I may be able
> to give you more useful feedback on the basis of testing it.

Thanks a lot for having a look.  I have attached the code; this version is extracted from biblio.el, for which it was originally written.  After loading it you should be able to use M-x help-with-major-mode and M-: (help-with-keymap occur-mode-map), for example.

There are shortcomings in the current implementation (it iterates over nested keymaps to flatten them, and that part is likely incomplete), including the fact that it displays <remap …> rather than actual keybindings when a keymap includes remapping.  All of these would have to be ironed out as part of turning this into a patch extending the existing facilities, of course.

> One related idea comes to mind: (…)> It would be useful, I think, if the mode author was able to customize
> the listing of commands while also "automatically" using the best
> standard for displaying them (with the added faces, indentation, etc.
> as you suggest) rather than having to write a doc string manually to
> achieve it.  For example, maybe we could come up with a way to group
> commands by categories and then have them displayed under proper
> headlines in key binding listings.

Yes, that would be great.  If the current discussion does get somewhere, it would be a natural next step to work on, I think.

> Footnotes:
> 1. As an aside, I have started converting 'substite-command-keys' from
> C to Lisp some time ago, but I got stuck somewhere and didn't complete
> it.  I hope I can find the energy and time to finish that job.

I think they may be some overlap with the code I posted, as I had to write much of the iteration over keymaps from scratch.  Maybe both implementations can benefit from each other and we can get something done together.  Or maybe I can be the first client of your new code :)

Cheers,
Clément.


[-- Attachment #2: help-with-major-mode.el --]
[-- Type: text/x-emacs-lisp, Size: 3852 bytes --]

(require 'seq)
(require 'pcase)

(defsubst biblio--as-list (x)
  "Make X a list, if it isn't."
  (if (consp x) x (list x)))

(defun biblio--map-keymap (func map)
  "Call `map-keymap' on FUNC and MAP, and collect the results."
  (let ((out))
    (map-keymap (lambda (&rest args) (push (apply func args) out)) map)
    (nreverse out)))

(defun biblio--flatten-map (keymap &optional prefix)
  "Flatten KEYMAP, prefixing its keys with PREFIX.
This should really be in Emacs core (in Elisp), instead of being
implemented in C (at least for sparse keymaps).  Don't run this on
non-sparse keymaps."
  (nreverse
   (cond
    ((keymapp keymap)
     (seq-map (lambda (key-value)
                "Add PREFIX to key in KEY-VALUE."
                (cons (append prefix (biblio--as-list (car key-value)))
                      (cdr key-value)))
              (delq nil
                    (apply
                     #'seq-concatenate
                     'list (biblio--map-keymap
                            (lambda (k v)
                              "Return a list of bindings in V, prefixed by K."
                              (biblio--flatten-map v (biblio--as-list k)))
                            keymap)))))
    ;; FIXME This breaks if keymap is a symbol whose function cell is a keymap
    ((symbolp keymap)
     (list (cons prefix keymap))))))

(defun biblio--group-alist (alist)
  "Return a copy of ALIST whose keys are lists of keys, grouped by value.
That is, if two key map to `eq' values, they are grouped."
  (let ((map (make-hash-table :test 'eq))
        (new-alist nil))
    (pcase-dolist (`(,key . ,value) alist)
      (puthash value (cons key (gethash value map)) map))
    (pcase-dolist (`(,_ . ,value) alist)
      (let ((keys (gethash value map)))
        (when keys
          (push (cons (nreverse keys) value) new-alist)
          (puthash value nil map))))
    (nreverse new-alist)))

(defun biblio--quote (str)
  "Quote STR and call `substitute-command-keys' on it."
  (if str (substitute-command-keys (concat "`" str "'")) ""))

(defun biblio--quote-keys (keys)
  "Quote and concatenate keybindings in KEYS."
  (mapconcat (lambda (keyseq)
               (let ((key (ignore-errors (help-key-description keyseq nil))))
                 (if (and nil key (string-match-p " " key))
                     (biblio--quote key)
                   key)))
             keys ", "))

(defun biblio--brief-docs (command)
  "Return first line of documentation of COMMAND."
  (let ((docs (or (ignore-errors (documentation command t)) "")))
    (string-match "\\(.*\\)$" docs)
    (match-string-no-properties 1 docs)))

(defun biblio--help-with-major-mode-1 (keyseqs-command)
  "Print help on KEYSEQS-COMMAND to standard output."
  ;; (biblio-with-fontification 'font-lock-function-name-face
  (insert (biblio--quote-keys (car keyseqs-command)) " ")
  (insert (propertize "\t" 'display '(space :align-to 10)))
  ;; (insert "(")
  (insert-text-button (format "%S" (cdr keyseqs-command)))
  ;; (insert ")")
  (insert "\n")
  (insert (propertize (format "  %s\n" (biblio--brief-docs (cdr keyseqs-command)))
                      'face '(font-lock-comment-face (:height 0.95))))
  (insert (propertize "\n" 'face '(:height 0.3))))

(defun help-with-keymap (map &optional buf title)
  "Display help for keymap MAP in buffer BUF, with a given TITLE."
  (setq buf (or buf "*Keymap help*"))
  (with-help-window buf
    (when title (princ title))
    (let ((bindings (nreverse (biblio--group-alist (biblio--flatten-map map)))))
      (with-current-buffer buf
        (seq-do #'biblio--help-with-major-mode-1 bindings)))))

(defun help-with-major-mode ()
  "Display help for current major mode."
  (interactive)
  (help-with-keymap
   (current-local-map)
   (format "*%S help*" major-mode)
   (format "Help with %s\n\n" (biblio--quote (symbol-name major-mode)))))

Re: Ideas to improve the output of C-h m?

[Jean-Christophe Helary]

> On May 1, 2020, at 23:34, Clément Pit-Claudel <clement.pitclaudel@live.com> wrote:
> 
> Hi all,
> 
> A while ago I tried to improve the discoverability of the keybindings of one of my packages by writing an enhanced version of the keybindings part of C-h m.  The idea was to list keybindings along with the first line of their docstrings, instead of just listing the name of the function.

This discussion, along with the one about transient about mode remind me of what which-key and helpful do. Those are not perfect solutions but they are pretty good considering the state of affairs in terms of discoverability.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Ideas to improve the output of C-h m?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Some modes, e.g. Info-mode, currently implement their own doc string
  > and never bother with 'substitute-command-keys'
  > ("\\{<foo>-mode-map}").[1]

When a doc string is long, or when the mode has many commands,
the current way of displaying the list of commands is inconvenient.
I think it would be good to develop a better way.

  > It would be useful, I think, if the mode author was able to customize
  > the listing of commands while also "automatically" using the best
  > standard for displaying them (with the added faces, indentation, etc.
  > as you suggest) rather than having to write a doc string manually to
  > achieve it.  For example, maybe we could come up with a way to group
  > commands by categories and then have them displayed under proper
  > headlines in key binding listings.

I agree.

Instead of including the whole list directly in the displayed doc string,
it might be better to have a link you can follow to see that whole list.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Ideas to improve the output of C-h m?

[Stefan Kangas]

Clément Pit-Claudel <cpitclaudel@gmail.com> writes:

> Thanks a lot for having a look.  I have attached the code; this
> version is extracted from biblio.el, for which it was originally
> written.  After loading it you should be able to use M-x
> help-with-major-mode and M-: (help-with-keymap occur-mode-map), for
> example.

Sorry for the delayed response.  I will try to take a proper look in
the next couple of days.

>> 1. As an aside, I have started converting 'substite-command-keys' from
>> C to Lisp some time ago, but I got stuck somewhere and didn't complete
>> it.  I hope I can find the energy and time to finish that job.
>
> I think they may be some overlap with the code I posted, as I had to
> write much of the iteration over keymaps from scratch.  Maybe both
> implementations can benefit from each other and we can get something
> done together.  Or maybe I can be the first client of your new code
> :)

Sounds good.  It will be interesting to see the overlap indeed, and it
sure is more motivating to work on the C to Lisp conversion if it can
actually be useful in terms of enabling improvements.

I suggest that I put my code on a branch, and we can take it from
there.  I have some loose ends to tie up before I can get back to it,
so please don't let that hold you back from what you're doing here.

Best regards,
Stefan Kangas

scratch/substitute-command-keys: C conversion of s-c-k

[Stefan Kangas]

Stefan Kangas <stefan@marxist.se> writes:

> Please find attached the first in a planned series of patches to replace
> `substitute-command-keys' with a Lisp version.
>
> - The goal is to produce byte-for-byte identical output to the old C
>   version.
>
> - This is a more or less direct translation of the C code, and so is not
>   very elegant Lisp.  It turned out to be easier to just "lift" the code
>   from C before working on any improvements.  (The code is not super
>   involved overall, but some details are a bit finicky.)

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Please find attached the first in a planned series of patches to replace
>> `substitute-command-keys' with a Lisp version.
>
> The strategy sounds good (I followed basically the same principles in
> various other rewrites from C to Elisp ;-).
>
> I don't know what Eli wants to do, but I think the better option is to
> push this to a branch where you can have this change followed by a few
> others, and when the patches's sum is clean we can merge it to `master`.

[The above discussion apparently never included emacs-devel.]

I now have a working translation of 'substitute-command-keys' from C to
Lisp on the scratch/substitute-command-keys branch.  For more background
see Bug#8951 and other bugs regarding s-c-k.

I have added some defsubrs that could be removed by writing new Lisp
equivalents.  It would be good if I could get advice on which is better
here:

   1) Just leave the defsubrs as is.
   2) Replace them with defun's in help.el and then either:
       a) Accept duplicate code on the C and Lisp level.
       b) Call the Lisp functions from C code, and remove the C
          functions altogether.

I also made some measurements to confirm that we get insubstantial
performance benefits by keepin 'substitute-command-keys' in C:

    (progn
      (with-temp-buffer (org-mode))
      (let* ((times 1000)
             (new (car (benchmark-run times
                         (substitute-command-keys "\\{org-mode-map}"))))
             (old (car (benchmark-run times
                         (substitute-command-keys-old "\\{org-mode-map}")))))
        (/ (- new old) times)))

    => 0.0010306464810000003

Looking around keymap.c got me thinking that maybe there is more stuff
that could be usefully lifted to Lisp there.  Obviously this would only
apply to functions that are called infrequently.  (Perhaps there is even
a case to be made for a new keymap.el where we could put this logically.)

Or maybe this is exactly the point where I stop, remove the C
implementation of 'substitute-command-keys', do some last cleanups, and
get all this merged.

Comments on all this are more than welcome.  Thanks in advance!

Best regards,
Stefan Kangas