Delegating user-reserved key binding space definition to users

Delegating user-reserved key binding space definition to users

[Psionic K]

[-- Attachment #1: Type: text/plain, Size: 2463 bytes --]

The purpose of this email is to identify if a package should be created or
if the technical implementation requires modification to Emacs.

The motivating problem was to eliminate the possibility of Emacs defaults
or packages creating many bindings sequences in two primary categories:

   - Multiple modifiers such as M-S-* or long sequences such as C-* C-* *.
   M-? is also an example of what I consider to require multiple-modifiers on
   a US key layout where '?' requires shift.
   - High-value space such as M-* and C-*

In order to enforce this choice, the user needs control over the effects of
all calls to create bindings, from an early phase and then persistently
through startup and future package loading.  The user needs a way to
express, in the same dimensions as the implementation, which keys should
receive no-op bindings if not explicitly created by the user.  A
complementary set of functions should allow the user to then make bindings
into the protected keyspace, free from any potential interference.

I was asked to relay my opinion that Emacs should ship with bindings not
much more densely populated than nano.  Nano does not have commands, so
Emacs needs M-x, for example.

It was suggested that advice might work for implementation, up to a point,
since define-key doesn't have a dedicated code of some type I'm unaware of.

The suggestion was made to live with emulation mode maps or even terminal
overriding maps.  I have two concerns about this technically.  Such a
solution provides no consultative feedback, a blind solution.  Second, it
doesn't help users who are already leaning on such maps for modal editing.

My personal preference is to find a path to a mostly complete
implementation as an independent package in order to figure out the problem
better, gaining the breadth of information necessary through incremental
adoption.

I am intending to explore a simple advice of define-key.  My implementation
will no-op any request to bind a command to a key sequence that doesn't
pass a predicate list.  Optionally a report will be stored about which
command bindings were no-op'd and by which filter predicate.  Support for
command and key translation inside of predicates would be my next feature.

If there are multiple key mechanisms I need to intercept, an enumeration of
them will be greatly appreciated.

-- 

Psionic K <psionik@positron.solutions>
Software Engineer

*Positron Solutions <https://positron.solutions>*

[-- Attachment #2: Type: text/html, Size: 3574 bytes --]

Re: Delegating user-reserved key binding space definition to users

[Stefan Monnier]

>    - Multiple modifiers such as M-S-* or long sequences such as C-* C-* *.
>    M-? is also an example of what I consider to require multiple-modifiers on
>    a US key layout where '?' requires shift.
>    - High-value space such as M-* and C-*
[...]
> It was suggested that advice might work for implementation, up to a point,
> since define-key doesn't have a dedicated code of some type I'm unaware of.

Doing it inside `define-key` is problematic because you can bind a key
to `C-a C-b C-c` via three calls to `define-key`, so you won't be able
to detect the problem reliably in all cases:

    (let* ((map1 (make-sparse-keymap))
           (map2 (make-sparse-keymap))
           (map3 (make-sparse-keymap)))
      (define-key map1 [?\C-a] map2)
      (define-key map2 [?\C-b] map3)
      (define-key map3 [?\C-c] 'foo))


-- Stefan

Re: Delegating user-reserved key binding space definition to users

[Phil Sainty]

AFAIK...

You can't arbitrarily prevent key sequences from being bound in
keymaps, because you can't control how or when that happens.

You can't control when it happens because that time might be "before
you started (or even installed) Emacs".  Keymaps are sometimes
generated at byte-compilation time, and thus even if you were to take
the extreme measure of preventing define-key from doing anything at
all, you will still acquire populated keymaps when certain libraries
are loaded (although if the libraries were byte-compiled while the
neutralised define-key was in place, you would then typically be
loading empty keymaps; but you would need to recompile everything to
get to that point; and of course most installations of Emacs will
include pre-compiled .elc files).

Native compilation might preclude even that, as IIRC it compiles
everything asynchronously in isolation, so the intended clobbering
of define-key might not actually be in effect when the native code
was being generated.  You would then have to edit the core code to
enforce your override irrespective of whether your custom code was
loaded.

You can't control how it happens because libraries can set bindings
without using define-key.  In the simplest case `use-local-map' for
one of the aforementioned pre-generated keymaps may easily occur; but
also a keymap is, after all, just a list, so it can be manipulated in
any number of ways.

(In practice I think that messing with define-key and recompiling All
Of The Things would affect almost all keymaps; but in general there's
no guarantee.)

I would suggest that your best option is to instead ensure that the
key lookup sequence always finds your preferred bindings.  Despite
your concerns about that approach, it seems more viable to me.

Re: Delegating user-reserved key binding space definition to users

[Psionic K]

[-- Attachment #1: Type: text/plain, Size: 7095 bytes --]

The problem I'm working to reach is allowing packages to correctly shadow
the user's bindings (and not incorrectly shadow) instead of ad-hoc
imitation of the global map defaults.  Ad-hoc imitation makes it very
tedious to, for instance, change the key sequence for expressing "next"
without losing coherence among all packages with a corresponding "next"
concept.

The solution we're discussing is to provide "abstract" commands.  I will
discuss the solution and then discuss what I'm currently working on.

Abstract commands are just command remap targets in the global map.  The
concrete global commands would directly command remap in the global map.
Abstract command bindings would be consumed when modes generate keymaps for
major & minor modes.  A Corfu map generator would for example see an
abstract command like "user-next" bound with the C-n sequence and choose to
shadow C-n in its map.  If the user rebinds the global abstract command,
the other modes could easily follow this change, perhaps even though a hook
to immediately propagate the change.

This scheme would scale much better than imitating global map defaults.
Any user or emulation system desiring to re-bind basic concepts like next
and undo could achieve and maintain coherence.  Such a mechanism should
also make it easier to maintain modes in the future.  Expressing abstract
commands could be a basis for coordinating special mode behavior in ways
that would currently be likely to break with many users' modifications.

I will not introduce a modal system I've worked on, but it has a quirky
notion of next / previous, and yet it will adapt fine to abstract key
bindings.  When switching keyboards to an alternative layout, I lose
physical correspondence.  "J" is a good high-frequency binding unless "J"
is moved to a physically inconvenient location.  Switching abstract maps
and regenerating is a solution for this as well.

Expressing "never shadow this key" would need a distinct map from the
global map.  The problem is that an abstract command like "user-nomap"
would not distinctly map to a concrete command if 20 sequences were to be
protected from shadowing.  There needs to be a global-no-map where all
sequences bound to user-nomap are prohibited by convention.

For generating maps where a binding would collide with an abstract binding,
a configurable successor function could be made available.  It could be set
to nil or return nil to flatly reject all sequence suggestions that were
bad to begin with instead of finding places to place every command.  We
have really good command completions these days, so personally I would opt
for a nil successor.

Abstract key definition would be adopted by first providing map generators
for many modes (similar to Evil collection) and then expecting those map
generators to migrate out to the modes themselves if the abstract map
support were to be adopted into Emacs itself.

Regarding where I'm starting out:

I'm working on a more modest and immediate goal of depopulating unwanted
bindings, focusing on overly complex or high-priority bindings first.  I
will define my future abstract keys implementation in the same package.
While users in the future should not have to battle with so many bindings
in the many mode maps, for now it will be appropriate to generate
mass-unbind statements for packages like general to consume.

I'm using a report-and-configure workflow.  This is the lowest effort
implementation to identify "bad" bindings and provide a fast remedy to the
user.  While I can see many maps in for example minor-mode-map-alist, I was
unsure of where to find a list of major modes except from inferring while
using mapatoms.  I could notice the cost of calling this to find major
modes / maps.

State tracking is necessary to differentiate user defined bindings from
defaults.  While I had not wanted to add the complexity, inheriting keymaps
may provide an elegant solution.  My hunch is to first destructively clean
out default maps and then define inheriting maps and swap the inheriting
maps into the keymap variables.  If the user adds bindings afterward, it's
almost clearly visible.  Is there a way I could add a sentinel to an
inherited map to indicate that I've cleaned the parent map already?  What
are some good choices for list elements of no consequence?  This would also
tell me that the child was created for the user.  I don't want to implement
package-private state tracking if possible.

I have no ambition to work with non-list maps at this time.  So far every
map I want to work with is just a plain list.

While working on my implementation, I have encountered the kinds of issue
that make me believe that alternatives such as key-sequence translation are
also not scalable.  (key-binding "\M-g") is incorrect after remapping M-g
to C-g unless I collect a key sequence interactively.   Keeping this kind
of indirection around has consequences.  I believe key sequence remapping
is a solution more aimed at misbehaving input systems where correct
bindings cannot accommodate the situation.

On Mon, Nov 21, 2022 at 8:07 PM Phil Sainty <psainty@orcon.net.nz> wrote:

> AFAIK...
>
> You can't arbitrarily prevent key sequences from being bound in
> keymaps, because you can't control how or when that happens.
>
> You can't control when it happens because that time might be "before
> you started (or even installed) Emacs".  Keymaps are sometimes
> generated at byte-compilation time, and thus even if you were to take
> the extreme measure of preventing define-key from doing anything at
> all, you will still acquire populated keymaps when certain libraries
> are loaded (although if the libraries were byte-compiled while the
> neutralised define-key was in place, you would then typically be
> loading empty keymaps; but you would need to recompile everything to
> get to that point; and of course most installations of Emacs will
> include pre-compiled .elc files).
>
> Native compilation might preclude even that, as IIRC it compiles
> everything asynchronously in isolation, so the intended clobbering
> of define-key might not actually be in effect when the native code
> was being generated.  You would then have to edit the core code to
> enforce your override irrespective of whether your custom code was
> loaded.
>
> You can't control how it happens because libraries can set bindings
> without using define-key.  In the simplest case `use-local-map' for
> one of the aforementioned pre-generated keymaps may easily occur; but
> also a keymap is, after all, just a list, so it can be manipulated in
> any number of ways.
>
> (In practice I think that messing with define-key and recompiling All
> Of The Things would affect almost all keymaps; but in general there's
> no guarantee.)
>
> I would suggest that your best option is to instead ensure that the
> key lookup sequence always finds your preferred bindings.  Despite
> your concerns about that approach, it seems more viable to me.
>
>

-- 

Psionic K <psionik@positron.solutions>
Software Engineer

*Positron Solutions <https://positron.solutions>*

[-- Attachment #2: Type: text/html, Size: 8757 bytes --]

Re: Delegating user-reserved key binding space definition to users

[Ihor Radchenko]

Psionic K <psionik@positron.solutions> writes:

> Abstract commands are just command remap targets in the global map.  The
> concrete global commands would directly command remap in the global map.
> Abstract command bindings would be consumed when modes generate keymaps for
> major & minor modes.  A Corfu map generator would for example see an
> abstract command like "user-next" bound with the C-n sequence and choose to
> shadow C-n in its map.  If the user rebinds the global abstract command,
> the other modes could easily follow this change, perhaps even though a hook
> to immediately propagate the change.

+1

Anyone who tried to re-bind default bindings for, say, C-n/C-p/n/p might
be familiar what the problem with current Emacs approach is. It is very
annoying to keep tweaking every single package bindings out there and
also deal with cascading key collisions.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Delegating user-reserved key binding space definition to users

[xenodasein--- via Emacs development discussions.]

Nov 25, 2022, 03:31 by yantar92@posteo.net:

> Psionic K <psionik@positron.solutions> writes:
>
>> Abstract commands are just command remap targets in the global map.  The
>> concrete global commands would directly command remap in the global map.
>> Abstract command bindings would be consumed when modes generate keymaps for
>> major & minor modes.  A Corfu map generator would for example see an
>> abstract command like "user-next" bound with the C-n sequence and choose to
>> shadow C-n in its map.  If the user rebinds the global abstract command,
>> the other modes could easily follow this change, perhaps even though a hook
>> to immediately propagate the change.
>>
>
> +1
>
> Anyone who tried to re-bind default bindings for, say, C-n/C-p/n/p might
> be familiar what the problem with current Emacs approach is. It is very
> annoying to keep tweaking every single package bindings out there and
> also deal with cascading key collisions.
>
> -- 
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <> https://liberapay.com/yantar92> >
>

Easy to configure key bindings?  Ugh who needs that, the bindings are
already perfect.  Emacs -Q is the supreme experience created in the
vein as a physics experiment colliding as many things as possible, so
everything has to be optimized for it's out of box magnificence!

Re: Delegating user-reserved key binding space definition to users

[Stefan Monnier]

> The problem I'm working to reach is allowing packages to correctly shadow
> the user's bindings (and not incorrectly shadow) instead of ad-hoc
> imitation of the global map defaults.  Ad-hoc imitation makes it very
> tedious to, for instance, change the key sequence for expressing "next"
> without losing coherence among all packages with a corresponding "next"
> concept.

I don't think anyone doubts that it's desirable.
The question is how to get that.


        Stefan

Re: Delegating user-reserved key binding space definition to users

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> The problem I'm working to reach is allowing packages to correctly shadow
>> the user's bindings (and not incorrectly shadow) instead of ad-hoc
>> imitation of the global map defaults.  Ad-hoc imitation makes it very
>> tedious to, for instance, change the key sequence for expressing "next"
>> without losing coherence among all packages with a corresponding "next"
>> concept.
>
> I don't think anyone doubts that it's desirable.
> The question is how to get that.

I suggest introducing a notion of "generalized" commands. Such commands
will represent common actions executed by users (like move to
next/previous element). Major and minor modes can then define specific
implementation of the actions.

In terms of OOP, I am talking about virtual methods and major/minor modes
providing the implementations. The implementations will be chosen
according to the current major mode or maybe some other condition.
Default implementation (like `next-line') can also be provided.

The generalized commands can then be freely re-bound by users with
immediate effect on all the major modes.

An example (not very clean) implementation of this concept is available
at
https://github.com/yantar92/meta-functions/blob/master/meta-functions.el:

; Quick implementation of meta-functions, which allows running multiple
;; functions, which do similar logical operations by one single "meta"
;; function in different major modes. For example, one may want to define
;; meta-next-line to call `next-line' normally, but
;; `org-agenda-next-line' in org-agenda mode. Both can be bound to, say
;; "M-j", without a need to change the key-bindings on both fundamental
;; and org-agenda modes.
;; Example usage:
;; (use-package meta-functions
;;   :config
;; (meta-defun meta-next-element ()
;;   "Go to next element."
;;   :mode org-agenda-mode (org-agenda-next-item 1) ; call org-agenda-next-item in org-agenda
;;   :cond (lambda () (and (eq major-mode 'org-mode) (org-at-heading-p))) (org-next-visible-heading 1) ; call org-next-visible-headting when at heading in org-mode
;;   (next-line)) ; call-next line in any other case
;; )
;; or the same can be written as
;; (meta-defun meta-next-element "Go to next element." next-line)
;; (meta-defun meta-next-element :mode org-agenda-mode (org-agenda-next-item 1))
;; (meta-defun meta-next-element :mode org-mode :cond org-at-heading-p (org-next-visible-heading 1))
;;
;; Similar packages:
;; https://gitlab.com/jjzmajic/handle implements similar functionality to unify functions across major modes
;; However, handle is very major-mode centric. meta-functions is more function-centric
;; Also, meta-functions is not only limited to current major mode defining which function to call.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Delegating user-reserved key binding space definition to users

[Stefan Monnier]

> I suggest introducing a notion of "generalized" commands. Such commands
> will represent common actions executed by users (like move to
> next/previous element). Major and minor modes can then define specific
> implementation of the actions.

I think it's a non-starter because it requires foresight: only those
commands defined with this mechanism will be extensible.  I agree that
an additional level of indirection is probably necessary, but I suspect
it needs to be placed elsewhere.

[ FWIW, you can get similar results with the current setup using
  command remapping.  ]


        Stefan

Re: Delegating user-reserved key binding space definition to users

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I suggest introducing a notion of "generalized" commands. Such commands
>> will represent common actions executed by users (like move to
>> next/previous element). Major and minor modes can then define specific
>> implementation of the actions.
>
> I think it's a non-starter because it requires foresight: only those
> commands defined with this mechanism will be extensible.  I agree that
> an additional level of indirection is probably necessary, but I suspect
> it needs to be placed elsewhere.

Auto-remapping will need some kind of grouping for commands one way or
another. There is no way we can do it auto-magically. Developer or users
should decide.

Currently, the commands in major mods are bound to specific key
bindings. The bindings are chosen either arbitrarily, according to major
mode author preferences, or according to semi-established default key
binding scheme (like C-f/C-M-f/C-n/C-v/etc). Either way, trying to
re-bind commands in multiple major modes is not easy.

Note that a number of commands like `comment-forward' already expect
major modes to fit into an established, pre-defined framework by
tweaking the `comment-forward' customization according to the major
mode. This also requires a foresight.

My suggestion is somewhat similar to the existing practices of major
mode writing where the author is required to configure comment handling,
paragraph handling, sentence rules, imenu, etc.

However, what I suggest is more flexible as it does not require Emacs
core developers to provide extensive configuration mechanism for, say,
`next-line' individually; `previous-line' individually, ...
Instead, it will be sufficient to declare generalized command and then
rely on major modes to hook in. Users should be able to do it easily
too.

> [ FWIW, you can get similar results with the current setup using
>   command remapping.  ]

You are absolutely right: when a major mode command is related to
built-in command in command map.

However, some major modes introduce new concepts. For example, think
about paredit-forward-slurp-sexp, which can be an equivalent of Org's
heading promotion or moving word at point forward in sentences. How
could you remap to group these 3 very different yet similar (for some
users) commands together?

I imagine that users can define a generalized command "move right"
themselves, give it a default binding in overriding global map, and then
make `paredit-forward-slurp-sexp', `org-promote', and `transpose-words'
be selected when running "move right" depending on the active major
mode. You cannot achieve this by simple remapping, AFAIK.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Delegating user-reserved key binding space definition to users

[Psionic K]

[-- Attachment #1: Type: text/plain, Size: 4846 bytes --]

I tested remap shadowing just now, and it works in 28.2.  A higher
precedence remap will shadow a lower precedence remap.  This means the user
can bind a sequence to an abstract command, give it a remap in the global
map, and have a different remap in a mode map.  When the user rebinds the
abstract command, everything goes with it.

> I suggest introducing a notion of "generalized" commands. Such commands
will represent common actions executed by users (like move to
next/previous element). Major and minor modes can then define specific
implementation of the actions.

We can do this with shadowing in higher precedence maps.  As above, this
works for command remapping, but the issue is that we need a solid place to
remap to, not a command that the user might replace.  If the user defines a
command like "next-line-unfold" in the the global map, a mode generating a
map by looking for "next-line" would not find the correct target.

This is the situation today, with ad-hoc imitation of the default global
map.  Modes map to sequences without knowing what the user wants those
sequences to be.  We basically need a convention to name the keys through
abstract commands instead of using sequences, which really are zero
indication of what the user wants that key to do.  Binding sequences to
abstract commands completely solves this.

>You cannot achieve this by simple remapping, AFAIK.

Each major mode map would shadow the global map's shadow of the
"user-right" abstract command.  Instead of the abstract command sequence
remapping to the global concrete command, the major mode's concrete command
remap would take precedence.  Totally works.


On Sat, Nov 26, 2022 at 11:44 PM Ihor Radchenko <yantar92@posteo.net> wrote:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> >> I suggest introducing a notion of "generalized" commands. Such commands
> >> will represent common actions executed by users (like move to
> >> next/previous element). Major and minor modes can then define specific
> >> implementation of the actions.
> >
> > I think it's a non-starter because it requires foresight: only those
> > commands defined with this mechanism will be extensible.  I agree that
> > an additional level of indirection is probably necessary, but I suspect
> > it needs to be placed elsewhere.
>
> Auto-remapping will need some kind of grouping for commands one way or
> another. There is no way we can do it auto-magically. Developer or users
> should decide.
>
> Currently, the commands in major mods are bound to specific key
> bindings. The bindings are chosen either arbitrarily, according to major
> mode author preferences, or according to semi-established default key
> binding scheme (like C-f/C-M-f/C-n/C-v/etc). Either way, trying to
> re-bind commands in multiple major modes is not easy.
>
> Note that a number of commands like `comment-forward' already expect
> major modes to fit into an established, pre-defined framework by
> tweaking the `comment-forward' customization according to the major
> mode. This also requires a foresight.
>
> My suggestion is somewhat similar to the existing practices of major
> mode writing where the author is required to configure comment handling,
> paragraph handling, sentence rules, imenu, etc.
>
> However, what I suggest is more flexible as it does not require Emacs
> core developers to provide extensive configuration mechanism for, say,
> `next-line' individually; `previous-line' individually, ...
> Instead, it will be sufficient to declare generalized command and then
> rely on major modes to hook in. Users should be able to do it easily
> too.
>
> > [ FWIW, you can get similar results with the current setup using
> >   command remapping.  ]
>
> You are absolutely right: when a major mode command is related to
> built-in command in command map.
>
> However, some major modes introduce new concepts. For example, think
> about paredit-forward-slurp-sexp, which can be an equivalent of Org's
> heading promotion or moving word at point forward in sentences. How
> could you remap to group these 3 very different yet similar (for some
> users) commands together?
>
> I imagine that users can define a generalized command "move right"
> themselves, give it a default binding in overriding global map, and then
> make `paredit-forward-slurp-sexp', `org-promote', and `transpose-words'
> be selected when running "move right" depending on the active major
> mode. You cannot achieve this by simple remapping, AFAIK.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>


-- 

Psionic K <psionik@positron.solutions>
Software Engineer

*Positron Solutions <https://positron.solutions>*

[-- Attachment #2: Type: text/html, Size: 6774 bytes --]

Re: Delegating user-reserved key binding space definition to users

[Psionic K]

[-- Attachment #1: Type: text/plain, Size: 6885 bytes --]

At this point, I'm thinking about supporting continuity and adoption.  Some
users will want to change and some will want to stay the same.

The short answer is not to define any abstract commands or forbidden /
reserved sequences if you want the current behavior.  We can start by only
defaulting abstract commands that are effectively in consensus already.

After adding the obvious abstract commands, there's not many bindings that
map to consistent concepts I can think of. However, this is not the problem
it seems like.  While a major mode author may wish to create more bindings,
they are not entitled to any of the user's keyspace.

It will suffice for mode writers to know which key sequences are:

   - designated for a purpose in other modes
   - designated for use by the user
   - never to be mapped under any circumstances

At the time of map generation, if a mode finishes mapping designated
abstract keys, they must choose keys that don't match the other two
groups.  These bindings may all be suboptimal or too few to implement the
mode writer's wishes, but this is the user's choice.  A user-overridable
sequence successor function could add bindings to a set of valid
sequences.  If a mode needs more than just a single prefix key, this should
suffice, and the user can define that function to nil to block all
extraneous bindings.

Mode authors should focus on introducing their commands through README's,
good docstrings for completions, and good command names.  Bindings are not
a good index of discovery for commands.  If a mode has a well implemented
data model and set of functions for building commands, this is much more
useful than having a bunch of maybe useful commands bound.

On Sun, Nov 27, 2022 at 5:26 AM Psionic K <psionik@positron.solutions>
wrote:

> I tested remap shadowing just now, and it works in 28.2.  A higher
> precedence remap will shadow a lower precedence remap.  This means the user
> can bind a sequence to an abstract command, give it a remap in the global
> map, and have a different remap in a mode map.  When the user rebinds the
> abstract command, everything goes with it.
>
> > I suggest introducing a notion of "generalized" commands. Such commands
> will represent common actions executed by users (like move to
> next/previous element). Major and minor modes can then define specific
> implementation of the actions.
>
> We can do this with shadowing in higher precedence maps.  As above, this
> works for command remapping, but the issue is that we need a solid place to
> remap to, not a command that the user might replace.  If the user defines a
> command like "next-line-unfold" in the the global map, a mode generating a
> map by looking for "next-line" would not find the correct target.
>
> This is the situation today, with ad-hoc imitation of the default global
> map.  Modes map to sequences without knowing what the user wants those
> sequences to be.  We basically need a convention to name the keys through
> abstract commands instead of using sequences, which really are zero
> indication of what the user wants that key to do.  Binding sequences to
> abstract commands completely solves this.
>
> >You cannot achieve this by simple remapping, AFAIK.
>
> Each major mode map would shadow the global map's shadow of the
> "user-right" abstract command.  Instead of the abstract command sequence
> remapping to the global concrete command, the major mode's concrete command
> remap would take precedence.  Totally works.
>
>
> On Sat, Nov 26, 2022 at 11:44 PM Ihor Radchenko <yantar92@posteo.net>
> wrote:
>
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>
>> >> I suggest introducing a notion of "generalized" commands. Such commands
>> >> will represent common actions executed by users (like move to
>> >> next/previous element). Major and minor modes can then define specific
>> >> implementation of the actions.
>> >
>> > I think it's a non-starter because it requires foresight: only those
>> > commands defined with this mechanism will be extensible.  I agree that
>> > an additional level of indirection is probably necessary, but I suspect
>> > it needs to be placed elsewhere.
>>
>> Auto-remapping will need some kind of grouping for commands one way or
>> another. There is no way we can do it auto-magically. Developer or users
>> should decide.
>>
>> Currently, the commands in major mods are bound to specific key
>> bindings. The bindings are chosen either arbitrarily, according to major
>> mode author preferences, or according to semi-established default key
>> binding scheme (like C-f/C-M-f/C-n/C-v/etc). Either way, trying to
>> re-bind commands in multiple major modes is not easy.
>>
>> Note that a number of commands like `comment-forward' already expect
>> major modes to fit into an established, pre-defined framework by
>> tweaking the `comment-forward' customization according to the major
>> mode. This also requires a foresight.
>>
>> My suggestion is somewhat similar to the existing practices of major
>> mode writing where the author is required to configure comment handling,
>> paragraph handling, sentence rules, imenu, etc.
>>
>> However, what I suggest is more flexible as it does not require Emacs
>> core developers to provide extensive configuration mechanism for, say,
>> `next-line' individually; `previous-line' individually, ...
>> Instead, it will be sufficient to declare generalized command and then
>> rely on major modes to hook in. Users should be able to do it easily
>> too.
>>
>> > [ FWIW, you can get similar results with the current setup using
>> >   command remapping.  ]
>>
>> You are absolutely right: when a major mode command is related to
>> built-in command in command map.
>>
>> However, some major modes introduce new concepts. For example, think
>> about paredit-forward-slurp-sexp, which can be an equivalent of Org's
>> heading promotion or moving word at point forward in sentences. How
>> could you remap to group these 3 very different yet similar (for some
>> users) commands together?
>>
>> I imagine that users can define a generalized command "move right"
>> themselves, give it a default binding in overriding global map, and then
>> make `paredit-forward-slurp-sexp', `org-promote', and `transpose-words'
>> be selected when running "move right" depending on the active major
>> mode. You cannot achieve this by simple remapping, AFAIK.
>>
>> --
>> Ihor Radchenko // yantar92,
>> Org mode contributor,
>> Learn more about Org mode at <https://orgmode.org/>.
>> Support Org development at <https://liberapay.com/org-mode>,
>> or support my work at <https://liberapay.com/yantar92>
>>
>
>
> --
>
> Psionic K <psionik@positron.solutions>
> Software Engineer
>
> *Positron Solutions <https://positron.solutions>*
>


-- 

Psionic K <psionik@positron.solutions>
Software Engineer

*Positron Solutions <https://positron.solutions>*

[-- Attachment #2: Type: text/html, Size: 9842 bytes --]

Re: Delegating user-reserved key binding space definition to users

[Stefan Monnier]

>>> I suggest introducing a notion of "generalized" commands. Such commands
>>> will represent common actions executed by users (like move to
>>> next/previous element). Major and minor modes can then define specific
>>> implementation of the actions.
>>
>> I think it's a non-starter because it requires foresight: only those
>> commands defined with this mechanism will be extensible.  I agree that
>> an additional level of indirection is probably necessary, but I suspect
>> it needs to be placed elsewhere.
>
> Auto-remapping will need some kind of grouping for commands one way or
> another. There is no way we can do it auto-magically. Developer or users
> should decide.

Agreed.  But it should apply to all bindings.

> Currently, the commands in major mods are bound to specific key
> bindings. The bindings are chosen either arbitrarily, according to major
> mode author preferences, or according to semi-established default key
> binding scheme (like C-f/C-M-f/C-n/C-v/etc). Either way, trying to
> re-bind commands in multiple major modes is not easy.

Yup.  I fully agree that it's a real problem.

I'd welcome a solution to it.  Even an "unrealistic" one would be good.
Currently, I don't even know what a good solution could look like.

To me a solution should allow packages to declare that command FOO
should be bound to some key based on SOME-INFO, such that it will be
bound to one key in "normal Emacs mode", and to another in `evil-mode`
and to yet another in `god-mode`, etc...

Some SOME-INFO needs to provide not a specific key, but some information
from which we can compute the "natural key" that fits the keybinding
style that the user selected.

Then there are also the issues of overloading several operations on
a single key (like TAB).  So far we've solved this along the lines you
suggest (a "generalized command", such as `indent-for-tab-command`), and
maybe that's good enough for this, tho it prevents changing this
overloading according to the keybinding style.

[ BTW, another way to look at it is not "how can we compute which key to
  bind FOO to" but rather "how can we compute which command to run when
  KEY is hit".  IOW, we could make keymaps more dynamic such that when
  you hit KEY, Emacs passes that to a "procedural keymap" which will
  *compute* (rather than lookup) which command to run, according to the
  current keybinding style.
  Not sure it would be better: I just mention it as one of the many
  things that we may want to consider in order to find a good solution
  to the problem.  ]

>> [ FWIW, you can get similar results with the current setup using
>>   command remapping.  ]
> You are absolutely right: when a major mode command is related to
> built-in command in command map.

Just to be clear: I don't think command remapping solves the problem
at hand.  It can sometimes be used for that, but not in general.

> However, some major modes introduce new concepts.  For example, think
> about paredit-forward-slurp-sexp, which can be an equivalent of Org's
> heading promotion or moving word at point forward in sentences.  How
> could you remap to group these 3 very different yet similar (for some
> users) commands together?

Great example, indeed, thanks.

In this context, I'd like the package to be able to explain that this
command is about a "sexp"-granularity operation and about
"forward"-direction operation, so that the keybinding style may
automatically find a natural/consistent key (or set of keys) to use for
it depending on whether the current keybinding style uses f/b, or j/k,
or left/right, or ... for forward/backward operations.


        Stefan

Re: Delegating user-reserved key binding space definition to users

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Psionic K <psionik@positron.solutions>,  Emacs developers
>  <emacs-devel@gnu.org>
> Date: Mon, 28 Nov 2022 13:15:26 -0500
> 
> > Currently, the commands in major mods are bound to specific key
> > bindings. The bindings are chosen either arbitrarily, according to major
> > mode author preferences, or according to semi-established default key
> > binding scheme (like C-f/C-M-f/C-n/C-v/etc). Either way, trying to
> > re-bind commands in multiple major modes is not easy.
> 
> Yup.  I fully agree that it's a real problem.
> 
> I'd welcome a solution to it.  Even an "unrealistic" one would be good.
> Currently, I don't even know what a good solution could look like.
> 
> To me a solution should allow packages to declare that command FOO
> should be bound to some key based on SOME-INFO, such that it will be
> bound to one key in "normal Emacs mode", and to another in `evil-mode`
> and to yet another in `god-mode`, etc...
> 
> Some SOME-INFO needs to provide not a specific key, but some information
> from which we can compute the "natural key" that fits the keybinding
> style that the user selected.
> 
> Then there are also the issues of overloading several operations on
> a single key (like TAB).  So far we've solved this along the lines you
> suggest (a "generalized command", such as `indent-for-tab-command`), and
> maybe that's good enough for this, tho it prevents changing this
> overloading according to the keybinding style.
> 
> [ BTW, another way to look at it is not "how can we compute which key to
>   bind FOO to" but rather "how can we compute which command to run when
>   KEY is hit".  IOW, we could make keymaps more dynamic such that when
>   you hit KEY, Emacs passes that to a "procedural keymap" which will
>   *compute* (rather than lookup) which command to run, according to the
>   current keybinding style.
>   Not sure it would be better: I just mention it as one of the many
>   things that we may want to consider in order to find a good solution
>   to the problem.  ]

How about adding this (and more) to etc/TODO?

Re: Delegating user-reserved key binding space definition to users

[Psionic K]

[-- Attachment #1: Type: text/plain, Size: 7547 bytes --]

> a solution should allow packages to declare that command FOO
> should be bound to some key based on SOME-INFO, such that it will be
> bound to one key in "normal Emacs mode", and to another in `evil-mode`
> and to yet another in `god-mode`, etc...

SOME-INFO will be of the form:
((concept command) (concept command) ...)) ; package provide
((concept (concept concept concept)) (concept concept)) ; overload
remappings, perhaps user or package provided or both
((concept binding) (concept binding)) ; user or emulation mode etc provided

The overloads might be embedded in the first and second lists, but it will
make for complicated reading later.

Either a built-in implementation in Emacs or customized implementations in
packages should consume the full SOME-INFO and generate keymaps.  If that
keymap generation is dynamic or the keymap itself embeds indirection for
contextual operations, we get essentially dynamic dispatch, and there's not
much need for new implementations.  emulation-mode-map-alists has its
structure for good reason.

In the beginning, before packages provide their half, the user will provide
it, likely through a package.  This is analogous to evil collection.

There likely will not be enough concepts to support relocating every key
this way.  While the default implementation that ships with Emacs can
attempt to rearrange keys usings current behavior and a heuristic where
that is not possible, if the user configures a different implementation,
even one that ignores the mode's SOME-INFO in part or whole, commands will
be unbound.  This is good.  We have to support the notion that the provided
commands are bad and that the users wants to write better ones or else we
are choking the evolution of the interaction models.  Package authors
should focus on good functions and editing data models.  This is what makes
it easy to write good commands.  Attempting to write all the good commands
is not possible without lots of user representation, which package authors
cannot individually have.

> To me a solution should allow packages to declare that command FOO
> should be bound to some key based on SOME-INFO, such that it will be
> bound to one key in "normal Emacs mode", and to another in `evil-mode`
> and to yet another in `god-mode`, etc...

This places too much emphasis on package authors.  They should only provide
the ((concept command)) information.  Package authors and users have to
meet in the middle, and remember to put the users first in that
discussion.  As stated above, I believe it's the job of modal systems to
decide how to consume the package-defined half of SOME-INFO.

> Agreed.  But it should apply to all bindings.

The concepts that can be good targets in SOME-INFO should arise only very
naturally.  This means there will be way too few concepts to handle the
current state of keymaps.  IMO this is because package authors feel
entitled to users' keyboards or obligated to shadow all of the random
commands bound in the default global map.  The implementations should make
it easier for bad commands / bindings to die.

Lots of Emacs applications handle lists of options because lists are simply
ubiquitous, and so navigating lists is naturally a common concept.  M-x is
the gateway to commands, and Emacs without such a gateway is degenerate, so
we need that as a concept.  Aborting a command that is asking for input or
blocking on network is a concept.  Let's get these figured out and make
them easily remappable before worrying about esoteric commands that the
user might want to redefine anyway.

On Mon, Nov 28, 2022 at 12:15 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> >>> I suggest introducing a notion of "generalized" commands. Such commands
> >>> will represent common actions executed by users (like move to
> >>> next/previous element). Major and minor modes can then define specific
> >>> implementation of the actions.
> >>
> >> I think it's a non-starter because it requires foresight: only those
> >> commands defined with this mechanism will be extensible.  I agree that
> >> an additional level of indirection is probably necessary, but I suspect
> >> it needs to be placed elsewhere.
> >
> > Auto-remapping will need some kind of grouping for commands one way or
> > another. There is no way we can do it auto-magically. Developer or users
> > should decide.
>
> Agreed.  But it should apply to all bindings.
>
> > Currently, the commands in major mods are bound to specific key
> > bindings. The bindings are chosen either arbitrarily, according to major
> > mode author preferences, or according to semi-established default key
> > binding scheme (like C-f/C-M-f/C-n/C-v/etc). Either way, trying to
> > re-bind commands in multiple major modes is not easy.
>
> Yup.  I fully agree that it's a real problem.
>
> I'd welcome a solution to it.  Even an "unrealistic" one would be good.
> Currently, I don't even know what a good solution could look like.
>
> To me a solution should allow packages to declare that command FOO
> should be bound to some key based on SOME-INFO, such that it will be
> bound to one key in "normal Emacs mode", and to another in `evil-mode`
> and to yet another in `god-mode`, etc...
>
> Some SOME-INFO needs to provide not a specific key, but some information
> from which we can compute the "natural key" that fits the keybinding
> style that the user selected.
>
> Then there are also the issues of overloading several operations on
> a single key (like TAB).  So far we've solved this along the lines you
> suggest (a "generalized command", such as `indent-for-tab-command`), and
> maybe that's good enough for this, tho it prevents changing this
> overloading according to the keybinding style.
>
> [ BTW, another way to look at it is not "how can we compute which key to
>   bind FOO to" but rather "how can we compute which command to run when
>   KEY is hit".  IOW, we could make keymaps more dynamic such that when
>   you hit KEY, Emacs passes that to a "procedural keymap" which will
>   *compute* (rather than lookup) which command to run, according to the
>   current keybinding style.
>   Not sure it would be better: I just mention it as one of the many
>   things that we may want to consider in order to find a good solution
>   to the problem.  ]
>
> >> [ FWIW, you can get similar results with the current setup using
> >>   command remapping.  ]
> > You are absolutely right: when a major mode command is related to
> > built-in command in command map.
>
> Just to be clear: I don't think command remapping solves the problem
> at hand.  It can sometimes be used for that, but not in general.
>
> > However, some major modes introduce new concepts.  For example, think
> > about paredit-forward-slurp-sexp, which can be an equivalent of Org's
> > heading promotion or moving word at point forward in sentences.  How
> > could you remap to group these 3 very different yet similar (for some
> > users) commands together?
>
> Great example, indeed, thanks.
>
> In this context, I'd like the package to be able to explain that this
> command is about a "sexp"-granularity operation and about
> "forward"-direction operation, so that the keybinding style may
> automatically find a natural/consistent key (or set of keys) to use for
> it depending on whether the current keybinding style uses f/b, or j/k,
> or left/right, or ... for forward/backward operations.
>
>
>         Stefan
>
>

-- 

Psionic K <psionik@positron.solutions>
Software Engineer

*Positron Solutions <https://positron.solutions>*

[-- Attachment #2: Type: text/html, Size: 9637 bytes --]

Re: Delegating user-reserved key binding space definition to users

[Stefan Monnier]

>> a solution should allow packages to declare that command FOO
>> should be bound to some key based on SOME-INFO, such that it will be
>> bound to one key in "normal Emacs mode", and to another in `evil-mode`
>> and to yet another in `god-mode`, etc...
>
> SOME-INFO will be of the form:
> ((concept command) (concept command) ...)) ; package provide
> ((concept (concept concept concept)) (concept concept)) ; overload
> remappings, perhaps user or package provided or both
> ((concept binding) (concept binding)) ; user or emulation mode etc provided

I'm afraid I don't know what this means.  Can you try and make it more
concrete with an example?

Note that in the text I wrote, SOME-INFO was supposed to be a piece of
information specific to the command FOO.  I get the impression that your
SOME-INFO includes info for several commands.  Maybe we'll have to do
that, but I was hoping we could avoid it.

> In the beginning, before packages provide their half, the user will provide
> it, likely through a package.  This is analogous to evil collection.

I don't want SOME-INFO to explicitly say what should happen for
`evil-mode`, vanilla Emacs mode, `god-mode`, `boon-mode`,
`ergoemacs-mode`, modalka, ...
This is the mess we already have.

Instead SOME-INFO should give just enough data to those modes such that
they can wisely (and predictably) choose a binding for that command.

>> To me a solution should allow packages to declare that command FOO
>> should be bound to some key based on SOME-INFO, such that it will be
>> bound to one key in "normal Emacs mode", and to another in `evil-mode`
>> and to yet another in `god-mode`, etc...
>
> This places too much emphasis on package authors.

I hope with my above explanation you can agree that it also gives a lot
of power to the keybinding styles.  And of course, ultimately the
end-user can override any of those decisions (this is Emacs we're
talking about, after all).

> As stated above, I believe it's the job of modal systems to
> decide how to consume the package-defined half of SOME-INFO.

IIUC we violently agree.

I personally don't yet have a clear idea of what SOME-INFO may look
like.  I suspect it could include some letters (used as hints to help
the keybinding-style choose keys) plus a set of "features" maybe (like
forward/up/down/backward for direction, or add/remove, or
char/word/symbol/sexp/buffer for granularity/subject).  It should likely
include also some notion of scope (global, buffer-local, local to
a region, specific to some particular set of major mode, only
meaningful when the region is active, ...).


        Stefan

Re: Delegating user-reserved key binding space definition to users

[Psionic K]

[-- Attachment #1: Type: text/plain, Size: 8339 bytes --]

> Can you try and make it more
> concrete with an example?

;; The user will specify concept / abstract commands -> key sequences
(setq-custom concept-sequences '((user-next ?\M-n) (user-prev ?\M-p)))
;; The package will specify concepts / abstract commands -> concrete
commands
;; So, something like a magit section mode intended to be derived from, the
package author writes:
(setq-local concept-commands '((user-next #'magit-section-next) (user-prev
#'magit-section-prev)))
;; And then the package tells the implementation to consume the
declarations, which might occur after the mode hook to let the user
intervene
(emacs-function-setup-concept-keymap)

`emacs-function-setup-concept-keymap' would be an implementation that ties
the user's declaration of sequence -> concept mappings together with the
package's concept -> command mappings.  The result is that the local map
should contain some command remap shadows corresponding to what the user
and the package declared.

I neglected overloading in my example, but basically a user might be okay
with several concepts being able to map to one key and the package might be
okay with mapping the same command to one of several concepts.

IMO these declarations one-to-many declarations should only be used with
two specific conventions:

   1. Package authors specify a list of increasingly more generic concepts,
   ending with `user-generic' which would just mean "any key that the user has
   bestowed upon packages by default.".  An example of more abstract
   definitions would be a declaration like (#'magit-section-next (user-next
   user-navigation)).   The implementation could try to map to user-next and
   if that's not available, use an available sequence from user-navigation.
   2. The user's convention for one-to-many mappings is different.  They
   might want to map a generic concept onto several key sequences in order, so
   they might declare something like '(user-navigation (?\M-p ?\M-n ?\M-f
   ?\M-b)).  If a package declares a user-next, user-prev, and two more
   user-navigation commands, the implementation will give them all keymap
   elements.

> I get the impression that your
> SOME-INFO includes info for several commands.

Yes.  The reason commands should not handle this declaration is because the
implementation would have to discover the commands, and this would be
fragile if the user starts replacing commands or wants to specify commands
that the package has no idea about.

I'm also preoccupied with automating the user-driven re-mapping of commands
to key sequences when those commands express an extremely similar idea,
such as "next".  I see this as a prerequisite for what I think you want.  I
don't think I agree with what I think you want, and this could be messing
with my interpretation, but it is a prerequisite still.

We have lots of convention, lots of similarity in mode keymaps, but we it's
ad-hoc, and we just need to make it official so that the conventions are
useful and user's can change one declaration to move, for example, C-g.

> It should likely
> include also some notion of scope (global, buffer-local, local to
> a region, specific to some particular set of major mode, only
> meaningful when the region is active, ...)

The purpose of an abstract key mapping scheme is to allow the user to
coherently move a binding whenever that binding expresses the same idea in
several modes.  Complex, highly configurable declarations would actually
make the situation worse, requiring more user effort to accomplish the
move.  There are already tools available for handling 1:1 edge cases, one
declaration at a time.  Many commands that need such configurability tweak
their behavior based on the call context or some state.  We don't need new
infrastructure for specifying a large number of edge cases.  Such schemes
become baked into modal systems like Evil.

> Emacs passes that to a "procedural keymap" which will
>   *compute* (rather than lookup) which command to run, according to the
>   current keybinding style.

This kind of dynamic dispatch is not actually necessary because commands
already can decide to do something differently.  A command can wrap several
commands.  Commands can react to variables.  Commands can inspect their
context.  The big reason we want this dynamic dispatch style behavior
encoded into commands themselves is because it plays well with the notion
of mapping commands and sequences.  Where-is doesn't work well with dynamic
maps like transient.  Users can't declare keys succinctly if they are
supposed to make their declarations aware of state that the commands should
instead know in their body.  An implementation for generating the concrete
bindings would be much more complex if it had to know about information
that should be internal to the command.

Package authors can always create the most power for users by having a good
data model and good functions with which users can construct their own
commands.  No matter how dynamic dispatch declarations are done, they are
going to be harder to understand and implement than simply re-writing new
command bodies.  Even writing schemes for evil, which only has to support
one implementation, makes some gnarly looking declarations.

The temptation to add power will harm the poor user with the humble goal to
coherently move C-n to M-n in modes where it makes obvious sense because
their pinky hurts.

On Mon, Nov 28, 2022 at 10:01 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> >> a solution should allow packages to declare that command FOO
> >> should be bound to some key based on SOME-INFO, such that it will be
> >> bound to one key in "normal Emacs mode", and to another in `evil-mode`
> >> and to yet another in `god-mode`, etc...
> >
> > SOME-INFO will be of the form:
> > ((concept command) (concept command) ...)) ; package provide
> > ((concept (concept concept concept)) (concept concept)) ; overload
> > remappings, perhaps user or package provided or both
> > ((concept binding) (concept binding)) ; user or emulation mode etc
> provided
>
> I'm afraid I don't know what this means.  Can you try and make it more
> concrete with an example?
>
> Note that in the text I wrote, SOME-INFO was supposed to be a piece of
> information specific to the command FOO.  I get the impression that your
> SOME-INFO includes info for several commands.  Maybe we'll have to do
> that, but I was hoping we could avoid it.
>
> > In the beginning, before packages provide their half, the user will
> provide
> > it, likely through a package.  This is analogous to evil collection.
>
> I don't want SOME-INFO to explicitly say what should happen for
> `evil-mode`, vanilla Emacs mode, `god-mode`, `boon-mode`,
> `ergoemacs-mode`, modalka, ...
> This is the mess we already have.
>
> Instead SOME-INFO should give just enough data to those modes such that
> they can wisely (and predictably) choose a binding for that command.
>
> >> To me a solution should allow packages to declare that command FOO
> >> should be bound to some key based on SOME-INFO, such that it will be
> >> bound to one key in "normal Emacs mode", and to another in `evil-mode`
> >> and to yet another in `god-mode`, etc...
> >
> > This places too much emphasis on package authors.
>
> I hope with my above explanation you can agree that it also gives a lot
> of power to the keybinding styles.  And of course, ultimately the
> end-user can override any of those decisions (this is Emacs we're
> talking about, after all).
>
> > As stated above, I believe it's the job of modal systems to
> > decide how to consume the package-defined half of SOME-INFO.
>
> IIUC we violently agree.
>
> I personally don't yet have a clear idea of what SOME-INFO may look
> like.  I suspect it could include some letters (used as hints to help
> the keybinding-style choose keys) plus a set of "features" maybe (like
> forward/up/down/backward for direction, or add/remove, or
> char/word/symbol/sexp/buffer for granularity/subject).  It should likely
> include also some notion of scope (global, buffer-local, local to
> a region, specific to some particular set of major mode, only
> meaningful when the region is active, ...).
>
>
>         Stefan
>
>

-- 

Psionic K <psionik@positron.solutions>
Software Engineer

*Positron Solutions <https://positron.solutions>*

[-- Attachment #2: Type: text/html, Size: 10236 bytes --]

Re: Delegating user-reserved key binding space definition to users

[John Yates]

On Mon, Nov 28, 2022 at 1:16 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> > However, some major modes introduce new concepts.  For example, think
> > about paredit-forward-slurp-sexp, which can be an equivalent of Org's
> > heading promotion or moving word at point forward in sentences.  How
> > could you remap to group these 3 very different yet similar (for some
> > users) commands together?
>
> Great example, indeed, thanks.
>
> In this context, I'd like the package to be able to explain that this
> command is about a "sexp"-granularity operation and about
> "forward"-direction operation, so that the keybinding style may
> automatically find a natural/consistent key (or set of keys) to use for
> it depending on whether the current keybinding style uses f/b, or j/k,
> or left/right, or ... for forward/backward operations.

[I have followed this thread only intermittently.
Please forgive any repetition of things already said.]

I agree with Stefan, that is a wonderful example.

Objects:  We need a codified vocabulary.
* Display objects: char, line, column, window, . . .
* Text containers: file, buffer, register, . . .
* Text objects: char, word, sexp, sentence, . . .

How are tree-sitter provided objects accomodated?

Clearly this vocabulary would be drawn largely from
current usage.  Codification is necessary for the
proposed package to be able to reason about these
concepts.

When a package introduces a new object, its author
should be able make statements like:
* X is a new object
* Object X is most akin to but coexists with object Y
* Object X subsumes and eclipses object Y

Ordering: The available key sequences have a vague
partial order related to the number and ergonomics
of keystrokes involved.  This should be formalized.
A user should be able to weigh in on this partial order.

To aid in aligning the choices of bindings with the
preceding partial order we need to formalize partial
orders over objects that emacs users already intuit:
* char < (line | column) < window < frame < display
* char < word < (sexp | sentence) . . .

Verbs: Once you have objects you clearly need
imperative verbs.
* insert
* delete
* move
* transpose

Properties: The applicability of properties to verbs
needs to be defined.  My sense is that direction is
inapplicable to verbs such as:
* yank
* save

Re: Delegating user-reserved key binding space definition to users

[Eli Zaretskii]

> From: Psionic K <psionik@positron.solutions>
> Date: Mon, 28 Nov 2022 23:22:24 -0600
> Cc: Ihor Radchenko <yantar92@posteo.net>,
>  Emacs developers <emacs-devel@gnu.org>
> 
> ;; The user will specify concept / abstract commands -> key sequences
> (setq-custom concept-sequences '((user-next ?\M-n) (user-prev ?\M-p)))
> ;; The package will specify concepts / abstract commands -> concrete commands
> ;; So, something like a magit section mode intended to be derived from, the package author writes:
> (setq-local concept-commands '((user-next #'magit-section-next) (user-prev #'magit-section-prev)))
> ;; And then the package tells the implementation to consume the declarations, which might occur after the
> mode hook to let the user intervene
> (emacs-function-setup-concept-keymap)
> 
> `emacs-function-setup-concept-keymap' would be an implementation that ties the user's declaration of
> sequence -> concept mappings together with the package's concept -> command mappings.  The result is
> that the local map should contain some command remap shadows corresponding to what the user and the
> package declared.
> 
> I neglected overloading in my example, but basically a user might be okay with several concepts being able
> to map to one key and the package might be okay with mapping the same command to one of several
> concepts.
> 
> IMO these declarations one-to-many declarations should only be used with two specific conventions:  
> 
> 1 Package authors specify a list of increasingly more generic concepts, ending with `user-generic' which
>  would just mean "any key that the user has bestowed upon packages by default.".  An example of more
>  abstract definitions would be a declaration like (#'magit-section-next (user-next user-navigation)).   The
>  implementation could try to map to user-next and if that's not available, use an available sequence from
>  user-navigation.
> 2 The user's convention for one-to-many mappings is different.  They might want to map a generic concept
>  onto several key sequences in order, so they might declare something like '(user-navigation (?\M-p ?\M-n
>  ?\M-f ?\M-b)).  If a package declares a user-next, user-prev, and two more user-navigation commands,
>  the implementation will give them all keymap elements.
> 
> > I get the impression that your
> > SOME-INFO includes info for several commands.
> 
> Yes.  The reason commands should not handle this declaration is because the implementation would have
> to discover the commands, and this would be fragile if the user starts replacing commands or wants to
> specify commands that the package has no idea about.
> 
> I'm also preoccupied with automating the user-driven re-mapping of commands to key sequences when
> those commands express an extremely similar idea, such as "next".  I see this as a prerequisite for what I
> think you want.  I don't think I agree with what I think you want, and this could be messing with my
> interpretation, but it is a prerequisite still.
> 
> We have lots of convention, lots of similarity in mode keymaps, but we it's ad-hoc, and we just need to make
> it official so that the conventions are useful and user's can change one declaration to move, for example,
> C-g.

I think this discussion started from the wrong starting point, and that's
why the common understanding is difficult and misunderstandings abundant:
there's no reason to believe you and Stefan (and myself, and others who
read this) have the same goals and the same issues in mind.

The basic problem here is that we don't have a clear commonly-shared idea of
what we want to support with these features.

My suggestion is to start from clarifying these requirements.  A good method
of understanding requirements is to describe typical use-cases and their
variations, as you probably know.  With that in mind, would you or someone
else please describe such use-cases?  Specifically:

  . what does the user specify?
  . what should happen with key bindings to various commands as result of
    the user specifications?

For instance, with your example:

> ;; The user will specify concept / abstract commands -> key sequences
> (setq-custom concept-sequences '((user-next ?\M-n) (user-prev ?\M-p)))
> ;; The package will specify concepts / abstract commands -> concrete commands
> ;; So, something like a magit section mode intended to be derived from, the package author writes:
> (setq-local concept-commands '((user-next #'magit-section-next) (user-prev #'magit-section-prev)))
> ;; And then the package tells the implementation to consume the declarations, which might occur after the
> mode hook to let the user intervene
> (emacs-function-setup-concept-keymap)

I have no idea what will be the binding of magit-section-next.  And, since
you only gave a single command as an example, I have no idea how is what you
propose better than just saying

  (local-set-key ?\M-n #'magit-section-next)

I understand what Stefan says about "letters" (probably meaning that 'n'
should be used in "next-FOO" commands and 'p' in "prev-FOO"), but I don't
understand what you are saying, because you didn't tell enough about your
eventual goal, in practical terms.

Re: Delegating user-reserved key binding space definition to users

[Psionic K]

[-- Attachment #1: Type: text/plain, Size: 5776 bytes --]

>   (local-set-key ?\M-n #'magit-section-next)

This is a good edge case solution.  It is not a good solution for moving or
removing bindings coherently across all current and future modes.

On Tue, Nov 29, 2022 at 7:03 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Psionic K <psionik@positron.solutions>
> > Date: Mon, 28 Nov 2022 23:22:24 -0600
> > Cc: Ihor Radchenko <yantar92@posteo.net>,
> >  Emacs developers <emacs-devel@gnu.org>
> >
> > ;; The user will specify concept / abstract commands -> key sequences
> > (setq-custom concept-sequences '((user-next ?\M-n) (user-prev ?\M-p)))
> > ;; The package will specify concepts / abstract commands -> concrete
> commands
> > ;; So, something like a magit section mode intended to be derived from,
> the package author writes:
> > (setq-local concept-commands '((user-next #'magit-section-next)
> (user-prev #'magit-section-prev)))
> > ;; And then the package tells the implementation to consume the
> declarations, which might occur after the
> > mode hook to let the user intervene
> > (emacs-function-setup-concept-keymap)
> >
> > `emacs-function-setup-concept-keymap' would be an implementation that
> ties the user's declaration of
> > sequence -> concept mappings together with the package's concept ->
> command mappings.  The result is
> > that the local map should contain some command remap shadows
> corresponding to what the user and the
> > package declared.
> >
> > I neglected overloading in my example, but basically a user might be
> okay with several concepts being able
> > to map to one key and the package might be okay with mapping the same
> command to one of several
> > concepts.
> >
> > IMO these declarations one-to-many declarations should only be used with
> two specific conventions:
> >
> > 1 Package authors specify a list of increasingly more generic concepts,
> ending with `user-generic' which
> >  would just mean "any key that the user has bestowed upon packages by
> default.".  An example of more
> >  abstract definitions would be a declaration like (#'magit-section-next
> (user-next user-navigation)).   The
> >  implementation could try to map to user-next and if that's not
> available, use an available sequence from
> >  user-navigation.
> > 2 The user's convention for one-to-many mappings is different.  They
> might want to map a generic concept
> >  onto several key sequences in order, so they might declare something
> like '(user-navigation (?\M-p ?\M-n
> >  ?\M-f ?\M-b)).  If a package declares a user-next, user-prev, and two
> more user-navigation commands,
> >  the implementation will give them all keymap elements.
> >
> > > I get the impression that your
> > > SOME-INFO includes info for several commands.
> >
> > Yes.  The reason commands should not handle this declaration is because
> the implementation would have
> > to discover the commands, and this would be fragile if the user starts
> replacing commands or wants to
> > specify commands that the package has no idea about.
> >
> > I'm also preoccupied with automating the user-driven re-mapping of
> commands to key sequences when
> > those commands express an extremely similar idea, such as "next".  I see
> this as a prerequisite for what I
> > think you want.  I don't think I agree with what I think you want, and
> this could be messing with my
> > interpretation, but it is a prerequisite still.
> >
> > We have lots of convention, lots of similarity in mode keymaps, but we
> it's ad-hoc, and we just need to make
> > it official so that the conventions are useful and user's can change one
> declaration to move, for example,
> > C-g.
>
> I think this discussion started from the wrong starting point, and that's
> why the common understanding is difficult and misunderstandings abundant:
> there's no reason to believe you and Stefan (and myself, and others who
> read this) have the same goals and the same issues in mind.
>
> The basic problem here is that we don't have a clear commonly-shared idea
> of
> what we want to support with these features.
>
> My suggestion is to start from clarifying these requirements.  A good
> method
> of understanding requirements is to describe typical use-cases and their
> variations, as you probably know.  With that in mind, would you or someone
> else please describe such use-cases?  Specifically:
>
>   . what does the user specify?
>   . what should happen with key bindings to various commands as result of
>     the user specifications?
>
> For instance, with your example:
>
> > ;; The user will specify concept / abstract commands -> key sequences
> > (setq-custom concept-sequences '((user-next ?\M-n) (user-prev ?\M-p)))
> > ;; The package will specify concepts / abstract commands -> concrete
> commands
> > ;; So, something like a magit section mode intended to be derived from,
> the package author writes:
> > (setq-local concept-commands '((user-next #'magit-section-next)
> (user-prev #'magit-section-prev)))
> > ;; And then the package tells the implementation to consume the
> declarations, which might occur after the
> > mode hook to let the user intervene
> > (emacs-function-setup-concept-keymap)
>
> I have no idea what will be the binding of magit-section-next.  And, since
> you only gave a single command as an example, I have no idea how is what
> you
> propose better than just saying
>
>   (local-set-key ?\M-n #'magit-section-next)
>
> I understand what Stefan says about "letters" (probably meaning that 'n'
> should be used in "next-FOO" commands and 'p' in "prev-FOO"), but I don't
> understand what you are saying, because you didn't tell enough about your
> eventual goal, in practical terms.
>


-- 

Psionic K <psionik@positron.solutions>
Software Engineer

*Positron Solutions <https://positron.solutions>*

[-- Attachment #2: Type: text/html, Size: 7602 bytes --]

Re: Delegating user-reserved key binding space definition to users

[xenodasein--- via Emacs development discussions.]

Nov 30, 2022, 06:23 by psionik@positron.solutions:

> >   (local-set-key ?\M-n #'magit-section-next)
>
> This is a good edge case solution.  It is not a good solution for moving or removing bindings coherently across all current and future modes.
>
> ...
>

Eli does have some rigid expectation as to how things should be, and
instead of making these explicit would rather toy with askers over
piece by piece mails like this.  I suggest to save time stop poking
around how things are done in defaults and write something brand-new
like transient.el then similarly send it for inclusion in Emacs.

Re: Delegating user-reserved key binding space definition to users

[Eli Zaretskii]

> From: Psionic K <psionik@positron.solutions>
> Date: Wed, 30 Nov 2022 00:23:02 -0600
> Cc: monnier@iro.umontreal.ca, yantar92@posteo.net, emacs-devel@gnu.org
> 
> >   (local-set-key ?\M-n #'magit-section-next)
> 
> This is a good edge case solution.  It is not a good solution for moving or removing bindings coherently
> across all current and future modes.

Thanks.  But if this is an answer to my questions, I'm afraid I don't
understand it.  I asked several questions whose answers would allow me to
understand better what you have in mind.