bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Phil Sainty]

Following up on 
https://lists.gnu.org/archive/html/emacs-devel/2021-10/msg00109.html

On 2021-10-02 21:53, Eli Zaretskii wrote:
>> From: Phil Sainty <psainty@orcon.net.nz>
>> On 2021-10-02 19:45, Eli Zaretskii wrote:
>> > There's a huge difference between breaking literal searches for
>> > symbols by text-searching tools, and breaking basic Emacs commands
>> > because the name the user sees and types is not known to Emacs.
>> 
>> But shorthands does *both* of those things.
>> 
>> The name the user sees is "s-foo".
>> The name known to Emacs is "string-library-foo" (or whatever).
>> The user types "C-h o s-foo RET" and Emacs says "no match".
> 
> If this is correct (I didn't try), please report it as a bug.


This was indeed the case in the build I'd compiled for testing:

GNU Emacs 28.0.50 of 2021-09-29
Repository revision: b02a7ad2631b6ac3a95e53cb26a0aa1b1ab7e98a
Repository branch: master

I tested with a copy of so-long.el in which I renamed all of the
so-long-* symbols to sl-* and then configured the local variable
;; elisp-shorthands: (("sl-" . "so-long-"))

Loading the new sl.el confirmed that Emacs didn't recognise the
shorthand symbols generally.

(Until now I was under the impression that this was all part and
parcel of the shorthands feature, but it turns out that it's a bug.)


-Phil

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> Date: Sat, 02 Oct 2021 22:20:09 +1300
> From: Phil Sainty <psainty@orcon.net.nz>
> 
> Following up on 
> https://lists.gnu.org/archive/html/emacs-devel/2021-10/msg00109.html
> 
> On 2021-10-02 21:53, Eli Zaretskii wrote:
> >> From: Phil Sainty <psainty@orcon.net.nz>
> >> On 2021-10-02 19:45, Eli Zaretskii wrote:
> >> > There's a huge difference between breaking literal searches for
> >> > symbols by text-searching tools, and breaking basic Emacs commands
> >> > because the name the user sees and types is not known to Emacs.
> >> 
> >> But shorthands does *both* of those things.
> >> 
> >> The name the user sees is "s-foo".
> >> The name known to Emacs is "string-library-foo" (or whatever).
> >> The user types "C-h o s-foo RET" and Emacs says "no match".
> > 
> > If this is correct (I didn't try), please report it as a bug.
> 
> 
> This was indeed the case in the build I'd compiled for testing:
> 
> GNU Emacs 28.0.50 of 2021-09-29
> Repository revision: b02a7ad2631b6ac3a95e53cb26a0aa1b1ab7e98a
> Repository branch: master
> 
> I tested with a copy of so-long.el in which I renamed all of the
> so-long-* symbols to sl-* and then configured the local variable
> ;; elisp-shorthands: (("sl-" . "so-long-"))
> 
> Loading the new sl.el confirmed that Emacs didn't recognise the
> shorthand symbols generally.

João, if this problem still exists after your changes yesterday, could
you please look into it?  TIA.

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> Loading the new sl.el confirmed that Emacs didn't recognise the
>> shorthand symbols generally.
>
> João, if this problem still exists after your changes yesterday, could
> you please look into it?  TIA.

It's not after my changes from yesterday, no.  This is by design.  Emacs
doesn't recognize the shorthands symbols generally (if "generally" is to
mean "globally") because shorthands _don't_ exist in the global obarray.
They exist in the buffer.  I'm going to type in the relevant part of my
reply to Phil in emacs-devel, when complaining about C-h o not finding
his shorthand.

    That's only when the user types that in the minibuffer and doesn't
    associate [it] in any way to the buffer where [the user] sets up
    that particular shorthand (remember, shorthands aren't global:
    that's the point).  Much like if I type 'import foo as bar' in my
    Python of JavaScript program and then go search [globally, on the
    internet] for 'bar' I don't get the results for 'foo'.
     
    But (have you seen the animated gif?)  if you type 'C-h o' with point ON
    TOP OF 's-foo', then M-x describe-symbol will be prepolulated with
    string-library-foo, and need only type RET.

In other words, Phil, if you wish me to do anything about this bug, you
must explain exactly what you are doing and what you expect.  What does
it mean precisely for "Emacs to recognise the shorthand symbols
generally".  Currently it "recognizes" them when the buffer where they
are setup is current.

João

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: Phil Sainty <psainty@orcon.net.nz>,  50959@debbugs.gnu.org
> Date: Sat, 02 Oct 2021 12:02:30 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Loading the new sl.el confirmed that Emacs didn't recognise the
> >> shorthand symbols generally.
> >
> > João, if this problem still exists after your changes yesterday, could
> > you please look into it?  TIA.
> 
> It's not after my changes from yesterday, no.  This is by design.  Emacs
> doesn't recognize the shorthands symbols generally (if "generally" is to
> mean "globally") because shorthands _don't_ exist in the global obarray.
> They exist in the buffer.

Would it be possible to see if the current buffer (from which the
minibuffer was entered) has shorthands, and if so, apply them to
minibuffer input?

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: João Távora <joaotavora@gmail.com>
>> Cc: Phil Sainty <psainty@orcon.net.nz>,  50959@debbugs.gnu.org
>> Date: Sat, 02 Oct 2021 12:02:30 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Loading the new sl.el confirmed that Emacs didn't recognise the
>> >> shorthand symbols generally.
>> >
>> > João, if this problem still exists after your changes yesterday, could
>> > you please look into it?  TIA.
>> 
>> It's not after my changes from yesterday, no.  This is by design.  Emacs
>> doesn't recognize the shorthands symbols generally (if "generally" is to
>> mean "globally") because shorthands _don't_ exist in the global obarray.
>> They exist in the buffer.
>
> Would it be possible to see if the current buffer (from which the
> minibuffer was entered) has shorthands, and if so, apply them to
> minibuffer input?

Yes, much like it is done with C-M-i, basically.

João

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

João Távora <joaotavora@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:

>> Would it be possible to see if the current buffer (from which the
>> minibuffer was entered) has shorthands, and if so, apply them to
>> minibuffer input?
>
> Yes, much like it is done with C-M-i, basically.

Let me improve on that.  That it _can_ be done doesn't mean that it
_should_ be done, but we can decide on that.  There are various levels
to this integration:

0. no integration

1. This is the current integration.  I.e. when C-h o is pressed on the
   symbol the global name is discovered and used as the default.  This
   is how SLIME work with CL's namespacing system.  SLIME is a very well
   tested and widely appreciated Common Lisp IDE for Emcas.

2. The shorthands from the buffer where the minibuffer was entered are
   _not_ in the completions list, but typing one of them interns the
   symbol with those shorthands present, so you get the desired result.
   This would fix Phil's visually-copy-and-type scenario.

3. (Eli's suggestion): the completion list would be augmented with the
   shorthands from the buffer where the minibuffer was entered from.

In levels 2 and 3 the user might be surprised that what once worked for
one 'C-h o' session no longer works for another 'C-h o' session.  The
only way I can see this being acceptable is if the user is somehow made
aware -- visually or otherwise -- that the list she is seeing is somehow
connected to the origin buffer.

In contrast, C-M-i (where level 3 happens) never really leaves the
buffer: its results are connected to it because they will be inserted
there.

João

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: psainty@orcon.net.nz,  50959@debbugs.gnu.org
> Date: Sat, 02 Oct 2021 12:54:13 +0100
> 
> 0. no integration
> 
> 1. This is the current integration.  I.e. when C-h o is pressed on the
>    symbol the global name is discovered and used as the default.  This
>    is how SLIME work with CL's namespacing system.  SLIME is a very well
>    tested and widely appreciated Common Lisp IDE for Emcas.
> 
> 2. The shorthands from the buffer where the minibuffer was entered are
>    _not_ in the completions list, but typing one of them interns the
>    symbol with those shorthands present, so you get the desired result.
>    This would fix Phil's visually-copy-and-type scenario.
> 
> 3. (Eli's suggestion): the completion list would be augmented with the
>    shorthands from the buffer where the minibuffer was entered from.

Are 2 and 3 significantly different (from the implementation POV)?

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

On Sat, Oct 2, 2021 at 1:30 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Cc: psainty@orcon.net.nz,  50959@debbugs.gnu.org
> > Date: Sat, 02 Oct 2021 12:54:13 +0100
> >
> > 0. no integration
> >
> > 1. This is the current integration.  I.e. when C-h o is pressed on the
> >    symbol the global name is discovered and used as the default.  This
> >    is how SLIME work with CL's namespacing system.  SLIME is a very well
> >    tested and widely appreciated Common Lisp IDE for Emcas.
> >
> > 2. The shorthands from the buffer where the minibuffer was entered are
> >    _not_ in the completions list, but typing one of them interns the
> >    symbol with those shorthands present, so you get the desired result.
> >    This would fix Phil's visually-copy-and-type scenario.
> >
> > 3. (Eli's suggestion): the completion list would be augmented with the
> >    shorthands from the buffer where the minibuffer was entered from.
>
> Are 2 and 3 significantly different (from the implementation POV)?

I think so.

I think 2 can be achieved by setting elisp-shorthands buffer-locally
in the minibuffer and removing the "require-match" flag requirement to
whatever completing-read call happens there.

3 is achieved by calculating the list of completions using
'elisp--completion-local-symbols` and then filtering it down as usual.
"require-match" is kept untouched.

João

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 2 Oct 2021 15:02:41 +0100
> Cc: Phil Sainty <psainty@orcon.net.nz>, 50959@debbugs.gnu.org
> 
> > > 0. no integration
> > >
> > > 1. This is the current integration.  I.e. when C-h o is pressed on the
> > >    symbol the global name is discovered and used as the default.  This
> > >    is how SLIME work with CL's namespacing system.  SLIME is a very well
> > >    tested and widely appreciated Common Lisp IDE for Emcas.
> > >
> > > 2. The shorthands from the buffer where the minibuffer was entered are
> > >    _not_ in the completions list, but typing one of them interns the
> > >    symbol with those shorthands present, so you get the desired result.
> > >    This would fix Phil's visually-copy-and-type scenario.
> > >
> > > 3. (Eli's suggestion): the completion list would be augmented with the
> > >    shorthands from the buffer where the minibuffer was entered from.
> >
> > Are 2 and 3 significantly different (from the implementation POV)?
> 
> I think so.
> 
> I think 2 can be achieved by setting elisp-shorthands buffer-locally
> in the minibuffer and removing the "require-match" flag requirement to
> whatever completing-read call happens there.
> 
> 3 is achieved by calculating the list of completions using
> 'elisp--completion-local-symbols` and then filtering it down as usual.
> "require-match" is kept untouched.

You are saying that 3 is easier than 2?  Then I think we should do 3,
as it's better from the user's POV, right?

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

On Sat, Oct 2, 2021 at 3:20 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Sat, 2 Oct 2021 15:02:41 +0100
> > Cc: Phil Sainty <psainty@orcon.net.nz>, 50959@debbugs.gnu.org
> >
> > > > 0. no integration
> > > >
> > > > 1. This is the current integration.  I.e. when C-h o is pressed on the
> > > >    symbol the global name is discovered and used as the default.  This
> > > >    is how SLIME work with CL's namespacing system.  SLIME is a very well
> > > >    tested and widely appreciated Common Lisp IDE for Emcas.
> > > >
> > > > 2. The shorthands from the buffer where the minibuffer was entered are
> > > >    _not_ in the completions list, but typing one of them interns the
> > > >    symbol with those shorthands present, so you get the desired result.
> > > >    This would fix Phil's visually-copy-and-type scenario.
> > > >
> > > > 3. (Eli's suggestion): the completion list would be augmented with the
> > > >    shorthands from the buffer where the minibuffer was entered from.
> > >
> > > Are 2 and 3 significantly different (from the implementation POV)?
> >
> > I think so.
> >
> > I think 2 can be achieved by setting elisp-shorthands buffer-locally
> > in the minibuffer and removing the "require-match" flag requirement to
> > whatever completing-read call happens there.
> >
> > 3 is achieved by calculating the list of completions using
> > 'elisp--completion-local-symbols` and then filtering it down as usual.
> > "require-match" is kept untouched.
>
> You are saying that 3 is easier than 2?  Then I think we should do 3,
> as it's better from the user's POV, right?

No, I don't know that for sure.  And I don't think it's better from
the user's POV.
See my reply to Phil.  It would mistakenly provide the idea that Shorthands
are some alias to the symbol (in the sense of defvaralias).  They are not.

The user would then be quite surprised to find the list of completions change
behind his back as he changes the place of origin of those C-h o calls.

It could only make sense if these interactive prompts were clearly tied to
the buffers they originated from.  The current "Describe function" prompt
should become "Describe function inside foobarbaz.el" .  Even then, I think
it is insufficient, and uncharted territory, whereas the current approach isn't
(it's how SLIME/SLY work in a Lisp-based IDE with local namespacing)

That _can_ be done, and I think all of Phil's list will eventually
funnel down to
a few existing function calls.  But it nevertheless needs more profound work
and a careful understanding of the consequences.

In summary, I think that with the exception of a shorthand-aware
'xref-backend-references',
something that I am working on (between the drops of the torrential emails,
some of them bordering on sheer harassment), this feature is currently
consistent
from a tools point of view.

Again, Shorthands are buffer-local textual indirections to symbols.  They
are not the symbol.  This will never change (not with Shorthands): so including
shorthands in a list of symbols is misguided.  Displaying them in
lists of fragments of
text to be completed in the buffer is not.

That is not to say I don't understand Phil's concerns: I do.  I just
don't understand
the feeling of imminent catastrophe.

As I wrote to Phil, I believe much of this anguish is improved if we font-lock
shorthands specially.  That doesn't seem hard at all.  If I understand Phil's
objections from day 1, he is talking about looking at something in an Elisp
buffer and being uncertain about the nature of the thing that his eyes are
focusing.  But if he is _looking_ at it, then we can quench much of that
doubt by changing the way it looks.

If, on the other hand, he is operating on the thing with some other tool [*],
not just his eyes, then I think the current tools are already doing
the right thing.

João

[*]: Yes, except when that tool is Grep or similar, but that's for all namespace
systems ever invented, so if the concern is still Grep, please open another
bug or (yet) another thread.

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 2 Oct 2021 15:43:03 +0100
> Cc: Phil Sainty <psainty@orcon.net.nz>, 50959@debbugs.gnu.org
> 
> > You are saying that 3 is easier than 2?  Then I think we should do 3,
> > as it's better from the user's POV, right?
> 
> No, I don't know that for sure.  And I don't think it's better from
> the user's POV.
> See my reply to Phil.  It would mistakenly provide the idea that Shorthands
> are some alias to the symbol (in the sense of defvaralias).  They are not.
> 
> The user would then be quite surprised to find the list of completions change
> behind his back as he changes the place of origin of those C-h o calls.

I'm not sure the user will be so much surprised.  We could document
that.  And shorthands aren't supposed to be used massively or
willy-nilly, so these surprises are not necessarily as acute as you
think.  they are certainly not worse than not showing these shorthands
at all.

> It could only make sense if these interactive prompts were clearly tied to
> the buffers they originated from.

But they are: we always know which buffer was the current one when the
minibuffer is entered.

> In summary, I think that with the exception of a shorthand-aware
> 'xref-backend-references',
> something that I am working on (between the drops of the torrential emails,
> some of them bordering on sheer harassment), this feature is currently
> consistent
> from a tools point of view.

You are saying that Help commands should allow asking about
shorthands, except if point is on the shorthand?  That'd be a grave
restriction, I think, worse than "depending on the buffer" which you
don't like: here it depends not only on the buffer, but also on
position of point in that buffer.

> Again, Shorthands are buffer-local textual indirections to symbols.  They
> are not the symbol.  This will never change (not with Shorthands): so including
> shorthands in a list of symbols is misguided.  Displaying them in
> lists of fragments of
> text to be completed in the buffer is not.

I think this is unnecessarily radical POV, and one that will cause
complaints.

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

On Sat, Oct 2, 2021 at 3:56 PM Eli Zaretskii <eliz@gnu.org> wrote:

> But they are: we always know which buffer was the current one when the
> minibuffer is entered.

We "the Emacs code" do. But it's not obvious to a user.  The commands
that invoke these things are global, remember.  I very very often don't know
where I pulled C-h o from.  And I don't want to know. Sometimes I do,
sometimes I don't.

In summary, I wish we can keep the what has been the invariant of Elisp
ever since it was created, and which is underlined many times in the manual:
there is only one obarray of symbols.  I believe my stance preserves this
accurate mental model more than yours.

> You are saying that Help commands should allow asking about
> shorthands, except if point is on the shorthand?

"should NOT allow".  That's what I am saying, yes.  Any command that operates
on _symbols_ should not offer shorthands as candidates unless the goal of that
command is to directly modify the buffer where those shorthands have meaning.

So C-h o is in the former group.  C-M-i s in the latter.

> That'd be a grave
> restriction, I think, worse than "depending on the buffer" which you
> don't like: here it depends not only on the buffer, but also on
> position of point in that buffer.

I don't agree, but ultimately it's your call.  Notice (maybe watch the
.gif again),
that what happens when you type C-h o on 's-concat' is that the prompt becomes:

   "Describe symbol (default magnar-string-concat): ... "

It does _not_ become:

   "Describe symbol (default s-concat): ... "

Because 's-concat' is _not_ a symbol.

> > Again, Shorthands are buffer-local textual indirections to symbols.  They
> > are not the symbol.  This will never change (not with Shorthands): so including
> > shorthands in a list of symbols is misguided.  Displaying them in
> > lists of fragments of
> > text to be completed in the buffer is not.
> I think this is unnecessarily radical POV, and one that will cause
> complaints.

It hasn't in SLIME/SLY and package-local-nicknames have existed for
quite some time there.

What is your opinion on the visually annotating font-lock idea?  I think
it's useful even if we decide to go with levels 2 or 3 of the above
integration (which, as I said, I think we shouldn't, not for now)

João

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 2 Oct 2021 16:22:51 +0100
> Cc: Phil Sainty <psainty@orcon.net.nz>, 50959@debbugs.gnu.org
> 
> > That'd be a grave
> > restriction, I think, worse than "depending on the buffer" which you
> > don't like: here it depends not only on the buffer, but also on
> > position of point in that buffer.
> 
> I don't agree, but ultimately it's your call.  Notice (maybe watch the
> .gif again),
> that what happens when you type C-h o on 's-concat' is that the prompt becomes:
> 
>    "Describe symbol (default magnar-string-concat): ... "
> 
> It does _not_ become:
> 
>    "Describe symbol (default s-concat): ... "
> 
> Because 's-concat' is _not_ a symbol.

I don't see the significance of the difference, from the usability
POV.  I'd still like to see Help commands support shorthands even if
point is not on a shorthand.

> What is your opinion on the visually annotating font-lock idea?  I think
> it's useful even if we decide to go with levels 2 or 3 of the above
> integration (which, as I said, I think we shouldn't, not for now)

I don't object, and think it could be useful.  But I don't think it
could supplant the recognition of shorthands in Help commands.

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

On Sat, Oct 2, 2021 at 4:53 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > Because 's-concat' is _not_ a symbol.
>
> I don't see the significance of the difference, from the usability
> POV.

You don't?  Maybe you are still somewhat thinking of shorthands
as similar to the products of defvaralias or defalias.  s-concat
may mean magnar-string-concat in a buffer, and stream-concat
in another.  In that some other buffer magnar-string-concat may
be accessible by the shorthand ms-concat.

This is how namespaces work also in other languages.  In Elisp
in proposals like Andreas Corallo's
(https://akrl.sdf.org/lexspaces/lexspaces.html)
or if Lars' proposed evolution of shorthands to be somehow local
to a specific region to a buffer (like Common Lisp's IN-PACKAGE).

> I'd still like to see Help commands support shorthands even if
> point is not on a shorthand.

Alright, it does need an implementation.  Of unforeseen complexity.
Certainly the most tricky of all the feature so far, I predict,  as it would
affect many more commands. I'm not confident I can muster the effort
to tackle it for Emacs 28, though I can of course experiment (and maybe
it's easier, especially if you help me restrict the feature in scope).

If it's ever done, I think at the very least it should be made
optional, and that the default should be off.

The least of its problems is that, when set to on, it would bring back
some form of namespace pollution: finding too "inexplicably" short
symbol names when perusing global lists of symbols.  What's more,
finding them there only sometimes.

The more serious of its problems is that it would make the evolution
of Shorthands to be local to a region (for example, as I hinted
above) much harder.

Then, it would raise the bar to adopting _other_ namespacing systems
even higher than it already is.

> I don't object, and think it could be useful.  But I don't think it
> could supplant the recognition of shorthands in Help commands.

I think it could, by helping the user keep the correct mental model
in mind.  It could even be made more sophisticated, like temporarily
displaying the full names of the symbols at the point of the symbols when
C-h f is entered. That would be level of integration 1.5 so to speak (in
the scale I gave a while ago).

I think we must focus on the problem being presented here, not on
the particular solution.  _Why_, fundamentally do you want to see
shorthands listed in an Help command? What, in layman's terms,
are the pieces of information that are missing for you when you _don't_
see them?  Is it more like:

   a. "I wonder what that that 's-thingy' over there is/means or what
    documentation there is for it.?"

Or is it:

  b. "I wonder what help on s-thingies I have at my disposal here
   (and yes I know perfectly well what 'here' is)?  I'm going to type a
   prefix, the TAB to find out."

Or is it both? Or is it a third thing?

The question goes for Phil too, though I think I have a better intuition of
what his problem is: I think it's 'a': he fears he won't be able to immediately
tell visually what he's looking at, say in one of his windows that has
an excerpt of a .el file open.  Disregarding the fact that that already
happens with things like  cl-flet or letf, for example (lexical function
definitions), I think it's a legitimate concern (especially coming from
someone with likely no experience of namespace systems at all).

João Távora

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 2 Oct 2021 18:46:18 +0100
> Cc: Phil Sainty <psainty@orcon.net.nz>, 50959@debbugs.gnu.org
> 
> On Sat, Oct 2, 2021 at 4:53 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > > Because 's-concat' is _not_ a symbol.
> >
> > I don't see the significance of the difference, from the usability
> > POV.
> 
> You don't?  Maybe you are still somewhat thinking of shorthands
> as similar to the products of defvaralias or defalias.  s-concat
> may mean magnar-string-concat in a buffer, and stream-concat
> in another.  In that some other buffer magnar-string-concat may
> be accessible by the shorthand ms-concat.

We are talking at cross-purposes.  You are talking about the
underlying mechanism; I'm talking about the user's experience.  From
the UX POV, it is unexpected to be unable to get help about a
symbol-like thing the user sees in the buffer.

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

On Sat, Oct 2, 2021 at 6:58 PM Eli Zaretskii <eliz@gnu.org> wrote:

> We are talking at cross-purposes.  You are talking about the
> underlying mechanism; I'm talking about the user's experience.  From
> the UX POV, it is unexpected to be unable to get help about a
> symbol-like thing the user sees in the buffer.

I'm taking both things into consideration. Does that mean that
your answer to my questionnaire is 'a.' or "mostly" 'a.'?

 a. "I wonder what that that 's-thingy' over there is/means or what
    documentation there is for it.?"

I'm assuming that since you wrote "sees in the buffer".

João

PS: Much better terminology "symbol-like thing" :-)

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Sat, 2 Oct 2021 19:03:43 +0100
> Cc: Phil Sainty <psainty@orcon.net.nz>, 50959@debbugs.gnu.org
> 
> On Sat, Oct 2, 2021 at 6:58 PM Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > We are talking at cross-purposes.  You are talking about the
> > underlying mechanism; I'm talking about the user's experience.  From
> > the UX POV, it is unexpected to be unable to get help about a
> > symbol-like thing the user sees in the buffer.
> 
> I'm taking both things into consideration. Does that mean that
> your answer to my questionnaire is 'a.' or "mostly" 'a.'?
> 
>  a. "I wonder what that that 's-thingy' over there is/means or what
>     documentation there is for it.?"
> 
> I'm assuming that since you wrote "sees in the buffer".

Yes.

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We are talking at cross-purposes.  You are talking about the
  > underlying mechanism; I'm talking about the user's experience.  From
  > the UX POV, it is unexpected to be unable to get help about a
  > symbol-like thing the user sees in the buffer.

I agree.  We should implement doing the right thing with these cases,
or at least whatever thing is the most right that we can come up with.
-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I don't object, and think it could be useful.  But I don't think it
  > could supplant the recognition of shorthands in Help commands.

Perhaps it would be useful to make a list of the help commands
(I suggest command names rather than their key bindings) in which
we are considering doing this.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Again, Shorthands are buffer-local textual indirections to symbols.  They
  > > are not the symbol.  This will never change (not with Shorthands): so including
  > > shorthands in a list of symbols is misguided.  Displaying them in
  > > lists of fragments of
  > > text to be completed in the buffer is not.

  > I think this is unnecessarily radical POV, and one that will cause
  > complaints.

It seems valid to me _in principle_, but can you think of specific
cases that might cause complaints?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: joaotavora@gmail.com, psainty@orcon.net.nz, 50959@debbugs.gnu.org
> Date: Sun, 03 Oct 2021 20:14:56 -0400
> 
>   > > Again, Shorthands are buffer-local textual indirections to symbols.  They
>   > > are not the symbol.  This will never change (not with Shorthands): so including
>   > > shorthands in a list of symbols is misguided.  Displaying them in
>   > > lists of fragments of
>   > > text to be completed in the buffer is not.
> 
>   > I think this is unnecessarily radical POV, and one that will cause
>   > complaints.
> 
> It seems valid to me _in principle_, but can you think of specific
> cases that might cause complaints?

I described them: when the user sees an identifier in the buffer, but
cannot call up Help for it unless point is on that identifier.

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

João Távora <joaotavora@gmail.com> writes:

> On Sat, Oct 2, 2021 at 1:30 PM Eli Zaretskii <eliz@gnu.org> wrote:

>> > 0. no integration
>> >
>> > 1. This is the current integration.  I.e. when C-h o is pressed on the
>> >    symbol the global name is discovered and used as the default.  This
>> >    is how SLIME work with CL's namespacing system.  SLIME is a very well
>> >    tested and widely appreciated Common Lisp IDE for Emcas.
>> >
>> > 2. The shorthands from the buffer where the minibuffer was entered are
>> >    _not_ in the completions list, but typing one of them interns the
>> >    symbol with those shorthands present, so you get the desired result.
>> >    This would fix Phil's visually-copy-and-type scenario.
>> >
>> > 3. The completion list would be augmented with the shorthands from
>> >    the buffer where the minibuffer was entered from.

Hello Eli,

I've implemented a variation on 2 based on the later suggestion you
gave in emacs-devel:

> That is, the user types "C-h o s-foo <SOME KEY SEQUENCE>" and that
> replaces s-foo with the expansion, the "real" symbol.  Is that
> feasible?

Yes, it is.  <SOME KEY SEQUENCE> is, of course, TAB.  Here is a patch
for people to try out which I will push in a few days time if there are
no objections.  Cc-ing completion-style specialist Stefan.

Patch just below,
João

commit f9f64c4b3287d7276c8edeacdecfa9c78194447b
Author: João Távora <joaotavora@gmail.com>
Date:   Wed Oct 6 11:30:29 2021 +0100

    Complete shorthands to longhands for symbol-completing tables
    
    Shorthands aren't symbols, they're text forms that 'read' into
    symbols.  As such, shorthands aren't candidates in these tables of
    symbols.  But in some situations, if no other candidates match the
    pattern, we can e.g. complete "x-foo" to "xavier-foo" if the shorthand
    
      (("x-" . "xavier-"))
    
    is set up in the buffer of origin.
    
    bug#50959
    
    * lisp/help-fns.el (help--symbol-completion-table): Report
    `symbol-help' category.
    
    * lisp/minibuffer.el (completion-styles-alist): New 'shorthand'
    style.
    (completion-category-defaults): Link 'symbol-help' category with
    'shorthand' style.
    (minibuffer--original-buffer): New variable.
    (completing-read-default): Setup minibuffer--original-buffer.
    (completion-shorthand-try-completion)
    (completion-shorthand-all-completions): New helpers.

diff --git a/lisp/help-fns.el b/lisp/help-fns.el
index 6be5cd4a50..03bbc979a9 100644
--- a/lisp/help-fns.el
+++ b/lisp/help-fns.el
@@ -176,8 +176,11 @@ help--symbol-completion-table-affixation
           completions))
 
 (defun help--symbol-completion-table (string pred action)
-  (if (and completions-detailed (eq action 'metadata))
-      '(metadata (affixation-function . help--symbol-completion-table-affixation))
+  (if (eq action 'metadata)
+      `(metadata
+        ,@(when completions-detailed
+            '((affixation-function . help--symbol-completion-table-affixation)))
+        (category . symbol-help))
     (when help-enable-completion-autoload
       (let ((prefixes (radix-tree-prefixes (help-definition-prefixes) string)))
         (help--load-prefixes prefixes)))
diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index 1e1a6f852e..48859585bc 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -943,7 +943,12 @@ completion-styles-alist
      completion-initials-try-completion completion-initials-all-completions
      "Completion of acronyms and initialisms.
 E.g. can complete M-x lch to list-command-history
-and C-x C-f ~/sew to ~/src/emacs/work."))
+and C-x C-f ~/sew to ~/src/emacs/work.")
+    (shorthand
+     completion-shorthand-try-completion completion-shorthand-all-completions
+     "Completion of symbol shorthands setup in `read-symbol-shorthands'.
+E.g. can complete \"x-foo\" to \"xavier-foo\" if the shorthand
+((\"x-\" . \"xavier-\")) is set up in the buffer of origin."))
   "List of available completion styles.
 Each element has the form (NAME TRY-COMPLETION ALL-COMPLETIONS DOC):
 where NAME is the name that should be used in `completion-styles',
@@ -990,7 +995,8 @@ completion-category-defaults
     ;; e.g. one that does not anchor to bos.
     (project-file (styles . (substring)))
     (xref-location (styles . (substring)))
-    (info-menu (styles . (basic substring))))
+    (info-menu (styles . (basic substring)))
+    (symbol-help (styles . (basic shorthand substring))))
   "Default settings for specific completion categories.
 Each entry has the shape (CATEGORY . ALIST) where ALIST is
 an association list that can specify properties such as:
@@ -1618,6 +1624,9 @@ minibuffer-confirm-exit-commands
 (defvar minibuffer--require-match nil
   "Value of REQUIRE-MATCH passed to `completing-read'.")
 
+(defvar minibuffer--original-buffer nil
+  "Buffer that was current when `completing-read' was called.")
+
 (defun minibuffer-complete-and-exit ()
   "Exit if the minibuffer contains a valid completion.
 Otherwise, try to complete the minibuffer contents.  If
@@ -4080,6 +4089,40 @@ completion-initials-try-completion
   (let ((newstr (completion-initials-expand string table pred)))
     (when newstr
       (completion-pcm-try-completion newstr table pred (length newstr)))))
+
+;; Shorthand completion
+;;
+;; Iff there is a (("x-" . "string-library-")) shorthand setup and
+;; string-library-foo is in candidates, complete x-foo to it.
+
+(defun completion-shorthand-try-completion (string table pred point)
+  "Try completion with `read-symbol-shorthands' of original buffer."
+  (cl-loop with expanded
+           for (short . long) in
+           (with-current-buffer minibuffer--original-buffer
+             read-symbol-shorthands)
+           for probe =
+           (and (> point (length short))
+                (string-prefix-p short string)
+                (try-completion (setq expanded
+                                      (concat long
+                                              (substring
+                                               string
+                                               (length short))))
+                                table pred))
+           when probe
+           do (message "Shorthand expansion")
+           and return (cons expanded (max (length long)
+                                          (+ (- point (length short))
+                                             (length long))))))
+
+(defun completion-shorthand-all-completions (string table pred _point)
+  ;; no-op: For now, we don't want shorthands to list all the possible
+  ;; locally active longhands.  For the completion categories where
+  ;; this style is active, it could hide other more interesting
+  ;; matches from subsequent styles.
+  nil)
+
 \f
 (defvar completing-read-function #'completing-read-default
   "The function called by `completing-read' to do its work.
@@ -4111,6 +4154,7 @@ completing-read-default
                     ;; in minibuffer-local-filename-completion-map can
                     ;; override bindings in base-keymap.
                     base-keymap)))
+         (buffer (current-buffer))
          (result
           (minibuffer-with-setup-hook
               (lambda ()
@@ -4119,7 +4163,8 @@ completing-read-default
                 ;; FIXME: Remove/rename this var, see the next one.
                 (setq-local minibuffer-completion-confirm
                             (unless (eq require-match t) require-match))
-                (setq-local minibuffer--require-match require-match))
+                (setq-local minibuffer--require-match require-match)
+                (setq-local minibuffer--original-buffer buffer))
             (read-from-minibuffer prompt initial-input keymap
                                   nil hist def inherit-input-method))))
     (when (and (equal result "") def)

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Yes, it is.  <SOME KEY SEQUENCE> is, of course, TAB.  Here is a patch
> for people to try out which I will push in a few days time if there are
> no objections.  Cc-ing completion-style specialist Stefan.

Implementing it as a completion style seems wrong.
Why not just tweak the completion table?


        Stefan

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

On Wed, Oct 6, 2021 at 1:19 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > Yes, it is.  <SOME KEY SEQUENCE> is, of course, TAB.  Here is a patch
> > for people to try out which I will push in a few days time if there are
> > no objections.  Cc-ing completion-style specialist Stefan.
>
> Implementing it as a completion style seems wrong.
>

What is "it"? "It" === "Eli's suggestion"?


> Why not just tweak the completion table?


What tweak do you suggest? The explicit point in this patch
was _not_ to list shorthands as completions, for the reasons
that I gave many times already and that led to Eli's suggestion.
If you can implement that without a completion style that's fine
(but it seems it's the nicest way).

João

[-- Attachment #2: Type: text/html, Size: 1369 bytes --]

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> > Yes, it is.  <SOME KEY SEQUENCE> is, of course, TAB.  Here is a patch
>> > for people to try out which I will push in a few days time if there are
>> > no objections.  Cc-ing completion-style specialist Stefan.
>> Implementing it as a completion style seems wrong.
> What is "it"? "It" === "Eli's suggestion"?

"it" == "making C-x o's completion pay attention to shorthands".


        Stefan

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 769 bytes --]

On Wed, Oct 6, 2021 at 1:50 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> >> > Yes, it is.  <SOME KEY SEQUENCE> is, of course, TAB.  Here is a patch
> >> > for people to try out which I will push in a few days time if there
> are
> >> > no objections.  Cc-ing completion-style specialist Stefan.
> >> Implementing it as a completion style seems wrong.
> > What is "it"? "It" === "Eli's suggestion"?
>
> "it" == "making C-x o's completion pay attention to shorthands".


In this thread, we've described many ways how that can be done.
C-x o already "pays attention" to them, from day one, but that attention
is somehow deemed insufficient.  So you'll have to be clearer about
what you're suggesting exactly, i.e., "what happens when".

João

[-- Attachment #2: Type: text/html, Size: 1267 bytes --]

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> In this thread, we've described many ways how that can be done.
> C-x o already "pays attention" to them, from day one, but that attention
> is somehow deemed insufficient.  So you'll have to be clearer about
> what you're suggesting exactly, i.e., "what happens when".

As an ELisp programmer, I'd want (and expect) that if I can successfully
use `st-bar` and `st-foo` in the current buffer, I will see them in the
possible completions at least when I complete `st-` and ideally in other
cases (e.g. it would be good for completion of `sba` to take `st-bar`
into account if `flex` completion style).


        Stefan

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 2027 bytes --]

On Wed, Oct 6, 2021 at 8:59 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> > In this thread, we've described many ways how that can be done.
> > C-x o already "pays attention" to them, from day one, but that attention
> > is somehow deemed insufficient.  So you'll have to be clearer about
> > what you're suggesting exactly, i.e., "what happens when".
>
> As an ELisp programmer, I'd want (and expect) that if I can successfully
> use `st-bar` and `st-foo` in the current buffer, I will see them in the
> possible completions at least when I complete `st-` and ideally in other
> cases (e.g. it would be good for completion of `sba` to take `st-bar`
> into account if `flex` completion style).
>

1) If by "use in the current buffer" you mean "insert in the current buffer"
then your wish makes full sense to me and is already granted, fully, via
C-M-i.

2) If you're talking about querying the single global database of symbols
using M-x describe-symbols, then you must address symbols by their
full name, as you have always done. It's easy to do so:

2a) by placing point under the form of interest.  If it happens to be
shorthand
it will automatically be picked up.  Shorthands are now visually highlighted
as such.

2b) by typing the shorthands as such and typing TAB to complete them to
the full symbol name.  This is only if you enter the query interface from
the buffer where the shorthands exist.

I think this is more than sufficient. I've worked for years and years with
a CL system that only does 1) and 2a). I've worked side-by-side with
people using it. I even forked and maintained such a system.  I've never
seen requests or complaints for anything more in this regard.  Sure, we
are all different and some of us are more demanding, and that's a good
thing, but it's a fact that the vast majority of Emacs users haven't used
a namespacing system in Elisp for a long time, possibly ever.  Shouldn't
we give this first  conservative approach a shot first?

João

[-- Attachment #2: Type: text/html, Size: 2755 bytes --]

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

João Távora <joaotavora@gmail.com> writes:

> João Távora <joaotavora@gmail.com> writes:
>
>> On Sat, Oct 2, 2021 at 1:30 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
>>> > 0. no integration
>>> >
>>> > 1. This is the current integration.  I.e. when C-h o is pressed on the
>>> >    symbol the global name is discovered and used as the default.  This
>>> >    is how SLIME work with CL's namespacing system.  SLIME is a very well
>>> >    tested and widely appreciated Common Lisp IDE for Emcas.
>>> >
>>> > 2. The shorthands from the buffer where the minibuffer was entered are
>>> >    _not_ in the completions list, but typing one of them interns the
>>> >    symbol with those shorthands present, so you get the desired result.
>>> >    This would fix Phil's visually-copy-and-type scenario.
>>> >
>>> > 3. The completion list would be augmented with the shorthands from
>>> >    the buffer where the minibuffer was entered from.
>
> Hello Eli,
>
> I've implemented a variation on 2 based on the later suggestion you
> gave in emacs-devel:
>
>> That is, the user types "C-h o s-foo <SOME KEY SEQUENCE>" and that
>> replaces s-foo with the expansion, the "real" symbol.  Is that
>> feasible?
>
> Yes, it is.  <SOME KEY SEQUENCE> is, of course, TAB.  Here is a patch
> for people to try out which I will push in a few days time if there are
> no objections.  Cc-ing completion-style specialist Stefan.
>
> Patch just below,

I've stashed this patch in branch scratch/bug-50959-fix.

The reason is Stefan's strong objections to it.

IIUC, Stefan, who originally designed the completion-style mechanism
would seem to favour a scheme where a new completion table category
'symbol' is linked to a new 'abbrev' style (instead of a 'shorthand'
style).  The 'abbrev' style would then query the table for an
appropriate source of abbreviations, which, in the case of table of the
'symbol' category, would be extracted from 'read-symbol-shorthands'.
Stefan do you confirm this?

If so, the end result would be the same and a functional level, and
Phil's original request would be satisfied.  I don't strongly object to
the above more complex approach, but I don't have much time right now to
implement those extra indirections either (not that they're rocket
science, but they need at least good formats, good names, good
docstrings, etc.), so I'll leave it up to you guys to figure out what to
do here.

João

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: Phil Sainty <psainty@orcon.net.nz>,  50959@debbugs.gnu.org,  Stefan
>  Monnier <monnier@iro.umontreal.ca>,  rms@gnu.org
> Date: Sun, 10 Oct 2021 11:46:11 +0100
> 
> I'll leave it up to you guys to figure out what to do here.

I already did, and that's why I asked you to install that change.
With it on a scratch branch, the problem is without any solution
whatsoever.  This is bad for users, IMNSHO.

And that is all I have to say on this matter.

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: João Távora <joaotavora@gmail.com>
>> Cc: Phil Sainty <psainty@orcon.net.nz>,  50959@debbugs.gnu.org,  Stefan
>>  Monnier <monnier@iro.umontreal.ca>,  rms@gnu.org
>> Date: Sun, 10 Oct 2021 11:46:11 +0100
>> 
>> I'll leave it up to you guys to figure out what to do here.
>
> I already did, and that's why I asked you to install that change.
> And that is all I have to say on this matter.

Done.  When someone strongly objects, I always seek confirmation from
the head maintainer before overriding them.

Also marking bug done.  If someone disagrees or finds bugs, let me know.

João

bug#50959: [PATCH] Re: bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Cc: psainty@orcon.net.nz,  50959-done@debbugs.gnu.org,
>   monnier@iro.umontreal.ca,  rms@gnu.org
> Date: Sun, 10 Oct 2021 14:39:03 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: João Távora <joaotavora@gmail.com>
> >> Cc: Phil Sainty <psainty@orcon.net.nz>,  50959@debbugs.gnu.org,  Stefan
> >>  Monnier <monnier@iro.umontreal.ca>,  rms@gnu.org
> >> Date: Sun, 10 Oct 2021 11:46:11 +0100
> >> 
> >> I'll leave it up to you guys to figure out what to do here.
> >
> > I already did, and that's why I asked you to install that change.
> > And that is all I have to say on this matter.
> 
> Done.  When someone strongly objects, I always seek confirmation from
> the head maintainer before overriding them.

Thank you!

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]


  > 2. The shorthands from the buffer where the minibuffer was entered are
  >    _not_ in the completions list, but typing one of them interns the
  >    symbol with those shorthands present, so you get the desired result.
  >    This would fix Phil's visually-copy-and-type scenario.

Would someone please give an explained example to make this concretely clear?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Phil Sainty]

Hi João,

On 2021-10-03 00:02, João Távora wrote:
> In other words, Phil, if you wish me to do anything about this bug, you
> must explain exactly what you are doing and what you expect.  What does
> it mean precisely for "Emacs to recognise the shorthand symbols
> generally".  Currently it "recognizes" them when the buffer where they
> are setup is current.

I'm just the messenger on this occasion.  I was describing to Eli how
shorthands exhibit the exact same problem that was objectionable in the
"nameless" approach, and he asked me to raise it as a bug.

I think that the shorthand symbols will need to be interned to avoid
this (or at least I can't think of any other solution which avoids the
problem).  Perhaps define them as aliases -- or even obsolete aliases,
if the long-term preference is for other libraries to migrate from s-*
and the likes to using the proper names?


-Phil

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Eli Zaretskii]

> Date: Sun, 03 Oct 2021 00:53:55 +1300
> From: Phil Sainty <psainty@orcon.net.nz>
> Cc: Eli Zaretskii <eliz@gnu.org>, 50959@debbugs.gnu.org
> 
> Hi João,
> 
> On 2021-10-03 00:02, João Távora wrote:
> > In other words, Phil, if you wish me to do anything about this bug, you
> > must explain exactly what you are doing and what you expect.  What does
> > it mean precisely for "Emacs to recognise the shorthand symbols
> > generally".  Currently it "recognizes" them when the buffer where they
> > are setup is current.
> 
> I'm just the messenger on this occasion.  I was describing to Eli how
> shorthands exhibit the exact same problem that was objectionable in the
> "nameless" approach, and he asked me to raise it as a bug.

Let me translate: is "C-h" command the only use case you have in mind
where you saw a problem with shorthands?  If there are others, and
they involve Emacs's internal functionalities (as opposed to, say,
searching with Grep or some other text-oriented tool), please describe
them.

> I think that the shorthand symbols will need to be interned to avoid
> this (or at least I can't think of any other solution which avoids the
> problem).

They are already, or something very similar.

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[Phil Sainty]

On 2021-10-03 01:10, Eli Zaretskii wrote:
> Let me translate: is "C-h" command the only use case you have in mind
> where you saw a problem with shorthands?  If there are others, and
> they involve Emacs's internal functionalities (as opposed to, say,
> searching with Grep or some other text-oriented tool), please describe
> them.

I think it's literally anything that hasn't has custom support added
as part of the shorthands feature?

E.g.:

* describe-function|-variable|-face|-symbol
* apropos|-command|-variable|-user-option|-symbol
* customize-apropos|-face|-variable|-option
* info-lookup-symbol
* execute-extended-command
* anything using read-command|-variable|-face-name
* anything using these interactive codes:

a -- Function name: symbol with a function definition.
C -- Command name: symbol with interactive function definition.
      This skips events that are integers or symbols.
S -- Any symbol.
v -- Variable name: symbol that is ‘custom-variable-p’.

etc, etc...

Basically everything?


-Phil

bug#50959: 28.0.50; Shorthand symbols are unknown to Emacs

[João Távora]

On Sat, Oct 2, 2021 at 1:37 PM Phil Sainty <psainty@orcon.net.nz> wrote:
>
> On 2021-10-03 01:10, Eli Zaretskii wrote:
> > Let me translate: is "C-h" command the only use case you have in mind
> > where you saw a problem with shorthands?  If there are others, and
> > they involve Emacs's internal functionalities (as opposed to, say,
> > searching with Grep or some other text-oriented tool), please describe
> > them.
>
> I think it's literally anything that hasn't has custom support added
> as part of the shorthands feature?
>
> E.g.:
>
> * describe-function|-variable|-face|-symbol
> * apropos|-command|-variable|-user-option|-symbol
> * customize-apropos|-face|-variable|-option
> * info-lookup-symbol
> * execute-extended-command
> * anything using read-command|-variable|-face-name
> * anything using these interactive codes:
>
> a -- Function name: symbol with a function definition.
> C -- Command name: symbol with interactive function definition.
>       This skips events that are integers or symbols.
> S -- Any symbol.
> v -- Variable name: symbol that is ‘custom-variable-p’.
>
> etc, etc...
>
> Basically everything?

As it is clear, that's not the case.  These things have and will always
deal with the global table of symbols obarray.  These commands operate on
symbols.  Shorthands are _not_ symbols! They are not! They are a textual
indirection to symbols, which -- as the manual explained long before I touched
it -- are objects composed of four things, blablabla.

Maybe this helps you think: it's just like (intern (concat "foo"
"bar")) is another
type of indirection to a symbol. A run-time indirection.  Shorthands
are read-time
indirections.

So: when the commands you reference are invoked in the buffer where
shorthands exist with point on the shorthand you which to describe, lookup, etc
all those functions and the interactive codes do the right thing.  What is that?
They follow the indirection to the symbol and operate on the symbol,
as they always
have.

If you find some case where they DON'T  follow this indirection, where
they clearly
COULD reasonably follow it then that is a bug and or feature request
for  shorthands.
But you should explain also why think it is reasonable.  Consider how SLIME (and
also proprietary Common Lisp IDE's, I think) have dealt with this issue (search
"package local nicknames" in your favourite search engine).

Soon I will propose that we font-lock shorthands specially (or at least make it
optional).  That should be easy to do, and should, in my opinion, extinguish at
least some of your anguish.

João