What's happened to M-<tab> `completion-at-point'?

What's happened to M-<tab> `completion-at-point'?

[Alan Mackenzie]

Hello, Emacs.

Suddenly, my M-<tab> `completion-at-point' isn't working.  Would
somebody please explain?

This is on a recent master branch build.

I'm not even entirely sure the problem is in Emacs.  Maybe a recent
update to Linux or the kbd package has messed things up.

More precisely, when in an Emacs Lisp Mode buffer, when I type M-<tab> I
get the error message

    <backtab> is undefined

..  I don't recall M-<tab> ever being called "<backtab>", so I at first
tried to find something recent changed in Linux.  No luck, there.

Have I missed some vigorous discussion on emacs-devel which concluded
that the key binding should be removed?  When I do C-h w
completion-at-point, I get told it's not bound to any key.

Why, why, why?

Would somebody please explain _what_ has changed here, and if relevant,
why.

I don't think this will be a popular change.  :-(

Thanks!

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's happened to M-<tab> `completion-at-point'?

[Eli Zaretskii]

> Date: Tue, 3 May 2022 21:06:00 +0000
> From: Alan Mackenzie <acm@muc.de>
> 
> Suddenly, my M-<tab> `completion-at-point' isn't working.  Would
> somebody please explain?

If you press "ESC TAB" instead, what does Emacs do then?

> This is on a recent master branch build.
> 
> I'm not even entirely sure the problem is in Emacs.  Maybe a recent
> update to Linux or the kbd package has messed things up.

Most likely.  M-TAB is frequently used by the OS and the WM on
different systems.

Re: What's happened to M-<tab> `completion-at-point'?

[Alan Mackenzie]

Hello, Eli.

On Wed, May 04, 2022 at 09:32:13 +0300, Eli Zaretskii wrote:
> > Date: Tue, 3 May 2022 21:06:00 +0000
> > From: Alan Mackenzie <acm@muc.de>

> > Suddenly, my M-<tab> `completion-at-point' isn't working.  Would
> > somebody please explain?

> If you press "ESC TAB" instead, what does Emacs do then?

I get the message

    [backtab] is undefined

> > This is on a recent master branch build.

> > I'm not even entirely sure the problem is in Emacs.  Maybe a recent
> > update to Linux or the kbd package has messed things up.

> Most likely.  M-TAB is frequently used by the OS and the WM on
> different systems.

I'm on the Linux console.  It seems that terminfo has started behaving
differently, as I outlined in my reply to Lars.

There doesn't seem to be any problem in X Windows.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's happened to M-<tab> `completion-at-point'?

[Eli Zaretskii]

> Date: Wed, 4 May 2022 16:34:38 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > > Suddenly, my M-<tab> `completion-at-point' isn't working.  Would
> > > somebody please explain?
> 
> > If you press "ESC TAB" instead, what does Emacs do then?
> 
> I get the message
> 
>     [backtab] is undefined

And what does "C-h l" show after that?

Re: What's happened to M-<tab> `completion-at-point'?

[Alan Mackenzie]

Hello, Eli.

On Wed, May 04, 2022 at 19:40:01 +0300, Eli Zaretskii wrote:
> > Date: Wed, 4 May 2022 16:34:38 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > > Suddenly, my M-<tab> `completion-at-point' isn't working.  Would
> > > > somebody please explain?

> > > If you press "ESC TAB" instead, what does Emacs do then?

> > I get the message

> >     [backtab] is undefined

> And what does "C-h l" show after that?

If I first press ESC TAB and C-h l, followed by C-M-i and C-h l, I get
this:

 ESC TAB ;; nil
 C-h l   ;; view-lossage
 ESC TAB ;; nil
 C-h l   ;; view-lossage

..

I think I've found the problem.  A new version of ncurses was
installed on my machine on 2022-05-01.  It contains a version of
terminfo, specifically /etc/terminfo/l/linux.

The new version is ncurses-6.3_p20211106.  The old version was
ncurses-6.2_p20210619.

So, possibly we need to amend Emacs (? src/term.c) to work properly on
this version of the Linux console.  I don't think this affects any other
type of terminal.  In particular, Emacs works OK for me under XFCE4 on X.

I think I'll be raising a bug report for this.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's happened to M-<tab> `completion-at-point'?

[Eli Zaretskii]

> Date: Wed, 4 May 2022 18:13:30 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > >     [backtab] is undefined
> 
> > And what does "C-h l" show after that?
> 
> If I first press ESC TAB and C-h l, followed by C-M-i and C-h l, I get
> this:
> 
>  ESC TAB ;; nil
>  C-h l   ;; view-lossage
>  ESC TAB ;; nil
>  C-h l   ;; view-lossage

Sp who is converting this to <backtab>?

> I think I've found the problem.  A new version of ncurses was
> installed on my machine on 2022-05-01.  It contains a version of
> terminfo, specifically /etc/terminfo/l/linux.
> 
> The new version is ncurses-6.3_p20211106.  The old version was
> ncurses-6.2_p20210619.
> 
> So, possibly we need to amend Emacs (? src/term.c) to work properly on
> this version of the Linux console.

Amend how?  I still don't understand where did <backtab> come from.
If you do understand, can you describe that?

Re: What's happened to M-<tab> `completion-at-point'?

[Alan Mackenzie]

Hello, Eli.

On Wed, May 04, 2022 at 21:55:28 +0300, Eli Zaretskii wrote:
> > Date: Wed, 4 May 2022 18:13:30 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > >     [backtab] is undefined

> > > And what does "C-h l" show after that?

> > If I first press ESC TAB and C-h l, followed by C-M-i and C-h l, I get
> > this:

> >  ESC TAB ;; nil
> >  C-h l   ;; view-lossage
> >  ESC TAB ;; nil
> >  C-h l   ;; view-lossage

> So who is converting this to <backtab>?

I don't know, yet.

> > I think I've found the problem.  A new version of ncurses was
> > installed on my machine on 2022-05-01.  It contains a version of
> > terminfo, specifically /etc/terminfo/l/linux.

> > The new version is ncurses-6.3_p20211106.  The old version was
> > ncurses-6.2_p20210619.

> > So, possibly we need to amend Emacs (? src/term.c) to work properly on
> > this version of the Linux console.

> Amend how?  I still don't understand where did <backtab> come from.
> If you do understand, can you describe that?

I don't understand it either, yet.  To be sure that terminfo is the
problem, I'll have to do something like reinstalling the old version of
ncurses, and seeing the problem is no longer there.

But in src/term.c L1258 appears:

      {"kB", "backtab"},    /* terminfo */

The "kB" is a terminfo code for back-tab, and the "backtab" might
somehow be the source for the "[backtab] is undefined" error message.

I'm still at an early stage of understanding this bug.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's happened to M-<tab> `completion-at-point'?

[Yuri Khan]

On Thu, 5 May 2022 at 02:10, Alan Mackenzie <acm@muc.de> wrote:

> I don't understand it either, yet.  To be sure that terminfo is the
> problem, I'll have to do something like reinstalling the old version of
> ncurses, and seeing the problem is no longer there.
>
> But in src/term.c L1258 appears:
>
>       {"kB", "backtab"},    /* terminfo */
>
> The "kB" is a terminfo code for back-tab, and the "backtab" might
> somehow be the source for the "[backtab] is undefined" error message.

You might be on to something.

    $ infocmp linux
    #       Reconstructed via infocmp from file: /lib/terminfo/l/linux
    linux|Linux console,
            am, bce, ccc, eo, mir, msgr, xenl, xon,
            …
            kb2=\E[G, kbs=^?, kcbt=\E^I, kcub1=\E[D, kcud1=\E[B,
            …

In other words, the terminfo database declares a capability named
‘kcbt’ with the value ESC TAB. And ‘man terminfo’ says ‘kcbt’ is the
backtab key.

Re: What's happened to M-<tab> `completion-at-point'?

[Stefan Monnier]

>     $ infocmp linux
>     #       Reconstructed via infocmp from file: /lib/terminfo/l/linux
>     linux|Linux console,
>             am, bce, ccc, eo, mir, msgr, xenl, xon,
>             …
>             kb2=\E[G, kbs=^?, kcbt=\E^I, kcub1=\E[D, kcud1=\E[B,
>             …
>
> In other words, the terminfo database declares a capability named
> ‘kcbt’ with the value ESC TAB. And ‘man terminfo’ says ‘kcbt’ is the
> backtab key.

Indeed.  And then Emacs's C code builds a default `input-decode-map`
which maps ESC TAB to `backtab`.

This is a weird choice, tho (on their side).
AFAIK the `backtab` is usually used for `S-tab` rather than `M-tab`.

What do you get if you hit the TAB key together with the Shift modifier?
Does Emacs also receive the ESC TAB byte sequence in that (and then maps
it back (correctly this time) to `backtab`)?

You should definitely report this as a problem.


        Stefan

Re: What's happened to M-<tab> `completion-at-point'?

[Yuri Khan]

On Thu, 5 May 2022 at 03:16, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> AFAIK the `backtab` is usually used for `S-tab` rather than `M-tab`.
> What do you get if you hit the TAB key together with the Shift modifier?
> Does Emacs also receive the ESC TAB byte sequence in that (and then maps
> it back (correctly this time) to `backtab`)?

On my Ubuntu 22.04 with Linux 5.15.0, in tty, both Shift+Tab and
Alt+Tab produce an ESC TAB sequence. So probably the problem on the
ncurses/terminfo side is induced by a problem on the Linux side.

(In my view, every instance where two distinct keys produce the same
code is a problem.)

Re: What's happened to M-<tab> `completion-at-point'?

[Stefan Monnier]

Yuri Khan [2022-05-05 03:31:24] wrote:
> On Thu, 5 May 2022 at 03:16, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> AFAIK the `backtab` is usually used for `S-tab` rather than `M-tab`.
>> What do you get if you hit the TAB key together with the Shift modifier?
>> Does Emacs also receive the ESC TAB byte sequence in that (and then maps
>> it back (correctly this time) to `backtab`)?
> On my Ubuntu 22.04 with Linux 5.15.0, in tty, both Shift+Tab and
> Alt+Tab produce an ESC TAB sequence.

Aha!

   Shift+Tab === Linux kernel ===> ESC TAB === ncurses ===> backtab

makes sense.

> So probably the problem on the
> ncurses/terminfo side is induced by a problem on the Linux side.

Indeed, the problem seems to be that both Shift+Tab and Alt+Tab emit the
same byte sequence (the byte sequence historically used for M-Tab) and
are hence indistinguishable :-(

In the mean time we can add a hack in lisp/term/linux.el to remove the
bogus `ESC TAB => backtab` remapping since I suspect that `Alt+Tab` is
used much more often than `Shift+Tab` in Emacs running in the
Linux console.


        Stefan

Re: What's happened to M-<tab> `completion-at-point'?

[Alan Mackenzie]

Hello, Yuri.

On Thu, May 05, 2022 at 03:31:24 +0700, Yuri Khan wrote:
> On Thu, 5 May 2022 at 03:16, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> > AFAIK the `backtab` is usually used for `S-tab` rather than `M-tab`.
> > What do you get if you hit the TAB key together with the Shift modifier?
> > Does Emacs also receive the ESC TAB byte sequence in that (and then maps
> > it back (correctly this time) to `backtab`)?

> On my Ubuntu 22.04 with Linux 5.15.0, in tty, both Shift+Tab and
> Alt+Tab produce an ESC TAB sequence. So probably the problem on the
> ncurses/terminfo side is induced by a problem on the Linux side.

I don't think this is actually the case.  If you look at the keyboard
mapping file in the kernel:

/usr/src/linux-5.15.32/drivers/tty/vt/defkeymap.map

, the entry for key 15, the tab key, looks like this:

#########################################################################
keycode  15 = Tab              Tab
        alt     keycode  15 = Meta_Tab
#########################################################################

..  I think that is specifying that both TAB and <shift>TAB produce just
the Tab character, whereas <alt>Tab produces Meta_Tab.  This is surely
correct.

Possibly it was wrong in the kernel, and has been fixed sometime before
5.15.32, but I don't think so.

Or, more likely, the collection of keyboard maps in /usr/share/keymaps
contains lots of erroneous keymaps.  Indeed, looking at just one,
/usr/share/keymaps/i386/qwerty/uk.map.gz we see this:

#########################################################################
keycode  15 = Tab
        shift   keycode  15 = Meta_Tab       <===========================
#########################################################################

..  So I would guess the ncurses maintainers felt themselves in an
impossible position, and made the workaround they felt would cause the
least damage.  Emacs on the Linux console is just an unfortunate
casualty.

Maybe the best thing we can do in Emacs is to remove that entry for "kB"
in src/term.c.  That would prevent Emacs recognising <shift>TAB on all
these misconfigured keyboards.  I think that is a lesser evil than
failing to recognise <meta>TAB.

> (In my view, every instance where two distinct keys produce the same
> code is a problem.)

I think you might be exaggerating a little, but I've got a lot of
sympathy for that view.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's happened to M-<tab> `completion-at-point'?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

On Wed, May 04, 2022 at 09:07:04PM +0000, Alan Mackenzie wrote:
> Hello, Yuri.
> 
> On Thu, May 05, 2022 at 03:31:24 +0700, Yuri Khan wrote:
> > On Thu, 5 May 2022 at 03:16, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
> > > AFAIK the `backtab` is usually used for `S-tab` rather than `M-tab`.
> > > What do you get if you hit the TAB key together with the Shift modifier?
> > > Does Emacs also receive the ESC TAB byte sequence in that (and then maps
> > > it back (correctly this time) to `backtab`)?
> 
> > On my Ubuntu 22.04 with Linux 5.15.0, in tty, both Shift+Tab and
> > Alt+Tab produce an ESC TAB sequence. So probably the problem on the
> > ncurses/terminfo side is induced by a problem on the Linux side.
> 
> I don't think this is actually the case.  If you look at the keyboard
> mapping file in the kernel:
> 
> /usr/src/linux-5.15.32/drivers/tty/vt/defkeymap.map
> 
> , the entry for key 15, the tab key, looks like this:
> 
> #########################################################################
> keycode  15 = Tab              Tab
>         alt     keycode  15 = Meta_Tab
> #########################################################################

I can confirm here that alt-tab and shift-tab both translate to esc+tab.
Only the Linux console, bash cat and hexdump -C were involved.

Debian buster, Linux kernel 5.10.0-14-amd64.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: What's happened to M-<tab> `completion-at-point'?

[Eli Zaretskii]

> Date: Wed, 4 May 2022 21:07:04 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Eli Zaretskii <eliz@gnu.org>,
>   Emacs developers <emacs-devel@gnu.org>
> From: Alan Mackenzie <acm@muc.de>
> 
> Maybe the best thing we can do in Emacs is to remove that entry for "kB"
> in src/term.c.  That would prevent Emacs recognising <shift>TAB on all
> these misconfigured keyboards.  I think that is a lesser evil than
> failing to recognise <meta>TAB.

This doesn't sound like a good idea to me.  Emacs shouldn't try
second-guessing the user's keyboard configuration, it isn't in our
mandate.

If the terminfo database you have doesn't do what you want, why can't
you modify it?  The tools to do that are available, and aren't part of
Emacs.  (We could include the instructions for making such a change in
PROBLEMS, if this is a common issue.)  Of course, the best solution
would be if the distro changes the terminfo database.

Re: What's happened to M-<tab> `completion-at-point'?

[Alan Mackenzie]

Hello, Eli.

On Thu, May 05, 2022 at 09:20:17 +0300, Eli Zaretskii wrote:
> > Date: Wed, 4 May 2022 21:07:04 +0000
> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Eli Zaretskii <eliz@gnu.org>,
> >   Emacs developers <emacs-devel@gnu.org>
> > From: Alan Mackenzie <acm@muc.de>

> > Maybe the best thing we can do in Emacs is to remove that entry for "kB"
> > in src/term.c.  That would prevent Emacs recognising <shift>TAB on all
> > these misconfigured keyboards.  I think that is a lesser evil than
> > failing to recognise <meta>TAB.

> This doesn't sound like a good idea to me.  Emacs shouldn't try
> second-guessing the user's keyboard configuration, it isn't in our
> mandate.

We have a bug here, Emacs does not work properly.  While it is true in
theory that a user might be able to fix it in her configuration, in
practice this is just too difficult for a user to diagnose and fix.
Note that all the key sequences M-tab, ESC TAB, and C-M-i are affected.

The change from ncurses-6.2 to ncurses-6.3 broke the Linux console
keyboard, in that terminfo now directs ESC TAB to be translated to
backtab.  This was almost certainly intentional, possibly prompted by
the misconfiguration of so many Linux keyboard layouts (in
/usr/share/keymaps/...), where the <shift>TAB key sequence produces the
characters ESC TAB.  I can imagine that there was a lot of strenuous
discussion on the ncurses mailing list before making this change, and
that it was done with regret.

> If the terminfo database you have doesn't do what you want, why can't
> you modify it?  The tools to do that are available, and aren't part of
> Emacs.

I could do this without too much difficulty for myself personally, but
that doesn't fix the bug for other users.  I don't think we want to
distribute a version of terminfo just for Linux Emacs users.

> (We could include the instructions for making such a change in
> PROBLEMS, if this is a common issue.)  Of course, the best solution
> would be if the distro changes the terminfo database.

There are many GNU/Linux distros all distributing Emacs, and I think it
likely that few, if any, will be prepared to fix terminfo (likely
"breaking" other programs) for the sake of Emacs.

Our problem here is caused by an ad hoc change to terminfo.  Why can't
we fix it likewise by an ad hoc change in Emacs, that would prevent ESC
TAB only on the Linux keyboard from being changed into backtab.  We
could make this optional, either by a run-time or a configure-time
option.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's happened to M-<tab> `completion-at-point'?

[Eli Zaretskii]

> Date: Thu, 5 May 2022 16:57:55 +0000
> Cc: yuri.v.khan@gmail.com, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > > From: Alan Mackenzie <acm@muc.de>
> 
> > > Maybe the best thing we can do in Emacs is to remove that entry for "kB"
> > > in src/term.c.  That would prevent Emacs recognising <shift>TAB on all
> > > these misconfigured keyboards.  I think that is a lesser evil than
> > > failing to recognise <meta>TAB.
> 
> > This doesn't sound like a good idea to me.  Emacs shouldn't try
> > second-guessing the user's keyboard configuration, it isn't in our
> > mandate.
> 
> We have a bug here, Emacs does not work properly.

That's your opinion.  It might also be the opinion of some others.
But the (mis)behavior is not due to Emacs code, it is due to ncurses
and the terminfo database that comes with it.  Why should we fix in
Emacs something that should be fixed outside of it, if at all, and
with such drastic measures on top of that?  Entirely ignoring a
terminfo capability is a drastic measure, which will affect not just
this particular aspect of the Emacs behavior.

> While it is true in
> theory that a user might be able to fix it in her configuration, in
> practice this is just too difficult for a user to diagnose and fix.

I don't see the difficulty.  Moreover, I don't even see the need to
fix it for every user out there.  We don't even know _why_ did ncurses
make this change.  Nor do I expect many Emacs users to use it on the
Linux console in the first place.  Your proposed "solution", OTOH,
affects _all_ users of TTY frames in Emacs, regardless of whether they
are or aren't affected by the M-TAB behavior of the Linux console.

> The change from ncurses-6.2 to ncurses-6.3 broke the Linux console
> keyboard, in that terminfo now directs ESC TAB to be translated to
> backtab.  This was almost certainly intentional, possibly prompted by
> the misconfiguration of so many Linux keyboard layouts (in
> /usr/share/keymaps/...), where the <shift>TAB key sequence produces the
> characters ESC TAB.  I can imagine that there was a lot of strenuous
> discussion on the ncurses mailing list before making this change, and
> that it was done with regret.

Why do we need to "imagine"? why not find those discussions and read
them, and/or ask the ncurses developers to reconsider and/or explain
to us why this change is TRT from their POV?

Armed with their opinions and rationale, we could then revisit this
issue and make up our own minds about the best course of action.

But making such serious changes with such wide consequences is not
something I'm interested in or agree to without a very good
understanding of the reasons why ncurses made the change.  Sorry, not
on my watch.

> > If the terminfo database you have doesn't do what you want, why can't
> > you modify it?  The tools to do that are available, and aren't part of
> > Emacs.
> 
> I could do this without too much difficulty for myself personally, but
> that doesn't fix the bug for other users.  I don't think we want to
> distribute a version of terminfo just for Linux Emacs users.

As long as this is a problem for only a few users, we can tell in
PROBLEMS how to make the change.  It isn't too complicated.

> > (We could include the instructions for making such a change in
> > PROBLEMS, if this is a common issue.)  Of course, the best solution
> > would be if the distro changes the terminfo database.
> 
> There are many GNU/Linux distros all distributing Emacs, and I think it
> likely that few, if any, will be prepared to fix terminfo (likely
> "breaking" other programs) for the sake of Emacs.

Each user can make up his/her own mind whether they want to make the
change and whether it will break anything for them.  We don't need
(and should not) second-guess their needs.

> Our problem here is caused by an ad hoc change to terminfo.  Why can't
> we fix it likewise by an ad hoc change in Emacs, that would prevent ESC
> TAB only on the Linux keyboard from being changed into backtab.  We
> could make this optional, either by a run-time or a configure-time
> option.

Because it isn't our problem to begin with, and we don't even
understand its nature well enough to make a good decision.  If you
want to facilitate the decision-making process, please find the
discussions that led to this change in ncurses and/or talk to the
ncurses developers about the issue and find out what is their take on
it.

Re: What's happened to M-<tab> `completion-at-point'?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > The change from ncurses-6.2 to ncurses-6.3 broke the Linux console
  > > keyboard, in that terminfo now directs ESC TAB to be translated to
  > > backtab.

Would a few people please volunteer to talk with the ncurses
developers about what problem they wanted to fix?  Given that this fix
ha a problem, it would be good to look for another one.  Have people
identified those who were involved?

The discussion should also include Thomas Dickey <dickey@his.com>,
the ncurses maintainer.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What's happened to M-<tab> `completion-at-point'?

[Alan Mackenzie]

Hello, Richard.

On Fri, May 06, 2022 at 19:19:56 -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

>   > > The change from ncurses-6.2 to ncurses-6.3 broke the Linux console
>   > > keyboard, in that terminfo now directs ESC TAB to be translated to
>   > > backtab.

> Would a few people please volunteer to talk with the ncurses
> developers about what problem they wanted to fix?  Given that this fix
> ha a problem, it would be good to look for another one.  Have people
> identified those who were involved?

It seems that the Linux keyboard layout is very old, and too few
keycodes were envisaged for existing key combinations.  This led to
Meta_Tab being used sometimes for <alt><tab> and sometimes (wrongly in
my view) for <shift><tab>.

Thomas Dickey has tracked down a bug "fix" in SuSE GNU/Linux from (I
think it was) 2007 where someone recorded that the change was made to
assigning <shift><tab> to Meta_Tab, but without noting down any reasons.

TD also justified the change in ncurses-6.3 by saying that the
<shift><tab> key assignment had come to prevail in the main GNU/Linux
distributions, and this change merely reflects current practice.

> The discussion should also include Thomas Dickey <dickey@his.com>,
> the ncurses maintainer.

Thomas Dickey is already in the conversation, having answered a post to
bug-ncurses@gnu.org with Subject: Emacs difficulties in linux console
with ncurses-6.3 caused by kcbt=\E^I.  That thread was cross-posted to
emacs-devel.

The conversation has not yet been resolved.

> -- 
> Dr Richard Stallman (https://stallman.org)
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's happened to M-<tab> `completion-at-point'?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Thomas Dickey is already in the conversation, having answered a post to
  > bug-ncurses@gnu.org with Subject: Emacs difficulties in linux console
  > with ncurses-6.3 caused by kcbt=\E^I.  That thread was cross-posted to
  > emacs-devel.

  > The conversation has not yet been resolved.

It sounds like this is a choice between two unsatisfying options.

Is it possible to do better than that, with the broad range hardware
that we have to support?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What's happened to M-<tab> `completion-at-point'?

[Alan Mackenzie]

Hello, Yuri.

On Thu, May 05, 2022 at 02:47:41 +0700, Yuri Khan wrote:
> On Thu, 5 May 2022 at 02:10, Alan Mackenzie <acm@muc.de> wrote:

> > I don't understand it either, yet.  To be sure that terminfo is the
> > problem, I'll have to do something like reinstalling the old version of
> > ncurses, and seeing the problem is no longer there.

> > But in src/term.c L1258 appears:

> >       {"kB", "backtab"},    /* terminfo */

> > The "kB" is a terminfo code for back-tab, and the "backtab" might
> > somehow be the source for the "[backtab] is undefined" error message.

> You might be on to something.

>     $ infocmp linux
>     #       Reconstructed via infocmp from file: /lib/terminfo/l/linux
>     linux|Linux console,
>             am, bce, ccc, eo, mir, msgr, xenl, xon,
>             …
>             kb2=\E[G, kbs=^?, kcbt=\E^I, kcub1=\E[D, kcud1=\E[B,
>             …

Thanks for the tip!  I didn't know about infocmp.

> In other words, the terminfo database declares a capability named
> ‘kcbt’ with the value ESC TAB. And ‘man terminfo’ says ‘kcbt’ is the
> backtab key.

I ran $ infocmp linux on the ncurses 6.3, then reinstalled 6.2 and did
it again.  A diff shows this:

--- /home/acm/infocmp-linux-6.2 2022-05-04 20:16:01.609557894 +0000
+++ /home/acm/infocmp-linux-6.3 2022-05-04 20:09:02.046581014 +0000
@@ -1,5 +1,5 @@
 #      Reconstructed via infocmp from file: /etc/terminfo/l/linux
-linux|linux console,
+linux|Linux console,
        am, bce, ccc, eo, mir, msgr, xenl, xon,
        colors#8, it#8, ncv#18, pairs#64,
        acsc=++\,\,--..00``aaffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
@@ -14,7 +14,7 @@
        home=\E[H, hpa=\E[%i%p1%dG, ht=^I, hts=\EH, ich=\E[%p1%d@,
        ich1=\E[@, il=\E[%p1%dL, il1=\E[L, ind=\n,
        initc=\E]P%p1%x%p2%{255}%*%{1000}%/%02x%p3%{255}%*%{1000}%/%02x%p4%{255}%*%{1000}%/%02x,
-       kb2=\E[G, kbs=^?, kcbt=\E[Z, kcub1=\E[D, kcud1=\E[B,
+       kb2=\E[G, kbs=^?, kcbt=\E^I, kcub1=\E[D, kcud1=\E[B,
        kcuf1=\E[C, kcuu1=\E[A, kdch1=\E[3~, kend=\E[4~, kf1=\E[[A,
        kf10=\E[21~, kf11=\E[23~, kf12=\E[24~, kf13=\E[25~,
        kf14=\E[26~, kf15=\E[28~, kf16=\E[29~, kf17=\E[31~,

..  So the one substantial change has been changing from kcbt=\E[Z to
kcbt=\E^I.  6.3 is instructing Emacs to interpret 0x19 0x09 as
back-tab, whereas before it was interpreted simply as ESC TAB.

This looks like it might be a bug in ncurses-6.3.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: What's happened to M-<tab> `completion-at-point'?

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> I think I've found the problem.  A new version of ncurses was
>> installed on my machine on 2022-05-01.  It contains a version of
>> terminfo, specifically /etc/terminfo/l/linux.
>> 
>> The new version is ncurses-6.3_p20211106.  The old version was
>> ncurses-6.2_p20210619.
>> 
>> So, possibly we need to amend Emacs (? src/term.c) to work properly
>> on this version of the Linux console.
>
> Amend how?  I still don't understand where did <backtab> come from.
> If you do understand, can you describe that?

I never use the Linux console (unless something is severely broken) but
can confirm Alan's observation.  With emacs -Q -nw , all of M-TAB, ESC
TAB, and C-M-i say <backtab> is undefined.  That happens with emacs 27,
28, and the current master.  I also have ncurses 6.3 which ships with
tons of files below /usr/share/terminfo/ including the l/linux Alan
mentioned.

I don't have a previous ncurses version handy to test if a downgrade
would help.

Bye,
Tassilo

Re: What's happened to M-<tab> `completion-at-point'?

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: Alan Mackenzie <acm@muc.de>, emacs-devel@gnu.org
> Date: Wed, 04 May 2022 21:16:42 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I think I've found the problem.  A new version of ncurses was
> >> installed on my machine on 2022-05-01.  It contains a version of
> >> terminfo, specifically /etc/terminfo/l/linux.
> >> 
> >> The new version is ncurses-6.3_p20211106.  The old version was
> >> ncurses-6.2_p20210619.
> >> 
> >> So, possibly we need to amend Emacs (? src/term.c) to work properly
> >> on this version of the Linux console.
> >
> > Amend how?  I still don't understand where did <backtab> come from.
> > If you do understand, can you describe that?
> 
> I never use the Linux console (unless something is severely broken) but
> can confirm Alan's observation.  With emacs -Q -nw , all of M-TAB, ESC
> TAB, and C-M-i say <backtab> is undefined.  That happens with emacs 27,
> 28, and the current master.  I also have ncurses 6.3 which ships with
> tons of files below /usr/share/terminfo/ including the l/linux Alan
> mentioned.

My point is that if <backtab> comes from outside of Emacs, there's
nothing we can do about it.

But Alan's report says that "C-h l" tells a different story: that
Emacs got "ESC TAB".  In which case I find it hard to understand how
ncurses or the terminfo database could be involved.  I'm probably
missing something.

Re: What's happened to M-<tab> `completion-at-point'?

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> I never use the Linux console (unless something is severely broken) but
>> can confirm Alan's observation.  With emacs -Q -nw , all of M-TAB, ESC
>> TAB, and C-M-i say <backtab> is undefined.  That happens with emacs 27,
>> 28, and the current master.  I also have ncurses 6.3 which ships with
>> tons of files below /usr/share/terminfo/ including the l/linux Alan
>> mentioned.
>
> My point is that if <backtab> comes from outside of Emacs, there's
> nothing we can do about it.
>
> But Alan's report says that "C-h l" tells a different story: that
> Emacs got "ESC TAB".

That's right, all of M-TAB, ESC TAB, C-M-i are shown as ESC TAB in C-h
l.

Bye,
Tassilo

Re: What's happened to M-<tab> `completion-at-point'?

[Andreas Schwab]

On Mai 04 2022, Eli Zaretskii wrote:

> But Alan's report says that "C-h l" tells a different story: that
> Emacs got "ESC TAB".  In which case I find it hard to understand how
> ncurses or the terminfo database could be involved.  I'm probably
> missing something.

The translation probably comes from input-decode-map, which is filled
from terminfo.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: What's happened to M-<tab> `completion-at-point'?

[Lars Ingebrigtsen]

Alan Mackenzie <acm@muc.de> writes:

> Have I missed some vigorous discussion on emacs-devel which concluded
> that the key binding should be removed?  When I do C-h w
> completion-at-point, I get told it's not bound to any key.

emacs -Q, `C-w completion-at-point':

completion-at-point is on C-M-i, <menu-bar> <lisp-interaction> <Complete Lisp Symbol>

And C-M-i is a different way to spell M-<tab>.  So perhaps you have some
local customisations?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: What's happened to M-<tab> `completion-at-point'?

[Alan Mackenzie]

Hello, Lars.

On Wed, May 04, 2022 at 08:33:52 +0200, Lars Ingebrigtsen wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > Have I missed some vigorous discussion on emacs-devel which concluded
> > that the key binding should be removed?  When I do C-h w
> > completion-at-point, I get told it's not bound to any key.

> emacs -Q, `C-w completion-at-point':

> completion-at-point is on C-M-i, <menu-bar> <lisp-interaction> <Complete Lisp Symbol>

> And C-M-i is a different way to spell M-<tab>.  So perhaps you have some
> local customisations?

OK, so at least there haven't been breaking changes in Emacs in the last
few weeks.

I think my problem lies in terminfo.

I've still got an old system on my machine (about 4 years old), and I
started Emacs 26.1 on it with emacs -Q.  I type

    M-: (lookup-key input-decode-map "\C-\M-i") RET

I get nil.

When I start Emacs 26.1, the same binary, on my up to date system and
type that incantation, I get [backtab].

I think this has happened very recently, within the last couple of weeks
or so.  I don't think the problem is in the kernel, rather it's in some
user-space supporting software close to the kernel.  The comments in
term.c suggest something for "backtab" is loaded from terminfo.

The trouble is, there is no package called "terminfo" on my GNU/Linux.
I don't know where to find it.  Have you any ideas?

My problem might even merit an entry in the PROBLEMS file.

> -- 
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no

-- 
Alan Mackenzie (Nuremberg, Germany).