bug#67470: 29.1; move-end-of-line behaves badly with eglot type annotations

bug#67470: 29.1; move-end-of-line behaves badly with eglot type annotations

[matthewktromp]

1. Go to a buffer with some code
2. Start eglot
3. Eglot adds type annotations
4. Go to some line that ends with a type annotation
5. Press C-e to go to the end of the line
6. Navigate around with C-n and C-p
7. Note that point will jump to the column of the position of the end of
the type annotation, rather than the end of the code.

For instance, if you have some code and some annotations (represented
with a) like so, with point at |:

some|code aaaaaaaaaa
here are some more lines which do not have annotations

Pressing C-e will move point to the end of the line

some cod| aaaaaaaaaa
here are some more lines which do not have annotations

Then when you press C-n, instead of point moving to the same column in
the next line, like so:

some code aaaaaaaaaa
here are|some more lines which do not have annotations

It will instead jump to the column of the end of the annotation, like
so:

some code aaaaaaaaaa
here are some more |ines which do not have annotations


In GNU Emacs 29.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.38,
cairo version 1.17.8)
Windowing system distributor 'Microsoft Corporation', version 11.0.12010000
System Description: Arch Linux

Configured using:
 'configure --sysconfdir=/etc --prefix=/usr --libexecdir=/usr/lib
 --with-tree-sitter --localstatedir=/var --with-cairo
 --disable-build-details --with-harfbuzz --with-libsystemd
 --with-modules --with-x-toolkit=gtk3 'CFLAGS=-march=x86-64
 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2
 -Wformat -Werror=format-security -fstack-clash-protection
 -fcf-protection -g
 -ffile-prefix-map=/build/emacs/src=/usr/src/debug/emacs -flto=auto'
 'LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Rust

Minor modes in effect:
  shell-dirtrack-mode: t
  delete-selection-mode: t
  repeat-mode: t
  eglot-inlay-hints-mode: t
  eglot--managed-mode: t
  flymake-mode: t
  windmove-mode: t
  global-corfu-mode: t
  corfu-mode: t
  pixel-scroll-precision-mode: t
  desktop-save-mode: t
  server-mode: t
  save-place-mode: t
  savehist-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tab-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rfc822 mml mml-sec epa epg
rfc6068 epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader nroff-mode ffap find-dired grep
cus-start display-line-numbers etags fileloop generator pcmpl-unix
sh-script executable shell pcomplete dabbrev face-remap files-x pulse
misearch multi-isearch mule-util cl-extra rng-xsd xsd-regexp rng-cmpct
rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt
rng-util rng-pttrn nxml-ns nxml-mode nxml-outln nxml-rap nxml-util
nxml-enc xmltok yank-media mhtml-mode css-mode smie eww xdg url-queue
shr pixel-fill kinsoku url-file svg xml puny mm-url gnus nnheader
gnus-util range color js c-ts-common treesit cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs sgml-mode
facemenu dom conf-mode dired-aux dired dired-loaddefs agda2-mode derived
agda2-queue agda2-abbrevs skeleton agda2-highlight agda-input quail
annotation eri time-date vc-git diff-mode easy-mmode vc-dispatcher
delsel rect rust-utils rust-mode rx rust-rustfmt rust-playpen
rust-compile rust-cargo repeat eglot external-completion array
filenotify jsonrpc ert ewoc debug backtrace help-mode find-func xref
flymake-proc flymake thingatpt warnings compile text-property-search
comint ansi-osc ansi-color project imenu windmove sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils corfu compat
pixel-scroll cua-base ring desktop frameset agda2 cus-edit pp cus-load
icons wid-edit server saveplace savehist modus-vivendi-theme
modus-themes pcase avy-autoloads corfu-autoloads compat-autoloads
debbugs-autoloads exwm-autoloads geiser-autoloads racket-mode-autoloads
rust-mode-autoloads info sicp-info-autoloads finder-inf vundo-autoloads
which-key-autoloads xelb-autoloads package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process emacs)

Memory information:
((conses 16 563494 56735)
 (symbols 48 25859 2)
 (strings 32 93540 3924)
 (string-bytes 1 2840663)
 (vectors 16 49716)
 (vector-slots 8 696206 66122)
 (floats 8 315 539)
 (intervals 56 29777 496)
 (buffers 984 133))

bug#67470: 29.1; move-end-of-line behaves badly with eglot type annotations

[Eli Zaretskii]

> From: matthewktromp@gmail.com
> Date: Sun, 26 Nov 2023 23:37:04 +0000
> 
> 
> 1. Go to a buffer with some code
> 2. Start eglot
> 3. Eglot adds type annotations
> 4. Go to some line that ends with a type annotation
> 5. Press C-e to go to the end of the line
> 6. Navigate around with C-n and C-p
> 7. Note that point will jump to the column of the position of the end of
> the type annotation, rather than the end of the code.
> 
> For instance, if you have some code and some annotations (represented
> with a) like so, with point at |:
> 
> some|code aaaaaaaaaa
> here are some more lines which do not have annotations
> 
> Pressing C-e will move point to the end of the line
> 
> some cod| aaaaaaaaaa
> here are some more lines which do not have annotations
> 
> Then when you press C-n, instead of point moving to the same column in
> the next line, like so:
> 
> some code aaaaaaaaaa
> here are|some more lines which do not have annotations
> 
> It will instead jump to the column of the end of the annotation, like
> so:
> 
> some code aaaaaaaaaa
> here are some more |ines which do not have annotations

João, how are those annotations shown?  I cannot run Eglot on my
system to try this recipe, but if I just put an overlay with a before-
or after-string at the end of a line, C-e moves to after the overlay
string, not before it.  Does Eglot do anything to change that?

In any case, I'm guessing that what the OP sees is a general-purpose
cursor movement in the presence of overlay strings, and can only be
changed if Eglot had its own commands for C-n/C-p, which set
goal-column or something before moving.

bug#67470: 29.1; move-end-of-line behaves badly with eglot type annotations

[João Távora]

On Mon, Nov 27, 2023 at 12:35 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: matthewktromp@gmail.com
> > Date: Sun, 26 Nov 2023 23:37:04 +0000
> >
> >
> > 1. Go to a buffer with some code
> > 2. Start eglot
> > 3. Eglot adds type annotations
> > 4. Go to some line that ends with a type annotation
> > 5. Press C-e to go to the end of the line
> > 6. Navigate around with C-n and C-p
> > 7. Note that point will jump to the column of the position of the end of
> > the type annotation, rather than the end of the code.
> >
> > For instance, if you have some code and some annotations (represented
> > with a) like so, with point at |:
> >
> > some|code aaaaaaaaaa
> > here are some more lines which do not have annotations
> >
> > Pressing C-e will move point to the end of the line
> >
> > some cod| aaaaaaaaaa
> > here are some more lines which do not have annotations
> >
> > Then when you press C-n, instead of point moving to the same column in
> > the next line, like so:
> >
> > some code aaaaaaaaaa
> > here are|some more lines which do not have annotations
> >
> > It will instead jump to the column of the end of the annotation, like
> > so:
> >
> > some code aaaaaaaaaa
> > here are some more |ines which do not have annotations

some code aaaaaaaaaa
> > here are|some more lines which do not have annotations


Hi Eli,

I have reproduced this.  I can even add the following curious example

  some code aaaaaaaaaa
  here are|some more lines which do not have annotations

Press C-p, get


  some cod| aaaaaaaaaa
  here are some more lines which do not have annotations

Which looks like just what you would get if you had pressed C-e
in the first line.  Now type C-n.

  some code aaaaaaaaaa
  here are|some more lines which do not have annotations

Which is exactly what you expect.  So, the manner in which
one arrives at the end of the line matters.

> João, how are those annotations shown?

With 'before-string' and 'after-string' properties, as you
guessed.

For example, here's a snippet of awkward but valid C++ code, where I
include the annotation:

auto bla: int
    = 42; heyheyheyheyhey


The ": int" 5-character string is the value of the before-string
property of the 1-char-long overlay that spans the newline character
separating the two lines.

> I cannot run Eglot on my
> system to try this recipe, but if I just put an overlay with a before-
> or after-string at the end of a line, C-e moves to after the overlay
> string, not before it.  Does Eglot do anything to change that?

Yes, it sets the property "cursor" to 1 in the first character of the
5-character string. This is intentionally done so that, among other reasons,
if you C-e to the end, you can edit the the 'bla' identifier without the
awkwardness of typing characters after the annotation and seeing them
pop into place after some time.

> cursor movement in the presence of overlay strings, and can only be
> changed if Eglot had its own commands for C-n/C-p, which set
> goal-column or something before moving.

Eglot does not change these commands.

João

bug#67470: 29.1; move-end-of-line behaves badly with eglot type annotations

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Mon, 27 Nov 2023 13:02:36 +0000
> Cc: matthewktromp@gmail.com, 67470@debbugs.gnu.org
> 
> I have reproduced this.  I can even add the following curious example
> 
>   some code aaaaaaaaaa
>   here are|some more lines which do not have annotations
> 
> Press C-p, get
> 
> 
>   some cod| aaaaaaaaaa
>   here are some more lines which do not have annotations
> 
> Which looks like just what you would get if you had pressed C-e
> in the first line.  Now type C-n.
> 
>   some code aaaaaaaaaa
>   here are|some more lines which do not have annotations
> 
> Which is exactly what you expect.  So, the manner in which
> one arrives at the end of the line matters.
> 
> > João, how are those annotations shown?
> 
> With 'before-string' and 'after-string' properties, as you
> guessed.

I'd appreciate an example I could play with that doesn't need Eglot.
I'm not sure we can fix this, but I'd like to understand better what
gets in the way before I decide.

Thanks.

bug#67470: 29.1; move-end-of-line behaves badly with eglot type annotations

[João Távora]

On Mon, Nov 27, 2023 at 1:18 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: João Távora <joaotavora@gmail.com>
> > Date: Mon, 27 Nov 2023 13:02:36 +0000
> > Cc: matthewktromp@gmail.com, 67470@debbugs.gnu.org
> >
> > I have reproduced this.  I can even add the following curious example
> >
> >   some code aaaaaaaaaa
> >   here are|some more lines which do not have annotations
> >
> > Press C-p, get
> >
> >
> >   some cod| aaaaaaaaaa
> >   here are some more lines which do not have annotations
> >
> > Which looks like just what you would get if you had pressed C-e
> > in the first line.  Now type C-n.
> >
> >   some code aaaaaaaaaa
> >   here are|some more lines which do not have annotations
> >
> > Which is exactly what you expect.  So, the manner in which
> > one arrives at the end of the line matters.
> >
> > > João, how are those annotations shown?
> >
> > With 'before-string' and 'after-string' properties, as you
> > guessed.
>
> I'd appreciate an example I could play with that doesn't need Eglot.
> I'm not sure we can fix this, but I'd like to understand better what
> gets in the way before I decide.
>
> Thanks.

You can try in the scratch buffer with your own overlay at the
end of the line with a before-string property that has the 'cursor'
property set to 1 at the first character.

João

bug#67470: 29.1; move-end-of-line behaves badly with eglot type annotations

[João Távora]

On Mon, Nov 27, 2023 at 1:38 PM João Távora <joaotavora@gmail.com> wrote:

> You can try in the scratch buffer with your own overlay at the
> end of the line with a before-string property that has the 'cursor'
> property set to 1 at the first character.

If you need some Elisp code, this seems to do it in the usual
starting scratch buffer contents:

(let ((o (make-overlay (line-end-position) (1+ (line-end-position)))))
  (overlay-put o 'before-string (propertize "HEYHO" 'cursor 1)))

bug#67470: 29.1; move-end-of-line behaves badly with eglot type annotations

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Mon, 27 Nov 2023 13:42:32 +0000
> Cc: matthewktromp@gmail.com, 67470@debbugs.gnu.org
> 
> On Mon, Nov 27, 2023 at 1:38 PM João Távora <joaotavora@gmail.com> wrote:
> 
> > You can try in the scratch buffer with your own overlay at the
> > end of the line with a before-string property that has the 'cursor'
> > property set to 1 at the first character.
> 
> If you need some Elisp code, this seems to do it in the usual
> starting scratch buffer contents:
> 
> (let ((o (make-overlay (line-end-position) (1+ (line-end-position)))))
>   (overlay-put o 'before-string (propertize "HEYHO" 'cursor 1)))

Thanks, I think I see the problem.  Hmm...