bug#13863: 24.2; Cursor position after after-string on overlay at point-max

bug#13863: 24.2; Cursor position after after-string on overlay at point-max

[Magnar Sveen]

- Write some text into a buffer
- Create this overlay:

    (let ((o (make-overlay (point-max) (point-max) nil t t))
          (s "There's\na multiline\nstring here"))
      (set-text-properties 0 (length s)
       `(face (:foreground ,(format "#%02x%02x%02x" 90 90 90))) s)
      (overlay-put o 'priority 100)
      (overlay-put o 'after-string s))

    ;; the text-properties aren't important, it's just there to
    ;; separate the overlay visually.
      
- Go to point-max.
- Type something.

Notice that the point of insertion is not reflected by the position of
the cursor.

Basically, the cursor is positioned after the after-string of an overlay
at point-max, even if it does not accurately reflect where text is
inserted. If the after-string is large enough, you could be inserting
text outside the screen and not know about it.

I am trying to use this so that I can display some extra information
after contents of the buffer. However, it gets mighty annoying when the
cursor is placed after all the second-order information I am displaying
there.

Maybe the placement of the cursor should depend on the the value of
rear-advance?

- Magnar


In GNU Emacs 24.2.1 (x86_64-apple-darwin10.8.0, NS apple-appkit-1038.36)
 of 2012-09-04 on fi-bp-202.local
Windowing system distributor `Apple', version 10.3.1038
Configured using:
 `configure '--prefix=/usr/local/Cellar/emacs/24.2' '--without-dbus'
 '--enable-locallisppath=/usr/local/share/emacs/site-lisp'
 '--infodir=/usr/local/Cellar/emacs/24.2/share/info/emacs' '--with-ns'
 '--disable-ns-self-contained' 'CC=/usr/bin/gcc-4.2' 'CFLAGS=-Os -w
 -pipe -march=core2 -msse4 -mmacosx-version-min=10.6'
 'LDFLAGS=-L/usr/local/lib''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8
  default enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  diff-auto-refine-mode: t
  shell-dirtrack-mode: t
  elisp-slime-nav-mode: t
  show-paren-mode: t
  global-hl-line-mode: t
  ido-ubiquitous-mode: t
  eproject-mode: t
  eldoc-mode: t
  paredit-mode: t
  persp-mode: t
  yas/global-mode: t
  yas/minor-mode: t
  global-undo-tree-mode: t
  undo-tree-mode: t
  global-subword-mode: t
  subword-mode: t
  recentf-mode: t
  delete-selection-mode: t
  global-auto-revert-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
s SPC <backspace> s-/ n a SPC m u l t i l i n e s-/ 
n s t r i n g SPC h e r e C-f C-f C-f C-j <S-left> 
C-n C-@ M-w <S-right> C-y C-p C-p C-p M-j C-n C-e C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-f C-d 
9 C-f C-f C-b C-b C-f C-f C-d 9 C-f C-f C-d 9 C-n C-p 
C-p C-n C-n C-p M-j C-n C-n C-n C-b C-b C-@ s C-p M-j 
C-e C-x 2 <S-down> C-c b H e r e ' s SPC s o m e SPC 
t e x t <S-up> C-@ M-w <S-down> M-: C-y <return> q 
<S-up> C-n C-n C-n C-n C-f C-f C-f C-f C-f C-f C-f 
C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f 
C-f C-f C-f C-f C-f C-@ C-@ C-æ o <return> C-e C-n 
C-e C-@ M-w <S-down> M-: <up> <down> C-y <return> C-p 
C-f C-f C-f C-f C-f f f f <home> C-n <S-up> C-n C-n 
C-p C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-b 
<S-right> <S-down> C-p C-f C-f C-f C-f C-f C-f C-f 
C-f a C-p C-f C-x k <return> C-c b <S-up> C-n C-n C-p 
C-p C-p C-p C-n C-e C-b C-b C-b C-b C-f C-f <backspace> 
n i l C-n C-n C-n C-n C-n C-x C-e <S-down> a s d f 
P <backspace> <S-up> f f C-_ C-p C-p C-p C-p C-_ C-_ 
C-n C-n C-n C-n C-n C-b f <backspace> C-f f C-_ C-p 
C-p C-p C-p C-p C-a C-@ M-w <S-down> M-: C-y <return> 
f f <S-up> C-@ M-w C-x k <return> <S-down> C-x k <return> 
C-x 0 M-x e m a c s - b u <return>

Recent messages:
windmove-do-window-select: No window right from selected window
line-move-visual: Beginning of buffer [2 times]
#("There's
a multiline
string here" 0 31 (face (:foreground "#5a5a5a")))
er/mark-outside-pairs
#("There's
a multiline
string here" 0 31 (face (:foreground "#5a5a5a")))
er/mark-outside-pairs

Load-path shadows:
/Users/fimasvee/.emacs.d/elpa/hardcore-mode-1.0.0/hardcore-mode hides /Users/fimasvee/.emacs.d/site-lisp/hardcore-mode/hardcore-mode
/Users/fimasvee/.emacs.d/site-lisp/fill-column-indicator/fill-column-indicator hides /Users/fimasvee/.emacs.d/site-lisp/fill-column-indicator
~/.emacs.d/users/fimasvee/init hides ~/.emacs.d/init
/Users/fimasvee/.emacs.d/elpa/magit-20130123.1617/rebase-mode hides /usr/local/share/emacs/site-lisp/rebase-mode
/Users/fimasvee/.emacs.d/site-lisp/delsel/delsel hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/delsel
~/.emacs.d/custom hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/custom
/usr/local/share/emacs/site-lisp/trampver hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/trampver
/usr/local/share/emacs/site-lisp/tramp hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp
/usr/local/share/emacs/site-lisp/tramp-uu hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp-uu
/usr/local/share/emacs/site-lisp/tramp-smb hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp-smb
/usr/local/share/emacs/site-lisp/tramp-sh hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp-sh
/usr/local/share/emacs/site-lisp/tramp-loaddefs hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp-loaddefs
/usr/local/share/emacs/site-lisp/tramp-gw hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp-gw
/usr/local/share/emacs/site-lisp/tramp-ftp hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp-ftp
/usr/local/share/emacs/site-lisp/tramp-compat hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp-compat
/usr/local/share/emacs/site-lisp/tramp-cmds hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp-cmds
/usr/local/share/emacs/site-lisp/tramp-cache hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/net/tramp-cache
/Users/fimasvee/.emacs.d/elpa/magit-20130123.1617/.dir-locals hides /usr/local/Cellar/emacs/24.2/share/emacs/24.2/lisp/gnus/.dir-locals

Features:
(shadow sort mail-extr emacsbug debug org-wl org-w3m org-vm org-rmail
org-mhe org-mew org-irc org-jsinfo org-infojs org-html org-exp ob-exp
org-exp-blocks find-func org-agenda org-info org-gnus org-docview
org-bibtex bibtex org-bbdb mule-util git-commit-mode setup-magit
magit-svn magit-bisect magit-key-mode magit diff-mode log-edit pcvs-util
add-log misearch multi-isearch cus-start cus-load ibuf-ext ibuffer
tabify network-stream starttls tls vc-git adventur-mode
alternatives-mode adv-typos adv-typo-list adv-refill refill
adv-indentation adv-keybindings adv-play adv-defuns adv-refactorings
adv-new-number adv-named-rooms adv-room-info adv-search adv-requirements
pp adv-enter adv-tab adv-escape adv-dash adv-jump-candidate
adv-syntax-highlighting adv-intelli nrepl arc-mode archive-mode
setup-clojure-mode clj-refactor clojure-mode-expansions clojure-mode
inf-lisp tramp tramp-compat tramp-loaddefs setup-shell shell-command
shell adv-utils adv-regexes regex-dsl unsafep elisp-slime-nav etags
oppdrag-mode my-defuns server setup-mu4e mu4e mu4e-speedbar speedbar
sb-image ezimage dframe mu4e-main mu4e-view epa epg epg-config
browse-url longlines mu4e-headers mu4e-compose mu4e-actions org-capture
org-mks setup-org org-mode-expansions org ob-emacs-lisp ob-tangle ob-ref
ob-lob ob-table org-footnote org-src ob-comint ob-keys ob ob-eval
org-pcomplete pcomplete org-list org-faces org-compat org-entities
org-macs noutline outline cal-menu calendar cal-loaddefs rfc2368
smtpmail auth-source gnus-util password-cache sendmail mu4e-mark
mu4e-message html2text mu4e-proc mu4e-utils doc-view jka-compr
image-mode setup-dired dired-details dired mu4e-about mu4e-vars message
format-spec rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mailabbrev
mail-utils gmm-utils mailheader mu4e-meta diminish mac my-misc
appearance zoom-frm frame-cmds frame-fns avoid paren hl-line
key-bindings windmove misc smex setup-ido ido-ubiquitous ido
browse-kill-ring fill-column-indicator multifiles change-inner
smart-forward wgrep setup-rgrep grep compile comint ansi-color eproject
esh-var esh-io esh-cmd esh-ext esh-proc esh-arg eldoc esh-groups eshell
esh-module esh-mode esh-util ace-jump-mode jump-char multiple-cursors
mc-separate-operations rectangular-region-mode mc-mark-more thingatpt
mc-cycle-cursors mc-edit-lines multiple-cursors-core rect
inline-string-rectangle mark-more-like-this mark-multiple expand-region
text-mode-expansions expand-region-custom expand-region-core file-defuns
imenu mode-mappings setup-paredit s dash regexp-opt paredit
setup-html-mode setup-ffip find-file-in-project setup-perspective
perspective setup-yasnippet buster-snippets buster-snippet-helpers
yasnippet dropdown-list derived help-mode view assoc setup-hippie
exec-path-from-shell sane-defaults uniquify undo-tree edmacro kmacro
diff cl smooth-scrolling warnings advice help-fns advice-preload subword
winner ring recentf tree-widget wid-edit easymenu delsel easy-mmode
autorevert setup-package elisp-slime-nav-autoloads es-lib-autoloads
exec-path-from-shell-autoloads gist-autoloads gh-autoloads eieio
byte-opt bytecomp byte-compile cconv macroexp git-commit-mode-autoloads
gitconfig-mode-autoloads gitignore-mode-autoloads
goto-last-change-autoloads hardcore-mode-autoloads htmlize-autoloads
logito-autoloads magit-autoloads move-text-autoloads nrepl-autoloads
clojure-mode-autoloads paredit-autoloads pcache-autoloads finder-inf
popwin-autoloads slime-js-autoloads slime-repl-autoloads slime-autoloads
package tabulated-list saveplace time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel ns-win tool-bar dnd fontset image fringe
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process ns multi-tty
emacs)

bug#13863: 24.2; Cursor position after after-string on overlay at point-max

[Eli Zaretskii]

> From: Magnar Sveen <magnars@gmail.com>
> Date: Sun, 03 Mar 2013 20:19:33 +0100
> 
> - Write some text into a buffer
> - Create this overlay:
> 
>     (let ((o (make-overlay (point-max) (point-max) nil t t))
>           (s "There's\na multiline\nstring here"))
>       (set-text-properties 0 (length s)
>        `(face (:foreground ,(format "#%02x%02x%02x" 90 90 90))) s)
>       (overlay-put o 'priority 100)
>       (overlay-put o 'after-string s))
> 
>     ;; the text-properties aren't important, it's just there to
>     ;; separate the overlay visually.
>       
> - Go to point-max.
> - Type something.
> 
> Notice that the point of insertion is not reflected by the position of
> the cursor.
> 
> Basically, the cursor is positioned after the after-string of an overlay
> at point-max, even if it does not accurately reflect where text is
> inserted. If the after-string is large enough, you could be inserting
> text outside the screen and not know about it.

Emacs always behaved like that with after-string display.

> I am trying to use this so that I can display some extra information
> after contents of the buffer. However, it gets mighty annoying when the
> cursor is placed after all the second-order information I am displaying
> there.
> 
> Maybe the placement of the cursor should depend on the the value of
> rear-advance?

Try this (note the marked line I added):

    (let ((o (make-overlay (point-max) (point-max) nil t t))
          (s "There's\na multiline\nstring here"))
      (set-text-properties 0 (length s)
       `(face (:foreground ,(format "#%02x%02x%02x" 90 90 90))) s)
      (put-text-property 0 1 'cursor t s)  ;;; <<<<<<<<<<<<<<<<<<<<<<
      (overlay-put o 'priority 100)
      (overlay-put o 'after-string s))

IOW, when Emacs is too dumb to put the cursor where you want, you can
guide it with the 'cursor' text property.

bug#13863: 24.2; Cursor position after after-string on overlay at point-max

[Magnar Sveen]

Eli Zaretskii writes:

> IOW, when Emacs is too dumb to put the cursor where you want, you can
> guide it with the 'cursor' text property.

Thanks Eli, this solved my problem.

> Emacs always behaved like that with after-string display.

Normally I'd say that something always having been like that, is not a
good argument against fixing unintuitive and buggy behavior. However, in
the case of Emacs and its exceedingly large code base and dependents, I
see greater value in backwards compatibility.

- Magnar

bug#13863: 24.2; Cursor position after after-string on overlay at point-max

[Eli Zaretskii]

> From: Magnar Sveen <magnars@gmail.com>
> Cc: 13863@debbugs.gnu.org
> Date: Mon, 04 Mar 2013 20:31:30 +0100
> 
> > IOW, when Emacs is too dumb to put the cursor where you want, you can
> > guide it with the 'cursor' text property.
> 
> Thanks Eli, this solved my problem.
> 
> > Emacs always behaved like that with after-string display.
> 
> Normally I'd say that something always having been like that, is not a
> good argument against fixing unintuitive and buggy behavior. However, in
> the case of Emacs and its exceedingly large code base and dependents, I
> see greater value in backwards compatibility.

I meant that after-string display was explicitly coded this way since
day one.  IOW, it's not a regression.

I will close the bug, then.

bug#13863: 24.2; Cursor position after after-string on overlay at point-max

[Stefan Monnier]

> Normally I'd say that something always having been like that, is not a
> good argument against fixing unintuitive and buggy behavior.  However, in
> the case of Emacs and its exceedingly large code base and dependents, I
> see greater value in backwards compatibility.

Actually, I don't think backward compatibility is a big issue in this
case.  The more real problem is that it's difficult to fix: IIRC last
time I looked at this code, at the point where the cursor is displayed,
we don't have easy access to the needed info (we have access to the
sequence of displayed chars, and their provenance, but their provenance
only says "this (after) string" without any reference to the
corresponding overlay).


        Stefan

bug#13863: 24.2; Cursor position after after-string on overlay at point-max

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: "Eli Zaretskii" <eliz@gnu.org>,  13863@debbugs.gnu.org
> Date: Mon, 04 Mar 2013 16:43:07 -0500
> 
> Actually, I don't think backward compatibility is a big issue in this
> case.  The more real problem is that it's difficult to fix: IIRC last
> time I looked at this code, at the point where the cursor is displayed,
> we don't have easy access to the needed info (we have access to the
> sequence of displayed chars, and their provenance, but their provenance
> only says "this (after) string" without any reference to the
> corresponding overlay).

That is correct.