From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Zack Weinberg Newsgroups: gmane.emacs.bugs Subject: bug#45054: 27.1; Can get point into the middle of a run of characters with a replacing display spec Date: Sat, 5 Dec 2020 10:54:57 -0500 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="2173"; mail-complaints-to="usenet@ciao.gmane.io" To: 45054@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Dec 05 16:56:22 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1klZv4-0000T7-Ip for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 05 Dec 2020 16:56:22 +0100 Original-Received: from localhost ([::1]:48738 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1klZv3-00024S-7W for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 05 Dec 2020 10:56:21 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:40722) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1klZuk-00022V-Ov for bug-gnu-emacs@gnu.org; Sat, 05 Dec 2020 10:56:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:36505) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1klZuk-0001fF-H8 for bug-gnu-emacs@gnu.org; Sat, 05 Dec 2020 10:56:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1klZuk-0003Dz-FF for bug-gnu-emacs@gnu.org; Sat, 05 Dec 2020 10:56:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Zack Weinberg Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 05 Dec 2020 15:56:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 45054 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.160718371512314 (code B ref -1); Sat, 05 Dec 2020 15:56:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 5 Dec 2020 15:55:15 +0000 Original-Received: from localhost ([127.0.0.1]:48051 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1klZtz-0003CX-3j for submit@debbugs.gnu.org; Sat, 05 Dec 2020 10:55:15 -0500 Original-Received: from lists.gnu.org ([209.51.188.17]:42384) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1klZtw-0003CN-P5 for submit@debbugs.gnu.org; Sat, 05 Dec 2020 10:55:13 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:40642) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1klZtw-0001kA-GH for bug-gnu-emacs@gnu.org; Sat, 05 Dec 2020 10:55:12 -0500 Original-Received: from mailbackend.panix.com ([166.84.1.89]:43159) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1klZtu-0001FV-C9 for bug-gnu-emacs@gnu.org; Sat, 05 Dec 2020 10:55:12 -0500 Original-Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by mailbackend.panix.com (Postfix) with ESMTPSA id 4CpDg136nWzVfM for ; Sat, 5 Dec 2020 10:55:09 -0500 (EST) Original-Received: by mail-ej1-f45.google.com with SMTP id x16so13150560ejj.7 for ; Sat, 05 Dec 2020 07:55:09 -0800 (PST) X-Gm-Message-State: AOAM532jiMa1WbmRXbvpLazna995FXstFKYzrH7nZY6fzNZZdD/SZY4o LOem7yJ0W3T/pltHtfVNnR9oVA0ht5TwkfNp2wk= X-Google-Smtp-Source: ABdhPJyIVMxrL917AGu4wCKXJtj7aqawrRIXTiXb9m46p66cLN4ijQdAe8LwfAGziS0RdhwXEc2N62b5FKNbctENcAM= X-Received: by 2002:a17:906:1758:: with SMTP id d24mr12454089eje.287.1607183708344; Sat, 05 Dec 2020 07:55:08 -0800 (PST) X-Gmail-Original-Message-ID: Received-SPF: pass client-ip=166.84.1.89; envelope-from=zackw@panix.com; helo=mailbackend.panix.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:195036 Archived-At: I'm implementing a minor mode that "folds" certain strings, causing them to be displayed as the actual single characters that they are notation for. For purposes of this bug report, I will use the running example of displaying C-style backslash escapes for Unicode characters as their actual characters, e.g. \u00B6 is displayed as =C2=B6 when the mode is active. The mode does this by adding a font-lock keywords rule to the buffer, that applies display properties that make each string look like it's a single character. The documentation says that Emacs will not let point get into the middle of a run of text that's hidden this way, and that's mostly true, but I found a way to do it. Starting from =E2=80=98emacs -Q=E2=80=99: In the *scratch* buffer, evaluate the following minor mode definition; this is a cut-down version of the real mode, that only replaces \u00B6. (define-minor-mode fold-demo-mode "Folding bug demo" :init-value nil :lighter " FD" :keymap nil (make-variable-buffer-local 'font-lock-extra-managed-props) (cl-flet ((refontify () (save-excursion (goto-char (point-min)) (while (re-search-forward "\\\\u00[bB]6" nil t) (save-excursion (font-lock-fontify-region (match-beginning 0) (match-end 0))))))) (let ((case-fold-search nil) (demo-fl-rule '(("\\\\u00[bB]6" 0 '(face font-lock-string-face display "=C2=B6"))))) (if fold-demo-mode (progn (push 'display font-lock-extra-managed-props) (font-lock-add-keywords nil demo-fl-rule) (refontify)) (font-lock-remove-keywords nil demo-fl-rule) (refontify) (setq font-lock-extra-managed-props (cl-remove 'display font-lock-extra-managed-props :count 1)))))) Then, in any convenient buffer, on an initially blank line, type the following. I=E2=80=99ve written the recipe out as it might appear in a keyboard-macro editing buffer, but instead of command names, the annotations show what you see at each stage. The *visible cursor* is indicated with a |; all other characters are real. \ ;; \| u00B6 ;; \u00B6| C-a ;; |\u00B6 M-x fold-demo-mode ;; |=C2=B6 C-d ;; |u00B6 \ ;; =C2=B6| x ;; \x|u00B6 The bug happens when you type \ for the second time. The font-lock rule matches again, =E2=80=9C\u00B6=E2=80=9D is converted to =E2=80=9C=C2=B6=E2= =80=9D, and the *visible cursor* is displaced to after the =E2=80=9C=C2=B6=E2=80=9D, but *point* is still where= it was, between the invisible \ and u characters. Any self-inserting character, like the =E2=80=98x=E2=80=99 shown in the recipe, will be inserted at the actual= location of point, instead of the position of the visible cursor. In the demo, this causes the font-lock rule to stop matching again. Any cursor-motion command while Emacs is in this state, will put point back in sync with the visible cursor; this can lead to odd but harmless phenomena, for instance a single C-f doesn=E2=80=99t appear to do anything, instead of moving the cursor to the next line like it normally would. (Because point moves within the hidden run of characters, and then gets displaced to the end of the run, where the visible cursor already is.) Questions: 1. Is this indeed a bug in Emacs? 2. Assuming it is, can you suggest a workaround? I=E2=80=99m hoping for a viable solution within my code that=E2=80=99s compatible at least as far= back as Emacs 24. 3. I think it would be better UI if the C-d in the recipe deleted the entire run of text that=E2=80=99s getting hidden (more generally, deleti= on commands should behave as if the =C2=B6 is really what=E2=80=99s in the = buffer). I can=E2=80=99t figure out how to implement that, can you suggest anythi= ng? 4. Is there a better way to do this sort of visual replacement of runs of text? This is the first time I=E2=80=99ve done any serious elisp programming, I might have missed something. Thanks for your attention, zw In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.23, cairo version 1.16.0) of 2020-11-07, modified by Debian built on x86-ubc-01 Windowing system distributor 'The X.Org Foundation', version 11.0.12009000 System Description: Debian GNU/Linux bullseye/sid Configured using: 'configure --build x86_64-linux-gnu --prefix=3D/usr --sharedstatedir=3D/var/lib --libexecdir=3D/usr/lib --localstatedir=3D/var/lib --infodir=3D/usr/share/info --mandir=3D/usr/share/man --enable-libsystemd --with-pop=3Dyes --enable-locallisppath=3D/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:= /usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share= /emacs/site-lisp --with-sound=3Dalsa --without-gconf --with-mailutils --build x86_64-linux-gnu --prefix=3D/usr --sharedstatedir=3D/var/lib --libexecdir=3D/usr/lib --localstatedir=3D/var/lib --infodir=3D/usr/share/info --mandir=3D/usr/share/man --enable-libsystemd --with-pop=3Dyes --enable-locallisppath=3D/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:= /usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share= /emacs/site-lisp --with-sound=3Dalsa --without-gconf --with-mailutils --with-cairo --with-x=3Dyes --with-x-toolkit=3Dgtk3 --with-toolkit-scroll-bars 'CFLAGS=3D-g -O2 -fdebug-prefix-map=3D/build/emacs-6jKC2B/emacs-27.1+1=3D. -fstack-protector-strong -Wformat -Werror=3Dformat-security -Wall' 'CPPFLAGS=3D-Wdate-time -D_FORTIFY_SOURCE=3D2' LDFLAGS=3D-Wl,-z,relro' Configured features: XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER LCMS2 GMP Important settings: value of $LANG: en_US.UTF-8 value of $XMODIFIERS: @im=3Dibus locale-coding-system: utf-8-unix Features: (shadow sort mail-extr emacsbug message rmc puny rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail rmail-loaddefs text-property-search mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils help-mode server visual-regexp-steroids visual-regexp tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat parse-time iso8601 time-date ls-lisp dired dired-loaddefs dbus xml solarized-light-theme solarized color dash typo paren grep cus-start cus-load erlang-start ess-site ess-toolbar ess-mouse mouseme ess-swv ess-noweb ess-noweb-font-lock-mode ess-jags-d ess-bugs-l essd-els ess-xls-d ess-vst-d ess-stata-mode ess-stata-lang cc-vars cc-defs make-regexp ess-sp6w-d ess-sp5-d ess-sp4-d ess-sas-d ess-sas-l ess-sas-a ess-s4-d ess-s3-d ess-omg-d ess-omg-l ess-arc-d ess-lsp-l ess-sp6-d ess-dde ess-sp3-d ess-julia julia-mode cl ess-r-mode ess-r-flymake flymake-proc flymake warnings thingatpt ess-r-xref xref project ess-trns ess-r-package shell pcomplete ess-r-syntax ess-r-completion ess-roxy ess-rd essddr noutline outline easy-mmode hideshow ess-s-lang ess-help info ess-mode ess ess-noweb-mode ess-inf ess-tracebug advice format-spec ess-generics compile ess-utils ido ess-custom executable comint ansi-color ring go-mode-autoloads package easymenu browse-url url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify lcms2 dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)