According to buffer-chars-modified-tick docstring: "By comparing the values returned by two individual calls of buffer-chars-modified-tick, you can tell whether a character change occurred in that buffer in between these calls" However, the return value can change even when no visible change is made to buffer text. Steps to reproduce: 1. emacs -Q 2. Evaluate the following code: (defun print-tick-before () (when (eq this-command 'self-insert-command) (warn "Tick before: %S" (buffer-chars-modified-tick)))) (defun print-tick-after () (when (eq this-command 'self-insert-command) (warn "Tick after: %S" (buffer-chars-modified-tick)))) (add-hook 'pre-command-hook #'print-tick-before) (add-hook 'post-command-hook #'print-tick-after) 3. Insert a latin symbol ?a twice. The warning buffer will print something like Warning (emacs): Tick before: 1698 Disable showing Disable logging Warning (emacs): Tick after: 1699 Disable showing Disable logging Warning (emacs): Tick before: 1699 Disable showing Disable logging Warning (emacs): Tick after: 1702 Disable showing Disable logging Note that second and third line show the same buffer-chars-modified-tick value. 4. Change input method (C-\) to russian-computer or i.e. arabic 5. Insert a non-latin symbol ?ф twice. The warning buffer will print something like Warning (emacs): Tick before: 1706 Disable showing Disable logging Warning (emacs): Tick after: 1707 Disable showing Disable logging Warning (emacs): Tick before: 1711 Disable showing Disable logging Warning (emacs): Tick after: 1712 Disable showing Disable logging Note that second and third line _do not_ show the same buffer-chars-modified-tick value even though buffer text has not been changed between the two self-insert commands Expected behaviour: return value of buffer-chars-modified-tick does not change when no changes in buffer text are made. This issue causes breakage in latest version of Org. See https://list.orgmode.org/87sfw2luhj.fsf@localhost/T/#you Best, Ihor In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, cairo version 1.16.0) of 2021-10-30 built on localhost Repository revision: c3499b8ddc357544a58917bfd3846f88caf5d97c Repository branch: master Windowing system distributor 'The X.Org Foundation', version 11.0.12013000 System Description: Gentoo/Linux Configured using: 'configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --datarootdir=/usr/share --disable-silent-rules --docdir=/usr/share/doc/emacs-29.0.9999 --htmldir=/usr/share/doc/emacs-29.0.9999/html --libdir=/usr/lib64 --program-suffix=-emacs-29-vcs --includedir=/usr/include/emacs-29-vcs --infodir=/usr/share/info/emacs-29-vcs --localstatedir=/var --enable-locallisppath=/etc/emacs:/usr/share/emacs/site-lisp --without-compress-install --without-hesiod --without-pop --with-file-notification=inotify --with-pdumper --enable-acl --with-dbus --with-modules --without-gameuser --with-libgmp --without-gpm --with-native-compilation --with-json --without-kerberos --without-kerberos5 --without-lcms2 --with-xml2 --without-mailutils --with-selinux --with-gnutls --without-libsystemd --with-threads --with-wide-int --with-zlib --with-sound=oss --with-x --without-ns --without-gconf --without-gsettings --without-toolkit-scroll-bars --with-gif --with-jpeg --with-png --with-rsvg --with-tiff --with-xpm --with-imagemagick --with-xft --with-cairo --with-harfbuzz --without-libotf --without-m17n-flt --with-x-toolkit=no --with-dumping=pdumper 'CFLAGS=-march=native -pipe -O2' CPPFLAGS= 'LDFLAGS=-Wl,-O1 -Wl,--as-needed'' Configured features: ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS HARFBUZZ IMAGEMAGICK JPEG JSON LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY OLDXMENU PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF WEBP X11 XDBE XIM XPM ZLIB Important settings: value of $LC_COLLATE: C value of $LANG: en_US.utf8 locale-coding-system: utf-8-unix Major mode: Lisp Interaction Minor modes in effect: pdf-occur-global-minor-mode: t TeX-PDF-mode: t org-edna-mode: t eros-mode: t which-key-mode: t diredfl-global-mode: t dired-async-mode: t winner-mode: t recentf-mode: t helm-adaptive-mode: t helm-mode: t helm--remap-mouse-mode: t async-bytecomp-package-mode: t eval-sexp-fu-flash-mode: t el-patch-use-package-mode: t global-git-commit-mode: t magit-auto-revert-mode: t shell-dirtrack-mode: t unpackaged/magit-log-date-headers-mode: t hl-todo-mode: t pretty-symbols-mode: t company-mode: t persistent-scratch-autosave-mode: t savehist-mode: t boon-mode: t boon-local-mode: t global-hl-line-mode: t global-page-break-lines-mode: t page-break-lines-mode: t shackle-mode: t gcmh-mode: t override-global-mode: t straight-use-package-mode: t straight-package-neutering-mode: t global-eldoc-mode: t eldoc-mode: t show-paren-mode: t electric-indent-mode: t mouse-wheel-mode: t global-prettify-symbols-mode: t prettify-symbols-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t window-divider-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t indent-tabs-mode: t transient-mark-mode: t abbrev-mode: t Load-path shadows: /home/yantar92/.emacs.d/straight/build/helm-org/helm-org hides /home/yantar92/.emacs.d/elpa/helm-org-20210324.1927/helm-org /home/yantar92/.emacs.d/straight/build/helm-org/helm-org-autoloads hides /home/yantar92/.emacs.d/elpa/helm-org-20210324.1927/helm-org-autoloads /home/yantar92/.emacs.d/straight/build/helm/helm-x-files hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-x-files /home/yantar92/.emacs.d/straight/build/helm/helm-utils hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-utils /home/yantar92/.emacs.d/straight/build/helm/helm-types hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-types /home/yantar92/.emacs.d/straight/build/helm/helm-tags hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-tags /home/yantar92/.emacs.d/straight/build/helm/helm-sys hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-sys /home/yantar92/.emacs.d/straight/build/helm/helm-shell hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-shell /home/yantar92/.emacs.d/straight/build/helm/helm-semantic hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-semantic /home/yantar92/.emacs.d/straight/build/helm/helm-ring hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-ring /home/yantar92/.emacs.d/straight/build/helm/helm-regexp hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-regexp /home/yantar92/.emacs.d/straight/build/helm/helm-occur hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-occur /home/yantar92/.emacs.d/straight/build/helm/helm-net hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-net /home/yantar92/.emacs.d/straight/build/helm/helm-mode hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-mode /home/yantar92/.emacs.d/straight/build/helm/helm-misc hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-misc /home/yantar92/.emacs.d/straight/build/helm/helm-man hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-man /home/yantar92/.emacs.d/straight/build/helm/helm-locate hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-locate /home/yantar92/.emacs.d/straight/build/helm/helm-info hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-info /home/yantar92/.emacs.d/straight/build/helm/helm-imenu hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-imenu /home/yantar92/.emacs.d/straight/build/helm/helm-id-utils hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-id-utils /home/yantar92/.emacs.d/straight/build/helm/helm-help hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-help /home/yantar92/.emacs.d/straight/build/helm/helm-grep hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-grep /home/yantar92/.emacs.d/straight/build/helm/helm-global-bindings hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-global-bindings /home/yantar92/.emacs.d/straight/build/helm/helm-for-files hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-for-files /home/yantar92/.emacs.d/straight/build/helm/helm-font hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-font /home/yantar92/.emacs.d/straight/build/helm/helm-find hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-find /home/yantar92/.emacs.d/straight/build/helm/helm-files hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-files /home/yantar92/.emacs.d/straight/build/helm/helm-fd hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-fd /home/yantar92/.emacs.d/straight/build/helm/helm-external hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-external /home/yantar92/.emacs.d/straight/build/helm/helm-eval hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-eval /home/yantar92/.emacs.d/straight/build/helm/helm-eshell hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-eshell /home/yantar92/.emacs.d/straight/build/helm/helm-epa hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-epa /home/yantar92/.emacs.d/straight/build/helm/helm-elisp hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-elisp /home/yantar92/.emacs.d/straight/build/helm/helm-elisp-package hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-elisp-package /home/yantar92/.emacs.d/straight/build/helm/helm-easymenu hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-easymenu /home/yantar92/.emacs.d/straight/build/helm/helm-dabbrev hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-dabbrev /home/yantar92/.emacs.d/straight/build/helm/helm-config hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-config /home/yantar92/.emacs.d/straight/build/helm/helm-command hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-command /home/yantar92/.emacs.d/straight/build/helm/helm-comint hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-comint /home/yantar92/.emacs.d/straight/build/helm/helm-color hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-color /home/yantar92/.emacs.d/straight/build/helm/helm-buffers hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-buffers /home/yantar92/.emacs.d/straight/build/helm/helm-bookmark hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-bookmark /home/yantar92/.emacs.d/straight/build/helm/helm-adaptive hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-adaptive /home/yantar92/.emacs.d/straight/build/helm/helm-autoloads hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-autoloads /home/yantar92/.emacs.d/straight/build/helm/helm-pkg hides /home/yantar92/.emacs.d/elpa/helm-20210827.1619/helm-pkg /home/yantar92/.emacs.d/straight/build/helm-core/helm hides /home/yantar92/.emacs.d/elpa/helm-core-20210822.952/helm /home/yantar92/.emacs.d/straight/build/helm-core/helm-source hides /home/yantar92/.emacs.d/elpa/helm-core-20210822.952/helm-source /home/yantar92/.emacs.d/straight/build/helm-core/helm-multi-match hides /home/yantar92/.emacs.d/elpa/helm-core-20210822.952/helm-multi-match /home/yantar92/.emacs.d/straight/build/helm-core/helm-lib hides /home/yantar92/.emacs.d/elpa/helm-core-20210822.952/helm-lib /home/yantar92/.emacs.d/straight/build/helm-core/helm-core-autoloads hides /home/yantar92/.emacs.d/elpa/helm-core-20210822.952/helm-core-autoloads /home/yantar92/.emacs.d/straight/build/helm-core/helm-core-pkg hides /home/yantar92/.emacs.d/elpa/helm-core-20210822.952/helm-core-pkg /home/yantar92/.emacs.d/straight/build/async/smtpmail-async hides /home/yantar92/.emacs.d/elpa/async-20210823.528/smtpmail-async /home/yantar92/.emacs.d/straight/build/async/dired-async hides /home/yantar92/.emacs.d/elpa/async-20210823.528/dired-async /home/yantar92/.emacs.d/straight/build/async/async hides /home/yantar92/.emacs.d/elpa/async-20210823.528/async /home/yantar92/.emacs.d/straight/build/async/async-bytecomp hides /home/yantar92/.emacs.d/elpa/async-20210823.528/async-bytecomp /home/yantar92/.emacs.d/straight/build/async/async-autoloads hides /home/yantar92/.emacs.d/elpa/async-20210823.528/async-autoloads /home/yantar92/.emacs.d/straight/build/org-ql/org-ql hides /home/yantar92/.emacs.d/elpa/org-ql-20210713.233/org-ql /home/yantar92/.emacs.d/straight/build/org-ql/org-ql-view hides /home/yantar92/.emacs.d/elpa/org-ql-20210713.233/org-ql-view /home/yantar92/.emacs.d/straight/build/org-ql/org-ql-search hides /home/yantar92/.emacs.d/elpa/org-ql-20210713.233/org-ql-search /home/yantar92/.emacs.d/straight/build/org-ql/org-ql-autoloads hides /home/yantar92/.emacs.d/elpa/org-ql-20210713.233/org-ql-autoloads /home/yantar92/.emacs.d/straight/build/f/f hides /home/yantar92/.emacs.d/elpa/f-20210624.1103/f /home/yantar92/.emacs.d/straight/build/f/f-autoloads hides /home/yantar92/.emacs.d/elpa/f-20210624.1103/f-autoloads /home/yantar92/.emacs.d/straight/build/org-super-agenda/org-super-agenda hides /home/yantar92/.emacs.d/elpa/org-super-agenda-20201211.918/org-super-agenda /home/yantar92/.emacs.d/straight/build/org-super-agenda/org-super-agenda-autoloads hides /home/yantar92/.emacs.d/elpa/org-super-agenda-20201211.918/org-super-agenda-autoloads /home/yantar92/.emacs.d/straight/build/ht/ht hides /home/yantar92/.emacs.d/elpa/ht-20210119.741/ht /home/yantar92/.emacs.d/straight/build/ht/ht-autoloads hides /home/yantar92/.emacs.d/elpa/ht-20210119.741/ht-autoloads /home/yantar92/.emacs.d/straight/build/ov/ov hides /home/yantar92/.emacs.d/elpa/ov-20200326.1042/ov /home/yantar92/.emacs.d/straight/build/ov/ov-autoloads hides /home/yantar92/.emacs.d/elpa/ov-20200326.1042/ov-autoloads /home/yantar92/.emacs.d/straight/build/peg/peg hides /home/yantar92/.emacs.d/elpa/peg-1.0/peg /home/yantar92/.emacs.d/straight/build/peg/peg-tests hides /home/yantar92/.emacs.d/elpa/peg-1.0/peg-tests /home/yantar92/.emacs.d/straight/build/peg/peg-autoloads hides /home/yantar92/.emacs.d/elpa/peg-1.0/peg-autoloads /home/yantar92/.emacs.d/straight/build/popup/popup hides /home/yantar92/.emacs.d/elpa/popup-20210625.400/popup /home/yantar92/.emacs.d/straight/build/popup/popup-autoloads hides /home/yantar92/.emacs.d/elpa/popup-20210625.400/popup-autoloads /home/yantar92/.emacs.d/straight/build/transient/transient hides /home/yantar92/.emacs.d/elpa/transient-20210819.2118/transient /home/yantar92/.emacs.d/straight/build/transient/transient-autoloads hides /home/yantar92/.emacs.d/elpa/transient-20210819.2118/transient-autoloads /home/yantar92/.emacs.d/straight/build/ts/ts hides /home/yantar92/.emacs.d/elpa/ts-20210813.1617/ts /home/yantar92/.emacs.d/straight/build/ts/ts-autoloads hides /home/yantar92/.emacs.d/elpa/ts-20210813.1617/ts-autoloads /home/yantar92/.emacs.d/straight/build/s/s hides /home/yantar92/.emacs.d/elpa/s-20210616.619/s /home/yantar92/.emacs.d/straight/build/s/s-autoloads hides /home/yantar92/.emacs.d/elpa/s-20210616.619/s-autoloads /home/yantar92/.emacs.d/straight/build/dash/dash hides /home/yantar92/.emacs.d/elpa/dash-20210826.1149/dash /home/yantar92/.emacs.d/straight/build/dash/dash-autoloads hides /home/yantar92/.emacs.d/elpa/dash-20210826.1149/dash-autoloads /usr/share/emacs/site-lisp/cmake-mode hides /usr/share/emacs/site-lisp/cmake/cmake-mode /home/yantar92/.emacs.d/straight/build/dash/dash hides /usr/share/emacs/site-lisp/dash/dash /usr/share/emacs/site-lisp/desktop-entry-mode hides /usr/share/emacs/site-lisp/desktop-file-utils/desktop-entry-mode /home/yantar92/.emacs.d/straight/build/f/f hides /usr/share/emacs/site-lisp/f/f /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-lib hides /usr/share/emacs/site-lisp/notmuch/notmuch-lib /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-compat hides /usr/share/emacs/site-lisp/notmuch/notmuch-compat /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-parser hides /usr/share/emacs/site-lisp/notmuch/notmuch-parser /home/yantar92/.emacs.d/straight/build/notmuch/notmuch hides /usr/share/emacs/site-lisp/notmuch/notmuch /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-query hides /usr/share/emacs/site-lisp/notmuch/notmuch-query /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-show hides /usr/share/emacs/site-lisp/notmuch/notmuch-show /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-tree hides /usr/share/emacs/site-lisp/notmuch/notmuch-tree /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-wash hides /usr/share/emacs/site-lisp/notmuch/notmuch-wash /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-hello hides /usr/share/emacs/site-lisp/notmuch/notmuch-hello /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-mua hides /usr/share/emacs/site-lisp/notmuch/notmuch-mua /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-address hides /usr/share/emacs/site-lisp/notmuch/notmuch-address /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-maildir-fcc hides /usr/share/emacs/site-lisp/notmuch/notmuch-maildir-fcc /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-message hides /usr/share/emacs/site-lisp/notmuch/notmuch-message /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-crypto hides /usr/share/emacs/site-lisp/notmuch/notmuch-crypto /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-tag hides /usr/share/emacs/site-lisp/notmuch/notmuch-tag /home/yantar92/.emacs.d/straight/build/notmuch/coolj hides /usr/share/emacs/site-lisp/notmuch/coolj /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-print hides /usr/share/emacs/site-lisp/notmuch/notmuch-print /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-jump hides /usr/share/emacs/site-lisp/notmuch/notmuch-jump /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-company hides /usr/share/emacs/site-lisp/notmuch/notmuch-company /home/yantar92/.emacs.d/straight/build/notmuch/notmuch-draft hides /usr/share/emacs/site-lisp/notmuch/notmuch-draft /home/yantar92/.emacs.d/straight/build/s/s hides /usr/share/emacs/site-lisp/s/s /home/yantar92/.emacs.d/straight/build/with-editor/with-editor hides /usr/share/emacs/site-lisp/with-editor/with-editor /home/yantar92/.emacs.d/straight/build/transient/transient hides /usr/share/emacs/29.0.50/lisp/transient /home/yantar92/.emacs.d/straight/build/org/ob-C hides /usr/share/emacs/29.0.50/lisp/org/ob-C /home/yantar92/.emacs.d/straight/build/org/ob-R hides /usr/share/emacs/29.0.50/lisp/org/ob-R /home/yantar92/.emacs.d/straight/build/org/ob-awk hides /usr/share/emacs/29.0.50/lisp/org/ob-awk /home/yantar92/.emacs.d/straight/build/org/ob-calc hides /usr/share/emacs/29.0.50/lisp/org/ob-calc /home/yantar92/.emacs.d/straight/build/org/ob-clojure hides /usr/share/emacs/29.0.50/lisp/org/ob-clojure /home/yantar92/.emacs.d/straight/build/org/ob-comint hides /usr/share/emacs/29.0.50/lisp/org/ob-comint /home/yantar92/.emacs.d/straight/build/org/ob-core hides /usr/share/emacs/29.0.50/lisp/org/ob-core /home/yantar92/.emacs.d/straight/build/org/ob-css hides /usr/share/emacs/29.0.50/lisp/org/ob-css /home/yantar92/.emacs.d/straight/build/org/ob-ditaa hides /usr/share/emacs/29.0.50/lisp/org/ob-ditaa /home/yantar92/.emacs.d/straight/build/org/ob-dot hides /usr/share/emacs/29.0.50/lisp/org/ob-dot /home/yantar92/.emacs.d/straight/build/org/ob-emacs-lisp hides /usr/share/emacs/29.0.50/lisp/org/ob-emacs-lisp /home/yantar92/.emacs.d/straight/build/org/ob-eshell hides /usr/share/emacs/29.0.50/lisp/org/ob-eshell /home/yantar92/.emacs.d/straight/build/org/ob-eval hides /usr/share/emacs/29.0.50/lisp/org/ob-eval /home/yantar92/.emacs.d/straight/build/org/ob-exp hides /usr/share/emacs/29.0.50/lisp/org/ob-exp /home/yantar92/.emacs.d/straight/build/org/ob-forth hides /usr/share/emacs/29.0.50/lisp/org/ob-forth /home/yantar92/.emacs.d/straight/build/org/ob-fortran hides /usr/share/emacs/29.0.50/lisp/org/ob-fortran /home/yantar92/.emacs.d/straight/build/org/ob-gnuplot hides /usr/share/emacs/29.0.50/lisp/org/ob-gnuplot /home/yantar92/.emacs.d/straight/build/org/ob-groovy hides /usr/share/emacs/29.0.50/lisp/org/ob-groovy /home/yantar92/.emacs.d/straight/build/org/ob-haskell hides /usr/share/emacs/29.0.50/lisp/org/ob-haskell /home/yantar92/.emacs.d/straight/build/org/ob-java hides /usr/share/emacs/29.0.50/lisp/org/ob-java /home/yantar92/.emacs.d/straight/build/org/ob-js hides /usr/share/emacs/29.0.50/lisp/org/ob-js /home/yantar92/.emacs.d/straight/build/org/ob-julia hides /usr/share/emacs/29.0.50/lisp/org/ob-julia /home/yantar92/.emacs.d/straight/build/org/ob-latex hides /usr/share/emacs/29.0.50/lisp/org/ob-latex /home/yantar92/.emacs.d/straight/build/org/ob-lilypond hides /usr/share/emacs/29.0.50/lisp/org/ob-lilypond /home/yantar92/.emacs.d/straight/build/org/ob-lisp hides /usr/share/emacs/29.0.50/lisp/org/ob-lisp /home/yantar92/.emacs.d/straight/build/org/ob-lob hides /usr/share/emacs/29.0.50/lisp/org/ob-lob /home/yantar92/.emacs.d/straight/build/org/ob-lua hides /usr/share/emacs/29.0.50/lisp/org/ob-lua /home/yantar92/.emacs.d/straight/build/org/ob-makefile hides /usr/share/emacs/29.0.50/lisp/org/ob-makefile /home/yantar92/.emacs.d/straight/build/org/ob-matlab hides /usr/share/emacs/29.0.50/lisp/org/ob-matlab /home/yantar92/.emacs.d/straight/build/org/ob-maxima hides /usr/share/emacs/29.0.50/lisp/org/ob-maxima /home/yantar92/.emacs.d/straight/build/org/ob-ocaml hides /usr/share/emacs/29.0.50/lisp/org/ob-ocaml /home/yantar92/.emacs.d/straight/build/org/ob-octave hides /usr/share/emacs/29.0.50/lisp/org/ob-octave /home/yantar92/.emacs.d/straight/build/org/ob-org hides /usr/share/emacs/29.0.50/lisp/org/ob-org /home/yantar92/.emacs.d/straight/build/org/ob-perl hides /usr/share/emacs/29.0.50/lisp/org/ob-perl /home/yantar92/.emacs.d/straight/build/org/ob-plantuml hides /usr/share/emacs/29.0.50/lisp/org/ob-plantuml /home/yantar92/.emacs.d/straight/build/org/ob-processing hides /usr/share/emacs/29.0.50/lisp/org/ob-processing /home/yantar92/.emacs.d/straight/build/org/ob-python hides /usr/share/emacs/29.0.50/lisp/org/ob-python /home/yantar92/.emacs.d/straight/build/org/ob-ref hides /usr/share/emacs/29.0.50/lisp/org/ob-ref /home/yantar92/.emacs.d/straight/build/org/ob-ruby hides /usr/share/emacs/29.0.50/lisp/org/ob-ruby /home/yantar92/.emacs.d/straight/build/org/ob-sass hides /usr/share/emacs/29.0.50/lisp/org/ob-sass /home/yantar92/.emacs.d/straight/build/org/ob-scheme hides /usr/share/emacs/29.0.50/lisp/org/ob-scheme /home/yantar92/.emacs.d/straight/build/org/ob-screen hides /usr/share/emacs/29.0.50/lisp/org/ob-screen /home/yantar92/.emacs.d/straight/build/org/ob-sed hides /usr/share/emacs/29.0.50/lisp/org/ob-sed /home/yantar92/.emacs.d/straight/build/org/ob-shell hides /usr/share/emacs/29.0.50/lisp/org/ob-shell /home/yantar92/.emacs.d/straight/build/org/ob-sql hides /usr/share/emacs/29.0.50/lisp/org/ob-sql /home/yantar92/.emacs.d/straight/build/org/ob-sqlite hides /usr/share/emacs/29.0.50/lisp/org/ob-sqlite /home/yantar92/.emacs.d/straight/build/org/ob-table hides /usr/share/emacs/29.0.50/lisp/org/ob-table /home/yantar92/.emacs.d/straight/build/org/ob-tangle hides /usr/share/emacs/29.0.50/lisp/org/ob-tangle /home/yantar92/.emacs.d/straight/build/org/ob hides /usr/share/emacs/29.0.50/lisp/org/ob /home/yantar92/.emacs.d/straight/build/org/oc-basic hides /usr/share/emacs/29.0.50/lisp/org/oc-basic /home/yantar92/.emacs.d/straight/build/org/oc-biblatex hides /usr/share/emacs/29.0.50/lisp/org/oc-biblatex /home/yantar92/.emacs.d/straight/build/org/oc-csl hides /usr/share/emacs/29.0.50/lisp/org/oc-csl /home/yantar92/.emacs.d/straight/build/org/oc-natbib hides /usr/share/emacs/29.0.50/lisp/org/oc-natbib /home/yantar92/.emacs.d/straight/build/org/oc hides /usr/share/emacs/29.0.50/lisp/org/oc /home/yantar92/.emacs.d/straight/build/org/ol-bbdb hides /usr/share/emacs/29.0.50/lisp/org/ol-bbdb /home/yantar92/.emacs.d/straight/build/org/ol-bibtex hides /usr/share/emacs/29.0.50/lisp/org/ol-bibtex /home/yantar92/.emacs.d/straight/build/org/ol-docview hides /usr/share/emacs/29.0.50/lisp/org/ol-docview /home/yantar92/.emacs.d/straight/build/org/ol-doi hides /usr/share/emacs/29.0.50/lisp/org/ol-doi /home/yantar92/.emacs.d/straight/build/org/ol-eshell hides /usr/share/emacs/29.0.50/lisp/org/ol-eshell /home/yantar92/.emacs.d/straight/build/org/ol-eww hides /usr/share/emacs/29.0.50/lisp/org/ol-eww /home/yantar92/.emacs.d/straight/build/org/ol-gnus hides /usr/share/emacs/29.0.50/lisp/org/ol-gnus /home/yantar92/.emacs.d/straight/build/org/ol-info hides /usr/share/emacs/29.0.50/lisp/org/ol-info /home/yantar92/.emacs.d/straight/build/org/ol-irc hides /usr/share/emacs/29.0.50/lisp/org/ol-irc /home/yantar92/.emacs.d/straight/build/org/ol-man hides /usr/share/emacs/29.0.50/lisp/org/ol-man /home/yantar92/.emacs.d/straight/build/org/ol-mhe hides /usr/share/emacs/29.0.50/lisp/org/ol-mhe /home/yantar92/.emacs.d/straight/build/org/ol-rmail hides /usr/share/emacs/29.0.50/lisp/org/ol-rmail /home/yantar92/.emacs.d/straight/build/org/ol-w3m hides /usr/share/emacs/29.0.50/lisp/org/ol-w3m /home/yantar92/.emacs.d/straight/build/org/ol hides /usr/share/emacs/29.0.50/lisp/org/ol /home/yantar92/.emacs.d/straight/build/org/org-agenda hides /usr/share/emacs/29.0.50/lisp/org/org-agenda /home/yantar92/.emacs.d/straight/build/org/org-archive hides /usr/share/emacs/29.0.50/lisp/org/org-archive /home/yantar92/.emacs.d/straight/build/org/org-attach-git hides /usr/share/emacs/29.0.50/lisp/org/org-attach-git /home/yantar92/.emacs.d/straight/build/org/org-attach hides /usr/share/emacs/29.0.50/lisp/org/org-attach /home/yantar92/.emacs.d/straight/build/org/org-capture hides /usr/share/emacs/29.0.50/lisp/org/org-capture /home/yantar92/.emacs.d/straight/build/org/org-clock hides /usr/share/emacs/29.0.50/lisp/org/org-clock /home/yantar92/.emacs.d/straight/build/org/org-colview hides /usr/share/emacs/29.0.50/lisp/org/org-colview /home/yantar92/.emacs.d/straight/build/org/org-compat hides /usr/share/emacs/29.0.50/lisp/org/org-compat /home/yantar92/.emacs.d/straight/build/org/org-crypt hides /usr/share/emacs/29.0.50/lisp/org/org-crypt /home/yantar92/.emacs.d/straight/build/org/org-ctags hides /usr/share/emacs/29.0.50/lisp/org/org-ctags /home/yantar92/.emacs.d/straight/build/org/org-datetree hides /usr/share/emacs/29.0.50/lisp/org/org-datetree /home/yantar92/.emacs.d/straight/build/org/org-duration hides /usr/share/emacs/29.0.50/lisp/org/org-duration /home/yantar92/.emacs.d/straight/build/org/org-element hides /usr/share/emacs/29.0.50/lisp/org/org-element /home/yantar92/.emacs.d/straight/build/org/org-entities hides /usr/share/emacs/29.0.50/lisp/org/org-entities /home/yantar92/.emacs.d/straight/build/org/org-faces hides /usr/share/emacs/29.0.50/lisp/org/org-faces /home/yantar92/.emacs.d/straight/build/org/org-feed hides /usr/share/emacs/29.0.50/lisp/org/org-feed /home/yantar92/.emacs.d/straight/build/org/org-footnote hides /usr/share/emacs/29.0.50/lisp/org/org-footnote /home/yantar92/.emacs.d/straight/build/org/org-goto hides /usr/share/emacs/29.0.50/lisp/org/org-goto /home/yantar92/.emacs.d/straight/build/org/org-habit hides /usr/share/emacs/29.0.50/lisp/org/org-habit /home/yantar92/.emacs.d/straight/build/org/org-id hides /usr/share/emacs/29.0.50/lisp/org/org-id /home/yantar92/.emacs.d/straight/build/org/org-indent hides /usr/share/emacs/29.0.50/lisp/org/org-indent /home/yantar92/.emacs.d/straight/build/org/org-inlinetask hides /usr/share/emacs/29.0.50/lisp/org/org-inlinetask /home/yantar92/.emacs.d/straight/build/org/org-install hides /usr/share/emacs/29.0.50/lisp/org/org-install /home/yantar92/.emacs.d/straight/build/org/org-keys hides /usr/share/emacs/29.0.50/lisp/org/org-keys /home/yantar92/.emacs.d/straight/build/org/org-lint hides /usr/share/emacs/29.0.50/lisp/org/org-lint /home/yantar92/.emacs.d/straight/build/org/org-list hides /usr/share/emacs/29.0.50/lisp/org/org-list /home/yantar92/.emacs.d/straight/build/org/org-macro hides /usr/share/emacs/29.0.50/lisp/org/org-macro /home/yantar92/.emacs.d/straight/build/org/org-macs hides /usr/share/emacs/29.0.50/lisp/org/org-macs /home/yantar92/.emacs.d/straight/build/org/org-mobile hides /usr/share/emacs/29.0.50/lisp/org/org-mobile /home/yantar92/.emacs.d/straight/build/org/org-mouse hides /usr/share/emacs/29.0.50/lisp/org/org-mouse /home/yantar92/.emacs.d/straight/build/org/org-num hides /usr/share/emacs/29.0.50/lisp/org/org-num /home/yantar92/.emacs.d/straight/build/org/org-pcomplete hides /usr/share/emacs/29.0.50/lisp/org/org-pcomplete /home/yantar92/.emacs.d/straight/build/org/org-plot hides /usr/share/emacs/29.0.50/lisp/org/org-plot /home/yantar92/.emacs.d/straight/build/org/org-protocol hides /usr/share/emacs/29.0.50/lisp/org/org-protocol /home/yantar92/.emacs.d/straight/build/org/org-refile hides /usr/share/emacs/29.0.50/lisp/org/org-refile /home/yantar92/.emacs.d/straight/build/org/org-src hides /usr/share/emacs/29.0.50/lisp/org/org-src /home/yantar92/.emacs.d/straight/build/org/org-table hides /usr/share/emacs/29.0.50/lisp/org/org-table /home/yantar92/.emacs.d/straight/build/org/org-tempo hides /usr/share/emacs/29.0.50/lisp/org/org-tempo /home/yantar92/.emacs.d/straight/build/org/org-timer hides /usr/share/emacs/29.0.50/lisp/org/org-timer /home/yantar92/.emacs.d/straight/build/org/org-version hides /usr/share/emacs/29.0.50/lisp/org/org-version /home/yantar92/.emacs.d/straight/build/org/org hides /usr/share/emacs/29.0.50/lisp/org/org /home/yantar92/.emacs.d/straight/build/org/ox-ascii hides /usr/share/emacs/29.0.50/lisp/org/ox-ascii /home/yantar92/.emacs.d/straight/build/org/ox-beamer hides /usr/share/emacs/29.0.50/lisp/org/ox-beamer /home/yantar92/.emacs.d/straight/build/org/ox-html hides /usr/share/emacs/29.0.50/lisp/org/ox-html /home/yantar92/.emacs.d/straight/build/org/ox-icalendar hides /usr/share/emacs/29.0.50/lisp/org/ox-icalendar /home/yantar92/.emacs.d/straight/build/org/ox-koma-letter hides /usr/share/emacs/29.0.50/lisp/org/ox-koma-letter /home/yantar92/.emacs.d/straight/build/org/ox-latex hides /usr/share/emacs/29.0.50/lisp/org/ox-latex /home/yantar92/.emacs.d/straight/build/org/ox-man hides /usr/share/emacs/29.0.50/lisp/org/ox-man /home/yantar92/.emacs.d/straight/build/org/ox-md hides /usr/share/emacs/29.0.50/lisp/org/ox-md /home/yantar92/.emacs.d/straight/build/org/ox-odt hides /usr/share/emacs/29.0.50/lisp/org/ox-odt /home/yantar92/.emacs.d/straight/build/org/ox-org hides /usr/share/emacs/29.0.50/lisp/org/ox-org /home/yantar92/.emacs.d/straight/build/org/ox-publish hides /usr/share/emacs/29.0.50/lisp/org/ox-publish /home/yantar92/.emacs.d/straight/build/org/ox-texinfo hides /usr/share/emacs/29.0.50/lisp/org/ox-texinfo /home/yantar92/.emacs.d/straight/build/org/ox hides /usr/share/emacs/29.0.50/lisp/org/ox /home/yantar92/.emacs.d/straight/build/org/org-loaddefs hides /usr/share/emacs/29.0.50/lisp/org/org-loaddefs /home/yantar92/.emacs.d/straight/build/let-alist/let-alist hides /usr/share/emacs/29.0.50/lisp/emacs-lisp/let-alist /home/yantar92/.emacs.d/straight/build/map/map hides /usr/share/emacs/29.0.50/lisp/emacs-lisp/map Features: (shadow emacsbug shell-pop shortdoc cl-print pyim-dhashcache org-pdftools org-noter pdf-view-restore pdf-sync pdf-annot facemenu pdf-outline pdf-links pdf-history pdf-occur tablist tablist-filter semantic/wisent/comp semantic/wisent semantic/wisent/wisent semantic/util-modes semantic/util semantic semantic/tag semantic/lex semantic/fw mode-local cedet pdf-isearch pdf-misc pdf-tools pdf-view pdf-cache pdf-info tq pdf-util pdf-macs magit-extras helm-imenu dired-open helm-ring qrencode helm-command helm-elisp helm-eval all-the-icons-dired dired-filter dired-hide-dotfiles network-stream url-cache qp thai-util thai-word helm-font avy mule-util cal-move ledger-mode ledger-check ledger-texi ledger-test ledger-sort ledger-report ledger-reconcile ledger-occur ledger-fonts ledger-fontify ledger-state ledger-complete ledger-schedule ledger-init ledger-xact ledger-post ledger-exec ledger-navigate eshell esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module esh-groups esh-util ledger-context ledger-commodities ledger-regex ox-org tabify elfeed-link cus-edit cus-start cus-load w3m-form w3m-symbol w3m-bookmark w3m w3m-hist w3m-fb bookmark-w3m w3m-ems w3m-favicon w3m-image tab-line w3m-proc w3m-util mm-archive org-datetree org-learn latex latex-flymake flymake-proc flymake tex-ispell tex-style tex sendmail boon-moves er-basic-expansions expand-region-core expand-region-custom sort footnote mail-extr boon-main boon-arguments multiple-cursors mc-separate-operations rectangular-region-mode mc-mark-pop mc-edit-lines mc-hide-unmatched-lines-mode mc-mark-more mc-cycle-cursors multiple-cursors-core boon-regs boon-utils boon-hl misearch multi-isearch org-duration cal-iso ffap org-table-sticky-header org-appear ol-eww eww mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect gnus-search eieio-opt speedbar ezimage dframe ol-docview doc-view jka-compr ol-bbdb ol-w3m ol-doi org-link-doi tramp-archive tramp-gvfs helm-x-files org-crypt helm-notmuch helm-notmuch-autoloads ol-notmuch org-eldoc org-appear-autoloads doom-themes-ext-org doom-themes doom-themes-base doom-themes-autoloads org-table-sticky-header-autoloads posframe ob-async ob-async-autoloads ob-latex ob-dot ob-calc calc-store calc-trail ob-gnuplot ob-ditaa ob-C cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs ob-python python tramp-sh ob-perl ob-org ob-shell ob-mathematica org-tempo tempo org-archive ox-md ox-beamer ox-extra doct ya-org-capture ya-org-capture-autoloads doct-autoloads org-capture-pop-frame org-capture-pop-frame-autoloads org-protocol org-analyzer-autoloads pomidor-autoloads alert-autoloads log4e-autoloads gntp-autoloads org-clock org-autosort org-autosort-autoloads helm-org-contacts helm-org-contacts-autoloads org-contacts gnus-art mm-uu mml2015 gnus-sum gnus-group gnus-undo gnus-start gnus-dbus gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int gnus-range gnus-win gnus nnheader helm-org-ql helm-org helm-org-ql-autoloads helm-org-autoloads org-ql-search org-ql-view ov org-super-agenda org-ql peg org-ql-autoloads peg-autoloads ov-autoloads org-super-agenda-autoloads map-autoloads org-quick-peek org-quick-peek-autoloads calfw-org calfw-org-autoloads calfw holidays hol-loaddefs calfw-autoloads org-attach cdlatex texmathp cdlatex-autoloads helm-recoll eieio-compat helm-for-files helm-bookmark helm-info helm-external helm-recoll-autoloads org-capture-ref org-ref-url-utils org-ref org-ref-helm-bibtex org-ref-helm helm-bibtex helm-net helm-config org-ref-core reftex-cite reftex reftex-loaddefs reftex-vars org-ref-glossary org-ref-bibtex org-ref-citeproc key-chord doi-utils org-ref-utils org-ref-pdf ol-bibtex htmlize bibtex-completion biblio biblio-download biblio-dissemin biblio-ieee biblio-hal biblio-dblp biblio-crossref biblio-arxiv timezone biblio-doi biblio-core ido parsebib org-ref-autoloads key-chord-autoloads ivy-autoloads helm-bibtex-autoloads bibtex-completion-autoloads biblio-autoloads biblio-core-autoloads parsebib-autoloads htmlize-autoloads scimax-inkscape scimax-inkscape-autoloads org-pdftools-autoloads org-noter-autoloads org-capture org-checklist org-habit org-edna org-edna-autoloads org-inlinetask org-drill persist org-drill-autoloads persist-autoloads speed-type speed-type-autoloads notmuch-calendar-x notmuch-calendar-x-autoloads notmuch notmuch-tree notmuch-jump notmuch-hello notmuch-show notmuch-print notmuch-crypto notmuch-mua notmuch-message notmuch-draft notmuch-maildir-fcc notmuch-address notmuch-company notmuch-parser notmuch-wash coolj notmuch-query goto-addr icalendar diary-lib diary-loaddefs notmuch-tag notmuch-lib notmuch-version notmuch-compat mm-view mml-smime smime dig w3m-load w3m-autoloads notmuch-autoloads elfeed-score elfeed-score-maint elfeed-score-scoring elfeed-score-serde elfeed-score-rule-stats elfeed-org elfeed-org-autoloads quick-peek quick-peek-autoloads elfeed-show elfeed-search vc-mtn vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs vc hideshow display-fill-column-indicator eros flycheck-tip error-tip notifications dbus flycheck-tip-autoloads flycheck rainbow-delimiters highlight-numbers parent-mode easy-escape yasnippet-snippets-autoloads yasnippet-snippets yasnippet elfeed-csv elfeed elfeed-curl elfeed-log elfeed-db elfeed-lib url-queue xml-query elfeed-score-rules elfeed-score-log elfeed-score-autoloads elfeed-autoloads qrencode-el-autoloads keycast keycast-autoloads gif-screencast gif-screencast-autoloads yaml-mode yaml-mode-autoloads mingus libmpdee cl mingus-autoloads libmpdee-autoloads calctex calc-sel calc-ext calctex-autoloads calc calc-loaddefs rect calc-macs shell-pop-autoloads eterm-256color-autoloads xterm-color-autoloads vterm term ehelp vterm-module term/xterm xterm vterm-autoloads ereader xml+ view shr kinsoku svg dom picture ereader-autoloads xml+-autoloads diffpdf diffpdf-autoloads pdf-view-restore-autoloads pdf-tools-autoloads tablist-autoloads wolfram-mode wolfram-mode-autoloads ledger-mode-autoloads auctex-autoloads tex-site ebuild-mode skeleton sh-script smie executable ebuild-mode-autoloads lua-mode lua-mode-autoloads gnuplot-autoloads eros-autoloads nameless lisp-mnt nameless-autoloads paredit paredit-autoloads which-key which-key-autoloads helm-descbinds helm-descbinds-autoloads elisp-demos elisp-demos-autoloads helpful edebug info-look help-fns radix-tree elisp-refs helpful-autoloads elisp-refs-autoloads tldr tldr-autoloads macrostep macrostep-autoloads font-lock-profiler font-lock-profiler-autoloads font-lock-studio font-lock-studio-autoloads memory-usage memory-usage-autoloads bug-hunter bug-hunter-autoloads lorem-ipsum lorem-ipsum-autoloads debug backtrace yasnippet-autoloads move-text move-text-autoloads aggressive-indent aggressive-indent-autoloads visual-regexp-autoloads magit-bookmark bookmark pp helm-bm helm-bm-autoloads bm bm-autoloads helm-dash dash-docs use-package-dash-docs xml helm-dash-autoloads dash-docs-autoloads disk-usage disk-usage-autoloads dired-git-info-autoloads dired-hide-dotfiles-autoloads dired-filter-autoloads diredfl diredfl-autoloads all-the-icons-dired-autoloads dired-async dired-open-autoloads dired-avfs dired-avfs-autoloads dired-narrow-autoloads dired-hacks-utils dired-hacks-utils-autoloads dired+ image-dired image-mode exif image-file image-converter dired-x dired-aux dired+-autoloads winner windower emacs-windower-autoloads goggles pulse skip-buffers-mode recentf tree-widget wid-edit helm-icons treemacs-icons treemacs-themes treemacs-core-utils treemacs-logging treemacs-customization pfuture inline helm-adaptive helm-mode helm-files tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat ls-lisp helm-buffers helm-occur helm-tags helm-locate helm-grep helm-regexp helm-utils helm-help helm-types helm async-bytecomp helm-global-bindings helm-source helm-multi-match helm-lib async eval-sexp-fu eval-sexp-fu-autoloads goggles-autoloads easy-escape-autoloads highlight-numbers-autoloads parent-mode-autoloads rainbow-delimiters-autoloads highlight-parentheses highlight-parentheses-autoloads flycheck-autoloads pkg-info-autoloads epl-autoloads langtool compile langtool-autoloads el-patch el-patch-autoloads flyspell ispell hi-lock ediff ediff-merg ediff-mult ediff-wind ediff-diff ediff-help ediff-init ediff-util browse-at-remote vc-git vc-dispatcher f browse-at-remote-autoloads forge-list forge-commands forge-semi forge-bitbucket buck forge-gogs gogs forge-gitea gtea forge-gitlab glab forge-github ghub-graphql treepy gsexp ghub let-alist gnutls forge-notify forge-revnote forge-pullreq forge-issue forge-topic yaml parse-time bug-reference forge-post markdown-mode thingatpt forge-repo forge forge-core forge-db closql emacsql-sqlite emacsql emacsql-compiler url-http url-auth url-gw nsm magit-submodule magit-obsolete magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit magit-sequence magit-notes magit-worktree magit-tag magit-merge magit-branch magit-reset magit-files magit-refs magit-status magit magit-repos magit-apply magit-wip magit-log which-func imenu magit-diff git-commit log-edit message rmc puny dired dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068 epg-config gnus-util rmail rmail-loaddefs text-property-search mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log magit-core magit-autorevert magit-margin magit-transient magit-process with-editor shell magit-mode transient magit-git magit-section magit-utils crm forge-autoloads yaml-autoloads markdown-mode-autoloads ghub-autoloads treepy-autoloads let-alist-autoloads closql-autoloads emacsql-sqlite-autoloads emacsql-autoloads unpackaged smerge-mode diff-mode diff ox-odt rng-loc rng-uri rng-parse rng-match rng-dt rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex ox-icalendar org-agenda ox-html table ox-ascii ox-publish ox org-element org-persist xdg avl-tree ibuf-ext ibuffer ibuffer-loaddefs use-package use-package-ensure use-package-delight ts ts-autoloads unpackaged-autoloads magit-autoloads magit-section-autoloads git-commit-autoloads with-editor-autoloads transient-autoloads autorevert filenotify disp-table hl-todo pretty-symbols company-oddmuse company-keywords company-etags etags fileloop generator xref project company-gtags company-dabbrev-code company-dabbrev company-files company-clang company-capf company-cmake company-semantic company-template company-bbdb company persistent-scratch persistent-scratch-autoloads savehist backup-walker-autoloads company-autoloads helm-icons-autoloads treemacs-autoloads cfrs-autoloads posframe-autoloads pfuture-autoloads ace-window-autoloads avy-autoloads f-autoloads helm-autoloads helm-core-autoloads popup-autoloads face-remap pyim pyim-hacks pyim-probe pyim-cregexp xr pyim-process pyim-cstring pyim-autoselector pyim-punctuation pyim-outcome pyim-indicator pyim-preview pyim-magic pyim-candidates pyim-codes pyim-imobjs pyim-pinyin pyim-pymap pyim-entered pyim-dcache pyim-dict pyim-page popup pyim-scheme pyim-common pyim-autoloads xr-autoloads async-autoloads reverse-im quail reverse-im-autoloads hydra lv boon-qwerty color olivetti straight-x boon boon-keys boon-core boon-loaddefs boon-autoloads pcre2el-autoloads multiple-cursors-autoloads expand-region-autoloads meta-functions org-id org-refile meta-functions-autoloads hl-line memoize memoize-autoloads info-colors info-colors-autoloads hl-todo-autoloads latex-pretty-symbols latex-pretty-symbols-autoloads pretty-symbols-autoloads page-break-lines page-break-lines-autoloads edmacro kmacro adaptive-wrap adaptive-wrap-autoloads olivetti-autoloads shackle trace shackle-autoloads all-the-icons all-the-icons-faces data-material data-weathericons data-octicons data-fileicons data-faicons data-alltheicons all-the-icons-autoloads org ob ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint org-pcomplete pcomplete comint ansi-color ring org-list org-faces org-entities noutline outline org-version ob-emacs-lisp ob-core ob-eval org-cycle org-table oc-basic bibtex iso8601 time-date ol org-fold org-fold-core org-keys oc org-compat advice org-macs org-loaddefs format-spec find-func cal-menu calendar cal-loaddefs modus-vivendi-theme modus-operandi-theme modus-themes modus-themes-autoloads gcmh gcmh-autoloads use-package-diminish s s-autoloads ht dash ht-autoloads dash-autoloads pcase asoc asoc.el-autoloads no-littering no-littering-autoloads hydra-autoloads lv-autoloads finder-inf use-package-bind-key org-contrib-autoloads comp comp-cstr warnings rx bind-key easy-mmode diminish diminish-autoloads use-package-core use-package-autoloads bind-key-autoloads straight-autoloads cl-extra help-mode straight server site-gentoo helm-easymenu info package browse-url url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache json map url-vars seq gv subr-x byte-opt bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify dynamic-setting font-render-setting cairo x multi-tty make-network-process native-compile emacs) Memory information: ((conses 16 6986171 2237182) (symbols 48 104486 98) (strings 32 752636 136785) (string-bytes 1 25502273) (vectors 16 561998) (vector-slots 8 24848211 822896) (floats 8 65913 13311) (intervals 56 655756 11352) (buffers 992 175))
> From: Ihor Radchenko <yantar92@gmail.com>
> Date: Thu, 11 Nov 2021 21:56:46 +0800
>
> According to buffer-chars-modified-tick docstring:
> "By comparing the values returned by two individual calls of
> buffer-chars-modified-tick, you can tell whether a character change
> occurred in that buffer in between these calls"
>
> However, the return value can change even when no visible change is made
> to buffer text.
>
> Steps to reproduce:
> 1. emacs -Q
> 2. Evaluate the following code:
>
> (defun print-tick-before ()
> (when (eq this-command 'self-insert-command)
> (warn "Tick before: %S" (buffer-chars-modified-tick))))
> (defun print-tick-after ()
> (when (eq this-command 'self-insert-command)
> (warn "Tick after: %S" (buffer-chars-modified-tick))))
> (add-hook 'pre-command-hook #'print-tick-before)
> (add-hook 'post-command-hook #'print-tick-after)
>
> 3. Insert a latin symbol ?a twice. The warning buffer will print
> something like
>
> Warning (emacs): Tick before: 1698 Disable showing Disable logging
> Warning (emacs): Tick after: 1699 Disable showing Disable logging
> Warning (emacs): Tick before: 1699 Disable showing Disable logging
> Warning (emacs): Tick after: 1702 Disable showing Disable logging
>
> Note that second and third line show the same buffer-chars-modified-tick
> value.
>
> 4. Change input method (C-\) to russian-computer or i.e. arabic
> 5. Insert a non-latin symbol ?ф twice. The warning buffer will print
> something like
>
> Warning (emacs): Tick before: 1706 Disable showing Disable logging
> Warning (emacs): Tick after: 1707 Disable showing Disable logging
> Warning (emacs): Tick before: 1711 Disable showing Disable logging
> Warning (emacs): Tick after: 1712 Disable showing Disable logging
>
> Note that second and third line _do not_ show the same
> buffer-chars-modified-tick value even though buffer text has not been
> changed between the two self-insert commands
>
> Expected behaviour: return value of buffer-chars-modified-tick does not
> change when no changes in buffer text are made.
How do you know there was no changes in the buffer? You call your
function from pre/post-command-hook, but why is it guaranteed that
there was no change in the buffer between post-command-hook and the
following pre-command-hook?
Eli Zaretskii <eliz@gnu.org> writes:
> How do you know there was no changes in the buffer? You call your
> function from pre/post-command-hook, but why is it guaranteed that
> there was no change in the buffer between post-command-hook and the
> following pre-command-hook?
I also tested the bug by setting debug-on-entry for self-insert-command.
Try the following:
1. emacs -Q
2. M-\ russian-computer <RET>
3. M-\
4. M-: (buffer-chars-modified-tick) <RET>. Note the return value.
5. M-x debug-on-entry <RET> self-insert-command <RET>
6. Insert ?a
7. The debugger window appears. ?a is not yet inserted
8. In the debugger window: e M-p <RET> (call
buffer-chars-modified-tick). The return value should be the same with
4.
9. Continue execution of self-insert-command in the debugger (c). The ?a
is inserted
10. Repeat 8. The return value correctly changes. Note the return value.
11. Continue execution to exit the debugger window (c)
12. Switch to russian input method (M-\)
13. Run M-: M-p <RET>. Note the return value. It is same with 10.
14. Run M-: (buffer-hash) <RET>. "(buffer-hash)" should by yanked to
avoid triggering debugger. Note that hash value.
13. Type ?ф (?a on qwerty keyboard)
14. The debugger appears again. ?ф is _not_ yet inserted
15. Repeat 8. The return value is different from 10 even though the
buffer text is not changed (and it can be confirmed if you run
e (buffer-hash) <RET>)
Of course, there might be some kind of invisible change in buffer. I.e.
text is added and immediately deleted from the buffer without redisplay.
However, even if there is any change like that, before-change-functions
and after-change-functions are not triggered. That would be another bug
then.
Best,
Ihor
> From: Ihor Radchenko <yantar92@gmail.com> > Cc: 51766@debbugs.gnu.org > Date: Thu, 11 Nov 2021 23:50:31 +0800 > > Of course, there might be some kind of invisible change in buffer. I.e. > text is added and immediately deleted from the buffer without redisplay. That's exactly what happens: quail.el deletes the inserted character and then reinserts it (for reasons unrelated to this issue). So the count of the changes is not equal to the number of characters actually inserted. I see no problem here, since the documentation never promises that the difference between the values returned by successive calls to buffer-chars-modified-tick will be exactly equal to the number of inserted or deleted characters. So if Org relies on such an equality, it's a bug in Org (but I didn't look at the relevant Org code, and don't have a clear idea of how exactly it uses the above function for whatever it is caching). > However, even if there is any change like that, before-change-functions > and after-change-functions are not triggered. That would be another bug > then. quail.el inhibit buffer modifications in places, since otherwise you'd have too many of them. It wants to pretend that just one character was inserted.
Eli Zaretskii <eliz@gnu.org> writes: > That's exactly what happens: quail.el deletes the inserted character > and then reinserts it (for reasons unrelated to this issue). So the > count of the changes is not equal to the number of characters actually > inserted. I see no problem here, since the documentation never > promises that the difference between the values returned by successive > calls to buffer-chars-modified-tick will be exactly equal to the > number of inserted or deleted characters. I agree that Emacs does not break any promises here. Though it is unfortunate. Feel free to close this bug report. > So if Org relies on such an equality, it's a bug in Org (but I didn't > look at the relevant Org code, and don't have a clear idea of how > exactly it uses the above function for whatever it is caching). Let me explain a little (hoping that you might have some idea about alternative solutions without using buffer-chars-modified-tick). Org has a caching mechanism (org-element-cache) that keeps parsed buffer representation in memory and updates it on the fly as the buffer changes. To make the mechanism work, Org must keep track of all the changes in buffer and update the affected Org elements in memory. Naturally, this is done using before/after-change-functions. However, some third-party code carelessly uses inhibit-modification-hooks and some edits may be missed by element cache. If we just ignore the possibility of such edits, cache can be broken badly. So, there is currently a control code that detects if buffer has been changed outside the Org's change functions. The control code uses buffer-chars-modified-tick. The behaviour of quail.el makes the control code useless - buffer-chars-modified-tick can no longer be reliably used to detect unfavourable "stealthy" changes. AFAIK, the only alternative way to detect the changes is buffer-hash/secure-hash. But calculating hash is very too slow when I try to put it into before/after-change-functions. I do not know any fast (as fast as buffer-chars-modified-tick) way to detect buffer changes. > quail.el inhibit buffer modifications in places, since otherwise you'd > have too many of them. It wants to pretend that just one character > was inserted. I understand the idea behind suppressing the modification hooks by quail. Though it would be helpful if before-change-functions were called before inserting+deleting a character by quail is done. Best, Ihor
> From: Ihor Radchenko <yantar92@gmail.com> > Cc: 51766@debbugs.gnu.org > Date: Fri, 12 Nov 2021 20:06:41 +0800 > > Org has a caching mechanism (org-element-cache) that keeps parsed buffer > representation in memory and updates it on the fly as the buffer > changes. To make the mechanism work, Org must keep track of all the > changes in buffer and update the affected Org elements in memory. > Naturally, this is done using before/after-change-functions. > > However, some third-party code carelessly uses > inhibit-modification-hooks and some edits may be missed by element > cache. If we just ignore the possibility of such edits, cache can be > broken badly. So, there is currently a control code that detects if > buffer has been changed outside the Org's change functions. The control > code uses buffer-chars-modified-tick. > > The behaviour of quail.el makes the control code useless - > buffer-chars-modified-tick can no longer be reliably used to detect > unfavourable "stealthy" changes. This last part I don't think I understand: why does quail's behavior make the control code useless? The value returned by buffer-chars-modified-tick still increases in your recipe, so what exactly is the aspect of that behavior that makes the control code useless? I think some additional details here are missing from your description which could explain the issue. > > quail.el inhibit buffer modifications in places, since otherwise you'd > > have too many of them. It wants to pretend that just one character > > was inserted. > > I understand the idea behind suppressing the modification hooks by > quail. Though it would be helpful if before-change-functions were called > before inserting+deleting a character by quail is done. I don't understand this, either. Are you saying that inserting a character via an input method doesn't call buffer-modification hooks even once? If the hooks are called, then what exactly is the problem with the hooks in this scenario?
Eli Zaretskii <eliz@gnu.org> writes: > This last part I don't think I understand: why does quail's behavior > make the control code useless? The value returned by > buffer-chars-modified-tick still increases in your recipe, so what > exactly is the aspect of that behavior that makes the control code > useless? I think some additional details here are missing from your > description which could explain the issue. The control code makes sure that all the changes made in buffer are processed by org-element-cache. It means that org-element--after-change-function saves the buffer-chars-modified-tick and the next org-element--before-change-function checks if the saved value is unchanged. If the saved value is changed, the buffer has been changed after org-element--after-change-function, but before next org-element--before-change-function. Such change may be arbitrary and the whole cache is potentially obsolete. In code, the described roughly looks like: (defun org-element--after-change-function (...) (setq org-element-chars-modified-tick (buffer-chars-modified-tick)) (org-element-cache-submit-request ...)) (defun org-element--before-change-function (...) (unless (eq org-element-chars-modified-tick (buffer-chars-modified-tick)) ;; Buffer has been changed without calling after-change-function ;; and we have no way to determine which part of buffer has been changed. )) quail changes the buffer after org-element--after-change-function call, but before org-element--before-change-function. So, all Org can see is that something has been changed in buffer, but there is no way to tell what it was. Org cannot distinguish between harmless buffer edits by quail (they do not change buffer text) and other kinds of "silent" changes. > I don't understand this, either. Are you saying that inserting a > character via an input method doesn't call buffer-modification hooks > even once? If the hooks are called, then what exactly is the problem > with the hooks in this scenario? The hooks are called, but after quail already triggered buffer-chars-modified-tick increase. If quail called before-change-functions before buffer-chars-modified-tick increases, it would be useful for my scenario. Though I am not sure how feasible it is. Just an idea. Best, Ihor
> From: Ihor Radchenko <yantar92@gmail.com> > Cc: 51766@debbugs.gnu.org > Date: Fri, 12 Nov 2021 20:53:33 +0800 > > The control code makes sure that all the changes made in buffer are > processed by org-element-cache. It means that > org-element--after-change-function saves the buffer-chars-modified-tick > and the next org-element--before-change-function checks if the saved > value is unchanged. If the saved value is changed, the buffer has been > changed after org-element--after-change-function, but before next > org-element--before-change-function. Such change may be arbitrary and > the whole cache is potentially obsolete. > > In code, the described roughly looks like: > > (defun org-element--after-change-function (...) > (setq org-element-chars-modified-tick (buffer-chars-modified-tick)) > (org-element-cache-submit-request ...)) > > (defun org-element--before-change-function (...) > (unless (eq org-element-chars-modified-tick (buffer-chars-modified-tick)) > ;; Buffer has been changed without calling after-change-function > ;; and we have no way to determine which part of buffer has been changed. > )) > > quail changes the buffer after org-element--after-change-function call, > but before org-element--before-change-function. So, all Org can see is > that something has been changed in buffer, but there is no way to tell > what it was. Org cannot distinguish between harmless buffer edits by > quail (they do not change buffer text) and other kinds of "silent" > changes. OK, but why does this invalidate what Org does? All it means, AFAIU, is that in some cases Org will do unnecessary processing. Those cases are probably not too frequent. IOW, why invalidating the cache unnecessarily is such a big deal? > > I don't understand this, either. Are you saying that inserting a > > character via an input method doesn't call buffer-modification hooks > > even once? If the hooks are called, then what exactly is the problem > > with the hooks in this scenario? > > The hooks are called, but after quail already triggered > buffer-chars-modified-tick increase. If quail called > before-change-functions before buffer-chars-modified-tick increases, it > would be useful for my scenario. Though I am not sure how feasible it > is. Just an idea. Would it help if Org looked at both buffer-modified-tick and buffer-chars-modified-tick?
Eli Zaretskii <eliz@gnu.org> writes: >> quail changes the buffer after org-element--after-change-function call, >> but before org-element--before-change-function. So, all Org can see is >> that something has been changed in buffer, but there is no way to tell >> what it was. Org cannot distinguish between harmless buffer edits by >> quail (they do not change buffer text) and other kinds of "silent" >> changes. > > OK, but why does this invalidate what Org does? All it means, AFAIU, > is that in some cases Org will do unnecessary processing. Those cases > are probably not too frequent. Normally, buffer changes under inhibit-modification-hooks are not frequent indeed. But not with quail + non-latin input methods. Every single self-insert-command triggers such change. > IOW, why invalidating the cache unnecessarily is such a big deal? It is critical for Org to know which part of buffer was changed (i.e. beg, end, length that are normally passed as arguments of after-change-functions). org-element-cache can contain >100k elements for especially large buffers. Manually checking which elements are changed without knowing the changed region is inefficient. Clearing the cache is not too much of a big deal, but causes slowdown when user runs a query on Org buffers (i.e. in agenda or sparse trees) - the buffer has to be re-parsed. When every edit triggers cache invalidation (that's what happens when user uses non-latin input method), the slowdown is pretty much guaranteed. Moreover, too frequent cache resets increase the load on Emacs' garbage collector (cache size is typically a multiple of buffer size). Overloading garbage collector leads to overall Emacs slowdown. >> The hooks are called, but after quail already triggered >> buffer-chars-modified-tick increase. If quail called >> before-change-functions before buffer-chars-modified-tick increases, it >> would be useful for my scenario. Though I am not sure how feasible it >> is. Just an idea. > > Would it help if Org looked at both buffer-modified-tick and > buffer-chars-modified-tick? When buffer-chars-modified-tick is changed, buffer-modified-tick is also changed. AFAIU, buffer-chars-modified-tick registers a subset of buffer modifications that actually change buffer text. buffer-modified-tick also registers text property changes. So, I do not see how buffer-modified-tick can help. Best, Ihor
> From: Ihor Radchenko <yantar92@gmail.com> > Cc: 51766@debbugs.gnu.org > Date: Fri, 12 Nov 2021 21:39:54 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> quail changes the buffer after org-element--after-change-function call, > >> but before org-element--before-change-function. So, all Org can see is > >> that something has been changed in buffer, but there is no way to tell > >> what it was. Org cannot distinguish between harmless buffer edits by > >> quail (they do not change buffer text) and other kinds of "silent" > >> changes. > > > > OK, but why does this invalidate what Org does? All it means, AFAIU, > > is that in some cases Org will do unnecessary processing. Those cases > > are probably not too frequent. > > Normally, buffer changes under inhibit-modification-hooks are not > frequent indeed. But not with quail + non-latin input methods. Every > single self-insert-command triggers such change. "Such change" being what exactly? the situation where buffer-chars-modified-tick changes between post-command-hook and the following pre-command-hook? or something else? I'm asking because I don't think I see the problem you are describing. With the following code: (defun my-before (beg end) (message "buf %s beg %s end %s" (current-buffer) beg end)) (defun my-after (beg end len) (message "buf %s beg %s end %s len %s" (current-buffer) beg end len)) (add-hook 'before-change-functions 'my-before) (add-hook 'after-change-functions 'my-after) if I activate the chinese-py input method, then inserting any character via the input method produces a single call to my-before and a single call to my-after, and with the expected values, for example: buf *scratch* beg 440 end 440 buf *scratch* beg 440 end 441 len 0 So what exactly is the problem with these hooks when non-latin input methods are used? Or what am I missing? > > IOW, why invalidating the cache unnecessarily is such a big deal? > > It is critical for Org to know which part of buffer was changed (i.e. > beg, end, length that are normally passed as arguments of > after-change-functions). org-element-cache can contain >100k elements > for especially large buffers. Manually checking which elements are > changed without knowing the changed region is inefficient. Clearing the > cache is not too much of a big deal, but causes slowdown when user runs > a query on Org buffers (i.e. in agenda or sparse trees) - the buffer has > to be re-parsed. When every edit triggers cache invalidation (that's > what happens when user uses non-latin input method), the slowdown is > pretty much guaranteed. Moreover, too frequent cache resets increase the > load on Emacs' garbage collector (cache size is typically a multiple of > buffer size). Overloading garbage collector leads to overall Emacs > slowdown. Perhaps Org developers should ask for infrastructure changes that will allow Org to maintain such a cache reliably and not too expensively? It sounds like Org currently applies all kinds of heuristics based on assumptions about how the internals work and using hooks and features that were never designed to support this kind of caching. Jumping through hoops in Lisp trying to implement something that might be much easier or even trivial in C is not the best way of getting such jobs done. So perhaps someone could describe on emacs-devel what does Org need to maintain this cache, and we could then see how to provide those features to Org. > >> The hooks are called, but after quail already triggered > >> buffer-chars-modified-tick increase. If quail called > >> before-change-functions before buffer-chars-modified-tick increases, it > >> would be useful for my scenario. Though I am not sure how feasible it > >> is. Just an idea. > > > > Would it help if Org looked at both buffer-modified-tick and > > buffer-chars-modified-tick? > > When buffer-chars-modified-tick is changed, buffer-modified-tick is also > changed. AFAIU, buffer-chars-modified-tick registers a subset of buffer > modifications that actually change buffer text. buffer-modified-tick > also registers text property changes. So, I do not see how > buffer-modified-tick can help. If you look at the implementation, you will see that when Emacs decides that the buffer's chars-modified-tick needs to be increased, it simply assigns to it the value of the buffer's modified-tick at that moment. So by tracking the value of buffer-modified-tick you could perhaps explain why buffer-chars-modified-tick jumps by more than you expected.
Eli Zaretskii <eliz@gnu.org> writes: > "Such change" being what exactly? the situation where > buffer-chars-modified-tick changes between post-command-hook and the > following pre-command-hook? or something else? The former. > So what exactly is the problem with these hooks when non-latin input > methods are used? Or what am I missing? There is no problem with the hooks in your example. However, consider the following: (let ((inhibit-modification-hooks t)) (insert "Insertion that will never trigger before/after-change-functions")) Org cache is bound to track all the changes in buffer. Any missed change will lead to cache corruption. So, situations like the above must be tracked somehow. This tracking can be done using buffer-chars-modified-tick or buffer-hash/secure-hash. The latter is too slow. If I understand your earlier explanation correctly, quail for non-latin input (I tested with russian-computer) does something like (let ((inhibit-modification-hooks t)) (insert ?char) (backward-delete-char)) ;; This increases buffer-chars-modified-tick (insert ?translated_char_according_to_input_method) The change hooks will only be called for the last insertion. However, the first insertion+deletion will change buffer-chars-modified-tick. The quail's insertion+deletion itself is not a problem for Org cache - it does not really alter the buffer text and cannot break the cache. The problem is that it cannot be distinguished from the first example - both cases will trigger buffer-chars-modified-tick increase. > Perhaps Org developers should ask for infrastructure changes that will > allow Org to maintain such a cache reliably and not too expensively? > It sounds like Org currently applies all kinds of heuristics based on > assumptions about how the internals work and using hooks and features > that were never designed to support this kind of caching. Jumping > through hoops in Lisp trying to implement something that might be much > easier or even trivial in C is not the best way of getting such jobs > done. > > So perhaps someone could describe on emacs-devel what does Org need to > maintain this cache, and we could then see how to provide those > features to Org. I am one of the Org developers. The only assumption I had it that Emacs does not frequently change buffer text without triggering modification hooks. Clearly, the assumption was wrong. Ideally, a way to track _all_ buffer modifications regardless of inhibit-modification-hooks would be useful. Currently, Org has to work around the possibilities that text can be inserted without triggering modification hooks: (1) when inhibit-modification-hooks/before-change-functions/after-change-functions are let-bound to nil; (2) when changes are made in indirect buffer with different buffer-local values of before/after-change-functions. Alternatively, Emacs could support language parsers. Org cache implements editing syntax tree generated by Org element parser. It is very similar to what tree-sitter editing API does: https://tree-sitter.github.io/tree-sitter/using-parsers#editing Native support for storing, modifying, and querying syntax trees using efficient data structures could be a great addition to Emacs from Org's perspective. Though it is not an easy feature to implement. AFAIR, something similar to my last suggestion has been already proposed: tree-sitter support. I can also propose the first idea about reliable buffer change tracking if you think that it is something reasonable. Best, Ihor
> From: Ihor Radchenko <yantar92@gmail.com> > Cc: 51766@debbugs.gnu.org > Date: Sat, 13 Nov 2021 17:10:11 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > "Such change" being what exactly? the situation where > > buffer-chars-modified-tick changes between post-command-hook and the > > following pre-command-hook? or something else? > > The former. > > > So what exactly is the problem with these hooks when non-latin input > > methods are used? Or what am I missing? > > There is no problem with the hooks in your example. However, consider > the following: > > (let ((inhibit-modification-hooks t)) > (insert "Insertion that will never trigger before/after-change-functions")) So the problem is that Org uses the modification hooks as the primary mechanism (with which quail presents no problem), and buffer-chars-modified-tick as the fallback, and that when some code inhibits the modification hooks, then the primary mechanism cannot work and quail breaks the fallback? > The quail's insertion+deletion itself is not a problem for Org cache - > it does not really alter the buffer text and cannot break the cache. The > problem is that it cannot be distinguished from the first example - both > cases will trigger buffer-chars-modified-tick increase. You didn't answer my question regarding buffer-modified-tick: it can explain to Org why buffer-chars-modified-tick jumped unexpectedly, and thus (hopefully) resolve this situation. If that helps, you could perhaps turn the table and use buffer-chars-modified-tick is the primary method of discovering changes, not as fallback. > > Perhaps Org developers should ask for infrastructure changes that will > > allow Org to maintain such a cache reliably and not too expensively? > > It sounds like Org currently applies all kinds of heuristics based on > > assumptions about how the internals work and using hooks and features > > that were never designed to support this kind of caching. Jumping > > through hoops in Lisp trying to implement something that might be much > > easier or even trivial in C is not the best way of getting such jobs > > done. > > > > So perhaps someone could describe on emacs-devel what does Org need to > > maintain this cache, and we could then see how to provide those > > features to Org. > > I am one of the Org developers. > > The only assumption I had it that Emacs does not frequently change > buffer text without triggering modification hooks. Clearly, the > assumption was wrong. I meant the assumptions about what buffer-chars-modified-tick does and what its value means. > Ideally, a way to track _all_ buffer modifications regardless of > inhibit-modification-hooks would be useful. But Org is not interested in just any moidification, AFAIU. It is only interested in modifications that change the buffer text. Isn't that true? Or what else is Org interested in for this purpose. > Alternatively, Emacs could support language parsers. I meant support on the low level, where changes to buffer text are considered and indicated. As I indicate below, the integration of tree-sitter simply uses the existing change indications, so I'm not sure how would a parser support help you in this matter. > Org cache > implements editing syntax tree generated by Org element parser. It is > very similar to what tree-sitter editing API does: https://tree-sitter.github.io/tree-sitter/using-parsers#editing > > Native support for storing, modifying, and querying syntax trees using > efficient data structures could be a great addition to Emacs from Org's > perspective. Though it is not an easy feature to implement. > > AFAIR, something similar to my last suggestion has been already > proposed: tree-sitter support. I can also propose the first idea about > reliable buffer change tracking if you think that it is something > reasonable. The Emacs code related to tree-sitter already uses change indications, and AFAIR didn't require any changes to the existing infrastructure. I wonder why Org cannot settle with what we have, if your needs are similar enough to those of tree-sitter.
Eli Zaretskii <eliz@gnu.org> writes: >> (let ((inhibit-modification-hooks t)) >> (insert "Insertion that will never trigger before/after-change-functions")) > > So the problem is that Org uses the modification hooks as the primary > mechanism (with which quail presents no problem), and > buffer-chars-modified-tick as the fallback, and that when some code > inhibits the modification hooks, then the primary mechanism cannot > work and quail breaks the fallback? Not exactly. Org uses modification hooks as the only mechanism to process buffer changes because Org needs to know the region where the buffer text changes. buffer-chars-modified-tick is used for error detection - when buffer text is changed, but Org modification hooks are not called for some reason. quail triggers false positives during the error detection. >> The quail's insertion+deletion itself is not a problem for Org cache - >> it does not really alter the buffer text and cannot break the cache. The >> problem is that it cannot be distinguished from the first example - both >> cases will trigger buffer-chars-modified-tick increase. > > You didn't answer my question regarding buffer-modified-tick: it can > explain to Org why buffer-chars-modified-tick jumped unexpectedly, and > thus (hopefully) resolve this situation. If that helps, you could > perhaps turn the table and use buffer-chars-modified-tick is the > primary method of discovering changes, not as fallback. > ... >> The only assumption I had it that Emacs does not frequently change >> buffer text without triggering modification hooks. Clearly, the >> assumption was wrong. > > I meant the assumptions about what buffer-chars-modified-tick does and > what its value means. It seems that we have some misunderstanding here. Org does not care about the value of buffer-chars-modified-tick - just whether buffer-chars-modified-tick is changed or not (see the above). Even if Org used the value of buffer-chars-modified-tick, it would not be useful. There is no information about buffer region where the edits happened if we just look at buffer-chars-modified-tick. AFAIK, only after-change-functions have access to the changed region bounds. >> Ideally, a way to track _all_ buffer modifications regardless of >> inhibit-modification-hooks would be useful. > > But Org is not interested in just any moidification, AFAIU. It is > only interested in modifications that change the buffer text. Isn't > that true? Or what else is Org interested in for this purpose. You are right. Org is interested in modifications that change buffer text. Also, Org is interested to be not affected by inhibit-modification-hooks. Making sure that Org modification hooks run for every modification is automatically making sure that no modifications that do change buffer text are missed. I thought that it can be the simplest approach to fix the issue. >> Alternatively, Emacs could support language parsers. > > I meant support on the low level, where changes to buffer text are > considered and indicated. As I indicate below, the integration of > tree-sitter simply uses the existing change indications, so I'm not > sure how would a parser support help you in this matter. I probably went too far with my suggestion here. I meant not only handling changes, but also adding API for working with syntax trees using C-level functions. It is out of scope of this bug report. > The Emacs code related to tree-sitter already uses change indications, > and AFAIR didn't require any changes to the existing infrastructure. > I wonder why Org cannot settle with what we have, if your needs are > similar enough to those of tree-sitter. I just checked https://github.com/casouri/emacs/blob/ts/src/insdel.c That ts Emacs branch directly modifies C-level Emacs buffer editing primitives (see the calls to ts_record_change). So, it is not affected by inhibit-modification-hooks. Best, Ihor
> From: Ihor Radchenko <yantar92@gmail.com> > Cc: 51766@debbugs.gnu.org > Date: Sat, 13 Nov 2021 19:29:36 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> (let ((inhibit-modification-hooks t)) > >> (insert "Insertion that will never trigger before/after-change-functions")) > > > > So the problem is that Org uses the modification hooks as the primary > > mechanism (with which quail presents no problem), and > > buffer-chars-modified-tick as the fallback, and that when some code > > inhibits the modification hooks, then the primary mechanism cannot > > work and quail breaks the fallback? > > Not exactly. Org uses modification hooks as the only mechanism to > process buffer changes because Org needs to know the region where the > buffer text changes. buffer-chars-modified-tick is used for error > detection - when buffer text is changed, but Org modification hooks are > not called for some reason. quail triggers false positives during the error > detection. Is that any different from what I said, which is that you need error detection only when the modification hooks are not called? And that the quail behavior is only the issue when using this error detection, i.e. when the modification hooks are not called? > >> The quail's insertion+deletion itself is not a problem for Org cache - > >> it does not really alter the buffer text and cannot break the cache. The > >> problem is that it cannot be distinguished from the first example - both > >> cases will trigger buffer-chars-modified-tick increase. > > > > You didn't answer my question regarding buffer-modified-tick: it can > > explain to Org why buffer-chars-modified-tick jumped unexpectedly, and > > thus (hopefully) resolve this situation. If that helps, you could > > perhaps turn the table and use buffer-chars-modified-tick is the > > primary method of discovering changes, not as fallback. > > ... > >> The only assumption I had it that Emacs does not frequently change > >> buffer text without triggering modification hooks. Clearly, the > >> assumption was wrong. > > > > I meant the assumptions about what buffer-chars-modified-tick does and > > what its value means. > > It seems that we have some misunderstanding here. Org does not care > about the value of buffer-chars-modified-tick - just whether > buffer-chars-modified-tick is changed or not (see the above). But if buffer-modified-tick completely explains the change in buffer-chars-modified-tick, you can conclude that buffer-chars-modified-tick didn't change for your purposes, and then all's well, no? > Even if Org used the value of buffer-chars-modified-tick, it would not > be useful. There is no information about buffer region where the edits > happened if we just look at buffer-chars-modified-tick. AFAIK, only > after-change-functions have access to the changed region bounds. So what does Org do if the modification hooks were not called, and buffer-chars-modified-tick says the text was changed? > > But Org is not interested in just any moidification, AFAIU. It is > > only interested in modifications that change the buffer text. Isn't > > that true? Or what else is Org interested in for this purpose. > > You are right. Org is interested in modifications that change buffer > text. Also, Org is interested to be not affected by > inhibit-modification-hooks. Then maybe this is the missing infrastructure you'd like to see implemented. > > The Emacs code related to tree-sitter already uses change indications, > > and AFAIR didn't require any changes to the existing infrastructure. > > I wonder why Org cannot settle with what we have, if your needs are > > similar enough to those of tree-sitter. > > I just checked https://github.com/casouri/emacs/blob/ts/src/insdel.c > That ts Emacs branch directly modifies C-level Emacs buffer editing > primitives (see the calls to ts_record_change). So, it is not affected > by inhibit-modification-hooks. That's what I meant by "existing infrastructure".
Eli Zaretskii <eliz@gnu.org> writes: >> Not exactly. Org uses modification hooks as the only mechanism to >> process buffer changes because Org needs to know the region where the >> buffer text changes. buffer-chars-modified-tick is used for error >> detection - when buffer text is changed, but Org modification hooks are >> not called for some reason. quail triggers false positives during the error >> detection. > > Is that any different from what I said, which is that you need error > detection only when the modification hooks are not called? And that > the quail behavior is only the issue when using this error detection, > i.e. when the modification hooks are not called? Your understanding is correct. >> It seems that we have some misunderstanding here. Org does not care >> about the value of buffer-chars-modified-tick - just whether >> buffer-chars-modified-tick is changed or not (see the above). > > But if buffer-modified-tick completely explains the change in > buffer-chars-modified-tick, you can conclude that > buffer-chars-modified-tick didn't change for your purposes, and then > all's well, no? I looked into it again and tried to play with non-cyrillic input looking at the values of buffer-chars-modified-tick and buffer-modified-tick. You are right, there seems to be a special relation between the values when I use non-latin input method (buffer-chars-modified-tick=buffer-modified-tick). Thanks! However, I am not sure if equality of the chars-modified-tick and modified-tick is unique to non-changing edits. Can test in the wild though. > So what does Org do if the modification hooks were not called, and > buffer-chars-modified-tick says the text was changed? The cache is potentially invalid, so it is dropped altogether by org-element-cache-reset. >> > But Org is not interested in just any moidification, AFAIU. It is >> > only interested in modifications that change the buffer text. Isn't >> > that true? Or what else is Org interested in for this purpose. >> >> You are right. Org is interested in modifications that change buffer >> text. Also, Org is interested to be not affected by >> inhibit-modification-hooks. > > Then maybe this is the missing infrastructure you'd like to see > implemented. Yes, I think. In practical terms, it may something like a new pair of hooks: before/after-change-no-inhibit-functions. The hooks work exactly like before/after-change-functions, but cannot be suppressed by let-binding inhibit-modification-hooks and before/after-change-functions. If necessary they can still be explicitly let-bound to nil, but it should be discouraged. WDYT? Best, Ihor
> From: Ihor Radchenko <yantar92@gmail.com> > Cc: 51766@debbugs.gnu.org > Date: Sat, 13 Nov 2021 22:43:17 +0800 > > > But if buffer-modified-tick completely explains the change in > > buffer-chars-modified-tick, you can conclude that > > buffer-chars-modified-tick didn't change for your purposes, and then > > all's well, no? > > I looked into it again and tried to play with non-cyrillic input looking > at the values of buffer-chars-modified-tick and buffer-modified-tick. > You are right, there seems to be a special relation between the values > when I use non-latin input method > (buffer-chars-modified-tick=buffer-modified-tick). Thanks! That's what the implementation does: it copies the value from the latter to the former. > However, I am not sure if equality of the chars-modified-tick and > modified-tick is unique to non-changing edits. Can test in the wild > though. I'd be surprised if the relation were violated. But weirder things have happened, so... > > Then maybe this is the missing infrastructure you'd like to see > > implemented. > > Yes, I think. In practical terms, it may something like a new pair of > hooks: before/after-change-no-inhibit-functions. The hooks work exactly > like before/after-change-functions, but cannot be suppressed by > let-binding inhibit-modification-hooks and > before/after-change-functions. If necessary they can still be explicitly > let-bound to nil, but it should be discouraged. WDYT? Something like that, yes. Just with shorter names ;-)
> (let ((inhibit-modification-hooks t)) > (insert "Insertion that will never trigger before/after-change-functions")) This is a severely broken piece of code. I don't think anyone should try and handle this in any other way than by screaming bloody murder when it detects the consequences. > (defun org-element--after-change-function (...) > (setq org-element-chars-modified-tick (buffer-chars-modified-tick)) > (org-element-cache-submit-request ...)) > > (defun org-element--before-change-function (...) > (unless (eq org-element-chars-modified-tick (buffer-chars-modified-tick)) > ;; Buffer has been changed without calling after-change-function > ;; and we have no way to determine which part of buffer has been changed. > )) So this `unless` is intended to detect the case where we should scream bloody murder, right? Why do you need it? AFAICT it should only be needed for debugging purposes until the offender is found and shamed publicly. [ I have a weird feeling that I might be one of the offenders. ] > Ideally, a way to track _all_ buffer modifications regardless of > inhibit-modification-hooks would be useful. I don't think this *can* exist: if we add a mechanism which ignores `inhibit-modification-hooks` it will still need some way to ignore some changes so we'll need another `inhibit-<foo>` variable to "silence" those changes and we'll be back at square one. I think the better way to proceed is to figure out why/when significant changes are made while `inhibit-modification-hooks` is non-nil, since that's the origin of your problems, AFAICT. Stefan
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>, 51766@debbugs.gnu.org
> Date: Thu, 16 Jun 2022 22:54:22 -0400
>
> I think the better way to proceed is to figure out why/when
> significant changes are made while `inhibit-modification-hooks` is
> non-nil, since that's the origin of your problems, AFAICT.
I thought that was clear from the rest of the discussion: it's quail's
input methods that cause the issue, because quail tries to pretend
that just one character was inserted, when in fact the user could type
several characters.
Stefan Monnier <monnier@iro.umontreal.ca> writes: >> (let ((inhibit-modification-hooks t)) >> (insert "Insertion that will never trigger before/after-change-functions")) > > This is a severely broken piece of code. I don't think anyone should > try and handle this in any other way than by screaming bloody murder > when it detects the consequences. >> (defun org-element--before-change-function (...) >> (unless (eq org-element-chars-modified-tick (buffer-chars-modified-tick)) >> ;; Buffer has been changed without calling after-change-function >> ;; and we have no way to determine which part of buffer has been changed. >> )) > > So this `unless` is intended to detect the case where we should scream > bloody murder, right? > > Why do you need it? AFAICT it should only be needed for debugging > purposes until the offender is found and shamed publicly. > [ I have a weird feeling that I might be one of the offenders. ] > >> Ideally, a way to track _all_ buffer modifications regardless of >> inhibit-modification-hooks would be useful. Well. This was also my assumption. I initially implemented the checking code to detect internal errors in Org. Then we received 12 bug reports on this. The offenders were identified in some cases. They were: - polymode, valign, and ox-hugo - Doom (unrelated to this particular issue, but Doom is let-binding major-mode...) - Emacs quail and also replace-match in some cases (because of false-positives caused by changing buffer-chars-modified-tick) ox-hugo and polymode fixed the issue. valign relies on disabling modification hooks because it is otherwise difficult to figure out pixel width of a string in current buffer: https://github.com/casouri/valign/issues/30 In other cases, the offenders were hard to identify because of remote debugging difficulty. What I want to say is that the problem is more widespread than you may think. And the consequences of missed modifications can cause very real data corruption in Org files (some editing Org commands are relying on cache being valid; if syntax boundaries are incorrect, the editing can mess up the text). It must be avoided at all costs, regardless of the recommended Elisp programming practices. On top of the misbehaving third-party code, there is an issue described in bug#46982. This discussion reminded me about clone-indirect-buffer-hook, but it is only executed by `clone-indirect-buffer', not by `make-indirect-buffer'. The latter is used ubiquitously. See https://github.com/search?q=make-indirect-buffer&type=code Again, some unsuspecting users can experience data corruption just because someone carelessly uses `make-indirect-buffer' in the code. > I don't think this *can* exist: if we add a mechanism which ignores > `inhibit-modification-hooks` it will still need some way to ignore some > changes so we'll need another `inhibit-<foo>` variable to "silence" > those changes and we'll be back at square one. > > I think the better way to proceed is to figure out why/when > significant changes are made while `inhibit-modification-hooks` is > non-nil, since that's the origin of your problems, AFAICT. See the above. There is real-world code doing things that make Emacs ignore after-change-functions. Combined with the issue revealed in this bug report, I am left with no Emacs tools to handle the problematic buffer modifications. Best, Ihor
> From: Ihor Radchenko <yantar92@gmail.com> > Cc: Eli Zaretskii <eliz@gnu.org>, 51766@debbugs.gnu.org > Date: Fri, 17 Jun 2022 18:05:36 +0800 > > Combined with the issue revealed in this bug report, I am left with no > Emacs tools to handle the problematic buffer modifications. You aren't supposed to try to do that in Lisp. I suggest to describe a generalization of the use cases you are aware of which you think need this, and then we could think about implementing some or all of it in C. > valign relies on disabling modification hooks because it is otherwise > difficult to figure out pixel width of a string in current buffer: > https://github.com/casouri/valign/issues/30 That discussion is very short and lacking in detail, but up front, why doesn't valign use the primitives we provide for determining the pixel width of a string?
Eli Zaretskii [2022-06-17 08:36:37] wrote:
>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: Eli Zaretskii <eliz@gnu.org>, 51766@debbugs.gnu.org
>> Date: Thu, 16 Jun 2022 22:54:22 -0400
>>
>> I think the better way to proceed is to figure out why/when
>> significant changes are made while `inhibit-modification-hooks` is
>> non-nil, since that's the origin of your problems, AFAICT.
>
> I thought that was clear from the rest of the discussion: it's quail's
> input methods that cause the issue, because quail tries to pretend
> that just one character was inserted, when in fact the user could type
> several characters.
AFAIK in the case of Quail the char-modified-ticks changes (so there's
some insertions/deletions going on) while `inhibit-modification-hooks`
is set, but the state of the buffer at the next
`before-change-functions` is correct, e.g. the buffer-hash is unchanged.
IOW in the cse of Quail the Org mode code doesn't need to flush the
whole parser's state, which means that the code that flushes the parser
state when char-modified-ticks is modified silently was written to
defend against *other* problems.
Stefan
> Well. This was also my assumption. I initially implemented the checking > code to detect internal errors in Org. > > Then we received 12 bug reports on this. The offenders were identified > in some cases. They were: > - polymode, valign, and ox-hugo > - Doom (unrelated to this particular issue, but Doom is let-binding major-mode...) > - Emacs quail and also replace-match in some cases (because of > false-positives caused by changing buffer-chars-modified-tick) IIUC Quail and replace-match shouldn't be in the above list: they're caught by your debugging check but they're false positives. > valign relies on disabling modification hooks because it is otherwise > difficult to figure out pixel width of a string in current buffer: > https://github.com/casouri/valign/issues/30 AFAIK this is *also* a false positive: the buffer is only temporarily modified within the `with-silent-modifications` and reverted to its previous state before the end of that macro, so there's no need to flush your parser's state. >> I don't think this *can* exist: if we add a mechanism which ignores >> `inhibit-modification-hooks` it will still need some way to ignore some >> changes so we'll need another `inhibit-<foo>` variable to "silence" >> those changes and we'll be back at square one. >> >> I think the better way to proceed is to figure out why/when >> significant changes are made while `inhibit-modification-hooks` is >> non-nil, since that's the origin of your problems, AFAICT. > > See the above. There is real-world code doing things that make > Emacs ignore after-change-functions. I don't see how this relates to what I'm saying: what I'm saying is that for the same reason there's code that has very valid reasons to inhibit `after-change-functions`, there will be code that has very valid reasons to inhibit some new `after-really-every-change-functions`, and then there will inevitably also be code that abuses this. The only real solution is to "push back" and get those abuses fixed. One thing you could do, for example is replace your char-modified-tick check with one based on buffer-size: it won't catch all cases, but it won't suffer from false positives, so you can really scream bloody murder when it happens. Stefan
Eli Zaretskii <eliz@gnu.org> writes:
>> valign relies on disabling modification hooks because it is otherwise
>> difficult to figure out pixel width of a string in current buffer:
>> https://github.com/casouri/valign/issues/30
>
> That discussion is very short and lacking in detail, but up front, why
> doesn't valign use the primitives we provide for determining the pixel
> width of a string?
Because string width in different buffers may be different depending on
the fontification, frame font size, face remapping,
wrap-prefix/line-prefix string properties (AFAIK, the built-in
string-pixel-width will return incorrect value on string with such
properties), invisibility specs in the buffer, line numbers mode, etc
We have implemented a number of workarounds in org-string-width on main,
but I am not 100% sure that I covered all the edge cases.
The most accurate way to measure the real string width inside current
buffer is actually inserting it and requesting the measurement.
Best,
Ihor
Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> See the above. There is real-world code doing things that make
>> Emacs ignore after-change-functions.
>
> I don't see how this relates to what I'm saying: what I'm saying is that
> for the same reason there's code that has very valid reasons to inhibit
> `after-change-functions`, there will be code that has very valid reasons
> to inhibit some new `after-really-every-change-functions`, and then
> there will inevitably also be code that abuses this.
>
> The only real solution is to "push back" and get those abuses fixed.
>
> One thing you could do, for example is replace your char-modified-tick
> check with one based on buffer-size: it won't catch all cases, but it
> won't suffer from false positives, so you can really scream bloody
> murder when it happens.
Checking the buffer-size is a great idea. Thanks!
It should be reliable enough for Org purposes.
Best,
Ihor
> From: Ihor Radchenko <yantar92@gmail.com>
> Cc: monnier@iro.umontreal.ca, 51766@debbugs.gnu.org
> Date: Tue, 21 Jun 2022 12:13:09 +0800
>
> Eli Zaretskii <eliz@gnu.org> writes:
>
> > That discussion is very short and lacking in detail, but up front, why
> > doesn't valign use the primitives we provide for determining the pixel
> > width of a string?
>
> Because string width in different buffers may be different depending on
> the fontification, frame font size, face remapping,
> wrap-prefix/line-prefix string properties (AFAIK, the built-in
> string-pixel-width will return incorrect value on string with such
> properties), invisibility specs in the buffer, line numbers mode, etc
> We have implemented a number of workarounds in org-string-width on main,
> but I am not 100% sure that I covered all the edge cases.
If you need such high accuracy, may I suggest window-text-pixel-size?
Eli Zaretskii <eliz@gnu.org> writes:
>> > That discussion is very short and lacking in detail, but up front, why
>> > doesn't valign use the primitives we provide for determining the pixel
>> > width of a string?
>>
>> Because string width in different buffers may be different depending on
>> the fontification, frame font size, face remapping,
>> wrap-prefix/line-prefix string properties (AFAIK, the built-in
>> string-pixel-width will return incorrect value on string with such
>> properties), invisibility specs in the buffer, line numbers mode, etc
>> We have implemented a number of workarounds in org-string-width on main,
>> but I am not 100% sure that I covered all the edge cases.
>
> If you need such high accuracy, may I suggest window-text-pixel-size?
window-text-pixel-size suffers from the same issues with
wrap-prefix/line-prefix and line numbers mode.
Also, in order to use it in current buffer on not-yet-inserted string,
you need to insert it. That where the issue in valign originated from.
Best,
Ihor
> From: Ihor Radchenko <yantar92@gmail.com> > Cc: monnier@iro.umontreal.ca, 51766@debbugs.gnu.org > Date: Tue, 21 Jun 2022 19:00:34 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> > That discussion is very short and lacking in detail, but up front, why > >> > doesn't valign use the primitives we provide for determining the pixel > >> > width of a string? > >> > >> Because string width in different buffers may be different depending on > >> the fontification, frame font size, face remapping, > >> wrap-prefix/line-prefix string properties (AFAIK, the built-in > >> string-pixel-width will return incorrect value on string with such > >> properties), invisibility specs in the buffer, line numbers mode, etc > >> We have implemented a number of workarounds in org-string-width on main, > >> but I am not 100% sure that I covered all the edge cases. > > > > If you need such high accuracy, may I suggest window-text-pixel-size? > > window-text-pixel-size suffers from the same issues with > wrap-prefix/line-prefix and line numbers mode. What issue are those? > Also, in order to use it in current buffer on not-yet-inserted string, > you need to insert it. That where the issue in valign originated from. The usual method is to use a temporary buffer.
Eli Zaretskii <eliz@gnu.org> writes: >> >> Because string width in different buffers may be different depending on >> >> the fontification, frame font size, face remapping, >> >> wrap-prefix/line-prefix string properties (AFAIK, the built-in >> >> string-pixel-width will return incorrect value on string with such >> >> properties), invisibility specs in the buffer, line numbers mode, etc >> >> We have implemented a number of workarounds in org-string-width on main, >> >> but I am not 100% sure that I covered all the edge cases. >> > >> > If you need such high accuracy, may I suggest window-text-pixel-size? >> >> window-text-pixel-size suffers from the same issues with >> wrap-prefix/line-prefix and line numbers mode. > > What issue are those? The length of line-prefix is added to the return value of window-text-pixel-size, which makes it wrong when the intention is measuring the actual string width. Not that window-text-pixel-size is misbehaving here - line-prefix does need to be included if the intention is to query the actual full width of text in buffer. The same goes for line numbers mode - line numbers are a part of window size. However, using window-text-pixel-size becomes awkward as the means to measure expected string width when it is going to be inserted into current buffer. >> Also, in order to use it in current buffer on not-yet-inserted string, >> you need to insert it. That where the issue in valign originated from. > > The usual method is to use a temporary buffer. Yes, and it is not accurate because temporary buffer may not have the same local environment for face remapping, invisibility specs, char-property-alias, etc To show all the trickery, let me share org-string-width I had to implement for the purposes of Org mode. I did not want this function to be complex and every single extra LOC there is fixing some edge case, test failure, or bug report: (defun org-string-width (string &optional pixels) "Return width of STRING when displayed in the current buffer. Return width in pixels when PIXELS is non-nil." (if (and (version< emacs-version "28") (not pixels)) ;; FIXME: Fallback to old limited version, because ;; `window-pixel-width' is buggy in older Emacs. (org--string-width-1 string) ;; Wrap/line prefix will make `window-text-pizel-size' return too ;; large value including the prefix. (remove-text-properties 0 (length string) '(wrap-prefix t line-prefix t) string) ;; Face should be removed to make sure that all the string symbols ;; are using default face with constant width. Constant char width ;; is critical to get right string width from pixel width (not needed ;; when PIXELS are requested though). (unless pixels (remove-text-properties 0 (length string) '(face t) string)) (let (;; We need to remove the folds to make sure that folded table ;; alignment is not messed up. (current-invisibility-spec (or (and (not (listp buffer-invisibility-spec)) buffer-invisibility-spec) (let (result) (dolist (el buffer-invisibility-spec) (unless (or (memq el '(org-fold-drawer org-fold-block org-fold-outline)) (and (listp el) (memq (car el) '(org-fold-drawer org-fold-block org-fold-outline)))) (push el result))) result))) (current-char-property-alias-alist char-property-alias-alist)) (with-temp-buffer (setq-local display-line-numbers nil) (setq-local buffer-invisibility-spec (if (listp current-invisibility-spec) (mapcar (lambda (el) ;; Consider elipsis to have 0 width. ;; It is what Emacs 28+ does, but we have ;; to force it in earlier Emacs versions. (if (and (consp el) (cdr el)) (list (car el)) el)) current-invisibility-spec) current-invisibility-spec)) (setq-local char-property-alias-alist current-char-property-alias-alist) (let (pixel-width symbol-width) (with-silent-modifications (setf (buffer-string) string) (setq pixel-width (if (get-buffer-window (current-buffer)) (car (window-text-pixel-size nil (line-beginning-position) (point-max))) (set-window-buffer nil (current-buffer)) (car (window-text-pixel-size nil (line-beginning-position) (point-max))))) (unless pixels (setf (buffer-string) "a") (setq symbol-width (if (get-buffer-window (current-buffer)) (car (window-text-pixel-size nil (line-beginning-position) (point-max))) (set-window-buffer nil (current-buffer)) (car (window-text-pixel-size nil (line-beginning-position) (point-max))))))) (if pixels pixel-width (/ pixel-width symbol-width))))))) Best, Ihor
> From: Ihor Radchenko <yantar92@gmail.com>
> Cc: monnier@iro.umontreal.ca, 51766@debbugs.gnu.org
> Date: Tue, 21 Jun 2022 20:39:10 +0800
>
> >> > If you need such high accuracy, may I suggest window-text-pixel-size?
> >>
> >> window-text-pixel-size suffers from the same issues with
> >> wrap-prefix/line-prefix and line numbers mode.
> >
> > What issue are those?
> [...]
> To show all the trickery, let me share org-string-width I had to
> implement for the purposes of Org mode. I did not want this function to
> be complex and every single extra LOC there is fixing some edge case,
> test failure, or bug report:
Ah, so you do use window-text-pixel-size... Then we are in violent
agreement.
Anyway, the beginning of this sub-thread, specifically about valign,
was in the context of Lisp programs that do buffer modifications under
with-silent-modifications or equivalent, and valign seems to do that
because it just needs to measure the pixel width of a string, and it
does that by inserting the string and then removing it. So in that
case, the "buffer modifications" are indeed null and void, and Org
shouldn't be bothered by such "modifications", because the buffer
really remains unmodified. Right?
Note that I changed the topic in this particular branch. My aim here is
to make you aware about the issues with current Emacs tools to measure
pixel-precise string width. Maybe, things can be improved on Emacs side
in this regard.
The below is going back to the initial topic.
Eli Zaretskii <eliz@gnu.org> writes:
> Anyway, the beginning of this sub-thread, specifically about valign,
> was in the context of Lisp programs that do buffer modifications under
> with-silent-modifications or equivalent, and valign seems to do that
> because it just needs to measure the pixel width of a string, and it
> does that by inserting the string and then removing it. So in that
> case, the "buffer modifications" are indeed null and void, and Org
> shouldn't be bothered by such "modifications", because the buffer
> really remains unmodified. Right?
In short, you are right. To clarify the problem on my side goes like:
1. Org has a real issue with bad third-party code inhibiting
before/after-change function + modifications in indirect buffers not
always triggering before/after-change
2. Because the issue is critical and can cause data corruption, we
cannot just ignore it
3. The first attempt to detect "stealthy" modifications was using
buffer-chars-modified-tick
4. But this method is not reliable because (a) quail does some legit
edits under inhibit-modification-hooks; (b) some other code, like
valign also does legit edits under inhibit-modification-hooks
These buffer modifications are harmless from Org perspective.
5. However, We end up with numerous false-positives using (3) and I am
clueless how to reliably detect or work around harmful "stealthy"
edits
- The suggestion to compare buffer size is helpful, but not 100%
reliable
- My other idea to request before/after-change function variants are
too specific to the problem at hand and may be not good for Emacs
in a whole
In any case, bug#51766 should not be considered a bug because quail does
modify the buffer and changes in buffer-chars-modified-tick are legit.
Best,
Ihor
> 1. Org has a real issue with bad third-party code inhibiting > before/after-change function + modifications in indirect buffers not > always triggering before/after-change > 2. Because the issue is critical and can cause data corruption, we > cannot just ignore it The bug is not yours. You don't have to ignore it, but you don't have to fix it either. > - The suggestion to compare buffer size is helpful, but not 100% > reliable To the extent that this just helps to detect other people's bug, I don't think it needs to be 100%. Stefan