bug#44466: 27.1; quail input fails at read-only boundary

bug#44466: 27.1; quail input fails at read-only boundary

[dick]

Run this:

emacs-27.1 -Q -nw --eval "(progn \
        (switch-to-buffer \"*scratch*\") \
        (erase-buffer) \
        (save-excursion (insert (propertize \"\n\" 'read-only t))) \
        (set-input-method \"russian-computer\" t))"

Now type "a".  Note it is incorrect.  We want cyrillic, not latin.

Exit emacs.  Run this:

emacs-26.3 -Q -nw --eval "(progn \
        (switch-to-buffer \"*scratch*\") \
        (erase-buffer) \
        (save-excursion (insert (propertize \"\n\" 'read-only t))) \
        (set-input-method \"russian-computer\" t))"

Now type "a".  Note it is correct.



In GNU Emacs 27.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2020-09-20 built on dick
Repository revision: dff02e85245dfbbeea7d57dc1a753a9ba24616d5
Repository branch: gnus-dev-27
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.4 LTS

Recent messages:
Wrote /tmp/foo
Mark activated
Mark deactivated
Mark set
Quit [2 times]
Mark set
Quit
Mark set [2 times]
Saving file /tmp/foo...
Wrote /tmp/foo

Configured using:
 'configure --prefix=/home/dick/.local --with-rsvg=yes --with-xml2=yes
 PKG_CONFIG_PATH=/home/dick/.local/lib/pkgconfig'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY LIBSELINUX
GNUTLS LIBXML2 FREETYPE HARFBUZZ XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE
XIM MODULES THREADS JSON PDUMPER GMP

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Shell-script

Minor modes in effect:
  pyvenv-mode: t
  sh-electric-here-document-mode: t
  show-paren-mode: t
  global-magit-file-mode: t
  magit-auto-revert-mode: t
  global-git-commit-mode: t
  async-bytecomp-package-mode: t
  flx-ido-mode: t
  projectile-mode: t
  ido-everywhere: t
  override-global-mode: t
  shell-dirtrack-mode: t
  beacon-mode: t
  global-hl-line-mode: t
  winner-mode: t
  cl-old-struct-compat-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/dick/ESS/lisp/obsolete/ess-swv hides /home/dick/ESS/lisp/ess-swv
/home/dick/ESS/lisp/obsolete/ess-rutils hides /home/dick/ESS/lisp/ess-rutils
/home/dick/ESS/lisp/obsolete/ess-noweb hides /home/dick/ESS/lisp/ess-noweb
/home/dick/ESS/lisp/obsolete/mouseme hides /home/dick/ESS/lisp/mouseme
/home/dick/ESS/lisp/obsolete/ess-mouse hides /home/dick/ESS/lisp/ess-mouse
/home/dick/ESS/lisp/obsolete/ess-noweb-mode hides /home/dick/ESS/lisp/ess-noweb-mode
/home/dick/ESS/lisp/obsolete/make-regexp hides /home/dick/ESS/lisp/make-regexp
/home/dick/ESS/lisp/obsolete/ess-r-a hides /home/dick/ESS/lisp/ess-r-a
/home/dick/ESS/lisp/obsolete/ess-noweb-font-lock-mode hides /home/dick/ESS/lisp/ess-noweb-font-lock-mode
/home/dick/gomacro-mode/gomacro-mode hides /home/dick/.emacs.d/elpa/gomacro-mode-20200326.1103/gomacro-mode
/home/dick/ESS/lisp/julia-mode-latexsubs hides /home/dick/.emacs.d/elpa/julia-mode-20200717.1915/julia-mode-latexsubs
/home/dick/ESS/lisp/julia-mode hides /home/dick/.emacs.d/elpa/julia-mode-20200717.1915/julia-mode
/home/dick/.emacs.d/elpa/hydra-20170924.2259/lv hides /home/dick/.emacs.d/elpa/lv-20191106.1238/lv
/home/dick/melpa-stats/melpa-stats hides /home/dick/.emacs.d/elpa/melpa-stats-20190720.1833/melpa-stats
/home/dick/.emacs.d/elpa/async-20200113.1745/async-autoloads hides /home/dick/.local/share/emacs/site-lisp/emacs-async/async-autoloads
/home/dick/.emacs.d/elpa/async-20200113.1745/async-bytecomp hides /home/dick/.local/share/emacs/site-lisp/emacs-async/async-bytecomp
/home/dick/.emacs.d/elpa/async-20200113.1745/smtpmail-async hides /home/dick/.local/share/emacs/site-lisp/emacs-async/smtpmail-async
/home/dick/.emacs.d/elpa/async-20200113.1745/dired-async hides /home/dick/.local/share/emacs/site-lisp/emacs-async/dired-async
/home/dick/.emacs.d/elpa/async-20200113.1745/async hides /home/dick/.local/share/emacs/site-lisp/emacs-async/async
/home/dick/.emacs.d/elpa/async-20200113.1745/async-pkg hides /home/dick/.local/share/emacs/site-lisp/emacs-async/async-pkg
/home/dick/.emacs.d/lisp/json hides /home/dick/.local/share/emacs/27.1/lisp/json
/home/dick/.emacs.d/elpa/map-2.1/map hides /home/dick/.local/share/emacs/27.1/lisp/emacs-lisp/map

Features:
(shadow sort flyspell ispell footnote mail-extr gnus-msg gnus-art mm-uu
mml2015 mm-view mml-smime smime dig emacsbug sendmail texinfo apropos
descr-text quail tabify rect company-oddmuse company-keywords company-etags
company-gtags company-dabbrev-code company-dabbrev company-files company-cmake
company-xcode company-eclim company-css company-nxml company-bbdb company-go
company-template company-c-headers cl-print debug backtrace mm-archive
timezone es-mode yasnippet-snippets yasnippet company-capf elpy elpy-rpc
pyvenv eshell esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module
esh-groups esh-util elpy-shell elpy-profile elpy-django elpy-refactor
ein-markdown-mode poly-rst rst mule-util org-element avl-tree ol-eww ol-rmail
ol-mhe ol-irc ol-info ol-gnus nnir gnus-sum gnus-group gnus-undo gnus-start
gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int
gnus-range gnus-win ol-docview doc-view image-mode exif ol-bibtex bibtex
ol-bbdb ol-w3m org-tempo tempo org org-macro org-footnote org-pcomplete
org-list org-faces org-entities org-version ob-jupyter jupyter-org-extensions
jupyter-org-client jupyter-repl jupyter-kernel-manager jupyter-channel
jupyter-widget-client simple-httpd jupyter-client jupyter-comm-layer
jupyter-messages hmac-def jupyter-mime shr svg dom jupyter-kernelspec
jupyter-env jupyter-base ob-emacs-lisp ob-ein ob ob-tangle org-src ob-ref
ob-lob ob-table ob-exp ob-comint ob-core ob-eval org-table ol org-keys
org-compat org-macs org-loaddefs cal-menu calendar cal-loaddefs ein-ipynb-mode
js ein-process ein-jupyter exec-path-from-shell ein-dev ein-notebook python
tramp-sh tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat
parse-time iso8601 ls-lisp ein-python-send ein-traceback ein-shared-output
ein-pytools ein-pager view ein-completer ein-notification ein-scratchsheet
ein-worksheet poly-ein display-line-numbers polymode poly-lock polymode-base
polymode-weave polymode-export polymode-compat polymode-methods polymode-core
polymode-classes eieio-custom eieio-base ein-kill-ring ein-cell
ein-output-area ein-kernelinfo ein-kernel ein-ipdb ein-events ein-websocket
websocket bindat ein-file ein-node ewoc ein-notebooklist eieio-opt speedbar
sb-image ezimage dframe ein-contents-api ein-query ein-log ein-classes
ein-core request ein-utils deferred ein help-fns radix-tree ein-gat
bug-reference magit-extras face-remap pulse google-c-style cc-mode cc-fonts
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs ag
vc-svn find-dired ivy delsel colir color ivy-overlay dumb-jump f s flycheck
find-func jka-compr let-alist go-eldoc gomacro-mode go-mode find-file ffap
network-stream url-http url-gw nsm url-cache url-auth misearch multi-isearch
sh-script executable paredit-ext paredit dired-x inf-ruby ruby-mode smie
melpa-stats ht dash-functional anaphora a pp sx key-combo company
haskell-interactive-mode haskell-presentation-mode haskell-process
haskell-session haskell-compile haskell-mode haskell-cabal haskell-utils
haskell-font-lock haskell-indentation haskell-string haskell-sort-imports
haskell-lexeme rx haskell-align-imports haskell-complete-module
haskell-ghc-support etags fileloop generator dabbrev haskell-customize hydra
lv use-package-ensure paren magit-patch-changelog magit-patch magit-submodule
magit-obsolete magit-popup magit-blame magit-stash magit-reflog magit-bisect
magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
magit-sequence magit-notes magit-worktree magit-tag magit-merge magit-branch
magit-reset magit-files magit-refs magit-status magit magit-repos magit-apply
magit-wip magit-log which-func imenu magit-diff smerge-mode diff diff-mode
magit-core magit-autorevert autorevert filenotify magit-margin magit-transient
magit-process magit-mode git-commit transient magit-git magit-section
magit-utils crm log-edit message rmc dired dired-loaddefs rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader pcvs-util add-log with-editor async-bytecomp async pcase server
dash flx-ido flx solarized-theme solarized-definitions projectile ibuf-ext
ibuffer ibuffer-loaddefs grep gnus nnheader gnus-util rmail rmail-loaddefs
rfc2047 rfc2045 ietf-drums text-property-search time-date mail-utils mm-util
mail-prsvr google-translate-default-ui google-translate-core-ui ido
google-translate-core google-translate-tk google-translate-backend
use-package-bind-key bind-key auto-complete popup cus-edit cus-start cus-load
wid-edit ess-r-mode ess-r-flymake flymake-proc flymake warnings thingatpt
ess-r-xref xref ess-trns ess-r-package shell pcomplete ess-r-completion
ess-roxy ess-r-syntax ess-rd noutline outline easy-mmode hideshow ess-s-lang
ess-help ess-mode ess-inf project format-spec ess-tracebug ess ess-utils
ess-custom compile comint ansi-color emms-player-mplayer emms-player-simple
emms emms-compat twittering-mode epa derived epg epg-config tls gnutls puny
url url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util mailcap advice xml cl fzf cl-extra help-mode
use-package-core beacon hl-line winner ring finder-inf howdoyou-autoloads
json-reformat-autoloads json-snatcher-autoloads edmacro kmacro
sml-mode-autoloads tornado-template-mode-autoloads info package easymenu
browse-url url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json subr-x map url-vars seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win
x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page tab-bar
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame minibuffer cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs button
faces cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote threads
dbusbind inotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 1083062 111029)
 (symbols 48 74547 1)
 (strings 32 291004 12083)
 (string-bytes 1 8642226)
 (vectors 16 114291)
 (vector-slots 8 2908784 288750)
 (floats 8 733 985)
 (intervals 56 57324 1194)
 (buffers 1000 97))

bug#44466: 27.1; quail input fails at read-only boundary

[Eli Zaretskii]

> From: dick <dick.r.chiang@gmail.com>
> Date: Thu, 05 Nov 2020 08:48:08 -0500
> 
> 
> Run this:
> 
> emacs-27.1 -Q -nw --eval "(progn \
>         (switch-to-buffer \"*scratch*\") \
>         (erase-buffer) \
>         (save-excursion (insert (propertize \"\n\" 'read-only t))) \
>         (set-input-method \"russian-computer\" t))"
> 
> Now type "a".  Note it is incorrect.  We want cyrillic, not latin.

Thanks.

This is due to an attempt to fix a different problem, see

  https://lists.gnu.org/archive/html/emacs-devel/2019-09/msg00093.html

Evgeny, could you please look into this?  Thinking about the original
problem now, I actually don't think I understand why the fix we
installed then is correct.  It seems to effectively disable an input
method in that case, but why is that TRT, even if we are on a button,
let alone in a buffer with point at a character with read-only
property?  So I'm inclined to revert the change.

bug#44466: 27.1; quail input fails at read-only boundary

[Evgeny Zajcev]

[-- Attachment #1: Type: text/plain, Size: 497 bytes --]

What I want with that patch is that if some key is pressed on read-only
area in non-read-only buffer, then quail should do what it does as if
buffer marked as read only, i.e. no input method translations are done, and
all single char bindings continue to work.

Possible we need to check front-stickyness of the char at point along with
'read-only property:

..
                   (and (get-char-property (point) 'read-only)
                        (get-char-property (point) 'front-sticky)))

..

[-- Attachment #2: Type: text/html, Size: 666 bytes --]

bug#44466: 27.1; quail input fails at read-only boundary

[Eli Zaretskii]

> From: Evgeny Zajcev <lg.zevlg@gmail.com>
> Date: Sun, 8 Nov 2020 01:24:26 +0300
> Cc: dick <dick.r.chiang@gmail.com>, 44466@debbugs.gnu.org
> 
> What I want with that patch is that if some key is pressed on read-only area in non-read-only buffer, then
> quail should do what it does as if buffer marked as read only, i.e. no input method translations are done, and
> all single char bindings continue to work.

When an input method is used in a read-only buffer, Emacs barfs
because it doesn't allow inserting text into such a buffer.  It
doesn't insert the untranslated character, as what your patch did.

> Possible we need to check front-stickyness of the char at point along with 'read-only property:
> 
> ..
>                    (and (get-char-property (point) 'read-only)
>                         (get-char-property (point) 'front-sticky)))

Does this solve the problem in this case?

bug#44466: 27.1; quail input fails at read-only boundary

[Evgeny Zajcev]

> 8 нояб. 2020 г., в 07:29, Eli Zaretskii <eliz@gnu.org> написал(а):
>
> 
>>
>> From: Evgeny Zajcev <lg.zevlg@gmail.com>
>> Date: Sun, 8 Nov 2020 01:24:26 +0300
>> Cc: dick <dick.r.chiang@gmail.com>, 44466@debbugs.gnu.org
>>
>> What I want with that patch is that if some key is pressed on read-only area in non-read-only buffer, then
>> quail should do what it does as if buffer marked as read only, i.e. no input method translations are done, and
>> all single char bindings continue to work.
>
> When an input method is used in a read-only buffer, Emacs barfs
> because it doesn't allow inserting text into such a buffer.  It
> doesn't insert the untranslated character, as what your patch did.
>
>> Possible we need to check front-stickyness of the char at point along with 'read-only property:
>>
>> ..
>>                   (and (get-char-property (point) 'read-only)
>>                        (get-char-property (point) 'front-sticky)))
>
> Does this solve the problem in this case?

Yes, because this mimics what is done in
verify_interval_modification() function from textprop.c.

Possibly the best solution would be to make
verify_interval_modification() visible from elisp side and use it in
quail-input-method to check for writability at point.

--
lg

bug#44466: 27.1; quail input fails at read-only boundary

[Eli Zaretskii]

> From: Evgeny Zajcev <lg.zevlg@gmail.com>
> Date: Sun, 8 Nov 2020 09:42:23 +0300
> Cc: dick.r.chiang@gmail.com, 44466@debbugs.gnu.org
> 
> > When an input method is used in a read-only buffer, Emacs barfs
> > because it doesn't allow inserting text into such a buffer.  It
> > doesn't insert the untranslated character, as what your patch did.
> >
> >> Possible we need to check front-stickyness of the char at point along with 'read-only property:
> >>
> >> ..
> >>                   (and (get-char-property (point) 'read-only)
> >>                        (get-char-property (point) 'front-sticky)))
> >
> > Does this solve the problem in this case?
> 
> Yes, because this mimics what is done in
> verify_interval_modification() function from textprop.c.

Any reason why pressing a key on a button should disregard the active
input method?  What if the button needs the user to type the character
which the input method would produce?  AFAIU, the patch we installed a
year ago makes this impossible.

> Possibly the best solution would be to make
> verify_interval_modification() visible from elisp side and use it in
> quail-input-method to check for writability at point.

I think this would be overkill.

bug#44466: 27.1; quail input fails at read-only boundary

[lg.zevlg]

> 8 нояб. 2020 г., в 19:02, Eli Zaretskii <eliz@gnu.org> написал(а):
> 
> 
>> 
>> From: Evgeny Zajcev <lg.zevlg@gmail.com>
>> Date: Sun, 8 Nov 2020 09:42:23 +0300
>> Cc: dick.r.chiang@gmail.com, 44466@debbugs.gnu.org
>> 
>>> When an input method is used in a read-only buffer, Emacs barfs
>>> because it doesn't allow inserting text into such a buffer.  It
>>> doesn't insert the untranslated character, as what your patch did.
>>> 
>>>> Possible we need to check front-stickyness of the char at point along with 'read-only property:
>>>> 
>>>> ..
>>>>                  (and (get-char-property (point) 'read-only)
>>>>                       (get-char-property (point) 'front-sticky)))
>>> 
>>> Does this solve the problem in this case?
>> 
>> Yes, because this mimics what is done in
>> verify_interval_modification() function from textprop.c.
> 
> Any reason why pressing a key on a button should disregard the active
> input method?  What if the button needs the user to type the character
> which the input method would produce?  AFAIU, the patch we installed a
> year ago makes this impossible.

The reason is the same as for read-only buffers, making single char bindings work. For example if you change input method in image-mode and press “q” key this will kill buffer, because correct command is executed.

Writable buffer might have read-only region with local-keymap installed with single char binding. If input method is applied, then instead of executing corresponding command you will get “text is read-only” error, because self-insert command is executed.


> 
>> Possibly the best solution would be to make
>> verify_interval_modification() visible from elisp side and use it in
>> quail-input-method to check for writability at point.
> 
> I think this would be overkill.

—
lg

bug#44466: 27.1; quail input fails at read-only boundary

[Eli Zaretskii]

> From: lg.zevlg@gmail.com
> Date: Sun, 8 Nov 2020 20:07:46 +0400
> Cc: dick.r.chiang@gmail.com, 44466@debbugs.gnu.org
> 
> > Any reason why pressing a key on a button should disregard the active
> > input method?  What if the button needs the user to type the character
> > which the input method would produce?  AFAIU, the patch we installed a
> > year ago makes this impossible.
> 
> The reason is the same as for read-only buffers, making single char bindings work. For example if you change input method in image-mode and press “q” key this will kill buffer, because correct command is executed.

If this is a feature, it should be documented somewhere.  Is it?

bug#44466: 27.1; quail input fails at read-only boundary

[lg.zevlg]

> 8 нояб. 2020 г., в 20:18, Eli Zaretskii <eliz@gnu.org> написал(а):
> 
> 
>> 
>> From: lg.zevlg@gmail.com
>> Date: Sun, 8 Nov 2020 20:07:46 +0400
>> Cc: dick.r.chiang@gmail.com, 44466@debbugs.gnu.org
>> 
>>> Any reason why pressing a key on a button should disregard the active
>>> input method?  What if the button needs the user to type the character
>>> which the input method would produce?  AFAIU, the patch we installed a
>>> year ago makes this impossible.
>> 
>> The reason is the same as for read-only buffers, making single char bindings work. For example if you change input method in image-mode and press “q” key this will kill buffer, because correct command is executed.
> 
> If this is a feature, it should be documented somewhere.  Is it?

I guess so, this feature was in Emacs for years. Usually people marks whole buffer as read-only and then mark special places to inhibit read-only, so quail translation inhibits for read-only parts and does the translation in writable parts. In telega.el project we do opposit - insert read-only parts in writable buffer and expect quail translation not to apply in that read-only parts of the buffer. We might change this behaviour, so patch to quail could be reverted, however I think it is better to fix this in quail, to reflect the logic in internal checks for region writability

—
lg

bug#44466: 27.1; quail input fails at read-only boundary

[Eli Zaretskii]

> From: lg.zevlg@gmail.com
> Date: Sun, 8 Nov 2020 23:50:54 +0400
> Cc: dick.r.chiang@gmail.com, 44466@debbugs.gnu.org
> 
> 
> > 8 нояб. 2020 г., в 20:18, Eli Zaretskii <eliz@gnu.org> написал(а):
> > 
> > 
> >> 
> >> From: lg.zevlg@gmail.com
> >> Date: Sun, 8 Nov 2020 20:07:46 +0400
> >> Cc: dick.r.chiang@gmail.com, 44466@debbugs.gnu.org
> >> 
> >>> Any reason why pressing a key on a button should disregard the active
> >>> input method?  What if the button needs the user to type the character
> >>> which the input method would produce?  AFAIU, the patch we installed a
> >>> year ago makes this impossible.
> >> 
> >> The reason is the same as for read-only buffers, making single char bindings work. For example if you change input method in image-mode and press “q” key this will kill buffer, because correct command is executed.
> > 
> > If this is a feature, it should be documented somewhere.  Is it?
> 
> I guess so, this feature was in Emacs for years. Usually people marks whole buffer as read-only and then mark special places to inhibit read-only, so quail translation inhibits for read-only parts and does the translation in writable parts. In telega.el project we do opposit - insert read-only parts in writable buffer and expect quail translation not to apply in that read-only parts of the buffer. We might change this behaviour, so patch to quail could be reverted, however I think it is better to fix this in quail, to reflect the logic in internal checks for region writability

Thanks, I've now fixed this on the emacs-27 branch.