bug#38153: 26.3; nxml with ENTITY: single quote misinterpreted as string delimiter

bug#38153: 26.3; nxml with ENTITY: single quote misinterpreted as string delimiter

[Jay Berkenbilt]

Create a file with the name "a.xml" and the following contents (between
the rows of dashes):

-------
<?xml version="1.0"?>
<!DOCTYPE anything [
<!ENTITY any "&#x03C0;">
]>
<anything>
  This is a quote, but it's not starting a string.
</anything>
-------

Observe that everything after the "'" is treated syntactically as a
string. This shows with the fontification of the following text, but it
also effects the behavior. For example, if you type <x> after the word
"quote" and then hit C-c C-f, the closing tag </x> is inserted as
expected. However, if you type <x> after the word "not", C-c C-f
erroneously attempts to close the <anything> tag.

If you remove the <!ENTITY ...> line, the problem goes away.

This behavior is observable with emacs -Q running in Ubuntu 19.10. It
did not occur with Ubuntu 19.04.

In GNU Emacs 26.3 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.11)
 of 2019-09-23, modified by Debian built on lgw01-amd64-032
Windowing system distributor 'The X.Org Foundation', version 11.0.12005000
System Description: Ubuntu 19.10

Recent messages:
Wrote /home/ejb/source/qpdf/qpdf/.git/rebase-merge/git-rebase-todo
When done with a buffer, type C-x #
Mark saved where search started
Quit [2 times]
Saving file /home/ejb/source/qpdf/qpdf/.git/rebase-merge/git-rebase-todo...
Wrote /home/ejb/source/qpdf/qpdf/.git/rebase-merge/git-rebase-todo
When done with a buffer, type C-x #
Saving file /home/ejb/source/qpdf/qpdf/.git/COMMIT_EDITMSG...
Wrote /home/ejb/source/qpdf/qpdf/.git/COMMIT_EDITMSG
Mark set
Quit
Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.3/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.3/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs-wvqver/emacs-26.3+1=.
 -fstack-protector-strong -Wformat -Werror=format-security -Wall'
 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Change Log

Minor modes in effect:
  shell-dirtrack-mode: t
  diff-auto-refine-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  auto-fill-function: do-auto-fill

Load-path shadows:
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-autoloads hides
/usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.0/debian-autoloads
/usr/share/emacs/site-lisp/elpa/debian-el-37/preseed hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/preseed
/usr/share/emacs/site-lisp/elpa/debian-el-37/gnus-BTS hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/gnus-BTS
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-el hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-el
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-el-pkg hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-el-pkg
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-el-autoloads hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-el-autoloads
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-bug hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-bug
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-autoloads hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/debian-autoloads
/usr/share/emacs/site-lisp/elpa/debian-el-37/deb-view hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/deb-view
/usr/share/emacs/site-lisp/elpa/debian-el-37/apt-utils hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/apt-utils
/usr/share/emacs/site-lisp/elpa/debian-el-37/apt-sources hides
/usr/share/emacs/site-lisp/elpa-src/debian-el-37/apt-sources
/usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.0/readme-debian hides
/usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.0/readme-debian
/usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.0/dpkg-dev-el hides
/usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.0/dpkg-dev-el
/usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.0/dpkg-dev-el-pkg hides
/usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.0/dpkg-dev-el-pkg
/usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.0/dpkg-dev-el-autoloads
hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.0/dpkg-dev-el-autoloads
/usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.0/debian-copyright
hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.0/debian-copyright
/usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.0/debian-control-mode
hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.0/debian-control-mode
/usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.0/debian-changelog-mode
hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.0/debian-changelog-mode
/usr/share/emacs/site-lisp/elpa/dpkg-dev-el-37.0/debian-bts-control
hides /usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.0/debian-bts-control
/usr/share/emacs/site-lisp/elpa/debian-el-37/debian-autoloads hides
/usr/share/emacs/site-lisp/elpa-src/dpkg-dev-el-37.0/debian-autoloads
/home/ejb/elisp/startup hides /usr/share/emacs/26.3/lisp/startup
/usr/share/emacs/site-lisp/flim/md4 hides /usr/share/emacs/26.3/lisp/md4
/usr/share/emacs/site-lisp/flim/hex-util hides
/usr/share/emacs/26.3/lisp/hex-util
/home/ejb/elisp/q/project hides /usr/share/emacs/26.3/lisp/progmodes/project
/usr/share/emacs/site-lisp/flim/sasl hides /usr/share/emacs/26.3/lisp/net/sasl
/usr/share/emacs/site-lisp/flim/sasl-ntlm hides
/usr/share/emacs/26.3/lisp/net/sasl-ntlm
/usr/share/emacs/site-lisp/flim/sasl-digest hides
/usr/share/emacs/26.3/lisp/net/sasl-digest
/usr/share/emacs/site-lisp/flim/sasl-cram hides
/usr/share/emacs/26.3/lisp/net/sasl-cram
/usr/share/emacs/site-lisp/flim/ntlm hides /usr/share/emacs/26.3/lisp/net/ntlm
/usr/share/emacs/site-lisp/flim/hmac-md5 hides
/usr/share/emacs/26.3/lisp/net/hmac-md5
/usr/share/emacs/site-lisp/flim/hmac-def hides
/usr/share/emacs/26.3/lisp/net/hmac-def

Features:
(shadow sort mail-extr warnings emacsbug make-mode cc-mode cc-fonts
cc-guess cc-menus cc-cmds shell pcomplete grep fg-lisp-indent elec-pair
eieio-opt speedbar sb-image ezimage dframe find-func files-x ibuf-ext
ibuffer ibuffer-loaddefs two-column iso-transl cl-extra pp cl-print
help-fns radix-tree dabbrev thai-util thai-word lao-util help-mode
goto-addr view add-log flyspell rng-xsd xsd-regexp rng-cmpct rng-nxml
rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt rng-util
rng-pttrn nxml-ns nxml-mode nxml-outln nxml-rap sgml-mode dom nxml-util
nxml-enc xmltok ispell misearch multi-isearch markdown-mode thingatpt
noutline outline smerge-mode vc-git diff-mode easy-mmode finder-inf
dpkg-dev-el info debian-el package url-handlers url-parse auth-source
cl-seq eieio eieio-core eieio-loaddefs url-vars vc-svn vc vc-dispatcher
mailcap doc-view image-mode timezone w3m-hist w3m-fb bookmark-w3m
w3m-ems w3m-ccl ccl w3m-favicon w3m-image w3m-proc w3m-util mime-w3m
qmime qmime-compose qmime-view filecache server compile ange-ftp comint
ansi-color ring message sendmail rmc puny seq byte-opt bytecomp
byte-compile cconv dired dired-loaddefs format-spec rfc822 mml mml-sec
password-cache epa derived epg epg-config gnus-util rmail rmail-loaddefs
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr mailabbrev mail-utils gmm-utils mailheader
cc-styles cc-align cc-engine cc-vars cc-defs project cl-macs jka-compr
cus-edit easymenu wid-edit cus-start cus-load edmacro kmacro advice cl
gv cl-loaddefs cl-lib w3m-load time-date mule-util tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 482720 128970)
 (symbols 48 46593 1)
 (miscs 40 774 1780)
 (strings 32 99324 14539)
 (string-bytes 1 2525152)
 (vectors 16 41095)
 (vector-slots 8 1626077 102230)
 (floats 8 126 428)
 (intervals 56 19379 112)
 (buffers 992 45))

bug#38153: 26.3; nxml with ENTITY: single quote misinterpreted as string delimiter

[Lars Ingebrigtsen]

Jay Berkenbilt <ejb@ql.org> writes:

> Create a file with the name "a.xml" and the following contents (between
> the rows of dashes):
>
> -------
> <?xml version="1.0"?>
> <!DOCTYPE anything [
> <!ENTITY any "&#x03C0;">
> ]>
> <anything>
>   This is a quote, but it's not starting a string.
> </anything>
> -------
>
> Observe that everything after the "'" is treated syntactically as a
> string. This shows with the fontification of the following text, but it
> also effects the behavior. For example, if you type <x> after the word
> "quote" and then hit C-c C-f, the closing tag </x> is inserted as
> expected. However, if you type <x> after the word "not", C-c C-f
> erroneously attempts to close the <anything> tag.

I'm not able to reproduce this bug in Emacs 27, but I am able to
reproduce it in Emacs 26.3, so I think this has been fixed already.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no