bug#34704: 26.1; Wrong line endings detected for XML files

[Akkana Peck <akkana@shallowsky.com>]

[-- Attachment #1: Type: text/plain, Size: 15495 bytes --]

I often use emacs to edit GPX files produced by OsmAnd.
Emacs displays these files with ^M at the end, failing to detect
the proper newlines. od shows the file as having \r\n (^J^M)
at the end of every line.

I initially thought this was because the final line of the file
lacks any newline characters, but if I edit the file and write it,
emacs adds a final ^J^M, and if I then try to read that file back
into emacs it still fails to detect the proper line endings, and
shows the ^M on every line.

The problem is related to the file containing XML:
if I edit the file and change all the < > to ( ) then open the file,
it opens in Fundamental mode and emacs uses DOS encoding and doesn't
show the ^M characters. But if I open the file normally (it uses nXML
mode) then switch to Fundamental, that doesn't fix the erroneous
newline detection. Of course I can fix it manually, but I'd like
to find a way to get emacs to open the file properly without my
needing to run extra commands.

I asked about this on #emacs and a helpful person commented:

    | I think auto-coding-functions is a problem.
    | sgml-xml-auto-coding-function specifically.
    | mule.el: (memq detected '(utf-8 'utf-8-with-signature 'utf-8-hfs undecided))
    | That looks a bit funny to me.
    | Ugh, no, it's both sgml-* functions in auto-coding-functions?
    | I must have screwed something up, sgml-html-meta-auto-coding-function returns nil for me.
    | But sgml-xml-auto-coding-function returns utf-8-unix.
    | I don't know enough about how Emacs chooses character sets to speculate at how this should be fixed, but I *think* that function should probably return utf-8 not utf-8-unix.

I'm attaching a small test file that shows the problem.



In GNU Emacs 26.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.4)
 of 2019-02-03, modified by Debian built on zam904
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description:	Debian GNU/Linux buster/sid

Recent messages:
Quit [2 times]
Saving file /tmp/test.paren...
Wrote /tmp/test.paren
Using vacuous schema
Type C-x 1 to delete the help window.
Using vacuous schema
Quit
Missing space after name token
Error during redisplay: (jit-lock-function 59) signaled (cl-assertion-failed (bobp))
Entering debugger...

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --enable-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --build
 x86_64-linux-gnu --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man --enable-libsystemd
 --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/26.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/26.1/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs-26.1+1=. -fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 THREADS LIBSYSTEMD LCMS2

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: nXML

Minor modes in effect:
  recentf-mode: t
  dubcaps-mode: t
  auto-insert-mode: t
  global-keys-minor-mode: t
  delete-selection-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-etags hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-etags
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-tng hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-tng
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-dabbrev hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-dabbrev
/usr/share/emacs/site-lisp/elpa/company-0.9.9/all hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/all
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-eclim hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-eclim
/usr/share/emacs/site-lisp/elpa/company-0.9.9/clang-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/clang-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-files hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-files
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-clang hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-clang
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-semantic hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-semantic
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-abbrev hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-abbrev
/usr/share/emacs/site-lisp/elpa/company-0.9.9/keywords-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/keywords-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/transformers-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/transformers-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/frontends-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/frontends-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-bbdb hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-bbdb
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-yasnippet hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-yasnippet
/usr/share/emacs/site-lisp/elpa/company-0.9.9/files-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/files-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/elisp-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/elisp-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/core-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/core-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-keywords hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-keywords
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-oddmuse hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-oddmuse
/usr/share/emacs/site-lisp/elpa/company-0.9.9/bbdb-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/bbdb-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-autoloads hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-autoloads
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-dabbrev-code hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-dabbrev-code
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-pkg hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-pkg
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-cmake hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-cmake
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-capf hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-capf
/usr/share/emacs/site-lisp/elpa/company-0.9.9/capf-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/capf-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-nxml hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-nxml
/usr/share/emacs/site-lisp/elpa/company-0.9.9/cmake-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/cmake-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-elisp hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-elisp
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-tempo hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-tempo
/usr/share/emacs/site-lisp/elpa/company-0.9.9/async-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/async-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-css hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-css
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-ispell hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-ispell
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-xcode hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-xcode
/usr/share/emacs/site-lisp/elpa/company-0.9.9/template-tests hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/template-tests
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-gtags hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-gtags
/usr/share/emacs/site-lisp/elpa/company-0.9.9/company-template hides /usr/share/emacs/site-lisp/elpa-src/company-0.9.9/company-template
/usr/share/emacs/site-lisp/elpa/elpy-1.28.0/elpy-profile hides /usr/share/emacs/site-lisp/elpa-src/elpy-1.28.0/elpy-profile
/usr/share/emacs/site-lisp/elpa/elpy-1.28.0/elpy-autoloads hides /usr/share/emacs/site-lisp/elpa-src/elpy-1.28.0/elpy-autoloads
/usr/share/emacs/site-lisp/elpa/elpy-1.28.0/elpy-pkg hides /usr/share/emacs/site-lisp/elpa-src/elpy-1.28.0/elpy-pkg
/usr/share/emacs/site-lisp/elpa/elpy-1.28.0/elpy-refactor hides /usr/share/emacs/site-lisp/elpa-src/elpy-1.28.0/elpy-refactor
/usr/share/emacs/site-lisp/elpa/elpy-1.28.0/elpy-shell hides /usr/share/emacs/site-lisp/elpa-src/elpy-1.28.0/elpy-shell
/usr/share/emacs/site-lisp/elpa/elpy-1.28.0/debian-autoloads hides /usr/share/emacs/site-lisp/elpa-src/elpy-1.28.0/debian-autoloads
/usr/share/emacs/site-lisp/elpa/elpy-1.28.0/elpy-django hides /usr/share/emacs/site-lisp/elpa-src/elpy-1.28.0/elpy-django
/usr/share/emacs/site-lisp/elpa/elpy-1.28.0/elpy hides /usr/share/emacs/site-lisp/elpa-src/elpy-1.28.0/elpy
/usr/share/emacs/site-lisp/elpa/find-file-in-project-5.7.2/find-file-in-project hides /usr/share/emacs/site-lisp/elpa-src/find-file-in-project-5.7.2/find-file-in-project
/usr/share/emacs/site-lisp/elpa/find-file-in-project-5.7.2/find-file-in-project-autoloads hides /usr/share/emacs/site-lisp/elpa-src/find-file-in-project-5.7.2/find-file-in-project-autoloads
/usr/share/emacs/site-lisp/elpa/find-file-in-project-5.7.2/find-file-in-project-pkg hides /usr/share/emacs/site-lisp/elpa-src/find-file-in-project-5.7.2/find-file-in-project-pkg
/usr/share/emacs/site-lisp/elpa/highlight-indentation-0.7.0/highlight-indentation hides /usr/share/emacs/site-lisp/elpa-src/highlight-indentation-0.7.0/highlight-indentation
/usr/share/emacs/site-lisp/elpa/highlight-indentation-0.7.0/highlight-indentation-pkg hides /usr/share/emacs/site-lisp/elpa-src/highlight-indentation-0.7.0/highlight-indentation-pkg
/usr/share/emacs/site-lisp/elpa/highlight-indentation-0.7.0/highlight-indentation-autoloads hides /usr/share/emacs/site-lisp/elpa-src/highlight-indentation-0.7.0/highlight-indentation-autoloads
/usr/share/emacs/site-lisp/elpa/ivy-0.11.0/ivy-autoloads hides /usr/share/emacs/site-lisp/elpa-src/ivy-0.11.0/ivy-autoloads
/usr/share/emacs/site-lisp/elpa/ivy-0.11.0/colir hides /usr/share/emacs/site-lisp/elpa-src/ivy-0.11.0/colir
/usr/share/emacs/site-lisp/elpa/ivy-0.11.0/ivy hides /usr/share/emacs/site-lisp/elpa-src/ivy-0.11.0/ivy
/usr/share/emacs/site-lisp/elpa/ivy-0.11.0/ivy-overlay hides /usr/share/emacs/site-lisp/elpa-src/ivy-0.11.0/ivy-overlay
/usr/share/emacs/site-lisp/elpa/ivy-0.11.0/ivy-pkg hides /usr/share/emacs/site-lisp/elpa-src/ivy-0.11.0/ivy-pkg
/usr/share/emacs/site-lisp/elpa/pyvenv-1.20/pyvenv-autoloads hides /usr/share/emacs/site-lisp/elpa-src/pyvenv-1.20/pyvenv-autoloads
/usr/share/emacs/site-lisp/elpa/pyvenv-1.20/pyvenv hides /usr/share/emacs/site-lisp/elpa-src/pyvenv-1.20/pyvenv
/usr/share/emacs/site-lisp/elpa/pyvenv-1.20/pyvenv-pkg hides /usr/share/emacs/site-lisp/elpa-src/pyvenv-1.20/pyvenv-pkg
/usr/share/emacs/site-lisp/elpa/s-1.12.0/s-pkg hides /usr/share/emacs/site-lisp/elpa-src/s-1.12.0/s-pkg
/usr/share/emacs/site-lisp/elpa/s-1.12.0/s hides /usr/share/emacs/site-lisp/elpa-src/s-1.12.0/s
/usr/share/emacs/site-lisp/elpa/s-1.12.0/s-autoloads hides /usr/share/emacs/site-lisp/elpa-src/s-1.12.0/s-autoloads
/usr/share/emacs/site-lisp/elpa/yasnippet-0.13.0/yasnippet hides /usr/share/emacs/site-lisp/elpa-src/yasnippet-0.13.0/yasnippet
/usr/share/emacs/site-lisp/elpa/yasnippet-0.13.0/yasnippet-pkg hides /usr/share/emacs/site-lisp/elpa-src/yasnippet-0.13.0/yasnippet-pkg
/usr/share/emacs/site-lisp/elpa/yasnippet-0.13.0/yasnippet-autoloads hides /usr/share/emacs/site-lisp/elpa-src/yasnippet-0.13.0/yasnippet-autoloads
/usr/share/emacs/site-lisp/rst hides /usr/share/emacs/26.1/lisp/textmodes/rst

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
format-spec rfc822 mml mml-sec epa epg gnus-util rmail rmail-loaddefs
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils cl-print debug cus-edit cus-start cus-load thingatpt cl-extra
eieio-opt speedbar sb-image ezimage dframe find-func completion help-fns
radix-tree help-mode misearch multi-isearch rng-xsd xsd-regexp rng-cmpct
rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt
rng-util rng-pttrn nxml-ns nxml-mode nxml-outln nxml-rap sgml-mode dom
nxml-util nxml-enc xmltok executable flyspell ispell elec-pair recentf
tree-widget wid-edit web-mode advice disp-table rx derived autoinsert
undo-tree diff finder-inf undo-tree-autoloads package easymenu
epg-config url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache url-vars seq byte-opt gv bytecomp
byte-compile cconv easy-mmode delsel edmacro kmacro cl-loaddefs cl-lib
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 242866 90418)
 (symbols 48 30048 2)
 (miscs 40 479 1132)
 (strings 32 58710 4561)
 (string-bytes 1 1505161)
 (vectors 16 32349)
 (vector-slots 8 1417747 146910)
 (floats 8 90 605)
 (intervals 56 1403 50)
 (buffers 992 20))

[-- Attachment #2: test.gpx --]
[-- Type: text/plain, Size: 146 bytes --]

<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>
<gpx version="1.1" creator="OsmAnd+">
  <trk>
    <trkseg>
    </trkseg>
  </trk>
</gpx>