bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Justus-dev--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

In gnus with the default shr html renderer, incoming HTML mails
containing <![if !supportLists]>...<![endif]> tags will show these
literally. Instead, since (as I understand) shr is not designed to
handle the surrounding CSS magic and does not in fact support "Lists",
it should simply eliminate the <![if !supportLists]> and <![endif]> tags
and display just the enclosed text.

This may in fact be a libxml2 limitation with no clean solution in shr,
but I don't at this point understand libxml2 nor shr well enough to
tell.


In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.34, cairo version 1.17.6)
 of 2022-07-06 built on piater
Repository revision: 15404818503e0d3df7a8c56af13e4123bd231989
Repository branch: master
System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
 --with-modules --without-libotf --without-m17n-flt --without-gconf
 --with-native-compilation --with-xinput2 --with-pgtk --without-xaw3d
 --with-sound=no --without-gpm --without-compress-install
 '--program-transform-name=s/\([ec]tags\)/\1.emacs/'
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions
 -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection -flto=auto'
 'LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -flto=auto''

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LCMS2 LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PGTK
PNG RSVG SECCOMP SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS WEBP XIM GTK3
ZLIB

Important settings:
  value of $LC_COLLATE: de_AT.UTF-8
  value of $LC_MONETARY: de_AT.UTF-8
  value of $LC_TIME: de_AT.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Article

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  override-global-mode: t
  shell-dirtrack-mode: t
  pdfgrep-mode: t
  pdf-occur-global-minor-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/piater/.config/emacs/elpa/transient-20220527.2213/transient hides /usr/share/emacs/29.0.50/lisp/transient

Features:
(shadow emacsbug finder-inf html2text gnus-html cl-print ps-mule
ps-print ps-print-loaddefs ps-def lpr calc-incom calc-alg calc-ext
calc-menu calc calc-loaddefs rect calc-macs nndoc ein-ipynb-mode js
ein-process ein-jupyter ein-dev ein-notebook ein-python-send
ein-traceback ein-pytools ein-pager ein-completer ein-notification
ein-scratchsheet ein-worksheet poly-ein quail ein-kill-ring ein-cell
ein-shared-output ein-output-area ein-kernelinfo ein-kernel ein-ipdb
ein-events ein-websocket websocket bindat ein-file ein-node
ein-notebooklist shortdoc ido ein-contents-api ein-query ein-log
ein-classes ein-core request anaphora ein-utils deferred cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs ein pcmpl-unix pcmpl-gnu locate magit-patch magit-subtree
magit-gitignore magit-ediff ediff ediff-merg ediff-mult ediff-wind
ediff-diff ediff-help ediff-init ediff-util conf-mode find-dired
latexenc poly-markdown markdown-mode mm-extern gnus-search eieio-opt
speedbar ezimage dframe org-clock vc-src vc-sccs vc-svn vc-cvs vc-rcs
log-view vc bug-reference magit-extras rng-xsd xsd-regexp rng-cmpct
rng-nxml rng-valid nxml-mode nxml-outln nxml-rap sgml-mode
magit-bookmark magit-submodule magit-obsolete magit-blame magit-stash
magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone
magit-remote magit-commit magit-sequence magit-notes magit-worktree
magit-tag magit-merge magit-branch magit-reset magit-files magit-refs
magit-status magit magit-repos magit-apply magit-wip magit-log
which-func magit-diff smerge-mode diff git-commit log-edit pcvs-util
add-log magit-core magit-autorevert autorevert magit-margin
magit-transient magit-process with-editor magit-mode magit-git
magit-base crm python tramp-cmds vc-hg vc-git diff-mode vc-bzr
vc-dispatcher tramp-sh tramp-archive tramp-gvfs tramp-cache zeroconf
view woman man arc-mode archive-mode pdf-sync pdf-annot facemenu
pdf-outline pdf-links pdf-history tex-mode diary-lib diary-loaddefs
cal-iso eudc-capf eudc cus-start eudc-vars display-line-numbers
ement-taxy taxy-magit-section magit-section dash compat-27 compat-26
taxy svg-lib ement-room-list cal-move sh-script smie executable timezone
dabbrev tabify poly-org polymode poly-lock polymode-base polymode-weave
polymode-export polymode-compat polymode-methods polymode-core
polymode-classes eieio-custom eieio-base ffap org-super-links
org-capture dired-aux org-indent ol-eww eww xdg url-queue thingatpt
ol-rmail ol-mhe ol-irc ol-info ol-gnus nnselect ol-docview ol-bibtex
ol-bbdb ol-w3m ol-doi org-link-doi mailalias smtpmail flow-fill
shr-color gnus-topic misearch multi-isearch gnus-draft mule-util smiley
gnus-cite matlab matlab-scan matlab-syntax matlab-compat pulse mail-extr
textsec uni-scripts idna-mapping ucs-normalize uni-confusable
textsec-check gnus-async gnus-bcklg sort gnus-ml disp-table mm-archive
url-http url-gw url-cache url-auth qp utf-7 network-stream nsm imap
rfc2104 epa-file nnrss nndraft nnmh nnfolder nnmaildir gnus-agent
gnus-srvr gnus-score score-mode nnvirtual gnus-msg gnus-art mm-uu
mml2015 mm-view mml-smime smime gnutls dig nntp gnus-cache gnus-sum
gnus-group gnus-undo mm-url gnus-start gnus-dbus gnus-cloud nnimap
nnmail mail-source utf7 netrc nnoo gnus-spec gnus-int gnus-range message
sendmail yank-media rfc822 mml mml-sec epa derived epg rfc6068
epg-config mailabbrev gmm-utils mailheader gnus-win gnus nnheader
gnus-util mail-utils range gnus-dired use-package-bind-key bind-key
ement ement-notify notifications dbus ement-room transient comp
comp-cstr edmacro kmacro compat compat-macs face-remap shr pixel-fill
kinsoku url-file url-dired puny svg dom ewoc ement-lib ement-api
ement-structs ement-macros warnings xml color dns plz quelpa-use-package
cl-extra use-package-core quelpa mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
lisp-mnt help-fns radix-tree help-mode ox-odt rng-loc rng-uri rng-parse
rng-match rng-dt rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok
nxml-util ox-latex ox-icalendar ox-html table ox-ascii ox-publish ox
org-element avl-tree generator org-agenda org-id org-refile yaml-mode
native-complete shell-here tramp tramp-loaddefs trampver
tramp-integration files-x tramp-compat shell parse-time ls-lisp
org-recoll doc-view filenotify org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-footnote org-src ob-comint org-pcomplete pcomplete
org-list org-faces org-entities noutline outline easy-mmode org-version
ob-emacs-lisp ob-core ob-eval org-table oc-basic bibtex iso8601
time-date ol rx org-keys oc org-compat org-macs org-loaddefs cal-menu
calendar cal-loaddefs pdfgrep grep pdf-occur ibuf-ext ibuffer
ibuffer-loaddefs tablist advice tablist-filter semantic/wisent/comp
semantic/wisent semantic/wisent/wisent semantic/util-modes semantic/util
semantic semantic/tag semantic/lex semantic/fw mode-local find-func
cedet pdf-isearch let-alist pdf-misc imenu pdf-tools compile comint
ansi-color ring cus-edit cus-load wid-edit pdf-view bookmark
text-property-search pp jka-compr pdf-cache pdf-info tq pdf-util
format-spec pdf-macs image-mode dired dired-loaddefs exif server
deadgrep-autoloads ein-autoloads ement-autoloads ess-autoloads
datetime-autoloads magit-autoloads git-commit-autoloads
pdf-tools-autoloads markdown-mode-autoloads rnc-mode-autoloads
rust-mode-autoloads magit-section-autoloads info compat-autoloads
request-autoloads package browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie generate-lisp-file
url-domsuf url-util mailcap url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
byte-opt gv bytecomp byte-compile cconv url-vars cl-loaddefs cl-lib rmc
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win term/common-win
pgtk-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list
replace newcomment text-mode lisp-mode prog-mode register page tab-bar
menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse
jit-lock font-lock syntax font-core term/tty-colors frame minibuffer
nadvice seq simple cl-generic indonesian philippine cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop
case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads dbusbind
inotify dynamic-setting system-font-setting font-render-setting cairo
gtk pgtk lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 5882393 730120)
 (symbols 48 77801 70)
 (strings 32 1594098 80708)
 (string-bytes 1 97327779)
 (vectors 16 569961)
 (vector-slots 8 8733979 876067)
 (floats 8 3658 3665)
 (intervals 56 373170 10181)
 (buffers 992 272))

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Lars Ingebrigtsen]

Justus-dev@Piater.name writes:

> In gnus with the default shr html renderer, incoming HTML mails
> containing <![if !supportLists]>...<![endif]> tags will show these
> literally.

Sounds odd.  Do you have an example email like that?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Visuwesh]

[திங்கள் ஜூலை 11, 2022] Lars Ingebrigtsen wrote:

> Justus-dev@Piater.name writes:
>
>> In gnus with the default shr html renderer, incoming HTML mails
>> containing <![if !supportLists]>...<![endif]> tags will show these
>> literally.
>
> Sounds odd.  Do you have an example email like that?

I don't have an example email but this site
http://www.ambedkar.org/ambcd/01.Caste%20in%20India.htm produces
something similar except it is for annotations.

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Lars Ingebrigtsen]

Visuwesh <visuweshm@gmail.com> writes:

>>> In gnus with the default shr html renderer, incoming HTML mails
>>> containing <![if !supportLists]>...<![endif]> tags will show these
>>> literally.
>>
>> Sounds odd.  Do you have an example email like that?
>
> I don't have an example email but this site
> http://www.ambedkar.org/ambcd/01.Caste%20in%20India.htm produces
> something similar except it is for annotations.

I didn't see any <![ things in that page when visiting it with eww?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Visuwesh]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]

[திங்கள் ஜூலை 11, 2022] Lars Ingebrigtsen wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>> I don't have an example email but this site
>> http://www.ambedkar.org/ambcd/01.Caste%20in%20India.htm produces
>> something similar except it is for annotations.
>
> I didn't see any <![ things in that page when visiting it with eww?

Could it be related to libxml version after all because I see it here
(it is in the beg and the end of the buffer).

    % apt show libxml2
    Package: libxml2
    Version: 2.9.14+dfsg-1
    Priority: optional
    Section: libs
    Maintainer: Debian XML/SGML Group <debian-xml-sgml-pkgs@lists.alioth.debian.org>
    Installed-Size: 1,937 kB
    Depends: libc6 (>= 2.33), libicu71 (>= 71.1-1~), liblzma5 (>= 5.1.1alpha+20120614), zlib1g (>= 1:1.2.3.3)
    Conflicts: w3c-dtd-xhtml
    Homepage: http://xmlsoft.org
    Tag: role::shared-lib
    Download-Size: 708 kB
    APT-Manual-Installed: no
    APT-Sources: http://deb.debian.org/debian testing/main amd64 Packages

Attached screenshot is from the end of the eww buffer,


[-- Attachment #2: Type: image/png, Size: 78514 bytes --]

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Lars Ingebrigtsen]

Visuwesh <visuweshm@gmail.com> writes:

> Could it be related to libxml version after all because I see it here
> (it is in the beg and the end of the buffer).
>
>     % apt show libxml2
>     Package: libxml2
>     Version: 2.9.14+dfsg-1

Version: 2.9.13+dfsg-1ubuntu0.1 here...

It definitely sounds like a libxml2 issue, but it's odd that the newer
version has the problem, while the older one doesn't have it.  (Unless
1ubuntu0.1 is a patch for the problem.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Justus-dev--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Lars Ingebrigtsen <larsi@gnus.org> wrote on Mon, 11 Jul 2022 13:12:47
+0200:

> Visuwesh <visuweshm@gmail.com> writes:

>>     Package: libxml2
>>     Version: 2.9.14+dfsg-1
>
> Version: 2.9.13+dfsg-1ubuntu0.1 here...
>
> It definitely sounds like a libxml2 issue, but it's odd that the newer
> version has the problem, while the older one doesn't have it.

Indeed this seems to be a libxml2 regression! I downgraded my libxml to
libxml2-2.9.13-2 (Arch Linux), and the tags are omitted as they should
be. I re-upgraded to the current libxml2-2.9.14-1, and the problem is
back.

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Lars Ingebrigtsen]

Justus-dev@Piater.name writes:

> Indeed this seems to be a libxml2 regression! I downgraded my libxml to
> libxml2-2.9.13-2 (Arch Linux), and the tags are omitted as they should
> be. I re-upgraded to the current libxml2-2.9.14-1, and the problem is
> back.

I'm starting to wonder...  is 

<![if !bar]>Foo<![endif]>

valid syntax?  It's apparently an MSIE-ism:

https://en.wikipedia.org/wiki/Conditional_comment

A comment is supposed to be <!--, after all.

Perhaps the libxml people have "fixed" the parser to not be MSIE
compatible any more?  Anybody know?  In which case, perhaps shr should
do some pre-processing to remove the <![ things, but that'd be a
bother...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Visuwesh]

[திங்கள் ஜூலை 11, 2022] Lars Ingebrigtsen wrote:

> Perhaps the libxml people have "fixed" the parser to not be MSIE
> compatible any more?  Anybody know?  In which case, perhaps shr should
> do some pre-processing to remove the <![ things, but that'd be a
> bother...

I think it is a regression?  I tried the git blame interface in GNOME's
gitlab interface and the last commit that seems to touch the conditional
comments fun was from 4 months ago, and this commit is not in the 2.9.13
tag.

Git blame for the conditional comment fun:
https://gitlab.gnome.org/GNOME/libxml2/-/blame/master/parser.c#L6682

parser.c in 2.9.13 tag:
https://gitlab.gnome.org/GNOME/libxml2/-/blob/v2.9.13/parser.c

(see how it says last commit 5 months ago, whilst the latest relevant
commit according to git blame is 4 months ago.)

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Lars Ingebrigtsen]

Visuwesh <visuweshm@gmail.com> writes:

> I think it is a regression?  I tried the git blame interface in GNOME's
> gitlab interface and the last commit that seems to touch the conditional
> comments fun was from 4 months ago, and this commit is not in the 2.9.13
> tag.
>
> Git blame for the conditional comment fun:
> https://gitlab.gnome.org/GNOME/libxml2/-/blame/master/parser.c#L6682

Hm, right.  Somebody should report this to the libxml2 people, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Phil Sainty]

On 2022-07-12 01:04, Lars Ingebrigtsen wrote:
> I'm starting to wonder...  is
> <![if !bar]>Foo<![endif]>
> valid syntax?  It's apparently an MSIE-ism:
> https://en.wikipedia.org/wiki/Conditional_comment

Yes, I believe that's invalid, and therefore it's expected
behaviour that it would have a visible effect upon rendering.

So unless HTML5 has legitimised it, it's not really a bug.

I recall encountering this unique-to-MSIE syntax well over
a decade ago at an old job, and establishing a safe way to
get the same outcome without using it, because it wasn't
valid to use.

It's unfortunate that this still exists in the wild.  Now
that MSIE is officially dead, and assuming it's *not* part
of HTML5, it would be nice if things generating that syntax
were updated to not do that.  (Perhaps libxml has dropped
support for it as a result of MSIE reaching end-of-life?)

I think what I came up with at the time was probably the
same thing I can see at that wikipedia link, which is that
instead of writing this:

<![if CONDITION]>
<link href="non-ie.css" rel="stylesheet">
<![endif]>

You instead structure it within valid comments like this:

<!--[if CONDITION]><!-->
<link href="non-ie.css" rel="stylesheet">
<!--<![endif]-->

Difficult to fix at the source if arbitrary people are
sending you this as email, though :(


-Phil

bug#56482: 29.0.50; shr displays <![if !supportLists]>...<![endif]> literally instead of dropping the tags

[Lars Ingebrigtsen]

Phil Sainty <psainty@orcon.net.nz> writes:

> Yes, I believe that's invalid, and therefore it's expected
> behaviour that it would have a visible effect upon rendering.
>
> So unless HTML5 has legitimised it, it's not really a bug.

Well, the libxml2 code tries to parse it, but the code has apparently
regressed.  So that's a bug.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no