bug#24893: 26.0.50; shr doesn't parse srcset <img> correctly

bug#24893: 26.0.50; shr doesn't parse srcset <img> correctly

[Lars Ingebrigtsen]

(shr--preferred-image
'(img ((src . "http://s3.amazonaws.com/ledejs/resized/allegiant/600/alle_winglede.jpg") (srcset . "
      //s3.amazonaws.com/ledejs/resized/allegiant/480/alle_winglede.jpg 480w,
      //s3.amazonaws.com/ledejs/resized/allegiant/480/alle_winglede-2x.jpg 960w,
      //s3.amazonaws.com/ledejs/resized/allegiant/600/alle_winglede.jpg 600w,
      //s3.amazonaws.com/ledejs/resized/allegiant/1024/alle_winglede.jpg 1024w,
      //s3.amazonaws.com/ledejs/resized/allegiant/600/alle_winglede-2x.jpg 1200w,
      //s3.amazonaws.com/ledejs/resized/allegiant/1500/alle_winglede.jpg 1500w,
      //s3.amazonaws.com/ledejs/resized/allegiant/1024/alle_winglede-2x.jpg 2048w,
      //s3.amazonaws.com/ledejs/resized/allegiant/1500/alle_winglede-2x.jpg 3000w
    ") (sizes . "
      (max-width: 479px) 480px,
      (min-width: 480px) and (max-width: 599px) 600px,
      (min-width: 600px) and (max-width: 1023px) 1024px,
      (min-width: 1024px) 1500px,
      100vw
  "))))

=> "
"

I'm guessing it's the newlines or something that's the problem.  Or the
protocol-less URLs?  I haven't examined.


In GNU Emacs 26.0.50.13 (x86_64-unknown-linux-gnu, GTK+ Version 3.14.5)
 of 2016-11-01 built on stories
Repository revision: c3640fcc96ed80368209c73d7ac9a0f0d1833d93
Windowing system distributor 'The X.Org Foundation', version 11.0.11604000
System Description:	Debian GNU/Linux 8.6 (jessie)

Recent messages:
" [2 times]
Entering debugger...
nil
Quit
" [2 times]
debugger-eval-expression: Symbol’s value as variable is void: base
nil
Back to top level
Quit
Making completion list... [2 times]

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Emacs-Lisp

Minor modes in effect:
  diff-auto-refine-mode: t
  global-whitespace-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t

Load-path shadows:
/home/larsi/src/cddb.el/expect hides /home/larsi/lisp/expect
/home/larsi/src/cddb.el/captitle hides /home/larsi/lisp/captitle
/home/larsi/src/clock.el/clock hides /home/larsi/lisp/clock
~/pgnus/contrib/vcard hides /home/larsi/lisp/vcard
/home/larsi/src/pvr.el/pvr hides /home/larsi/lisp/pvr
~/lisp/zenirc-2.112/src/zenirc-example hides /home/larsi/lisp/zenirc-example
~/pgnus/contrib/compface hides /home/larsi/src/emacs/trunk/lisp/image/compface

Features:
(shadow emacsbug pulse xref project org-element org-rmail org-mhe
org-irc org-info org-gnus org-docview org-bibtex bibtex org-bbdb org-w3m
org org-macro org-footnote org-pcomplete pcomplete org-list org-faces
org-entities noutline outline org-version ob-emacs-lisp ob ob-tangle
ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint comint ring
ob-core ob-eval org-compat org-macs org-loaddefs perl-mode sh-script
smie executable dired-aux jukebox id3 humanely-sort lyric-wiki discogs
json dae musicbrainz scan scrobble tellstick wave cddb captitle expect
vc-annotate thingatpt cal-move cal-menu calendar cal-loaddefs mailalias
smtpmail sendmail eieio-opt speedbar sb-image ezimage dframe find-func
ecomplete misearch multi-isearch log-view pcvs-util vc vc-dispatcher
vc-git pp canlock map diff-mode shr-color color eww flow-fill copyright
vc-cvs gnus-html url-queue help-fns radix-tree url-cache qp sort
gnus-cite smiley ansi-color mm-archive gnus-async gnus-dup gnus-ml gmane
spam-gmane dns mm-url disp-table gnus-fun gnus-mdrtn gnus-topic pop3
nndoc nnmbox utf-7 nnml nnfolder network-stream starttls nnir
spam-report spam spam-stat gnus-uu yenc gnus-delay gnus-draft gnus-agent
gnus-srvr gnus-score score-mode nnvirtual nntp gnus-cache gnus-msg
gnus-art mm-uu mml2015 mm-view mml-smime smime dig gnus-sum nndraft nnmh
gnus-group gnus-undo gnus-start gnus-cloud nnimap utf7 netrc nnoo
parse-time gnus-spec gnus-win nnmail gnus-int gnus-range mail-source
message format-spec rfc822 mml mml-sec epa epg mailabbrev gmm-utils
mailheader gnus nnheader gnus-util rmail rmail-loaddefs mail-utils
whitespace movie mkv shr svg imdb dom pvr debug debbugs-gnu easy-mmode
derived debbugs soap-client mm-decode mm-bodies mm-encode url-http tls
gnutls url-auth mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums
mail-prsvr url-gw nsm puny url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap warnings
rng-xsd rng-dt rng-util xsd-regexp xml ido flyspell ispell benchmark w3m
browse-url doc-view subr-x dired dired-loaddefs image-mode timezone
w3m-hist w3m-fb w3m-ems wid-edit w3m-ccl ccl w3m-favicon w3m-image
w3m-proc w3m-util add-log mail-extr jka-compr cl finder-inf package
epg-config url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache url-vars seq byte-opt gv bytecomp
byte-compile cl-extra help-mode easymenu cconv cl-loaddefs pcase cl-lib
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev
obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face
macroexp files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
dbusbind inotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 7241253 564715)
 (symbols 48 181511 479)
 (miscs 40 2567 3353)
 (strings 32 499022 79531)
 (string-bytes 1 52214498)
 (vectors 16 54970)
 (vector-slots 8 1927348 186360)
 (floats 8 7817 13807)
 (intervals 56 1603036 13249)
 (buffers 976 145)
 (heap 1024 556677 241444))

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24893: 26.0.50; shr doesn't parse srcset <img> correctly

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 263 bytes --]

On Mon, 07 Nov 2016 12:11:51 +0100, Lars Ingebrigtsen wrote:
> (shr--preferred-image
[...]
> => "
> "
> I'm guessing it's the newlines or something that's the problem.  Or the
> protocol-less URLs?  I haven't examined.

Though this is no more than a workaround:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 954 bytes --]

--- shr.el~	2016-11-04 10:32:58.932040100 +0000
+++ shr.el	2016-11-08 05:54:35.912122700 +0000
@@ -1529,7 +1529,7 @@
       (setq srcset
             (sort (mapcar
                    (lambda (elem)
-                     (let ((spec (split-string elem " ")))
+                     (let ((spec (split-string elem "[\t\n ]+")))
                        (cond
                         ((= (length spec) 1)
                          ;; Make sure it's well formed.
@@ -1543,7 +1543,9 @@
                         (t
                          (list (car spec)
                                (string-to-number (cadr spec)))))))
-                   (split-string srcset ", "))
+                   (split-string (replace-regexp-in-string
+				  "\\`[\t\n ]+\\|[\t\n ]+\\'" "" srcset)
+				 "[\t\n ]*,[\t\n ]*"))
                   (lambda (e1 e2)
                     (> (cadr e1) (cadr e2)))))
       ;; Choose the smallest picture that's bigger than the current

bug#24893: 26.0.50; shr doesn't parse srcset <img> correctly

[Andreas Schwab]

On Nov 08 2016, Katsumi Yamaoka <yamaoka@jpl.org> wrote:

> On Mon, 07 Nov 2016 12:11:51 +0100, Lars Ingebrigtsen wrote:
>> (shr--preferred-image
> [...]
>> => "
>> "
>> I'm guessing it's the newlines or something that's the problem.  Or the
>> protocol-less URLs?  I haven't examined.
>
> Though this is no more than a workaround:
>
> --- shr.el~	2016-11-04 10:32:58.932040100 +0000
> +++ shr.el	2016-11-08 05:54:35.912122700 +0000
> @@ -1529,7 +1529,7 @@
>        (setq srcset
>              (sort (mapcar
>                     (lambda (elem)
> -                     (let ((spec (split-string elem " ")))
> +                     (let ((spec (split-string elem "[\t\n ]+")))

Shouldn't this ignore leading and trailing whitespace as well?

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#24893: 26.0.50; shr doesn't parse srcset <img> correctly

[Katsumi Yamaoka]

On Tue, 08 Nov 2016 09:39:46 +0100, Andreas Schwab wrote:
> On Nov 08 2016, Katsumi Yamaoka <yamaoka@jpl.org> wrote:
>> -                     (let ((spec (split-string elem " ")))
>> +                     (let ((spec (split-string elem "[\t\n ]+")))

> Shouldn't this ignore leading and trailing whitespace as well?

It's done by the second part?

-                   (split-string srcset ", "))
+                   (split-string (replace-regexp-in-string
+				  "\\`[\t\n ]+\\|[\t\n ]+\\'" "" srcset)
+				 "[\t\n ]*,[\t\n ]*"))

bug#24893: 26.0.50; shr doesn't parse srcset <img> correctly

[Andreas Schwab]

On Nov 09 2016, Katsumi Yamaoka <yamaoka@jpl.org> wrote:

> On Tue, 08 Nov 2016 09:39:46 +0100, Andreas Schwab wrote:
>> On Nov 08 2016, Katsumi Yamaoka <yamaoka@jpl.org> wrote:
>>> -                     (let ((spec (split-string elem " ")))
>>> +                     (let ((spec (split-string elem "[\t\n ]+")))
>
>> Shouldn't this ignore leading and trailing whitespace as well?
>
> It's done by the second part?
>
> -                   (split-string srcset ", "))
> +                   (split-string (replace-regexp-in-string
> +				  "\\`[\t\n ]+\\|[\t\n ]+\\'" "" srcset)
> +				 "[\t\n ]*,[\t\n ]*"))

Looks like.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#24893: 26.0.50; shr doesn't parse srcset <img> correctly

[Lars Ingebrigtsen]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

>> I'm guessing it's the newlines or something that's the problem.  Or the
>> protocol-less URLs?  I haven't examined.
>
> Though this is no more than a workaround:

I haven't tested the code, but it looks reasonable to me.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#24893: 26.0.50; shr doesn't parse srcset <img> correctly

[Katsumi Yamaoka]

On Thu, 10 Nov 2016 19:09:58 +0100, Lars Ingebrigtsen wrote:
> I haven't tested the code, but it looks reasonable to me.  :-)

Committed.  Thanks.

P.S. I'm trying improving the bug#24831 code so as to render
not only strings but also another invalid table form such as:
<table> ...no tr/td/th...<table>...
Jidanni mailed me an example in which such an unseen table
contains vital links.