bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Eli Zaretskii]

To reproduce:

  emacs -Q
  M-x eww RET https://sourceware.org/gdb/current/onlinedocs/gdb/Connecting.html#Connecting RET

Scroll down to the link that says "see set remote exec-file", with
"set remote exec-file" a hyper-link, and press RET on the link

Result: you get an empty page.
Expected: You should get this page:

  https://sourceware.org/gdb/current/onlinedocs/gdb/Remote-Configuration.html#set%20remote%20exec-file

My crystal ball of a former Texinfo hacker says that the offending
link goes through a short HTML page that just redirects to another
page, and EWW somehow gets confused by that.  But that's just a guess.


In GNU Emacs 25.0.50.206 (i686-pc-mingw32)
 of 2015-12-24
Repository revision: d107eda498f94423d846b0e2f1be7facab417b2a
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
Configured using:
 'configure --prefix=/d/usr --enable-checking=yes,glyphs --with-wide-int
 --with-modules 'CFLAGS=-O0 -gdwarf-4 -g3''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS MODULES

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: eww

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Contacting host: sourceware.org:443
https://sourceware.org/gdb/current/onlinedocs/gdb/set-remote-exec_002dfile.html#set-remote-exec_002dfile
Contacting host: sourceware.org:443

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mailabbrev gmm-utils mailheader sendmail
gnutls network-stream nsm starttls url-http tls mail-parse rfc2231
rfc2047 rfc2045 ietf-drums url-gw url-cache url-auth eww mm-url gnus
gnus-ems nnheader mail-utils wid-edit url-queue url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util url-parse auth-source cl-seq eieio byte-opt bytecomp
byte-compile cl-extra cconv eieio-core cl-macs gv gnus-util mm-util
help-fns help-mode easymenu mail-prsvr password-cache url-vars mailcap
shr dom cl-loaddefs cl-lib subr-x pcase browse-url format-spec time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table w32-win w32-vars
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
w32notify w32 multi-tty make-network-process emacs)

Memory information:
((conses 16 130951 7808)
 (symbols 56 24086 0)
 (miscs 48 40 99)
 (strings 16 27476 5599)
 (string-bytes 1 751769)
 (vectors 16 15676)
 (vector-slots 8 457223 3748)
 (floats 8 206 61)
 (intervals 40 3601 53)
 (buffers 856 12))

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Result: you get an empty page.
> Expected: You should get this page:
>
>   https://sourceware.org/gdb/current/onlinedocs/gdb/Remote-Configuration.html#set%20remote%20exec-file
>
> My crystal ball of a former Texinfo hacker says that the offending
> link goes through a short HTML page that just redirects to another
> page, and EWW somehow gets confused by that.  But that's just a guess.

Yes, the page the link points to just contains this:

<meta http-equiv="refresh" content="0; url=Remote-Configuration.html#set%20remote%20exec%2dfile">

eww doesn't support "refresh" metas, but I guess it could for just the
"0" timeout (i.e., redirect) case...

I mean, it could for other timeouts, too, but I rather think that that
is probably not something we want.  Having Emacs stop randomly and
re-retrieve the page.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 22234@debbugs.gnu.org
> Date: Fri, 25 Dec 2015 17:52:10 +0100
> 
> > My crystal ball of a former Texinfo hacker says that the offending
> > link goes through a short HTML page that just redirects to another
> > page, and EWW somehow gets confused by that.  But that's just a guess.
> 
> Yes, the page the link points to just contains this:
> 
> <meta http-equiv="refresh" content="0; url=Remote-Configuration.html#set%20remote%20exec%2dfile">
> 
> eww doesn't support "refresh" metas, but I guess it could for just the
> "0" timeout (i.e., redirect) case...

All GNU manuals use this technique, so I think we should indeed try to
support that.

> I mean, it could for other timeouts, too, but I rather think that that
> is probably not something we want.  Having Emacs stop randomly and
> re-retrieve the page.

Isn't that also for those "redirecting in NNN seconds" pages?

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> All GNU manuals use this technique, so I think we should indeed try to
> support that.

I've implemented it now...  commit after a bit more testing...

It's a new feature, though, so I'm not sure it should go to Emacs 25.1....

>> I mean, it could for other timeouts, too, but I rather think that that
>> is probably not something we want.  Having Emacs stop randomly and
>> re-retrieve the page.
>
> Isn't that also for those "redirecting in NNN seconds" pages?

When the timeout is other than 0, I'm rendering the <meta> as a link, so
that people can hit it manually.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 22234@debbugs.gnu.org
> Date: Fri, 25 Dec 2015 18:10:46 +0100
> 
> I've implemented it now...  commit after a bit more testing...

Thanks.

> It's a new feature, though, so I'm not sure it should go to Emacs 25.1....

Unless it's risky, please commit to emacs-25.

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I've implemented it now...  commit after a bit more testing...

What stumped me was this:

Remote-Configuration.html#set%20remote%20exec%2dfile

The anchor here is "set remote exec-file", and it didn't go there in the
final page.  But there is no such anchor there!  There's a 

<a name="set-remote-exec_002dfile">

anchor there, though.  Is this a bug on the web pages?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> It's a new feature, though, so I'm not sure it should go to Emacs 25.1....
>
> Unless it's risky, please commit to emacs-25.

It is slightly risky, since it may lead to redirect loops (if I messed
up the redirect counter thing, or if there is some strange interaction
with URL's redirect header parsing).

Hm...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

>> It's a new feature, though, so I'm not sure it should go to Emacs 25.1....
> Unless it's risky, please commit to emacs-25.

emacs-25 is also accepting "refinements", just not brand new features (i.e.,
anything that would justify another entry in the NEWS file).

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 22234@debbugs.gnu.org
> Date: Fri, 25 Dec 2015 18:22:12 +0100
> 
> What stumped me was this:
> 
> Remote-Configuration.html#set%20remote%20exec%2dfile
> 
> The anchor here is "set remote exec-file", and it didn't go there in the
> final page.  But there is no such anchor there!  There's a 
> 
> <a name="set-remote-exec_002dfile">
> 
> anchor there, though.  Is this a bug on the web pages?

I'm not an HTML expert, but aren't you supposed to use the part after
the hash in the original link, not the redirecting one?

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 22234@debbugs.gnu.org
> Date: Fri, 25 Dec 2015 18:24:03 +0100
> 
> > Unless it's risky, please commit to emacs-25.
> 
> It is slightly risky, since it may lead to redirect loops (if I messed
> up the redirect counter thing, or if there is some strange interaction
> with URL's redirect header parsing).

We have enough time till the release to find out if that can happen.

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Cc: 22234@debbugs.gnu.org
>> Date: Fri, 25 Dec 2015 18:22:12 +0100
>> 
>> What stumped me was this:
>> 
>> Remote-Configuration.html#set%20remote%20exec%2dfile
>> 
>> The anchor here is "set remote exec-file", and it didn't go there in the
>> final page.  But there is no such anchor there!  There's a 
>> 
>> <a name="set-remote-exec_002dfile">
>> 
>> anchor there, though.  Is this a bug on the web pages?
>
> I'm not an HTML expert, but aren't you supposed to use the part after
> the hash in the original link, not the redirecting one?

I didn't consider that...

Apparently there is much confusion around the issue.  The spec
originally didn't allow #fragments in HTTP redirects, but then it did:

"With strict reading RFC2616 does not allow fragments in Location header
values, since they are not part of absolute URIs. However, with the
IETF's HTTP rewrite draft this was fixed."

http://stackoverflow.com/questions/1284021/can-i-send-a-http-redirect-to-a-url-with-an-anchor

But this is with HTTP redirects...  I wonder whether HTML <meta>
redirects suffer the same confusion...

I looked at what Firefox did with the page in question, and it, too,
didn't seem to go to the intended anchor.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#22234: 25.0.50; EWW doesn't display some links in GNU manuals

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Cc: 22234@debbugs.gnu.org
>> Date: Fri, 25 Dec 2015 18:24:03 +0100
>> 
>> > Unless it's risky, please commit to emacs-25.
>> 
>> It is slightly risky, since it may lead to redirect loops (if I messed
>> up the redirect counter thing, or if there is some strange interaction
>> with URL's redirect header parsing).
>
> We have enough time till the release to find out if that can happen.

Ok, if Emacs 25.1 isn't happening "soon" soon, then it should be OK.
I'll backport it...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no