bug#38713: 27.0.50; eww doesn't handle protocols other than https?

bug#38713: 27.0.50; eww doesn't handle protocols other than https?

[Juri Linkov]

[-- Attachment #1: Type: text/plain, Size: 334 bytes --]

0. emacs -Q
1. M-x eww RET about:protocols RET hangs with the following line:

Loading http://about:protocols/...

whereas after applying the following patch it displays the nicely
formatted HTML page:

  Supported Protocols - URL vEmacs
   Protocol   Properties   Description
   about   Synchronous  N/A
     Default Port: 0
   ...


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: eww-protocols.patch --]
[-- Type: text/x-diff, Size: 1446 bytes --]

diff --git a/lisp/net/eww.el b/lisp/net/eww.el
index dccaf0497f..64386921c7 100644
--- a/lisp/net/eww.el
+++ b/lisp/net/eww.el
@@ -319,7 +319,8 @@ eww--dwim-expand-url
         (t
 	 ;; Anything that starts with something that vaguely looks
 	 ;; like a protocol designator is interpreted as a full URL.
-         (if (or (string-match "\\`[A-Za-z]+:" url)
+         (unless (string-match-p "\\`about:" url)
+         (if (or (string-match-p "\\`[A-Za-z]+:" url)
 		 ;; Also try to match "naked" URLs like
 		 ;; en.wikipedia.org/wiki/Free software
 		 (string-match "\\`[A-Za-z_]+\\.[A-Za-z._]+/" url)
@@ -335,7 +336,7 @@ eww--dwim-expand-url
                  (setq url (concat url "/"))))
            (setq url (concat eww-search-prefix
                              (mapconcat
-                              #'url-hexify-string (split-string url) "+"))))))
+                              #'url-hexify-string (split-string url) "+")))))))
   url)
 
 (defun eww--preprocess-html (start end)
diff --git a/lisp/url/url-about.el b/lisp/url/url-about.el
index e9d9d6346e..ee79ee1aab 100644
--- a/lisp/url/url-about.el
+++ b/lisp/url/url-about.el
@@ -91,7 +95,7 @@ url-about
     (if (fboundp func)
 	(progn
 	  (set-buffer (generate-new-buffer " *about-data*"))
-	  (insert "Content-type: text/plain\n\n")
+	  (insert "Content-type: text/html\n\n")
 	  (funcall func url)
 	  (current-buffer))
       (error "URL does not know about `%s'" item))))

bug#38713: 27.0.50; eww doesn't handle protocols other than https?

[Lars Ingebrigtsen]

Juri Linkov <juri@linkov.net> writes:

> 0. emacs -Q
> 1. M-x eww RET about:protocols RET hangs with the following line:
>
> Loading http://about:protocols/...
>
> whereas after applying the following patch it displays the nicely
> formatted HTML page:
>
>   Supported Protocols - URL vEmacs
>    Protocol   Properties   Description
>    about   Synchronous  N/A
>      Default Port: 0
>    ...

I'm not sure that makes much sense, really.  Emacs has other ways of
documenting these things other than putting them in an about:
pseudo-protocol thing.  (While other browsers do not.)

eww does not support everything that the url* library does, and
shouldn't.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38713: 27.0.50; eww doesn't handle protocols other than https?

[Juri Linkov]

>> 1. M-x eww RET about:protocols RET hangs with the following line:
>>
>> Loading http://about:protocols/...
>>
>> whereas after applying the following patch it displays the nicely
>> formatted HTML page:
>>
>>   Supported Protocols - URL vEmacs
>>    Protocol   Properties   Description
>>    about   Synchronous  N/A
>>      Default Port: 0
>>    ...
>
> I'm not sure that makes much sense, really.  Emacs has other ways of
> documenting these things other than putting them in an about:
> pseudo-protocol thing.  (While other browsers do not.)

This is part of the planned bigger effort - browsers already support
the 'about' URI scheme, but Emacs is still lagging behind.  I'd like
to expend lisp/url/url-about.el to give information on Emacs internals
including about:memory, about:buffers, about:processes, about:timers and
many more, using HTML instead of semi-obsolete Info or other ad-hoc formats.

Do you think this is worth the effort?

bug#38713: 27.0.50; eww doesn't handle protocols other than https?

[Lars Ingebrigtsen]

Juri Linkov <juri@linkov.net> writes:

> This is part of the planned bigger effort - browsers already support
> the 'about' URI scheme, but Emacs is still lagging behind.  I'd like
> to expend lisp/url/url-about.el to give information on Emacs internals
> including about:memory, about:buffers, about:processes, about:timers and
> many more, using HTML instead of semi-obsolete Info or other ad-hoc formats.
>
> Do you think this is worth the effort?

Reimplementing the Emacs Lisp manual piece-by-piece in eww doesn't sound
like a very promising path, I think?  And for live data on things, `M-x
list-timers' is always going to be better than HTML.

The long-term goal is to have the TeXinfo target HTML instead of Info,
so the lispref manual will end up being HTML.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38713: 27.0.50; eww doesn't handle protocols other than https?

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Date: Tue, 24 Dec 2019 01:12:15 +0200
> Cc: 38713@debbugs.gnu.org
> 
> >> 1. M-x eww RET about:protocols RET hangs with the following line:
> >>
> >> Loading http://about:protocols/...
> >>
> >> whereas after applying the following patch it displays the nicely
> >> formatted HTML page:
> >>
> >>   Supported Protocols - URL vEmacs
> >>    Protocol   Properties   Description
> >>    about   Synchronous  N/A
> >>      Default Port: 0
> >>    ...
> >
> > I'm not sure that makes much sense, really.  Emacs has other ways of
> > documenting these things other than putting them in an about:
> > pseudo-protocol thing.  (While other browsers do not.)
> 
> This is part of the planned bigger effort - browsers already support
> the 'about' URI scheme, but Emacs is still lagging behind.  I'd like
> to expend lisp/url/url-about.el to give information on Emacs internals
> including about:memory, about:buffers, about:processes, about:timers and
> many more, using HTML instead of semi-obsolete Info or other ad-hoc formats.
> 
> Do you think this is worth the effort?

Can you tell more about the plan?  For example, the following
questions seem to be pertinent:

 . why implement this as part of eww? Emacs is not a browser, which
   can only react to protocols, it has other ways of interaction

 . where will all the HTML content come from? will it be taken from
   the existing documentation or written from scratch?

 . eww is not very sophisticated browser, so are you sure displaying
   HTML with it will be significantly better than "semi-obsolete"
   Info?

 . more generally, what will be the advantages of providing
   documentation this way, as opposed to the way we provide it now?

Thanks.

Using eww as browser for Emacs internals and documentation (was: bug#38713: 27.0.50; eww doesn't handle protocols other than https?)

[Juri Linkov]

> Can you tell more about the plan?

Web browsers support the officially registered URI scheme 'about'
to reveal internal state and built-in functions:
https://en.wikipedia.org/wiki/About_URI_scheme

There is one 'about' URI, namely 'about:protocols'
already implemented in Emacs in lisp/url/url-about.el
Then why not to support more 'about' URIs like:

about:memory - with data returned from 'garbage-collect'
about:processes
about:timers
about:buffers
etc.

If it's worth going that path, then would it be nice also
to browse docstrings in eww as well with such URI as e.g.
about:function/cdr

There is one problem that some links might require executing
Lisp code, e.g. to visit C source code.  In this case where
Web browsers execute JavaScript, use Emacs-specific EMACScript:

<a href="emacscript:(find-function 'car)">Click to visit C source code</a>

or

<a href="#" onclick="(find-function-C-source 'car)">C source code</a>

> For example, the following questions seem to be pertinent:
>
>  . why implement this as part of eww? Emacs is not a browser, which
>    can only react to protocols, it has other ways of interaction

Because eww provides basic capabilities present in web browsers such
as rendering the standard HTML format and using hypertext links
for navigation.

OTOH, I think it would be useful only for browsing, not for interaction:
it would not make sense to implement so-called "web apps" to be able to
e.g. mark and delete buffers in the buffer list displayed in eww -
this would be an inappropriate thing to do.

>  . where will all the HTML content come from? will it be taken from
>    the existing documentation or written from scratch?

HTML content will be generated by functions like the existing
'url-about-protocols' in lisp/url/url-about.el.  These functions
might extract data from various data sources, including docstrings.

>  . eww is not very sophisticated browser, so are you sure displaying
>    HTML with it will be significantly better than "semi-obsolete"
>    Info?

Many features of the Info browser are already supported in eww
such as search and next/prev page navigation.  But eww could be
extended further with capabilities of the Info browser.

>  . more generally, what will be the advantages of providing
>    documentation this way, as opposed to the way we provide it now?

Advantages:

- consistent unified UI for browsing Emacs internals and documentation;
- browser-like navigation using links where every resource is represented
  by standardized URI scheme;
- navigation history shared between various resources including Help and Info;
- easy bookmarking of every available resource;
...

Re: Using eww as browser for Emacs internals and documentation

[Lars Ingebrigtsen]

Juri Linkov <juri@linkov.net> writes:

> Web browsers support the officially registered URI scheme 'about'
> to reveal internal state and built-in functions:
> https://en.wikipedia.org/wiki/About_URI_scheme
>
> There is one 'about' URI, namely 'about:protocols'
> already implemented in Emacs in lisp/url/url-about.el
> Then why not to support more 'about' URIs like:
>
> about:memory - with data returned from 'garbage-collect'
> about:processes
> about:timers
> about:buffers
> etc.

Web browsers support this sort of thing because that's a nice ad-hoc way
to display data that you don't want to have in the menus.

I don't see why Emacs should emulate that odd approach to hiding
information, especially since Emacs already has commands to display all
this data in a sensible manner.  Like I said, `M-x list-timers' will
always be better (for the users) than dumping some data into HTML and
letting shr render it.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Using eww as browser for Emacs internals and documentation

[Juri Linkov]

>> Web browsers support the officially registered URI scheme 'about'
>> to reveal internal state and built-in functions:
>> https://en.wikipedia.org/wiki/About_URI_scheme
>>
>> There is one 'about' URI, namely 'about:protocols'
>> already implemented in Emacs in lisp/url/url-about.el
>> Then why not to support more 'about' URIs like:
>>
>> about:memory - with data returned from 'garbage-collect'
>> about:processes
>> about:timers
>> about:buffers
>> etc.
>
> Web browsers support this sort of thing because that's a nice ad-hoc way
> to display data that you don't want to have in the menus.
>
> I don't see why Emacs should emulate that odd approach to hiding
> information, especially since Emacs already has commands to display all
> this data in a sensible manner.  Like I said, `M-x list-timers' will
> always be better (for the users) than dumping some data into HTML and
> letting shr render it.

It would take a lot time and huge effort to implement all these
URI resources.  It is a great relief for me to know this is not
something that would be desirable.  Thanks for saving my time :)

Re: Using eww as browser for Emacs internals and documentation

[Stefan Monnier]

> about:memory - with data returned from 'garbage-collect'
> about:processes
> about:timers
> about:buffers
> etc.

I don't think we should create new HTML-rendered UIs for the above.
But maybe it would be nice to arrange for EWW to somehow redirect
`about:buffers` to `list-buffers`.


        Stefan

Re: Using eww as browser for Emacs internals and documentation

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I don't think we should create new HTML-rendered UIs for the above.
> But maybe it would be nice to arrange for EWW to somehow redirect
> `about:buffers` to `list-buffers`.

I don't really see why anybody would prefer to type `M-x eww RET
about:buffers RET' instead of `M-x list-buffers'.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Using eww as browser for Emacs internals and documentation

[T.V Raman]

Lars Ingebrigtsen <larsi@gnus.org> writes:

1+. Also the proposed "new" approach would require reinventing many
wheels that emacs already has --  and as Lars points out, the already
existing emacs means for doing this are already better.
>
>> Web browsers support the officially registered URI scheme 'about'
>> to reveal internal state and built-in functions:
>> https://en.wikipedia.org/wiki/About_URI_scheme
>>
>> There is one 'about' URI, namely 'about:protocols'
>> already implemented in Emacs in lisp/url/url-about.el
>> Then why not to support more 'about' URIs like:
>>
>> about:memory - with data returned from 'garbage-collect'
>> about:processes
>> about:timers
>> about:buffers
>> etc.
>
> Web browsers support this sort of thing because that's a nice ad-hoc way
> to display data that you don't want to have in the menus.
>
> I don't see why Emacs should emulate that odd approach to hiding
> information, especially since Emacs already has commands to display all
> this data in a sensible manner.  Like I said, `M-x list-timers' will
> always be better (for the users) than dumping some data into HTML and
> letting shr render it.

-- 

Re: Using eww as browser for Emacs internals and documentation

[T.V Raman]

This form of redirection would be much better --though I fail to see the
use cases for it -- perhaps as a means of sharing such info across sessions/users?
-- 

bug#38713: 27.0.50; eww doesn't handle protocols other than https?

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Reimplementing the Emacs Lisp manual piece-by-piece in eww doesn't sound
> like a very promising path, I think?  And for live data on things, `M-x
> list-timers' is always going to be better than HTML.

After the discussion on emacs-devel, it didn't sound like anybody was
very enthusiastic about this idea, so I'm closing the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no