Why Emacs should have a good web-browser

Why Emacs should have a good web-browser

[Fernando]

Hello.
Sorry if this is not the right place, but I wanted to ask some
questions to the community and expose some arguments.

First, I would like to know if you agree about the reasons for having
a web browser in Emacs (either as part of it or as an external lisp
package).

1) One of the main purposes of Emacs is programming. Web development,
css and JavaScript are emerging languages present a lot in the
internet since a long time and now they are even extending to the
desktop (gnome-shell, seed, adium themes...). Emacs has modes and
tools to edit on this languages, but the integration is not as good as
it could be if you had js and html interpreters integrated in Emacs
itself.

2) There are very few web browsers that are comfortable to use in a
keyboard-only interface. Emacs would be very good in this sense
because its keyboard navigation is very usable and as it's designed
for editing text, it will be perfect for all the form editing, comment
writing and all the editing related actions you have to do often in a
browser (editing wikipedia articles, etc).

3) Emacspeak  has turned Emacs into a very accessible environment for
the visual-impaired and it would offer  these people a highly
customizable interface to help them browse the web, along with the
keybindings.

4) Emacs since long time has been one of the greatest tools for an
operative system. During the Age Of The Usenet it was an good
newsreader. Now that the newsgroups have started to die slowly and the
HTTP protocol and Javascript are the kings of the big cloud Emacs
should adapt to it.

5) Browsers are turning into the next generation Emacs! they can
browse ftp, access IRC channels, check your mail, read pdf and other
things with embeded applications, now they can even play video/audio
as a core functionality, they are often used for editing text (web
forms, comments in blogs, etc)... there's even the whole "Google
Chrome OS" designed around a browser. Sooner or later they will be
able to edit code (there's even prototypes for this already) when this
happens Emacs has to compete or it will slowly die. Web browsers are
turning into the main program for the end/power-user in a PC, when
they reach Emacs in functionality I'm sure a lot of people (even Emacs
users) would end up switching to hack Javascript instead of LISP.

6) The special features of Lisp and the extensibility of Emacs make it
be the perfect candidate for an extensible and modular web browser.
Current browsers are tending to improve their extensibility by means
of "plugins" and "extensions". Emacs has since long time a powerful
scripting that a lot of browsers would envy to have.

- Emacs-w3m is not enough and it's not an Emacs module that can be extended.
- Emacs/w3 was a very good idea but soon it has passed more than 1
year since the last single commit to the git repository, it doesn't
look very active at all (am I wrong?).

So.. I just want to know what's the general feeling of the emacs-dev
community about having an emacs web-browser and what expectations
should we have in this regard, is there any other work being done by
anyone? how much is the interest on this?

Not long ago a new (alpha) version of Guile was released that
introduced some basic support for ECMAscript, announcing that there's
a goal to support up to version 3.1 of the spec. Would this make it
possible for a (distant) future Guile Emacs to be able to have an
efficient Javascript-capable web browser?
There also seems that a bit more of work was put on the Guile elisp
compiler lately, although it's still far from being mature.

-- 
Fernando
(sorry for my english)

Re: Why Emacs should have a good web-browser

[Chong Yidong]

Fernando <ferkiwi@gmail.com> writes:

> First, I would like to know if you agree about the reasons for having
> a web browser in Emacs (either as part of it or as an external lisp
> package).

Rendering the modern web is a very complicated task, and I doubt it's
worthwhile to try to implement this independently in Emacs.

An easier route might be to use Gecko or Webkit to embed webpages in
Emacs windows, in the spirit of how we use the GTK library to draw the
tool-bar and scroll-bar.  Emacs might either link directly to
Gecko/Webkit, or use XEmbed to fit a separate mini-browser process
inside an Emacs window.

AFAIK, no one is currently working on anything like this.

Re: Why Emacs should have a good web-browser

[joakim]

Chong Yidong <cyd@stupidchicken.com> writes:

> Fernando <ferkiwi@gmail.com> writes:
>
>> First, I would like to know if you agree about the reasons for having
>> a web browser in Emacs (either as part of it or as an external lisp
>> package).
>
> Rendering the modern web is a very complicated task, and I doubt it's
> worthwhile to try to implement this independently in Emacs.
>
> An easier route might be to use Gecko or Webkit to embed webpages in
> Emacs windows, in the spirit of how we use the GTK library to draw the
> tool-bar and scroll-bar.  Emacs might either link directly to
> Gecko/Webkit, or use XEmbed to fit a separate mini-browser process
> inside an Emacs window.
>
> AFAIK, no one is currently working on anything like this.

I would like to point to:
http://www.emacswiki.org/emacs/EmacsXembed

I have some screenshots there of my xembed patch for emacs that allows
xembedding for instance a video player(mplayer in the screenshot) in emacs.

I am currently making a small xembeddable wrapper on webkit that I will
make some screenshots with to show Emacs embedding a browser.

My original aproach was to use Firefox for embedding, since theres a
nice integration with Emacs called MozRepl, but the mozembed component
crashed when trying embedding.

I need to make small wrappers around apps because most apps doesnt
support to be xembedded out of the box. Emacs and Mplayer are easily
xembeddable without modification.

>
-- 
Joakim Verona

Re: Why Emacs should have a good web-browser

[Lennart Borgman]

On Wed, Jul 8, 2009 at 11:55 PM, <joakim@verona.se> wrote:
> Chong Yidong <cyd@stupidchicken.com> writes:
>
>> Fernando <ferkiwi@gmail.com> writes:
>>
>>> First, I would like to know if you agree about the reasons for having
>>> a web browser in Emacs (either as part of it or as an external lisp
>>> package).
>>
>> Rendering the modern web is a very complicated task, and I doubt it's
>> worthwhile to try to implement this independently in Emacs.
>>
>> An easier route might be to use Gecko or Webkit to embed webpages in
>> Emacs windows, in the spirit of how we use the GTK library to draw the
>> tool-bar and scroll-bar.  Emacs might either link directly to
>> Gecko/Webkit, or use XEmbed to fit a separate mini-browser process
>> inside an Emacs window.
>>
>> AFAIK, no one is currently working on anything like this.
>
> I would like to point to:
> http://www.emacswiki.org/emacs/EmacsXembed
>
> I have some screenshots there of my xembed patch for emacs that allows
> xembedding for instance a video player(mplayer in the screenshot) in emacs.
>
> I am currently making a small xembeddable wrapper on webkit that I will
> make some screenshots with to show Emacs embedding a browser.
>
> My original aproach was to use Firefox for embedding, since theres a
> nice integration with Emacs called MozRepl, but the mozembed component
> crashed when trying embedding.
>
> I need to make small wrappers around apps because most apps doesnt
> support to be xembedded out of the box. Emacs and Mplayer are easily
> xembeddable without modification.

It is probably a very nice piece of job you are doing, but for many
people just controlling Firefox from Emacs would probably be good
enough.

BTW, I have heard that espresso offers even better interaction than
MozRepl but I am not sure. Is there anyone who knows more about this?
Some examples of how to interact from Espresso/MozRepl?

Re: Why Emacs should have a good web-browser

[Daniel Colascione]

On Jul 8, 2009, at 6:42 PM, Lennart Borgman wrote:
> BTW, I have heard that espresso offers even better interaction than
> MozRepl but I am not sure. Is there anyone who knows more about this?
> Some examples of how to interact from Espresso/MozRepl?


Here is an example of some code from espresso (after the rename to js- 
mode). I hope the JS interaction  is self-explanatory, at least for  
the purpose of discussion.

(FWIW, maybe the elisp<->js bridge really should be moved to moz.el?)

(defun js--get-tabs ()
   "Enumerate all the contexts available. Each one is a list:

    The list is (TITLE URL BROWSER TAB TABBROWSER) for content documents
    The list is (TITLE URL WINDOW) for windows

    All tabs of a given window are grouped together. The most
    recent window is first. Within each window, the tabs are
    returned left-to-right.
"
   (with-js
    (let (windows)

      (loop with window-mediator = (js! ("Components" "classes"
                                         "@mozilla.org/appshell/window- 
mediator;1"
                                         "getService")
                                        (js< "Components" "interfaces"
                                             "nsIWindowMediator"))
            with enumerator = (js! (window-mediator "getEnumerator")  
nil)

            while (js? (js! (enumerator "hasMoreElements")))
            for window = (js! (enumerator "getNext"))
            for window-info = (js-list window
                                       (js< window "document" "title")
                                       (js! (window "location"  
"toString"))
                                       (js< window "closed")
                                       (js< window "windowState"))

            unless (or (js? (fourth window-info))
                       (eq (fifth window-info) 2))
            do (push window-info windows))

      (loop for window-info in windows
            for window = (first window-info)
            collect (list (second window-info)
                          (third window-info)
                          window)

            for gbrowser = (js< window "gBrowser")
            if (js-handle? gbrowser)
            nconc (loop
                   for x below (js< gbrowser "browsers" "length")
                   collect (js-list (js< gbrowser
                                         "browsers"
                                         x
                                         "contentDocument"
                                         "title")

                                    (js! (gbrowser
                                          "browsers"
                                          x
                                          "contentWindow"
                                          "location"
                                          "toString"))
                                    (js< gbrowser
                                         "browsers"
                                         x)

                                    (js! (gbrowser
                                          "tabContainer"
                                          "childNodes"
                                          "item")
                                         x)

                                    gbrowser))))))

Re: Why Emacs should have a good web-browser

[Chong Yidong]

joakim@verona.se writes:

> I would like to point to:
> http://www.emacswiki.org/emacs/EmacsXembed
>
> I have some screenshots there of my xembed patch for emacs that allows
> xembedding for instance a video player(mplayer in the screenshot) in emacs.
>
> I am currently making a small xembeddable wrapper on webkit that I will
> make some screenshots with to show Emacs embedding a browser.

That's excellent news.

On a technical note, I think it makes more sense to associate embedded
applications with Emacs windows, rather than buffers as you're
apparently trying to do.  Otherwise, we run into the problem of handling
the situation where the same buffer is displayed in more than one
window.  Basically, we should have a way to say "the contents of this
window are handled by an embedded program, rather than by Emacs".  WDYT?

Re: Why Emacs should have a good web-browser

[Lennart Borgman]

On Thu, Jul 9, 2009 at 12:55 AM, Chong Yidong<cyd@stupidchicken.com> wrote:
> joakim@verona.se writes:
>
>> I would like to point to:
>> http://www.emacswiki.org/emacs/EmacsXembed
>>
>> I have some screenshots there of my xembed patch for emacs that allows
>> xembedding for instance a video player(mplayer in the screenshot) in emacs.
>>
>> I am currently making a small xembeddable wrapper on webkit that I will
>> make some screenshots with to show Emacs embedding a browser.
>
> That's excellent news.
>
> On a technical note, I think it makes more sense to associate embedded
> applications with Emacs windows, rather than buffers as you're
> apparently trying to do.  Otherwise, we run into the problem of handling
> the situation where the same buffer is displayed in more than one
> window.  Basically, we should have a way to say "the contents of this
> window are handled by an embedded program, rather than by Emacs".  WDYT?

You perhaps still need some buffer elements otherwise the embedded
application would go away when you closed a window. That could maybe
be ok, but it is not the way use to behave in Emacs.

Other than that I think it is a good suggestion.

Re: Why Emacs should have a good web-browser

[Davis Herring]

>> On a technical note, I think it makes more sense to associate embedded
>> applications with Emacs windows, rather than buffers as you're
>> apparently trying to do.  Otherwise, we run into the problem of
>> handling
>> the situation where the same buffer is displayed in more than one
>> window.  Basically, we should have a way to say "the contents of this
>> window are handled by an embedded program, rather than by Emacs".
>>  WDYT?
>
> You perhaps still need some buffer elements otherwise the embedded
> application would go away when you closed a window. That could maybe
> be ok, but it is not the way use to behave in Emacs.
>
> Other than that I think it is a good suggestion.

I would say that there should be an "embed object" (sort of like a process
object), and you can do

(set-window-embed window embed)

which fails if EMBED is already associated with some other window.  You
would of course have `window-embed' and `embed-window' which would tell
you that association, so that you could do (set-window-embed (embed-window
embed) nil) to detach it and move it around.

(One could also write it as `set-embed-window', which might make the
single-valuedness of the "embed's window" property clearer.  But I think
it's important to consider it a property of the window because it
overrides everything else that window could ever do.)

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: Why Emacs should have a good web-browser

[joakim]

Chong Yidong <cyd@stupidchicken.com> writes:

> joakim@verona.se writes:
>
>> I would like to point to:
>> http://www.emacswiki.org/emacs/EmacsXembed>
>> I have some screenshots there of my xembed patch for emacs that allows
>> xembedding for instance a video player(mplayer in the screenshot) in emacs.
>>
>> I am currently making a small xembeddable wrapper on webkit that I will
>> make some screenshots with to show Emacs embedding a browser.
>
> That's excellent news.
>
> On a technical note, I think it makes more sense to associate embedded
> applications with Emacs windows, rather than buffers as you're
> apparently trying to do.  Otherwise, we run into the problem of handling
> the situation where the same buffer is displayed in more than one
> window.  Basically, we should have a way to say "the contents of this
> window are handled by an embedded program, rather than by Emacs".  WDYT?

One aproach does not exclude the other.

I modelled my xembed support on the way images are handled in Emacs.

The use-case im really trying to achieve is having any sort of gtk
controller embedded in a buffer. It kind of works already. Embedding an
xembeddable browser, was just a special case of embedding the gtk socket
component.

I want this so I can make guis for controlling multimedia stuff in a
hybrid emacs/traditional gui way.

Personally I dont think embedding a browser in emacs is the way to go
even though it makes for a fun demo. I think the emacs-w3m aproach is
better. I'm also interested in making a similar aproach that would use
mozilla-headless to render the web page in memory, and read the DOM
through something MozRepl like into Emacs buffers.


-- 
Joakim Verona

Re: Why Emacs should have a good web-browser

[Jason Rumney]

Chong Yidong wrote:
> That's excellent news.
>
> On a technical note, I think it makes more sense to associate embedded
> applications with Emacs windows, rather than buffers as you're
> apparently trying to do.  Otherwise, we run into the problem of handling
> the situation where the same buffer is displayed in more than one
> window.  Basically, we should have a way to say "the contents of this
> window are handled by an embedded program, rather than by Emacs".  WDYT?
>   
It's not that difficult to handle, you just need to take snapshots of 
the application's display now and then.  The active window contains the 
real application, and any other windows can contain a snapshot of the 
display of that program.

Re: Why Emacs should have a good web-browser

[joakim]

Jason Rumney <jasonr@gnu.org> writes:

> Chong Yidong wrote:
>> That's excellent news.
>>
>> On a technical note, I think it makes more sense to associate embedded
>> applications with Emacs windows, rather than buffers as you're
>> apparently trying to do.  Otherwise, we run into the problem of handling
>> the situation where the same buffer is displayed in more than one
>> window.  Basically, we should have a way to say "the contents of this
>> window are handled by an embedded program, rather than by Emacs".  WDYT?
>>   
> It's not that difficult to handle, you just need to take snapshots of
> the application's display now and then.  The active window contains
> the real application, and any other windows can contain a snapshot of
> the display of that program.

This is basically what I'm aiming for with my xwidget patch. The active
window contains the live component, the other windows contain inactive
snapshots of the components(currently only grey rectangles in the
inactive windows).

Still, if the list finds the possibility of defering drawing of an
entire emacs window to an xembeddable component, I could try to
implement that on top of my current patch. I would need help on the
design of the interface, that is, where should the properties governing
this go, and what should their names and functions be. I think this
would be useful also, and it could go into the emacs tree way faster
than my current slow-moving xwidget patch.



-- 
Joakim Verona

Re: Why Emacs should have a good web-browser

[Chong Yidong]

joakim@verona.se writes:

>> It's not that difficult to handle, you just need to take snapshots of
>> the application's display now and then.  The active window contains
>> the real application, and any other windows can contain a snapshot of
>> the display of that program.
>
> This is basically what I'm aiming for with my xwidget patch. The active
> window contains the live component, the other windows contain inactive
> snapshots of the components(currently only grey rectangles in the
> inactive windows).

Is this "snapshot" handled automatically by X (e.g., as an XEmbed
feature), or is it something that would have to be done "by hand",
e.g. by copying one X bitmap buffer into another?  If the latter,
doesn't that mean that clicking on a button in the "copied" application
wouldn't do anything?

Re: Why Emacs should have a good web-browser

[joakim]

Chong Yidong <cyd@stupidchicken.com> writes:

> joakim@verona.se writes:
>
>>> It's not that difficult to handle, you just need to take snapshots of
>>> the application's display now and then.  The active window contains
>>> the real application, and any other windows can contain a snapshot of
>>> the display of that program.
>>
>> This is basically what I'm aiming for with my xwidget patch. The active
>> window contains the live component, the other windows contain inactive
>> snapshots of the components(currently only grey rectangles in the
>> inactive windows).
>
> Is this "snapshot" handled automatically by X (e.g., as an XEmbed
> feature), or is it something that would have to be done "by hand",
> e.g. by copying one X bitmap buffer into another?  If the latter,
> doesn't that mean that clicking on a button in the "copied" application
> wouldn't do anything?

The copying needs to be done manualy yes. However, the xembedded
component is moved to the active window. When clicking on a component in
a non-active window, that window becomes active, so the component is
moved there, and the previously active window gets a copied
component(grey rectangle) instead.


-- 
Joakim Verona

Re: Why Emacs should have a good web-browser

[Richard Stallman]

    On a technical note, I think it makes more sense to associate embedded
    applications with Emacs windows, rather than buffers as you're
    apparently trying to do.  Otherwise, we run into the problem of handling
    the situation where the same buffer is displayed in more than one
    window.  Basically, we should have a way to say "the contents of this
    window are handled by an embedded program, rather than by Emacs".

I see what you mean.  In a way, that would be more logical.

But it is hard to fit that into the rest of the way Emacs handles
windows: as spaces within a frame, whose contents are controlled by
the buffer specified for display in the window.  To control a window's
contents in some other way would require a conceptual redesign.

It might be much simpler to avoid that redesign and instead represent
the embedded whatever through a buffer that has a name and can be
displayed in a window.  In addition, this would give a way to ask to
show it, or to replace it with something else: all the usual commands
for switching buffers.

Re: Why Emacs should have a good web-browser

[Richard Riley]

Fernando <ferkiwi@gmail.com> writes:

> Hello.
> Sorry if this is not the right place, but I wanted to ask some
> questions to the community and expose some arguments.
>
> First, I would like to know if you agree about the reasons for having
> a web browser in Emacs (either as part of it or as an external lisp
> package).
>
> 1) One of the main purposes of Emacs is programming. Web development,
> css and JavaScript are emerging languages present a lot in the
> internet since a long time and now they are even extending to the
> desktop (gnome-shell, seed, adium themes...). Emacs has modes and
> tools to edit on this languages, but the integration is not as good as
> it could be if you had js and html interpreters integrated in Emacs
> itself.
>
> 2) There are very few web browsers that are comfortable to use in a
> keyboard-only interface. Emacs would be very good in this sense
> because its keyboard navigation is very usable and as it's designed
> for editing text, it will be perfect for all the form editing, comment
> writing and all the editing related actions you have to do often in a
> browser (editing wikipedia articles, etc).
>
> 3) Emacspeak  has turned Emacs into a very accessible environment for
> the visual-impaired and it would offer  these people a highly
> customizable interface to help them browse the web, along with the
> keybindings.
>
> 4) Emacs since long time has been one of the greatest tools for an
> operative system. During the Age Of The Usenet it was an good
> newsreader. Now that the newsgroups have started to die slowly and the
> HTTP protocol and Javascript are the kings of the big cloud Emacs
> should adapt to it.
>
> 5) Browsers are turning into the next generation Emacs! they can
> browse ftp, access IRC channels, check your mail, read pdf and other
> things with embeded applications, now they can even play video/audio
> as a core functionality, they are often used for editing text (web
> forms, comments in blogs, etc)... there's even the whole "Google
> Chrome OS" designed around a browser. Sooner or later they will be
> able to edit code (there's even prototypes for this already) when this
> happens Emacs has to compete or it will slowly die. Web browsers are
> turning into the main program for the end/power-user in a PC, when
> they reach Emacs in functionality I'm sure a lot of people (even Emacs
> users) would end up switching to hack Javascript instead of LISP.

You might be interested in looking at controlling/interacting with
Firefox using emacs. Google has more info. I use this below to "refresh"
my firefox when I save local web code. I dont know how well polished it
is. I guess there is a huge load you can do wit whatever that script
language is.


,----
| 
| (autoload 'moz-minor-mode "moz" "Mozilla Minor and Inferior Mozilla Modes" t)
|     (add-hook 'espresso-mode-hook 'espresso-custom-setup)
| 
| (require 'moz)
| 
| (defun espresso-custom-setup ()
|   (moz-minor-mode 1))
| 
| (defun auto-reload-firefox-on-after-save-hook ()         
|   (add-hook 'after-save-hook
| 	    '(lambda ()
| 	       (interactive)
| 	       (comint-send-string (inferior-moz-process)
| 				   "setTimeout(BrowserReload(), \"1000\");"))
| 	    'append 'local)) ; buffer-local
| 
| ;; Example - you may want to add hooks for your own modes.
| ;; I also add this to python-mode when doing django development.
| (add-hook 'nxhtml-mode-hook 'auto-reload-firefox-on-after-save-hook)
| (add-hook 'php-mode-hook 'auto-reload-firefox-on-after-save-hook)
| (add-hook 'html-mode-hook 'auto-reload-firefox-on-after-save-hook)
| (add-hook 'css-mode-hook 'auto-reload-firefox-on-after-save-hook)
`----

Re: Why Emacs should have a good web-browser

[Paul R]

Hi,

Fernando> First, I would like to know if you agree about the reasons for
Fernando> having a web browser in Emacs (either as part of it or as an
Fernando> external lisp package).

As Chong pointed out, rendering today web is a full-time-full-team job.

You might find uzbl[1] of interest. It is basically a frame rendering
the html/css+js, and that's it. Control, bookmarks, history ... well,
everything else than rendering is let to third party tools. IOW, very
suitable for an uzbl-mode.

[1] http://www.uzbl.org/

-- 
  Paul

Re: Why Emacs should have a good web-browser

[Stefan Monnier]

> First, I would like to know if you agree about the reasons for having
> a web browser in Emacs (either as part of it or as an external lisp
> package).

I'd like to see Emacs support things like javascript better, as well as
web-browsing.  The Emacs/W3 way is sadly unworkable because of the
amount of effort this requires (compounded by the performance
limitations of Elisp).  Embedding a whole web-browser like Firefox might
be a good direction, tho it's probably too coarse to be satisfactory.
E.g. I'd like to be able to use Emacs's own completion code when typing
the URL or when filling any text in the page.

Maybe the most promising direction I can see is to try and make some
library (like webkit) render directly into an Emacs buffer (rather than
X11/Cairo/w32/plaintext/younameit).  Still, to make it handle tables
properly, we may need to extend Emacs's rendering engine to better
support them (which would be handy in any case for various applications
like list-buffer which should columns of data).


        Stefan

Re: Why Emacs should have a good web-browser

[Robert D. Crawford]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Maybe the most promising direction I can see is to try and make some
> library (like webkit) render directly into an Emacs buffer (rather than
> X11/Cairo/w32/plaintext/younameit).  

From a personal standpoint, this is the best idea I've seen in this
thread.  I'm a daily user of emacs/w3 and an more than occasional user
of emacs-w3m.  Being partially sighted and reliant on emacspeak, I
_really_ like the idea of having text rendered into a buffer where it
acts just like any other buffer.

rdc
-- 
Robert D. Crawford                                      rdc1x@comcast.net

Re: Why Emacs should have a good web-browser

[Ken Raeburn]

On Jul 11, 2009, at 16:24, Stefan Monnier wrote:
>  The Emacs/W3 way is sadly unworkable because of the
> amount of effort this requires (compounded by the performance
> limitations of Elisp).

Guile has had some performance work done lately, and has a bytecode  
engine of its own; no JIT compilation to machine code yet, but people  
are certainly keeping it in mind.  I don't know how the ECMAScript  
performance compares to a random browser's Javascript engine  
currently, and (according to http://wingolog.org/archives/2009/02/22/ecmascript-for-guile 
  though maybe things have changed) I guess Andy Wingo doesn't  
either.  But the ECMAScript support is a second language reader for  
the compiler, so when the Scheme compilation and optimization support  
improves, so should the ECMAScript support.

Ken

Re: Why Emacs should have a good web-browser

[Paul R]

Stefan> Maybe the most promising direction I can see is to try and make
Stefan> some library (like webkit) render directly into an Emacs buffer
Stefan> (rather than X11/Cairo/w32/plaintext/younameit). Still, to make
Stefan> it handle tables properly, we may need to extend Emacs's
Stefan> rendering engine to better support them

How about doing that the other way, that is rendering emacs buffers in
webkit (or an other xhtml/css+js lib), making it the default rendering
backend for emacs in graphic mode. It would probably cut off some large
amount of emacs plateform-specific code and make web integration tight.

-- 
  Paul

Re: Why Emacs should have a good web-browser

[Miles Bader]

Paul R <paul.r.ml@gmail.com> writes:
> How about doing that the other way, that is rendering emacs buffers in
> webkit (or an other xhtml/css+js lib), making it the default rendering
> backend for emacs in graphic mode. It would probably cut off some large
> amount of emacs plateform-specific code and make web integration tight.

Please give more details about what you mean.  Emacs' display engine is
not simple, and much code relies on its (many) features.  It would seem
quit difficult to make it use a high-level display toolkit with very
different abstractions.

-Miles

-- 
/\ /\
(^.^)
(")")
*This is the cute kitty virus, please copy this into your sig so it can spread.

Re: Why Emacs should have a good web-browser

[Paul R]

Hi Miles,

Miles> Please give more details about what you mean. Emacs' display
Miles> engine is not simple, and much code relies on its (many)
Miles> features. It would seem quit difficult to make it use
Miles> a high-level display toolkit with very different abstractions.

This is a very naïve suggestion, as I dont know much of emacs display
engine. I just notice that emacs wants to display formated text, with
hyperlinks, colors, pictures, antialiasing, etc. All of that (and much
more, like complex layouts, tables, etc) is already very well adressed
in an xhtml/css+js engine such as gecko or webkit, both free software
and available for integration in third party applications.

-- 
  Paul

Re: Why Emacs should have a good web-browser

[Stefan Monnier]

Miles> Please give more details about what you mean. Emacs' display
Miles> engine is not simple, and much code relies on its (many)
Miles> features. It would seem quit difficult to make it use
Miles> a high-level display toolkit with very different abstractions.

> This is a very naïve suggestion, as I dont know much of emacs display
> engine. I just notice that emacs wants to display formated text, with
> hyperlinks, colors, pictures, antialiasing, etc. All of that (and much
> more, like complex layouts, tables, etc) is already very well adressed
> in an xhtml/css+js engine such as gecko or webkit, both free software
> and available for integration in third party applications.

It might be possible to use one of those engines as Emacs's rendering
engine, indeed.  To me, it wouldn't seem like an good solution to the
problem at hand because I don't think it would allow me to control the
web-browser from Emacs (e.g., how would I access from Elisp the content
of pages generated from HTML?).  So it'd be more like embedding Emacs
inside a normal browser.  It's not a bad idea, but I don't think it'll
provide as many benefits from Emacs's point of view.


        Stefan

Re: Why Emacs should have a good web-browser

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> This is a very naïve suggestion, as I dont know much of emacs display
>> engine. I just notice that emacs wants to display formated text, with
>> hyperlinks, colors, pictures, antialiasing, etc. All of that (and much
>> more, like complex layouts, tables, etc) is already very well adressed
>> in an xhtml/css+js engine such as gecko or webkit, both free software
>> and available for integration in third party applications.
>
> It might be possible to use one of those engines as Emacs's rendering
> engine, indeed.  To me, it wouldn't seem like an good solution to the
> problem at hand because I don't think it would allow me to control the
> web-browser from Emacs (e.g., how would I access from Elisp the content
> of pages generated from HTML?).  So it'd be more like embedding Emacs
> inside a normal browser.  It's not a bad idea, but I don't think it'll
> provide as many benefits from Emacs's point of view.

If Emacs used the "high level" rendering component, I dunno how well it
would work, as an Emacs buffer is pretty different than a web page.

Most obviously, it's dynamic, and changing it is expected to be very
cheap even if the buffer is huge.

Another issue might be that, to the best of my knowledge, html rendering
engines tend to generate a "rendered" representation of the entire page,
no matter how little of it is displayed (as opposed to Emacs, which does
the most expensive processing mostly only the parts of a buffer that are
displayed).  This has obvious benefits (e.g., scrolling around after the
long initial setup can be fast, and your scrollbar can easily show
physical display units), but has obvious problems too:  displaying a 1GB
file might take a l.o.n.g time to show the first page....

I don't know how well these engines deal with the underlying text
changing; given that a small text change might affect the _entire_
"rendered" data structure, there seems a good chance the answer might be
"not very well."

But all of this is mostly speculation on my part...

-Miles

-- 
Bore, n. A person who talks when you wish him to listen.

Re: Why Emacs should have a good web-browser

[Thomas Lord]

On Wed, 2009-07-22 at 01:31 +0900, Miles Bader wrote:

> I don't know how well these engines deal with the underlying text
> changing; given that a small text change might affect the _entire_
> "rendered" data structure, there seems a good chance the answer might be
> "not very well."
> 
> But all of this is mostly speculation on my part...


A modern browser rendering engine presents (e.g.
to Javascript programs) a "DOM" (data object model)
interface to the contents of a page.   The DOM
is a tree structure that functions as a hierarchical
display list and window system windows are automatically
updated from that display list and from changes made
to it.   All of the mechanism of CSS is available to 
control the rendering of that display list.

The natural way to use this for an Emacs display
is *not* to try to store all of a buffer's text 
in the DOM but rather to use the DOM for only the 
visible content of an Emacs frame.   For example,
the scrollbars on a typical browser window would not
be used here:  Emacs would have to implement its own
style of scrollbars as DOM elements.

In terms of performance:  modern browser rendering
kits are fast enough -- although probably only "barely"
so.  They'll get better (through code improvement,
not faster hardware).     People have been building
GUI toolkits that use the DOM as their display engine
for several years now and each year the results 
become more and more impressive.

(A question nagging me, lately:  how good would
the performance be of an implementation of Elisp
in Javascript....  The result would be a second
implementation of GNU Emacs, not a replacement for
the GNU Emacs we have.).

-t

Re: Why Emacs should have a good web-browser

[Paul R]

On Wed, 22 Jul 2009 01:31:23 +0900, Miles Bader <miles@gnu.org> said:

> Most obviously, it's dynamic, and changing it is expected to be very
> cheap even if the buffer is huge.

As some others pointed out, you can easily access to a branch of the DOM
tree without touching the rest, pretty much like with Emacs. If you want
to play with that, install Firefox and its "firebug" extension. Within
minute you will be able to live-change parts of your web pages, both the
content and the style.

> Another issue might be that, to the best of my knowledge, html
> rendering engines tend to generate a "rendered" representation of the
> entire page, no matter how little of it is displayed (as opposed to
> Emacs, which does the most expensive processing mostly only the parts
> of a buffer that are displayed). This has obvious benefits (e.g.,
> scrolling around after the long initial setup can be fast, and your
> scrollbar can easily show physical display units), but has obvious
> problems too: displaying a 1GB file might take a l.o.n.g time to show
> the first page....

Emacs could easily use the big-file-in-many-chunks strategy, the
rendering engine would only be a rendering engine, i.e. no direct access
to file content.

> I don't know how well these engines deal with the underlying text
> changing; given that a small text change might affect the _entire_
> "rendered" data structure, there seems a good chance the answer might
> be "not very well."

Live-editing parts of a big wikipedia article with Firebug is
instantaneous on my good old laptop. The only cascading effect I can
think of is the visual filling, but now emacs does it as well so I'm not
sure it would be worst with a xhtml display engine.

-- 
  Paul

Re: Why Emacs should have a good web-browser

[David Reitter]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

On Jul 21, 2009, at 4:52 PM, Stefan Monnier wrote:
> It might be possible to use one of those engines as Emacs's rendering
> engine, indeed.  To me, it wouldn't seem like an good solution to the
> problem at hand because I don't think it would allow me to control the
> web-browser from Emacs (e.g., how would I access from Elisp the  
> content
> of pages generated from HTML?).

JS / DOM has worked this out quite well, and the implementations  
provide a security model.

>  So it'd be more like embedding Emacs
> inside a normal browser.  It's not a bad idea, but I don't think it'll
> provide as many benefits from Emacs's point of view.

It would be fantastic to be able to use Emacs directly inside text  
areas of Firefox.  There's a simple plugin that lets users edit text  
from text boxes in an external text editor, but that's nowhere nearly  
as nice as having a little Emacs window displayed right in place of  
the text box.

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 2193 bytes --]

Re: Why Emacs should have a good web-browser

[Chong Yidong]

David Reitter <david.reitter@gmail.com> writes:

> It would be fantastic to be able to use Emacs directly inside text
> areas of Firefox.  There's a simple plugin that lets users edit text
> from text boxes in an external text editor, but that's nowhere nearly
> as nice as having a little Emacs window displayed right in place of
> the text box.

This should be doable on X, with the existing XEmbed support.

Re: Why Emacs should have a good web-browser

[Thomas Lord]

On Tue, 2009-07-21 at 11:52 -0400, Stefan Monnier wrote:

> > This is a very naïve suggestion, as I dont know much of emacs display
> > engine. I just notice that emacs wants to display formated text, with
> > hyperlinks, colors, pictures, antialiasing, etc. All of that (and much
> > more, like complex layouts, tables, etc) is already very well adressed
> > in an xhtml/css+js engine such as gecko or webkit, both free software
> > and available for integration in third party applications.
> 
> It might be possible to use one of those engines as Emacs's rendering
> engine, indeed.  To me, it wouldn't seem like an good solution to the
> problem at hand because I don't think it would allow me to control the
> web-browser from Emacs (e.g., how would I access from Elisp the content
> of pages generated from HTML?).  So it'd be more like embedding Emacs
> inside a normal browser.  It's not a bad idea, but I don't think it'll
> provide as many benefits from Emacs's point of view.

It might make an interesting experiment for someone
who has the time and inclination to try writing an 
Elisp interpreter and Emacs primitives in Javascript.

-t

Re: Why Emacs should have a good web-browser

[Adam Wołk]

Dnia 21-07-2009 o 19:13:47 Thomas Lord <lord@emf.net> napisał(a):

> On Tue, 2009-07-21 at 11:52 -0400, Stefan Monnier wrote:

>> It might be possible to use one of those engines as Emacs's rendering
>> engine, indeed.  To me, it wouldn't seem like an good solution to the
>> problem at hand because I don't think it would allow me to control the
>> web-browser from Emacs (e.g., how would I access from Elisp the content
>> of pages generated from HTML?).  So it'd be more like embedding Emacs
>> inside a normal browser.  It's not a bad idea, but I don't think it'll
>> provide as many benefits from Emacs's point of view.
>

There already exists a browser that tries to implement things the Emacs  
way. It's called conkeror.

 From their website (http://conkeror.org):
"Conkeror is a keyboard-oriented, highly-customizable, highly-extensible  
web browser based on Mozilla XULRunner, written mainly in JavaScript, and  
inspired by exceptional software such as Emacs and vi. Conkeror features a  
sophisticated keyboard system, allowing users to run commands and interact  
with content in powerful and novel ways. It is self-documenting, featuring  
a powerful interactive help system."

Benefits of helping out with this project:
* Conkeror can be fully controlled from withing emacs using mozrepl
* Doesn't have problems with most pages that Firefox can handle
* Can use existing Firefox extensions (Adblock etc.)
* No problems with websites blocking less popular clients (a very common  
problem even for browsers like Opera)
* Shares many concepts with Emacs

Instead of implementing a full blown browser inside of Emacs, which by  
itself is a massive complicated task both from a usability point of view  
and security. Maybe we should focus on improving the interoperability  
between Emacs and conkeror which already exists and is available.

>
> It might make an interesting experiment for someone
> who has the time and inclination to try writing an
> Elisp interpreter and Emacs primitives in Javascript.
>

As far as I understand, they are not implementing an elisp interpreter but  
javascript is used the same way to provide functionality to the browser.  
Which by itself is quite interesting considering how far they have gone.

Re: Why Emacs should have a good web-browser

[Lennart Borgman]

On Tue, Jul 21, 2009 at 8:21 PM, Adam Wołk<netprobe@gmail.com> wrote:

> Benefits of helping out with this project:
> * Conkeror can be fully controlled from withing emacs using mozrepl

Please explain in what way it is better to use Conkeror than Firefox for this.

Re: Why Emacs should have a good web-browser

[Adam Wołk]

Dnia 21-07-2009 o 21:01:21 Lennart Borgman <lennart.borgman@gmail.com>  
napisał(a):

> On Tue, Jul 21, 2009 at 8:21 PM, Adam Wołk<netprobe@gmail.com> wrote:
>
>> Benefits of helping out with this project:
>> * Conkeror can be fully controlled from withing emacs using mozrepl
>
> Please explain in what way it is better to use Conkeror than Firefox for  
> this.

If You refer to mozrepl and scripting only. Then I would assume that  
concepts taken from Emacs and implemented in conkeror will make it a more  
natural platform to extend for current Emacs users. Unfortunately this is  
a wild guess on my part as I really haven't tried to code for either  
platform (conkeror and Firefox).



Some other things that came to my mind:

Conkeror has implemented massive amounts of functionality in ways that  
Emacs users expect.
For example I quickly checked the following set of common tasks with basic  
text edition:
* C-n, C-p, C-f, C-b, M-f, M-b, C-t, M-t, C-a, C-e, C-v, M-v
* C-s, C-r
* C-w,M-w,C-y,C-k
* Marks and regions are used for operations (C-SPACE, C-x C-x)

All of them work as expected. This makes it easier and more natural to tab  
between both applications (Emacs and conkeror).

The interface is styled after Emacs and is designed to be fully keyboard  
driven.
The browser works with 'buffers' while firefox still defaults to tabs.
It's actively developed and mostly by people who already are Emacs users.
Do to the nature of the project it should be easier to have specific  
features accepted in the upstream then with Firefox.
Consider sending a patch for Firefox to improve Emacs compatibility - I  
don't think it would concern most developers.

Of course this could be done as an extensions to Firefox. In fact conkeror  
started that way and they decided to go all the way down.

I believe that having a good default and supported browser that integrates  
well with Emacs would be great.

Re: Why Emacs should have a good web-browser

[Lennart Borgman]

2009/7/21 Adam Wołk <netprobe@gmail.com>:
> Dnia 21-07-2009 o 21:01:21 Lennart Borgman <lennart.borgman@gmail.com>
> napisał(a):
>
>> On Tue, Jul 21, 2009 at 8:21 PM, Adam Wołk<netprobe@gmail.com> wrote:
>>
>>> Benefits of helping out with this project:
>>> * Conkeror can be fully controlled from withing emacs using mozrepl
>>
>> Please explain in what way it is better to use Conkeror than Firefox for
>> this.
>
> If You refer to mozrepl and scripting only. Then I would assume that
> concepts taken from Emacs and implemented in conkeror will make it a more
> natural platform to extend for current Emacs users. Unfortunately this is a
> wild guess on my part as I really haven't tried to code for either platform
> (conkeror and Firefox).

Thanks for explaining.


> Some other things that came to my mind:
>
> Conkeror has implemented massive amounts of functionality in ways that Emacs
> users expect.
> For example I quickly checked the following set of common tasks with basic
> text edition:
> * C-n, C-p, C-f, C-b, M-f, M-b, C-t, M-t, C-a, C-e, C-v, M-v
> * C-s, C-r
> * C-w,M-w,C-y,C-k
> * Marks and regions are used for operations (C-SPACE, C-x C-x)
>
> All of them work as expected. This makes it easier and more natural to tab
> between both applications (Emacs and conkeror).


There are some users of Emacs (like me) that never use these key
bindings since we prefer Viper. So we would really want a combination
of vi+emacs keybindings in other applications too.

Re: Why Emacs should have a good web-browser

[Adam Wołk]

Dnia 21-07-2009 o 21:33:33 Lennart Borgman <lennart.borgman@gmail.com>  
napisał(a):


> There are some users of Emacs (like me) that never use these key
> bindings since we prefer Viper. So we would really want a combination
> of vi+emacs keybindings in other applications too.

I saw this question recently on the conkeror mailing list.
It seems that there are no ready to use bindings or modes for people who  
prefer vim key bindings but I believe that this could be implemented (at  
least partially?).

Below are some quotes from the conkeror mailing list archives:

quote from http://article.gmane.org/gmane.comp.mozilla.conkeror/1580 :
> Some vi-like search bindings:
>
>   define_key(content_buffer_normal_keymap, "/", "isearch-forward");
>   define_key(content_buffer_normal_keymap, "?", "isearch-backward");
>   define_key(content_buffer_normal_keymap, "n",  
> "isearch-continue-forward");
>   define_key(content_buffer_normal_keymap, "N",  
> "isearch-continue-backward");
>
> The commands isearch-continue-[forward/backward] are bound on S and R
> by default, and you may just want to use those unless you are planning
> to make a whole new binding set.
>
>> open link in new buffer with F instead of C-u f
>
>   define_key(content_buffer_normal_keymap, "F", "follow-new-buffer");


quote from http://article.gmane.org/gmane.comp.mozilla.conkeror/1587 :
>> Thanks for the tips. I understand that conkeror could be made to behave  
>> like
>> vimperator by changing keybindings. What I actually wanted to know is  
>> if it
>> is possible to make conkeror "completely" behave like Vimperator "only"  
>> by
>> changing the keybindings or does one need to do more than that (for  
>> example,
>> some deeper hacking)?
>
> Changing key bindings will likely get you much of the way to vimperator,
> but it will indeed not get you an exact clone of vimperator.  For one
> thing, not all UI decisions are represented as key binding choices, and
> for another, there are some features in vimperator not supported in
> Conkeror (though also some things in Conkeror not supported in
> Vimperator).
>
> I would think that changing the key bindings would be mostly sufficient,
> though.

Re: Why Emacs should have a good web-browser

[Robert D. Crawford]

Adam Wołk <netprobe@gmail.com> writes:

> I believe that having a good default and supported browser that
> integrates well with Emacs would be great.

Correct me if I am wrong, but this does not sound like "default" but
more like "de facto," the difference being that we are talking about
separate applications that run independently of each other.  

I will not pretend to understand everything that is being discussed.  I
am interested as a blind user, dependent on w3 and or emacs-w3m.  So
far, I don't see how I will be able to use emacspeak to speak the
contents of a conqueror buffer like I can any other emacs buffer.  Am I
mistaken?  

rdc
-- 
Robert D. Crawford                                      rdc1x@comcast.net

Re: Why Emacs should have a good web-browser

[Lennart Borgman]

On Tue, Jul 21, 2009 at 10:02 PM, Robert D. Crawford<rdc1x@comcast.net> wrote:

> I will not pretend to understand everything that is being discussed.  I
> am interested as a blind user, dependent on w3 and or emacs-w3m.  So
> far, I don't see how I will be able to use emacspeak to speak the
> contents of a conqueror buffer like I can any other emacs buffer.  Am I
> mistaken?

Hi Robert,

Is emacspeak in any way dependent on w3 or could it do equally well if
emacs-w3m could use another web browser to do what it does now?

Re: Why Emacs should have a good web-browser

[Robert D. Crawford]

Lennart Borgman <lennart.borgman@gmail.com> writes:

> Is emacspeak in any way dependent on w3 or could it do equally well if
> emacs-w3m could use another web browser to do what it does now?

emacspeak is not dependent on any particular browser, it is just
dependent on emacs.  I can use w3 or w3m... both work equally well as
far as getting the content rendered to a buffer so that it can be
spoken.  The differences are mainly in what can be done in each browser
e.g. in w3 you can navigate tables cell-by-cell and there are some
limitations imposed by the underlying w3m that mean it cannot render
certain tags in different faces.

So, yes, if emacs-w3m used a different backend it should work equally
well.  I think the main point is that the web page gets rendered as
_text_, marked up in different faces to denote various tags and
presented just as any other buffer.  One of the nice things about this
sort of set-up is that transformations can be made to the text before it
is rendered.  A good example is transforming table based layouts so that
each cell becomes a paragraph, making it possible to read in a linear
fashion.

Thanks,
rdc
-- 
Robert D. Crawford                                      rdc1x@comcast.net

Re: Why Emacs should have a good web-browser

[Adam Wołk]

Dnia 21-07-2009 o 22:02:28 Robert D. Crawford <rdc1x@comcast.net>  
napisał(a):

> Adam Wołk <netprobe@gmail.com> writes:
>
>> I believe that having a good default and supported browser that
>> integrates well with Emacs would be great.
>
> Correct me if I am wrong, but this does not sound like "default" but
> more like "de facto," the difference being that we are talking about
> separate applications that run independently of each other.
>

They run independently as applications but thanks to extensions like  
mozrepl they can communicate the same way as Emacs + SLIME can with many  
Common Lisp implementations. So really embracing mozrepl would allow  
building a bridge between regular Emacs usage and browsing, focusing on  
conkeror allows us to have a more familiar environment both for usage and  
extending.

> I will not pretend to understand everything that is being discussed.  I
> am interested as a blind user, dependent on w3 and or emacs-w3m.  So
> far, I don't see how I will be able to use emacspeak to speak the
> contents of a conqueror buffer like I can any other emacs buffer.  Am I
> mistaken?
>
> rdc

quote from mozrepl website:
> Connect to Firefox and other Mozilla apps, explore and modify them from  
> the inside, while they're running.
> Execute Javascript, play with browser GUI, sneak into HTML pages,  
> examine functions and variables, redefine them on the fly, hot-fix
> bugs, ... MozRepl itself is programmable from within MozRepl.

Conkeror can be connected both ways with Emacs using mozrepl so I can  
imagine (but can't confirm) that one could implement a feature that would  
send website text content directly to emacspeak.

So my guess is that You could not only pass every browser buffer to  
emacspeak but also wouldn't have problems with pages using heavy  
javascript and flash for navigation. Before You take my words for granted  
it would be wise to wait for confirmation of this possibility from someone  
with actual experience with mozrepl.

I also saw a browser extension for Firefox called 'It's all text' that  
could send text input elements from forms and allow to edit them in  
external editors, sending it back when the editor saved the file. If  
exporting text from regular Firefox this way is possible then I assume  
that the website text content wouldn't be much different.

Re: Why Emacs should have a good web-browser

[Robert D. Crawford]

Adam Wołk <netprobe@gmail.com> writes:

> Dnia 21-07-2009 o 22:02:28 Robert D. Crawford <rdc1x@comcast.net>
> napisał(a):
>
>> Adam Wołk <netprobe@gmail.com> writes:
>>
>>> I believe that having a good default and supported browser that
>>> integrates well with Emacs would be great.
>>
>> Correct me if I am wrong, but this does not sound like "default" but
>> more like "de facto," the difference being that we are talking about
>> separate applications that run independently of each other.
>
> They run independently as applications but thanks to extensions like
> mozrepl they can communicate the same way as Emacs + SLIME can with
> many  Common Lisp implementations. So really embracing mozrepl would
> allow  building a bridge between regular Emacs usage and browsing,
> focusing on  conkeror allows us to have a more familiar environment
> both for usage and  extending.

Granted.  I like the interaction of a REPL.  I've played with the python
REPL in emacs and I see it as working like emacs-w3m in that the user
sends commands and the output gets sent to an emacs buffer.  This works
well as it just becomes more text that can be "read" through like any
other buffer.  

> quote from mozrepl website:
>> Connect to Firefox and other Mozilla apps, explore and modify them
>> from the inside, while they're running.
>> Execute Javascript, play with browser GUI, sneak into HTML pages,
>> examine functions and variables, redefine them on the fly, hot-fix
>> bugs, ... MozRepl itself is programmable from within MozRepl.
>
> Conkeror can be connected both ways with Emacs using mozrepl so I can
> imagine (but can't confirm) that one could implement a feature that
> would  send website text content directly to emacspeak.

If I understand what you are saying, the text would be sent to the
speech server but not be rendered in an emacs buffer.  This will not
work as it would prevent scrolling through the text, killing/yanking,
sending URLs to other processes (mplayer and pdf2text come to mind).
I'm not even sure how that would work with emacspeak as it relies on
emacs to get its input... at least that is how I understand it.

> So my guess is that You could not only pass every browser buffer to
> emacspeak but also wouldn't have problems with pages using heavy
> javascript and flash for navigation. Before You take my words for
> granted  it would be wise to wait for confirmation of this possibility
> from someone  with actual experience with mozrepl.

There is some integration between emacspeak and the mozrepl.  I've not
played with it in a very long time so I don't know exactly what it can
do.  I do know that Dr. Raman was using it for javascript development
but not for browsing.

> I also saw a browser extension for Firefox called 'It's all text' that
> could send text input elements from forms and allow to edit them in
> external editors, sending it back when the editor saved the file. If
> exporting text from regular Firefox this way is possible then I assume
> that the website text content wouldn't be much different.

I guess what I'm really hoping for is the speed of emacs-w3m with the
extensibility of emacs/w3.  The main purpose of my posting is to try to
move things in the direction of adding more accessibility to emacs.
Plus, having a browser that can replace w3, which is getting really old,
is a good idea.

Thanks for listening,
rdc
-- 
Robert D. Crawford                                      rdc1x@comcast.net

Why Emacs should have a good web-browser

[T.V. Raman]

Note that emacspeak does integrate with firefox via mozrepl ---
Google for "firebox" --- code is in emacspeak-moz.el.

That code needs some updating to make keyboard input work with
Firefox 3 --- things changed on the firefox side post FF2, and
I've not had the chance to finish the work.

But roughly, here is how it works:

0. I start Firefox with mozrepl enabled --- in my case I tell FF
to use an Xvfb created framebuffer --- but that's just an aside,
since I dont start Gnome or any other graphical desktop --- my
desktop is Emacs.

1. From emacs, I open a repl interaction handle via which I can
send forms to FF  for evaluation.

2. On the FF side, I load in a JS  library I wrote called adom.js
http://emacspeak.googlecode.com/svn/trunk/js/

that defines a set of helper functions in JS  -- facilitate
receiving HTML in emacs back from Firefox.


-- 
Best Regards,
--raman

Title:  Research Scientist
Email:  raman@google.com
WWW:    http://emacspeak.sf.net/raman/
Google: tv+raman
GTalk:  raman@google.com, tv.raman.tv@gmail.com
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc


3. Finally, I define a special Mozilla interaction mode where I
can send js commands to FF  from Emacs, and the response back as
HTML  --- I then hand off this HTML  to either W3 or W3M  for
processing.

All this works upto a point --- except that at present keyboard
input to FF from the emacs side is broken.

In effect, the above turns FF  into a DOM-server for Emacs.

-- 
Best Regards,
--raman

Title:  Research Scientist
Email:  raman@google.com
WWW:    http://emacspeak.sf.net/raman/
Google: tv+raman
GTalk:  raman@google.com, tv.raman.tv@gmail.com
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc



On 7/21/09, Robert D. Crawford <rdc1x@comcast.net> wrote:
> Adam Wołk <netprobe@gmail.com> writes:
>
>> Dnia 21-07-2009 o 22:02:28 Robert D. Crawford <rdc1x@comcast.net>
>> napisał(a):
>>
>>> Adam Wołk <netprobe@gmail.com> writes:
>>>
>>>> I believe that having a good default and supported browser that
>>>> integrates well with Emacs would be great.
>>>
>>> Correct me if I am wrong, but this does not sound like "default" but
>>> more like "de facto," the difference being that we are talking about
>>> separate applications that run independently of each other.
>>
>> They run independently as applications but thanks to extensions like
>> mozrepl they can communicate the same way as Emacs + SLIME can with
>> many  Common Lisp implementations. So really embracing mozrepl would
>> allow  building a bridge between regular Emacs usage and browsing,
>> focusing on  conkeror allows us to have a more familiar environment
>> both for usage and  extending.
>
> Granted.  I like the interaction of a REPL.  I've played with the python
> REPL in emacs and I see it as working like emacs-w3m in that the user
> sends commands and the output gets sent to an emacs buffer.  This works
> well as it just becomes more text that can be "read" through like any
> other buffer.
>
>> quote from mozrepl website:
>>> Connect to Firefox and other Mozilla apps, explore and modify them
>>> from the inside, while they're running.
>>> Execute Javascript, play with browser GUI, sneak into HTML pages,
>>> examine functions and variables, redefine them on the fly, hot-fix
>>> bugs, ... MozRepl itself is programmable from within MozRepl.
>>
>> Conkeror can be connected both ways with Emacs using mozrepl so I can
>> imagine (but can't confirm) that one could implement a feature that
>> would  send website text content directly to emacspeak.
>
> If I understand what you are saying, the text would be sent to the
> speech server but not be rendered in an emacs buffer.  This will not
> work as it would prevent scrolling through the text, killing/yanking,
> sending URLs to other processes (mplayer and pdf2text come to mind).
> I'm not even sure how that would work with emacspeak as it relies on
> emacs to get its input... at least that is how I understand it.
>
>> So my guess is that You could not only pass every browser buffer to
>> emacspeak but also wouldn't have problems with pages using heavy
>> javascript and flash for navigation. Before You take my words for
>> granted  it would be wise to wait for confirmation of this possibility
>> from someone  with actual experience with mozrepl.
>
> There is some integration between emacspeak and the mozrepl.  I've not
> played with it in a very long time so I don't know exactly what it can
> do.  I do know that Dr. Raman was using it for javascript development
> but not for browsing.
>
>> I also saw a browser extension for Firefox called 'It's all text' that
>> could send text input elements from forms and allow to edit them in
>> external editors, sending it back when the editor saved the file. If
>> exporting text from regular Firefox this way is possible then I assume
>> that the website text content wouldn't be much different.
>
> I guess what I'm really hoping for is the speed of emacs-w3m with the
> extensibility of emacs/w3.  The main purpose of my posting is to try to
> move things in the direction of adding more accessibility to emacs.
> Plus, having a browser that can replace w3, which is getting really old,
> is a good idea.
>
> Thanks for listening,
> rdc
> --
> Robert D. Crawford                                      rdc1x@comcast.net
>
>
>
>

Re: Why Emacs should have a good web-browser

[Robert D. Crawford]

"T.V. Raman" <tv.raman.tv@gmail.com> writes:

> Note that emacspeak does integrate with firefox via mozrepl ---
> Google for "firebox" --- code is in emacspeak-moz.el.

Thanks.  I'll have to have a look at the code and see what I can figure
out.  What are the disadvantages to doing things like this?  Would it,
with some work, be able to replace w3 or w3m?


>
>
-- 
Robert D. Crawford                                      rdc1x@comcast.net

Re: Why Emacs should have a good web-browser

[Adam Wołk]

>> Conkeror can be connected both ways with Emacs using mozrepl so I can
>> imagine (but can't confirm) that one could implement a feature that
>> would  send website text content directly to emacspeak.
>
> If I understand what you are saying, the text would be sent to the
> speech server but not be rendered in an emacs buffer.  This will not
> work as it would prevent scrolling through the text, killing/yanking,
> sending URLs to other processes (mplayer and pdf2text come to mind).
> I'm not even sure how that would work with emacspeak as it relies on
> emacs to get its input... at least that is how I understand it.
>
I wrongly expressed what I had in mind by 'sending directly to emacspeak'.
What I was trying to say is that the text could be pulled over from the  
browser
to Emacs which would trigger emacspeak to read it. This of course was a
mistake on my side as I have no experience with emacspeak and wrongly  
assumed
how emacspeak is used and works. I stand corrected now but still never  
meant to imply
that Emacs would be excluded from the browsing process.

Re: Why Emacs should have a good web-browser

[Robert D. Crawford]

Adam Wołk <netprobe@gmail.com> writes:

>>> Conkeror can be connected both ways with Emacs using mozrepl so I can
>>> imagine (but can't confirm) that one could implement a feature that
>>> would  send website text content directly to emacspeak.
>>
>> If I understand what you are saying, the text would be sent to the
>> speech server but not be rendered in an emacs buffer.  This will not
>> work as it would prevent scrolling through the text, killing/yanking,
>> sending URLs to other processes (mplayer and pdf2text come to mind).
>> I'm not even sure how that would work with emacspeak as it relies on
>> emacs to get its input... at least that is how I understand it.
>>
> I wrongly expressed what I had in mind by 'sending directly to
> emacspeak'.  What I was trying to say is that the text could be pulled
> over from the browser to Emacs which would trigger emacspeak to read
> it. This of course was a mistake on my side as I have no experience
> with emacspeak and wrongly assumed how emacspeak is used and works. I
> stand corrected now but still never meant to imply that Emacs would be
> excluded from the browsing process.

No worries.  I think it was as much my misunderstanding what you were
saying.  

I'm going to have a look at the mozrepl integration with emacspeak as
soon as I get a chance to really look.  Maybe I can report back with
some useful information before the thread dies.

Thanks again,
rdc
-- 
Robert D. Crawford                                      rdc1x@comcast.net

Re: Why Emacs should have a good web-browser

[Tassilo Horn]

Adam Wołk <netprobe@gmail.com> writes:

Hi Adam,

> There already exists a browser that tries to implement things the
> Emacs way. It's called conkeror.

I agree that Conkeror is a good browser for emacs users, but I'd like to
add another browser to the discussion: uzbl [1].

It's not specially designed for emacs users (keybindings etc.), but
instead it uses by default vi-like bindings (but those can be modified.)

It uses the slogan that it adheres the UNIX philosophy, e.g. it does one
thing very good (rendering web pages using WebKit) and it delegates
everything else to other tools (downloads, cookies, bookmarks, history,
...), which could be elisp handlers.

IMHO, this approach makes it suited best for creating a browser *inside*
emacs, not just a browser designed *for* emacs users.

Bye,
Tassilo
__________
[1] http://www.uzbl.org/

Re: Why Emacs should have a good web-browser

[Paul R]

On Tue, 21 Jul 2009 11:52:28 -0400, Stefan Monnier <monnier@iro.umontreal.ca> said:

> It might be possible to use one of those engines as Emacs's rendering
> engine, indeed. To me, it wouldn't seem like an good solution to the
> problem at hand because I don't think it would allow me to control the
> web-browser from Emacs (e.g., how would I access from Elisp the
> content of pages generated from HTML?).

I am not sure I see what type of use case you mean. Like counting
occurences of word « dog » in a page ?

> So it'd be more like embedding Emacs inside a normal browser. It's not
> a bad idea, but I don't think it'll provide as many benefits from
> Emacs's point of view.

I was mainly thinking about the benefits Emacs could get to entrust
buffer rendering to a third party component. In the case of
a xhtml/css+js engine, the advantage is that it is highly standardised,
higly available, and highly maintained, so it would not be a potentialy
dead-end dependance.

-- 
  Paul

Re: Why Emacs should have a good web-browser

[Stefan Monnier]

>> It might be possible to use one of those engines as Emacs's rendering
>> engine, indeed. To me, it wouldn't seem like an good solution to the
>> problem at hand because I don't think it would allow me to control the
>> web-browser from Emacs (e.g., how would I access from Elisp the
>> content of pages generated from HTML?).

> I am not sure I see what type of use case you mean. Like counting
> occurences of word « dog » in a page ?

Maybe not the one I was thinking of, but yes, why not.  Or opening a Web
page, and then navigating it with M-C-f, changing its major mode,
killing part of it and yanking it into some other buffer, displaying
various parts of that web page into a bunch of different (Emacs)
windows, ...

> I was mainly thinking about the benefits Emacs could get to entrust
> buffer rendering to a third party component. In the case of
> a xhtml/css+js engine, the advantage is that it is highly standardised,
> higly available, and highly maintained, so it would not be a potentialy
> dead-end dependance.

Yes, of course.  But the rendering from Emacs buffers to those might be
the tricky part (which would require maintenance and work just like
the current rendering engine to X11).  Of course, maybe it'd be easy and
simple enough to be a significant win.  As long as nobody tries, nobody
will know.


        Stefan

Why Emacs should have a good web-browser

[Stephen Eilert]

[ Forgot to reply to the list. My bad ]



On Fri, Jul 17, 2009 at 10:29 PM, Miles Bader<miles@gnu.org> wrote:
>
> Paul R <paul.r.ml@gmail.com> writes:
>>
>> How about doing that the other way, that is rendering emacs buffers in
>> webkit (or an other xhtml/css+js lib), making it the default rendering
>> backend for emacs in graphic mode. It would probably cut off some large
>> amount of emacs plateform-specific code and make web integration tight.
>
> Please give more details about what you mean.  Emacs' display engine is
> not simple, and much code relies on its (many) features.  It would seem
> quit difficult to make it use a high-level display toolkit with very
> different abstractions.

I don't think there's any problem that Emacs doesn't look like a
"native" application. The problem is that it looks like an "old"
application.
If you take screenshots from Emacs and Textmate, both with their
"factory" defaults, Textmate looks *much* nicer (even though its
window consists mostly of text too). However, my heavily customized
Emacs looks prettier (IMHO) than the default, as does this one:
http://ozmm.org/posts/textmate_minor_mode.html

So, I think there are two issues here: first, better defaults. This
not only includes CUA-mode (even though I don't use it), but also XFT,
and the best font rendering we can get. Emacs 23 is close.

But it also needs more in the way of graphics. Even though Emacs
buffers *can* display graphics, their use is fairly limited(I see some
icons in Gnus, avatar faces in twit.el, line breaks, and the
speedbar). But these icons are small, not able to be changed in an
obvious way, are bitmaps, and are ugly.

So, we either need a better way to package graphics, as the preferred
distribution method seems to be EmacsWiki .el script files, so a
proper package manager. Or a way to create vector and bitmap images
programatically (some Cairo-like elisp library, or bindings for Cairo
itself).

I am a developer, though I've only hacked my personal .emacs so far,
but those are both interesting projects (package manager or graphics
library).


--Stephen

programmer, n:
      A red eyed, mumbling mammal capable of conversing with inanimate monsters.

Re: Why Emacs should have a good web-browser

[Chong Yidong]

Stephen Eilert <spedrosa@gmail.com> writes:

> Or a way to create vector and bitmap images programatically (some
> Cairo-like elisp library, or bindings for Cairo itself).

Emacs 23 can already do this, because it can be compiled with librsvg.

Re: Why Emacs should have a good web-browser

[Stephen Eilert]

On Tue, Jul 21, 2009 at 8:05 PM, Chong Yidong<cyd@stupidchicken.com> wrote:
> Stephen Eilert <spedrosa@gmail.com> writes:
>
>> Or a way to create vector and bitmap images programatically (some
>> Cairo-like elisp library, or bindings for Cairo itself).
>
> Emacs 23 can already do this, because it can be compiled with librsvg.
>

And how do I call it from elisp? I couldn't figure out how.


--Stephen

programmer, n:
        A red eyed, mumbling mammal capable of conversing with
inanimate monsters.

Re: Why Emacs should have a good web-browser

[Chong Yidong]

Stephen Eilert <spedrosa@gmail.com> writes:

> On Tue, Jul 21, 2009 at 8:05 PM, Chong Yidong<cyd@stupidchicken.com> wrote:
>> Stephen Eilert <spedrosa@gmail.com> writes:
>>
>>> Or a way to create vector and bitmap images programatically (some
>>> Cairo-like elisp library, or bindings for Cairo itself).
>>
>> Emacs 23 can already do this, because it can be compiled with librsvg.
>>
>
> And how do I call it from elisp? I couldn't figure out how.

Here's an example:

(setq my-svg-data
      "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?><svg xmlns:svg=\"http://www.w3.org/2000/svg\" xmlns=\"http://www.w3.org/2000/svg\" version=\"1.0\" width=\"223.51666\" height=\"223.76907\" id=\"svg2\"><defs id=\"defs4\" /><g transform=\"translate(-191.24778,-211.49836)\" id=\"layer1\"><path d=\"M 408.57145,369.50505 L 333.90781,368.26904 L 290.56166,429.07444 L 268.66485,357.68315 L 197.44078,335.24841 L 258.57144,292.36219 L 257.8987,217.69136 L 317.57633,262.5775 L 388.38462,238.86313 L 364.13677,309.49051 L 408.57145,369.50505 z\" id=\"path2399\" style=\"opacity:1;fill:#ff0a0a;fill-opacity:0.264574;stroke:#000000;stroke-width:12.38599968;stroke-linecap:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1\" /></g></svg>")

(setq my-svg-descriptor (create-image my-svg-data 'svg t))
(insert-image my-svg-descriptor)

Re: Why Emacs should have a good web-browser

[joakim]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> First, I would like to know if you agree about the reasons for having
>> a web browser in Emacs (either as part of it or as an external lisp
>> package).
>
> I'd like to see Emacs support things like javascript better, as well as
> web-browsing.  The Emacs/W3 way is sadly unworkable because of the
> amount of effort this requires (compounded by the performance
> limitations of Elisp).  Embedding a whole web-browser like Firefox might
> be a good direction, tho it's probably too coarse to be satisfactory.
> E.g. I'd like to be able to use Emacs's own completion code when typing
> the URL or when filling any text in the page.
>
> Maybe the most promising direction I can see is to try and make some
> library (like webkit) render directly into an Emacs buffer (rather than
> X11/Cairo/w32/plaintext/younameit).  Still, to make it handle tables
> properly, we may need to extend Emacs's rendering engine to better
> support them (which would be handy in any case for various applications
> like list-buffer which should columns of data).
>

Would it be posible to consolidate one or the other of the window groups
proposals, and tables? That is, a table cell is somewhat like an Emacs
window.

I played a little bit with having firefox print to pdf, and render pdf
to ascii, and view that in emacs, but it didnt look very good. Better
tables would likely help.


>         Stefan
>
-- 
Joakim Verona

Re: Why Emacs should have a good web-browser

[martin rudalics]

 > Would it be posible to consolidate one or the other of the window groups
 > proposals, and tables? That is, a table cell is somewhat like an Emacs
 > window.

IIUC tables can have hundreds of cells.  Do you want to create that many
windows?

martin

Re: Why Emacs should have a good web-browser

[joakim]

martin rudalics <rudalics@gmx.at> writes:

>> Would it be posible to consolidate one or the other of the window groups
>> proposals, and tables? That is, a table cell is somewhat like an Emacs
>> window.
>
> IIUC tables can have hundreds of cells.  Do you want to create that many
> windows?

It was just a thought.
Heres some other thoughts:

Use case: text flow around images, which would improve w3m
experience.

a test hack could be to use emacs image-slicing to slice an image in
equal row-height slices, to display text around the image. (maybe the
image slices could be locked in place, dunno)

this is like "rowspan" if an emacs window is a html table. would maybe
be cool with a :rowspan display property.

Some other ideas:
- check out "row overlap" mentioned in dispextern.h
- see if an image could automatically fill a number of "display matrix
cells"


> martin
-- 
Joakim Verona

Re: Why Emacs should have a good web-browser

[Deniz Dogan]

2009/7/8 Fernando <ferkiwi@gmail.com>:
> Hello.
> Sorry if this is not the right place, but I wanted to ask some
> questions to the community and expose some arguments.
>
> First, I would like to know if you agree about the reasons for having
> a web browser in Emacs (either as part of it or as an external lisp
> package).
>
> 1) One of the main purposes of Emacs is programming. Web development,
> css and JavaScript are emerging languages present a lot in the
> internet since a long time and now they are even extending to the
> desktop (gnome-shell, seed, adium themes...). Emacs has modes and
> tools to edit on this languages, but the integration is not as good as
> it could be if you had js and html interpreters integrated in Emacs
> itself.
>
> 2) There are very few web browsers that are comfortable to use in a
> keyboard-only interface. Emacs would be very good in this sense
> because its keyboard navigation is very usable and as it's designed
> for editing text, it will be perfect for all the form editing, comment
> writing and all the editing related actions you have to do often in a
> browser (editing wikipedia articles, etc).
>
> 3) Emacspeak  has turned Emacs into a very accessible environment for
> the visual-impaired and it would offer  these people a highly
> customizable interface to help them browse the web, along with the
> keybindings.
>
> 4) Emacs since long time has been one of the greatest tools for an
> operative system. During the Age Of The Usenet it was an good
> newsreader. Now that the newsgroups have started to die slowly and the
> HTTP protocol and Javascript are the kings of the big cloud Emacs
> should adapt to it.
>
> 5) Browsers are turning into the next generation Emacs! they can
> browse ftp, access IRC channels, check your mail, read pdf and other
> things with embeded applications, now they can even play video/audio
> as a core functionality, they are often used for editing text (web
> forms, comments in blogs, etc)... there's even the whole "Google
> Chrome OS" designed around a browser. Sooner or later they will be
> able to edit code (there's even prototypes for this already) when this
> happens Emacs has to compete or it will slowly die. Web browsers are
> turning into the main program for the end/power-user in a PC, when
> they reach Emacs in functionality I'm sure a lot of people (even Emacs
> users) would end up switching to hack Javascript instead of LISP.
>
> 6) The special features of Lisp and the extensibility of Emacs make it
> be the perfect candidate for an extensible and modular web browser.
> Current browsers are tending to improve their extensibility by means
> of "plugins" and "extensions". Emacs has since long time a powerful
> scripting that a lot of browsers would envy to have.
>
> - Emacs-w3m is not enough and it's not an Emacs module that can be extended.
> - Emacs/w3 was a very good idea but soon it has passed more than 1
> year since the last single commit to the git repository, it doesn't
> look very active at all (am I wrong?).
>
> So.. I just want to know what's the general feeling of the emacs-dev
> community about having an emacs web-browser and what expectations
> should we have in this regard, is there any other work being done by
> anyone? how much is the interest on this?
>
> Not long ago a new (alpha) version of Guile was released that
> introduced some basic support for ECMAscript, announcing that there's
> a goal to support up to version 3.1 of the spec. Would this make it
> possible for a (distant) future Guile Emacs to be able to have an
> efficient Javascript-capable web browser?
> There also seems that a bit more of work was put on the Guile elisp
> compiler lately, although it's still far from being mature.
>
> --
> Fernando
> (sorry for my english)
>
>
>

I'm surprised this hasn't been mentioned in this thread yet:

http://www.haxney.org/2009/08/its-alive.html

-- 
Deniz Dogan