porting to Wayland

porting to Wayland

[Daiki Ueno]

Hi,

As a learning project, I'm trying to port Emacs to the Wayland display
server[1].  Now the very basic redisplay code started working, although
it's still buggy (it even doesn't start without -Q):

Video: http://du-a.org/~ueno/junk/emacs-wl.webm
Code: https://github.com/ueno/emacs/tree/wip/wl

Rendering in Wayland is entirely done by the client, and the server
simply reveals the part of buffer sent from the client.  So, the main
task is to replace all the Xlib-based rendering stuff with the
buffer-based rendering using cairo.

I originally thought it could be done as an "add-on" to the current X
toolkit support.  As GTK3 already has the Wayland backend, it should be
able to embed the Emacs surface.  However, after looking at the code
(xterm.c/xfns.c), I realized that large part of it relies on Xlib and
seems too hard (for me) to experiment, without breaking the existing
code.  So, my initial attempt is to add it as another "window system"
(like NS or W32).

Currently it's missing:

- proper color handling
- images
- some text decoration (underline, overline, overstrike)
- toolkit stuff (window decoration, menubar, toolbar, scrollbar)

and many more.  Perhaps anybody here already considered/tried this?

Regards,

Footnotes: 
[1]  http://wayland.freedesktop.org

-- 
Daiki Ueno

Re: porting to Wayland

[Jan Djärv]

Hello.

I have updated the YAMAMOTO Mitsuharu patch:
http://lists.gnu.org/archive/html/emacs-devel/2009-04/msg00390.html

It all works except some image glitches.
It does not add another window system, rather expands the xterm.c code.
I was expecting that Gtk3+ and Cairo together would let us re-enable double 
buffering in Gtk and thus get that for free.  Unfortunately this did not work, 
as Gtk needs to do its drawing in the event loop, like the NS port needs to do 
as mentioned in #17124.

I will probably check this in if I get round to writing all those ChangeLog 
entries and fix the image things.  There is actually not much user benefit 
from using Cairo, except Mitsuharus printing extension.  For Wayland, there is 
XWayland.  Using Cairo is actually slower than using X, sometimes visible so. 
  Making Cairo draw the same way we draw in X may be a suboptimal way to use 
Cairo.

	Jan D.

2014-07-13 02:12, Daiki Ueno skrev:
> Hi,
>
> As a learning project, I'm trying to port Emacs to the Wayland display
> server[1].  Now the very basic redisplay code started working, although
> it's still buggy (it even doesn't start without -Q):
>
> Video: http://du-a.org/~ueno/junk/emacs-wl.webm
> Code: https://github.com/ueno/emacs/tree/wip/wl
>
> Rendering in Wayland is entirely done by the client, and the server
> simply reveals the part of buffer sent from the client.  So, the main
> task is to replace all the Xlib-based rendering stuff with the
> buffer-based rendering using cairo.
>
> I originally thought it could be done as an "add-on" to the current X
> toolkit support.  As GTK3 already has the Wayland backend, it should be
> able to embed the Emacs surface.  However, after looking at the code
> (xterm.c/xfns.c), I realized that large part of it relies on Xlib and
> seems too hard (for me) to experiment, without breaking the existing
> code.  So, my initial attempt is to add it as another "window system"
> (like NS or W32).
>
> Currently it's missing:
>
> - proper color handling
> - images
> - some text decoration (underline, overline, overstrike)
> - toolkit stuff (window decoration, menubar, toolbar, scrollbar)
>
> and many more.  Perhaps anybody here already considered/tried this?
>
> Regards,
>
> Footnotes:
> [1]  http://wayland.freedesktop.org
>

Re: porting to Wayland

[Daiki Ueno]

Hi Jan,

Thanks for the comment.  Glad to hear from the right person.

Jan Djärv <jan.h.d@swipnet.se> writes:

> I have updated the YAMAMOTO Mitsuharu patch:
> http://lists.gnu.org/archive/html/emacs-devel/2009-04/msg00390.html

I missed it, thanks for the pointer.

> It does not add another window system, rather expands the xterm.c code.
> I was expecting that Gtk3+ and Cairo together would let us re-enable
> double buffering in Gtk and thus get that for free.  Unfortunately
> this did not work, as Gtk needs to do its drawing in the event loop,
> like the NS port needs to do as mentioned in #17124.

Yeah, it would be ideal that the single stack (Gtk3+ and Cairo) supports
all the underlying display system.

> I will probably check this in if I get round to writing all those
> ChangeLog entries and fix the image things.  There is actually not
> much user benefit from using Cairo, except Mitsuharus printing
> extension.  For Wayland, there is XWayland.

> Using Cairo is actually slower than using X, sometimes visible
> so. Making Cairo draw the same way we draw in X may be a suboptimal
> way to use Cairo.

However, on Wayland, I feel rendering through XWayland rather slower
than using Cairo directly.  Even the 'M-x butterfly' animation is not
very smooth (few glyphs are visible before being cleared).  Maybe I
should have measured the speed before coding.

Anyway, my intention was to understand more about the redisplay stuff,
not to propose code for merge, so I'm already rewarded :-)

Regards,
-- 
Daiki Ueno

Re: porting to Wayland

[Jan Djärv]

Hello.

13 jul 2014 kl. 11:56 skrev Daiki Ueno <ueno@gnu.org>:

> Jan Djärv <jan.h.d@swipnet.se> writes:
> 
> 
>> Using Cairo is actually slower than using X, sometimes visible
>> so. Making Cairo draw the same way we draw in X may be a suboptimal
>> way to use Cairo.
> 
> However, on Wayland, I feel rendering through XWayland rather slower
> than using Cairo directly.  Even the 'M-x butterfly' animation is not
> very smooth (few glyphs are visible before being cleared).  Maybe I
> should have measured the speed before coding.

XWayland does have extra overhead, so this is not surprising.  But it is disturbing that the overhead is so large that it is noticable.

	Jan D.

Re: porting to Wayland

[Lars Magne Ingebrigtsen]

Jan Djärv <jan.h.d@swipnet.se> writes:

> I have updated the YAMAMOTO Mitsuharu patch:
> http://lists.gnu.org/archive/html/emacs-devel/2009-04/msg00390.html
>
> It all works except some image glitches.
> It does not add another window system, rather expands the xterm.c code.
> I was expecting that Gtk3+ and Cairo together would let us re-enable
> double buffering in Gtk and thus get that for free.  Unfortunately
> this did not work, as Gtk needs to do its drawing in the event loop,
> like the NS port needs to do as mentioned in #17124.
>
> I will probably check this in if I get round to writing all those
> ChangeLog entries and fix the image things.

I just got a Jolla phone running Sailfish, which is apparently a Wayland
server without XWayland.  Would this Wayland port enable running a
graphical Emacs on the phone, I wonder?  Somebody else has apparently
ported a non-graphical Emacs to the phone already...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: porting to Wayland

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I just got a Jolla phone running Sailfish, which is apparently a Wayland
> server without XWayland.  Would this Wayland port enable running a
> graphical Emacs on the phone, I wonder?  Somebody else has apparently
> ported a non-graphical Emacs to the phone already...

I forgot to mention why it might be interesting to run Emacs on the
phone:  Somebody is making a keyboard to the Jolla.

https://www.kickstarter.com/projects/2028347278/tohkbd-the-other-half-keyboard-for-your-jolla


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: porting to Wayland

[Jan D.]

Hi.

Lars Magne Ingebrigtsen skrev den 2014-11-12 16:50:
> Jan Djärv <jan.h.d@swipnet.se> writes:
>
>> I have updated the YAMAMOTO Mitsuharu patch:
>> http://lists.gnu.org/archive/html/emacs-devel/2009-04/msg00390.html
>>
>> It all works except some image glitches.
>> It does not add another window system, rather expands the xterm.c code.
>> I was expecting that Gtk3+ and Cairo together would let us re-enable
>> double buffering in Gtk and thus get that for free.  Unfortunately
>> this did not work, as Gtk needs to do its drawing in the event loop,
>> like the NS port needs to do as mentioned in #17124.
>>
>> I will probably check this in if I get round to writing all those
>> ChangeLog entries and fix the image things.
>
> I just got a Jolla phone running Sailfish, which is apparently a Wayland
> server without XWayland.  Would this Wayland port enable running a
> graphical Emacs on the phone, I wonder?  Somebody else has apparently
> ported a non-graphical Emacs to the phone already...
>

This is not a Wayland port, it is a Cairo port.  It still uses X for 
events (mouse keyboard) as Cairo does not have any of that.
Wayland would be a different beast.

	Jan D.

Re: porting to Wayland

[Lars Magne Ingebrigtsen]

"Jan D." <jan.h.d@swipnet.se> writes:

> This is not a Wayland port, it is a Cairo port.  It still uses X for
> events (mouse keyboard) as Cairo does not have any of that.
> Wayland would be a different beast.

Right.  So this is totally unrelated to what Daiki was experimenting
with?

(I don't know what Cairo is.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: porting to Wayland

[Achim Gratz]

Lars Magne Ingebrigtsen writes:
> (I don't know what Cairo is.)

It's a mostly device agnostic rendering API
http://cairographics.org/


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: porting to Wayland

[Jan D.]

Hi.

> 12 nov 2014 kl. 18:05 skrev Lars Magne Ingebrigtsen <larsi@gnus.org>:
> 
> "Jan D." <jan.h.d@swipnet.se> writes:
> 
>> This is not a Wayland port, it is a Cairo port.  It still uses X for
>> events (mouse keyboard) as Cairo does not have any of that.
>> Wayland would be a different beast.
> 
> Right.  So this is totally unrelated to what Daiki was experimenting
> with?
> 

I don't know what Daiki was experimenting with.

> (I don't know what Cairo is.)

Its a 2D drawing API http://en.wikipedia.org/wiki/Cairo_(graphics).  Wayland does not have any drawing API so you could use Cairo. You can also use something else, more or less whatever you choose.  Wayland leaves all that up to the application.

	Jan D.

Re: porting to Wayland

[Daiki Ueno]

"Jan D." <jan.h.d@swipnet.se> writes:

>>> This is not a Wayland port, it is a Cairo port.  It still uses X for
>>> events (mouse keyboard) as Cairo does not have any of that.
>>> Wayland would be a different beast.
>> 
>> Right.  So this is totally unrelated to what Daiki was experimenting
>> with?
>> 
>
> I don't know what Daiki was experimenting with.

The Cairo part is an overlap between those patches.  I believe mine is
much less quality.  The differences are: Yamamoto-san's patch adds
printing support using GTK+ print dialogs, and mine was trying to add
Wayland keyboard event handling.

By the way, some of my friends are really interested in the Cairo based
printing support.  Could you perhaps post an update of Yamamoto-san's
patch?  I'm happy to test it.

Regards,
--
Daiki Ueno

Re: porting to Wayland

[Jan D.]

Daiki Ueno skrev den 2014-11-13 02:58:
> "Jan D." <jan.h.d@swipnet.se> writes:
>
>>>> This is not a Wayland port, it is a Cairo port.  It still uses X for
>>>> events (mouse keyboard) as Cairo does not have any of that.
>>>> Wayland would be a different beast.
>>>
>>> Right.  So this is totally unrelated to what Daiki was experimenting
>>> with?
>>>
>>
>> I don't know what Daiki was experimenting with.
>
> The Cairo part is an overlap between those patches.  I believe mine is
> much less quality.  The differences are: Yamamoto-san's patch adds
> printing support using GTK+ print dialogs, and mine was trying to add
> Wayland keyboard event handling.
>
> By the way, some of my friends are really interested in the Cairo based
> printing support.  Could you perhaps post an update of Yamamoto-san's
> patch?  I'm happy to test it.

I haven't kept it up to date with trunk lately.  I'll see if I can 
update it and then post it.

	Jan D.

Re: porting to Wayland

[Jan Djärv]

[-- Attachment #1: Type: text/plain, Size: 1476 bytes --]

Ok, here goes.

Image support sucks, only PNG from files.  Some redraw issues (esp. Gtk 
scrollbars) remain.
If you choose Gtk3 as toolkit, cairo is forced, otherwise give--with-cairo to 
configure.  Tool bar is blank when using native tool bar (i.e. Lucid, 
Lesstif/Motif, no toolkit) because of no xpm-support for images.

Performance is not that great, straight X is faster.

It will probably destroy your computer and burn down your house also.

	Jan D.

Den 2014-11-13 10:08, Jan D. skrev:
> Daiki Ueno skrev den 2014-11-13 02:58:
>> "Jan D." <jan.h.d@swipnet.se> writes:
>>
>>>>> This is not a Wayland port, it is a Cairo port.  It still uses X for
>>>>> events (mouse keyboard) as Cairo does not have any of that.
>>>>> Wayland would be a different beast.
>>>>
>>>> Right.  So this is totally unrelated to what Daiki was experimenting
>>>> with?
>>>>
>>>
>>> I don't know what Daiki was experimenting with.
>>
>> The Cairo part is an overlap between those patches.  I believe mine is
>> much less quality.  The differences are: Yamamoto-san's patch adds
>> printing support using GTK+ print dialogs, and mine was trying to add
>> Wayland keyboard event handling.
>>
>> By the way, some of my friends are really interested in the Cairo based
>> printing support.  Could you perhaps post an update of Yamamoto-san's
>> patch?  I'm happy to test it.
>
> I haven't kept it up to date with trunk lately.  I'll see if I can update it
> and then post it.
>
>      Jan D.
>
>


[-- Attachment #2: cr-diff.gz --]
[-- Type: application/x-zip-compressed, Size: 21796 bytes --]

Re: porting to Wayland

[Lars Magne Ingebrigtsen]

Jan Djärv <jan.h.d@swipnet.se> writes:

> Ok, here goes.

Hey, great!

> Performance is not that great, straight X is faster.
>
> It will probably destroy your computer and burn down your house also.

:-)

Is this Cairo only, or does it support direct rendering on Wayland
without X?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: porting to Wayland

[Jan D.]

Hi.

> 24 nov 2014 kl. 01:07 skrev Lars Magne Ingebrigtsen <larsi@gnus.org>:
> 
> Jan Djärv <jan.h.d@swipnet.se> writes:
> 
>> Ok, here goes.
> 
> Hey, great!
> 
>> Performance is not that great, straight X is faster.
>> 
>> It will probably destroy your computer and burn down your house also.
> 
> :-)
> 
> Is this Cairo only, or does it support direct rendering on Wayland
> without X?

Cairo on X only.

	Jan D.

Re: porting to Wayland

[Daiki Ueno]

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

Jan Djärv <jan.h.d@swipnet.se> writes:

> Ok, here goes.

Thanks!

> Image support sucks, only PNG from files.  Some redraw issues
> (esp. Gtk scrollbars) remain.
> If you choose Gtk3 as toolkit, cairo is forced, otherwise
> give--with-cairo to configure.

I've tried the patch.  For PNG support, I had to apply the attached
patch and run configure with:

  ./configure --with-x-toolkit=gtk3 --with-cairo --without-imagemagick

The output for printing (with x-export-frames) looks great.  

> Performance is not that great, straight X is faster.

It's already fast enough for me, but it doesn't do rendering in ::draw
signal, right?  If it did, would the speed considerablly slow down?

Regards,
--
Daiki Ueno

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: png.patch --]
[-- Type: text/x-patch, Size: 693 bytes --]

diff --git a/configure.ac b/configure.ac
index ac0b86e..a84637d 100644
--- a/configure.ac
+++ b/configure.ac
@@ -3042,7 +3042,6 @@ if test "${HAVE_X11}" = "yes"; then
     fi
     with_xpm=no
     with_jpeg=no
-    with_png=no
     with_gif=no
     with_tiff=no
 
diff --git a/src/image.c b/src/image.c
index 77e5523..ad31719 100644
--- a/src/image.c
+++ b/src/image.c
@@ -5459,7 +5459,7 @@ pbm_load (struct frame *f, struct image *img)
 				 PNG
  ***********************************************************************/
 
-#if defined (HAVE_XPM) || defined (HAVE_NS) || defined (USE_CAIRO)
+#if defined (HAVE_PNG) || defined (HAVE_NS) || defined (USE_CAIRO)
 
 /* Function prototypes.  */
 

Re: porting to Wayland

[Jan D.]

Hi.

> 25 nov 2014 kl. 08:49 skrev Daiki Ueno <ueno@gnu.org>:
> 
> Jan Djärv <jan.h.d@swipnet.se> writes:
> 
>> Ok, here goes.
> 
> Thanks!
> 
>> Image support sucks, only PNG from files.  Some redraw issues
>> (esp. Gtk scrollbars) remain.
>> If you choose Gtk3 as toolkit, cairo is forced, otherwise
>> give--with-cairo to configure.
> 
> I've tried the patch.  For PNG support, I had to apply the attached
> patch and run configure with:
> 
>  ./configure --with-x-toolkit=gtk3 --with-cairo --without-imagemagick

Hmm, it seems I made a typo, thanks.

> 
> The output for printing (with x-export-frames) looks great.  
> 
>> Performance is not that great, straight X is faster.
> 
> It's already fast enough for me, but it doesn't do rendering in ::draw
> signal, right?  If it did, would the speed considerablly slow down?

Do you mean the Gtk+ draw signal?  Emacs bypasses that even without cairo.

	Jan D.

> 
> Regards,
> --
> Daiki Ueno
> diff --git a/configure.ac b/configure.ac
> index ac0b86e..a84637d 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -3042,7 +3042,6 @@ if test "${HAVE_X11}" = "yes"; then
>     fi
>     with_xpm=no
>     with_jpeg=no
> -    with_png=no
>     with_gif=no
>     with_tiff=no
> 
> diff --git a/src/image.c b/src/image.c
> index 77e5523..ad31719 100644
> --- a/src/image.c
> +++ b/src/image.c
> @@ -5459,7 +5459,7 @@ pbm_load (struct frame *f, struct image *img)
> 				 PNG
>  ***********************************************************************/
> 
> -#if defined (HAVE_XPM) || defined (HAVE_NS) || defined (USE_CAIRO)
> +#if defined (HAVE_PNG) || defined (HAVE_NS) || defined (USE_CAIRO)
> 
> /* Function prototypes.  */
> 

Re: porting to Wayland

[Stefan Monnier]

> Ok, here goes.
> Image support sucks, only PNG from files.  Some redraw issues (esp. Gtk
> scrollbars) remain.
> If you choose Gtk3 as toolkit, cairo is forced, otherwise give--with-cairo
> to configure.  Tool bar is blank when using native tool bar (i.e. Lucid,
> Lesstif/Motif, no toolkit) because of no xpm-support for images.

I think this should be in a branch in emacs.git.


        Stefan

Re: porting to Wayland

[Jan D.]

Hello.

> 25 nov 2014 kl. 15:30 skrev Stefan Monnier <monnier@iro.umontreal.ca>:
> 
>> Ok, here goes.
>> Image support sucks, only PNG from files.  Some redraw issues (esp. Gtk
>> scrollbars) remain.
>> If you choose Gtk3 as toolkit, cairo is forced, otherwise give--with-cairo
>> to configure.  Tool bar is blank when using native tool bar (i.e. Lucid,
>> Lesstif/Motif, no toolkit) because of no xpm-support for images.
> 
> I think this should be in a branch in emacs.git.

My plan was to wait until the generation of ChangeLog:s are in place.

	Jan D.

Re: porting to Wayland

[Stefan Monnier]

> My plan was to wait until the generation of ChangeLog:s are in place.

I think waiting is not a good plan.


        Stefan

Re: generating ChangeLogs automatically (was: porting to Wayland)

[Paul Eggert]

>> My plan was to wait until the generation of ChangeLog:s are in place.
>
> I think waiting is not a good plan.

True.  I have been waiting for the switch-to-git brouhaha to die down before 
installing further changes for automatic ChangeLog generation.  Perhaps things 
are becoming calm enough now.  This will entail changes to ChangeLog habits and 
so will start a new brouhaha (a smaller one I hope).  The current draft for this 
is in Bug#19113, at:

http://bugs.gnu.org/19113#23

Think of it as an echo to Eric Raymond's big bang.