Re: Introducing emacs-webkit and more thoughts on Emacs rendering (was Rethinking the design of xwidgets)

[Akira Kyle <akira@akirakyle.com>]

On Mon, Nov 23, 2020 at 04:47 PM, Alan Third <alan@idiocy.org> 
wrote:

>> Just a curious question: have you ever looked at libmpv?
>> Would it be possible to embedd video via livmpv in an image 
>> container
>> just as jpg, png etc?
>> 
>> libmpv is a C library for mpv player ment to be embedded, and 
>> works in
>> all 3 major desktop platforms.
>> 
>> https://github.com/mpv-player/mpv-examples/tree/master/libmpv
>> 
>> Mpv itself comes with crap-load of video codecs and image 
>> formats already
>> built-in; it is a fork mplayer2. What is good, if one could 
>> create a
>> OpenGL context for an Emacs window (X11/Win32 Window - 
>> shouldn't be
>> hard), you could controll rendering directly, since they don't 
>> take over
>> your even loop:
>> 
>> "You will have to simulate this with the 
>> mouse/keypress/keydown/keyup commands."
>> 
>> It would probably fit into Emacs well.
>> 
>> I don't know myself how to implement "new image format" for 
>> Emacs; too
>> much to look through the spaghetty of if-defs in image.c, 
>
> IIRC it's actually possible to load videos using the imagemagick
> backend, the trouble is that Emacs can't really handle them as 
> it
> decodes and stores each frame individually.
>
> Emacs has no way to "remember" where it is in a file, as each 
> image is
> loaded and stored individually.
>
> For example, to display an animated gif Emacs opens the file, 
> decodes
> frame one, closes the file and then displays the frame. When 
> asked to
> display frame two it opens the file, decodes frame two (which in 
> the
> case of a gif involves decoding frame one again) closes the file 
> and
> displays it. To display frame three it opens the file, decodes 
> frame
> three (which involves decoding frames one and two again), closes 
> the
> file and displays it. And so on and so on.
>
> I think it's like this because it was only intended for 
> displaying
> occasional graphics like the fringe bitmaps and similar, and, 
> IMHO, to
> do anything different would require a major overhaul of image.c, 
> and
> probably the lisp image API while maintaining backwards 
> compatibility.

I second this and would also add that I think the emacs event loop 
would need modification as well, since my understanding is that 
it's pretty lazy, rendering things only when it needs to. So while 
its good that mpv wouldn't demand taking over the event loop, I 
think emacs might really struggle to display video frames at 30fps 
without some modifications or work around.