SVG hack for display engine

SVG hack for display engine

[Anand Tamariya]

[-- Attachment #1.1: Type: text/plain, Size: 369 bytes --]

Here's throwing an idea into the wild. How about using SVG for rendering in
the display engine? The attached image shows the rendering as a proof of
concept for feasibility. The top window shows the SVG rendering of the
buffer shown in the bottom window. Though I'm not sure about the
practicality at this point of time. I'd love to hear your thoughts.

Regards,
Anand

[-- Attachment #1.2: Type: text/html, Size: 452 bytes --]

[-- Attachment #2: display_engine.png --]
[-- Type: image/png, Size: 84177 bytes --]

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Mon, 15 Nov 2021 08:09:22 +0530
> 
> How about using SVG for rendering in the display engine?

Please elaborate what you mean by that.  How would the display engine
work under this idea?

The image you posted doesn't tell enough to understand what's
involved, and thus it is very hard to think about possible
implications of such a change.

Thanks.

Re: SVG hack for display engine

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

In this hack, the elisp code navigates from (window-start) to (window-end)
reading a line of text from the associated buffer and displays the same in
SVG. I've used (read-event) and (lookup-key) to allow for key translations
for navigation commands. Hopefully with a tighter integration, all of this
will not be required.

Positioning of the point is one open area. We would need the bounding box
of text for accurate positioning. We have three sources for this - librsvg,
pango or the current display engine.

On Mon, Nov 15, 2021 at 6:15 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Anand Tamariya <atamariya@gmail.com>
> > Date: Mon, 15 Nov 2021 08:09:22 +0530
> >
> > How about using SVG for rendering in the display engine?
>
> Please elaborate what you mean by that.  How would the display engine
> work under this idea?
>
> The image you posted doesn't tell enough to understand what's
> involved, and thus it is very hard to think about possible
> implications of such a change.
>
> Thanks.
>

[-- Attachment #2: Type: text/html, Size: 1437 bytes --]

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Mon, 15 Nov 2021 18:57:20 +0530
> Cc: Emacs Devel <emacs-devel@gnu.org>
> 
> In this hack, the elisp code navigates from (window-start) to (window-end) reading a line of text from the
> associated buffer and displays the same in SVG. I've used (read-event) and (lookup-key) to allow for key
> translations for navigation commands. Hopefully with a tighter integration, all of this will not be required.

So you use the results of the display engine to display the same stuff
differently, is that right?

If so, I'm not sure I understand how to do it "properly".  The Emacs
display engine performs the layout calculations (which determine, for
example, where each screen line ends and the next one begins, or what
stuff fits into a window) by examining each character of buffer text
in turn.  Without this examination, how can we know how to display the
text as SVG?  And if we use the current code to produce this
information, then what did we gain?

I'm probably missing something.

Re: SVG hack for display engine

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 980 bytes --]

> So you use the results of the display engine to display the same stuff
> differently, is that right?
>
> If so, I'm not sure I understand how to do it "properly".  The Emacs
> display engine performs the layout calculations (which determine, for
> example, where each screen line ends and the next one begins, or what
> stuff fits into a window) by examining each character of buffer text
> in turn.  Without this examination, how can we know how to display the
> text as SVG?  And if we use the current code to produce this
> information, then what did we gain?
>
> I'm probably missing something.
>
Do you notice the popup in the top window in the image attached in OP? It
has clean lines over buffer content with mixed (fixed + variable pitch)
fonts. Is it possible to have that in graphical Emacs? Precise positioning
of popups can open up graphical Emacs to immense possibilities like feature
filled tooltips, inline completion popup, multi-column layouts, watermarks
etc.

[-- Attachment #2: Type: text/html, Size: 1247 bytes --]

Re: SVG hack for display engine

[Po Lu]

Anand Tamariya <atamariya@gmail.com> writes:

> Do you notice the popup in the top window in the image attached in OP?
> It has clean lines over buffer content with mixed (fixed + variable
> pitch) fonts. Is it possible to have that in graphical Emacs? Precise
> positioning of popups can open up graphical Emacs to immense
> possibilities like feature filled tooltips, inline completion popup,

Don't we have child frames which do exactly that?

Also, on X-Windows, tooltips are implemented as a special kind of Emacs
frame.

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Tue, 16 Nov 2021 08:31:24 +0530
> Cc: Emacs Devel <emacs-devel@gnu.org>
> 
> Do you notice the popup in the top window in the image attached in OP? It has clean lines over buffer
> content with mixed (fixed + variable pitch) fonts. Is it possible to have that in graphical Emacs? Precise
> positioning of popups can open up graphical Emacs to immense possibilities like feature filled tooltips, inline
> completion popup, multi-column layouts, watermarks etc.  

Why would it be impossible to implement something like that without
changing the current basic display code?  IOW, what does the SVG
enable, technically, that cannot be done with the current display and
prevents implementation of such popups?

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Emacs Devel <emacs-devel@gnu.org>
> Date: Tue, 16 Nov 2021 11:28:53 +0800
> 
> Anand Tamariya <atamariya@gmail.com> writes:
> 
> > Do you notice the popup in the top window in the image attached in OP?
> > It has clean lines over buffer content with mixed (fixed + variable
> > pitch) fonts. Is it possible to have that in graphical Emacs? Precise
> > positioning of popups can open up graphical Emacs to immense
> > possibilities like feature filled tooltips, inline completion popup,
> 
> Don't we have child frames which do exactly that?
> 
> Also, on X-Windows, tooltips are implemented as a special kind of Emacs
> frame.

The MS-Windows implementation of tooltips also uses special-purpose
frames.

Re: SVG hack for display engine

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> The MS-Windows implementation of tooltips also uses special-purpose
> frames.

As they say, you learn something new every day :)

Thanks.

Re: SVG hack for display engine

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

Hi,

Please don’t restrict your deliberations to pop-ups; that would be way too narrow a scope. Think of tables with different size faces in lines, rows, or even individual cells. Our tables using variable pitch fonts. Then, there other day, there was a discussion on this very list, about “universal tab stops” when using variable pitch fonts. 

All of these, and probably many more, use cases would be enabled by an SVG based display engine. 

  --alex

> On 16. Nov 2021, at 13:56, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> 
>> 
>> From: Anand Tamariya <atamariya@gmail.com>
>> Date: Tue, 16 Nov 2021 08:31:24 +0530
>> Cc: Emacs Devel <emacs-devel@gnu.org>
>> 
>> Do you notice the popup in the top window in the image attached in OP? It has clean lines over buffer
>> content with mixed (fixed + variable pitch) fonts. Is it possible to have that in graphical Emacs? Precise
>> positioning of popups can open up graphical Emacs to immense possibilities like feature filled tooltips, inline
>> completion popup, multi-column layouts, watermarks etc.  
> 
> Why would it be impossible to implement something like that without
> changing the current basic display code?  IOW, what does the SVG
> enable, technically, that cannot be done with the current display and
> prevents implementation of such popups?
> 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 1944 bytes --]

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Alexander Adolf <alexander.adolf@condition-alpha.com>
> Date: Tue, 16 Nov 2021 15:08:00 +0100
> Cc: Anand Tamariya <atamariya@gmail.com>, emacs-devel@gnu.org
> 
> All of these, and probably many more, use cases would be enabled by an SVG based display engine. 

What is "an SVG based display engine"?  How will it work?  Which parts
of the existing display engine will it replace, and how?

A serious discussion of these matters should answer all those
questions and many more.  The Emacs display engine isn't just
something that takes text and shows it on the screen in different
colors, it implements a large number of features without which Emacs
wouldn't be Emacs.  They all need to be supported in any new
implementation, so we should have a more-or-less clear idea how this
would be possible with this proposal.

Re: SVG hack for display engine

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]


> On 16. Nov 2021, at 15:17, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> 
>> 
>> From: Alexander Adolf <alexander.adolf@condition-alpha.com>
>> Date: Tue, 16 Nov 2021 15:08:00 +0100
>> Cc: Anand Tamariya <atamariya@gmail.com>, emacs-devel@gnu.org
>> 
>> All of these, and probably many more, use cases would be enabled by an SVG based display engine. 
> 
> What is "an SVG based display engine"?  How will it work?  Which parts
> of the existing display engine will it replace, and how?
> 
> A serious discussion of these matters should answer all those
> questions and many more. […]

Fully agree. 

And I’ll happily leave it to the OP to clarify those. 

  —alex 

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 1944 bytes --]

Re: SVG hack for display engine

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 1770 bytes --]

On Tue, Nov 16, 2021 at 7:46 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Alexander Adolf <alexander.adolf@condition-alpha.com>
> > Date: Tue, 16 Nov 2021 15:08:00 +0100
> > Cc: Anand Tamariya <atamariya@gmail.com>, emacs-devel@gnu.org
> >
> > All of these, and probably many more, use cases would be enabled by an
> SVG based display engine.
>
> What is "an SVG based display engine"?  How will it work?  Which parts
> of the existing display engine will it replace, and how?
>
> A serious discussion of these matters should answer all those
> questions and many more.  The Emacs display engine isn't just
> something that takes text and shows it on the screen in different
> colors, it implements a large number of features without which Emacs
> wouldn't be Emacs.  They all need to be supported in any new
> implementation, so we should have a more-or-less clear idea how this
> would be possible with this proposal.
>
Here's some details on how I think it might work. Emacs exposes two Lisp
APIs:
- svg-render (svg x y width height)
svg is a Lisp DOM object as returned by (dom-node). This should use librsvg
to generate a bitmap image of svg of size (width x height) and superimpose
it at position (x, y) on the current bitmap being displayed by Emacs. The
current image API takes either a filename or XML string for SVG. This can
be avoided if svg is a Lisp DOM.

- svg-bbox (node)
This returns the bounding box of the DOM node. This is basically a wrapper
around librsvg API.

Emacs also adds a hook, say, display-svg-hook which runs only on a
graphical display. Any mode can use this hook to display the SVG content
using the above API.

This should mean minimal changes to the display engine and it continues to
run as is for terminals. What do you think?

[-- Attachment #2: Type: text/html, Size: 2433 bytes --]

Re: SVG hack for display engine

[Po Lu]

Anand Tamariya <atamariya@gmail.com> writes:

> This should mean minimal changes to the display engine and it
> continues to run as is for terminals. What do you think?

FWIW I am working on a port of Emacs to GTK 4, which doesn't use Cairo
(which has been deprecated by the GTK developers), instead using the
native GSK scene graph kit to draw.  It does something very similar to
what your SVG rendering would be like, keeping a tree of render nodes
around past calls to gtk_widget_draw, and modifying it from the RIF.

Getting it to work mostly correctly was extremely painful, and it still
struggles with leaks (where obscured render nodes are not pruned
correctly by xg_fixup_render_table) and the lack of the scroll
optimization.

(If someone wants details on this, wait a few weeks.  I will probably
have it in a usable state by then.)

Thanks.

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Wed, 17 Nov 2021 10:03:25 +0530
> Cc: Alexander Adolf <alexander.adolf@condition-alpha.com>, Emacs Devel <emacs-devel@gnu.org>
> 
> Here's some details on how I think it might work. Emacs exposes two Lisp APIs:
> - svg-render (svg x y width height)
> svg is a Lisp DOM object as returned by (dom-node).

What code will produce that DOM object, and how will it be produced?

> This should use librsvg to generate a bitmap image of
> svg of size (width x height) and superimpose it at position (x, y) on the current bitmap being displayed by
> Emacs. The current image API takes either a filename or XML string for SVG. This can be avoided if svg is a
> Lisp DOM.
> 
> - svg-bbox (node)
> This returns the bounding box of the DOM node. This is basically a wrapper around librsvg API.
> 
> Emacs also adds a hook, say, display-svg-hook which runs only on a graphical display. Any mode can use
> this hook to display the SVG content using the above API.

Please describe the intended use of these APIs and the hook to provide
the display feature(s) you have in mind, and how will that work in
concert with the display engine.  I don't think I have a clear picture
of that yet.

Also, please describe what happens with use of these APIs when just a
small part of the Emacs display needs to be redrawn.

Thanks.

Re: SVG hack for display engine

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 3511 bytes --]

> Please describe the intended use of these APIs and the hook to provide
> the display feature(s) you have in mind, and how will that work in
> concert with the display engine.  I don't think I have a clear picture
> of that yet.
>
> Also, please describe what happens with use of these APIs when just a
> small part of the Emacs display needs to be redrawn.
>
> Maybe a demo ( https://youtu.be/1343m3LY4mo ) will help you visualize my
suggestion. In the video, an event loop is started in the scratch buffer.
All the edit operations update text and text property in a temporary
buffer, say a.txt. Based on this information, in-memory SVG DOM is updated.
Ultimately, the display engine updates the view with this SVG image. In
cases like this, the mode can have full control over the display if so
desired.

Since you have a problem accessing my repo, I've added the code of a
function I call in my event loop at the end of this mail.

With a tighter integration, Emacs would define a new data structure {svg,
x, y, w, h} and two new variables with this type - new_svg and prev_svg.
Then the sequence of operations would be:
- Display engine refreshes the view bitmap with current logic - full or
partial refresh as the case may be.
- If it's a graphical display
- prev_svg = new_svg
- Run display-svg-hook. The registered user function will call (svg-render)
function passing required information. This API sets the value in new_svg.
- If the bounding box of new_svg < the bounding box of prev_svg, display
engine refreshes the area of prev_svg in the view bitmap.
- Generate a bitmap for new_svg and superimpose it on the view bitmap.
- Display the view bitmap to the user.

Hope this helps.

(defun formula-canvas-mode (key svg)
  (let* ((func (lookup-key global-map (vector key)))
         (i 0)
         (size 20)
         mkey win
         text x y id point pos start end opts)
    ;; (message "Before: %s %s" key func)
    (unless func
      (setq mkey (lookup-key local-function-key-map (vector key))
            func (lookup-key global-map mkey)))
    ;; (message "After: %s %s" key func)

    (with-current-buffer (get-buffer-create "a.txt")
      (when func
        (condition-case var
            (if (eq func 'self-insert-command)
                (funcall func 1 key)
              (funcall func 1))
          (error (message (format "Error: %s" (car var)))))

        (save-excursion
          (setq win (get-buffer-window))
          (goto-char (window-start win))
          ;; Need to keep popup at the end
          ;; (setq svg (subseq svg 0 2))
          (dom-remove-node svg (car (dom-by-id svg "_popup")))
          (while (< (point) (window-end win))
            (setq start (point)
                  end (progn (end-of-line) (point))
                  text (buffer-substring start end)
                  id (number-to-string i)
                  x 0
                  y (* (1+ i) size))
            (setq i (1+ i))
            (forward-line)
            ;; Text
            (svg-text svg text :id id :x x :y y :xml:space "preserve")))

        ;; Cursor
        (setq pos (posn-col-row (posn-at-point (point) win))
              point (* 4 (car pos))
              y (* size (cdr pos)))
        (svg-rectangle svg point y 5 size
                       :id "_cursor" :fill "black" :opacity "0.4")
        ;; Popup
        (setq opts '("a" "b" "c" ))
        (if opts
            (formula-canvas-popup svg point (+ y size) opts))
        (svg-possibly-update-image svg)
        )
      )))

[-- Attachment #2: Type: text/html, Size: 4744 bytes --]

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Thu, 18 Nov 2021 05:17:47 +0530
> Cc: Alexander Adolf <alexander.adolf@condition-alpha.com>, Emacs Devel <emacs-devel@gnu.org>
> 
>  Please describe the intended use of these APIs and the hook to provide
>  the display feature(s) you have in mind, and how will that work in
>  concert with the display engine.  I don't think I have a clear picture
>  of that yet.
> 
>  Also, please describe what happens with use of these APIs when just a
>  small part of the Emacs display needs to be redrawn.
> 
> Maybe a demo ( https://youtu.be/1343m3LY4mo ) will help you visualize my suggestion. In the video, an
> event loop is started in the scratch buffer. All the edit operations update text and text property in a temporary
> buffer, say a.txt. Based on this information, in-memory SVG DOM is updated. Ultimately, the display engine
> updates the view with this SVG image. In cases like this, the mode can have full control over the display if so
> desired.
> 
> Since you have a problem accessing my repo, I've added the code of a function I call in my event loop at the
> end of this mail.
> 
> With a tighter integration, Emacs would define a new data structure {svg, x, y, w, h} and two new variables
> with this type - new_svg and prev_svg. Then the sequence of operations would be:
> - Display engine refreshes the view bitmap with current logic - full or partial refresh as the case may be.
> - If it's a graphical display
> - prev_svg = new_svg
> - Run display-svg-hook. The registered user function will call (svg-render) function passing required
> information. This API sets the value in new_svg.
> - If the bounding box of new_svg < the bounding box of prev_svg, display engine refreshes the area of
> prev_svg in the view bitmap.
> - Generate a bitmap for new_svg and superimpose it on the view bitmap.
> - Display the view bitmap to the user.

Thanks, but I'm not sure yet how to connect this to what the display
engine actually does.  Are you familiar with what happens in the
display code during a single redisplay cycle?  If so, could you
describe that and insert into that description what you say above?
That would go a long way towards making sure we are on the same page,
which I think is a necessary precondition for discussing your
proposal.

Failing that, I could try asking questions whose answers might help
reconstructing those missing links.  For example:

 . does this mean you expect the display engine to tell that
   display-svg-hook which parts of the display were redrawn?
 . what is the "required information" to be passed to
   display-svg-hook, and how will it be generated from the information
   that exists in the display engine?

Re: SVG hack for display engine

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 1272 bytes --]

> Thanks, but I'm not sure yet how to connect this to what the display
> engine actually does.  Are you familiar with what happens in the
> display code during a single redisplay cycle?  If so, could you
> describe that and insert into that description what you say above?
> That would go a long way towards making sure we are on the same page,
> which I think is a necessary precondition for discussing your
> proposal.
>
I admit I don't know about the internal working of the display engine.

Failing that, I could try asking questions whose answers might help
> reconstructing those missing links.  For example:
>
>  . does this mean you expect the display engine to tell that
>    display-svg-hook which parts of the display were redrawn?
>
No, that won't be necessary. After change hooks and other APIs are enough
to derive change and context information. In my experience, the characters
around the point are usually the most relevant.

 . what is the "required information" to be passed to
>    display-svg-hook, and how will it be generated from the information
>    that exists in the display engine?
>
 "required information" in my sentence above refers to the parameters of
(svg-render) API viz. svg x y width height. display-svg-hook can be a
no-arg function.

[-- Attachment #2: Type: text/html, Size: 1910 bytes --]

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Thu, 18 Nov 2021 15:43:01 +0530
> Cc: Alexander Adolf <alexander.adolf@condition-alpha.com>, Emacs Devel <emacs-devel@gnu.org>
> 
>   . does this mean you expect the display engine to tell that
>     display-svg-hook which parts of the display were redrawn?
> 
> No, that won't be necessary. After change hooks and other APIs are enough to derive change and context
> information. In my experience, the characters around the point are usually the most relevant.

But that's not generally true.  Redisplay has its own criteria to
decide which parts need to be redrawn, and those criteria are not
necessarily trivial to guess from buffer contents alone, because
redisplay has access to all kinds of information recorded specifically
for that purpose, and not exposed to Lisp.

>   . what is the "required information" to be passed to
>     display-svg-hook, and how will it be generated from the information
>     that exists in the display engine?
> 
>  "required information" in my sentence above refers to the parameters of (svg-render) API viz. svg x y width
> height. display-svg-hook can be a no-arg function.

What are X, Y, WIDTH and HEIGHT here?

And how will those hooks or svg-render know what to display an where
on the screen?

Re: SVG hack for display engine

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 1781 bytes --]

> But that's not generally true.  Redisplay has its own criteria to
> decide which parts need to be redrawn, and those criteria are not
> necessarily trivial to guess from buffer contents alone, because
> redisplay has access to all kinds of information recorded specifically
> for that purpose, and not exposed to Lisp.
>
Display engine must continue to do what it does presently to not break any
functionality in the terminal and elsewhere.  I'm only asking to extend the
functionality to allow a developer more freedom to play with SVG should he
desire.

>   . what is the "required information" to be passed to
> >     display-svg-hook, and how will it be generated from the information
> >     that exists in the display engine?
> >
> >  "required information" in my sentence above refers to the parameters of
> (svg-render) API viz. svg x y width
> > height. display-svg-hook can be a no-arg function.
>
> What are X, Y, WIDTH and HEIGHT here?
>
- svg-render (svg x y width height)
*svg* is a Lisp DOM object as returned by (dom-node). This should use
librsvg to generate a bitmap image of svg of *size (width x height)* and
superimpose it at *position (x, y)* on the current bitmap being displayed
by Emacs. The current image API takes either a filename or XML string for
SVG. This can be avoided if svg is a Lisp DOM. *Edit: (dom-node) *is
defined in dom.el .

And how will those hooks or svg-render know what to display an where
> on the screen?
>
Take some simple cases like a tooltip or an auto-completion popup.  They
usually display near the point with some offset. (posn-at-point) provides
that information. For complete window display control like my formula
editor, they can specify to display at (0,0) an image of size
(window-pixel-wdth) x (window-pixel-height).

[-- Attachment #2: Type: text/html, Size: 2445 bytes --]

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Thu, 18 Nov 2021 18:59:10 +0530
> Cc: Alexander Adolf <alexander.adolf@condition-alpha.com>, Emacs Devel <emacs-devel@gnu.org>
> 
>  But that's not generally true.  Redisplay has its own criteria to
>  decide which parts need to be redrawn, and those criteria are not
>  necessarily trivial to guess from buffer contents alone, because
>  redisplay has access to all kinds of information recorded specifically
>  for that purpose, and not exposed to Lisp.
> 
> Display engine must continue to do what it does presently to not break any functionality in the terminal and
> elsewhere.  I'm only asking to extend the functionality to allow a developer more freedom to play with SVG
> should he desire.

OK, but I still don't understand what you need from the display engine
for that purpose.

>  >   . what is the "required information" to be passed to
>  >     display-svg-hook, and how will it be generated from the information
>  >     that exists in the display engine?
>  > 
>  >  "required information" in my sentence above refers to the parameters of (svg-render) API viz. svg x y
>  width
>  > height. display-svg-hook can be a no-arg function.
> 
>  What are X, Y, WIDTH and HEIGHT here?
> 
> - svg-render (svg x y width height)
> svg is a Lisp DOM object as returned by (dom-node). This should use librsvg to generate a bitmap image of
> svg of size (width x height) and superimpose it at position (x, y) on the current bitmap being displayed by
> Emacs. The current image API takes either a filename or XML string for SVG. This can be avoided if svg is a
> Lisp DOM. Edit: (dom-node) is defined in dom.el .

I know that dom-node is defined in dom.el.  What I don't understand is
which code will generate a DOM, and based on what information.

>  And how will those hooks or svg-render know what to display an where
>  on the screen?
> 
> Take some simple cases like a tooltip or an auto-completion popup.  They usually display near the point with
> some offset. (posn-at-point) provides that information. For complete window display control like my formula
> editor, they can specify to display at (0,0) an image of size (window-pixel-wdth) x (window-pixel-height).

So this is only for popping up some additional image on top of the
"normal" Emacs display?  Or is the intent to replace some parts of the
"normal" Emacs display, i.e. what we show in the windows of the
regular frames?

And if this is just for popups, isn't there already some hook ypou
could use for that purpose?  If not, what is missing in the existing
hooks?

Re: SVG hack for display engine

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 1745 bytes --]

> So this is only for popping up some additional image on top of the
> "normal" Emacs display?  Or is the intent to replace some parts of the
> "normal" Emacs display, i.e. what we show in the windows of the
> regular frames?
>
> And if this is just for popups, isn't there already some hook ypou
> could use for that purpose?  If not, what is missing in the existing
> hooks?
>
Two problems in current approach:


> I've used (read-event) and (lookup-key) to allow for key translations for
> navigation commands. Hopefully with a tighter integration, all of this will
> not be required.
>
- Do you remember I said I am running my own event loop? This means I'm
missing out on the key translations that Emacs would normally handle for
me. So anything the developer hasn't coded for will remain inaccessible
while the mode is in effect.



> > - svg-render (svg x y width height)
> > svg is a Lisp DOM object as returned by (dom-node). This should use
> librsvg to generate a bitmap image of
> > svg of size (width x height) and superimpose it at position (x, y) on
> the current bitmap being displayed by
> > Emacs.
> *The current image API takes either a filename or XML string for SVG. This
> can be avoided if svg is a > Lisp DOM.* Edit: (dom-node) is defined in
> dom.el .
>
Can you visualize the amount of string serialization, deserialization and
SVG parsing happening in a draw or move operation with live feedback? Do
you see it performing well for even a moderately sized SVG?

Without direct SVG support in the display engine ie. a way for a developer
to tell the display engine to use a particular SVG  in a DOM form during
it's normal display cycle, any such implementation will remain a hack - it
works but it could be much better.

[-- Attachment #2: Type: text/html, Size: 2421 bytes --]

Re: SVG hack for display engine

[Eli Zaretskii]

> From: Anand Tamariya <atamariya@gmail.com>
> Date: Fri, 19 Nov 2021 07:02:20 +0530
> Cc: Alexander Adolf <alexander.adolf@condition-alpha.com>, Emacs Devel <emacs-devel@gnu.org>
> 
>  So this is only for popping up some additional image on top of the
>  "normal" Emacs display?  Or is the intent to replace some parts of the
>  "normal" Emacs display, i.e. what we show in the windows of the
>  regular frames?
> 
>  And if this is just for popups, isn't there already some hook ypou
>  could use for that purpose?  If not, what is missing in the existing
>  hooks?
> 
> Two problems in current approach:
>  
>  I've used (read-event) and (lookup-key) to allow for key translations for navigation commands.
>  Hopefully with a tighter integration, all of this will not be required.
> 
> - Do you remember I said I am running my own event loop? This means I'm missing out on the key
> translations that Emacs would normally handle for me. So anything the developer hasn't coded for will
> remain inaccessible while the mode is in effect.

Sorry, I don't follow: what do these issues have to do with what we
are discussing, and in general to the display engine?  What am I
missing?

>  > - svg-render (svg x y width height)
>  > svg is a Lisp DOM object as returned by (dom-node). This should use librsvg to generate a bitmap
>  image of
>  > svg of size (width x height) and superimpose it at position (x, y) on the current bitmap being displayed
>  by
>  > Emacs. The current image API takes either a filename or XML string for SVG. This can be
>  avoided if svg is a
>  > Lisp DOM. Edit: (dom-node) is defined in dom.el .
> 
> Can you visualize the amount of string serialization, deserialization and SVG parsing happening in a draw or
> move operation with live feedback? Do you see it performing well for even a moderately sized SVG?
> 
> Without direct SVG support in the display engine ie. a way for a developer to tell the display engine to use a
> particular SVG  in a DOM form during it's normal display cycle, any such implementation will remain a hack -
> it works but it could be much better. 

I understand that you want Emacs to be able to support display of an
SVG specified via a DOM.  What I'm trying to understand, and so far
failing, is where would that DOM come from, assuming the SVG will
display text of some buffer or string.  Can you explain the connection
between the text to be displayed and the DOM?

Re: SVG hack for display engine

[Alan Third]

On Fri, Nov 19, 2021 at 07:02:20AM +0530, Anand Tamariya wrote:
> > > - svg-render (svg x y width height)
> > > svg is a Lisp DOM object as returned by (dom-node). This should use
> > librsvg to generate a bitmap image of
> > > svg of size (width x height) and superimpose it at position (x, y) on
> > the current bitmap being displayed by
> > > Emacs.
> > *The current image API takes either a filename or XML string for SVG. This
> > can be avoided if svg is a > Lisp DOM.* Edit: (dom-node) is defined in
> > dom.el .
> >
> Can you visualize the amount of string serialization, deserialization and
> SVG parsing happening in a draw or move operation with live feedback? Do
> you see it performing well for even a moderately sized SVG?
> 
> Without direct SVG support in the display engine ie. a way for a developer
> to tell the display engine to use a particular SVG  in a DOM form during
> it's normal display cycle, any such implementation will remain a hack - it
> works but it could be much better.

I'm curious. I can't see any functionality in librsvg for loading an
SVG DOM directly without doing a round-trip into a string. Am I
missing something?

Does this idea rely on using some Cairo functionality?
-- 
Alan Third

Re: SVG hack for display engine

[Anand Tamariya]

[-- Attachment #1: Type: text/plain, Size: 995 bytes --]

>
> I'm curious. I can't see any functionality in librsvg for loading an
> SVG DOM directly without doing a round-trip into a string. Am I
> missing something?
>
Good point. I hope the following will perform better.

RsvgHandle <https://developer-old.gnome.org/rsvg/unstable/RsvgHandle.html> *
rsvg_handle_new_from_stream_sync (*GInputStream
<https://developer-old.gnome.org/gio/stable/GInputStream.html#GInputStream-struct>
*input_stream*,
                                  *GFile
<https://developer-old.gnome.org/gio/stable/GFile.html#GFile-struct>
*base_file*,
                                  *RsvgHandleFlags
<https://developer-old.gnome.org/rsvg/unstable/rsvg-Using-RSVG-with-GIO.html#RsvgHandleFlags>
flags*,
                                  *GCancellable
<https://developer-old.gnome.org/gio/stable/GCancellable.html#GCancellable-struct>
*cancellable*,
                                  *GError
<https://developer-old.gnome.org/glib/stable/glib-Error-Reporting.html#GError>
**error*);

[-- Attachment #2: Type: text/html, Size: 1847 bytes --]

Re: SVG hack for display engine

[Alan Third]

On Mon, Nov 22, 2021 at 08:18:31AM +0530, Anand Tamariya wrote:
> >
> > I'm curious. I can't see any functionality in librsvg for loading an
> > SVG DOM directly without doing a round-trip into a string. Am I
> > missing something?
> >
> Good point. I hope the following will perform better.
> 
> RsvgHandle * rsvg_handle_new_from_stream_sync (*GInputStream *input_stream*, *GFile *base_file*, *RsvgHandleFlags flags*,
> *GCancellable *cancellable*, *GError 
> **error*);

AFAICT that still reads in text, just in the form of a "stream".

-- 
Alan Third

Re: SVG hack for display engine

[Gerry Agbobada]

Hello,

On Wed, Nov 17, 2021, at 05:46, Po Lu wrote:
> FWIW I am working on a port of Emacs to GTK 4, which doesn't use Cairo
> (which has been deprecated by the GTK developers), instead using the
> native GSK scene graph kit to draw.  It does something very similar to
> what your SVG rendering would be like, keeping a tree of render nodes
> around past calls to gtk_widget_draw, and modifying it from the RIF.
>
> Getting it to work mostly correctly was extremely painful, and it still
> struggles with leaks (where obscured render nodes are not pruned
> correctly by xg_fixup_render_table) and the lack of the scroll
> optimization.
>
> (If someone wants details on this, wait a few weeks.  I will probably
> have it in a usable state by then.)
>
> Thanks.


Is there a forge/repository online where you push your work on GTK4, I wonder what the changes/work to do look like.


-- 
Gerry Agbobada