There should be an option to set the display size of an image to zero

There should be an option to set the display size of an image to zero

[Marc Feeley]

Images added to a buffer with put-image always occupy an area in the buffer that is equal to their size.  It would be nice to allow the image to be overlaid over or under the text which would allow annotating the text with arrows, bubbles, etc. This could be as simple as indicating a zero width and height for the image (for determining the placing of text around it), but using the actual image size for displaying it on the screen.

Marc Feeley

Re: There should be an option to set the display size of an image to zero

[Eli Zaretskii]

> From: Marc Feeley <feeley@iro.umontreal.ca>
> Date: Mon, 16 Sep 2013 22:37:34 -0400
> 
> Images added to a buffer with put-image always occupy an area in the buffer that is equal to their size.

You mean, occupy an area _on_display_, not in the buffer, right?
Because images don't occupy any area in the buffer at all, they are
displayed instead of some buffer text which can be as short as 1
character, or even _before_ some text (in which case it doesn't
replace any characters).  See this note in the ELisp manual:

     Internally, [`put-image'] creates an overlay, and gives it a
     `before-string' property containing text that has a `display'
     property whose value is the image.

This arrangement of the overlay has the effect that the image is
displayed before the text at certain buffer position, without taking
any place in the buffer itself.

> It would be nice to allow the image to be overlaid over or under the text which would allow annotating the text with arrows, bubbles, etc. This could be as simple as indicating a zero width and height for the image (for determining the placing of text around it), but using the actual image size for displaying it on the screen.

The actual problem is not with the size of the image on screen, the
actual problem is with layout.  Emacs still lays out text and images
by horizontal "rows", top to bottom.  When it needs to display an
image, it enlarges the height of the row to accommodate the image,
displays the image, then goes on with displaying text.  If I
understand correctly what you are asking for, it would need a radical
change in the display layout algorithms.  Patches are welcome.

P.S.  If I misunderstood your suggestion, apologies, and please
elaborate.

Re: There should be an option to set the display size of an image to zero

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > If I understand correctly what you are asking for, it would need a
 > radical change in the display layout algorithms.

As he phrased it, yes.  But the use-case he described could be
accomodated fairly simply by blitting the image to the *background*
rather than treating it as a (more or less) giant character in the
foreground.  I think Emacs allows background pixmaps, doesn't it?
What I do (in XEmacs) when translating or inputting from a
photographed page image is to make that image the background pixmap,
then just type right over the text I'm working on.

Of course it wouldn't be particularly robust to scrolling and such.
To get a relatively robust version, you'd just need to be able to
"pin" rectangular objects to the background canvas, which shouldn't be
too hard.  Allowing you to pin those things relative to a particular
marker in text might be quite a bit more complex, though.

Re: There should be an option to set the display size of an image to zero

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Tue, 17 Sep 2013 16:19:48 +0900
> Cc: Marc Feeley <feeley@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> As he phrased it, yes.  But the use-case he described could be
> accomodated fairly simply by blitting the image to the *background*
> rather than treating it as a (more or less) giant character in the
> foreground.

I'm not sure we have that capability.  I may be wrong, though.

> I think Emacs allows background pixmaps, doesn't it?

I don't think so.  We only support :stipple.  Again, I know almost
nothing about this.

> What I do (in XEmacs) when translating or inputting from a
> photographed page image is to make that image the background pixmap,
> then just type right over the text I'm working on.
> 
> Of course it wouldn't be particularly robust to scrolling and such.

Right.  The Emacs redisplay never redraws the background, except when
it completely erases the frame.

> To get a relatively robust version, you'd just need to be able to
> "pin" rectangular objects to the background canvas, which shouldn't be
> too hard.  Allowing you to pin those things relative to a particular
> marker in text might be quite a bit more complex, though.

We don't have a canvas in Emacs, I think.

Re: There should be an option to set the display size of an image to zero

[Marc Feeley]

Let me explain the context so that you better understand my specific needs.  I want to use emacs as a programming tool to display the result of some program analyses of the code currently in the buffer.  Specifically I want to display arrows that indicate how data flows in the program.  For example, what are all the places in the code where the value resulting from a given constructor are referenced, or inversely, what are all the calls to constructors whose resulting value can flow to a particular point in the code.

My plan is to display arrows indicating these relationships and to overlay them on top of the program source code.  It would be ideal if it was possible to place an image on top of the text that scrolls with the text.  The arrows would only appear if the cursor (or mouse) is placed on top of a constructor or accessor, so it must be efficient to change the image dynamically.  An SVG image would be ideal because the image can easily be constructed and takes little space.

I've prototyped such an interface in JavaScript by using a web browser as a display engine.  However, editing is not easy to achieve.  I really want emacs for this.

How hard would it be to implement such a feature in emacs?

Here's a wild idea: has anyone compiled emacs using emscripten to allow emacs to run in a web browser?  It would then be easy to overlay graphics on top of the emacs text.

Marc


On 2013-09-17, at 3:31 AM, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: "Stephen J. Turnbull" <stephen@xemacs.org>
>> Date: Tue, 17 Sep 2013 16:19:48 +0900
>> Cc: Marc Feeley <feeley@iro.umontreal.ca>, emacs-devel@gnu.org
>> 
>> As he phrased it, yes.  But the use-case he described could be
>> accomodated fairly simply by blitting the image to the *background*
>> rather than treating it as a (more or less) giant character in the
>> foreground.
> 
> I'm not sure we have that capability.  I may be wrong, though.
> 
>> I think Emacs allows background pixmaps, doesn't it?
> 
> I don't think so.  We only support :stipple.  Again, I know almost
> nothing about this.
> 
>> What I do (in XEmacs) when translating or inputting from a
>> photographed page image is to make that image the background pixmap,
>> then just type right over the text I'm working on.
>> 
>> Of course it wouldn't be particularly robust to scrolling and such.
> 
> Right.  The Emacs redisplay never redraws the background, except when
> it completely erases the frame.
> 
>> To get a relatively robust version, you'd just need to be able to
>> "pin" rectangular objects to the background canvas, which shouldn't be
>> too hard.  Allowing you to pin those things relative to a particular
>> marker in text might be quite a bit more complex, though.
> 
> We don't have a canvas in Emacs, I think.

Re: There should be an option to set the display size of an image to zero

[Eli Zaretskii]

> From: Marc Feeley <feeley@iro.umontreal.ca>
> Date: Tue, 17 Sep 2013 08:50:45 -0400
> Cc: "Stephen J. Turnbull" <stephen@xemacs.org>,
>  emacs-devel@gnu.org
> 
> Let me explain the context so that you better understand my specific needs.  I want to use emacs as a programming tool to display the result of some program analyses of the code currently in the buffer.  Specifically I want to display arrows that indicate how data flows in the program.  For example, what are all the places in the code where the value resulting from a given constructor are referenced, or inversely, what are all the calls to constructors whose resulting value can flow to a particular point in the code.

I vaguely understand, but it is still not clear enough how you would
achieve those goals with "arrows".  Where will the arrows originate,
and where will they point?  Perhaps you could show a screenshot?

> My plan is to display arrows indicating these relationships and to overlay them on top of the program source code.  It would be ideal if it was possible to place an image on top of the text that scrolls with the text.  The arrows would only appear if the cursor (or mouse) is placed on top of a constructor or accessor, so it must be efficient to change the image dynamically.  An SVG image would be ideal because the image can easily be constructed and takes little space.

Did you consider using display margins?  Emacs can show display
strings and images in the display margins, to the left or right of the
text.  These display strings and images are "pinned" to specific
buffer positions, so they scroll with the text.

Since I don't have a clear idea of what you want to accomplish, I
cannot be sure this feature gives you what you need.  If they do, then
making them show when the mouse is at some specific place should not
be too hard, I think.

Re: There should be an option to set the display size of an image to zero

[Marc Feeley]

On 2013-09-17, at 10:02 AM, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Marc Feeley <feeley@iro.umontreal.ca>
>> Date: Tue, 17 Sep 2013 08:50:45 -0400
>> Cc: "Stephen J. Turnbull" <stephen@xemacs.org>,
>> emacs-devel@gnu.org
>> 
>> Let me explain the context so that you better understand my specific needs.  I want to use emacs as a programming tool to display the result of some program analyses of the code currently in the buffer.  Specifically I want to display arrows that indicate how data flows in the program.  For example, what are all the places in the code where the value resulting from a given constructor are referenced, or inversely, what are all the calls to constructors whose resulting value can flow to a particular point in the code.
> 
> I vaguely understand, but it is still not clear enough how you would
> achieve those goals with "arrows".  Where will the arrows originate,
> and where will they point?  Perhaps you could show a screenshot?

Here's a screenshot showing the type of annotation I'm trying to implement:

http://planet.racket-lang.org/package-source/mflatt/scribble-paper.plt/2/1/planet-docs/scribble/collatz.png

The screenshot is from the racket Scheme implementation which supports arrow annotations.

>> My plan is to display arrows indicating these relationships and to overlay them on top of the program source code.  It would be ideal if it was possible to place an image on top of the text that scrolls with the text.  The arrows would only appear if the cursor (or mouse) is placed on top of a constructor or accessor, so it must be efficient to change the image dynamically.  An SVG image would be ideal because the image can easily be constructed and takes little space.
> 
> Did you consider using display margins?  Emacs can show display
> strings and images in the display margins, to the left or right of the
> text.  These display strings and images are "pinned" to specific
> buffer positions, so they scroll with the text.

But can the image in a margin overflow the margin to overlay the main text?  I assume not.

Marc

Re: There should be an option to set the display size of an image to zero

[Eli Zaretskii]

> From: Marc Feeley <feeley@iro.umontreal.ca>
> Date: Tue, 17 Sep 2013 10:43:47 -0400
> Cc: stephen@xemacs.org,
>  emacs-devel@gnu.org
> 
> Here's a screenshot showing the type of annotation I'm trying to implement:
> 
> http://planet.racket-lang.org/package-source/mflatt/scribble-paper.plt/2/1/planet-docs/scribble/collatz.png

I don't think this is possible in Emacs at this time.

But since you want the arrow to appear only when the mouse pointer
hovers above one of the places that are involved in a relationship,
how about highlighting those places in some way, when mouse is over
one of them, without actually drawing an arrow?

> But can the image in a margin overflow the margin to overlay the main text?  I assume not.

No, it cannot.  It will be displayed entirely in the margin.

Re: There should be an option to set the display size of an image to zero

[Marc Feeley]

On 2013-09-17, at 12:10 PM, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Marc Feeley <feeley@iro.umontreal.ca>
>> Date: Tue, 17 Sep 2013 10:43:47 -0400
>> Cc: stephen@xemacs.org,
>> emacs-devel@gnu.org
>> 
>> Here's a screenshot showing the type of annotation I'm trying to implement:
>> 
>> http://planet.racket-lang.org/package-source/mflatt/scribble-paper.plt/2/1/planet-docs/scribble/collatz.png
> 
> I don't think this is possible in Emacs at this time.
> 
> But since you want the arrow to appear only when the mouse pointer
> hovers above one of the places that are involved in a relationship,
> how about highlighting those places in some way, when mouse is over
> one of them, without actually drawing an arrow?

That's a solution, but it has its problems.  For one, you can only see the highlighted parts that are in the visible part of the buffer.  With arrows, you can see that the arrow goes "off the window", so you get a clue that you should follow the arrow to see what it points to.

> 
>> But can the image in a margin overflow the margin to overlay the main text?  I assume not.
> 
> No, it cannot.  It will be displayed entirely in the margin.

Ok.

Oh well.  I guess a web-browser solution will have to do for now.

Marc

Re: There should be an option to set the display size of an image to zero

[chad]

[-- Attachment #1: Type: text/plain, Size: 440 bytes --]


On 17 Sep 2013, at 09:15, Marc Feeley <feeley@iro.umontreal.ca> wrote:

> That's a solution, but it has its problems.  For one, you can only see the highlighted parts that are in the visible part of the buffer.  With arrows, you can see that the arrow goes "off the window", so you get a clue that you should follow the arrow to see what it points to.

Is it possible to create a tooltip window with a transparent background?

~Chad

[-- Attachment #2: Type: text/html, Size: 1596 bytes --]

Re: There should be an option to set the display size of an image to zero

[Stefan Monnier]

[ Some context: Marc is my office neighbor.  ]

> I don't think this is possible in Emacs at this time.

Yes, we know.  The question is how feasible it would be to add to the
C code some kind of support for such a feature.


        Stefan

Re: There should be an option to set the display size of an image to zero

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Marc Feeley <feeley@iro.umontreal.ca>, stephen@xemacs.org,
>  emacs-devel@gnu.org
> Date: Tue, 17 Sep 2013 17:57:36 -0400
> 
> > I don't think this is possible in Emacs at this time.
> 
> Yes, we know.

The OP didn't seem to know.

> The question is how feasible it would be to add to the C code some
> kind of support for such a feature.

The feature was not defined well enough for me to answer this
question, sorry.  If indeed the feature needs a complete redesign and
rewrite of the display layout code, it is IMO not feasible, unless
Someone™ with enough motivation and resources steps forward and
offers to do the job.  I'm not that person.

You know how Emacs lays out its display (one "display element" at a
time, left to right, top to bottom), and (Marc being your neighbor)
you can probably discuss possible ways to express on the screen what
he has in mind.  So if you can suggest an idea of implementing
something that would not require such a complete redesign, I can try
assessing its feasibility.

One possible idea -- not sure if it fits the purpose -- is to overlay
transparent-background images on top of the displayed text, not unlike
what we do with mouse pointers.  For this, I'd rather ask Jan to
comment, as this kind of graphics, most of which should be implemented
in the terminal-dependent display back-end, is way out of my area of
expertise.

Re: There should be an option to set the display size of an image to zero

[Jan Djärv]

Hello.

18 sep 2013 kl. 08:56 skrev Eli Zaretskii <eliz@gnu.org>:

> One possible idea -- not sure if it fits the purpose -- is to overlay
> transparent-background images on top of the displayed text, not unlike
> what we do with mouse pointers.  For this, I'd rather ask Jan to
> comment, as this kind of graphics, most of which should be implemented
> in the terminal-dependent display back-end, is way out of my area of
> expertise.

For the backends it is just a matter of putting an image glyph string at the correct coordinates.  I don't know if transparent images has been tested well, but in theory that should be handeled already.

So the display engine needs to make the appropriate image glyph string, and send them to the backend after the text has been drawn.

	Jan D.

Re: There should be an option to set the display size of an image to zero

[Stefan Monnier]

> One possible idea -- not sure if it fits the purpose -- is to overlay
> transparent-background images on top of the displayed text, not unlike
> what we do with mouse pointers.

That's the idea, I think, yes.

Combined with posn-point which lets us find the XY coordinates of
a particular buffer position, it should be possible to build (in Elisp)
the right SVG image and tell Emacs to overlay it on top the text.

Ideally, the primitive could look like a `display' spec which says
"overlay the image at offset XY relative to the current position".
But if it's too difficult to handle it that way, a simpler (from the
C code PoV) approach might be a window property which says "overlay this
image over the whole window's text".


        Stefan

Re: There should be an option to set the display size of an image to zero

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Date: Wed, 18 Sep 2013 12:10:44 -0400
> Cc: feeley@IRO.UMontreal.CA, stephen@xemacs.org, emacs-devel@gnu.org
> 
> > One possible idea -- not sure if it fits the purpose -- is to overlay
> > transparent-background images on top of the displayed text, not unlike
> > what we do with mouse pointers.
> 
> That's the idea, I think, yes.
> 
> Combined with posn-point which lets us find the XY coordinates of
> a particular buffer position

It's the other way around: given XY coordinates, it finds buffer
position at those coordinates.  I guess you meant posn-at-point
instead.

> it should be possible to build (in Elisp) the right SVG image and
> tell Emacs to overlay it on top the text.

I believe you; I know nothing about SVG, except that it is a vector
image and is specified in XML.

> Ideally, the primitive could look like a `display' spec which says
> "overlay the image at offset XY relative to the current position".

I guess you mean "relative to specified POSITION", because there's
only one current position, which sounds too restrictive.

> But if it's too difficult to handle it that way, a simpler (from the
> C code PoV) approach might be a window property which says "overlay this
> image over the whole window's text".

I think the difficulty is the same: we need a separate display
operation invoked when the redisplay cycle is completed.  We also need
to figure out what to do when redisplay is preempted before it
completes.  Determining the anchor position, relative to which the
image is displayed, could also need some thought, because that
position might not be visible.

Re: There should be an option to set the display size of an image to zero

[Stefan Monnier]

>> Ideally, the primitive could look like a `display' spec which says
>> "overlay the image at offset XY relative to the current position".
> I guess you mean "relative to specified POSITION", because there's
> only one current position, which sounds too restrictive.

Since a display spec is applied over a particular chunk of text in the
buffer, by "current position" I meant "the place where the chunk of
text covered by the display spec is (or would be) drawn".

>> But if it's too difficult to handle it that way, a simpler (from the
>> C code PoV) approach might be a window property which says "overlay this
>> image over the whole window's text".
> I think the difficulty is the same:

If it's the same, then it would be more useful for it to be a display
spec, so it moves with the buffer's text.


        Stefan