Re: No possibility of determining image position/object from click

Re: No possibility of determining image position/object from click

[Richard Stallman]

    GNU Emacs is missing an API for finding out where a click occured.  If
    I have a string or an image as part of a display property, or as a
    before-string or after-string, the only way to let Emacs react to
    clicks on different images differently is to generate individual
    keymaps for each of them.

I see.  All it provides is the buffer position, which won't tell
you the position within the before-string.

I think we should replace the buffer position with a list that gives
the full story.

      Even then, you cannot find out the
    relative coordinates of a click on such an image.

It does give the x and y pixel coordinates of the click.  If you can
determine the coordinates of a corner of the image, you can subtract
and get the relative position.  But I am not sure there is any way to
determine the corner's coordinates, if the image comes from a
before-string.

In principle, these are good features to add.  Do you have an
immediate practical application for them?

Re: No possibility of determining image position/object from click

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     GNU Emacs is missing an API for finding out where a click occured.  If
>     I have a string or an image as part of a display property, or as a
>     before-string or after-string, the only way to let Emacs react to
>     clicks on different images differently is to generate individual
>     keymaps for each of them.

I have also encountered another "bug" in connection with that for
which an actual example is pretty hard to produce: if a displayed
window changes asynchronously because a sentinel is placing image
properties on screen, and Emacs is basically busy without getting to
redisplay, then tooltip displays (for example) will still pop up for
whatever the mouse cursor is covering, as a user would expect.  If one
does an actual _click_ on such an image, the positions(?) but at least
the keymaps that get consulted for this click are the keymaps that
belong to the display _after_ Emacs updates the screen.  This means,
for example, that you were sure to click on an image in the buffer
(and display and corresponding tooltip suggested you would), but if
Emacs actually gets to look at the click, it will instead yank the cut
buffer into a different location than that at the time of the click.

I am not sure this would be easy to fix: it would probably imply
having to associate things of relevance (object clicked on, buffer
position and a few other things) to an event at the time of the
click, or at least before redisplay destroys the information about
the screen state at the time of the event.

> I see.  All it provides is the buffer position, which won't tell
> you the position within the before-string.

Also, if a larger body of text is displayed as an image by way of a
'display property, the buffer position does not buy me anything (it
will probably be that of the first letter of the text).  This is
quite similar to the before-string problem, except that in that case
one does not have a string-object easily available with which to
associate.

> I think we should replace the buffer position with a list that gives
> the full story.
> 
>       Even then, you cannot find out the
>     relative coordinates of a click on such an image.
> 
> It does give the x and y pixel coordinates of the click.  If you can
> determine the coordinates of a corner of the image,

at the time of the click...

> you can subtract and get the relative position.  But I am not sure
> there is any way to determine the corner's coordinates, if the image
> comes from a before-string.

At least I have failed to see one.

> In principle, these are good features to add.  Do you have an
> immediate practical application for them?

Well, as you should remember from the amounts of display engine
related bug reports I sent in before Emacs-21.1 got released, I have
employed Emacs-21's new display engine for providing WYSIWYG-like
editing of LaTeX compositions (formulas, section headings, figures and
other configurable stuff).  Info/homepage/downloads at
<URL:http://preview-latex.sourceforge.net>.  Currently, we trace
clicks back to single images by giving each of them an individual
keymap with a keybinding to a quoted function call including the
image overlay.  It would be quite more convenient if all images could
just get the same keybinding.

A future plan is to provide relative positioning: click close to the
bottom of such an image, and you get placed close to the lower border
of the source text.  Another would be table editing: those are
currently one big image.  I could get the LaTeX component of
preview-latex to deliver the relative pixel coordinates in such a
table to Emacs, so that I can reliably edit a particular table cell
in the source code when the image gets clicked.

All of those refinements would require the relative position of the
click on the image.  The keymap tomfoolery has made it at least
possible to identify the image itself reliably, but the position
appears quite out of reach for me at the current point of time.

Would you be interested in getting info about the functions XEmacs
uses for reporting those properties of events, so that programmers of
external packages will have less work in order to make their stuff
work under both Emacsen?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum
Email: David.Kastrup@t-online.de

Re: No possibility of determining image position/object from click

[Richard Stallman]

      If one
    does an actual _click_ on such an image, the positions(?) but at least
    the keymaps that get consulted for this click are the keymaps that
    belong to the display _after_ Emacs updates the screen.  This means,
    for example, that you were sure to click on an image in the buffer
    (and display and corresponding tooltip suggested you would), but if
    Emacs actually gets to look at the click, it will instead yank the cut
    buffer into a different location than that at the time of the click.

I am trying to read this and understand it, but having trouble parsing
it.  I can't really grasp the sequence of user actions and Emacs
behaviors.

It seems correct that Emacs uses the latest data to interpret a click;
what else should it do?  If the text is changing rapidly, in general
you are taking a risk by clicking on it, since what you intended to
click on could have moved or changed by the time the mouse button
makes contact.  But I don't understand enough to say that there is no
bug here.  I am not sure if this is a bug, or an unfortunate
consequence of correct Emacs actions, or just what the user deserves.

    Also, if a larger body of text is displayed as an image by way of a
    'display property, the buffer position does not buy me anything (it
    will probably be that of the first letter of the text).

The buffer position tells you what image was clicked on.  Doesn't that
suffice for your immediate goal, which is to use just one keymap for
all the images?

We could add the relative position within in the image to the event,
and we could add other things to the event too.  Would someone like to
propose an upward compatible data representation that does the whole
job?

Re: No possibility of determining image position/object from click

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>       If one
>     does an actual _click_ on such an image, the positions(?) but at least
>     the keymaps that get consulted for this click are the keymaps that
>     belong to the display _after_ Emacs updates the screen.  This means,
>     for example, that you were sure to click on an image in the buffer
>     (and display and corresponding tooltip suggested you would), but if
>     Emacs actually gets to look at the click, it will instead yank the cut
>     buffer into a different location than that at the time of the click.
> 
> I am trying to read this and understand it, but having trouble parsing
> it.  I can't really grasp the sequence of user actions and Emacs
> behaviors.
> 
> It seems correct that Emacs uses the latest data to interpret a click;
> what else should it do?

Emacs uses data to interpret a click that has not yet made it to the
screen at the moment of the click.

> If the text is changing rapidly, in general you are taking a risk by
> clicking on it, since what you intended to click on could have moved
> or changed by the time the mouse button makes contact.

This is not the problem, since this would just introduce a small
time window for thwarting the user.  But Emacs does not update the
screen while it is busy, which can easily last for a few seconds.  If
I click onto the screen during that time, buffer positions and
keymaps will be calculated corresponding to what the screen will look
like after the next screen update finally gets done.  This is hardly
what the user intended when clicking onto something.  Of course,
there is a problem for Emacs if the object displayed at the time of
the click has already been deleted, but this deletion has not yet
been reflected by a display update.

>     Also, if a larger body of text is displayed as an image by way
>     of a 'display property, the buffer position does not buy me
>     anything (it will probably be that of the first letter of the
>     text).
> 
> The buffer position tells you what image was clicked on.

No.  I might have more than one image at a specific buffer position, I
might have overlays not covering any character.  The buffer position
will not tell me whether I clicked on the image of an empty overlay,
or on the character following it, or on which image from several
overlays at the same character position, or on some before-string or
after-string property of an overlay.

> Doesn't that suffice for your immediate goal, which is to use just
> one keymap for all the images?

Not really as well as to warrant changing the current behavior.

> We could add the relative position within in the image to the event,
> and we could add other things to the event too.  Would someone like
> to propose an upward compatible data representation that does the
> whole job?

I would guess the internal structure of events need not be so very
much upward compatible as long as the interface routines to it (like
posn-xy or similar) returned the same values as before.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum
Email: David.Kastrup@t-online.de

Re: No possibility of determining image position/object from click

[Eli Zaretskii]

> From: David.Kastrup@t-online.de
> Date: 15 Jun 2002 16:34:25 +0200
> 
> 
> Emacs does not update the
> screen while it is busy, which can easily last for a few seconds.  If
> I click onto the screen during that time, buffer positions and
> keymaps will be calculated corresponding to what the screen will look
> like after the next screen update finally gets done.

How is this different from the situation where a user types C-w in
the middle of a lengthy calculation which will eventually change the
text inside the region (and thus the result of C-w).

In other words, typing ahead when Emacs is busy always runs a risk of
doing the wrong thing.  Am I missing something?

Re: No possibility of determining image position/object from click

[David Kastrup]

"Eli Zaretskii" <eliz@is.elta.co.il> writes:

> > From: David.Kastrup@t-online.de
> > Date: 15 Jun 2002 16:34:25 +0200
> > 
> > 
> > Emacs does not update the
> > screen while it is busy, which can easily last for a few seconds.  If
> > I click onto the screen during that time, buffer positions and
> > keymaps will be calculated corresponding to what the screen will look
> > like after the next screen update finally gets done.
> 
> How is this different from the situation where a user types C-w in
> the middle of a lengthy calculation which will eventually change the
> text inside the region (and thus the result of C-w).
> 
> In other words, typing ahead when Emacs is busy always runs a risk of
> doing the wrong thing.  Am I missing something?

When clicking on something, we establish a visual correlation between
the object we are clicking on and the mouse pointer, not between the
naked screen position and the mouse pointer.  That is, when we are
queing keyboard events, we are conceptually queuing
"process C-x C-w"
"process C-d" ...

With mouse clicks, we are conceptually queuing "click on object/buffer
position ...", not "click on screen pixel coordinates ...".  The
perceived discrepancy in processing is quite disconcerting.

During such busy calculations, for example tooltips are still being
processed, and they are processed taking into account the look of the
buffer _before_ redisplay (which is the only thing that makes sense
to the user, anyhow), at least that is my impression.  The tooltip in
my case then promises something like "[button-2] toggles preview",
but if you press the button, and this gets queued, and then some
other stuff gets updated in a process sentinel or wherever else
before the event gets processed, something quite different from what
the tooltip promised will happen.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum
Email: David.Kastrup@t-online.de

Re: No possibility of determining image position/object from click

[Richard Stallman]

    This is not the problem, since this would just introduce a small
    time window for thwarting the user.  But Emacs does not update the
    screen while it is busy, which can easily last for a few seconds.  If
    I click onto the screen during that time, buffer positions and
    keymaps will be calculated corresponding to what the screen will look
    like after the next screen update finally gets done.

Now I understand what the issue is.  I agree this is a real issue.

If we solve your other problem, of reporting which object (and which
part of it) the click occurred on, in the input event, that could
enable you to avoid this problem--right?

So I think the next item of business is to draw up a precise proposal for
what added information to include in the event, and in what format.