23.0.60; Image in before-string of overlay interferes with keymap

23.0.60; Image in before-string of overlay interferes with keymap

[Ralf Angeli]

As soon as an image is used in the before-string property of an overlay,
mouse clicks as defined in a keymap for the propertized string are not
recognized anymore.

In order to reproduce this, start Emacs with `emacs -Q', execute the
code snippet included below and click with mouse-1, mouse-2 or mouse-3
on the image in the new buffer.  If the `display' property is omitted
from the code below, clicking on the letter "A" after executing the code
will show "Mouse-1", "Mouse-2" or "Mouse-3" in the echo area, depending
on which mouse button was used when clicking on it.


(progn
  (pop-to-buffer "*foo*")
  (insert "xxx")
  (let ((ov (make-overlay 1 4))
	(map (make-sparse-keymap)))
    (define-key map [mouse-1] (lambda () (interactive) (message "Mouse-1")))
    (define-key map [mouse-2] (lambda () (interactive) (message "Mouse-2")))
    (define-key map [mouse-3] (lambda () (interactive) (message "Mouse-3")))
    (overlay-put ov 'before-string
		 (propertize
		  "A"
		  'keymap map
		  'display `(image :type xpm
				   :file ,(concat data-directory
						  "/images/attach.xpm"))
		  'mouse-face 'highlight))))



In GNU Emacs 23.0.60.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.12.7)
 of 2008-02-06 on photon
Windowing system distributor `The X.Org Foundation', version 11.0.10400090
configured using `configure  '--enable-font-backend''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
<mouse-1> <down-mouse-2> <mouse-2> C-_ <down-mouse-3> 
<mouse-3> C-x o C-p C-p C-p C-p C-f C-SPC C-n C-n C-n 
C-p C-p C-p C-SPC C-SPC C-n C-n C-n M-; C-x 0 C-x k 
<return> C-n C-b C-x C-e <down-mouse-1> <mouse-1> <down-mouse-2> 
<mouse-2> <down-mouse-3> <mouse-3> C-x 1 M-x r e p 
o r t - e m <tab> <return> C-g C-x k <return> C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p 
C-a C-SPC C-SPC C-n C-n C-n C-n M-w M-> <return> C-y 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p 
C-p C-p C-p C-n C-n C-n C-n C-n C-n C-p C-p C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-SPC C-SPC C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
M-w <return> C-y C-k C-k C-k C-k C-k C-k C-k C-k C-k 
C-k C-p C-p C-p C-p C-p C-p C-p C-b <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> d i s p l a y C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-b C-x C-e <down-mouse-1> 
<mouse-1> <down-mouse-2> <mouse-2> <down-mouse-3> <mouse-3> 
C-x k <return> C-x 0 C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-SPC C-SPC C-n C-n C-n M-; C-n C-n C-b C-x C-e <down-mouse-2> 
<mouse-2> C-x k <return> C-x 0 C-n C-n C-n C-n C-n 
C-n C-n C-n C-SPC M-> C-SPC C-SPC C-p C-p C-p C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-w 
C-p C-p C-p C-p C-SPC C-SPC C-n C-n C-n M-; M-< C-SPC 
C-M-f M-w M-x r e p o r t - e m a <tab> <return>

Recent messages:
Mouse-3
Mark set
Transient-mark-mode temporarily enabled
#("A" 0 1 (mouse-face highlight display (image :type xpm :file "/usr/src/emacs/etc//images/attach.xpm") keymap (keymap (mouse-3 lambda nil ... ...) (mouse-2 lambda nil ... ...) (mouse-1 lambda nil ... ...))))
Mouse-2
Mark set [3 times]
Transient-mark-mode temporarily enabled
Mark set
Transient-mark-mode temporarily enabled
Mark set [2 times]

-- 
Ralf

Re: 23.0.60; Image in before-string of overlay interferes with keymap

[Richard Stallman]

Would someone please fix this and ack?

Message-Id: <87fxw593iu.fsf@photon.caeruleus.net>
From: Ralf Angeli <angeli@caeruleus.net>
To: emacs-pretest-bug@gnu.org
Date: Wed, 06 Feb 2008 22:30:17 +0100
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: 
Subject: 23.0.60; Image in before-string of overlay interferes with keymap

As soon as an image is used in the before-string property of an overlay,
mouse clicks as defined in a keymap for the propertized string are not
recognized anymore.

In order to reproduce this, start Emacs with `emacs -Q', execute the
code snippet included below and click with mouse-1, mouse-2 or mouse-3
on the image in the new buffer.  If the `display' property is omitted
from the code below, clicking on the letter "A" after executing the code
will show "Mouse-1", "Mouse-2" or "Mouse-3" in the echo area, depending
on which mouse button was used when clicking on it.


(progn
  (pop-to-buffer "*foo*")
  (insert "xxx")
  (let ((ov (make-overlay 1 4))
	(map (make-sparse-keymap)))
    (define-key map [mouse-1] (lambda () (interactive) (message "Mouse-1")))
    (define-key map [mouse-2] (lambda () (interactive) (message "Mouse-2")))
    (define-key map [mouse-3] (lambda () (interactive) (message "Mouse-3")))
    (overlay-put ov 'before-string
		 (propertize
		  "A"
		  'keymap map
		  'display `(image :type xpm
				   :file ,(concat data-directory
						  "/images/attach.xpm"))
		  'mouse-face 'highlight))))



In GNU Emacs 23.0.60.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.12.7)
 of 2008-02-06 on photon
Windowing system distributor `The X.Org Foundation', version 11.0.10400090
configured using `configure  '--enable-font-backend''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
<mouse-1> <down-mouse-2> <mouse-2> C-_ <down-mouse-3> 
<mouse-3> C-x o C-p C-p C-p C-p C-f C-SPC C-n C-n C-n 
C-p C-p C-p C-SPC C-SPC C-n C-n C-n M-; C-x 0 C-x k 
<return> C-n C-b C-x C-e <down-mouse-1> <mouse-1> <down-mouse-2> 
<mouse-2> <down-mouse-3> <mouse-3> C-x 1 M-x r e p 
o r t - e m <tab> <return> C-g C-x k <return> C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p 
C-a C-SPC C-SPC C-n C-n C-n C-n M-w M-> <return> C-y 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p 
C-p C-p C-p C-n C-n C-n C-n C-n C-n C-p C-p C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-SPC C-SPC C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
M-w <return> C-y C-k C-k C-k C-k C-k C-k C-k C-k C-k 
C-k C-p C-p C-p C-p C-p C-p C-p C-b <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> d i s p l a y C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-b C-x C-e <down-mouse-1> 
<mouse-1> <down-mouse-2> <mouse-2> <down-mouse-3> <mouse-3> 
C-x k <return> C-x 0 C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-SPC C-SPC C-n C-n C-n M-; C-n C-n C-b C-x C-e <down-mouse-2> 
<mouse-2> C-x k <return> C-x 0 C-n C-n C-n C-n C-n 
C-n C-n C-n C-SPC M-> C-SPC C-SPC C-p C-p C-p C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-p C-w 
C-p C-p C-p C-p C-SPC C-SPC C-n C-n C-n M-; M-< C-SPC 
C-M-f M-w M-x r e p o r t - e m a <tab> <return>

Recent messages:
Mouse-3
Mark set
Transient-mark-mode temporarily enabled
#("A" 0 1 (mouse-face highlight display (image :type xpm :file "/usr/src/emacs/etc//images/attach.xpm") keymap (keymap (mouse-3 lambda nil ... ...) (mouse-2 lambda nil ... ...) (mouse-1 lambda nil ... ...))))
Mouse-2
Mark set [3 times]
Transient-mark-mode temporarily enabled
Mark set
Transient-mark-mode temporarily enabled
Mark set [2 times]

-- 
Ralf

Re: 23.0.60; Image in before-string of overlay interferes with keymap

[Ralf Angeli]

* Richard Stallman (2008-02-10) writes:

> Would someone please fix this and ack?
>
> From: Ralf Angeli <angeli@caeruleus.net>
> To: emacs-pretest-bug@gnu.org
> Date: Wed, 06 Feb 2008 22:30:17 +0100
> Subject: 23.0.60; Image in before-string of overlay interferes with keymap
>
> As soon as an image is used in the before-string property of an overlay,
> mouse clicks as defined in a keymap for the propertized string are not
> recognized anymore.

As additional info, AFAICS the following change introduced the bug:

2007-10-16  YAMAMOTO Mitsuharu  <mituharu@math.s.chiba-u.ac.jp>

	* xdisp.c (handle_display_prop): Ignore display specs after
	replacing one when string text is being replaced.
	(handle_single_display_spec): Pretend as if characters with display
	property haven't been consumed only when buffer text is being replaced.

-- 
Ralf

Re: 23.0.60; Image in before-string of overlay interferes with keymap

[Ralf Angeli]

* Ralf Angeli (2008-02-25) writes:

> * Richard Stallman (2008-02-10) writes:
>
>> Would someone please fix this and ack?
>>
>> From: Ralf Angeli <angeli@caeruleus.net>
>> To: emacs-pretest-bug@gnu.org
>> Date: Wed, 06 Feb 2008 22:30:17 +0100
>> Subject: 23.0.60; Image in before-string of overlay interferes with keymap
>>
>> As soon as an image is used in the before-string property of an overlay,
>> mouse clicks as defined in a keymap for the propertized string are not
>> recognized anymore.
>
> As additional info, AFAICS the following change introduced the bug:
>
> 2007-10-16  YAMAMOTO Mitsuharu  <mituharu@math.s.chiba-u.ac.jp>
>
> 	* xdisp.c (handle_display_prop): Ignore display specs after
> 	replacing one when string text is being replaced.
> 	(handle_single_display_spec): Pretend as if characters with display
> 	property haven't been consumed only when buffer text is being replaced.

Some more information ...

The original report and discussion which lead to the change can be found
here: <URL:http://thread.gmane.org/gmane.emacs.devel/80216>

The ironic part here is that while the reports Joe sent were concerned
with helping to fix Emacs to better support preview-latex and folding of
TeX macros, the resulting change actually broke preview-latex.

I could narrow down the problematic part of the change.  If the
following part of the patch (diff between revision 1.1170 and 1.1171 of
xdisp.c) is reverted, the test case I sent in my original report will
work again.

@@ -4245,7 +4260,8 @@
 	  /* Say that we haven't consumed the characters with
 	     `display' property yet.  The call to pop_it in
 	     set_iterator_to_next will clean this up.  */
-	  *position = start_pos;
+	  if (BUFFERP (object))
+	    it->current.pos = start_pos;
 	}
 #endif /* HAVE_WINDOW_SYSTEM */
 

The test case Joe sent in his report will work as well, but this is
likely only due to the fact that the part shown above is not related to
strings (where an analogous change was made.)

Could somebody please help me fix this before Emacs 22.2 is released?
I'd hate to see preview-latex broken until the successor of 22.2 will
be released.

-- 
Ralf

Re: 23.0.60; Image in before-string of overlay interferes with keymap

[YAMAMOTO Mitsuharu]

>>>>> On Thu, 28 Feb 2008 20:03:59 +0100, Ralf Angeli <angeli@caeruleus.net> said:

> I could narrow down the problematic part of the change.  If the
> following part of the patch (diff between revision 1.1170 and 1.1171 of
> xdisp.c) is reverted, the test case I sent in my original report will
> work again.

> @@ -4245,7 +4260,8 @@
>  	  /* Say that we haven't consumed the characters with
>  	     `display' property yet.  The call to pop_it in
>  	     set_iterator_to_next will clean this up.  */
> -	  *position = start_pos;
> +	  if (BUFFERP (object))
> +	    it->current.pos = start_pos;
>  	}
>  #endif /* HAVE_WINDOW_SYSTEM */
 
> The test case Joe sent in his report will work as well, but this is
> likely only due to the fact that the part shown above is not related to
> strings (where an analogous change was made.)

Nice observation.  I changed all the cases where a (overlay) string is
replaced as an effect of some `display' property (string, space, or
image) because that looks more consistent.  But that introduced the
problem you described.  The crucial case in Joe's report was
string-replacing-string one, so the other cases can be reverted.

> Could somebody please help me fix this before Emacs 22.2 is released?
> I'd hate to see preview-latex broken until the successor of 22.2 will
> be released.

I agree with you.  If nobody objects, I'll revert the changes about
the space and image cases in the EMACS_22_BASE branch.

				     YAMAMOTO Mitsuharu
				mituharu@math.s.chiba-u.ac.jp

Re: 23.0.60; Image in before-string of overlay interferes with keymap

[Ralf Angeli]

* YAMAMOTO Mitsuharu (2008-02-29) writes:

> I agree with you.  If nobody objects, I'll revert the changes about
> the space and image cases in the EMACS_22_BASE branch.

Thank you.  Any plan for the trunk?  It suffers from the same problem
because the change in the trunk was merged to it.

-- 
Ralf

Re: 23.0.60; Image in before-string of overlay interferes with keymap

[Stefan Monnier]

>> I agree with you.  If nobody objects, I'll revert the changes about
>> the space and image cases in the EMACS_22_BASE branch.

> Thank you.  Any plan for the trunk?  It suffers from the same problem
> because the change in the trunk was merged to it.

Hopefully, at some point, all regular contributors to Emacs will
understand that changes in old versions get merged into newer versions
as well.  I.e. all commits to the 22 branch will be merged into
the trunk.


        Stefan