Issues with X selection handling

Issues with X selection handling

[Davis Herring]

[Emacs version: 21.3.1 of May 22 2003, although I see few relevant 
changes in CVS]
I have been experimenting with different semantics for handling X 
selections (particularly `PRIMARY'), and have run across several snags in 
the x-*-selection-* functions and hooks:

 + `x-disown-selection-internal' doesn't seem to do anything.
 + `x-lost-selection-hooks' is not always executed when one would expect:
  - using the mouse to select text in Emacs and then elsewhere seems to 
never call the hooks
  - losing the clipboard usually (but not always) calls the hooks
  - explicitly owning a selection with M-: (x-own-selection-internal
'PRIMARY "foo") seems to cause the hooks to be called (later)
  - killing text with the keyboard also seems to guarantee hooks
 + Occasionally, Emacs itself (i.e. xselect.c) seems to miss a
notification that it has lost the selection -- I have, once, seen Emacs
insist that it owns the clipboard selection despite the fact that other
applications are owning and using it.

There may be some sort of race condition in xselect.c, since (if I read it
correctly) it uses 1-second-resolution time values to decide on the
relevance of X events.  But most of the time, `Vselection-alist' has the
correct values (as reported by the x-get-* methods) even when the Lisp
hooks aren't called.

I am unfortunately sufficiently familiar with neither X nor Emacs' C to
properly diagnose (much less correct) these problems; however, I have
attached an Emacs Lisp file with X-selection investigative tools;
hopefully it will be of use in discovering and/or testing fixes for the
bug(s).  Could someone please have a look at this?

Thanks in advance,
Davis Herring

-- 
This product is sold by volume, not by mass.  If it seems too dense or too 
sparse, it means mass-energy conversion has occurred during shipping.

Re: Issues with X selection handling

[Jan D.]

Davis Herring wrote:
> [Emacs version: 21.3.1 of May 22 2003, although I see few relevant 
> changes in CVS]
> I have been experimenting with different semantics for handling X 
> selections (particularly `PRIMARY'), and have run across several snags in 
> the x-*-selection-* functions and hooks:
> 
>  + `x-disown-selection-internal' doesn't seem to do anything.

Well, it disowns the selection, but it does not call any hooks if that is what
you mean.  Can you please say what you expected it to do?

>  + `x-lost-selection-hooks' is not always executed when one would expect:
>   - using the mouse to select text in Emacs and then elsewhere seems to 
> never call the hooks

I have not been able to reproduce this with the CVS version or 21.3.
The lost and send hooks are always called, for PRIMARY and CLIPBOARD.
Can you test the CVS version?

>   - losing the clipboard usually (but not always) calls the hooks
>   - explicitly owning a selection with M-: (x-own-selection-internal
> 'PRIMARY "foo") seems to cause the hooks to be called (later)
>   - killing text with the keyboard also seems to guarantee hooks
>  + Occasionally, Emacs itself (i.e. xselect.c) seems to miss a
> notification that it has lost the selection -- I have, once, seen Emacs
> insist that it owns the clipboard selection despite the fact that other
> applications are owning and using it.

We really need a solid test case that always fails to be able to find any bug
in this area.

	Jan D.

Re: Issues with X selection handling

[Davis Herring]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4109 bytes --]

[These experiments conducted on 21.3.1 on GNU/Linux with XFree86.]

On Thu, 19 Aug 2004, Jan D. wrote:

> >  + `x-disown-selection-internal' doesn't seem to do anything.
> 
> Well, it disowns the selection, but it does not call any hooks if that is what
> you mean.  Can you please say what you expected it to do?

I see -- it does disown the selection, but not only are no hooks called
(the docstring for `x-lost-selection-hooks' explicitly says it should be
called), but the entry is not removed from the internal variable
`Vselection-alist'.  This means Emacs (in Lisp) thinks it still owns the 
selection and that it has the value it had before the disownment.

> >  + `x-lost-selection-hooks' is not always executed when one would expect:
> >   - using the mouse to select text in Emacs and then elsewhere seems to 
> > never call the hooks
> 
> I have not been able to reproduce this with the CVS version or 21.3.
> The lost and send hooks are always called, for PRIMARY and CLIPBOARD.
> Can you test the CVS version?

Unfortunately I can't test the CVS version -- at least, not quickly.  
Beyond that I don't know what to say to this, except to try out the 
attached test code (I've updated it somewhat since my prior postings).  
What I'm trying now is running two Emacsen with the same 
continuous-sampling setup, and alternating between them making selections.

They disagree a notable amount of the time, and `x-lost-selection-hooks' 
is never called with PRIMARY as an argument unless the keyboard is used to 
kill text.  Meanwhile CLIPBOARD appears to call the hooks every time.  
Stranger still is that occasionally one Emacs generates messages in the 
other that look like this:

Sent selection at Mon Aug 23 18:57:44 2004: (CLIPBOARD nil nil)
Sent selection at Mon Aug 23 18:57:44 2004: (CLIPBOARD nil nil)
Sent selection at Mon Aug 23 18:57:44 2004: (PRIMARY nil nil)
Sent selection at Mon Aug 23 18:57:44 2004: (PRIMARY nil nil)

The Emacs requesting the selection isn't sending any type with its 
request, apparently; it gets declined, and decides that there is no such 
selection.

As for the mouse-selection, a mouse-drag after a keyboard event doesn't
even call `post-command-hook' until the next event is received (at which
point the hook is called twice); if I mouse-drag, then tap a key, and
-then- select text elsewhere, the lost-hook is called as I expect.  So
there may be a race between the "completion" of the mouse-drag and the
reception of the "selection stolen" event.  This would explain the 
bizarre "keyboard-dependence".

Of course, it seems to me as though the very delay of such "completion"  
of a mouse command is a much worse problem.  Is there some reason for 
this?

> >   - losing the clipboard usually (but not always) calls the hooks

As an unexpectedly final test, aiming to decide this issue, I 
mouse-selected text in one Emacs, then immediately went to the other and 
evaluated Lisp to grab only the clipboard.  Both Emacsen immediately 
froze.  This, I suppose, is a separate issue worthy of consideration.

The CLIPBOARD/PRIMARY asymmetry is surely some sort of indication as to 
the nature of the bug (since there's no real reason for the selections to 
behave differently), but it might be something as trivial as "the first 
event is lost, and that's usually PRIMARY".

> >  + Occasionally, Emacs itself (i.e. xselect.c) seems to miss a
> > notification that it has lost the selection -- I have, once, seen Emacs
> > insist that it owns the clipboard selection despite the fact that other
> > applications are owning and using it.

This must have been the result of manually disowning a selection; I see
now that this isn't a separate issue.

> We really need a solid test case that always fails to be able to find any bug
> in this area.

All these behaviors are pretty deterministic as far as I can tell.  
Hopefully this additional information (and updated testing code) is 
helpful.

Davis Herring

-- 
This product is sold by volume, not by mass.  If it seems too dense or too 
sparse, it means mass-energy conversion has occurred during shipping.

[-- Attachment #2: X Selection Test Code --]
[-- Type: TEXT/PLAIN, Size: 2655 bytes --]

;; [From my excessive .emacs: used for cut-buffers below.]
(require 'cl)
(defmacro table (var count form)
	"Evaluate FORM COUNT times, with VAR bound to integers from 0 to COUNT-1.
Return a list of the results obtained.
If VAR is `nil', bind nothing, just loop."
	(let ((listsym (gensym "table::list"))
				(tailsym (gensym "table::tail"))
				(iterator (or var (gensym "table::iterator")))
				(countsym (gensym "table::count")))
		`(let* ((,iterator -1) (,countsym ,count)
						(,listsym (list nil)) (,tailsym ,listsym))
			 (while (< (setq ,iterator (1+ ,iterator)) ,countsym)
				 (setq ,tailsym (setcdr ,tailsym (list ,form))))
			 (cdr ,listsym))))

(defvar x-selection-errors nil)

(defun x-get-selection-no-error (sel type)
	(condition-case e (x-get-selection-internal sel type)
		(error (add-to-list 'x-selection-errors e t) nil)))

(defun x-selection-report (s)
	(list s (x-selection-owner-p s)
				(let ((ct (x-get-selection-no-error s 'COMPOUND_TEXT))
							(str (x-get-selection-no-error s 'STRING)))
					(if ct (list 'COMPOUND_TEXT ct) (list 'STRING str)))))

(defvar x-selection-counter 0)

(defun x-describe-selections (&rest ignore)	; allow use on any hook
	(interactive)
	(with-current-buffer (get-buffer-create "*X Selections*")
		(erase-buffer)
		(pp (list (list (format "%-3s"
														(make-string (setq x-selection-counter
																							 (% (1+ x-selection-counter) 4)) ?*))
										(input-pending-p))
							(x-selection-report 'CLIPBOARD)
							(x-selection-report 'PRIMARY)
							(x-selection-report 'SECONDARY)
							(table i 8 (cons i (x-get-cut-buffer i)))) (current-buffer))))

;; Returns a lambda which pretty-prints list of its arguments with a header
(defun pp-lambda (buf str)
	`(lambda (&rest args)
		 (with-current-buffer (get-buffer-create ,buf)
			 (goto-char (point-max))
			 (insert ,str " at " (current-time-string) ": ")
			 (pp args (current-buffer)))))

(add-hook 'post-command-hook 'x-describe-selections)
(setq x-selection-timer (run-with-timer 0 0.25 'x-describe-selections))
(add-hook 'x-sent-selection-hooks
					(pp-lambda "*X Selection News*" "Sent selection"))
(add-hook 'x-lost-selection-hooks
					(pp-lambda "*X Selection News*" "Lost selection"))
(add-hook 'x-sent-selection-hooks 'x-describe-selections)
(add-hook 'x-lost-selection-hooks 'x-describe-selections)

;; For use with `eval-last-sexp'
;; (x-own-selection-internal 'CLIPBOARD "foo")
;; (x-own-selection-internal 'PRIMARY "bar")
;; (x-disown-selection-internal 'CLIPBOARD)
;; (x-disown-selection-internal 'PRIMARY)
;; (cancel-timer x-selection-timer)

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: Issues with X selection handling

[Davis Herring]

...Ok, an update; after sending the previous mail, the Emacs sessions
responded to my C-g, albeit rather fitfully.  They eventually decided that
they each owned one selection (the one where I manually grabbed the
clipboard did think it owned CLIPBOARD, and the other thought it (still)
owned PRIMARY)  and that -no one- owned the other.

Checking with an external program revealed that there was no selection of 
either type active.

Both Emacsen recovered with the message "condition-case: Quit", but who
knows which `condition-case' that was.  So I set `debug-on-quit' and
managed to repeat the lock, this time getting a traceback that showed
`x-get-selection-internal' calling `x-sent-selection-hooks'; this caused
an infinite recursion because my selection-reporting code (which obviously
calls `x-get-selection-internal') was on that hook.  The arguments to the
hook function alternated between "CLIPBOARD COMPOUND_TEXT t" and
"CLIPBOARD STRING t".

I don't see how `x-get-selection-internal' could ever call that hook; 
perhaps the traceback merely means that `x-get-selection-internal' was 
blocked and something -else- called the hook?  That would make sense if 
the two Emacs were deadlocked, each waiting for the other's answer before 
giving its own...  Disowning the selections seems to wake up a frozen 
Emacs, which makes sense with the deadlock-ish idea.

Davis Herring

-- 
This product is sold by volume, not by mass.  If it seems too dense or too 
sparse, it means mass-energy conversion has occurred during shipping.

Re: Issues with X selection handling

[Jan D.]

Davis Herring wrote:

>>> + `x-disown-selection-internal' doesn't seem to do anything.
>>
>>Well, it disowns the selection, but it does not call any hooks if that is what
>>you mean.  Can you please say what you expected it to do?
> 
> 
> I see -- it does disown the selection, but not only are no hooks called
> (the docstring for `x-lost-selection-hooks' explicitly says it should be
> called), but the entry is not removed from the internal variable
> `Vselection-alist'.  This means Emacs (in Lisp) thinks it still owns the 
> selection and that it has the value it had before the disownment.

The mishandling of Vselection-alist is a bug.  I have corrected it in CVS.
The hooks are also called after this correction.


> They disagree a notable amount of the time, and `x-lost-selection-hooks' 
> is never called with PRIMARY as an argument unless the keyboard is used to 
> kill text.  Meanwhile CLIPBOARD appears to call the hooks every time.

I still can not reproduce this.  You need to supply an explicit test case, i.e.
a step by step description of how to reproduce this.

> Stranger still is that occasionally one Emacs generates messages in the 
> other that look like this:
> 
> Sent selection at Mon Aug 23 18:57:44 2004: (CLIPBOARD nil nil)
> Sent selection at Mon Aug 23 18:57:44 2004: (CLIPBOARD nil nil)
> Sent selection at Mon Aug 23 18:57:44 2004: (PRIMARY nil nil)
> Sent selection at Mon Aug 23 18:57:44 2004: (PRIMARY nil nil)
> 
> The Emacs requesting the selection isn't sending any type with its 
> request, apparently; it gets declined, and decides that there is no such 
> selection.

It is.  It is just that the Emacs that prints this does not have the selection.
Then the second and third argument will be nil.  It is probably a race 
condition in your test set up.


	Jan D.

Mouse-drags wait to call post-command-hook

[Davis Herring]

I think this may be its own issue, so I'll isolate it (and talk more about 
X selections in later email):

> > They disagree a notable amount of the time, and `x-lost-selection-hooks' 
> > is never called with PRIMARY as an argument unless the keyboard is used to 
> > kill text.  Meanwhile CLIPBOARD appears to call the hooks every time.
> 
> I still can not reproduce this.  You need to supply an explicit test case, i.e.
> a step by step description of how to reproduce this.

The mouse-dependence may be related to the fact that mouse-drags do not
call `post-command-hook' until another event is received.

emacs -q -f ielm

*** Welcome to IELM ***  Type (describe-mode) for help.
ELISP> (defvar pcm-counter 0
	 "Counter of `post-command-hook' executions.")
pcm-counter
ELISP> (defun pcm-message ()
	 "Count `post-command-hook' executions."
	 (message "PCM: %s" (setq pcm-counter (1+ pcm-counter))))
pcm-message
ELISP> (add-hook 'post-command-hook 'pcm-message)
(pcm-message)

ELISP> 

Then drag-select some text in the buffer; no message is generated.  But 
then press C-f, and the counter increments twice.  This leads me to 
suspect that the drag isn't "finalized" in some way until the next 
command, which would explain why the X selection behavior depends on 
whether the most recent event was a mouse-drag.

Is there some reason for this behavior?  If it's just a bug, perhaps 
fixing it will fix (or make easier to fix) the X issues.

Davis Herring

-- 
This product is sold by volume, not by mass.  If it seems too dense or too 
sparse, it means mass-energy conversion has occurred during shipping.