bug#6720: battle with x-clipboard vs. Chinese

bug#6720: battle with x-clipboard vs. Chinese

[jidanni]

I just updated to
emacs-snapshot:
  Installed: 1:20100724-1
and am having to battle getting Chinese in correctly from the X
clipboard.
It is turning into ###, ???, \u{5432}, etc. etc.

bug#6720: battle with x-clipboard vs. Chinese

[David De La Harpe Golden]

On 24/07/10 18:36, jidanni@jidanni.org wrote:
> I just updated to
> emacs-snapshot:
>    Installed: 1:20100724-1
> and am having to battle getting Chinese in correctly from the X
> clipboard.
> It is turning into ###, ???, \u{5432}, etc. etc.
>

Do you see this when using middle mouse button or C-y or both?

This sounds like it might be bug #6635, an encoding handling problem in 
mouse-yank-primary, which could affect people more due to recent changes:

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6635

bug#6720: battle with x-clipboard vs. Chinese

[jidanni]

>>>>> "DDLHG" == David De La Harpe Golden <david@harpegolden.net> writes:

DDLHG> On 24/07/10 18:36, jidanni@jidanni.org wrote:
>> I just updated to
>> emacs-snapshot:
>> Installed: 1:20100724-1
>> and am having to battle getting Chinese in correctly from the X
>> clipboard.
>> It is turning into ###, ???, \u{5432}, etc. etc.
>> 

DDLHG> Do you see this when using middle mouse button or C-y or both?

All I know is starting with 
   $ apt-cache policy emacs-snapshot 
   emacs-snapshot:
     Installed: 1:20100724-1
     Candidate: 1:20100724-1
     Version table:
    *** 1:20100724-1 0
           500 http://emacs.orebokech.com sid/main Packages
           100 /var/lib/dpkg/status

I am having two kinds of pain:

1. Having to write things into /tmp/files to be able to read them into
emacs correctly as xclip now can't if I want Chinese intact.
I try every button and end up having to resort to /tmp.


2. Even more disconnection between C-y, Shift-insert, the middle mouse
button, between emacs and the outside world. I wish I could make them
all connected.

DDLHG> This sounds like it might be bug #6635, an encoding handling problem
DDLHG> in mouse-yank-primary, which could affect people more due to recent
DDLHG> changes:

DDLHG> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=6635

bug#6720: battle with x-clipboard vs. Chinese

[David De La Harpe Golden]

[-- Attachment #1: Type: text/plain, Size: 1311 bytes --]

On 25/07/10 02:08, jidanni@jidanni.org wrote:

> All I know is starting with
>   $ apt-cache policy emacs-snapshot
>   emacs-snapshot:
>     Installed: 1:20100724-1
>     Candidate: 1:20100724-1


Yes, this appears to be Romain Francoise's snapshot packaging of latest 
emacs development code. Bug reports for development code are of course 
helpful and welcome, but please be aware that isn't an official release 
if you aren't already.

> I am having two kinds of pain:
>
> 1. Having to write things into /tmp/files to be able to read them into
> emacs correctly as xclip now can't if I want Chinese intact.
> I try every button and end up having to resort to /tmp.


Middle mouse button is currently known to be broken due to #6635., 
symptoms being strings like  "\u4e2d\u6587" or "中文".

"M-x load-file" of the attached fix6635.el file in emacs will help 
there, until a fix goes into the tree.

"xclip": you mean the command line tool to manipulate the clipboard?

Using xclip to put text on the clipboard for C-y to pull in does seem to 
work as expected on my system:

# shell
$ echo "中文" | xclip -selection clipboard

# in emacs, C-y
中文


But I don't actually know chinese.  Do you have a sample string that is 
breaking if you do the above?


[-- Attachment #2: fix6635.el --]
[-- Type: text/x-emacs-lisp, Size: 1312 bytes --]

;; Temporary workaround for GNU Emacs bug #6635 on x11 systems.
;; 2010-07-24 DG

(defun mouse-yank-primary (click)
  "Insert the primary selection at the position clicked on.
Move point to the end of the inserted text.
If `mouse-yank-at-point' is non-nil, insert at point
regardless of where you click."
  (interactive "e")
  ;; Give temporary modes such as isearch a chance to turn off.
  (run-hooks 'mouse-leave-buffer-hook)
  (when select-active-regions
    ;; Without this, confusing things happen upon e.g. inserting into
    ;; the middle of an active region.
    (deactivate-mark))
  (or mouse-yank-at-point (mouse-set-point click))
  (let ((primary (x-selection-value 'PRIMARY)))
    (if primary
        (insert primary)
      (error "No primary selection"))))

(defun mouse-yank-secondary (click)
  "Insert the secondary selection at the position clicked on.
Move point to the end of the inserted text.
If `mouse-yank-at-point' is non-nil, insert at point
regardless of where you click."
  (interactive "e")
  ;; Give temporary modes such as isearch a chance to turn off.
  (run-hooks 'mouse-leave-buffer-hook)
  (or mouse-yank-at-point (mouse-set-point click))
  (let ((secondary (x-selection-value 'SECONDARY)))
    (if secondary
        (insert secondary)
      (error "No secondary selection"))))

bug#6720: battle with x-clipboard vs. Chinese

[jidanni]

And the most scary thing of all is now things that I kill with C-k in
emacs are not necessarily what Shift-Insert will now paste in xterm.
That has never happened before and is causing me to send dangers wads of
text of some previous yank to the shell.

bug#6720: battle with x-clipboard vs. Chinese

[David De La Harpe Golden]

On 25/07/10 02:28, jidanni@jidanni.org wrote:
> And the most scary thing of all is now things that I kill with C-k in
> emacs are not necessarily what Shift-Insert will now paste in xterm.

This is a likely separate issue to the encoding problem and expected 
behaviour.   Recently a decision was made to make emacs act more like 
and play nicer with more recent x11 applications in the area of cut and 
paste.  xterm is ...not a recent application in this respect.

Things killed in emacs presently go to the _system clipboard_, but 
Shift-Insert in xterm inserts the "x11 primary selection" (unless you 
reconfigure it to use the clipboard).

The below does not directly address encoding problems (though step 3 
will mask the known encoding bug):

*** The new settings are only a revised default, you can still 
reconfigure emacs to act like it used to.  This will mean it won't work 
well with more recent x11 applications anymore, much like the way xterm 
doesn't work well with them.  Of course, you may not care.

1. M-x customize-group killing

Turn off x-select-enable-clipboard
Turn on x-select-enable-primary
Turn off select-active-regions

2. M-x customize-group mouse

Turn on mouse-drag-copy-region

3. Add the following to your ~/.emacs:

(global-set-key [mouse-2] 'mouse-yank-at-click)


**** various forms of xterm reconfiguration are also possible, though 
may not prove satisfactory. Using gnome-terminal (or xfce4-terminal or 
konsole, etc.) might be a better option.

Firstly, you can easily make xterm use the clipboard, but then it uses 
it all the time:  Hold down ctrl and middle mouse button on the xterm 
window. Select menu option "Select to Clipboard".  If you like that, 
making that permanent involves setting X resource 
XTerm.selectToClipboard as per the xterm manual.

http://www.davidsimmons.com/soft/xtermhacks/#copynpaste has another take 
on the problem with xterm in this area, and offers some more advanced 
hackery for changing xterm's behaviour.

bug#6720: battle with x-clipboard vs. Chinese

[jidanni]

Cool, using
   (setq
    x-select-enable-clipboard nil
    x-select-enable-primary t
    select-active-regions nil
    mouse-drag-copy-region t)
   (global-set-key [mouse-2] 'mouse-yank-at-click)
I didn't even need to load that .el you sent and all seems OK now. Thanks!