Re: Reporting UTF-8 related problems?

[Kenichi Handa <handa@etl.go.jp>]

In article <sh65yxniuf.fsf@tux.gnu.franken.de>, Karl Eichwalder <keichwa@gmx.net> writes:
> Yes, but once in the X selection I'd like to see Emacs honor them.

> The spacing problem also occurs when I try to cut and paste from Markus
> Kuhn's demo file
> (http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt):

As far as I understand, that's not a spacing problem.  As
those clients send Emacs the designation sequence of
jisx0208 characters, Emacs just decodes them correctly (i.e.
honoring them) and displaying them by Japanese double-width
font.

> When I insert (C-x RET c utf-8 RET C-x C-f UTF-8-demo.txt RET), things
> are correctly displayed (the characters are different):

That's because the file is correclty encoded in utf-8, thus
Emacs can decode it correctly.

> Cut and paste both these examples from Emacs (this mail buffer) to a
> UTF-8 xterm doesn't work neither; instead of the quotes I see "-1" and
> garbage.

Yes because I have not yet installed a code for encoding
Emacs string to what UTF-8 xterm expect.

I confirmed that UTF-8 xterm surely request the target type
UTF8_STRING at first.   I'm now finding a way to handle it.

While tracing the the whole procedure of Emacs to handle a
selection request, I found the followings.

Could someone else also check if I miss something?

When Emacs receives a selection request,
x_handle_selction_request (xselect.c) is called.

The flow is as this:

x_handle_selction_request (EVENT)  -- xselect.c
  x_get_local_selection (SELECTION, TARGET_TYPE)  -- xselect.c
    xselect-convert-to-string (SELECTION, TARGET-TYPE, VALUE)  -- select.el
       => returns MULTIBYTE-STRING
    => returns MULTIBYTE-STRING
  lisp_data_to_selection_data (EVENT, MULTIBYTE-STRING, ...)
     => returns encoded string
  x_reply_selection_request (EVENT, above returned encoded string)
     ;; sends selection data to the other client

So, it seems that we can perform the encoding in the lisp
function xselect-convert-to-string, not in
lisp_data_to_selection_data.  BUT...

xselect-convert-to-string is also called in this way:

yank  -- simple.el
  current-kill  -- simple.el
    x-cur-buffer-or-selection-value  -- x-win.el
      x-get-selection  -- select.el
        Fx_get_selection_internal  -- xselect.c
          x_get_local_selection  -- xselect.c
             xselect-convert-to-string  -- select.el  !!!

And, in the latter case, xselect-convert-to-string must
return an Emacs string without encoding it.  Currently,
xselect-convert-to-string has no way to know in which
situation it is called.

So, how about calling xselect-convert-to-string with
TARGET-TYPE nil in the latter case?  This can be done by
adding one more arg LOCAL-REQUEST to x_get_local_selection.

If the above analysis is correct, we can implement the
rather sensitive/delicate code for handling string in
lisp_data_to_selection_data and x_encode_text in Lisp, which
makes the Emacs' reaction to selection request more flexible
and also makes the future maintanance easier.

What do you think?

---
Ken'ichi HANDA
handa@etl.go.jp