Emacs failes to communicate with other X clients

Emacs failes to communicate with other X clients

[Robin Hu]

Hi everyone:

   Emacs failes to communicate with other X clients, due to it's
   wrongly decoding compound text from others. 

   For example, a chinese word is encoded in compound text
   as : "^[$(AV1^[%/2\200\210gbk-0^B\351F;y", as I have provide gbk
   coding system myself, emacs can decode the first two characters
   correctly, but it miss the last character, because it doesn't know
   "\351F;y" should be decoded follow "^[$(AV1".

   I try to correct this bug myself, but it seems quite diffcult in the
   exist compound-text-with-extension coding system.  ;-(

   I'd like to know how about leaving this decoding stuff to X itself, I
   knew XmbText family can do this kind of thing. And can anyone point me
   out where is the suitable place if this change can be applied?

   Robin.Hu

Re: Emacs failes to communicate with other X clients

[Eli Zaretskii]

> From: Robin Hu <huxw@knight.6test.edu.cn>
> Date: Fri, 23 May 2003 09:19:10 +0000
> 
>    For example, a chinese word is encoded in compound text
>    as : "^[$(AV1^[%/2\200\210gbk-0^B\351F;y", as I have provide gbk
>    coding system myself, emacs can decode the first two characters
>    correctly, but it miss the last character, because it doesn't know
>    "\351F;y" should be decoded follow "^[$(AV1".
> 
>    I try to correct this bug myself, but it seems quite diffcult in the
>    exist compound-text-with-extension coding system.  ;-(

I'm not sure I understand correctly the situation, but IIRC, gbk is
not supported by compound-text-with-extensions.  If you want to add
such a support, you need to modify the alist of non-standard ICCCM
encodings used by Emacs to match known coding-systems to the encoding
name mentioned in the X selection encoding.  See mule.el for the
definitions of those alists.

>    I'd like to know how about leaving this decoding stuff to X itself, I
>    knew XmbText family can do this kind of thing.

How can Emacs leave that to X?  We need to convert the X selection
text into the internal representation of characters used by Emacs; X
knows nothing about that representation.  Am I missing something?

Re: Emacs failes to communicate with other X clients

[Robin Hu]

>>>>> "Eli" == Eli Zaretskii <eliz@elta.co.il> writes:

    >> From: Robin Hu <huxw@knight.6test.edu.cn> Date: Fri, 23 May 2003
    >> 09:19:10 +0000

    Eli> I'm not sure I understand correctly the situation, but IIRC,
    Eli> gbk is not supported by compound-text-with-extensions.  If you
    Eli> want to add such a support, you need to modify the alist of
    Eli> non-standard ICCCM encodings used by Emacs to match known
    Eli> coding-systems to the encoding name mentioned in the X
    Eli> selection encoding.  See mule.el for the definitions of those
    Eli> alists.

    Yeah, icccm list had been appended with GBK-0, that's why my emacs
    can decode some gbk characters correctly, but problems are still
    there. In the example I gived in the previous post, one embeded gbk
    character will make all characters fail to be decoded. ;-(

    Eli> How can Emacs leave that to X?  We need to convert the X
    Eli> selection text into the internal representation of characters
    Eli> used by Emacs; X knows nothing about that representation.  Am I
    Eli> missing something?

    But selection-coding-sytem can do this translation. For example, we
    can leave X to decode compound text to gbk locale coding, then
    set-selection-coding-system to chinese-gbk, then everything should
    go fine. Why don't we just totally leave this copy/paste working to
    be transparent to Emacs, to make this work just like a keyboard? If
    I understand the source corrently, ntEmacs works in this way. Any
    comments?

Re: Emacs failes to communicate with other X clients

[Eli Zaretskii]

> From: Robin Hu <huxw@knight.6test.edu.cn>
> Date: Sat, 24 May 2003 11:10:06 +0000
> 
>     Eli> I'm not sure I understand correctly the situation, but IIRC,
>     Eli> gbk is not supported by compound-text-with-extensions.  If you
>     Eli> want to add such a support, you need to modify the alist of
>     Eli> non-standard ICCCM encodings used by Emacs to match known
>     Eli> coding-systems to the encoding name mentioned in the X
>     Eli> selection encoding.  See mule.el for the definitions of those
>     Eli> alists.
> 
>     Yeah, icccm list had been appended with GBK-0, that's why my emacs
>     can decode some gbk characters correctly, but problems are still
>     there. In the example I gived in the previous post, one embeded gbk
>     character will make all characters fail to be decoded. ;-(

Sounds like a bug.  Unfortunately, I don't know enough about gbk to
guess what might be wrong, and don't have time to debug
compound-text-with-extensions myself.

>     Why don't we just totally leave this copy/paste working to
>     be transparent to Emacs, to make this work just like a keyboard?

Keyboard input is nowhere as transparent as you seem to think.  Emacs
decodes keyboard input similarly to what it does with X selections.

Re: Emacs failes to communicate with other X clients

[Robin Hu]

[-- Attachment #1: Type: text/plain, Size: 1513 bytes --]

>>>>> "Eli" == Eli Zaretskii <eliz@elta.co.il> writes:

    >> From: Robin Hu <huxw@knight.6test.edu.cn> Date: Sat, 24 May 2003
    >> 11:10:06 +0000
    >> 
    Eli> Sounds like a bug.  Unfortunately, I don't know enough about
    Eli> gbk to guess what might be wrong, and don't have time to debug
    Eli> compound-text-with-extensions myself.

    I'll try to clearify myself, and attachment is a dirty patch to
    xselect.c, I also hope this can be any help out of my buggy english
    ;-(

    My point is that compound-text decoding should be left to X itself,
    rather then by emacs itself. It should be better be transparent to
    emacs, for better portable and better extensible.

    My patch implements this idea, it use XmbTextPropertyToTextList to
    decode compound-text, then delegate the result to
    selection_data_to_lisp_data, who will call decode_region with
    chinese-gbk to translate gbk coding characters to emacs-mule. It
    works fine for me these days.

    But there is still some problems I am not so sure. First is that I
    dont know whether this change of behavior will impact any other
    packages exists. Second is that I still wonder if there are any
    better places for this change.

    Eli> Keyboard input is nowhere as transparent as you seem to think.
    Eli> Emacs decodes keyboard input similarly to what it does with X
    Eli> selections.

    I mean decoding X selections should be seperated into 2 stages, then
    compound text is transparent to emacs.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch to Xselect.c --]
[-- Type: text/x-patch, Size: 2193 bytes --]

--- xselect.c	2003-04-07 04:35:06.000000000 +0800
+++ /home/huxw/tmp_src/emacs/src/xselect.c	2003-05-26 10:29:46.641097720 +0800
@@ -1496,6 +1496,11 @@
      Lisp_Object target_type;	/* for error messages only */
      Atom selection_atom;	/* for error messages only */
 {
+	// by huxw start here
+	XTextProperty text_prop;
+	char** local_list;
+	int local_number;
+	// by huxw end here
   Atom actual_type;
   int actual_format;
   unsigned long actual_size;
@@ -1554,12 +1559,50 @@
 
   /* It's been read.  Now convert it to a lisp object in some semi-rational
      manner.  */
+  //by huxw start here
+  if (XSupportsLocale()) {
+	  int local_status;
+
+	  text_prop.value = (char*)data;
+	  text_prop.encoding = actual_type;
+	  text_prop.format = actual_format;
+	  text_prop.nitems = actual_size;
+
+	  local_status = XmbTextPropertyToTextList(display, &text_prop, &local_list, &local_number);
+	  if (local_status < Success || !local_number || !*local_list ) {
+		  printf("failed due to XmbTextPropertyToTextList\n");
+		  printf("XNoMemory is %d, XLocaleNotSupported is %d, XConverterNotFound is %d\n", XNoMemory, XLocaleNotSupported, XConverterNotFound);
+		  printf("status is %d, number is %d, \n", local_status, local_number );
+	  } else {
+		  printf("XmbTextPropertyToTextList successfullly\n");
+		  xfree((char*)data);
+		  printf("xfree data successfully\n");
+		  data = strdup(*local_list);
+		  printf("strdup data successfully, data is %s\n", (char*)data);
+		  XFreeStringList(local_list);
+		  printf("XFreeStringList successfully\n");
+	  }
+  } else {
+	  printf("no support this locale\n");
+  }
+  //by huxw end here
+
+#if 0
   val = selection_data_to_lisp_data (display, data, bytes,
 				     actual_type, actual_format);
+#else
+	val = selection_data_to_lisp_data (display, data, strlen(data), actual_type, actual_format);
+#endif
 
   /* Use xfree, not XFree, because x_get_window_property
      calls xmalloc itself.  */
-  xfree ((char *) data);
+
+  // by huxw start here
+//  xfree ((char *) data);
+  printf("selection_data_to_lisp_data successfully\n");
+  free(data);
+  printf("free data successfully\n");
+  // by huxw end here
   return val;
 }
 \f

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Emacs failes to communicate with other X clients

[Kenichi Handa]

In article <uznla2z61.fsf@knight.6test.edu.cn>, Robin Hu <huxw@knight.6test.edu.cn> writes:
>     My point is that compound-text decoding should be left
> to X itself, rather then by emacs itself. It should be
> better be transparent to emacs, for better portable and
> better extensible.

>     My patch implements this idea, it use
> XmbTextPropertyToTextList to decode compound-text, then
> delegate the result to selection_data_to_lisp_data, who
> will call decode_region with chinese-gbk to translate gbk
> coding characters to emacs-mule. It works fine for me
> these days.

XmbTextPropertyToTextList can handle only such compound-text
that contains characters supported in the current X locale.
So, in your way, if you are in GBK locale, Emacs can't
receive, for instance, Hangul chacaters even if
compound-text can correctly encode Hangul characters and
Emacs itself can handle Hangul in any locales.

So, I beleive we should avoid such kind of change.

By the way, the coding system chinese-gbk is not yet
supported in the current Emacs.  Or, are you using
emacs-unicode?

---
Ken'ichi HANDA
handa@m17n.org

Re: Emacs failes to communicate with other X clients

[Robin Hu]

>>>>> "Kenichi" == Kenichi Handa <handa@m17n.org> writes:

    Kenichi> XmbTextPropertyToTextList can handle only such
    Kenichi> compound-text that contains characters supported in the
    Kenichi> current X locale.  So, in your way, if you are in GBK
    Kenichi> locale, Emacs can't receive, for instance, Hangul chacaters
    Kenichi> even if compound-text can correctly encode Hangul
    Kenichi> characters and Emacs itself can handle Hangul in any
    Kenichi> locales.

    Thank you Kenichi for your answering ;-) But I still have some
    different ideas here.
    
    Of course Hangul characters can not be received when we set locale
    to zh_CN.GBK, but I think this is the right behavior. For example,
    while I set keyboard-coding-system to chinese-gbk, I can not input
    Hangul characters, because it's my responsibility to set correct
    keyboard coding system for chinese input. So I think that's also my
    responsibility to set correct locale for X paste. 

    Another problem with current implementation is, some characters can
    be encoded in different char-settings. For example, I set file
    coding system to chinese-iso-8bit, and selection coding system to
    compound-text-with-extension, and copy/paste a very long chinese
    article from mozilla. Every thing seems to go fine, but this article
    just cannot be saved. This is because X encode some characters as if
    they are not chinese-iso-8bit characters, but emacs decode them to
    emacs-mule successfully, and finally file coding system cannot
    encode emacs-mule to chinese-iso-8bit.

    Kenichi> By the way, the coding system chinese-gbk is not yet
    Kenichi> supported in the current Emacs.  Or, are you using
    Kenichi> emacs-unicode?

    Chinese-gbk is my extension to Emacs, I believe I have post it in
    m17n's maillist. I can read/write gbk encoding file, and unicode
    file with utf-translate-gbk-mode turned on. Everything goes fine,
    except copy/paste in X (ntEmacs is fine). ;-(

Re: Emacs failes to communicate with other X clients

[Kenichi Handa]

In article <usmr21bsz.fsf@knight.6test.edu.cn>, Robin Hu <huxw@knight.6test.edu.cn> writes:
>     Of course Hangul characters can not be received when
> we set locale to zh_CN.GBK, but I think this is the right
> behavior. For example, while I set keyboard-coding-system
> to chinese-gbk, I can not input Hangul characters, because
> it's my responsibility to set correct keyboard coding
> system for chinese input. So I think that's also my
> responsibility to set correct locale for X paste.

The reason why you can't input Hangul characters is that
your input method doesn't support it or it generates data
only in chiense-gbk encoding which can't contain Hangul
characters.

Provided that you have an input method that produces both
Chinese and Hangul and sends data to Emacs in compound-text
encoding, and you set keyboard-coding-system to
compound-text, Emacs should accept all characters, shouldn't
it?

Emacs is a multilingual editor and its functionality should
not be limited by locale.

>     Another problem with current implementation is, some
> characters can be encoded in different char-settings. For
> example, I set file coding system to chinese-iso-8bit, and
> selection coding system to compound-text-with-extension,
> and copy/paste a very long chinese article from
> mozilla. Every thing seems to go fine, but this article
> just cannot be saved. This is because X encode some
> characters as if they are not chinese-iso-8bit characters,
> but emacs decode them to emacs-mule successfully, and
> finally file coding system cannot encode emacs-mule to
> chinese-iso-8bit.

In that case, Emacs asks you to select some of safe coding
systems.  If your Emacs supports chinese-gbk, it should also
be listed as a safe coding system.  I think that behaviour
is better than refusing characters not supported by
chinese-iso-8bit from the beginning.

Kenichi>  By the way, the coding system chinese-gbk is not
Kenichi> yet supported in the current Emacs.  Or, are you
Kenichi> using emacs-unicode?

>     Chinese-gbk is my extension to Emacs, I believe I have
> post it in m17n's maillist.

Oops, then, I'm very sorry that I completely forgot about
that mail.  Could you send it to me again?  Then I'll check
why:

>    Yeah, icccm list had been appended with GBK-0, that's why my emacs
>    can decode some gbk characters correctly, but problems are still
>    there. In the example I gived in the previous post, one embeded gbk
>    character will make all characters fail to be decoded. ;-(

that happens.

By the way, the name non-standard-icccm-encodings-alist is
wrong.  "icccm" should actually be "ctext".  I'll fix it
soon.

---
Ken'ichi HANDA
handa@m17n.org

Re: Emacs failes to communicate with other X clients

[Robin Hu]

[-- Attachment #1: Type: text/plain, Size: 1148 bytes --]

>>>>> "Kenichi" == Kenichi Handa <handa@m17n.org> writes:

    Kenichi> Provided that you have an input method that produces both
    Kenichi> Chinese and Hangul and sends data to Emacs in compound-text
    Kenichi> encoding, and you set keyboard-coding-system to
    Kenichi> compound-text, Emacs should accept all characters,
    Kenichi> shouldn't it?

    Kenichi> Emacs is a multilingual editor and its functionality should
    Kenichi> not be limited by locale.

    Kenichi> In that case, Emacs asks you to select some of safe coding
    Kenichi> systems.  If your Emacs supports chinese-gbk, it should
    Kenichi> also be listed as a safe coding system.  I think that
    Kenichi> behaviour is better than refusing characters not supported
    Kenichi> by chinese-iso-8bit from the beginning.

    Well, I think you successfully persuade me to believe your idea is
    a better choice, though it takes more effort. ;-)

    The first attachment is my extension for gbk coding. The second is a
    sample gbk coding file, it helps to show copy/paste problems. Hope
    they can be any help.

    Kenichi> --- Ken'ichi HANDA handa@m17n.org


[-- Attachment #2: Chinese-gbk --]
[-- Type: application/octet-stream, Size: 104177 bytes --]

[-- Attachment #3: Sample.txt --]
[-- Type: application/octet-stream, Size: 34 bytes --]

[-- Attachment #4: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/emacs-devel

Re: Emacs failes to communicate with other X clients

[Kenichi Handa]

In article <uk7certe8.fsf@knight.6test.edu.cn>, Robin Hu <huxw@knight.6test.edu.cn> writes:
>     The first attachment is my extension for gbk
> coding. The second is a sample gbk coding file, it helps
> to show copy/paste problems. Hope they can be any help.

Thank you.  I loaded your chinese-gbk.el, evaluate this,
  (push '("gbk-0" . chinese-gbk) non-standard-icccm-encodings-alist)
and tried your failure example as below (as I don't have an
X client that sends such a data):

(decode-coding-string "\e$(AV1\e%/2\200\210gbk-0\C-b\351F;y"
		      'compound-text-with-extensions)

It retursn four characters string:
 1. chinese-gb2312 (zhi2)
 2. chinese-cns11643-5 (can't see because I don't have a gbk font)
 3. ';'
 4. 'y'

If I encode that string into chinese-gbk, I get this:
"\326\261\351F;y"

Isn't it the correct string?

---
Ken'ichi HANDA
handa@m17n.org