* Emacs failes to communicate with other X clients @ 2003-05-23 9:19 Robin Hu 2003-05-23 17:22 ` Eli Zaretskii 0 siblings, 1 reply; 10+ messages in thread From: Robin Hu @ 2003-05-23 9:19 UTC (permalink / raw) Hi everyone: Emacs failes to communicate with other X clients, due to it's wrongly decoding compound text from others. For example, a chinese word is encoded in compound text as : "^[$(AV1^[%/2\200\210gbk-0^B\351F;y", as I have provide gbk coding system myself, emacs can decode the first two characters correctly, but it miss the last character, because it doesn't know "\351F;y" should be decoded follow "^[$(AV1". I try to correct this bug myself, but it seems quite diffcult in the exist compound-text-with-extension coding system. ;-( I'd like to know how about leaving this decoding stuff to X itself, I knew XmbText family can do this kind of thing. And can anyone point me out where is the suitable place if this change can be applied? Robin.Hu ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Emacs failes to communicate with other X clients 2003-05-23 9:19 Emacs failes to communicate with other X clients Robin Hu @ 2003-05-23 17:22 ` Eli Zaretskii 2003-05-24 11:10 ` Robin Hu 0 siblings, 1 reply; 10+ messages in thread From: Eli Zaretskii @ 2003-05-23 17:22 UTC (permalink / raw) Cc: emacs-devel > From: Robin Hu <huxw@knight.6test.edu.cn> > Date: Fri, 23 May 2003 09:19:10 +0000 > > For example, a chinese word is encoded in compound text > as : "^[$(AV1^[%/2\200\210gbk-0^B\351F;y", as I have provide gbk > coding system myself, emacs can decode the first two characters > correctly, but it miss the last character, because it doesn't know > "\351F;y" should be decoded follow "^[$(AV1". > > I try to correct this bug myself, but it seems quite diffcult in the > exist compound-text-with-extension coding system. ;-( I'm not sure I understand correctly the situation, but IIRC, gbk is not supported by compound-text-with-extensions. If you want to add such a support, you need to modify the alist of non-standard ICCCM encodings used by Emacs to match known coding-systems to the encoding name mentioned in the X selection encoding. See mule.el for the definitions of those alists. > I'd like to know how about leaving this decoding stuff to X itself, I > knew XmbText family can do this kind of thing. How can Emacs leave that to X? We need to convert the X selection text into the internal representation of characters used by Emacs; X knows nothing about that representation. Am I missing something? ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Emacs failes to communicate with other X clients 2003-05-23 17:22 ` Eli Zaretskii @ 2003-05-24 11:10 ` Robin Hu 2003-05-24 17:38 ` Eli Zaretskii 0 siblings, 1 reply; 10+ messages in thread From: Robin Hu @ 2003-05-24 11:10 UTC (permalink / raw) Cc: emacs-devel >>>>> "Eli" == Eli Zaretskii <eliz@elta.co.il> writes: >> From: Robin Hu <huxw@knight.6test.edu.cn> Date: Fri, 23 May 2003 >> 09:19:10 +0000 Eli> I'm not sure I understand correctly the situation, but IIRC, Eli> gbk is not supported by compound-text-with-extensions. If you Eli> want to add such a support, you need to modify the alist of Eli> non-standard ICCCM encodings used by Emacs to match known Eli> coding-systems to the encoding name mentioned in the X Eli> selection encoding. See mule.el for the definitions of those Eli> alists. Yeah, icccm list had been appended with GBK-0, that's why my emacs can decode some gbk characters correctly, but problems are still there. In the example I gived in the previous post, one embeded gbk character will make all characters fail to be decoded. ;-( Eli> How can Emacs leave that to X? We need to convert the X Eli> selection text into the internal representation of characters Eli> used by Emacs; X knows nothing about that representation. Am I Eli> missing something? But selection-coding-sytem can do this translation. For example, we can leave X to decode compound text to gbk locale coding, then set-selection-coding-system to chinese-gbk, then everything should go fine. Why don't we just totally leave this copy/paste working to be transparent to Emacs, to make this work just like a keyboard? If I understand the source corrently, ntEmacs works in this way. Any comments? ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Emacs failes to communicate with other X clients 2003-05-24 11:10 ` Robin Hu @ 2003-05-24 17:38 ` Eli Zaretskii 2003-05-26 2:56 ` Robin Hu 0 siblings, 1 reply; 10+ messages in thread From: Eli Zaretskii @ 2003-05-24 17:38 UTC (permalink / raw) Cc: emacs-devel > From: Robin Hu <huxw@knight.6test.edu.cn> > Date: Sat, 24 May 2003 11:10:06 +0000 > > Eli> I'm not sure I understand correctly the situation, but IIRC, > Eli> gbk is not supported by compound-text-with-extensions. If you > Eli> want to add such a support, you need to modify the alist of > Eli> non-standard ICCCM encodings used by Emacs to match known > Eli> coding-systems to the encoding name mentioned in the X > Eli> selection encoding. See mule.el for the definitions of those > Eli> alists. > > Yeah, icccm list had been appended with GBK-0, that's why my emacs > can decode some gbk characters correctly, but problems are still > there. In the example I gived in the previous post, one embeded gbk > character will make all characters fail to be decoded. ;-( Sounds like a bug. Unfortunately, I don't know enough about gbk to guess what might be wrong, and don't have time to debug compound-text-with-extensions myself. > Why don't we just totally leave this copy/paste working to > be transparent to Emacs, to make this work just like a keyboard? Keyboard input is nowhere as transparent as you seem to think. Emacs decodes keyboard input similarly to what it does with X selections. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Emacs failes to communicate with other X clients 2003-05-24 17:38 ` Eli Zaretskii @ 2003-05-26 2:56 ` Robin Hu 2003-05-26 4:54 ` Kenichi Handa 0 siblings, 1 reply; 10+ messages in thread From: Robin Hu @ 2003-05-26 2:56 UTC (permalink / raw) Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1513 bytes --] >>>>> "Eli" == Eli Zaretskii <eliz@elta.co.il> writes: >> From: Robin Hu <huxw@knight.6test.edu.cn> Date: Sat, 24 May 2003 >> 11:10:06 +0000 >> Eli> Sounds like a bug. Unfortunately, I don't know enough about Eli> gbk to guess what might be wrong, and don't have time to debug Eli> compound-text-with-extensions myself. I'll try to clearify myself, and attachment is a dirty patch to xselect.c, I also hope this can be any help out of my buggy english ;-( My point is that compound-text decoding should be left to X itself, rather then by emacs itself. It should be better be transparent to emacs, for better portable and better extensible. My patch implements this idea, it use XmbTextPropertyToTextList to decode compound-text, then delegate the result to selection_data_to_lisp_data, who will call decode_region with chinese-gbk to translate gbk coding characters to emacs-mule. It works fine for me these days. But there is still some problems I am not so sure. First is that I dont know whether this change of behavior will impact any other packages exists. Second is that I still wonder if there are any better places for this change. Eli> Keyboard input is nowhere as transparent as you seem to think. Eli> Emacs decodes keyboard input similarly to what it does with X Eli> selections. I mean decoding X selections should be seperated into 2 stages, then compound text is transparent to emacs. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: patch to Xselect.c --] [-- Type: text/x-patch, Size: 2193 bytes --] --- xselect.c 2003-04-07 04:35:06.000000000 +0800 +++ /home/huxw/tmp_src/emacs/src/xselect.c 2003-05-26 10:29:46.641097720 +0800 @@ -1496,6 +1496,11 @@ Lisp_Object target_type; /* for error messages only */ Atom selection_atom; /* for error messages only */ { + // by huxw start here + XTextProperty text_prop; + char** local_list; + int local_number; + // by huxw end here Atom actual_type; int actual_format; unsigned long actual_size; @@ -1554,12 +1559,50 @@ /* It's been read. Now convert it to a lisp object in some semi-rational manner. */ + //by huxw start here + if (XSupportsLocale()) { + int local_status; + + text_prop.value = (char*)data; + text_prop.encoding = actual_type; + text_prop.format = actual_format; + text_prop.nitems = actual_size; + + local_status = XmbTextPropertyToTextList(display, &text_prop, &local_list, &local_number); + if (local_status < Success || !local_number || !*local_list ) { + printf("failed due to XmbTextPropertyToTextList\n"); + printf("XNoMemory is %d, XLocaleNotSupported is %d, XConverterNotFound is %d\n", XNoMemory, XLocaleNotSupported, XConverterNotFound); + printf("status is %d, number is %d, \n", local_status, local_number ); + } else { + printf("XmbTextPropertyToTextList successfullly\n"); + xfree((char*)data); + printf("xfree data successfully\n"); + data = strdup(*local_list); + printf("strdup data successfully, data is %s\n", (char*)data); + XFreeStringList(local_list); + printf("XFreeStringList successfully\n"); + } + } else { + printf("no support this locale\n"); + } + //by huxw end here + +#if 0 val = selection_data_to_lisp_data (display, data, bytes, actual_type, actual_format); +#else + val = selection_data_to_lisp_data (display, data, strlen(data), actual_type, actual_format); +#endif /* Use xfree, not XFree, because x_get_window_property calls xmalloc itself. */ - xfree ((char *) data); + + // by huxw start here +// xfree ((char *) data); + printf("selection_data_to_lisp_data successfully\n"); + free(data); + printf("free data successfully\n"); + // by huxw end here return val; } \f [-- Attachment #3: Type: text/plain, Size: 142 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://mail.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Emacs failes to communicate with other X clients 2003-05-26 2:56 ` Robin Hu @ 2003-05-26 4:54 ` Kenichi Handa 2003-05-26 6:06 ` Robin Hu 0 siblings, 1 reply; 10+ messages in thread From: Kenichi Handa @ 2003-05-26 4:54 UTC (permalink / raw) Cc: emacs-devel In article <uznla2z61.fsf@knight.6test.edu.cn>, Robin Hu <huxw@knight.6test.edu.cn> writes: > My point is that compound-text decoding should be left > to X itself, rather then by emacs itself. It should be > better be transparent to emacs, for better portable and > better extensible. > My patch implements this idea, it use > XmbTextPropertyToTextList to decode compound-text, then > delegate the result to selection_data_to_lisp_data, who > will call decode_region with chinese-gbk to translate gbk > coding characters to emacs-mule. It works fine for me > these days. XmbTextPropertyToTextList can handle only such compound-text that contains characters supported in the current X locale. So, in your way, if you are in GBK locale, Emacs can't receive, for instance, Hangul chacaters even if compound-text can correctly encode Hangul characters and Emacs itself can handle Hangul in any locales. So, I beleive we should avoid such kind of change. By the way, the coding system chinese-gbk is not yet supported in the current Emacs. Or, are you using emacs-unicode? --- Ken'ichi HANDA handa@m17n.org ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Emacs failes to communicate with other X clients 2003-05-26 4:54 ` Kenichi Handa @ 2003-05-26 6:06 ` Robin Hu 2003-05-26 7:09 ` Kenichi Handa 0 siblings, 1 reply; 10+ messages in thread From: Robin Hu @ 2003-05-26 6:06 UTC (permalink / raw) Cc: emacs-devel >>>>> "Kenichi" == Kenichi Handa <handa@m17n.org> writes: Kenichi> XmbTextPropertyToTextList can handle only such Kenichi> compound-text that contains characters supported in the Kenichi> current X locale. So, in your way, if you are in GBK Kenichi> locale, Emacs can't receive, for instance, Hangul chacaters Kenichi> even if compound-text can correctly encode Hangul Kenichi> characters and Emacs itself can handle Hangul in any Kenichi> locales. Thank you Kenichi for your answering ;-) But I still have some different ideas here. Of course Hangul characters can not be received when we set locale to zh_CN.GBK, but I think this is the right behavior. For example, while I set keyboard-coding-system to chinese-gbk, I can not input Hangul characters, because it's my responsibility to set correct keyboard coding system for chinese input. So I think that's also my responsibility to set correct locale for X paste. Another problem with current implementation is, some characters can be encoded in different char-settings. For example, I set file coding system to chinese-iso-8bit, and selection coding system to compound-text-with-extension, and copy/paste a very long chinese article from mozilla. Every thing seems to go fine, but this article just cannot be saved. This is because X encode some characters as if they are not chinese-iso-8bit characters, but emacs decode them to emacs-mule successfully, and finally file coding system cannot encode emacs-mule to chinese-iso-8bit. Kenichi> By the way, the coding system chinese-gbk is not yet Kenichi> supported in the current Emacs. Or, are you using Kenichi> emacs-unicode? Chinese-gbk is my extension to Emacs, I believe I have post it in m17n's maillist. I can read/write gbk encoding file, and unicode file with utf-translate-gbk-mode turned on. Everything goes fine, except copy/paste in X (ntEmacs is fine). ;-( ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Emacs failes to communicate with other X clients 2003-05-26 6:06 ` Robin Hu @ 2003-05-26 7:09 ` Kenichi Handa 2003-05-26 8:42 ` Robin Hu 0 siblings, 1 reply; 10+ messages in thread From: Kenichi Handa @ 2003-05-26 7:09 UTC (permalink / raw) Cc: emacs-devel In article <usmr21bsz.fsf@knight.6test.edu.cn>, Robin Hu <huxw@knight.6test.edu.cn> writes: > Of course Hangul characters can not be received when > we set locale to zh_CN.GBK, but I think this is the right > behavior. For example, while I set keyboard-coding-system > to chinese-gbk, I can not input Hangul characters, because > it's my responsibility to set correct keyboard coding > system for chinese input. So I think that's also my > responsibility to set correct locale for X paste. The reason why you can't input Hangul characters is that your input method doesn't support it or it generates data only in chiense-gbk encoding which can't contain Hangul characters. Provided that you have an input method that produces both Chinese and Hangul and sends data to Emacs in compound-text encoding, and you set keyboard-coding-system to compound-text, Emacs should accept all characters, shouldn't it? Emacs is a multilingual editor and its functionality should not be limited by locale. > Another problem with current implementation is, some > characters can be encoded in different char-settings. For > example, I set file coding system to chinese-iso-8bit, and > selection coding system to compound-text-with-extension, > and copy/paste a very long chinese article from > mozilla. Every thing seems to go fine, but this article > just cannot be saved. This is because X encode some > characters as if they are not chinese-iso-8bit characters, > but emacs decode them to emacs-mule successfully, and > finally file coding system cannot encode emacs-mule to > chinese-iso-8bit. In that case, Emacs asks you to select some of safe coding systems. If your Emacs supports chinese-gbk, it should also be listed as a safe coding system. I think that behaviour is better than refusing characters not supported by chinese-iso-8bit from the beginning. Kenichi> By the way, the coding system chinese-gbk is not Kenichi> yet supported in the current Emacs. Or, are you Kenichi> using emacs-unicode? > Chinese-gbk is my extension to Emacs, I believe I have > post it in m17n's maillist. Oops, then, I'm very sorry that I completely forgot about that mail. Could you send it to me again? Then I'll check why: > Yeah, icccm list had been appended with GBK-0, that's why my emacs > can decode some gbk characters correctly, but problems are still > there. In the example I gived in the previous post, one embeded gbk > character will make all characters fail to be decoded. ;-( that happens. By the way, the name non-standard-icccm-encodings-alist is wrong. "icccm" should actually be "ctext". I'll fix it soon. --- Ken'ichi HANDA handa@m17n.org ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Emacs failes to communicate with other X clients 2003-05-26 7:09 ` Kenichi Handa @ 2003-05-26 8:42 ` Robin Hu 2003-05-27 12:26 ` Kenichi Handa 0 siblings, 1 reply; 10+ messages in thread From: Robin Hu @ 2003-05-26 8:42 UTC (permalink / raw) Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1148 bytes --] >>>>> "Kenichi" == Kenichi Handa <handa@m17n.org> writes: Kenichi> Provided that you have an input method that produces both Kenichi> Chinese and Hangul and sends data to Emacs in compound-text Kenichi> encoding, and you set keyboard-coding-system to Kenichi> compound-text, Emacs should accept all characters, Kenichi> shouldn't it? Kenichi> Emacs is a multilingual editor and its functionality should Kenichi> not be limited by locale. Kenichi> In that case, Emacs asks you to select some of safe coding Kenichi> systems. If your Emacs supports chinese-gbk, it should Kenichi> also be listed as a safe coding system. I think that Kenichi> behaviour is better than refusing characters not supported Kenichi> by chinese-iso-8bit from the beginning. Well, I think you successfully persuade me to believe your idea is a better choice, though it takes more effort. ;-) The first attachment is my extension for gbk coding. The second is a sample gbk coding file, it helps to show copy/paste problems. Hope they can be any help. Kenichi> --- Ken'ichi HANDA handa@m17n.org [-- Attachment #2: Chinese-gbk --] [-- Type: application/octet-stream, Size: 104177 bytes --] [-- Attachment #3: Sample.txt --] [-- Type: application/octet-stream, Size: 34 bytes --] [-- Attachment #4: Type: text/plain, Size: 142 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://mail.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Emacs failes to communicate with other X clients 2003-05-26 8:42 ` Robin Hu @ 2003-05-27 12:26 ` Kenichi Handa 0 siblings, 0 replies; 10+ messages in thread From: Kenichi Handa @ 2003-05-27 12:26 UTC (permalink / raw) Cc: emacs-devel In article <uk7certe8.fsf@knight.6test.edu.cn>, Robin Hu <huxw@knight.6test.edu.cn> writes: > The first attachment is my extension for gbk > coding. The second is a sample gbk coding file, it helps > to show copy/paste problems. Hope they can be any help. Thank you. I loaded your chinese-gbk.el, evaluate this, (push '("gbk-0" . chinese-gbk) non-standard-icccm-encodings-alist) and tried your failure example as below (as I don't have an X client that sends such a data): (decode-coding-string "\e$(AV1\e%/2\200\210gbk-0\C-b\351F;y" 'compound-text-with-extensions) It retursn four characters string: 1. chinese-gb2312 (zhi2) 2. chinese-cns11643-5 (can't see because I don't have a gbk font) 3. ';' 4. 'y' If I encode that string into chinese-gbk, I get this: "\326\261\351F;y" Isn't it the correct string? --- Ken'ichi HANDA handa@m17n.org ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2003-05-27 12:26 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2003-05-23 9:19 Emacs failes to communicate with other X clients Robin Hu 2003-05-23 17:22 ` Eli Zaretskii 2003-05-24 11:10 ` Robin Hu 2003-05-24 17:38 ` Eli Zaretskii 2003-05-26 2:56 ` Robin Hu 2003-05-26 4:54 ` Kenichi Handa 2003-05-26 6:06 ` Robin Hu 2003-05-26 7:09 ` Kenichi Handa 2003-05-26 8:42 ` Robin Hu 2003-05-27 12:26 ` Kenichi Handa
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).