bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

[Петров Андрей]

Hello, team!

Please check the following behavior:

1. Start two instances of emacs -Q.
2. Switch to the "*scratch*" buffer of the first one and type the following:
   M-: (insert (format "%S"  "000\000000")) [RET]
   You should see "000^@000" string.
3. Copy the string into clipboard and paste into the other instance of
   Emacs (which you have started before).
   I get "000\0000" instead of "000\000000"
4. Try evaluating inserted text ("000\0000")
   C-c C-e
   I get "000^@0" instead of "000^@000".

It seems that "^@" is transcoded between applications into "\0"
 instead of "\000". If I do the same test inside single instance
 of Emacs I get the correct result.


In GNU Emacs 29.0.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.34, cairo version 1.17.6)
 of 2022-05-30 built on pva-msi
Repository revision: 004e6ae3f42bdc8cd632060f56dbf8c1ea850c4a
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
System Description: Arch Linux

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY
PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LANG: ru_RU.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media rmc puny
dired dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg
rfc6068 epg-config gnus-util text-property-search seq gv byte-opt
bytecomp byte-compile cconv mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils time-date subr-x cl-loaddefs
cl-lib cyril-util iso-transl tooltip eldoc paren electric uniquify
ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice simple cl-generic indonesian philippine cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
make-network-process emacs)

Memory information:
((conses 16 43966 8699)
 (symbols 48 5718 1)
 (strings 32 15667 1941)
 (string-bytes 1 509558)
 (vectors 16 11167)
 (vector-slots 8 162892 11831)
 (floats 8 23 52)
 (intervals 56 457 0)
 (buffers 992 11))


Best regards,
Andrey Petrov.

bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Петров Андрей <an.petrov@pay-lab.ru> writes:

> Hello, team!
>
> Please check the following behavior:
>
> 1. Start two instances of emacs -Q.
> 2. Switch to the "*scratch*" buffer of the first one and type the following:
>    M-: (insert (format "%S"  "000\000000")) [RET]
>    You should see "000^@000" string.
> 3. Copy the string into clipboard and paste into the other instance of
>    Emacs (which you have started before).
>    I get "000\0000" instead of "000\000000"
> 4. Try evaluating inserted text ("000\0000")
>    C-c C-e
>    I get "000^@0" instead of "000^@000".
>
> It seems that "^@" is transcoded between applications into "\0"
>  instead of "\000". If I do the same test inside single instance
>  of Emacs I get the correct result.

That's not a bug.  Most programs cannot tolerate NUL bytes inside
selection strings, so the Emacs selection converter deliberately
replaces those inside a string with "\0" before sending the string to a
requestor.

When one Emacs instance owns the selection, the contents of the kill
ring are used directly when yanking text, so that conversion doesn't
take place.

bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

[Eli Zaretskii]

> From: Петров Андрей <an.petrov@pay-lab.ru>
> Date: Mon, 30 May 2022 13:08:52 +0000
> 
> 1. Start two instances of emacs -Q.
> 2. Switch to the "*scratch*" buffer of the first one and type the following:
>    M-: (insert (format "%S"  "000\000000")) [RET]
>    You should see "000^@000" string.
> 3. Copy the string into clipboard and paste into the other instance of
>    Emacs (which you have started before).
>    I get "000\0000" instead of "000\000000"
> 4. Try evaluating inserted text ("000\0000")
>    C-c C-e
>    I get "000^@0" instead of "000^@000".
> 
> It seems that "^@" is transcoded between applications into "\0"
>  instead of "\000". If I do the same test inside single instance
>  of Emacs I get the correct result.

Yes.  And it's a feature.  From select.el:

      ;; Most programs are unable to handle NUL bytes in strings.
      (setq str (string-replace "\0" "\\0" str))

bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

[Петров Андрей]

Hi, Po.

OK, You can close the bug then.

Just tell me if Emacs have ready to use solution for passing binary
strings between applications in a form what can be correctly restored
 with `read' function. I found that output of `print' also can not be
transferred through application boundary.

С уважением,
Андрей Петров,
программист ООО Инпас-Софт.


________________________________________
От: Po Lu <luangruo@yahoo.com>
Отправлено: 30 мая 2022 г. 18:42:51
Кому: Петров Андрей
Копия: 55727@debbugs.gnu.org
Тема: Re: bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

Петров Андрей <an.petrov@pay-lab.ru> writes:

> Hello, team!
>
> Please check the following behavior:
>
> 1. Start two instances of emacs -Q.
> 2. Switch to the "*scratch*" buffer of the first one and type the following:
>    M-: (insert (format "%S"  "000\000000")) [RET]
>    You should see "000^@000" string.
> 3. Copy the string into clipboard and paste into the other instance of
>    Emacs (which you have started before).
>    I get "000\0000" instead of "000\000000"
> 4. Try evaluating inserted text ("000\0000")
>    C-c C-e
>    I get "000^@0" instead of "000^@000".
>
> It seems that "^@" is transcoded between applications into "\0"
>  instead of "\000". If I do the same test inside single instance
>  of Emacs I get the correct result.

That's not a bug.  Most programs cannot tolerate NUL bytes inside
selection strings, so the Emacs selection converter deliberately
replaces those inside a string with "\0" before sending the string to a
requestor.

When one Emacs instance owns the selection, the contents of the kill
ring are used directly when yanking text, so that conversion doesn't
take place.

bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

[Eli Zaretskii]

> Cc: "55727@debbugs.gnu.org" <55727@debbugs.gnu.org>
> From: Петров Андрей <an.petrov@pay-lab.ru>
> Date: Mon, 30 May 2022 14:27:43 +0000
> 
> OK, You can close the bug then.

Done.

> Just tell me if Emacs have ready to use solution for passing binary
> strings between applications in a form what can be correctly restored
>  with `read' function. I found that output of `print' also can not be
> transferred through application boundary.

Did you consider encoding it in base64?

bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

[Петров Андрей]

Hi, Eli!

Looks like I have to use hex, base64 or other ASCII encodings.
It seems that I will use something like this:

(defun my-escape-char (c)
  (insert (if (< c ?\s) (format "\\%03o" c) c)))

(print "000\000000" 'my-escape-char)


Best regards,
Andrey Petrov.
________________________________________
От: Eli Zaretskii <eliz@gnu.org>
Отправлено: 30 мая 2022 г. 20:42:49
Кому: Петров Андрей
Копия: luangruo@yahoo.com; 55727-done@debbugs.gnu.org
Тема: Re: bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

> Cc: "55727@debbugs.gnu.org" <55727@debbugs.gnu.org>
> From: Петров Андрей <an.petrov@pay-lab.ru>
> Date: Mon, 30 May 2022 14:27:43 +0000
>
> OK, You can close the bug then.

Done.

> Just tell me if Emacs have ready to use solution for passing binary
> strings between applications in a form what can be correctly restored
>  with `read' function. I found that output of `print' also can not be
> transferred through application boundary.

Did you consider encoding it in base64?

bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

[Eli Zaretskii]

> From: Петров Андрей <an.petrov@pay-lab.ru>
> CC: "luangruo@yahoo.com" <luangruo@yahoo.com>, "55727-done@debbugs.gnu.org"
> 	<55727-done@debbugs.gnu.org>
> Date: Mon, 30 May 2022 16:23:46 +0000
> 
> Hi, Eli!
> 
> Looks like I have to use hex, base64 or other ASCII encodings.
> It seems that I will use something like this:
> 
> (defun my-escape-char (c)
>   (insert (if (< c ?\s) (format "\\%03o" c) c)))
> 
> (print "000\000000" 'my-escape-char)

Emacs has base64-encode-region and base64-decode-region, so you could
use that instead.

bug#55727: 29.0.50; Incorrect zero symbol transcoding between applications

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Петров Андрей <an.petrov@pay-lab.ru> writes:

> Hi, Po.
>
> OK, You can close the bug then.

Thanks, closed.

> Just tell me if Emacs have ready to use solution for passing binary
> strings between applications in a form what can be correctly restored
>  with `read' function. I found that output of `print' also can not be
> transferred through application boundary.

You could encode the data in a format such as base64, or alternatively
pass it as input to a subprocess.  Selections are probably the wrong
thing to use for this.