bug#64697: 29.0.92: cannot paste NUL on macOS (regression from Emacs 28)

bug#64697: 29.0.92: cannot paste NUL on macOS (regression from Emacs 28)

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 406 bytes --]

In Emacs 29, text pasted from the clipboard on macOS will be truncated if containing NULs.

Reproduction: copy text with NUL in the middle from Emacs 28 (or any other application), and paste into Emacs 29.

This was probably caused by 7e3c2b553f, where construction of a Lisp string was changed from make_string to build_string.

Suggested patch attached. If valid, would it qualify for emacs-29?


[-- Attachment #2: lispString.diff --]
[-- Type: application/octet-stream, Size: 453 bytes --]

diff --git a/src/nsfns.m b/src/nsfns.m
index 8804a7df7cf..d7ad0fa8341 100644
--- a/src/nsfns.m
+++ b/src/nsfns.m
@@ -3829,7 +3829,8 @@ handled fairly well by the NS libraries (displayed with distinct
 /* Make a Lisp string from an NSString.  */
 - (Lisp_Object)lispString
 {
-  return build_string ([self UTF8String]);
+  return make_string ([self UTF8String],
+                      [self lengthOfBytesUsingEncoding: NSUTF8StringEncoding]);
 }
 @end
 

bug#64697: 29.0.92: cannot paste NUL on macOS (regression from Emacs 28)

[Alan Third]

On Tue, Jul 18, 2023 at 10:30:38AM +0200, Mattias Engdegård wrote:
> In Emacs 29, text pasted from the clipboard on macOS will be truncated if containing NULs.
> 
> Reproduction: copy text with NUL in the middle from Emacs 28 (or any other application), and paste into Emacs 29.
> 
> This was probably caused by 7e3c2b553f, where construction of a Lisp string was changed from make_string to build_string.

LGTM

I considered this approach but wasn't aware of any situation where
make_string would be better. You've found one. :)

> Suggested patch attached. If valid, would it qualify for emacs-29?

That's up to the maintainers, but I will say that it looks safe, if
that's a concern.

-- 
Alan Third

bug#64697: 29.0.92: cannot paste NUL on macOS (regression from Emacs 28)

[Eli Zaretskii]

> Cc: 64697@debbugs.gnu.org
> Date: Tue, 18 Jul 2023 10:22:21 +0100
> From: Alan Third <alan@idiocy.org>
> 
> > This was probably caused by 7e3c2b553f, where construction of a Lisp string was changed from make_string to build_string.
> 
> I considered this approach but wasn't aware of any situation where
> make_string would be better. You've found one. :)

I would actually recommend the opposite rule of thumb: only use
build_string if you have good reasons to do so.

Basically, build_string is unreliable with anything but fixed-value
pure-ASCII strings.

bug#64697: 29.0.92: cannot paste NUL on macOS (regression from Emacs 28)

[Eli Zaretskii]

> Cc: Alan Third <alan@idiocy.org>
> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Tue, 18 Jul 2023 10:30:38 +0200
> 
> In Emacs 29, text pasted from the clipboard on macOS will be truncated if containing NULs.
> 
> Reproduction: copy text with NUL in the middle from Emacs 28 (or any other application), and paste into Emacs 29.
> 
> This was probably caused by 7e3c2b553f, where construction of a Lisp string was changed from make_string to build_string.
> 
> Suggested patch attached. If valid, would it qualify for emacs-29?

This is OK for emacs-29, but I wonder: do we want a unibyte string
here or a multibyte string?  We should try to call either
make_unibyte_string or make_multibyte_string accordingly, because
make_string has its own ideas about which one to produce.

Thanks.

bug#64697: 29.0.92: cannot paste NUL on macOS (regression from Emacs 28)

[Mattias Engdegård]

18 juli 2023 kl. 13.28 skrev Eli Zaretskii <eliz@gnu.org>:

> do we want a unibyte string
> here or a multibyte string?
>  We should try to call either
> make_unibyte_string or make_multibyte_string accordingly, because
> make_string has its own ideas about which one to produce.

That is more than I expected to change in emacs-29, but we could do that too if you like.

Since the input is UTF-8 the result, with the patch, will always be either a multibyte string or a unibyte ASCII string. Always making a multibyte string would also do and be marginally faster, but not more correct in any meaningful way.

bug#64697: 29.0.92: cannot paste NUL on macOS (regression from Emacs 28)

[Eli Zaretskii]

> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Tue, 18 Jul 2023 14:00:49 +0200
> Cc: 64697@debbugs.gnu.org,
>  alan@idiocy.org
> 
> 18 juli 2023 kl. 13.28 skrev Eli Zaretskii <eliz@gnu.org>:
> 
> > do we want a unibyte string
> > here or a multibyte string?
> >  We should try to call either
> > make_unibyte_string or make_multibyte_string accordingly, because
> > make_string has its own ideas about which one to produce.
> 
> That is more than I expected to change in emacs-29, but we could do that too if you like.

Let's install your change on emacs-29, and do the rest (if needed) on
master.

> Since the input is UTF-8 the result, with the patch, will always be either a multibyte string or a unibyte ASCII string. Always making a multibyte string would also do and be marginally faster, but not more correct in any meaningful way.

OK, but please add a comment there with the above rationale.

Thanks.

bug#64697: 29.0.92: cannot paste NUL on macOS (regression from Emacs 28)

[Mattias Engdegård]

18 juli 2023 kl. 15.02 skrev Eli Zaretskii <eliz@gnu.org>:

> Let's install your change on emacs-29, and do the rest (if needed) on
> master.

Now installed on emacs-29. Any improvement will be added on master after the merge.

Alan and Eli, thank you both.

bug#64697: 29.0.92: cannot paste NUL on macOS (regression from Emacs 28)

[Mattias Engdegård]

> Now installed on emacs-29. Any improvement will be added on master after the merge.

A faster conversion was added in 722b1ebc6e. Closing.