bug#48029: 28.1; [native-comp] Function names with non-ascii characters

[Andrea Corallo <akrl@sdf.org>]

[-- Attachment #1: Type: text/plain, Size: 1973 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>, 48029@debbugs.gnu.org,
>>         jakanakaevangeli@chiru.no
>> Date: Thu, 07 Jul 2022 20:51:48 +0000
>> 
>> We have the symbol `nonascii-test-žžž' correct into our pipeline.
>> 
>> Then we prepare the file that will be executed by the subprocess that
>> will do the actual compilation, we do this in `comp-final'.
>> 
>> Here when we print `nonascii-test-žžž' using `prin1-to-string' this
>> becomes "nonascii-test-\x017e\x017e\x017e" (exploring it with the
>> debugger).
>> 
>> This is then stored in the temp file I mentioned, the file has on top
>> ";; -*-coding: nil; -*- " and the symbol is shown as
>> "nonascii-test-\305\276\305\276\305\276" if I visit it on my emacs.
>> 
>> Now two options, either we are not printing it as we should or either
>> the reader doesn't know he has to handle these non ascii characters
>> while reading back.
>> 
>> I'm no expert in this area, I guess will be evident to experts what we
>> are not doing correctly here.
>
> Why do we have ";; -*-coding: nil; -*- "?  I think that's the problem:
> it should be ";; -*-coding: utf-8; -*- " instead.  Where does the nil
> come from?

AFAIR at the time we encountered a situation when one of these files was
read back with some other encoding (I guess it's heuristic?) so fixing
the encoding to nil solved the issue.

> Lisp sources are by default UTF-8 encoded, so that should be the
> default for the temp file we write.  Bonus points for using the actual
> encoding of the Lisp source file there (which in very rare cases can
> be something other than UTF-8).

Ok attached the patch that sets it to utf-8, seems to work for me.

I'll have a look on how to spill the original coding system and get the
bonus points hopefully next week.

Should this patch go in master or into the release branch?

Regards

  Andrea


[-- Attachment #2: 0001-Mark-async-worker-tmp-file-as-utf-8-bug-48029.patch --]
[-- Type: text/x-diff, Size: 2295 bytes --]

From 76f18b38989e13136c191d40ef5c497161947394 Mon Sep 17 00:00:00 2001
From: Andrea Corallo <akrl@sdf.org>
Date: Fri, 8 Jul 2022 10:27:45 +0200
Subject: [PATCH] Mark async worker tmp file as utf-8 (bug#48029)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* lisp/emacs-lisp/comp.el (comp-final): Mark async worker tmp file
as utf-8.
* test/src/comp-tests.el (48029-1): New test.
* test/src/comp-resources/comp-test-funcs.el
(comp-test-48029-nonascii-žžž-f): New function.
---
 lisp/emacs-lisp/comp.el                    | 2 +-
 test/src/comp-resources/comp-test-funcs.el | 2 ++
 test/src/comp-tests.el                     | 5 +++++
 3 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
index 73285e0f24..9555bc8bd1 100644
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -3693,7 +3693,7 @@ comp-final
 				 (file-name-base output) "-")
 			 nil ".el")))
 	(with-temp-file temp-file
-          (insert ";; -*-coding: nil; -*-\n")
+          (insert ";; -*-coding: utf-8; -*-\n")
           (mapc (lambda (e)
                   (insert (prin1-to-string e)))
                 expr))
diff --git a/test/src/comp-resources/comp-test-funcs.el b/test/src/comp-resources/comp-test-funcs.el
index 0a60f4d6cc..9092f040c8 100644
--- a/test/src/comp-resources/comp-test-funcs.el
+++ b/test/src/comp-resources/comp-test-funcs.el
@@ -515,6 +515,8 @@ comp-test-47868-3-f
 (defun comp-test-47868-4-f ()
   #(" " 0 1 (face font-lock-keyword-face)))
 
+(defun comp-test-48029-nonascii-žžž-f (arg)
+  (when arg t))
 
 \f
 ;;;;;;;;;;;;;;;;;;;;
diff --git a/test/src/comp-tests.el b/test/src/comp-tests.el
index e7b534d00e..c7e81298d2 100644
--- a/test/src/comp-tests.el
+++ b/test/src/comp-tests.el
@@ -532,6 +532,11 @@ comp-test-47868-1
   (should (eq (comp-test-47868-1-f) (comp-test-47868-3-f)))
   (should (eq (comp-test-47868-2-f) (comp-test-47868-4-f))))
 
+(comp-deftest 48029-1 ()
+  "<https://lists.gnu.org/archive/html/bug-gnu-emacs/2022-07/msg00666.html>"
+  (should (subr-native-elisp-p
+           (symbol-function 'comp-test-48029-nonascii-žžž-f))))
+
 \f
 ;;;;;;;;;;;;;;;;;;;;;
 ;; Tromey's tests. ;;
-- 
2.25.1