bug#24206: 25.1; Curly quotes generate invalid strings, leading to a segfault

[Paul Eggert <eggert@cs.ucla.edu>]

[-- Attachment #1: Type: text/plain, Size: 1322 bytes --]

Andreas Schwab wrote:
> Unibyte strings should not be used for anything but pre-decoding /
> post-encoding situations directly after / before doing the actual I/O
> operation.  Thus substitute-command-keys should never be called with an
> undecoded unibyte string.  IMHO it would be ok to return something
> useless in this case (but it shouldn't cause Emacs to crash, of course).

Yes. This is in the Elisp manual, which says "We recommend that
you never use unibyte buffers and strings except for manipulating
encoded text or binary non-text data."

Eli Zaretskii wrote:
> as the original string is
> unibyte, the output of "\200≠", which is multibyte, might not be what
> the users expect.  They might expect "\200\342\211\240" instead.

No, as per Andreas's comment and the Elisp reference manual, users should not 
expect substitute-command-keys to do that. As long as it doesn't crash on 
non-ASCII unibyte data we needn't sweat the details about whether it returns 
unibyte or multibyte strings for such data.

That being said, it shouldn't hurt to return the original string if there is no 
substitution. Although I think your first patch in 
<http://bugs.gnu.org/24206#61> should work, it'd be safer and simpler to 
backport that part of master, as in the attached patch to emacs-25.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-core-dump-with-unibyte-substitute-command-keys.patch --]
[-- Type: text/x-diff; name="0001-Fix-core-dump-with-unibyte-substitute-command-keys.patch", Size: 1021 bytes --]

From c4cdebc78783f337cb504b6fea71295cfcaaa516 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Mon, 15 Aug 2016 11:34:57 -0700
Subject: [PATCH] Fix core dump with unibyte substitute-command-keys

Backport from master (Bug#24206).
* src/doc.c (Fsubstitute_command_keys): Convert string to
multibyte first, but return the original if there is no
substitution.
---
 src/doc.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/src/doc.c b/src/doc.c
index 7cdb0d0..4f5f7a4 100644
--- a/src/doc.c
+++ b/src/doc.c
@@ -763,6 +763,8 @@ Otherwise, return a new string.  */)
 
   enum text_quoting_style quoting_style = text_quoting_style ();
 
+  Lisp_Object orig_string = string;
+  string = Fstring_make_multibyte (string);
   multibyte = STRING_MULTIBYTE (string);
   nchars = 0;
 
@@ -1024,7 +1026,7 @@ Otherwise, return a new string.  */)
 	}
     }
   else
-    tem = string;
+    tem = orig_string;
   xfree (buf);
   return tem;
 }
-- 
2.7.4