bug#24206: 25.1; Curly quotes generate invalid strings, leading to a segfault

[Paul Eggert <eggert@cs.ucla.edu>]

[-- Attachment #1: Type: text/plain, Size: 2051 bytes --]

Eli Zaretskii wrote:
> I've reviewed the changes you pushed to master for fixing this bug,
> and I must say that most of them look like purely stylistic changes,
> to make the code more to your personal liking.  The actual bugs were
> very few and minor, and didn't necessitate such thorough changes.

The changes were motivated by bug fixes, not style. Although the bugs were 
mostly minor (e.g., generating bogus NUL bytes due to miscounting) it's fine to 
fix minor bugs. I did change nearby style (indenting as per GNU style, switching 
some locals to C99-style decl-after-statement, etc.) but none of the changes 
were pervasive or were intended for the emacs-25 branch, and it's fine to make 
such changes in master.

One of the bugs was O(N**2) performance when reallocating a temporary buffer. 
While I was at it, I changed the code to allocate a small temp buffer on the 
stack to avoid a malloc/free in the usual case, which should be a small win. 
This accounts for many changes that a quick glance might give the mistaken 
impression of being stylistic.

> without any comments as to why we handle the input string as
> multibyte for the rest of the function, I think this will confuse
> someone down the road.

OK, I added some comments along those lines (see attached patch).

> I think the refactoring already introduced a
> regression.

This comment appears to be about changes made in May for Bug#23425, not about 
code changes I made recently. The May changes were not a regression; they were 
intended and are documented in etc/NEWS. Alan Mackenzie felt strongly that some 
changes were needed in this area. See commit 
433d366dc7b053048abf710d790ff62421dd1570.

> maybe we should just go to the original code, after fixing the
> immediate bugs, as I proposed in a patch yesterday?

No, the code in master should be uniformly better than what's in emacs-25.

It would be nice to have good tests for substitute-command-keys, of course. (We 
can all add this to our lists of things to do. :-)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Omit-substitute-command-keys-code-no-longer-needed.patch --]
[-- Type: text/x-diff; name="0001-Omit-substitute-command-keys-code-no-longer-needed.patch", Size: 1420 bytes --]

From bc5a55262382da9cc4b1999df7ce195b7f825d4b Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 16 Aug 2016 13:16:49 -0700
Subject: [PATCH] Omit substitute-command-keys code no longer needed

* src/doc.c (Fsubstitute_command_keys):
Remove duplicate initializations.
---
 src/doc.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/src/doc.c b/src/doc.c
index 37a731b..4b91831 100644
--- a/src/doc.c
+++ b/src/doc.c
@@ -743,10 +743,12 @@ Otherwise, return a new string.  */)
   if (NILP (string))
     return Qnil;
 
+  /* If STRING contains non-ASCII unibyte data, process its
+     properly-encoded multibyte equivalent instead.  This simplifies
+     the implementation and is OK since substitute-command-keys is
+     intended for use only on text strings.  Keep STRING around, since
+     it will be returned if no changes occur.  */
   Lisp_Object str = Fstring_make_multibyte (string);
-  tem = Qnil;
-  keymap = Qnil;
-  name = Qnil;
 
   enum text_quoting_style quoting_style = text_quoting_style ();
 
@@ -905,6 +907,8 @@ Otherwise, return a new string.  */)
 	 }
 
 	subst_string:
+	  /* Convert non-ASCII unibyte data to properly-encoded multibyte,
+	     for the same reason STRING was converted to STR.  */
 	  tem = Fstring_make_multibyte (tem);
 	  start = SDATA (tem);
 	  length = SCHARS (tem);
-- 
2.7.4