bug#38191: incorrect text properties in result of `format' with multibyte(?) characters

bug#38191: incorrect text properties in result of `format' with multibyte(?) characters

[Paul Pogonyshev]

"Multibyte" is a guess, I don't really know the underlying reason.

Examples:

(format (propertize "`foo' %s bar" 'face 'bold) "xxx")
  => #("`foo' xxx bar" 0 13 (face bold))

(format (propertize "‘foo’ %s bar" 'face 'bold) "xxx")
  => #("‘foo’ xxx bar" 0 10 (face bold))

Length of the string is the same in both cases. In the first example
the face is correctly applied to the whole string, in the second
example 3 last characters incorrectly lack a face.

This is a regression, it used to work correctly before, but I don't
know when it became broken.

Tested on: GNU Emacs 27.0.50 (build 3, x86_64-pc-linux-gnu, GTK+
Version 2.24.32) of 2019-11-13

Paul

bug#38191: incorrect text properties in result of `format' with multibyte(?) characters

[Lars Ingebrigtsen]

Paul Pogonyshev <pogonyshev@gmail.com> writes:

> "Multibyte" is a guess, I don't really know the underlying reason.
>
> Examples:
>
> (format (propertize "`foo' %s bar" 'face 'bold) "xxx")
>   => #("`foo' xxx bar" 0 13 (face bold))
>
> (format (propertize "‘foo’ %s bar" 'face 'bold) "xxx")
>   => #("‘foo’ xxx bar" 0 10 (face bold))
>
> Length of the string is the same in both cases. In the first example
> the face is correctly applied to the whole string, in the second
> example 3 last characters incorrectly lack a face.
>
> This is a regression, it used to work correctly before, but I don't
> know when it became broken.

It's always off by the length of the inserted string, so it's at least
systematic:

(format (propertize "ççfoo %s bar" 'face 'bold) "xxx")
=> #("ççfoo xxx bar" 0 10 (face bold))

(format (propertize "ççfoo %s bar" 'face 'bold) "xxxx")
=> #("ççfoo xxxx bar" 0 10 (face bold))

It doesn't happen if there's just one multibyte character in the format
spec -- there has to be two or more.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38191: incorrect text properties in result of `format' with multibyte(?) characters

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 69 bytes --]

Thanks for reporting that. I installed the attached patch to fix it.

[-- Attachment #2: 0001-Fix-byte-counting-error-in-format.patch --]
[-- Type: text/x-patch, Size: 2619 bytes --]

From 63aafe766e7d81aa7b2e75bddf2ce3eca4611ae5 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Thu, 14 Nov 2019 14:42:39 -0800
Subject: [PATCH] =?UTF-8?q?Fix=20byte-counting=20error=20in=20=E2=80=98for?=
 =?UTF-8?q?mat=E2=80=99?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Problem reported by Paul Pogonyshev (Bug#38191).
* src/editfns.c (styled_format): When checking for adjacent
%-sequences, use byte position rather than character position.
* test/src/editfns-tests.el (format-properties): Test for fix.
---
 src/editfns.c             | 6 +++---
 test/src/editfns-tests.el | 4 ++++
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/src/editfns.c b/src/editfns.c
index 1b33f39711..8fc866d391 100644
--- a/src/editfns.c
+++ b/src/editfns.c
@@ -3143,7 +3143,7 @@ styled_format (ptrdiff_t nargs, Lisp_Object *args, bool message)
     /* The start and end bytepos in the output string.  */
     ptrdiff_t start, end;
 
-    /* The start of the spec in the format string.  */
+    /* The start bytepos of the spec in the format string.  */
     ptrdiff_t fbeg;
 
     /* Whether the argument is a string with intervals.  */
@@ -3954,7 +3954,7 @@ styled_format (ptrdiff_t nargs, Lisp_Object *args, bool message)
 		    {
 		      position++;
 		      if (fieldn < nspec
-			  && position > info[fieldn].fbeg
+			  && bytepos >= info[fieldn].fbeg
 			  && translated == info[fieldn].start)
 			{
 			  translated += info[fieldn].end - info[fieldn].start;
@@ -3976,7 +3976,7 @@ styled_format (ptrdiff_t nargs, Lisp_Object *args, bool message)
 		    {
 		      position++;
 		      if (fieldn < nspec
-			  && position > info[fieldn].fbeg
+			  && bytepos >= info[fieldn].fbeg
 			  && translated == info[fieldn].start)
 			{
 			  translated += info[fieldn].end - info[fieldn].start;
diff --git a/test/src/editfns-tests.el b/test/src/editfns-tests.el
index a1060808f6..238fb409f0 100644
--- a/test/src/editfns-tests.el
+++ b/test/src/editfns-tests.el
@@ -90,6 +90,10 @@ format-properties
                                    (propertize "45" 'face 'italic)))
            #("012345    "
              0 2 (face bold) 2 4 (face underline) 4 10 (face italic))))
+  ;; Bug #38191
+  (should (ert-equal-including-properties
+           (format (propertize "‘foo’ %s bar" 'face 'bold) "xxx")
+           #("‘foo’ xxx bar" 0 13 (face bold))))
   ;; Bug #32404
   (should (ert-equal-including-properties
            (format (concat (propertize "%s" 'face 'bold)
-- 
2.23.0