bug#9389: 23.3.50; unencodable-char-position has buffer relocation problem

bug#9389: 23.3.50; unencodable-char-position has buffer relocation problem

[Kazuhiro Ito]

When I start precompiled Windows binary with -Q and evaluate below
code, I have unexpected result.

(with-temp-buffer
  (insert (make-string 16 ?A))
  (insert #x80)
  (unencodable-char-position 1 18 'ctext-unix))

-> 13 (Emacs 23.1)
-> 5  (Emacs 23.3)

If I evaluate it twice, it returns expected result (17).

I think the cause of the problem is similar to bug#9318.
unencodable-char-position uses char_charset(), which could cause a
relocation of buffes.  After using it, pointers must be updated as
needed.


=== modified file 'src/coding.c'
--- src/coding.c	2011-05-09 09:59:23 +0000
+++ src/coding.c	2011-08-27 04:29:23 +0000
@@ -8861,7 +8924,7 @@
   Lisp_Object attrs, charset_list, translation_table;
   Lisp_Object positions;
   int from, to;
-  const unsigned char *p, *stop, *pend;
+  const unsigned char *p, *stop, *pend, *orig;
   int ascii_compatible;
 
   setup_coding_system (Fcheck_coding_system (coding_system), &coding);
@@ -8881,7 +8944,7 @@
 	  || (ascii_compatible
 	      && (to - from) == (CHAR_TO_BYTE (to) - (CHAR_TO_BYTE (from)))))
 	return Qnil;
-      p = CHAR_POS_ADDR (from);
+      p = orig = CHAR_POS_ADDR (from);
       pend = CHAR_POS_ADDR (to);
       if (from < GPT && to >= GPT)
 	stop = GPT_ADDR;
@@ -8918,6 +8981,7 @@
   while (1)
     {
       int c;
+      struct charset *charset;
 
       if (ascii_compatible)
 	while (p < stop && ASCII_BYTE_P (*p))
@@ -8931,9 +8995,21 @@
 	}
 
       c = STRING_CHAR_ADVANCE (p);
+
+      charset_map_loaded = 0;
+      charset = char_charset (translate_char (translation_table, c),
+			      charset_list, NULL);
+      if (charset_map_loaded && NILP (string))
+	{
+	  EMACS_INT offset = CHAR_POS_ADDR (from) - orig;
+	  orig += offset;
+	  p += offset;
+	  pend += offset;
+	  stop += offset;
+	}
+
       if (! (ASCII_CHAR_P (c) && ascii_compatible)
-	  && ! char_charset (translate_char (translation_table, c),
-			     charset_list, NULL))
+	  && ! charset)
 	{
 	  positions = Fcons (make_number (from), positions);
 	  n--;

-- 
Kazuhiro Ito

bug#9389: 23.3.50; unencodable-char-position has buffer relocation problem

[Kenichi Handa]

In article <20110828000802.B9D1B34803A@msa103.auone-net.jp>, Kazuhiro Ito <kzhr@d1.dion.ne.jp> writes:

> When I start precompiled Windows binary with -Q and evaluate below
> code, I have unexpected result.

> (with-temp-buffer
>   (insert (make-string 16 ?A))
>   (insert #x80)
>   (unencodable-char-position 1 18 'ctext-unix))

> -> 13 (Emacs 23.1)
> -> 5  (Emacs 23.3)

> If I evaluate it twice, it returns expected result (17).

> I think the cause of the problem is similar to bug#9318.
> unencodable-char-position uses char_charset(), which could cause a
> relocation of buffes.  After using it, pointers must be updated as
> needed.

You are right.  I've just installed the attached patch
(which is a little bit different from yours).

---
Kenichi Handa
handa@m17n.org


=== modified file 'src/coding.c'
--- src/coding.c	2011-12-08 05:54:20 +0000
+++ src/coding.c	2011-12-11 11:14:15 +0000
@@ -8756,6 +8756,7 @@
     }
 
   positions = Qnil;
+  charset_map_loaded = 0;
   while (1)
     {
       int c;
@@ -8783,6 +8784,16 @@
 	}
 
       from++;
+      if (charset_map_loaded && NILP (string))
+	{
+	  p = CHAR_POS_ADDR (from);
+	  pend = CHAR_POS_ADDR (to);
+	  if (from < GPT && to >= GPT)
+	    stop = GPT_ADDR;
+	  else
+	    stop = pend;
+	  charset_map_loaded = 0;
+	}
     }
 
   return (NILP (count) ? Fcar (positions) : Fnreverse (positions));

bug#9389: 23.3.50; unencodable-char-position has buffer relocation problem

[Kazuhiro Ito]

> > When I start precompiled Windows binary with -Q and evaluate below
> > code, I have unexpected result.
> 
> > (with-temp-buffer
> >   (insert (make-string 16 ?A))
> >   (insert #x80)
> >   (unencodable-char-position 1 18 'ctext-unix))
> 
> > -> 13 (Emacs 23.1)
> > -> 5  (Emacs 23.3)
> 
> > If I evaluate it twice, it returns expected result (17).
> 
> > I think the cause of the problem is similar to bug#9318.
> > unencodable-char-position uses char_charset(), which could cause a
> > relocation of buffes.  After using it, pointers must be updated as
> > needed.
> 
> You are right.  I've just installed the attached patch
> (which is a little bit different from yours).

I confirmed the problem was fixed.  Thank you.

-- 
Kazuhiro Ito