Re: Assertion failure in set_iterator_to_next

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> While playing with my nhexl-mode.el I bumped into some assertion failure
>> in the redisplay code.  More specifically the assertion
>> 
>> xassert (IT_BYTEPOS (*it) == CHAR_TO_BYTE (IT_CHARPOS (*it)));
>> 
>> at line 6183.
>> I think the problem is that the non-bidi code above that line uses
>> it-> len assuming it holds the length of the char at point (well, at
>> IT_CHARPOS(it)), whereas IIUC it holds the length of "the" glyph.

> No, it->len should be the length of the multibyte sequence of the
> character at IT_CHARPOS (*it).  This is how the basic iteration works,
> and if the length is incorrect, we will get garbled display, in
> anything but pure 7-bit ASCII buffers.

OK, good.  Can you update/complete dispextern.h correspondingly?
It currently says:

  /* If what == IT_CHARACTER, character and length in bytes.  This is
     a character from a buffer or string.  It may be different from
     the character displayed in case that
     unibyte_display_via_language_environment is set.

     If what == IT_COMPOSITION, the first component of a composition
     and length in bytes of the composition.  */
  int c, len;

which lead me to think that it->len is strongly linked to it->c.

>> In my case it->c holds sometimes the "lf symbol" char (u240a) which comes
>> from a display property and it->len is then 3, which is obviously not
>> the length of the char at IT_CHARPOS since the buffer I was looking at
>> only happens to contain ASCII chars.

> I think it->c is not always the character at IT_CHARPOS (*it).  In
> particular, this code is under `case GET_FROM_BUFFER', so the iterator
> is iterating through the buffer, not through any string.  But as you
> say, it->c still holds a character from a string.  I think it->c is
> loaded only in get_next_display_element, which is called _after_
> set_iterator_to_next advances to the next character.  And if it->what
> is anything other than a character, it->c will not be updated at all.

Fair enough.  So the problem now is "why is it->len ==3 rather than ==1"?

>> I use the quick patch below, which seems to fix the problem, but I don't
>> know enough of the redisplay to know whether that's the right fix, or
>> whether the right fix is to make sure it->len holds some other value.
> I don't think it's right.  Can you show a simple test case where this
> problem happens?

I wish I could.  I'll send one as soon as I find it, thanks,


        Stefan