Re: Tracking down assertion failure

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> /* Skip from tlbufpos to PT and see where it is.  Note that
>> PT may be in invisible text.  If so, we will end at the
>> next visible position.  */
>> init_iterator (&it, w, CHARPOS (tlbufpos), BYTEPOS (tlbufpos),
>> NULL, DEFAULT_FACE_ID);
>> xassert (IT_BYTEPOS (it) == CHAR_TO_BYTE (IT_CHARPOS (it)));
>> 
>> the problem is that as I keep adding such assertions earlier and earlier
>> in the code I seem to start hitting another problem: I'm not sure at
>> which point this condition should be true and and at which point it's OK
>> for it not to be true (because the charpos and bytepos recorded refer
>> to out-of-data data which will/should simply not be used).
>> 
>> Can someone help me out?  

> I may be forgetting something, but isn't the iterator moving by
> characters?  It uses the pair

>       IT_BYTEPOS (*it) += it->len;
>       IT_CHARPOS (*it) += 1;

> to advance, so IT_BYTEPOS and IT_CHARPOS should always be in sync,
> right?  Am I missing something?

Yes, they should.  I'm pretty sure that the above xassert should
not fail.  Yet it does (typically the bytepos is 1 larger than the
charpos, even tho the buffer contains only ascii, and typically it
happens when moving around the buffer with the arrow keys, it seems to
have to do with handling of the cursor).

The problem is that when we enter the cursor code, it's not clear to me
which charpos/bytepos pairs are expected to be valid and which ones are
expected to be potentially out-of-date and I don't understand either
enough about how/when we check them being up-to-date.

The bug cannot be reproduced deterministically, but I hit it several
times a day.  After adding xasserts similar to the above further up the
call chain, all I managed to get is that those asserts fail more often,
but I'm not convinced that their failure is really a bug, because I'm
not sure whether the values I check should actually be up-to-date.


        Stefan