> From: Stefan Monnier > Date: Mon, 26 Mar 2012 00:07:32 -0400 > > Extract the test.cpio and cpio-mode.el files attached, and then try: > > src/emacs -Q -l .../cpio-mode.el .../test.cpio Ouch! (Btw, test.cpio is an invalid archive, according to 2 different implementations of the cpio command. The one from GNU/Linux says "cpio: premature end of file". Not that it matters for the purposes of this discussion.) > First, you'll note that the cursor is displayed at end-of-line (whereas > in reality point as it BOB), then move forward with C-f and you'll see > the cursor jumping in odd ways (often being pushed to end-of-line even > when we're still in the middle of the line). Right. The end-of-line placement of the cursor usually means that the display engine knows the cursor should be on this line, but it didn't find a proper place to put it. And in this case, for a good reason, see below. > In Emacs-23, the cursor jumps correctly over the various display and > invisible fields For some value of "correctly". E.g., position the cursor over the ".", which is the first file name in the archive, and type "C-x =". You will see "63", which is a lie: the actual value of point is 111. This lie becomes more evident if you position the cursor on the first 's' of "scripts", the file name on the second line: "C-x =" says "175" (whereas the truth is 223), but if you now type "C-f C-x =", you will see "224". Huh? Anyway, that this "works" in Emacs 23 and before is more by accident than by anything else. Your cpio-mode.el almost completely conceals the buffer contents from the display engine, either by 'invisible' property or behind display strings. The cursor-positioning code is therefore completely helpless when it comes to decide where to put the cursor, because all it has is (a) the value of point, and (b) the glyphs generated for each display line (a.k.a. "glyph row"). With most of the buffer positions covered by 'invisible' and 'display' properties, no glyphs to place the cursor on can be found for most values of point, since there are no glyphs which "tell" the display engine that they came from those buffer positions. The old, unidirectional display engine could get away (albeit by the skin of its teeth) with such code because it relied on buffer positions to increase monotonically with screen positions. So once it found a glyph from buffer position N that is greater or equal to the value of point, it could place the cursor there, because it "knew" all the later glyphs _must_ correspond to positions beyond N. That is why it "works" in Emacs 23. A case in point is the position of cursor at BOB. The value of point is 1, whereas the first glyph on the first screen lines comes from buffer position 15(!). The old display code could put the cursor there _only_ because that is the first glyph whose buffer position is greater or equal to 1. However, the bidirectional display engine can no longer make such assumptions, so it is helpless without at least _some_ clue about the buffer positions corresponding to the glyphs. And your code gives almost no such clues. I think we should either (1) from now on discourage (i.e., maintain that we don't support) code that covers too much of buffer text by display strings, or (2) completely redesign 'struct glyph', so that it keeps more information about the buffer positions "associated" with the glyph, and then refactor the code which needs that information. (The second alternative is, of course, not for Emacs 24.) Alternatively, may I suggest that you use the 'cursor' property to provide some rope for the display engine to do its job even in this situation? The attached variant of your code demonstrates how to do that; it works equally well with Emacs 23 and with the current trunk. Bottom line, to resolve this bug for Emacs 24.1, I suggest a judicial use of the 'cursor' property as the only safe (and IMO the only sane) alternative. If you still think we should support your original code, we should schedule some post-24.1 redesign and refactoring. Let me know what you think. Here's a modified version of cpio-mode.el. Note that it removes most of the 'invisible' properties, because they are not really needed, and because they have an unpleasant side effect of reporting incorrect value of point due to "point adjustments". I suspect that all the other 'invisible' properties should be removed as well, but I didn't test the situations where they would come into play.