From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#11094: Wrong cursor positioning with display+invisible Date: Sat, 31 Mar 2012 12:33:17 +0300 Message-ID: <83vclljesi.fsf@gnu.org> References: Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="Boundary_(ID_JufyzPVdrRs1IMG5yD9Erw)" X-Trace: dough.gmane.org 1333186449 5670 80.91.229.3 (31 Mar 2012 09:34:09 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sat, 31 Mar 2012 09:34:09 +0000 (UTC) Cc: 11094@debbugs.gnu.org To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Mar 31 11:34:08 2012 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1SDuhD-0000Jy-Rc for geb-bug-gnu-emacs@m.gmane.org; Sat, 31 Mar 2012 11:34:08 +0200 Original-Received: from localhost ([::1]:34874 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SDuhC-0004PM-SZ for geb-bug-gnu-emacs@m.gmane.org; Sat, 31 Mar 2012 05:34:06 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:49558) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SDuh8-0004Oj-8s for bug-gnu-emacs@gnu.org; Sat, 31 Mar 2012 05:34:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1SDuh5-0005MX-VV for bug-gnu-emacs@gnu.org; Sat, 31 Mar 2012 05:34:01 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:35328) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1SDuh5-0005MP-RV for bug-gnu-emacs@gnu.org; Sat, 31 Mar 2012 05:33:59 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.72) (envelope-from ) id 1SDuh8-0005lk-K0 for bug-gnu-emacs@gnu.org; Sat, 31 Mar 2012 05:34:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 31 Mar 2012 09:34:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 11094 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 11094-submit@debbugs.gnu.org id=B11094.133318643722165 (code B ref 11094); Sat, 31 Mar 2012 09:34:02 +0000 Original-Received: (at 11094) by debbugs.gnu.org; 31 Mar 2012 09:33:57 +0000 Original-Received: from localhost ([127.0.0.1]:60099 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SDuh2-0005lR-Gz for submit@debbugs.gnu.org; Sat, 31 Mar 2012 05:33:56 -0400 Original-Received: from mtaout22.012.net.il ([80.179.55.172]:42282) by debbugs.gnu.org with esmtp (Exim 4.72) (envelope-from ) id 1SDugy-0005lH-Sr for 11094@debbugs.gnu.org; Sat, 31 Mar 2012 05:33:54 -0400 Original-Received: from conversion-daemon.a-mtaout22.012.net.il by a-mtaout22.012.net.il (HyperSendmail v2007.08) id <0M1Q00C00T445D00@a-mtaout22.012.net.il> for 11094@debbugs.gnu.org; Sat, 31 Mar 2012 12:33:18 +0300 (IDT) Original-Received: from HOME-C4E4A596F7 ([84.228.100.223]) by a-mtaout22.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0M1Q00BMZT7GD3N0@a-mtaout22.012.net.il>; Sat, 31 Mar 2012 12:33:18 +0300 (IDT) In-reply-to: X-012-Sender: halo1@inter.net.il X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.13 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:58372 Archived-At: --Boundary_(ID_JufyzPVdrRs1IMG5yD9Erw) Content-type: text/plain > From: Stefan Monnier > Date: Mon, 26 Mar 2012 00:07:32 -0400 > > Extract the test.cpio and cpio-mode.el files attached, and then try: > > src/emacs -Q -l .../cpio-mode.el .../test.cpio Ouch! (Btw, test.cpio is an invalid archive, according to 2 different implementations of the cpio command. The one from GNU/Linux says "cpio: premature end of file". Not that it matters for the purposes of this discussion.) > First, you'll note that the cursor is displayed at end-of-line (whereas > in reality point as it BOB), then move forward with C-f and you'll see > the cursor jumping in odd ways (often being pushed to end-of-line even > when we're still in the middle of the line). Right. The end-of-line placement of the cursor usually means that the display engine knows the cursor should be on this line, but it didn't find a proper place to put it. And in this case, for a good reason, see below. > In Emacs-23, the cursor jumps correctly over the various display and > invisible fields For some value of "correctly". E.g., position the cursor over the ".", which is the first file name in the archive, and type "C-x =". You will see "63", which is a lie: the actual value of point is 111. This lie becomes more evident if you position the cursor on the first 's' of "scripts", the file name on the second line: "C-x =" says "175" (whereas the truth is 223), but if you now type "C-f C-x =", you will see "224". Huh? Anyway, that this "works" in Emacs 23 and before is more by accident than by anything else. Your cpio-mode.el almost completely conceals the buffer contents from the display engine, either by 'invisible' property or behind display strings. The cursor-positioning code is therefore completely helpless when it comes to decide where to put the cursor, because all it has is (a) the value of point, and (b) the glyphs generated for each display line (a.k.a. "glyph row"). With most of the buffer positions covered by 'invisible' and 'display' properties, no glyphs to place the cursor on can be found for most values of point, since there are no glyphs which "tell" the display engine that they came from those buffer positions. The old, unidirectional display engine could get away (albeit by the skin of its teeth) with such code because it relied on buffer positions to increase monotonically with screen positions. So once it found a glyph from buffer position N that is greater or equal to the value of point, it could place the cursor there, because it "knew" all the later glyphs _must_ correspond to positions beyond N. That is why it "works" in Emacs 23. A case in point is the position of cursor at BOB. The value of point is 1, whereas the first glyph on the first screen lines comes from buffer position 15(!). The old display code could put the cursor there _only_ because that is the first glyph whose buffer position is greater or equal to 1. However, the bidirectional display engine can no longer make such assumptions, so it is helpless without at least _some_ clue about the buffer positions corresponding to the glyphs. And your code gives almost no such clues. I think we should either (1) from now on discourage (i.e., maintain that we don't support) code that covers too much of buffer text by display strings, or (2) completely redesign 'struct glyph', so that it keeps more information about the buffer positions "associated" with the glyph, and then refactor the code which needs that information. (The second alternative is, of course, not for Emacs 24.) Alternatively, may I suggest that you use the 'cursor' property to provide some rope for the display engine to do its job even in this situation? The attached variant of your code demonstrates how to do that; it works equally well with Emacs 23 and with the current trunk. Bottom line, to resolve this bug for Emacs 24.1, I suggest a judicial use of the 'cursor' property as the only safe (and IMO the only sane) alternative. If you still think we should support your original code, we should schedule some post-24.1 redesign and refactoring. Let me know what you think. Here's a modified version of cpio-mode.el. Note that it removes most of the 'invisible' properties, because they are not really needed, and because they have an unpleasant side effect of reporting incorrect value of point due to "point adjustments". I suspect that all the other 'invisible' properties should be removed as well, but I didn't test the situations where they would come into play. --Boundary_(ID_JufyzPVdrRs1IMG5yD9Erw) Content-type: application/emacs-lisp Content-transfer-encoding: quoted-printable Content-disposition: attachment; filename=cpio-mode1.el ;;; cpio-mode.el --- View/edit cpio archives ;; Copyright (C) 2012 Stefan Monnier ;; Author: Stefan Monnier ;; Keywords:=20 ;; This program is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with this program. If not, see . ;;; Commentary: ;;=20 ;;; Code: ;; The format supported is the "New ASCII Format" as documented in ;; http://people.freebsd.org/~kientzle/libarchive/man/cpio.5.txt. (eval-when-compile (require 'cl)) ;;;###autoload (add-to-list 'auto-mode-alist '("\\.cpio\\'" . cpio-mode)) ;;;###autoload (when (boundp 'inhibit-local-variables-regexps) (add-to-list 'inhibit-local-variables-regexps "\\.cpio\\'")) ;;;###autoload (define-derived-mode cpio-mode special-mode "CPIO" "View/edit cpio archives." (add-hook 'change-major-mode-hook (lambda () (with-silent-modifications (save-restriction (widen) (remove-text-properties (point-min) (point-max) '(display nil invisible nil))= )))) (cpio--parse-buffer)) (defun cpio--parse-mode (mode) (string (case (logand #o170000 mode) (#o140000 ?s) (#o120000 ?l) (#o100000 ?-) (#o060000 ?b) (#o040000 ?d) (#o020000 ?c) (#o010000 ?p) (t error "Unknown mode %S" mode)) (if (zerop (logand #o400 mode)) ?- ?r) (if (zerop (logand #o200 mode)) ?- ?w) (if (zerop (logand #o100 mode)) (if (zerop (logand #o4000 mode)) ?- ?S) (if (zerop (logand #o4000 mode)) ?x ?s)) (if (zerop (logand #o040 mode)) ?- ?r) (if (zerop (logand #o020 mode)) ?- ?w) (if (zerop (logand #o010 mode)) (if (zerop (logand #o2000 mode)) ?- ?S) (if (zerop (logand #o2000 mode)) ?x ?s)) (if (zerop (logand #o004 mode)) ?- ?r) (if (zerop (logand #o002 mode)) ?- ?w) (if (zerop (logand #o001 mode)) ?- ?x))) (defun put-display-string-with-cursor (start end string &optional count) "Display STRING between START and END, with `cursor' property on 1st char. Optional argument COUNT controls the value of the `cursor' property. If it is nil or omitted, the value will be an integer number of buffer positions between START and END. Any other value will be used as the value of `cursor' property. Useful kinds of values include an integer number or a symbol." (put-text-property start end 'display (concat (propertize (substring string 0 1) 'cursor (or count (- end start 1))) (substring string 1)))) (defun cpio--parse-buffer () (save-excursion (save-restriction (widen) (goto-char (point-min)) (with-silent-modifications (while (not (eobp)) (assert (zerop (mod (- (point) (point-min)) 4))) (cond ((not (looking-at "07070[12]\\([[:xdigit:]]\\{8\\}\\)\\([[:xdigi= t:]]\\{8\\}\\)\\([[:xdigit:]]\\{8\\}\\)\\([[:xdigit:]]\\{8\\}\\)\\([[:xdigi= t:]]\\{8\\}\\)\\([[:xdigit:]]\\{8\\}\\)\\([[:xdigit:]]\\{8\\}\\)\\([[:xdigi= t:]]\\{8\\}\\)\\([[:xdigit:]]\\{8\\}\\)\\([[:xdigit:]]\\{8\\}\\)\\([[:xdigi= t:]]\\{8\\}\\)\\([[:xdigit:]]\\{8\\}\\)[[:xdigit:]]\\{8\\}\\|\0+\\'")) (error "Unrecognized cpio header format")) ((not (match-beginning 1)) ;; Reached the trailing padding, just skip it. (put-text-property (point) (point-max) 'invisible t) (goto-char (match-end 0))) (t (let* ((ino (string-to-number (match-string 1) 16)) (mode (string-to-number (match-string 2) 16)) (uid (string-to-number (match-string 3) 16)) (gid (string-to-number (match-string 4) 16)) (nlink (string-to-number (match-string 5) 16)) (mtime (string-to-number (match-string 6) 16)) (filesize (string-to-number (match-string 7) 16)) (devmajor (string-to-number (match-string 8) 16)) (devminor (string-to-number (match-string 9) 16)) (rdevmajor (string-to-number (match-string 10) 16)) (rdevminor (string-to-number (match-string 11) 16)) (namesize (string-to-number (match-string 12) 16)) (namebeg (match-end 0)) (name (buffer-substring namebeg (+ namebeg namesize -1))) (filebeg (+ (match-end 0) 2 (* (/ (+ namesize 1) 4) 4))) (next (+ filebeg (* (/ (+ filesize 3) 4) 4)))) (if (and (zerop ino) (zerop mode) (zerop filesize) (equal name "TRAILER!!!")) (put-display-string-with-cursor (match-beginning 0) next "\n") (if (bobp) (put-display-string-with-cursor (point) (match-end 2) (cpio--parse-mode mode) t) (put-text-property (point) (match-end 1) 'display "\n") (put-display-string-with-cursor (match-end 1) (match-end 2) (cpio--parse-mode mode) t)) (put-display-string-with-cursor (match-beginning 3) (match-end 3) (format "%10d" uid) t) (put-display-string-with-cursor (match-beginning 4) (match-end 4) (format "/%-10d" gid) t) (put-display-string-with-cursor (match-beginning 5) namebeg (format "%10d " filesize) t) (if (=3D (logand #o170000 mode) #o120000) ;Symlink (progn (put-text-property (+ namebeg namesize -1) filebeg 'display " -> ") (put-text-property (+ filebeg filesize) next 'invisible t)) (put-text-property (+ namebeg namesize -1) next 'invisible t)) (message "Parsed %S of size %d" name filesize)) (goto-char next))))))))) (provide 'cpio-mode) ;;; cpio-mode.el ends here --Boundary_(ID_JufyzPVdrRs1IMG5yD9Erw)--