From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#9771: 24.0.90; Redisplay problems with control characters Date: Tue, 18 Oct 2011 08:56:34 -0400 Message-ID: References: <871uucr2w1.fsf@gnu.org> <874nz79vo1.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: dough.gmane.org 1318942654 8154 80.91.229.12 (18 Oct 2011 12:57:34 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 18 Oct 2011 12:57:34 +0000 (UTC) Cc: 9771@debbugs.gnu.org To: Johan =?UTF-8?Q?Bockg=C3=A5rd?= , handa@m17n.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Tue Oct 18 14:57:24 2011 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RG9EM-0006Os-IU for geb-bug-gnu-emacs@m.gmane.org; Tue, 18 Oct 2011 14:57:19 +0200 Original-Received: from localhost ([::1]:49303 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RG9EL-00030T-US for geb-bug-gnu-emacs@m.gmane.org; Tue, 18 Oct 2011 08:57:17 -0400 Original-Received: from eggs.gnu.org ([140.186.70.92]:59382) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RG9EG-0002z7-1W for bug-gnu-emacs@gnu.org; Tue, 18 Oct 2011 08:57:16 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RG9E9-0002FG-Ad for bug-gnu-emacs@gnu.org; Tue, 18 Oct 2011 08:57:11 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:55139) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RG9E9-0002F3-8L for bug-gnu-emacs@gnu.org; Tue, 18 Oct 2011 08:57:05 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.69) (envelope-from ) id 1RG9F3-0004em-PW for bug-gnu-emacs@gnu.org; Tue, 18 Oct 2011 08:58:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 18 Oct 2011 12:58:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 9771 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 9771-submit@debbugs.gnu.org id=B9771.131894265517863 (code B ref 9771); Tue, 18 Oct 2011 12:58:01 +0000 Original-Received: (at 9771) by debbugs.gnu.org; 18 Oct 2011 12:57:35 +0000 Original-Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RG9Ed-0004e4-K1 for submit@debbugs.gnu.org; Tue, 18 Oct 2011 08:57:35 -0400 Original-Received: from fencepost.gnu.org ([140.186.70.10] ident=Debian-exim) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1RG9Eb-0004dx-Cr for 9771@debbugs.gnu.org; Tue, 18 Oct 2011 08:57:34 -0400 Original-Received: from eliz by fencepost.gnu.org with local (Exim 4.71) (envelope-from ) id 1RG9De-0004Zm-5N; Tue, 18 Oct 2011 08:56:34 -0400 In-reply-to: <874nz79vo1.fsf@gnu.org> (message from Johan =?UTF-8?Q?Bockg=C3=A5rd?= on Mon, 17 Oct 2011 23:04:46 +0200) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list Resent-Date: Tue, 18 Oct 2011 08:58:01 -0400 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:52782 Archived-At: > From: Johan Bockgård > Cc: 9771@debbugs.gnu.org > Date: Mon, 17 Oct 2011 23:04:46 +0200 > > Eli Zaretskii writes: > > >> According to the condition above, the position in column 0 before the > >> ^ glyph (dpvec_index = 0) is not a possible stop point, but the position > >> between ^ and @ is. > > > > Okay, but what is the practical problem with this? > > Nothing, really. But it doesn't seem correct. I'm not sure. Maybe I'm missing something (move_it_in_display_line_to is one of the trickiest functions in the display engine), so please bear with me as I explain why I think the test is correct. You wrote: > #define BUFFER_POS_REACHED_P() \ > [...] > && (it->method == GET_FROM_BUFFER \ > || (it->method == GET_FROM_DISPLAY_VECTOR \ > && it->dpvec + it->current.dpvec_index + 1 >= it->dpend))) > > According to the condition above, the position in column 0 before the > ^ glyph (dpvec_index = 0) is not a possible stop point, but the position > between ^ and @ is. A display vector has its glyphs stored at indices 0..it->dpend-1. IOW, the last glyph's index is it->dpend-1. BUFFER_POS_REACHED_P is invoked always after a call to get_next_display_element, but before the call to set_iterator_to_next. The former just consumes the glyph at it->current.dpvec_index; the latter advances the index to the next glyph of the display vector. Therefore, after a call to get_next_display_element, the glyph at dpvec_index was already consumed, but the index was not yet advanced to the next position. Thus, when the index is at dpend-1, as the condition in BUFFER_POS_REACHED_P says, we have already consumed the last glyph. Consuming the last glyph of a display vector means that the next call to set_iterator_to_next will detect that the display vector is exhausted, and will advance to the next buffer position, the one after the position which we just passed by consuming all the glyphs from the display vector used to display the character at that position. Thus, "buffer position reached". Does this make sense? In practice, after correcting the bug that caused the assertion violation, I can no longer reproduce the situation where we stop in the middle of the ^@ character. If you can show me a recipe for winding up in the middle of a display vector under these or similar circumstances, I will have another look. > > Binary nulls in a file generally cause Emacs to make the buffer > > unibyte, where bidi reordering is disabled. > > That doesn't seem to be working, then. > > If I do > > emacs -Q /usr/bin/emacs > > the resulting buffer is multibyte. Indeed. However, buffer-file-coding-system is no-conversion, which is confusingly inconsistent with "C-x RET c no-conversion C-x C-f". Perhaps Handa-san could shed some light on this inconsistency. Anyway, this being so, I found a way to optimize a couple of expensive loops inside bidi.c for the important special case of a plain L2R text in the buffer. With these optimizations, the bidi redisplay of the entire window showing a buffer with 2000 binary nulls is only a few milliseconds slower than the unidirectional one. I will install those changes later today, after some more testing. > Type M-> and Emacs hangs. (I waited 5 minutes and had to kill the > process.) It takes 1.5 seconds now (with a 44MB emacs binary ;-)