bug#17585: 24.4.50; vertical-motion erroneously adds points

bug#17585: 24.4.50; vertical-motion erroneously adds points

[Keith David Bershatsky]

The following test function demonstrates how vertical-motion erroneously adds points when dealing with long lines without spaces that go beyond the window-width:

    (defun test ()
    (interactive)
      (switch-to-buffer (get-buffer-create "foo.txt"))
      (text-mode)
      (linum-mode t)
      (visual-line-mode t)
      (dotimes (i (* (window-width) 4))
        (insert ";"))
     (dotimes (i 10)
        (insert "\n"))
      (goto-char (point-min))
      (vertical-motion 1)
      (message "Second Row + 2 points:  %s" (current-column))
      (vertical-motion 1)
      (message "Third Row + 4 points:  %s" (current-column)))

Additional incorrect behavior can be seen by moving the cursor to the beginning of the buffer after running the above test function and evaluating `(vertical-motion 1`).  Instead of moving to line number two, the cursor will move to column number 2.  I have written a workaround to compensate for the problem -- if point is on the first wrapped line, then ...

    (vertical-motion 1)
    (when (= (current-column) 2 )
      (move-to-column (+ (current-column) (- window-width 2))))

This bug affects the popular functions that rely upon the behavior of vertical-motion -- e.g., scrolling with the mouse; and using the arrow keys to navigate the buffer.

If the wrapped line is at the top of the buffer, and there are many such wrapped lines that don't break nicely at the far right edge (e.g., no space and no tab), then the mouse scroll becomes inoperable unless one uses the scroll bars.

Thanks,

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

In GNU Emacs 24.4.50.1 (x86_64-apple-darwin10.8.0, NS apple-appkit-1038.36)
 of 2014-04-28 on MP.local
Repository revision: 117030 eggert@cs.ucla.edu-20140428165941-zk00wxiq6dy6xcg4
Windowing system distributor `Apple', version 10.3.1038
Configured using:
 `configure --with-ns'

Configured features:
ACL LIBXML2 ZLIB

Important settings:
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  lawlist-scroll-bar-mode: t
  lawlist-delete-selection-mode: t
  desktop-save-mode: t
  tabbar-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  transient-mark-mode: t

Recent input:
<escape> x e m a c s - r e p o r t - b u g <return> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> r e p o r t - b u g - e m a c s <return> 
<backspace> <return> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> e m a c s - b u g <return> 
<backspace> <backspace> <backspace> r e p o r t <return> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> r e p o r t - <return> b u 
g <return>

Recent messages:
Loading /Users/HOME/.0.data/.0.emacs/init.el (source)...
Loading /Users/HOME/.0.data/.0.emacs/.multiple_cursors...done
Loading /Users/HOME/.0.data/.0.emacs/init.el (source)...done
For information about GNU Emacs and the GNU system, type C-h C-a.
Wrote /Users/HOME/.0.data/.0.emacs/.lock
Turned ON highlighting current column when Emacs is idle.
*beep* [5 times]

Load-path shadows:
/Users/HOME/.0.data/.0.emacs/.0.flim/md4 hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/md4
/Users/HOME/.0.data/.0.emacs/.0.flim/hex-util hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/hex-util
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/net/sasl
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl-ntlm hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/net/sasl-ntlm
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl-digest hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/net/sasl-digest
/Users/HOME/.0.data/.0.emacs/.0.flim/sasl-cram hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/net/sasl-cram
/Users/HOME/.0.data/.0.emacs/.0.flim/ntlm hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/net/ntlm
/Users/HOME/.0.data/.0.emacs/.0.flim/hmac-md5 hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/net/hmac-md5
/Users/HOME/.0.data/.0.emacs/.0.flim/hmac-def hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/net/hmac-def
/Users/HOME/.0.data/.0.emacs/.0.wl/rfc2368 hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/mail/rfc2368
/Users/HOME/.0.data/.0.emacs/.0.wl/utf7 hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/gnus/utf7
/Users/HOME/.0.data/.0.emacs/.0.simi/smime hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/gnus/smime
/Users/HOME/.emacs.d/elpa/tabulated-list-20120406.1351/tabulated-list hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/emacs-lisp/tabulated-list
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/obsolete/pgg
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-pgp5 hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/obsolete/pgg-pgp5
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-pgp hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/obsolete/pgg-pgp
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-parse hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/obsolete/pgg-parse
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-gpg hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/obsolete/pgg-gpg
/Users/HOME/.0.data/.0.emacs/.0.simi/pgg-def hides /Users/HOME/.0.data/.0.emacs/Emacs_04_28_2014.app/Contents/Resources/lisp/obsolete/pgg-def

Features:
(shadow disp-table wl-mime mime-edit pgg-parse pccl pccl-20 signature
mime-setup mail-mime-setup semi-setup mime-pgp pgg-def mime-play
filename mime-image modb-legacy modb-standard elmo-imap4 emacsbug
bbdb-autoloads lawlist-wl wl-demo wl-draft eword-encode wl-template
sendmail elmo-net elmo-cache elmo-map elmo-dop wl-news wl-address
wl-thread wl-folder wl wl-e21 wl-spam wl-action wl-summary wl-refile
wl-util pp elmo-flag elmo-localdir wl-message elmo-mime mmelmo-buffer
mmelmo-imap mime-view mime-conf calist semi-def mmimap mime-parse
mmbuffer mmgeneric elmo-multi elmo-spam elsp-header elsp-generic elmo
elmo-signal wl-highlight wl-vars wl-version elmo-msgdb modb
modb-generic modb-entity luna mime elmo-util emu invisible inv-23 poem
poem-e20 poem-e20_3 eword-decode std11 elmo-date elmo-vars
elmo-version w3m-load mime-w3m w3m browse-url doc-view jka-compr
image-mode w3m-hist w3m-fb bookmark-w3m w3m-ems w3m-ccl ccl
w3m-favicon w3m-image w3m-proc w3m-util savehist lawlist-vcol
lawlist-whitespace lawlist-github conf-mode log-edit add-log find-lisp
package esh-var esh-io esh-cmd esh-opt esh-ext esh-proc esh-arg eldoc
esh-groups eshell esh-module esh-mode esh-util dired-x view tramp
tramp-compat tramp-loaddefs trampver server grep epa epg epg-config
diff-mode autorevert filenotify log-view pcvs-util ido time-stamp
vc-git vc vc-dispatcher ediff-merg ediff-wind ediff-diff ediff-mult
ediff-help ediff-init ediff-util ediff rx ert ewoc debug eieio-base
lawlist-calculator ps-print ps-def lpr flyspell ispell bbdb timezone
find-func dired-aux yasnippet help-mode multiple-cursors
mc-separate-operations rectangular-region-mode mc-mark-more thingatpt
mc-cycle-cursors mc-edit-lines multiple-cursors-core rect saveplace
lawlist-tex-mode pcase compile shell pcomplete comint ansi-color ring
skeleton compare-w desktop frameset lawlist-tabbar lawlist-org
lawlist-calendar edmacro kmacro derived lawlist-toodledo advice
url-http url-auth url-gw url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util url-parse
auth-source eieio byte-opt bytecomp byte-compile cconv eieio-core
password-cache url-vars mailcap json xml noutline outline easy-mmode
gnus-sum gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail
mail-source tls utf7 mel path-util mime-def alist mcharset mcs-20
mcs-e20 pcustom pces pces-e20 pces-20 broken poe pym static apel-ver
product netrc nnoo parse-time gnus-spec gnus-int gnus-range message
cl-macs dired format-spec rfc822 mml easymenu mml-sec mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
mailabbrev gmm-utils mailheader gnus-win gnus gnus-ems nnheader
gnus-util mail-utils mm-util help-fns mail-prsvr wid-edit cl gv
cl-loaddefs cl-lib time-date tooltip electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel ns-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
cocoa ns multi-tty emacs)

Memory information:
((conses 16 718578 45882)
 (symbols 48 54623 0)
 (miscs 40 83 241)
 (strings 32 112318 16394)
 (string-bytes 1 3740329)
 (vectors 16 40884)
 (vector-slots 8 721743 31900)
 (floats 8 957 235)
 (intervals 56 470 88)
 (buffers 960 13))

bug#17585: 24.4.50; vertical-motion erroneously adds points

[Eli Zaretskii]

> Date: Sun, 25 May 2014 00:23:20 -0700
> From: Keith David Bershatsky <esq@lawlist.com>
> 
> The following test function demonstrates how vertical-motion erroneously adds points when dealing with long lines without spaces that go beyond the window-width:
> 
>     (defun test ()
>     (interactive)
>       (switch-to-buffer (get-buffer-create "foo.txt"))
>       (text-mode)
>       (linum-mode t)
>       (visual-line-mode t)
>       (dotimes (i (* (window-width) 4))
>         (insert ";"))
>      (dotimes (i 10)
>         (insert "\n"))
>       (goto-char (point-min))
>       (vertical-motion 1)
>       (message "Second Row + 2 points:  %s" (current-column))
>       (vertical-motion 1)
>       (message "Third Row + 4 points:  %s" (current-column)))

Fixed in revision 117154 on the emacs-24 branch.  The patch is below,
in case you want to try it without waiting for it to be merged to the
trunk.

Thanks.

--- src/xdisp.c	2014-05-24 21:28:43 +0000
+++ src/xdisp.c	2014-05-25 14:18:46 +0000
@@ -8556,7 +8556,7 @@ move_it_in_display_line_to (struct it *i
 	}
       else
 	{
-	  if (it->line_wrap == WORD_WRAP)
+	  if (it->line_wrap == WORD_WRAP && it->area == TEXT_AREA)
 	    {
 	      if (IT_DISPLAYING_WHITESPACE (it))
 		may_wrap = 1;

bug#17585: 24.4.50; vertical-motion erroneously adds points

[Keith David Bershatsky]

The `test` function yields the same result with Emacs built (--with-ns) from the Trunk as of May 29, 2014 at 11:18 P.S.T.  Approximately when, please, should I expect to see the fix merged into the Trunk?

I'm using macports / baazar:

/macports/bin/bzr branch --stacked bzr://bzr.savannah.gnu.org/emacs/trunk emacs-trunk

Thanks,

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

At Sun, 25 May 2014 17:23:31 +0300,
Eli Zaretskii wrote:
> 
> > Date: Sun, 25 May 2014 00:23:20 -0700
> > From: Keith David Bershatsky <esq@lawlist.com>
> > 
> > The following test function demonstrates how vertical-motion erroneously adds points when dealing with long lines without spaces that go beyond the window-width:
> > 
> >     (defun test ()
> >     (interactive)
> >       (switch-to-buffer (get-buffer-create "foo.txt"))
> >       (text-mode)
> >       (linum-mode t)
> >       (visual-line-mode t)
> >       (dotimes (i (* (window-width) 4))
> >         (insert ";"))
> >      (dotimes (i 10)
> >         (insert "\n"))
> >       (goto-char (point-min))
> >       (vertical-motion 1)
> >       (message "Second Row + 2 points:  %s" (current-column))
> >       (vertical-motion 1)
> >       (message "Third Row + 4 points:  %s" (current-column)))
> 
> Fixed in revision 117154 on the emacs-24 branch.  The patch is below,
> in case you want to try it without waiting for it to be merged to the
> trunk.
> 
> Thanks.
> 
> --- src/xdisp.c	2014-05-24 21:28:43 +0000
> +++ src/xdisp.c	2014-05-25 14:18:46 +0000
> @@ -8556,7 +8556,7 @@ move_it_in_display_line_to (struct it *i
>  	}
>        else
>  	{
> -	  if (it->line_wrap == WORD_WRAP)
> +	  if (it->line_wrap == WORD_WRAP && it->area == TEXT_AREA)
>  	    {
>  	      if (IT_DISPLAYING_WHITESPACE (it))
>  		may_wrap = 1;

bug#17585: 24.4.50; vertical-motion erroneously adds points

[Eli Zaretskii]

> Date:  Thu, 29 May 2014 23:21:05 -0700
> From:  Keith David Bershatsky <esq@lawlist.com>
> Cc:  17585-done@debbugs.gnu.org
> 
> The `test` function yields the same result with Emacs built (--with-ns) from the Trunk as of May 29, 2014 at 11:18 P.S.T.  Approximately when, please, should I expect to see the fix merged into the Trunk?

It was merged to the trunk on May 26, as trunk revision 117154 (yes,
the same revision number as on the emacs-24 branch; it happens).

> I'm using macports / baazar:
> 
> /macports/bin/bzr branch --stacked bzr://bzr.savannah.gnu.org/emacs/trunk emacs-trunk

Do you have revision 117154 in your bzr trunk branch?  If you do, you
should see this entry in src/ChangeLog:

  2014-05-26  Eli Zaretskii  <eliz@gnu.org>

	  * xdisp.c (move_it_in_display_line_to): Don't record wrap position
	  if we are iterating over an object that generates glyphs for
	  marginal areas.  (Bug#17585)

With the latest trunk, vertical motion in the foo.txt buffer created
by your recipe works OK interactively (i.e. by using arrow keys or
M-:); it didn't work correctly before.  Please try that after your
'test' function finishes.

Your test case, which uses vertical-motion non-interactively, indeed
still moves point horizontally as well, but that's because linum-mode
does its line number display in a post-command hook, so it defers the
initial display of the line numbers until _after_ vertical-motion did
its thing.  You can see that by inserting (sit-for 1) before each call
to vertical-motion.  So I think this is an unrelated problem; if it
gives you trouble in some real-life use case, please describe that use
case.

bug#17585: 24.4.50; vertical-motion erroneously adds points

[Stefan Monnier]

> the Trunk as of May 29, 2014 at 11:18 P.S.T.  Approximately when, please,
> should I expect to see the fix merged into the Trunk?

Please use the `emacs-24' branch instead, which is the one we're
focusing on right now (and which does have this fix applied).


        Stefan

bug#17585: 24.4.50; vertical-motion erroneously adds points

[Keith David Bershatsky]

Eli & Stefan:

My confusion regarding whether the fix for Bug#17585 had been implemented in the version of Emacs Trunk that I've been testing was due to the reason mentioned by Eli below.  Indeed, the arrow keys and scrolling with the mouse now work correctly when `visual-line-mode` and `linum-mode` are both active and the line of text (without any spaces) exceeds the `window-width`.

I've been using the following command line prior to building a new Emacs Trunk version, and I believe it gives me the most recent Emacs-24 branch:

bzr branch --stacked bzr://bzr.savannah.gnu.org/emacs/trunk emacs-trunk

`M-x emacs version` yields the following message:

GNU Emacs 24.4.50.1 (x86_64-apple-darwin10.8.0, NS apple-appkit-1038.36) of 2014-05-29 on MP.local

I am working on writing a custom library that draws a thin vertical line (i.e., "|") and a thin horizontal line (i.e., `underline`), which is compatible with `visual-line-mode` and `linum-mode`.  The visual effect is similar to the `crosshairs.el` library, except that mine is done with thin lines only -- i.e., no background highlighting.  One of my key functions uses (vertical-motion 1) for every line in the visible window.  To deal with the situation best described by Eli (below), I use the following immediately after each execution of (vertical-motion 1):

    (when (= (current-column) 2 )
        (move-to-column (+ (current-column) (- window-width 2))))

I have not yet done any experimentation with removing the line number overlays for purposes of calculating and placing my cross-hair overlays.  My best guess is that adding the line numbers after the cross-hairs have been placed would skew/break the vertical line.

Thanks again for implementing the bug fix.  It certainly nice to be able to effectively use the arrow keys and the mouse scroll when dealing with the situation described in the first paragraph above.

Keith

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

At Fri, 30 May 2014 12:21:58 -0400,
Stefan Monnier wrote:
> 
> > the Trunk as of May 29, 2014 at 11:18 P.S.T.  Approximately when, please,
> > should I expect to see the fix merged into the Trunk?
> 
> Please use the `emacs-24' branch instead, which is the one we're
> focusing on right now (and which does have this fix applied).
> 
> 
>         Stefan


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

> Date:  Thu, 29 May 2014 23:21:05 -0700
> From:  Keith David Bershatsky <esq@lawlist.com>
> Cc:  17585-done@debbugs.gnu.org
> 
> The `test` function yields the same result with Emacs built (--with-ns) from the Trunk as of May 29, 2014 at 11:18 P.S.T.  Approximately when, please, should I expect to see the fix merged into the Trunk?

It was merged to the trunk on May 26, as trunk revision 117154 (yes,
the same revision number as on the emacs-24 branch; it happens).

> I'm using macports / baazar:
> 
> /macports/bin/bzr branch --stacked bzr://bzr.savannah.gnu.org/emacs/trunk emacs-trunk

Do you have revision 117154 in your bzr trunk branch?  If you do, you
should see this entry in src/ChangeLog:

  2014-05-26  Eli Zaretskii  <eliz@gnu.org>

	  * xdisp.c (move_it_in_display_line_to): Don't record wrap position
	  if we are iterating over an object that generates glyphs for
	  marginal areas.  (Bug#17585)

With the latest trunk, vertical motion in the foo.txt buffer created
by your recipe works OK interactively (i.e. by using arrow keys or
M-:); it didn't work correctly before.  Please try that after your
'test' function finishes.

Your test case, which uses vertical-motion non-interactively, indeed
still moves point horizontally as well, but that's because linum-mode
does its line number display in a post-command hook, so it defers the
initial display of the line numbers until _after_ vertical-motion did
its thing.  You can see that by inserting (sit-for 1) before each call
to vertical-motion.  So I think this is an unrelated problem; if it
gives you trouble in some real-life use case, please describe that use
case.

bug#17585: 24.4.50; vertical-motion erroneously adds points

[Eli Zaretskii]

> Date:  Sat, 31 May 2014 12:00:59 -0700
> From:  Keith David Bershatsky <esq@lawlist.com>
> Cc:  17585@debbugs.gnu.org
> 
> I am working on writing a custom library that draws a thin vertical line (i.e., "|") and a thin horizontal line (i.e., `underline`), which is compatible with `visual-line-mode` and `linum-mode`.  The visual effect is similar to the `crosshairs.el` library, except that mine is done with thin lines only -- i.e., no background highlighting.  One of my key functions uses (vertical-motion 1) for every line in the visible window.  To deal with the situation best described by Eli (below), I use the following immediately after each execution of (vertical-motion 1):
> 
>     (when (= (current-column) 2 )
>         (move-to-column (+ (current-column) (- window-width 2))))

This shouldn't be necessary.  Try invoking linum-update (or whatever
function is put by linum-mode on the post-command hook) before/when
you display the buffer for the first time, and I think the problem
should go away.  It is caused by the fact that the line numbers are
not displayed until the first command that displays the buffer
finishes, and therefore any functions in that first command will not
be aware that the window has non-zero left margin.