bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Joe Corneli]

My eyesight is very bad without my glasses, and I was experimenting with
a giant font on my laptop that I could comfortably read without them.

To replicate my set-up: shift+click and select "Increase Buffer Text
Size" several times, to get letters that are about an inch tall!  The
buffer is in fundamental mode.  Emacs is compiled with Xft support.

I was able to replicate the following weird behavior by following the
above recipe with emacs -q --no-site-file, using the font that is the
default there.

Here's a sample text generated by simply typing with a giant-sized font:

SAMPLE:

Maybe it's when the lines are particularly long, like if I keep writing
after a ?seog rosruc eht erehw fo yltnednepedni ,elihw

(DECRYPTED):

Maybe it's when the lines are particularly long, like if I keep writing
after a while, independently of where the cursor goes?

It seems that quite reliably, with this giant font, the lines will flip
to RTL after writing for a while.  If I write in the same buffer in a
window that doesn't have the giant font set, it doesn't trigger RTL.
Here's a text I wrote, starting in window in which the buffer is
normal-sized, and then switching, back to the giant sized buffer after a
while:

SAMPLE:

Is it true even if the font size is a bit smaller, I wonder, or does it
only happen with my glasses off?  Is there a particular point when it
hits, or does it only happen if I get going with some particular word in
the way?  ...ezis tnof yb dereggirt eb ot yletinifed smees tI

Finally, here's a text written in the giant-sized buffer that can be
used to estimate where it switches:

SAMPLE:

a b c d e f g h i j k l m n o p q r s t u v w x y z a b c d e f g h i j k l m n u t s r q p o

... hm, it seems to flip at EXACTLY AT 80 CHARACTERS!  (That matches the
first sample text above, too.)




In GNU Emacs 24.3.50.2 (x86_64-apple-darwin13.1.0, X toolkit, Xaw scroll bars)
 of 2014-04-19 on Joes-MacBook-Pro.local
Repository revision: 116800 lekktu@gmail.com-20140319022451-z8fp3icgftf8zjg6
Windowing system distributor `The X.Org Foundation', version 11.0.11406000
Configured using:
 `configure --with-x-toolkit=lucid CPPFLAGS=-I/opt/local/include
 LDFLAGS=-L/opt/local/lib'

Important settings:
  locale-coding-system: nil

Major mode: Fundamental

Minor modes in effect:
  show-paren-mode: t
  TeX-PDF-mode: t
  shell-dirtrack-mode: t
  global-hl-line-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: 1

Recent input:
i n g SPC q u i t e SPC l i k e SPC i t SPC b e f o 
r e , SPC c e r t a i n l y . SPC SPC A n d SPC i t 
SPC d o e s n ' t SPC h a p p e n SPC e v e r y SPC 
t i m e C-e C-e C-e C-e <down> M-> <return> <return> 
M a y b e SPC i t ' s SPC w h e n SPC t h e SPC l i 
n e s SPC a r e SPC p a r t i c u l a r l y SPC l o 
n g , SPC l i k e SPC i f SPC t h e SPC <backspace> 
<backspace> <backspace> <backspace> <backspace> SPC 
I S-SPC k e e p SPC w r i t i n g SPC a f t e r SPC 
a SPC w h i l e , SPC i n d e p e n d e n t l y SPC 
o f SPC w h e r e SPC t h e SPC c u r s o r SPC g o 
e s ? C-e <down> <down> <return> <return> <up> <up> 
C-e <down> C-SPC <up> <up> <up> <up> <up> <up> <up> 
<up> M-w M-x C-g C-x C-w <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> b i 
g <return> M-x s w <tab> - <tab> b u <tab> <tab> - 
<tab> f <tab> <return> s m a l l <return> C-y <up> 
<up> <down> <down> <down> <up> <up> C-SPC M-> M-w M-x 
r e p <tab> o <tab> r <tab> <return>

Recent messages:
Saved text until "osruc eht erehw fo yltnednepedni ,elihw
"
Quit
Saving file /Users/joe/big...
Wrote /Users/joe/big
Making completion list...
Mark set
End of buffer
Mark activated
Making completion list... [2 times]

Load-path shadows:
/Users/joe/.emacs.d/elpa/mediawiki-2.2.3/mediawiki hides ~/thesis.git/config/elisp/mediawiki
~/diaspora.el/libraries/markdown-mode hides ~/thesis.git/config/elisp/markdown-mode
~/thesis.git/config/elisp/tex-mode hides /usr/local/share/emacs/24.3.50/lisp/textmodes/tex-mode
~/elisp/emms/lisp/tq hides /usr/local/share/emacs/24.3.50/lisp/emacs-lisp/tq

Features:
(shadow emacsbug align rect novice etags tutorial grep scheme js imenu
cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine
cc-vars cc-defs plain-tex tex-buf python mmm-cmds eieio-opt texmathp pp
url-queue bibtex shr-color timezone parse-time dabbrev ispell qp
network-stream starttls mailalias mail-extr sort face-remap sh-script
smie executable pcmpl-unix vc-git font-latex latexenc misearch
multi-isearch dired-aux help-mode paren cus-start cus-load w3m-load
w3m-proc w3m-util mule-util dired-x info-look info tex-mode latex
tex-style tex crm compile compare-w org org-macro org-footnote
org-pcomplete org-list org-faces org-entities org-version ob-emacs-lisp
ob ob-tangle org-src ob-ref ob-lob ob-table ob-keys ob-exp ob-comint
ob-core ob-eval org-compat org-macs org-loaddefs cal-menu calendar
cal-loaddefs tramp tramp-compat tramp-loaddefs trampver shell pcomplete
eww google-translate ert find-func ewoc debug diaspora diaspora-bookmark
diaspora-stream-mode diaspora-misc diaspora-main diaspora-messages htmlr
sgml-mode diaspora-contacts diaspora-aspects diaspora-notifications
diaspora-stream diaspora-comments diaspora-post diaspora-http-errors
diaspora-post-edit-mode skeleton diaspora-new diaspora-urls
diaspora-mode markdown-translator json mmm-mode mmm-class mmm-region
mmm-utils mmm-univ mmm-auto mmm-vars mmm-compat gnutls shr mu4e
mu4e-speedbar speedbar sb-image ezimage dframe mu4e-main mu4e-view epa
epg epg-config browse-url comint ansi-color mu4e-headers mu4e-compose
mu4e-draft mu4e-actions ido rfc2368 mu4e-mark mu4e-message html2text
mu4e-proc mu4e-utils doc-view jka-compr image-mode mu4e-lists mu4e-about
mu4e-vars message rfc822 mailabbrev gmm-utils mailheader mu4e-meta
smtpmail-multi smtpmail sendmail load-theme-buffer-local
highlight-indentation poly-markdown markdown-mode derived thingatpt
edmacro kmacro polymode pcase poly-base polymode-weave polymode-export
polymode-methods polymode-classes polymode-common cl-macs gv format-spec
eieio-custom eieio-base color jabber-autoloads package apropos emms-mark
emms-history emms-cache emms-info-ogginfo emms-info-mp3info emms-info
later-do emms-playlist-mode emms-player-vlc advice emms-player-mplayer
emms-player-simple emms-source-playlist emms-source-file dired
emms-setup emms emms-compat emstar cl hl-line mediawiki url-cache ring
mm-url gnus gnus-ems nnheader mail-utils wid-edit cl-loaddefs cl-lib mml
easymenu mml-sec mm-decode mm-bodies mm-encode url-http tls url
url-proxy url-privacy url-expand url-methods url-history mailcap
url-auth mail-parse rfc2231 rfc2047 rfc2045 ietf-drums url-cookie
url-domsuf url-util url-parse auth-source eieio byte-opt bytecomp
byte-compile cconv eieio-core gnus-util mm-util help-fns mail-prsvr
password-cache url-gw url-vars outline-magic noutline outline easy-mmode
tex-site auto-loads time-date tooltip electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
gfilenotify dynamic-setting system-font-setting font-render-setting
x-toolkit x multi-tty emacs)

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Eli Zaretskii]

> From: Joe Corneli <holtzermann17@gmail.com>
> Date: Wed, 02 Jul 2014 23:30:33 +0100
> 
> My eyesight is very bad without my glasses, and I was experimenting with
> a giant font on my laptop that I could comfortably read without them.
> 
> To replicate my set-up: shift+click and select "Increase Buffer Text
> Size" several times, to get letters that are about an inch tall!  The
> buffer is in fundamental mode.  Emacs is compiled with Xft support.
> 
> I was able to replicate the following weird behavior by following the
> above recipe with emacs -q --no-site-file, using the font that is the
> default there.
> 
> Here's a sample text generated by simply typing with a giant-sized font:
> 
> SAMPLE:
> 
> Maybe it's when the lines are particularly long, like if I keep writing
> after a ?seog rosruc eht erehw fo yltnednepedni ,elihw
> 
> (DECRYPTED):
> 
> Maybe it's when the lines are particularly long, like if I keep writing
> after a while, independently of where the cursor goes?
> 
> It seems that quite reliably, with this giant font, the lines will flip
> to RTL after writing for a while.  If I write in the same buffer in a
> window that doesn't have the giant font set, it doesn't trigger RTL.
> Here's a text I wrote, starting in window in which the buffer is
> normal-sized, and then switching, back to the giant sized buffer after a
> while:
> 
> SAMPLE:
> 
> Is it true even if the font size is a bit smaller, I wonder, or does it
> only happen with my glasses off?  Is there a particular point when it
> hits, or does it only happen if I get going with some particular word in
> the way?  ...ezis tnof yb dereggirt eb ot yletinifed smees tI
> 
> Finally, here's a text written in the giant-sized buffer that can be
> used to estimate where it switches:
> 
> SAMPLE:
> 
> a b c d e f g h i j k l m n o p q r s t u v w x y z a b c d e f g h i j k l m n u t s r q p o
> 
> ... hm, it seems to flip at EXACTLY AT 80 CHARACTERS!  (That matches the
> first sample text above, too.)

Could you please provide a complete self-contained recipe for
reproducing this?  Something like this:

  - start with "emacs -Q"
  - enlarge the font by shift-mouse and selecting "Increase Buffer
    Text Size" N times (please tell what is the value of N)
  - type the following text: ....

etc.

Given your description, I have hard time understanding what to do to
reproduce this.

Thanks.

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Joe Corneli]

On Thu, Jul 3, 2014 at 3:51 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Joe Corneli <holtzermann17@gmail.com>
>> I was able to replicate the following weird behavior by following the
>> above recipe with emacs -q --no-site-file, using the font that is the
>> default there.

>   - start with "emacs -Q"
>   - enlarge the font by shift-mouse and selecting "Increase Buffer
>     Text Size" N times (please tell what is the value of N)
>   - type the following text: ....

... where N=12 was a value that triggered the effect for me.  Also
note that the Emacs window was full-screen.

A sample text would be: type the alphabet about 2 and a half times,
space separated, and if it works the same way for you as it did for
me, reversal will start after the 3rd "h":

a b c d e f g h i j k l m n o p q r s t u v w x y z a b c d e f g h i
j k l m n o p q r s t u v w x y z a b c d e f g h q p o n m l k j i

However, as a further update to what I wrote before, the exact point
where the reversal is triggered seems to depend on the size of the
window.  With the same setup as above, except, with a 50% smaller
window, I only had to type:

a b c d e f g h i j k l m n o p q r s t u v w x y z a b f e d c

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Stefan Monnier]

>> - start with "emacs -Q"
>> - enlarge the font by shift-mouse and selecting "Increase Buffer
>> Text Size" N times (please tell what is the value of N)
>> - type the following text: ....

> ... where N=12 was a value that triggered the effect for me.  Also
> note that the Emacs window was full-screen.

Ah, yes, I can reproduce it: it's not RTL-rendering.
It's some issue with cursor positioning, such that after some
line-wrapping, when you enter a char, somehow point ends up before the
new char rather than after, so you end up typing "backwards".


        Stefan

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>, 17905@debbugs.gnu.org
> Date: Thu, 03 Jul 2014 09:51:29 -0400
> 
> >> - start with "emacs -Q"
> >> - enlarge the font by shift-mouse and selecting "Increase Buffer
> >> Text Size" N times (please tell what is the value of N)
> >> - type the following text: ....
> 
> > ... where N=12 was a value that triggered the effect for me.  Also
> > note that the Emacs window was full-screen.
> 
> Ah, yes, I can reproduce it:

I can't.

> it's not RTL-rendering.
> It's some issue with cursor positioning, such that after some
> line-wrapping, when you enter a char, somehow point ends up before the
> new char rather than after, so you end up typing "backwards".

Are you saying that completely redisplaying the window (e.g., with
"C-x 0" followed by "C-x b THAT-BUFFER RET") makes the display correct
again?  I guess not, in which case this is not about _cursor_
positioning, but rather about _point_ positioning.  Right?

Anyway, since Joe complicated the recipe with going full-screen (which
makes everything dependent on the monitor dimensions), I would ask you
two to answer the following questions:

  - Can the issue be reproduced without enlarging the font?

  - Does the issue happen only in continuation lines?  If so, does it
    happen as soon as the line is continued, or do you have to have
    several continuation lines?

  - Stefan, did you see it on the emacs-24 branch or on the trunk (or
    both)?

  - Is it important what text to type, or will any text do?

(I cannot reproduce this no matter what I do.  I'm tempted to say it's
X-specific, but I fail to see how could it be.)

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Stefan Monnier]

> again?  I guess not, in which case this is not about _cursor_
> positioning, but rather about _point_ positioning.  Right?

Exactly.

> Anyway, since Joe complicated the recipe with going full-screen (which
> makes everything dependent on the monitor dimensions), I would ask you
> two to answer the following questions:

> - Can the issue be reproduced without enlarging the font?

I think enlarging the font is necessary because I think an important
element is that there be a partially displayed line.

> - Does the issue happen only in continuation lines?

Yes.

>   If so, does it happen as soon as the line is continued, or do you
>   have to have several continuation lines?

I think the issue shows up when:
- the current logical line starts before window-start.
- point is at the bottom of the window.
- More specifically, point is on the last line and is only
  partially displayed.

I can reproduce it as follows:
- emacs -Q
- C-x C-+
- hit "g" (other chars work as well) and keep it pressed until the
  screen is completely filled with "g"s (window-start should be
  continuation of the current line (there's a little curly arrow in the
  left fringe of every displayed line, including the very first one)),
  at which point the "g"'s keep being inserted but all the same position
  with the cursor (and point) being at the left of the bottom
  (partially-displayed) line.

With just a single C-x C-+, it takes a while to fill the window, so you
can speed it up by using many more C-x C-+ or by making the
window smaller (but do make sure that the window should not be
a whole multiple of lines, so that the last line is partially hidden by
the mode-line).


        Stefan

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: holtzermann17@gmail.com, 17905@debbugs.gnu.org
> Date: Thu, 03 Jul 2014 12:17:18 -0400
> 
> I think the issue shows up when:
> - the current logical line starts before window-start.
> - point is at the bottom of the window.
> - More specifically, point is on the last line and is only
>   partially displayed.

Thanks.

If it only shows up when point is in a partially visible line, then
the bug is that Emacs lets point be displayed in such a line.  It is
supposed to scroll the display to have the line with point be fully
visible.

And indeed, I have a very hard time forcing Emacs to let me have point
in a partially visible line.

But Joe didn't seem to talk about partially visible lines.  Joe, is
that right?

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Eli Zaretskii]

> Date: Thu, 03 Jul 2014 19:57:30 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: holtzermann17@gmail.com, 17905@debbugs.gnu.org
> 
> If it only shows up when point is in a partially visible line, then
> the bug is that Emacs lets point be displayed in such a line.  It is
> supposed to scroll the display to have the line with point be fully
> visible.
> 
> And indeed, I have a very hard time forcing Emacs to let me have point
> in a partially visible line.

If I'm right in what I think is the root cause for this, it goes back
at least to Emacs 22.

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Joe Corneli]

On Thu, Jul 3, 2014 at 5:57 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> But Joe didn't seem to talk about partially visible lines.  Joe, is
> that right?

I think the diagnosis from Stefan is totally correct, and it's a good
observation.  It simply hadn't occurred to me when I posted, that's
all.  Using _ (underscore) as a test character is useful for
confirming the theory, since, on an affected window, the final
underscores don't appear -- they are off-screen!

It does seem relevant to mention at this point that I run Emacs under
Ratpoison, so all windows are typically adjusted to run full screen.
Windows that aren't forced by the WM to be a fixed height might be
less likely to display partially visible lines?

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Stefan Monnier]

> If it only shows up when point is in a partially visible line, then
> the bug is that Emacs lets point be displayed in such a line.

Probably (tho I don't know why this causes point to be changed).

> It is supposed to scroll the display to have the line with point be
> fully visible.

I know, but I confirm that point (and the corresponding cursor) is at
the leftmost position of the partially displayed screen line in that case.


        Stefan

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: holtzermann17@gmail.com,  17905@debbugs.gnu.org
> Date: Thu, 03 Jul 2014 15:49:29 -0400
> 
> > If it only shows up when point is in a partially visible line, then
> > the bug is that Emacs lets point be displayed in such a line.
> 
> Probably (tho I don't know why this causes point to be changed).

The display engine is probably confused by what happened, so it does
something utterly inappropriate.  There are situations where redisplay
moves point, but this is not one of them.

> I confirm that point (and the corresponding cursor) is at the
> leftmost position of the partially displayed screen line in that
> case.

That's the immediate cause of the bug, I'm quite sure.  And it goes a
long way back.

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Eli Zaretskii]

> Date: Fri, 04 Jul 2014 09:39:03 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: holtzermann17@gmail.com, 17905@debbugs.gnu.org
> 
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: holtzermann17@gmail.com,  17905@debbugs.gnu.org
> > Date: Thu, 03 Jul 2014 15:49:29 -0400
> > 
> > > If it only shows up when point is in a partially visible line, then
> > > the bug is that Emacs lets point be displayed in such a line.
> > 
> > Probably (tho I don't know why this causes point to be changed).
> 
> The display engine is probably confused by what happened, so it does
> something utterly inappropriate.  There are situations where redisplay
> moves point, but this is not one of them.
> 
> > I confirm that point (and the corresponding cursor) is at the
> > leftmost position of the partially displayed screen line in that
> > case.
> 
> That's the immediate cause of the bug, I'm quite sure.  And it goes a
> long way back.

I think I fixed this in revision 117347 on the emacs-24 branch.

In general, I recommend to stay away of resizing the font of the
default face.  The display engine does a lot of pixel calculations
using the size of the frame's default font; when that is different
from the font of the default face, all kinds of features begin to
subtly break.  E.g., set scroll-margin to a non-zero value, then type
"C-x C-+" enough time to enlarge the font significantly, and you will
see that Emacs lets point enter the scroll margin (because it
internally computes the scroll margin in pixels using the frame's
default font).

If you want to use a larger font by default, it is much better to
start Emacs with the corresponding -fn switch, or customize the
frame's font in your ~/.emacs or X resources.  That will cause Emacs
to use that font as the frame's default font, and all these issues
won't pop up.

This bug was caused by something similar to the scroll-margin problem
when the default face uses a resized font.  It caused the code which
handles redisplay of window malfunction when point ended up being
displayed in a partially visible screen line at the end of the window.

There was a similar problem in scroll commands, which caused
inappropriate moving of point in the last line of the window; I fixed
that as well.

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Stefan Monnier]

> subtly break.  E.g., set scroll-margin to a non-zero value, then type
> "C-x C-+" enough time to enlarge the font significantly, and you will
> see that Emacs lets point enter the scroll margin (because it
> internally computes the scroll margin in pixels using the frame's
> default font).

I think this is OK: the scroll-margin's unit is "lines" but it is really
just a proxy for pixels.  Same as for window-margins.

> There was a similar problem in scroll commands, which caused
> inappropriate moving of point in the last line of the window; I fixed
> that as well.

Thanks Eli,


        Stefan

bug#17905: 24.3.50; writing with a giant font triggers RTL text entry

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: holtzermann17@gmail.com,  17905-done@debbugs.gnu.org
> Date: Fri, 04 Jul 2014 10:15:25 -0400
> 
> > subtly break.  E.g., set scroll-margin to a non-zero value, then type
> > "C-x C-+" enough time to enlarge the font significantly, and you will
> > see that Emacs lets point enter the scroll margin (because it
> > internally computes the scroll margin in pixels using the frame's
> > default font).
> 
> I think this is OK: the scroll-margin's unit is "lines" but it is really
> just a proxy for pixels.  Same as for window-margins.

Well, I'm sure someone will be surprised...