right-char and left-char

right-char and left-char

[Chong Yidong]

In the Emacs manual, the documentation for right-char says:

`<right>'
     Move one character to the right (`right-char').  This moves one
     character forward in text that is read in the usual left-to-right
     direction, but one character _backward_ if the text is read
     right-to-left, as needed for right-to-left scripts such as Arabic.

But in fact, the directionality of right-char and left-char depends on
current-bidi-paragraph-direction.  If you have RTL text embedded in LTR
paragraphs, like in the HELLO file, <right> moves left while in the RTL
segment.  This contradicts the manual description, which merely states
"if the text is read right-to-left".

Which is the intended behavior?

Re: right-char and left-char

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Date: Thu, 06 Oct 2011 12:45:16 -0400
> 
> In the Emacs manual, the documentation for right-char says:
> 
> `<right>'
>      Move one character to the right (`right-char').  This moves one
>      character forward in text that is read in the usual left-to-right
>      direction, but one character _backward_ if the text is read
>      right-to-left, as needed for right-to-left scripts such as Arabic.
> 
> But in fact, the directionality of right-char and left-char depends on
> current-bidi-paragraph-direction.  If you have RTL text embedded in LTR
> paragraphs, like in the HELLO file, <right> moves left while in the RTL
> segment.  This contradicts the manual description, which merely states
> "if the text is read right-to-left".
> 
> Which is the intended behavior?

The intended behavior is what you see now in Emacs.  The manual is
correct for the important special case of the paragraph direction that
coincides with the text direction.  Most L2R text in the world is in
L2R paragraphs and most R2L text is in R2L paragraphs, so the
description covers most of the use cases.

As for the less common use case you mention, yes, the description in
the manual could mislead if read too literally; "text is read
left-to-right" is intentionally vague to at least not be blatantly
wrong.

There was a long thread about this which started here:

  http://lists.gnu.org/archive/html/emacs-devel/2011-06/msg00005.html

In that thread, I explained the difficulties of describing the
behavior clearly and concisely to someone who doesn't already know
what is the expected behavior.  (Basically, only users of R2L scripts
will know what to expect.)  If you can come up with a better (more
accurate) description, please suggest it.  I don't think referencing
current-bidi-paragraph-direction is the way to do it, because it
sounds wrong to describe such a simple and basic editing key through
such a complex notion as base paragraph direction or by reference to
an obscure internal function.

Re: right-char and left-char

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

> The intended behavior is what you see now in Emacs.  The manual is
> correct for the important special case of the paragraph direction that
> coincides with the text direction.  Most L2R text in the world is in
> L2R paragraphs and most R2L text is in R2L paragraphs, so the
> description covers most of the use cases.
>
> As for the less common use case you mention, yes, the description in
> the manual could mislead if read too literally; "text is read
> left-to-right" is intentionally vague to at least not be blatantly
> wrong.

This lands us in the unfortunate situation where a command named
`left-char' sometimes moves right, and a command named `right-char'
sometimes moves left.  Their names seem to imply that what distinguishes
these commands from `forward-char'/`backward-char' is their
directionality.

RE: right-char and left-char

[Drew Adams]

> This lands us in the unfortunate situation where a command named
> `left-char' sometimes moves right, and a command named `right-char'
> sometimes moves left.  Their names seem to imply that what 
> distinguishes these commands from `forward-char'/`backward-char'
> is their directionality.

http://lists.gnu.org/archive/html/emacs-devel/2011-05/msg00836.html

Wrt the _key_ named `left', Eli said:

"Because `left' and `right' behave differently depending on the bidirectional
context,"...

and  "if `left' sometimes moves _forward_ in the buffer, binding it to a command
called `backward-char' [would be] a lie."

[Of course back then he also said, "[Bidi] _is_ optional: you can set
bidi-display-reordering to nil" (his emphasis).  Now he says that that variable
is not to make bidi optional for users but is only for bidi debugging.  (I hope
the paraphrase is accurate.)]

> Another question: how come C-<right> is bound to `right-word' but
> M-<right> is still bound to `forward-word'?  An oversight?

Eli spoke to that to some extent, as well:

"the same "problem" exists with `M-f'/`M-b' and
`C-<right>'/`C-<left>' as well.  We even discussed the possibility to
make `<Home>' and `<End>' behaving differently"

It can perhaps help to read the whole thread ("`C-b' is backward-char, `left' is
left-char - why?"), if the doc is not clear enough about some of these things.

Re: right-char and left-char

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Thu, 6 Oct 2011 15:45:53 -0700
> Cc: emacs-devel@gnu.org
> 
> Wrt the _key_ named `left', Eli said:
> 
> "Because `left' and `right' behave differently depending on the bidirectional
> context,"...
> 
> and  "if `left' sometimes moves _forward_ in the buffer, binding it to a command
> called `backward-char' [would be] a lie."
> 
> [Of course back then he also said, "[Bidi] _is_ optional: you can set
> bidi-display-reordering to nil" (his emphasis).  Now he says that that variable
> is not to make bidi optional for users but is only for bidi debugging. [...]]

Translation: Don't believe a word of what the lying Eli says.  Nice.

Of course, back then I also said

  The only reason why the bidi-display-reordering flag will stay is
  because unibyte buffers should not be reordered.

(which in the meantime is also OBE, since the value of
bidi-display-reordering is ignored in unibyte buffers).  I guess
you've conveniently "forgot" to cite that.

How to reconcile this "contradiction" in two sentences I wrote in the
same message, is left as an exercise for the reader.

Re: right-char and left-char

[Stefan Monnier]

> This lands us in the unfortunate situation where a command named
> `left-char' sometimes moves right, and a command named `right-char'
> sometimes moves left.  Their names seem to imply that what distinguishes
> these commands from `forward-char'/`backward-char' is their
> directionality.

None of this sounds very satisfactory, but at the same time giving
a name that faithfully reflects the behavior seems impossibly hard.
Basically "left-char" means "do what is intended by pressing the left
arrow", which is of course circular.

I think we're better off with the current approximate names and
description than with something that goes into too many details.


        Stefan "who still finds it odd that keeping <right> pressed may
                end up stopping in the middle of the buffer"

Re: right-char and left-char

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Thu, 06 Oct 2011 21:49:51 -0400
> 
>         Stefan "who still finds it odd that keeping <right> pressed may
>                 end up stopping in the middle of the buffer"

FWIW, the same happens in several other bidi-aware apps that support
paragraphs of different direction in the same document.

Of course, C-f/C-b don't have these problems, so can be used as a
workaround.

Re: right-char and left-char

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Cc: emacs-devel@gnu.org
> Date: Thu, 06 Oct 2011 18:04:55 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > The intended behavior is what you see now in Emacs.  The manual is
> > correct for the important special case of the paragraph direction that
> > coincides with the text direction.  Most L2R text in the world is in
> > L2R paragraphs and most R2L text is in R2L paragraphs, so the
> > description covers most of the use cases.
> >
> > As for the less common use case you mention, yes, the description in
> > the manual could mislead if read too literally; "text is read
> > left-to-right" is intentionally vague to at least not be blatantly
> > wrong.
> 
> This lands us in the unfortunate situation where a command named
> `left-char' sometimes moves right, and a command named `right-char'
> sometimes moves left.

That's unfortunate, I agree, but I couldn't find better names.
Suggestions welcome, it's not too late to change these if we want.

> Their names seem to imply that what distinguishes
> these commands from `forward-char'/`backward-char' is their
> directionality.

That isn't _entirely_ false...

Re: right-char and left-char

[Chong Yidong]

Another question: how come C-<right> is bound to `right-word' but
M-<right> is still bound to `forward-word'?  An oversight?

Re: right-char and left-char

[Eli Zaretskii]

> From: Chong Yidong <cyd@stupidchicken.com>
> Cc: emacs-devel@gnu.org
> Date: Thu, 06 Oct 2011 18:20:53 -0400
> 
> Another question: how come C-<right> is bound to `right-word' but
> M-<right> is still bound to `forward-word'?  An oversight?

Not an oversight.  Users of bidirectional scripts expect
Control-arrows to move by words in the paragraph direction, but they
don't expect Meta-arrows to do that.  In fact, I'm not aware of many
applications that bind Meta-arrows to cursor motion commands.

That said, I have no objections to make Meta-arrows do the same, if
you think being consistent is better.  I don't think it will surprise
users of bidirectional scripts if we make Meta-arrows behave similarly
to Control-arrows.

Re: right-char and left-char

[Chong Yidong]

Eli Zaretskii <eliz@gnu.org> writes:

>> Another question: how come C-<right> is bound to `right-word' but
>> M-<right> is still bound to `forward-word'?  An oversight?
>
> Not an oversight.  Users of bidirectional scripts expect
> Control-arrows to move by words in the paragraph direction, but they
> don't expect Meta-arrows to do that.  In fact, I'm not aware of many
> applications that bind Meta-arrows to cursor motion commands.
>
> That said, I have no objections to make Meta-arrows do the same, if
> you think being consistent is better.  I don't think it will surprise
> users of bidirectional scripts if we make Meta-arrows behave similarly
> to Control-arrows.

Yes, if we're going by the heuristic that "arrow keys obey
bidi-paragraph-direction", it should be consistently applied.  I'll make
this change.