From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eric Abrahamsen Newsgroups: gmane.emacs.devel Subject: Re: Message Mode and bidi Date: Wed, 28 Feb 2024 08:54:54 -0800 Message-ID: <87edcwcyw1.fsf@ericabrahamsen.net> References: <87v86ldzpw.fsf@aura.christopherculver.com> <867cj1qg4m.fsf@gnu.org> <8734to16tw.fsf@aura.christopherculver.com> <86bk8cw20e.fsf@p200300d627023a0ad1f3c3db8ccb4c50.dip0.t-ipconnect.de> <87frxoyuq4.fsf@aura.christopherculver.com> <87ttm4klnl.fsf@ericabrahamsen.net> <86y1bfolgf.fsf@gnu.org> <87h6i3lnp8.fsf@ericabrahamsen.net> <86o7cbnqxb.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="14915"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) To: emacs-devel@gnu.org Cancel-Lock: sha1:EYe6Zp54Z2c1xsfxsheAFCAYyUE= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Feb 28 18:10:30 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rfNRt-0003az-U9 for ged-emacs-devel@m.gmane-mx.org; Wed, 28 Feb 2024 18:10:30 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rfNQH-000808-To; Wed, 28 Feb 2024 12:08:49 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfNDL-0000N2-CC for emacs-devel@gnu.org; Wed, 28 Feb 2024 11:55:30 -0500 Original-Received: from ciao.gmane.io ([116.202.254.214]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rfND3-0004QJ-NZ for emacs-devel@gnu.org; Wed, 28 Feb 2024 11:55:22 -0500 Original-Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1rfNCz-0002Zg-Lb for emacs-devel@gnu.org; Wed, 28 Feb 2024 17:55:05 +0100 X-Injected-Via-Gmane: http://gmane.org/ Received-SPF: pass client-ip=116.202.254.214; envelope-from=ged-emacs-devel@m.gmane-mx.org; helo=ciao.gmane.io X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_HELO_TEMPERROR=0.01, T_SPF_TEMPERROR=0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 28 Feb 2024 12:08:47 -0500 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:316625 Archived-At: Eli Zaretskii writes: >> From: Eric Abrahamsen >> Date: Mon, 19 Feb 2024 21:16:51 -0800 >> >> >> Eli Zaretskii writes: >> >> >> From: Eric Abrahamsen >> >> Date: Mon, 19 Feb 2024 16:46:22 -0800 >> >> >> >> Christopher Culver via "Emacs development discussions." >> >> writes: >> >> >> >> > Joost Kremers writes: >> >> >> When you compose a new message, is there a line "--text follows this line--" >> >> >> separating the headers and the message text? In my case, there is (I use mu4e) >> >> >> and when I type Arabic text on the line below this text, I get the effect you >> >> >> mention. If I leave an empty line after "--text follows this line--", bidi works >> >> >> as expected. >> >> > >> >> > Indeed, if I just go down one line and then begin typing, bidi works as >> >> > expected. I am feeling very foolish that I did not even try this. Thank >> >> > you for clearing up this problem, and for shedding light on how Emacs >> >> > considers paragraphs. >> >> >> >> This is a bit weird, because the value of `mail-header-separator' >> >> ("--text follows this line--") is added to both `paragraph-start' and >> >> `paragraph-separate') in `message-mode'. You'd think one of those would >> >> do it. >> > >> > Emacs doesn't use paragraph-separate and paragraph-start to define >> > where a paragraph starts and ends, for the purposes of determining the >> > base directionality of a paragraph. It uses separate variables for >> > that, see the node "Bidirectional Editing" in the Emacs user manual. >> > The reason for using separate variables is because several modes, >> > including (but not limited to) message-mode, set the former variables >> > to regexps that get in the way of bidi reordering, and could easily >> > produce wrong results on display. >> >> Do you think we'd stand a chance of finding values for >> bidi-paragraph-start|separate-re that would resolve this particular >> issue? > > We already have those values described in the Emacs manual: > > Each paragraph of bidirectional text can have its own “base > direction”, either right-to-left or left-to-right. Text in > left-to-right paragraphs begins on the screen at the left margin of the > window and is truncated or continued when it reaches the right margin. > By contrast, text in right-to-left paragraphs is displayed starting at > the right margin and is continued or truncated at the left margin. By > default, paragraph boundaries are empty lines, i.e., lines consisting > entirely of whitespace characters. To change that, you can customize > the two variables ‘bidi-paragraph-start-re’ and > ‘bidi-paragraph-separate-re’, whose values should be regular expressions > (strings); e.g., to have a single newline start a new paragraph, set > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > both of these variables to ‘"^"’. These two variables are buffer-local > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > (*note Locals::). > > But beware: doing this in Emacs will cause effects that are unpleasant > to readers of bidirectional text, because you could have "chess-like" > text display, like this: > > asasasasasasasasasasasasassa > ASASASASASASASASASASASASA > xcxcxcxcxcxcxcxcxcxcxcxcxc > JKJNJKKKNKNKNKNKNK > > etc. Here upper-case letters stand for RTL (like Arabic or Farsi) > text and lower-case letters stand for LTR (like Latin or Cyrillic) > text. This is the effect I was imagining in the header section of the message buffer, and why I thought we should probably skip trying to handle this. message-mode headers can be "continuation headers", as well, effectively line-wrapped, which would make it even harder. > Let me explain. UBA, the Unicode Bidirectional Algorithm which Emacs > implements, was designed for text-processing programs that wrap long > lines without inserting hard newlines. For those applications, a > single hard newline indicates a new paragraph, and thus recalculating > the base direction of a paragraph after a newline is justified. > > But in Emacs, we have a lot of text filled and wrapped using hard > newlines, and filling can insert a newline in an arbitrary place > inside the paragraph. If it happens that a newline was inserted > before a strong right-to-left character, under the strict UBA rules > the next line will be considered as a paragraph with right-to-left > base direction, and rendered starting at the right window edge. Which > leads to the above "chess-like" display. Since that is basically > unacceptable for Emacs users, we require at least one empty line to > separate paragraphs. The price of a single empty line before the body > of an email message that needs to be rendered right to left is a small > price to pay for solving the horrible display effect of changing the > paragraph's base direction after each newline. An alternative would > be to use visual-line-mode instead of wrapping using hard newlines, > but that is still relatively rare in Emacs, especially in email > messages. Thanks for the background! I guess I was hoping that we could at least support OP's original request, which is making the first paragraph of the message body independent of the mail header separator as regards BIDI display. I experimented with putting the value of `mail-header-separator' into `bidi-paragraph-start|separate-re', but couldn't get it to display that first paragraph starting on the right. Do you think this is feasible, and worth the effort?