* bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode @ 2024-01-14 10:21 Thamer Mahmoud 2024-01-14 11:24 ` Eli Zaretskii 0 siblings, 1 reply; 9+ messages in thread From: Thamer Mahmoud @ 2024-01-14 10:21 UTC (permalink / raw) To: 68446 Hi, 1. In emacs -Q, create an empty buffer with Org mode active and type: a [[link]] 2. Evaluate: (setq bidi-paragraph-direction 'right-to-left) 3. Note the "a" and link are no longer visible. In GNU Emacs 29.1.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.39, cairo version 1.18.0) Repository branch: emacs-29 Windowing system distributor 'The X.Org Foundation', version 11.0.12101009 System Description: Debian GNU/Linux trixie/sid Configured features: CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB -- Regards, Thamer ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode 2024-01-14 10:21 bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode Thamer Mahmoud @ 2024-01-14 11:24 ` Eli Zaretskii 2024-01-14 12:11 ` Ihor Radchenko 0 siblings, 1 reply; 9+ messages in thread From: Eli Zaretskii @ 2024-01-14 11:24 UTC (permalink / raw) To: Thamer Mahmoud, Ihor Radchenko; +Cc: 68446 > From: Thamer Mahmoud <thamer.mahmoud@gmail.com> > Date: Sun, 14 Jan 2024 13:21:57 +0300 > > 1. In emacs -Q, create an empty buffer with Org mode active and type: > > a [[link]] > > 2. Evaluate: (setq bidi-paragraph-direction 'right-to-left) > 3. Note the "a" and link are no longer visible. I think it's an Org bug: it should prevent bidi reordering inside the "[[link]]" string. For example, wrap the "[[link]]" thing in LRO..PDF bidi controls. Because without that, the brackets can be mirrored by bidi reordering and the BPA algorithm, and the link is no longer in the form that Org expects. The result is that the entire text becomes invisible. A work-around is to do one of the following: . insert one or more L2R characters after the "[[link]]", or . set bidi-inhibit-bpa to a non-nil value I don't see an Emacs bug here, surprising as it may sound. Lisp programs that depend on particular sequence of characters on display should be aware that bidi reordering can affect that. Adding Ihor to the discussion. ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode 2024-01-14 11:24 ` Eli Zaretskii @ 2024-01-14 12:11 ` Ihor Radchenko 2024-01-14 13:00 ` Eli Zaretskii 0 siblings, 1 reply; 9+ messages in thread From: Ihor Radchenko @ 2024-01-14 12:11 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 68446, Thamer Mahmoud Eli Zaretskii <eliz@gnu.org> writes: >> 1. In emacs -Q, create an empty buffer with Org mode active and type: >> >> a [[link]] >> >> 2. Evaluate: (setq bidi-paragraph-direction 'right-to-left) >> 3. Note the "a" and link are no longer visible. > > I think it's an Org bug: it should prevent bidi reordering inside the > "[[link]]" string. For example, wrap the "[[link]]" thing in LRO..PDF > bidi controls. Because without that, the brackets can be mirrored by > bidi reordering and the BPA algorithm, and the link is no longer in > the form that Org expects. The result is that the entire text becomes > invisible. May you please elaborate? I do not see anything unexpected in the text properties that Org mode applies to the link and the letter "a". In fact, after following the reproducer, and enabling visible-mode, M-x describe-text-properties on "a" yields Text content at position 101: There are text properties here: fontified t Yet, "a" is rendered invisible. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode 2024-01-14 12:11 ` Ihor Radchenko @ 2024-01-14 13:00 ` Eli Zaretskii 2024-01-14 14:41 ` Ihor Radchenko 0 siblings, 1 reply; 9+ messages in thread From: Eli Zaretskii @ 2024-01-14 13:00 UTC (permalink / raw) To: Ihor Radchenko; +Cc: 68446, thamer.mahmoud > From: Ihor Radchenko <yantar92@posteo.net> > Cc: Thamer Mahmoud <thamer.mahmoud@gmail.com>, 68446@debbugs.gnu.org > Date: Sun, 14 Jan 2024 12:11:03 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > > I think it's an Org bug: it should prevent bidi reordering inside the > > "[[link]]" string. For example, wrap the "[[link]]" thing in LRO..PDF > > bidi controls. Because without that, the brackets can be mirrored by > > bidi reordering and the BPA algorithm, and the link is no longer in > > the form that Org expects. The result is that the entire text becomes > > invisible. > > May you please elaborate? I do not see anything unexpected in the text > properties that Org mode applies to the link and the letter "a". (For some value of "expected". Me, I don't understand more than half of the properties there, wrt their expected effect on the text and its display. Are you surprised?) Anyway, after performing the recipe, do this: . M-< . C-x = . C-f . C-x = . C-f . C-x = etc. IOW, move 1 buffer position forward at a time from BOB, and each time show the character at that position. Do you see something strange? Then do the same in a buffer under Fundamental mode with the same text and the same value of bidi-paragraph-direction. Can you explain what Org does to cause the difference, which makes some characters be "skipped" by C-f (due to point-adjustment feature), and probably also makes all the characters invisible as result? I can tell you what I see when stepping through the display code: it skips all the characters as invisible. I see that org-link is in the buffer-invisibility-spec, but that's probably just the tip of the iceberg, so I'm asking you below to describe what are all those faces and properties for, and how they are related to the invisibility. > In fact, after following the reproducer, and enabling visible-mode, M-x > describe-text-properties on "a" yields > > Text content at position 101: > > > There are text properties here: > fontified t > > Yet, "a" is rendered invisible. When you invoke visible-mode, the text is magically shown as expected. Can you explain why is that? If you invoke describe-text-properties on any character in the buffer, there's no character with 'invisible' face, either. And yet they are invisible. How come? If you describe in enough details how the [[link]] thing is handled by Org, maybe I could elaborate more than the general observations and questions above. To summarize, my point is simple: the Emacs bidirectional display is responsible for reordering plain text with faces, and while doing so, it more-or-less ignores invisible text. Anything fancier: overlays, images, text properties that affect display in some other ways, etc. -- all these are not guaranteed to behave in reordered text as they do in strict L2R text, and Lisp programs which create visual effects using those fancier features should look out for problems, and should use bidi formatting controls judiciously to prevent such disasters. (My guess is that the brackets in this case are reordered, and as result cause everything to become invisible. But that's just a guess, and I don't expect to understand why without a lot more background and help from Org experts.) ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode 2024-01-14 13:00 ` Eli Zaretskii @ 2024-01-14 14:41 ` Ihor Radchenko 2024-01-14 14:58 ` Eli Zaretskii 0 siblings, 1 reply; 9+ messages in thread From: Ihor Radchenko @ 2024-01-14 14:41 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 68446, thamer.mahmoud Eli Zaretskii <eliz@gnu.org> writes: >> May you please elaborate? I do not see anything unexpected in the text >> properties that Org mode applies to the link and the letter "a". > > ... > If you describe in enough details how the [[link]] thing is handled by > Org, maybe I could elaborate more than the general observations and > questions above. Let's simplify the reproducer, getting rid of Org mode: (let ((str (concat "a " (propertize "[" 'invisible t) "b" (propertize "]" 'invisible t)))) (with-temp-buffer (display-buffer (current-buffer)) (insert str) (read-char "'a b' is visible. Press any key.") (setq bidi-paragraph-direction 'right-to-left) (read-char "'a b' is invisible. Press any key."))) Is this the above expected? -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode 2024-01-14 14:41 ` Ihor Radchenko @ 2024-01-14 14:58 ` Eli Zaretskii 2024-02-04 9:51 ` Eli Zaretskii 0 siblings, 1 reply; 9+ messages in thread From: Eli Zaretskii @ 2024-01-14 14:58 UTC (permalink / raw) To: Ihor Radchenko; +Cc: 68446, thamer.mahmoud > From: Ihor Radchenko <yantar92@posteo.net> > Cc: thamer.mahmoud@gmail.com, 68446@debbugs.gnu.org > Date: Sun, 14 Jan 2024 14:41:21 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> May you please elaborate? I do not see anything unexpected in the text > >> properties that Org mode applies to the link and the letter "a". > > > > ... > > If you describe in enough details how the [[link]] thing is handled by > > Org, maybe I could elaborate more than the general observations and > > questions above. Actually, I think I see the reason, and the problem is indeed in the display code's logic in this case. > Let's simplify the reproducer, getting rid of Org mode: > > (let ((str (concat "a " (propertize "[" 'invisible t) "b" (propertize "]" 'invisible t)))) > (with-temp-buffer > (display-buffer (current-buffer)) > (insert str) > (read-char "'a b' is visible. Press any key.") > (setq bidi-paragraph-direction 'right-to-left) > (read-char "'a b' is invisible. Press any key."))) > > Is this the above expected? No. ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode 2024-01-14 14:58 ` Eli Zaretskii @ 2024-02-04 9:51 ` Eli Zaretskii 2024-02-04 13:10 ` Ihor Radchenko 0 siblings, 1 reply; 9+ messages in thread From: Eli Zaretskii @ 2024-02-04 9:51 UTC (permalink / raw) To: yantar92, thamer.mahmoud; +Cc: 68446 > Cc: 68446@debbugs.gnu.org, thamer.mahmoud@gmail.com > Date: Sun, 14 Jan 2024 16:58:43 +0200 > From: Eli Zaretskii <eliz@gnu.org> > > > From: Ihor Radchenko <yantar92@posteo.net> > > Cc: thamer.mahmoud@gmail.com, 68446@debbugs.gnu.org > > Date: Sun, 14 Jan 2024 14:41:21 +0000 > > > > Eli Zaretskii <eliz@gnu.org> writes: > > > > >> May you please elaborate? I do not see anything unexpected in the text > > >> properties that Org mode applies to the link and the letter "a". > > > > > > ... > > > If you describe in enough details how the [[link]] thing is handled by > > > Org, maybe I could elaborate more than the general observations and > > > questions above. > > Actually, I think I see the reason, and the problem is indeed in the > display code's logic in this case. There was an algorithmic flaw in handling invisible property when the invisible text starts inside embedding level above the base paragraph level: the code failed to account for the "backward" iteration which is part of the bidi reordering, and instead always looked forward for the first visible position. > > Let's simplify the reproducer, getting rid of Org mode: > > > > (let ((str (concat "a " (propertize "[" 'invisible t) "b" (propertize "]" 'invisible t)))) > > (with-temp-buffer > > (display-buffer (current-buffer)) > > (insert str) > > (read-char "'a b' is visible. Press any key.") > > (setq bidi-paragraph-direction 'right-to-left) > > (read-char "'a b' is invisible. Press any key."))) > > > > Is this the above expected? > > No. Should be fixed now on the master branch. ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode 2024-02-04 9:51 ` Eli Zaretskii @ 2024-02-04 13:10 ` Ihor Radchenko 2024-02-04 13:43 ` Eli Zaretskii 0 siblings, 1 reply; 9+ messages in thread From: Ihor Radchenko @ 2024-02-04 13:10 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 68446, thamer.mahmoud Eli Zaretskii <eliz@gnu.org> writes: > Should be fixed now on the master branch. I confirm that I can no longer reproduce the problem following the original steps and my reproducer. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at <https://orgmode.org/>. Support Org development at <https://liberapay.com/org-mode>, or support my work at <https://liberapay.com/yantar92> ^ permalink raw reply [flat|nested] 9+ messages in thread
* bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode 2024-02-04 13:10 ` Ihor Radchenko @ 2024-02-04 13:43 ` Eli Zaretskii 0 siblings, 0 replies; 9+ messages in thread From: Eli Zaretskii @ 2024-02-04 13:43 UTC (permalink / raw) To: Ihor Radchenko; +Cc: 68446-done, thamer.mahmoud > From: Ihor Radchenko <yantar92@posteo.net> > Cc: thamer.mahmoud@gmail.com, 68446@debbugs.gnu.org > Date: Sun, 04 Feb 2024 13:10:14 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Should be fixed now on the master branch. > > I confirm that I can no longer reproduce the problem following the > original steps and my reproducer. Thanks, I'm therefore closing this bug. We can reopen if someone reports some issues or left-overs. ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2024-02-04 13:43 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-01-14 10:21 bug#68446: 29.1.90; Bidi right-to-left paragraphs missing text in Org mode Thamer Mahmoud 2024-01-14 11:24 ` Eli Zaretskii 2024-01-14 12:11 ` Ihor Radchenko 2024-01-14 13:00 ` Eli Zaretskii 2024-01-14 14:41 ` Ihor Radchenko 2024-01-14 14:58 ` Eli Zaretskii 2024-02-04 9:51 ` Eli Zaretskii 2024-02-04 13:10 ` Ihor Radchenko 2024-02-04 13:43 ` Eli Zaretskii
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).