bug#52587: 29.0.50; Wrong block header/footer background in Org

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Rudolf Adamkovič via Bug reports for GNU Emacs, the Swiss army knife of text editors]

I keep seeing buggy drawing in Org with Modus Themes.

Reproduction steps:

1. run 'emacs -Q'
2. type 'M-x load-theme'
3. type 'modus-operandi'
4. type 'C-x C-f'
5. type 'test.org'
6. type:

* foo
#+begin_quote
bar
#+end_quote
* baz

7. move point to 'foo'
8. type 'TAB'

Expected: headings with default (white) background

Actual: one heading has buggy (partially gray) background

Rudy


In GNU Emacs 29.0.50 (build 11, x86_64-apple-darwin21.2.0, NS appkit-2113.20 Version 12.1 (Build 21C52))
 of 2021-12-17 built on Workstation.local
Repository revision: efdfe22704581dca8bc8b32672f7b5ae142f1fd5
Repository branch: master
Windowing system distributor 'Apple', version 10.3.2113
System Description:  macOS 12.1

Configured using:
 'configure --with-json --with-xwidgets --with-native-compilation'

Configured features:
ACL DBUS GIF GLIB GMP GNUTLS JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP
NOTIFY KQUEUE NS PDUMPER PNG RSVG SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS WEBP XIM XWIDGETS ZLIB

Important settings:
  value of $LC_ALL: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  paredit-mode: t
  display-fill-column-indicator-mode: t
  corfu-global-mode: t
  corfu-mode: t
  vertico-mode: t
  marginalia-mode: t
  global-diff-hl-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  global-hl-todo-mode: t
  hl-todo-mode: t
  global-subword-mode: t
  subword-mode: t
  save-place-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  savehist-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/Users/salutis/.emacs.d/elpa/transient-20211208.1819/transient hides /Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/transient
/Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/emacs-lisp/eieio-compat hides /Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/obsolete/eieio-compat

Features:
(shadow sort bbdb-message fortune flymake-proc flymake project compile
thingatpt mail-extr emacsbug message yank-media rmc puny ls-lisp dired
dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068 epg-config
gnus-util rmail rmail-loaddefs text-property-search mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils char-fold
orderless cursor-sensor paredit edmacro kmacro
display-fill-column-indicator bbdb bbdb-site timezone
modus-vivendi-theme modus-operandi-theme modus-themes corfu vertico
marginalia pdf-loader diff-hl log-view pcvs-util vc-dir ewoc vc
vc-dispatcher diff-mode diminish yasnippet hl-todo finder-inf ob-sqlite
ob-sql ob-C cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles
cc-align cc-engine cc-vars cc-defs ob-R org-clock cl cap-words superword
subword saveplace autorevert filenotify comp comp-cstr warnings delsel
savehist cl-extra help-mode org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-footnote org-src ob-comint org-pcomplete pcomplete
comint ansi-color ring org-list org-faces org-entities noutline outline
easy-mmode org-version ob-emacs-lisp ob-core ob-eval org-table oc-basic
bibtex iso8601 time-date ol rx org-keys oc org-compat advice org-macs
org-loaddefs format-spec find-func cal-menu calendar cal-loaddefs
tex-site info package browse-url url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache json map url-vars seq gv subr-x byte-opt
bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl tooltip eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget keymap hashtable-print-readable backquote threads
xwidget-internal dbusbind kqueue cocoa ns lcms2 multi-tty
make-network-process native-compile emacs)

Memory information:
((conses 16 621646 20332)
 (symbols 48 30239 11)
 (strings 32 167903 5449)
 (string-bytes 1 5056735)
 (vectors 16 47217)
 (vector-slots 8 815311 33172)
 (floats 8 230 28)
 (intervals 56 4144 0)
 (buffers 992 11))

-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and if it were so, it would be; but as it isn't, it ain't. That's logic.'" -- Lewis Carroll, Through the Looking Glass

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Lars Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 559 bytes --]

Rudolf Adamkovič <salutis@me.com> writes:

> I keep seeing buggy drawing in Org with Modus Themes.
>
> Reproduction steps:
>
> 1. run 'emacs -Q'
> 2. type 'M-x load-theme'
> 3. type 'modus-operandi'
> 4. type 'C-x C-f'
> 5. type 'test.org'
> 6. type:
>
> * foo
>
> #+begin_quote
> bar
> #+end_quote
>
> * baz
>
> 7. move point to 'foo'
> 8. type 'TAB'
>
> Expected: headings with default (white) background
>
> Actual: one heading has buggy (partially gray) background

After following this recipe on the current trunk, I get:


[-- Attachment #2: Type: image/png, Size: 19317 bytes --]

[-- Attachment #3: Type: text/plain, Size: 214 bytes --]


So I seem to be unable to reproduce the problem (on Debian/bookworm).
Are there any additional steps needed?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Kévin Le Gouguec]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> So I seem to be unable to reproduce the problem (on Debian/bookworm).
> Are there any additional steps needed?

Try removing the extra newline between "#+end_quote" and "* baz".

AFAIR this is a long-standing issue in outline, to reproduce without Org
nor specific themes:

1. emacs -Q
2. C-x b repro
3. M-x outline-mode
4. type in: "* foo RET bar RET * baz"
5. move to "bar"
6. M-: 
   (add-text-properties
    (point-at-bol)
    (1+ (point-at-eol))
    '(font-lock-face (:background "red")))
7. S-TAB (to fold all sections)

Here, I see a single column with red background after the ellipses.  If
I add ":extend t" to the face definition, then the whole line has a red
background.

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Eli Zaretskii]

> From: Kévin Le Gouguec <kevin.legouguec@gmail.com>
> Date: Sun, 19 Dec 2021 13:10:37 +0100
> Cc: 52587@debbugs.gnu.org, Rudolf Adamkovič <salutis@me.com>
> 
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
> > So I seem to be unable to reproduce the problem (on Debian/bookworm).
> > Are there any additional steps needed?
> 
> Try removing the extra newline between "#+end_quote" and "* baz".
> 
> AFAIR this is a long-standing issue in outline, to reproduce without Org
> nor specific themes:
> 
> 1. emacs -Q
> 2. C-x b repro
> 3. M-x outline-mode
> 4. type in: "* foo RET bar RET * baz"
> 5. move to "bar"
> 6. M-: 
>    (add-text-properties
>     (point-at-bol)
>     (1+ (point-at-eol))
>     '(font-lock-face (:background "red")))
> 7. S-TAB (to fold all sections)
> 
> Here, I see a single column with red background after the ellipses.

Why is that a problem?  That single column has buffer position 10, and
is the only visible buffer position that has the red color before
S-TAB.

So I think Emacs behaves as expected here.

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Kévin Le Gouguec]

[-- Attachment #1: Type: text/plain, Size: 1544 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Kévin Le Gouguec <kevin.legouguec@gmail.com>
>> Date: Sun, 19 Dec 2021 13:10:37 +0100
>> Cc: 52587@debbugs.gnu.org, Rudolf Adamkovič <salutis@me.com>
>> 
>> 1. emacs -Q
>> 2. C-x b repro
>> 3. M-x outline-mode
>> 4. type in: "* foo RET bar RET * baz"
>> 5. move to "bar"
>> 6. M-: 
>>    (add-text-properties
>>     (point-at-bol)
>>     (1+ (point-at-eol))
>>     '(font-lock-face (:background "red")))
>> 7. S-TAB (to fold all sections)
>> 
>> Here, I see a single column with red background after the ellipses.
>
> Why is that a problem?  That single column has buffer position 10, and
> is the only visible buffer position that has the red color before
> S-TAB.
>
> So I think Emacs behaves as expected here.

From an implementation POV, the current behaviour makes perfect sense,
yes, because outline-mode essentially hides everything ranging from the
newline at the end of "* foo" up to but *excluding* the newline after
"bar".

But I think this bug report shows that some users have different
expectations, rooted in an abstract sense of what is and is not "part"
of the subsection to hide.  Specifically, they consider that the
end-of-line decoration of a subsection's last line is "part" of the
subsection, and so expect it to be hidden when the subsection is folded.

To put it in picture in case that wasn't clear, assuming a derived mode
of Outline which would also fontify a headline's final newline, some
users expect that in this buffer:


[-- Attachment #2: unfolded.png --]
[-- Type: image/png, Size: 45789 bytes --]

[-- Attachment #3: Type: text/plain, Size: 33 bytes --]


Folding should not yield this:


[-- Attachment #4: folded-current.png --]
[-- Type: image/png, Size: 46093 bytes --]

[-- Attachment #5: Type: text/plain, Size: 19 bytes --]


But rather that:


[-- Attachment #6: folded-requested.png --]
[-- Type: image/png, Size: 43594 bytes --]

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Lars Ingebrigtsen]

Kévin Le Gouguec <kevin.legouguec@gmail.com> writes:

> But I think this bug report shows that some users have different
> expectations, rooted in an abstract sense of what is and is not "part"
> of the subsection to hide.  Specifically, they consider that the
> end-of-line decoration of a subsection's last line is "part" of the
> subsection, and so expect it to be hidden when the subsection is folded.

Hm, yes.  Makes sense to me.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Eli Zaretskii]

> From: Kévin Le Gouguec <kevin.legouguec@gmail.com>
> Cc: larsi@gnus.org,  salutis@me.com,  52587@debbugs.gnu.org
> Date: Mon, 20 Dec 2021 07:33:07 +0100
> 
> But I think this bug report shows that some users have different
> expectations, rooted in an abstract sense of what is and is not "part"
> of the subsection to hide.  Specifically, they consider that the
> end-of-line decoration of a subsection's last line is "part" of the
> subsection, and so expect it to be hidden when the subsection is folded.

Which means the Lisp code which defines which parts are hidden should
hide slightly different parts?  Because the display engine does its
job perfectly here, and so do the faces: they do what you tell them to
do.

IOW, while I see your point, I don't think I understand how would you
like this to be resolved.

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Rudolf Adamkovič via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 154 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> After following this recipe on the current trunk, I get: [image]

This is how it looks on NS (macOS), FYI:


[-- Attachment #2: Org Mode glitch --]
[-- Type: image/png, Size: 190495 bytes --]

[-- Attachment #3: Type: text/plain, Size: 263 bytes --]


Rudy
-- 
"Logic is a science of the necessary laws of thought, without which no employment of the understanding and the reason takes place." -- Immanuel Kant, 1785

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Kévin Le Gouguec]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Kévin Le Gouguec <kevin.legouguec@gmail.com>
>> Cc: larsi@gnus.org,  salutis@me.com,  52587@debbugs.gnu.org
>> Date: Mon, 20 Dec 2021 07:33:07 +0100
>> 
>> But I think this bug report shows that some users have different
>> expectations, rooted in an abstract sense of what is and is not "part"
>> of the subsection to hide.  Specifically, they consider that the
>> end-of-line decoration of a subsection's last line is "part" of the
>> subsection, and so expect it to be hidden when the subsection is folded.
>
> Which means the Lisp code which defines which parts are hidden should
> hide slightly different parts?  Because the display engine does its
> job perfectly here, and so do the faces: they do what you tell them to
> do.

That sounds about right; FWIW I'd imagine this to be fixed/worked around
either in outline-mode itself, or in derived modes which feature these
:extended :backgrounds.

> IOW, while I see your point, I don't think I understand how would you
> like this to be resolved.

I don't have a concrete implementation proposal yet (this itch has
remained stuck in my "minor enough that I'll let someone else file a
report" drawer ever since I discovered it; thanks Rudolf for the report
btw 😉).

On the "vague handwavy half-baked idea" front, I'd imagine that when
folding a headline's subsections, Outline could (1) check whether the
headline has :extended decorations at end-of-line (2) pull off some
shenanigans to "copy" these decorations onto the hidden content's final
newline (by hacking its {font-lock-,}face property or slapping a display
property on top of it maybe?), but that sounds pretty brittle.

On the "gazing at the other side of fence where the grass looks greener"
front, I wonder if there are things to learn from Jonas's
magit-section[1], which AFAICT quacks like outline-mode (i.e. lets users
define/show/hide/navigate arbitrarily nested hierarchies of subsections)
and does not seem to suffer from this problem despite making heavy use
of extended backgrounds.

The "actionable suggestion" front is late to the battlefield, sorry 😕


[1] Code:
    https://git.savannah.gnu.org/cgit/emacs/nongnu.git/plain/lisp/magit-section.el?h=elpa/magit-section
    Manual:
    https://magit.vc/manual/magit-section/

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Protesilaos Stavrou]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

On 2021-12-20, 23:16 +0100, Rudolf Adamkovič wrote:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> After following this recipe on the current trunk, I get: [image]
>
> This is how it looks on NS (macOS), FYI:

While what you have observed with the outline folding is not specific to
the themes, there is a minor bug on my end as well: the default
delimiter faces should not have the ':extend t' attribute.  Please check
the attached screenshots.

[ I will sync with emacs.git as soon as I publish the next tagged
  version (2.0.0).  It is a big one and the change log will take some
  time to prepare...  Maybe this week. ]

-- 
Protesilaos Stavrou
https://protesilaos.com

[-- Attachment #2: screenshot_region_2021-12-21-11:52:07.png --]
[-- Type: image/png, Size: 19371 bytes --]

[-- Attachment #3: screenshot_region_2021-12-21-11:52:38.png --]
[-- Type: image/png, Size: 18711 bytes --]

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Rudolf Adamkovič via General discussions about Org-mode.]

Protesilaos Stavrou <info@protesilaos.com> writes:

> [ I will sync with emacs.git as soon as I publish the next tagged
>   version (2.0.0).  It is a big one and the change log will take some
>   time to prepare...  Maybe this week. ]

I noticed "Update modus-themes to version 2.0.0" in the Git log.  Yet, I
compiled Emacs today (2021-12-26), and the gray area extends
edge-to-edge while modus-themes-org-blocks equals nil.  Perhaps I
misunderstood your intention?

Rudy
-- 
"Logic is a science of the necessary laws of thought, without which no employment of the understanding and the reason takes place." -- Immanuel Kant, 1785

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Protesilaos Stavrou]

[-- Attachment #1: Type: text/plain, Size: 1152 bytes --]

On 2021-12-26, 11:22 +0100, Rudolf Adamkovič <salutis@me.com> wrote:

> Protesilaos Stavrou <info@protesilaos.com> writes:
>
>> [ I will sync with emacs.git as soon as I publish the next tagged
>>   version (2.0.0).  It is a big one and the change log will take some
>>   time to prepare...  Maybe this week. ]
>
> I noticed "Update modus-themes to version 2.0.0" in the Git log.  Yet, I
> compiled Emacs today (2021-12-26), and the gray area extends
> edge-to-edge while modus-themes-org-blocks equals nil.  Perhaps I
> misunderstood your intention?

No, you did not misunderstand my intention.  Maybe something else is
going on, as I am getting the expected results.  Check the attached
screenshot.

Steps to reproduce the screenshot:

- Create a file with demo content, such as ~/test-org-block.org:

    * Testing

    This is a test

    #+begin_src emacs-lisp
    (message "Hello world")
    #+end_src

    * Test

- Execute 'emacs -Q' on the command-line.
- C-x C-f test-org-block.org RET
- M-x load-theme RET modus-operandi

Do you get the same results?

-- 
Protesilaos Stavrou
https://protesilaos.com

[-- Attachment #2: screenshot_region_2021-12-26-13:18:19.png --]
[-- Type: image/png, Size: 25281 bytes --]

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Rudolf Adamkovič via General discussions about Org-mode.]

[-- Attachment #1: Type: text/plain, Size: 344 bytes --]

Protesilaos Stavrou <info@protesilaos.com> writes:

> Do you get the same results?

Oh, I do!  That said, if I reverse last two steps, I do not:

> - Create a file with demo content, such as ~/test-org-block.org
> - Execute 'emacs -Q' on the command-line.
> - M-x load-theme RET modus-operandi
> - C-x C-f test-org-block.org RET

Then, I get:


[-- Attachment #2: Bug --]
[-- Type: image/png, Size: 372296 bytes --]

[-- Attachment #3: Type: text/plain, Size: 296 bytes --]


Rudy
-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and
if it were so, it would be; but as it isn't, it ain't. That's logic.'"
-- Lewis Carroll, Through the Looking Glass

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Protesilaos Stavrou]

On 2021-12-26, 22:04 +0100, Rudolf Adamkovič <salutis@me.com> wrote:

> Protesilaos Stavrou <info@protesilaos.com> writes:
>
>> Do you get the same results?
>
> Oh, I do!  That said, if I reverse last two steps, I do not:
>
>> - Create a file with demo content, such as ~/test-org-block.org
>> - Execute 'emacs -Q' on the command-line.
>> - M-x load-theme RET modus-operandi
>> - C-x C-f test-org-block.org RET

Gotcha!  I can reproduce that.  It is not a theme issue.  I checked the
Org code a bit.  There is a function in org-compat.el called
'org--set-faces-extend' and in org.el we see it being used in the
(define-derived-mode org-mode ... part.

In other words, M-x org-mode wants those faces to have ':extend t'.

I must investigate this further, though my first impression is that we
need to notify the Org folks about it.

-- 
Protesilaos Stavrou
https://protesilaos.com

bug#52587: 29.0.50; Wrong block header/footer background in Org

[Kévin Le Gouguec]

Protesilaos Stavrou <info@protesilaos.com> writes:

> On 2021-12-26, 22:04 +0100, Rudolf Adamkovič <salutis@me.com> wrote:
>
>>> - Create a file with demo content, such as ~/test-org-block.org
>>> - Execute 'emacs -Q' on the command-line.
>>> - M-x load-theme RET modus-operandi
>>> - C-x C-f test-org-block.org RET
>
> Gotcha!  I can reproduce that.  It is not a theme issue.  I checked the
> Org code a bit.  There is a function in org-compat.el called
> 'org--set-faces-extend' and in org.el we see it being used in the
> (define-derived-mode org-mode ... part.
>
> In other words, M-x org-mode wants those faces to have ':extend t'.

Only when org-fontify-whole-block-delimiter-line is set (which it is, by
default, I think).