[BUG] Writing table to a folded drawer unfolds it. [9.7.12 (release_9.7.12 @ /home/natrys/.emacs.d/elpa/org-mode/lisp/)]

[BUG] Writing table to a folded drawer unfolds it. [9.7.12 (release_9.7.12 @ /home/natrys/.emacs.d/elpa/org-mode/lisp/)]

[Imran Khan]

Also any attempt to fold the drawer immediately afterwards (in same
elisp thread of execution) does nothing.

It seems to be specific to tables in my testing because writing anything
else doesn't cause the drawer to unfold like that.

For setup, given this simple org-mode file (test1.org):

```
* Headline
:DRAWER:
00000000
:END:

Content.
```

And a function that writes a table to current drawer at point:

```
(defun my-edit-drawer ()
  (let* ((element (org-element-at-point))
         (begin (org-element-property :contents-begin element))
         (end (org-element-property :contents-end element)))
    (delete-region begin end)
    (goto-char begin)
    
    (insert "| time |\n")
    (insert "|-+|\n")
    (insert "| " (format "%f" (time-to-seconds)) " |\n")

    ;; problem happens without this align call too
    ;; so this is just for good visual
    (org-table-align)))
```

If we edit the test1.org with elisp:

```
(with-current-buffer "test1.org"
  ;; reset visibility
  (org-show-all)

  ;; make sure all the drawers are folded at first
  (org-fold-hide-drawer-all)

  (goto-char (point-min))
  (search-forward ":DRAWER:")
  (beginning-of-line)

  ;; At the drawer now, let's write a table to it
  (when (org-at-drawer-p) (my-edit-drawer))

  ;; Once done with writing, really make sure drawer is folded again
  (org-fold-hide-drawer-all))
```

Not only had the originally folded drawer been unfolded, but that last
`org-fold-hide-drawer-all` call that should have folded it back again,
doesn't (but it works if I independently call it afterwards).

----

For some extra context, I arrived here because this causes really
annoying visual glitches in org-fc (a package for flashcards and spaced
repetition in org-mode). In org-fc, you can mark certain headlines as
questions for flashcard with answers below, then org-fc lets you review
those, and your performance data is stored in a table inside a drawer
under that question.

Let's say you have a top level headline, and then a few questions below
it (test2.org):

```
* Subject

** Question 1
:DRAWER:
0000
:END:

Content.

** Question 1
:DRAWER:
0000
:END:

Content.
```

I am simplifying this but when you are reviewing a question, org-fc lets
you see ancestor headlines (e.g. subject here) for extra context (but
none of the siblings) with: `(org-fold-show-set-visibility 'ancestors)`.

Let's say you just reviewed Question 1 (which wrote to table in its
drawer), and moved on to Question 2:

```
(with-current-buffer "test2.org"
  ;; reset visibility
  (org-show-all)

  ;; make sure all the drawers are folded
  (org-fold-hide-drawer-all)

  (goto-char (point-min))
  (search-forward ":DRAWER:")
  (beginning-of-line)

  ;; we review Question 1 and write to its drawer
  (when (org-at-drawer-p) (my-edit-drawer))

  ;; really make sure drawer is folded after writing
  (org-fold-hide-drawer-all)

  ;;; and now we want to review Question 2

  ;; but first fold everything
  (while (org-up-heading-safe))
  (org-fold-subtree t)

  ;; now only show Question 2 with ancestor context
  (search-forward "Question 2")
  (org-fold-show-set-visibility 'ancestors)
  (org-fold-show-subtree))
```

This should show Question 2, with only Subject headline above.

But this issue causes the table in Question 1's drawer somehow bleed
into the view:

```
* Top Level...
|              time |
|-------------------|
| 1728928902.770990 |
:END:...
** Question 2
:DRAWER:...

Content.
```

----

Anyway, I am using Emacs from master branch, and latest release of
org-mode (9.7.12). But I have checked that I can reproduce this with
`emacs -Q` which loads bundled org-mode (9.7.11) with no other custom
setting.

But I can't reproduce this on Emacs 29.4 with its bundled org-mode (9.6.15).
I am presuming this is an org-mode issue, rather than overlay rendering
problem or something else in Emacs itself, but I am not sure.

----

Emacs  : GNU Emacs 31.0.50 (build 1, x86_64-unknown-linux-gnu, cairo version 1.18.0) of 2024-10-14
Package: Org mode version 9.7.12 (release_9.7.12 @ /home/natyrs/.emacs.d/elpa/org-mode/lisp/)

Re: [BUG] Writing table to a folded drawer unfolds it. [9.7.12 (release_9.7.12 @ /home/natrys/.emacs.d/elpa/org-mode/lisp/)]

[Ihor Radchenko]

Imran Khan <imran@khan.ovh> writes:

> ... Writing table to a folded drawer unfolds it
> ...
>     (insert "| " (format "%f" (time-to-seconds)) " |\n")

This is expected.
In the middle of the editing, you are breaking the drawer structure,
transiently having something like

:DRAWER:
...
| :END:

So, Org mode unfolds the drawer to avoid having a fold that cannot be
unfolded by normal means.

> Also any attempt to fold the drawer immediately afterwards (in same
> elisp thread of execution) does nothing.

This one is a bit tricky.
The above unfolding does not happen immediately after the edit - we
cannot reliably do it from inside after-change-functions.  So, the
unfolding of drawers is delayed until `post-command-hook' is evaluated.

Since your attempt to fold the drawer happens _before_
`post-command-hook', your folding is effectively undone shortly after.

To work around the problem, you can use
`org-fold-core-ignore-fragility-checks' macro around your code modifying
the drawer contents; as long as you know that you are not actually
breaking the drawer structure.

P.S. The above is all quite tricky and should be eventually simplified
by using track-changes.el library (will be a part of Emacs 30). But not
yet, and not until we drop Emacs 29 support.

Wontfix.
Canceled.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] Writing table to a folded drawer unfolds it. [9.7.12 (release_9.7.12 @ /home/natrys/.emacs.d/elpa/org-mode/lisp/)]

[Imran Khan]

Ihor Radchenko <yantar92@posteo.net> writes:

> Imran Khan <imran@khan.ovh> writes:
>
>> ... Writing table to a folded drawer unfolds it
>> ...
>>     (insert "| " (format "%f" (time-to-seconds)) " |\n")
>
> This is expected.
> In the middle of the editing, you are breaking the drawer structure,
> transiently having something like
>
> :DRAWER:
> ...
> | :END:
>
> So, Org mode unfolds the drawer to avoid having a fold that cannot be
> unfolded by normal means.

Oh, not only does it make complete sense, but turns out that I had been
operating under the misassumption all this while that one `insert' call
amounts to one atomic change to the buffer (after concatenation of
arguments). I had to do a double take now when I tested that `(insert a
b)' is in fact two changes, oof. Shows the danger of such implicit
assumptions.

I can now also construct the table in a way that doesn't leave the
drawer structure broken at any point. Regardless, thanks for also
telling me about `org-fold-core-ignore-fragility-checks'.