[BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Sebastian Miele]

In an emacs -Q, create an Org buffer with the following contents:

--8<---------------cut here---------------start------------->8---
* AB
** C
--8<---------------cut here---------------end--------------->8---

Fold the subtree under the heading AB, so that only a single line is
displayed (ending in "...").  With point between A and B, hit
C-S-<backspace> (kill-whole-line).

Expected: The whole _visible_ line, i.e., the entire contents of the
buffer is erased.  Actual behavior: The line with heading C remains.

Contrast this with the same experiment, except that the point is at the
beginning of the line containing AB when hitting C-S-<backspace>.  Then
the expected behavior happens.  According to the source of
kill-whole-line, the intended effect indeed is to kill a whole _visible_
line.

The following patch to the Emacs sources fixes the issue:

diff --git a/lisp/simple.el b/lisp/simple.el
index abd587245fe..44221f3fc24 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -6649,9 +6649,7 @@ kill-whole-line
 			     (unless (bobp) (backward-char))
 			     (point))))
 	(t
-	 (save-excursion
-	   (kill-region (point) (progn (forward-visible-line 0) (point))))
-	 (kill-region (point)
+         (kill-region (save-excursion (forward-visible-line 0) (point))
 		      (progn (forward-visible-line arg) (point))))))
 
 (defun forward-visible-line (arg)

The reason for the issue probably is: Without the patch, the killing
happens in two stages.  The first kill-region kills from the beginning
of the line until after the A.  That kills the leading *.  That probably
somehow triggers Org visibility changes.  With the patch applied the
whole killing happens in one stage, probably without causing an
intermediate change of visibility.

I first reported this to bug-gnu-emacs@gnu.org (see
https://debbugs.gnu.org/65734).  However, Eli asks:

> I'm not sure I understand why this is deemed a problem in Emacs.
> Shouldn't Org redefine C-S-<backspace> if the default binding doesn't
> suit what happens in Org buffers?  Did you discuss this with Org
> developers?

Emacs  : GNU Emacs 29.1.50 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.38, cairo version 1.17.8)
 of 2023-09-04
Package: Org mode version 9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)

Re: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Sebastian Miele <iota@whxvd.name> writes:

> I first reported this to bug-gnu-emacs@gnu.org (see
> https://debbugs.gnu.org/65734).  However, Eli asks:
>
>> I'm not sure I understand why this is deemed a problem in Emacs.
>> Shouldn't Org redefine C-S-<backspace> if the default binding doesn't
>> suit what happens in Org buffers?  Did you discuss this with Org
>> developers?

Confirmed.

I am CCing debbugs as I'd like to clarify things to be in sync with
Emacs.


> In an emacs -Q, create an Org buffer with the following contents:
>
> --8<---------------cut here---------------start------------->8---
> * AB
> ** C
> --8<---------------cut here---------------end--------------->8---

This will produce

* AB...

> Fold the subtree under the heading AB, so that only a single line is
> displayed (ending in "...").  With point between A and B, hit
> C-S-<backspace> (kill-whole-line).
>
> Expected: The whole _visible_ line, i.e., the entire contents of the
> buffer is erased.  Actual behavior: The line with heading C remains.

This indeed happens because `kill-whole-line' deletes the line in two
steps: "* A" and then the rest.

The first deletion leaves

B<begin invisible>
** C<end invisible>

which drastically alters the outline structure and triggers or to
automatically unfold the subtree, leaving

B
** C

visible.
Then, `kill-whole-line' proceeds with the second part of the deletion
and deletes the now visible line, leading to the observed behaviour.

The first deletion would be an equivalent of deleting "(defun"

(defun foo ()...

in outline-mode and would make it hard to unfold the body, if such
single deletion where performed.

In Org mode, because of frequent user requests about accidental
deletions of hidden text, we try our best to protect deletions of
invisible folded outlines. Automatic unfolding is one of the ways to
attract user's attention to potential accidental edit.

> Contrast this with the same experiment, except that the point is at the
> beginning of the line containing AB when hitting C-S-<backspace>.  Then
> the expected behavior happens.  According to the source of
> kill-whole-line, the intended effect indeed is to kill a whole _visible_
> line.

Currently, Org mode, similar to Eli's suggestion re-binds `kill-line' to
Org's own version - `org-kill-line'. But not `kill-whole-line'.

We can certainly do the same for `kill-whole-line', but in our previous
discussion https://yhetil.org/emacs-devel/87tu8rq2l6.fsf@localhost/, you
asked to consider extending the built-in Emacs commands instead of
overriding them.

As I described in the above, Org needs more control over the behaviour of
`kill-line'/`kill-whole-line' when the visible line contains multiple
lines of hidden text - to protect accidental deletions.
A hook, where Org can intervene with a yes/no prompt, would be useful.
It would also make sense to group the two edits together via
`combine-after-change-calls', although a more universal way to know that
certain edits are a part of the same known command (even when called
non-interactively) would be useful.

In addition, `org-kill-line' acts specially in certain scenarios:

For
* Heading <point> text :tag1:tag2:

`org-kill-line' will keep and re-align ":tag1:tag2:":

* Heading <point>      :tag1:tag2:

It would be nice if we could express such behavior without overriding
the `kill-line' command.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> Cc: 65734@debbugs.gnu.org, emacs-orgmode@gnu.org
> From: Ihor Radchenko <yantar92@posteo.net>
> Date: Tue, 05 Sep 2023 10:29:20 +0000
> 
> As I described in the above, Org needs more control over the behaviour of
> `kill-line'/`kill-whole-line' when the visible line contains multiple
> lines of hidden text - to protect accidental deletions.
> A hook, where Org can intervene with a yes/no prompt, would be useful.
> It would also make sense to group the two edits together via
> `combine-after-change-calls', although a more universal way to know that
> certain edits are a part of the same known command (even when called
> non-interactively) would be useful.

The command kills in two parts for a good reason, which is explained
in the comments to the code.  So making a single group will not work,
I think, at least not in all situations.  And relying on after-change
hooks to fix this use case sounds too obscure and fragile to me.

Moreover, I don't think this is specific to Org: any mode that folds
or hides portions of text might hit the same problem.

So we could decide that this command needs to become smarter when the
visual line includes invisible text.  That is, improve the command
without making any Org-specific changes anywhere.  Patches to that
effect are welcome.

> In addition, `org-kill-line' acts specially in certain scenarios:
> 
> For
> * Heading <point> text :tag1:tag2:
> 
> `org-kill-line' will keep and re-align ":tag1:tag2:":
> 
> * Heading <point>      :tag1:tag2:
> 
> It would be nice if we could express such behavior without overriding
> the `kill-line' command.

This could be handled by a suitable extension to end-of-visible-line.
For example, introduce a new text property which end-of-visible-line
would then handle the same as it currently handles invisible text.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Max Nikulin]

On 05/09/2023 17:29, Ihor Radchenko wrote:
> Confirmed.

It is a regression in comparison to e.g. org-mode-9.3.1.

I noticed it for multiline plain list items

- ab
   cd

Only first line is removed by C-S-<backspace> when the item is folded by 
TAB.

For headings I usually use C-c C-x C-w, so I had less chance to face 
this issue.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Sebastian Miele]

> From: Eli Zaretskii <eliz@gnu.org>
> Date: Tue, 2023-09-05 14:54 +0300
>
> […]
>
> So we could decide that this command needs to become smarter when the
> visual line includes invisible text.  That is, improve the command
> without making any Org-specific changes anywhere.  Patches to that
> effect are welcome.

The following would do it.  I think I tested it rather thoroughly.
During testing I found another bug that is addressed by the let-binding
of kill-read-only-ok during the first kill-region below.

diff --git a/lisp/simple.el b/lisp/simple.el
index abd587245fe..d983cb85ab3 100644
--- a/lisp/simple.el
+++ b/lisp/simple.el
@@ -6631,28 +6631,50 @@ kill-whole-line
   (unless (eq last-command 'kill-region)
     (kill-new "")
     (setq last-command 'kill-region))
-  (cond ((zerop arg)
-	 ;; We need to kill in two steps, because the previous command
-	 ;; could have been a kill command, in which case the text
-	 ;; before point needs to be prepended to the current kill
-	 ;; ring entry and the text after point appended.  Also, we
-	 ;; need to use save-excursion to avoid copying the same text
-	 ;; twice to the kill ring in read-only buffers.
-	 (save-excursion
-	   (kill-region (point) (progn (forward-visible-line 0) (point))))
-	 (kill-region (point) (progn (end-of-visible-line) (point))))
-	((< arg 0)
-	 (save-excursion
-	   (kill-region (point) (progn (end-of-visible-line) (point))))
-	 (kill-region (point)
-		      (progn (forward-visible-line (1+ arg))
-			     (unless (bobp) (backward-char))
-			     (point))))
-	(t
-	 (save-excursion
-	   (kill-region (point) (progn (forward-visible-line 0) (point))))
-	 (kill-region (point)
-		      (progn (forward-visible-line arg) (point))))))
+  ;; - We need to kill in two steps, because the previous command
+  ;;   could have been a kill command, in which case the text before
+  ;;   point needs to be prepended to the current kill ring entry and
+  ;;   the text after point appended.
+  ;; - We need to be careful to avoid copying text twice to the kill
+  ;;   ring in read-only buffers.
+  ;; - We need to determine the boundaries of visible lines before we
+  ;;   do the first kill, because that may change visibility
+  ;;   (bug#65734).
+  (let ((regions-begin (point-marker))
+        region1-end)
+    (cond ((zerop arg)
+           (setq region1-end (save-excursion
+                               (forward-visible-line 0)
+                               (point-marker)))
+           (end-of-visible-line))
+	  ((< arg 0)
+	   (setq region1-end (save-excursion
+                               (end-of-visible-line)
+                               (point-marker)))
+           (forward-visible-line (1+ arg))
+	   (unless (bobp) (backward-char)))
+	  (t
+	   (setq region1-end (save-excursion
+                               (forward-visible-line 0)
+                               (point-marker)))
+	   (progn (forward-visible-line arg))))
+    ;; - Pass the marker positions and not the markers themselves.
+    ;;   kill-region determines whether to prepend or append to a
+    ;;   previous kill by checking the direction of the region.  But
+    ;;   it deletes the content and hence moves the markers before
+    ;;   that.  That effectively makes every region delimited by
+    ;;   markers an (empty) forward region.
+    ;; - Make the first kill-region emit a non-local exit only if the
+    ;;   second kill-region below would not operate on a non-empty
+    ;;   region.
+    (let ((kill-read-only-ok (or kill-read-only-ok
+                                 (/= regions-begin (point)))))
+      (kill-region (marker-position regions-begin)
+                   (marker-position region1-end)))
+    (kill-region (marker-position regions-begin)
+                 (point))
+    (set-marker regions-begin nil)
+    (set-marker region1-end nil)))
 
 (defun forward-visible-line (arg)
   "Move forward by ARG lines, ignoring currently invisible newlines only.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> Cc: emacs-orgmode@gnu.org, 65734@debbugs.gnu.org
> Date: Tue, 5 Sep 2023 21:30:58 +0700
> From: Max Nikulin <manikulin@gmail.com>
> 
> On 05/09/2023 17:29, Ihor Radchenko wrote:
> > Confirmed.
> 
> It is a regression in comparison to e.g. org-mode-9.3.1.

What changed since org-mode-9.3.1?  Was it some change in Emacs, and
if so, which one?

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Sebastian Miele <iota@whxvd.name>
> Cc: Ihor Radchenko <yantar92@posteo.net>, 65734@debbugs.gnu.org,
>  emacs-orgmode@gnu.org
> Date: Tue, 05 Sep 2023 17:25:38 +0200
> 
> > From: Eli Zaretskii <eliz@gnu.org>
> > Date: Tue, 2023-09-05 14:54 +0300
> >
> > […]
> >
> > So we could decide that this command needs to become smarter when the
> > visual line includes invisible text.  That is, improve the command
> > without making any Org-specific changes anywhere.  Patches to that
> > effect are welcome.
> 
> The following would do it.  I think I tested it rather thoroughly.
> During testing I found another bug that is addressed by the let-binding
> of kill-read-only-ok during the first kill-region below.

Thanks.  Sadly, we don't have any tests for this function in our test
suite, so verifying this non-trivial change will not be easy...

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> On 05/09/2023 17:29, Ihor Radchenko wrote:
>> > Confirmed.
>> 
>> It is a regression in comparison to e.g. org-mode-9.3.1.
>
> What changed since org-mode-9.3.1?  Was it some change in Emacs, and
> if so, which one?

The reported bug is a side effect of a feature when Org automatically
reveals hidden outlines that are "broken" due to edits and thus could
not be unfolded easily. For example, when destroying parent heading in a
folding subtree:

<point>* Heading...

   |
   V

 Heading
<unfolded>

The feature was introduced in Or 9.5.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Max Nikulin]

On 05/09/2023 22:50, Ihor Radchenko wrote:
> Eli Zaretskii writes:
> 
>>> On 05/09/2023 17:29, Ihor Radchenko wrote:
>>>> Confirmed.
>>>
>>> It is a regression in comparison to e.g. org-mode-9.3.1.
>>
>> What changed since org-mode-9.3.1?  Was it some change in Emacs, and
>> if so, which one?

That was a comparison on emacs-26.3

> The reported bug is a side effect of a feature when Org automatically
> reveals hidden outlines that are "broken" due to edits and thus could
> not be unfolded easily. For example, when destroying parent heading in a
> folding subtree:
> 
> <point>* Heading...
> 
>     |
>     V
> 
>   Heading
> <unfolded>
> 
> The feature was introduced in Or 9.5.

Or in 9.6? On Emacs-28.2 I have compared Org-9.5.5 and 9.6.8. Unfolding 
on removing of a heading marker does not work in 9.5.5, but collapsed 
list items and headers are removed correctly.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> The feature was introduced in Or 9.5.
>
> Or in 9.6?

Oops. Yup. Org 9.6 - the latest release. Together with org-fold library.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Max Nikulin <manikulin@gmail.com>, iota@whxvd.name,
>  65734@debbugs.gnu.org, emacs-orgmode@gnu.org
> Date: Tue, 05 Sep 2023 15:50:58 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> On 05/09/2023 17:29, Ihor Radchenko wrote:
> >> > Confirmed.
> >> 
> >> It is a regression in comparison to e.g. org-mode-9.3.1.
> >
> > What changed since org-mode-9.3.1?  Was it some change in Emacs, and
> > if so, which one?
> 
> The reported bug is a side effect of a feature when Org automatically
> reveals hidden outlines that are "broken" due to edits and thus could
> not be unfolded easily. For example, when destroying parent heading in a
> folding subtree:
> 
> <point>* Heading...
> 
>    |
>    V
> 
>  Heading
> <unfolded>
> 
> The feature was introduced in Or 9.5.

Understood, thanks.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> The following would do it.  I think I tested it rather thoroughly.
>> During testing I found another bug that is addressed by the let-binding
>> of kill-read-only-ok during the first kill-region below.
>
> Thanks.  Sadly, we don't have any tests for this function in our test
> suite, so verifying this non-trivial change will not be easy...

Then, what should we do to move things forward? I guess the first step
will be writing these missing tests. Anything else?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> It would also make sense to group the two edits together via
>> `combine-after-change-calls', although a more universal way to know that
>> certain edits are a part of the same known command (even when called
>> non-interactively) would be useful.
>
> The command kills in two parts for a good reason, which is explained
> in the comments to the code.  So making a single group will not work,
> I think, at least not in all situations.

I think there is misunderstanding. `combine-after-change-calls' will not
affect the two-step modification of the kill ring, if we put it around
`kill-whole-line'. Or do I miss something?

> ...  And relying on after-change
> hooks to fix this use case sounds too obscure and fragile to me.

Indeed. I did not talk about this particular bug report. What I meant is
some way to group change hooks executed by the same function/command.

>> In addition, `org-kill-line' acts specially in certain scenarios:
>> 
>> For
>> * Heading <point> text :tag1:tag2:
>> 
>> `org-kill-line' will keep and re-align ":tag1:tag2:":
>> 
>> * Heading <point>      :tag1:tag2:
>> 
>> It would be nice if we could express such behavior without overriding
>> the `kill-line' command.
>
> This could be handled by a suitable extension to end-of-visible-line.
> For example, introduce a new text property which end-of-visible-line
> would then handle the same as it currently handles invisible text.

I am not sure if I like the idea of text property - marking all the tags
in buffer with text property is expensive. What about something like
`end-of-visible-line-function'?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: Sebastian Miele <iota@whxvd.name>, 65734@debbugs.gnu.org,
>  emacs-orgmode@gnu.org
> Date: Wed, 06 Sep 2023 08:23:23 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The following would do it.  I think I tested it rather thoroughly.
> >> During testing I found another bug that is addressed by the let-binding
> >> of kill-read-only-ok during the first kill-region below.
> >
> > Thanks.  Sadly, we don't have any tests for this function in our test
> > suite, so verifying this non-trivial change will not be easy...
> 
> Then, what should we do to move things forward? I guess the first step
> will be writing these missing tests.

Yes, that'd be most welcome.

> Anything else?

How about asking on emacs-devel that people who use kill-whole-line
frequently install the patch and run with it for some time?  (We could
do that after installing the changes on master, of course.)

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: iota@whxvd.name, 65734@debbugs.gnu.org, emacs-orgmode@gnu.org
> Date: Wed, 06 Sep 2023 08:30:36 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> In addition, `org-kill-line' acts specially in certain scenarios:
> >> 
> >> For
> >> * Heading <point> text :tag1:tag2:
> >> 
> >> `org-kill-line' will keep and re-align ":tag1:tag2:":
> >> 
> >> * Heading <point>      :tag1:tag2:
> >> 
> >> It would be nice if we could express such behavior without overriding
> >> the `kill-line' command.
> >
> > This could be handled by a suitable extension to end-of-visible-line.
> > For example, introduce a new text property which end-of-visible-line
> > would then handle the same as it currently handles invisible text.
> 
> I am not sure if I like the idea of text property - marking all the tags
> in buffer with text property is expensive.

Then perhaps just a special value for buffer-invisibility-spec, or
some other simple variation of a property Org already uses?

> What about something like `end-of-visible-line-function'?

That is also a possibility, but it will then affect kill-line
_anywhere_ in the buffer, whereas a text property can have a more
localized effect.  Are you sure kill-line will need this customization
on the whole buffer?

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Sebastian Miele]

> From: Eli Zaretskii <eliz@gnu.org>
> Date: Wed, 2023-09-06 15:16 +0300
>
>> From: Ihor Radchenko <yantar92@posteo.net>
>> Date: Wed, 06 Sep 2023 08:23:23 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> The following would do it.  I think I tested it rather thoroughly.
>> >> During testing I found another bug that is addressed by the let-binding
>> >> of kill-read-only-ok during the first kill-region below.
>> >
>> > Thanks.  Sadly, we don't have any tests for this function in our test
>> > suite, so verifying this non-trivial change will not be easy...
>> 
>> Then, what should we do to move things forward? I guess the first step
>> will be writing these missing tests.
>
> Yes, that'd be most welcome.

I will write the tests.  And I will probably come up with an updated
version of the original patch.  There is at least one cosmetic change.
And something else that I want to have tried.  May take some time.

>> Anything else?
>
> How about asking on emacs-devel that people who use kill-whole-line
> frequently install the patch and run with it for some time?  (We could
> do that after installing the changes on master, of course.)

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Sebastian Miele]

> From: Ihor Radchenko <yantar92@posteo.net>
> Date: Wed, 2023-09-06 08:30 +0000
>
> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> It would also make sense to group the two edits together via
>>> `combine-after-change-calls', although a more universal way to know that
>>> certain edits are a part of the same known command (even when called
>>> non-interactively) would be useful.
>>
>> The command kills in two parts for a good reason, which is explained
>> in the comments to the code.  So making a single group will not work,
>> I think, at least not in all situations.
>
> I think there is misunderstanding. `combine-after-change-calls' will not
> affect the two-step modification of the kill ring, if we put it around
> `kill-whole-line'. Or do I miss something?

I tried to wrap the problematic portion of `kill-whole-line' into
`combine-after-change-calls'.  It seems to have no effect.  The
after-change function `org-fold-core--fix-folded-region' still gets
called twice, not fixing the bug.  I did not dig deeper, because the
stuff that makes `combine-after-change-calls' work at least partially
goes in C and seems to be scattered over several places.

The Emacs Lisp manual states that `combine-after-change-calls' "arranges
to call the after-change functions just once for a series of several
changes—if that seems safe."  So this case does not seem safe.  Apart
from that, there is no stated guarantee for when it would seem it safe.

I conclude that, although this path looked possibly elegant at first,
and I wanted to give it a try, this cannot work out.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> >> In addition, `org-kill-line' acts specially in certain scenarios:
>> >> 
>> >> For
>> >> * Heading <point> text :tag1:tag2:
>> >> 
>> >> `org-kill-line' will keep and re-align ":tag1:tag2:":
>> >> 
>> >> * Heading <point>      :tag1:tag2:
>> >> 
>> >> It would be nice if we could express such behavior without overriding
>> >> the `kill-line' command.
>> ...
>> I am not sure if I like the idea of text property - marking all the tags
>> in buffer with text property is expensive.
>
> Then perhaps just a special value for buffer-invisibility-spec, or
> some other simple variation of a property Org already uses?

We may have a misunderstanding here.
In "* Heading text :tag1:tag2:", everything is visible yet Org needs to
protect ":tag1:tag2: from being killed by `kill-line', but not from
`kill-whole-line'. Moreover, the behaviour also depends on the point
position - if point is inside ":tag1:tag2:", we fall back to the default
behaviour. And the whole "special" behaviour can also be switched off by
flipping `org-special-ctrl-k'.

Invisibility has nothing to do with this need.

>> What about something like `end-of-visible-line-function'?
>
> That is also a possibility, but it will then affect kill-line
> _anywhere_ in the buffer, whereas a text property can have a more
> localized effect.  Are you sure kill-line will need this customization
> on the whole buffer?

Applying text property is not free - (1) we need to do it across the
whole buffer, which scales poorly; (2) we need to keep it updated as the
buffer changes, which scales even worse. In addition, adding more text
properties slow down redisplay, which Org already strains to its limits.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Sebastian Miele <iota@whxvd.name> writes:

>> I think there is misunderstanding. `combine-after-change-calls' will not
>> affect the two-step modification of the kill ring, if we put it around
>> `kill-whole-line'. Or do I miss something?
>
> I tried to wrap the problematic portion of `kill-whole-line' into
> `combine-after-change-calls'.  It seems to have no effect.  The
> after-change function `org-fold-core--fix-folded-region' still gets
> called twice, not fixing the bug.  I did not dig deeper, because the
> stuff that makes `combine-after-change-calls' work at least partially
> goes in C and seems to be scattered over several places.

Oops. Of course, I meant `combine-change-calls'.
`combine-after-change-calls' does not always have effect. In particular,
Org sets `before-change-functions' effectively disabling the macro you tried.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: iota@whxvd.name, 65734@debbugs.gnu.org, emacs-orgmode@gnu.org
> Date: Thu, 07 Sep 2023 10:00:49 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Then perhaps just a special value for buffer-invisibility-spec, or
> > some other simple variation of a property Org already uses?
> 
> We may have a misunderstanding here.
> In "* Heading text :tag1:tag2:", everything is visible yet Org needs to
> protect ":tag1:tag2: from being killed by `kill-line', but not from
> `kill-whole-line'. Moreover, the behaviour also depends on the point
> position - if point is inside ":tag1:tag2:", we fall back to the default
> behaviour. And the whole "special" behaviour can also be switched off by
> flipping `org-special-ctrl-k'.
> 
> Invisibility has nothing to do with this need.

Isn't it true that invisibility is what causes the user expectations
in this case to begin with?  Then saying that "invisibility has
nothing to do with this" is not really accurate, is it?

> >> What about something like `end-of-visible-line-function'?
> >
> > That is also a possibility, but it will then affect kill-line
> > _anywhere_ in the buffer, whereas a text property can have a more
> > localized effect.  Are you sure kill-line will need this customization
> > on the whole buffer?
> 
> Applying text property is not free - (1) we need to do it across the
> whole buffer, which scales poorly; (2) we need to keep it updated as the
> buffer changes, which scales even worse. In addition, adding more text
> properties slow down redisplay, which Org already strains to its limits.

I understand, but in my book correctness tramps performance, and I was
trying to make the point that perhaps modifying the behavior of
kill-line everywhere in Org buffers might be incorrect for some cases.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Sebastian Miele]

> From: Eli Zaretskii <eliz@gnu.org>
> Date: Thu, 2023-09-07 13:19 +0300
>
>> From: Ihor Radchenko <yantar92@posteo.net>
>> Cc: iota@whxvd.name, 65734@debbugs.gnu.org, emacs-orgmode@gnu.org
>> Date: Thu, 07 Sep 2023 10:00:49 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Then perhaps just a special value for buffer-invisibility-spec, or
>> > some other simple variation of a property Org already uses?
>> 
>> We may have a misunderstanding here.
>> In "* Heading text :tag1:tag2:", everything is visible yet Org needs to
>> protect ":tag1:tag2: from being killed by `kill-line', but not from
>> `kill-whole-line'. Moreover, the behaviour also depends on the point
>> position - if point is inside ":tag1:tag2:", we fall back to the default
>> behaviour. And the whole "special" behaviour can also be switched off by
>> flipping `org-special-ctrl-k'.
>> 
>> Invisibility has nothing to do with this need.
>
> Isn't it true that invisibility is what causes the user expectations
> in this case to begin with?  Then saying that "invisibility has
> nothing to do with this" is not really accurate, is it?

I am not 100 % sure what exactly the misunderstanding is.  My first
guess would be: You assume that the tags are invisible, too.  But that
is not the case.  Because of that the special handling of tags by
`org-kill-line' has nothing to do with visibility.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> We may have a misunderstanding here.
>> In "* Heading text :tag1:tag2:", everything is visible yet Org needs to
>> protect ":tag1:tag2: from being killed by `kill-line', but not from
>> `kill-whole-line'. Moreover, the behaviour also depends on the point
>> position - if point is inside ":tag1:tag2:", we fall back to the default
>> behaviour. And the whole "special" behaviour can also be switched off by
>> flipping `org-special-ctrl-k'.
>> 
>> Invisibility has nothing to do with this need.
>
> Isn't it true that invisibility is what causes the user expectations
> in this case to begin with?  Then saying that "invisibility has
> nothing to do with this" is not really accurate, is it?

There are two things I raised in the previous messages:
1. The specific bug reported, related to invisibility changes in
   after-change-functions
2. A request to extend `kill-whole-line' and `kill-line' to cater Org
   needs:
   - for invisible text in folded headings
   - for visible text, when `kill-line' is called on a heading

In this branch of the discussion, I am describing a request to deal with
visible text.

Hope I made things more clear now.

>> >> What about something like `end-of-visible-line-function'?
>> >
>> > That is also a possibility, but it will then affect kill-line
>> > _anywhere_ in the buffer, whereas a text property can have a more
>> > localized effect.  Are you sure kill-line will need this customization
>> > on the whole buffer?
>> 
>> Applying text property is not free - (1) we need to do it across the
>> whole buffer, which scales poorly; (2) we need to keep it updated as the
>> buffer changes, which scales even worse. In addition, adding more text
>> properties slow down redisplay, which Org already strains to its limits.
>
> I understand, but in my book correctness tramps performance, and I was
> trying to make the point that perhaps modifying the behavior of
> kill-line everywhere in Org buffers might be incorrect for some cases.

I think that `end-of-visible-line-function' might be more appropriate -
it does not reduce correctness (we can simply fall back to the default
behaviour when needed) and less problematic performance-wise, as I
described.

Or maybe some other idea. But not text properties - they are
problematic performance-wise. In addition, `org-kill-line' is sensitive
to point position (do different things at bol, before ":tag:...", and
inside it). I am not sure how to make things depend on point position
with text properties only.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Sebastian Miele <iota@whxvd.name> writes:

>>> Then, what should we do to move things forward? I guess the first step
>>> will be writing these missing tests.
>>
>> Yes, that'd be most welcome.
>
> I will write the tests.  And I will probably come up with an updated
> version of the original patch.  There is at least one cosmetic change.
> And something else that I want to have tried.  May take some time.

Thanks in advance!
Feel free to ask us anything if you need any help.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Stefan Monnier]

> The reported bug is a side effect of a feature when Org automatically
> reveals hidden outlines that are "broken" due to edits and thus could
> not be unfolded easily. For example, when destroying parent heading in a
> folding subtree:

I'd be in favor of changing `kill-whole-line` to do the kill in a single
step rather than killing the "before" and "after" separately.
I understand why it doesn't do that, but I'm not sure it's worth the
trouble (or we should change the `kill-region` thingy to be more
robust, e.g. record the position of the last kill so that it doesn't
need to rely on (< end beg) to guess whether to append or prepend and
it can automatically notice when the new kill is *around* the previous
one).

But in addition to that, I suspect that Org should probably not modify
visibility directly from the modification hooks.  Instead, its
modification hook function should just stash the info somewhere and then
update the visibility later on, such as in a `post-command-hook`, timer,
`pre-redisplay-functions`, younameit.

As a rule of thumb, I think modification hooks should be treated a bit
like POSIX signal handlers: just record the event somewhere but don't do
any substantial work in there.


        Stefan

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> But in addition to that, I suspect that Org should probably not modify
> visibility directly from the modification hooks.  Instead, its
> modification hook function should just stash the info somewhere and then
> update the visibility later on, such as in a `post-command-hook`, timer,
> `pre-redisplay-functions`, younameit.

Good idea. At least, for this specific feature in Org mode.

> As a rule of thumb, I think modification hooks should be treated a bit
> like POSIX signal handlers: just record the event somewhere but don't do
> any substantial work in there.

Yet, it is sometimes necessary to modify text right inside the
modification hooks. Otherwise, it is very hard (and sometimes
impossible) to keep track of the original text region when multiple
modifications happen there one by one.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Stefan Monnier]

>> As a rule of thumb, I think modification hooks should be treated a bit
>> like POSIX signal handlers: just record the event somewhere but don't do
>> any substantial work in there.
> Yet, it is sometimes necessary to modify text right inside the
> modification hooks. Otherwise, it is very hard (and sometimes
> impossible) to keep track of the original text region when multiple
> modifications happen there one by one.

It's always "possible", but yes there can be undesirable effects:
usually as you delay the updates you're forced to reduce the granularity
of the recorded changes, so in you may end up treating a series of small
changes as one big-ass change for which your updates may not be able to
do as good a job.

E.g. in `diff-mode` we try to allow editing the patch, and dynamically
update the hunk headers accordingly.  This works well for an edit within
a single hunk but doing it for larger edits is somewhere between hard
and undesirable so we don't do it.

If a command changes just two lines but in two different hunks, our code
treats it a a single change that spans two hunks and thus refrains from
updating the hunk headers, as if the command had actually deleted and
then reinserted all the text between those two lines.

That's why I said "rule of thumb": there can be tradeoffs.
In practice 99% of Emacs commands modify only a single contiguous chunk
of text, so the tradeoff comes into play fairly rarely.


        Stefan

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> That's why I said "rule of thumb": there can be tradeoffs.
> In practice 99% of Emacs commands modify only a single contiguous chunk
> of text, so the tradeoff comes into play fairly rarely.

It would be nice if this advice were added to the relevant docstrings.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> But in addition to that, I suspect that Org should probably not modify
>> visibility directly from the modification hooks.  Instead, its
>> modification hook function should just stash the info somewhere and then
>> update the visibility later on, such as in a `post-command-hook`, timer,
>> `pre-redisplay-functions`, younameit.
>
> Good idea. At least, for this specific feature in Org mode.

Now, done.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f9702a09e

Although, I am still interested to pursue feature request to allow
customizing `kill-whole-line' and `kill-line' by major modes.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Sebastian Miele <iota@whxvd.name> writes:

> In an emacs -Q, create an Org buffer with the following contents:
>
> --8<---------------cut here---------------start------------->8---
> * AB
> ** C
> --8<---------------cut here---------------end--------------->8---
>
> Fold the subtree under the heading AB, so that only a single line is
> displayed (ending in "...").  With point between A and B, hit
> C-S-<backspace> (kill-whole-line).
>
> Expected: The whole _visible_ line, i.e., the entire contents of the
> buffer is erased.  Actual behavior: The line with heading C remains.

Fixed, on main. This is a workaround, but it was suggested by
Stefan (Emacs maintained) that changing visibility in the modification
hooks might be unsafe in general.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f9702a09e

For unrelated reasons, `kill-whole-line' in Emacs will also be changed
to not use multi-step edit.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Stefan Monnier]

>>> But in addition to that, I suspect that Org should probably not modify
>>> visibility directly from the modification hooks.  Instead, its
>>> modification hook function should just stash the info somewhere and then
>>> update the visibility later on, such as in a `post-command-hook`, timer,
>>> `pre-redisplay-functions`, younameit.
>> Good idea. At least, for this specific feature in Org mode.
> Now, done.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=f9702a09e

Great, thanks!

> Although, I am still interested to pursue feature request to allow
> customizing `kill-whole-line' and `kill-line' by major modes.

[ I'm sorry I didn't pay very much attention to this part.
  Naively, I'd suggest you use `remap` bindings for that, but I'm sure
  you're aware of that option, so you presumably have good reasons to
  want something else.  ]


        Stefan

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Stefan Monnier]

>> That's why I said "rule of thumb": there can be tradeoffs.
>> In practice 99% of Emacs commands modify only a single contiguous chunk
>> of text, so the tradeoff comes into play fairly rarely.
> It would be nice if this advice were added to the relevant docstrings.

Not sure exactly which advice you're referring to (the text you quoted
above is misleading: it would encourage the naive reader to just do the
work in `after-change-functions` since it's called only once per command
anyway, which they already instinctively do and which is exactly what we
don't want), nor which docstring you're referring to.


        Stefan

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> It would be nice if this advice were added to the relevant docstrings.
>
> Not sure exactly which advice you're referring to (the text you quoted
> above is misleading: it would encourage the naive reader to just do the
> work in `after-change-functions` since it's called only once per command
> anyway, which they already instinctively do and which is exactly what we
> don't want), nor which docstring you're referring to.

I was mostly referring to the docstring of `after-change-functions'.
Maybe also
`modification-hooks'\‘insert-in-front-hooks’\‘insert-behind-hooks’ text
properties.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Although, I am still interested to pursue feature request to allow
>> customizing `kill-whole-line' and `kill-line' by major modes.
>
> [ I'm sorry I didn't pay very much attention to this part.
>   Naively, I'd suggest you use `remap` bindings for that, but I'm sure
>   you're aware of that option, so you presumably have good reasons to
>   want something else.  ]

Eli told me in the past that Org mode should not remap keys.
See the discussion branch starting at
https://yhetil.org/emacs-devel/8735gfs3is.fsf@localhost/

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Stefan Monnier]

> Eli told me in the past that Org mode should not remap keys.
> See the discussion branch starting at
> https://yhetil.org/emacs-devel/8735gfs3is.fsf@localhost/

Then I'll let Eli take care of this :-)


        Stefan

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: emacs-orgmode@gnu.org, Eli Zaretskii <eliz@gnu.org>,
>  65734@debbugs.gnu.org, Max Nikulin <manikulin@gmail.com>, iota@whxvd.name
> Date: Tue, 09 Jan 2024 22:33:58 +0000
> 
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> >> Although, I am still interested to pursue feature request to allow
> >> customizing `kill-whole-line' and `kill-line' by major modes.
> >
> > [ I'm sorry I didn't pay very much attention to this part.
> >   Naively, I'd suggest you use `remap` bindings for that, but I'm sure
> >   you're aware of that option, so you presumably have good reasons to
> >   want something else.  ]
> 
> Eli told me in the past that Org mode should not remap keys.
> See the discussion branch starting at
> https://yhetil.org/emacs-devel/8735gfs3is.fsf@localhost/

I said that remapping widely-used keys to commands that behave
significantly differently places a non-trivial burden on users,
especially on those who use the remapping mode relatively rarely.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Eli told me in the past that Org mode should not remap keys.
>> See the discussion branch starting at
>> https://yhetil.org/emacs-devel/8735gfs3is.fsf@localhost/
>
> I said that remapping widely-used keys to commands that behave
> significantly differently places a non-trivial burden on users,
> especially on those who use the remapping mode relatively rarely.

Sure. From which I concluded that Org mode should avoid remapping and
instead prefer other means to adjust the behaviour of standard Emacs
commands.

Then, we discussed that Emacs commands to not always provide enough
toggles. So, I am asking to add one in this thread.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, emacs-orgmode@gnu.org, 65734@debbugs.gnu.org,
>  manikulin@gmail.com, iota@whxvd.name
> Date: Wed, 10 Jan 2024 13:05:19 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I said that remapping widely-used keys to commands that behave
> > significantly differently places a non-trivial burden on users,
> > especially on those who use the remapping mode relatively rarely.
> 
> Sure. From which I concluded that Org mode should avoid remapping and
> instead prefer other means to adjust the behaviour of standard Emacs
> commands.
> 
> Then, we discussed that Emacs commands to not always provide enough
> toggles. So, I am asking to add one in this thread.

I don't think I understand what kind of toggle are we talking about.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Stefan Monnier]

>> I said that remapping widely-used keys to commands that behave
>> significantly differently places a non-trivial burden on users,
>> especially on those who use the remapping mode relatively rarely.
>
> Sure. From which I concluded that Org mode should avoid remapping

I don't think that's what it means.  It means that it depends on what is
the end-user-visible effect.  If the remapped version of the command
"does the same" conceptually, then it's OK (even more so if the
non-remapped version of the command ends up misbehaving).

I think the main question is: can we expect that some users out there
will want to use the non-remapped version of the command because they
prefer its behavior?

So, when the alternative is to directly modify the behavior of the
non-remapped command, remapping is always acceptable.
[ But sometimes, remapping just doesn't do what we want, because the
  command is also used as a function from other places.  ]


        Stefan

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-orgmode@gnu.org,
>   65734@debbugs.gnu.org,  manikulin@gmail.com,  iota@whxvd.name
> Date: Wed, 10 Jan 2024 11:26:08 -0500
> 
> >> I said that remapping widely-used keys to commands that behave
> >> significantly differently places a non-trivial burden on users,
> >> especially on those who use the remapping mode relatively rarely.
> >
> > Sure. From which I concluded that Org mode should avoid remapping
> 
> I don't think that's what it means.  It means that it depends on what is
> the end-user-visible effect.  If the remapped version of the command
> "does the same" conceptually, then it's OK (even more so if the
> non-remapped version of the command ends up misbehaving).

Yes.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> I said that remapping widely-used keys to commands that behave
>>> significantly differently places a non-trivial burden on users,
>>> especially on those who use the remapping mode relatively rarely.
>>
>> Sure. From which I concluded that Org mode should avoid remapping
>
> I don't think that's what it means.  It means that it depends on what is
> the end-user-visible effect.  If the remapped version of the command
> "does the same" conceptually, then it's OK (even more so if the
> non-remapped version of the command ends up misbehaving).

Ok. I stand corrected.

> I think the main question is: can we expect that some users out there
> will want to use the non-remapped version of the command because they
> prefer its behavior?

We received no bug reports complaining about non-standard behaviour of
remapped functions AFAIK. So, I do not think that Org mode has problems
here.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Then, we discussed that Emacs commands to not always provide enough
>> toggles. So, I am asking to add one in this thread.
>
> I don't think I understand what kind of toggle are we talking about.

What I would like to request is a way to handle the following situation:

* Heading<... few hundreds of lines of invisible text>

If the user calls `kill-whole-line', a large part of the buffer will get
deleted. We had complains from the users about accidentally deleting a
lot of text in Org files in such situations.

So, I'd like some way to configure `kill-whole-line'/`kill-line' to warn
user about killing hidden text when we detect that we are deleting a
folded heading. Something like:

   (y-or-n-p "Kill hidden subtree along with headline? ")

I believe that it might be useful in other situations as well. Like in
outline-mode or outline-minor-mode.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, emacs-orgmode@gnu.org, 65734@debbugs.gnu.org,
>  manikulin@gmail.com, iota@whxvd.name
> Date: Thu, 11 Jan 2024 15:50:10 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Then, we discussed that Emacs commands to not always provide enough
> >> toggles. So, I am asking to add one in this thread.
> >
> > I don't think I understand what kind of toggle are we talking about.
> 
> What I would like to request is a way to handle the following situation:
> 
> * Heading<... few hundreds of lines of invisible text>
> 
> If the user calls `kill-whole-line', a large part of the buffer will get
> deleted. We had complains from the users about accidentally deleting a
> lot of text in Org files in such situations.
> 
> So, I'd like some way to configure `kill-whole-line'/`kill-line' to warn
> user about killing hidden text when we detect that we are deleting a
> folded heading. Something like:
> 
>    (y-or-n-p "Kill hidden subtree along with headline? ")
> 
> I believe that it might be useful in other situations as well. Like in
> outline-mode or outline-minor-mode.

How would kill-line know that it's about to kill a subtree?  All it
knows is that it is killing some invisible text.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> So, I'd like some way to configure `kill-whole-line'/`kill-line' to warn
>> user about killing hidden text when we detect that we are deleting a
>> folded heading. Something like:
>> 
>>    (y-or-n-p "Kill hidden subtree along with headline? ")
>> 
>> I believe that it might be useful in other situations as well. Like in
>> outline-mode or outline-minor-mode.
>
> How would kill-line know that it's about to kill a subtree?  All it
> knows is that it is killing some invisible text.

I imagine the following:

1. `kill-*-line' function will, by default, test if invisible text of
   length size is killed and query the user when called interactively.

2. Major modes could also set buffer-local `kill-line-query-function'
   that will return nil when killing should proceed without query or a
   string with query text.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, emacs-orgmode@gnu.org, 65734@debbugs.gnu.org,
>  manikulin@gmail.com, iota@whxvd.name
> Date: Thu, 11 Jan 2024 16:15:13 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> So, I'd like some way to configure `kill-whole-line'/`kill-line' to warn
> >> user about killing hidden text when we detect that we are deleting a
> >> folded heading. Something like:
> >> 
> >>    (y-or-n-p "Kill hidden subtree along with headline? ")
> >> 
> >> I believe that it might be useful in other situations as well. Like in
> >> outline-mode or outline-minor-mode.
> >
> > How would kill-line know that it's about to kill a subtree?  All it
> > knows is that it is killing some invisible text.
> 
> I imagine the following:
> 
> 1. `kill-*-line' function will, by default, test if invisible text of
>    length size is killed and query the user when called interactively.
> 
> 2. Major modes could also set buffer-local `kill-line-query-function'
>    that will return nil when killing should proceed without query or a
>    string with query text.

If the command is only sensitive to invisible text, it could warn
about so-and-so many invisible characters being killed, but it could
not warn about "subtrees", which is what you wanted.  Invisible text
in a buffer could have nothing to do with subtrees, even if the buffer
is under org-mode.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> > How would kill-line know that it's about to kill a subtree?  All it
>> > knows is that it is killing some invisible text.
>> 
>> I imagine the following:
>> 
>> 1. `kill-*-line' function will, by default, test if invisible text of
>>    length size is killed and query the user when called interactively.
>> 
>> 2. Major modes could also set buffer-local `kill-line-query-function'
>>    that will return nil when killing should proceed without query or a
>>    string with query text.
>
> If the command is only sensitive to invisible text, it could warn
> about so-and-so many invisible characters being killed, but it could
> not warn about "subtrees", which is what you wanted.  Invisible text
> in a buffer could have nothing to do with subtrees, even if the buffer
> is under org-mode.

Let me elaborate. In Elisp, I am thinking about something like:

(defvar-local kill-line-query-function #'kill-line-query-default)
(defun kill-line-query-default (beg end)
  (let ((nlines <count invisible lines between beg end>))
  (when (> nlines threshold)
    (format "Kill %d invisible lines? " nlines))))

Then, Org mode can instead have

(setq-local kill-line-query-function #'org-kill-line-query)
(defun org-kill-line-query (beg end)
  (org-with-point-at beg
    (when (and (org-at-heading-p)
               (progn
                 (end-of-line)
                 (and (< (point) end)
                      (org-fold-folded-p))))
       "Kill hidden subtree along with headline? ")))

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, emacs-orgmode@gnu.org, 65734@debbugs.gnu.org,
>  manikulin@gmail.com, iota@whxvd.name
> Date: Thu, 11 Jan 2024 18:08:11 +0000
> 
> Then, Org mode can instead have
> 
> (setq-local kill-line-query-function #'org-kill-line-query)
> (defun org-kill-line-query (beg end)
>   (org-with-point-at beg
>     (when (and (org-at-heading-p)
>                (progn
>                  (end-of-line)
>                  (and (< (point) end)
>                       (org-fold-folded-p))))
>        "Kill hidden subtree along with headline? ")))

I don't know what org-with-point-at and org-fold-folded-p do, but my
point is that you should consider the case when kill-line kills
invisible text that has nothing to do with Org's headings and trees,
so I suggest to either make the detection code smarter (so it could
distinguish between the two), or make the prompt text vaguer (to not
claim that the text must be a subtree).

And if that is still not clear or you disagree, let's leave it at
that.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Then, Org mode can instead have
>> 
>> (setq-local kill-line-query-function #'org-kill-line-query)
>> (defun org-kill-line-query (beg end)
>>   (org-with-point-at beg
>>     (when (and (org-at-heading-p)
>>                (progn
>>                  (end-of-line)
>>                  (and (< (point) end)
>>                       (org-fold-folded-p))))
>>        "Kill hidden subtree along with headline? ")))
>
> I don't know what org-with-point-at and org-fold-folded-p do, but my
> point is that you should consider the case when kill-line kills
> invisible text that has nothing to do with Org's headings and trees,
> so I suggest to either make the detection code smarter (so it could
> distinguish between the two), or make the prompt text vaguer (to not
> claim that the text must be a subtree).
>
> And if that is still not clear or you disagree, let's leave it at
> that.

I think that I see what you mean. What we can do to achieve this is
changing `kill-line-query-function' into abnormal hook
`kill-line-query-functions'. Then, `kill-line'/`kill-whole-line' will
use (run-hook-with-args-until-success 'kill-line-query-functions) to
decide whether to show a query.

Does it make sense?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, emacs-orgmode@gnu.org, 65734@debbugs.gnu.org,
>  manikulin@gmail.com, iota@whxvd.name
> Date: Fri, 12 Jan 2024 12:24:01 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Then, Org mode can instead have
> >> 
> >> (setq-local kill-line-query-function #'org-kill-line-query)
> >> (defun org-kill-line-query (beg end)
> >>   (org-with-point-at beg
> >>     (when (and (org-at-heading-p)
> >>                (progn
> >>                  (end-of-line)
> >>                  (and (< (point) end)
> >>                       (org-fold-folded-p))))
> >>        "Kill hidden subtree along with headline? ")))
> >
> > I don't know what org-with-point-at and org-fold-folded-p do, but my
> > point is that you should consider the case when kill-line kills
> > invisible text that has nothing to do with Org's headings and trees,
> > so I suggest to either make the detection code smarter (so it could
> > distinguish between the two), or make the prompt text vaguer (to not
> > claim that the text must be a subtree).
> >
> > And if that is still not clear or you disagree, let's leave it at
> > that.
> 
> I think that I see what you mean. What we can do to achieve this is
> changing `kill-line-query-function' into abnormal hook
> `kill-line-query-functions'. Then, `kill-line'/`kill-whole-line' will
> use (run-hook-with-args-until-success 'kill-line-query-functions) to
> decide whether to show a query.
> 
> Does it make sense?

It might, yes.  But do we have to _replace_ the hook? cannot we have
both?  I.e. if the new one is defined, call it, otherwise call the old
one.  That would be more backward-compatible, I think.

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> I think that I see what you mean. What we can do to achieve this is
>> changing `kill-line-query-function' into abnormal hook
>> `kill-line-query-functions'. Then, `kill-line'/`kill-whole-line' will
>> use (run-hook-with-args-until-success 'kill-line-query-functions) to
>> decide whether to show a query.
>> 
>> Does it make sense?
>
> It might, yes.  But do we have to _replace_ the hook? cannot we have
> both?  I.e. if the new one is defined, call it, otherwise call the old
> one.  That would be more backward-compatible, I think.

May you please elaborate what you mean by "old one"?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, emacs-orgmode@gnu.org, 65734@debbugs.gnu.org,
>  manikulin@gmail.com, iota@whxvd.name
> Date: Fri, 12 Jan 2024 12:39:18 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I think that I see what you mean. What we can do to achieve this is
> >> changing `kill-line-query-function' into abnormal hook
> >> `kill-line-query-functions'. Then, `kill-line'/`kill-whole-line' will
> >> use (run-hook-with-args-until-success 'kill-line-query-functions) to
> >> decide whether to show a query.
> >> 
> >> Does it make sense?
> >
> > It might, yes.  But do we have to _replace_ the hook? cannot we have
> > both?  I.e. if the new one is defined, call it, otherwise call the old
> > one.  That would be more backward-compatible, I think.
> 
> May you please elaborate what you mean by "old one"?

kill-line-query-function

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> May you please elaborate what you mean by "old one"?
>
> kill-line-query-function

There is no such variable in Emacs. I proposed to introduce a new
variable `kill-line-query-function' earlier in this thread. There is no
backwards-compatibility problem we need to solve here.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Stefan Monnier]

> (setq-local kill-line-query-function #'org-kill-line-query)

Please use `add-function` for such things.
That's its raison d'être.


        Stefan

Re: bug#65734: [BUG] kill-whole-line on folded subtrees [9.6.8 (release_9.6.8-3-g21171d @ /home/w/usr/emacs/0/29/0/lisp/org/)]

[Ihor Radchenko]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> (setq-local kill-line-query-function #'org-kill-line-query)
>
> Please use `add-function` for such things.
> That's its raison d'être.

When comparing imaginary kill-line-query-functions (abnormal hook) +
add-hook vs. kill-line-query-function + add-function, which one will be
more preferred?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>