Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

[Gustavo Barros]

Hi all,

When the capture template ends with a newline character and the capture 
process is canceled, the following heading gets demoted. And it 
shouldn’t.


Consider the following scenario. We have an agenda file with the 
following content:

#+name: ~/org/agenda.org
#+begin_src org
,* Capture

,* Following heading

#+end_src

We start ~emacs -Q~, and do some basic setup:

#+begin_src emacs-lisp
(package-initialize)
(global-set-key (kbd "C-c c") 'org-capture)
(setq org-agenda-files
      '("~/org/agenda.org"))
(setq org-refile-targets
      '((org-agenda-files :maxlevel . 2)))
(setq org-capture-templates
      '(("t" "TODO entry" entry
         (file+headline "~/org/agenda.org" "Capture")
         "* TODO %?\n")))
#+end_src

Now we start capture with "C-c c t" and cancel it with "C-c 
k". Examination of "agenda.org" will show:

#+name: ~/org/agenda.org
#+begin_src org
,* Capture
,** Following heading

#+end_src

Both the line between them is gone, and "* Following heading" got 
demoted, which is particularly unfortunate.

I’m not sure that I found a general rule in this respect.  I had met 
this effect before in my workflow and vaguely associated it with a 
trailing "\n".  Investigating it further, I could find a recipe to 
reproduce the effect, as shown, but I don’t really know how general it 
is.

This effect does not occur if "\n" is removed from the template, and I 
don’t know if its inclusion is to be considered bad practice, 
particularly as the capture templates already have a structure to handle 
empty lines.  But, if my memory does not betray me in this respect, one 
will find plenty of those laying around in folks configs.


Best regards,
Gustavo Barros.



Emacs  : GNU Emacs 26.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 
3.22.30)
 of 2019-04-19
Package: Org mode version 9.2.4 (9.2.4-11-g1c3eae-elpaplus @ 
/home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)

current state:
==============
(setq
 org-src-mode-hook '(org-src-babel-configure-edit-buffer
		     org-src-mode-configure-edit-buffer)
 org-metadown-hook '(org-babel-pop-to-session-maybe)
 org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
 org-refile-targets '((org-agenda-files :maxlevel . 2))
 org-agenda-files '("~/org/agenda.org")
 org-mode-hook '(#[0 "\300\301\302\303\304$\207"
		   [add-hook change-major-mode-hook org-show-all append 
		   local]
		   5]
		 #[0 "\300\301\302\303\304$\207"
		   [add-hook change-major-mode-hook 
		   org-babel-show-result-all
		    append local]
		   5]
		 org-babel-result-hide-spec org-babel-hide-all-hashes
		 org-eldoc-load)
 org-archive-hook '(org-attach-archive-delete-maybe)
 org-confirm-elisp-link-function 'yes-or-no-p
 org-agenda-before-write-hook '(org-agenda-add-entry-text)
 org-metaup-hook '(org-babel-load-in-session-maybe)
 org-bibtex-headline-format-function #[257 "\300\236A\207" [:title] 3 
 "\n\n(fn ENTRY)"]
 org-babel-pre-tangle-hook '(save-buffer)
 org-tab-first-hook '(org-babel-hide-result-toggle-maybe
		      org-babel-header-arg-expand)
 org-src-lang-modes '(("arduino" . arduino) ("redis" . redis) ("php" 
 . php)
		      ("C" . c) ("C++" . c++) ("asymptote" . asy)
		      ("bash" . sh) ("beamer" . latex) ("calc" 
		      . fundamental)
		      ("cpp" . c++) ("ditaa" . artist) ("dot" 
		      . fundamental)
		      ("elisp" . emacs-lisp) ("ocaml" . tuareg)
		      ("screen" . shell-script) ("shell" . sh)
		      ("sqlite" . sql))
 org-occur-hook '(org-first-headline-recenter)
 org-cycle-hook '(org-cycle-hide-archived-subtrees 
 org-cycle-show-empty-lines
		  org-optimize-window-after-visibility-change)
 org-speed-command-hook '(org-speed-command-activate
			  org-babel-speed-command-activate)
 org-confirm-shell-link-function 'yes-or-no-p
 org-link-parameters '(("id" :follow org-id-open)
		       ("eww" :follow eww :store org-eww-store-link)
		       ("rmail" :follow org-rmail-open :store
			org-rmail-store-link)
		       ("mhe" :follow org-mhe-open :store 
		       org-mhe-store-link)
		       ("irc" :follow org-irc-visit :store 
		       org-irc-store-link
			:export org-irc-export)
		       ("info" :follow org-info-open :export 
		       org-info-export
			:store org-info-store-link)
		       ("gnus" :follow org-gnus-open :store
			org-gnus-store-link)
		       ("docview" :follow org-docview-open :export
			org-docview-export :store 
			org-docview-store-link)
		       ("bibtex" :follow org-bibtex-open :store
			org-bibtex-store-link)
		       ("bbdb" :follow org-bbdb-open :export 
		       org-bbdb-export
			:complete org-bbdb-complete-link :store
			org-bbdb-store-link)
		       ("w3m" :store org-w3m-store-link) ("file+sys")
		       ("file+emacs")
		       ("elfeed" :follow elfeed-link-open :store
			elfeed-link-store-link)
		       ("doi" :follow org--open-doi-link)
		       ("elisp" :follow org--open-elisp-link)
		       ("file" :complete org-file-complete-link)
		       ("ftp" :follow
			(lambda (path) (browse-url (concat "ftp:" 
			path))))
		       ("help" :follow org--open-help-link)
		       ("http" :follow
			(lambda (path) (browse-url (concat "http:" 
			path))))
		       ("https" :follow
			(lambda (path) (browse-url (concat "https:" 
			path))))
		       ("mailto" :follow
			(lambda (path) (browse-url (concat "mailto:" 
			path))))
		       ("news" :follow
			(lambda (path) (browse-url (concat "news:" 
			path))))
		       ("shell" :follow org--open-shell-link))
 org-capture-templates '(("t" "TODO entry" entry
			  (file+headline "~/org/agenda.org" "Capture")
			  "* TODO %?\n")
			 )
 )

Re: Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

[Carsten Dominik]

[-- Attachment #1: Type: text/plain, Size: 7347 bytes --]

Hi Gustavo,

I tried to reproduce your example, and things worked properly....

Carsten

On Sun, Jul 28, 2019 at 9:34 PM Gustavo Barros <gusbrs.2016@gmail.com>
wrote:

> Hi all,
>
> When the capture template ends with a newline character and the capture
> process is canceled, the following heading gets demoted. And it
> shouldn’t.
>
>
> Consider the following scenario. We have an agenda file with the
> following content:
>
> #+name: ~/org/agenda.org
> #+begin_src org
> ,* Capture
>
> ,* Following heading
>
> #+end_src
>
> We start ~emacs -Q~, and do some basic setup:
>
> #+begin_src emacs-lisp
> (package-initialize)
> (global-set-key (kbd "C-c c") 'org-capture)
> (setq org-agenda-files
>       '("~/org/agenda.org"))
> (setq org-refile-targets
>       '((org-agenda-files :maxlevel . 2)))
> (setq org-capture-templates
>       '(("t" "TODO entry" entry
>          (file+headline "~/org/agenda.org" "Capture")
>          "* TODO %?\n")))
> #+end_src
>
> Now we start capture with "C-c c t" and cancel it with "C-c
> k". Examination of "agenda.org" will show:
>
> #+name: ~/org/agenda.org
> #+begin_src org
> ,* Capture
> ,** Following heading
>
> #+end_src
>
> Both the line between them is gone, and "* Following heading" got
> demoted, which is particularly unfortunate.
>
> I’m not sure that I found a general rule in this respect.  I had met
> this effect before in my workflow and vaguely associated it with a
> trailing "\n".  Investigating it further, I could find a recipe to
> reproduce the effect, as shown, but I don’t really know how general it
> is.
>
> This effect does not occur if "\n" is removed from the template, and I
> don’t know if its inclusion is to be considered bad practice,
> particularly as the capture templates already have a structure to handle
> empty lines.  But, if my memory does not betray me in this respect, one
> will find plenty of those laying around in folks configs.
>
>
> Best regards,
> Gustavo Barros.
>
>
>
> Emacs  : GNU Emacs 26.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version
> 3.22.30)
>  of 2019-04-19
> Package: Org mode version 9.2.4 (9.2.4-11-g1c3eae-elpaplus @
> /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)
>
> current state:
> ==============
> (setq
>  org-src-mode-hook '(org-src-babel-configure-edit-buffer
>                      org-src-mode-configure-edit-buffer)
>  org-metadown-hook '(org-babel-pop-to-session-maybe)
>  org-clock-out-hook '(org-clock-remove-empty-clock-drawer)
>  org-refile-targets '((org-agenda-files :maxlevel . 2))
>  org-agenda-files '("~/org/agenda.org")
>  org-mode-hook '(#[0 "\300\301\302\303\304$\207"
>                    [add-hook change-major-mode-hook org-show-all append
>                    local]
>                    5]
>                  #[0 "\300\301\302\303\304$\207"
>                    [add-hook change-major-mode-hook
>                    org-babel-show-result-all
>                     append local]
>                    5]
>                  org-babel-result-hide-spec org-babel-hide-all-hashes
>                  org-eldoc-load)
>  org-archive-hook '(org-attach-archive-delete-maybe)
>  org-confirm-elisp-link-function 'yes-or-no-p
>  org-agenda-before-write-hook '(org-agenda-add-entry-text)
>  org-metaup-hook '(org-babel-load-in-session-maybe)
>  org-bibtex-headline-format-function #[257 "\300\236A\207" [:title] 3
>  "\n\n(fn ENTRY)"]
>  org-babel-pre-tangle-hook '(save-buffer)
>  org-tab-first-hook '(org-babel-hide-result-toggle-maybe
>                       org-babel-header-arg-expand)
>  org-src-lang-modes '(("arduino" . arduino) ("redis" . redis) ("php"
>  . php)
>                       ("C" . c) ("C++" . c++) ("asymptote" . asy)
>                       ("bash" . sh) ("beamer" . latex) ("calc"
>                       . fundamental)
>                       ("cpp" . c++) ("ditaa" . artist) ("dot"
>                       . fundamental)
>                       ("elisp" . emacs-lisp) ("ocaml" . tuareg)
>                       ("screen" . shell-script) ("shell" . sh)
>                       ("sqlite" . sql))
>  org-occur-hook '(org-first-headline-recenter)
>  org-cycle-hook '(org-cycle-hide-archived-subtrees
>  org-cycle-show-empty-lines
>                   org-optimize-window-after-visibility-change)
>  org-speed-command-hook '(org-speed-command-activate
>                           org-babel-speed-command-activate)
>  org-confirm-shell-link-function 'yes-or-no-p
>  org-link-parameters '(("id" :follow org-id-open)
>                        ("eww" :follow eww :store org-eww-store-link)
>                        ("rmail" :follow org-rmail-open :store
>                         org-rmail-store-link)
>                        ("mhe" :follow org-mhe-open :store
>                        org-mhe-store-link)
>                        ("irc" :follow org-irc-visit :store
>                        org-irc-store-link
>                         :export org-irc-export)
>                        ("info" :follow org-info-open :export
>                        org-info-export
>                         :store org-info-store-link)
>                        ("gnus" :follow org-gnus-open :store
>                         org-gnus-store-link)
>                        ("docview" :follow org-docview-open :export
>                         org-docview-export :store
>                         org-docview-store-link)
>                        ("bibtex" :follow org-bibtex-open :store
>                         org-bibtex-store-link)
>                        ("bbdb" :follow org-bbdb-open :export
>                        org-bbdb-export
>                         :complete org-bbdb-complete-link :store
>                         org-bbdb-store-link)
>                        ("w3m" :store org-w3m-store-link) ("file+sys")
>                        ("file+emacs")
>                        ("elfeed" :follow elfeed-link-open :store
>                         elfeed-link-store-link)
>                        ("doi" :follow org--open-doi-link)
>                        ("elisp" :follow org--open-elisp-link)
>                        ("file" :complete org-file-complete-link)
>                        ("ftp" :follow
>                         (lambda (path) (browse-url (concat "ftp:"
>                         path))))
>                        ("help" :follow org--open-help-link)
>                        ("http" :follow
>                         (lambda (path) (browse-url (concat "http:"
>                         path))))
>                        ("https" :follow
>                         (lambda (path) (browse-url (concat "https:"
>                         path))))
>                        ("mailto" :follow
>                         (lambda (path) (browse-url (concat "mailto:"
>                         path))))
>                        ("news" :follow
>                         (lambda (path) (browse-url (concat "news:"
>                         path))))
>                        ("shell" :follow org--open-shell-link))
>  org-capture-templates '(("t" "TODO entry" entry
>                           (file+headline "~/org/agenda.org" "Capture")
>                           "* TODO %?\n")
>                          )
>  )
>
>

[-- Attachment #2: Type: text/html, Size: 10160 bytes --]

Re: Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

[Gustavo Barros]

Hi Carsten,

thank you for looking into this.

On Sat, Aug 10 2019, Carsten Dominik wrote:

> I tried to reproduce your example, and things worked properly....

I followed the described steps to the letter (except for the clear typo, 
where I should have written 'and cancel it with "C-c C-k"'). And the 
result is regularly the one described.

I have no idea how I could further isolate things. Could 
'(package-initialize)' be of any relevance? The only purpose of it here 
is to load the most recent version of org, instead of the built-in 
one. In your experience, what could be the source of the difference here 
and there? (OS? WM?)

Can anyone else reproduce?

I'm at your disposal to test any other possible intervening 
factors. But, as it stands, I don't know where to look at.

Best regards,
Gustavo.

Re: Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

[Carsten Dominik]

[-- Attachment #1: Type: text/plain, Size: 1124 bytes --]

Hi Gustavo,

I am also on Emacs 26.2, and I don't know where to look if I cannot
reproduce the problem.

It would be useful if someone else tries your minimal example and reports
back.

Carsten

On Sat, Aug 10, 2019 at 12:54 PM Gustavo Barros <gusbrs.2016@gmail.com>
wrote:

> Hi Carsten,
>
> thank you for looking into this.
>
> On Sat, Aug 10 2019, Carsten Dominik wrote:
>
> > I tried to reproduce your example, and things worked properly....
>
> I followed the described steps to the letter (except for the clear typo,
> where I should have written 'and cancel it with "C-c C-k"'). And the
> result is regularly the one described.
>
> I have no idea how I could further isolate things. Could
> '(package-initialize)' be of any relevance? The only purpose of it here
> is to load the most recent version of org, instead of the built-in
> one. In your experience, what could be the source of the difference here
> and there? (OS? WM?)
>
> Can anyone else reproduce?
>
> I'm at your disposal to test any other possible intervening
> factors. But, as it stands, I don't know where to look at.
>
> Best regards,
> Gustavo.
>

[-- Attachment #2: Type: text/html, Size: 1606 bytes --]

Re: Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

[Gustavo Barros]

Hi Carsten,

yes, let’s see if anyone else can reproduce. If not, feel free to file 
it under the old "little piece between the keyboard and the chair".

Best,
Gustavo.


On Sat, Aug 10 2019, Carsten Dominik wrote:

> Hi Gustavo,
>
> I am also on Emacs 26.2, and I don't know where to look if I cannot
> reproduce the problem.
>
> It would be useful if someone else tries your minimal example and 
> reports
> back.
>
> Carsten
>
> On Sat, Aug 10, 2019 at 12:54 PM Gustavo Barros 
> <gusbrs.2016@gmail.com>
> wrote:
>
>> Hi Carsten,
>>
>> thank you for looking into this.
>>
>> On Sat, Aug 10 2019, Carsten Dominik wrote:
>>
>> > I tried to reproduce your example, and things worked properly....
>>
>> I followed the described steps to the letter (except for the clear 
>> typo,
>> where I should have written 'and cancel it with "C-c C-k"'). And the
>> result is regularly the one described.
>>
>> I have no idea how I could further isolate things. Could
>> '(package-initialize)' be of any relevance? The only purpose of it 
>> here
>> is to load the most recent version of org, instead of the built-in
>> one. In your experience, what could be the source of the difference 
>> here
>> and there? (OS? WM?)
>>
>> Can anyone else reproduce?
>>
>> I'm at your disposal to test any other possible intervening
>> factors. But, as it stands, I don't know where to look at.
>>
>> Best regards,
>> Gustavo.
>>

Re: Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

[Gustavo Barros]

Hi Carsten,

On Sat, Aug 10 2019, Carsten Dominik wrote:

> Hi Gustavo,
>
> I am also on Emacs 26.2, and I don't know where to look if I cannot
> reproduce the problem.
>
> It would be useful if someone else tries your minimal example and 
> reports
> back.
>
> Carsten
>

I’ve tried to put my hands on possible intervening factors I could think 
of.

I’ve reduced the setup further to:

#+begin_src emacs-lisp
(add-to-list 'load-path 
"/home/gustavo/.emacs.d/elpa/org-plus-contrib-20190805")
(load-library "org")
(global-set-key (kbd "C-c c") 'org-capture)
(setq org-agenda-files
     '("~/org/agenda.org"))
(setq org-capture-templates
     '(("t" "TODO entry" entry
        (file+headline "~/org/agenda.org" "Capture")
        "* TODO %?\n")))
#+end_src

I’ve tried to rebuilt Emacs from source with default =./configure= 
options and run =src/emacs -Q=.

I’ve tried to create the agenda file from the =emacs -Q= session in case 
any encoding options of mine could intervene.

I’ve also tried my different Linux flavors (Mint 19.2 Cinnamon and MX 
Linux 18 Xfce).

And in every case the behavior was the same.

The only way I could make it not happen was to remove the preexisting 
line between "* Capture" and "* Following heading" headings.

As mentioned, if anyone can think of anything else I can try to narrow 
this down further, I’m at your disposal.

Best regards,
Gustavo.

Re: Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

[Gustavo Barros]

Hi All,

On Sun, 28 Jul 2019 at 16:34, Gustavo Barros <gusbrs.2016@gmail.com> wrote:
> When the capture template ends with a newline character and the capture
> process is canceled, the following heading gets demoted. And it
> shouldn’t.

a respectful bump on this one. I can still reproduce this with Emacs
28.2 and Org 9.5.5.

Best regards,
Gustavo.

Re: Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

[Ihor Radchenko]

Gustavo Barros <gusbrs.2016@gmail.com> writes:

> On Sun, 28 Jul 2019 at 16:34, Gustavo Barros <gusbrs.2016@gmail.com> wrote:
>> When the capture template ends with a newline character and the capture
>> process is canceled, the following heading gets demoted. And it
>> shouldn’t.
>
> a respectful bump on this one. I can still reproduce this with Emacs
> 28.2 and Org 9.5.5.

Thanks for bumping!
I am unable to reproduce your recipe on the latest Org main.
Note that we have several capture-related fixed on main but not on
stable Org 9.5.5.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Bug: canceled capture operation results in demoted following heading when template ends with newline [9.2.4 (9.2.4-11-g1c3eae-elpaplus @ /home/gustavo/.emacs.d/elpa/org-plus-contrib-20190722/)]

[Gustavo Barros]

Hi Ihor,

On Mon, 24 Oct 2022 at 01:21, Ihor Radchenko <yantar92@posteo.net> wrote:

> I am unable to reproduce your recipe on the latest Org main.

Indeed, I can confirm! True, the empty line between the headings still
goes missing, but there's no longer structural change which was the
real problem.

And, sorry for the noise. I had assumed that for such an old one the
latest release was good enough approximation, it turned out it wasn't.

> Note that we have several capture-related fixed on main but not on
> stable Org 9.5.5.

I was tempted to call the "lucky us" card again, but twice in a row
makes it unlikely. It seems that what's really going on is folks
working hard for the upcoming release, which is much appreciated.  :-)
Thank you all! And particularly to whoever got this one fixed.

Best regards,
Gustavo.