[BUG] org-agenda-prepare-buffers does not save restrictions

[BUG] org-agenda-prepare-buffers does not save restrictions

[Al Haji-Ali]

[-- Attachment #1: Type: text/plain, Size: 1146 bytes --]


I am not sure if this has been reported before, of if the behaviour is intended. But to reproduce this "bug":
- Open an org file from the agenda.
- Restrict to some headline
- Execute the following command:

(with-temp-buffer
  (org-agenda-prepare-buffers (org-agenda-files t)))

- The restriction is removed.

Note that executing `(org-agenda-prepare-buffers (org-agenda-files t))` in the buffer (without `with-temp-buffer`) restores the restriction upon completion.

The reason is that `save-restriction` (and `save-excursion`) are called in `org-agenda-prepare-buffers`
before switching to the buffer of the org file so the restriction is not saved (the point is manually restored but not the restriction).

This has implications on running `org-map-entries` where all org buffers in the agenda (beside the current one) lose their restrictions.

Attached is a possible patch to fix this.

Emacs  : GNU Emacs 28.1 (build 3, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0)
 of 2022-04-08
Package: Org mode version 9.5.3 (9.5.3-g428076 @ /home/abdo/Work/Projects/dotfiles/emacs-radian/.emacs.d/.elocal/straight/build-28.1/org/)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: org-agenda-prepare-buffers.patch --]
[-- Type: text/x-diff, Size: 3925 bytes --]

diff --git a/lisp/org.el b/lisp/org.el
index 661efeb9c..e75b7c93c 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -15031,49 +15031,48 @@ When a buffer is unmodified, it is just killed.  When modified, it is saved
   "Create buffers for all agenda files, protect archived trees and comments."
   (interactive)
   (let ((inhibit-read-only t)
-	(org-inhibit-startup org-agenda-inhibit-startup)
-	pos)
+	(org-inhibit-startup org-agenda-inhibit-startup))
     (setq org-tag-alist-for-agenda nil
 	  org-tag-groups-alist-for-agenda nil)
     (save-excursion
-      (save-restriction
-	(dolist (file files)
-	  (catch 'nextfile
-	    (if (bufferp file)
-		(set-buffer file)
-	      (org-check-agenda-file file)
-	      (set-buffer (org-get-agenda-file-buffer file)))
-	    (widen)
-	    (org-set-regexps-and-options 'tags-only)
-	    (setq pos (point))
-	    (or (memq 'category org-agenda-ignore-properties)
-		(org-refresh-category-properties))
-	    (or (memq 'stats org-agenda-ignore-properties)
-		(org-refresh-stats-properties))
-	    (or (memq 'effort org-agenda-ignore-properties)
-                (unless org-element-use-cache
-		  (org-refresh-effort-properties)))
-	    (or (memq 'appt org-agenda-ignore-properties)
-		(org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime))
-	    (setq org-todo-keywords-for-agenda
-		  (append org-todo-keywords-for-agenda org-todo-keywords-1))
-	    (setq org-done-keywords-for-agenda
-		  (append org-done-keywords-for-agenda org-done-keywords))
-	    (setq org-todo-keyword-alist-for-agenda
-		  (append org-todo-keyword-alist-for-agenda org-todo-key-alist))
-	    (setq org-tag-alist-for-agenda
-		  (org--tag-add-to-alist
-		   org-tag-alist-for-agenda
-		   org-current-tag-alist))
-	    ;; Merge current file's tag groups into global
-	    ;; `org-tag-groups-alist-for-agenda'.
-	    (when org-group-tags
-	      (dolist (alist org-tag-groups-alist)
-		(let ((old (assoc (car alist) org-tag-groups-alist-for-agenda)))
-		  (if old
-		      (setcdr old (org-uniquify (append (cdr old) (cdr alist))))
-		    (push alist org-tag-groups-alist-for-agenda)))))
-	    (goto-char pos)))))
+      (dolist (file files)
+	(catch 'nextfile
+	  (if (bufferp file)
+	      (set-buffer file)
+	    (org-check-agenda-file file)
+	    (set-buffer (org-get-agenda-file-buffer file)))
+	  (save-excursion
+            (save-restriction
+              (widen)
+	      (org-set-regexps-and-options 'tags-only)
+	      (setq pos (point))
+	      (or (memq 'category org-agenda-ignore-properties)
+		  (org-refresh-category-properties))
+	      (or (memq 'stats org-agenda-ignore-properties)
+		  (org-refresh-stats-properties))
+	      (or (memq 'effort org-agenda-ignore-properties)
+                  (unless org-element-use-cache
+		    (org-refresh-effort-properties)))
+	      (or (memq 'appt org-agenda-ignore-properties)
+		  (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime))
+	      (setq org-todo-keywords-for-agenda
+		    (append org-todo-keywords-for-agenda org-todo-keywords-1))
+	      (setq org-done-keywords-for-agenda
+		    (append org-done-keywords-for-agenda org-done-keywords))
+	      (setq org-todo-keyword-alist-for-agenda
+		    (append org-todo-keyword-alist-for-agenda org-todo-key-alist))
+	      (setq org-tag-alist-for-agenda
+		    (org--tag-add-to-alist
+		     org-tag-alist-for-agenda
+		     org-current-tag-alist))
+	      ;; Merge current file's tag groups into global
+	      ;; `org-tag-groups-alist-for-agenda'.
+	      (when org-group-tags
+	        (dolist (alist org-tag-groups-alist)
+		  (let ((old (assoc (car alist) org-tag-groups-alist-for-agenda)))
+		    (if old
+		        (setcdr old (org-uniquify (append (cdr old) (cdr alist))))
+		      (push alist org-tag-groups-alist-for-agenda))))))))))
     (setq org-todo-keywords-for-agenda
           (org-uniquify org-todo-keywords-for-agenda))
     (setq org-todo-keyword-alist-for-agenda

Re: [BUG] org-agenda-prepare-buffers does not save restrictions

[Ihor Radchenko]

Al Haji-Ali <abdo.haji.ali@gmail.com> writes:

> I am not sure if this has been reported before, of if the behaviour is intended. But to reproduce this "bug":
> - Open an org file from the agenda.
> - Restrict to some headline
> - Execute the following command:
>
> (with-temp-buffer
>   (org-agenda-prepare-buffers (org-agenda-files t)))
>
> - The restriction is removed.
>
> Note that executing `(org-agenda-prepare-buffers (org-agenda-files t))` in the buffer (without `with-temp-buffer`) restores the restriction upon completion.
>
> The reason is that `save-restriction` (and `save-excursion`) are called in `org-agenda-prepare-buffers`
> before switching to the buffer of the org file so the restriction is not saved (the point is manually restored but not the restriction).
>
> This has implications on running `org-map-entries` where all org buffers in the agenda (beside the current one) lose their restrictions.

I agree that the current behavior might be unexpected.
I also do not see any potential side effects of applying the proposed
patch.

> Attached is a possible patch to fix this.

Thanks for the patch! The patch looks good.

Could you please create a proper patch following
https://orgmode.org/worg/org-contribute.html#first-patch instructions?

You will need to add the proper commit message and add TINYCHANGE line
to the message. All the details are described in the link.

Best,
Ihor

Re: [BUG] org-agenda-prepare-buffers does not save restrictions

[Al Haji-Ali]

[-- Attachment #1: Type: text/plain, Size: 208 bytes --]


On 08/07/2022, Ihor Radchenko wrote:
> Could you please create a proper patch

See attached. I made a further change to use `with-current-buffer` instead of
`set-buffer+save-excursion`.

Best regards,
-- Al

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: save-restrictions-agenda-buf.patch --]
[-- Type: text/x-patch, Size: 4556 bytes --]

From 398fee0c235c53399cd8cc481f9e732f64ae88cb Mon Sep 17 00:00:00 2001
From: Al Haji-Ali <abdo.haji.ali@gmail.com>
Date: Fri, 8 Jul 2022 10:24:08 +0100
Subject: [PATCH] lisp/org.el: Save restrictions in all agenda file buffers.

* lisp/org.el (org-agenda-prepare-buffers): Call
`save-restriction'/`save-excursion' for every buffer in the agenda
instead of just once for the current buffer. Use `with-current-buffer'
instead of `set-buffer'/`save-excursion'.

TINYCHANGE
---
 lisp/org.el | 80 ++++++++++++++++++++++++++---------------------------
 1 file changed, 39 insertions(+), 41 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 661efeb9c..ef3e4b950 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -15031,49 +15031,47 @@ When a buffer is unmodified, it is just killed.  When modified, it is saved
   "Create buffers for all agenda files, protect archived trees and comments."
   (interactive)
   (let ((inhibit-read-only t)
-	(org-inhibit-startup org-agenda-inhibit-startup)
-	pos)
+	(org-inhibit-startup org-agenda-inhibit-startup))
     (setq org-tag-alist-for-agenda nil
 	  org-tag-groups-alist-for-agenda nil)
-    (save-excursion
-      (save-restriction
-	(dolist (file files)
-	  (catch 'nextfile
-	    (if (bufferp file)
-		(set-buffer file)
-	      (org-check-agenda-file file)
-	      (set-buffer (org-get-agenda-file-buffer file)))
-	    (widen)
-	    (org-set-regexps-and-options 'tags-only)
-	    (setq pos (point))
-	    (or (memq 'category org-agenda-ignore-properties)
-		(org-refresh-category-properties))
-	    (or (memq 'stats org-agenda-ignore-properties)
-		(org-refresh-stats-properties))
-	    (or (memq 'effort org-agenda-ignore-properties)
-                (unless org-element-use-cache
-		  (org-refresh-effort-properties)))
-	    (or (memq 'appt org-agenda-ignore-properties)
-		(org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime))
-	    (setq org-todo-keywords-for-agenda
-		  (append org-todo-keywords-for-agenda org-todo-keywords-1))
-	    (setq org-done-keywords-for-agenda
-		  (append org-done-keywords-for-agenda org-done-keywords))
-	    (setq org-todo-keyword-alist-for-agenda
-		  (append org-todo-keyword-alist-for-agenda org-todo-key-alist))
-	    (setq org-tag-alist-for-agenda
-		  (org--tag-add-to-alist
-		   org-tag-alist-for-agenda
-		   org-current-tag-alist))
-	    ;; Merge current file's tag groups into global
-	    ;; `org-tag-groups-alist-for-agenda'.
-	    (when org-group-tags
-	      (dolist (alist org-tag-groups-alist)
-		(let ((old (assoc (car alist) org-tag-groups-alist-for-agenda)))
-		  (if old
-		      (setcdr old (org-uniquify (append (cdr old) (cdr alist))))
-		    (push alist org-tag-groups-alist-for-agenda)))))
-	    (goto-char pos)))))
+    (dolist (file files)
+      (catch 'nextfile
+        (with-current-buffer
+            (if (bufferp file)
+                file
+              (org-check-agenda-file file)
+              (org-get-agenda-file-buffer file))
+          (save-excursion
+            (save-restriction
+              (widen)
+	      (org-set-regexps-and-options 'tags-only)
+	      (or (memq 'category org-agenda-ignore-properties)
+		  (org-refresh-category-properties))
+	      (or (memq 'stats org-agenda-ignore-properties)
+		  (org-refresh-stats-properties))
+	      (or (memq 'effort org-agenda-ignore-properties)
+                  (unless org-element-use-cache
+		    (org-refresh-effort-properties)))
+	      (or (memq 'appt org-agenda-ignore-properties)
+		  (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime))
+	      (setq org-todo-keywords-for-agenda
+		    (append org-todo-keywords-for-agenda org-todo-keywords-1))
+	      (setq org-done-keywords-for-agenda
+		    (append org-done-keywords-for-agenda org-done-keywords))
+	      (setq org-todo-keyword-alist-for-agenda
+		    (append org-todo-keyword-alist-for-agenda org-todo-key-alist))
+	      (setq org-tag-alist-for-agenda
+		    (org--tag-add-to-alist
+		     org-tag-alist-for-agenda
+		     org-current-tag-alist))
+	      ;; Merge current file's tag groups into global
+	      ;; `org-tag-groups-alist-for-agenda'.
+	      (when org-group-tags
+	        (dolist (alist org-tag-groups-alist)
+		  (let ((old (assoc (car alist) org-tag-groups-alist-for-agenda)))
+		    (if old
+		        (setcdr old (org-uniquify (append (cdr old) (cdr alist))))
+		      (push alist org-tag-groups-alist-for-agenda))))))))))
     (setq org-todo-keywords-for-agenda
           (org-uniquify org-todo-keywords-for-agenda))
     (setq org-todo-keyword-alist-for-agenda
--
2.24.4

Re: [BUG] org-agenda-prepare-buffers does not save restrictions

[Ihor Radchenko]

Al Haji-Ali <abdo.haji.ali@gmail.com> writes:

> On 08/07/2022, Ihor Radchenko wrote:
>> Could you please create a proper patch
>
> See attached. I made a further change to use `with-current-buffer` instead of
> `set-buffer+save-excursion`.

Thanks!
Two more comments.

> * lisp/org.el (org-agenda-prepare-buffers): Call
> `save-restriction'/`save-excursion' for every buffer in the agenda
> instead of just once for the current buffer. Use `with-current-buffer'
> instead of `set-buffer'/`save-excursion'.

Please use double space between sentences.

> +          (save-excursion
> +            (save-restriction
> +              (widen)

A more concise way is

(org-with-wide-buffer ...)

Best,
Ihor

Re: [BUG] org-agenda-prepare-buffers does not save restrictions

[Al Haji-Ali]

[-- Attachment #1: Type: text/plain, Size: 290 bytes --]


On 09/07/2022, Ihor Radchenko wrote:
> Two more comments.
Attached is the updated patch.

PS: I noticed that the `org-timer.el` example in
https://orgmode.org/worg/org-contribute.html
has a single space between sentences. Other examples on the page seem fine though. 

Best regards,
-- Al

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: save-restrictions-agenda-buf-v2.patch --]
[-- Type: text/x-diff, Size: 4520 bytes --]

From fcaff9b583c46c0d147aaf4c7e3647f689d54894 Mon Sep 17 00:00:00 2001
From: Al Haji-Ali <abdo.haji.ali@gmail.com>
Date: Sun, 10 Jul 2022 09:46:26 +0100
Subject: [PATCH] lisp/org.el: Save restrictions in all agenda file buffers.

* lisp/org.el (org-agenda-prepare-buffers): Call
`org-with-wide-buffer' for every buffer in the agenda, instead of
calling `save-excursion'/'`save-restrictions' just for the current
buffer, to save restrictions in all buffers.  Use
`with-current-buffer' instead of `save-excursion'/'`set-buffer'.

TINYCHANGE
---
 lisp/org.el | 78 +++++++++++++++++++++++++----------------------------
 1 file changed, 37 insertions(+), 41 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 661efeb9c..3d4de5b4f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -15031,49 +15031,45 @@ When a buffer is unmodified, it is just killed.  When modified, it is saved
   "Create buffers for all agenda files, protect archived trees and comments."
   (interactive)
   (let ((inhibit-read-only t)
-	(org-inhibit-startup org-agenda-inhibit-startup)
-	pos)
+	(org-inhibit-startup org-agenda-inhibit-startup))
     (setq org-tag-alist-for-agenda nil
 	  org-tag-groups-alist-for-agenda nil)
-    (save-excursion
-      (save-restriction
-	(dolist (file files)
-	  (catch 'nextfile
-	    (if (bufferp file)
-		(set-buffer file)
-	      (org-check-agenda-file file)
-	      (set-buffer (org-get-agenda-file-buffer file)))
-	    (widen)
-	    (org-set-regexps-and-options 'tags-only)
-	    (setq pos (point))
-	    (or (memq 'category org-agenda-ignore-properties)
-		(org-refresh-category-properties))
-	    (or (memq 'stats org-agenda-ignore-properties)
-		(org-refresh-stats-properties))
-	    (or (memq 'effort org-agenda-ignore-properties)
-                (unless org-element-use-cache
-		  (org-refresh-effort-properties)))
-	    (or (memq 'appt org-agenda-ignore-properties)
-		(org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime))
-	    (setq org-todo-keywords-for-agenda
-		  (append org-todo-keywords-for-agenda org-todo-keywords-1))
-	    (setq org-done-keywords-for-agenda
-		  (append org-done-keywords-for-agenda org-done-keywords))
-	    (setq org-todo-keyword-alist-for-agenda
-		  (append org-todo-keyword-alist-for-agenda org-todo-key-alist))
-	    (setq org-tag-alist-for-agenda
-		  (org--tag-add-to-alist
-		   org-tag-alist-for-agenda
-		   org-current-tag-alist))
-	    ;; Merge current file's tag groups into global
-	    ;; `org-tag-groups-alist-for-agenda'.
-	    (when org-group-tags
-	      (dolist (alist org-tag-groups-alist)
-		(let ((old (assoc (car alist) org-tag-groups-alist-for-agenda)))
-		  (if old
-		      (setcdr old (org-uniquify (append (cdr old) (cdr alist))))
-		    (push alist org-tag-groups-alist-for-agenda)))))
-	    (goto-char pos)))))
+    (dolist (file files)
+      (catch 'nextfile
+        (with-current-buffer
+            (if (bufferp file)
+                file
+              (org-check-agenda-file file)
+              (org-get-agenda-file-buffer file))
+          (org-with-wide-buffer
+	   (org-set-regexps-and-options 'tags-only)
+	   (or (memq 'category org-agenda-ignore-properties)
+	       (org-refresh-category-properties))
+	   (or (memq 'stats org-agenda-ignore-properties)
+	       (org-refresh-stats-properties))
+	   (or (memq 'effort org-agenda-ignore-properties)
+               (unless org-element-use-cache
+		 (org-refresh-effort-properties)))
+	   (or (memq 'appt org-agenda-ignore-properties)
+	       (org-refresh-properties "APPT_WARNTIME" 'org-appt-warntime))
+	   (setq org-todo-keywords-for-agenda
+		 (append org-todo-keywords-for-agenda org-todo-keywords-1))
+	   (setq org-done-keywords-for-agenda
+		 (append org-done-keywords-for-agenda org-done-keywords))
+	   (setq org-todo-keyword-alist-for-agenda
+		 (append org-todo-keyword-alist-for-agenda org-todo-key-alist))
+	   (setq org-tag-alist-for-agenda
+		 (org--tag-add-to-alist
+		  org-tag-alist-for-agenda
+		  org-current-tag-alist))
+	   ;; Merge current file's tag groups into global
+	   ;; `org-tag-groups-alist-for-agenda'.
+	   (when org-group-tags
+	     (dolist (alist org-tag-groups-alist)
+	       (let ((old (assoc (car alist) org-tag-groups-alist-for-agenda)))
+		 (if old
+		     (setcdr old (org-uniquify (append (cdr old) (cdr alist))))
+		   (push alist org-tag-groups-alist-for-agenda)))))))))
     (setq org-todo-keywords-for-agenda
           (org-uniquify org-todo-keywords-for-agenda))
     (setq org-todo-keyword-alist-for-agenda
--
2.25.1

Re: [BUG] org-agenda-prepare-buffers does not save restrictions

[Ihor Radchenko]

Al Haji-Ali <abdo.haji.ali@gmail.com> writes:

> On 09/07/2022, Ihor Radchenko wrote:
>> Two more comments.
> Attached is the updated patch.

Thanks!
Applied onto main via bc33c0133.

> PS: I noticed that the `org-timer.el` example in
> https://orgmode.org/worg/org-contribute.html
> has a single space between sentences. Other examples on the page seem fine though. 

Fixed.

Best,
Ihor