Proposal: 'executable' org-capture-templaes

Proposal: 'executable' org-capture-templaes

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 973 bytes --]


Hi guys,

I was playing with org-capture today, and it strike me that it could be used as
a simple and lightweight alternative to create GUIs for user options, somewhat
resembling the use of well-known hydra/transient or built-in help macro or easy
menu.

I would like to propose a small 'feature/change' to org-capture templates, to
inlude a new target: 'exec'. It should be followed by a function to be
executed.

I believe that this is a simple change that does not intrude on anything else in
org-capture, you can see the attached patch. The goal is to just circumwent the
createion of capture buffer, bookmark and other processing done by org-capture
etc. However there might be things I am not aware of, so if someone have more
insight, it is welcomed to hear.

There is a small illustration in the attached patch as well.

The error handling could probably be much better, so I would like to hear
opinion and suggestion how can I improve it.

best regards
/arthur


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Introduce-executable-org-capture-templates.patch --]
[-- Type: text/x-patch, Size: 3388 bytes --]

From b8a910235c688d9ea1cb717a186699da489987af Mon Sep 17 00:00:00 2001
From: Arthur Miller <arthur.miller@live.com>
Date: Thu, 26 May 2022 17:15:37 +0200
Subject: [PATCH] Introduce 'executable' org-capture-templates

* etc/ORG-NEWS: Documented new feature.
* doc/misc/org.org: Documented new template target.
* lisp/org/org-capture.el: 'org-capture' add handler for 'exec' target.
---
 doc/misc/org.org        | 14 ++++++++++++++
 etc/ORG-NEWS            | 28 ++++++++++++++++++++++++++++
 lisp/org/org-capture.el |  5 +++++
 3 files changed, 47 insertions(+)

diff --git a/doc/misc/org.org b/doc/misc/org.org
index 3dce83c936..2445c57942 100644
--- a/doc/misc/org.org
+++ b/doc/misc/org.org
@@ -7629,6 +7629,20 @@ Now lets look at the elements of a template definition.  Each entry in
     the target entry or as a top-level entry.  The target file should
     be an Org file.
 
+  - ~exec~ ::
+
+    An executable function. Function will be executed and the result
+    returned immidiately. No further processing by org-capture will be
+    performed, no capture buffer will be created, no last capture
+    bookmark etc, will be created. The eventual other template
+    parameters are ignored. Use this when you need to execute a Lisp
+    function for the side effects. Useful if you would like to create
+    lightweight GUIs for user choices based on org-capture mechanism.
+
+    Simple example:
+
+    '("h" "Say hello" exec (lambda () (message "Hello, World!")))
+
   - ~item~ ::
 
     A plain list item, placed in the first plain list at the target
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 37a39131d9..53ac10ba45 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -108,6 +108,34 @@ If you prefer to keep the keybinding, you can add it back to
 #+end_src
 
 ** New features
+*** 'Executable' org-capture-templates
+
+New target, "exec" is added to templates which provides an easy option
+to run a function from within org-capture dialog. It is meant as a
+lightweight option to create GUIs for user options based on
+org-capture mechanism.
+
+Exec should be followed by a lisp function, a lambda or a function
+object that will be executed and result returned without further
+processing by org-capture. As a consequence other template parameters
+are not used with this target.
+
+Illustration:
+
+#+begin_src emacs-lisp
+(defvar my-templates
+  `(("h" "Hello World" exec
+     (lambda ()
+       (message "Hello, World")))
+    ("f" "Find file" exec ,(function find-file))))
+
+(defun simple-menu ()
+  (interactive)
+  (let ((org-capture-templates my-templates))
+    (org-capture)))
+
+(define-key global-map (kbd "C-S-n") #'simple-menu)
+#+end_src
 
 *** New citation engine
 
diff --git a/lisp/org/org-capture.el b/lisp/org/org-capture.el
index 2fd9a9c74d..1a9b59de2f 100644
--- a/lisp/org/org-capture.el
+++ b/lisp/org/org-capture.el
@@ -665,6 +665,11 @@ org-capture
 	(remove-text-properties 0 (length annotation)
 				'(read-only t) annotation))
       (cond
+       ((equal (nth 2 entry) 'exec)
+        (let ((f (plist-get entry 'exec)))
+          (if (not f) (error "Missing function specification.")
+            (if (commandp f) (call-interactively f)
+              (if (functionp f) (funcall f) (error "Invalid function specification."))))))
        ((equal entry "C")
 	(customize-variable 'org-capture-templates))
        ((equal entry "q")
-- 
2.36.1

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

> I was playing with org-capture today, and it strike me that it could be used as
> a simple and lightweight alternative to create GUIs for user options, somewhat
> resembling the use of well-known hydra/transient or built-in help macro or easy
> menu.

Is there any reason why you dislike transient (now part of Emacs core)?
We actually had multiple threads discussing possibility to port all the
Org dialogues to transient.

> I would like to propose a small 'feature/change' to org-capture templates, to
> inlude a new target: 'exec'. It should be followed by a function to be
> executed.
>
> I believe that this is a simple change that does not intrude on anything else in
> org-capture, you can see the attached patch. The goal is to just circumwent the
> createion of capture buffer, bookmark and other processing done by org-capture
> etc. However there might be things I am not aware of, so if someone have more
> insight, it is welcomed to hear.

It seems to be quite out of scope of org-capture. If anything, capture
menu might be factored out to a generic menu framework. But again, we
already have transient achieving the same goal.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Ihor Radchenko <yantar92@gmail.com> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>> I was playing with org-capture today, and it strike me that it could be used as
>> a simple and lightweight alternative to create GUIs for user options, somewhat
>> resembling the use of well-known hydra/transient or built-in help macro or easy
>> menu.
>
> Is there any reason why you dislike transient (now part of Emacs core)?

It is not about me disliking transient, I am sure it is a great library. It is
more about simplicity and efficiency. Entire org-capture is ~2/3 of the size of
transient alone, and as an org user I have org-mode loaded almost always, while
I don't have transient loaded, since I am using magit very, very rarely, which I
believe is the only consumer of transient in my Emacs. So it is more about
lightness and efficiency. 

Simplicity comes from the org-templates. Me, and I guess other people are
familiar with org-catpure templates already, and I mean, can it be simpler to
create a menu of choices then a simple list:

'(("key1" "label1" exec (lambda () ...))
  ("key2" "label2" exec (labmda () ...))
   ...
   )

People are already writing those as part of org-capture, so there is a
familiarity factor. Transient has to be learned, and the complexity is much
bigger.

For the record, there are other alternatives to transient that were already in Emacs
for example macro 'make-help-screen', as used in Emacs help, but the complexity
is also bigger, I am sure you are aware of it's usage, but for other readers, an
example usage is in help.el, starting on line 238, where help menu is built. It
is not terribly complicated to use, but it is still more involved then
org-capture template would be.

Of course, those are more capapble than org-capture, but most often we need just
a simple choice.

> We actually had multiple threads discussing possibility to port all the
> Org dialogues to transient.

I have unfortunately missed those discussions. But as said, I am not in to argue
for or against transient at all. I would just like to re-use the org-capture
code, since it is already in-place.

>> I would like to propose a small 'feature/change' to org-capture templates, to
>> inlude a new target: 'exec'. It should be followed by a function to be
>> executed.
>>
>> I believe that this is a simple change that does not intrude on anything else in
>> org-capture, you can see the attached patch. The goal is to just circumwent the
>> createion of capture buffer, bookmark and other processing done by org-capture
>> etc. However there might be things I am not aware of, so if someone have more
>> insight, it is welcomed to hear.
>
> It seems to be quite out of scope of org-capture.

Maybe, but than what is in scope of org-mode or org-capture or Emacs? We are
constantly bending code to do what it is not really meant to do. Since to be a
DNA of Emacs :).

>                                                   If anything, capture
> menu might be factored out to a generic menu framework.

Please no. Not every single piece of Emacs has to be a generalized
framework. Generalized frameworks add an extra layer of complexity, and it this
case, as you point out, we have other frameworks, so yet another framework is
*definitely* not what I ask for.

> already have transient achieving the same goal.

Yes, and other alternatives, but to higher cost mentally and computationally.

I just wish to use org-capture for its simplicity and lighntess. Considering
it's five lines of code, and no additional learning required to use it, I think
it is rather "cheap". We also had easy-menu and make-help-screen when we got
org-capture and transient. I can still understand the feeling that it is "out of
scope", but I don't think it is terribly out of org-capture waters.

Anyway, it is a proposal, and if you don't want it in org it is OK. I personally
think it could make life simpler in cases people need a simple choice menus to
fire up an action, and it comes with very small cost.

best regards
/arthur

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 27/05/2022 19:17, Arthur Miller wrote:
> Ihor Radchenko writes:
>>
>> Is there any reason why you dislike transient (now part of Emacs core)?
> 
> It is not about me disliking transient, I am sure it is a great library. It is
> more about simplicity and efficiency. Entire org-capture is ~2/3 of the size of
> transient alone, and as an org user I have org-mode loaded almost always, while
> I don't have transient loaded, since I am using magit very, very rarely, which I
> believe is the only consumer of transient in my Emacs. So it is more about
> lightness and efficiency.
> 
> '(("key1" "label1" exec (lambda () ...))
>    ("key2" "label2" exec (labmda () ...))
>     ...
>     )

If you are seeking simplicity, I suppose, `completing-read' should be 
more than enough. TAB shows list of options and it is quite similar to menu.

I have seen `org-mks' function but I am unsure if it will suite for you 
to define capture-like interface.

I do not think, you patch should be applied. Org-capture is a particular 
case of menu-like interface, but its purpose is capture and that should 
not be abused.

I can not say that I really like org-capture UI. It blocks other windows 
and frame. I believe, menu windows should just define own keymap without 
affecting of minibuffer.

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

> Simplicity comes from the org-templates. Me, and I guess other people are
> familiar with org-catpure templates already, and I mean, can it be simpler to
> create a menu of choices then a simple list:
>
> '(("key1" "label1" exec (lambda () ...))
>   ("key2" "label2" exec (labmda () ...))
>    ...
>    )
>
> People are already writing those as part of org-capture, so there is a
> familiarity factor. Transient has to be learned, and the complexity is much
> bigger.

>>                                                   If anything, capture
>> menu might be factored out to a generic menu framework.
>
> Please no. Not every single piece of Emacs has to be a generalized
> framework. Generalized frameworks add an extra layer of complexity, and it this
> case, as you point out, we have other frameworks, so yet another framework is
> *definitely* not what I ask for.

By "generic" I did not mean general-purpose all-functional framework.
We just need something to remove code duplication in
org-export-dispatch, org-agenda, org-capture, org-set-tags-command, etc
They all share pretty similar code to generate dialogues.

As for familiarity, I understand and it is exactly the reason why I
suggested to factor out the menu code from capture templates.

I am strongly not in favor of adding exec to org-capture itself. It's
a bit like if you were to add :activate-func for a link that actually
downloads some file from internet, then generates and exports agenda,
and meanwhile also restarts your remote server. This will also kind of
save the code, but completely out of purpose of :activate-func. Of
course, I am exaggerating here, but just want to make my point clear.

>> We actually had multiple threads discussing possibility to port all the
>> Org dialogues to transient.
>
> I have unfortunately missed those discussions. But as said, I am not in to argue
> for or against transient at all. I would just like to re-use the org-capture
> code, since it is already in-place.

The last one was
https://orgmode.org/list/8c364693bf6856e60cdd3e8b63ab0c9284d16733.camel@heagren.com
And we had multiple complaints that Org menus are not searchable and do
not allow recursive edit.

>> It seems to be quite out of scope of org-capture.
>
> Maybe, but than what is in scope of org-mode or org-capture or Emacs? We are
> constantly bending code to do what it is not really meant to do. Since to be a
> DNA of Emacs :).

Sure. But another feature of Emacs is being consistent. Emacs does
provide flexible interfaces, but they stick to they purpose. Abusing
them is up to the user (with no guarantees to work in future versions),
but not up to Emacs itself.

If we were to include your suggestion, we would also need to maintain
the new generalized functionality in future. Even if we need to change
some internals of org-capture. Over-generalization will put an extra
burden on future maintenance.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Ihor Radchenko <yantar92@gmail.com> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>> Simplicity comes from the org-templates. Me, and I guess other people are
>> familiar with org-catpure templates already, and I mean, can it be simpler to
>> create a menu of choices then a simple list:
>>
>> '(("key1" "label1" exec (lambda () ...))
>>   ("key2" "label2" exec (labmda () ...))
>>    ...
>>    )
>>
>> People are already writing those as part of org-capture, so there is a
>> familiarity factor. Transient has to be learned, and the complexity is much
>> bigger.
>
>>>                                                   If anything, capture
>>> menu might be factored out to a generic menu framework.
>>
>> Please no. Not every single piece of Emacs has to be a generalized
>> framework. Generalized frameworks add an extra layer of complexity, and it this
>> case, as you point out, we have other frameworks, so yet another framework is
>> *definitely* not what I ask for.
>
> By "generic" I did not mean general-purpose all-functional framework.
> We just need something to remove code duplication in
> org-export-dispatch, org-agenda, org-capture, org-set-tags-command, etc
> They all share pretty similar code to generate dialogues.
>
> As for familiarity, I understand and it is exactly the reason why I
> suggested to factor out the menu code from capture templates.

I am not really familiar with those other dialogues but org-capture, so I only
had that one in the mind. Yes, I agree if the similar code is used/shared in
several places than it does make sense to refactor it out.

> I am strongly not in favor of adding exec to org-capture itself. It's
> a bit like if you were to add :activate-func for a link that actually
> downloads some file from internet, then generates and exports agenda,
> and meanwhile also restarts your remote server. This will also kind of
> save the code, but completely out of purpose of :activate-func. Of
> course, I am exaggerating here, but just want to make my point clear.

I understand, it is ok :).

>>> We actually had multiple threads discussing possibility to port all the
>>> Org dialogues to transient.
>>
>> I have unfortunately missed those discussions. But as said, I am not in to argue
>> for or against transient at all. I would just like to re-use the org-capture
>> code, since it is already in-place.
>
> The last one was
> https://orgmode.org/list/8c364693bf6856e60cdd3e8b63ab0c9284d16733.camel@heagren.com
> And we had multiple complaints that Org menus are not searchable and do
> not allow recursive edit.

I am not sure what complain about searchability is/was, so I should not say much
there, but those are just a list of lists, saved in a well-known place,
org-caputre-templates var, so one can always traverse that list, or search the
menu buffer itself. Anyway thanks for the link, I have read through
that discussion. Seems to me like most of you are in favour of refactoring org
to use transient everywhere?

>>> It seems to be quite out of scope of org-capture.
>>
>> Maybe, but than what is in scope of org-mode or org-capture or Emacs? We are
>> constantly bending code to do what it is not really meant to do. Since to be a
>> DNA of Emacs :).
>
> Sure. But another feature of Emacs is being consistent. Emacs does
> provide flexible interfaces, but they stick to they purpose. Abusing
> them is up to the user (with no guarantees to work in future versions),
> but not up to Emacs itself.
>
> If we were to include your suggestion, we would also need to maintain
> the new generalized functionality in future. Even if we need to change
> some internals of org-capture. Over-generalization will put an extra
> burden on future maintenance.

Np, I understand. Thanks for the answer anyway.

Best regards
/a

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

>> By "generic" I did not mean general-purpose all-functional framework.
>> We just need something to remove code duplication in
>> org-export-dispatch, org-agenda, org-capture, org-set-tags-command, etc
>> They all share pretty similar code to generate dialogues.
>>
>> As for familiarity, I understand and it is exactly the reason why I
>> suggested to factor out the menu code from capture templates.
>
> I am not really familiar with those other dialogues but org-capture, so I only
> had that one in the mind. Yes, I agree if the similar code is used/shared in
> several places than it does make sense to refactor it out.

This refactoring could be a practical way to get something similar to
your proposal into Org core. At least, if the menus are factored out
appropriately.

The above statement is a hint that patches are welcome :)

>> The last one was
>> https://orgmode.org/list/8c364693bf6856e60cdd3e8b63ab0c9284d16733.camel@heagren.com
>> And we had multiple complaints that Org menus are not searchable and do
>> not allow recursive edit.
>
> I am not sure what complain about searchability is/was, so I should not say much
> there, but those are just a list of lists, saved in a well-known place,
> org-caputre-templates var, so one can always traverse that list, or search the
> menu buffer itself. Anyway thanks for the link, I have read through
> that discussion. Seems to me like most of you are in favour of refactoring org
> to use transient everywhere?

The complaint was mostly from users who wanted to interrupt, say, the
capture process, switch to the menu buffer, and search text there using
isearch. Similar to what you can do with vanilla *Completions* buffer.

Transient has met similar complaints when it was in the process of
merging into Emacs core and now transient does support this "searching"
possibility. That's why using transient could be an easy way for Org to
address such complaints without a need to increase the maintenance
burden.

However, despite general agreement that Org should switch to transient
menus, we will still keep backwards compatibility with the existing
menus. So, one way or another, the existing menus will be retained.
Ideally, also factored out and possibly merged into Emacs core (as
requested by RMS
https://orgmode.org/list/E1kIPh1-0001Lu-Rg@fencepost.gnu.org).

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Ihor Radchenko <yantar92@gmail.com> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>>> By "generic" I did not mean general-purpose all-functional framework.
>>> We just need something to remove code duplication in
>>> org-export-dispatch, org-agenda, org-capture, org-set-tags-command, etc
>>> They all share pretty similar code to generate dialogues.
>>>
>>> As for familiarity, I understand and it is exactly the reason why I
>>> suggested to factor out the menu code from capture templates.
>>
>> I am not really familiar with those other dialogues but org-capture, so I only
>> had that one in the mind. Yes, I agree if the similar code is used/shared in
>> several places than it does make sense to refactor it out.
>
> This refactoring could be a practical way to get something similar to
> your proposal into Org core. At least, if the menus are factored out
> appropriately.

As I see from 'org-capture' function, it does not seem to be terribly hard to
factor menu creation out. There seem to be two parts: template selection which
is already done by 'org-capture-select-template' function, and then the main
work that one has to implement on its own, which is specific to whatever one
would like to implement. I just did a quick refactor to test the idea:

#+begin_src emacs-lisp
(require 'org-capture)
(defun org-menu (&optional goto keys)
  (interactive "P")
  (let* ((entry (org-capture-select-template keys)))
    (cond
     ((equal entry "C")
      (customize-variable 'org-capture-templates))
     ((equal entry "q")
      (user-error "Abort"))
     (t
      (let ((f (nth 2 entry)))
        (if (not f) (error "Missing function specification.")
          (if (commandp f) (call-interactively f)
            (if (functionp f) (funcall f)
              (error "Invalid function specification.")))))))))

(defun org-capture-some-menu ()
  (interactive)
  (let ((org-capture-templates
         `(("F" "Functions")
           ("Fh" "Hello World"
            (lambda ()
              (message "Hello, World")))
           ("Ff" "Find file" ,(function find-file)))))
    (org-menu)))

(define-key global-map (kbd "C-S-m") #'org-capture-some-menu)
#+end_src

Instead of hardcoding the actual work in the conditional statement, there should
be a function to be called, so org-capture would setup its own work, some random
"exec" menu like here would setup its own and so on. I haven't look at other
parts of org you have mentioned, so I am not yet sure if the approach would work
for all the kids in the block. I don't think it would that much harder to
refactor this out, but I might be wrong, since I am not that familiar with org code.

Factoring this out of Org itself, as suggested by RMS in the link you posted
might be much more work though. I haven't looked at that, and question is if
that is really worth the effort? I would agree with him that things like
org-table and date/time handling would be great to have in entire Emacs, without
need to load org, at least bigger parts of it. If I remember well, table mode
started outside of org as its own minor mode and got merged into org.

> The above statement is a hint that patches are welcome :)

As said, I am not that well familiar with org in-depth, and with other places
that might need to be factored out, so I don't promise anything. Initially I
just got a quick idea while working on a project of mine with org-capture, and
hacked the 'org-capture' function to implement my idea :).

/a

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

> Instead of hardcoding the actual work in the conditional statement, there should
> be a function to be called, so org-capture would setup its own work, some random
> "exec" menu like here would setup its own and so on. I haven't look at other
> parts of org you have mentioned, so I am not yet sure if the approach would work
> for all the kids in the block. I don't think it would that much harder to
> refactor this out, but I might be wrong, since I am not that familiar with org code.

There are several slightly more complex things in org-agenda (you can
set extra switches in agenda menu to affect the subsequent command - see
agenda restriction), but please by all means go ahead if you have
interest in this area.

> Factoring this out of Org itself, as suggested by RMS in the link you posted
> might be much more work though. I haven't looked at that, and question is if
> that is really worth the effort? I would agree with him that things like
> org-table and date/time handling would be great to have in entire Emacs, without
> need to load org, at least bigger parts of it. If I remember well, table mode
> started outside of org as its own minor mode and got merged into org.

This is not an immediate goal. Rather a justification that we generally
aim for a more modular code structure.

>> The above statement is a hint that patches are welcome :)
>
> As said, I am not that well familiar with org in-depth, and with other places
> that might need to be factored out, so I don't promise anything. Initially I
> just got a quick idea while working on a project of mine with org-capture, and
> hacked the 'org-capture' function to implement my idea :).

Feel free at ask anything if you encounter difficulties. It is not
always trivial to convert something that works for you personally into
something suitable for all the people using Org.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Ihor Radchenko <yantar92@gmail.com> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>> Instead of hardcoding the actual work in the conditional statement, there should
>> be a function to be called, so org-capture would setup its own work, some random
>> "exec" menu like here would setup its own and so on. I haven't look at other
>> parts of org you have mentioned, so I am not yet sure if the approach would work
>> for all the kids in the block. I don't think it would that much harder to
>> refactor this out, but I might be wrong, since I am not that familiar with org code.
>
> There are several slightly more complex things in org-agenda (you can
> set extra switches in agenda menu to affect the subsequent command - see
> agenda restriction), but please by all means go ahead if you have
> interest in this area.
>
>> Factoring this out of Org itself, as suggested by RMS in the link you posted
>> might be much more work though. I haven't looked at that, and question is if
>> that is really worth the effort? I would agree with him that things like
>> org-table and date/time handling would be great to have in entire Emacs, without
>> need to load org, at least bigger parts of it. If I remember well, table mode
>> started outside of org as its own minor mode and got merged into org.
>
> This is not an immediate goal. Rather a justification that we generally
> aim for a more modular code structure.
>
>>> The above statement is a hint that patches are welcome :)
>>
>> As said, I am not that well familiar with org in-depth, and with other places
>> that might need to be factored out, so I don't promise anything. Initially I
>> just got a quick idea while working on a project of mine with org-capture, and
>> hacked the 'org-capture' function to implement my idea :).
>
> Feel free at ask anything if you encounter difficulties. It is not
> always trivial to convert something that works for you personally into
> something suitable for all the people using Org.

Thank you for the help, I'll come back if I have to ask something for sure. I
have looked little bit into org-agenda-capture and org-mks that does the real
work, but I will have to familiarize myself more with org-agenda and other
places before I dare to suggest something.

best regards
/a

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 30/05/2022 09:04, Arthur Miller wrote:
> Ihor Radchenko writes:
>> Arthur Miller writes:
>>
>>> Simplicity comes from the org-templates. Me, and I guess other people are
>>> familiar with org-catpure templates already, and I mean, can it be simpler to
>>> create a menu of choices then a simple list:
>>>
>>> '(("key1" "label1" exec (lambda () ...))
>>>    ("key2" "label2" exec (labmda () ...))
>>>     ...
>>>     )
> 
> I am not really familiar with those other dialogues but org-capture, so I only
> had that one in the mind.

There is `org-insert-structure-template' C-c C-,

Doesn't the following code do what you are asking for?

(let ((entry (org-mks `(("a" "A entry" ,#'identity)
		       ("b" "B entry" ,(lambda (x) (* 2 x))))
		     "A or B")))
   (funcall (nth 2 entry) 10))

RE: Proposal: 'executable' org-capture-templaes

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 1468 bytes --]

Hi Max, thank you for the answer.

I have abandoned the idea 😀. Yes, org-mks is the one that does the lifting, you can see my other answers to Ihor., where I am playing with an example "usage pattern", that uses org-select-template which uses org-mks underto achieve a similar goal as you present here.

Sorry for a bad formatting, I am typing this from my phone.


-------- Originalmeddelande --------
Från: Max Nikulin <manikulin@gmail.com>
Datum: 2022-05-31 18:37 (GMT+01:00)
Till: Arthur Miller <arthur.miller@live.com>
Kopia: emacs-orgmode@gnu.org
Ämne: Re: Proposal: 'executable' org-capture-templaes

On 30/05/2022 09:04, Arthur Miller wrote:
> Ihor Radchenko writes:
>> Arthur Miller writes:
>>
>>> Simplicity comes from the org-templates. Me, and I guess other people are
>>> familiar with org-catpure templates already, and I mean, can it be simpler to
>>> create a menu of choices then a simple list:
>>>
>>> '(("key1" "label1" exec (lambda () ...))
>>>    ("key2" "label2" exec (labmda () ...))
>>>     ...
>>>     )
>
> I am not really familiar with those other dialogues but org-capture, so I only
> had that one in the mind.

There is `org-insert-structure-template' C-c C-,

Doesn't the following code do what you are asking for?

(let ((entry (org-mks `(("a" "A entry" ,#'identity)
                       ("b" "B entry" ,(lambda (x) (* 2 x))))
                     "A or B")))
   (funcall (nth 2 entry) 10))

[-- Attachment #2: Type: text/html, Size: 2715 bytes --]

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Ihor Radchenko <yantar92@gmail.com> writes:

>>> The above statement is a hint that patches are welcome :)
>>
>> As said, I am not that well familiar with org in-depth, and with other places
>> that might need to be factored out, so I don't promise anything. Initially I
>> just got a quick idea while working on a project of mine with org-capture, and
>> hacked the 'org-capture' function to implement my idea :).
>
> Feel free at ask anything if you encounter difficulties. It is not
> always trivial to convert something that works for you personally into
> something suitable for all the people using Org.

Hello again; I have been playing with this, and so far the biggest problem is
that I get too many ideas as longer I refactor the existing code :).

However, I have a question: would it be acceptable for org-capture to change the
input model?

I have implemented more flexible replacement for org-mks, and refactored it out
of org-mode under new nick 'quick-menu' to indicate that it does not pull any of
org mode with it. I have one implementation based on "read-key" function from
org-macs and used in original org-mks. A proptotype works, but I am not done
completely.

However before I continue, I am thinking of ditching the 'read-key' completely
and switching to "standard" Emacs way of implementing interactivity via mode and
mode-map. I am currently playing with such implementation, which to me appears
both simpler (code reduction) and more flexible, but it does change the mental
model of how clients of org-mks are used, for example org-capture.

I don't think it should be a deal-breaker, but that is my personal opinion, so I
would like to hear if that change would be acceptable or not?

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

> Hello again; I have been playing with this, and so far the biggest problem is
> that I get too many ideas as longer I refactor the existing code :).

:)

> However, I have a question: would it be acceptable for org-capture to change the
> input model?
>
> I have implemented more flexible replacement for org-mks, and refactored it out
> of org-mode under new nick 'quick-menu' to indicate that it does not pull any of
> org mode with it. I have one implementation based on "read-key" function from
> org-macs and used in original org-mks. A proptotype works, but I am not done
> completely.

This sounds promising.

> However before I continue, I am thinking of ditching the 'read-key' completely
> and switching to "standard" Emacs way of implementing interactivity via mode and
> mode-map. I am currently playing with such implementation, which to me appears
> both simpler (code reduction) and more flexible, but it does change the mental
> model of how clients of org-mks are used, for example org-capture.
>
> I don't think it should be a deal-breaker, but that is my personal opinion, so I
> would like to hear if that change would be acceptable or not?

Could you provide a bit more details? How exactly will the usage differ
from read-key?

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 04/06/2022 22:35, Arthur Miller wrote:
 >
> However before I continue, I am thinking of ditching the 'read-key' completely
> and switching to "standard" Emacs way of implementing interactivity via mode and
> mode-map. I am currently playing with such implementation, which to me appears
> both simpler (code reduction) and more flexible, but it does change the mental
> model of how clients of org-mks are used, for example org-capture.

Frankly speaking, I am quite confused concerning what you are trying to 
do in particular. At some moment I had an impression that you were going 
to factor out of `org-capture' the menu that is already a separate 
function `org-mks'.

While I appreciate the goal to improve `org-capture' behavior, I have 
some warnings.

Interface is blocking for purpose. Capture has single-task workflow 
currently. Capture data are stored in global variables, so parallel 
captures may cause problems. Likely it is assumed that a user quickly 
selects template and necessary data are added to the target document 
buffer. Unsure if some intermediate persistent store would be an 
improvement.

The following complain is mainly related to selection of a window to 
show the menu, but it should have in mind that some people use Emacs as 
window manager and menu should not hide windows related to current activity.

Eric S Fraga. Re: Bug: org-no-popups disregards 
display-buffer-fallback-action. Mon, 15 Nov 2021 09:57:46 +0000. 
https://list.orgmode.org/87fsrxeo6d.fsf@ucl.ac.uk

Likely nobody performed any steps toward `transient' as the interface, 
but due to such requests it would be nice to have possibility to switch 
between menu implementations.

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Max Nikulin <manikulin@gmail.com> writes:

> On 04/06/2022 22:35, Arthur Miller wrote:
>>
>> However before I continue, I am thinking of ditching the 'read-key' completely
>> and switching to "standard" Emacs way of implementing interactivity via mode and
>> mode-map. I am currently playing with such implementation, which to me appears
>> both simpler (code reduction) and more flexible, but it does change the mental
>> model of how clients of org-mks are used, for example org-capture.
>
> Frankly speaking, I am quite confused concerning what you are trying to do in
> particular. At some moment I had an impression that you were going to factor out
> of `org-capture' the menu that is already a separate function `org-mks'.

From the beginning I relized I can easily create menus with org-capture, bu just
definiing org-templates, which are simply lists, and wanted to generalize the
org-capture to create menus that can execute ordinary functions, which 'exec'
keyword did. After input from Ihor I agree that it isn't the best way, and was
able to refactor org-mks to create a menu where I can execute any lisp form,
when it comes in a list like this : ("h" "hello-word" (message "Hello,
World")), where third element is just a lisp form. I have something like this:

#+begin_src emacs-lisp

(defun demo1 ()
  "Simple illustration to recreate org-capture menu (visually only)."
  (interactive)
  (let ((quick-menu-key-decorator-chars "[]")
        (return
         (quick-menu
          ;; table
          test-templates
          ;; description
          '(:label "*Quick Select*"
                   :text "Select a capture template\n=========================")
          ;; more tables
          '(("C" "Customize org-capture-templates"
             (customize-variable 'org-capture-templates))
            ("q" "Abort" (user-error "Abort"))))))
    (if (called-interactively-p 'interactive)
        (message "%S" return)
      return)))

(defun demo3 ()
  "Illustrate nested menus, unicode separator and alternative decorator."
  (interactive)
  (let ((quick-menu-key-decorator-chars "<>")
        (quick-menu-vertical-separator ?─))
    (quick-menu
     ;; table
     '(("g" "Greetings")
       ("gh" "Hello, World!" (message "Hello, World!"))
       ("gb" "Bar" (message "Hello, Bar!")))
     ;; description
     nil
     ;; more tables
     '(("f" "Functions")
       ("ff" "Find File" (call-interactively #'find-file))
       ("fo" "Open File" (flet ((next-read-file-uses-dialog-p () t))
                           (call-interactively 'find-file))))
     '(("q" "Abort" (user-error "Abort"))))))
     
"quick-menu" is my refactoring of org-mks, definition looks like this:



(defun quick-menu (table &optional description &rest tables)
  "Select a member of an alist with multiple keys.

TABLE is an alist which should contain entries where the car is a string.
There should be two types of entries.

1. prefix descriptions like (\"a\" \"Description\")
   This indicates that `a' is a prefix key for multi-letter selection, and
   that there are entries following with keys like \"ab\", \"ax\"...

2. Select-able members must have more than two elements, with the first
   being the string of keys that lead to selecting it, and the second a
   short description string of the item. 

The command will then make a temporary buffer listing all entries
that can be selected with a single key, and all the single key
prefixes.  When you press the key for a single-letter entry, it is selected.
When you press a prefix key, the commands (and maybe further prefixes)
under this key will be shown and offered for selection.

DESCRIPTON is a property list containing following members:

:text          a string placed over the selection in the buffer.
:label         a string used for the selections buffer name.
:prompt        a string used when prompting for a key.
:always        when `t', this menu is shown; even descended into submenus
:horizontal    when `t', if multiple menus are present they are rendered from
left to right, otherwise from top to bottom.
:key-decorator a two-character string used to decorate command characters. When
this string is specified, it will take precedence over the global variable
`quick-menu-key-decorator-chars'.

TABLES are additional menus in the same format as TABLE. If there are more
then one menus, they will be separated by a separator line rendered with
character as specified in `quick-menu-vertical-separator'")

#+end_src

I have paramterized decorator character for shortcut keys as they appear in the
buffer, org-capture uses "[]", as well as menu separator, which is currently
hard-coded in org-capture, and I am currently trying to implement horizontal
layout, where menus are stacked from left to right. I also have a not so nice
bug when drawing nested menu that it leaves undesired space where menus not
visible after descension into current are; I have to redraw the entire menu but
haven't yet implemented it so I don't want to post a demo yet. But before I fix
redrawing and implement horizontal layout, I would like to switch to the
different model of interaction and use ordinary mode map idioms instead of
blocking read key. Since I need to rework current prototype for the re-drawing
part, I can as well rework it to skip read-key at the same time.

> Interface is blocking for purpose. Capture has single-task workflow
> currently.

Yes, I am aware of that. That is why I ask if it would be acceptable to switch
away from non-blocking interface. I totally agree that capture is a single-task
workflow, however more generalized menu should allow for other applications as
well. We can still achieve single-task workflow with org-capture, by simply not
allowing more then one org-capture menu buffer at a time, it is just that it
won't block entire Emacs. So one could have more than one application of
quick-menu, where for example org-capture is one application, some imaingary
greating-app would be a different application, etc.

>            Capture data are stored in global variables, so parallel captures may
> cause problems. Likely it is assumed that a user quickly selects template and
> necessary data are added to the target document buffer.

Exactly.  It is important that org-capture is one capture at the time so people
don't mess their note files, agenda files etc.

>                                                         Unsure if some
> intermediate persistent store would be an improvement.

Not sure what you mean here, but I don't plan to change anything in org-capture
itself, it should still be one-task at the time.

> The following complain is mainly related to selection of a window to show the
> menu, but it should have in mind that some people use Emacs as window manager
> and menu should not hide windows related to current activity.
>
> Eric S Fraga. Re: Bug: org-no-popups disregards
> display-buffer-fallback-action. Mon, 15 Nov 2021 09:57:46
> +0000. https://list.orgmode.org/87fsrxeo6d.fsf@ucl.ac.uk
>
> Likely nobody performed any steps toward `transient' as the interface, but due
> to such requests it would be nice to have possibility to switch between menu
> implementations.

I am not building some generalized framework, as I said in my first respone to
Ihor :-). 

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Ihor Radchenko <yantar92@gmail.com> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>> However before I continue, I am thinking of ditching the 'read-key' completely
>> and switching to "standard" Emacs way of implementing interactivity via mode and
>> mode-map. I am currently playing with such implementation, which to me appears
>> both simpler (code reduction) and more flexible, but it does change the mental
>> model of how clients of org-mks are used, for example org-capture.
>>
>> I don't think it should be a deal-breaker, but that is my personal opinion, so I
>> would like to hear if that change would be acceptable or not?
>
> Could you provide a bit more details? How exactly will the usage differ
> from read-key?

I just wrote a much more detailed descrpition of what I have done so far, and
some dificulties I have to solve before I continue, and some discussion of how
it might work in answer to Mikulins question and concerns.

Short here: it will be ordinary text buffer, read only of course, with its own
major mode derived from special mode and buffer local key maps, instead of major
mode global maps, so user can just press a key in the buffer itself instead of
being prompted.

Single task workflow, I believe, can be guaranteed by allowing
only one menu buffer per application, for example one org-capture menu at a
time, but multiple applications could work since they will have different named
buffers.

This is a suggestions. I really dislike the read-key implementation of org-mks,
I don't think it is very easy to hack it in order to extend it, but I don't know
if it is possible to block Emacs when using ordinary key map mechanism. If
someone knows how to do it, I am all ears :).

Hope it explains a bit.

Thanks for the help!

Re: Proposal: 'executable' org-capture-templaes

[Tim Cross]

Arthur Miller <arthur.miller@live.com> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> Arthur Miller <arthur.miller@live.com> writes:
>>
>>> However before I continue, I am thinking of ditching the 'read-key' completely
>>> and switching to "standard" Emacs way of implementing interactivity via mode and
>>> mode-map. I am currently playing with such implementation, which to me appears
>>> both simpler (code reduction) and more flexible, but it does change the mental
>>> model of how clients of org-mks are used, for example org-capture.
>>>
>>> I don't think it should be a deal-breaker, but that is my personal opinion, so I
>>> would like to hear if that change would be acceptable or not?
>>
>> Could you provide a bit more details? How exactly will the usage differ
>> from read-key?
>
> I just wrote a much more detailed descrpition of what I have done so far, and
> some dificulties I have to solve before I continue, and some discussion of how
> it might work in answer to Mikulins question and concerns.
>
> Short here: it will be ordinary text buffer, read only of course, with its own
> major mode derived from special mode and buffer local key maps, instead of major
> mode global maps, so user can just press a key in the buffer itself instead of
> being prompted.
>
> Single task workflow, I believe, can be guaranteed by allowing
> only one menu buffer per application, for example one org-capture menu at a
> time, but multiple applications could work since they will have different named
> buffers.
>
> This is a suggestions. I really dislike the read-key implementation of org-mks,
> I don't think it is very easy to hack it in order to extend it, but I don't know
> if it is possible to block Emacs when using ordinary key map mechanism. If
> someone knows how to do it, I am all ears :).
>
> Hope it explains a bit.
>
> Thanks for the help!


I'm not sure I really understand the exact goal you have here. To me, it
feels like yet another input selection/menu/completion scheme and I'm
not clear on how it will be an improvement or why do something
'different' in org compared to other modes etc. However, I also don't
have any problems using the existing capture interface, so perhaps I
just don't have the number or complexity of capture choices to expose
issues/limitations wiht the existing approach. 

The main 'concern' (well, not really a concern, but ....) I have is that
Emacs already has far too many solutions along this line, which makes it
harder to get a consistent UI. I also feel this is one of those areas
which appears to be really easy to 'fix' or improve, but also has a lot
of hidden complexity which only becomes evident once lots of different
users and workflows try to use it. 

One very big warning I would like to raise to ensure it is taken into
consideration is accessibility. This can have two significant effects
with respect to the types of things you are doing -

1. I am a vision impaired user. While considered legally to be blind, I
do have some very limited vision (less than 5%). I use very large fonts.
(assume a 25 x 80 screen). 

2. I use Emacspeak to provide text-to-speech support. Emacspeak works
primarily by adding 'advice' to key functions which take the core bit of
text and send it to a text-to-speech synthesizer to generate spoken
output. One thing which is important with any 'menu' or selection
solution is that we can move through the choices so that they get spoken
again i.e. review the choices. 

Choice review is very important, especially when there are lots of
choices or when there are nested choices. Some solutions are better at
this than others. Sometimes, this is only possible by selecting the
'prompt' window and using normal emacs navigation keys to move around
and get the options spoken - clunky, but usable. Others solutions are
structured such that when you move to a new option, it is spoken and you
can just move up/down or in/out of selections to hear them spoken. The
key point here is that in some situations, you may need to allow the
user to switch windows and then switch back (i.e. switch to the window
displaying the choices, move around to listen to them, switch back to
the minibuffer and enter the choice or provide key bindings which will
allow the 'choices' buffer to be scrolled up/down and have the contents
spoken without leaving the minibuffer etc. The key point is that for
blind and VI users, the ability to 'scan' the choices is more limited.
Often you will need to go down one selection tree, come back up and go
down a different branch. You cannot just glance at the screen and get an
overview which helps to give context. As an example, the org export
'menu' is not good for most blind/VI users. The choices quickly exceed
the number you can fit on a 25.80 screen and it is difficult to review
choices. 

One reason I like it when modes stick to core or built-in modes/packages
to provide UI elements is that typically, they will already have
emacspeak support. When a mode does something fundamentally different,
it requires someone to implement the advice to make it work with
Emacspeak. Most 'popular' approaches have already got support i.e. helm,
ivy, company, transient mode, vertico, embark, ido, etc. Some modes are
easier to support than others. 

It can be difficult for someone unfamiliar with accessibility
requirements to know how to ensure these are met and met adequately.
Unfortunately, there is no simple answer. However, when ti comes to
emacs, one approach which might help is to see if you can make your
implementation echo choices to the message buffer and/or echo area when
the user moves between options or requests a review of available
options. While not perfect, in general, if you can do this then it won't
be too difficult to add Emacspeak support. 

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 05/06/2022 22:07, Arthur Miller wrote:
> Max Nikulin writes:
> 
> After input from Ihor I agree that it isn't the best way, and was
> able to refactor org-mks to create a menu where I can execute any lisp form,
> when it comes in a list like this : ("h" "hello-word" (message "Hello,
> World")), where third element is just a lisp form. I have something like this:

This message is merely my opinion that you may disagree. I am not trying 
to prevent you from going forward.

 From my point of view current `org-mks' is more general giving you 
opportunity to treat extra elements as you wish. A thin wrapper allows 
to evaluate 3rd element of returned list. You have not convinced me that 
built-in executable form is the essential feature.

> (defun demo3 ()
>    "Illustrate nested menus, unicode separator and alternative decorator."
>    (interactive)
>    (let ((quick-menu-key-decorator-chars "<>")
>          (quick-menu-vertical-separator ?─))
>      (quick-menu
>       ;; table
>       '(("g" "Greetings")
>         ("gh" "Hello, World!" (message "Hello, World!"))
>         ("gb" "Bar" (message "Hello, Bar!")))
>       ;; description
>       nil
>       ;; more tables
>       '(("f" "Functions")
>         ("ff" "Find File" (call-interactively #'find-file))
>         ("fo" "Open File" (flet ((next-read-file-uses-dialog-p () t))
>                             (call-interactively 'find-file))))
>       '(("q" "Abort" (user-error "Abort"))))))

It is tightly based on org-mks, but actually it is way to represent list 
of choices with some hints to possible hierarchy and accelerator keys. 
Quite similar list may be fed to completion read or represented as a 
hierarchical GUI menu. The only specific is "always visible" elements 
like "abort". When Ubuntu used Unity desktop their had a nice feature of 
searching in application menu.

There should be an extension point that allows users to replace 
representation e.g. to improve accessibility.

> DESCRIPTON is a property list containing following members:
...
> :horizontal    when `t', if multiple menus are present they are rendered from
> left to right, otherwise from top to bottom.

It may depend on whether a window created to display menu is tall and 
narrow or wide.

> I have paramterized decorator character for shortcut keys as they appear in the
> buffer, org-capture uses "[]", as well as menu separator, which is currently
> hard-coded in org-capture,

I agree that org-mks may have additional argument to specify menu 
decoration.

> Exactly.  It is important that org-capture is one capture at the time so people
> don't mess their note files, agenda files etc.
> 
>>                                                          Unsure if some
>> intermediate persistent store would be an improvement.
> 
> Not sure what you mean here, but I don't plan to change anything in org-capture
> itself, it should still be one-task at the time.

Changing interface to less degree of blocking may be confusing for 
users. Capture template selection menu may be displayed in another frame 
hidden behind other application, on another monitor, on another virtual 
desktop, so invisible. So a user earlier distracted by something urgent 
may try to start another capture. Captures may be initiated from other 
applications using org-protocol.

To avoid data loss while other capture is in progress, the data of next 
capture may be temporary saved to some place till the user pops it from 
the queue. I mentioned persistence since something may unexpectedly 
crash, so it should be possible to resurrect enqueued captures in next 
Emacs session.

>> Likely nobody performed any steps toward `transient' as the interface, but due
>> to such requests it would be nice to have possibility to switch between menu
>> implementations.
> 
> I am not building some generalized framework, as I said in my first respone to
> Ihor :-).

I am not requesting for a framework, I mean API compatible with other 
frameworks to let user choose their preferred ones.

So tunables to control decoration sounds interesting. I am in doubts 
concerning fixing some element as executable. Mode-map instead of 
minibuffer may be a great step to more convenient interface (it 
resembles help buffers), but may require more changes in functions that 
do actual job. From other messages on this list my impression is that 
API should be designed having in mind flexibility and other UI packages.

Re: Proposal: 'executable' org-capture-templaes

[Samuel Wales]

i must be confused.  menus aside, you can currently capture and it
uses the org forest itself, so it is both crash-proof and snappy.  and
you can yakshave as much as you want, starting a capture while doing a
capture.  those were design goals.

you can even be in the middle of a capture, get distracted, navigate
around your forest, and find where you are in the middle of a capture.
goes with the original crash-proof and yakshave and snappy
use-original-buffer design goal.

so are we talking about menus then?  is there truly a need to make
/menu state/ persistent or yakshaveable?


On 6/6/22, Max Nikulin <manikulin@gmail.com> wrote:
> On 05/06/2022 22:07, Arthur Miller wrote:
>> Max Nikulin writes:
>>
>> After input from Ihor I agree that it isn't the best way, and was
>> able to refactor org-mks to create a menu where I can execute any lisp
>> form,
>> when it comes in a list like this : ("h" "hello-word" (message "Hello,
>> World")), where third element is just a lisp form. I have something like
>> this:
>
> This message is merely my opinion that you may disagree. I am not trying
> to prevent you from going forward.
>
>  From my point of view current `org-mks' is more general giving you
> opportunity to treat extra elements as you wish. A thin wrapper allows
> to evaluate 3rd element of returned list. You have not convinced me that
> built-in executable form is the essential feature.
>
>> (defun demo3 ()
>>    "Illustrate nested menus, unicode separator and alternative
>> decorator."
>>    (interactive)
>>    (let ((quick-menu-key-decorator-chars "<>")
>>          (quick-menu-vertical-separator ?─))
>>      (quick-menu
>>       ;; table
>>       '(("g" "Greetings")
>>         ("gh" "Hello, World!" (message "Hello, World!"))
>>         ("gb" "Bar" (message "Hello, Bar!")))
>>       ;; description
>>       nil
>>       ;; more tables
>>       '(("f" "Functions")
>>         ("ff" "Find File" (call-interactively #'find-file))
>>         ("fo" "Open File" (flet ((next-read-file-uses-dialog-p () t))
>>                             (call-interactively 'find-file))))
>>       '(("q" "Abort" (user-error "Abort"))))))
>
> It is tightly based on org-mks, but actually it is way to represent list
> of choices with some hints to possible hierarchy and accelerator keys.
> Quite similar list may be fed to completion read or represented as a
> hierarchical GUI menu. The only specific is "always visible" elements
> like "abort". When Ubuntu used Unity desktop their had a nice feature of
> searching in application menu.
>
> There should be an extension point that allows users to replace
> representation e.g. to improve accessibility.
>
>> DESCRIPTON is a property list containing following members:
> ...
>> :horizontal    when `t', if multiple menus are present they are rendered
>> from
>> left to right, otherwise from top to bottom.
>
> It may depend on whether a window created to display menu is tall and
> narrow or wide.
>
>> I have paramterized decorator character for shortcut keys as they appear
>> in the
>> buffer, org-capture uses "[]", as well as menu separator, which is
>> currently
>> hard-coded in org-capture,
>
> I agree that org-mks may have additional argument to specify menu
> decoration.
>
>> Exactly.  It is important that org-capture is one capture at the time so
>> people
>> don't mess their note files, agenda files etc.
>>
>>>                                                          Unsure if some
>>> intermediate persistent store would be an improvement.
>>
>> Not sure what you mean here, but I don't plan to change anything in
>> org-capture
>> itself, it should still be one-task at the time.
>
> Changing interface to less degree of blocking may be confusing for
> users. Capture template selection menu may be displayed in another frame
> hidden behind other application, on another monitor, on another virtual
> desktop, so invisible. So a user earlier distracted by something urgent
> may try to start another capture. Captures may be initiated from other
> applications using org-protocol.
>
> To avoid data loss while other capture is in progress, the data of next
> capture may be temporary saved to some place till the user pops it from
> the queue. I mentioned persistence since something may unexpectedly
> crash, so it should be possible to resurrect enqueued captures in next
> Emacs session.
>
>>> Likely nobody performed any steps toward `transient' as the interface,
>>> but due
>>> to such requests it would be nice to have possibility to switch between
>>> menu
>>> implementations.
>>
>> I am not building some generalized framework, as I said in my first
>> respone to
>> Ihor :-).
>
> I am not requesting for a framework, I mean API compatible with other
> frameworks to let user choose their preferred ones.
>
> So tunables to control decoration sounds interesting. I am in doubts
> concerning fixing some element as executable. Mode-map instead of
> minibuffer may be a great step to more convenient interface (it
> resembles help buffers), but may require more changes in functions that
> do actual job. From other messages on this list my impression is that
> API should be designed having in mind flexibility and other UI packages.
>
>
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: Proposal: 'executable' org-capture-templaes

[Samuel Wales]

[i put an unnecessary and not really meant word truly there.
accessbililty reasons could of course make it necessary.]

i think it is needed to do accessibility and great to do factoring.
shows our maturity as a project and developers.

[accessbilty/refactoring aside, i want to say i really like many
aspects of the care taken to make many of our menus.  e.g. kw
selection or tag selection use colors, have low key count.  date
selection too.  i wonder how much of this will survive?]


On 6/6/22, Samuel Wales <samologist@gmail.com> wrote:
> i must be confused.  menus aside, you can currently capture and it
> uses the org forest itself, so it is both crash-proof and snappy.  and
> you can yakshave as much as you want, starting a capture while doing a
> capture.  those were design goals.
>
> you can even be in the middle of a capture, get distracted, navigate
> around your forest, and find where you are in the middle of a capture.
> goes with the original crash-proof and yakshave and snappy
> use-original-buffer design goal.
>
> so are we talking about menus then?  is there truly a need to make
> /menu state/ persistent or yakshaveable?
>
>
> On 6/6/22, Max Nikulin <manikulin@gmail.com> wrote:
>> On 05/06/2022 22:07, Arthur Miller wrote:
>>> Max Nikulin writes:
>>>
>>> After input from Ihor I agree that it isn't the best way, and was
>>> able to refactor org-mks to create a menu where I can execute any lisp
>>> form,
>>> when it comes in a list like this : ("h" "hello-word" (message "Hello,
>>> World")), where third element is just a lisp form. I have something like
>>> this:
>>
>> This message is merely my opinion that you may disagree. I am not trying
>> to prevent you from going forward.
>>
>>  From my point of view current `org-mks' is more general giving you
>> opportunity to treat extra elements as you wish. A thin wrapper allows
>> to evaluate 3rd element of returned list. You have not convinced me that
>> built-in executable form is the essential feature.
>>
>>> (defun demo3 ()
>>>    "Illustrate nested menus, unicode separator and alternative
>>> decorator."
>>>    (interactive)
>>>    (let ((quick-menu-key-decorator-chars "<>")
>>>          (quick-menu-vertical-separator ?─))
>>>      (quick-menu
>>>       ;; table
>>>       '(("g" "Greetings")
>>>         ("gh" "Hello, World!" (message "Hello, World!"))
>>>         ("gb" "Bar" (message "Hello, Bar!")))
>>>       ;; description
>>>       nil
>>>       ;; more tables
>>>       '(("f" "Functions")
>>>         ("ff" "Find File" (call-interactively #'find-file))
>>>         ("fo" "Open File" (flet ((next-read-file-uses-dialog-p () t))
>>>                             (call-interactively 'find-file))))
>>>       '(("q" "Abort" (user-error "Abort"))))))
>>
>> It is tightly based on org-mks, but actually it is way to represent list
>> of choices with some hints to possible hierarchy and accelerator keys.
>> Quite similar list may be fed to completion read or represented as a
>> hierarchical GUI menu. The only specific is "always visible" elements
>> like "abort". When Ubuntu used Unity desktop their had a nice feature of
>> searching in application menu.
>>
>> There should be an extension point that allows users to replace
>> representation e.g. to improve accessibility.
>>
>>> DESCRIPTON is a property list containing following members:
>> ...
>>> :horizontal    when `t', if multiple menus are present they are rendered
>>> from
>>> left to right, otherwise from top to bottom.
>>
>> It may depend on whether a window created to display menu is tall and
>> narrow or wide.
>>
>>> I have paramterized decorator character for shortcut keys as they appear
>>> in the
>>> buffer, org-capture uses "[]", as well as menu separator, which is
>>> currently
>>> hard-coded in org-capture,
>>
>> I agree that org-mks may have additional argument to specify menu
>> decoration.
>>
>>> Exactly.  It is important that org-capture is one capture at the time so
>>> people
>>> don't mess their note files, agenda files etc.
>>>
>>>>                                                          Unsure if some
>>>> intermediate persistent store would be an improvement.
>>>
>>> Not sure what you mean here, but I don't plan to change anything in
>>> org-capture
>>> itself, it should still be one-task at the time.
>>
>> Changing interface to less degree of blocking may be confusing for
>> users. Capture template selection menu may be displayed in another frame
>> hidden behind other application, on another monitor, on another virtual
>> desktop, so invisible. So a user earlier distracted by something urgent
>> may try to start another capture. Captures may be initiated from other
>> applications using org-protocol.
>>
>> To avoid data loss while other capture is in progress, the data of next
>> capture may be temporary saved to some place till the user pops it from
>> the queue. I mentioned persistence since something may unexpectedly
>> crash, so it should be possible to resurrect enqueued captures in next
>> Emacs session.
>>
>>>> Likely nobody performed any steps toward `transient' as the interface,
>>>> but due
>>>> to such requests it would be nice to have possibility to switch between
>>>> menu
>>>> implementations.
>>>
>>> I am not building some generalized framework, as I said in my first
>>> respone to
>>> Ihor :-).
>>
>> I am not requesting for a framework, I mean API compatible with other
>> frameworks to let user choose their preferred ones.
>>
>> So tunables to control decoration sounds interesting. I am in doubts
>> concerning fixing some element as executable. Mode-map instead of
>> minibuffer may be a great step to more convenient interface (it
>> resembles help buffers), but may require more changes in functions that
>> do actual job. From other messages on this list my impression is that
>> API should be designed having in mind flexibility and other UI packages.
>>
>>
>>
>
>
> --
> The Kafka Pandemic
>
> A blog about science, health, human rights, and misopathy:
> https://thekafkapandemic.blogspot.com
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

>> Could you provide a bit more details? How exactly will the usage differ
>> from read-key?
>
> Short here: it will be ordinary text buffer, read only of course, with its own
> major mode derived from special mode and buffer local key maps, instead of major
> mode global maps, so user can just press a key in the buffer itself instead of
> being prompted.

Sounds reasonable.

> Single task workflow, I believe, can be guaranteed by allowing
> only one menu buffer per application, for example one org-capture menu at a
> time, but multiple applications could work since they will have different named
> buffers.

Again, reasonable. Though I did not see how it is possible in your demo.

> This is a suggestions. I really dislike the read-key implementation of org-mks,
> I don't think it is very easy to hack it in order to extend it, but I don't know
> if it is possible to block Emacs when using ordinary key map mechanism. If
> someone knows how to do it, I am all ears :).

There were other people who really dislike read-key implementation.
Notably Jean Louis and Eduardo Ochs.

A kind of hack you are asking for can be binding every other key to
function that aborts the menu. It will not restrict users, say, from
creating another frame. But otherwise it will pretty much work like
read-key.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

> The command will then make a temporary buffer listing all entries
> that can be selected with a single key, and all the single key
> prefixes.  When you press the key for a single-letter entry, it is selected.
> When you press a prefix key, the commands (and maybe further prefixes)
> under this key will be shown and offered for selection.

Is there a way to select an entry without exiting the menu?
What I have in mind is C-c C-e interface where we have on/off switches
to control other sub-menus. See, Body-only, Export scope, etc

Also, note the fontification. I guess it will also be possible in your
menu generator if we provide propertized strings.

Selecting a group may also echo the description to minibuffer (as Tim
Cross suggested).

Finally, you are only allowing a single key selectors. What about C-key
or M-key?

> I have paramterized decorator character for shortcut keys as they appear in the
> buffer, org-capture uses "[]", as well as menu separator, which is currently
> hard-coded in org-capture

More generally, it can be a function to create the decorator.

> , and I am currently trying to implement horizontal
> layout, where menus are stacked from left to right.

Please refer to org-fast-tag-selection and org-fast-todo-selection.

> I also have a not so nice
> bug when drawing nested menu that it leaves undesired space where menus not
> visible after descension into current are; I have to redraw the entire menu but
> haven't yet implemented it so I don't want to post a demo yet. But before I fix
> redrawing and implement horizontal layout, I would like to switch to the
> different model of interaction and use ordinary mode map idioms instead of
> blocking read key. Since I need to rework current prototype for the re-drawing
> part, I can as well rework it to skip read-key at the same time.

Since no details are given, I cannot provide any help here. So, good
luck :)

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

> I'm not sure I really understand the exact goal you have here. To me, it
> feels like yet another input selection/menu/completion scheme and I'm
> not clear on how it will be an improvement or why do something
> 'different' in org compared to other modes etc. However, I also don't
> have any problems using the existing capture interface, so perhaps I
> just don't have the number or complexity of capture choices to expose
> issues/limitations wiht the existing approach. 
>
> The main 'concern' (well, not really a concern, but ....) I have is that
> Emacs already has far too many solutions along this line, which makes it
> harder to get a consistent UI. I also feel this is one of those areas
> which appears to be really easy to 'fix' or improve, but also has a lot
> of hidden complexity which only becomes evident once lots of different
> users and workflows try to use it. 

Let me clarify my vision of this thread.

1. Arthur is interested to implement something similar to org-capture
   menu. We can help him with this regardless of our stance on whether
   to include the result into Org.

2. Org mode has multiple implementations of menu. Menus for org-capture,
   org-export, org-todo, org-set-tags-command, and org-agenda are all
   implemented independently creating redundancy in our codebase.

3. Because of the redundancy, there has been a proposal in the past to
   switch from our existing menus to transient. However, it will be a
   breaking change. We would prefer to support old menus as well (at
   least for a handful of years)

4. If Arthur's implementation turns out sufficient to replicate the
   "look and feel" or our existing menus, we can use it instead. This
   will at least reduce the amount of menu code in Org. We can also take
   this opportunity to make the menu backend selectable: the old menus,
   Arthur's menu backend, transient. Then, we can eventually drop the
   old menus backend and leave Arthur's + transient. They will be much
   easier to maintain, especially if Arthur's implementation can be
   distributed as separate package (even if not, one menu implementation
   is easier than multiple that we have now).

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Samuel Wales <samologist@gmail.com> writes:

> [accessbilty/refactoring aside, i want to say i really like many
> aspects of the care taken to make many of our menus.  e.g. kw
> selection or tag selection use colors, have low key count.  date
> selection too.  i wonder how much of this will survive?]

Everything must survive. We will try our best to get backwards
compatibility and will try our best to not remove features.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> I'm not sure I really understand the exact goal you have here. To me, it
>> feels like yet another input selection/menu/completion scheme and I'm
>> not clear on how it will be an improvement or why do something
>> 'different' in org compared to other modes etc. However, I also don't
>> have any problems using the existing capture interface, so perhaps I
>> just don't have the number or complexity of capture choices to expose
>> issues/limitations wiht the existing approach. 
>>
>> The main 'concern' (well, not really a concern, but ....) I have is that
>> Emacs already has far too many solutions along this line, which makes it
>> harder to get a consistent UI. I also feel this is one of those areas
>> which appears to be really easy to 'fix' or improve, but also has a lot
>> of hidden complexity which only becomes evident once lots of different
>> users and workflows try to use it. 
>
> Let me clarify my vision of this thread.
>
> 1. Arthur is interested to implement something similar to org-capture
>    menu. We can help him with this regardless of our stance on whether
>    to include the result into Org.
>
> 2. Org mode has multiple implementations of menu. Menus for org-capture,
>    org-export, org-todo, org-set-tags-command, and org-agenda are all
>    implemented independently creating redundancy in our codebase.
>
> 3. Because of the redundancy, there has been a proposal in the past to
>    switch from our existing menus to transient. However, it will be a
>    breaking change. We would prefer to support old menus as well (at
>    least for a handful of years)
>
> 4. If Arthur's implementation turns out sufficient to replicate the
>    "look and feel" or our existing menus, we can use it instead. This
>    will at least reduce the amount of menu code in Org. We can also take
>    this opportunity to make the menu backend selectable: the old menus,
>    Arthur's menu backend, transient. Then, we can eventually drop the
>    old menus backend and leave Arthur's + transient. They will be much
>    easier to maintain, especially if Arthur's implementation can be
>    distributed as separate package (even if not, one menu implementation
>    is easier than multiple that we have now).
>
Hi Ihor,

I think I totally get where your coming from and I agree with all
points. However, I don't quite get exactly what Arthur is proposing at a
concrete level. 

Overall, I guess my main concern is that this is one of those areas
where it looks deceptively easy to improve and it is only once you get
down into the weeds and start to see all the competing perspectives, you
realise how much more complicated it actually is. 

To some extent, it reminds me of what I always found so frustrating wiht
CL. In general, it was so easy to create a new library representing some
functionality that everyone did it. Instead of having one good solution,
we end up with 20 ok ones. Worse still, we end up with 10 different
implementations within the same code base. 

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Tim Cross <theophilusx@gmail.com> writes:

> I think I totally get where your coming from and I agree with all
> points. However, I don't quite get exactly what Arthur is proposing at a
> concrete level. 
>
> Overall, I guess my main concern is that this is one of those areas
> where it looks deceptively easy to improve and it is only once you get
> down into the weeds and start to see all the competing perspectives, you
> realise how much more complicated it actually is. 

You may or may not be right. It is not important in this case.

We can just let Arthur try and help him in the process.

If he manages to go through all the obstacles and develop something
equivalent to our existing menus (no need for anything more powerful or
"generic"), we can use it.

Let's not discourage him by imaginary difficulties, which may or may not
appear.

At the end, our existing menu code combined is not even that large. So,
developing equivalent should not be prohibitively hard. (I am not
talking about full-fledged menu library here. Arthur clearly stated that
it is not his intention).

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 07/06/2022 10:09, Samuel Wales wrote:
> i must be confused.  menus aside, you can currently capture and it
> uses the org forest itself, so it is both crash-proof and snappy.  and
> you can yakshave as much as you want, starting a capture while doing a
> capture.  those were design goals.
> 
> you can even be in the middle of a capture, get distracted, navigate
> around your forest, and find where you are in the middle of a capture.
> goes with the original crash-proof and yakshave and snappy
> use-original-buffer design goal.
> 
> so are we talking about menus then?  is there truly a need to make
> /menu state/ persistent or yakshaveable?

As soon as capture template is chosen, content is landed to the target 
file and may be autosaved. I do not expect problems here.

However if two org-protocol handlers are launched without specified 
template then behavior of Org becomes confusing. I meant this case. 
Currently reading key from minibuffer serves as a kind of 
synchronization tool.

Imagine what would happen if Emacs decided to show several capture menus 
with keymap non-blocking interface in different virtual desktops. 
Capture data should be saved somewhere till the user would discover 
initially hidden menu.

For me applications spread over 4 virtual desktops and more than one 
emacs frame is a usual case. I have not realized yet exact conditions 
when capture buffer appears in the frame other than one displayed on the 
active virtual desktop.

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> so are we talking about menus then?  is there truly a need to make
>> /menu state/ persistent or yakshaveable?
>
> As soon as capture template is chosen, content is landed to the target 
> file and may be autosaved. I do not expect problems here.
>
> However if two org-protocol handlers are launched without specified 
> template then behavior of Org becomes confusing. I meant this case. 
> Currently reading key from minibuffer serves as a kind of 
> synchronization tool.
>
> Imagine what would happen if Emacs decided to show several capture menus 
> with keymap non-blocking interface in different virtual desktops. 
> Capture data should be saved somewhere till the user would discover 
> initially hidden menu.

Note that there is not much happening when capture menu is called. Only
the link is stored into link ting. Otherwise, no capture data is
altered. All the fragile staff is happening after selecting capture
template.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 7074 bytes --]

Ihor Radchenko <yantar92@gmail.com> writes:

> Tim Cross <theophilusx@gmail.com> writes:
>
>> I'm not sure I really understand the exact goal you have here. To me, it
>> feels like yet another input selection/menu/completion scheme and I'm
>> not clear on how it will be an improvement or why do something
>> 'different' in org compared to other modes etc. However, I also don't
>> have any problems using the existing capture interface, so perhaps I
>> just don't have the number or complexity of capture choices to expose
>> issues/limitations wiht the existing approach. 
>>
>> The main 'concern' (well, not really a concern, but ....) I have is that
>> Emacs already has far too many solutions along this line, which makes it
>> harder to get a consistent UI. I also feel this is one of those areas
>> which appears to be really easy to 'fix' or improve, but also has a lot
>> of hidden complexity which only becomes evident once lots of different
>> users and workflows try to use it. 
>
> Let me clarify my vision of this thread.
>
> 1. Arthur is interested to implement something similar to org-capture
>    menu. We can help him with this regardless of our stance on whether
>    to include the result into Org.
>
> 2. Org mode has multiple implementations of menu. Menus for org-capture,
>    org-export, org-todo, org-set-tags-command, and org-agenda are all
>    implemented independently creating redundancy in our codebase.
>
> 3. Because of the redundancy, there has been a proposal in the past to
>    switch from our existing menus to transient. However, it will be a
>    breaking change. We would prefer to support old menus as well (at
>    least for a handful of years)
>
> 4. If Arthur's implementation turns out sufficient to replicate the
>    "look and feel" or our existing menus, we can use it instead. This
>    will at least reduce the amount of menu code in Org. We can also take
>    this opportunity to make the menu backend selectable: the old menus,
>    Arthur's menu backend, transient. Then, we can eventually drop the
>    old menus backend and leave Arthur's + transient. They will be much
>    easier to maintain, especially if Arthur's implementation can be
>    distributed as separate package (even if not, one menu implementation
>    is easier than multiple that we have now).

Hello, and sorry for long time no hear ... thought I would had something last
weekend, but it took a bit longer time.

Anyway, I have been playing and testing a bit, and didn't want to prolong
discussion untill I have something to show. So here is a small prototype. It is
just a rough sketch of the idea.

The idea is simple: just ordinary keymap, with automated mode and keymap
creation from templates.

It uses simple template format to specify a key and a label to display in a
buffer for the user. It can either return the template back to some callback, or
it can use the 3rd argument as "executable" and wrap it in an interactive lambda
to tuck into the keymap. I think that it is the minimum required. Rest is a
boilerplate. It also puts declaration of gui and logic in same place (the
template).

For example org-capture defines its own template language, so it is just to give
the chosen template to org-capture. This is what org-mks does, pretty much. I
have just refactored the org-capture in an example to show that it is possible
to implement the equivalent with almost no changes, more than it does not use
org-mks under the hood. There is no code saving there.

However, when it comes to org-agenda, as I see from the current implementation
it does not use org-mks at all, but does something very similar on it's own,
with ui and logic hardcoded in `org-agenda-get-restriction-and-command'. In
this example the mode map approach seems slightly more convenient. I don't know,
in org-agenda-test, I haven't implemented all of org-agenda, restrictions,
prefixes and some other stuff, mostly because I don't really understand the
implementation. I didn't want to sitt too long and figure out how stuff works,
if the fundamental approach is not acceptable, but I have implemented quite few
of the menu choices, at least to show the pattern.

As said, it is just a rough sketch to illustrate the idea. I am not sure myself
if it is good idea or not. I have implemented it so it works with org-capture
templates, and I hope it wasn't too much of extra "customizations" tossed
in. "Horizontal" menu was needed to simulate org-agenda looks, otherwise the
code would be much smaller. Also to note is that the "logic" does not use
anything in buffer on display, so it would be possible for someone interested to
"rice" it up after the drawing is done, so the customization options could be
further reduced.

To answer some questions I have seen in mails, sorry for late answeres:

@Ihor
I really don't have problem with "read key". Originally I just wanted to extend
org-capture templates to do a bit extra :).

Actually org-mks and similar approach is really efficient in terms of
resource (no minor/major modes etc). It is only the thing if same framework is
to be used by non-modal applications too, than there have to be other way to
read user input, and since the other way are keymaps, read-key becomes redundant.

Sometimes, something like 'read-key' is what is needed, and sometimes that is
just enough. When I first thought of using org-capture templates for
"executable" definitions, I really didn't know how org-capture worked under the
hood. Max is correct about wrapper, that is how org-capture works. But since it
is so easy, we can also automate it and skip writing wrappers and lambdas every
time we use it. That is the idea behind the "default handler" in the
example.

Big difference with org-mks vs ordinary mode-map based menu, is that org-mks
locks entire session. Modal behaviour is used to ensure that just one task at
the time is manipulating org files. I think it can be achieved by other means
too. I have not done it correctly in the example :), but I think it is possible.

I am including also an older test which I have later refactored, that has
"read-key" interface (in org-select-modal); i.e it behaves similar to org-mks,
just to show that such modal interface can be tucked on. It just reads a key
from the user and then invokes the command from the mode map. It is very crappy,
but it shows that both

@Tim
Thank you for mentioning emacspeak. I have never used it so I don't know how it
works, but I have taken a look at some code in Emacspeak after your mail.

Also if I understand what you and Ihor say, it needs to get labels echoed to
minibuffer in order to work with Emacspeak? I have done it so, I am not sure if
works always though :).

@Max
I agree with you that completing read is a good alternative, but it is a bit
like discussion about GUI vs. terminal. I am personally heavy user of Helm, but
not everyone is I believe.

About the name: org-select; i really have no idea what to call it, so better
name would be nice. Sorry for the bugs, I am aware of many, but it still
displays the idea I think.


[-- Attachment #2: org-select.el --]
[-- Type: text/plain, Size: 22798 bytes --]

;;; org-select.el --- Build custom menus from declarative templates  -*- lexical-binding: t; -*-

;; Copyright (C) 2022  Arthur Miller

;; Author: Arthur Miller <arthur.miller@live.com>
;; Keywords: tools

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;; 
;; 

;;; Code:
\f
(require 'org-macs)

;;; User vars
\f
(defgroup org-select nil
  "Create menus from declarative templates."
  :prefix "org-select-"
  :prefix "osl--"
  :tag "Org Select"
  :group 'org)

(defcustom org-select-back-key [f10]
  "Used to render string for the horizontal separator."
  :type 'character
  :group 'org-select)

(defcustom org-select-horizontal-separator "|"
  "Used to render string for the horizontal separator."
  :type 'string
  :group 'org-select)

(defcustom org-select-vertical-separator "-"
  "Used to render string for the vetical separator."
  :type 'string
  :group 'org-select)

(defcustom org-select-key-decorator-chars ""
  "Characters used to decorate shortcut keys.

This string should contain only two characters, the first one for the left
decorator and the second one for the right decorator.

Example: string \"[]\" will render key \"C\" as \"[C]\"."
  :type 'string
  :group 'org-select)

(defcustom org-select-label-decorators (cons "..." "...")
  "Used to render string for the vetical separator."
  :type 'cons
  :group 'org-select)
\f
;;; Implementation
\f
(defvar-local osl--init nil)
(defvar-local osl--args nil)
(defvar-local osl--buffer nil)
(defvar-local osl--menu-begin nil)
(defvar-local osl--buffer-menu nil)
(defvar-local osl--longest-label 0)
(defvar-local osl--buffer-window nil)
(defvar-local org-select-mode-map nil)
(defvar-local osl--horizontal-layout nil)
(defvar-local osl--default-handler-fn nil)
(defvar-local osl--current-menu-column nil)

(define-minor-mode org-select-mode ""
  :interactive nil :global nil)

;;;; Help-functions
(defun osl--arg (key)
  (plist-get osl--args key))

(defun osl--init ()
  (buffer-local-value 'osl--init (current-buffer)))

(defun osl--default-handler-fn (entry)
  "Try to execute form found in ENTRY if any."
  (let ((form (nth 2 entry)))
    (cond
     ((listp form)
      (eval form))
     (t (if (commandp form)
            (call-interactively form)
          (eval form))))))

(with-eval-after-load
    (setq osl--default-handler-fn #'osl--default-handler-fn))

(defun osl--ignore-key ()
  (interactive)
  (message
   "Invalid key %S"
   ;; I am not happy but it works somewhat
   (edmacro-format-keys (vector last-input-event))))

(defun org-select-quit (&optional abort-message buffer-name)
  (interactive)
  (let ((window (if buffer-name
                    (get-buffer-window buffer-name)
                  osl--buffer-window))
        (kill-buffer (buffer-local-value 'osl--buffer (current-buffer))))
    (when (window-live-p window)
      (select-window window)
      (quit-window kill-buffer window))
    (message (or abort-message "Org Select Quit"))))

(defun osl--make-mode-map ()
  (let ((map (make-sparse-keymap)))
    (define-key map [?q] #'org-select-quit)
    (define-key map [?\C-g] #'org-select-abort)
    (define-key map [left] #'osl--back)
    (define-key map [?\C-p] #'osl--back)
    (define-key map [remap newline] #'osl--ignore-key)
    (define-key map [remap self-insert-command] #'osl--ignore-key)
    (setq org-select-mode-map map)
    (use-local-map org-select-mode-map)))

(defun org-select-abort ()
  (interactive)
  (org-select-quit "Aborted"))

(defun osl--back ()
  (interactive)
  (when (bound-and-true-p org-select-mode)
    (osl--make-mode-map)
    (osl--draw)))

(defun osl--longest-line ()
  "Return the length of the longest line in current buffer."
  (let ((n 1) (L 0) (e 0) (E (point-max)) l)
    (while (< e E)
      (setq e (line-end-position n)
            l (- e (line-beginning-position n))
            n (1+ n))
      (if (> l L) (setq L l)))
    L))

(defun osl--decorate-key (key)
  "Place string KEY between characters specified in DECORATOR string."
  (let ((kd (if (> (length org-select-key-decorator-chars) 0)
                org-select-key-decorator-chars
              (osl--arg :key-decorator))))
    (if (= (length kd) 2)
        (concat (substring kd 0 1) key (substring kd 1))
      key)))

(defun osl--decorate-label (entry)
  "Place string LABEL between strings specified in DECORATORS strings.
DECOARATOR  is a cons containing two elements: left and right decorators."
  (let ((left (car org-select-label-decorators))
        (right (cdr org-select-label-decorators)))
    (if (= (length entry) 2)
        (concat left (cadr entry) right)
      (cadr entry))))

(defun osl--make-separator (&optional marker length)
  (let ((len (or length (osl--longest-line)))
        (sep (if (osl--arg :horizontal)
                 org-select-horizontal-separator
               org-select-vertical-separator)))
    (if marker
        (concat "sep" sep)
      (make-string len (string-to-char sep)))))

(defun osl--insert-horizontal-separator (sep &optional _length)
  (goto-char 1)
  (let ((lol (osl--longest-line))
        (sep (or org-select-horizontal-separator sep)))
    (while (not (eobp))
      (let* ((eol (line-end-position))
             (bol (line-beginning-position))
             (fill (- (+ bol lol) eol)))
        (goto-char eol)
        (if (> fill 0)
            (while (> fill 0) (insert " ") (setq fill (1- fill)))
          (while (> 0 fill) (delete-char 1) (setq fill (1+ fill))))
        (insert " " sep " "))
      (forward-line))
    (setq osl--current-menu-column (+ lol (length sep) 2))))

(defun osl--insert-separator (sep &optional _length)
  (if (osl--arg :horizontal)
      (osl--insert-horizontal-separator sep)
    (insert sep)))

(defun osl--insert (&rest strings)
  (cond
   ((and (osl--arg :horizontal)
         (> osl--current-menu-column 0))
    (goto-char (+ (line-beginning-position)
                  osl--current-menu-column))
    (apply #'insert strings)
    (if (char-after)
        (forward-line)
      (insert "\n")))
   (t 
    (apply #'insert strings)
    (insert "\n"))))

(defun osl--forward-menu ()
  (cond
   ((osl--arg :horizontal)
    (goto-char (point-min))
    (goto-char (line-end-position))
    (setq osl--current-menu-column
          (- (point) (line-beginning-position))))
   (t (insert "\n"))))

;;;; Menu drawing
(defun osl--setup-buffer (tables args)
  "Setup buffer local variables needed for an org-select buffer."
  (let* ((buffer (or (plist-get args :label) "*Org-select: "))
         (window (get-buffer-window buffer)))
      (if window
          (select-window window)
        (org-switch-to-buffer-other-window buffer))
    (with-current-buffer (get-buffer buffer)
      (special-mode)
      ;;(setq cursor-type nil)
      (org-select-mode)
      (osl--make-mode-map)
      (setq osl--args args
            osl--buffer-menu tables
            osl--current-menu-column 0
            osl--buffer (current-buffer)
            osl--buffer-window (get-buffer-window)
            osl--default-handler-fn 'osl--default-handler-fn))))

;; menu  is a list of tables, display one table at a time
(defun osl--draw ()
  "Starts menu parsing and insertig."
  (with-silent-modifications
    (erase-buffer)
    (setq osl--init nil)
    (let ((marker (osl--make-separator 'marker))
          (text (osl--arg :text))
          (menus (buffer-local-value
                  'osl--buffer-menu (current-buffer))))
      (setq osl--menu-begin (point))
      (dolist (menu menus)
        (if (symbolp menu) (setq menu (eval menu)))
        (osl--do-menu menu)
        (setq menus (cdr menus))
        (when menus
          (osl--insert-separator marker)
          (osl--forward-menu)))
      (goto-char 1)
      (let ((sep (osl--make-separator nil (osl--longest-line)))
            ;; (osl--make-separator nil fill-column))
            )
        (while (search-forward marker nil t)
          (replace-match "")
          (osl--insert-separator sep)))
      (when text
        (goto-char 1)
        (insert "\n" text "\n"))
      (org-fit-window-to-buffer)
      (setq osl--init t)
      (goto-char 1)))) ; unnecessary but prettier if beacon-mode is active

;; iterate through menu and render a single entry or a group of entries on each
;; iteration
(defun osl--do-menu (menu)
  "Insert one menu at a time."
  (while menu
    (let ((entry (car menu)))
      (setq menu
            (if (> (length entry) 2)
                (osl--do-entry menu)
              (osl--do-group menu))))))

(defun osl--do-group (menu)
  "Do a menu with group nodes."
  (let ((group (car menu))
        (transient (osl--arg :transient))
        newmenu)
    (osl--do-entry menu)
    (while (> (length (cadr menu)) 2)
      (let (entry newentry key)
        (setq menu (cdr menu) entry (car menu))
        (setq key (substring (car entry) 1))
        (push key newentry)
        (dolist (elt (cdr entry)) (push elt newentry))
        (push (nreverse newentry) newmenu)))
    (setq newmenu (nreverse newmenu))
    (define-key org-select-mode-map (kbd (car group))
                (lambda ()
                  (interactive)
                  (with-silent-modifications
                    (erase-buffer)
                    (setq osl--current-menu-column 0)
                    (osl--do-menu newmenu)
                    (if transient (org-select-quit "")))))
    (cdr menu))) ;; return next group in chain

;; we send in the entire menu so we can return next piece in chain,
;; but *the* entry we work with is just the very first one (car menu)
(defun osl--do-entry (menu)
  "Display a single entry in the buffer."
  (let* ((entry (car menu))
         (key (car entry))
         (line-length 0)
         (transient (osl--arg :transient)))
    (define-key org-select-mode-map (kbd key)
                (lambda ()
                  (interactive)
                  (let ((label (nth 1 entry))
                        (handler (or (osl--arg :handler)
                                     osl--default-handler-fn))
                        (init (buffer-local-value 'osl--init osl--buffer))
                        msg)
                    (and init handler
                         (setq msg (funcall handler entry)))
                    (if transient (org-select-quit ""))
                    (message (or msg label)))))
    (osl--insert (osl--decorate-key key) "    " (osl--decorate-label entry))
    (setq line-length (- (line-end-position) (line-beginning-position)))
    (if (> line-length osl--longest-label)
        (setq osl--longest-label line-length))
    (cdr menu)))
\f
;;; API
\f
(defun org-select (tables &rest args)
  "Select a member of an alist with multiple keys.

TABLE is an alist which should contain entries where the car is a string.
There should be two types of entries.

1. prefix descriptions like (\"a\" \"Description\")
   This indicates that `a' is a prefix key for multi-letter selection, and
   that there are entries following with keys like \"ab\", \"ax\"...

2. Select-able members must have more than two elements, with the first
   being the string of keys that lead to selecting it, and the second a
   short description string of the item. 

The command will then make a temporary buffer listing all entries
that can be selected with a single key, and all the single key
prefixes.  When you press the key for a single-letter entry, it is selected.
When you press a prefix key, the commands (and maybe further prefixes)
under this key will be shown and offered for selection.

ARGS is a property list containing following members:

:text          a string placed over the selection in the buffer.
:label         a string used for the selections buffer name.
:prompt        a string used when prompting for a key.
:always        when `t', this menu is shown; even descended into submenus
:transient     when `t', the menu is dissmised after user perform an action
:key-decorator a two-character string used to decorate command characters. When
this string is specified, it will take precedence over the global variable
`org-select-key-decorator-chars'.

TABLES are additional menus in the same format as TABLE. If there are more
than one menus, they will be separated by a separator line rendered with
character as specified in `org-select-horizontal-separator'"
  (osl--setup-buffer tables args)
  (osl--draw))
\f
;;; Demo
\f
;;;; org-capture
\f
(require 'org)
(require 'org-capture)

(defvar org-capture--current-goto nil)
(defvar org-capture--current-keys nil)
(defvar org-capture--old-window-config nil)

(defun org-capture-test (&optional goto keys)
  "Simple illustration to recreate org-capture menu (visually only)."
  (interactive "P")
  (let ((org-select-vertical-separator "-")
        (org-capture-templates
	   (or (org-contextualize-keys
	        (org-capture-upgrade-templates org-capture-templates)
	        org-capture-templates-contexts)
	       '(("t" "Task" entry (file+headline "" "Tasks")
		  "* TODO %?\n  %u\n  %a")))))
      (if keys
	  (or (assoc keys org-capture-templates)
	      (error
               "No capture template referred to by \"%s\" keys" keys)))
  (cond
   ((equal goto '(4))  (org-capture-goto-target keys))
   ((equal goto '(16)) (org-capture-goto-last-stored))
   (t
    (if goto (setq org-capture--current-goto goto))
    (setq org-capture--old-window-config (current-window-configuration))
    (org-select
     ;; tables
     '(org-capture-templates
       (("C" "Customize org-capture-templates"
         (customize-variable 'org-capture-templates))
        ("q" "Abort" (org-select-quit "Abort"))))
     ;; description
     :transient t
     :handler #'org-capture--handle
     :label "*Capture*" :key-decorator "[]"
     :text "Select a capture template\n========================="))))
    (message "Org Capture"))
(define-key global-map (kbd "C-v c") #'org-capture-test)

(defun org-capture--handle (entry)
  (org-select-quit "")
  (cond
   ((or (equal "C" (car entry)) (equal "q" (car entry)))
    (eval (nth 2 entry)))
   (t
    (let* ((orig-buf (current-buffer))
	   (annotation (if (and (boundp 'org-capture-link-is-already-stored)
			        org-capture-link-is-already-stored)
			   (plist-get org-store-link-plist :annotation)
		         (ignore-errors (org-store-link nil))))
	   (entry (or org-capture-entry entry))
           (goto org-capture--current-goto)
           (inhibit-read-only t)
	   initial)
      (setq initial (or org-capture-initial
		        (and (org-region-active-p)
			     (buffer-substring (point) (mark)))))
      (when (stringp initial)
        (remove-text-properties 0 (length initial) '(read-only t) initial))
      (when (stringp annotation)
        (remove-text-properties 0 (length annotation)
			        '(read-only t) annotation))
      (org-capture-set-plist entry)
      (org-capture-get-template)
      (org-capture-put :original-buffer orig-buf
		       :original-file (or (buffer-file-name orig-buf)
					  (and (featurep 'dired)
					       (car (rassq orig-buf
							   dired-buffers))))
		       :original-file-nondirectory
		       (and (buffer-file-name orig-buf)
			    (file-name-nondirectory
			     (buffer-file-name orig-buf)))
		       :annotation annotation
		       :initial initial
		       :return-to-wconf (current-window-configuration)
		       :default-time (or org-overriding-default-time
				         (org-current-time)))
      (org-capture-set-target-location (and (equal goto 0) 'here))
      (condition-case error
	  (org-capture-put :template (org-capture-fill-template))
        ((error quit)
         ;;(if (get-buffer "*Capture*") (kill-buffer "*Capture*"))
         (org-select-quit "" "*Capture*")
         (error "Capture abort: %s" (error-message-string error))))
      (setq org-capture-clock-keep (org-capture-get :clock-keep))
      (condition-case error
	  (org-capture-place-template
	   (eq (car (org-capture-get :target)) 'function))
        ((error quit)
         (when (and (buffer-base-buffer (current-buffer))
		    (string-prefix-p "CAPTURE-" (buffer-name)))
	   (kill-buffer (current-buffer)))
         (set-window-configuration (org-capture-get :return-to-wconf))
         (error "Capture template `%s': %s"
	        (org-capture-get :key)
	        (error-message-string error))))
      (when (and (derived-mode-p 'org-mode) (org-capture-get :clock-in))
        (condition-case nil
	    (progn
	      (when (org-clock-is-active)
	        (org-capture-put :interrupted-clock
			         (copy-marker org-clock-marker)))
	      (org-clock-in)
	      (setq-local org-capture-clock-was-started t))
	  (error "Could not start the clock in this capture buffer")))
      (when (org-capture-get :immediate-finish)
        (org-capture-finalize))))))
\f
;;;; Org Agenda
\f
(require 'org-agenda)
(defvar org-agenda--arg nil)
(defvar org-agenda--keys nil)
(defvar org-agenda--restriction nil)

(defun org-agenda--exec (action &rest args)
  "Execute ACTION and exit org-agenda menu."
  (interactive)
  (org-select-quit "")
  (apply action args))

(defvar org-agenda--menu
  '((("a" "Agenda for current week or day" (org-agenda--exec
                                            'org-agenda-list))
     ("t" "List of all TODO entries"       (org-agenda--exec
                                            'org-todo-list))
     ("m" "Match a TAGS/PROP/TODO query"   (org-agenda--exec
                                            'org-tags-view))
     ("s" "Search for keywords"            (org-agenda--exec
                                            'org-search-view))
     ("/" "Multi-occur"                    (call-interactively
                                            'org-occur-in-agenda-files))
     ("?" "Find :FLAGGED: entries"         (org-agenda--exec
                                            'org-tags-view
                                            nil "+FLAGGED"))
     ("*" "Toggle sticky agenda views"     (call-interactively
                                            #'org-toggle-sticky-agenda)))
    (("<" "Buffer, subtree/region restriction" ignore)
     (">" "Remove restriction"             ignore)
     ("e" "Export agenda views"            org-store-agenda-views)
     ("T" "Entries with special TODO kwd" (org-agenda--exec
                                           'org-call-with-arg
                                           'org-todo-list
                                           (or org-agenda--arg '(4))))
     ("M" "Like m, but only TODO entries" (org-agenda--exec
                                           'org-call-with-arg
                                           'org-tags-view
                                           (or org-agenda--arg '(4))))
     ("S" "Like s, but only TODO entries" (org-agenda--exec
                                           'org-call-with-arg
                                           'org-search-view
                                           (or org-agenda--arg '(4))))
     ("C" "Configure custom agenda commands"
      (org-agenda--exec 'customize-variable 'org-agenda-custom-commands))
     ("#" "List stuck projects" (org-agenda--exec
                                 'org-agenda-list-stuck-projects))
     ("!" "Configure stuck projects"
      (org-agenda--exec 'customize-variable 'org-stuck-projects)))))

(defun org-agenda-test (&optional _arg _keys _restriction)
  (interactive "P")
  (let ((org-select-horizontal-separator " "))
    (org-select
     org-agenda--menu
     :text
     "Press key for an agenda command:
--------------------------------\n"
     :horizontal t)
    (org-agenda-fit-window-to-buffer)))
\f
(defun test1 ()
  "Stays after a choice is made."
  (interactive)
  (let ((org-select-horizontal-separator "│"))
    (org-select
     ;; table
     '((("1" "One" (message "One!"))
        ("2" "Two" (message "Two!!"))
        ("3" "Three" (message "Three!!!")))
       (("C-4" "Four" (message "Four!!!!"))
        ("C-5" "Five" (message "Five!!!!!"))
        ("C-6" "six" (message "Six!")))
       (("M-7" "Seven" (message "Seven!"))
        ("M-8" "Eight" (message "Eight!"))
        ("M-9" "Nine" (message "Nine!"))))
     ;; description
     :horizontal t :key-decorator "<>")))

(defun test2 ()
  "Dissapears after a choice is made."
  (interactive)
  (let ((org-select-horizontal-separator "│"))
    (org-select
     ;; menus
     '((("h" "Hello, World!" (message "Hello, World!"))
        ("b" "Bar" (message "Hello, Bar!")))
       (("f" "Find File" find-file)
        ("o" "Open File" (flet ((next-read-file-uses-dialog-p () t))
                           (call-interactively #'find-file)))))
     ;; description
     :key-decorator "\"\"" :transient t)
      ;; Hints
      (setq header-line-format
	    (if (not (pos-visible-in-window-p (point-max)))
	        "Use C-v, M-v, C-n or C-p to navigate. C-g, q to quit."
              "Use C-p/Left to go back, C-g, q to quit."))))

(defun test3 ()
  "Illustrate nested menus, unicode separator and alternative decorator."
  (interactive)
  (let ((org-select-vertical-separator "─"))
    (org-select
     ;; tables
     '((("g" "Greetings")
        ("gh" "Hello, World!" (message "Hello, World!"))
        ("gb" "Bar" (message "Hello, Bar!")))
       (("f" "Functions")
        ("ff" "Find File" find-file)
        ("fo" "Open File" (flet ((next-read-file-uses-dialog-p () t))
                            (call-interactively #'find-file)))))))
      ;; Hints
      (setq header-line-format
	    (if (not (pos-visible-in-window-p (point-max)))
	        "Use C-v, M-v, C-n or C-p to navigate. C-g, q to quit."
              "Use C-p/Left to go back, C-g, q to quit.")))

  (provide 'org-select)

;;; org-select.el ends here

[-- Attachment #3: org-select-modal.el --]
[-- Type: text/plain, Size: 16425 bytes --]

;;; org-select.el --- Build custom menus from declarative templates  -*- lexical-binding: t; -*-

;; Copyright (C) 2022  Arthur Miller

;; Author: Arthur Miller <arthur.miller@live.com>
;; Keywords: tools

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;; 
;; 

;;; Code:
\f
(require 'org-macs)

;;; User vars
\f
(defgroup org-select nil
  "Create menus from declarative templates."
  :prefix "org-select-"
  :prefix "osl--"
  :tag "Org Select"
  :group 'org)

(defcustom org-select-back-key [f10]
  "Used to render string for the horizontal separator."
  :type 'character
  :group 'org-select)

(defcustom org-select-horizontal-separator "|"
  "Used to render string for the horizontal separator."
  :type 'string
  :group 'org-select)

(defcustom org-select-vertical-separator "-"
  "Used to render string for the vetical separator."
  :type 'string
  :group 'org-select)

(defcustom org-select-key-decorator-chars ""
  "Characters used to decorate shortcut keys.

This string should contain only two characters, the first one for the left
decorator and the second one for the right decorator.

Example: string \"[]\" will render key \"C\" as \"[C]\"."
  :type 'string
  :group 'org-select)

(defcustom org-select-label-decorators (cons "..." "...")
  "Used to render string for the vetical separator."
  :type 'cons
  :group 'org-select)
\f
;;; Implementation
\f
(defvar-local osl--args nil)
(defvar-local osl--menu-begin nil)
(defvar-local osl--buffer-menu nil)
(defvar-local osl--longest-label 0)
(defvar-local osl--allowed-keys nil)
(defvar-local osl--buffer-window nil)
(defvar-local org-select-mode-map nil)
(defvar-local osl--horizontal-layout nil)
(defvar-local osl--default-handler-fn nil)
(defvar-local osl--current-menu-column nil)

(define-minor-mode org-select-mode ""
  :interactive nil :global nil)

;;;; Help-functions
(defun osl--arg (key)
  (plist-get osl--args key))

(defun osl--default-handler-fn (entry)
  "Try to execute form found in ENTRY if any."
  (let ((form (nth 2 entry)))
    (cond
     ((listp form)
      (eval form))
     (t (if (commandp form)
            (call-interactively form)
          (eval form))))))

(with-eval-after-load
    (setq osl--default-handler-fn #'osl--default-handler-fn))

(defun osl--ignore-key ()
  (interactive)
  (message
   "Invalid key %S"
   ;; I am not happy but it works somewhat
   (edmacro-format-keys (vector last-input-event))))

(defun osl--read-key ()
  (let ((key (read-key-sequence
              (concat
               (or (osl--arg :label) "Org-select") ": "))))
    (funcall (local-key-binding key))))

(defun org-select-quit (&optional abort-message)
  (interactive)
  (catch 'exit
    (when (> 0 (recursion-depth))
      (exit-recursive-edit)
      (top-level)))
    (while osl--buffer-window
      (quit-window t osl--buffer-window)
      (message (or abort-message "Org Select Quit"))))

(defun osl--back ()
  (interactive)
  (osl--draw))

(defun osl--line-length ()
  (- (line-end-position) (line-beginning-position)))

(defun osl--decorate-key (key)
  "Place string KEY between characters specified in DECORATOR string."
  (let ((kd (if (> (length org-select-key-decorator-chars) 0)
                org-select-key-decorator-chars
              (osl--arg :key-decorator))))
    (if (= (length kd) 2)
        (concat (substring kd 0 1) key (substring kd 1))
      key)))

(defun osl--decorate-label (entry)
  "Place string LABEL between strings specified in DECORATORS strings.
DECOARATOR  is a cons containing two elements: left and right decorators."
  (let ((left (car org-select-label-decorators))
        (right (cdr org-select-label-decorators)))
    (if (= (length entry) 2)
        (concat left (cadr entry) right)
      (cadr entry))))

(defun osl--make-separator (&optional marker length)
  (let ((length (or length osl--longest-label))
        (sepch (if (osl--arg :horizontal)
                   (string-to-char org-select-horizontal-separator)
                 (string-to-char org-select-vertical-separator))))
    (if marker
        (concat "sep" (char-to-string sepch))
      (make-string length sepch))))

(defun osl--insert-separator (sep)
  (if (osl--arg :horizontal)
      (osl--insert-horizontal-separator sep)
    (insert sep "\n")))

(defun osl--longest-menu-length ()
  (let ((longest-menu-length 0)
        (menus (buffer-local-value
                'osl--buffer-menu (current-buffer)))
        length)
    (dolist (m menus)
      (setq length (if (symbolp m) (length (eval m)) (length m)))
      (if (> length longest-menu-length) (setq longest-menu-length length)))
    longest-menu-length))

(defun osl--insert-horizontal-separator (sep)
  (goto-char osl--menu-begin)
  (dotimes (i (osl--longest-menu-length))
    (let* ((eol (line-end-position))
           (bol (line-beginning-position))
           (lol osl--longest-label)
           (sep (or org-select-horizontal-separator sep))
           (fill (abs (- eol (+ bol lol)))))
      (goto-char eol)
      (while (> fill 0) (insert " ") (setq fill (1- fill)))
      (goto-char (line-end-position))
      (insert " ")
      (if (> (length sep) 0)
          (insert sep " "))
    (forward-line)
    (setq i (1+ i))))
  (setq osl--current-menu-column (1- (point))))

(defun osl--insert (&rest strings)
  (if (osl--arg :horizontal)
    (goto-char (line-end-position)))
  (apply #'insert strings))

(defun osl--forward-menu ()
  (cond ((osl--arg :horizontal)
         (goto-char osl--menu-begin)
         (setq osl--current-menu-column
               (+ osl--current-menu-column osl--longest-label)))
        (t ;;(insert "\n")
           )))

;;;; Menu drawing
(defun osl--setup-buffer (tables args)
  "Setup buffer local variables needed for an org-select buffer."
  (let* ((buffer (or (plist-get args :label) "*Org-select: "))
         (window (get-buffer-window buffer)))
    (if window (select-window window)
      (org-switch-to-buffer-other-window buffer))
    (with-current-buffer (get-buffer buffer)
      (special-mode)
      (setq cursor-type nil)
      (org-select-mode)
      (setq org-select-mode-map
            (let ((map (make-sparse-keymap)))
              (define-key map [?q] #'org-select-quit)
              (define-key map [?\C-g] #'org-select-quit)
              (define-key map [left] #'osl--back)
              (define-key map [?\C-p] #'osl--back)
              (define-key map [remap newline] #'osl--ignore-key)
              (define-key map [remap self-insert-command] #'osl--ignore-key)
              map))
      (use-local-map org-select-mode-map)
      (setq osl--args args
            osl--buffer-menu tables
            osl--current-menu-column 0
            osl--buffer-window (get-buffer-window)
            osl--default-handler-fn 'osl--default-handler-fn))))

;; menu  is a list of tables, display one table at a time
(defun osl--draw ()
  "Starts menu parsing and insertig."
  (with-silent-modifications
    (erase-buffer)
    (let ((marker (osl--make-separator 'marker))
          (modal (osl--arg :modal))
          (text (osl--arg :text))
          (menus (buffer-local-value
                  'osl--buffer-menu (current-buffer))))
      (when text
        (insert text "\n"))
      (setq osl--menu-begin (point))
      (dolist (menu menus)
        (if (symbolp menu) (setq menu (eval menu)))
        (osl--do-menu menu)
        (setq menus (cdr menus))
        (when menus
          (osl--insert-separator marker)
          (osl--forward-menu)))
      (let ((separator (osl--make-separator)))
        (while (search-backward marker nil t)
          (replace-match "")
          (osl--insert-separator separator)))
      (org-fit-window-to-buffer)
      (goto-char 1) ;; unnecessary but looks prettier if beacon-mode is active
      (if modal (osl--read-key)))))

;; iterate through menu and render a single entry or a group of entries on each
;; iteration
(defun osl--do-menu (menu)
  "Insert one menu at a time."
  (while menu
    (let ((entry (car menu)))
      (setq menu
            (if (> (length entry) 2)
                (osl--do-entry menu)
              (osl--do-group menu))))))

(defun osl--do-group (menu)
  "Do a menu with group nodes."
  (let ((group (car menu))
        (modal (osl--arg :modal))
        (transient (osl--arg :transient))
        newmenu)
    (osl--do-entry menu)
    (while (> (length (cadr menu)) 2)
      (let (entry newentry key)
        (setq menu (cdr menu) entry (car menu))
        (setq key (substring (car entry) 1))
        (push key newentry)
        (dolist (elt (cdr entry)) (push elt newentry))
        (push (nreverse newentry) newmenu)))
    (setq newmenu (nreverse newmenu))
    (define-key org-select-mode-map (kbd (car group))
                (lambda ()
                  (interactive)
                  (with-silent-modifications
                    (erase-buffer)
                    (setq osl--current-menu-column 0)
                    (osl--do-menu newmenu)
                    (if modal (osl--read-key))
                    (if transient (org-select-quit "")))))
    (cdr menu))) ;; return next group in chain

;; we send in the entire menu so we can return next piece in chain,
;; but *the* entry we work with is just the first one (car menu)
(defun osl--do-entry (menu)
  "Display a single entry in the buffer."
  (let* ((entry (car menu))
         (key (car entry))
         (line-length 0)
         (transient (osl--arg :transient)))
    (push key osl--allowed-keys)
    (define-key org-select-mode-map (kbd key)
                (lambda ()
                  (interactive)
                  (let ((label (nth 1 entry))
                        (handler (or (plist-get :handler entry)
                                     osl--default-handler-fn)))
                    (if handler (funcall handler entry))
                    (if transient (org-select-quit ""))
                    (message label))))
    (osl--insert (osl--decorate-key key) "    " (osl--decorate-label entry))
    (setq line-length (- (line-end-position) (line-beginning-position)))
    (if (> line-length osl--longest-label)
        (setq osl--longest-label line-length))
    (if (= 0 osl--current-menu-column)
        (insert "\n")
      (forward-line))
    (cdr menu)))
\f
;;; API
\f
(defun org-select (tables &rest args)
  "Select a member of an alist with multiple keys.

TABLE is an alist which should contain entries where the car is a string.
There should be two types of entries.

1. prefix descriptions like (\"a\" \"Description\")
   This indicates that `a' is a prefix key for multi-letter selection, and
   that there are entries following with keys like \"ab\", \"ax\"...

2. Select-able members must have more than two elements, with the first
   being the string of keys that lead to selecting it, and the second a
   short description string of the item. 

The command will then make a temporary buffer listing all entries
that can be selected with a single key, and all the single key
prefixes.  When you press the key for a single-letter entry, it is selected.
When you press a prefix key, the commands (and maybe further prefixes)
under this key will be shown and offered for selection.

ARGS is a property list containing following members:

:text          a string placed over the selection in the buffer.
:label         a string used for the selections buffer name.
:prompt        a string used when prompting for a key.
:modal         when `t', read minibuffer until dialog is dismissed
:always        when `t', this menu is shown; even descended into submenus
:transient     when `t', the menu is dissmised after user perform an action
:key-decorator a two-character string used to decorate command characters. When
this string is specified, it will take precedence over the global variable
`org-select-key-decorator-chars'.

TABLES are additional menus in the same format as TABLE. If there are more
than one menus, they will be separated by a separator line rendered with
character as specified in `org-select-horizontal-separator'"
  (osl--setup-buffer tables args)
  (osl--draw))
\f
;;; Demo
\f
(require 'org)
(require 'org-capture)

(defun demo1 ()
  "Simple illustration to recreate org-capture menu (visually only)."
  (interactive)
  (org-select
   ;; tables
   '(org-capture-templates
     (("C" "Customize org-capture-templates"
       (customize-variable 'org-capture-templates))
      ("q" "Abort" (org-select-quit "Abort"))))
   ;; description
   :label "*Quick Select*" :key-decorator "[]" :horizontal t
   :text "Select a capture template\n========================="))

(defun demo2 ()
  "Menu composition with automatic separator."
  (interactive)
  (let ((org-select-key-decorator-chars "<>"))
    (org-select
     ;; menus
     '((("h" "Hello, World!" (message "Hello, World!"))
        ("b" "Bar" (message "Hello, Bar!")))
       (("f" "Find File" find-file)
        ("o" "Open File" (flet ((next-read-file-uses-dialog-p () t))
                           (call-interactively #'find-file))))
       (("q" "Abort" (org-select-quit "Abort"))))
     ;; description
     :key-decorator "<>")))

(defun demo3 ()
  "Menu dissapears after a choice is made."
  (interactive)
  (org-select
   ;; menus
   '((("h" "Hello, World!" (message "Hello, World!"))
      ("b" "Bar" (message "Hello, Bar!")))
     (("f" "Find File" find-file)
      ("o" "Open File" (flet ((next-read-file-uses-dialog-p () t))
                         (call-interactively #'find-file))))
     (("q" "Abort" (message "Abort"))))
   ;; description
   :key-decorator "<>" :transient t :horizontal t))

(defun demo4 ()
  "Illustrate nested menus, unicode separator and alternative decorator."
  (interactive)
  (let ((org-select-vertical-separator "─"))
    (org-select
     ;; tables
     '((("g" "Greetings")
        ("gh" "Hello, World!" (message "Hello, World!"))
        ("gb" "Bar" (message "Hello, Bar!")))
       (("f" "Functions")
        ("ff" "Find File" find-file)
        ("fo" "Open File" (flet ((next-read-file-uses-dialog-p () t))
                            (call-interactively #'find-file))))
       (("q" "Abort" (org-select-quit "Abort"))))
       ;; description
     :key-decorator "<>")))

(defun demo5 ()
  "Same as demo4 but modal."
  (interactive)
  (let ((org-select-vertical-separator "─"))
    (org-select
     ;; table
     '((("g" "Greetings")
        ("gh" "Hello, World!" (message "Hello, World!"))
        ("gb" "Bar" (message "Hello, Bar!")))
       ;; more tables
       (("f" "Functions")
        ("ff" "Find File" (call-interactively #'find-file))
        ("fo" "Open File" (flet ((next-read-file-uses-dialog-p () t))
                            (call-interactively 'find-file))))
       (("q" "Abort" (org-select-quit "Abort"))))
     ;; description
     :modal t :transient t)))

(defun demo6 ()
  "Horizontal menus."
  (interactive)
  (let ((org-select-vertical-separator "─"))
    (org-select
     ;; table
     '((("1" "One" (message "One!"))
        ("2" "Two" (message "Two!!"))
        ("3" "Three" (message "Three!!!")))
       (("4" "Four" (message "Four!!!!"))
        ("5" "Five" (message "Five!!!!!"))
        ("6" "six" (message "Six!")))
       (("7" "Seven" (message "Seven!"))
        ("8" "Eight" (message "Eight!"))
        ("9" "Nine" (message "Nine!"))))
     ;; description
     :transient t :horizontal t)))

  (provide 'org-select)

;;; org-select.el ends here

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

> Anyway, I have been playing and testing a bit, and didn't want to prolong
> discussion untill I have something to show. So here is a small prototype. It is
> just a rough sketch of the idea.

Thanks!
I will not comment on Elisp part just yet. Let's first settle down the design.

> this example the mode map approach seems slightly more convenient. I don't know,
> in org-agenda-test, I haven't implemented all of org-agenda, restrictions,
> prefixes and some other stuff, mostly because I don't really understand the
> implementation.

In the nutshell, agenda restrictions will execute some elisp that sets
certain global variables affecting other agenda commands. selecting the
restriction should not leave the agenda menu.

Also, unlike other selections being echoed literally upon selection,
restriction echo must depend on the global state. If you press "<" in
the menu, the menu prompt should change between "Press key for agenda
command (unrestricted):", Press key for agenda command (restricted to
buffer):, ... etc

Note that there is not much point echoing the selection.
Tim, do I understand correctly that changed minibuffer prompt will be
also spoken out by emacspeak?

> About the name: org-select; i really have no idea what to call it, so better
> name would be nice. Sorry for the bugs, I am aware of many, but it still
> displays the idea I think.

It does not matter at this point. We can discuss it at the very end. It
is not like M-x replace-regexp is difficult to run later.

> (defun osl--make-mode-map ()
>   (let ((map (make-sparse-keymap)))
>     (define-key map [?q] #'org-select-quit)
>     (define-key map [?\C-g] #'org-select-abort)
>     (define-key map [left] #'osl--back)
>     (define-key map [?\C-p] #'osl--back)

binding left and C-p is unexpected. C-n and C-p are usually dedicated to
moving around. A very conservative alternative could be creating a new
buffer for sub-menus and letting the user use the usual previous-buffer
command.

> ;;; API
> \f
> (defun org-select (tables &rest args)
>   "Select a member of an alist with multiple keys.
>
> TABLE is an alist which should contain entries where the car is a string.
> There should be two types of entries.

This is confusing because there is no TABLE argument in the command.
I it not immediately clear that you can use multiple tables to build a
single menu.

> ARGS is a property list containing following members:

Does it apply to individual menu entries? I am a bit confused here.

> :text          a string placed over the selection in the buffer.
> :label         a string used for the selections buffer name.

:label name is confusing. Why not directly :buffer-name?

> :always        when `t', this menu is shown; even descended into submenus
> :transient     when `t', the menu is dissmised after user perform an
> action

Does it apply to the whole table? Individual entries?

> TABLES are additional menus in the same format as TABLE. If there are more
> than one menus, they will be separated by a separator line rendered with
> character as specified in `org-select-horizontal-separator'"

> (defun org-capture-test (&optional goto keys)
> (defun org-capture--handle (entry)
>   (org-select-quit "")
>   (cond
>    ((or (equal "C" (car entry)) (equal "q" (car entry)))
>     (eval (nth 2 entry)))
>    (t
>     (let* ((orig-buf (current-buffer))

This looks fragile. It is hard to know what is the value of
(current-buffer) after exiting the manu. It is not guaranteed to be the
buffer where the menu was called initially.

A more proper alternative could be allowing the menu to setup some
variables _before_ running the menu. For example, org-select can store
some setting as buffer-local variables in the menu buffer. This way, the
caller can fill some initial settings at the calling time. The settings
will then be available at the time user actually select the menu option.

> (defvar org-agenda--menu
>   '((("a" "Agenda for current week or day" (org-agenda--exec
>                                             'org-agenda-list))

It's nice that I can instead do

`((("a" ,(propertize "Agenda for current week or day"
                       'face 'highlight)
      (org-agenda--exec
       'org-agenda-list))

and the face is actually preserved (:

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>> this example the mode map approach seems slightly more convenient. I don't know,
>> in org-agenda-test, I haven't implemented all of org-agenda, restrictions,
>> prefixes and some other stuff, mostly because I don't really understand the
>> implementation.
>
> In the nutshell, agenda restrictions will execute some elisp that sets
> certain global variables affecting other agenda commands. selecting the
> restriction should not leave the agenda menu.
>
> Also, unlike other selections being echoed literally upon selection,
> restriction echo must depend on the global state. If you press "<" in
> the menu, the menu prompt should change between "Press key for agenda
> command (unrestricted):", Press key for agenda command (restricted to
> buffer):, ... etc
>
> Note that there is not much point echoing the selection.
> Tim, do I understand correctly that changed minibuffer prompt will be
> also spoken out by emacspeak?
>

I think so, but this would need to be verified. A lot depends on
how/where Emacs does things - for example, because Emacspeak relies on
the advice mechanism, it cannot pickup changes/actions which occur in
pure C code. Provided Emacspeak is able to get the prompt in some
before, after or around advice call, all should be fine. 

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 11/06/2022 12:26, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>> However if two org-protocol handlers are launched without specified
>> template then behavior of Org becomes confusing. I meant this case.
>> Currently reading key from minibuffer serves as a kind of
>> synchronization tool.
>>
>> Imagine what would happen if Emacs decided to show several capture menus
>> with keymap non-blocking interface in different virtual desktops.
>> Capture data should be saved somewhere till the user would discover
>> initially hidden menu.
> 
> Note that there is not much happening when capture menu is called. Only
> the link is stored into link ting. Otherwise, no capture data is
> altered. All the fragile staff is happening after selecting capture
> template.

Ihor, magic is impossible. If several captures may be requested in 
parallel then snapshot of data required to fill capture template should 
be stored somewhere at the moment when capture is initiated. Otherwise 
the user may kill the buffer she is going to capture before selecting 
particular template.

There are enough side band communication channels in Org. I did not 
remember a variable from which properties are obtained. Before I have 
realized that it is `org-store-link-plist', I noticed at least 
`org-overriding-default-time', `org-capture-initial'. Unsure that the 
list is complete.

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>> Note that there is not much happening when capture menu is called. Only
>> the link is stored into link ting. Otherwise, no capture data is
>> altered. All the fragile staff is happening after selecting capture
>> template.
>
> Ihor, magic is impossible. If several captures may be requested in 
> parallel then snapshot of data required to fill capture template should 
> be stored somewhere at the moment when capture is initiated. Otherwise 
> the user may kill the buffer she is going to capture before selecting 
> particular template.

Sure. That somewhere can be buffer-local variable inside the capture
menu buffer. Or global variable. Or closure. How is it relevant to the
capture menu?

Of course, using global variables will limit things to a single capture,
but it just means that if a user starts capture, leaves the capture menu
buffer, and then starts another capture, only the last capture will be
handled. Just like we have now.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 06/06/2022 06:05, Tim Cross wrote:
> 
> One very big warning I would like to raise to ensure it is taken into
> consideration is accessibility. This can have two significant effects
> with respect to the types of things you are doing -

Out of curiosity, you mentioned export menu. Would it help if if only 
top-level were presented at first (settings, HTML, LaTeX&PDF, ODT)? E.g. 
some custom variable would control how many options may be presented 
before hierarchical representation is activated.

For web pages I have seen recommendations to use ARIA (Web Accessibility 
Initiative – Accessible Rich Internet Applications) attributes like 
aria-role="button" or aria-ignore="true". Is there something similar in 
Emacs, e.g. text properties that helps Emacspeak to filter content and 
to "explain" its meaning?

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Max Nikulin <manikulin@gmail.com> writes:

> On 11/06/2022 12:26, Ihor Radchenko wrote:
>> Max Nikulin writes:
>> 
>>> However if two org-protocol handlers are launched without specified
>>> template then behavior of Org becomes confusing. I meant this case.
>>> Currently reading key from minibuffer serves as a kind of
>>> synchronization tool.
>>>
>>> Imagine what would happen if Emacs decided to show several capture menus
>>> with keymap non-blocking interface in different virtual desktops.
>>> Capture data should be saved somewhere till the user would discover
>>> initially hidden menu.
>> Note that there is not much happening when capture menu is called. Only
>> the link is stored into link ting. Otherwise, no capture data is
>> altered. All the fragile staff is happening after selecting capture
>> template.
>
> Ihor, magic is impossible. If several captures may be requested in parallel then
> snapshot of data required to fill capture template should be stored somewhere at
> the moment when capture is initiated. Otherwise the user may kill the buffer she
> is going to capture before selecting particular template.
>
> There are enough side band communication channels in Org. I did not remember a
> variable from which properties are obtained. Before I have realized that it is
> `org-store-link-plist', I noticed at least `org-overriding-default-time',
> `org-capture-initial'. Unsure that the list is complete.

I have a question here: what is meant by this:

>>> Imagine what would happen if Emacs decided to show several capture menus
>>> with keymap non-blocking interface in different virtual desktops.

Different Emacs processes, or just different Emacs frames?

In case of different Emacs processes, there is no way to guarantee consistence
unless one locks file in the file system. Windows can do it, I am not sure what
is Linux API to do this, don't know if Emacs exposes this functionality, have
never tried.

Otherewise, if it is only different Emacs frames/clients, the capture should
always find the capture buffer and return that one instead of creating new
ones. That way there is only one capture buffer, so multiple captures should not
be possible, i.el, it creates same effect as locking the input to minibuffer. I
am not sure how org-capture does, I haven't studied the code in-depth yet, but
what I see currently a user cancels it with C-c C-k. org-capture buffer could
setup hooks to clean everything, even if user kills buffer by other means, c-x
k, or whatever. It maybe already does, as said I haven't looked at those
details.

I just haven't done that in demo yet, so that is why I said when I posted the
code, I haven't implemented that "correctly", bit I am quite sure it is not very
hard to do.

Am I correct about the principle? If not, than I will have to rething about it.

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 18/06/2022 22:05, Arthur Miller wrote:
> Max Nikulin writes:
>> On 11/06/2022 12:26, Ihor Radchenko wrote:
>>> Max Nikulin writes:
>>>
>>>> Imagine what would happen if Emacs decided to show several capture menus
>>>> with keymap non-blocking interface in different virtual desktops.
> 
> Different Emacs processes, or just different Emacs frames?

I mean single Emacs process perhaps with multiple frames spread over 
monitors and virtual desktops. I am unsure if Emacs can create windows 
for different X11 displays, but let's leave it aside and inter-process 
file locks as well.

> In case of different Emacs processes, there is no way to guarantee consistence
> unless one locks file in the file system. Windows can do it, I am not sure what
> is Linux API to do this, don't know if Emacs exposes this functionality, have
> never tried.
> 
> Otherewise, if it is only different Emacs frames/clients, the capture should
> always find the capture buffer and return that one instead of creating new
> ones. That way there is only one capture buffer, so multiple captures should not
> be possible, i.el, it creates same effect as locking the input to minibuffer. I
> am not sure how org-capture does, I haven't studied the code in-depth yet, but
> what I see currently a user cancels it with C-c C-k. org-capture buffer could
> setup hooks to clean everything, even if user kills buffer by other means, c-x
> k, or whatever. It maybe already does, as said I haven't looked at those
> details.

Arthur, be reasonably skeptical concerning my ideas or suggestions, it 
seems I have not managed to convince e.g. Ihor.

I believe, multiple capture menus should be possible in parallel even 
within single frame since it may contain several windows and user 
experience should be better than now. Keymap-based selection opens a 
road in this direction since menu may be non-modal, but it requires a 
bit different design.

Waiting for return value to get capture template is not possible any 
more. A kind of continuations should be passed to the function creating 
menu buffer instead. E.g. it can be some state object that stores 
snapshot of data necessary to fill capture template, export options, 
etc. and functions in menu entries that accept this state object as 
argument or obtain it from a dynamic variable. The state object likely 
should be a buffer-local variable. For non-blocking menu, entries should 
contain callbacks or menu may have single callback that is able to 
process static data from menu entries.

As a result, capture can not be processed till filling of a template 
(and so till adding it to the target buffer) within a single function. 
Instead first function prepares set of callbacks and renders a buffer 
with menu. When user activates a menu entry, a callback either just 
modifies the state object to change some option or starts some action 
(fills capture template and inserts result to the target document) and 
notifies caller that the menu should be destroyed. E.g. if some special 
symbol is returned from the menu entry callback than it means change of 
some option, so menu should be retained, otherwise it is action and the 
menu buffer is not necessary any more.

So despite I earlier opposed to executable menu entries, they are quite 
natural way to implement non-blocking menu. State object specific to 
menu instance should be added in some way convenient for developers.

More work may be necessary however to make org-capture really convenient 
and reliable. Capture menu should display some summary of captured data 
otherwise it is impossible to decide which template should be chosen in 
the case of several simultaneous capture buffers. It is better to save 
capture data somewhere on disk while while menu is displayed to recover 
it after a crash.

> I agree with you that completing read is a good alternative, but it is a
> bit like discussion about GUI vs. terminal. I am personally heavy user
> of Helm, but not everyone is I believe.

I mentioned completing-read because I consider it as a test of API 
quality. It should be possible to plug alternative menu implementation 
and completing read may be a simple enough variant. It is blocking, but 
in the case of capture "push to capture queue" could be used to postpone 
the action.

P.S. Notice text properties for entries in the following modal menu:
Timothy to emacs-orgmode. [PATCH] New remote resource download policy. 
Sun, 12 Jun 2022 22:43:07 +0800. 
https://list.orgmode.org/87mteiq6ou.fsf@gmail.com

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 18/06/2022 15:25, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
>>> Note that there is not much happening when capture menu is called. Only
>>> the link is stored into link ting. Otherwise, no capture data is
>>> altered. All the fragile staff is happening after selecting capture
>>> template.
>>
>> Ihor, magic is impossible. If several captures may be requested in
>> parallel then snapshot of data required to fill capture template should
>> be stored somewhere at the moment when capture is initiated. Otherwise
>> the user may kill the buffer she is going to capture before selecting
>> particular template.
> 
> Sure. That somewhere can be buffer-local variable inside the capture
> menu buffer. Or global variable. Or closure. How is it relevant to the
> capture menu?

Before menu buffer is created, caller can not assign a buffer-local 
variable. So to be transparent for snapshot of capture data, menu should 
support such variable and should pass it further when template is 
chosen. Otherwise the capture data may be lost with temporary buffer. 
The function calling menu should gather values from all variables 
necessary for capture to build some state passed to menu implementation.

> Of course, using global variables will limit things to a single capture,
> but it just means that if a user starts capture, leaves the capture menu
> buffer, and then starts another capture, only the last capture will be
> handled. Just like we have now.

Now user may leave capture menu by either selection of a template or by 
aborting menu. In the case of keymap-based menu there is no such 
restriction, so org-capture and menu implementation should be adjusted 
in some way to avoid bad surprises for users.

Currently several capture menu instances may be requested though 
org-protocol (or calling org-capture from emacsclient). The behavior is 
rather confusing. New menu may help to fix the issue, that is why I 
raised the question about parallel captures.

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Max Nikulin <manikulin@gmail.com> writes:

> On 18/06/2022 22:05, Arthur Miller wrote:
>> Max Nikulin writes:
>>> On 11/06/2022 12:26, Ihor Radchenko wrote:
>>>> Max Nikulin writes:
>>>>
>>>>> Imagine what would happen if Emacs decided to show several capture menus
>>>>> with keymap non-blocking interface in different virtual desktops.
>> Different Emacs processes, or just different Emacs frames?
>
> I mean single Emacs process perhaps with multiple frames spread over monitors
> and virtual desktops. I am unsure if Emacs can create windows for different X11
> displays, but let's leave it aside and inter-process file locks as well.

Allright then; in that case what Ihor suggests, a variable somewhere, should
do. I suggest that variable be the capture buffer itself. I'll try to implement
this in a day or two; just need some time from other things in life.

>> In case of different Emacs processes, there is no way to guarantee consistence
>> unless one locks file in the file system. Windows can do it, I am not sure what
>> is Linux API to do this, don't know if Emacs exposes this functionality, have
>> never tried.
>> Otherewise, if it is only different Emacs frames/clients, the capture should
>> always find the capture buffer and return that one instead of creating new
>> ones. That way there is only one capture buffer, so multiple captures should not
>> be possible, i.el, it creates same effect as locking the input to minibuffer. I
>> am not sure how org-capture does, I haven't studied the code in-depth yet, but
>> what I see currently a user cancels it with C-c C-k. org-capture buffer could
>> setup hooks to clean everything, even if user kills buffer by other means, c-x
>> k, or whatever. It maybe already does, as said I haven't looked at those
>> details.
>
> Arthur, be reasonably skeptical concerning my ideas or suggestions, it seems I
> have not managed to convince e.g. Ihor.

:-). I think this would be quite a big and radical change in some important
parts of Org, so I really want to make sure that a fundamental error does not
happen. It would be a horrible thing if someones file with maybe lots of data
from the past gets messed up. That must not happen, so I really value the input
and would like to make sure use cases we have to take care of.

> I believe, multiple capture menus should be possible in parallel even within
> single frame since it may contain several windows and user experience should be
> better than now. Keymap-based selection opens a road in this direction since
> menu may be non-modal, but it requires a bit different design.

That sounds like a new feature. You are correct, keymaps do open in that
direction. That would be possible to tuck on the routine that saves the
temporary buffer (whatever is called with C-c C-c). When user press C-c C-c in a
capture buffer, that is the only time when interaction with real a file on the
disk happends, right? Everythign else is done in a temporary buffer in ram. User
can physically press it only in one capture buffer, so it least in theory,
multiple capture shouldn't be impossible or too hard to implement. But, lets do
it after the rework of the menu thing?

> Waiting for return value to get capture template is not possible any more. A
> kind of continuations should be passed to the function creating menu buffer
> instead. E.g. it can be some state object that stores snapshot of data necessary
> to fill capture template, export options, etc. and functions in menu entries
> that accept this state object as argument or obtain it from a dynamic
> variable. The state object likely should be a buffer-local variable. For
> non-blocking menu, entries should contain callbacks or menu may have single
> callback that is able to process static data from menu entries.

I happened to send the code yesterday by misstake only to one participant,
unfortunately, it wasn't ment; I seem to have pressed S w instead of S W and
wasn't looking at the address on top, so unfortunately you probably haven't seen
the code. However I have to rework it again because I did a bit bad design in
handling of one feature; I'll try to do it tomorrow.

However the idea is that the selection menu framework have no idea who is
calling it and why. It can not possibly know that either. It is entirely up to
the caller, so for that reason a caller can setup a handler that gets entire
template passed back and can do what it wants with it.

Current org-mks & Co don't really know who is calling than and why, but locking
happens automatically due to nature how the input is read (via minibuffer). In
case of new menu handling, the client has to do locking on its own, so it is a
bit more extra, but client can just buffer-get-create to get an existing buffer
before calling org-select, so it shouldn't be too much of the work. I
think. I'll test tomorrow or so.

> As a result, capture can not be processed till filling of a template (and so
> till adding it to the target buffer) within a single function. Instead first
> function prepares set of callbacks and renders a buffer with menu. When user
> activates a menu entry, a callback either just modifies the state object to
> change some option or starts some action (fills capture template and inserts
> result to the target document) and notifies caller that the menu should be
> destroyed. E.g. if some special symbol is returned from the menu entry callback
> than it means change of some option, so menu should be retained, otherwise it is
> action and the menu buffer is not necessary any more.

I have used a property 'transient' to flag that menu should be destroyed, and I
have done that before a handler is called, in osl--do-entry. But I it is a bad
thing. I came up with a different plan yesterday, and I will rework the meaning
of "transient" property there (btw if you have better word to skip confusion with
transient.el please suggest).

The idea is to have something like this:

    (defun org-capture--handler (entry &optional menu-buffer)

When a menu buffer is marked as "transient", it means that a client will take
ownership of the menu buffer and will can do whichever it prefer: keep it alive
or destroy it; in most cases destry it by reusing it, as in example of
org-capture where current code seems to recreate capture buffer in same
buffer.

> So despite I earlier opposed to executable menu entries, they are quite natural
> way to implement non-blocking menu. State object specific to menu instance
> should be added in some way convenient for developers.

Current proposition is meant to be backwards compatible with org-capture, so it
has to work with org-capture-templates modell of state. Means a list of form

(key label ....)

Sometimes it is not even a proper plist, but anyway, it is a list, i.e. a state
carrier. So IMO menu "framework" itself is just a presentation of choices to a
user. Hanlder is the logic, so I don't think menu framwork should do too much
here; just present choices to a user, and pass the choice to the handler. There
is default handler

(defun osl--default-handler-fn (entry)
  "Try to execute form found in ENTRY if any."
  (let* ((form (nth 2 entry))
         (message
          (cond
           ((commandp form)
            (call-interactively form))
           ((functionp form)
            (apply form (cddr entry)))
           (t (eval form)))))
    (if (stringp message) message)))

(I have to rework, that is the one from yesterday )

That can cover lots of simple cases, so most use-cases don't need to write
handler in one-shot actions, (even org-agenda seems to work with this scenario),
but more advanced uses should give a handler and deal with the state in hanlder.

> More work may be necessary however to make org-capture really convenient and
> reliable. Capture menu should display some summary of captured data otherwise it
> is impossible to decide which template should be chosen in the case of several
> simultaneous capture buffers. It is better to save capture data somewhere on
> disk while while menu is displayed to recover it after a crash.

See the comment about new feature above; I think that covers same issue. Sure
temporary file might be something, but that would be more work. Lets leave it
out for the particular moment, and come to it later.

>> I agree with you that completing read is a good alternative, but it is a
>> bit like discussion about GUI vs. terminal. I am personally heavy user
>> of Helm, but not everyone is I believe.
>
> I mentioned completing-read because I consider it as a test of API quality. It
> should be possible to plug alternative menu implementation and completing read
> may be a simple enough variant. It is blocking, but in the case of capture "push
> to capture queue" could be used to postpone the action.

I don't think it matters there that completing read is blocking. Users do
completing read just to pick one action to get executed. I don't think I
understand how qeue of captures to do would work, to be honest.

> P.S. Notice text properties for entries in the following modal menu:
> Timothy to emacs-orgmode. [PATCH] New remote resource download policy. Sun, 12
> Jun 2022 22:43:07 +0800. https://list.orgmode.org/87mteiq6ou.fsf@gmail.com

I am not sure what do you bring that cod up? I looked at the link, but I just
see text styling with faces. Do  you mean to use text properties to communicate
the state? Sure text properties can be used for different things, but already
have "template" (a list) we are passing around. Isn't it sufficient as a state
carrier. You can push/pop things off of it to communicate whatever. Forgive me
if I don't understand what you meant there.

best regards
/a

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

>>> Ihor, magic is impossible. If several captures may be requested in
>>> parallel then snapshot of data required to fill capture template should
>>> be stored somewhere at the moment when capture is initiated. Otherwise
>>> the user may kill the buffer she is going to capture before selecting
>>> particular template.
>> 
>> Sure. That somewhere can be buffer-local variable inside the capture
>> menu buffer. Or global variable. Or closure. How is it relevant to the
>> capture menu?
>
> Before menu buffer is created, caller can not assign a buffer-local 
> variable. So to be transparent for snapshot of capture data, menu should 
> support such variable and should pass it further when template is 
> chosen. Otherwise the capture data may be lost with temporary buffer. 
> The function calling menu should gather values from all variables 
> necessary for capture to build some state passed to menu implementation.

Sure. I was hinting about such functionality in the new library we are
discussing. Multiple capture menus are not possible now anyway, so it
will be a new feature if we decide that we need it (I am not 100% sure
if having multiple menus is not going to be confusing).

>> Of course, using global variables will limit things to a single capture,
>> but it just means that if a user starts capture, leaves the capture menu
>> buffer, and then starts another capture, only the last capture will be
>> handled. Just like we have now.
>
> Now user may leave capture menu by either selection of a template or by 
> aborting menu. In the case of keymap-based menu there is no such 
> restriction, so org-capture and menu implementation should be adjusted 
> in some way to avoid bad surprises for users.
>
> Currently several capture menu instances may be requested though 
> org-protocol (or calling org-capture from emacsclient). The behavior is 
> rather confusing. New menu may help to fix the issue, that is why I 
> raised the question about parallel captures.

I am not sure which behavior you have in mind.
What I was thinking as a conservative implementation that would not
introduce any new features is replacing the old menu with the new one
every time the same menu is called. So, every time the user calls menu
(e.g. capture menu), only the last capture environment is preserved. The
previous, potentially unfinished, capture menus will be destroyed.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 20/06/2022 19:10, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
> Sure. I was hinting about such functionality in the new library we are
> discussing. Multiple capture menus are not possible now anyway, so it
> will be a new feature if we decide that we need it (I am not 100% sure
> if having multiple menus is not going to be confusing).

In my opinion application logic should be consistent with UI. Do you 
think current behavior of help buffers has no problems? If so I have no 
chance to convince you that upgrade of menu should be accompanied by 
changes in org-capture.

Consider the following case. A user has 2 virtual desktops dedicated to 
rather independent tasks and an Emacs frame on each desktop. On desktop 
1 she opens help for some function with aim to evaluate if it is 
appropriate for some particular purpose. Next she has to switch to 
another task and applications for it reside on virtual desktop 2. To 
proceed further she has to read several help pages relevant for task 2. 
After completion of this task it is time to resume task 1 and to switch 
to virtual desktop 1. Does a window with the help buffer still display 
the same content as before switching to task 2? No, context on desktop 1 
lost due to some actions in the Emacs frame on desktop 2.

Such synchronization is against multitasking and I do not like the idea 
to get it for org-capture as well. Currently read-key and modal prompt 
is a kind of excuse, but the idea of new menu library is non-blocking 
keymap-based selection.

>> Currently several capture menu instances may be requested though
>> org-protocol (or calling org-capture from emacsclient). The behavior is
>> rather confusing. New menu may help to fix the issue, that is why I
>> raised the question about parallel captures.
> 
> I am not sure which behavior you have in mind.

try the following commands without selecting a template in an Emacs 
frame in between

     emacsclient 'org-protocol:/capture?url=url-A&title=title-A&body=body=A'
     emacsclient 'org-protocol:/capture?url=url-B&title=title-B&body=body=B'

> What I was thinking as a conservative implementation that would not
> introduce any new features is replacing the old menu with the new one
> every time the same menu is called. So, every time the user calls menu
> (e.g. capture menu), only the last capture environment is preserved. The
> previous, potentially unfinished, capture menus will be destroyed.

Causing loss of user data. Currently it is hard to start new capture 
before selecting a template.

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> Consider the following case. A user has 2 virtual desktops dedicated to 
> rather independent tasks and an Emacs frame on each desktop. On desktop 
> 1 she opens help for some function with aim to evaluate if it is 
> appropriate for some particular purpose. Next she has to switch to 
> another task and applications for it reside on virtual desktop 2. To 
> proceed further she has to read several help pages relevant for task 2. 
> After completion of this task it is time to resume task 1 and to switch 
> to virtual desktop 1. Does a window with the help buffer still display 
> the same content as before switching to task 2? No, context on desktop 1 
> lost due to some actions in the Emacs frame on desktop 2.
>
> Such synchronization is against multitasking and I do not like the idea 
> to get it for org-capture as well. Currently read-key and modal prompt 
> is a kind of excuse, but the idea of new menu library is non-blocking 
> keymap-based selection.

I understand.
From the perspective of the library being discussed, what you want does
not look impossible.

The other question is altering the org-capture code.
I think that it is too early to discuss org-capture part just yet.
Lets first finalize the generic library itself and discuss the
appropriate adjustments to
org-capture/org-agenda/org-export/org-todo/etc code later.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Max Nikulin <manikulin@gmail.com> writes:

> On 20/06/2022 19:10, Ihor Radchenko wrote:
>> Max Nikulin writes:
>> Sure. I was hinting about such functionality in the new library we are
>> discussing. Multiple capture menus are not possible now anyway, so it
>> will be a new feature if we decide that we need it (I am not 100% sure
>> if having multiple menus is not going to be confusing).
>
> In my opinion application logic should be consistent with UI. Do you think
> current behavior of help buffers has no problems? If so I have no chance to
> convince you that upgrade of menu should be accompanied by changes in
> org-capture.
>
> Consider the following case. A user has 2 virtual desktops dedicated to rather
> independent tasks and an Emacs frame on each desktop. On desktop 1 she opens
> help for some function with aim to evaluate if it is appropriate for some
> particular purpose. Next she has to switch to another task and applications for
> it reside on virtual desktop 2. To proceed further she has to read several help
> pages relevant for task 2. After completion of this task it is time to resume
> task 1 and to switch to virtual desktop 1. Does a window with the help buffer
> still display the same content as before switching to task 2? No, context on
> desktop 1 lost due to some actions in the Emacs frame on desktop 2.
>
> Such synchronization is against multitasking and I do not like the idea to get
> it for org-capture as well. Currently read-key and modal prompt is a kind of
> excuse, but the idea of new menu library is non-blocking keymap-based selection.

Yes, that is true, current Emacs help-buffer implementation is not
designed with your use-case in mind. If we leave out "virtual desktops",
it is unimportant detail. What you speak about is that user is working with
two different contexts at the same time and prefers to keep each context
in it's own frame. Emacs as a whole is not designed to work in the way I
percieve it has clean separation between contexts in each frame. Menu
buffer is "global" for entire Emacs process, and there are other
features of Emacs that does not work well in such scenarion. Some people
prefer to keep an Emacs process per project/task for that reason.

I can agree with you, that it would be nice if things were not designed
the way they are, but I don't have an opinion how org-capture should
behave in that regard, and how much work would it be to design it the
way you suggest. With other words, I say neither yes or no to anything
:). I would just like to remind that we are speaking of two things here;
one is menu buffer, that is replacing org-mks & co, as in org-agenda and
other places Ihor mentioned in earlier mails, and the other one is
org-capture itself.

>>> Currently several capture menu instances may be requested though
>>> org-protocol (or calling org-capture from emacsclient). The behavior is
>>> rather confusing. New menu may help to fix the issue, that is why I
>>> raised the question about parallel captures.
>> I am not sure which behavior you have in mind.
>
> try the following commands without selecting a template in an Emacs frame in
> between
>
>     emacsclient 'org-protocol:/capture?url=url-A&title=title-A&body=body=A'
>     emacsclient 'org-protocol:/capture?url=url-B&title=title-B&body=body=B'
>
>> What I was thinking as a conservative implementation that would not
>> introduce any new features is replacing the old menu with the new one
>> every time the same menu is called. So, every time the user calls menu
>> (e.g. capture menu), only the last capture environment is preserved. The
>> previous, potentially unfinished, capture menus will be destroyed.
>
> Causing loss of user data. Currently it is hard to start new capture before
> selecting a template.

Current org-capture is one at a time because of how org-mks works. There
is nothing that prevents org-capture to open enumerated buffers,
org-caputre<1>, org-capture<2> etc. User has to manually serialize data
anyway, via C-c C-c from withing capture buffer? So in principle it is
still one capture buffer at a time that manipulates the file on the disk
itself?

Problem would arise if serialization takes long time, i.e. there is lots
of data to write to file, and user switches to another org-capture
buffer and tries to save that one. So the serialization itself has to be
guarded somehow, but a mutex or file lock or something. At least I think
so, I am not sure if I am correct completely. Just trying to think loud
about it.

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Ihor Radchenko <yantar92@gmail.com> writes:

> Max Nikulin <manikulin@gmail.com> writes:
>
>> Consider the following case. A user has 2 virtual desktops dedicated to 
>> rather independent tasks and an Emacs frame on each desktop. On desktop 
>> 1 she opens help for some function with aim to evaluate if it is 
>> appropriate for some particular purpose. Next she has to switch to 
>> another task and applications for it reside on virtual desktop 2. To 
>> proceed further she has to read several help pages relevant for task 2. 
>> After completion of this task it is time to resume task 1 and to switch 
>> to virtual desktop 1. Does a window with the help buffer still display 
>> the same content as before switching to task 2? No, context on desktop 1 
>> lost due to some actions in the Emacs frame on desktop 2.
>>
>> Such synchronization is against multitasking and I do not like the idea 
>> to get it for org-capture as well. Currently read-key and modal prompt 
>> is a kind of excuse, but the idea of new menu library is non-blocking 
>> keymap-based selection.
>
> I understand.
> From the perspective of the library being discussed, what you want does
> not look impossible.
>
> The other question is altering the org-capture code.
> I think that it is too early to discuss org-capture part just yet.
> Lets first finalize the generic library itself and discuss the
> appropriate adjustments to
> org-capture/org-agenda/org-export/org-todo/etc code later.
>
I think it sounds like a wise proposal :).

Thanks.

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 21/06/2022 11:07, Ihor Radchenko wrote:
> Max Nikulin writes:
> 
> The other question is altering the org-capture code.
> I think that it is too early to discuss org-capture part just yet.
> Lets first finalize the generic library itself and discuss the
> appropriate adjustments to
> org-capture/org-agenda/org-export/org-todo/etc code later.

 From my point of view changing menu without adjusting calling logic is 
a kind of trade-off with uncertain net result. New UI may significantly 
affect different user expectations. It seems this is the point of our 
disagreement. You tend to consider menu in isolation trying to postpone 
the question concerning interaction of code running before menu creation 
and handlers of menu items (they are not belong to the same call stack 
any more).

What I am really afraid is words like:

> Sure. That somewhere can be buffer-local variable inside the capture
> menu buffer. Or global variable. Or closure. How is it relevant to the
> capture menu?

I am trying to avoid implementation of menu that would make passing 
state to handlers unreliable or prohibitively complicated.

Global variables are certainly not an option because such approach does 
not allow several instances of menu created in parallel.

To use buffer-local variables some support from the side of menu library 
is required. Buffer does not exist when menu function is called. So 
either some callback should be passed to menu to be invoked when a new 
buffer is created for menu or that buffer should be returned from the 
menu creating function. Unsure if (current-buffer) would be a reliable 
approach.

Closures should work, but would be it convenient enough?

Maybe some intermediate layer should be introduced to make alternative 
implementation of menu (transient, completing read, etc.) easier.

I believe, discussing design of menu without evaluation if it is 
suitable to features that should use it (capture, agenda, etc.), there 
is a risk to create unreasonable restrictions on further steps.

That is why I consider the following aspects of menu design as essential 
ones:
- A clearly determined way to pass some state from a function that 
requests creation of menu to handlers of menu items.
- When a non-blocking technique to wait user decision is used, multiple 
instances of the same menu should be allowed.
- Defined extension points for alternative implementations to avoid code 
duplication and divergence of behavior.

My fear is that if such points are ignored in the beginning, it may 
become impossible later.

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

[-- Attachment #1: Type: text/plain, Size: 6581 bytes --]

Max Nikulin <manikulin@gmail.com> writes:

> On 21/06/2022 11:07, Ihor Radchenko wrote:
>> Max Nikulin writes:
>> The other question is altering the org-capture code.
>> I think that it is too early to discuss org-capture part just yet.
>> Lets first finalize the generic library itself and discuss the
>> appropriate adjustments to
>> org-capture/org-agenda/org-export/org-todo/etc code later.
>
> From my point of view changing menu without adjusting calling logic is a kind of
> trade-off with uncertain net result. New UI may significantly affect different
> user expectations. It seems this is the point of our disagreement. You tend to
> consider menu in isolation trying to postpone the question concerning
> interaction of code running before menu creation and handlers of menu items
> (they are not belong to the same call stack any more).
>
> What I am really afraid is words like:
>
>> Sure. That somewhere can be buffer-local variable inside the capture
>> menu buffer. Or global variable. Or closure. How is it relevant to the
>> capture menu?

Yepp, it can't saved in a buffer-local before the buffer is created.

> I am trying to avoid implementation of menu that would make passing state to
> handlers unreliable or prohibitively complicated.

Menu should and application should be separated in my eyes. Menu is just a
graphical/audio? presentation of selectable choice to the user. As such it
should display choices, let user pick a choice, and return to the application
the picked choice. Compare to completing-read, or org-mks. I don't think we
should mingle application states and menu states.

However, there is a slight complication compared to completing-read or org-mks
in regard that those are used to make only one selection and exit, while this
menu is supposed to be used that way, or to be used in a way that it survives
multiple selections.

To me this seems as clearly per menu entry behaviour. For exmaple in org-agenda
menu, some actions will make menu exit, while some will keep the menu alive (for
example '*'). One solution is to have some flag passed per entry to signal to
the menu to kill itself upon selection. The other one, the easier, is to let the
applicaiton code handle the menu kill. I took the second one to minimize on
number of flags that needs to be passed and documented, but I agree it is not
the cleanest way, so I can change to use a flag if it is considered a better
option.

> Global variables are certainly not an option because such approach does not
> allow several instances of menu created in parallel.

Yes, please, lets try to minimize number of globals. Actually I would be
happiest with a functional approach where everything is communicated via
function arguments and return values.

> To use buffer-local variables some support from the side of menu library is
> required. Buffer does not exist when menu function is called. So either some
> callback should be passed to menu to be invoked when a new buffer is created for
> menu or that buffer should be returned from the menu creating function. Unsure
> if (current-buffer) would be a reliable approach.
>
> Closures should work, but would be it convenient enough?

We are using list to pass both menus and menu entries. Application can stick in
some args used by the menu. 

> Maybe some intermediate layer should be introduced to make alternative
> implementation of menu (transient, completing read, etc.) easier.

I don't think it is the topic and scope of this menu. It would be another layer
on its own, and definitely another project, I think.

> I believe, discussing design of menu without evaluation if it is suitable to
> features that should use it (capture, agenda, etc.), there is a risk to create
> unreasonable restrictions on further steps.

I definitely agree. Designing a good API is hard. I personally always think that
frameworks should not be designed in advance. I am more of a pragmatic, and
prefer to see actual working use-cases and design a framework after the
experience, not try to desing a framework by guessing the potential
use-cases. That is why I said that I am not interested in turning this into a
framework. I have try to design here something that can be used to implement
existing org-capture, org-agenda & co on top of it.

> That is why I consider the following aspects of menu design as essential ones:
> - A clearly determined way to pass some state from a function that requests
>  creation of menu to handlers of menu items.

Menu entries (lists)

> - When a non-blocking technique to wait user decision is used, multiple
>   instances of the same menu should be allowed.

Still have to be worked on.

> - Defined extension points for alternative implementations to avoid code
>   duplication and divergence of behavior.

That would be some framework layer.

> My fear is that if such points are ignored in the beginning, it may become
> impossible later.

It's lisp, anything is possible ^^ ;-).

No, but serioiusly, there is not much of extension points here. It is a keymap
and buffer text generated automatically from a list of templates and few
"visual" options. An application can even discard entire buffer and draw its own
text out of templates as it pleases, it wouldn't matter.

My goal was to create something very simple to use from a programming
perspective. Compared to other similar options like make-help-screen and org-mks
this puts both key handling, actions and labels into same spot, so there is only
one place to maintain later on, and client code needs needs just to specify the
most minimal lisp one is interested to execute, ie. I don't need to create
lambdas if some function is not already defined. Sort of, that is how at least I
would like to use it. I don't know if that is good or bad practice, or how
robust the code is, I am not super familiar with lisp and org-mode to be honest.

Attached is another experiment. In this version each entry can have attached
handler, and I provide two simple handers: run-once and run-multiple times. For
applications that need more advanced handling, they are supposed to provide own
handler and to take care of killing the menu buffer.

I do have thoughts to rework the drawing code once more to allow entries in form
of: (:separator sepeartor-string), similar as it takes group entries for
submenus. It would allow for mixing horizontal and vertical layouts, but I am
not sure if that is a good feature to have. It certainly isn't needed.

I have reworked a bit org-capture, but I haven't yet worked on org-agenda
restrictions and other details.


[-- Attachment #2: org-select.el --]
[-- Type: text/plain, Size: 24253 bytes --]

;;; org-select.el --- Build custom menus from declarative templates  -*- lexical-binding: t; -*-

;; Copyright (C) 2022  Arthur Miller

;; Author: Arthur Miller <arthur.miller@live.com>
;; Keywords: tools

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;; Org-select is a selection framework meant to be simple and easy to use. Its
;; job is to display a list of choices for a user to pick from and to hand in
;; the selection to the client application. Org-select can be used to allow
;; either a single selection at a time, or for repeated selections from a
;; menu-like text-buffer. 

;; One of goals with this framework is to be easy to setup for the client
;; code. For that reason, org-select uses simple template language modeled after
;; org-capture templates.

;;; Code:
\f
(require 'org-macs)

;;; User vars
\f
(defgroup org-select nil
  "Create menus from declarative templates."
  :prefix "org-select-"
  :prefix "osl--"
  :tag "Org Select"
  :group 'org)

(defcustom org-select-back-key [f10]
  "Used to render string for the horizontal separator."
  :type 'character
  :group 'org-select)

(defcustom org-select-horizontal-separator "|"
  "Used to render string for the horizontal separator."
  :type 'string
  :group 'org-select)

(defcustom org-select-vertical-separator "-"
  "Used to render string for the vetical separator."
  :type 'string
  :group 'org-select)

(defcustom org-select-key-decorator-chars ""
  "Characters used to decorate shortcut keys.

This string should contain only two characters, the first one for the left
decorator and the second one for the right decorator.

Example: string \"[]\" will render key \"C\" as \"[C]\"."
  :type 'string
  :group 'org-select)

(defcustom org-select-label-decorators (cons "..." "...")
  "Used to render string for the vetical separator."
  :type 'cons
  :group 'org-select)
\f
;;; Implementation
\f
(defvar-local osl--init nil)
(defvar-local osl--args nil)
(defvar-local osl--buffer nil)
(defvar-local osl--menu-begin nil)
(defvar-local osl--buffer-menu nil)
(defvar-local osl--longest-label 0)
(defvar-local osl--buffer-window nil)
(defvar-local org-select-mode-map nil)
(defvar-local osl--horizontal-layout nil)
(defvar-local osl--current-menu-column nil)

(defvar-local osl--handler-fn nil
  "The handler invoked when per-menu handler is not specified.

The default one is org-select-run-once.")

(define-minor-mode org-select-mode ""
  :interactive nil :global nil)

;;;; Help-functions
(defun osl--arg (key)
  (plist-get osl--args key))

(defun osl--init ()
  (buffer-local-value 'osl--init (current-buffer)))

(defun osl--prop (property list)
  "Return value of PROPERTY from irregular plist LIST."
  (cadr (member property list)))

(defun osl--ignore-key ()
  (interactive)
  (message "Invalid key %S"
           ;; I am not happy but it works somewhat
           (edmacro-format-keys (vector last-input-event))))

(defun org-select-quit (&optional abort-message buffer-or-name)
  "Callback to quit an org-select buffer.

If given, and optional ABORT-MESSAGE will be printed instead of the default
one. BUFFER-NAME can be used to quit org-select mode from a non org-select
buffer."
  (interactive)
  (let ((window (if buffer-or-name
                    (get-buffer-window buffer-or-name)
                  osl--buffer-window)))
    (when (window-live-p window)
      (select-window window)
      (quit-window (buffer-live-p buffer-or-name) window))
    (message (or abort-message "Org Select Quit"))))

(defun osl--make-mode-map ()
  (let ((map (make-sparse-keymap)))
    (define-key map [?q] #'org-select-quit)
    (define-key map [?\C-g] #'org-select-abort)
    (define-key map [?\C-p] #'osl--back)
    (define-key map [remap newline] #'osl--ignore-key)
    (define-key map [remap self-insert-command] #'osl--ignore-key)
    (setq org-select-mode-map map)
    (use-local-map org-select-mode-map)))

(defun org-select-abort ()
  (interactive)
  (org-select-quit "Aborted"))

(defun osl--back ()
  (interactive)
  (when (bound-and-true-p org-select-mode)
    (osl--make-mode-map)
    (osl--draw)))

(defun osl--longest-line ()
  "Return the length of the longest line in current buffer."
  (let ((n 1) (L 0) (e 0) (E (point-max)) l)
    (while (< e E)
      (setq e (line-end-position n)
            l (- e (line-beginning-position n))
            n (1+ n))
      (if (> l L) (setq L l)))
    L))

(defun osl--decorate-key (key)
  "Place string KEY between characters specified in DECORATOR string."
  (let ((kd (if (> (length org-select-key-decorator-chars) 0)
                org-select-key-decorator-chars
              (osl--arg :key-decorator))))
    (if (= (length kd) 2)
        (concat (substring kd 0 1) key (substring kd 1))
      key)))

(defun osl--decorate-label (entry)
  (let ((left (car org-select-label-decorators))
        (right (cdr org-select-label-decorators)))
    (if (= (length entry) 2)
        (concat left (cadr entry) right)
      (cadr entry))))

(defun osl--make-separator (&optional marker length)
  (let ((len (or length (osl--longest-line)))
        (sep (if (osl--arg :horizontal)
                 org-select-horizontal-separator
               org-select-vertical-separator)))
    (if marker
        (concat "sep" sep)
      (make-string len (string-to-char sep)))))

(defun osl--insert-horizontal-separator (sep)
  (goto-char 1)
  (let ((lol (osl--longest-line))
        (sep (or org-select-horizontal-separator sep)))
    (while (not (eobp))
      (let* ((eol (line-end-position))
             (bol (line-beginning-position))
             (fill (- (+ bol lol) eol)))
        (goto-char eol)
        (if (> fill 0)
            (while (> fill 0) (insert " ") (setq fill (1- fill)))
          (while (> 0 fill) (delete-char 1) (setq fill (1+ fill))))
        (insert " " sep " "))
      (forward-line))
    (setq osl--current-menu-column (+ lol (length sep) 2))))

(defun osl--insert-separator (sep)
  (if (osl--arg :horizontal)
      (osl--insert-horizontal-separator sep)
    (insert sep)))

(defun osl--insert (&rest strings)
  (cond
   ((and (osl--arg :horizontal)
         (> osl--current-menu-column 0))
    (goto-char (+ (line-beginning-position)
                  osl--current-menu-column))
    (apply #'insert strings)
    (if (char-after)
        (forward-line)
      (insert "\n")))
   (t 
    (apply #'insert strings)
    (insert "\n"))))

(defun osl--forward-menu ()
  (cond
   ((osl--arg :horizontal)
    (goto-char (point-min))
    (goto-char (line-end-position))
    (setq osl--current-menu-column
          (- (point) (line-beginning-position))))
   (t (insert "\n"))))

;;;; Menu drawing
(defun osl--setup-buffer (tables args)
  "Setup buffer local variables needed for an org-select buffer."
  (let* ((buffer (or (plist-get args :buffer-name) "*Org-select: "))
         (window (get-buffer-window buffer)))
      (if window
          (select-window window)
        (org-switch-to-buffer-other-window buffer))
    (with-current-buffer (get-buffer buffer)
      (special-mode)
      (setq cursor-type nil)
      (org-select-mode)
      (osl--make-mode-map)
      (setq osl--args args
            osl--buffer-menu tables
            osl--current-menu-column 0
            osl--buffer (current-buffer)
            osl--buffer-window (get-buffer-window)))))

(defun osl--draw ()
  "Starts menu parsing."
  (with-silent-modifications
    (erase-buffer)
    (setq osl--init nil)
    (let ((marker (osl--make-separator 'marker))
          (text (osl--arg :text))
          (menus (buffer-local-value
                  'osl--buffer-menu (current-buffer))))
      (setq osl--menu-begin (point))
      ;; given a list of menus, display one menu at a time
      (dolist (menu menus)
        (cond ((symbolp menu) (setq menu (eval menu)))
              ((symbolp (car menu)) (setq menu (eval (car menu)))))
        (let ((handler (osl--prop :org-select-handler menu)))
          (when handler
            (setq menu (delete :org-select-handler (delete handler menu))))
          (osl--do-menu menu (or handler #'org-select-run-once)))
        (setq menus (cdr menus))
        (when menus
          (osl--insert-separator marker)
          (osl--forward-menu)))
      ;; redraw markers with real separator strings
      (goto-char 1)
      (let ((sep (osl--make-separator nil (osl--longest-line))))
        (while (search-forward marker nil t)
          (replace-match "")
          (osl--insert-separator sep)))
      ;; insert info text if any
      (when text
        (goto-char 1)
        (insert "\n" text "\n"))
      (org-fit-window-to-buffer)
      (setq osl--init t)
      (goto-char 1)))) ; unnecessary but prettier if beacon-mode is active

;; iterate through menu and render a single entry or a group of entries on each
;; iteration
(defun osl--do-menu (menu handler)
  "Insert one menu at a time."
  (while menu
    (let ((entry (car menu)))
      (setq menu
            (if (> (length entry) 2)
                (osl--do-entry menu handler)
              (osl--do-group menu handler))))))

(defun osl--do-group (menu handler)
  "Do a menu with group nodes."
  (let ((group (car menu))
        newmenu)
    (osl--do-entry menu handler)
    (while (> (length (cadr menu)) 2)
      (let (entry newentry key)
        (setq menu (cdr menu) entry (car menu))
        (setq key (substring (car entry) 1))
        (push key newentry)
        (dolist (elt (cdr entry)) (push elt newentry))
        (push (nreverse newentry) newmenu)))
    (setq newmenu (nreverse newmenu))
    (define-key org-select-mode-map (kbd (car group))
                (lambda ()
                  (interactive)
                  (with-silent-modifications
                    (erase-buffer)
                    (setq osl--current-menu-column 0)
                    (osl--do-menu newmenu handler))))
    (cdr menu))) ;; return next group in chain

;; we send in the entire menu so we can return next piece in chain,
;; but *the* entry we work with is just the very first one (car menu)
(defun osl--do-entry (menu handler)
  "Display a single entry in the buffer."
  (let* ((entry (car menu))
         (key (car entry))
         (line-length 0)
         (handler (or (osl--prop :org-select-handler entry) handler)))
    (define-key org-select-mode-map (kbd key)
                (lambda ()
                  (interactive)
                  (let ((label (nth 1 entry))
                        (init (buffer-local-value 'osl--init osl--buffer)))
                    (and init handler
                         (message (or (funcall handler entry (current-buffer))
                                      label))))))
    (osl--insert (osl--decorate-key key) "    " (osl--decorate-label entry))
    (setq line-length (- (line-end-position) (line-beginning-position)))
    (if (> line-length osl--longest-label)
        (setq osl--longest-label line-length))
    (cdr menu)))

(defun org-select-run (entry &optional _org-select-buffer)
  "Try to execute form found in ENTRY if any leave ORG-SELECT-BUFFER live.

This handler provides an easy way to use the framework for the simple
use-cases for multiple choices. It relies on the user to press built-in choice
`q' or `C-g' to exit the menu."
  (let* ((form (nth 2 entry))
         (message
          (cond
           ((commandp form)
            (call-interactively form))
           ((functionp form)
            (apply form (cddr entry)))
           (t (eval form)))))
    (if (stringp message) message)))

(defun org-select-run-once (entry &optional org-select-buffer)
  "Try to execute form found in ENTRY if any and kill ORG-SELECT-BUFFER.

Provides an easy way to use the framework for the simple use-cases for multiple
choices. It relies on the user to press built-in choice `q' or `C-g' to exit the
menu."
  (if org-select-buffer (org-select-quit ""))
  (let* ((form (nth 2 entry))
         (message
          (cond
           ((commandp form)
            (call-interactively form))
           ((functionp form)
            (apply form (cddr entry)))
           (t (eval form)))))
    (if (stringp message) message)))
\f
;;; API
\f
(defun org-select (menus &rest args)
  "Select a member of an alist with multiple keys.

MENUS is a list of menus which themselves are lists containing entries in one of
following two formats:

1. prefix descriptions like (\"a\" \"Description\")
   This indicates that `a' is a prefix key for multi-letter selection, and
   that there are entries following with keys like \"ab\", \"ax\"...

2. Select-able members must have more than two elements, with the first
   being the string of keys that lead to selecting it, and the second a
   short description string of the item. 

The command will then make a temporary buffer listing all entries
that can be selected with a single key, and all the single key
prefixes.  When you press the key for a single-letter entry, it is selected.
When you press a prefix key, the commands (and maybe further prefixes)
under this key will be shown and offered for selection.

Each menu can be followed by some properties in form of a keu-value pair. The
entire menu or entry does not need to be a regular plist. Following keys are
recognized:

:org-select-pin     Pin this menu in org-select buffer. If group nodes are used,
                    when this option is `t', keep this menu visible even when
                    descending into a submenu. ;; FIXME Not implemented yet.
:org-select-handler Use this function to handle this particular menu or
                    entry. When none is specified, org-select uses
                    `org-select-run-once' to hande the menu. Entry handler
                    takes precedence over menu handler.

If there are more than one menus, they will be separated by a separator line
rendered with character as specified in `org-select-horizontal-separator'.

ARGS is a org-select buffer or entry property list containing following members:

:text          a string placed over selections in the buffer.
:buffer-name   a string used for the selections buffer name.
:key-decorator a two-character string used to decorate command characters. A
               menu can specify this string, but the precedence will be given
               the global variable `org-select-key-decorator-chars'. This to
               ensure that users can customize the appearance of the menus.

Properties in ARGS list are global for the entire org-select buffer."
  
  (osl--setup-buffer menus args)
  (osl--draw))
\f
;;; Demo
\f
;;;; org-capture
\f
(require 'org)
(require 'org-capture)

(defvar org-capture--current-goto nil)
(defvar org-capture--current-keys nil)
(defvar org-capture--old-window-config nil)

(defun org-capture-test (&optional goto keys)
  "Simple illustration to recreate org-capture menu (visually only)."
  (interactive "P")
  (let ((org-select-vertical-separator "-")
        (org-capture-templates
	   (or (org-contextualize-keys
	        (org-capture-upgrade-templates org-capture-templates)
	        org-capture-templates-contexts)
	       '(("t" "Task" entry (file+headline "" "Tasks")
		  "* TODO %?\n  %u\n  %a")))))
      (if keys
	  (or (assoc keys org-capture-templates)
	      (error
               "No capture template referred to by \"%s\" keys" keys)))
  (cond
   ((equal goto '(4))  (org-capture-goto-target keys))
   ((equal goto '(16)) (org-capture-goto-last-stored))
   (t
    (if goto (setq org-capture--current-goto goto))
    (push :org-select-handler org-capture-templates)
    (push #'org-capture--handler org-capture-templates)
    (org-select
     ;; tables
     `(,(nreverse org-capture-templates)
       (("C" "Customize org-capture-templates"
         (customize-variable 'org-capture-templates))
        ("q" "Abort" (org-select-quit "Abort"))))
     ;; description
     :buffer-name "*Capture*" :key-decorator "[]"
     :text "Select a capture template\n========================="))))
  (message "Org Capture"))
;;(define-key global-map (kbd "C-v c") #'org-capture-test)

(defun org-capture--handler (entry org-select-buf)
  (org-select-quit "" org-select-buf)
  (let* ((capture-buf (generate-new-buffer "*Capture*"))
         (annotation (if (and (boundp 'org-capture-link-is-already-stored)
			      org-capture-link-is-already-stored)
			 (plist-get org-store-link-plist :annotation)
		       (ignore-errors (org-store-link nil))))
	 (entry (or org-capture-entry entry))
         (goto org-capture--current-goto)
         (inhibit-read-only t)
	 initial)
    (setq initial (or org-capture-initial
		      (and (org-region-active-p)
			   (buffer-substring (point) (mark)))))
    (when (stringp initial)
      (remove-text-properties 0 (length initial) '(read-only t) initial))
    (when (stringp annotation)
      (remove-text-properties 0 (length annotation)
			      '(read-only t) annotation))
    (org-capture-set-plist entry)
    (org-capture-get-template)
    (org-capture-put :original-buffer capture-buf
		     :original-file (or (buffer-file-name capture-buf)
					(and (featurep 'dired)
					     (car (rassq capture-buf
							 dired-buffers))))
		     :original-file-nondirectory
		     (and (buffer-file-name capture-buf)
			  (file-name-nondirectory
			   (buffer-file-name capture-buf)))
		     :annotation annotation
		     :initial initial
		     :return-to-wconf (current-window-configuration)
		     :default-time (or org-overriding-default-time
				       (org-current-time)))
    (org-capture-set-target-location (and (equal goto 0) 'here))
    (condition-case error
	(org-capture-put :template (org-capture-fill-template))
      ((error quit)
       (if (get-buffer capture-buf) (kill-buffer capture-buf))
       (error "Capture abort: %s" (error-message-string error))))
    (setq org-capture-clock-keep (org-capture-get :clock-keep))
    (condition-case error
	(org-capture-place-template
	 (eq (car (org-capture-get :target)) 'function))
      ((error quit)
       (when (and (buffer-base-buffer (current-buffer))
		  (string-prefix-p "CAPTURE-" (buffer-name)))
	 (kill-buffer (current-buffer)))
       (set-window-configuration (org-capture-get :return-to-wconf))
       (error "Capture template `%s': %s"
	      (org-capture-get :key)
	      (error-message-string error))))
    (when (and (derived-mode-p 'org-mode) (org-capture-get :clock-in))
      (condition-case nil
	  (progn
	    (when (org-clock-is-active)
	      (org-capture-put :interrupted-clock
			       (copy-marker org-clock-marker)))
	    (org-clock-in)
	    (setq-local org-capture-clock-was-started t))
	(error "Could not start the clock in this capture buffer")))
    (when (org-capture-get :immediate-finish)
      (org-capture-finalize))))
\f
;;;; Org Agenda
\f
(require 'org-agenda)
(defvar org-agenda--arg nil)
(defvar org-agenda--keys nil)
(defvar org-agenda--restriction nil)

(defun org-agenda-test (&optional _arg _keys _restriction)
  (interactive "P")
  (let ((org-select-horizontal-separator " "))
    (org-select
     '((("a" "Agenda for current week or day" org-agenda-list)
        ("t" "List of all TODO entries"       org-todo-list)
        ("m" "Match a TAGS/PROP/TODO query"   org-tags-view)
        ("s" "Search for keywords"            org-search-view)
        ("/" "Multi-occur"                    (call-interactively
                                               'org-occur-in-agenda-files)
                                               :org-select-inhibit-transient t)
        ("?" "Find :FLAGGED: entries"         (org-tags-view
                                               nil "+FLAGGED"))
        ("*" "Toggle sticky agenda views"     org-toggle-sticky-agenda
                                              :org-select-handler org-select-run))
       (("<" "Buffer, subtree/region restriction" ignore)
        (">" "Remove restriction"             ignore)
        ("e" "Export agenda views"            org-store-agenda-views)
        ("T" "Entries with special TODO kwd" (org-call-with-arg
                                              'org-todo-list
                                              (or org-agenda--arg '(4))))
        ("M" "Like m, but only TODO entries" (org-call-with-arg
                                              'org-tags-view
                                              (or org-agenda--arg '(4))))
        ("S" "Like s, but only TODO entries" (org-call-with-arg
                                              'org-search-view
                                              (or org-agenda--arg '(4))))
        ("C" "Configure custom agenda commands"
         (customize-variable 'org-agenda-custom-commands))
        ("#" "List stuck projects" (org-agenda--exec
                                    'org-agenda-list-stuck-projects))
        ("!" "Configure stuck projects"
         (customize-variable 'org-stuck-projects))))
    :text
    "Press key for an agenda command:
--------------------------------\n"
    :horizontal t :buffer-name "*Agenda Commands*")))
\f
;;;; Various tests
\f
(defun test1 ()
  "Stays after a choice is made."
  (interactive)
  (let ((org-select-horizontal-separator "│"))
    (org-select
     ;; table
     '((("1" "One" (message "One!"))
        ("2" "Two" (message "Two!!"))
        ("3" "Three" (message "Three!!!")))
       (("C-4" "Four" (message "Four!!!!"))
        ("C-5" "Five" (message "Five!!!!!"))
        ("C-6" "six" (message "Six!")))
       (("M-7" "Seven" (message "Seven!"))
        ("M-8" "Eight" (message "Eight!"))
        ("M-9" "Nine" (message "Nine!"))))
     ;; description
     :horizontal t :key-decorator "<>")))

(defun test2 ()
  "Dissapears after a choice is made."
  (interactive)
  (let ((org-select-horizontal-separator "│"))
    (org-select
     ;; menus
     '((("h" "Hello, World!" (message "Hello, World!"))
        ("b" "Bar" (message "Hello, Bar!")))
       (("f" "Find File" find-file)
        ("o" "Open File" (flet ((next-read-file-uses-dialog-p () t))
                           (call-interactively #'find-file)))))
     ;; description
     :key-decorator "\"\"" :transient t)
      ;; Hints
      (setq header-line-format
	    (if (not (pos-visible-in-window-p (point-max)))
	        "Use C-v, M-v, C-n or C-p to navigate. C-g, q to quit."
              "Use C-p/Left to go back, C-g, q to quit."))))

(defun test3 ()
  "Illustrate nested menus, unicode separator and alternative decorator."
  (interactive)
  (let ((org-select-vertical-separator "─"))
    (org-select
     ;; tables
     '((("g" "Greetings")
        ("gh" "Hello, World!" (message "Hello, World!"))
        ("gb" "Bar" (message "Hello, Bar!")))
       (("f" "Functions")
        ("ff" "Find File" find-file)
        ("fo" "Open File" (flet ((next-read-file-uses-dialog-p () t))
                            (call-interactively #'find-file)))))))
      ;; Hints
      (setq header-line-format
	    (if (not (pos-visible-in-window-p (point-max)))
	        "Use C-v, M-v, C-n or C-p to navigate. C-g, q to quit."
              "Use C-p/Left to go back, C-g, q to quit.")))

  (provide 'org-select)

;;; org-select.el ends here

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 22/06/2022 19:13, Arthur Miller wrote:
> Max Nikulin writes:
> 
> Menu should and application should be separated in my eyes. Menu is just a
> graphical/audio? presentation of selectable choice to the user. As such it
> should display choices, let user pick a choice, and return to the application
> the picked choice. Compare to completing-read, or org-mks. I don't think we
> should mingle application states and menu states.

By state I mean some structure opaque to menu package. It just receives 
it from caller as an optional argument and (when given) later passes it 
to handler. Maybe it is alien approach in LISP, but in C (where closures 
are impossible) it is a widely used technique. Functions having callback 
argument usually have another void* one that is later passed as an 
argument of the callback function in addition to other data.

(org-buffer-menu
  '(("a" "Option A" (1))
    ("b" "Option B" (2)))
  :handler
  (lambda (entry state)
   (message "value %S state %S" (nth 2 entry) state))
  :state
  '(1 2 3))

Menu caller does not care whether some buffer is created to present 
menu. Menu has no idea what state may contain, it just passes it to the 
handler. When menu is implemented as a buffer with keymap then state is 
stored as a buffer-local variable.

As to your current implementation of org-select, I do not like that it 
is responsibility of caller to create different buffer names to enable 
multiple instances of menu. I would prefer to control single/multiple 
instances through a boolean argument of org-select.

Arthur, I see that I should response to more your comments. However I am 
unable to identify source of disagreement, likely it is some different 
assumptions. That is why I decided to concentrate on a couple of 
questions in this message.

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

> I have reworked a bit org-capture, but I haven't yet worked on org-agenda
> restrictions and other details.

I do not think that you need to prioritize re-creating
org-capture/org-agenda/etc The main point is making sure that org-select
provides the required functionality.
I'd prefer to first finalize the API and get a cleaner code of
org-select itself.

> (define-minor-mode org-select-mode ""
>   :interactive nil :global nil)

Why don't you just make it a major mode derived from special-mode? It
will make more sense considering that you are setting special-mode,
keymap, etc.

> (defun osl--prop (property list)
>   "Return value of PROPERTY from irregular plist LIST."
>   (cadr (member property list)))

FYI, there is plist-get

> (defun osl--init ()
>   (buffer-local-value 'osl--init (current-buffer)))

This is no different than just saying osl--init.

> (defun org-select-abort ()
>   (interactive)
>   (org-select-quit "Aborted"))

Please make sure that all the functions and variables have docstrings.
This is not just a boring convention, it really helps when you come back
to the code in future and when other people are reading your code.

> (defun osl--longest-line ()
>   "Return the length of the longest line in current buffer."
>   (let ((n 1) (L 0) (e 0) (E (point-max)) l)
>     (while (< e E)
>       (setq e (line-end-position n)
>             l (- e (line-beginning-position n))
>             n (1+ n))
>       (if (> l L) (setq L l)))
>     L))

Please do not use single-char variable names for non-trivial variables.
It is always better to provide self-explanatory names. It is not a
programming context. We are targeting better readability, not fast
typing.

>       (dolist (menu menus)
>         (cond ((symbolp menu) (setq menu (eval menu)))
>               ((symbolp (car menu)) (setq menu (eval (car menu)))))
>         (let ((handler (osl--prop :org-select-handler menu)))
>           (when handler
>             (setq menu (delete :org-select-handler (delete handler menu))))

Destructive modifications of arguments is a bad idea. I expect future
bugs in such code. Please avoid this approach.

> ;; we send in the entire menu so we can return next piece in chain,
> ;; but *the* entry we work with is just the very first one (car menu)
> (defun osl--do-entry (menu handler)
>   "Display a single entry in the buffer."

AFAIU, the comment on top belongs to the docstring. At least the part
talking about the return value. If the function is expected to return
something, it should be documented. Otherwise, I expect bugs in future.

> (defun org-select-run (entry &optional _org-select-buffer)
>   "Try to execute form found in ENTRY if any leave ORG-SELECT-BUFFER live.
>
> This handler provides an easy way to use the framework for the simple
> use-cases for multiple choices. It relies on the user to press built-in choice
> `q' or `C-g' to exit the menu."

Please, do not use key bindings verbatim in docstring. Prefer commands.
Docstrings do have special format for auto-detecting command bindings.
See D.6 Tips for Documentation Strings section of Elisp manual.

>       ;; given a list of menus, display one menu at a time
>       (dolist (menu menus)
> 	(cond ((symbolp menu) (setq menu (eval menu)))
> 	      ((symbolp (car menu)) (setq menu (eval (car menu)))))

Please avoid eval when possible. It can behave not nicely in
lexical/dynamic scope.

> Each menu can be followed by some properties in form of a keu-value
> pair. The
                                                            ^key
> entire menu or entry does not need to be a regular plist. Following keys are
> recognized:
>
> :org-select-pin     Pin this menu in org-select buffer. If group nodes are used,
>                     when this option is `t', keep this menu visible even when
>                     descending into a submenu. ;; FIXME Not implemented yet.
> :org-select-handler Use this function to handle this particular menu or
>                     entry. When none is specified, org-select uses
>                     `org-select-run-once' to hande the menu. Entry handler
>                     takes precedence over menu handler.

This is confusing. What do you mean by "does not need to be a regular
plist"? What do you mean by "menu can be followed"? Do you imply that
MENUS may not be a list of MENU lists, but can also contain :key value
pairs?

In general, I'd prefer format similar to
https://github.com/progfolio/doct/
or at least similar to org-capture-templates
The global ARGS in org-select could be defined using cl-defun style. See
2.1 Argument Lists section of CL manual.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Ihor Radchenko <yantar92@gmail.com> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>> I have reworked a bit org-capture, but I haven't yet worked on org-agenda
>> restrictions and other details.
>
> I do not think that you need to prioritize re-creating
> org-capture/org-agenda/etc The main point is making sure that org-select
> provides the required functionality.

That is just my way to test if it provides features needed, and how they
would reflect on real use-case such as org-capture and org-agenda. It is
just for demo purpose so to say.

> I'd prefer to first finalize the API and get a cleaner code of
> org-select itself.
 
Yes, that is my goal too. I just thought it was illustrative in context
of those two.

>> (define-minor-mode org-select-mode ""
>>   :interactive nil :global nil)
>
> Why don't you just make it a major mode derived from special-mode? It
> will make more sense considering that you are setting special-mode,
> keymap, etc.

I don't remember any more :). The first version I made was derived from
special mode, but than I switched to minor mode. I don't remember why
atm, to be honset, but I don't think there is anything prohibitive to
derive it from special mode either.

>> (defun osl--prop (property list)
>>   "Return value of PROPERTY from irregular plist LIST."
>>   (cadr (member property list)))
>
> FYI, there is plist-get

Yes I know, I have been using it. However this one is a bit
different. plist-get works on regular lists, this one works on
irregular. This one just match key-value in any list, or to be correct,
it matches key and return next element as the value.

I should have documented, but I meant to document them later on, this
is just for my testing and demoing.

>> (defun osl--init ()
>>   (buffer-local-value 'osl--init (current-buffer)))
>
> This is no different than just saying osl--init.
>
>> (defun org-select-abort ()
>>   (interactive)
>>   (org-select-quit "Aborted"))
>
> Please make sure that all the functions and variables have docstrings.
> This is not just a boring convention, it really helps when you come back
> to the code in future and when other people are reading your code.

Yepp. As said, this was just while I am testing and demoing. Since I
tend to change lots in the code, I am finding myself constantly typing
and erasing stuff, but I guess I should have been more clear when
sending in code to others. I also forgott to make patch and sent in
everything, wasn't really meaning :):

>> (defun osl--longest-line ()
>>   "Return the length of the longest line in current buffer."
>>   (let ((n 1) (L 0) (e 0) (E (point-max)) l)
>>     (while (< e E)
>>       (setq e (line-end-position n)
>>             l (- e (line-beginning-position n))
>>             n (1+ n))
>>       (if (> l L) (setq L l)))
>>     L))
>
> Please do not use single-char variable names for non-trivial variables.
> It is always better to provide self-explanatory names. It is not a
> programming context. We are targeting better readability, not fast
> typing.

Ah, that one is special :). e = end, l = line length L = longest
length. I espect myself to rework that one, but yes normally I use
self-docummented code. Sure np, I'll try to not send in short-named ones.

>>       (dolist (menu menus)
>>         (cond ((symbolp menu) (setq menu (eval menu)))
>>               ((symbolp (car menu)) (setq menu (eval (car menu)))))
>>         (let ((handler (osl--prop :org-select-handler menu)))
>>           (when handler
>>             (setq menu (delete :org-select-handler (delete handler menu))))
>
> Destructive modifications of arguments is a bad idea. I expect future
> bugs in such code. Please avoid this approach.

Ok. I can rework it by copying the list, but I am not sure if it adds
much of the value since the original one is not used after this
point. But if it is believed to lead to bugs, I can of course change it,
its not a problem.

>> ;; we send in the entire menu so we can return next piece in chain,
>> ;; but *the* entry we work with is just the very first one (car menu)
>> (defun osl--do-entry (menu handler)
>>   "Display a single entry in the buffer."
>
> AFAIU, the comment on top belongs to the docstring. At least the part
> talking about the return value. If the function is expected to return
> something, it should be documented. Otherwise, I expect bugs in future.

Ok.

>> (defun org-select-run (entry &optional _org-select-buffer)
>>   "Try to execute form found in ENTRY if any leave ORG-SELECT-BUFFER live.
>>
>> This handler provides an easy way to use the framework for the simple
>> use-cases for multiple choices. It relies on the user to press built-in choice
>> `q' or `C-g' to exit the menu."
>
> Please, do not use key bindings verbatim in docstring. Prefer commands.
> Docstrings do have special format for auto-detecting command bindings.
> See D.6 Tips for Documentation Strings section of Elisp manual.

Allright, I didn't know about key bindings format in docs. I'll check on
it, thanks.

>>       ;; given a list of menus, display one menu at a time
>>       (dolist (menu menus)
>> 	(cond ((symbolp menu) (setq menu (eval menu)))
>> 	      ((symbolp (car menu)) (setq menu (eval (car menu)))))
>
> Please avoid eval when possible. It can behave not nicely in
> lexical/dynamic scope.
>
>> Each menu can be followed by some properties in form of a keu-value
>> pair. The
>                                                             ^key
>> entire menu or entry does not need to be a regular plist. Following keys are
>> recognized:
>>
>> :org-select-pin     Pin this menu in org-select buffer. If group nodes are used,
>>                     when this option is `t', keep this menu visible even when
>>                     descending into a submenu. ;; FIXME Not implemented yet.
>> :org-select-handler Use this function to handle this particular menu or
>>                     entry. When none is specified, org-select uses
>>                     `org-select-run-once' to hande the menu. Entry handler
>>                     takes precedence over menu handler.
>
> This is confusing. What do you mean by "does not need to be a regular
> plist"? What do you mean by "menu can be followed"? Do you imply that
> MENUS may not be a list of MENU lists, but can also contain :key value
> pairs?
Yepp. That is also the reason for osl--prop above, and why am I using
"destructive approach" when I remove the handler from the list before
rendering it.

Consider simple case of (key label form) that can be executed directly
as in tests at the end. I can than add a key-value pair like, and get
the value with osl--prop. Otherwise (key label form) would need an extra
keyword or some other mean: (key label :form form)?

> In general, I'd prefer format similar to
> https://github.com/progfolio/doct/
> or at least similar to org-capture-templates
> The global ARGS in org-select could be defined using cl-defun style. See
> 2.1 Argument Lists section of CL manual.

Ok. I am not familiar with doct, but I did try to keep it similar to
org-capture-templates. Or at least it must start with key and label:

(key label ... )

rest after label is user data. This entry will be passed back to user
callback if user specify a callback. The special case is a simplistic
case where client code does not have any data but will just like to have
a lisp form executed when user chooses a key. That one is (key label
form). I think that will be the most often use-case. That would provide
for really simple way to create menus, so I would like to make the most
often use-case the most simple to use too. In more general case, as in
org-capture for example, I exect client code to pass in a handler and
parse its user data itself. If I relate to Max last mail about C structs
and void* to user data, that is pretty much that void* user
gets. Org-capture define its own "template language", some other project
could define their own, etc. The only thing this library cares about is
the first two elements so it can draw stuff in buffer and create
mode-map to execute commands.

Thank you for the input and sorry for sending in the entire program
instead of just patch, I tottally forgot it and remembered first long
after I sent the mail.

I'll definitely think about all the issues you bring up, and
hear from me when I have reworked it a bit more.

best regards
/a

Re: Proposal: 'executable' org-capture-templaes

[Ihor Radchenko]

Arthur Miller <arthur.miller@live.com> writes:

> Ihor Radchenko <yantar92@gmail.com> writes:
>
>> Arthur Miller <arthur.miller@live.com> writes:
>>
>>> I have reworked a bit org-capture, but I haven't yet worked on org-agenda
>>> restrictions and other details.
>>
>> I do not think that you need to prioritize re-creating
>> org-capture/org-agenda/etc The main point is making sure that org-select
>> provides the required functionality.
>
> That is just my way to test if it provides features needed, and how they
> would reflect on real use-case such as org-capture and org-agenda. It is
> just for demo purpose so to say.

Demo is fine.
However, since you are already asking about comments on the elisp parts,
docstrings would be helpful to understand the code. Without docstrings
and following the conventions, it is much harder to read the code.

Best,
Ihor

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Max Nikulin <manikulin@gmail.com> writes:

> On 22/06/2022 19:13, Arthur Miller wrote:
>> Max Nikulin writes:
>> Menu should and application should be separated in my eyes. Menu is just a
>> graphical/audio? presentation of selectable choice to the user. As such it
>> should display choices, let user pick a choice, and return to the application
>> the picked choice. Compare to completing-read, or org-mks. I don't think we
>> should mingle application states and menu states.
>
> By state I mean some structure opaque to menu package. It just receives it from
> caller as an optional argument and (when given) later passes it to
> handler. Maybe it is alien approach in LISP, but in C (where closures are
> impossible) it is a widely used technique. Functions having callback argument
> usually have another void* one that is later passed as an argument of the
> callback function in addition to other data.

I understand, I have done my share of C, and know what you mean. Say
posix thread will take a void* to user data, and pass it on. This is
exactly what this is about. It is just that we don't need an extra
structure to pass around. We have a menu entry, which *is* the user
data. Well at least most of it :). This structure looks like:

(key label user-data)

Where user-data is any elements in the list user wishes to pass on. I
don't even require the list to be a well-formed property list, I thought
it is left to the user. key = shortcut used to create mode-map and label
is the string displayed in the buffer. Later this string could be used
for compleating read/isearch or whatever, if it is desired to provide
such interface. But really those two elements are the required
ones and in the first place in the template. The rest is up to client
code.

Compare to org-capture-templates as illustration. We don need to keep it
compatible with org-capture, so it has to recognize two-elements form:
(key label) which has special meaning, and that translates to all other
clients. I could transform those, but I don't think it is worth work. I
will also see if I can rework it to use another two element form:

(:separator separator-string).

I disslike the tripple nesting of lists: a menu is a list of tables
which are each list of lists. I used this approach to implicitly tell
where separators are to be drawn, but after second thought, I think it
will be prettier and more intuitive for client code to explicitly note
where separator is. Maybe I could even mix vertical and horizontal
separators in same menu. I'll see if I have time and interest to
implement that one :).


> (org-buffer-menu
>  '(("a" "Option A" (1))
>    ("b" "Option B" (2)))
>  :handler
>  (lambda (entry state)
>   (message "value %S state %S" (nth 2 entry) state))
>  :state
>  '(1 2 3))
>
> Menu caller does not care whether some buffer is created to present menu. Menu
> has no idea what state may contain, it just passes it to the handler. When menu
> is implemented as a buffer with keymap then state is stored as a buffer-local
> variable.

Exactly, and this is exactly what happens when you pass in a
handler. The handler gets the entry, which can contain whatever client
data is. Again, compare to org-capture-template.

One can mentally see a menu entry as two lists:

(key label) and (user-data1 ... user-dataN).

They are just lumped into one list (key label user-data1 ... user-dataN).

There is no point of separating this into two lists, since we need
backward compatibility with org-capture-templates.

> As to your current implementation of org-select, I do not like that it is
> responsibility of caller to create different buffer names to enable multiple
> instances of menu. I would prefer to control single/multiple instances through a
> boolean argument of org-select.

How can we know what client application would be called? In which way
can we guess we are called from org-capture or org-agenda org some
user-app? Or I don't udnerstand what you mean here?

Reason I removed :transient as the flag to say the menu should dissapear
after the choice is made, is that it puts control into two places and
messes up the code. Since a hanlder can do anything anyway, it has to be
checked in both places. So I removed the transient flag, and lets just
the handler do it. In that case we know where the buffer handling
happens. I think it eases up usage and implementation. If you have some
cleaner approach sure, it is appreaciated. 

> Arthur, I see that I should response to more your comments. However I am unable
> to identify source of disagreement, likely it is some different

I didn't know we are in disagreement :-).

>  likely it is some different
> assumptions. 

Could be. Could be that we think about same thing, but we see it a bit
differently. People solve things differently, so would probably have
different solutions if we worked separately on this. I think it is normal.

> assumptions. That is why I decided to concentrate on a couple of questions in
> this message.

It is ok. I hope I made it more clear how I am thinking about this.

best regards
/a

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Ihor Radchenko <yantar92@gmail.com> writes:

> Arthur Miller <arthur.miller@live.com> writes:
>
>> Ihor Radchenko <yantar92@gmail.com> writes:
>>
>>> Arthur Miller <arthur.miller@live.com> writes:
>>>
>>>> I have reworked a bit org-capture, but I haven't yet worked on org-agenda
>>>> restrictions and other details.
>>>
>>> I do not think that you need to prioritize re-creating
>>> org-capture/org-agenda/etc The main point is making sure that org-select
>>> provides the required functionality.
>>
>> That is just my way to test if it provides features needed, and how they
>> would reflect on real use-case such as org-capture and org-agenda. It is
>> just for demo purpose so to say.
>
> Demo is fine.
> However, since you are already asking about comments on the elisp parts,
> docstrings would be helpful to understand the code. Without docstrings
> and following the conventions, it is much harder to read the code.
>
> Best,
> Ihor

Yepp, I understand. I'll think of it for the next patch.

best regards
/a

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 26/06/2022 11:50, Arthur Miller wrote:
> Max Nikulin writes:
>>
>> By state I mean some structure opaque to menu package. It just receives it from
>> caller as an optional argument and (when given) later passes it to
>> handler. Maybe it is alien approach in LISP, but in C (where closures are
>> impossible) it is a widely used technique. Functions having callback argument
>> usually have another void* one that is later passed as an argument of the
>> callback function in addition to other data.
> 
> I understand, I have done my share of C, and know what you mean. Say
> posix thread will take a void* to user data, and pass it on. This is
> exactly what this is about. It is just that we don't need an extra
> structure to pass around. We have a menu entry, which *is* the user
> data.

You a right, it is not strictly necessary that menu should be aware of 
state. I expect some helper though, e.g.

;;; -*- lexical-binding: t; -*-
(defun org-menu-multiinstance-stateful (menus &rest args)
   (let* ((buffer-name (or (plist-get args :buffer-name)
			  "*Select menu*"))
	 (state (plist-get args :state))
	 (handler (plist-get args :handler))
	 (handler-closure
	  (and handler
	       state
	       (lambda (entry &rest _)
		 (funcall handler entry state)))))
     (when state (org-plist-delete args :state))
     (when handler (org-plist-delete args :handler))
     (plist-put args
	       :buffer-name
	       (generate-new-buffer-name buffer-name))
     (apply #'org-select
	   (if handler-closure
	       (mapcar
		(lambda (menu)
		  (append menu (list :org-select-handler
				     handler-closure)))
		menus)
	     menus)
	   args)))
(provide 'org-multimenu)

To be able to call menu as

(load (expand-file-name "org-multimenu"))
(org-menu-multiinstance-stateful
  `((("1" "one" 1)
     ("2" "two" 2)))
  :state (format-time-string "%T")
  :text "Some heading"
  :buffer-name "*Test menu*"
  :handler (lambda (entry state)
	    (org-select-quit) ; it does not kill the buffer
	    (message "handler %S %S" entry state)))

I do not like how to closures are represented in elisp stack traces 
(e.g. JavaScript engines in browsers tries hard to give some debug names 
for anonymous functions). It should not be a problem when passed handler 
is a named function (not a lambda) despite closure in the adjacent frame.

However I would expect that menu handler is not aware if menu is 
implemented using buffers. From my point of view handler should not have 
buffer argument.

What I missed in your current implementation is ability to change text 
of menu entries in response to action. E.g. "C-c C-e" ox dispatcher has 
some options and user should see current values.

P.S. I am sorry for long delay.

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Max Nikulin <manikulin@gmail.com> writes:

> On 26/06/2022 11:50, Arthur Miller wrote:
>> Max Nikulin writes:
>>>
>>> By state I mean some structure opaque to menu package. It just receives it from
>>> caller as an optional argument and (when given) later passes it to
>>> handler. Maybe it is alien approach in LISP, but in C (where closures are
>>> impossible) it is a widely used technique. Functions having callback argument
>>> usually have another void* one that is later passed as an argument of the
>>> callback function in addition to other data.
>> I understand, I have done my share of C, and know what you mean. Say
>> posix thread will take a void* to user data, and pass it on. This is
>> exactly what this is about. It is just that we don't need an extra
>> structure to pass around. We have a menu entry, which *is* the user
>> data.
>
> You a right, it is not strictly necessary that menu should be aware of state. I
> expect some helper though, e.g.
>
> ;;; -*- lexical-binding: t; -*-
> (defun org-menu-multiinstance-stateful (menus &rest args)
>   (let* ((buffer-name (or (plist-get args :buffer-name)
> 			  "*Select menu*"))
> 	 (state (plist-get args :state))
> 	 (handler (plist-get args :handler))
> 	 (handler-closure
> 	  (and handler
> 	       state
> 	       (lambda (entry &rest _)
> 		 (funcall handler entry state)))))
>     (when state (org-plist-delete args :state))
>     (when handler (org-plist-delete args :handler))
>     (plist-put args
> 	       :buffer-name
> 	       (generate-new-buffer-name buffer-name))
>     (apply #'org-select
> 	   (if handler-closure
> 	       (mapcar
> 		(lambda (menu)
> 		  (append menu (list :org-select-handler
> 				     handler-closure)))
> 		menus)
> 	     menus)
> 	   args)))
> (provide 'org-multimenu)
>
> To be able to call menu as
>
> (load (expand-file-name "org-multimenu"))
> (org-menu-multiinstance-stateful
>  `((("1" "one" 1)
>     ("2" "two" 2)))
>  :state (format-time-string "%T")
>  :text "Some heading"
>  :buffer-name "*Test menu*"
>  :handler (lambda (entry state)
> 	    (org-select-quit) ; it does not kill the buffer
> 	    (message "handler %S %S" entry state)))

I might be missunderstanding you now, but from this example it seems to me that
you see menu entries as something that aims for the menu itself, while state is
some user data?

The "menu data" here is just: "1" and "one" in the first entry, and simillary
"2" and "two" in the second entry. After those, client code can put whatever it
desires: ("1" "one" ..... lots of client state .....). So for example client
could have something like ("1" "one" 1 (format-time-string "%T")).

However that might be tedious to type for each entry, so maybe we could pass an
optional "menu-global" state (or entry) that is passed to the user. So it would
be:

(lambda (entry &optional state)
 	    (org-select-quit) ; it does not kill the buffer
 	    (message "handler %S %S" entry state)))

Reason for optional: I prefer simplicity. I think the idea to lump together
menu selection with user state as it is done in org-capture-templates is really
good. Kudos to whomever came up with that one!

I like it because it puts everything into one place, we don't need to update
display and logic separately but everything is in an entry. I think it is good,
and I would like to keep that simplicity. I wouldnt like to suddenly separate
user state and menu state, because it would negate that fundamental idea.

> I do not like how to closures are represented in elisp stack traces
> (e.g. JavaScript engines in browsers tries hard to give some debug names for
> anonymous functions). It should not be a problem when passed handler is a named
> function (not a lambda) despite closure in the adjacent frame.
>
> However I would expect that menu handler is not aware if menu is implemented
> using buffers. From my point of view handler should not have buffer argument.

I understand, and I agree it is not very beautiful design :). The problem is when
client code provides its own handler. In the beginning I used a flag,
org-select-transient, to signal the menu should go away, but that wasn't very
clean either. The solution is maybe to never give total control to user handler,
but to always wrap/call user handler and finnish in menu's own handler. That way
we can use a flag to signal some state. I don't know, just thinking loud at the
moment; will have to test.

> What I missed in your current implementation is ability to change text of menu
> entries in response to action. E.g. "C-c C-e" ox dispatcher has some options and
> user should see current values.

The buffer text is just dead text; menu entries are a graphical display for the
user, not really interactive in the sense that it will update itself upon some
user action, unless user picks submenu or returns from one. 

>                                E.g. "C-c C-e" ox dispatcher has some options and
> user should see current values.

Can that be implemented as a submenu (group node as in org-capture-templates)?

> P.S. I am sorry for long delay.

Don't worry. It was midsummer celebration here and lots of sun so I was out on
the beach with the family, not much behind the computer last weekend and entire
week. We are expecting rain next week, so I will probably work on it more over
the weekend and early next week. I have rebuild it for now, I have removed
tripple nesting and incorporated some (most) of Ihors remarks, and removed some
ugly iteration for recursion but I am not completely done with everything I wish
to do yet.

best regards
/a

Re: Proposal: 'executable' org-capture-templates

[Max Nikulin]

On 01/07/2022 06:30, Arthur Miller wrote:
> Max Nikulin writes:
>>
>> (load (expand-file-name "org-multimenu"))
>> (org-menu-multiinstance-stateful
>>   `((("1" "one" 1)
>>      ("2" "two" 2)))
>>   :state (format-time-string "%T")
>>   :text "Some heading"
>>   :buffer-name "*Test menu*"
>>   :handler (lambda (entry state)
>> 	    (org-select-quit) ; it does not kill the buffer
>> 	    (message "handler %S %S" entry state)))
> 
> I might be missunderstanding you now, but from this example it seems to me that
> you see menu entries as something that aims for the menu itself, while state is
> some user data?

I am feeling myself confused now. It seems you have got the idea mostly 
right, but I suspect some resistance though.

For org-capture there are menu entries data in `org-capture-templates' 
that is the same set of rules (how to expand templates where to store 
result) for all instances of menu. I do not see any point to make each 
entry an executable closure.

State specific to each menu instance consists from the following parts:
- requested action (prefix argument: goto target location, insert at point)
- data required to expand template when it is selected, namely 
`org-overriding-default-time', `org-capture-initial' and maybe something 
else.

Template properties and capture data are quite orthogonal and for me it 
is rather natural to keep them independently. I would prefer to have 
support of state at low level, but it can be added on the top of basic 
functions through a closure. In a different use case, when menu is just 
a collection of independent actions, there is no point in the state 
shared by all entries of the menu instance.

>> However I would expect that menu handler is not aware if menu is implemented
>> using buffers. From my point of view handler should not have buffer argument.
> 
> I understand, and I agree it is not very beautiful design :). The problem is when
> client code provides its own handler. In the beginning I used a flag,
> org-select-transient, to signal the menu should go away, but that wasn't very
> clean either.

There are two types of actionable menu entries:
1. Ones that modify the state associated with menu instance without real 
action. Currently they are e.g. toggles like body only or complete 
document with headers for the export dispatcher. Menu should be retained 
when such action is called.
2. Entries that call real action. In the case of export dispatcher or 
`org-capture' menu buffer should be destroyed before a new one to 
display export result or expanded template is created or selected.

So generally it is not possible to determine in advance if menu should 
be "closed" before calling its handler. It is too late to close menu 
when the handler returns some value. I considered a predicate function 
that accepts menu entry data and the state and returns decision if menu 
should be retained. Looking into your implementation I realized that 
`org-select-quit' is almost ready to be a better solution. If the 
handler just changes the state then it does not call the quit function. 
Actions must invoke `org-select-quit' to notify menu implementation that 
the instance may (or must) be destroyed. The only problem is currently 
with multi-instance menu. Buffer is not destroyed by this function. 
Despite it is possible to kill the buffer since it is passed as an 
argument, I would prefer to delegate this action to `org-select-quit', 
so menu caller becomes completely unaware if menu implementation uses 
buffers or something else.

There are may be a convenience keyword argument that adds call of 
`org-select-quit' before invoking a handler. It should be handy for 
simple menu with no options that can be changed before calling an action.

> The buffer text is just dead text;

It is unfortunate, I hope, you will reconsider it.

>>                                 E.g. "C-c C-e" ox dispatcher has some options and
>> user should see current values.
> 
> Can that be implemented as a submenu (group node as in org-capture-templates)?

I suppose, it is necessary to provide some visual feedback, so it is 
obvious to user that some options is changed (e.g. "C-b" for "C-c C-e" 
export). It may be a kind of always visible status line, however in some 
cases it is better to display option value close to its menu entry.

In a minimal variant there may be a function to set status line and menu 
keeps a pair of markers, so text between them are erased and new status 
is inserted when the function is called.

In the beginning I was afraid that menu instance state may become a 
serious problem with implementation you proposed. Actually it appeared 
to be a matter of convenience. Entry-dependent behavior (if menu should 
be kept or closed) would benefit from some polishing. Presenting to the 
user changes in menu state in response to their actions should be 
addressed somehow.

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 21/06/2022 14:37, Arthur Miller wrote:
> Emacs as a whole is not designed to work in the way I
> percieve it has clean separation between contexts in each frame. Menu
> buffer is "global" for entire Emacs process, and there are other
> features of Emacs that does not work well in such scenarion. Some people
> prefer to keep an Emacs process per project/task for that reason.

A side note rather unrelated to menu for Org mode.

My impression is that Emacs process per task scenario is not supported. 
It is almost certainly requires different init files, but request for a 
command line option overriding init.el was refused:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=15539
setting user-emacs-directory at command line invocation
> So I think the conclusion to this long thread was that we don't want to
> add a specific switch for this, and instead people can say
> "XDG_CONFIG_HOME=/whatever emacs" when they want to change these paths.
> So I'm closing this bug report.

Unfortunately initialization in Emacs is rather tricky and
     emacs -q -l init.el
may behave differently.

On the other hand the latest variant of org-select is quite close to 
reasonable level of support of multiple instances of the same menu.

>>>> Currently several capture menu instances may be requested though
>>>> org-protocol (or calling org-capture from emacsclient). The behavior is
>>>> rather confusing. New menu may help to fix the issue, that is why I
>>>> raised the question about parallel captures.
>>> I am not sure which behavior you have in mind.
>>
>> try the following commands without selecting a template in an Emacs frame in
>> between
>>
>>      emacsclient 'org-protocol:/capture?url=url-A&title=title-A&body=body=A'
>>      emacsclient 'org-protocol:/capture?url=url-B&title=title-B&body=body=B'
>>
>>> What I was thinking as a conservative implementation that would not
>>> introduce any new features is replacing the old menu with the new one
>>> every time the same menu is called. So, every time the user calls menu
>>> (e.g. capture menu), only the last capture environment is preserved. The
>>> previous, potentially unfinished, capture menus will be destroyed.
>>
>> Causing loss of user data. Currently it is hard to start new capture before
>> selecting a template.
> 
> Current org-capture is one at a time because of how org-mks works. There
> is nothing that prevents org-capture to open enumerated buffers,
> org-caputre<1>, org-capture<2> etc. User has to manually serialize data
> anyway, via C-c C-c from withing capture buffer? So in principle it is
> still one capture buffer at a time that manipulates the file on the disk
> itself?

I would like to avoid confusion here. "*CAPTURE*" buffers are created 
after selection of template. Menu is gone away and content is already 
added to the target document, so it will be saved in response to C-x C-s 
or autosaved after some interval. There is no need of additional 
persistence at this stage.

And to be clear, example I provided is broken, it creates 2 template 
selection menus existing at the same time, but second request overwrites 
capture data. When one template is selected, menu disappears, but 
session is still blocked by waiting for a key specifying second 
template. It is really ugly. I expect that new menu implementation will 
allow to improve user experience.

I was writing about interval between invocation of `org-capture' and 
selection of some template. During this period capture data exist only 
as values of runtime variable. Currently it is acceptable because it is 
almost impossible to do anything else in Emacs till capture template is 
selected.

Non-blocking menu makes the issue more severe for some (likely small) 
fraction of users. Of course, it is responsibility of code calling menu 
and arranging menu item handler, not the code implementing menu.

The priority of this issue is certainly less than choosing proper menu 
API and implementing it. However it should not be forgotten at the 
moment when new menu implementation will be committed to Org.

Re: Proposal: 'executable' org-capture-templaes

[Max Nikulin]

On 19/06/2022 22:34, Arthur Miller wrote:
> Max Nikulin writes:
>> I believe, multiple capture menus should be possible in parallel even within
>> single frame since it may contain several windows and user experience should be
>> better than now. Keymap-based selection opens a road in this direction since
>> menu may be non-modal, but it requires a bit different design.
> 
> That sounds like a new feature. You are correct, keymaps do open in that
> direction. That would be possible to tuck on the routine that saves the
> temporary buffer (whatever is called with C-c C-c). When user press C-c C-c in a
> capture buffer, that is the only time when interaction with real a file on the
> disk happends, right?

I am going through older messages trying to find possible sources of 
different assumptions. I am at least partially repeating myself, but I 
consider this point as an important one.

When a capture template is selected, `org-capture-fill-template' adds 
capture text to the proper heading of the target document. Capture 
windows display narrowed down buffers of target documents. Disk 
operations happen on C-c C-s hit by user (it can be done for a capture 
buffer as well) or on autosave timer. Aborting capture by C-c C-k 
reverts the change by erasing earlier added text from the target 
document. It is safe to have multiple capture buffers, but some amount 
of work is required to make parallel menu buffers straightforward.

>> I mentioned completing-read because I consider it as a test of API quality. It
>> should be possible to plug alternative menu implementation and completing read
>> may be a simple enough variant. It is blocking, but in the case of capture "push
>> to capture queue" could be used to postpone the action.
> 
> I don't think it matters there that completing read is blocking. Users do
> completing read just to pick one action to get executed. I don't think I
> understand how qeue of captures to do would work, to be honest.

It is related to code calling menu rather than implementation of menu. 
It is a way to improve handling of parallel capture template selection 
menus even with `org-mks'. Certainly I expect that new menu library will 
not add any new obstacles. Assume a modal template selection is 
configured, e.g. based on completing read and such "menu" is currently 
active. The user may switch to another application and request new 
capture through "org-protocol:" URI. The best way to handle such case 
from my point of view is to detect that the menu is active, collect 
values necessary to expand template and put this capture state to some 
queue. As soon as the user selects template for the capture started 
earlier, next state is popped from the queue and next template selection 
prompt is activated.

>> P.S. Notice text properties for entries in the following modal menu:
>> Timothy to emacs-orgmode. [PATCH] New remote resource download policy. Sun, 12
>> Jun 2022 22:43:07 +0800. https://list.orgmode.org/87mteiq6ou.fsf@gmail.com
> 
> I am not sure what do you bring that cod up? I looked at the link, but I just
> see text styling with faces. Do  you mean to use text properties to communicate
> the state? Sure text properties can be used for different things, but already
> have "template" (a list) we are passing around. Isn't it sufficient as a state
> carrier. You can push/pop things off of it to communicate whatever. Forgive me
> if I don't understand what you meant there.

At that moment I considered it as a feature request to add text 
properties to keys displayed in the menu. Later I realized that it is 
not a problem even for `org-mks' since strings with keys may be 
propertized in advance.

It is unrelated to user state, but the patch adds another implementation 
of menu. I hope that it will be possible to avoid dedicated menu code 
for this particular query and rely on org-select your proposed.

Timothy wrote that a reason for custom menu was modal nature of 
`org-mks', so it would made impossible to inspect the document before 
decision if a remote URL should be allowed. org-select should relieve 
this limitation.

Re: Proposal: 'executable' org-capture-templaes

[Arthur Miller]

Max Nikulin <manikulin@gmail.com> writes:

> On 21/06/2022 14:37, Arthur Miller wrote:
>> Emacs as a whole is not designed to work in the way I
>> percieve it has clean separation between contexts in each frame. Menu
>> buffer is "global" for entire Emacs process, and there are other
>> features of Emacs that does not work well in such scenarion. Some people
>> prefer to keep an Emacs process per project/task for that reason.
>
> A side note rather unrelated to menu for Org mode.
>
> My impression is that Emacs process per task scenario is not supported.

I am not sure if we think of same thing, but "process per task" means simply how
people use Emacs. Some people open each file in a new Emacs process, some people
like to keep different tasks in different instances of Emacs, for example Gnus
in one Emacs, and editing work in another etc. It is not that Emacs has some
feature that would make one prohibit to start Gnus in other session, or to load
same set of files in different Emacs processes, but I don't think it is
necessary either.

> almost certainly requires different init files, but request for a command line

Again, I am not sure if we think of same things here, but I don't think it is
needed, but anyway: 

> option overriding init.el was refused:

do we need it? Aren't -q and -l option enough to load a different file after
startup? There is also -e and -f option that could be used for the purpose of
more customization. 

> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=15539
> setting user-emacs-directory at command line invocation
>> So I think the conclusion to this long thread was that we don't want to
>> add a specific switch for this, and instead people can say
>> "XDG_CONFIG_HOME=/whatever emacs" when they want to change these paths.
>> So I'm closing this bug report.
>
> Unfortunately initialization in Emacs is rather tricky and
>     emacs -q -l init.el
> may behave differently.
>
> On the other hand the latest variant of org-select is quite close to reasonable
> level of support of multiple instances of the same menu.
>
>>>>> Currently several capture menu instances may be requested though
>>>>> org-protocol (or calling org-capture from emacsclient). The behavior is
>>>>> rather confusing. New menu may help to fix the issue, that is why I
>>>>> raised the question about parallel captures.
>>>> I am not sure which behavior you have in mind.
>>>
>>> try the following commands without selecting a template in an Emacs frame in
>>> between
>>>
>>>      emacsclient 'org-protocol:/capture?url=url-A&title=title-A&body=body=A'
>>>      emacsclient 'org-protocol:/capture?url=url-B&title=title-B&body=body=B'
>>>
>>>> What I was thinking as a conservative implementation that would not
>>>> introduce any new features is replacing the old menu with the new one
>>>> every time the same menu is called. So, every time the user calls menu
>>>> (e.g. capture menu), only the last capture environment is preserved. The
>>>> previous, potentially unfinished, capture menus will be destroyed.
>>>
>>> Causing loss of user data. Currently it is hard to start new capture before
>>> selecting a template.
>> Current org-capture is one at a time because of how org-mks works. There
>> is nothing that prevents org-capture to open enumerated buffers,
>> org-caputre<1>, org-capture<2> etc. User has to manually serialize data
>> anyway, via C-c C-c from withing capture buffer? So in principle it is
>> still one capture buffer at a time that manipulates the file on the disk
>> itself?
>
> I would like to avoid confusion here. "*CAPTURE*" buffers are created after
> selection of template. Menu is gone away and content is already added to the
> target document, so it will be saved in response to C-x C-s or autosaved after
> some interval. There is no need of additional persistence at this stage.
>
> And to be clear, example I provided is broken, it creates 2 template selection
> menus existing at the same time, but second request overwrites capture
> data. When one template is selected, menu disappears, but session is still
> blocked by waiting for a key specifying second template. It is really ugly. I
> expect that new menu implementation will allow to improve user experience.
>
> I was writing about interval between invocation of `org-capture' and selection
> of some template. During this period capture data exist only as values of
> runtime variable. Currently it is acceptable because it is almost impossible to
> do anything else in Emacs till capture template is selected.
>
> Non-blocking menu makes the issue more severe for some (likely small) fraction
> of users. Of course, it is responsibility of code calling menu and arranging
> menu item handler, not the code implementing menu.
>
> The priority of this issue is certainly less than choosing proper menu API and
> implementing it. However it should not be forgotten at the moment when new menu
> implementation will be committed to Org.

I'll think more about what you write here and your next mail, so I'll answer at
later time, I am not sure I can give some constructive answer at the moment.

best regards
/a

Re: Proposal: 'executable' org-capture-templates

[Max Nikulin]

On 03/07/2022 22:12, Arthur Miller wrote:
> Max Nikulin writes:
>>
>> My impression is that Emacs process per task scenario is not supported.
> 
> I am not sure if we think of same thing, but "process per task" means simply how
> people use Emacs. Some people open each file in a new Emacs process, some people
> like to keep different tasks in different instances of Emacs, for example Gnus
> in one Emacs, and editing work in another etc. It is not that Emacs has some
> feature that would make one prohibit to start Gnus in other session, or to load
> same set of files in different Emacs processes, but I don't think it is
> necessary either.

I am aware of a workaround with a separate process for gnus since it is 
implemented as a synchronous single thread application, so network 
latency becomes a disaster. Emacs process per file is inconvenient when 
emacs server is desired (e.g. for org-protocol). True init directory is 
required for easy customization. There are may be subtle differences 
whether a file loaded as a part of initialization of as -l argument and, 
of course, bugs.

After all, it would be much more convenient to tune init.el file if it 
were possible to specify just config directory. It is disappointing that 
emacs offers rather strange options instead: --user and XDG_CONFIG_HOME. 
Even mount namespace might be more convenient than the latter one to 
override just .emacs.d without touching of over desktop settings.

So menu with multitasking support sounds promising to avoid multiple 
processes, but requires some care at the call sites.