post-command-hook to slow?

post-command-hook to slow?

[Thorsten Jolitz]

Hi List, 

I call a command to be executed in a temporary buffer, and that command
runs a post-command-hook. Here is some pseudocode:

,----------------------------------
| open-temp-buffer
| set-buffer-mode
| call-interactively cmd
| copy-content-and-exit-temp-buffer
`----------------------------------

Unfortunately the post-command-hook of cmd is executed only after the
temp buffer is already exited (i.e. in the original buffer in a
different/wrong major-mode with the needed markers lost).

Is that to be expected? Will `copy-content-and-exit-temp-buffer' be
executed before the post-command-hook of cmd has a chance to be
executed?

-- 
cheers,
Thorsten

Re: post-command-hook to slow?

[Pascal J. Bourguignon]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Hi List, 
>
> I call a command to be executed in a temporary buffer, and that command
> runs a post-command-hook. Here is some pseudocode:
>
> ,----------------------------------
> | open-temp-buffer
> | set-buffer-mode
> | call-interactively cmd
> | copy-content-and-exit-temp-buffer
> `----------------------------------
>
> Unfortunately the post-command-hook of cmd is executed only after the
> temp buffer is already exited (i.e. in the original buffer in a
> different/wrong major-mode with the needed markers lost).
>
> Is that to be expected? Will `copy-content-and-exit-temp-buffer' be
> executed before the post-command-hook of cmd has a chance to be
> executed?

post-command-hook is called after each command.  Ie. after each
character you type!  So indeed, it should be very fast.  Perhaps you
could keep your temp buffer around (you can hide such a buffer by
prefixing its name with a space).


From a quick scan, I don't see anything in call-interactively that would
make it not execute the command synchronously.
Furthermore, post-command-hooks are not called by call-interactively.

If cmd is this-command, then what you are doing, is to call
this-command a second time, after it has already been called.  This
call-interactively works with your temp buffer, but the original command
call was done before post-command-hooks were called of course.


If what you want to do is to wrap some commands to make them work in a
different buffer, then you cannot use post-command-hook to do that, and
just both pre-command-hook and post-command-hook wouldn't be very safe
(if anything breaks, you could end with a dangling state).  

In that case, I would advise you to use an around advice on the command
instead.  cf. defadvice.



Some commands may rely on the selected window in a hidden way, so if the
commands you call in the post-command-hook use a window, wrapping them in
a (with-selected-window window (call-interactively …)) could help.

Do you use with-temp-buffer?  You probably should, instead of creating
the buffer and later killing it.



-- 
__Pascal Bourguignon__
http://www.informatimago.com/
"Le mercure monte ?  C'est le moment d'acheter !"

Re: post-command-hook to slow?

[Thorsten Jolitz]

"Pascal J. Bourguignon" <pjb@informatimago.com> writes:

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>
>> Hi List, 
>>
>> I call a command to be executed in a temporary buffer, and that command
>> runs a post-command-hook. Here is some pseudocode:
>>
>> ,----------------------------------
>> | open-temp-buffer
>> | set-buffer-mode
>> | call-interactively cmd
>> | copy-content-and-exit-temp-buffer
>> `----------------------------------
>>
>> Unfortunately the post-command-hook of cmd is executed only after the
>> temp buffer is already exited (i.e. in the original buffer in a
>> different/wrong major-mode with the needed markers lost).
>>
>> Is that to be expected? Will `copy-content-and-exit-temp-buffer' be
>> executed before the post-command-hook of cmd has a chance to be
>> executed?
>
> post-command-hook is called after each command.  Ie. after each
> character you type!  So indeed, it should be very fast.  Perhaps you
> could keep your temp buffer around (you can hide such a buffer by
> prefixing its name with a space).

right, when tends to forget that ...

> From a quick scan, I don't see anything in call-interactively that would
> make it not execute the command synchronously.
> Furthermore, post-command-hooks are not called by call-interactively.
>
> If cmd is this-command, then what you are doing, is to call
> this-command a second time, after it has already been called.  This
> call-interactively works with your temp buffer, but the original command
> call was done before post-command-hooks were called of course.

hmmm ... may be a bit more context:

what I do is call `outorg-edit-as-org' in a programming language buffer,
e.g. an emacs-lisp buffer. Then a temporary edit buffer in Org-mode is
opened (not using with-temp-buffer since normally it should stay around
for editing) until it is closed again.

Now I reuse that mechanism for calling Org commands without any user
editing - open the Org-mode temp buffer, execute the command, close the
temp buffer and copy the results to the source buffer.

So the post-command-hooks are not mine, but those from some Org-mode
functions, I just have to deal with them. 

> If what you want to do is to wrap some commands to make them work in a
> different buffer, then you cannot use post-command-hook to do that, and
> just both pre-command-hook and post-command-hook wouldn't be very safe
> (if anything breaks, you could end with a dangling state).  
>
> In that case, I would advise you to use an around advice on the command
> instead.  cf. defadvice.

unfortunately its not my choice, since very basic Org commands like
org-todo run post-command-hook. 

> Some commands may rely on the selected window in a hidden way, so if the
> commands you call in the post-command-hook use a window, wrapping them in
> a (with-selected-window window (call-interactively …)) could help.

cmd and its hook function is Org-mode code I cannot change ... but
still, I'm going to experiment with this, looks promising.

> Do you use with-temp-buffer?  You probably should, instead of creating
> the buffer and later killing it. 

It would be better for this use case, but I don't want to reimplement
existing functionality, just reuse what is already there and works. 

I put some message-calls printing out `current-buffer' in `cmd' and its
 post-command-hook function, and its really strange - cmd is executed in
 the temp buffer (in Org-mode), the hook in the original emacs-lisp
 source-code buffer

  <emacs-lisp-buffer>

          |
 ,----------------------------------
 | open-temp-buffer 
 | set-buffer-mode
 | call-interactively cmd             }<org-mode-buffer>
 | copy-content-and-exit-temp-buffer
 `----------------------------------
          |

  <emacs-lisp-buffer>

although it shouldn't in my view, and everything you wrote seems to
indicate that 

 ,-----------------------
 | call-interactively cmd
 `-----------------------

should quickly run cmd and post-command-hook before 

 ,----------------------------------
 | copy-content-and-exit-temp-buffer
 `----------------------------------

is executed - but apparently not. 

Thanks for your answer.

-- 
cheers,
Thorsten

Re: post-command-hook to slow?

[Pascal J. Bourguignon]

Thorsten Jolitz <tjolitz@gmail.com> writes:
> […]
>
>  ,-----------------------
>  | call-interactively cmd
>  `-----------------------
>
> should quickly run cmd and post-command-hook before 
>
>  ,----------------------------------
>  | copy-content-and-exit-temp-buffer
>  `----------------------------------
>
> is executed - but apparently not. 


I'm still not sure to understand exactly what you're trying, but if what
you want is to have some function in post-command-hook to be called
after you call-interactively cmd, perhaps you could just do that
explicitely, since indeed, post-command-hooks won't be run while
processing post-command-hooks.


-- 
__Pascal Bourguignon__
http://www.informatimago.com/
"Le mercure monte ?  C'est le moment d'acheter !"

Re: post-command-hook to slow?

[Thorsten Jolitz]

"Pascal J. Bourguignon" <pjb@informatimago.com> writes:

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>> […]
>>
>>  ,-----------------------
>>  | call-interactively cmd
>>  `-----------------------
>>
>> should quickly run cmd and post-command-hook before 
>>
>>  ,----------------------------------
>>  | copy-content-and-exit-temp-buffer
>>  `----------------------------------
>>
>> is executed - but apparently not. 
>
>
> I'm still not sure to understand exactly what you're trying, but if what
> you want is to have some function in post-command-hook to be called
> after you call-interactively cmd, perhaps you could just do that
> explicitely, since indeed, post-command-hooks won't be run while
> processing post-command-hooks.

Its a bit hard to explain, in fact I don't want to run a
post-command-hook, I just want to call an Org function in a temporary
buffer - but that function runs a post-command hook, and I have to deal
with that (it is somehow run too late, when the temp buffer is already
closed).

An MWE is difficult unless you are an Org user, but anyway:

Evaluate this in an emacs-lisp-mode buffer (without the
surrounding #+begin_ and #+end_ delimiters):

#+begin_src emacs-lisp
(setq org-todo-keywords
      (quote
       ((sequence "TODO(t)" "NEXT(n)" "|" "DONE(d!/!)")
	(sequence
	 "WAITING(w@/!)" "HOLD(h@/!)" "|"
	 "CANCELLED(c@/!)" "PHONE"))))

(with-temp-buffer
  (org-mode)
  (insert "* Level 1\nSome Text\n")
  (org-todo)
  (message "%s" (buffer-substring-no-properties
     (point-min) (point-max))))
#+end_src

you should be prompted for a state, choose TODO first (t) and it should
 work.

Then evaluate 'with-temp-buffer again, this time chosing a state with an
 @ in its definition:

 "WAITING" or "HOLD" or "CANCELLED

because thats triggers taking a log note (and calling a
post-command-hook function).

You should get the error:

,-----------------------------------------------
| Error in post-command-hook (org-add-log-note):
| (error "Marker does not point anywhere")
`-----------------------------------------------

When you insert a message statement at the very beginning of
`org-add-log-note' that print current-buffer an major-mode, you will see
that its called in the emacs-lisp buffer and not in the temp buffer. 

-- 
cheers,
Thorsten

Re: post-command-hook to slow?

[Pascal J. Bourguignon]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> Its a bit hard to explain, in fact I don't want to run a
> post-command-hook, I just want to call an Org function in a temporary
> buffer - but that function runs a post-command hook, and I have to deal
> with that (it is somehow run too late, when the temp buffer is already
> closed).
>
> An MWE is difficult unless you are an Org user, but anyway:
>
> Evaluate this in an emacs-lisp-mode buffer (without the
> surrounding #+begin_ and #+end_ delimiters):
>
> #+begin_src emacs-lisp
> (setq org-todo-keywords
>       (quote
>        ((sequence "TODO(t)" "NEXT(n)" "|" "DONE(d!/!)")
> 	(sequence
> 	 "WAITING(w@/!)" "HOLD(h@/!)" "|"
> 	 "CANCELLED(c@/!)" "PHONE"))))
>
> (with-temp-buffer
>   (org-mode)
>   (insert "* Level 1\nSome Text\n")
>   (org-todo)
>   (message "%s" (buffer-substring-no-properties
>      (point-min) (point-max))))
> #+end_src
>
> you should be prompted for a state, choose TODO first (t) and it should
>  work.
>
> Then evaluate 'with-temp-buffer again, this time chosing a state with an
>  @ in its definition:
>
>  "WAITING" or "HOLD" or "CANCELLED
>
> because thats triggers taking a log note (and calling a
> post-command-hook function).
>
> You should get the error:
>
> ,-----------------------------------------------
> | Error in post-command-hook (org-add-log-note):
> | (error "Marker does not point anywhere")
> `-----------------------------------------------
>
> When you insert a message statement at the very beginning of
> `org-add-log-note' that print current-buffer an major-mode, you will see
> that its called in the emacs-lisp buffer and not in the temp buffer. 

Ok.   So you see how things go.  This is a general technique in emacs
applications, since we start from an editing event loop. 

org-todo opens a menu buffer with a list of choices.    In this case, it
reads the user choice modaly, with org-icompleting-read: you cannot do
anything else than select a menu item.

But then, in some cases, it needs to let the user edit a note.
To do that,
org-todo adds org-add-log-note to post-command-hook, and
org-add-log-note removes itself from it.  This chains the org-todo and
org-add-log-note commands in the main editing event loop.

org-add-log-note sets up a *Org Note* buffer, and similarly, returns, to
let the user edit it in the main editing event loop.  When the user
types C-c C-c, the org-ctrl-c-ctrl-c command is called which store the
note.  In the mean time a lot of commands can be given by the user.  He
may switch to other buffers and come back to the note a long time later.

Since the org application doesn't expect to do anything else once the
note is stored, org-store-log-note doesn't have any final hook.  We
would have to advice it, to add one.  On the other hand,
org-store-log-note is called from the org-finish-function hook, set up
by org-add-log-note, so we could set this hook to call
org-store-log-note, and our own function.


In any case, we have two architectures, either:

- we may use the same modeless model as the org application (and any
  other emacs application), therefore not using with-temp-buffer, but
  creating a temporary buffer, and by way of hooks, chain the activities
  we need.  or:

- we can still use with-temp-buffer, but we have to call
  (recursive-edit) to insert an editing event loop to let the user gain
  control until we (OR the user!) choose to exit or abort the recursive
  edit.

Here is how you would implement this later case:

(defun pjb-finish-store-log-note-and-exit-recursive-edit ()
  (org-store-log-note)
  (exit-recursive-edit))

(with-temp-buffer
  (org-mode)
  (insert "* Level 1\nSome Text\n")
  (org-todo)
  (message "BEFORE: %s" (buffer-substring-no-properties
                         (point-min) (point-max)))
  (when (marker-buffer org-log-note-marker)
    (org-add-log-note)
    (org-set-local 'org-finish-function 'pjb-finish-store-log-note-and-exit-recursive-edit)
    (recursive-edit))
  (message "AFTER: %s" (buffer-substring-no-properties
                        (point-min) (point-max))))


Notice that in this problem post-command-hook is only incidental, even
if the error message reporte mentionned it.  It's used to chain the
commands, but the problem is not related to commands and
post-command-hook at all.

-- 
__Pascal Bourguignon__
http://www.informatimago.com/
"Le mercure monte ?  C'est le moment d'acheter !"

Re: post-command-hook to slow?

[Stefan Monnier]

> that command runs a post-command-hook.

That doesn't make sense.  `post-command-hook' is run by the read-eval loop.

> ,----------------------------------
> | open-temp-buffer
> | set-buffer-mode
> | call-interactively cmd
> | copy-content-and-exit-temp-buffer
> `----------------------------------

call-interactively does not run post-command-hook.


        Stefan

Re: post-command-hook to slow?

[Thorsten Jolitz]

"Pascal J. Bourguignon" <pjb@informatimago.com> writes:

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>
>> Its a bit hard to explain, in fact I don't want to run a
>> post-command-hook, I just want to call an Org function in a temporary
>> buffer - but that function runs a post-command hook, and I have to deal
>> with that (it is somehow run too late, when the temp buffer is already
>> closed).
>>
>> An MWE is difficult unless you are an Org user, but anyway:
>>
>> Evaluate this in an emacs-lisp-mode buffer (without the
>> surrounding #+begin_ and #+end_ delimiters):
>>
>> #+begin_src emacs-lisp
>> (setq org-todo-keywords
>>       (quote
>>        ((sequence "TODO(t)" "NEXT(n)" "|" "DONE(d!/!)")
>> 	(sequence
>> 	 "WAITING(w@/!)" "HOLD(h@/!)" "|"
>> 	 "CANCELLED(c@/!)" "PHONE"))))
>>
>> (with-temp-buffer
>>   (org-mode)
>>   (insert "* Level 1\nSome Text\n")
>>   (org-todo)
>>   (message "%s" (buffer-substring-no-properties
>>      (point-min) (point-max))))
>> #+end_src
>>
>> you should be prompted for a state, choose TODO first (t) and it should
>>  work.
>>
>> Then evaluate 'with-temp-buffer again, this time chosing a state with an
>>  @ in its definition:
>>
>>  "WAITING" or "HOLD" or "CANCELLED
>>
>> because thats triggers taking a log note (and calling a
>> post-command-hook function).
>>
>> You should get the error:
>>
>> ,-----------------------------------------------
>> | Error in post-command-hook (org-add-log-note):
>> | (error "Marker does not point anywhere")
>> `-----------------------------------------------
>>
>> When you insert a message statement at the very beginning of
>> `org-add-log-note' that print current-buffer an major-mode, you will see
>> that its called in the emacs-lisp buffer and not in the temp buffer. 
>
> Ok.   So you see how things go.  This is a general technique in emacs
> applications, since we start from an editing event loop. 

Thank you very much for your detailled and very informative reply!

> org-todo opens a menu buffer with a list of choices.    In this case, it
> reads the user choice modaly, with org-icompleting-read: you cannot do
> anything else than select a menu item.
>
> But then, in some cases, it needs to let the user edit a note.
> To do that,
> org-todo adds org-add-log-note to post-command-hook, and
> org-add-log-note removes itself from it.  This chains the org-todo and
> org-add-log-note commands in the main editing event loop.
>
> org-add-log-note sets up a *Org Note* buffer, and similarly, returns, to
> let the user edit it in the main editing event loop.  When the user
> types C-c C-c, the org-ctrl-c-ctrl-c command is called which store the
> note.  In the mean time a lot of commands can be given by the user.  He
> may switch to other buffers and come back to the note a long time later.
>
> Since the org application doesn't expect to do anything else once the
> note is stored, org-store-log-note doesn't have any final hook.  We
> would have to advice it, to add one.  On the other hand,
> org-store-log-note is called from the org-finish-function hook, set up
> by org-add-log-note, so we could set this hook to call
> org-store-log-note, and our own function.

I had to write it down like this to get a clear picture:

,----------------------------------------------------------------------
| * Program Flow
|   
|   1. user calls (outshine-use-outorg 'org-todo)
| 
|      - calls outorg-edit-as-org, which sets up a *outorg-edit-buffer*
|        in org-mode and makes it current buffer
|      
|      - calls org-todo
|      
|        + adds org-add-log-note to post-command-hook
| 
|        + org-add-log-note is called SOMETIMES
| 
|          * removes itself from post-command-hook
| 
|          * sets up a *Org Note* buffer
| 
|          * sets up org-finish-function hook
| 
|          * returns control to main editing event loop
| 
|       - calls outorg-copy-edits-and-exit => TOO EARLY       
| 
|   [2. user eventually types C-c C-c to store note]
| 
|      [- org-finish-function hook calls org-store-log-note]
`----------------------------------------------------------------------


> In any case, we have two architectures, either:
>
> - we may use the same modeless model as the org application (and any
>   other emacs application), therefore not using with-temp-buffer, but
>   creating a temporary buffer, and by way of hooks, chain the activities
>   we need.  or:

This is the way to go for me, since this is the way outorg works. I
figured out how to do it more or less like this:

#+begin_src emacs-lisp
(defun outshine-use-outorg-finish-store-log-note ()
  "Finish store-log-note and exit recursive edit"
  (org-store-log-note)
  (outorg-copy-edits-and-exit))
#+end_src

[these are just relevant excerpts, not a complete working function]
#+begin_src emacs-lisp
      (outorg-edit-as-org)
      (call-interactively fun))
      (when (marker-buffer org-log-note-marker)
      	(org-add-log-note)
	(org-set-local
	 'org-finish-function
	 'outshine-use-outorg-finish-store-log-note))
#+end_src

This works fine in cases when org-add-log-note is actually called, but
for state changes to e.g. TODO when no log is taken, the finish
function and thus outorg-copy-edits-and-exit is never called, thus the
*outorg-edit-buffer* stays around.

To solve the problem completely I would need need something like

#+begin_src emacs-lisp
      (if (not (marker-buffer org-log-note-marker))
	  (outorg-copy-edits-and-exit)	  
      	(org-add-log-note)
	(org-set-local
	 'org-finish-function
	 'outshine-use-outorg-finish-store-log-note)))))
#+end_src

but this does not work ...

> - we can still use with-temp-buffer, but we have to call
>   (recursive-edit) to insert an editing event loop to let the user gain
>   control until we (OR the user!) choose to exit or abort the recursive
>   edit.
>
> Here is how you would implement this later case:
>
> (defun pjb-finish-store-log-note-and-exit-recursive-edit ()
>   (org-store-log-note)
>   (exit-recursive-edit))
>
> (with-temp-buffer
>   (org-mode)
>   (insert "* Level 1\nSome Text\n")
>   (org-todo)
>   (message "BEFORE: %s" (buffer-substring-no-properties
>                          (point-min) (point-max)))
>   (when (marker-buffer org-log-note-marker)
>     (org-add-log-note)
>     (org-set-local 'org-finish-function 'pjb-finish-store-log-note-and-exit-recursive-edit)
>     (recursive-edit))
>   (message "AFTER: %s" (buffer-substring-no-properties
>                         (point-min) (point-max))))

This works like a charm, and might even be better, but I want to reuse
existing code. 

> Notice that in this problem post-command-hook is only incidental, even
> if the error message reporte mentionned it.  It's used to chain the
> commands, but the problem is not related to commands and
> post-command-hook at all.

Thanks again, not only for helping solve the problem, but for giving
valuable insight into Emacs patterns.

-- 
cheers,
Thorsten

Re: post-command-hook to slow?

[Pascal J. Bourguignon]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> This is the way to go for me, since this is the way outorg works. I
> figured out how to do it more or less like this:
>
> #+begin_src emacs-lisp
> (defun outshine-use-outorg-finish-store-log-note ()
>   "Finish store-log-note and exit recursive edit"
>   (org-store-log-note)
>   (outorg-copy-edits-and-exit))
> #+end_src
>
> [these are just relevant excerpts, not a complete working function]
> #+begin_src emacs-lisp
>       (outorg-edit-as-org)
>       (call-interactively fun))
>       (when (marker-buffer org-log-note-marker)
>       	(org-add-log-note)
> 	(org-set-local
> 	 'org-finish-function
> 	 'outshine-use-outorg-finish-store-log-note))
> #+end_src
>
> This works fine in cases when org-add-log-note is actually called, but
> for state changes to e.g. TODO when no log is taken, the finish
> function and thus outorg-copy-edits-and-exit is never called, thus the
> *outorg-edit-buffer* stays around.
>
> To solve the problem completely I would need need something like
>
> #+begin_src emacs-lisp
>       (if (not (marker-buffer org-log-note-marker))
> 	  (outorg-copy-edits-and-exit)	  
>       	(org-add-log-note)
> 	(org-set-local
> 	 'org-finish-function
> 	 'outshine-use-outorg-finish-store-log-note)))))
> #+end_src
>
> but this does not work ...


It should work, unless outorg-copy-edits-and-exit does or expects
something.  You should see what it does.
-- 
__Pascal Bourguignon__
http://www.informatimago.com/
"Le mercure monte ?  C'est le moment d'acheter !"