Avoiding 'with-temp-buffer' in tight loops

Avoiding 'with-temp-buffer' in tight loops

[martin rudalics]

Ever since 'substitute-command-keys' has been transferred to help.el,
putting initial breakpoints in Fkill_buffer has become virtually
impossible.

Could we pretty please replace the

     (let ((keymap overriding-local-map)
           (inhibit-modification-hooks t)
           (orig-buf (current-buffer)))
       (with-temp-buffer

with something like

     (let* ((keymap overriding-local-map)
            (inhibit-modification-hooks t)
            (orig-buf (current-buffer))
	   (buffer-name " *substitute-command-keys*")
	   (buffer (or (get-buffer buffer-name)
		       (generate-new-buffer buffer-name t))))
       (with-current-buffer buffer
	(erase-buffer)

This would eliminate the overhead associated with continuously creating
and killing a buffer and make debugging amenable.  Maybe we could also
add a note somewhere in 'with-temp-buffer' so people avoid calling it in
tight loops.  Then we could also do something similar for
'image-type-from-file-header', possibly erasing the contents of its
buffer after being done with it.

Thanks, martin

Re: Avoiding 'with-temp-buffer' in tight loops

[Eli Zaretskii]

> Date: Fri, 9 Aug 2024 10:30:42 +0200
> From: martin rudalics <rudalics@gmx.at>
> 
> Ever since 'substitute-command-keys' has been transferred to help.el,
> putting initial breakpoints in Fkill_buffer has become virtually
> impossible.

It should be possible to make the breakpoint conditional on the first
character of a temporary buffer not being SPC.

> Could we pretty please replace the
> 
>      (let ((keymap overriding-local-map)
>            (inhibit-modification-hooks t)
>            (orig-buf (current-buffer)))
>        (with-temp-buffer
> 
> with something like
> 
>      (let* ((keymap overriding-local-map)
>             (inhibit-modification-hooks t)
>             (orig-buf (current-buffer))
> 	   (buffer-name " *substitute-command-keys*")
> 	   (buffer (or (get-buffer buffer-name)
> 		       (generate-new-buffer buffer-name t))))
>        (with-current-buffer buffer
> 	(erase-buffer)

This needs to be amended to support several threads calling
substitute-command-keys, I think.

An alternative for the above is to have substitute-command-keys bind
some variable non-nil, and have a breakpoint be conditional on that
variable's value.  If this works, I think it's simpler and cleaner.

Re: Avoiding 'with-temp-buffer' in tight loops

[martin rudalics]

 >> Ever since 'substitute-command-keys' has been transferred to help.el,
 >> putting initial breakpoints in Fkill_buffer has become virtually
 >> impossible.
 >
 > It should be possible to make the breakpoint conditional on the first
 > character of a temporary buffer not being SPC.

This fails when investigating a general misbehavior of 'kill-buffer'
that might also happen when the buffer name starts with a space.  And it
won't catch the case where such a buffer is killed inadvertently as
"current buffer".

 > This needs to be amended to support several threads calling
 > substitute-command-keys, I think.

Protecting access to any such buffer with the help of a mutex should be
trivial.  But IIUC we don't do such a thing anywhere and it's certainly
not done with Vprin1_to_string_buffer which was used earlier for this
purpose.  So my faith in the correct resolution of locks is limited.

 > An alternative for the above is to have substitute-command-keys bind
 > some variable non-nil, and have a breakpoint be conditional on that
 > variable's value.  If this works, I think it's simpler and cleaner.

This means that every tight loop using 'with-temp-buffer' would have to
set up and clear that variable.  It would fail to catch all cases where
killing a buffer associated with that variable is the cause of the
problem to fix.

martin

Re: Avoiding 'with-temp-buffer' in tight loops

[Eli Zaretskii]

> Date: Sat, 10 Aug 2024 15:59:29 +0200
> Cc: emacs-devel@gnu.org
> From: martin rudalics <rudalics@gmx.at>
> 
>  >> Ever since 'substitute-command-keys' has been transferred to help.el,
>  >> putting initial breakpoints in Fkill_buffer has become virtually
>  >> impossible.
>  >
>  > It should be possible to make the breakpoint conditional on the first
>  > character of a temporary buffer not being SPC.
> 
> This fails when investigating a general misbehavior of 'kill-buffer'
> that might also happen when the buffer name starts with a space.  And it
> won't catch the case where such a buffer is killed inadvertently as
> "current buffer".
> 
>  > This needs to be amended to support several threads calling
>  > substitute-command-keys, I think.
> 
> Protecting access to any such buffer with the help of a mutex should be
> trivial.  But IIUC we don't do such a thing anywhere and it's certainly
> not done with Vprin1_to_string_buffer which was used earlier for this
> purpose.  So my faith in the correct resolution of locks is limited.
> 
>  > An alternative for the above is to have substitute-command-keys bind
>  > some variable non-nil, and have a breakpoint be conditional on that
>  > variable's value.  If this works, I think it's simpler and cleaner.
> 
> This means that every tight loop using 'with-temp-buffer' would have to
> set up and clear that variable.  It would fail to catch all cases where
> killing a buffer associated with that variable is the cause of the
> problem to fix.

With respect, if you reject every solution, don't expect this to be
solved any time soon.

Re: Avoiding 'with-temp-buffer' in tight loops

[martin rudalics]

 > With respect, if you reject every solution,

I did not reject any solution.  I'm using conditional breakpoints when
debugging 'kill-buffer' all the time, in particular to trace only those
cases where the buffer to be killed is shown in some window.

But here I'm investigating a problem where Emacs hangs and kill -TSTP
doesn't get me anything useful.  I suspect that 'kill-buffer' might be
involved but have no real clue yet.  So the buffer may be any buffer
including those created by 'with-temp-buffer' and not tracing those
buffers with the debugger would be of no help.

 > don't expect this to be
 > solved any time soon.

Don't worry.  I fixed this locally so it won't bother me any more here.

Apologies for wasting your time.  It won't happen again.

martin

Re: Avoiding 'with-temp-buffer' in tight loops

[Eli Zaretskii]

> Date: Mon, 12 Aug 2024 09:51:36 +0200
> Cc: emacs-devel@gnu.org
> From: martin rudalics <rudalics@gmx.at>
> 
>  > With respect, if you reject every solution,
> 
> I did not reject any solution.  I'm using conditional breakpoints when
> debugging 'kill-buffer' all the time, in particular to trace only those
> cases where the buffer to be killed is shown in some window.
> 
> But here I'm investigating a problem where Emacs hangs and kill -TSTP
> doesn't get me anything useful.  I suspect that 'kill-buffer' might be
> involved but have no real clue yet.  So the buffer may be any buffer
> including those created by 'with-temp-buffer' and not tracing those
> buffers with the debugger would be of no help.
> 
>  > don't expect this to be
>  > solved any time soon.
> 
> Don't worry.  I fixed this locally so it won't bother me any more here.
> 
> Apologies for wasting your time.  It won't happen again.

You didn't waste my time, I was trying to help you solve your problem,
given what I understood from your descriptions.  But if no proposal
satisfies your needs, I guess there's no solution, at least not one
that you haven't tried already.