How do I debug errors in post-command-hook?

How do I debug errors in post-command-hook?

[Tassilo Horn]

Hi all,

how do I debug errors that happen while executing a function in
`post-command-hook'?  The usual debugging facilities like
`toggle-debug-on-error' and `edebug-function' are not triggered in
there...

Background: There seem to be some packages that error when emacs is
started with the --daemon option.  One example is the lusty-explorer's
`lusty-buffer-explorer' function.  It works fine when run in a "normal"
emacs, but I get an error in `post-command-hook' when emacs was
initially started as daemon, and I'd like to know what's going wrong.

Bye,
Tassilo

Re: How do I debug errors in post-command-hook?

[Lennart Borgman]

On Thu, Jul 29, 2010 at 10:30 PM, Tassilo Horn <tassilo@member.fsf.org> wrote:
> Hi all,
>
> how do I debug errors that happen while executing a function in
> `post-command-hook'?  The usual debugging facilities like
> `toggle-debug-on-error' and `edebug-function' are not triggered in
> there...
>
> Background: There seem to be some packages that error when emacs is
> started with the --daemon option.  One example is the lusty-explorer's
> `lusty-buffer-explorer' function.  It works fine when run in a "normal"
> emacs, but I get an error in `post-command-hook' when emacs was
> initially started as daemon, and I'd like to know what's going wrong.

I usually split up functions in post-command-hook like this:

(defun post-fun ()
   (condition-case err
         (post-fun-1)
     (error (message "post-fun: %s" (error-message-string err)))))

(defun post-fun-1 ()
   ...)

When I get an error I do

   M-: (post-fun-1)

Re: How do I debug errors in post-command-hook?

[Johan Bockgård]

Tassilo Horn <tassilo@member.fsf.org> writes:

> Hi all,
>
> how do I debug errors that happen while executing a function in
> `post-command-hook'?  The usual debugging facilities like
> `toggle-debug-on-error' and `edebug-function' are not triggered in
> there...
>
> Background: There seem to be some packages that error when emacs is
> started with the --daemon option.  One example is the lusty-explorer's
> `lusty-buffer-explorer' function.  It works fine when run in a "normal"
> emacs, but I get an error in `post-command-hook' when emacs was
> initially started as daemon, and I'd like to know what's going wrong.

;; I spent a few hours hacking together a solution using
;; signal-hook-function and had written a reply, and just before sending
;; the message I noticed that this actually works:
(defadvice lusty--post-command-function (around intercept activate)
  (condition-case err
      ad-do-it
    ;; Let the debugger run
    ((debug error) (signal (car err) (cdr err)))))

Re: How do I debug errors in post-command-hook?

[Tassilo Horn]

On Friday 30 July 2010 02:11:38 Johan Bockgård wrote:

Hi Johan,

> (defadvice lusty--post-command-function (around intercept activate)
>   (condition-case err
>       ad-do-it
>     ;; Let the debugger run
>     ((debug error) (signal (car err) (cdr err)))))

That didn't enter the debugger.  I guess you missed the error as
condition handler.  So I use this advice now:

--8<---------------cut here---------------start------------->8---
(defadvice lusty--post-command-function (around intercept activate)
  (condition-case err
      ad-do-it
    ;; Let the debugger run
    (error (progn (message "%s" err) (debug err)))))
--8<---------------cut here---------------end--------------->8---

And here's the backtrace.

--8<---------------cut here---------------start------------->8---
Debugger entered: ((error "Attempt to modify read-only object"))
  (progn (message "%s" err) (debug err))
  (condition-case err (setq ad-return-value (ad-Orig-lusty--post-command-function)) (error (progn ... ...)))
  (let (ad-return-value) (condition-case err (setq ad-return-value ...) (error ...)) ad-return-value)
  lusty--post-command-function()
  run-hooks(post-command-hook)
  ad-Orig-read-buffer(">> " nil nil)
  (setq ad-return-value (ad-Orig-read-buffer prompt def require-match))
  (let ((completion-styles ...)) (setq ad-return-value (ad-Orig-read-buffer prompt def require-match)))
  (let (ad-return-value) (let (...) (setq ad-return-value ...)) ad-return-value)
  read-buffer(">> ")
  apply(read-buffer ">> " nil)
  (save-window-excursion (apply read-fn lusty-prompt args))
  (unwind-protect (save-window-excursion (apply read-fn lusty-prompt args)) (remove-hook (quote post-command-hook) (quote lusty--post-command-function)) (setq lusty--previous-minibuffer-contents nil lusty--initial-window-config nil lusty--current-idle-timer nil))
  (let ((lusty--highlighted-coords ...) (lusty--matches-matrix ...) (lusty--matrix-column-widths ...) (lusty--matrix-truncated-p nil)) (add-hook (quote post-command-hook) (quote lusty--post-command-function) t) (unwind-protect (save-window-excursion ...) (remove-hook ... ...) (setq lusty--previous-minibuffer-contents nil lusty--initial-window-config nil lusty--current-idle-timer nil)))
  lusty--run(read-buffer)
  (let* ((lusty--active-mode :buffer-explorer) (minibuffer-local-completion-map lusty-mode-map) (buffer ...)) (when buffer (switch-to-buffer buffer)))
  lusty-buffer-explorer()
  call-interactively(lusty-buffer-explorer nil nil)
--8<---------------cut here---------------end--------------->8---

Well, that doesn't help me, right?  The error message was printed
anyway, and the backtrace is the backtrace after leaving
`lusty--post-command-function'.

Bye,
Tassilo

Re: How do I debug errors in post-command-hook?

[Johan Bockgård]

Tassilo Horn <tassilo@member.fsf.org> writes:

> On Friday 30 July 2010 02:11:38 Johan Bockgård wrote:
>
> Hi Johan,
>
>> (defadvice lusty--post-command-function (around intercept activate)
>>   (condition-case err
>>       ad-do-it
>>     ;; Let the debugger run
>>     ((debug error) (signal (car err) (cdr err)))))
>
> That didn't enter the debugger.

You need to turn on debug-on-error.

Buffer names are sometimes read-only objects in daemonized emacs (was: How do I debug errors in post-command-hook?)

[Tassilo Horn]

On Friday 30 July 2010 13:58:59 Johan Bockgård wrote:

Hi Johan,

> >> (defadvice lusty--post-command-function (around intercept activate)
> >>   (condition-case err
> >>       ad-do-it
> >>     ;; Let the debugger run
> >>     ((debug error) (signal (car err) (cdr err)))))
> >
> > That didn't enter the debugger.
> 
> You need to turn on debug-on-error.

Oh, indeed.  Then it did work, and here's the backtrace.

--8<---------------cut here---------------start------------->8---
Debugger entered--Lisp error: (error "Attempt to modify read-only object")
  put-text-property(0 11 face font-lock-string-face "*GNU Emacs*")
  byte-code("\b\306\x19\211\x1a\203r\0\n@\x11\v\f	\211\x1dGS\x1e\x0eH\307=\2035\0\310\311\x0e\312\x0e\x0e%\210\310\x0e\211T\312\x0e\x0f%\210\202@\0\310\311\x0eT\312\x0e\x10%\210))I\210\fT\211\x14\x0e\x11Y\203k\0\x0e\x12T\211\x16\x12\x0e\x13Y\203c\0\313\314\306\"\210\202i\0\x0e\x14\x0e\x12H\x13\311\x14\nA\211\x12\204\b\0*\306\207" [items item --cl-dolist-temp-- col-vec y path nil 47 put-text-property 0 face throw --cl-block-nil-- last lusty-directory-face lusty-slash-face lusty-file-face optimal-n-rows x n-columns matrix] 9)
  lusty--compute-layout-matrix(("*GNU Emacs*" "*Messages*" "diary" "*scratch*"))
  lusty-refresh-matches-buffer()
  ad-Orig-lusty--post-command-function()
  (setq ad-return-value (ad-Orig-lusty--post-command-function))
  (condition-case err (setq ad-return-value (ad-Orig-lusty--post-command-function)) ((debug error) (signal ... ...)))
  (let (ad-return-value) (condition-case err (setq ad-return-value ...) (... ...)) ad-return-value)
  lusty--post-command-function()
  run-hooks(post-command-hook)
  ad-Orig-read-buffer(">> " nil nil)
  (setq ad-return-value (ad-Orig-read-buffer prompt def require-match))
  (let ((completion-styles ...)) (setq ad-return-value (ad-Orig-read-buffer prompt def require-match)))
  (let (ad-return-value) (let (...) (setq ad-return-value ...)) ad-return-value)
  read-buffer(">> ")
  apply(read-buffer ">> " nil)
  lusty--run(read-buffer)
  lusty-buffer-explorer()
  call-interactively(lusty-buffer-explorer nil nil)
--8<---------------cut here---------------end--------------->8---

Looking at the lusty-explorer code, `lusty--compute-layout-matrix' calls
a defsubst `lusty-propertize-path' which eventually propertizes the
string given as parameter `path', like indicated by the top frame of the
backtrace:

    put-text-property(0 11 face font-lock-string-face "*GNU Emacs*")

But why does that throw an error when emacs is started as a daemon?

And, even in a daemonized emacs, I can eval

   (lusty-propertize-path "*GNU Emacs*")

without any problem.  So it looks like the string "*GNU Emacs*" passed
to that defsubst is somehow read-only.  Grepping the emacs sources the
error seems to be triggered by having that string in pure storage.  But
why?

The string is retrieved by collecting buffers using `walk-windows'
followed by `window-buffer' in combination with `buffer-list'.  Then for
all collected buffers, `buffer-name' is called, and those strings are
then passed to the `lusty-propertize-path' defsubst.

But even this does work in a daemonized emacs, even when I put it at the
front of my .emacs and then start it with emacs --daemon:

(dolist (b (buffer-list))
  (let ((n (buffer-name (get-buffer b))))
    (put-text-property 0 (length n) 'face font-lock-string-face n)))

Thereby I realized that after evaling that, (buffer-name <some-buf>)
will return the propertized string, so it seems that `buffer-name'
doesn't return a string copy but a place in the CL sense.  I'm not sure
if that's correct?  (The same is true for a "normal" emacs.)

Can anyone enlighten me what's going on?

Bye,
Tassilo

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Johan Bockgård]

Tassilo Horn <tassilo@member.fsf.org> writes:

> And, even in a daemonized emacs, I can eval
>
>    (lusty-propertize-path "*GNU Emacs*")

This is obviously a fresh string.

> So it looks like the string "*GNU Emacs*" passed to that defsubst is
> somehow read-only. Grepping the emacs sources the error seems to be
> triggered by having that string in pure storage.

Yes.

> But even this does work in a daemonized emacs, even when I put it at the
> front of my .emacs and then start it with emacs --daemon:
>
> (dolist (b (buffer-list))
>   (let ((n (buffer-name (get-buffer b))))
>     (put-text-property 0 (length n) 'face font-lock-string-face n)))

But this list does not contain "*GNU Emacs*", does it? ("*GNU Emacs*" is
the splash screen buffer. It is created after loading the init files.)

The daemon creates the splash screen via `normal-splash-screen' (instead
of `fancy-startup-screen'). It contains a call to `(rename-buffer "*GNU
Emacs*" t)'. Unlike `get-buffer-create', `rename-buffer' does not make a
copy of the provided string. This is how the pure string ends up in
`buffer-name', AFAICT.

(Modifying a string returned by `buffer-name' is still evil, as it does
not promise to return a fresh string.)

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Tassilo Horn]

On Friday 30 July 2010 19:46:28 Johan Bockgård wrote:

> > But even this does work in a daemonized emacs, even when I put it at
> > the front of my .emacs and then start it with emacs --daemon:
> >
> > (dolist (b (buffer-list))
> >   (let ((n (buffer-name (get-buffer b))))
> >     (put-text-property 0 (length n) 'face font-lock-string-face n)))
> 
> But this list does not contain "*GNU Emacs*", does it?

Nope.

> ("*GNU Emacs*" is the splash screen buffer. It is created after
> loading the init files.)
> 
> The daemon creates the splash screen via `normal-splash-screen'
> (instead of `fancy-startup-screen'). It contains a call to
> `(rename-buffer "*GNU Emacs*" t)'. Unlike `get-buffer-create',
> `rename-buffer' does not make a copy of the provided string. This is
> how the pure string ends up in `buffer-name', AFAICT.

But why is the string "*GNU Emacs*" put in pure space?  Because
everything that gets created when loading startup.el will be in pure
space?

> (Modifying a string returned by `buffer-name' is still evil, as it
> does not promise to return a fresh string.)

IMO a function shouldn't need to allow modification of it's return value
explicitly, but instead should warn if that's possible.  But indeed,
evaluating

  (aset (buffer-name) 0 ?g) 

makes my kmailCh3711.tmp buffer a gmailCh3711.tmp buffer.  To be honest,
this is really awkward.  Nobody would guess that modifying a string
returned by some function could have any side effects.

Bye,
Tassilo

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Wojciech Meyer]

Tassilo Horn <tassilo@member.fsf.org>  writes:

> IMO a function shouldn't need to allow modification of it's return value
> explicitly, but instead should warn if that's possible.  But indeed,
> evaluating
>
>   (aset (buffer-name) 0 ?g)

How would you do this? I see that the check can be performed on only at
runtime. Bytecompiler will not warn, because the value might be
propagated, and Lisp is dynamicaly typed. (in simple cases like this it
is apparent, but consider just returning this value from function).

Wojciech

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Tassilo Horn]

On Friday 30 July 2010 21:54:15 Wojciech Meyer wrote:
> Tassilo Horn <tassilo@member.fsf.org>  writes:
> 
> > IMO a function shouldn't need to allow modification of it's return
> > value explicitly, but instead should warn if that's possible.  But
> > indeed, evaluating
> >
> >   (aset (buffer-name) 0 ?g)
> 
> How would you do this? I see that the check can be performed on only
> at runtime. Bytecompiler will not warn, because the value might be
> propagated, and Lisp is dynamicaly typed. (in simple cases like this
> it is apparent, but consider just returning this value from function).

No no, you got me wrong.  I just meant the docstring of `buffer-name'
(and any other function that returns some thing whose modification would
have undesired, non-obvious side-effects) should explicitly state that
modifying that returned string will indeed modify the buffer name.

Johan argued the other way round, that it's clear that you should not do
that because the function doesn't explicitly state that the returned
string is a fresh copy.

Bye,
Tassilo

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Wojciech Meyer]

Tassilo Horn <tassilo@member.fsf.org> writes:

>> How would you do this? I see that the check can be performed on only
>> at runtime. Bytecompiler will not warn, because the value might be
>> propagated, and Lisp is dynamicaly typed. (in simple cases like this
>> it is apparent, but consider just returning this value from function).
>
> No no, you got me wrong.  I just meant the docstring of `buffer-name'
> (and any other function that returns some thing whose modification would
> have undesired, non-obvious side-effects) should explicitly state that
> modifying that returned string will indeed modify the buffer name.

Sry. I now got it.
The thing is that you should never modify inplace other string refs then
returned from make-string, the same applies to string literals.  e.g
text surounded by double quotes. (this could land easily in some read
only area). You could assume that all the string refs, are passed by
ref, and the string (mainly for performance reasons) will the same
object as in the buffer-list. So the advise is, modify inplace only if
you are sure it will not clobber the data in the other corner of Emacs
Also, Elisp has highly stateful nature, and it is hard to avoid such
quirks.

>
> Johan argued the other way round, that it's clear that you should not do
> that because the function doesn't explicitly state that the returned
> string is a fresh copy.

Yes, that's indeed better idea, as it will happen less often.

>
> Bye,
> Tassilo

Wojciech

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Stefan Monnier]

> No no, you got me wrong.  I just meant the docstring of `buffer-name'
> (and any other function that returns some thing whose modification would
> have undesired, non-obvious side-effects) should explicitly state that
> modifying that returned string will indeed modify the buffer name.

Rather, `aset' should warn that modifying strings is evil and dangerous.


        Stefan "whose aset refuses to do it"

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Lennart Borgman]

On Fri, Jul 30, 2010 at 11:48 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>> No no, you got me wrong.  I just meant the docstring of `buffer-name'
>> (and any other function that returns some thing whose modification would
>> have undesired, non-obvious side-effects) should explicitly state that
>> modifying that returned string will indeed modify the buffer name.
>
> Rather, `aset' should warn that modifying strings is evil and dangerous.
>
>
>        Stefan "whose aset refuses to do it"


When will this aset be grown up enough to play outside of home?

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Johan Bockgård]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> No no, you got me wrong.  I just meant the docstring of `buffer-name'
>> (and any other function that returns some thing whose modification would
>> have undesired, non-obvious side-effects) should explicitly state that
>> modifying that returned string will indeed modify the buffer name.
>
> Rather, `aset' should warn that modifying strings is evil and dangerous.

Should `put-text-property' do that too?

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Stefan Monnier]

>>> No no, you got me wrong.  I just meant the docstring of `buffer-name'
>>> (and any other function that returns some thing whose modification would
>>> have undesired, non-obvious side-effects) should explicitly state that
>>> modifying that returned string will indeed modify the buffer name.
>> Rather, `aset' should warn that modifying strings is evil and dangerous.
> When will this aset be grown up enough to play outside of home?

Never (as in "not for the foreseeable future"),  It would not bring any
benefit to the users, and would break backward compatibility: sound like
a clear loser to me.


        Stefan

Re: Buffer names are sometimes read-only objects in daemonized emacs

[Stefan Monnier]

>>> No no, you got me wrong.  I just meant the docstring of `buffer-name'
>>> (and any other function that returns some thing whose modification would
>>> have undesired, non-obvious side-effects) should explicitly state that
>>> modifying that returned string will indeed modify the buffer name.
>> Rather, `aset' should warn that modifying strings is evil and dangerous.
> Should `put-text-property' do that too?

Probably, but that will break a lot more packages (removing the "aset on
strings" feature breaks fairly few packages, surprisingly enough).


        Stefan