bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[Drew Adams]

emacs -Q

M-: (add-hook 'temp-buffer-show-hook (lambda () (debug)) 'append)

C-h v auto-mode-interpreter-regexp

The hook is never invoked.  It is invoked in all previous Emacs
releases.  Do the same thing in Emacs 24.3, for example, and you
enter the debugger.

This has an effect on my code and setup, where I have
`fit-frame-if-one-window' on the hook: buffer *Help*, alone in its
frame, must always be refit to the latest *Help* text.  This is now
broken.

In GNU Emacs 24.4.50.1 (i686-pc-mingw32)
 of 2014-04-29 on ODIEONE
Bzr revision: 117031 monnier@iro.umontreal.ca-20140429151607-qnkgbymwfaj5ut08
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --prefix=/c/Devel/emacs/snapshot/trunk
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -g3'
 LDFLAGS=-Lc:/Devel/emacs/lib 'CPPFLAGS=-DGC_MCHECK=1
 -Ic:/Devel/emacs/include''

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[Drew Adams]

Needless to say, this bug is present also in the 24.4 pretest.

(I am using this development version at Stefan's request, to try
to catch the crash of bug #17340 with his patch.)

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[Drew Adams]

I would really like to get some help with this. So far, no
response at all.

Help functions using buffer *Help* previously used
`with-output-to-temp-buffer', which runs `temp-buffer-setup-hook'
and `temp-buffer-show-hook'.  And previously the setup hook
usually put the buffer in help mode.

Now, *Help* is set up by `with-help-window', and neither of those
hooks is run by it.  This is a regression, to me - any code or any
user who relies on those hooks being run for *Help* buffers is up
the creek without a paddle, AFAICS.

So be it.  But what is that code or that user supposed to do now?

`with-help-window' invokes `with-temp-buffer-window', which runs
`temp-buffer-window-setup-hook' and `temp-buffer-window-show-hook'.

To get *Help* to DTRT for the code or user that previously used
`temp-buffer-setup-hook' or `temp-buffer-show-hook, is it
appropriate to put the same things on `temp-buffer-window-setup-hook'
and `temp-buffer-window-show-hook' instead?  Or will that maybe
have unwanted effects because `with-temp-buffer-window' is perhaps
used for more than just *Help*?

Not clear to me.  And I find no help in the doc or NEWS - or in
this bug thread, so far.

Should I be adding `fit-frame-if-one-window' to `temp-buffer-window'
for Emacs 24.4, to get the effect it has in Emacs 24.3 (and prior)
by being on `temp-buffer-show-hook'?

If not, how to get the same effect?  If yes, how to deal with other
possible uses of `with-temp-buffer-window', which have nothing to do
with *Help*?  (In the case of `fit-frame-if-one-window', that would
probably not hurt anything, but the question is more general.)

--

The existing doc is anyway wrong in some cases, AFAICT: The doc
string of `with-temp-buffer-window' says:

  This construct is similar to `with-output-to-temp-buffer'
  but, neither runs `temp-buffer-setup-hook' which usually puts
  the buffer in Help mode, nor `temp-buffer-show-function' (the
  ACTION argument replaces this).

And the doc string of `temp-buffer-setup-hook', likewise, says:

  This hook is normally set up with a function to put the buffer
  in Help mode.

I don't get the impression that either of those statements is
true anymore.  It seems like there is *no relation* anymore
between such temp-buffer things and Help mode.  Furthermore,
grepping for `temp-buffer-setup-hook' in the Emacs sources
shows that it is not used for this at all anymore.

A little help, please.

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[martin rudalics]

 > To get *Help* to DTRT for the code or user that previously used
 > `temp-buffer-setup-hook' or `temp-buffer-show-hook, is it
 > appropriate to put the same things on `temp-buffer-window-setup-hook'
 > and `temp-buffer-window-show-hook' instead?  Or will that maybe
 > have unwanted effects because `with-temp-buffer-window' is perhaps
 > used for more than just *Help*?

Both `with-output-to-temp-buffer' and `with-temp-buffer-window' are and
can be used for other things than displaying *Help*.

 > Not clear to me.  And I find no help in the doc or NEWS - or in
 > this bug thread, so far.

The manual says about `with-temp-buffer-window' that

      This macro uses the normal hooks `temp-buffer-window-setup-hook'
      and `temp-buffer-window-show-hook' in place of the analogous hooks
      run by `with-output-to-temp-buffer'.

What else do you need?

 > Should I be adding `fit-frame-if-one-window' to `temp-buffer-window'
 > for Emacs 24.4, to get the effect it has in Emacs 24.3 (and prior)
 > by being on `temp-buffer-show-hook'?

I suppose you want to add it to `temp-buffer-window-show-hook'.

 > If not, how to get the same effect?  If yes, how to deal with other
 > possible uses of `with-temp-buffer-window', which have nothing to do
 > with *Help*?  (In the case of `fit-frame-if-one-window', that would
 > probably not hurt anything, but the question is more general.)

Emacs has a `temp-buffer-resize-mode' which is tied to temporary buffers
and not to `help-mode'.  To "deal with other possible uses" of temporary
buffers, a function run by `temp-buffer-show-hook' or
`temp-buffer-window-show-hook' should probably check whether the buffer
is in help mode.  In this regard nothing has changed wrt earlier version
where a user removed `help-mode-setup' from `temp-buffer-setup-hook' and
`help-mode-finish' from `temp-buffer-show-hook'.

Maybe you could also use `help-mode-hook' to do something for help
buffers exclusively and avoid other temporary buffers.

 > The existing doc is anyway wrong in some cases, AFAICT: The doc
 > string of `with-temp-buffer-window' says:
 >
 >    This construct is similar to `with-output-to-temp-buffer'
 >    but, neither runs `temp-buffer-setup-hook' which usually puts
 >    the buffer in Help mode, nor `temp-buffer-show-function' (the
 >    ACTION argument replaces this).

What is wrong here?

 > And the doc string of `temp-buffer-setup-hook', likewise, says:
 >
 >    This hook is normally set up with a function to put the buffer
 >    in Help mode.

This is still the case for the release version.  IIRC Leo Liu changed it
on the trunk so the doc-string should be probably updated there.

 > I don't get the impression that either of those statements is
 > true anymore.  It seems like there is *no relation* anymore
 > between such temp-buffer things and Help mode.  Furthermore,
 > grepping for `temp-buffer-setup-hook' in the Emacs sources
 > shows that it is not used for this at all anymore.

The connection between temporary buffers and help mode is established by
`with-help-window' which uses `with-temp-buffer-window'.

martin

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[Drew Adams]

>  > To get *Help* to DTRT for the code or user that previously used
>  > `temp-buffer-setup-hook' or `temp-buffer-show-hook, is it
>  > appropriate to put the same things on `temp-buffer-window-setup-hook'
>  > and `temp-buffer-window-show-hook' instead?  Or will that maybe
>  > have unwanted effects because `with-temp-buffer-window' is perhaps
>  > used for more than just *Help*?
> 
> Both `with-output-to-temp-buffer' and `with-temp-buffer-window' are
> and can be used for other things than displaying *Help*.

For `with-output-to-temp-buffer':  NOW, yes.  Before, no - it was
pretty much hardwired to help mode.

So there will be users and existing 3rd-party code that depend on
that behavior, which endured for 30 years or so.

This is why I have argued that the right fix for that problem was
not to change the behavior of `with-output-to-temp-buffer', and thus
gratuitously break such code and user expectations, but to make a
new macro for dealing with temporary buffer display that was not
hardwired to help.

Emacs itself could move to using that new macro, of course, but there
should be no reason to break the longstanding behavior of
`with-output-to-temp-buffer'.

>  > Not clear to me.  And I find no help in the doc or NEWS - or in
>  > this bug thread, so far.
> 
> The manual says about `with-temp-buffer-window' that
> 
>       This macro uses the normal hooks `temp-buffer-window-setup-hook'
>       and `temp-buffer-window-show-hook' in place of the analogous
>       hooks run by `with-output-to-temp-buffer'.
> 
> What else do you need?

What did I say?  Yes, I read that in the manual.  What about a function
that has been on `temp-buffer-setup-hook' or `temp-buffer-show-hook',
and that was specifically put there for help, i.e., that expects help
mode?

Is it necessarily appropriate to put that function on the *-window-*
hooks, without modifying it to work around use in a non-help mode?
I don't think that is the case, in general.  And I don't see a hook
that is analogous and specifically for help.

In my case of `fit-frame-if-one-window', this is not a problem AFAICT
- I added it to `temp-buffer-window-show-hook'.  But in the general
case there is a problem in moving a function from a temp-buffer hook to
a temp-buffer-window hook, whenever that function is specifically aimed
at help (expects help mode).

Sure, users and libraries can also change such a function, so that it
tests the buffer or mode to see whether it involves help.  But this is
gratuitous hassling, IMO - it should not be necessary.

I made the further point that none of this is documented, AFAICT.
Nothing about migrating one's use of temp-buffer stuff to whatever
is appropriate for Emacs 24.4+.  Nothing about incompatible
behavior changes for the temp-buffer stuff.

>  > Should I be adding `fit-frame-if-one-window' to `temp-buffer-window'
>  > for Emacs 24.4, to get the effect it has in Emacs 24.3 (and prior)
>  > by being on `temp-buffer-show-hook'?
> 
> I suppose you want to add it to `temp-buffer-window-show-hook'.

I supposed so too, and I did.  I asked the question several times,
hoping to incite adding some migration explanation to the doc.  But
thanks anyway for confirming.  I still hope someone updates the doc.

But again, though that is OK for `fit-frame-if-one-window', since I
want to invoke that regardless of which temp buffer is displayed, it
is not OK in general for functions that have been on
`temp-buffer-show-hook'.

It is likely that at least some such functions are specific to help
mode, since `temp-buffer-show-hook' was dedicated to help previously
(and for so long).

What do you tell library maintainers or users who have a function on
`temp-buffer-show-hook' that is appropriate for help mode but not
for other temp buffers?  Such information should be in the doc, IMO.

>  > If not, how to get the same effect?  If yes, how to deal with other
>  > possible uses of `with-temp-buffer-window', which have nothing to do
>  > with *Help*?  (In the case of `fit-frame-if-one-window', that would
>  > probably not hurt anything, but the question is more general.)
> 
> Emacs has a `temp-buffer-resize-mode' which is tied to temporary buffers
> and not to `help-mode'.  To "deal with other possible uses" of temporary
> buffers, a function run by `temp-buffer-show-hook' or
> `temp-buffer-window-show-hook' should probably check whether the buffer
> is in help mode.

It will have to, now.  Too bad.

> In this regard nothing has changed wrt earlier version where a user
> removed `help-mode-setup' from `temp-buffer-setup-hook' and
> `help-mode-finish' from `temp-buffer-show-hook'.

That is a very roundabout way of saying that IF there previously were
a user who for some reason coded things up to approximate what the
code does now, THEN ... "Nothing has changed"!

So-called temp-buffer display was previously coupled with help display.
Functions on `temp-buffer-show-hook' in existing code or user setups
could well depend on that behavior.  It was expected that the temp
buffer displayed would be in help mode - because it WAS - for 30 years.

Sure, someone could have jumped through hoops to use
`with-output-to-temp-buffer' without help mode.  But that would have
been relatively rare.  What percentage of the uses of
`with-output-to-temp-buffer' in the Emacs code corresponded to this?
Not more than 1% would be my guess - maybe 0%.

It is hardly the common use case of `with-output-to-temp-buffer'.

To suggest that it is, that "nothing has changed", would be
disingenuous.  Yes, I note that you did not claim that in general -
you qualified it as being the case only for such exceptional uses
(without acknowledging that they are exceptional).  But the
impression can be got from a cursory reading that you are saying
that nothing has changed in general, i.e., for most uses of
`with-output-to-temp-buffer'.

In fact, I have jumped through that hoop that in some of my code,
so I do recognize such an exception:

(defmacro bmkp-with-output-to-plain-temp-buffer (buf &rest body)
  "Like `with-output-to-temp-buffer', but with no *Help* navigation stuff."
  `(unwind-protect
    (progn
      (remove-hook 'temp-buffer-setup-hook 'help-mode-setup)
      (remove-hook 'temp-buffer-show-hook  'help-mode-finish)
      (with-output-to-temp-buffer ,buf ,@body))
    (add-hook 'temp-buffer-setup-hook 'help-mode-setup)
    (add-hook 'temp-buffer-show-hook  'help-mode-finish)))

But doing things like this was no doubt uncommon.  That was the point
of my original bug report asking that Emacs decouple temporary buffer
display from help display, i.e., that it provide a real temp-buffer
display that does not involve help (as does the macro above, in a
workaround way).

What was wrong was the way this decoupling was done in Emacs:
changing the behavior of macro `with-output-to-temp-buffer'.  That
was misguided, IMHO, and not necessary.  That boat has apparently
sailed, however.

> Maybe you could also use `help-mode-hook' to do something for help
> buffers exclusively and avoid other temporary buffers.
> 
>  > The existing doc is anyway wrong in some cases, AFAICT: The doc
>  > string of `with-temp-buffer-window' says:
>  >
>  >    This construct is similar to `with-output-to-temp-buffer'
>  >    but, neither runs `temp-buffer-setup-hook' which usually puts
>  >    the buffer in Help mode, nor `temp-buffer-show-function' (the
>  >    ACTION argument replaces this).
> 
> What is wrong here?

`temp-buffer-setup-hook' no longer "usually puts the buffer in Help
mode".

>  > And the doc string of `temp-buffer-setup-hook', likewise, says:
>  >
>  >    This hook is normally set up with a function to put the buffer
>  >    in Help mode.
> 
> This is still the case for the release version.  IIRC Leo Liu changed it
> on the trunk so the doc-string should be probably updated there.

The doc string is not updated in the trunk build I cited, from April 29.
I don't have a more recent build.  But yes, the doc should be updated -
that was what I was pointing out.

>  > I don't get the impression that either of those statements is
>  > true anymore.  It seems like there is *no relation* anymore
>  > between such temp-buffer things and Help mode.  Furthermore,
>  > grepping for `temp-buffer-setup-hook' in the Emacs sources
>  > shows that it is not used for this at all anymore.
> 
> The connection between temporary buffers and help mode is established
> by `with-help-window' which uses `with-temp-buffer-window'.

Yes, I know.

And previously such a connection was hardwired in
`with-output-to-temp-buffer'.  That behavior should not have changed.
Not because it was good to couple the two behaviors, but because they
have been so coupled for a long time in `with-output-to-temp-buffers'.

The right approach would have been to (a) leave
`with-output-to-temp-buffer' the way it was, (b) decouple temp-buffer
display from help mode by coming up with new macros, and (c) use the
new macros in the Emacs source code (this has been done).

That way, the Emacs code would no longer use `with-output-to-temp-buffer',
and thus would no longer depend on a coupling of temp-buffer display
with help, BUT users would not be bothered and existing 3rd-party code
would not be broken by a change to `with-output-to-temp-buffer': it would
continue to work as usual, coupling temp with help as before.  Eventually,
`with-output-to-temp-buffer' would be abandoned everywhere, naturally.

I pointed this out in my original bug report.  That is not what was
done.  We now have to live with the consequences of the incompatible
change.  That calls for doc that explains the change and how to deal
with it.

The doc (manual and/or NEWS) should clearly point out (a) what
`with-output-to-temp-buffer' does now, and (b) that it does not do what
it has always done before, and (c) this is how to migrate existing code
that uses it: A, B, C,...Z.

For Emacs's source code the change was trivial: replace uses of
`with-output-to-temp-buffer' with uses of the new macro.  But for
3rd-party code that supports multiple Emacs versions the change is not
so trivial.

It is too bad that we have arrived here, but we have.  Now let's patch
things up by at least letting users know about the damage done and how
to make do with it.

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[martin rudalics]

 >> Both `with-output-to-temp-buffer' and `with-temp-buffer-window' are
 >> and can be used for other things than displaying *Help*.
 >
 > For `with-output-to-temp-buffer':  NOW, yes.  Before, no - it was
 > pretty much hardwired to help mode.
 >
 > So there will be users and existing 3rd-party code that depend on
 > that behavior, which endured for 30 years or so.
 >
 > This is why I have argued that the right fix for that problem was
 > not to change the behavior of `with-output-to-temp-buffer', and thus
 > gratuitously break such code and user expectations, but to make a
 > new macro for dealing with temporary buffer display that was not
 > hardwired to help.
 >
 > Emacs itself could move to using that new macro, of course, but there
 > should be no reason to break the longstanding behavior of
 > `with-output-to-temp-buffer'.

It wasn't my intention to change the longstanding behavior of
`with-output-to-temp-buffer'.

 > What did I say?  Yes, I read that in the manual.  What about a function
 > that has been on `temp-buffer-setup-hook' or `temp-buffer-show-hook',
 > and that was specifically put there for help, i.e., that expects help
 > mode?

Generally assuming that a temporary buffer is always in help mode is
wrong.  Callers of `with-output-to-temp-buffer' can always change the
mode as, for example, ediff does.

 > Is it necessarily appropriate to put that function on the *-window-*
 > hooks, without modifying it to work around use in a non-help mode?
 > I don't think that is the case, in general.  And I don't see a hook
 > that is analogous and specifically for help.
 >
 > In my case of `fit-frame-if-one-window', this is not a problem AFAICT
 > - I added it to `temp-buffer-window-show-hook'.  But in the general
 > case there is a problem in moving a function from a temp-buffer hook to
 > a temp-buffer-window hook, whenever that function is specifically aimed
 > at help (expects help mode).
 >
 > Sure, users and libraries can also change such a function, so that it
 > tests the buffer or mode to see whether it involves help.  But this is
 > gratuitous hassling, IMO - it should not be necessary.
 >
 > I made the further point that none of this is documented, AFAICT.
 > Nothing about migrating one's use of temp-buffer stuff to whatever
 > is appropriate for Emacs 24.4+.  Nothing about incompatible
 > behavior changes for the temp-buffer stuff.

It was wrong to put a function on `temp-buffer-show-hook' without
testing whether the associated buffer was in help mode when the
intention is to run the function for help mode buffers only.

 >>   > Should I be adding `fit-frame-if-one-window' to `temp-buffer-window'
 >>   > for Emacs 24.4, to get the effect it has in Emacs 24.3 (and prior)
 >>   > by being on `temp-buffer-show-hook'?
 >>
 >> I suppose you want to add it to `temp-buffer-window-show-hook'.
 >
 > I supposed so too, and I did.  I asked the question several times,
 > hoping to incite adding some migration explanation to the doc.  But
 > thanks anyway for confirming.  I still hope someone updates the doc.
 >
 > But again, though that is OK for `fit-frame-if-one-window', since I
 > want to invoke that regardless of which temp buffer is displayed, it
 > is not OK in general for functions that have been on
 > `temp-buffer-show-hook'.
 >
 > It is likely that at least some such functions are specific to help
 > mode, since `temp-buffer-show-hook' was dedicated to help previously
 > (and for so long).

Such functions were based on a wrong assumption.

 > What do you tell library maintainers or users who have a function on
 > `temp-buffer-show-hook' that is appropriate for help mode but not
 > for other temp buffers?  Such information should be in the doc, IMO.

I tell them here on this list that such a function should have tested
whether the buffer is in help mode.

 >> Emacs has a `temp-buffer-resize-mode' which is tied to temporary buffers
 >> and not to `help-mode'.  To "deal with other possible uses" of temporary
 >> buffers, a function run by `temp-buffer-show-hook' or
 >> `temp-buffer-window-show-hook' should probably check whether the buffer
 >> is in help mode.
 >
 > It will have to, now.  Too bad.

On the contrary.  Older versions of Emacs running your code will benefit
from more correct code.

 > That is a very roundabout way of saying that IF there previously were
 > a user who for some reason coded things up to approximate what the
 > code does now, THEN ... "Nothing has changed"!
 >
 > So-called temp-buffer display was previously coupled with help display.
 > Functions on `temp-buffer-show-hook' in existing code or user setups
 > could well depend on that behavior.  It was expected that the temp
 > buffer displayed would be in help mode - because it WAS - for 30 years.
 >
 > Sure, someone could have jumped through hoops to use
 > `with-output-to-temp-buffer' without help mode.  But that would have
 > been relatively rare.  What percentage of the uses of
 > `with-output-to-temp-buffer' in the Emacs code corresponded to this?
 > Not more than 1% would be my guess - maybe 0%.
 >
 > It is hardly the common use case of `with-output-to-temp-buffer'.
 >
 > To suggest that it is, that "nothing has changed", would be
 > disingenuous.  Yes, I note that you did not claim that in general -
 > you qualified it as being the case only for such exceptional uses
 > (without acknowledging that they are exceptional).  But the
 > impression can be got from a cursory reading that you are saying
 > that nothing has changed in general, i.e., for most uses of
 > `with-output-to-temp-buffer'.
 >
 > In fact, I have jumped through that hoop that in some of my code,
 > so I do recognize such an exception:
 >
 > (defmacro bmkp-with-output-to-plain-temp-buffer (buf &rest body)
 >    "Like `with-output-to-temp-buffer', but with no *Help* navigation stuff."
 >    `(unwind-protect
 >      (progn
 >        (remove-hook 'temp-buffer-setup-hook 'help-mode-setup)
 >        (remove-hook 'temp-buffer-show-hook  'help-mode-finish)
 >        (with-output-to-temp-buffer ,buf ,@body))
 >      (add-hook 'temp-buffer-setup-hook 'help-mode-setup)
 >      (add-hook 'temp-buffer-show-hook  'help-mode-finish)))
 >
 > But doing things like this was no doubt uncommon.  That was the point
 > of my original bug report asking that Emacs decouple temporary buffer
 > display from help display, i.e., that it provide a real temp-buffer
 > display that does not involve help (as does the macro above, in a
 > workaround way).

So you fit a frame shown by `bmkp-with-output-to-plain-temp-buffer' even
if the buffer is not in help mode?

 > What was wrong was the way this decoupling was done in Emacs:
 > changing the behavior of macro `with-output-to-temp-buffer'.  That
 > was misguided, IMHO, and not necessary.  That boat has apparently
 > sailed, however.

Maybe.

 >>   >    This construct is similar to `with-output-to-temp-buffer'
 >>   >    but, neither runs `temp-buffer-setup-hook' which usually puts
 >>   >    the buffer in Help mode, nor `temp-buffer-show-function' (the
 >>   >    ACTION argument replaces this).
 >>
 >> What is wrong here?
 >
 > `temp-buffer-setup-hook' no longer "usually puts the buffer in Help
 > mode".

I see.  This is due to Leo's change.

 >>   > And the doc string of `temp-buffer-setup-hook', likewise, says:
 >>   >
 >>   >    This hook is normally set up with a function to put the buffer
 >>   >    in Help mode.
 >>
 >> This is still the case for the release version.  IIRC Leo Liu changed it
 >> on the trunk so the doc-string should be probably updated there.
 >
 > The doc string is not updated in the trunk build I cited, from April 29.
 > I don't have a more recent build.  But yes, the doc should be updated -
 > that was what I was pointing out.

Agreed.

 >> The connection between temporary buffers and help mode is established
 >> by `with-help-window' which uses `with-temp-buffer-window'.
 >
 > Yes, I know.
 >
 > And previously such a connection was hardwired in
 > `with-output-to-temp-buffer'.  That behavior should not have changed.
 > Not because it was good to couple the two behaviors, but because they
 > have been so coupled for a long time in `with-output-to-temp-buffers'.
 >
 > The right approach would have been to (a) leave
 > `with-output-to-temp-buffer' the way it was, (b) decouple temp-buffer
 > display from help mode by coming up with new macros, and (c) use the
 > new macros in the Emacs source code (this has been done).
 >
 > That way, the Emacs code would no longer use `with-output-to-temp-buffer',
 > and thus would no longer depend on a coupling of temp-buffer display
 > with help, BUT users would not be bothered and existing 3rd-party code
 > would not be broken by a change to `with-output-to-temp-buffer': it would
 > continue to work as usual, coupling temp with help as before.  Eventually,
 > `with-output-to-temp-buffer' would be abandoned everywhere, naturally.

I might agree with you here.

 > I pointed this out in my original bug report.  That is not what was
 > done.  We now have to live with the consequences of the incompatible
 > change.  That calls for doc that explains the change and how to deal
 > with it.
 >
 > The doc (manual and/or NEWS) should clearly point out (a) what
 > `with-output-to-temp-buffer' does now,

Here ...

 > and (b) that it does not do what
 > it has always done before, and (c) this is how to migrate existing code
 > that uses it: A, B, C,...Z.
 >
 > For Emacs's source code the change was trivial: replace uses of
 > `with-output-to-temp-buffer' with uses of the new macro.

... and here ...

 > But for
 > 3rd-party code that supports multiple Emacs versions the change is not
 > so trivial.
 >
 > It is too bad that we have arrived here, but we have.  Now let's patch
 > things up by at least letting users know about the damage done and how
 > to make do with it.

... you're lumping together a change in `with-help-window' with a change
affecting `with-output-to-temp-buffer'.  I can only address the former.

martin

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[Drew Adams]

>  >> Both `with-output-to-temp-buffer' and `with-temp-buffer-window' are
>  >> and can be used for other things than displaying *Help*.
>  >
>  > For `with-output-to-temp-buffer':  NOW, yes.  Before, no - it was
>  > pretty much hardwired to help mode.
>  >
>  > So there will be users and existing 3rd-party code that depend on
>  > that behavior, which endured for 30 years or so.
>  >
>  > This is why I have argued that the right fix for that problem was
>  > not to change the behavior of `with-output-to-temp-buffer', and thus
>  > gratuitously break such code and user expectations, but to make a
>  > new macro for dealing with temporary buffer display that was not
>  > hardwired to help.
>  >
>  > Emacs itself could move to using that new macro, of course, but there
>  > should be no reason to break the longstanding behavior of
>  > `with-output-to-temp-buffer'.
> 
> It wasn't my intention to change the longstanding behavior of
> `with-output-to-temp-buffer'.

Well, it's done.  Is it your intention to revert that change? ;-)

>  > What did I say?  Yes, I read that in the manual.  What about a function
>  > that has been on `temp-buffer-setup-hook' or `temp-buffer-show-hook',
>  > and that was specifically put there for help, i.e., that expects help
>  > mode?
> 
> Generally assuming that a temporary buffer is always in help mode is
> wrong.  Callers of `with-output-to-temp-buffer' can always change the
> mode as, for example, ediff does.

As we have discussed at length elsewhere, `with-output-to-temp-buffer' was
NOT just about temporary buffers.  In spite of its name, it was about a
temporary buffer in help mode.  That's a fact of life.  I lamented the
name-vs-behavior mismatch, and filed a bug about that, but that's the way
it became over time, and it's been that way for a very long time now.

>  > Is it necessarily appropriate to put that function on the *-window-*
>  > hooks, without modifying it to work around use in a non-help mode?
>  > I don't think that is the case, in general.  And I don't see a hook
>  > that is analogous and specifically for help.
>  >
>  > In my case of `fit-frame-if-one-window', this is not a problem AFAICT
>  > - I added it to `temp-buffer-window-show-hook'.  But in the general
>  > case there is a problem in moving a function from a temp-buffer hook to
>  > a temp-buffer-window hook, whenever that function is specifically aimed
>  > at help (expects help mode).
>  >
>  > Sure, users and libraries can also change such a function, so that it
>  > tests the buffer or mode to see whether it involves help.  But this is
>  > gratuitous hassling, IMO - it should not be necessary.
>  >
>  > I made the further point that none of this is documented, AFAICT.
>  > Nothing about migrating one's use of temp-buffer stuff to whatever
>  > is appropriate for Emacs 24.4+.  Nothing about incompatible
>  > behavior changes for the temp-buffer stuff.
> 
> It was wrong to put a function on `temp-buffer-show-hook' without
> testing whether the associated buffer was in help mode when the
> intention is to run the function for help mode buffers only.

I disagree.  `with-output-to-temp-buffer' simply has never (almost never;
I cannot find an old enough version where it was ever the case) been about
temp buffers other than help-mode buffers.

You cannot just reinterpret things based on what the name suggests it
should do.  It has not done what its name says for 30 years now.

On the contrary, in the very rare cases where someone actually tried to
use `w-o-t-t-b' for something other than help mode, s?he had to jump
through a hoop to get that behavior.

It is not up to all functions that have been on that hook to test whether
the buffer is in help mode.  The behavior has been to ALWAYS put the buffer
in help mode. To get a different behavior you needed to jump through a
hoop (removing stuff from `temp-buffer-setup-hook').

>  >>   > Should I be adding `fit-frame-if-one-window' to `temp-buffer-window'
>  >>   > for Emacs 24.4, to get the effect it has in Emacs 24.3 (and prior)
>  >>   > by being on `temp-buffer-show-hook'?
>  >>
>  >> I suppose you want to add it to `temp-buffer-window-show-hook'.
>  >
>  > I supposed so too, and I did.  I asked the question several times,
>  > hoping to incite adding some migration explanation to the doc.  But
>  > thanks anyway for confirming.  I still hope someone updates the doc.
>  >
>  > But again, though that is OK for `fit-frame-if-one-window', since I
>  > want to invoke that regardless of which temp buffer is displayed, it
>  > is not OK in general for functions that have been on
>  > `temp-buffer-show-hook'.
>  >
>  > It is likely that at least some such functions are specific to help
>  > mode, since `temp-buffer-show-hook' was dedicated to help previously
>  > (and for so long).
> 
> Such functions were based on a wrong assumption.

No, they were not.  They were based on the longstanding and documented
behavior of `with-output-to-temp-buffer'.  See above.  The macro was
simply misnamed for what it did.  And there unfortunately was no macro
that did what its name suggests.  That was the original bug I reported.

>  > What do you tell library maintainers or users who have a function on
>  > `temp-buffer-show-hook' that is appropriate for help mode but not
>  > for other temp buffers?  Such information should be in the doc, IMO.
> 
> I tell them here on this list that such a function should have tested
> whether the buffer is in help mode.

Not helpful, IMO.

>  >> Emacs has a `temp-buffer-resize-mode' which is tied to temporary buffers
>  >> and not to `help-mode'.  To "deal with other possible uses" of temporary
>  >> buffers, a function run by `temp-buffer-show-hook' or
>  >> `temp-buffer-window-show-hook' should probably check whether the buffer
>  >> is in help mode.
>  >
>  > It will have to, now.  Too bad.
> 
> On the contrary.  Older versions of Emacs running your code will benefit
> from more correct code.

"It", not "I".  I don't have any such code.

Anyway, that is not "on the contrary".  You confirm that "It will have to,
now."  What is too bad is that the function code needs to be changed at all.

There was no need to change the misnamed `with-output-to-temp-buffer'
behavior.  You even claimed above that that was not your intention.  If the
behavior had not been changed then there would be no "too bad" here - no need
to change a function someone has on the hook.

> So you fit a frame shown by `bmkp-with-output-to-plain-temp-buffer' even
> if the buffer is not in help mode?

Yes.  I already made it clear that my use of `fit-frame-if-one-window'
on `temp-buffer-show-hook' is not a problem for non-help modes.  And I
said clearly that I have added it to `temp-buffer-window-show-hook'.

That will not be true in general for functions that users and libraries
have on `with-output-to-temp-buffer'.  That's the point here.

>  > What was wrong was the way this decoupling was done in Emacs:
>  > changing the behavior of macro `with-output-to-temp-buffer'.  That
>  > was misguided, IMHO, and not necessary.  That boat has apparently
>  > sailed, however.
> 
> Maybe.

You mean there's hope?

> I see.  This is due to Leo's change.

>  > The right approach would have been to (a) leave
>  > `with-output-to-temp-buffer' the way it was, (b) decouple temp-buffer
>  > display from help mode by coming up with new macros, and (c) use the
>  > new macros in the Emacs source code (this has been done).
>  >
>  > That way, the Emacs code would no longer use `with-output-to-temp-
>  > buffer', and thus would no longer depend on a coupling of temp-buffer
>  > display with help, BUT users would not be bothered and existing 3rd-party
>  > code would not be broken by a change to `with-output-to-temp-buffer': it
>  > would continue to work as usual, coupling temp with help as before.
>  > Eventually, `with-output-to-temp-buffer' would be abandoned everywhere,
>  > naturally.
> 
> I might agree with you here.

You mean there's hope? ;-)

>  > I pointed this out in my original bug report.  That is not what was
>  > done.  We now have to live with the consequences of the incompatible
>  > change.  That calls for doc that explains the change and how to deal
>  > with it.
>  >
>  > The doc (manual and/or NEWS) should clearly point out (a) what
>  > `with-output-to-temp-buffer' does now,
> 
> Here ...

If you mean that it suffices to let users know about such things in a
bug thread then I disagree.  In particular, NEWS should mention incompatible
changes and tell users how to migrate code to accommodate those changes.

>  > and (b) that it does not do what
>  > it has always done before, and (c) this is how to migrate existing code
>  > that uses it: A, B, C,...Z.
>  >
>  > For Emacs's source code the change was trivial: replace uses of
>  > `with-output-to-temp-buffer' with uses of the new macro.
> 
> ... and here ...

Dunno what you are saying here; sorry.  "Here ... and here ..." what?

> ... you're lumping together a change in `with-help-window' with a change
> affecting `with-output-to-temp-buffer'.  I can only address the former.

Why do you say that?  I have no problem with the creation of
`with-help-window' and its use in Emacs code, including instead of
`with-output-to-temp-buffer'.  It is the incompatible
change to `with-output-to-temp-buffer' that is the problem.  That, and a
lack of doc mentioning such things to users.

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[martin rudalics]

 >> It wasn't my intention to change the longstanding behavior of
 >> `with-output-to-temp-buffer'.
 >
 > Well, it's done.  Is it your intention to revert that change? ;-)

No.  I'm too surprised that it didn't cause any misbehavior so far.

 > As we have discussed at length elsewhere, `with-output-to-temp-buffer' was
 > NOT just about temporary buffers.  In spite of its name, it was about a
 > temporary buffer in help mode.  That's a fact of life.  I lamented the
 > name-vs-behavior mismatch, and filed a bug about that, but that's the way
 > it became over time, and it's been that way for a very long time now.

Since after Leo has taken it out, I haven't seen any complaints yet.  So
even if it is a fact of life it apparently didn't have any real impact.
Now if only all facts of life were like this ...

 > I disagree.  `with-output-to-temp-buffer' simply has never (almost never;
 > I cannot find an old enough version where it was ever the case) been about
 > temp buffers other than help-mode buffers.
 >
 > You cannot just reinterpret things based on what the name suggests it
 > should do.  It has not done what its name says for 30 years now.
 >
 > On the contrary, in the very rare cases where someone actually tried to
 > use `w-o-t-t-b' for something other than help mode, s?he had to jump
 > through a hoop to get that behavior.
 >
 > It is not up to all functions that have been on that hook to test whether
 > the buffer is in help mode.

Obviously not.  But if a function wants to act only upon buffers in help
mode, that function should check whether the buffer is in help mode.

 > The behavior has been to ALWAYS put the buffer
 > in help mode.

If the intention was to ALWAYS put the buffer in help mode, there would
not have been any reason to do it with a hook.  Rather, the `help-mode'
call would have been hardcoded in `with-output-to-temp-buffer'.

 > To get a different behavior you needed to jump through a
 > hoop (removing stuff from `temp-buffer-setup-hook').

IIRC clients do that by simply changing the mode in the BODY of
`with-output-to-temp-buffer'.

 >>   > It is likely that at least some such functions are specific to help
 >>   > mode, since `temp-buffer-show-hook' was dedicated to help previously
 >>   > (and for so long).
 >>
 >> Such functions were based on a wrong assumption.
 >
 > No, they were not.  They were based on the longstanding and documented
 > behavior of `with-output-to-temp-buffer'.  See above.  The macro was
 > simply misnamed for what it did.  And there unfortunately was no macro
 > that did what its name suggests.  That was the original bug I reported.

I see only one manual reference for `temp-buffer-setup-hook' which says
that "This hook is normally set up with a function to put the buffer in
Help mode.".  The term "normally" clearly implies that an application in
order to be safe must handle behavior that is not normal.

 >>   > What do you tell library maintainers or users who have a function on
 >>   > `temp-buffer-show-hook' that is appropriate for help mode but not
 >>   > for other temp buffers?  Such information should be in the doc, IMO.
 >>
 >> I tell them here on this list that such a function should have tested
 >> whether the buffer is in help mode.
 >
 > Not helpful, IMO.

Our opinions differ.

 >>   >> Emacs has a `temp-buffer-resize-mode' which is tied to temporary buffers
 >>   >> and not to `help-mode'.  To "deal with other possible uses" of temporary
 >>   >> buffers, a function run by `temp-buffer-show-hook' or
 >>   >> `temp-buffer-window-show-hook' should probably check whether the buffer
 >>   >> is in help mode.
 >>   >
 >>   > It will have to, now.  Too bad.
 >>
 >> On the contrary.  Older versions of Emacs running your code will benefit
 >> from more correct code.
 >
 > "It", not "I".  I don't have any such code.

Then I don't understand why you've been asking for help before.

 > Anyway, that is not "on the contrary".  You confirm that "It will have to,
 > now."  What is too bad is that the function code needs to be changed at all.

It should never have been written that way.  So "now" is a good occasion
to fix this.

 > There was no need to change the misnamed `with-output-to-temp-buffer'
 > behavior.  You even claimed above that that was not your intention.  If the
 > behavior had not been changed then there would be no "too bad" here - no need
 > to change a function someone has on the hook.

So let's delay this discussion until "someone" shows up.

 > You mean there's hope?

That the `with-output-to-temp-buffer' change will be reverted?  If
someone finds a real problem in it, yes.

 >> I might agree with you here.
 >
 > You mean there's hope? ;-)

See above.

 >>   > I pointed this out in my original bug report.  That is not what was
 >>   > done.  We now have to live with the consequences of the incompatible
 >>   > change.  That calls for doc that explains the change and how to deal
 >>   > with it.
 >>   >
 >>   > The doc (manual and/or NEWS) should clearly point out (a) what
 >>   > `with-output-to-temp-buffer' does now,
 >>
 >> Here ...
 >
 > If you mean that it suffices to let users know about such things in a
 > bug thread then I disagree.  In particular, NEWS should mention incompatible
 > changes and tell users how to migrate code to accommodate those changes.
 >
 >>   > and (b) that it does not do what
 >>   > it has always done before, and (c) this is how to migrate existing code
 >>   > that uses it: A, B, C,...Z.
 >>   >
 >>   > For Emacs's source code the change was trivial: replace uses of
 >>   > `with-output-to-temp-buffer' with uses of the new macro.
 >>
 >> ... and here ...
 >
 > Dunno what you are saying here; sorry.  "Here ... and here ..." what?
 >
 >> ... you're lumping together a change in `with-help-window' with a change
 >> affecting `with-output-to-temp-buffer'.  I can only address the former.
 >
 > Why do you say that?  I have no problem with the creation of
 > `with-help-window' and its use in Emacs code, including instead of
 > `with-output-to-temp-buffer'.  It is the incompatible
 > change to `with-output-to-temp-buffer' that is the problem.  That, and a
 > lack of doc mentioning such things to users.

If you mean that we should document the removal of `help-mode' from
`temp-buffer-setup-hook' then I obviously agree.  I have no idea what
else you mean to do.

martin

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[Drew Adams]

>  >> It wasn't my intention to change the longstanding behavior of
>  >> `with-output-to-temp-buffer'.
>  >
>  > Well, it's done.  Is it your intention to revert that change? ;-)
> 
> No.  I'm too surprised that it didn't cause any misbehavior so far.

Bug reports of regressions don't count?

>  > As we have discussed at length elsewhere, `with-output-to-temp-buffer'
>  > was NOT just about temporary buffers.  In spite of its name, it was about
>  > a temporary buffer in help mode.  That's a fact of life.  I lamented the
>  > name-vs-behavior mismatch, and filed a bug about that, but that's the way
>  > it became over time, and it's been that way for a very long time now.
> 
> Since after Leo has taken it out, I haven't seen any complaints yet.  So
> even if it is a fact of life it apparently didn't have any real impact.
> Now if only all facts of life were like this ...

This and related bug reports are complaints.  And they describe real impact.

>  > I disagree.  `with-output-to-temp-buffer' simply has never (almost never;
>  > I cannot find an old enough version where it was ever the case) been
>  > about temp buffers other than help-mode buffers.
>  >
>  > You cannot just reinterpret things based on what the name suggests it
>  > should do.  It has not done what its name says for 30 years now.
>  >
>  > On the contrary, in the very rare cases where someone actually tried to
>  > use `w-o-t-t-b' for something other than help mode, s?he had to jump
>  > through a hoop to get that behavior.
>  >
>  > It is not up to all functions that have been on that hook to test whether
>  > the buffer is in help mode.
> 
> Obviously not.  But if a function wants to act only upon buffers in help
> mode, that function should check whether the buffer is in help mode.

Not if the macro used is `with-output-to-temp-buffer', since it has always
taken care of establishing help mode.  Until now.

>  > The behavior has been to ALWAYS put the buffer in help mode.
> 
> If the intention was to ALWAYS put the buffer in help mode, there would
> not have been any reason to do it with a hook.  Rather, the `help-mode'
> call would have been hardcoded in `with-output-to-temp-buffer'.

Do you really want to argue about what the original intentions were?
`with-output-to-temp-buffer' was co-opted to serve only help, fairly
early in its history.  The name itself betrays a mismatch between its
(longstanding) behavior and what its original intention might have been
(as reflected by its name).

You seem to be stretching, to find any justification whatsoever for this
incompatible change to the behavior of `with-output-to-temp-buffer'.  If
you hide your eyes, of course you won't see - nothing I can do about that.

>  >>   > It is likely that at least some such functions are specific to help
>  >>   > mode, since `temp-buffer-show-hook' was dedicated to help previously
>  >>   > (and for so long).
>  >>
>  >> Such functions were based on a wrong assumption.
>  >
>  > No, they were not.  They were based on the longstanding and documented
>  > behavior of `with-output-to-temp-buffer'.  See above.  The macro was
>  > simply misnamed for what it did.  And there unfortunately was no macro
>  > that did what its name suggests.  That was the original bug I reported.
> 
> I see only one manual reference for `temp-buffer-setup-hook' which says
> that "This hook is normally set up with a function to put the buffer in
> Help mode.".  The term "normally" clearly implies that an application in
> order to be safe must handle behavior that is not normal.

You are ignoring reality.  Code and users have had a reasonable expectation
for 30 years about the behavior of `with-output-to-temp-buffer', and that
has now been thrown out the window gratuitously.

Gratuitously, because nothing was gained by it.  Emacs could just as well
have moved on to use new and different macros, without changing the
`w-o-t-t-b' behavior.  That was totally uncalled for and serves no purpose.
Not necessary.

>  >>   > What do you tell library maintainers or users who have a function on
>  >>   > `temp-buffer-show-hook' that is appropriate for help mode but not
>  >>   > for other temp buffers?  Such information should be in the doc, IMO.
>  >>
>  >> I tell them here on this list that such a function should have tested
>  >> whether the buffer is in help mode.
>  >
>  > Not helpful, IMO.
> 
> Our opinions differ.

Yes.  Some info from a developer that is only buried in some bug thread
might be helpful to the few who read that thread.  But not more useful
than that.

Emacs users deserve better communication.  Especially about incompatible
changes.  Especially about behavior that has remained stable for 30 years.
Not communicating such info properly is irresponsible, IMHO.

> If you mean that we should document the removal of `help-mode' from
> `temp-buffer-setup-hook' then I obviously agree.

Good.  And consequently from `with-output-to-temp-buffer-buffer',
which is what users will see in their code.

Start with that.  Plus an explanation of how to migrate code that
expects `with-output-to-temp-buffer-buffer' to establish help mode.
(Mention `with-help-window', `with-temp-buffer-window', etc.)

Better would of course be to not screw `with-output-to-temp-buffer' at
all.  Emacs gains nothing by messing with that, and would lose nothing
by restoring its behavior.  I've seen no argument showing a loss by
doing that.  Do you have such an argument?

Changing `w-o-t-t-b' is not at all necessary to accomplish the clean
separation of temp buffer setup from help buffer setup that was the
goal (and which I was the one to propose, FWIW).

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[martin rudalics]

 > You are ignoring reality.

Indeed.  I'll stop doing that now.

martin

bug#17397: 24.4.50; REGRESSION: `temp-buffer-show-hook' is no longer invoked for `describe-variable'

[Stefan Kangas]

tags 17397 + wontfix notabug
close 17397
thanks

martin rudalics <rudalics@gmx.at> writes:

>>> It wasn't my intention to change the longstanding behavior of
>>> `with-output-to-temp-buffer'.
>>
>> Well, it's done.  Is it your intention to revert that change? ;-)
>
> No.  I'm too surprised that it didn't cause any misbehavior so far.

It's kind of hard to follow this bug report.

From what I gather, it's about reverting some change that was made many
years ago, and the conclusion was that we don't intend to do that.

In any case, that's even more unlikely to happen now, seven years later,
so it hardly makes sense to keep this open.  I'm therefore closing this
bug report.