bug#5273: 23.1; format-alist encode vs write-region-post-annotation-function

bug#5273: 23.1; format-alist encode vs write-region-post-annotation-function

[Kevin Ryde]

In a format-alist "TO-FN" encode function, the buffer provided to that
function to operate on has write-region-post-annotation-function set to
`kill-buffer'.

This is extremely dangerous, and almost certainly incompatible with past
emacs, as it means any write-region done by the encode function will
kill the buffer it's operating on, causing it to then mangle some other
buffer, quite possibly with no indication that this happened.

A write-region in an encode function is likely if the function works by
putting data through an external program.


I struck this in three of my file formats where I use make-temp-file.
(Two for an error files for call-process-region, one because the program
needed an actual file to write to, not a pipe etc.)  Because
make-temp-file does its job by a writing an empty string to the new
file, it ran the kill-buffer from write-region-post-annotation-function.

I'm was unsure if make-temp-file ought to guard itself against this.  My
inclination is not.  There's an awful lot of read/write hooks and stuff,
and to set them so "write a string to a file" means "kill the current
buffer" is so unreasonable that there's no point anticipating it in one
particular place when so many things will be similarly affected.


I couldn't tell what that write-region-post-annotation-function is
trying to achieve.  If it's to kill the temporary buffer after writing
then an unwind-protect would sound far better, or at least apply the
kill setting only after running the format-alist encode func.


In GNU Emacs 23.1.1 (i486-pc-linux-gnu, GTK+ Version 2.16.5)
 of 2009-09-14 on raven, modified by Debian
configured using `configure  '--build=i486-linux-gnu' '--host=i486-linux-gnu' '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib' '--localstatedir=/var/lib' '--infodir=/usr/share/info' '--mandir=/usr/share/man' '--with-pop=yes' '--enable-locallisppath=/etc/emacs23:/etc/emacs:/usr/local/share/emacs/23.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/23.1/site-lisp:/usr/share/emacs/site-lisp:/usr/share/emacs/23.1/leim' '--with-x=yes' '--with-x-toolkit=gtk' '--with-toolkit-scroll-bars' 'build_alias=i486-linux-gnu' 'host_alias=i486-linux-gnu' 'CFLAGS=-DDEBIAN -g -O2' 'LDFLAGS=-g' 'CPPFLAGS=''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_AU
  value of $XMODIFIERS: nil
  locale-coding-system: iso-latin-1-unix
  default-enable-multibyte-characters: t

bug#5273: 23.1; format-alist encode vs write-region-post-annotation-function

[Kevin Ryde]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> Does the patch below help?

Yep.

The format-count thing should keep a lid on temp buffers if the to-fn
errors out (I do that if the contents are no good), though I still
slightly wonder that it ought to be with-temp-buffer or something.

bug#5273: 23.1; format-alist encode vs write-region-post-annotation-function

[Stefan Monnier]

>> Does the patch below help?
> Yep.

Thanks, installed.

> The format-count thing should keep a lid on temp buffers if the to-fn
> errors out (I do that if the contents are no good), though I still
> slightly wonder that it ought to be with-temp-buffer or something.

Not sure what you're referring to, but at least the issue of temp
buffers accumulating if to-fn errors out is indeed a problem, but not
a new one, so the patch doesn't make things worse.
with-temp-buffer is not an option, since the new "temp" buffer (usually)
needs to be returned as the new current-buffer at the end of
the function.


        Stefan

bug#5273: 23.1; format-alist encode vs write-region-post-annotation-function

[Kevin Ryde]

(Belated followup! ...)

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> the issue of temp
> buffers accumulating if to-fn errors out is indeed a problem, but not
> a new one,

I presume kill-on-write was an attempt to address that though.

As I say, I suspect the `format-count' passed from build_annotations()
naming the buffer "Format Temp 0" etc the same on each call will keep a
lid on how many leftovers.

I suspect that also means it's not reentrant -- if the TO-FN writes from
another buffer with a buffer-file-format then "Format Temp 0" will get
re-used and clobbered too soon.  I might start a separate bug for that.

> with-temp-buffer is not an option, since the new "temp" buffer (usually)
> needs to be returned as the new current-buffer at the end of
> the function.

There must be some far-enough outer scope of write-region or whatever
where temporaries can be discarded ... but the amount of stuff done just
to save some bytes is scary! :-)

bug#5273: 23.1; format-alist encode vs write-region-post-annotation-function

[Stefan Monnier]

>> the issue of temp buffers accumulating if to-fn errors out is indeed
>> a problem, but not a new one,

> I presume kill-on-write was an attempt to address that though.

No, not at all, the kill-on-write is done so that the buffer gets killed
after the calling's write-region terminates (the
post-annotation-function returns the buffer to write-region, so it
cannot kill it itself and is not involved any more after that so it has
no direct way to kill the buffer other than via this
write-region-post-annotation-function).

> As I say, I suspect the `format-count' passed from build_annotations()
> naming the buffer "Format Temp 0" etc the same on each call will keep a
> lid on how many leftovers.

> I suspect that also means it's not reentrant -- if the TO-FN writes from
> another buffer with a buffer-file-format then "Format Temp 0" will get
> re-used and clobbered too soon.

Could be.

>> with-temp-buffer is not an option, since the new "temp" buffer (usually)
>> needs to be returned as the new current-buffer at the end of
>> the function.

> There must be some far-enough outer scope of write-region or whatever
> where temporaries can be discarded ... but the amount of stuff done just
> to save some bytes is scary! :-)

Yes, but at that outside point, we don't know if such a temp buffer will
be needed, and worse, we don't know how many (there can be several
annotation functions that each take a buffer as input and produce the
result into a new output buffer).


        Stefan

bug#5273: 23.1; format-alist encode vs write-region-post-annotation-function

[Kevin Ryde]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>
> Yes, but at that outside point, we don't know if such a temp buffer will
> be needed, and worse, we don't know how many (there can be several
> annotation functions that each take a buffer as input and produce the
> result into a new output buffer).

Oh, well, a let-bound defvar holding a list of temp buffers to be added
to by sub-functions.  Or alternately make write-region (or
build_annotations or whatever) take responsibility for temp buffers
returned to it.  It's too complicated for me, but one of those would
seem better than kill-on-write.