threads and kill-buffer

threads and kill-buffer

[Tom Tromey]

Suppose thread T has buffer B as its current buffer, and then some other
thread evaluates (kill-buffer B).  What should happen in thread T?

I thought of a few options, but I'm not really sure which is best.


In the single-threaded case, Emacs picks some other buffer to be the
current buffer.  So, one option would be to do this in the
multi-threaded case.

As long as we assume cooperative threading, this seems reasonable
enough -- it is analogous to what would happen if a process filter were
to call kill-buffer.


However, in the long run it would be good to make threads preemptive.
In this case, it seems to me that switching the current buffer like this
would be very surprising.  (But maybe still ok?)

One idea here would be to make various operations on buffers throw an
exception if the killed buffer were used.  For example, "insert" would
do this.

One problem with this approach is that this may mean that 'let' could
now throw a "killed-buffer" exception in some cases.  This seems
moderately surprising.

Another problem with this approach is just that it is hard to implement.
I see 1085 references to 'current_buffer' in the tree, and presumably
I'd have to examine each one...


Yet another idea is to make kill-buffer refuse to do this.

Tom

Re: threads and kill-buffer

[Lars Ingebrigtsen]

Tom Tromey <tromey@redhat.com> writes:

> However, in the long run it would be good to make threads preemptive.
> In this case, it seems to me that switching the current buffer like this
> would be very surprising.  (But maybe still ok?)

This happens today with poorly-written process filters and is a total
nightmare to debug.

-- 
(domestic pets only, the antidote for overdose, milk.)
  http://lars.ingebrigtsen.no  *  Sent from my Emacs

Re: threads and kill-buffer

[Paul Eggert]

On 09/04/2012 01:36 PM, Tom Tromey wrote:
> Yet another idea is to make kill-buffer refuse to do this.

This sounds simplest.  Maybe we could also change things
so that kill-buffer refuses the pull the rug from under
a process-filter too, to address the problems Lars mentions.

Re: threads and kill-buffer

[Eli Zaretskii]

> From: Tom Tromey <tromey@redhat.com>
> Date: Tue, 04 Sep 2012 14:36:22 -0600
> 
> One idea here would be to make various operations on buffers throw an
> exception if the killed buffer were used.  For example, "insert" would
> do this.
> 
> One problem with this approach is that this may mean that 'let' could
> now throw a "killed-buffer" exception in some cases.  This seems
> moderately surprising.
> 
> Another problem with this approach is just that it is hard to implement.
> I see 1085 references to 'current_buffer' in the tree, and presumably
> I'd have to examine each one...
> 
> 
> Yet another idea is to make kill-buffer refuse to do this.

How about letting kill-buffer succeed, but delay the actual deletion
of the buffer until no thread has it as current, like what Posix
filesystems do with file deletion?

Re: threads and kill-buffer

[Sam Steingold]

> * Eli Zaretskii <ryvm@tah.bet> [2012-09-05 05:53:56 +0300]:
>
> How about letting kill-buffer succeed, but delay the actual deletion
> of the buffer until no thread has it as current, like what Posix
> filesystems do with file deletion?

I am not sure that Posix file systems do this "by design" and not as a
side effect of an implementation detail.

At any rate, I don't think this behavior is very useful.

It is a certain way to lose data: imagine thread T1 moving data from
buffer B1 to buffer B2 and thread T2 killing one of these buffers.
Killing B1 is fine, but killing B2 will lose data. Note that being the
current buffer of another thread is an insufficient protection because a
thread may be writing to several buffers and only one of them may be
current to it, so a thread should be able to mark a buffer as used by
it, so that an attempt to kill a buffer used by a live thread would
result in an exception.


-- 
Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X 11.0.11103000
http://www.childpsy.net/ http://palestinefacts.org http://www.memritv.org
http://jihadwatch.org http://honestreporting.com
When we break the law, they fine us, when we comply, they tax us.

Re: threads and kill-buffer

[Eli Zaretskii]

> From: Sam Steingold <sds@gnu.org>
> Date: Wed, 05 Sep 2012 10:20:47 -0400
> 
> > * Eli Zaretskii <ryvm@tah.bet> [2012-09-05 05:53:56 +0300]:
> >
> > How about letting kill-buffer succeed, but delay the actual deletion
> > of the buffer until no thread has it as current, like what Posix
> > filesystems do with file deletion?
> 
> I am not sure that Posix file systems do this "by design" and not as a
> side effect of an implementation detail.

Why does it matter?  If we implement the same logic for buffers, it
will surely be by design.

> It is a certain way to lose data: imagine thread T1 moving data from
> buffer B1 to buffer B2 and thread T2 killing one of these buffers.
> Killing B1 is fine, but killing B2 will lose data. Note that being the
> current buffer of another thread is an insufficient protection because a
> thread may be writing to several buffers and only one of them may be
> current to it, so a thread should be able to mark a buffer as used by
> it, so that an attempt to kill a buffer used by a live thread would
> result in an exception.

That's an entirely different problem than the one that started this
thread.  The original problem was that each thread must have a current
buffer, and what to do when it is deleted by another thread.  That
problem is not about losing data at all, it's about gobs of Emacs
internals that assume there's always a current buffer that is a live
buffer.

Re: threads and kill-buffer

[Sam Steingold]

> * Eli Zaretskii <ryvm@tah.bet> [2012-09-05 19:35:35 +0300]:
>
>> From: Sam Steingold <sds@gnu.org>
>> Date: Wed, 05 Sep 2012 10:20:47 -0400
>> 
>> > * Eli Zaretskii <ryvm@tah.bet> [2012-09-05 05:53:56 +0300]:
>> >
>> > How about letting kill-buffer succeed, but delay the actual deletion
>> > of the buffer until no thread has it as current, like what Posix
>> > filesystems do with file deletion?
>> 
>> I am not sure that Posix file systems do this "by design" and not as a
>> side effect of an implementation detail.
>
> Why does it matter?  If we implement the same logic for buffers, it
> will surely be by design.

Because this is a bad behavior.
The fact that posix does it, does not prove that it is good.
Why do we need to copy other people's mistakes, especially when those
mistakes might be unintended side effects of other design decisions
(i.e., even its authors would agree that it was a mistake)?

>> It is a certain way to lose data: imagine thread T1 moving data from
>> buffer B1 to buffer B2 and thread T2 killing one of these buffers.
>> Killing B1 is fine, but killing B2 will lose data. Note that being the
>> current buffer of another thread is an insufficient protection because a
>> thread may be writing to several buffers and only one of them may be
>> current to it, so a thread should be able to mark a buffer as used by
>> it, so that an attempt to kill a buffer used by a live thread would
>> result in an exception.
>
> That's an entirely different problem than the one that started this
> thread.  The original problem was that each thread must have a current
> buffer, and what to do when it is deleted by another thread.  That
> problem is not about losing data at all, it's about gobs of Emacs
> internals that assume there's always a current buffer that is a live
> buffer.

Yes, but sometimes it is a good idea to look at a problem in a wider
context.
Specifically, the problem I describe suggests a solution to the problem
you describe: when a thread sets its current buffer, that thread should
be added to that buffer's users list which would prevent all other
threads from killing it.

-- 
Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X 11.0.11103000
http://www.childpsy.net/ http://memri.org http://camera.org http://truepeace.org
http://mideasttruth.com http://pmw.org.il http://iris.org.il
If you do not move, you will not feel the shackles.

Re: threads and kill-buffer

[Eli Zaretskii]

> From: Sam Steingold <sds@gnu.org>
> Date: Wed, 05 Sep 2012 12:50:46 -0400
> 
> > * Eli Zaretskii <ryvm@tah.bet> [2012-09-05 19:35:35 +0300]:
> >
> >> From: Sam Steingold <sds@gnu.org>
> >> Date: Wed, 05 Sep 2012 10:20:47 -0400
> >> 
> >> > * Eli Zaretskii <ryvm@tah.bet> [2012-09-05 05:53:56 +0300]:
> >> >
> >> > How about letting kill-buffer succeed, but delay the actual deletion
> >> > of the buffer until no thread has it as current, like what Posix
> >> > filesystems do with file deletion?
> >> 
> >> I am not sure that Posix file systems do this "by design" and not as a
> >> side effect of an implementation detail.
> >
> > Why does it matter?  If we implement the same logic for buffers, it
> > will surely be by design.
> 
> Because this is a bad behavior.
> The fact that posix does it, does not prove that it is good.

It is used by quite a lot of programs, including GNU Coreutils
(AFAIR).

> Why do we need to copy other people's mistakes, especially when those
> mistakes might be unintended side effects of other design decisions
> (i.e., even its authors would agree that it was a mistake)?

We don't _need_ to copy anything, unless we decide that this paradigm
is useful for the problem at hand.

> > That's an entirely different problem than the one that started this
> > thread.  The original problem was that each thread must have a current
> > buffer, and what to do when it is deleted by another thread.  That
> > problem is not about losing data at all, it's about gobs of Emacs
> > internals that assume there's always a current buffer that is a live
> > buffer.
> 
> Yes, but sometimes it is a good idea to look at a problem in a wider
> context.

But you aren't looking at the same problem.  You are looking at a
different problem, with potentially a quite different solution.

Re: threads and kill-buffer

[Stephen J. Turnbull]

Sam Steingold writes:

 > Because [one thread killing another's buffer] is a bad behavior.

Sure, but you're prosecuting an innocent man, Sam.  The culprit is not
kill-buffer semantics[1], it's threads, which *by design* impose the
responsibility for data safety on the programmer.  If one thread's
code can't destroy another thread's data with kill-buffer, then it can
use erase-buffer or `(progn (overwrite-mode) (goto-char (point-min))
(while (not (eobp)) (insert "DEADBEEF")))'.  More likely, it will just
insert "The boss is an idiot!" in a footnote somewhere in the 100-page
report you're turning in to said boss in 31 minutes. :-)

So threads are dangerous if the programmer is not extremely careful.
This is not news, and hamstringing kill-buffer isn't going to make
them all that much safer.  All we can do here is try to prevent Very
Bad Things from happening to third parties who never operate on that
buffer.  Eg, A starts writing to the buffer, B kills it, it gets GC'd
and reallocated, this time for byte-code running in thread C, and A
starts playing "core wars" with what it thinks is its current buffer.[2]

That's why POSIX file deletion semantics are a good idea.  Not because
they save data from users determined to destroy it, but because they
prevent at least some kinds of collateral damage to other processes or
the OS.

 > The fact that posix does it, does not prove that it is good.

Agreed.  Down With Threads!  :-)


Footnotes: 
[1]  I mean, really.  There's a reason we call it "killing the buffer!"

[2]  Before the thread advocates start stoning me, let me acknowledge
that this scenario is highly unlikely even in the worst case, and
more or less impossible for the kinds of applications normally
proposed for Emacs Lisp threads (eg, asynchronous I/O).  But it's at
least theoretically possible with preemptive threads and Sufficiently
Stupid Code[tm].

Re: threads and kill-buffer

[Tom Tromey]

>>>>> "Stephen" == Stephen J Turnbull <stephen@xemacs.org> writes:

Stephen> Eg, A starts writing to the buffer, B kills it, it gets GC'd
Stephen> and reallocated, this time for byte-code running in thread C,
Stephen> and A starts playing "core wars" with what it thinks is its
Stephen> current buffer.[2]

Stephen> [2]  Before the thread advocates start stoning me, let me acknowledge
Stephen> that this scenario is highly unlikely even in the worst case, and
Stephen> more or less impossible for the kinds of applications normally
Stephen> proposed for Emacs Lisp threads (eg, asynchronous I/O).  But it's at
Stephen> least theoretically possible with preemptive threads and Sufficiently
Stephen> Stupid Code[tm].

No, it really is impossible in the context of Emacs, even with
preemptive threads.  The impossibility lies in the "GCd and reallocated"
step.  If thread A has a handle on the buffer, then it cannot be GCd.

Tom

Re: threads and kill-buffer

[Stephen J. Turnbull]

Tom Tromey writes:

 > No, it really is impossible in the context of Emacs, even with
 > preemptive threads.  The impossibility lies in the "GCd and reallocated"
 > step.  If thread A has a handle on the buffer, then it cannot be GCd.

Ah you're right.  ... but subtract GC, and just have the buffer
reallocated in thread B, and guess what happens?[1]

So I didn't get it right, and you miss things, too.  "Threading is
hard, and then you deadlock."

What was the application that makes general threading so attractive,
again?

I have to wonder if it might not be a better idea to develop ways to
share specified data across processes cheaply, eg, via shared memory.

Footnotes: 
[1]  enlarge_buffer_text() @ buffer.c, ~l.4819.

Re: threads and kill-buffer

[Tom Tromey]

Tom> No, it really is impossible in the context of Emacs, even with
Tom> preemptive threads.  The impossibility lies in the "GCd and reallocated"
Tom> step.  If thread A has a handle on the buffer, then it cannot be GCd.

Stephen> Ah you're right.  ... but subtract GC, and just have the buffer
Stephen> reallocated in thread B, and guess what happens?[1]

Subtracting GC doesn't make sense, since we're only talking about Emacs
here.

Stephen> So I didn't get it right, and you miss things, too.

Sure, I do, but not this:

Stephen> Footnotes: 
Stephen> [1]  enlarge_buffer_text() @ buffer.c, ~l.4819.

Things like this are why the threading implementation is cooperative.

But actually, even that is too strong.  I think the implementation only
has to be cooperative as far as the C code in Emacs is concerned; and
that it can appear to be preemptive from the Lisp VM perspective.  Task
switching during I/O is a step in that direction.

Full preemption at the C level is a harder proposition.  I don't intend
to work on that.

Tom

Re: threads and kill-buffer

[Stephen J. Turnbull]

Tom Tromey writes:

 > Subtracting GC doesn't make sense, since we're only talking about Emacs
 > here.

Subtract GC from the scenario, not from Emacs.  With preemptive
threading (at the C level) the enlarge_buffer_text scenario doesn't
need a GC to occur.

 > But actually, even that is too strong.  I think the implementation
 > only has to be cooperative as far as the C code in Emacs is
 > concerned; and that it can appear to be preemptive from the Lisp VM
 > perspective.

But isn't that pretty horrible from the programmer's perspective?
There will be no obviously atomic operations except for variable
reference and null-ary function calls to primitives written in C.  Any
function that accepts list arguments and wants them to be consistent
is either going to have to copy it, or lock it (and ISTM that locking
a list means locking each cons), or assume that its data can be
changed out from under it at any time (setcar and setcdr, not to
mention nconc and friends).

Re: threads and kill-buffer

[Stefan Monnier]

>> How about letting kill-buffer succeed, but delay the actual deletion
>> of the buffer until no thread has it as current, like what Posix
>> filesystems do with file deletion?
> I am not sure that Posix file systems do this "by design" and not as a
> side effect of an implementation detail.

I highly doubt it was a side effect of implementation.
More specifically, the implementation has to go to extra length to do
this.  And it is also a natural result of the design decision that aimed
at making read&write as lean&mean as possible, pushing the heavy lifting
to `open'.

> At any rate, I don't think this behavior is very useful.

To the extent that you can get virtually the same result by renaming the
file to a well-hidden name, it's pretty useful since it turns this
"special situation" into a situation that client code has to deal
with anyway thus reducing the number of special cases.

Note that for buffers, it's a bit different since a buffer can only have
a single name and that it always has one name, so while you can't
reliably ask for the file-name corresponding to a POSIX file-descriptor,
you can reliably ask for the buffer-name of the current buffer.


        Stefan

Re: threads and kill-buffer

[Tom Tromey]

>>>>> "Sam" == Sam Steingold <sds@gnu.org> writes:

Sam> It is a certain way to lose data: imagine thread T1 moving data from
Sam> buffer B1 to buffer B2 and thread T2 killing one of these buffers.

We may eventually need a mechanism to let a thread lock a buffer.  This
kind of thing is harder than it seems.  For example, in your scenario,
you'd presumably want to lock both buffers -- but then you can easily
end up in deadlocks.

Anyway, as Eli said, this particular question is really just about a
single case, of kill-buffer and current_buffer.

Due to the current implementation, right now threads only present a few
problems not already possible with process filters.  For example, your
scenario is possible already.

I'm not trying to minimize the problems you point out; just to defer
them.  The current_buffer thing came up because it is a new problem
arising from the concurrency changes, and one that I think we need to
solve before the branch can be merged.

I'll send a note soon about the other problems in this category.

Tom

Re: threads and kill-buffer

[Stefan Monnier]

> We may eventually need a mechanism to let a thread lock a buffer.  This
> kind of thing is harder than it seems.

We'd clearly need to distinguish read-lock vs write-lock, yes.


        Stefan

Re: threads and kill-buffer

[Tom Tromey]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

Eli> How about letting kill-buffer succeed, but delay the actual deletion
Eli> of the buffer until no thread has it as current, like what Posix
Eli> filesystems do with file deletion?

I think that's a very interesting idea, thanks.

I have been thinking about it and the only issue I see is that, since it
introduces a new buffer state, we would have to decide what
buffer-live-p returns for a buffer that is killed-but-not-yet-dead.

I suppose we may also want a new predicate to detect this state.

Tom

Re: threads and kill-buffer

[Stephen J. Turnbull]

Tom Tromey writes:

 > I have been thinking about it and the only issue I see is that,
 > since it introduces a new buffer state,

Not one that's visible to Lisp.  Why would it be?

 > we would have to decide what buffer-live-p returns for a buffer
 > that is killed-but-not-yet-dead.

nil.  If a thread calls buffer-live-p, that means it's prepared to
recover if b-l-p returns nil.

I suppose you're thinking that the thread might want to snarf the data
before the memory gets reallocated or something like that, but if
other threads are killing your buffers, who knows what they might be
doing to the contents?  Better to just give up.

Re: threads and kill-buffer

[Tom Tromey]

>> I have been thinking about it and the only issue I see is that,
>> since it introduces a new buffer state,

Stephen> Not one that's visible to Lisp.  Why would it be?

It is visible because in thread T, the buffer is still the current
buffer, but is semi-live.  On the trunk (buffer-live-p (current-buffer))
can never be nil, but under this proposed change, it could be.

Stephen> but if
Stephen> other threads are killing your buffers, who knows what they might be
Stephen> doing to the contents?  Better to just give up.

Yeah, but the question is exactly how to give up.

Tom

Re: threads and kill-buffer

[Eli Zaretskii]

> From: Tom Tromey <tromey@redhat.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> Date: Wed, 05 Sep 2012 13:10:00 -0600
> 
> >> I have been thinking about it and the only issue I see is that,
> >> since it introduces a new buffer state,
> 
> Stephen> Not one that's visible to Lisp.  Why would it be?
> 
> It is visible because in thread T, the buffer is still the current
> buffer, but is semi-live.  On the trunk (buffer-live-p (current-buffer))
> can never be nil, but under this proposed change, it could be.

I think we should try to arrange things in such a way that for a
thread that has this buffer as current everything looks and feels as
if the buffer were not killed at all.

Re: threads and kill-buffer

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > I think we should try to arrange things in such a way that for a
 > thread that has this buffer as current everything looks and feels
 > as if the buffer were not killed at all.

This isn't possible.  If the thread wants to do anything further with
this buffer, it must do so before switching buffers, because the
current (dead) buffer will become inaccessible as soon as it does
switch buffers.

Even to the extent that it is possible, I don't see why it's a good
idea.  Suppose the buffer is being used as (gasp!) a buffer, more
precisely, a message queue.  If the writer is done, and kills the
buffer, the reader wants to detect this after it's done reading, and
quit reading.  Under your semantics (including your proposal that the
KILLED_BUT_NOT_DEAD state not be available to Lisp), it can't.
Similarly, if the reader is done, and kills the buffer, the writer
wants to detect this *before* writing any more, and quit writing.

So here we have two extreme scenarios, one where the thread whose
current buffer has met an untimely death wants to stop immediately,
and another where it wants to finish up what it's doing completely.
It's not hard to imagine cases where a killed current buffer is an
error.  So the code needs to be able to find out the state of a
buffer.

The question is whether it's useful for there to be a third state.  I
don't think so; everything you need to know is encapsulated in the
fact that the buffer is dead (to the rest of Emacs) and current.

Re: threads and kill-buffer

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Tom Tromey <tromey@redhat.com>,
>     emacs-devel@gnu.org
> Date: Thu, 06 Sep 2012 05:48:18 +0900
> 
> If the thread wants to do anything further with this buffer, it must
> do so before switching buffers, because the current (dead) buffer
> will become inaccessible as soon as it does switch buffers.

This is another problem, one that exists today: switching buffers
could kill the buffer we are switching away of (e.g., via some hook).

> Even to the extent that it is possible, I don't see why it's a good
> idea.

Because it keeps the old code in working condition, and it keeps the
programmer sane.

> Suppose the buffer is being used as (gasp!) a buffer, more
> precisely, a message queue.  If the writer is done, and kills the
> buffer, the reader wants to detect this after it's done reading, and
> quit reading.  Under your semantics (including your proposal that the
> KILLED_BUT_NOT_DEAD state not be available to Lisp), it can't.

Why can't it?  The buffer is killed, so it doesn't exist, and the
reader will detect that right away, like it does now.  Maybe I don't
understand the situation you are describing.

> So here we have two extreme scenarios, one where the thread whose
> current buffer has met an untimely death wants to stop immediately,
> and another where it wants to finish up what it's doing completely.
> It's not hard to imagine cases where a killed current buffer is an
> error.  So the code needs to be able to find out the state of a
> buffer.

If you mean that the same buffer is current in both the reader and the
writer, then that's a situation that cannot exist in the current code,
so we don't need to be bothered by it too much, I think.  If and when
this is implemented, people who write Lisp code where the reader and
the writer run in different threads will need to use synchronization
devices other than buffer-live-p.

Re: threads and kill-buffer

[Stephen J. Turnbull]

Eli Zaretskii writes:
 > > From: "Stephen J. Turnbull" <stephen@xemacs.org>

 > > If the thread wants to do anything further with this buffer, it must
 > > do so before switching buffers, because the current (dead) buffer
 > > will become inaccessible as soon as it does switch buffers.
 > 
 > This is another problem, one that exists today: switching buffers
 > could kill the buffer we are switching away of (e.g., via some hook).

No, because there's nothing we can do about such a hook unless we put
it there, and nobody does.  Under the proposed "POSIX deletion"
semantics, however, we can know it's coming, as long as we can detect
that the current buffer is killed.  And we are guaranteed that we can
read it, we can save it, etc.

 > > Even to the extent that it is possible, I don't see why it's a good
 > > idea.
 > 
 > Because it keeps the old code in working condition, and it keeps the
 > programmer sane.

Well, no.  Old code won't work (it will deadlock, interleave writes
that should be sequenced, etc) without being rewritten for threaded
environments, unless it's only ever used single-threaded anyway.  In
that case there's no problem in the first place.  Unless you're
talking about some random other thread killing your buffers, in which
case killing a buffer is only one of the more drastic of nasty things
that other thread could do.

Having a buffer killed out from under my code is hardly the first
thing I'd worry about if I were writing a threaded program.

 > > quit reading.  Under your semantics (including your proposal that the
 > > KILLED_BUT_NOT_DEAD state not be available to Lisp), it can't.
 > 
 > Why can't it?  The buffer is killed, so it doesn't exist, and the
 > reader will detect that right away, like it does now.  Maybe I don't
 > understand the situation you are describing.

I'm describing the semantics of a buffer X killed by thread A as seen
from thread B where it is current, and according to your proposal to
thread B thqe buffer will appear to be live indefinitely.  If thread B
is the reader, it will never detect that the buffer is dead, and will
keep on trying to read from it forever when it could be doing useful
work.

Yes, you could use an alternative synchronization method where thread
A explicitly sends thread B a message "hey, I killed your buffer", but
why make the user do it when "buffer is dying, so let's read what's
left and then get on with our life" will automatically DTRT?

 > If you mean that the same buffer is current in both the reader and the
 > writer, then that's a situation that cannot exist in the current code,

What do you mean by "current code"?  The trunk or Tom's branch?  And
why can't it exist?  Eg,

(defvar read-mark (make-marker))

(defun writer ()
  (set-marker read-mark (point))
  (insert "Hi, Eli!\n")
  (reader))

(defun reader ()
  (mail-to-eli (buffer-substring read-mark (point)))
  (writer))

(writer)

No set-buffers needed.  Everything in the current buffer.

Of course this is an error in Emacs Lisp as it stands, since it will
blow the stack.  But it could run quite a while (as far as your
overful mailbox is concerned, anyway).  And once you've got threads,
instead of mutually recursive cooperative coroutines, you can
implement them as ordinary loops.

You could also remove the recursive calls and just loop:

(while t (writer) (reader))

 > so we don't need to be bothered by it too much, I think.

I don't understand this at all.

Re: threads and kill-buffer

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: tromey@redhat.com,
>     emacs-devel@gnu.org
> Date: Thu, 06 Sep 2012 18:22:01 +0900
> 
> Eli Zaretskii writes:
>  > > From: "Stephen J. Turnbull" <stephen@xemacs.org>
> 
>  > > If the thread wants to do anything further with this buffer, it must
>  > > do so before switching buffers, because the current (dead) buffer
>  > > will become inaccessible as soon as it does switch buffers.
>  > 
>  > This is another problem, one that exists today: switching buffers
>  > could kill the buffer we are switching away of (e.g., via some hook).
> 
> No, because there's nothing we can do about such a hook unless we put
> it there, and nobody does.  Under the proposed "POSIX deletion"
> semantics, however, we can know it's coming, as long as we can detect
> that the current buffer is killed.  And we are guaranteed that we can
> read it, we can save it, etc.

Again, I was primarily bothered with preserving the existing
semantics, not to allow new features.

>  > > Even to the extent that it is possible, I don't see why it's a good
>  > > idea.
>  > 
>  > Because it keeps the old code in working condition, and it keeps the
>  > programmer sane.
> 
> Well, no.  Old code won't work (it will deadlock, interleave writes
> that should be sequenced, etc) without being rewritten for threaded
> environments, unless it's only ever used single-threaded anyway.

I meant to say that the old code that runs in a single thread will
still work, in the presence of other threads that do unrelated things,
like fetch news group articles.  If the old code is rewritten to take
advantage of threads, it will have to be adapted to this issue as part
of the rewrite.

> Unless you're talking about some random other thread killing your
> buffers

Yes.

> in which case killing a buffer is only one of the more drastic of
> nasty things that other thread could do.

That's the "nasty thing" Tom raised in this discussion, so that's what
I'm talking about.  Other nasty things warrant separate discussions.

> Having a buffer killed out from under my code is hardly the first
> thing I'd worry about if I were writing a threaded program.

And you'd be wrong: lots of low-level internals in Emacs assume
without checking that the current buffer exists and is valid.  E.g.,
redisplay will crash and burn if that somehow became false in the
middle of its work.

> I'm describing the semantics of a buffer X killed by thread A as seen
> from thread B where it is current, and according to your proposal to
> thread B thqe buffer will appear to be live indefinitely.  If thread B
> is the reader, it will never detect that the buffer is dead, and will
> keep on trying to read from it forever when it could be doing useful
> work.
> 
> Yes, you could use an alternative synchronization method where thread
> A explicitly sends thread B a message "hey, I killed your buffer", but
> why make the user do it when "buffer is dying, so let's read what's
> left and then get on with our life" will automatically DTRT?

If that's what is desired, then thread B could switch away from that
buffer, after killing it, and Bob's our uncle.

IOW, you can write your code so that a killed buffer disappears
shortly, if that's what you need.

>  > If you mean that the same buffer is current in both the reader and the
>  > writer, then that's a situation that cannot exist in the current code,
> 
> What do you mean by "current code"?

The code on the trunk.

> (defvar read-mark (make-marker))
> 
> (defun writer ()
>   (set-marker read-mark (point))
>   (insert "Hi, Eli!\n")
>   (reader))
> 
> (defun reader ()
>   (mail-to-eli (buffer-substring read-mark (point)))
>   (writer))
> 
> (writer)
> 
> No set-buffers needed.  Everything in the current buffer.

Invalid programs are not interesting for discussing design decisions.

> You could also remove the recursive calls and just loop:
> 
> (while t (writer) (reader))

And why is this a problem in the context of this discussion?  No
buffer is killed anywhere in sight, AFAICS.

Re: threads and kill-buffer

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > Again, I was primarily bothered with preserving the existing
 > semantics, not to allow new features.

You don't need to do anything to preserve existing semantics of
accessing buffers.  Lisp can already run asynchronously, including
killing the current buffer out from under a running function.  The
feature you propose (and I agree with) allows a thread with a dying
current buffer to do something useful with it before current buffer
changes.  The new semantics is due to your feature.  The question is,
in these new semantics, what is the return value of (buffer-live-p
(current-buffer))?  Returning t is not useful.  Returning nil
preserves existing semantics of code that actually checks for liveness
of the current buffer, and allows code that knows about the new
semantics to do something more useful (if that option is available,
which depends on the code).

 > I meant to say that the old code that runs in a single thread will
 > still work, in the presence of other threads that do unrelated things,
 > like fetch news group articles.

Sure, but it doesn't matter what (buffer-live-p (current-buffer))
returns while current buffer is dying, that old code will still work.

 > > Having a buffer killed out from under my code is hardly the first
 > > thing I'd worry about if I were writing a threaded program.
 > 
 > And you'd be wrong: lots of low-level internals in Emacs assume
 > without checking that the current buffer exists and is valid.  E.g.,
 > redisplay will crash and burn if that somehow became false in the
 > middle of its work.

Yeah, and this just isn't going to happen in correct programs.  The
risk is low except in experimental or one-off code, and even there I
only kill buffers that I've created.  Do you really do

    (let ((bl (buffer-list)))
      (kill-buffer (nth (random (length bl)) bl)))

so frequently?  Yes, it's something to worry about in the internals,
but (a) at this point in time it's relatively low priority vs getting
the semantics of threading right and (b) your proposal fixes that.

 > If that's what is desired, then thread B could switch away from that
 > buffer, after killing it, and Bob's our uncle.

In the scenario we're concerned about (defective code that kills
somebody else's buffer), that's not what is desired.  You're
completely missing the point.  This entire discussion is "Is there a
more useful return value than t for (buffer-live-p (current-buffer))
when we are surprised by the current buffer being killed by another
thread?"  (Assuming that the current buffer is preserved for our
thread until it switches away.)  

 > > You could also remove the recursive calls and just loop:
 > > 
 > > (while t (writer) (reader))
 > 
 > And why is this a problem in the context of this discussion?  No
 > buffer is killed anywhere in sight, AFAICS.

But that's the *whole* point of this discussion!  *The buffer killing
takes place out of sight.*

Re: threads and kill-buffer

[Stephen J. Turnbull]

Tom Tromey writes:
 > >> I have been thinking about it and the only issue I see is that,
 > >> since it introduces a new buffer state,
 > 
 > Stephen> Not one that's visible to Lisp.  Why would it be?
 > 
 > It is visible because in thread T, the buffer is still the current
 > buffer, but is semi-live.  On the trunk (buffer-live-p (current-buffer))
 > can never be nil, but under this proposed change, it could be.

Yeah, well, threads mess with your mind.  What else is new?  I mean,
on the trunk (buffer-live-p (current-buffer)) can't return
'killed-but-not-dead, either.  Why would it start doing so now? ;-)

My point is that at this point the thread is clearly not manipulating
the buffer, so buffer-live-p can safely change the current buffer (in
the sense that it won't mess up Emacs's internal structures, although
it might play havoc with user data).[1]  However, it may as well return
nil which tells the thread that what it thinks is the current buffer
is dead.

Of course, you could introduce the notion of object owner, and
disallow any thread but the object's owner from killing/deleting an
object.  But that still isn't going to save you from all the other
nefarious things that other threads can do to your objects.

 > Yeah, but the question is exactly how to give up.

Concurrently.

Footnotes: 
[1]  In practice I would expect that actually the Lisp engine does
this at some point.

Re: threads and kill-buffer

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,
>     emacs-devel@gnu.org
> Date: Thu, 06 Sep 2012 04:00:19 +0900
> 
>  > we would have to decide what buffer-live-p returns for a buffer
>  > that is killed-but-not-yet-dead.
> 
> nil.

I think it should return non-nil to the thread(s) that have that
buffer as current, and nil to all the rest.

If we don't pretend the buffer is still there and in perfectly good
state to the threads that have the buffer as current, then all this
method breaks down.

Re: threads and kill-buffer

[Eli Zaretskii]

> From: Tom Tromey <tromey@redhat.com>
> Cc: emacs-devel@gnu.org
> Date: Wed, 05 Sep 2012 12:20:38 -0600
> 
> I suppose we may also want a new predicate to detect this state.

At least internally, yes.  Not sure we should expose that to Lisp.

Re: threads and kill-buffer

[Stefan Monnier]

>> I suppose we may also want a new predicate to detect this state.
> At least internally, yes.  Not sure we should expose that to Lisp.

Can someone explain to me why don't want to make kill-buffer signal an error?


        Stefan

Re: threads and kill-buffer

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Tom Tromey <tromey@redhat.com>,  emacs-devel@gnu.org
> Date: Wed, 05 Sep 2012 22:28:19 -0400
> 
> >> I suppose we may also want a new predicate to detect this state.
> > At least internally, yes.  Not sure we should expose that to Lisp.
> 
> Can someone explain to me why don't want to make kill-buffer signal an error?

Because, if we find a solution that doesn't signal an error, it will
be better: existing code will still work.

Re: threads and kill-buffer

[Stefan Monnier]

>> >> I suppose we may also want a new predicate to detect this state.
>> > At least internally, yes.  Not sure we should expose that to Lisp.
>> Can someone explain to me why don't want to make kill-buffer signal
>> an error?
> Because, if we find a solution that doesn't signal an error, it will
> be better: existing code will still work.

I wonder/doubt if there is any existing working code which would
be affected.


        Stefan

Re: threads and kill-buffer

[SAKURAI Masashi]

Hi Tom,

> Suppose thread T has buffer B as its current buffer, and then some other
> thread evaluates (kill-buffer B).  What should happen in thread T?
> 
> I thought of a few options, but I'm not really sure which is best.

In java, some appropriate exception is thrown, such as
InterruptedException, ConcurrentModificationException and so on.

Preventing the exceptions, some thread controlling is also needed,
such as monitor, semaphore, lock and so on.


> Another problem with this approach is just that it is hard to implement.
> I see 1085 references to 'current_buffer' in the tree, and presumably
> I'd have to examine each one...

I think that this discussion is very important and the first step for
introducing the Emacs thread support. Before implementation, I would
hope that the behavior and specifications for the language (VM) and
library level should be discussed and defined well.
If it has been done, I would apologize for my meddler comment.

Though I have worked on development for server and client applications
with Java, C and Ruby, I feel the concurrency of the design and
programing is very difficult.

This book is introductory and very practical, even in Java.

Amazon.com: Java Concurrency in Practice:
Brian Goetz, Tim Peierls, Joshua Bloch, Joseph Bowbeer, David Holmes, Doug Lea: Books
http://www.amazon.com/Java-Concurrency-Practice-Brian-Goetz/dp/0321349601

In addition to the book, there are many good books for the concurrency design.
The design and implementation in the other languages also may be helpful.


--
SAKURAI, Masashi (family, given)
m.sakurai@kiwanami.net

Re: threads and kill-buffer

[Dmitry Antipov]

On 09/05/2012 12:36 AM, Tom Tromey wrote:

> Suppose thread T has buffer B as its current buffer, and then some other
> thread evaluates (kill-buffer B).  What should happen in thread T?

(assuming other thread is P) T should continue editing, but P marks B
as no longer available to set as current (so B can't be used as current
in another thread); when T switches from B to another buffer, B is really
killed.

> Another problem with this approach is just that it is hard to implement.
> I see 1085 references to 'current_buffer' in the tree, and presumably
> I'd have to examine each one...

Why not just define current_buffer to something like:

#define current_buffer current_thread ()->data->current_buffer

and see what happens?

Dmitry

Re: threads and kill-buffer

[martin rudalics]

 >> Suppose thread T has buffer B as its current buffer, and then some other
 >> thread evaluates (kill-buffer B).  What should happen in thread T?
 >
 > (assuming other thread is P) T should continue editing,

... with B not current?

 > but P marks B
 > as no longer available to set as current (so B can't be used as current
 > in another thread); when T switches from B to another buffer,

... and what if this buffer was meanwhile revived by another thread?

 > B is really
 > killed.

Unless we come up with a robust set of invariants to handle this and
similar scenarios, this war is lost.

martin

Re: threads and kill-buffer

[Eli Zaretskii]

> Date: Wed, 05 Sep 2012 11:44:11 +0200
> From: martin rudalics <rudalics@gmx.at>
> Cc: Tom Tromey <tromey@redhat.com>, Emacs discussions <emacs-devel@gnu.org>
> 
>  > but P marks B
>  > as no longer available to set as current (so B can't be used as current
>  > in another thread); when T switches from B to another buffer,
> 
> ... and what if this buffer was meanwhile revived by another thread?

No other thread can revive that buffer, because the buffer is
invisible to any buffer but those who have it as current.

The "revived" buffer will be a different buffer, perhaps with the same
name.

Re: threads and kill-buffer

[martin rudalics]

 >> ... and what if this buffer was meanwhile revived by another thread?
 >
 > No other thread can revive that buffer, because the buffer is
 > invisible to any buffer but those who have it as current.
 >
 > The "revived" buffer will be a different buffer, perhaps with the same
 > name.

How could that work with buffers like *Help* or *Backtrace* which are
revived all the time?  Do you want to clone them?

martin

Re: threads and kill-buffer

[Eli Zaretskii]

> Date: Thu, 06 Sep 2012 09:19:36 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: tromey@redhat.com, dmantipov@yandex.ru, emacs-devel@gnu.org
> 
>  >> ... and what if this buffer was meanwhile revived by another thread?
>  >
>  > No other thread can revive that buffer, because the buffer is
>  > invisible to any buffer but those who have it as current.
>  >
>  > The "revived" buffer will be a different buffer, perhaps with the same
>  > name.
> 
> How could that work with buffers like *Help* or *Backtrace* which are
> revived all the time?  Do you want to clone them?

I don't understand the problem that bothers you.  Please elaborate.

Re: threads and kill-buffer

[martin rudalics]

 >>  >> ... and what if this buffer was meanwhile revived by another thread?
 >>  >
 >>  > No other thread can revive that buffer, because the buffer is
 >>  > invisible to any buffer but those who have it as current.
 >>  >
 >>  > The "revived" buffer will be a different buffer, perhaps with the same
 >>  > name.
 >>
 >> How could that work with buffers like *Help* or *Backtrace* which are
 >> revived all the time?  Do you want to clone them?
 >
 > I don't understand the problem that bothers you.  Please elaborate.

Suppose *Backtrace* is current in thread A and gets killed by thread B.
Before making another buffer current for A, a debugger buffer must be
revived for B.  Would that be a different buffer from the *Backtrace*
seen in A?

martin

Re: threads and kill-buffer

[Eli Zaretskii]

> Date: Thu, 06 Sep 2012 16:41:39 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: tromey@redhat.com, dmantipov@yandex.ru, emacs-devel@gnu.org
> 
> Suppose *Backtrace* is current in thread A and gets killed by thread B.

Why does it get killed?

> Before making another buffer current for A, a debugger buffer must be
> revived for B.

Why is there a need to make "another buffer current for A"?.  And why
"a debugger buffer must be revived for B"?  The "must" part is
particularly confusing.

> Would that be a different buffer from the *Backtrace* seen in A?

Yes.

Re: threads and kill-buffer

[martin rudalics]

 >> Suppose *Backtrace* is current in thread A and gets killed by thread B.
 >
 > Why does it get killed?

Because killing a buffer was part of the OP's scenario.  In the case at
hand suppose it's killed because the user quits the debugger.

 >> Before making another buffer current for A, a debugger buffer must be
 >> revived for B.
 >
 > Why is there a need to make "another buffer current for A"?.  And why
 > "a debugger buffer must be revived for B"?  The "must" part is
 > particularly confusing.

Because of a bug in thread B causing it to enter the debugger.

 >> Would that be a different buffer from the *Backtrace* seen in A?
 >
 > Yes.

After trying to understand the internals of `debug' in the past days,
I'd be curious how this could be done.

martin

Re: threads and kill-buffer

[Stefan Monnier]

>>> Would that be a different buffer from the *Backtrace* seen in A?
>> Yes.
> After trying to understand the internals of `debug' in the past days,
> I'd be curious how this could be done.

Why?  When you killed the first *Backtrace* buffer, it lost its name, so
when B pops a new backtrace, it creates a new *Backtrace* buffer.


        Stefan

Re: threads and kill-buffer

[martin rudalics]

 > Why?  When you killed the first *Backtrace* buffer, it lost its name, so
 > when B pops a new backtrace, it creates a new *Backtrace* buffer.

And if, at the time it gets killed, *Backtrace* appears in a window
managed by thread A what do I see after it has been killed?

martin

Re: threads and kill-buffer

[Eli Zaretskii]

> Date: Thu, 06 Sep 2012 19:37:10 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: Eli Zaretskii <eliz@gnu.org>, tromey@redhat.com, dmantipov@yandex.ru, 
>  emacs-devel@gnu.org
> 
>  > Why?  When you killed the first *Backtrace* buffer, it lost its name, so
>  > when B pops a new backtrace, it creates a new *Backtrace* buffer.
> 
> And if, at the time it gets killed, *Backtrace* appears in a window
> managed by thread A what do I see after it has been killed?

*Backtrace*, of course.

And if you are going to ask next about buffer B's backtrace buffer, I
guess we will have to arrange for it to be called *Backtrace*<2> in
this case.

Re: threads and kill-buffer

[Eli Zaretskii]

> Date: Thu, 06 Sep 2012 21:22:14 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: tromey@redhat.com, dmantipov@yandex.ru, monnier@iro.umontreal.ca,
> 	emacs-devel@gnu.org
> 
> And if you are going to ask next about buffer B's backtrace buffer, I
                                         ^^^^^^
s/buffer/thread

> guess we will have to arrange for it to be called *Backtrace*<2> in
> this case.

Re: threads and kill-buffer

[martin rudalics]

 > And if you are going to ask next about buffer B's backtrace buffer, I
 > guess we will have to arrange for it to be called *Backtrace*<2> in
 > this case.

And that would break the old `debug' code and any code that relies on
hard-coded buffer names.  String constants like *Backtrace* or *Help*
are part of the global Emacs state.  Changing that might be profitable
but is certainly more tedious than implementing threads in the first
place.

martin

Re: threads and kill-buffer

[Stefan Monnier]

>> Why?  When you killed the first *Backtrace* buffer, it lost its name, so
>> when B pops a new backtrace, it creates a new *Backtrace* buffer.
> And if, at the time it gets killed, *Backtrace* appears in a window
> managed by thread A what do I see after it has been killed?

Some other buffer: a window cannot display a killed buffer.


        Stefan

Re: threads and kill-buffer

[martin rudalics]

 >> And if, at the time it gets killed, *Backtrace* appears in a window
 >> managed by thread A what do I see after it has been killed?
 >
 > Some other buffer: a window cannot display a killed buffer.

So killing a buffer will change the current buffer when it's displayed
in a window and leave it unchanged otherwise.  Would we really like such
behavior?

martin

Re: threads and kill-buffer

[Stefan Monnier]

>>> And if, at the time it gets killed, *Backtrace* appears in a window
>>> managed by thread A what do I see after it has been killed?
>> Some other buffer: a window cannot display a killed buffer.
> So killing a buffer will change the current buffer when it's displayed
> in a window and leave it unchanged otherwise.  Would we really like such
> behavior?

No, killing a buffer will undisplay it from wherever it is displayed
(as is the case now).  But this has no influence of what
`current-buffer' is.

IOW: currently kill-buffer does:
1- undisplay the buffer.
2- change current-buffer if needed.
3- set buffer-name to nil (and remove from buffer-list).
4- throw away some internal data (like the buffer's text).

So one way to handle the problem with threading is that kill-buffer
doesn't do 2 and 4 any more, instead 4 is delayed to the moment when the
buffer is not current anywhere any more (could be detected by the GC,
for example).


        Stefan

Re: threads and kill-buffer

[martin rudalics]

 >> So killing a buffer will change the current buffer when it's displayed
 >> in a window and leave it unchanged otherwise.  Would we really like such
 >> behavior?
 >
 > No, killing a buffer will undisplay it from wherever it is displayed
 > (as is the case now).  But this has no influence of what
 > `current-buffer' is.

Undisplaying changes the current buffer, provided it's displayed in the
selected window, with the next command.

martin

Re: threads and kill-buffer

[Stefan Monnier]

>>> So killing a buffer will change the current buffer when it's displayed
>>> in a window and leave it unchanged otherwise.  Would we really like such
>>> behavior?
>> No, killing a buffer will undisplay it from wherever it is displayed
>> (as is the case now).  But this has no influence of what
>> `current-buffer' is.
> Undisplaying changes the current buffer, provided it's displayed in the
> selected window, with the next command.

It's not undisplaying that changes the current buffer.  It's the
top-level repl that changes the current-buffer to the buffer displayed
in the selected-window at the beginning of a command.
But this top-level interaction should be completely unaffected by the
issue we're discussing where one thread kill a buffer which another
thread is actively using (i.e. it's not in the repl waiting for the
user).


        Stefan

Re: threads and kill-buffer

[PJ Weisberg]

On Thu, Sep 6, 2012 at 7:41 AM, martin rudalics <rudalics@gmx.at> wrote:
>>>  >> ... and what if this buffer was meanwhile revived by another thread?
>>>  >
>>>  > No other thread can revive that buffer, because the buffer is
>>>  > invisible to any buffer but those who have it as current.
>>>  >
>>>  > The "revived" buffer will be a different buffer, perhaps with the same
>>>  > name.
>>>
>>> How could that work with buffers like *Help* or *Backtrace* which are
>>> revived all the time?  Do you want to clone them?
>>
>> I don't understand the problem that bothers you.  Please elaborate.
>
> Suppose *Backtrace* is current in thread A and gets killed by thread B.
> Before making another buffer current for A, a debugger buffer must be
> revived for B.  Would that be a different buffer from the *Backtrace*
> seen in A?

A more interesting question is, what does (get-buffer "*Backtrace*")
return in thread A?

I can think of a few commands in Magit that would not behave properly
if (get-buffer-create some-string) returned a buffer that couldn't be
displayed after the command finished.  (Though I'm having a much
harder time imagining how one of those functions could be called with
a current-buffer that's in the process of being killed.)

-PJ

Gehm's Corollary to Clark's Law: Any technology distinguishable from
magic is insufficiently advanced.

Re: threads and kill-buffer

[Eli Zaretskii]

> Date: Thu, 6 Sep 2012 13:49:29 -0700
> From: PJ Weisberg <pj@irregularexpressions.net>
> Cc: Eli Zaretskii <eliz@gnu.org>, tromey@redhat.com, dmantipov@yandex.ru, 
> 	emacs-devel@gnu.org
> 
> A more interesting question is, what does (get-buffer "*Backtrace*")
> return in thread A?

As long as A has *Backtrace* as its current-buffer, get-buffer will
return it.

> I can think of a few commands in Magit that would not behave properly
> if (get-buffer-create some-string) returned a buffer that couldn't be
> displayed after the command finished.

What significance is here to the fact that the command finishes?

Anyway, if the buffer in question is killed, but is needed elsewhere
in Emacs, then that's a bug that needs to be fixed.  We don't need to
come up with a design that works regardless of bugs, do we?

> (Though I'm having a much harder time imagining how one of those
> functions could be called with a current-buffer that's in the
> process of being killed.)

Exactly.

Re: threads and kill-buffer

[PJ Weisberg]

On Thu, Sep 6, 2012 at 10:52 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Thu, 6 Sep 2012 13:49:29 -0700
>> From: PJ Weisberg <pj@irregularexpressions.net>
>> Cc: Eli Zaretskii <eliz@gnu.org>, tromey@redhat.com, dmantipov@yandex.ru,
>>       emacs-devel@gnu.org
>>
>> A more interesting question is, what does (get-buffer "*Backtrace*")
>> return in thread A?
>
> As long as A has *Backtrace* as its current-buffer, get-buffer will
> return it.
>
>> I can think of a few commands in Magit that would not behave properly
>> if (get-buffer-create some-string) returned a buffer that couldn't be
>> displayed after the command finished.
>
> What significance is here to the fact that the command finishes?

I was thinking of a command that populates a buffer like
"*magit-log*"(creating it if necessary), and then leaves the user *in*
that buffer so he can interact with it further.

> Anyway, if the buffer in question is killed, but is needed elsewhere
> in Emacs, then that's a bug that needs to be fixed.  We don't need to
> come up with a design that works regardless of bugs, do we?

No, I was just brainstorming things that used to be safe assumptions
(`get-buffer-create' returns a displayable buffer) and now might not
be.

-PJ

Gehm's Corollary to Clark's Law: Any technology distinguishable from
magic is insufficiently advanced.

Re: threads and kill-buffer

[Nix]

On 7 Sep 2012, PJ Weisberg spake thusly:
> No, I was just brainstorming things that used to be safe assumptions
> (`get-buffer-create' returns a displayable buffer) and now might not
> be.

It's not safe now. A process sentinel could come along and destroy it
before you have finished running. The only difference here is that
another thread can do the same as well.

-- 
NULL && (void)

Re: threads and kill-buffer

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Cc: Eli Zaretskii <eliz@gnu.org>, tromey@redhat.com, rudalics@gmx.at,
>         dmantipov@yandex.ru, emacs-devel@gnu.org
> Emacs: the road to Hell is paved with extensibility.
> Date: Sat, 08 Sep 2012 15:58:16 +0100
> 
> On 7 Sep 2012, PJ Weisberg spake thusly:
> > No, I was just brainstorming things that used to be safe assumptions
> > (`get-buffer-create' returns a displayable buffer) and now might not
> > be.
> 
> It's not safe now. A process sentinel could come along and destroy it
> before you have finished running.

But a process sentinel can run only when Emacs is idle, right?

Re: threads and kill-buffer

[Stefan Monnier]

BTW, besides the "signal an error" and "mark the buffer as killed but
let the thread keep using it" another option is to treat the thread
using the buffer like we do with sub-processes: either just kill it or
prompt to kill it depending on the moral equivalent of its
process-query-on-exit-flag.


        Stefan

Re: threads and kill-buffer

[Stefan Monnier]

> However, in the long run it would be good to make threads preemptive.
> In this case, it seems to me that switching the current buffer like this
> would be very surprising.  (But maybe still ok?)

If/when we make threads preemptive, I think the answer should be that
set-buffer would lock the buffer and kill-buffer would block until the
buffer is unlocked.
But preemption seems to be pretty far into the future, so I wouldn't
worry too much about it.

> One idea here would be to make various operations on buffers throw an
> exception if the killed buffer were used.  For example, "insert" would
> do this.

I don't think this is a good solution.

> Yet another idea is to make kill-buffer refuse to do this.

Rather than block, we could also make kill-buffer signal an error.
The advantage being that we can also provide this behavior for the
non-preemptive case and I think it would also help debug the problem
Lars points out.


        Stefan

Re: threads and kill-buffer

[Tom Tromey]

Stefan> If/when we make threads preemptive, I think the answer should be that
Stefan> set-buffer would lock the buffer and kill-buffer would block until the
Stefan> buffer is unlocked.

FWIW, an earlier version of the threading patch gave each buffer a lock
and had set-buffer acquire it.  This doesn't work well in practice.  For
example, if you make a new thread, it will not run, since it is created
with the same current buffer as the parent thread.

For fully preemptive threads some kind of buffer lock will be needed,
but I think it would have to be on operations on the buffer, not
set-buffer.

Tom