Re: advice needed for multi-threading patch

[Ken Raeburn <raeburn@raeburn.org>]

On Sep 28, 2009, at 20:27, Stefan Monnier wrote:

>> In my view, a let-binding should always be thread-local, and I think
>> buffer-localness shouldn't affect this.  This is enough of a rule to
>> answer your questions -- but it is just one choice, we could make
>> others, I suppose.
>
> So what should happen in the following case:
> Thread A, running in buffer B1, runs the following code:
>
>  (let ((default-directory "/foo"))
>    (with-current-buffer B2 default-directory))
>
> You seem to say that it should return "/foo", but currently it returns
> something else.  Maybe we should try to come up with a run-time test  
> to
> catch such cases in current Lisp code.

I suspect that any sensible approach we come up with is going to  
involve a change in behavior that may affect some existing code.  So  
coming up with ways to detect code that changes meaning is probably  
going to be a good idea.

>  A related case is when a process
> filter or a sentinel is run via accept-process-output: we'd need to be
> careful to make sure the code is run in the same thread as the code  
> that
> called accept-process-output.

Not necessarily; two threads could each be running helper programs in  
subprocesses (or using network connections), setting up let-bound  
variables, and calling accept-process-output, expecting the process  
filters to have access to the let-bound variables.  We might want to,  
by default, bind the filters to run only in the thread that originally  
created the process, so long as it exists.  It could be less  
efficient, but I think it might easily preserve more of the current  
behavior of packages written for single-threaded Emacs, even if they  
get run concurrently (but independently) in multiple threads.  And we  
could make it easy for code to allow other threads to process the  
output too.

>> There are some oddities implied by making buffer-local let-bindings
>> also be thread-specific.  For example, some buffer-locals affect
>> redisplay, so what the user sees will depend on the thread in which
>> redisplay is run.
>
> Redisplay should be run in a completely separate thread (at least
> conceptually).

I think it's less likely, but a package might also let-bind some  
variables that affect the display (e.g., truncate-lines, selective- 
display, show-trailing-whitespace, all of which are buffer-local, or  
anything referenced via mode-line) and then call y-or-n-p or read-file- 
name or some other routine that will trigger redisplay and prompt the  
user for some info that may depend on what's displayed....  Be that as  
it may, I think making it conceptually a separate thread is probably  
the right approach.  And this minor incompatibility is something we  
can test for now, for the most part -- if redisplay examines a Lisp  
variable, it should check for the presence of that variable in specpdl  
and cause a warning to be issued later if it's found.  (Uses of  
":eval" in mode-line-format are probably too expensive to process.)

The other approach that came to my mind was, use the context of the  
thread that's currently taking the keyboard input if there is one, to  
try to approximate the "right thing" for any code already doing what I  
described above.  (If one thread is prompting the user for input, and  
then another thread triggers redisplay but doesn't prompt for input, I  
suspect seeing what's intended by the prompting thread is probably  
more important.)  Of course that would mean that when switching which  
thread "owns" the current keyboard input, you might need to redisplay  
everything because of the change of context, or at least redisplay the  
affected buffers, which could be many of them, depending on the  
behavior of such code as:

  (let ((default-directory foo))
     (with-current buffer buf1
       (let ((default-directory bar))
          (with-current-buffer buf2
             ...repeat ad nauseam...

So... yeah, I think I like the conceptually-separate-thread approach  
better, even if it could break existing code.  I'm not certain we  
couldn't do better in terms of backwards compatibility, but it's  
simple and clean.

Ken