From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Tom Tromey Newsgroups: gmane.emacs.devel Subject: Re: advice needed for multi-threading patch Date: Tue, 22 Sep 2009 21:16:07 -0600 Message-ID: References: Reply-To: Tom Tromey NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1253675794 24369 80.91.229.12 (23 Sep 2009 03:16:34 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 23 Sep 2009 03:16:34 +0000 (UTC) Cc: Emacs development discussions To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Sep 23 05:16:26 2009 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1MqILC-0001IV-Ep for ged-emacs-devel@m.gmane.org; Wed, 23 Sep 2009 05:16:26 +0200 Original-Received: from localhost ([127.0.0.1]:55642 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MqILB-0002BR-KX for ged-emacs-devel@m.gmane.org; Tue, 22 Sep 2009 23:16:25 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1MqIL6-00029k-BW for emacs-devel@gnu.org; Tue, 22 Sep 2009 23:16:20 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1MqIL1-00023c-64 for emacs-devel@gnu.org; Tue, 22 Sep 2009 23:16:19 -0400 Original-Received: from [199.232.76.173] (port=33167 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MqIL1-00023U-24 for emacs-devel@gnu.org; Tue, 22 Sep 2009 23:16:15 -0400 Original-Received: from mx1.redhat.com ([209.132.183.28]:55236) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1MqIL0-0006nk-Cm for emacs-devel@gnu.org; Tue, 22 Sep 2009 23:16:14 -0400 Original-Received: from int-mx01.intmail.prod.int.phx2.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id n8N3GAc7004299; Tue, 22 Sep 2009 23:16:10 -0400 Original-Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx01.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id n8N3G9Ig016572; Tue, 22 Sep 2009 23:16:10 -0400 Original-Received: from opsy.redhat.com (ovpn01.gateway.prod.ext.phx2.redhat.com [10.5.9.1]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id n8N3G8SH001359; Tue, 22 Sep 2009 23:16:08 -0400 Original-Received: by opsy.redhat.com (Postfix, from userid 500) id 10E97378183; Tue, 22 Sep 2009 21:16:08 -0600 (MDT) X-Attribution: Tom In-Reply-To: (Stefan Monnier's message of "Tue, 22 Sep 2009 10:24:15 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) X-Scanned-By: MIMEDefang 2.67 on 10.5.11.11 X-detected-operating-system: by monty-python.gnu.org: Genre and OS details not recognized. X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:115537 Archived-At: Tom> This works great as long as the C code follows lispy rules. However, I Tom> don't think that is always the case -- there are places doing direct Tom> assignment to some of these variables where, really, a per-thread Tom> assignment is meant. Stefan> I'm not sure what you mean. Are you still talking about objfwd Stefan> variables? Yeah. What I meant here is that specbind has a little extra code in it to install a Lisp_ThreadLocal object when needed. So, if the C code uses specbind to do let-like binding, everything will work fine. If the C code does not do this, then things will break. For example, I am not certain that Vquit_flag is handled in a "safe" way. It is directly assigned to in a number of places, but those assignments should probably be thread-local. (This isn't the greatest example, maybe, because Vquit_flag is also bound up in keyboard locking and maybe other issues.) Tom> Right now when a thread yields it does not release its buffer lock. Stefan> I think it should, unless some of the buffer-local variables are Stefan> let-bound by the thread. I've been thinking about this a bit. It is trickier than it seems, because a thread can actually have let-bindings for buffer-locals from multiple buffers at the same time. There's an example of this in the elisp manual. (This is also an area the threading code does not handle well yet.) I think this means it does not suffice to keep a simple per-thread count of buffer-local bindings. While thinking about this I realized that (I think) the current code does the wrong thing if one thread let-binds a variable and then another thread calls make-variable-buffer-local on it. Stefan> Maybe another way to look at all these problems is to take an Stefan> "agent" point of view: rather than threads moving around, we Stefan> could consider each keyboard and each buffer as an active object Stefan> (i.e. with its own thread), which communicate among each other. Stefan> I.e. a buffer-thread never leaves its buffer, instead it does an Stefan> RPC to another buffer-thread, or to a keyboard-thread, ... I had considered the agent model -- that, plus a discussion on irc, is actually what lead me to ask about running a second Emacs as a subprocess. It is hard for me to see how this could be done in a compatible way. Right now elisp operates with few constraints; an elisp program can call select-frame, which seems to imply that per-frame or per-keyboard threads can't be done. Tom> Yeah. I just wonder why nobody has done it and whether it would not be Tom> a better approach. Stefan> IIUC people do it all the time, tho not with another Emacs Stefan> process: with an hexl process, a shell process, an openssl Stefan> process, ... Emacs currently is not very good at using Stefan> stdin/stdout so running an inferior Emacs process is poorly Stefan> supported. If I were doing it "for real" I might consider coming up with a higher-bandwidth way to transfer an entire buffer between processes. But, a princ/read approach could be done without modifying Emacs, by having the parent Emacs make a server socket, and then passing the port number to the subprocess as an argument. This avoids the stdin/stdout difficulties. One thing that would help me is having a concrete idea of what subset of features would make this work be actually useful. I mostly implemented it because it was cool, and because Giuseppe's initial patch convinced me that it was possible (before that I'd written it off as totally impractical). Now that a proof of concept works it would be nice to have a concrete goal. Tom