From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Tom Tromey Newsgroups: gmane.emacs.devel Subject: Re: advice needed for multi-threading patch Date: Mon, 21 Sep 2009 15:50:32 -0600 Message-ID: References: Reply-To: Tom Tromey NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1253570230 16677 80.91.229.12 (21 Sep 2009 21:57:10 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 21 Sep 2009 21:57:10 +0000 (UTC) Cc: Emacs development discussions To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Sep 21 23:57:03 2009 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1MpqsZ-00040I-19 for ged-emacs-devel@m.gmane.org; Mon, 21 Sep 2009 23:57:03 +0200 Original-Received: from localhost ([127.0.0.1]:49807 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MpqsY-0003Mm-Hy for ged-emacs-devel@m.gmane.org; Mon, 21 Sep 2009 17:57:02 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1MpqmQ-0008HM-8N for emacs-devel@gnu.org; Mon, 21 Sep 2009 17:50:42 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1MpqmL-0008FZ-AN for emacs-devel@gnu.org; Mon, 21 Sep 2009 17:50:41 -0400 Original-Received: from [199.232.76.173] (port=40269 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MpqmL-0008FS-6b for emacs-devel@gnu.org; Mon, 21 Sep 2009 17:50:37 -0400 Original-Received: from mx1.redhat.com ([209.132.183.28]:47608) by monty-python.gnu.org with esmtp (Exim 4.60) (envelope-from ) id 1MpqmJ-0002S7-Tj for emacs-devel@gnu.org; Mon, 21 Sep 2009 17:50:36 -0400 Original-Received: from int-mx08.intmail.prod.int.phx2.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by mx1.redhat.com (8.13.8/8.13.8) with ESMTP id n8LLoZiC032623; Mon, 21 Sep 2009 17:50:35 -0400 Original-Received: from ns3.rdu.redhat.com (ns3.rdu.redhat.com [10.11.255.199]) by int-mx08.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id n8LLoYMa027195; Mon, 21 Sep 2009 17:50:34 -0400 Original-Received: from opsy.redhat.com (ovpn01.gateway.prod.ext.phx2.redhat.com [10.5.9.1]) by ns3.rdu.redhat.com (8.13.8/8.13.8) with ESMTP id n8LLoWx2020563; Mon, 21 Sep 2009 17:50:33 -0400 Original-Received: by opsy.redhat.com (Postfix, from userid 500) id 75D42378198; Mon, 21 Sep 2009 15:50:32 -0600 (MDT) X-Attribution: Tom In-Reply-To: (Stefan Monnier's message of "Mon, 21 Sep 2009 17:19:42 -0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) X-Scanned-By: MIMEDefang 2.67 on 10.5.11.21 X-detected-operating-system: by monty-python.gnu.org: Genre and OS details not recognized. X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:115502 Archived-At: >>>>> "Stefan" == Stefan Monnier writes: Tom> * Many, but not all, of the 9 kinds of variables that Emacs implements Tom> work properly with threads. In particular, working ones are Tom> defvaralias, objfwd, and ordinary. These all should do the right Tom> thing in all cases. Buffer-local and buffer objfwd mostly work, Tom> because we do buffer locking. (However, I think make-local-variable Tom> doesn't work properly with let-binding and multiple threads.) Not Tom> working on intfwd, boolfwd, keyboard-local and frame-local. Stefan> How do you handle objfwd objects? Do you swap them in&out during Stefan> context switches? We have a script to edit the Emacs source. In the end all DEFUN_LISP variables are defined like: #define Vafter_change_functions *find_variable_location (&impl_Vafter_change_functions) Then, find_variable_location understands Lisp_ThreadLocal binding. A Lisp_ThreadLocal is put into an impl_V* by specbind. This works great as long as the C code follows lispy rules. However, I don't think that is always the case -- there are places doing direct assignment to some of these variables where, really, a per-thread assignment is meant. We don't swap anything in and out during a context switch. I've tried to make all the decisions with an eye on the long term: preemptive threading. Ultimately I'd like to get rid of the global lock and have all threads run free. Tom> * Buffer locking is actually quite a pain. Tom> If you have a background thread locking some buffer, you can't switch Tom> to that buffer. Emacs will just pause. I don't think we allow Tom> interrupting a lock acquisition (clearly a bug). Stefan> Yes, we may have to relax the locking somehow. E.g. we should clearly Stefan> allow to display a buffer in any window, if the buffer is "locked" by Stefan> a thread. I.e. things like switch-to-buffer should not take the Stefan> buffer's lock. Stefan> Of course, if the buffer is locked by a thread, we also need to be Stefan> careful what we do if the user then types a command in it (should we Stefan> wait for the thread to yield or should we signal an erreur?) Giuseppe has some code to let a thread acquire a buffer without locking. But, this is done in an unsafe way; I think it is just a hack to let him experiment with Gnus a bit. Right now when a thread yields it does not release its buffer lock. The only way to do that is to set-buffer to another buffer. One way forward would be to work out the semantics properly, and eliminate buffer locking. Some of the semantics seems tricky. And, the work itself is hard, because it means adding tons of accessor macros and then updating a lot of source code to use them. (I did this, once, for a lot of Emacs, thinking to implement a software write barrier. It is really quite painful.) Tom> * Giuseppe implemented minibuffer exclusion. But really the keyboard Tom> should only be available to one thread at a time. Stefan> You mean "each keyboard", right? Yes. Locking a keyboard will probably have strange effects. There are some things here that I have not thought through; like, what is the right behavior of debug-on-error? Under what conditions would the main thread ever release the keyboard? Tom> Sometimes I wonder whether it would be better to just fork a second Tom> emacs and communicate with it using pipes + princ + read. This uses Tom> more memory but the implementation would be a lot simpler and (I think) Tom> it would suffice for the one case that seems most important: Gnus. Stefan> But you can already do that right now, so it's a different "solution". Yeah. I just wonder why nobody has done it and whether it would not be a better approach. Tom