From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Multithreading, again and again Date: Wed, 19 Oct 2011 17:33:02 -0400 Message-ID: References: <4E82C377.4040007@yandex.ru> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1319059999 23447 80.91.229.12 (19 Oct 2011 21:33:19 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Wed, 19 Oct 2011 21:33:19 +0000 (UTC) Cc: Dmitry Antipov , emacs-devel@gnu.org To: Tom Tromey Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Oct 19 23:33:11 2011 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RGdl8-0003XO-JU for ged-emacs-devel@m.gmane.org; Wed, 19 Oct 2011 23:33:10 +0200 Original-Received: from localhost ([::1]:50547 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RGdl7-0005EY-UW for ged-emacs-devel@m.gmane.org; Wed, 19 Oct 2011 17:33:09 -0400 Original-Received: from eggs.gnu.org ([140.186.70.92]:42391) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RGdl5-0005ES-8u for emacs-devel@gnu.org; Wed, 19 Oct 2011 17:33:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RGdl4-0005C0-2e for emacs-devel@gnu.org; Wed, 19 Oct 2011 17:33:07 -0400 Original-Received: from chene.dit.umontreal.ca ([132.204.246.20]:36282) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RGdl3-0005Bw-Sy for emacs-devel@gnu.org; Wed, 19 Oct 2011 17:33:06 -0400 Original-Received: from faina.iro.umontreal.ca (lechon.iro.umontreal.ca [132.204.27.242]) by chene.dit.umontreal.ca (8.14.1/8.14.1) with ESMTP id p9JLX2rP012545; Wed, 19 Oct 2011 17:33:02 -0400 Original-Received: by faina.iro.umontreal.ca (Postfix, from userid 20848) id CFF10130010; Wed, 19 Oct 2011 17:33:02 -0400 (EDT) In-Reply-To: (Tom Tromey's message of "Wed, 19 Oct 2011 13:00:01 -0600") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.90 (gnu/linux) X-NAI-Spam-Flag: NO X-NAI-Spam-Threshold: 5 X-NAI-Spam-Score: 0 X-NAI-Spam-Rules: 1 Rules triggered RV4015=0 X-NAI-Spam-Version: 2.2.0.9286 : core <4015> : streams <693265> : uri <987346> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 132.204.246.20 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:145353 Archived-At: Stefan> I also recently noticed that sometimes my Gnus is unresponsive for Stefan> a while (waiting for a remote server, maybe in DNS, tho I think not) and Stefan> while C-g does work, other things don't, most importantly clipboard Stefan> requests get delayed so if my Gnus session happens to own the clipboard Stefan> then C-y in another Emacs session hangs until the remote server replies. > I have noticed this as well. Annoyingly, it makes my ERC connections > time out. I guess whatever is pausing is not properly accepting process > output or processing timers or something like that. > I was blaming opportunistic TLS for this but I didn't actually > investigate. Maybe you're right. That deserves a separate bug report. Tom> The other is the multiple keyboard case. I think it should be possible Tom> to have each 'emacsclient -t' run in a separate thread. Stefan> This OTOH is pretty much "the general case" since it needs to deal with Stefan> running Elisp in different threads concurrently (and with recursive Stefan> edits and minibuffers, we want to be able to run one terminal's command Stefan> while some other terminal still has active let-bindings). > I think it is doable, given the context-switching constraints I > mentioned earlier. Yes a "simple" switch-on-yield should be sufficient (i.e. the old concurrency branch should be able to handle it), but that still means that solving such a problem requires handling interaction between the various kinds of Elisp variable bindings (thread-local, buffer-local, ...). > The long-term thing that would be cool is context-switching during QUIT. I'm not sure it would be significantly easier than true concurrency. > I don't think unconstrained context-switching is possible without some > drastic changes to the implementation. Why do you think so? Tom> My plan was to follow the Bordeaux threads API, more or less. Stefan> I'm afraid that using locks is not going to work well for Elisp. Stefan> I have much higher expectations for approaches based on yield Stefan> (i.e. approaches which are inherently safe(r), with the main Stefan> risk being lack of concurrency rather than race-conditions and Stefan> corruption). > Can you expand on this? I'm not sure what kind of expansion you'd like to see: if you forget a lock you can get data corruption, whereas if you forget a yield you may just block a bit more often than desired (but still less often than in Emacs-23). Tom> I think right now you can do things like: Tom> (let ((some-binding)) Tom> (while (condition) Tom> (sit-for 0))) Tom> ... and expect that some-binding will be visible to the process filters. Stefan> Yes, that's a code pattern that doesn't interact well with Stefan> threads. We'll need to adjust/fix those cases one by one, I Stefan> think. > I was thinking of constraining a process to a particular thread by > default, then letting code set the process-thread to nil to mean that it > doesn't care where it is run. That might work, yes. Good idea. > But, if you think it is simpler to just fix the code, then I guess we > could drop this. My general approach has been to be conservative about > changes to elisp semantics. Indeed if we can avoid having to fix the code, it's usually better (the "usually" is for the case where compatibility has significant adverse effects on new clean code and the incompatibility is minor). Stefan