From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Christopher Allan Webber Newsgroups: gmane.emacs.devel Subject: Re: Concurrency, again Date: Thu, 27 Oct 2016 15:55:02 -0500 Message-ID: <87r3717brt.fsf@dustycloud.org> References: <87wq97i78i.fsf@earlgrey.lan> <86k2dk77w6.fsf@molnjunk.nocrew.org> <9D64B8EA-DB52-413D-AE6A-264416C391F3@iotcl.com> <83int1g0s5.fsf@gnu.org> <83twckekqq.fsf@gnu.org> <83funkwfzf.fsf@gnu.org> <87k2cwe4wl.fsf@jupiter.lan> <8360odu2gp.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1477601758 25760 195.159.176.226 (27 Oct 2016 20:55:58 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 27 Oct 2016 20:55:58 +0000 (UTC) User-Agent: mu4e 0.9.16; emacs 25.1.1 Cc: Eli Zaretskii , stefan.huchler@mail.de, Philipp Stephani , emacs-devel@gnu.org To: Daniel Colascione Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Oct 27 22:55:54 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bzriD-0005DN-Mf for ged-emacs-devel@m.gmane.org; Thu, 27 Oct 2016 22:55:45 +0200 Original-Received: from localhost ([::1]:44569 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzriG-00019P-9S for ged-emacs-devel@m.gmane.org; Thu, 27 Oct 2016 16:55:48 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:56386) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzrhg-0000rz-E1 for emacs-devel@gnu.org; Thu, 27 Oct 2016 16:55:13 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bzrhe-0006nu-SP for emacs-devel@gnu.org; Thu, 27 Oct 2016 16:55:12 -0400 Original-Received: from dustycloud.org ([50.116.34.160]:51762) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bzrhb-0006fg-6t; Thu, 27 Oct 2016 16:55:07 -0400 Original-Received: from oolong (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id DAFED266D7; Thu, 27 Oct 2016 16:55:02 -0400 (EDT) In-reply-to: X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 50.116.34.160 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208894 Archived-At: Daniel Colascione writes: > On 10/27/2016 10:27 AM, Eli Zaretskii wrote: >>> From: Philipp Stephani >>> Date: Tue, 25 Oct 2016 23:28:51 +0000 >>> >>> I've pushed the experimental branch to 'concurrency-libtask'. It's essentially a simple wrapper around libtask, >>> which implements CSP based on setcontext. I've also implemented a Windows equivalent based on >>> Windows native fibers, but haven't tried that yet. >> >> Thanks. >> >> Could you perhaps summarize the relative advantages and disadvantages >> of the two concurrency branches? Your branch doesn't have any >> documentation besides doc strings of the new primitives, so it's not >> easy to grasp the high-level picture by looking at the details. >> >> One issue that bothers me is whether it's wise to use libtask here, >> because the changes you did there seem to imply that we will have to >> maintain the library (which is pretty low-level stuff) as part of >> Emacs. Isn't using system threads better? > > Agreed on system threads vs libtask. Fibers of the sort libtask provides > (similar to GNU Pth) have some claimed advantages in efficiency in large > programs with lots of concurrent tasks, but for us, I vote for > Python-style use of OS threads with a Python-style GIL that we can > release to do long-running computations and recover from parallelism. Python's GIL + threading is probably not the best model to work off of... while the GIL is largely misunderstood, there have been a lot of challenges with getting this model to work nice. At one point, it turned out that the GIL + threads would result in constant CPU thrashing. This isn't the case any more, but it was once, and was a challenging thing to fix: http://dabeaz.com/python/UnderstandingGIL.pdf Even now, my understanding is that Python does a ton of context switching, and while it's better than it was, you don't really get a lot of the performance you would over threads being used in other systems, because it's hard for Python to really run things concurrently. Thus the rise of things like asyncio, where instead of parallelizing, you break things into a lot of coroutines which don't block on any IO. Other mechanisms like green threading + coroutines for most IO bound async stuff and, in the case that you need CPU bound stuff to be optimized, something that spawns independent processes (or okay, maybe threads) that communicate with message passing is probably much better than a GIL + system threads route, I'd think?