From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Ken Raeburn Newsgroups: gmane.emacs.devel Subject: Re: Concurrency, again Date: Wed, 19 Oct 2016 06:18:18 -0400 Message-ID: <646C7DAF-F7AB-48C2-AFDF-6881D2990617@raeburn.org> References: <87wq97i78i.fsf@earlgrey.lan> <86k2dk77w6.fsf@molnjunk.nocrew.org> <9D64B8EA-DB52-413D-AE6A-264416C391F3@iotcl.com> <83int1g0s5.fsf@gnu.org> <83twckekqq.fsf@gnu.org> <874m4aic0g.fsf@tromey.com> <7D150317-7A01-464D-8352-942631A3116B@raeburn.org> <8337juxb8h.fsf@gnu.org> <31A629C9-7C3B-4B5D-A5B5-38F556C4E064@raeburn.org> <83wph6vt0f.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1476872617 24775 195.159.176.226 (19 Oct 2016 10:23:37 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 19 Oct 2016 10:23:37 +0000 (UTC) Cc: Emacs-Devel devel To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Oct 19 12:23:29 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bwo1m-0003xs-Ko for ged-emacs-devel@m.gmane.org; Wed, 19 Oct 2016 12:23:18 +0200 Original-Received: from localhost ([::1]:46799 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bwo1o-0007ke-Re for ged-emacs-devel@m.gmane.org; Wed, 19 Oct 2016 06:23:20 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33436) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bwnx2-0005AD-I7 for emacs-devel@gnu.org; Wed, 19 Oct 2016 06:18:26 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bwnwz-0001U7-BD for emacs-devel@gnu.org; Wed, 19 Oct 2016 06:18:24 -0400 Original-Received: from mail-qk0-x22b.google.com ([2607:f8b0:400d:c09::22b]:34413) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1bwnwz-0001Tl-5M for emacs-devel@gnu.org; Wed, 19 Oct 2016 06:18:21 -0400 Original-Received: by mail-qk0-x22b.google.com with SMTP id f128so24962113qkb.1 for ; Wed, 19 Oct 2016 03:18:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raeburn-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=seMasNcWyhGhemgzXoN47j/fQPRcHx3v5LYiD6W7QI8=; b=qaoe4wrnUPkCVo0aW8JXkXLk2kg0qERwXVdRupVYVdhQNXGGT9w+7GCWDqTkiuycfp cgFLYsYmfs53MVESVZ/v6Vt8GRR+OQ2F1tof/7UM00FsBot+bgW+8lx29tyW5ZOvHv0H k9zzi7o7eaHpV7sO0bPikyCZgAsvJHaTxmTBLTuJLFf8su8Gubigu/390Uo7j64hZNY8 1wD5/42lM4LhkF88VIN83i0eA0ZeAnFBrHlFR0JPxsH7tdS8Uwa8Kz35rhKa3g5umX9I xbuH6dl2B3FEo+SwIegfWeHpuIhjkcMdvyocYoCqq22M5plWPzuCZJ70P2GexB7AlMrT GJ+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=seMasNcWyhGhemgzXoN47j/fQPRcHx3v5LYiD6W7QI8=; b=cRU3UE6mM8Y+vvo5QAihPlF/Q2FHvgp78WwhakZhogjufBSmBTS26unXn3UBbUGSSj KYg2aGqyTkRXuSKDPHi+Tm7u6iojdmtJu3u7O/KVcunA70W0zQSP5qX0PF03v32DAUSa Y7HmKtnB0pDriGV6VhCx1wP0l5gVBLNY80NTAO3VH/C4rDwPvQEAiLjW+gaVQQBoe2zi LGmPZt5QxO+B3dZQgLoaEgL2IXuIPCCqNSuKuzUQmJ52pMC9lLo14pm5YykDdwrwZ4p0 R55l1I6sx2sfo21KUktu1ZlaMsvFGUIVsZWv23mMO3r6wEduZmqrnbvkq02EANxCVRWG oqiQ== X-Gm-Message-State: ABUngvcJK34UYmmcb07BN9X3Ml886ET7R9vwUvbzCwMLjB2Cx7nzuvOsy+6tWHGZc4Lacw== X-Received: by 10.55.89.197 with SMTP id n188mr4933516qkb.306.1476872300490; Wed, 19 Oct 2016 03:18:20 -0700 (PDT) Original-Received: from [192.168.23.52] (c-50-138-183-136.hsd1.ma.comcast.net. [50.138.183.136]) by smtp.gmail.com with ESMTPSA id e36sm19004852qtd.42.2016.10.19.03.18.19 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 19 Oct 2016 03:18:20 -0700 (PDT) In-Reply-To: <83wph6vt0f.fsf@gnu.org> X-Mailer: Apple Mail (2.3124) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:400d:c09::22b X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208469 Archived-At: > On Oct 18, 2016, at 06:41, Eli Zaretskii wrote: >=20 >> From: Ken Raeburn >> Date: Tue, 18 Oct 2016 06:08:49 -0400 >> Cc: tom@tromey.com, >> emacs-devel@gnu.org >>=20 >>>> * header inclusion order requirement is weird; can generating one = big header help? >>>=20 >>> I'm not sure I understand what inclusion order is being alluded to >>> here. Can you elaborate? >>=20 >> I think it=E2=80=99s the ordering and recursive inclusion involving = thread.h relative to the other headers. For example lisp.h includes = thread.h which includes sysselect.h which includes lisp.h; thread.h uses = struct vectorlike_header, so it has to be included in lisp.h after that = structure is defined but before struct thread_state gets used; thread.h = also includes regex.h which includes lisp.h. You commented at one = point, =E2=80=9CWe now have an unfortunate situation whereby lisp.h = cannot be included before some of the other headers, due to this.=E2=80=9D= >=20 > This could be solved by moving parts of thread.h into lisp.h, with the > appropriate #ifdef guards. I think Lisp objects should be in lisp.h > anyway, even if they are optional; anything else is confusing. For big, specialized objects like buffers, I think the modularity can = help keep things organized, but where we=E2=80=99ve got a structure with = all the per-thread state from random parts of the program, like = condition handlers and regex state, we don=E2=80=99t have the luxury of = separating things. But even if we move most of thread.h into lisp.h, I = think there may still be mutual recursion between it and the other = headers. Perhaps if the thread state contained struct pointers instead of = structures, we could forward-declare some of the types and not have to = pull in the other headers in such a fashion. >=20 >>>> * one thread per terminal? >>>=20 >>> Why? >>>=20 >>>> * file notifications and such shouldn=E2=80=99t go through same = queue as keyboard events >>>=20 >>> Why? >>=20 >> Stefan=E2=80=99s message: = http://lists.gnu.org/archive/html/emacs-devel/2013-08/msg00755.html >=20 > I don't see that as a critical problem, perhaps because we don't yet > realize how serious it can be. The whole purpose of trying to merge > the concurrency branch is to collect practical experience as to what > should and shouldn't be in this kind of Emacs feature. So I'd tend to > let this be, until we find out we can't, and why. >=20 >>>> * interaction of SIGCHLD handling and threads? >>>=20 >>> Details? >>=20 >> Your message = http://lists.gnu.org/archive/html/emacs-devel/2013-08/msg00738.html = raised questions. If they were ever satisfactorily answered, I = overlooked it when putting together my notes=E2=80=A6. >=20 > That should be easy: since a subprocess is locked to a single thread, by default, but if that thread exits, that lock disappears > SIGCHLD should be delivered to that thread. If we don't have that > already, we should add that, it doesn't sound hard, given the > infrastructure we already have (deliver_thread_signal etc.). It=E2=80=99s not completely trivial. Under POSIX, we get no control = over which thread receives SIGCHLD due to a subprocess exiting, except = if we want certain threads to block receiving the signal completely. We = can use pthread_kill to explicitly send SIGCHLD to a specific thread, if = it=E2=80=99s not blocking the signal, but we don=E2=80=99t know which = thread until we=E2=80=99ve fetched the child=E2=80=99s pid and looked it = up in some data structure; we do that by calling waitpid() but then the = kernel discards the child process=E2=80=99s status info, so the = =E2=80=9Ccorrect=E2=80=9D thread can no longer respond to the signal by = making another waitpid() call to collect the status info. We=E2=80=99d = have to save the info the first time we call waitpid(). But doing it = from within the signal handler could be tricky, because in that context = we=E2=80=99re limited to async-signal-safe functions, and helpful = routines like malloc() and pthread_mutex_lock() aren=E2=80=99t on the = list. On the other hand, perhaps we can create one special thread to do all = the waitpid() calls and pass info to the Lisp-running threads. If = that=E2=80=99s all it=E2=80=99s doing, the non-signal-handler portion of = the thread=E2=80=99s code can loop calling waitpid and locking mutexes = and updating data structures, and the Lisp-running threads can check in = their thread-state structures for messages from the child-reaping = thread. I=E2=80=99m not sure how safe it would be to block SIGCHLD in = threads that might call into the various system UI libraries and such, = but we can probably just use a simple SIGCHLD handler to let any thread = wake the reaper thread, which can then do its waitpid() calls. >=20 >> It=E2=80=99s easy enough to disable stack overflow checking when = enabling thread support. >=20 > Or add some simple code in the stack overflow handler to check if we > are in the main thread, and if not, punt (i.e. crash). >=20 >> If only one thread is allowed into the image processing code at a = time (i.e., don=E2=80=99t release the global lock for that code) then = that=E2=80=99s probably fine for now, and there=E2=80=99s probably other = state there that different threads shouldn=E2=80=99t be mucking around = with in parallel. >=20 > Redisplay runs in the main thread anyway, right? If so, there's no > problem. If some random thread calls (redisplay) or (sit-for =E2=80=A6)? I think = it=E2=80=99ll run in whichever Lisp-running thread triggers it. But, = it=E2=80=99ll be the one holding the lock. >=20 >> The keyboard.c one is the only one I=E2=80=99m a bit concerned about, = in part because I haven=E2=80=99t looked at it. >=20 > What part(s) of keyboard.c, exactly? Anything looking at getcjmp; that means read_event_from_main_queue and = read_char. Like I said, I haven=E2=80=99t looked very closely; if the = static storage isn=E2=80=99t ever used across a point where the global = lock could be released to allow a thread switch, it may be fine.=