From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Ken Raeburn Newsgroups: gmane.emacs.devel Subject: Re: Concurrency, again Date: Thu, 20 Oct 2016 02:08:14 -0400 Message-ID: <9DABFA8E-064D-4647-A4FE-0041C8B0051C@raeburn.org> References: <87wq97i78i.fsf@earlgrey.lan> <86k2dk77w6.fsf@molnjunk.nocrew.org> <9D64B8EA-DB52-413D-AE6A-264416C391F3@iotcl.com> <83int1g0s5.fsf@gnu.org> <83twckekqq.fsf@gnu.org> <874m4aic0g.fsf@tromey.com> <7D150317-7A01-464D-8352-942631A3116B@raeburn.org> <8337juxb8h.fsf@gnu.org> <31A629C9-7C3B-4B5D-A5B5-38F556C4E064@raeburn.org> <83wph6vt0f.fsf@gnu.org> <646C7DAF-F7AB-48C2-AFDF-6881D2990617@raeburn.org> <83mvi0v9f2.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: blaine.gmane.org 1476943755 6626 195.159.176.226 (20 Oct 2016 06:09:15 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 20 Oct 2016 06:09:15 +0000 (UTC) Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Oct 20 08:09:11 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bx6X7-0007A4-CW for ged-emacs-devel@m.gmane.org; Thu, 20 Oct 2016 08:08:53 +0200 Original-Received: from localhost ([::1]:52617 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bx6X8-0005jj-CQ for ged-emacs-devel@m.gmane.org; Thu, 20 Oct 2016 02:08:54 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48636) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bx6Wb-0005je-Bm for emacs-devel@gnu.org; Thu, 20 Oct 2016 02:08:22 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bx6WY-00050Q-6y for emacs-devel@gnu.org; Thu, 20 Oct 2016 02:08:21 -0400 Original-Received: from mail-qt0-x241.google.com ([2607:f8b0:400d:c0d::241]:32855) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1bx6WY-0004zZ-1V for emacs-devel@gnu.org; Thu, 20 Oct 2016 02:08:18 -0400 Original-Received: by mail-qt0-x241.google.com with SMTP id m5so1662770qtb.0 for ; Wed, 19 Oct 2016 23:08:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raeburn-org.20150623.gappssmtp.com; s=20150623; h=subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=IRQJmF64Jvy18MnECDDYDswrgPAqOF7biy90JtOIty8=; b=MlwX9tmzdyndBL0p9+MZHpSEoDGeeBhzcCPBDQHveNx0QIOOiwnBaGvdlZX/D7PpRw e4HrUT4H8vD7hNWgehag5xzmMQ8ZylOfea4rN4v/AOqH/lWIFhAFFAqkbw33uNYxhoPN iX4Nzhpzo9ZcxIL5Dv48RiqLflAtr92hICOrjLQ2IbDv82AM7RbMamE0hZrBhhBbqbQX 7vPKgGpsdy3RtEPpsfphXsps8yG7XQ8kT+oU4GwgQZNKy+U2KE+MY+cXyaxK3X8u9RTR Umy//YRcm7o1i0UYOvmy6FVihUWMipevsvzHtCaCs/BkPGxmwV0B0EPkb0wdMdnDhwDq 6K7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=IRQJmF64Jvy18MnECDDYDswrgPAqOF7biy90JtOIty8=; b=R4G408pHmpc6rh00imTwvCZeEH+uutwnbXE0lUPAlhl0ozpKVMTrGK1aGkMyvR5Jkm PR9D83pqd4k5GjP+KjjG2+MxR20lufBqvSi2Em8UCAEgvr4ApXn5oPKupxtzYdqMXLLY ERIuyVtji+DiogZ2S3Y1YVBNH0iRN2+JkHD6bd/9L8u2GisqwJRaUCBa5byPv5KLA9QQ bUlrTQh63+GlImzS6hGyXHOPC0KAWWZ+lK+Jtsv+xkZTLU6aHd3DuX/FnEVq6Mshtg+R sOdaiOlGEbYTyPT/Yl7fiv8SAhI7imwXYa2s8PScnpIbL+wUA7RCUBKSYv5csMXlWPk3 8bog== X-Gm-Message-State: AA6/9RneSFfdKHW7JHarZveAs29DQ7OC6AaYiHB8VP4qN5w+8Y2OD/cdWmwXLsD5kFdjSg== X-Received: by 10.200.50.78 with SMTP id y14mr9328181qta.57.1476943697020; Wed, 19 Oct 2016 23:08:17 -0700 (PDT) Original-Received: from [192.168.23.52] (c-50-138-183-136.hsd1.ma.comcast.net. [50.138.183.136]) by smtp.gmail.com with ESMTPSA id b12sm22282144qta.4.2016.10.19.23.08.15 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 19 Oct 2016 23:08:16 -0700 (PDT) In-Reply-To: <83mvi0v9f2.fsf@gnu.org> X-Mailer: Apple Mail (2.3124) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2607:f8b0:400d:c0d::241 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:208519 Archived-At: On Oct 19, 2016, at 07:57, Eli Zaretskii wrote: >>> That should be easy: since a subprocess is locked to a single = thread, >>=20 >> by default, but if that thread exits, that lock disappears >=20 > And the process gets locked to some other thread. Not that I can see, unless you explicitly call set-process-thread; = update_processes_for_thread_death sets the process thread field to Qnil. >=20 >>=20 >>> SIGCHLD should be delivered to that thread. If we don't have that >>> already, we should add that, it doesn't sound hard, given the >>> infrastructure we already have (deliver_thread_signal etc.). >>=20 >> It=E2=80=99s not completely trivial. [=E2=80=A6] >=20 > So you are saying this problem was never encountered before in any > other program out there, and doesn't already have a solution? I find > that hard to believe. Not at all, just that it may take some work. Though, I would expect = that most such programs we might go look at are committed to a = multi-threaded design, and aren=E2=80=99t written to support both = multi-threaded and single-threaded environments. >=20 >> On the other hand, perhaps we can create one special thread to do all = the waitpid() calls and pass info to the Lisp-running threads. >=20 > Sounds like an unnecessary complication, but if that's how others = solve > this problem, so shall we. I don=E2=80=99t know. Aside from checking a few man pages while writing = the previous email, I haven=E2=80=99t researched it. I think it=E2=80=99s= probably the way I=E2=80=99d be most inclined to do such a thing, if = the ability to use such helper threads could be assumed; it can=E2=80=99t = be in this case. I just took a quick look at some bits of a Thunderbird source tree. In = the Netscape Portable Runtime = (thunderbird-49.0b1/mozilla/nsprpub/pr/src/md/unix/uxproces.c) they=E2=80=99= ve got a =E2=80=9Cwaitpid daemon thread=E2=80=9D; whichever thread gets = SIGCHLD delivered to it writes a byte to a pipe to wake up that daemon = thread, which then calls waitpid until there are no more child process = status updates to process. It updates global (mutex-protected) data = structures and optionally uses a condition variable to notify some other = thread of the process change. >=20 >>>> It=E2=80=99s easy enough to disable stack overflow checking when = enabling thread support. >>>=20 >>> Or add some simple code in the stack overflow handler to check if we >>> are in the main thread, and if not, punt (i.e. crash). >>>=20 >>>> If only one thread is allowed into the image processing code at a = time (i.e., don=E2=80=99t release the global lock for that code) then = that=E2=80=99s probably fine for now, and there=E2=80=99s probably other = state there that different threads shouldn=E2=80=99t be mucking around = with in parallel. >>>=20 >>> Redisplay runs in the main thread anyway, right? If so, there's no >>> problem. >>=20 >> If some random thread calls (redisplay) or (sit-for =E2=80=A6)? I = think it=E2=80=99ll run in whichever Lisp-running thread triggers it. = But, it=E2=80=99ll be the one holding the lock. >=20 > No, sit-for causes a thread switch. I believe that=E2=80=99s when it=E2=80=99s checking for input, after = calling redisplay, though. I just tried a test with a breakpoint in redisplay_internal; it can get = called from threads other than the main thread. As far as I know, we = can=E2=80=99t have multiple threads in the redisplay code at the same = time, though. >=20 >>>> The keyboard.c one is the only one I=E2=80=99m a bit concerned = about, in part because I haven=E2=80=99t looked at it. >>>=20 >>> What part(s) of keyboard.c, exactly? >>=20 >> Anything looking at getcjmp; that means read_event_from_main_queue = and read_char. Like I said, I haven=E2=80=99t looked very closely; if = the static storage isn=E2=80=99t ever used across a point where the = global lock could be released to allow a thread switch, it may be fine. >=20 > That should already be solved, or else threads cannot receive keyboard > input safely. I hope so. I haven=E2=80=99t convinced myself one way or the other yet, = though. Ken=