From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.org!.POSTED!not-for-mail
From: Ken Raeburn <raeburn@raeburn.org>
Newsgroups: gmane.emacs.devel
Subject: Re: Concurrency, again
Date: Thu, 20 Oct 2016 02:08:14 -0400
Message-ID: <9DABFA8E-064D-4647-A4FE-0041C8B0051C@raeburn.org>
References: <87wq97i78i.fsf@earlgrey.lan>
	<jwviokn4n6w.fsf-monnier+emacs@gnu.org>
	<86k2dk77w6.fsf@molnjunk.nocrew.org>
	<m2zimgw0i8.fsf@newartisans.com>
	<9D64B8EA-DB52-413D-AE6A-264416C391F3@iotcl.com>
	<83int1g0s5.fsf@gnu.org> <m2mvicx62q.fsf@newartisans.com>
	<83twckekqq.fsf@gnu.org>
	<jwvbmyp20ym.fsf-monnier+gmane.emacs.devel@gnu.org>
	<m2fuo1a59i.fsf@newartisans.com>
	<m2r37l77mo.fsf@newartisans.com> <874m4aic0g.fsf@tromey.com>
	<7D150317-7A01-464D-8352-942631A3116B@raeburn.org>
	<8337juxb8h.fsf@gnu.org>
	<31A629C9-7C3B-4B5D-A5B5-38F556C4E064@raeburn.org>
	<83wph6vt0f.fsf@gnu.org>
	<646C7DAF-F7AB-48C2-AFDF-6881D2990617@raeburn.org>
	<83mvi0v9f2.fsf@gnu.org>
NNTP-Posting-Host: blaine.gmane.org
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Trace: blaine.gmane.org 1476943755 6626 195.159.176.226 (20 Oct 2016 06:09:15 GMT)
X-Complaints-To: usenet@blaine.gmane.org
NNTP-Posting-Date: Thu, 20 Oct 2016 06:09:15 +0000 (UTC)
Cc: emacs-devel@gnu.org
To: Eli Zaretskii <eliz@gnu.org>
Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Oct 20 08:09:11 2016
Return-path: <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>
Envelope-to: ged-emacs-devel@m.gmane.org
Original-Received: from lists.gnu.org ([208.118.235.17])
	by blaine.gmane.org with esmtp (Exim 4.84_2)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>)
	id 1bx6X7-0007A4-CW
	for ged-emacs-devel@m.gmane.org; Thu, 20 Oct 2016 08:08:53 +0200
Original-Received: from localhost ([::1]:52617 helo=lists.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>)
	id 1bx6X8-0005jj-CQ
	for ged-emacs-devel@m.gmane.org; Thu, 20 Oct 2016 02:08:54 -0400
Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:48636)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <raeburn@raeburn.org>) id 1bx6Wb-0005je-Bm
	for emacs-devel@gnu.org; Thu, 20 Oct 2016 02:08:22 -0400
Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <raeburn@raeburn.org>) id 1bx6WY-00050Q-6y
	for emacs-devel@gnu.org; Thu, 20 Oct 2016 02:08:21 -0400
Original-Received: from mail-qt0-x241.google.com ([2607:f8b0:400d:c0d::241]:32855)
	by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
	(Exim 4.71) (envelope-from <raeburn@raeburn.org>) id 1bx6WY-0004zZ-1V
	for emacs-devel@gnu.org; Thu, 20 Oct 2016 02:08:18 -0400
Original-Received: by mail-qt0-x241.google.com with SMTP id m5so1662770qtb.0
	for <emacs-devel@gnu.org>; Wed, 19 Oct 2016 23:08:17 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=raeburn-org.20150623.gappssmtp.com; s=20150623;
	h=subject:mime-version:from:in-reply-to:date:cc
	:content-transfer-encoding:message-id:references:to;
	bh=IRQJmF64Jvy18MnECDDYDswrgPAqOF7biy90JtOIty8=;
	b=MlwX9tmzdyndBL0p9+MZHpSEoDGeeBhzcCPBDQHveNx0QIOOiwnBaGvdlZX/D7PpRw
	e4HrUT4H8vD7hNWgehag5xzmMQ8ZylOfea4rN4v/AOqH/lWIFhAFFAqkbw33uNYxhoPN
	iX4Nzhpzo9ZcxIL5Dv48RiqLflAtr92hICOrjLQ2IbDv82AM7RbMamE0hZrBhhBbqbQX
	7vPKgGpsdy3RtEPpsfphXsps8yG7XQ8kT+oU4GwgQZNKy+U2KE+MY+cXyaxK3X8u9RTR
	Umy//YRcm7o1i0UYOvmy6FVihUWMipevsvzHtCaCs/BkPGxmwV0B0EPkb0wdMdnDhwDq
	6K7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20130820;
	h=x-gm-message-state:subject:mime-version:from:in-reply-to:date:cc
	:content-transfer-encoding:message-id:references:to;
	bh=IRQJmF64Jvy18MnECDDYDswrgPAqOF7biy90JtOIty8=;
	b=R4G408pHmpc6rh00imTwvCZeEH+uutwnbXE0lUPAlhl0ozpKVMTrGK1aGkMyvR5Jkm
	PR9D83pqd4k5GjP+KjjG2+MxR20lufBqvSi2Em8UCAEgvr4ApXn5oPKupxtzYdqMXLLY
	ERIuyVtji+DiogZ2S3Y1YVBNH0iRN2+JkHD6bd/9L8u2GisqwJRaUCBa5byPv5KLA9QQ
	bUlrTQh63+GlImzS6hGyXHOPC0KAWWZ+lK+Jtsv+xkZTLU6aHd3DuX/FnEVq6Mshtg+R
	sOdaiOlGEbYTyPT/Yl7fiv8SAhI7imwXYa2s8PScnpIbL+wUA7RCUBKSYv5csMXlWPk3
	8bog==
X-Gm-Message-State: AA6/9RneSFfdKHW7JHarZveAs29DQ7OC6AaYiHB8VP4qN5w+8Y2OD/cdWmwXLsD5kFdjSg==
X-Received: by 10.200.50.78 with SMTP id y14mr9328181qta.57.1476943697020;
	Wed, 19 Oct 2016 23:08:17 -0700 (PDT)
Original-Received: from [192.168.23.52] (c-50-138-183-136.hsd1.ma.comcast.net.
	[50.138.183.136]) by smtp.gmail.com with ESMTPSA id
	b12sm22282144qta.4.2016.10.19.23.08.15
	(version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);
	Wed, 19 Oct 2016 23:08:16 -0700 (PDT)
In-Reply-To: <83mvi0v9f2.fsf@gnu.org>
X-Mailer: Apple Mail (2.3124)
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2607:f8b0:400d:c0d::241
X-BeenThere: emacs-devel@gnu.org
X-Mailman-Version: 2.1.21
Precedence: list
List-Id: "Emacs development discussions." <emacs-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/emacs-devel/>
List-Post: <mailto:emacs-devel@gnu.org>
List-Help: <mailto:emacs-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=subscribe>
Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org
Original-Sender: "Emacs-devel" <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>
Xref: news.gmane.org gmane.emacs.devel:208519
Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/208519>


On Oct 19, 2016, at 07:57, Eli Zaretskii <eliz@gnu.org> wrote:

>>> That should be easy: since a subprocess is locked to a single =
thread,
>>=20
>> by default, but if that thread exits, that lock disappears
>=20
> And the process gets locked to some other thread.

Not that I can see, unless you explicitly call set-process-thread; =
update_processes_for_thread_death sets the process thread field to Qnil.


>=20
>>=20
>>> SIGCHLD should be delivered to that thread.  If we don't have that
>>> already, we should add that, it doesn't sound hard, given the
>>> infrastructure we already have (deliver_thread_signal etc.).
>>=20
>> It=E2=80=99s not completely trivial.  [=E2=80=A6]
>=20
> So you are saying this problem was never encountered before in any
> other program out there, and doesn't already have a solution?  I find
> that hard to believe.

Not at all, just that it may take some work.  Though, I would expect =
that most such programs we might go look at are committed to a =
multi-threaded design, and aren=E2=80=99t written to support both =
multi-threaded and single-threaded environments.

>=20
>> On the other hand, perhaps we can create one special thread to do all =
the waitpid() calls and pass info to the Lisp-running threads.
>=20
> Sounds like an unnecessary complication, but if that's how others =
solve
> this problem, so shall we.

I don=E2=80=99t know.  Aside from checking a few man pages while writing =
the previous email, I haven=E2=80=99t researched it.  I think it=E2=80=99s=
 probably the way I=E2=80=99d be most inclined to do such a thing, if =
the ability to use such helper threads could be assumed; it can=E2=80=99t =
be in this case.

I just took a quick look at some bits of a Thunderbird source tree.  In =
the Netscape Portable Runtime =
(thunderbird-49.0b1/mozilla/nsprpub/pr/src/md/unix/uxproces.c) they=E2=80=99=
ve got a =E2=80=9Cwaitpid daemon thread=E2=80=9D; whichever thread gets =
SIGCHLD delivered to it writes a byte to a pipe to wake up that daemon =
thread, which then calls waitpid until there are no more child process =
status updates to process.  It updates global (mutex-protected) data =
structures and optionally uses a condition variable to notify some other =
thread of the process change.

>=20
>>>> It=E2=80=99s easy enough to disable stack overflow checking when =
enabling thread support.
>>>=20
>>> Or add some simple code in the stack overflow handler to check if we
>>> are in the main thread, and if not, punt (i.e. crash).
>>>=20
>>>> If only one thread is allowed into the image processing code at a =
time (i.e., don=E2=80=99t release the global lock for that code) then =
that=E2=80=99s probably fine for now, and there=E2=80=99s probably other =
state there that different threads shouldn=E2=80=99t be mucking around =
with in parallel.
>>>=20
>>> Redisplay runs in the main thread anyway, right?  If so, there's no
>>> problem.
>>=20
>> If some random thread calls (redisplay) or (sit-for =E2=80=A6)?  I =
think it=E2=80=99ll run in whichever Lisp-running thread triggers it.  =
But, it=E2=80=99ll be the one holding the lock.
>=20
> No, sit-for causes a thread switch.

I believe that=E2=80=99s when it=E2=80=99s checking for input, after =
calling redisplay, though.

I just tried a test with a breakpoint in redisplay_internal; it can get =
called from threads other than the main thread.  As far as I know, we =
can=E2=80=99t have multiple threads in the redisplay code at the same =
time, though.

>=20
>>>> The keyboard.c one is the only one I=E2=80=99m a bit concerned =
about, in part because I haven=E2=80=99t looked at it.
>>>=20
>>> What part(s) of keyboard.c, exactly?
>>=20
>> Anything looking at getcjmp; that means read_event_from_main_queue =
and read_char.  Like I said, I haven=E2=80=99t looked very closely; if =
the static storage isn=E2=80=99t ever used across a point where the =
global lock could be released to allow a thread switch, it may be fine.
>=20
> That should already be solved, or else threads cannot receive keyboard
> input safely.

I hope so.  I haven=E2=80=99t convinced myself one way or the other yet, =
though.

Ken=