From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Tom Tromey Newsgroups: gmane.emacs.devel Subject: Re: call-process blocks the active thread Date: Fri, 08 Sep 2017 10:12:17 -0600 Message-ID: <87lglpdva6.fsf@bapiya> References: <837ex9znba.fsf@gnu.org> <83k219xu58.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1504887198 16061 195.159.176.226 (8 Sep 2017 16:13:18 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 8 Sep 2017 16:13:18 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.50 (gnu/linux) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Sep 08 18:13:13 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dqLtp-0003CW-5V for ged-emacs-devel@m.gmane.org; Fri, 08 Sep 2017 18:12:57 +0200 Original-Received: from localhost ([::1]:46318 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dqLtw-00016q-40 for ged-emacs-devel@m.gmane.org; Fri, 08 Sep 2017 12:13:04 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:60877) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dqLtl-00015v-Ra for emacs-devel@gnu.org; Fri, 08 Sep 2017 12:12:54 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dqLti-0005vk-Lj for emacs-devel@gnu.org; Fri, 08 Sep 2017 12:12:53 -0400 Original-Received: from gproxy1-pub.mail.unifiedlayer.com ([69.89.25.95]:59806 helo=outbound-ss-1812.hostmonster.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dqLti-0005cr-5w for emacs-devel@gnu.org; Fri, 08 Sep 2017 12:12:50 -0400 Original-Received: from cmgw4 (cmgw5 [10.0.90.85]) by gproxy1.mail.unifiedlayer.com (Postfix) with ESMTP id 5FE40175EF3 for ; Fri, 8 Sep 2017 10:12:28 -0600 (MDT) Original-Received: from box522.bluehost.com ([74.220.219.122]) by cmgw4 with id 74CM1w01k2f2jeq014CQXv; Fri, 08 Sep 2017 10:12:28 -0600 X-Authority-Analysis: v=2.2 cv=G8xsK5s5 c=1 sm=1 tr=0 a=GsOEXm/OWkKvwdLVJsfwcA==:117 a=GsOEXm/OWkKvwdLVJsfwcA==:17 a=2JCJgTwv5E4A:10 a=Va1hJJt0djawKbCUBWEA:9 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=tromey.com; s=default; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date: References:Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=+2EgRyPLr4/W+xg78fS3LHh6MgX+VvQfqxRBZtfKxG8=; b=DsN02bXSauwj162zrMKF3FE5oM dGUY9DK9DtDNGseakMHMdR/V+ewFtq9KgxBQu3ChqjzHl3cmM6m8ya8WrvfbybhT7o44ORJux4gF8 SsRjohs6LdSlzYDgci9hQFyFg; Original-Received: from 75-166-76-94.hlrn.qwest.net ([75.166.76.94]:37118 helo=bapiya) by box522.bluehost.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.87) (envelope-from ) id 1dqLtF-003R89-F8 for emacs-devel@gnu.org; Fri, 08 Sep 2017 10:12:21 -0600 In-Reply-To: <83k219xu58.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 08 Sep 2017 15:16:35 +0300") X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - box522.bluehost.com X-AntiAbuse: Original Domain - gnu.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - tromey.com X-BWhitelist: no X-Source-IP: 75.166.76.94 X-Exim-ID: 1dqLtF-003R89-F8 X-Source: X-Source-Args: X-Source-Dir: X-Source-Sender: 75-166-76-94.hlrn.qwest.net (bapiya) [75.166.76.94]:37118 X-Source-Auth: tom+tromey.com X-Email-Count: 1 X-Source-Cap: ZWx5bnJvYmk7ZWx5bnJvYmk7Ym94NTIyLmJsdWVob3N0LmNvbQ== X-Local-Domain: yes X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 69.89.25.95 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:218021 Archived-At: Eli> Of course, it's possible I don't see some clever way to make Eli> call-process thread-aware, which is why I said that if someone has Eli> concrete ideas for how to do that, they are encouraged to speak up. I looked at it briefly. I have two ideas. Maybe the simplest idea is to rewrite call-process in Lisp, using make-process and accept-process-output. This would automatically allow other threads to run. I didn't really investigate this in depth (so for instance I don't know if this can be done without adding extensions to make-process), but it seems to me that it would be nice to migrate code out of C when possible. Otherwise, I think call_process can be made to yield. A bit of background (I know Eli doesn't need this but maybe others do): Only one thread can run Lisp at a time. A global lock enforces this. However, threads can run non-Lisp code while not holding the global lock -- this is why Emacs can wait for input while Lisp runs in another thread. The way this is done is that a thread calls flush_stack_call_func to save registers on the stack (so that GC will do the right thing); this calls a provided callback, and the callback handles releasing the global lock, doing whatever non-Lisp thing it wants to do, and then re-acquiring the lock before returning. A good example of this is thread.c:thread_select, which winds up in thread.c:really_call_select. So, the idea for call-process would be to modify the inner loop of call_process to do this. I think the best spot would be to call flush_stack_call_func around the call to emacs_read_quit. However, the tricky thing here is that call-process relies on some global state. For example, it assumes that current-buffer will not change while it is working. So, after emacs_read_quit returns, this code would have to be careful to restore the state it needs. The hard part of the project is figuring out which bits of state must be preserved. It might also be reasonable to wrap the wait_for_termination call in a lock-releasing function. Tom