From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Spencer Baugh Newsgroups: gmane.emacs.devel Subject: call-process should not block process filters from running Date: Tue, 27 Jun 2023 17:55:00 -0400 Message-ID: Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="32892"; mail-complaints-to="usenet@ciao.gmane.io" Cc: app-emacs-dev@janestreet.com To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Jun 28 04:26:27 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qEKt1-0008KW-Ee for ged-emacs-devel@m.gmane-mx.org; Wed, 28 Jun 2023 04:26:27 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qEKsK-0001Un-Jb; Tue, 27 Jun 2023 22:25:44 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEGeT-0006Af-6x for emacs-devel@gnu.org; Tue, 27 Jun 2023 17:55:09 -0400 Original-Received: from mxout5.mail.janestreet.com ([64.215.233.18]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qEGeQ-0004LC-8j for emacs-devel@gnu.org; Tue, 27 Jun 2023 17:55:08 -0400 Received-SPF: pass client-ip=64.215.233.18; envelope-from=sbaugh@janestreet.com; helo=mxout5.mail.janestreet.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Tue, 27 Jun 2023 22:25:41 -0400 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:307269 Archived-At: When Lisp code calls call-process, then while call-process is running, all Lisp is blocked from running, including process filters and timers created beforehand by other Lisp. This call-process behavior is harmful, but we can fix call-process to not behave this way. This call-process behavior is harmful: Many packages rely on their process filters and timers being able to run: - Packages which communicate over the network (such as ERC) rely on being able to respond to heartbeats and prevent timeouts. - Packages which respond to requests from other programs (such as EXWM) rely on being able to respond to those requests. A simple (shell-command "sleep 60") will cause most such packages to break or behave poorly. We can fix call-process to not behave this way: It is straightforward to reimplement call-process in Lisp, using accept-process-output, so that process filters and timers and other Lisp can run while waiting on call-process. There's been at least one attempt at doing so: https://github.com/tromey/emacs/tree/rewrite-call-process My suggestion is that we should create a new helper function in Lisp, perhaps called "process-run", which has an identical interface as call-process, except that while it is running, other process filters and other Lisp are still able to run. Then we can move users of call-process over to this new function. Most (but perhaps not all) Lisp using call-process should be using process-run, since most Lisp doesn't actually want to block process filters from running. As a bonus, process-run will be usable by Lisp code running in Lisp threads created by make-thread, as will any functions which use process-run, since accept-process-output supports threads. (call-process does not) A few clarifications in advance for things which I suspect might be confusing: - This is not about switching Lisp from running processes synchronously (call-process) to asynchronously (make-process). process-run would still be a synchronous API. It is much easier to fix this one bad aspect of the call-process API than to move lots of Lisp over to make-process. Even if we moved all Lisp over to be run asynchronous processes, we would still want a fixed call-process for Lisp thread support. - This does not require changing the C core of Emacs at all, nor changing the call-process implementation. The existing accept-process-output API is sufficient for creating a synchronous process-running API which does not block Lisp from running in process filters and timers and so on. - MS-DOS can't support an implementation of call-process which allows other Lisp to run. On MS-DOS, call-process and process-run will be identical. This is fine and not a problem. Furthermore, even if it was a problem, MS-DOS should not hold us back from improving Emacs on other platforms.