From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Releasing the thread global_lock from the module API Date: Fri, 01 Mar 2024 18:47:32 +0200 Message-ID: <86cysdrja3.fsf@gnu.org> References: Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="11338"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org, phst@google.com To: Spencer Baugh Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Mar 01 17:48:16 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rg63U-0002he-PD for ged-emacs-devel@m.gmane-mx.org; Fri, 01 Mar 2024 17:48:16 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rg62w-0006oo-HQ; Fri, 01 Mar 2024 11:47:42 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rg62s-0006gR-JG for emacs-devel@gnu.org; Fri, 01 Mar 2024 11:47:39 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rg62r-0007OC-4Q; Fri, 01 Mar 2024 11:47:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=8sN6ZlEYTg5tZGQw9Iwc0vYiAKyDGuVueprkYxilr50=; b=qliraXC4QezP K1Em09MOuGJt6WgU/CQwiqBAbdi6I11vMBhZOVRG1drhBuZF6w5NORqz6M+Ff0XWsT8TobG5+bwBJ CaSkD+UIDzzEIvFSNtxoJzacWykqhCuigg7Q+B7wrtOI3RPMQ4S0iwbK3zgeaDnX2VPP+Cj8RqgmW OkOdn7ZVDYjb6vtowM1/kxLiHGtK3aENDsAHSVH0ULRETkj7S1wJ2W5bE0MWV5ZAdgOhrB4GMwAhn SIC7vTrAm79ogUnO+Qa43x5y7IaGJw7yNXXeKyq3bR7fv6ZsblRFKEwhwzCyk9OV26XZ67+j2H48n 2zhYJnZMnLRt5q03i4Kh/g==; In-Reply-To: (message from Spencer Baugh on Fri, 01 Mar 2024 09:53:41 -0500) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:316664 Archived-At: > From: Spencer Baugh > Cc: Philipp Stephani > Date: Fri, 01 Mar 2024 09:53:41 -0500 > > In other languages with an FFI and a global interpreter lock taken by > threads, such as Python, it's possible for a native function called from > the interpreter to release the global lock, and then re-acquire it > before calling any interpreter functions (or automatically re-acquire it > upon returning to the interpreter). > > This allows for a limited form of actual parallelism: if a native > function is doing work that doesn't involve the Python interpreter, > e.g. doing numerical computations in native code, it can release the > lock so that it can run in parallel with other threads. If the native > function needs to call a function which does involve the Python > interpreter, it can re-acquire the global lock around that call. > > Could the same functionality be added to the Emacs module API? Then > module code could release the global lock when doing Emacs-independent > computation, and re-acquire the lock when calling into Emacs. This > would allow a limited form of parallelism: if a Lisp thread is calling > module code which releases the lock, then the module code could run in > parallel with the rest of Emacs on that thread. It isn't clear what exactly do you want added, since calls to thread-yield and similar APIs (like sleep-for and accept-process-output), which release and re-acquire the global lock, are already possible from Lisp threads. What exactly is missing? > This should be safe, because this is in effect already possibly through > thread-yield: that can be called from anywhere in a Lisp thread, and > releases the global lock, calls the operating system thread_yield > function, and then re-acquires the global lock. If instead of doing > thread_yield, it did some computation, e.g. > > int x = 0 > for (int i = 0; i<999999; i++) > x += i; > > that would have the same effect as yielding, but the computation would > run in parallel with the main Emacs thread. This is what would happen > if module code released the lock, did some Emacs-independent work, and > then re-acquired the lock. If a module wants to do some Emacs-independent work, it can start a native thread and do it there, no? So again, what is missing? > As an additional bonus, this would allow the module API to extend Lisp > threads: if a C library provides some blocking function which does some > complicated form of IO, a module providing bindings for that C library > can release global_lock before calling that function, and then > re-acquire the lock after the function returns. Then Lisp threads > calling this module function will not block the main Emacs thread. I think this is already possible, see above. I guess I'm missing something.