From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Releasing the thread global_lock from the module API Date: Sun, 03 Mar 2024 08:13:06 +0200 Message-ID: <864jdnq1vx.fsf@gnu.org> References: <86cysdrja3.fsf@gnu.org> <86a5nhrdv0.fsf@gnu.org> <868r31rbxn.fsf@gnu.org> <865xy5r8e3.fsf@gnu.org> <864jdpr5zy.fsf@gnu.org> <8634t9qgl2.fsf@gnu.org> <87plwck2q5.fsf@catern.com> <868r30pnxv.fsf@gnu.org> <87le70jrwr.fsf@catern.com> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35317"; mail-complaints-to="usenet@ciao.gmane.io" Cc: sbaugh@janestreet.com, emacs-devel@gnu.org To: Spencer Baugh Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Mar 03 07:13:54 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rgf6g-00090i-1V for ged-emacs-devel@m.gmane-mx.org; Sun, 03 Mar 2024 07:13:54 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rgf64-0007GI-AY; Sun, 03 Mar 2024 01:13:16 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rgf5y-0007Fv-V2 for emacs-devel@gnu.org; Sun, 03 Mar 2024 01:13:12 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rgf5y-0003ib-6S; Sun, 03 Mar 2024 01:13:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=fEAEawAiJ5wMzmar9lygXVeexNmS+Kn3zPQBCa/Kljk=; b=EY3fOO9P/+6a +TgVJohSYCl0dcrGPIhNSrdcLDybwWooSpcf+WqS3C79DlZsR8NdZwes58yaWb+JCeOTc6sYq+AzF CvIWPdyKJxnwEgJFJftu2Dvhr05G5TDOSbQfkJuWP5e1mW7VUl5o5ZgbIEHbsQITUPS5nSQ9JzXDD /eD1M0TrxpdMcU5EvH4z4Kk39NBiiHreLgBBzcMHym1v+bFvHWJJKAOaxaQce7RZopM8NcdrcJJHb WqXCe+mRz26dL99DfqgWq+WimmiVIttX+rNi0ZE/g3qVYV6B82bK0muolxAk90D6deqdfLGvA+ww0 KopEBLXmTwQnoDYA63Duvw==; In-Reply-To: <87le70jrwr.fsf@catern.com> (message from Spencer Baugh on Sat, 02 Mar 2024 20:33:05 +0000 (UTC)) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:316722 Archived-At: > From: Spencer Baugh > Date: Sat, 02 Mar 2024 20:33:05 +0000 (UTC) > Cc: sbaugh@janestreet.com, emacs-devel@gnu.org > > >> So, the main difference between the "new thread" model and the "release > >> lock" model is that creating a native thread takes a nontrivial amount > >> of time; maybe around 0.1 milliseconds. If some_native_function would > >> takes less time than that, the thread creation cost will slow down > >> Emacs, especially because the native module creates the native thread > >> while holding the Lisp global_lock. > > > > Why are you worried by 0.1 msec slowdown (if it indeed takes that > > long; I think it should be around 10 to 20 usec at most)? If this > > kind of slowdown is important for you, you are using the wrong > > software package (and probably the wrong OS as well). > > Because a Lisp program that uses a native module might make thousands of > module calls. This is fine when each call takes a microsecond. If we > add, for example, 500 microseconds of overhead to each module call, then > 1000 module calls will take half a second. > > For example: I have a project.el backend which uses a native module. > Looking up the project for the current directory and then getting the > name of the project makes around 5 module calls. I have around 200 > projects. That works out to 1000 module calls to get the names of all > my projects. Currently with the native module backend this takes around > a millisecond. With 500 extra microseconds per call, it will take half > a second. A module whose call takes this little time to complete and whose calls from Lisp are so massive should not, and need not, use this technique, because releasing the global lock for such a short time will not benefit anything. Releasing a global lock is only beneficial if a module call does a lot of work, during which other Lisp threads could do something useful. but if a module call takes a significant time (say, a few seconds), then adding a 20 usecs to that time is insignificant. > >> The "release lock" model fits this need. > > > > But it exposes the sensitive internals and runs the risk of more than > > one Lisp thread running at the same time, and thus is not acceptable. > > Yes. But of course in practice we would find a design which allows > releasing the lock but is hard to misuse. Sorry, I don't believe this is reliably possible, or safe enough to justify exposing the lock to Lisp. Certainly not to modules, which is an open-ended gateway into the bowels of the Emacs Lisp machine. > How about this: > > native_output = env->call_without_lock(some_native_function, native_input); > > call_without_lock would be a function which releases the global lock, > calls a specified function, then acquires the global lock again. Sorry, I don't agree to adding such interfaces to Emacs. And if you are still not convinced, let's agree to disagree.