From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Spencer Baugh Newsgroups: gmane.emacs.devel Subject: Re: Releasing the thread global_lock from the module API Date: Fri, 01 Mar 2024 16:56:55 -0500 Message-ID: References: <86cysdrja3.fsf@gnu.org> <86a5nhrdv0.fsf@gnu.org> <868r31rbxn.fsf@gnu.org> <865xy5r8e3.fsf@gnu.org> <864jdpr5zy.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="29401"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Mar 02 07:26:57 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1rgIpl-0007Tz-7I for ged-emacs-devel@m.gmane-mx.org; Sat, 02 Mar 2024 07:26:57 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rgIo8-0000re-0e; Sat, 02 Mar 2024 01:25:16 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rgAsF-0008IT-FV for emacs-devel@gnu.org; Fri, 01 Mar 2024 16:56:59 -0500 Original-Received: from mxout6.mail.janestreet.com ([64.215.233.21]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rgAsD-0000rd-Ll for emacs-devel@gnu.org; Fri, 01 Mar 2024 16:56:59 -0500 In-Reply-To: <864jdpr5zy.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 01 Mar 2024 23:34:25 +0200") DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com; s=waixah; t=1709330216; bh=l1vWa6x5pRe6e9Vz03Fh7zfioiPux9lfPCqiyIpqX7I=; h=From:To:Cc:Subject:In-Reply-To:References:Date; b=epZcRApPCHCsn9yqsSMyd4Kz/BQiJ9xbYQncp6Nv7go6uixmuVeRScLP6/v96G6Ls 4xIJytHkR/WdfizK4+Fhxxq6bjQd5w/25QH8nhUed0T6J/lRJ1dW/lblTUvFM6/Ms2 R8HzzH4Ca1FKhc5WFDef/rYHY4ysR6JknE02BLoO2mrF1h6sWtJwaznzzRE0Z/8GSF xrKUUt6d4pznwTxGCA5syHVi8PS6N3291swaPcXJ5fPxNIggz9P6dF/CoR8ItefTtT sgqzuDTRn0IUpuEbTGqPRS302uIiU71/VzmigmL0SJtQxe4pPCDN3pSHIskz0gOiWa rL2iTIIEkwRaQ== Received-SPF: pass client-ip=64.215.233.21; envelope-from=sbaugh@janestreet.com; helo=mxout6.mail.janestreet.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Sat, 02 Mar 2024 01:25:14 -0500 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:316690 Archived-At: Eli Zaretskii writes: >> From: Spencer Baugh >> Date: Fri, 01 Mar 2024 16:21:03 -0500 >> >> Eli Zaretskii writes: >> >> > Where would you take the data for opening the socket? doesn't that >> > come from some Lisp program or from a Lisp variable? And how would >> > you know what kind of request to send? doesn't that come from Lisp as >> > well? >> >> Yes: I get those things as arguments from Lisp and then convert them >> into the native datastructures of my library, which can be used without >> further interacting with the Lisp machine. >> >> Then I would release the lock and call into my library, which does some >> useful work which takes a while. > > How is this different from starting your own native thread, then > releasing the lock? They are completely different things. Let me say again what this sub-thread is about: > If a C library provides some blocking function which does some > complicated form of IO, a module providing bindings for that C library > can release global_lock before calling that function, and then > re-acquire the lock after the function returns. Then Lisp threads > calling this module function will not block the main Emacs thread. So, if I am running the following program in a Lisp thread: (while (do-emacs-things) (let ((input (do-emacs-things))) (let ((result (call-into-native-module input))) (do-emacs-things result)))) and call-into-native-module unlocks the global lock around the calls into my library, then during the part of call-into-native-module which calls into my library, the main Emacs thread will not be blocked and can run in parallel with call-into-native-module. >> > And what would you do with the stuff you get in response? don't >> > you want to put this in some Lisp variable or provide as input for >> > some Lisp program? Etc. etc. >> >> Yes: After I finish the call into my library, I would take the lock >> again and call further Lisp functions to put the results back into the >> Lisp machine. > > How is this different from doing the same when your native thread > finishes? > >> >> Since it's useful for me, I'd like to write a patch which allow modules >> >> to do this; would it be considered? >> > >> > Once again: what cannot you do from a native thread started by the >> > module? IOW, why would you need access to the global lock machinery >> > in the first place, if all you want to do is something that is >> > unrelated to Emacs and its global state? >> >> See above: The call into my library, which takes a while, and is useful, >> does not touch the Lisp machine. But other code around that call does >> touch the Lisp machine, and so needs to run with the lock. The ability >> to release and re-acquire the lock means my module doesn't hold the lock >> when it doesn't need to. > > You didn't answer my question about doing this from a native thread. I hope my answer above clarifies it. > As for re-acquiring the lock: you cannot do that with how the Lisp > threads are currently implemented. You can only wait for the lock to > be released, and try re-acquiring it; whether you succeed is anyone's > guess, if there are other threads competing for the lock. Yes, that is the normal behavior of locks. I am well aware of it. By "re-acquire", I mean (in your words) "wait for the lock to be released, and try re-acquiring it". It is okay if my module has to wait for the main Emacs Lisp thread to stop running before it gets the global lock again.