From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Ihor Radchenko Newsgroups: gmane.emacs.devel Subject: Re: Concurrency via isolated process/thread Date: Thu, 13 Jul 2023 14:23:36 +0000 Message-ID: <87fs5rnb4n.fsf@localhost> References: <871qhnr4ty.fsf@localhost> <87zg49xfke.fsf@localhost> <83sfa1gjns.fsf@gnu.org> <87r0plxbep.fsf@localhost> <83ilawhpi6.fsf@gnu.org> <87zg48apwr.fsf@localhost> <83edljg8ub.fsf@gnu.org> <87o7knbxr7.fsf@localhost> <838rbrg4mg.fsf@gnu.org> <87ilavbvdr.fsf@localhost> <834jmffvhy.fsf@gnu.org> <878rbrbmwr.fsf@localhost> <83fs5zecpo.fsf@gnu.org> <87351zbi72.fsf@localhost> <83351yevde.fsf@gnu.org> <87cz12ad2w.fsf@localhost> <83a5w6cwdr.fsf@gnu.org> <87pm518m0g.fsf@localhost> <17bc586130847440bbf6@heytings.org> <87bkgk6lps.fsf@localhost> <489f350ce60aea470c35@heytings.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="6642"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , luangruo@yahoo.com, emacs-devel@gnu.org To: Gregory Heytings Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Jul 13 16:24:35 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qJxFD-0001WK-4y for ged-emacs-devel@m.gmane-mx.org; Thu, 13 Jul 2023 16:24:35 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qJxEI-0002D2-Pa; Thu, 13 Jul 2023 10:23:38 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qJxEG-0002Cj-47 for emacs-devel@gnu.org; Thu, 13 Jul 2023 10:23:36 -0400 Original-Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qJxEB-0003Gn-Gq for emacs-devel@gnu.org; Thu, 13 Jul 2023 10:23:35 -0400 Original-Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id E576E240103 for ; Thu, 13 Jul 2023 16:23:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1689258208; bh=CkogwYrdUIvFm1Dqz7W3cYM3Dd2Xni/ICVyfykZNKOk=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version:From; b=d6XBUGkcpyjrN2IZmi/uq+GeOr8fJoJxK0sVe7ieot7/FwviSu+tgeKPXikghI6a1 PB1r3s1vg6b74qRydjWCBblEHLuZjKsUFiUxqJnm4gJ9tb98ww7IP5BzqbYXRI97PH zlg3TeBGEQ4iOkvv//UAmbfX9GIh+xT0O42CgNTReupdMoVoiP2MAMKSsJKHKjS95C lT5/3/A+OIdMK++XgWjgZ3vXcmHYL8QmTy4+savAhSRauueQDyI7IupqCgdIVGHbHW KOPnxbEgbqc3gJUSgFkyNEaF0GOdKqLEswQycGV3X9zrf/GumP1Lt55/uevFgqg4dK qLBQOnoqiO+Gg== Original-Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4R1xfl6fspz6tvb; Thu, 13 Jul 2023 16:23:27 +0200 (CEST) In-Reply-To: <489f350ce60aea470c35@heytings.org> Received-SPF: pass client-ip=185.67.36.66; envelope-from=yantar92@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:307822 Archived-At: Gregory Heytings writes: > It's not completely clear (to me) what you mean by "locking". But I > assume you mean two operations lock/unlock with which a thread can request > and waive exclusive access to an object, and sleeps until that exclusive > access is granted. IOW, mutexes. Yes. Although mutexes are not the only way to achieve this. > If my guess is correct, that is not > possible. You cannot use mutexes in a program whose data structures have > not been organized in a way that makes the use of such synchronization > primitives possible, without having deadlocks. Given that in Emacs > objects are part of an enormous unstructured graph, with pointers leading > from anywhere to anywhere, that's clearly not the case, and all you can > use is a single global lock. I do not think so. There is usually no need to ensure that a given Elisp object is locked recursively, with all the linked objects (like all elements in a list). We just need to ensure that we protect individual Lisp_Object's from data races. >> But the very need to access global Elisp variables/objects from async >> threads should not be considered a good practice (except near start/end >> of thread execution). Most of processing should be done using threads' >> local lexical scope. > > ... if what you have in mind are async threads that should in fact not > access objects of the main thread, why is it necessary to lock > objects? > You can prepare the arguments to the async thread in the main thread, > start the async thread, and when its execution completes process the > return values of the async thread in the main thread, which is what > emacs-async already does. > May I ask you if you have a concrete example of a task that you would like > to perform with such threads, and that cannot already be done with > emacs-async? In other words, what are the limitations of emacs-async you > try to overcome? You cannot pass non-printable objects like (like markers of buffer). You cannot pass Emacs local state, like user customization, except a small subset. You have to go through print/read loop to pass the objects around. You have to deal with startup overheads and large memory overheads associated with creating a new Emacs process. Examples: 1. Background parsing of large buffers, where you cannot afford to pass the whole buffer and parser state back-and-forth every time the user makes changes. 2. Searching across multiple buffers, while keeping all the buffer-local customizations (including runtime, this-session-only, customizations) honored. 3. Stealth fontification of buffers. Not just on idle, but asynchronously. 4. Doing something as basic as loop parallelism on multiple CPUs for CPU-heavy processing. (passing the data between Emacs process would defer the whole purpose here - that's inter-process communication overheads + sexp parsing !!!) 5. The very async process communication where process sentinels are not trivial and have to utilize significant CPU processing. 6. Basically, anything more complex than what can be done using a bunch of foo& + wait() in bash. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at