From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Po Lu Newsgroups: gmane.emacs.devel Subject: Re: Multithread or multiprocess in emacs?? Date: Mon, 17 Apr 2023 21:58:59 +0800 Message-ID: <87jzyahaos.fsf@yahoo.com> References: <7w4yjuxtu6rm325oeddtl3fu4oe64fsarru3s4fnh7on5m7udl.ref@wzxqo5pm6es7> <7w4yjuxtu6rm325oeddtl3fu4oe64fsarru3s4fnh7on5m7udl@wzxqo5pm6es7> <87354yiq8q.fsf@posteo.net> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="17437"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Ergus , emacs-devel@gnu.org To: Philip Kaludercic Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Apr 17 16:01:57 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1poPQa-0004JH-PS for ged-emacs-devel@m.gmane-mx.org; Mon, 17 Apr 2023 16:01:56 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1poPNy-00021r-Jy; Mon, 17 Apr 2023 09:59:14 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1poPNv-000205-DF for emacs-devel@gnu.org; Mon, 17 Apr 2023 09:59:11 -0400 Original-Received: from sonic306-21.consmr.mail.ne1.yahoo.com ([66.163.189.83]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1poPNt-0002kv-Pi for emacs-devel@gnu.org; Mon, 17 Apr 2023 09:59:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681739946; bh=IzDo4QnTk037IWZ21ti4WAh9vmz7bfdm9HTC+oqqOaw=; h=From:To:Cc:Subject:In-Reply-To:References:Date:From:Subject:Reply-To; b=HMGX3Sot2c2y6iJmMTTNnLI3tlh+TK5/iZUUizTIwTwAnc8Sv/KtfKbMY6k+6E8mZo6JxQJ1XcIRGsfipy1gDkT3zFyZC+Z3Ilsq8Ym4M2PiHQNOOaRId8TJyXd2AqvIM12w5iy/3M4xFvJrJmDgX4SR6QmzRUqRFH85U7v3AN/IvtHGG6RVnwOCHTEkYB1KDwQa0gQIbhHRKSKcpJ8qP1gHd++qgfGInJi3Hkh+jfHxmJ+OCaVAv0I84dIS0rZonJL6IzwO3KB/Mnoj7hsj8fCuhwSVaBntJoteW6gLVx65btXxwQe0HL8jM9PsYzxL8QHRrytTiBCqBvfMogKYbg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681739946; bh=eOUIF+L4elsggkUhmMxW68nCEU0XasJs1+zGPgKRD1H=; h=X-Sonic-MF:From:To:Subject:Date:From:Subject; b=SG5PaoDlzevqwThzTZ6I5wLTThySP6pKyfZhKtFAjLTFV3SMT7VUuO67WpcoL7ulNoHMKoHcCvpHE9x10O5zgt+nFD+rWrG9vZsYEpQk3359lYQQ5seQP4m+BRe6EBKUJwwK863iAQjWzqlT6qA2/pwQtEjfD/fCXZZJodCMkh3r2RMuc6CCnfnKEQt7g4Jx+eYAX/FDiWl6cdItulDQOTGnvBm93XcxKWSBSxAwC0Sxmp0kiSWPkTbltbPWUguoG7CY0kde0ZcSgS55AXVnYsDfAsp6S7fBpAZChBsHl5Vi4XqsHXjLUccXG6bGFRrTGgQ+XDkHZEedeLIkfSR66Q== X-YMail-OSG: Ft9uOKIVM1ntxeS.k_kFADFVhJ_FFXryeHRoQ5Z35h.3Fq1DHim_fJFBmvkDFQK OkokUZgl5ZjY0HfSwqV7WHuEyzwSSne3iTaRrnsr9HDiI2KBiEfnnxHHSiuU6xceOqTOXNVUPPj9 mZeBUJ3fSalXeKjyqfzreQER25pH9cGdhVXqDQ5m7oa.00KQEv5otIx5Wku7M4yEbjF8wTVFiXby u4mFbIN1sfPZmRO9nLgVfT2xXNRMOx.tKQ5wdwkQmPMf8qe6nA5WgLeSHJMDcAGPvwrM2ptjCfDR MXsgwytZJL3It2ewLuzhVNhjzdkdURjTsDBLEklXLpjnG0v0v2rkMA6y1skj36L.qKWl_ppBwD86 sHPWG5ODc4dfLotEOAcB9YxNYIDeW1vzMrcxV7U4cl5J.tognzZ1KbWBRZ260fUipVJo1JdMXHjP eKyYWaMhrDrLUf_MDg.07iOuVKY3JoM.UNuzEc.dSAU2ofEQ7fUisXepJgTDmysKpO1shlWtMIET QDz5d.M6zieHcjgaaVo19Z0puUAAFFmTl7VBwi5BxyXmWhFe7.BHUIYILs0wHirp.GC7LaRw8NXy V65soiN_VRXLpNxujxdxZz1zpj_gW9AskhDrC6luhihBE4V25BSen33C6hQQ9InTTciHmAKxigJC sQmCHuOB.nJUh4HKdAVwkBmFNsOw4PA3suZ9o0YDioMT9cnBxkwlH4FE8YOfqCpq.W6wdpGGjl9d l7mu0VnTe2yES8ba9lBrvjaR_rJWC198boHJpV7YrTM0HEyhKOmCvokM4QVeGPhPcQ6p6ZdaZ85_ 0N4VDhqJX1c6JCA8WTFiVAy.Qcg8R9RU6fMQDZB5dU X-Sonic-MF: X-Sonic-ID: 97d9b410-08e4-4792-a521-408fb7013e8f Original-Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.ne1.yahoo.com with HTTP; Mon, 17 Apr 2023 13:59:06 +0000 Original-Received: by hermes--production-sg3-6d6fb994f6-94jgf (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID dcd1461c0fe7882c9f509ccf68ca912a; Mon, 17 Apr 2023 13:59:03 +0000 (UTC) In-Reply-To: <87354yiq8q.fsf@posteo.net> (Philip Kaludercic's message of "Mon, 17 Apr 2023 13:37:41 +0000") X-Mailer: WebService/1.1.21365 mail.backend.jedi.jws.acl:role.jedi.acl.token.atz.jws.hermes.yahoo Received-SPF: pass client-ip=66.163.189.83; envelope-from=luangruo@yahoo.com; helo=sonic306-21.consmr.mail.ne1.yahoo.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:305366 Archived-At: Philip Kaludercic writes: > Ergus writes: > >> Hi: >> >> Very recently I have seen a code in an elpa package that starts async >> subprocesses; but instead of using make-process the package uses >> make-thread + process-file. This was to reduce the latency and some >> lagging (not evident in my system, but apparently it annoyed the >> package's author). >> >> So, my question is now: Has anyone measured the overhead created by >> make-process in critical parts of the code like flymake checkers, or >> ispell? and compared with creating new threads? >> >> If this is somehow significant maybe we may consider adding a helper >> thread or thread-pool for some purposes as now the C11 standard has the >> threads.h header. > > Note that C11 threads are controversial[0], but also optional. Indeed, threads cannot be relied on to exist in Standard C. Many compilers don't support C11 anyway, and even less come with a C11 runtime. Threads are an optional feature in Emacs too. > Also, I might be mistaken, but shouldn't Pthreads in some form be > available on platforms that Emacs runs on? Also, see (elisp) C > Dialect. pthreads are not always suitable for complex language runtimes. But even that is besides the point. The big problem with threads is to make an Emacs capable of running more than one thread at the same time work safely. Consider this pattern, which is very common throughout Emacs: Lisp_Object cons; struct Lisp_String *sp; cons = < some cons >; CHECK_STRING (XCAR (cons)); sp = XSTRING (XCAR (cons)); even assuming that reads and writes of Lisp_Object are coherently propagated between threads (which is not true on some kinds of CPU, and also --with-wide-int configurations in general), there is still the problem of `XCAR (cons)' changing to something other than a string between `CHECK_STRING' and `XSTRING'. So you will first have to get rid of all of these not-so-constant subexpressions. Next, you will have to interlock everything that is reasonable to use from multiple threads at the same time, such as all of alloc.c, editfns.c, fileio.c, and so on. And do so correctly. Then, you will have to make everything else report an error when used outside the main thread. Finally, you will have to make the resulting Emacs with all of the extra interlocking overhead run almost as fast as it used to. (Not to mention all the Lisp that currently assumes two threads can't run at the same time. Just avoid thinking about that for now.) This work will require a lot of manpower and time! Just count the number of years it took to interlock the Unix kernel (which, like Emacs, previously used set-priority-level to handle the limited amount of reentrancy present when running on a single CPU with interrupts.)