From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Blocking calls and threads Date: Thu, 20 Apr 2023 17:36:23 +0300 Message-ID: <837cu662oo.fsf@gnu.org> References: <838ren6mpp.fsf@gnu.org> <83cz3y65ev.fsf@gnu.org> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="15790"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Lynn Winebarger Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Apr 20 16:37:18 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ppVPS-0003tZ-1L for ged-emacs-devel@m.gmane-mx.org; Thu, 20 Apr 2023 16:37:18 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ppVOZ-0001PG-M7; Thu, 20 Apr 2023 10:36:23 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ppVOY-0001Oz-B0 for emacs-devel@gnu.org; Thu, 20 Apr 2023 10:36:22 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ppVOW-0003L2-4b; Thu, 20 Apr 2023 10:36:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=clBUkroMni3/bPHEX9EcQOeEV/NcQ4w3Vv/wEIEBjbo=; b=d2pO0O2r0IgE m0T25PQUlEACxRgMB7NjYX3yaVBn0Wx/64uQgjwCxlU2/koMqcSUx3P49IbKYjgolc9k9gAOCdFEK gdfSCMr9ZTPk/IgqNm4ONR/NzfVoHZk7tf4R3i86jp/PUyNNi5/yXfcwrFXrVsq/C/9LuHQ4NBfYk FVQxs6gtHkF90bIdIVUXuekl4Cc+mA4CBMAKSMM0/jKmkyRC0XgBjzJg2/r03AIwoJrtkY6BQb9Pi /hycLDVHQq10ft/Vh5C84wXkls9bYCbbfBjPEGBcidZnio/h2LCEvJPgLrZX7+QwO9OQ0zQpMcLQW pEOks4gvsRmBi+xIkLsqGQ==; Original-Received: from [87.69.77.57] (helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ppVOL-0001Tg-Cn; Thu, 20 Apr 2023 10:36:19 -0400 In-Reply-To: (message from Lynn Winebarger on Thu, 20 Apr 2023 10:19:11 -0400) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:305515 Archived-At: > From: Lynn Winebarger > Date: Thu, 20 Apr 2023 10:19:11 -0400 > Cc: emacs-devel@gnu.org > > > If you yield before issuing the system call, the system call will wait > > until you re-acquire the lock. So how will this help? > > You're talking about yielding the system thread, I'm talking about > yielding the Lisp machine thread. No, I'm also talking about the Lisp machine thread. The thread which calls insert-file-contents and runs the C code of insert-file-contents and of the subroutines called by insert-file-contents. > Even though Lisp machine threads are implemented by mapping them to > the underlying system thread, the Lisp machine execution state is > kept coherent by the global (interpreter) lock. Releasing the lock > is effectively yielding the Lisp machine thread. The system thread > will yield implicitly if the read blocks. What you say here is not relevant to the issue at hand. > the read operation should either use some temporary buffer and copy > into the lisp buffer, or wait for data to be ready, then reacquire > the GIL before invoking the read syscall with a pointer into the > Lisp buffer object. Yes, and that's exactly where we will lose: most of the heavy processing cannot be done in a separate temporary buffer, because it calls functions from the Lisp machine, and those are written assuming nothing else is running in the Lisp machine concurrently. For example, take the code which decodes the file's contents we have just read. I encourage you to take a good look at that code (most of it is in coding.c) to appreciate the magnitude of the problem. So the code which can run in parallel with another Lisp thread will be able to do only very simple jobs, and will also add overhead due to the need of copying stuff from temporary buffers to Lisp objects. Of course, we could redesign and reimplement this stuff, but that's a lot of non-trivial work. My assumption was that you are considering relatively lightweight changes on top of the existing code, not a complete redesign of how these primitives work.