From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: John Shahid Newsgroups: gmane.emacs.devel Subject: Re: User interaction from multiple threads Date: Thu, 30 Aug 2018 16:15:57 -0400 Message-ID: <874lfbr4oy.fsf@gmail.com> References: <838t59j821.fsf@gnu.org> <87lg92q7ih.fsf@runbox.com> <87bm9xqg46.fsf@runbox.com> <838t51dl10.fsf@gnu.org> <87efehqdlv.fsf@gmail.com> <3c3e954d-65fc-58e6-9165-1c775ead69a2@orcon.net.nz> <87bm9kr3ad.fsf@gmail.com> <83y3co0zmb.fsf@gnu.org> <875zzrrg5b.fsf@gmail.com> <83k1o721qm.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1535660061 9215 195.159.176.226 (30 Aug 2018 20:14:21 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 30 Aug 2018 20:14:21 +0000 (UTC) User-Agent: mu4e 1.1.0; emacs 27.0.50 Cc: psainty@orcon.net.nz, gazally@runbox.com, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Aug 30 22:14:16 2018 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fvTKZ-0002GT-5p for ged-emacs-devel@m.gmane.org; Thu, 30 Aug 2018 22:14:15 +0200 Original-Received: from localhost ([::1]:50770 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fvTMf-0001Ng-9E for ged-emacs-devel@m.gmane.org; Thu, 30 Aug 2018 16:16:25 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33290) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fvTMV-0001NQ-Hv for emacs-devel@gnu.org; Thu, 30 Aug 2018 16:16:19 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fvTML-0002F5-Pt for emacs-devel@gnu.org; Thu, 30 Aug 2018 16:16:14 -0400 Original-Received: from mail-qt0-x22d.google.com ([2607:f8b0:400d:c0d::22d]:42566) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fvTML-0002EI-IU; Thu, 30 Aug 2018 16:16:05 -0400 Original-Received: by mail-qt0-x22d.google.com with SMTP id z8-v6so12081030qto.9; Thu, 30 Aug 2018 13:16:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=references:user-agent:from:to:cc:subject:in-reply-to:date :message-id:mime-version; bh=DLdCj/oxs2MPGS0DbyAziDAksy3Se/Y2WMGGW43/jcQ=; b=ivTOevYHetubeKfSspuXuIPotg/hvt07f6mauKP5LE2P8BOYX5BM1ejgmNbTzU8MIB XdKcnF2v85MURDVRKGN/MB0dh5VsGa+5lJgrd6MGyG3zjR5yBnBSNeUjKyGsO9APBXp1 e3GM8HCUt2K3feF3haiBnJUGSeXj2+i0FC2GHtGSHK3QLkp0965jgoXInl55Ps7AfgH1 77F/4kGHMjxnFHMcSvqDqv0ikmDHSsgw6nbc7zp2nQ8suFAC/y3k5tyLV5ni5M+n2O4H c2Re3ZnHzdsW9Z+O58YmI1TTo/twPqdjTRtZBPz90R18FnGOcsAxtOOMr4lIwoFyIZeg g49A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:cc:subject :in-reply-to:date:message-id:mime-version; bh=DLdCj/oxs2MPGS0DbyAziDAksy3Se/Y2WMGGW43/jcQ=; b=VVOca4qDcZEjqinsoAw/Z6ep6s7xWHeaMW3Hf6c95nVsAnkRSmGWQhKvZERnsReoK7 nsq8zwKro9Tlyp7XWon60YP5fdqA7sRvB6/GyZmihFNnVDKrzCqgCZAKR/6p+LJSxXeb RIUHsIkOfY5NlTiqAfsLUjDLFXNbXYwhMPY4Dhs24kNI6zB0BGXMnFr5OUeKxxU+wDNC BmiLWpmERQ+hfLTEBvvwz87s6vXjnuYHypHiD0Jumg+4GQ5ByDUSJaE59v0e6YHfH/xF y/dWGXWH3H5b04aaxEDI3K6UfBaRnqcwnMZnx8aJ1loyzma8nva4EFHNshvFXckK2cja km7A== X-Gm-Message-State: APzg51BRtekhA6S9KcELMe5ncYlVbdJozNDkKMr3s/VERUAN3ZQs9Uu/ KJv/t8AbMErwXmEo11jP2ZCWo/sm X-Google-Smtp-Source: ANB0VdYulw8/wR0zMRlVJC4xBgKNVUQeay0FBXKHrzq6VUFb4m6Bpw2v/JJ6lkwyNwD2Rk0hUrVwXg== X-Received: by 2002:a37:c5d2:: with SMTP id k79-v6mr9359881qkl.206.1535660162509; Thu, 30 Aug 2018 13:16:02 -0700 (PDT) Original-Received: from amun (cpe-104-162-86-217.nyc.res.rr.com. [104.162.86.217]) by smtp.gmail.com with ESMTPSA id 53-v6sm4844377qto.61.2018.08.30.13.16.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 30 Aug 2018 13:16:01 -0700 (PDT) In-reply-to: <83k1o721qm.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400d:c0d::22d X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:229116 Archived-At: Eli Zaretskii writes: >> From: John Shahid >> Cc: psainty@orcon.net.nz, gazally@runbox.com, emacs-devel@gnu.org >> Date: Thu, 30 Aug 2018 12:08:32 -0400 >> >> > I'm afraid there's too much to encapsulate. E.g., every buffer-local >> > variable in every buffer to be used by the thread will need to be >> > encapsulated; and how will the main thread know in advance all that? >> >> Why do we need buffer local variables for all the buffers ? > > Not all the buffers, only those which the thread function will make > current at some point. Surely, you've seen and written code that > switches buffers as part of doing whatever it needs. Yeah, but I'm not sure why the portion that is prompting the user should be switching buffers. Even in that case the code will look like the following where all the necessary variables are captured by the closure: (on-main-thread (read-file-name ...) (with-current-buffer other-buffer ;; captured by the closure (do-something) (read-file-name ..))) >> Isn't the current buffer during the prompt the only relevant buffer. > > No, it is not. There are also various other parts of the global state > that should be specific to a thread, like the list of condition-case > handlers; see the members of 'struct thread_state' as it is defined > now, for more examples. I am not sure I totally understand the impact of every field in that struct has on my proposal. Speaking of signals, I think those should be re-emitted on the child thread stack. For example, the response of the main thread can indicate the error symbol and data, which can be used in the child thread to raise/signal again. >> My idea, is that each thread is started with a pipe which is used to >> interact with the main thread when it needs to prompt the user. > > If you mean literally a pipe, then does this mean you suggest that > Emacs forks itself and runs each thread in a separate process, talking > via pipes with other processes? That's a completely different > architecture from what we have now, and it wasn't my intent to discuss > such a complete rewrite of the concurrency code. No, I am not proposing changing the architecture. I was proposing to literally use pipes to communicate between the different threads. In fact that doesn't have to be a pipe it could any implementation of a queue, as was proposed earlier. > If this is not what you mean, please clarify what you mean by "pipe" > in this context. > >> The child thread write the lambda that needs to run on the main >> thread then wait for main thread response using >> `accept-process-output'. The main thread handles the request in the >> process filter displaying the prompt and writing the result back to >> the pipe. Since the child and main thread are sharing the same >> memory, the communication can simply be the symbol name that holds >> the closure and the response, respectively. > > We already have the async package written by John Wiegley; it sounds > like you are describing the idea which he already implemented. > > The limitation of this is that you need to communicate to much of a > state to the subordinate Emacs, and some stuff is very hard, sometimes > impossible, to share this way. I am proposing something similar but taking advantage of the fact that threads are sharing the same memory space. Similar in the sense of preferring using the existing communication primitives (e.g. `make-pipe-process' and `accept-process-output'). > In any case, let's stay in the framework of the current basic design > of how threads work in Emacs. I sincerely apologize if what I'm being off topic or not helping this conversation. This is either due to my inability to explain my thoughts clearly or my flawed understanding of Emacs internals. May be trying to hack together the solution I have in mind will educate me and prove test the feasibility of this solution. Thanks, -js