From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Tomas Hlavaty Newsgroups: gmane.emacs.devel Subject: Re: continuation passing in Emacs vs. JUST-THIS-ONE Date: Tue, 11 Apr 2023 21:59:18 +0200 Message-ID: <874jpmfaw9.fsf@logand.com> References: <87leizif4r.fsf@logand.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33692"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Jim Porter , Karthik Chikmagalur , "emacs-devel@gnu.org" To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Apr 11 22:00:32 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pmKAK-0008bM-C4 for ged-emacs-devel@m.gmane-mx.org; Tue, 11 Apr 2023 22:00:32 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pmK9J-0007Rm-KQ; Tue, 11 Apr 2023 15:59:29 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pmK9H-0007RZ-OT for emacs-devel@gnu.org; Tue, 11 Apr 2023 15:59:27 -0400 Original-Received: from logand.com ([37.48.87.44]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pmK9C-0006Lz-Qg for emacs-devel@gnu.org; Tue, 11 Apr 2023 15:59:25 -0400 Original-Received: by logand.com (Postfix, from userid 1001) id 1388219E65B; Tue, 11 Apr 2023 21:59:20 +0200 (CEST) X-Mailer: emacs 28.2 (via feedmail 11-beta-1 I) In-Reply-To: Received-SPF: pass client-ip=37.48.87.44; envelope-from=tom@logand.com; helo=logand.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:305251 Archived-At: On Mon 10 Apr 2023 at 22:53, Stefan Monnier wrot= e: >>> And the use `await` above means that your Emacs will block while waiting >>> for one result. `futur-let*` instead lets you compose async operations >>> without blocking Emacs, and thus works more like Javascript's `await`. >> Blocking the current thread for one result is fine, because all the >> futures already run in other threads in "background" so there is nothing >> else to do. > > You can't know that. There can be other async processes whose > filters should be run, timers to be executed, other threads to run, > ... I do know that, because in this case I have been talking about the specific case where the future runs in a thread, i.e. my async-thread and await-thread code. Looks like similar use-case partially sketched in futur-wait, unfortunately without working code to try. >> If you mean that you want to use the editor at the same time, just run >> the example in another thread. > > The idea is to use `futur.el` *instead of* threads. What do you mean? I can see thread-join and thread-signal in futur.el. It is useful to acknowledge, that there are 3 different use-cases: a) asynchronous processes b) threads c) iter My impression was that futur.el was trying to address a) and b) but now you say it does address a) only. That is rather limited. >> No, the iter case does map directly to futures: >> >> (await >> (async-iter >> (let ((a (async-iter >> (message "a1") >> (await-iter (sleep-iter3 3)) >> (message "a2") >> 1)) >> (b (async-iter >> (message "b1") >> (let ((c (async-iter >> (message "c1") >> (await-iter (sleep-iter3 3)) >> (message "c2") >> 2))) >> (message "b2") >> (+ 3 (await-iter c)))))) >> (+ (await-iter a) (await-iter b))))) > > I must say I don't understand this example: in which sense is it using > "iter"? I don't see any `iter-yield`. await-iter and async-iter macros are using iter under the hood. One could use iter-yield explicitly too but that example does not need to do that. I have already sent the definitions of those macros earlier but here the whole thing self-contained: ;;; -*- lexical-binding: t -*- (defmacro async-iter (&rest body) (declare (indent 0)) `(let ((i (iter-make (iter-yield (progn ,@body)))) (z 'EAGAIN)) (setq z (iter-next i)) (lambda () (when (eq 'EAGAIN z) (setq z (iter-next i))) (unless (eq 'EAGAIN z) (iter-close i)) z))) (defmacro await-iter (future) (let ((f (gensym))) `(let ((,f ,future)) (let (z) (while (eq 'EAGAIN (setq z (funcall ,f))) (iter-yield 'EAGAIN)) z)))) (defun sleep-iter3 (sec) (async-iter (let ((end (+ (float-time (current-time)) sec))) (while (< (float-time (current-time)) end) (iter-yield 'EAGAIN))) (iter-yield sec))) (defun await-in-background (future &optional callback secs repeat) (let ((z (funcall future))) (if (eq 'EAGAIN z) (let (timer) (setq timer (run-with-timer (or secs 0.2) (or repeat 0.2) (lambda () (let ((z (funcall future))) (unless (eq 'EAGAIN z) (cancel-timer timer) (when callback (funcall callback z)))))))) (when callback (funcall callback z))) z)) (await-in-background (async-iter (let ((a (async-iter (message "a1") (await-iter (sleep-iter3 3)) (message "a2") 1)) (b (async-iter (message "b1") (let ((c (async-iter (message "c1") (await-iter (sleep-iter3 3)) (message "c2") 2))) (message "b2") (+ 3 (await-iter c)))))) (message "Result =3D %s" (+ (await-iter a) (await-iter b)))))) > `futur.el` also "queues the continuations in the event loop". I get: futur.el:97:8: Warning: the function =E2=80=98funcall-later=E2=80=99 is not= known to be defined. >>>> Calling await immediately after async is useless (simply use blocking >>>> call). The point of future is to make the distance between those calls >>>> as big as possible so that the sum of times in the sequential case is >>>> replaced with max of times in the parallel case. >>> You're looking for parallelism. I'm not. >> What do you mean exactly? > > That `futur.el` is not primarily concerned with allowing you to run > several subprocesses to exploit your multiple cores. It's instead > primarily concerned with making it easier to write asynchronous code. > > One of the intended use case would be for completion tables to return > futures (which, in many cases, will have already been computed > synchronously, but not always). > >> I am asking because: >> >> https://wiki.haskell.org/Parallelism_vs._Concurrency >> >> Warning: Not all programmers agree on the meaning of the terms >> 'parallelism' and 'concurrency'. They may define them in different >> ways or do not distinguish them at all. > > Yet I have never heard of anyone disagree with the definitions given at > the beginning of that very same page. More specifically those who may > disagree are those who didn't know there was a distinction :-) strawman I was asking in order to understand why you dismissed my examples by saying: You're looking for parallelism. I'm not. It looks to me that the reason is that futur.el cannot do those things demonstrated in my examples. >> But it seems that you insist on composing promises sequentially: > > No, I'm merely making it easy to do that. > >> Also futur.el does seems to run callbacks synchronously: > > I don't think so: it runs them via `funcall-later`. > >> In this javascript example, a and b appear to run in parallel (shall I >> say concurrently?): >> >> function sleep(sec) { >> return new Promise(resolve =3D> { >> setTimeout(() =3D> {resolve(sec);}, sec * 1000); >> }); >> } >> async function test() { >> const a =3D sleep(9); >> const b =3D sleep(8); >> const z =3D await a + await b; >> console.log(z); >> } >> test(); >> >> Here the console log will show 17 after 9sec. >> It will not show 17 after 17sec. >> >> Can futur.el do that? > > Of course. You could do something like > > (futur-let* > ((a (futur-let* ((_ <- (futur-process-make > :command '("sleep" "9")))) > 9)) > (b (futur-let* ((_ <- (futur-process-make > :command '("sleep" "8")))) > 8)) > (a-val <- a) > (b-val <- b)) > (message "Result =3D %s" (+ a-val b-val)))) So will futur.el take 9sec or 17sec? I cannot test this because it does not work: I get: Debugger entered--Lisp error: (wrong-type-argument stringp nil) #(:sentinel #f(compiled-function (proc state) #) :command ("sleep" "9")) make-process--with-editor-process-filter(# :sentinel #= f(compiled-function (proc state) #) :command = ("sleep" "9")) apply(make-process--with-editor-process-filter # (:sen= tinel #f(compiled-function (proc state) #) :c= ommand ("sleep" "9"))) make-process(:sentinel #f(compiled-function (proc state) #) :command ("sleep" "9")) apply(make-process :sentinel #f(compiled-function (proc state) #) (:command ("sleep" "9"))) #f(compiled-function (f) #)(#s(futur :clients= nil :value nil)) futur-new(#f(compiled-function (f) #)) futur-process-make(:command ("sleep" "9")) (futur-bind (futur-process-make :command '("sleep" "9")) #'(lambda (_) 9)) (let ((a (futur-bind (futur-process-make :command '("sleep" "9")) #'(lamb= da (_) 9)))) (let ((b (futur-bind (futur-process-make :command '("sleep" "8= ")) #'(lambda (_) 8)))) (futur-bind a #'(lambda (a-val) (futur-bind b #'(la= mbda ... ...)))))) (progn (let ((a (futur-bind (futur-process-make :command '("sleep" "9")) = #'(lambda (_) 9)))) (let ((b (futur-bind (futur-process-make :command '...)= #'(lambda ... 8)))) (futur-bind a #'(lambda (a-val) (futur-bind b #'...)))= ))) (setq elisp--eval-defun-result (progn (let ((a (futur-bind (futur-process= -make :command '...) #'(lambda ... 9)))) (let ((b (futur-bind (futur-proces= s-make :command ...) #'...))) (futur-bind a #'(lambda (a-val) (futur-bind b= ...))))))) elisp--eval-defun() eval-defun(nil) funcall-interactively(eval-defun nil) command-execute(eval-defun) >> I think that your confusion is caused by the decision that >> futur-process-make yields exit code. That is wrong, exit code is >> logically not the resolved value (promise resolution), it indicates >> failure (promise rejection). > > Not necessarily, it all depends on what the process is doing. > > Similarly the "intended return value" of a process will depend on what > the process does. In some cases it will be the stdout, but I see no > reason to restrict my fundamental function to such a choice. This overgeneralized thinking is beyond usefulness and harmfully leads to the problem of how to maintain state. It is better to have the process future return the output on success and error on failure. The error can contain the error code which covers the specialized use-case you were after. > It's easy to build on top of `futur-process-make` a higher-level > function which returns the stdout as the result of the future. It might be easy but unnecessary. That overgeneralized thinking also lead you astray: From: Stefan Monnier Date: Thu, 16 Mar 2023 23:08:25 -0400 id:jwvpm98nlqz.fsf-monnier+emacs@gnu.org BTW the above code can't work right now. Part of the issue is the management of `current-buffer`: should the composition of futures with `futur-let*` save&restore `current-buffer` to mimic more closely the behavior one would get with plain old sequential execution? If so, should we do the same with `point`? What about other such state? It is better not to open this can of worms.