From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Tomas Hlavaty Newsgroups: gmane.emacs.devel Subject: Re: continuation passing in Emacs vs. JUST-THIS-ONE Date: Fri, 17 Mar 2023 01:17:41 +0100 Message-ID: <87mt4c6xju.fsf@logand.com> References: <627090382.312345.1678539189382@office.mailbox.org> <87sfe7suog.fsf@gmail.com> <1c6fedae-10b4-5d97-5036-eaa736e1b816@gmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="13834"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Karthik Chikmagalur , Thomas Koch , "emacs-devel@gnu.org" To: Stefan Monnier , Jim Porter Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Mar 17 01:18:37 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pcxnp-0003Ov-2C for ged-emacs-devel@m.gmane-mx.org; Fri, 17 Mar 2023 01:18:37 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pcxnB-0000i5-Uf; Thu, 16 Mar 2023 20:17:57 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pcxnA-0000hh-9b for emacs-devel@gnu.org; Thu, 16 Mar 2023 20:17:56 -0400 Original-Received: from logand.com ([37.48.87.44]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pcxn8-0005Ok-C4 for emacs-devel@gnu.org; Thu, 16 Mar 2023 20:17:55 -0400 Original-Received: by logand.com (Postfix, from userid 1001) id BEE9619E65B; Fri, 17 Mar 2023 01:17:43 +0100 (CET) X-Mailer: emacs 28.1 (via feedmail 11-beta-1 I) In-Reply-To: Received-SPF: pass client-ip=37.48.87.44; envelope-from=tom@logand.com; helo=logand.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:304534 Archived-At: Hi Stefan, On Wed 15 Mar 2023 at 13:48, Stefan Monnier wrote: >> I seem to recall that Stefan Monnier (CCed) mentioned having some WIP >> code to make generator.el easier to use for asynchronous code... > I think my WiP thingy is very similar to emacs-aio. I haven't had > time to work on it and I'd welcome help with it (attached). Interesting. >From futur.el: > ;; (futur-let* > ;; (exitcode <- (futur-process-make :command cmd :buffer t)) > ;; (out (buffer-string)) ;; Get the process's output. > ;; (cmd2 (build-second-arg-list exitcode out)) > ;; (otherexit <- (futur-process-make :command cmd :buffer t))) > ;; (futur-pure (buffer-string))) Seems like beautiful lisp code has no futur. :-) There is something very ugly about this code. It looks like assembly, 1 dimensional vertical code. It is hard to see the structure of the code and what it actually does. I do not think it is practical to write non-trivial code in this style. Nice lisp code is usually 2 dimensional, with indentation and top-left to bottom-right direction. It is usually much clearer to see what is an argument to what based on the position in the syntax tree. Is it possible to make the syntax more structured (lispy)? Meaning tree-like, not list-like? Something in the spirit of: (futur-progn (futur-process-make :command (futur-let ((exitcode (futur-process-make :command (build-arg-list) :buffer t))) (build-second-arg-list exitcode (buffer-string))) :buffer t) (buffer-string)) or would it need some fancy syntax rewriting like other async/cps syntax rewriting libraries? Second question: I see that futur-wait blocks the whole emacs due to the while loop. How can one use futur without blocking emacs? I usually prefer pull based code as it does not steal control from me. Lets say I want to do something nontrivial but not block emacs. I would split computation into chunks, identify state explicitly and move it to the heap and suspend the computation without needing to reserve a stack for it. I.e. manually write a kind of stream that yields items or nil as EOF (without syntax rewriting ala generators.el). I need EAGAIN for stuff happening asynchronously. Stuff that blocks simply needs to be such a small chunk that it does not negatively affect emacs useability. Unfortunately futur.el does not have executable example so I'll invent one. Example (requires lexical bindings): Traverse filesystem, find *.el files and do something for each one (here I just count the length of the absolute path for simplicity). And the whole thing should not block emacs. (defun stream-pull-in-background (stream &optional secs repeat) (let (timer) (setq timer (run-with-timer (or secs 1) (or repeat 1) (lambda () ;;(message "@@@ polling!") (unless (funcall stream) (cancel-timer timer))))))) (defun line-stream (buffer) ;; yield buffer lines, follow process output if any (let (start) (lambda () (with-current-buffer buffer (save-excursion (unless start (setq start (point-min))) (goto-char start) (let ((end (line-beginning-position 2))) ;;(message "@@@ %s %s" start end) (if (< start end) (prog1 (buffer-substring-no-properties start (line-end-position 1)) (setq start end)) (let ((process (get-buffer-process buffer))) (if (and process (process-live-p process)) 'EAGAIN (let ((end (point-max))) (when (< start end) (prog1 (buffer-substring-no-properties start end) (setq start end))))))))))))) (defun burst-stream (stream &optional secs) ;; pull available data during SECS time window ;; this is very crude "scheduler" but keeps emacs mostly useable (let ((secs (or secs 0.2))) (lambda () (when secs (let ((z 'EAGAIN) (end (+ secs (float-time (current-time))))) ;;(message "@@@ burst %s %s:" (float-time (current-time)) end) (while (and (< (float-time (current-time)) end) (setq z (funcall stream)) (not (eq 'EAGAIN z)))) (unless z (setq secs nil)) z))))) (defun message2-stream (stream) (lambda () (let ((x (funcall stream))) (when x (unless (eq 'EAGAIN x) (message "@@@ %s %s" (length x) x)) x)))) (defun test-buffer (name) (let ((b (get-buffer-create name))) (with-current-buffer b (buffer-disable-undo) (erase-buffer)) b)) (defun test3 (buffer-name command) (stream-pull-in-background (let ((b (test-buffer buffer-name))) (make-process :name buffer-name :command command :buffer b) (burst-stream (message2-stream (line-stream b)))))) ;;(test3 "test3" '("cat" "/tmp/a.el")) ;;(test3 "test3" '("find" "/home/tomas/mr/" "-type" "f" "-name" "*.el")) ;;(list-processes) ;;(list-timers) Last question: How would similar functionality be implemented using futur? Cheers Tomas