From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: [Emacs-diffs] master f6b5db6: Add support for generators Date: Tue, 03 Mar 2015 13:05:23 -0500 Message-ID: References: <20150302234252.19883.48595@vcs.savannah.gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1425405970 16039 80.91.229.3 (3 Mar 2015 18:06:10 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 3 Mar 2015 18:06:10 +0000 (UTC) Cc: Daniel Colascione To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Mar 03 19:06:03 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1YSrCl-0006l6-2I for ged-emacs-devel@m.gmane.org; Tue, 03 Mar 2015 19:06:03 +0100 Original-Received: from localhost ([::1]:40233 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YSrCk-0005oy-HE for ged-emacs-devel@m.gmane.org; Tue, 03 Mar 2015 13:06:02 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:49288) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YSrCK-0005om-2j for emacs-devel@gnu.org; Tue, 03 Mar 2015 13:05:42 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YSrCE-0007R9-Rj for emacs-devel@gnu.org; Tue, 03 Mar 2015 13:05:35 -0500 Original-Received: from chene.dit.umontreal.ca ([132.204.246.20]:58900) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YSrCE-0007Qp-MU for emacs-devel@gnu.org; Tue, 03 Mar 2015 13:05:30 -0500 Original-Received: from fmsmemgm.homelinux.net (lechon.iro.umontreal.ca [132.204.27.242]) by chene.dit.umontreal.ca (8.14.1/8.14.1) with ESMTP id t23I5OGI025024; Tue, 3 Mar 2015 13:05:25 -0500 Original-Received: by fmsmemgm.homelinux.net (Postfix, from userid 20848) id D5D13AE187; Tue, 3 Mar 2015 13:05:23 -0500 (EST) In-Reply-To: (Daniel Colascione's message of "Mon, 02 Mar 2015 23:42:52 +0000") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) X-NAI-Spam-Flag: NO X-NAI-Spam-Level: X-NAI-Spam-Threshold: 5 X-NAI-Spam-Score: 0.3 X-NAI-Spam-Rules: 2 Rules triggered BDY_STRT_NGRT=0.3, RV5234=0 X-NAI-Spam-Version: 2.3.0.9393 : core <5234> : inlines <2342> : streams <1399371> : uri <1870104> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 132.204.246.20 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:183607 Archived-At: > Add support for generators > > diff --git a/doc/lispref/ChangeLog b/doc/lispref/ChangeLog [...] > + * automated/generator-tests.el: New tests The content of the commit log is fine, but the form sucks. Please get rid of the diff-cruft in those commit messages. > + A @dfn{generator} is a function that produces a potentially-infinite ^^^^^^^^ Do we want to document it as a function? Maybe it would be better to document it as an "object" of unspecified implementation (i.e. calling it via funcall rather than via iter-next is unsupported). > +@defmac iter-yield-from iterator > +@code{iter-yield-from} yields all the values that @var{iterator} > +produces and evaluates to the value that @var{iterator}'s generator > +function returns normally. While it has control, @var{iterator} > +receives sent to the iterator using @code{iter-next}. ^^^^^^^^^^^^^ Some word(s) is missing here. > + To use a generator function, first call it normally, producing a > +@dfn{iterator} object. Ah, right, so that's why you defined generators as functions. OK, now that makes sense. Maybe this description of the difference between a generator and an iterator should be placed earlier, tho. > +@defun iter-close iterator > +If @var{iterator} is suspended inside a @code{unwind-protect} and > +becomes unreachable, Emacs will eventually run unwind handlers after a > +garbage collection pass. To ensure that these handlers are run before > +then, use @code{iter-close}. > +@end defun Hmm... but earlier you said: +@code{iter-yield} and @code{iter-yield-from} cannot appear inside +@code{unwind-protect} forms. Oh, wait, I think I understand: we can use iter-yield inside the main body of unwind-protect but not inside the unwind forms, right? I think this should be clarified. To get back to iter-close, this seems to be a tricky aspect of the implementation. Could you give me more details, such as an example problematic case where calling iter-close makes a difference? I guess this has to do with those finalizers, so if you could explain how/why these are used, I'd really appreciate it. > +@example > +(iter-defun my-iter (x) > + (iter-yield (1+ (iter-yield (1+ x)))) > + -1 ;; Return normally > + ) Please don't leave a close-paren alone on its line in such example code. > - * vc/vc.el (vc-responsible-backend): Add autoload cooking for > + * vc/vc.el (vc-responsible-backend): Add autoload cookie for Thanks ;-) > +(defvar *cps-bindings* nil) > +(defvar *cps-states* nil) > +(defvar *cps-value-symbol* nil) > +(defvar *cps-state-symbol* nil) > +(defvar *cps-cleanup-table-symbol* nil) > +(defvar *cps-cleanup-function* nil) > + > +(defvar *cps-dynamic-wrappers* '(identity) > + "List of transformer functions to apply to atomic forms we > +evaluate in CPS context.") This CL naming style is not used in Elisp. I.e. just drop the surrounding stars. > + (let* ((state (cl-gensym (format "cps-state-%s-" kind)))) ^^^^^^^^^ Elisp prefers make-symbol. > +(defvar cps-disable-atomic-optimization nil We usually use "inhibit" rather than "disable". > + ;; Unfortunately, because elisp lacks a mechanism for generically > + ;; capturing the reason for an arbitrary non-local control > + ;; transfer and restarting the transfer at a later point, we > + ;; cannot reify non-local transfers and cannot allow > + ;; continuation-passing code inside UNWINDFORMS. I think it's an acceptable limitation. Unwind-forms should be "short and to the point". We actually should run them with inhibit-quit (tho we currently don't, which leads to some problems when the user hits C-g repeatedly in panic). > +(put 'iter-end-of-sequence 'error-conditions '(iter-end-of-sequence)) > +(put 'iter-end-of-sequence 'error-message "iteration terminated") This should use the newish `define-error'. > +(defmacro iter-yield-from (value) > + "When used inside a generator function, delegate to a sub-iterator. > +The values that the sub-iterator yields are passed directly to > +the caller, and values supplied to `iter-next' are sent to the > +sub-iterator. `iter-yield-from' evaluates to the value that the > +sub-iterator function returns via `iter-end-of-sequence'." > + (let ((errsym (cl-gensym "yield-from-result")) > + (valsym (cl-gensym "yield-from-value"))) > + `(let ((,valsym ,value)) > + (unwind-protect > + (condition-case ,errsym > + (let ((vs nil)) > + (while t > + (setf vs (iter-yield (iter-next ,valsym vs))))) > + (iter-end-of-sequence (cdr ,errsym))) > + (iter-close ,valsym))))) [Mostly out of curiosity:] Could this be implemented as `(iter--blabla ,value (lambda (x) (iter-yield x))) > +(defmacro iter-defun (name arglist &rest body) > + "Creates a generator NAME. > +When called as a function, NAME returns an iterator value that > +encapsulates the state of a computation that produces a sequence > +of values. Callers can retrieve each value using `iter-next'." > + (declare (indent defun)) > + (cl-assert lexical-binding) > + `(defun ,name ,arglist > + ,(cps-generate-evaluator > + `(cl-macrolet ((iter-yield (value) `(cps-internal-yield ,value))) > + ,@body)))) IIUC you don't support any declarations between ARGLIST and BODY. You could use macroexp-parse-body for that. In cps-generate-evaluator I see you do: > + (macroexpand-all form) IIUC this will lose any local macro stored in macroexpand-all-environment. IOW you need to pass macroexpand-all-environment to this macroexpand-all call. Also, both calls to cps-generate-evaluator look identical, so I think it'd be better to just pass `body' to cps-generate-evaluator, and then have cps-generate-evaluator handle the local macro definition of iter-yield. > +(eval-after-load 'elisp-mode > + (lambda () > + (font-lock-add-keywords > + 'emacs-lisp-mode > + '(("(\\(iter-defun\\)\\_>\\s *\\(\\(?:\\sw\\|\\s_\\)+\\)?" > + (1 font-lock-keyword-face nil t) > + (2 font-lock-function-name-face nil t)) > + ("(\\(iter-next\\)\\_>" > + (1 font-lock-keyword-face nil t)) > + ("(\\(iter-lambda\\)\\_>" > + (1 font-lock-keyword-face nil t)) > + ("(\\(iter-yield\\)\\_>" > + (1 font-lock-keyword-face nil t)) > + ("(\\(iter-yield-from\\)\\_>" > + (1 font-lock-keyword-face nil t)))))) Using fewer entries (at most 2) would make this more efficient (especially since they all start with "(iter-)"). Stefan