RFC: rough draft of Python-style generators for elisp

RFC: rough draft of Python-style generators for elisp

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 949 bytes --]

Over at https://github.com/dcolascione/elisp-generators, I have a
pure-elisp implementation of Python-style generators for elisp.
Perhaps the feature is best illustrate by example:

(defgenerator mygen (i)
  (yield 1)
  (yield i)
  (yield 3))

(let ((gen (mygen 100)))
  (list  (funcall gen)
         (funcall gen)
         (funcall gen)))

-> (1 100 3)

Yields can appear in arbitrary code:

(defgenerator mygen2 (lim)
  (loop for x from 0 to lim do (yield x)))

The package works by rewriting elisp into continuation-passing form
and closing over the resulting continuations with a driver loop that
transitions from one continuation-state to the next. After the last
yield, the facility signals generator-ended.

Please take a look. It'd be nice if there were cl-loop extensions to
iterate over the things, and if generators were available with regular
defun the way yield is available with regular "def" in Python.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: RFC: rough draft of Python-style generators for elisp

[Burton Samograd]

Daniel Colascione <dancol@dancol.org> writes:

> Over at https://github.com/dcolascione/elisp-generators, I have a
> pure-elisp implementation of Python-style generators for elisp.
> Perhaps the feature is best illustrate by example:
>
> (defgenerator mygen (i)
>   (yield 1)
>   (yield i)
>   (yield 3))
>
> (let ((gen (mygen 100)))
>   (list  (funcall gen)
>          (funcall gen)
>          (funcall gen)))
>
> -> (1 100 3)
>
> Yields can appear in arbitrary code:
>
> (defgenerator mygen2 (lim)
>   (loop for x from 0 to lim do (yield x)))
>
> The package works by rewriting elisp into continuation-passing form
> and closing over the resulting continuations with a driver loop that
> transitions from one continuation-state to the next. After the last
> yield, the facility signals generator-ended.
>
> Please take a look. It'd be nice if there were cl-loop extensions to
> iterate over the things, and if generators were available with regular
> defun the way yield is available with regular "def" in Python.

When I try and byte-compile your package I get the following:

~/src/emacs-git/lisp/emacs-lisp/generators $ emacs -q --batch --eval "(byte-compile-file \"generator.el\")"
... <some gensym warnings> ...
generator.el:244:1:Warning: Argument "%s not supported in generators" is not a lexical variable
generator.el:244:1:Warning: Unused lexical argument `defvar'
generator.el:244:1:Warning: Unused lexical argument `error'
generator.el:244:1:Error: Invalid lambda variable %s not supported in generators

~/src/emacs-git/lisp/emacs-lisp/generators $ emacs --batch --eval "(print (emacs-version))"

"GNU Emacs 24.2.90.1 (i686-pc-cygwin, X toolkit, Xaw3d scroll bars)
 of 2012-11-27 on CDW764-BURTONS"

--
Burton Samograd

Re: RFC: rough draft of Python-style generators for elisp

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1469 bytes --]

On 11/27/12 9:31 AM, Burton Samograd wrote:
> Daniel Colascione <dancol@dancol.org> writes:
> 
>> Over at https://github.com/dcolascione/elisp-generators, I have a
>> pure-elisp implementation of Python-style generators for elisp.
>> Perhaps the feature is best illustrate by example:
>>
>> (defgenerator mygen (i)
>>   (yield 1)
>>   (yield i)
>>   (yield 3))
>>
>> (let ((gen (mygen 100)))
>>   (list  (funcall gen)
>>          (funcall gen)
>>          (funcall gen)))
>>
>> -> (1 100 3)
>>
>> Yields can appear in arbitrary code:
>>
>> (defgenerator mygen2 (lim)
>>   (loop for x from 0 to lim do (yield x)))
>>
>> The package works by rewriting elisp into continuation-passing form
>> and closing over the resulting continuations with a driver loop that
>> transitions from one continuation-state to the next. After the last
>> yield, the facility signals generator-ended.
>>
>> Please take a look. It'd be nice if there were cl-loop extensions to
>> iterate over the things, and if generators were available with regular
>> defun the way yield is available with regular "def" in Python.
> 
> When I try and byte-compile your package I get the following:

Thanks for trying the package. I've updated it to use pcase, and I
fixed the compilation issues by side effect. A few warnings about
unused lexical variables remain, but I'm convinced these warnings are
spurious.

Please let me know if you have any other problems.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: RFC: rough draft of Python-style generators for elisp

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1469 bytes --]

On 11/27/12 9:31 AM, Burton Samograd wrote:
> Daniel Colascione <dancol@dancol.org> writes:
> 
>> Over at https://github.com/dcolascione/elisp-generators, I have a
>> pure-elisp implementation of Python-style generators for elisp.
>> Perhaps the feature is best illustrate by example:
>>
>> (defgenerator mygen (i)
>>   (yield 1)
>>   (yield i)
>>   (yield 3))
>>
>> (let ((gen (mygen 100)))
>>   (list  (funcall gen)
>>          (funcall gen)
>>          (funcall gen)))
>>
>> -> (1 100 3)
>>
>> Yields can appear in arbitrary code:
>>
>> (defgenerator mygen2 (lim)
>>   (loop for x from 0 to lim do (yield x)))
>>
>> The package works by rewriting elisp into continuation-passing form
>> and closing over the resulting continuations with a driver loop that
>> transitions from one continuation-state to the next. After the last
>> yield, the facility signals generator-ended.
>>
>> Please take a look. It'd be nice if there were cl-loop extensions to
>> iterate over the things, and if generators were available with regular
>> defun the way yield is available with regular "def" in Python.
> 
> When I try and byte-compile your package I get the following:

Thanks for trying the package. I've updated it to use pcase, and I
fixed the compilation issues by side effect. A few warnings about
unused lexical variables remain, but I'm convinced these warnings are
spurious.

Please let me know if you have any other problems.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: RFC: rough draft of Python-style generators for elisp

[Levin Du]

[-- Attachment #1: Type: text/plain, Size: 2587 bytes --]

Hi Daniel,

My Emacs version is "GNU Emacs 24.3.50.1".

I tried:

(defgenerator mygen (i)
  (yield 1)
  (yield i)
  (yield 3))

(let ((gen (mygen 100)))
  (list  (funcall gen)
         (funcall gen)
         (funcall gen)))

but failed with:

Debugger entered--Lisp error: (void-variable cps-current-state-91642)
  (funcall cps-current-state-91642)
  (while t (funcall cps-current-state-91642))
  (catch (quote cps-yield) (while t (funcall cps-current-state-91642)))
  (lambda nil (catch (quote cps-yield) (while t (funcall
cps-current-state-91642))))()
  funcall((lambda nil (catch (quote cps-yield) (while t (funcall
cps-current-state-91642)))))
  (list (funcall gen) (funcall gen) (funcall gen))
  (let ((gen (mygen 100))) (list (funcall gen) (funcall gen) (funcall gen)))
  eval((let ((gen (mygen 100))) (list (funcall gen) (funcall gen) (funcall
gen))) nil)
  eval-last-sexp-1(nil)
  eval-last-sexp(nil)
  call-interactively(eval-last-sexp)
  ex-eval-region-or-sexp()
  call-interactively(ex-eval-region-or-sexp nil nil)



2012/11/28 Daniel Colascione <dancol@dancol.org>

> On 11/27/12 9:31 AM, Burton Samograd wrote:
> > Daniel Colascione <dancol@dancol.org> writes:
> >
> >> Over at https://github.com/dcolascione/elisp-generators, I have a
> >> pure-elisp implementation of Python-style generators for elisp.
> >> Perhaps the feature is best illustrate by example:
> >>
> >> (defgenerator mygen (i)
> >>   (yield 1)
> >>   (yield i)
> >>   (yield 3))
> >>
> >> (let ((gen (mygen 100)))
> >>   (list  (funcall gen)
> >>          (funcall gen)
> >>          (funcall gen)))
> >>
> >> -> (1 100 3)
> >>
> >> Yields can appear in arbitrary code:
> >>
> >> (defgenerator mygen2 (lim)
> >>   (loop for x from 0 to lim do (yield x)))
> >>
> >> The package works by rewriting elisp into continuation-passing form
> >> and closing over the resulting continuations with a driver loop that
> >> transitions from one continuation-state to the next. After the last
> >> yield, the facility signals generator-ended.
> >>
> >> Please take a look. It'd be nice if there were cl-loop extensions to
> >> iterate over the things, and if generators were available with regular
> >> defun the way yield is available with regular "def" in Python.
> >
> > When I try and byte-compile your package I get the following:
>
> Thanks for trying the package. I've updated it to use pcase, and I
> fixed the compilation issues by side effect. A few warnings about
> unused lexical variables remain, but I'm convinced these warnings are
> spurious.
>
> Please let me know if you have any other problems.
>
>

[-- Attachment #2: Type: text/html, Size: 4850 bytes --]

Re: RFC: rough draft of Python-style generators for elisp

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 585 bytes --]

On 11/29/12 1:41 AM, Levin Du wrote:
> Hi Daniel,
> 
> My Emacs version is "GNU Emacs 24.3.50.1".
> 
> I tried:
> 
> (defgenerator mygen (i)
>   (yield 1)
>   (yield i)
>   (yield 3))
> 
> (let ((gen (mygen 100)))
>   (list  (funcall gen)
>          (funcall gen)
>          (funcall gen)))
> 
> but failed with:
> 
> Debugger entered--Lisp error: (void-variable cps-current-state-91642)

lexical-binding needs to be t. Is it okay for defgenerator to let-bind
it at top-level as long as it preserves dynamic binding semantics for
the actual code generated?



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 235 bytes --]

Re: RFC: rough draft of Python-style generators for elisp

[Stefan Monnier]

> lexical-binding needs to be t. Is it okay for defgenerator to let-bind
> it at top-level as long as it preserves dynamic binding semantics for
> the actual code generated?

No, better signal an error if lexical-binding is non-nil.


        Stefan