[ANN] New library stream.el in ELPA

[ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 1632 bytes --]

Hi,

I just pushed to ELPA the first version of `stream', a new library that
provides an implementation of streams. Streams are implemented as
delayed evaluation of cons cells, and are immutable. In that sense they
are similar to streams in the SICP book[1] or to Clojure's lazy
sequences.

`stream' requires Emacs >= 25.1, as it leverages the extensibility of
seq.el (the version currently in master, not the one in ELPA): all
functions defined in seq.el will work on streams, meaning that it's
possible to consume a stream using `seq-take', map and filter it using
`seq-map` and `seq-filter`, and so forth.

Streams could be created from any sequential input data:
 - sequences, making operation on them lazy
 - a set of 2 forms (first and rest), making it easy to represent infinite sequences
 - buffers (by character)
 - buffers (by line)
 - buffers (by page)
 - IO streams
 - orgmode table cells
 - ...

The generic `stream' function currently accepts lists, strings, arrays and
buffers as input, but it can be cleanly extended to support pretty much any kind
of data.

All operations on streams are lazy (including the functions from seq.el), unless
data is actually needed.

Here is an example implementation of the Fibonacci numbers
implemented as in infinite stream:

    (defun fib (a b)
      (stream-cons a (fib b (+ a b))))
    (fib 0 1)

The following example returns a stream of the first 50 characters of the
current buffer:

    (seq-take (stream (current-buffer)) 50)
      

Cheers,
Nico

[1] https://mitpress.mit.edu/sicp/full-text/book/book-Z-H-24.html#%_sec_3.5

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[Michael Heerdegen]

Nicolas Petton <nicolas@petton.fr> writes:

> I just pushed to ELPA the first version of `stream', a new library that
> provides an implementation of streams. Streams are implemented as
> delayed evaluation of cons cells, and are immutable. In that sense they
> are similar to streams in the SICP book[1] or to Clojure's lazy
> sequences.

FWIW, I implemented quite the same some years ago, but never got happy
with it.

The problem was that streams defined via delayed conses end up to be
massively nested lambda structures.  Emacs has no tail recursion, and
sooner or later, I always ended with very simple examples, like
computing the 500nth prime in the stream of primes, hitting the
`max-lisp-eval-depth' limit.  I don't recall the details of a
counterexample, but I ended up dismissing the approach completely,
because it was hopeless.

Then I switched to working with generator.el, and defined a different
form of streams I called "ilists".  These are like normal lists, but
(speaking simplified) the last cdr contains a rule to compute more
elements on the fly when they are referenced.  That approach gives you a
quite similar kind of semantical thing offering the same functionality,
without the deadly recursion problem.

My fear is that you will soon see `max-lisp-eval-depth' related errors
when doing very simple things with your streams and trying to reference
element 500 or so.  Sorry, but I really don't recall a good
counterexample.  It would be great if I'm proven wrong by your
implementation, but I'm not optimistic.

I tried with your package

(defun fib (a b)
      (stream-cons a (fib b (+ a b))))

(seq-elt (fib 0 1) 10)

but got the error

unless: Symbol’s value as variable is void: #:forced


Regards,

Michael.

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 318 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I tried with your package
>
> (defun fib (a b)
>       (stream-cons a (fib b (+ a b))))
>
> (seq-elt (fib 0 1) 10)
>
> but got the error
>
> unless: Symbol’s value as variable is void: #:forced

Have you tried turning on lexical binding?

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[Michael Heerdegen]

Nicolas Petton <nicolas@petton.fr> writes:

> > unless: Symbol’s value as variable is void: #:forced
>
> Have you tried turning on lexical binding?

Not in the Gnus Article buffer ;-)

That was it, the example works now - thanks.


Regards,

Michael.


P.S. BTW, forgot to say that the ilist def is "iterators.el".

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 531 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> My fear is that you will soon see `max-lisp-eval-depth' related errors
> when doing very simple things with your streams and trying to reference
> element 500 or so.  Sorry, but I really don't recall a good
> counterexample.

That could happen if you tried to read an entire large stream at once,
but why would you do that, and not use instead a sequence?

If you do stream it, using `stream-first' and `stream-rest' for
instance, you shouldn't have this issue, right?

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[Michael Heerdegen]

Nicolas Petton <nicolas@petton.fr> writes:

> If you do stream it, using `stream-first' and `stream-rest' for
> instance, you shouldn't have this issue, right?

I don't remember.  Maybe your implementation is cleverer than mine was
;-)

I tried this stress test with your lib:

--8<---------------cut here---------------start------------->8---
(defun stream-append (s1 s2)
  (if (stream-empty-p s1) s2
      (stream-cons
       (stream-first s1)
       (stream-append (stream-rest s1) s2))))

(seq-elt (stream-append
          (stream (cl-loop for i from 1 to 100 collect i))
          (seq-filter
           #'cl-oddp
           (stream-append
            (stream (cl-loop for i from 1 to 50000 collect i))
            (seq-map #'1+ (stream (cl-loop for i from 50001 to 100001 collect i))))))
         40000)
--8<---------------cut here---------------end--------------->8---

What can I say - it work well!

BTW, is `stream-append' missing in your library, or should that be done
via some `seq' function?


Regards,

Michael.

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 1146 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I tried this stress test with your lib:
>
> --8<---------------cut here---------------start------------->8---
> (defun stream-append (s1 s2)
>   (if (stream-empty-p s1) s2
>       (stream-cons
>        (stream-first s1)
>        (stream-append (stream-rest s1) s2))))
>
> (seq-elt (stream-append
>           (stream (cl-loop for i from 1 to 100 collect i))
>           (seq-filter
>            #'cl-oddp
>            (stream-append
>             (stream (cl-loop for i from 1 to 50000 collect i))
>             (seq-map #'1+ (stream (cl-loop for i from 50001 to 100001 collect i))))))
>          40000)
> --8<---------------cut here---------------end--------------->8---
>
> What can I say - it work well!

:-)

>
> BTW, is `stream-append' missing in your library, or should that be done
> via some `seq' function?

It could be done with `seq-concatenate', but it would not be lazy, and
would return a sequence (consuming the entire stream!).  Adding
`stream-append' would indeed make sense.

The repository is https://github.com/NicolasPetton/stream, if you want
to send me a patch.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[John Wiegley]

>>>>> Nicolas Petton <nicolas@petton.fr> writes:

> Here is an example implementation of the Fibonacci numbers implemented as in
> infinite stream:

>     (defun fib (a b)
>       (stream-cons a (fib b (+ a b))))
>     (fib 0 1)

How bizarre, just yesterday I started a file called "thunks.el" to generalize
this kind of behavior. Since I didn't get very far, I'm glad to see the idea
taking shape elsewhere!

What I wonder is, can you generalize this beyond cons cells?  I mean, can we
have a general lazy evaluation framework, instead of just lazy lists?  The API
I was thinking of was:

    (let ((x (thunk-create FORM)))
      (thunk-eval x))

The idea being that 'x' is a self-modifying thunk which only evaluates FORM
the first time it is forced, but immediately returns the result value after.

I think that stream-cons and seq-take could be implemented on top of this
lower-level behavior.

John

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 1028 bytes --]

John Wiegley <johnw@newartisans.com> writes:

> How bizarre, just yesterday I started a file called "thunks.el" to generalize
> this kind of behavior. Since I didn't get very far, I'm glad to see the idea
> taking shape elsewhere!
>
> What I wonder is, can you generalize this beyond cons cells?  I mean, can we
> have a general lazy evaluation framework, instead of just lazy lists?  The API
> I was thinking of was:
>
>     (let ((x (thunk-create FORM)))
>       (thunk-eval x))
>
> The idea being that 'x' is a self-modifying thunk which only evaluates FORM
> the first time it is forced, but immediately returns the result value after.
>
> I think that stream-cons and seq-take could be implemented on top of this
> lower-level behavior.

stream.el is based on lazy form evaluation with memoization (which is
exactly what you describe IIUC), that I didn't need to expose it so I
kept it private to the library, see `stream--delay' and `stream-force':

https://github.com/NicolasPetton/stream/blob/master/stream.el#L62

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[John Wiegley]

>>>>> Nicolas Petton <nicolas@petton.fr> writes:

> stream.el is based on lazy form evaluation with memoization (which is
> exactly what you describe IIUC), that I didn't need to expose it so I kept
> it private to the library, see `stream--delay' and `stream-force':

> https://github.com/NicolasPetton/stream/blob/master/stream.el#L62

What if you expose the lower-level API, and then make lazy streaming a useful
example. I.e., call your module "thunk.el" or "lazy.el".

I would have uses for such a library beyond streams of value: for example,
querying an external process for a tidbit of information, but abstracting this
interaction to appear as a "value" within a thunk.

John

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 998 bytes --]

John Wiegley <johnw@newartisans.com> writes:

>>>>>> Nicolas Petton <nicolas@petton.fr> writes:
>
>> stream.el is based on lazy form evaluation with memoization (which is
>> exactly what you describe IIUC), that I didn't need to expose it so I kept
>> it private to the library, see `stream--delay' and `stream-force':
>
>> https://github.com/NicolasPetton/stream/blob/master/stream.el#L62
>
> What if you expose the lower-level API, and then make lazy streaming a useful
> example. I.e., call your module "thunk.el" or "lazy.el".

What if I extract it in a thunk.el library and install it in master
(with proper tests), and have stream.el use that? (I could also put
stream.el in Emacs, but Stefan thought it would be better to have it in
ELPA).

> I would have uses for such a library beyond streams of value: for example,
> querying an external process for a tidbit of information, but abstracting this
> interaction to appear as a "value" within a thunk.

Yes, that would be really nice!

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[John Wiegley]

>>>>> Nicolas Petton <nicolas@petton.fr> writes:

> What if I extract it in a thunk.el library and install it in master (with
> proper tests), and have stream.el use that? (I could also put stream.el in
> Emacs, but Stefan thought it would be better to have it in ELPA).

Probably I'd prefer to see thunk.el in master, and stream.el in ELPA, since
the latter is a bit more special-cased and "implemented in terms of core". If
it ends up proving to be a valuable core addition, we can always bring it into
master later.

We should talk about where to draw "the ELPA divide" at some point too. Core
Emacs sort of has at least two separate roles: Functionality the Emacs
developers have standardized on -- as a common framework to build packages
upon -- and packages that should really be installed anywhere Emacs is
installed.

John

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]

John Wiegley <johnw@newartisans.com> writes:

>>>>>> Nicolas Petton <nicolas@petton.fr> writes:
>
>> What if I extract it in a thunk.el library and install it in master (with
>> proper tests), and have stream.el use that? (I could also put stream.el in
>> Emacs, but Stefan thought it would be better to have it in ELPA).
>
> Probably I'd prefer to see thunk.el in master, and stream.el in ELPA, since
> the latter is a bit more special-cased and "implemented in terms of
> core".

Agreed.

> We should talk about where to draw "the ELPA divide" at some point too. Core
> Emacs sort of has at least two separate roles: Functionality the Emacs
> developers have standardized on -- as a common framework to build packages
> upon -- and packages that should really be installed anywhere Emacs is
> installed.

I think that would be a fair definition of "the ELPA divide".

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[raman]

Speaking from the perspective of building  Emacspeak, I only feel
comfortable using modules that are already in Core, since I dont want
the average emacspeak user to have to pull things from elpa to get the
system started up. From that perspective, I would prefer to see
streams.el in Master --- though I could likely implement some of what I
was thinking of e.g., rapid interactive browsing of content  by chunks,
where the chunk can be interactively determined etc using think.el just
as much as streams.el -- though streams is likely  closer to what I
would use than the lower  level thunk in that case.
-- 

Re: [ANN] New library stream.el in ELPA

[John Wiegley]

>>>>> raman  <raman@google.com> writes:

> Speaking from the perspective of building Emacspeak, I only feel comfortable
> using modules that are already in Core, since I dont want the average
> emacspeak user to have to pull things from elpa to get the system started
> up.

Raman, with due respect to your tremendous efforts on Emacspeak, the way that
non-Core packages are distributed should not have a bearing on what does and
does not go into Core; otherwise, we would be making our choices based on
undefinable criteria, like which packages are the most popular.

If a package is in ELPA, it can just as easily download its dependencies as
itself; if it is not in ELPA, it is outside the scope of this discussion.

John

Re: [ANN] New library stream.el in ELPA

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1486 bytes --]

On 10/14/2015 02:31 PM, John Wiegley wrote:
>>>>>> Nicolas Petton <nicolas@petton.fr> writes:
> 
>> What if I extract it in a thunk.el library and install it in master (with
>> proper tests), and have stream.el use that? (I could also put stream.el in
>> Emacs, but Stefan thought it would be better to have it in ELPA).
> 
> Probably I'd prefer to see thunk.el in master, and stream.el in ELPA, since
> the latter is a bit more special-cased and "implemented in terms of core". If
> it ends up proving to be a valuable core addition, we can always bring it into
> master later.

I prefer keeping code in master. That way, it's easily maintained and
distributed automatically. There's very little downside: nobody is going
to miss a few tens of kilobytes of compiled elisp.

> We should talk about where to draw "the ELPA divide" at some point too. Core
> Emacs sort of has at least two separate roles: Functionality the Emacs
> developers have standardized on -- as a common framework to build packages
> upon -- and packages that should really be installed anywhere Emacs is
> installed.

I see no need to flake off useful parts of Emacs and put them into ELPA
when the core is not resource-constrained.

For me, using ELPA packages is a bit more awkward, since I don't use
package.el and pull in the bits of ELPA I want manually. If your package
is only in ELPA, I probably won't hear about it, and unless it's
particularly compelling, I won't use it.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [ANN] New library stream.el in ELPA

[Stephen J. Turnbull]

Daniel Colascione writes:

 > I prefer keeping code in master.

Hackers always do.  The vast majority of users aren't hackers,
though.  The vast majority of that vast majority (the half-vast
majority?) loves community package repositories, and for good reason.

N.B.  I have no opinion on stream.el itself.  My opinion is that these
stdlib/package repo issues need to be decided case by case, and that in
general a bias toward putting stuff in ELPA is not a bad thing.  A
bias toward putting everything in core should be avoided in the
current state of the art.

 > That way, it's easily maintained and distributed automatically.

Assuming it's maintained at all, and FVO "distributed" == "to the
hackers who pull from master daily", yes.  To industrial users who get
their core Emacs from Debian or Red Hat, not so.

 > There's very little downside: nobody is going to miss a few tens of
 > kilobytes of compiled elisp.

True but irrelevant, even when the specious "few kilobytes" (due to
one small module) is replaced by the more realistic "many megabytes"
(due to a plethora of small modules and a few monster packages).
There is a lot of experience with standard libraries nowadays.  A
short list of the real tradeoffs involves:

1.  Obsolescence/bitrot of stdlib code.

2.  Maintainer effort to prevent/remediate (1).

3.  Users who need to keep a few modules up to date, but prefer to get
    their core Emacs + from an OS distro.

4.  Developers who always have a current build of master to hand, and
    can easily rollback using the VCS if that breaks, vs. (3).

5.  Paranoid enterprise environments where ordinary programmers have
    to choose software from a list approved by HQ, and there's no
    blanket approval for the package repository.  (Such users' best
    hope for access to new features is a short release cycle that fits
    the convenience of the enterprise's software audit team.)  This is
    really the killer application for "everything in core", but I
    haven't heard this claim in Emacs channels (and the data I've seen
    on distribution of versions in a couple of large enterprises
    suggests that a large majority of industrial users are perfectly
    happy with 5-year-old Emacsen, and a significant minority often
    uses 10-year-old Emacsen).

6.  Variations in development cycles between core and modules, and
    among modules.  Similarity in cycles argues for joint release and
    distribution.

7.  Discoverability of modules in the stdlib is probably higher,
    although as we've seen in several threads recently, even those
    with great interest in the subject can be sadly underinformed
    about the elephant in the room (eg CEDET/EDE).

 > I see no need to flake off useful parts of Emacs and put them into ELPA
 > when the core is not resource-constrained.

You haven't noticed that there just aren't enough developers to fix
all the bugs yet, let alone develop all of the attractive proposals?
That there's only one volunteer for lead maintainer, and he's
politically nonaligned in a flagship product of a political
organization (not to mention having reservations about top management
policy)?[1]

8.  Of course to some extent those resource constraints argue for
    putting everything in core for the convenience of those who *do*
    do a lot of maintenance work on core.

 > For me, using ELPA packages is a bit more awkward,

Emacs is not just about you and other core developers whose practices
are basically similar to yours, though.

 > since I don't use package.el and pull in the bits of ELPA I want
 > manually. If your package is only in ELPA, I probably won't hear
 > about it, and unless it's particularly compelling, I won't use it.

You may be more diligent, but Emacs (like other sprawling language and
framework projects I know of) is such a mess[2] that most hackers won't
use core code if they have to look for it.  It's easier to build your
own, submit/commit, and let Stefan et al. tell you there's a standard
way that also has benefits X, Y, and Z for your application.

9.  This is often a good thing, as the existing code may have
    limitations or even be quite inappropriate for the application to
    hand.  Such experimentation is encouraged by package repos.

I could go on, but I'll stop at 9, according to Guido's Rule of Single
Digits.


Footnotes: 
[1]  I expect these issues to be resolved satisfactorily to all
parties.  But I wonder if John would have stepped forward if someone
who is an enthusiastic advocate of the GNU approach to software
freedom had their hat in the ring?

[2]  Organization of modules is worse than NP-hard.  I have found a
wonderful proof of this theorem but unfortunately it won't fit in a
4-line .sig.

Re: [ANN] New library stream.el in ELPA

[John Wiegley]

>>>>> Stephen J Turnbull <stephen@xemacs.org> writes:

> Daniel Colascione writes:
>> I prefer keeping code in master.

> N.B. I have no opinion on stream.el itself. My opinion is that these
> stdlib/package repo issues need to be decided case by case, and that in
> general a bias toward putting stuff in ELPA is not a bad thing. A bias
> toward putting everything in core should be avoided in the current state of
> the art. [...]

Stephen summed up what I wanted to say quite nicely. Thank you!

Although we have no hard and fast rule yet for what goes into core and ELPA, I
expect moving things into core to offer a better reason than "because I want
it there". Such a package should provide:

  - a basic functionality we expect to be commonly used,
  - supports a package already in core,
  - provides some justification for being in the default distribution and
    thus subject to maintenance by the core maintainers.

If it can't answer those questions, it should be in ELPA.

John

Re: [ANN] New library stream.el in ELPA

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1096 bytes --]

On 10/19/2015 11:55 PM, John Wiegley wrote:
>>>>>> Stephen J Turnbull <stephen@xemacs.org> writes:
> 
>> Daniel Colascione writes:
>>> I prefer keeping code in master.
> 
>> N.B. I have no opinion on stream.el itself. My opinion is that these
>> stdlib/package repo issues need to be decided case by case, and that in
>> general a bias toward putting stuff in ELPA is not a bad thing. A bias
>> toward putting everything in core should be avoided in the current state of
>> the art. [...]
> 
> Stephen summed up what I wanted to say quite nicely. Thank you!
> 
> Although we have no hard and fast rule yet for what goes into core and ELPA, I
> expect moving things into core to offer a better reason than "because I want
> it there". Such a package should provide:
> 
>   - a basic functionality we expect to be commonly used,
>   - supports a package already in core,
>   - provides some justification for being in the default distribution and
>     thus subject to maintenance by the core maintainers.

So, today, M-x butterfly wouldn't have made the cut? That's sad.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [ANN] New library stream.el in ELPA

[John Wiegley]

>>>>> Daniel Colascione <dancol@dancol.org> writes:

> So, today, M-x butterfly wouldn't have made the cut? That's sad.

Probably not. Originally, there was a lot less code like this, and the
developers were amused to include it. M-x tetris makes a great demo during
conferences.

But today, there are a LOT of these sorts of cute packages. We shouldn't feel
compelled to include nyancat.el into core because of past whimsy.

John

Re: [ANN] New library stream.el in ELPA

[Jay Belanger]

Daniel Colascione <dancol@dancol.org> writes:
...
> So, today, M-x butterfly wouldn't have made the cut? That's sad.

I agree.  In my opinion, some whimsy and diversions that are built in
add texture to a program.

Re: [ANN] New library stream.el in ELPA

[Stephen J. Turnbull]

John Wiegley writes:

 > Although we have no hard and fast rule yet for what goes into core
 > and ELPA, I expect moving things into core to offer a better reason
 > than "because I want it there". Such a package should provide:
 > 
 >   - a basic functionality we expect to be commonly used,
 >   - supports a package already in core,
 >   - provides some justification for being in the default
 >     distribution and thus subject to maintenance by the core
 >     maintainers.
 > 
 > If it can't answer those questions, it should be in ELPA.

Of course, as Daniel was at pains to point out, there are cases where
"it's just too Hello-Kitty-cute to not be in core" is an answer to all
three (though the connection of cuteness to "supports package" is as
tenuous as the smile on a maintainer's face, I admit ;-).

Not-entirely-tongue-in-cheek-ly y'rs,

Re: [ANN] New library stream.el in ELPA

[David Kastrup]

Daniel Colascione <dancol@dancol.org> writes:

> I see no need to flake off useful parts of Emacs and put them into
> ELPA when the core is not resource-constrained.
>
> For me, using ELPA packages is a bit more awkward, since I don't use
> package.el and pull in the bits of ELPA I want manually.

So?  Emacs doesn't try catering for users who don't type M-x for
religious reasons either.  If you want to pull in the bits of ELPA
manually, you are not the person Emacs is built and maintained for.
There is no point in Emacs trying to particularly support people who
don't want to use the core facilities of Emacs.

If you choose to do things your own way, you are responsible for it.

> If your package is only in ELPA, I probably won't hear about it, and
> unless it's particularly compelling, I won't use it.

Why would you "hear" about some random addition to the core?  The
package manager, in contrast, actually lists new packages.

-- 
David Kastrup

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

John Wiegley <johnw@newartisans.com> writes:

>>>>>> Nicolas Petton <nicolas@petton.fr> writes:
>
>> What if I extract it in a thunk.el library and install it in master (with
>> proper tests), and have stream.el use that? (I could also put stream.el in
>> Emacs, but Stefan thought it would be better to have it in ELPA).
>
> Probably I'd prefer to see thunk.el in master, and stream.el in ELPA

Hi John,

I just installed thunk.el in master, and added unit tests to it.  Next
I'll refactor stream.el to use it.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[John Wiegley]

>>>>> Nicolas Petton <nicolas@petton.fr> writes:

> I just installed thunk.el in master, and added unit tests to it. Next I'll
> refactor stream.el to use it.

Thank you, Nicolas, I'm glad to hear it!

John

Re: [ANN] New library stream.el in ELPA

[Michael Heerdegen]

"John Wiegley" <johnw@newartisans.com> writes:

> What I wonder is, can you generalize this beyond cons cells?  I mean,
> can we have a general lazy evaluation framework, instead of just lazy
> lists?  The API I was thinking of was:
>
>     (let ((x (thunk-create FORM)))
>       (thunk-eval x))
>
> The idea being that 'x' is a self-modifying thunk which only evaluates FORM
> the first time it is forced, but immediately returns the result value after.

Note that there is actually already something like that in Emacs, though
it's hard or nearly impossible to find:

  lisp/url/url-future.el

You may want to see the thread "Generators (iterators) for Gnu Emacs"
from Dec of last year, where this had been mentioned.

Stefan meant he wanted to wait giving it this thing a "future-" prefix
until we have understood the consequences of this kind of abstraction.

Just want to mention this here, I'm not against writing a new lib.


Regards,

Michael.

Re: [ANN] New library stream.el in ELPA

[John Wiegley]

>>>>> Michael Heerdegen <michael_heerdegen@web.de> writes:

> Note that there is actually already something like that in Emacs, though
> it's hard or nearly impossible to find:

>   lisp/url/url-future.el

This is good to know, and it certainly doesn't belong buried down there.  I'll
leave it to Nicolas to determine if that implementation is already sufficient
and should simply be moved, or whether he can improve upon it.

Thanks for the heads up,
  John

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

John Wiegley <johnw@newartisans.com> writes:

>>>>>> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> Note that there is actually already something like that in Emacs, though
>> it's hard or nearly impossible to find:
>
>>   lisp/url/url-future.el
>
> This is good to know, and it certainly doesn't belong buried down
> there.

Indeed, that's definitely interesting!  I haven't looked at it in
details yet, but it seems to provide async promises working with
callbacks rather than lazy form evaluation.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[John Mastro]

[-- Attachment #1: Type: text/plain, Size: 823 bytes --]

Hi Nico,

> I just pushed to ELPA the first version of `stream', a new library that
> provides an implementation of streams.
>
> `stream' requires Emacs >= 25.1, as it leverages the extensibility of
> seq.el (the version currently in master, not the one in ELPA): all
> functions defined in seq.el will work on streams, meaning that it's
> possible to consume a stream using `seq-take', map and filter it using
> `seq-map` and `seq-filter`, and so forth.

Thanks for working on this, it's definitely useful.

Specifying the dependency as (emacs "25.1") seems to prevent it from
being installable via package.el until Emacs 25.1 is released. At least,
it's listed as "incompat" in the *Packages* menu in my Emacs, built from
master fairly recently. I think listing the dependency as (emacs "25")
would avoid this.

-- 
john

[-- Attachment #2: Type: text/html, Size: 986 bytes --]

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 472 bytes --]

John Mastro <john.b.mastro@gmail.com> writes:

> Hi Nico,

Hi John,

> Thanks for working on this, it's definitely useful.
>
> Specifying the dependency as (emacs "25.1") seems to prevent it from
> being installable via package.el until Emacs 25.1 is released. At least,
> it's listed as "incompat" in the *Packages* menu in my Emacs, built from
> master fairly recently. I think listing the dependency as (emacs "25")
> would avoid this.

Thanks, I will update it!

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: [ANN] New library stream.el in ELPA

[Michael Heerdegen]

Nicolas Petton <nicolas@petton.fr> writes:

> The generic `stream' function currently accepts lists, strings, arrays
> and buffers as input, but it can be cleanly extended to support pretty
> much any kind of data.

I have a question: For implementing the diverse generic seq functions
(`seq-p', `seq-elt', `seq-length') for streams you use `cl-defgeneric'
all the time, e.g.

--8<---------------cut here---------------start------------->8---
(cl-defgeneric seq-elt ((stream stream) n)
  "Return the element of STREAM at index N."
  (while (> n 0)
    (setq stream (stream-rest stream))
    (setq n (1- n)))
  (stream-first stream))
--8<---------------cut here---------------end--------------->8---


But according to the doc of `cl-defgeneric' and `cl-defmethod', this
seems the perfect use case for `cl-defmethod':

"[...] it defines the implementation of NAME to use for invocations where
the value of the dispatch argument matches the specified TYPE."

So why do you use `cl-defgeneric' - what do I miss?


Regards,

Michael.

Re: [ANN] New library stream.el in ELPA

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 428 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> But according to the doc of `cl-defgeneric' and `cl-defmethod', this
> seems the perfect use case for `cl-defmethod':
>
> "[...] it defines the implementation of NAME to use for invocations where
> the value of the dispatch argument matches the specified TYPE."
>
> So why do you use `cl-defgeneric' - what do I miss?

Nothing, it's a mistake I made, I will fix it.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]