unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Streams: add stream-delay and streams of directory files
@ 2016-02-11 13:32 Michael Heerdegen
  2016-02-11 13:36 ` Michael Heerdegen
                   ` (2 more replies)
  0 siblings, 3 replies; 24+ messages in thread
From: Michael Heerdegen @ 2016-02-11 13:32 UTC (permalink / raw)
  To: Emacs Development; +Cc: Nicolas Petton

[-- Attachment #1: Type: text/plain, Size: 3944 bytes --]

Hello,

attached (at the end) is a patch for stream.el adding two things: (1) a
new macro `stream-delay' and (2) a function `stream-of-directory-files'
returning streams of directory files, optionally recursively.
Especially in the recursive case, a stream of directory files is better
to handle than a list: since it's delayed, it just encapsulates a
directory search to do.  No huge list needs to be accumulated; you can
stop when you have found what you have searched for, resume searching
later, etc. - just as with any stream.


Some notes about semantics and implementation:

(1) It turned out that the template of the implementation of `seq-copy'
for streams is useful in other scenarios, too.  I made the template a
macro `stream-delay', fixed the case of an empty stream that the old
`stream-copy' didn't handle correctly, and got that:

--8<---------------cut here---------------start------------->8---
(defmacro stream-delay (expr)
  "Return a new stream to be obtained by evaluating EXPR.
EXPR will be evaluated once when an element of the resulting
stream is requested for the first time, and must return a stream.
EXPR will be evaluated in the lexical environment present when
calling this function."
  (let ((stream (make-symbol "stream")))
    `(stream-make (let ((,stream ,expr))
                    (if (stream-empty-p ,stream)
                        nil
                      (cons (stream-first ,stream)
                            (stream-rest ,stream)))))))
--8<---------------cut here---------------end--------------->8---


So, now

--8<---------------cut here---------------start------------->8---
(cl-defmethod seq-copy ((stream stream))
  "Return a shallow copy of STREAM."
  (stream-delay stream))
--8<---------------cut here---------------end--------------->8---

As a function, it is different from `stream-delay': the argument is
evaluated.  OTOH `stream-delay' accepts an arbitrary expression that is
evaluated delayed (i.e. when an element is requested the firs ttime) and
must return a stream.  I need this for (2), see there.  When the EXPR is
evaluated later, this is done in the lexical environment present when
`stream-delay' was called (this happens automatically, because
`thunk-delay' creates a closure).


An example of what `stream-delay' does, and what not:

--8<---------------cut here---------------start------------->8---
ELISP> (let* ((a 0))
         (setq f (lambda (x) (setq a x)))
         (setq s (stream-delay (stream-range a 10))))
(--stream--..21..)

ELISP> s
(--stream--..21..)

ELISP> (funcall f 5)
5 (#o5, #x5, ?\C-e)

ELISP> (seq-into s 'list)
(5 6 7 8 9)

ELISP> (funcall f 1)
1 (#o1, #x1, ?\C-a)

ELISP> (seq-into s 'list)
(5 6 7 8 9)
--8<---------------cut here---------------end--------------->8---


(2) I chose the signature

 stream-of-directory-files (directory &optional full nosort recurse follow-links filter)

where FULL and NOSORT are directly passed to `directory-files', RECURSE
can be boolean or a predicate (of a dir name) to decide into which dirs
to recurse.  FILTER can be a predicate to limit the resulting stream to
files fulfilling it.  It replaces the MATCH argument of
`directory-files'.

A typical call to `stream-of-directory-files' looks like this:

(setq s (stream-of-directory-files
         "~/gnu-emacs" t nil
         (lambda (dir) (not (string= ".git" (file-name-nondirectory dir))))
         nil
         #'file-directory-p))

This sets `s' to a stream of files under "~/gnu-emacs" that will recurse
into all directories not named ".git".  The last (the FILTER) argument
specifies that only directories will be included.

The function call terminates immediately without looking at the file
system (as we expect for streams).  This is exactly the reason for why I
needed something like `stream-delay' - see the implementation.

For testing: to list the elements of s, one can do e.g.:

(seq-doseq (file s) (message "%s" file))


Finally the patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: stream.patch --]
[-- Type: text/x-diff, Size: 3701 bytes --]

diff --git a/stream.el b/stream.el
index 567a9e3..4d61cf1 100644
--- a/stream.el
+++ b/stream.el
@@ -152,7 +152,7 @@ range is infinite."
        (eq (car stream) stream--identifier)))
 
 (defun stream-empty ()
-  "Return an empty stream."
+  "Return a new empty stream."
   (list stream--identifier (thunk-delay nil)))
 
 (defun stream-empty-p (stream)
@@ -317,10 +317,69 @@ kind of nonlocal exit."
        (cons (stream-first stream)
              (seq-filter pred (stream-rest stream)))))))
 
+(defmacro stream-delay (expr)
+  "Return a new stream to be obtained by evaluating EXPR.
+EXPR will be evaluated once when an element of the resulting
+stream is requested for the first time, and must return a stream.
+EXPR will be evaluated in the lexical environment present when
+calling this function."
+  (let ((stream (make-symbol "stream")))
+    `(stream-make (let ((,stream ,expr))
+                    (if (stream-empty-p ,stream)
+                        nil
+                      (cons (stream-first ,stream)
+                            (stream-rest ,stream)))))))
+
 (cl-defmethod seq-copy ((stream stream))
   "Return a shallow copy of STREAM."
-  (stream-cons (stream-first stream)
-               (stream-rest stream)))
+  (stream-delay stream))
+
+(defun stream-of-directory-files-1 (directory &optional nosort recurse follow-links)
+  "Helper for `stream-of-directory-files'."
+  (stream-delay
+   (if (file-accessible-directory-p directory)
+       (let (files dirs (reverse-fun (if nosort #'identity #'nreverse)))
+         (dolist (file (directory-files directory t nil nosort))
+           (let ((is-dir (file-directory-p file)))
+             (unless (and is-dir
+                          (member (file-name-nondirectory (directory-file-name file))
+                                  '("." "..")))
+               (push file files)
+               (when (and is-dir
+                          (or follow-links (not (file-symlink-p file)))
+                          (if (functionp recurse) (funcall recurse file) recurse))
+                 (push file dirs)))))
+         (apply #'stream-append
+                (stream (funcall reverse-fun files))
+                (mapcar
+                 (lambda (dir) (stream-of-directory-files-1 dir nosort recurse follow-links))
+                 (funcall reverse-fun dirs))))
+     (stream-empty))))
+
+(defun stream-of-directory-files (directory &optional full nosort recurse follow-links filter)
+  "Return a stream of names of files in DIRECTORY.
+Call `directory-files' to list file names in DIRECTORY and make
+the result a stream.  Don't include files named \".\" or \"..\".
+The arguments FULL and NOSORT are directly passed to
+`directory-files'.
+
+Third optional argument RECURSE non-nil means recurse on
+subdirectories.  If RECURSE is a function, it should be a
+predicate accepting one argument, an absolute file name of a
+directory, and return non-nil when the returned stream should
+recurse into that directory.  Any other non-nil value means
+recurse into every readable subdirectory.
+
+Even with recurse non-nil, don't descent into directories by
+following symlinks unless FOLLOW-LINKS is non-nil.
+
+If FILTER is non-nil, it should be a predicate accepting one
+argument, an absolute file name.  It is used to limit the
+resulting stream to the files fulfilling this predicate."
+  (let* ((stream (stream-of-directory-files-1 directory nosort recurse follow-links))
+         (filtered-stream (if filter (seq-filter filter stream) stream)))
+    (if full filtered-stream
+      (seq-map (lambda (file) (file-relative-name file directory)) filtered-stream))))
 
 (provide 'stream)
 ;;; stream.el ends here
-- 
2.7.0


^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-11 13:32 Streams: add stream-delay and streams of directory files Michael Heerdegen
@ 2016-02-11 13:36 ` Michael Heerdegen
  2016-02-11 20:33 ` Nicolas Petton
  2016-02-11 20:48 ` Eli Zaretskii
  2 siblings, 0 replies; 24+ messages in thread
From: Michael Heerdegen @ 2016-02-11 13:36 UTC (permalink / raw)
  To: emacs-devel

Michael Heerdegen <michael_heerdegen@web.de> writes:

> attached (at the end) is a patch for stream.el adding two things:
> [...]

BTW @Nicolas I can make a PR at Github if you prefer after discussing
the changes here.


Regards,

Michael.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-11 13:32 Streams: add stream-delay and streams of directory files Michael Heerdegen
  2016-02-11 13:36 ` Michael Heerdegen
@ 2016-02-11 20:33 ` Nicolas Petton
  2016-02-12 14:18   ` Michael Heerdegen
  2016-02-24 16:38   ` Michael Heerdegen
  2016-02-11 20:48 ` Eli Zaretskii
  2 siblings, 2 replies; 24+ messages in thread
From: Nicolas Petton @ 2016-02-11 20:33 UTC (permalink / raw)
  To: Michael Heerdegen, Emacs Development

[-- Attachment #1: Type: text/plain, Size: 837 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Hello,

Hi Michael,

> attached (at the end) is a patch for stream.el adding two things: (1) a
> new macro `stream-delay' and (2) a function `stream-of-directory-files'
> returning streams of directory files, optionally recursively.
> Especially in the recursive case, a stream of directory files is better
> to handle than a list: since it's delayed, it just encapsulates a
> directory search to do.  No huge list needs to be accumulated; you can
> stop when you have found what you have searched for, resume searching
> later, etc. - just as with any stream.

It looks really good, thank you!

Would you mind adding tests for `stream-delay' and
`stream-of-directory-files'?  (A regression test for the stream
implementation of `seq-copy' would be great too!)

Thanks again,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-11 13:32 Streams: add stream-delay and streams of directory files Michael Heerdegen
  2016-02-11 13:36 ` Michael Heerdegen
  2016-02-11 20:33 ` Nicolas Petton
@ 2016-02-11 20:48 ` Eli Zaretskii
  2016-02-11 21:10   ` Nicolas Petton
  2 siblings, 1 reply; 24+ messages in thread
From: Eli Zaretskii @ 2016-02-11 20:48 UTC (permalink / raw)
  To: Michael Heerdegen; +Cc: nicolas, emacs-devel

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Date: Thu, 11 Feb 2016 14:32:13 +0100
> Cc: Nicolas Petton <nicolas@petton.fr>
> 
> attached (at the end) is a patch for stream.el adding two things: (1) a
> new macro `stream-delay' and (2) a function `stream-of-directory-files'
> returning streams of directory files, optionally recursively.
> Especially in the recursive case, a stream of directory files is better
> to handle than a list: since it's delayed, it just encapsulates a
> directory search to do.  No huge list needs to be accumulated; you can
> stop when you have found what you have searched for, resume searching
> later, etc. - just as with any stream.

Thanks.

Would you please consider adding suitable changes to the documentation
for these patches?



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-11 20:48 ` Eli Zaretskii
@ 2016-02-11 21:10   ` Nicolas Petton
  2016-02-11 21:22     ` Eli Zaretskii
  0 siblings, 1 reply; 24+ messages in thread
From: Nicolas Petton @ 2016-02-11 21:10 UTC (permalink / raw)
  To: Eli Zaretskii, Michael Heerdegen; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 216 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

> Would you please consider adding suitable changes to the documentation
> for these patches?

seq.el and map.el are in Emacs, but stream.el is currently in ELPA.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-11 21:10   ` Nicolas Petton
@ 2016-02-11 21:22     ` Eli Zaretskii
  2016-02-12 14:14       ` Michael Heerdegen
  0 siblings, 1 reply; 24+ messages in thread
From: Eli Zaretskii @ 2016-02-11 21:22 UTC (permalink / raw)
  To: Nicolas Petton; +Cc: michael_heerdegen, emacs-devel

> From: Nicolas Petton <nicolas@petton.fr>
> Cc: emacs-devel@gnu.org
> Date: Thu, 11 Feb 2016 22:10:53 +0100
> 
> > Would you please consider adding suitable changes to the documentation
> > for these patches?
> 
> seq.el and map.el are in Emacs, but stream.el is currently in ELPA.

Sorry, I thought the patch was being proposed for inclusion in Emacs.



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-11 21:22     ` Eli Zaretskii
@ 2016-02-12 14:14       ` Michael Heerdegen
  2016-02-12 15:22         ` Eli Zaretskii
  0 siblings, 1 reply; 24+ messages in thread
From: Michael Heerdegen @ 2016-02-12 14:14 UTC (permalink / raw)
  To: emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

> > > Would you please consider adding suitable changes to the
> > > documentation for these patches?
> > 
> > seq.el and map.el are in Emacs, but stream.el is currently in ELPA.
>
> Sorry, I thought the patch was being proposed for inclusion in Emacs.

Sorry for the confusion.  ELPA is considered a part of Emacs - that's
why I wanted to discuss this here.  Is this unwanted?  I think it is
better than discussing at Github.

Anyway, the next time I'll use the prefix Elpa: ... in the message's
subject.


Michael.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-11 20:33 ` Nicolas Petton
@ 2016-02-12 14:18   ` Michael Heerdegen
  2016-02-13 19:25     ` Nicolas Petton
  2016-02-24 16:38   ` Michael Heerdegen
  1 sibling, 1 reply; 24+ messages in thread
From: Michael Heerdegen @ 2016-02-12 14:18 UTC (permalink / raw)
  To: emacs-devel

Nicolas Petton <nicolas@petton.fr> writes:

> Would you mind adding tests for `stream-delay' and
> `stream-of-directory-files'?  (A regression test for the stream
> implementation of `seq-copy' would be great too!)

I can do that - though I don't know how a useful test of
`stream-of-directory-files' could look like.

In the meantime, could you please address the following compiler
warnings of the test file:

  Compiling file /home/micha/software/stream/test/stream-tests.el at Fri
  Feb 12 15:15:53 2016
  
  In end of data: stream-tests.el:216:1:Warning: the following functions
  are not known to be defined: seq-p, oddp, evenp


Regards,

Michael.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-12 14:14       ` Michael Heerdegen
@ 2016-02-12 15:22         ` Eli Zaretskii
  0 siblings, 0 replies; 24+ messages in thread
From: Eli Zaretskii @ 2016-02-12 15:22 UTC (permalink / raw)
  To: Michael Heerdegen; +Cc: emacs-devel

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Date: Fri, 12 Feb 2016 15:14:01 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > > > Would you please consider adding suitable changes to the
> > > > documentation for these patches?
> > > 
> > > seq.el and map.el are in Emacs, but stream.el is currently in ELPA.
> >
> > Sorry, I thought the patch was being proposed for inclusion in Emacs.
> 
> Sorry for the confusion.  ELPA is considered a part of Emacs - that's
> why I wanted to discuss this here.  Is this unwanted?

No, not at all.  Just mention somewhere it's for ELPA, that's all.

Thanks.



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-12 14:18   ` Michael Heerdegen
@ 2016-02-13 19:25     ` Nicolas Petton
  2016-02-14 12:03       ` Michael Heerdegen
  0 siblings, 1 reply; 24+ messages in thread
From: Nicolas Petton @ 2016-02-13 19:25 UTC (permalink / raw)
  To: Michael Heerdegen, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 494 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Nicolas Petton <nicolas@petton.fr> writes:
>
>> Would you mind adding tests for `stream-delay' and
>> `stream-of-directory-files'?  (A regression test for the stream
>> implementation of `seq-copy' would be great too!)
>
> I can do that - though I don't know how a useful test of
> `stream-of-directory-files' could look like.
>
> In the meantime, could you please address the following compiler
> warnings of the test file:

Done!

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-13 19:25     ` Nicolas Petton
@ 2016-02-14 12:03       ` Michael Heerdegen
  2016-02-14 12:07         ` Nicolas Petton
  2016-02-14 13:55         ` Nicolas Petton
  0 siblings, 2 replies; 24+ messages in thread
From: Michael Heerdegen @ 2016-02-14 12:03 UTC (permalink / raw)
  To: emacs-devel

Nicolas Petton <nicolas@petton.fr> writes:

> Done!

Thanks.

But now stream.el in Gnu Elpa and on github have diverged: they both
have a change that is not present in the other one.

Can you please fix this before we proceed?


Thanks,

Michael.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-14 12:03       ` Michael Heerdegen
@ 2016-02-14 12:07         ` Nicolas Petton
  2016-02-14 13:55         ` Nicolas Petton
  1 sibling, 0 replies; 24+ messages in thread
From: Nicolas Petton @ 2016-02-14 12:07 UTC (permalink / raw)
  To: Michael Heerdegen, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 363 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Nicolas Petton <nicolas@petton.fr> writes:
>
>> Done!
>
> Thanks.
>
> But now stream.el in Gnu Elpa and on github have diverged: they both
> have a change that is not present in the other one.
>
> Can you please fix this before we proceed?

I was waiting for your changes before doing that, but sure.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-14 12:03       ` Michael Heerdegen
  2016-02-14 12:07         ` Nicolas Petton
@ 2016-02-14 13:55         ` Nicolas Petton
  2016-02-14 14:24           ` Michael Heerdegen
  1 sibling, 1 reply; 24+ messages in thread
From: Nicolas Petton @ 2016-02-14 13:55 UTC (permalink / raw)
  To: Michael Heerdegen, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 142 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Can you please fix this before we proceed?

I just pushed on both repositories.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-14 13:55         ` Nicolas Petton
@ 2016-02-14 14:24           ` Michael Heerdegen
  0 siblings, 0 replies; 24+ messages in thread
From: Michael Heerdegen @ 2016-02-14 14:24 UTC (permalink / raw)
  To: emacs-devel

Nicolas Petton <nicolas@petton.fr> writes:

> I just pushed on both repositories.

Thanks; will update my patch ASAP.


Regards,

Michael.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-11 20:33 ` Nicolas Petton
  2016-02-12 14:18   ` Michael Heerdegen
@ 2016-02-24 16:38   ` Michael Heerdegen
  2016-02-24 17:01     ` Nicolas Petton
                       ` (2 more replies)
  1 sibling, 3 replies; 24+ messages in thread
From: Michael Heerdegen @ 2016-02-24 16:38 UTC (permalink / raw)
  To: Nicolas Petton; +Cc: Emacs Development

[-- Attachment #1: Type: text/plain, Size: 359 bytes --]

Nicolas Petton <nicolas@petton.fr> writes:

> Would you mind adding tests for `stream-delay' and
> `stream-of-directory-files'?  (A regression test for the stream
> implementation of `seq-copy' would be great too!)

I've added some tests to the last patch.

But I can't image how a test for `stream-of-directory-files' could look
like that is not nonsense.



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: stream.patch --]
[-- Type: text/x-diff, Size: 5957 bytes --]

diff --git a/packages/stream/stream.el b/packages/stream/stream.el
index 567a9e3..4d61cf1 100644
--- a/packages/stream/stream.el
+++ b/packages/stream/stream.el
@@ -152,7 +152,7 @@ range is infinite."
        (eq (car stream) stream--identifier)))
 
 (defun stream-empty ()
-  "Return an empty stream."
+  "Return a new empty stream."
   (list stream--identifier (thunk-delay nil)))
 
 (defun stream-empty-p (stream)
@@ -317,10 +317,69 @@ kind of nonlocal exit."
        (cons (stream-first stream)
              (seq-filter pred (stream-rest stream)))))))
 
+(defmacro stream-delay (expr)
+  "Return a new stream to be obtained by evaluating EXPR.
+EXPR will be evaluated once when an element of the resulting
+stream is requested for the first time, and must return a stream.
+EXPR will be evaluated in the lexical environment present when
+calling this function."
+  (let ((stream (make-symbol "stream")))
+    `(stream-make (let ((,stream ,expr))
+                    (if (stream-empty-p ,stream)
+                        nil
+                      (cons (stream-first ,stream)
+                            (stream-rest ,stream)))))))
+
 (cl-defmethod seq-copy ((stream stream))
   "Return a shallow copy of STREAM."
-  (stream-cons (stream-first stream)
-               (stream-rest stream)))
+  (stream-delay stream))
+
+(defun stream-of-directory-files-1 (directory &optional nosort recurse follow-links)
+  "Helper for `stream-of-directory-files'."
+  (stream-delay
+   (if (file-accessible-directory-p directory)
+       (let (files dirs (reverse-fun (if nosort #'identity #'nreverse)))
+         (dolist (file (directory-files directory t nil nosort))
+           (let ((is-dir (file-directory-p file)))
+             (unless (and is-dir
+                          (member (file-name-nondirectory (directory-file-name file))
+                                  '("." "..")))
+               (push file files)
+               (when (and is-dir
+                          (or follow-links (not (file-symlink-p file)))
+                          (if (functionp recurse) (funcall recurse file) recurse))
+                 (push file dirs)))))
+         (apply #'stream-append
+                (stream (funcall reverse-fun files))
+                (mapcar
+                 (lambda (dir) (stream-of-directory-files-1 dir nosort recurse follow-links))
+                 (funcall reverse-fun dirs))))
+     (stream-empty))))
+
+(defun stream-of-directory-files (directory &optional full nosort recurse follow-links filter)
+  "Return a stream of names of files in DIRECTORY.
+Call `directory-files' to list file names in DIRECTORY and make
+the result a stream.  Don't include files named \".\" or \"..\".
+The arguments FULL and NOSORT are directly passed to
+`directory-files'.
+
+Third optional argument RECURSE non-nil means recurse on
+subdirectories.  If RECURSE is a function, it should be a
+predicate accepting one argument, an absolute file name of a
+directory, and return non-nil when the returned stream should
+recurse into that directory.  Any other non-nil value means
+recurse into every readable subdirectory.
+
+Even with recurse non-nil, don't descent into directories by
+following symlinks unless FOLLOW-LINKS is non-nil.
+
+If FILTER is non-nil, it should be a predicate accepting one
+argument, an absolute file name.  It is used to limit the
+resulting stream to the files fulfilling this predicate."
+  (let* ((stream (stream-of-directory-files-1 directory nosort recurse follow-links))
+         (filtered-stream (if filter (seq-filter filter stream) stream)))
+    (if full filtered-stream
+      (seq-map (lambda (file) (file-relative-name file directory)) filtered-stream))))
 
 (provide 'stream)
 ;;; stream.el ends here
diff --git a/packages/stream/tests/stream-tests.el b/packages/stream/tests/stream-tests.el
index 88edf91..400f2ee 100644
--- a/packages/stream/tests/stream-tests.el
+++ b/packages/stream/tests/stream-tests.el
@@ -171,10 +171,40 @@
   (should (= 3 (stream-first (stream-rest (seq-filter #'cl-oddp (stream-range 0 4))))))
   (should (stream-empty-p (stream-rest (stream-rest (seq-filter #'cl-oddp (stream-range 0 4)))))))
 
+(ert-deftest stream-delay-test ()
+  (should (streamp (stream-delay (stream-range))))
+  (should (= 0 (stream-first (stream-delay (stream-range)))))
+  (should (= 1 (stream-first (stream-rest (stream-delay (stream-range))))))
+  (should (let ((stream (stream-range 3 7)))
+            (equal (seq-into (stream-delay stream) 'list)
+                   (seq-into               stream  'list))))
+  (should (null (seq-into (stream-delay (stream-empty)) 'list)))
+  (should (let* ((evaluated nil)
+                 (one-plus (lambda (el)
+                             (setq evaluated t)
+                             (1+ el)))
+                 (stream (seq-map one-plus (stream '(1)))))
+            (equal '(nil 2 t)
+                   (list evaluated (stream-first stream) evaluated))))
+  (should (let* ((a 0)
+                 (set-a (lambda (x) (setq a x)))
+                 (s (stream-delay (stream (list a))))                 
+                 res1 res2)
+            (funcall set-a 5)
+            (setq res1 (stream-first s))
+            (funcall set-a 11)
+            (setq res2 (stream-first s))
+            (and (equal res1 5)
+                 (equal res2 5)))))
+
 (ert-deftest stream-seq-copy-test ()
   (should (streamp (seq-copy (stream-range))))
   (should (= 0 (stream-first (seq-copy (stream-range)))))
-  (should (= 1 (stream-first (stream-rest (seq-copy (stream-range)))))))
+  (should (= 1 (stream-first (stream-rest (seq-copy (stream-range))))))
+  (should (let ((stream (stream-range 3 7)))
+            (equal (seq-into (seq-copy stream) 'list)
+                   (seq-into           stream  'list))))
+  (should (null (seq-into (seq-copy (stream-empty)) 'list))))
 
 (ert-deftest stream-range-test ()
   (should (stream-empty-p (stream-range 0 0)))

[-- Attachment #3: Type: text/plain, Size: 21 bytes --]



Regards,

Michael.

^ permalink raw reply related	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-24 16:38   ` Michael Heerdegen
@ 2016-02-24 17:01     ` Nicolas Petton
  2016-02-29 15:05     ` Nicolas Petton
  2016-03-03 14:42     ` Nicolas Petton
  2 siblings, 0 replies; 24+ messages in thread
From: Nicolas Petton @ 2016-02-24 17:01 UTC (permalink / raw)
  To: Michael Heerdegen; +Cc: Emacs Development

[-- Attachment #1: Type: text/plain, Size: 135 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I've added some tests to the last patch.

Thanks, I'll have a look ASAP.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-24 16:38   ` Michael Heerdegen
  2016-02-24 17:01     ` Nicolas Petton
@ 2016-02-29 15:05     ` Nicolas Petton
  2016-03-03 14:42     ` Nicolas Petton
  2 siblings, 0 replies; 24+ messages in thread
From: Nicolas Petton @ 2016-02-29 15:05 UTC (permalink / raw)
  To: Michael Heerdegen; +Cc: Emacs Development

[-- Attachment #1: Type: text/plain, Size: 190 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I've added some tests to the last patch.

Hi Michael,

I'm swamped today, but I'll review and apply the patch as soon as I can.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-02-24 16:38   ` Michael Heerdegen
  2016-02-24 17:01     ` Nicolas Petton
  2016-02-29 15:05     ` Nicolas Petton
@ 2016-03-03 14:42     ` Nicolas Petton
  2016-03-03 15:26       ` Michael Heerdegen
  2 siblings, 1 reply; 24+ messages in thread
From: Nicolas Petton @ 2016-03-03 14:42 UTC (permalink / raw)
  To: Michael Heerdegen; +Cc: Emacs Development

[-- Attachment #1: Type: text/plain, Size: 224 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> I've added some tests to the last patch.

Hi Michael,

I applied the patch (and will update the on github too).  Sorry for
taking so long to review and apply it.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-03-03 14:42     ` Nicolas Petton
@ 2016-03-03 15:26       ` Michael Heerdegen
  2016-03-03 15:29         ` Nicolas Petton
  0 siblings, 1 reply; 24+ messages in thread
From: Michael Heerdegen @ 2016-03-03 15:26 UTC (permalink / raw)
  To: Nicolas Petton; +Cc: Emacs Development

Nicolas Petton <nicolas@petton.fr> writes:

> I applied the patch (and will update the on github too).  Sorry for
> taking so long to review and apply it.

Thanks!  Don't worry about the delay.

Hoping `stream-delay' was not a bad idea...


Regards,

Michael.



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-03-03 15:26       ` Michael Heerdegen
@ 2016-03-03 15:29         ` Nicolas Petton
  2016-03-03 22:05           ` Artur Malabarba
  0 siblings, 1 reply; 24+ messages in thread
From: Nicolas Petton @ 2016-03-03 15:29 UTC (permalink / raw)
  To: Michael Heerdegen; +Cc: Emacs Development

[-- Attachment #1: Type: text/plain, Size: 134 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Hoping `stream-delay' was not a bad idea...

I think it's quite good :)

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-03-03 15:29         ` Nicolas Petton
@ 2016-03-03 22:05           ` Artur Malabarba
  2016-03-04  9:02             ` Nicolas Petton
  2016-03-04 12:15             ` Michael Heerdegen
  0 siblings, 2 replies; 24+ messages in thread
From: Artur Malabarba @ 2016-03-03 22:05 UTC (permalink / raw)
  To: Nicolas Petton; +Cc: Michael Heerdegen, Emacs Development

[-- Attachment #1: Type: text/plain, Size: 363 bytes --]

Sorry if this is a silly question, but isn't stream-delay just
reimplementing thunk.el (the functionality that stream itself is built-on)?

Artur

On 3 March 2016 at 12:29, Nicolas Petton <nicolas@petton.fr> wrote:

> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
> > Hoping `stream-delay' was not a bad idea...
>
> I think it's quite good :)
>
> Nico
>

[-- Attachment #2: Type: text/html, Size: 828 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-03-03 22:05           ` Artur Malabarba
@ 2016-03-04  9:02             ` Nicolas Petton
  2016-03-04 12:15             ` Michael Heerdegen
  1 sibling, 0 replies; 24+ messages in thread
From: Nicolas Petton @ 2016-03-04  9:02 UTC (permalink / raw)
  To: bruce.connor.am; +Cc: Michael Heerdegen, Emacs Development

[-- Attachment #1: Type: text/plain, Size: 378 bytes --]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

> Sorry if this is a silly question, but isn't stream-delay just
> reimplementing thunk.el (the functionality that stream itself is
> built-on)?

No, stream-delay will return a new stream, while thunk-delay will just
delay the evaluation of a form.

stream-delay uses stream-make, which will in turn use thunk-delay :)

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-03-03 22:05           ` Artur Malabarba
  2016-03-04  9:02             ` Nicolas Petton
@ 2016-03-04 12:15             ` Michael Heerdegen
  2016-03-04 14:21               ` Artur Malabarba
  1 sibling, 1 reply; 24+ messages in thread
From: Michael Heerdegen @ 2016-03-04 12:15 UTC (permalink / raw)
  To: emacs-devel

Artur Malabarba <bruce.connor.am@gmail.com> writes:

> Sorry if this is a silly question, but isn't stream-delay just
> reimplementing thunk.el (the functionality that stream itself is
> built-on)?

Yes, that's a very congruous question.

Let me add to Nicolas comment:

The semantics is very similar to that of `thunk-delay', and it serves
the same purpose at the end: delaying an expression.  The idea behind
`stream-delay' was to transfer the idea of delaying to the level of
streams:

`stream-delay' takes an expression E that must (later) evaluate to a
stream S, and returns another stream S' (and not a thunk).  And unlike
`thunk-delay', when referencing an element of S', E is _implicitly_
evaluated to get S.

I just defined that because it made sense for defining
`stream-of-directory-files'.  Maybe you have a nicer idea for reaching
the same goal...?


Regards,

Michael.




^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: Streams: add stream-delay and streams of directory files
  2016-03-04 12:15             ` Michael Heerdegen
@ 2016-03-04 14:21               ` Artur Malabarba
  0 siblings, 0 replies; 24+ messages in thread
From: Artur Malabarba @ 2016-03-04 14:21 UTC (permalink / raw)
  To: Michael Heerdegen; +Cc: emacs-devel

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

On 4 Mar 2016 9:15 am, "Michael Heerdegen" <michael_heerdegen@web.de> wrote:
> I just defined that because it made sense for defining
> `stream-of-directory-files'.  Maybe you have a nicer idea for reaching
> the same goal...?

Not at all, I was just curious. :-)
Thanks for explaining.

[-- Attachment #2: Type: text/html, Size: 419 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2016-03-04 14:21 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-11 13:32 Streams: add stream-delay and streams of directory files Michael Heerdegen
2016-02-11 13:36 ` Michael Heerdegen
2016-02-11 20:33 ` Nicolas Petton
2016-02-12 14:18   ` Michael Heerdegen
2016-02-13 19:25     ` Nicolas Petton
2016-02-14 12:03       ` Michael Heerdegen
2016-02-14 12:07         ` Nicolas Petton
2016-02-14 13:55         ` Nicolas Petton
2016-02-14 14:24           ` Michael Heerdegen
2016-02-24 16:38   ` Michael Heerdegen
2016-02-24 17:01     ` Nicolas Petton
2016-02-29 15:05     ` Nicolas Petton
2016-03-03 14:42     ` Nicolas Petton
2016-03-03 15:26       ` Michael Heerdegen
2016-03-03 15:29         ` Nicolas Petton
2016-03-03 22:05           ` Artur Malabarba
2016-03-04  9:02             ` Nicolas Petton
2016-03-04 12:15             ` Michael Heerdegen
2016-03-04 14:21               ` Artur Malabarba
2016-02-11 20:48 ` Eli Zaretskii
2016-02-11 21:10   ` Nicolas Petton
2016-02-11 21:22     ` Eli Zaretskii
2016-02-12 14:14       ` Michael Heerdegen
2016-02-12 15:22         ` Eli Zaretskii

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).