From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Michael Heerdegen Newsgroups: gmane.emacs.devel Subject: Streams: add stream-delay and streams of directory files Date: Thu, 11 Feb 2016 14:32:13 +0100 Message-ID: <87pow3mkyq.fsf@web.de> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1455197567 4823 80.91.229.3 (11 Feb 2016 13:32:47 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 11 Feb 2016 13:32:47 +0000 (UTC) Cc: Nicolas Petton To: Emacs Development Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Feb 11 14:32:38 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aTrMJ-0000Ha-Uy for ged-emacs-devel@m.gmane.org; Thu, 11 Feb 2016 14:32:36 +0100 Original-Received: from localhost ([::1]:50075 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aTrME-0003Vr-66 for ged-emacs-devel@m.gmane.org; Thu, 11 Feb 2016 08:32:30 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:49610) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aTrM8-0003Td-Rp for emacs-devel@gnu.org; Thu, 11 Feb 2016 08:32:26 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aTrM4-0000Ef-PK for emacs-devel@gnu.org; Thu, 11 Feb 2016 08:32:24 -0500 Original-Received: from mout.web.de ([212.227.17.12]:55485) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aTrM4-0000ES-F8 for emacs-devel@gnu.org; Thu, 11 Feb 2016 08:32:20 -0500 Original-Received: from drachen.dragon ([92.74.178.250]) by smtp.web.de (mrweb103) with ESMTPSA (Nemesis) id 0Lxwme-1ZxwMs3FJs-015MbA; Thu, 11 Feb 2016 14:32:15 +0100 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.90 (gnu/linux) X-Provags-ID: V03:K0:jCiw4XZIMMnw9L6JOMwEkVTC6NF2i9KpuPp1txrrlIZIodYwDKZ BRuolBCsCi1c1rz1LQmFHH+bN4lrqmpu6TWHLuVUFxnSg47YilyFH4jbrU6120EjG1UAjwg YO8BsZJ46BfxhUXjpagg2n6jyyFdFA0yy7GKF00yAc7ksnkPenI11yCoipAnUiEB7nvDQ/t K8YbIyICcLcBmMPYEc0Mw== X-UI-Out-Filterresults: notjunk:1;V01:K0:hpREUKEJwBU=:ej9hGbUd0Mh1a729+ktVqz GMt8bY3o+LM3n1rNLxNum5xlf71kbzx4p07nThvisbcwaa6DVGMV7y4mdjJJpLZsViwk7wYpJ a4Ky+OpfAENhjZdx5EQMCGFa6ACUPsZGbebZ5HGhf4oWss7bQ/XAMB8bzQwWSSxqIuwaOyigd CZ67LCsRcgHOVzin2T/Xjt19aiMYvoRb4QlFL8TtVmdFQpv+wmykhVKtkkUL68ARKvwLeiO9p EpK54A8eMH0o0t4Yfu6fVNylcvpGsBRe5ureZuXPMtmGrU8GQQBvHwirZeXCjc6uFf8hGUJtZ 4wQET7BoyLYU2WRt11yOVD303ED5jHV23t76SOJAvUzSXhBtK0OocAwSaa+i3fpersgMB06oc qLhSXw6zRpsmeVXfBqFxq5aZ5Ul6/oiZrdO0obNhRqGpsbj811BGx3N1I3dQpjF103K3Eb3Nz pxcSFwBPldJY/Ci03M+Av98vYH4y/yW6ByLwsGau5i2L5bjaaH6vEyE8BSjdNkcQaFb9MduW1 skT+hxrwXS6kE7R9m0803nTymt0Dwk8nNPj1Jxnq8NEZwH+oAwdfXMObPkAKJVEwtcKNe/9mU g5y7MVZV0XybtcEaI66Uz8wDpOg+XT5QVrq3LBBUr5Zo+pTcZgSEPUFjbZhBlSd6zN4E6Mbsz MLUmCTGQ6+sIru5vOB7GORI/t3uLNQjOMd995EY8+RxwQvM47WZ1QVtUQWP1uAkuw2bR/E4Eq 0LoH4lzyHXP7Ww4A9DLxpv1YMB+kpyzV3kBKzRZD6Q/SIU6XW1PliZimSP80S/9L+0/OI5S5 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 212.227.17.12 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:199744 Archived-At: --=-=-= Content-Type: text/plain Hello, attached (at the end) is a patch for stream.el adding two things: (1) a new macro `stream-delay' and (2) a function `stream-of-directory-files' returning streams of directory files, optionally recursively. Especially in the recursive case, a stream of directory files is better to handle than a list: since it's delayed, it just encapsulates a directory search to do. No huge list needs to be accumulated; you can stop when you have found what you have searched for, resume searching later, etc. - just as with any stream. Some notes about semantics and implementation: (1) It turned out that the template of the implementation of `seq-copy' for streams is useful in other scenarios, too. I made the template a macro `stream-delay', fixed the case of an empty stream that the old `stream-copy' didn't handle correctly, and got that: --8<---------------cut here---------------start------------->8--- (defmacro stream-delay (expr) "Return a new stream to be obtained by evaluating EXPR. EXPR will be evaluated once when an element of the resulting stream is requested for the first time, and must return a stream. EXPR will be evaluated in the lexical environment present when calling this function." (let ((stream (make-symbol "stream"))) `(stream-make (let ((,stream ,expr)) (if (stream-empty-p ,stream) nil (cons (stream-first ,stream) (stream-rest ,stream))))))) --8<---------------cut here---------------end--------------->8--- So, now --8<---------------cut here---------------start------------->8--- (cl-defmethod seq-copy ((stream stream)) "Return a shallow copy of STREAM." (stream-delay stream)) --8<---------------cut here---------------end--------------->8--- As a function, it is different from `stream-delay': the argument is evaluated. OTOH `stream-delay' accepts an arbitrary expression that is evaluated delayed (i.e. when an element is requested the firs ttime) and must return a stream. I need this for (2), see there. When the EXPR is evaluated later, this is done in the lexical environment present when `stream-delay' was called (this happens automatically, because `thunk-delay' creates a closure). An example of what `stream-delay' does, and what not: --8<---------------cut here---------------start------------->8--- ELISP> (let* ((a 0)) (setq f (lambda (x) (setq a x))) (setq s (stream-delay (stream-range a 10)))) (--stream--..21..) ELISP> s (--stream--..21..) ELISP> (funcall f 5) 5 (#o5, #x5, ?\C-e) ELISP> (seq-into s 'list) (5 6 7 8 9) ELISP> (funcall f 1) 1 (#o1, #x1, ?\C-a) ELISP> (seq-into s 'list) (5 6 7 8 9) --8<---------------cut here---------------end--------------->8--- (2) I chose the signature stream-of-directory-files (directory &optional full nosort recurse follow-links filter) where FULL and NOSORT are directly passed to `directory-files', RECURSE can be boolean or a predicate (of a dir name) to decide into which dirs to recurse. FILTER can be a predicate to limit the resulting stream to files fulfilling it. It replaces the MATCH argument of `directory-files'. A typical call to `stream-of-directory-files' looks like this: (setq s (stream-of-directory-files "~/gnu-emacs" t nil (lambda (dir) (not (string= ".git" (file-name-nondirectory dir)))) nil #'file-directory-p)) This sets `s' to a stream of files under "~/gnu-emacs" that will recurse into all directories not named ".git". The last (the FILTER) argument specifies that only directories will be included. The function call terminates immediately without looking at the file system (as we expect for streams). This is exactly the reason for why I needed something like `stream-delay' - see the implementation. For testing: to list the elements of s, one can do e.g.: (seq-doseq (file s) (message "%s" file)) Finally the patch: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=stream.patch diff --git a/stream.el b/stream.el index 567a9e3..4d61cf1 100644 --- a/stream.el +++ b/stream.el @@ -152,7 +152,7 @@ range is infinite." (eq (car stream) stream--identifier))) (defun stream-empty () - "Return an empty stream." + "Return a new empty stream." (list stream--identifier (thunk-delay nil))) (defun stream-empty-p (stream) @@ -317,10 +317,69 @@ kind of nonlocal exit." (cons (stream-first stream) (seq-filter pred (stream-rest stream))))))) +(defmacro stream-delay (expr) + "Return a new stream to be obtained by evaluating EXPR. +EXPR will be evaluated once when an element of the resulting +stream is requested for the first time, and must return a stream. +EXPR will be evaluated in the lexical environment present when +calling this function." + (let ((stream (make-symbol "stream"))) + `(stream-make (let ((,stream ,expr)) + (if (stream-empty-p ,stream) + nil + (cons (stream-first ,stream) + (stream-rest ,stream))))))) + (cl-defmethod seq-copy ((stream stream)) "Return a shallow copy of STREAM." - (stream-cons (stream-first stream) - (stream-rest stream))) + (stream-delay stream)) + +(defun stream-of-directory-files-1 (directory &optional nosort recurse follow-links) + "Helper for `stream-of-directory-files'." + (stream-delay + (if (file-accessible-directory-p directory) + (let (files dirs (reverse-fun (if nosort #'identity #'nreverse))) + (dolist (file (directory-files directory t nil nosort)) + (let ((is-dir (file-directory-p file))) + (unless (and is-dir + (member (file-name-nondirectory (directory-file-name file)) + '("." ".."))) + (push file files) + (when (and is-dir + (or follow-links (not (file-symlink-p file))) + (if (functionp recurse) (funcall recurse file) recurse)) + (push file dirs))))) + (apply #'stream-append + (stream (funcall reverse-fun files)) + (mapcar + (lambda (dir) (stream-of-directory-files-1 dir nosort recurse follow-links)) + (funcall reverse-fun dirs)))) + (stream-empty)))) + +(defun stream-of-directory-files (directory &optional full nosort recurse follow-links filter) + "Return a stream of names of files in DIRECTORY. +Call `directory-files' to list file names in DIRECTORY and make +the result a stream. Don't include files named \".\" or \"..\". +The arguments FULL and NOSORT are directly passed to +`directory-files'. + +Third optional argument RECURSE non-nil means recurse on +subdirectories. If RECURSE is a function, it should be a +predicate accepting one argument, an absolute file name of a +directory, and return non-nil when the returned stream should +recurse into that directory. Any other non-nil value means +recurse into every readable subdirectory. + +Even with recurse non-nil, don't descent into directories by +following symlinks unless FOLLOW-LINKS is non-nil. + +If FILTER is non-nil, it should be a predicate accepting one +argument, an absolute file name. It is used to limit the +resulting stream to the files fulfilling this predicate." + (let* ((stream (stream-of-directory-files-1 directory nosort recurse follow-links)) + (filtered-stream (if filter (seq-filter filter stream) stream))) + (if full filtered-stream + (seq-map (lambda (file) (file-relative-name file directory)) filtered-stream)))) (provide 'stream) ;;; stream.el ends here -- 2.7.0 --=-=-=--