From: Noam Postavsky <npostavs@gmail.com>
To: Michael Heerdegen <michael_heerdegen@web.de>
Cc: Nicolas Petton <nicolas@petton.fr>, 30626@debbugs.gnu.org
Subject: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files'
Date: Wed, 24 Apr 2019 23:20:44 -0400 [thread overview]
Message-ID: <87imv2rcs3.fsf@gmail.com> (raw)
In-Reply-To: <87bmg91ity.fsf@web.de> (Michael Heerdegen's message of "Wed, 28 Feb 2018 11:58:49 +0100")
[-- Attachment #1: Type: text/plain, Size: 1382 bytes --]
Michael Heerdegen <michael_heerdegen@web.de> writes:
>> I don't have a quick answer for the general case, but I think it's a bug
>> in stream.el that it's creating such large structures in the first
>> place. As far as I understand it, the point of streams is to handle
>> long lists by encoding them as
>>
>> (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST)
>
> Yes, that's exactly how it's implemented. When requesting more elements
> from the stream, that becomes
>
> (FIRST-VALUE .
> (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST))
>
> When we loop over the string, the cons whose car is the FIRST-VALUE,
> let's call it cons1, is immediately thrown away, and we continue with
>
> (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST)
Coming back to this again. I think I got lost in the weeds of the
byte-code function objects before. The core problem is that streams are
not exactly encoded like the above, because even after forcing it you
don't have just a plain SECOND-VALUE stored in the stream. The original
FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST stays around and keeps referencing
all the code and all the following elements. So a possible solution is
change the stream to get rid of the lambda part and just leave the
computed value after it's forced. With the following patch
(stream-flush (stream-range 1 1000000)) succeeds:
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 2937 bytes --]
From 5e7139618f1b4cebbe9785ea5a9303b80d1b2c92 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@users.sourceforge.net>
Date: Wed, 24 Apr 2019 22:51:19 -0400
Subject: [PATCH] [WIP] Drop forced lambda's from stream (Bug#30626)
Let the stream id distinguish between forced and unforced stream
values, replacing (think-delay BODY) with just (lambda () BODY). When
the value is forced, replace the lambda with its result.
---
packages/stream/stream.el | 30 +++++++++++++++++++++---------
1 file changed, 21 insertions(+), 9 deletions(-)
diff --git a/packages/stream/stream.el b/packages/stream/stream.el
index 3f6bc4b5b..fa7a3b520 100644
--- a/packages/stream/stream.el
+++ b/packages/stream/stream.el
@@ -65,18 +65,28 @@
(eval-when-compile (require 'cl-lib))
(require 'seq)
-(require 'thunk)
(eval-and-compile
- (defconst stream--identifier '--stream--
- "Symbol internally used to identify streams."))
+ (defconst stream--fresh-identifier '--stream-fresh--
+ "Symbol internally used to streams whose head was not evaluated.")
+ (defconst stream--evald-identifier '--stream-evald--
+ "Symbol internally used to streams whose head was evaluated."))
(defmacro stream-make (&rest body)
"Return a stream built from BODY.
BODY must return nil or a cons cell whose cdr is itself a
stream."
(declare (debug t))
- `(list ',stream--identifier (thunk-delay ,@body)))
+ `(list ',stream--fresh-identifier (lambda () ,@body)))
+
+(defun stream-force (stream)
+ (cond
+ ((eq (car-safe stream) stream--evald-identifier)
+ (cadr stream))
+ ((eq (car-safe stream) stream--fresh-identifier)
+ (setf (car stream) stream--evald-identifier)
+ (setf (cadr stream) (funcall (cadr stream))))
+ (t (signal 'wrong-type-argument (list 'streamp stream)))))
(defmacro stream-cons (first rest)
"Return a stream built from the cons of FIRST and REST.
@@ -159,24 +169,26 @@ (defun stream-range (&optional start end step)
(defun streamp (stream)
"Return non-nil if STREAM is a stream, nil otherwise."
- (eq (car-safe stream) stream--identifier))
+ (let ((car (car-safe stream)))
+ (or (eq car stream--fresh-identifier)
+ (eq car stream--evald-identifier))))
(defun stream-empty ()
"Return a new empty stream."
- (list stream--identifier (thunk-delay nil)))
+ (list stream--evald-identifier nil))
(defun stream-empty-p (stream)
"Return non-nil if STREAM is empty, nil otherwise."
- (null (thunk-force (cadr stream))))
+ (null (cadr (stream-force stream))))
(defun stream-first (stream)
"Return the first element of STREAM.
Return nil if STREAM is empty."
- (car (thunk-force (cadr stream))))
+ (car (stream-force stream)))
(defun stream-rest (stream)
"Return a stream of all but the first element of STREAM."
- (or (cdr (thunk-force (cadr stream)))
+ (or (cdr (stream-force stream))
(stream-empty)))
(defun stream-append (&rest streams)
--
2.11.0
[-- Attachment #3: Type: text/plain, Size: 494 bytes --]
The reason I didn't do this in thunk.el is that thunks are just bare
lambdas, so there is no way to get rid it "from the inside", as it were.
Whereas for streams, it's just a matter of list editing.
Some additional things that I thought of changing, but I didn't yet:
- stream--identifier vs just using '--stream-- directly, I don't see
what's the benefit of indirection here.
- stream-make should use cons instead of list (or maybe a struct?).
- stream-empty should just be a constant.
next prev parent reply other threads:[~2019-04-25 3:20 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-02-27 9:22 bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' Michael Heerdegen
2018-02-27 11:21 ` Eli Zaretskii
2018-02-27 11:39 ` Michael Heerdegen
2018-02-27 12:08 ` Michael Heerdegen
2018-02-27 18:08 ` Eli Zaretskii
2018-02-28 1:29 ` Noam Postavsky
2018-02-28 10:58 ` Michael Heerdegen
2018-02-28 16:00 ` Eli Zaretskii
2018-02-28 16:20 ` Michael Heerdegen
2018-02-28 17:22 ` Eli Zaretskii
2018-02-28 18:25 ` Michael Heerdegen
2018-03-01 11:25 ` Michael Heerdegen
2018-03-01 15:00 ` Eli Zaretskii
2018-03-02 14:11 ` Noam Postavsky
2018-03-02 15:06 ` Michael Heerdegen
2018-03-02 15:43 ` Eli Zaretskii
2018-03-02 20:16 ` Nicolas Petton
2018-03-02 20:58 ` Nicolas Petton
2018-03-03 7:56 ` Michael Heerdegen
2018-03-03 7:54 ` Michael Heerdegen
2018-03-03 8:47 ` Nicolas Petton
2018-03-02 21:48 ` John Mastro
2018-03-03 23:00 ` Noam Postavsky
2018-03-04 15:56 ` Noam Postavsky
2018-03-04 17:02 ` Eli Zaretskii
2018-03-11 18:52 ` Noam Postavsky
2018-03-11 20:31 ` Eli Zaretskii
2018-03-11 21:51 ` Noam Postavsky
2018-03-12 3:28 ` Eli Zaretskii
2018-03-13 1:59 ` Noam Postavsky
2018-03-13 16:06 ` Eli Zaretskii
2018-03-14 0:09 ` Noam Postavsky
2018-03-15 16:34 ` Eli Zaretskii
2018-03-17 15:53 ` Noam Postavsky
2018-03-17 16:10 ` Eli Zaretskii
2018-03-17 16:27 ` Eli Zaretskii
2018-03-17 17:28 ` Noam Postavsky
2018-03-19 20:05 ` Eli Zaretskii
2019-04-25 3:20 ` Noam Postavsky [this message]
2019-04-25 5:19 ` Michael Heerdegen
2019-05-10 13:20 ` Michael Heerdegen
2019-05-25 20:29 ` Noam Postavsky
2019-05-26 0:32 ` Michael Heerdegen
2019-05-26 0:40 ` Noam Postavsky
2019-05-26 1:15 ` Michael Heerdegen
2019-06-04 0:26 ` Noam Postavsky
2018-02-28 11:05 ` Michael Heerdegen
2018-02-28 13:20 ` Nicolas Petton
2018-03-01 10:44 ` Daniel Colascione
2018-03-01 15:51 ` Noam Postavsky
2018-03-01 16:54 ` Michael Heerdegen
2018-03-01 17:15 ` Noam Postavsky
2018-03-02 7:08 ` Michael Heerdegen
2018-03-02 13:01 ` Noam Postavsky
2018-03-02 13:13 ` Michael Heerdegen
2018-03-02 13:04 ` Michael Heerdegen
2018-02-27 18:00 ` Eli Zaretskii
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87imv2rcs3.fsf@gmail.com \
--to=npostavs@gmail.com \
--cc=30626@debbugs.gnu.org \
--cc=michael_heerdegen@web.de \
--cc=nicolas@petton.fr \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).