From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Noam Postavsky Newsgroups: gmane.emacs.bugs Subject: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' Date: Wed, 24 Apr 2019 23:20:44 -0400 Message-ID: <87imv2rcs3.fsf@gmail.com> References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="132031"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) Cc: Nicolas Petton , 30626@debbugs.gnu.org To: Michael Heerdegen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Thu Apr 25 05:27:29 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hJV2n-000YEH-18 for geb-bug-gnu-emacs@m.gmane.org; Thu, 25 Apr 2019 05:27:29 +0200 Original-Received: from localhost ([127.0.0.1]:50516 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hJV2l-0000bM-Rr for geb-bug-gnu-emacs@m.gmane.org; Wed, 24 Apr 2019 23:27:27 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:51686) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hJV2a-0000SE-Gu for bug-gnu-emacs@gnu.org; Wed, 24 Apr 2019 23:27:17 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hJUwY-0000eA-Ih for bug-gnu-emacs@gnu.org; Wed, 24 Apr 2019 23:21:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:43529) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hJUwY-0000e0-Ej for bug-gnu-emacs@gnu.org; Wed, 24 Apr 2019 23:21:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hJUwY-0003IQ-9s for bug-gnu-emacs@gnu.org; Wed, 24 Apr 2019 23:21:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Noam Postavsky Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 25 Apr 2019 03:21:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 30626 X-GNU-PR-Package: emacs Original-Received: via spool by 30626-submit@debbugs.gnu.org id=B30626.155616245512634 (code B ref 30626); Thu, 25 Apr 2019 03:21:02 +0000 Original-Received: (at 30626) by debbugs.gnu.org; 25 Apr 2019 03:20:55 +0000 Original-Received: from localhost ([127.0.0.1]:57072 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hJUwQ-0003He-Qf for submit@debbugs.gnu.org; Wed, 24 Apr 2019 23:20:55 -0400 Original-Received: from mail-qk1-f179.google.com ([209.85.222.179]:41866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hJUwP-0003HL-4C for 30626@debbugs.gnu.org; Wed, 24 Apr 2019 23:20:53 -0400 Original-Received: by mail-qk1-f179.google.com with SMTP id l199so2523173qke.8 for <30626@debbugs.gnu.org>; Wed, 24 Apr 2019 20:20:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=B613M59uLOwonp+Ve9ApLfmN8duCQrbn/dSoIbyLdFs=; b=n329/+92cyWcithXwjj8PYvleBj8uot47XBEru/vy73diyjpl4upokWiNQ3jjwekst RbEtIuY0wX5CVH2Fps2CfvdXX5eS+Pn9tQH3c0nekWzjt6BX8/92aSsAMZOapeNlUyPx FWdtbA1fwKt7teW+0Q65bqdQVDur4HL8Mi2v+hfhuvu+aLPZLhyrNQFI0F3AE0r5zt4Y TNLfl84L7RpViv49MBAFkycfriQQor6AMvTz7aUrAlImC4gRN3iFHeue/YWmtzSGob2q Hf8ej5SOKF+AU792CVaCOQAVb5TF3wAPdtA87D0Xidy+n6lznjs8o9qfN0NhI1jFJ7n3 TmqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=B613M59uLOwonp+Ve9ApLfmN8duCQrbn/dSoIbyLdFs=; b=JTfOuRFbPHIWZj2D7RjEwn1c8GjZkpqmL1ckN4XFB/hjI0nM3YVDLF3gonET7bR/bR 3k28Da6Ocbzo07RcQClhIbJ27bVmzcPvyXsWWuUxt6gJILNCYT/HVDbDNPZcRh8Fk/WC pc/2F9wLbEzqTkkdnAFTboYPyp2hBxTG72hr/grMjaRfY0FogfB1qXJU7XpG3ZxUpnNc GDamfXplYT4baQQ6WimAYG9Sz1I4PXlJiHwDZWyHyVxuXKDGIOpoVuqlLfF/rkwFsNln eUWXV5cF1DPpAwAbDUW/z+1xl8AjbtplJikG1lyMn1G4kdgiTTlpAOs3CVnDc3qi5Pz0 7QYg== X-Gm-Message-State: APjAAAXgnOpH8vq+/stiKFGzBnpqncR5MbMkm3lf7cTgD6KJdl++m9cw UG+jUz5u6rCXFIVFBxs7bp4= X-Google-Smtp-Source: APXvYqz56Ri/ufllS++XelCK3FVVU32bTPQlSLNOEunK87KKHnnP3hyLrrtdNdEWjmEGrfvLZwfSDg== X-Received: by 2002:a37:4ed5:: with SMTP id c204mr27811819qkb.68.1556162447473; Wed, 24 Apr 2019 20:20:47 -0700 (PDT) Original-Received: from minid (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id b187sm1216445qkd.73.2019.04.24.20.20.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 24 Apr 2019 20:20:45 -0700 (PDT) In-Reply-To: <87bmg91ity.fsf@web.de> (Michael Heerdegen's message of "Wed, 28 Feb 2018 11:58:49 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:158222 Archived-At: --=-=-= Content-Type: text/plain Michael Heerdegen writes: >> I don't have a quick answer for the general case, but I think it's a bug >> in stream.el that it's creating such large structures in the first >> place. As far as I understand it, the point of streams is to handle >> long lists by encoding them as >> >> (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST) > > Yes, that's exactly how it's implemented. When requesting more elements > from the stream, that becomes > > (FIRST-VALUE . > (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST)) > > When we loop over the string, the cons whose car is the FIRST-VALUE, > let's call it cons1, is immediately thrown away, and we continue with > > (SECOND-VALUE . FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST) Coming back to this again. I think I got lost in the weeds of the byte-code function objects before. The core problem is that streams are not exactly encoded like the above, because even after forcing it you don't have just a plain SECOND-VALUE stored in the stream. The original FUNCTION-TO-PRODUCE-MORE-REST-OF-LIST stays around and keeps referencing all the code and all the following elements. So a possible solution is change the stream to get rid of the lambda part and just leave the computed value after it's forced. With the following patch (stream-flush (stream-range 1 1000000)) succeeds: --=-=-= Content-Type: text/x-diff Content-Disposition: inline; filename=0001-WIP-Drop-forced-lambda-s-from-stream-Bug-30626.patch Content-Description: patch >From 5e7139618f1b4cebbe9785ea5a9303b80d1b2c92 Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Wed, 24 Apr 2019 22:51:19 -0400 Subject: [PATCH] [WIP] Drop forced lambda's from stream (Bug#30626) Let the stream id distinguish between forced and unforced stream values, replacing (think-delay BODY) with just (lambda () BODY). When the value is forced, replace the lambda with its result. --- packages/stream/stream.el | 30 +++++++++++++++++++++--------- 1 file changed, 21 insertions(+), 9 deletions(-) diff --git a/packages/stream/stream.el b/packages/stream/stream.el index 3f6bc4b5b..fa7a3b520 100644 --- a/packages/stream/stream.el +++ b/packages/stream/stream.el @@ -65,18 +65,28 @@ (eval-when-compile (require 'cl-lib)) (require 'seq) -(require 'thunk) (eval-and-compile - (defconst stream--identifier '--stream-- - "Symbol internally used to identify streams.")) + (defconst stream--fresh-identifier '--stream-fresh-- + "Symbol internally used to streams whose head was not evaluated.") + (defconst stream--evald-identifier '--stream-evald-- + "Symbol internally used to streams whose head was evaluated.")) (defmacro stream-make (&rest body) "Return a stream built from BODY. BODY must return nil or a cons cell whose cdr is itself a stream." (declare (debug t)) - `(list ',stream--identifier (thunk-delay ,@body))) + `(list ',stream--fresh-identifier (lambda () ,@body))) + +(defun stream-force (stream) + (cond + ((eq (car-safe stream) stream--evald-identifier) + (cadr stream)) + ((eq (car-safe stream) stream--fresh-identifier) + (setf (car stream) stream--evald-identifier) + (setf (cadr stream) (funcall (cadr stream)))) + (t (signal 'wrong-type-argument (list 'streamp stream))))) (defmacro stream-cons (first rest) "Return a stream built from the cons of FIRST and REST. @@ -159,24 +169,26 @@ (defun stream-range (&optional start end step) (defun streamp (stream) "Return non-nil if STREAM is a stream, nil otherwise." - (eq (car-safe stream) stream--identifier)) + (let ((car (car-safe stream))) + (or (eq car stream--fresh-identifier) + (eq car stream--evald-identifier)))) (defun stream-empty () "Return a new empty stream." - (list stream--identifier (thunk-delay nil))) + (list stream--evald-identifier nil)) (defun stream-empty-p (stream) "Return non-nil if STREAM is empty, nil otherwise." - (null (thunk-force (cadr stream)))) + (null (cadr (stream-force stream)))) (defun stream-first (stream) "Return the first element of STREAM. Return nil if STREAM is empty." - (car (thunk-force (cadr stream)))) + (car (stream-force stream))) (defun stream-rest (stream) "Return a stream of all but the first element of STREAM." - (or (cdr (thunk-force (cadr stream))) + (or (cdr (stream-force stream)) (stream-empty))) (defun stream-append (&rest streams) -- 2.11.0 --=-=-= Content-Type: text/plain The reason I didn't do this in thunk.el is that thunks are just bare lambdas, so there is no way to get rid it "from the inside", as it were. Whereas for streams, it's just a matter of list editing. Some additional things that I thought of changing, but I didn't yet: - stream--identifier vs just using '--stream-- directly, I don't see what's the benefit of indirection here. - stream-make should use cons instead of list (or maybe a struct?). - stream-empty should just be a constant. --=-=-=--