From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: John Mastro Newsgroups: gmane.emacs.bugs Subject: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' Date: Fri, 2 Mar 2018 13:48:23 -0800 Message-ID: References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> <87lgfd52by.fsf@gmail.com> <87bmg91ity.fsf@web.de> <83h8q1yuin.fsf@gnu.org> <87po4pnl0a.fsf@web.de> <837eqxyqoe.fsf@gnu.org> <87lgfdnf86.fsf@web.de> <87muzs11hk.fsf@web.de> <87606e4lel.fsf@gmail.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: blaine.gmane.org 1520027235 24024 195.159.176.226 (2 Mar 2018 21:47:15 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 2 Mar 2018 21:47:15 +0000 (UTC) Cc: Michael Heerdegen , Nicolas Petton , Noam Postavsky To: 30626@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Mar 02 22:47:10 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ersWC-0005Lk-G5 for geb-bug-gnu-emacs@m.gmane.org; Fri, 02 Mar 2018 22:47:08 +0100 Original-Received: from localhost ([::1]:37598 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ersYE-0004w4-Mb for geb-bug-gnu-emacs@m.gmane.org; Fri, 02 Mar 2018 16:49:14 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:60602) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ersY5-0004ut-RE for bug-gnu-emacs@gnu.org; Fri, 02 Mar 2018 16:49:06 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ersY2-0000uG-Nl for bug-gnu-emacs@gnu.org; Fri, 02 Mar 2018 16:49:05 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:33672) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ersY2-0000u7-JO for bug-gnu-emacs@gnu.org; Fri, 02 Mar 2018 16:49:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ersY2-0003gu-7m for bug-gnu-emacs@gnu.org; Fri, 02 Mar 2018 16:49:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: John Mastro Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 02 Mar 2018 21:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 30626 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 30626-submit@debbugs.gnu.org id=B30626.152002733114172 (code B ref 30626); Fri, 02 Mar 2018 21:49:02 +0000 Original-Received: (at 30626) by debbugs.gnu.org; 2 Mar 2018 21:48:51 +0000 Original-Received: from localhost ([127.0.0.1]:41569 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ersXr-0003gV-Iu for submit@debbugs.gnu.org; Fri, 02 Mar 2018 16:48:51 -0500 Original-Received: from mail-qk0-f175.google.com ([209.85.220.175]:45622) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ersXq-0003gH-BP for 30626@debbugs.gnu.org; Fri, 02 Mar 2018 16:48:50 -0500 Original-Received: by mail-qk0-f175.google.com with SMTP id g2so13768810qkd.12 for <30626@debbugs.gnu.org>; Fri, 02 Mar 2018 13:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=EFpfmREqGYP06GiSZ4rWHh17p3ivcCfe3pTRKT5DfBI=; b=ZzL5yBjDsDgJ34syzAgv56Y/2OPhI2rLLHcniPjlbldqJRXOJjF7zynNRm0ywP2AyG 6VcU0tYzN7Lvw5N8lq75JSGO34zRQN7SdlX8MtL5C7kfW1Inq5qh5WPRMjxd5IWhuT1K TSlJ+fBcIDUbIc9pcX7/s8hisxqigM7PmQIfK+2hsAK6yIb+DiXReG6QjHCuJwiKoVNp 8uY/5S4KBCb9t+l0CqO/vhAC52/HAxQiayG9YDMQZc7SObRD8FVCdKepE6ZGcOgcDHmB cmqucT/LlbMe2RHN7N6jEwEB/AsHU82RJh/CtGZqHe4cvwgNlgkobA2uuex5DR7vuHuV PqBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=EFpfmREqGYP06GiSZ4rWHh17p3ivcCfe3pTRKT5DfBI=; b=E49jvqKaS+VdmW08M4dNOAyw1mRHJzC/g5xqdrKE1P3Bx4KhSmMkDMK0lrVtdQqSHy tdjPFCYncn4LmYMxtSCfVbF7l3DRZoI/mYXIhxLGzjrI0uV9sE0TTlkZTyhHxRDzn7RO x6/454i1I0Ua17VZuU+D10SZ8egMu2ziKthUQBAR26CNe+PZtdJhE/eztZOIU7ELg9pR K66X5BQnVeoZzk2+NDDiuvu/ZE6X2/Uwzua016l7gRHIKTgw/GbWe7TvfYcmmLnFxW6f Ktq+x+ULY0Y2W71qIBVcXijrcEUiDlkysPEhlIAK1OKzNwcSNxw4nc2GnkaDBkMD8n1m Gr5w== X-Gm-Message-State: AElRT7E7fvZUH8I+88KfisdWcVo3xhjqEZ4Gk5XSDoJm2U2I88U0OoD1 UMAnvK77QCvI5Fc0HUS+dD9a1djZo8eJlU6WxO8KSloc X-Google-Smtp-Source: AG47ELvC+LL7nH/Q27qTI11GnXmLnSZm1YnevYgCXL4Px/Qwec6/Eq46vGFh7o9FPLJeU6DUXC+1D8kYBKg0/5ZSEeE= X-Received: by 10.55.78.212 with SMTP id c203mr10596248qkb.351.1520027324330; Fri, 02 Mar 2018 13:48:44 -0800 (PST) Original-Received: by 10.200.52.16 with HTTP; Fri, 2 Mar 2018 13:48:23 -0800 (PST) In-Reply-To: <87606e4lel.fsf@gmail.com> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:143847 Archived-At: Noam Postavsky wrote: > Aha, but the following also crashes, whether compiled or not: > > ;; -*- lexical-binding: t -*- > > (require 'stream) > > (let* ((stream0 (stream-range 1 1000000)) > (stream stream0)) > (while (not (stream-empty-p stream)) > (cl-callf stream-rest stream))) > > So the problem is that the initial stream0 object can reach the entire > unfolding stream as it goes, and just holding on to that reference is > enough to keep the whole thing in memory. > > Now, I can see that letting stream0 automagically get access to the > unfolded result can be an optimization in some cases, although in this > case it's a pessimization. It could also affect the semantics if > unfolding the stream has side effects, not sure if stream.el makes > guarantees about that though. Clojure has/had a similar issue. They describe this scenario (having a live reference to the beginning of the stream, preventing GC from collecting it) as "holding onto the head" of the stream (in Clojure they're called lazy seqs). Their solution was what they call "locals clearing". The compiler tracks the lifetimes of local bindings and "clears" them (by setting them to nil/null) after their point of last use, e.g.: (let* ((stream0 (stream-range 1 1000000)) (stream stream0)) (setq stream0 nil) ;; <<< Inserted by compiler (while (not (stream-empty-p stream)) (cl-callf stream-rest stream))) If the code does reference stream0 later, locals clearing can't help you, but that's considered a "if it hurts, don't do it" situation. This probably isn't practical for Emacs, especially since it could only work for byte-compiled code, but thought the prior art may be of interest. John