From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Noam Postavsky Newsgroups: gmane.emacs.bugs Subject: bug#30626: 26.0.91; Crash when traversing a `stream-of-directory-files' Date: Tue, 27 Feb 2018 20:29:21 -0500 Message-ID: <87lgfd52by.fsf@gmail.com> References: <87inaiss6l.fsf@web.de> <6FCF6ACA-4F29-4B6B-BE9D-D7130C6E9495@gnu.org> <87fu5moe4c.fsf@web.de> <877eqyocro.fsf@web.de> <83zi3uz4nb.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1519781295 20386 195.159.176.226 (28 Feb 2018 01:28:15 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Wed, 28 Feb 2018 01:28:15 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.90 (gnu/linux) Cc: Michael Heerdegen , 30626@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Feb 28 02:28:11 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqqXS-0004hh-Rn for geb-bug-gnu-emacs@m.gmane.org; Wed, 28 Feb 2018 02:28:11 +0100 Original-Received: from localhost ([::1]:41273 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqqZT-0006VO-Ho for geb-bug-gnu-emacs@m.gmane.org; Tue, 27 Feb 2018 20:30:15 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:53913) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqqZL-0006V8-K9 for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 20:30:08 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqqZG-0003Gd-H6 for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 20:30:07 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:56603) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eqqZG-0003GV-DW for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 20:30:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1eqqZG-00023y-4x for bug-gnu-emacs@gnu.org; Tue, 27 Feb 2018 20:30:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Noam Postavsky Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 28 Feb 2018 01:30:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 30626 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 30626-submit@debbugs.gnu.org id=B30626.15197813697873 (code B ref 30626); Wed, 28 Feb 2018 01:30:02 +0000 Original-Received: (at 30626) by debbugs.gnu.org; 28 Feb 2018 01:29:29 +0000 Original-Received: from localhost ([127.0.0.1]:36267 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqqYj-00022v-Kw for submit@debbugs.gnu.org; Tue, 27 Feb 2018 20:29:29 -0500 Original-Received: from mail-it0-f49.google.com ([209.85.214.49]:40704) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eqqYi-00022i-0I for 30626@debbugs.gnu.org; Tue, 27 Feb 2018 20:29:28 -0500 Original-Received: by mail-it0-f49.google.com with SMTP id v186so1509606itc.5 for <30626@debbugs.gnu.org>; Tue, 27 Feb 2018 17:29:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=jU8JUjE6W55VY6I37oKWn09TjMLw2bqetg/bhOTki3g=; b=b1sFUlg2c/0p86qMHVI0A6G4Cuef0V0lN92r+W1JoLkvH50AEj+HUz32gBIuq0I0IU lBG5i1MJV19FATD6xjV6VeXoVgG8g8gPQDxdPeoDWXmX+il7V1uYjNxaCtmtIOgUA1sZ atyOtXm0D5rCBufkwp0Hh9iCaxHHPfUI9QwbLYmUoTZynNhjkeqIAv92HCcgwui2mDWw PrZ7agsTYoN98tf9jSqu4OsLpdlDOs5j+vE+ugmXUeyPtjqlA7V93p76z0o1O+c75SCq 37meonZOa+8XSKN5ajzIgISwh0GhCBMoPesoltFYyQNDxg+a+/okl4IxkL6h1TZBs4xk NQyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=jU8JUjE6W55VY6I37oKWn09TjMLw2bqetg/bhOTki3g=; b=Uijut9rvzW1/hMqIIgycpcaVSg7/oBu0WdRxzKn2jpt848bwe7xFpADkhIuSjIfJ9x LPjj0e/5EA2zjrdw/BBcXKzKMyLJ8pPY4/mghcld1UYs0ZoQpY72MEkm5XziHXWxQzf3 XWLu5G8CDa9injIwlLT9gKLmmLZsb1d2Zp4gOrloRQOmOHGG/tpHBE88xSQqxtbCLHeS VeAUir7+XrBznZhAnl2dnYggac22/x3DG0ywc54svKe3uxyo8dXb4W6C0nIe4H2f2QUw 03UwY7XeuiY03UE/sHOXlFB+imbz2kgp2oQZNvAnjNj2Dzxno1FsOwIOhm4glEb8OHMa FlWg== X-Gm-Message-State: APf1xPBmX0JPUyobkCYKRubPmLFhpeFFzsF8ju62DUT6VALi64FIMVZ5 CEqjxH5S4s3dHAQzIsl9j6/2HQ== X-Google-Smtp-Source: AG47ELtuM8/FMdTKLAc+W7C0nSbU3hvIe8XbADqTWfqrkJJM6x1QT7EIXEhSndLFCCFGyq5ZCnSrfw== X-Received: by 10.36.144.7 with SMTP id x7mr7071226itd.128.1519781362550; Tue, 27 Feb 2018 17:29:22 -0800 (PST) Original-Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id m32sm9372369iti.3.2018.02.27.17.29.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 27 Feb 2018 17:29:22 -0800 (PST) In-Reply-To: <83zi3uz4nb.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 27 Feb 2018 20:08:56 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:143727 Archived-At: Eli Zaretskii writes: >> From: Michael Heerdegen >> Cc: bug-gnu-emacs@gnu.org, 30626@debbugs.gnu.org >> Date: Tue, 27 Feb 2018 13:08:59 +0100 >> >> #+begin_src emacs-lisp >> (seq-doseq (_ (stream-range 1 1000000)) nil) >> #+end_src >> >> Note that this is executed as a loop due how to streams are implemented, >> although the definition of `seq-doseq' looks recursive. Doesn't look recursive to me, it expands to a call to seq-do, which uses a simple loop. >> But it seems that gc has a problem with the large number of conses >> created when processing that. > > What can we do instead in such cases? Stack-overflow protection > cannot work in GC, so you are shooting yourself in the foot by > creating such large recursive structures. By the time we get to GC, > where the problem will happen, it's too late, because the memory was > already allocated. > > Does anyone has a reasonable idea for avoiding the crash in such > programs? I don't have a quick answer for the general case, but I think it's a bug in stream.el that it's creating such large structures in the first place. As far as I understand it, the point of streams is to handle long lists by encoding them as (FIRST-VALUE . FUNCTION-TO-PRODUCE-REST-OF-LIST) so as to avoid allocating large amounts of memory. Is there an easy way to find out what the large structures are, and where they are coming from?