From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Bindat can exhaust memory when unpacking to vector Date: Mon, 23 Oct 2023 14:25:16 +0300 Message-ID: <83zg09oa1v.fsf@gnu.org> References: <87pm16fnzv.fsf@iki.fi> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="38059"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Petteri Hintsanen Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Oct 23 13:26:18 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qut4c-0009cm-2u for ged-emacs-devel@m.gmane-mx.org; Mon, 23 Oct 2023 13:26:18 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qut3h-0005Ua-Lh; Mon, 23 Oct 2023 07:25:21 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qut3c-0005Sg-9B for emacs-devel@gnu.org; Mon, 23 Oct 2023 07:25:16 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qut3a-0002EN-UF; Mon, 23 Oct 2023 07:25:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=qQTeZg4+I3dANd6Mwqixwb6YRdxEg591KbV8VziJBqs=; b=qTW7eftTyYW2 jfNukp99whyuPn+mfjxZ7vMRdCyM40p2QHpWecSF0MKokXHE4G33/jxr4uxsK7HOQUS4+7PCoSxoB qM7ZWoMyxVWbUBCLHdxD/SFHnBVzlEf9q9ML5OmREeetBUC40ptRFYaOLNt0/LV24+PRtZ+1eA+Py 0VXMigbYXTuMGuNAf8CLEsC8xIJK0CCJ4jaI4L3prGnzeSu7n6CrMrRPjUtOH2JUMTqqSrweMGy57 yJy7STaOUPsYzs12C8iURzqSBU7m1S/ZWlm3wIidkmyTK8LkcvA/nLQ6EudKq384txihZFsJdKXGJ E5ToUDi5W0HLkP0IU5/GxQ==; In-Reply-To: <87pm16fnzv.fsf@iki.fi> (message from Petteri Hintsanen on Sun, 22 Oct 2023 22:36:36 +0300) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:311703 Archived-At: > From: Petteri Hintsanen > Date: Sun, 22 Oct 2023 22:36:36 +0300 > > It appears that bindat-unpack can exhaust memory when it unpacks to a > vector. Consider the following Bindat type expression: > > (require 'bindat) > (defconst foo-bindat-spec > (bindat-type > (length uint 32) > (data vec length))) > > If you then, for example, eval this > > (bindat-unpack foo-bindat-spec [255 255 255 255 0 1 2 3 4 5]) > > it will result in (make-vector 4294967295 0), which makes Emacs hang. Hang or just slow down tremendously due to paging? How much time did you wait before deciding that Emacs hung? And how much memory do you have on that machine? The above call to make-vector is supposed to signal a memory-full error if the number of elements is more than the system can support, but it cannot be smart enough to consider the amount of available VM. > Now it can be argued whether this is a problem or not. Can you argue why this should be considered a problem for Emacs to solve, and not a cockpit error on the part of the Lisp program that makes such calls? > Nonetheless, I think it would be prudent to mention this potential > pitfall in the Emacs Lisp manual so that users become aware of it. > I can write a texinfo patch if you agree. What would we say? that unpacking vectors larger than available memory would cause Emacs to run out of memory? > [Side note: vec seems to be the only bindat type that suffers from this. > str (string) is another type that has a varying length, but it is > unpacked by `substring' which will implicitly guard against nonsensical > lengths.] See above: make-vector is also protected.