From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Bindat can exhaust memory when unpacking to vector Date: Sat, 04 Nov 2023 09:48:56 +0200 Message-ID: <83pm0q557b.fsf@gnu.org> References: <87pm16fnzv.fsf@iki.fi> <83zg09oa1v.fsf@gnu.org> <87a5s9m8ri.fsf@iki.fi> Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40676"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Petteri Hintsanen , Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Nov 04 08:50:03 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qzBPu-000AMj-Ez for ged-emacs-devel@m.gmane-mx.org; Sat, 04 Nov 2023 08:50:02 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qzBP1-0008L6-LU; Sat, 04 Nov 2023 03:49:07 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qzBOu-0008Kn-OQ for emacs-devel@gnu.org; Sat, 04 Nov 2023 03:49:01 -0400 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qzBOt-0006Fs-6Q; Sat, 04 Nov 2023 03:48:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=+N5e13wk2KAA97/XtrzxWQLzVUJlpe+zUFzpSWEjJF8=; b=WUUnn+e9oZoS 33j80PwV2Ll7apvUwsf1NA3q3cuY+G/FmB+49skB8USn85o1qODW8hMUYRUurPMVGunw3DAWOHd3p KWuvN8Ajtq5CiVuGY1Oi8RHzYs2qRHjXjLi5/1fdsUq/bWJp2vUHWAiu44LGrTk1iy4m94f7I57HE EJYXOVeGNtoHzGS93g1n373NJ2PCwURv3usjd3o0/hwxIdEx4EDFI6r3XvV8dtQ2K/xxdYDoVydXF KIeFXRJ5UIaX0ZrTmSxv3LK2SpbURabTgRTH8phhUgICZviHsD5dtQDtFkvlE02ZR8ZgPY4GcJBBh TOd7AEp8lF/uDbP+ALg1gA==; In-Reply-To: <87a5s9m8ri.fsf@iki.fi> (message from Petteri Hintsanen on Mon, 23 Oct 2023 22:36:01 +0300) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:312191 Archived-At: > X-Spam-Status: No, score=0.653 tagged_above=-9999 required=6 > tests=[SPF_HELO_NONE=0.001, SPF_NEUTRAL=0.652] autolearn=disabled > From: Petteri Hintsanen > Cc: emacs-devel@gnu.org > Date: Mon, 23 Oct 2023 22:36:01 +0300 > > Eli Zaretskii writes: > > > Hang or just slow down tremendously due to paging? How much time did > > you wait before deciding that Emacs hung? And how much memory do you > > have on that machine? > > Sorry, I was careless with wording; obviously what I meant was "slowed > down due to paging." I did not wait for long, maybe a minute or so. > This machine has ample memory, 16 GB RAM plus another 16 GB swap space. > > > Can you argue why this should be considered a problem for Emacs to > > solve, and not a cockpit error on the part of the Lisp program that > > makes such calls? > > No, not really. Now when I think about it, it is fairly obvious that > the programmer should realize this when using such low level mechanism > as Bindat. > > >> Nonetheless, I think it would be prudent to mention this potential > >> pitfall in the Emacs Lisp manual so that users become aware of it. > >> I can write a texinfo patch if you agree. > > > > What would we say? that unpacking vectors larger than available memory > > would cause Emacs to run out of memory? > > I was thinking of some kind of note about validation, e.g.: > > Notice that unpacking data to a vector of length LEN will > unconditionally preallocate memory with at least LEN bytes. If LEN is > calculated from the input, it is a good idea to validate its value > before unpacking. > > But, again, users of Bindat can be expected to know that tainted data > should be sanitized. And what is described above is an implementation > detail. So it does not make much sense to mention this in the manual. Stefan, any comments?