From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lynn Winebarger Newsgroups: gmane.emacs.bugs Subject: bug#56793: Infinite loop in pure_alloc when dumping strings longer than 10K bytes Date: Wed, 27 Jul 2022 09:49:13 -0400 Message-ID: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000006a416105e4c9af49" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40437"; mail-complaints-to="usenet@ciao.gmane.io" To: 56793@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Jul 27 15:51:32 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oGhRk-000APS-KD for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 27 Jul 2022 15:51:32 +0200 Original-Received: from localhost ([::1]:40336 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oGhRj-0005uC-HT for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 27 Jul 2022 09:51:31 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41228) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oGhQK-0005P2-Rj for bug-gnu-emacs@gnu.org; Wed, 27 Jul 2022 09:50:13 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:37407) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oGhQI-0005wB-Pd for bug-gnu-emacs@gnu.org; Wed, 27 Jul 2022 09:50:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oGhQI-0004Gz-Es for bug-gnu-emacs@gnu.org; Wed, 27 Jul 2022 09:50:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Lynn Winebarger Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 27 Jul 2022 13:50:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 56793 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.165892978216391 (code B ref -1); Wed, 27 Jul 2022 13:50:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 27 Jul 2022 13:49:42 +0000 Original-Received: from localhost ([127.0.0.1]:55389 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oGhPx-0004GI-Ts for submit@debbugs.gnu.org; Wed, 27 Jul 2022 09:49:42 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:54376) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oGhPs-0004G3-AT for submit@debbugs.gnu.org; Wed, 27 Jul 2022 09:49:40 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:40854) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oGhPm-0004ow-NU for bug-gnu-emacs@gnu.org; Wed, 27 Jul 2022 09:49:36 -0400 Original-Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]:55885) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oGhPk-0005s4-LP for bug-gnu-emacs@gnu.org; Wed, 27 Jul 2022 09:49:30 -0400 Original-Received: by mail-pj1-x1030.google.com with SMTP id b10so16437498pjq.5 for ; Wed, 27 Jul 2022 06:49:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to; bh=RV7X4lj+2xsb7/6HiqkSJ8Jg/Dlqwp7jHmiUdUgJPuI=; b=FniHQ47xe/4WWTTVs+ipL9R/zUZqNS9XHJyCxn4w+9eR+RGWd/HvsfEyzOFt5mcnqG kQQj/Kbhe9T7saemaVbu5xah6KDtdWjgk9/H3+Nm8k/AKtvXdYlYege0C1/JNvmFFetj 8Lv+hwFYEPzsBSi5xhTXhDGS0tD/qTpXbR827OGShPUxSPdj00RsftfHKWLCNj9I3uhr zhZuRMPDkk0bCLQAz/OpkzXPByfyz6Nbl4k2iza9WpLgV4gorxAG712bsBqdXqE9mMAb x1a2KYfiI27GjU3iE8yJhD1OWLgsaYx7xhnQ98M9nh4465NhEzkl5Tm1TpnQ5ZRpwWK8 rYgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=RV7X4lj+2xsb7/6HiqkSJ8Jg/Dlqwp7jHmiUdUgJPuI=; b=k++5ihJlE15Y70gjcgz9khZB5f39ObdIygIfxox/HkJGxw9AFWbNycY+fKQYnY58iR RpKD3EpUh/OyighKdFnPtAymuE44jygqgRnEoPhi7qVeCytVW1sGa5toK+EHyug2jPuP 08Xcj2l5Z1Syb1NVtSo6iiE/EFoakOtSPSzjmAxxLh/4F496gnwDthkj6oIUnsDIVGmy S8kFAO0cVwFg/6UnNoYA/xJkwtaDGkWvk0eXDcoT4TTL8/bTAl8PBYt+KAYRJDyGtzSp 8CCT41abuSOBFRYL9TaUNZatfVAulOrUiLxuohvcxijBAY7Eyz718OO6juYrCq8zUy4C cjYg== X-Gm-Message-State: AJIora+2EKRFvTZxrvHtIiaLwcOhLU91yhA4zWMgA+YtP1uysnXgFt8y +tiM+pSf0ML6LMpXUMpI47+be1LIx6ZkDTtPKP/1XpFVctE= X-Google-Smtp-Source: AGRyM1t1nwR7tlRcVIBmUUiMS0vzXiB+u3vK4HpQ9qIdL/c3qGstmfDdqoTGgnxJPQ2tXfkdUFg4rTxehipIlZrTmWQ= X-Received: by 2002:a17:903:260f:b0:16d:bafc:86c with SMTP id jd15-20020a170903260f00b0016dbafc086cmr1131189plb.121.1658929766087; Wed, 27 Jul 2022 06:49:26 -0700 (PDT) Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=owinebar@gmail.com; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:238037 Archived-At: --0000000000006a416105e4c9af49 Content-Type: text/plain; charset="UTF-8" I apologize for not being able to include significant details of the build, as this is happening on a sandboxed system in a proprietary context. I've been attempting to dump emacs built from the 28.1 tarball with a large number of core libraries preloaded. I have observed runaway allocation when attempting to dump with native-compilation enabled and with native-compilation disabled. Using gdb I was able to identify the culprit as an infinite "goto again" loop in pure_alloc: static void * pure_alloc (size_t size, int type) { void *result; again: if (type >= 0) { /* Allocate space for a Lisp object from the beginning of the free space with taking account of alignment. */ result = pointer_align (purebeg + pure_bytes_used_lisp, LISP_ALIGNMENT); pure_bytes_used_lisp = ((char *)result - (char *)purebeg) + size; } else { /* Allocate space for a non-Lisp object from the end of the free space. */ ptrdiff_t unaligned_non_lisp = pure_bytes_used_non_lisp + size; char *unaligned = purebeg + pure_size - unaligned_non_lisp; int decr = (intptr_t) unaligned & (-1 - type); pure_bytes_used_non_lisp = unaligned_non_lisp + decr; result = unaligned - decr; } pure_bytes_used = pure_bytes_used_lisp + pure_bytes_used_non_lisp; if (pure_bytes_used <= pure_size) return result; /* Don't allocate a large amount here, because it might get mmap'd and then its address might not be usable. */ int small_amount = 10000; eassert (size <= small_amount - LISP_ALIGNMENT); purebeg = xzalloc (small_amount); pure_size = small_amount; pure_bytes_used_before_overflow += pure_bytes_used - size; pure_bytes_used = 0; pure_bytes_used_lisp = pure_bytes_used_non_lisp = 0; /* Can't GC if pure storage overflowed because we can't determine if something is a pure object or not. */ garbage_collection_inhibited++; goto again; } For example, the issue was triggered while attempting to dump ibuffer.el - a docstring has 10,655 characters, and the "eassert" is turned off with the usual CFLAGS, so it just never stops attempting to allocate. Since unexec is disabled, I don't even understand why avoiding mmap'ed memory is important. Pure space is just going to be copied into the cold storage area of the pdmp file anyway, no? In any case, changing the small amount to 20000 made the issue go away for the purpose of testing, but I'm not sure why there isn't some comparison of the size of the object to the small amount in the logic - either determining the minimum to allocate in one chunk, or attempting to obtain a contiguous chunk by successive xzallocs until the required size is obtained. Lynn --0000000000006a416105e4c9af49 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I apologize for not being able to include significant deta= ils of the build, as this is happening on a sandboxed system in a proprieta= ry context.
I've been attempting to dump emacs built from the 28.1 t= arball with a large number of core libraries preloaded.=C2=A0 I have observ= ed runaway allocation when attempting to dump with native-compilation enabl= ed and with native-compilation disabled. =C2=A0
Using gdb I was able to = identify the culprit as an infinite "goto again" loop in pure_all= oc:
stat= ic void *
pure_alloc (size_t size, int type)
{
void *result;
ag= ain:
if (type >=3D 0)
{
/* Allocate space for a Lisp object fro= m the beginning of the free
space with taking account of alignment. */result =3D pointer_align (purebeg + pure_bytes_used_lisp, LISP_ALIGNMENT)= ;
pure_bytes_used_lisp =3D ((char *)result - (char *)purebeg) + size;}
else
{

/* Allocate space for a non-Lisp object from the end o= f the free
space. */
ptrdiff_t unaligned_non_lisp =3D pure_bytes_used= _non_lisp + size;
char *unaligned =3D purebeg + pure_size - unaligned_no= n_lisp;
int decr =3D (intptr_t) unaligned & (-1 - type);
pure_byt= es_used_non_lisp =3D unaligned_non_lisp + decr;
result =3D unaligned - d= ecr;
}
pure_bytes_used =3D pure_bytes_used_lisp + pure_bytes_used_non= _lisp;
if (pure_bytes_used <=3D pure_size)
return result;
/* Do= n't allocate a large amount here,
because it might get mmap'd an= d then its address
might not be usable. */
int small_amount =3D 10000= ;
eassert (size <=3D small_amount - LISP_ALIGNMENT);
purebeg =3D x= zalloc (small_amount);
pure_size =3D small_amount;
pure_bytes_used_be= fore_overflow +=3D pure_bytes_used - size;
pure_bytes_used =3D 0;
pur= e_bytes_used_lisp =3D pure_bytes_used_non_lisp =3D 0;
/* Can't GC if= pure storage overflowed because we can't determine
if something is = a pure object or not. */
garbage_collection_inhibited++;
goto again;<= br>}
For example, the issue was triggered while attempting to d= ump ibuffer.el=C2=A0 - a docstring has 10,655 characters, and the "eas= sert" is turned off with the usual CFLAGS, so it just never stops atte= mpting to allocate.
Since unexec is disabled, I don't even understa= nd why avoiding mmap'ed memory is important.=C2=A0 Pure space is just g= oing to be copied into the cold storage area of the pdmp file anyway, no?= =C2=A0=C2=A0
In any case, changing the small amount to 20000 made= the issue go away for the purpose of testing, but I'm not sure why the= re isn't some comparison of the size of the object to the small amount = in the logic - either determining the minimum to allocate in one chunk, or = attempting to obtain a contiguous chunk by successive xzallocs until the re= quired size is obtained.

Lynn


--0000000000006a416105e4c9af49--