From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lynn Winebarger Newsgroups: gmane.emacs.bugs Subject: bug#55972: 28.1; Package quickstart generated for large number of packages generates byte-code string larger than 64K, triggering bytecode overflow error Date: Fri, 17 Jun 2022 16:06:01 -0400 Message-ID: References: <1058D1B4-9A9F-41D9-BE59-55BFA2A69A10@acm.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000527e4705e1aa4946" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12806"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 55972@debbugs.gnu.org, Stefan Monnier To: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Fri Jun 17 22:07:28 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o2IFc-00037K-IE for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 17 Jun 2022 22:07:28 +0200 Original-Received: from localhost ([::1]:43956 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o2IFb-0003pg-4d for geb-bug-gnu-emacs@m.gmane-mx.org; Fri, 17 Jun 2022 16:07:27 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:56840) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o2IFC-0003pT-CU for bug-gnu-emacs@gnu.org; Fri, 17 Jun 2022 16:07:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:53132) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o2IFC-00023z-4d for bug-gnu-emacs@gnu.org; Fri, 17 Jun 2022 16:07:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1o2IFB-00083s-TC for bug-gnu-emacs@gnu.org; Fri, 17 Jun 2022 16:07:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Lynn Winebarger Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 17 Jun 2022 20:07:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 55972 X-GNU-PR-Package: emacs Original-Received: via spool by 55972-submit@debbugs.gnu.org id=B55972.165549638230940 (code B ref 55972); Fri, 17 Jun 2022 20:07:01 +0000 Original-Received: (at 55972) by debbugs.gnu.org; 17 Jun 2022 20:06:22 +0000 Original-Received: from localhost ([127.0.0.1]:47029 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o2IEY-00082x-7m for submit@debbugs.gnu.org; Fri, 17 Jun 2022 16:06:22 -0400 Original-Received: from mail-vk1-f175.google.com ([209.85.221.175]:39773) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1o2IEW-00082l-0v for 55972@debbugs.gnu.org; Fri, 17 Jun 2022 16:06:20 -0400 Original-Received: by mail-vk1-f175.google.com with SMTP id p83so2455995vkf.6 for <55972@debbugs.gnu.org>; Fri, 17 Jun 2022 13:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hvqzGd+Jm00hqAAaE+b3GatCCQCSFZTCroIcUtV62ag=; b=jvChPmizpUfwvigyHzuM/vqhGE1U8fGUrwnMAvlQd7sZ6dJyuaN2Mro0UJT6Pxn77F 8kBPyLYCakGP7U/mg70kcj+acwHerZMLstukrMEkfpjlqfogZdJNvDk5ft/s8eVuqz0H hOAVrrUKWFIMecnGgiegEK7Mhv8AUyUoy+N2ey4oAOJv95eEnZ54ZwzSKTp3a8udUh/Z eKp2VKUB+9T+VnzsRLwyE1doWET1MmOlZ9NK71Aixjc0KZvj7DmYvvx8jZ2V+p/xX7ae HroAtZzzKJ0fc4Wjovca4mPy/SdvHW2N6cduDqJ/j2Te688mlHv+GNMkuyvqpHGr004o WVhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hvqzGd+Jm00hqAAaE+b3GatCCQCSFZTCroIcUtV62ag=; b=yjM4bDtUDN/ylY55yISaPYTouLwIiJe/zpip3UaC+KIr17O7miiT9oH3JerUl26N8K ncUSFPzpUkTkcU7HTI3Pevt6vDbTjNzUydkDSUS3MPKmbOIUIYIDat1qWfylwG0qJVKK TEJs8eqLr1KKeWMoZjDo3iGeKDx+SIbbMimXvV4HB6ZOrwXyZhI3XoW+Gd2AzrNyQuYw MfR9HWMVe2G5XBse/aKOFVpXUvC7k3DX6UXW0k103RGXF/V6G21kP0aBUWOT3QP2kTnN Dk6NfnLsQaF/mYazmy9/35E4j66GpCgoNsiICML0xIsVVwPbO+xqxsFZJmQMg6VdyNj/ Hjfw== X-Gm-Message-State: AJIora+UQMkHN6g44FOiUwC/WSSQIJZs6fyaHGl32n1bu44PG7IVFof3 +/7dXcoE5L7fHy85s0wToLKEqqvU3tAhfHS1y18= X-Google-Smtp-Source: AGRyM1v7CG796DLOsjLEcPpuBlHuv4fLC4HMW465W1/WgoXO2Ig5xMBdnTTk1sa6QKinqjvjxp2E5Y9uhbfkaEf+nss= X-Received: by 2002:a1f:ab56:0:b0:36b:e8de:afe2 with SMTP id u83-20020a1fab56000000b0036be8deafe2mr143733vke.10.1655496374385; Fri, 17 Jun 2022 13:06:14 -0700 (PDT) In-Reply-To: <1058D1B4-9A9F-41D9-BE59-55BFA2A69A10@acm.org> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:234722 Archived-At: --000000000000527e4705e1aa4946 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Jun 14, 2022 at 12:29 PM Mattias Engdeg=C3=A5rd = wrote: > If, as seems to be the case, byte-compile-keep-pending is only used for > top-level forms, then this patch may even be correct. Does it solve your > problems? > It still seems to generate far bigger bytecode chunks than the 300 cutoff > would imply but that's perhaps just a matter of calling the function in > more places. > > Thanks, Mattias, it does work. I was going to ask about directly addressing the underlying problem by tail-calling or trampolining to a byte-code vector in the constants array, but then realized you would have to either make sure there could be no "gotos" between the segments or do a real trampoline to an explicit label. And in either case you would have to save the contents of the stack frame and reinstate them in the continuation byte-code call, and I don't see any byte-codes that would support that. Otherwise you could only do it when you know there is no stack in use, which is what I believe your solution effectively does. On the other hand, given the code for patching up byte-code in byte-compile-lapcode, you could explicitly byte-compile a thunk for every top-level expression, then glue them together until they would exceed the 65K pc limit, then do another segment, etc, and do a simple trampoline between the resulting byte-code vectors, or no trampoline if there were only one required. Strictly speaking, gluing them all together is really an optimization of creating byte-code vectors (thunks) for each top-level expression, and looping over the collection of them, invoking each one in turn. As long as I'm looking at the compile log, I also see a lot of errors of the form: package-quickstart2.el:14739:39: Warning: The compiler ignores =E2=80=98autoload=E2=80=99 except at top level. You should probably put the autoload of the macro =E2=80=98bind-map-for-minor= -mode=E2=80=99 at top-level. This message is only reported for macros - there are plenty of autoload expressions that do not generate this message despite being in the same kind of "let" form. Lynn --000000000000527e4705e1aa4946 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Tue, Jun 14, 2022 at 12:29 PM Mattias Engdeg=C3=A5rd <mattiase@acm.org> wrote:
If, as seems to be the case, b= yte-compile-keep-pending is only used for top-level forms, then this patch = may even be correct. Does it solve your problems?
It still seems to generate far bigger bytecode chunks than the 300 cutoff w= ould imply but that's perhaps just a matter of calling the function in = more places.


Thanks, Mattias, it does work.=C2=A0= =C2=A0

I was going to ask about directly addressin= g the underlying problem by tail-calling
or= trampolining to a byte-code vector in the constants array, but then realiz= ed you would have to
either make sure there could be no "got= os" between the segments or do a real trampoline to
an expli= cit label.=C2=A0 And in either case you would have to save the contents of = the stack frame=C2=A0
and reinstate them in the continuation byte= -code call,=C2=A0and I don't see any byte-codes that=C2=A0would=C2=A0
support=C2=A0that.=C2=A0 =C2=A0Otherwise you could only do it when= you know there is no stack in use, which is what I believe
your = solution effectively does.

On the other hand, give= n the code for patching up byte-code in byte-compile-lapcode, you could
explicitly byte-compile a thunk for every top-level expression, then= glue them together until they would exceed=C2=A0
the 65K pc limi= t, then do another segment, etc, and do a simple trampoline between the res= ulting byte-code vectors,
or no trampoline if there were only one= required.=C2=A0 Strictly speaking, gluing them all together is really an o= ptimization=C2=A0
of creating byte-code vectors (thunks) for each= top-level expression, and looping over the collection of them, invoking=C2= =A0
each one in turn.

As long as I'm= looking at the compile log, I also see a lot of errors of the form:
<= div>=C2=A0 =C2=A0 =C2=A0package-quickstart2.el:14739:39: Warning: The compi= ler ignores =E2=80=98autoload=E2=80=99 except at top level.=C2=A0 You shoul= d
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0probably put the autoload of the mac= ro =E2=80=98bind-map-for-minor-mode=E2=80=99 at top-level.
Th= is message is only reported for macros - there are plenty of autoload expre= ssions that do not generate this
message despite being in the sam= e kind of "let" form.

Lynn
=C2=A0
--000000000000527e4705e1aa4946--