From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lynn Winebarger Newsgroups: gmane.emacs.devel Subject: Re: emacs empty startup time increased from v 27.1 to 29.0 by factor 3? Date: Sat, 28 Jan 2023 04:00:53 -0500 Message-ID: References: <873581tkc0.fsf@no.workgroup> <83pmb5i90e.fsf@gnu.org> <87a624zw0e.fsf@no.workgroup> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000d63e0f05f34f38a5" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="38414"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Emacs developers , Andrea Corallo To: John Yates Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sat Jan 28 10:01:53 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pLh5t-0009ri-IM for ged-emacs-devel@m.gmane-mx.org; Sat, 28 Jan 2023 10:01:53 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pLh5D-0000ot-LM; Sat, 28 Jan 2023 04:01:11 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pLh5B-0000oT-QH for emacs-devel@gnu.org; Sat, 28 Jan 2023 04:01:09 -0500 Original-Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pLh59-0007Mv-1V for emacs-devel@gnu.org; Sat, 28 Jan 2023 04:01:09 -0500 Original-Received: by mail-pj1-x1036.google.com with SMTP id z1-20020a17090a66c100b00226f05b9595so6871219pjl.0 for ; Sat, 28 Jan 2023 01:01:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=lgStK7Jdh+N6Ry5ctoA778zymCNvIkj13efz/5Zuht0=; b=N9P2WNPVdhuUcuK+/qyPMg60hUXB5bxeZM+w+Dm/5/VN7kE3s6S5LpJ6b4lKxk5jrj lnRfuFvXMTvoCVaM6Ly2TkeViPOp5mxRAhqje+K3HZyDKsPJiAAUh7fdc0v3RSbCqH9g uVLSwkB8AyY0R6IqTeng5TwjIVcim8UPxO+0IxE8b6bAHl1PyQju3ylLn2kt7/2Go0W3 GlCvKJ63GTTr7na4rEJ6sheiUYicfb3U/FUCdAgJUKMCcnni9QB50cjFr7AXXgZK0aXE gEQ/pr61qzQ92zBQh0SkXToIKKu6T6d8TYwEkjcujp/pQhXvmNzVThVEG6KS6NUXXOVP d/aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=lgStK7Jdh+N6Ry5ctoA778zymCNvIkj13efz/5Zuht0=; b=rqhlJN8wlomI/UsF7ND0wBOzLdC23wjubzqx0NZn4WsTNCYrdMEuh+IaNHjCeYks0C R/3v7scDE6N18nX3uXooZojo5n8azZixtiQkAO9mFNV5f2lQ63D2BciSvmnb/g7kiyK0 N0ize2iJJlvY+v7rir/vkX1K0frqfp/Imea3mIphKGIctKyVjVSKd/4UZBEVbCFYppP5 LIdq8BE+c+05xT9ZooBoBezWCn99341CJNfGOGCHC/8n4hP1jrhXNLWAfZE5zDIj/UdV 13KF2sUv2peluYxz6jyHd8B1xVJbWrzZRBFs/qk04WWeSUKLVizakEbFYm9zjCvUcWoP NlMg== X-Gm-Message-State: AFqh2kr6j/eRAmjZhMXLd7e51DmgY4565HH7wzTuELPG6Zt5vSTfq+5h 9SH67DV+t4bG5B1WMDnAFE54opUuCIIIpsXsc6o= X-Google-Smtp-Source: AMrXdXsymkklT/dm0qqpOWEQCWT++NQodfqr1UbejwpXpdVICzl1ZPKbNxfZkMemCllPLMYtD/7QcBXCLqq6V57m6dk= X-Received: by 2002:a17:90a:ba10:b0:229:1fc2:108f with SMTP id s16-20020a17090aba1000b002291fc2108fmr5392887pjr.86.1674896465082; Sat, 28 Jan 2023 01:01:05 -0800 (PST) In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=owinebar@gmail.com; helo=mail-pj1-x1036.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:302707 Archived-At: --000000000000d63e0f05f34f38a5 Content-Type: text/plain; charset="UTF-8" Hi, John, The closest thing in Emacs to what you describe is dumping, which currently isn't supported for native compilation, unless something has changed that I'm not aware of. Last summer I did an experiment and modified 28.1 to support dumping native compilation. I had a startup that was loading all libraries included with emacs and approximately an additional 1000 packages installed in site-lisp (lots of manual modifications to make that happen). Loading those packages at startup as regular packages, after ensuring all were compiled ahead of time, increased startup time to 15 minutes or more. The dumped version only took 5 seconds or so. I discarded that experiment, but have done a similar site-lisp installation with maybe half of those additional packages. The dump produced by "dump-emacs-portable" starts almost instantly. Even without dumping, startup time is only 30 seconds or so. For that matter, it takes 12 minutes to dump the byte-compiled function, where dumping the natively compiled version would take over 30 minutes. It's not a completely fair comparison since (if memory serves) the natively compiled version loaded 4000-5000 files where this version is only loading ~2200 libraries. Loading natively compiled libraries does do some additional checking to compare source file to the compiled library by checksum. Maybe that's part of the 3x you're seeing? Lynn On Fri, Jan 27, 2023, 12:22 PM John Yates wrote: > Reposting to emacs-devel at Eli's behest. > > Re my suggested parallel, Eli asked: > > > How many hundreds? Emacs 29 loads almost 300 *.eln files at startup. > > I have never needed to answer that question. > > I work on the code generation middle-end. When I modify one of its > fundamental headers over 350 modules get rebuilt. (That number > reflects poor dependency management, which is being addressed.) > There are probably just as many modules unrelated to code generation > (e.g. UI, graphics, etc). > > /john > ---------- Forwarded message --------- > From: John Yates > Date: Thu, Jan 26, 2023 at 8:16 PM > Subject: Re: emacs empty startup time increased from v 27.1 to 29.0 by > factor 3? > To: > > > I work on a product whose structure involves literally hundreds of > DLLs / SOs. We tolerate slow start-up in-house because it optimizes > the compile / edit / debug cycle. > > Packaging the final product involves an extra step to collapse our > hundreds of images into just a handful via a secondary link step. > I wonder if there is anything analogous that Emacs could do. > > --000000000000d63e0f05f34f38a5 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi, John,
The closest thing in Emacs to = what you describe is dumping, which currently isn't supported for nativ= e compilation, unless something has changed that I'm not aware of.
Last summer I did an experiment and modified 28.1 to sup= port dumping native compilation.=C2=A0 I had a startup that was loading all= libraries included with emacs and approximately an additional 1000 package= s installed in site-lisp (lots of manual modifications to make that happen)= .
Loading those packages at startup as regular packa= ges, after ensuring all were compiled ahead of time, increased startup time= to 15 minutes or more. The dumped version only took 5 seconds or so.
=
I discarded that experiment, but have done a similar site= -lisp installation with maybe half of those additional packages.=C2=A0 The = dump produced by "dump-emacs-portable" starts almost instantly.= =C2=A0 Even without dumping, startup time is only 30 seconds or so.=C2=A0 F= or that matter, it takes 12 minutes to dump the byte-compiled function, whe= re dumping the natively compiled version would take over 30 minutes.=C2=A0 = It's not a completely fair comparison since (if memory serves) the nati= vely compiled version loaded 4000-5000 files where this version is only loa= ding ~2200 libraries.

Lo= ading natively compiled libraries does do some additional checking to compa= re source file to the compiled library by checksum.=C2=A0 Maybe that's = part of the 3x you're seeing?

Lynn


On Fri, Jan 27, 2023, 12:22= PM John Yates <john@yates-shee= ts.org> wrote:
Reposting to emacs-devel at Eli's behest.

R= e my suggested parallel, Eli asked:

>=C2=A0How many hundreds?=C2=A0 Ema= cs 29 loads almost 300 *.eln files at startup.

I have never needed to answer t= hat=C2=A0question.

I work on the code generation middle-end.= =C2=A0 When I modify one of its
fundamental headers over 350 modules get= rebuilt. =C2=A0(That number
reflects poor dependency management, which = is being addressed.)
There are probably just as many modules unrelated t= o code generation
(e.g. UI, graphics, etc).

/jo= hn
---------- Forwarded message ---------
From: John Yates <j= ohn@yates-sheets.org>
Date: Thu, Jan 26, 2023 at 8:16 PM
Subje= ct: Re: emacs empty startup time increased from v 27.1 to 29.0 by factor 3?=
To: <help-gnu-emacs@gnu.org>


I work on a produ= ct whose structure involves literally hundreds of
DLLs / SOs.=C2=A0 We t= olerate slow start-up in-house because it optimizes
the compile / edit /= debug cycle.

Packaging the final product involves an extra step to = collapse our
hundreds of images into just a handful via a secondary link= step.
I wonder if there is anything analogous that Emacs could do.
<= br>
--000000000000d63e0f05f34f38a5--