From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lynn Winebarger Newsgroups: gmane.emacs.devel Subject: Re: Regression in dump-emacs-portable Date: Thu, 16 Feb 2023 04:31:10 -0500 Message-ID: References: <83ttzocomk.fsf@gnu.org> <834jrncd6a.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="00000000000022d86805f4cddcf0" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1507"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Feb 16 10:32:03 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pSacU-0000Ge-P9 for ged-emacs-devel@m.gmane-mx.org; Thu, 16 Feb 2023 10:32:02 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pSabw-0001lL-Es; Thu, 16 Feb 2023 04:31:28 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pSabv-0001kB-8t for emacs-devel@gnu.org; Thu, 16 Feb 2023 04:31:27 -0500 Original-Received: from mail-pj1-x102d.google.com ([2607:f8b0:4864:20::102d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pSabt-00006D-3M; Thu, 16 Feb 2023 04:31:27 -0500 Original-Received: by mail-pj1-x102d.google.com with SMTP id oa11-20020a17090b1bcb00b002341a2656e5so1568398pjb.1; Thu, 16 Feb 2023 01:31:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=QYG71x6kjt88ruihntHvVQh6TinQ3eWvk8qBUdpt72Q=; b=bHw1yTdl9Eiwg48yWLdwZr2231/Sn/FyoBqVyVm7j8MK9jm+bfLc62PNK/Uw5jzMY8 WHKAhsPiZ3dit5HnT6aoim7c5apSKba72Y1dSU1wMX76cH+0NAz9yBACEKUHQz70P3vH prHmKatqYrsLVHjZeDdpKkL6vCS80mTn8BF4ShnYaQIHTPZmgHClqnosYiTEsRee6W5Z 37tawjvkxnDrOGoLniFJhMRZjRFwrnBwHtyd+VbcrTe9Zd4uxomKH6CHnuBzn1FcIYYP kjtCMqylmi0wvkvJhkQiwQhOogwswHz9hzMfaTZlChGT25wZo+xRAf2+76/bf50pRXME 1aDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=QYG71x6kjt88ruihntHvVQh6TinQ3eWvk8qBUdpt72Q=; b=gDrZP7T9RB79J2JTPGcNqsXn67435C971KWesRHwtsdgS3p+E5xQSfY4/dX+Xf7OnO cG4kfNxBgLhizWQfvmWssG5H3c+HodDLxGtiAGM7P5wYpUviWRDK7CbqNBU0+gnLVWgl daoL/tQeMS22RUmQKQ77cJleekWbYimGb7iDh67Q667HviYZbUdv1+R8gdIFT2nA7x6n Ao/EsnnmGhWkGTNc61+539UxB5d/p8qVqKy4Nyt6CWu+SyXq0pTf36H4RY0cRWUjcBTa ObsKZ9IVy8KNxXXRRPjzu8mSRyKVkQ9SMc/6wRVtJKZkBknGi5hdXkgFWLcnKofIh6UL WTng== X-Gm-Message-State: AO0yUKXsGf/Nfdgjg7xO+WC8U/ik3ddxqpS46I9t7PNEaBjVACQQ+5nB psef26hWllH31WCOoSkEOQg43R/idFDUARFJfnQmnY9LxcY= X-Google-Smtp-Source: AK7set+kD2M6a+0fQGoyfOb/LwHEIvn/8WYW8413PUkUEzZrKwV9wOUFNhlUNCb3EAFKifyWFr/zRG0LZX4isA/2HuI= X-Received: by 2002:a17:90b:2784:b0:232:ce4d:8da8 with SMTP id pw4-20020a17090b278400b00232ce4d8da8mr327922pjb.143.1676539882295; Thu, 16 Feb 2023 01:31:22 -0800 (PST) In-Reply-To: <834jrncd6a.fsf@gnu.org> Received-SPF: pass client-ip=2607:f8b0:4864:20::102d; envelope-from=owinebar@gmail.com; helo=mail-pj1-x102d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:303389 Archived-At: --00000000000022d86805f4cddcf0 Content-Type: text/plain; charset="UTF-8" On Wed, Feb 15, 2023, 7:43 AM Eli Zaretskii wrote: > > From: Lynn Winebarger > > Date: Tue, 14 Feb 2023 18:26:07 -0500 > > Cc: emacs-devel@gnu.org > > > > On Tue, Feb 14, 2023 at 9:23 AM Eli Zaretskii wrote: > > > > > > What do these tests actually test? > > > > Whether libraries expected to be redumpable are in fact redumpable. > > That's the goal, not the actual testing algorithm. I asked about the > latter. How do you intend to test that a dump succeeded (assuming > there's no crash)? > The dump will have to be performed in a separate emacs process. The easiest criteria to judge is whether the dump file exists and is greater than 0 bytes. Emacs appears to create a 0 byte file when dump-emacs-portable is invoked, which is just not updated if the dump terminates unsuccessfully. A second criteria is then to invoke emacs with the dump-file and evaluate some simple expression to verify no unexpected errors were encountered on load. I've started automating my process with some simple shell scripting tracked at https://github.com/owinebar/emacs-redumping. It's not much yet, but at least I was able to align my efforts between 28.2 and 30.0.50. The next step will be to create a proper load-time dependency graph, so I can automate the calculation of the minimal list of features that need to be provided so that the maximum number of libraries can be loaded for the dump, with the artificially provided features loaded on an after-init hook (because before-init happens prior to the X frame initialization). Once these dependencies are identified and lists are calculated, then creating a set of canned tests should be straightforward. Some makefile-based approach should be adequate for determining which parts of the dependency graph need to be recalculated after an update. I want to calculate these dependencies (and compile-time dependencies) to construct a more robust native-compilation build process anyway. For a regression test, I would want to record the results from 28.2 as a basis for measuring 29 and 30, at least as a starting point. In any case, I never see an "abort signal" termination in 28, or even a "weird pseudovector" message. It's either something incompatible (because I blindly attempted to load the world) as in the "term" subdirectory or dos/w32 libraries under linux, or some redefinition of a character table (which is why I calculate the files loaded in the baseline dump and exclude them). I got some very lengthy error messages printing out explicit objects from some obsolete libraries, so I exclude them as well. And viper demands user input at startup when it's loaded, so it has to be excluded from dumping. There might be some variable to turn off that annoying behavior, I'm just not interested in investigating. > Almost every library in 28.2 could be redumped, excepting those which > > simply failed to load for whatever reason. > > Don't we have Lisp objects that cannot be dumped? If we do, then not > every library could be dumped even in principle. > In 28.2, using dump-emacs-portable, the answer is, not many in the libraries in included in the Emacs source distribution. I excluded the term and obsolete subdirectories from generating the set of libraries to dump (but not from the final set determined from load-history). Even outside of the emacs distribution, the only problematic objects are dynamic modules. I assume this is due to dumping in batch-mode. My exclusion on wid-edit.el is because dumping it in batch-mode appears to bar it from ever subsequently creating proper buttons in a graphic terminal. But dumping it still succeeds. Another potential issue with this is (assuming you suggest to actually > try dumping every library) that it will take too long, and thus will > be likely to be skipped in any "normal" run of the test suite, thus > missing the point. > My 2017-vintage laptop dumps the 1252 files, including all of leim, in 34 seconds, for a 135MB dump file. When I added leim to the exclusions list, 1172 libraries are dumped in 24 seconds for a 83MB dump file, which explains why my effort with 30.0.50 produces a 75MB dump. I excluded leim for 30.0.50 because I was encountering too many errors to deal with manually, which explains most of the size reduction. I'm not sure how the tests are normally run, but I would think anyone working on pdumper should be interested in a comprehensive test at some point. Aside from testing on a per-commit basis, isn't there a more comprehensive set of regression tests run pre-release? Does emacs have a CI process regularly running the test suite, or is it more ad hoc? If nothing else, failures reported from such a routine run could be used to create a more targeted test set for someone actively working on pdumper. Lynn --00000000000022d86805f4cddcf0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Wed, Feb 15, 2023, 7:43 AM Eli Zaretskii &= lt;eliz@gnu.org> w= rote:
> From: Lynn Winebarger &l= t;owinebar@gmail.com>
> Date: Tue, 14 Feb 2023 18:26:07 -0500
> Cc: emacs-devel@gnu.org
>
> On Tue, Feb 14, 2023 at 9:23 AM Eli Zaretskii <eliz@gnu.org> wrote= :
> >
> > What do these tests actually test?
>
> Whether libraries expected to be redumpable are in fact redumpable.
That's the goal, not the actual testing algorithm.=C2=A0 I asked about = the
latter.=C2=A0 How do you intend to test that a dump succeeded (assuming
there's no crash)?

The dump will have to be performed in a separate emac= s process. The easiest criteria to judge is whether the dump file exists an= d is greater than 0 bytes.=C2=A0 Emacs appears to create a 0 byte file when= dump-emacs-portable is invoked, which is just not updated if the dump term= inates unsuccessfully.
A second criteria is then to = invoke emacs with the dump-file and evaluate some simple expression to veri= fy no unexpected=C2=A0 errors were encountered on load.=C2=A0=C2=A0
I've started automating my process with some simple shell scripting = tracked at=C2=A0htt= ps://github.com/owinebar/emacs-redumping.=C2=A0 It's not much yet, = but at least I was able to align my efforts between 28.2 and 30.0.50.=C2=A0= The next step will be to create a proper load-time dependency graph, so I = can automate the calculation of the minimal list of features that need to b= e provided so that the maximum number of libraries can be loaded for the du= mp, with the artificially provided features loaded on an after-init hook (b= ecause before-init happens prior to the X frame initialization).=C2=A0=C2= =A0
Once these dependencies are identified and lists are calculat= ed, then creating a set of canned tests should be straightforward.=C2=A0 So= me makefile-based approach should be adequate for determining which parts o= f the dependency graph need to be recalculated after an update.
I= want to calculate these dependencies (and compile-time dependencies) to co= nstruct a more robust native-compilation build process anyway.
Fo= r a regression test, I would want to record the results from 28.2 as a basi= s for measuring 29 and 30, at least as a starting point.=C2=A0 =C2=A0In any= case, I never see an "abort signal" termination in 28, or even a= "weird pseudovector" message.=C2=A0 It's either something in= compatible (because I blindly attempted to load the world) as in the "= term" subdirectory or dos/w32 libraries under linux, or some redefinit= ion of a character table (which is why I calculate the files loaded in the = baseline dump and exclude them).=C2=A0 I got some very lengthy error messag= es printing out explicit objects from some obsolete libraries, so I exclude= them as well.
And viper demands user input at startup when it= 9;s loaded, so it has to be excluded from dumping.=C2=A0 There might be som= e variable to turn off that annoying behavior, I'm just not interested = in investigating.

> Almost every library in 28.2 co= uld be redumped, excepting those which
> simply failed to load for whatever reason.

Don't we have Lisp objects that cannot be dumped?=C2=A0 If we do, then = not
every library could be dumped even in principle.

In 28.2, using dump-emacs-p= ortable, the answer is, not many in the libraries in included in the Emacs = source distribution.=C2=A0 I excluded the term and obsolete subdirectories = from generating the set of libraries to dump (but not from the final set de= termined from load-history).=C2=A0 Even outside of the emacs distribution, = the only problematic objects are dynamic modules.=C2=A0 I assume this is du= e to dumping in batch-mode.=C2=A0 My exclusion on wid-edit.el is because du= mping it in batch-mode appears to bar it from ever subsequently creating pr= oper buttons in a graphic terminal.=C2=A0 =C2=A0But dumping it still succee= ds.

Another potential issue with this is (assuming you suggest to actually
try dumping every library) that it will take too long, and thus will
be likely to be skipped in any "normal" run of the test suite, th= us
missing the point.
My 2017-vi= ntage laptop dumps the 1252 files, including all of leim, in 34 seconds, fo= r a 135MB dump file.
When I added leim to the exclusions list,=C2= =A0 1172 libraries are dumped in 24 seconds for a 83MB dump file, which exp= lains why my effort with 30.0.50 produces a 75MB dump.=C2=A0 I excluded lei= m for 30.0.50 because I was encountering too many errors to deal with manua= lly, which explains most of the size reduction.
I'm not sure = how the tests are normally run, but I would think anyone working on pdumper= should be interested in a comprehensive test at some point.=C2=A0 Aside fr= om testing on a per-commit basis, isn't there a more comprehensive set = of regression tests run pre-release? Does emacs have a CI process regularly= running the test suite, or is it more ad hoc?=C2=A0 If nothing else, failu= res reported from such a routine run could be used to create a more targete= d test set for someone actively working on pdumper.

Lynn


--00000000000022d86805f4cddcf0--