From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lynn Winebarger Newsgroups: gmane.emacs.devel Subject: Re: Docstring hack Date: Tue, 2 Aug 2022 12:55:32 -0400 Message-ID: References: <83bkt6687a.fsf@gnu.org> <871qu2lo29.fsf@yahoo.com> <837d3u63ez.fsf@gnu.org> <834jyy61w6.fsf@gnu.org> <831qu25z5x.fsf@gnu.org> <87mtcqhvot.fsf@yahoo.com> <83sfmh4x0o.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="22714"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , Po Lu , emacs-devel To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Aug 02 18:56:33 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oIvC4-0005gc-0h for ged-emacs-devel@m.gmane-mx.org; Tue, 02 Aug 2022 18:56:32 +0200 Original-Received: from localhost ([::1]:36802 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oIvC2-0004GO-Qb for ged-emacs-devel@m.gmane-mx.org; Tue, 02 Aug 2022 12:56:30 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:43330) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oIvBM-0003UE-Mx for emacs-devel@gnu.org; Tue, 02 Aug 2022 12:55:48 -0400 Original-Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]:36484) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oIvBL-0007Hm-1Z; Tue, 02 Aug 2022 12:55:48 -0400 Original-Received: by mail-pg1-x533.google.com with SMTP id s206so12838245pgs.3; Tue, 02 Aug 2022 09:55:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=e62b/G9du+GdhpOH6nv5L/Ur46MuShZSBrfPEWatbco=; b=Ceb0aJn/fd9bA/zUTbLj+uTPQmjiRdeAQGgzbwwvqmPD0iLECeZbwfyPJthLi2Ihe2 ixtPDuen52dp1am3GfKrSkgagsm84iN9DoJfHEuhRQYYbe4UuM6a/oXrxCmy8sJNA84p QLiKYCg/Z7KiDWQVpidxHQJ4aq2AebP8or7XsCsO335hK+DdFyoBh+Kcur0B4wb3uDz8 R0ayxY6Uuwf0qM+VXvYv8gWmd7mhFAPOr3zlDMVuseCpQiBVYgJg4Gk/X3vKKcOnftTx K19uGVUF1G68bOdo8CV9H8L7+QCqq71W+bm1wNk106iaQlL+WYkJSdvIYubOgec0gp6B 7bMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=e62b/G9du+GdhpOH6nv5L/Ur46MuShZSBrfPEWatbco=; b=Of/SJUR4TRK8gbfzm/UWEg40sXyW1C7b5MNrUBEZs6YdYOg4lcjra7FwPzotepWz5u ghz07wO9H3E5+HqLUZHLVB1qgVVMdZNgOdY1t24uKJ+FYHqV0k8LamSn5QASqjGIjER1 909SffY6Xs6cPqAfe0RphdkZKtwP/pKJHbj6T9g6pURRF9uwVqTpQQmgyl/yP6jndRoS pfyN4NB6LIE0/BbS+3izyGqzc0loPM/F0Py8PJTd14WPMZlFnmyVOHJr3WobW9U4Csfo X0UhQo0qHuDwcmJcGsaAQbhjUqA3HHbmH5WVLv4k6B7Gif262jgYatoBxcAcLWrGUpLL U0GA== X-Gm-Message-State: AJIora8AuqBetAjB/TvYZla1Ot3CVjXeC4Qf1sD0fyq9M9QaCQmERf6x ZJQykMjKiYJZSzk3S4CJpp0BWEY+RJa6fetnYUcICjaL X-Google-Smtp-Source: AGRyM1ts2PAts5lggiJEMLTaEBmhkbtoPbfmetaK0u459qkSexSrQaPE0kygUqvWr4JboEXkdKEUvvOR5RrQ3TbWYJ4= X-Received: by 2002:a63:2bc1:0:b0:412:706e:73ad with SMTP id r184-20020a632bc1000000b00412706e73admr17348535pgr.488.1659459345283; Tue, 02 Aug 2022 09:55:45 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::533; envelope-from=owinebar@gmail.com; helo=mail-pg1-x533.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:292994 Archived-At: On Sun, Jul 31, 2022 at 5:32 PM Stefan Monnier wrote: > > Loading `site-load.el` in the first dump is a bad idea because files > haven't been compiled yet. > > Loading `site-load.el` in the second dump is a bad idea because: > - as currently written, the site-loaded files aren't compiled, so it's > too early to dump them. > - if you change the build to byte-compile them before the second dump, > you'll be byte-compiling them with the bootstrap-emacs which might > work but will lead to a slower compilation. > > So, I suggest something like: > > mv lisp/site-load.el lisp/my-site-load.el > make > rm src/emacs > mv lisp/my-site-load.el lisp/site-load.el > mv lisp/my-site-load.elc lisp/site-load.elc > make > > So the first 2 dumps are "normal" without any site-loaded files, and > that's followed by a 3rd dump, where all the ELisp files are already > byte-compiled. I have now successfully dumped (byte-compiled only) an additional 511 core emacs files by calling a shell script during the current second-stage dump to make the actual second stage dump and build the required elc files, as well as finder-inf and cus-load. The bad news is, when I just turn off Vpurify_flag at points in the site-load to avoid a segfault or other problem (and then re-enable it after the problem libraries), site-load will finish without error, but the actual dumping process will fail for some reason I haven't debugged. The good news is that I was able to resolve the remaining problems with a handful of changes to the C code in alloc and lread, and changes to a couple of elisp files. * alloc - put in a hash table of objects that have been purified during a call to Fpurecopy, so cycles are not followed. * Also changed the "small_amount" in pure_alloc to 20000 and printed a message on every allocation going over, since I can't rely on the process actually finishing if many megabytes of additional pure space are required. * lread - put in a docstring-hack flag used as an extra and conditional for the hack, so I could turn that off in site-load without changing any variables used by the existing code * easy-mmode - easy-mmode-defmap produces a defconst for a keymap variable, which I changed to defvar * tramp-sh - the defconsts for the tramp-completion-function-alist-* variables changed to defvar, since for some reason the values are getting modified, probably during tramp-startup-hook I am going to try native-compilation again. I can't see why I was getting an "incoherent" library message, now that I understand the loadup code that sets the compilation unit file name before the dump. So I've made the table of compilation units registered by the load_compilation_unit visible in lisp. That way I can compare the method used by loadup, which depends on subrs in compilation units being accessible through function symbol values, in the site-load file directly. Or just do the conversion directly based on that table, which should guarantee any NCU encountered by pdumper has the file field set "coherently". Given the elimination of pure space will not be back ported to 27 and 28, and the problem with pdump when the purify flag is turned off while loading files, I think some of these changes (or similar) should be included maintenance releases of those two major versions, so there will be some way for users stuck with those versions to effectively dump significant chunks of core emacs beyond what's in loadup. Lynn