From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lynn Winebarger Newsgroups: gmane.emacs.devel Subject: Re: Docstring hack Date: Sun, 31 Jul 2022 08:43:09 -0400 Message-ID: References: <83bkt6687a.fsf@gnu.org> <871qu2lo29.fsf@yahoo.com> <837d3u63ez.fsf@gnu.org> <834jyy61w6.fsf@gnu.org> <831qu25z5x.fsf@gnu.org> <87mtcqhvot.fsf@yahoo.com> <83sfmh4x0o.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000007fc6f105e5193a37" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="7767"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , Po Lu , emacs-devel To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Sun Jul 31 14:45:00 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oI8JX-0001nw-Mq for ged-emacs-devel@m.gmane-mx.org; Sun, 31 Jul 2022 14:44:59 +0200 Original-Received: from localhost ([::1]:44210 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oI8JW-0007bi-M0 for ged-emacs-devel@m.gmane-mx.org; Sun, 31 Jul 2022 08:44:58 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:34948) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oI8I1-0005uo-MV for emacs-devel@gnu.org; Sun, 31 Jul 2022 08:43:26 -0400 Original-Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]:55102) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oI8Hz-0001W5-Ny; Sun, 31 Jul 2022 08:43:25 -0400 Original-Received: by mail-pj1-x102f.google.com with SMTP id y1so8359644pja.4; Sun, 31 Jul 2022 05:43:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qW910LbRfPb0hkB81D28yIb53W65c0jT/Rs0iX/HLlI=; b=Rw0pzK0FUgej5yvH6no8mp2xkbx0r1325jkDHm69lkgSzvpvpwJBiRiDL/yOWMCwCi iUw2fYfNPTmlAVcdsH3O+Y4HE8xJoq/hykfTfWj2IjodC1tgWT4gIgg+Sv55fcUNQFgy MdtQbNIJc5kdjUVo5xXzV2zsmv0PY22xcruydf1/30hLC+qE2XzDZUGxb/w8YsMMNMMN BPUk0iMz8PepZRBZI9JGTm7a0EsHoekWsUtXrByioCQ6pYes8ZaHEt1lSTuSrXVOf9GE sRu5H7sZr6hvoOlnavsmoTwD3pCDk0Kuf+N2kEwSLjkYKMDTE5nE9Nncsef8lS7kBCOx d6cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qW910LbRfPb0hkB81D28yIb53W65c0jT/Rs0iX/HLlI=; b=6c+xQ8k7xgjRfrvXHfSqa7XE1S6e8LUfb5rdKa7GLGhSzZl5tuBTR55NlSbcLPT5jp LuDhOQLqRK3HVCwEJWvqC6ns6+SCGWcDGSZSiuh6xdBisNXLpa8kj/66LNllw3N6ObYj rupMQevIWnHxSRjEi76efWYfsnHPuAhPmU7l+BWeGOML6woNZlrUuiz8Nv590dm0EMvO 0vLcxH5MqgjL6fUvrYqjS98nI2mw6nwjHTLJAMrtuYc45RyECEOF9eRf1XVRZHUMoaCi Lj5+JXDq1nQGpRYcbAewa9DP4xJDwP60biUhHC+wHlU1s+qKKeVGskHn9qTDoA8oFxUw 27og== X-Gm-Message-State: ACgBeo0XqyWS/bUm9TJi1hYzoUeRvy32m/FaM4qi4QJA1/pU8Hrf2u8F BQlI36FVnSR71sL932m5CUZKLTxySzweowJ7oZI= X-Google-Smtp-Source: AA6agR5cSCx86jyDvkmRequ5od93AdlGrhYpVFMOq1plYnLjkMPOtPFfkCLJZQ0FiruS8rBz3Ds5JB30hNo68709YIk= X-Received: by 2002:a17:90a:2948:b0:1f3:1b42:a810 with SMTP id x8-20020a17090a294800b001f31b42a810mr14316946pjf.203.1659271401947; Sun, 31 Jul 2022 05:43:21 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=owinebar@gmail.com; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:292914 Archived-At: --0000000000007fc6f105e5193a37 Content-Type: text/plain; charset="UTF-8" On Sun, Jul 31, 2022, 4:03 AM Stefan Monnier wrote: > > OK, but I still lack some glue to understand the issue. Specifically: > > > > . the OP said "strings that are erroneously treated as docstrings in > > dump mode" -- where's the code which makes that mistake, and how > > is read_literal_string related to that mistake? > > . why isn't there an alternative to fix read_literal_string not to > > generate zero instead of the format template? the other > alternatives all look like partial kludges to me The first 4 yes, but the fifth one was to remove that and just modify the evaluator to not set those properties during dump mode. But if the only need for this feature is in loading loaddefs.el in dump mode, then that is overkill. I don't understand why the docstrings are even being extracted for the byte compiled files at all, since they would be lazy loaded anyway. Then you could remove the files from lisp.mk from the dependencies of DOC in the Makefile and just leave loaddefs and the docstrings from C source files. In `read_literal_string` there is a hack that dates back to Emacs's > early life where we drop the string we just read and return the > 0 literal instead. We do that for those strings we think are docstrings > that will be found in etc/DOC and will be re-provided later when we call > `Snarf-documentation` (which should then replace those 0 literals with > appropriate integers pointing into etc/DOC). > > The reason for this hack is to avoid allocating the string in the heap > (or worse, the purespace) since it's to be found lazily in > etc/DOC instead. > > But we don't have a sure-fire way to recognize those strings, so we use > a convention that they start with "double-quote backslash newline" (this > same convention is then used in `make-docfile` in order to find those > strings). But some non-preloaded files also use "double-quote backslash > newline" for other reasons, such as in `eieio-defclass-autoload`. > They are everywhere in fact, not just as arguments to format. > Not sure why it's a problem for Lynn, tho: he should not try to preload > `eieio-core.el` but only `eieio-core.elc` where the problem should not > appear any more. This is the conundrum of trying to do anything significant in site-load without Makefile support. If you're bootstrapping, then none of those files are compiled. So I put in a check to only load after the bootstrap during the dump. But nothing is byte compiled at this point other than the files in loadup. Trying to do the byte compile from within site-load after the bootstrap but during the dump requires pre loading the source of all dependencies, because require and autoload hit the panic button in dump mode. I finally just wrote a shell script yesterday that I could invoke from site-load to add the compiled versions of the libraries loaded in site-load to lisp.mk (so they will get added to the list of preloaded libraries in the final dump), then invoke make on them. This can cause some issues since a significant number of them require loaddefs to be loaded. I might be better off just adding the extra libraries directly to the environment variable the compiler references (to determine preloaded libraries). I don't know if that will make a difference in determining coherence for dumping during native-compilation. In any case, when the shell script calls make, it needs to arrange for the byte compiler to load loaddefs. Lynn --0000000000007fc6f105e5193a37 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sun, Jul 31, 2022, 4:03 AM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
<= /div>
> OK, but I still lack some glue to = understand the issue.=C2=A0 Specifically:
>
>=C2=A0 =C2=A0. the OP said "strings that are erroneously treated a= s docstrings in
>=C2=A0 =C2=A0 =C2=A0dump mode" -- where's the code which makes= that mistake, and how
>=C2=A0 =C2=A0 =C2=A0is read_literal_string related to that mistake?
>=C2=A0 =C2=A0. why isn't there an alternative to fix read_literal_s= tring not to
>=C2=A0 =C2=A0 =C2=A0generate zero instead of the format template? the o= ther
>=C2=A0 =C2=A0 =C2=A0alternatives all look like partial kludges to me
The first 4 yes, but the fifth one = was to remove that and just modify the evaluator to not set those propertie= s during dump mode.=C2=A0=C2=A0
But if the only need= for this feature is in loading loaddefs.el in dump mode, then that is over= kill.=C2=A0 I don't understand why the docstrings are even being extrac= ted for the byte compiled files at all, since they would be lazy loaded any= way.=C2=A0 Then you could remove the files from = lisp.mk from the dependencies of DOC in the Makefile and just leave loa= ddefs and the docstrings from C source files.=C2=A0
=
In `read_literal_string` there is a hack that dates back to Emac= s's
early life where we drop the string we just read and return the
0 literal instead.=C2=A0 We do that for those strings we think are docstrin= gs
that will be found in etc/DOC and will be re-provided later when we call `Snarf-documentation` (which should then replace those 0 literals with
appropriate integers pointing into etc/DOC).

The reason for this hack is to avoid allocating the string in the heap
(or worse, the purespace) since it's to be found lazily in
etc/DOC instead.

But we don't have a sure-fire way to recognize those strings, so we use=
a convention that they start with "double-quote backslash newline"= ; (this
same convention is then used in `make-docfile` in order to find those
strings).=C2=A0 But some non-preloaded files also use "double-quote ba= ckslash
newline" for other reasons, such as in `eieio-defclass-autoload`.
<= /blockquote>

They = are everywhere in fact, not just as arguments to format.


Not sure why it's a problem for Lynn, tho: he should not try to preload=
`eieio-core.el` but only `eieio-core.elc` where the problem should not
appear any more.

This is the conundrum of trying to do a= nything significant in site-load without Makefile support.=C2=A0 If you'= ;re bootstrapping, then none of those files are compiled.=C2=A0 So I put in= a check to only load after the bootstrap during the dump. But nothing is b= yte compiled at this point other than the files in loadup.=C2=A0 Trying to = do the byte compile from within site-load after the bootstrap but during th= e dump requires pre loading the source of all dependencies, because require= and autoload hit the panic button in dump mode.=C2=A0=C2=A0

<= div dir=3D"auto">
--0000000000007fc6f105e5193a37--