From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Lynn Winebarger Newsgroups: gmane.emacs.devel Subject: Re: native compilation units Date: Sun, 19 Jun 2022 21:39:28 -0400 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000868da005e1d72df9" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="12314"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Andrea Corallo , emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Jun 20 03:40:36 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1o36P5-00031e-BS for ged-emacs-devel@m.gmane-mx.org; Mon, 20 Jun 2022 03:40:35 +0200 Original-Received: from localhost ([::1]:37214 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1o36P3-000569-Mr for ged-emacs-devel@m.gmane-mx.org; Sun, 19 Jun 2022 21:40:33 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59682) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1o36OH-0004R4-BO for emacs-devel@gnu.org; Sun, 19 Jun 2022 21:39:45 -0400 Original-Received: from mail-oi1-x22f.google.com ([2607:f8b0:4864:20::22f]:36472) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1o36OE-00023C-Rc for emacs-devel@gnu.org; Sun, 19 Jun 2022 21:39:45 -0400 Original-Received: by mail-oi1-x22f.google.com with SMTP id s16so2063694oie.3 for ; Sun, 19 Jun 2022 18:39:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KZyhGsP9hvXoejMob1EAIHLzm2nk0WxXJQVhkVCjyKQ=; b=qlFw2ppwc8MLNoxwvPaANfSAldKEZtZqKJEL0ekQFLfTIBFi/dn2CzEHsV2LzKESzn y+80tYMXamrPA1TkZor0msvn0nlLE39d97GMwwWCbwFFrWP8JVhinIK/rS1EIC6qqs7O QmlEyQfNLMpSqOpHr2JAEwTDm4+msqQsUduC2prSa7UWSWoqC/KBKcxHpuy2ZA4GxwMt AMLGIYK5Fvn9VGpiW6X+iRYdI8PHtMJS1u0h/k361lqXjAs6RuUPj8HaVBrnWbhv/H0O RhCzk8teTaO+v64y0TyZBOf3z63I2T++0Ol98IvUR86kVA5cyB4UY+g4AfM6vPi1cH/C GHGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KZyhGsP9hvXoejMob1EAIHLzm2nk0WxXJQVhkVCjyKQ=; b=hfephqnOae8XfWLQp72QxS+jNwNLpBjq5i+GKzxkAWcFC81wCQaWUw8MJkAeadlT37 tLRRJoztc2Bz/dTcKq0vjrLD8PNnxe05O+9fPL8YD6Me6HVItdNLrr/2IXbywICxqJ13 FV1jSkDJGqCVaSJmm5nt/BZLw1ihE3tdBk+DF0wvDJtiBBl/jwjUHUVZMLCODGasWpUw tXgVvnYuqCOb/MwNS3yVuxJ2kwQyNbiFPSk9kHmXt1LR8oImeApcmJJaEaKWmkLW+pju bjPU0b4BxZDAjLnDsp7CT9ocWG8tm6eqfJgoQnmiafgSxSEBw+5JLIY+PuhicUPmkP74 o43Q== X-Gm-Message-State: AJIora+RIXGWUmWEHzPP3yFs5LmIsK7nKC7Y8Suqqp1Ps8aaDay5xUVQ 67UpTR9I7lOS5Y4vx5eZtxSZ+bcXDbX79MXvzZI= X-Google-Smtp-Source: AGRyM1sUSk4WJQY039Kzqu3We4H5/YCHxNtR4lBQFPAalfDUvJBtNZBPb5AaZzJCECQ1NtN8UG17vtxeQP5VjwnFG0U= X-Received: by 2002:a05:6808:144f:b0:32f:56f5:7754 with SMTP id x15-20020a056808144f00b0032f56f57754mr10781940oiv.162.1655689181563; Sun, 19 Jun 2022 18:39:41 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::22f; envelope-from=owinebar@gmail.com; helo=mail-oi1-x22f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:291447 Archived-At: --000000000000868da005e1d72df9 Content-Type: text/plain; charset="UTF-8" On Sun, Jun 19, 2022 at 7:02 PM Stefan Monnier wrote: > > Currently compiling a top-level expression wrapped in > > eval-when-compile by itself leaves no residue in the compiled output, > > `eval-when-compile` has 2 effects: > > 1- Run the code within the compiler's process. > E.g. (eval-when-compile (require 'cl-lib)). > This is somewhat comparable to loading a gcc plugin during > a compilation: it affects the GCC process itself, rather than the > code it emits. > > 2- It replaces the (eval-when-compile ...) thingy with the value > returned by the evaluation of this code. So you can do (defvar > my-str (eval-when-compile (concat "foo" "bar"))) and you know that > the concatenation will be done during compilation. > > > but I would want to make the above evaluate to an object at run-time > > where the exported symbols in the obstack are immutable. > > Then it wouldn't be called `eval-when-compile` because it would do > something quite different from what `eval-when-compile` does :-) > > The informal semantics of "eval-when-compile" from the elisp info file are that This form marks BODY to be evaluated at compile time but not when the compiled program is loaded. The result of evaluation by the compiler becomes a constant which appears in the compiled program. If you load the source file, rather than compiling it, BODY is evaluated normally. I'm not sure what I have proposed that would be inconsistent with "the result of evaluation by the compiler becomes a constant which appears in the compiled program". The exact form of that appearance in the compiled program is not specified. For example, the byte-compile of (eval-when-compile (cl-labels ((f...) (g ...))) currently produces a byte-code vector in which f and g are byte-code vectors with shared structure. However, that representation is only one choice. It is inconsistent with the semantics of *symbols* as they currently stand, as I have already admitted. Even there, you could advance a model where it is not inconsistent. For example, if you view the binding of symbol to value as having two components - the binding and the cell holding the mutable value during the extent of the symbol as a global/dynamically scoped variable, then having the binding of the symbol to the final value of the cell before the dynamic extent of the variable terminates would be consistent. That's not how it's currently implemented, because there is no way to express the final compile-time environment as a value after compilation has completed with the current semantics. The part that's incompatible with current semantics of symbols is importing that symbol as an immutable symbolic reference. Not really a "variable" reference, but as a binding of a symbol to a value in the run-time namespace (or package in CL terminology, although CL did not allow any way to specify what I'm suggesting either, as far as I know). However, that would capture the semantics of ELF shared objects with the text and ro_data segments loaded into memory that is in fact immutable for a userspace program. > > byte-code (or native-code) instruction arrays. This would in turn enable > > implementing proper tail recursion as "goto with arguments". > > Proper tail recursion elimination would require changing the *normal* > function call protocol. I suspect you're thinking of a smaller-scale version of it specifically tailored to self-recursion, kind of like > what `named-let` provides. Note that such ad-hoc TCO tends to hit the same > semantic issues as the -O3 optimization of the native compiler. > E.g. in code like the following: > > (defun vc-foo-register (file) > (when (some-hint-is-true) > (load "vc-foo") > (vc-foo-register file))) > > the final call to `vc-foo-register` is in tail position but is not > a self call because loading `vc-foo` is expected to redefine > `vc-foo-register` with the real implementation. > > I'm only talking about the steps that are required to allow the compiler to produce code that implements proper tail recursion. With the abstract machine currently implemented by the byte-code VM, the "call[n]" instructions will always be needed to call out according to the C calling conventions. The call[-absolute/relative] or [goto-absolute] instructions I suggested *would be* used in the "normal" function-call protocol in place of the current funcall dispatch, at least to functions defined in lisp. This is necessary but not sufficient for proper tail recursion. To actually get proper tail recursion requires the compiler to use the instructions for implementing the appropriate function call protocol, especially if "goto-absolute" is the instruction provided for changing the PC register. Other instructions would have to be issued to manage the stack frame explicitly if that were the route taken. Or, a more CISCish call-absolute type of instruction could be used that would perform that stack frame management implicitly. EIther way, it's the compiler that has to determine whether a return instruction following a control transfer can be safely eliminated or not. If the "goto-absolute" instruction were used, the compiler would have to decide whether the address following the "goto-absolute" should be pushed in a new frame, or if it can be "pre-emptively garbage collected" at compile time because it's a tail call. > > I'm not familiar with emacs's profiling facilities. Is it possible to > > tell how much of the allocated space/time spent in gc is due to the > > constant vectors of lexical closures? In particular, how much of the > > constant vectors are copied elements independent of the lexical > > environment? That would provide some measure of any gc-related > > benefit that *might* be gained from using an explicit environment > > register for closures, instead of embedding it in the > > byte-code vector. > > No, I can't think of any profiling tool we currently have that can help > with that, sorry :-( > > Note that when support for native closures is added to the native > compiler, it will hopefully not be using this clunky representation > where capture vars are mixed in with the vector of constants, so that > might be a more promising direction (may be able to skip the step where > we need to change the bytecode). > > The trick is to make the implementation of the abstract machine by each of the compilers have enough in common to support calling one from the other. The extensions I've suggested for the byte-code VM and lisp semantics are intended to support that interoperation, so the semantics of the byte-code implementation won't unnecessarily constrain the semantics of the native-code implementation. Lynn --000000000000868da005e1d72df9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Sun, Jun 19, 2022 at 7:02 PM Stefa= n Monnier <monnier@iro.umont= real.ca> wrote:
> Currently compiling a top-level expression wrapped in
> eval-when-compile by itself leaves no residue in the compiled=C2=A0 ou= tput,

`eval-when-compile` has 2 effects:

1- Run the code within the compiler's process.
=C2=A0 =C2=A0E.g.=C2=A0 (eval-when-compile=C2=A0 (require 'cl-lib)). =C2=A0 =C2=A0This is somewhat comparable to loading a gcc plugin during
=C2=A0 =C2=A0a compilation: it affects the GCC process itself, rather than = the
=C2=A0 =C2=A0code it emits.

2- It replaces the (eval-when-compile ...) thingy with the value
=C2=A0 =C2=A0returned by the evaluation of this code.=C2=A0 So you can do (= defvar
=C2=A0 =C2=A0my-str (eval-when-compile (concat "foo" "bar&qu= ot;))) and you know that
=C2=A0 =C2=A0the concatenation will be done during compilation.

> but I would want to make the above evaluate to an object at run-time > where the exported symbols in the obstack are immutable.

Then it wouldn't be called `eval-when-compile` because it would do
something quite different from what `eval-when-compile` does :-)


The informal semantics of "eval-w= hen-compile" from the elisp info file are that=C2=A0
=C2=A0 = =C2=A0 =C2=A0This form marks BODY to be evaluated at compile time but not w= hen
=C2=A0 =C2=A0 =C2=A0the compiled program is loaded.=C2=A0 The = result of evaluation by the
=C2=A0 =C2=A0 =C2=A0compiler becomes a const= ant which appears in the compiled program.
=C2=A0 =C2=A0 =C2=A0If you lo= ad the source file, rather than compiling it, BODY is
=C2=A0 =C2=A0 =C2= =A0evaluated normally.
I'm not sure what I have proposed that w= ould be inconsistent with "the result of evaluation=C2=A0
by= the compiler becomes a constant which appears in the compiled program"= ;.
The exact form of that appearance in the compiled program is n= ot specified.
For example, the byte-compile of (eval-when-com= pile (cl-labels ((f...) (g ...)))
currently produces a byte-code = vector in which f and g are byte-code vectors with
shared structu= re.=C2=A0 However, that representation is only one choice.
<= br>
It is inconsistent with the semantics of *symbols* as they cu= rrently stand, as I have already admitted.
Even there, you could = advance a model where it is not inconsistent.=C2=A0 For example,
= if you view the binding of symbol to value as having two components - the b= inding and the cell
holding the mutable value during the ex= tent of the symbol as a global/dynamically scoped variable,
then = having the binding of the symbol to the final value of the cell before the = dynamic extent of the variable
terminates would be consistent.=C2= =A0 That's not how it's currently implemented, because there is no = way to
express the final compile-time environment as a value afte= r compilation has completed with the
current semantics.

The part that's incompatible with current semantics of = symbols is importing that symbol as=C2=A0
an immutable symbolic r= eference.=C2=A0 Not really a "variable" reference, but as a bindi= ng
of a symbol to a value in the run-time namespace (or package i= n CL terminology, although
CL did not allow any way to specify wh= at I'm suggesting either, as far as I know).

H= owever, that would capture the semantics of ELF shared objects with the tex= t and ro_data
segments loaded into memory that is in fact immutab= le for a userspace program.
=C2=A0
> byte-code (or native-code) instruction arrays.=C2=A0 This would in tur= n enable
> implementing proper tail recursion as "goto with arguments".=

Proper tail recursion elimination would require changing the *normal*
function call protocol.=C2=A0 I suspect you're thinking of a smaller-sc= ale
version of it specifically tailored to self-recursion, kind of like
what `named-let` provides.=C2=A0 Note that such ad-hoc TCO tends to hit the= same
semantic issues as the -O3 optimization of the native compiler.
E.g. in code like the following:

=C2=A0 =C2=A0 (defun vc-foo-register (file)
=C2=A0 =C2=A0 =C2=A0 (when (some-hint-is-true)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (load "vc-foo")
=C2=A0 =C2=A0 =C2=A0 =C2=A0 (vc-foo-register file)))

the final call to `vc-foo-register` is in tail position but is not
a self call because loading `vc-foo` is expected to redefine
`vc-foo-register` with the real implementation.

I'm only talking about the steps that are require= d to allow the compiler to=C2=A0
produce code that implements pro= per tail recursion.
With the abstract machine currently implement= ed by the byte-code VM,
the "call[n]" instructions will= always be needed to call out according to
the C calling conventi= ons.
The call[-absolute/relative] or [goto-absolute] instructions= I suggested
*would be* used in the "normal" function-c= all protocol in place of the current
funcall dispatch, at least t= o functions defined in lisp.=C2=A0=C2=A0
This is necessary but no= t sufficient for proper tail recursion.
To actually get proper ta= il recursion requires the compiler to use the instructions
for im= plementing the appropriate function call protocol, especially if
= "goto-absolute" is the instruction provided for changing the PC r= egister.
Other instructions would have to be issued to manage the= stack frame
explicitly if that were the route taken.=C2=A0 Or,= =C2=A0 a more CISCish call-absolute
type of instruction could be = used that would perform that stack frame
management implicitly.
EIther way, it's the compiler that has to determine whether a = return
instruction following a control transfer can be safely eli= minated or not.
If the "goto-absolute" instruction were= used, the compiler would
have to decide whether the address foll= owing the "goto-absolute"
should be pushed in a new fra= me, or if it can be "pre-emptively
garbage collected"= =C2=A0 at compile time because it's a tail call.
=C2=A0
> I'm not familiar with emacs's profiling facilities.=C2=A0 Is i= t possible to
> tell how much of the allocated space/time spent in gc is due to the > constant vectors of lexical closures?=C2=A0 In particular, how much of= the
> constant vectors are copied elements independent of the lexical
> environment?=C2=A0 That would provide some measure of any gc-related > benefit that *might* be gained from using an explicit environment
> register for closures, instead of embedding it in the
> byte-code vector.

No, I can't think of any profiling tool we currently have that can help=
with that, sorry :-(

Note that when support for native closures is added to the native
compiler, it will hopefully not be using this clunky representation
where capture vars are mixed in with the vector of constants, so that
might be a more promising direction (may be able to skip the step where
we need to change the bytecode).


Th= e trick is to make the implementation of the abstract machine by each of th= e
compilers have enough in common to support calling one from the= other.
The extensions I've suggested for the byte-code VM an= d lisp semantics
are intended to support that interoperation, so = the semantics of the byte-code
implementation won't unnecessa= rily constrain the semantics of the native-code
implementation.

Lynn



=C2=A0
--000000000000868da005e1d72df9--