bug#58509: 29.0.50; Synchronous nativecomp

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

I thought I'd open a bug report for something I mentioned elsewhere: It
should be possible for --batch Emacsen to do native compilation.

I've got a bunch of Elisp scripts, and some of them do a lot of stuff,
and it'd be nice if they were native-compiled.  This will happen if I
also use the same packages interactively, but that often doesn't happen.
(And besides, since there's a bunch of different Emacs versions, they
won't use the same eln cache anyway.)

So I'd like to be able to say "emacs --batch --native-compile -l ..." or
something along those lines.

However, this also almost requires that we allow the nativecomp to
happen synchronously -- the batch job may end before nativecomp finished
otherwise, leading to cleanup problems and loss of efficiency.  So in
this mode, loading an .elc file (with missing .eln file) should instead
just trigger .eln generation, and then we load the .eln file.

And then perhaps we don't actually have to fork an Emacs to do the .eln
generation?  We can just do it in the currently running one, leading to
further efficiency gains.

This is also useful as a general tool for debugging: Today, things
aren't really reproducible, because things happen asynchronously and
unpredictably.  So if we have this, we can also use it as a debugging
tool in general, and not just in --batch.



In GNU Emacs 29.0.50 (build 144, x86_64-pc-linux-gnu, GTK+ Version
 3.24.33, cairo version 1.16.0) of 2022-10-13 built on downe
Repository revision: cbd04ad3d572850775f18bde868c71abcde733ed
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101003
System Description: Ubuntu 22.04.1 LTS

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> Cc: Andrea Corallo <akrl@sdf.org>
> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Fri, 14 Oct 2022 12:38:30 +0200
> 
> 
> I thought I'd open a bug report for something I mentioned elsewhere: It
> should be possible for --batch Emacsen to do native compilation.

We already have that, and we use it when users build a release
tarball.  Here's the lisp/Makefile snippet which  does it:

  ifeq ($(HAVE_NATIVE_COMP),yes)
  .PHONY: $(THEFILE)n
  $(THEFILE)n:
	  $(AM_V_ELN)$(emacs) $(BYTE_COMPILE_FLAGS) \
		  -l comp -f byte-compile-refresh-preloaded \
		  --eval '(batch-native-compile t)' $(THEFILE)

We also have the batch-native-compile function, which is probably a
better match for what you want to do in your case (compile a bunch of
your own Lisp files, AFAIU).

> However, this also almost requires that we allow the nativecomp to
> happen synchronously -- the batch job may end before nativecomp finished
> otherwise, leading to cleanup problems and loss of efficiency.

The above-mentioned functions do work synchronously.

So what is actually missing?

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> However, this also almost requires that we allow the nativecomp to
>> happen synchronously -- the batch job may end before nativecomp finished
>> otherwise, leading to cleanup problems and loss of efficiency.
>
> The above-mentioned functions do work synchronously.

`batch-native-compile' just calls `byte-compile-file' on the command
line arguments.  That's not what's needed here at all.

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 58509@debbugs.gnu.org,  akrl@sdf.org
> Date: Fri, 14 Oct 2022 12:56:07 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> However, this also almost requires that we allow the nativecomp to
> >> happen synchronously -- the batch job may end before nativecomp finished
> >> otherwise, leading to cleanup problems and loss of efficiency.
> >
> > The above-mentioned functions do work synchronously.
> 
> `batch-native-compile' just calls `byte-compile-file' on the command
> line arguments.

It does?  I see this:

    (cl-loop for file in command-line-args-left
             if (or (null byte+native-compile)
                    (cl-notany (lambda (re) (string-match re file))
                               native-comp-bootstrap-deny-list))
             collect (comp--native-compile file)  <<<<<<<<<<<<<<<<<<<<<<
             else
             collect (byte-compile-file file))))

What did I miss?

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> `batch-native-compile' just calls `byte-compile-file' on the command
>> line arguments.
>
> It does?  I see this:
>
>     (cl-loop for file in command-line-args-left
>              if (or (null byte+native-compile)
>                     (cl-notany (lambda (re) (string-match re file))
>                                native-comp-bootstrap-deny-list))
>              collect (comp--native-compile file)  <<<<<<<<<<<<<<<<<<<<<<
>              else
>              collect (byte-compile-file file))))
>
> What did I miss?

Sorry, I meant "it just calls comp--native-compile on the command line
arguments".  Which is irrelevant for the proposed use case, which is
enabling normal opportunistic .eln generation, but synchronously.

I.e., if you say

emacs --batch --native-comp -l foo

and foo requires bar, then bar should be native-compiled before being
used.

So I'm not sure why you're bringing up batch-native-compile at all -- it
seems irrelevant, but perhaps I'm missing something?

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 58509@debbugs.gnu.org,  akrl@sdf.org
> Date: Fri, 14 Oct 2022 13:13:40 +0200
> 
> So I'm not sure why you're bringing up batch-native-compile at all -- it
> seems irrelevant, but perhaps I'm missing something?

It is documented as "Native compilation equivalent of
`batch-byte-compile'", so if it doesn't do what it's advertised to do,
it should be fixed to do it.

Let's wait for Andrea to chime in.

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> It is documented as "Native compilation equivalent of
> `batch-byte-compile'", so if it doesn't do what it's advertised to do,
> it should be fixed to do it.

It is, but it seems fully irrelevant for the use case described in this
bug report, unless I'm missing something?

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 58509@debbugs.gnu.org,  akrl@sdf.org
> Date: Fri, 14 Oct 2022 13:51:27 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It is documented as "Native compilation equivalent of
> > `batch-byte-compile'", so if it doesn't do what it's advertised to do,
> > it should be fixed to do it.
> 
> It is, but it seems fully irrelevant for the use case described in this
> bug report, unless I'm missing something?

Then maybe I'm misunderstanding the use case?  Didn't you say you
wanted to compile a bunch of Lisp files natively?

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Then maybe I'm misunderstanding the use case?  Didn't you say you
> wanted to compile a bunch of Lisp files natively?

I said I wanted Emacs to native-compile stuff (almost as normal) when
using --batch.

But perhaps it'd be helpful if you read the original report; I don't
really have much to add to it, and I've just been repeating things
already in it in these responses.

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 58509@debbugs.gnu.org,  akrl@sdf.org
> Date: Fri, 14 Oct 2022 13:59:47 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Then maybe I'm misunderstanding the use case?  Didn't you say you
> > wanted to compile a bunch of Lisp files natively?
> 
> I said I wanted Emacs to native-compile stuff (almost as normal) when
> using --batch.
> 
> But perhaps it'd be helpful if you read the original report; I don't
> really have much to add to it, and I've just been repeating things
> already in it in these responses.

The original report said:

> I thought I'd open a bug report for something I mentioned elsewhere: It
> should be possible for --batch Emacsen to do native compilation.
[...]
> So I'd like to be able to say "emacs --batch --native-compile -l ..." or
> something along those lines.

So it seemed to me like being able to say

  emacs --batch -f batch-native-compile FILE...

is what you want?  If not, please tell what I missed in the original
report.

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> The original report said:
>
>> I thought I'd open a bug report for something I mentioned elsewhere: It
>> should be possible for --batch Emacsen to do native compilation.
> [...]
>> So I'd like to be able to say "emacs --batch --native-compile -l ..." or
>> something along those lines.
>
> So it seemed to me like being able to say
>
>   emacs --batch -f batch-native-compile FILE...
>
> is what you want?  If not, please tell what I missed in the original
> report.

You missed that the --native-compile there is in no way related to the
-l that follows.  I.e., for instance:

emacs --batch --native-compile --eval '(gnus)'

should native-compile all parts of running Gnus.

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 58509@debbugs.gnu.org,  akrl@sdf.org
> Date: Fri, 14 Oct 2022 14:20:24 +0200
> 
> You missed that the --native-compile there is in no way related to the
> -l that follows.  I.e., for instance:
> 
> emacs --batch --native-compile --eval '(gnus)'

(There was no "--eval '(gnus)" in the original report.)

> should native-compile all parts of running Gnus.

Gnus is an interactive program, and I'm quite sure what exactly is
loaded as side effect of "M-x gnus RET" depends on what you do next,
after Gnus starts up.  So the above could perhaps compile everything
that just starting Gnus loads, but it cannot compile anything else.

Besides, what is the semantics of

  emacs --batch --eval '(gnus)

?  IOW, what does it mean to run Gnus in batch mode?

So I still don't understand the request and its semantics.  I also
don't understand how it differs from just starting Gnus interactively
and having whatever is needed compiled asynchronously.  But since my
questions seem to annoy you, I will stop asking them; I'm sorry I
started.  Just one last thing: if the intent is to tell Emacs "compile
natively, but do that in the same Emacs session instead of in
subprocesses", then how is this different from

  emacs --batch --batch-native-compile lisp/gnus/gnus*.el --eval '(gnus)

which is what I had in mind?

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> You missed that the --native-compile there is in no way related to the
>> -l that follows.  I.e., for instance:
>> 
>> emacs --batch --native-compile --eval '(gnus)'
>
> (There was no "--eval '(gnus)" in the original report.)

It's just an example.  Substitute with
`(calculate-the-535325th-prime-number)' or whatever.

> Just one last thing: if the intent is to tell Emacs "compile
> natively, but do that in the same Emacs session instead of in
> subprocesses", then how is this different from
> 
>   emacs --batch --batch-native-compile lisp/gnus/gnus*.el --eval '(gnus)
> 
> which is what I had in mind?

You're suggesting that users should compute with files a command will
load, then compile those, and then start the command?  That doesn't
sound like a good interface, and fortunately that's not how Emacs works
interactively.

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 58509@debbugs.gnu.org,  akrl@sdf.org
> Date: Fri, 14 Oct 2022 14:59:32 +0200
> 
> >   emacs --batch --batch-native-compile lisp/gnus/gnus*.el --eval '(gnus)
> > 
> > which is what I had in mind?
> 
> You're suggesting that users should compute with files a command will
> load, then compile those, and then start the command?

What else is possible in --batch?

> That doesn't sound like a good interface, and fortunately that's not
> how Emacs works interactively.

Interactively, async compilation is triggered by actually _loading_
stuff.  You cannot have Emacs load everything that is need for some
function call, unless you let the code of that function actually run,
because only running it will reveal all that's needed to be loaded.
Think about autoloads, eval-after-load, etc.  How can Emacs know what
files to compile ahead of running the code you are asking it to run?
Or what am I missing?

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> That doesn't sound like a good interface, and fortunately that's not
>> how Emacs works interactively.
>
> Interactively, async compilation is triggered by actually _loading_
> stuff.  You cannot have Emacs load everything that is need for some
> function call, unless you let the code of that function actually run,
> because only running it will reveal all that's needed to be loaded.
> Think about autoloads, eval-after-load, etc.  How can Emacs know what
> files to compile ahead of running the code you are asking it to run?
> Or what am I missing?

It really sounds like you're not familiar with how the nativecomp
machinery works?

When Emacs is asked to load a file, no matter how it is asked to do so,
it (eventually) looks at the .elc file (if any).  This makes the
native-comp machinery kick in, and at present (like I said in the
original description here) it adds the .el file to a list of files that
will be native-compiled in the background, and then loads the .eln file
later.

And like I said, at that point, we could instead just go ahead and
native-compile the .el file, and then load the .eln file instead of
loading the .elc file, synchronously.

Nothing needs to be known in advance, because the nativecomp machinery
already has all this under control.

And, yes, this thread has been really annoying, because I explained all
this in the first message already.

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

[...]

> Let's wait for Andrea to chime in.

Hi Eli,

to get practical I think what Lars'd like to have is: at the end of
'maybe_swap_for_eln', in case the .eln is not found, a syncronous
invocation of the native compiler to produce and use the .eln file
(indeed only when this flag is used).

In principal as decribed it should be relatively easy, the only
technical caveat I can think of now are potential circular dependecies
generated by loading comp.el.

The comment I have is that we could have this mode capable of being used
also in interactive sessions, in case I think we should really think of
a better name for the flag, something like --native-compile-sync?  Or
anyway something that highlights the "synchronousness" of the enabled
mechanism.

Best Regards

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>, 58509@debbugs.gnu.org
> Date: Fri, 14 Oct 2022 21:40:06 +0000
> 
> to get practical I think what Lars'd like to have is: at the end of
> 'maybe_swap_for_eln', in case the .eln is not found, a syncronous
> invocation of the native compiler to produce and use the .eln file
> (indeed only when this flag is used).
> 
> In principal as decribed it should be relatively easy, the only
> technical caveat I can think of now are potential circular dependecies
> generated by loading comp.el.

Exactly.  Couldn't this cause an infinite recursion, if compiling a
file needs to load some other file that also needs to be compiled,
etc. etc.?  For example, if some primitive is advised, we'd need to
compile the same trampoline, recursively, no?  How do we break such
vicious circles?

> The comment I have is that we could have this mode capable of being used
> also in interactive sessions, in case I think we should really think of
> a better name for the flag, something like --native-compile-sync?  Or
> anyway something that highlights the "synchronousness" of the enabled
> mechanism.

I don't think the synchronous part is important here.  What is
important is that the Lisp code is natively-compiled before it is
loaded into Emacs.  So something like native-compile-before-load,
perhaps?

Which also suggests a way out of the circular dependency thing: we
could start the compilation in a subprocess and wait for it to finish.
Then any circular dependencies would be solved in the subprocess --
which will run with --batch, so it will simply use byte-compiled Lisp
for its compilation when the corresponding .eln files aren't
available.  Of course, this means when the "main" Emacs loads the
just-produced .eln file, it will find that dependency .el files were
not yet compiled, and will have to stop and wait for their
compilation.  But that is inevitable, I think, and will simply mean we
are serializing the recursion in a brute-force manner.

Does this make sense?

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Andrea Corallo <akrl@sdf.org> writes:

> to get practical I think what Lars'd like to have is: at the end of
> 'maybe_swap_for_eln', in case the .eln is not found, a syncronous
> invocation of the native compiler to produce and use the .eln file
> (indeed only when this flag is used).
>
> In principal as decribed it should be relatively easy, the only
> technical caveat I can think of now are potential circular dependecies
> generated by loading comp.el.

It's kinda the same problem as with the infinite recursion problem with
the trampolines, so perhaps this could have the same solution?  I.e., in
the Emacs that we fork to do the nativecomp, we should switch off both
JIT and trampoline generation?  (We'd need to add such a mechanism, of
course.)

> The comment I have is that we could have this mode capable of being used
> also in interactive sessions, in case I think we should really think of
> a better name for the flag, something like --native-compile-sync?  Or
> anyway something that highlights the "synchronousness" of the enabled
> mechanism.

Yes, that's a better name for the switch.

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Andrea Corallo <akrl@sdf.org> writes:
>
>> to get practical I think what Lars'd like to have is: at the end of
>> 'maybe_swap_for_eln', in case the .eln is not found, a syncronous
>> invocation of the native compiler to produce and use the .eln file
>> (indeed only when this flag is used).
>>
>> In principal as decribed it should be relatively easy, the only
>> technical caveat I can think of now are potential circular dependecies
>> generated by loading comp.el.
>
> It's kinda the same problem as with the infinite recursion problem with
> the trampolines, so perhaps this could have the same solution?  I.e., in
> the Emacs that we fork to do the nativecomp, we should switch off both
> JIT and trampoline generation?  (We'd need to add such a mechanism, of
> course.)

Yes, ATM we break circularity as in noninteractive sessions we disable
native compilation, if we allow for it we just have to handle the case.
The mechanism discussed for trampolines on emacs-devel should be
extended for lisp code as well and should do the job.

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Andrea Corallo <akrl@sdf.org> writes:

> Yes, ATM we break circularity as in noninteractive sessions we disable
> native compilation, if we allow for it we just have to handle the case.
> The mechanism discussed for trampolines on emacs-devel should be
> extended for lisp code as well and should do the job.

Yup.  Perhaps we could just have a special value for
`inhibit-automatic-native-compilation' to also skip trampoline
generation totally?

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Andrea Corallo <akrl@sdf.org> writes:
>
>> Yes, ATM we break circularity as in noninteractive sessions we disable
>> native compilation, if we allow for it we just have to handle the case.
>> The mechanism discussed for trampolines on emacs-devel should be
>> extended for lisp code as well and should do the job.
>
> Yup.  Perhaps we could just have a special value for
> `inhibit-automatic-native-compilation' to also skip trampoline
> generation totally?

I think `inhibit-automatic-native-compilation' does already too many
things (with an abstraction I find definitely too weak), that's the
reason I'm against it.  I don't think I want it to do even more.

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Andrea Corallo <akrl@sdf.org> writes:

> I think `inhibit-automatic-native-compilation' does already too many
> things (with an abstraction I find definitely too weak), that's the
> reason I'm against it.  I don't think I want it to do even more.

Well, the "more" here is very small.  A new value like `full' or `total'
to `inhibit-automatic-native-compilation' would solve the fork bomb subr
problem.

I.e., EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION=full when forking the
compiling Emacs would do the trick without introducing yet another
twiddle here.

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I.e., EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION=full when forking the
> compiling Emacs would do the trick without introducing yet another
> twiddle here.

Something like the following simple patch.

I think this should fix the fork bomb problem, and it doesn't seem to
introduce any other problems, but I've only tested it for a couple
minutes.

diff --git a/doc/lispref/compile.texi b/doc/lispref/compile.texi
index 6f8431c55c..3ac4e90c2a 100644
--- a/doc/lispref/compile.texi
+++ b/doc/lispref/compile.texi
@@ -993,10 +993,14 @@ Native-Compilation Variables
 While setting this variable disables automatic compilation of Lisp
 files, the compiler may still be invoked to install @dfn{trampolines}
 if any built-in functions are redefined.  However, these trampolines
-will not get written to your cache directory.
+will not get written to your cache directory.  If this variable has
+the special value @code{full}, even trampoline installation is
+inhibited, which means that redefinition of built-in functions isn't
+possible.
 
 You can also use the @samp{EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION}
-environment variable to disable native compilation.
+environment variable to disable automatic native compilation.  If it's
+@samp{"full"}, trampoline generation is also inhibited.
 @end defvar
 
 @defopt native-comp-speed
diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
index c300c44a8d..91851ad0f4 100644
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -693,6 +693,7 @@ 'native-compiler-error-empty-byte
 (defun comp-subr-trampoline-install (subr-name)
   "Make SUBR-NAME effectively advice-able when called from native code."
   (unless (or (null comp-enable-subr-trampolines)
+              (eq inhibit-automatic-native-compilation 'full)
               (memq subr-name native-comp-never-optimize-functions)
               (gethash subr-name comp-installed-trampolines-h))
     (cl-assert (subr-primitive-p (symbol-function subr-name)))
@@ -3713,9 +3714,13 @@ comp-final
 	(with-temp-buffer
           (unwind-protect
               (if (zerop
-                   (call-process (expand-file-name invocation-name
-                                                   invocation-directory)
-				 nil t t "--batch" "-l" temp-file))
+                   (let ((process-environment
+                          (cons
+                           "EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION=full"
+                           process-environment)))
+                     (call-process (expand-file-name invocation-name
+                                                     invocation-directory)
+				   nil t t "--batch" "-l" temp-file)))
                   (progn
                     (delete-file temp-file)
                     output)
diff --git a/lisp/startup.el b/lisp/startup.el
index 725984b815..7238bcc1cf 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -580,8 +580,12 @@ normal-top-level
 reads the initialization files, etc.
 It is the default value of the variable `top-level'."
   ;; Allow disabling automatic .elc->.eln processing.
-  (setq inhibit-automatic-native-compilation
-        (getenv "EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION"))
+  (let ((inhibit (getenv "EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION")))
+    (when inhibit
+      (setq inhibit-automatic-native-compilation
+            (if (equal inhibit "full")
+                'full
+              t))))
 
   (if command-line-processed
       (message internal--top-level-message)
diff --git a/src/comp.c b/src/comp.c
index 14012634cc..946816e069 100644
--- a/src/comp.c
+++ b/src/comp.c
@@ -5682,6 +5682,12 @@ syms_of_comp (void)
 	       Vinhibit_automatic_native_compilation,
 	       doc: /* If non-nil, inhibit automatic native compilation of loaded .elc files.
 
+If `full', also inhibit trampoline generation.  Any other non-nil
+value will inhibit automatic native compilation of loaded .elc files,
+but will still generate trampolines (without writing them to the eln
+cache).  If `full', it's no longer possible to reliably redefine
+built-in functions.
+
 After compilation, each function definition is updated to the native
 compiled one.  */);
   Vinhibit_automatic_native_compilation = Qnil;

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  58509@debbugs.gnu.org
> Date: Mon, 17 Oct 2022 13:57:21 +0200
> 
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
> > I.e., EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION=full when forking the
> > compiling Emacs would do the trick without introducing yet another
> > twiddle here.
> 
> Something like the following simple patch.

Thanks, but passing information via environment variable is not a good
idea, IME.  It is much better to pass that via command-line arguments.
We've been moving away of the environment variables, albeit slowly, so
adding a new one would be a step back.

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks, but passing information via environment variable is not a good
> idea, IME.  It is much better to pass that via command-line arguments.
> We've been moving away of the environment variables, albeit slowly, so
> adding a new one would be a step back.

Sure, we can do it via a command-line argument instead.  Perhaps this
fits well with the proposed --native-compile-sync switch, even.

That is, instead of that, we could have a --native-compile-mode command
line switch with the values --native-compile-mode=none,
--native-compile-mode=sync and --native-compile-mode=normal?

Or a better name for the switch, if anybody has an idea -- I don't much
like "mode" here, since that can be confused with Emacs modes...

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: akrl@sdf.org,  58509@debbugs.gnu.org
> Date: Mon, 17 Oct 2022 21:30:24 +0200
> 
> Sure, we can do it via a command-line argument instead.  Perhaps this
> fits well with the proposed --native-compile-sync switch, even.
> 
> That is, instead of that, we could have a --native-compile-mode command
> line switch with the values --native-compile-mode=none,
> --native-compile-mode=sync and --native-compile-mode=normal?

I doubt we really need to have an option with 3 values.  The default
already does TRT in almost all cases, so all we need is to suppress
async compilation of trampolines, because that could cause trouble.

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

[-- Attachment #1: Type: text/plain, Size: 1012 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Cc: akrl@sdf.org,  58509@debbugs.gnu.org
>> Date: Mon, 17 Oct 2022 21:30:24 +0200
>> 
>> Sure, we can do it via a command-line argument instead.  Perhaps this
>> fits well with the proposed --native-compile-sync switch, even.
>> 
>> That is, instead of that, we could have a --native-compile-mode command
>> line switch with the values --native-compile-mode=none,
>> --native-compile-mode=sync and --native-compile-mode=normal?
>
> I doubt we really need to have an option with 3 values.  The default
> already does TRT in almost all cases, so all we need is to suppress
> async compilation of trampolines, because that could cause trouble.

The attached works for me for fixing the trampoline issue (unless I did
something wrong reproducing).

I named the flag -no-comp-spawn from the variable it controls.  Happy to
rename it if we have something better, but being for internal use only
maybe is already okay?

BR

  Andrea


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Set-comp-no-spawn-earlier-using-no-comp-spawn.patch --]
[-- Type: text/x-diff, Size: 4183 bytes --]

From 01e8bc45a34f754af13e6ff05de71b320a15ffc0 Mon Sep 17 00:00:00 2001
From: Andrea Corallo <andrea.corallo@arm.com>
Date: Tue, 18 Oct 2022 15:14:32 +0200
Subject: [PATCH] Set `comp-no-spawn' earlier using -no-comp-spawn

* src/emacs.c (standard_args): Add '-no-comp-spawn' cmd line option.
* lisp/startup.el (command-line): Parse '-no-comp-spawn' cmd line
option.
* lisp/emacs-lisp/comp.el (comp-run-async-workers, comp-final):
Use '-no-comp-spawn'.
---
 lisp/emacs-lisp/comp.el | 10 +++++-----
 lisp/startup.el         |  5 ++++-
 src/emacs.c             |  1 +
 3 files changed, 10 insertions(+), 6 deletions(-)

diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
index 2c9b79334b..ddfdbf286a 100644
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -3689,8 +3689,7 @@ comp-final
              (print-circle t)
              (print-escape-multibyte t)
              (expr `((require 'comp)
-                     (setf comp-no-spawn t
-                           native-comp-verbose ,native-comp-verbose
+                     (setf native-comp-verbose ,native-comp-verbose
                            comp-libgccjit-reproducer ,comp-libgccjit-reproducer
                            comp-ctxt ,comp-ctxt
                            native-comp-eln-load-path ',native-comp-eln-load-path
@@ -3716,7 +3715,8 @@ comp-final
               (if (zerop
                    (call-process (expand-file-name invocation-name
                                                    invocation-directory)
-				 nil t t "--batch" "-l" temp-file))
+				 nil t t "-no-comp-spawn" "--batch" "-l"
+                                 temp-file))
                   (progn
                     (delete-file temp-file)
                     output)
@@ -3947,7 +3947,6 @@ comp-run-async-workers
                      source-file (comp-el-to-eln-filename source-file))))
          do (let* ((expr `((require 'comp)
                            (setq comp-async-compilation t
-                                 comp-no-spawn t
                                  warning-fill-column most-positive-fixnum)
                            ,(let ((set (list 'setq)))
                               (dolist (var '(comp-file-preloaded-p
@@ -4004,7 +4003,8 @@ comp-run-async-workers
                              :command (list
                                        (expand-file-name invocation-name
                                                          invocation-directory)
-                                       "--batch" "-l" temp-file)
+                                       "-no-comp-spawn" "--batch" "-l"
+                                       temp-file)
                              :sentinel
                              (lambda (process _event)
                                (run-hook-with-args
diff --git a/lisp/startup.el b/lisp/startup.el
index 725984b815..70267fc857 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -1198,7 +1198,7 @@ command-line
                          ("--user") ("--iconic") ("--icon-type") ("--quick")
 			 ("--no-blinking-cursor") ("--basic-display")
                          ("--dump-file") ("--temacs") ("--seccomp")
-                         ("--init-directory")))
+                         ("--init-directory" "--no-comp-spawn")))
              (argi (pop args))
              (orig-argi argi)
              argval)
@@ -1255,6 +1255,9 @@ command-line
 	 ((equal argi "-no-site-file")
 	  (setq site-run-file nil)
 	  (put 'site-run-file 'standard-value '(nil)))
+         ((equal argi "-no-comp-spawn")
+          (defvar comp-no-spawn)
+          (setq comp-no-spawn t))
 	 ((equal argi "-debug-init")
 	  (setq init-file-debug t))
 	 ((equal argi "-iconic")
diff --git a/src/emacs.c b/src/emacs.c
index 43e81b912c..8ad70fecd4 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -2564,6 +2564,7 @@ main (int argc, char **argv)
   { "-init-directory", "--init-directory", 30, 1 },
   { "-no-x-resources", "--no-x-resources", 40, 0 },
   { "-no-site-file", "--no-site-file", 40, 0 },
+  { "-no-comp-spawn", "--no-comp-spawn", 60, 0 },
   { "-u", "--user", 30, 1 },
   { "-user", 0, 30, 1 },
   { "-debug-init", "--debug-init", 20, 0 },
-- 
2.25.1

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Andrea Corallo <akrl@sdf.org> writes:

[...]

>  	  (put 'site-run-file 'standard-value '(nil)))
> +         ((equal argi "-no-comp-spawn")

That is not a good command line argument name, and you haven't said why
you want to introduce yet another command line argument to tweak
behaviour in this area when we've already discussed adding the
--native-compile-sync command line argument.

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Andrea Corallo <akrl@sdf.org> writes:
>
> [...]
>
>>  	  (put 'site-run-file 'standard-value '(nil)))
>> +         ((equal argi "-no-comp-spawn")
>
> That is not a good command line argument name,

As I mentioned it would be for internal use only, but feel free to offer
a better one.

> and you haven't said why
> you want to introduce yet another command line argument to tweak
> behaviour in this area when we've already discussed adding the
> --native-compile-sync command line argument.

IIUC --native-compile-sync has a different semantic (and it might go in
at a different time as well), this patch is avoid compilation recursion.
I posted it in this bug as (as you mentioned) there was some related
discussion.

Best Regards

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>, 58509@debbugs.gnu.org
> Date: Tue, 18 Oct 2022 13:55:18 +0000
> 
> The attached works for me for fixing the trampoline issue (unless I did
> something wrong reproducing).
> 
> I named the flag -no-comp-spawn from the variable it controls.  Happy to
> rename it if we have something better, but being for internal use only
> maybe is already okay?

On second thought: why do we need a special knob for that?  Could we
instead _always_ compile trampolines "without spawning" in a --batch
session?  IOW, when do we need --batch to compile trampolines
asynchronously?

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: Lars Ingebrigtsen <larsi@gnus.org>, 58509@debbugs.gnu.org
>> Date: Tue, 18 Oct 2022 13:55:18 +0000
>> 
>> The attached works for me for fixing the trampoline issue (unless I did
>> something wrong reproducing).
>> 
>> I named the flag -no-comp-spawn from the variable it controls.  Happy to
>> rename it if we have something better, but being for internal use only
>> maybe is already okay?
>
> On second thought: why do we need a special knob for that?  Could we
> instead _always_ compile trampolines "without spawning" in a --batch
> session?

Hi Eli,

unfortunately we cannot as libgccjit leaks memory :/

BR

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
> Date: Wed, 19 Oct 2022 19:02:59 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Andrea Corallo <akrl@sdf.org>
> >> Cc: Lars Ingebrigtsen <larsi@gnus.org>, 58509@debbugs.gnu.org
> >> Date: Tue, 18 Oct 2022 13:55:18 +0000
> >> 
> >> The attached works for me for fixing the trampoline issue (unless I did
> >> something wrong reproducing).
> >> 
> >> I named the flag -no-comp-spawn from the variable it controls.  Happy to
> >> rename it if we have something better, but being for internal use only
> >> maybe is already okay?
> >
> > On second thought: why do we need a special knob for that?  Could we
> > instead _always_ compile trampolines "without spawning" in a --batch
> > session?
> 
> Hi Eli,
> 
> unfortunately we cannot as libgccjit leaks memory :/

I think there could be a misunderstanding here.  What I meant is to do
whatever we do under -no-comp-spawn, but do it always for trampolines
in the --batch invocations, without the need to see -no-comp-spawn on
the command line.

Or maybe I misunderstand what -no-comp-spawn does, in which case can
you tell me what I missed?

Thanks.

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
>> Date: Wed, 19 Oct 2022 19:02:59 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Andrea Corallo <akrl@sdf.org>
>> >> Cc: Lars Ingebrigtsen <larsi@gnus.org>, 58509@debbugs.gnu.org
>> >> Date: Tue, 18 Oct 2022 13:55:18 +0000
>> >> 
>> >> The attached works for me for fixing the trampoline issue (unless I did
>> >> something wrong reproducing).
>> >> 
>> >> I named the flag -no-comp-spawn from the variable it controls.  Happy to
>> >> rename it if we have something better, but being for internal use only
>> >> maybe is already okay?
>> >
>> > On second thought: why do we need a special knob for that?  Could we
>> > instead _always_ compile trampolines "without spawning" in a --batch
>> > session?
>> 
>> Hi Eli,
>> 
>> unfortunately we cannot as libgccjit leaks memory :/
>
> I think there could be a misunderstanding here.  What I meant is to do
> whatever we do under -no-comp-spawn, but do it always for trampolines
> in the --batch invocations, without the need to see -no-comp-spawn on
> the command line.

> "do it always for trampolines in the --batch invocations"

So you mean identifying that we are doing a trampoline compilation and
disable the native compiler without a specific flag?

But I'm not sure I understand 100%, hope the explaination that follows
helps clarifying.

> Or maybe I misunderstand what -no-comp-spawn does, in which case can
> you tell me what I missed?

-no-comp-spawn makes sure that in the spawend compilation processes, no
 matter what, we never spawn again other compilation processes.

We have two invocations for spawning processes, one for sync
compilations and one of async (none of the invocation is specific to
trampolines).  This patch is using -no-comp-spawn for both.

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
> Date: Wed, 19 Oct 2022 19:31:05 +0000
> 
> > "do it always for trampolines in the --batch invocations"
> 
> So you mean identifying that we are doing a trampoline compilation and
> disable the native compiler without a specific flag?

Yes, but only in -batch sessions.

I believe this is our logic now: if we are going to compile a
trampoline, we invoke an async subprocess with both -batch and
the -no-comp-spawn options.  But if the --batch session can figure
out that it's compiling a trampoline, it can automatically behave as
if -no-comp-spawn was passed on the command line, no?

> -no-comp-spawn makes sure that in the spawend compilation processes, no
>  matter what, we never spawn again other compilation processes.
> 
> We have two invocations for spawning processes, one for sync
> compilations and one of async (none of the invocation is specific to
> trampolines).  This patch is using -no-comp-spawn for both.

I'm asking why compilation of trampolines could behave like that
automatically, when the compilation is done in a -batch invocation.

Thanks.

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
>> Date: Wed, 19 Oct 2022 19:31:05 +0000
>> 
>> > "do it always for trampolines in the --batch invocations"
>> 
>> So you mean identifying that we are doing a trampoline compilation and
>> disable the native compiler without a specific flag?
>
> Yes, but only in -batch sessions.
>
> I believe this is our logic now: if we are going to compile a
> trampoline, we invoke an async subprocess

We invoke a sync compilation for trampolines (but I think it's not
relevant here)

> with both -batch and
> the -no-comp-spawn options.  But if the --batch session can figure
> out that it's compiling a trampoline, it can automatically behave as
> if -no-comp-spawn was passed on the command line, no?

Okay so IIUC your suggestion would be to: when we identify '--batch' we
search for a signature in the commandline to identify the trampoline
compilation and set in case `comp-no-spawn'?

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
> Date: Thu, 20 Oct 2022 07:23:55 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Andrea Corallo <akrl@sdf.org>
> >> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
> >> Date: Wed, 19 Oct 2022 19:31:05 +0000
> >> 
> >> > "do it always for trampolines in the --batch invocations"
> >> 
> >> So you mean identifying that we are doing a trampoline compilation and
> >> disable the native compiler without a specific flag?
> >
> > Yes, but only in -batch sessions.
> >
> > I believe this is our logic now: if we are going to compile a
> > trampoline, we invoke an async subprocess
> 
> We invoke a sync compilation for trampolines (but I think it's not
> relevant here)
> 
> > with both -batch and
> > the -no-comp-spawn options.  But if the --batch session can figure
> > out that it's compiling a trampoline, it can automatically behave as
> > if -no-comp-spawn was passed on the command line, no?
> 
> Okay so IIUC your suggestion would be to: when we identify '--batch' we
> search for a signature in the commandline to identify the trampoline
> compilation and set in case `comp-no-spawn'?

Yes.  And if that works, my next question is: can we then remove the
new -no-comp-spawn command-line option, or do we need it for some
other cases?

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
>> Date: Thu, 20 Oct 2022 07:23:55 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Andrea Corallo <akrl@sdf.org>
>> >> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
>> >> Date: Wed, 19 Oct 2022 19:31:05 +0000
>> >> 
>> >> > "do it always for trampolines in the --batch invocations"
>> >> 
>> >> So you mean identifying that we are doing a trampoline compilation and
>> >> disable the native compiler without a specific flag?
>> >
>> > Yes, but only in -batch sessions.
>> >
>> > I believe this is our logic now: if we are going to compile a
>> > trampoline, we invoke an async subprocess
>> 
>> We invoke a sync compilation for trampolines (but I think it's not
>> relevant here)
>> 
>> > with both -batch and
>> > the -no-comp-spawn options.  But if the --batch session can figure
>> > out that it's compiling a trampoline, it can automatically behave as
>> > if -no-comp-spawn was passed on the command line, no?
>> 
>> Okay so IIUC your suggestion would be to: when we identify '--batch' we
>> search for a signature in the commandline to identify the trampoline
>> compilation and set in case `comp-no-spawn'?
>
> Yes.  And if that works, my next question is: can we then remove the
> new -no-comp-spawn command-line option, or do we need it for some
> other cases?

If it works I think it should be equivalent at that point.  But at the
moment is not so trivial to identify this condition as we have no clear
marker of it.

The current invocation for compilations is just like:

emacs --batch -l sometmpfile.el

I don't know if we have some other option for adding a marker other than
the most obvious (the dedicated flag).

Best Regards

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
> Date: Sun, 23 Oct 2022 10:51:14 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Okay so IIUC your suggestion would be to: when we identify '--batch' we
> >> search for a signature in the commandline to identify the trampoline
> >> compilation and set in case `comp-no-spawn'?
> >
> > Yes.  And if that works, my next question is: can we then remove the
> > new -no-comp-spawn command-line option, or do we need it for some
> > other cases?
> 
> If it works I think it should be equivalent at that point.  But at the
> moment is not so trivial to identify this condition as we have no clear
> marker of it.
> 
> The current invocation for compilations is just like:
> 
> emacs --batch -l sometmpfile.el
> 
> I don't know if we have some other option for adding a marker other than
> the most obvious (the dedicated flag).

At some point during compilation, we surely know that we are compiling
a trampoline, right?  So I thought to avoid forking at that point, so
that we don't need yet another command-line option for internal
purposes.

But if this is too complicated and cumbersome, I'm okay with a
command-line switch.

Thanks.

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
>> Date: Sun, 23 Oct 2022 10:51:14 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Okay so IIUC your suggestion would be to: when we identify '--batch' we
>> >> search for a signature in the commandline to identify the trampoline
>> >> compilation and set in case `comp-no-spawn'?
>> >
>> > Yes.  And if that works, my next question is: can we then remove the
>> > new -no-comp-spawn command-line option, or do we need it for some
>> > other cases?
>> 
>> If it works I think it should be equivalent at that point.  But at the
>> moment is not so trivial to identify this condition as we have no clear
>> marker of it.
>> 
>> The current invocation for compilations is just like:
>> 
>> emacs --batch -l sometmpfile.el
>> 
>> I don't know if we have some other option for adding a marker other than
>> the most obvious (the dedicated flag).
>
> At some point during compilation, we surely know that we are compiling
> a trampoline, right?  So I thought to avoid forking at that point, so
> that we don't need yet another command-line option for internal
> purposes.

That's correct, but I believe the issue is that when we realize we are
compiling a trampoline it's too late, and we might have been decided
already a new trampoline needs to be compiled and installed.  The
parsing of the command line args happens way earlier and that's why this
technique works at solving this issue.

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Andrea Corallo <akrl@sdf.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Andrea Corallo <akrl@sdf.org>
>>> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
>>> Date: Sun, 23 Oct 2022 10:51:14 +0000
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> >> Okay so IIUC your suggestion would be to: when we identify '--batch' we
>>> >> search for a signature in the commandline to identify the trampoline
>>> >> compilation and set in case `comp-no-spawn'?
>>> >
>>> > Yes.  And if that works, my next question is: can we then remove the
>>> > new -no-comp-spawn command-line option, or do we need it for some
>>> > other cases?
>>> 
>>> If it works I think it should be equivalent at that point.  But at the
>>> moment is not so trivial to identify this condition as we have no clear
>>> marker of it.
>>> 
>>> The current invocation for compilations is just like:
>>> 
>>> emacs --batch -l sometmpfile.el
>>> 
>>> I don't know if we have some other option for adding a marker other than
>>> the most obvious (the dedicated flag).
>>
>> At some point during compilation, we surely know that we are compiling
>> a trampoline, right?  So I thought to avoid forking at that point, so
>> that we don't need yet another command-line option for internal
>> purposes.
>
> That's correct, but I believe the issue is that when we realize we are
> compiling a trampoline it's too late, and we might have been decided
> already a new trampoline needs to be compiled and installed.  The
> parsing of the command line args happens way earlier and that's why this
> technique works at solving this issue.
>
>   Andrea

What do we want to do?  Shall I push my patch that fixes the trampoline
bomb with the use of -no-comp-spawn?

Best Regards

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
> Date: Tue, 25 Oct 2022 19:54:27 +0000
> 
> >>> The current invocation for compilations is just like:
> >>> 
> >>> emacs --batch -l sometmpfile.el
> >>> 
> >>> I don't know if we have some other option for adding a marker other than
> >>> the most obvious (the dedicated flag).
> >>
> >> At some point during compilation, we surely know that we are compiling
> >> a trampoline, right?  So I thought to avoid forking at that point, so
> >> that we don't need yet another command-line option for internal
> >> purposes.
> >
> > That's correct, but I believe the issue is that when we realize we are
> > compiling a trampoline it's too late, and we might have been decided
> > already a new trampoline needs to be compiled and installed.  The
> > parsing of the command line args happens way earlier and that's why this
> > technique works at solving this issue.
> >
> >   Andrea
> 
> What do we want to do?  Shall I push my patch that fixes the trampoline
> bomb with the use of -no-comp-spawn?

Oh, I thought you already did?

Yes, please install that.  AFAIU, the simplification I had in mind
doesn't really simplify things, so your proposal seems to be the best
we have on the table.

Thanks.

bug#58509: 29.0.50; Synchronous nativecomp

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <akrl@sdf.org>
>> Cc: larsi@gnus.org, 58509@debbugs.gnu.org
>> Date: Tue, 25 Oct 2022 19:54:27 +0000
>> 
>> >>> The current invocation for compilations is just like:
>> >>> 
>> >>> emacs --batch -l sometmpfile.el
>> >>> 
>> >>> I don't know if we have some other option for adding a marker other than
>> >>> the most obvious (the dedicated flag).
>> >>
>> >> At some point during compilation, we surely know that we are compiling
>> >> a trampoline, right?  So I thought to avoid forking at that point, so
>> >> that we don't need yet another command-line option for internal
>> >> purposes.
>> >
>> > That's correct, but I believe the issue is that when we realize we are
>> > compiling a trampoline it's too late, and we might have been decided
>> > already a new trampoline needs to be compiled and installed.  The
>> > parsing of the command line args happens way earlier and that's why this
>> > technique works at solving this issue.
>> >
>> >   Andrea
>> 
>> What do we want to do?  Shall I push my patch that fixes the trampoline
>> bomb with the use of -no-comp-spawn?
>
> Oh, I thought you already did?
>
> Yes, please install that.

Done, into trunk as 5ad5b797f7.

Thanks

  Andrea

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> That is, instead of that, we could have a --native-compile-mode command
>> line switch with the values --native-compile-mode=none,
>> --native-compile-mode=sync and --native-compile-mode=normal?
>
> I doubt we really need to have an option with 3 values.  The default
> already does TRT in almost all cases, so all we need is to suppress
> async compilation of trampolines, because that could cause trouble.

For bug bug#58509 we need the sync case; for the trampoline fork bomb we
need the don't-do-anything case -- that's two cases, right?

Then the default, normal, NO-OP case is the third, which is pretty
normal practice when doing command line switches like this, so that you
don't have to mogrify the command line arguments when exec-ing and
choosing between the three cases.

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: akrl@sdf.org,  58509@debbugs.gnu.org
> Date: Tue, 18 Oct 2022 20:14:15 +0200
> 
> > I doubt we really need to have an option with 3 values.  The default
> > already does TRT in almost all cases, so all we need is to suppress
> > async compilation of trampolines, because that could cause trouble.
> 
> For bug bug#58509 we need the sync case; for the trampoline fork bomb we
> need the don't-do-anything case -- that's two cases, right?

The sync case doesn't need a command-line switch, though?  You can do
it with --eval.

bug#58509: 29.0.50; Synchronous nativecomp

[Eli Zaretskii]

> Cc: 58509@debbugs.gnu.org, akrl@sdf.org
> Date: Tue, 18 Oct 2022 21:32:57 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Lars Ingebrigtsen <larsi@gnus.org>
> > Cc: akrl@sdf.org,  58509@debbugs.gnu.org
> > Date: Tue, 18 Oct 2022 20:14:15 +0200
> > 
> > > I doubt we really need to have an option with 3 values.  The default
> > > already does TRT in almost all cases, so all we need is to suppress
> > > async compilation of trampolines, because that could cause trouble.
> > 
> > For bug bug#58509 we need the sync case; for the trampoline fork bomb we
> > need the don't-do-anything case -- that's two cases, right?
> 
> The sync case doesn't need a command-line switch, though?  You can do
> it with --eval.

Or with -f.

bug#58509: 29.0.50; Synchronous nativecomp

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> The sync case doesn't need a command-line switch, though?  You can do
>> it with --eval.
>
> Or with -f.

OK, I'm done.