limit native compilation - how

limit native compilation - how

[Colin Baxter 😺]

I want to limit native compilation to emacs' own libraries and not
include any of my own lisp files. I tried a .dir-locals.el file 

--8<---------------cut here---------------start------------->8---
((emacs-lisp-mode .
((eval: (setq no-native-compile t)))))
--8<---------------cut here---------------end--------------->8---

in the directory I want native compilation to avoid. That didn't
work. Perhaps I can't see the wood for the trees, but I don't understand
what's wrong with the .dir-locals. Can anyone help?

Thanks - Colin.

Re: limit native compilation - how

[Eli Zaretskii]

> From: Colin Baxter 😺 <m43cap@yandex.com>
> Date: Sat, 15 Jan 2022 15:42:34 +0000
> 
> 
> I want to limit native compilation to emacs' own libraries and not
> include any of my own lisp files. I tried a .dir-locals.el file 
> 
> --8<---------------cut here---------------start------------->8---
> ((emacs-lisp-mode .
> ((eval: (setq no-native-compile t)))))
> --8<---------------cut here---------------end--------------->8---
> 
> in the directory I want native compilation to avoid. That didn't
> work. Perhaps I can't see the wood for the trees, but I don't understand
> what's wrong with the .dir-locals. Can anyone help?

Did you try to customize native-comp-deferred-compilation-deny-list?

Re: limit native compilation - how

[Emanuel Berg via Users list for the GNU Emacs text editor]

Eli Zaretskii wrote:

> Did you try to customize
> native-comp-deferred-compilation-deny-list?

Don't have that here ...

GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, cairo version
1.16.0) of 2021-12-31

-- 
underground experts united
https://dataswamp.org/~incal

Re: limit native compilation - how

[Eli Zaretskii]

> Date: Sat, 15 Jan 2022 17:01:51 +0100
> From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> 
> Eli Zaretskii wrote:
> 
> > Did you try to customize
> > native-comp-deferred-compilation-deny-list?
> 
> Don't have that here ...
> 
> GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, cairo version
> 1.16.0) of 2021-12-31

In an Emacs compiled with native-compilation?  Then look closer,
because you do have it.

Re: limit native compilation - how

[Emanuel Berg via Users list for the GNU Emacs text editor]

Eli Zaretskii wrote:

>> Don't have that here ...
>> 
>> GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, cairo
>> version 1.16.0) of 2021-12-31
>
> In an Emacs compiled with native-compilation? Then look
> closer, because you do have it.

Nope,

  Debugger entered--Lisp error: (void-variable
  native-comp-deferred-compilation-deny-list)

while

  system-configuration-options is a variable defined in ‘C source code’.

  Its value is "--with-x-toolkit=no --with-native-compilation"

  String containing the configuration options Emacs was built with.

-- 
underground experts united
https://dataswamp.org/~incal

Re: limit native compilation - how

[Eli Zaretskii]

> Date: Sat, 15 Jan 2022 17:51:15 +0100
> From:  Emanuel Berg via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org>
> 
> Eli Zaretskii wrote:
> 
> >> Don't have that here ...
> >> 
> >> GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, cairo
> >> version 1.16.0) of 2021-12-31
> >
> > In an Emacs compiled with native-compilation? Then look
> > closer, because you do have it.
> 
> Nope,
> 
>   Debugger entered--Lisp error: (void-variable
>   native-comp-deferred-compilation-deny-list)
> 
> while
> 
>   system-configuration-options is a variable defined in ‘C source code’.
> 
>   Its value is "--with-x-toolkit=no --with-native-compilation"

You need to "M-x load-library RET comp RET" to have it defined.

Re: limit native compilation - how

[Colin Baxter 😺]

 Customizing "native-comp-deferred-compilation-deny-list" worked (after I
 got the right regexp!). My eln-cache is still enormous though ~
 40Mb. Is that usual?

Best wishes

Re: limit native compilation - how

[Eli Zaretskii]

> From: Colin Baxter 😺 <m43cap@yandex.com>
> Cc: help-gnu-emacs@gnu.org
> Cc: 
> Date: Sat, 15 Jan 2022 18:58:31 +0000
> 
>  Customizing "native-comp-deferred-compilation-deny-list" worked (after I
>  got the right regexp!). My eln-cache is still enormous though ~
>  40Mb. Is that usual?

Yes.  Mine is 49MB.  Each .eln file is about 300KB.

Re: limit native compilation - how

[H. Dieter Wilhelm]

Hello Colin

Colin Baxter 😺 <m43cap@yandex.com> writes:
>  Customizing "native-comp-deferred-compilation-deny-list" worked (after I
>  got the right regexp!). My eln-cache is still enormous though ~
>  40Mb. Is that usual?

I'm sorry, I don't have as yet experiences with the eln-cache.

But could you please tell me what the size of your native-lisp/ folder
is?  Just compiled 28.0.91 on Windows and its size is 106 Mb. 

Thank you

      Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Colin Baxter 😺]

Hello Dieter,
>>>>> H Dieter Wilhelm <dieter@duenenhof-wilhelm.de> writes:

    > Hello Colin Colin Baxter 😺 <m43cap@yandex.com> writes:
    >> Customizing "native-comp-deferred-compilation-deny-list" worked
    >> (after I got the right regexp!). My eln-cache is still enormous
    >> though ~ 40Mb. Is that usual?

    > I'm sorry, I don't have as yet experiences with the eln-cache.

    > But could you please tell me what the size of your native-lisp/
    > folder is?  Just compiled 28.0.91 on Windows and its size is 106
    > Mb.

I run emacs-29.0.50 directly from git without a "make install" and the
size of the lisp directory is 140Mb. For my installed emacs-28.0.90
(pre-test), its lisp is 81Mb. For my emacs-27.2, it's 79Mb.

I hope that helps.

Best wishes,

Colin.

Re: limit native compilation - how

[H. Dieter Wilhelm]

Hello Colin

Colin Baxter 😺 <m43cap@yandex.com> writes:
>     > Hello Colin Colin Baxter 😺 <m43cap@yandex.com> writes:
>     >> Customizing "native-comp-deferred-compilation-deny-list" worked
>     >> (after I got the right regexp!). My eln-cache is still enormous
>     >> though ~ 40Mb. Is that usual?
>
>     > I'm sorry, I don't have as yet experiences with the eln-cache.
>
>     > But could you please tell me what the size of your native-lisp/
>     > folder is?  Just compiled 28.0.91 on Windows and its size is 106
>     > Mb.
>
> I run emacs-29.0.50 directly from git without a "make install" and the
> size of the lisp directory is 140Mb. For my installed emacs-28.0.90
> (pre-test), its lisp is 81Mb. For my emacs-27.2, it's 79Mb.

Thank you for the statistics.  I assumed that you compiled
--with-native-compilation and that there appears then a "native-lisp/"
directory in the Emacs tree.  (But it might be a Windows build
peculiarity...)

        Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Eli Zaretskii]

On January 16, 2022 12:15:35 AM GMT+02:00, "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> wrote:
> Hello Colin
> 
> Colin Baxter 😺 <m43cap@yandex.com> writes:
> >     > Hello Colin Colin Baxter 😺 <m43cap@yandex.com> writes:
> >     >> Customizing "native-comp-deferred-compilation-deny-list" worked
> >     >> (after I got the right regexp!). My eln-cache is still enormous
> >     >> though ~ 40Mb. Is that usual?
> >
> >     > I'm sorry, I don't have as yet experiences with the eln-cache.
> >
> >     > But could you please tell me what the size of your native-lisp/
> >     > folder is?  Just compiled 28.0.91 on Windows and its size is 106
> >     > Mb.
> >
> > I run emacs-29.0.50 directly from git without a "make install" and the
> > size of the lisp directory is 140Mb. For my installed emacs-28.0.90
> > (pre-test), its lisp is 81Mb. For my emacs-27.2, it's 79Mb.
> 
> Thank you for the statistics.  I assumed that you compiled
> --with-native-compilation and that there appears then a "native-lisp/"
> directory in the Emacs tree.  (But it might be a Windows build
> peculiarity...)
> 
>         Dieter

It isn't a Windows peculiarity, the directory tree looks the same as on Unix.

Re: limit native compilation - how

[Eli Zaretskii]

On January 15, 2022 10:31:16 PM GMT+02:00, "Colin Baxter 😺" <m43cap@yandex.com> wrote:
> Hello Dieter,
> >>>>> H Dieter Wilhelm <dieter@duenenhof-wilhelm.de> writes:
> 
>     > Hello Colin Colin Baxter 😺 <m43cap@yandex.com> writes:
>     >> Customizing "native-comp-deferred-compilation-deny-list" worked
>     >> (after I got the right regexp!). My eln-cache is still enormous
>     >> though ~ 40Mb. Is that usual?
> 
>     > I'm sorry, I don't have as yet experiences with the eln-cache.
> 
>     > But could you please tell me what the size of your native-lisp/
>     > folder is?  Just compiled 28.0.91 on Windows and its size is 106
>     > Mb.
> 
> I run emacs-29.0.50 directly from git without a "make install" and the
> size of the lisp directory is 140Mb. For my installed emacs-28.0.90
> (pre-test), its lisp is 81Mb. For my emacs-27.2, it's 79Mb.
> 
> I hope that helps.
> 
> Best wishes,
> 
> Colin.
> 
> 

He asked about native-lisp directory, not about the lisp directory.

Re: limit native compilation - how

[Colin Baxter 😺]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

    > On January 15, 2022 10:31:16 PM GMT+02:00, "Colin Baxter 😺"
    > <m43cap@yandex.com> wrote:
    >> Hello Dieter, >>>>> H Dieter Wilhelm
    >> <dieter@duenenhof-wilhelm.de> writes:
    >> 
    >> > Hello Colin Colin Baxter 😺 <m43cap@yandex.com> writes: >>
    >> Customizing "native-comp-deferred-compilation-deny-list" worked
    >> >> (after I got the right regexp!). My eln-cache is still
    >> enormous >> though ~ 40Mb. Is that usual?
    >> 
    >> > I'm sorry, I don't have as yet experiences with the eln-cache.
    >> 
    >> > But could you please tell me what the size of your native-lisp/
    >> > folder is?  Just compiled 28.0.91 on Windows and its size is
    >> 106 > Mb.
    >> 
    >> I run emacs-29.0.50 directly from git without a "make install"
    >> and the size of the lisp directory is 140Mb. For my installed
    >> emacs-28.0.90 (pre-test), its lisp is 81Mb. For my emacs-27.2,
    >> it's 79Mb.
    >> 
    >> I hope that helps.
    >> 
    >> Best wishes,
    >> 
    >> Colin.
    >> 
    >> 

    > He asked about native-lisp directory, not about the lisp
    > directory.

15Mb, with 12Mb "preloaded".

Re: limit native compilation - how

[Eli Zaretskii]

> From: Colin Baxter 😺 <m43cap@yandex.com>
> Cc: help-gnu-emacs@gnu.org,  "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>     > He asked about native-lisp directory, not about the lisp
>     > directory.
> 
> 15Mb, with 12Mb "preloaded".

The Windows *.eln files are larger because we produce them with some
debug info, as an aid debugging Emacs.  (GDB has special tricks for
ELF-based platforms, so they don't work on Windows.)

Re: limit native compilation - how

[H. Dieter Wilhelm]

Colin Baxter 😺 <m43cap@yandex.com> writes:
>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>     > He asked about native-lisp directory, not about the lisp
>     > directory.

> 15Mb, with 12Mb "preloaded".

That's roughly the same - 14 Mb, 12 Mb respectively - I see now on a
Debian system, thanks

       Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Colin Baxter 😺]

>>>>> H Dieter Wilhelm <dieter@duenenhof-wilhelm.de> writes:

    > Colin Baxter 😺 <m43cap@yandex.com> writes:
    >>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
    >> > He asked about native-lisp directory, not about the lisp >
    >> directory.

    >> 15Mb, with 12Mb "preloaded".

    > That's roughly the same - 14 Mb, 12 Mb respectively - I see now on
    > a Debian system, thanks

Excellent. I am pleased you have a satisfactory outcome. Best wishes. 

Re: limit native compilation - how

[Eli Zaretskii]

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Cc: Eli Zaretskii <eliz@gnu.org>,  ,  help-gnu-emacs@gnu.org
> Date: Sat, 15 Jan 2022 20:21:57 +0100
> 
> But could you please tell me what the size of your native-lisp/ folder
> is?  Just compiled 28.0.91 on Windows and its size is 106 Mb. 

106MB sounds a bit too much; I have 43MB.

Did you use NATIVE_FULL_AOT=1, per chance?

Re: limit native compilation - how

[H. Dieter Wilhelm]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  ,  help-gnu-emacs@gnu.org
>> Date: Sat, 15 Jan 2022 20:21:57 +0100
>> 
>> But could you please tell me what the size of your native-lisp/ folder
>> is?  Just compiled 28.0.91 on Windows and its size is 106 Mb. 
>
> 106MB sounds a bit too much; I have 43MB.
>
> Did you use NATIVE_FULL_AOT=1, per chance?

Not to my knowledge. :-(

By the way, just compiled Emacs-28.0.91, from archived sources and not
the git sources, on a second machine and here the size of native-lisp/
is 55.5 MB.  I'm confused..

     Dieter

-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Eli Zaretskii]

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Cc: help-gnu-emacs@gnu.org
> Date: Sun, 16 Jan 2022 12:30:44 +0100
> 
> > 106MB sounds a bit too much; I have 43MB.
> >
> > Did you use NATIVE_FULL_AOT=1, per chance?
> 
> Not to my knowledge. :-(
> 
> By the way, just compiled Emacs-28.0.91, from archived sources and not
> the git sources, on a second machine and here the size of native-lisp/
> is 55.5 MB.  I'm confused..

55MB sounds about right to me.

Do you have the same GCC/libgccjit versions on both systems?

Re: limit native compilation - how

[H. Dieter Wilhelm]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>> Cc: help-gnu-emacs@gnu.org
>> Date: Sun, 16 Jan 2022 12:30:44 +0100
>> 
>> > 106MB sounds a bit too much; I have 43MB.
>> >
>> > Did you use NATIVE_FULL_AOT=1, per chance?
>> 
>> Not to my knowledge. :-(
>> 
>> By the way, just compiled Emacs-28.0.91, from archived sources and not
>> the git sources, on a second machine and here the size of native-lisp/
>> is 55.5 MB.  I'm confused..
>
> 55MB sounds about right to me.
>
> Do you have the same GCC/libgccjit versions on both systems?

machine 55MB: gcc 10.3.0
            : mingw-w64-x86_64-libgccjit 11.2.0-1
            (mingw-w64-x86_64-toolchain)  [installed]: 10.3.0-2
machine 106MB: gcc 10.3.0
            : mingw-w64-x86_64-libgccjit 11.2.0-1
            (mingw-w64-x86_64-toolchain)  [installed]: 10.3.0-2

Could the difference in size result from opening my Emacs, I think I
checked Emacs-28.0.91 on the machine 106MB?  On machine 55MB I didn't
run Emacs yet.

        Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Emanuel Berg via Users list for the GNU Emacs text editor]

H. Dieter Wilhelm wrote:

> machine 55MB: gcc 10.3.0
>             : mingw-w64-x86_64-libgccjit 11.2.0-1
>             (mingw-w64-x86_64-toolchain)  [installed]: 10.3.0-2
> machine 106MB: gcc 10.3.0
>             : mingw-w64-x86_64-libgccjit 11.2.0-1
>             (mingw-w64-x86_64-toolchain)  [installed]: 10.3.0-2
>
> Could the difference in size result from opening my Emacs,
> I think I checked Emacs-28.0.91 on the machine 106MB?
> On machine 55MB I didn't run Emacs yet.

Sounds possible since native-compile will happen on `load' if
there is a byte-compiled version available next to the
original Elisp source.

Mine is 42M but I did `system-configuration-options' as
"--with-x-toolkit=no --with-native-compilation" on Debian so
maybe the missing 13M are X-rated ... I mean related.

-- 
underground experts united
https://dataswamp.org/~incal

Re: limit native compilation - how

[Eli Zaretskii]

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Cc: help-gnu-emacs@gnu.org
> Date: Sun, 16 Jan 2022 13:01:37 +0100
> 
> Could the difference in size result from opening my Emacs, I think I
> checked Emacs-28.0.91 on the machine 106MB?  On machine 55MB I didn't
> run Emacs yet.

The contents of the native-lisp/ directory doesn't change once Emacs
is built.

If you cannot figure this out, compare the contents of the two
native-lisp/ directories.

Re: limit native compilation - how

[H. Dieter Wilhelm]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>> Cc: help-gnu-emacs@gnu.org
>> Date: Sun, 16 Jan 2022 13:01:37 +0100
>> 
>> Could the difference in size result from opening my Emacs, I think I
>> checked Emacs-28.0.91 on the machine 106MB?  On machine 55MB I didn't
>> run Emacs yet.
>
> The contents of the native-lisp/ directory doesn't change once Emacs
> is built.

Really, please see below, could it be you meant perhaps the preloaded/
folder? (Just guessing.)

> If you cannot figure this out, compare the contents of the two
> native-lisp/ directories.

machine 105 MB:
|- native-lisp/
   |- 28.0.91-59ee6325/
   |  |- Some .eln files
   |  |- preloaded/
   |     | 45 MB .eln
   |- 28.0.91-aa40abb8/
      | Some other .eln files
      |- preloaded/
         |- 45 Mb .eln files

machine 55 MB:
|- native-lisp/
   |- 28.0.91-59ee6325/
   |- Some .eln files
      |- preloaded/
         | 45 MB .eln files

Might the differences come from multiple compilations?

Thanks
        Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Emanuel Berg via Users list for the GNU Emacs text editor]

H. Dieter Wilhelm wrote:

> machine 105 MB:
> |- native-lisp/
>    |- 28.0.91-59ee6325/
>    |  |- Some .eln files
>    |  |- preloaded/
>    |     | 45 MB .eln
>    |- 28.0.91-aa40abb8/
>       | Some other .eln files
>       |- preloaded/
>          |- 45 Mb .eln files
>
> machine 55 MB:
> |- native-lisp/
>    |- 28.0.91-59ee6325/
>    |- Some .eln files
>       |- preloaded/
>          | 45 MB .eln files

I thought we were talking about the eln-cache directory.

Here, native-lisp as in

  /usr/local/lib/emacs/29.0.50/native-lisp/29.0.50-9e08bfb0

is 13M.

-- 
underground experts united
https://dataswamp.org/~incal

Re: limit native compilation - how

[H. Dieter Wilhelm]

Emanuel Berg via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> H. Dieter Wilhelm wrote:
>
>> machine 105 MB:
>> |- native-lisp/
>>    |- 28.0.91-59ee6325/
>>    |  |- Some .eln files
>>    |  |- preloaded/
>>    |     | 45 MB .eln
>>    |- 28.0.91-aa40abb8/
>>       | Some other .eln files
>>       |- preloaded/
>>          |- 45 Mb .eln files
>>
>> machine 55 MB:
>> |- native-lisp/
>>    |- 28.0.91-59ee6325/
>>    |- Some .eln files
>>       |- preloaded/
>>          | 45 MB .eln files
>
> I thought we were talking about the eln-cache directory.

On Windows I haven't seen yet the cache..

> Here, native-lisp as in
>
>   /usr/local/lib/emacs/29.0.50/native-lisp/29.0.50-9e08bfb0
>
> is 13M.

Thanks, Eli suggested that compiling with debug symbols on Windows might
bloat the size compared to Gnu-Linux.


        Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Eli Zaretskii]

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Date: Sun, 16 Jan 2022 14:52:48 +0100
> 
> > I thought we were talking about the eln-cache directory.
> 
> On Windows I haven't seen yet the cache..

The cache isn't created until you actually run the Emacs you've built.

Re: limit native compilation - how

[H. Dieter Wilhelm]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>> Date: Sun, 16 Jan 2022 14:52:48 +0100
>> 
>> > I thought we were talking about the eln-cache directory.
>> 
>> On Windows I haven't seen yet the cache..
>
> The cache isn't created until you actually run the Emacs you've built.

All right, didn't know that its location is under ~/.emacs.d/.

After using Emacs, with slight delays, for example when opening dired,
the cache is filling up.  The used machine is really, really slow, need
to investigate further.

Thanks

        Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Eli Zaretskii]

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Cc: help-gnu-emacs@gnu.org
> Date: Sun, 16 Jan 2022 19:37:50 +0100
> 
> After using Emacs, with slight delays, for example when opening dired,
> the cache is filling up.  The used machine is really, really slow, need
> to investigate further.

This typically happens during the first few minutes of a session, and
then subsides.  It's JIT compilation in action, and is normal.  The
JIT compilation uses half the execution cores of the machine.

Re: limit native compilation - how

[H. Dieter Wilhelm]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>> Cc: help-gnu-emacs@gnu.org
>> Date: Sun, 16 Jan 2022 19:37:50 +0100
>> 
>> After using Emacs, with slight delays, for example when opening dired,
>> the cache is filling up.  The used machine is really, really slow, need
>> to investigate further.
>
> This typically happens during the first few minutes of a session, and

For a session only?!

I hoped that the cache (or the .eln files) can last a live time of an
Emacs build!  Except maybe when packages will be updated.

I just updated some Elpa packages and it seems they won't be natively
compiled at this stage.

> then subsides.  It's JIT compilation in action, and is normal.  The
> JIT compilation uses half the execution cores of the machine.

Half the cores, I see, thanks

  Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Eli Zaretskii]

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Cc: help-gnu-emacs@gnu.org
> Date: Mon, 17 Jan 2022 14:38:46 +0100
> 
> > This typically happens during the first few minutes of a session, and
> 
> For a session only?!

The first session (assuming your use patterns are repeatable enough).

> I hoped that the cache (or the .eln files) can last a live time of an
> Emacs build!  Except maybe when packages will be updated.

That's what I meant, except that I was talking about the bundled
packages.

> I just updated some Elpa packages and it seems they won't be natively
> compiled at this stage.

They will be compiled when Emacs loads them for the first time,
assuming that they were byte-compiled into *.elc.

> > then subsides.  It's JIT compilation in action, and is normal.  The
> > JIT compilation uses half the execution cores of the machine.
> 
> Half the cores, I see, thanks

(And of course there's a defcustom to tune this.)

Re: limit native compilation - how

[H. Dieter Wilhelm]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>> Cc: help-gnu-emacs@gnu.org
>> Date: Mon, 17 Jan 2022 14:38:46 +0100
>> 
>> > This typically happens during the first few minutes of a session, and
>> 
>> For a session only?!
>
> The first session (assuming your use patterns are repeatable enough).
>
>> I hoped that the cache (or the .eln files) can last a live time of an
>> Emacs build!  Except maybe when packages will be updated.
>
> That's what I meant, except that I was talking about the bundled
> packages.

All right, I felt it was 70 % clear but I wanted it 100 %, thank you!

>> I just updated some Elpa packages and it seems they won't be natively
>> compiled at this stage.
>
> They will be compiled when Emacs loads them for the first time,
> assuming that they were byte-compiled into *.elc.

Micromanagement alert:

I'm wondering if it would be more fluid to compile packages natively
during installation and updating?  The user expects then a delay,
anyway, and might be spared small "jerkings" during normal operation..

>> > then subsides.  It's JIT compilation in action, and is normal.  The
>> > JIT compilation uses half the execution cores of the machine.
>> 
>> Half the cores, I see, thanks
>
> (And of course there's a defcustom to tune this.)

  :-)    

-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Corwin Brust]

Howdy,

On Mon, Jan 17, 2022, 10:14 H. Dieter Wilhelm <dieter@duenenhof-wilhelm.de>
wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> >> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> >> Cc: help-gnu-emacs@gnu.org
> >> Date: Mon, 17 Jan 2022 14:38:46 +0100
> >>
> >> > This typically happens during the first few minutes of a session, and
> >>
> >> For a session only?!
> >
> > The first session (assuming your use patterns are repeatable enough).
> >
> >> I hoped that the cache (or the .eln files) can last a live time of an
> >> Emacs build!  Except maybe when packages will be updated.
> >
> > That's what I meant, except that I was talking about the bundled
> > packages.
>
> All right, I felt it was 70 % clear but I wanted it 100 %, thank you!
>
> >> I just updated some Elpa packages and it seems they won't be natively
> >> compiled at this stage.
> >
> > They will be compiled when Emacs loads them for the first time,
> > assuming that they were byte-compiled into *.elc.
>
> Micromanagement alert:
>
> I'm wondering if it would be more fluid to compile packages natively
> during installation and updating?  The user expects then a delay,
> anyway, and might be spared small "jerkings" during normal operation..
>

You can pass make an argument to cause this.  This has been my usual
routine for the last several months. (IIRC) NATIVE_FULL_AOT=1


> >> > then subsides.  It's JIT compilation in action, and is normal.  The
> >> > JIT compilation uses half the execution cores of the machine.
> >>
> >> Half the cores, I see, thanks
> >
> > (And of course there's a defcustom to tune this.)
>

I have been away on vacation this weekend, returning home in a few hours.
I plan to rebuild emacs-28 with the recently added patches when I have
access to my desktop system again.

Just before leaving I was able to successfully sign my packages.  (I didn't
find it very difficult to perform the signing under windows.)

Perhaps it is time to discuss getting the two of us access to ftp up our
work.



>   :-)
>
> --
> Best wishes
> H. Dieter Wilhelm
> Zwingenberg, Germany
>
>

Re: limit native compilation - how

[H. Dieter Wilhelm]

Hi Corwin

Corwin Brust <corwin@bru.st> writes:
> Howdy,
I'm very fine, hope you too.

>  I'm wondering if it would be more fluid to compile packages natively
>  during installation and updating?  The user expects then a delay,
>  anyway, and might be spared small "jerkings" during normal operation..
>
> You can pass make an argument to cause this.  This has been my usual
> routine for the last several months. (IIRC) NATIVE_FULL_AOT=1

Thanks, I've read about this option but I can't find where to specify
it?

> ..
>
> I have been away on vacation this weekend, returning home in a few
> hours.  I plan to rebuild emacs-28 with the recently added patches
> when I have access to my desktop system again.

And how are you bringing the native-lisp/ folder into the Windows
package?

> Just before leaving I was able to successfully sign my packages.  (I
> didn't find it very difficult to perform the signing under windows.)

Very good!  Please show me how..

> Perhaps it is time to discuss getting the two of us access to ftp up
> our work.

I'm afraid I'm not yet so far. :-/ But please go for it!

    Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Corwin Brust]

On Mon, Jan 17, 2022, 13:04 H. Dieter Wilhelm <dieter@duenenhof-wilhelm.de>
wrote:

>
> Hi Corwin
>
> Corwin Brust <corwin@bru.st> writes:
> > Howdy,
> I'm very fine, hope you too.
>

Getting there. 🙂

>
> >  I'm wondering if it would be more fluid to compile packages natively
> >  during installation and updating?  The user expects then a delay,
> >  anyway, and might be spared small "jerkings" during normal operation..
> >
> > You can pass make an argument to cause this.  This has been my usual
> > routine for the last several months. (IIRC) NATIVE_FULL_AOT=1
>
> Thanks, I've read about this option but I can't find where to specify
> it?
>

I add it too the end of make, so after running configure.  E.g make -j20
NATIVE_FULL_AOT=1

I will reply again when we reach home, maybe another 30m, and share my full
recipes as well as reply to the bits of your last note I don't address here.

Very glad to be able to 'tag team' all this with you Dieter!

(Looking forward to using more than my thumbs to reply.)

>
>

Re: limit native compilation - how

[Corwin Brust]

Home, safe and sound!

On Mon, Jan 17, 2022 at 1:04 PM H. Dieter Wilhelm
<dieter@duenenhof-wilhelm.de> wrote:
> Corwin Brust <corwin@bru.st> writes:
> >  I'm wondering if it would be more fluid to compile packages natively
> >  during installation and updating?  The user expects then a delay,
> >  anyway, and might be spared small "jerkings" during normal operation..
> >
> > You can pass make an argument to cause this.  This has been my usual
> > routine for the last several months. (IIRC) NATIVE_FULL_AOT=1
>
> Thanks, I've read about this option but I can't find where to specify
> it?

Here's the "usual" recipe I've been using.  Sometimes (when I screw up
my build tree) I use "make maintainer-clean" instead of "make clean".

git pull && make clean && ./autogen.sh && ./configure
--with-native-compilation && make -j 20 NATIVE_FULL_AOT=1

> > I have been away on vacation this weekend, returning home in a few
> > hours.  I plan to rebuild emacs-28 with the recently added patches
> > when I have access to my desktop system again.
>
> And how are you bringing the native-lisp/ folder into the Windows
> package?

This is what I haven't figured out yet.

I wonder if we need to ask Andrea about this:

I don't know if it is it safe to simply include the native-lisp folder
or if this would causes issues for Windows users who do have libgccjib
but don't have the same CPU as I do.   (Locally I have: Intel(R)
Core(TM) i7-6950X CPU @ 3.00GHz   3.00 GHz)   I don't know if the eln
files are CPU/arch-specific or if this should work fine for (e.g.) i9,
AMD chips, those with newer chipsets than my x99, and so forth.

> > Just before leaving I was able to successfully sign my packages.  (I
> > didn't find it very difficult to perform the signing under windows.)
>
> Very good!  Please show me how..

Given a working install of GNU PGP for Windows (which I already had)
it was quite easy:

Ensure your gpg is working at that you have your keys setup:

  # gpg --list-keys

And, given that works correctly, in ~/emacs-upload use something like:

  # for f in *.zip ; do gpg -b $f ; done

From there I believe we just need to create the directive files.
Regarding which, see:

  https://www.gnu.org/prep/maintain/html_node/FTP-Upload-Directive-File.html#FTP-Upload-Directive-File

> > Perhaps it is time to discuss getting the two of us access to ftp up
> > our work.
>
> I'm afraid I'm not yet so far. :-/ But please go for it!

OK.  I will create a new thread with Eli to discuss.  I will put you
in copy (as I assume you are not far behind :)

Warm regards,
Corwin

Re: limit native compilation - how

[Eli Zaretskii]

> From: Corwin Brust <corwin@bru.st>
> Date: Mon, 17 Jan 2022 15:54:55 -0600
> Cc: Eli Zaretskii <eliz@gnu.org>, Help Gnu Emacs mailing list <help-gnu-emacs@gnu.org>
> 
> I don't know if it is it safe to simply include the native-lisp folder
> or if this would causes issues for Windows users who do have libgccjib
> but don't have the same CPU as I do.   (Locally I have: Intel(R)
> Core(TM) i7-6950X CPU @ 3.00GHz   3.00 GHz)   I don't know if the eln
> files are CPU/arch-specific or if this should work fine for (e.g.) i9,
> AMD chips, those with newer chipsets than my x99, and so forth.

This is not about the CPU you have, this is about the CPU-specific
code-tuning options you pass to libgccjit when you natively-compile
the *.el files.  Since I find it hard to believe you tweaked
native-comp-compiler-options to pass non-default options to the
compiler, the resulting code should be generic enough to run on any
64-bit machine running Windows (if you build for x86_64) or on any
machine running any version of Windows if you build for x86.

Again, this is the theory.  But here we have at leas partial proof
from the fact that Emacs itself runs on all those other systems, since
emacs.exe is compiled and linked by the same utilities used to produce
the *.eln files.

Re: limit native compilation - how

[H. Dieter Wilhelm]

Corwin Brust <corwin@bru.st> writes:

> Home, safe and sound!
Nice to read. :-)

> Here's the "usual" recipe I've been using.  Sometimes (when I screw up
> my build tree) I use "make maintainer-clean" instead of "make clean".
>
> git pull && make clean && ./autogen.sh && ./configure
> --with-native-compilation && make -j 20 NATIVE_FULL_AOT=1

Thanks, that's really helpful for me, but I wanted to inspect your
recipes for doing distros! :-)

> Given a working install of GNU PGP for Windows (which I already had)
> it was quite easy:

There's gpg on MSYS2, my key ID is
8027EC54B495EC88959B97B7B63FB3E2D6A2F7B8, we could sign our respective
keys, if that's helpful..

> Ensure your gpg is working at that you have your keys setup:
>   # gpg --list-keys
> And, given that works correctly, in ~/emacs-upload use something like:
>   # for f in *.zip ; do gpg -b $f ; done
> From there I believe we just need to create the directive files.
> Regarding which, see:
>
>   https://www.gnu.org/prep/maintain/html_node/FTP-Upload-Directive-File.html#FTP-Upload-Directive-File

Oh, what's a directive file?  I'll need to check the documentation..

> OK.  I will create a new thread with Eli to discuss.  I will put you
> in copy (as I assume you are not far behind :)

Great, please do so.

       Dieter

-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Eli Zaretskii]

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Cc: help-gnu-emacs@gnu.org
> Date: Mon, 17 Jan 2022 17:11:02 +0100
> 
> I'm wondering if it would be more fluid to compile packages natively
> during installation and updating?  The user expects then a delay,
> anyway, and might be spared small "jerkings" during normal operation..

That's not an Emacs problem, that's a package.el problem -- and maybe
not even that.  As long as we don't endorse NATIVE_FULL_AOT=1 as the
default, it makes little sense to me to natively-compile installed
packages before they are actually used.

Re: limit native compilation - how

[Stefan Monnier via Users list for the GNU Emacs text editor]

> I'm wondering if it would be more fluid to compile packages natively
> during installation and updating?  The user expects then a delay,
> anyway, and might be spared small "jerkings" during normal operation..

The native-compiled files will only work with the current Emacs
executable, so those packages's .elc files will need to be recompiled
when you upgrade to a new Emacs.  Hence the current behavior of "compile
.elc to native code upon first use" is more or less indispensable.

We could additionally eagerly native-compile the .el files when the
package is installed, of course, and that should even be easy to do via
some hook (or at worst advice) without changing `package.el`.
We'll see how it goes.


        Stefan

Re: limit native compilation - how

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier via Users list for the GNU Emacs text editor wrote:

> The native-compiled files will only work with the current
> Emacs executable, so those packages's .elc files will need
> to be recompiled when you upgrade to a new Emacs. Hence the
> current behavior of "compile .elc to native code upon first
> use" is more or less indispensable.
>
> We could additionally eagerly native-compile the .el files
> when the package is installed, of course, and that should
> even be easy to do via some hook (or at worst advice)
> without changing `package.el`. We'll see how it goes.

Very interesting, this!

-- 
underground experts united
https://dataswamp.org/~incal

Re: limit native compilation - how

[Emanuel Berg via Users list for the GNU Emacs text editor]

Eli Zaretskii wrote:

>>> I thought we were talking about the eln-cache directory.
>> 
>> On Windows I haven't seen yet the cache..
>
> The cache isn't created until you actually run the Emacs
> you've built.

In general, a computer cache, a part of the memory hierarchy,
is populated and works like that ...

Why is this called that, because it is populated when possible
on `load', or does it behave like a cache later on in some
other sense as well?

If you are just to read onee text on Emacs native-compile,
what text would that be?

-- 
underground experts united
https://dataswamp.org/~incal

Re: limit native compilation - how

[Emanuel Berg via Users list for the GNU Emacs text editor]

> In general, a computer cache, a part of the memory
> hierarchy, is populated and works like that ...

https://dataswamp.org/~incal/hs-linux/docs/report/figures/llc/llc.pdf

Proximity in time and space ...

Up - fast, expensive, small

Down - slow, cheap, big

-- 
underground experts united
https://dataswamp.org/~incal

Re: limit native compilation - how

[Michael Heerdegen]

Emanuel Berg via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> If you are just to read onee text on Emacs native-compile, what text
> would that be?

Not a text - but I enjoyed this very informative talk given at the Emacs
Conf:

  https://emacsconf.org/2021/talks/native/

Very interesting, includes some demonstrations and also answers a lot of
questions.

Michael.

Re: limit native compilation - how

[Eli Zaretskii]

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Cc: help-gnu-emacs@gnu.org
> Date: Sun, 16 Jan 2022 14:24:52 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > The contents of the native-lisp/ directory doesn't change once Emacs
> > is built.
> 
> Really, please see below, could it be you meant perhaps the preloaded/
> folder? (Just guessing.)

No, I meant the entire native-lisp/28.0.91-xxxxx directory (of which
there should be just one when you build a single version for a single
configuration).

> 
> > If you cannot figure this out, compare the contents of the two
> > native-lisp/ directories.
> 
> machine 105 MB:
> |- native-lisp/
>    |- 28.0.91-59ee6325/
>    |  |- Some .eln files
>    |  |- preloaded/
>    |     | 45 MB .eln
>    |- 28.0.91-aa40abb8/
>       | Some other .eln files
>       |- preloaded/
>          |- 45 Mb .eln files
> 
> machine 55 MB:
> |- native-lisp/
>    |- 28.0.91-59ee6325/
>    |- Some .eln files
>       |- preloaded/
>          | 45 MB .eln files
> 
> Might the differences come from multiple compilations?

Yes, you have two different builds in the first case.  I don't know
how that happened, assuming you built the pretest tarball.  Did you
build 2 different configurations in that tree, perhaps?

Re: limit native compilation - how

[Andy Moreton]

On Sun 16 Jan 2022, H. Dieter Wilhelm wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>>> Cc: help-gnu-emacs@gnu.org
>>> Date: Sun, 16 Jan 2022 13:01:37 +0100
>>> 
>>> Could the difference in size result from opening my Emacs, I think I
>>> checked Emacs-28.0.91 on the machine 106MB?  On machine 55MB I didn't
>>> run Emacs yet.
>>
>> The contents of the native-lisp/ directory doesn't change once Emacs
>> is built.
>
> Really, please see below, could it be you meant perhaps the preloaded/
> folder? (Just guessing.)
>
>> If you cannot figure this out, compare the contents of the two
>> native-lisp/ directories.
>
> machine 105 MB:
> |- native-lisp/
>    |- 28.0.91-59ee6325/
>    |  |- Some .eln files
>    |  |- preloaded/
>    |     | 45 MB .eln
>    |- 28.0.91-aa40abb8/
>       | Some other .eln files
>       |- preloaded/
>          |- 45 Mb .eln files

This shows native builds for two emacs versions with different native
compilation hash signatures (which are used to ensure that the .eln
DLLs are compatible with the emacs executable that they are loaded
into).

The variable `comp-native-version-dir' contains the matching name for a
running emacs with native comilation support.

> machine 55 MB:
> |- native-lisp/
>    |- 28.0.91-59ee6325/
>    |- Some .eln files
>       |- preloaded/
>          | 45 MB .eln files
>
> Might the differences come from multiple compilations?

Yes.

    AndyM

Re: limit native compilation - how

[H. Dieter Wilhelm]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> On Sun 16 Jan 2022, H. Dieter Wilhelm wrote:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>>> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>>>> Cc: help-gnu-emacs@gnu.org
>>>> Date: Sun, 16 Jan 2022 13:01:37 +0100
>>>> 
>>>> Could the difference in size result from opening my Emacs, I think I
>>>> checked Emacs-28.0.91 on the machine 106MB?  On machine 55MB I didn't
>>>> run Emacs yet.
>>>
>>> The contents of the native-lisp/ directory doesn't change once Emacs
>>> is built.
>>
>> Really, please see below, could it be you meant perhaps the preloaded/
>> folder? (Just guessing.)
>>
>>> If you cannot figure this out, compare the contents of the two
>>> native-lisp/ directories.
>>
>> machine 105 MB:
>> |- native-lisp/
>>    |- 28.0.91-59ee6325/
>>    |  |- Some .eln files
>>    |  |- preloaded/
>>    |     | 45 MB .eln
>>    |- 28.0.91-aa40abb8/
>>       | Some other .eln files
>>       |- preloaded/
>>          |- 45 Mb .eln files
>
> This shows native builds for two emacs versions with different native
> compilation hash signatures (which are used to ensure that the .eln
> DLLs are compatible with the emacs executable that they are loaded
> into).

As far as I remember, I updated with pacman - after the first
compilation - the MSYS/MinGW toolchain and wanted to see if it would
make a difference to the native compilation.

> The variable `comp-native-version-dir' contains the matching name for a
> running emacs with native comilation support.

I see, thank you for the information!

  Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany

Re: limit native compilation - how

[Colin Baxter 😺]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

    >> From: Colin Baxter 😺 <m43cap@yandex.com> Date: Sat, 15 Jan 2022
    >> 15:42:34 +0000
    >> 
    >> 
    >> I want to limit native compilation to emacs' own libraries and
    >> not include any of my own lisp files. I tried a .dir-locals.el
    >> file
    >> 
    >> --8<---------------cut here---------------start------------->8---
    >> ((emacs-lisp-mode .  ((eval: (setq no-native-compile t)))))
    >> --8<---------------cut here---------------end--------------->8---
    >> 
    >> in the directory I want native compilation to avoid. That didn't
    >> work. Perhaps I can't see the wood for the trees, but I don't
    >> understand what's wrong with the .dir-locals. Can anyone help?

    > Did you try to customize
    > native-comp-deferred-compilation-deny-list?

Ah ... no. Thanks.

Best wishes,

Re: limit native compilation - how

[Emanuel Berg via Users list for the GNU Emacs text editor]

Colin Baxter wrote:

> I want to limit native compilation to emacs' own libraries
> and not include any of my own lisp files. I tried
> a .dir-locals.el file [...]
>
> in the directory I want native compilation to avoid.
> That didn't work. Perhaps I can't see the wood for the
> trees, but I don't understand what's wrong with the
> .dir-locals. Can anyone help?

If you byte-compile them they will be natively-compiled if the
.elc files end up in the same directory as the original
.el files.

See what you got in 

  ~/.emacs.d/eln-cache/

-- 
underground experts united
https://dataswamp.org/~incal