unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Bootstrap Compilation Speed
@ 2022-01-14 11:36 phillip.lord
  2022-01-14 13:21 ` Eli Zaretskii
                   ` (2 more replies)
  0 siblings, 3 replies; 42+ messages in thread
From: phillip.lord @ 2022-01-14 11:36 UTC (permalink / raw)
  To: emacs-devel



Has anyone else seen a regression in bootstrap compilation speed?

I am currently building Emacs-28 trunk. I am still on the 
bootstrap-emacs phase after half an hour or so and am getting no more 
than 2 or 3 ELC+ELN steps complete per minute, with a parallel build.

I haven't been doing bootstrap builds consistently, so I do not know if 
this has changed recently or whether I am just seeing it for the first 
time, so this is not a very consistent "bug report", rather just to see 
if anyone else is seeing something similar.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Bootstrap Compilation Speed
  2022-01-14 11:36 Bootstrap Compilation Speed phillip.lord
@ 2022-01-14 13:21 ` Eli Zaretskii
  2022-01-14 14:16   ` phillip.lord
  2022-01-15  9:34 ` Eli Zaretskii
  2022-01-15 10:29 ` H. Dieter Wilhelm
  2 siblings, 1 reply; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-14 13:21 UTC (permalink / raw)
  To: phillip.lord; +Cc: emacs-devel

> Date: Fri, 14 Jan 2022 11:36:26 +0000
> From: phillip.lord@russet.org.uk
> 
> Has anyone else seen a regression in bootstrap compilation speed?
> 
> I am currently building Emacs-28 trunk. I am still on the 
> bootstrap-emacs phase after half an hour or so and am getting no more 
> than 2 or 3 ELC+ELN steps complete per minute, with a parallel build.

How many parallel jobs did you allow Make to have?  And on what OS is
this?

Also, do you see this slow compilation only for the first few files,
before comp.el is natively-compiled, or even after that?



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Bootstrap Compilation Speed
  2022-01-14 13:21 ` Eli Zaretskii
@ 2022-01-14 14:16   ` phillip.lord
  0 siblings, 0 replies; 42+ messages in thread
From: phillip.lord @ 2022-01-14 14:16 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

On 2022-01-14 13:21, Eli Zaretskii wrote:
>> Date: Fri, 14 Jan 2022 11:36:26 +0000
>> From: phillip.lord@russet.org.uk
>> 
>> Has anyone else seen a regression in bootstrap compilation speed?
>> 
>> I am currently building Emacs-28 trunk. I am still on the
>> bootstrap-emacs phase after half an hour or so and am getting no more
>> than 2 or 3 ELC+ELN steps complete per minute, with a parallel build.
> 
> How many parallel jobs did you allow Make to have?  And on what OS is
> this?
> 
> Also, do you see this slow compilation only for the first few files,
> before comp.el is natively-compiled, or even after that?


Ubuntu 20.04. I tried both single threaded (so I could see how long 
individual files take) and my normal build (-j 20). Again, based on my 
ad hoc experience rather than any careful measurement, it was for all 
the elc+eln using bootstrap-emacs. Once emacs proper was generated, 
things went rather more quickly and upto my expectations.

It's probably something local to me. I thought to mention it here 
because I think we still do not have good metrics set up for bootstrap 
build speed performance.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Bootstrap Compilation Speed
  2022-01-14 11:36 Bootstrap Compilation Speed phillip.lord
  2022-01-14 13:21 ` Eli Zaretskii
@ 2022-01-15  9:34 ` Eli Zaretskii
  2022-01-15 11:29   ` phillip.lord
  2022-01-15 10:29 ` H. Dieter Wilhelm
  2 siblings, 1 reply; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-15  9:34 UTC (permalink / raw)
  To: phillip.lord; +Cc: emacs-devel

> Date: Fri, 14 Jan 2022 11:36:26 +0000
> From: phillip.lord@russet.org.uk
> 
> Has anyone else seen a regression in bootstrap compilation speed?
> 
> I am currently building Emacs-28 trunk. I am still on the 
> bootstrap-emacs phase after half an hour or so and am getting no more 
> than 2 or 3 ELC+ELN steps complete per minute, with a parallel build.

With "make -j4" recompiling natively all the preloaded files and their
dependencies on the emacs-28 branch, I see this:

  . compiling the 16 COMPILE_FIRST files takes 22 min, with comp.el
    being the worst offender: about 20 minutes
  . compiling all the rest of the preloaded files takes an additional
    25 minutes

If someone can compare this with past versions, maybe we could see if
the slowdown is real.

Thanks.



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Bootstrap Compilation Speed
  2022-01-14 11:36 Bootstrap Compilation Speed phillip.lord
  2022-01-14 13:21 ` Eli Zaretskii
  2022-01-15  9:34 ` Eli Zaretskii
@ 2022-01-15 10:29 ` H. Dieter Wilhelm
  2022-01-15 11:28   ` phillip.lord
  2 siblings, 1 reply; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-15 10:29 UTC (permalink / raw)
  To: phillip.lord; +Cc: emacs-devel

Hello Phil :-)

phillip.lord@russet.org.uk writes:

> Has anyone else seen a regression in bootstrap compilation speed?
>
> I am currently building Emacs-28 trunk. I am still on the
> bootstrap-emacs phase after half an hour or so and am getting no more
> than 2 or 3 ELC+ELN steps complete per minute, with a parallel build.

Are you doing these builds for Windows?

    Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Bootstrap Compilation Speed
  2022-01-15 10:29 ` H. Dieter Wilhelm
@ 2022-01-15 11:28   ` phillip.lord
  2022-01-15 16:24     ` Corwin Brust
  0 siblings, 1 reply; 42+ messages in thread
From: phillip.lord @ 2022-01-15 11:28 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: emacs-devel

On 2022-01-15 10:29, H. Dieter Wilhelm wrote:
> Hello Phil :-)
> 
> phillip.lord@russet.org.uk writes:
> 
>> Has anyone else seen a regression in bootstrap compilation speed?
>> 
>> I am currently building Emacs-28 trunk. I am still on the
>> bootstrap-emacs phase after half an hour or so and am getting no more
>> than 2 or 3 ELC+ELN steps complete per minute, with a parallel build.
> 
> Are you doing these builds for Windows?
> 


No, I am afraid not.

The performance that I got on Windows builds when I did do it was highly 
variable (between half an hour and three), but I think this was mostly 
the shared CPU infrastructure I was using. I never worked out how to get 
native comp working in a way that I could package it for windows, so the 
builds were all without it.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Bootstrap Compilation Speed
  2022-01-15  9:34 ` Eli Zaretskii
@ 2022-01-15 11:29   ` phillip.lord
  0 siblings, 0 replies; 42+ messages in thread
From: phillip.lord @ 2022-01-15 11:29 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: emacs-devel

On 2022-01-15 09:34, Eli Zaretskii wrote:
>> Date: Fri, 14 Jan 2022 11:36:26 +0000
>> From: phillip.lord@russet.org.uk
>> 
>> Has anyone else seen a regression in bootstrap compilation speed?
>> 
>> I am currently building Emacs-28 trunk. I am still on the
>> bootstrap-emacs phase after half an hour or so and am getting no more
>> than 2 or 3 ELC+ELN steps complete per minute, with a parallel build.
> 
> With "make -j4" recompiling natively all the preloaded files and their
> dependencies on the emacs-28 branch, I see this:
> 
>   . compiling the 16 COMPILE_FIRST files takes 22 min, with comp.el
>     being the worst offender: about 20 minutes
>   . compiling all the rest of the preloaded files takes an additional
>     25 minutes
> 
> If someone can compare this with past versions, maybe we could see if
> the slowdown is real.
> 
> Thanks.

Okay, that's similar to what I am seeing.

I can probably do a little testing over the weekend (while my computer 
is otherwise inactive) to see if it has slowed down.

Phil




^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Bootstrap Compilation Speed
  2022-01-15 11:28   ` phillip.lord
@ 2022-01-15 16:24     ` Corwin Brust
  2022-01-15 19:06       ` Native compilation on Windows, was " H. Dieter Wilhelm
  2022-01-16 14:31       ` H. Dieter Wilhelm
  0 siblings, 2 replies; 42+ messages in thread
From: Corwin Brust @ 2022-01-15 16:24 UTC (permalink / raw)
  To: Phillip Lord; +Cc: H. Dieter Wilhelm, Emacs developers

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

Hi :)

On Sat, Jan 15, 2022, 05:32 <phillip.lord@russet.org.uk> wrote:

> On 2022-01-15 10:29, H. Dieter Wilhelm wrote:
> > Hello Phil :-)
> >
> > phillip.lord@russet.org.uk writes:
> >
>
> >
> > Are you doing these builds for Windows?
> >
>
>
> No, I am afraid not.
>
> The performance that I got on Windows builds when I did do it was highly
> variable (between half an hour and three), but I think this was mostly
> the shared CPU infrastructure I was using. I never worked out how to get
> native comp working in a way that I could package it for windows, so the
> builds were all without it.
>

This is my present situation also.  I can build with native comp but not
(yet) found a way to package that version.

Regards,
Corwin

>

[-- Attachment #2: Type: text/html, Size: 1463 bytes --]

^ permalink raw reply	[flat|nested] 42+ messages in thread

* Native compilation on Windows,  was Re: Bootstrap Compilation Speed
  2022-01-15 16:24     ` Corwin Brust
@ 2022-01-15 19:06       ` H. Dieter Wilhelm
  2022-01-15 19:15         ` Eli Zaretskii
  2022-01-15 22:14         ` phillip.lord
  2022-01-16 14:31       ` H. Dieter Wilhelm
  1 sibling, 2 replies; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-15 19:06 UTC (permalink / raw)
  To: Corwin Brust; +Cc: Emacs developers, Phillip Lord

Corwin Brust <corwin@bru.st> writes:
>  > phillip.lord@russet.org.uk writes:
>  The performance that I got on Windows builds when I did do it was highly 
>  variable (between half an hour and three), but I think this was mostly 
>  the shared CPU infrastructure I was using. I never worked out how to get 
>  native comp working in a way that I could package it for windows, so the 
>  builds were all without it.
>
> This is my present situation also.  I can build with native comp but
> not (yet) found a way to package that version.

I don't understand yet the packaging requirements, is it not possible to
copy additionally the native-lisp/ folder to the package?

So far I could compile (locally) the current pretest 28.0.91 version
--with-native-compilation.

Then I tried to build (with build.zips.sh) a snapshot of the
masterbranch --with-native-compilation on the same system.  And now
./configure is complaining:

  configure: error: Elisp native compiler was requested, but libgccjit was not found.

Does Emacs-29 has different requirements?

   Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-15 19:06       ` Native compilation on Windows, was " H. Dieter Wilhelm
@ 2022-01-15 19:15         ` Eli Zaretskii
  2022-01-15 19:27           ` H. Dieter Wilhelm
  2022-01-15 22:14         ` phillip.lord
  1 sibling, 1 reply; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-15 19:15 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: corwin, phillip.lord, emacs-devel

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Date: Sat, 15 Jan 2022 20:06:18 +0100
> Cc: Emacs developers <emacs-devel@gnu.org>,
>  Phillip Lord <phillip.lord@russet.org.uk>
> 
> Then I tried to build (with build.zips.sh) a snapshot of the
> masterbranch --with-native-compilation on the same system.

I'm not familiar with that script, so I don't think I understand the
implications.

> And now ./configure is complaining:
> 
>   configure: error: Elisp native compiler was requested, but libgccjit was not found.
> 
> Does Emacs-29 has different requirements?

No.



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-15 19:15         ` Eli Zaretskii
@ 2022-01-15 19:27           ` H. Dieter Wilhelm
  2022-01-16 11:47             ` H. Dieter Wilhelm
  0 siblings, 1 reply; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-15 19:27 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: corwin, emacs-devel, phillip.lord

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
>> Date: Sat, 15 Jan 2022 20:06:18 +0100

>> Then I tried to build (with build.zips.sh) a snapshot of the
>> masterbranch --with-native-compilation on the same system.
>
> I'm not familiar with that script, so I don't think I understand the
> implications.

Sorry, I meant the script admin/nt/dist-build/build-zips.sh

   $ build-zips.sh -s

>> And now ./configure is complaining:
>>   configure: error: Elisp native compiler was requested, but libgccjit was not found.
>> Does Emacs-29 has different requirements?
>
> No.

OK thank you, for the moment I'll focus on emacs-28

    Dieter

-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows,  was Re: Bootstrap Compilation Speed
  2022-01-15 19:06       ` Native compilation on Windows, was " H. Dieter Wilhelm
  2022-01-15 19:15         ` Eli Zaretskii
@ 2022-01-15 22:14         ` phillip.lord
  2022-01-15 22:32           ` H. Dieter Wilhelm
  2022-01-16  8:54           ` Eli Zaretskii
  1 sibling, 2 replies; 42+ messages in thread
From: phillip.lord @ 2022-01-15 22:14 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: Corwin Brust, Emacs developers

On 2022-01-15 19:06, H. Dieter Wilhelm wrote:
> Corwin Brust <corwin@bru.st> writes:
>>  > phillip.lord@russet.org.uk writes:
>>  The performance that I got on Windows builds when I did do it was 
>> highly
>>  variable (between half an hour and three), but I think this was 
>> mostly
>>  the shared CPU infrastructure I was using. I never worked out how to 
>> get
>>  native comp working in a way that I could package it for windows, so 
>> the
>>  builds were all without it.
>> 
>> This is my present situation also.  I can build with native comp but
>> not (yet) found a way to package that version.
> 
> I don't understand yet the packaging requirements, is it not possible 
> to
> copy additionally the native-lisp/ folder to the package?
> 
> So far I could compile (locally) the current pretest 28.0.91 version
> --with-native-compilation.
> 
> Then I tried to build (with build.zips.sh) a snapshot of the
> masterbranch --with-native-compilation on the same system.  And now
> ./configure is complaining:
> 
>   configure: error: Elisp native compiler was requested, but libgccjit
> was not found.
> 
> Does Emacs-29 has different requirements?


The problem is that native comp needs libgccjit at compile time but also 
at
runtime. To make an executable install package like we have at the 
moment
essentially means packaging libgccjit. Which means gcc and half of the 
msys2
toolchain.

We were there a long time back when Emacs used to be packaged with lots 
of other
stuff, including a full python installation. It didn't make sense then 
and it's
not a great thing to go back to.

So, I think, Emacs releases for windows will be without native-comp. 
Anyone who
wants it will have to install it over an msys2 installation, or with 
chocolaty
or something equivalent.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows,  was Re: Bootstrap Compilation Speed
  2022-01-15 22:14         ` phillip.lord
@ 2022-01-15 22:32           ` H. Dieter Wilhelm
  2022-01-16  8:54           ` Eli Zaretskii
  1 sibling, 0 replies; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-15 22:32 UTC (permalink / raw)
  To: phillip.lord; +Cc: Corwin Brust, Emacs developers

phillip.lord@russet.org.uk writes:
> The problem is that native comp needs libgccjit at compile time but
> also at runtime. To make an executable install package like we have at
> the moment essentially means packaging libgccjit. Which means gcc and
> half of the msys2 toolchain.

As far as I understood Eli it should be possible to build Emacs on
Windows --with-native-compilation and distribute it even on systems
without libgccjit.  Only Emacs internal lisp files will then be natively
compiled and if, for other lisp packages, there are no .eln files
available Emacs will just load the respective .elc files.

Do you think I'm mistaken?

    Dieter

-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows,  was Re: Bootstrap Compilation Speed
  2022-01-15 22:14         ` phillip.lord
  2022-01-15 22:32           ` H. Dieter Wilhelm
@ 2022-01-16  8:54           ` Eli Zaretskii
  2022-01-17  9:34             ` Andrea Corallo
  2022-01-17  9:56             ` Phillip Lord
  1 sibling, 2 replies; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-16  8:54 UTC (permalink / raw)
  To: phillip.lord; +Cc: dieter, corwin, emacs-devel

> Date: Sat, 15 Jan 2022 22:14:56 +0000
> From: phillip.lord@russet.org.uk
> Cc: Corwin Brust <corwin@bru.st>, Emacs developers <emacs-devel@gnu.org>
> 
> The problem is that native comp needs libgccjit at compile time but
> also at runtime. To make an executable install package like we have
> at the moment essentially means packaging libgccjit. Which means gcc
> and half of the msys2 toolchain.

We made a point of implementing native-compilation on MS-Windows in a
way that doesn't require such extreme measures.  Emacs built with
native-compilation support should be able to run on a Windows system
without GCC and Binutils installed, it just won't be able to
natively-compile any new or modified Lisp files.  It will use the
*.elc files instead.

Or at least that is the theory.



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-15 19:27           ` H. Dieter Wilhelm
@ 2022-01-16 11:47             ` H. Dieter Wilhelm
  2022-01-16 12:13               ` Eli Zaretskii
  2022-01-17  9:44               ` Phillip Lord
  0 siblings, 2 replies; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-16 11:47 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: corwin, phillip.lord, emacs-devel

"H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:
> Sorry, I meant the script admin/nt/dist-build/build-zips.sh

I could build Gnu-Emacs-28.0.91 when I used the archived sources

  $ ./configure --with-native-compilation

The difference to the build-zips.sh seems to me some more options from
the script (and that is build from the git tree):

        echo [build] Configuring Emacs
        $REPO_DIR/$BRANCH/configure \
            --without-dbus \
            --without-compress-install \
            --with-native-compilation \
            $CACHE \
            CFLAGS="$CFLAGS"

Could it be that these additional options are spoiling the build?
(CACHE=-C , I think.)

      Dieter

-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-16 11:47             ` H. Dieter Wilhelm
@ 2022-01-16 12:13               ` Eli Zaretskii
  2022-01-17  9:39                 ` Phillip Lord
  2022-01-17  9:44               ` Phillip Lord
  1 sibling, 1 reply; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-16 12:13 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: corwin, phillip.lord, emacs-devel

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Cc: corwin@bru.st,  emacs-devel@gnu.org,  phillip.lord@russet.org.uk
> Date: Sun, 16 Jan 2022 12:47:24 +0100
> 
> "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:
> > Sorry, I meant the script admin/nt/dist-build/build-zips.sh
> 
> I could build Gnu-Emacs-28.0.91 when I used the archived sources
> 
>   $ ./configure --with-native-compilation
> 
> The difference to the build-zips.sh seems to me some more options from
> the script (and that is build from the git tree):
> 
>         echo [build] Configuring Emacs
>         $REPO_DIR/$BRANCH/configure \
>             --without-dbus \
>             --without-compress-install \
>             --with-native-compilation \
>             $CACHE \
>             CFLAGS="$CFLAGS"
> 
> Could it be that these additional options are spoiling the build?
> (CACHE=-C , I think.)

I'd drop the CACHE part, so that the configure script starts from a
clean state.



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Bootstrap Compilation Speed
  2022-01-15 16:24     ` Corwin Brust
  2022-01-15 19:06       ` Native compilation on Windows, was " H. Dieter Wilhelm
@ 2022-01-16 14:31       ` H. Dieter Wilhelm
  1 sibling, 0 replies; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-16 14:31 UTC (permalink / raw)
  To: Corwin Brust; +Cc: Emacs developers, Phillip Lord


Hello Corwin,

some progress on my side.. :-)

Corwin Brust <corwin@bru.st> writes:
> On Sat, Jan 15, 2022, 05:32 <phillip.lord@russet.org.uk> wrote:
>  On 2022-01-15 10:29, H. Dieter Wilhelm wrote:
>  > Are you doing these builds for Windows?

>  No, I am afraid not.
>
>  The performance that I got on Windows builds when I did do it was highly 
>  variable (between half an hour and three), but I think this was mostly 
>  the shared CPU infrastructure I was using. I never worked out how to get 
>  native comp working in a way that I could package it for windows, so the 
>  builds were all without it.
> ...
> Corwin:

> This is my present situation also.  I can build with native comp but
> not (yet) found a way to package that version.

When I'm removing all the configure options and include
--with-native-compilation in build-zips.sh then - at last - the build
process is creating .eln files...

Wouldn't it be sufficient to copy the native-lisp/ folder into the
distro tree?

     Dieter

-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows,  was Re: Bootstrap Compilation Speed
  2022-01-16  8:54           ` Eli Zaretskii
@ 2022-01-17  9:34             ` Andrea Corallo
  2022-01-17  9:56             ` Phillip Lord
  1 sibling, 0 replies; 42+ messages in thread
From: Andrea Corallo @ 2022-01-17  9:34 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: dieter, corwin, emacs-devel, phillip.lord

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sat, 15 Jan 2022 22:14:56 +0000
>> From: phillip.lord@russet.org.uk
>> Cc: Corwin Brust <corwin@bru.st>, Emacs developers <emacs-devel@gnu.org>
>> 
>> The problem is that native comp needs libgccjit at compile time but
>> also at runtime. To make an executable install package like we have
>> at the moment essentially means packaging libgccjit. Which means gcc
>> and half of the msys2 toolchain.
>
> We made a point of implementing native-compilation on MS-Windows in a
> way that doesn't require such extreme measures.  Emacs built with
> native-compilation support should be able to run on a Windows system
> without GCC and Binutils installed, it just won't be able to
> natively-compile any new or modified Lisp files.  It will use the
> *.elc files instead.
>
> Or at least that is the theory.

Agree, before we rely on this behavior would be nice if someone could
test it tho.

Best Regards

  Andrea



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-16 12:13               ` Eli Zaretskii
@ 2022-01-17  9:39                 ` Phillip Lord
  2022-01-17 12:35                   ` Eli Zaretskii
  0 siblings, 1 reply; 42+ messages in thread
From: Phillip Lord @ 2022-01-17  9:39 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: H. Dieter Wilhelm, corwin, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> I could build Gnu-Emacs-28.0.91 when I used the archived sources
>> 
>>   $ ./configure --with-native-compilation
>> 
>> The difference to the build-zips.sh seems to me some more options from
>> the script (and that is build from the git tree):
>> 
>>         echo [build] Configuring Emacs
>>         $REPO_DIR/$BRANCH/configure \
>>             --without-dbus \
>>             --without-compress-install \
>>             --with-native-compilation \
>>             $CACHE \
>>             CFLAGS="$CFLAGS"
>> 
>> Could it be that these additional options are spoiling the build?
>> (CACHE=-C , I think.)
>
> I'd drop the CACHE part, so that the configure script starts from a
> clean state.

I wouldn't!

CACHE=-C is used only on a snapshot build. These were always built
incrementally from the previous version because I judged that the risk
of a broken build was small compared to the significant time and cost
saving that the incremental build provided. A full build is easy to
achieve by simply deleting the build tree, as build-zips does an
out-of-tree. Reconfiguring requires deleting the config.cache as well.

Full release builds are built from a clean tree. The time saving is less
important there.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-16 11:47             ` H. Dieter Wilhelm
  2022-01-16 12:13               ` Eli Zaretskii
@ 2022-01-17  9:44               ` Phillip Lord
  2022-01-17 16:08                 ` Óscar Fuentes
  1 sibling, 1 reply; 42+ messages in thread
From: Phillip Lord @ 2022-01-17  9:44 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: Eli Zaretskii, corwin, emacs-devel

"H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:

> "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:
>> Sorry, I meant the script admin/nt/dist-build/build-zips.sh
>
> I could build Gnu-Emacs-28.0.91 when I used the archived sources
>
>   $ ./configure --with-native-compilation
>
> The difference to the build-zips.sh seems to me some more options from
> the script (and that is build from the git tree):
>
>         echo [build] Configuring Emacs
>         $REPO_DIR/$BRANCH/configure \
>             --without-dbus \
>             --without-compress-install \
>             --with-native-compilation \
>             $CACHE \
>             CFLAGS="$CFLAGS"
>
> Could it be that these additional options are spoiling the build?
> (CACHE=-C , I think.)


--without-compress-install switches off the gzipping of various files,
such as the info files. It was necessary, if I remember correctly, when
I added it because Emacs uses external calls to gzip which wasn't
necessarily bundled.

I do not remember why I added --without-dbus.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows,  was Re: Bootstrap Compilation Speed
  2022-01-16  8:54           ` Eli Zaretskii
  2022-01-17  9:34             ` Andrea Corallo
@ 2022-01-17  9:56             ` Phillip Lord
  2022-01-17 12:37               ` Eli Zaretskii
  1 sibling, 1 reply; 42+ messages in thread
From: Phillip Lord @ 2022-01-17  9:56 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: dieter, corwin, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sat, 15 Jan 2022 22:14:56 +0000
>> From: phillip.lord@russet.org.uk
>> Cc: Corwin Brust <corwin@bru.st>, Emacs developers <emacs-devel@gnu.org>
>> 
>> The problem is that native comp needs libgccjit at compile time but
>> also at runtime. To make an executable install package like we have
>> at the moment essentially means packaging libgccjit. Which means gcc
>> and half of the msys2 toolchain.
>
> We made a point of implementing native-compilation on MS-Windows in a
> way that doesn't require such extreme measures.  Emacs built with
> native-compilation support should be able to run on a Windows system
> without GCC and Binutils installed, it just won't be able to
> natively-compile any new or modified Lisp files.  It will use the
> *.elc files instead.
>
> Or at least that is the theory.


Ah, okay. I remember asking about this some time back when I got the
impression this would not work. It's a little bit late in the release
process to be testing the theory now. The last snapshot is a year old,
and that is not native comp.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17  9:39                 ` Phillip Lord
@ 2022-01-17 12:35                   ` Eli Zaretskii
  2022-01-17 14:07                     ` Phillip Lord
  0 siblings, 1 reply; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-17 12:35 UTC (permalink / raw)
  To: Phillip Lord; +Cc: dieter, corwin, emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>,  corwin@bru.st,
>   emacs-devel@gnu.org
> Date: Mon, 17 Jan 2022 09:39:34 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I could build Gnu-Emacs-28.0.91 when I used the archived sources
> >> 
> >>   $ ./configure --with-native-compilation
> >> 
> >> The difference to the build-zips.sh seems to me some more options from
> >> the script (and that is build from the git tree):
> >> 
> >>         echo [build] Configuring Emacs
> >>         $REPO_DIR/$BRANCH/configure \
> >>             --without-dbus \
> >>             --without-compress-install \
> >>             --with-native-compilation \
> >>             $CACHE \
> >>             CFLAGS="$CFLAGS"
> >> 
> >> Could it be that these additional options are spoiling the build?
> >> (CACHE=-C , I think.)
> >
> > I'd drop the CACHE part, so that the configure script starts from a
> > clean state.
> 
> I wouldn't!
> 
> CACHE=-C is used only on a snapshot build. These were always built
> incrementally from the previous version because I judged that the risk
> of a broken build was small compared to the significant time and cost
> saving that the incremental build provided.

AFAIU, Dieter said he did more than one build in the same tree for
several different configurations, so I suggested to eliminate the
cache as a possible factor for the problems he saw in the second
build.  Apologies if I misunderstood.



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows,  was Re: Bootstrap Compilation Speed
  2022-01-17  9:56             ` Phillip Lord
@ 2022-01-17 12:37               ` Eli Zaretskii
  0 siblings, 0 replies; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-17 12:37 UTC (permalink / raw)
  To: Phillip Lord; +Cc: dieter, corwin, emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: dieter@duenenhof-wilhelm.de,  corwin@bru.st,  emacs-devel@gnu.org
> Date: Mon, 17 Jan 2022 09:56:44 +0000
> 
> I remember asking about this some time back when I got the
> impression this would not work.

AFAIR, it did work in my (very limited) testing.

> It's a little bit late in the release process to be testing the
> theory now. The last snapshot is a year old, and that is not native
> comp.

I don't see how we could do better, given the (un)availability of
snapshots and pretest binaries for Windows.  Certainly, delaying Emacs
28.1 release until this is tested well enough would be a worse
alternative.



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 12:35                   ` Eli Zaretskii
@ 2022-01-17 14:07                     ` Phillip Lord
  2022-01-17 14:09                       ` Eli Zaretskii
  2022-01-17 16:25                       ` H. Dieter Wilhelm
  0 siblings, 2 replies; 42+ messages in thread
From: Phillip Lord @ 2022-01-17 14:07 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: dieter, corwin, emacs-devel

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>,  corwin@bru.st,
>>   emacs-devel@gnu.org
>> Date: Mon, 17 Jan 2022 09:39:34 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> I could build Gnu-Emacs-28.0.91 when I used the archived sources
>> >> 
>> >>   $ ./configure --with-native-compilation
>> >> 
>> >> The difference to the build-zips.sh seems to me some more options from
>> >> the script (and that is build from the git tree):
>> >> 
>> >>         echo [build] Configuring Emacs
>> >>         $REPO_DIR/$BRANCH/configure \
>> >>             --without-dbus \
>> >>             --without-compress-install \
>> >>             --with-native-compilation \
>> >>             $CACHE \
>> >>             CFLAGS="$CFLAGS"
>> >> 
>> >> Could it be that these additional options are spoiling the build?
>> >> (CACHE=-C , I think.)
>> >
>> > I'd drop the CACHE part, so that the configure script starts from a
>> > clean state.
>> 
>> I wouldn't!
>> 
>> CACHE=-C is used only on a snapshot build. These were always built
>> incrementally from the previous version because I judged that the risk
>> of a broken build was small compared to the significant time and cost
>> saving that the incremental build provided.
>
> AFAIU, Dieter said he did more than one build in the same tree for
> several different configurations, so I suggested to eliminate the
> cache as a possible factor for the problems he saw in the second
> build.  Apologies if I misunderstood.


Oh, sure, that's fine. Just saying that the `CACHE=-C` is there for
mostly good reasons!

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 14:07                     ` Phillip Lord
@ 2022-01-17 14:09                       ` Eli Zaretskii
  2022-01-17 16:25                       ` H. Dieter Wilhelm
  1 sibling, 0 replies; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-17 14:09 UTC (permalink / raw)
  To: Phillip Lord; +Cc: dieter, corwin, emacs-devel

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: dieter@duenenhof-wilhelm.de,  corwin@bru.st,  emacs-devel@gnu.org
> Date: Mon, 17 Jan 2022 14:07:29 +0000
> 
> >> CACHE=-C is used only on a snapshot build. These were always built
> >> incrementally from the previous version because I judged that the risk
> >> of a broken build was small compared to the significant time and cost
> >> saving that the incremental build provided.
> >
> > AFAIU, Dieter said he did more than one build in the same tree for
> > several different configurations, so I suggested to eliminate the
> > cache as a possible factor for the problems he saw in the second
> > build.  Apologies if I misunderstood.
> 
> 
> Oh, sure, that's fine. Just saying that the `CACHE=-C` is there for
> mostly good reasons!

Of course.  I'm a happy user of -C myself.



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17  9:44               ` Phillip Lord
@ 2022-01-17 16:08                 ` Óscar Fuentes
  2022-01-17 17:23                   ` H. Dieter Wilhelm
  2022-01-17 20:21                   ` Phillip Lord
  0 siblings, 2 replies; 42+ messages in thread
From: Óscar Fuentes @ 2022-01-17 16:08 UTC (permalink / raw)
  To: emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> --without-compress-install switches off the gzipping of various files,
> such as the info files. It was necessary, if I remember correctly, when
> I added it because Emacs uses external calls to gzip which wasn't
> necessarily bundled.

Isn't the built-in zip support used for uncompressing things (.el.gz and
.info.gz) when gzip is missing? IIRC that was recently added.

> I do not remember why I added --without-dbus.

Probably because the dbus libraries might be present on the system and
Emacs would pick up them, which is not a good thing.




^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 14:07                     ` Phillip Lord
  2022-01-17 14:09                       ` Eli Zaretskii
@ 2022-01-17 16:25                       ` H. Dieter Wilhelm
  2022-01-17 18:07                         ` Corwin Brust
                                           ` (2 more replies)
  1 sibling, 3 replies; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-17 16:25 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Eli Zaretskii, corwin, emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Phillip Lord <phillip.lord@russet.org.uk>
>>> Cc: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>,  corwin@bru.st,
>>>   emacs-devel@gnu.org
>>> Date: Mon, 17 Jan 2022 09:39:34 +0000
>>> 
>>> Eli Zaretskii <eliz@gnu.org> writes:
>>> 
>>> >> I could build Gnu-Emacs-28.0.91 when I used the archived sources
>>> >> 
>>> >>   $ ./configure --with-native-compilation
>>> >> 
>>> >> The difference to the build-zips.sh seems to me some more options from
>>> >> the script (and that is build from the git tree):
>>> >> 
>>> >>         echo [build] Configuring Emacs
>>> >>         $REPO_DIR/$BRANCH/configure \
>>> >>             --without-dbus \
>>> >>             --without-compress-install \
>>> >>             --with-native-compilation \
>>> >>             $CACHE \
>>> >>             CFLAGS="$CFLAGS"
>>> >> 
>>> >> Could it be that these additional options are spoiling the build?
>>> >> (CACHE=-C , I think.)
>>> >
>>> > I'd drop the CACHE part, so that the configure script starts from a
>>> > clean state.
>>> 
>>> I wouldn't!
>>> 
>>> CACHE=-C is used only on a snapshot build. These were always built
>>> incrementally from the previous version because I judged that the risk
>>> of a broken build was small compared to the significant time and cost
>>> saving that the incremental build provided.
>>
>> AFAIU, Dieter said he did more than one build in the same tree for
>> several different configurations, so I suggested to eliminate the
>> cache as a possible factor for the problems he saw in the second
>> build.  Apologies if I misunderstood.
>
>
> Oh, sure, that's fine. Just saying that the `CACHE=-C` is there for
> mostly good reasons!

I'm too inexperienced to judge the options.

But, for the moment, it seems to me when the $CACHE line is active then
there is NO native compilation going on for my builds.  (No idea why,
I'll try to confirm this.)

The current idea is to build 28.0.91 with build-zips.sh - in the usual
way but --with-native compilation - and then copy the lisp-native/
folder into the archive.  Then checking if such a build is running and
actually loading the .eln files on a Windows system without MSYS/MinGW.

Hmm, how can I test if Emacs is actually loading some .eln and not the
.elc files?

       Dieter


-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 16:08                 ` Óscar Fuentes
@ 2022-01-17 17:23                   ` H. Dieter Wilhelm
  2022-01-17 17:46                     ` Óscar Fuentes
  2022-01-17 20:21                   ` Phillip Lord
  1 sibling, 1 reply; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-17 17:23 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

Óscar Fuentes <ofv@wanadoo.es> writes:

> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> --without-compress-install switches off the gzipping of various files,
>> such as the info files. It was necessary, if I remember correctly, when
>> I added it because Emacs uses external calls to gzip which wasn't
>> necessarily bundled.
>
> Isn't the built-in zip support used for uncompressing things (.el.gz and
> .info.gz) when gzip is missing? IIRC that was recently added.

OK, I'll note this onto the todo list.

Thanks for the hint Óscar

       Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 17:23                   ` H. Dieter Wilhelm
@ 2022-01-17 17:46                     ` Óscar Fuentes
  2022-01-17 19:09                       ` H. Dieter Wilhelm
  0 siblings, 1 reply; 42+ messages in thread
From: Óscar Fuentes @ 2022-01-17 17:46 UTC (permalink / raw)
  To: emacs-devel

"H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:

>> Isn't the built-in zip support used for uncompressing things (.el.gz and
>> .info.gz) when gzip is missing? IIRC that was recently added.
>
> OK, I'll note this onto the todo list.

That was a genuine question. I'm not sure about the status of that
feature.




^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 16:25                       ` H. Dieter Wilhelm
@ 2022-01-17 18:07                         ` Corwin Brust
  2022-01-17 18:13                         ` Eli Zaretskii
  2022-01-17 20:25                         ` Phillip Lord
  2 siblings, 0 replies; 42+ messages in thread
From: Corwin Brust @ 2022-01-17 18:07 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: Eli Zaretskii, Emacs developers, Phillip Lord

[-- Attachment #1: Type: text/plain, Size: 496 bytes --]

On Mon, Jan 17, 2022, 10:25 H. Dieter Wilhelm <dieter@duenenhof-wilhelm.de>
wrote:

>
> Hmm, how can I test if Emacs is actually loading some .eln and not the
> .elc files?
>


I have been using describe-function (C-h f) for this  It can tell us if a
given command has been natively compiled (or byte-compiled or neither of
these) for the present session.

Aside: I would first check `native-comp-available-p' to confirm the build
has the native compiler and that the Emacs instance is using it.

[-- Attachment #2: Type: text/html, Size: 938 bytes --]

^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 16:25                       ` H. Dieter Wilhelm
  2022-01-17 18:07                         ` Corwin Brust
@ 2022-01-17 18:13                         ` Eli Zaretskii
  2022-01-17 20:25                         ` Phillip Lord
  2 siblings, 0 replies; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-17 18:13 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: corwin, emacs-devel, phillip.lord

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Date: Mon, 17 Jan 2022 17:25:00 +0100
> Cc: Eli Zaretskii <eliz@gnu.org>, corwin@bru.st, emacs-devel@gnu.org
> 
> Hmm, how can I test if Emacs is actually loading some .eln and not the
> .elc files?

Start Process Explorer, and look at the DLLs loaded by Emacs: you
should see there all the *.eln files Emacs has currently loaded into
the session.



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 17:46                     ` Óscar Fuentes
@ 2022-01-17 19:09                       ` H. Dieter Wilhelm
  0 siblings, 0 replies; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-17 19:09 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

Óscar Fuentes <ofv@wanadoo.es> writes:

> "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:
>
>>> Isn't the built-in zip support used for uncompressing things (.el.gz and
>>> .info.gz) when gzip is missing? IIRC that was recently added.
>>
>> OK, I'll note this onto the todo list.
>
> That was a genuine question. I'm not sure about the status of that
> feature.

No problem, I meant to check if this feature is working or not. :-)

   Dieter

-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 16:08                 ` Óscar Fuentes
  2022-01-17 17:23                   ` H. Dieter Wilhelm
@ 2022-01-17 20:21                   ` Phillip Lord
  2022-01-18  9:03                     ` Robert Pluim
  1 sibling, 1 reply; 42+ messages in thread
From: Phillip Lord @ 2022-01-17 20:21 UTC (permalink / raw)
  To: Óscar Fuentes; +Cc: emacs-devel

Óscar Fuentes <ofv@wanadoo.es> writes:

> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> --without-compress-install switches off the gzipping of various files,
>> such as the info files. It was necessary, if I remember correctly, when
>> I added it because Emacs uses external calls to gzip which wasn't
>> necessarily bundled.
>
> Isn't the built-in zip support used for uncompressing things (.el.gz and
> .info.gz) when gzip is missing? IIRC that was recently added.


I haven't tried! If it does do that now then this switch needs
removing. It didn't when I added it many years back.


>> I do not remember why I added --without-dbus.
>
> Probably because the dbus libraries might be present on the system and
> Emacs would pick up them, which is not a good thing.

Glad I go something right!

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 16:25                       ` H. Dieter Wilhelm
  2022-01-17 18:07                         ` Corwin Brust
  2022-01-17 18:13                         ` Eli Zaretskii
@ 2022-01-17 20:25                         ` Phillip Lord
  2022-01-17 20:46                           ` H. Dieter Wilhelm
  2 siblings, 1 reply; 42+ messages in thread
From: Phillip Lord @ 2022-01-17 20:25 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: Eli Zaretskii, corwin, emacs-devel

"H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:

>>>> I wouldn't!
>>>> 
>>>> CACHE=-C is used only on a snapshot build. These were always built
>>>> incrementally from the previous version because I judged that the risk
>>>> of a broken build was small compared to the significant time and cost
>>>> saving that the incremental build provided.
>>>
>>> AFAIU, Dieter said he did more than one build in the same tree for
>>> several different configurations, so I suggested to eliminate the
>>> cache as a possible factor for the problems he saw in the second
>>> build.  Apologies if I misunderstood.
>>
>>
>> Oh, sure, that's fine. Just saying that the `CACHE=-C` is there for
>> mostly good reasons!
>
> I'm too inexperienced to judge the options.
>
> But, for the moment, it seems to me when the $CACHE line is active then
> there is NO native compilation going on for my builds.  (No idea why,
> I'll try to confirm this.)
>
> The current idea is to build 28.0.91 with build-zips.sh - in the usual
> way but --with-native compilation - and then copy the lisp-native/
> folder into the archive.  Then checking if such a build is running and
> actually loading the .eln files on a Windows system without MSYS/MinGW.
>
> Hmm, how can I test if Emacs is actually loading some .eln and not the
> .elc files?


If you change the configure options in build-zips.sh, I would go for a
full clean build for sure. "make extraclean" I think will do the trick,
then run build-zips.sh with modified options.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 20:25                         ` Phillip Lord
@ 2022-01-17 20:46                           ` H. Dieter Wilhelm
  2022-01-18 16:38                             ` Phillip Lord
  0 siblings, 1 reply; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-17 20:46 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Eli Zaretskii, corwin, emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:

>> The current idea is to build 28.0.91 with build-zips.sh - in the usual
>> way but --with-native compilation - and then copy the lisp-native/
>> folder into the archive.  Then checking if such a build is running and
>> actually loading the .eln files on a Windows system without MSYS/MinGW.
>>
>> Hmm, how can I test if Emacs is actually loading some .eln and not the
>> .elc files?
>
>
> If you change the configure options in build-zips.sh, I would go for a
> full clean build for sure. "make extraclean" I think will do the trick,
> then run build-zips.sh with modified options.

Thank you Phil

And when we are at it, please tell me your recipe how to build a
snapshot of the master branch.  I think my build tree is damaged, I
don't manage to build a snapshot of emacs-29 with a date included..

Shouldn't it be just:

$ build-zips.sh -g
$ build-zips.sh -s

      Dieter

-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 20:21                   ` Phillip Lord
@ 2022-01-18  9:03                     ` Robert Pluim
  2022-01-18 14:50                       ` Eli Zaretskii
                                         ` (2 more replies)
  0 siblings, 3 replies; 42+ messages in thread
From: Robert Pluim @ 2022-01-18  9:03 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Óscar Fuentes, emacs-devel

>>>>> On Mon, 17 Jan 2022 20:21:53 +0000, Phillip Lord <phillip.lord@russet.org.uk> said:

    Phillip> Óscar Fuentes <ofv@wanadoo.es> writes:
    >> Phillip Lord <phillip.lord@russet.org.uk> writes:
    >> 
    >>> --without-compress-install switches off the gzipping of various files,
    >>> such as the info files. It was necessary, if I remember correctly, when
    >>> I added it because Emacs uses external calls to gzip which wasn't
    >>> necessarily bundled.
    >> 
    >> Isn't the built-in zip support used for uncompressing things (.el.gz and
    >> .info.gz) when gzip is missing? IIRC that was recently added.


    Phillip> I haven't tried! If it does do that now then this switch needs
    Phillip> removing. It didn't when I added it many years back.

I donʼt know if the builtin zip support works here, but itʼs moot:
zlib support is optional, so you canʼt rely on it always being
available (unless youʼre going to ensure that here).

Robert
-- 



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-18  9:03                     ` Robert Pluim
@ 2022-01-18 14:50                       ` Eli Zaretskii
  2022-01-18 15:09                       ` Óscar Fuentes
  2022-01-18 16:40                       ` Phillip Lord
  2 siblings, 0 replies; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-18 14:50 UTC (permalink / raw)
  To: Robert Pluim; +Cc: ofv, emacs-devel, phillip.lord

> From: Robert Pluim <rpluim@gmail.com>
> Gmane-Reply-To-List: yes
> Date: Tue, 18 Jan 2022 10:03:55 +0100
> Cc: Óscar Fuentes <ofv@wanadoo.es>, emacs-devel@gnu.org
> 
> >>>>> On Mon, 17 Jan 2022 20:21:53 +0000, Phillip Lord <phillip.lord@russet.org.uk> said:
> 
>     Phillip> Óscar Fuentes <ofv@wanadoo.es> writes:
>     >> Phillip Lord <phillip.lord@russet.org.uk> writes:
>     >> 
>     >>> --without-compress-install switches off the gzipping of various files,
>     >>> such as the info files. It was necessary, if I remember correctly, when
>     >>> I added it because Emacs uses external calls to gzip which wasn't
>     >>> necessarily bundled.
>     >> 
>     >> Isn't the built-in zip support used for uncompressing things (.el.gz and
>     >> .info.gz) when gzip is missing? IIRC that was recently added.
> 
> 
>     Phillip> I haven't tried! If it does do that now then this switch needs
>     Phillip> removing. It didn't when I added it many years back.
> 
> I donʼt know if the builtin zip support works here, but itʼs moot:
> zlib support is optional, so you canʼt rely on it always being
> available (unless youʼre going to ensure that here).

For Emacs with native compilation enabled, the zlib support takes
another role: if the *.el files are installed compressed, then Emacs
_must_ have zlib support, or else it will be unable to use the *.eln
files, because it must verify the *.el source files correspond to the
*.eln files.  So if you build an Emacs with native compilation, but
without zlib, do NOT compress the *.el file at "make install" time.



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-18  9:03                     ` Robert Pluim
  2022-01-18 14:50                       ` Eli Zaretskii
@ 2022-01-18 15:09                       ` Óscar Fuentes
  2022-01-18 16:40                       ` Phillip Lord
  2 siblings, 0 replies; 42+ messages in thread
From: Óscar Fuentes @ 2022-01-18 15:09 UTC (permalink / raw)
  To: emacs-devel

Robert Pluim <rpluim@gmail.com> writes:

>     >> Isn't the built-in zip support used for uncompressing things (.el.gz and
>     >> .info.gz) when gzip is missing? IIRC that was recently added.
>
>
>     Phillip> I haven't tried! If it does do that now then this switch needs
>     Phillip> removing. It didn't when I added it many years back.
>
> I donʼt know if the builtin zip support works here, but itʼs moot:
> zlib support is optional, so you canʼt rely on it always being
> available (unless youʼre going to ensure that here).

We are talking about a binary distribution. Its creator can ensure that
the required dependencies are included on the binary package.




^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-17 20:46                           ` H. Dieter Wilhelm
@ 2022-01-18 16:38                             ` Phillip Lord
  2022-01-18 19:37                               ` H. Dieter Wilhelm
  0 siblings, 1 reply; 42+ messages in thread
From: Phillip Lord @ 2022-01-18 16:38 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: Eli Zaretskii, corwin, emacs-devel

"H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:

>> If you change the configure options in build-zips.sh, I would go for a
>> full clean build for sure. "make extraclean" I think will do the trick,
>> then run build-zips.sh with modified options.
>
> Thank you Phil
>
> And when we are at it, please tell me your recipe how to build a
> snapshot of the master branch.  I think my build tree is damaged, I
> don't manage to build a snapshot of emacs-29 with a date included..
>
> Shouldn't it be just:
>
> $ build-zips.sh -g
> $ build-zips.sh -s
>


Yes, I think that should do it.

Then, the I build the source like so.

./make-dist --snapshot --no-check --no-changelog
rename .tar _`date +%Y-%m-%d`.tar emacs*.tar.gz
mv emacs-*.tar.gz ~/emacs-upload

I don't think I ever automated this bit.

The model of build-zips.sh is probably a bit complicated and could be
simplified. The addition of snapshots and the ability to build from a
branch makes it much more so and the documentation isn't always well
done. I have been trying to remember why I called the variable
"OF_VERSION" so, or what "OF" stands for, and have failed!

Apologies for this.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-18  9:03                     ` Robert Pluim
  2022-01-18 14:50                       ` Eli Zaretskii
  2022-01-18 15:09                       ` Óscar Fuentes
@ 2022-01-18 16:40                       ` Phillip Lord
  2 siblings, 0 replies; 42+ messages in thread
From: Phillip Lord @ 2022-01-18 16:40 UTC (permalink / raw)
  To: Robert Pluim; +Cc: Óscar Fuentes, emacs-devel

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Mon, 17 Jan 2022 20:21:53 +0000, Phillip Lord <phillip.lord@russet.org.uk> said:
>
>     Phillip> Óscar Fuentes <ofv@wanadoo.es> writes:
>     >> Phillip Lord <phillip.lord@russet.org.uk> writes:
>     >> 
>     >>> --without-compress-install switches off the gzipping of various files,
>     >>> such as the info files. It was necessary, if I remember correctly, when
>     >>> I added it because Emacs uses external calls to gzip which wasn't
>     >>> necessarily bundled.
>     >> 
>     >> Isn't the built-in zip support used for uncompressing things (.el.gz and
>     >> .info.gz) when gzip is missing? IIRC that was recently added.
>
>
>     Phillip> I haven't tried! If it does do that now then this switch needs
>     Phillip> removing. It didn't when I added it many years back.
>
> I donʼt know if the builtin zip support works here, but itʼs moot:
> zlib support is optional, so you canʼt rely on it always being
> available (unless youʼre going to ensure that here).

From the point of the windows package, it is under the control of the
packager. If builtin zlib support works on the compressed installed
files, when it can be guaranteed to be packaged.

Phil



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-18 16:38                             ` Phillip Lord
@ 2022-01-18 19:37                               ` H. Dieter Wilhelm
  2022-01-18 19:46                                 ` Eli Zaretskii
  0 siblings, 1 reply; 42+ messages in thread
From: H. Dieter Wilhelm @ 2022-01-18 19:37 UTC (permalink / raw)
  To: Phillip Lord; +Cc: Eli Zaretskii, corwin, emacs-devel

Phillip Lord <phillip.lord@russet.org.uk> writes:

> "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de> writes:
>
>>> If you change the configure options in build-zips.sh, I would go for a
>>> full clean build for sure. "make extraclean" I think will do the trick,
>>> then run build-zips.sh with modified options.
>>
>> Thank you Phil
>>
>> And when we are at it, please tell me your recipe how to build a
>> snapshot of the master branch.  I think my build tree is damaged, I
>> don't manage to build a snapshot of emacs-29 with a date included..
>>
>> Shouldn't it be just:
>>
>> $ build-zips.sh -g
>> $ build-zips.sh -s
>
> Yes, I think that should do it.
>
> Then, the I build the source like so.
>
> ./make-dist --snapshot --no-check --no-changelog
> rename .tar _`date +%Y-%m-%d`.tar emacs*.tar.gz
> mv emacs-*.tar.gz ~/emacs-upload
>
> I don't think I ever automated this bit.

Uups, I thought this - or similar - would be done by below part of
build-zips.sh?


    make -j 4 $INSTALL_TARGET \
         prefix=$HOME/emacs-build/install/emacs-$VERSION
    cd $HOME/emacs-build/install/emacs-$VERSION
    zip -r -9 emacs-$OF_VERSION-no-deps.zip *
    mv emacs-$OF_VERSION-no-deps.zip $HOME/emacs-upload

    if [ -z $SNAPSHOT ];
    then
        DEPS_FILE=$HOME/emacs-build/deps/emacs-$MAJOR_VERSION-deps.zip
    else
        ## Pick the most recent snapshot whatever that is
        DEPS_FILE=`ls $HOME/emacs-build/deps/emacs-$MAJOR_VERSION-*-deps.zip | tail -n 1`
    fi

    echo [build] Using $DEPS_FILE
    unzip -d bin $DEPS_FILE

    zip -r -9 emacs-$OF_VERSION.zip *
    mv emacs-$OF_VERSION.zip ~/emacs-upload

> The model of build-zips.sh is probably a bit complicated and could be
> simplified. The addition of snapshots and the ability to build from a
> branch makes it much more so and the documentation isn't always well
> done. I have been trying to remember why I called the variable
> "OF_VERSION" so, or what "OF" stands for, and have failed!

Not only above is confusing, but also why is

    INSTALL_TARGET="install-strip"

in the script?  (Back to pretest building) I've got an error for

$ build-zips.sh -V28

in the emacs-28 worktree, maybe, that is a wrong usage of the scrip?
Anyway, "install-strip" seems not to be a valid target for make!?

> Apologies for this.

No reason to apologise, it was working for you.  Rather, thank you for
your work. :-)

   Dieter
-- 
Best wishes
H. Dieter Wilhelm
Zwingenberg, Germany



^ permalink raw reply	[flat|nested] 42+ messages in thread

* Re: Native compilation on Windows, was Re: Bootstrap Compilation Speed
  2022-01-18 19:37                               ` H. Dieter Wilhelm
@ 2022-01-18 19:46                                 ` Eli Zaretskii
  0 siblings, 0 replies; 42+ messages in thread
From: Eli Zaretskii @ 2022-01-18 19:46 UTC (permalink / raw)
  To: H. Dieter Wilhelm; +Cc: corwin, emacs-devel, phillip.lord

> From: "H. Dieter Wilhelm" <dieter@duenenhof-wilhelm.de>
> Cc: Eli Zaretskii <eliz@gnu.org>,  corwin@bru.st,  emacs-devel@gnu.org
> Date: Tue, 18 Jan 2022 20:37:53 +0100
> 
> Anyway, "install-strip" seems not to be a valid target for make!?

It is not?  How do you see that?



^ permalink raw reply	[flat|nested] 42+ messages in thread

end of thread, other threads:[~2022-01-18 19:46 UTC | newest]

Thread overview: 42+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-14 11:36 Bootstrap Compilation Speed phillip.lord
2022-01-14 13:21 ` Eli Zaretskii
2022-01-14 14:16   ` phillip.lord
2022-01-15  9:34 ` Eli Zaretskii
2022-01-15 11:29   ` phillip.lord
2022-01-15 10:29 ` H. Dieter Wilhelm
2022-01-15 11:28   ` phillip.lord
2022-01-15 16:24     ` Corwin Brust
2022-01-15 19:06       ` Native compilation on Windows, was " H. Dieter Wilhelm
2022-01-15 19:15         ` Eli Zaretskii
2022-01-15 19:27           ` H. Dieter Wilhelm
2022-01-16 11:47             ` H. Dieter Wilhelm
2022-01-16 12:13               ` Eli Zaretskii
2022-01-17  9:39                 ` Phillip Lord
2022-01-17 12:35                   ` Eli Zaretskii
2022-01-17 14:07                     ` Phillip Lord
2022-01-17 14:09                       ` Eli Zaretskii
2022-01-17 16:25                       ` H. Dieter Wilhelm
2022-01-17 18:07                         ` Corwin Brust
2022-01-17 18:13                         ` Eli Zaretskii
2022-01-17 20:25                         ` Phillip Lord
2022-01-17 20:46                           ` H. Dieter Wilhelm
2022-01-18 16:38                             ` Phillip Lord
2022-01-18 19:37                               ` H. Dieter Wilhelm
2022-01-18 19:46                                 ` Eli Zaretskii
2022-01-17  9:44               ` Phillip Lord
2022-01-17 16:08                 ` Óscar Fuentes
2022-01-17 17:23                   ` H. Dieter Wilhelm
2022-01-17 17:46                     ` Óscar Fuentes
2022-01-17 19:09                       ` H. Dieter Wilhelm
2022-01-17 20:21                   ` Phillip Lord
2022-01-18  9:03                     ` Robert Pluim
2022-01-18 14:50                       ` Eli Zaretskii
2022-01-18 15:09                       ` Óscar Fuentes
2022-01-18 16:40                       ` Phillip Lord
2022-01-15 22:14         ` phillip.lord
2022-01-15 22:32           ` H. Dieter Wilhelm
2022-01-16  8:54           ` Eli Zaretskii
2022-01-17  9:34             ` Andrea Corallo
2022-01-17  9:56             ` Phillip Lord
2022-01-17 12:37               ` Eli Zaretskii
2022-01-16 14:31       ` H. Dieter Wilhelm

Code repositories for project(s) associated with this inbox:

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).