msys2 building issue

msys2 building issue

[Phillip Lord]

I'm struggling with an issue with the snapshot build for windows. I have
a small script that in turn runs admin/nt/dist-build/build-zips.sh. The
build times (based on when the output files are created) make no sense.

x86_64
no-deps zip    17min
Deps-zip       3min
installer      25min

i686
no deps zip    31min
deps zip       47min
installer      61min

So, the installer is really slow to build (probably because of the
compression I guess). But the second i686 step is really slow,
especially the building the zips and installer.

I don't think this is the Emacs build per se; it looks to me like a
memory leak; the build process just gets slower and slower. In fact,
I've had to reduce the parallalism of make or the whole process crashes
with a resource allocation error. But Process Manager reports that the
VM has memory left.

I ask about all this, because I have no idea how to debug this further,
and I thought someone might have an idea.


My launch script is below, and build-zips.sh is in the repo.


#!/bin/bash

set -o errexit

## List the ssh-keys -- this has the crashes the script if there are none
ssh-add -l

pushd master/admin/nt/dist-build

## Update repo
git pull

./build-zips.sh -sn | tee ../../../../build.log

popd

pushd master
## Make the source so we can distribute it, but don't update because
## we've build in a different directory. --no-changelog is also needed
## because otherwise we run make which runs configure
./make-dist --snapshot --no-check --no-changelog
rename .tar _`date +%Y-%m-%d`.tar emacs*.tar.gz
mv emacs-*.tar.gz ~/emacs-upload
popd

Re: msys2 building issue

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Tue, 27 Mar 2018 22:20:34 +0100
> 
> x86_64
> no-deps zip    17min
> Deps-zip       3min
> installer      25min
> 
> i686
> no deps zip    31min
> deps zip       47min
> installer      61min
> 
> So, the installer is really slow to build (probably because of the
> compression I guess). But the second i686 step is really slow,
> especially the building the zips and installer.

Does this happen only when building both the 64-bit and the 32-bit
zips?  What if you build only the 32-bit one -- do you get the same
slow times?

My next suspect is the way MSYS2 lets you build 32-bit binaries.  I
know nothing about that, all I see is that you set 2 environment
variables, but could that cause Bash or Make or some other component
to run out of memory due to a bug or a memory leak?

> I don't think this is the Emacs build per se; it looks to me like a
> memory leak; the build process just gets slower and slower. In fact,
> I've had to reduce the parallalism of make or the whole process crashes
> with a resource allocation error.

Which program reports the error?

> But Process Manager reports that the VM has memory left.

What does Task Manager say about memory consumption of the various
programs involved in the build, like Bash, Make, and Python?  Do you
see their memory growing to values that are unreasonably large?

The slow-down could be caused by memory consumption, so it's possible
the actual problem is with memory, not with slow builds.

Re: msys2 building issue

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Date: Tue, 27 Mar 2018 22:20:34 +0100
>> 
>> x86_64
>> no-deps zip    17min
>> Deps-zip       3min
>> installer      25min
>> 
>> i686
>> no deps zip    31min
>> deps zip       47min
>> installer      61min
>> 
>> So, the installer is really slow to build (probably because of the
>> compression I guess). But the second i686 step is really slow,
>> especially the building the zips and installer.
>
> Does this happen only when building both the 64-bit and the 32-bit
> zips?  What if you build only the 32-bit one -- do you get the same
> slow times?
>
> My next suspect is the way MSYS2 lets you build 32-bit binaries.  I
> know nothing about that, all I see is that you set 2 environment
> variables, but could that cause Bash or Make or some other component
> to run out of memory due to a bug or a memory leak?

I've tested this out since. And I am having a hard time making head nor
tail of it. Building on it's own, the i686 will build fast
enough. Building x86_64 sometimes give very inconsistent behaviour
between the two builds (with the second slower). I could, of course,
build the x86 and i686 separately, but I want to launch the build once
and let it finish.

I've had occasions when the build time goes up to many hours.

>> I don't think this is the Emacs build per se; it looks to me like a
>> memory leak; the build process just gets slower and slower. In fact,
>> I've had to reduce the parallalism of make or the whole process crashes
>> with a resource allocation error.
>
> Which program reports the error?
>
>> But Process Manager reports that the VM has memory left.
>
> What does Task Manager say about memory consumption of the various
> programs involved in the build, like Bash, Make, and Python?  Do you
> see their memory growing to values that are unreasonably large?
>
> The slow-down could be caused by memory consumption, so it's possible
> the actual problem is with memory, not with slow builds.

Actually, no Task Manager reports that CPU is 100%, memory is between 50
and 70%.

I'm coming to the conclusion that it's an oddity of the VM
infrastructure. I think it's going to be hard to test out further,
because the slow build makes experiments a bit of a PITA.

Never mind, even if it is slow it always seems to build now that I am
using only dual threaded build. I guess that is ultimately all I care
about.

Phil

Re: msys2 building issue

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: emacs-devel@gnu.org
> Date: Tue, 10 Apr 2018 13:00:20 +0100
> 
> I'm coming to the conclusion that it's an oddity of the VM
> infrastructure.

What is "VM" in this context?

Re: msys2 building issue

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Cc: emacs-devel@gnu.org
>> Date: Tue, 10 Apr 2018 19:52:09 +0100
>> 

For Emacs-25, windows was built on a physical (dual boot) machine, but
the window partition started to run out of space. I switched to an
externally hosted VM in November.

Generally, it's been a positive experience, except that the windows VM
is low-powered because I am too tight-fisted to pay for anything
faster. Perhaps this is the core of the issue.

Phil