More build times

More build times

[Lars Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 119 bytes --]

The last week, my build machine has been running a whole bunch of
builds, and here's a chart for the last few months:


[-- Attachment #2: Type: image/png, Size: 70049 bytes --]

[-- Attachment #3: Type: text/plain, Size: 43 bytes --]


What's odd are those peaks.  Let's zoom:


[-- Attachment #4: Type: image/png, Size: 64132 bytes --]

[-- Attachment #5: Type: text/plain, Size: 89 bytes --]


Some builds consistently take 40s longer to run.  And here's what's even
more bizarre:


[-- Attachment #6: Type: image/png, Size: 68965 bytes --]

[-- Attachment #7: Type: text/plain, Size: 1190 bytes --]


Those exact same builds take a lot less CPU to run.  So...  less CPU
usage; more real time?  It's bizarre, and perhaps it's just a measuring
artefact of some kind?

Anybody see the same phenomenon on their systems?  (These are clean
builds -- made from a fresh git clone.)

Here's the data in CSV form:

20220602,f71cfd3084,92.74,1101.11
20220603,8f279c8666,92.73,1099.27
20220604,6f69284ada,94.41,1096.24
20220605,1b8719835a,132.86,951.54
20220606,0a36671f41,92.30,1109.85
20220607,8c252e2326,93.21,1114.62
20220608,88b025f815,93.08,1116.36
20220609,d02c94090c,131.41,952.02
20220610,8ef3862fa0,92.34,1111.84
20220611,3247c31d67,91.62,1101.95
20220612,98365c7b1e,93.69,1090.07
20220613,72e0ef74d0,93.58,1114.55
20220614,5678829a62,92.87,1112.83
20220615,787c4ad8b0,94.85,1105.01
20220616,556c304007,93.11,1109.08
20220617,f419de6eca,132.49,957.99
20220618,0d103e6f79,89.38,1084.58
20220619,8b45e7c681,88.23,1074.94
20220620,3947037a33,89.66,1085.17
20220621,4ae315f7c3,89.31,1084.66
20220622,b1af8c2c00,68.61,700.41
20220623,b1af8c2c00,68.15,699.24
20220624,b1af8c2c00,68.16,700.60


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: More build times

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Anybody see the same phenomenon on their systems?  (These are clean
> builds -- made from a fresh git clone.)

It's a git artefact.  For instance, this is what "git log" says:

commit 61a312ba0cae10e8e19b7424540751a71d0170b1
Author: Po Lu <luangruo@yahoo.com>
Date:   Fri Jun 17 10:24:05 2022 +0800

    Update last event time during DND operations
    
    * lisp/x-dnd.el (x-dnd-handle-xdnd, x-dnd-handle-motif): Set
    last user time to provided timestamp.
    * src/xfns.c (Fx_display_last_user_time): New function.
    (syms_of_xfns): New defsubr.
    
    * src/xterm.c (x_set_last_user_time_from_lisp): New function.
    * src/xterm.h: Update prototypes.

commit f419de6eca4ca6a6d03db1eec4b9086a3d1e5b86
Author: Stefan Kangas <stefan@marxist.se>
Date:   Thu Jun 16 09:48:05 2022 +0200

    * lisp/textmodes/artist.el: Minor doc fixes.

No intervening commits or anything, but checking out f419de6eca and
61a312ba0c give you vastly different trees -- presumably because of
merges and the like (since f419de6eca4 originated in emacs-28).

Is there a way to make git cough up the commit on the master branch for
a specific date?  I.e., "what was the state on the master branch at
Fri Jun 17 06:30:35 2022 +0200"?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: More build times

[Tor Kringeland]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Is there a way to make git cough up the commit on the master branch for
> a specific date?  I.e., "what was the state on the master branch at
> Fri Jun 17 06:30:35 2022 +0200"?

Yes, someting like

  git rev-list -n 1 --before="Fri Jun 17 06:30:35 2022 +0200" master

should work.  And then pass the output to git checkout to check the
commit.

Re: More build times

[Lars Ingebrigtsen]

Tor Kringeland <tor.kringeland@ntnu.no> writes:

> Yes, someting like
>
>   git rev-list -n 1 --before="Fri Jun 17 06:30:35 2022 +0200" master
>
> should work.  And then pass the output to git checkout to check the
> commit.

Unfortunately not.  What I'm using is:

larsi@ns3206854:/mnt/ram/bench$ git rev-list -n 1 --before="Fri Jun 17 00:00:00 2022 +0200" HEAD
f419de6eca4ca6a6d03db1eec4b9086a3d1e5b86

Which gives me the commit that didn't exist in master at that point.  

larsi@ns3206854:/mnt/ram/bench$ git rev-list -n 1 --before="Fri Jun 17 00:00:00 2022 +0200" master
f419de6eca4ca6a6d03db1eec4b9086a3d1e5b86

And this seems to give me the same thing.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: More build times

[Andreas Schwab]

On Jun 25 2022, Lars Ingebrigtsen wrote:

> larsi@ns3206854:/mnt/ram/bench$ git rev-list -n 1 --before="Fri Jun 17 00:00:00 2022 +0200" HEAD
> f419de6eca4ca6a6d03db1eec4b9086a3d1e5b86
>
> Which gives me the commit that didn't exist in master at that point.  

But it is reachable from master.  That's what you have asked for.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: More build times

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 25 Jun 2022 15:44:02 +0200
> 
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
> > Anybody see the same phenomenon on their systems?  (These are clean
> > builds -- made from a fresh git clone.)
> 
> It's a git artefact.  For instance, this is what "git log" says:

You mean the build times include the time it took Git to checkout the
tree at a specific commit?  Because otherwise I cannot understand how
this:

> No intervening commits or anything, but checking out f419de6eca and
> 61a312ba0c give you vastly different trees -- presumably because of
> merges and the like (since f419de6eca4 originated in emacs-28).

could explain that the elapsed time increased whereas CPU time
decreased: if you get more files that look modified to Make, you
should see more CPU time invested into rebuilding.  Am I missing
something?

Re: More build times

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> You mean the build times include the time it took Git to checkout the
> tree at a specific commit?

No, it just measures the "make".

>> No intervening commits or anything, but checking out f419de6eca and
>> 61a312ba0c give you vastly different trees -- presumably because of
>> merges and the like (since f419de6eca4 originated in emacs-28).
>
> could explain that the elapsed time increased whereas CPU time
> decreased: if you get more files that look modified to Make, you
> should see more CPU time invested into rebuilding.  Am I missing
> something?

Yes.  "git checkout f419de6eca4ca6" (on the master branch) does not give
you the state on master branch at that time.  Try it yourself:

git checkout f419de6eca4ca6
...
HEAD is now at f419de6eca * lisp/textmodes/artist.el: Minor doc fixes.
larsi@xo:~/src/emacs/bench$ ls -l lisp/emacs-lisp/load*
ls: cannot access 'lisp/emacs-lisp/load*': No such file or directory

And that file does exist on master at that time.

git is so much fun.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: More build times

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Emacs-devel@gnu.org
> Date: Sat, 25 Jun 2022 16:44:55 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > You mean the build times include the time it took Git to checkout the
> > tree at a specific commit?
> 
> No, it just measures the "make".

Then how the Git issues you described can explain the oscillations of
the elapsed build time, whereby the elapsed time increases while CPU
time decreases?

> Yes.  "git checkout f419de6eca4ca6" (on the master branch) does not give
> you the state on master branch at that time.  Try it yourself:

How is that relevant to the build times?

Re: More build times

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Then how the Git issues you described can explain the oscillations of
> the elapsed build time, whereby the elapsed time increases while CPU
> time decreases?
>
>> Yes.  "git checkout f419de6eca4ca6" (on the master branch) does not give
>> you the state on master branch at that time.  Try it yourself:
>
> How is that relevant to the build times?

That was the cause of the oscillations.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: More build times

[Gregory Heytings]

[-- Attachment #1: Type: text/plain, Size: 2227 bytes --]


Here is the graph you wanted.

I built Emacs at 500 points, from 2018 to today, on an up-to-date and 
unloaded Debian bookworm computer, and took the average wall clock time of 
10 builds with "make -j20".

The build times are in general constant.  The few commits that change the 
build time significantly, figured by breaks of the red line in the graph, 
are:

97b7e58c4d improvement from 86s to 78s (9%) Stefan Monnier Try and fix the more obvious sources of bug#30635
d12e5d003d deterioration from 78s to 95s (22%) Daniel Colascione Add portable dumper
.......... gradual deterioration from 95s to 108s (14%) from January 2019 to February 2021
a350ae058c deterioration from 108s to 129s (19%) Stefan Monnier <monnier@iro.umontreal.ca> * lisp/emacs-lisp/cconv.el: Improve line-nb info of unused var warnings
fddd63f8b8 and b3d310fa96 deterioration from 129s to 140s (9%) Glenn Morris <rgm@gnu.org> Distribute the real source for some doc/misc manuals (bug#45143)
ff09851a1f improvement from 140s to 130s (9%) Stefan Monnier * lisp/emacs-lisp/macroexp.el: Rewrite the code warning about '(lambda ...)
1362a9fec4 improvement from 130s to 119s (8%) Mattias Engdegård Make lambda-lifting work again
.......... gradual deterioration from 119s to 126s (6%) from March 2021 to January 2022
fd31ef21c5 deterioration from 126s to 132s (5%) Alan Mackenzie Don't use 'load-read-function' in byte-compile-from-buffer
f687e62ac5 deterioration from 132s to 145s (10%) Alan Mackenzie Fix symbols with position appearing in the output of 'compile-defun'
6092ee1c3f improvement from 145s to 130s (10%) Alan Mackenzie Amend byte-run-strip-symbol-positions so that an unexec build builds
68cdb95019 deterioration from 130s to 136s (5%) Alan Mackenzie Restore call to byte-run-strip-symbol-positions in byte-compile-out
1d4e903417 improvement from 136s to 109s (20%) Lars Ingebrigtsen Speed up generation of loaddefs files
3e312d11ce improvement from 109s to 83s (24%) Lars Ingebrigtsen Reshuffle the generation of the OKURO-NASI entries to speed up build
73a384a986 improvement from 83s to 60s (28%) Lars Ingebrigtsen Increase gc-cons-percentage in -batch Emacs

Congrats for 1d4e903417, 3e312d11ce and 73a384a986!

[-- Attachment #2: emacs-build.csv --]
[-- Type: text/csv, Size: 13299 bytes --]

fc31788beb;20180106;84.1
c569e686b0;20180115;85.3
71e458505f;20180123;87.1
29abae3572;20180129;84.7
327d251f8a;20180203;86.5
47106da23a;20180216;85.1
2ea62d635d;20180226;84.9
a4605cd60d;20180301;87.5
461e681822;20180309;86.7
165803c691;20180316;86.3
bcc146a668;20180317;88.7
3a3aa0e056;20180318;86.8
1979bce57d;20180319;85.8
5c8c8f0a49;20180319;87.8
97608c60b8;20180320;87.7
16559146f9;20180320;87.1
5648746756;20180321;87.2
0a53c71fe2;20180322;87.7
2849477af2;20180322;86.5
011186279c;20180322;87.5
311bb33218;20180322;85.2
86d6417bf1;20180322;86.4
97b7e58c4d;20180322;77.9
dfea6d5a4a;20180322;77.7
b61b3c804c;20180328;77.5
25b22772a7;20180404;76.8
271d8510c4;20180412;78.2
42141da5b0;20180414;79.1
a37a14e0a8;20180415;77.8
0585bd643d;20180420;77.2
a0aac5741e;20180428;78.4
60ff810144;20180517;75.6
fd6f08840e;20180529;77.6
4a51deb993;20180605;78.6
0626d5aba5;20180611;77.1
92b609c572;20180616;76.1
de05ecd116;20180623;79.2
40db29fc55;20180702;78.5
cda7e1850f;20180713;79.1
7eef590870;20180808;77.1
eb787d749f;20180813;77.6
f18af6cd5c;20180821;77.3
ee2509bd82;20180831;77.6
3b38cb0516;20180913;76.5
21fc322763;20180927;77.4
333f0bfe76;20181008;77.9
487931cd06;20181019;79.3
1f38454e00;20181103;75.4
73ba6f16d1;20181112;76.6
e0799e6713;20181121;76.0
3b852da52f;20181129;79.1
51170f34ac;20181208;75.8
8664ba18c7;20181217;78.1
a0108f7871;20181223;78.2
f136a4bf7c;20190102;75.8
cdb082322d;20190114;77.1
9fc02ff5ea;20190115;76.9
2a3bd6798e;20190115;78.5
d12e5d003d;20190115;94.8
02976d6736;20190115;94.5
b2497ef695;20190116;95.2
8832de4f93;20190116;94.0
3fd369becd;20190116;96.1
8ca414de0e;20190117;96.0
03b8903ee7;20190117;96.1
d8da0916ff;20190118;95.9
957090b795;20190118;95.4
78725e49d2;20190118;95.2
8bb5939efa;20190119;94.7
67b1053dcd;20190120;93.7
6c2ee11d8f;20190127;94.1
b32ac17c32;20190203;93.9
dd319f2711;20190213;94.9
3700375334;20190218;94.7
a80e10458a;20190228;96.7
3635be5aee;20190305;93.4
1d65985a2b;20190311;94.6
107215596c;20190321;94.3
d7e4426517;20190327;94.3
3e8f9482fe;20190402;94.2
0797897f34;20190417;93.0
6f334b6bc0;20190419;94.4
db2c930849;20190423;93.3
86b6e91091;20190426;91.8
d4fa998c31;20190501;92.4
ab4619e7c3;20190505;93.5
655634808a;20190510;93.1
88cc910abe;20190516;93.2
5f9671e57e;20190518;93.2
464770d2e9;20190527;92.9
4b117065e9;20190528;93.5
146486f8a6;20190528;93.3
fe0cb43fb8;20190529;94.1
b367a3dee1;20190530;94.0
b354935902;20190530;94.3
f2a7acdde7;20190601;94.1
c6f4f5debd;20190601;94.1
6028662649;20190601;98.4
d101e87715;20190601;98.1
b5e41e8ba3;20190601;97.9
cfb592fd4b;20190607;97.0
03b66d23a8;20190610;92.8
769e9fa5ae;20190613;94.1
f6a1647a8b;20190615;94.2
df937c2080;20190616;93.8
1b855aa8dd;20190618;94.0
0a580c187d;20190624;93.3
1bf5be5d37;20190702;92.8
ef389fd9a5;20190706;94.9
7d8e759c09;20190710;94.7
936d074d7c;20190713;92.3
7c30ad184f;20190714;94.0
d2758820cd;20190716;95.2
cf285946be;20190721;95.0
8e0ebb9a3c;20190722;94.9
7f95d2d407;20190722;94.8
d0eeb62c43;20190722;95.3
056cbcb7a9;20190730;95.3
0173962570;20190730;96.7
f7cf9199ab;20190730;97.3
0aa4bfaab8;20190731;96.6
3018f6d832;20190731;96.2
07ce3be6aa;20190731;97.2
4dcb692de0;20190731;97.5
a79e96f0f9;20190731;96.8
3c31775527;20190801;96.9
75690d7fac;20190801;96.9
3134137bdd;20190801;97.0
b60b6ffb35;20190803;96.4
25baa7d20c;20190807;99.5
370f07046b;20190814;99.4
f38a16ee89;20190817;99.2
280cf93f31;20190920;98.2
87b685f436;20190923;96.2
84ef1ea8b5;20190926;97.8
19405291aa;20190929;97.9
b0043f688a;20191001;100.7
2ff16a4837;20191004;98.2
3f9ad4e725;20191008;98.9
e4efab3f03;20191010;96.7
3fcfa8071a;20191012;96.9
d2d658320e;20191014;98.5
1610d67051;20191017;99.0
d9a3c8974f;20191020;98.7
d5fdde42d2;20191021;99.7
75b9f4652b;20191026;98.6
9e10ee754c;20191030;99.6
3b2f11f268;20191102;98.7
0c293f1520;20191106;98.1
027f218ad2;20191110;98.7
31be86c347;20191114;98.5
a15431ffe2;20191119;98.4
b006095bc9;20191126;98.7
e6e0654daa;20191130;99.0
48373e7540;20191206;98.2
47a767c24e;20191211;99.2
0f7e3430bb;20191220;98.6
273b815767;20191225;98.4
56ce263f50;20200213;99.2
c4ca8219dd;20200223;98.8
3274b8090b;20200307;99.2
7725fbc5b1;20200320;101.0
3273e2ace7;20200330;100.5
333f63d537;20200405;100.5
d0b9cf876f;20200413;101.1
3f8b771da9;20200419;100.5
1844bcf556;20200428;100.5
6bd47f4477;20200503;101.1
7f7a8fbfd7;20200510;101.1
788c2480f4;20200516;103.3
294495de8e;20200523;103.4
43caa9680b;20200531;102.9
64e25cde32;20200608;103.9
b6c7780bb0;20200618;103.6
be7a07f38d;20200626;103.5
fcc04678b9;20200708;104.4
bcb58180c7;20200717;101.7
f7ebba5724;20200723;101.0
35564bea4d;20200730;101.9
fd50b3fc45;20200803;101.0
361baa451a;20200806;101.4
b8b88fad58;20200810;102.4
2b69a4df78;20200812;101.1
727ffbaaf4;20200813;101.8
64da2e8096;20200818;100.6
bd0523901b;20200820;101.9
3a99f966dc;20200823;100.7
1926fe6527;20200826;102.1
766cd0c7c3;20200828;101.1
2ea34662c2;20200831;100.1
a30268907c;20200905;103.0
6fd03b0fbe;20200907;102.9
27b711f834;20200910;102.5
1a4d8feb17;20200913;102.3
92f342f38d;20200916;102.1
db6bdef2dd;20200920;100.5
e1290f4713;20200922;100.2
9fd9c9c2c0;20200925;99.5
f679bcbb39;20200928;100.5
648de09e71;20201001;101.3
a14321ff69;20201006;98.6
7db2879a81;20201011;98.3
5474603c4c;20201014;97.1
0022935a31;20201016;101.8
6d0b2a60f6;20201017;101.7
8b6b4176f6;20201017;102.2
9513aa8f48;20201017;101.5
ffd201c47e;20201017;103.5
3b5a43f3d7;20201018;103.6
a67fed2e0c;20201018;103.5
9bc5c016a1;20201018;103.1
d3c99b6b6b;20201019;103.2
422fdabe7b;20201019;103.7
bacebc89af;20201019;103.6
954a4decfc;20201022;103.3
5daa6a6a03;20201024;104.5
608c821716;20201027;103.5
f018cffca0;20201030;104.2
2a4b0da28c;20201101;104.0
37c0208aaa;20201105;104.8
79d04ae13f;20201110;102.9
81588748bd;20201115;104.3
b4b1bd6e03;20201120;103.2
781dd876e4;20201123;104.7
e3d8f4b98f;20201126;103.6
fd578af547;20201202;102.8
c86dc8d488;20201205;104.8
6614b67d9e;20201208;105.1
4adc79641f;20201209;104.8
204d151940;20201212;104.9
10bc4eac5b;20201214;104.3
fdaaf886b7;20201218;104.4
188b09d6d9;20201222;104.6
47799cc02b;20201225;103.7
03bab768be;20201229;105.2
144b883eb0;20201231;106.2
ad2567fb1e;20210103;105.0
c1daeb4c28;20210105;103.7
918a5eae17;20210110;102.1
138486cddb;20210115;103.6
297edbebec;20210119;103.3
a6f030fc7b;20210121;102.0
3c314f3dd2;20210126;105.5
a8caa66906;20210128;106.6
e226357c3b;20210131;107.2
9a67da98a2;20210202;106.2
a6f23c226e;20210205;107.1
69943ae70c;20210208;106.9
5a598fa414;20210210;108.2
6b0de9f830;20210213;108.5
211731b3a9;20210215;108.6
546f552e7b;20210218;108.8
d0c47652e5;20210221;108.0
d56fbc375d;20210224;107.1
65f458ba5e;20210224;107.5
db09267de3;20210224;107.4
4d43b9a0b0;20210225;107.4
056b468f74;20210225;107.3
f83775b4d3;20210226;107.0
e95b3e4d1d;20210226;107.1
99340ad17a;20210226;107.4
a350ae058c;20210226;128.6
9abee2acda;20210227;128.6
3b1fb6186b;20210227;129.1
3984044ad3;20210226;128.6
b3d310fa96;20210226;140.2
2a9ce136ed;20210227;140.2
c9f5c314ad;20210227;139.8
7ac99eeefe;20210227;140.2
f8b4623db3;20210227;139.6
2c639a35a6;20210227;138.9
0f91948372;20210227;139.3
697aaafca3;20210228;139.7
9a110cb0d9;20210228;139.7
9eff23573d;20210228;139.8
ff09851a1f;20210227;129.2
aad8ffafa8;20210227;129.7
bdea1883cc;20210228;129.8
797f0ba218;20210228;129.6
150d06644e;20210228;129.3
8bdaac3457;20210228;130.0
59e1867a1f;20210301;129.5
08b11a02f4;20210301;129.6
165353674e;20210301;129.9
aa14398aa1;20210302;130.0
6458e16f33;20210303;130.3
7b99fc0069;20210304;129.7
8e759d60cc;20210304;130.1
6ff90ef0bb;20210304;130.0
4c2820ce3f;20210304;129.9
51e78e9146;20210305;129.9
03ada27cb8;20210305;130.7
1ff46018f6;20210305;131.1
1362a9fec4;20210305;119.1
533c659b6c;20210305;119.3
0f85f2c0e5;20210305;119.4
106e6f48bf;20210306;119.1
8fb33bae32;20210306;118.6
432c1aaa80;20210308;118.9
4dda6db3ab;20210308;119.2
56297c0f42;20210311;119.5
7a7bc15242;20210314;119.5
5d2f319eec;20210321;119.9
66a36f1e5a;20210427;119.5
647e4a92bb;20210508;121.4
3f8f3a9027;20210515;120.9
1f82c85bff;20210519;121.0
ce7a78d0a8;20210605;121.5
beb4e9f828;20210614;121.4
492a0ae592;20210704;118.5
35906d60e6;20210815;118.4
4dd1f56f29;20211111;122.5
30a00f14c8;20211114;123.2
37738c32ac;20211115;123.0
bc53310aaf;20211123;122.2
209662c0bd;20211124;123.4
127be1e8b3;20211130;124.4
b3277883f3;20211130;123.6
c3a440e520;20211130;123.8
6e5c2fb468;20211201;122.9
f638541785;20211203;124.3
aacc371ac5;20211206;125.3
d9b3f665bc;20211207;123.6
cc0f147180;20211210;125.0
b9c1e1d73b;20211211;123.5
13edadb492;20211212;124.2
a7794c6dd5;20211218;126.1
cb10eed97c;20211218;125.0
09af054dab;20211219;125.0
1a923e5ac6;20211221;125.7
552b74ba9e;20211224;126.0
d8fc436bbb;20211226;124.9
8719abe4ca;20211229;125.8
4521a3a19c;20211231;125.2
417a3ebaa2;20220104;125.0
0ae38ec94a;20220106;126.2
59ede5cce8;20220109;126.5
368b8fe765;20220111;127.3
fa1dd64ba8;20220114;126.4
7055104358;20220115;125.7
c99e28207a;20220118;124.9
d2a23c7441;20220120;126.6
a51a2a13ed;20220122;126.5
a604e877cd;20220122;127.3
bf3db92bd6;20220122;128.5
7e596463be;20220122;126.4
71b433f035;20220122;128.5
3b33a14380;20220122;126.2
79e54a1b75;20220122;126.9
1edde2fc7a;20220122;126.2
fd31ef21c5;20220122;131.3
bdcbe897d1;20220123;132.4
1bcbca3750;20220123;133.2
db454c98f1;20220123;132.4
80098a2830;20220123;133.0
dfae76c991;20220124;132.8
7f7b418ae6;20220124;130.4
824440a7c8;20220127;131.1
7e17749ada;20220129;128.9
4efab4330e;20220131;129.3
c86e4b0950;20220202;133.1
b4f1ceaf24;20220204;133.8
bff8354ec4;20220206;132.3
fe37e49a97;20220208;131.5
a9d54814b7;20220211;133.0
0ba1ecc816;20220214;130.3
c0fa3f2a6b;20220217;131.9
a1617f070b;20220217;131.8
df6e989f7a;20220217;133.9
3d0f5bc219;20220217;130.7
86ebc88cd8;20220217;131.8
c2913c30b1;20220218;131.8
1f755a74b7;20220218;132.4
7829f1b5e6;20220219;132.1
5b90b05e42;20220219;132.4
76ee4dc823;20220219;132.6
e015dc77f5;20220219;132.3
767619595c;20220219;132.1
f687e62ac5;20220219;145.3
f273a504be;20220219;143.9
9eefe59d57;20220219;145.1
76b91671a1;20220221;145.3
ec1aaac5c0;20220224;145.1
2db149539b;20220224;144.5
6092ee1c3f;20220224;129.9
bd17fa2c75;20220224;130.6
794fbd1c07;20220224;130.1
8d2fbd4a14;20220224;129.3
d29cc1e8a0;20220225;129.6
6fd6bea8c4;20220225;129.0
018eeb655a;20220226;129.4
e1757c00ca;20220226;129.7
796bb6f1e7;20220227;129.7
704a40ad75;20220227;129.8
f995169c72;20220228;129.9
122b189219;20220228;131.3
e05478050a;20220301;129.7
b305f5a3ab;20220304;131.2
f685bcb65d;20220304;130.6
6ad223fa9f;20220304;131.8
4564ae5a3d;20220305;131.7
2fbd48f733;20220304;130.4
98d2dc6522;20220305;130.8
0adb2d507c;20220306;130.7
450f3f8725;20220306;132.1
ea5a0142ce;20220306;131.2
6eeab90632;20220306;132.6
68cdb95019;20220306;137.1
93c2daa33d;20220306;137.2
36b31d38cd;20220307;136.9
231a1ba30f;20220307;138.3
8b66adfe06;20220307;137.9
7ad1df5e16;20220311;136.3
ea3c147d2d;20220313;133.9
d5e8f483f9;20220316;136.7
ae9a0b7808;20220319;133.9
c9c3d5d830;20220322;132.9
b4f504a0ea;20220324;133.9
2dfeea8962;20220326;135.0
c3ad47b7e4;20220401;133.0
c12a48c335;20220403;133.1
78ecd67888;20220406;132.3
d9bffa1f3b;20220408;132.3
60f66cc2a2;20220412;132.4
0a368e0077;20220413;132.7
c4768cda7f;20220415;133.4
cb953504ab;20220416;133.5
6662079b90;20220417;133.1
67940394c0;20220419;132.4
5bc5e56576;20220423;134.8
631a8ae9ee;20220426;132.9
c2c1976d0e;20220428;132.9
7b7a124afa;20220430;133.0
51186ed69c;20220501;134.5
3346b94b73;20220503;134.3
be374c18b3;20220505;135.4
e13744780f;20220507;135.0
d8a4782310;20220509;136.6
ec01391ab3;20220510;136.5
4e7879f807;20220511;136.4
dae431bf0a;20220513;135.1
b65a905edf;20220514;135.5
164ff4a012;20220517;135.7
75c90c7c51;20220519;136.7
208102fa47;20220521;135.1
a9b393c773;20220523;136.0
abd3411827;20220525;136.9
6c4d767019;20220527;136.3
52d41f2750;20220530;135.8
774880c84c;20220530;139.4
4132223d89;20220530;137.2
75bf80e2b1;20220531;136.5
f8d07fbeb3;20220531;135.4
f1e21a0341;20220531;135.7
0a7bd8c07c;20220531;136.5
41a2def162;20220531;135.3
1d4e903417;20220531;108.3
8ebfcff3a6;20220531;108.7
576d750915;20220531;108.7
71aaa9ac1e;20220531;108.7
7d1c44c2b3;20220531;109.7
b1ac48bd7e;20220531;109.2
3b9bbb24eb;20220601;108.6
3e312d11ce;20220601;83.4
3d354b1fba;20220601;83.2
a3d3fef2bc;20220601;82.8
a6322e7f13;20220601;83.4
c791a3e9f3;20220601;83.0
88b88c08a2;20220601;83.0
c50718dcfa;20220601;83.0
d9e106c72e;20220602;83.0
804415b8cc;20220601;83.7
2a40fa3d85;20220602;83.2
77854b1af6;20220602;83.4
f0189819d8;20220604;83.8
dc9337aacd;20220606;83.8
d7270b4445;20220609;83.6
cb0c697e21;20220611;83.4
df081b9f9c;20220613;83.9
8ee9e20f8c;20220615;83.7
17b3f8d56e;20220617;82.7
0dc75daf11;20220618;82.6
45dc99dcb9;20220620;82.8
0c8bc8e6d9;20220620;82.7
25e69968b0;20220620;82.8
c175984e2c;20220621;83.0
7934bad23b;20220621;83.2
32906819ad;20220621;83.3
0939465300;20220621;83.0
73a384a986;20220621;60.3
49137311a4;20220621;60.4
b1af8c2c00;20220621;60.4
be35c92c90;20220621;60.4
16d48cf8a0;20220621;60.5
de30e8c0de;20220621;60.3
fa6d9c1e84;20220622;60.3
f3c78c6ea7;20220622;60.0
c2e07f2258;20220626;59.7
baec3c4974;20220628;59.7
d3492bcf30;20220629;60.7

[-- Attachment #3: emacs-build.png --]
[-- Type: image/png, Size: 375767 bytes --]

Re: More build times

[Lars Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 469 bytes --]

Gregory Heytings <gregory@heytings.org> writes:

> Here is the graph you wanted.

Hm; I'm unable to view that image -- Emacs says that it's invalid?  I'm
not sure whether it's Gnus decoding it wrong or something, though.

But speaking of which, I've been letting an 8-code AMD server out there
do builds for a half a week now.  It's not done with all the builds
(it's doing three builds per day just to get the noise level down a
bit), but here's a preliminary graph:


[-- Attachment #2: Type: image/png, Size: 70615 bytes --]

[-- Attachment #3: Type: text/plain, Size: 1787 bytes --]


> The build times are in general constant.  The few commits that change
> the build time significantly, figured by breaks of the red line in the
> graph, are:

Thanks for identifying the significant commits.

> d12e5d003d deterioration from 78s to 95s (22%) Daniel Colascione Add
> portable dumper

That's that big jump in January 2019 in the chart above.  I wonder
whether anybody's taken a look at it again to see if there's anything
that can be done more efficiently there.

> a350ae058c deterioration from 108s to 129s (19%) Stefan Monnier
> <monnier@iro.umontreal.ca> * lisp/emacs-lisp/cconv.el: Improve line-nb
> info of unused var warnings

Hm...  is that something that can be improved (if that code is still
there, that is)?

> fddd63f8b8 and b3d310fa96 deterioration from 129s to 140s (9%) Glenn
> Morris <rgm@gnu.org> Distribute the real source for some doc/misc
> manuals (bug#45143)

Ah, I suspected that that was the cause of one of the major jumps.

> fd31ef21c5 deterioration from 126s to 132s (5%) Alan Mackenzie Don't
> use 'load-read-function' in byte-compile-from-buffer
> f687e62ac5 deterioration from 132s to 145s (10%) Alan Mackenzie Fix
> symbols with position appearing in the output of 'compile-defun'
> 6092ee1c3f improvement from 145s to 130s (10%) Alan Mackenzie Amend
> byte-run-strip-symbol-positions so that an unexec build builds
> 68cdb95019 deterioration from 130s to 136s (5%) Alan Mackenzie Restore
> call to byte-run-strip-symbol-positions in byte-compile-out

Right; so these weren't in one big commit, so it's more spread out.

> Congrats for 1d4e903417, 3e312d11ce and 73a384a986!

Thanks.  😃

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: More build times

[Gregory Heytings]

>
> Hm; I'm unable to view that image -- Emacs says that it's invalid?  I'm 
> not sure whether it's Gnus decoding it wrong or something, though.
>

I don't use Gnus, so I don't know.  It's a very big image (20000 x 1200 
pixels), but I can view it with xview, feh, Chromium, and with emacs -Q 
emacs-build.png.  In case it's a Gnus decoding bug, I've put it here, too: 
https://www.heytings.org/20220629-emacs-build.png .

Re: More build times

[Stephen Berman]

On Thu, 30 Jun 2022 08:30:13 +0000 Gregory Heytings <gregory@heytings.org> wrote:

>>
>> Hm; I'm unable to view that image -- Emacs says that it's invalid?  I'm not
>> sure whether it's Gnus decoding it wrong or something, though.
>>
>
> I don't use Gnus, so I don't know.  It's a very big image (20000 x 1200
> pixels), but I can view it with xview, feh, Chromium, and with emacs -Q
> emacs-build.png.

I cannot view it in Emacs 27-29 with Gnus, but after saving it (from
Gnus) to a file, I can view it with display (ImageMagick), Ristretto
(xfce4) and Firefox, but still not with Emacs (with or without -Q).

Steve Berman

Re: More build times

[Lars Ingebrigtsen]

Gregory Heytings <gregory@heytings.org> writes:

> I don't use Gnus, so I don't know.  It's a very big image (20000 x
> 1200 pixels),

Ah, that explains it.  Saving it and viewing it with feh works fine
here, too.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: More build times

[Andreas Schwab]

On Jun 30 2022, Lars Ingebrigtsen wrote:

> Hm; I'm unable to view that image -- Emacs says that it's invalid?

Probably just too large and exceeds max-image-size.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: More build times

[Stephen Berman]

On Thu, 30 Jun 2022 10:42:21 +0200 Andreas Schwab <schwab@suse.de> wrote:

> On Jun 30 2022, Lars Ingebrigtsen wrote:
>
>> Hm; I'm unable to view that image -- Emacs says that it's invalid?
>
> Probably just too large and exceeds max-image-size.

Yep.  After setting max-image-size to t I can view it in Emacs.

Steve Berman

Re: More build times

[Juri Linkov]

> Hm; I'm unable to view that image -- Emacs says that it's invalid?  I'm
> not sure whether it's Gnus decoding it wrong or something, though.

Strange, I can see that image in Gnus.  What am I doing wrong?
max-image-size is at its default.

Re: More build times

[Andreas Schwab]

On Jun 30 2022, Juri Linkov wrote:

> max-image-size is at its default.

The default is a limit relative to the frame size.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: More build times

[Juri Linkov]

>> max-image-size is at its default.
>
> The default is a limit relative to the frame size.

This explains everything: the image is not displayed after resizing the frame
to a smaller size.

Re: More build times

[Mattias Engdegård]

29 juni 2022 kl. 18.22 skrev Gregory Heytings <gregory@heytings.org>:

> I built Emacs at 500 points, from 2018 to today, on an up-to-date and unloaded Debian bookworm computer, and took the average wall clock time of 10 builds with "make -j20".

Thank you!

> 1362a9fec4 improvement from 130s to 119s (8%) Mattias Engdegård Make lambda-lifting work again

That just fixed a typo introduced a week or so before, which should cancel it out.

Re: More build times

[Gregory Heytings]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]


>> 1362a9fec4 improvement from 130s to 119s (8%) Mattias Engdegård Make 
>> lambda-lifting work again
>
> That just fixed a typo introduced a week or so before, which should 
> cancel it out.
>

Indeed.  The typo in which that typo was introduced was

a350ae058c deterioration from 108s to 129s (19%) Stefan Monnier 
<monnier@iro.umontreal.ca> * lisp/emacs-lisp/cconv.el: Improve line-nb 
info of unused var warnings

And I just checked that fixing that typo in that commit removes the 19% 
slowdown.  This answers one of Lars' questions:

>> a350ae058c deterioration from 108s to 129s (19%) Stefan Monnier 
>> <monnier@iro.umontreal.ca> * lisp/emacs-lisp/cconv.el: Improve line-nb 
>> info of unused var warnings
>
> Hm...  is that something that can be improved (if that code is still 
> there, that is)?
>

Re: More build times

[Gregory Heytings]

>
> The typo in which that typo was introduced
>

Wow, a recursive typo.  I meant: The commit in which that typo was 
introduced.

Re: More build times

[Yuri Khan]

> It's a git artefact.  For instance, this is what "git log" says:
>
> commit 61a312ba0cae10e8e19b7424540751a71d0170b1
> Date:   Fri Jun 17 10:24:05 2022 +0800
>
> commit f419de6eca4ca6a6d03db1eec4b9086a3d1e5b86
> Date:   Thu Jun 16 09:48:05 2022 +0200

> No intervening commits or anything, but checking out f419de6eca and
> 61a312ba0c give you vastly different trees -- presumably because of
> merges and the like (since f419de6eca4 originated in emacs-28).

Yes:

    $ git fetch
    $ git log --oneline --graph --decorate --date-order origin/master
    […]
    * | | cce197c630 Fix instances of not using a usable Motif drag atom
    * | | d1a10a1cf2 Merge from origin/emacs-28
    |\| |
    * | | a8cf6567dd Improve window manager user time reporting mode switching
→   * | | 61a312ba0c Update last event time during DND operations
→   | * | f419de6eca * lisp/textmodes/artist.el: Minor doc fixes.
    * | | e75ce9ca38 * src/fns.c (Fmapconcat): Optimize the case where
separator==""
    * | | cd9b920217 Fix Tramp test
    […]

The two commits are adjacent in date order but live on different
branches, one having been merged into the other a couple commits
later.

> Is there a way to make git cough up the commit on the master branch for
> a specific date?  I.e., "what was the state on the master branch at
> Fri Jun 17 06:30:35 2022 +0200"?

Not really. In Git, commits do not store the information about the
branch they were originally committed to. Branches keep timestamped
reflogs so you can ask them where they used to point at certain
moments of time, but, for remote tracking branches, that depends on
how regularly you update them (do a ‘git fetch’).

For example:

1. You have a clone of a repository as of 2022-05-16. Its
origin/master points at commit A.
2. On 2022-05-17, someone pushes commit B.
3. On 2022-05-18, someone else pushes commit C.
4. On 2022-05-19, you do a ‘git fetch’.

Now your remote tracking branch origin/master points at the commit C,
and the last two records of its reflog are “on 2022-05-19, I moved
from A to C as a result of a ‘git fetch’” and “on 2022-05-16, I moved
from […] to A as a result of […]”.

‘git rev-list -n 1 --before=<date> origin/master’ will give you *some*
commit that is reachable from origin/master and was committed before
the specified date, but it might be from a branch that was merged
later than that date.

What you can do is ‘git rev-list -n 1 --before=<date> --first-parent
origin/master’ and hope people don’t do things like “first merge
master into my feature branch, then fast-forward master to the
resulting merge commit” because this messes up the order of parent
links in the merge.

Compare the above commit graph with the following:

    $ git log --oneline --graph --decorate --date-order --first-parent
origin/master
    […]
    * cce197c630 Fix instances of not using a usable Motif drag atom
    * d1a10a1cf2 Merge from origin/emacs-28
    * a8cf6567dd Improve window manager user time reporting mode switching
→   * 61a312ba0c Update last event time during DND operations
    * e75ce9ca38 * src/fns.c (Fmapconcat): Optimize the case where separator==""
    * cd9b920217 Fix Tramp test
    […]

Note that f419de6eca is nowhere to be seen. It is only reachable from
origin/master over the second parent link in the merge d1a10a1cf2.

Re: More build times

[Lars Ingebrigtsen]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> What you can do is ‘git rev-list -n 1 --before=<date> --first-parent
> origin/master’ and hope people don’t do things like “first merge
> master into my feature branch, then fast-forward master to the
> resulting merge commit” because this messes up the order of parent
> links in the merge.

Ah, thanks!  With --first-parent, I seem to be getting the state I'm
looking for (I think).  I've now re-started the benchmarking thing, so
in a week we should know whether that makes the graph smoother.  😅

(And I don't think we ever fast-forward master?)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: More build times

[Stefan Monnier]

> Ah, thanks!  With --first-parent, I seem to be getting the state I'm
> looking for (I think).  I've now re-started the benchmarking thing, so
> in a week we should know whether that makes the graph smoother.  😅

`--first-parent` will probably avoid the commits that are actually on
`emacs-28`, because I think we do these commits "correctly", but I'm not
sure it'll be reliable enough in general.

`git bisect` needs to solve the same problem, so maybe we should look at
how it does it.


        Stefan

Re: More build times

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> `--first-parent` will probably avoid the commits that are actually on
> `emacs-28`, because I think we do these commits "correctly", but I'm not
> sure it'll be reliable enough in general.
>
> `git bisect` needs to solve the same problem, so maybe we should look at
> how it does it.

Yup.  But I feel like I've sometimes had the same problem with git
bisect?  I.e., landing on commits from branches that have been merged in
later.  (I'm not sure, though.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no