How to make aot native-compilation and pdmp creation reproduceable

How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Hey,

I noticed that building Emacs isn't enterily reproduceable.
The compiled byte code it reproduceable, however I notied that
the pdmp files and some of the native compiled elisp for each variant
change each time the package is rebuild.

The specific build option don't seem to make a difference.

Strangely the only eln file that changes is x-win.el:
/usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln differs in ELF sections
--- old /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
+++ new /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
@@ -2950,7 +2950,7 @@
  0e7c0 6f726564 20736573 73696f6e 20646174  ored session dat
  0e7d0 61222022 4865776c 6574742d 5061636b  a" "Hewlett-Pack
  0e7e0 61726420 496e636f 72706f72 61746564  ard Incorporated
- 0e7f0 22202334 3d282831 3638202e 206d7574  " #4=((168 . mut
+ 0e7f0 22202336 3d282831 3638202e 206d7574  " #6=((168 . mut
  0e800 652d6163 75746529 20283136 39202e20  e-acute) (169 . 
  0e810 6d757465 2d677261 76652920 28313730  mute-grave) (170
  0e820 202e206d 7574652d 61736369 69636972   . mute-asciicir
@@ -2976,7 +2976,7 @@
  0e960 20224865 776c6574 742d5061 636b6172   "Hewlett-Packar
  0e970 6420436f 6d70616e 79222022 5831312f  d Company" "X11/
  0e980 4e655753 202d2053 756e204d 6963726f  NeWS - Sun Micro
- 0e990 73797374 656d7320 496e632e 22202336  systems Inc." #6
+ 0e990 73797374 656d7320 496e632e 22202334  systems Inc." #4
  0e9a0 3d282833 39323937 36202e20 66333629  =((392976 . f36)
  0e9b0 20283339 32393737 202e2066 33372920   (392977 . f37) 
  0e9c0 28333933 30353620 2e207265 71292028  (393056 . req) (

One example of pdmp of one of the variants:
/usr/bin/emacs-x11 differs in ELF sections
--- old /usr/bin/emacs-x11 (objdump)
+++ new /usr/bin/emacs-x11 (objdump)
@@ -519888,8 +519888,8 @@
  7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
  7f1420 00400000 00000000 00000000 00000000  .@..............
  7f1430 00000000 00000000 00000000 00000000  ................
- 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
- 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
+ 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
+ 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
  7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
  7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
  7f1480 12000000 00000000 00000000 00000000  ................

The build log of the package including a binary diff can be found here:
https://build.opensuse.org/build/editors/openSUSE_Tumbleweed/x86_64/emacs/_log

The spec file of the package can be fond here:
https://build.opensuse.org/package/view_file/editors/emacs/emacs.spec?expand=1

How can I debug why they differ after each rebuild?

Br,

Björn Bidar

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

> Hey,
>
> I noticed that building Emacs isn't enterily reproduceable.
> The compiled byte code it reproduceable, however I notied that
> the pdmp files and some of the native compiled elisp for each variant
> change each time the package is rebuild.
>
> The specific build option don't seem to make a difference.
>
> Strangely the only eln file that changes is x-win.el:
> /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln differs in ELF sections
> --- old /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
> +++ new /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
> @@ -2950,7 +2950,7 @@
>   0e7c0 6f726564 20736573 73696f6e 20646174  ored session dat
>   0e7d0 61222022 4865776c 6574742d 5061636b  a" "Hewlett-Pack
>   0e7e0 61726420 496e636f 72706f72 61746564  ard Incorporated
> - 0e7f0 22202334 3d282831 3638202e 206d7574  " #4=((168 . mut
> + 0e7f0 22202336 3d282831 3638202e 206d7574  " #6=((168 . mut
>   0e800 652d6163 75746529 20283136 39202e20  e-acute) (169 . 
>   0e810 6d757465 2d677261 76652920 28313730  mute-grave) (170
>   0e820 202e206d 7574652d 61736369 69636972   . mute-asciicir
> @@ -2976,7 +2976,7 @@
>   0e960 20224865 776c6574 742d5061 636b6172   "Hewlett-Packar
>   0e970 6420436f 6d70616e 79222022 5831312f  d Company" "X11/
>   0e980 4e655753 202d2053 756e204d 6963726f  NeWS - Sun Micro
> - 0e990 73797374 656d7320 496e632e 22202336  systems Inc." #6
> + 0e990 73797374 656d7320 496e632e 22202334  systems Inc." #4
>   0e9a0 3d282833 39323937 36202e20 66333629  =((392976 . f36)
>   0e9b0 20283339 32393737 202e2066 33372920   (392977 . f37) 
>   0e9c0 28333933 30353620 2e207265 71292028  (393056 . req) (

These two look like comp.el prints with print-circle being t, and
doesn't use print-continuous-numbering/print-number-table.

> One example of pdmp of one of the variants:
> /usr/bin/emacs-x11 differs in ELF sections
> --- old /usr/bin/emacs-x11 (objdump)
> +++ new /usr/bin/emacs-x11 (objdump)
> @@ -519888,8 +519888,8 @@
>   7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
>   7f1420 00400000 00000000 00000000 00000000  .@..............
>   7f1430 00000000 00000000 00000000 00000000  ................
> - 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
> - 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
> + 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
> + 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
>   7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
>   7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
>   7f1480 12000000 00000000 00000000 00000000  ................

That's confusing.  Is that an executable, if you do an objdump?  You
write it's an example of pdmp.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

>> One example of pdmp of one of the variants:
>> /usr/bin/emacs-x11 differs in ELF sections
>> --- old /usr/bin/emacs-x11 (objdump)
>> +++ new /usr/bin/emacs-x11 (objdump)
>> @@ -519888,8 +519888,8 @@
>>   7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
>>   7f1420 00400000 00000000 00000000 00000000  .@..............
>>   7f1430 00000000 00000000 00000000 00000000  ................
>> - 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
>> - 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
>> + 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
>> + 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
>>   7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
>>   7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
>>   7f1480 12000000 00000000 00000000 00000000  ................
>
> That's confusing.  Is that an executable, if you do an objdump?  You
> write it's an example of pdmp.
Th Emacs executable also changes each time.
Oh that's an oversight on my side. Here is the diff:
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp differs at offset '17' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
@@ -1,6 +1,6 @@
 00000000  44 55 4d 50 45 44 47 4e  55 45 4d 41 43 53 00 00  |DUMPEDGNUEMACS..|
-00000010  81 e3 b8 e4 33 01 1e c5  18 db 85 f2 95 86 19 7c  |....3..........||
-00000020  be 6f b7 03 f7 61 af d4  c6 d7 bd 74 f2 16 8b e2  |.o...a.....t....|
+00000010  a2 c0 5c 34 51 54 0c 6d  ea e9 6b ed 21 a9 0a 3c  |..\4QT.m..k.!..<|
+00000020  c0 cf 32 1a e7 c6 34 d2  89 ae 77 91 2f e9 09 67  |..2...4...w./..g|
 00000030  98 ce c2 00 bc d7 08 00  88 2d e6 00 5b 00 00 00  |.........-..[...|
 00000040  f8 2e e6 00 b8 16 00 00  d8 89 e6 00 e3 e9 03 00  |................|
 00000050  68 31 f6 00 03 0d 00 00  20 4c 98 00 00 00 9c 00  |h1...... L......|

Here is the diff of the matching Emacs binary:
/usr/bin/emacs-x11 differs in ELF sections
--- old /usr/bin/emacs-x11 (objdump)
+++ new /usr/bin/emacs-x11 (objdump)
@@ -519888,8 +519888,8 @@
  7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
  7f1420 00400000 00000000 00000000 00000000  .@..............
  7f1430 00000000 00000000 00000000 00000000  ................
- 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
- 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
+ 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
+ 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
  7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
  7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
  7f1480 12000000 00000000 00000000 00000000  ................

(I had to rebuild to get new logs)

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

[-- Attachment #1: Type: text/plain, Size: 2977 bytes --]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

>> One example of pdmp of one of the variants:
>> /usr/bin/emacs-x11 differs in ELF sections
>> --- old /usr/bin/emacs-x11 (objdump)
>> +++ new /usr/bin/emacs-x11 (objdump)
>> @@ -519888,8 +519888,8 @@
>>   7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
>>   7f1420 00400000 00000000 00000000 00000000  .@..............
>>   7f1430 00000000 00000000 00000000 00000000  ................
>> - 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
>> - 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
>> + 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
>> + 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
>>   7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
>>   7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
>>   7f1480 12000000 00000000 00000000 00000000  ................
>
> That's confusing.  Is that an executable, if you do an objdump?  You
> write it's an example of pdmp.
Th Emacs executable also changes each time.
Oh that's an oversight on my side. Here is the diff:
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp differs at offset '17' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
@@ -1,6 +1,6 @@
 00000000  44 55 4d 50 45 44 47 4e  55 45 4d 41 43 53 00 00  |DUMPEDGNUEMACS..|
-00000010  81 e3 b8 e4 33 01 1e c5  18 db 85 f2 95 86 19 7c  |....3..........||
-00000020  be 6f b7 03 f7 61 af d4  c6 d7 bd 74 f2 16 8b e2  |.o...a.....t....|
+00000010  a2 c0 5c 34 51 54 0c 6d  ea e9 6b ed 21 a9 0a 3c  |..\4QT.m..k.!..<|
+00000020  c0 cf 32 1a e7 c6 34 d2  89 ae 77 91 2f e9 09 67  |..2...4...w./..g|
 00000030  98 ce c2 00 bc d7 08 00  88 2d e6 00 5b 00 00 00  |.........-..[...|
 00000040  f8 2e e6 00 b8 16 00 00  d8 89 e6 00 e3 e9 03 00  |................|
 00000050  68 31 f6 00 03 0d 00 00  20 4c 98 00 00 00 9c 00  |h1...... L......|

Here is the diff of the matching Emacs binary:
/usr/bin/emacs-x11 differs in ELF sections
--- old /usr/bin/emacs-x11 (objdump)
+++ new /usr/bin/emacs-x11 (objdump)
@@ -519888,8 +519888,8 @@
  7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
  7f1420 00400000 00000000 00000000 00000000  .@..............
  7f1430 00000000 00000000 00000000 00000000  ................
- 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
- 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
+ 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
+ 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
  7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
  7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
  7f1480 12000000 00000000 00000000 00000000  ................

(I had to rebuild to get new logs)

I also attach the full diff to this message.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nachbau.emacs.log --]
[-- Type: text/x-log, Size: 21073 bytes --]


/var/tmp/build-root.//.mount/.build.packages/SRPMS/emacs-29.1-4.1.src.rpm

/var/tmp/build-root.//.mount/.build.packages/RPMS/noarch/emacs-el-29.1-4.1.noarch.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/noarch/emacs-info-29.1-4.1.noarch.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-x11-debuginfo-29.1-4.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-x11-29.1-4.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-29.1-4.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-eln-29.1-4.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-nox-29.1-4.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-debugsource-29.1-4.1.x86_64.rpm/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-nox-debuginfo-29.1-4.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-debuginfo-29.1-4.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/etags-debuginfo-29.1-4.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/etags-29.1-4.1.x86_64.rpm
+ echo 0
+ cp -a binaries.nachbau/.osc-build-retval .
+ cp -a /var/tmp/build-root.//.build.log binaries.nachbau/
+ cp -a binaries.nachbau/.build.log .
+ test -e binaries.nachbau/.build.log
+ tail -1 binaries.nachbau/.build.log
+ perl -ne '/\[\s*(\d+)s\]/ && print $1'
+ find -mtime -0 -print0
+ xargs --no-run-if-empty -0 touch
++ cat binaries.nachbau/.osc-build-retval
+ '[' 0 -gt 0 ']'
+ calcchecksums ../emacs/binaries.nachbau/emacs-29.1-4.1.src.rpm ../emacs/binaries.nachbau/emacs-29.1-4.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-debuginfo-29.1-4.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-debugsource-29.1-4.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-el-29.1-4.1.noarch.rpm ../emacs/binaries.nachbau/emacs-eln-29.1-4.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-info-29.1-4.1.noarch.rpm ../emacs/binaries.nachbau/emacs-nox-29.1-4.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-nox-debuginfo-29.1-4.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-x11-29.1-4.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-x11-debuginfo-29.1-4.1.x86_64.rpm ../emacs/binaries.nachbau/etags-29.1-4.1.x86_64.rpm ../emacs/binaries.nachbau/etags-debuginfo-29.1-4.1.x86_64.rpm
+ sed -e 's!\.\./!!'
++ find binaries.nachbau/ -maxdepth 1 -name '*.rpm' -print -quit
+ test -n binaries.nachbau/emacs-el-29.1-4.1.noarch.rpm
+ : binaries
+ : binaries.nachbau
+ pushd binaries
~/dev/opensuse/buildservice/openSUSE/Factory/emacs/binaries ~/dev/opensuse/buildservice/openSUSE/Factory/emacs
++ ls emacs-29.1-4.1.x86_64.rpm emacs-el-29.1-4.1.noarch.rpm emacs-eln-29.1-4.1.x86_64.rpm emacs-info-29.1-4.1.noarch.rpm emacs-nox-29.1-4.1.x86_64.rpm emacs-x11-29.1-4.1.x86_64.rpm etags-29.1-4.1.x86_64.rpm
++ grep -v '^::import::'
+ origrpms='emacs-29.1-4.1.x86_64.rpm
emacs-el-29.1-4.1.noarch.rpm
emacs-eln-29.1-4.1.x86_64.rpm
emacs-info-29.1-4.1.noarch.rpm
emacs-nox-29.1-4.1.x86_64.rpm
emacs-x11-29.1-4.1.x86_64.rpm
etags-29.1-4.1.x86_64.rpm'
+ popd
~/dev/opensuse/buildservice/openSUSE/Factory/emacs
+ [[ -n emacs-29.1-4.1.x86_64.rpm
emacs-el-29.1-4.1.noarch.rpm
emacs-eln-29.1-4.1.x86_64.rpm
emacs-info-29.1-4.1.noarch.rpm
emacs-nox-29.1-4.1.x86_64.rpm
emacs-x11-29.1-4.1.x86_64.rpm
etags-29.1-4.1.x86_64.rpm ]]
+ pkgdiff=/home/bidar/build-compare/pkg-diff.sh
+ test -e /home/bidar/build-compare/pkg-diff.sh
+ pkgdiff=/usr/lib/build/pkg-diff.sh
+ test -e /usr/lib/build/pkg-diff.sh
+ overall=0
+ overalldiffered=0
+ for f in $origrpms
++ echo emacs-29.1-4.1.x86_64.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-29.1-*.x86_64.rpm'
+ f1=binaries/emacs-29.1-4.1.x86_64.rpm
+ f2='binaries.nachbau/emacs-29.1-*.x86_64.rpm'
+ cmp binaries/emacs-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-29.1-4.1.x86_64.rpm
binaries/emacs-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-29.1-4.1.x86_64.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=1
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-29.1-4.1.x86_64.rpm
Comparing emacs-29.1-4.1.x86_64.rpm to emacs-29.1-4.1.x86_64.rpm
comparing the rpm tags of emacs
RPM meta information is identical
Extracting packages
Package content is identical
+ compareresult=0
+ [[ 0 = 0 ]]
+ for f in $origrpms
++ echo emacs-el-29.1-4.1.noarch.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-el-29.1-*.noarch.rpm'
+ f1=binaries/emacs-el-29.1-4.1.noarch.rpm
+ f2='binaries.nachbau/emacs-el-29.1-*.noarch.rpm'
+ cmp binaries/emacs-el-29.1-4.1.noarch.rpm binaries.nachbau/emacs-el-29.1-4.1.noarch.rpm
binaries/emacs-el-29.1-4.1.noarch.rpm binaries.nachbau/emacs-el-29.1-4.1.noarch.rpm differ: byte 226, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=2
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-el-29.1-4.1.noarch.rpm binaries.nachbau/emacs-el-29.1-4.1.noarch.rpm
Comparing emacs-el-29.1-4.1.noarch.rpm to emacs-el-29.1-4.1.noarch.rpm
comparing the rpm tags of emacs-el
RPM meta information is identical
Extracting packages
Package content is identical
+ compareresult=0
+ [[ 0 = 0 ]]
+ for f in $origrpms
++ echo emacs-eln-29.1-4.1.x86_64.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-eln-29.1-*.x86_64.rpm'
+ f1=binaries/emacs-eln-29.1-4.1.x86_64.rpm
+ f2='binaries.nachbau/emacs-eln-29.1-*.x86_64.rpm'
+ cmp binaries/emacs-eln-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-eln-29.1-4.1.x86_64.rpm
binaries/emacs-eln-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-eln-29.1-4.1.x86_64.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=3
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-eln-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-eln-29.1-4.1.x86_64.rpm
Comparing emacs-eln-29.1-4.1.x86_64.rpm to emacs-eln-29.1-4.1.x86_64.rpm
comparing the rpm tags of emacs-eln
--- old-rpm-tags
+++ new-rpm-tags
@@ -711 +711 @@
-/usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln 36acc5c9856aae02f65b275f1a0432dbea3fcb8e00a64a52d6a946be7352c3a8 0
+/usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln c49b4771f320e07a6a790e61ec71a44306d2afc60eb369dcc1c7721f2bf6cbe9 0
@@ -870 +870 @@
-/usr/lib64/emacs/29.1/native-lisp/29.1-5abdf415/preloaded/x-win-3c49581f-6ddb7e08.eln 4e587b95433867fb385c9a52ce7f0716296ccd70463132132a5f26edf10a56d1 0
+/usr/lib64/emacs/29.1/native-lisp/29.1-5abdf415/preloaded/x-win-3c49581f-6ddb7e08.eln 239f9b90f7e8943b54b37ec7800f566d808273a12dff98320fcbeb4b21b4f5ed 0
comparing rpmtags
comparing RELEASE
comparing PROVIDES
comparing scripts
comparing filelist
comparing file checksum
creating rename script
RPM file checksum differs.
Extracting packages
/usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln differs in ELF sections
--- old /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
+++ new /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
@@ -2976,7 +2976,7 @@
  0e960 20224865 776c6574 742d5061 636b6172   "Hewlett-Packar
  0e970 6420436f 6d70616e 79222022 5831312f  d Company" "X11/
  0e980 4e655753 202d2053 756e204d 6963726f  NeWS - Sun Micro
- 0e990 73797374 656d7320 496e632e 22202336  systems Inc." #6
+ 0e990 73797374 656d7320 496e632e 22202335  systems Inc." #5
  0e9a0 3d282833 39323937 36202e20 66333629  =((392976 . f36)
  0e9b0 20283339 32393737 202e2066 33372920   (392977 . f37) 
  0e9c0 28333933 30353620 2e207265 71292028  (393056 . req) (
@@ -2987,7 +2987,7 @@
  0ea10 20283339 33303736 202e2070 61737465   (393076 . paste
  0ea20 29202833 39333037 37202e20 63757429  ) (393077 . cut)
  0ea30 29202258 20436f6e 736f7274 69756d22  ) "X Consortium"
- 0ea40 2023353d 28283635 32383020 2e207265   #5=((65280 . re
+ 0ea40 2023363d 28283635 32383020 2e207265   #6=((65280 . re
  0ea50 6d6f7665 29292067 75692d2d 73656c65  move)) gui--sele
  0ea60 6374696f 6e2d7661 6c75652d 696e7465  ction-value-inte
  0ea70 726e616c 20434c49 50424f41 52442073  rnal CLIPBOARD s
/usr/lib64/emacs/29.1/native-lisp/29.1-5abdf415/preloaded/x-win-3c49581f-6ddb7e08.eln differs in ELF sections
--- old /usr/lib64/emacs/29.1/native-lisp/29.1-5abdf415/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
+++ new /usr/lib64/emacs/29.1/native-lisp/29.1-5abdf415/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
@@ -2976,7 +2976,7 @@
  0e960 20224865 776c6574 742d5061 636b6172   "Hewlett-Packar
  0e970 6420436f 6d70616e 79222022 5831312f  d Company" "X11/
  0e980 4e655753 202d2053 756e204d 6963726f  NeWS - Sun Micro
- 0e990 73797374 656d7320 496e632e 22202334  systems Inc." #4
+ 0e990 73797374 656d7320 496e632e 22202335  systems Inc." #5
  0e9a0 3d282833 39323937 36202e20 66333629  =((392976 . f36)
  0e9b0 20283339 32393737 202e2066 33372920   (392977 . f37) 
  0e9c0 28333933 30353620 2e207265 71292028  (393056 . req) (
@@ -2987,7 +2987,7 @@
  0ea10 20283339 33303736 202e2070 61737465   (393076 . paste
  0ea20 29202833 39333037 37202e20 63757429  ) (393077 . cut)
  0ea30 29202258 20436f6e 736f7274 69756d22  ) "X Consortium"
- 0ea40 2023353d 28283635 32383020 2e207265   #5=((65280 . re
+ 0ea40 2023343d 28283635 32383020 2e207265   #4=((65280 . re
  0ea50 6d6f7665 29292067 75692d2d 73656c65  move)) gui--sele
  0ea60 6374696f 6e2d7661 6c75652d 696e7465  ction-value-inte
  0ea70 726e616c 20434c49 50424f41 52442073  rnal CLIPBOARD s
+ compareresult=1
+ [[ 1 = 0 ]]
+ overall=1
+ for f in $origrpms
++ echo emacs-info-29.1-4.1.noarch.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-info-29.1-*.noarch.rpm'
+ f1=binaries/emacs-info-29.1-4.1.noarch.rpm
+ f2='binaries.nachbau/emacs-info-29.1-*.noarch.rpm'
+ cmp binaries/emacs-info-29.1-4.1.noarch.rpm binaries.nachbau/emacs-info-29.1-4.1.noarch.rpm
binaries/emacs-info-29.1-4.1.noarch.rpm binaries.nachbau/emacs-info-29.1-4.1.noarch.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=4
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-info-29.1-4.1.noarch.rpm binaries.nachbau/emacs-info-29.1-4.1.noarch.rpm
Comparing emacs-info-29.1-4.1.noarch.rpm to emacs-info-29.1-4.1.noarch.rpm
comparing the rpm tags of emacs-info
RPM meta information is identical
Extracting packages
Package content is identical
+ compareresult=0
+ [[ 0 = 0 ]]
+ for f in $origrpms
++ echo emacs-nox-29.1-4.1.x86_64.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-nox-29.1-*.x86_64.rpm'
+ f1=binaries/emacs-nox-29.1-4.1.x86_64.rpm
+ f2='binaries.nachbau/emacs-nox-29.1-*.x86_64.rpm'
+ cmp binaries/emacs-nox-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-nox-29.1-4.1.x86_64.rpm
binaries/emacs-nox-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-nox-29.1-4.1.x86_64.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=5
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-nox-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-nox-29.1-4.1.x86_64.rpm
Comparing emacs-nox-29.1-4.1.x86_64.rpm to emacs-nox-29.1-4.1.x86_64.rpm
comparing the rpm tags of emacs-nox
--- old-rpm-tags
+++ new-rpm-tags
@@ -158 +158 @@
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp 8f6f6b5d7d6428b5eaf56230883887dd72d559fff493a21c999a14862c0cddfe 0
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp 3f4b5c83f2e9514eb2e1b1998ca54551d6566c1008d8f1390808cb767e150bbc 0
comparing rpmtags
comparing RELEASE
comparing PROVIDES
comparing scripts
comparing filelist
comparing file checksum
creating rename script
RPM file checksum differs.
Extracting packages
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp differs at offset '60273' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp (hex)
@@ -1,7 +1,7 @@
 0000eb40  00 50 00 03 00 00 00 40  08 e7 00 00 00 00 00 00  |.P.....@........|
 0000eb50  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 0000eb60  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
-0000eb70  3a 1d 8b 5e ed 4f 28 5e  02 28 6b ee 00 00 00 00  |:..^.O(^.(k.....|
+0000eb70  f2 0b ff 94 2e f1 30 5e  02 28 6b ee 00 00 00 00  |......0^.(k.....|
 0000eb80  47 00 00 1f 00 00 00 40  00 00 00 00 00 00 00 00  |G......@........|
 0000eb90  00 00 00 00 00 00 00 00  60 30 00 00 00 00 00 00  |........`0......|
 0000eba0  60 ef 00 00 00 00 00 00  c0 ed 00 00 00 00 00 00  |`...............|
+ compareresult=1
+ [[ 1 = 0 ]]
+ overall=1
+ for f in $origrpms
++ echo emacs-x11-29.1-4.1.x86_64.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-x11-29.1-*.x86_64.rpm'
+ f1=binaries/emacs-x11-29.1-4.1.x86_64.rpm
+ f2='binaries.nachbau/emacs-x11-29.1-*.x86_64.rpm'
+ cmp binaries/emacs-x11-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-x11-29.1-4.1.x86_64.rpm
binaries/emacs-x11-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-x11-29.1-4.1.x86_64.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=6
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-x11-29.1-4.1.x86_64.rpm binaries.nachbau/emacs-x11-29.1-4.1.x86_64.rpm
Comparing emacs-x11-29.1-4.1.x86_64.rpm to emacs-x11-29.1-4.1.x86_64.rpm
comparing the rpm tags of emacs-x11
--- old-rpm-tags
+++ new-rpm-tags
@@ -234 +234 @@
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11-81e3b8e433011ec518db85f29586197cbe6fb703f761afd4c6d7bd74f2168be2.pdmp 0 (none) 120777 root root 0 4294967295 emacs-x11.pdmp
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11-a2c05c3451540c6deae96bed21a90a3cc0cf321ae7c634d289ae77912fe90967.pdmp 0 (none) 120777 root root 0 4294967295 emacs-x11.pdmp
@@ -261 +261 @@
-/usr/bin/emacs-x11 41c41eac4c0b4d024b10abed80e118f815504ff61333c98f9933cec9a407667d 0
+/usr/bin/emacs-x11 a2d14d6c2fe8fb7a2f3b4294a52fa84464eee3bb48810f51b168c44862cf401e 0
@@ -263 +263 @@
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp 53164a45ed0046fd84510219d5c54de1d0837643732b090c1c0447edf9d6c133 0
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp eb0e82ead89a96c87c78900a5268a1222a5cc2113edd0c6e932886dc664fbe59 0
@@ -265,3 +265,3 @@
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp 6bcf94cafd2351a365d457d438fc346291eb905d2eae590a644753ba8215efce 0
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11-81e3b8e433011ec518db85f29586197cbe6fb703f761afd4c6d7bd74f2168be2.pdmp  0
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp 18fdf1bbeddae449be1d1d9ef7c70107fe3e11c80944e18be5c48efb82eed68e 0
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp a0b7b0ebdac55ab6915523f621661834008b63c8312f530cd6407ea91e7b548d 0
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11-a2c05c3451540c6deae96bed21a90a3cc0cf321ae7c634d289ae77912fe90967.pdmp  0
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp d8d654a13cb0d4b6986aa9d053e9a48b7f2a21e3ed206f97bf5bb62d65afec3c 0
comparing rpmtags
comparing RELEASE
comparing PROVIDES
comparing scripts
comparing filelist
--- old-filelist
+++ new-filelist
@@ -5,7 +5,7 @@
 /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp 0 (none) 100644 root root 0 4294967295 
 /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland-0f6013953f74f885f9e1d4e3773fc22d291ea6a8440ce64b23847ef94c73a6f9.pdmp 0 (none) 120777 root root 0 4294967295 emacs-wayland.pdmp
 /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp 0 (none) 100644 root root 0 4294967295 
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11-81e3b8e433011ec518db85f29586197cbe6fb703f761afd4c6d7bd74f2168be2.pdmp 0 (none) 120777 root root 0 4294967295 emacs-x11.pdmp
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11-a2c05c3451540c6deae96bed21a90a3cc0cf321ae7c634d289ae77912fe90967.pdmp 0 (none) 120777 root root 0 4294967295 emacs-x11.pdmp
 /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp 0 (none) 100644 root root 0 4294967295 
 /usr/share/X11/app-defaults 0 (none) 40755 root root 0 4294967295 
 /usr/share/X11/app-defaults/Emacs 0 (none) 100644 root root 0 4294967295 
comparing file checksum
creating rename script
RPM meta information is different
Extracting packages
/usr/bin/emacs-x11 differs in ELF sections
--- old /usr/bin/emacs-x11 (objdump)
+++ new /usr/bin/emacs-x11 (objdump)
@@ -519888,8 +519888,8 @@
  7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
  7f1420 00400000 00000000 00000000 00000000  .@..............
  7f1430 00000000 00000000 00000000 00000000  ................
- 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
- 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
+ 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
+ 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
  7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
  7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
  7f1480 12000000 00000000 00000000 00000000  ................
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp differs at offset '60273' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp (hex)
@@ -1,7 +1,7 @@
 0000eb40  00 50 00 03 00 00 00 40  08 e7 00 00 00 00 00 00  |.P.....@........|
 0000eb50  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 0000eb60  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
-0000eb70  6e 5f e0 f4 fb 50 28 5e  02 28 6b ee 00 00 00 00  |n_...P(^.(k.....|
+0000eb70  f2 9b bf e6 e2 f1 30 5e  02 28 6b ee 00 00 00 00  |......0^.(k.....|
 0000eb80  b0 c2 77 00 00 00 00 00  90 eb 00 00 00 00 00 00  |..w.............|
 0000eb90  f0 ee 00 00 00 00 00 00  a0 eb 00 00 00 00 00 00  |................|
 0000eba0  70 ee 00 00 00 00 00 00  b0 eb 00 00 00 00 00 00  |p...............|
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp differs at offset '60274' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp (hex)
@@ -1,7 +1,7 @@
 0000eb40  00 50 00 03 00 00 00 40  08 e7 00 00 00 00 00 00  |.P.....@........|
 0000eb50  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 0000eb60  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
-0000eb70  66 11 30 2f 7e 51 28 5e  02 28 6b ee 00 00 00 00  |f.0/~Q(^.(k.....|
+0000eb70  66 67 9e 6e 3c f2 30 5e  02 28 6b ee 00 00 00 00  |fg.n<.0^.(k.....|
 0000eb80  08 8f 76 00 00 00 00 00  90 eb 00 00 00 00 00 00  |..v.............|
 0000eb90  f0 ee 00 00 00 00 00 00  a0 eb 00 00 00 00 00 00  |................|
 0000eba0  70 ee 00 00 00 00 00 00  b0 eb 00 00 00 00 00 00  |p...............|
+ compareresult=1
+ [[ 1 = 0 ]]
+ overall=1
+ for f in $origrpms
++ echo etags-29.1-4.1.x86_64.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='etags-29.1-*.x86_64.rpm'
+ f1=binaries/etags-29.1-4.1.x86_64.rpm
+ f2='binaries.nachbau/etags-29.1-*.x86_64.rpm'
+ cmp binaries/etags-29.1-4.1.x86_64.rpm binaries.nachbau/etags-29.1-4.1.x86_64.rpm
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=7
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/etags-29.1-4.1.x86_64.rpm binaries.nachbau/etags-29.1-4.1.x86_64.rpm
+ compareresult=0
+ [[ 0 = 0 ]]
+ echo 7
+ echo 'overalldiffered=7 (number of pkgs that are not bit-by-bit identical: 0 is good)'
+ echo overall=1
+ echo 1
+ exit 7
Missing in old package: /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11-a2c05c3451540c6deae96bed21a90a3cc0cf321ae7c634d289ae77912fe90967.pdmp  0
Missing in old package: /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11-a2c05c3451540c6deae96bed21a90a3cc0cf321ae7c634d289ae77912fe90967.pdmp  0
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp differs at offset '17' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
@@ -1,6 +1,6 @@
 00000000  44 55 4d 50 45 44 47 4e  55 45 4d 41 43 53 00 00  |DUMPEDGNUEMACS..|
-00000010  81 e3 b8 e4 33 01 1e c5  18 db 85 f2 95 86 19 7c  |....3..........||
-00000020  be 6f b7 03 f7 61 af d4  c6 d7 bd 74 f2 16 8b e2  |.o...a.....t....|
+00000010  a2 c0 5c 34 51 54 0c 6d  ea e9 6b ed 21 a9 0a 3c  |..\4QT.m..k.!..<|
+00000020  c0 cf 32 1a e7 c6 34 d2  89 ae 77 91 2f e9 09 67  |..2...4...w./..g|
 00000030  98 ce c2 00 bc d7 08 00  88 2d e6 00 5b 00 00 00  |.........-..[...|
 00000040  f8 2e e6 00 b8 16 00 00  d8 89 e6 00 e3 e9 03 00  |................|
 00000050  68 31 f6 00 03 0d 00 00  20 4c 98 00 00 00 9c 00  |h1...... L......|
binaries/etags-29.1-4.1.x86_64.rpm binaries.nachbau/etags-29.1-4.1.x86_64.rpm differ: byte 225, line 1
Comparing etags-29.1-4.1.x86_64.rpm to etags-29.1-4.1.x86_64.rpm
comparing the rpm tags of etags
RPM meta information is identical
Extracting packages
Package content is identical
overalldiffered=7 (number of pkgs that are not bit-by-bit identical: 0 is good)
overall=1
processing .
result:emacs=7
result:emacs=notbuilt

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>>> One example of pdmp of one of the variants:
>>> /usr/bin/emacs-x11 differs in ELF sections
>>> --- old /usr/bin/emacs-x11 (objdump)
>>> +++ new /usr/bin/emacs-x11 (objdump)
>>> @@ -519888,8 +519888,8 @@
>>>   7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
>>>   7f1420 00400000 00000000 00000000 00000000  .@..............
>>>   7f1430 00000000 00000000 00000000 00000000  ................
>>> - 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
>>> - 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
>>> + 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
>>> + 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
>>>   7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
>>>   7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
>>>   7f1480 12000000 00000000 00000000 00000000  ................
>>
>> That's confusing.  Is that an executable, if you do an objdump?  You
>> write it's an example of pdmp.
> Th Emacs executable also changes each time.
> Oh that's an oversight on my side. Here is the diff:
> /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp differs at offset '17' (GNU/Emacs pdumper image)
> --- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
> +++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
> @@ -1,6 +1,6 @@
>  00000000  44 55 4d 50 45 44 47 4e  55 45 4d 41 43 53 00 00  |DUMPEDGNUEMACS..|
> -00000010  81 e3 b8 e4 33 01 1e c5  18 db 85 f2 95 86 19 7c  |....3..........||
> -00000020  be 6f b7 03 f7 61 af d4  c6 d7 bd 74 f2 16 8b e2  |.o...a.....t....|
> +00000010  a2 c0 5c 34 51 54 0c 6d  ea e9 6b ed 21 a9 0a 3c  |..\4QT.m..k.!..<|
> +00000020  c0 cf 32 1a e7 c6 34 d2  89 ae 77 91 2f e9 09 67  |..2...4...w./..g|
>  00000030  98 ce c2 00 bc d7 08 00  88 2d e6 00 5b 00 00 00  |.........-..[...|
>  00000040  f8 2e e6 00 b8 16 00 00  d8 89 e6 00 e3 e9 03 00  |................|
>  00000050  68 31 f6 00 03 0d 00 00  20 4c 98 00 00 00 9c 00
>  |h1...... L......|

Offset 17 is the stat of the dump fingerprint (the first 16 bytes are a
magic number).

struct dump_header
{
  /* File type magic.  */
  char magic[sizeof (dump_magic)];

  /* Associated Emacs binary.  */
  unsigned char fingerprint[sizeof fingerprint];

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Mon, 09 Oct 2023 06:19:16 +0200
> 
> Björn Bidar <bjorn.bidar@thaodan.de> writes:
> 
> > Gerd Möllmann <gerd.moellmann@gmail.com> writes:
> >
> >>> One example of pdmp of one of the variants:
> >>> /usr/bin/emacs-x11 differs in ELF sections
> >>> --- old /usr/bin/emacs-x11 (objdump)
> >>> +++ new /usr/bin/emacs-x11 (objdump)
> >>> @@ -519888,8 +519888,8 @@
> >>>   7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
> >>>   7f1420 00400000 00000000 00000000 00000000  .@..............
> >>>   7f1430 00000000 00000000 00000000 00000000  ................
> >>> - 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
> >>> - 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
> >>> + 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
> >>> + 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
> >>>   7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
> >>>   7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
> >>>   7f1480 12000000 00000000 00000000 00000000  ................
> >>
> >> That's confusing.  Is that an executable, if you do an objdump?  You
> >> write it's an example of pdmp.
> > Th Emacs executable also changes each time.
> > Oh that's an oversight on my side. Here is the diff:
> > /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp differs at offset '17' (GNU/Emacs pdumper image)
> > --- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
> > +++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
> > @@ -1,6 +1,6 @@
> >  00000000  44 55 4d 50 45 44 47 4e  55 45 4d 41 43 53 00 00  |DUMPEDGNUEMACS..|
> > -00000010  81 e3 b8 e4 33 01 1e c5  18 db 85 f2 95 86 19 7c  |....3..........||
> > -00000020  be 6f b7 03 f7 61 af d4  c6 d7 bd 74 f2 16 8b e2  |.o...a.....t....|
> > +00000010  a2 c0 5c 34 51 54 0c 6d  ea e9 6b ed 21 a9 0a 3c  |..\4QT.m..k.!..<|
> > +00000020  c0 cf 32 1a e7 c6 34 d2  89 ae 77 91 2f e9 09 67  |..2...4...w./..g|
> >  00000030  98 ce c2 00 bc d7 08 00  88 2d e6 00 5b 00 00 00  |.........-..[...|
> >  00000040  f8 2e e6 00 b8 16 00 00  d8 89 e6 00 e3 e9 03 00  |................|
> >  00000050  68 31 f6 00 03 0d 00 00  20 4c 98 00 00 00 9c 00
> >  |h1...... L......|
> 
> Offset 17 is the stat of the dump fingerprint (the first 16 bytes are a
> magic number).
> 
> struct dump_header
> {
>   /* File type magic.  */
>   char magic[sizeof (dump_magic)];
> 
>   /* Associated Emacs binary.  */
>   unsigned char fingerprint[sizeof fingerprint];

The fingerprint is supposed to remain unchanged if you rebuild the
same code base.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> Offset 17 is the stat of the dump fingerprint (the first 16 bytes are a
>> magic number).
>> 
>> struct dump_header
>> {
>>   /* File type magic.  */
>>   char magic[sizeof (dump_magic)];
>> 
>>   /* Associated Emacs binary.  */
>>   unsigned char fingerprint[sizeof fingerprint];
>
> The fingerprint is supposed to remain unchanged if you rebuild the
> same code base.

Right, that's the problem -- apparently it isn't unchanged in this case
as the diff of the executables' objdump shows.

My hypothesis is that x-win.eln changes between builds (print-circle
etc.) => make-fingerprint computes a different ingerprint and so on.

But that's a hypothesis, I can't check that on my system.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: bjorn.bidar@thaodan.de,  emacs-devel@gnu.org
> Date: Mon, 09 Oct 2023 13:26:03 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Offset 17 is the stat of the dump fingerprint (the first 16 bytes are a
> >> magic number).
> >> 
> >> struct dump_header
> >> {
> >>   /* File type magic.  */
> >>   char magic[sizeof (dump_magic)];
> >> 
> >>   /* Associated Emacs binary.  */
> >>   unsigned char fingerprint[sizeof fingerprint];
> >
> > The fingerprint is supposed to remain unchanged if you rebuild the
> > same code base.
> 
> Right, that's the problem -- apparently it isn't unchanged in this case
> as the diff of the executables' objdump shows.
> 
> My hypothesis is that x-win.eln changes between builds (print-circle
> etc.) => make-fingerprint computes a different ingerprint and so on.

If x-win.el didn't change, x-win.eln will not be regenerated, so
something else is at work here.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: bjorn.bidar@thaodan.de,  emacs-devel@gnu.org
>> Date: Mon, 09 Oct 2023 13:26:03 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Offset 17 is the stat of the dump fingerprint (the first 16 bytes are a
>> >> magic number).
>> >> 
>> >> struct dump_header
>> >> {
>> >>   /* File type magic.  */
>> >>   char magic[sizeof (dump_magic)];
>> >> 
>> >>   /* Associated Emacs binary.  */
>> >>   unsigned char fingerprint[sizeof fingerprint];
>> >
>> > The fingerprint is supposed to remain unchanged if you rebuild the
>> > same code base.
>> 
>> Right, that's the problem -- apparently it isn't unchanged in this case
>> as the diff of the executables' objdump shows.
>> 
>> My hypothesis is that x-win.eln changes between builds (print-circle
>> etc.) => make-fingerprint computes a different ingerprint and so on.
>
> If x-win.el didn't change, x-win.eln will not be regenerated, so
> something else is at work here.

But x-win.elc _did_ change.  To cite one of Björn's mails

> Strangely the only eln file that changes is x-win.el:
> /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln differs in ELF sections
> --- old /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
> +++ new /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
> @@ -2950,7 +2950,7 @@
>   0e7c0 6f726564 20736573 73696f6e 20646174  ored session dat
>   0e7d0 61222022 4865776c 6574742d 5061636b  a" "Hewlett-Pack
>   0e7e0 61726420 496e636f 72706f72 61746564  ard Incorporated
> - 0e7f0 22202334 3d282831 3638202e 206d7574  " #4=((168 . mut
> + 0e7f0 22202336 3d282831 3638202e 206d7574  " #6=((168 . mut
>   0e800 652d6163 75746529 20283136 39202e20  e-acute) (169 . 
>   0e810 6d757465 2d677261 76652920 28313730  mute-grave) (170
>   0e820 202e206d 7574652d 61736369 69636972   . mute-asciicir
> @@ -2976,7 +2976,7 @@
>   0e960 20224865 776c6574 742d5061 636b6172   "Hewlett-Packar
>   0e970 6420436f 6d70616e 79222022 5831312f  d Company" "X11/
>   0e980 4e655753 202d2053 756e204d 6963726f  NeWS - Sun Micro
> - 0e990 73797374 656d7320 496e632e 22202336  systems Inc." #6
> + 0e990 73797374 656d7320 496e632e 22202334  systems Inc." #4
>   0e9a0 3d282833 39323937 36202e20 66333629  =((392976 . f36)
>   0e9b0 20283339 32393737 202e2066 33372920   (392977 . f37) 
>   0e9c0 28333933 30353620 2e207265 71292028  (393056 . req) (

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> But x-win.elc _did_ change.  To cite one of Björn's mails

I meant .eln, of course.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: bjorn.bidar@thaodan.de,  emacs-devel@gnu.org
> Date: Mon, 09 Oct 2023 13:55:30 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> My hypothesis is that x-win.eln changes between builds (print-circle
> >> etc.) => make-fingerprint computes a different ingerprint and so on.
> >
> > If x-win.el didn't change, x-win.eln will not be regenerated, so
> > something else is at work here.
> 
> But x-win.elc _did_ change.  To cite one of Björn's mails

If x-win.elc or x-win.eln change between builds (meaning x-win.el or
one of the *.el files it 'require's changed), why do we expect the
builds to be identical?

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: bjorn.bidar@thaodan.de,  emacs-devel@gnu.org
>> Date: Mon, 09 Oct 2023 13:55:30 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> My hypothesis is that x-win.eln changes between builds (print-circle
>> >> etc.) => make-fingerprint computes a different ingerprint and so on.
>> >
>> > If x-win.el didn't change, x-win.eln will not be regenerated, so
>> > something else is at work here.
>> 
>> But x-win.elc _did_ change.  To cite one of Björn's mails
>
> If x-win.elc or x-win.eln change between builds (meaning x-win.el or
> one of the *.el files it 'require's changed), why do we expect the
> builds to be identical?

AIAIU, the assumption that an .el file has changed doesn't hold.
Instead, comp.el prints a constant list and produces a different $n=
label in both cases.  The diff of the .eln's shows that.  If there are
no further diffs in the .eln that weren't shown, everything else is
identical.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Cc: bjorn.bidar@thaodan.de,  emacs-devel@gnu.org
> Date: Mon, 09 Oct 2023 15:33:59 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> >> Cc: bjorn.bidar@thaodan.de,  emacs-devel@gnu.org
> >> Date: Mon, 09 Oct 2023 13:55:30 +0200
> >> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> >> My hypothesis is that x-win.eln changes between builds (print-circle
> >> >> etc.) => make-fingerprint computes a different ingerprint and so on.
> >> >
> >> > If x-win.el didn't change, x-win.eln will not be regenerated, so
> >> > something else is at work here.
> >> 
> >> But x-win.elc _did_ change.  To cite one of Björn's mails
> >
> > If x-win.elc or x-win.eln change between builds (meaning x-win.el or
> > one of the *.el files it 'require's changed), why do we expect the
> > builds to be identical?
> 
> AIAIU, the assumption that an .el file has changed doesn't hold.
> Instead, comp.el prints a constant list and produces a different $n=
> label in both cases.  The diff of the .eln's shows that.  If there are
> no further diffs in the .eln that weren't shown, everything else is
> identical.

Once again: why is x-win.el being ELN-compiled in the first place?
Are you saying that every time you say "make", we compile all the
preloaded *.el files, even if they didn't change?  That is definitely
not true.  So the question IMO is: why is x-win.eln being regenerated?

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Eli Zaretskii <eliz@gnu.org> writes:

>> AIAIU, the assumption that an .el file has changed doesn't hold.
>> Instead, comp.el prints a constant list and produces a different $n=
>> label in both cases.  The diff of the .eln's shows that.  If there are
>> no further diffs in the .eln that weren't shown, everything else is
>> identical.
>
> Once again: why is x-win.el being ELN-compiled in the first place?
> Are you saying that every time you say "make", we compile all the
> preloaded *.el files, even if they didn't change?  That is definitely
> not true.  So the question IMO is: why is x-win.eln being regenerated?

I think we are miscommunicating.

I build nothing, Björn does.  As I understand him, he builds 2 times
_from scratch_, finds that the builds have diffs, which he showed, and
asked what could be the cause, which I answered to.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Gerd Möllmann <gerd.moellmann@gmail.com>
>> Cc: bjorn.bidar@thaodan.de,  emacs-devel@gnu.org
>> Date: Mon, 09 Oct 2023 13:55:30 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> My hypothesis is that x-win.eln changes between builds (print-circle
>> >> etc.) => make-fingerprint computes a different ingerprint and so on.
>> >
>> > If x-win.el didn't change, x-win.eln will not be regenerated, so
>> > something else is at work here.
>> 
>> But x-win.elc _did_ change.  To cite one of Björn's mails
>
> If x-win.elc or x-win.eln change between builds (meaning x-win.el or
> one of the *.el files it 'require's changed), why do we expect the
> builds to be identical?

x-win.elc didn't change only x-win.eln (and the emacs executeable and pdmp) did. 

All uncompiled and the byte-compiled Emacs Lisp is inside the emacs-el
package which is identical.

See my message here:
https://mail.gnu.org/archive/html/emacs-devel/2023-10/msg00133.html

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Björn Bidar <bjorn.bidar@thaodan.de>
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>  emacs-devel@gnu.org
> Date: Mon, 09 Oct 2023 17:48:05 +0300
> 
> x-win.elc didn't change only x-win.eln (and the emacs executeable and pdmp) did. 

Then I think this is not an Emacs problem, this is a GCC/Binutils
problem.  Are your GCC and Binutils built in a way that they generate
reproducible binaries?  (AFAIR, by default compiling a source file
with GCC embeds date and time of the build inside the binary
products.)

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Björn Bidar <bjorn.bidar@thaodan.de>
>> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>>  emacs-devel@gnu.org
>> Date: Mon, 09 Oct 2023 17:48:05 +0300
>> 
>> x-win.elc didn't change only x-win.eln (and the emacs executeable and pdmp) did. 
>
> Then I think this is not an Emacs problem, this is a GCC/Binutils
> problem.  Are your GCC and Binutils built in a way that they generate
> reproducible binaries?

Yes. Other packages already show this in action.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> Offset 17 is the stat of the dump fingerprint (the first 16 bytes are a
>>> magic number).
>>> 
>>> struct dump_header
>>> {
>>>   /* File type magic.  */
>>>   char magic[sizeof (dump_magic)];
>>> 
>>>   /* Associated Emacs binary.  */
>>>   unsigned char fingerprint[sizeof fingerprint];
>>
>> The fingerprint is supposed to remain unchanged if you rebuild the
>> same code base.
>
> Right, that's the problem -- apparently it isn't unchanged in this case
> as the diff of the executables' objdump shows.
>
> My hypothesis is that x-win.eln changes between builds (print-circle
> etc.) => make-fingerprint computes a different ingerprint and so on

But the emacs execute and pdmp even changes for builds without native-comp, see my earlier message:
https://mail.gnu.org/archive/html/emacs-devel/2023-10/msg00133.html

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

>> My hypothesis is that x-win.eln changes between builds (print-circle
>> etc.) => make-fingerprint computes a different ingerprint and so on
>
> But the emacs execute and pdmp even changes for builds without
> native-comp, see my earlier message:
> https://mail.gnu.org/archive/html/emacs-devel/2023-10/msg00133.html

Is that the right link?  Where does it say that that is w/o native-comp?
Anyway, so the facts are which now?

- all builds are from scratch?
- Executable and pdmp change with every build, independent of
  native-comp?
- x-win.eln changes with native-comp?

Correct?

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Björn Bidar <bjorn.bidar@thaodan.de>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Mon, 09 Oct 2023 18:25:42 +0300
> 
> > - Executable and pdmp change with every build, independent of
> >   native-comp?
> yes

You build each time in the exact same directory?  That is, remove the
source tree, recreate it again in the same place under the same name,
and then rebuild?

I see that you use "make -j4", what happens if you don't use the -j4
option (i.e. build in a deterministic sequential order)?

> Build log without native-comp:
> 
> (build.wnativecomp.log (1MB))

Ouch! who knows what all those additional operations do and what
effect they have on the built Emacs?

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Björn Bidar <bjorn.bidar@thaodan.de>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
>> Date: Mon, 09 Oct 2023 18:25:42 +0300
>> 
>> > - Executable and pdmp change with every build, independent of
>> >   native-comp?
>> yes
>
> You build each time in the exact same directory?  That is, remove the
> source tree, recreate it again in the same place under the same name,
> and then rebuild?

The builder is recreated on each build. Obs sets up a fresh install in
each worker with just the packages required and nothing else but those
and the package sources.

> I see that you use "make -j4", what happens if you don't use the -j4
> option (i.e. build in a deterministic sequential order)?

How should that make a difference?

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Björn Bidar <bjorn.bidar@thaodan.de>
> Cc: gerd.moellmann@gmail.com,  emacs-devel@gnu.org
> Date: Mon, 09 Oct 2023 19:06:41 +0300
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Björn Bidar <bjorn.bidar@thaodan.de>
> >> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> >> Date: Mon, 09 Oct 2023 18:25:42 +0300
> >> 
> >> > - Executable and pdmp change with every build, independent of
> >> >   native-comp?
> >> yes
> >
> > You build each time in the exact same directory?  That is, remove the
> > source tree, recreate it again in the same place under the same name,
> > and then rebuild?
> 
> The builder is recreated on each build. Obs sets up a fresh install in
> each worker with just the packages required and nothing else but those
> and the package sources.

I don't think this answers my question.  Are the absolute file names
of each file in the source tree exactly identical in all the builds,
including all the leading directories?  If not, that can explain some
differences.

> > I see that you use "make -j4", what happens if you don't use the -j4
> > option (i.e. build in a deterministic sequential order)?
> 
> How should that make a difference?

"make -j4" can compile files in different order, so when file1.el is
byte-compiled and 'require's file2.el, file2.el may or may not be
already byte-compiled, so byte-compilation of file1.el may load
file2.el or file2.elc, thus producing some differences.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Björn Bidar <bjorn.bidar@thaodan.de>
>> Cc: gerd.moellmann@gmail.com,  emacs-devel@gnu.org
>> Date: Mon, 09 Oct 2023 19:06:41 +0300
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: Björn Bidar <bjorn.bidar@thaodan.de>
>> >> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
>> >> Date: Mon, 09 Oct 2023 18:25:42 +0300
>> >> 
>> >> > - Executable and pdmp change with every build, independent of
>> >> >   native-comp?
>> >> yes
>> >
>> > You build each time in the exact same directory?  That is, remove the
>> > source tree, recreate it again in the same place under the same name,
>> > and then rebuild?
>> 
>> The builder is recreated on each build. Obs sets up a fresh install in
>> each worker with just the packages required and nothing else but those
>> and the package sources.
>
> I don't think this answers my question.  Are the absolute file names
> of each file in the source tree exactly identical in all the builds,
> including all the leading directories?  If not, that can explain some
> differences.

They are exactly the same. That's the point it was trying to make, each
builder has a fresh set of packages, plus the extracted sources.

>> > I see that you use "make -j4", what happens if you don't use the -j4
>> > option (i.e. build in a deterministic sequential order)?
>> 
>> How should that make a difference?
>
> "make -j4" can compile files in different order, so when file1.el is
> byte-compiled and 'require's file2.el, file2.el may or may not be
> already byte-compiled, so byte-compilation of file1.el may load
> file2.el or file2.elc, thus producing some differences.
OK I try if that makes a difference.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> Björn Bidar <bjorn.bidar@thaodan.de> writes:
>>
>>>> My hypothesis is that x-win.eln changes between builds (print-circle
>>>> etc.) => make-fingerprint computes a different ingerprint and so on
>>>
>>> But the emacs execute and pdmp even changes for builds without
>>> native-comp, see my earlier message:
>>> https://mail.gnu.org/archive/html/emacs-devel/2023-10/msg00133.html
>>
>> Is that the right link?  Where does it say that that is w/o native-comp?
>> Anyway, so the facts are which now?
>
> The link only contains the diff. I attach the full build log now yes.

To be honest, I don't understand what you mean by the first sentence.
The linked message didn't say it's w/o native-compilation?

>
>> - all builds are from scratch?
>
> Yes they are from scratch in a clean build environment, a open build
> service worker (see attachment).
>
>> - Executable and pdmp change with every build, independent of
>>   native-comp?
> yes
>> - x-win.eln changes with native-comp?
> yes

Let me add another question, just to be as precise as possible:

I understand We have differences between 2 independent builds with
native compilation.  When you say executable and pdmp differ without
native compilation, do you mean between 2 builds both without native
comp?

> Build log without native-comp:

Not of great help, I'm afraid.  Lots of stuff nobody wants to know, here
at least :-).

Anyway--what are these CFLAGS for, esp. -DPDMP_BASE=...?  I don't know
what they do.

> [  187s] + CFLAGS='-O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -D_GNU_SOURCE -DGDK_DISABLE_DEPRECATION_WARNINGS -DGLIB_DISABLE_DEPRECATION_WARNINGS -pipe -Wno-pointer-sign -Wno-unused-variable -Wno-unused-label -fno-optimize-sibling-calls -DPDMP_BASE='\''"emacs-wayland"'\'''
> [  187s] + ./configure --disable-build-details ...

I'd recommend simplifying things by building Emacs with standard options
(no CFLAGS=...) and without any of your build tools.  Just 2 clean build
directories, and then compare.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Björn Bidar <bjorn.bidar@thaodan.de> writes:
>
>> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>>
>>> Björn Bidar <bjorn.bidar@thaodan.de> writes:
>>>
>>>>> My hypothesis is that x-win.eln changes between builds (print-circle
>>>>> etc.) => make-fingerprint computes a different ingerprint and so on
>>>>
>>>> But the emacs execute and pdmp even changes for builds without
>>>> native-comp, see my earlier message:
>>>> https://mail.gnu.org/archive/html/emacs-devel/2023-10/msg00133.html
>>>
>>> Is that the right link?  Where does it say that that is w/o native-comp?
>>> Anyway, so the facts are which now?
>>
>> The link only contains the diff. I attach the full build log now yes.
>
> To be honest, I don't understand what you mean by the first sentence.
> The linked message didn't say it's w/o native-compilation?

The first log I send only did contain the log of the diff between the
two builds. The attached log was the full build log.

>>
>>> - all builds are from scratch?
>>
>> Yes they are from scratch in a clean build environment, a open build
>> service worker (see attachment).
>>
>>> - Executable and pdmp change with every build, independent of
>>>   native-comp?
>> yes
>>> - x-win.eln changes with native-comp?
>> yes
>
> Let me add another question, just to be as precise as possible:
>
> I understand We have differences between 2 independent builds with
> native compilation.  When you say executable and pdmp differ without
> native compilation, do you mean between 2 builds both without native
> comp?
>
>> Build log without native-comp:
>
> Not of great help, I'm afraid.  Lots of stuff nobody wants to know, here
> at least :-).
>
> Anyway--what are these CFLAGS for, esp. -DPDMP_BASE=...?  I don't know
> what they do.

These are general cflags (-pipe), harding and lto cflags but they have
no impact on Emacs being reproduceable or not.

PDMP base is so that each variant of the Emacs binary picks up the right
pdmp, again no impact on being reproduceable or not. 

>> [ 187s] + CFLAGS='-O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3
>> -fstack-protector-strong -funwind-tables
>> -fasynchronous-unwind-tables -fstack-clash-protection
>> -Werror=return-type -flto=auto -g -D_GNU_SOURCE
>> -DGDK_DISABLE_DEPRECATION_WARNINGS
>> -DGLIB_DISABLE_DEPRECATION_WARNINGS -pipe -Wno-pointer-sign
>> -Wno-unused-variable -Wno-unused-label -fno-optimize-sibling-calls
>> -DPDMP_BASE='\''"emacs-wayland"'\'''
>> [  187s] + ./configure --disable-build-details ...
>
> I'd recommend simplifying things by building Emacs with standard options
> (no CFLAGS=...) and without any of your build tools.  Just 2 clean build
> directories, and then compare.

I don't think how that simplifies things, cflags don't impact if the
build is reproduceable or not.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

>> Let me add another question, just to be as precise as possible:
>>
>> I understand We have differences between 2 independent builds with
>> native compilation.  When you say executable and pdmp differ without
>> native compilation, do you mean between 2 builds both without native
>> comp?

What about this?

>> I'd recommend simplifying things by building Emacs with standard options
>> (no CFLAGS=...) and without any of your build tools.  Just 2 clean build
>> directories, and then compare.
>
> I don't think how that simplifies things, cflags don't impact if the
> build is reproduceable or not.

You know that I'm not talking about CFLAGS alone?

But I'm just giving you an advice.  You can of course free to ignore it.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Alan Mackenzie]

Hello, everyone.

On Mon, Oct 09, 2023 at 19:19:23 +0200, Gerd Möllmann wrote:
> Björn Bidar <bjorn.bidar@thaodan.de> writes:

> >> Let me add another question, just to be as precise as possible:
> >>
> >> I understand We have differences between 2 independent builds with
> >> native compilation.  When you say executable and pdmp differ without
> >> native compilation, do you mean between 2 builds both without native
> >> comp?

> What about this?

> >> I'd recommend simplifying things by building Emacs with standard options
> >> (no CFLAGS=...) and without any of your build tools.  Just 2 clean build
> >> directories, and then compare.
> >
> > I don't think how that simplifies things, cflags don't impact if the
> > build is reproduceable or not.

> You know that I'm not talking about CFLAGS alone?

> But I'm just giving you an advice.  You can of course free to ignore it.

For what it's worth, I just built Emacs twice on one of my development
repositories on GNU/Linux, and the two emacsen produced were identical,
checked by cmp and md5sum.

Each build was set off with

    $ make -j17 bootstrap

..  My configuration was created by:

    $ configure --with-gpm --with-native-compilation --with-gif=no --with-tiff=no

..  Maybe I don't understand what the problem is, but I'm seeing
repeatable builds.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Alan Mackenzie <acm@muc.de> writes:

>     $ configure --with-gpm --with-native-compilation --with-gif=no
>     --with-tiff=no

I think he's using --with-native-compilation=aot

Re: How to make aot native-compilation and pdmp creation reproduceable

[Alan Mackenzie]

Hello, Gerd.

On Tue, Oct 10, 2023 at 08:21:51 +0200, Gerd Möllmann wrote:
> Alan Mackenzie <acm@muc.de> writes:

> >     $ configure --with-gpm --with-native-compilation --with-gif=no
> >     --with-tiff=no

> I think he's using --with-native-compilation=aot

Yes.  I've tried it again with aot, and get the same binary twice over.

However, when I build the same source tree in two different directories,
the results differ.  In native-lisp/30.0.50-15c37793, there are ~22 .eln
files differing between the two trees.  As the OP reported, one of the
differing files is native-lisp/30.0.50-15c37793/preloaded/x-win-....eln.

The other pertinent files are: cmacexp, cpp, dictionary, el, erc,
erc-button, erc-nicks, extract, find, grammar, ia, insert, make, mode,
oc-csl, org-habit, ox-html, semantic, srt, tramp-message, utf7 (all
appended with their hex signatures and .eln).

What do these files have in common?  As pure speculation, it may be that
they communicate with external processes, somehow.  Or something like
that.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Gerd.

Hi Alan,

>
> On Tue, Oct 10, 2023 at 08:21:51 +0200, Gerd Möllmann wrote:
>> Alan Mackenzie <acm@muc.de> writes:
>
>> >     $ configure --with-gpm --with-native-compilation --with-gif=no
>> >     --with-tiff=no
>
>> I think he's using --with-native-compilation=aot
>
> Yes.  I've tried it again with aot, and get the same binary twice
> over.
>
> However, when I build the same source tree in two different directories,
> the results differ.  In native-lisp/30.0.50-15c37793, there are ~22 .eln
> files differing between the two trees.  As the OP reported, one of the
> differing files is
> native-lisp/30.0.50-15c37793/preloaded/x-win-....eln.
>
> The other pertinent files are: cmacexp, cpp, dictionary, el, erc,
> erc-button, erc-nicks, extract, find, grammar, ia, insert, make, mode,
> oc-csl, org-habit, ox-html, semantic, srt, tramp-message, utf7 (all
> appended with their hex signatures and .eln).
>
> What do these files have in common?  As pure speculation, it may be that
> they communicate with external processes, somehow.  Or something like
> that.

Hm.  If my hypothesis of comp.el printing with print-circle etc. is
right, they would differ in some label of the form '#nnn=' only, where
nnn is some integer.

If I understood how print-continuous-numbering + print-number-table
actually work and should be used in comp.el, I'd offer a patch, but alas
... :-).

Re: How to make aot native-compilation and pdmp creation reproduceable

[Michael Albinus]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Gerd.

Hi Alan,

> However, when I build the same source tree in two different directories,
> the results differ.  In native-lisp/30.0.50-15c37793, there are ~22 .eln
> files differing between the two trees.  As the OP reported, one of the
> differing files is native-lisp/30.0.50-15c37793/preloaded/x-win-....eln.
>
> The other pertinent files are: cmacexp, cpp, dictionary, el, erc,
> erc-button, erc-nicks, extract, find, grammar, ia, insert, make, mode,
> oc-csl, org-habit, ox-html, semantic, srt, tramp-message, utf7 (all
> appended with their hex signatures and .eln).
>
> What do these files have in common?  As pure speculation, it may be that
> they communicate with external processes, somehow.  Or something like
> that.

In tramp-message, the variables emacs-version, tramp-version,
tramp-repository-branch and tramp-repository-version are
used. Furthermore, (locate-library "tramp") is called. All of this
should happen at runtime, but perhaps there are compile time
optimizations?

Best regards, Michael.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> Date: Tue, 10 Oct 2023 10:39:06 +0000
> Cc: Björn Bidar <bjorn.bidar@thaodan.de>,
>   Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> However, when I build the same source tree in two different directories,
> the results differ.  In native-lisp/30.0.50-15c37793, there are ~22 .eln
> files differing between the two trees.  As the OP reported, one of the
> differing files is native-lisp/30.0.50-15c37793/preloaded/x-win-....eln.

This is expected, since the absolute file names are recorded in
various places of the Emacs binary and the pdumper file.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Alan Mackenzie]

Hello, Eli.

On Tue, Oct 10, 2023 at 15:17:39 +0300, Eli Zaretskii wrote:
> > Date: Tue, 10 Oct 2023 10:39:06 +0000
> > Cc: Björn Bidar <bjorn.bidar@thaodan.de>,
> >   Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > However, when I build the same source tree in two different directories,
> > the results differ.  In native-lisp/30.0.50-15c37793, there are ~22 .eln
> > files differing between the two trees.  As the OP reported, one of the
> > differing files is native-lisp/30.0.50-15c37793/preloaded/x-win-....eln.

> This is expected, since the absolute file names are recorded in
> various places of the Emacs binary and the pdumper file.

Ah.  Indeed this is the case.  There is even an absolute file name in
cc-mode.elc, the place where a doc string is to be found.  :-(

Given this, it would seem to be premature to try and build reproducible
builds if the build directory name is recorded in the binary.  Surely we
need to identify all the places this happens and tidy these up, first.
That's assuming a reproducible build is actually required.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: How to make aot native-compilation and pdmp creation reproduceable

[Andrea Corallo]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Gerd.
>
> On Tue, Oct 10, 2023 at 08:21:51 +0200, Gerd Möllmann wrote:
>> Alan Mackenzie <acm@muc.de> writes:
>
>> >     $ configure --with-gpm --with-native-compilation --with-gif=no
>> >     --with-tiff=no
>
>> I think he's using --with-native-compilation=aot
>
> Yes.  I've tried it again with aot, and get the same binary twice over.
>
> However, when I build the same source tree in two different directories,
> the results differ.  In native-lisp/30.0.50-15c37793, there are ~22 .eln
> files differing between the two trees.  As the OP reported, one of the
> differing files is native-lisp/30.0.50-15c37793/preloaded/x-win-....eln.

Certanly the directory Emacs is built in does make a difference in what
we store into the eln files.

At loadup.el:534 `invocation-directory' comes into play when we fixup
native compilation units before dump.  This is to have Emacs (when we
resurrect from the dump file) working if installed or if the whole build
directory was moved.

Best Regards

  Andrea

Re: How to make aot native-compilation and pdmp creation reproduceable

[Ulrich Mueller]

>>>>> On Mon, 09 Oct 2023, Alan Mackenzie wrote:

> For what it's worth, I just built Emacs twice on one of my development
> repositories on GNU/Linux, and the two emacsen produced were identical,
> checked by cmp and md5sum.

> Each build was set off with

>     $ make -j17 bootstrap

> ..  My configuration was created by:

>     $ configure --with-gpm --with-native-compilation --with-gif=no --with-tiff=no

> ..  Maybe I don't understand what the problem is, but I'm seeing
> repeatable builds.

When building on a Gentoo system, I get a reproducible emacs binary,
but the dump is different every time:

$ USE="-*" MAKEOPTS="-j1" EVCS_OFFLINE=1 ebuild emacs-30.0.9999.ebuild clean compile
[...]
 * Emacs branch: master
 * Commit: 336c3674119f61bd78a056476769ce83b97230bb
[...]
$ md5sum /tmp/portage/app-editors/emacs-30.0.9999/work/emacs/src/emacs{,.pdmp}
805903ff5a7a5efc289e1e66240cddff  /tmp/portage/app-editors/emacs-30.0.9999/work/emacs/src/emacs
d3a98def495185bb818f6521f36417e0  /tmp/portage/app-editors/emacs-30.0.9999/work/emacs/src/emacs.pdmp

$ USE="-*" MAKEOPTS="-j1" EVCS_OFFLINE=1 ebuild emacs-30.0.9999.ebuild clean compile
[...]
 * Emacs branch: master
 * Commit: 336c3674119f61bd78a056476769ce83b97230bb
[...]
$ md5sum /tmp/portage/app-editors/emacs-30.0.9999/work/emacs/src/emacs{,.pdmp}
805903ff5a7a5efc289e1e66240cddff  /tmp/portage/app-editors/emacs-30.0.9999/work/emacs/src/emacs
2f9c59181684c3070a1684cd2d030dea  /tmp/portage/app-editors/emacs-30.0.9999/work/emacs/src/emacs.pdmp

I have verified that all *.el and *.elc files are identical in both
cases. I get different checksums for lib/libgnu.a, which is expected
because the archive contains timestamps.

My configuration is this (with USE="-* almost all options are disabled):

./configure --prefix=/usr --build=x86_64-pc-linux-gnu
--host=x86_64-pc-linux-gnu --mandir=/usr/share/man
--infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc
--localstatedir=/var/lib --datarootdir=/usr/share --disable-silent-rules
--docdir=/usr/share/doc/emacs-30.0.9999
--htmldir=/usr/share/doc/emacs-30.0.9999/html --libdir=/usr/lib64
--program-suffix=-emacs-30-vcs --includedir=/usr/include/emacs-30-vcs
--infodir=/usr/share/info/emacs-30-vcs --localstatedir=/var
--enable-locallisppath=/etc/emacs:/usr/share/emacs/site-lisp
--without-compress-install --without-hesiod --without-pop
--with-file-notification=no --with-pdumper --disable-acl --disable-xattr
--without-dbus --without-modules --without-gameuser --without-libgmp
--without-gpm --without-native-compilation --without-json
--without-kerberos --without-kerberos5 --without-lcms2 --without-xml2
--without-mailutils --without-selinux --without-small-ja-dic
--without-sqlite3 --without-gnutls --without-libsystemd --without-threads
--without-tree-sitter --without-wide-int --with-sound=no --without-zlib
--without-x --without-pgtk --without-ns --with-dumping=pdumper

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Ulrich Mueller <ulm@gentoo.org> writes:

> When building on a Gentoo system, I get a reproducible emacs binary,
> but the dump is different every time:

> --without-gpm --without-native-compilation --without-json

Can you find out where the pdmp files differ?  Maybe the offset or the
contents of the .pdmp files around the diffs tell us something.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Ulrich Mueller]

>>>>> On Tue, 10 Oct 2023, Gerd Möllmann wrote:

>> When building on a Gentoo system, I get a reproducible emacs binary,
>> but the dump is different every time:

>> --without-gpm --without-native-compilation --without-json

> Can you find out where the pdmp files differ?  Maybe the offset or the
> contents of the .pdmp files around the diffs tell us something.

Not sure what to learn from this, but here's the cmp output:

$ cmp -l emacs.pdmp.old emacs.pdmp
   60273 106   6
   60274 103 150
   60275 300 257
   60276  31  42
   60277 153 172
   60278 335 337
 2331737 142  72
 2331738 216 227
 2331753 132 166
 2331754 346 273
 2331755   7   1
 2331769   2 302
 2331770 207 156
 2331771  50  22
 9050945  46  22
 9050946  40  71
 9050947 254 226
 9050948  32 332
 9050949  22 377
 9050950  64 316

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Ulrich Mueller <ulm@gentoo.org> writes:

>>>>>> On Tue, 10 Oct 2023, Gerd Möllmann wrote:
>
>>> When building on a Gentoo system, I get a reproducible emacs binary,
>>> but the dump is different every time:
>
>>> --without-gpm --without-native-compilation --without-json
>
>> Can you find out where the pdmp files differ?  Maybe the offset or the
>> contents of the .pdmp files around the diffs tell us something.
>
> Not sure what to learn from this, but here's the cmp output:
>
> $ cmp -l emacs.pdmp.old emacs.pdmp
>    60273 106   6
>    60274 103 150
>    60275 300 257
>    60276  31  42
>    60277 153 172
>    60278 335 337
>  2331737 142  72
>  2331738 216 227
>  2331753 132 166
>  2331754 346 273
>  2331755   7   1
>  2331769   2 302
>  2331770 207 156
>  2331771  50  22
>  9050945  46  22
>  9050946  40  71
>  9050947 254 226
>  9050948  32 332
>  9050949  22 377
>  9050950  64 316

That's not very enlightning, indeed.  Sorry for the hassle.

Differences in *.elc files (was: Re: How to make aot native-compilation and pdmp creation reproduceable)

[Ulrich Mueller]

Not sure if it is related, but when I compile Emacs with make -j1 and
with make -j8 (both for commit 336c3674119 on master), it results in the
file emacs-lisp/eieio-opt.elc having a different size:

-rw-r--r-- 1 ulm users 8716 Oct 10 12:14 eieio-opt.elc-j1
-rw-r--r-- 1 ulm users 9102 Oct 10 12:23 eieio-opt.elc-j8

Is there a good way to compare byte-compiled files? diff outputs the
following for the first hunk:

$ diff -a -U0 eieio-opt.elc-{j1,j8} | head -n5
--- eieio-opt.elc-j1     2023-10-10 12:14:29.447911151 +0200
+++ eieio-opt.elc-j8     2023-10-10 12:23:58.083301315 +0200
@@ -19 +19 @@
-(defalias 'eieio-browse-tree #[771 "\300!\204\301\302\303\304E\"\210\305!\306N\211\307H\262\310P\311P\312P\313\261\210A\203<\314@#\210A\262\202)\205F\314@#\207" [class-p signal wrong-type-argument class this-root symbol-name cl--class 6 "  +--" "  |  " "     " "\n" eieio-browse-tree] 12 (#$ . 890)])
+(defalias 'eieio-browse-tree #[771 "\301!\204\302\303\304\305E\"\210\306!\307N\310>\204#\302\303\311D\"\210\211\312H\262\313P\314P\315P\316\261\210A\203K\317@#\210A\262\2028\205U\317@#\207" [cl-struct-eieio--class-tags class-p signal wrong-type-argument class this-root symbol-name cl--class type-of eieio--class 6 "  +--" "  |  " "     " "\n" eieio-browse-tree] 12 (#$ . 890)])

After this there are many more differences because all the positions
have changed.

Parallel build problem because of a missing prerequisite?

Re: Differences in *.elc files

[Gerd Möllmann]

Ulrich Mueller <ulm@gentoo.org> writes:

> Not sure if it is related, but when I compile Emacs with make -j1 and
> with make -j8 (both for commit 336c3674119 on master), it results in the
> file emacs-lisp/eieio-opt.elc having a different size:
>
> -rw-r--r-- 1 ulm users 8716 Oct 10 12:14 eieio-opt.elc-j1
> -rw-r--r-- 1 ulm users 9102 Oct 10 12:23 eieio-opt.elc-j8
>
> Is there a good way to compare byte-compiled files? diff outputs the
> following for the first hunk:

I don't know a way.

>
> $ diff -a -U0 eieio-opt.elc-{j1,j8} | head -n5
> --- eieio-opt.elc-j1     2023-10-10 12:14:29.447911151 +0200
> +++ eieio-opt.elc-j8     2023-10-10 12:23:58.083301315 +0200
> @@ -19 +19 @@
> -(defalias 'eieio-browse-tree #[771
> "\300!\204\301\302\303\304E\"\210\305!\306N\211\307H\262\310P\311P\312P\313\261\210A\203<\314@#\210A\262\202)\205F\314@#\207"
> [class-p signal wrong-type-argument class this-root symbol-name
> cl--class 6 " +--" " | " " " "\n" eieio-browse-tree] 12 (#$ . 890)])
> +(defalias 'eieio-browse-tree #[771
> "\301!\204\302\303\304\305E\"\210\306!\307N\310>\204#\302\303\311D\"\210\211\312H\262\313P\314P\315P\316\261\210A\203K\317@#\210A\262\2028\205U\317@#\207"
> [cl-struct-eieio--class-tags class-p signal wrong-type-argument class
> this-root symbol-name cl--class type-of eieio--class 6 " +--" " | " "
> " "\n" eieio-browse-tree] 12 (#$ . 890)])
>
> After this there are many more differences because all the positions
> have changed.
>
> Parallel build problem because of a missing prerequisite?

I guess so.

If I had to, I'd bet on the cl-check-type in eieio-browse-tree, which in
the -j1 and -j8 case seems to expand to different things.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Ulrich Mueller <ulm@gentoo.org>
> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>   Björn Bidar
>  <bjorn.bidar@thaodan.de>,  Eli Zaretskii <eliz@gnu.org>,
>   emacs-devel@gnu.org
> Date: Tue, 10 Oct 2023 10:30:10 +0200
> 
> I have verified that all *.el and *.elc files are identical in both
> cases. I get different checksums for lib/libgnu.a, which is expected
> because the archive contains timestamps.

So this difference in libgnu.a is the reason for differences in
checksums?  Isn't there a way to build Binutils in a way that libgnu.a
and similar archives don't record time stamps?

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ulrich Mueller <ulm@gentoo.org>
>> Cc: Gerd Möllmann <gerd.moellmann@gmail.com>,
>>   Björn Bidar
>>  <bjorn.bidar@thaodan.de>,  Eli Zaretskii <eliz@gnu.org>,
>>   emacs-devel@gnu.org
>> Date: Tue, 10 Oct 2023 10:30:10 +0200
>> 
>> I have verified that all *.el and *.elc files are identical in both
>> cases. I get different checksums for lib/libgnu.a, which is expected
>> because the archive contains timestamps.
>
> So this difference in libgnu.a is the reason for differences in
> checksums?  Isn't there a way to build Binutils in a way that libgnu.a
> and similar archives don't record time stamps?

binuils uses source date epoch but the changes have done quite recently:
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=6f56739807051e82a6327ff184b01be67be37670
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=6badd1020f5bebd3f60a780b8e41a1b581046087

None of them are in the current release (2.41) but require backports.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Ulrich Mueller]

>>>>> On Tue, 10 Oct 2023, Eli Zaretskii wrote:

> So this difference in libgnu.a is the reason for differences in
> checksums?

I don't think so. I've done a further test where I first built libgnu.a,
then replaced it by the libgnu.a from the previous build, and continued
to build Emacs. The dump still ends up having a different checksum.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

> Here is the diff of the matching Emacs binary:
> /usr/bin/emacs-x11 differs in ELF sections
> --- old /usr/bin/emacs-x11 (objdump)
> +++ new /usr/bin/emacs-x11 (objdump)
> @@ -519888,8 +519888,8 @@
>   7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
>   7f1420 00400000 00000000 00000000 00000000  .@..............
>   7f1430 00000000 00000000 00000000 00000000  ................
> - 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
> - 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
> + 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
> + 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
>   7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
>   7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
>   7f1480 12000000 00000000 00000000 00000000  ................
>

What Eli said.  Do you ./configure --disable-build-details?

Re: How to make aot native-compilation and pdmp creation reproduceable

[Andrea Corallo]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

> Hey,
>
> I noticed that building Emacs isn't enterily reproduceable.
> The compiled byte code it reproduceable, however I notied that
> the pdmp files and some of the native compiled elisp for each variant
> change each time the package is rebuild.
>
> The specific build option don't seem to make a difference.
>
> Strangely the only eln file that changes is x-win.el:
> /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln differs in ELF sections
> --- old /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
> +++ new /usr/lib64/emacs/29.1/native-lisp/29.1-115521d4/preloaded/x-win-3c49581f-6ddb7e08.eln (objdump)
> @@ -2950,7 +2950,7 @@
>   0e7c0 6f726564 20736573 73696f6e 20646174  ored session dat
>   0e7d0 61222022 4865776c 6574742d 5061636b  a" "Hewlett-Pack
>   0e7e0 61726420 496e636f 72706f72 61746564  ard Incorporated
> - 0e7f0 22202334 3d282831 3638202e 206d7574  " #4=((168 . mut
> + 0e7f0 22202336 3d282831 3638202e 206d7574  " #6=((168 . mut
>   0e800 652d6163 75746529 20283136 39202e20  e-acute) (169 . 
>   0e810 6d757465 2d677261 76652920 28313730  mute-grave) (170
>   0e820 202e206d 7574652d 61736369 69636972   . mute-asciicir
> @@ -2976,7 +2976,7 @@
>   0e960 20224865 776c6574 742d5061 636b6172   "Hewlett-Packar
>   0e970 6420436f 6d70616e 79222022 5831312f  d Company" "X11/
>   0e980 4e655753 202d2053 756e204d 6963726f  NeWS - Sun Micro
> - 0e990 73797374 656d7320 496e632e 22202336  systems Inc." #6
> + 0e990 73797374 656d7320 496e632e 22202334  systems Inc." #4
>   0e9a0 3d282833 39323937 36202e20 66333629  =((392976 . f36)
>   0e9b0 20283339 32393737 202e2066 33372920   (392977 . f37) 
>   0e9c0 28333933 30353620 2e207265 71292028  (393056 . req) (
>
> One example of pdmp of one of the variants:
> /usr/bin/emacs-x11 differs in ELF sections
> --- old /usr/bin/emacs-x11 (objdump)
> +++ new /usr/bin/emacs-x11 (objdump)
> @@ -519888,8 +519888,8 @@
>   7f1410 2d2d6e6f 2d73706c 61736800 00000000  --no-splash.....
>   7f1420 00400000 00000000 00000000 00000000  .@..............
>   7f1430 00000000 00000000 00000000 00000000  ................
> - 7f1440 81e3b8e4 33011ec5 18db85f2 9586197c  ....3..........|
> - 7f1450 be6fb703 f761afd4 c6d7bd74 f2168be2  .o...a.....t....
> + 7f1440 a2c05c34 51540c6d eae96bed 21a90a3c  ..\4QT.m..k.!..<
> + 7f1450 c0cf321a e7c634d2 89ae7791 2fe90967  ..2...4...w./..g
>   7f1460 2c010000 00000000 20003900 00000000  ,....... .9.....
>   7f1470 ffffffff 15000000 3e000000 7c000000  ........>...|...
>   7f1480 12000000 00000000 00000000 00000000  ................
>
> The build log of the package including a binary diff can be found here:
> https://build.opensuse.org/build/editors/openSUSE_Tumbleweed/x86_64/emacs/_log
>
> The spec file of the package can be fond here:
> https://build.opensuse.org/package/view_file/editors/emacs/emacs.spec?expand=1
>
> How can I debug why they differ after each rebuild?
>
> Br,
>
> Björn Bidar


Hi Björn,

mmhh interesting, is the build without native compilation reproducible
in terms of dump file and elc ones?

Thanks

  Andrea

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

[-- Attachment #1: Type: text/plain, Size: 412 bytes --]

Andrea Corallo <acorallo@gnu.org> writes:
>
> Hi Björn,
>
> mmhh interesting, is the build without native compilation reproducible
> in terms of dump file and elc ones?

The elc ones don't show up in the diff (full diff attached to an answer
to Gerd Möllmann).

I tried a build without native compilation, the pdmp still differs, same
for te Emacs binary.

I attach the pkg-diff to this message.

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nachbau.emacs.wnativecomp.log --]
[-- Type: text/x-log, Size: 13849 bytes --]

/var/tmp/build-root.//.mount/.build.packages/SRPMS/emacs-29.1-436.1.src.rpm

/var/tmp/build-root.//.mount/.build.packages/RPMS/noarch/emacs-el-29.1-436.1.noarch.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/noarch/emacs-info-29.1-436.1.noarch.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-nox-29.1-436.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-x11-29.1-436.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-29.1-436.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-x11-debuginfo-29.1-436.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-debugsource-29.1-436.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-nox-debuginfo-29.1-436.1.x86_64.rpm
/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/emacs-debuginfo-29.1-436.1.x86_64.rpm/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/etags-debuginfo-29.1-436.1.x86_64.rpm/var/tmp/build-root.//.mount/.build.packages/RPMS/x86_64/etags-29.1-436.1.x86_64.rpm
+ echo 0
+ cp -a binaries.nachbau/.osc-build-retval .
+ cp -a /var/tmp/build-root.//.build.log binaries.nachbau/
+ cp -a binaries.nachbau/.build.log .
+ test -e binaries.nachbau/.build.log
+ tail -1 binaries.nachbau/.build.log
+ perl -ne '/\[\s*(\d+)s\]/ && print $1'
+ find -mtime -0 -print0
+ xargs --no-run-if-empty -0 touch
++ cat binaries.nachbau/.osc-build-retval
+ '[' 0 -gt 0 ']'
+ calcchecksums ../emacs/binaries.nachbau/emacs-29.1-436.1.src.rpm ../emacs/binaries.nachbau/emacs-29.1-436.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-debuginfo-29.1-436.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-debugsource-29.1-436.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-el-29.1-436.1.noarch.rpm ../emacs/binaries.nachbau/emacs-info-29.1-436.1.noarch.rpm ../emacs/binaries.nachbau/emacs-nox-29.1-436.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-nox-debuginfo-29.1-436.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-x11-29.1-436.1.x86_64.rpm ../emacs/binaries.nachbau/emacs-x11-debuginfo-29.1-436.1.x86_64.rpm ../emacs/binaries.nachbau/etags-29.1-436.1.x86_64.rpm ../emacs/binaries.nachbau/etags-debuginfo-29.1-436.1.x86_64.rpm
+ sed -e 's!\.\./!!'
++ find binaries.nachbau/ -maxdepth 1 -name '*.rpm' -print -quit
+ test -n binaries.nachbau/emacs-el-29.1-436.1.noarch.rpm
+ : binaries
+ : binaries.nachbau
+ pushd binaries
~/dev/opensuse/buildservice/home/Thaodan/branches/openSUSE/Factory/emacs/binaries ~/dev/opensuse/buildservice/home/Thaodan/branches/openSUSE/Factory/emacs
++ ls emacs-29.1-436.1.x86_64.rpm emacs-el-29.1-436.1.noarch.rpm emacs-info-29.1-436.1.noarch.rpm emacs-nox-29.1-436.1.x86_64.rpm emacs-x11-29.1-436.1.x86_64.rpm etags-29.1-436.1.x86_64.rpm
++ grep -v '^::import::'
+ origrpms='emacs-29.1-436.1.x86_64.rpm
emacs-el-29.1-436.1.noarch.rpm
emacs-info-29.1-436.1.noarch.rpm
emacs-nox-29.1-436.1.x86_64.rpm
emacs-x11-29.1-436.1.x86_64.rpm
etags-29.1-436.1.x86_64.rpm'
+ popd
+ [[ -n emacs-29.1-436.1.x86_64.rpm
emacs-el-29.1-436.1.noarch.rpm
emacs-info-29.1-436.1.noarch.rpm
emacs-nox-29.1-436.1.x86_64.rpm
emacs-x11-29.1-436.1.x86_64.rpm
~/dev/opensuse/buildservice/home/Thaodan/branches/openSUSE/Factory/emacs
etags-29.1-436.1.x86_64.rpm ]]
+ pkgdiff=/home/bidar/build-compare/pkg-diff.sh
+ test -e /home/bidar/build-compare/pkg-diff.sh
+ pkgdiff=/usr/lib/build/pkg-diff.sh
+ test -e /usr/lib/build/pkg-diff.sh
+ overall=0
+ overalldiffered=0
+ for f in $origrpms
++ echo emacs-29.1-436.1.x86_64.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-29.1-*.x86_64.rpm'
+ f1=binaries/emacs-29.1-436.1.x86_64.rpm
+ f2='binaries.nachbau/emacs-29.1-*.x86_64.rpm'
+ cmp binaries/emacs-29.1-436.1.x86_64.rpm binaries.nachbau/emacs-29.1-436.1.x86_64.rpm
binaries/emacs-29.1-436.1.x86_64.rpm binaries.nachbau/emacs-29.1-436.1.x86_64.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=1
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-29.1-436.1.x86_64.rpm binaries.nachbau/emacs-29.1-436.1.x86_64.rpm
Comparing emacs-29.1-436.1.x86_64.rpm to emacs-29.1-436.1.x86_64.rpm
comparing the rpm tags of emacs
RPM meta information is identical
Extracting packages
Package content is identical
+ compareresult=0
+ [[ 0 = 0 ]]
+ for f in $origrpms
++ echo emacs-el-29.1-436.1.noarch.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-el-29.1-*.noarch.rpm'
+ f1=binaries/emacs-el-29.1-436.1.noarch.rpm
+ f2='binaries.nachbau/emacs-el-29.1-*.noarch.rpm'
+ cmp binaries/emacs-el-29.1-436.1.noarch.rpm binaries.nachbau/emacs-el-29.1-436.1.noarch.rpm
binaries/emacs-el-29.1-436.1.noarch.rpm binaries.nachbau/emacs-el-29.1-436.1.noarch.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=2
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-el-29.1-436.1.noarch.rpm binaries.nachbau/emacs-el-29.1-436.1.noarch.rpm
Comparing emacs-el-29.1-436.1.noarch.rpm to emacs-el-29.1-436.1.noarch.rpm
comparing the rpm tags of emacs-el
RPM meta information is identical
Extracting packages
Package content is identical
+ compareresult=0
+ [[ 0 = 0 ]]
+ for f in $origrpms
++ echo emacs-info-29.1-436.1.noarch.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-info-29.1-*.noarch.rpm'
+ f1=binaries/emacs-info-29.1-436.1.noarch.rpm
+ f2='binaries.nachbau/emacs-info-29.1-*.noarch.rpm'
+ cmp binaries/emacs-info-29.1-436.1.noarch.rpm binaries.nachbau/emacs-info-29.1-436.1.noarch.rpm
binaries/emacs-info-29.1-436.1.noarch.rpm binaries.nachbau/emacs-info-29.1-436.1.noarch.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=3
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-info-29.1-436.1.noarch.rpm binaries.nachbau/emacs-info-29.1-436.1.noarch.rpm
Comparing emacs-info-29.1-436.1.noarch.rpm to emacs-info-29.1-436.1.noarch.rpm
comparing the rpm tags of emacs-info
RPM meta information is identical
Extracting packages
Package content is identical
+ compareresult=0
+ [[ 0 = 0 ]]
+ for f in $origrpms
++ echo emacs-nox-29.1-436.1.x86_64.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-nox-29.1-*.x86_64.rpm'
+ f1=binaries/emacs-nox-29.1-436.1.x86_64.rpm
+ f2='binaries.nachbau/emacs-nox-29.1-*.x86_64.rpm'
+ cmp binaries/emacs-nox-29.1-436.1.x86_64.rpm binaries.nachbau/emacs-nox-29.1-436.1.x86_64.rpm
binaries/emacs-nox-29.1-436.1.x86_64.rpm binaries.nachbau/emacs-nox-29.1-436.1.x86_64.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=4
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-nox-29.1-436.1.x86_64.rpm binaries.nachbau/emacs-nox-29.1-436.1.x86_64.rpm
Comparing emacs-nox-29.1-436.1.x86_64.rpm to emacs-nox-29.1-436.1.x86_64.rpm
comparing the rpm tags of emacs-nox
--- old-rpm-tags
+++ new-rpm-tags
@@ -149 +149 @@
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp 3112553b40327c589fec1b96fe60c604584443e6c25c6abebd0dacf35bf876b6 0
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp 92e68b6d5661f65bfc8c537feb9a5a00f1e17b8adacf84b9aa3f84be5d9e1a51 0
comparing rpmtags
comparing RELEASE
comparing PROVIDES
comparing scripts
comparing filelist
comparing file checksum
creating rename script
RPM file checksum differs.
Extracting packages
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp differs at offset '60273' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-nox.pdmp (hex)
@@ -1,7 +1,7 @@
 0000eb40  00 50 00 03 00 00 00 40  08 e7 00 00 00 00 00 00  |.P.....@........|
 0000eb50  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 0000eb60  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
-0000eb70  da 6f eb 72 de f9 30 5e  02 28 6b ee 00 00 00 00  |.o.r..0^.(k.....|
+0000eb70  6e 51 7e 6a 7f fc 30 5e  02 28 6b ee 00 00 00 00  |nQ~j..0^.(k.....|
 0000eb80  47 00 00 1f 00 00 00 40  00 00 00 00 00 00 00 00  |G......@........|
 0000eb90  00 00 00 00 00 00 00 00  60 30 00 00 00 00 00 00  |........`0......|
 0000eba0  60 ef 00 00 00 00 00 00  c0 ed 00 00 00 00 00 00  |`...............|
+ compareresult=1
+ [[ 1 = 0 ]]
+ overall=1
+ for f in $origrpms
++ echo emacs-x11-29.1-436.1.x86_64.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='emacs-x11-29.1-*.x86_64.rpm'
+ f1=binaries/emacs-x11-29.1-436.1.x86_64.rpm
+ f2='binaries.nachbau/emacs-x11-29.1-*.x86_64.rpm'
+ cmp binaries/emacs-x11-29.1-436.1.x86_64.rpm binaries.nachbau/emacs-x11-29.1-436.1.x86_64.rpm
binaries/emacs-x11-29.1-436.1.x86_64.rpm binaries.nachbau/emacs-x11-29.1-436.1.x86_64.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=5
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/emacs-x11-29.1-436.1.x86_64.rpm binaries.nachbau/emacs-x11-29.1-436.1.x86_64.rpm
Comparing emacs-x11-29.1-436.1.x86_64.rpm to emacs-x11-29.1-436.1.x86_64.rpm
comparing the rpm tags of emacs-x11
--- old-rpm-tags
+++ new-rpm-tags
@@ -254 +254 @@
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp a9e8541d54d196b2728f11aa9ca95d4937232f3aeea1e0238f5edc52d6f12209 0
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp cfcdba04e21757299b54609aeb7a3c6ff17ccbb5519df3a3cc8a6f47a7bd7c87 0
@@ -256 +256 @@
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp 5ceda5d362f4a7a2f7e0f393d16c799f14b3875b54b5c037168d9419eb9be2b2 0
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp 933d7f2ce61c681288d589a751c7dbf5eaa3f1cf7eb2fcd3e39176666bde57f8 0
@@ -258 +258 @@
-/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp 7e921a5abca4093ee01e86758ad8e1b0b01df779dfa4b031bb97e0b9f13398b6 0
+/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp 52afdb381253acfef2cbb727af2fb0a2bf5764a7e8a60c2d0e609767c8348947 0
comparing rpmtags
comparing RELEASE
comparing PROVIDES
comparing scripts
comparing filelist
comparing file checksum
creating rename script
RPM file checksum differs.
Extracting packages
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp differs at offset '60273' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-gtk.pdmp (hex)
@@ -1,7 +1,7 @@
 0000eb40  00 50 00 03 00 00 00 40  08 e7 00 00 00 00 00 00  |.P.....@........|
 0000eb50  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 0000eb60  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
-0000eb70  de ef 72 be 7a fa 30 5e  02 28 6b ee 00 00 00 00  |..r.z.0^.(k.....|
+0000eb70  0a f1 5e a3 ed fc 30 5e  02 28 6b ee 00 00 00 00  |..^...0^.(k.....|
 0000eb80  a8 c8 48 00 00 00 00 00  90 eb 00 00 00 00 00 00  |..H.............|
 0000eb90  f0 ee 00 00 00 00 00 00  a0 eb 00 00 00 00 00 00  |................|
 0000eba0  70 ee 00 00 00 00 00 00  b0 eb 00 00 00 00 00 00  |p...............|
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp differs at offset '60273' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-wayland.pdmp (hex)
@@ -1,7 +1,7 @@
 0000eb40  00 50 00 03 00 00 00 40  08 e7 00 00 00 00 00 00  |.P.....@........|
 0000eb50  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 0000eb60  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
-0000eb70  a2 44 74 3f c3 fa 30 5e  02 28 6b ee 00 00 00 00  |.Dt?..0^.(k.....|
+0000eb70  7e e3 de dc 21 fd 30 5e  02 28 6b ee 00 00 00 00  |~...!.0^.(k.....|
 0000eb80  c8 3e 48 00 00 00 00 00  90 eb 00 00 00 00 00 00  |.>H.............|
 0000eb90  f0 ee 00 00 00 00 00 00  a0 eb 00 00 00 00 00 00  |................|
 0000eba0  70 ee 00 00 00 00 00 00  b0 eb 00 00 00 00 00 00  |p...............|
/usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp differs at offset '60273' (GNU/Emacs pdumper image)
--- old /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
+++ new /usr/libexec/emacs/29.1/x86_64-suse-linux/emacs-x11.pdmp (hex)
@@ -1,7 +1,7 @@
 0000eb40  00 50 00 03 00 00 00 40  08 e7 00 00 00 00 00 00  |.P.....@........|
 0000eb50  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
 0000eb60  01 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  |................|
-0000eb70  96 f8 21 bf 2c fa 30 5e  02 28 6b ee 00 00 00 00  |..!.,.0^.(k.....|
+0000eb70  02 f8 d3 0b b6 fc 30 5e  02 28 6b ee 00 00 00 00  |......0^.(k.....|
 0000eb80  d8 af 48 00 00 00 00 00  90 eb 00 00 00 00 00 00  |..H.............|
 0000eb90  f0 ee 00 00 00 00 00 00  a0 eb 00 00 00 00 00 00  |................|
 0000eba0  70 ee 00 00 00 00 00 00  b0 eb 00 00 00 00 00 00  |p...............|
+ compareresult=1
+ [[ 1 = 0 ]]
+ overall=1
+ for f in $origrpms
++ echo etags-29.1-436.1.x86_64.rpm
++ sed 's/-[0-9.]*\(\.[a-z0-9_]*\.rpm\)$/-*\1/'
+ rpm='etags-29.1-*.x86_64.rpm'
+ f1=binaries/etags-29.1-436.1.x86_64.rpm
+ f2='binaries.nachbau/etags-29.1-*.x86_64.rpm'
+ cmp binaries/etags-29.1-436.1.x86_64.rpm binaries.nachbau/etags-29.1-436.1.x86_64.rpm
binaries/etags-29.1-436.1.x86_64.rpm binaries.nachbau/etags-29.1-436.1.x86_64.rpm differ: byte 225, line 1
+ compareresult=1
+ [[ 1 = 0 ]]
+ overalldiffered=6
+ [[ 1 = 0 ]]
+ /usr/lib/build/pkg-diff.sh -a binaries/etags-29.1-436.1.x86_64.rpm binaries.nachbau/etags-29.1-436.1.x86_64.rpm
Comparing etags-29.1-436.1.x86_64.rpm to etags-29.1-436.1.x86_64.rpm
comparing the rpm tags of etags
RPM meta information is identical
Extracting packages
Package content is identical
+ compareresult=0
+ [[ 0 = 0 ]]
+ echo 6
+ echo 'overalldiffered=6 (number of pkgs that are not bit-by-bit identical: 0 is good)'
+ echo overall=1
overalldiffered=6 (number of pkgs that are not bit-by-bit identical: 0 is good)
+ echo 1
overall=1
+ exit 6
processing .
result:emacs=6
result:emacs=notbuilt

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Björn Bidar <bjorn.bidar@thaodan.de>
> Cc: emacs-devel@gnu.org
> Date: Mon, 09 Oct 2023 00:38:21 +0300
> 
> Andrea Corallo <acorallo@gnu.org> writes:
> >
> > Hi Björn,
> >
> > mmhh interesting, is the build without native compilation reproducible
> > in terms of dump file and elc ones?
> 
> The elc ones don't show up in the diff (full diff attached to an answer
> to Gerd Möllmann).
> 
> I tried a build without native compilation, the pdmp still differs, same
> for te Emacs binary.

Did you configure Emacs with --disable-build-details ?

Re: How to make aot native-compilation and pdmp creation reproduceable

[Björn Bidar]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Björn Bidar <bjorn.bidar@thaodan.de>
>> Cc: emacs-devel@gnu.org
>> Date: Mon, 09 Oct 2023 00:38:21 +0300
>> 
>> Andrea Corallo <acorallo@gnu.org> writes:
>> >
>> > Hi Björn,
>> >
>> > mmhh interesting, is the build without native compilation reproducible
>> > in terms of dump file and elc ones?
>> 
>> The elc ones don't show up in the diff (full diff attached to an answer
>> to Gerd Möllmann).
>> 
>> I tried a build without native compilation, the pdmp still differs, same
>> for te Emacs binary.
>
> Did you configure Emacs with --disable-build-details ?

Yes it is.

Each build has the configure flags (besides gtk or pgtk):

./configure --disable-build-details --without-pop --with-mailutils
--with-native-compilation --without-hesiod --with-gameuser=:games
--with-kerberos --with-kerberos5 --with-file-notification=inotify
--with-modules --enable-autodepend --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --datadir=/usr/share --localstatedir=/var
--sharedstatedir=/var/lib --libexecdir=/usr/libexec
--with-file-notification=yes --libdir=/usr/lib64
--enable-locallisppath=/usr/share/emacs/29.1/site-lisp:/usr/share/emacs/site-lisp
--build=x86_64-suse-linux --with-dumping=pdumper

Should the build use SOURCE_DATE_EPOCH if set even if
--disable-build-details isn't set in any case?

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

>> Did you configure Emacs with --disable-build-details ?
>
> Yes it is.

Hm, maybe addr2line could help here?  I don't have that here (macOS), so
I can't try.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Björn Bidar <bjorn.bidar@thaodan.de> writes:
>
>>> Did you configure Emacs with --disable-build-details ?
>>
>> Yes it is.
>
> Hm, maybe addr2line could help here?  I don't have that here (macOS), so
> I can't try.

Errr, the fingerprint is of course also stored in the executable...

  volatile unsigned char fingerprint[] = ...

That could explain the difference.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> Gerd Möllmann <gerd.moellmann@gmail.com> writes:
>
>> Björn Bidar <bjorn.bidar@thaodan.de> writes:
>>
>>>> Did you configure Emacs with --disable-build-details ?
>>>
>>> Yes it is.
>>
>> Hm, maybe addr2line could help here?  I don't have that here (macOS), so
>> I can't try.
>
> Errr, the fingerprint is of course also stored in the executable...
>
>   volatile unsigned char fingerprint[] = ...
>
> That could explain the difference.

Can you determine if x-win.eln gets dumped into the pdmp file?  That
would explain the differences, and it would all boil down to the
print-circle thing.

Re: How to make aot native-compilation and pdmp creation reproduceable

[Eli Zaretskii]

> From: Björn Bidar <bjorn.bidar@thaodan.de>
> Cc: acorallo@gnu.org,  emacs-devel@gnu.org
> Date: Mon, 09 Oct 2023 07:25:38 +0300
> 
> Should the build use SOURCE_DATE_EPOCH if set even if
> --disable-build-details isn't set in any case?

Not IMO, no.  Not until Emacs supports truly reproducible builds; we
are not there yet.