bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[John Paul Adrian Glaubitz]

Hello!

Starting with version 27, Emacs has started failing to build from source
on m68k (Linux) with the following error message:

/bin/mkdir -p ../etc
/usr/bin/make -C ../lisp update-subdirs
make[4]: Entering directory '/<<BUILDDIR>>/emacs-27.1+1/debian/build-gtk/lisp'
make[4]: Leaving directory '/<<BUILDDIR>>/emacs-27.1+1/debian/build-gtk/lisp'
cp -f temacs bootstrap-emacs
rm -f bootstrap-emacs.pdmp
./temacs --batch  -l loadup --temacs=pbootstrap
Loading loadup.el (source)...
dump mode: pbootstrap
Using load-path (/<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/emacs-lisp /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/progmodes /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/language /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/international /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/textmodes /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/vc)
Loading emacs-lisp/byte-run (source)...
Loading emacs-lisp/backquote (source)...
Loading subr (source)...
Loading version (source)...
(...)
Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/vc/vc-hooks.el (source)...
Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/vc/ediff-hook.el (source)...
Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/uniquify.el (source)...
Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/electric.el (source)...
Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/emacs-lisp/eldoc.el (source)...
Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/cus-start.el (source)...
Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/tooltip.el (source)...
Finding pointers to doc strings...
Finding pointers to doc strings...done
Dumping under the name bootstrap-emacs.pdmp
dumping fingerprint: 7b5c59c589dc151eb1e4269bd83fbe809616b5cb9bb5c80014d5b560b391dfb6
dump relocation out of range

Full build log available in [1].

I have not been able to figure out what "dump relocation out of range"
actually means, so any explanation is much appreciated.

Thanks,
Adrian

> [1] https://buildd.debian.org/status/fetch.php?pkg=emacs&arch=m68k&ver=1%3A27.1%2B1-3&stamp=1604857999&raw=0

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

bug#44531: Acknowledgement (27.1; Emacs 27 fails to build from source on m68k (regression))

[John Paul Adrian Glaubitz]

Hi!

The problem can be worked around by configuring emacs with "--with-dumping=unexec",
so it seems the new portable dumper is incompatible with m68k.

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[Eli Zaretskii]

> From: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
> Date: Mon, 09 Nov 2020 14:30:34 +0100
> 
> Hello!
> 
> Starting with version 27, Emacs has started failing to build from source
> on m68k (Linux) with the following error message:
> 
> /bin/mkdir -p ../etc
> /usr/bin/make -C ../lisp update-subdirs
> make[4]: Entering directory '/<<BUILDDIR>>/emacs-27.1+1/debian/build-gtk/lisp'
> make[4]: Leaving directory '/<<BUILDDIR>>/emacs-27.1+1/debian/build-gtk/lisp'
> cp -f temacs bootstrap-emacs
> rm -f bootstrap-emacs.pdmp
> ./temacs --batch  -l loadup --temacs=pbootstrap
> Loading loadup.el (source)...
> dump mode: pbootstrap
> Using load-path (/<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/emacs-lisp /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/progmodes /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/language /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/international /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/textmodes /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/vc)
> Loading emacs-lisp/byte-run (source)...
> Loading emacs-lisp/backquote (source)...
> Loading subr (source)...
> Loading version (source)...
> (...)
> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/vc/vc-hooks.el (source)...
> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/vc/ediff-hook.el (source)...
> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/uniquify.el (source)...
> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/electric.el (source)...
> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/emacs-lisp/eldoc.el (source)...
> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/cus-start.el (source)...
> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/tooltip.el (source)...
> Finding pointers to doc strings...
> Finding pointers to doc strings...done
> Dumping under the name bootstrap-emacs.pdmp
> dumping fingerprint: 7b5c59c589dc151eb1e4269bd83fbe809616b5cb9bb5c80014d5b560b391dfb6
> dump relocation out of range
> 
> Full build log available in [1].
> 
> I have not been able to figure out what "dump relocation out of range"
> actually means, so any explanation is much appreciated.

Daniel, any advice for how to go about this problem?

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[Daniel Colascione]

On November 20, 2020 11:48:09 PM Eli Zaretskii <eliz@gnu.org> wrote:

>> From: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
>> Date: Mon, 09 Nov 2020 14:30:34 +0100
>>
>> Hello!
>>
>> Starting with version 27, Emacs has started failing to build from source
>> on m68k (Linux) with the following error message:


m68k is big endian, isn't it? Try building with enable-checking to see 
whether we can see where the bit rearrangement problem might be.

>>
>> /bin/mkdir -p ../etc
>> /usr/bin/make -C ../lisp update-subdirs
>> make[4]: Entering directory '/<<BUILDDIR>>/emacs-27.1+1/debian/build-gtk/lisp'
>> make[4]: Leaving directory '/<<BUILDDIR>>/emacs-27.1+1/debian/build-gtk/lisp'
>> cp -f temacs bootstrap-emacs
>> rm -f bootstrap-emacs.pdmp
>> ./temacs --batch  -l loadup --temacs=pbootstrap
>> Loading loadup.el (source)...
>> dump mode: pbootstrap
>> Using load-path (/<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp 
>> /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/emacs-lisp 
>> /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/progmodes 
>> /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/language 
>> /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/international 
>> /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/textmodes 
>> /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/vc)
>> Loading emacs-lisp/byte-run (source)...
>> Loading emacs-lisp/backquote (source)...
>> Loading subr (source)...
>> Loading version (source)...
>> (...)
>> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/vc/vc-hooks.el 
>> (source)...
>> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/vc/ediff-hook.el 
>> (source)...
>> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/uniquify.el 
>> (source)...
>> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/electric.el 
>> (source)...
>> Loading 
>> /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/emacs-lisp/eldoc.el 
>> (source)...
>> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/cus-start.el 
>> (source)...
>> Loading /<<BUILDDIR>>/emacs-27.1+1/debian/build-src/lisp/tooltip.el (source)...
>> Finding pointers to doc strings...
>> Finding pointers to doc strings...done
>> Dumping under the name bootstrap-emacs.pdmp
>> dumping fingerprint: 
>> 7b5c59c589dc151eb1e4269bd83fbe809616b5cb9bb5c80014d5b560b391dfb6
>> dump relocation out of range
>>
>> Full build log available in [1].
>>
>> I have not been able to figure out what "dump relocation out of range"
>> actually means, so any explanation is much appreciated.
>
> Daniel, any advice for how to go about this problem?

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[John Paul Adrian Glaubitz]

(CC'ing Andreas)

On 11/21/20 9:07 AM, Daniel Colascione wrote:
>>> Starting with version 27, Emacs has started failing to build from source
>>> on m68k (Linux) with the following error message:
> 
> 
> m68k is big endian, isn't it? Try building with enable-checking to see whether we can see where the bit rearrangement problem might be.

Thanks, I'll give it a try. My suspicion would be that it's an alignment problem
since the native alignment for m68k on Linux/sysv is 16 bits, not 32 bits.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[Lars Ingebrigtsen]

John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> writes:

>> m68k is big endian, isn't it? Try building with enable-checking to
>> see whether we can see where the bit rearrangement problem might be.
>
> Thanks, I'll give it a try. My suspicion would be that it's an
> alignment problem since the native alignment for m68k on Linux/sysv is
> 16 bits, not 32 bits.

This was almost a year ago -- did you get any further in debugging this
problem?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[John Paul Adrian Glaubitz]

Hi Lars!

On 9/8/21 11:48, Lars Ingebrigtsen wrote:
> John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> writes:
> 
>>> m68k is big endian, isn't it? Try building with enable-checking to
>>> see whether we can see where the bit rearrangement problem might be.
>>
>> Thanks, I'll give it a try. My suspicion would be that it's an
>> alignment problem since the native alignment for m68k on Linux/sysv is
>> 16 bits, not 32 bits.
> 
> This was almost a year ago -- did you get any further in debugging this
> problem?

I think the problem was the new dumper that is used by default on emacs27 [1].

I know that I built emacs27 successfully on m68k with the option changed back,
let me try that again.

Adrian

> [1] https://lists.debian.org/debian-68k/2020/11/msg00009.html

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> This was almost a year ago -- did you get any further in debugging this
> problem?

More information was requested, but no response was given within a
month, so it seems unlikely that there'll be further progress here, and
I'm closing this bug report.  If progress can be made, please respond to
this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> More information was requested, but no response was given within a
> month, so it seems unlikely that there'll be further progress here, and
> I'm closing this bug report.  If progress can be made, please respond to
> this email and we'll reopen the bug report.

Please disregard -- I hit the wrong button when responding here.  The
bug report has not been closed.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[John Paul Adrian Glaubitz]

Hi Lars!

On 10/7/21 11:19, Lars Ingebrigtsen wrote:
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
>> This was almost a year ago -- did you get any further in debugging this
>> problem?
> 
> More information was requested, but no response was given within a
> month, so it seems unlikely that there'll be further progress here, and
> I'm closing this bug report.  If progress can be made, please respond to
> this email and we'll reopen the bug report.

As I said, building with the old dumper fixes the problem. Using the new
pdumper causes the crash. The issue still hasn't been fixed.

I'm not an emacs expert, so I don't know how to fix this bug.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> dumping fingerprint: 7b5c59c589dc151eb1e4269bd83fbe809616b5cb9bb5c80014d5b560b391dfb6
> dump relocation out of range

This error message basically says that the relation is not a multiple of
the minimum expected alignment.  So it seems to be a direct consequence
of the 16bit alignment used on m68k combined with the following from
pdumper.el:

   [...]
   DUMP_RELOC_ALIGNMENT_BITS = 2,

   /* Minimum alignment required by dump file format.  */
   DUMP_RELOCATION_ALIGNMENT = 1 << DUMP_RELOC_ALIGNMENT_BITS,
   [...]

I can't see anything in the code which explains what this alignment
requirement is about.  You can try lowering DUMP_RELOC_ALIGNMENT_BITS
to 1 and see if that works (long shot).
Daniel?


        Stefan

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[Pip Cet]

On Sat, Jul 2, 2022 at 7:29 PM Stefan Monnier via Bug reports for GNU
Emacs, the Swiss army knife of text editors <bug-gnu-emacs@gnu.org>
wrote:
> > dumping fingerprint: 7b5c59c589dc151eb1e4269bd83fbe809616b5cb9bb5c80014d5b560b391dfb6
> > dump relocation out of range
>
> This error message basically says that the relation is not a multiple of
> the minimum expected alignment.  So it seems to be a direct consequence
> of the 16bit alignment used on m68k combined with the following from
> pdumper.el:

pdumper.c, I think :-)

>    [...]
>    DUMP_RELOC_ALIGNMENT_BITS = 2,
>
>    /* Minimum alignment required by dump file format.  */
>    DUMP_RELOCATION_ALIGNMENT = 1 << DUMP_RELOC_ALIGNMENT_BITS,
>    [...]
>
> I can't see anything in the code which explains what this alignment
> requirement is about.  You can try lowering DUMP_RELOC_ALIGNMENT_BITS
> to 1 and see if that works (long shot).

IIUC, the top (DUMP_RELOC_TYPE_BITS - DUMP_RELOC_ALIGNMENT_BITS) of
the relocation offsets stored by pdumper must be 0. That means we can
only address the first 512 MB of the dump in ordinary 32-bit pdumper
builds, and 256 MB on m68k with your fix. I'm not sure how useful a
data point this is without real silicon, but I tried on an m68k
emulator (qemu), could reproduce the bug, and your fix works there.

Pip

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>>    [...]
>>    DUMP_RELOC_ALIGNMENT_BITS = 2,
>>
>>    /* Minimum alignment required by dump file format.  */
>>    DUMP_RELOCATION_ALIGNMENT = 1 << DUMP_RELOC_ALIGNMENT_BITS,
>>    [...]
>>
>> I can't see anything in the code which explains what this alignment
>> requirement is about.  You can try lowering DUMP_RELOC_ALIGNMENT_BITS
>> to 1 and see if that works (long shot).
>
> IIUC, the top (DUMP_RELOC_TYPE_BITS - DUMP_RELOC_ALIGNMENT_BITS) of
> the relocation offsets stored by pdumper must be 0. That means we can
> only address the first 512 MB of the dump in ordinary 32-bit pdumper
> builds, and 256 MB on m68k with your fix. I'm not sure how useful a
> data point this is without real silicon, but I tried on an m68k
> emulator (qemu), could reproduce the bug, and your fix works there.

Any hope you can turn that into a patch?
Maybe with something like `#ifdef (__m68k__)`?


        Stefan

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[John Paul Adrian Glaubitz]

Hello!

On 7/2/22 21:28, Stefan Monnier wrote:
>> dumping fingerprint: 7b5c59c589dc151eb1e4269bd83fbe809616b5cb9bb5c80014d5b560b391dfb6
>> dump relocation out of range
> 
> This error message basically says that the relation is not a multiple of
> the minimum expected alignment.  So it seems to be a direct consequence
> of the 16bit alignment used on m68k combined with the following from
> pdumper.el:
> 
>     [...]
>     DUMP_RELOC_ALIGNMENT_BITS = 2,
> 
>     /* Minimum alignment required by dump file format.  */
>     DUMP_RELOCATION_ALIGNMENT = 1 << DUMP_RELOC_ALIGNMENT_BITS,
>     [...]
> 
> I can't see anything in the code which explains what this alignment
> requirement is about.  You can try lowering DUMP_RELOC_ALIGNMENT_BITS
> to 1 and see if that works (long shot).
> Daniel?

I just gave lowering DUMP_RELOC_ALIGNMENT_BITS to 1 and it fixes the problem
for me. Maybe DUMP_RELOC_ALIGNMENT_BITS could be set depending on the native
alignment of the host machine?

Adrian

-- 
  .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer
`. `'   Physicist
   `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

bug#44531: 27.1; Emacs 27 fails to build from source on m68k (regression)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I just gave lowering DUMP_RELOC_ALIGNMENT_BITS to 1 and it fixes the problem
> for me. Maybe DUMP_RELOC_ALIGNMENT_BITS could be set depending on the native
> alignment of the host machine?

That's right.
If someone can come up with a corresponding patch that would be great.
The alignment is a question of ABI convention, so I'm not sure how best
to do it, but maybe something like:

    struct dummy
    {
      char a;
      Lisp_Object b;
    }

    #define DUMP_RELOC_ALIGNMENT_BITS log2 (offsetof (struct dummy, b))

?


        Stefan