bug#43878: emacs fails to build on recent macOS 11.0 ARM betas

bug#43878: emacs fails to build on recent macOS 11.0 ARM betas

[Itai Seggev]

In the last few betas of macOS on ARM, Apple has start enforcing a requirement
that all code be properly signed.  The linker automatically adds an "ad-hoc"
signature.  (At least for now, this is not required on x86_64, though I imagine
it is only a matter of time given Apple's public statements on code signing.)

The emacs build fails when the temacs is called to compile the Lisp files.
I've tracked this down to the call to make-fingerprint on temacs.tmp.  The call
modifies the Macho-O temacs.tmp after it was linked and signed, invaldinating
the code signature.  When it is launched, it is killed with a SIGABORT by the
OS due to the invalid signature.

I've come up with a couple of workarounds in my local build.  First, if I
modifiy make-fingerprint to not store the result in the Mach-O, then everything
seems to build fine.  It's not entirely clear to me what the purpose of this
modification of the Macho-O is, so I don't know if such a solution is
acceptable upstream.

If it is not, then the signature _must_ be repaired after make-fingerprint is
run.  This can be done quite simply, using 'codesign -s - -f temacs.tmp', which
creates a new "ad-hoc" signature for the executable.

If necessary, I am happy to test a patch / branch on my machine.

--
Itai

In 1997 a group of programmers started writing a desktop environment to fix a
travesty they didn't create.  Their program promptly found its way onto un*x
systems everywhere. Today, still opposed by a software monopolist, they survive
as soldiers of fortune.  If you share their vision, if you know you can help,
and if you can connect to internet, maybe you can join... the K-Team.

bug#43878: emacs fails to build on recent macOS 11.0 ARM betas

[Unknown]

Itai Seggev <is@cs.hmc.edu> writes:

> In the last few betas of macOS on ARM, Apple has start enforcing a requirement
> that all code be properly signed.  The linker automatically adds an "ad-hoc"
> signature.  (At least for now, this is not required on x86_64, though I imagine
> it is only a matter of time given Apple's public statements on code signing.)
>
> The emacs build fails when the temacs is called to compile the Lisp files.
> I've tracked this down to the call to make-fingerprint on temacs.tmp.  The call
> modifies the Macho-O temacs.tmp after it was linked and signed, invaldinating
> the code signature.  When it is launched, it is killed with a SIGABORT by the
> OS due to the invalid signature.
>
> I've come up with a couple of workarounds in my local build.  First, if I
> modifiy make-fingerprint to not store the result in the Mach-O, then everything
> seems to build fine.  It's not entirely clear to me what the purpose of this
> modification of the Macho-O is, so I don't know if such a solution is
> acceptable upstream.
>
> If it is not, then the signature _must_ be repaired after make-fingerprint is
> run.  This can be done quite simply, using 'codesign -s - -f temacs.tmp', which
> creates a new "ad-hoc" signature for the executable.
>
> If necessary, I am happy to test a patch / branch on my machine.
>

The approach to resign the executable after temacs invalidates the
digital signature seems like a good approach to me. It's also
the recommended approach in Apple's release notes:
https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-universal-apps-beta-release-notes

Note that we may want to preserve some metadata from the original
digital signature and resign the Mach-O file with something like:

codesign -s - —preserve-metadata=identifier,entitlements,flags,runtime -f temacs.tmp

But I'm not sure if it'd make a significant difference.

bug#43878: patch for 43878

[Itai Seggev]

Please find attached a patch for this bug.  I'm neither an autoconf nor emacs
build system expert, so it might be a bit naive, but it works for me.  (Also, I
hope attachments survive.  If they don't, I can send this in the body.)

--
Itai

In 1997 a group of programmers started writing a desktop environment to fix a
travesty they didn't create.  Their program promptly found its way onto un*x
systems everywhere. Today, still opposed by a software monopolist, they survive
as soldiers of fortune.  If you share their vision, if you know you can help,
and if you can connect to internet, maybe you can join... the K-Team.

bug#43878: patch for 43878

[Lars Ingebrigtsen]

Itai Seggev <is+apple@cs.hmc.edu> writes:

> Please find attached a patch for this bug.  I'm neither an autoconf
> nor emacs build system expert, so it might be a bit naive, but it
> works for me.  (Also, I hope attachments survive.  If they don't, I
> can send this in the body.)

The attachments didn't survive, apparently.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#43878: patch for 43878

[Itai Seggev]

On Mon, Nov 16, 2020 at 10:44:30PM +0100, Lars Ingebrigtsen wrote:
> Itai Seggev <is+apple@cs.hmc.edu> writes:
> 
> > Please find attached a patch for this bug.  I'm neither an autoconf
> > nor emacs build system expert, so it might be a bit naive, but it
> > works for me.  (Also, I hope attachments survive.  If they don't, I
> > can send this in the body.)
> 
> The attachments didn't survive, apparently.

This time inlined:

diff --git a/src/Makefile.in b/src/Makefile.in
index c5fb2ea3ab..6b09125e06 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -653,6 +653,9 @@ temacs$(EXEEXT):
 	  $(ALLOBJS) $(LIBEGNU_ARCHIVE) $(W32_RES_LINK) $(LIBES)
 ifeq ($(HAVE_PDUMPER),yes)
 	$(AM_V_at)$(MAKE_PDUMPER_FINGERPRINT) $@.tmp
+ifeq ($(shell uname),Darwin)
+	codesign -s - -f $@.tmp
+endif
 endif
 	$(AM_V_at)mv $@.tmp $@
 	$(MKDIR_P) $(etc)


--
Itai

In 1997 a group of programmers started writing a desktop environment to fix a
travesty they didn't create.  Their program promptly found its way onto un*x
systems everywhere. Today, still opposed by a software monopolist, they survive
as soldiers of fortune.  If you share their vision, if you know you can help,
and if you can connect to internet, maybe you can join... the K-Team.

bug#43878: patch for 43878

[Alan Third]

On Tue, Nov 17, 2020 at 12:36:10PM -0600, Itai Seggev wrote:
> +ifeq ($(shell uname),Darwin)

Is $DARWIN_OS available in the Makefile?

-- 
Alan Third

bug#43878: patch for 43878

[Itai Seggev]

On Wed, Nov 18, 2020 at 11:34:15PM +0000, Alan Third wrote:
> On Tue, Nov 17, 2020 at 12:36:10PM -0600, Itai Seggev wrote:
> > +ifeq ($(shell uname),Darwin)
> 
> Is $DARWIN_OS available in the Makefile?

Not as far as I can tell.  But here's a revised patch that is both more
targeted and only using autoconf variables:

diff --git a/src/Makefile.in b/src/Makefile.in
index c5fb2ea3ab..02d50bb7ca 100644
--- a/src/Makefile.in
+++ b/src/Makefile.in
@@ -336,6 +336,10 @@ DUMPING=
 CHECK_STRUCTS = @CHECK_STRUCTS@
 HAVE_PDUMPER = @HAVE_PDUMPER@
 
+## ARM Macs require that all code have a valid signature.  Since pump
+## invalidates the signature, we must re-sign to fix it.
+DO_CODESIGN=$(patsubst @configuration@,aarch64-apple-darwin%,yes)
+
 # 'make' verbosity.
 AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@
 
@@ -653,6 +657,9 @@ temacs$(EXEEXT):
 	  $(ALLOBJS) $(LIBEGNU_ARCHIVE) $(W32_RES_LINK) $(LIBES)
 ifeq ($(HAVE_PDUMPER),yes)
 	$(AM_V_at)$(MAKE_PDUMPER_FINGERPRINT) $@.tmp
+ifeq ($(DO_CODESIGN),yes)
+	codesign -s - -f $@.tmp
+endif
 endif
 	$(AM_V_at)mv $@.tmp $@
 	$(MKDIR_P) $(etc)


--
Itai

In 1997 a group of programmers started writing a desktop environment to fix a
travesty they didn't create.  Their program promptly found its way onto un*x
systems everywhere. Today, still opposed by a software monopolist, they survive
as soldiers of fortune.  If you share their vision, if you know you can help,
and if you can connect to internet, maybe you can join... the K-Team.

bug#43878: patch for 43878

[Lars Ingebrigtsen]

Itai Seggev <is+apple@cs.hmc.edu> writes:

> Not as far as I can tell.  But here's a revised patch that is both more
> targeted and only using autoconf variables:

I don't have an ARM Apple machine to test on (yet), but I guess the
patch looks reasonable, so I've applied it to the trunk, and we'll see
whether anybody complains.

I had to fix up the patch, though -- the syntax was wrong, and led to
codesigning on all platforms.  I did

DO_CODESIGN=$(patsubst aarch64-apple-darwin%,yes,@configuration@)

instead, which may or may not work (as I've got nothing to test on).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#43878: patch for 43878

[Itai Seggev]

Tested master.  Works on my ARM Mac.  Thanks!

On Tue, Nov 24, 2020 at 06:54:07AM +0100, Lars Ingebrigtsen wrote:
> Itai Seggev <is+apple@cs.hmc.edu> writes:
> 
> > Not as far as I can tell.  But here's a revised patch that is both more
> > targeted and only using autoconf variables:
> 
> I don't have an ARM Apple machine to test on (yet), but I guess the
> patch looks reasonable, so I've applied it to the trunk, and we'll see
> whether anybody complains.
> 
> I had to fix up the patch, though -- the syntax was wrong, and led to
> codesigning on all platforms.  I did
> 
> DO_CODESIGN=$(patsubst aarch64-apple-darwin%,yes,@configuration@)
> 
> instead, which may or may not work (as I've got nothing to test on).
> 
> -- 
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no
--
Itai

In 1997 a group of programmers started writing a desktop environment to fix a
travesty they didn't create.  Their program promptly found its way onto un*x
systems everywhere. Today, still opposed by a software monopolist, they survive
as soldiers of fortune.  If you share their vision, if you know you can help,
and if you can connect to internet, maybe you can join... the K-Team.