* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location @ 2021-06-29 11:58 Jim Myhrberg 2021-06-29 19:18 ` Alan Third 0 siblings, 1 reply; 18+ messages in thread From: Jim Myhrberg @ 2021-06-29 11:58 UTC (permalink / raw) To: 49271 Commit 5dd2d50 seems to have moved the native-lisp folder within self-contained Emacs.app bundles: - from: "Contents/Resources/native-lisp" - to: "Contents/MacOS/lib/emacs/28.0.50/native-lisp" Unfortunately, Apple's codesign utility blows up with an error if there is any folder within "Contents/MacOS" (recursively) which contains two dots within its name. Here is an example of what the error looks like: /Users/runner/work/emacs-builds/emacs-builds/builds/Emacs.2021-06-26.b8f9e58.master.macOS-10-15.x86_64/Emacs.app: bundle format unrecognized, invalid, or unsuitable In subcomponent: /Users/runner/work/emacs-builds/emacs-builds/builds/Emacs.2021-06-26.b8f9e58.master.macOS-10-15.x86_64/Emacs.app/Contents/MacOS/lib/emacs/28.0.50 The above error is from this GitHub Actions workflow run: https://github.com/jimeh/emacs-builds/runs/2923284812?check_suite_focus=true The same issue happens locally for me too. I have so far not found a way to make codesign not blow up on such folders, so for now I've had to workaround the issue in my build script with the following changes: - https://github.com/jimeh/build-emacs-for-macos/commit/eeca7b798de236a3ffc1ab04b0f7735a37ce5af4 - https://github.com/jimeh/build-emacs-for-macos/commit/23b8236e0a66fb09810e8422bedf02f7192a53e4 In summary, I rename: - from: Emacs.app/Contents/MacOS/lib/emacs/28.0.50/native-lisp/28.0.50-852ecda2 - to: Emacs.app/Contents/MacOS/lib/emacs/28-0-50/native-lisp/28-0-50-852ecda2 (Note that I had to rename both "28.0.50" and "28.0.50-852ecda2" folders, as both trigger the error from codesign.) I create a symlink from "Contents/native-lisp" to "Contents/MacOS/lib/emacs/28-0-50/native-lisp". And finally I patch Emacs.pdmp (Emacs.app/Contents/MacOS/libexec/Emacs.pdmp), performing the following raw text replacements: - "lib/emacs/28.0.50/native-lisp/28.0.50-852ecda2/" --> "lib/emacs/28-0-50/native-lisp/28-0-50-852ecda2/" - "../native-lisp/28.0.50-852ecda2/" --> "../native-lisp/28-0-50-852ecda2/" This yields a seemingly actually working Emacs.app bundle, but it re-compiles all *.eln files in the bundle into the user cache folder. However, there's a separate bug causing re-compiling which I've reported as bug#49270, so for now I don't know if my patching would prevent Emacs from dynamically finding existing *.eln which are not referenced from Emacs.pdmp. This patching/workaround to get code signing to work feels very hacky and fragile. It would be great if we could avoid the need for this kind of patch to be able to codesign the .app bundle. The way I see it, there's three possible solutions: - Figure out a way to make Apple's codesign play nice. - Move the native-lisp folder back to "Contents/Resources/native-lisp", as codesign does not complain about folders with two dots in their names within the Resources directory. - Change paths used to store *.eln files within Contents/MacOS so no single folder name contains two dots. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-29 11:58 bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location Jim Myhrberg @ 2021-06-29 19:18 ` Alan Third 2021-06-30 10:04 ` Jim Myhrberg 0 siblings, 1 reply; 18+ messages in thread From: Alan Third @ 2021-06-29 19:18 UTC (permalink / raw) To: Jim Myhrberg; +Cc: 49271 On Tue, Jun 29, 2021 at 12:58:44PM +0100, Jim Myhrberg wrote: > Commit 5dd2d50 seems to have moved the native-lisp folder within self-contained > Emacs.app bundles: > > - from: "Contents/Resources/native-lisp" > - to: "Contents/MacOS/lib/emacs/28.0.50/native-lisp" > > Unfortunately, Apple's codesign utility blows up with an error if there is any > folder within "Contents/MacOS" (recursively) which contains two dots within its > name. Here is an example of what the error looks like: > > /Users/runner/work/emacs-builds/emacs-builds/builds/Emacs.2021-06-26.b8f9e58.master.macOS-10-15.x86_64/Emacs.app: > bundle format unrecognized, invalid, or unsuitable > In subcomponent: > /Users/runner/work/emacs-builds/emacs-builds/builds/Emacs.2021-06-26.b8f9e58.master.macOS-10-15.x86_64/Emacs.app/Contents/MacOS/lib/emacs/28.0.50 Bummer. I had three options: * Contents/MacOS/lib * Contents/Resources/<something> * Contents/lib and a close reading of the Apple documentation left me none-the-wiser as to which option I should use. Executable binaries go under MacOS, but these aren't executables. Framework libraries go somewhere else entirely, but these aren't framework libraries. Resources is intended for images and things, not libs. Lib is entirely non-standard. I really don't know where the best place is. I'm still thinking Resources is definitely not the right place, but none of the other existing places make any sense, so perhaps the non-standard /Contents/lib is the best option... Can you try that? In order to sort it edit configure.ac, search for the first occurrence of "ns_applibdir" and change the path. If that fails then I guess we move them back under Resources. Unless you have any better ideas. -- Alan Third ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-29 19:18 ` Alan Third @ 2021-06-30 10:04 ` Jim Myhrberg 2021-06-30 12:20 ` Eli Zaretskii 0 siblings, 1 reply; 18+ messages in thread From: Jim Myhrberg @ 2021-06-30 10:04 UTC (permalink / raw) To: Alan Third, Jim Myhrberg, 49271 I've just tried "Contents/lib", and it allowed me to sign and notarize the .app bundle. And combined with your patch from bug#49270, all bundled *.eln files are also correctly located and loaded :) I did a bit of searching myself for alternatives and found "Contents/PlugIns" as a potentially suitable place, but a quick test revealed codesign fails the same way with it as it does with Contents/MacOS. Personally I think Contents/lib is probably fine, as both codesign and Apple's notarization process are happy with it. And the notarization process seems very picky. For example, when *.eln files were in Resources/native-lisp, my initial notarization attempts failed because it considered the *.eln files to be binaries, and they had not been signed by codesign despite the --deep flag being used. Hence I'm individually signing all the *.eln files before signing the app bundle itself to get the app through notarization. Also, I should probably mention that I generally know little about macOS app development, and have mostly just tested my way forward with figuring out how to sign and notarize Emacs :) On Tue, Jun 29, 2021 at 8:19 PM Alan Third <alan@idiocy.org> wrote: > > On Tue, Jun 29, 2021 at 12:58:44PM +0100, Jim Myhrberg wrote: > > Commit 5dd2d50 seems to have moved the native-lisp folder within self-contained > > Emacs.app bundles: > > > > - from: "Contents/Resources/native-lisp" > > - to: "Contents/MacOS/lib/emacs/28.0.50/native-lisp" > > > > Unfortunately, Apple's codesign utility blows up with an error if there is any > > folder within "Contents/MacOS" (recursively) which contains two dots within its > > name. Here is an example of what the error looks like: > > > > /Users/runner/work/emacs-builds/emacs-builds/builds/Emacs.2021-06-26.b8f9e58.master.macOS-10-15.x86_64/Emacs.app: > > bundle format unrecognized, invalid, or unsuitable > > In subcomponent: > > /Users/runner/work/emacs-builds/emacs-builds/builds/Emacs.2021-06-26.b8f9e58.master.macOS-10-15.x86_64/Emacs.app/Contents/MacOS/lib/emacs/28.0.50 > > Bummer. > > I had three options: > > * Contents/MacOS/lib > * Contents/Resources/<something> > * Contents/lib > > and a close reading of the Apple documentation left me none-the-wiser > as to which option I should use. Executable binaries go under MacOS, > but these aren't executables. Framework libraries go somewhere else > entirely, but these aren't framework libraries. Resources is intended > for images and things, not libs. Lib is entirely non-standard. > > I really don't know where the best place is. I'm still thinking > Resources is definitely not the right place, but none of the other > existing places make any sense, so perhaps the non-standard > /Contents/lib is the best option... > > Can you try that? In order to sort it edit configure.ac, search for > the first occurrence of "ns_applibdir" and change the path. > > If that fails then I guess we move them back under Resources. Unless > you have any better ideas. > -- > Alan Third ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-30 10:04 ` Jim Myhrberg @ 2021-06-30 12:20 ` Eli Zaretskii 2021-06-30 12:39 ` Jim Myhrberg 2021-06-30 12:42 ` Alan Third 0 siblings, 2 replies; 18+ messages in thread From: Eli Zaretskii @ 2021-06-30 12:20 UTC (permalink / raw) To: Jim Myhrberg; +Cc: alan, contact, 49271 > From: Jim Myhrberg <contact@jimeh.me> > Date: Wed, 30 Jun 2021 11:04:43 +0100 > > I've just tried "Contents/lib", and it allowed me to sign and notarize > the .app bundle. And combined with your patch from bug#49270, all > bundled *.eln files are also correctly located and loaded :) > > I did a bit of searching myself for alternatives and found > "Contents/PlugIns" as a potentially suitable place, but a quick test > revealed codesign fails the same way with it as it does with > Contents/MacOS. > > Personally I think Contents/lib is probably fine, as both codesign and > Apple's notarization process are happy with it. And the notarization > process seems very picky. For example, when *.eln files were in > Resources/native-lisp, my initial notarization attempts failed because > it considered the *.eln files to be binaries, and they had not been > signed by codesign despite the --deep flag being used. Hence I'm > individually signing all the *.eln files before signing the app bundle > itself to get the app through notarization. The *.eln files are shared libraries. What is the canonical place to install shared libraries specific to an application? ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-30 12:20 ` Eli Zaretskii @ 2021-06-30 12:39 ` Jim Myhrberg 2021-06-30 12:52 ` Alan Third 2021-06-30 12:42 ` Alan Third 1 sibling, 1 reply; 18+ messages in thread From: Jim Myhrberg @ 2021-06-30 12:39 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Alan Third, 49271 On Wed, Jun 30, 2021 at 1:20 PM Eli Zaretskii <eliz@gnu.org> wrote: > The *.eln files are shared libraries. What is the canonical place to > install shared libraries specific to an application? This page indicates that "Contents/Frameworks" is the place for shared libraries: https://wiki.lazarus.freepascal.org/Application_Bundle I just poked around the internals of a bunch of Apple's apps and a few third party apps, and it looks like "Contents/Frameworks" is where they're storing *.dylib files among other things. So it might be the sensible place for shared libraries. However, the same "two dots in folder name" issue applies with codesign there too, so to use it we'd need to drop the emacs version (28.0.50) from all folder paths. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-30 12:39 ` Jim Myhrberg @ 2021-06-30 12:52 ` Alan Third 2021-06-30 13:10 ` Jim Myhrberg 0 siblings, 1 reply; 18+ messages in thread From: Alan Third @ 2021-06-30 12:52 UTC (permalink / raw) To: Jim Myhrberg; +Cc: 49271 On Wed, Jun 30, 2021 at 01:39:10PM +0100, Jim Myhrberg wrote: > On Wed, Jun 30, 2021 at 1:20 PM Eli Zaretskii <eliz@gnu.org> wrote: > > > The *.eln files are shared libraries. What is the canonical place to > > install shared libraries specific to an application? > > This page indicates that "Contents/Frameworks" is the place for shared > libraries: https://wiki.lazarus.freepascal.org/Application_Bundle > > I just poked around the internals of a bunch of Apple's apps and a few > third party apps, and it looks like "Contents/Frameworks" is where > they're storing *.dylib files among other things. So it might be the > sensible place for shared libraries. However, the same "two dots in > folder name" issue applies with codesign there too, so to use it we'd > need to drop the emacs version (28.0.50) from all folder paths. I just found this page which explains the deal with the dots in directory names: https://developer.apple.com/library/archive/technotes/tn2206/_index.html search for "period". It also explains why you need to put code in certain directories. I don't know why that page has been so hard to find. Anyway, I guess we do need to change the install location so it doesn't have the dots. It shouldn't really matter for the application bundle since you can't have multiple versions of Emacs installed in one bundle (without some hardcore messing around like the emacsformacosx.com builds). -- Alan Third ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-30 12:52 ` Alan Third @ 2021-06-30 13:10 ` Jim Myhrberg 2021-06-30 19:05 ` Alan Third 0 siblings, 1 reply; 18+ messages in thread From: Jim Myhrberg @ 2021-06-30 13:10 UTC (permalink / raw) To: Alan Third, Jim Myhrberg, Eli Zaretskii, 49271 On Wed, Jun 30, 2021 at 1:53 PM Alan Third <alan@idiocy.org> wrote: > I just found this page which explains the deal with the dots in > directory names: > > https://developer.apple.com/library/archive/technotes/tn2206/_index.html > > search for "period". > > It also explains why you need to put code in certain directories. I > don't know why that page has been so hard to find. Damn, yeah it makes sense now :) > Anyway, I guess we do need to change the install location so it > doesn't have the dots. It shouldn't really matter for the application > bundle since you can't have multiple versions of Emacs installed in > one bundle (without some hardcore messing around like the > emacsformacosx.com builds). For multi-arch builds I think special care will need to be taken anyway. The emacsformacosx.com builds for example have duplicate shared libraries for each supported architecture. Personally though I'm planning to just create separate builds/dmgs for each when I'm able to support anything other than x86_64. The main self-contained Emacs.app my builds are producing at the moment are already 500MB thanks to full native AoT and essentially copying most of GCC into the bundle for libgccjit to work. Also, I'd obviously be more than happy to help with testing any further changes to eln file locations and related things :) ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-30 13:10 ` Jim Myhrberg @ 2021-06-30 19:05 ` Alan Third 2021-07-01 7:13 ` Eli Zaretskii 2021-07-01 14:53 ` Jim Myhrberg 0 siblings, 2 replies; 18+ messages in thread From: Alan Third @ 2021-06-30 19:05 UTC (permalink / raw) To: Jim Myhrberg; +Cc: 49271 [-- Attachment #1: Type: text/plain, Size: 561 bytes --] On Wed, Jun 30, 2021 at 02:10:53PM +0100, Jim Myhrberg wrote: > > Also, I'd obviously be more than happy to help with testing any > further changes to eln file locations and related things :) Please try the attached patch. Eli, is this an acceptable way of modifying a lisp string, or is there a better way? I considered just removing the version since it doesn't matter much for the app bundle, but then I realised that would also remove it in the eln cache in .emacs.d, which isn't ideal, so I ended up replacing the dots with underscores. -- Alan Third [-- Attachment #2: 0001-Fix-NS-self-contained-eln-location-bug-49271.patch --] [-- Type: text/plain, Size: 2746 bytes --] From 2d408e5db6f4d111bdf687eea939b0897dad58cd Mon Sep 17 00:00:00 2001 From: Alan Third <alan@idiocy.org> Date: Wed, 30 Jun 2021 19:58:13 +0100 Subject: [PATCH] Fix NS self contained eln location (bug#49271) * Makefile.in: * configure.ac: Change eln file install location to Contents/Frameworks. * src/comp.c (hash_native_abi): Replace dots with underscores in the eln install location as the macOS code-signing tool won't sign the files if the parent directories have dots. --- Makefile.in | 2 +- configure.ac | 4 ++-- src/comp.c | 20 +++++++++++++++++++- 3 files changed, 22 insertions(+), 4 deletions(-) diff --git a/Makefile.in b/Makefile.in index 8fccdf7580..8c14c5cc7d 100644 --- a/Makefile.in +++ b/Makefile.in @@ -333,7 +333,7 @@ BIN_DESTDIR= ELN_DESTDIR = $(DESTDIR)${libdir}/emacs/${version}/ else BIN_DESTDIR='${ns_appbindir}/' -ELN_DESTDIR = ${ns_applibdir}/emacs/${version}/ +ELN_DESTDIR = ${ns_applibdir}/ endif all: ${SUBDIR} info diff --git a/configure.ac b/configure.ac index 6e2cda947a..c924634d5b 100644 --- a/configure.ac +++ b/configure.ac @@ -1895,7 +1895,7 @@ AC_DEFUN ns_appdir=`pwd`/nextstep/Emacs.app ns_appbindir=${ns_appdir}/Contents/MacOS ns_applibexecdir=${ns_appdir}/Contents/MacOS/libexec - ns_applibdir=${ns_appdir}/Contents/MacOS/lib + ns_applibdir=${ns_appdir}/Contents/Frameworks ns_appresdir=${ns_appdir}/Contents/Resources ns_appsrc=Cocoa/Emacs.base ns_fontfile=macfont.o @@ -1954,7 +1954,7 @@ AC_DEFUN ns_appdir=`pwd`/nextstep/Emacs.app ns_appbindir=${ns_appdir} ns_applibexecdir=${ns_appdir}/libexec - ns_applibdir=${ns_appdir}/lib + ns_applibdir=${ns_appdir}/Frameworks ns_appresdir=${ns_appdir}/Resources ns_appsrc=GNUstep/Emacs.base ns_fontfile=nsfont.o diff --git a/src/comp.c b/src/comp.c index ea05952627..ac77ae704d 100644 --- a/src/comp.c +++ b/src/comp.c @@ -744,8 +744,26 @@ hash_native_abi (void) Vsystem_configuration_options), Fmapconcat (intern_c_string ("comp--subr-signature"), Vcomp_subr_list, build_string ("")))); + + Lisp_Object version = Vemacs_version; + +#ifdef NS_SELF_CONTAINED + /* MacOS self contained app bundles do not like having dots in the + directory names under the Contents/Frameworks directory, so + convert them to underscores. */ + + char *str = xstrdup (SSDATA(version)); + + for (ptrdiff_t c = 0 ; c < SBYTES (version) ; c++) + if (*(str + c) == '.') + *(str + c) = '_'; + + version = build_string (str); + xfree (str); +#endif + Vcomp_native_version_dir = - concat3 (Vemacs_version, build_string ("-"), Vcomp_abi_hash); + concat3 (version, build_string ("-"), Vcomp_abi_hash); } static void -- 2.29.2 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-30 19:05 ` Alan Third @ 2021-07-01 7:13 ` Eli Zaretskii 2021-07-01 18:45 ` Alan Third 2021-07-01 14:53 ` Jim Myhrberg 1 sibling, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2021-07-01 7:13 UTC (permalink / raw) To: Alan Third; +Cc: contact, 49271 > Date: Wed, 30 Jun 2021 20:05:50 +0100 > From: Alan Third <alan@idiocy.org> > Cc: Eli Zaretskii <eliz@gnu.org>, 49271@debbugs.gnu.org > > Eli, is this an acceptable way of modifying a lisp string, or is there > a better way? > [...] > +#ifdef NS_SELF_CONTAINED > + /* MacOS self contained app bundles do not like having dots in the > + directory names under the Contents/Frameworks directory, so > + convert them to underscores. */ > + > + char *str = xstrdup (SSDATA(version)); > + > + for (ptrdiff_t c = 0 ; c < SBYTES (version) ; c++) > + if (*(str + c) == '.') > + *(str + c) = '_'; > + > + version = build_string (str); > + xfree (str); > +#endif It should work, although I'd prefer writing such code the other way around: first create an uninitialized Lisp string (with make_uninit_multibyte_string or make_uninit_string), then copy the bytes while making the conversions. The reason for this preference is that you could then make sure the produced string has the same multibyte-ness as the original, whereas the way you did it relies on whatever build_string decides, which is not necessarily the same. Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-07-01 7:13 ` Eli Zaretskii @ 2021-07-01 18:45 ` Alan Third 2021-07-01 19:03 ` Eli Zaretskii 0 siblings, 1 reply; 18+ messages in thread From: Alan Third @ 2021-07-01 18:45 UTC (permalink / raw) To: Eli Zaretskii; +Cc: contact, 49271 [-- Attachment #1: Type: text/plain, Size: 611 bytes --] On Thu, Jul 01, 2021 at 10:13:04AM +0300, Eli Zaretskii wrote: > It should work, although I'd prefer writing such code the other way > around: first create an uninitialized Lisp string (with > make_uninit_multibyte_string or make_uninit_string), then copy the > bytes while making the conversions. The reason for this preference is > that you could then make sure the produced string has the same > multibyte-ness as the original, whereas the way you did it relies on > whatever build_string decides, which is not necessarily the same. Thanks. I think the attached is doing what you suggested. -- Alan Third [-- Attachment #2: v2-0001-Fix-NS-self-contained-eln-location-bug-49271.patch --] [-- Type: text/plain, Size: 3155 bytes --] From 8de89ce0d2ae7f84ab2df025ae405b15994dfdb6 Mon Sep 17 00:00:00 2001 From: Alan Third <alan@idiocy.org> Date: Wed, 30 Jun 2021 19:58:13 +0100 Subject: [PATCH v2] Fix NS self contained eln location (bug#49271) * Makefile.in: * configure.ac: Change eln file install location to Contents/Frameworks. * src/comp.c (hash_native_abi): Replace dots with underscores in the eln install location as the macOS code-signing tool won't sign the files if the parent directories have dots. --- Makefile.in | 2 +- configure.ac | 4 ++-- src/comp.c | 35 ++++++++++++++++++++++++++++++++++- 3 files changed, 37 insertions(+), 4 deletions(-) diff --git a/Makefile.in b/Makefile.in index 8fccdf7580..8c14c5cc7d 100644 --- a/Makefile.in +++ b/Makefile.in @@ -333,7 +333,7 @@ BIN_DESTDIR= ELN_DESTDIR = $(DESTDIR)${libdir}/emacs/${version}/ else BIN_DESTDIR='${ns_appbindir}/' -ELN_DESTDIR = ${ns_applibdir}/emacs/${version}/ +ELN_DESTDIR = ${ns_applibdir}/ endif all: ${SUBDIR} info diff --git a/configure.ac b/configure.ac index 6e2cda947a..c924634d5b 100644 --- a/configure.ac +++ b/configure.ac @@ -1895,7 +1895,7 @@ AC_DEFUN ns_appdir=`pwd`/nextstep/Emacs.app ns_appbindir=${ns_appdir}/Contents/MacOS ns_applibexecdir=${ns_appdir}/Contents/MacOS/libexec - ns_applibdir=${ns_appdir}/Contents/MacOS/lib + ns_applibdir=${ns_appdir}/Contents/Frameworks ns_appresdir=${ns_appdir}/Contents/Resources ns_appsrc=Cocoa/Emacs.base ns_fontfile=macfont.o @@ -1954,7 +1954,7 @@ AC_DEFUN ns_appdir=`pwd`/nextstep/Emacs.app ns_appbindir=${ns_appdir} ns_applibexecdir=${ns_appdir}/libexec - ns_applibdir=${ns_appdir}/lib + ns_applibdir=${ns_appdir}/Frameworks ns_appresdir=${ns_appdir}/Resources ns_appsrc=GNUstep/Emacs.base ns_fontfile=nsfont.o diff --git a/src/comp.c b/src/comp.c index ea05952627..c45f9e3a7f 100644 --- a/src/comp.c +++ b/src/comp.c @@ -744,8 +744,41 @@ hash_native_abi (void) Vsystem_configuration_options), Fmapconcat (intern_c_string ("comp--subr-signature"), Vcomp_subr_list, build_string ("")))); + + Lisp_Object version = Vemacs_version; + +#ifdef NS_SELF_CONTAINED + /* MacOS self contained app bundles do not like having dots in the + directory names under the Contents/Frameworks directory, so + convert them to underscores. */ + unsigned char *to; + const unsigned char *from = SDATA (Vemacs_version); + + int multibyte = STRING_MULTIBYTE (Vemacs_version); + + version = multibyte + ? make_uninit_multibyte_string (SCHARS (Vemacs_version), + SBYTES (Vemacs_version)) + : make_uninit_string (SBYTES (Vemacs_version)); + + to = SDATA (version); + + while (from < SDATA (Vemacs_version) + SBYTES (Vemacs_version)) + { + int c = multibyte ? string_char_advance (&from) : *from++; + + if (c == '.') + c = '_'; + + if (multibyte) + to += CHAR_STRING (c, to); + else + *to++ = c; + } +#endif + Vcomp_native_version_dir = - concat3 (Vemacs_version, build_string ("-"), Vcomp_abi_hash); + concat3 (version, build_string ("-"), Vcomp_abi_hash); } static void -- 2.29.2 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-07-01 18:45 ` Alan Third @ 2021-07-01 19:03 ` Eli Zaretskii 2021-07-01 19:56 ` Alan Third 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2021-07-01 19:03 UTC (permalink / raw) To: Alan Third; +Cc: contact, 49271 > Date: Thu, 1 Jul 2021 19:45:46 +0100 > From: Alan Third <alan@idiocy.org> > Cc: contact@jimeh.me, 49271@debbugs.gnu.org > > > It should work, although I'd prefer writing such code the other way > > around: first create an uninitialized Lisp string (with > > make_uninit_multibyte_string or make_uninit_string), then copy the > > bytes while making the conversions. The reason for this preference is > > that you could then make sure the produced string has the same > > multibyte-ness as the original, whereas the way you did it relies on > > whatever build_string decides, which is not necessarily the same. > > Thanks. I think the attached is doing what you suggested. Yes. But I think you can simplify (and make it a tad faster) if you just copy bytes in either case. Since a multibyte string is represented by a superset of UTF-8, you are guaranteed that no multibyte sequence will ever include '.'. So converting from multibyte sequences to characters and back, as well as calls to string_char_advance, can be avoided. Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-07-01 19:03 ` Eli Zaretskii @ 2021-07-01 19:56 ` Alan Third 0 siblings, 0 replies; 18+ messages in thread From: Alan Third @ 2021-07-01 19:56 UTC (permalink / raw) To: Eli Zaretskii; +Cc: contact, 49271 On Thu, Jul 01, 2021 at 10:03:09PM +0300, Eli Zaretskii wrote: > > Date: Thu, 1 Jul 2021 19:45:46 +0100 > > From: Alan Third <alan@idiocy.org> > > Cc: contact@jimeh.me, 49271@debbugs.gnu.org > > > > > It should work, although I'd prefer writing such code the other way > > > around: first create an uninitialized Lisp string (with > > > make_uninit_multibyte_string or make_uninit_string), then copy the > > > bytes while making the conversions. The reason for this preference is > > > that you could then make sure the produced string has the same > > > multibyte-ness as the original, whereas the way you did it relies on > > > whatever build_string decides, which is not necessarily the same. > > > > Thanks. I think the attached is doing what you suggested. > > Yes. But I think you can simplify (and make it a tad faster) if you > just copy bytes in either case. Since a multibyte string is > represented by a superset of UTF-8, you are guaranteed that no > multibyte sequence will ever include '.'. So converting from > multibyte sequences to characters and back, as well as calls to > string_char_advance, can be avoided. I'm glad I asked about this because I had completely misunderstood how multibyte strings are represented. This makes a lot more sense. Thanks! -- Alan Third ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-30 19:05 ` Alan Third 2021-07-01 7:13 ` Eli Zaretskii @ 2021-07-01 14:53 ` Jim Myhrberg 2021-07-01 20:13 ` Alan Third 2021-07-01 21:16 ` Alan Third 1 sibling, 2 replies; 18+ messages in thread From: Jim Myhrberg @ 2021-07-01 14:53 UTC (permalink / raw) To: Alan Third; +Cc: 49271 > On 30 Jun 2021, at 20:05, Alan Third <alan@idiocy.org> wrote: > > Please try the attached patch. I've successfully built, signed and notarized a full AoT build with this patch. I also applied the patch from bug#49270 to ensure *.eln files are located correctly within the bundle. Everything is working as expected, and in fact, codesign even automatically finds and signs all *.eln files within Contents/Frameworks when signing the app bundle itself, so I no longer need to individually sign each *.eln file before signing the whole app bundle :) ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-07-01 14:53 ` Jim Myhrberg @ 2021-07-01 20:13 ` Alan Third 2021-07-01 20:43 ` Jim Myhrberg 2021-07-01 21:16 ` Alan Third 1 sibling, 1 reply; 18+ messages in thread From: Alan Third @ 2021-07-01 20:13 UTC (permalink / raw) To: Jim Myhrberg; +Cc: 49271 On Thu, Jul 01, 2021 at 03:53:13PM +0100, Jim Myhrberg wrote: > I've successfully built, signed and notarized a full AoT build with > this patch. I also applied the patch from bug#49270 to ensure *.eln > files are located correctly within the bundle. > > Everything is working as expected, and in fact, codesign even > automatically finds and signs all *.eln files within > Contents/Frameworks when signing the app bundle itself, so I no > longer need to individually sign each *.eln file before signing the > whole app bundle :) I'm curious what you're actually doing. We have some code that runs on ARM based Macs only that does this codesign -s - -f $@.tmp where I think $@.tmp is... something... executable? I'm wondering if we should just install everything into the app bundle and sign the whole damn thing. I don't really understand this whole codesigning thing. It seems to make no difference here, but I'm not trying to run the app on another Mac or on an ARM Mac, so I think it's normal that I shouldn't see any difference. -- Alan Third ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-07-01 20:13 ` Alan Third @ 2021-07-01 20:43 ` Jim Myhrberg 0 siblings, 0 replies; 18+ messages in thread From: Jim Myhrberg @ 2021-07-01 20:43 UTC (permalink / raw) To: Alan Third; +Cc: 49271 > On 1 Jul 2021, at 21:13, Alan Third <alan@idiocy.org> wrote: > > I'm curious what you're actually doing. We have some code that runs on > ARM based Macs only that does this > > codesign -s - -f $@.tmp > > where I think $@.tmp is... something... executable? > > I'm wondering if we should just install everything into the app bundle > and sign the whole damn thing. > > I don't really understand this whole codesigning thing. It seems to > make no difference here, but I'm not trying to run the app on another > Mac or on an ARM Mac, so I think it's normal that I shouldn't see any > difference. I can't say I know what the `$@.tmp` thing is either, or really understand codesign for that matter. I've managed to make it work through trial and error... lol To explain a bit more of what I'm doing though; I basically got bored of making my laptop sound like a jet engine for 25 minutes every other day as I made new builds from the master branch, so I set out to make nightly builds using GitHub Actions which I could just download and/or install/upgrade via brew cask: https://github.com/jimeh/emacs-builds With the heavy lifting done by my custom build script: https://github.com/jimeh/build-emacs-for-macos The signing and notarizing stuff also lives in the build script repo, but as part of a new (and somewhat hacky) "emacs-builder" CLI tool written in Go, which is designed to automated a bunch of the steps. Because of this, you can see the full build process I'm doing on GitHub Actions: https://github.com/jimeh/emacs-builds/runs/2957583013?check_suite_focus=true As for the *.eln file, previously when they were in Contents/Resources I had to sign each one individually before signing the bundle as a whole. But I didn't check if that was still needed after they were moved to Contents/MacOS a few days ago. But I have checked it with them under Contents/Frameworks, and signing the *.eln files is no longer needed as codesign finds them when signing the .app bundle itself.. As for exactly what I'm doing, this is essentially the main command which signs the .app bundle: codesign --sign <cert-id> --deep --timestamp --force --verbose \ --options runtime --entitlements <path-to-entitlements.plist> \ /path/to/Emacs.app And the entitlements I use are: - com.apple.security.automation.apple-events - com.apple.security.cs.allow-jit - com.apple.security.cs.disable-library-validation - com.apple.security.network.client Which seem to be enough for everything I do with Emacs to work correctly. I do also sign a custom little "emacs" shell-script in Contents/MacOS/bin, which I add to make it easy to expose a "emacs" CLI tool, all it does is correctly resolve the path to the .app bundle, and execute the main MacOS/Emacs binary from its absolute real path so it can find everything it depends on within the app bundle even when symlinked to somewhere else. I'm happy to answer any questions you might have :) ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-07-01 14:53 ` Jim Myhrberg 2021-07-01 20:13 ` Alan Third @ 2021-07-01 21:16 ` Alan Third 1 sibling, 0 replies; 18+ messages in thread From: Alan Third @ 2021-07-01 21:16 UTC (permalink / raw) To: Jim Myhrberg; +Cc: 49271-done On Thu, Jul 01, 2021 at 03:53:13PM +0100, Jim Myhrberg wrote: > > On 30 Jun 2021, at 20:05, Alan Third <alan@idiocy.org> wrote: > > > > Please try the attached patch. > > I've successfully built, signed and notarized a full AoT build with > this patch. I also applied the patch from bug#49270 to ensure *.eln > files are located correctly within the bundle. > > Everything is working as expected, and in fact, codesign even > automatically finds and signs all *.eln files within > Contents/Frameworks when signing the app bundle itself, so I no > longer need to individually sign each *.eln file before signing the > whole app bundle :) I've pushed an updated version of this patch to master, so I'll close this bug report. (I'm going to reply to your other email, but I'll redirect it to emacs-devel.) -- Alan Third ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-30 12:20 ` Eli Zaretskii 2021-06-30 12:39 ` Jim Myhrberg @ 2021-06-30 12:42 ` Alan Third 2021-06-30 12:51 ` Jim Myhrberg 1 sibling, 1 reply; 18+ messages in thread From: Alan Third @ 2021-06-30 12:42 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Jim Myhrberg, 49271 On Wed, Jun 30, 2021 at 03:20:41PM +0300, Eli Zaretskii wrote: > The *.eln files are shared libraries. What is the canonical place to > install shared libraries specific to an application? AFAICT there isn't one. Apple don't expect you to use any, or at least not to copy them into your application bundle. The Objective C equivalent are Frameworks, and there's a place for them, but they have a specific file structure and I don't think we want to try and emulate that for standard shared libraries. There also doesn't appear to be any consensus in the community as I've seen several places suggested. I'm inclined to towards Contents/lib. It works for Jim and I think it should be fairly clear to outside observers what's going on. Although I've just done another search and found multiple people suggesting Contents/Frameworks is the correct place to put them. Jim, does putting them in Frameworks work or does that have signing implications too? (None of this seems to matter at all on my machine, so I can't test it myself.) -- Alan Third ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location 2021-06-30 12:42 ` Alan Third @ 2021-06-30 12:51 ` Jim Myhrberg 0 siblings, 0 replies; 18+ messages in thread From: Jim Myhrberg @ 2021-06-30 12:51 UTC (permalink / raw) To: Alan Third, Eli Zaretskii, Jim Myhrberg, 49271 On Wed, Jun 30, 2021 at 1:42 PM Alan Third <alan@idiocy.org> wrote: > Jim, does putting them in Frameworks work or does that have signing > implications too? You must not have seen my email from a couple of minutes ago, I'm afraid I tested Frameworks and it has the same folder naming issues. From the apps I poked around in, they've just put all their *.dylib files directly in the root of the Frameworks directory. > (None of this seems to matter at all on my machine, so I can't test it > myself.) What version of macOS/Xcode do you have? Case I've seen slight differences in behavior with codesign on Catalina and Big Sur at least (unrelated to the two dots issue though). I've written a small CLI tool to make life easier for myself when it comes to signing, dmg packaging, and notarizing Emacs. I'm happy to point you towards it if you want. ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2021-07-01 21:16 UTC | newest] Thread overview: 18+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-06-29 11:58 bug#49271: 28.0.50: native-comp: Signing macOS self-contained .app bundle fails due to new *.eln location Jim Myhrberg 2021-06-29 19:18 ` Alan Third 2021-06-30 10:04 ` Jim Myhrberg 2021-06-30 12:20 ` Eli Zaretskii 2021-06-30 12:39 ` Jim Myhrberg 2021-06-30 12:52 ` Alan Third 2021-06-30 13:10 ` Jim Myhrberg 2021-06-30 19:05 ` Alan Third 2021-07-01 7:13 ` Eli Zaretskii 2021-07-01 18:45 ` Alan Third 2021-07-01 19:03 ` Eli Zaretskii 2021-07-01 19:56 ` Alan Third 2021-07-01 14:53 ` Jim Myhrberg 2021-07-01 20:13 ` Alan Third 2021-07-01 20:43 ` Jim Myhrberg 2021-07-01 21:16 ` Alan Third 2021-06-30 12:42 ` Alan Third 2021-06-30 12:51 ` Jim Myhrberg
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.