bug#54677: Creating core files on macOS (Monterey)

[Perry Smith <pedz@easesoftware.com>]

The CONTRIBUTE document says to submit feature requests, etc to this
list.

I wish I could provide more expertise on this topic but I’m not a
serious macOS developer.  I've cobbled together the info below over
the past few days.

Apple (like AIX) has decided that core files are bad and so has
disabled them in the normal activities.  So, a few things need to be
done to get a core file from a particular applicaiton.

Another Apple trait seems to be that if they did it yesterday, it must
be changed for today.  Thus, what follows is ONLY for Monterey (I've
been told).  The previous Apple releases require some but not all of
these changes.  AND... it appears that what is required for the M1
chips (Arm / Apple silicon) is not the same as what is required for
Intel chips.  So... I know that without these changes core dumps on
Monterey with an M1 chip were not created and after these changes they
are.

As I mentioned in an email on the developers list, making emacs
remains a drop dead easy process where

./configure ; make ; make install

creates nextstep/Emacs.app

ls -ld@ nextstep/Emacs.app

shows:

drwxr-xr-x@ 3 pedz  staff  96 Apr  1 18:08 nextstep/Emacs.app
	com.apple.quarantine	57 

To get the Emacs.app to launch, the quarantine needs to be removed and
that is done with:

sudo xattr -rds com.apple.quarantine nextstep/Emacs.app

The app can then be moved to /Applications and it will launch via
Finder, etc but will not create a core file.

To get an application to create a core file two particular things need
to be done.  The resource limit for core files needs to be set to the
max.  And an entitlement needs to be applied to the application.

The first I accomplished with this code change:

diff -r emacs-orig-27.2/src/emacs.c emacs-new-27.2/src/emacs.c
1226c1226,1234
< 
---
> #if defined (HAVE_SETRLIMIT) && defined (RLIMIT_CORE) && !defined (CYGWIN)
>   if (getrlimit (RLIMIT_CORE, &rlim) == 0
>       && 0 <= rlim.rlim_cur && rlim.rlim_cur <= rlim.rlim_max)
>     {
>       rlim.rlim_cur = rlim.rlim_max;
>       setrlimit (RLIMIT_CORE, &rlim); /* Error??? */
>     }  
> #endif /* HAVE_SETRLIMIT and RLIMIT_CORE and not CYGWIN */
>   

The second I accomplished with this small script:

#!/bin/zsh

EMACS=/Applications/Emacs.app
if [[ -d nextstep/Emacs.app ]] ; then
  sudo rm -rf ${EMACS}
  sudo mv nextstep/Emacs.app ${EMACS}
  rm -f /tmp/entitlements
  /usr/libexec/PlistBuddy -c "Add :com.apple.security.get-task-allow bool true" /tmp/entitlements
  sudo codesign -s - -f --deep --entitlements tmp.entitlements ${EMACS}
else
  echo nextstep/Emacs.app is not present
fi

The PlistBuddy command will spew out a message:

File Doesn't Exist, Will Create: /tmp/entitlements