From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail
From: Torrekie <me@torrekie.dev>
Newsgroups: gmane.lisp.guile.bugs
Subject: bug#61476: mmap() with RWX cannot correctly invalidate cache
Date: Mon, 13 Feb 2023 17:09:54 +0800
Message-ID: <5327BA39-E711-4178-8E7F-37645137BF51@torrekie.dev>
Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\))
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_7FD80609-3E3C-4B38-A565-5728DCC1029B"
Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214";
	logging-data="40401"; mail-complaints-to="usenet@ciao.gmane.io"
To: 61476@debbugs.gnu.org
Original-X-From: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Mon Feb 13 15:35:20 2023
Return-path: <bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org>
Envelope-to: guile-bugs@m.gmane-mx.org
Original-Received: from lists.gnu.org ([209.51.188.17])
	by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
	(Exim 4.92)
	(envelope-from <bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org>)
	id 1pRZvK-000AFW-AD
	for guile-bugs@m.gmane-mx.org; Mon, 13 Feb 2023 15:35:18 +0100
Original-Received: from localhost ([::1] helo=lists1p.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.90_1)
	(envelope-from <bug-guile-bounces@gnu.org>)
	id 1pRZv8-00028R-5B; Mon, 13 Feb 2023 09:35:06 -0500
Original-Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>)
 id 1pRZv5-000284-Kz
 for bug-guile@gnu.org; Mon, 13 Feb 2023 09:35:03 -0500
Original-Received: from debbugs.gnu.org ([209.51.188.43])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>)
 id 1pRZv5-0004ES-5B
 for bug-guile@gnu.org; Mon, 13 Feb 2023 09:35:03 -0500
Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2)
 (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1pRZv4-0005Z4-LK
 for bug-guile@gnu.org; Mon, 13 Feb 2023 09:35:02 -0500
X-Loop: help-debbugs@gnu.org
Resent-From: Torrekie <me@torrekie.dev>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org>
Resent-CC: bug-guile@gnu.org
Resent-Date: Mon, 13 Feb 2023 14:35:02 +0000
Resent-Message-ID: <handler.61476.B.167629884321290@debbugs.gnu.org>
Resent-Sender: help-debbugs@gnu.org
X-GNU-PR-Message: report 61476
X-GNU-PR-Package: guile
X-Debbugs-Original-To: bug-guile@gnu.org
Original-Received: via spool by submit@debbugs.gnu.org id=B.167629884321290
 (code B ref -1); Mon, 13 Feb 2023 14:35:02 +0000
Original-Received: (at submit) by debbugs.gnu.org; 13 Feb 2023 14:34:03 +0000
Original-Received: from localhost ([127.0.0.1]:48053 helo=debbugs.gnu.org)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <debbugs-submit-bounces@debbugs.gnu.org>)
 id 1pRZu4-0005X9-2Q
 for submit@debbugs.gnu.org; Mon, 13 Feb 2023 09:34:03 -0500
Original-Received: from lists.gnu.org ([209.51.188.17]:45830)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <me@torrekie.dev>) id 1pRUqm-0000Kb-Dk
 for submit@debbugs.gnu.org; Mon, 13 Feb 2023 04:10:18 -0500
Original-Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <me@torrekie.dev>) id 1pRUql-0007ME-Ts
 for bug-guile@gnu.org; Mon, 13 Feb 2023 04:10:16 -0500
Original-Received: from mta-15-4.privateemail.com ([198.54.127.111])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <me@torrekie.dev>) id 1pRUqi-0007CK-6o
 for bug-guile@gnu.org; Mon, 13 Feb 2023 04:10:15 -0500
Original-Received: from mta-15.privateemail.com (localhost [127.0.0.1])
 by mta-15.privateemail.com (Postfix) with ESMTP id 45DA018000A7
 for <bug-guile@gnu.org>; Mon, 13 Feb 2023 04:09:59 -0500 (EST)
Original-Received: from [127.0.0.1] (unknown [104.245.98.35])
 by mta-15.privateemail.com (Postfix) with ESMTPA id E3EA518000A2
 for <bug-guile@gnu.org>; Mon, 13 Feb 2023 04:09:57 -0500 (EST)
X-Mailer: Apple Mail (2.3654.60.0.2.21)
X-Virus-Scanned: ClamAV using ClamSMTP
Received-SPF: pass client-ip=198.54.127.111; envelope-from=me@torrekie.dev;
 helo=MTA-15-4.privateemail.com
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, HTML_MESSAGE=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Mailman-Approved-At: Mon, 13 Feb 2023 09:33:59 -0500
X-BeenThere: debbugs-submit@debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
X-BeenThere: bug-guile@gnu.org
List-Id: "Bug reports for GUILE,
 GNU's Ubiquitous Extension Language" <bug-guile.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/bug-guile>,
 <mailto:bug-guile-request@gnu.org?subject=unsubscribe>
List-Archive: <https://lists.gnu.org/archive/html/bug-guile>
List-Post: <mailto:bug-guile@gnu.org>
List-Help: <mailto:bug-guile-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/bug-guile>,
 <mailto:bug-guile-request@gnu.org?subject=subscribe>
Errors-To: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org
Original-Sender: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org
Xref: news.gmane.io gmane.lisp.guile.bugs:10545
Archived-At: <http://permalink.gmane.org/gmane.lisp.guile.bugs/10545>


--Apple-Mail=_7FD80609-3E3C-4B38-A565-5728DCC1029B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

I'm building Guile 3.0.9 on iOS 14.5.1 (Darwin 20.4.0) which is =
suffering some system API differences within pthread_jit* stuff.


cat alist.doc array-handle.doc array-map.doc arrays.doc async.doc =
atomic.doc backtrace.doc boolean.doc bitvectors.doc bytevectors.doc =
chars.doc control.doc continuations.doc debug.doc deprecated.doc =
deprecation.doc dynl.doc dynwind.doc eq.doc error.doc eval.doc =
evalext.doc exceptions.doc expand.doc extensions.doc fdes-finalizers.doc =
feature.doc filesys.doc fluids.doc foreign.doc fports.doc gc-malloc.doc =
gc.doc gettext.doc generalized-vectors.doc goops.doc gsubr.doc =
guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc init.doc ioext.doc =
keywords.doc list.doc load.doc macros.doc mallocs.doc memoize.doc =
modules.doc numbers.doc objprop.doc options.doc pairs.doc ports.doc =
print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc random.doc =
rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc smob.doc =
sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc =
srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc =
strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc =
unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc =
vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc =
posix.doc net_db.doc socket.doc regex-posix.doc | GUILE_AUTO_COMPILE=3D0 =
../meta/build-env guild snarf-check-and-output-texi          > =
guile-procedures.texi || { rm guile-procedures.texi; false; }
cat: write error: Broken pipe

After inspecting the crash logs, it might be some unexpected freeing of =
mmap cache, the librecompat.0.dylib includes a sys_icache_invalidate =
wrapper for __clear_cache function (as the same implementation with =
macOS one since they are all come from LLVM) which has not been compiled =
to system libcompiler_rt.dylib for iOS.

Exception Type:  EXC_BAD_ACCESS (SIGBUS)
Exception Subtype: KERN_PROTECTION_FAILURE at 0x0000000104ccb9d8
VM Region Info: 0x104ccb9d8 is in 0x104cc4000-0x104ccc000;  bytes after =
start: 31192  bytes before end: 1575
      REGION TYPE                 START - END      [ VSIZE] PRT/MAX =
SHRMOD  REGION DETAIL
      __LINKEDIT               104bc0000-104cc4000 [ 1040K] r--/rw- =
SM=3DCOW  ...e-3.0.1.dylib
--->  __TEXT                   104cc4000-104ccc000 [   32K] r--/rw- =
SM=3DCOW  ...ompat.0.dylib
      __DATA_CONST             104ccc000-104cd0000 [   16K] rw-/rw- =
SM=3DCOW  ...ompat.0.dylib

Termination Signal: Bus error: 10
Termination Reason: Namespace SIGNAL, Code 0xa
Terminating Process: exc handler [9957]
Triggered by Thread:  0

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0     + librecompat.0.dylib           	0x104ccb9d8 0x104cc4000 + 0x79d8	=
// __clear_cache + 0x0
1       libguile-3.0.1.dylib          	0x104b4b728 0x104a44000 + =
0x107728	// =
/buildroot/guile-3.0.9/libguile/./lightening/lightening/aarch64.c:217
2       libguile-3.0.1.dylib          	0x104b4b3b0 0x104a44000 + =
0x1073b0	// =
/buildroot/guile-3.0.9/libguile/./lightening/lightening/lightening.c:241
3       libguile-3.0.1.dylib          	0x104aa12e0 0x104a44000 + =
0x5d2e0	// /buildroot/guile-3.0.9/libguile/jit.c:1424
4       libguile-3.0.1.dylib          	0x104aa0c38 0x104a44000 + =
0x5cc38	// /buildroot/guile-3.0.9/libguile/jit.c:5895
5       libsystem_pthread.dylib       	0x1def01f60 0x1deeff000 + 0x2f60	=
// __pthread_once_handler + 0x50
6       libsystem_platform.dylib      	0x1deefb964 0x1deef7000 + 0x4964	=
// _os_once_callout + 0x20
7       libsystem_pthread.dylib       	0x1def01ef4 0x1deeff000 + 0x2ef4	=
// pthread_once + 0x64
8       libguile-3.0.1.dylib          	0x104aa0558 0x104a44000 + =
0x5c558	// /buildroot/guile-3.0.9/libguile/jit.c:0
9       libguile-3.0.1.dylib          	0x104aa0410 0x104a44000 + =
0x5c410	// /buildroot/guile-3.0.9/libguile/jit.c:6031
10      libguile-3.0.1.dylib          	0x104b35774 0x104a44000 + =
0xf1774	// /buildroot/guile-3.0.9/libguile/./vm-engine.c:370
11      libguile-3.0.1.dylib          	0x104b323d0 0x104a44000 + =
0xee3d0	// /buildroot/guile-3.0.9/libguile/vm.c:1615
12      libguile-3.0.1.dylib          	0x104a6776c 0x104a44000 + =
0x2376c	// /buildroot/guile-3.0.9/libguile/eval.c:496
13      libguile-3.0.1.dylib          	0x104af9e20 0x104a44000 + =
0xb5e20	// /buildroot/guile-3.0.9/libguile/read.c:1553
14      libguile-3.0.1.dylib          	0x104abb158 0x104a44000 + =
0x77158	// /buildroot/guile-3.0.9/libguile/load.c:124
15      libguile-3.0.1.dylib          	0x104abc3d4 0x104a44000 + =
0x783d4	// /buildroot/guile-3.0.9/libguile/load.c:1267
16      libguile-3.0.1.dylib          	0x104abcb28 0x104a44000 + =
0x78b28	// /buildroot/guile-3.0.9/libguile/load.c:1275
17      libguile-3.0.1.dylib          	0x104a8e0ec 0x104a44000 + =
0x4a0ec	// /buildroot/guile-3.0.9/libguile/init.c:230
18      libguile-3.0.1.dylib          	0x104a8e458 0x104a44000 + =
0x4a458	// /buildroot/guile-3.0.9/libguile/init.c:510
19      libguile-3.0.1.dylib          	0x104b2a89c 0x104a44000 + =
0xe689c	// /buildroot/guile-3.0.9/libguile/threads.c:578
20      libguile-3.0.1.dylib          	0x104b2d47c 0x104a44000 + =
0xe947c	// /buildroot/guile-3.0.9/libguile/threads.c:642
21    + libgc.1.dylib                 	0x1053688e4 0x105354000 + =
0x148e4	// /buildroot/gc-8.2.2/misc.c:2173
22      libguile-3.0.1.dylib          	0x104b2a97c 0x104a44000 + =
0xe697c	// /buildroot/guile-3.0.9/libguile/threads.c:692
23      libguile-3.0.1.dylib          	0x104b2a930 0x104a44000 + =
0xe6930	// /buildroot/guile-3.0.9/libguile/threads.c:698
24      libguile-3.0.1.dylib          	0x104a8e190 0x104a44000 + =
0x4a190	// /buildroot/guile-3.0.9/libguile/init.c:295
25      guile (*)                     	0x104a37d90 0x104a30000 + 0x7d90	=
// /buildroot/guile-3.0.9/libguile/guile.c:94
26      libdyld.dylib                 	0x193c92cf8 0x193c91000 + 0x1cf8	=
// start + 0x4

But according to the JIT allocation problems which has been met by other =
projects:
=
https://patchwork.kernel.org/project/qemu-devel/patch/20201108232425.1705-=
7-j@getutm.app/#23841029 =
<https://patchwork.kernel.org/project/qemu-devel/patch/20201108232425.1705=
-7-j@getutm.app/#23841029>

I attempted to patch libguile/jit.c like that:
--- libguile/jit.c      1676274604.835054871
+++ libguile/jit.c.old  1676204975.415324076
@@ -47,8 +47,17 @@
 #include <sys/mman.h>
 #endif
=20
-#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#if defined __APPLE__
 #include <libkern/OSCacheControl.h>
+#include "tcg-apple-jit.h"
+
+static void tb_exec_change(bool locked)
+{
+     if (jit_write_protect_supported()) {
+         jit_write_protect(locked);
+     }
+}
+
 #endif
=20
 #include "jit.h"
@@ -1414,8 +1435,9 @@
=20
       uint8_t *ret =3D jit_address (j->jit);
=20
-#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
-      pthread_jit_write_protect_np(0);
+#if defined __APPLE__
+//      pthread_jit_write_protect_np(0);
+      tb_exec_change(0);
 #endif
=20
       emit (j);
@@ -1423,10 +1445,11 @@
       size_t size;
       if (!jit_has_overflow (j->jit) && jit_end (j->jit, &size))
         {
-#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#if defined __APPLE__
           /* protect previous code arena. leave unprotected after =
emit()
              since jit_end() also writes to code arena. */
-          pthread_jit_write_protect_np(1);
+//          pthread_jit_write_protect_np(1);
+          tb_exec_change(1);
           sys_icache_invalidate(arena->base, arena->size);
 #endif
           ASSERT (size <=3D (arena->size - arena->used));
@@ -1442,9 +1465,10 @@
         }
       else
         {
-#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#if defined __APPLE__
           /* protect previous code arena */
-          pthread_jit_write_protect_np(1);
+//          pthread_jit_write_protect_np(1);
+          tb_exec_change(1);
           sys_icache_invalidate(arena->base, arena->size);
 #endif
           jit_reset (j->jit);

Which replaces the previous `pthread_jit_write_protect_np` with another =
iOS capable implementation that provided in the link above:
/*
 * Apple Silicon functions for JIT handling
 *
 * Copyright (c) 2020 osy
 *
 * This library is free software; you can redistribute it and/or
 * modify it under the terms of the GNU Lesser General Public
 * License as published by the Free Software Foundation; either
 * version 2.1 of the License, or (at your option) any later version.
 *
 * This library is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 * Lesser General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public
 * License along with this library; if not, see =
<http://www.gnu.org/licenses/>.
 */

#ifndef TCG_APPLE_JIT_H
#define TCG_APPLE_JIT_H

/*
 * APRR handling
 * Credits to: https://siguza.github.io/APRR/
 * Reversed from /usr/lib/system/libsystem_pthread.dylib
 */

#if defined(__aarch64__) && defined(__APPLE__)

#define _COMM_PAGE_START_ADDRESS        (0x0000000FFFFFC000ULL) /* In =
TTBR0 */
#define _COMM_PAGE_APRR_SUPPORT         (_COMM_PAGE_START_ADDRESS + =
0x10C)
#define _COMM_PAGE_APPR_WRITE_ENABLE    (_COMM_PAGE_START_ADDRESS + =
0x110)
#define _COMM_PAGE_APRR_WRITE_DISABLE   (_COMM_PAGE_START_ADDRESS + =
0x118)

static __attribute__((__always_inline__)) bool =
jit_write_protect_supported(void)
{
    /* Access shared kernel page at fixed memory location. */
    uint8_t aprr_support =3D *(volatile uint8_t =
*)_COMM_PAGE_APRR_SUPPORT;
    return aprr_support > 0;
}

/* write protect enable =3D write disable */
static __attribute__((__always_inline__)) void jit_write_protect(int =
enabled)
{
    /* Access shared kernel page at fixed memory location. */
    uint8_t aprr_support =3D *(volatile uint8_t =
*)_COMM_PAGE_APRR_SUPPORT;
    if (aprr_support =3D=3D 0 || aprr_support > 3) {
        return;
    } else if (aprr_support =3D=3D 1) {
        __asm__ __volatile__ (
            "mov x0, %0\n"
            "ldr x0, [x0]\n"
            "msr S3_4_c15_c2_7, x0\n"
            "isb sy\n"
            :: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE
                            : _COMM_PAGE_APPR_WRITE_ENABLE)
            : "memory", "x0"
        );
    } else {
        __asm__ __volatile__ (
            "mov x0, %0\n"
            "ldr x0, [x0]\n"
            "msr S3_6_c15_c1_5, x0\n"
            "isb sy\n"
            :: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE
                            : _COMM_PAGE_APPR_WRITE_ENABLE)
            : "memory", "x0"
        );
    }
}

#else /* defined(__aarch64__) && defined(__APPLE__) */

static __attribute__((__always_inline__)) bool =
jit_write_protect_supported(void)
{
    return false;
}

static __attribute__((__always_inline__)) void jit_write_protect(int =
enabled)
{
}

#endif

#endif /* define TCG_APPLE_JIT_H */

Then we have a successful jit allocation in the first process, but while =
it comes to stage 0, every allocation fails like that:
make[3]: Entering directory '/buildroot/guile-3.0.9/libguile'
cat alist.doc array-handle.doc array-map.doc arrays.doc async.doc =
atomic.doc backtrace.doc boolean.doc bitvectors.doc bytevectors.doc =
chars.doc control.doc continuations.doc debug.doc deprecated.doc =
deprecation.doc dynl.doc dynwind.doc eq.doc error.doc eval.doc =
evalext.doc exceptions.doc expand.doc extensions.doc fdes-finalizers.doc =
feature.doc filesys.doc fluids.doc foreign.doc fports.doc gc-malloc.doc =
gc.doc gettext.doc generalized-vectors.doc goops.doc gsubr.doc =
guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc init.doc ioext.doc =
keywords.doc list.doc load.doc macros.doc mallocs.doc memoize.doc =
modules.doc numbers.doc objprop.doc options.doc pairs.doc ports.doc =
print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc random.doc =
rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc smob.doc =
sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc =
srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc =
strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc =
unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc =
vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc =
posix.doc net_db.doc socket.doc regex-posix.doc | GUILE_AUTO_COMPILE=3D0 =
../meta/build-env guild snarf-check-and-output-texi          > =
guile-procedures.texi || { rm guile-procedures.texi; false; }
make[3]: Leaving directory '/buildroot/guile-3.0.9/libguile'
make[2]: Leaving directory '/buildroot/guile-3.0.9/libguile'
Making all in module
make[2]: Entering directory '/buildroot/guile-3.0.9/module'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/buildroot/guile-3.0.9/module'
Making all in stage0
make[2]: Entering directory '/buildroot/guile-3.0.9/stage0'
GUILE_BOOTSTRAP_STAGE=3Dstage0 ../meta/build-env guild compile =
--target=3D"aarch64-apple-darwin20.4.0" -W0 -O1 -L =
"/buildroot/guile-3.0.9/module" -o "ice-9/eval.go" =
"../module/ice-9/eval.scm"
allocating JIT code buffer failed: Invalid argument
JIT failed due to resource exhaustion
disabling automatic JIT compilation
The issue #47429 mentions a similar issue related with JIT allocation =
under Darwin20+, but alongside to the implementation of =
`allocate_code_arena` method in libguile/jit.c in 3.0.5 we don't have =
any processes regarding to the Darwin20+ JIT write protections, the =
culprit might different.
Now what I can confirm is that
- MAP_JIT is working on both macOS and iOS
- If not providing an alternative method of pthread_write_protect_np, we =
then should properly handle the flags that passing to mmaped pointers to =
make sure RWX does not applied same time when doing W/X operations.
- Current Guile does not have a working JIT on arm64 iOS if there's no =
HAVE_PTHREAD_JIT_WRITE_PROTECT_NP defined=

--Apple-Mail=_7FD80609-3E3C-4B38-A565-5728DCC1029B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=us-ascii

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;" class=3D"">I'm =
building Guile 3.0.9 on iOS 14.5.1 (Darwin 20.4.0) which is suffering =
some system API differences within pthread_jit* stuff.<div class=3D""><br =
class=3D""></div><div class=3D""><br class=3D""></div><div class=3D""><div=
 class=3D"">cat alist.doc array-handle.doc array-map.doc arrays.doc =
async.doc atomic.doc backtrace.doc boolean.doc bitvectors.doc =
bytevectors.doc chars.doc control.doc continuations.doc debug.doc =
deprecated.doc deprecation.doc dynl.doc dynwind.doc eq.doc error.doc =
eval.doc evalext.doc exceptions.doc expand.doc extensions.doc =
fdes-finalizers.doc feature.doc filesys.doc fluids.doc foreign.doc =
fports.doc gc-malloc.doc gc.doc gettext.doc generalized-vectors.doc =
goops.doc gsubr.doc guardians.doc hash.doc hashtab.doc hooks.doc =
i18n.doc init.doc ioext.doc keywords.doc list.doc load.doc macros.doc =
mallocs.doc memoize.doc modules.doc numbers.doc objprop.doc options.doc =
pairs.doc ports.doc print.doc procprop.doc procs.doc promises.doc =
r6rs-ports.doc random.doc rdelim.doc read.doc rw.doc scmsigs.doc =
script.doc simpos.doc smob.doc sort.doc srcprop.doc srfi-1.doc =
srfi-4.doc srfi-13.doc srfi-14.doc srfi-60.doc stackchk.doc stacks.doc =
stime.doc strings.doc strorder.doc strports.doc struct.doc symbols.doc =
syntax.doc threads.doc throw.doc unicode.doc uniform.doc values.doc =
variable.doc vectors.doc version.doc vports.doc weak-set.doc =
weak-table.doc weak-vector.doc dynl.doc posix.doc net_db.doc socket.doc =
regex-posix.doc | GUILE_AUTO_COMPILE=3D0 ../meta/build-env guild =
snarf-check-and-output-texi &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;&gt; =
guile-procedures.texi || { rm guile-procedures.texi; false; }</div><div =
class=3D"">cat: write error: Broken pipe</div></div><div class=3D""><br =
class=3D""></div><div class=3D"">After inspecting the crash logs, it =
might be some unexpected freeing of mmap cache, the librecompat.0.dylib =
includes a sys_icache_invalidate wrapper for __clear_cache function (as =
the same implementation with macOS one since they are all come from =
LLVM) which has not been compiled to system libcompiler_rt.dylib for =
iOS.</div><div class=3D""><br class=3D""></div><div class=3D""><pre =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: =
8pt;" class=3D"">Exception Type:  EXC_BAD_ACCESS (SIGBUS)
Exception Subtype: KERN_PROTECTION_FAILURE at 0x0000000104ccb9d8
VM Region Info: 0x104ccb9d8 is in 0x104cc4000-0x104ccc000;  bytes after =
start: 31192  bytes before end: 1575
      REGION TYPE                 START - END      [ VSIZE] PRT/MAX =
SHRMOD  REGION DETAIL
      __LINKEDIT               104bc0000-104cc4000 [ 1040K] r--/rw- =
SM=3DCOW  ...e-3.0.1.dylib
---&gt;  __TEXT                   104cc4000-104ccc000 [   32K] r--/rw- =
SM=3DCOW  ...ompat.0.dylib
      __DATA_CONST             104ccc000-104cd0000 [   16K] rw-/rw- =
SM=3DCOW  ...ompat.0.dylib

Termination Signal: Bus error: 10
Termination Reason: Namespace SIGNAL, Code 0xa
Terminating Process: exc handler [9957]
Triggered by Thread:  0

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0     + librecompat.0.dylib           	0x104ccb9d8 0x104cc4000 + 0x79d8	=
// __clear_cache + 0x0
1       libguile-3.0.1.dylib          	0x104b4b728 0x104a44000 + =
0x107728	// =
/buildroot/guile-3.0.9/libguile/./lightening/lightening/aarch64.c:217
2       libguile-3.0.1.dylib          	0x104b4b3b0 0x104a44000 + =
0x1073b0	// =
/buildroot/guile-3.0.9/libguile/./lightening/lightening/lightening.c:241
3       libguile-3.0.1.dylib          	0x104aa12e0 0x104a44000 + =
0x5d2e0	// /buildroot/guile-3.0.9/libguile/jit.c:1424
4       libguile-3.0.1.dylib          	0x104aa0c38 0x104a44000 + =
0x5cc38	// /buildroot/guile-3.0.9/libguile/jit.c:5895
5       libsystem_pthread.dylib       	0x1def01f60 0x1deeff000 + 0x2f60	=
// __pthread_once_handler + 0x50
6       libsystem_platform.dylib      	0x1deefb964 0x1deef7000 + 0x4964	=
// _os_once_callout + 0x20
7       libsystem_pthread.dylib       	0x1def01ef4 0x1deeff000 + 0x2ef4	=
// pthread_once + 0x64
8       libguile-3.0.1.dylib          	0x104aa0558 0x104a44000 + =
0x5c558	// /buildroot/guile-3.0.9/libguile/jit.c:0
9       libguile-3.0.1.dylib          	0x104aa0410 0x104a44000 + =
0x5c410	// /buildroot/guile-3.0.9/libguile/jit.c:6031
10      libguile-3.0.1.dylib          	0x104b35774 0x104a44000 + =
0xf1774	// /buildroot/guile-3.0.9/libguile/./vm-engine.c:370
11      libguile-3.0.1.dylib          	0x104b323d0 0x104a44000 + =
0xee3d0	// /buildroot/guile-3.0.9/libguile/vm.c:1615
12      libguile-3.0.1.dylib          	0x104a6776c 0x104a44000 + =
0x2376c	// /buildroot/guile-3.0.9/libguile/eval.c:496
13      libguile-3.0.1.dylib          	0x104af9e20 0x104a44000 + =
0xb5e20	// /buildroot/guile-3.0.9/libguile/read.c:1553
14      libguile-3.0.1.dylib          	0x104abb158 0x104a44000 + =
0x77158	// /buildroot/guile-3.0.9/libguile/load.c:124
15      libguile-3.0.1.dylib          	0x104abc3d4 0x104a44000 + =
0x783d4	// /buildroot/guile-3.0.9/libguile/load.c:1267
16      libguile-3.0.1.dylib          	0x104abcb28 0x104a44000 + =
0x78b28	// /buildroot/guile-3.0.9/libguile/load.c:1275
17      libguile-3.0.1.dylib          	0x104a8e0ec 0x104a44000 + =
0x4a0ec	// /buildroot/guile-3.0.9/libguile/init.c:230
18      libguile-3.0.1.dylib          	0x104a8e458 0x104a44000 + =
0x4a458	// /buildroot/guile-3.0.9/libguile/init.c:510
19      libguile-3.0.1.dylib          	0x104b2a89c 0x104a44000 + =
0xe689c	// /buildroot/guile-3.0.9/libguile/threads.c:578
20      libguile-3.0.1.dylib          	0x104b2d47c 0x104a44000 + =
0xe947c	// /buildroot/guile-3.0.9/libguile/threads.c:642
21    + libgc.1.dylib                 	0x1053688e4 0x105354000 + =
0x148e4	// /buildroot/gc-8.2.2/misc.c:2173
22      libguile-3.0.1.dylib          	0x104b2a97c 0x104a44000 + =
0xe697c	// /buildroot/guile-3.0.9/libguile/threads.c:692
23      libguile-3.0.1.dylib          	0x104b2a930 0x104a44000 + =
0xe6930	// /buildroot/guile-3.0.9/libguile/threads.c:698
24      libguile-3.0.1.dylib          	0x104a8e190 0x104a44000 + =
0x4a190	// /buildroot/guile-3.0.9/libguile/init.c:295
25      guile (*)                     	0x104a37d90 0x104a30000 + 0x7d90	=
// /buildroot/guile-3.0.9/libguile/guile.c:94
26      libdyld.dylib                 	0x193c92cf8 0x193c91000 + 0x1cf8	=
// start + 0x4</pre><pre style=3D"caret-color: rgb(0, 0, 0); color: =
rgb(0, 0, 0); font-size: 8pt;" class=3D""><br class=3D""></pre><pre =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: =
8pt;" class=3D"">But according to the JIT allocation problems which has =
been met by other projects:</pre><pre class=3D""><font color=3D"#000000" =
size=3D"2" class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" =
class=3D""><a =
href=3D"https://patchwork.kernel.org/project/qemu-devel/patch/202011082324=
25.1705-7-j@getutm.app/#23841029" =
class=3D"">https://patchwork.kernel.org/project/qemu-devel/patch/202011082=
32425.1705-7-j@getutm.app/#23841029</a></span></font></pre><pre =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: =
8pt;" class=3D""><br class=3D""></pre><pre style=3D"caret-color: rgb(0, =
0, 0); color: rgb(0, 0, 0); font-size: 8pt;" class=3D"">I attempted to =
patch libguile/jit.c like that:</pre><pre class=3D""><font =
color=3D"#000000" size=3D"2" class=3D""><span style=3D"caret-color: =
rgb(0, 0, 0);" class=3D"">--- libguile/jit.c      1676274604.835054871
+++ libguile/jit.c.old  1676204975.415324076
@@ -47,8 +47,17 @@
 #include &lt;sys/mman.h&gt;
 #endif
=20
-#if defined __APPLE__ &amp;&amp; HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#if defined __APPLE__
 #include &lt;libkern/OSCacheControl.h&gt;
+#include "tcg-apple-jit.h"
+
+static void tb_exec_change(bool locked)
+{
+     if (jit_write_protect_supported()) {
+         jit_write_protect(locked);
+     }
+}
+
 #endif
=20
 #include "jit.h"
@@ -1414,8 +1435,9 @@
=20
       uint8_t *ret =3D jit_address (j-&gt;jit);
=20
-#if defined __APPLE__ &amp;&amp; HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
-      pthread_jit_write_protect_np(0);
+#if defined __APPLE__
+//      pthread_jit_write_protect_np(0);
+      tb_exec_change(0);
 #endif
=20
       emit (j);
@@ -1423,10 +1445,11 @@
       size_t size;
       if (!jit_has_overflow (j-&gt;jit) &amp;&amp; jit_end (j-&gt;jit, =
&amp;size))
         {
-#if defined __APPLE__ &amp;&amp; HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#if defined __APPLE__
           /* protect previous code arena. leave unprotected after =
emit()
              since jit_end() also writes to code arena. */
-          pthread_jit_write_protect_np(1);
+//          pthread_jit_write_protect_np(1);
+          tb_exec_change(1);
           sys_icache_invalidate(arena-&gt;base, arena-&gt;size);
 #endif
           ASSERT (size &lt;=3D (arena-&gt;size - arena-&gt;used));
@@ -1442,9 +1465,10 @@
         }
       else
         {
-#if defined __APPLE__ &amp;&amp; HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#if defined __APPLE__
           /* protect previous code arena */
-          pthread_jit_write_protect_np(1);
+//          pthread_jit_write_protect_np(1);
+          tb_exec_change(1);
           sys_icache_invalidate(arena-&gt;base, arena-&gt;size);
 #endif
           jit_reset (j-&gt;jit);</span></font></pre><pre =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: =
8pt;" class=3D""><br class=3D""></pre><pre style=3D"caret-color: rgb(0, =
0, 0); color: rgb(0, 0, 0); font-size: 8pt;" class=3D"">Which replaces =
the previous `<span style=3D"caret-color: rgb(255, 255, 255); font-size: =
small;" class=3D"">pthread_jit_write_protect_np` with another iOS =
capable implementation that provided in the link above:</span></pre><pre =
class=3D""><font color=3D"#000000" size=3D"2" class=3D""><span =
style=3D"caret-color: rgb(0, 0, 0);" class=3D"">/*
 * Apple Silicon functions for JIT handling
 *
 * Copyright (c) 2020 osy
 *
 * This library is free software; you can redistribute it and/or
 * modify it under the terms of the GNU Lesser General Public
 * License as published by the Free Software Foundation; either
 * version 2.1 of the License, or (at your option) any later version.
 *
 * This library is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 * Lesser General Public License for more details.
 *
 * You should have received a copy of the GNU Lesser General Public
 * License along with this library; if not, see &lt;<a =
href=3D"http://www.gnu.org/licenses/" =
class=3D"">http://www.gnu.org/licenses/</a>&gt;.
 */

#ifndef TCG_APPLE_JIT_H
#define TCG_APPLE_JIT_H

/*
 * APRR handling
 * Credits to: <a href=3D"https://siguza.github.io/APRR/" =
class=3D"">https://siguza.github.io/APRR/</a>
 * Reversed from /usr/lib/system/libsystem_pthread.dylib
 */

#if defined(__aarch64__) &amp;&amp; defined(__APPLE__)

#define _COMM_PAGE_START_ADDRESS        (0x0000000FFFFFC000ULL) /* In =
TTBR0 */
#define _COMM_PAGE_APRR_SUPPORT         (_COMM_PAGE_START_ADDRESS + =
0x10C)
#define _COMM_PAGE_APPR_WRITE_ENABLE    (_COMM_PAGE_START_ADDRESS + =
0x110)
#define _COMM_PAGE_APRR_WRITE_DISABLE   (_COMM_PAGE_START_ADDRESS + =
0x118)

static __attribute__((__always_inline__)) bool =
jit_write_protect_supported(void)
{
    /* Access shared kernel page at fixed memory location. */
    uint8_t aprr_support =3D *(volatile uint8_t =
*)_COMM_PAGE_APRR_SUPPORT;
    return aprr_support &gt; 0;
}

/* write protect enable =3D write disable */
static __attribute__((__always_inline__)) void jit_write_protect(int =
enabled)
{
    /* Access shared kernel page at fixed memory location. */
    uint8_t aprr_support =3D *(volatile uint8_t =
*)_COMM_PAGE_APRR_SUPPORT;
    if (aprr_support =3D=3D 0 || aprr_support &gt; 3) {
        return;
    } else if (aprr_support =3D=3D 1) {
        __asm__ __volatile__ (
            "mov x0, %0\n"
            "ldr x0, [x0]\n"
            "msr S3_4_c15_c2_7, x0\n"
            "isb sy\n"
            :: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE
                            : _COMM_PAGE_APPR_WRITE_ENABLE)
            : "memory", "x0"
        );
    } else {
        __asm__ __volatile__ (
            "mov x0, %0\n"
            "ldr x0, [x0]\n"
            "msr S3_6_c15_c1_5, x0\n"
            "isb sy\n"
            :: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE
                            : _COMM_PAGE_APPR_WRITE_ENABLE)
            : "memory", "x0"
        );
    }
}

#else /* defined(__aarch64__) &amp;&amp; defined(__APPLE__) */

static __attribute__((__always_inline__)) bool =
jit_write_protect_supported(void)
{
    return false;
}

static __attribute__((__always_inline__)) void jit_write_protect(int =
enabled)
{
}

#endif

#endif /* define TCG_APPLE_JIT_H */</span></font></pre><pre class=3D""><br=
 class=3D""></pre><pre class=3D"">Then we have a successful jit =
allocation in the first process, but while it comes to stage 0, every =
allocation fails like that:</pre><pre class=3D"">make[3]: Entering =
directory '/buildroot/guile-3.0.9/libguile'
cat alist.doc array-handle.doc array-map.doc arrays.doc async.doc =
atomic.doc backtrace.doc boolean.doc bitvectors.doc bytevectors.doc =
chars.doc control.doc continuations.doc debug.doc deprecated.doc =
deprecation.doc dynl.doc dynwind.doc eq.doc error.doc eval.doc =
evalext.doc exceptions.doc expand.doc extensions.doc fdes-finalizers.doc =
feature.doc filesys.doc fluids.doc foreign.doc fports.doc gc-malloc.doc =
gc.doc gettext.doc generalized-vectors.doc goops.doc gsubr.doc =
guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc init.doc ioext.doc =
keywords.doc list.doc load.doc macros.doc mallocs.doc memoize.doc =
modules.doc numbers.doc objprop.doc options.doc pairs.doc ports.doc =
print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc random.doc =
rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc smob.doc =
sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc =
srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc =
strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc =
unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc =
vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc =
posix.doc net_db.doc socket.doc regex-posix.doc | GUILE_AUTO_COMPILE=3D0 =
../meta/build-env guild snarf-check-and-output-texi          &gt; =
guile-procedures.texi || { rm guile-procedures.texi; false; }
make[3]: Leaving directory '/buildroot/guile-3.0.9/libguile'
make[2]: Leaving directory '/buildroot/guile-3.0.9/libguile'
Making all in module
make[2]: Entering directory '/buildroot/guile-3.0.9/module'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/buildroot/guile-3.0.9/module'
Making all in stage0
make[2]: Entering directory '/buildroot/guile-3.0.9/stage0'
GUILE_BOOTSTRAP_STAGE=3Dstage0 ../meta/build-env guild compile =
--target=3D"aarch64-apple-darwin20.4.0" -W0 -O1 -L =
"/buildroot/guile-3.0.9/module" -o "ice-9/eval.go" =
"../module/ice-9/eval.scm"
allocating JIT code buffer failed: Invalid argument
JIT failed due to resource exhaustion
disabling automatic JIT compilation</pre><pre class=3D"">The issue =
#47429 mentions a similar issue related with JIT allocation under =
Darwin20+, but alongside to the implementation of `<font color=3D"#000000"=
 size=3D"3" class=3D"">allocate_code_arena` method in libguile/jit.c in =
3.0.5 we don't have any processes regarding to the Darwin20+ JIT write =
protections, the culprit might different.</font></pre><pre =
class=3D""><font color=3D"#000000" size=3D"3" class=3D""><span =
style=3D"caret-color: rgb(0, 0, 0);" class=3D"">Now what I can confirm =
is that</span></font></pre><pre class=3D""><font color=3D"#000000" =
size=3D"3" class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" =
class=3D"">- MAP_JIT is working on both macOS and =
iOS</span></font></pre><pre class=3D""><font color=3D"#000000" size=3D"3" =
class=3D"">- If not providing an alternative method of =
pthread_write_protect_np, we then should properly handle the flags that =
passing to mmaped pointers to make sure RWX does not applied same time =
when doing W/X operations.</font></pre><pre class=3D""><font =
color=3D"#000000" size=3D"3" class=3D"">- Current Guile does not have a =
working JIT on arm64 iOS if there's no HAVE_PTHREAD_JIT_WRITE_PROTECT_NP =
defined</font></pre></div></body></html>=

--Apple-Mail=_7FD80609-3E3C-4B38-A565-5728DCC1029B--