From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Torrekie <me@torrekie.dev> Newsgroups: gmane.lisp.guile.bugs Subject: bug#61476: mmap() with RWX cannot correctly invalidate cache Date: Mon, 13 Feb 2023 17:09:54 +0800 Message-ID: <5327BA39-E711-4178-8E7F-37645137BF51@torrekie.dev> Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\)) Content-Type: multipart/alternative; boundary="Apple-Mail=_7FD80609-3E3C-4B38-A565-5728DCC1029B" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40401"; mail-complaints-to="usenet@ciao.gmane.io" To: 61476@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Mon Feb 13 15:35:20 2023 Return-path: <bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org> Envelope-to: guile-bugs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org>) id 1pRZvK-000AFW-AD for guile-bugs@m.gmane-mx.org; Mon, 13 Feb 2023 15:35:18 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <bug-guile-bounces@gnu.org>) id 1pRZv8-00028R-5B; Mon, 13 Feb 2023 09:35:06 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1pRZv5-000284-Kz for bug-guile@gnu.org; Mon, 13 Feb 2023 09:35:03 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1pRZv5-0004ES-5B for bug-guile@gnu.org; Mon, 13 Feb 2023 09:35:03 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1pRZv4-0005Z4-LK for bug-guile@gnu.org; Mon, 13 Feb 2023 09:35:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Torrekie <me@torrekie.dev> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org> Resent-CC: bug-guile@gnu.org Resent-Date: Mon, 13 Feb 2023 14:35:02 +0000 Resent-Message-ID: <handler.61476.B.167629884321290@debbugs.gnu.org> Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 61476 X-GNU-PR-Package: guile X-Debbugs-Original-To: bug-guile@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.167629884321290 (code B ref -1); Mon, 13 Feb 2023 14:35:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 13 Feb 2023 14:34:03 +0000 Original-Received: from localhost ([127.0.0.1]:48053 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces@debbugs.gnu.org>) id 1pRZu4-0005X9-2Q for submit@debbugs.gnu.org; Mon, 13 Feb 2023 09:34:03 -0500 Original-Received: from lists.gnu.org ([209.51.188.17]:45830) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <me@torrekie.dev>) id 1pRUqm-0000Kb-Dk for submit@debbugs.gnu.org; Mon, 13 Feb 2023 04:10:18 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <me@torrekie.dev>) id 1pRUql-0007ME-Ts for bug-guile@gnu.org; Mon, 13 Feb 2023 04:10:16 -0500 Original-Received: from mta-15-4.privateemail.com ([198.54.127.111]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <me@torrekie.dev>) id 1pRUqi-0007CK-6o for bug-guile@gnu.org; Mon, 13 Feb 2023 04:10:15 -0500 Original-Received: from mta-15.privateemail.com (localhost [127.0.0.1]) by mta-15.privateemail.com (Postfix) with ESMTP id 45DA018000A7 for <bug-guile@gnu.org>; Mon, 13 Feb 2023 04:09:59 -0500 (EST) Original-Received: from [127.0.0.1] (unknown [104.245.98.35]) by mta-15.privateemail.com (Postfix) with ESMTPA id E3EA518000A2 for <bug-guile@gnu.org>; Mon, 13 Feb 2023 04:09:57 -0500 (EST) X-Mailer: Apple Mail (2.3654.60.0.2.21) X-Virus-Scanned: ClamAV using ClamSMTP Received-SPF: pass client-ip=198.54.127.111; envelope-from=me@torrekie.dev; helo=MTA-15-4.privateemail.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 13 Feb 2023 09:33:59 -0500 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" <bug-guile.gnu.org> List-Unsubscribe: <https://lists.gnu.org/mailman/options/bug-guile>, <mailto:bug-guile-request@gnu.org?subject=unsubscribe> List-Archive: <https://lists.gnu.org/archive/html/bug-guile> List-Post: <mailto:bug-guile@gnu.org> List-Help: <mailto:bug-guile-request@gnu.org?subject=help> List-Subscribe: <https://lists.gnu.org/mailman/listinfo/bug-guile>, <mailto:bug-guile-request@gnu.org?subject=subscribe> Errors-To: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.lisp.guile.bugs:10545 Archived-At: <http://permalink.gmane.org/gmane.lisp.guile.bugs/10545> --Apple-Mail=_7FD80609-3E3C-4B38-A565-5728DCC1029B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii I'm building Guile 3.0.9 on iOS 14.5.1 (Darwin 20.4.0) which is = suffering some system API differences within pthread_jit* stuff. cat alist.doc array-handle.doc array-map.doc arrays.doc async.doc = atomic.doc backtrace.doc boolean.doc bitvectors.doc bytevectors.doc = chars.doc control.doc continuations.doc debug.doc deprecated.doc = deprecation.doc dynl.doc dynwind.doc eq.doc error.doc eval.doc = evalext.doc exceptions.doc expand.doc extensions.doc fdes-finalizers.doc = feature.doc filesys.doc fluids.doc foreign.doc fports.doc gc-malloc.doc = gc.doc gettext.doc generalized-vectors.doc goops.doc gsubr.doc = guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc init.doc ioext.doc = keywords.doc list.doc load.doc macros.doc mallocs.doc memoize.doc = modules.doc numbers.doc objprop.doc options.doc pairs.doc ports.doc = print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc random.doc = rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc smob.doc = sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc = srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc = strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc = unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc = vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc = posix.doc net_db.doc socket.doc regex-posix.doc | GUILE_AUTO_COMPILE=3D0 = ../meta/build-env guild snarf-check-and-output-texi > = guile-procedures.texi || { rm guile-procedures.texi; false; } cat: write error: Broken pipe After inspecting the crash logs, it might be some unexpected freeing of = mmap cache, the librecompat.0.dylib includes a sys_icache_invalidate = wrapper for __clear_cache function (as the same implementation with = macOS one since they are all come from LLVM) which has not been compiled = to system libcompiler_rt.dylib for iOS. Exception Type: EXC_BAD_ACCESS (SIGBUS) Exception Subtype: KERN_PROTECTION_FAILURE at 0x0000000104ccb9d8 VM Region Info: 0x104ccb9d8 is in 0x104cc4000-0x104ccc000; bytes after = start: 31192 bytes before end: 1575 REGION TYPE START - END [ VSIZE] PRT/MAX = SHRMOD REGION DETAIL __LINKEDIT 104bc0000-104cc4000 [ 1040K] r--/rw- = SM=3DCOW ...e-3.0.1.dylib ---> __TEXT 104cc4000-104ccc000 [ 32K] r--/rw- = SM=3DCOW ...ompat.0.dylib __DATA_CONST 104ccc000-104cd0000 [ 16K] rw-/rw- = SM=3DCOW ...ompat.0.dylib Termination Signal: Bus error: 10 Termination Reason: Namespace SIGNAL, Code 0xa Terminating Process: exc handler [9957] Triggered by Thread: 0 Thread 0 name: Dispatch queue: com.apple.main-thread Thread 0 Crashed: 0 + librecompat.0.dylib 0x104ccb9d8 0x104cc4000 + 0x79d8 = // __clear_cache + 0x0 1 libguile-3.0.1.dylib 0x104b4b728 0x104a44000 + = 0x107728 // = /buildroot/guile-3.0.9/libguile/./lightening/lightening/aarch64.c:217 2 libguile-3.0.1.dylib 0x104b4b3b0 0x104a44000 + = 0x1073b0 // = /buildroot/guile-3.0.9/libguile/./lightening/lightening/lightening.c:241 3 libguile-3.0.1.dylib 0x104aa12e0 0x104a44000 + = 0x5d2e0 // /buildroot/guile-3.0.9/libguile/jit.c:1424 4 libguile-3.0.1.dylib 0x104aa0c38 0x104a44000 + = 0x5cc38 // /buildroot/guile-3.0.9/libguile/jit.c:5895 5 libsystem_pthread.dylib 0x1def01f60 0x1deeff000 + 0x2f60 = // __pthread_once_handler + 0x50 6 libsystem_platform.dylib 0x1deefb964 0x1deef7000 + 0x4964 = // _os_once_callout + 0x20 7 libsystem_pthread.dylib 0x1def01ef4 0x1deeff000 + 0x2ef4 = // pthread_once + 0x64 8 libguile-3.0.1.dylib 0x104aa0558 0x104a44000 + = 0x5c558 // /buildroot/guile-3.0.9/libguile/jit.c:0 9 libguile-3.0.1.dylib 0x104aa0410 0x104a44000 + = 0x5c410 // /buildroot/guile-3.0.9/libguile/jit.c:6031 10 libguile-3.0.1.dylib 0x104b35774 0x104a44000 + = 0xf1774 // /buildroot/guile-3.0.9/libguile/./vm-engine.c:370 11 libguile-3.0.1.dylib 0x104b323d0 0x104a44000 + = 0xee3d0 // /buildroot/guile-3.0.9/libguile/vm.c:1615 12 libguile-3.0.1.dylib 0x104a6776c 0x104a44000 + = 0x2376c // /buildroot/guile-3.0.9/libguile/eval.c:496 13 libguile-3.0.1.dylib 0x104af9e20 0x104a44000 + = 0xb5e20 // /buildroot/guile-3.0.9/libguile/read.c:1553 14 libguile-3.0.1.dylib 0x104abb158 0x104a44000 + = 0x77158 // /buildroot/guile-3.0.9/libguile/load.c:124 15 libguile-3.0.1.dylib 0x104abc3d4 0x104a44000 + = 0x783d4 // /buildroot/guile-3.0.9/libguile/load.c:1267 16 libguile-3.0.1.dylib 0x104abcb28 0x104a44000 + = 0x78b28 // /buildroot/guile-3.0.9/libguile/load.c:1275 17 libguile-3.0.1.dylib 0x104a8e0ec 0x104a44000 + = 0x4a0ec // /buildroot/guile-3.0.9/libguile/init.c:230 18 libguile-3.0.1.dylib 0x104a8e458 0x104a44000 + = 0x4a458 // /buildroot/guile-3.0.9/libguile/init.c:510 19 libguile-3.0.1.dylib 0x104b2a89c 0x104a44000 + = 0xe689c // /buildroot/guile-3.0.9/libguile/threads.c:578 20 libguile-3.0.1.dylib 0x104b2d47c 0x104a44000 + = 0xe947c // /buildroot/guile-3.0.9/libguile/threads.c:642 21 + libgc.1.dylib 0x1053688e4 0x105354000 + = 0x148e4 // /buildroot/gc-8.2.2/misc.c:2173 22 libguile-3.0.1.dylib 0x104b2a97c 0x104a44000 + = 0xe697c // /buildroot/guile-3.0.9/libguile/threads.c:692 23 libguile-3.0.1.dylib 0x104b2a930 0x104a44000 + = 0xe6930 // /buildroot/guile-3.0.9/libguile/threads.c:698 24 libguile-3.0.1.dylib 0x104a8e190 0x104a44000 + = 0x4a190 // /buildroot/guile-3.0.9/libguile/init.c:295 25 guile (*) 0x104a37d90 0x104a30000 + 0x7d90 = // /buildroot/guile-3.0.9/libguile/guile.c:94 26 libdyld.dylib 0x193c92cf8 0x193c91000 + 0x1cf8 = // start + 0x4 But according to the JIT allocation problems which has been met by other = projects: = https://patchwork.kernel.org/project/qemu-devel/patch/20201108232425.1705-= 7-j@getutm.app/#23841029 = <https://patchwork.kernel.org/project/qemu-devel/patch/20201108232425.1705= -7-j@getutm.app/#23841029> I attempted to patch libguile/jit.c like that: --- libguile/jit.c 1676274604.835054871 +++ libguile/jit.c.old 1676204975.415324076 @@ -47,8 +47,17 @@ #include <sys/mman.h> #endif =20 -#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP +#if defined __APPLE__ #include <libkern/OSCacheControl.h> +#include "tcg-apple-jit.h" + +static void tb_exec_change(bool locked) +{ + if (jit_write_protect_supported()) { + jit_write_protect(locked); + } +} + #endif =20 #include "jit.h" @@ -1414,8 +1435,9 @@ =20 uint8_t *ret =3D jit_address (j->jit); =20 -#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP - pthread_jit_write_protect_np(0); +#if defined __APPLE__ +// pthread_jit_write_protect_np(0); + tb_exec_change(0); #endif =20 emit (j); @@ -1423,10 +1445,11 @@ size_t size; if (!jit_has_overflow (j->jit) && jit_end (j->jit, &size)) { -#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP +#if defined __APPLE__ /* protect previous code arena. leave unprotected after = emit() since jit_end() also writes to code arena. */ - pthread_jit_write_protect_np(1); +// pthread_jit_write_protect_np(1); + tb_exec_change(1); sys_icache_invalidate(arena->base, arena->size); #endif ASSERT (size <=3D (arena->size - arena->used)); @@ -1442,9 +1465,10 @@ } else { -#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP +#if defined __APPLE__ /* protect previous code arena */ - pthread_jit_write_protect_np(1); +// pthread_jit_write_protect_np(1); + tb_exec_change(1); sys_icache_invalidate(arena->base, arena->size); #endif jit_reset (j->jit); Which replaces the previous `pthread_jit_write_protect_np` with another = iOS capable implementation that provided in the link above: /* * Apple Silicon functions for JIT handling * * Copyright (c) 2020 osy * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public * License as published by the Free Software Foundation; either * version 2.1 of the License, or (at your option) any later version. * * This library is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU * Lesser General Public License for more details. * * You should have received a copy of the GNU Lesser General Public * License along with this library; if not, see = <http://www.gnu.org/licenses/>. */ #ifndef TCG_APPLE_JIT_H #define TCG_APPLE_JIT_H /* * APRR handling * Credits to: https://siguza.github.io/APRR/ * Reversed from /usr/lib/system/libsystem_pthread.dylib */ #if defined(__aarch64__) && defined(__APPLE__) #define _COMM_PAGE_START_ADDRESS (0x0000000FFFFFC000ULL) /* In = TTBR0 */ #define _COMM_PAGE_APRR_SUPPORT (_COMM_PAGE_START_ADDRESS + = 0x10C) #define _COMM_PAGE_APPR_WRITE_ENABLE (_COMM_PAGE_START_ADDRESS + = 0x110) #define _COMM_PAGE_APRR_WRITE_DISABLE (_COMM_PAGE_START_ADDRESS + = 0x118) static __attribute__((__always_inline__)) bool = jit_write_protect_supported(void) { /* Access shared kernel page at fixed memory location. */ uint8_t aprr_support =3D *(volatile uint8_t = *)_COMM_PAGE_APRR_SUPPORT; return aprr_support > 0; } /* write protect enable =3D write disable */ static __attribute__((__always_inline__)) void jit_write_protect(int = enabled) { /* Access shared kernel page at fixed memory location. */ uint8_t aprr_support =3D *(volatile uint8_t = *)_COMM_PAGE_APRR_SUPPORT; if (aprr_support =3D=3D 0 || aprr_support > 3) { return; } else if (aprr_support =3D=3D 1) { __asm__ __volatile__ ( "mov x0, %0\n" "ldr x0, [x0]\n" "msr S3_4_c15_c2_7, x0\n" "isb sy\n" :: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE : _COMM_PAGE_APPR_WRITE_ENABLE) : "memory", "x0" ); } else { __asm__ __volatile__ ( "mov x0, %0\n" "ldr x0, [x0]\n" "msr S3_6_c15_c1_5, x0\n" "isb sy\n" :: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE : _COMM_PAGE_APPR_WRITE_ENABLE) : "memory", "x0" ); } } #else /* defined(__aarch64__) && defined(__APPLE__) */ static __attribute__((__always_inline__)) bool = jit_write_protect_supported(void) { return false; } static __attribute__((__always_inline__)) void jit_write_protect(int = enabled) { } #endif #endif /* define TCG_APPLE_JIT_H */ Then we have a successful jit allocation in the first process, but while = it comes to stage 0, every allocation fails like that: make[3]: Entering directory '/buildroot/guile-3.0.9/libguile' cat alist.doc array-handle.doc array-map.doc arrays.doc async.doc = atomic.doc backtrace.doc boolean.doc bitvectors.doc bytevectors.doc = chars.doc control.doc continuations.doc debug.doc deprecated.doc = deprecation.doc dynl.doc dynwind.doc eq.doc error.doc eval.doc = evalext.doc exceptions.doc expand.doc extensions.doc fdes-finalizers.doc = feature.doc filesys.doc fluids.doc foreign.doc fports.doc gc-malloc.doc = gc.doc gettext.doc generalized-vectors.doc goops.doc gsubr.doc = guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc init.doc ioext.doc = keywords.doc list.doc load.doc macros.doc mallocs.doc memoize.doc = modules.doc numbers.doc objprop.doc options.doc pairs.doc ports.doc = print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc random.doc = rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc smob.doc = sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc = srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc = strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc = unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc = vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc = posix.doc net_db.doc socket.doc regex-posix.doc | GUILE_AUTO_COMPILE=3D0 = ../meta/build-env guild snarf-check-and-output-texi > = guile-procedures.texi || { rm guile-procedures.texi; false; } make[3]: Leaving directory '/buildroot/guile-3.0.9/libguile' make[2]: Leaving directory '/buildroot/guile-3.0.9/libguile' Making all in module make[2]: Entering directory '/buildroot/guile-3.0.9/module' make[2]: Nothing to be done for 'all'. make[2]: Leaving directory '/buildroot/guile-3.0.9/module' Making all in stage0 make[2]: Entering directory '/buildroot/guile-3.0.9/stage0' GUILE_BOOTSTRAP_STAGE=3Dstage0 ../meta/build-env guild compile = --target=3D"aarch64-apple-darwin20.4.0" -W0 -O1 -L = "/buildroot/guile-3.0.9/module" -o "ice-9/eval.go" = "../module/ice-9/eval.scm" allocating JIT code buffer failed: Invalid argument JIT failed due to resource exhaustion disabling automatic JIT compilation The issue #47429 mentions a similar issue related with JIT allocation = under Darwin20+, but alongside to the implementation of = `allocate_code_arena` method in libguile/jit.c in 3.0.5 we don't have = any processes regarding to the Darwin20+ JIT write protections, the = culprit might different. Now what I can confirm is that - MAP_JIT is working on both macOS and iOS - If not providing an alternative method of pthread_write_protect_np, we = then should properly handle the flags that passing to mmaped pointers to = make sure RWX does not applied same time when doing W/X operations. - Current Guile does not have a working JIT on arm64 iOS if there's no = HAVE_PTHREAD_JIT_WRITE_PROTECT_NP defined= --Apple-Mail=_7FD80609-3E3C-4B38-A565-5728DCC1029B Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D"">I'm = building Guile 3.0.9 on iOS 14.5.1 (Darwin 20.4.0) which is suffering = some system API differences within pthread_jit* stuff.<div class=3D""><br = class=3D""></div><div class=3D""><br class=3D""></div><div class=3D""><div= class=3D"">cat alist.doc array-handle.doc array-map.doc arrays.doc = async.doc atomic.doc backtrace.doc boolean.doc bitvectors.doc = bytevectors.doc chars.doc control.doc continuations.doc debug.doc = deprecated.doc deprecation.doc dynl.doc dynwind.doc eq.doc error.doc = eval.doc evalext.doc exceptions.doc expand.doc extensions.doc = fdes-finalizers.doc feature.doc filesys.doc fluids.doc foreign.doc = fports.doc gc-malloc.doc gc.doc gettext.doc generalized-vectors.doc = goops.doc gsubr.doc guardians.doc hash.doc hashtab.doc hooks.doc = i18n.doc init.doc ioext.doc keywords.doc list.doc load.doc macros.doc = mallocs.doc memoize.doc modules.doc numbers.doc objprop.doc options.doc = pairs.doc ports.doc print.doc procprop.doc procs.doc promises.doc = r6rs-ports.doc random.doc rdelim.doc read.doc rw.doc scmsigs.doc = script.doc simpos.doc smob.doc sort.doc srcprop.doc srfi-1.doc = srfi-4.doc srfi-13.doc srfi-14.doc srfi-60.doc stackchk.doc stacks.doc = stime.doc strings.doc strorder.doc strports.doc struct.doc symbols.doc = syntax.doc threads.doc throw.doc unicode.doc uniform.doc values.doc = variable.doc vectors.doc version.doc vports.doc weak-set.doc = weak-table.doc weak-vector.doc dynl.doc posix.doc net_db.doc socket.doc = regex-posix.doc | GUILE_AUTO_COMPILE=3D0 ../meta/build-env guild = snarf-check-and-output-texi > = guile-procedures.texi || { rm guile-procedures.texi; false; }</div><div = class=3D"">cat: write error: Broken pipe</div></div><div class=3D""><br = class=3D""></div><div class=3D"">After inspecting the crash logs, it = might be some unexpected freeing of mmap cache, the librecompat.0.dylib = includes a sys_icache_invalidate wrapper for __clear_cache function (as = the same implementation with macOS one since they are all come from = LLVM) which has not been compiled to system libcompiler_rt.dylib for = iOS.</div><div class=3D""><br class=3D""></div><div class=3D""><pre = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: = 8pt;" class=3D"">Exception Type: EXC_BAD_ACCESS (SIGBUS) Exception Subtype: KERN_PROTECTION_FAILURE at 0x0000000104ccb9d8 VM Region Info: 0x104ccb9d8 is in 0x104cc4000-0x104ccc000; bytes after = start: 31192 bytes before end: 1575 REGION TYPE START - END [ VSIZE] PRT/MAX = SHRMOD REGION DETAIL __LINKEDIT 104bc0000-104cc4000 [ 1040K] r--/rw- = SM=3DCOW ...e-3.0.1.dylib ---> __TEXT 104cc4000-104ccc000 [ 32K] r--/rw- = SM=3DCOW ...ompat.0.dylib __DATA_CONST 104ccc000-104cd0000 [ 16K] rw-/rw- = SM=3DCOW ...ompat.0.dylib Termination Signal: Bus error: 10 Termination Reason: Namespace SIGNAL, Code 0xa Terminating Process: exc handler [9957] Triggered by Thread: 0 Thread 0 name: Dispatch queue: com.apple.main-thread Thread 0 Crashed: 0 + librecompat.0.dylib 0x104ccb9d8 0x104cc4000 + 0x79d8 = // __clear_cache + 0x0 1 libguile-3.0.1.dylib 0x104b4b728 0x104a44000 + = 0x107728 // = /buildroot/guile-3.0.9/libguile/./lightening/lightening/aarch64.c:217 2 libguile-3.0.1.dylib 0x104b4b3b0 0x104a44000 + = 0x1073b0 // = /buildroot/guile-3.0.9/libguile/./lightening/lightening/lightening.c:241 3 libguile-3.0.1.dylib 0x104aa12e0 0x104a44000 + = 0x5d2e0 // /buildroot/guile-3.0.9/libguile/jit.c:1424 4 libguile-3.0.1.dylib 0x104aa0c38 0x104a44000 + = 0x5cc38 // /buildroot/guile-3.0.9/libguile/jit.c:5895 5 libsystem_pthread.dylib 0x1def01f60 0x1deeff000 + 0x2f60 = // __pthread_once_handler + 0x50 6 libsystem_platform.dylib 0x1deefb964 0x1deef7000 + 0x4964 = // _os_once_callout + 0x20 7 libsystem_pthread.dylib 0x1def01ef4 0x1deeff000 + 0x2ef4 = // pthread_once + 0x64 8 libguile-3.0.1.dylib 0x104aa0558 0x104a44000 + = 0x5c558 // /buildroot/guile-3.0.9/libguile/jit.c:0 9 libguile-3.0.1.dylib 0x104aa0410 0x104a44000 + = 0x5c410 // /buildroot/guile-3.0.9/libguile/jit.c:6031 10 libguile-3.0.1.dylib 0x104b35774 0x104a44000 + = 0xf1774 // /buildroot/guile-3.0.9/libguile/./vm-engine.c:370 11 libguile-3.0.1.dylib 0x104b323d0 0x104a44000 + = 0xee3d0 // /buildroot/guile-3.0.9/libguile/vm.c:1615 12 libguile-3.0.1.dylib 0x104a6776c 0x104a44000 + = 0x2376c // /buildroot/guile-3.0.9/libguile/eval.c:496 13 libguile-3.0.1.dylib 0x104af9e20 0x104a44000 + = 0xb5e20 // /buildroot/guile-3.0.9/libguile/read.c:1553 14 libguile-3.0.1.dylib 0x104abb158 0x104a44000 + = 0x77158 // /buildroot/guile-3.0.9/libguile/load.c:124 15 libguile-3.0.1.dylib 0x104abc3d4 0x104a44000 + = 0x783d4 // /buildroot/guile-3.0.9/libguile/load.c:1267 16 libguile-3.0.1.dylib 0x104abcb28 0x104a44000 + = 0x78b28 // /buildroot/guile-3.0.9/libguile/load.c:1275 17 libguile-3.0.1.dylib 0x104a8e0ec 0x104a44000 + = 0x4a0ec // /buildroot/guile-3.0.9/libguile/init.c:230 18 libguile-3.0.1.dylib 0x104a8e458 0x104a44000 + = 0x4a458 // /buildroot/guile-3.0.9/libguile/init.c:510 19 libguile-3.0.1.dylib 0x104b2a89c 0x104a44000 + = 0xe689c // /buildroot/guile-3.0.9/libguile/threads.c:578 20 libguile-3.0.1.dylib 0x104b2d47c 0x104a44000 + = 0xe947c // /buildroot/guile-3.0.9/libguile/threads.c:642 21 + libgc.1.dylib 0x1053688e4 0x105354000 + = 0x148e4 // /buildroot/gc-8.2.2/misc.c:2173 22 libguile-3.0.1.dylib 0x104b2a97c 0x104a44000 + = 0xe697c // /buildroot/guile-3.0.9/libguile/threads.c:692 23 libguile-3.0.1.dylib 0x104b2a930 0x104a44000 + = 0xe6930 // /buildroot/guile-3.0.9/libguile/threads.c:698 24 libguile-3.0.1.dylib 0x104a8e190 0x104a44000 + = 0x4a190 // /buildroot/guile-3.0.9/libguile/init.c:295 25 guile (*) 0x104a37d90 0x104a30000 + 0x7d90 = // /buildroot/guile-3.0.9/libguile/guile.c:94 26 libdyld.dylib 0x193c92cf8 0x193c91000 + 0x1cf8 = // start + 0x4</pre><pre style=3D"caret-color: rgb(0, 0, 0); color: = rgb(0, 0, 0); font-size: 8pt;" class=3D""><br class=3D""></pre><pre = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: = 8pt;" class=3D"">But according to the JIT allocation problems which has = been met by other projects:</pre><pre class=3D""><font color=3D"#000000" = size=3D"2" class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" = class=3D""><a = href=3D"https://patchwork.kernel.org/project/qemu-devel/patch/202011082324= 25.1705-7-j@getutm.app/#23841029" = class=3D"">https://patchwork.kernel.org/project/qemu-devel/patch/202011082= 32425.1705-7-j@getutm.app/#23841029</a></span></font></pre><pre = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: = 8pt;" class=3D""><br class=3D""></pre><pre style=3D"caret-color: rgb(0, = 0, 0); color: rgb(0, 0, 0); font-size: 8pt;" class=3D"">I attempted to = patch libguile/jit.c like that:</pre><pre class=3D""><font = color=3D"#000000" size=3D"2" class=3D""><span style=3D"caret-color: = rgb(0, 0, 0);" class=3D"">--- libguile/jit.c 1676274604.835054871 +++ libguile/jit.c.old 1676204975.415324076 @@ -47,8 +47,17 @@ #include <sys/mman.h> #endif =20 -#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP +#if defined __APPLE__ #include <libkern/OSCacheControl.h> +#include "tcg-apple-jit.h" + +static void tb_exec_change(bool locked) +{ + if (jit_write_protect_supported()) { + jit_write_protect(locked); + } +} + #endif =20 #include "jit.h" @@ -1414,8 +1435,9 @@ =20 uint8_t *ret =3D jit_address (j->jit); =20 -#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP - pthread_jit_write_protect_np(0); +#if defined __APPLE__ +// pthread_jit_write_protect_np(0); + tb_exec_change(0); #endif =20 emit (j); @@ -1423,10 +1445,11 @@ size_t size; if (!jit_has_overflow (j->jit) && jit_end (j->jit, = &size)) { -#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP +#if defined __APPLE__ /* protect previous code arena. leave unprotected after = emit() since jit_end() also writes to code arena. */ - pthread_jit_write_protect_np(1); +// pthread_jit_write_protect_np(1); + tb_exec_change(1); sys_icache_invalidate(arena->base, arena->size); #endif ASSERT (size <=3D (arena->size - arena->used)); @@ -1442,9 +1465,10 @@ } else { -#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP +#if defined __APPLE__ /* protect previous code arena */ - pthread_jit_write_protect_np(1); +// pthread_jit_write_protect_np(1); + tb_exec_change(1); sys_icache_invalidate(arena->base, arena->size); #endif jit_reset (j->jit);</span></font></pre><pre = style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-size: = 8pt;" class=3D""><br class=3D""></pre><pre style=3D"caret-color: rgb(0, = 0, 0); color: rgb(0, 0, 0); font-size: 8pt;" class=3D"">Which replaces = the previous `<span style=3D"caret-color: rgb(255, 255, 255); font-size: = small;" class=3D"">pthread_jit_write_protect_np` with another iOS = capable implementation that provided in the link above:</span></pre><pre = class=3D""><font color=3D"#000000" size=3D"2" class=3D""><span = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">/* * Apple Silicon functions for JIT handling * * Copyright (c) 2020 osy * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public * License as published by the Free Software Foundation; either * version 2.1 of the License, or (at your option) any later version. * * This library is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU * Lesser General Public License for more details. * * You should have received a copy of the GNU Lesser General Public * License along with this library; if not, see <<a = href=3D"http://www.gnu.org/licenses/" = class=3D"">http://www.gnu.org/licenses/</a>>. */ #ifndef TCG_APPLE_JIT_H #define TCG_APPLE_JIT_H /* * APRR handling * Credits to: <a href=3D"https://siguza.github.io/APRR/" = class=3D"">https://siguza.github.io/APRR/</a> * Reversed from /usr/lib/system/libsystem_pthread.dylib */ #if defined(__aarch64__) && defined(__APPLE__) #define _COMM_PAGE_START_ADDRESS (0x0000000FFFFFC000ULL) /* In = TTBR0 */ #define _COMM_PAGE_APRR_SUPPORT (_COMM_PAGE_START_ADDRESS + = 0x10C) #define _COMM_PAGE_APPR_WRITE_ENABLE (_COMM_PAGE_START_ADDRESS + = 0x110) #define _COMM_PAGE_APRR_WRITE_DISABLE (_COMM_PAGE_START_ADDRESS + = 0x118) static __attribute__((__always_inline__)) bool = jit_write_protect_supported(void) { /* Access shared kernel page at fixed memory location. */ uint8_t aprr_support =3D *(volatile uint8_t = *)_COMM_PAGE_APRR_SUPPORT; return aprr_support > 0; } /* write protect enable =3D write disable */ static __attribute__((__always_inline__)) void jit_write_protect(int = enabled) { /* Access shared kernel page at fixed memory location. */ uint8_t aprr_support =3D *(volatile uint8_t = *)_COMM_PAGE_APRR_SUPPORT; if (aprr_support =3D=3D 0 || aprr_support > 3) { return; } else if (aprr_support =3D=3D 1) { __asm__ __volatile__ ( "mov x0, %0\n" "ldr x0, [x0]\n" "msr S3_4_c15_c2_7, x0\n" "isb sy\n" :: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE : _COMM_PAGE_APPR_WRITE_ENABLE) : "memory", "x0" ); } else { __asm__ __volatile__ ( "mov x0, %0\n" "ldr x0, [x0]\n" "msr S3_6_c15_c1_5, x0\n" "isb sy\n" :: "r" (enabled ? _COMM_PAGE_APRR_WRITE_DISABLE : _COMM_PAGE_APPR_WRITE_ENABLE) : "memory", "x0" ); } } #else /* defined(__aarch64__) && defined(__APPLE__) */ static __attribute__((__always_inline__)) bool = jit_write_protect_supported(void) { return false; } static __attribute__((__always_inline__)) void jit_write_protect(int = enabled) { } #endif #endif /* define TCG_APPLE_JIT_H */</span></font></pre><pre class=3D""><br= class=3D""></pre><pre class=3D"">Then we have a successful jit = allocation in the first process, but while it comes to stage 0, every = allocation fails like that:</pre><pre class=3D"">make[3]: Entering = directory '/buildroot/guile-3.0.9/libguile' cat alist.doc array-handle.doc array-map.doc arrays.doc async.doc = atomic.doc backtrace.doc boolean.doc bitvectors.doc bytevectors.doc = chars.doc control.doc continuations.doc debug.doc deprecated.doc = deprecation.doc dynl.doc dynwind.doc eq.doc error.doc eval.doc = evalext.doc exceptions.doc expand.doc extensions.doc fdes-finalizers.doc = feature.doc filesys.doc fluids.doc foreign.doc fports.doc gc-malloc.doc = gc.doc gettext.doc generalized-vectors.doc goops.doc gsubr.doc = guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc init.doc ioext.doc = keywords.doc list.doc load.doc macros.doc mallocs.doc memoize.doc = modules.doc numbers.doc objprop.doc options.doc pairs.doc ports.doc = print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc random.doc = rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc smob.doc = sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc = srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc = strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc = unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc = vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc = posix.doc net_db.doc socket.doc regex-posix.doc | GUILE_AUTO_COMPILE=3D0 = ../meta/build-env guild snarf-check-and-output-texi > = guile-procedures.texi || { rm guile-procedures.texi; false; } make[3]: Leaving directory '/buildroot/guile-3.0.9/libguile' make[2]: Leaving directory '/buildroot/guile-3.0.9/libguile' Making all in module make[2]: Entering directory '/buildroot/guile-3.0.9/module' make[2]: Nothing to be done for 'all'. make[2]: Leaving directory '/buildroot/guile-3.0.9/module' Making all in stage0 make[2]: Entering directory '/buildroot/guile-3.0.9/stage0' GUILE_BOOTSTRAP_STAGE=3Dstage0 ../meta/build-env guild compile = --target=3D"aarch64-apple-darwin20.4.0" -W0 -O1 -L = "/buildroot/guile-3.0.9/module" -o "ice-9/eval.go" = "../module/ice-9/eval.scm" allocating JIT code buffer failed: Invalid argument JIT failed due to resource exhaustion disabling automatic JIT compilation</pre><pre class=3D"">The issue = #47429 mentions a similar issue related with JIT allocation under = Darwin20+, but alongside to the implementation of `<font color=3D"#000000"= size=3D"3" class=3D"">allocate_code_arena` method in libguile/jit.c in = 3.0.5 we don't have any processes regarding to the Darwin20+ JIT write = protections, the culprit might different.</font></pre><pre = class=3D""><font color=3D"#000000" size=3D"3" class=3D""><span = style=3D"caret-color: rgb(0, 0, 0);" class=3D"">Now what I can confirm = is that</span></font></pre><pre class=3D""><font color=3D"#000000" = size=3D"3" class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" = class=3D"">- MAP_JIT is working on both macOS and = iOS</span></font></pre><pre class=3D""><font color=3D"#000000" size=3D"3" = class=3D"">- If not providing an alternative method of = pthread_write_protect_np, we then should properly handle the flags that = passing to mmaped pointers to make sure RWX does not applied same time = when doing W/X operations.</font></pre><pre class=3D""><font = color=3D"#000000" size=3D"3" class=3D"">- Current Guile does not have a = working JIT on arm64 iOS if there's no HAVE_PTHREAD_JIT_WRITE_PROTECT_NP = defined</font></pre></div></body></html>= --Apple-Mail=_7FD80609-3E3C-4B38-A565-5728DCC1029B--