From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Aleix_Conchillo_Flaqu=C3=A9?= Newsgroups: gmane.lisp.guile.devel Subject: Re: [PATCH] fix Apple Silicon JIT compilation Date: Thu, 17 Nov 2022 17:37:12 -0800 Message-ID: References: <20221118013615.81209-1-aconchillo@gmail.com> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="00000000000057593305edb4bf1c" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="33720"; mail-complaints-to="usenet@ciao.gmane.io" To: guile-devel@gnu.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Fri Nov 18 02:37:38 2022 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ovqK1-0008XZ-UZ for guile-devel@m.gmane-mx.org; Fri, 18 Nov 2022 02:37:38 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ovqJu-000771-Qy; Thu, 17 Nov 2022 20:37:30 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ovqJr-00076p-Jr for guile-devel@gnu.org; Thu, 17 Nov 2022 20:37:27 -0500 Original-Received: from mail-vk1-xa35.google.com ([2607:f8b0:4864:20::a35]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ovqJp-0004Aw-DG for guile-devel@gnu.org; Thu, 17 Nov 2022 20:37:27 -0500 Original-Received: by mail-vk1-xa35.google.com with SMTP id bk51so1754557vkb.4 for ; Thu, 17 Nov 2022 17:37:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=VksDEDILV4Zxz8OWkiHdHHYCS6MZ9qyPgpw5pyr7SlY=; b=gRGO6QB5g7SG2Pnl/yoiVgxd5vyIwZZOzoxRviEg8hY15qDWj/QDU1WScGfYhNXHH7 Ind5ni1QvTbslMEfr4g0MDlocl5yBaIvT1po854+O0ldZKt+Hk+R2InBfAsoDb+BMM8y tFgI+BqS5E+9wUEcPsyKrTSmLS5206x17WLCpJekP8K9dHoAn90bx0ECe8oTz8zPJuvJ J6B2GsxY0Y22X20pSoq9+0E+gwyJFPUPPaACV39KpozCfXy/MS9vBKt3S01MAtxP6KSC kT/JMoYpnRuTtDuelvZ6BrCZ4/tKpvHzADOguA7Cz2eGwimwEq2ienwDXuvezZ0KrgQo rmOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VksDEDILV4Zxz8OWkiHdHHYCS6MZ9qyPgpw5pyr7SlY=; b=7tJWh5ZcP2DMgEflybQyauBy3/Jjpv1z+kcQ89jcalPIgHjaig5RAGxp6OQ4w05xL/ 4lmScQrIaQt0hB6jR0/mIoS587vrms00ci5f8SIiGMn02okPOC1eTqVf6oRIneG4qdpv xBi7a3U75tZgkVGigUfr7RwV5GgC3X4K91xsFlJX92Ooqa0b3K8V14RwwB0emQYs7jF9 yBn6Jf1fuf9R9n+Ra03naNUoTJI3Epc66tKqVippV50G8npp+0D5xWIY1xx744mjjv8h 7beZzZiMwJHgVydk18EanoVNNVYlJo9v2NEZBKJaXbRHGx9F0gmC2nZAp3KBAiGzwhBJ 1OHA== X-Gm-Message-State: ANoB5pnFUHGKuiHz9cvp/JFw4dcuMf28D6FsvPWJIGD1mu0LM1IYYE8j au9bKuOdBxxzIWhwf3paVixKArDuY0EO4owBLtY7T5UkYhc= X-Google-Smtp-Source: AA0mqf4/CK95tXusmvituYYDJ8dLMHX4xVKgzwRyHGz+QYQgWIXWWGNtgwb7IHtzxZYT1hydBwut6+BgbJfBajavsF4= X-Received: by 2002:a05:6122:990:b0:3bc:721:4fec with SMTP id g16-20020a056122099000b003bc07214fecmr2918576vkd.8.1668735443620; Thu, 17 Nov 2022 17:37:23 -0800 (PST) In-Reply-To: <20221118013615.81209-1-aconchillo@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::a35; envelope-from=aconchillo@gmail.com; helo=mail-vk1-xa35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.lisp.guile.devel:21479 Archived-At: --00000000000057593305edb4bf1c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Apologies for re-sending this. Just realized this should be called Apple Silicon and not just Apple M1. On Thu, Nov 17, 2022 at 5:36 PM Aleix Conchillo Flaqu=C3=A9 wrote: > * configure.ac: check for pthread_jit_write_protect_np. > > * libguile/jit.c: add support for Apple Silicon JIT compilation. > > Fixes https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D44505 > --- > configure.ac | 3 +++ > libguile/jit.c | 25 ++++++++++++++++++++++++- > 2 files changed, 27 insertions(+), 1 deletion(-) > > diff --git a/configure.ac b/configure.ac > index b3879df1f..f8c12f0d7 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -1187,6 +1187,9 @@ case "$with_threads" in > pthread_get_stackaddr_np pthread_attr_get_np pthread_sigmask \ > pthread_cancel]) > > + # Apple Silicon JIT code arena needs to be unprotected before writin= g. > + AC_CHECK_FUNCS([pthread_jit_write_protect_np]) > + > # On past versions of Solaris, believe 8 through 10 at least, you > # had to write "pthread_once_t foo =3D { PTHREAD_ONCE_INIT };". > # This is contrary to POSIX: > diff --git a/libguile/jit.c b/libguile/jit.c > index 8420829b4..5cef8fae3 100644 > --- a/libguile/jit.c > +++ b/libguile/jit.c > @@ -47,6 +47,10 @@ > #include > #endif > > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > +#include > +#endif > + > #include "jit.h" > > > @@ -1349,9 +1353,13 @@ allocate_code_arena (size_t size, struct code_aren= a > *prev) > ret->size =3D size; > ret->prev =3D prev; > #ifndef __MINGW32__ > + int flags =3D MAP_PRIVATE | MAP_ANONYMOUS; > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > + flags |=3D MAP_JIT; > +#endif > ret->base =3D mmap (NULL, ret->size, > PROT_EXEC | PROT_READ | PROT_WRITE, > - MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); > + flags, -1, 0); > if (ret->base =3D=3D MAP_FAILED) > { > perror ("allocating JIT code buffer failed"); > @@ -1406,11 +1414,21 @@ emit_code (scm_jit_state *j, void (*emit) > (scm_jit_state *)) > > uint8_t *ret =3D jit_address (j->jit); > > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > + pthread_jit_write_protect_np(0); > +#endif > + > emit (j); > > size_t size; > if (!jit_has_overflow (j->jit) && jit_end (j->jit, &size)) > { > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > + /* protect previous code arena. leave unprotected after emit() > + since jit_end() also writes to code arena. */ > + pthread_jit_write_protect_np(1); > + sys_icache_invalidate(arena->base, arena->size); > +#endif > ASSERT (size <=3D (arena->size - arena->used)); > DEBUG ("mcode: %p,+%zu\n", ret, size); > arena->used +=3D size; > @@ -1424,6 +1442,11 @@ emit_code (scm_jit_state *j, void (*emit) > (scm_jit_state *)) > } > else > { > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > + /* protect previous code arena */ > + pthread_jit_write_protect_np(1); > + sys_icache_invalidate(arena->base, arena->size); > +#endif > jit_reset (j->jit); > if (arena->used =3D=3D 0) > { > -- > 2.37.1 (Apple Git-137.1) > > --00000000000057593305edb4bf1c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Apologies for re-sending this. Just realized this shoul= d be called Apple Silicon and not just Apple M1.

On Thu, Nov 17, 2022= at 5:36 PM Aleix Conchillo Flaqu=C3=A9 <aconchillo@gmail.com> wrote:
* configure.ac: check for pthread_jit_write_prot= ect_np.

* libguile/jit.c: add support for Apple Silicon JIT compilation.

Fixes https://debbugs.gnu.org/cgi/bugreport.cgi= ?bug=3D44505
---
=C2=A0= configure.ac=C2=A0 =C2=A0|=C2=A0 3 +++
=C2=A0libguile/jit.c | 25 ++++++++++++++++++++++++-
=C2=A02 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index b3879df1f..f8c12f0d7 100644
--- a/= configure.ac
+++ b/= configure.ac
@@ -1187,6 +1187,9 @@ case "$with_threads" in
=C2=A0 =C2=A0 =C2=A0 =C2=A0pthread_get_stackaddr_np pthread_attr_get_np pth= read_sigmask=C2=A0 =C2=A0 =C2=A0\
=C2=A0 =C2=A0 =C2=A0 =C2=A0pthread_cancel])

+=C2=A0 =C2=A0 # Apple Silicon JIT code arena needs to be unprotected befor= e writing.
+=C2=A0 =C2=A0 AC_CHECK_FUNCS([pthread_jit_write_protect_np])
+
=C2=A0 =C2=A0 =C2=A0# On past versions of Solaris, believe 8 through 10 at = least, you
=C2=A0 =C2=A0 =C2=A0# had to write "pthread_once_t foo =3D { PTHREAD_O= NCE_INIT };".
=C2=A0 =C2=A0 =C2=A0# This is contrary to POSIX:
diff --git a/libguile/jit.c b/libguile/jit.c
index 8420829b4..5cef8fae3 100644
--- a/libguile/jit.c
+++ b/libguile/jit.c
@@ -47,6 +47,10 @@
=C2=A0#include <sys/mman.h>
=C2=A0#endif

+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#include <libkern/OSCacheControl.h>
+#endif
+
=C2=A0#include "jit.h"


@@ -1349,9 +1353,13 @@ allocate_code_arena (size_t size, struct code_arena = *prev)
=C2=A0 =C2=A0ret->size =3D size;
=C2=A0 =C2=A0ret->prev =3D prev;
=C2=A0#ifndef __MINGW32__
+=C2=A0 int flags =3D MAP_PRIVATE | MAP_ANONYMOUS;
+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+=C2=A0 flags |=3D MAP_JIT;
+#endif
=C2=A0 =C2=A0ret->base =3D mmap (NULL, ret->size,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0PROT_EXEC | PROT_READ | PROT_WRITE,
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 MAP_= PRIVATE | MAP_ANONYMOUS, -1, 0);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 flag= s, -1, 0);
=C2=A0 =C2=A0if (ret->base =3D=3D MAP_FAILED)
=C2=A0 =C2=A0 =C2=A0{
=C2=A0 =C2=A0 =C2=A0 =C2=A0perror ("allocating JIT code buffer failed&= quot;);
@@ -1406,11 +1414,21 @@ emit_code (scm_jit_state *j, void (*emit) (scm_jit_= state *))

=C2=A0 =C2=A0 =C2=A0 =C2=A0uint8_t *ret =3D jit_address (j->jit);

+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+=C2=A0 =C2=A0 =C2=A0 pthread_jit_write_protect_np(0);
+#endif
+
=C2=A0 =C2=A0 =C2=A0 =C2=A0emit (j);

=C2=A0 =C2=A0 =C2=A0 =C2=A0size_t size;
=C2=A0 =C2=A0 =C2=A0 =C2=A0if (!jit_has_overflow (j->jit) && jit= _end (j->jit, &size))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{
+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /* protect previous code arena. leave u= nprotected after emit()
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0since jit_end() also write= s to code arena. */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pthread_jit_write_protect_np(1);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sys_icache_invalidate(arena->base, a= rena->size);
+#endif
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ASSERT (size <=3D (arena->si= ze - arena->used));
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0DEBUG ("mcode: %p,+%zu\n"= ;, ret, size);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0arena->used +=3D size;
@@ -1424,6 +1442,11 @@ emit_code (scm_jit_state *j, void (*emit) (scm_jit_s= tate *))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
=C2=A0 =C2=A0 =C2=A0 =C2=A0else
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{
+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /* protect previous code arena */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pthread_jit_write_protect_np(1);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sys_icache_invalidate(arena->base, a= rena->size);
+#endif
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0jit_reset (j->jit);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (arena->used =3D=3D 0)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{
--
2.37.1 (Apple Git-137.1)

--00000000000057593305edb4bf1c--