From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Aleix_Conchillo_Flaqu=C3=A9?= Newsgroups: gmane.lisp.guile.devel Subject: Re: [PATCH] fix Apple Silicon JIT compilation Date: Sun, 18 Dec 2022 11:54:52 -0800 Message-ID: References: <20221118013615.81209-1-aconchillo@gmail.com> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000001edf0305f01f948f" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31181"; mail-complaints-to="usenet@ciao.gmane.io" To: guile-devel@gnu.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Sun Dec 18 20:55:40 2022 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p6zl5-0007v9-Tb for guile-devel@m.gmane-mx.org; Sun, 18 Dec 2022 20:55:40 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p6zkb-0002Or-3P; Sun, 18 Dec 2022 14:55:09 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p6zkZ-0002Oi-No for guile-devel@gnu.org; Sun, 18 Dec 2022 14:55:07 -0500 Original-Received: from mail-vs1-xe33.google.com ([2607:f8b0:4864:20::e33]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p6zkW-0005Qw-Uv for guile-devel@gnu.org; Sun, 18 Dec 2022 14:55:06 -0500 Original-Received: by mail-vs1-xe33.google.com with SMTP id i2so7004344vsc.1 for ; Sun, 18 Dec 2022 11:55:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=+s/tpmxkn7jnoLMjGHD/hAFDjyEtpLq078bT1TaQ9Qg=; b=dpkGg/MZAoIA+8lr/Lvvd+NeFsQUqkIxXAwAicMeeYZJ7GvjD4GNatnOIt47UdaxWF 6au+g0PEgjjckENfDhQhpZqGe+tnUDklJu/rm12AWJFJJc4OIdYROAHrHG0nRVbLBIIQ QCafSGMvPS8iAwAfzSIhqln20rv61XEgky3QMi1R4E5lP/Q9HEX7PWRzZNXvjwpmhA7V q3J5LrrTNxFYuAGnSx56bYB5vdn2WJUn3t+ZlNvatt+hOk+2mPfuGtmi4wU1hEJ7zRMA LsDNmUgiebChE+CTbTCVrBq/RpQ0/vsys2c3eDEwxj5VnVLE7nI+vqG9EnCqxPsvHu2z nfkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+s/tpmxkn7jnoLMjGHD/hAFDjyEtpLq078bT1TaQ9Qg=; b=H/3JH/0CgqT66BorkPFLbLiOBjJiBkbytfgcoE6GKio8zpUkXgLlGKqwVUEhRJkiBe ClWPLae3D8nWlfdHyKwkxQpBwhX/4tewLhRWPO7YJlue5rtGse0pKyanxDmNp/J2gF4p xT+rIyLD61jfdIzCvUXyL8a10NBruuzw+6Hpfkv3/oBQWpHNWIiHcKjrkjNud6ljeWGY FE/QM7qOZP1UlBXtGbVezYQ5Y5EVaKBWDhQJpKdHPR0eP2mT9Gx3pts4GU6oDwIXXw0C iUruktFA5hhocC+KTJvG1ij1flaixMIt/iW+2BA1EWEtq/Am0rN8uCRCdYiLQ/wRcqTy +wog== X-Gm-Message-State: ANoB5pmkemBa8UZZQ7oP45Gyf+WdKEjreZ4xAZGVb/PIotVNR0s/uMun uKyfybQA51s3kLFFWSiAIhguCxpDGs76+eTaxUDKOgPm X-Google-Smtp-Source: AA0mqf4qKP+2J1/JstULy2XhQWDWhFUaMoTfJotD6RX0fu0+xdtiI4ijKdZu6gOeN5gz7XS03z/LJCxZJQL3fZXj8nA= X-Received: by 2002:a67:e19e:0:b0:3b3:1169:61b4 with SMTP id e30-20020a67e19e000000b003b3116961b4mr2457575vsl.56.1671393303255; Sun, 18 Dec 2022 11:55:03 -0800 (PST) In-Reply-To: <20221118013615.81209-1-aconchillo@gmail.com> Received-SPF: pass client-ip=2607:f8b0:4864:20::e33; envelope-from=aconchillo@gmail.com; helo=mail-vs1-xe33.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.lisp.guile.devel:21509 Archived-At: --0000000000001edf0305f01f948f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Ping. This one is important to run Guile with JIT on new Apple hardware. On Thu, Nov 17, 2022 at 5:36 PM Aleix Conchillo Flaqu=C3=A9 wrote: > * configure.ac: check for pthread_jit_write_protect_np. > > * libguile/jit.c: add support for Apple Silicon JIT compilation. > > Fixes https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D44505 > --- > configure.ac | 3 +++ > libguile/jit.c | 25 ++++++++++++++++++++++++- > 2 files changed, 27 insertions(+), 1 deletion(-) > > diff --git a/configure.ac b/configure.ac > index b3879df1f..f8c12f0d7 100644 > --- a/configure.ac > +++ b/configure.ac > @@ -1187,6 +1187,9 @@ case "$with_threads" in > pthread_get_stackaddr_np pthread_attr_get_np pthread_sigmask \ > pthread_cancel]) > > + # Apple Silicon JIT code arena needs to be unprotected before writin= g. > + AC_CHECK_FUNCS([pthread_jit_write_protect_np]) > + > # On past versions of Solaris, believe 8 through 10 at least, you > # had to write "pthread_once_t foo =3D { PTHREAD_ONCE_INIT };". > # This is contrary to POSIX: > diff --git a/libguile/jit.c b/libguile/jit.c > index 8420829b4..5cef8fae3 100644 > --- a/libguile/jit.c > +++ b/libguile/jit.c > @@ -47,6 +47,10 @@ > #include > #endif > > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > +#include > +#endif > + > #include "jit.h" > > > @@ -1349,9 +1353,13 @@ allocate_code_arena (size_t size, struct code_aren= a > *prev) > ret->size =3D size; > ret->prev =3D prev; > #ifndef __MINGW32__ > + int flags =3D MAP_PRIVATE | MAP_ANONYMOUS; > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > + flags |=3D MAP_JIT; > +#endif > ret->base =3D mmap (NULL, ret->size, > PROT_EXEC | PROT_READ | PROT_WRITE, > - MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); > + flags, -1, 0); > if (ret->base =3D=3D MAP_FAILED) > { > perror ("allocating JIT code buffer failed"); > @@ -1406,11 +1414,21 @@ emit_code (scm_jit_state *j, void (*emit) > (scm_jit_state *)) > > uint8_t *ret =3D jit_address (j->jit); > > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > + pthread_jit_write_protect_np(0); > +#endif > + > emit (j); > > size_t size; > if (!jit_has_overflow (j->jit) && jit_end (j->jit, &size)) > { > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > + /* protect previous code arena. leave unprotected after emit() > + since jit_end() also writes to code arena. */ > + pthread_jit_write_protect_np(1); > + sys_icache_invalidate(arena->base, arena->size); > +#endif > ASSERT (size <=3D (arena->size - arena->used)); > DEBUG ("mcode: %p,+%zu\n", ret, size); > arena->used +=3D size; > @@ -1424,6 +1442,11 @@ emit_code (scm_jit_state *j, void (*emit) > (scm_jit_state *)) > } > else > { > +#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP > + /* protect previous code arena */ > + pthread_jit_write_protect_np(1); > + sys_icache_invalidate(arena->base, arena->size); > +#endif > jit_reset (j->jit); > if (arena->used =3D=3D 0) > { > -- > 2.37.1 (Apple Git-137.1) > > --0000000000001edf0305f01f948f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Ping. This one is important to run Guile with JIT on ne= w Apple hardware.

On Thu, Nov 17, 2022 at 5:36 PM Aleix Conchillo Flaq= u=C3=A9 <aconchillo@gmail.com> wrote:
* <= a href=3D"http://configure.ac" rel=3D"noreferrer" target=3D"_blank">configu= re.ac: check for pthread_jit_write_protect_np.

* libguile/jit.c: add support for Apple Silicon JIT compilation.

Fixes https://debbugs.gnu.org/cgi/bugreport.cgi= ?bug=3D44505
---
=C2=A0= configure.ac=C2=A0 =C2=A0|=C2=A0 3 +++
=C2=A0libguile/jit.c | 25 ++++++++++++++++++++++++-
=C2=A02 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index b3879df1f..f8c12f0d7 100644
--- a/= configure.ac
+++ b/= configure.ac
@@ -1187,6 +1187,9 @@ case "$with_threads" in
=C2=A0 =C2=A0 =C2=A0 =C2=A0pthread_get_stackaddr_np pthread_attr_get_np pth= read_sigmask=C2=A0 =C2=A0 =C2=A0\
=C2=A0 =C2=A0 =C2=A0 =C2=A0pthread_cancel])

+=C2=A0 =C2=A0 # Apple Silicon JIT code arena needs to be unprotected befor= e writing.
+=C2=A0 =C2=A0 AC_CHECK_FUNCS([pthread_jit_write_protect_np])
+
=C2=A0 =C2=A0 =C2=A0# On past versions of Solaris, believe 8 through 10 at = least, you
=C2=A0 =C2=A0 =C2=A0# had to write "pthread_once_t foo =3D { PTHREAD_O= NCE_INIT };".
=C2=A0 =C2=A0 =C2=A0# This is contrary to POSIX:
diff --git a/libguile/jit.c b/libguile/jit.c
index 8420829b4..5cef8fae3 100644
--- a/libguile/jit.c
+++ b/libguile/jit.c
@@ -47,6 +47,10 @@
=C2=A0#include <sys/mman.h>
=C2=A0#endif

+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+#include <libkern/OSCacheControl.h>
+#endif
+
=C2=A0#include "jit.h"


@@ -1349,9 +1353,13 @@ allocate_code_arena (size_t size, struct code_arena = *prev)
=C2=A0 =C2=A0ret->size =3D size;
=C2=A0 =C2=A0ret->prev =3D prev;
=C2=A0#ifndef __MINGW32__
+=C2=A0 int flags =3D MAP_PRIVATE | MAP_ANONYMOUS;
+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+=C2=A0 flags |=3D MAP_JIT;
+#endif
=C2=A0 =C2=A0ret->base =3D mmap (NULL, ret->size,
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0PROT_EXEC | PROT_READ | PROT_WRITE,
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 MAP_= PRIVATE | MAP_ANONYMOUS, -1, 0);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 flag= s, -1, 0);
=C2=A0 =C2=A0if (ret->base =3D=3D MAP_FAILED)
=C2=A0 =C2=A0 =C2=A0{
=C2=A0 =C2=A0 =C2=A0 =C2=A0perror ("allocating JIT code buffer failed&= quot;);
@@ -1406,11 +1414,21 @@ emit_code (scm_jit_state *j, void (*emit) (scm_jit_= state *))

=C2=A0 =C2=A0 =C2=A0 =C2=A0uint8_t *ret =3D jit_address (j->jit);

+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+=C2=A0 =C2=A0 =C2=A0 pthread_jit_write_protect_np(0);
+#endif
+
=C2=A0 =C2=A0 =C2=A0 =C2=A0emit (j);

=C2=A0 =C2=A0 =C2=A0 =C2=A0size_t size;
=C2=A0 =C2=A0 =C2=A0 =C2=A0if (!jit_has_overflow (j->jit) && jit= _end (j->jit, &size))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{
+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /* protect previous code arena. leave u= nprotected after emit()
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0since jit_end() also write= s to code arena. */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pthread_jit_write_protect_np(1);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sys_icache_invalidate(arena->base, a= rena->size);
+#endif
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ASSERT (size <=3D (arena->si= ze - arena->used));
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0DEBUG ("mcode: %p,+%zu\n"= ;, ret, size);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0arena->used +=3D size;
@@ -1424,6 +1442,11 @@ emit_code (scm_jit_state *j, void (*emit) (scm_jit_s= tate *))
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0}
=C2=A0 =C2=A0 =C2=A0 =C2=A0else
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{
+#if defined __APPLE__ && HAVE_PTHREAD_JIT_WRITE_PROTECT_NP
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 /* protect previous code arena */
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pthread_jit_write_protect_np(1);
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sys_icache_invalidate(arena->base, a= rena->size);
+#endif
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0jit_reset (j->jit);
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (arena->used =3D=3D 0)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0{
--
2.37.1 (Apple Git-137.1)

--0000000000001edf0305f01f948f--