From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Nala Ginrut Newsgroups: gmane.comp.gcc.jit,gmane.lisp.guile.user Subject: Re: AOT compiler (was: Running Compiled Guile Objects) Date: Sun, 15 Dec 2024 10:59:28 +0900 Message-ID: References: <769073d434c2ed5fb7937c85da240aa5df4d854a.camel@starynkevitch.net> <49d9827f4455076cc066add3e51f0e882b59e9b7.camel@starynkevitch.net> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000a8dc540629456adb" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31797"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Maxime Devos , team-rHw2apc0TIwgsBAKwltoeQ@public.gmane.org, jit-/MQLu3FmUzdAfugRpC6u6w@public.gmane.org, "dmalcolm-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org" , basile-VdE74OAlGqnvXjIo7pOF+l6hYfS7NtTn@public.gmane.org To: Guile User Original-X-From: jit-bounces~gcgj-jit=m.gmane-mx.org-/MQLu3FmUzdAfugRpC6u6w@public.gmane.org Sun Dec 15 03:00:09 2024 Return-path: Envelope-to: gcgj-jit@m.gmane-mx.org Original-Received: from server2.sourceware.org ([8.43.85.97]) by ciao.gmane.io with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tMdvU-00085G-JV for gcgj-jit@m.gmane-mx.org; Sun, 15 Dec 2024 03:00:08 +0100 Original-Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 604B73858CDB for ; Sun, 15 Dec 2024 02:00:07 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 604B73858CDB Authentication-Results: sourceware.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=UpgTgM+x Original-Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by sourceware.org (Postfix) with ESMTPS id 9952C3858D21 for ; Sun, 15 Dec 2024 01:59:40 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 9952C3858D21 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=gmail.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 9952C3858D21 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=2607:f8b0:4864:20::102e ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1734227980; cv=none; b=sz8SePUxknuNHrfFR0C3Tlx2KyoGgXQAv0JXlWj2RYIDa8FB6NvjGScV7WUHGgNr38olAbXqHtOLVOd81XdXgorAJIj7/g6m3havPESkQGsfeF4G3E2zibPQCqk94/3K57o+zZzstR1XKstjTUDvLn/lH6a6foMvUArY3+4x10o= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1734227980; c=relaxed/simple; bh=IvJHqFHIAskKAHLi9PaDRflcFR3Ikz/coyZThq8jtA0=; h=DKIM-Signature:MIME-Version:From:Date:Message-ID:Subject:To; b=gP0FCk26upiezXgXKgzDvgPJ3FUyRO4KBkEooajQAtBBtYCwfP++cMqJ3OBbaQnpKCgOhu72i6QJW8hB2nvzGAKJC2ctGGQQP52DRGU6iymXBpXHLiguLzoDWB2ylBKMyZXAuPaD8Mlb6JMhsWy2eZUcLP4gJc6DWvAJ/fDLCsQ= ARC-Authentication-Results: i=1; server2.sourceware.org DKIM-Filter: OpenDKIM Filter v2.11.0 sourceware.org 9952C3858D21 Original-Received: by mail-pj1-x102e.google.com with SMTP id 98e67ed59e1d1-2efded08c79so2039601a91.0 for ; Sat, 14 Dec 2024 17:59:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734227979; x=1734832779; darn=gcc.gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=vuomIeIL3KOIlQCOmCV3r/H3oZ0m9/LNTwWcq6B8St8=; b=UpgTgM+x4WpK4zex+XMbYu44z9L47Zojsa71XqgWq6Nx5KLqUccXcyl11+yDfJ9a1/ 61cxXJJ6hE/6+jONQbG7MoLHIgWN3PHxTpoeykmO2M+ojwhKqbVttAY+Me63xVG5CiUr SkNMAMklzqQBYQ28Mgkd3Kupkn2ztMjE6kHEoDRbzM3AqKssp51H/81Kj2pauwOy9VMD usgfU4hbytR6vzYqHTaWCtmqsDEFN5ZezyrnQ5TPE0N3oq0Sz7uGXcIVNz0SIa84ezE3 KvKFO1KkqcO5+jNgbm5f2So96IJrFrEYsb0CoyxnpilWWIcxuDtNYtNkBSMg7wxV8+ZD lFag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734227979; x=1734832779; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vuomIeIL3KOIlQCOmCV3r/H3oZ0m9/LNTwWcq6B8St8=; b=dQWip0vbE3mjqmbaczgplPgfeYxVu92m+rOkE4Dw+5XaAVSzeDI4wXPEZKOSJUnTU2 XQGuLR9oBv/AV7M0W9avGxDmGFoUrP0rm5rinOGp29Cnc7try2YA+1CzJ6NxJj2pFYHt SvgASfNGyfLvaURB158TH79NKKT36UerIgISUA3QED48qi6CQn35mTpGDX6PcnL7+4bK 976w9bt6YHqdMAOSP5CEKYKSxk/U7bSuSSiWM/1YFh9EcRe65Jf3IhFzgecQ36+UzCJe lbol3DTb0aHnn+epYlSe9sZtwxH4Ws7BX/89Hv0mPLlmi795EoZRdWm+nTZyKjJp5gVb kVRA== X-Forwarded-Encrypted: i=1; AJvYcCUvQa/+awlw/xWfeItjse+rPWEgra+yc9obqckZLUeQqO89qrvb/fGEtjWQWOYgxWodqgs=@gcc.gnu.org X-Gm-Message-State: AOJu0YxD3e2slitZkH40A0zdQ11JPQYI/ClG/4OuuAOZhdbg7pY7ycpd P4Wn53GNOS9n8RXQwHyaCE4sxiiISKFDbECOBPoZs9zey+4D7MO5+6YLlca4HkN4upi3YjMgOkM DBS6B+AlDTMzyVEArkEVbLVhSw/Q= X-Gm-Gg: ASbGncuXT9VBjkvOkP1fsjtPbABdrukWJ7tUk7jfgf484vJqMx+WzUsncrcQE1u1qqQ YkZzXqjPa2PCQ13GACwBsPmomlOabtzN3OehCew== X-Google-Smtp-Source: AGHT+IHu45ttZUw0X9rBWDh5qN97EP2UD3MMAWQbq6XFCRsu3iltPPvl2lEFt/GIVV0l3dTngSh9XpuczErU/WO5rk8= X-Received: by 2002:a17:90b:43:b0:2ea:aa56:499 with SMTP id 98e67ed59e1d1-2f28fa55105mr11410745a91.1.1734227979195; Sat, 14 Dec 2024 17:59:39 -0800 (PST) In-Reply-To: X-BeenThere: jit-/MQLu3FmUzdAfugRpC6u6w@public.gmane.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Jit mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: jit-bounces~gcgj-jit=m.gmane-mx.org-/MQLu3FmUzdAfugRpC6u6w@public.gmane.org Xref: news.gmane.io gmane.comp.gcc.jit:1947 gmane.lisp.guile.user:20001 Archived-At: --000000000000a8dc540629456adb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi folks! Please reply AOT topic in this thread. > Indeed, it turns out that everyone using libgccjit is using it for ahead-of-time compilation, rather than jit-compilation. Sorry about picking a bad name :) Thanks for the work! At least in Guile community, now that we already have JIT with GNU Lightening, we won't redundantly talk libgccjit for JIT. But I think it's still interesting to try libgccjit for JIT in my other project. :-p > Probably of most interest to guile developers might be the gccemacs project here: https://akrl.sdf.org/gccemacs.html It looks super interesting! They created a high level IR just like GIMPLE, which named LIMPLE. It's the way I preferred in streaming-fist. The basic idea is to provide such an IR, then try to replace 'bytecode' in Guile compiler tower codegen. And they choose SBCL for it, I think it can inspire people who are interested in Guile AOT. Or maybe they could consider to change to Guile to save a lot of time for both Guile and Emacs. The compiler tower is so flexible that it's possible to try it as a plugin. I agreed with @mikael about losing weight of Guile. The AOT feature can be a plugin to install as guild tools. Of course, plugin unnecessarily mean it will be small...... Best regards. On Sun, Dec 15, 2024, 02:45 Nala Ginrut wrote: > Thanks for the inspiring sharing! =F0=9F=91=8F > > On Sun, Dec 15, 2024, 02:39 Basile Starynkevitch > wrote: > >> On Sun, 2024-12-15 at 02:21 +0900, Nala Ginrut wrote: >> >> @basile I'm glad you raise this topic, I've played lobgccjit with a toy >> project. >> https://gitlab.com/hardenedlinux/screaming-fist >> >> I would say libgccjit is a wrong name since it's more like a tool for AO= T. >> Of course, one may still use it for JIT, however you have to do your own >> work for JIT and finally use libgccjit for codegen. =F0=9F=98=81 >> >> Best regards. >> >> >> You basically are right. If you want to use get a fast just-in-time >> compilation, libgccjit might not be the right tool. >> >> But in practice, current computers are so fast that I think that in >> practice libgccjit is quite usable, and it can be tuned to various GCC >> optimization strategy. >> >> A few years ago I did experiment (see https://arxiv.org/abs/1109.0779 ..= .) >> generation of C++ code which (on Linux desktop) was GCC compiled to a >> plugin and dlopen-ed. This works quite well with an elapsed time suitabl= e >> for human users. >> >> A related experiment is my manydl.c thing on >> https://github.com/bstarynk/misc-basile - it is/was a toy program that I >> wrote which generates a lot of random C code (in many thousands of C >> files), compile it to a plugin, and use dlopen and dlsym. It shows that = on >> Linux a process can successfully dlopen do many hundred thousands of >> plugins (and never bother dlclose-ing them) >> >> BTW in France the Lisp syntax and Scheme semantics of GNU guile is sadly >> becoming impopular. I know few persons using it. >> >> Just in case I am attaching a few PDF files on RefPerSys. >> >> Some ideas of RefPerSys originated from books and papers by by Jacques >> Pitrat >> https://en.wikipedia.org/wiki/Jacques_Pitrat >> >> Please mention RefPerSys to your colleagues and forward them this email. >> >> Thanks >> >> >> >> ---------- Forwarded message --------- >> From: Basile Starynkevitch >> Date: Sun, Dec 15, 2024, 02:11 >> Subject: Re: Running Compiled Guile Objects >> To: Nala Ginrut , Hakan Candar < >> hakancandar-g/b1ySJe57IN+BqQ9rBEUg@public.gmane.org> >> Cc: guile-user-mXXj517/zsQ@public.gmane.org , >> >> >> On Sat, 2024-12-14 at 09:15 +0900, Nala Ginrut wrote: >> > Hi Hakan! >> > The current Guile is not AOT yet. Although the object file is ELF, >> > it's >> > just bytecode wrapped ELF header. So you can't run it as a regular >> > executable file. >> > >> >> Those willing to contribute a proper ahead-of-time compiler to GNU >> guile could use the GNU CC libgccjit library which is part of the GCC >> compiler. >> https://gcc.gnu.org/onlinedocs/jit/ >> >> This libgccjit layer of the GCC compiler is stable and maintained C API >> and has some obsolete C++ API (which seems unmaintained in december >> 2024). Most of the libgccjit code is internally coded (under GPL >> license) in C++, but the stable API is for C. >> >> I am using the C API of libgccjit in the RefPerSys open source >> inference engine project (GPLv3+ licensed) on >> https://github.com/RefPerSys/RefPerSys/ >> >> Both libgccjit and GNU lightning (see >> https://www.gnu.org/software/lightning/ ...) could be a basis for >> adding a genuine compilation layer to GNU guile. And RefPerSys uses >> both. >> >> I guess a significant issue would be to use libgccjit (or GNU >> lightning) with GUILE's garbage collector (which seems to be Boehm >> conservative GC). >> >> The RefPerSys project has a precise garbage collector and some >> persistence (to textual files). Since it is GPLv3+ licensed, its code >> could be reused in a future GUILE major version. RefPerSys is mostly >> coded in C++. >> >> I did contribute to GCC long time ago and hope that RefPerSys could >> become a GNU project (but don't know how to make that happen) >> >> >> -- >> >> Basile STARYNKEVITCH 8 rue de la Fa= =C3=AFencerie >> 92340 Bourg-la-Reine, France http://starynkevitch.net/Basile & https://github.com= /bstarynk >> >> --000000000000a8dc540629456adb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

Hi folks! Please reply AOT topic in this t= hread.


> Indeed, it turns out t= hat everyone using libgccjit is using it for
ahead-of-time compilation, rather than jit-compilation.=C2=A0 Sorry about picking a bad name :)

Thanks for the work!=C2=A0

At least in Guile community, now that we already have JIT with GN= U Lightening, we won't redundantly talk libgccjit for JIT. But I think = it's still interesting to try libgccjit for JIT in my other project. :-= p

> Probably of most interest to guile developers might be = the gccemacs
project here:
=C2=A0 https://akrl.sdf.org/gccemacs.html


It looks super interesting!

They created a high level IR just like GIMPLE, which named LIMPLE. It= 9;s the way I preferred in streaming-fist. The basic idea is to provide suc= h an IR, then try to replace 'bytecode' in Guile compiler tower cod= egen. And they choose SBCL for it, I think it can inspire people who are in= terested in Guile AOT. Or maybe they could consider to change to Guile to s= ave a lot of time for both Guile and Emacs.

The compiler = tower is so flexible that it's possible to try it as a plugin.=C2=A0

I agreed with @mikael about losing weight of Guile. The AOT= feature can be a plugin to install as guild tools. Of course, plugin unnec= essarily mean it will be small......


Best regards.

On Sun, Dec 15, = 2024, 02:45 Nala Ginrut <nalaginrut-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote= :

Thanks for the ins= piring sharing! =F0=9F=91=8F


On Sun= , Dec 15, 2024, 02:39 Basile Starynkevitch <= basile-VdE74OAlGqnvXjIo7pOF+l6hYfS7NtTn@public.gmane.org> wrote:
On Sun, 2024-12-15 at 02:21 +0900, Nala Ginrut wrote:
<= blockquote type=3D"cite" style=3D"margin:0 0 0 .8ex;border-left:2px #729fcf= solid;padding-left:1ex">
@basile I'm glad you ra= ise this topic, I've played lobgccjit with a toy project.

I would say libgccjit is a wrong name since it's mo= re like a tool for AOT.
Of course, one may still use= it for JIT, however you have to do your own work for JIT and finally use l= ibgccjit for codegen. =F0=9F=98=81

Best regards.

You ba= sically are right. If you want to use get a fast just-in-time compilation, = libgccjit might not be the right tool.

But in prac= tice, current computers are so fast that I think that in practice libgccjit= is quite usable, and it can be tuned to various GCC optimization strategy.=

A few years ago I did experiment (see=C2=A0https://arxiv.org/abs/1109.0779=C2=A0.= ..) generation of C++ code which (on Linux desktop) was GCC compiled to a p= lugin and dlopen-ed. This works quite well with an elapsed time suitable fo= r human users.

A related experiment is my manydl.c= thing on=C2=A0https://githu= b.com/bstarynk/misc-basile=C2=A0- it is/was a toy program that I wrote = which generates a lot of random C code (in many thousands of C files), comp= ile it to a plugin, and use dlopen and dlsym. It shows that on Linux a proc= ess can successfully dlopen do many hundred thousands of plugins (and never= bother dlclose-ing them)

BTW in France the Lisp s= yntax and Scheme semantics of GNU guile is sadly becoming impopular. I know= few persons using it.

Just in case I am attaching= a few PDF files on RefPerSys.

Some ideas of RefPe= rSys originated from books and papers by by Jacques Pitrat=C2=A0

Please mention RefPerSys= to your colleagues and forward them this email.

T= hanks



---------- Forwarded message ---------
From: Basile Starynkevitch = <basile@staryn= kevitch.net>
Date: Sun, Dec 15, 2024, 02:11
Subject: Re= : Running Compiled Guile Objects
To: Nala Ginrut <nalaginrut-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>, Hakan Candar <hakancandar-g/b1ySJe57IN+BqQ9rBEUg@public.gmane.org>
Cc: guile-user-mXXj517/zsQ@public.gmane.org <guile-user-mXXj517/zsQ@public.gmane.org>, <jit-/MQLu3FmUzdAfugRpC6u6w@public.gmane.org>


On Sat, 2024-12-14 at 09:15 +090= 0, Nala Ginrut wrote:
> Hi Hakan!
> The current Guile is not AO= T yet. Although the object file is ELF,
> it's
> just bytec= ode wrapped ELF header. So you can't run it as a regular
> execut= able file.
>

Those willing to contribute a proper ahead-of-ti= me compiler to GNU
guile could use the GNU CC libgccjit library which is= part of the GCC
compiler.
https://gcc.gnu.org/onlinedocs/jit/

This = libgccjit layer of the GCC compiler is stable and maintained C API
and h= as some obsolete C++ API (which seems unmaintained in december
2024). Mo= st of the libgccjit code is internally coded (under GPL
license) in C++,= but the stable API is for C.

I am using the C API of libgccjit in t= he RefPerSys open source
inference engine project (GPLv3+ licensed) onhttp= s://github.com/RefPerSys/RefPerSys/

Both libgccjit and GNU light= ning (see
https://www.gnu.org/software/lightning/ ...) could be a basis fo= r
adding a genuine compilation layer to GNU guile. And RefPerSys usesboth.

I guess a significant issue would be to use libgccjit (or GNU=
lightning) with GUILE's garbage collector (which seems to be Boehm<= br>conservative GC).

The RefPerSys project has a precise garbage col= lector and some
persistence (to textual files). Since it is GPLv3+ licen= sed, its code
could be reused in a future GUILE major version. RefPerSys= is mostly
coded in C++.

I did contribute to GCC long time ago an= d hope that RefPerSys could
become a GNU project (but don't know how= to make that happen)

-- 
Basile STARYNKEVITCH           <basile-VdE74OAlGqnvXjIo7pOF+l6hYfS7NtTn@public.gmane.org>
8 rue de la Fa=C3=
=AFencerie
92340 Bourg-la-Reine,          France
http://starynkevitch.net/Basile =
& https://github.com/bstarynk=20
--000000000000a8dc540629456adb--