From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Wojciech Meyer Newsgroups: gmane.emacs.devel Subject: Re: Compiling Elisp to a native code with a GCC plugin Date: Tue, 14 Sep 2010 23:37:50 +0100 Message-ID: <87bp802bpd.fsf@gmail.com> References: <87bp805ecr.fsf@gmail.com> <874ods5ctf.fsf@gmail.com> <877hio3urh.fsf@gmail.com> <87wrqo2ev4.fsf@gmail.com> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1284503939 31879 80.91.229.12 (14 Sep 2010 22:38:59 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 14 Sep 2010 22:38:59 +0000 (UTC) Cc: Wojciech Meyer , emacs-devel@gnu.org To: Tom Tromey Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Sep 15 00:38:57 2010 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Ove9P-0002yr-B9 for ged-emacs-devel@m.gmane.org; Wed, 15 Sep 2010 00:38:55 +0200 Original-Received: from localhost ([127.0.0.1]:57786 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Ove9O-0002GP-D7 for ged-emacs-devel@m.gmane.org; Tue, 14 Sep 2010 18:38:54 -0400 Original-Received: from [140.186.70.92] (port=40391 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Ove9H-0002El-3p for emacs-devel@gnu.org; Tue, 14 Sep 2010 18:38:48 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1Ove9F-00084i-Fo for emacs-devel@gnu.org; Tue, 14 Sep 2010 18:38:47 -0400 Original-Received: from mail-ww0-f49.google.com ([74.125.82.49]:41164) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Ove9F-00084X-73 for emacs-devel@gnu.org; Tue, 14 Sep 2010 18:38:45 -0400 Original-Received: by wwb24 with SMTP id 24so8758292wwb.30 for ; Tue, 14 Sep 2010 15:38:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:subject:references :date:in-reply-to:message-id:user-agent:mime-version:content-type; bh=B66SqkTBLTImgFCNI08gAgH92INqcdgMc469tjahCOY=; b=O0ZvGwHLJ1ghjxIjClKxJNzGd9TwpvGT4cyzoxXLqNvfxoJyThkNmONjdo699iC6mh RxZu65cDnjLq9fgZbigW3vK8rQ1SqKRQv5VgyNVZ3PLoTTzPxw9z+sfBOpqafzrP7rCV HP83IdaQX+lod+0E6OycGQkvNDHUOUuq8PEHQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-type; b=KPT6mR8Qa9jhjCThj03ma/Wykc/tHjvXbPF6lVijc3eSyfwexfOz34LelMowqTHBT3 dJZCcJ9iTitxVD/jmKrwSXNQAnq8iW/zqICKIMZfwyH0U90gk86LgRemNY8ko0/BETR8 1olUcppSW/CDjtCeLgla88wRskkxqyLjC5BVs= Original-Received: by 10.216.232.144 with SMTP id n16mr565962weq.1.1284503921620; Tue, 14 Sep 2010 15:38:41 -0700 (PDT) Original-Received: from spec-desktop.specuu.com (host86-133-35-46.range86-133.btcentralplus.com [86.133.35.46]) by mx.google.com with ESMTPS id u32sm528133weq.35.2010.09.14.15.38.34 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 14 Sep 2010 15:38:40 -0700 (PDT) In-Reply-To: (Tom Tromey's message of "Tue, 14 Sep 2010 15:59:50 -0600") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.0.50 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:130157 Archived-At: Tom Tromey writes: >>>>>> "Wojciech" == Wojciech Meyer writes: > > Tom> So, lots of grunge work, just to get the point where you could start > Tom> actually working on the GC. I would look at automated rewriting to > Tom> make this work -- that worked out great on the concurrent branch. > > Wojciech> Maybe that work should be actually done even without thinking > Wojciech> currently about GC. AFAIR MT Emacs rewriting was in Elisp, > Wojciech> ideally maybe using GCC would be better at some point. > > You would have to hack GCC a little bit, because most of the code > locations you want to change arise from macro expansion, and GCC does > not keep all that information. (Though there's a WIP patch for this.) Yes, I am aware about impreciseness of this. Currently I may not think about this for some other unrelated reasons as well... > > Maybe it could be done more simply using a simple parser in elisp that > recognizes just the needed forms. Or maybe something based on clang. > > This would get you most of the way there, though there are still some > bad things you have to fix up by hand. > > > For the concurrency stuff, we did two kinds of automated rewriting. If you could point me out with the tools you used for this job, I would be grateful, any points to git? > > One was just pure elisp that searched the .c for DEFVAR_LISP and then > made various changes. > > The other one modified the source (in a compile-breaking way), then ran > the compiler, then visited each error to perform a rewrite. This > approach might also work for the GC problem, I am not certain. That is clever and cheap to do, and it is worth to try (and it looks a bit like humans do re-factoring). However clang error messages are currently are more precise than GCC (yes, we can match-replace regex, and it will work fine in most cases). > > These scripts are both in src/ on the concurrency branch. > > One problem with any compiler-based approach is that it only works on > the sources it sees. That is, the not-taken #if branches won't get > rewritten. This argues for trying some kind of custom parser. Yes, this is a major problem, different configurations, different systems, and you are never sure if it will not break something, on some machine (assuming that the changes are required to generate correct code). > > Another problem we ran into is that this approach doesn't work if the > problem code itself appears in a macro. There were a few spots that we > had to fix by hand -- no big deal, the automation is still worthwhile > even if it only does 85% of the work. Definitely it is worthwhile, however I need to know what to rewrite at first... > > > My advice is to try to do this bulk rewriting work on head, so that it > doesn't rot. I think that's been a problem for the concurrency work > :-( Any chances to get it back to life? It would be nice to push it back... If you would like to get it back, I would volunteer with any help. If the maintainers are happy to accept patches with incremental improvements then it is OK, however I still think in terms `I want - but it will be hard and even nobody started it before'. If there will be any progress on this I will just let everybody know. (anyway starting such project is even OK for just *learning* about internals). Shall we setup a public repo somewhere then? I can commit there my generational gc as a sub-module, straight away. (however it is not the best starting point, as was said). > > Tom Thanks, Wojciech