From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Ken Raeburn Newsgroups: gmane.emacs.devel Subject: Re: advice needed for multi-threading patch Date: Wed, 26 Aug 2009 14:50:38 -0400 Message-ID: <5C8D2D84-C4B5-47A2-8392-370D7C3651CF@raeburn.org> References: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 (Apple Message framework v936) Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1251314896 12900 80.91.229.12 (26 Aug 2009 19:28:16 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 26 Aug 2009 19:28:16 +0000 (UTC) Cc: Emacs development discussions To: Tom Tromey Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Aug 26 21:28:09 2009 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1MgOAC-0002pV-Is for ged-emacs-devel@m.gmane.org; Wed, 26 Aug 2009 21:28:09 +0200 Original-Received: from localhost ([127.0.0.1]:46012 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MgOAB-0002kJ-VC for ged-emacs-devel@m.gmane.org; Wed, 26 Aug 2009 15:28:08 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1MgNaA-0003Mz-NG for emacs-devel@gnu.org; Wed, 26 Aug 2009 14:50:54 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1MgNa7-0003KR-0q for emacs-devel@gnu.org; Wed, 26 Aug 2009 14:50:54 -0400 Original-Received: from [199.232.76.173] (port=60592 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MgNa6-0003KJ-Nt for emacs-devel@gnu.org; Wed, 26 Aug 2009 14:50:50 -0400 Original-Received: from splat.raeburn.org ([69.25.196.39]:62178 helo=raeburn.org) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1MgNZy-0001ji-CM for emacs-devel@gnu.org; Wed, 26 Aug 2009 14:50:50 -0400 Original-Received: from [10.0.0.172] (squish.raeburn.org [10.0.0.172]) by raeburn.org (8.14.3/8.14.1) with ESMTP id n7QIob5Y014408; Wed, 26 Aug 2009 14:50:39 -0400 (EDT) In-Reply-To: X-Mailer: Apple Mail (2.936) X-detected-operating-system: by monty-python.gnu.org: Genre and OS details not recognized. X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:114624 Archived-At: On Aug 26, 2009, at 12:08, Tom Tromey wrote: > Ken> I hope you and Daniel and I aren't doing redundant work. > > Unless your changes apply to Emacs I don't think they are really > redundant. I think the key thing is agree how it ought to work, so we > end up with the same model, but I don't anticipate any problems there. Well, in their current form, my Emacs changes are kind of ugly, so they won't be applied in their current form. Some of them are hacks to get things working, so we can (eventually) evaluate the feasibility of merging in Guile for real. It's not that I don't want to fix things properly; but fixing some of them properly requires either large-scale divergence from the Emacs repository -- in small but pervasive and less well automated ways -- or changes to how things are done in the Emacs repository that can't be enforced very cleanly, and which my only argument for is, "well, my Guile work that may or may not ever get folded in is going to need it." The big issue -- the *huge* one -- is that all-bits-zero, the default C initializer for a static Lisp_Object and the result of bzero, is an integer-zero in Emacs currently and an invalid object representation in Guile. There are lots of places where such values get used, or Lisp_Object variables or fields are made visible to the GC system before being properly "initialized" with valid representations. I could go change all of the places like that to first initialize a location with a valid object before using it, but I don't see how to tweak the non-Guile version to help me catch such cases without, say, changing Lisp_Int to use a nonzero tag, which would affect performance (how much?), and would stop XINT/XUINT from being a shortcut for "gimme all the bits" in the hash code and perhaps a few other places I haven't tracked down yet. Or, I could change things like staticpro to first store a valid object value, which then requires that the assignment of the desired value in the various init_foo functions *must* happen after the staticpro call, whereas now the ordering is inconsistent. That might be more enforceable, because I can make staticpro and the like complain if they see anything other than all-bits-zero, which implies that the code tried to store the desired value (other than integer-zero) first. I'd still have to patch up anything using bzero to follow it up with explicit initialization of some fields. It's a bit annoying that the unexec setup basically prohibits the use of static initializers for Lisp_Object variables because they might become read-only; otherwise static initialization of V* and Q* variables with a macro expanding to a valid object representation would be an option. I was figuring on pushing this work a bit further along, and having something I could show off better, to make a stronger case for integrating the guile-emacs work, before trying to argue for such changes in the main Emacs code base. (Even if the decision eventually went against integrating Guile specifically, I might be able to make a case for going through a little pain to better isolate the Lisp engine so it could perhaps be replaced someday by something else.) And in the meantime, I'd prefer to keep it easier to resync with upstream changes, by not rewriting all that code, just as long as I know roughly what needs to be fixed later. But, I digress. :-) My own changes are, in part, to insulate the C Lisp implementation in Emacs as much as possible from the rest of Emacs, and in part, to replace the most fundamental layer -- representation, allocation and GC -- with Guile. Whether this project eventually leads to replacing the Lisp evaluator itself, I don't know; some people seem to think it would be a great idea, and some seem to think it would be a bad idea. I'm on the fence until I can see some performance numbers and serious testing, and that requires a more or less complete implementation; there's been some encouraging work happening on the Guile side, but it's not done yet. But it's appealing to think we might someday be able to have elisp available as an extension language for other programs, via Guile; and if that happens, do we really need two implementations? (And if we go with only one for both Emacs and Guile, would it be the current Guile one, or taken from Emacs and dropped into Guile? Dunno.) I'm not worrying about things like multithreading, but my changes will need to ensure that symbol value lookups and updates are sufficiently isolated through macros or functions that it should be easy to slip in thread-specific processing. Daniel Kraft and others are working on the Guile elisp implementation, which in part requires learning the specs of how elisp functions in various corner cases as well as normal usage. There has been some discussion of multithreading, but as far as I know there's currently no implementation of buffer-local variables. Daniel's work is using Guile's string and symbol representations. I started with the "smob" application-defined-object handles so I could store pointers to Lisp_String and Lisp_Symbol and so forth, and not have to rewrite all that code right away; that's coming up. So there's a bit of work needed before Daniel's work and mine can possibly work together. > Ken> I've written similar elisp code for doing transformations of > patterns > Ken> within the Emacs code -- mostly more localized things like "XCONS > Ken> (expr)->car" to "XCAR (expr)", but with support for somewhat > complex > Ken> values of "expr" that have to have balanced parens, etc. > > If you still have these, I would appreciate a copy, so that I don't > have > to reinvent them. Are they in your git repository? No, but I've got them floating around somewhere, and I'll send you a copy. They're not pretty, but they get the job done. > I will have to think about this. This would imply removing all those > slots from struct buffer; I suppose my concern would be that the > result > would be too inefficient. Not necessarily. With Lisp_Misc_Buffer_Objfwd, one of the existing fields is the offset into the buffer structure. Maybe that could be where the thread-local object handle is stored. > I've also been considering a similar thing for the Vfoo globals: > remove > them and have the C code always refer to them via SYMBOL_VALUE. I think that might also simplify some things for my work. Ken