From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Kevin Ryde Newsgroups: gmane.lisp.guile.devel Subject: Re: Evolution & optimization of the module system Date: Mon, 19 Feb 2007 10:32:29 +1100 Message-ID: <877iufhwj6.fsf@zip.com.au> References: <87sld4g6io.fsf@chbouib.org> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Trace: sea.gmane.org 1171841641 341 80.91.229.12 (18 Feb 2007 23:34:01 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Sun, 18 Feb 2007 23:34:01 +0000 (UTC) Cc: guile-devel@gnu.org To: ludo@chbouib.org (Ludovic =?iso-8859-1?Q?Court=E8s?=) Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Mon Feb 19 00:33:53 2007 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1HIvXT-0002Ky-A8 for guile-devel@m.gmane.org; Mon, 19 Feb 2007 00:33:51 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1HIvXS-00021s-OR for guile-devel@m.gmane.org; Sun, 18 Feb 2007 18:33:50 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1HIvXM-00021e-Kd for guile-devel@gnu.org; Sun, 18 Feb 2007 18:33:44 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1HIvXL-00021S-18 for guile-devel@gnu.org; Sun, 18 Feb 2007 18:33:44 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1HIvXK-00021P-Qw for guile-devel@gnu.org; Sun, 18 Feb 2007 18:33:42 -0500 Original-Received: from mailout1-8.pacific.net.au ([61.8.2.215] helo=mailout1.pacific.net.au) by monty-python.gnu.org with esmtp (Exim 4.52) id 1HIvXJ-0003Ba-It for guile-devel@gnu.org; Sun, 18 Feb 2007 18:33:42 -0500 Original-Received: from mailproxy1.pacific.net.au (mailproxy1.pacific.net.au [61.8.2.162]) by mailout1.pacific.net.au (Postfix) with ESMTP id C04315A0D30; Mon, 19 Feb 2007 10:33:30 +1100 (EST) Original-Received: from localhost (ppp2349.dyn.pacific.net.au [61.8.35.73]) by mailproxy1.pacific.net.au (Postfix) with ESMTP id 8FC408C14; Mon, 19 Feb 2007 10:33:23 +1100 (EST) Original-Received: from gg by localhost with local (Exim 4.63) (envelope-from ) id 1HIvWA-00028J-Hb; Mon, 19 Feb 2007 10:32:30 +1100 Mail-Copies-To: never In-Reply-To: <87sld4g6io.fsf@chbouib.org> (Ludovic =?iso-8859-1?Q?Court=E8?= =?iso-8859-1?Q?s's?= message of "Sat, 17 Feb 2007 16:15:11 +0100") User-Agent: Gnus/5.110006 (No Gnus v0.6) Emacs/21.4 (gnu/linux) X-detected-kernel: Linux 2.6, seldom 2.4 (older, 4) X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:6534 Archived-At: ludo@chbouib.org (Ludovic Court=E8s) writes: > > (1) has to do mainly with `module-use!' vs. `module-use-interfaces!' (as > was discussed recently). Namely the fact that duplicate processing is > not always performed, depending on whether one uses `module-use!' or > some other means to use a module. There's nothing broken though is there, it's just a trap for the unwary (like me for instance, wrestling with use-srfis)? > There's still more to do to achieve (2) (notably actual documentation > ;-)) I've put that off, because a lot looks like internals, and I'd wondered if r6rs might offer some of its own "introspection" stuff. > The patch solves this issue by making > duplicate processing inescapable. Likewise, variable lookup currently > has two implementations (which have the same behavior, though): the C > `module_variable ()' and the Scheme `module-variable'. The patch leaves > only one implementation of that. Moving stuff into C is a good thing. You can propose that separately if you like, it should be able to go straight in. (And could be a candidate for the 1.8 branch if the performance help is noticable.) > Although duplicates should be the exception rather than the rule[*], > duplicate processing is pretty costly: the current `process-duplicates' > is roughly O(N*USES), USES is supposed to be smallish of course though. > Likewise, > variable lookup (e.g., in `module_variable ()') is O(USES). I believe > that both may have a sensible impact on startup time. Something definitely slows down startup over 1.6. I'd guessed it was strings, but I failed miserably at making either gprof or functioncheck give some call counts to prove or disprove that. > The patch addresses this by changing the data structures used by > modules: instead of a list of used modules, it uses a second "obarray", > called the "import obarray", that maps symbols to the modules providing > them. That would increase the memory used by a module though, would it? Big modules imported into lots of other places would have their binding list much copied would they? (I'm thinking for instance of gtk and gnome, and thinking purely selfishly since I import gtk and/or gdk into a bunch of my modules.) > This has several implications. First, duplicate processing occurs the > same way for dynamically added bindings than for "statically imported" > bindings. Second, it makes load-time-dependent duplicate policies such > as `last' and `first' irrelevant (since they are inherently > non-deterministic). Last and first still sound pretty sensible. > Third, it makes dynamic addition of bindings relatively costly. Depends how often that happens I suppose. There's no actual documented way to do it is there? :-) Maybe some tomfoolery with macros and eval ... > (2) the > SMOB classes are added to a separate module called `(oop goops > smob-classes)'. Since only `(oop goops)' uses it, it is the only one > that needs to re-process duplicates as new SMOB classes are added. Sounds ok in principle. > From the measurements I've made, the new version is > around 40 times faster than the other one. Is that in part due to moving the variable lookup to C? > So the question is: is the `beautify-user-module!' overhead compensated > by the variable lookup and duplicate processing gains? I'd be inclined to say probably not, but that maybe there's another way. Duplicates is trying to find the intersection between one set of symbols (from one module) and some reasonably smallish number N of other sets (from other modules) is it? I wonder if there's some way to find that more efficiently. I guess the `warn' in the default duplicates handling makes it necessary to find all the duplicates. Nice enough feature, but without it there'd be no need to check any duplicates at all, or only check against the few `#:replace' lists. Maybe the worst afflicted programs should be advised to turn it off, or even turn it off globally when confident there's no accidental clashes. And for a start I guess we don't need any duplicates work in the core, at least up until reaching the `--use-srfi' stage of startup, if that isn't already the case. > + (define %default-import-size > + ;; This should be the size of the pre-module obarray. > + 500) Which also ends up being a minimum size of course ... > +/* Copy the given alist, i.e., duplicate all its pairs recursively. */ > +static inline SCM > +alist_copy (SCM alist) If you end up using this you could move scm_srfi1_alist_copy from srfi/srfi-1.c into the core (leaving behind a #define and a re-export). > -/* > - TODO: should export this function? --hwn. > - */ > -static SCM > -scm_export (SCM module, SCM namelist) > +SCM > +scm_module_export (SCM module, SCM namelist) That could be an "scm_i_" (for now at least) if it's not documented. _______________________________________________ Guile-devel mailing list Guile-devel@gnu.org http://lists.gnu.org/mailman/listinfo/guile-devel