From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: "Stephen J. Turnbull" Newsgroups: gmane.emacs.devel Subject: Re: `make' written in elisp Date: Mon, 03 Jan 2005 20:05:43 +0900 Organization: The XEmacs Project Message-ID: <87k6qu4vso.fsf@tleepslib.sk.tsukuba.ac.jp> References: NNTP-Posting-Host: deer.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: sea.gmane.org 1104750587 15656 80.91.229.6 (3 Jan 2005 11:09:47 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Mon, 3 Jan 2005 11:09:47 +0000 (UTC) Cc: schierlm@gmx.de, rms@gnu.org, emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Jan 03 12:09:35 2005 Return-path: Original-Received: from lists.gnu.org ([199.232.76.165]) by deer.gmane.org with esmtp (Exim 3.35 #1 (Debian)) id 1ClQ5e-0006ib-00 for ; Mon, 03 Jan 2005 12:09:34 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.33) id 1ClQGn-0000AB-UN for ged-emacs-devel@m.gmane.org; Mon, 03 Jan 2005 06:21:05 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.33) id 1ClQDW-0008Fu-1s for emacs-devel@gnu.org; Mon, 03 Jan 2005 06:17:43 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.33) id 1ClQDS-0008Et-Ug for emacs-devel@gnu.org; Mon, 03 Jan 2005 06:17:39 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.33) id 1ClQDS-0008Dp-Rd for emacs-devel@gnu.org; Mon, 03 Jan 2005 06:17:38 -0500 Original-Received: from [130.158.98.109] (helo=tleepslib.sk.tsukuba.ac.jp) by monty-python.gnu.org with esmtp (TLSv1:RC4-SHA:128) (Exim 4.34) id 1ClQ2B-0000ud-VV; Mon, 03 Jan 2005 06:06:02 -0500 Original-Received: from steve by tleepslib.sk.tsukuba.ac.jp with local (Exim 4.34) id 1ClQ1w-0000MU-DV; Mon, 03 Jan 2005 20:05:45 +0900 Original-To: Stefan In-Reply-To: (Stefan's message of "Sun, 02 Jan 2005 19:26:41 -0500") User-Agent: Gnus/5.1006 (Gnus v5.10.6) XEmacs/21.5 (chayote, linux) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: main.gmane.org gmane.emacs.devel:31727 X-Report-Spam: http://spam.gmane.org/gmane.emacs.devel:31727 >>>>> "Stefan" == Stefan writes: Stefan> I tend to think that most of the autoconfiguration should Stefan> be done upon use rather than during installation, which Stefan> basically implies "in elisp rather than in autoconf". A word to the wise: This is expensive at startup, and will probably take some effort to shake out. Mike Sperber put in many hours removing unnecessary stats from our naive package location code, despite the very regular layout of our package hierarchy. It's still pretty straightforward, but the efficiency is not an accident. It would theoretically be possible to have a cache which collects all the auto-autoloads, but we felt that was over-engineering; startup is not something that should happen very often in Emacs. dak> a) a directory layout. XEmacs packages have a standard way dak> of placing Elisp, info, data and other files in a layout dak> under the top package directory. XEmacs's layout is inverted from what makes sense for third-party packages: we have a standard GNU Emacs-style tree with lisp/$package, info/$package.info*, etc as needed. Really, however, third-party packages want $package/{lisp,lib-src,etc,...}. The Emacspeak and JDEE maintainers absolutely insist on this, for two examples. XEmacs is moving toward flattening out the core lisp, and eventually inverting the tree for packages. The only exceptions will be package meta-data (manifests used to ensure that package upgrades and removals get rid of everything installed by unwanted packages), and info files. This is probably not an appropriate strategy for Emacs, but you should be aware that XEmacs is not necessarily going to stick with the current structure. dak> In addition there are some standard file names for dak> initializing of packages and installation. In short, this dak> part of the package system is basically just file layout dak> conventions relative to a package-specific top directory. We have auto-autoloads.el, custom-loads.el, and _pkg.el. The first is what it looks like, the second handles attaching Custom's menus and groups and so on, and the third contains meta-data which actually is inserted into auto-autoloads.el, and never actually loaded itself. Stefan> I can't find any documentation about the in-package layout Stefan> used by XEmacs, tho. Anybody knows where it's described? The hierarchy is simple, and not really explicitly documented. There is some information on internal structure, especially for the source trees, in XEmacs Users' Guide (node Packages) and Lispref (node Packaging), see http://www.xemacs.org/Documentation/21.5/html/. There may be some commentary in the Internals Manual in the "Future Work" nodes, but they're quite disorganized, and mostly written by Ben who is not authoritative for the packages. The installed package hierarchy is as described above: $top/{etc,info,lib-src,lisp,man,pkginfo,...} with packages constrained to install into like-named subdirectories, except for $top/info, where the name should be $package.info*, and $top/pkginfo, which contains the MANIFEST.$package files used when uninstalling. We're policy-free with respect to source directory organization. Basically in source the required files are an XEmacs-style Makefile and a metadata file, package-info.in. The three files referred to above are automatically generated. dak> b) version management. Versions must basically be floats, dak> higher versions correspond to later versions, are usually dak> based on some CVS archive and not at all with actual package dak> versions. Uh. Stefan> Right, it seems to be used only to discover whether a Stefan> newer version is available or not. Yeah, we need to do something about that. I don't know what Steve Baur was thinking. dak> c) a package manager that will download and upgrade packages. dak> It might do so in user-specific and system-wide packages. XEmacs's is currently in flux. Its #1 design bug is absence of an HTTP transport, both because it makes EFS a single point of failure, and because of the firewall issue. However, unless there's some easily findable index of URLs for downloadable packages, this is not terribly useful, I think. dak> I am not convinced about the value of the centralized server dak> approach to package management from XEmacs: it does not seem dak> to lend itself Stefan> Since `install' specifically targets external packages, a Stefan> centralized server doesn't make much sense indeed. For Stefan> XEmacs, it was probably a good simplification. The centralized server is _not_ designed as part of the package management system, but as a service provided to the community. Due to the convenience for maintainers who don't necessarily have commit access to the upstream project (assuming it exists), it has become a dependency. Doing something about that would require some effort. There is one point that needs to be mentioned, however, and that is that for complex packages that depend on other packages, the "calling macros from byte-compiled code" kind of bug has dropped from FAQ to fossil status. dak> But I do think that we should try to offer some way of dak> installing external Lisp code if one has acquired it as a zip dak> or tar archive in some manner already. Stefan> Not just tarballs but also single elisp files. Most elisp Stefan> packages are single files. This we really are not set up to handle efficiently. I think we should, so I'll be interested to see how you handle it. Stefan> The DWIM-aspect of `install' (just look for *.el and Stefan> *.info files) only works up to a point. I've bumped into Stefan> problems where some *.el files should be ignored, or other We handle this by having an explicit list as a make variable. Usually the default setting of "all .el files" works fine, of course, and that's easily implemented with `wildcard'. Stefan> problems where the *.info file doesn't have the `info' Stefan> extension, or where the info file simply doesn't exist and Stefan> it's not clear how to build it based on a bunch of *.texi Stefan> files, ... Yep. Info files and package-specific data (eg, pixmaps, the Yow! database, SGML DTDs, JDEE's Java code, etc) are the reason for 1/2 of the hair in the package build infrastructure. Most of the rest has to do with supporting building on Windows. Stefan> We could keep adding intelligence and heuristics, but it Stefan> makes more sense to devise some conventions. I agree in theory, but in practice, you'll run into resistance (ie, they just won't bother) from 3rd-party maintainers, all the more since you're focusing on "external" packages. dak> All of those certainly sound reasonable. In particular, I'd dak> second having separate commands for installing system-wide dak> and user-local packages. Stefan> Agreed, it makes more sense. XEmacs experience with "guessing" the target tree has not been positive; we probably should consider moving in that direction ourselves. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business;