* `make' written in elisp @ 2003-12-30 19:31 Michael Schierl 2003-12-31 22:57 ` Richard Stallman ` (2 more replies) 0 siblings, 3 replies; 30+ messages in thread From: Michael Schierl @ 2003-12-30 19:31 UTC (permalink / raw) Hi, I'm not sure if I am right here ("Emacs development discussions." Discussions about developing Emacs, developing for Emacs or developing with Emacs...?) so if i'm wrong I'd like if you pointed me into the right direction... For most users of Emacs who are not on *ix, it is harder to install new packages than for those on *ix since most (GNU) Emacs "extensions" are shipped with makefiles or autoconf scripts. This works flawlessly on Unix but not on other systems - especially because most of the simpler scripts make assumptions where your emacs binary or your site-lisp directory is. You can adjust this for every single package, but what for? most makefiles don't do more than runninge batch-byte-compile for the .el files, makeinfo for the .texi files and install-info for the resulting info files (plus copying the files where they belong). So I tried to write a framework completely in Elisp (except makeinfo at the moment, because `texinfo-format-buffer' seems to be just crashy/buggy/whatever when applying it to "in-the-wild" texinfo files) called elMake¹. Its syntax is similar to that of apache ant, except it is in sexps instead of xml tags (Users writing Elisp should be familiar with that syntax). It seems to work quite well so far (example "elMakefiles" for jde, auctex and of course elmake itself seem to work as desired). You can have a look at it at http://elmake.sourceforge.net/ (the website is quite "minimalistic", but what do you need more for a 42KB download?). So, and why do i write here? I'd like to know - what you think of that idea - if you could think to use it for your own emacs addons (if you maintain any) - what is bad at the particular implementation - if you find any bugs - what features are missing - how to install such a thing in your opinion (atm it is a .el file to be loaded and pressed C-x C-e at the correct position) - and everything else you can think of (if related to elMake). For answers you might use the elmake-devl mailing list at sourceforge, but of course I'll read answers on this list as well. TIA, Michael ¹ hit 2 on Google for "elmake" is the (empty) sf.net project site atm... ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2003-12-30 19:31 `make' written in elisp Michael Schierl @ 2003-12-31 22:57 ` Richard Stallman 2003-12-31 23:14 ` Michael Schierl 2004-01-04 23:25 ` Stefan Monnier 2004-03-31 22:31 ` patch for locate-file-completion? Nic Ferrier 2 siblings, 1 reply; 30+ messages in thread From: Richard Stallman @ 2003-12-31 22:57 UTC (permalink / raw) Cc: emacs-devel Have you designed this so that the same packages can use both this and their current configure files? Use of a configure file and `make install' is the GNU standard, and we don't want any packages that now support the standard to stop supporting it. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2003-12-31 22:57 ` Richard Stallman @ 2003-12-31 23:14 ` Michael Schierl 2004-01-01 21:10 ` Richard Stallman 0 siblings, 1 reply; 30+ messages in thread From: Michael Schierl @ 2003-12-31 23:14 UTC (permalink / raw) Cc: emacs-devel Richard Stallman schrieb: > > Have you designed this so that the same packages can use both this and > their current configure files? Yes. There is no change needed for the packages i tested until now (except placing an additional file called elMakefile into the install directory). And if a package would require to change its directory structure/whatever, that is clearly a fault of elMake then and needs to be fixed. This will of course mean for developers to maintain two different install methods. > Use of a configure file and `make > install' is the GNU standard, and we don't want any packages that now > support the standard to stop supporting it. Sure. I don't want that either. Michael ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2003-12-31 23:14 ` Michael Schierl @ 2004-01-01 21:10 ` Richard Stallman 2004-01-01 21:37 ` Michael Schierl 0 siblings, 1 reply; 30+ messages in thread From: Richard Stallman @ 2004-01-01 21:10 UTC (permalink / raw) Cc: emacs-devel Yes. There is no change needed for the packages i tested until now (except placing an additional file called elMakefile into the install directory). And if a package would require to change its directory structure/whatever, that is clearly a fault of elMake then and needs to be fixed. Then it might be a good idea. Can you show me an example of an elMakefile? ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2004-01-01 21:10 ` Richard Stallman @ 2004-01-01 21:37 ` Michael Schierl 0 siblings, 0 replies; 30+ messages in thread From: Michael Schierl @ 2004-01-01 21:37 UTC (permalink / raw) Cc: emacs-devel Richard Stallman schrieb: > > Yes. There is no change needed for the packages i tested until now > (except placing an additional file called elMakefile into the install > directory). And if a package would require to change its directory > structure/whatever, that is clearly a fault of elMake then and needs to > be fixed. > > Then it might be a good idea. Can you show me an example of an > elMakefile? Surely. Directly from CVS: <URL:http://cvs.sourceforge.net/viewcvs.py/*checkout*/elmake/elmake/samples/auctex.elMake_?content-type=text%2Fplain&rev=1.1> <URL:http://cvs.sourceforge.net/viewcvs.py/*checkout*/elmake/elmake/samples/emptyB.elMake?content-type=text%2Fplain&rev=1.2> <URL:http://cvs.sourceforge.net/viewcvs.py/*checkout*/elmake/elmake/samples/jde.elMake_?content-type=text%2Fplain&rev=1.1> A "general" sample: <URL:http://cvs.sourceforge.net/viewcvs.py/*checkout*/elmake/elmake/samples/sample.elMake?content-type=text%2Fplain&rev=1.3> And the one used for "bootstrapping" elMake itself: <URL:http://cvs.sourceforge.net/viewcvs.py/*checkout*/elmake/elmake/elMakefile?content-type=text%2Fplain&rev=1.5> You might as well download <URL:http://prdownloads.sourceforge.net/elmake/elmake-0.1.zip> They are all in there as well. Michael ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2003-12-30 19:31 `make' written in elisp Michael Schierl 2003-12-31 22:57 ` Richard Stallman @ 2004-01-04 23:25 ` Stefan Monnier 2005-01-02 16:06 ` Richard Stallman 2004-03-31 22:31 ` patch for locate-file-completion? Nic Ferrier 2 siblings, 1 reply; 30+ messages in thread From: Stefan Monnier @ 2004-01-04 23:25 UTC (permalink / raw) Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1106 bytes --] > For most users of Emacs who are not on *ix, it is harder to install new > packages than for those on *ix since most (GNU) Emacs "extensions" are > shipped with makefiles or autoconf scripts. This works flawlessly on Unix > but not on other systems - especially because most of the simpler scripts > make assumptions where your emacs binary or your site-lisp directory is. > You can adjust this for every single package, but what for? most makefiles > don't do more than runninge batch-byte-compile for the .el files, makeinfo > for the .texi files and install-info for the resulting info files (plus > copying the files where they belong). For the same reason, I wrote the install.el elisp package which can take a single .el file or a tarball of an elisp package and install it. It is a completely different solution to your problem, but I think that your code addresses just the issues that are left in install.el. It's still very simple (or rather: simplistic) and needs more work, but it might be interesting to integrate it with your code. I have attached my latest install.el. Stefan [-- Attachment #2: install.el --] [-- Type: application/emacs-lisp, Size: 21208 bytes --] [-- Attachment #3: Type: text/plain, Size: 141 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://mail.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2004-01-04 23:25 ` Stefan Monnier @ 2005-01-02 16:06 ` Richard Stallman 2005-01-02 23:22 ` David Kastrup 0 siblings, 1 reply; 30+ messages in thread From: Richard Stallman @ 2005-01-02 16:06 UTC (permalink / raw) Cc: schierlm, emacs-devel A year after you posted it, I finally read the source code of install.el. (I was too backlogged to read source code last January.) I found that it was extremely clean and appealing code. If a package system for Emacs can mean something like this, I am in favor of it. I would suggest some small changes. * Instead of asking the user every time whether to install for everyone or just for him, I think it would be better to have different command names, or use a prefix argument to mean "global". It is not feasible to use the commands from programs if they always ask questions. * It seems to assume that auto-compression-mode is enabled, that just writing a file whose name ends in .gz will compress it. * A number of functions need doc strings. Here's the version that was posted a year ago; does anyone want to work on it further? ;;; install.el --- Package to ease installation of Elisp packages ;; Copyright (C) 2001, 2003 Stefan Monnier ;; Author: Stefan Monnier <monnier@cs.yale.edu> ;; This file is free software; you can redistribute it and/or modify ;; it under the terms of the GNU General Public License as published by ;; the Free Software Foundation; either version 2, or (at your option) ;; any later version. ;; This file is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; You should have received a copy of the GNU General Public License ;; along with GNU Emacs; see the file COPYING. If not, write to ;; the Free Software Foundation, Inc., 59 Temple Place - Suite 330, ;; Boston, MA 02111-1307, USA. ;;; Commentary: ;; This little package is meant to ease up the task of installing ;; third party ELisp packages. I.e. it takes care of placing it in ;; an appropriate location, finds out what code is necessary to get ;; the package activated, sets up your .emacs file to activate the ;; package, and byte-compiles the package. ;; It should work on both single-file packages and tarballs. ;; On tarball packages, it does a bit of guess work to figure out ;; where are which files and how to use them. This is bound to ;; fail sometimes. ;; Tested on: ;; + ProofGeneral ;; + sml-mode ;; + AUCTeX (missed the info page) ;; + X-Symbol (as an XEmacs package) ;; + Gnus (but doesn't install the info doc :-( ) ;; + BBDB (misses the main info page) ;; - WhizzyTeX (needs to hack the perl script and stuff :-( ) ;; ? ECB ;; ? JDEE ;; ? preview-latex ;; ? VM ;; ? mmm-mode ;; ? Semantic ;; The on-disk structure is as follows: ;; - there are two area: the `home' and the `site' each with their ;; respective directory (~/lib/emacs and /usr/share/emacs/site-lisp) ;; and file (.emacs and site-start). ;; - There is a distinction between installing and activating. ;; Installing only places the files on disk, whereas activating sets up ;; autoloads and friends. ;; - Activation is done on a directory by directory basis. Each directory ;; has an `autoloads' file. Loading it activates the package(s) ;; in directory. ;; - Single-file packages are placed together in the toplevel directory ;; whereas tarball-packages are placed in their own subdirectory (so they ;; can be activated independently). ;;; Todo: ;; - don't ask whether to activate site-wide packages installed in home. ;; - Create Info from Texinfo when needed. ;; - Try harder to find Info files such as doc/auctex. ;; - UI to (un)install and (de)activate packages, get a list, ... ;; - If a single-file package lacks ;;;###autoload, try to add them ;; based on the Commentary section or something. ;; - don't leave out reams of `autoloads~' backup files. ;;; Code: (require 'em-glob) (defgroup install nil "Elisp package installation") (defmacro install-filter (list exp) (declare (debug t)) `(let ((res nil)) (dolist (x ,list (nreverse res)) (if ,exp (push x res))))) (defcustom install-site-file (or (locate-file (or site-run-file "site-start") load-path load-suffixes) (let ((lp (mapcar 'abbreviate-file-name load-path))) ;; Prefer non-user directories. (setq lp (or (install-filter lp (not (string-match "\\`~/" x))) lp)) ;; Prefer site-lisp directories. (setq lp (or (install-filter lp (string-match "/site-lisp\\'" x)) lp)) ;; Prefer shorter directory names (i.e. parents rather than subdirs). (setq lp (sort lp (lambda (d1 d2) (< (length d1) (length d2))))) ;; (expand-file-name (concat (or site-run-file "site-start") ".el") (car lp)))) "Site-wide customization file." :type 'file) (defcustom install-site-dir (file-name-directory install-site-file) "Directory where site-wide packages should be installed." :type 'directory) (defcustom install-home-file (or user-init-file (convert-standard-filename "~/.emacs")) "Main customization file into which Install should place `load' commands." :type 'file) (defcustom install-home-dir ;; FIXME: We should be careful never to choose one of Emacs's own ;; directories, even if the user installed Emacs in his home dir. (let ((lp (mapcar 'abbreviate-file-name load-path))) ;; Only consider writable directories. (setq lp (install-filter lp (file-writable-p x))) ;; Only consider user directories. (setq lp (install-filter lp (string-match "\\`~/" x))) ;; Prefer shorter directory names (i.e. parents rather than subdirs). (setq lp (sort lp (lambda (d1 d2) (< (length d1) (length d2))))) ;; Default to ~/lib/emacs. (if (or (null lp) ;; If it's a subdir of lib/emacs, use lib/emacs. This can happen ;; because Install does not automatically add lib/emacs to the ;; load-path if it only installs tar packages underneath. (string-match "\\`~/lib/emacs/" (car lp))) "~/lib/emacs/" (car lp))) "Directory into which elisp packages should be placed by Install." :type 'directory) (defcustom install-autoload-file "autoloads" "Name of autoload files used by Install.") (defcustom install-compress-source-files nil ;; ".gz" "If non-nil, Install will try to compress file.") (defcustom install-byte-compile t "If non-nil, elisp files are byte-compiled during installation." :type 'boolean) ;; (defun install-get-dir () "Return the directory into which to install packages." (or (and (file-writable-p install-site-dir) (y-or-n-p "Install site-wide? ") install-site-dir) (progn (unless (file-writable-p install-home-dir) (setq install-home-dir (let ((default-directory install-home-dir)) (read-directory-name "Directory to install into: "))) (unless (file-directory-p install-home-dir) (make-directory install-home-dir t))) install-home-dir))) (defun install-get-file () "Return the file into which to activate packages." (or (and (file-writable-p install-site-file) (y-or-n-p "Activate site-wide? ") install-site-file) install-home-file)) (defmacro install-with-file (file &rest body) (declare (debug t) (indent 1)) `(let ((install-with-existing-file (find-buffer-visiting ,file))) (with-current-buffer (or install-with-existing-file (find-file-noselect ,file)) (prog1 (save-current-buffer ,@body) (unless install-with-existing-file (kill-buffer (current-buffer))))))) ;;;###autoload (defun install-file (file) (interactive "fFile to install: ") (with-current-buffer (find-file-noselect file) (install-buffer))) ;;;###autoload (defun install-buffer () "Install the current elisp buffer as a package. The package is install in `install-home-dir', autoloads are added to the `install-autoload-file' in that directory and the `install-custom-file' is then updated to load these autoloads." (interactive) (cond ((derived-mode-p 'tar-mode) (install-tar-buffer)) ((not (derived-mode-p 'emacs-lisp-mode)) (error "I only know how to install tar.gz and elisp files.")) (t (let* ((install-dir (install-get-dir)) (package (file-name-nondirectory buffer-file-name)) (file (expand-file-name package install-dir)) (autoload (expand-file-name install-autoload-file install-dir))) (when (and install-compress-source-files (string-match "\\.el\\'" file)) (setq file (concat file (if (stringp install-compress-source-files) install-compress-source-files ".gz")))) ;; Install the elisp file. (write-region (point-min) (point-max) file) ;; Extract the autoloads into a separate file. (install-update-autoloads autoload) ;; Activate. (install-activate autoload) ;; Finally, byte compile. In the present case (a single-file package), ;; this could be done before activation. (if install-byte-compile (byte-compile-file file)))))) (defun install-tar-buffer () "Like `install-buffer' but for a tar package rather than single file." (let* ((name (file-name-nondirectory buffer-file-name)) ;; Strip off ".tar.gz", ".tar", ".tgz", ".tar.Z", ... (name (if (string-match "\\.[tT][^.]+\\(\\.[^.]+\\)?\\'" name) (substring name 0 (match-beginning 0)) name)) (install-dir (install-get-dir)) (default-directory (expand-file-name name install-dir))) ;; Install the files. ;; FIXME: check what `tar-untar-buffer' does with symlinks and stuff. ;; FIXME: the dir might already exist. (make-directory default-directory) (tar-untar-buffer) (let ((files (directory-files default-directory nil "\\`\\([^.]\\|\\.[^.]\\|\\.\\..\\)" t))) ;; If the tar file already had everything under a single directory, ;; remove the redundant level of directory. (when (and (= (length files) 1) (file-directory-p (car files))) (let* ((f (car files)) ;; Keep the longest name of the two, assuming that the ;; difference is that the longer one has a version number. (final (if (> (length name) (length f)) name f)) (temp (if (= (length name) (length f)) (concat f ".tmp") f))) ;; FIXME: the dir might already exist. (rename-file f (expand-file-name temp install-dir)) (setq default-directory install-dir) (delete-directory name) ;; FIXME: the dir might already exist. (unless (equal final temp) (rename-file temp final)) (setq name final) (setq default-directory (expand-file-name final))))) (install-directory-inplace))) ;;;###autoload (defun install-directory-inplace () "Prepare and activate the current directory for use by Emacs. Sets up the autoload files, activates them and byte-compiles if needed." (interactive) ;; Extract the autoloads. (install-setup-tree) ;; Activate the package. (install-activate (expand-file-name (install-get-activation-file))) ;; Finally, byte-compile the files. (if install-byte-compile (install-byte-compile-dir))) (defun install-dirs-of-files (files) "Return a list of subdirs containing elisp files." (let ((dirs nil) (ignore (regexp-opt (cons ;; Ignore contrib directories because they tend to contain ;; either less-debugged code, or packages that might ;; already be installed and can thus interfere. "contrib/" (let ((exts nil)) (dolist (ext completion-ignored-extensions exts) (if (eq (aref ext (1- (length ext))) ?/) (push ext exts)))))))) ;; Collect the dirs that hold elisp files. (dolist (file files dirs) (let ((dir (file-name-directory file))) (unless (or (member dir dirs) (and dir (string-match ignore dir))) (push dir dirs)))))) (defun install-find-elisp-dirs () "Return a list of subdirs containing elisp files." (install-dirs-of-files (install-glob "**/*.el"))) (defun install-byte-compile-dir () "Byte compile all elisp files under the current directory." (let ((load-path (append (mapcar (lambda (dir) (if dir (expand-file-name dir) default-directory)) (install-find-elisp-dirs)) load-path))) (byte-recompile-directory default-directory 0))) (defun install-glob (pattern) (let ((res (eshell-extended-glob pattern))) (if (listp res) res))) (defun install-get-activation-file () "Return the file to load to activate the package. This is usually \"./autoloads\", but it can also be \"lisp/foo-site.el\"." (if (file-exists-p install-autoload-file) install-autoload-file (or (car (install-glob (concat "**/" install-autoload-file))) (car (install-glob "**/auto-autoloads.el")) (car (install-glob "**/*-site.el"))))) (defun install-setup-tree () (eshell-glob-initialize) ;; Look for elisp files. (let ((dirs (install-find-elisp-dirs)) (autoload-files nil) (toplevel nil)) ;; Prepare each elisp subdir and collect info along the way. (dolist (dir dirs) (let ((default-directory (expand-file-name (or dir default-directory)))) ;; Remove *.elc files, in case they were not compiled for our version. (mapc 'delete-file (install-glob "*.elc")) ;; Extract autoloads. (let ((sites (or (install-glob "auto-autoloads.el") (install-glob "*-site.el")))) (if (= 1 (length sites)) ;; Some packages come with a <pkg>-site.el file instead ;; of using autoloads. In that case, just load that file. (push (concat dir (car sites)) autoload-files) ;; Otherwise. Make an autoloads file and load it. ;; FIXME: Don't make hundreds of autoload files. (let ((exists (file-exists-p install-autoload-file))) (if (not (install-update-autoloads install-autoload-file)) ;; Don't stupidly add empty autoloads files. (unless exists (delete-file install-autoload-file)) (push (concat dir install-autoload-file) autoload-files))))))) (mapc 'install-ensure-autoloads-file autoload-files) ;; Setup the toplevel activation file. (if (and (= 1 (length autoload-files)) (equal (car autoload-files) (install-get-activation-file))) (setq toplevel (car autoload-files)) (setq toplevel install-autoload-file) (dolist (file autoload-files) (unless (equal file toplevel) (install-activate `(expand-file-name ,(file-relative-name (expand-file-name file) (file-name-directory (expand-file-name toplevel))) (file-name-directory load-file-name)) toplevel)))) ;; Make up an info/dir file if necessary and register the info dirs. (let ((info-dirs (install-make-info))) (when info-dirs (install-with-file toplevel (unless (derived-mode-p 'emacs-lisp-mode) (emacs-lisp-mode)) (goto-char (point-min)) (unless (re-search-forward "(add-to-list[ \t\n]+'Info-default-directory-list" nil t) (forward-comment (point-max)) (while (re-search-backward "^\f" nil t)) (unless (bolp) (newline)) (let ((top-dir (file-name-directory (expand-file-name toplevel)))) (dolist (dir info-dirs) (setq dir (expand-file-name (or dir default-directory))) (if (equal dir top-dir) (insert "(add-to-list 'Info-default-directory-list (file-name-directory load-file-name))\n") (let ((text (pp-to-string (file-relative-name dir top-dir)))) (if (string-match "\n\\'" text) (setq text (substring text 0 -1))) (insert "(add-to-list 'Info-default-directory-list\n" " (expand-file-name " text " (file-name-directory load-file-name)))\n"))))) (save-buffer))))))) (defun install-ensure-autoloads-file (file) "Make sure that the autoload file FILE exists and if not create it." (install-with-file file (unless (derived-mode-p 'emacs-lisp-mode) (emacs-lisp-mode)) (when buffer-read-only (set-file-modes buffer-file-name (logior ?\200 (file-modes buffer-file-name))) (toggle-read-only)) (goto-char (point-min)) ;; Insert a little boiler plate if there's nothing yet. (when (eobp) (insert ";;; " (file-name-nondirectory file) " --- automatically extracted autoloads\n" ";;\n" ";;; Code:\n\n" "\f\n;; Local Variables:\n" ";; version-control: never\n" ";; no-byte-compile: t\n" ";; no-update-autoloads: t\n" ";; End:\n" ";;; " (file-name-nondirectory file) " ends here\n") (goto-char (point-min))) ;; Make sure it will setup the load path properly. (unless (re-search-forward "\\<load-file-name\\>" nil t) (forward-comment (point-max)) (while (re-search-backward "^\f" nil t)) (unless (bolp) (newline)) (unless (eq (char-before (1- (point))) ?\n) (newline)) (insert ";; Tell Emacs to look for elisp files in this directory." ;; Add some sort of signature. " -- Install\n") (insert "(add-to-list 'load-path (or (file-name-directory load-file-name) (car load-path)))\n\n") (save-buffer))) file) (defvar generated-autoload-file) (defun install-update-autoloads (autoload) "Update file AUTOLOAD. This will create the file if necessary. Returns non-nil if there is anything autoloaded into it." (setq autoload (expand-file-name autoload)) (let ((bufp (find-buffer-visiting autoload))) (let ((generated-autoload-file (install-ensure-autoloads-file autoload))) ;; (update-file-autoloads file) (update-directory-autoloads (file-name-directory autoload))) ;; Make sure the file sets up the load-path appropriately. (with-current-buffer (find-file-noselect autoload) (unless (derived-mode-p 'emacs-lisp-mode) (emacs-lisp-mode)) (goto-char (point-min)) (re-search-forward "^\f" nil t) ;Find the first autoload entry. (forward-comment (point-max)) (prog1 (not (eobp)) (unless bufp (kill-buffer (current-buffer))))))) (defun install-activate (autoload &optional into) "Update INTO to make sure it loads AUTOLOAD. AUTOLOAD can be an expression. If it is a string, this also loads it into the currently running Emacs. If provided, INTO specifies the file which should load AUTOLOAD. The default is to use `install-get-file'." (when (stringp autoload) (setq autoload (abbreviate-file-name autoload)) (load autoload)) (install-with-file (or into (install-get-file)) (unless (derived-mode-p 'emacs-lisp-mode) (emacs-lisp-mode)) (save-excursion (let ((text (pp-to-string autoload))) (if (string-match "\n\\'" text) (setq text (substring text 0 -1))) (goto-char (point-min)) (unless (re-search-forward (regexp-quote text) nil t) (goto-char (point-min)) (forward-comment (point-max)) (while (re-search-backward "^\f" nil t)) (unless (bolp) (newline)) ;; Pass `install' as argument to load: this both makes Emacs ;; ignore the load if the file is missing and is used as a marker ;; indicating that this load statement was introduced by us. (insert "(load " text " 'install)\n") (save-buffer)))))) ;;;###autoload (defun install-list-packages () "Show the installed packages." (interactive) (dired (install-get-dir))) ;; Info files and DIR files. ;; Some of this should probably be moved to info.el. (defconst install-info-dir "-*- Text -*-\n\x1f\n\ File: dir Node: Top This is the top of the INFO tree\ \n\n* Menu:\n\n" "Text content of a barebones empty `info/dir' file.") (defun install-find-info-files () (let ((files (or (install-glob "**/*.info*") (install-glob "**/info/*"))) (tmp nil)) (dolist (f files) (unless (or (member f '("dir" "localdir")) (and (string-match "-[0-9]+" f) (member (replace-match "" t t f) files)) (not (string-match "\\.info\\>\\|\\(\\`\\|/\\)[^.]+\\(\\'\\|\\.\\(gz\\|Z\\)\\)" f))) (push f tmp))) tmp)) (defun install-make-info () "Make an info/dir file if necessary and return the info directories." ;; FIXME: This should create the info files from the Texinfo files ;; if necessary !! ;; Problems to do that: ;; - detect when necessary. E.g. BBDB comes with an info page for ;; the bbdb-filters stuff, but the main bbdb doc is in texinfo. ;; - figure out how to makeinfo the thing. E.g. AucTeX comes with ;; a whole bunch of Texinfo files and it's really not clear which ;; is the right one. ;; - The info file might be there, but not found. E.e. AucTeX has its ;; page in doc/auctex. (let* ((files (install-find-info-files)) (dirs (install-dirs-of-files files)) (dir-files nil)) ;; Remove files that were in ignored directories. (dolist (file files) (unless (member (file-name-directory file) dirs) (setq files (delq file files)))) ;; Check that there's something to do. (when files (assert dirs) (dolist (dir dirs) (if (file-exists-p (expand-file-name "dir" dir)) (push (expand-file-name "dir" dir) dir-files))) (unless dir-files ;; Pick the dir closest to the toplevel to put the main dir file. (setq dirs (sort dirs (lambda (s1 s2) (< (length s1) (length s2))))) (install-with-file (expand-file-name "dir" (car dirs)) (assert (= (point-min) (point-max))) (insert install-info-dir) (narrow-to-region (point) (point-max)) (dolist (file files) (let ((section "Miscellaneous") (entry nil)) (install-with-file file (goto-char (point-min)) (if (not (re-search-forward (concat "^START-INFO-DIR-ENTRY\n" "\\([* \t].*\n\\)+" "END-INFO-DIR-ENTRY$") nil t)) ;; No entry in the file, let's build a default one. (let ((base (file-name-nondirectory (file-name-sans-extension file)))) (setq entry (concat "* " (upcase base) ": (" base ").\n"))) (setq entry (match-string 1)) (goto-char (point-min)) (when (re-search-forward "^INFO-DIR-SECTION[ \t]+\\(.*[^ \t\n]\\)" nil t) (setq section (match-string 1))))) (goto-char (point-min)) (unless (search-forward entry nil t) (unless (re-search-forward (concat "^" (regexp-quote section) "[ \t]*\n") nil 'move) (unless (bobp) (newline)) (insert section) (newline)) (insert entry)))) (save-buffer) (kill-buffer (current-buffer)))) dirs))) \f (provide 'install) ;;; install.el ends here ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-02 16:06 ` Richard Stallman @ 2005-01-02 23:22 ` David Kastrup 2005-01-02 23:55 ` Ralf Angeli 2005-01-03 0:26 ` Stefan 0 siblings, 2 replies; 30+ messages in thread From: David Kastrup @ 2005-01-02 23:22 UTC (permalink / raw) Cc: emacs-devel, Stefan Monnier, schierlm Richard Stallman <rms@gnu.org> writes: > A year after you posted it, I finally read the source code of install.el. > (I was too backlogged to read source code last January.) > > I found that it was extremely clean and appealing code. If a > package system for Emacs can mean something like this, I am in favor > of it. Judging from the comments, this installer might more or less get along at least with some XEmacs packages, as well as some other packages with a regular enough structure (AUCTeX has in the mean time gotten a new autoconf-based installer very similar to that of preview-latex, so the latest versions probably will not work with install.el). As far as I understand the XEmacs package system (at which I have taken a cursory glance at best), it consists of several parts: a) a directory layout. XEmacs packages have a standard way of placing Elisp, info, data and other files in a layout under the top package directory. In addition there are some standard file names for initializing of packages and installation. In short, this part of the package system is basically just file layout conventions relative to a package-specific top directory. We don't have any conventions for packages intended for Emacs as far as I can see. It certainly would not be bad to have something like that: if people make code for Emacs, and if we can agree on some layout that makes it easier for stuff to work out of the box, it certainly might be worthwhile to state some conventions. b) version management. Versions must basically be floats, higher versions correspond to later versions, are usually based on some CVS archive and not at all with actual package versions. Uh. c) a package manager that will download and upgrade packages. It might do so in user-specific and system-wide packages. d) central download archives that will provide all packages, updated by packagers (often different from the actual creators of software) on central servers. e) it does not offer AFAIR: dependencies and standard tests (for executables, directories and other stuff) that autoconf provides. A bit of framework for that might be nice for obliterating some necessities for installation. Even though installing MSYS is not particularly hard, it still seems to be a barrier for some people. I am not convinced about the value of the centralized server approach to package management from XEmacs: it does not seem to lend itself overly well to getting reasonably updated packages from third developers (who often have their own download areas from which they offer their stuff, and those would not normally be visible to the package manager that just looks at one server for all packages at one time IIUC). But I do think that we should try to offer some way of installing external Lisp code if one has acquired it as a zip or tar archive in some manner already. And in those respects where the directory structure and layout conventions of an XEmacs package would seem reasonable, there seems little point in inventing something else. Stefan, having met those problems when writing install.el already, would probably be a lot more qualified to comment on my impressions and on the versatility of the package layout he has been catering for. > I would suggest some small changes. [...] All of those certainly sound reasonable. In particular, I'd second having separate commands for installing system-wide and user-local packages. IIRC (I might well be mistaken), XEmacs automatically picked one or the other automatically, based on whether one had previous private packages, and that was less than great. In particular, I tend to install a CVS version of Emacs into its own tree (from my own user), and this means that I can, as a user, install packages into this Emacs' system tree. But that does not imply that I don't want to install some stuff also in my user-local tree (which would get shared by the regular Emacs as well). Having separate commands would appear to be beneficial. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-02 23:22 ` David Kastrup @ 2005-01-02 23:55 ` Ralf Angeli 2005-01-03 0:07 ` David Kastrup 2005-01-03 0:26 ` Stefan 1 sibling, 1 reply; 30+ messages in thread From: Ralf Angeli @ 2005-01-02 23:55 UTC (permalink / raw) * David Kastrup (2005-01-03) writes: > Richard Stallman <rms@gnu.org> writes: > >> I found that it was extremely clean and appealing code. If a >> package system for Emacs can mean something like this, I am in favor >> of it. [...] > But I do think that we should try to offer some way of installing > external Lisp code if one has acquired it as a zip or tar archive in > some manner already. Have you ever looked at source-based package management systems like FreeBSD's ports system or NetBSD's pkgsrc? I think something like this would be suited very well for installing Emacs packages. They basically download the source code from the original site, patch it if necessary and call commands for building it, e.g. `configure', `make' and `make install'. Thus, an installation like the one of AUCTeX would not be a problem. In fact, FreeBSD's port for AUCTeX is ridiculously simple. -- Ralf ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-02 23:55 ` Ralf Angeli @ 2005-01-03 0:07 ` David Kastrup 2005-01-03 0:25 ` Ralf Angeli 0 siblings, 1 reply; 30+ messages in thread From: David Kastrup @ 2005-01-03 0:07 UTC (permalink / raw) Cc: emacs-devel Ralf Angeli <angeli@iwi.uni-sb.de> writes: > * David Kastrup (2005-01-03) writes: > >> Richard Stallman <rms@gnu.org> writes: >> >>> I found that it was extremely clean and appealing code. If a >>> package system for Emacs can mean something like this, I am in favor >>> of it. > [...] >> But I do think that we should try to offer some way of installing >> external Lisp code if one has acquired it as a zip or tar archive in >> some manner already. > > Have you ever looked at source-based package management systems like > FreeBSD's ports system or NetBSD's pkgsrc? I think something like > this would be suited very well for installing Emacs packages. They > basically download the source code from the original site, patch it > if necessary and call commands for building it, e.g. `configure', > `make' and `make install'. Thus, an installation like the one of > AUCTeX would not be a problem. In fact, FreeBSD's port for AUCTeX > is ridiculously simple. The problem is that the only reliable common available infrastructure we have for all operating systems on which Emacs is able to run, is Emacs. And that takes us back to the subject title of this posting. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 0:07 ` David Kastrup @ 2005-01-03 0:25 ` Ralf Angeli 2005-01-03 4:32 ` Richard Stallman 0 siblings, 1 reply; 30+ messages in thread From: Ralf Angeli @ 2005-01-03 0:25 UTC (permalink / raw) * David Kastrup (2005-01-03) writes: > Ralf Angeli <angeli@iwi.uni-sb.de> writes: [source-based package management systems] > The problem is that the only reliable common available infrastructure > we have for all operating systems on which Emacs is able to run, is > Emacs. And that takes us back to the subject title of this posting. Assuming the `make' part is solved by install.el or elmake, we "only" need an Elisp-based substitute for `configure'. If a package like AUCTeX doesn't provide it, one has to require external tools. I don't see a problem here. The point is that the package system should be able to support `configure'. -- Ralf ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 0:25 ` Ralf Angeli @ 2005-01-03 4:32 ` Richard Stallman 2005-01-03 8:02 ` David Kastrup 2005-01-03 9:10 ` Ralf Angeli 0 siblings, 2 replies; 30+ messages in thread From: Richard Stallman @ 2005-01-03 4:32 UTC (permalink / raw) Cc: emacs-devel Assuming the `make' part is solved by install.el or elmake, we "only" need an Elisp-based substitute for `configure'. That is too heavy-weight; I am not interested in installing that kind of package system in Emacs. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 4:32 ` Richard Stallman @ 2005-01-03 8:02 ` David Kastrup 2005-01-03 9:36 ` Eli Zaretskii 2005-01-03 9:10 ` Ralf Angeli 1 sibling, 1 reply; 30+ messages in thread From: David Kastrup @ 2005-01-03 8:02 UTC (permalink / raw) Cc: Ralf Angeli, emacs-devel Richard Stallman <rms@gnu.org> writes: > Assuming the `make' part is solved by install.el or elmake, we "only" > need an Elisp-based substitute for `configure'. > > That is too heavy-weight; I am not interested in installing that kind > of package system in Emacs. Well, at least a standard function for finding the path name of an executable in PATH and/or in general a file in a set of locations might be nice to have. Of course a complete autoconf system would be ridiculous. I guess that once we start talking about actually compiling stuff (and autoconf's main point is about figuring compiler specialties), we might as well assume a complete GNU build system to be present, so we would not need to implement this in Emacs. But as long as we are just talking about figuring out some locations and the availability/name of some executables, there is little point not to have a few available functions for that. Even though it might in some cases be more convenient if they run at runtime instead of install time, making it possible to switch between setups/PATHs without reinstalling/reconfiguring (or have an explicit command PACKAGENAME-reconfigure for reconfiguring?). -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 8:02 ` David Kastrup @ 2005-01-03 9:36 ` Eli Zaretskii 2005-01-03 16:45 ` Stefan Monnier 0 siblings, 1 reply; 30+ messages in thread From: Eli Zaretskii @ 2005-01-03 9:36 UTC (permalink / raw) Cc: angeli, emacs-devel > From: David Kastrup <dak@gnu.org> > Date: Mon, 03 Jan 2005 09:02:37 +0100 > Cc: Ralf Angeli <angeli@iwi.uni-sb.de>, emacs-devel@gnu.org > > Well, at least a standard function for finding the path name of an > executable in PATH and/or in general a file in a set of locations > might be nice to have. We already have that, I think: locate-file-internal is a built-in function in `C source code'. (locate-file-internal FILENAME PATH &optional SUFFIXES PREDICATE) Search for FILENAME through PATH. If SUFFIXES is non-nil, it should be a list of suffixes to append to file name when searching. If non-nil, PREDICATE is used instead of `file-readable-p'. PREDICATE can also be an integer to pass to the access(2) function, in which case file-name-handlers are ignored. Isn't this what you wanted? ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 9:36 ` Eli Zaretskii @ 2005-01-03 16:45 ` Stefan Monnier 0 siblings, 0 replies; 30+ messages in thread From: Stefan Monnier @ 2005-01-03 16:45 UTC (permalink / raw) Cc: angeli, emacs-devel >> Well, at least a standard function for finding the path name of an >> executable in PATH and/or in general a file in a set of locations >> might be nice to have. > We already have that, I think: > locate-file-internal is a built-in function in `C source code'. > (locate-file-internal FILENAME PATH &optional SUFFIXES PREDICATE) I'd rather advertise locate-file. There's also executable-find. Stefan ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 4:32 ` Richard Stallman 2005-01-03 8:02 ` David Kastrup @ 2005-01-03 9:10 ` Ralf Angeli 2005-01-03 18:29 ` Richard Stallman 1 sibling, 1 reply; 30+ messages in thread From: Ralf Angeli @ 2005-01-03 9:10 UTC (permalink / raw) Cc: emacs-devel * Richard Stallman (2005-01-03) writes: > Assuming the `make' part is solved by install.el or elmake, we "only" > need an Elisp-based substitute for `configure'. > > That is too heavy-weight; I am not interested in installing that kind > of package system in Emacs. At least it should be able to call things like `configure' which is a mandatory step in the build process of packages like AUCTeX and preview-latex. During this step the values of some important variables are determined and written to the package's site configuration file. For example XEmacs' package system suffers from its inability to do that. AUCTeX users who want to use the package provided by the XEmacs package system have to do the respective configuration manually which the configure script does automatically. -- Ralf ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 9:10 ` Ralf Angeli @ 2005-01-03 18:29 ` Richard Stallman 2005-01-03 19:28 ` Ralf Angeli 0 siblings, 1 reply; 30+ messages in thread From: Richard Stallman @ 2005-01-03 18:29 UTC (permalink / raw) Cc: emacs-devel At least it should be able to call things like `configure' which is a mandatory step in the build process of packages like AUCTeX and preview-latex. During this step the values of some important variables are determined and written to the package's site configuration file. Why is it important for Emacs Lisp packages to do that sort of thing? ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 18:29 ` Richard Stallman @ 2005-01-03 19:28 ` Ralf Angeli 2005-01-03 23:10 ` David Kastrup 2005-01-04 3:38 ` Richard Stallman 0 siblings, 2 replies; 30+ messages in thread From: Ralf Angeli @ 2005-01-03 19:28 UTC (permalink / raw) Cc: emacs-devel * Richard Stallman (2005-01-03) writes: > At least it should be able to call things like `configure' which is a > mandatory step in the build process of packages like AUCTeX and > preview-latex. During this step the values of some important > variables are determined and written to the package's site > configuration file. > > Why is it important for Emacs Lisp packages to do that sort of thing? Automatically determining variables specific to the site the package is being installed at configuration time is a means to cut down load times. There are things where it is not efficient or feasible to carry out the respective tests each time the package is loaded. As an example, in AUCTeX we are currently using `configure' to determine locations where TeX trees are located. The resulting value is used at runtime for speeding up file searches. Another example (which is not yet implemented) is the determination of programs for viewing DVI or PDF files on different platforms. `configure' can check which alternative is present on the system at hand and insert the choice into the site configuration file. This would allow us to get rid of the different platform-specific files we now have. Besides writing values to init files the configuration process is used to check if external tools required for building or running the package are present and provide necessary features. A special case might be preview-latex which has a TeX part besides the Elisp part. For the respective file the target directory for its installation is determined by `configure'. -- Ralf ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 19:28 ` Ralf Angeli @ 2005-01-03 23:10 ` David Kastrup 2005-01-04 3:38 ` Richard Stallman 1 sibling, 0 replies; 30+ messages in thread From: David Kastrup @ 2005-01-03 23:10 UTC (permalink / raw) Cc: rms, emacs-devel Ralf Angeli <angeli@iwi.uni-sb.de> writes: > * Richard Stallman (2005-01-03) writes: > >> At least it should be able to call things like `configure' >> which is a mandatory step in the build process of packages like >> AUCTeX and preview-latex. During this step the values of some >> important variables are determined and written to the package's >> site configuration file. >> >> Why is it important for Emacs Lisp packages to do that sort of >> thing? > > Automatically determining variables specific to the site the package > is being installed at configuration time is a means to cut down load > times. There are things where it is not efficient or feasible to > carry out the respective tests each time the package is loaded. > > As an example, in AUCTeX we are currently using `configure' to > determine locations where TeX trees are located. The resulting > value is used at runtime for speeding up file searches. I am not actually convinced that this is a good idea: I'd prefer it that _if_ AUCTeX is made to match some package layout, then the whole installation/configuration will be performed by Lisp. Probably cached in customize variables, and with an option to redo the detection on demand (in case you want to experiment with different setups of TeX distributions or similar). I'd really like it if we could skip all of the autoconf magic (which is particularly painful and error-prone dealing with paths with spaces and backslashes) for a reasonably easy Elisp solution. It would appear that with some of the mentioned functions in this thread, this might be possible. -- David Kastrup, Kriemhildstr. 15, 44793 Bochum ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 19:28 ` Ralf Angeli 2005-01-03 23:10 ` David Kastrup @ 2005-01-04 3:38 ` Richard Stallman 2005-01-04 11:17 ` Ralf Angeli 1 sibling, 1 reply; 30+ messages in thread From: Richard Stallman @ 2005-01-04 3:38 UTC (permalink / raw) Cc: emacs-devel Automatically determining variables specific to the site the package is being installed at configuration time is a means to cut down load times. There are things where it is not efficient or feasible to carry out the respective tests each time the package is loaded. I am surprised to hear that. Many Emacs Lisp packages search for files they need to use, and they generally do so either when first loaded into a session, or each time they are executed. We have never tried to make any of them save the results between sessions, but it seems to be fast enough. Is AUCTeX doing something that is particularly slow? Besides writing values to init files the configuration process is used to check if external tools required for building or running the package are present and provide necessary features. A special case might be preview-latex which has a TeX part besides the Elisp part. Why is it necessary to do this? It seems to me that it would be just fine to look for these things when the user tries to use them. to try to use the command when the user asks to do ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-04 3:38 ` Richard Stallman @ 2005-01-04 11:17 ` Ralf Angeli 0 siblings, 0 replies; 30+ messages in thread From: Ralf Angeli @ 2005-01-04 11:17 UTC (permalink / raw) Cc: emacs-devel * Richard Stallman (2005-01-04) writes: > I am surprised to hear that. Many Emacs Lisp packages search for > files they need to use, and they generally do so either when first > loaded into a session, or each time they are executed. We have never > tried to make any of them save the results between sessions, but it > seems to be fast enough. > > Is AUCTeX doing something that is particularly slow? That depends on how you define slow. I just extracted the relevant code for testing purposes and using `time' with the resulting shell script I got the following execution times on a Pentium M 1,7 (currently running at 600MHz): real 0m0.114s user 0m0.084s sys 0m0.018s In the meantime I reimplemented the functionality in Elisp. Using (abs (- (prog1 (float-time (current-time)) (TeX-input-dirs)) (float-time (current-time)))) for measuring the execution time I got an average of about 0.06 seconds for executing the function `TeX-input-dirs'. While I am not really fond of having the function executed every time the package is loaded, it is probably an acceptable delay. And, as David already mentioned, we could find ways to save its output and subsequently inhibit its execution until a user calls some reconfiguration procedure. > Besides writing values to init files the configuration process is used > to check if external tools required for building or running the > package are present and provide necessary features. A special case > might be preview-latex which has a TeX part besides the Elisp part. > > Why is it necessary to do this? > It seems to me that it would be just fine > to look for these things when the user tries to use them. > > to try to use the command when the user asks to do I don't think it is feasible to check for program versions every time a program is called. And instead of letting the user alone with error messages which don't give him a hint that the program is too old, I consider it better practice to tell him when he tries to install the software which relies on the external program. -- Ralf ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-02 23:22 ` David Kastrup 2005-01-02 23:55 ` Ralf Angeli @ 2005-01-03 0:26 ` Stefan 2005-01-03 11:05 ` Stephen J. Turnbull 1 sibling, 1 reply; 30+ messages in thread From: Stefan @ 2005-01-03 0:26 UTC (permalink / raw) Cc: emacs-devel, rms, schierlm [-- Attachment #1: Type: text/plain, Size: 4493 bytes --] > Judging from the comments, this installer might more or less get along > at least with some XEmacs packages, as well as some other packages > with a regular enough structure (AUCTeX has in the mean time gotten a > new autoconf-based installer very similar to that of preview-latex, so > the latest versions probably will not work with install.el). It might still work, depending on how the configuration is done. I tend to think that most of the autoconfiguration should be done upon use rather than during installation, which basically implies "in elisp rather than in autoconf". > a) a directory layout. XEmacs packages have a standard way of placing > Elisp, info, data and other files in a layout under the top package > directory. In addition there are some standard file names for > initializing of packages and installation. In short, this part of the > package system is basically just file layout conventions relative to a > package-specific top directory. > We don't have any conventions for packages intended for Emacs as far > as I can see. It certainly would not be bad to have something like > that: if people make code for Emacs, and if we can agree on some > layout that makes it easier for stuff to work out of the box, it > certainly might be worthwhile to state some conventions. Yes, `install' does not impose a particular structure (it just looks recursively for *.el files). But it would benefit from some conventions (so it can safely ignore the contrib/foo.el or compat/bar.el files). I can't find any documentation about the in-package layout used by XEmacs, tho. Anybody knows where it's described? > b) version management. Versions must basically be floats, higher > versions correspond to later versions, are usually based on some CVS > archive and not at all with actual package versions. Uh. Right, it seems to be used only to discover whether a newer version is available or not. > c) a package manager that will download and upgrade packages. It > might do so in user-specific and system-wide packages. > d) central download archives that will provide all packages, updated > by packagers (often different from the actual creators of software) on > central servers. > e) it does not offer AFAIR: dependencies and standard tests (for > executables, directories and other stuff) that autoconf provides. A > bit of framework for that might be nice for obliterating some > necessities for installation. Even though installing MSYS is not > particularly hard, it still seems to be a barrier for some people. `install' doesn't get involved with version management, dependencies, ... It only tries to make it easier for users to install and setup external packages. A package manager to also allow activation/deactivation/deletion would be a natural addition. > I am not convinced about the value of the centralized server approach > to package management from XEmacs: it does not seem to lend itself Since `install' specifically targets external packages, a centralized server doesn't make much sense indeed. For XEmacs, it was probably a good simplification. > But I do think that we should try to offer some way of installing > external Lisp code if one has acquired it as a zip or tar archive in > some manner already. Not just tarballs but also single elisp files. Most elisp packages are single files. > And in those respects where the directory structure and layout > conventions of an XEmacs package would seem reasonable, there seems > little point in inventing something else. Agreed. > Stefan, having met those problems when writing install.el already, > would probably be a lot more qualified to comment on my impressions > and on the versatility of the package layout he has been catering for. The DWIM-aspect of `install' (just look for *.el and *.info files) only works up to a point. I've bumped into problems where some *.el files should be ignored, or other problems where the *.info file doesn't have the `info' extension, or where the info file simply doesn't exist and it's not clear how to build it based on a bunch of *.texi files, ... We could keep adding intelligence and heuristics, but it makes more sense to devise some conventions. > All of those certainly sound reasonable. In particular, I'd second having > separate commands for installing system-wide and user-local packages. Agreed, it makes more sense. Stefan PS: I attached my latest version, which is still "work in progress". [-- Attachment #2: install.el --] [-- Type: application/emacs-lisp, Size: 22759 bytes --] [-- Attachment #3: Type: text/plain, Size: 142 bytes --] _______________________________________________ Emacs-devel mailing list Emacs-devel@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-devel ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 0:26 ` Stefan @ 2005-01-03 11:05 ` Stephen J. Turnbull 2005-01-03 17:16 ` Stefan Monnier 0 siblings, 1 reply; 30+ messages in thread From: Stephen J. Turnbull @ 2005-01-03 11:05 UTC (permalink / raw) Cc: schierlm, rms, emacs-devel >>>>> "Stefan" == Stefan <monnier@iro.umontreal.ca> writes: Stefan> I tend to think that most of the autoconfiguration should Stefan> be done upon use rather than during installation, which Stefan> basically implies "in elisp rather than in autoconf". A word to the wise: This is expensive at startup, and will probably take some effort to shake out. Mike Sperber put in many hours removing unnecessary stats from our naive package location code, despite the very regular layout of our package hierarchy. It's still pretty straightforward, but the efficiency is not an accident. It would theoretically be possible to have a cache which collects all the auto-autoloads, but we felt that was over-engineering; startup is not something that should happen very often in Emacs. dak> a) a directory layout. XEmacs packages have a standard way dak> of placing Elisp, info, data and other files in a layout dak> under the top package directory. XEmacs's layout is inverted from what makes sense for third-party packages: we have a standard GNU Emacs-style tree with lisp/$package, info/$package.info*, etc as needed. Really, however, third-party packages want $package/{lisp,lib-src,etc,...}. The Emacspeak and JDEE maintainers absolutely insist on this, for two examples. XEmacs is moving toward flattening out the core lisp, and eventually inverting the tree for packages. The only exceptions will be package meta-data (manifests used to ensure that package upgrades and removals get rid of everything installed by unwanted packages), and info files. This is probably not an appropriate strategy for Emacs, but you should be aware that XEmacs is not necessarily going to stick with the current structure. dak> In addition there are some standard file names for dak> initializing of packages and installation. In short, this dak> part of the package system is basically just file layout dak> conventions relative to a package-specific top directory. We have auto-autoloads.el, custom-loads.el, and _pkg.el. The first is what it looks like, the second handles attaching Custom's menus and groups and so on, and the third contains meta-data which actually is inserted into auto-autoloads.el, and never actually loaded itself. Stefan> I can't find any documentation about the in-package layout Stefan> used by XEmacs, tho. Anybody knows where it's described? The hierarchy is simple, and not really explicitly documented. There is some information on internal structure, especially for the source trees, in XEmacs Users' Guide (node Packages) and Lispref (node Packaging), see http://www.xemacs.org/Documentation/21.5/html/. There may be some commentary in the Internals Manual in the "Future Work" nodes, but they're quite disorganized, and mostly written by Ben who is not authoritative for the packages. The installed package hierarchy is as described above: $top/{etc,info,lib-src,lisp,man,pkginfo,...} with packages constrained to install into like-named subdirectories, except for $top/info, where the name should be $package.info*, and $top/pkginfo, which contains the MANIFEST.$package files used when uninstalling. We're policy-free with respect to source directory organization. Basically in source the required files are an XEmacs-style Makefile and a metadata file, package-info.in. The three files referred to above are automatically generated. dak> b) version management. Versions must basically be floats, dak> higher versions correspond to later versions, are usually dak> based on some CVS archive and not at all with actual package dak> versions. Uh. Stefan> Right, it seems to be used only to discover whether a Stefan> newer version is available or not. Yeah, we need to do something about that. I don't know what Steve Baur was thinking. dak> c) a package manager that will download and upgrade packages. dak> It might do so in user-specific and system-wide packages. XEmacs's is currently in flux. Its #1 design bug is absence of an HTTP transport, both because it makes EFS a single point of failure, and because of the firewall issue. However, unless there's some easily findable index of URLs for downloadable packages, this is not terribly useful, I think. dak> I am not convinced about the value of the centralized server dak> approach to package management from XEmacs: it does not seem dak> to lend itself Stefan> Since `install' specifically targets external packages, a Stefan> centralized server doesn't make much sense indeed. For Stefan> XEmacs, it was probably a good simplification. The centralized server is _not_ designed as part of the package management system, but as a service provided to the community. Due to the convenience for maintainers who don't necessarily have commit access to the upstream project (assuming it exists), it has become a dependency. Doing something about that would require some effort. There is one point that needs to be mentioned, however, and that is that for complex packages that depend on other packages, the "calling macros from byte-compiled code" kind of bug has dropped from FAQ to fossil status. dak> But I do think that we should try to offer some way of dak> installing external Lisp code if one has acquired it as a zip dak> or tar archive in some manner already. Stefan> Not just tarballs but also single elisp files. Most elisp Stefan> packages are single files. This we really are not set up to handle efficiently. I think we should, so I'll be interested to see how you handle it. Stefan> The DWIM-aspect of `install' (just look for *.el and Stefan> *.info files) only works up to a point. I've bumped into Stefan> problems where some *.el files should be ignored, or other We handle this by having an explicit list as a make variable. Usually the default setting of "all .el files" works fine, of course, and that's easily implemented with `wildcard'. Stefan> problems where the *.info file doesn't have the `info' Stefan> extension, or where the info file simply doesn't exist and Stefan> it's not clear how to build it based on a bunch of *.texi Stefan> files, ... Yep. Info files and package-specific data (eg, pixmaps, the Yow! database, SGML DTDs, JDEE's Java code, etc) are the reason for 1/2 of the hair in the package build infrastructure. Most of the rest has to do with supporting building on Windows. Stefan> We could keep adding intelligence and heuristics, but it Stefan> makes more sense to devise some conventions. I agree in theory, but in practice, you'll run into resistance (ie, they just won't bother) from 3rd-party maintainers, all the more since you're focusing on "external" packages. dak> All of those certainly sound reasonable. In particular, I'd dak> second having separate commands for installing system-wide dak> and user-local packages. Stefan> Agreed, it makes more sense. XEmacs experience with "guessing" the target tree has not been positive; we probably should consider moving in that direction ourselves. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 11:05 ` Stephen J. Turnbull @ 2005-01-03 17:16 ` Stefan Monnier 2005-01-04 12:00 ` Stephen J. Turnbull 0 siblings, 1 reply; 30+ messages in thread From: Stefan Monnier @ 2005-01-03 17:16 UTC (permalink / raw) Cc: schierlm, rms, emacs-devel Stefan> I tend to think that most of the autoconfiguration should Stefan> be done upon use rather than during installation, which Stefan> basically implies "in elisp rather than in autoconf". > A word to the wise: This is expensive at startup, and will probably > take some effort to shake out. Mike Sperber put in many hours > removing unnecessary stats from our naive package location code, > despite the very regular layout of our package hierarchy. It's still > pretty straightforward, but the efficiency is not an accident. I think we're not talking about the same thing. IIUC You're talking about the time it takes for XEmacs to find all the packages and update the load-path accordingly. `install' does not suffer from this problem since it instead requires the user to write this list directly in his .emacs file (which also has the advantage that the user can choose between Gnus-5.8 and Gnus-5.10 when both are installed). Also `install' only handles external packages which are likely to be much fewer than for XEmacs-with-Sumo. What I was talking about is things like latex-preview searching for some LaTeX files. This is not done at startup: it is currently done at install time, and I was arguing about that it should be done when latex-preview is loaded. Of course it may also take too much time, but doing it at use time has the advantage that it does not require a re-install whenever LaTeX files are moved from /usr/local/texmf to /usr/texmf, or /usr/lib/texmf, or /usr/share/texmf, or /opt/texmf, or god knows where else. Stefan> I can't find any documentation about the in-package layout Stefan> used by XEmacs, tho. Anybody knows where it's described? [...] > We're policy-free with respect to source directory organization. > Basically in source the required files are an XEmacs-style Makefile > and a metadata file, package-info.in. The three files referred to > above are automatically generated. So it's similar to `install' in this sense, except that you require a Makefile to describe where the various files are located. That sounds like a good way to do it (tho for `install', it would make more sense to use a .el file for that info since `install' does the job of `make' for typical simple packages). > There is one point that needs to be mentioned, however, and that is > that for complex packages that depend on other packages, the "calling > macros from byte-compiled code" kind of bug has dropped from FAQ to > fossil status. So there is some amount of version-dependency checking now? Stefan> Not just tarballs but also single elisp files. Most elisp Stefan> packages are single files. > This we really are not set up to handle efficiently. I think we > should, so I'll be interested to see how you handle it. Take a look at install.el. The case of single files was the first that worked. It's much simpler than tarballs: place it in the desired directory, add the autoloads to the directory's autoload file, byte-compile, done. Remember, `install' aims for simplicity and tries as much as possible to stick to what a real user would manually do. I haven't tried to handle autoconf'd packages yet, but there's no reason why `install' couldn't just say: oh wait, there's a `configure' file, let's just run "EMACS=<myself> ./configure --prefix=<myownpackagedir>; make; make install". Stefan ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-03 17:16 ` Stefan Monnier @ 2005-01-04 12:00 ` Stephen J. Turnbull 2005-01-04 13:59 ` Stefan 0 siblings, 1 reply; 30+ messages in thread From: Stephen J. Turnbull @ 2005-01-04 12:00 UTC (permalink / raw) Cc: emacs-devel, schierlm, rms BTW, I'm going to be out of town for a week, so any further responses will be delayed. >>>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes: Stefan> What I was talking about is things like latex-preview Stefan> searching for some LaTeX files. This is not done at Stefan> startup: it is currently done at install time, and I was Stefan> arguing about that it should be done when latex-preview is Stefan> loaded. Of course it may also take too much time, That's my point. People are just not going to find that acceptable, at least they didn't with year 2000 hardware with XEmacs. Even if it's done while you're getting morning coffee---because it will take longer than that. Waiting for it to load every time you reload AUCTeX will wear out the AUCTeX developers. Stefan> So [XEmacs packaging is] similar to `install' in this Stefan> sense, except that you require a Makefile to describe Stefan> where the various files are located. That sounds like a Stefan> good way to do it (tho for `install', it would make more Stefan> sense to use a .el file for that info since `install' does Stefan> the job of `make' for typical simple packages). Agreed. >> There is one point that needs to be mentioned, however, and >> that is that for complex packages that depend on other >> packages, the "calling macros from byte-compiled code" kind of >> bug has dropped from FAQ to fossil status. Stefan> So there is some amount of version-dependency checking Stefan> now? No. These are package dependencies, not version dependencies. If a package uses macros from another one, it is placed in the "REQUIRES" make variable. Then the libraries from those REQUIRE'd packages are preloaded using the -l argument to emacs. Theoretically these could be autogenerated, but the dependency trackers we've seen so far have all been buggy, and miss more dependencies than when doing it by hand. Stefan> Remember, `install' aims for simplicity and tries as much Stefan> as possible to stick to what a real user would manually Stefan> do. Sure. I don't think XEmacs will change its commitment to a more sophisticated ("intrusive and inflexible" if you prefer :-) packaging system. I think we should be able to achieve interoperability in most cases though. -- Institute of Policy and Planning Sciences http://turnbull.sk.tsukuba.ac.jp University of Tsukuba Tennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-04 12:00 ` Stephen J. Turnbull @ 2005-01-04 13:59 ` Stefan 2005-01-04 14:07 ` Miles Bader 0 siblings, 1 reply; 30+ messages in thread From: Stefan @ 2005-01-04 13:59 UTC (permalink / raw) Cc: emacs-devel, schierlm, rms Stefan> What I was talking about is things like latex-preview Stefan> searching for some LaTeX files. This is not done at Stefan> startup: it is currently done at install time, and I was Stefan> arguing about that it should be done when latex-preview is Stefan> loaded. Of course it may also take too much time, > That's my point. People are just not going to find that acceptable, > at least they didn't with year 2000 hardware with XEmacs. But you're comparing apples and oranges. With XEmacs you're talking about lots and lots of searches at every startup, whereas with AUCTeX we're talking about several searches done only when the feature is used. So, yes, maybe it'll be slow, but the XEmacs experience with its package system is no evidence of it. Stefan> So there is some amount of version-dependency checking Stefan> now? > No. These are package dependencies, not version dependencies. If a > package uses macros from another one, it is placed in the "REQUIRES" > make variable. Then the libraries from those REQUIRE'd packages are > preloaded using the -l argument to emacs. > Theoretically these could be autogenerated, but the dependency > trackers we've seen so far have all been buggy, and miss more > dependencies than when doing it by hand. The way to fix those things is to call them bugs. The packages should have (require 'foobar) or (eval-when-compile (require 'foobar)) so that you get a clean error rather than a miscompile. Of course, you don't always have much control over the upstream code, but you can apply local patches. The only case where (require 'foobar) can still lead to miscompile is when old versions of `foobar' don't provide the macro, which is why I thought you used version-dependency checking. Stefan ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: `make' written in elisp 2005-01-04 13:59 ` Stefan @ 2005-01-04 14:07 ` Miles Bader 0 siblings, 0 replies; 30+ messages in thread From: Miles Bader @ 2005-01-04 14:07 UTC (permalink / raw) Cc: Stephen J. Turnbull, schierlm, rms, emacs-devel > > That's my point. People are just not going to find that acceptable, > > at least they didn't with year 2000 hardware with XEmacs. > > But you're comparing apples and oranges. With XEmacs you're talking about > lots and lots of searches at every startup, whereas with AUCTeX we're talking > about several searches done only when the feature is used. Yeah, anyway don't we already do this sort of thing for configuring e.g., grep at runtime? Searching for a few programs and invoking them to test options, _the first time you use a specific feature_, doesn't seems like a big deal at all. Does auxtex require something else? -Miles ^ permalink raw reply [flat|nested] 30+ messages in thread
* patch for locate-file-completion? 2003-12-30 19:31 `make' written in elisp Michael Schierl 2003-12-31 22:57 ` Richard Stallman 2004-01-04 23:25 ` Stefan Monnier @ 2004-03-31 22:31 ` Nic Ferrier 2004-04-01 17:34 ` Richard Stallman 2 siblings, 1 reply; 30+ messages in thread From: Nic Ferrier @ 2004-03-31 22:31 UTC (permalink / raw) I think locate-file-completion is broken, here's how: M-x load-library somepath-that-should-complete TAB Mine causes an exception because of a nil path element in my load-path. I'm not sure why I've got a nil path element in my load-path but the docs for load-path say it is legal to have a nil element. Apologies if it's not really a problem, if there's some reason for the exception I'd like to know. If not, and there is a problem, here's a patch that fixes it: Index: files.el =================================================================== RCS file: /cvsroot/emacs/emacs/lisp/files.el,v retrieving revision 1.686 diff -c -r1.686 files.el *** files.el 27 Mar 2004 04:05:23 -0000 1.686 --- files.el 31 Mar 2004 22:26:37 -0000 *************** *** 617,623 **** (string-dir (file-name-directory string))) (dolist (dir (car path-and-suffixes)) (if string-dir (setq dir (expand-file-name string-dir dir))) ! (when (file-directory-p dir) (dolist (file (file-name-all-completions (file-name-nondirectory string) dir)) (push (if string-dir (concat string-dir file) file) names) --- 617,623 ---- (string-dir (file-name-directory string))) (dolist (dir (car path-and-suffixes)) (if string-dir (setq dir (expand-file-name string-dir dir))) ! (when (and dir (file-directory-p dir)) (dolist (file (file-name-all-completions (file-name-nondirectory string) dir)) (push (if string-dir (concat string-dir file) file) names) -- Nic Ferrier http://www.tapsellferrier.co.uk ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: patch for locate-file-completion? 2004-03-31 22:31 ` patch for locate-file-completion? Nic Ferrier @ 2004-04-01 17:34 ` Richard Stallman 0 siblings, 0 replies; 30+ messages in thread From: Richard Stallman @ 2004-04-01 17:34 UTC (permalink / raw) Cc: emacs-devel It looks to me like right fix is this one. Does this look right to you? diff -c /home/rms/emacs/lisp/files.el.\~80\~ /home/rms/emacs/lisp/files.el *** /home/rms/emacs/lisp/files.el.~80~ Mon Mar 29 23:58:25 2004 --- /home/rms/emacs/lisp/files.el Thu Apr 1 10:31:51 2004 *************** *** 616,621 **** --- 616,623 ---- (suffix (concat (regexp-opt (cdr path-and-suffixes) t) "\\'")) (string-dir (file-name-directory string))) (dolist (dir (car path-and-suffixes)) + (unless dir + (setq dir default-directory)) (if string-dir (setq dir (expand-file-name string-dir dir))) (when (file-directory-p dir) (dolist (file (file-name-all-completions Diff finished. Thu Apr 1 10:32:37 2004 ^ permalink raw reply [flat|nested] 30+ messages in thread
[parent not found: <E1CloEh-0004Sl-Hg@monty-python.gnu.org>]
* Re: `make' written in elisp [not found] <E1CloEh-0004Sl-Hg@monty-python.gnu.org> @ 2005-01-04 16:10 ` Eric M. Ludlam 0 siblings, 0 replies; 30+ messages in thread From: Eric M. Ludlam @ 2005-01-04 16:10 UTC (permalink / raw) [ ... ] >> Besides writing values to init files the configuration process is used >> to check if external tools required for building or running the >> package are present and provide necessary features. A special case >> might be preview-latex which has a TeX part besides the Elisp part. >> >> Why is it necessary to do this? >> It seems to me that it would be just fine >> to look for these things when the user tries to use them. >> >> to try to use the command when the user asks to do > >I don't think it is feasible to check for program versions every time >a program is called. And instead of letting the user alone with error >messages which don't give him a hint that the program is too old, I >consider it better practice to tell him when he tries to install the >software which relies on the external program. Hi, On the topic of checking version numbers at runtime, I wrote a tool to do this for me which is in the CEDET stuff I maintain. Many tools have a variable, for example, foo.el, might have foo-version be a string that is the version number. This tool can scan for that, and make sure the version you want is compatible via various settings. I also use it to manager save file version numbers so that upgrades can occur among various tools that are not a part of Emacs. The one dilemma is that a package author has to explicitly specify when a version is no longer compatible with older versions. ie, current version is 2.4, but tools depending on 1.5 won't work with 2.4, this needs to be specified for inversion to handle all cases. Here is a CVS link: http://cvs.sourceforge.net/viewcvs.py/cedet/cedet/common/inversion.el?rev=1.20&view=log Enjoy Eric -- Eric Ludlam: zappo@gnu.org, eric@siege-engine.com Home: http://www.ludlam.net Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net GNU: www.gnu.org ^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2005-01-04 16:10 UTC | newest] Thread overview: 30+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2003-12-30 19:31 `make' written in elisp Michael Schierl 2003-12-31 22:57 ` Richard Stallman 2003-12-31 23:14 ` Michael Schierl 2004-01-01 21:10 ` Richard Stallman 2004-01-01 21:37 ` Michael Schierl 2004-01-04 23:25 ` Stefan Monnier 2005-01-02 16:06 ` Richard Stallman 2005-01-02 23:22 ` David Kastrup 2005-01-02 23:55 ` Ralf Angeli 2005-01-03 0:07 ` David Kastrup 2005-01-03 0:25 ` Ralf Angeli 2005-01-03 4:32 ` Richard Stallman 2005-01-03 8:02 ` David Kastrup 2005-01-03 9:36 ` Eli Zaretskii 2005-01-03 16:45 ` Stefan Monnier 2005-01-03 9:10 ` Ralf Angeli 2005-01-03 18:29 ` Richard Stallman 2005-01-03 19:28 ` Ralf Angeli 2005-01-03 23:10 ` David Kastrup 2005-01-04 3:38 ` Richard Stallman 2005-01-04 11:17 ` Ralf Angeli 2005-01-03 0:26 ` Stefan 2005-01-03 11:05 ` Stephen J. Turnbull 2005-01-03 17:16 ` Stefan Monnier 2005-01-04 12:00 ` Stephen J. Turnbull 2005-01-04 13:59 ` Stefan 2005-01-04 14:07 ` Miles Bader 2004-03-31 22:31 ` patch for locate-file-completion? Nic Ferrier 2004-04-01 17:34 ` Richard Stallman [not found] <E1CloEh-0004Sl-Hg@monty-python.gnu.org> 2005-01-04 16:10 ` `make' written in elisp Eric M. Ludlam
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).