bug#10970: 24.0.93; package.el doesn't load autoloads before compiling

bug#10970: 24.0.93; package.el doesn't load autoloads before compiling

[Donald Ephraim Curtis]

Occasionally, when you install a package via `package.el' you have a
package that stores it's elisp files in a subdirectory under the main
package directory.  One solution is to add elisp like,

(when (boundp 'load-path)
  (add-to-list 'load-path
               (file-name-as-directory
                (expand-file-name
                 "lisp"
                 (file-name-directory load-file-name)))))

where "lisp" is the subdirectory under the package directory containing
the elisp files (this example is ESS).

However, before compiling, the autoloads file (pkg-autoloads.el) is not
loaded, so anything that updates the load-path is also not sourced.

It may be best that packages simply put all their lisp files in the main
package directory, but it also may be worthwhile to simply add,

(load (expand-file-name (concat (symbol-name name) "-autoloads") pkg-dir) nil t)

in the `package-unpack' function after the autoloads are generated.
There may also be some problems with this that I am not seeing.

      


In GNU Emacs 24.0.93.1 (i386-apple-darwin11.3.0, NS apple-appkit-1138.32)
of 2012-02-07 on bird.local
Windowing system distributor `Apple', version 10.3.1138
Configured using:
`configure '--prefix=/usr/local/Cellar/emacs/HEAD' '--without-dbus'
'--enable-locallisppath=/usr/local/share/emacs/site-lisp'
'--infodir=/usr/local/Cellar/emacs/HEAD/share/info/emacs' '--with-ns'
'--disable-ns-self-contained' 'CC=/usr/bin/llvm-gcc' 'CFLAGS=-O3 -w
-pipe -march=core2 -msse4''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: Package Menu

Minor modes in effect:
  shell-dirtrack-mode: t
  savehist-mode: t
  recentf-mode: t
  ido-ubiquitous-mode: t
  ido-everywhere: t
  show-paren-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-x b C-g C-x RET p a c k <return> C-n C-n C-n C-p 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n i x C-g C-x b i n i <return> C-e C-x 
C-e C-x b <return> x y C-n C-n C-n C-n C-x o C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-s E r r o r C-s C-s C-s 
C-g C-g C-p C-p C-p C-p C-p C-p C-p C-x o C-p C-p C-p 
C-p C-p C-p C-p C-p C-p C-p C-p C-p C-n C-n C-n C-n 
C-n C-n C-x RET b u g C-n C-n <return>

Recent messages:
Checking /Users/dcurtis/.emacs.d/elpa/ess-20120305/etc/useR-2006-ESS/RefCards...
Checking /Users/dcurtis/.emacs.d/elpa/ess-20120305/etc/useR-2006-ESS/Lab1/auto...
Compiling /Users/dcurtis/.emacs.d/elpa/ess-20120305/etc/useR-2006-ESS/Lab1/auto/cph-1-lab.el...done
Wrote /Users/dcurtis/.emacs.d/elpa/ess-20120305/etc/useR-2006-ESS/Lab1/auto/cph-1-lab.elc
Checking /Users/dcurtis/.emacs.d/elpa/ess-20120305/etc/useR-2006-ESS/Lab1/auto...
Checking /Users/dcurtis/.emacs.d/elpa/ess-20120305/etc/useR-2006-ESS/Lab2/auto...
Compiling /Users/dcurtis/.emacs.d/elpa/ess-20120305/etc/useR-2006-ESS/Lab2/auto/Sweave-Example.el...done
Wrote /Users/dcurtis/.emacs.d/elpa/ess-20120305/etc/useR-2006-ESS/Lab2/auto/Sweave-Example.elc
Checking /Users/dcurtis/.emacs.d/elpa/ess-20120305/etc/useR-2006-ESS/Lab2/auto...
Done (Total of 68 files compiled in 6 directories)
Quit

Load-path shadows:
~/.emacs.d/custom hides /usr/local/Cellar/emacs/HEAD/share/emacs/24.0.93/lisp/custom
/Users/dcurtis/.emacs.d/elpa/ruby-mode-20120122/ruby-mode hides /usr/local/Cellar/emacs/HEAD/share/emacs/24.0.93/lisp/progmodes/ruby-mode
/Users/dcurtis/.emacs.d/elpa/magit-20120213/.dir-locals hides /usr/local/Cellar/emacs/HEAD/share/emacs/24.0.93/lisp/gnus/.dir-locals
~/.emacs.d/package hides /usr/local/Cellar/emacs/HEAD/share/emacs/24.0.93/lisp/emacs-lisp/package

Features:
(shadow sort flyspell ispell mail-extr message rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mailabbrev gmm-utils mailheader emacsbug
multi-isearch hideshow ess-help ess-dde ess-omg-l reporter tramp
tramp-compat format-spec tramp-loaddefs ess-toolbar ess-mouse mouseme
browse-url ess-menu speedbar sb-image ezimage dframe ess-swv ess-noweb
essd-els ess-sas-d ess-sas-l ess-sas-a executable shell pcomplete
ess-arc-d ess-vst-d ess-xls-d ess-sta-d ess-sta-l cc-vars cc-defs
make-regexp ess-sp6-d ess-sp5-d ess-sp3-d ess-r-d ess-tracebug
ess-developer ess-r-args ess-s-l ess-inf ess-mode noweb-mode ess
ess-custom ess-site ess-utils ess-bugs-l ess-compat ess-lsp-l compile
comint noutline outline easy-mmode regexp-opt autoload tar-mode
time-stamp vc-git mail-utils network-stream starttls url-cache url-http
tls mail-parse rfc2231 rfc2047 rfc2045 ietf-drums url-gw url-auth url
url-proxy url-privacy url-expand url-methods url-history url-cookie
url-util url-parse auth-source eieio gnus-util password-cache url-vars
mm-util mail-prsvr mailcap finder-inf eldoc elisp-slime-nav etags ring
paredit hl-line saveplace idle-highlight-mode thingatpt dropdown-list
yasnippet help-mode view assoc rainbow-delimiters whitespace melpa
sanityinc-tomorrow-bright-theme color-theme-sanityinc-tomorrow savehist
recentf tree-widget wid-edit autorevert cua-base cus-start cus-load
server misc midnight auctex-autoloads tex-site info easymenu
clojure-test-mode-autoloads clojurescript-mode-autoloads
clojure-mode-autoloads color-theme-sanityinc-solarized-autoloads
color-theme-sanityinc-tomorrow-autoloads deft-autoloads evil-autoloads
flymake-cursor-autoloads full-ack-autoloads haskell-mode-autoloads
markdown-mode+-autoloads markdown-mode-autoloads melpa-autoloads
byte-opt warnings bytecomp byte-compile cconv macroexp
multi-term-autoloads pastels-on-dark-theme-autoloads
rainbow-delimiters-autoloads rinari-autoloads jump-autoloads
inflections-autoloads findr-autoloads ruby-compilation-autoloads
ruby-mode-autoloads slime-autoloads starter-kit-bindings-autoloads
windmove starter-kit-eshell-autoloads starter-kit-lisp-autoloads
elisp-slime-nav-autoloads starter-kit-ruby-autoloads
starter-kit-autoloads edmacro kmacro smex starter-kit-misc ffap
ido-ubiquitous ido paren starter-kit-defuns uniquify advice help-fns
advice-preload magit-autoloads ido-ubiquitous-autoloads smex-autoloads
find-file-in-project-autoloads idle-highlight-mode-autoloads
paredit-autoloads inf-ruby-autoloads undo-tree-autoloads
yasnippet-autoloads package derived tabulated-list defun cl time-date
tooltip ediff-hook vc-hooks lisp-float-type mwheel ns-win tool-bar dnd
fontset image fringe lisp-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer loaddefs button faces cus-face files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process ns
multi-tty emacs)

bug#10970: 24.0.93; package.el doesn't load autoloads before compiling

[Chong Yidong]

Donald Ephraim Curtis <dcurtis@milkbox.net> writes:

> However, before compiling, the autoloads file (pkg-autoloads.el) is not
> loaded, so anything that updates the load-path is also not sourced.
>
> It may be best that packages simply put all their lisp files in the main
> package directory, but it also may be worthwhile to simply add,
>
> (load (expand-file-name (concat (symbol-name name) "-autoloads") pkg-dir) nil t)
>
> in the `package-unpack' function after the autoloads are generated.

I see the problem.  OTOH I don't see any actual bug that would arise
from loading the autoloads file before byte compilation.  But on the
other hand, it's kind of conceptually nice that the autoloads file is
only loaded when the package is activated, as a separate step from
unpacking.

Maybe it's better for the affected files to use `eval-when-compile' if
they need to alter the load path.

Anyone have another opinion?

bug#10970: 24.0.93; package.el doesn't load autoloads before compiling

[Stefan Monnier]

> I see the problem.  OTOH I don't see any actual bug that would arise
> from loading the autoloads file before byte compilation.

I think it's the right thing to do.

> But on the other hand, it's kind of conceptually nice that the
> autoloads file is only loaded when the package is activated, as
> a separate step from unpacking.

It would be separate, because byte-compiling is a separate step
from unpacking (although it's almost always done together).


        Stefan

bug#10970: 24.0.93; package.el doesn't load autoloads before compiling

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I see the problem.  OTOH I don't see any actual bug that would arise
>> from loading the autoloads file before byte compilation.
>
> I think it's the right thing to do.

Fine by me.  I committed a patch to do that.