Re: master 1d4e903417: Speed up generation of loaddefs files

Re: master 1d4e903417: Speed up generation of loaddefs files

[Basil L. Contovounesios]

[-- Attachment #1: Type: text/plain, Size: 2318 bytes --]

Lars Ingebrigtsen [2022-05-31 12:09 -0400] wrote:

> branch: master
> commit 1d4e90341782030cc7d8c29c639450b079587908
> Author: Lars Ingebrigtsen <larsi@gnus.org>
> Commit: Lars Ingebrigtsen <larsi@gnus.org>
>
>     Speed up generation of loaddefs files

[...]

>     * lisp/emacs-lisp/package.el: Use loaddefs-generate instead of
>     make-directory-autoloads.

[...]

> diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
> index b340848a6f..48551f59b4 100644
> --- a/lisp/emacs-lisp/package.el
> +++ b/lisp/emacs-lisp/package.el
> @@ -1003,6 +1003,7 @@ untar into a directory named DIR; otherwise, signal an error."
>  
>  (defun package-autoload-ensure-default-file (file)
>    "Make sure that the autoload file FILE exists and if not create it."
> +  (declare (obsolete nil "29.1"))
>    (unless (file-exists-p file)
>      (require 'autoload)
>      (let ((coding-system-for-write 'utf-8-emacs-unix))
> @@ -1021,8 +1022,11 @@ untar into a directory named DIR; otherwise, signal an error."
>           (autoload-timestamps nil)
>           (backup-inhibited t)
>           (version-control 'never))
> -    (package-autoload-ensure-default-file output-file)
> -    (make-directory-autoloads pkg-dir output-file)
> +    (loaddefs-generate
> +     pkg-dir output-file
> +     nil
> +     "(add-to-list 'load-path (directory-file-name
> +                         (or (file-name-directory #$) (car load-path))))")
>      (let ((buf (find-buffer-visiting output-file)))
>        (when buf (kill-buffer buf)))
>      auto-name))

Since this change, installation and activation of packages lacking
autoloads signals a (demoted) error.  For example:

$ cd "$(mktemp -d)"
$ export HOME="$PWD" XDG_CONFIG_HOME="$PWD/.config"
$ emacs -Q -f toggle-debug-on-error
(setq package-archives '(("melpa" . "https://melpa.org/packages/")))
C-x C-e
M-x package-install RET sicp RET

Error loading autoloads:
(file-missing
 Cannot open load file
 No such file or directory
 /tmp/tmp.6gK5WLWds9/.emacs.d/elpa/sicp-20200512.1137/sicp-autoloads)

Backtrace attached.  The same error occurs with a subsequent:

C-x C-c
$ emacs -Q
M-x package-initialize RET

I guess either the -autoloads.el file should continue to be generated
unconditionally, or its nonexistence should not be regarded as an error.

Thanks,

-- 
Basil


[-- Attachment #2: backtrace.txt --]
[-- Type: text/plain, Size: 3691 bytes --]

Debugger entered--Lisp error: (file-missing "Cannot open load file" "No such file or directory" "/tmp/tmp.6gK5WLWds9/.emacs.d/elpa/sicp-20200512.1137/sicp-autoloads")
  load("/tmp/tmp.6gK5WLWds9/.emacs.d/elpa/sicp-20200512.1137/sicp-autoloads" nil t)
  package-activate-1(#s(package-desc :name sicp :version (20200512 1137) :summary "Structure and Interpretation of Computer Programs in info format" :reqs nil :kind nil :archive nil :dir "/tmp/tmp.6gK5WLWds9/.emacs.d/elpa/sicp-20200512.1137" :extras ((:url . "https://mitpress.mit.edu/sicp") (:maintainer "Hal Abelson") (:authors ("Hal Abelson") ("Jerry Sussman") ("Julie Sussman")) (:commit . "4002d83083d520c6b5ede2df36cc2cee885d450a")) :signed nil) :reload :deps)
  package-unpack(#s(package-desc :name sicp :version (20200512 1137) :summary "Structure and Interpretation of Computer Programs in info format" :reqs nil :kind tar :archive "melpa" :dir nil :extras ((:commit . "4002d83083d520c6b5ede2df36cc2cee885d450a") (:authors ("Hal Abelson") ("Jerry Sussman") ("Julie Sussman")) (:maintainer "Hal Abelson") (:url . "https://mitpress.mit.edu/sicp")) :signed nil))
  #f(compiled-function (&optional good-sigs) #<bytecode 0x11cd866634739475>)(nil)
  #f(compiled-function () #<bytecode 0x1a1912937020e64e>)()
  package--with-response-buffer-1("https://melpa.org/packages/" #f(compiled-function () #<bytecode 0x6c060a56f580dc5>) :file "sicp-20200512.1137.tar.sig" :async nil :error-function #f(compiled-function () #<bytecode 0x1a1912937020e64e>) :noerror t)
  package--check-signature("https://melpa.org/packages/" "sicp-20200512.1137.tar" "sicp-20200512.1137/\0\0\0..." nil #f(compiled-function (&optional good-sigs) #<bytecode 0x11cd866634739475>))
  #f(compiled-function () #<bytecode -0xfbc3e316bb0b219>)()
  package--with-response-buffer-1("https://melpa.org/packages/" #f(compiled-function () #<bytecode -0xfbc3e316bb0b219>) :file "sicp-20200512.1137.tar" :async nil :error-function #f(compiled-function () #<bytecode 0x1dc000153e91>) :noerror nil)
  package-install-from-archive(#s(package-desc :name sicp :version (20200512 1137) :summary "Structure and Interpretation of Computer Programs in info format" :reqs nil :kind tar :archive "melpa" :dir nil :extras ((:commit . "4002d83083d520c6b5ede2df36cc2cee885d450a") (:authors ("Hal Abelson") ("Jerry Sussman") ("Julie Sussman")) (:maintainer "Hal Abelson") (:url . "https://mitpress.mit.edu/sicp")) :signed nil))
  mapc(package-install-from-archive (#s(package-desc :name sicp :version (20200512 1137) :summary "Structure and Interpretation of Computer Programs in info format" :reqs nil :kind tar :archive "melpa" :dir nil :extras ((:commit . "4002d83083d520c6b5ede2df36cc2cee885d450a") (:authors ("Hal Abelson") ("Jerry Sussman") ("Julie Sussman")) (:maintainer "Hal Abelson") (:url . "https://mitpress.mit.edu/sicp")) :signed nil)))
  package-download-transaction((#s(package-desc :name sicp :version (20200512 1137) :summary "Structure and Interpretation of Computer Programs in info format" :reqs nil :kind tar :archive "melpa" :dir nil :extras ((:commit . "4002d83083d520c6b5ede2df36cc2cee885d450a") (:authors ("Hal Abelson") ("Jerry Sussman") ("Julie Sussman")) (:maintainer "Hal Abelson") (:url . "https://mitpress.mit.edu/sicp")) :signed nil)))
  package-install(sicp nil)
  funcall-interactively(package-install sicp nil)
  call-interactively(package-install record nil)
  command-execute(package-install record)
  execute-extended-command(nil "package-install" "package-install")
  funcall-interactively(execute-extended-command nil "package-install" "package-install")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

[-- Attachment #3: Type: text/plain, Size: 3093 bytes --]


In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu)
 of 2022-06-11 built on tia
Repository revision: 1d4e90341782030cc7d8c29c639450b079587908
Repository branch: HEAD
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --prefix=/home/blc/.local --with-x-toolkit=no --without-x'

Configured features:
ACL DBUS GMP GNUTLS GPM JSON LCMS2 LIBSELINUX LIBSYSTEMD LIBXML2 MODULES
NOTIFY INOTIFY PDUMPER SECCOMP SOUND SQLITE3 THREADS XIM ZLIB

Important settings:
  value of $LANG: en_IE.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Debugger

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug cl-extra help-fns cl-print debug
backtrace help-mode tool-bar find-func loaddefs-gen lisp-mnt radix-tree
tar-mode arc-mode archive-mode cus-edit pp wid-edit mm-archive message
sendmail yank-media dired dired-loaddefs rfc822 mml mml-sec epa derived
gnus-util text-property-search time-date mailabbrev gmm-utils mailheader
mm-decode mm-bodies mm-encode mail-utils gnutls network-stream url-http
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr url-gw
nsm rmc puny url-cache url-auth epg rfc6068 epg-config finder-inf
package browse-url url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map url-vars seq cl-loaddefs cl-lib cus-start
cus-load term/xterm xterm gv subr-x byte-opt bytecomp byte-compile cconv
regexp-opt iso-transl tooltip eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice simple cl-generic indonesian
philippine cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese composite
emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help
abbrev obarray oclosure cl-preloaded button loaddefs faces cus-face
macroexp files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 258881 11354)
 (symbols 48 15448 0)
 (strings 32 84223 1957)
 (string-bytes 1 3537351)
 (vectors 16 22445)
 (vector-slots 8 276160 19702)
 (floats 8 43 252)
 (intervals 56 795 162)
 (buffers 992 15))

Re: master 1d4e903417: Speed up generation of loaddefs files

[Lars Ingebrigtsen]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> Backtrace attached.  The same error occurs with a subsequent:
>
> C-x C-c
> $ emacs -Q
> M-x package-initialize RET
>
> I guess either the -autoloads.el file should continue to be generated
> unconditionally, or its nonexistence should not be regarded as an error.

I guess generating the file unconditionally would be possible...  but
perhaps it'd be better if this didn't signal an error?  Perhaps Stefan
has an opinion here; added to the CCs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: master 1d4e903417: Speed up generation of loaddefs files

[Stefan Monnier]

Lars Ingebrigtsen [2022-06-11 13:03:40] wrote:

> "Basil L. Contovounesios" <contovob@tcd.ie> writes:
>
>> Backtrace attached.  The same error occurs with a subsequent:
>>
>> C-x C-c
>> $ emacs -Q
>> M-x package-initialize RET
>>
>> I guess either the -autoloads.el file should continue to be generated
>> unconditionally, or its nonexistence should not be regarded as an error.
>
> I guess generating the file unconditionally would be possible...  but
> perhaps it'd be better if this didn't signal an error?  Perhaps Stefan
> has an opinion here; added to the CCs.

Even in the absence of any autoload cookies, the generated file is
supposed to contain:

     "(add-to-list 'load-path (directory-file-name
                         (or (file-name-directory #$) (car load-path))))")

so it's necessary to produce the file.


        Stefan

Re: master 1d4e903417: Speed up generation of loaddefs files

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Even in the absence of any autoload cookies, the generated file is
> supposed to contain:
>
>      "(add-to-list 'load-path (directory-file-name
>                          (or (file-name-directory #$) (car load-path))))")
>
> so it's necessary to produce the file.

Ah, right.  I've now fixed this on the trunk, I think -- at least
Basil's test case works correctly for me now.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: master 1d4e903417: Speed up generation of loaddefs files

[Basil L. Contovounesios]

Lars Ingebrigtsen [2022-06-11 17:59 +0200] wrote:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> Even in the absence of any autoload cookies, the generated file is
>> supposed to contain:
>>
>>      "(add-to-list 'load-path (directory-file-name
>>                          (or (file-name-directory #$) (car load-path))))")
>>
>> so it's necessary to produce the file.
>
> Ah, right.  I've now fixed this on the trunk, I think -- at least
> Basil's test case works correctly for me now.

LFTM here as well, thanks,

-- 
Basil