bug#53527: 29.0.50; package-reinstall fails to initialize (enough of) package.el

bug#53527: 29.0.50; package-reinstall fails to initialize (enough of) package.el

[Matt Armstrong]

The `package-reinstall' function is broken when called before package.el
is initialized, or more precisely, before `package-load-all-descriptors'
has been called.

It looks like most of the other top level functions in package.el
attempt to handle this case, and do it successfully.

I had trouble formulating an emacs -Q repro because Emacs must be in a
"quick start" state, which requires installation of at least one package
and then restarting Emacs.  I could not figure out how to do that simply
and hermetically without a lot of infrastructure (like a separate user
on a Linux system, or a VM, or https://github.com/plexus/chemacs2).

Stefan's commit dde33727b2ace3ce417d97475d074f0a82b7c2b8 on 2019-01-22
introduced this function and called it in several places but not
`package-reinstall'.

+(defun package--alist ()
+  "Return `package-alist', after computing it if needed."
+  (or package-alist
+      (progn (package-load-all-descriptors)
+             package-alist)))

I found this bug after removing a long standing call to
`package-initialize` from my init.el, then calling this function of
mine:

(defun my-reinstall-packages ()
  "Re-install all Emacs packages that are installed presently."
  (interactive)
  (dolist (package-name package-activated-list)
    (message (format "Reinstalling %s" package-name))
    (package-reinstall package-name)))


I confirmed this bug is in Emacs 27 through master.


In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.31, cairo version 1.16.0)
 of 2022-01-13 built on naz
Repository revision: 53dba3202205b5fab078a47da77be4140b7f93b1
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --with-native-compilation'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  bug-reference-prog-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  auto-revert-mode: t
  pyvenv-mode: t
  msb-mode: t
  display-time-mode: t
  shell-dirtrack-mode: t
  auto-insert-mode: t
  keyfreq-autosave-mode: t
  keyfreq-mode: t
  icomplete-mode: t
  savehist-mode: t
  vertico-mode: t
  which-key-mode: t
  electric-pair-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
~/env/elisp/ol-notmuch hides /home/matt/.config/emacs/elpa/ol-notmuch-20210530.2054/ol-notmuch
/home/matt/.config/emacs/elpa/transient-20220121.1603/transient hides /home/matt/git/emacs-build/lisp/transient

Features:
(shadow sort mail-extr emacsbug sendmail notmuch notmuch-tree
notmuch-jump notmuch-hello notmuch-show notmuch-print notmuch-crypto
notmuch-mua notmuch-message notmuch-draft notmuch-maildir-fcc
notmuch-address notmuch-company notmuch-parser notmuch-wash coolj
notmuch-query icalendar diary-lib diary-loaddefs notmuch-tag notmuch-lib
notmuch-compat hl-line mm-view mml-smime smime dig goto-addr
bug-reference magit-extras magit-bookmark magit-submodule magit-obsolete
magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull
magit-fetch magit-clone magit-remote magit-commit magit-sequence
magit-notes magit-worktree magit-tag magit-merge magit-branch
magit-reset magit-files magit-refs magit-status magit magit-repos
magit-apply magit-wip magit-log which-func magit-diff smerge-mode diff
git-commit log-edit message yank-media rmc puny rfc822 mml mml-sec epa
derived epg rfc6068 epg-config gnus-util mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log magit-core
magit-autorevert autorevert magit-margin magit-transient magit-process
with-editor magit-mode transient comp comp-cstr magit-git magit-section
meson-mode dabbrev markdown-mode color vc-mtn vc-hg vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs vc eglot array filenotify jsonrpc ert ewoc
debug backtrace tabify imenu dired-aux dired dired-loaddefs mule-util
jka-compr magit-utils crm dash copyright sh-script smie executable
misearch multi-isearch pylint company-oddmuse company-keywords
company-etags company-gtags company-dabbrev-code company-dabbrev
company-files company-clang company-template company-cmake company-bbdb
yasnippet highlight-indentation flymake-proc flymake warnings thingatpt
company-capf company pcase help-fns radix-tree elpy elpy-rpc pyvenv
eshell esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module
esh-groups esh-util elpy-shell elpy-profile elpy-django s elpy-refactor
ido grep compile etags fileloop generator xref project cus-edit
cus-start cus-load wid-edit make-mode man protbuf cc-mode cc-fonts
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
vc-git diff-mode vc-dispatcher bookmark text-property-search pp
face-remap msb time clang-format xml server dirtrack ob-ruby ob-python
python tramp-sh tramp tramp-loaddefs trampver tramp-integration files-x
tramp-compat shell parse-time ls-lisp ob-dot org-protocol org ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src
ob-comint org-pcomplete pcomplete comint ansi-color ring org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp
ob-core ob-eval org-table oc-basic bibtex iso8601 time-date org-keys oc
org-loaddefs find-func cal-menu calendar cal-loaddefs ol-notmuch ol rx
org-compat org-macs format-spec skeleton autoinsert advice keyfreq
icomplete edmacro kmacro savehist vertico orderless modus-operandi-theme
modus-themes which-key cl-extra help-mode elec-pair use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core finder-inf
package browse-url url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map url-vars seq gv subr-x byte-opt bytecomp
byte-compile cconv cl-loaddefs cl-lib info iso-transl tooltip eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice
button loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 648085 59613)
 (symbols 48 38951 83)
 (strings 32 141858 8213)
 (string-bytes 1 5316259)
 (vectors 16 70636)
 (vector-slots 8 1253041 72408)
 (floats 8 437 990)
 (intervals 56 21885 480)
 (buffers 992 61))

bug#53527: 29.0.50; package-reinstall fails to initialize (enough of) package.el

[Matt Armstrong]

[-- Attachment #1: Type: text/plain, Size: 335 bytes --]

tags 53527 + patch
quit

This is one way to fix it.  Another would have been to call
`package--alist' instead of reference `package-alist' in
`package-reinstall'.  ...but that wouldn't have been an optimization,
since `package-install' inevitably calls `package--archives-initialize'
anyway.  This patch just makes it happen earlier.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Patch to fix Bug#53527 --]
[-- Type: text/x-diff, Size: 1538 bytes --]

From fdf15b2985d237a70277362ad0b7d771f20c5f38 Mon Sep 17 00:00:00 2001
From: Matt Armstrong <matt@rfc20.org>
Date: Tue, 25 Jan 2022 11:22:10 -0800
Subject: [PATCH] Fix calling `package-reinstall' just after quick
 initialization.

* lisp/emacs-lisp/package.el (package-reinstall): call
`package--archives-initialize', just like `package-install' does.
This populates `package-alist', and so fixes calling
`package-reinstall' as the first thing done after package "quick
init".  (Bug#53527)
---
 lisp/emacs-lisp/package.el | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index 7679ba2fae..6a8692c983 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2379,10 +2379,14 @@ package-reinstall
   "Reinstall package PKG.
 PKG should be either a symbol, the package name, or a `package-desc'
 object."
-  (interactive (list (intern (completing-read
-                              "Reinstall package: "
-                              (mapcar #'symbol-name
-                                      (mapcar #'car package-alist))))))
+  (interactive
+   (progn
+     (package--archives-initialize)
+     (list (intern (completing-read
+                    "Reinstall package: "
+                    (mapcar #'symbol-name
+                            (mapcar #'car package-alist)))))))
+  (package--archives-initialize)
   (package-delete
    (if (package-desc-p pkg) pkg (cadr (assq pkg package-alist)))
    'force 'nosave)
-- 
2.34.1

bug#53527: 29.0.50; package-reinstall fails to initialize (enough of) package.el

[Lars Ingebrigtsen]

Matt Armstrong <matt@rfc20.org> writes:

> * lisp/emacs-lisp/package.el (package-reinstall): call
> `package--archives-initialize', just like `package-install' does.
> This populates `package-alist', and so fixes calling
> `package-reinstall' as the first thing done after package "quick
> init".  (Bug#53527)

It seems like this patch was missed; sorry.

The change looks reasonable to me, but just double-checking before
applying it -- is it still needed on the current trunk?  I vaguely seem
to remember there being some other work done in this area recently, but
I may well be mistaken.

bug#53527: 29.0.50; package-reinstall fails to initialize (enough of) package.el

[Matt Armstrong]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Matt Armstrong <matt@rfc20.org> writes:
>
>> * lisp/emacs-lisp/package.el (package-reinstall): call
>> `package--archives-initialize', just like `package-install' does.
>> This populates `package-alist', and so fixes calling
>> `package-reinstall' as the first thing done after package "quick
>> init".  (Bug#53527)
>
> It seems like this patch was missed; sorry.
>
> The change looks reasonable to me, but just double-checking before
> applying it -- is it still needed on the current trunk?  I vaguely seem
> to remember there being some other work done in this area recently, but
> I may well be mistaken.

Hey Lars, I just verified that the bug is still there on current master,
and that my patch fixes it.  Looking at the git log there are a few
commits to package.el related to fixes, but none directly related to
this issue.

Thanks!

bug#53527: 29.0.50; package-reinstall fails to initialize (enough of) package.el

[Lars Ingebrigtsen]

Matt Armstrong <matt@rfc20.org> writes:

> Hey Lars, I just verified that the bug is still there on current master,
> and that my patch fixes it.  Looking at the git log there are a few
> commits to package.el related to fixes, but none directly related to
> this issue.

Thanks for checking -- I've now pushed your patch to Emacs 29.