bug#60155: 29.0.60; package-vc doesn't detect subdirectories when installing from repositories

bug#60155: 29.0.60; package-vc doesn't detect subdirectories when installing from repositories

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 721 bytes --]


  M-x package-vc-install RET https://codeberg.org/martianh/mastodon.el RET

This will fetch the package sources, but since the actual code is
located in a "lisp" subdirectory, none of the actual files will be added
to the load-path, making it appear as tough nothing were installed.

This could be fixed by adding a heuristic to package-vc--unpack-1 that
checks if there is a ./lisp/ directory (with .el files?) and that would
add that to the load-path.  Are there any other directory names or
structures that should be considered as well?  Is it common for packages
to have loadable files in both the root directory of a repository and a
sub-directory?

I could imagine that this would be enough to resolve the issue:


[-- Attachment #2: Type: text/plain, Size: 725 bytes --]

diff --git a/lisp/emacs-lisp/package-vc.el b/lisp/emacs-lisp/package-vc.el
index 8f0eedd2f8..eea5ad6c26 100644
--- a/lisp/emacs-lisp/package-vc.el
+++ b/lisp/emacs-lisp/package-vc.el
@@ -443,6 +443,12 @@ package-vc--unpack-1
            (auto-name (format "%s-autoloads.el" name))
            (extras (package-desc-extras pkg-desc))
            (lisp-dir (alist-get :lisp-dir extras)))
+      ;; Heuristic to guess a sub-directory with lisp files.
+      (when-let (((null lisp-dir))
+                 (dir (expand-file-name "lisp" pkg-dir))
+                 ((file-directory-p dir)))
+        (setq lisp-dir dir))
+
       (package-generate-autoloads
        name (file-name-concat pkg-dir lisp-dir))
       (when lisp-dir

[-- Attachment #3: Type: text/plain, Size: 7875 bytes --]


But I can also imagine that this could cause issues in some other
edge-cases.

In GNU Emacs 29.0.60 (build 5, x86_64-pc-linux-gnu, GTK+ Version
 3.24.35, cairo version 1.16.0) of 2022-12-14 built on quetzal
Repository revision: 622838b957e240d700585050e9ddbd036e690513
Repository branch: emacs-29
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --with-pgtk --with-imagemagick'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ
IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES
NOTIFY INOTIFY PDUMPER PGTK PNG RSVG SECCOMP SOUND THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER XIM GTK3 ZLIB

Important settings:
  value of $EMACSLOADPATH: 
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Magit Log

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  shell-dirtrack-mode: t
  server-mode: t
  rcirc-color-mode: t
  rcirc-track-minor-mode: t
  editorconfig-mode: t
  repeat-mode: t
  display-battery-mode: t
  display-time-mode: t
  diff-hl-flydiff-mode: t
  winner-mode: t
  windmove-mode: t
  corfu-history-mode: t
  vertico-multiform-mode: t
  vertico-mode: t
  electric-pair-mode: t
  recentf-mode: t
  save-place-mode: t
  savehist-mode: t
  pixel-scroll-precision-mode: t
  pixel-scroll-mode: t
  xterm-mouse-mode: t
  which-function-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tab-bar-mode: t
  file-name-shadow-mode: t
  context-menu-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/home/philip/.config/emacs/site-lisp/sbbs-pld/sbbs hides /home/philip/.config/emacs/site-lisp/sbbs/sbbs
/home/philip/.config/emacs/site-lisp/flymake-proselint/flymake-proselint hides /home/philip/.config/emacs/elpa/flymake-proselint-0.3.0/flymake-proselint
/home/philip/.config/emacs/site-lisp/writegood-mode/writegood-mode hides /home/philip/.config/emacs/elpa/writegood-mode-2.2.0/writegood-mode
/home/philip/.config/emacs/site-lisp/writegood-mode/writegood-mode-autoloads hides /home/philip/.config/emacs/elpa/writegood-mode-2.2.0/writegood-mode-autoloads
/home/philip/.config/emacs/site-lisp/writegood-mode/writegood-mode-pkg hides /home/philip/.config/emacs/elpa/writegood-mode-2.2.0/writegood-mode-pkg
/home/philip/.config/emacs/elpa/transient-0.3.7/transient hides /home/philip/Source/emacs/lisp/transient

Features:
(shadow emacsbug goto-addr bug-reference magit-extras face-remap
magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
magit-commit magit-sequence magit-notes magit-worktree magit-tag
magit-merge magit-branch magit-reset magit-files magit-refs magit-status
magit magit-repos magit-apply magit-wip magit-log magit-diff git-commit
log-edit add-log magit-core magit-autorevert autorevert filenotify
magit-margin magit-transient magit-process with-editor shell pcomplete
server magit-mode transient edmacro kmacro magit-git magit-section
magit-utils crm dash vc-hg vc-bzr vc-src vc-sccs vc-cvs vc-rcs vc-svn
grep xref rcirc-color rcirc vc-git package-vc inline url-cache url-http
url-auth url-gw display-line-numbers shortdoc eudc-capf eudc cus-start
eudc-vars cl-print debug backtrace find-func smerge-mode jka-compr
mm-archive qp char-fold misearch multi-isearch mailalias smtpmail
orderless vertico-directory vertico-flat ffap buffer-env compat
compat-macs ecomplete mule-util sort smiley gnus-cite mail-extr textsec
uni-scripts idna-mapping ucs-normalize uni-confusable textsec-check
gnus-async gnus-bcklg gnus-ml disp-table nndraft nnmh utf-7 nnfolder
vc-backup copyright time-stamp epa-file network-stream nsm gnus-agent
gnus-srvr gnus-score score-mode nnvirtual gnus-msg gnus-art mm-uu
mml2015 mm-view mml-smime smime gnutls dig nntp gnus-cache gnus-sum shr
pixel-fill kinsoku url-file svg dom gnus-group gnus-undo gnus-start
gnus-dbus gnus-cloud nnimap nnmail mail-source utf7 nnoo parse-time
iso8601 gnus-spec gnus-int gnus-range message yank-media puny dired-x
dired dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068
epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader gnus-win noutline outline checkdoc flymake-proc
flymake yasnippet-snippets warnings yasnippet editorconfig
editorconfig-core editorconfig-core-handle editorconfig-fnmatch flyspell
ispell init repeat project format-spec battery dbus xml shell-command+
thingatpt time sendmail rfc2047 rfc2045 ietf-drums gnus nnheader
gnus-util time-date mail-utils range mm-util mail-prsvr finder-inf
diff-hl-flydiff diff diff-hl log-view pcvs-util vc-dir ewoc vc
vc-dispatcher diff-mode hippie-exp winner windmove corfu-history corfu
vertico-multiform vertico cl-extra elec-pair recentf tree-widget
saveplace savehist pixel-scroll cua-base xt-mouse modus-operandi-theme
modus-themes which-func imenu cus-edit pp cus-load icons wid-edit
package browse-url url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core password-cache json subr-x
map byte-opt url-vars setup site-lisp auto-site compile easy-mmode
files-x cl-macs gv pcase rx text-property-search comint ansi-osc
ansi-color bytecomp byte-compile loaddefs-gen generate-lisp-file
lisp-mnt info consult-autoloads flymake-proselint-autoloads
corfu-autoloads diff-hl-autoloads focus-autoloads haskell-mode-autoloads
buffer-env-autoloads avy-autoloads yasnippet-snippets-autoloads
magit-autoloads geiser-mit-autoloads inspector-autoloads
bash-completion-autoloads ef-themes-autoloads auctex-autoloads tex-site
git-commit-autoloads with-editor-autoloads geiser-guile-autoloads
geiser-impl help-fns radix-tree help-mode cl-loaddefs cl-lib
geiser-custom geiser-base ring geiser-autoloads transient-autoloads
magit-section-autoloads vertico-autoloads 0x0-autoloads
debbugs-autoloads compat-autoloads editorconfig-autoloads
rcirc-color-autoloads orderless-autoloads yasnippet-autoloads
markdown-mode-autoloads slime-autoloads macrostep-autoloads
writegood-mode-autoloads go-mode-autoloads dash-autoloads rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win term/common-win
pgtk-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list
replace newcomment text-mode lisp-mode prog-mode register page tab-bar
menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse
jit-lock font-lock syntax font-core term/tty-colors frame minibuffer
nadvice seq simple cl-generic indonesian philippine cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop
case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo gtk pgtk lcms2 multi-tty make-network-process
emacs)

Memory information:
((conses 16 739347 76383)
 (symbols 48 33750 18)
 (strings 32 159911 10527)
 (string-bytes 1 4359661)
 (vectors 16 93122)
 (vector-slots 8 1939887 121376)
 (floats 8 555 2277)
 (intervals 56 25166 3922)
 (buffers 984 36))

bug#60155: 29.0.60; package-vc doesn't detect subdirectories when installing from repositories

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

>   M-x package-vc-install RET https://codeberg.org/martianh/mastodon.el RET
>
> This will fetch the package sources, but since the actual code is
> located in a "lisp" subdirectory, none of the actual files will be added
> to the load-path, making it appear as tough nothing were installed.
>
> This could be fixed by adding a heuristic to package-vc--unpack-1 that
> checks if there is a ./lisp/ directory (with .el files?) and that would
> add that to the load-path.  Are there any other directory names or
> structures that should be considered as well?  Is it common for packages
> to have loadable files in both the root directory of a repository and a
> sub-directory?
>
> I could imagine that this would be enough to resolve the issue:
>
> diff --git a/lisp/emacs-lisp/package-vc.el b/lisp/emacs-lisp/package-vc.el
> index 8f0eedd2f8..eea5ad6c26 100644
> --- a/lisp/emacs-lisp/package-vc.el
> +++ b/lisp/emacs-lisp/package-vc.el
> @@ -443,6 +443,12 @@ package-vc--unpack-1
>             (auto-name (format "%s-autoloads.el" name))
>             (extras (package-desc-extras pkg-desc))
>             (lisp-dir (alist-get :lisp-dir extras)))
> +      ;; Heuristic to guess a sub-directory with lisp files.
> +      (when-let (((null lisp-dir))
> +                 (dir (expand-file-name "lisp" pkg-dir))
> +                 ((file-directory-p dir)))
> +        (setq lisp-dir dir))
> +
>        (package-generate-autoloads
>         name (file-name-concat pkg-dir lisp-dir))
>        (when lisp-dir

I have installed a variation upon this diff in n06368fef.  Please report
if this heuristic is too aggressive.