bug#71789: 29.3; Handling of nested directories of ELPA packages

[Jeremy Bryant via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

Xiyue Deng <manphiz@gmail.com> writes:

> Hi,
>
> I'm opening a bug to follow up the discussion at [1].  Given an ELPA
> package `elpafoo' that contains another directory `elpabar' that also
> holds source files, e.g. like the structure below:
>
> ,----
> | ~/.config/emacs/elpa/elpafoo/
> | ~/.config/emacs/elpa/elpafoo/elpafoo.el
> | ~/.config/emacs/elpa/elpafoo/elpabar
> | ~/.config/emacs/elpa/elpafoo/elpabar/elpabar.el
> `----
>
> When installing through package.el, both elpafoo.el and
> elpabar/elpabar.el will be byte-compiled, but only the root source path
> `~/.config/emacs/elpa/elpafoo' is added to `load-path', without any of
> the nested directories.
>
> I wonder whether this is the intended behavior regarding `load-path'
> handling.  If nested directories are added, it has the advantage to
> allow more fine-grained source organization; as a downside, it may
> unnecessarily add more unneeded stuff and processing overhead (as
> Michael pointed out at [2]), and potentially break existing code,
> e.g. auctex, which seems to vendor stuff in the `style/' directories but
> not loaded so it's not causing any trouble yet.
>
> So I wonder what is the stance of the Emacs maintainer on this subject,
> or whether this is documented anywhere.  Also one more step: whether
> there is a recommended layout for source code organization for ELPA
> packages.
>
> [1] https://lists.gnu.org/archive/html/emacs-devel/2024-05/msg00658.html
> [2] https://lists.gnu.org/archive/html/emacs-devel/2024-05/msg01039.html
>
>
>
> In GNU Emacs 29.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.38,
>  cairo version 1.16.0) of 2024-05-20, modified by Debian built on sbuild
> Windowing system distributor 'The X.Org Foundation', version 11.0.12101007
> System Description: Debian GNU/Linux 12 (bookworm)
>
> Configured using:
>  'configure --build x86_64-linux-gnu --prefix=/usr
>  --sharedstatedir=/var/lib --libexecdir=/usr/libexec
>  --localstatedir=/var/lib --infodir=/usr/share/info
>  --mandir=/usr/share/man --with-libsystemd --with-pop=yes
>  --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/29.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/29.3/site-lisp:/usr/share/emacs/site-lisp
>  --with-sound=alsa --without-gconf --with-mailutils
>  --with-native-compilation --build x86_64-linux-gnu --prefix=/usr
>  --sharedstatedir=/var/lib --libexecdir=/usr/libexec
>  --localstatedir=/var/lib --infodir=/usr/share/info
>  --mandir=/usr/share/man --with-libsystemd --with-pop=yes
>  --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/29.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/29.3/site-lisp:/usr/share/emacs/site-lisp
>  --with-sound=alsa --without-gconf --with-mailutils
>  --with-native-compilation --with-cairo --with-x=yes
>  --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
>  -ffile-prefix-map=/build/reproducible-path/emacs-29.3+1=. -fstack-protector-strong
>  -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
>  -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'
>
> Configured features:
> ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
> JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
> NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
> THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2
> XPM GTK3 ZLIB
>
> Important settings:
>   value of $LANG: en_US.UTF-8
>   value of $XMODIFIERS: @im=ibus
>   locale-coding-system: utf-8-unix
>
> Major mode: Lisp Interaction
>
> Minor modes in effect:
>   global-git-commit-mode: t
>   magit-auto-revert-mode: t
>   shell-dirtrack-mode: t
>   windmove-mode: t
>   rcirc-track-minor-mode: t
>   server-mode: t
>   subword-mode: t
>   bug-reference-prog-mode: t
>   whitespace-mode: t
>   yas-minor-mode: t
>   xclip-mode: t
>   global-treesit-auto-mode: t
>   treemacs-project-follow-mode: t
>   treemacs-follow-mode: t
>   treemacs-git-mode: t
>   treemacs-fringe-indicator-mode: t
>   corfu-terminal-mode: t
>   corfu-popupinfo-mode: t
>   corfu-echo-mode: t
>   global-corfu-mode: t
>   corfu-mode: t
>   activities-tabs-mode: t
>   activities-mode: t
>   fido-vertical-mode: t
>   icomplete-vertical-mode: t
>   icomplete-mode: t
>   fido-mode: t
>   override-global-mode: t
>   global-display-line-numbers-mode: t
>   display-line-numbers-mode: t
>   global-auto-revert-mode: t
>   tooltip-mode: t
>   global-eldoc-mode: t
>   eldoc-mode: t
>   show-paren-mode: t
>   electric-indent-mode: t
>   mouse-wheel-mode: t
>   tool-bar-mode: t
>   tab-bar-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   column-number-mode: t
>   line-number-mode: t
>   transient-mark-mode: t
>   auto-composition-mode: t
>   auto-encryption-mode: t
>   auto-compression-mode: t
>
> Load-path shadows:
> /usr/share/emacs/site-lisp/elpa/debian-el-37.13/debian-autoloads hides /usr/share/emacs/site-lisp/elpa/gnuplot-0.8.1/debian-autoloads
> /usr/share/emacs/site-lisp/elpa/magit-3.3.0/magit-section hides /usr/share/emacs/site-lisp/elpa/magit-section-3.3.0/magit-section
>
> Features:
> (shadow emacsbug git-rebase tabify ert-x python etags fileloop generator
> java-ts-mode c-ts-common sh-script smie executable debian-copyright
> debian-changelog-mode noutline outline shortdoc completion
> debian-control-mode cl-print help-fns radix-tree tramp-cmds
> gitignore-mode conf-mode tramp-container tramp-sh tramp-cache time-stamp
> dockerfile-ts-mode yaml-ts-mode reporter debian-bts-control debian-bug
> reveal mailalias goto-addr vterm tramp tramp-loaddefs trampver
> tramp-integration files-x tramp-compat term ehelp vterm-module misearch
> multi-isearch shr-color flow-fill matlab matlab-scan matlab-syntax
> matlab-compat qp url-http url-gw url-auth url-queue url-cache mm-archive
> sort gnus-cite mail-extr textsec uni-scripts idna-mapping ucs-normalize
> uni-confusable textsec-check gnus-async gnus-bcklg gnus-ml magit-extras
> eglot external-completion array jsonrpc ert ewoc debug backtrace
> find-func xref pcase perl-mode face-remap magit-bookmark magit-submodule
> magit-obsolete magit-blame magit-stash magit-reflog magit-bisect
> magit-push magit-pull magit-fetch magit-clone magit-remote magit-commit
> magit-sequence magit-notes magit-worktree magit-tag magit-merge
> magit-branch magit-reset magit-files magit-refs magit-status magit
> magit-repos magit-apply magit-wip magit-log which-func imenu magit-diff
> smerge-mode diff git-commit log-edit add-log magit-core magit-autorevert
> magit-margin magit-transient magit-process with-editor shell pcomplete
> magit-mode transient format-spec magit-git magit-section magit-utils crm
> dired-aux mule-util jka-compr gnus-topic cursor-sensor utf-7 nnfolder
> gnus-demon nnml ezgnus gnus-delay gnus-draft gnus-agent gnus-srvr
> gnus-score score-mode nnvirtual nntp gnus-cache gnus-msg gnus-art mm-uu
> mml2015 mm-view mml-smime smime dig gnus-sum shr pixel-fill kinsoku
> url-file svg dom nndraft nnmh gnus-group gnus-undo smtpmail gnus-start
> gnus-dbus dbus xml gnus-cloud nnimap nnmail mail-source utf7 nnoo
> gnus-spec gnus-int gnus-range message sendmail yank-media rfc822 mml
> mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
> ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus nnheader
> gnus-util mail-utils range mm-util mail-prsvr windmove flyspell ispell
> gnutls network-stream puny nsm epa-file epa derived epg rfc6068
> epg-config rcirc parse-time iso8601 time-date term/xterm xterm comp
> comp-cstr server cap-words superword subword vc-hg vc-git diff-mode
> vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs log-view pcvs-util vc
> vc-dispatcher bug-reference disp-table whitespace yasnippet-snippets
> yasnippet cus-edit cus-start wid-edit init zenburn-theme xclip
> treesit-auto treesit treemacs-project-follow-mode treemacs-follow-mode
> treemacs-rendering treemacs-annotations treemacs-async treemacs-visuals
> treemacs-fringe-indicator pulse color treemacs-workspaces treemacs-dom
> treemacs-icons treemacs-themes treemacs-scope treemacs-core-utils
> treemacs-logging treemacs-customization pfuture inline ht s hl-line dash
> keychain-environment exec-path-from-shell corfu-terminal popon
> corfu-popupinfo corfu-echo corfu compat activities-tabs activities
> persist bookmark pp edmacro kmacro advice icomplete cus-load
> flymake-proc flymake project compile text-property-search comint
> ansi-osc ansi-color ring warnings icons thingatpt cl-extra help-mode
> use-package use-package-ensure use-package-delight use-package-diminish
> use-package-bind-key bind-key easy-mmode use-package-core
> display-line-numbers autorevert filenotify
> keychain-environment-autoloads treesit-auto-autoloads xclip-autoloads rx
> info debian-el dired dired-loaddefs finder-inf package browse-url url
> url-proxy url-privacy url-expand url-methods url-history url-cookie
> generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
> auth-source cl-seq eieio eieio-core cl-macs password-cache json subr-x
> map byte-opt gv bytecomp byte-compile url-vars cl-loaddefs cl-lib rmc
> iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
> vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
> term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
> tabulated-list replace newcomment text-mode lisp-mode prog-mode register
> page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
> scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
> frame minibuffer nadvice seq simple cl-generic indonesian philippine
> cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
> korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
> european ethiopic indian cyrillic chinese composite emoji-zwj charscript
> charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
> cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
> files window text-properties overlay sha1 md5 base64 format env
> code-pages mule custom widget keymap hashtable-print-readable backquote
> threads dbusbind inotify lcms2 dynamic-setting system-font-setting
> font-render-setting cairo move-toolbar gtk x-toolkit xinput2 x multi-tty
> make-network-process native-compile emacs)
>
> Memory information:
> ((conses 16 4151836 235820)
>  (symbols 48 38894 35)
>  (strings 32 186143 40188)
>  (string-bytes 1 6165556)
>  (vectors 16 121009)
>  (vector-slots 8 3140568 287319)
>  (floats 8 1184 5120)
>  (intervals 56 274782 3076)
>  (buffers 984 145))

Adding Philip who may have thoughts regarding ELPA structure.