bug#71817: 29.3; Sub-directory handling of ELPA package

[Xiyue Deng <manphiz@gmail.com>]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Currently as observed, ELPA packages only get their root path added to
>> `load-path', but source code in sub-directories will still get
>> byte-compiled.  That is, for an ELPA package elpafoo with a nested
>> sub-directory of the following structure (installed through package.el):
>
> The recursive compilation is somewhat of an "accident": it was the
> easiest to implement (and seemed like a good idea anyway).
>
> The `load-path` behavior is conscious: it's easy for a package to add
> more subdirectories to the `load-path` but it would be much harder to
> remove undesired ones.  [ And of course, the current behavior is also
> the easiest one to implement.  ]
>
>> If this is not yet a policy, I wonder whether this will be the path
>> forward for `load-path' handling.
>
> In `elpafoo.el`, include something like:
>
>     ;;;###autoload
>     (add-to-list 'load-path
>                  (expand-file-name
>                   "elpabar" (file-name-directory load-file-name)))
>
> This assumes that you want `elpabar` to be in your `load-path` right
> from the start (i.e. that an entry point to your package is in the
> `elpabar` subdirectory).  If `elpabar` can only ever be used from code
> that's in the `elpafoo` directory, there are other options (such as
> `(require 'elpabar/elpabar)` or using an auxiliary `elpafoo-loaddefs.el`
> which you load when `elpafoo.el` is loaded, ...
>
>> I see some pros of adding sub-directories recursively,
>
> I don't.  Most of the packages which use subdirectories have a complex
> enough layout that some of those directories should not be in
> `load-path`: it's better to let them add entries "manually" at the
> appropriate time than to try and do it automatically.
>
> The more real problem is that the way `elpafoo-autoloads.el` is created
> does *not* scan ELisp files in subdirectories.  The way this is handled
> typically in that the ELPA tarball comes with its own
> `elpabar/elpabar-autoloads.el` file and `elpafoo.el` then needs to
> contain something like
>
>     ;;;###autoload
>     (require 'elpabar/elpabar-autoloads)
>
> The main downside here is that the current elpa.gnu.org scripts don't
> know how to build such a `elpabar/elpabar-autoloads.el`, so you either
> need to store it in the Git (which is ugly since it's a generated file),
> or use an ad-hoc `:make` rule.
>
> IMO we should change the ELPA protocol so that the
> `elpafoo-autoloads.el` is not created during installation but is instead
> included in the tarball, so it can be generated any way we like.
>
>
>         Stefan
>

Thanks for the explanations and for exploring options to load sources in
sub-directories without recursive loading by default.  I take that the
current status - byte-compile recursively, only add source root path to
`load-path' - will continue to be the path forward.

For now, it makes sense that loading sources from sub-directories
requires some manual `load-path' handling.  Maybe when using
sub-directories to organize source files becomes more common upstream
may consider adding some convenient functions/macros to make it easier,
but it will be for the future.

Thanks again!

-- 
Xiyue Deng