bug#55305: 28.0.50: With async nativecomp, package manager fails to load hyperbole-autoloads.el before compilation

[Robert Weiner <rsw@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 4159 bytes --]

On Thu, May 12, 2022 at 3:22 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Robert Weiner <rsw@gnu.org>
> > Date: Thu, 12 May 2022 02:21:37 -0400
> > Cc: 55305@debbugs.gnu.org
> >
> >  Does this last fact mean there's an assumption in Hyperbole that the
> >  package is always activated before its *.el files are compiled?  If
> >  so, perhaps this is why it fails during native-compilation, where the
> >  package is not activated prior to the compilation?
> >
> > Said another way, there is an assumption that the hyperbole-autoloads.el
> file is loaded prior to any
> > compilation, yes.  This is similar to assumptions that loaddefs.el are
> loaded prior to their reference in other
> > Emacs Lisp files.

Hi Eli:

Again, thanks for your feedback.  I don't expect any change to be made on
this in Emacs at this point but wanted to finish the discussion with a few
final thoughts.

> loaddefs.el is preloaded into Emacs when it is built, so the analogy
> doesn't work in practice.
>

Emacs loads autoloads from a file when it is built (prior to dumping its
image) and I am simply suggesting that both the package manager and the
native compiler do the same for packages.


> I think you should look at bundled packages like Calc.  Calc has
> calc-loaddefs.el, but I just now forced Emacs 28.1 to native-compile
> Calc and didn't see any problems.  And I see that calc.el does say
> explicitly
>
>   ;;;; (Autoloads here)
>   (load "calc-loaddefs.el" nil t)
>

Exactly, calc.el works around this missing feature by explicitly loading
the loaddefs and then having every other calc module require the 'calc'
library.  This is equivalent to a manual load of the autoloads in every
module of the package, i.e. there is no autoloading since the autoload
definitions are required everywhere.  Any package can do this but then
nothing is autoloaded at build time when a definition is referenced.  The
calc package goes further and adds a hack at the end of certain files:

;; Local variables:
;; generated-autoload-file: "calc-loaddefs.el"
;; End:

to force magic ;;;###autoload definitions to be written to the
calc-loaddefs.el file.  All of this is necessary because certain automated
handling of the default package autoloading file is missing from Emacs.


> > The point of the autoloads file is to include definitions that must
> exist in the Lisp
> > environment prior to their reference in any Lisp files, whether this is
> during package use or package
> > build-time.
>
> That is true, but AFAIU packages that have their own separate
> autoloads file should proactively do something to make sure those
> autoloads are loaded before they are needed.
>

My question is why?  If we want definitions within packages autoloaded just
as they can be outside of packages, why do we not want to simply fix the
issue so that each package's autoload file is actually autoloaded by the
package manager and the native compiler at both build and package
activation time (the latter already being done)?  We have a standardized
naming for such files, package-<package-name>.el.  They are generated by
the package manager but presently not autoloaded at build initialization.
It is immensely more work for each large package to require this in each of
their files and makes little sense since such a file does not exist at
least for Elpa packages until build time.

And this is not related to native-compilation in any way: the same
> will happen if one tries to byte-compile Hyperbole files without first
> loading its autoloads.  Right?
>

Yes, if you manually byte-compile a package file, you have to ensure its
autoloads have been loaded, but this is for a manual process.  I am
suggesting that in an automated context of package building that this too
should be automated and autoloads should automatically be loaded by the
build automation systems.  Otherwise, my argument is that these are not
treated as autoload files at the package level but are simply Lisp
libraries that have to be manually loaded by every library in the package,
a tedious affair for large packages.  Certainly not the end of the world
but difficult to manage and get right all the time.

[-- Attachment #2: Type: text/html, Size: 6816 bytes --]