Has something changed in the autoload mechanism in Emacs 29?

Has something changed in the autoload mechanism in Emacs 29?

[Pierre Rouleau]

 Hi all,

I stumbled upon a behaviour that is surprising to me when using something I
wrote in my
PEL system ( https://github.com/pierre-rouleau/pel) :  the autoloading of
the PEL functions
fails for at least one function (a non-interactive one) and when I use
`describe-symbol` on
the name of that function, the help buffer shows information identifying
that function as
being implemented in the wrong file.

This is the first time I noticed this.  I assume something must be wrong in
my code,
but I've used it on emacs 26, 27, 28 on various computers without any
problem.

My system probably differs from others in the way it starts:
- My ~/.emacs.d/init.el only hold some basic logic that must be located
there
   but does not activate the packages I used, for the most part;
   that job is done by my pel-init function which then checks user options
   and activates the features requested by it.

The pel-init() function has the following code:

(unless (fboundp 'pel-build)
  ;; autoload all PEL functions
  (require 'pel-autoload)
  (if (fboundp 'pel--autoload-init)
      (pel--autoload-init)))

The 'pel-build symbol is normally not bound, that's only used for testing.

The pel--autoload-init function, located inside the pel-autuload.el file,
has the following:
(defmacro pel-autoload-function (fname for: &rest functions)
  "Schedule the autoloading of FNAME for specified FUNCTIONS.

Argument FOR: just a required separator keyword to make code look better."
  (declare (indent 2))
  (ignore for:)
  (if (> (length functions) 1)
      `(dolist (fct (quote (,@functions)))
         (autoload fct ,fname))
    `(autoload (quote ,@functions) ,fname)))

(defmacro pel-autoload (fname for: &rest functions)
  "Schedule the autoloading of FNAME for specified interactive FUNCTIONS.

Argument FOR: just a required separator keyword to make code look better."
  (declare (indent 2))
  (ignore for:)
  (if (> (length functions) 1)
      `(dolist (fct (quote (,@functions)))
         (autoload fct ,fname nil :interactive))
    `(autoload (quote ,@functions) ,fname nil :interactive)))

And then the definition of pel--autoload-init which schedules the
autoloading
of the PEL functions explicitly with the above macros:

It start with this:

(defun pel--autoload-init ()
  "Intialize the PEL system -- prepare automatic loading of all function."

  (require 'pel--options)

And then has several blocks like the following:

  (pel-autoload "pel-abbrev" for:
    pel-ispell-word-then-abbrev
    pel-abbrev-info)

  (pel-autoload "pel-align" for:
    pel-newline-and-indent-below
    pel-align-info
    pel-toggle-newline-indent-align
    pel-multi-align-regexp)

  (when (eq system-type 'darwin)
    (pel-autoload "pel-applescript" for:
      pel-pel-say-word
      pel-say-sentence
      pel-say-paragraph
      pel-say-region
      pel-say
      pel-say-words)
    (pel-autoload-function "pel-applescript" for:
      pel-run-applescript))

These all work, but one does not seem to work:

  (pel-autoload-function "pel-ffind" for:
    pel-ffind
    pel-generic-find-file)

The pel--find is found in the right file, but not pel-generic-find-file.

It registers as if it was defined in the pel_keys.el file where pel-init()
is defined:
the *help* message displays :
  pel-generic-find-file is an autoloaded byte-compiled Lisp function in
  ‘pel_keys.el’.
And this is wrong.
The rest of the help is correct.  It shows the function signature and the
correct docstring.
 Note I byte-compile *all* my code.  I have a make file that does that.

Does anyone have ideas as to where I should be looking?

Thanks (and sorry for the long description).


-- 
/Pierre

Re: Has something changed in the autoload mechanism in Emacs 29?

[Michael Heerdegen via Users list for the GNU Emacs text editor]

Pierre Rouleau <prouleau001@gmail.com> writes:

> My system probably differs from others in the way it starts:
> - My ~/.emacs.d/init.el only hold some basic logic that must be located
> there
>    but does not activate the packages I used, for the most part;
>    that job is done by my pel-init function which then checks user options
>    and activates the features requested by it.
> [...]
> These all work, but one does not seem to work:
>
>   (pel-autoload-function "pel-ffind" for:
>     pel-ffind
>     pel-generic-find-file)

Is "pel-ffind" in `load-path' when you get the unexpected result?


Michael.

Re: Has something changed in the autoload mechanism in Emacs 29?

[Pierre Rouleau]

On Thu, Jul 25, 2024 at 2:38 PM Michael Heerdegen via Users list for the
GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

>
>


> Is "pel-ffind" in `load-path' when you get the unexpected result?
>
>
Yes.  The directory where all PEL files are located is always in the
load-path list.
I checked explicitly in the case where the help shows the wrong file
location of
the function and it's there.

Note that when the error shows up, if I byte-compile the pel-ffind.el file
 (where the pel-generic-find-file function is located)
and issue the describe-symbol command again,
then the information shown in the help buffer shows the correct file name:
pel--ffind.el

-- 
/Pierre

Re: Has something changed in the autoload mechanism in Emacs 29?

[Michael Heerdegen via Users list for the GNU Emacs text editor]

Pierre Rouleau <prouleau001@gmail.com> writes:

> Note that when the error shows up, if I byte-compile the pel-ffind.el file
>  (where the pel-generic-find-file function is located)
> and issue the describe-symbol command again,
> then the information shown in the help buffer shows the correct file name:
> pel--ffind.el

What's the result of evaluating (symbol-function 'pel-generic-find-file)
in the situation where the describe command reports the wrong location?


I currently don't have a concrete idea.  Nothing fundamental changed in
the autoload mechanism I think.  Smaller changes happened over the time,
sure.

Are you even sure that the problem is new and not reproducible using
older Emacs versions?


When I visit the link to the source of your code, is it easy for me to
reproduce this issue?


Michael.

Re: Has something changed in the autoload mechanism in Emacs 29?

[Pierre Rouleau]

On Fri, Jul 26, 2024 at 7:59 AM Michael Heerdegen via Users list for the
GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> Pierre Rouleau <prouleau001@gmail.com> writes:
>
> > Note that when the error shows up, if I byte-compile the pel-ffind.el
> file
> >  (where the pel-generic-find-file function is located)
> > and issue the describe-symbol command again,
> > then the information shown in the help buffer shows the correct file
> name:
> > pel--ffind.el
>
> What's the result of evaluating (symbol-function 'pel-generic-find-file)
> in the situation where the describe command reports the wrong location?
>
> The result is:

#[513 "\300


 \3011^\^@\211\203^P^@\211^BB\202^Q^@^A\302^A!\303^E^B\"\266\2020\207\210\304\207"
 [pel-ffind-project-directory (user-error) delete-dups pel-ffind nil] 8
 ("/Users/roup/my/dv/elisp/pel/pel-ffind.elc" . 3424)]


The path /Users/roup/my/dv/elisp/pel/pel-ffind.elc is correct.

Yet the *Help* buffer created by (describe-symbol 'pel-generic-find-file) ,
which returns 1,
shows the function being defined in pel_keys.el instead of the pel-ffind.el,
followed by the proper docstring.

pel-generic-find-file is an autoloaded byte-compiled Lisp function in
‘pel_keys.el’.

(pel-generic-find-file FILENAME &optional DIRECTORIES)

Find a file FILENAME from the project holding the current buffer file.

Return a list of found file names with complete absolute path.
Return nil if nothing found.

All file searches are done in directory *trees* identified by the current
project and optionally by the extra DIRECTORIES.

If DIRECTORIES argument is specified it may be a single directory
path string or a list of directory path strings to search on top
of the current project directory.

The ‘pel-generic-find-file’ is the most generic method to search
for files, can be set as the default value for the global value
of the ‘pel-filename-at-point-finders’ buffer local variable used
by ‘pel--find-by-finders’.

However, that may not be sufficient for some programming
languages.  In that case you should be using a language specific function.
There is one implemented for Erlang: ‘pel-erlang-find-file’.



> I currently don't have a concrete idea.  Nothing fundamental changed in
> the autoload mechanism I think.  Smaller changes happened over the time,
> sure.
>

Was there not a change in Emacs 28?  I'll have to go back and read the
release notes.


> Are you even sure that the problem is new and not reproducible using
> older Emacs versions?
>

I will have to retest on Emacs 28 and 27.
I tried several times on Emacs 26.1 that I have to use on some old linux
distros
and I could not see the problem.

When I visit the link to the source of your code, is it easy for me to
> reproduce this issue?
>

Probably not.  PEL requires a specific init.el,
provided as an example to update in
https://github.com/pierre-rouleau/pel/blob/master/example/init/init.el
And then basic configuration.

I will probably have to try to disable some packages
in the environment where I see the problem just in case
the issue is caused by something in there.
I can't use emacs -Q to test it because then I don't use PEL.

If there's nothing obvious as to what changed in the auto loading
mechanism then I'll need to test more on the affected platform and
test on others.

I should probably look at the code that describe-symbol and see
where it gets the source file associated with the symbol.


-- 
/Pierre

Re: Has something changed in the autoload mechanism in Emacs 29?

[Michael Heerdegen via Users list for the GNU Emacs text editor]

Pierre Rouleau <prouleau001@gmail.com> writes:

> > What's the result of evaluating (symbol-function 'pel-generic-find-file)
> > in the situation where the describe command reports the wrong location?
> >
> > The result is:
>
> #[513 "\300
>
>
>  \3011^\^@\211\203^P^@\211^BB\202^Q^@^A\302^A!\303^E^B\"\266\2020\207\210\304\207"
>  [pel-ffind-project-directory (user-error) delete-dups pel-ffind nil] 8
>  ("/Users/roup/my/dv/elisp/pel/pel-ffind.elc" . 3424)]

> The path /Users/roup/my/dv/elisp/pel/pel-ffind.elc is correct.

But isn't this a byte-code function and not an autoload function?  With
a reference to the source of the docstring (when using
`byte-compile-dynamic-docstrings')?


> Yet the *Help* buffer created by (describe-symbol 'pel-generic-find-file) ,
> which returns 1,
> shows the function being defined in pel_keys.el instead of the pel-ffind.el,
> followed by the proper docstring.
>
> pel-generic-find-file is an autoloaded byte-compiled Lisp function in
> ‘pel_keys.el’.

Maybe looking at `load-history' can reveal why this file is mentioned.


> Was there not a change in Emacs 28?  I'll have to go back and read the
> release notes.

Ok, yes, there had been the switch from autoloads.el to loaddefs-gen.el
with Emacs 29.


> If there's nothing obvious as to what changed in the auto loading
> mechanism then I'll need to test more on the affected platform and
> test on others.
>
> I should probably look at the code that describe-symbol and see
> where it gets the source file associated with the symbol.

When `find-lisp-object-file-name' is involved, I would try to edebug
that, as a promising starting point.


Michael.

Re: Has something changed in the autoload mechanism in Emacs 29?

[Pierre Rouleau]

On Fri, Jul 26, 2024 at 11:59 AM Michael Heerdegen via Users list for the
GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> Pierre Rouleau <prouleau001@gmail.com> writes:
>
> > > What's the result of evaluating (symbol-function
> 'pel-generic-find-file)
> > > in the situation where the describe command reports the wrong location?
> > >
> > > The result is:
> >
> > #[513 "\300
> >
> >
> >
> \3011^\^@\211\203^P^@\211^BB\202^Q^@^A\302^A!\303^E^B\"\266\2020\207\210\304\207"
> >  [pel-ffind-project-directory (user-error) delete-dups pel-ffind nil] 8
> >  ("/Users/roup/my/dv/elisp/pel/pel-ffind.elc" . 3424)]
>
> > The path /Users/roup/my/dv/elisp/pel/pel-ffind.elc is correct.
>
> But isn't this a byte-code function and not an autoload function?  With
> a reference to the source of the docstring (when using
> `byte-compile-dynamic-docstrings')?
>
>
> > Yet the *Help* buffer created by (describe-symbol
> 'pel-generic-find-file) ,
> > which returns 1,
> > shows the function being defined in pel_keys.el instead of the
> pel-ffind.el,
> > followed by the proper docstring.
> >
> > pel-generic-find-file is an autoloaded byte-compiled Lisp function in
> > ‘pel_keys.el’.
>
> Maybe looking at `load-history' can reveal why this file is mentioned.
>
>
> > Was there not a change in Emacs 28?  I'll have to go back and read the
> > release notes.
>
> Ok, yes, there had been the switch from autoloads.el to loaddefs-gen.el
> with Emacs 29.
>
>
> > If there's nothing obvious as to what changed in the auto loading
> > mechanism then I'll need to test more on the affected platform and
> > test on others.
> >
> > I should probably look at the code that describe-symbol and see
> > where it gets the source file associated with the symbol.
>
> When `find-lisp-object-file-name' is involved, I would try to edebug
> that, as a promising starting point.
>
> Thanks for the info Michael,  I will try these once I get a little bit of
time and will report the outcome.

I'm a little swamped by other things at the moment.


-- 
/Pierre

Re: Has something changed in the autoload mechanism in Emacs 29?

[Michael Heerdegen via Users list for the GNU Emacs text editor]

Pierre Rouleau <prouleau001@gmail.com> writes:

> I'm a little swamped by other things at the moment.

You are very kind.  Take your time, really no problem.


Michael.