bug#48015: 28.0.50; ELPA package compilation fails

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> The core problem that I see is the following:
>>
>> - Emacs's tramp gets loaded
>> - We go to a Tramp-controlled default-directory
>> - We call `package--load-files-for-activation`
>>   - This starts by loading ~/.emacs.d/elpa/tramp-NN.MM/tramp-autoloads.el
>>     This calls `tramp-register-autoload-file-name-handlers`.
>>   - At this point, `package--load-files-for-activation` would like to
>>     continue by (re)loading the new Tramp files, such as `tramp-compat`
>>     and friends in the same order that they have been loaded
>>     (i.e. `tramp-compat.el` before `tramp.el`).
>>   - But before it gets a chance to do that, the file-name handlers
>>     call `tramp-autoload-file-name-handler` because of `default-directory`,
>>     which does (load "tramp" 'noerror 'nomessage), which loads the new
>>     `tramp.el` before we got a change to load the new `tramp-compat.el`,
>>     which then leads to an error when the new code in `tramp.el` calls
>>     a new function from `tramp-compat.el` (which happens to be
>>     `tramp-compat-thread-yield` AFAICT).
>>
>> At this point, I'm not sure how best to fix the problem.
>> Maybe replacing (load "tramp" 'noerror 'nomessage) with
>> (require 'tramp nil t) is all it takes.
>> Or maybe a better option is to arrange the autoloads such that
>> `tramp-register-autoload-file-name-handlers` doesn't "unload/unregister" file
>> handlers that have already been loaded so that the directory that was
>> already under Tramp's control doesn't re-trigger a call to
>> `tramp-autoload-file-name-handler`?
>
> What I don't understand: when (load "tramp" 'noerror 'nomessage) is
> called, default-directory is already local due to a let-binding.

That's fine, this load functions properly.  The problem is elsewhere:
- loading the new `tramp.el` works properly but results in a broken
  setup because it will not load the new `tramp-compat.el`
  because that feature is already provided.  So you end up with a new`
  tramp.el` calling functions it expects to be provided by the new
  `tramp-compat.el`.
- the Tramp default-directory is not active during the load, but it is
  active right before it (it is the trigger that causes the load, actually)
  and it is active right after it (before package.el gets to reload
  `tramp-compat.el`) and that's when the new code from `tramp.el`
  signals an error because of the missing function from the new
  `tramp-compat.el`.

> Anyway, even if the compilation runs through in case of a local
> default-directory, the resulting *.elc files have errors.
>
> I have quit Emacs (from the first recipe), and then I have started
>
>     emacs -Q -L ~/.emacs.d/elpa/tramp-2.5.0.3/ /ssh::
>
> There are further errors, which are related to a wrong
> tramp-compat.el. Only my new command tramp-recompile-elpa fixes this.

I suspect this is the result of the `find-library-name` problem I fixed
yesterday on `master` (which causes `package.el` not to load the new
files to override the old ones before compiling the new files).

You can circumvent it by loading `find-func` before doing the
`package-install`.  I'm not sure how the GNU ELPA package of Tramp can
protect itself from this bug yet (beside `tramp-recompile-elpa`, of
course).

BTW, maybe one option to circumvent both problems is to replace

    (load "tramp" 'noerror 'nomessage)

with

    (load "tramp-compat" 'noerror 'nomessage)
    (load "tramp" 'noerror 'nomessage)

?


-- Stefan