bug#63757: 29.0.91 order of package paths changed: random old versions of packages in load-path

[Philip Kaludercic <philipk@posteo.net>]

Евгений Бойков <artscan@list.ru> writes:

>> Done.  Should we close the issue then, or wait for more feedback?
>
> fa8135f8916 * emacs-29 origin/emacs-29 Revert changes to the order in which
> package descs are loaded
> -      (dolist (pkg-dir (directory-files dir t "\\`[^.]" t))
> +      (dolist (pkg-dir (directory-files dir t "\\`[^.]"))
>
> it doesn't fix the initial problem: `load-path` and `package-alist` are
> still broken in my test.

What does "broken" mean in this case?  I fear that your patch just
changes what is necessary to resolve the issue in your case, but the
underlying problem (the destructive modification of `package-list')
still persists?

> Test:
> To reproduce the case we need emacs 29.0.91 + fa8135f8916 fix  and
>     ~/.emacs.d/elpa/go-mode-20170726.555/...
>     ~/.emacs.d/elpa/go-mode-20220114.2239/...
> Run `emacs -Q` and `M-x package-initialize`. After that `package-alist` and
> `load-path` contain the 2017 version as the first item in the list (instead
> of 2022).
>
> It can be fixed like this:
>
> diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
> index ba0e3618f28..6eaf261f5f4 100644
> --- a/lisp/emacs-lisp/package.el
> +++ b/lisp/emacs-lisp/package.el
> @@ -922,11 +922,13 @@ correspond to previously loaded files."
>                                   (v2 (package-desc-version p2)))
>                               (or
>                                ;; Prefer VC packages.
> -                              (package-vc-p p1)
> -                              (package-vc-p p2)
> +                              (and
> +                               (package-vc-p p1)
> +                               (not (package-vc-p p2)))
>                                ;; Prefer builtin packages.
> -                              (package-disabled-p p1 v1)
> -                              (not (package-disabled-p p2 v2))))))))
> +                              (and
> +                               (package-built-in-p p1 v1)
> +                               (not (package-built-in-p p2 v2)))))))))
>      ;; Check if PACKAGE is available in `package-alist'.
>      (while
>          (when pkg-descs
>
> But if we leave it until 29.2, I will not insist :)
>
>
> On Sun, Jun 4, 2023 at 11:21 PM Philip Kaludercic <philipk@posteo.net>
> wrote:
>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> >> From: Philip Kaludercic <philipk@posteo.net>
>> >> Cc: artscan@list.ru,  monnier@iro.umontreal.ca,  63757@debbugs.gnu.org
>> >> Date: Sun, 04 Jun 2023 12:32:17 +0000
>> >>
>> >> Eli Zaretskii <eliz@gnu.org> writes:
>> >>
>> >> >> -      (dolist (pkg-dir (directory-files dir t "\\`[^.]" t))
>> >> >> +      (dolist (pkg-dir (directory-files dir t "\\`[^.]"))
>> >> >>          (when (file-directory-p pkg-dir)
>> >> >>            (package-load-descriptor pkg-dir))))))
>> >> >>
>> >> >> (I attached the *vc-diff* buffer, but before sending the message I
>> >> >> invoked vc-diff again which modified the buffer contents.  Should
>> have
>> >> >> just copied the diff manually...)
>> >> >
>> >> > Thanks, now it at least makes sense.
>> >> >
>> >> > But why does it matter whether the package directory names are sorted
>> >> > or not?  Is sorting package versions the same as sorting the names of
>> >> > their directories?
>> >>
>> >> No, because directories are sorted by their names and by that metric,
>> >> "foo-10" is less than "foo-9".  But since package archives like MELPA
>> >> use ISO 8601-like release-dates as version numbers, the order "works".
>> >> If we disable sorting, the files returned in any order the operating
>> >> system might have arbitrary stored the directory entries in which might
>> >> even change depending on the file system.  So at the very least we can
>> >> argue that while sorting does not solve the issue, it provides a quick
>> >> hack to prevent sudden breakage from Emacs 28 to Emacs 29, due to users
>> >> relying on this kind of behaviour.  A proper solution, that takes
>> >> version numbers into account should be prepared on the master branch.
>> >
>> > OK, thanks.  Please install on emacs-29, and let's hope this doesn't
>> > cause new problems.
>>
>> Done.  Should we close the issue then, or wait for more feedback?
>>