ELPA and built-in packages

ELPA and built-in packages

[Lars Ingebrigtsen]

If Emacs comes with a built-in package that has a version 3.4, and the
version you've installed from ELPA is 3.3, does Emacs know to use the
built-in version automatically instead of the version from ELPA?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: ELPA and built-in packages

[Stefan Monnier]

Lars Ingebrigtsen [2021-10-31 19:08:37] wrote:
> If Emacs comes with a built-in package that has a version 3.4, and the
> version you've installed from ELPA is 3.3, does Emacs know to use the
> built-in version automatically instead of the version from ELPA?

It should, yes.
Reality will sometimes disagree, because it depends if `package.el` is
truly informed about the built-in package.
[ E.g. it currently isn't informed of the built-in modus-theme
  packages.  ]


        Stefan

Re: ELPA and built-in packages

[Phil Sainty]

On 2021-11-01 07:55, Stefan Monnier wrote:
> Reality will sometimes disagree, because it depends if `package.el`
> is truly informed about the built-in package.

What is it about the built-in package that "informs" package.el?

I just did a test with so-long.el and got slightly odd results.

1. I ran Emacs 27.2 (which includes so-long 1.0) with a fresh profile.
2. I then installed v1.1 from 
https://elpa.gnu.org/packages/so-long-1.1.tar.lz
3. I then quit and ran Emacs 28 (which includes so-long 1.1.2).
4. M-x package-refresh-contents
5. M-x package-list-packages
6. M-x occur RET so-long

Result:

3 matches for "so-long" in buffer: *Packages*
     310:  so-long         1.1.2         available         gnu
     388:  so-long         1.1           installed
     444:  so-long         1.0           built-in

The actual so-long functionality is good (the built-in version 1.1.2 is
used in practice, as desired), but that listing is definitely confused.


-Phil

Re: ELPA and built-in packages

[Phil Sainty]

Some missing details:

On 2021-11-01 20:44, Phil Sainty wrote:
> 2. I then installed v1.1 from 
> https://elpa.gnu.org/packages/so-long-1.1.tar.lz

Initially I used package-install-from-buffer because I forgot that
package-install-file requires tar files to be uncompressed.  I've
now also tested with M-x package-install-file, after uncompressing.

> 3. I then quit and ran Emacs 28 (which includes so-long 1.1.2).

More specifically, GNU Emacs 28.0.60 ... of 2021-10-07

Re: ELPA and built-in packages

[Stefan Monnier]

Phil Sainty [2021-11-01 20:44:52] wrote:

> On 2021-11-01 07:55, Stefan Monnier wrote:
>> Reality will sometimes disagree, because it depends if `package.el`
>> is truly informed about the built-in package.
>
> What is it about the built-in package that "informs" package.el?

The info is stored in `package--builtin-versions` which is populated
from the autoloads information in `lisp/loaddefs.el`, which is
generated by `lisp/emacs-lisp/autoload.el`:

                      (when autoload-builtin-package-versions
                        (let ((version (lm-header "version"))
                              package)
                          (and version
                               (setq version (ignore-errors (version-to-list version)))
                               (setq package (or (lm-header "package")
                                                 (file-name-sans-extension
                                                  (file-name-nondirectory file))))
                               (setq output-start (autoload--setup-output
                                                   otherbuf outbuf absfile
                                                   load-name outfile))
                               (let ((standard-output (marker-buffer output-start))
                                     (print-quoted t))
                                 (princ `(push (purecopy
                                                ',(cons (intern package) version))
                                               package--builtin-versions))
                                 (princ "\n")))))

> I just did a test with so-long.el and got slightly odd results.
>
> 1. I ran Emacs 27.2 (which includes so-long 1.0) with a fresh profile.
> 2. I then installed v1.1 from
>    https://elpa.gnu.org/packages/so-long-1.1.tar.lz
> 3. I then quit and ran Emacs 28 (which includes so-long 1.1.2).
> 4. M-x package-refresh-contents
> 5. M-x package-list-packages
> 6. M-x occur RET so-long
>
> Result:
>
> 3 matches for "so-long" in buffer: *Packages*
>     310:  so-long         1.1.2         available         gnu
>     388:  so-long         1.1           installed
>     444:  so-long         1.0           built-in
>
> The actual so-long functionality is good (the built-in version 1.1.2 is
> used in practice, as desired), but that listing is definitely confused.

Quite odd, indeed.  Any chance your `loaddefs.el` was stale?
Try `(cd lisp; make autoloads); make` to rebuild your Emacs after having
explicitly refreshed your autoloads.

You can also try `grep package--builtin-versions **/*.el | grep long` ?


        Stefan

Re: ELPA and built-in packages

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> It should, yes.

Cool; I was having some problems here when working on a built-in package
that I also had installed from ELPA, but ---

> The info is stored in `package--builtin-versions` which is populated
> from the autoloads information in `lisp/loaddefs.el`, which is
> generated by `lisp/emacs-lisp/autoload.el`:

--- so I guess my problem was that I hadn't bumped the version number in
the built-in version and/or hadn't regenerated the loaddefs?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: ELPA and built-in packages

[Stefan Monnier]

Lars Ingebrigtsen [2021-11-01 14:35:41] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> It should, yes.
> Cool; I was having some problems here when working on a built-in package
> that I also had installed from ELPA, but ---
>> The info is stored in `package--builtin-versions` which is populated
>> from the autoloads information in `lisp/loaddefs.el`, which is
>> generated by `lisp/emacs-lisp/autoload.el`:
> --- so I guess my problem was that I hadn't bumped the version number in
> the built-in version and/or hadn't regenerated the loaddefs?

Could be.  Our makefile doesn't automatically refresh
`lisp/loaddefs.el`, IIRC.

IIRC a known problem that can happen is when the package is preloaded
(e.g. `cl-generic`) in which case `autoload.el` skips it altogether,
which is why `cl-generic.el` and `nadvice.el` have an explicit

    (push (purecopy '(<PKG> 1 0)) package--builtin-versions)

Of course, there might be other problems,


        Stefan