bug#71356: use-package doesn't load org from elpa

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 5118 bytes --]

A minimal init.el:
-----
(package-initialize)
(package-refresh-contents)
(use-package org
  :ensure t
  :pin gnu)
-----
Expected result would be C-h v org-version returning 9.7.2, but I see
9.6.15 (the builtin package)

ls -lR in the init directory doesn't show any org-.. files

Emacs version:

In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.33, cairo version 1.16.0) of 2024-06-03 built on 4fcd1a4aa6cf
Repository revision: 760b54de080c238ea9f7b16055e820862d3e8896
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12201001
System Description: Ubuntu 22.04.4 LTS

Configured using:
 'configure --prefix=/usr --program-suffix=30 --with-x
 --with-x-toolkit=gtk3 --with-cairo --with-compress-install
 --with-modules=yes --with-threads --with-included-regex --with-zlib
 --with-json --with-rsvg --with-small-ja-dic
 --with-native-compilation=aot --with-tree-sitter=no 'CFLAGS=-g -O2
 -ffile-prefix-map=/home/paag/emacs=. -flto=auto -ffat-lto-objects
 -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat
 -Werror=format-security' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
 'LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects
 -flto=auto -Wl,-z,relro''

Configured features:
CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG
LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG
SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XINPUT2 XPM
GTK3 ZLIB

Important settings:
  value of $LC_MONETARY: es_ES.UTF-8
  value of $LC_NUMERIC: es_ES.UTF-8
  value of $LC_TIME: es_ES.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
/usr/share/emacs/site-lisp/latex-cjk-thai/thai-word hides
/usr/share/emacs/30.0.50/lisp/language/thai-word

Features:
(shadow sort mail-extr emacsbug compile comp-run comp-common org ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-src ob-comint
org-pcomplete pcomplete comint ansi-osc ansi-color ring org-list
org-footnote org-faces org-entities noutline outline org-version
ob-emacs-lisp ob-core ob-eval org-cycle org-table ol rx org-fold
org-fold-core org-keys oc org-loaddefs find-func cal-menu calendar
cal-loaddefs org-compat org-macs format-spec cl-extra help-mode
use-package-ensure use-package-core mm-archive message sendmail
yank-media dired dired-loaddefs rfc822 mml mml-sec epa derived gnus-util
text-property-search time-date mailabbrev gmm-utils mailheader mm-decode
mm-bodies mm-encode mail-utils gnutls network-stream url-cache url-http
url-auth mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums
mail-prsvr url-gw nsm puny epg rfc6068 epg-config finder-inf package
browse-url url url-proxy url-privacy url-expand url-methods url-history
url-cookie generate-lisp-file url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs icons
password-cache json subr-x map byte-opt gv bytecomp byte-compile
url-vars cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc paren
electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify
dynamic-setting system-font-setting font-render-setting cairo gtk
x-toolkit xinput2 x multi-tty move-toolbar make-network-process
native-compile emacs)

Memory information:
((conses 16 192752 19341) (symbols 48 14741 0) (strings 32 58804 3103)
 (string-bytes 1 1841578) (vectors 16 24106)
 (vector-slots 8 293920 10503) (floats 8 121 18) (intervals 56 462 0)
 (buffers 984 13))




-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 6449 bytes --]

bug#71356: use-package doesn't load org from elpa

[Andrea Corallo]

[-- Attachment #1: Type: text/plain, Size: 718 bytes --]

tags 71356 patch
thanks

Pedro Andres Aranda Gutierrez <paaguti@gmail.com> writes:

> A minimal init.el:
> -----
> (package-initialize)
> (package-refresh-contents)
> (use-package org
>   :ensure t
>   :pin gnu)
> -----
> Expected result would be C-h v org-version returning 9.7.2, but I see 9.6.15 (the builtin package)

I can reproduce it.

Seems the issue is in 'use-package-ensure-elpa' where we gate any
installation with "(unless (package-installed-p package)".  I think we
should progress also if we see that the package is built-in and is
actually pinned.

The attached seems to do the job for me, but I'm not 100% sure it's the
best/right fix so I'd appretiate someone else to have a look.

Thanks

  Andrea


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-use-package-for-built-in-pinned-packages.patch --]
[-- Type: text/x-diff, Size: 3255 bytes --]

From 4942ef85c2db0fb7b24e87da57456be208e83605 Mon Sep 17 00:00:00 2001
From: Andrea Corallo <acorallo@gnu.org>
Date: Tue, 4 Jun 2024 23:30:07 +0200
Subject: [PATCH] Fix use-package for built-in pinned packages

* lisp/use-package/use-package-ensure.el (use-package-ensure-elpa):
Always install built-in pinned packages.
---
 lisp/use-package/use-package-ensure.el | 47 ++++++++++++++------------
 1 file changed, 26 insertions(+), 21 deletions(-)

diff --git a/lisp/use-package/use-package-ensure.el b/lisp/use-package/use-package-ensure.el
index 5f75b6b59ea..a6ed980610f 100644
--- a/lisp/use-package/use-package-ensure.el
+++ b/lisp/use-package/use-package-ensure.el
@@ -157,28 +157,33 @@ use-package-ensure-elpa
                ensure)))
       (when package
         (require 'package)
-        (when (consp package)
-          (use-package-pin-package (car package) (cdr package))
-          (setq package (car package)))
-        (unless (package-installed-p package)
-          (condition-case-unless-debug err
-              (progn
-                (when (assoc package (bound-and-true-p
-                                      package-pinned-packages))
-                  (package-read-all-archive-contents))
-                (if (assoc package package-archive-contents)
-                    (package-install package)
-                  (package-refresh-contents)
-                  (when (assoc package (bound-and-true-p
-                                        package-pinned-packages))
+        (let* ((pinned (assoc package (bound-and-true-p
+                                       package-pinned-packages)))
+               (need-upgrade (and pinned (package-built-in-p package))))
+          (when (consp package)
+            (use-package-pin-package (car package) (cdr package))
+            (setq package (car package)))
+          (when (or (not (package-installed-p package)) need-upgrade)
+            (condition-case-unless-debug err
+                (progn
+                  (when pinned
                     (package-read-all-archive-contents))
-                  (package-install package))
-                t)
-            (error
-             (display-warning 'use-package
-                              (format "Failed to install %s: %s"
-                                      name (error-message-string err))
-                              :error))))))))
+                  (if (assoc package package-archive-contents)
+                      (if need-upgrade
+                          (package-upgrade package)
+                        (package-install package))
+                    (package-refresh-contents)
+                    (when pinned
+                      (package-read-all-archive-contents))
+                    (if need-upgrade
+                        (package-upgrade package)
+                      (package-install package)))
+                  t)
+              (error
+               (display-warning 'use-package
+                                (format "Failed to install %s: %s"
+                                        name (error-message-string err))
+                                :error)))))))))
 
 ;;;###autoload
 (defun use-package-handler/:ensure (name _keyword ensure rest state)
-- 
2.34.1

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 1130 bytes --]

Seems to for me too

Thanks, /PA

On Tue, 4 Jun 2024 at 23:44, Andrea Corallo <acorallo@gnu.org> wrote:

> tags 71356 patch
> thanks
>
> Pedro Andres Aranda Gutierrez <paaguti@gmail.com> writes:
>
> > A minimal init.el:
> > -----
> > (package-initialize)
> > (package-refresh-contents)
> > (use-package org
> >   :ensure t
> >   :pin gnu)
> > -----
> > Expected result would be C-h v org-version returning 9.7.2, but I see
> 9.6.15 (the builtin package)
>
> I can reproduce it.
>
> Seems the issue is in 'use-package-ensure-elpa' where we gate any
> installation with "(unless (package-installed-p package)".  I think we
> should progress also if we see that the package is built-in and is
> actually pinned.
>
> The attached seems to do the job for me, but I'm not 100% sure it's the
> best/right fix so I'd appretiate someone else to have a look.
>
> Thanks
>
>   Andrea
>
>

-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 1895 bytes --]

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> Cc: 71356@debbugs.gnu.org
> From: Andrea Corallo <acorallo@gnu.org>
> Date: Tue, 04 Jun 2024 17:44:37 -0400
> 
> Seems the issue is in 'use-package-ensure-elpa' where we gate any
> installation with "(unless (package-installed-p package)".  I think we
> should progress also if we see that the package is built-in and is
> actually pinned.
> 
> The attached seems to do the job for me, but I'm not 100% sure it's the
> best/right fix so I'd appretiate someone else to have a look.

Isn't this because we require an explicit directive by the user in
order to upgrade a built-in package?  The Emacs user manual says:

     By default, ‘package-install’ doesn't consider built-in packages for
  which new versions are available from the archives.  (A package is
  built-in if it is included in the Emacs distribution.)  In particular,
  it will not show built-in packages in the list of completion candidates
  when you type at its prompt.  But if you invoke ‘package-install’ with a
  prefix argument, it will also consider built-in packages that can be
  upgraded.  You can make this behavior the default by customizing the
  variable ‘package-install-upgrade-built-in’: if its value is non-‘nil’,
  ‘package-install’ will consider built-in packages even when invoked
  without a prefix argument.  Note that the package-menu commands (*note
  Package Menu::) are also affected by ‘package-install-upgrade-built-in’.

     By contrast, ‘package-upgrade’ and ‘package-upgrade-all’ never
  consider built-in packages.  If you want to use these commands for
  upgrading some built-in packages, you need to upgrade each of those
  packages, once, either via ‘C-u M-x package-install <RET>’, or by
  customizing ‘package-install-upgrade-built-in’ to a non-‘nil’ value, and
  then upgrading the package once via the package menu or by
  ‘package-install’.

We had a long (and somewhat heated) discussion about this a year ago,
see bug#62720.

Philip, am I missing something?

bug#71356: use-package doesn't load org from elpa

[Andrea Corallo]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: 71356@debbugs.gnu.org
>> From: Andrea Corallo <acorallo@gnu.org>
>> Date: Tue, 04 Jun 2024 17:44:37 -0400
>>
>> Seems the issue is in 'use-package-ensure-elpa' where we gate any
>> installation with "(unless (package-installed-p package)".  I think we
>> should progress also if we see that the package is built-in and is
>> actually pinned.
>>
>> The attached seems to do the job for me, but I'm not 100% sure it's the
>> best/right fix so I'd appretiate someone else to have a look.
>
> Isn't this because we require an explicit directive by the user in
> order to upgrade a built-in package?  The Emacs user manual says:
>
>      By default, ‘package-install’ doesn't consider built-in packages for
>   which new versions are available from the archives.  (A package is
>   built-in if it is included in the Emacs distribution.)  In particular,
>   it will not show built-in packages in the list of completion candidates
>   when you type at its prompt.  But if you invoke ‘package-install’ with a
>   prefix argument, it will also consider built-in packages that can be
>   upgraded.  You can make this behavior the default by customizing the
>   variable ‘package-install-upgrade-built-in’: if its value is non-‘nil’,
>   ‘package-install’ will consider built-in packages even when invoked
>   without a prefix argument.  Note that the package-menu commands (*note
>   Package Menu::) are also affected by ‘package-install-upgrade-built-in’.
>
>      By contrast, ‘package-upgrade’ and ‘package-upgrade-all’ never
>   consider built-in packages.  If you want to use these commands for
>   upgrading some built-in packages, you need to upgrade each of those
>   packages, once, either via ‘C-u M-x package-install <RET>’, or by
>   customizing ‘package-install-upgrade-built-in’ to a non-‘nil’ value, and
>   then upgrading the package once via the package menu or by
>   ‘package-install’.
>
> We had a long (and somewhat heated) discussion about this a year ago,
> see bug#62720.

I see thanks, OTOH this report is about the use-package macro not
package itself.

use-package doc doesn't mention built-in packages, but describes the two
keyword parameters as:

:ensure          Loads the package using package.el if necessary.
:pin             Pin the package to an archive.

So I found reasonable that for the reported case the user expects the
package to be loaded using package.el.  But as I mentioned I'm no expert
in this area so I might very well be off :)

Thanks

  Andrea

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 3514 bytes --]

Hi,

Andrea is right. Reading through all the documentation, I implied
that use-package would upgrade built-in packages if I pinned them to an
archive
and I :ensure'd them.

Use case: want to upgrade org from the 9.6.x version packaged with master
to the
9.7.x version available in elpa.

Maybe this is more a FR and if so, we could move this to the list and have
an informed
discussion there.

Best, /PA

On Wed, 5 Jun 2024 at 20:09, Andrea Corallo <acorallo@gnu.org> wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> >> Cc: 71356@debbugs.gnu.org
> >> From: Andrea Corallo <acorallo@gnu.org>
> >> Date: Tue, 04 Jun 2024 17:44:37 -0400
> >>
> >> Seems the issue is in 'use-package-ensure-elpa' where we gate any
> >> installation with "(unless (package-installed-p package)".  I think we
> >> should progress also if we see that the package is built-in and is
> >> actually pinned.
> >>
> >> The attached seems to do the job for me, but I'm not 100% sure it's the
> >> best/right fix so I'd appretiate someone else to have a look.
> >
> > Isn't this because we require an explicit directive by the user in
> > order to upgrade a built-in package?  The Emacs user manual says:
> >
> >      By default, ‘package-install’ doesn't consider built-in packages for
> >   which new versions are available from the archives.  (A package is
> >   built-in if it is included in the Emacs distribution.)  In particular,
> >   it will not show built-in packages in the list of completion candidates
> >   when you type at its prompt.  But if you invoke ‘package-install’ with
> a
> >   prefix argument, it will also consider built-in packages that can be
> >   upgraded.  You can make this behavior the default by customizing the
> >   variable ‘package-install-upgrade-built-in’: if its value is non-‘nil’,
> >   ‘package-install’ will consider built-in packages even when invoked
> >   without a prefix argument.  Note that the package-menu commands (*note
> >   Package Menu::) are also affected by
> ‘package-install-upgrade-built-in’.
> >
> >      By contrast, ‘package-upgrade’ and ‘package-upgrade-all’ never
> >   consider built-in packages.  If you want to use these commands for
> >   upgrading some built-in packages, you need to upgrade each of those
> >   packages, once, either via ‘C-u M-x package-install <RET>’, or by
> >   customizing ‘package-install-upgrade-built-in’ to a non-‘nil’ value,
> and
> >   then upgrading the package once via the package menu or by
> >   ‘package-install’.
> >
> > We had a long (and somewhat heated) discussion about this a year ago,
> > see bug#62720.
>
> I see thanks, OTOH this report is about the use-package macro not
> package itself.
>
> use-package doc doesn't mention built-in packages, but describes the two
> keyword parameters as:
>
> :ensure          Loads the package using package.el if necessary.
> :pin             Pin the package to an archive.
>
> So I found reasonable that for the reported case the user expects the
> package to be loaded using package.el.  But as I mentioned I'm no expert
> in this area so I might very well be off :)
>
> Thanks
>
>   Andrea
>
>

-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 4699 bytes --]

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> Date: Thu, 6 Jun 2024 07:46:31 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, Philip Kaludercic <philipk@posteo.net>, 71356@debbugs.gnu.org
> 
> Andrea is right. Reading through all the documentation, I implied
> that use-package would upgrade built-in packages if I pinned them to an archive
> and I :ensure'd them. 
> 
> Use case: want to upgrade org from the 9.6.x version packaged with master to the
> 9.7.x version available in elpa. 
> 
> Maybe this is more a FR and if so, we could move this to the list and have an informed 
> discussion there.

I'd like to hear from Philip.  If use-package just uses package.el,
then the behavior is expected.

Do you have package-install-upgrade-built-in set non-nil?  If not, can
you set it non-nil and try the recipe again?

As for a feature request: what exactly is the feature requested here?
Are you saying that use-package should automatically upgrade built-in
packages?  If so, I don't think this will fly, since it would mean
inconsistencies with package-install.

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 2185 bytes --]

Answers inline

On Thu, 6 Jun 2024 at 08:02, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> > Date: Thu, 6 Jun 2024 07:46:31 +0200
> > Cc: Eli Zaretskii <eliz@gnu.org>, Philip Kaludercic <philipk@posteo.net>,
> 71356@debbugs.gnu.org
> >
> > Andrea is right. Reading through all the documentation, I implied
> > that use-package would upgrade built-in packages if I pinned them to an
> archive
> > and I :ensure'd them.
> >
> > Use case: want to upgrade org from the 9.6.x version packaged with
> master to the
> > 9.7.x version available in elpa.
> >
> > Maybe this is more a FR and if so, we could move this to the list and
> have an informed
> > discussion there.
>
> I'd like to hear from Philip.  If use-package just uses package.el,
> then the behavior is expected.
>

Let's see


> Do you have package-install-upgrade-built-in set non-nil?  If not, can
> you set it non-nil and try the recipe again?
>

Retried on an unpatched emacs master with
package-install-upgrade-builtin set to t and had the same behaviour.


> As for a feature request: what exactly is the feature requested here?
> Are you saying that use-package should automatically upgrade built-in
> packages?  If so, I don't think this will fly, since it would mean
> inconsistencies with package-install.
>

This is exactly what I would like to discuss ;-) What options do we have to
allow built-in packages to be
upgradable from the archives? Maybe something that would emulate

#+begin_src
(require 'package)
(setq package-install-upgrade-built-in t)
(package-initialize)
(when (not package-archive-contents)
  (package-refresh-contents))
(package-install 'org)
#+end-src

A new keyword in combination with :pin so that we can cherry-pick which
packages we want to actually refresh from elpa
and which ones we are fine with if they are built in?

I just want to raise the question (see below ;-) )

Best, /PA
-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 3566 bytes --]

bug#71356: use-package doesn't load org from elpa

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
>> Date: Thu, 6 Jun 2024 07:46:31 +0200
>> Cc: Eli Zaretskii <eliz@gnu.org>, Philip Kaludercic <philipk@posteo.net>, 71356@debbugs.gnu.org
>> 
>> Andrea is right. Reading through all the documentation, I implied
>> that use-package would upgrade built-in packages if I pinned them to an archive
>> and I :ensure'd them. 
>> 
>> Use case: want to upgrade org from the 9.6.x version packaged with master to the
>> 9.7.x version available in elpa. 
>> 
>> Maybe this is more a FR and if so, we could move this to the list and have an informed 
>> discussion there.
>
> I'd like to hear from Philip.  If use-package just uses package.el,
> then the behavior is expected.

Sorry for the delayed response;  I don't think that has to be expected.
While use-package can utilise package.el for package management, my
impression is that it is at liberty to be more flexible/declarative.  

> Do you have package-install-upgrade-built-in set non-nil?  If not, can
> you set it non-nil and try the recipe again?

I have tried it out myself, and it doesn't appear to do anything.  The
issue looks like that `package-installed-p' doesn't respect
package-install-upgrade-built-in or :pin.

> As for a feature request: what exactly is the feature requested here?
> Are you saying that use-package should automatically upgrade built-in
> packages?  If so, I don't think this will fly, since it would mean
> inconsistencies with package-install.

IIUC the feature would be that if a use-package form has a

     :pin gnu

argument, then this is an indication that we want to install the package
from GNU ELPA, disregarding the fact that Emacs already has a built-in
version of the same package.  Sort of a package-local version of
`package-install-upgrade-built-in'.

I am not familiar with the use-package code, but it seems like we could
implement this generally in package-install, by checking
`package-pinned-packages'.

-- 
	Philip Kaludercic on peregrine

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> Date: Thu, 6 Jun 2024 08:11:37 +0200
> Cc: acorallo@gnu.org, philipk@posteo.net, 71356@debbugs.gnu.org
> 
>  Do you have package-install-upgrade-built-in set non-nil?  If not, can
>  you set it non-nil and try the recipe again?
> 
> Retried on an unpatched emacs master with package-install-upgrade-builtin set to t and had the same
> behaviour.

I think this is a bug.

>  As for a feature request: what exactly is the feature requested here?
>  Are you saying that use-package should automatically upgrade built-in
>  packages?  If so, I don't think this will fly, since it would mean
>  inconsistencies with package-install.
> 
> This is exactly what I would like to discuss ;-) What options do we have to allow built-in packages to be 
> upgradable from the archives?

I quoted from the Emacs manual what should be done to allow that.  If
and when use-package supports that, it's how we decided to handle
built-in packages: by default not upgraded, unless the user says to
upgrade them in one of those ways.

> A new keyword in combination with :pin so that we can cherry-pick which packages we want to actually refresh
> from elpa
> and which ones we are fine with if they are built in?

Maybe.  First we should see that use-package obeys the user option I
mentioned.

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>,  acorallo@gnu.org,
>   71356@debbugs.gnu.org
> Date: Thu, 06 Jun 2024 06:15:44 +0000
> 
> Sorry for the delayed response;  I don't think that has to be expected.
> While use-package can utilise package.el for package management, my
> impression is that it is at liberty to be more flexible/declarative.  

Doesn't use-package utilize package.el already?

If not, how does it handle installation and upgrades? by its own code?

> > Do you have package-install-upgrade-built-in set non-nil?  If not, can
> > you set it non-nil and try the recipe again?
> 
> I have tried it out myself, and it doesn't appear to do anything.  The
> issue looks like that `package-installed-p' doesn't respect
> package-install-upgrade-built-in or :pin.

We should fix that, I think.  If package-install-upgrade-built-in is
non-nil, use-package should upgrade built-in packages.

> > As for a feature request: what exactly is the feature requested here?
> > Are you saying that use-package should automatically upgrade built-in
> > packages?  If so, I don't think this will fly, since it would mean
> > inconsistencies with package-install.
> 
> IIUC the feature would be that if a use-package form has a
> 
>      :pin gnu
> 
> argument, then this is an indication that we want to install the package
> from GNU ELPA, disregarding the fact that Emacs already has a built-in
> version of the same package.  Sort of a package-local version of
> `package-install-upgrade-built-in'.

I'm not sure.  People tend to copy/paste recipes from the Internet
without really understanding what they do.  I think a simple :pin
should not be sufficient, we need some specialized keyword (in
addition to supporting package-install-upgrade-built-in).

> I am not familiar with the use-package code, but it seems like we could
> implement this generally in package-install, by checking
> `package-pinned-packages'.

I would prefer not to introduce another indication of whether built-in
packages should or should not be upgraded.  If we do, we will next
need to decide which one "wins" when they contradict each other.

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 3304 bytes --]

Answers inline

On Thu, 6 Jun 2024 at 11:21, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Philip Kaludercic <philipk@posteo.net>
> > Cc: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>,  acorallo@gnu.org
> ,
> >   71356@debbugs.gnu.org
> > Date: Thu, 06 Jun 2024 06:15:44 +0000
> >
> > Sorry for the delayed response;  I don't think that has to be expected.
> > While use-package can utilise package.el for package management, my
> > impression is that it is at liberty to be more flexible/declarative.
>
> Doesn't use-package utilize package.el already?
>
> If not, how does it handle installation and upgrades? by its own code?
>
> > > Do you have package-install-upgrade-built-in set non-nil?  If not, can
> > > you set it non-nil and try the recipe again?
> >
> > I have tried it out myself, and it doesn't appear to do anything.  The
> > issue looks like that `package-installed-p' doesn't respect
> > package-install-upgrade-built-in or :pin.
>
> We should fix that, I think.  If package-install-upgrade-built-in is
> non-nil, use-package should upgrade built-in packages.
>
> > > As for a feature request: what exactly is the feature requested here?
> > > Are you saying that use-package should automatically upgrade built-in
> > > packages?  If so, I don't think this will fly, since it would mean
> > > inconsistencies with package-install.
> >
> > IIUC the feature would be that if a use-package form has a
> >
> >      :pin gnu
> >
> > argument, then this is an indication that we want to install the package
> > from GNU ELPA, disregarding the fact that Emacs already has a built-in
> > version of the same package.  Sort of a package-local version of
> > `package-install-upgrade-built-in'.
>
> I'm not sure.  People tend to copy/paste recipes from the Internet
> without really understanding what they do.  I think a simple :pin
> should not be sufficient, we need some specialized keyword (in
> addition to supporting package-install-upgrade-built-in).


I didn't arrive at trying :pin gnu from anything in the Internet, but from
reading the use-package documentation (just this time ;-) )

> I am not familiar with the use-package code, but it seems like we could
> > implement this generally in package-install, by checking
> > `package-pinned-packages'.
>
> I would prefer not to introduce another indication of whether built-in
> packages should or should not be upgraded.  If we do, we will next
> need to decide which one "wins" when they contradict each other.
>

My feeling is that if I set package-install-upgrade-built-in to t and pin
a package to (say) gnu elpa, that should be enough. I may resort to
use-package from everything, but would not use :pin on built-in packages
that I don't want to upgrade (makes no sense, right?).

It's a bit like adding a load-path to the use-package call
and download the package externally
(for example cloning with git) into that directory, which is
the way I use to get the development version of, for example, org-mode
when I want to contribute to it.

Does it sound strange?
 /PA


-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 4763 bytes --]

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> Date: Thu, 6 Jun 2024 17:07:02 +0200
> Cc: Philip Kaludercic <philipk@posteo.net>, acorallo@gnu.org, 71356@debbugs.gnu.org
> 
>  > IIUC the feature would be that if a use-package form has a
>  > 
>  >      :pin gnu
>  > 
>  > argument, then this is an indication that we want to install the package
>  > from GNU ELPA, disregarding the fact that Emacs already has a built-in
>  > version of the same package.  Sort of a package-local version of
>  > `package-install-upgrade-built-in'.
> 
>  I'm not sure.  People tend to copy/paste recipes from the Internet
>  without really understanding what they do.  I think a simple :pin
>  should not be sufficient, we need some specialized keyword (in
>  addition to supporting package-install-upgrade-built-in).
> 
> I didn't arrive at trying :pin gnu from anything in the Internet, but from
> reading the use-package documentation (just this time ;-) )
> 
>  > I am not familiar with the use-package code, but it seems like we could
>  > implement this generally in package-install, by checking
>  > `package-pinned-packages'.
> 
>  I would prefer not to introduce another indication of whether built-in
>  packages should or should not be upgraded.  If we do, we will next
>  need to decide which one "wins" when they contradict each other.
> 
>  
> My feeling is that if I set package-install-upgrade-built-in to t and pin
> a package to (say) gnu elpa, that should be enough.

I agree.  I was responding to the suggestion that just :pin should be
enough.  That use-package currently ignores
package-install-upgrade-built-in is a bug we should surely fix.

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 2599 bytes --]

Just for the record. Further tests I have done:

--- init.el ---
(require 'benchmark)
(require 'package)
(setq custom-file (locate-user-emacs-file "custom.el"))

(when (not package-archive-contents)
  (package-refresh-contents))
(message "Loading org from elpa took %f s"
         (benchmark-elapse
           (setq-default package-install-upgrade-built-in t)
           (use-package org
             :ensure t
             :pin gnu)
  (message "org-version: %s" org-version)))
---

M-x org-version yields 9.6.15 (expected was 9.7.3)

Best, /PA

On Thu, 6 Jun 2024 at 17:19, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> > Date: Thu, 6 Jun 2024 17:07:02 +0200
> > Cc: Philip Kaludercic <philipk@posteo.net>, acorallo@gnu.org,
> 71356@debbugs.gnu.org
> >
> >  > IIUC the feature would be that if a use-package form has a
> >  >
> >  >      :pin gnu
> >  >
> >  > argument, then this is an indication that we want to install the
> package
> >  > from GNU ELPA, disregarding the fact that Emacs already has a built-in
> >  > version of the same package.  Sort of a package-local version of
> >  > `package-install-upgrade-built-in'.
> >
> >  I'm not sure.  People tend to copy/paste recipes from the Internet
> >  without really understanding what they do.  I think a simple :pin
> >  should not be sufficient, we need some specialized keyword (in
> >  addition to supporting package-install-upgrade-built-in).
> >
> > I didn't arrive at trying :pin gnu from anything in the Internet, but
> from
> > reading the use-package documentation (just this time ;-) )
> >
> >  > I am not familiar with the use-package code, but it seems like we
> could
> >  > implement this generally in package-install, by checking
> >  > `package-pinned-packages'.
> >
> >  I would prefer not to introduce another indication of whether built-in
> >  packages should or should not be upgraded.  If we do, we will next
> >  need to decide which one "wins" when they contradict each other.
> >
> >
> > My feeling is that if I set package-install-upgrade-built-in to t and pin
> > a package to (say) gnu elpa, that should be enough.
>
> I agree.  I was responding to the suggestion that just :pin should be
> enough.  That use-package currently ignores
> package-install-upgrade-built-in is a bug we should surely fix.
>


-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 3851 bytes --]

bug#71356: use-package doesn't load org from elpa

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>,  acorallo@gnu.org,
>>   71356@debbugs.gnu.org
>> Date: Thu, 06 Jun 2024 06:15:44 +0000
>> 
>> Sorry for the delayed response;  I don't think that has to be expected.
>> While use-package can utilise package.el for package management, my
>> impression is that it is at liberty to be more flexible/declarative.  
>
> Doesn't use-package utilize package.el already?
>
> If not, how does it handle installation and upgrades? by its own code?

By default it uses package.el, but there is an option to change it.

>> > Do you have package-install-upgrade-built-in set non-nil?  If not, can
>> > you set it non-nil and try the recipe again?
>> 
>> I have tried it out myself, and it doesn't appear to do anything.  The
>> issue looks like that `package-installed-p' doesn't respect
>> package-install-upgrade-built-in or :pin.
>
> We should fix that, I think.  If package-install-upgrade-built-in is
> non-nil, use-package should upgrade built-in packages.
>
>> > As for a feature request: what exactly is the feature requested here?
>> > Are you saying that use-package should automatically upgrade built-in
>> > packages?  If so, I don't think this will fly, since it would mean
>> > inconsistencies with package-install.
>> 
>> IIUC the feature would be that if a use-package form has a
>> 
>>      :pin gnu
>> 
>> argument, then this is an indication that we want to install the package
>> from GNU ELPA, disregarding the fact that Emacs already has a built-in
>> version of the same package.  Sort of a package-local version of
>> `package-install-upgrade-built-in'.
>
> I'm not sure.  People tend to copy/paste recipes from the Internet
> without really understanding what they do.  I think a simple :pin
> should not be sufficient, we need some specialized keyword (in
> addition to supporting package-install-upgrade-built-in).

To me :pin would make perfect sense, as it explicitly expresses what
archive we want to follow for package upgrades.

>> I am not familiar with the use-package code, but it seems like we could
>> implement this generally in package-install, by checking
>> `package-pinned-packages'.
>
> I would prefer not to introduce another indication of whether built-in
> packages should or should not be upgraded.  If we do, we will next
> need to decide which one "wins" when they contradict each other.

One idea would be that use-package would check :pin and then
conditionally bind `package-install-upgrade-built-in' when invoking
`package-install'.  That being said, I am not a fan of the user option
any way, and wouldn't mind if we came up with a cleaner solution.

-- 
	Philip Kaludercic on peregrine

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 3454 bytes --]

On Mon, 10 Jun 2024 at 08:02, Philip Kaludercic <philipk@posteo.net> wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> >> From: Philip Kaludercic <philipk@posteo.net>
> >> Cc: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>,
> acorallo@gnu.org,
> >>   71356@debbugs.gnu.org
> >> Date: Thu, 06 Jun 2024 06:15:44 +0000
> >>
> >> Sorry for the delayed response;  I don't think that has to be expected.
> >> While use-package can utilise package.el for package management, my
> >> impression is that it is at liberty to be more flexible/declarative.
> >
> > Doesn't use-package utilize package.el already?
> >
> > If not, how does it handle installation and upgrades? by its own code?
>
> By default it uses package.el, but there is an option to change it.
>
> >> > Do you have package-install-upgrade-built-in set non-nil?  If not, can
> >> > you set it non-nil and try the recipe again?
> >>
> >> I have tried it out myself, and it doesn't appear to do anything.  The
> >> issue looks like that `package-installed-p' doesn't respect
> >> package-install-upgrade-built-in or :pin.
> >
> > We should fix that, I think.  If package-install-upgrade-built-in is
> > non-nil, use-package should upgrade built-in packages.
> >
> >> > As for a feature request: what exactly is the feature requested here?
> >> > Are you saying that use-package should automatically upgrade built-in
> >> > packages?  If so, I don't think this will fly, since it would mean
> >> > inconsistencies with package-install.
> >>
> >> IIUC the feature would be that if a use-package form has a
> >>
> >>      :pin gnu
> >>
> >> argument, then this is an indication that we want to install the package
> >> from GNU ELPA, disregarding the fact that Emacs already has a built-in
> >> version of the same package.  Sort of a package-local version of
> >> `package-install-upgrade-built-in'.
> >
> > I'm not sure.  People tend to copy/paste recipes from the Internet
> > without really understanding what they do.  I think a simple :pin
> > should not be sufficient, we need some specialized keyword (in
> > addition to supporting package-install-upgrade-built-in).
>
> To me :pin would make perfect sense, as it explicitly expresses what
> archive we want to follow for package upgrades.
>
> >> I am not familiar with the use-package code, but it seems like we could
> >> implement this generally in package-install, by checking
> >> `package-pinned-packages'.
> >
> > I would prefer not to introduce another indication of whether built-in
> > packages should or should not be upgraded.  If we do, we will next
> > need to decide which one "wins" when they contradict each other.
>
> One idea would be that use-package would check :pin and then
> conditionally bind `package-install-upgrade-built-in' when invoking
> `package-install'.  That being said, I am not a fan of the user option
> any way, and wouldn't mind if we came up with a cleaner solution.
>
> --
>         Philip Kaludercic on peregrine
>

Just as a "parachute", one could obey the global
`package-install-upgrade-built-in`
value and make :pin cry out for built-in packages when the user indicates a
repo and
`package-install-upgrade-built-in`  is nil

My .2cents, /PA
-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 4867 bytes --]

bug#71356: use-package doesn't load org from elpa

[Andrea Corallo]

Philip Kaludercic <philipk@posteo.net> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Philip Kaludercic <philipk@posteo.net>
>>> Cc: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>,  acorallo@gnu.org,
>>>   71356@debbugs.gnu.org
>>> Date: Thu, 06 Jun 2024 06:15:44 +0000
>>> 
>>> Sorry for the delayed response;  I don't think that has to be expected.
>>> While use-package can utilise package.el for package management, my
>>> impression is that it is at liberty to be more flexible/declarative.  
>>
>> Doesn't use-package utilize package.el already?
>>
>> If not, how does it handle installation and upgrades? by its own code?
>
> By default it uses package.el, but there is an option to change it.
>
>>> > Do you have package-install-upgrade-built-in set non-nil?  If not, can
>>> > you set it non-nil and try the recipe again?
>>> 
>>> I have tried it out myself, and it doesn't appear to do anything.  The
>>> issue looks like that `package-installed-p' doesn't respect
>>> package-install-upgrade-built-in or :pin.
>>
>> We should fix that, I think.  If package-install-upgrade-built-in is
>> non-nil, use-package should upgrade built-in packages.
>>
>>> > As for a feature request: what exactly is the feature requested here?
>>> > Are you saying that use-package should automatically upgrade built-in
>>> > packages?  If so, I don't think this will fly, since it would mean
>>> > inconsistencies with package-install.
>>> 
>>> IIUC the feature would be that if a use-package form has a
>>> 
>>>      :pin gnu
>>> 
>>> argument, then this is an indication that we want to install the package
>>> from GNU ELPA, disregarding the fact that Emacs already has a built-in
>>> version of the same package.  Sort of a package-local version of
>>> `package-install-upgrade-built-in'.
>>
>> I'm not sure.  People tend to copy/paste recipes from the Internet
>> without really understanding what they do.  I think a simple :pin
>> should not be sufficient, we need some specialized keyword (in
>> addition to supporting package-install-upgrade-built-in).
>
> To me :pin would make perfect sense, as it explicitly expresses what
> archive we want to follow for package upgrades.

+1, also use-package interface is very declarative and I'm not sure
having it influenced by a dynamic var would match user expected
behavior.

  Andrea

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: paaguti@gmail.com,  acorallo@gnu.org,  71356@debbugs.gnu.org
> Date: Mon, 10 Jun 2024 06:02:08 +0000
> 
> >> IIUC the feature would be that if a use-package form has a
> >> 
> >>      :pin gnu
> >> 
> >> argument, then this is an indication that we want to install the package
> >> from GNU ELPA, disregarding the fact that Emacs already has a built-in
> >> version of the same package.  Sort of a package-local version of
> >> `package-install-upgrade-built-in'.
> >
> > I'm not sure.  People tend to copy/paste recipes from the Internet
> > without really understanding what they do.  I think a simple :pin
> > should not be sufficient, we need some specialized keyword (in
> > addition to supporting package-install-upgrade-built-in).
> 
> To me :pin would make perfect sense, as it explicitly expresses what
> archive we want to follow for package upgrades.

Bitter experience should have taught us that what makes perfect sense
to us does not necessarily make such perfect sense to others.  Which
is why we don't like to make incompatible changes in behavior even
though the new behavior sounds like a definite TRT to us.

Let me remind you that similar arguments were voiced to make
package-install-upgrade-built-in be the default.  We didn't.

So I'd like us to trod cautiously here, abiding by the same logic as
package-install-upgrade-built-in.  If nothing else, that's consistent
with other methods of upgrading built-in packages.

> >> I am not familiar with the use-package code, but it seems like we could
> >> implement this generally in package-install, by checking
> >> `package-pinned-packages'.
> >
> > I would prefer not to introduce another indication of whether built-in
> > packages should or should not be upgraded.  If we do, we will next
> > need to decide which one "wins" when they contradict each other.
> 
> One idea would be that use-package would check :pin and then
> conditionally bind `package-install-upgrade-built-in' when invoking
> `package-install'.  That being said, I am not a fan of the user option
> any way, and wouldn't mind if we came up with a cleaner solution.

Cleaner that package-install-upgrade-built-in?  Why is that "unclean"?
Given that users may or may not want the built-in packages to be
updated en-masse with their usual updates, a user option lets everyone
have what they prefer, so it's the cleanest possible solution, IMO.

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  paaguti@gmail.com,  71356@debbugs.gnu.org
> Date: Mon, 10 Jun 2024 04:17:21 -0400
> 
> Philip Kaludercic <philipk@posteo.net> writes:
> 
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> >>> From: Philip Kaludercic <philipk@posteo.net>
> >>> Cc: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>,  acorallo@gnu.org,
> >>>   71356@debbugs.gnu.org
> >>> Date: Thu, 06 Jun 2024 06:15:44 +0000
> >>> 
> >>> Sorry for the delayed response;  I don't think that has to be expected.
> >>> While use-package can utilise package.el for package management, my
> >>> impression is that it is at liberty to be more flexible/declarative.  
> >>
> >> Doesn't use-package utilize package.el already?
> >>
> >> If not, how does it handle installation and upgrades? by its own code?
> >
> > By default it uses package.el, but there is an option to change it.
> >
> >>> > Do you have package-install-upgrade-built-in set non-nil?  If not, can
> >>> > you set it non-nil and try the recipe again?
> >>> 
> >>> I have tried it out myself, and it doesn't appear to do anything.  The
> >>> issue looks like that `package-installed-p' doesn't respect
> >>> package-install-upgrade-built-in or :pin.
> >>
> >> We should fix that, I think.  If package-install-upgrade-built-in is
> >> non-nil, use-package should upgrade built-in packages.
> >>
> >>> > As for a feature request: what exactly is the feature requested here?
> >>> > Are you saying that use-package should automatically upgrade built-in
> >>> > packages?  If so, I don't think this will fly, since it would mean
> >>> > inconsistencies with package-install.
> >>> 
> >>> IIUC the feature would be that if a use-package form has a
> >>> 
> >>>      :pin gnu
> >>> 
> >>> argument, then this is an indication that we want to install the package
> >>> from GNU ELPA, disregarding the fact that Emacs already has a built-in
> >>> version of the same package.  Sort of a package-local version of
> >>> `package-install-upgrade-built-in'.
> >>
> >> I'm not sure.  People tend to copy/paste recipes from the Internet
> >> without really understanding what they do.  I think a simple :pin
> >> should not be sufficient, we need some specialized keyword (in
> >> addition to supporting package-install-upgrade-built-in).
> >
> > To me :pin would make perfect sense, as it explicitly expresses what
> > archive we want to follow for package upgrades.
> 
> +1, also use-package interface is very declarative and I'm not sure
> having it influenced by a dynamic var would match user expected
> behavior.

If you prefer, we could add a new :foo keyword to mean this.  But
unconditionally changing what :pin means in these cases is out of the
question.

bug#71356: use-package doesn't load org from elpa

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 3014 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andrea Corallo <acorallo@gnu.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  paaguti@gmail.com,  71356@debbugs.gnu.org
>> Date: Mon, 10 Jun 2024 04:17:21 -0400
>> 
>> Philip Kaludercic <philipk@posteo.net> writes:
>> 
>> > Eli Zaretskii <eliz@gnu.org> writes:
>> >
>> >>> From: Philip Kaludercic <philipk@posteo.net>
>> >>> Cc: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>,  acorallo@gnu.org,
>> >>>   71356@debbugs.gnu.org
>> >>> Date: Thu, 06 Jun 2024 06:15:44 +0000
>> >>> 
>> >>> Sorry for the delayed response;  I don't think that has to be expected.
>> >>> While use-package can utilise package.el for package management, my
>> >>> impression is that it is at liberty to be more flexible/declarative.  
>> >>
>> >> Doesn't use-package utilize package.el already?
>> >>
>> >> If not, how does it handle installation and upgrades? by its own code?
>> >
>> > By default it uses package.el, but there is an option to change it.
>> >
>> >>> > Do you have package-install-upgrade-built-in set non-nil?  If not, can
>> >>> > you set it non-nil and try the recipe again?
>> >>> 
>> >>> I have tried it out myself, and it doesn't appear to do anything.  The
>> >>> issue looks like that `package-installed-p' doesn't respect
>> >>> package-install-upgrade-built-in or :pin.
>> >>
>> >> We should fix that, I think.  If package-install-upgrade-built-in is
>> >> non-nil, use-package should upgrade built-in packages.
>> >>
>> >>> > As for a feature request: what exactly is the feature requested here?
>> >>> > Are you saying that use-package should automatically upgrade built-in
>> >>> > packages?  If so, I don't think this will fly, since it would mean
>> >>> > inconsistencies with package-install.
>> >>> 
>> >>> IIUC the feature would be that if a use-package form has a
>> >>> 
>> >>>      :pin gnu
>> >>> 
>> >>> argument, then this is an indication that we want to install the package
>> >>> from GNU ELPA, disregarding the fact that Emacs already has a built-in
>> >>> version of the same package.  Sort of a package-local version of
>> >>> `package-install-upgrade-built-in'.
>> >>
>> >> I'm not sure.  People tend to copy/paste recipes from the Internet
>> >> without really understanding what they do.  I think a simple :pin
>> >> should not be sufficient, we need some specialized keyword (in
>> >> addition to supporting package-install-upgrade-built-in).
>> >
>> > To me :pin would make perfect sense, as it explicitly expresses what
>> > archive we want to follow for package upgrades.
>> 
>> +1, also use-package interface is very declarative and I'm not sure
>> having it influenced by a dynamic var would match user expected
>> behavior.
>
> If you prefer, we could add a new :foo keyword to mean this.  But
> unconditionally changing what :pin means in these cases is out of the
> question.

We wouldn't change what :pin means directly, but just have
package-install respect `package-pinned-packages'.  It seems that all we
have to change is this:


[-- Attachment #2: Type: text/plain, Size: 2047 bytes --]

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index fda855d2143..562dc5dbca3 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2173,7 +2173,8 @@ package-installed-p
             (version-list-<= min-version
                              (package-desc-version (car pkg-descs)))))
      ;; Also check built-in packages.
-     (package-built-in-p package min-version)))))
+     (and (not (package-install-upgrade-built-in-p package))
+          (package-built-in-p package min-version))))))
 
 (defun package-download-transaction (packages)
   "Download and install all the packages in PACKAGES.
@@ -2197,6 +2198,11 @@ package-install-upgrade-built-in
   :type 'boolean
   :version "29.1")
 
+(defun package-install-upgrade-built-in-p (pkg)
+  "Return non-nil if PKG should be upgraded."
+  (or (assq pkg package-pinned-packages)
+      package-install-upgrade-built-in))
+
 ;;;###autoload
 (defun package-install (pkg &optional dont-select)
   "Install the package PKG.
@@ -2226,7 +2232,7 @@ package-install
                     (mapcan
                      (lambda (elt)
                        (and (or (and (or current-prefix-arg
-                                         package-install-upgrade-built-in)
+                                         (package-install-upgrade-built-in-p elt))
                                      (package--active-built-in-p (car elt)))
                                 (not (package-installed-p (car elt))))
                             (list (symbol-name (car elt)))))
@@ -2241,7 +2247,7 @@ package-install
     (unless (or dont-select (package--user-selected-p name))
       (package--save-selected-packages
        (cons name package-selected-packages)))
-    (when (and (or current-prefix-arg package-install-upgrade-built-in)
+    (when (and (or current-prefix-arg (package-install-upgrade-built-in-p name))
                (package--active-built-in-p pkg))
       (setq pkg (or (cadr (assq name package-archive-contents)) pkg)))
     (if-let* ((transaction

[-- Attachment #3: Type: text/plain, Size: 130 bytes --]


(not thoroughly tested, just a sketch that makes (use-package org
:ensure t :pin gnu) work)

-- 
	Philip Kaludercic on peregrine

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: Andrea Corallo <acorallo@gnu.org>,  paaguti@gmail.com,
>   71356@debbugs.gnu.org
> Date: Mon, 10 Jun 2024 15:40:58 +0000
> 
> >> > To me :pin would make perfect sense, as it explicitly expresses what
> >> > archive we want to follow for package upgrades.
> >> 
> >> +1, also use-package interface is very declarative and I'm not sure
> >> having it influenced by a dynamic var would match user expected
> >> behavior.
> >
> > If you prefer, we could add a new :foo keyword to mean this.  But
> > unconditionally changing what :pin means in these cases is out of the
> > question.
> 
> We wouldn't change what :pin means directly, but just have
> package-install respect `package-pinned-packages'.

How is that different?  It's an incompatible change of behavior, and I
cannot agree to that, sorry.

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 1636 bytes --]

We don't want to change what :pin means, IMHO we need it to obey
`package-install-upgrade-built-in`.
That wouldn't be incompatible, right?

So the sketch that actually loads org from elpa would be

(let ((package-install-upgrade-built-in t))
  (use-package org
    :ensure t :pin gnu))

whereas

  (use-package org
    :ensure t :pin gnu)

would keep the original org packaged with emacs (called with emacs -Q)

Best, /PA


On Mon, 10 Jun 2024 at 18:12, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Philip Kaludercic <philipk@posteo.net>
> > Cc: Andrea Corallo <acorallo@gnu.org>,  paaguti@gmail.com,
> >   71356@debbugs.gnu.org
> > Date: Mon, 10 Jun 2024 15:40:58 +0000
> >
> > >> > To me :pin would make perfect sense, as it explicitly expresses what
> > >> > archive we want to follow for package upgrades.
> > >>
> > >> +1, also use-package interface is very declarative and I'm not sure
> > >> having it influenced by a dynamic var would match user expected
> > >> behavior.
> > >
> > > If you prefer, we could add a new :foo keyword to mean this.  But
> > > unconditionally changing what :pin means in these cases is out of the
> > > question.
> >
> > We wouldn't change what :pin means directly, but just have
> > package-install respect `package-pinned-packages'.
>
> How is that different?  It's an incompatible change of behavior, and I
> cannot agree to that, sorry.
>


-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 2814 bytes --]

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> Date: Mon, 10 Jun 2024 18:51:08 +0200
> Cc: Philip Kaludercic <philipk@posteo.net>, acorallo@gnu.org, 71356@debbugs.gnu.org
> 
> We don't want to change what :pin means, IMHO we need it to obey `package-install-upgrade-built-in`.
> That wouldn't be incompatible, right?

Right.  But AFAIU, that was not what Philip was proposing.  If I
misunderstood, my apologies.

> So the sketch that actually loads org from elpa would be
> 
> (let ((package-install-upgrade-built-in t))
>   (use-package org
>     :ensure t :pin gnu))

No, you are supposed to customize that option if you want built-in
packages to be upgraded as the rest of them.  You aren't supposed to
let-bind user options.

bug#71356: use-package doesn't load org from elpa

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
>> Date: Mon, 10 Jun 2024 18:51:08 +0200
>> Cc: Philip Kaludercic <philipk@posteo.net>, acorallo@gnu.org, 71356@debbugs.gnu.org
>> 
>> We don't want to change what :pin means, IMHO we need it to obey `package-install-upgrade-built-in`.
>> That wouldn't be incompatible, right?
>
> Right.  But AFAIU, that was not what Philip was proposing.  If I
> misunderstood, my apologies.

No it wasn't.  I was proposing to add `package-pinned-packages' as an
alternative to `package-install-upgrade-built-in' to upgrade specific
packages.

>> So the sketch that actually loads org from elpa would be
>> 
>> (let ((package-install-upgrade-built-in t))
>>   (use-package org
>>     :ensure t :pin gnu))
>
> No, you are supposed to customize that option if you want built-in
> packages to be upgraded as the rest of them.  You aren't supposed to
> let-bind user options.

-- 
	Philip Kaludercic on peregrine

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 1542 bytes --]

On Mon, 10 Jun 2024 at 19:47, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> > Date: Mon, 10 Jun 2024 18:51:08 +0200
> > Cc: Philip Kaludercic <philipk@posteo.net>, acorallo@gnu.org,
> 71356@debbugs.gnu.org
> >
> > We don't want to change what :pin means, IMHO we need it to obey
> `package-install-upgrade-built-in`.
> > That wouldn't be incompatible, right?
>
> Right.  But AFAIU, that was not what Philip was proposing.  If I
> misunderstood, my apologies.
>

So, in this, we are in "violent agreement" :-)

>
> > So the sketch that actually loads org from elpa would be
> >
> > (let ((package-install-upgrade-built-in t))
> >   (use-package org
> >     :ensure t :pin gnu))
>
> No, you are supposed to customize that option if you want built-in
> packages to be upgraded as the rest of them.  You aren't supposed to
> let-bind user options.
>

Ahh, OK. I was just trying to sketch a test case. So the thing would be:

(custom-set-variables '(package-install-upgrade-built-in t))
(use-package org
  :ensure t :pin gnu) ;; to upgrade from elpa
  :init (message "org-version: %s" org-version))
(use-package eglot
 :ensure t :: just to configure and initialise the built-in package
 :init (message "init eglot"))

Is that what you mean?

-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 2741 bytes --]

bug#71356: use-package doesn't load org from elpa

[Eli Zaretskii]

> From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> Date: Tue, 11 Jun 2024 07:27:39 +0200
> Cc: philipk@posteo.net, acorallo@gnu.org, 71356@debbugs.gnu.org
> 
>  > So the sketch that actually loads org from elpa would be
>  > 
>  > (let ((package-install-upgrade-built-in t))
>  >   (use-package org
>  >     :ensure t :pin gnu))
> 
>  No, you are supposed to customize that option if you want built-in
>  packages to be upgraded as the rest of them.  You aren't supposed to
>  let-bind user options.
> 
>  
> Ahh, OK. I was just trying to sketch a test case. So the thing would be:
> 
> (custom-set-variables '(package-install-upgrade-built-in t))
> (use-package org
>   :ensure t :pin gnu) ;; to upgrade from elpa
>   :init (message "org-version: %s" org-version))
> (use-package eglot
>  :ensure t :: just to configure and initialise the built-in package
>  :init (message "init eglot"))
> 
> Is that what you mean?

Yes.

bug#71356: use-package doesn't load org from elpa

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]

Perfect :-) We have (hopefully) the test case, Best, /PA

On Tue, 11 Jun 2024 at 09:29, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Pedro Andres Aranda Gutierrez <paaguti@gmail.com>
> > Date: Tue, 11 Jun 2024 07:27:39 +0200
> > Cc: philipk@posteo.net, acorallo@gnu.org, 71356@debbugs.gnu.org
> >
> >  > So the sketch that actually loads org from elpa would be
> >  >
> >  > (let ((package-install-upgrade-built-in t))
> >  >   (use-package org
> >  >     :ensure t :pin gnu))
> >
> >  No, you are supposed to customize that option if you want built-in
> >  packages to be upgraded as the rest of them.  You aren't supposed to
> >  let-bind user options.
> >
> >
> > Ahh, OK. I was just trying to sketch a test case. So the thing would be:
> >
> > (custom-set-variables '(package-install-upgrade-built-in t))
> > (use-package org
> >   :ensure t :pin gnu) ;; to upgrade from elpa
> >   :init (message "org-version: %s" org-version))
> > (use-package eglot
> >  :ensure t :: just to configure and initialise the built-in package
> >  :init (message "init eglot"))
> >
> > Is that what you mean?
>
> Yes.
>


-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 2343 bytes --]

bug#71356: Follow-up on bug#71356

[Pedro A. Aranda]

HI,

I just wanted to know what the situation with this is.

My use-case is I'm using org-mode on different OSes with Emacs packaged 
versions.

To have the same environment, I currently upgrade it manually with M-x 
p-l-p to the current elpa version. Would be nice to be able to automate 
this in my .emacs.d/init.el

BR,/PA