Changes to make in elpa-packages file for nongnu elpa

Changes to make in elpa-packages file for nongnu elpa

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 1457 bytes --]


I see that Helm installation from nongnu elpa doesn't include
emacs-helm.sh script, would it be possible to modify this?

        diff --git a/elpa-packages b/elpa-packages
        index 77d7a5409e..0923724b62 100644
        --- a/elpa-packages
        +++ b/elpa-packages
        @@ -312,11 +312,11 @@
           :ignored-files "COPYING")
         
          (helm			:url "https://github.com/emacs-helm/helm"
        -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-core.el" "helm.el"
        +  :ignored-files ("images" "Makefile" "helm-core.el" "helm.el"
         		  "helm-lib.el" "helm-source.el" "helm-multi-match.el"))
         
          (helm-core		:url "https://github.com/emacs-helm/helm"
        -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
        +  :ignored-files ("images" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
         		  "helm-bookmark.el" "helm-buffers.el" "helm-color.el" "helm-comint.el"
         		  "helm-command.el" "helm-config.el" "helm-dabbrev.el" "helm-easymenu.el"
         		  "helm-elisp.el" "helm-elisp-package.el" "helm-epa.el" "helm-eshell.el"

Also I asked here about a new dependency for Helm called wfnames and had
no answer (2 times), so I ask again differently now:
When a package is available in Melpa do we have to add it in nongnu as
well to satisfy dependencies for a package already in nongnu?

Thanks.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> I see that Helm installation from nongnu elpa doesn't include
> emacs-helm.sh script, would it be possible to modify this?
>
>         diff --git a/elpa-packages b/elpa-packages
>         index 77d7a5409e..0923724b62 100644
>         --- a/elpa-packages
>         +++ b/elpa-packages
>         @@ -312,11 +312,11 @@
>            :ignored-files "COPYING")
>          
>           (helm			:url "https://github.com/emacs-helm/helm"
>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-core.el" "helm.el"
>         +  :ignored-files ("images" "Makefile" "helm-core.el" "helm.el"
>          		  "helm-lib.el" "helm-source.el" "helm-multi-match.el"))
>          
>           (helm-core		:url "https://github.com/emacs-helm/helm"
>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>         +  :ignored-files ("images" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>          		  "helm-bookmark.el" "helm-buffers.el" "helm-color.el" "helm-comint.el"
>          		  "helm-command.el" "helm-config.el" "helm-dabbrev.el" "helm-easymenu.el"
>          		  "helm-elisp.el" "helm-elisp-package.el" "helm-epa.el" "helm-eshell.el"

There shouldn't be any issue in applying this patch, but could you
explain the propose of this script?  I see something involving
straight.el, which is a package.el-incompatible package manager, at
which point I am confused why this should be distributed along with a
"package.el"-package.

> Also I asked here about a new dependency for Helm called wfnames and had
> no answer (2 times), 

Sorry about that, you can always feel free to CC me if you have specific
questions that might get lost on the mailing list.  I suppose Stefan
Monnier wouldn't object to be mentioned either.

>                      so I ask again differently now:
>
> When a package is available in Melpa do we have to add it in nongnu as
> well to satisfy dependencies for a package already in nongnu?

Yes, if the dependency is missing someone only using GNU ELPA and NonGNU
ELPA wouldn't be able to install the package due to a missing dependency
and that is obviously not intended.

> Thanks.

Re: Changes to make in elpa-packages file for nongnu elpa

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 3241 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> I see that Helm installation from nongnu elpa doesn't include
>> emacs-helm.sh script, would it be possible to modify this?
>>
>>         diff --git a/elpa-packages b/elpa-packages
>>         index 77d7a5409e..0923724b62 100644
>>         --- a/elpa-packages
>>         +++ b/elpa-packages
>>         @@ -312,11 +312,11 @@
>>            :ignored-files "COPYING")
>>          
>>           (helm			:url "https://github.com/emacs-helm/helm"
>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-core.el" "helm.el"
>>         +  :ignored-files ("images" "Makefile" "helm-core.el" "helm.el"
>>          		  "helm-lib.el" "helm-source.el" "helm-multi-match.el"))
>>          
>>           (helm-core		:url "https://github.com/emacs-helm/helm"
>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>         +  :ignored-files ("images" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>          		  "helm-bookmark.el" "helm-buffers.el" "helm-color.el" "helm-comint.el"
>>          		  "helm-command.el" "helm-config.el" "helm-dabbrev.el" "helm-easymenu.el"
>>          		  "helm-elisp.el" "helm-elisp-package.el" "helm-epa.el" "helm-eshell.el"
>
> There shouldn't be any issue in applying this patch, but could you
> explain the propose of this script?

It is used specially for reproducing bugs in a clean environment, see it
as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
with only Helm loaded, this ensure the bug if one comes from Helm and
not another package. This is important especially nowaday people are
using "Emacs distribution" with the world list of packages installed.
Apart that the script is useful to quickly launch Emacs with helm, one
can use it from the Helm directory or symlinked to e.g. ~/bin. 

> I see something involving straight.el, which is a
> package.el-incompatible package manager, at which point I am confused
> why this should be distributed along with a "package.el"-package.

The script doesn't need straight, but it ensures the script is usable for
somebody using straight (to find package directory etc...).

>> Also I asked here about a new dependency for Helm called wfnames and had
>> no answer (2 times), 
>
> Sorry about that, you can always feel free to CC me if you have specific
> questions that might get lost on the mailing list.  I suppose Stefan
> Monnier wouldn't object to be mentioned either.

Ok thanks, I will next time now I know you and Stefan are in charge of
this.

>>                      so I ask again differently now:
>>
>> When a package is available in Melpa do we have to add it in nongnu as
>> well to satisfy dependencies for a package already in nongnu?
>
> Yes, if the dependency is missing someone only using GNU ELPA and NonGNU
> ELPA wouldn't be able to install the package due to a missing dependency
> and that is obviously not intended.

Ok, it's what I thought, so yes it would be nice to install wfnames
package in Nongnu.
Here is the link: https://github.com/thierryvolpiatto/wfnames

Thanks.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 3760 bytes --]

Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Thierry Volpiatto <thievol@posteo.net> writes:
>>
>>> I see that Helm installation from nongnu elpa doesn't include
>>> emacs-helm.sh script, would it be possible to modify this?
>>>
>>>         diff --git a/elpa-packages b/elpa-packages
>>>         index 77d7a5409e..0923724b62 100644
>>>         --- a/elpa-packages
>>>         +++ b/elpa-packages
>>>         @@ -312,11 +312,11 @@
>>>            :ignored-files "COPYING")
>>>          
>>>           (helm			:url "https://github.com/emacs-helm/helm"
>>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-core.el" "helm.el"
>>>         +  :ignored-files ("images" "Makefile" "helm-core.el" "helm.el"
>>>          		  "helm-lib.el" "helm-source.el" "helm-multi-match.el"))
>>>          
>>>           (helm-core		:url "https://github.com/emacs-helm/helm"
>>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>>         +  :ignored-files ("images" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>>          		  "helm-bookmark.el" "helm-buffers.el" "helm-color.el" "helm-comint.el"
>>>          		  "helm-command.el" "helm-config.el" "helm-dabbrev.el" "helm-easymenu.el"
>>>          		  "helm-elisp.el" "helm-elisp-package.el" "helm-epa.el" "helm-eshell.el"
>>
>> There shouldn't be any issue in applying this patch, but could you
>> explain the propose of this script?
>
> It is used specially for reproducing bugs in a clean environment, see it
> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
> with only Helm loaded, this ensure the bug if one comes from Helm and
> not another package. This is important especially nowaday people are
> using "Emacs distribution" with the world list of packages installed.
> Apart that the script is useful to quickly launch Emacs with helm, one
> can use it from the Helm directory or symlinked to e.g. ~/bin. 

I see.  In that case is there any reason you implement this as a shell
script?  (It might be interesting to provide something like this for
package.el, to test packages in a generic way.)

>> I see something involving straight.el, which is a
>> package.el-incompatible package manager, at which point I am confused
>> why this should be distributed along with a "package.el"-package.
>
> The script doesn't need straight, but it ensures the script is usable for
> somebody using straight (to find package directory etc...).

Ah OK my bad.

>>> Also I asked here about a new dependency for Helm called wfnames and had
>>> no answer (2 times), 
>>
>> Sorry about that, you can always feel free to CC me if you have specific
>> questions that might get lost on the mailing list.  I suppose Stefan
>> Monnier wouldn't object to be mentioned either.
>
> Ok thanks, I will next time now I know you and Stefan are in charge of
> this.

I am sort of an unofficial co-maintainer, without access to any
server-side infrastructure, but other than that I'm glad to help within
my possibilities.

>>>                      so I ask again differently now:
>>>
>>> When a package is available in Melpa do we have to add it in nongnu as
>>> well to satisfy dependencies for a package already in nongnu?
>>
>> Yes, if the dependency is missing someone only using GNU ELPA and NonGNU
>> ELPA wouldn't be able to install the package due to a missing dependency
>> and that is obviously not intended.
>
> Ok, it's what I thought, so yes it would be nice to install wfnames
> package in Nongnu.
> Here is the link: https://github.com/thierryvolpiatto/wfnames

Sure, but could you first take a look at these comments, suggestions and
questions I have made while skimming over the source?


[-- Attachment #2: Type: text/plain, Size: 6559 bytes --]

diff --git a/wfnames.el b/wfnames.el
index e5a83b5..a61a18d 100644
--- a/wfnames.el
+++ b/wfnames.el
@@ -34,20 +34,20 @@
 
 ;; Usage:
 ;; Once in the Wfnames buffer, edit your filenames and hit C-c C-c to
-;; save your changes. You have completion on filenames and directories
+;; save your changes.  You have completion on filenames and directories
 ;; with TAB but if you are using Iedit package and it is in action use =M-TAB=.
 
 ;;; Code:
 
-(require 'cl-lib)
+(eval-when-compile (require 'cl-lib))	;you are only using macros, so this should be OK
 
 ;; Internal.
-(defvar wfnames-buffer "*Wfnames*")
-(defvar wfnames--modified nil)
+(defvar wfnames-buffer "*Wfnames*")	;perhaps `defconst'
+(defvar-local wfnames--modified nil)
 \f
 (defgroup wfnames nil
   "A mode to edit filenames."
-  :group 'wfnames)
+  :group 'wfnames)			;watch out, this is a recursive group
 
 (defcustom wfnames-create-parent-directories t
   "Create parent directories when non nil."
@@ -57,16 +57,21 @@
   "Ask confirmation when overwriting."
   :type 'boolean)
 
-(defvar wfnames-after-commit-hook nil)
+(defcustom wfnames-after-commit-hook nil
+  "Hook to run after `wfnames-commit-buffer'." ;rephrase this
+  :type 'hook)
 
 (defcustom wfnames-after-commit-function #'kill-buffer
   "A function to call on `wfnames-buffer' when done."
   :type 'function)
 
 (defcustom wfnames-make-backup nil
-  "Backup files before overwriting when non nil."
+  "Non-nil means files are backed up before overwriting."
   :type 'boolean)
 \f
+;; instead of defining new faces and colours, do you think it would be
+;; possible to inherit from existing faces?
+
 (defface wfnames-modified
     '((t :background "LightBlue" :foreground "black"))
   "Face used when filename is modified.")
@@ -104,27 +109,27 @@
   "Provide filename completion in wfnames buffer."
   (let ((beg (line-beginning-position))
         (end (point)))
+    ;; Does it make sense to extend beyond END to allow completing
+    ;; file names mid-string?
     (list beg end #'completion-file-name-table
           :exit-function (lambda (str _status)
                              (when (and (stringp str)
                                         (eq (char-after) ?/))
                                (delete-char -1))))))
 
-(define-derived-mode wfnames-mode
-    text-mode "wfnames"
+(define-derived-mode wfnames-mode text-mode "wfnames"
     "Major mode to edit filenames.
 
 Special commands:
 \\{wfnames-mode-map}"
   (add-hook 'after-change-functions #'wfnames-after-change-hook nil t)
-  (make-local-variable 'wfnames--modified)
-  (set (make-local-variable 'completion-at-point-functions) #'wfnames-capf)
-  (set (make-local-variable 'revert-buffer-function) #'wfnames-revert-changes))
+  (setq-local completion-at-point-functions #'wfnames-capf)
+  (setq-local revert-buffer-function #'wfnames-revert-changes))
 
 (defun wfnames-abort ()
   "Quit and kill wfnames buffer."
   (interactive)
-  (quit-window t))
+  (quit-window t))			;isn't this `kill-buffer-and-window'
 
 (defun wfnames-after-change-hook (beg end _len)
   "Put overlay on current line when modified.
@@ -137,9 +142,10 @@ Args BEG and END delimit changes on line."
                (eol (line-end-position))
                (old (get-text-property bol 'old-name))
                (new (buffer-substring-no-properties bol eol))
-               ov face)
-          (setq face (if (file-exists-p new)
-                         'wfnames-modified-exists 'wfnames-modified))
+               (face (if (file-exists-p new)
+                         'wfnames-modified-exists
+		       'wfnames-modified))
+	       ov)
           (setq-local wfnames--modified
                       (cons old (delete old wfnames--modified)))
           (cl-loop for o in (overlays-in bol eol)
@@ -184,10 +190,10 @@ When APPEND is specified, append FILES to existing `wfnames-buffer'."
                                          "* "
                                          'face 'wfnames-prefix))
                           "\n"))
-      (when append (delete-duplicate-lines (point-min) (point-max))))
+      (when append (delete-duplicate-lines (point-min) (point-max)))) ;this requires Emacs 24.4
     (unless append
       ;; Go to beginning of basename on first line.
-      (while (re-search-forward "/" (line-end-position) t))
+      (re-search-forward "\\(?:/[^/]*\\)*/" (line-end-position) t)
       (wfnames-mode)
       (funcall display-fn wfnames-buffer))))
 \f
@@ -202,7 +208,7 @@ When APPEND is specified, append FILES to existing `wfnames-buffer'."
   "Backup FILE."
   (when wfnames-make-backup
     (with-current-buffer (find-file-noselect file)
-      (let ((backup-by-copying t))
+      (let ((backup-by-copying t))	;why is this bound?  isn't this a user preference?
         (backup-buffer))
       (kill-buffer))))
 
@@ -252,7 +258,7 @@ When APPEND is specified, append FILES to existing `wfnames-buffer'."
                                    (let ((basedir (file-name-directory
                                                    (directory-file-name new))))
                                      (unless (file-directory-p basedir)
-                                       (mkdir basedir 'parents))))
+                                       (make-directory basedir 'parents))))
                                  (if (and ow (wfnames-ask-for-overwrite new))
                                      ;; Direct overwrite i.e. first loop.
                                      (progn
@@ -300,21 +306,22 @@ With a numeric prefix ARG, revert the ARG next lines."
     (wfnames-revert-current-line-1)
     (when (eobp) (forward-line -1))
     (goto-char (line-beginning-position))
-    (while (re-search-forward "/" (line-end-position) t))))
+    (re-search-forward "\\(?:/[^/]*\\)*/" (line-end-position) t)))
 
 (defun wfnames-revert-changes (_ignore-auto _no-confirm)
   "Revert wfnames buffer to its initial state.
 
 This is used as `revert-buffer-function' for `wfnames-mode'."
   (with-current-buffer wfnames-buffer
-    (cl-loop for o in (overlays-in (point-min) (point-max))
-             when (overlay-get o 'hff-changed)
-             do (delete-overlay o))
+    (dolist (o (overlays-in (point-min) (point-max)))
+      (when (overlay-get o 'hff-changed)
+	(delete-overlay o)))
     (goto-char (point-min))
     (save-excursion
       (while (not (eobp))
         (wfnames-revert-current-line-1)))
-    (while (re-search-forward "/" (line-end-position) t))))
+    (re-search-forward "\\(?:/[^/]*\\)*/" (line-end-position) t)))
+
 
 (provide 'wfnames)
 

[-- Attachment #3: Type: text/plain, Size: 11 bytes --]


> Thanks.

Re: Changes to make in elpa-packages file for nongnu elpa

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 10671 bytes --]


Hello Philip,

Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Thierry Volpiatto <thievol@posteo.net> writes:
>>>
>>>> I see that Helm installation from nongnu elpa doesn't include
>>>> emacs-helm.sh script, would it be possible to modify this?
>>>>
>>>>         diff --git a/elpa-packages b/elpa-packages
>>>>         index 77d7a5409e..0923724b62 100644
>>>>         --- a/elpa-packages
>>>>         +++ b/elpa-packages
>>>>         @@ -312,11 +312,11 @@
>>>>            :ignored-files "COPYING")
>>>>          
>>>>           (helm			:url "https://github.com/emacs-helm/helm"
>>>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-core.el" "helm.el"
>>>>         +  :ignored-files ("images" "Makefile" "helm-core.el" "helm.el"
>>>>          		  "helm-lib.el" "helm-source.el" "helm-multi-match.el"))
>>>>          
>>>>           (helm-core		:url "https://github.com/emacs-helm/helm"
>>>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>>>         +  :ignored-files ("images" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>>>          		  "helm-bookmark.el" "helm-buffers.el" "helm-color.el" "helm-comint.el"
>>>>          		  "helm-command.el" "helm-config.el" "helm-dabbrev.el" "helm-easymenu.el"
>>>>          		  "helm-elisp.el" "helm-elisp-package.el" "helm-epa.el" "helm-eshell.el"
>>>
>>> There shouldn't be any issue in applying this patch, but could you
>>> explain the propose of this script?
>>
>> It is used specially for reproducing bugs in a clean environment, see it
>> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
>> with only Helm loaded, this ensure the bug if one comes from Helm and
>> not another package. This is important especially nowaday people are
>> using "Emacs distribution" with the world list of packages installed.
>> Apart that the script is useful to quickly launch Emacs with helm, one
>> can use it from the Helm directory or symlinked to e.g. ~/bin. 
>
> I see.  In that case is there any reason you implement this as a shell
> script?

Well when I wrote the script, packages where not existing and from
outside emacs it is actually the only way to run a package isolated.

> (It might be interesting to provide something like this for
> package.el, to test packages in a generic way.)

Yes, this would be interesting, it would be something like this:

Emacs -Q
M-x <A command that run a package alone, isolated from other
packages nuisances>

> I am sort of an unofficial co-maintainer, without access to any
> server-side infrastructure, but other than that I'm glad to help within
> my possibilities.

Ok thanks.

>> Ok, it's what I thought, so yes it would be nice to install wfnames
>> package in Nongnu.
>> Here is the link: https://github.com/thierryvolpiatto/wfnames
>
> Sure, but could you first take a look at these comments, suggestions and
> questions I have made while skimming over the source?
>
> diff --git a/wfnames.el b/wfnames.el
> index e5a83b5..a61a18d 100644
> --- a/wfnames.el
> +++ b/wfnames.el
> @@ -34,20 +34,20 @@
>  
>  ;; Usage:
>  ;; Once in the Wfnames buffer, edit your filenames and hit C-c C-c to
> -;; save your changes. You have completion on filenames and directories
> +;; save your changes.  You have completion on filenames and directories
>  ;; with TAB but if you are using Iedit package and it is in action use =M-TAB=.
>  
>  ;;; Code:
>  
> -(require 'cl-lib)
> +(eval-when-compile (require 'cl-lib))	;you are only using macros, so this should be OK

Ok done.
  
>  ;; Internal.
> -(defvar wfnames-buffer "*Wfnames*")
> -(defvar wfnames--modified nil)
> +(defvar wfnames-buffer "*Wfnames*")	;perhaps `defconst'
> +(defvar-local wfnames--modified nil)

Ok done.

>  \f
>  (defgroup wfnames nil
>    "A mode to edit filenames."
> -  :group 'wfnames)
> +  :group 'wfnames)			;watch out, this is a recursive group

Good catch, done.

>  (defcustom wfnames-create-parent-directories t
>    "Create parent directories when non nil."
> @@ -57,16 +57,21 @@
>    "Ask confirmation when overwriting."
>    :type 'boolean)
>  
> -(defvar wfnames-after-commit-hook nil)
> +(defcustom wfnames-after-commit-hook nil
> +  "Hook to run after `wfnames-commit-buffer'." ;rephrase this

Not sure how to rephrase this:

"Hook that run after `wfnames-commit-buffer'."?

> 
> + :type 'hook)

Othewise defcustom make sense, done.

>  (defcustom wfnames-after-commit-function #'kill-buffer
>    "A function to call on `wfnames-buffer' when done."
>    :type 'function)
>  
>  (defcustom wfnames-make-backup nil
> -  "Backup files before overwriting when non nil."
> +  "Non-nil means files are backed up before overwriting."

Ok done.

> 
>    :type 'boolean)
>  \f
> +;; instead of defining new faces and colours, do you think it would be
> +;; possible to inherit from existing faces?

Apart requiring a package just for the faces no, font-lock family
doesn't provide :background faces.

>  (defface wfnames-modified
>      '((t :background "LightBlue" :foreground "black"))
>    "Face used when filename is modified.")
> @@ -104,27 +109,27 @@
>    "Provide filename completion in wfnames buffer."
>    (let ((beg (line-beginning-position))
>          (end (point)))
> +    ;; Does it make sense to extend beyond END to allow completing
> +    ;; file names mid-string?

This one for now I don't know, I transformed your comment with a FIXME.

> 
>      (list beg end #'completion-file-name-table :exit-function (lambda
> (str _status) (when (and (stringp str) (eq (char-after) ?/))
> (delete-char -1))))))
>  
> -(define-derived-mode wfnames-mode
> -    text-mode "wfnames"
> +(define-derived-mode wfnames-mode text-mode "wfnames"
>      "Major mode to edit filenames.

Ok.

>  Special commands:
>  \\{wfnames-mode-map}"
>    (add-hook 'after-change-functions #'wfnames-after-change-hook nil t)
> -  (make-local-variable 'wfnames--modified)
> -  (set (make-local-variable 'completion-at-point-functions) #'wfnames-capf)
> -  (set (make-local-variable 'revert-buffer-function) #'wfnames-revert-changes))
> +  (setq-local completion-at-point-functions #'wfnames-capf)
> +  (setq-local revert-buffer-function #'wfnames-revert-changes))

Yes done.
  
>  (defun wfnames-abort ()
>    "Quit and kill wfnames buffer."
>    (interactive)
> -  (quit-window t))
> +  (quit-window t))			;isn't this `kill-buffer-and-window'

Yes, but quit-window is fine as well (I am used to it).
  
>  (defun wfnames-after-change-hook (beg end _len)
>    "Put overlay on current line when modified.
> @@ -137,9 +142,10 @@ Args BEG and END delimit changes on line."
>                 (eol (line-end-position))
>                 (old (get-text-property bol 'old-name))
>                 (new (buffer-substring-no-properties bol eol))
> -               ov face)
> -          (setq face (if (file-exists-p new)
> -                         'wfnames-modified-exists 'wfnames-modified))
> +               (face (if (file-exists-p new)
> +                         'wfnames-modified-exists
> +		       'wfnames-modified))
> +	       ov)

Ok.

>            (setq-local wfnames--modified
>                        (cons old (delete old wfnames--modified)))
>            (cl-loop for o in (overlays-in bol eol)
> @@ -184,10 +190,10 @@ When APPEND is specified, append FILES to existing `wfnames-buffer'."
>                                           "* "
>                                           'face 'wfnames-prefix))
>                            "\n"))
> -      (when append (delete-duplicate-lines (point-min) (point-max))))
> +      (when append (delete-duplicate-lines (point-min) (point-max)))) ;this requires Emacs 24.4
>      (unless append
>        ;; Go to beginning of basename on first line.
> -      (while (re-search-forward "/" (line-end-position) t))
> +      (re-search-forward "\\(?:/[^/]*\\)*/" (line-end-position) t)

Ok.

>        (wfnames-mode)
>        (funcall display-fn wfnames-buffer))))
>  \f
> @@ -202,7 +208,7 @@ When APPEND is specified, append FILES to existing `wfnames-buffer'."
>    "Backup FILE."
>    (when wfnames-make-backup
>      (with-current-buffer (find-file-noselect file)
> -      (let ((backup-by-copying t))
> +      (let ((backup-by-copying t))	;why is this bound?  isn't this
>        a user preference?

Yes removed.

> 
>          (backup-buffer)) (kill-buffer))))
>  
> @@ -252,7 +258,7 @@ When APPEND is specified, append FILES to existing `wfnames-buffer'."
>                                     (let ((basedir (file-name-directory
>                                                     (directory-file-name new))))
>                                       (unless (file-directory-p basedir)
> -                                       (mkdir basedir 'parents))))
> +                                       (make-directory basedir
> 'parents))))

Ok, make sense.

>                                   (if (and ow (wfnames-ask-for-overwrite new))
>                                       ;; Direct overwrite i.e. first loop.
>                                       (progn
> @@ -300,21 +306,22 @@ With a numeric prefix ARG, revert the ARG next lines."
>      (wfnames-revert-current-line-1)
>      (when (eobp) (forward-line -1))
>      (goto-char (line-beginning-position))
> -    (while (re-search-forward "/" (line-end-position) t))))
> +    (re-search-forward "\\(?:/[^/]*\\)*/" (line-end-position) t)))

Ok, same as above.
  
>  (defun wfnames-revert-changes (_ignore-auto _no-confirm)
>    "Revert wfnames buffer to its initial state.
>  
>  This is used as `revert-buffer-function' for `wfnames-mode'."
>    (with-current-buffer wfnames-buffer
> -    (cl-loop for o in (overlays-in (point-min) (point-max))
> -             when (overlay-get o 'hff-changed)
> -             do (delete-overlay o))
> +    (dolist (o (overlays-in (point-min) (point-max)))
> +      (when (overlay-get o 'hff-changed)
> +	(delete-overlay o)))

If you want ;-) done.

>      (goto-char (point-min))
>      (save-excursion
>        (while (not (eobp))
>          (wfnames-revert-current-line-1)))
> -    (while (re-search-forward "/" (line-end-position) t))))
> +    (re-search-forward "\\(?:/[^/]*\\)*/" (line-end-position) t)))
> +

Ok same as above.
  
>  (provide 'wfnames)
>  
>
>
>> Thanks.

Thanks.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> Hello Philip,
>
> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Thierry Volpiatto <thievol@posteo.net> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> Thierry Volpiatto <thievol@posteo.net> writes:
>>>>
>>>>> I see that Helm installation from nongnu elpa doesn't include
>>>>> emacs-helm.sh script, would it be possible to modify this?
>>>>>
>>>>>         diff --git a/elpa-packages b/elpa-packages
>>>>>         index 77d7a5409e..0923724b62 100644
>>>>>         --- a/elpa-packages
>>>>>         +++ b/elpa-packages
>>>>>         @@ -312,11 +312,11 @@
>>>>>            :ignored-files "COPYING")
>>>>>          
>>>>>           (helm			:url "https://github.com/emacs-helm/helm"
>>>>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-core.el" "helm.el"
>>>>>         +  :ignored-files ("images" "Makefile" "helm-core.el" "helm.el"
>>>>>          		  "helm-lib.el" "helm-source.el" "helm-multi-match.el"))
>>>>>          
>>>>>           (helm-core		:url "https://github.com/emacs-helm/helm"
>>>>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>>>>         +  :ignored-files ("images" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>>>>          		  "helm-bookmark.el" "helm-buffers.el" "helm-color.el" "helm-comint.el"
>>>>>          		  "helm-command.el" "helm-config.el" "helm-dabbrev.el" "helm-easymenu.el"
>>>>>          		  "helm-elisp.el" "helm-elisp-package.el" "helm-epa.el" "helm-eshell.el"
>>>>
>>>> There shouldn't be any issue in applying this patch, but could you
>>>> explain the propose of this script?
>>>
>>> It is used specially for reproducing bugs in a clean environment, see it
>>> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
>>> with only Helm loaded, this ensure the bug if one comes from Helm and
>>> not another package. This is important especially nowaday people are
>>> using "Emacs distribution" with the world list of packages installed.
>>> Apart that the script is useful to quickly launch Emacs with helm, one
>>> can use it from the Helm directory or symlinked to e.g. ~/bin. 
>>
>> I see.  In that case is there any reason you implement this as a shell
>> script?
>
> Well when I wrote the script, packages where not existing and from
> outside emacs it is actually the only way to run a package isolated.
>
>> (It might be interesting to provide something like this for
>> package.el, to test packages in a generic way.)
>
> Yes, this would be interesting, it would be something like this:
>
> Emacs -Q
> M-x <A command that run a package alone, isolated from other
> packages nuisances>

I was actually thinking of a command like

M-x package-isolate RET foo,bar,baz RET

and a new instance of Emacs using -Q is spun up, with all the packages
you have listed loaded, and nothing else... Sounds like a fun little
weekend project ;^)

>> I am sort of an unofficial co-maintainer, without access to any
>> server-side infrastructure, but other than that I'm glad to help within
>> my possibilities.
>
> Ok thanks.
>
>>> Ok, it's what I thought, so yes it would be nice to install wfnames
>>> package in Nongnu.
>>> Here is the link: https://github.com/thierryvolpiatto/wfnames
>>
>> Sure, but could you first take a look at these comments, suggestions and
>> questions I have made while skimming over the source?
>>
>> diff --git a/wfnames.el b/wfnames.el
>> index e5a83b5..a61a18d 100644
>> --- a/wfnames.el
>> +++ b/wfnames.el
>> @@ -34,20 +34,20 @@
>>  
>>  ;; Usage:
>>  ;; Once in the Wfnames buffer, edit your filenames and hit C-c C-c to
>> -;; save your changes. You have completion on filenames and directories
>> +;; save your changes.  You have completion on filenames and directories
>>  ;; with TAB but if you are using Iedit package and it is in action use =M-TAB=.
>>  
>>  ;;; Code:
>>  
>> -(require 'cl-lib)
>> +(eval-when-compile (require 'cl-lib))	;you are only using macros, so this should be OK
>
> Ok done.
>   
>>  ;; Internal.
>> -(defvar wfnames-buffer "*Wfnames*")
>> -(defvar wfnames--modified nil)
>> +(defvar wfnames-buffer "*Wfnames*")	;perhaps `defconst'
>> +(defvar-local wfnames--modified nil)
>
> Ok done.
>
>>  \f
>>  (defgroup wfnames nil
>>    "A mode to edit filenames."
>> -  :group 'wfnames)
>> +  :group 'wfnames)			;watch out, this is a recursive group
>
> Good catch, done.
>
>>  (defcustom wfnames-create-parent-directories t
>>    "Create parent directories when non nil."
>> @@ -57,16 +57,21 @@
>>    "Ask confirmation when overwriting."
>>    :type 'boolean)
>>  
>> -(defvar wfnames-after-commit-hook nil)
>> +(defcustom wfnames-after-commit-hook nil
>> +  "Hook to run after `wfnames-commit-buffer'." ;rephrase this
>
> Not sure how to rephrase this:
>
> "Hook that run after `wfnames-commit-buffer'."?

If it is OK the way it is, then you can keep it, I just wasn't sure if
you would have a more descriptive name for it.

>> + :type 'hook)
>
> Othewise defcustom make sense, done.
>
>>  (defcustom wfnames-after-commit-function #'kill-buffer
>>    "A function to call on `wfnames-buffer' when done."
>>    :type 'function)
>>  
>>  (defcustom wfnames-make-backup nil
>> -  "Backup files before overwriting when non nil."
>> +  "Non-nil means files are backed up before overwriting."
>
> Ok done.
>
>> 
>>    :type 'boolean)
>>  \f
>> +;; instead of defining new faces and colours, do you think it would be
>> +;; possible to inherit from existing faces?
>
> Apart requiring a package just for the faces no, font-lock family
> doesn't provide :background faces.

I wasn't thinking just of font-lock, you could pick any face,
eg. wfnames-modified could inherit from diff-refine-changed.  That way
themes don't have to add extra support, and the package looks more
"native" without any additional effort.

>>  (defface wfnames-modified
>>      '((t :background "LightBlue" :foreground "black"))
>>    "Face used when filename is modified.")
>> @@ -104,27 +109,27 @@
>>    "Provide filename completion in wfnames buffer."
>>    (let ((beg (line-beginning-position))
>>          (end (point)))
>> +    ;; Does it make sense to extend beyond END to allow completing
>> +    ;; file names mid-string?
>
> This one for now I don't know, I transformed your comment with a FIXME.

That is fine, this is just a general thing I wonder about with CAP
functions.

>> 
>>      (list beg end #'completion-file-name-table :exit-function (lambda
>> (str _status) (when (and (stringp str) (eq (char-after) ?/))
>> (delete-char -1))))))
>>  
>> -(define-derived-mode wfnames-mode
>> -    text-mode "wfnames"
>> +(define-derived-mode wfnames-mode text-mode "wfnames"
>>      "Major mode to edit filenames.
>
> Ok.
>
>>  Special commands:
>>  \\{wfnames-mode-map}"
>>    (add-hook 'after-change-functions #'wfnames-after-change-hook nil t)
>> -  (make-local-variable 'wfnames--modified)
>> -  (set (make-local-variable 'completion-at-point-functions) #'wfnames-capf)
>> -  (set (make-local-variable 'revert-buffer-function) #'wfnames-revert-changes))
>> +  (setq-local completion-at-point-functions #'wfnames-capf)
>> +  (setq-local revert-buffer-function #'wfnames-revert-changes))
>
> Yes done.
>   
>>  (defun wfnames-abort ()
>>    "Quit and kill wfnames buffer."
>>    (interactive)
>> -  (quit-window t))
>> +  (quit-window t))			;isn't this `kill-buffer-and-window'
>
> Yes, but quit-window is fine as well (I am used to it).
>   
>>  (defun wfnames-after-change-hook (beg end _len)
>>    "Put overlay on current line when modified.
>> @@ -137,9 +142,10 @@ Args BEG and END delimit changes on line."
>>                 (eol (line-end-position))
>>                 (old (get-text-property bol 'old-name))
>>                 (new (buffer-substring-no-properties bol eol))
>> -               ov face)
>> -          (setq face (if (file-exists-p new)
>> -                         'wfnames-modified-exists 'wfnames-modified))
>> +               (face (if (file-exists-p new)
>> +                         'wfnames-modified-exists
>> +		       'wfnames-modified))
>> +	       ov)
>
> Ok.
>
>>            (setq-local wfnames--modified
>>                        (cons old (delete old wfnames--modified)))
>>            (cl-loop for o in (overlays-in bol eol)
>> @@ -184,10 +190,10 @@ When APPEND is specified, append FILES to existing `wfnames-buffer'."
>>                                           "* "
>>                                           'face 'wfnames-prefix))
>>                            "\n"))
>> -      (when append (delete-duplicate-lines (point-min) (point-max))))
>> +      (when append (delete-duplicate-lines (point-min) (point-max)))) ;this requires Emacs 24.4
>>      (unless append
>>        ;; Go to beginning of basename on first line.
>> -      (while (re-search-forward "/" (line-end-position) t))
>> +      (re-search-forward "\\(?:/[^/]*\\)*/" (line-end-position) t)
>
> Ok.
>
>>        (wfnames-mode)
>>        (funcall display-fn wfnames-buffer))))
>>  \f
>> @@ -202,7 +208,7 @@ When APPEND is specified, append FILES to existing `wfnames-buffer'."
>>    "Backup FILE."
>>    (when wfnames-make-backup
>>      (with-current-buffer (find-file-noselect file)
>> -      (let ((backup-by-copying t))
>> +      (let ((backup-by-copying t))	;why is this bound?  isn't this
>>        a user preference?
>
> Yes removed.
>
>> 
>>          (backup-buffer)) (kill-buffer))))
>>  
>> @@ -252,7 +258,7 @@ When APPEND is specified, append FILES to existing `wfnames-buffer'."
>>                                     (let ((basedir (file-name-directory
>>                                                     (directory-file-name new))))
>>                                       (unless (file-directory-p basedir)
>> -                                       (mkdir basedir 'parents))))
>> +                                       (make-directory basedir
>> 'parents))))
>
> Ok, make sense.
>
>>                                   (if (and ow (wfnames-ask-for-overwrite new))
>>                                       ;; Direct overwrite i.e. first loop.
>>                                       (progn
>> @@ -300,21 +306,22 @@ With a numeric prefix ARG, revert the ARG next lines."
>>      (wfnames-revert-current-line-1)
>>      (when (eobp) (forward-line -1))
>>      (goto-char (line-beginning-position))
>> -    (while (re-search-forward "/" (line-end-position) t))))
>> +    (re-search-forward "\\(?:/[^/]*\\)*/" (line-end-position) t)))
>
> Ok, same as above.
>   
>>  (defun wfnames-revert-changes (_ignore-auto _no-confirm)
>>    "Revert wfnames buffer to its initial state.
>>  
>>  This is used as `revert-buffer-function' for `wfnames-mode'."
>>    (with-current-buffer wfnames-buffer
>> -    (cl-loop for o in (overlays-in (point-min) (point-max))
>> -             when (overlay-get o 'hff-changed)
>> -             do (delete-overlay o))
>> +    (dolist (o (overlays-in (point-min) (point-max)))
>> +      (when (overlay-get o 'hff-changed)
>> +	(delete-overlay o)))
>
> If you want ;-) done.

I am totally indifferent, they expand to almost the same code anyway:

--8<---------------cut here---------------start------------->8---
(disassemble
 (byte-compile
  '(cl-loop for o in olist
	    when (foo)
	    do (bar))))

byte code:
  args: nil
0	varref	  olist
1	constant  nil
2:1	stack-ref 1
3	consp	  
4	goto-if-nil 3
7	discard	  
8	dup	  
9	car	  
10	constant  foo
11	call	  0
12	goto-if-nil 2
15	constant  bar
16	call	  0
17	discard	  
18:2	stack-ref 1
19	cdr	  
20	stack-set 2
22	goto	  1
25:3	constant  nil
26	return	  


(disassemble
 (byte-compile
  '(dolist (o olist)
     (when (foo)
       (bar)))))

byte code:
  args: nil
0	varref	  olist
1:1	dup	  
2	goto-if-nil-else-pop 3
5	dup	  
6	car	  
7	discard	  
8	constant  nil
9	constant  foo
10	call	  0
11	goto-if-nil 2
14	constant  bar
15	call	  0
16	discard	  
17:2	discard	  
18	cdr	  
19	goto	  1
22:3	return	  
--8<---------------cut here---------------end--------------->8---


>>      (goto-char (point-min))
>>      (save-excursion
>>        (while (not (eobp))
>>          (wfnames-revert-current-line-1)))
>> -    (while (re-search-forward "/" (line-end-position) t))))
>> +    (re-search-forward "\\(?:/[^/]*\\)*/" (line-end-position) t)))
>> +
>
> Ok same as above.
>   
>>  (provide 'wfnames)
>>  
>>
>>
>>> Thanks.
>
> Thanks.

Great, I'll add the package then.

Re: Changes to make in elpa-packages file for nongnu elpa

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 1092 bytes --]


Ok I commited your changes with some corrections from my previous post,
see below.

Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>    (let ((beg (line-beginning-position))
>          (end (point)))
> +    ;; Does it make sense to extend beyond END to allow completing
> +    ;; file names mid-string?
>      (list beg end #'completion-file-name-table
>            :exit-function (lambda (str _status)
>                               (when (and (stringp str)
>                                          (eq (char-after) ?/))
>                                 (delete-char -1))))))

Yes, we want to complete from bol to point:

     START and END delimit the text to complete (which should enclose
     point).  COLLECTION is a completion table for completing that text,

> -      (let ((backup-by-copying t))
> +      (let ((backup-by-copying t))	;why is this bound?

We want to backup by copying, not renaming, so ensure it is done by copying.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Changes to make in elpa-packages file for nongnu elpa

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 5243 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Hello Philip,
>>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Thierry Volpiatto <thievol@posteo.net> writes:
>>>
>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>
>>>>> Thierry Volpiatto <thievol@posteo.net> writes:
>>>>>
>>>>>> I see that Helm installation from nongnu elpa doesn't include
>>>>>> emacs-helm.sh script, would it be possible to modify this?
>>>>>>
>>>>>>         diff --git a/elpa-packages b/elpa-packages
>>>>>>         index 77d7a5409e..0923724b62 100644
>>>>>>         --- a/elpa-packages
>>>>>>         +++ b/elpa-packages
>>>>>>         @@ -312,11 +312,11 @@
>>>>>>            :ignored-files "COPYING")
>>>>>>          
>>>>>>           (helm			:url "https://github.com/emacs-helm/helm"
>>>>>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-core.el" "helm.el"
>>>>>>         +  :ignored-files ("images" "Makefile" "helm-core.el" "helm.el"
>>>>>>          		  "helm-lib.el" "helm-source.el" "helm-multi-match.el"))
>>>>>>          
>>>>>>           (helm-core		:url "https://github.com/emacs-helm/helm"
>>>>>>         -  :ignored-files ("images" "Cask" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>>>>>         +  :ignored-files ("images" "Makefile" "emacs-helm.sh" "helm-adaptive.el"
>>>>>>          		  "helm-bookmark.el" "helm-buffers.el" "helm-color.el" "helm-comint.el"
>>>>>>          		  "helm-command.el" "helm-config.el" "helm-dabbrev.el" "helm-easymenu.el"
>>>>>>          		  "helm-elisp.el" "helm-elisp-package.el" "helm-epa.el" "helm-eshell.el"
>>>>>
>>>>> There shouldn't be any issue in applying this patch, but could you
>>>>> explain the propose of this script?
>>>>
>>>> It is used specially for reproducing bugs in a clean environment, see it
>>>> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
>>>> with only Helm loaded, this ensure the bug if one comes from Helm and
>>>> not another package. This is important especially nowaday people are
>>>> using "Emacs distribution" with the world list of packages installed.
>>>> Apart that the script is useful to quickly launch Emacs with helm, one
>>>> can use it from the Helm directory or symlinked to e.g. ~/bin. 
>>>
>>> I see.  In that case is there any reason you implement this as a shell
>>> script?
>>
>> Well when I wrote the script, packages where not existing and from
>> outside emacs it is actually the only way to run a package isolated.
>>
>>> (It might be interesting to provide something like this for
>>> package.el, to test packages in a generic way.)
>>
>> Yes, this would be interesting, it would be something like this:
>>
>> Emacs -Q
>> M-x <A command that run a package alone, isolated from other
>> packages nuisances>
>
> I was actually thinking of a command like
>
> M-x package-isolate RET foo,bar,baz RET

Then you want completion on packages and return a list of packages, you
can involve helm without requiring it by using completing-read if
helm-mode is enabled otherwise fallback to completing-read-multiple.
You will have to let-bound (and declare it on top of your file)
helm-comp-read-use-marked.

Example from mu4e:

    (cond
       ((bound-and-true-p helm-mode)
        ;; tweaks for "helm"; it's not nice to have to special-case for
        ;; completion frameworks, but this has been supported for while.
        ;; basically, with helm, helm-comp-read-use-marked + completing-read
        ;; is preferred over completing-read-multiple
        (let ((helm-comp-read-use-marked t))
          (completing-read prompt candidates)))
       (multi
        (completing-read-multiple prompt candidates))
       (t
        (completing-read prompt candidates)))


> and a new instance of Emacs using -Q is spun up,

Yes much better.

> with all the packages you have listed loaded, and nothing
> else... Sounds like a fun little weekend project ;^)

Yes, nice.

>> Apart requiring a package just for the faces no, font-lock family
>> doesn't provide :background faces.
>
> I wasn't thinking just of font-lock, you could pick any face,
> eg. wfnames-modified could inherit from diff-refine-changed.

Yes, but I will have to require diff, right?

> That way themes don't have to add extra support, and the package looks
> more "native" without any additional effort.

Of course that would be nice.

>>>    "Provide filename completion in wfnames buffer."
>>>    (let ((beg (line-beginning-position))
>>>          (end (point)))
>>> +    ;; Does it make sense to extend beyond END to allow completing
>>> +    ;; file names mid-string?
>>
>> This one for now I don't know, I transformed your comment with a FIXME.
>
> That is fine, this is just a general thing I wonder about with CAP
> functions.

So yes, as explained in previous post it is what we want, i.e. get the
completion of the part of the line from bol to point.

> I am totally indifferent, they expand to almost the same code anyway:

Yes, I have no problems either to use dolist here.

> Great, I'll add the package then.

Great thanks!

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Changes to make in elpa-packages file for nongnu elpa

[Michael Albinus]

Philip Kaludercic <philipk@posteo.net> writes:

Hi Philip,

>>> Sorry about that, you can always feel free to CC me if you have specific
>>> questions that might get lost on the mailing list.  I suppose Stefan
>>> Monnier wouldn't object to be mentioned either.
>>
>> Ok thanks, I will next time now I know you and Stefan are in charge of
>> this.
>
> I am sort of an unofficial co-maintainer, without access to any
> server-side infrastructure, but other than that I'm glad to help within
> my possibilities.

Perhaps it is worth to mention this in admin/MAINTAINERS?

Best regards, Michael.

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

Michael Albinus <michael.albinus@gmx.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
> Hi Philip,
>
>>>> Sorry about that, you can always feel free to CC me if you have specific
>>>> questions that might get lost on the mailing list.  I suppose Stefan
>>>> Monnier wouldn't object to be mentioned either.
>>>
>>> Ok thanks, I will next time now I know you and Stefan are in charge of
>>> this.
>>
>> I am sort of an unofficial co-maintainer, without access to any
>> server-side infrastructure, but other than that I'm glad to help within
>> my possibilities.
>
> Perhaps it is worth to mention this in admin/MAINTAINERS?

There seems to be no information on ELPA in admin/MAINTAINERS.

> Best regards, Michael.

Re: Changes to make in elpa-packages file for nongnu elpa

[Michael Albinus]

Philip Kaludercic <philipk@posteo.net> writes:

Hi Philip,

>>> I am sort of an unofficial co-maintainer, without access to any
>>> server-side infrastructure, but other than that I'm glad to help within
>>> my possibilities.
>>
>> Perhaps it is worth to mention this in admin/MAINTAINERS?
>
> There seems to be no information on ELPA in admin/MAINTAINERS.

Yes, the file is incomplete. So it is time to say something about GNU
ELPA, IMHO.

Best regards, Michael.

Re: Changes to make in elpa-packages file for nongnu elpa

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > When a package is available in Melpa do we have to add it in nongnu as
  > > well to satisfy dependencies for a package already in nongnu?

  > Yes, if the dependency is missing someone only using GNU ELPA and NonGNU
  > ELPA wouldn't be able to install the package due to a missing dependency
  > and that is obviously not intended.

Our policy for Emacs is that we do not refer to MELPA for any reason.
That is because MELPA does not follow our policies about promoting
only free sofware.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

Michael Albinus <michael.albinus@gmx.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
> Hi Philip,
>
>>>> I am sort of an unofficial co-maintainer, without access to any
>>>> server-side infrastructure, but other than that I'm glad to help within
>>>> my possibilities.
>>>
>>> Perhaps it is worth to mention this in admin/MAINTAINERS?
>>
>> There seems to be no information on ELPA in admin/MAINTAINERS.
>
> Yes, the file is incomplete. So it is time to say something about GNU
> ELPA, IMHO.

In that case I'd like to request access to elpa.gnu.org, and ideally
also the list of users who have signed the CA.

I've CC'ed Eli and Stefan.

> Best regards, Michael.

Re: Changes to make in elpa-packages file for nongnu elpa

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: emacs-devel@gnu.org, "'Eli Zaretskii'" <eliz@gnu.org>, Stefan Monnier
>  <monnier@iro.umontreal.ca>
> Date: Wed, 09 Aug 2023 07:06:30 +0000
> 
> >>> Perhaps it is worth to mention this in admin/MAINTAINERS?
> >>
> >> There seems to be no information on ELPA in admin/MAINTAINERS.
> >
> > Yes, the file is incomplete. So it is time to say something about GNU
> > ELPA, IMHO.
> 
> In that case I'd like to request access to elpa.gnu.org

You mean you cannot access it now?

> and ideally also the list of users who have signed the CA.

No can do, for privacy reasons.  You can always ask me or Stefan to
check a name for you, though.

Thanks.

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: emacs-devel@gnu.org, "'Eli Zaretskii'" <eliz@gnu.org>, Stefan Monnier
>>  <monnier@iro.umontreal.ca>
>> Date: Wed, 09 Aug 2023 07:06:30 +0000
>> 
>> >>> Perhaps it is worth to mention this in admin/MAINTAINERS?
>> >>
>> >> There seems to be no information on ELPA in admin/MAINTAINERS.
>> >
>> > Yes, the file is incomplete. So it is time to say something about GNU
>> > ELPA, IMHO.
>> 
>> In that case I'd like to request access to elpa.gnu.org
>
> You mean you cannot access it now?

I can access it as a user, but I cannot access it to fix something.

>> and ideally also the list of users who have signed the CA.
>
> No can do, for privacy reasons.  You can always ask me or Stefan to
> check a name for you, though.

OK, but most of the time I don't have to check a single user but would
like to see check a set of names to see how many have or haven't signed
the CA.  Is that still a manageable effort on your end?

> Thanks.

Re: Changes to make in elpa-packages file for nongnu elpa

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: michael.albinus@gmx.de,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca
> Date: Wed, 09 Aug 2023 14:53:12 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> In that case I'd like to request access to elpa.gnu.org
> >
> > You mean you cannot access it now?
> 
> I can access it as a user, but I cannot access it to fix something.

Maybe I'm misremembering, but AFAIR if you have write access to the
Emacs Git repository, you also have write access to the ELPA Git
repository.

> >> and ideally also the list of users who have signed the CA.
> >
> > No can do, for privacy reasons.  You can always ask me or Stefan to
> > check a name for you, though.
> 
> OK, but most of the time I don't have to check a single user but would
> like to see check a set of names to see how many have or haven't signed
> the CA.  Is that still a manageable effort on your end?

Yes, feel free.

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: michael.albinus@gmx.de,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca
>> Date: Wed, 09 Aug 2023 14:53:12 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> In that case I'd like to request access to elpa.gnu.org
>> >
>> > You mean you cannot access it now?
>> 
>> I can access it as a user, but I cannot access it to fix something.
>
> Maybe I'm misremembering, but AFAIR if you have write access to the
> Emacs Git repository, you also have write access to the ELPA Git
> repository.

No I have access to emacs.git, elpa.git and nongnu.git, what I was
referent to was the server associated with elpa.gnu.org.

>> >> and ideally also the list of users who have signed the CA.
>> >
>> > No can do, for privacy reasons.  You can always ask me or Stefan to
>> > check a name for you, though.
>> 
>> OK, but most of the time I don't have to check a single user but would
>> like to see check a set of names to see how many have or haven't signed
>> the CA.  Is that still a manageable effort on your end?
>
> Yes, feel free.

OK, thank you for the clarification.

Re: Changes to make in elpa-packages file for nongnu elpa

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: michael.albinus@gmx.de,  emacs-devel@gnu.org,  monnier@iro.umontreal.ca
> Date: Wed, 09 Aug 2023 15:24:30 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Maybe I'm misremembering, but AFAIR if you have write access to the
> > Emacs Git repository, you also have write access to the ELPA Git
> > repository.
> 
> No I have access to emacs.git, elpa.git and nongnu.git, what I was
> referent to was the server associated with elpa.gnu.org.

Oh, you mean the server itself?  Then I think only Stefan can help
you.

Re: Changes to make in elpa-packages file for nongnu elpa

[No Wayman]

Thierry Volpiatto <thievol@posteo.net> writes:

>>> It is used specially for reproducing bugs in a clean 
>>> environment, see it
>
>> (It might be interesting to provide something like this for
>> package.el, to test packages in a generic way.)
>
> Yes, this would be interesting, it would be something like this:
>
> Emacs -Q
> M-x <A command that run a package alone, isolated from other
> packages nuisances>

I've written various macros that do as much.
See:

- straight.el's straight-bug-report macro 

https://github.com/radian-software/straight.el/blob/ff63b154bef1ef8d92c141bd189001bff74f6982/straight.el#L7049

- Elpaca's elpaca-test macro

https://github.com/progfolio/elpaca/blob/master/elpaca-test.el

- yodel

https://github.com/progfolio/yodel

All three are variations on the idea that it's useful to have a 
declarative form that can be shared to install packages in a clean 
environment, perform some testing, and report the results. Test 
environments can be run interactively or in batch mode.
e.g.


(elpaca-test
  :init
  (elpaca helm
    (require 'helm)
    (do-something-with-helm))
  (elpaca-wait))

Yodel was my attempt at providing something that could be 
package-manager agnostic and can be made to easily reformat the 
output of the report. Evaluating the following test:

(yodel :packages* helm
       :formatter yodel-format-as-mailing-list-message
       :post*
       (require 'helm)
       (message "good bye"))

Produces the following report:

Yodel[1] Report 2023-08-09 20:25:25
===================================

--8<---------------cut here---------------start------------->8---
(yodel
  :packages*
  helm
  :formatter yodel-format-as-mailing-list-message
  :post*
  (require 'helm) (message "good bye"))
--8<---------------cut here---------------end--------------->8---

STDOUT
======

> INFO     Scraping files for loaddefs... 
>   INFO     Scraping files for loaddefs...done
>   GEN      ../elpaca-autoloads.el
> Cloning into '/tmp/yodel-KSS7te/elpaca/repos/elpaca'...
> Your branch is up to date with 'origin/master'.
> Checking /tmp/yodel-KSS7te/elpaca/repos/elpaca...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca-info.el...
> Compiling /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca-log.el...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca-manager.el...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca-menu-elpa.el...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca-menu-melpa.el...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca-menu-org.el...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca-process.el...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca-test.el...
> Compiling /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca-ui.el...
> Compiling /tmp/yodel-KSS7te/elpaca/repos/elpaca/elpaca.el...
> Checking /tmp/yodel-KSS7te/elpaca/repos/elpaca/doc...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/doc/early-init.el...
> Compiling /tmp/yodel-KSS7te/elpaca/repos/elpaca/doc/init.el...
> Checking /tmp/yodel-KSS7te/elpaca/repos/elpaca/extensions...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/extensions/elpaca-use-package.el...
> Checking /tmp/yodel-KSS7te/elpaca/repos/elpaca/images...
> Checking /tmp/yodel-KSS7te/elpaca/repos/elpaca/test...
> Compiling 
> /tmp/yodel-KSS7te/elpaca/repos/elpaca/test/elpaca-test.el...
> Done (Total of 11 files compiled, 3 skipped in 4 directories)

> Downloading MELPA recipes...
> Downloading MELPA recipes...100%
> Downloading NonGNU-devel ELPA...
> Downloading GNU-devel ELPA...
> Downloading NonGNU ELPA...
> Downloading GNU ELPA...
> good bye

Environment
===========

- emacs version: GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, 
  GTK+ Version 3.24.38, cairo version 1.17.8)
 of 2023-08-09
- system type: gnu/linux

Packages
========

- async 
  https://github.com/jwiegley/emacs-async/commit/bb0509060ddc9c94bc7db7bddb12995acb3b53e4
- elpaca 
  https://github.com/progfolio/elpaca/commit/48da538b084870ef2126ca607af521eeee890fa7
- helm 
  https://github.com/emacs-helm/helm/commit/a968b9bdd243f9d8d9d572656268421cb1ea8b39
- helm-core 
  https://github.com/emacs-helm/helm/commit/a968b9bdd243f9d8d9d572656268421cb1ea8b39
- popup 
  https://github.com/auto-complete/popup-el/commit/707bb336c6eb25743f1516b6a2165b40659a9d02
- wfnames 
  https://github.com/thierryvolpiatto/wfnames/commit/e4d60978921b8a52693aebae988717261c89ed38

[1] https://www.github.com/progfolio/yodel

I pitched the idea a couple years ago on a bug report. The 
following message gives more of the rationale for the package:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=50842#35

Eli and Lars didn't seem too impressed at the time, but it clearly 
inspired "erts-mode", which Lars implemented shortly after. That's 
a step in the right direction, but not enough.
Having a declarative testing macro has saved me many hours in 
debugging and confirming elisp bugs across packages I maintain.

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 1436 bytes --]

Philip Kaludercic <philipk@posteo.net> writes:

>>>> It is used specially for reproducing bugs in a clean environment, see it
>>>> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
>>>> with only Helm loaded, this ensure the bug if one comes from Helm and
>>>> not another package. This is important especially nowaday people are
>>>> using "Emacs distribution" with the world list of packages installed.
>>>> Apart that the script is useful to quickly launch Emacs with helm, one
>>>> can use it from the Helm directory or symlinked to e.g. ~/bin. 
>>>
>>> I see.  In that case is there any reason you implement this as a shell
>>> script?
>>
>> Well when I wrote the script, packages where not existing and from
>> outside emacs it is actually the only way to run a package isolated.
>>
>>> (It might be interesting to provide something like this for
>>> package.el, to test packages in a generic way.)
>>
>> Yes, this would be interesting, it would be something like this:
>>
>> Emacs -Q
>> M-x <A command that run a package alone, isolated from other
>> packages nuisances>
>
> I was actually thinking of a command like
>
> M-x package-isolate RET foo,bar,baz RET
>
> and a new instance of Emacs using -Q is spun up, with all the packages
> you have listed loaded, and nothing else... Sounds like a fun little
> weekend project ;^)

Here is my first attempt at providing this kind of a command.  Any
comments?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-Add-command-to-start-Emacs-with-specific-packages.patch --]
[-- Type: text/x-diff, Size: 4095 bytes --]

From ae254f69f789ab4fa2b83bcf2cdc64291ae8aff6 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Tue, 15 Aug 2023 18:39:14 +0200
Subject: [PATCH 2/2] Add command to start Emacs with specific packages

* lisp/emacs-lisp/package.el (package-isolate): Add command.
* etc/NEWS: Announce it.
---
 etc/NEWS                   |  6 +++++
 lisp/emacs-lisp/package.el | 50 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 56 insertions(+)

diff --git a/etc/NEWS b/etc/NEWS
index 57f04609679..c374695a571 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -505,6 +505,12 @@ project, that you can quickly select using 'project-switch-project'
 When non-nil, package specifications with side-effects for building
 software will be used when building a package.
 
+---
+*** New command to start Emacs only with specific packages
+The command 'package-isolate' is equivalent to starting Emacs with the
+-Q flag and loading specific packages (and their dependencies)
+manually.
+
 ** Flymake
 
 +++
diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index b3062d2608b..c0bbdacec84 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2625,6 +2625,56 @@ package-autoremove
                   removable))
         (message "Nothing to autoremove")))))
 
+(defun package-isolate (packages)
+  "Start a new instance of Emacs and load only PACKAGES."
+  (interactive
+   (cl-loop for p in (cl-loop for p in (package--alist) append (cdr p))
+	    unless (package-built-in-p p)
+	    collect (cons (package-desc-full-name p) p) into table
+	    finally return
+	    (list (cl-loop for c in (completing-read-multiple
+                                     "Isolate packages: " table
+                                     nil t)
+		           collect (alist-get c table nil nil #'string=))
+                  current-prefix-arg)))
+  (cl-assert (cl-every #'package-desc-p packages))
+  (let* ((name (concat "package-isolate-" (mapconcat #'package-desc-full-name
+                                                     packages ",")))
+         (tmp-init (make-temp-file name t))
+         (elpa (expand-file-name "elpa" tmp-init))
+         args)
+    (make-directory elpa)
+    (dolist (package packages)
+      ;; We need to recursively expand all the dependencies of the
+      ;; requested packages and all of them to `load-path'.
+      (dolist (package (named-let loop ((pkg-desc package))
+                         (let (deps)
+                           (dolist (req (package-desc-reqs pkg-desc))
+                             (setq deps (nconc
+                                         (catch 'found
+                                           (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
+                                             (when (and (string= (car req) (package-desc-name p))
+                                                        (version-list-<= (cadr req) (package-desc-version p)))
+                                               (throw 'found (loop p)))))
+                                         deps)))
+                           (cons pkg-desc deps))))
+        (let* ((real (package-desc-dir package))
+               (link (expand-file-name (file-name-nondirectory real) elpa)))
+          (make-symbolic-link real link t)
+          (push (format "--directory=%s" link) args))
+        (let* ((load-suffixes '(".el" ".elc"))
+               (autoload (locate-library (package--autoloads-file-name package))))
+          (push (format "--load=%s" autoload) args))))
+    (apply #'start-process (concat "*" name "*") nil
+           (append (list (or (emacs-executable) "emacs")
+                         (format "--eval=%S"
+                                 '(progn
+                                    (require 'warnings)
+                                    (add-to-list 'warning-suppress-log-types 'initialization)))
+                         "--quick" "--debug-init"
+                         "--init-directory" tmp-init)
+                   args))))
+
 \f
 ;;;; Package description buffer.
 
-- 
2.39.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-Add-a-function-to-query-the-Emacs-executable.patch --]
[-- Type: text/x-diff, Size: 1330 bytes --]

From bc9ee566cb9fee337f94870aa718d9b5e57c007b Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Tue, 15 Aug 2023 18:37:59 +0200
Subject: [PATCH 1/2] Add a function to query the Emacs executable

* src/emacs.c (Femacs_executable): Add new function.
(syms_of_emacs): Announce it.
---
 src/emacs.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/src/emacs.c b/src/emacs.c
index 687b8c7f81f..4869093ce09 100644
--- a/src/emacs.c
+++ b/src/emacs.c
@@ -3039,6 +3039,18 @@ DEFUN ("kill-emacs", Fkill_emacs, Skill_emacs, 0, 2, "P",
   exit (exit_code);
 }
 
+DEFUN ("emacs-executable", Femacs_executable, Semacs_executable, 0, 0, "",
+       doc: /* Return a string with the file name of the Emacs executable.
+If this is not known, nil will be returned instead. */)
+  (void)
+{
+  static Lisp_Object *string;
+  if (string)
+    return string;
+  if (initial_emacs_executable)
+    return string = build_string(initial_emacs_executable);
+  return Qnil;
+}
 
 /* Perform an orderly shutdown of Emacs.  Autosave any modified
    buffers, kill any child processes, clean up the terminal modes (if
@@ -3566,6 +3578,7 @@ syms_of_emacs (void)
 #endif
 
   defsubr (&Skill_emacs);
+  defsubr (&Semacs_executable);
 
   defsubr (&Sinvocation_name);
   defsubr (&Sinvocation_directory);
-- 
2.39.2

Re: Changes to make in elpa-packages file for nongnu elpa

[Eshel Yaron]

Hi Philip,

Philip Kaludercic <philipk@posteo.net> writes:

> Here is my first attempt at providing this kind of a command.  Any
> comments?

I haven't followed the thread very closely, but this looks quite useful!
A few minor suggestions:

> diff --git a/etc/NEWS b/etc/NEWS
> index 57f04609679..c374695a571 100644
> --- a/etc/NEWS
> +++ b/etc/NEWS
> @@ -505,6 +505,12 @@ project, that you can quickly select using 'project-switch-project'
>  When non-nil, package specifications with side-effects for building
>  software will be used when building a package.
>  
> +---
> +*** New command to start Emacs only with specific packages

This should probably end with a period.

> +The command 'package-isolate' is equivalent to starting Emacs with the
> +-Q flag and loading specific packages (and their dependencies)
> +manually.

I'd rephrase this as: "The command 'package-isolate' starts a new Emacs
instance with minimum customizations (similar to 'emacs -Q') and a given
set of packages (with their dependencies)."

> +(defun package-isolate (packages)
> +  "Start a new instance of Emacs and load only PACKAGES."

I'd clarify that the new instance disregards the user's
config/customizations, how about "Start a new Emacs instance with
minimum customizations and load PACKAGES."

> +DEFUN ("emacs-executable", Femacs_executable, Semacs_executable, 0, 0, "",
> +       doc: /* Return a string with the file name of the Emacs executable.
> +If this is not known, nil will be returned instead. */)

Maybe say "Return the file name of the Emacs executable, or nil if
unknown." to steer clear of the passive voice.


Best,

Eshel

Re: Changes to make in elpa-packages file for nongnu elpa

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: emacs-devel@gnu.org
> Date: Tue, 15 Aug 2023 16:55:03 +0000
> 
> +*** New command to start Emacs only with specific packages
> +The command 'package-isolate' is equivalent to starting Emacs with the
> +-Q flag and loading specific packages (and their dependencies)
> +manually.

Seems strange to me to have a command to start another Emacs.  Why not
implement this as a command-line option instead?  That would be
consistent with the several options we already have, like -q, -Q, -D,
which already contrl what happens at startup.

> +        (let* ((real (package-desc-dir package))
> +               (link (expand-file-name (file-name-nondirectory real) elpa)))
> +          (make-symbolic-link real link t)

Using symbolic links makes the program less portable, so it is best to
avoid them.

> +    (apply #'start-process (concat "*" name "*") nil
> +           (append (list (or (emacs-executable) "emacs")

I don't think it's a good idea to invoke just "emacs", it could be a
completely different version of Emacs.

> +DEFUN ("emacs-executable", Femacs_executable, Semacs_executable, 0, 0, "",
> +       doc: /* Return a string with the file name of the Emacs executable.
> +If this is not known, nil will be returned instead. */)
> +  (void)

I don't understand why you need this primitive.  What's wrong with the
usual paradigm we use everywhere else:

  (expand-file-name invocation-name invocation-directory)

Proposal for 'package-isolate' command

[Philip Kaludercic]

Eshel Yaron <me@eshelyaron.com> writes:

> Hi Philip,
>
> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Here is my first attempt at providing this kind of a command.  Any
>> comments?
>
> I haven't followed the thread very closely, but this looks quite useful!
> A few minor suggestions:

This is mostly a tangent, so the previous discussion is not that
relevant.  I've changed the subject accordingly.

>> diff --git a/etc/NEWS b/etc/NEWS
>> index 57f04609679..c374695a571 100644
>> --- a/etc/NEWS
>> +++ b/etc/NEWS
>> @@ -505,6 +505,12 @@ project, that you can quickly select using 'project-switch-project'
>>  When non-nil, package specifications with side-effects for building
>>  software will be used when building a package.
>>  
>> +---
>> +*** New command to start Emacs only with specific packages
>
> This should probably end with a period.
>
>> +The command 'package-isolate' is equivalent to starting Emacs with the
>> +-Q flag and loading specific packages (and their dependencies)
>> +manually.
>
> I'd rephrase this as: "The command 'package-isolate' starts a new Emacs
> instance with minimum customizations (similar to 'emacs -Q') and a given
> set of packages (with their dependencies)."
>
>> +(defun package-isolate (packages)
>> +  "Start a new instance of Emacs and load only PACKAGES."
>
> I'd clarify that the new instance disregards the user's
> config/customizations, how about "Start a new Emacs instance with
> minimum customizations and load PACKAGES."
>
>> +DEFUN ("emacs-executable", Femacs_executable, Semacs_executable, 0, 0, "",
>> +       doc: /* Return a string with the file name of the Emacs executable.
>> +If this is not known, nil will be returned instead. */)
>
> Maybe say "Return the file name of the Emacs executable, or nil if
> unknown." to steer clear of the passive voice.

Thanks, I obviously cannot write documentation.  I've addressed the
changes locally, and will send an updated patch in response to Eli's
message.

> Best,
>
> Eshel

Proposal for 'package-isolate' command

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 2305 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: emacs-devel@gnu.org
>> Date: Tue, 15 Aug 2023 16:55:03 +0000
>> 
>> +*** New command to start Emacs only with specific packages
>> +The command 'package-isolate' is equivalent to starting Emacs with the
>> +-Q flag and loading specific packages (and their dependencies)
>> +manually.
>
> Seems strange to me to have a command to start another Emacs.  Why not
> implement this as a command-line option instead?  That would be
> consistent with the several options we already have, like -q, -Q, -D,
> which already contrl what happens at startup.

Mainly because this wouldn't support a completing-read interface, that
simplifies prompting the user for a set of packages.

But perhaps the bulk of this command could be implemented as a CLI
option, that this command could wrap?

>> +        (let* ((real (package-desc-dir package))
>> +               (link (expand-file-name (file-name-nondirectory real) elpa)))
>> +          (make-symbolic-link real link t)
>
> Using symbolic links makes the program less portable, so it is best to
> avoid them.

The reason I used them here, instead of just adding the directories
under ~/.config/emacs/elpa/ is that startup.el issues a warning along
the lines of

  Your `load-path' seems to contain your `user-emacs-directory' ...

My previous patch actually included the code that suppresses all
warnings during initialisation, so I've removed this hack.

>> +    (apply #'start-process (concat "*" name "*") nil
>> +           (append (list (or (emacs-executable) "emacs")
>
> I don't think it's a good idea to invoke just "emacs", it could be a
> completely different version of Emacs.
>
>> +DEFUN ("emacs-executable", Femacs_executable, Semacs_executable, 0, 0, "",
>> +       doc: /* Return a string with the file name of the Emacs executable.
>> +If this is not known, nil will be returned instead. */)
>> +  (void)
>
> I don't understand why you need this primitive.  What's wrong with the
> usual paradigm we use everywhere else:
>
>   (expand-file-name invocation-name invocation-directory)

Because I failed to remember it.  This also appears to not have the risk
of returning nil, as my proposed command could have.  I've applies these
and related changes below:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-command-to-start-Emacs-with-specific-packages.patch --]
[-- Type: text/x-diff, Size: 3999 bytes --]

From 47dde2be5732ec9dfa153813649e385e0227751e Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Tue, 15 Aug 2023 18:39:14 +0200
Subject: [PATCH] Add command to start Emacs with specific packages

* lisp/emacs-lisp/package.el (package-isolate): Add command.
* etc/NEWS: Announce it.
---
 etc/NEWS                   |  6 +++++
 lisp/emacs-lisp/package.el | 47 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/etc/NEWS b/etc/NEWS
index 57f04609679..c374695a571 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -505,6 +505,12 @@ project, that you can quickly select using 'project-switch-project'
 When non-nil, package specifications with side-effects for building
 software will be used when building a package.
 
+---
+*** New command to start Emacs only with specific packages
+The command 'package-isolate' is equivalent to starting Emacs with the
+-Q flag and loading specific packages (and their dependencies)
+manually.
+
 ** Flymake
 
 +++
diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index b3062d2608b..838b2823466 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2625,6 +2625,53 @@ package-autoremove
                   removable))
         (message "Nothing to autoremove")))))
 
+(defun package-isolate (packages)
+  "Start an uncustomised Emacs and only load a set of PACKAGES."
+  (interactive
+   (cl-loop for p in (cl-loop for p in (package--alist) append (cdr p))
+	    unless (package-built-in-p p)
+	    collect (cons (package-desc-full-name p) p) into table
+	    finally return
+	    (list (cl-loop for c in (completing-read-multiple
+                                     "Isolate packages: " table
+                                     nil t)
+		           collect (alist-get c table nil nil #'string=)))))
+  (cl-assert (cl-every #'package-desc-p packages))
+  (let* ((name (concat "package-isolate-" (mapconcat #'package-desc-full-name
+                                                     packages ",")))
+         (tmp-init (make-temp-file name t))
+         (elpa (expand-file-name "elpa" tmp-init))
+         args)
+    (make-directory elpa)
+    (dolist (package packages)
+      ;; We need to recursively expand all the dependencies of the
+      ;; requested packages and all of them to `load-path'.
+      (dolist (package (named-let loop ((pkg-desc package))
+                         (let (deps)
+                           (dolist (req (package-desc-reqs pkg-desc))
+                             (setq deps (nconc
+                                         (catch 'found
+                                           (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
+                                             (when (and (string= (car req) (package-desc-name p))
+                                                        (version-list-<= (cadr req) (package-desc-version p)))
+                                               (throw 'found (loop p)))))
+                                         deps)))
+                           (cons pkg-desc deps))))
+        (push (format "--directory=%s" (package-desc-dir package)) args)
+        (let* ((load-suffixes '(".el" ".elc"))
+               (autoload (locate-library (package--autoloads-file-name package))))
+          (push (format "--load=%s" autoload) args))))
+    (apply #'start-process (concat "*" name "*") nil
+           (append (list (expand-file-name invocation-name invocation-directory)
+                         "--quick" "--debug-init"
+                         (format "--eval=%S"
+                                 '(progn
+                                    (require 'warnings)
+                                    (add-to-list 'warning-suppress-log-types 'initialization)))
+                         "--init-directory" tmp-init)
+                   args))
+    (message "Started Emacs with the init directory: %s" tmp-init)))
+
 \f
 ;;;; Package description buffer.
 
-- 
2.39.2

Re: Changes to make in elpa-packages file for nongnu elpa

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 3279 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>>>>> It is used specially for reproducing bugs in a clean environment, see it
>>>>> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
>>>>> with only Helm loaded, this ensure the bug if one comes from Helm and
>>>>> not another package. This is important especially nowaday people are
>>>>> using "Emacs distribution" with the world list of packages installed.
>>>>> Apart that the script is useful to quickly launch Emacs with helm, one
>>>>> can use it from the Helm directory or symlinked to e.g. ~/bin. 
>>>>
>>>> I see.  In that case is there any reason you implement this as a shell
>>>> script?
>>>
>>> Well when I wrote the script, packages where not existing and from
>>> outside emacs it is actually the only way to run a package isolated.
>>>
>>>> (It might be interesting to provide something like this for
>>>> package.el, to test packages in a generic way.)
>>>
>>> Yes, this would be interesting, it would be something like this:
>>>
>>> Emacs -Q
>>> M-x <A command that run a package alone, isolated from other
>>> packages nuisances>
>>
>> I was actually thinking of a command like
>>
>> M-x package-isolate RET foo,bar,baz RET
>>
>> and a new instance of Emacs using -Q is spun up, with all the packages
>> you have listed loaded, and nothing else... Sounds like a fun little
>> weekend project ;^)
>
> Here is my first attempt at providing this kind of a command.  Any
> comments?

Why not reusing package.el functions?
Why do you want to start in an isolated elpa directory?
Isn't something like this suffice? (just missing to fallback to CRM when
helm is not available)

    (defun package-isolate (packages)
      "Start an uncustomised Emacs and only load a set of PACKAGES."
      (interactive
       (list (let ((helm-comp-read-use-marked t))
               (completing-read "Packages: " (mapcar #'car (package--alist))))))
      (let* ((name (concat "package-isolate-" (mapconcat #'identity
                                                         packages ",")))
             (deps (cl-loop for p in packages
                            for sym = (intern p)
                            append (package--dependencies sym))))
        (apply #'start-process (concat "*" name "*") nil
               (list (file-truename (expand-file-name invocation-name invocation-directory))
                     "--quick" "--debug-init"
                     (format "--eval=%S"
                             `(progn
                               (require 'warnings)
                               (add-to-list 'warning-suppress-log-types 'initialization)
                               (require 'package)
                               (setq package-load-list
                                ',(append (mapcar (lambda (p) (list (intern p) t)) packages)
                                          (mapcar (lambda (p) (list p t)) deps)))
                               (package-initialize)))))))

> [2. text/x-diff; 0002-Add-command-to-start-Emacs-with-specific-packages.patch]...
>
> [3. text/x-diff; 0001-Add-a-function-to-query-the-Emacs-executable.patch]...


-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 4087 bytes --]

Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>>>>> It is used specially for reproducing bugs in a clean environment, see it
>>>>>> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
>>>>>> with only Helm loaded, this ensure the bug if one comes from Helm and
>>>>>> not another package. This is important especially nowaday people are
>>>>>> using "Emacs distribution" with the world list of packages installed.
>>>>>> Apart that the script is useful to quickly launch Emacs with helm, one
>>>>>> can use it from the Helm directory or symlinked to e.g. ~/bin. 
>>>>>
>>>>> I see.  In that case is there any reason you implement this as a shell
>>>>> script?
>>>>
>>>> Well when I wrote the script, packages where not existing and from
>>>> outside emacs it is actually the only way to run a package isolated.
>>>>
>>>>> (It might be interesting to provide something like this for
>>>>> package.el, to test packages in a generic way.)
>>>>
>>>> Yes, this would be interesting, it would be something like this:
>>>>
>>>> Emacs -Q
>>>> M-x <A command that run a package alone, isolated from other
>>>> packages nuisances>
>>>
>>> I was actually thinking of a command like
>>>
>>> M-x package-isolate RET foo,bar,baz RET
>>>
>>> and a new instance of Emacs using -Q is spun up, with all the packages
>>> you have listed loaded, and nothing else... Sounds like a fun little
>>> weekend project ;^)
>>
>> Here is my first attempt at providing this kind of a command.  Any
>> comments?
>
> Why not reusing package.el functions?
> Why do you want to start in an isolated elpa directory?
> Isn't something like this suffice? (just missing to fallback to CRM when
> helm is not available)

I don't know much about Helm, but does it not support CRM?

>     (defun package-isolate (packages)
>       "Start an uncustomised Emacs and only load a set of PACKAGES."
>       (interactive
>        (list (let ((helm-comp-read-use-marked t))
>                (completing-read "Packages: " (mapcar #'car (package--alist))))))

This doesn't allow selecting specific package versions, in case multiple
are installed (which might easily happen if you use package-vc).  I
stole the code in my patch from package-delete, and I guess it would be
possible to generalise it into a utility function.

>       (let* ((name (concat "package-isolate-" (mapconcat #'identity
>                                                          packages ",")))

This doesn't work, because the above returns a string, not a list of strings.

>              (deps (cl-loop for p in packages
>                             for sym = (intern p)
>                             append (package--dependencies sym))))
>         (apply #'start-process (concat "*" name "*") nil
>                (list (file-truename (expand-file-name invocation-name invocation-directory))
>                      "--quick" "--debug-init"
>                      (format "--eval=%S"
>                              `(progn
>                                (require 'warnings)
>                                (add-to-list 'warning-suppress-log-types 'initialization)
>                                (require 'package)
>                                (setq package-load-list
>                                 ',(append (mapcar (lambda (p) (list (intern p) t)) packages)
>                                           (mapcar (lambda (p) (list p t)) deps)))
>                                (package-initialize)))))))

This is actually a good idea, assuming there are no issues with lexical
scoping that might arise from --eval.  I didn't think of using
package-load-list here, but it seems that this only works if we don't
add --init-directory, because otherwise the elpa/ directory is not
populated.  It seems to me, that for a proper isolated environment, we
should add a --init-directory option.  This is easy to fix though, we
just need to specify `package-user-dir' at startup.  Here is my updated
patch, merging our approaches:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-command-to-start-Emacs-with-specific-packages.patch --]
[-- Type: text/x-diff, Size: 4878 bytes --]

From 6ae48d6d28112d487ea09ca905814da47c2a26cf Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Tue, 15 Aug 2023 18:39:14 +0200
Subject: [PATCH] Add command to start Emacs with specific packages

* lisp/emacs-lisp/package.el (package--dependencies): Extend function
to handle and return package descriptors.
(package-isolate): Add new command.
* etc/NEWS: Announce new command.
---
 etc/NEWS                   |  6 ++++
 lisp/emacs-lisp/package.el | 57 ++++++++++++++++++++++++++++++++++----
 2 files changed, 57 insertions(+), 6 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 57f04609679..c374695a571 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -505,6 +505,12 @@ project, that you can quickly select using 'project-switch-project'
 When non-nil, package specifications with side-effects for building
 software will be used when building a package.
 
+---
+*** New command to start Emacs only with specific packages
+The command 'package-isolate' is equivalent to starting Emacs with the
+-Q flag and loading specific packages (and their dependencies)
+manually.
+
 ** Flymake
 
 +++
diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index b3062d2608b..9e969ce1024 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2330,12 +2330,26 @@ package-upgrade-all
       (mapc #'package-upgrade upgradeable))))
 
 (defun package--dependencies (pkg)
-  "Return a list of all dependencies PKG has.
-This is done recursively."
-  ;; Can we have circular dependencies?  Assume "nope".
-  (when-let* ((desc (cadr (assq pkg package-archive-contents)))
-              (deps (mapcar #'car (package-desc-reqs desc))))
-    (delete-dups (apply #'nconc deps (mapcar #'package--dependencies deps)))))
+  "Return a list of all recursive dependencies of PKG.
+If PKG is a package descriptor, the return value will consist of
+a list of package descriptors.  If PKG is a symbol, designating a
+package, the return value will be a list of symbols designating
+packages."
+  (when-let* ((desc (if (package-desc-p pkg) pkg
+                      (cadr (assq pkg package-archive-contents)))))
+    ;; Can we have circular dependencies?  Assume "nope".
+    (let ((all (named-let more ((pkg-desc desc))
+                 (let (deps)
+                   (dolist (req (package-desc-reqs pkg-desc))
+                     (setq deps (nconc
+                                 (catch 'found
+                                   (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
+                                     (when (and (string= (car req) (package-desc-name p))
+                                                (version-list-<= (cadr req) (package-desc-version p)))
+                                       (throw 'found (more p)))))
+                                 deps)))
+                   (delete-dups (cons pkg-desc deps))))))
+      (remq pkg (mapcar (if (package-desc-p pkg) #'identity #'package-desc-name) all)))))
 
 (defun package-strip-rcs-id (str)
   "Strip RCS version ID from the version string STR.
@@ -2625,6 +2639,37 @@ package-autoremove
                   removable))
         (message "Nothing to autoremove")))))
 
+(defun package-isolate (packages)
+  "Start an uncustomised Emacs and only load a set of PACKAGES."
+  (interactive
+   (cl-loop for p in (cl-loop for p in (package--alist) append (cdr p))
+	    unless (package-built-in-p p)
+	    collect (cons (package-desc-full-name p) p) into table
+	    finally return
+	    (list (cl-loop for c in (completing-read-multiple
+                                     "Isolate packages: " table
+                                     nil t)
+		           collect (alist-get c table nil nil #'string=)))))
+  (let* ((name (concat "package-isolate-"
+                       (mapconcat #'package-desc-full-name packages ",")))
+         package-load-list)
+    (dolist (package (append packages (mapcan #'package--dependencies packages)))
+      (push (list (package-desc-name package)
+                  (package-version-join (package-desc-version package)))
+              package-load-list))
+    (apply #'start-process (concat "*" name "*") nil
+           (list (expand-file-name invocation-name invocation-directory)
+                 "--quick" "--debug-init"
+                 "--init-directory" (make-temp-file name t)
+                 (format "--eval=%S"
+                         `(progn
+                            (require 'warnings)
+                            (add-to-list 'warning-suppress-log-types 'initialization)
+                            (require 'package)
+                            (setq package-user-dir ,(expand-file-name package-user-dir)
+                                  package-load-list ',package-load-list)
+                            (package-initialize)))))))
+
 \f
 ;;;; Package description buffer.
 
-- 
2.39.2


[-- Attachment #3: Type: text/plain, Size: 1400 bytes --]


>> [2. text/x-diff; 0002-Add-command-to-start-Emacs-with-specific-packages.patch]...
>>
>> [3. text/x-diff; 0001-Add-a-function-to-query-the-Emacs-executable.patch]...

I have slightly modified your version, fixing issues I had, in case
anyone else wants to try it out:

        (defun package-isolate (packages)
          "Start an uncustomised Emacs and only load a set of PACKAGES."
          (interactive
           (list (mapcar #'intern (completing-read-multiple "Packages: " (mapcar #'car (package--alist))))))
          (let* ((name (concat "package-isolate-" (mapconcat #'symbol-name packages ",")))
                 (deps (mapcan #'package--dependencies packages)))
            (apply #'start-process (concat "*" name "*") nil
                   (list (file-truename (expand-file-name invocation-name invocation-directory))
                         "--quick" "--debug-init"
                         (format "--eval=%S"
                                 `(progn
                                    (require 'warnings)
                                    (add-to-list 'warning-suppress-log-types 'initialization)
                                    (require 'package)
                                    (setq package-load-list
                                          ',(mapcar (lambda (p) (list p t)) (append packages deps)))
                                    (package-initialize)))))))

Re: Changes to make in elpa-packages file for nongnu elpa

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 6276 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>>>>> It is used specially for reproducing bugs in a clean environment, see it
>>>>>>> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
>>>>>>> with only Helm loaded, this ensure the bug if one comes from Helm and
>>>>>>> not another package. This is important especially nowaday people are
>>>>>>> using "Emacs distribution" with the world list of packages installed.
>>>>>>> Apart that the script is useful to quickly launch Emacs with helm, one
>>>>>>> can use it from the Helm directory or symlinked to e.g. ~/bin. 
>>>>>>
>>>>>> I see.  In that case is there any reason you implement this as a shell
>>>>>> script?
>>>>>
>>>>> Well when I wrote the script, packages where not existing and from
>>>>> outside emacs it is actually the only way to run a package isolated.
>>>>>
>>>>>> (It might be interesting to provide something like this for
>>>>>> package.el, to test packages in a generic way.)
>>>>>
>>>>> Yes, this would be interesting, it would be something like this:
>>>>>
>>>>> Emacs -Q
>>>>> M-x <A command that run a package alone, isolated from other
>>>>> packages nuisances>
>>>>
>>>> I was actually thinking of a command like
>>>>
>>>> M-x package-isolate RET foo,bar,baz RET
>>>>
>>>> and a new instance of Emacs using -Q is spun up, with all the packages
>>>> you have listed loaded, and nothing else... Sounds like a fun little
>>>> weekend project ;^)
>>>
>>> Here is my first attempt at providing this kind of a command.  Any
>>> comments?
>>
>> Why not reusing package.el functions?
>> Why do you want to start in an isolated elpa directory?
>> Isn't something like this suffice? (just missing to fallback to CRM when
>> helm is not available)
>
> I don't know much about Helm, but does it not support CRM?

It does, but this is a much better interface than CRM.

>>     (defun package-isolate (packages)
>>       "Start an uncustomised Emacs and only load a set of PACKAGES."
>>       (interactive
>>        (list (let ((helm-comp-read-use-marked t))
>>                (completing-read "Packages: " (mapcar #'car (package--alist))))))
>
> This doesn't allow selecting specific package versions, in case multiple
> are installed (which might easily happen if you use package-vc).  I
> stole the code in my patch from package-delete, and I guess it would be
> possible to generalise it into a utility function.

Ok, I don't know much how package-vc works.

>>       (let* ((name (concat "package-isolate-" (mapconcat #'identity
>>                                                          packages ",")))
>
> This doesn't work, because the above returns a string, not a list of strings.

No, it works, the above returns a list of strings (the completing-read
allows marking and returns always a list).

>>              (deps (cl-loop for p in packages
>>                             for sym = (intern p)
>>                             append (package--dependencies sym))))
>>         (apply #'start-process (concat "*" name "*") nil
>>                (list (file-truename (expand-file-name invocation-name invocation-directory))
>>                      "--quick" "--debug-init"
>>                      (format "--eval=%S"
>>                              `(progn
>>                                (require 'warnings)
>>                                (add-to-list 'warning-suppress-log-types 'initialization)
>>                                (require 'package)
>>                                (setq package-load-list
>>                                 ',(append (mapcar (lambda (p) (list (intern p) t)) packages)
>>                                           (mapcar (lambda (p) (list p t)) deps)))
>>                                (package-initialize)))))))
>
> This is actually a good idea, assuming there are no issues with lexical
> scoping that might arise from --eval.  I didn't think of using
> package-load-list here, but it seems that this only works if we don't
> add --init-directory, because otherwise the elpa/ directory is not
> populated.

Yes of course.

> It seems to me, that for a proper isolated environment, we should add
> a --init-directory option.

Why as long as other directories in elpa are not in load-path?

> This is easy to fix though, we just need to specify `package-user-dir'
> at startup.  Here is my updated patch, merging our approaches:
>
> [2. text/x-diff; 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...
>
>
>>> [2. text/x-diff; 0002-Add-command-to-start-Emacs-with-specific-packages.patch]...
>>>
>>> [3. text/x-diff; 0001-Add-a-function-to-query-the-Emacs-executable.patch]...
>
> I have slightly modified your version, fixing issues I had, in case
> anyone else wants to try it out:

Thanks, sorry to not provide the CRM, I quickly wrote this.

>         (defun package-isolate (packages)
>           "Start an uncustomised Emacs and only load a set of PACKAGES."
>           (interactive
>            (list (mapcar #'intern (completing-read-multiple "Packages: " (mapcar #'car (package--alist))))))
>           (let* ((name (concat "package-isolate-" (mapconcat #'symbol-name packages ",")))
>                  (deps (mapcan #'package--dependencies packages)))
>             (apply #'start-process (concat "*" name "*") nil
>                    (list (file-truename (expand-file-name invocation-name invocation-directory))
>                          "--quick" "--debug-init"
>                          (format "--eval=%S"
>                                  `(progn
>                                     (require 'warnings)
>                                     (add-to-list 'warning-suppress-log-types 'initialization)
>                                     (require 'package)
>                                     (setq package-load-list
>                                           ',(mapcar (lambda (p) (list p t)) (append packages deps)))
>                                     (package-initialize)))))))


-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Thierry Volpiatto <thievol@posteo.net> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>
>>>>>>>> It is used specially for reproducing bugs in a clean environment, see it
>>>>>>>> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
>>>>>>>> with only Helm loaded, this ensure the bug if one comes from Helm and
>>>>>>>> not another package. This is important especially nowaday people are
>>>>>>>> using "Emacs distribution" with the world list of packages installed.
>>>>>>>> Apart that the script is useful to quickly launch Emacs with helm, one
>>>>>>>> can use it from the Helm directory or symlinked to e.g. ~/bin. 
>>>>>>>
>>>>>>> I see.  In that case is there any reason you implement this as a shell
>>>>>>> script?
>>>>>>
>>>>>> Well when I wrote the script, packages where not existing and from
>>>>>> outside emacs it is actually the only way to run a package isolated.
>>>>>>
>>>>>>> (It might be interesting to provide something like this for
>>>>>>> package.el, to test packages in a generic way.)
>>>>>>
>>>>>> Yes, this would be interesting, it would be something like this:
>>>>>>
>>>>>> Emacs -Q
>>>>>> M-x <A command that run a package alone, isolated from other
>>>>>> packages nuisances>
>>>>>
>>>>> I was actually thinking of a command like
>>>>>
>>>>> M-x package-isolate RET foo,bar,baz RET
>>>>>
>>>>> and a new instance of Emacs using -Q is spun up, with all the packages
>>>>> you have listed loaded, and nothing else... Sounds like a fun little
>>>>> weekend project ;^)
>>>>
>>>> Here is my first attempt at providing this kind of a command.  Any
>>>> comments?
>>>
>>> Why not reusing package.el functions?
>>> Why do you want to start in an isolated elpa directory?
>>> Isn't something like this suffice? (just missing to fallback to CRM when
>>> helm is not available)
>>
>> I don't know much about Helm, but does it not support CRM?
>
> It does, but this is a much better interface than CRM.

I don't think it makes sense to add support like this in the core, but
is there something we should keep in mind to not hinder integration with
Helm?

>>>     (defun package-isolate (packages)
>>>       "Start an uncustomised Emacs and only load a set of PACKAGES."
>>>       (interactive
>>>        (list (let ((helm-comp-read-use-marked t))
>>>                (completing-read "Packages: " (mapcar #'car (package--alist))))))
>>
>> This doesn't allow selecting specific package versions, in case multiple
>> are installed (which might easily happen if you use package-vc).  I
>> stole the code in my patch from package-delete, and I guess it would be
>> possible to generalise it into a utility function.
>
> Ok, I don't know much how package-vc works.

I can have a classical tarball package installed next to a VC package,
just like you can have a built-in package or a system package, and it
makes sense to be able to decide which one to isolate.

>>>       (let* ((name (concat "package-isolate-" (mapconcat #'identity
>>>                                                          packages ",")))
>>
>> This doesn't work, because the above returns a string, not a list of strings.
>
> No, it works, the above returns a list of strings (the completing-read
> allows marking and returns always a list).

Not in my case, I got a type error.

>>>              (deps (cl-loop for p in packages
>>>                             for sym = (intern p)
>>>                             append (package--dependencies sym))))
>>>         (apply #'start-process (concat "*" name "*") nil
>>>                (list (file-truename (expand-file-name invocation-name invocation-directory))
>>>                      "--quick" "--debug-init"
>>>                      (format "--eval=%S"
>>>                              `(progn
>>>                                (require 'warnings)
>>>                                (add-to-list 'warning-suppress-log-types 'initialization)
>>>                                (require 'package)
>>>                                (setq package-load-list
>>>                                 ',(append (mapcar (lambda (p) (list (intern p) t)) packages)
>>>                                           (mapcar (lambda (p) (list p t)) deps)))
>>>                                (package-initialize)))))))
>>
>> This is actually a good idea, assuming there are no issues with lexical
>> scoping that might arise from --eval.  I didn't think of using
>> package-load-list here, but it seems that this only works if we don't
>> add --init-directory, because otherwise the elpa/ directory is not
>> populated.
>
> Yes of course.
>
>> It seems to me, that for a proper isolated environment, we should add
>> a --init-directory option.
>
> Why as long as other directories in elpa are not in load-path?

Mainly to avoid issues with packages that might place files in the
configuration directory, which might hinder the reproduction of bugs.

Upon reflection, my approach might have an issue if the user wishes to
install a package, for the sake of testing within the isolated
environment.  Perhaps it would be better to set `package-directory-list'
instead?

Also, would it make sense to have some visual/textual indication that
this is a testing environment?  Perhaps the *scratch* buffer string
could be modified or the default background colour could be set to
something else.

>> This is easy to fix though, we just need to specify `package-user-dir'
>> at startup.  Here is my updated patch, merging our approaches:
>>
>> [2. text/x-diff;
>> 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...
>>
>>
>>>> [2. text/x-diff;
>>>> 0002-Add-command-to-start-Emacs-with-specific-packages.patch]...
>>>>
>>>> [3. text/x-diff; 0001-Add-a-function-to-query-the-Emacs-executable.patch]...
>>
>> I have slightly modified your version, fixing issues I had, in case
>> anyone else wants to try it out:
>
> Thanks, sorry to not provide the CRM, I quickly wrote this.

np.

>>         (defun package-isolate (packages)
>>           "Start an uncustomised Emacs and only load a set of PACKAGES."
>>           (interactive
>>            (list (mapcar #'intern (completing-read-multiple "Packages: " (mapcar #'car (package--alist))))))
>>           (let* ((name (concat "package-isolate-" (mapconcat #'symbol-name packages ",")))
>>                  (deps (mapcan #'package--dependencies packages)))
>>             (apply #'start-process (concat "*" name "*") nil
>>                    (list (file-truename (expand-file-name invocation-name invocation-directory))
>>                          "--quick" "--debug-init"
>>                          (format "--eval=%S"
>>                                  `(progn
>>                                     (require 'warnings)
>>                                     (add-to-list 'warning-suppress-log-types 'initialization)
>>                                     (require 'package)
>>                                     (setq package-load-list
>>                                           ',(mapcar (lambda (p) (list p t)) (append packages deps)))
>>                                     (package-initialize)))))))

Re: Proposal for 'package-isolate' command

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: thievol@posteo.net,  emacs-devel@gnu.org
> Date: Tue, 15 Aug 2023 19:52:32 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Seems strange to me to have a command to start another Emacs.  Why not
> > implement this as a command-line option instead?  That would be
> > consistent with the several options we already have, like -q, -Q, -D,
> > which already contrl what happens at startup.
> 
> Mainly because this wouldn't support a completing-read interface, that
> simplifies prompting the user for a set of packages.

Shells can complete as well, right?  And once you typed the command
once, shells have features to help you invoke commands from history,
so you won't need to retype it more than once.

> But perhaps the bulk of this command could be implemented as a CLI
> option, that this command could wrap?

Maybe.  But again, it's strange to have to launch another process from
Emacs for such a simple functionality.  And if the number of packages
to load is large, typing them manually would be annoying anyway, so
people will probably store the list in a file or something?  So maybe
we should allow specifying a file with the list of packages as well?

> >> +          (make-symbolic-link real link t)
> >
> > Using symbolic links makes the program less portable, so it is best to
> > avoid them.
> 
> The reason I used them here, instead of just adding the directories
> under ~/.config/emacs/elpa/ is that startup.el issues a warning along
> the lines of
> 
>   Your `load-path' seems to contain your `user-emacs-directory' ...
> 
> My previous patch actually included the code that suppresses all
> warnings during initialisation, so I've removed this hack.

I don't understand how this is different from a "normal" startup of
Emacs with all the packages.  Here, you just want to load part of the
packages that are loaded normally, so why does this warning pop up in
this case, but not when Emacs starts "normally"?  I'm probably missing
something.

Re: Changes to make in elpa-packages file for nongnu elpa

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 8327 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Thierry Volpiatto <thievol@posteo.net> writes:
>>>
>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>
>>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>>
>>>>>>>>> It is used specially for reproducing bugs in a clean environment, see it
>>>>>>>>> as emacs -Q for Emacs when reporting bugs. This script starts Emacs -Q
>>>>>>>>> with only Helm loaded, this ensure the bug if one comes from Helm and
>>>>>>>>> not another package. This is important especially nowaday people are
>>>>>>>>> using "Emacs distribution" with the world list of packages installed.
>>>>>>>>> Apart that the script is useful to quickly launch Emacs with helm, one
>>>>>>>>> can use it from the Helm directory or symlinked to e.g. ~/bin. 
>>>>>>>>
>>>>>>>> I see.  In that case is there any reason you implement this as a shell
>>>>>>>> script?
>>>>>>>
>>>>>>> Well when I wrote the script, packages where not existing and from
>>>>>>> outside emacs it is actually the only way to run a package isolated.
>>>>>>>
>>>>>>>> (It might be interesting to provide something like this for
>>>>>>>> package.el, to test packages in a generic way.)
>>>>>>>
>>>>>>> Yes, this would be interesting, it would be something like this:
>>>>>>>
>>>>>>> Emacs -Q
>>>>>>> M-x <A command that run a package alone, isolated from other
>>>>>>> packages nuisances>
>>>>>>
>>>>>> I was actually thinking of a command like
>>>>>>
>>>>>> M-x package-isolate RET foo,bar,baz RET
>>>>>>
>>>>>> and a new instance of Emacs using -Q is spun up, with all the packages
>>>>>> you have listed loaded, and nothing else... Sounds like a fun little
>>>>>> weekend project ;^)
>>>>>
>>>>> Here is my first attempt at providing this kind of a command.  Any
>>>>> comments?
>>>>
>>>> Why not reusing package.el functions?
>>>> Why do you want to start in an isolated elpa directory?
>>>> Isn't something like this suffice? (just missing to fallback to CRM when
>>>> helm is not available)
>>>
>>> I don't know much about Helm, but does it not support CRM?
>>
>> It does, but this is a much better interface than CRM.
>
> I don't think it makes sense to add support like this in the core,

Perhaps yes. However it doesn't requires helm:

     (let ((helm-comp-read-use-marked t))
        (if (and (boundp 'helm-mode) helm-mode)
            (completing-read ...)
          (completing-read-multiple ...)

> but is there something we should keep in mind to not hinder
> integration with Helm?

No don't worry, helm will work in any cases, thanks.

>>>>     (defun package-isolate (packages)
>>>>       "Start an uncustomised Emacs and only load a set of PACKAGES."
>>>>       (interactive
>>>>        (list (let ((helm-comp-read-use-marked t))
>>>>                (completing-read "Packages: " (mapcar #'car (package--alist))))))
>>>
>>> This doesn't allow selecting specific package versions, in case multiple
>>> are installed (which might easily happen if you use package-vc).  I
>>> stole the code in my patch from package-delete, and I guess it would be
>>> possible to generalise it into a utility function.
>>
>> Ok, I don't know much how package-vc works.
>
> I can have a classical tarball package installed next to a VC package,
> just like you can have a built-in package or a system package, and it
> makes sense to be able to decide which one to isolate.

Ok, I understand.

>>>>       (let* ((name (concat "package-isolate-" (mapconcat #'identity
>>>>                                                          packages ",")))
>>>
>>> This doesn't work, because the above returns a string, not a list of strings.
>>
>> No, it works, the above returns a list of strings (the completing-read
>> allows marking and returns always a list).
>
> Not in my case, I got a type error.

Yes, because the let-bounded var had no effect in your case, otherwise
when helm is installed and helm-mode enabled the completing-read will
always return a list.

>>>>              (deps (cl-loop for p in packages
>>>>                             for sym = (intern p)
>>>>                             append (package--dependencies sym))))
>>>>         (apply #'start-process (concat "*" name "*") nil
>>>>                (list (file-truename (expand-file-name invocation-name invocation-directory))
>>>>                      "--quick" "--debug-init"
>>>>                      (format "--eval=%S"
>>>>                              `(progn
>>>>                                (require 'warnings)
>>>>                                (add-to-list 'warning-suppress-log-types 'initialization)
>>>>                                (require 'package)
>>>>                                (setq package-load-list
>>>>                                 ',(append (mapcar (lambda (p) (list (intern p) t)) packages)
>>>>                                           (mapcar (lambda (p) (list p t)) deps)))
>>>>                                (package-initialize)))))))
>>>
>>> This is actually a good idea, assuming there are no issues with lexical
>>> scoping that might arise from --eval.  I didn't think of using
>>> package-load-list here, but it seems that this only works if we don't
>>> add --init-directory, because otherwise the elpa/ directory is not
>>> populated.
>>
>> Yes of course.
>>
>>> It seems to me, that for a proper isolated environment, we should add
>>> a --init-directory option.
>>
>> Why as long as other directories in elpa are not in load-path?
>
> Mainly to avoid issues with packages that might place files in the
> configuration directory, which might hinder the reproduction of bugs.

Hmm, maybe, I don't have an example in mind though.

> Upon reflection, my approach might have an issue if the user wishes to
> install a package, for the sake of testing within the isolated
> environment.

Yes, one more reason to use the original elpa dir ;-)

> Perhaps it would be better to set `package-directory-list' instead?

Don't know yet.

> Also, would it make sense to have some visual/textual indication that
> this is a testing environment?  Perhaps the *scratch* buffer string
> could be modified or the default background colour could be set to
> something else.

No particular opinions about this.

>>> This is easy to fix though, we just need to specify `package-user-dir'
>>> at startup.  Here is my updated patch, merging our approaches:
>>>
>>> [2. text/x-diff;
>>> 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...
>>>
>>>
>>>>> [2. text/x-diff;
>>>>> 0002-Add-command-to-start-Emacs-with-specific-packages.patch]...
>>>>>
>>>>> [3. text/x-diff; 0001-Add-a-function-to-query-the-Emacs-executable.patch]...
>>>
>>> I have slightly modified your version, fixing issues I had, in case
>>> anyone else wants to try it out:
>>
>> Thanks, sorry to not provide the CRM, I quickly wrote this.
>
> np.
>
>>>         (defun package-isolate (packages)
>>>           "Start an uncustomised Emacs and only load a set of PACKAGES."
>>>           (interactive
>>>            (list (mapcar #'intern (completing-read-multiple "Packages: " (mapcar #'car (package--alist))))))
>>>           (let* ((name (concat "package-isolate-" (mapconcat #'symbol-name packages ",")))
>>>                  (deps (mapcan #'package--dependencies packages)))
>>>             (apply #'start-process (concat "*" name "*") nil
>>>                    (list (file-truename (expand-file-name invocation-name invocation-directory))
>>>                          "--quick" "--debug-init"
>>>                          (format "--eval=%S"
>>>                                  `(progn
>>>                                     (require 'warnings)
>>>                                     (add-to-list 'warning-suppress-log-types 'initialization)
>>>                                     (require 'package)
>>>                                     (setq package-load-list
>>>                                           ',(mapcar (lambda (p) (list p t)) (append packages deps)))
>>>                                     (package-initialize)))))))


-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Changes to make in elpa-packages file for nongnu elpa

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Cc: emacs-devel@gnu.org
> Date: Wed, 16 Aug 2023 10:10:46 +0000
> 
> +---
> +*** New command to start Emacs only with specific packages
> +The command 'package-isolate' is equivalent to starting Emacs with the
> +-Q flag and loading specific packages (and their dependencies)
> +manually.

If this is how this feature is going to be implemented, I suggest to
reword this NEWS entry to make the intent and the feature easier to
understand:

  *** New command to start an inferior Emacs loading only specific packages.
  The new command 'package-isolate' will start a new Emacs process, as
  a sub-process of Emacs where you invoke the command, in a way that
  causes the new process to load only some of the installed packages.
  The command prompts for the packages to activate in this
  sub-process, and is intended for testing Emacs and/or the packages
  in a clean environment.

Proposal for 'package-isolate' command

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

>>>> I don't know much about Helm, but does it not support CRM?
>>>
>>> It does, but this is a much better interface than CRM.
>>
>> I don't think it makes sense to add support like this in the core,
>
> Perhaps yes. However it doesn't requires helm:
>
>      (let ((helm-comp-read-use-marked t))
>         (if (and (boundp 'helm-mode) helm-mode)
>             (completing-read ...)
>           (completing-read-multiple ...)
>
>> but is there something we should keep in mind to not hinder
>> integration with Helm?
>
> No don't worry, helm will work in any cases, thanks.

OK, in that case I'd prefer to keep it the way it is.


[...]

>>>>>       (let* ((name (concat "package-isolate-" (mapconcat #'identity
>>>>>                                                          packages ",")))
>>>>
>>>> This doesn't work, because the above returns a string, not a list of strings.
>>>
>>> No, it works, the above returns a list of strings (the completing-read
>>> allows marking and returns always a list).
>>
>> Not in my case, I got a type error.
>
> Yes, because the let-bounded var had no effect in your case, otherwise
> when helm is installed and helm-mode enabled the completing-read will
> always return a list.

Right, but that is the issue here, because using completing-read,
especially in the core, shouldn't make any assumptions like that.  In
the end this boils down to the fact that completing-read and co. are a
over-burned interface...

[...]

>>>> It seems to me, that for a proper isolated environment, we should add
>>>> a --init-directory option.
>>>
>>> Why as long as other directories in elpa are not in load-path?
>>
>> Mainly to avoid issues with packages that might place files in the
>> configuration directory, which might hinder the reproduction of bugs.
>
> Hmm, maybe, I don't have an example in mind though.

It might not be the best example, but my package autocrypt generates a
file in the `user-emacs-directory'.  If we don't use --init-directory,
the existing file would be re-used.

>> Upon reflection, my approach might have an issue if the user wishes to
>> install a package, for the sake of testing within the isolated
>> environment.
>
> Yes, one more reason to use the original elpa dir ;-)

No, because in that case the package installed in the temporary
environment would be retained in the original elpa/ directory, which I
think we /don't/ want.

>> Perhaps it would be better to set `package-directory-list' instead?
>
> Don't know yet.

Even better, I have added all directories that
`package-load-all-descriptors' would load to `package-directory-list',
which should be more robust.

>> Also, would it make sense to have some visual/textual indication that
>> this is a testing environment?  Perhaps the *scratch* buffer string
>> could be modified or the default background colour could be set to
>> something else.
>
> No particular opinions about this.

I guess this is too controversial.  Perhaps just adjusting
`frame-title-format' would be enough.


[...]

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: thievol@posteo.net,  emacs-devel@gnu.org
>> Date: Tue, 15 Aug 2023 19:52:32 +0000
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Seems strange to me to have a command to start another Emacs.  Why not
>> > implement this as a command-line option instead?  That would be
>> > consistent with the several options we already have, like -q, -Q, -D,
>> > which already contrl what happens at startup.
>> 
>> Mainly because this wouldn't support a completing-read interface, that
>> simplifies prompting the user for a set of packages.
>
> Shells can complete as well, right?  And once you typed the command
> once, shells have features to help you invoke commands from history,
> so you won't need to retype it more than once.

Each shell requires individual completion support, and figuring out what
packages are installed outside of Emacs to ensure input like
"magit-3.3.0,corfu-0.37" is well formed, seems too much work to me.

What would be reasonable, might be a function that could be invoked from
batch mode.

  emacs -Q --batch -f package-isolate-batch magit corfu

>> But perhaps the bulk of this command could be implemented as a CLI
>> option, that this command could wrap?
>
> Maybe.  But again, it's strange to have to launch another process from
> Emacs for such a simple functionality.  And if the number of packages
> to load is large, typing them manually would be annoying anyway, so
> people will probably store the list in a file or something?  So maybe
> we should allow specifying a file with the list of packages as well?

I don't know how this package would be used, but the way I imagine it is
that a package maintainer asks the user to run

  M-x package-isolate RET foo-1.2.3 RET

with only a few packages, to make the reproduction of bugs easier, as
is the case with Helm, which is what prompted this tangent.

>> >> +          (make-symbolic-link real link t)
>> >
>> > Using symbolic links makes the program less portable, so it is best to
>> > avoid them.
>> 
>> The reason I used them here, instead of just adding the directories
>> under ~/.config/emacs/elpa/ is that startup.el issues a warning along
>> the lines of
>> 
>>   Your `load-path' seems to contain your `user-emacs-directory' ...
>> 
>> My previous patch actually included the code that suppresses all
>> warnings during initialisation, so I've removed this hack.
>
> I don't understand how this is different from a "normal" startup of
> Emacs with all the packages.  Here, you just want to load part of the
> packages that are loaded normally, so why does this warning pop up in
> this case, but not when Emacs starts "normally"?  I'm probably missing
> something.

The point is moot anyway, since the approach that Thierry proposed
circumvents the issue to begin with.

Re: Proposal for 'package-isolate' command

[Stefan Kangas]

Philip Kaludercic <philipk@posteo.net> writes:

> I guess this is too controversial.  Perhaps just adjusting
> `frame-title-format' would be enough.

Would it be controversial to print a message in the scratch buffer?

Re: Changes to make in elpa-packages file for nongnu elpa

[Philip Kaludercic]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Philip Kaludercic <philipk@posteo.net>
>> Cc: emacs-devel@gnu.org
>> Date: Wed, 16 Aug 2023 10:10:46 +0000
>> 
>> +---
>> +*** New command to start Emacs only with specific packages
>> +The command 'package-isolate' is equivalent to starting Emacs with the
>> +-Q flag and loading specific packages (and their dependencies)
>> +manually.
>
> If this is how this feature is going to be implemented, I suggest to
> reword this NEWS entry to make the intent and the feature easier to
> understand:
>
>   *** New command to start an inferior Emacs loading only specific packages.
>   The new command 'package-isolate' will start a new Emacs process, as
>   a sub-process of Emacs where you invoke the command, in a way that
>   causes the new process to load only some of the installed packages.
>   The command prompts for the packages to activate in this
>   sub-process, and is intended for testing Emacs and/or the packages
>   in a clean environment.

As usual, I prefer your phrasing.  Will update the patch.

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 339 bytes --]

Stefan Kangas <stefankangas@gmail.com> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> I guess this is too controversial.  Perhaps just adjusting
>> `frame-title-format' would be enough.
>
> Would it be controversial to print a message in the scratch buffer?

That is also a good idea, this patch integrates that approach:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-command-to-start-Emacs-with-specific-packages.patch --]
[-- Type: text/x-diff, Size: 5673 bytes --]

From ecc6b37a2a1cb9b3629521fc92b2d9af5c5ea653 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Tue, 15 Aug 2023 18:39:14 +0200
Subject: [PATCH] Add command to start Emacs with specific packages

* lisp/emacs-lisp/package.el (package--dependencies): Extend function
to handle and return package descriptors.
(package-isolate): Add new command.
* etc/NEWS: Announce new command.
---
 etc/NEWS                   |  9 +++++
 lisp/emacs-lisp/package.el | 70 ++++++++++++++++++++++++++++++++++----
 2 files changed, 73 insertions(+), 6 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 57f04609679..dc213dcee42 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -505,6 +505,15 @@ project, that you can quickly select using 'project-switch-project'
 When non-nil, package specifications with side-effects for building
 software will be used when building a package.
 
+---
+*** New command to start an inferior Emacs loading only specific packages.
+The new command 'package-isolate' will start a new Emacs process, as
+a sub-process of Emacs where you invoke the command, in a way that
+causes the new process to load only some of the installed packages.
+The command prompts for the packages to activate in this
+sub-process, and is intended for testing Emacs and/or the packages
+in a clean environment.
+
 ** Flymake
 
 +++
diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index b3062d2608b..76cd39cc263 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2330,12 +2330,26 @@ package-upgrade-all
       (mapc #'package-upgrade upgradeable))))
 
 (defun package--dependencies (pkg)
-  "Return a list of all dependencies PKG has.
-This is done recursively."
-  ;; Can we have circular dependencies?  Assume "nope".
-  (when-let* ((desc (cadr (assq pkg package-archive-contents)))
-              (deps (mapcar #'car (package-desc-reqs desc))))
-    (delete-dups (apply #'nconc deps (mapcar #'package--dependencies deps)))))
+  "Return a list of all recursive dependencies of PKG.
+If PKG is a package descriptor, the return value will consist of
+a list of package descriptors.  If PKG is a symbol, designating a
+package, the return value will be a list of symbols designating
+packages."
+  (when-let* ((desc (if (package-desc-p pkg) pkg
+                      (cadr (assq pkg package-archive-contents)))))
+    ;; Can we have circular dependencies?  Assume "nope".
+    (let ((all (named-let more ((pkg-desc desc))
+                 (let (deps)
+                   (dolist (req (package-desc-reqs pkg-desc))
+                     (setq deps (nconc
+                                 (catch 'found
+                                   (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
+                                     (when (and (string= (car req) (package-desc-name p))
+                                                (version-list-<= (cadr req) (package-desc-version p)))
+                                       (throw 'found (more p)))))
+                                 deps)))
+                   (delete-dups (cons pkg-desc deps))))))
+      (remq pkg (mapcar (if (package-desc-p pkg) #'identity #'package-desc-name) all)))))
 
 (defun package-strip-rcs-id (str)
   "Strip RCS version ID from the version string STR.
@@ -2625,6 +2639,50 @@ package-autoremove
                   removable))
         (message "Nothing to autoremove")))))
 
+(defun package-isolate (packages)
+  "Start an uncustomised Emacs and only load a set of PACKAGES."
+  (interactive
+   (cl-loop for p in (cl-loop for p in (package--alist) append (cdr p))
+	    unless (package-built-in-p p)
+	    collect (cons (package-desc-full-name p) p) into table
+	    finally return
+	    (list (cl-loop for c in (completing-read-multiple
+                                     "Isolate packages: " table
+                                     nil t)
+		           collect (alist-get c table nil nil #'string=)))))
+  (let* ((name (concat "package-isolate-"
+                       (mapconcat #'package-desc-full-name packages ",")))
+         initial-scratch-message package-load-list)
+    (with-temp-buffer
+      (insert ";; Emacs has been started by `package-isolate' without any configuration, and\n"
+              ";; only the following packages have been loaded:\n;;\n")
+
+      (dolist (package (append packages (mapcan #'package--dependencies packages)))
+        (push (list (package-desc-name package)
+                    (package-version-join (package-desc-version package)))
+              package-load-list)
+        (insert ";; - " (package-desc-full-name package))
+        (unless (memq package packages)
+          (insert " (dependency)"))
+        (insert "\n"))
+      (insert "\n")
+      (setq initial-scratch-message (buffer-string)))
+    (apply #'start-process (concat "*" name "*") nil
+           (list (expand-file-name invocation-name invocation-directory)
+                 "--quick" "--debug-init"
+                 "--init-directory" (make-temp-file name t)
+                 (format "--eval=%S"
+                         `(progn
+                            (setq initial-scratch-message ,initial-scratch-message)
+
+                            (require 'package)
+                            ,@(mapcar
+                               (lambda (dir)
+                                 `(add-to-list 'package-directory-list ,dir))
+                               (cons package-user-dir package-directory-list))
+                            (setq package-load-list ',package-load-list)
+                            (package-initialize)))))))
+
 \f
 ;;;; Package description buffer.
 
-- 
2.39.2

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 1092 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

>> No don't worry, helm will work in any cases, thanks.
>
> OK, in that case I'd prefer to keep it the way it is.

Ok, fair enough.

>>> Mainly to avoid issues with packages that might place files in the
>>> configuration directory, which might hinder the reproduction of bugs.
>>
>> Hmm, maybe, I don't have an example in mind though.
>
> It might not be the best example, but my package autocrypt generates a
> file in the `user-emacs-directory'.  If we don't use --init-directory,
> the existing file would be re-used.

Ok, for such case it is easy to either rename the file temporarily to
foo_ori or foo_save or to set the variable handling the file (generally
it is not harcoded) to something else. But another issue is if your
isolated package needs the history file or tramp file or whatever file
~/.emacs.d is handling; e.g. a package providing completion on history.
I still think it is more handy to reuse user-emacs-directory and its
elpa directory where the packages are already installed.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>>> No don't worry, helm will work in any cases, thanks.
>>
>> OK, in that case I'd prefer to keep it the way it is.
>
> Ok, fair enough.
>
>>>> Mainly to avoid issues with packages that might place files in the
>>>> configuration directory, which might hinder the reproduction of bugs.
>>>
>>> Hmm, maybe, I don't have an example in mind though.
>>
>> It might not be the best example, but my package autocrypt generates a
>> file in the `user-emacs-directory'.  If we don't use --init-directory,
>> the existing file would be re-used.
>
> Ok, for such case it is easy to either rename the file temporarily to
> foo_ori or foo_save or to set the variable handling the file (generally
> it is not harcoded) to something else. But another issue is if your
> isolated package needs the history file or tramp file or whatever file
> ~/.emacs.d is handling; e.g. a package providing completion on history.
> I still think it is more handy to reuse user-emacs-directory and its
> elpa directory where the packages are already installed.

How about this patch, that will use a temporary directory when
`package-isolate' is invoked with a prefix argument (not sure what the
default should be, I guess reusing `user-emacs-directory' is less
surprising):


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-command-to-start-Emacs-with-specific-packages.patch --]
[-- Type: text/x-diff, Size: 6128 bytes --]

From dd5da3d2f64f89994084012a3b7379aa78ae33f2 Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Tue, 15 Aug 2023 18:39:14 +0200
Subject: [PATCH] Add command to start Emacs with specific packages

* lisp/emacs-lisp/package.el (package--dependencies): Extend function
to handle and return package descriptors.
(package-isolate): Add new command.
* etc/NEWS: Announce new command.
---
 etc/NEWS                   |  9 +++++
 lisp/emacs-lisp/package.el | 77 +++++++++++++++++++++++++++++++++++---
 2 files changed, 80 insertions(+), 6 deletions(-)

diff --git a/etc/NEWS b/etc/NEWS
index 57f04609679..dc213dcee42 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -505,6 +505,15 @@ project, that you can quickly select using 'project-switch-project'
 When non-nil, package specifications with side-effects for building
 software will be used when building a package.
 
+---
+*** New command to start an inferior Emacs loading only specific packages.
+The new command 'package-isolate' will start a new Emacs process, as
+a sub-process of Emacs where you invoke the command, in a way that
+causes the new process to load only some of the installed packages.
+The command prompts for the packages to activate in this
+sub-process, and is intended for testing Emacs and/or the packages
+in a clean environment.
+
 ** Flymake
 
 +++
diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index b3062d2608b..169d0d1360d 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2330,12 +2330,26 @@ package-upgrade-all
       (mapc #'package-upgrade upgradeable))))
 
 (defun package--dependencies (pkg)
-  "Return a list of all dependencies PKG has.
-This is done recursively."
-  ;; Can we have circular dependencies?  Assume "nope".
-  (when-let* ((desc (cadr (assq pkg package-archive-contents)))
-              (deps (mapcar #'car (package-desc-reqs desc))))
-    (delete-dups (apply #'nconc deps (mapcar #'package--dependencies deps)))))
+  "Return a list of all recursive dependencies of PKG.
+If PKG is a package descriptor, the return value will consist of
+a list of package descriptors.  If PKG is a symbol, designating a
+package, the return value will be a list of symbols designating
+packages."
+  (when-let* ((desc (if (package-desc-p pkg) pkg
+                      (cadr (assq pkg package-archive-contents)))))
+    ;; Can we have circular dependencies?  Assume "nope".
+    (let ((all (named-let more ((pkg-desc desc))
+                 (let (deps)
+                   (dolist (req (package-desc-reqs pkg-desc))
+                     (setq deps (nconc
+                                 (catch 'found
+                                   (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
+                                     (when (and (string= (car req) (package-desc-name p))
+                                                (version-list-<= (cadr req) (package-desc-version p)))
+                                       (throw 'found (more p)))))
+                                 deps)))
+                   (delete-dups (cons pkg-desc deps))))))
+      (remq pkg (mapcar (if (package-desc-p pkg) #'identity #'package-desc-name) all)))))
 
 (defun package-strip-rcs-id (str)
   "Strip RCS version ID from the version string STR.
@@ -2625,6 +2639,57 @@ package-autoremove
                   removable))
         (message "Nothing to autoremove")))))
 
+(defun package-isolate (packages &optional temp-init)
+  "Start an uncustomised Emacs and only load a set of PACKAGES.
+If TEMP-INIT is non-nil, or when invoked with a prefix argument,
+the Emacs user directory is set to a temporary directory."
+  (interactive
+   (cl-loop for p in (cl-loop for p in (package--alist) append (cdr p))
+	    unless (package-built-in-p p)
+	    collect (cons (package-desc-full-name p) p) into table
+	    finally return
+	    (list (cl-loop for c in (completing-read-multiple
+                                     "Isolate packages: " table
+                                     nil t)
+		           collect (alist-get c table nil nil #'string=))
+                  current-prefix-arg)))
+  (let* ((name (concat "package-isolate-"
+                       (mapconcat #'package-desc-full-name packages ",")))
+         (all-packages (delete-consecutive-dups
+                        (sort (append packages (mapcan #'package--dependencies packages))
+                              (lambda (p0 p1)
+                                (string< (package-desc-name p0) (package-desc-name p1))))))
+         initial-scratch-message package-load-list)
+    (with-temp-buffer
+      (insert ";; This is an isolated testing environment, with these packages enabled:\n\n")
+      (dolist (package all-packages)
+        (push (list (package-desc-name package)
+                    (package-version-join (package-desc-version package)))
+              package-load-list)
+        (insert ";; - " (package-desc-full-name package))
+        (unless (memq package packages)
+          (insert " (dependency)"))
+        (insert "\n"))
+      (insert "\n")
+      (setq initial-scratch-message (buffer-string)))
+    (apply #'start-process (concat "*" name "*") nil
+           (list (expand-file-name invocation-name invocation-directory)
+                 "--quick" "--debug-init"
+                 "--init-directory" (if temp-init
+                                        (make-temp-file name t)
+                                      user-emacs-directory)
+                 (format "--eval=%S"
+                         `(progn
+                            (setq initial-scratch-message ,initial-scratch-message)
+
+                            (require 'package)
+                            ,@(mapcar
+                               (lambda (dir)
+                                 `(add-to-list 'package-directory-list ,dir))
+                               (cons package-user-dir package-directory-list))
+                            (setq package-load-list ',package-load-list)
+                            (package-initialize)))))))
+
 \f
 ;;;; Package description buffer.
 
-- 
2.39.2

Re: Proposal for 'package-isolate' command

[Eshel Yaron]

Hi Philip,

Philip Kaludercic <philipk@posteo.net> writes:

> How about this patch, that will use a temporary directory when
> `package-isolate' is invoked with a prefix argument (not sure what the
> default should be, I guess reusing `user-emacs-directory' is less
> surprising):

I'm again looking forward to trying this out, I think it could be
helpful for testing the Compat integration I'm working on for a package
of mine.  I also again have a couple of minor suggestions around
documentation:

>  (defun package--dependencies (pkg)
> -  "Return a list of all dependencies PKG has.
> -This is done recursively."
> -  ;; Can we have circular dependencies?  Assume "nope".
> -  (when-let* ((desc (cadr (assq pkg package-archive-contents)))
> -              (deps (mapcar #'car (package-desc-reqs desc))))
> -    (delete-dups (apply #'nconc deps (mapcar #'package--dependencies deps)))))
> +  "Return a list of all recursive dependencies of PKG.

I'd say "transitive" rather than "recursive" dependencies, because I
think a recursive dependency is more commonly used to refer to a case in
which a package (or library, etc.) depends on itself.

> +If PKG is a package descriptor, the return value will consist of
> +a list of package descriptors.  If PKG is a symbol, designating a
> +package, the return value will be a list of symbols designating
> +packages."

Personally, I find the future tense less clear than the present tense
when describing deterministic software behavior.  So usually I prefer it
when the docs say what the program "does", not what it "will do".  In
this case that be "if PKG is a package descriptor, the return value is a
list of package descriptors..."


Best,

Eshel

Re: Proposal for 'package-isolate' command

[Adam Porter]

FWIW, I've been using this script of mine, with-emacs.sh, to do the 
equivalent for nearly four years now:

https://github.com/alphapapa/with-emacs.sh

It works on older Emacs versions from before the "--init-directory" 
option was added, as well as Emacs 29.1.

For example:

   with-emacs.sh --install org-ql

That starts Emacs in a clean, temporary config directory (which is 
deleted after exit), runs package-refresh-contents, and installs org-ql 
and its dependencies.

Or if you want to run in a specific config directory, which will not be 
deleted on exit, use the option, e.g.

   with-emacs.sh -d ~/src/emacs/configs/ap.el

It sets the frame title accordingly, so multiple Emacs sessions in 
multiple config directories can be run simultaneously and distinguished:

https://github.com/alphapapa/with-emacs.sh/blob/0bc4f216ed101d86d2e5d52919bad39bc041bdbe/with-emacs.sh#L187-L188

I've found it invaluable for debugging problems that users of my 
packages report.

It's mostly implemented in Bash, with some embedded Elisp.  It could be 
written entirely as an Elisp library, if desired.

Anyway, feel free to draw any inspiration or code from this script if it 
helps.

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 2660 bytes --]


Hello Philip,

Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>>> No don't worry, helm will work in any cases, thanks.
>>>
>>> OK, in that case I'd prefer to keep it the way it is.
>>
>> Ok, fair enough.
>>
>>>>> Mainly to avoid issues with packages that might place files in the
>>>>> configuration directory, which might hinder the reproduction of bugs.
>>>>
>>>> Hmm, maybe, I don't have an example in mind though.
>>>
>>> It might not be the best example, but my package autocrypt generates a
>>> file in the `user-emacs-directory'.  If we don't use --init-directory,
>>> the existing file would be re-used.
>>
>> Ok, for such case it is easy to either rename the file temporarily to
>> foo_ori or foo_save or to set the variable handling the file (generally
>> it is not harcoded) to something else. But another issue is if your
>> isolated package needs the history file or tramp file or whatever file
>> ~/.emacs.d is handling; e.g. a package providing completion on history.
>> I still think it is more handy to reuse user-emacs-directory and its
>> elpa directory where the packages are already installed.
>
> How about this patch, that will use a temporary directory when
> `package-isolate' is invoked with a prefix argument (not sure what the
> default should be, I guess reusing `user-emacs-directory' is less
> surprising):

Yes.

Isn't these two loop the same? (save one loop)

    (cl-loop for p in (cl-loop for p in (package--alist) append (cdr p))
    	 unless (package-built-in-p p)
    	 collect (cons (package-desc-full-name p) p) into table
    	 finally return
    	 (list (cl-loop for c in (completing-read-multiple
                                      "Isolate packages: " table
                                      nil t)
    		        collect (alist-get c table nil nil #'string=))
                   current-prefix-arg))

    (cl-loop for p in (package--alist)
             for desc = (cadr p)
             for name = (car p)
             unless (package-built-in-p desc)
             collect (cons (package-desc-full-name desc) p) into table
             finally return
             (list (cl-loop for c in (completing-read-multiple
                                      "Isolate packages: " table
                                      nil t)
    		            append (cdr (alist-get c table nil nil #'string=)))
                   current-prefix-arg))


> [2. text/x-diff; 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...


-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 3002 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>>> No don't worry, helm will work in any cases, thanks.
>>>
>>> OK, in that case I'd prefer to keep it the way it is.
>>
>> Ok, fair enough.
>>
>>>>> Mainly to avoid issues with packages that might place files in the
>>>>> configuration directory, which might hinder the reproduction of bugs.
>>>>
>>>> Hmm, maybe, I don't have an example in mind though.
>>>
>>> It might not be the best example, but my package autocrypt generates a
>>> file in the `user-emacs-directory'.  If we don't use --init-directory,
>>> the existing file would be re-used.
>>
>> Ok, for such case it is easy to either rename the file temporarily to
>> foo_ori or foo_save or to set the variable handling the file (generally
>> it is not harcoded) to something else. But another issue is if your
>> isolated package needs the history file or tramp file or whatever file
>> ~/.emacs.d is handling; e.g. a package providing completion on history.
>> I still think it is more handy to reuse user-emacs-directory and its
>> elpa directory where the packages are already installed.
>
> How about this patch, that will use a temporary directory when
> `package-isolate' is invoked with a prefix argument (not sure what the
> default should be, I guess reusing `user-emacs-directory' is less
> surprising):

However it is not working properly:

Tried to isolate two packages, osm and w3m, and osm use compat as
dependency:

Unable to activate package ‘osm’.
Required package ‘compat-29.1.4.0’ is unavailable

With my simple version of package-isolate I have not this error, both
osm w3m are installed correctly.

(defun package-isolate (packages)
  "Start an uncustomised Emacs and only load a set of PACKAGES."
  (interactive
   (list (completing-read-multiple
          "Packages: " (mapcar #'car (package--alist)))))
  (let* ((name (concat "package-isolate-" (mapconcat #'identity packages ",")))
         (deps (cl-loop for p in packages
                        for sym = (intern p)
                        nconc (package--dependencies sym))))
    (apply #'start-process (concat "*" name "*") nil
           (list (file-truename (expand-file-name
                                 invocation-name invocation-directory))
                 "--quick" "--debug-init"
                 (format "--eval=%S"
                         `(progn
                           (require 'package)
                           (setq package-load-list
                            ',(append (mapcar (lambda (p) (list (intern p) t))
                                              packages)
                                      (mapcar (lambda (p) (list p t)) deps)))
                           (package-initialize)))))))

> [2. text/x-diff; 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...


-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> Hello Philip,
>
> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Thierry Volpiatto <thievol@posteo.net> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>>> No don't worry, helm will work in any cases, thanks.
>>>>
>>>> OK, in that case I'd prefer to keep it the way it is.
>>>
>>> Ok, fair enough.
>>>
>>>>>> Mainly to avoid issues with packages that might place files in the
>>>>>> configuration directory, which might hinder the reproduction of bugs.
>>>>>
>>>>> Hmm, maybe, I don't have an example in mind though.
>>>>
>>>> It might not be the best example, but my package autocrypt generates a
>>>> file in the `user-emacs-directory'.  If we don't use --init-directory,
>>>> the existing file would be re-used.
>>>
>>> Ok, for such case it is easy to either rename the file temporarily to
>>> foo_ori or foo_save or to set the variable handling the file (generally
>>> it is not harcoded) to something else. But another issue is if your
>>> isolated package needs the history file or tramp file or whatever file
>>> ~/.emacs.d is handling; e.g. a package providing completion on history.
>>> I still think it is more handy to reuse user-emacs-directory and its
>>> elpa directory where the packages are already installed.
>>
>> How about this patch, that will use a temporary directory when
>> `package-isolate' is invoked with a prefix argument (not sure what the
>> default should be, I guess reusing `user-emacs-directory' is less
>> surprising):
>
> Yes.
>
> Isn't these two loop the same? (save one loop)

No, because that assumes that we are only interested in the first
package descriptor in every package--alist entry, while there might be
multiple.

>     (cl-loop for p in (cl-loop for p in (package--alist) append (cdr p))
>     	 unless (package-built-in-p p)

Also, do we want this or wouldn't it be useful to be able to force a
built-in package being loaded?

>     	 collect (cons (package-desc-full-name p) p) into table
>     	 finally return
>     	 (list (cl-loop for c in (completing-read-multiple
>                                       "Isolate packages: " table
>                                       nil t)
>     		        collect (alist-get c table nil nil #'string=))
>                    current-prefix-arg))
>
>     (cl-loop for p in (package--alist)
>              for desc = (cadr p)
>              for name = (car p)
>              unless (package-built-in-p desc)
>              collect (cons (package-desc-full-name desc) p) into table
>              finally return
>              (list (cl-loop for c in (completing-read-multiple
>                                       "Isolate packages: " table
>                                       nil t)
>     		            append (cdr (alist-get c table nil nil #'string=)))
>                    current-prefix-arg))
>
>
>> [2. text/x-diff;
>> 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 2496 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Hello Philip,
>>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Thierry Volpiatto <thievol@posteo.net> writes:
>>>
>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>
>>>>>> No don't worry, helm will work in any cases, thanks.
>>>>>
>>>>> OK, in that case I'd prefer to keep it the way it is.
>>>>
>>>> Ok, fair enough.
>>>>
>>>>>>> Mainly to avoid issues with packages that might place files in the
>>>>>>> configuration directory, which might hinder the reproduction of bugs.
>>>>>>
>>>>>> Hmm, maybe, I don't have an example in mind though.
>>>>>
>>>>> It might not be the best example, but my package autocrypt generates a
>>>>> file in the `user-emacs-directory'.  If we don't use --init-directory,
>>>>> the existing file would be re-used.
>>>>
>>>> Ok, for such case it is easy to either rename the file temporarily to
>>>> foo_ori or foo_save or to set the variable handling the file (generally
>>>> it is not harcoded) to something else. But another issue is if your
>>>> isolated package needs the history file or tramp file or whatever file
>>>> ~/.emacs.d is handling; e.g. a package providing completion on history.
>>>> I still think it is more handy to reuse user-emacs-directory and its
>>>> elpa directory where the packages are already installed.
>>>
>>> How about this patch, that will use a temporary directory when
>>> `package-isolate' is invoked with a prefix argument (not sure what the
>>> default should be, I guess reusing `user-emacs-directory' is less
>>> surprising):
>>
>> Yes.
>>
>> Isn't these two loop the same? (save one loop)
>
> No, because that assumes that we are only interested in the first
> package descriptor in every package--alist entry, while there might be
> multiple.

Ah ok.

>>     (cl-loop for p in (cl-loop for p in (package--alist) append (cdr p))
>>     	 unless (package-built-in-p p)
>
> Also, do we want this or wouldn't it be useful to be able to force a
> built-in package being loaded?

Some packages seems to require a specific version of a package for their
dependency e.g. seq, by excluding it the package may not work correctly,
this is my understanding but I may be wrong. Also perhaps the package
e.g. seq is selected later when computing dependencies but maybe user
wants to select a particular version manually in the first place?

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Thierry Volpiatto <thievol@posteo.net> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>>> No don't worry, helm will work in any cases, thanks.
>>>>
>>>> OK, in that case I'd prefer to keep it the way it is.
>>>
>>> Ok, fair enough.
>>>
>>>>>> Mainly to avoid issues with packages that might place files in the
>>>>>> configuration directory, which might hinder the reproduction of bugs.
>>>>>
>>>>> Hmm, maybe, I don't have an example in mind though.
>>>>
>>>> It might not be the best example, but my package autocrypt generates a
>>>> file in the `user-emacs-directory'.  If we don't use --init-directory,
>>>> the existing file would be re-used.
>>>
>>> Ok, for such case it is easy to either rename the file temporarily to
>>> foo_ori or foo_save or to set the variable handling the file (generally
>>> it is not harcoded) to something else. But another issue is if your
>>> isolated package needs the history file or tramp file or whatever file
>>> ~/.emacs.d is handling; e.g. a package providing completion on history.
>>> I still think it is more handy to reuse user-emacs-directory and its
>>> elpa directory where the packages are already installed.
>>
>> How about this patch, that will use a temporary directory when
>> `package-isolate' is invoked with a prefix argument (not sure what the
>> default should be, I guess reusing `user-emacs-directory' is less
>> surprising):
>
> However it is not working properly:
>
> Tried to isolate two packages, osm and w3m, and osm use compat as
> dependency:
>
> Unable to activate package ‘osm’.
> Required package ‘compat-29.1.4.0’ is unavailable

Sadly I cannot reproduce it, w3m is not in GNU or NonGNU ELPA, so unless
that is causing the issue, something like M-x package-isolate RET
avy-0.5.0,osm-0.13 RET shouldn't make an issue when it comes to
resolving dependencies, as the scratch message indicates:

--8<---------------cut here---------------start------------->8---
;; This is an isolated testing environment, with these packages enabled:

;; - avy-0.5.0
;; - compat-29.1.4.2 (dependency)
;; - osm-0.13
--8<---------------cut here---------------end--------------->8---

And M-x load-library RET compat RET works as well.

> With my simple version of package-isolate I have not this error, both
> osm w3m are installed correctly.
>
> (defun package-isolate (packages)
>   "Start an uncustomised Emacs and only load a set of PACKAGES."
>   (interactive
>    (list (completing-read-multiple
>           "Packages: " (mapcar #'car (package--alist)))))
>   (let* ((name (concat "package-isolate-" (mapconcat #'identity packages ",")))
>          (deps (cl-loop for p in packages
>                         for sym = (intern p)
>                         nconc (package--dependencies sym))))
>     (apply #'start-process (concat "*" name "*") nil
>            (list (file-truename (expand-file-name
>                                  invocation-name invocation-directory))
>                  "--quick" "--debug-init"
>                  (format "--eval=%S"
>                          `(progn
>                            (require 'package)
>                            (setq package-load-list
>                             ',(append (mapcar (lambda (p) (list (intern p) t))
>                                               packages)
>                                       (mapcar (lambda (p) (list p t)) deps)))
>                            (package-initialize)))))))
>
>> [2. text/x-diff; 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

Eshel Yaron <me@eshelyaron.com> writes:

> Hi Philip,
>
> Philip Kaludercic <philipk@posteo.net> writes:
>
>> How about this patch, that will use a temporary directory when
>> `package-isolate' is invoked with a prefix argument (not sure what the
>> default should be, I guess reusing `user-emacs-directory' is less
>> surprising):
>
> I'm again looking forward to trying this out, I think it could be
> helpful for testing the Compat integration I'm working on for a package
> of mine.  I also again have a couple of minor suggestions around
> documentation:
>
>>  (defun package--dependencies (pkg)
>> -  "Return a list of all dependencies PKG has.
>> -This is done recursively."
>> -  ;; Can we have circular dependencies?  Assume "nope".
>> -  (when-let* ((desc (cadr (assq pkg package-archive-contents)))
>> -              (deps (mapcar #'car (package-desc-reqs desc))))
>> -    (delete-dups (apply #'nconc deps (mapcar #'package--dependencies deps)))))
>> +  "Return a list of all recursive dependencies of PKG.
>
> I'd say "transitive" rather than "recursive" dependencies, because I
> think a recursive dependency is more commonly used to refer to a case in
> which a package (or library, etc.) depends on itself.

I wouldn't say recursive but cyclical in that case, but either way I
think transitive is a better word here.  Will change it.

>> +If PKG is a package descriptor, the return value will consist of
>> +a list of package descriptors.  If PKG is a symbol, designating a
>> +package, the return value will be a list of symbols designating
>> +packages."
>
> Personally, I find the future tense less clear than the present tense
> when describing deterministic software behavior.  So usually I prefer it
> when the docs say what the program "does", not what it "will do".  In
> this case that be "if PKG is a package descriptor, the return value is a
> list of package descriptors..."

Right, makes sense as well.

> Best,
>
> Eshel

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 1915 bytes --]


Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Thierry Volpiatto <thievol@posteo.net> writes:
>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>>> No don't worry, helm will work in any cases, thanks.
>>>>
>>>> OK, in that case I'd prefer to keep it the way it is.
>>>
>>> Ok, fair enough.
>>>
>>>>>> Mainly to avoid issues with packages that might place files in the
>>>>>> configuration directory, which might hinder the reproduction of bugs.
>>>>>
>>>>> Hmm, maybe, I don't have an example in mind though.
>>>>
>>>> It might not be the best example, but my package autocrypt generates a
>>>> file in the `user-emacs-directory'.  If we don't use --init-directory,
>>>> the existing file would be re-used.
>>>
>>> Ok, for such case it is easy to either rename the file temporarily to
>>> foo_ori or foo_save or to set the variable handling the file (generally
>>> it is not harcoded) to something else. But another issue is if your
>>> isolated package needs the history file or tramp file or whatever file
>>> ~/.emacs.d is handling; e.g. a package providing completion on history.
>>> I still think it is more handy to reuse user-emacs-directory and its
>>> elpa directory where the packages are already installed.
>>
>> How about this patch, that will use a temporary directory when
>> `package-isolate' is invoked with a prefix argument (not sure what the
>> default should be, I guess reusing `user-emacs-directory' is less
>> surprising):
>
> However it is not working properly:
>
> Tried to isolate two packages, osm and w3m, and osm use compat as
> dependency:
>
> Unable to activate package ‘osm’.
> Required package ‘compat-29.1.4.0’ is unavailable

Forget it, it is working properly, just forget you had modified
package--dependencies as well.
Sorry for the noise.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> Thierry Volpiatto <thievol@posteo.net> writes:
>>>
>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>
>>>>>> No don't worry, helm will work in any cases, thanks.
>>>>>
>>>>> OK, in that case I'd prefer to keep it the way it is.
>>>>
>>>> Ok, fair enough.
>>>>
>>>>>>> Mainly to avoid issues with packages that might place files in the
>>>>>>> configuration directory, which might hinder the reproduction of bugs.
>>>>>>
>>>>>> Hmm, maybe, I don't have an example in mind though.
>>>>>
>>>>> It might not be the best example, but my package autocrypt generates a
>>>>> file in the `user-emacs-directory'.  If we don't use --init-directory,
>>>>> the existing file would be re-used.
>>>>
>>>> Ok, for such case it is easy to either rename the file temporarily to
>>>> foo_ori or foo_save or to set the variable handling the file (generally
>>>> it is not harcoded) to something else. But another issue is if your
>>>> isolated package needs the history file or tramp file or whatever file
>>>> ~/.emacs.d is handling; e.g. a package providing completion on history.
>>>> I still think it is more handy to reuse user-emacs-directory and its
>>>> elpa directory where the packages are already installed.
>>>
>>> How about this patch, that will use a temporary directory when
>>> `package-isolate' is invoked with a prefix argument (not sure what the
>>> default should be, I guess reusing `user-emacs-directory' is less
>>> surprising):
>>
>> However it is not working properly:
>>
>> Tried to isolate two packages, osm and w3m, and osm use compat as
>> dependency:
>>
>> Unable to activate package ‘osm’.
>> Required package ‘compat-29.1.4.0’ is unavailable
>
> Forget it, it is working properly, just forget you had modified
> package--dependencies as well.
> Sorry for the noise.

It was either that or a new function had to be added, not sure which
approach is worse.  The current implementation seems to have been
hastily added by Lars last year, and it is a bit regrettable in
retrospect.  It would be possible to modify my change, and have the
function always return package-desc objects, since the function is only
used in one other spot in another part of the file.  While there might
be others (packages or individuals) that depend on the function behaving
the way it does, but on the other hand, convention designates it as
being an "internal" function.

Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Thierry Volpiatto <thievol@posteo.net> writes:
>>
>>> Hello Philip,
>>>
>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>
>>>> Thierry Volpiatto <thievol@posteo.net> writes:
>>>>
>>>>> Philip Kaludercic <philipk@posteo.net> writes:
>>>>>
>>>>>>> No don't worry, helm will work in any cases, thanks.
>>>>>>
>>>>>> OK, in that case I'd prefer to keep it the way it is.
>>>>>
>>>>> Ok, fair enough.
>>>>>
>>>>>>>> Mainly to avoid issues with packages that might place files in the
>>>>>>>> configuration directory, which might hinder the reproduction of bugs.
>>>>>>>
>>>>>>> Hmm, maybe, I don't have an example in mind though.
>>>>>>
>>>>>> It might not be the best example, but my package autocrypt generates a
>>>>>> file in the `user-emacs-directory'.  If we don't use --init-directory,
>>>>>> the existing file would be re-used.
>>>>>
>>>>> Ok, for such case it is easy to either rename the file temporarily to
>>>>> foo_ori or foo_save or to set the variable handling the file (generally
>>>>> it is not harcoded) to something else. But another issue is if your
>>>>> isolated package needs the history file or tramp file or whatever file
>>>>> ~/.emacs.d is handling; e.g. a package providing completion on history.
>>>>> I still think it is more handy to reuse user-emacs-directory and its
>>>>> elpa directory where the packages are already installed.
>>>>
>>>> How about this patch, that will use a temporary directory when
>>>> `package-isolate' is invoked with a prefix argument (not sure what the
>>>> default should be, I guess reusing `user-emacs-directory' is less
>>>> surprising):
>>>
>>> Yes.
>>>
>>> Isn't these two loop the same? (save one loop)
>>
>> No, because that assumes that we are only interested in the first
>> package descriptor in every package--alist entry, while there might be
>> multiple.
>
> Ah ok.
>
>>>     (cl-loop for p in (cl-loop for p in (package--alist) append (cdr p))
>>>     	 unless (package-built-in-p p)
>>
>> Also, do we want this or wouldn't it be useful to be able to force a
>> built-in package being loaded?
>
> Some packages seems to require a specific version of a package for their
> dependency e.g. seq, by excluding it the package may not work correctly,
> this is my understanding but I may be wrong. Also perhaps the package
> e.g. seq is selected later when computing dependencies but maybe user
> wants to select a particular version manually in the first place?

The current algorithm should pick the first package in the package alist
that satisfies the necessary dependencies.  Perhaps that should be
re-thought or the selection should be more clever, e.g. if the user
explicitly specifies a dependency with a version, then it should be
preferred to whatever other dependency might be determined, at the
possible expense of triggering runtime bugs.

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 3077 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

>> Forget it, it is working properly, just forget you had modified
>> package--dependencies as well.
>> Sorry for the noise.
>
> It was either that or a new function had to be added, not sure which
> approach is worse.  The current implementation seems to have been
> hastily added by Lars last year, and it is a bit regrettable in
> retrospect.  It would be possible to modify my change, and have the
> function always return package-desc objects, since the function is only
> used in one other spot in another part of the file.  While there might
> be others (packages or individuals) that depend on the function behaving
> the way it does, but on the other hand, convention designates it as
> being an "internal" function.

The actual version is something like 4 or 5 lines long, so external
packages can inline this version under another name if really needed,
but your version is covering the both so it's ok I think.
OTOH The problem with package.el is inconsistency within its functions,
sometimes you have to use a pkg-desc as arg, sometimes not, sometimes
functions return a list of symbols sometimes a list of pkg-desc, what is
a package name, a string or a symbol? What is a pkg-desc exactly,
sometimes it is the cdr in other places the cadr is used...
Also built-in packages don't have the same format unless they are distributed
in Elpa etc...

>> Some packages seems to require a specific version of a package for their
>> dependency e.g. seq, by excluding it the package may not work correctly,
>> this is my understanding but I may be wrong. Also perhaps the package
>> e.g. seq is selected later when computing dependencies but maybe user
>> wants to select a particular version manually in the first place?
>
> The current algorithm should pick the first package in the package alist
> that satisfies the necessary dependencies.  Perhaps that should be
> re-thought or the selection should be more clever, e.g. if the user
> explicitly specifies a dependency with a version, then it should be
> preferred to whatever other dependency might be determined, at the
> possible expense of triggering runtime bugs.

I think it is hard to cover all cases, but after all it is more a
developer tool for debugging a particular package than a end user tool
to run packages, so perhaps the developer can give directives about
dependencies to use when needed.

Another thing, you use actually 

    (expand-file-name invocation-name invocation-directory)

I suggest you use the truename of this instead as emacs can be symlinked
in some installations.  It should work with the symlink name, but it is
clearer which emacs version is used.

Now your function is working fine and nearly finish, did you think how
you are going to distribute it? It seems you are going to merge it in
master, but what about providing it as well as a Elpa package so that
users of old emacs (at some point of course, say emacs-27) can use it to
report bugs?

Thanks.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Eli Zaretskii]

> From: Thierry Volpiatto <thievol@posteo.net>
> Cc: emacs-devel@gnu.org
> Date: Fri, 18 Aug 2023 04:57:06 +0000
> 
> Another thing, you use actually 
> 
>     (expand-file-name invocation-name invocation-directory)
> 
> I suggest you use the truename of this instead as emacs can be symlinked
> in some installations.  It should work with the symlink name, but it is
> clearer which emacs version is used.

We use the above in quite a few places, including when we start async
native-compilation.  So if it has some problems, we had better
discussed them, because I don't quite see why would the above need to
use truename.  If the OS succeeded to start Emacs via a symlink when
the current session was started, why would it fail to start it from
inside Emacs?

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>>> Forget it, it is working properly, just forget you had modified
>>> package--dependencies as well.
>>> Sorry for the noise.
>>
>> It was either that or a new function had to be added, not sure which
>> approach is worse.  The current implementation seems to have been
>> hastily added by Lars last year, and it is a bit regrettable in
>> retrospect.  It would be possible to modify my change, and have the
>> function always return package-desc objects, since the function is only
>> used in one other spot in another part of the file.  While there might
>> be others (packages or individuals) that depend on the function behaving
>> the way it does, but on the other hand, convention designates it as
>> being an "internal" function.
>
> The actual version is something like 4 or 5 lines long, so external
> packages can inline this version under another name if really needed,
> but your version is covering the both so it's ok I think.
> OTOH The problem with package.el is inconsistency within its functions,
> sometimes you have to use a pkg-desc as arg, sometimes not, sometimes
> functions return a list of symbols sometimes a list of pkg-desc, what is
> a package name, a string or a symbol? What is a pkg-desc exactly,
> sometimes it is the cdr in other places the cadr is used...
> Also built-in packages don't have the same format unless they are distributed
> in Elpa etc...

Right, there is certainly work to be done.

>>> Some packages seems to require a specific version of a package for their
>>> dependency e.g. seq, by excluding it the package may not work correctly,
>>> this is my understanding but I may be wrong. Also perhaps the package
>>> e.g. seq is selected later when computing dependencies but maybe user
>>> wants to select a particular version manually in the first place?
>>
>> The current algorithm should pick the first package in the package alist
>> that satisfies the necessary dependencies.  Perhaps that should be
>> re-thought or the selection should be more clever, e.g. if the user
>> explicitly specifies a dependency with a version, then it should be
>> preferred to whatever other dependency might be determined, at the
>> possible expense of triggering runtime bugs.
>
> I think it is hard to cover all cases, but after all it is more a
> developer tool for debugging a particular package than a end user tool
> to run packages, so perhaps the developer can give directives about
> dependencies to use when needed.
>
> Another thing, you use actually 
>
>     (expand-file-name invocation-name invocation-directory)
>
> I suggest you use the truename of this instead as emacs can be symlinked
> in some installations.  It should work with the symlink name, but it is
> clearer which emacs version is used.

I second Eli's question here, what difference would it make?  Clearer to
whom?

> Now your function is working fine and nearly finish, did you think how
> you are going to distribute it? It seems you are going to merge it in
> master, but what about providing it as well as a Elpa package so that
> users of old emacs (at some point of course, say emacs-27) can use it to
> report bugs?

I am not a fan of small ELPA packages, but what I'd like to bring up
again was the proposal to add package.el itself to ELPA.  I wrote a
patch in <873559q83j.fsf@posteo.net> that should arrange everything
necessary for this move, there are still a few points that should be
discussed in terms of stability and recovering from a possibly
inconsistent state.  I think I'll create a feature branch some day soon
to make the proposal more concrete.

> Thanks.

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 4147 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>>> Forget it, it is working properly, just forget you had modified
>>>> package--dependencies as well.
>>>> Sorry for the noise.
>>>
>>> It was either that or a new function had to be added, not sure which
>>> approach is worse.  The current implementation seems to have been
>>> hastily added by Lars last year, and it is a bit regrettable in
>>> retrospect.  It would be possible to modify my change, and have the
>>> function always return package-desc objects, since the function is only
>>> used in one other spot in another part of the file.  While there might
>>> be others (packages or individuals) that depend on the function behaving
>>> the way it does, but on the other hand, convention designates it as
>>> being an "internal" function.
>>
>> The actual version is something like 4 or 5 lines long, so external
>> packages can inline this version under another name if really needed,
>> but your version is covering the both so it's ok I think.
>> OTOH The problem with package.el is inconsistency within its functions,
>> sometimes you have to use a pkg-desc as arg, sometimes not, sometimes
>> functions return a list of symbols sometimes a list of pkg-desc, what is
>> a package name, a string or a symbol? What is a pkg-desc exactly,
>> sometimes it is the cdr in other places the cadr is used...
>> Also built-in packages don't have the same format unless they are distributed
>> in Elpa etc...
>
> Right, there is certainly work to be done.
>
>>>> Some packages seems to require a specific version of a package for their
>>>> dependency e.g. seq, by excluding it the package may not work correctly,
>>>> this is my understanding but I may be wrong. Also perhaps the package
>>>> e.g. seq is selected later when computing dependencies but maybe user
>>>> wants to select a particular version manually in the first place?
>>>
>>> The current algorithm should pick the first package in the package alist
>>> that satisfies the necessary dependencies.  Perhaps that should be
>>> re-thought or the selection should be more clever, e.g. if the user
>>> explicitly specifies a dependency with a version, then it should be
>>> preferred to whatever other dependency might be determined, at the
>>> possible expense of triggering runtime bugs.
>>
>> I think it is hard to cover all cases, but after all it is more a
>> developer tool for debugging a particular package than a end user tool
>> to run packages, so perhaps the developer can give directives about
>> dependencies to use when needed.
>>
>> Another thing, you use actually 
>>
>>     (expand-file-name invocation-name invocation-directory)
>>
>> I suggest you use the truename of this instead as emacs can be symlinked
>> in some installations.  It should work with the symlink name, but it is
>> clearer which emacs version is used.
>
> I second Eli's question here, what difference would it make?  Clearer to
> whom?

OK, this is not that important.

>> Now your function is working fine and nearly finish, did you think how
>> you are going to distribute it? It seems you are going to merge it in
>> master, but what about providing it as well as a Elpa package so that
>> users of old emacs (at some point of course, say emacs-27) can use it to
>> report bugs?
>
> I am not a fan of small ELPA packages, but what I'd like to bring up
> again was the proposal to add package.el itself to ELPA.  I wrote a
> patch in <873559q83j.fsf@posteo.net> that should arrange everything
> necessary for this move, there are still a few points that should be
> discussed in terms of stability and recovering from a possibly
> inconsistent state.  I think I'll create a feature branch some day soon
> to make the proposal more concrete.

Ok, looking forward for having a package-isolate function fetchable for
everybody, I will use my own version that fit my needs as a helm action
for its package manager.

Thanks for your work.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Adding package and package-vc to ELPA

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 1237 bytes --]

Philip Kaludercic <philipk@posteo.net> writes:

>> Now your function is working fine and nearly finish, did you think how
>> you are going to distribute it? It seems you are going to merge it in
>> master, but what about providing it as well as a Elpa package so that
>> users of old emacs (at some point of course, say emacs-27) can use it to
>> report bugs?
>
> I am not a fan of small ELPA packages, but what I'd like to bring up
> again was the proposal to add package.el itself to ELPA.  I wrote a
> patch in <873559q83j.fsf@posteo.net> that should arrange everything
> necessary for this move, there are still a few points that should be
> discussed in terms of stability and recovering from a possibly
> inconsistent state.  I think I'll create a feature branch some day soon
> to make the proposal more concrete.

I have pushed the patch from that thread, together with a patch for
package-vc and a custom command that would revert Emacs to use the
built-in version of package.el.  The branch is called
"feature/elpa-package", and I have added to tarballs with packages built
by elpa-admin.el in case anyone wants to try out the packages.  I'll be
testing these tarballs with older versions of Emacs, to see how that
works out.


[-- Attachment #2: package.tar --]
[-- Type: application/x-tar, Size: 204800 bytes --]

[-- Attachment #3: package-vc.tar --]
[-- Type: application/x-tar, Size: 51200 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0001-elpa-packages-package-package-vc-New-packages.patch --]
[-- Type: text/x-diff, Size: 910 bytes --]

From 3834109bf379fae333b60adf49dd119bd538635a Mon Sep 17 00:00:00 2001
From: Philip Kaludercic <philipk@posteo.net>
Date: Fri, 18 Aug 2023 20:33:24 +0200
Subject: [PATCH] * elpa-packages (package,package-vc): New packages

---
 elpa-packages | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/elpa-packages b/elpa-packages
index 3949d8fe14..ced827955b 100644
--- a/elpa-packages
+++ b/elpa-packages
@@ -521,7 +521,9 @@
  (paced			:url "bzr::bzr://bzr.savannah.nongnu.org/paced-el/trunk"
   ;; The Bzr<->Git bridge wasn't working well enough last time I tried.
   :manual-sync t)
+ (package		:core "lisp/emacs-lisp/package.el")
  (package-fixes		:url nil)
+ (package-vc		:core "lisp/emacs-lisp/package-vc.el")
  (parsec              	:url "https://github.com/cute-jumper/parsec.el.git")
  (parser-generator	:url "https://github.com/cjohansson/emacs-parser-generator")
  (path-iterator		:url nil)
-- 
2.41.0

Re: Adding package and package-vc to ELPA

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 1411 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>>> Now your function is working fine and nearly finish, did you think how
>>> you are going to distribute it? It seems you are going to merge it in
>>> master, but what about providing it as well as a Elpa package so that
>>> users of old emacs (at some point of course, say emacs-27) can use it to
>>> report bugs?
>>
>> I am not a fan of small ELPA packages, but what I'd like to bring up
>> again was the proposal to add package.el itself to ELPA.  I wrote a
>> patch in <873559q83j.fsf@posteo.net> that should arrange everything
>> necessary for this move, there are still a few points that should be
>> discussed in terms of stability and recovering from a possibly
>> inconsistent state.  I think I'll create a feature branch some day soon
>> to make the proposal more concrete.
>
> I have pushed the patch from that thread, together with a patch for
> package-vc and a custom command that would revert Emacs to use the
> built-in version of package.el.  The branch is called
> "feature/elpa-package", and I have added to tarballs with packages built
> by elpa-admin.el in case anyone wants to try out the packages.  I'll be
> testing these tarballs with older versions of Emacs, to see how that
> works out.

Thanks, looking forward for this to be available.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 2225 bytes --]


Hello Philip,

Philip Kaludercic <philipk@posteo.net> writes:

> How about this patch, that will use a temporary directory when
> `package-isolate' is invoked with a prefix argument (not sure what the
> default should be, I guess reusing `user-emacs-directory' is less
> surprising):
>
> [2. text/x-diff; 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...

I was reading the code of the new version of `package--dependencies` and
had some questions:

--8<---------------cut here---------------start------------->8---
    (named-let more ((pkg-desc desc))
      (let (deps)
        (dolist (req (package-desc-reqs pkg-desc))
          (setq deps (nconc
                      (catch 'found
                        (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
                          (when (and (string= (car req) (package-desc-name p))
                                     (version-list-<= (cadr req) (package-desc-version p)))
                            (throw 'found (more p)))))
                      deps)))
        (delete-dups (cons pkg-desc deps))))
--8<---------------cut here---------------end--------------->8---

Why are you using `string=` to compare (car req) 
with (package-desc-name p)?

Isn't (apply #'append (mapcar #'cdr (package--alist))) same as
(mapcar #'cadr (package--alist))?

I am not a fan of named-let but isn't using deps as a second arg of
'more' more in the named-let spirit? (fully not tested)

--8<---------------cut here---------------start------------->8---
    (named-let more ((pkg-desc desc)
                     (deps nil))
      (dolist (req (package-desc-reqs pkg-desc))
        (setq deps (nconc
                    (catch 'found
                      (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
                        (when (and (string= (car req) (package-desc-name p))
                                   (version-list-<= (cadr req) (package-desc-version p)))
                          (throw 'found (more p deps)))))
                    deps)))
      (delete-dups (cons pkg-desc deps)))
--8<---------------cut here---------------end--------------->8---

Thanks.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> Hello Philip,
>
> Philip Kaludercic <philipk@posteo.net> writes:
>
>> How about this patch, that will use a temporary directory when
>> `package-isolate' is invoked with a prefix argument (not sure what the
>> default should be, I guess reusing `user-emacs-directory' is less
>> surprising):
>>
>> [2. text/x-diff;
>> 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...
>
> I was reading the code of the new version of `package--dependencies` and
> had some questions:
>
>     (named-let more ((pkg-desc desc))
>       (let (deps)
>         (dolist (req (package-desc-reqs pkg-desc))
>           (setq deps (nconc
>                       (catch 'found
>                         (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
>                           (when (and (string= (car req) (package-desc-name p))
>                                      (version-list-<= (cadr req) (package-desc-version p)))
>                             (throw 'found (more p)))))
>                       deps)))
>         (delete-dups (cons pkg-desc deps))))
>
>
> Why are you using `string=` to compare (car req) 
> with (package-desc-name p)?

As opposed to what alternative?  The advantage of `string=' compared to
something like equal is that 1. that I do not have to care if a package
name is a symbol or a string 2. that all other values would trigger an
error.

> Isn't (apply #'append (mapcar #'cdr (package--alist))) same as
> (mapcar #'cadr (package--alist))?

Iff every package in (package--alist) only had one package descriptor,
which is not the case

(seq-filter
 (lambda (e) (length> e 2))
 (package--alist))
;=> non-nil

Also, see

(seq-set-equal-p
 (apply #'append (mapcar #'cdr (package--alist)))
 (mapcar #'cadr (package--alist)))
;=> nil

> I am not a fan of named-let but isn't using deps as a second arg of
> 'more' more in the named-let spirit? (fully not tested)
>
>     (named-let more ((pkg-desc desc)
>                      (deps nil))
>       (dolist (req (package-desc-reqs pkg-desc))
>         (setq deps (nconc
>                     (catch 'found
>                       (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
>                         (when (and (string= (car req) (package-desc-name p))
>                                    (version-list-<= (cadr req) (package-desc-version p)))
>                           (throw 'found (more p deps)))))
>                     deps)))
>       (delete-dups (cons pkg-desc deps)))

The only difference I believe would occur in this case, is that we would
always maintain "the same" list of dependencies, throughout traversing
all package dependencies.  In the end that shouldn't change much, and it
is only more work for `delete-dups'.

> Thanks.

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 4553 bytes --]


Thierry Volpiatto <thievol@posteo.net> writes:

> Hello Philip,
>
> Philip Kaludercic <philipk@posteo.net> writes:
>
>> How about this patch, that will use a temporary directory when
>> `package-isolate' is invoked with a prefix argument (not sure what the
>> default should be, I guess reusing `user-emacs-directory' is less
>> surprising):
>>
>> [2. text/x-diff; 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...
>
> I was reading the code of the new version of `package--dependencies` and
> had some questions:
>
> --8<---------------cut here---------------start------------->8---
>     (named-let more ((pkg-desc desc))
>       (let (deps)
>         (dolist (req (package-desc-reqs pkg-desc))
>           (setq deps (nconc
>                       (catch 'found
>                         (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
>                           (when (and (string= (car req) (package-desc-name p))
>                                      (version-list-<= (cadr req) (package-desc-version p)))
>                             (throw 'found (more p)))))
>                       deps)))
>         (delete-dups (cons pkg-desc deps))))
> --8<---------------cut here---------------end--------------->8---
>
> Why are you using `string=` to compare (car req) 
> with (package-desc-name p)?
>
> Isn't (apply #'append (mapcar #'cdr (package--alist))) same as
> (mapcar #'cadr (package--alist))?
>
> I am not a fan of named-let but isn't using deps as a second arg of
> 'more' more in the named-let spirit? (fully not tested)
>
> --8<---------------cut here---------------start------------->8---
>     (named-let more ((pkg-desc desc)
>                      (deps nil))
>       (dolist (req (package-desc-reqs pkg-desc))
>         (setq deps (nconc
>                     (catch 'found
>                       (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
>                         (when (and (string= (car req) (package-desc-name p))
>                                    (version-list-<= (cadr req) (package-desc-version p)))
>                           (throw 'found (more p deps)))))
>                     deps)))
>       (delete-dups (cons pkg-desc deps)))
> --8<---------------cut here---------------end--------------->8---

After some tests this create circular lists, so forget it.

Also now from Emacs 29 (magit is NOT installed):

(package-initialize)
(package--dependencies 'magit)
=> (emacs compat dash git-commit magit-section seq transient with-editor)

Now if I eval your new package--dependencies from Emacs 30:
(package-initialize)
(package--dependencies 'magit)
=> (compat)

What is wrong is you are looping in package-alist instead of
package-archive-contents, in fact you don't need to loop at all, just
using assq or package-get-descriptor:

--8<---------------cut here---------------start------------->8---
    (defun package--dependencies (pkg)
      "Return a list of all transitive dependencies of PKG.
    If PKG is a package descriptor, the return value is a list of
    package descriptors.  If PKG is a symbol designating a package,
    the return value is a list of symbols designating packages."
      (when-let* ((desc (if (package-desc-p pkg) pkg
                          (cadr (assq pkg package-archive-contents)))))
        ;; Can we have circular dependencies?  Assume "nope".
        (let ((all (cl-labels ((more (pkg-desc)
                                 (let (deps)
                                   (dolist (req (package-desc-reqs pkg-desc))
                                     (setq deps (nconc
                                                 (when-let* ((matched (package-get-descriptor (car req)))
                                                             (version<= (version-list-<=
                                                                         (cadr req)
                                                                         (package-desc-version matched))))
                                                   (more matched))
                                                 deps)))
                                   (delete-dups (cons pkg-desc deps)))))
                     (more desc))))
          (remq pkg (mapcar (if (package-desc-p pkg) #'identity #'package-desc-name) all)))))
--8<---------------cut here---------------end--------------->8---

PS: Sorry I used cl-labels to test as I am not easy with named-let.


-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> Thierry Volpiatto <thievol@posteo.net> writes:
>
>> Hello Philip,
>>
>> Philip Kaludercic <philipk@posteo.net> writes:
>>
>>> How about this patch, that will use a temporary directory when
>>> `package-isolate' is invoked with a prefix argument (not sure what the
>>> default should be, I guess reusing `user-emacs-directory' is less
>>> surprising):
>>>
>>> [2. text/x-diff; 0001-Add-command-to-start-Emacs-with-specific-packages.patch]...
>>
>> I was reading the code of the new version of `package--dependencies` and
>> had some questions:
>>
>> --8<---------------cut here---------------start------------->8---
>>     (named-let more ((pkg-desc desc))
>>       (let (deps)
>>         (dolist (req (package-desc-reqs pkg-desc))
>>           (setq deps (nconc
>>                       (catch 'found
>>                         (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
>>                           (when (and (string= (car req) (package-desc-name p))
>>                                      (version-list-<= (cadr req) (package-desc-version p)))
>>                             (throw 'found (more p)))))
>>                       deps)))
>>         (delete-dups (cons pkg-desc deps))))
>> --8<---------------cut here---------------end--------------->8---
>>
>> Why are you using `string=` to compare (car req) 
>> with (package-desc-name p)?
>>
>> Isn't (apply #'append (mapcar #'cdr (package--alist))) same as
>> (mapcar #'cadr (package--alist))?
>>
>> I am not a fan of named-let but isn't using deps as a second arg of
>> 'more' more in the named-let spirit? (fully not tested)
>>
>> --8<---------------cut here---------------start------------->8---
>>     (named-let more ((pkg-desc desc)
>>                      (deps nil))
>>       (dolist (req (package-desc-reqs pkg-desc))
>>         (setq deps (nconc
>>                     (catch 'found
>>                       (dolist (p (apply #'append (mapcar #'cdr (package--alist))))
>>                         (when (and (string= (car req) (package-desc-name p))
>>                                    (version-list-<= (cadr req) (package-desc-version p)))
>>                           (throw 'found (more p deps)))))
>>                     deps)))
>>       (delete-dups (cons pkg-desc deps)))
>> --8<---------------cut here---------------end--------------->8---
>
> After some tests this create circular lists, so forget it.

I guess you could avoid that by replacing 'nconc' with 'append'.

> Also now from Emacs 29 (magit is NOT installed):
>
> (package-initialize)
> (package--dependencies 'magit)
> => (emacs compat dash git-commit magit-section seq transient with-editor)
>
> Now if I eval your new package--dependencies from Emacs 30:
> (package-initialize)
> (package--dependencies 'magit)
> => (compat)
>
> What is wrong is you are looping in package-alist instead of
> package-archive-contents, in fact you don't need to loop at all, just
> using assq or package-get-descriptor:
>
>     (defun package--dependencies (pkg)
>       "Return a list of all transitive dependencies of PKG.
>     If PKG is a package descriptor, the return value is a list of
>     package descriptors.  If PKG is a symbol designating a package,
>     the return value is a list of symbols designating packages."
>       (when-let* ((desc (if (package-desc-p pkg) pkg
>                           (cadr (assq pkg package-archive-contents)))))
>         ;; Can we have circular dependencies?  Assume "nope".
>         (let ((all (cl-labels ((more (pkg-desc)
>                                  (let (deps)
>                                    (dolist (req (package-desc-reqs pkg-desc))
>                                      (setq deps (nconc
>                                                  (when-let* ((matched (package-get-descriptor (car req)))
>                                                              (version<= (version-list-<=
>                                                                          (cadr req)
>                                                                          (package-desc-version matched))))

An issue I see here, is that a required version might not be satisfied
by the version of the package returned by `package-get-descriptor'.  Or
am I mistaken?

>                                                    (more matched))
>                                                  deps)))
>                                    (delete-dups (cons pkg-desc deps)))))
>                      (more desc))))
>           (remq pkg (mapcar (if (package-desc-p pkg) #'identity #'package-desc-name) all)))))
>
> PS: Sorry I used cl-labels to test as I am not easy with named-let.

"Not easy" in the sense that you are not familiar with how it works, or
thin k it shouldn't be used?  All in all, `named-let' just compiles down
to a cl-labels call, similar to the one you propose.

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]


Philip Kaludercic <philipk@posteo.net> writes:

> An issue I see here, is that a required version might not be satisfied
> by the version of the package returned by `package-get-descriptor'.  Or
> am I mistaken?

So I guess we need a function that get the dependencies from installed
packages and one from all to fit with what we previously had.
If one wants to have all the dependencies for a given package not
already installed, there is actually nothing available.
Also `package-menu--list-to-prompt` is now wrong when used as prompt for
package-install.

> "Not easy" in the sense that you are not familiar with how it works, or
> thin k it shouldn't be used?  All in all, `named-let' just compiles down
> to a cl-labels call, similar to the one you propose.

Not familiar.

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Thierry Volpiatto]

[-- Attachment #1: Type: text/plain, Size: 2264 bytes --]


Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> An issue I see here, is that a required version might not be satisfied
>> by the version of the package returned by `package-get-descriptor'.  Or
>> am I mistaken?
>
> So I guess we need a function that get the dependencies from installed
> packages and one from all to fit with what we previously had.

Or use an optional arg:

--8<---------------cut here---------------start------------->8---
    (defun package--dependencies (pkg &optional only-local)
      "Return a list of all transitive dependencies of PKG.
    If PKG is a package descriptor, the return value is a list of
    package descriptors.  If PKG is a symbol designating a package,
    the return value is a list of symbols designating packages."
      (when-let* ((desc (if (package-desc-p pkg) pkg
                          (cadr (assq pkg package-archive-contents)))))
        ;; Can we have circular dependencies?  Assume "nope".
        (let ((all (cl-labels ((more (pkg-desc)
                                 (let (deps)
                                   (dolist (req (package-desc-reqs pkg-desc))
                                     (setq deps (nconc
                                                 (when-let* ((matched (if only-local
                                                                          (cadr (assq (car req) package-alist))
                                                                        (package-get-descriptor (car req))))
                                                             (version<= (version-list-<=
                                                                         (cadr req)
                                                                         (package-desc-version matched))))
                                                   (more matched))
                                                 deps)))
                                   (delete-dups (cons pkg-desc deps)))))
                     (more desc))))
          (remq pkg (mapcar (if (package-desc-p pkg) #'identity #'package-desc-name) all)))))
--8<---------------cut here---------------end--------------->8---

-- 
Thierry

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 686 bytes --]

Re: Proposal for 'package-isolate' command

[Philip Kaludercic]

Thierry Volpiatto <thievol@posteo.net> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> An issue I see here, is that a required version might not be satisfied
>> by the version of the package returned by `package-get-descriptor'.  Or
>> am I mistaken?
>
> So I guess we need a function that get the dependencies from installed
> packages and one from all to fit with what we previously had.
> If one wants to have all the dependencies for a given package not
> already installed, there is actually nothing available.
> Also `package-menu--list-to-prompt` is now wrong when used as prompt for
> package-install.

I think a sufficient solution might be just to extend
`package-get-descriptor' with an optional MIN-VERSION argument.

>> "Not easy" in the sense that you are not familiar with how it works, or
>> thin k it shouldn't be used?  All in all, `named-let' just compiles down
>> to a cl-labels call, similar to the one you propose.
>
> Not familiar.

Ok.