Add the archive name column in the list-packages table

Add the archive name column in the list-packages table

[Darren Hoo]

Hi guys,

For those who use third-party archives other than GNU elpa it is not
easy to tell where a package is from without bringing up the summary
info of a package. And it is even worse for those who use more than two
archive locations as packages listed are mingled together(no sorting).   

So How about adding a column in list-packages that specifies the archive
name where one package is from?

Re: Add the archive name column in the list-packages table

[Glenn Morris]

Darren Hoo wrote:

> So How about adding a column in list-packages that specifies the archive
> name where one package is from?

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=12775

Re: Add the archive name column in the list-packages table

[Ted Zlatanov]

On Fri, 25 Oct 2013 12:09:05 -0400 Glenn Morris <rgm@gnu.org> wrote: 

GM> Darren Hoo wrote:
>> So How about adding a column in list-packages that specifies the archive
>> name where one package is from?

GM> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=12775

Can you try this patch?  It seems to DTRT for me, leaving the archive to
an empty string for nil archives.  Sorting works too.  I could make a
new `package-desc-archive-print' function I guess, if you think the
hard-coded "" is ugly.

Ted

=== modified file 'lisp/emacs-lisp/package.el'
--- lisp/emacs-lisp/package.el  2013-10-03 07:11:27 +0000
+++ lisp/emacs-lisp/package.el  2013-10-25 18:24:44 +0000
@@ -1575,6 +1575,7 @@
   (setq tabulated-list-format [("Package" 18 package-menu--name-predicate)
                               ("Version" 12 nil)
                               ("Status"  10 package-menu--status-predicate)
+                              ("Archive" 10 package-menu--archive-predicate)
                               ("Description" 0 nil)])
   (setq tabulated-list-padding 2)
   (setq tabulated-list-sort-key (cons "Status" nil))
@@ -1697,6 +1698,8 @@
                                (package-desc-version pkg-desc))
                              'font-lock-face face)
                  (propertize status 'font-lock-face face)
+                 (propertize (or (package-desc-archive pkg-desc) "")
+                              'font-lock-face face)
                  (propertize (package-desc-summary pkg-desc)
                               'font-lock-face face)))))
 
@@ -1913,6 +1916,10 @@
   (string< (symbol-name (package-desc-name (car A)))
           (symbol-name (package-desc-name (car B)))))
 
+(defun package-menu--archive-predicate (A B)
+  (string< (or (package-desc-archive (car A)) "")
+          (or (package-desc-archive (car B)) "")))
+
 ;;;###autoload
 (defun list-packages (&optional no-fetch)
   "Display a list of packages.

Re: Add the archive name column in the list-packages table

[Stefan Monnier]

> For those who use third-party archives other than GNU elpa it is not
> easy to tell where a package is from without bringing up the summary
> info of a package. And it is even worse for those who use more than two
> archive locations as packages listed are mingled together(no sorting).   

I'm not sure if you're basing your comments on the 24.3 version of
package.el or the latest code in trunk.

The code in trunk only shows a package once in lisp-packages, regardless
of whether it's installed, built-in, and/or available at a hundred
different archives.


        Stefan

Re: Add the archive name column in the list-packages table

[Dmitry Gutov]

[-- Attachment #1: Type: text/plain, Size: 267 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The code in trunk only shows a package once in lisp-packages, regardless
> of whether it's installed, built-in, and/or available at a hundred
> different archives.

Not really. I see most of them mentioned twice:


[-- Attachment #2: list-packages.png --]
[-- Type: image/png, Size: 178995 bytes --]

Re: Add the archive name column in the list-packages table

[Stefan Monnier]

>> The code in trunk only shows a package once in lisp-packages, regardless
>> of whether it's installed, built-in, and/or available at a hundred
>> different archives.
> Not really. I see most of them mentioned twice:

Hmm... I indeed see similar things here...
I thought I had solved this.  Damn!


        Stefan

Re: Add the archive name column in the list-packages table

[Darren Hoo]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> The code in trunk only shows a package once in lisp-packages, regardless
>>> of whether it's installed, built-in, and/or available at a hundred
>>> different archives.
>> Not really. I see most of them mentioned twice:
>
> Hmm... I indeed see similar things here...
> I thought I had solved this.  Damn!

I've not seen your fix yet, but Packages with the same name being listed
only once is not really what I want. what if I want to install a specific
version? Are they listed in the summary buffer twice, thrice etc and
have different install buttons so that I can choose?

Re: Add the archive name column in the list-packages table

[Ted Zlatanov]

On Fri, 25 Oct 2013 14:27:11 -0400 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Fri, 25 Oct 2013 12:09:05 -0400 Glenn Morris <rgm@gnu.org> wrote: 
GM> Darren Hoo wrote:
>>> So How about adding a column in list-packages that specifies the archive
>>> name where one package is from?

GM> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=12775

TZ> Can you try this patch?  It seems to DTRT for me, leaving the archive to
TZ> an empty string for nil archives.  Sorting works too.  I could make a
TZ> new `package-desc-archive-print' function I guess, if you think the
TZ> hard-coded "" is ugly.

Ping?  I didn't want to assume it was OK to install my patch, but it
works great for me and I think it makes sense to install it.

Ted

Re: Add the archive name column in the list-packages table

[Stefan Monnier]

> Ping?  I didn't want to assume it was OK to install my patch, but it
> works great for me and I think it makes sense to install it.

Go ahead.


        Stefan

Re: Add the archive name column in the list-packages table

[Ted Zlatanov]

On Sun, 03 Nov 2013 09:12:21 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> Ping?  I didn't want to assume it was OK to install my patch, but it
>> works great for me and I think it makes sense to install it.

SM> Go ahead.

OK, installed.  I didn't document it because it's very self-explanatory
but please let me know if any tutorials or docs need to be updated.

Ted

Re: Add the archive name column in the list-packages table

[Xue Fuqiao]

On Tue, Nov 5, 2013 at 12:12 AM, Ted Zlatanov <tzz@lifelogs.com> wrote:
> OK, installed.  I didn't document it because it's very self-explanatory
> but please let me know if any tutorials or docs need to be updated.

Thank you, it looks good.  IMHO it would also be useful to add the
archive name to the buffer created by `C-h P' (`describe-package').

-- 
http://www.gnu.org/software/emacs/

Re: Add the archive name column in the list-packages table

[Ted Zlatanov]

On Tue, 19 Nov 2013 15:36:58 +0800 Xue Fuqiao <xfq.free@gmail.com> wrote: 

XF> On Tue, Nov 5, 2013 at 12:12 AM, Ted Zlatanov <tzz@lifelogs.com> wrote:
>> OK, installed.  I didn't document it because it's very self-explanatory
>> but please let me know if any tutorials or docs need to be updated.

XF> Thank you, it looks good.  IMHO it would also be useful to add the
XF> archive name to the buffer created by `C-h P' (`describe-package').

Agreed, and done.  The archive is shown in the "available from" line but
it really looks better on its own line.

I think the Keywords should also be shown in the package description and
maybe available for filtering, WDYT?

Thanks
Ted

Re: Add the archive name column in the list-packages table

[Xue Fuqiao]

On Thu, Nov 21, 2013 at 5:06 AM, Ted Zlatanov <tzz@lifelogs.com> wrote:
>>> OK, installed.  I didn't document it because it's very self-explanatory
>>> but please let me know if any tutorials or docs need to be updated.
>
> XF> Thank you, it looks good.  IMHO it would also be useful to add the
> XF> archive name to the buffer created by `C-h P' (`describe-package').
>
> Agreed, and done.  The archive is shown in the "available from" line but
> it really looks better on its own line.
>
> I think the Keywords should also be shown in the package description and
> maybe available for filtering, WDYT?

I agree.  Showing the dependencies would be useful, too.

-- 
http://www.gnu.org/software/emacs/

add keyword search and display in details for list-packages (was: Add the archive name column in the list-packages table)

[Ted Zlatanov]

On Thu, 21 Nov 2013 06:34:50 +0800 Xue Fuqiao <xfq.free@gmail.com> wrote: 

XF> On Thu, Nov 21, 2013 at 5:06 AM, Ted Zlatanov <tzz@lifelogs.com> wrote:

>> I think the Keywords should also be shown in the package description and
>> maybe available for filtering, WDYT?

XF> I agree.  Showing the dependencies would be useful, too.

OK.  Any objections to:

* adding filtering by Keywords in the packages list?
* showing the Keywords in the packages list?
* showing the Keywords in the package description?

The package.el TODO says:

;; - should store the package's keywords in archive-contents, then
;;   let the users filter the package-menu by keyword.  See
;;   finder-by-keyword.

so it seems this will require backend work by the GNU ELPA tools to put
the Keywords in the package description.  Stefan, WDYT?

Finally, in addition to the Keywords, is there a package category,
meaning a single word to describe its purpose?  It might be more
suitable for display in the packages list and could let us set up a
hierarchy in the packages list to avoid the current "2500+ entries in a
flat list" situation.

Ted

Re: Add the archive name column in the list-packages table

[Jambunathan K]

> XF> Thank you, it looks good.  IMHO it would also be useful to add the
> XF> archive name to the buffer created by `C-h P' (`describe-package').
>
> Agreed, and done.  The archive is shown in the "available from" line but
> it really looks better on its own line.

It would be nice to show little visual icons (think favicons) for
repositories.  "Available" is taking too much space on my netbook
screen.  Nice little icons to give visual feedback.

2 cents.

> Thanks
> Ted

Re: Add the archive name column in the list-packages table

[Ted Zlatanov]

On Thu, 21 Nov 2013 17:07:02 +0530 Jambunathan K <kjambunathan@gmail.com> wrote: 

XF> Thank you, it looks good.  IMHO it would also be useful to add the
XF> archive name to the buffer created by `C-h P' (`describe-package').
>> 
>> Agreed, and done.  The archive is shown in the "available from" line but
>> it really looks better on its own line.

JK> It would be nice to show little visual icons (think favicons) for
JK> repositories.  "Available" is taking too much space on my netbook
JK> screen.  Nice little icons to give visual feedback.

The "available from" information could be on a new line to show all the
info.

I think icons are typically used for familiar things, so it would be
puzzling to the users to see an icon for a repository name.  They
wouldn't know what icon corresponds to what repository.

You can, of course, do this on your own with text properties.  But I
don't think it makes sense for all users.

Ted

Re: Add the archive name column in the list-packages table

[Jambunathan K]

Ted Zlatanov <tzz@lifelogs.com> writes:

> They wouldn't know what icon corresponds to what repository.

You mean Gnu Icon wouldn't be recognized by Emacs users?

> You can, of course, do this on your own with text properties.

I am just making a general remark: Can or should Emacs use Icons.  Your
post was just an excuse.

Re: add keyword search and display in details for list-packages

[Stefan Monnier]

>>> I think the Keywords should also be shown in the package description and
>>> maybe available for filtering, WDYT?
XF> I agree.  Showing the dependencies would be useful, too.

Adding dependencies in the package description sounds good, indeed.

> OK.  Any objections to:
> * adding filtering by Keywords in the packages list?

OK.

> * showing the Keywords in the packages list?

Not sure how you plan to do that.

> * showing the Keywords in the package description?

OK.

> ;; - should store the package's keywords in archive-contents, then
> ;;   let the users filter the package-menu by keyword.  See
> ;;   finder-by-keyword.
> so it seems this will require backend work by the GNU ELPA tools to put
> the Keywords in the package description.  Stefan, WDYT?

If you want to filter by keywords, I see 3 options:
- add keywords in archive-contents.
- let package.el download all packages to extract keywords from them.
- use a separate file alongside archive-contents.
Option 2 is only included for completeness since it makes little sense.

Option 1 probably makes most sense.

Option 3 could make sense if filtering by keyword is rare and if the
size of the keywords data is significantly larger than the size of
archive-contents.

IOW, option 1 sounds best.

> Finally, in addition to the Keywords, is there a package category,
> meaning a single word to describe its purpose?

No, and I don't think we'll ever be able to categorize this way.

> It might be more suitable for display in the packages list and could
> let us set up a hierarchy in the packages list to avoid the current
> "2500+ entries in a flat list" situation.

We could just as well browse by keywords.


        Stefan

Re: Add the archive name column in the list-packages table

[Stefan Monnier]

>> Agreed, and done.  The archive is shown in the "available from" line but
>> it really looks better on its own line.
> It would be nice to show little visual icons (think favicons) for
> repositories.

I don't mind either way.  But the "Archive" title is taking too much
space, we should find a way to shorten it.

> "Available" is taking too much space on my netbook
> screen.  Nice little icons to give visual feedback.

Indeed, "available" uses a lot of space, but if we shorten it,
"built-in" will also need to be shortened (it's only 1 char shorter).

We could try to stick to "three-letter" maximum for the package state.

Icons would be fine but don't help in the tty case, so shortening the
text is also important.


        Stefan

Re: Add the archive name column in the list-packages table

[Ted Zlatanov]

On Thu, 21 Nov 2013 20:05:38 +0530 Jambunathan K <kjambunathan@gmail.com> wrote: 

JK> Ted Zlatanov <tzz@lifelogs.com> writes:
>> They wouldn't know what icon corresponds to what repository.

JK> You mean Gnu Icon wouldn't be recognized by Emacs users?

That's not what I said.  The user has to know that the GNU ELPA ("gnu")
repository is represented by the Gnu icon, while marmalade is a
marmalade jar and melpa is a Melba wafer and ELPA is Tom Tromey's face :)

This mapping is not intuitive or particularly useful IMO.  Sometimes
text really is better than an icon.

>> You can, of course, do this on your own with text properties.

JK> I am just making a general remark: Can or should Emacs use Icons.  Your
JK> post was just an excuse.

Well, no, your remark was very specific about using icons in the package
listings.  That's a context where security (you have to be sure where a
package is coming from) and usability (it must work in a terminal) are
more important than in most Emacs use cases.

I think in general Emacs should use icons more, but with care.  See
http://permalink.gmane.org/gmane.emacs.devel/137682 for a proposal
(iconset.el) I still intend to pursue, though it's not high on my TODO
list due to other priorities.

Ted

Re: add keyword search and display in details for list-packages

[Ted Zlatanov]

On Thu, 21 Nov 2013 09:44:39 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>>>> I think the Keywords should also be shown in the package description and
>>>> maybe available for filtering, WDYT?
XF> I agree.  Showing the dependencies would be useful, too.

SM> Adding dependencies in the package description sounds good, indeed.

It's already there in `describe-package-1':

#+begin_src lisp
    (setq reqs (if desc (package-desc-reqs desc)))
    (when reqs
      (insert "   " (propertize "Requires" 'font-lock-face 'bold) ": ")
      (let ((first t)
	    name vers text)
	(dolist (req reqs)
	  (setq name (car req)
		vers (cadr req)
		text (format "%s-%s" (symbol-name name)
			     (package-version-join vers)))
	  (cond (first (setq first nil))
		((>= (+ 2 (current-column) (length text))
		     (window-width))
		 (insert ",\n               "))
		(t (insert ", ")))
	  (help-insert-xref-button text 'help-package name))
	(insert "\n")))
#+end_src

It seems that the list of requirements is not making it into the
description.

>> OK.  Any objections to:
>> * adding filtering by Keywords in the packages list?

SM> OK.

>> * showing the Keywords in the packages list?

SM> Not sure how you plan to do that.

I was thinking of appending "k1,k2,..." to the description.

>> * showing the Keywords in the package description?

SM> OK.

>> ;; - should store the package's keywords in archive-contents, then
>> ;;   let the users filter the package-menu by keyword.  See
>> ;;   finder-by-keyword.
>> so it seems this will require backend work by the GNU ELPA tools to put
>> the Keywords in the package description.  Stefan, WDYT?

SM> If you want to filter by keywords, I see 3 options:
SM> - add keywords in archive-contents.
SM> - let package.el download all packages to extract keywords from them.
SM> - use a separate file alongside archive-contents.
[SM: 1 is best]

(1) works for me as well.  Do you or I need to do work on the backend
tools to make it happen?  That's the prerequisite for the other work
listed here, which I can do on my own afterwards.

>> Finally, in addition to the Keywords, is there a package category,
>> meaning a single word to describe its purpose?

SM> No, and I don't think we'll ever be able to categorize this way.

OK.

>> It might be more suitable for display in the packages list and could
>> let us set up a hierarchy in the packages list to avoid the current
>> "2500+ entries in a flat list" situation.

SM> We could just as well browse by keywords.

OK.

Ted

Re: Add the archive name column in the list-packages table

[Jambunathan K]

The description is pushed to the last column which means that it is
truncated.  Additional details can go as part of tool tip.  The tool-tip
is per-table cell.  May be it should be per-table row (at least in this
case)

Visual feedback need not mean icons.  It could be color coding.

The *entire* table-row can be highlighted in green if it available etc.
(I see that unsigned packages are highlighted in with foreground red.)

> I think in general Emacs should use icons more, but with care.  See
> http://permalink.gmane.org/gmane.emacs.devel/137682 for a proposal
> (iconset.el) I still intend to pursue, though it's not high on my TODO
> list due to other priorities.

It would be nice if Emacs is less verbal and more visual.

> Ted

Re: Add the archive name column in the list-packages table

[Ted Zlatanov]

On Thu, 21 Nov 2013 23:28:43 +0530 Jambunathan K <kjambunathan@gmail.com> wrote: 

JK> The description is pushed to the last column which means that it is
JK> truncated.  Additional details can go as part of tool tip.  The tool-tip
JK> is per-table cell.  May be it should be per-table row (at least in this
JK> case)

Consider me completely opposed to tooltips.  They require hovering,
don't work in a terminal, and are hard to read.

JK> Visual feedback need not mean icons.  It could be color coding.

JK> The *entire* table-row can be highlighted in green if it available etc.
JK> (I see that unsigned packages are highlighted in with foreground red.)

(`M-x list-packages' already has some color coding.)

That's an interesting idea, but it doesn't work in all terminals and
it's hard to read, especially for color-blind people.  Remember, this is
*the* way to install packages, so all users have to use it.  It's not an
optional package so the default UI has to be very conservative.

Also user-specific default faces and themes may conflict.  For instance
I already use green on black for regular text.

Maybe the shortened status ("avl" for available or whatever) can be
color-coded, bur the whole row is IMO too much.

>> I think in general Emacs should use icons more, but with care.  See
>> http://permalink.gmane.org/gmane.emacs.devel/137682 for a proposal
>> (iconset.el) I still intend to pursue, though it's not high on my TODO
>> list due to other priorities.

JK> It would be nice if Emacs is less verbal and more visual.

That's pretty vague.  Maybe you'd like to start a new thread with
specific suggestions.

Ted

RE: Add the archive name column in the list-packages table

[Drew Adams]

> Consider me completely opposed to tooltips.  They require hovering,
> don't work in a terminal, and are hard to read.

But what's a "tooltip"?

Emacs can use the echo area for "tooltip" text.  Does that use
provide a "tooltip"?  Not in the usual sense.  But Emacs is not so
usual, in general.

We used to have variable `tooltip-use-echo-area', whose name even
suggested that Emacs was using the echo area as a tooltip.

Call it what you might.  It shows the same info as a tooltip.

And no, you do *not* have to hover over the zone.  You do not even
have to have a mouse.

emacs -Q -nw

C-h f forward-char
C-h f backward-char
C-x o ; to get to *Help*
TAB   ; see :help ("tooltip") text in echo area
TAB   ; see next zone's :help text in echo area

Without -nw, just turn off `tooltip-mode'.  Same behavior as above.

Re: add keyword search and display in details for list-packages

[Stefan Monnier]

SM> Not sure how you plan to do that.
> I was thinking of appending "k1,k2,..." to the description.

But that means it'll be "out of view" in many/most cases.  Not sure it's
worth the trouble.

> (1) works for me as well.  Do you or I need to do work on the backend
> tools to make it happen?  That's the prerequisite for the other work
> listed here, which I can do on my own afterwards.

If you can work on the elpa/admin/* scripts to make them DTRT, then I'll
take care of installing them in elpa.gnu.org.


        Stefan

Re: add keyword search and display in details for list-packages

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 1440 bytes --]

On Thu, 21 Nov 2013 20:23:13 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> Not sure how you plan to do that.
>> I was thinking of appending "k1,k2,..." to the description.

SM> But that means it'll be "out of view" in many/most cases.  Not sure it's
SM> worth the trouble.

Well, it would be visible at least... better than having to hit `?' on
every package.  Maybe it can be toggled as an addition or as a column
instead with `k' in the packages listing?

>> (1) works for me as well.  Do you or I need to do work on the backend
>> tools to make it happen?  That's the prerequisite for the other work
>> listed here, which I can do on my own afterwards.

SM> If you can work on the elpa/admin/* scripts to make them DTRT, then I'll
SM> take care of installing them in elpa.gnu.org.

It was hard to get around the compilation failure of `packages/diff-hl'.
I had to remove it temporarily in order to test the `make' output.  Is
there a way to do "make ignore-errors" in ELPA?  I tested with this
commit as HEAD: 30809c7e797f57c50a63f64ac19eb8b78bc21939 (master from
today).

Either way, I believe I added the keywords correctly, see the attached
patch.  But I couldn't test it properly, sorry :) Doing all combinations
of `make clean; make; make archive-full' didn't refresh
`archive-contents' with the keywords, even though they were in place.
Maybe you can update the README or I'm missing something obvious?

Thanks
Ted


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: elpa-include-keywords.patch --]
[-- Type: text/x-diff, Size: 1221 bytes --]

diff --git a/admin/archive-contents.el b/admin/archive-contents.el
index 7ee9c1c..8fee66b 100644
--- a/admin/archive-contents.el
+++ b/admin/archive-contents.el
@@ -187,13 +187,18 @@ Otherwise, return nil."
                  (requires-str (lm-header "package-requires"))
                  (pt (lm-header "package-type"))
                  (simple (if pt (equal pt "simple") (= (length files) 1)))
+                 (keywords (split-string (or (lm-header "keywords") "")
+                                         "," t "[ ]+"))
                  (url (or (lm-header "url")
                           (format "http://elpa.gnu.org/packages/%s.html" pkg)))
                  (req
                   (if requires-str
                       (mapcar 'archive--convert-require
                               (car (read-from-string requires-str))))))
-            (list simple version description req (list (cons :url url)))))))
+            (list simple version description req
+                  ;; extra parameters
+                  (list (cons :url url)
+                        (cons :keywords keywords)))))))
      ((not (file-exists-p pkg-file))
       (error "Can find single file nor package desc file in %s" dir)))))
 

Re: add keyword search and display in details for list-packages

[Stefan Monnier]

> of `make clean; make; make archive-full' didn't refresh

Hmm... elpa/README is outdated: don't "make archive-full".

The way to reproduce what happens on elpa.gnu.org is to run
"admin/update-archive.sh".  IIRC you need to do it like this:

   git clone .../elpa.git
   mkdir build
   cd build
   ln -s ../elpa/{admin,GNUmakefile} .
   admin/update-archive.sh


-- Stefan

Re: add keyword search and display in details for list-packages

[Ted Zlatanov]

On Fri, 22 Nov 2013 20:49:39 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> of `make clean; make; make archive-full' didn't refresh
SM> Hmm... elpa/README is outdated: don't "make archive-full".

SM> The way to reproduce what happens on elpa.gnu.org is to run
SM> "admin/update-archive.sh".  IIRC you need to do it like this:

SM>    git clone .../elpa.git
SM>    mkdir build
SM>    cd build
SM>    ln -s ../elpa/{admin,GNUmakefile} .
SM>    admin/update-archive.sh

I got `staging/packages/archive-contents' with the right contents for a
multi-file package (w3) but the single-file package keywords are not
getting generated.  I spent about 2 hours trying to figure out what's
going on, following your instructions (which worked) and debugging
things ad-hoc.  It seems that the presence of the "PACKAGE-pkg.el" file
is causing the keyword evaluation to be skipped as a sort of cache takes
over, but I lost track of the indirection layers and couldn't figure it
out.

If you could commit my patch and test it once, that would be great.  I
believe it will DTRT even though I couldn't test it thoroughly.  Once
that's pushed I can proceed with the keyword-related features for
package.el.

A documented GNU ELPA clean build+stage process would be wonderful; the
previously-discussed package signing stage should also be inserted in
that flow.  Let me know if you want to work on this with me.  I'd do it
myself but after the 2-hour debugging session I just had, I'm not sure I
understand enough of the process to fix it on my own.  Some sort of CI
tool would make sense, though, that much is clear.

Ted

Re: add keyword search and display in details for list-packages

[Stefan Monnier]

> I got `staging/packages/archive-contents' with the right contents for a
> multi-file package (w3) but the single-file package keywords are not
> getting generated.

Good start, thanks.

> I spent about 2 hours trying to figure out what's going on, following
> your instructions (which worked) and debugging things ad-hoc.
> It seems that the presence of the "PACKAGE-pkg.el" file is causing the
> keyword evaluation to be skipped as a sort of cache takes over, but
> I lost track of the indirection layers and couldn't figure it out.

I can take a look.

> If you could commit my patch and test it once, that would be great.

I must have missed your patch.  Can you resend it?


        Stefan

Re: add keyword search and display in details for list-packages

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 873 bytes --]

On Mon, 25 Nov 2013 10:20:42 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> I got `staging/packages/archive-contents' with the right contents for a
>> multi-file package (w3) but the single-file package keywords are not
>> getting generated.

SM> Good start, thanks.

>> I spent about 2 hours trying to figure out what's going on, following
>> your instructions (which worked) and debugging things ad-hoc.
>> It seems that the presence of the "PACKAGE-pkg.el" file is causing the
>> keyword evaluation to be skipped as a sort of cache takes over, but
>> I lost track of the indirection layers and couldn't figure it out.

SM> I can take a look.

>> If you could commit my patch and test it once, that would be great.

SM> I must have missed your patch.  Can you resend it?

It was earlier in this thread, re-attaching here.  Thanks for looking
into it!

Ted



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: elpa-include-keywords.patch --]
[-- Type: text/x-diff, Size: 1221 bytes --]

diff --git a/admin/archive-contents.el b/admin/archive-contents.el
index 7ee9c1c..8fee66b 100644
--- a/admin/archive-contents.el
+++ b/admin/archive-contents.el
@@ -187,13 +187,18 @@ Otherwise, return nil."
                  (requires-str (lm-header "package-requires"))
                  (pt (lm-header "package-type"))
                  (simple (if pt (equal pt "simple") (= (length files) 1)))
+                 (keywords (split-string (or (lm-header "keywords") "")
+                                         "," t "[ ]+"))
                  (url (or (lm-header "url")
                           (format "http://elpa.gnu.org/packages/%s.html" pkg)))
                  (req
                   (if requires-str
                       (mapcar 'archive--convert-require
                               (car (read-from-string requires-str))))))
-            (list simple version description req (list (cons :url url)))))))
+            (list simple version description req
+                  ;; extra parameters
+                  (list (cons :url url)
+                        (cons :keywords keywords)))))))
      ((not (file-exists-p pkg-file))
       (error "Can find single file nor package desc file in %s" dir)))))
 

Re: add keyword search and display in details for list-packages

[Stefan Monnier]

>>> I spent about 2 hours trying to figure out what's going on, following
>>> your instructions (which worked) and debugging things ad-hoc.
>>> It seems that the presence of the "PACKAGE-pkg.el" file is causing the
>>> keyword evaluation to be skipped as a sort of cache takes over, but
>>> I lost track of the indirection layers and couldn't figure it out.
SM> I can take a look.

The patch you show only applies to `archive--simple-package-p', so
indeed it won't affect packages that have a <foo>-pkg.el.
For those with a <foo>-pkg.el, the keywords will have to be provided in
the <foo>-pkg.el file.  But you may opt not to bother, because the
<foo>-pkg.el files are on the way out (I'm speaking here about the
`elpa' git repository, not the tarballs distributed from elpa.gnu.org).
Currently only `auctex', `sokoban' and `uni-confusables' are left which
use a <foo>-pkg.el and those should be fixed as well.


        Stefan

Re: add keyword search and display in details for list-packages

[Stefan Monnier]

> diff --git a/admin/archive-contents.el b/admin/archive-contents.el
> index 7ee9c1c..8fee66b 100644
> --- a/admin/archive-contents.el
> +++ b/admin/archive-contents.el
> @@ -187,13 +187,18 @@ Otherwise, return nil."
>                   (requires-str (lm-header "package-requires"))
>                   (pt (lm-header "package-type"))
>                   (simple (if pt (equal pt "simple") (= (length files) 1)))
> +                 (keywords (split-string (or (lm-header "keywords") "")
> +                                         "," t "[ ]+"))
>                   (url (or (lm-header "url")
>                            (format "http://elpa.gnu.org/packages/%s.html" pkg)))
>                   (req
>                    (if requires-str
>                        (mapcar 'archive--convert-require
>                                (car (read-from-string requires-str))))))
> -            (list simple version description req (list (cons :url url)))))))
> +            (list simple version description req
> +                  ;; extra parameters
> +                  (list (cons :url url)
> +                        (cons :keywords keywords)))))))
>       ((not (file-exists-p pkg-file))
>        (error "Can find single file nor package desc file in %s" dir)))))
 
Looks good, please commit it, and I'll install it into elpa.gnu.org.


        Stefan

Re: add keyword search and display in details for list-packages

[Ted Zlatanov]

On Mon, 25 Nov 2013 22:25:44 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>>>> I spent about 2 hours trying to figure out what's going on, following
>>>> your instructions (which worked) and debugging things ad-hoc.
>>>> It seems that the presence of the "PACKAGE-pkg.el" file is causing the
>>>> keyword evaluation to be skipped as a sort of cache takes over, but
>>>> I lost track of the indirection layers and couldn't figure it out.
SM> I can take a look.

SM> The patch you show only applies to `archive--simple-package-p', so
SM> indeed it won't affect packages that have a <foo>-pkg.el.
SM> For those with a <foo>-pkg.el, the keywords will have to be provided in
SM> the <foo>-pkg.el file.  But you may opt not to bother, because the
SM> <foo>-pkg.el files are on the way out (I'm speaking here about the
SM> `elpa' git repository, not the tarballs distributed from elpa.gnu.org).
SM> Currently only `auctex', `sokoban' and `uni-confusables' are left which
SM> use a <foo>-pkg.el and those should be fixed as well.

OK, I won't bother.  `uni-confusables' is mine actually; what do I need
to do exactly?

On Mon, 25 Nov 2013 22:27:34 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> Looks good, please commit it, and I'll install it into elpa.gnu.org.

OK; done.  I assumed you don't keep a ChangeLog, but if you need the
entry let me know.

Thanks
Ted

Re: add keyword search and display in details for list-packages

[Stefan Monnier]

SM> Looks good, please commit it, and I'll install it into elpa.gnu.org.
> OK; done.  I assumed you don't keep a ChangeLog, but if you need the
> entry let me know.

We use the Git commit log.  But please next time use the usual
formatting conventions that we follow for ChangeLog messages.


        Stefan

Re: add keyword search and display in details for list-packages

[Ted Zlatanov]

On Tue, 26 Nov 2013 14:26:41 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> Looks good, please commit it, and I'll install it into elpa.gnu.org.
>> OK; done.  I assumed you don't keep a ChangeLog, but if you need the
>> entry let me know.

SM> We use the Git commit log.  But please next time use the usual
SM> formatting conventions that we follow for ChangeLog messages.

Understood, sorry.  Let me know when I can proceed with the package.el
changes that use the keywords.

(If there's a good way to enter ChangeLog editing mode in a VC commit
message, let me know.  Basically something like `C-x 4 a' but into the
commit message.)

Ted

Re: add keyword search and display in details for list-packages

[Dmitry Gutov]

Ted Zlatanov <tzz@lifelogs.com> writes:

> (If there's a good way to enter ChangeLog editing mode in a VC commit
> message, let me know.  Basically something like `C-x 4 a' but into the
> commit message.)

That would be http://debbugs.gnu.org/3462

Re: add keyword search and display in details for list-packages

[Stefan Monnier]

> Understood, sorry.  Let me know when I can proceed with the package.el
> changes that use the keywords.

I just installed your new archive-contents.el into elpa.gnu.org (tho
I had to tweak it because the Emacs used there is Emacs-24.3 which
doesn't have the `trim' argument of `split-string').

> (If there's a good way to enter ChangeLog editing mode in a VC commit
> message, let me know.

There isn't so far.  And it's not quite the same (e.g. we don't need/want
the "<date>  <author>  <emacs>" lines, nor the leading TAB chars of
indentation).

> Basically something like `C-x 4 a' but into the commit message.)

I have some tentative hacks to make `C-x 4 a' work with the *VC-Log*
buffer, but it rather sucks so far.


        Stefan

Re: add keyword search and display in details for list-packages

[Juri Linkov]

> +                 (keywords (split-string (or (lm-header "keywords") "")
> +                                         "," t "[ ]+"))

Isn't `lm-keywords-list' a better function to do this?

Re: add keyword search and display in details for list-packages

[Ted Zlatanov]

On Thu, 28 Nov 2013 02:52:38 +0200 Juri Linkov <juri@jurta.org> wrote: 

>> +                 (keywords (split-string (or (lm-header "keywords") "")
>> +                                         "," t "[ ]+"))

JL> Isn't `lm-keywords-list' a better function to do this?

It would be normally, but it doesn't trim space off the strings (the
last argument is missing).  This is compatible with older Emacsen but, I
think, is less useful.  So can I make the change to `lm-keywords-list'
to add that last argument?  Or do you and Stefan feel it should be used
without worrying about trimming space?

Sorry I didn't mention this earlier, I simply forgot.  It was on my TODO
list :)

Ted

Re: add keyword search and display in details for list-packages

[Stefan Monnier]

>>> +                 (keywords (split-string (or (lm-header "keywords") "")
>>> +                                         "," t "[ ]+"))
JL> Isn't `lm-keywords-list' a better function to do this?
> It would be normally, but it doesn't trim space off the strings (the
> last argument is missing).  This is compatible with older Emacsen but, I
> think, is less useful.  So can I make the change to `lm-keywords-list'
> to add that last argument?

Yes.

> Or do you and Stefan feel it should be used
> without worrying about trimming space?

I agree to this as well (remember that elpa.gnu.org uses Emacs-24.3 to
build its web pages).


        Stefan

Re: add keyword search and display in details for list-packages

[Ted Zlatanov]

On Wed, 04 Dec 2013 14:53:30 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>>>> +                 (keywords (split-string (or (lm-header "keywords") "")
>>>> +                                         "," t "[ ]+"))
JL> Isn't `lm-keywords-list' a better function to do this?
>> It would be normally, but it doesn't trim space off the strings (the
>> last argument is missing).  This is compatible with older Emacsen but, I
>> think, is less useful.  So can I make the change to `lm-keywords-list'
>> to add that last argument?

SM> Yes.

OK, pushed to Emacs trunk.

>> Or do you and Stefan feel it should be used
>> without worrying about trimming space?

SM> I agree to this as well (remember that elpa.gnu.org uses Emacs-24.3 to
SM> build its web pages).

OK, pushed to elpa.git.  I see it's the right approach to rely on
`lm-keywords-list' to DTRT.

Thanks
Ted