Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Philip Kaludercic]

Stefan Kangas <stefankangas@gmail.com> writes:

> branch: main
> commit 74116339a852129b98ff79e2eb3aa35b387aa006
> Author: Stefan Kangas <stefan@marxist.se>
> Commit: Stefan Kangas <stefan@marxist.se>
>
>     * elpa-packages (anzu): New package

Hasn't anzu been obsoleted by isearch-lazy-count?

> ---
>  elpa-packages | 5 +++++
>  1 file changed, 5 insertions(+)
>
> diff --git a/elpa-packages b/elpa-packages
> index 01923a66b7..667bfd132a 100644
> --- a/elpa-packages
> +++ b/elpa-packages
> @@ -22,6 +22,11 @@
>   ("anti-zenburn-theme"	:url "https://github.com/m00natic/anti-zenburn-theme"
>    :ignored-files ("anti-zenburn-snapshot.jpeg"))
>  
> + ("anzu"                :url "https://github.com/emacsorphanage/anzu.git"
> +  :readme "README.md"
> +  :news "Changes"
> +  :ignored-files (".github" "image" "Cask" "Makefile"))
> +
>   ("apache-mode"		:url "https://github.com/emacs-php/apache-mode"
>    :ignored-files ("LICENSE"))
>  
>
>

-- 
	Philip Kaludercic

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Stefan Kangas]

Philip Kaludercic <philipk@posteo.net> writes:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>> branch: main
>> commit 74116339a852129b98ff79e2eb3aa35b387aa006
>> Author: Stefan Kangas <stefan@marxist.se>
>> Commit: Stefan Kangas <stefan@marxist.se>
>>
>>     * elpa-packages (anzu): New package
>
> Hasn't anzu been obsoleted by isearch-lazy-count?

I don't know; I don't use isearch these days.  I remember using anzu
when I did.

I add packages mostly based on our formal rules, not on if I think they
are useful or good.  Some notable exceptions to this are s.el and f.el,
where we have decided that it is strongly undesirable to promote such
short package prefixes.  I have also skipped some other packages that
are clearly obsolete/no longer developed/bad or low effort, etc.

I am currently focusing on highly popular packages, and anzu clearly fit
the bill with over 1 million downloads on MELPA.org.  It also seems like
it is still maintained, with a new maintainer since March 2020, and
commits as late as October 2021.

But feel free to tell me why I'm wrong; I'm not married to it and if you
think we should remove it again, I don't think I will have any
objections.

BTW, maybe this should be raised as a ticket on the anzu bug tracker?

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Philip Kaludercic]

Stefan Kangas <stefan@marxist.se> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Stefan Kangas <stefankangas@gmail.com> writes:
>>
>>> branch: main
>>> commit 74116339a852129b98ff79e2eb3aa35b387aa006
>>> Author: Stefan Kangas <stefan@marxist.se>
>>> Commit: Stefan Kangas <stefan@marxist.se>
>>>
>>>     * elpa-packages (anzu): New package
>>
>> Hasn't anzu been obsoleted by isearch-lazy-count?
>
> I don't know; I don't use isearch these days.  I remember using anzu
> when I did.
>
> I add packages mostly based on our formal rules, not on if I think they
> are useful or good.  Some notable exceptions to this are s.el and f.el,
> where we have decided that it is strongly undesirable to promote such
> short package prefixes.  I have also skipped some other packages that
> are clearly obsolete/no longer developed/bad or low effort, etc.

I think this might be worth discussing.  The impression I get is that a
lot of packages/libraries are not being used because they are better,
but because they are still recommended on outdated blog posts and fora
(think of linum-mode vs. display-line-numbers-mode).  More on this
below.

> I am currently focusing on highly popular packages, and anzu clearly fit
> the bill with over 1 million downloads on MELPA.org.  It also seems like
> it is still maintained, with a new maintainer since March 2020, and
> commits as late as October 2021.

The "danger" is that MELPA might give the wrong impression of
popularity: Their download counter is an absolute number, and does not
account for the age of a package or the number of downloads that have
been updates (this skews towards older packages and packages with
frequent updates), or when a package was updated (this can be manually
approximated for some packages via archive.org).

> But feel free to tell me why I'm wrong; I'm not married to it and if you
> think we should remove it again, I don't think I will have any
> objections.

This seems to be more of an general, open question on the direction that
NonGNU ELPA should head:

Do we want to collect as many packages as possible, even if the
implementations and practices are sub-optimal, are displaced by
alternative implementations in Emacs or ELPA, etc. or should we try to
restrict the packages to popular, "good citizens" of the Emacs package
space, in an effort to raise the standards and clean up "obsolete" and
"redundant" packages.  It is probably clear that I have an inclination
towards the latter position: Going forward it seems preferable to have
as many useful and idiomatic packages available directly via the ELPAs,
without burdening newcomers with duplicate functionalities.  My
motivation in contributing to NonGNU ELPA is to further this goal.

From my restricted understanding of Emacs' history (mostly due to my
age), MELPA and EmacsWiki before it contributed a substantial
improvement (more packages, cleaner code, usage of libraries, ...) next
to their respective formal improvements (package.el support, a
centralised location for packages).  It seems to me that MELPA leans
towards completeness, adding anything from serious packages to fun and
jokes.  While I do understand the motivation/advantages, it appears to
fuel the "there is a package for that" mentality (parallel to Apple's
"there is an app for that"-slogan), that implies to download a package
for every little change, instead of writing or even copying some Elisp.
Given that package.el still makes it difficult to maintain your own
changes while updating packages, I understand why people falsely claim
Emacs has a "plug-in" system.

(From what I remember you were intending to present a talk on a related
topic at EmacsConf last year, right?)

> BTW, maybe this should be raised as a ticket on the anzu bug tracker?

What could they do with this information?  I guess they would disagree,
and that would be it.

-- 
	Philip Kaludercic

Packages quality (was: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package)

[Stefan Monnier]

> Do we want to collect as many packages as possible, even if the
> implementations and practices are sub-optimal, are displaced by
> alternative implementations in Emacs or ELPA, etc. or should we try to
> restrict the packages to popular, "good citizens" of the Emacs package
> space, in an effort to raise the standards and clean up "obsolete" and
> "redundant" packages.  It is probably clear that I have an inclination
> towards the latter position: Going forward it seems preferable to have
> as many useful and idiomatic packages available directly via the ELPAs,
> without burdening newcomers with duplicate functionalities.  My
> motivation in contributing to NonGNU ELPA is to further this goal.

Note that (Non)GNU ELPA in the long term will inevitably also contain
old/redundant/outdated packages unless we go and actively remove such
packages (which we haven't done so far).

So, I think if we want to improve the quality, in the long term, the way
to do that is not just by restricting which packages we add, but by
finding ways to regularly re-assess the quality of packages and coming
up with good ways to remove/demote packages based on that (and similarly
promote those packages that are currently particularly good).


        Stefan

Re: Packages quality (was: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package)

[Bozhidar Batsov]

[-- Attachment #1: Type: text/plain, Size: 1705 bytes --]

I'm also curious what "redundant package" even means in this context. There are always many ways to achieve something and usually there's no clear way to decide if some approach is much better than the alternatives. Given how early we are with NonGNU ELPA I think that concerns about "obsolete" and "redundant" packages are quite overdone. 

On Thu, Jan 6, 2022, at 4:30 PM, Stefan Monnier wrote:
> > Do we want to collect as many packages as possible, even if the
> > implementations and practices are sub-optimal, are displaced by
> > alternative implementations in Emacs or ELPA, etc. or should we try to
> > restrict the packages to popular, "good citizens" of the Emacs package
> > space, in an effort to raise the standards and clean up "obsolete" and
> > "redundant" packages.  It is probably clear that I have an inclination
> > towards the latter position: Going forward it seems preferable to have
> > as many useful and idiomatic packages available directly via the ELPAs,
> > without burdening newcomers with duplicate functionalities.  My
> > motivation in contributing to NonGNU ELPA is to further this goal.
> 
> Note that (Non)GNU ELPA in the long term will inevitably also contain
> old/redundant/outdated packages unless we go and actively remove such
> packages (which we haven't done so far).
> 
> So, I think if we want to improve the quality, in the long term, the way
> to do that is not just by restricting which packages we add, but by
> finding ways to regularly re-assess the quality of packages and coming
> up with good ways to remove/demote packages based on that (and similarly
> promote those packages that are currently particularly good).
> 
> 
>         Stefan
> 
> 
> 

[-- Attachment #2: Type: text/html, Size: 2339 bytes --]

Re: Packages quality

[Stefan Monnier]

> I'm also curious what "redundant package" even means in this context. There
> are always many ways to achieve something and usually there's no clear way
> to decide if some approach is much better than the alternatives. Given how
> early we are with NonGNU ELPA I think that concerns about "obsolete" and
> "redundant" packages are quite overdone. 

I don't consider NonGNU ELPA separately from GNU ELPA in this
discussion, and GNU ELPA is just as old as MELPA.

What I'm getting at is that users would benefit from extra info about
the packages, e.g. notions of popularity and health, some lists of
related packages including alternatives.

And it'd be good for us to make efforts at consolidating packages
(i.e. reach out and help package maintainers integrate their new package
with the older one they thought was crap, as happens too often).


        Stefan


> On Thu, Jan 6, 2022, at 4:30 PM, Stefan Monnier wrote:
>> > Do we want to collect as many packages as possible, even if the
>> > implementations and practices are sub-optimal, are displaced by
>> > alternative implementations in Emacs or ELPA, etc. or should we try to
>> > restrict the packages to popular, "good citizens" of the Emacs package
>> > space, in an effort to raise the standards and clean up "obsolete" and
>> > "redundant" packages.  It is probably clear that I have an inclination
>> > towards the latter position: Going forward it seems preferable to have
>> > as many useful and idiomatic packages available directly via the ELPAs,
>> > without burdening newcomers with duplicate functionalities.  My
>> > motivation in contributing to NonGNU ELPA is to further this goal.
>> 
>> Note that (Non)GNU ELPA in the long term will inevitably also contain
>> old/redundant/outdated packages unless we go and actively remove such
>> packages (which we haven't done so far).
>> 
>> So, I think if we want to improve the quality, in the long term, the way
>> to do that is not just by restricting which packages we add, but by
>> finding ways to regularly re-assess the quality of packages and coming
>> up with good ways to remove/demote packages based on that (and similarly
>> promote those packages that are currently particularly good).
>> 
>> 
>>         Stefan
>> 
>> 
>> 

Re: Packages quality

[Tim Cross]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> What I'm getting at is that users would benefit from extra info about
> the packages, e.g. notions of popularity and health, some lists of
> related packages including alternatives.
>

I think this would be very useful. The Arch GNU Linux distro has a
similar concept with their AUR repository. When you are looking to
install a new package, there are often multiple candidates which would
meet the user's requirements. The AUR provides information on how many
installs there are of a package and a basic user 'score' representing
user recommendation/satisfaction (as well as alternative/similar
packages in the repository).  

> And it'd be good for us to make efforts at consolidating packages
> (i.e. reach out and help package maintainers integrate their new package
> with the older one they thought was crap, as happens too often).
>

Yes, if resources are available. Might be useful if there was also some
automated package analysis to help pinpoint packages which may need
attention (for example, prior to a new Emacs release). Even something
basic, such as checking packages for use of functions/variables which
have been flagged as obsolete or compilations with warnings which exceed
some cut off count or linting warning counts, might be useful. While such heuristics would only
provide indicators, they might help prioritise which packages need
attention first or which ones might have more problems after a new Emacs
release etc.

Re: Packages quality

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> What I'm getting at is that users would benefit from extra info about
> the packages, e.g. notions of popularity and health, some lists of
> related packages including alternatives.

Agreed.  We should work on adding such features.

If someone would volunteer to open bug reports with their feature
suggestions, we could take it from there.

> And it'd be good for us to make efforts at consolidating packages
> (i.e. reach out and help package maintainers integrate their new package
> with the older one they thought was crap, as happens too often).

I would be interested in hearing more about this.  Could you expand on
this, or perhaps even point to some examples?

I don't know if this is what you mean, but one example that comes to
mind is how straight.el is not at all integrated with package.el (which
in your opinion, IIRC, could be rectified).

Re: Packages quality

[John Yates]

On Fri, Jan 7, 2022 at 2:56 AM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> I don't know if this is what you mean, but one example that comes to
> mind is how straight.el is not at all integrated with package.el (which
> in your opinion, IIRC, could be rectified).

As a very happy user of straight I feel that Radon Rosborough,
straight's author, is  being unfairly maligned.  He has integrated
straight exactly as John Wiegley intended.  From use-package's
README.md:

Usage with other package managers

By overriding use-package-ensure-function and/or
use-package-pre-ensure-function, other package managers
can override :ensure to use them instead of package.el. At
the present time, the only package manager that does this
is straight.el.

Re: Packages quality

[Stefan Kangas]

John Yates <john@yates-sheets.org> writes:

> On Fri, Jan 7, 2022 at 2:56 AM Stefan Kangas <stefankangas@gmail.com> wrote:
>>
>> I don't know if this is what you mean, but one example that comes to
>> mind is how straight.el is not at all integrated with package.el (which
>> in your opinion, IIRC, could be rectified).
>
> As a very happy user of straight I feel that Radon Rosborough,
> straight's author, is  being unfairly maligned.

I don't understand.  How is it maligning, unfairly or otherwise, the
author of some package to say that it could be integrated with another
package?

> He has integrated straight exactly as John Wiegley intended.  From
> use-package's README.md:

I believe the discussion was about package.el, not use-package.el.

Re: Packages quality

[John Yates]

Stefan, you did indeed write 'package.el' which I then  misread
as 'use-package.el'.  My apologies.

I agree that finding a way to integrate straight-like capabilities
into emac's core package management is much to be desired.

/john

On Fri, Jan 7, 2022 at 7:16 AM Stefan Kangas <stefankangas@gmail.com> wrote:
>
> John Yates <john@yates-sheets.org> writes:
>
> > On Fri, Jan 7, 2022 at 2:56 AM Stefan Kangas <stefankangas@gmail.com> wrote:
> >>
> >> I don't know if this is what you mean, but one example that comes to
> >> mind is how straight.el is not at all integrated with package.el (which
> >> in your opinion, IIRC, could be rectified).
> >
> > As a very happy user of straight I feel that Radon Rosborough,
> > straight's author, is  being unfairly maligned.
>
> I don't understand.  How is it maligning, unfairly or otherwise, the
> author of some package to say that it could be integrated with another
> package?
>
> > He has integrated straight exactly as John Wiegley intended.  From
> > use-package's README.md:
>
> I believe the discussion was about package.el, not use-package.el.



-- 
John Yates
505 Tremont St, #803
Boston, MA 02116

Re: Packages quality (was: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package)

[Stefan Kangas]

"Bozhidar Batsov" <bozhidar@batsov.dev> writes:

> I'm also curious what "redundant package" even means in this
> context. There are always many ways to achieve something and usually
> there's no clear way to decide if some approach is much better than
> the alternatives. Given how early we are with NonGNU ELPA I think that
> concerns about "obsolete" and "redundant" packages are quite overdone.

I don't think it's time to start pruning NGE, indeed, but we could avoid
adding packages that are on MELPA that are already obviously
obsolete/superseded.

For example, I use the software "anki" quite a lot, and there are four
different packages for it.  I have tried all of them, and frankly
speaking only one of those packages is relevant.  Therefore, I would
advise against adding the others; as Philip points out, this is more
likely to waste users time than be very helpful.

Re: Packages quality (was: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package)

[Bozhidar Batsov]

[-- Attachment #1: Type: text/plain, Size: 1550 bytes --]

Fair enough. I'm a bit concerned that often evaluations of the quality/usefulness of something are subjective (e.g. the people who like flymake would argue that flycheck might be "redundant", etc). To me - it's fine to try to solve the same problem in multiple ways if there's no clearly superior way of doing something. I guess a package is never really obsolete until no one uses it and maintains it. 

In general I agree that some quality criteria have to be met for a package to be included on NGE, I just hope that those criteria are going to be of the objective kind. 

On Fri, Jan 7, 2022, at 9:55 AM, Stefan Kangas wrote:
> "Bozhidar Batsov" <bozhidar@batsov.dev> writes:
> 
> > I'm also curious what "redundant package" even means in this
> > context. There are always many ways to achieve something and usually
> > there's no clear way to decide if some approach is much better than
> > the alternatives. Given how early we are with NonGNU ELPA I think that
> > concerns about "obsolete" and "redundant" packages are quite overdone.
> 
> I don't think it's time to start pruning NGE, indeed, but we could avoid
> adding packages that are on MELPA that are already obviously
> obsolete/superseded.
> 
> For example, I use the software "anki" quite a lot, and there are four
> different packages for it.  I have tried all of them, and frankly
> speaking only one of those packages is relevant.  Therefore, I would
> advise against adding the others; as Philip points out, this is more
> likely to waste users time than be very helpful.
> 
> 

[-- Attachment #2: Type: text/html, Size: 2125 bytes --]

Re: Packages quality (was: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package)

[Stefan Kangas]

"Bozhidar Batsov" <bozhidar@batsov.dev> writes:

> Fair enough. I'm a bit concerned that often evaluations of the
> quality/usefulness of something are subjective (e.g. the people who
> like flymake would argue that flycheck might be "redundant", etc). To
> me - it's fine to try to solve the same problem in multiple ways if
> there's no clearly superior way of doing something. I guess a package
> is never really obsolete until no one uses it and maintains it.

FWIW, this makes sense to me.

> In general I agree that some quality criteria have to be met for a
> package to be included on NGE, I just hope that those criteria are
> going to be of the objective kind.

Here's how I see the current situation:

We currently have the luxury to pick and choose which packages get
added.  In the future, users or developers will increasingly ask us to
include this or that package.  I think it will be rare that we should
have any reason to deny such requests.

Currently, I won't spend my limited time working on adding things that I
see little value in, as there are so many important packages that are
still missing.  However, if someone makes an explicit request for
something, I would think about encouraging such participation by
prioritizing it above something that I would otherwise see as higher
priority.

Re: Packages quality

[Philip Kaludercic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Do we want to collect as many packages as possible, even if the
>> implementations and practices are sub-optimal, are displaced by
>> alternative implementations in Emacs or ELPA, etc. or should we try to
>> restrict the packages to popular, "good citizens" of the Emacs package
>> space, in an effort to raise the standards and clean up "obsolete" and
>> "redundant" packages.  It is probably clear that I have an inclination
>> towards the latter position: Going forward it seems preferable to have
>> as many useful and idiomatic packages available directly via the ELPAs,
>> without burdening newcomers with duplicate functionalities.  My
>> motivation in contributing to NonGNU ELPA is to further this goal.
>
> Note that (Non)GNU ELPA in the long term will inevitably also contain
> old/redundant/outdated packages unless we go and actively remove such
> packages (which we haven't done so far).

Of course, but this seems to be a separate issue.  Thinking about if a
package is already deprecated or could be improved now is a different
consideration than the (still real) issue of maintaining and preserving
this for future releases.

> So, I think if we want to improve the quality, in the long term, the way
> to do that is not just by restricting which packages we add, but by
> finding ways to regularly re-assess the quality of packages and coming
> up with good ways to remove/demote packages based on that (and similarly
> promote those packages that are currently particularly good).

What do you mean by promote/demote packages?  Something like a
"featured" section in GNOME Software
(https://wiki.gnome.org/Apps/Software)?

>
>         Stefan
>

-- 
	Philip Kaludercic

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Juri Linkov]

>>>> branch: main
>>>> commit 74116339a852129b98ff79e2eb3aa35b387aa006
>>>> Author: Stefan Kangas <stefan@marxist.se>
>>>> Commit: Stefan Kangas <stefan@marxist.se>
>>>>
>>>>     * elpa-packages (anzu): New package
>>>
>>> Hasn't anzu been obsoleted by isearch-lazy-count?

The external packages often provide more features
that don't exist in core packages.

> The "danger" is that MELPA might give the wrong impression of
> popularity: Their download counter is an absolute number, and does not
> account for the age of a package or the number of downloads that have
> been updates (this skews towards older packages and packages with
> frequent updates), or when a package was updated (this can be manually
> approximated for some packages via archive.org).

Some package archives handle this problem by displaying an additional
counter of downloads over a shorter time period, e.g. the last year.

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Stefan Kangas]

Philip Kaludercic <philipk@posteo.net> writes:

> I think this might be worth discussing.  The impression I get is that a
> lot of packages/libraries are not being used because they are better,

Agreed.

> The "danger" is that MELPA might give the wrong impression of
> popularity:

Yes, I see the same problems and hope that we will eventually be able to
do better on (Non-)GNU ELPA (and I hope MELPA will, too)!  For example,
we could show download count in the last 12 months in addition to the
total downloads.  If we could add a "?major-version=28.1" to the URL or
somesuch, we could even start counting downloads per major version.
(Though I'm not sure if there are any privacy implications, or if it
should be opt-in or opt-out.)

I also think we should have a "rating" system in place, which should
also somehow include information on which version/date the user rated
the package in.

> Do we want to collect as many packages as possible, even if the
> implementations and practices are sub-optimal, are displaced by
> alternative implementations in Emacs or ELPA, etc. or should we try to
> restrict the packages to popular, "good citizens" of the Emacs package
> space, in an effort to raise the standards and clean up "obsolete" and
> "redundant" packages.  It is probably clear that I have an inclination
> towards the latter position: Going forward it seems preferable to have
> as many useful and idiomatic packages available directly via the ELPAs,
> without burdening newcomers with duplicate functionalities.  My
> motivation in contributing to NonGNU ELPA is to further this goal.

I basically agree.  I won't be adding redundant or novelty packages
myself -- unless I do it by mistake! -- but there is also some balance:
I don't want to impose my opinions on everyone else.

So when it comes to "evil-*" packages, I feel like I did my part by
adding the 10-15 most downloaded ones on MELPA.  I don't use evil, so I
have no other criteria to go by, really.

But when it comes to packages I do use, like ace-jump, I noticed that it
had been superseded by avy.  So I added this to my private NGE notes:

    *** Packages not to add
    - ace-jump: seems superseded by avy (in GNU ELPA)

I'm not sure if we should have a public record of packages we decide
*not* to add.  If we did, one place to put it would be just directly
`nongnu/elpa-packages` itself, in the regular alphabetical order, which
is a place you're less likely to forget looking in.

>  It seems to me that MELPA leans
> towards completeness, adding anything from serious packages to fun and
> jokes.

I think we should take the middle ground: let's not *only* add the most
technically sound and excellent packages, because you'll end up with a
lot of stuff that is actually used by people that won't make the cut.

But let's also steer clear of the obvious low
effort/patch/bad/novelty/obsolete/unmaintained packages.

I *hope* that in general you will find that my additions have been
balanced, but please point out if you see any examples (such as anzu)
that you think we could discuss.

>  While I do understand the motivation/advantages, it appears to
> fuel the "there is a package for that" mentality (parallel to Apple's
> "there is an app for that"-slogan), that implies to download a package
> for every little change, instead of writing or even copying some Elisp.
[snip]
> (From what I remember you were intending to present a talk on a related
> topic at EmacsConf last year, right?)

Correct.  Unfortunately, I had two consecutive colds and couldn't make
it.  But as you can imagine, I strongly agree with you.  ;-)

I am not against any efforts in this direction, of course, and I would
welcome any kind of initiative to do "community outreach" here.
However, I think the place when we can really start doing such work is
when people start coming to *us* to ask to be added, and not the other
way around.  This will happen eventually, I think, but we have a bit of
an up-hill battle to get the first critical mass (getting Emacs 28, 29,
30 out the door will help of course).

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Philip Kaludercic]

Stefan Kangas <stefan@marxist.se> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> I think this might be worth discussing.  The impression I get is that a
>> lot of packages/libraries are not being used because they are better,
>
> Agreed.
>
>> The "danger" is that MELPA might give the wrong impression of
>> popularity:
>
> Yes, I see the same problems and hope that we will eventually be able to
> do better on (Non-)GNU ELPA (and I hope MELPA will, too)!  For example,
> we could show download count in the last 12 months in addition to the
> total downloads.  If we could add a "?major-version=28.1" to the URL or
> somesuch, we could even start counting downloads per major version.
> (Though I'm not sure if there are any privacy implications, or if it
> should be opt-in or opt-out.)

Interesting idea, though it should probably be opt-in, at the risk of
skewing the popularity data towards enthusiasts.  Just one thing: It
seems like this would either require a special server or to scrub the
server logs.  Is either of that feasible?

> I also think we should have a "rating" system in place, which should
> also somehow include information on which version/date the user rated
> the package in.

Would also be nice. 

>> Do we want to collect as many packages as possible, even if the
>> implementations and practices are sub-optimal, are displaced by
>> alternative implementations in Emacs or ELPA, etc. or should we try to
>> restrict the packages to popular, "good citizens" of the Emacs package
>> space, in an effort to raise the standards and clean up "obsolete" and
>> "redundant" packages.  It is probably clear that I have an inclination
>> towards the latter position: Going forward it seems preferable to have
>> as many useful and idiomatic packages available directly via the ELPAs,
>> without burdening newcomers with duplicate functionalities.  My
>> motivation in contributing to NonGNU ELPA is to further this goal.
>
> I basically agree.  I won't be adding redundant or novelty packages
> myself -- unless I do it by mistake! -- but there is also some balance:
> I don't want to impose my opinions on everyone else.
>
> So when it comes to "evil-*" packages, I feel like I did my part by
> adding the 10-15 most downloaded ones on MELPA.  I don't use evil, so I
> have no other criteria to go by, really.

If we have Evil, it seems necessary to add some of extensions, as Evil
appears to not be complete on it's own.  But as this is an entire
package-space onto itself, I am uncertain which of these are actually
being used and which are obsoleted by either other evil-specific
packages, other general packages or even features in Emacs.

> But when it comes to packages I do use, like ace-jump, I noticed that it
> had been superseded by avy.  So I added this to my private NGE notes:
>
>     *** Packages not to add
>     - ace-jump: seems superseded by avy (in GNU ELPA)
>
> I'm not sure if we should have a public record of packages we decide
> *not* to add.  If we did, one place to put it would be just directly
> `nongnu/elpa-packages` itself, in the regular alphabetical order, which
> is a place you're less likely to forget looking in.

It might not be bad to collect these notes somewhere.  I also have my
private notes, and if more people want to contribute, being able to
quickly check what the status of a package is (waiting for a patch to be
applied, obsoleted, ...) would be useful.

>>  It seems to me that MELPA leans
>> towards completeness, adding anything from serious packages to fun and
>> jokes.
>
> I think we should take the middle ground: let's not *only* add the most
> technically sound and excellent packages, because you'll end up with a
> lot of stuff that is actually used by people that won't make the cut.
>
> But let's also steer clear of the obvious low
> effort/patch/bad/novelty/obsolete/unmaintained packages.
>
> I *hope* that in general you will find that my additions have been
> balanced, but please point out if you see any examples (such as anzu)
> that you think we could discuss.

Most of the packages certainly are good and appreciated additions, and
would have liked to help if I had more time.

>>  While I do understand the motivation/advantages, it appears to
>> fuel the "there is a package for that" mentality (parallel to Apple's
>> "there is an app for that"-slogan), that implies to download a package
>> for every little change, instead of writing or even copying some Elisp.
> [snip]
>> (From what I remember you were intending to present a talk on a related
>> topic at EmacsConf last year, right?)
>
> Correct.  Unfortunately, I had two consecutive colds and couldn't make
> it.  But as you can imagine, I strongly agree with you.  ;-)
>
> I am not against any efforts in this direction, of course, and I would
> welcome any kind of initiative to do "community outreach" here.

What do you mean by this?

> However, I think the place when we can really start doing such work is
> when people start coming to *us* to ask to be added, and not the other
> way around.  This will happen eventually, I think, but we have a bit of
> an up-hill battle to get the first critical mass (getting Emacs 28, 29,
> 30 out the door will help of course).

What difference would this make? I agree that it would be preferable,
but a tone for what packages are added to NonGNU ELPA can already be
set.

On the topic of critical mass:  It might be possible to spread NonGNU
ELPA via compat (https://git.sr.ht/~pkal/compat), as soon as it is
released.  It seems that there is already some interest in using this
package, most notably from transient.  But as the package is already
borderline heretical, I hesitate to do something like this (the same
applies to updating the default rcirc and erc server lists).

-- 
	Philip Kaludercic

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Stefan Monnier]

> Interesting idea, though it should probably be opt-in, at the risk of
> skewing the popularity data towards enthusiasts.  Just one thing: It
> seems like this would either require a special server or to scrub the
> server logs.  Is either of that feasible?

The webserver on elpa.gnu.org is under our control, so yes we have
access to the log.  I believe the same holds for MELPA.

As for "opt-in", I don't see any need to add user-control to it, but we
should rather show more complete data (like a histogram) so the users
can get a better sense of what's going on.

>> I also think we should have a "rating" system in place, which should
>> also somehow include information on which version/date the user rated
>> the package in.
> Would also be nice.

I don't know how to do that, OTOH (at least in a way that's not too
susceptible to ridiculous amounts of bias).

> If we have Evil, it seems necessary to add some of extensions, as Evil
> appears to not be complete on it's own.  But as this is an entire
> package-space onto itself, I am uncertain which of these are actually
> being used and which are obsoleted by either other evil-specific
> packages, other general packages or even features in Emacs.

There seems to be a fair bit of duplication here, in that many of those
Evil-specific packages would benefit from being made non-specific to
Evil (or merged with similar non-Evil-specific packages).

>> I'm not sure if we should have a public record of packages we decide
>> *not* to add.  If we did, one place to put it would be just directly
>> `nongnu/elpa-packages` itself, in the regular alphabetical order, which
>> is a place you're less likely to forget looking in.
> It might not be bad to collect these notes somewhere.

I agree with Stefan (the usurper) in that `elpa-packages` is a good
place for that.

> On the topic of critical mass:  It might be possible to spread NonGNU
> ELPA via compat (https://git.sr.ht/~pkal/compat), as soon as it is
> released.

I don't understand hat you meant by that.

Do you mean to have `compat` do

    (add-to-list 'package-archives '("nongnu" . ...))

?

> But as the package is already borderline heretical, I hesitate to do
> something like this (the same applies to updating the default rcirc
> and erc server lists).

Maybe we can make it safe enough via something like

    (when (member package-archives
                  '((("gnu" . "https://..."))
                    (("gnu" . "http://..."))))
      (add-to-list 'package-archives '("nongnu" . ...)))

so it only adds the entry when nothing's been changed yet.


        Stefan

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Philip Kaludercic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> I also think we should have a "rating" system in place, which should
>>> also somehow include information on which version/date the user rated
>>> the package in.
>> Would also be nice.
>
> I don't know how to do that, OTOH (at least in a way that's not too
> susceptible to ridiculous amounts of bias).

"Bias" in what sense?

>>> I'm not sure if we should have a public record of packages we decide
>>> *not* to add.  If we did, one place to put it would be just directly
>>> `nongnu/elpa-packages` itself, in the regular alphabetical order, which
>>> is a place you're less likely to forget looking in.
>> It might not be bad to collect these notes somewhere.
>
> I agree with Stefan (the usurper) in that `elpa-packages` is a good
> place for that.

Ok, I am fine with that.

>> But as the package is already borderline heretical, I hesitate to do
>> something like this (the same applies to updating the default rcirc
>> and erc server lists).
>
> Maybe we can make it safe enough via something like
>
>     (when (member package-archives
>                   '((("gnu" . "https://..."))
>                     (("gnu" . "http://..."))))
>       (add-to-list 'package-archives '("nongnu" . ...)))
>
> so it only adds the entry when nothing's been changed yet.

Yes, this was my idea.

-- 
	Philip Kaludercic

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Stefan Monnier]

>> I don't know how to do that, OTOH (at least in a way that's not too
>> susceptible to ridiculous amounts of bias).
> "Bias" in what sense?

In the sense that some users may be given a lot more weight than others,
or in the sense of being susceptible to conscious efforts to influence
the outcome (like a user writing a script that votes a million times
because they really like that SuperFoo package and they're frustrated
that it scores below package SimpleFoo).

To some extent it's unavoidable (counting downloads is also subject to
those kinds of problems), but arguably less so than a pure voting
system, because it comes as a side effect of a real action.

>>> But as the package is already borderline heretical, I hesitate to do
>>> something like this (the same applies to updating the default rcirc
>>> and erc server lists).
>>
>> Maybe we can make it safe enough via something like
>>
>>     (when (member package-archives
>>                   '((("gnu" . "https://..."))
>>                     (("gnu" . "http://..."))))
>>       (add-to-list 'package-archives '("nongnu" . ...)))
>>
>> so it only adds the entry when nothing's been changed yet.
>
> Yes, this was my idea.

Than it looks quite safe and acceptable to me.


        Stefan

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Stefan Kangas]

Philip Kaludercic <philipk@posteo.net> writes:

> Interesting idea, though it should probably be opt-in, at the risk of
> skewing the popularity data towards enthusiasts.  Just one thing: It
> seems like this would either require a special server or to scrub the
> server logs.  Is either of that feasible?

I don't know what you mean by scrubbing the server logs, but my idea is
that we just extract whatever data we need from it, while ensuring that
we remove any personal identifiers.

>  But as this is an entire
> package-space onto itself, I am uncertain which of these are actually
> being used and which are obsoleted by either other evil-specific
> packages, other general packages or even features in Emacs.

Yes, but at the same time I don't think it's too bad if we catch the odd
package that perhaps shouldn't have made it.  At some point in the
future, we will want to implement some mechanism for obsoleting
packages, so we will be able to fix that.  Eventually, hopefully.

> It might not be bad to collect these notes somewhere.  I also have my
> private notes, and if more people want to contribute, being able to
> quickly check what the status of a package is (waiting for a patch to be
> applied, obsoleted, ...) would be useful.

I wouldn't mind just keeping a separate org-file in nongnu.git or even
just notes directly in the elpa-packages file.  The main problem will be
to ensure that such notes don't go too wildly out of date.

>> I am not against any efforts in this direction, of course, and I would
>> welcome any kind of initiative to do "community outreach" here.
>
> What do you mean by this?

I think it would be beneficial to think of ways to encourage people to
consider sending patches to Emacs or packages instead of writing up yet
another package.  This could be in the form of posts on Reddit, IRC,
blogs, etc.

>  I agree that it would be preferable,
> but a tone for what packages are added to NonGNU ELPA can already be
> set.

I think we agree; my point is merely that the tone we want to set will
carry more weight to the extent that NonGNU ELPA is successful and
widely used.

>  It might be possible to spread NonGNU
> ELPA via compat (https://git.sr.ht/~pkal/compat), as soon as it is
> released.  It seems that there is already some interest in using this
> package, most notably from transient.  But as the package is already
> borderline heretical, I hesitate to do something like this (the same
> applies to updating the default rcirc and erc server lists).

Personally, I would just do it.  The new defaults are arguably just
better in those cases; no one will benefit from not having NonGNU ELPA
or trying to connect to an almost dead IRC network.

BTW, could you consider adding the fix for the vulnerability in
enriched-text-mode on Emacs < 25.3, as detailed in etc/NEWS.25?

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Ergus]

>
>I think it would be beneficial to think of ways to encourage people to
>consider sending patches to Emacs or packages instead of writing up yet
>another package.  This could be in the form of posts on Reddit, IRC,
>blogs, etc.
>


To not reinvent the wheel maybe look at the AUR repository page for Arch Linux.

AUR repository just have a button on every package's page where the users can report obsolete outdated packages. And where the package maintainer can adopt or abandon (orphan) the package.

When there are issues usually the same users can reply and propose solutions or workarounds to the others or some patches.

Then when a user updates or install those packages a warning is printed if the package is outdated or orphan. 







-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Philip Kaludercic]

Ergus <spacibba@aol.com> writes:

>>
>>I think it would be beneficial to think of ways to encourage people to
>>consider sending patches to Emacs or packages instead of writing up yet
>>another package.  This could be in the form of posts on Reddit, IRC,
>>blogs, etc.
>>
>
>
> To not reinvent the wheel maybe look at the AUR repository page for Arch Linux.
>
> AUR repository just have a button on every package's page where the
> users can report obsolete outdated packages. And where the package
> maintainer can adopt or abandon (orphan) the package.

I don't think that "obsolete" or "outdated" means the same thing here.
While the AUR is a collection of build-scripts that might fall behind an
upstream source, I was thinking more of packages that have been
"obsoleted" by upstream changes, or by other packages (think of Avy
vs. ace-jump, where I think the consensus is that Avy has superseded
ace-jump).

> When there are issues usually the same users can reply and propose solutions or workarounds to the others or some patches.
>
> Then when a user updates or install those packages a warning is printed if the package is outdated or orphan. 

-- 
	Philip Kaludercic

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Stefan Monnier]

>>> I am not against any efforts in this direction, of course, and I would
>>> welcome any kind of initiative to do "community outreach" here.
>> What do you mean by this?
> I think it would be beneficial to think of ways to encourage people to
> consider sending patches to Emacs or packages instead of writing up yet
> another package.  This could be in the form of posts on Reddit, IRC,
> blogs, etc.

I think in many cases it's illusory to expect this because it's easier
for the users to write their own package custom-tailored to their
specific case (oftentimes without realizing to what extent it's custom
tailored) than to try and understand the generic code which has to take
into account many more cases that they've never even dreamed of.

But we should at least aim to make it so those users find it natural to
send us a bug report showing describing the problem they encountered and
the way they fixed it in their package.  And also make sure that we then
followup on it, to try and integrate a fix or the features, encouraging
him to take part in it, without necessarily forcing him to do the work.

> Personally, I would just do it.  The new defaults are arguably just
> better in those cases; no one will benefit from not having NonGNU ELPA
> or trying to connect to an almost dead IRC network.

Actually for `package-archives` I was thinking of doing it in
`gnu-elpa-keyring-update`.


        Stefan

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Philip Kaludercic]

Stefan Kangas <stefan@marxist.se> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> Interesting idea, though it should probably be opt-in, at the risk of
>> skewing the popularity data towards enthusiasts.  Just one thing: It
>> seems like this would either require a special server or to scrub the
>> server logs.  Is either of that feasible?
>
> I don't know what you mean by scrubbing the server logs, but my idea is
> that we just extract whatever data we need from it, while ensuring that
> we remove any personal identifiers.

Not sure why I wrote "scrubbing", I just meant to say that since
the packages are to my knowledge hosted statically, one would have to
use the HTTP logs to check for queries in the URL.

>> It might not be bad to collect these notes somewhere.  I also have my
>> private notes, and if more people want to contribute, being able to
>> quickly check what the status of a package is (waiting for a patch to be
>> applied, obsoleted, ...) would be useful.
>
> I wouldn't mind just keeping a separate org-file in nongnu.git or even
> just notes directly in the elpa-packages file.  The main problem will be
> to ensure that such notes don't go too wildly out of date.

In that case elpa-packages might be preferable.

>>  I agree that it would be preferable,
>> but a tone for what packages are added to NonGNU ELPA can already be
>> set.
>
> I think we agree; my point is merely that the tone we want to set will
> carry more weight to the extent that NonGNU ELPA is successful and
> widely used.

Yes, I agree.

>>  It might be possible to spread NonGNU
>> ELPA via compat (https://git.sr.ht/~pkal/compat), as soon as it is
>> released.  It seems that there is already some interest in using this
>> package, most notably from transient.  But as the package is already
>> borderline heretical, I hesitate to do something like this (the same
>> applies to updating the default rcirc and erc server lists).
>
> Personally, I would just do it.  The new defaults are arguably just
> better in those cases; no one will benefit from not having NonGNU ELPA
> or trying to connect to an almost dead IRC network.

Ok, I will use this thread as an excuse if anyone were to complain ^^.

> BTW, could you consider adding the fix for the vulnerability in
> enriched-text-mode on Emacs < 25.3, as detailed in etc/NEWS.25?

I can take a look at that, thanks for the reminder!

-- 
	Philip Kaludercic

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Jean Louis]

> > I also think we should have a "rating" system in place, which should
> > also somehow include information on which version/date the user rated
> > the package in.
> 
> Would also be nice. 

While it is good idea, it would be good to add the care of genuity of
such ratings.

If such rating system goes over HTTP anonymously and without
registration, that opens door to false statistics, thus false reports.

It is good to have genuine rating system that would tell something
about background of the rating.

If such rating is automated then full description of the rating system
in simple English shall be provided.

Let me give the example of what I think is false report based on
download count on melpa.org domain, that is similar to ratings
system. It gives download count.

> dash, A modern list library for Emacs, 20210826.1149, github, download
> count being 3,951,288

That line alone is vague to its meanings:

- I am asking myself, was the version 20210826.1149 downloaded
  3,951,288 times?

- or was that the total download count for all versions?

- then I am pretty sure that "dash" is not a package users would
  download manually, rather automatically

- it appears most popular package by download count, while it is
  probably not well mentioned package among people in their written
  and verbal communication;

Thus ratings method in ELPA shall be described in public for people to
understand the background.






-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Philip Kaludercic]

I agree with all of this, but this seems to imply that some sort of
identification would be necessary (I would guess automatised, using some
kind of public key cryptography, instead of manually with accounts).

Jean Louis <bugs@gnu.support> writes:

>> > I also think we should have a "rating" system in place, which should
>> > also somehow include information on which version/date the user rated
>> > the package in.
>> 
>> Would also be nice. 
>
> While it is good idea, it would be good to add the care of genuity of
> such ratings.
>
> If such rating system goes over HTTP anonymously and without
> registration, that opens door to false statistics, thus false reports.
>
> It is good to have genuine rating system that would tell something
> about background of the rating.
>
> If such rating is automated then full description of the rating system
> in simple English shall be provided.
>
> Let me give the example of what I think is false report based on
> download count on melpa.org domain, that is similar to ratings
> system. It gives download count.
>
>> dash, A modern list library for Emacs, 20210826.1149, github, download
>> count being 3,951,288
>
> That line alone is vague to its meanings:
>
> - I am asking myself, was the version 20210826.1149 downloaded
>   3,951,288 times?
>
> - or was that the total download count for all versions?
>
> - then I am pretty sure that "dash" is not a package users would
>   download manually, rather automatically
>
> - it appears most popular package by download count, while it is
>   probably not well mentioned package among people in their written
>   and verbal communication;
>
> Thus ratings method in ELPA shall be described in public for people to
> understand the background.

-- 
	Philip Kaludercic

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Stefan Kangas]

Philip Kaludercic <philipk@posteo.net> writes:

> What could they do with this information?  I guess they would disagree,
> and that would be it.

If they agree, they could say in their README that their package is
obsolete with Emacs version NN or later.

But Juri says that it adds more features that we don't yet have in Emacs
OOTB, so I don't know if that is true.

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Stefan Monnier]

Stefan Kangas [2022-01-07 04:02:25] wrote:
> But Juri says that it adds more features that we don't yet have in Emacs
> OOTB, so I don't know if that is true.

That's almost always the case (e.g. linum.el, nlinum.el).

It's almost always impossible to provide a good replacement that is
really a strict superset (e.g. I tried to make `nlinum.el` a strict
superset of `linum.el` but that required very significant changes which
reintroduced some of the problems I wanted to get away from when
I decided to implement `nlinum.el`).


        Stefan

Re: [nongnu] main 74116339a8 2/3: * elpa-packages (anzu): New package

[Augusto Stoffel]

On Thu,  6 Jan 2022 at 10:02, Philip Kaludercic <philipk@posteo.net> wrote:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>> branch: main
>> commit 74116339a852129b98ff79e2eb3aa35b387aa006
>> Author: Stefan Kangas <stefan@marxist.se>
>> Commit: Stefan Kangas <stefan@marxist.se>
>>
>>     * elpa-packages (anzu): New package
>
> Hasn't anzu been obsoleted by isearch-lazy-count?

Anzu also provides lazy count/highlight for `query-replace' and friends,
as well as previews of the replacements.

This seems like a good feature to introduce in Emacs 29.  I started
working on this a while back, but it's quite stressful to modify
isearch.el and ensure nothing broke in the process.