RE: Imports / inclusion of s.el into Emacs

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> > Renaming 'concat' seems also like spurious inconvenience
> > in the name of rigidity -- the Lisp equivalent of 
> > bureaucratese.

Yes.  Spurious inconvenience for users, even if
well-intentioned.

> We all do keep in mind that 'concat' is about much
> more than strings, yes?  Such polymorphic APIs are
> IMO a problem when a simplistic re/naming is considered.

Exactly my thought.  Likewise, `format' - an even
better example.
___

The idea that every function that returns an XYZ
instance, or that operates on one, should have a
name prefix (e.g. `xyz-') that advertises that
fact is, well, somewhat silly, and it can border
on dogmatism.

My guess is that this is coming, not just from an
Emacs UI question involving prefix completion and
discoverability, but more generally from the
widespread influence of OOP, where categorizing
or grouping of functions (methods) etc. by type
(class) of object acted on or returned has become
ingrained mentally as the only or the best or the
main way to organize them. 

That things _can_ be usefully grouped thus is not
a reason why they always should or must.  That may
be a requirement in some OOP languages, but it's
not so in theory or in other programming contexts.

Functions are more varied than what's necessary to
define an abstract data type (class).  Yes, a
particular set of functions do define an ADT, but
other functions can also make use of it.

It would be absurd to name every function that
uses or returns an integer with prefix `int-'.
(Polymorphism can help get around this in OOP.) 

In Haskell, do you name every function with a
prefix that advertises the type of its return
value or one of its main arguments?  No, of course
not.  How do you find functions that return or use
a value of a given type?  You check signatures
or doc.

As for "alist", "ass(q|oc)", and the like: each
such choice has its drawbacks.  But yes, some name
consistency can help, in general.  But no, because
history.  And no, because different uses/contexts.

The discussion was motivated by considering new
users, in particular.  Well then, what does a new
user look for, when it comes to alists?  Does s?he
know the term "alist"?  Or is the thought about
"association"? "key-value"? "pair"? "relation"?
"attribute-value"? "property-value"?

Consider "plists".  Does a newbie whose heard
something about them and tries to find relevant
functions look for "plist"? "property"? "prop"?
"tag"? something else?

There is no blanket, simple "solution" to this,
because people come from different places/contexts
and think in different terms/languages.  Should
Emacs generally be consistent?  Sure.  Could it
have been more consistent at the outset (Lisp Day
One)?  Probably.  Is there now, here & there, room
for improvement in naming functions?  Sure.

Should we be wholesale-renaming stuff, especially
to try to fit what the latest generation of
newbies might be most used to?  Absolutely not.
Names should be considered case by case.  And
there's zero "fierce urgency of now" for any
particular renaming Long March (initiative).
___

Don't downplay the use of matching against more
than the function name.  As Eli mentioned:
`C-h d', which matches also against a function's
doc.  And as I and others have mentioned: more
flexible matching methods.

Consider `C-h f alist', with substring matching.
In my current session I get 156 candidates, all
of which are relevant to alists.  Great.

If I match `ass' instead, I get 235 candidates,
many of which are about alists, but many of which
are not.  If I then chip away from those matches
the matches for `class' and `pass', I get only 51
matches, of which 34 are about alists.

(The 17 matches that are not about alists:
ad-assemble-advised-definition
2C-associated-buffer
ad-assemble-advised-definition
assert
byte-compile-associative
byte-optimize-associative-math
byte-optimize-nonassociative-math
cl--assertion-failed
cl-assert
disassemble
disassemble-offset
glasses-mode
gnus-atomic-progn-assign
image-dired-associated-dired-buffer
image-dired-associated-dired-buffer-window
message-disassociate-draft
nndraft-request-associate-buffer)

If I use `C-h d alist' I find 560 functions
whose doc mentions "alist".
`C-h d association list' finds 56 functions.
`C-h d assoc' finds 52 functions.
___

Finally, the habit of assuming that functions
should be grouped only by argument or return
type, either in terms of name or doc, is not
a guide.  API doc (e.g. JavaDoc) does have its
uses.  And it's especially useful, perhaps
even necessary, for certain kinds of languages
(e.g. OOP).  But it's certainly not the be-all
and end-all.

Lisp is not Java.  Discoverability for Lisp is
not discoverability for Java.  Doc for Lisp is
not doc for Java.

It does take some time for a newbie to learn
to "ask Emacs", including wrt using Lisp to
ask.  And sure, we can help ease entry.  But
that's not a reason to try to mold Emacs help
(including naming) into only what newbies
might be used to in other contexts.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 2434 bytes --]

>
> In Haskell, do you name every function with a
> prefix that advertises the type of its return
> value or one of its main arguments?  No, of course
> not.  How do you find functions that return or use
> a value of a given type?  You check signatures
> or doc.
>

Except if I'm missing somethign Haskell also groups related functions
together:
https://hackage.haskell.org/package/strings-1.1/docs/Data-Strings.html



> As for "alist", "ass(q|oc)", and the like: each
> such choice has its drawbacks.  But yes, some name
> consistency can help, in general.  But no, because
> history.  And no, because different uses/contexts.
>
> The discussion was motivated by considering new
> users, in particular.  Well then, what does a new
> user look for, when it comes to alists?  Does s?he
> know the term "alist"?  Or is the thought about
> "association"? "key-value"? "pair"? "relation"?
> "attribute-value"? "property-value"?
>

It has already been said that alist was a bad example. Consider process or
regexp instead.



> Consider `C-h f alist', with substring matching.
> In my current session I get 156 candidates, all
> of which are relevant to alists.  Great.
>
> If I match `ass' instead, I get 235 candidates,
> many of which are about alists, but many of which
> are not.  If I then chip away from those matches
> the matches for `class' and `pass', I get only 51
> matches, of which 34 are about alists.
>
> If I use `C-h d alist' I find 560 functions
> whose doc mentions "alist".
> `C-h d association list' finds 56 functions.
> `C-h d assoc' finds 52 functions.
>

When I see this it only confuses me. Sure those are great tools to find
things in a broad sense, but there are so much noise that you have to
filter. The only relevant functions I'd like to find are:

assoc, rassoc, assq, alist-get, rassq, assoc-default, copy-alist,
assq-delete-all, assoc-delete-all, rassq-delete-all

Anything else listed there is noise IMHO, except if searching in a broad
scope.



> Finally, the habit of assuming that functions
> should be grouped only by argument or return
> type, either in terms of name or doc, is not
> a guide.  API doc (e.g. JavaDoc) does have its
> uses.  And it's especially useful, perhaps
> even necessary, for certain kinds of languages
> (e.g. OOP).  But it's certainly not the be-all
> and end-all.
>

Agreed. Still I think there is a lot of room for improvement, without going
"all the way".

Philippe

[-- Attachment #2: Type: text/html, Size: 3691 bytes --]

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

[Please use plain-text, not HTML.]

>> In Haskell, do you name every function with a
>> prefix that advertises the type of its return
>> value or one of its main arguments?  No, of course
>> not.  How do you find functions that return or use
>> a value of a given type?  You check signatures
>> or doc.
>
> Except if I'm missing somethign Haskell also groups related functions together: https://urldefense.com/v3/__https://hackage.haskell.org/package/strings-1.1/docs/Data-Strings.html__;!!GqivPVa7Brio!LCpoSrliut7R6447MW0bxoZVqjxyDCCan6SeRxySTEZFcwKTa4KnqazRkAcDMCGd$

According to what you've said before, instead of
those functions being named `text', `lazyText',
`bytes', `lazyBytes', `charToByte', and
`byteToChar' they should be named `string-text',
`string-lazyText', `string-bytes',
`string-lazyBytes', `string-charToByte',
and `string-byteToChar'.

Re: Imports / inclusion of s.el into Emacs

[Stefan Kangas]

Drew Adams <drew.adams@oracle.com> writes:

> According to what you've said before, instead of
> those functions being named `text', `lazyText',
> `bytes', `lazyBytes', `charToByte', and
> `byteToChar' they should be named `string-text',
> `string-lazyText', `string-bytes',
> `string-lazyBytes', `string-charToByte',
> and `string-byteToChar'.

Please have another look beyond the first few examples on that page
where the "str" and "s" prefix feature prominently.

Also note the different kinds of uses for the function you mention
(e.g. "bytes :: String -> ByteString") and the examples further down
(e.g. "strTrim :: a -> a").  This shows that the library designers
have been judicious in their use of prefixes.  We would probably be
wise to exercise similar care.

Best regards,
Stefan Kangas

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1063 bytes --]

On Sat, May 2, 2020 at 10:53 PM Drew Adams <drew.adams@oracle.com> wrote:
>
> [Please use plain-text, not HTML.]
>
> >> In Haskell, do you name every function with a
> >> prefix that advertises the type of its return
> >> value or one of its main arguments?  No, of course
> >> not.  How do you find functions that return or use
> >> a value of a given type?  You check signatures
> >> or doc.
> >
> > Except if I'm missing somethign Haskell also groups related functions
together:
https://urldefense.com/v3/__https://hackage.haskell.org/package/strings-1.1/docs/Data-Strings.html__;!!GqivPVa7Brio!LCpoSrliut7R6447MW0bxoZVqjxyDCCan6SeRxySTEZFcwKTa4KnqazRkAcDMCGd$
>
> According to what you've said before, instead of
> those functions being named `text', `lazyText',
> `bytes', `lazyBytes', `charToByte', and
> `byteToChar' they should be named `string-text',
> `string-lazyText', `string-bytes',
> `string-lazyBytes', `string-charToByte',
> and `string-byteToChar'.

No, they all are in the namespace Data.Strings

I'm not an haskell expert so I might be wrong.

[-- Attachment #2: Type: text/html, Size: 1783 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 23:50, Drew Adams wrote:
> According to what you've said before, instead of
> those functions being named `text', `lazyText',
> `bytes', `lazyBytes', `charToByte', and
> `byteToChar' they should be named `string-text',
> `string-lazyText', `string-bytes',
> `string-lazyBytes', `string-charToByte',
> and `string-byteToChar'.

You're missing the point: all this functions are in the same namespace 
(Data.Strings). That's why they're all on the same page there.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 2 May 2020 22:30:49 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, Richard Stallman <rms@gnu.org>, Emacs developers <emacs-devel@gnu.org>
> 
> When I see this it only confuses me. Sure those are great tools to find things in a broad sense, but there are
> so much noise that you have to filter. The only relevant functions I'd like to find are:
> 
> assoc, rassoc, assq, alist-get, rassq, assoc-default, copy-alist, assq-delete-all, assoc-delete-all,
> rassq-delete-all

How is Emacs supposed to know that only these are expected, and all
the rest are "noise"?  Here are a few other functions related to
alists that might be what you are looking for:

  alist-get
  nested-alist-p
  set-nested-alist

And that's just in "emacs -Q".

If we want more spot-on results, we should improve our completion
scoring.

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> In Haskell, do you name every function with a
> prefix that advertises the type of its return
> value or one of its main arguments?

Like in all the other languages with
modules/packages/classes/namespaces/structures/clusters, yes.


        Stefan


PS: In case you're not familiar with some of those terms, "structures"
is the term used in Standard ML (and then OCaml) and "clusters" was used
in CLU which was one of the precursors in this area (which had
a profound influence, tho AFAIK noone kept that name).

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> In Haskell, do you name every function with a
> prefix that advertises the type of its return
> value or one of its main arguments?

[... replies pointing out that some Haskell
 functions do prefix their names with type names...]

I suggest those of you who think that Haskell
too deserves type-name prefixes for all its
function names start an initiative to rename
its `map', `add', `zip', `head', ... functions.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1335 bytes --]

> > In Haskell, do you name every function with a
> > prefix that advertises the type of its return
> > value or one of its main arguments?
>
> [... replies pointing out that some Haskell
>  functions do prefix their names with type names...]
>
> I suggest those of you who think that Haskell
> too deserves type-name prefixes for all its
> function names start an initiative to rename
> its `map', `add', `zip', `head', ... functions.
>

https://hackage.haskell.org/package/containers-0.6.2.1/docs/doc-index-M.html

Map is always namespaced here (look at the right)? I expect other functions
to be too.

Anyway, you try to focus on the weak points of the arguments. I understand,
it's a natural defense mechanism. What I think is more relevant is to
notice how in Haskell there is a strong will to group things together.
That's what you'd take out of the example.

That said, yes, some generic functions will have troube finding a good
home. We already addressed this point several time but here it is once
more: those function could either be left untouched, or they could be moved
to `seq-`, or yet another solution...

But I suggest we don't talk about these corner cases for and and focus only
on concrete proposals: What we take from s.el and put under the string-
namespace, how we alias some of the regexp functions.

Philippe

[-- Attachment #2: Type: text/html, Size: 1839 bytes --]

Re: Imports / inclusion of s.el into Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 531 bytes --]

On Sun, May 03, 2020 at 09:33:06AM +0200, Philippe Vaucher wrote:

[...]

> Anyway, you try to focus on the weak points of the arguments. I understand,
> it's a natural defense mechanism.

Here it is again.

See, I have no strong part in this discussion [1]. I just watch the
kind of interaction.

Psychologizing your opponent is dangerous, may be insulting and won't
bring the quest forward.

Cheers

[1] Well, actually I have a bit. I'm for change, but for adaptive,
   slow, well-measured change. So somewhere in-between.

-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1714 bytes --]

>> Anyway, you try to focus on the weak points of the arguments. I
understand,
>> it's a natural defense mechanism.

> Here it is again.

> Psychologizing your opponent is dangerous, may be insulting and won't
> bring the quest forward.

Oops, seems I derailed again. Thanks for pointing these out, it helps me
notice.

I have problems when someone looks to me like putting a lot of efforts into
not seeing where we are talking from, and only pointing things from
their perspective (all this while, I think, we try to understand theirs).
But I do realise now that it's just myself who has an understanding
problem: if I truly understood the other person I'd not be surprised. What
would work wonders for me is to say something like "I understand your
position and why you think it's better but for me it's useless because...".

It looks like most of our missunderstandings come from the fact that I
expect others to be familiar with a lot of concepts or how a significant
portion of the developers in the world work & function. Based on previous
messages I discovered that's not the case at all, and it looks like this
community is sometimes a bit "unaware" of how things work for a lot of
people "outside". I'm sorry if that sounds offensive, I just don't know how
to say it. I really don't mean this in a bad way, maybe it's because you
know of these other ways but decided the Emacs way is superior and with
time you forgot a bit? More specifically I was surprised to have to explain
why namespaces have advantages, why tutorials & stackoverflow has a lot of
success. From my perspective these are obvious and thus it's hard for me to
realise when someone simply needs basic explanations about these.

Philippe

[-- Attachment #2: Type: text/html, Size: 1922 bytes --]

Re: Imports / inclusion of s.el into Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 3271 bytes --]

On Sun, May 03, 2020 at 10:24:54AM +0200, Philippe Vaucher wrote:
> >> Anyway, you try to focus on the weak points of the arguments. I
> understand,
> >> it's a natural defense mechanism.
> 
> > Here it is again.
> 
> > Psychologizing your opponent is dangerous, may be insulting and won't
> > bring the quest forward.
> 
> Oops, seems I derailed again. Thanks for pointing these out, it helps me
> notice.

I'm in the nice position of a somewhat external observer. Having
been involved in that kind of discussions in far more unconfortable
positions in the past, I'm just trying to offer what I can.

Of course, I may be wrong and all that -- in that case I'd hope
someone sets me straight (on-list or off-list). I can take quite
some abuse whenever I can assume it's well-intentioned :-)

> I have problems when someone looks to me like putting a lot of efforts into
> not seeing where we are talking from, and only pointing things from
> their perspective (all this while, I think, we try to understand theirs).

Often, such is the feeling on all sides...

> But I do realise now that it's just myself who has an understanding
> problem: if I truly understood the other person I'd not be surprised. What
> would work wonders for me is to say something like "I understand your
> position and why you think it's better but for me it's useless because...".

Of course that's not only you. It's always easier to see the other's
understanding problem. One's own can only be inferred (by making the
reasonable assumption that the other is, on the average, as smart as
oneself), but that is an abstract thing.

> It looks like most of our missunderstandings come from the fact that I
> expect others to be familiar with a lot of concepts or how a significant
> portion of the developers in the world work & function. Based on previous
> messages I discovered that's not the case at all, and it looks like this
> community is sometimes a bit "unaware" of how things work for a lot of
> people "outside".

That's why I try to compare it with cultures. Basically, I'm convinced
that the same mechanisms are at work.

Cultures gather around languages, but in a very subtle and complex
way. The official translation into English of "pavé de bœuf" is just
"beef steak". But the association cloud stirred up in the mind of a
person rooted in French culture (think Bourgongne :) will be totally
different to the cloud stirred up in someone, say, accultured in
Berlin.

When writing (and even more when reading) code, similar mechanisms
are at work. A wise person (I think it was Donald Knuth) once said
that he writes programs not to convince the machine to do something,
but to convince his colleagues that the program is doing the right
thing. So computer languages are (a strange kind) of human languages.

Now assume Alan is proofreading a patch coming from somewhere else.
Would you like him to read it as he reads his native language,
or as he reads a foreign language, dictionary in hand?

I guess the first choice will contribute towards a higher quality
in Emacs!

OTOH cultures and languages are living organisms. It doesn't make
sense to pretend they're static (unless they are dead).

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1092 bytes --]

> Now assume Alan is proofreading a patch coming from somewhere else.
> Would you like him to read it as he reads his native language,
> or as he reads a foreign language, dictionary in hand?

I agree with everything you said, I just wanted to point out this: of
course I prefer Alan to use his native language. But at the same time
languages do evolve... the french I speak nowadays is different than the
one that was there when I was born, and VERY different than the one that
was spoken 150 years ago. The thing is that it changed gradually, and I
guess my proposal makes it look like it'd change too rapidly.

So, basically we have to find a pace where most agree. From what I
understand there are 3 point of views:

- Alan / Richard: 0 new aliases. Only new APIs.
- Philippe: as many new aliases as needed where "obvious" (regexp, process,
etc), probably around 50 (wild guess)
- Most people's: Not against testing with just one topic (regexp), maximum
10 new aliases.... then wait a few years to see how it went before moving
on to other topics.

Do you feel that's accurate?

Philippe

[-- Attachment #2: Type: text/html, Size: 1213 bytes --]

Re: Imports / inclusion of s.el into Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2224 bytes --]

On Sun, May 03, 2020 at 11:14:33AM +0200, Philippe Vaucher wrote:
> > Now assume Alan is proofreading a patch coming from somewhere else.
> > Would you like him to read it as he reads his native language,
> > or as he reads a foreign language, dictionary in hand?
> 
> I agree with everything you said, I just wanted to point out this: of
> course I prefer Alan to use his native language. But at the same time
> languages do evolve... the french I speak nowadays is different than the
> one that was there when I was born, and VERY different than the one that
> was spoken 150 years ago. The thing is that it changed gradually, and I
> guess my proposal makes it look like it'd change too rapidly.

Definitely. A language that doesn't evolve is basically dead.

> So, basically we have to find a pace where most agree. From what I
> understand there are 3 point of views:
> 
> - Alan / Richard: 0 new aliases. Only new APIs.
> - Philippe: as many new aliases as needed where "obvious" (regexp, process,
> etc), probably around 50 (wild guess)
> - Most people's: Not against testing with just one topic (regexp), maximum
> 10 new aliases.... then wait a few years to see how it went before moving
> on to other topics.

I don't know if I could dare to quantify things. Watching what happens with
languages, "local dialects" just pop up in places and get used because there
are people who use them. OTOH, a language's job is to connect people, so
some process is needed to regulate all of that. In most regular languages,
it's just traveling and trading.

Your (native) language has the Académie [1], but some other languages don't
have that luxury :-)

> Do you feel that's accurate?

I think at the end it comes up to whether and  how people use it. If the
center of gravity of Emacs development moves, there's very little some
"cabal in power" (if there is any) can do. There are quite a few examples
(which were resolved in different ways) in that short history (Xorg,
gcc/egcs, Open vs LibreOffice, Own vs NextCloud...)

So the more experienced among us (I'm *not* among them!) are just some
kind of moderators.

Cheers

[1] Just a guess, based on your name.

-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The official translation into English of "pavé de bœuf" is just
  > "beef steak". But the association cloud stirred up in the mind of a
  > person rooted in French culture (think Bourgongne :)

What associations would those be?  I don't know French culture well enough
to have any idea of that.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 3323 bytes --]

On Sun, May 03, 2020 at 11:12:54PM -0400, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>   > The official translation into English of "pavé de bœuf" is just
>   > "beef steak". But the association cloud stirred up in the mind of a
>   > person rooted in French culture (think Bourgongne :)
> 
> What associations would those be?  I don't know French culture well enough
> to have any idea of that.

Difficult to put into few words, but let me try :-)

As a disclaimer up front: all generalizations suck. It is very
dangerous to ascribe attributes to persons just because they
are an X (for X in women, French people, mathematicians, muslims,
what not). Moreover, reducing persons to being "elements" of one
of those "sets" has always caused no end of grief and pain.

I don't say this because I think you don't know it, rather to
clarify the frame of reference I'm anchoring this.

Back to the pavé. In general (but see above ;-), eating is a
central part of French culture. Three-course meals are widespread,
you'll often find very special food shops (butcher, sweet
shop [1]) in very small villages (say 1000 to 4000 inhabitants).

In contrast, Germans tend to eat just to top off their energy
requirements. If you go to a sweet shop in Berlin, the buns
tend to be double the size of those you'd get in France, but
the taste... (OTOH you might not perceive the difference if you
haven't been sensitized to it).

So a French person (see above) tends to associate to a pavé
the kind of cattle those come from (Charolais, of course [2]),
the sauces those are served with (red wine sauce) or whatever.
Of course, memories of those meals (with all the talks that
went with them) will follow in this association. For a French
mind, a food item is a highly connected node.

On the contrary, for most German minds, one meal resembles
the other. The name of the game is to top off bodily needs
as efficiently (quickly, cheaply) as possible. A food item
is a moderately connected node in that brain's association
graph.

You can't translate that without loss, as you can't translate
the Inuit's snow without loss. The graphs aren't isomorphic
(and hey, that's what makes diving into other cultures
an exciting experience, after all, no?)

And you can't translate `cdr' or `concat' without loss,
either!

This is, of course, an image with a highly exaggerated contrast.
For one, things change over time; with the advancement of
austerity politics, the less rich French are less and less capable
of affording things -- while the Germans (at least those who
can afford) learn to appreciate other ways of living. Then
there are regional variations (you can see French traits in
German regions next to the French border).

Sorry for that wall-of-text

[1] See? There isn't even a word in English for confiserie!
[2] Those clarifications are a bit tongue-in-cheek and just
   try to illustrate the tone such a discussion might take
   (because you aren't eating alone, and of course you are
   talking about food and cooking meanwhile, aren't you?)

-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

Sous les pavés de bœuf, la plage!

Re: Imports / inclusion of s.el into Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 115 bytes --]

On Mon, May 04, 2020 at 10:12:58AM -0700, Drew Adams wrote:
> Sous les pavés de bœuf, la plage!

:-)

-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I mistakenly sent my question about French culture to the list.
I meant to send it only to Tomas.

Sorry for the noise.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

>>> In Haskell, do you name every function with a
>>> prefix that advertises the type of its return
>>> value or one of its main arguments?
>>
>> [... replies pointing out that some Haskell
>> functions do prefix their names with type names...]
>>
>> I suggest those of you who think that Haskell
>> too deserves type-name prefixes for all its
>> function names start an initiative to rename
>> its `map', `add', `zip', `head', ... functions.
>
> notice how in Haskell there is a strong will to
> group things together. That's what you'd take out
> of the example.

Of course.  In Haskell and in life generally.  Things
that are alike generally belong together.

But this discussion is about _naming_ functions, vars,
etc., not just grouping them.  In Elisp, grouping them
in a library already gives them a library prefix.  And
in general that has nothing to do with the type of
objects they use or return.

I should also point out that my initial question was,
in particular, about Haskell code that you write,
(e.g. for an application), not Haskell library code.

In the wide world there are many, many more functions
that _use_ objects of various kinds (types, classes)
than just the functions that are needed to _define_
those kinds.

That's especially true of a language like Lisp.
What you call "generic functions" are not corner
cases in Lisp.

It's even true of a typed functional language like
Haskell, I think: lots of functions need not be
associated with just one data type.  (They're
type-grouped by their signatures, of course.)  But
forget Haskell as example, if you're not convinced.

It's true of Lisp, in any case (even CLOS).

But it's not true of OOP.  In most OOP languages
you pretty much _have_ to place a function in some
class, e.g. as a method (which typically privileges
one of its args as the target object (which is in
that class).  (But in most OOP, there's no need to
add the class name as part of each method name.)

It can make sense to adopt a prefix convention for
functions that define a type of thing (e.g. buffer,
window, vector, string) than it does to impose it
willy-nilly way on functions that might just return
such a thing or accept it as one of their args.

Most functions do not fit that bill of defining a
thing type.  And many (most?) do not fit the bill
of acting only/mainly on one particular type of
thing or having as their main purpose to return
such a thing.

That was really the point.  Perhaps there are some
cases in Elisp where it would make sense to prefix
more functions that involve a particular kind of
thing.  I've acknowledged that.  And others have
said the same thing, citing strings as a type that
might be looked at more in this regard.

My preference would not be to name string-related
functions as a whole with a `string-' prefix, but
instead to consider them on a case-by-case basis.

(That said, I don't disagree with what Tomás said
about having "some vision as to which direction
those renames should tend to, if one wants to have
some overall consistency."  And I agree with him
that your proposal can serve discussion, and so
is part of the process.)

The functions in a library (especially 3rd-party)
are a different story.  They should have the
library prefix, not some object-type prefix.

E.g., if s.el functions were added to Elisp, and
if the library prefix remained as `s-', then they
should get that prefix.  If the library prefix
were renamed to `str-' or `foo' then that should
be the prefix for its function names.

But on its own, that has nothing to do with the
type of objects the functions act on or return.
The fact that most functions in that package
might involve strings in some way is something
else.  Even a function, macro, etc. in it that
has nothing to do with strings (if such exists)
should get its library prefix.

> yes, some generic functions will have trouble finding
> a good home. ...those function could either be left
> untouched, or they could be moved to `seq-`,...

That's where the disagreement is, I think - wrt
the degree to which such a renaming is needed or
useful.

Your point of view is apparently that functions
should generally, i.e., by default, be prefixed
with a thing name, and you acknowledge that it
might be difficult for "some generic functions"
to find a good thing type to name them with.

My point of view is farther down the spectrum.
It's not about generic functions.  It's about many,
probably MOST, functions in Lisp.  They don't have
a "good home" in the sense you mean it (some
privileged object type), and IMO they shouldn't
have a type-name prefix.

IOW, you seem to want to repaint everything, but
you acknowledge that it might be difficult to find
the right color paint for some things.  I don't.
I think it makes more sense to paint only things
that really need a type-labeling.

I contrasted painting the bikeshed with patching
holes in its roof.  I suggested dealing with names
case by case, and I even suggested filing bugs for
that, to track, discuss, and debate case by case.

A given case/bug could involve more than one
function name.  But I'd rather we err in the
direction of too few at a time than too many.

There's no right or wrong approach here.  It's a
question of aim, and view of the problem & cure.

> But I suggest we don't talk about these corner cases

See above.  For you, the corner cases (to ignore)
are a few "generic functions" that aren't easily
assigned a thing type, and you propose to leave
them out of the painting initiative.

The corner cases for me are the relatively few
functions whose names cry out for labeling as
specific to some type, and I'd propose fixing
their names, case by case.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sun, May 3, 2020 at 8:45 PM Drew Adams <drew.adams@oracle.com> wrote:

> that class).  (But in most OOP, there's no need to
> add the class name as part of each method name.)

Exactly. The compiler does it for you, "mangling"
the names according to the classes.  100% consistent,
predictable and horrible. Fortunately only for machine
consumption (...and occasional gdb pain).

João

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 2020 bytes --]

> > notice how in Haskell there is a strong will to
> > group things together. That's what you'd take out
> > of the example.
>
> Of course.  In Haskell and in life generally.  Things
> that are alike generally belong together.
>
> But this discussion is about _naming_ functions, vars,
> etc., not just grouping them.  In Elisp, grouping them
> in a library already gives them a library prefix.  And
> in general that has nothing to do with the type of
> objects they use or return.

No, to me this discussion is about _grouping_ things, that's why we push
for library prefixes.

> It can make sense to adopt a prefix convention for
> functions that define a type of thing (e.g. buffer,
> window, vector, string) than it does to impose it
> willy-nilly way on functions that might just return
> such a thing or accept it as one of their args.

Just to be clear, "type-grouping" you more or less see the point but
"topic-grouping" (e.g regexp, string) not so much?

> Most functions do not fit that bill of defining a
> thing type.  And many (most?) do not fit the bill
> of acting only/mainly on one particular type of
> thing or having as their main purpose to return
> such a thing.
>
> That was really the point.  Perhaps there are some
> cases in Elisp where it would make sense to prefix
> more functions that involve a particular kind of
> thing.  I've acknowledged that.  And others have
> said the same thing, citing strings as a type that
> might be looked at more in this regard.

Okay, but I think we already agree on this? I propose to alias/rename
functions where it's "obvious", but as we saw that what is obvious for me
is not obvious for everyone, so that's complicated. I think we reached a
point where everything proposed was rejected so this is a dead end.

> The corner cases for me are the relatively few
> functions whose names cry out for labeling as
> specific to some type, and I'd propose fixing
> their names, case by case.

The ones I proposed were all rejected :-/ Do you have one in mind?

[-- Attachment #2: Type: text/html, Size: 2370 bytes --]

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

>> It can make [more] sense to adopt a prefix convention
>> for functions that define a type of thing (e.g. buffer,
>> window, vector, string) than it does to impose it
>> willy-nilly way on functions that might just return
>> such a thing or accept it as one of their args.
>
> Just to be clear, "type-grouping" you more or less see
> the point but "topic-grouping" (e.g regexp, string)
> not so much?

No.  I was distinguishing the definition of an
abstract data type, and the function definitions
needed for that, as a more focused kind of grouping
from grouping functions just by the type of an
argument or the return value.

In OOP, a function that merely uses an object in
a given class is often required to itself be a
method of that class.  That's not so for other
kinds of programming, and it's not so, in general,
for formal definition of ADTs.

>> Most functions do not fit that bill of defining a
>> thing type.  And many (most?) do not fit the bill
>> of acting only/mainly on one particular type of
>> thing or having as their main purpose to return
>> such a thing.
>>
>> That was really the point.  Perhaps there are some
>> cases in Elisp where it would make sense to prefix
>> more functions that involve a particular kind of
>> thing.  I've acknowledged that.  And others have
>> said the same thing, citing strings as a type that
>> might be looked at more in this regard.
>
> Okay, but I think we already agree on this?

Not so sure.  If so, good.

> I propose to alias/rename functions where it's
> "obvious", but as we saw that what is obvious for
> me is not obvious for everyone, so that's
> complicated.

Yes.  And I'm guessing that the more you think
about it, the more you might find that some of
what you thought was obvious is not quite so
straightforward.  Just a thought.

In some of what you wrote, you seemed to think
that your background of having worked with many
languages etc. made things obvious to you, and
that others might not have such a rich background
and so they don't see easily the light.

You might be surprised that some here also have
long and rich backgrounds with many languages.
That they might not agree with you - might not
see what you see as "obvious" - might not mean
what you think.

> I think we reached a point where everything
> proposed was rejected so this is a dead end.

I haven't seen a clear proposal.  And I don't
know that anything has been rejected.  The
discussion has been rather mixed and somewhat
unclear, IMO.

I don't think anyone has rejected the idea of
renaming some functions or aliases them to new
names.  The devil is in the details.

I suggested that any proposed name changes
(including aliases) be submitted (and tracked)
as bug/enhancement requests.

> > The corner cases for me are the relatively few
> > functions whose names cry out for labeling as
> > specific to some type, and I'd propose fixing
> > their names, case by case.
>
> The ones I proposed were all rejected :-/ Do you
> have one in mind?

No, not at the moment.

And I'm not aware that anything has been rejected.
If you submit them as a bug report (or bug reports,
if they're not very related) then it will be clear
if the bug gets fixed, stalls, or is closed as
won't fix.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

> I haven't seen a clear proposal.  And I don't
> know that anything has been rejected.  The
> discussion has been rather mixed and somewhat
> unclear, IMO.
>
> I suggested that any proposed name changes
> (including aliases) be submitted (and tracked)
> as bug/enhancement requests.
>
> And I'm not aware that anything has been rejected.
> If you submit them as a bug report (or bug reports,
> if they're not very related) then it will be clear
> if the bug gets fixed, stalls, or is closed as
> won't fix.

Well I thought I did a concrete proposal on `string-` and I'm thinking
that if I made the same proposal throught the bug tracker I'd get the
same answers.

Anyway, I'll try if you tell me that's how I'd do it. I'm a bit
reluctant because I hoped that we'd do this all together here and
reach some form of consensus with rough guidelines like "yes I think
for A, B, & C it makes sense, for D & E not so much" and thus the real
concrete proposal would already be pretty accurate.

I'll make a bug report anyway and we'll see how it goes. Should it be
already patches or just a message with "new aliases / new api"
sections like I did a few emails above in that thread?

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> > I suggested that any proposed name changes
> > (including aliases) be submitted (and tracked)
> > as bug/enhancement requests.
> >
> > And I'm not aware that anything has been rejected.
> > If you submit them as a bug report (or bug reports,
> > if they're not very related) then it will be clear
> > if the bug gets fixed, stalls, or is closed as
> > won't fix.
> 
> Well I thought I did a concrete proposal on `string-` and I'm thinking
> that if I made the same proposal throught the bug tracker I'd get the
> same answers.
> 
> Anyway, I'll try if you tell me that's how I'd do it. I'm a bit
> reluctant because I hoped that we'd do this all together here and
> reach some form of consensus with rough guidelines like "yes I think
> for A, B, & C it makes sense, for D & E not so much" and thus the real
> concrete proposal would already be pretty accurate.
> 
> I'll make a bug report anyway and we'll see how it goes. Should it be
> already patches or just a message with "new aliases / new api"
> sections like I did a few emails above in that thread?

I can't speak for anyone but myself.  Others will
perhaps offer other suggestions.

I'd suggest filing an enhancement request (M-x
report-emacs-bug) for some specific renamings or
aliases, providing brief reasons.

If you think it will help, you can reference, or
quote from, specific messages in this list (at 
https://lists.gnu.org/archive/html/emacs-devel/).

But maybe wait a bit, to see first if you get
other suggestions, including any that might
disagree.  Eli, in particular, may have a
preference about handling this.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Date: Tue, 5 May 2020 08:03:56 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, Emacs developers <emacs-devel@gnu.org>,
>         Stefan Monnier <monnier@iro.umontreal.ca>,
>         Richard Stallman <rms@gnu.org>
> 
> I'd suggest filing an enhancement request (M-x
> report-emacs-bug) for some specific renamings or
> aliases, providing brief reasons.
> 
> If you think it will help, you can reference, or
> quote from, specific messages in this list (at 
> https://lists.gnu.org/archive/html/emacs-devel/).
> 
> But maybe wait a bit, to see first if you get
> other suggestions, including any that might
> disagree.  Eli, in particular, may have a
> preference about handling this.

I personally don't mind either way, but keep in mind that the bug list
is read by significantly less people than emacs-devel.  Whether this
is an advantage or a disadvantage, is left to each one to decide ;-)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'd suggest filing an enhancement request (M-x
  > report-emacs-bug) for some specific renamings or
  > aliases, providing brief reasons.

There's no need for that.  That is just a way to start a conversation.
We have already gone past that point -- we already have a conversation
here about which string functions or names would be good to add.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

>   > I'd suggest filing an enhancement request (M-x
>   > report-emacs-bug) for some specific renamings or
>   > aliases, providing brief reasons.
> 
> There's no need for that.  That is just a way to start a conversation.
> We have already gone past that point -- we already have a conversation
> here about which string functions or names would be good to add.

OK, but this thread has gone well beyond a request
for specific renamings or aliases.  Is it clear here
what the ask is?  If so, great.

Possible renamings of some string functions.

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > OK, but this thread has gone well beyond a request
  > for specific renamings or aliases.

I'm asking for proposals for specific renamings or aliases.  Since
this is not the discussion of s.el, I have changed the Subject field.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> >  > > So, if I would rename concat, it would be concat-to-string.
> >
> >   > But then people who need to concatenate strings will not find it,
> >   > because they will type string- TAB.
> >
> > My solution, extending apropos and its variants so it "finds"
> > 'string-' in the name 'concat', would deal with that.
> 
> The main motivation for "renaming" was to have completion find those
> names.  People who advance that proposal don't want to use apropos
> instead of completion.  So we will need to extend the completion to do
> the same trick you had in mind for apropos.

Don't forget that completion can now use substring
and other kinds of matching.

So depending on what you mean, perhaps there's no
need to extend completion - there's just a need to
let users get the kind of completion they want in
any context.

The question, for any given _use_ of completion,
is what kind of completion someone wants.  For
code-completion maybe prefix completion is great.
For `C-h f' maybe substring completion is better.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Mon, May 4, 2020 at 6:39 PM Drew Adams <drew.adams@oracle.com> wrote:
>
> > >  > > So, if I would rename concat, it would be concat-to-string.
> > >
> > >   > But then people who need to concatenate strings will not find it,
> > >   > because they will type string- TAB.
> > >
> > > My solution, extending apropos and its variants so it "finds"
> > > 'string-' in the name 'concat', would deal with that.
> >
> > The main motivation for "renaming" was to have completion find those
> > names.  People who advance that proposal don't want to use apropos
> > instead of completion.  So we will need to extend the completion to do
> > the same trick you had in mind for apropos.

> The question, for any given _use_ of completion,
> is what kind of completion someone wants.  For
> code-completion maybe prefix completion is great.
> For `C-h f' maybe substring completion is better.

+1.  Or maybe 'substring' completion with that
"also mentioned in the manual" scoring boost
(could be another completion style).

João Távora

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

> Don't forget that completion can now use substring
> and other kinds of matching.

You understand that substring completion fails as soon as the term is
too generic? e.g "string", "regexp" or "list", the list is just a lot
of noise. Also it doesn't quite work when the order is "reversed", e.g
C-h f "string TAB multibyte" would not return the desired function
because it has to be "multibyte TAB string".

With other topics where most functions are really about the topic I
agree it works reasonably well.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 8:26 AM Philippe Vaucher
<philippe.vaucher@gmail.com> wrote:
>
> > Don't forget that completion can now use substring
> > and other kinds of matching.
>
> You understand that substring completion fails as soon as the term is
> too generic? e.g "string", "regexp" or "list", the list is just a lot
> of noise.

Have you actually tried Drew's suggestion? Starts an Emacs -Q and (setq
completion-styles '(flex)), or just M-x fido-mode.  Then C-h f for
"string".  You get a _little_ noise but it's not "a lot", by far.  And
the little noise you _do_ get from internal functions in other packages
is pretty easy to remove if you boost things by their mentions in the
manual.  In fact there's a patch at the end of this message.

> Also it doesn't quite work when the order is "reversed", e.g> C-h f
> "string TAB multibyte" would not return the desired function
> because it has to be "multibyte TAB string".

But that's never going to work either way, unless you alias all the
multi-word symbols to all their possible permutations.  You'll always
get a group of people that really expected it to be multibyte-string and
group of people that expect string-multibyte.  Oh, maybe we should all
be searching for "multibyte" instead!  Bam.  First 5 results:

multibyte-string-p
string-to-multibyte
string-as-multibyte
set-buffer-multibyte
multibyte-char-to-unibyte

To be clear.  I'm not presenting these examples to be antagonistic.  I'm
kind of sympathetic to your laziness to read the manuals, I'm lazy
myself, a reasonably good trait in programmers.  So I really do use
these tools (and work on them) to get around an imperfect world instead
of demanding that the world adjust to my rigid way of thinking.

That said, it is possible to devise a completion style that will take
two words and match them against candidates in different ways, maybe
even matching them against other properties besides their names.  I
haven't found it really useful, yet.  But it's not absurd to think about
that and writing a smart completion style for Emacs is a much better
(and more interesting) contribution to it than the renaming
sledgehammer.

Also sometimes, I'll read the manual and be really impressed by its
quality, too.

João

Anyway here's the "use the manual to bump up flex score"
patch.

diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
index f6e2b236f3..1590b954b7 100644
--- a/lisp/minibuffer.el
+++ b/lisp/minibuffer.el
@@ -3191,6 +3191,12 @@ flex-score-match-tightness
 than the latter (which has two \"holes\" and three
 one-letter-long matches).")

+(defvar flex-score-adjustment nil
+  "If non nil a function of to adjust score of a flex match.
+Function is passed three arguments: MATCH, PATTERN and
+preliminary SCORE.  It should return a factor by which to
+multiply SCORE to reach the final value.")
+
 (defun completion-pcm--hilit-commonality (pattern completions)
   (when completions
     (let* ((re (completion-pcm--pattern->regex pattern 'group))
@@ -3279,9 +3285,16 @@ completion-pcm--hilit-commonality
                 'completions-first-difference
                 nil str))
            (unless (zerop (length str))
-             (put-text-property
-              0 1 'completion-score
-              (/ score-numerator (* len (1+ score-denominator)) 1.0) str)))
+             (let ((prelim
+                    (/ score-numerator (* len (1+ score-denominator)) 1.0)))
+               (put-text-property
+                0 1 'completion-score
+                (*
+                 (if (functionp flex-score-adjustment)
+                     (funcall flex-score-adjustment str pattern prelim)
+                   1)
+                 prelim)
+                str))))
          str)p
        completions))))


Then do this:

(setq flex-score-adjustment
      (lambda (m _p _s)
        (if (assoc m (info-lookup->completions 'symbol 'emacs-lisp-mode))
            4
          1)))

Then try C-h f string,  C-h f process

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

> > You understand that substring completion fails as soon as the term is
> > too generic? e.g "string", "regexp" or "list", the list is just a lot
> > of noise.
>
> Have you actually tried Drew's suggestion? Starts an Emacs -Q and (setq
> completion-styles '(flex)), or just M-x fido-mode.  Then C-h f for
> "string".  You get a _little_ noise but it's not "a lot", by far.  And
> the little noise you _do_ get from internal functions in other packages
> is pretty easy to remove if you boost things by their mentions in the
> manual.  In fact there's a patch at the end of this message.

I must admit I'm a bit of an extremist here, because even when trying
this the little noise annoys me. Also the noise is bigger with
"regexp". But I understand this solution seems good enough from your
perspective.

> > Also it doesn't quite work when the order is "reversed", e.g> C-h f
> > "string TAB multibyte" would not return the desired function
> > because it has to be "multibyte TAB string".
>
> But that's never going to work either way, unless you alias all the
> multi-word symbols to all their possible permutations.  You'll always
> get a group of people that really expected it to be multibyte-string and
> group of people that expect string-multibyte.

I think "all in string-" or "all in string- or multibyte-" would work
for me. Sure, the first time you try to find "multibyte" in in
"string-" you fail but you rapidly know that from now only multibyte
starts with multibyte. For example your list I'd do:

multibyte-string-p -> multibyte-string-p
string-to-multibyte -> multibyte-from-string
string-as-multibyte -> multibyte-from-string-unsafe
set-buffer-multibyte -> buffer-set-multibyte
multibyte-char-to-unibyte -> multibyte-char-to-unibyte

> To be clear.  I'm not presenting these examples to be antagonistic.  I'm
> kind of sympathetic to your laziness to read the manuals, I'm lazy
> myself, a reasonably good trait in programmers.  So I really do use
> these tools (and work on them) to get around an imperfect world instead
> of demanding that the world adjust to my rigid way of thinking.

I agree you have a point. But even without renaming as much as I
suggest, I think renaming the functions that does not follow the
"standard" of Emacs Lisp should be done (once it is defined). It's
much easier for the brain to think in terms of patterns, even if they
are not as straight forward as namespaces they should at least be
consistent.


> That said, it is possible to devise a completion style that will take
> two words and match them against candidates in different ways, maybe
> even matching them against other properties besides their names.  I
> haven't found it really useful, yet.  But it's not absurd to think about
> that and writing a smart completion style for Emacs is a much better
> (and more interesting) contribution to it than the renaming
> sledgehammer.

Yes, personally I'd be more enclined toward the proposal of grouping
functions according to
https://www.gnu.org/software/emacs/manual/html_node/elisp/index.html,
this should be fairly easy to implement.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 12:58 PM Philippe Vaucher
<philippe.vaucher@gmail.com> wrote:

> I think "all in string-" or "all in string- or multibyte-" would work
> for me. Sure, the first time you try to find "multibyte" in in
> "string-" you fail but you rapidly know that from now only multibyte
> starts with multibyte. For example your list I'd do:
>
> multibyte-string-p -> multibyte-string-p
> string-to-multibyte -> multibyte-from-string
> string-as-multibyte -> multibyte-from-string-unsafe
> set-buffer-multibyte -> buffer-set-multibyte
> multibyte-char-to-unibyte -> multibyte-char-to-unibyte

[1]

Do you know what this would make to fine completion list
I gave you?  It would make it suck.  So you, the "bit of an
extremist" who "is annoyed by a little noise", are prepared to
introduce  an unimaginable amount of it into my and other's
workflows at the slightest difficulty and resistance to read
a fine manual.  I like Emacs because it respects people's
established workflows, and allows for programmers
to build on it, so they can use whatever workflow they
prefer.

Unfortunately, you're finding it a bit hard to support
your -- absolutely legitimate, mind you -- ruby-esque
ways.  But you should be working to have proper
namespacing so you can work with a magnar-string.el
or ruby-regexp.el library the way you feel more
confortable.  That takes work, yes, but at least it's a
win-win, not a lose-lose. If proper namespacing takes a lot
of work, then work on a powerful completion tool. I can
help you with that.  You're French, right? Imagine if
Google  decided they'd to the complete works of Raymond
Roussell from French to modern, easy-to-search, French.

João

PS: BTW let a monkey bite me if for each of those
proposed renames you don't start a separate
100-mail long bike-shedding war.

Re: Imports / inclusion of s.el into Emacs

[조성빈]

João Távora <joaotavora@gmail.com> 작성:

> On Tue, May 5, 2020 at 12:58 PM Philippe Vaucher
> <philippe.vaucher@gmail.com> wrote:
>
>> I think "all in string-" or "all in string- or multibyte-" would work
>> for me. Sure, the first time you try to find "multibyte" in in
>> "string-" you fail but you rapidly know that from now only multibyte
>> starts with multibyte. For example your list I'd do:
>>
>> multibyte-string-p -> multibyte-string-p
>> string-to-multibyte -> multibyte-from-string
>> string-as-multibyte -> multibyte-from-string-unsafe
>> set-buffer-multibyte -> buffer-set-multibyte
>> multibyte-char-to-unibyte -> multibyte-char-to-unibyte
>
> [1]
>
> Do you know what this would make to fine completion list
> I gave you?  It would make it suck.  So you, the "bit of an
> extremist" who "is annoyed by a little noise", are prepared to
> introduce  an unimaginable amount of it into my and other's
> workflows at the slightest difficulty and resistance to read
> a fine manual.  I like Emacs because it respects people's
> established workflows, and allows for programmers
> to build on it, so they can use whatever workflow they
> prefer.

Really? This whole mail thread’s 80% is about why just usual
prefix-searching isn’t good and you should use ‘C-h <something>’
in order to search for a function. I can’t say that’s how it’s
respecting a person’s ‘established’ workflow… I think it’s fair
enough to say that Emacs is geared towards the workflow the
majority of emacs-devel is used to. (Which is very different
from usual Emacs users.)

> Unfortunately, you're finding it a bit hard to support
> your -- absolutely legitimate, mind you -- ruby-esque
> ways.

It’s not ruby-esque, mind you - it’s consistency, that almost
every language aims to be. I can’t understand why you’re keep
calling it as ‘ruby’ or something implying that it’s something
new that doesn’t work with Emacs.

> But you should be working to have proper
> namespacing so you can work with a magnar-string.el
> or ruby-regexp.el library the way you feel more
> comfortable.  That takes work, yes, but at least it's a
> win-win, not a lose-lose. If proper namespacing takes a lot
> of work, then work on a powerful completion tool. I can
> help you with that.  You're French, right? Imagine if
> Google  decided they'd to the complete works of Raymond
> Roussell from French to modern, easy-to-search, French.
>
> João
>
> PS: BTW let a monkey bite me if for each of those
> proposed renames you don't start a separate
> 100-mail long bike-shedding war.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 2:18 PM 조성빈 <pcr910303@icloud.com> wrote:

> > a fine manual.  I like Emacs because it respects people's
> > established workflows, and allows for programmers
> > to build on it, so they can use whatever workflow they
> > prefer.
> Really? This whole mail thread’s 80% is about why just usual
> prefix-searching isn’t good and you should use ‘C-h <something>’
> in order to search for a function.

You're confused and should read the thread.  I was not
talking about prefix-searching.  My workflow doesn't use
prefix searching (anymore) I use "flex". But I haven't impinged
my  workflow on other people.  Well I lie.  I do so on occasion,
because I think it's cool. But I'm not going to _force_ it on
other people.

> It’s not ruby-esque, mind you - it’s consistency, that almost
> every language aims to be. I can’t understand why you’re keep
> calling it as ‘ruby’ or something implying that it’s something
> new that doesn’t work with Emacs.

I was answering Philippe, and he mentioned earlier that he's
a big fan of "Convention over Configuration" a philosophy
popularized in the 2000's by the RoR community. He's also
mentioned he has done Ruby. As have I, and I have nothing
against it, quite the contrary.  Pretty nice language.  And I
also love the API lists and the naming consistency there.

> Something implying that it’s something new that doesn’t
> work with Emacs.

Quite the contrary. I'm trying to convince you that it can
work with closer to that if you work towards it without
futzing up other people's, equally legitimate, ways of
thinking about symbol names.  Hey, if Emacs had Common
Lisp, you could probably program the reader and put most
of Ruby itself in Elisp if you wanted to. Proper namespaces,
manual-extracted  API lists, existing completion styles, new
completion styles. All  these things would go a long way before
you take the thickest brush to a 40-year old painstakingly
crafted painting.  A living painting, yes and never finished,
just like a real one, but still.

João Távora

Re: Imports / inclusion of s.el into Emacs

[조성빈]

João Távora <joaotavora@gmail.com> 작성:

> On Tue, May 5, 2020 at 2:18 PM 조성빈 <pcr910303@icloud.com> wrote:
>
>>> a fine manual.  I like Emacs because it respects people's
>>> established workflows, and allows for programmers
>>> to build on it, so they can use whatever workflow they
>>> prefer.
>> Really? This whole mail thread’s 80% is about why just usual
>> prefix-searching isn’t good and you should use ‘C-h <something>’
>> in order to search for a function.
>
> You're confused and should read the thread.  I was not
> talking about prefix-searching.  My workflow doesn't use
> prefix searching (anymore) I use "flex". But I haven't impinged
> my  workflow on other people.  Well I lie.  I do so on occasion,
> because I think it's cool. But I'm not going to _force_ it on
> other people.

I’m not saying only about completion - I was thinking about things
like how C-h f isn’t really useful for function finding and we
should use C-h a and C-h d. And arguably emacs lisp function names
are forcing us to use them. How should an Emacs user fix one’s
init.el in Vim because his init.el is broken to make Emacs not
work? (And let’s not say emacs -Q is the answer, because it’s just
one example. Just think that he/she doesn’t want to use emacs -Q
because it litters *~ files in the file system.)

>> It’s not ruby-esque, mind you - it’s consistency, that almost
>> every language aims to be. I can’t understand why you’re keep
>> calling it as ‘ruby’ or something implying that it’s something
>> new that doesn’t work with Emacs.
>
> I was answering Philippe, and he mentioned earlier that he's
> a big fan of "Convention over Configuration" a philosophy
> popularized in the 2000's by the RoR community.

I can’t see why CoC applies here - function names aren’t
‘configuration’, right? - AFAIK CoC is something like sensible
defaults which, while Emacs have a long way to get some more
sensible defaults IMO, is for another thread.

(BTW, while the term CoC was coined by RoR, there are various
  similar concepts that existed before that - it’s not something
  RoR has invented.)

> He's also
> mentioned he has done Ruby. As have I, and I have nothing
> against it, quite the contrary.  Pretty nice language.  And I
> also love the API lists and the naming consistency there.
>
>> Something implying that it’s something new that doesn’t
>> work with Emacs.
>
> Quite the contrary. I'm trying to convince you that it can
> work with closer to that if you work towards it without
> futzing up other people's, equally legitimate, ways of
> thinking about symbol names.

Actually, IMO that ways of thinking symbol names should be
codified and be recommended - could you help in making some
emacs lisp API guidelines with that knowledge?

> Hey, if Emacs had Common
> Lisp, you could probably program the reader and put most
> of Ruby itself in Elisp if you wanted to. Proper namespaces,
> manual-extracted  API lists, existing completion styles, new
> completion styles. All  these things would go a long way before
> you take the thickest brush to a 40-year old painstakingly
> crafted painting.  A living painting, yes and never finished,
> just like a real one, but still.
>
> João Távora

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 2:55 PM 조성빈 <pcr910303@icloud.com> wrote:

> I’m not saying only about completion - I was thinking about things
> like how C-h f isn’t really useful for function finding and we
> should use C-h a and C-h d.

But I'm saying it _is_ useful. It IS useful to me. With flex. And
your proposal will make it much less useful, and that's bad.

> > I was answering Philippe, and he mentioned earlier that he's
> > a big fan of "Convention over Configuration" a philosophy
> > popularized in the 2000's by the RoR community.
> I can’t see why CoC applies here - function names aren’t
> ‘configuration’, right?

No, but using `(setq completion-styles 'flex)` is. A pretty useful
bit of it too.  Maybe you try it for a while? It works with Company,
completion-at-point and, theoretically, with a new version of Helm.
You can also try M-x fido-mode.

Anyway, CoC is what Phillipe brought up, not me. It's a RoR thing,
and I was answering him. And Convention is pretty good, sure,
when you're starting from scratch.  So in your next new language,
by all means, use the Best Convention (TM).

> > Quite the contrary. I'm trying to convince you that it can
> > work with closer to that if you work towards it without
> > futzing up other people's, equally legitimate, ways of
> > thinking about symbol names.
>
> Actually, IMO that ways of thinking symbol names should be
> codified and be recommended - could you help in making some
> emacs lisp API guidelines with that knowledge?

For new domains, surely? Like washing-machines.el or
little-green-men.el?  Sure, I can do that.  I have my own
conventions on how to name things (and they've changed
over the years, a lot, and they will probably change until
I'm dead.  I wouldn't recommend putting  a lot of weight in
them, though. You can check my latest packages, like
eglot.el or jsonrpc.el to see how I name things.

João

Re: Imports / inclusion of s.el into Emacs

[조성빈]

João Távora <joaotavora@gmail.com> 작성:

> On Tue, May 5, 2020 at 2:55 PM 조성빈 <pcr910303@icloud.com> wrote:
>
>> I’m not saying only about completion - I was thinking about things
>> like how C-h f isn’t really useful for function finding and we
>> should use C-h a and C-h d.
>
> But I'm saying it _is_ useful. It IS useful to me. With flex.

Yes, that’s true, but flex, while is super useful (and I’m grateful to you
developing it), is still a hack that exists because the function names don’t
have good rules. (Why should everybody use flex, just to find functions?)

> And
> your proposal will make it much less useful, and that's bad.


>>> I was answering Philippe, and he mentioned earlier that he's
>>> a big fan of "Convention over Configuration" a philosophy
>>> popularized in the 2000's by the RoR community.
>> I can’t see why CoC applies here - function names aren’t
>> ‘configuration’, right?
>
> No, but using `(setq completion-styles 'flex)` is. A pretty useful
> bit of it too.  Maybe you try it for a while? It works with Company,
> completion-at-point and, theoretically, with a new version of Helm.

In fact, I already use it - I use flex for both Company & Helm, it’s
wonderful. (As I have said already) Thanks for flex-completion, it’s super
useful to me.

However, IMO the fact that you can now search functions in C-h f a bit easily
doesn't mean that the functions can be less consistent.

> You can also try M-x fido-mode.
>
> Anyway, CoC is what Phillipe brought up, not me. It's a RoR thing,
> and I was answering him.

Oh, I might have missed that. Sorry.

> And Convention is pretty good, sure,
> when you're starting from scratch.  So in your next new language,
> by all means, use the Best Convention (TM).
>
>>> Quite the contrary. I'm trying to convince you that it can
>>> work with closer to that if you work towards it without
>>> futzing up other people's, equally legitimate, ways of
>>> thinking about symbol names.
>>
>> Actually, IMO that ways of thinking symbol names should be
>> codified and be recommended - could you help in making some
>> emacs lisp API guidelines with that knowledge?
>
> For new domains, surely?
------------------------------------------------------------------------------
Well, Elisp will gain some more core APIs in the future, so I was thinking
about that, but except from package prefixes, wouldn’t it be similar?

> Like washing-machines.el or
> little-green-men.el?  Sure, I can do that.  I have my own
> conventions on how to name things (and they've changed
> over the years, a lot, and they will probably change until
> I'm dead.  I wouldn't recommend putting  a lot of weight in
> them, though. You can check my latest packages, like
> eglot.el or jsonrpc.el to see how I name things.

How do you think about this:

> I think some preliminary conventions that Elisp already follows is that the
> <action>-<object> scheme is for actions on the object like rename-file or
> clear-string, and <object>-<property> scheme is for getting properties of  
> the
> object like string-width and file-name-extension. (I’m not considering
> polymorphic ones.)
>
> But then there are exceptions, like string-trim (which should then be
> trim-string) or string-join (which should then be join-string).

> João

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 5:47 PM 조성빈 <pcr910303@icloud.com> wrote:
>
> João Távora <joaotavora@gmail.com> 작성:
>
> > On Tue, May 5, 2020 at 2:55 PM 조성빈 <pcr910303@icloud.com> wrote:
> >
> >> I’m not saying only about completion - I was thinking about things
> >> like how C-h f isn’t really useful for function finding and we
> >> should use C-h a and C-h d.
> >
> > But I'm saying it _is_ useful. It IS useful to me. With flex.
>
> Yes, that’s true, but flex, while is super useful (and I’m grateful to you
> developing it), is still a hack that exists because the function names don’t
> have good rules. (Why should everybody use flex, just to find functions?)

Good question. Because one can use flex for every idiosyncratically
named out there, which is the majority, not just functions, or at least
not just the very small subset of functions we use in the core.

Again, it's languages with built-in namespaces or implicit
OO namespaces like Ruby, C++ , Java, etc that give you the
illusion that things can be like that in Lisp. But they can't.
_Especially_ in Elisp, which has a big shared namespace.

(BTW I very much appreciate that you like flex. You're welcome.)

> However, IMO the fact that you can now search functions in C-h f a bit easily
> doesn't mean that the functions can be less consistent.

I'm not proposing we make new name inconsistent. I'm proposing
we don't touch the names we've already done.  Or that we touch
them very cautiously.

> > For new domains, surely?
> ------------------------------------------------------------------------------
> Well, Elisp will gain some more core APIs in the future, so I was thinking
> about that, but except from package prefixes, wouldn’t it be similar?

OK, a new core API (say thread-support, or emoji-support)  is a new
"domain" (again, it is kinda hard to draw the line because one big
namespace).

> How do you think about this:
>
> > I think some preliminary conventions that Elisp already follows is that the
> > <action>-<object> scheme is for actions on the object like rename-file or
> > clear-string, and <object>-<property> scheme is for getting properties of
> > the
> > object like string-width and file-name-extension. (I’m not considering
> > polymorphic ones.)
> >
> > But then there are exceptions, like string-trim (which should then be
> > trim-string) or string-join (which should then be join-string).

It's not bad. You could program Common Lisp with that, but in Elisp
goes against Stefan's rule which is: always prefix by the name
of the file. But even if we ignore it, it's hard. Say I'm writing a library
for music files,  a music.el.  I want a function to play a piece of music.
Should I name it play-music or music-play or music-play-music?
Whoever wrote package-list-packages, package-menu-hide-package,
package-install, describe-package, list-packages must have had
nightmares  with this. I shiver every time I see those names popup
in my minibufer.

A true story: some people in CL like to suffix all slot accessors with
"-OF". NAME-OF, TYPE-OF, etc. I did that for a while, but now I don't
do it. It's just silly. Once you start doing  it, it's disappointing not to
see it consistently applied everywhere.

You can't "tame" names in Lisp, is my feeling.

João

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

> > multibyte-string-p -> multibyte-string-p
> > string-to-multibyte -> multibyte-from-string
> > string-as-multibyte -> multibyte-from-string-unsafe
> > set-buffer-multibyte -> buffer-set-multibyte
> > multibyte-char-to-unibyte -> multibyte-char-to-unibyte
>
> [1]
>
> Do you know what this would make to fine completion list
> I gave you?  It would make it suck.  So you, the "bit of an
> extremist" who "is annoyed by a little noise", are prepared to
> introduce  an unimaginable amount of it into my and other's
> workflows at the slightest difficulty and resistance to read
> a fine manual.  I like Emacs because it respects people's
> established workflows, and allows for programmers
> to build on it, so they can use whatever workflow they
> prefer.

I think I already acknowledged that point in
https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg00508.html
Which is why I don't think any of my renames proposal will be
accepted. Maybe one or two but so far I haven't found a single example
where everyone agreed. In the multibyte example I was just
illustrating how I think about things.


> Unfortunately, you're finding it a bit hard to support
> your -- absolutely legitimate, mind you -- ruby-esque
> ways.  But you should be working to have proper
> namespacing so you can work with a magnar-string.el
> or ruby-regexp.el library the way you feel more
> confortable.  That takes work, yes, but at least it's a
> win-win, not a lose-lose. If proper namespacing takes a lot
> of work, then work on a powerful completion tool. I can
> help you with that.  You're French, right? Imagine if
> Google  decided they'd to the complete works of Raymond
> Roussell from French to modern, easy-to-search, French.

Actually I'm swiss, from the french speaking part of Switzerland.
Funnily enough we think we speak a better french than the frenchs
themselves, given we say the more logical "seventy-five" instead of
"sixty-fifteen" like they do. Maybe that's why I think I know better
how to name things than the actual authors :-)

Anyway, at this point I'll make a "concrete proposal" like I was
asked, something very simple and hopefully very uncontroversial, but I
think the bikeshedding can stop. We obviously disagree how APIs should
be designed.

Basically I focus more on the advantages of putting the
discoverability/consistency inside the language itself instead of its
tools, while you focus more on how disruptive this approch will be and
how we could still get what I want with new tools instead of the
language.

I thought we had a golden opportunity to put s.el inside Emacs:

- The author is willing.
- The number of contributors is small, most of them already signed the
papers including the author.
- A lot of "github" Emacs users would see this as a good sign.

But I realise now that even if this was done, what would likely happen
is that s.el get stale because adding/modifying it is not as simple &
smooth as doing so on other platforms, and a new s.el will appear and
we can repeat what happened these past days in a few years.

On the topic of s.el inclusion here are my conclusions (please correct
me if they are wrong):

- I can try to suggest a few aliases, and maybe one or two will be
accepted, but certainly not a lot.
- I can try to suggest a few new functions, and maybe one or two will
be accepted.
- Whatever is introduced is likely to be "Emacsified" as not to look
too much like clojure.
- I can notify the author of s.el that only a tiny subset of s.el (if
any) is likely to be imported, but he should know he's very welcome to
put s.el on ELPA.

Kind regards,
Philippe

Re: Imports / inclusion of s.el into Emacs

[Stefan Kangas]

Philippe Vaucher <philippe.vaucher@gmail.com> writes:

> I thought we had a golden opportunity to put s.el inside Emacs:

We can still put it in GNU ELPA, which is just as good.  Frankly, it's
better.  We can pick and choose all the best ideas and include them in
a way that fits with the rest of Emacs.  And users who still prefer
s.el are free to use it, including in other GNU ELPA packages.

> - I can try to suggest a few aliases, and maybe one or two will be
> accepted, but certainly not a lot.
> - I can try to suggest a few new functions, and maybe one or two will
> be accepted.

Sounds good, thank you.  (I'm not sure why you think that most of your
suggestions would be rejected, though.)

> - Whatever is introduced is likely to be "Emacsified" as not to look
> too much like clojure.

The way I understand this discussion, I don't think the point is that
we just want to be different for the sake of being different.  It's
just that we can't import functions wholesale simply to be similar to
another language when we already have perfectly good alternatives in
ELisp.  Or at least we can't do that in core (OTOH it's fine if a
library like s.el wants to do that).

> - I can notify the author of s.el that only a tiny subset of s.el (if
> any) is likely to be imported, but he should know he's very welcome to
> put s.el on ELPA.

If I read you correctly, you're a bit disappointed.  Even so, please
focus on the positive, that we would like to add s.el to GNU ELPA,
rather than the negative "it will not be added to Emacs core".  I
believe adding it to GNU ELPA will send most of the positive and
encouraging signals that we all want to communicate to the
outside-of-ELPA package author crowd.

Best regards,
Stefan Kangas

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> > I thought we had a golden opportunity to put s.el inside Emacs:
> 
> We can still put it in GNU ELPA, which is just as good.  Frankly, it's
> better.  We can pick and choose all the best ideas and include them in
> a way that fits with the rest of Emacs.  And users who still prefer
> s.el are free to use it, including in other GNU ELPA packages.
> 
> > - I can try to suggest a few aliases, and maybe one or two will be
> > accepted, but certainly not a lot.
> > - I can try to suggest a few new functions, and maybe one or two will
> > be accepted.
> 
> Sounds good, thank you.  (I'm not sure why you think that most of your
> suggestions would be rejected, though.)
> 
> > - Whatever is introduced is likely to be "Emacsified" as not to look
> > too much like clojure.
> 
> The way I understand this discussion, I don't think the point is that
> we just want to be different for the sake of being different.  It's
> just that we can't import functions wholesale simply to be similar to
> another language when we already have perfectly good alternatives in
> ELisp.  Or at least we can't do that in core (OTOH it's fine if a
> library like s.el wants to do that).
> 
> > - I can notify the author of s.el that only a tiny subset of s.el (if
> > any) is likely to be imported, but he should know he's very welcome
> to
> > put s.el on ELPA.
> 
> If I read you correctly, you're a bit disappointed.  Even so, please
> focus on the positive, that we would like to add s.el to GNU ELPA,
> rather than the negative "it will not be added to Emacs core".  I
> believe adding it to GNU ELPA will send most of the positive and
> encouraging signals that we all want to communicate to the
> outside-of-ELPA package author crowd.

+1 to everything Stefan K said there.
(And expressed constructively, as usual.)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I don't think we should distribute or recommend s.el in any way.
It would be self-defeating to do that.

This is not a moral issue.  s.el is free software; it is not evil.
But we have to exercize good technical judgment as well as moral judgment.
We don't want people to make their Emacs Lisp code depend on s.el,
because that would not fit into Emacs well.  So we should avoid
encouraging making Emacs Lisp code depend on s.el.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> This is not a moral issue.  s.el is free software; it is not evil.
> But we have to exercize good technical judgment as well as moral judgment.
> We don't want people to make their Emacs Lisp code depend on s.el,
> because that would not fit into Emacs well.  So we should avoid
> encouraging making Emacs Lisp code depend on s.el.

I have the impression that you don't live in the same universe as mine:
in my world, `s.el` is already used by the majority of new packages even
tho it's neither part of Emacs nor of GNU ELPA.  Not including it in GNU
ELPA just increases the difficulty of accepting other packages in GNU ELPA.

We can try and provide something better, but as this long thread about
trying to have a more organized namespace has shown, I don't think this
is going to happen any time soon.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Wed, May 6, 2020 at 2:37 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> > This is not a moral issue.  s.el is free software; it is not evil.
> > But we have to exercize good technical judgment as well as moral judgment.
> > We don't want people to make their Emacs Lisp code depend on s.el,
> > because that would not fit into Emacs well.  So we should avoid
> > encouraging making Emacs Lisp code depend on s.el.
>
> I have the impression that you don't live in the same universe as mine:
> in my world, `s.el` is already used by the majority of new packages even
> tho it's neither part of Emacs nor of GNU ELPA.

Yes, but that does not make Richard's technical points less valid,
I think.  Though, Richard, I do think it is somewhat immoral. It is
somewhat immoral not to let widely used and Free software into our
universe just because we don't have the technical apparatus to
control the damage it would bring with it. We should all work together
to acquire that apparatus instead.

João

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    not to let widely used and Free software into our
  > universe just because we don't have the technical apparatus to
  > control the damage it would bring with it.

You have stated an inaccurate picture of the situation, but I can't
respond to that statement on a technical level because you have mixed
personal criticism into it.  If you bring up the issues in purely
techical terms, I can respond in that way.

We need to be kind to each other even when if we disagree, so please
don't try to argue by putdowns.  Please see the Kind Communication
Guidelines, https://gnu.org/philosophy/kind-communication.html.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Thu, May 7, 2020 at 3:45 AM Richard Stallman <rms@gnu.org> wrote:
>
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>     not to let widely used and Free software into our
>   > universe just because we don't have the technical apparatus to
>   > control the damage it would bring with it.
>
> You have stated an inaccurate picture of the situation, but I can't
> respond to that statement on a technical level because you have mixed
> personal criticism into it.  If you bring up the issues in purely
> techical terms, I can respond in that way.

You've completely misunderstood me, Richard. I was not attacking your
position at all, much less attack you personally. Or Stefan.  I *agree* with
you: puting s.el in Emacs or GNU ELPA is not good *for* exactly the reasons
you state.  But I also *agree*  with Stefan, when he laments that that library
which is free software and used widely in third party package can't easily
be added to GNU ELPA and/or Emacs.

Now, what I stated is that it would be immoral to permit this to continue:
Immoral is a strong word. I should have used "unfair" or "untenable".
Let me try again:

  Is is untenable not to let widely used and Free software into our universe
  just because we don't have the technical apparatus to control the damage.

Q: What damage is that? A: Namespace pollution from a package with a
very short prefix.

Q: What could we do to control it? A: Change the package to not have a very
short prefix. Q: What is the problem with that? A: Many package users like it
and already use it with the short prefix.

Q: So what could we do, technically, to remedy the situation? A: Provide
a way for the package to have a longer prefix *and*  package users
to use it with a short prefix.

This is my only and main contribution to this discussion.  I have surveyed
existing solutions to last question and have counted about 4 or 5 existing
ones, and many new ideas.

Thank you,
João

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Now, what I stated is that it would be immoral to permit this to continue:
  > Immoral is a strong word. I should have used

Thank you for taking back that word.  It hurt my feelings.

   > "unfair" or "untenable".

Or perhaps "Would have a significant drawback"?  That I can agree with
you about.  Still, it may be the best choice available under the
circumstances.

  > Q: So what could we do, technically, to remedy the situation? A: Provide
  > a way for the package to have a longer prefix *and*  package users
  > to use it with a short prefix.

I agree that would be the ideal outcome, but it is easier said than done.

In the 1980s, developing the Lisp Machine and Common Lisp, we designed
a namespace (called "package") system the best way we could see, and
it turned out so problematic in use that I concluded the best practice
was never to use it nontrivially.  Thus I decided, when writing Emacs
Lisp to, to avoid conflicts by means of name prefixes, and not have
packages at all.

However, the state of the art may have advanced since then.  This week
you said there was a kind of namespace system for Lisp that works well
and avoids those problems.  If that is true, it could be a good
solution.

I couldn't follow how that worksn or why it is better.  That is not
your fault.  I was so overloaded and hurried that I didn't have time
for careful reading of your technical arguments.  Thus, at present I
have not seen a demonstration that we have a good solution.

I am not saying we don't, only that I don't know enough to judge.

Another obstacle for my reading those messages was that you were
responding to other people's questions, which were not the same
questions that I need to understand.

Can you show me programmer's intro to using a package system of the
sort you're advocating?  I think I could tell what I need to know from
that.

After that, would you be willing to talk with me by voice so I can
understand enough to see whether this is a good solution?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 4554 bytes --]

On Fri, May 8, 2020 at 3:49 AM Richard Stallman <rms@gnu.org> wrote:

> Thank you for taking back that word.  It hurt my feelings.

My apologies, again

> Or perhaps "Would have a significant drawback"?  That I can agree with
> you about.  Still, it may be the best choice available under the
> circumstances.

I agree.  Under the current circumstances.  But we can work to change
the circumstances.

> was never to use it nontrivially.  Thus I decided, when writing Emacs
> Lisp to, to avoid conflicts by means of name prefixes, and not have
> packages at all.
> However, the state of the art may have advanced since then.

It has indeed evolved since the 1980's.  Most implementations of Common
Lisp now have something called "package local nicknames", which many
find advantageous.  There are proponents and adversaries.  I'm one
of the proponents, and if you wish I can sing you the joys of the CL
package system.  But that can be for later.

Currently, I count around 7 different implementations and even more
vapourware.  The debate over which to choose is likely to be very
lengthy.  In the end we can even choose more than one system.

But we should keep our eyes on the prize, as they say, and resolve
the s.el and dash.el (and also f.el) situation in the shorter term, so
that those free programs can join our ranks without hurting us,
our existing users, or their existing users.

> This week you said there was a kind of namespace system for Lisp that
> works well and avoids those problems.  If that is true, it could be a
> good solution.

Indeed I said so: I was referring to a simple shorthand system.  In this
system, there are no profound changes to how symbols are organized in
Emacs.  Things can have different names in different contexts, much as
"RMS" means something different in Electrical Engineering or in
this mailing list.

> Another obstacle for my reading those messages was that you were
> responding to other people's questions, which were not the same
> questions that I need to understand.
>
> Can you show me programmer's intro to using a package system of the
> sort you're advocating?  I think I could tell what I need to know from
> that.

Yes, that's a good idea.  So our goal is to use the problematic s.el
library without having it pollute our namespace, right?  To do that, we
first load the library shorthand.el (attached to this message) into
our Emacs.

Then, we must change s.el minimally.  Its contents are (very truncanted):

   ;;; s.el --- The long lost Emacs string manipulation library.
   ;; Author: Magnar Sveen <magnars@gmail.com>

   (defun s-collapse-whitespace (s)
      "Convert all adjacent whitespace characters to a single space."
    ...)

   (defun s-lines (s)
     "Splits S into a list of strings on newline characters."
    ...)

   (provide 's)
   ;;; s.el ends here,

We need to ask the author to add a few lines to the end of the
file and maybe also change the file name:

   ;;; magnar-string.el --- Now with 99% less namespace pain

   (defun s-collapse-whitespace (s) ...)

   (defun s-lines (s) ...)

   (provide 'magnar-string)
   ;;; magnar-string.el ends here,
   ;; Local Variables:
   ;; shorthand-shorthands: (("^s-" . "magnar-string-"))
   ;; End:

Theoretically, we could avoid this step altogether and "guess" the
shorthand from a list of known problematic cases.

Now, Richard, when you load the new magnar-string.el file into your
Emacs namespace won't be polluted with those two names.  Instead, the
the two symbols will be called `magnar-string-lines' and
`magnar-string-collapse-whitespace'.

In your hypothetical "foobarbaz.el"  you can now:

   (require 'magnar-string)
   (defun f-test () (interactive) (message (cadr (s-lines "this\nworks"))))
   ;; Local Variables:
   ;; shorthand-shorthands: (("^s-" . "magnar-string-") ("^f" . "foobarfaz"))
   ;; End:

Again, by loading foobarbaz.el you will have created the function
`foobarbaz-test', despite having typed `f-test'.

> After that, would you be willing to talk with me by voice so I can
> understand enough to see whether this is a good solution?

Yes.  What voice system do you prefer?  I live in GMT, btw.

João

PS: I attach to this message the shorthand.el file mentioned above,
followed by the modified magnar-string.el.  Please note that
shorthand.el is EXPERIMENTAL.  If you want to try it, you should read it
beforehand (it's less than 50 lines) and load it into a separate Emacs.

[-- Attachment #2: magnar-string.el --]
[-- Type: text/x-emacs-lisp, Size: 24761 bytes --]

;;; magnar-string.el --- Formerly s.el, now with 99% less namespace pain!

;; Copyright (C) 2012-2015 Magnar Sveen

;; Author: Magnar Sveen <magnars@gmail.com>
;; Hacked-by: João Távora <joaotavora@gmail.com>
;; Version: 1.12.0-xxx
;; Keywords: strings

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <http://www.gnu.org/licenses/>.

;;; Commentary:

;; The long lost Emacs string manipulation library,
;; now with 99% less namespace pain!
;;
;; See documentation on https://github.com/magnars/s.el#functions

;;; Code:

;; Silence byte-compiler
(defvar ucs-normalize-combining-chars)  ; Defined in `ucs-normalize'
(autoload 'slot-value "eieio")

(defun s-trim-left (s)
  "Remove whitespace at the beginning of S."
  (declare (pure t) (side-effect-free t))
  (save-match-data
    (if (string-match "\\`[ \t\n\r]+" s)
        (replace-match "" t t s)
      s)))

(defun s-trim-right (s)
  "Remove whitespace at the end of S."
  (save-match-data
    (declare (pure t) (side-effect-free t))
    (if (string-match "[ \t\n\r]+\\'" s)
        (replace-match "" t t s)
      s)))

(defun s-trim (s)
  "Remove whitespace at the beginning and end of S."
  (declare (pure t) (side-effect-free t))
  (s-trim-left (s-trim-right s)))

(defun s-collapse-whitespace (s)
  "Convert all adjacent whitespace characters to a single space."
  (declare (pure t) (side-effect-free t))
  (replace-regexp-in-string "[ \t\n\r]+" " " s))

(defun s-split (separator s &optional omit-nulls)
  "Split S into substrings bounded by matches for regexp SEPARATOR.
If OMIT-NULLS is non-nil, zero-length substrings are omitted.

This is a simple wrapper around the built-in `split-string'."
  (declare (side-effect-free t))
  (save-match-data
    (split-string s separator omit-nulls)))

(defun s-split-up-to (separator s n &optional omit-nulls)
  "Split S up to N times into substrings bounded by matches for regexp SEPARATOR.

If OMIT-NULLS is non-nil, zero-length substrings are omitted.

See also `s-split'."
  (declare (side-effect-free t))
  (save-match-data
    (let ((op 0)
          (r nil))
      (with-temp-buffer
        (insert s)
        (setq op (goto-char (point-min)))
        (while (and (re-search-forward separator nil t)
                    (< 0 n))
          (let ((sub (buffer-substring op (match-beginning 0))))
            (unless (and omit-nulls
                         (equal sub ""))
              (push sub r)))
          (setq op (goto-char (match-end 0)))
          (setq n (1- n)))
        (let ((sub (buffer-substring op (point-max))))
          (unless (and omit-nulls
                       (equal sub ""))
            (push sub r))))
      (nreverse r))))

(defun s-lines (s)
  "Splits S into a list of strings on newline characters."
  (declare (pure t) (side-effect-free t))
  (s-split "\\(\r\n\\|[\n\r]\\)" s))

(defun s-join (separator strings)
  "Join all the strings in STRINGS with SEPARATOR in between."
  (declare (pure t) (side-effect-free t))
  (mapconcat 'identity strings separator))

(defun s-concat (&rest strings)
  "Join all the string arguments into one string."
  (declare (pure t) (side-effect-free t))
  (apply 'concat strings))

(defun s-prepend (prefix s)
  "Concatenate PREFIX and S."
  (declare (pure t) (side-effect-free t))
  (concat prefix s))

(defun s-append (suffix s)
  "Concatenate S and SUFFIX."
  (declare (pure t) (side-effect-free t))
  (concat s suffix))

(defun s-repeat (num s)
  "Make a string of S repeated NUM times."
  (declare (pure t) (side-effect-free t))
  (let (ss)
    (while (> num 0)
      (setq ss (cons s ss))
      (setq num (1- num)))
    (apply 'concat ss)))

(defun s-chop-suffix (suffix s)
  "Remove SUFFIX if it is at end of S."
  (declare (pure t) (side-effect-free t))
  (let ((pos (- (length suffix))))
    (if (and (>= (length s) (length suffix))
             (string= suffix (substring s pos)))
        (substring s 0 pos)
      s)))

(defun s-chop-suffixes (suffixes s)
  "Remove SUFFIXES one by one in order, if they are at the end of S."
  (declare (pure t) (side-effect-free t))
  (while suffixes
    (setq s (s-chop-suffix (car suffixes) s))
    (setq suffixes (cdr suffixes)))
  s)

(defun s-chop-prefix (prefix s)
  "Remove PREFIX if it is at the start of S."
  (declare (pure t) (side-effect-free t))
  (let ((pos (length prefix)))
    (if (and (>= (length s) (length prefix))
             (string= prefix (substring s 0 pos)))
        (substring s pos)
      s)))

(defun s-chop-prefixes (prefixes s)
  "Remove PREFIXES one by one in order, if they are at the start of S."
  (declare (pure t) (side-effect-free t))
  (while prefixes
    (setq s (s-chop-prefix (car prefixes) s))
    (setq prefixes (cdr prefixes)))
  s)

(defun s-shared-start (s1 s2)
  "Returns the longest prefix S1 and S2 have in common."
  (declare (pure t) (side-effect-free t))
  (let ((search-length (min (length s1) (length s2)))
        (i 0))
    (while (and (< i search-length)
                (= (aref s1 i) (aref s2 i)))
      (setq i (1+ i)))
    (substring s1 0 i)))

(defun s-shared-end (s1 s2)
  "Returns the longest suffix S1 and S2 have in common."
  (declare (pure t) (side-effect-free t))
  (let* ((l1 (length s1))
         (l2 (length s2))
         (search-length (min l1 l2))
         (i 0))
    (while (and (< i search-length)
                (= (aref s1 (- l1 i 1)) (aref s2 (- l2 i 1))))
      (setq i (1+ i)))
    ;; If I is 0, then it means that there's no common suffix between
    ;; S1 and S2.
    ;;
    ;; However, since (substring s (- 0)) will return the whole
    ;; string, `s-shared-end' should simply return the empty string
    ;; when I is 0.
    (if (zerop i)
        ""
      (substring s1 (- i)))))

(defun s-chomp (s)
  "Remove one trailing `\\n`, `\\r` or `\\r\\n` from S."
  (declare (pure t) (side-effect-free t))
  (s-chop-suffixes '("\n" "\r") s))

(defun s-truncate (len s &optional ellipsis)
  "If S is longer than LEN, cut it down and add ELLIPSIS to the end.

The resulting string, including ellipsis, will be LEN characters
long.

When not specified, ELLIPSIS defaults to ‘...’."
  (declare (pure t) (side-effect-free t))
  (unless ellipsis
    (setq ellipsis "..."))
  (if (> (length s) len)
      (format "%s%s" (substring s 0 (- len (length ellipsis))) ellipsis)
    s))

(defun s-word-wrap (len s)
  "If S is longer than LEN, wrap the words with newlines."
  (declare (side-effect-free t))
  (save-match-data
    (with-temp-buffer
      (insert s)
      (let ((fill-column len))
        (fill-region (point-min) (point-max)))
      (buffer-substring (point-min) (point-max)))))

(defun s-center (len s)
  "If S is shorter than LEN, pad it with spaces so it is centered."
  (declare (pure t) (side-effect-free t))
  (let ((extra (max 0 (- len (length s)))))
    (concat
     (make-string (ceiling extra 2) ? )
     s
     (make-string (floor extra 2) ? ))))

(defun s-pad-left (len padding s)
  "If S is shorter than LEN, pad it with PADDING on the left."
  (declare (pure t) (side-effect-free t))
  (let ((extra (max 0 (- len (length s)))))
    (concat (make-string extra (string-to-char padding))
            s)))

(defun s-pad-right (len padding s)
  "If S is shorter than LEN, pad it with PADDING on the right."
  (declare (pure t) (side-effect-free t))
  (let ((extra (max 0 (- len (length s)))))
    (concat s
            (make-string extra (string-to-char padding)))))

(defun s-left (len s)
  "Returns up to the LEN first chars of S."
  (declare (pure t) (side-effect-free t))
  (if (> (length s) len)
      (substring s 0 len)
    s))

(defun s-right (len s)
  "Returns up to the LEN last chars of S."
  (declare (pure t) (side-effect-free t))
  (let ((l (length s)))
    (if (> l len)
        (substring s (- l len) l)
      s)))

(defun s-ends-with? (suffix s &optional ignore-case)
  "Does S end with SUFFIX?

If IGNORE-CASE is non-nil, the comparison is done without paying
attention to case differences.

Alias: `s-suffix?'"
  (declare (pure t) (side-effect-free t))
  (let ((start-pos (- (length s) (length suffix))))
    (and (>= start-pos 0)
         (eq t (compare-strings suffix nil nil
                                s start-pos nil ignore-case)))))

(defun s-starts-with? (prefix s &optional ignore-case)
  "Does S start with PREFIX?

If IGNORE-CASE is non-nil, the comparison is done without paying
attention to case differences.

Alias: `s-prefix?'. This is a simple wrapper around the built-in
`string-prefix-p'."
  (declare (pure t) (side-effect-free t))
  (string-prefix-p prefix s ignore-case))

(defun s--truthy? (val)
  (declare (pure t) (side-effect-free t))
  (not (null val)))

(defun s-contains? (needle s &optional ignore-case)
  "Does S contain NEEDLE?

If IGNORE-CASE is non-nil, the comparison is done without paying
attention to case differences."
  (declare (pure t) (side-effect-free t))
  (let ((case-fold-search ignore-case))
    (s--truthy? (string-match-p (regexp-quote needle) s))))

(defun s-equals? (s1 s2)
  "Is S1 equal to S2?

This is a simple wrapper around the built-in `string-equal'."
  (declare (pure t) (side-effect-free t))
  (string-equal s1 s2))

(defun s-less? (s1 s2)
  "Is S1 less than S2?

This is a simple wrapper around the built-in `string-lessp'."
  (declare (pure t) (side-effect-free t))
  (string-lessp s1 s2))

(defun s-matches? (regexp s &optional start)
  "Does REGEXP match S?
If START is non-nil the search starts at that index.

This is a simple wrapper around the built-in `string-match-p'."
  (declare (side-effect-free t))
  (s--truthy? (string-match-p regexp s start)))

(defun s-blank? (s)
  "Is S nil or the empty string?"
  (declare (pure t) (side-effect-free t))
  (or (null s) (string= "" s)))

(defun s-blank-str? (s)
  "Is S nil or the empty string or string only contains whitespace?"
  (declare (pure t) (side-effect-free t))
  (or (s-blank? s) (s-blank? (s-trim s))))

(defun s-present? (s)
  "Is S anything but nil or the empty string?"
  (declare (pure t) (side-effect-free t))
  (not (s-blank? s)))

(defun s-presence (s)
  "Return S if it's `s-present?', otherwise return nil."
  (declare (pure t) (side-effect-free t))
  (and (s-present? s) s))

(defun s-lowercase? (s)
  "Are all the letters in S in lower case?"
  (declare (side-effect-free t))
  (let ((case-fold-search nil))
    (not (string-match-p "[[:upper:]]" s))))

(defun s-uppercase? (s)
  "Are all the letters in S in upper case?"
  (declare (side-effect-free t))
  (let ((case-fold-search nil))
    (not (string-match-p "[[:lower:]]" s))))

(defun s-mixedcase? (s)
  "Are there both lower case and upper case letters in S?"
  (let ((case-fold-search nil))
    (s--truthy?
     (and (string-match-p "[[:lower:]]" s)
          (string-match-p "[[:upper:]]" s)))))

(defun s-capitalized? (s)
  "In S, is the first letter upper case, and all other letters lower case?"
  (declare (side-effect-free t))
  (let ((case-fold-search nil))
    (s--truthy?
     (string-match-p "^[[:upper:]][^[:upper:]]*$" s))))

(defun s-numeric? (s)
  "Is S a number?"
  (declare (pure t) (side-effect-free t))
  (s--truthy?
   (string-match-p "^[0-9]+$" s)))

(defun s-replace (old new s)
  "Replaces OLD with NEW in S."
  (declare (pure t) (side-effect-free t))
  (replace-regexp-in-string (regexp-quote old) new s t t))

(defalias 's-replace-regexp 'replace-regexp-in-string)

(defun s--aget (alist key)
  (declare (pure t) (side-effect-free t))
  (cdr (assoc-string key alist)))

(defun s-replace-all (replacements s)
  "REPLACEMENTS is a list of cons-cells. Each `car` is replaced with `cdr` in S."
  (declare (pure t) (side-effect-free t))
  (replace-regexp-in-string (regexp-opt (mapcar 'car replacements))
                            (lambda (it) (s--aget replacements it))
                            s t t))

(defun s-downcase (s)
  "Convert S to lower case.

This is a simple wrapper around the built-in `downcase'."
  (declare (side-effect-free t))
  (downcase s))

(defun s-upcase (s)
  "Convert S to upper case.

This is a simple wrapper around the built-in `upcase'."
  (declare (side-effect-free t))
  (upcase s))

(defun s-capitalize (s)
  "Convert the first word's first character to upper case and the rest to lower case in S."
  (declare (side-effect-free t))
  (concat (upcase (substring s 0 1)) (downcase (substring s 1))))

(defun s-titleize (s)
  "Convert each word's first character to upper case and the rest to lower case in S.

This is a simple wrapper around the built-in `capitalize'."
  (declare (side-effect-free t))
  (capitalize s))

(defmacro s-with (s form &rest more)
  "Threads S through the forms. Inserts S as the last item
in the first form, making a list of it if it is not a list
already. If there are more forms, inserts the first form as the
last item in second form, etc."
  (declare (debug (form &rest [&or (function &rest form) fboundp])))
  (if (null more)
      (if (listp form)
          `(,(car form) ,@(cdr form) ,s)
        (list form s))
    `(s-with (s-with ,s ,form) ,@more)))

(put 's-with 'lisp-indent-function 1)

(defun s-index-of (needle s &optional ignore-case)
  "Returns first index of NEEDLE in S, or nil.

If IGNORE-CASE is non-nil, the comparison is done without paying
attention to case differences."
  (declare (pure t) (side-effect-free t))
  (let ((case-fold-search ignore-case))
    (string-match-p (regexp-quote needle) s)))

(defun s-reverse (s)
  "Return the reverse of S."
  (declare (pure t) (side-effect-free t))
  (save-match-data
    (if (multibyte-string-p s)
        (let ((input (string-to-list s))
              output)
          (require 'ucs-normalize)
          (while input
            ;; Handle entire grapheme cluster as a single unit
            (let ((grapheme (list (pop input))))
              (while (memql (car input) ucs-normalize-combining-chars)
                (push (pop input) grapheme))
              (setq output (nconc (nreverse grapheme) output))))
          (concat output))
      (concat (nreverse (string-to-list s))))))

(defun s-match-strings-all (regex string)
  "Return a list of matches for REGEX in STRING.

Each element itself is a list of matches, as per
`match-string'. Multiple matches at the same position will be
ignored after the first."
  (declare (side-effect-free t))
  (save-match-data
    (let ((all-strings ())
          (i 0))
      (while (and (< i (length string))
                  (string-match regex string i))
        (setq i (1+ (match-beginning 0)))
        (let (strings
              (num-matches (/ (length (match-data)) 2))
              (match 0))
          (while (/= match num-matches)
            (push (match-string match string) strings)
            (setq match (1+ match)))
          (push (nreverse strings) all-strings)))
      (nreverse all-strings))))

(defun s-matched-positions-all (regexp string &optional subexp-depth)
  "Return a list of matched positions for REGEXP in STRING.
SUBEXP-DEPTH is 0 by default."
  (declare (side-effect-free t))
  (if (null subexp-depth)
      (setq subexp-depth 0))
  (save-match-data
    (let ((pos 0) result)
      (while (and (string-match regexp string pos)
                  (< pos (length string)))
        (let ((m (match-end subexp-depth)))
          (push (cons (match-beginning subexp-depth) (match-end subexp-depth)) result)
          (setq pos (match-end 0))))
      (nreverse result))))

(defun s-match (regexp s &optional start)
  "When the given expression matches the string, this function returns a list
of the whole matching string and a string for each matched subexpressions.
If it did not match the returned value is an empty list (nil).

When START is non-nil the search will start at that index."
  (declare (side-effect-free t))
  (save-match-data
    (if (string-match regexp s start)
        (let ((match-data-list (match-data))
              result)
          (while match-data-list
            (let* ((beg (car match-data-list))
                   (end (cadr match-data-list))
                   (subs (if (and beg end) (substring s beg end) nil)))
              (setq result (cons subs result))
              (setq match-data-list
                    (cddr match-data-list))))
          (nreverse result)))))

(defun s-slice-at (regexp s)
  "Slices S up at every index matching REGEXP."
  (declare (side-effect-free t))
  (if (= 0 (length s)) (list "")
    (save-match-data
      (let (i)
        (setq i (string-match regexp s 1))
        (if i
            (cons (substring s 0 i)
                  (s-slice-at regexp (substring s i)))
          (list s))))))

(defun s-split-words (s)
  "Split S into list of words."
  (declare (side-effect-free t))
  (s-split
   "[^[:word:]0-9]+"
   (let ((case-fold-search nil))
     (replace-regexp-in-string
      "\\([[:lower:]]\\)\\([[:upper:]]\\)" "\\1 \\2"
      (replace-regexp-in-string "\\([[:upper:]]\\)\\([[:upper:]][0-9[:lower:]]\\)" "\\1 \\2" s)))
   t))

(defun s--mapcar-head (fn-head fn-rest list)
  "Like MAPCAR, but applies a different function to the first element."
  (if list
      (cons (funcall fn-head (car list)) (mapcar fn-rest (cdr list)))))

(defun s-lower-camel-case (s)
  "Convert S to lowerCamelCase."
  (declare (side-effect-free t))
  (s-join "" (s--mapcar-head 'downcase 'capitalize (s-split-words s))))

(defun s-upper-camel-case (s)
  "Convert S to UpperCamelCase."
  (declare (side-effect-free t))
  (s-join "" (mapcar 'capitalize (s-split-words s))))

(defun s-snake-case (s)
  "Convert S to snake_case."
  (declare (side-effect-free t))
  (s-join "_" (mapcar 'downcase (s-split-words s))))

(defun s-dashed-words (s)
  "Convert S to dashed-words."
  (declare (side-effect-free t))
  (s-join "-" (mapcar 'downcase (s-split-words s))))

(defun s-capitalized-words (s)
  "Convert S to Capitalized words."
  (declare (side-effect-free t))
  (let ((words (s-split-words s)))
    (s-join " " (cons (capitalize (car words)) (mapcar 'downcase (cdr words))))))

(defun s-titleized-words (s)
  "Convert S to Titleized Words."
  (declare (side-effect-free t))
  (s-join " " (mapcar 's-titleize (s-split-words s))))

(defun s-word-initials (s)
  "Convert S to its initials."
  (declare (side-effect-free t))
  (s-join "" (mapcar (lambda (ss) (substring ss 0 1))
                     (s-split-words s))))

;; Errors for s-format
(progn
  (put 's-format-resolve
       'error-conditions
       '(error s-format s-format-resolve))
  (put 's-format-resolve
       'error-message
       "Cannot resolve a template to values"))

(defun s-format (template replacer &optional extra)
  "Format TEMPLATE with the function REPLACER.

REPLACER takes an argument of the format variable and optionally
an extra argument which is the EXTRA value from the call to
`s-format'.

Several standard `s-format' helper functions are recognized and
adapted for this:

    (s-format \"${name}\" 'gethash hash-table)
    (s-format \"${name}\" 'aget alist)
    (s-format \"$0\" 'elt sequence)

The REPLACER function may be used to do any other kind of
transformation."
  (let ((saved-match-data (match-data)))
    (unwind-protect
        (replace-regexp-in-string
         "\\$\\({\\([^}]+\\)}\\|[0-9]+\\)"
         (lambda (md)
           (let ((var
                  (let ((m (match-string 2 md)))
                    (if m m
                      (string-to-number (match-string 1 md)))))
                 (replacer-match-data (match-data)))
             (unwind-protect
                 (let ((v
                        (cond
                         ((eq replacer 'gethash)
                          (funcall replacer var extra))
                         ((eq replacer 'aget)
                          (funcall 's--aget extra var))
                         ((eq replacer 'elt)
                          (funcall replacer extra var))
                         ((eq replacer 'oref)
                          (funcall #'slot-value extra (intern var)))
                         (t
                          (set-match-data saved-match-data)
                          (if extra
                              (funcall replacer var extra)
                            (funcall replacer var))))))
                   (if v (format "%s" v) (signal 's-format-resolve md)))
               (set-match-data replacer-match-data)))) template
               ;; Need literal to make sure it works
               t t)
      (set-match-data saved-match-data))))

(defvar s-lex-value-as-lisp nil
  "If `t' interpolate lisp values as lisp.

`s-lex-format' inserts values with (format \"%S\").")

(defun s-lex-fmt|expand (fmt)
  "Expand FMT into lisp."
  (declare (side-effect-free t))
  (list 's-format fmt (quote 'aget)
        (append '(list)
                (mapcar
                 (lambda (matches)
                   (list
                    'cons
                    (cadr matches)
                    `(format
                      (if s-lex-value-as-lisp "%S" "%s")
                      ,(intern (cadr matches)))))
                 (s-match-strings-all "${\\([^}]+\\)}" fmt)))))

(defmacro s-lex-format (format-str)
  "`s-format` with the current environment.

FORMAT-STR may use the `s-format' variable reference to refer to
any variable:

 (let ((x 1))
   (s-lex-format \"x is: ${x}\"))

The values of the variables are interpolated with \"%s\" unless
the variable `s-lex-value-as-lisp' is `t' and then they are
interpolated with \"%S\"."
  (declare (debug (form)))
  (s-lex-fmt|expand format-str))

(defun s-count-matches (regexp s &optional start end)
  "Count occurrences of `regexp' in `s'.

`start', inclusive, and `end', exclusive, delimit the part of `s' to
match.  `start' and `end' are both indexed starting at 1; the initial
character in `s' is index 1.

This function starts looking for the next match from the end of the
previous match.  Hence, it ignores matches that overlap a previously
found match.  To count overlapping matches, use
`s-count-matches-all'."
  (declare (side-effect-free t))
  (save-match-data
    (with-temp-buffer
      (insert s)
      (goto-char (point-min))
      (count-matches regexp (or start 1) (or end (point-max))))))

(defun s-count-matches-all (regexp s &optional start end)
  "Count occurrences of `regexp' in `s'.

`start', inclusive, and `end', exclusive, delimit the part of `s' to
match.  `start' and `end' are both indexed starting at 1; the initial
character in `s' is index 1.

This function starts looking for the next match from the second
character of the previous match.  Hence, it counts matches that
overlap a previously found match.  To ignore matches that overlap a
previously found match, use `s-count-matches'."
  (declare (side-effect-free t))
  (let* ((anchored-regexp (format "^%s" regexp))
         (match-count 0)
         (i 0)
         (narrowed-s (substring s
                                (when start (1- start))
                                (when end (1- end)))))
    (save-match-data
      (while (< i (length narrowed-s))
        (when (s-matches? anchored-regexp (substring narrowed-s i))
          (setq match-count (1+ match-count)))
        (setq i (1+ i))))
    match-count))

(defun s-wrap (s prefix &optional suffix)
  "Wrap string S with PREFIX and optionally SUFFIX.

Return string S with PREFIX prepended.  If SUFFIX is present, it
is appended, otherwise PREFIX is used as both prefix and
suffix."
  (declare (pure t) (side-effect-free t))
  (concat prefix s (or suffix prefix)))

\f
;;; Aliases

(defalias 's-blank-p 's-blank?)
(defalias 's-blank-str-p 's-blank-str?)
(defalias 's-capitalized-p 's-capitalized?)
(defalias 's-contains-p 's-contains?)
(defalias 's-ends-with-p 's-ends-with?)
(defalias 's-equals-p 's-equals?)
(defalias 's-less-p 's-less?)
(defalias 's-lowercase-p 's-lowercase?)
(defalias 's-matches-p 's-matches?)
(defalias 's-mixedcase-p 's-mixedcase?)
(defalias 's-numeric-p 's-numeric?)
(defalias 's-prefix-p 's-starts-with?)
(defalias 's-prefix? 's-starts-with?)
(defalias 's-present-p 's-present?)
(defalias 's-starts-with-p 's-starts-with?)
(defalias 's-suffix-p 's-ends-with?)
(defalias 's-suffix? 's-ends-with?)
(defalias 's-uppercase-p 's-uppercase?)

\f
(provide 'magnar-string)
;;; magnar-string.el ends here,
;; Local Variables:
;; shorthand-shorthands: (("^s-" . "magnar-string-"))
;; End:

[-- Attachment #3: shorthand.el --]
[-- Type: text/x-emacs-lisp, Size: 2956 bytes --]

;;; shorthand.el --- namespacing system  -*- lexical-binding: t; -*-

;; Copyright (C) 2020  Free Software Foundation

;; Author: João Távora <joaotavora@gmail.com>
;; Keywords: languages, lisp

;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation, either version 3 of the License, or
;; (at your option) any later version.

;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;; GNU General Public License for more details.

;; You should have received a copy of the GNU General Public License
;; along with this program.  If not, see <https://www.gnu.org/licenses/>.

;;; Commentary:

;;; Code:

(require 'cl-lib)

(defvar shorthand-shorthands nil)
(put 'shorthand-shorthands 'safe-local-variable #'consp)

(defun shorthand--expand-shorthand (form)
  (cl-typecase form
    (cons (setcar form (shorthand--expand-shorthand (car form)))
          (setcdr form (shorthand--expand-shorthand (cdr form))))
    (vector (cl-loop for i from 0 for e across form
                     do (aset form i (shorthand--expand-shorthand e))))
    (symbol (let* ((name (symbol-name form)))
              (cl-loop for (short-pat . long-pat) in shorthand-shorthands
                       when (string-match short-pat name)
                       do (setq name (replace-match long-pat t nil name)))
              (setq form (intern name))))
    (string) (number)
    (t       (message "[shorthand] unexpectged %s" (type-of form))))
  form)

(defun shorthand-read-wrapper (wrappee stream &rest stuff)
  (if (and load-file-name (string-match "\\.elc$" load-file-name))
      (apply wrappee stream stuff)
    (shorthand--expand-shorthand
     (let ((obarray (obarray-make))) (apply wrappee stream stuff)))))

(defun shorthand-intern-soft-wrapper (wrappee name &rest stuff)
  (let ((res (apply wrappee name stuff)))
    (or res (cl-loop
             for (short-pat . long-pat) in shorthand-shorthands
             thereis (apply wrappee
                            (replace-regexp-in-string short-pat
                                                      long-pat name)
                            stuff)))))

(defun shorthand-load-wrapper (wrappee file &rest stuff)
  (let (file-local-shorthands)
    (when (file-readable-p file)
      (with-temp-buffer
        (insert-file-contents file)
        (hack-local-variables)
        (setq file-local-shorthands shorthand-shorthands)))
    (let ((shorthand-shorthands file-local-shorthands))
      (apply wrappee file stuff)))))

(advice-add 'read        :around #'shorthand-read-wrapper)
(advice-add 'intern-soft :around #'shorthand-intern-soft-wrapper)
(advice-add 'load        :around #'shorthand-load-wrapper)

(provide 'shorthand)
;;; shorthand.el ends here

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It is clear that shorthand.el avoids the Lisp namespace traps.
It aims to do something far less sophisticated, and I think
that if it ever doesn't do what the user wants, 
perse can see how to work around it.

What objections have people stated to it?
Have people found problems in it, or have they only
criticized it for not being a real namespace system?

Of course, what we actually install should not be implemented using
advice.  It should be added to the C code.

Perhaps if you load a file, magnars.el, for instance, which specifies
symbol renaming rules, Emacs should remembers which symbols were
renamed and how.  Then it could automatically do precisely those
renamings to each on each file that somehow says it uses magnars.el.

Would this have a flaw?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[João Távora]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

> Have people found problems in it, or have they only
> criticized it for not being a real namespace system?

Namespace systems are a broad concept.  If one takes such things to
posess a first-class object holding each individual space of names, then
that feature is absent from shorthand.el.

This means it's hard to do automatic checking of consistency in naming,
much as the Common Lisp package facility does (and does it very well).

That is what we're trading off in return for a very simple facility.

However, if people are looking for other constructs to manage
namespaces, it very much possible to develop shorthand.el so that you
can have:

   (shorthand-in-namespace FOO :as "f")

For more complicated cases where the library foo.el doesn't have a
single prefix or renaming rule, we can use more sophisticated forms.

I just provided the buffer-local-variable technique:

   ;; Local Variables:
   ;; shorthand-shorthands: (("^s-" . "magnar-string-"))
   ;; End:

because it's practical, understood, and would be self-evident, i.e. work
without much "magic".

João

Re: Imports / inclusion of s.el into Emacs

[Adam Porter]

João Távora <joaotavora@gmail.com> writes:

> However, if people are looking for other constructs to manage
> namespaces, it very much possible to develop shorthand.el so that you
> can have:
>
>    (shorthand-in-namespace FOO :as "f")
>
> For more complicated cases where the library foo.el doesn't have a
> single prefix or renaming rule, we can use more sophisticated forms.
>
> I just provided the buffer-local-variable technique:
>
>    ;; Local Variables:
>    ;; shorthand-shorthands: (("^s-" . "magnar-string-"))
>    ;; End:
>
> because it's practical, understood, and would be self-evident, i.e. work
> without much "magic".

I'm hopeful that this solution, or something like it, will make it into
Emacs!  Thank you for working on this, João.

One minor drawback I see is that the symbol renaming is placed at the
bottom of an Elisp file, while the loading of the library whose symbols
are renamed is (usually) at the top of the file.

Would a macro something like this be feasible, to (require ...) the
library and specify the renamings together?

  (shorthand-require 'magnar-string :with "^s-" :as "magnar-string-")

Re: Imports / inclusion of s.el into Emacs

[João Távora]

Adam Porter <adam@alphapapa.net> writes:

> João Távora <joaotavora@gmail.com> writes:
>
> I'm hopeful that this solution, or something like it, will make it into
> Emacs!  Thank you for working on this, João.

You're welcome.  Let's hope we find a practical solution this time
around!

> One minor drawback I see is that the symbol renaming is placed at the
> bottom of an Elisp file, while the loading of the library whose symbols
> are renamed is (usually) at the top of the file.
>
> Would a macro something like this be feasible, to (require ...) the
> library and specify the renamings together?
>
>   (shorthand-require 'magnar-string :with "^s-" :as "magnar-string-")

Yes, something like that is reasoanble.  Or we can just have a table of
"notable renamings", and then the bare (require 'magnar-string) would
come with that.  Or as Richard, suggested, the default renamings could
be given in the magnar-string.el file itself.  They could be collected
within each byte-compile-file or load-file, then restored.  

One thing that I'd like to discuss is whether it's a good idea or not to
rename s.el to magnar-string.el.  Maybe there's a way to keep calling it
s.el and let every client keep using (require 's).

Another thing that has to implemented is a special syntax to escape the
renamings (imagine that you want to use some legitimate "s-foo" function
that your renaming has shadowed)

João

Re: Imports / inclusion of s.el into Emacs

[Adam Porter]

João Távora <joaotavora@gmail.com> writes:

>> One minor drawback I see is that the symbol renaming is placed at the
>> bottom of an Elisp file, while the loading of the library whose symbols
>> are renamed is (usually) at the top of the file.
>>
>> Would a macro something like this be feasible, to (require ...) the
>> library and specify the renamings together?
>>
>>   (shorthand-require 'magnar-string :with "^s-" :as "magnar-string-")
>
> Yes, something like that is reasoanble.  Or we can just have a table of
> "notable renamings", and then the bare (require 'magnar-string) would
> come with that.  Or as Richard, suggested, the default renamings could
> be given in the magnar-string.el file itself.  They could be collected
> within each byte-compile-file or load-file, then restored.  

I may be missing something, but it strikes me as almost duplicating the
original problem, if libraries are allowed to define their own symbol
abbreviations.  IOW, it seems almost like solving the problem of
libraries' "polluting" the global symbol namespace by allowing libraries
to "pollute" the global symbol *abbreviation* namespace (in the sense
that one would exist).

Similarly to what you said, what if one user wants to write a package
using magnar-string.el with the "^s-" abbreviation, but another user
wants to write a package using a hypothetical super.el with the same
abbreviation, and both magnar-string.el and super.el define a symbol
`foo'?  Or what if both magnar-string.el and super.el defined the same
default abbreviation (since there is no enforcing body to prevent
conflicts)?  Shouldn't it be up to the "requiring" library to decide
which shorthands it uses for libraries it requires?

> One thing that I'd like to discuss is whether it's a good idea or not to
> rename s.el to magnar-string.el.  Maybe there's a way to keep calling it
> s.el and let every client keep using (require 's).

I suppose that would be ideal if it's possible.  On the other hand, if a
user wanted to write a package using s.el with a prefix like
"^mstring-", and a hypothetical super.el with the "^s-" prefix, it would
seem confusing for symbols like "s-foo" to not refer to those loaded by
(require 's).  The "don't do that if it hurts" axiom may apply, of
course, but it would seem preferable to avoid such a situation.

> Another thing that has to implemented is a special syntax to escape the
> renamings (imagine that you want to use some legitimate "s-foo" function
> that your renaming has shadowed)

Would a table of buffer-local symbol aliases work?  Maybe it could even
work something like:

  (defalias-local :canonical 's-foo :local 'real-s-foo)

Which would make the canonical s-foo accessible as real-s-foo in the
local buffer's code.  (The keyword arguments may be superfluous, but it
may also help clarify that the s-foo referred to is the canonical one
rather than a "shorthand"-abbreviated one.)  If that were present in the
file before the (shorthand-require ...), I suppose the "'s-foo" would
not be intended to be read with the abbreviation.  (Not commenting on
the feasibility of the implementation, just musing on the user-facing
API.)

Re: Imports / inclusion of s.el into Emacs

[João Távora]

Adam Porter <adam@alphapapa.net> writes:

> João Távora <joaotavora@gmail.com> writes:

>> Yes, something like that is reasoanble.  Or we can just have a table of
>> "notable renamings", and then the bare (require 'magnar-string) would
>> come with that.  Or as Richard, suggested, the default renamings could
>> be given in the magnar-string.el file itself.  They could be collected
>> within each byte-compile-file or load-file, then restored.  
> I may be missing something, but it strikes me as almost duplicating the
> original problem, if libraries are allowed to define their own symbol
> abbreviations.

You can say that yes, but you wouldn't be duplicating the problem
because the symbols of the library foobarfaz.el that the library
imperiously wants you to access as with the prefix `f-` are still filed
under the prefix `foobarbaz-`, where now you they are filed under `f-`.

But yes, my suggestion to let the library itself decide opens the door
to some contention.

But if Emacs itself curates a list of notable cases, it's probably not
so bad.

> IOW, it seems almost like solving the problem of libraries'
> "polluting" the global symbol namespace by allowing libraries to
> "pollute" the global symbol *abbreviation* namespace (in the sense
> that one would exist).

Yes, exacly.

So maybe not such a good idea. :-)

>> Another thing that has to implemented is a special syntax to escape the
>> renamings (imagine that you want to use some legitimate "s-foo" function
>> that your renaming has shadowed)
>
> Would a table of buffer-local symbol aliases work?  Maybe it could even
> work something like:

I was thinking of making say sth like ##s-lines really tell the reader
to intern the symbol "s-lines", even if there's a renaming rule for
"^s-" active.

so (##s-lines) would call the function s-lines and (defface ##s-lines)
would define that face, etc...

Anyway, you raise good points.  I'm not married to any of these
solutions and I hope I made the code simple enough that others can hack
on it and experiment.

João

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I may be missing something, but it strikes me as almost duplicating the
  > original problem, if libraries are allowed to define their own symbol
  > abbreviations.

That depends on how a library would specify them.  If it could be
arbitrary string substitutions, it might have this consequence.
Joao and I are talking about how to make it more controllable.

  > Similarly to what you said, what if one user wants to write a package
  > using magnar-string.el with the "^s-" abbreviation, but another user
  > wants to write a package using a hypothetical super.el with the same
  > abbreviation, and both magnar-string.el and super.el define a symbol
  > `foo'?

There is no conflict.  In the first package, s-foo expands into magnars-foo.
In the second package, s-foo expands into super-foo.
Neither one actually uses the symbol s-foo.

For that reason, we must not try to specify what the "canonical meaning"
of s-foo is.  It can have multiple ones.

If you wanted to use both magnars.el and super.el in one program,
you'd have to turn off the renamings for one or bot of those two.



-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > One thing that I'd like to discuss is whether it's a good idea or not to
  > rename s.el to magnar-string.el.  Maybe there's a way to keep calling it
  > s.el and let every client keep using (require 's).

We could have a file s.el which is not an ordinary file of Lisp,
but rather a special stub for 'require' to access.

(require 's) would load magnars.el, then set up the renamings for the
rest of the containing file so that it can say s-foo and that will be
renamed to magnars-foo.

The s.el file can include a precise list of the necessary renamings
for callers to use.  This woud be updated by scanning magnars.el.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[João Távora]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > One thing that I'd like to discuss is whether it's a good idea or not to
>   > rename s.el to magnar-string.el.  Maybe there's a way to keep calling it
>   > s.el and let every client keep using (require 's).
>
> We could have a file s.el which is not an ordinary file of Lisp,
> but rather a special stub for 'require' to access.

As I told you off-list, this can certainly be done, if we really can't
or don't change existing code that uses the present s.el library.

The stub s.el technique is only, in my opinion, a "nice to have"
requirement.  I don't expect users of magnars.el to be very bothered to
change one or two lines of each file that currently uses s.el.

The hypothesis here is that these users _want_ to be in GNU ELPA, they
have fulfilled all other requirements, but can't easily switch to our
string manipulation techniques.  So, presumably, they won't mind
changing a bit of their code, especially if we show them that they can
do it in a backward compatible way.

> (require 's) would load magnars.el, then set up the renamings for the
> rest of the containing file so that it can say s-foo and that will be
> renamed to magnars-foo.

Exactly.

> The s.el file can include a precise list of the necessary renamings
> for callers to use.  This woud be updated by scanning magnars.el.

I don't understand this particular bit , the "precise list of necessary
renamings").  It simply isn't needed, from my understanding of how the
system works.  Here's an example: say a user.el file has this content
today:

    ;; user.el -- a file which is a user of s.el
     
    (require 's)
     
    (defun user-thingy ()
       (message "%s" (s-lines "hello\nworld")))

    (provide 'user)
    ;; user.el ends here

It can remain completely unaltered.  

Thus, when compiling user.el, the byte-compiler will evaluate some
forms, including top-level require forms.  After it has evaluated the
first line

   (require 's)

It has loaded the file s.el[c].  That, in turn will have `require`d
magnars.el and set up the `s-` -> `magnars-` translation for the current
compilation session (probably in a buffer-local version of the
shorthand-shorthands variable).  

From this point on, back in the user.el file, the reader knows to
replace the 's-' prefix with 'magnars-'.  This means that the .elc file
produced that will eventually be loaded will have references to longname
version of every symbol.  Not only that but during the reading itself,
the `s-lines` symbol was never interned in the global obarray.

So this is really a shorthand system, very much the way shorthands work
in real life: we write AFAIK and IMNSHO for brevity but we the long
names of these things.

Anyway, when loading files something similar happens.

The only think I'm not sure how it should work is if the value of
shorthand-shorthands setup by requiring s.el should then remain in the
buffer for further byte-compilations of evaluations of specific
functions.  I think it should.

João

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Please forgive me for taking so long to respond.  In late October
email started was pouring in on me, and I am now 700 messages
backlogged.  I just saw your message today, because I decided to
look for mail from you.

  > > The s.el file can include a precise list of the necessary renamings
  > > for callers to use.  This woud be updated by scanning magnars.el.

  > I don't understand this particular bit , the "precise list of necessary
  > renamings").

The idea is that (require 's) in file foo.el specifies renamings for
the rest of foo.el.  The question here is precisely how to specify them.

  > Thus, when compiling user.el, the byte-compiler will evaluate some
  > forms, including top-level require forms.  After it has evaluated the
  > first line

  >    (require 's)

  > It has loaded the file s.el[c].  That, in turn will have `require`d
  > magnars.el

Yes.

	       and set up the `s-` -> `magnars-` translation for the current
  > compilation session (probably in a buffer-local version of the
  > shorthand-shorthands variable).  

That is the part I want to change.

The current shorthands.el idea is to specify renamings with a pattern
(or multiple patterns) -- for instance, "rename everything that starts
with s- to start with magnars- instead."

That would work, but any change in shorthand.el could change the list
of symbols to be renamed.  Such changes could get in without being noticed.

So I am thinking that s.el should specify a list of specific symbol
renamings:

  (("s-prepend" . magnars-prepend)
   ("s-append" . magnars-append)
   ("s-concat" . magnars-concat)
   ...)

WDYT?

BTW, instead of magnars.el, I suggest the name magnar-string.el.
Its meaning will be clearer.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This means it's hard to do automatic checking of consistency in naming,

That is an abstract concept, and I can understand it abstractly,
but I don't know concretely what that would do.

  > However, if people are looking for other constructs to manage
  > namespaces, it very much possible to develop shorthand.el so that you
  > can have:

  >    (shorthand-in-namespace FOO :as "f")

What would that do?  I see values as argunments but what do they mean?



-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[João Távora]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > This means it's hard to do automatic checking of consistency in naming,
>
> That is an abstract concept, and I can understand it abstractly,
> but I don't know concretely what that would do.

You're right, it is abstract.  So let me explain what I meant with a
real-life analogy.  A namespace facility is for managing different
things with the same name in distinct contexts.  At some point you may
want to merge two contexts for convenience, much like the way teachers
will bring two smaller classes together on a class trip.  In those
occasions, a better equipped namespacing facility will have the means to
detect a possible clash in names, and warn the teachers explicitly that
there are now two kids known as "Jimmy".  The teachers may elect to:

- Rename both Jimmys
- Rename just of the Jimmys
- Let one of the Jimmy's be "shadowed" by the other (poor Jimmy...)
- Evict one of the Jimmy's from the class (harsher still)
- Abort the class trip entirely

In all of this example, each Jimmy keeps its identity (i.e. renaming him
doesn't change who he is).

It's these kinds of choices that a good namespacing facility like CL
packages give the user.  It's easier to implement them there because
each namespace is a first-class object: it can be queried
and modified programatically.

In shorthand.el, as in Elisp, a namespace is just a concept that is
managed entirely in the programmer's mind.

However, that doesn't mean conflicts can't arise in shorthand.el,
because naming conflicts are a hazard that comes with naming itself.

Say that you are using the symbol RICHARD-STALLMAN-* symbols from
richard-stallman.el happily in your myprogram.el as RMS-*.

Then, you realize you need to calculate the "Root Mean Square Error", a
mathematical concept, and you load a library mathematical.el where that
is done by the RMS-ERROR function.

You will have a problem, because if you type RMS-ERROR the system will
think you mean RICHARD-STALLMAN-ERROR, a function that may or may not
exist, but at any rate is not the one you intended.

This is why we probably need special syntax to "escape" the renaming, so
that you _can_ call RMS-ERROR from myprogram.el without the
aforementioned problem, perhaps by writing it ##RMS-ERROR or something
like that.  Another option is to give up on RMS-ERROR completely, or
convince the authors of mathematical.el to give their symbols another
prefix.

At any rate, and this was my main point, shorthand.el is _not_ designed
to warn you about these conflicts: it is just a very dumb renaming
system, and lets you shoot yourself in the foot.

However, the argument can be made that Elisp _already_ lets you shoot
yourself in the foot with names, and Elisp programmers will presumably
know to take precautions anyway.  The opponents of CL packages will
probably also argue that the CL package facility has too broad an array
of choices which confuses the beginner programmer.

>   > However, if people are looking for other constructs to manage
>   > namespaces, it very much possible to develop shorthand.el so that you
>   > can have:
>
>   >    (shorthand-in-namespace FOO :as "f")
>
> What would that do?  I see values as argunments but what do they mean?

This enacts the shorhand rule "^f-" -> "foo-" for the following forms:

(f-bar)                                         ;; this really does mean (f-bar)
(shorthand-in-namespace foo :as "f")
(f-bar)                                         ;; but this means (foo-bar)
(shorthand-in-namespace foo :as "heyhey")
(heyhey-bar)                                    ;; this also means (foo-bar)
(f-bar)                                         ;; this again means (f-bar)

João

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This means it's hard to do automatic checking of consistency in naming,

That is an abstract concept, and I can understand it abstractly,
but I don't know concretely what that would do.

  > However, if people are looking for other constructs to manage
  > namespaces, it very much possible to develop shorthand.el so that you
  > can have:

  >    (shorthand-in-namespace FOO :as "f")

What would that do?  I see values as argunments but what do they mean?

  >    ;; shorthand-shorthands: (("^s-" . "magnar-string-"))
  >    ;; End:

  > because it's practical, understood, and would be self-evident, i.e. work
  > without much "magic".

You didn't say what that call means, but I was able to figure it out.

I think this feature is good when used in a sane way, but we need some
control to make sure people don't use it in absurd ways.  Also, when a
file uses magnarstring.el, it should not have to duplicate the
renamings of magnarstring.el.  It should be able to say, "This refers
to magnarstring.el".

Also, the renamings in the using program should only apply to the
names specified somehow by magnarstring.el.  If that file happens to
contain a symbol named s-means-stop, that shouldn't get renamed to
magnarstring-means-stop.

I think this calls for magnarstring.el to contain a list of the
individual renamings.  When you edit the file you should be able to
run a command that would update that list in the file.

Then files which use magnarstring.el should be able to snarf that list
out of magnarstring.el and apply those specific renamings.

WDYT?



-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

> > This is not a moral issue.  s.el is free software; it is not evil.
> > But we have to exercize good technical judgment as well as moral judgment.
> > We don't want people to make their Emacs Lisp code depend on s.el,
> > because that would not fit into Emacs well.  So we should avoid
> > encouraging making Emacs Lisp code depend on s.el.
>
> I have the impression that you don't live in the same universe as mine:
> in my world, `s.el` is already used by the majority of new packages even
> tho it's neither part of Emacs nor of GNU ELPA.  Not including it in GNU
> ELPA just increases the difficulty of accepting other packages in GNU ELPA.

Richard, Eli: please decide wether you want s.el into ELPA or not.
This is a bit embarassing for me to have done the work of getting
magnars to agree to put it there just to be refused at the last
minute, especially since in the past days I receive nothing but
requests to at least get s.el into ELPA, if the namespace way lead to
nowhere.

I'm not sure why there's this sudden turnaround on this issue, maybe
I'm missing something.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Richard, Eli: please decide wether you want s.el into ELPA or not.

In its current form, I think it would be a grave mistake to include
s.el in ELPA.  Its purpose is to make Emacs Lisp mimic object-oriented
languages which are alien to Emacs Lisp.

See my message to Stefan for a change that would make s.sl ok to add.

  > I'm not sure why there's this sudden turnaround on this issue, maybe
  > I'm missing something.

I don't think there was a turnaround.  We never decided to include
s.el in GNU ELPA.

Before yesterday, we were talking about a different question: whether
to adopt the s.el functions and their names in Emacs Lisp.  When I saw
concretely what those actually were, I said no.

Then yesterday I saw a message proposing to include s.el in ELPA, and
_presuming_ that that was ok.  I responded no, saying that it would
send Emacs Lisp down the same wrong path.  We should not have code in
Emacs that uses string-prepend instead of concat.  We should fix that
code to use concat.

  > This is a bit embarassing for me to have done the work of getting
  > magnars to agree to put it there just to be refused at the last
  > minute,

I am sorry for your disappointment, because I feel for your eagerness
to make a change you consider an improvement.  But we have to make the
decision that is right.

There is no need for you to feel embarrassed.  The people you talked
with will understand that there is no reason to blame or criticize you
for this.

Meanwhile, maybe we could include it with some changes, as clostring.el.
Please see my message to Stefan.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> See my message to Stefan for a change that would make s.sl ok to add.

There's no change under discussion: either it's in GNU ELPA or it's not
(in which case it will keep living happily ever after in MELPA).

>   > I'm not sure why there's this sudden turnaround on this issue, maybe
>   > I'm missing something.
> I don't think there was a turnaround.  We never decided to include
> s.el in GNU ELPA.

Yes, I did.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

> >   > I'm not sure why there's this sudden turnaround on this issue, maybe
> >   > I'm missing something.
> > I don't think there was a turnaround.  We never decided to include
> > s.el in GNU ELPA.
>
> Yes, I did.

I was sure that at least 3 people asked for it but now I can only find
you. I think I misremembered the following message from Richard to be
about `s.el`:

> Would someone with good social skills like to ask the developers of
> dash to move their development into GNU ELPA, or ask why they have not
> done so?  I am sure they can see the advantage of having their latest
> version available through Emacs.

There's something I don't quite understand tho, please explain it to
me: from my uneducated eye dash.el is very similar to s.el. If I
understood correctly having dash.el active in ELPA is something you'd
like but having s.el in ELPA would be "bad". I just don't understand,
almost all of the criticism that you did to s.el I can do to dash.el
as well ("useless" functions, clojure-like, already exists in Emacs
core but named differently, etc).

Philippe

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

>> Would someone with good social skills like to ask the developers of
>> dash to move their development into GNU ELPA, or ask why they have not
>> done so?  I am sure they can see the advantage of having their latest
>> version available through Emacs.
> There's something I don't quite understand tho, please explain it to
> me: from my uneducated eye dash.el is very similar to s.el.

My guess: he doesn't know dash.el (just like he doesn't know about most
of elpa.git, and of course even more so about the rest of the Emacs
world outside of emacs-devel and emacs.git: he doesn't have the time to
learn about those things).

With a bit of luck, he'll now ask for dash.el to be removed from GNU ELPA.


        Stefan "depressed"

Re: Imports / inclusion of s.el into Emacs

[Alfred M. Szmidt]

Please, can everyone start behaving in kind and simply accept that
people have different frames of reference and stop with the petty
messages back and forth?

They have nothing to do with Emacs, Emacs development, or remotely try
to encourage to a productive discourse.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 07.05.2020 17:18, Alfred M. Szmidt wrote:
> Please, can everyone start behaving in kind and simply accept that
> people have different frames of reference and stop with the petty
> messages back and forth?

Did you just suggest everybody shut up?

> They have nothing to do with Emacs, Emacs development, or remotely try
> to encourage to a productive discourse.

You might want to pay attention to who you are replying to.

Re: Imports / inclusion of s.el into Emacs

[Alfred M. Szmidt]

On this list, and other GNU mailing lists we try to follow the GNU
Kind Communication Guidelines, no matter whom you are.  

  https://www.gnu.org/philosophy/kind-communication.html

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 07.05.2020 22:47, Alfred M. Szmidt wrote:
> On this list, and other GNU mailing lists we try to follow the GNU
> Kind Communication Guidelines, no matter whom you are.
> 
>    https://www.gnu.org/philosophy/kind-communication.html

I'm aware.

Entering a sensitive discussion and saying "you all behave like 
children, stop it" is not kind.

Especially when one of the issues is a power imbalance, and "stopping 
it" heavily favors one particular outcome.

Re: Imports / inclusion of s.el into Emacs

[Alfred M. Szmidt]

There is the old joke that Emacs includes everything including the
kitchen sink, that doesn't mean that every kitchen sink out there has
to be included.  Or that the kitchen sink has to be changed.

Emacs is a kitchen sink used by many, having every possible variant of
a faucet or sink variation would be unfeasible, some will always
prefer it to be slightly, or vastly different.  On some you turn the
faucet counter clock wise, on others clock wise.  Some are square or
round, deep or shallow.  Hot water on the left or right?

Someone has to draw the line as to how many different faucets, pipe
sizes, etc to support to have some sort of cohesive sink.  That will
always cause some disagreements, since we all have a different vision
of what our perfect sink is.

The really wonderful thing about Emacs is that it comes with a
documented -- slightly strange pipe sizes or threadings on the odd
ocassion -- that allow you to change it to you ideal sink.  Given some
patience, all of these things can be changed if they make sense or at
least make it possible to change them in the persuit of a kitchen sink
that can be made perfect for everyone.

Nobody is stopping the existance of modes or libraries that can be
used with Emacs that change things to do all kind of strange and
wonderful things.  It just happens that not everything is suitable, at
the discretion of the maintainer(s), to be part of Emacs proper, and
there is nothing wrong with that, and nothing to get upset about.

It is after all they who will need to see that pipes stop leaking
water, see that the faucet doesn't drip, warm water isn't to hot or
cold, etc when the day ends and you want to wash your dishes.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > My guess: he doesn't know dash.el 

Of course I don't!  How would I?  With so many responsibilities, I
can't learn everything that it would be useful for me to know.  I
can't try everything in GNU ELPA in my copious spare time.

The only way I would find out about dash.el is if you tell me.
So why didn't you tell me?

It is not too late.  Would you please tell me what I need to know
about it?  It sounds like you know something problematic about it, or
that you think might be problematic.  What is that?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> It is not too late.  Would you please tell me what I need to know
> about it?  It sounds like you know something problematic about it, or
> that you think might be problematic.  What is that?

I made the first efforts to get it integrated into GNU ELPA (and Phil
did the heavy lifting), so no I definitely don't think it's problematic.


        Stefan

ELPA policy (was: Imports / inclusion of s.el into Emacs)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: philippe.vaucher@gmail.com,  stefan@marxist.se,  emacs-devel@gnu.org,
>   joaotavora@gmail.com,  pcr910303@icloud.com,  dgutov@yandex.ru,
>   eliz@gnu.org,  drew.adams@oracle.com
> Date: Thu, 07 May 2020 23:38:23 -0400
> 
> I made the first efforts to get it integrated into GNU ELPA (and Phil
> did the heavy lifting), so no I definitely don't think it's problematic.

FWIW, I took a cursory look at dash.el in ELPA, and was surprised to
see that its doc strings are not according to our coding conventions,
and functions/macros that clearly aren't internal don't follow the
naming conventions.

So maybe it's time we defined the minimum requirements for packages to
be included in ELPA?

Re: ELPA policy

[Stefan Monnier]

> FWIW, I took a cursory look at dash.el in ELPA, and was surprised to
> see that its doc strings are not according to our coding conventions,
> and functions/macros that clearly aren't internal don't follow the
> naming conventions.

Then I suggest you file a bug report with the author.

> So maybe it's time we defined the minimum requirements for packages to
> be included in ELPA?

What good would it do?


        Stefan

Re: ELPA policy

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Fri, 08 May 2020 10:57:58 -0400
> 
> > FWIW, I took a cursory look at dash.el in ELPA, and was surprised to
> > see that its doc strings are not according to our coding conventions,
> > and functions/macros that clearly aren't internal don't follow the
> > naming conventions.
> 
> Then I suggest you file a bug report with the author.

Aren't ELPA bugs reported via our bug tracker?

> > So maybe it's time we defined the minimum requirements for packages to
> > be included in ELPA?
> 
> What good would it do?

<Shrug> It will get us on the same page when we are reviewing packages
and patches for ELPA.

Re: ELPA policy

[Stefan Monnier]

>> > FWIW, I took a cursory look at dash.el in ELPA, and was surprised to
>> > see that its doc strings are not according to our coding conventions,
>> > and functions/macros that clearly aren't internal don't follow the
>> > naming conventions.
>> Then I suggest you file a bug report with the author.
> Aren't ELPA bugs reported via our bug tracker?

We tell users that they can submit bug reports via our bug tracker.
And maintainers are free to use other bug trackers.
Ideally they should keep an eye on our tracker, or else someone needs to
forward our bugs to their tracker when such things happen.

IOW, as a user of dash.el you can submit your bug report via `M-x
report-emacs-bug` (and if needed, hopefully someone will forward it to
where the maintainer will see it).

>> > So maybe it's time we defined the minimum requirements for packages to
>> > be included in ELPA?
>> What good would it do?
> <Shrug> It will get us on the same page when we are reviewing packages
> and patches for ELPA.

That's not a very high concern for me, compared to the concern of keeping
GNU ELPA relevant and vaguely up-to-date.


        Stefan

Re: ELPA policy

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Fri, 08 May 2020 19:16:03 -0400
> 
> >> > So maybe it's time we defined the minimum requirements for packages to
> >> > be included in ELPA?
> >> What good would it do?
> > <Shrug> It will get us on the same page when we are reviewing packages
> > and patches for ELPA.
> 
> That's not a very high concern for me, compared to the concern of keeping
> GNU ELPA relevant and vaguely up-to-date.

Excuse me, but what do you mean by "for me"?  Isn't GNU ELPA an
important part of the Emacs project?  I'd expect something like "for
us" or "for Emacs", in which case not just you personally have a say.

And why do I have to submit bug reports against an ELPA package for
violation of our coding conventions?  I'd expect the maintainer of the
package to be asked to fix those as a precondition for accepting the
package in GNU ELPA, or at least as a long-term plan to which the
maintainer agreed in advance (in which case no bug report would have
been necessary).

What am I missing here?

Re: ELPA policy

[David Engster]

> And why do I have to submit bug reports against an ELPA package for
> violation of our coding conventions?  I'd expect the maintainer of the
> package to be asked to fix those as a precondition for accepting the
> package in GNU ELPA, or at least as a long-term plan to which the
> maintainer agreed in advance (in which case no bug report would have
> been necessary).
>
> What am I missing here?

The README for GNU ELPA states:

We do not impose a particular coding style on GNU ELPA packages, but of
course we recommend the coding style used in Emacs's own source code.
Furthermore we recommend the following:
- Use `cl-lib` rather than `cl` if it all possible.
- Use lexical-binding if it all possible.
- Try and fix the warnings emitted when compiling the package with a
  recent Emacs.

From: http://git.savannah.gnu.org/cgit/emacs/elpa.git/plain/README

So from my understanding: Following Emacs coding guidelines is a
recommendation, but not a precondition for getting packages into
GNU ELPA.

If we start bundling certain ELPA packages with Emacs proper, then of
course these special "core packages" would need to adhere to the Emacs
coding style. I don't see any difficulty in making this distinction
between core packages and the rest. And I also don't see any problem to
put s.el in ELPA and say: note that using this package is against the
Emacs coding style, so as long as you depend on this packages, it cannot
become a "core package" in the future (same for dash.el).

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 09:35:49 +0200
> 
> From: http://git.savannah.gnu.org/cgit/emacs/elpa.git/plain/README
> 
> So from my understanding: Following Emacs coding guidelines is a
> recommendation, but not a precondition for getting packages into
> GNU ELPA.
> 
> If we start bundling certain ELPA packages with Emacs proper, then of
> course these special "core packages" would need to adhere to the Emacs
> coding style. I don't see any difficulty in making this distinction
> between core packages and the rest. And I also don't see any problem to
> put s.el in ELPA and say: note that using this package is against the
> Emacs coding style, so as long as you depend on this packages, it cannot
> become a "core package" in the future (same for dash.el).

If we mark some ELPA packages up front as unsuitable for inclusion in
core, then I guess this could be acceptable.  It does mean that such a
decision would make it hard to change our minds later, though.  But at
least we should have an indication in each package whether it is or
isn't exempt from our conventions, something we don't have now.

Moreover, our conventions not being an absolute requirement, I think
_some_ requirements should still be non-negotiable, because we cannot
just accept anything.  Otherwise any request for a change could be met
with an argument like "but I'm not under any obligation to comply" or
somesuch.  I don't see any such requirements described in README, so
my question still stands, I think.

Re: ELPA policy

[David Engster]

>> From: David Engster <deng@randomsample.de>
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
>> Date: Sat, 09 May 2020 09:35:49 +0200
>
>> 
>> From: http://git.savannah.gnu.org/cgit/emacs/elpa.git/plain/README
>> 
>> So from my understanding: Following Emacs coding guidelines is a
>> recommendation, but not a precondition for getting packages into
>> GNU ELPA.
>> 
>> If we start bundling certain ELPA packages with Emacs proper, then of
>> course these special "core packages" would need to adhere to the Emacs
>> coding style. I don't see any difficulty in making this distinction
>> between core packages and the rest. And I also don't see any problem to
>> put s.el in ELPA and say: note that using this package is against the
>> Emacs coding style, so as long as you depend on this packages, it cannot
>> become a "core package" in the future (same for dash.el).
>
> If we mark some ELPA packages up front as unsuitable for inclusion in
> core, then I guess this could be acceptable.  It does mean that such a
> decision would make it hard to change our minds later, though.  But at
> least we should have an indication in each package whether it is or
> isn't exempt from our conventions, something we don't have now.

If you look here

http://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/externals-list

you'll see that there are already packages marked as ":core" which all
currently refer to a path in Emacs proper. This way, they could be
overriden through GNU ELPA, and they of course need to adhere to the
Emacs coding style.

You are right that it will be difficult if we decide a package should
become "core" and currently does not adhere to the Emacs coding style,
but frankly, we should be more worried that this ship has already sailed
far away. Many packages which are absolutely essential for a modern
programmer's editor are already only available through MELPA. If we want
to make GNU ELPA more relevant, we need to lower the hurdle for
inclusion. I also fully agree with Stefan that we should make it
possible for packages that have non-FSF copyright to be included. Of
course these packages could never become "core", but having them
installable throught GNU ELPA would be the next best thing.

> Moreover, our conventions not being an absolute requirement, I think
> _some_ requirements should still be non-negotiable, because we cannot
> just accept anything.  Otherwise any request for a change could be met
> with an argument like "but I'm not under any obligation to comply" or
> somesuch.  I don't see any such requirements described in README, so
> my question still stands, I think.

Yes. For instance, it is clear that a package in GNU ELPA must not
recommend or even depend on non-free software.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 10:16:38 +0200
> 
> You are right that it will be difficult if we decide a package should
> become "core" and currently does not adhere to the Emacs coding style,
> but frankly, we should be more worried that this ship has already sailed
> far away. Many packages which are absolutely essential for a modern
> programmer's editor are already only available through MELPA.

Why is that a problem?  We can never do anything to prevent people
from concocting whatever packages they want and making them available
for others.  Nor do I think we should: this is, after all, Free
Software: people should be free to choose whatever software they like
that does the job for them.  We can never control that, and we
shouldn't try.

> I also fully agree with Stefan that we should make it possible for
> packages that have non-FSF copyright to be included. Of course these
> packages could never become "core", but having them installable
> throught GNU ELPA would be the next best thing.

If we are going to drop requirements, then what will distinguish ELPA
from MELPA?  And what's the problem with having non-core packages
available through MELPA, anyway? why do we need to have them in ELPA?

This all goes back to my confusion about the relation between ELPA and
Emacs, something I thought I understood, but now I conclude that I
don't.

Re: ELPA policy

[David Engster]

>> From: David Engster <deng@randomsample.de>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Sat, 09 May 2020 10:16:38 +0200
>> 
>> You are right that it will be difficult if we decide a package should
>> become "core" and currently does not adhere to the Emacs coding style,
>> but frankly, we should be more worried that this ship has already sailed
>> far away. Many packages which are absolutely essential for a modern
>> programmer's editor are already only available through MELPA.
>
> Why is that a problem?  We can never do anything to prevent people
> from concocting whatever packages they want and making them available
> for others.  Nor do I think we should: this is, after all, Free
> Software: people should be free to choose whatever software they like
> that does the job for them.  We can never control that, and we
> shouldn't try.
>
>> I also fully agree with Stefan that we should make it possible for
>> packages that have non-FSF copyright to be included. Of course these
>> packages could never become "core", but having them installable
>> throught GNU ELPA would be the next best thing.
>
> If we are going to drop requirements, then what will distinguish ELPA
> from MELPA?  And what's the problem with having non-core packages
> available through MELPA, anyway? why do we need to have them in ELPA?

In principal, I agree with you. The problem is mainly Richard's stance
on this issue, which says that we must not recommend packages which are
not in Emacs or GNU ELPA, but that we should rather re-implement them. I
think that's a terrible waste.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 10:43:55 +0200
> 
> > If we are going to drop requirements, then what will distinguish ELPA
> > from MELPA?  And what's the problem with having non-core packages
> > available through MELPA, anyway? why do we need to have them in ELPA?
> 
> In principal, I agree with you. The problem is mainly Richard's stance
> on this issue, which says that we must not recommend packages which are
> not in Emacs or GNU ELPA, but that we should rather re-implement them. I
> think that's a terrible waste.

Is this only about "recommending" or not "recommending" a package?  Is
this why we created GNU ELPA and invest non-trivial amount of effort
in maintaining and developing it?  I very much hope there's more to
it than just that.

I could understand if you'd say "use" instead of "recommend",
i.e. have code in Emacs, which, if a package is installed, would use
it.  That'd actually have the package's name in our sources, and would
constitute some kind of "endorsement".  But as long as we don't use
any of those packages, why should we care what other people like or
don't like?

Re: ELPA policy

[David Engster]

>> From: David Engster <deng@randomsample.de>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Sat, 09 May 2020 10:43:55 +0200
>
>> 
>> > If we are going to drop requirements, then what will distinguish ELPA
>> > from MELPA?  And what's the problem with having non-core packages
>> > available through MELPA, anyway? why do we need to have them in ELPA?
>> 
>> In principal, I agree with you. The problem is mainly Richard's stance
>> on this issue, which says that we must not recommend packages which are
>> not in Emacs or GNU ELPA, but that we should rather re-implement them. I
>> think that's a terrible waste.
>
> Is this only about "recommending" or not "recommending" a package?  Is
> this why we created GNU ELPA and invest non-trivial amount of effort
> in maintaining and developing it?  I very much hope there's more to
> it than just that.

We created GNU ELPA because we wanted a package system in Emacs. It was
simply the first, then things evolved to the state we have now, where
MELPA is the dominant package repository.

> I could understand if you'd say "use" instead of "recommend",
> i.e. have code in Emacs, which, if a package is installed, would use
> it.  That'd actually have the package's name in our sources, and would
> constitute some kind of "endorsement".  But as long as we don't use
> any of those packages, why should we care what other people like or
> don't like?

Sorry, but you lost me there. All I'm saying is that there's a whole lot
of terrific packages out there but which we must not recommend to users,
although they are free software and often vastly superior to the things
that are built into Emacs. For instance, there's a discussion going on
about making a video showing Emacs' capabilities, but I assume we'd not
be allowed to show Magit. That's a huge loss.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 12:13:01 +0200
> 
> > I could understand if you'd say "use" instead of "recommend",
> > i.e. have code in Emacs, which, if a package is installed, would use
> > it.  That'd actually have the package's name in our sources, and would
> > constitute some kind of "endorsement".  But as long as we don't use
> > any of those packages, why should we care what other people like or
> > don't like?
> 
> Sorry, but you lost me there. All I'm saying is that there's a whole lot
> of terrific packages out there but which we must not recommend to users,
> although they are free software and often vastly superior to the things
> that are built into Emacs. For instance, there's a discussion going on
> about making a video showing Emacs' capabilities, but I assume we'd not
> be allowed to show Magit. That's a huge loss.

So package maintainers are supposed to want to be on ELPA so that they
could appear in a video, or in someone's message on a GNU mailing
list?  Really?

Once again, I wasn't asking whether it was okay or not to show off
Magit in a video or promote it on this list, I was asking why do we
need ELPA when MELPA is out there and has many more packages (and
always will)?

Re: ELPA policy

[David Engster]

>> From: David Engster <deng@randomsample.de>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Sat, 09 May 2020 12:13:01 +0200
>
>> 
>> > I could understand if you'd say "use" instead of "recommend",
>> > i.e. have code in Emacs, which, if a package is installed, would use
>> > it.  That'd actually have the package's name in our sources, and would
>> > constitute some kind of "endorsement".  But as long as we don't use
>> > any of those packages, why should we care what other people like or
>> > don't like?
>> 
>> Sorry, but you lost me there. All I'm saying is that there's a whole lot
>> of terrific packages out there but which we must not recommend to users,
>> although they are free software and often vastly superior to the things
>> that are built into Emacs. For instance, there's a discussion going on
>> about making a video showing Emacs' capabilities, but I assume we'd not
>> be allowed to show Magit. That's a huge loss.
>
> So package maintainers are supposed to want to be on ELPA so that they
> could appear in a video, or in someone's message on a GNU mailing
> list?  Really?

No, package maintainers usually don't care. I think this should be in
*our* interest.

> Once again, I wasn't asking whether it was okay or not to show off
> Magit in a video or promote it on this list, I was asking why do we
> need ELPA when MELPA is out there and has many more packages (and
> always will)?

Indeed, one possibility would be to simply close GNU ELPA for everything
but core or maybe-in-the-future-core packages. Is that what you're
proposing?

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 12:29:22 +0200
> 
> > So package maintainers are supposed to want to be on ELPA so that they
> > could appear in a video, or in someone's message on a GNU mailing
> > list?  Really?
> 
> No, package maintainers usually don't care. I think this should be in
> *our* interest.

If it's in our interest, then why do we accept packages in exactly the
same form as they are on MELPA?  What do we gain from this?

> > Once again, I wasn't asking whether it was okay or not to show off
> > Magit in a video or promote it on this list, I was asking why do we
> > need ELPA when MELPA is out there and has many more packages (and
> > always will)?
> 
> Indeed, one possibility would be to simply close GNU ELPA for everything
> but core or maybe-in-the-future-core packages. Is that what you're
> proposing?

I don't have any proposals yet, because I still don't have a clear
idea of what is (or should be) the relation between Emacs and ELPA,
nor even what are the goals of adding packages to ELPA which clearly
couldn't be added to core even if we wanted.  And with Stefan's
proposal to drop the copyright assignment requirement, bringing them
into core will even harder.  But OTOH Stefan says that requesting the
assignment is harmful to the Emacs project.  So I'm utterly confused
regarding our goals in this regard.

Re: ELPA policy

[David Engster]

>> Indeed, one possibility would be to simply close GNU ELPA for everything
>> but core or maybe-in-the-future-core packages. Is that what you're
>> proposing?
>
> I don't have any proposals yet, because I still don't have a clear
> idea of what is (or should be) the relation between Emacs and ELPA,
> nor even what are the goals of adding packages to ELPA which clearly
> couldn't be added to core even if we wanted.  And with Stefan's
> proposal to drop the copyright assignment requirement, bringing them
> into core will even harder.  But OTOH Stefan says that requesting the
> assignment is harmful to the Emacs project.  So I'm utterly confused
> regarding our goals in this regard.

Even if we drop the FSF copyright requirement, we still would have
requirements that MELPA does not have, like not supporting/promoting
non-free software, GPLv3-or-later license (MELPA only requires
GPL-compatible), and probably (and unfortunately, IMHO) also the
requirement to not depend on free software that is seen critical by the
FSF, like LLVM/clang.

My hope is that if we could drop the FSF copyright requirement for
non-core packages, we at least could try to get popular and useful
packages into GNU ELPA, we could make them prominently visible and easy
to install without the user having to put anything into her dotemacs,
and we could show the power of Emacs that is visible in these packages
on our main web site, in videos, etc.

-David

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Indeed, one possibility would be to simply close GNU ELPA for everything
  > but core or maybe-in-the-future-core packages. Is that what you're
  > proposing?

I think that is unnecessarily strict.  It presumes that when adding a
package to GNU ELPA we always know, already, whether we will ever want
to put it in the core.  I expect there are lots of cases  where
that decision will depend on circumstances.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: ELPA policy

[Alfred M. Szmidt]

I think one of the reasons is simply that the GNU project doesn't want
to be in a situation where a third party (i.e., MELPA) is recommending
non-free software, or where the license is unclear, etc.

So ELPA was setup so that this cannot happen, since that is part
(AFAIK) of Emacs.

Re: ELPA policy

[Dmitry Gutov]

On 09.05.2020 09:22, Eli Zaretskii wrote:
> And why do I have to submit bug reports against an ELPA package for
> violation of our coding conventions?

Because it's the only way how things can work well. Once we have a bug 
tracker that most maintainers like to use, that could change.

> I'd expect the maintainer of the
> package to be asked to fix those as a precondition for accepting the
> package in GNU ELPA, or at least as a long-term plan to which the
> maintainer agreed in advance (in which case no bug report would have
> been necessary).
> 
> What am I missing here?

You're missing that the maintainers don't have any contractual 
obligations. The more conditions we add, and the less effort we want to 
expend ourselves [forwarding bug reports], the higher the odds are that 
we end up with fewer contributions/stale code/untended bug reports.

Re: ELPA policy

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 9 May 2020 18:06:05 +0300
> 
> On 09.05.2020 09:22, Eli Zaretskii wrote:
> > And why do I have to submit bug reports against an ELPA package for
> > violation of our coding conventions?
> 
> Because it's the only way how things can work well. Once we have a bug 
> tracker that most maintainers like to use, that could change.

There's no chance in the world someone will be serious about such bug
reports, given that ELPA's README basically tells that these
requirements are "recommendations".  Why would someone invest any
effort if they can avoid doing that?

> You're missing that the maintainers don't have any contractual 
> obligations. The more conditions we add, and the less effort we want to 
> expend ourselves [forwarding bug reports], the higher the odds are that 
> we end up with fewer contributions/stale code/untended bug reports.

And what's the problem with that?  These packages are available from
elsewhere anyway.

Our coding standards aren't arbitrary.  Either we think they are solid
and should apply to every code that could end up in Emacs, or we
should stop requiring adherence to them.  Anything else makes no
sense.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Our coding standards aren't arbitrary.  Either we think they are solid
  > and should apply to every code that could end up in Emacs, or we
  > should stop requiring adherence to them.  Anything else makes no
  > sense.

In principle, they should apply to ELPA packages.  But it is ok, I think,
if we don't rush to bring all of them completely up to snuff.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: ELPA policy

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Date: Mon, 11 May 2020 23:16:20 -0400
> 
>   > Our coding standards aren't arbitrary.  Either we think they are solid
>   > and should apply to every code that could end up in Emacs, or we
>   > should stop requiring adherence to them.  Anything else makes no
>   > sense.
> 
> In principle, they should apply to ELPA packages.  But it is ok, I think,
> if we don't rush to bring all of them completely up to snuff.

I could agree to that, but I wouldn't agree to drop all of the
requirements.  And even if we loosen some of them, it's already
problematic: the contributors of code for the core, which are
generally required to adhere to all of the requirements, could
rightfully feel they are treated unfairly, even though developing the
core benefits Emacs more than an unbundled package.

Re: ELPA policy

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: philippe.vaucher@gmail.com,  stefan@marxist.se,  emacs-devel@gnu.org,
>>   joaotavora@gmail.com,  pcr910303@icloud.com,  dgutov@yandex.ru,
>>   eliz@gnu.org,  drew.adams@oracle.com
>> Date: Thu, 07 May 2020 23:38:23 -0400
>> 
>> I made the first efforts to get it integrated into GNU ELPA (and Phil
>> did the heavy lifting), so no I definitely don't think it's problematic.
>
> FWIW, I took a cursory look at dash.el in ELPA, and was surprised to
> see that its doc strings are not according to our coding conventions,
> and functions/macros that clearly aren't internal don't follow the
> naming conventions.
>
> So maybe it's time we defined the minimum requirements for packages to
> be included in ELPA?


dash.el has an extensive README with a very large number of
examples. The README is produced by combining the examples from a lisp
file which also generates a test file. I think Magnar values doc tests
more than he values Emacs doc string conventions.

Phil

Re: ELPA policy (was: Imports / inclusion of s.el into Emacs)

[Philippe Vaucher]

> So maybe it's time we defined the minimum requirements for packages to
> be included in ELPA?

Why not simply use MELPA's one?

https://github.com/melpa/melpa/blob/master/CONTRIBUTING.org

Re: ELPA policy (was: Imports / inclusion of s.el into Emacs)

[Philippe Vaucher]

> > So maybe it's time we defined the minimum requirements for packages to
> > be included in ELPA?
>
> Why not simply use MELPA's one?
>
> https://github.com/melpa/melpa/blob/master/CONTRIBUTING.org

Before someone nitpicks, of course I mean select the parts that'd apply to us.

Re: ELPA policy (was: Imports / inclusion of s.el into Emacs)

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 9 May 2020 09:21:27 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Emacs developers <emacs-devel@gnu.org>
> 
> > So maybe it's time we defined the minimum requirements for packages to
> > be included in ELPA?
> 
> Why not simply use MELPA's one?
> 
> https://github.com/melpa/melpa/blob/master/CONTRIBUTING.org

I'd prefer to start from our CONTRIBUTE, since (AFAIU) GNU ELPA is
supposed to be part of the Emacs project.  Commonality of requirements
is important IMO to make it easier to contribute a package: we could
then decide relatively easily whether to add the package to core or to
ELPA.

Re: ELPA policy (was: Imports / inclusion of s.el into Emacs)

[Philippe Vaucher]

> > > So maybe it's time we defined the minimum requirements for packages to
> > > be included in ELPA?
> >
> > Why not simply use MELPA's one?
> >
> > https://github.com/melpa/melpa/blob/master/CONTRIBUTING.org
>
> I'd prefer to start from our CONTRIBUTE, since (AFAIU) GNU ELPA is
> supposed to be part of the Emacs project.  Commonality of requirements
> is important IMO to make it easier to contribute a package: we could
> then decide relatively easily whether to add the package to core or to
> ELPA.

Sure, but from a quick look at it CONTRIBUTE lacks a lot of what
https://github.com/melpa/melpa/blob/master/CONTRIBUTING.org#making-your-package-ready-for-inclusion
proposes and would make sense for inclusion in ELPA. I'll copy the
relevant parts of that section here:

- Coding style and conventions: The Emacs Lisp files should follow the
Emacs Lisp conventions and the Emacs Lisp Style Guide.
- Package metadata: Package descriptions should adhere to the
package.el format as specified by (info "(elisp) Packaging")
documentation. More information on this format is provided by the
marmalade package manual.
- Use quality-checking tools: Use flycheck, package-lint and
flycheck-package to help you identify common errors in your package
metadata. Use checkdoc to make sure that your package follows the
conventions for documentation strings.
- Avoid long functions: The longer a function the harder it is for a
MELPA maintainer to understand what is happening and to give feedback.
It is also much harder to point to a specific portion of the code we
believe could be improved. Please spend time decomposing your long
functions into smaller, well-named and documented, ones.

I'm sure that if we read the MELPA recommendations more in depth we'd
find even more inspiration as to what ELPA's policy could be.

Re: ELPA policy (was: Imports / inclusion of s.el into Emacs)

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 9 May 2020 12:42:01 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>  Emacs developers <emacs-devel@gnu.org>
> 
> > I'd prefer to start from our CONTRIBUTE, since (AFAIU) GNU ELPA is
> > supposed to be part of the Emacs project.  Commonality of requirements
> > is important IMO to make it easier to contribute a package: we could
> > then decide relatively easily whether to add the package to core or to
> > ELPA.
> 
> Sure, but from a quick look at it CONTRIBUTE lacks a lot of what
> https://github.com/melpa/melpa/blob/master/CONTRIBUTING.org#making-your-package-ready-for-inclusion
> proposes and would make sense for inclusion in ELPA. I'll copy the
> relevant parts of that section here:

I think you are missing my point.  I didn't mean the details on the
level you seem to focus, I meant the deviations from our requirements
for code accepted into Emacs.

I have comments on your specific points (most of them are in ELPA's
README), but I see no reason to start a tangent, because IMO much more
important issues are at stake.

Re: ELPA policy (was: Imports / inclusion of s.el into Emacs)

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 513 bytes --]

>
> > proposes and would make sense for inclusion in ELPA. I'll copy the
> > relevant parts of that section here:
>
> I think you are missing my point.  I didn't mean the details on the
> level you seem to focus, I meant the deviations from our requirements
> for code accepted into Emacs.
>
> I have comments on your specific points (most of them are in ELPA's
> README), but I see no reason to start a tangent, because IMO much more
> important issues are at stake.
>

Ah, yes I was. Thanks for clarifying! :-)

[-- Attachment #2: Type: text/html, Size: 1007 bytes --]

Re: ELPA policy (was: Imports / inclusion of s.el into Emacs)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'm sure that if we read the MELPA recommendations more in depth we'd
  > find even more inspiration as to what ELPA's policy could be.

The MELPA recommendations you cited are all technical.  I agree with
the first three.  I think the fourth one, rejecting long functions in
a blanket way, is too strict, but I agree with that it is good to
encourage functions to be shorter.

GNU and GNU Emacs have many other coding conventions.  I suppose the
people who run MELPA have some too.  Maybe they are the same ones, or
maybe not.  We should apply our conventions to the code we adopt.

However, the most important area of possible difference is in the
_nontechnical_ conventions.  I think Eli posted a message listing some
of them, regarding licensing, and what other programs to recommend.
This is the most important area of all.  This is why we can't
recommend MELPA.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Alfred M. Szmidt]

     > My guess: he doesn't know dash.el 

Neither do I, I looked it up.  

The manual for dash.el doesn't explain what dash.el is about, it
advertises it self as a 'modern list library for Emacs'.

From the looks it contains a big mix of various functions that operate
on lists, anaphoric variants of Emacs lisp ones, some functions
borrowed from Haskell, some functions to work on tree, and a thin
layer for various Emacs Lisp functions to follow the dash.el libraries
internal naming converntion of prefixing everything with a dash, and
the Scheme naming style for predicate.  At the end, I still do not
know what dash.el does, or what to use it for.

s.el seems to be much simpler, and a simple layer over strings, often
aliasing standard Emacs Lisp functions to follow the Scheme convention
of using a question mark to indicate a predicate.  Some of which look
useful in function, but have a non-standard naming convention.

dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Joost Kremers]

On Fri, May 08 2020, Alfred M. Szmidt wrote:
>      > My guess: he doesn't know dash.el 
>
> Neither do I, I looked it up.  
>
> The manual for dash.el doesn't explain what dash.el is about, it
> advertises it self as a 'modern list library for Emacs'.
>
> From the looks it contains a big mix of various functions that 
> operate
> on lists, anaphoric variants of Emacs lisp ones, some functions
> borrowed from Haskell, some functions to work on tree, and a 
> thin
> layer for various Emacs Lisp functions to follow the dash.el 
> libraries
> internal naming converntion of prefixing everything with a dash, 
> and
> the Scheme naming style for predicate.  At the end, I still do 
> not
> know what dash.el does, 

But you just described what dash does. ;-) It is just a collection 
of list-handling functions such as they exist in modern functional 
programming languages. If you're used to thinking in this paradigm 
and then come (back) to Emacs Lisp, it feels like a hopelessly 
clunky language. `dash.el` was written to remedy this.

I assume the short prefix was chosen because list handling 
functions are part of the core language in Clojure (and I assume 
other functional programming languages) and using a dash was the 
best way to not conflict with existing names and still create the 
feeling that you're not actually using a library. (I assume 
"easier to type" won't have much to do with it, we have 
completion, after all.)

The convention of using `?` for predicates is probably also 
borrowed more from Clojure than from Scheme (though Clojure 
obviously borrowed it from Scheme).

> or what to use it for.

Well, you use it if you want to program in a Clojure-like style.


-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   But you just described what dash does. ;-) It is just a collection 
   of list-handling functions such as they exist in modern functional 
   programming languages. If you're used to thinking in this paradigm 
   and then come (back) to Emacs Lisp, it feels like a hopelessly 
   clunky language. `dash.el` was written to remedy this.

Thank you, I was and still am unsure what the purpose of dash.el is!
It is simply not somehing I would ever find myself using.

While I'm very much used to programming in functional langugaes, the
way that Clojure and Haskell do it is very much alien to me.  And
quite often, confusing to follow and reason about (IMHO).

While some might prefer that, they do not seem well fitting for Emacs,
how you program in Emacs Lisp and how Emacs Lisp is intended to be
used.

   > or what to use it for.

   Well, you use it if you want to program in a Clojure-like style.

Isn't that the crux then? 

I as a user want to program in an Emacs Lisp style, not Clojure-style.

As a user, I found many of the functions to have alien names, strange
behaviour and very unintuitve to use in the style that is Emacs lisp
(Classical Lisp?).  But some functions seem useful, though with
non-Emacsy names and calling conventions.

Adding functions like `s-reverse' instead of just using `reverse'
seems strange.  Obviously, that isn't all that s.el provides,
s-split-up-to (splits a string N times into substrings based on a
regexp) seem very much useful for example that could be added with a
different name that fits Emacs Lisp, or maybe there is already a way
of doing that.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

ams@gnu.org (Alfred M. Szmidt) writes:

>    But you just described what dash does. ;-) It is just a collection 
>    of list-handling functions such as they exist in modern functional 
>    programming languages. If you're used to thinking in this paradigm 
>    and then come (back) to Emacs Lisp, it feels like a hopelessly 
>    clunky language. `dash.el` was written to remedy this.
>
> Thank you, I was and still am unsure what the purpose of dash.el is!
> It is simply not somehing I would ever find myself using.
>
> While I'm very much used to programming in functional langugaes, the
> way that Clojure and Haskell do it is very much alien to me.  And
> quite often, confusing to follow and reason about (IMHO).
>
> While some might prefer that, they do not seem well fitting for Emacs,
> how you program in Emacs Lisp and how Emacs Lisp is intended to be
> used.



Personally, I find dash very comfortable to use, because I can stop
thinking about loops, and think more about expressions. Of course, the
addition of seq.el a couple of years back has bought that to. pcase.el
has provided some of the other things in dash (as well as more). Indeed,
I wrote my own library, so I could stop thinking about loops when
manipulating buffer contents. An dash.el anaphoric macros are nice also,
and save typing `lambda' a lot.

Finally, dash just adds a lot of things ready rolled. If the history of
programming languages tells us anything at all, it is that languages
tend toward higher and higher levels. Emacs could do with more of this.

Consider:

(while (re-search-forward "foo[ \t]+bar" nil t)
   (replace-match "foobar"))

from the documentation. And, if you want to not nobble the user facing
location of point

(save-excursion
  (while (re-search-forward "foo[ \t]+bar" nil t)
    (replace-match "foobar")))

And, don't forget to beware what you are searching for and what you are
replacing. Because if point does not advance, you'll loop for ever.

I don't want to have to remember all of this.


>
>    > or what to use it for.
>
>    Well, you use it if you want to program in a Clojure-like style.
>
> Isn't that the crux then? 
>
> I as a user want to program in an Emacs Lisp style, not Clojure-style.

Emacs lisp is not a constant, of course. One of the joys of lisp having
very little syntax it is possible to re-invent for different
generations. Many people do find 



> As a user, I found many of the functions to have alien names, strange
> behaviour and very unintuitve to use in the style that is Emacs lisp
> (Classical Lisp?).  But some functions seem useful, though with
> non-Emacsy names and calling conventions.

Yes, this is true. seq.el does a better job there. But, again, it
assumes we cannot re-invent Emacs. The question is whether we
should. 2million downloads for dash.el tells me many users think the
answer is yes.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   Finally, dash just adds a lot of things ready rolled. If the history of
   programming languages tells us anything at all, it is that languages
   tend toward higher and higher levels. Emacs could do with more of this.

I think Emacs already does that; only that the POV is just different.

The way we manipulate buffers doesn't lend it self easily to a
`functional' paradigm -- consider only forward-char!

   Consider:

   (while (re-search-forward "foo[ \t]+bar" nil t)
      (replace-match "foobar"))

   from the documentation. And, if you want to not nobble the user facing
   location of point

   (save-excursion
     (while (re-search-forward "foo[ \t]+bar" nil t)
       (replace-match "foobar")))

   And, don't forget to beware what you are searching for and what you are
   replacing. Because if point does not advance, you'll loop for ever.

   I don't want to have to remember all of this.

I find both forms easy enough, they are easy to understand and easy to
write.  I barley need to understand what each of those functions do to
understand what the code does.  I don't see how dash.el or s.el could
make the above easier to grok (one reads code more than writes it).

What do you suggest as the alternative?

   >    > or what to use it for.
   >
   >    Well, you use it if you want to program in a Clojure-like style.
   >
   > Isn't that the crux then? 
   >
   > I as a user want to program in an Emacs Lisp style, not Clojure-style.

   Emacs lisp is not a constant, of course. One of the joys of lisp having
   very little syntax it is possible to re-invent for different
   generations. Many people do find 

End of sentence missing?

I disagree that Lisp lacks syntax (it has probobly more of it than
other languages) or that one should `re-invent' things.  But that is
more of a philosophical debate than anything.

   > As a user, I found many of the functions to have alien names, strange
   > behaviour and very unintuitve to use in the style that is Emacs lisp
   > (Classical Lisp?).  But some functions seem useful, though with
   > non-Emacsy names and calling conventions.

   Yes, this is true. seq.el does a better job there. But, again, it
   assumes we cannot re-invent Emacs. The question is whether we
   should. 2million downloads for dash.el tells me many users think the
   answer is yes.

I don't know what it means to re-invent Emacs, or what it would entail
so that is making a leap of faith into an abyss of unknowns.  

What I do know is that "millions of downloads" is not good metric of
what is good, e.g. many more people download non-free software than
free software.

We do not know, or so I assume, how many of those supposed millions
activley use dash.el, or if it is simply a dependency that is
automatically pulled in by some other package.

And even so, it doesn't automatically mean that it would be a good
addition to Emacs -- such things shouldn't be decided only based on
popularity.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

ams@gnu.org (Alfred M. Szmidt) writes:

>    Finally, dash just adds a lot of things ready rolled. If the history of
>    programming languages tells us anything at all, it is that languages
>    tend toward higher and higher levels. Emacs could do with more of this.
>
> I think Emacs already does that; only that the POV is just different.
>
> The way we manipulate buffers doesn't lend it self easily to a
> `functional' paradigm -- consider only forward-char!
>
> I find both forms easy enough, they are easy to understand and easy to
> write.  I barley need to understand what each of those functions do to
> understand what the code does.  I don't see how dash.el or s.el could
> make the above easier to grok (one reads code more than writes it).
>
> What do you suggest as the alternative?

Using my own library (m-buffer) this switches all "a" with "x".

(defun switch-a-1 ()
  (interactive)
  (m-buffer-replace-match
   (m-buffer-match (current-buffer) "a")
   "x"))

or just the first two occurences

(defun switch-a-2 ()
  (interactive)
  (m-buffer-replace-match
   (-take 2
          (m-buffer-match (current-buffer) "a"))
   "x"))

or if you don't like too much indentation

(defun switch-a-3 ()
  (interactive)
  (--> (m-buffer-match (current-buffer) "a")
       (-take 2 it)
       (m-buffer-replace-match it "x")))


>
>    >    > or what to use it for.
>    >
>    >    Well, you use it if you want to program in a Clojure-like style.
>    >
>    > Isn't that the crux then? 
>    >
>    > I as a user want to program in an Emacs Lisp style, not Clojure-style.
>
>    Emacs lisp is not a constant, of course. One of the joys of lisp having
>    very little syntax it is possible to re-invent for different
>    generations. Many people do find 
>
> End of sentence missing?


"Many people find that they want to program in different styles", or
something like that!


> I disagree that Lisp lacks syntax (it has probobly more of it than
> other languages)

Lisp has parens and some other stuff. It's hard to add control-flow
structures in most other languages, while it's trivial in lisp.

> or that one should `re-invent' things.  But that is more of a
> philosophical debate than anything.

And the soul of what we are talking about I think.



>    > As a user, I found many of the functions to have alien names, strange
>    > behaviour and very unintuitve to use in the style that is Emacs lisp
>    > (Classical Lisp?).  But some functions seem useful, though with
>    > non-Emacsy names and calling conventions.
>
>    Yes, this is true. seq.el does a better job there. But, again, it
>    assumes we cannot re-invent Emacs. The question is whether we
>    should. 2million downloads for dash.el tells me many users think the
>    answer is yes.
>
> I don't know what it means to re-invent Emacs, or what it would entail
> so that is making a leap of faith into an abyss of unknowns.  
>
> What I do know is that "millions of downloads" is not good metric of
> what is good, e.g. many more people download non-free software than
> free software.

It's not a bad metric either! Yes, there are many people downloading


> We do not know, or so I assume, how many of those supposed millions
> activley use dash.el, or if it is simply a dependency that is
> automatically pulled in by some other package.

Well, it's a programmatic library, so "actively using" is really about
being a dependency. So, I downloaded MELPA till I got to packages
starting with "i" and grepped for "require 'dash", which gets me 160
matches. It's a lot.


> And even so, it doesn't automatically mean that it would be a good
> addition to Emacs -- such things shouldn't be decided only based on
> popularity.

Happy to hear the criteria that you would apply.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Kangas]

Phillip Lord <phillip.lord@russet.org.uk> writes:

>                     So, I downloaded MELPA till I got to packages
> starting with "i" and grepped for "require 'dash", which gets me 160
> matches. It's a lot.

There are 471 packages depending on dash.el according to this list:

    https://melpa.org/#/dash

There are 4585 packages in MELPA.

Best regards,
Stefan Kangas

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   (defun switch-a-3 ()
     (interactive)
     (--> (m-buffer-match (current-buffer) "a")
	  (-take 2 it)
	  (m-buffer-replace-match it "x")))

How does that even the same thing as:

  (save-excursion
    (while (re-search-forward "foo[ \t]+bar" nil t)
      (replace-match "foobar")))

is beyond me.  And I think that applies to the majority of Emacs Lisp
users, -->, m-buffer-match, m-buffer-replace-match say nothing, they
do not even suggest what they do.

   It's not a bad metric either! Yes, there are many people
   downloading

It is a terrible metric, Emacs isn't a popularity contest.  End of
sentence missing again?

   > And even so, it doesn't automatically mean that it would be a good
   > addition to Emacs -- such things shouldn't be decided only based on
   > popularity.

   Happy to hear the criteria that you would apply.

The same critera that is used when anything is added to Emacs.  I
think that has and is working well, people give their input, you try
to convince the maintainer(s) that it makes sense and follows their
idea of what is "Emacs', and it gets added to Emacs if so.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

ams@gnu.org (Alfred M. Szmidt) writes:

>    (defun switch-a-3 ()
>      (interactive)
>      (--> (m-buffer-match (current-buffer) "a")
> 	  (-take 2 it)
> 	  (m-buffer-replace-match it "x")))
>
> How does that even the same thing as:
>
>   (save-excursion
>     (while (re-search-forward "foo[ \t]+bar" nil t)
>       (replace-match "foobar")))
>
> is beyond me.  And I think that applies to the majority of Emacs Lisp
> users, -->, m-buffer-match, m-buffer-replace-match say nothing, they
> do not even suggest what they do.

m-buffer-match returns the matches in the buffer as a
list. m-buffer-replace-match replaces those matches with the
placement. -take takes the first two elements from a list. I agree -->
doesn't tell you much -- it's a threading macro.


>    It's not a bad metric either! Yes, there are many people
>    downloading
>
> It is a terrible metric, Emacs isn't a popularity contest.

So I have heard.

>    > And even so, it doesn't automatically mean that it would be a good
>    > addition to Emacs -- such things shouldn't be decided only based on
>    > popularity.
>
>    Happy to hear the criteria that you would apply.
>
> The same critera that is used when anything is added to Emacs.  I
> think that has and is working well, people give their input, you try
> to convince the maintainer(s) that it makes sense and follows their
> idea of what is "Emacs', and it gets added to Emacs if so.

Quite clearly, dash has and is working well because as 10% of MELPA uses
it. Dash have taken lots of input (indeed, the size of the contributor
list is what has caused problems with ELPA and FSF copyright
policy). Your last criteria doesn't help at all, just begs the question
of what criteria the maintainer(s) use.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But you just described what dash does. ;-) It is just a collection 
  > of list-handling functions such as they exist in modern functional 
  > programming languages. If you're used to thinking in this paradigm 
  > and then come (back) to Emacs Lisp, it feels like a hopelessly 
  > clunky language. `dash.el` was written to remedy this.

That sounds like they could be useful facilities.  Since they are real
features, not mere aliases and trivialities, they would not have the
main flaw of s.el.

However, we should look carefully at the specific interfaces
of the entry points.  Including something into Emacs is a time
for regularizing little details.

Also, I get the impression the names don't fit Emacs's conventions.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[조성빈]

> 2020. 5. 9. 오후 12:56, Richard Stallman <rms@gnu.org> 작성:
> 
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>> But you just described what dash does. ;-) It is just a collection 
>> of list-handling functions such as they exist in modern functional 
>> programming languages. If you're used to thinking in this paradigm 
>> and then come (back) to Emacs Lisp, it feels like a hopelessly 
>> clunky language. `dash.el` was written to remedy this.
> 
> That sounds like they could be useful facilities.  Since they are real
> features, not mere aliases and trivialities, they would not have the
> main flaw of s.el.

How is s.el ‘mere aliases and trivialities’ and dash.el isn’t? Both provides convenience APIs for easier use...

Below is a random long function in s.el... 
(defun s-shared-end (s1 s2)
  "Returns the longest suffix S1 and S2 have in common."
  (declare (pure t) (side-effect-free t))
  (let* ((l1 (length s1))
         (l2 (length s2))
         (search-length (min l1 l2))
         (i 0))
    (while (and (< i search-length)
                (= (aref s1 (- l1 i 1)) (aref s2 (- l2 i 1))))
      (setq i (1+ i)))
    ;; If I is 0, then it means that there's no common suffix between
    ;; S1 and S2.
    ;;
    ;; However, since (substring s (- 0)) will return the whole
    ;; string, `s-shared-end' should simply return the empty string
    ;; when I is 0.
    (if (zerop i)
        ""
      (substring s1 (- i)))))

And below is a random short function from dash.el...

(defun -map (fn list)
  "Return a new list consisting of the result of applying FN to the items in LIST."
  (mapcar fn list))

Just that some aliases exist for  consistency doesn’t mean that the whole package is filled with aliases.

> However, we should look carefully at the specific interfaces
> of the entry points.  Including something into Emacs is a time
> for regularizing little details.
> 
> Also, I get the impression the names don't fit Emacs's conventions.
> 
> -- 
> Dr Richard Stallman
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)
> 
> 
> 

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 686 bytes --]

   Just that some aliases exist for consistency doesn’t mean that the
   whole package is filled with aliases.

Nobody claimed that, quite the opposite.  The main flaw of s.el is
still that it is _mainly_ trivial wrappers.

A function for finding the common suffix seems like a useful addition,
that is very concrete and can most surley be added to e.g. subr-x.el
or subr.el as maybe 'string-common-suffix STRING1 STRING' or something
similar, and implemented using `compare-strings'

It would be interesting to hear how people use this function, is it as
a predicate (where string-suffix-p on both strings would suffice) or
do you use the returned value which seems uninteresting?

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: ams@gnu.org (Alfred M. Szmidt)
> Date: Sat, 09 May 2020 06:57:54 -0400
> Cc: joostkremers@fastmail.fm, rms@gnu.org, Emacs-devel@gnu.org
> 
> A function for finding the common suffix seems like a useful addition,
> that is very concrete and can most surley be added to e.g. subr-x.el
> or subr.el as maybe 'string-common-suffix STRING1 STRING' or something
> similar, and implemented using `compare-strings'

subr.el is preloaded, and so we don't add there functions unless they
are used very frequently, or are called by preloaded code.  But we
have other places for functions that are deemed important, but whuich
we don't want to preload.

P.S.  Would you please fix your MUA to not mangle non-ASCII names of
your correspondents?  They come out as octal escapes, for some
reason.  Maybe it's because your message doesn't state an encoding.

octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Alfred M. Szmidt]

   P.S.  Would you please fix your MUA to not mangle non-ASCII names of
   your correspondents?  They come out as octal escapes, for some
   reason.  Maybe it's because your message doesn't state an encoding.

Thank you for the hint. I'm using rmail (and Emacs 26.3), maybe
someone who knows rmail here can pitch in why it might be doing that
or where to look.

Re: octal escapes with rmail

[Eli Zaretskii]

> From: ams@gnu.org (Alfred M. Szmidt)
> CC: pcr910303@icloud.com, joostkremers@fastmail.fm, rms@gnu.org,
> 	Emacs-devel@gnu.org
> Date: Sat, 09 May 2020 07:29:46 -0400
> 
>    P.S.  Would you please fix your MUA to not mangle non-ASCII names of
>    your correspondents?  They come out as octal escapes, for some
>    reason.  Maybe it's because your message doesn't state an encoding.
> 
> Thank you for the hint. I'm using rmail (and Emacs 26.3), maybe
> someone who knows rmail here can pitch in why it might be doing that
> or where to look.

I'm also using Rmail, so it is not Rmail that causes it, at least not
directly.  Most likely, it's some setting of coding-systems in your
init files.

Re: octal escapes with rmail

[Alfred M. Szmidt]

   >    P.S.  Would you please fix your MUA to not mangle non-ASCII names of
   >    your correspondents?  They come out as octal escapes, for some
   >    reason.  Maybe it's because your message doesn't state an encoding.
   > 
   > Thank you for the hint. I'm using rmail (and Emacs 26.3), maybe
   > someone who knows rmail here can pitch in why it might be doing that
   > or where to look.

   I'm also using Rmail, so it is not Rmail that causes it, at least not
   directly.  Most likely, it's some setting of coding-systems in your
   init files.

Hm, I don't have any coding-systems settings in my init file.  I'll
see if I can reproduce this using emacs -Q.

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Andreas Schwab]

On Mai 09 2020, Alfred M. Szmidt wrote:

> Thank you for the hint. I'm using rmail (and Emacs 26.3), maybe
> someone who knows rmail here can pitch in why it might be doing that
> or where to look.

What does (rfc2047-decode-string "=?utf-8?B?7KGw7ISx67mI?=") return?

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Sat, 09 May 2020 14:53:10 +0200
> Cc: joostkremers@fastmail.fm, Eli Zaretskii <eliz@gnu.org>, rms@gnu.org,
>  pcr910303@icloud.com, Emacs-devel@gnu.org
> 
> What does (rfc2047-decode-string "=?utf-8?B?7KGw7ISx67mI?=") return?

Another detail, which may or may not be relevant: what is your value
of mail-user-agent?

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Alfred M. Szmidt]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 195 bytes --]

   Another detail, which may or may not be relevant: what is your value
   of mail-user-agent?

mail-user-agent is a variable defined in ‘simple.el’.
Its value is ‘sendmail-user-agent’

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Eli Zaretskii]

> From: "Alfred M. Szmidt" <ams@gnu.org>
> Cc: schwab@linux-m68k.org, Emacs-devel@gnu.org
> Date: Sat, 09 May 2020 09:20:27 -0400
> 
>    Another detail, which may or may not be relevant: what is your value
>    of mail-user-agent?
> 
> mail-user-agent is a variable defined in ‘simple.el’.
> Its value is ‘sendmail-user-agent’

Same as mine, thanks.

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Alfred M. Szmidt]

   > Thank you for the hint. I'm using rmail (and Emacs 26.3), maybe
   > someone who knows rmail here can pitch in why it might be doing
   > that or where to look.

   What does (rfc2047-decode-string "=?utf-8?B?7KGw7ISx67mI?=")

It returns the following (converted to hex, so I don't send raw
garbage):

  22ec a1b0 ec84 b1eb b988 22          

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Eli Zaretskii]

> From: "Alfred M. Szmidt" <ams@gnu.org>
> Cc: eliz@gnu.org, joostkremers@fastmail.fm, rms@gnu.org,
> 	pcr910303@icloud.com, Emacs-devel@gnu.org
> Date: Sat, 09 May 2020 09:20:30 -0400
> 
>    > Thank you for the hint. I'm using rmail (and Emacs 26.3), maybe
>    > someone who knows rmail here can pitch in why it might be doing
>    > that or where to look.
> 
>    What does (rfc2047-decode-string "=?utf-8?B?7KGw7ISx67mI?=")
> 
> It returns the following (converted to hex, so I don't send raw
> garbage):
> 
>   22ec a1b0 ec84 b1eb b988 22          

That's wrong, it should be just 3 characters, c870 c131 be48.

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Stefan Monnier]

>    P.S.  Would you please fix your MUA to not mangle non-ASCII names of
>    your correspondents?  They come out as octal escapes, for some
>    reason.  Maybe it's because your message doesn't state an encoding.

FWIW, I don't see any octal sequences in Alfred's message (using
Gnus to read it).


        Stefan

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>    P.S.  Would you please fix your MUA to not mangle non-ASCII names of
>>    your correspondents?  They come out as octal escapes, for some
>>    reason.  Maybe it's because your message doesn't state an encoding.
>
> FWIW, I don't see any octal sequences in Alfred's message (using
> Gnus to read it).

The sequences are in the "To:" header.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Andreas Schwab]

On Mai 09 2020, Stefan Monnier wrote:

> FWIW, I don't see any octal sequences in Alfred's message (using
> Gnus to read it).

You mean <E1jXNB4-0000Gp-9c@fencepost.gnu.org>?  Look at the To
header.

Also, <E1jXPP1-0004sY-6K@fencepost.gnu.org> lacks a charset declaration,
that's why it is broken in the archive.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  joostkremers@fastmail.fm,  rms@gnu.org,
>   pcr910303@icloud.com,  Emacs-devel@gnu.org
> Date: Sat, 09 May 2020 10:22:43 -0400
> 
> >    P.S.  Would you please fix your MUA to not mangle non-ASCII names of
> >    your correspondents?  They come out as octal escapes, for some
> >    reason.  Maybe it's because your message doesn't state an encoding.
> 
> FWIW, I don't see any octal sequences in Alfred's message (using
> Gnus to read it).

Likely because your locale is UTF-8 (or maybe you are looking at the
wrong message).  Or maybe what you see is one of those old tricks Gnus
had added long ago to "handle" broken emails.

Alfred's messages arrive with the non-ASCII characters in the From
header in UTF-8, but there's no MIME charset header anywhere in sight.
You can download the archived copies of his messages and look at the
mbox file.

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Andreas Schwab]

On Mai 09 2020, Eli Zaretskii wrote:

> Alfred's messages arrive with the non-ASCII characters in the From
> header in UTF-8, but there's no MIME charset header anywhere in sight.

MIME headers are only relevant for the body.  All header fields are
encoded independently.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Re: octal escapes with rmail [was: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Stefan Monnier]

> Likely because your locale is UTF-8

That must be it, then.

> Alfred's messages arrive with the non-ASCII characters in the From
> header in UTF-8, but there's no MIME charset header anywhere in sight.

The charset headers wouldn't help.  The only valid format, AFAIK is when
encoded with the ?utf8?blabla? format (tho it can use another charset
than utf-8, of course).


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

>   > But you just described what dash does. ;-) It is just a collection
>   > of list-handling functions such as they exist in modern functional
>   > programming languages. If you're used to thinking in this paradigm
>   > and then come (back) to Emacs Lisp, it feels like a hopelessly
>   > clunky language. `dash.el` was written to remedy this.
>
> That sounds like they could be useful facilities.  Since they are real
> features, not mere aliases and trivialities, they would not have the
> main flaw of s.el.

Richard, I think this is a good opportunity for you to actually go and
really see what dash.el is and also what s.el is, maybe even code a
little with them. Your comments really make it look like you vaguely
understand what they are about. I'm sorry if that's not the case.

But what is more disturbing is that you take strong positions about
them that are in conflict with Stefan Monnier for example. These
tensions needs to be resolved. Maybe Eli can facilitate this?

Except if I missed it, here are the questions I didn't see an answer to:

- As far as I know you are the only one who objected strongly against
s.el in ELPA (others please voice your opinion if you think like
Richard). This created an atmosphere of people fearing that you would
use your authority to "veto" such adoption despite Stefan Monnier &
others wanting s.el in ELPA. I think you need to clarify your
position/power here.
- For most users, dash.el and s.el are very similar in nature. dash.el
is already in ELPA. If we refuse s.el, isn't it inconsistent? What
about the message we send?

Thanks for taking the time to address these.

Philippe

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 9 May 2020 09:38:04 +0200
> Cc: Joost Kremers <joostkremers@fastmail.fm>,
>  Emacs developers <emacs-devel@gnu.org>
> 
> But what is more disturbing is that you take strong positions about
> them that are in conflict with Stefan Monnier for example. These
> tensions needs to be resolved. Maybe Eli can facilitate this?

I don't know what you mean.  We each have our strong positions
sometimes, and there's nothing wrong with two or more people having
opposite strong positions.  It's normal to have differences of
opinions, nothing to find disturbing about that.  The important thing
is how we go about finding ways of reconciling the differences of
opinions, not how to cause us all to think the same (which I think is
impossible and even damaging).

> - As far as I know you are the only one who objected strongly against
> s.el in ELPA (others please voice your opinion if you think like
> Richard). This created an atmosphere of people fearing that you would
> use your authority to "veto" such adoption despite Stefan Monnier &
> others wanting s.el in ELPA. I think you need to clarify your
> position/power here.

Richard also proposed a compromise which AFAIU would allow it to be
added.  For some reason, that proposal got no responses at all.

> - For most users, dash.el and s.el are very similar in nature. dash.el
> is already in ELPA. If we refuse s.el, isn't it inconsistent? What
> about the message we send?

Are you saying that popularity and similarity of a package is the only
criterion we should apply when deciding whether to add a package to
ELPA?  IOW, are you saying that the technical details of the package's
implementation should not matter, for fear of sending the wrong
message?

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> > - As far as I know you are the only one who objected strongly against
> > s.el in ELPA (others please voice your opinion if you think like
> > Richard). This created an atmosphere of people fearing that you would
> > use your authority to "veto" such adoption despite Stefan Monnier &
> > others wanting s.el in ELPA. I think you need to clarify your
> > position/power here.
>
> Richard also proposed a compromise which AFAIU would allow it to be
> added.  For some reason, that proposal got no responses at all.

I tried to find that compromise again, all I found was "See my message
to Stefan for a change that would make s.el ok to add." but could not
find this message. If you would be so kind as to quote Richard again
so I have his perspective.


> > - For most users, dash.el and s.el are very similar in nature. dash.el
> > is already in ELPA. If we refuse s.el, isn't it inconsistent? What
> > about the message we send?
>
> Are you saying that popularity and similarity of a package is the only
> criterion we should apply when deciding whether to add a package to
> ELPA?  IOW, are you saying that the technical details of the package's
> implementation should not matter, for fear of sending the wrong
> message?

No, that's not what I am saying. I'd be interesting to see if you can
understand what I am trying to say without me explaining it in
details, because I feel that many people would understand what I am
trying to say, even if it is not said perfectly. This would be good
exercice to figure out how I could say things differently and how you
could try to understand things differently.

Can you give it another try? And if you still don't understand I'll
explain it another way.

Philippe

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 9 May 2020 12:56:03 +0200
> Cc: Richard Stallman <rms@gnu.org>, Joost Kremers <joostkremers@fastmail.fm>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> > > - As far as I know you are the only one who objected strongly against
> > > s.el in ELPA (others please voice your opinion if you think like
> > > Richard). This created an atmosphere of people fearing that you would
> > > use your authority to "veto" such adoption despite Stefan Monnier &
> > > others wanting s.el in ELPA. I think you need to clarify your
> > > position/power here.
> >
> > Richard also proposed a compromise which AFAIU would allow it to be
> > added.  For some reason, that proposal got no responses at all.
> 
> I tried to find that compromise again, all I found was "See my message
> to Stefan for a change that would make s.el ok to add." but could not
> find this message. If you would be so kind as to quote Richard again
> so I have his perspective.

Look here:

  https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg00850.html

> Can you give it another try? And if you still don't understand I'll
> explain it another way.

Please believe me when I say that my interpretation of what you wrote
is such and such.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> > > Richard also proposed a compromise which AFAIU would allow it to be
> > > added.  For some reason, that proposal got no responses at all.
> >
> > I tried to find that compromise again, all I found was "See my message
> > to Stefan for a change that would make s.el ok to add." but could not
> > find this message. If you would be so kind as to quote Richard again
> > so I have his perspective.
>
> Look here:
>
>   https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg00850.html

Thanks!

> > > > - For most users, dash.el and s.el are very similar in nature. dash.el
> > > > is already in ELPA. If we refuse s.el, isn't it inconsistent? What
> > > > about the message we send?
>
> > > Are you saying that popularity and similarity of a package is the only
> > > criterion we should apply when deciding whether to add a package to
> >  > ELPA?  IOW, are you saying that the technical details of the package's
> > > implementation should not matter, for fear of sending the wrong
> > >message?
>
> > Can you give it another try? And if you still don't understand I'll
> > explain it another way.
>
> Please believe me when I say that my interpretation of what you wrote
> is such and such.

I believe you it is, I was asking you to *try* to find another
interpretation. Since you don't want to do that, here is my
re-formulation of what I said:

"For most users of dash.el and s.el, they will be surprised to see
dash.el accepted in ELPA but not s.el because they might feel these
packages are very similar in nature (provide high-order programming
discoverable functions to Emacs). To them it might look inconsistant
and they might wrongly assume emacs-devel is driven by arbitrary
decisions when it comes to accepting what goes into ELPA. Without a
good communication on why s.el is refused but dash.el is not, many
people could deduce that ELPA is a dead end and that only MELPA is the
sane route, further distancing ELPA from "where the real development
of emacs packages happens"".

There, hope it's more clear. Surprised you couldn't infer that from my
first statement, but ok it's not the first time, I'll try to be extra
explicit from now on.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 9 May 2020 14:13:29 +0200
> Cc: Richard Stallman <rms@gnu.org>, Joost Kremers <joostkremers@fastmail.fm>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> > > Can you give it another try? And if you still don't understand I'll
> > > explain it another way.
> >
> > Please believe me when I say that my interpretation of what you wrote
> > is such and such.
> 
> I believe you it is, I was asking you to *try* to find another
> interpretation. Since you don't want to do that

Can we please assume that each one of us reads the other's messages
attentively, and tries to understand and interpret it in good faith?

> "For most users of dash.el and s.el, they will be surprised to see
> dash.el accepted in ELPA but not s.el because they might feel these
> packages are very similar in nature (provide high-order programming
> discoverable functions to Emacs). To them it might look inconsistant
> and they might wrongly assume emacs-devel is driven by arbitrary
> decisions when it comes to accepting what goes into ELPA. Without a
> good communication on why s.el is refused but dash.el is not, many
> people could deduce that ELPA is a dead end and that only MELPA is the
> sane route, further distancing ELPA from "where the real development
> of emacs packages happens"".

How is consistency relevant here?  They are 2 different packages
targeting different domains.  Each one of them should be assessed
separately and on its own merit.  Thus, I see no reason for people to
be surprised that two different packages are handled differently.
(And the discussion is not yet over, so what will be the conclusion is
so far anyone's guess.)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1497 bytes --]

>
> > I believe you it is, I was asking you to *try* to find another
> > interpretation. Since you don't want to do that
>
> Can we please assume that each one of us reads the other's messages
> attentively, and tries to understand and interpret it in good faith?
>

You are correct. Sorry.


> "For most users of dash.el and s.el, they will be surprised to see
> > dash.el accepted in ELPA but not s.el because they might feel these
> > packages are very similar in nature (provide high-order programming
> > discoverable functions to Emacs). To them it might look inconsistant
> > and they might wrongly assume emacs-devel is driven by arbitrary
> > decisions when it comes to accepting what goes into ELPA. Without a
> > good communication on why s.el is refused but dash.el is not, many
> > people could deduce that ELPA is a dead end and that only MELPA is the
> > sane route, further distancing ELPA from "where the real development
> > of emacs packages happens"".
>
> How is consistency relevant here?  They are 2 different packages
> targeting different domains.  Each one of them should be assessed
> separately and on its own merit.  Thus, I see no reason for people to
> be surprised that two different packages are handled differently.
> (And the discussion is not yet over, so what will be the conclusion is
> so far anyone's guess.)
>

Fair point. I'm probably not very objective as I am the one who asked
magnars for help and I'd hate to tell him that it was all "for nothing".

>

[-- Attachment #2: Type: text/html, Size: 2304 bytes --]

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I tried to find that compromise again, all I found was "See my message
  > to Stefan for a change that would make s.el ok to add." but could not
  > find this message. If you would be so kind as to quote Richard again
  > so I have his perspective.

I proposed to prefix the function names with 'clo' on the
understanding that the names it defines are inspired by Clojure.  I
said this would result in names such as 'clostring-prepend', because a
message I had read had led me to think that they started with
'string-'.

But maybe I was misled by what I had read.  Do they actually start with 's-'?

If so, I still propose the same compromise, prefixing with 'clo', but
it would result in names starting with 'clos-'.

Would you like to send me the API documentation of s.el?


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

>   > I tried to find that compromise again, all I found was "See my message
>   > to Stefan for a change that would make s.el ok to add." but could not
>   > find this message. If you would be so kind as to quote Richard again
>   > so I have his perspective.
>
> I proposed to prefix the function names with 'clo' on the
> understanding that the names it defines are inspired by Clojure.  I
> said this would result in names such as 'clostring-prepend', because a
> message I had read had led me to think that they started with
> 'string-'.

I think it's a reasonable approach in theory. Practically I don't
think the maintainer would be ok to maintain two prefixes (one for
MELPA, one for ELPA) or willing to rename it because so many packages
depends on it.

I guess if one day we have namespaces or if it's easy to write a tool
that takes the existing s.el and rename it automatically to
clojure-string then we could automate it in ELPA under a different
name, but then I'm not sure that anyone would use it in ELPA except to
"try it out".


> But maybe I was misled by what I had read.  Do they actually start with 's-'?

Yes it starts with `s-`.


> Would you like to send me the API documentation of s.el?

https://github.com/magnars/s.el

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I think it's a reasonable approach in theory. Practically I don't
  > think the maintainer would be ok to maintain two prefixes (one for
  > MELPA, one for ELPA) or willing to rename it because so many packages
  > depends on it.

The first thing to do is ask the maintainer to simply change them. 

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

>   > I think it's a reasonable approach in theory. Practically I don't
>   > think the maintainer would be ok to maintain two prefixes (one for
>   > MELPA, one for ELPA) or willing to rename it because so many packages
>   > depends on it.
>
> The first thing to do is ask the maintainer to simply change them.

I'd like additional people to agree with you on this first. I mean
sure I can try but it sounds so unlikely to happen, the author already
gave us all of his code but asked for not too much work for it to be
included in ELPA.

The only thing I can see myself do is to ask on a humorous tone as if
we all know that this is not going to happen, and on a lucky moon he
might reply with "you know what? I was actually thinking about it" :-)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Would you like to send me the API documentation of s.el?

  > https://github.com/magnars/s.el

The URL suggests it is actually the source code.  I can make do with
that, but I would rather get just the API spec, so I can honestly say
I never read the source code.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Amin Bandali]

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > Would you like to send me the API documentation of s.el?
>
>   > https://github.com/magnars/s.el
>
> The URL suggests it is actually the source code.  I can make do with
> that, but I would rather get just the API spec, so I can honestly say
> I never read the source code.

Indeed, the address may suggest it's an el file, but in fact it is not,
and it's the URL for the git repository containing s.el and other files.

If you would like a direct link to their README which contains examples
of invoking each function (i.e. in a way, their API spec), you could use
either of these:

https://github.com/magnars/s.el/blob/master/README.md
https://raw.githubusercontent.com/magnars/s.el/master/README.md

The former is an HTML rendered version of the README, and the latter its
raw Markdown markup source.

Hope this helps.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> >   > > Would you like to send me the API documentation of s.el?
> >
> >   > https://github.com/magnars/s.el
> >
> > The URL suggests it is actually the source code.  I can make do with
> > that, but I would rather get just the API spec, so I can honestly say
> > I never read the source code.
>
> Indeed, the address may suggest it's an el file, but in fact it is not,
> and it's the URL for the git repository containing s.el and other files.

And when you arrive on this page, the README is displayed which is the API spec.

It starts with a short summary of the function (from which I already
understand 95% of what it does just by reading the signature) and then
for each function if you click you have a longer description of what
it does and a small example of use.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If you would like a direct link to their README which contains examples
  > of invoking each function (i.e. in a way, their API spec), you could use
  > either of these:

  > https://github.com/magnars/s.el/blob/master/README.md
  > https://raw.githubusercontent.com/magnars/s.el/master/README.md

Thanks.

Someone emailed me that file, so now I see what is in s.el.
65 functions, some of them simply renamed to add 's-',
some of them with calling conventions changed, some of them
useful added functionality (though nothing profound),
some of them useless additions (s-prepend, etc).

Magnars is very skilled and very capable.  If he were working in
cooperation with us, I am sure he could contribute a geat deal.
However, the directionb he has chosen to go in leaves us facing
two bad alternatives.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > I tried to find that compromise again, all I found was "See my message
>   > to Stefan for a change that would make s.el ok to add." but could not
>   > find this message. If you would be so kind as to quote Richard again
>   > so I have his perspective.
>
> I proposed to prefix the function names with 'clo' on the
> understanding that the names it defines are inspired by Clojure.  I
> said this would result in names such as 'clostring-prepend', because a
> message I had read had led me to think that they started with
> 'string-'.
>
> But maybe I was misled by what I had read.  Do they actually start with 's-'?
>
> If so, I still propose the same compromise, prefixing with 'clo', but
> it would result in names starting with 'clos-'.
>
> Would you like to send me the API documentation of s.el?


The s.el API is actually quite a bit richer than that of
clojure. Indeed, I found at least one non-standard clojure library that
looks like s.el. So I think the logic is flowing the other way.

So "clostring-prepend" doesn't really achieve anything other than making
it unreadable. Magnar's clever choice of naming is, I think, one of the
reasons people like his libraries. Why change a feature and turn it into
unreadability.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So "clostring-prepend"

Rather, 'clos-prepend'.

			   doesn't really achieve anything other than making
  > it unreadable. Magnar's clever choice of naming is, I think, one of the
  > reasons people like his libraries. Why change a feature and turn it into
  > unreadability.

'clos-prepend' is not harder to read than 's-prepend'.

Ostensibly it is "just an optional package", but if "most packages use it"
it would really be a replacement installed half-way.

The s- functions, by themselves, are clean.  s.el by itself is not a
mess.  But Emacs plus s.el is a mess.

It contains 65 functions, similar to the Emacs string functions but
with idiosyncratic incompatibilities scattered throughout.  Some of
the changes are good, some are gratuitously bad.  Two different string
functiob interface, the one we developed and the one that was dropped
on us from outside.

Would we have any say over further development of the s- interface?
I get the impression we would not.




-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[조성빈]

Richard Stallman <rms@gnu.org> 작성:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>> So "clostring-prepend"
>
> Rather, 'clos-prepend'.
>
>> 			   doesn't really achieve anything other than making
>> it unreadable. Magnar's clever choice of naming is, I think, one of the
>> reasons people like his libraries. Why change a feature and turn it into
>> unreadability.
>
> 'clos-prepend' is not harder to read than 's-prepend'.

The problem is that lots of packages use ‘s-prepend’, rather than
‘clos-prepend’. There are too many packages that are blocked on being ELPA
because of s.el and f.el (as dash.el is on ELPA).

> Ostensibly it is "just an optional package", but if "most packages use it"
> it would really be a replacement installed half-way.
>
> The s- functions, by themselves, are clean.  s.el by itself is not a
> mess.  But Emacs plus s.el is a mess.

That’s true. And there was a big previous discussion where some people here
tried to explain why people tend to use the s.el functions instead of the
Elisp ones, and rename some Elisp functions to make them more sense. But
nobody wanted to add s.el into Emacs core, I think everybody would be opposed
to that.

> It contains 65 functions, similar to the Emacs string functions but
> with idiosyncratic incompatibilities scattered throughout.  Some of
> the changes are good, some are gratuitously bad.  Two different string
> function interface, the one we developed and the one that was dropped
> on us from outside.
>
> Would we have any say over further development of the s- interface?
> I get the impression we would not.
>
>
>
>
> -- 
> Dr Richard Stallman
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   > 'clos-prepend' is not harder to read than 's-prepend'.

   The problem is that lots of packages use `s-prepend', rather than
   `clos-prepend'. There are too many packages that are blocked on being ELPA
   because of s.el and f.el (as dash.el is on ELPA).

Then those packages should be modified to not use s.el/dash.el/f.el
before they are included in ELPA.  Just like that would be a
requirement for them to be included in Emacs.  

s.el could be modified in such a way that it would have compiler
warnings for the functins that Emacs Lisp already has, and where there
are different semantics encourage to use the Emacs Lisp way of
writting.  The functions that do not have a equivalent could added in
subr-x.  Users would then, slowly, migrate their code, and make it
easier to include things in ELPA in the future.

This is just a matter of following the good practises that already
exist in Emacs.  It would be a bad idea to start making a mess, and
then encouraging this mess to become larger.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

ams@gnu.org (Alfred M. Szmidt) writes:

>    > 'clos-prepend' is not harder to read than 's-prepend'.
>
>    The problem is that lots of packages use `s-prepend', rather than
>    `clos-prepend'. There are too many packages that are blocked on being ELPA
>    because of s.el and f.el (as dash.el is on ELPA).
>
> Then those packages should be modified to not use s.el/dash.el/f.el
> before they are included in ELPA.  Just like that would be a
> requirement for them to be included in Emacs.  
>
> s.el could be modified in such a way that it would have compiler
> warnings for the functins that Emacs Lisp already has, and where there
> are different semantics encourage to use the Emacs Lisp way of
> writting.  The functions that do not have a equivalent could added in
> subr-x.  Users would then, slowly, migrate their code, and make it
> easier to include things in ELPA in the future.
>
> This is just a matter of following the good practises that already
> exist in Emacs.  It would be a bad idea to start making a mess, and
> then encouraging this mess to become larger.


Posited on s.el being a mess, which neither it, nor dash.el is. They are
both nice APIs that are nice to use.

I did manage to drop a dash.el dependency form one of my libraries and
replace it with seq.el. That worked because it was close to a drop
in. But people have already chosen to work with dash, or s, or f, even
though it means adding a dependency because they are nice.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   > This is just a matter of following the good practises that already
   > exist in Emacs.  It would be a bad idea to start making a mess, and
   > then encouraging this mess to become larger.

   Posited on s.el being a mess, which neither it, nor dash.el is. They are
   both nice APIs that are nice to use.

In isolation these libraries do not create a mess, and it was never
claimed that they are so.  The issue is making them part of
Emacs/ELPA, thus encouraging people to start using non-standard Emacs
Lisp conventions in Emacs.  _That_ would be the mess.

I really urge people to carefully read what people have written to
minimize these type of misunderstandings.

   I did manage to drop a dash.el dependency form one of my libraries and
   replace it with seq.el. That worked because it was close to a drop
   in. But people have already chosen to work with dash, or s, or f, even
   though it means adding a dependency because they are nice.

Would you like to suggest which parts of those libraries are nice in
your opinon so that they could maybe be added to Emacs, following
normal Emacs conventions?

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[조성빈]

> 2020. 5. 12. 오전 1:42, Alfred M. Szmidt <ams@gnu.org> 작성:
> 
> 
>> 
>> This is just a matter of following the good practises that already
>> exist in Emacs.  It would be a bad idea to start making a mess, and
>> then encouraging this mess to become larger.
> 
>   Posited on s.el being a mess, which neither it, nor dash.el is. They are
>   both nice APIs that are nice to use.
> 
> In isolation these libraries do not create a mess, and it was never
> claimed that they are so.  The issue is making them part of
> Emacs/ELPA, thus encouraging people to start using non-standard Emacs
> Lisp conventions in Emacs.  _That_ would be the mess.
> 
> I really urge people to carefully read what people have written to
> minimize these type of misunderstandings.
> 
>   I did manage to drop a dash.el dependency form one of my libraries and
>   replace it with seq.el. That worked because it was close to a drop
>   in. But people have already chosen to work with dash, or s, or f, even
>   though it means adding a dependency because they are nice.
> 
> Would you like to suggest which parts of those libraries are nice in
> your opinon so that they could maybe be added to Emacs, following
> normal Emacs conventions?

There was a long, long controversial thread about this. (And I think this thread is a branch of it?)

All of this discussion started about having some more string functions and renaming some of them so that people didn’t need to use s.el. 

FWIW, nobody asked to add it in Emacs core. The only request was to add it in ELPA, which looks like everybody has a different idea of what it is. (I thought of it as a blessed package repo, but apparently a lot of people here thinks that it’s an extension to Emacs.)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> in ELPA, which looks like everybody has a different idea of what it is. (I
> thought of it as a blessed package repo, but apparently a lot of people here
> thinks that it’s an extension to Emacs.)

It's only "blessed" in the sense that they've been blessed with the luck
of having contributors who are willing to sign the relevant paperwork
and are willing to accommodate the few requests we make (mostly imposed
by the restrictions of the scripts that I was able to write to build
the packages).


        Stefan

elpa part of emacs? [Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Alfred M. Szmidt]

I as a user of Emacs have always considered ELPA to be part of Emacs
-- if I run M-x package-install I get package from ELPA.  That too me
means that someone in the Emacs team has done something to add them,
and see that they vetted to be sensible.  And it follows that it is
some how part of Emacs, and not a third party.

So ELPA to me is an curated list of Emacs packages that the
maintainers of Emacs feel are useful, and try to follow the same
standards as the rest of Emacs.

As for the main reason I think that people would want to keep their
package in ELPA is that they have a different release cadance than
Emacs, which is relatively slow.  

Allowing one to follow a specific package more closely than waiting
for it to be upgraded in Emacs, and then waiting for a new Emacs
release -- or even the situation when one is on an older version of
Emacs.

Re: elpa part of emacs? [Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Stefan Monnier]

> So ELPA to me is an curated list of Emacs packages that the
> maintainers of Emacs feel are useful, and try to follow the same
> standards as the rest of Emacs.

While it is indeed the case for some of the GNU ELPA packages, it's
definitely not the case for all of them, because it's not "curated" in
this sense.
Some would likely qualify as useless, others as poorly written, many
don't follow the coding standards, ...

[ Then again, that also applies to several packages bundled with
  Emacs ;-)  ]


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   All of this discussion started about having some more string
   functions and renaming some of them so that people didn't need to
   use s.el.

Nobody has said no to adding specific functions to Emacs, nobody has
rejected the idea of renaming functions where they make sense.

The original proposal suggested renaming functions in s.el to be
called string-* something.  So it seems that from the start, renaming
functions in s.el to follow a more Emacs Lisp style convention isn't
entierly alien.  But this seems to have had many objections recently.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > FWIW, nobody asked to add it in Emacs core. The only request was
  > to add it in ELPA, which looks like everybody has a different idea
  > of what it is.

I see the distinction, but either way it would cause the same problem.
The problem is a second, incoherent set of string functions.

You can _say_ that they are an extension we can ignore, but that is
not true in practice.  If "most packages use them", then no other
programs would be able to define those names for _anything_
and we would need to document them.  In effect, they would
be a part of the Emacs Lisp programming interface and not under
our control.

We can have those functions in Emacs (core, or GNU ELPA) if they
are in niche in the namespace, with a substantial prefix such
as a well-behaved Lisp library should have.
-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

>   > FWIW, nobody asked to add it in Emacs core. The only request was
>   > to add it in ELPA, which looks like everybody has a different idea
>   > of what it is.
> I see the distinction, but either way it would cause the same problem.
> The problem is a second, incoherent set of string functions.

But it's a problem we can't solve, because the library is out there are
people use it.  Furthermore it's only hypothetical.
The library's been in wide use for many years already, yet Emacs still
chugs along exactly as before.  Including it into GNU ELPA would make no
difference in this respect.

In any case, at this point I'm not really interested in adding any other
of those packages to GNU ELPA.  I'll just recommend people add MELPA to
their `package-archives` and move on.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 11 May 2020 23:55:10 -0400
> Cc: joostkremers@fastmail.fm, ams@gnu.org, phillip.lord@russet.org.uk,
>  pcr910303@icloud.com, Emacs-devel@gnu.org
> 
> > I see the distinction, but either way it would cause the same problem.
> > The problem is a second, incoherent set of string functions.
> 
> But it's a problem we can't solve, because the library is out there are
> people use it.  Furthermore it's only hypothetical.

It won't stay hypothetical if we allow incoherent packages into ELPA
and start accepting their use in other packages and eventually in
core.  People will ask us to use them more, people will ask us to
document them, people will ask us to fix bugs in them, and eventually
to use them in our own code.  In a word, people will rightfully expect
us to take full responsibility on every such package.  The costs will
come, I have no doubt about it.

> I'll just recommend people add MELPA to their `package-archives` and
> move on.

Please don't.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Date: Mon, 11 May 2020 23:55:10 -0400
>> Cc: joostkremers@fastmail.fm, ams@gnu.org, phillip.lord@russet.org.uk,
>>  pcr910303@icloud.com, Emacs-devel@gnu.org
>> 
>> > I see the distinction, but either way it would cause the same problem.
>> > The problem is a second, incoherent set of string functions.
>> 
>> But it's a problem we can't solve, because the library is out there are
>> people use it.  Furthermore it's only hypothetical.
>
> It won't stay hypothetical if we allow incoherent packages into ELPA
> and start accepting their use in other packages and eventually in
> core.  People will ask us to use them more, people will ask us to
> document them, people will ask us to fix bugs in them, and eventually
> to use them in our own code.  In a word, people will rightfully expect
> us to take full responsibility on every such package.  The costs will
> come, I have no doubt about it.


Well, dash, f, and s are already been maintained, already being used,
already being fixed, documented (rather well, actually), and tested.

Perhaps, if the people doing this work saw that their work was
gratefully accepted, and allowed to continue doing it with the minimum
of fuss and not too much design by committee, then they would continue
doing it.

>> I'll just recommend people add MELPA to their `package-archives` and
>> move on.
>
> Please don't.

Please understand why.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  rms@gnu.org,
>   joostkremers@fastmail.fm,  ams@gnu.org,  pcr910303@icloud.com,
>   Emacs-devel@gnu.org
> Date: Tue, 12 May 2020 18:30:18 +0100
> 
> Perhaps, if the people doing this work saw that their work was
> gratefully accepted, and allowed to continue doing it with the minimum
> of fuss and not too much design by committee, then they would continue
> doing it.

Is this what you think of how I'm doing my job for Emacs?  All I'm
doing here, most of the time, is make sure the code we get from
various contributions is up to our standards.  You seem to be saying
that I'm harming the project by doing so, because that drives the
contributors away.  Would you like me to step down?

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 12.05.2020 20:46, Eli Zaretskii wrote:
> Is this what you think of how I'm doing my job for Emacs?  All I'm
> doing here, most of the time, is make sure the code we get from
> various contributions is up to our standards.

Standards etched in stone many moons ago.

> You seem to be saying
> that I'm harming the project by doing so, because that drives the
> contributors away.  Would you like me to step down?

Could we perhaps discuss these things without blackmail? The problems 
are real, and you have the power to make them better.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> Cc: rms@gnu.org, joostkremers@fastmail.fm, Emacs-devel@gnu.org, ams@gnu.org,
>  monnier@iro.umontreal.ca, pcr910303@icloud.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 12 May 2020 21:03:40 +0300
> 
> > You seem to be saying
> > that I'm harming the project by doing so, because that drives the
> > contributors away.  Would you like me to step down?
> 
> Could we perhaps discuss these things without blackmail? The problems 
> are real, and you have the power to make them better.

The only problem I see is your unfriendly conduct, and I'm evidently
completely powerless to do anything about that.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Could we perhaps discuss these things without blackmail? The problems 
  > are real, and you have the power to make them better.

We are carefully trying to decide what will make the situation better
rather than worse.  In the mean time, please stop trying to badger
people.  It won't win any argument, but it makes this list painful to
be on.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 13.05.2020 07:04, Richard Stallman wrote:
>    > Could we perhaps discuss these things without blackmail? The problems
>    > are real, and you have the power to make them better.
> 
> We are carefully trying to decide what will make the situation better
> rather than worse.  In the mean time, please stop trying to badger
> people.  It won't win any argument, but it makes this list painful to
> be on.

A clear acknowledgment of the problem and some mutual understanding 
would have made this discussion flow a lot smoother.

It's not like anybody's asking for a fix right this minute.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> Is this what you think of how I'm doing my job for Emacs?  All I'm
> doing here, most of the time, is make sure the code we get from
> various contributions is up to our standards.

This whole discussion is not about the standards we should apply to
Emacs, but the standards we should apply to GNU ELPA.

The answer to "Why are so many great packages not trying to get included
in GNU Emacs?" is exactly this whole discussion: it's because they don't
want to have to deal with all the rules to follow.

> You seem to be saying that I'm harming the project by doing so,
> because that drives the contributors away.

No, but I do think that imposing the same rules to GNU ELPA as we impose
on Emacs drives contributors away (and it imposes yet more work on us,
so it's a lose-lose option).


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Phillip Lord <phillip.lord@russet.org.uk>
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  rms@gnu.org,
>>   joostkremers@fastmail.fm,  ams@gnu.org,  pcr910303@icloud.com,
>>   Emacs-devel@gnu.org
>> Date: Tue, 12 May 2020 18:30:18 +0100
>> 
>> Perhaps, if the people doing this work saw that their work was
>> gratefully accepted, and allowed to continue doing it with the minimum
>> of fuss and not too much design by committee, then they would continue
>> doing it.
>
> Is this what you think of how I'm doing my job for Emacs?  All I'm
> doing here, most of the time, is make sure the code we get from
> various contributions is up to our standards.  You seem to be saying
> that I'm harming the project by doing so, because that drives the
> contributors away.  Would you like me to step down?


Eli

No, I wouldn't want you to step down, and this was not directed
specifically at you. Overall, emacs-devel is a fairly conservative
place, though. That, combined with a slow and onerous copyright
assignment proceedure (proceedure not policy: the proceedure could be
fixed without changing policy), means that the ecosystem outside core is
richer than inside.

I believe that this produces a relatively poor new user
experience. Tools like magit, or CIDER draw people in; and for
developers of Emacs packages dash.el does the same. But people have to
know that they are there; as it stands, a new developer picking Emacs
would assuming it does not do Clojure, or Rust, or GO, or
autocompletion, or lsp. I watch my students faced with Emacs and this is
just too much of a struggle.

I understand why you worry about code quality, and this is valid. I
think most of Magnar's code has proven itself; the usage stats speak for
themselves. If packages like this do not meet Emacs standards, perhaps,
this it is Emacs standards that need to change to fit.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 731 bytes --]



> On May 12, 2020, at 5:38 PM, Phillip Lord <phillip.lord@russet.org.uk> wrote:
> 
> 
> I understand why you worry about code quality, and this is valid. I
> think most of Magnar's code has proven itself; the usage stats speak for
> themselves. If packages like this do not meet Emacs standards, perhaps,
> this it is Emacs standards that need to change to fit.
> 

Good code quality along doesn’t mean a package is suitable to be added to Emacs core. One should also consider the tradition and the overall style of the interface. E.g., how do you feel about having both string-prefix-p and s-starts-with? in Emacs? If we have a set of good string functions, should Clojure developers add them to their core?

Yuan

[-- Attachment #2: Type: text/html, Size: 4882 bytes --]

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Yuan Fu <casouri@gmail.com> writes:

>> On May 12, 2020, at 5:38 PM, Phillip Lord <phillip.lord@russet.org.uk> wrote:
>> 
>> 
>> I understand why you worry about code quality, and this is valid. I
>> think most of Magnar's code has proven itself; the usage stats speak for
>> themselves. If packages like this do not meet Emacs standards, perhaps,
>> this it is Emacs standards that need to change to fit.
>> 
>
> Good code quality along doesn’t mean a package is suitable to be added
> to Emacs core. One should also consider the tradition and the overall
> style of the interface. E.g., how do you feel about having both
> string-prefix-p and s-starts-with? in Emacs? If we have a set of good
> string functions, should Clojure developers add them to their core?

Clojure has taken stuff from lots of other places. The language is
nothing really new there, it's just built up from a while loads of other
language features.

So, yes, in answer to the question.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Phillip Lord <phillip.lord@russet.org.uk>
> Cc: monnier@iro.umontreal.ca,  rms@gnu.org,  joostkremers@fastmail.fm,
>   ams@gnu.org,  pcr910303@icloud.com,  Emacs-devel@gnu.org
> Date: Tue, 12 May 2020 22:38:31 +0100
> 
> No, I wouldn't want you to step down, and this was not directed
> specifically at you.

The message had me in the To header and was a direct response to
something I wrote.  I interpreted it as being aimed at me, yes.

> Overall, emacs-devel is a fairly conservative place, though. That,
> combined with a slow and onerous copyright assignment proceedure
> (proceedure not policy: the proceedure could be fixed without
> changing policy), means that the ecosystem outside core is richer
> than inside.

We can discuss which parts of our requirements are overly conservative
and need to be relaxed (but then we should talk about specifics, not
in such general terms).  But let's agree about one thing: none of
these requirements have anything to do with being ungrateful to our
contributors.  Every contribution is very welcome, and the standards
are not meant to be obstacles for the sake of obstacles, they are
meant to keep the quality high enough for us all to be proud of what
we achieved.

> If packages like this do not meet Emacs standards, perhaps, this it
> is Emacs standards that need to change to fit.

You make it sound as if asking the developer of a package, e.g., to
reformat the doc strings so that the first line is a complete sentence
somehow means the whole package is garbage.  IOW, the delta between
the original package and what would make it on par with the rest of
the code is usually quite small.  It is not a rejection of the
package, it is a small set of minor requests.

It is still possible that our standards should change in some ways,
but let's again keep this in perspective and not blow it out of
proportion.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 13.05.2020 17:38, Eli Zaretskii wrote:
> But let's agree about one thing: none of
> these requirements have anything to do with being ungrateful to our
> contributors.

At least some contributors have certain expectations on gratitude that 
don't end with "thanks", and include a certain degree of autonomy. Like 
having the last word in a technical discussion, for example.

> Every contribution is very welcome, and the standards
> are not meant to be obstacles for the sake of obstacles, they are
> meant to keep the quality high enough for us all to be proud of what
> we achieved.

No one will argue with it, at least in abstract.

> You make it sound as if asking the developer of a package, e.g., to
> reformat the doc strings so that the first line is a complete sentence
> somehow means the whole package is garbage.

"incoherent packages" were your words.

Are we discussing docstrings now? The bulk of the preceding discussion 
had to do with either copyright assignment or function naming (e.g. 
using the s- prefix).

I am confident that if we just ask Magnar to keep the docstrings up to 
standard, it will be done without much fuss.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > At least some contributors have certain expectations on gratitude that 
  > don't end with "thanks", and include a certain degree of autonomy. Like 
  > having the last word in a technical discussion.

Strict autonomy for the developer of one part of a software package
(GNU Emacs in this case) is incompatible, inherently, with doing a
good job of maintaining the whole package.

What we can do, in practice, is leave it to the contributor to decide
nearly all the technical decisions of per part of the package,
except for the occasional issue where we can't.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 14.05.2020 08:09, Richard Stallman wrote:
>    > At least some contributors have certain expectations on gratitude that
>    > don't end with "thanks", and include a certain degree of autonomy. Like
>    > having the last word in a technical discussion.
> 
> Strict autonomy for the developer of one part of a software package
> (GNU Emacs in this case) is incompatible, inherently, with doing a
> good job of maintaining the whole package.
> 
> What we can do, in practice, is leave it to the contributor to decide
> nearly all the technical decisions of per part of the package,
> except for the occasional issue where we can't.

Well, yes. There have to be some limits (say, borne out of ethical 
considerations).

But we'd do well not to intrude on technical autonomy without good reason.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Perhaps, if the people doing this work saw that their work was
  > gratefully accepted, and allowed to continue doing it with the minimum
  > of fuss and not too much design by committee, then they would continue
  > doing it.

If they coordinated with us in designing it, to make sure we could add
it to Emacs or GNU ELPA without causing a mess, we would want to
encourage them to continue doing it.

Another message today  explains what the mess is.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> It won't stay hypothetical if we allow incoherent packages into ELPA
> and start accepting their use in other packages and eventually in core.

We can simply refuse to incorporate `s.el` into core and then any
package which wants to be in core will first have to sop using `s.el`.

Accepting `s.el` into GNU ELPA does not mean we will accept it
into core.  Emacs and GNU ELP are both under our control, but we don't
need to (and we don't) apply the same rules to the two.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rms@gnu.org,  joostkremers@fastmail.fm,  ams@gnu.org,
>   phillip.lord@russet.org.uk,  pcr910303@icloud.com,  Emacs-devel@gnu.org
> Date: Tue, 12 May 2020 14:42:09 -0400
> 
> > It won't stay hypothetical if we allow incoherent packages into ELPA
> > and start accepting their use in other packages and eventually in core.
> 
> We can simply refuse to incorporate `s.el` into core and then any
> package which wants to be in core will first have to sop using `s.el`.
> 
> Accepting `s.el` into GNU ELPA does not mean we will accept it
> into core.  Emacs and GNU ELP are both under our control, but we don't
> need to (and we don't) apply the same rules to the two.

Do we have a mechanism to declare that a package is not intended to be
brought into core, unless changed to follow the same standards and
guidelines as the core does?  If not, can we come up with such a
mechanism?  A package that is thus declared can then be exempt from
some of the requirements (we still need to agree on which ones,
though).

maybe such packages should be in a separate subdirectory of ELPA.
Unless that unduly complicates management or package.el.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> Do we have a mechanism to declare that a package is not intended to be
> brought into core, unless changed to follow the same standards and
> guidelines as the core does?

Being a GNU ELPA package does not mean "intends to be brought into core".
Never has, never will.

There are packages in GNU ELPA which I'd like to see included in Emacs's
tarball (either by moving them to emacs.git or by bundling them when
creating the tarball).  Currently, these are just plans, tho.

> If not, can we come up with such a mechanism?

I guess we could.  We'd first have to clarify exactly how this mechanism
would be used in order to know how to design it.  But at least we could
start by just adding a special header `On-The-Way-To-Emacs: yes` for
those packages which we "intend" to include in the tarball.

> A package that is thus declared can then be exempt from some of the
> requirements (we still need to agree on which ones, though).

If we only label those rare ones for which we do have plans to integrate
them, we don't need to "agree first" since in case of disagreement the
package just won't be integrated.

Personally I don't see much benefit in such labeling: the way I expect
it to work is rather:
- Shouldn't we include SuperFoo into the tarball?
- Oh, yes, good idea.  Let's see ... is it in a good enough shape?
- Almost, we just have to fix this and that.
- OK, let's do that.
- [time passes]
- Alright, now this and that has been fixed, let's include it.
- Great, thanks, done.
No labeling involved.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rms@gnu.org,  joostkremers@fastmail.fm,  ams@gnu.org,
>   phillip.lord@russet.org.uk,  pcr910303@icloud.com,  Emacs-devel@gnu.org
> Date: Tue, 12 May 2020 15:50:52 -0400
> 
> > Do we have a mechanism to declare that a package is not intended to be
> > brought into core, unless changed to follow the same standards and
> > guidelines as the core does?
> 
> Being a GNU ELPA package does not mean "intends to be brought into core".
> Never has, never will.

Others seem to think otherwise.  So maybe we should begin even farther
back: by defining what it means to be a GNU ELPA package that is
considered "compatible with core".

Here's my definition: it is a package which we can move in or out of
core whenever we like, and/or distribute it as part of an Emacs
release, whether the Emacs source tarball or any auxiliary tarballs
that are considered part of an official Emacs distribution.

> > A package that is thus declared can then be exempt from some of the
> > requirements (we still need to agree on which ones, though).
> 
> If we only label those rare ones for which we do have plans to integrate
> them, we don't need to "agree first" since in case of disagreement the
> package just won't be integrated.

I think we do need to agree up front.  The reason is simple: a package
that will never be "core-compatible" is of no special interest to us.
When accepting it, we should only verify several simple
technicalities.  We don't need to pay any attention to later changes,
we don't need to check its sources for use of any deprecated APIs, we
don't need to be bothered by its documentation and by its being
consistent with the rest of our conventions -- because our code will
never directly call any of that package's APIs.  In effect, we are
just hosting the package in ELPA so that we could freely recommend it
to our users and to other 3rd-party packages, and make package.el
install it without any non-default configuration.

That is not so for packages that we would decide to keep "compatible".

> Personally I don't see much benefit in such labeling: the way I expect
> it to work is rather:
> - Shouldn't we include SuperFoo into the tarball?
> - Oh, yes, good idea.  Let's see ... is it in a good enough shape?
> - Almost, we just have to fix this and that.
> - OK, let's do that.
> - [time passes]
> - Alright, now this and that has been fixed, let's include it.
> - Great, thanks, done.
> No labeling involved.

This is not a good idea, IMO.  It would mean that deciding to include
a package in a tarball or even optionally call from our sources would
mean we need at that time to do all the review work we didn't before,
which is a much larger job.  More so if we are doing this with several
packages at once, due to their dependencies.  It is much easier to do
this piecemeal over many moons.

We would also need to ask the package developers to make all those
adjustments at once, and they could rightfully ask us why we suddenly
come up with all those requests when the package already happily lives
several years on ELPA in its present form.  It would be hard to come
up with a good answer to that.

So no, your proposal doesn't really work for me, sorry.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

>> > Do we have a mechanism to declare that a package is not intended to be
>> > brought into core, unless changed to follow the same standards and
>> > guidelines as the core does?
>> Being a GNU ELPA package does not mean "intends to be brought into core".
>> Never has, never will.
> Others seem to think otherwise.

Then they should look at what GNU ELPA contains.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Others seem to think otherwise.  So maybe we should begin even farther
  > back: by defining what it means to be a GNU ELPA package that is
  > considered "compatible with core".

  > Here's my definition: it is a package which we can move in or out of
  > core whenever we like, and/or distribute it as part of an Emacs
  > release, whether the Emacs source tarball or any auxiliary tarballs
  > that are considered part of an official Emacs distribution.

If we want to define "compatible with core" (i.e., ready to be moved
to the core if we wish), that is a good definition.

The crucial policy decision is the next step: under what circumstances
should we insist that a GNU ELPA package be compatible with core, as a
condition of admiting it to GNU ELPA?

  > > Personally I don't see much benefit in such labeling: the way I expect
  > > it to work is rather:
  > > - Shouldn't we include SuperFoo into the tarball?
  > > - Oh, yes, good idea.  Let's see ... is it in a good enough shape?
  > > - Almost, we just have to fix this and that.
  > > - OK, let's do that.
  > > - [time passes]
  > > - Alright, now this and that has been fixed, let's include it.
  > > - Great, thanks, done.

It would be unwise to put ourselves in this position.
Delaying before handling the papers will make it harder,
perhaps impossible.

(I explained the reasons earlier this week.)

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Do we have a mechanism to declare that a package is not intended to be
  > brought into core, unless changed to follow the same standards and
  > guidelines as the core does?  If not, can we come up with such a
  > mechanism?  A package that is thus declared can then be exempt from
  > some of the requirements (we still need to agree on which ones,
  > though).

For that purpose, we need to determine that we _certainly_ don't want
to bring a certain package into the core.  If we made this decision
for package P, we could forego assignments for P.

The hard part is not _saying_ that, but how we would
decide about that conclusion.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> For that purpose, we need to determine that we _certainly_ don't want
> to bring a certain package into the core.  If we made this decision
> for package P, we could forego assignments for P.

I'd estimate that at least ~95% of current GNU ELPA packages will never
make it to emacs.git.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > For that purpose, we need to determine that we _certainly_ don't want
  > > to bring a certain package into the core.  If we made this decision
  > > for package P, we could forego assignments for P.

  > I'd estimate that at least ~95% of current GNU ELPA packages will never
  > make it to emacs.git.

That may be true, but it is a different question.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: monnier@iro.umontreal.ca, joostkremers@fastmail.fm,
> 	Emacs-devel@gnu.org, ams@gnu.org, pcr910303@icloud.com,
> 	phillip.lord@russet.org.uk
> Date: Wed, 13 May 2020 00:07:16 -0400
> 
> For that purpose, we need to determine that we _certainly_ don't want
> to bring a certain package into the core.  If we made this decision
> for package P, we could forego assignments for P.
> 
> The hard part is not _saying_ that, but how we would
> decide about that conclusion.

I don't think it's hard, it basically boils down to whether the
package developer(s) will be willing to work with us on fixing the
minor issues that we find when the package is first considered for
addition.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We can simply refuse to incorporate `s.el` into core and then any
  > package which wants to be in core will first have to sop using `s.el`.

  > Accepting `s.el` into GNU ELPA does not mean we will accept it
  > into core.  Emacs and GNU ELP are both under our control, but we don't
  > need to (and we don't) apply the same rules to the two.

Keeping s.el out of the core would not avoid the problem that s.el causes.
Simply having it in GNU ELPA causes the problems.

The problems are that (1) we have two incogruent series of string
functions, and (2) we are compelled to cede control of the s-... name
space to people who con't coordinate with us.

Even in the core, we would have to treat the s-... name space as
reserved simply because "many packages" use s.el.

The only way I can see to fix this problem is with symbol renamings.
If we fix it that way, we could have s.el in GNU ELPA and even in the core.
At least I think so.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > We can simply refuse to incorporate `s.el` into core and then any
>   > package which wants to be in core will first have to sop using `s.el`.
>
>   > Accepting `s.el` into GNU ELPA does not mean we will accept it
>   > into core.  Emacs and GNU ELP are both under our control, but we don't
>   > need to (and we don't) apply the same rules to the two.
>
> Keeping s.el out of the core would not avoid the problem that s.el causes.
> Simply having it in GNU ELPA causes the problems.
>
> The problems are that (1) we have two incogruent series of string
> functions, and (2) we are compelled to cede control of the s-... name
> space to people who con't coordinate with us.
>
> Even in the core, we would have to treat the s-... name space as
> reserved simply because "many packages" use s.el.
>
> The only way I can see to fix this problem is with symbol renamings.
> If we fix it that way, we could have s.el in GNU ELPA and even in the core.
> At least I think so.

1) Incongruent functions: A situation we already have with deprecated
functions, function aliases, and things like seq which overlaps with
other emacs functions (albeit somewhat differently).

2) The s- namespace: This has already happened. If we put a function
called `s-starts-with' or `-map' into core, we will break quite a lot of
MELPA. This would look poor and potentially like bad will.

3) "who don't coordinate with us". Magnar will respond if emailed. Is
reading this mailing list really a criteria that makes sense.


And you have missed the advantages: having libraries like s and dash
available, will bring to Emacs a set of functionality that many people
like, as well as bringing in a lot of developers who have already
contributed to s.el, dash and the rest.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   1) Incongruent functions: A situation we already have with
   deprecated functions, function aliases, and things like seq which
   overlaps with other emacs functions (albeit somewhat differently).

Deprecated functions are marked as such, function aliases as
well. Some functions, specifically in s.el, could have been aliases
but are instead fresh implementations.  It is definitly not the same
situation.

And don't you think that if there are badly named, or overlapping
functions (in the context of overall Emacs, in isolation they might be
well named), then instead of continuting adding more of the same
situation, that one should instead try and fix it?

   2) The s- namespace: This has already happened. If we put a function
   called `s-starts-with' or `-map' into core, we will break quite a lot of
   MELPA. This would look poor and potentially like bad will.

Emacs cannot control what happens outside of Emacs -- if the library
was called string-x.el and was super popular, and the prefix was
string- the situation would have been exactly the same.  Adding a
function would break code, Emacs cannot control that.

   3) "who don't coordinate with us". Magnar will respond if
   emailed. Is reading this mailing list really a criteria that makes
   sense.

That is asking Emacs for coordinating with someone else.

   And you have missed the advantages: having libraries like s and
   dash available, will bring to Emacs a set of functionality that
   many people like, as well as bringing in a lot of developers who
   have already contributed to s.el, dash and the rest.

And nobody is against adding functionality -- the issue is adding many
overlapping, slightly similar, differently named functions.  It is not
the functionality that is the topic here.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > 2) The s- namespace: This has already happened. If we put a function
  > called `s-starts-with' or `-map' into core, we will break quite a lot of
  > MELPA. This would look poor and potentially like bad will.

That would be blaming us for their treating us badly.

I suppose some people might do that -- but I don't want to
cringe in fear of it.

  > 3) "who don't coordinate with us". Magnar will respond if emailed.

It can't be our responsibility to contact everyone working on a
package that might be useful.  We don't know who they are, or what
they are working on, until they release it.

The only way that can work is if they contact us.

  > And you have missed the advantages: having libraries like s and dash
  > available, will bring to Emacs a set of functionality that many people
  > like, as well as bringing in a lot of developers who have already
  > contributed to s.el, dash and the rest.

This is not the way we want contributions to arrive.



-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 14.05.2020 08:12, Richard Stallman wrote:
>    > 2) The s- namespace: This has already happened. If we put a function
>    > called `s-starts-with' or `-map' into core, we will break quite a lot of
>    > MELPA. This would look poor and potentially like bad will.
> 
> That would be blaming us for their treating us badly.
> 
> I suppose some people might do that -- but I don't want to
> cringe in fear of it.

Regardless of who is to blame, doing that on our part would be 
counter-productive. So we're very unlikely to even do it.

Thus, the "s" namespace is unfortunately already taken.

If Elisp grew to support namespaces, that could be fixed, of course.

RE: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Drew Adams]

> Thus, the "s" namespace is unfortunately already taken.

Is it?  What's to prevent some other 3rd-party
library, with whatever library name, using
prefix `s-'?

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 14.05.2020 20:23, Drew Adams wrote:
>> Thus, the "s" namespace is unfortunately already taken.
> Is it?  What's to prevent some other 3rd-party
> library, with whatever library name, using
> prefix `s-'?

The same considerations that usually apply when choosing a name that 
conflicts with an existing one.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

>> I'll just recommend people add MELPA to their `package-archives` and
>> move on.
> Please don't.

It's not like I have much of a choice,


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But it's a problem we can't solve, because the library is out there are
  > people use it.

That is not our resposibility or our problem.  But if we put s.el into
GNU ELPA, it will be our resposibility and our problem.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

>   > But it's a problem we can't solve, because the library is out there are
>   > people use it.
> That is not our resposibility or our problem.  But if we put s.el into
> GNU ELPA, it will be our resposibility and our problem.

We could choose to take it upon ourselves, yes, but adding it to GNU
ELPA does not *impose* this responsability on us any more than it is
already the case.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >   > But it's a problem we can't solve, because the library is out there are
  > >   > people use it.
  > > That is not our resposibility or our problem.  But if we put s.el into
  > > GNU ELPA, it will be our resposibility and our problem.

  > We could choose to take it upon ourselves,

Including a package in GNU ELPA implies we invite people to
report problems with it to us, and expect us to fix them.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> Including a package in GNU ELPA implies we invite people to
> report problems with it to us, and expect us to fix them.

No, that is simply not the case.

When people find a bug in a package, they don't instinctively report it
to the place from where they installed it.

We do *accept* bug reports to GNU ELPA packages in our debbugs instance,
and we occasionally invite people to submit them there, but in my
experience people do not expect us to accept bug reports and even less
to fix those bugs.

Most people (somewhat wrongly) think of GNU ELPA as a *distribution*
site for third party packages, like MELPA.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 191 bytes --]

>
> Most people (somewhat wrongly) think of GNU ELPA as a *distribution*
> site for third party packages, like MELPA.


I just wanted to confirm that it's indeed the case for a lot of users.

[-- Attachment #2: Type: text/html, Size: 402 bytes --]

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Thu, 14 May 2020 17:28:24 +0200
> Cc: Richard Stallman <rms@gnu.org>, Joost Kremers <joostkremers@fastmail.fm>,
>  Emacs developers <Emacs-devel@gnu.org>, "Alfred M. Szmidt" <ams@gnu.org>,
>  조성빈 <pcr910303@icloud.com>,
>  Phillip Lord <phillip.lord@russet.org.uk>
> 
>  Most people (somewhat wrongly) think of GNU ELPA as a *distribution*
>  site for third party packages, like MELPA.
> 
> I just wanted to confirm that it's indeed the case for a lot of users.

It sounds like the fundamental disagreements regarding what ELPA is
and why it exists are even wider than I thought.  The more we discuss
this, the wider it seems to become.  It's little wonder we cannot
agree on more complex issues wrt ELPA when something so basic is
subject to controversy.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 14.05.2020 18:28, Philippe Vaucher wrote:
>     Most people (somewhat wrongly) think of GNU ELPA as a *distribution*
>     site for third party packages, like MELPA.
> 
> 
> I just wanted to confirm that it's indeed the case for a lot of users.

And [at least some] contributors.

What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >     Most people (somewhat wrongly) think of GNU ELPA as a *distribution*
  > >     site for third party packages, like MELPA.
  > > 
  > > 
  > > I just wanted to confirm that it's indeed the case for a lot of users.

What do we say now to explain what GNU ELPA is?
And where do we say it?
Will users see that explanation
in the normal workflow of using packages from GNU ELPA?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Dmitry Gutov]

On 15.05.2020 06:19, Richard Stallman wrote:
> What do we say now to explain what GNU ELPA is?
> And where do we say it?

On ELPA's home page: https://elpa.gnu.org/

   GNU Emacs Lisp Package Archive

   The default package repository for GNU Emacs.

And efaq.texi:

   The easiest way to add more features to your Emacs is to use the
command @kbd{M-x list-packages}.  This contacts the
@uref{https://elpa.gnu.org, GNU ELPA} (``Emacs Lisp Package Archive'')
server and fetches the list of additional packages that it offers.
These are GNU packages that are available for use with Emacs, but are
distributed separately from Emacs itself, for reasons of space, etc.
You can browse the resulting @file{*Packages*} buffer to see what is
available, and then Emacs can automatically download and install the
packages that you select.  @xref{Packages,,, emacs, The GNU Emacs Manual}.

It also says:

   There are other, non-GNU, Emacs Lisp package servers, including:
@uref{https://melpa.org, MELPA}; and
@uref{https://marmalade-repo.org, Marmalade}.  To use additional
package servers, customize the @code{package-archives} variable.  Be
aware that installing a package can run arbitrary code, so only add
sources that you trust.

...apparently implying that the user can trust GNU ELPA.

> Will users see that explanation
> in the normal workflow of using packages from GNU ELPA?

I'm not sure they need to. But the FAQ explanation is pretty good.

Re: What is GNU ELPA?

[Jean-Christophe Helary]

> On May 15, 2020, at 12:46, Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
>  There are other, non-GNU, Emacs Lisp package servers, including:
> @uref{https://melpa.org, MELPA}; and
> @uref{https://marmalade-repo.org, Marmalade}.

I thought marmalade was not updated anymore ?

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: What is GNU ELPA?

[Dmitry Gutov]

On 15.05.2020 07:00, Jean-Christophe Helary wrote:
>> On May 15, 2020, at 12:46, Dmitry Gutov<dgutov@yandex.ru>  wrote:
>>
>>   There are other, non-GNU, Emacs Lisp package servers, including:
>> @uref{https://melpa.org, MELPA}; and
>> @uref{https://marmalade-repo.org, Marmalade}.
> I thought marmalade was not updated anymore ?

Indeed. Good catch.

Re: What is GNU ELPA?

[Stefan Monnier]

> aware that installing a package can run arbitrary code, so only add
> sources that you trust.
>
> ...apparently implying that the user can trust GNU ELPA.

Strictly speaking it only implies you can trust it to be safe to
*install* a GNU ELPA package, but not necessarily to run it.

I had no recollection of this text, but indeed, I try to keep on eye on
GNU ELPA to make sure *installation* is harmless (not only in terms of
safety but also annoyance).  I wouldn't guarantee it is harmless, but
that's the part that I do try and make an effort to check.


        Stefan

Re: What is GNU ELPA?

[Alfred M. Szmidt]

      There are other, non-GNU, Emacs Lisp package servers, including:
   @uref{https://melpa.org, MELPA}; and
   @uref{https://marmalade-repo.org, Marmalade}.  To use additional
   package servers, customize the @code{package-archives} variable.  Be
   aware that installing a package can run arbitrary code, so only add
   sources that you trust.

This should probobly be removed, since melpa contains software that
recommends non-free software.

Re: What is GNU ELPA?

[Howard Melman]

ams@gnu.org (Alfred M. Szmidt) writes:

>       There are other, non-GNU, Emacs Lisp package servers, including:
>    @uref{https://melpa.org, MELPA}; and
>    @uref{https://marmalade-repo.org, Marmalade}.  To use additional
>    package servers, customize the @code{package-archives} variable.  Be
>    aware that installing a package can run arbitrary code, so only add
>    sources that you trust.
>
> This should probobly be removed, since melpa contains software that
> recommends non-free software.

I have an alternative approach. What if instead of removing
the text you explained the rationale?

      There are other, non-GNU, Emacs Lisp package servers which
      you can access by customizing the @code{package-archives}
      variable.  Be aware that installing a package can run
      arbitrary code, so only add sources that you trust. A very
      popular server is @uref{https://melpa.org, MELPA} but note
      that it contains some software that recommends non-free
      software.

This allows people to find additional popular emacs
packages, including truly free ones that are on melpa and
not elpa and cautions them about finding non-free software,
raising the issue for those that might not fully understand
it. This seems better to me than having emacs remain mute on
the subject.

-- 

Howard

Re: What is GNU ELPA?

[Alfred M. Szmidt]

In the GNU project we don't link to sites that recommend, or could
cause a user to install non-free software.  Linking to such a site
would be self defeating -- the whole point is to make such software
irrelevant.

Not staying silent on the subject seems like a good idea, but then one
can do so without linking to any offending site, and instead explain
why non-free software is bad.

Re: What is GNU ELPA?

[Dmitry Gutov]

On 15.05.2020 18:08, Howard Melman wrote:
> A very
>        popular server is @uref{https://melpa.org, MELPA} but note
>        that it contains some software that recommends non-free
>        software.

This sounds very circumspect. As a user, *I* don't have to fear 
something recommending me something. Or else that would imply some 
ostrich-like behavior. Those who avoid proprietary software, take care 
to do it. Those who don't, don't care.

If we must say something, let's try factual: some packages in there 
depend on non-free software being installed. Even so, that sounds 
unnecessarily negative to me, given that (by some very rough estimate) 
the fraction of such packages there is <5%.

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

The basic idea of the free software movement is that nonfree software
is an injustice and it should not exist.  We tell people it is a problem
we aim to eliminate.

We take this issue very seriously, so we have a rule not to say
anything that might suggest the use of nonfree software.  We don't
mention the existence of a program if it can't be used in the Free
World.  See the GNU Coding standards in gnu.org/prep/standards, node
References.

That rule is needed because if we ever recommended a nonfree program
it would cast doubt on our commitment to our principles.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Dmitry Gutov]

On 18.05.2020 06:46, Richard Stallman wrote:
> We take this issue very seriously, so we have a rule not to say
> anything that might suggest the use of nonfree software.  We don't
> mention the existence of a program if it can't be used in the Free
> World.  See the GNU Coding standards in gnu.org/prep/standards, node
> References.

But the situation here is twice removed. MELPA itself is libre software. 
All packages there are libre as well. Some small minority integrate with 
non-free software, but that is it.

We don't shy away from mentioning e.g. MS Windows, and the vast majority 
of software for it is proprietary.

Re: What is GNU ELPA?

[Alfred M. Szmidt]

   > We take this issue very seriously, so we have a rule not to say
   > anything that might suggest the use of nonfree software.  We
   > don't mention the existence of a program if it can't be used in
   > the Free World.  See the GNU Coding standards in
   > gnu.org/prep/standards, node References.

   But the situation here is twice removed. MELPA itself is libre
   software.  All packages there are libre as well. Some small
   minority integrate with non-free software, but that is it.

And that is enough to cause a problem, there are several programs that
are free software which are not included in the Free Software
Directory for example because of that reason.

Another issue is that MELPA is it is hosted on Gitblob, that requires
users to run non-free Javascript, that is another problem making it
unsuitable to reference in any GNU website.

   We don't shy away from mentioning e.g. MS Windows, and the vast
   majority of software for it is proprietary.

When we mention Windows, we mention it in passing, we don't give it
legitimacy.

This is all well explained in
https://www.gnu.org/prep/standards/standards.html#References .

Re: What is GNU ELPA?

[Dmitry Gutov]

On 18.05.2020 16:01, Alfred M. Szmidt wrote:
> Another issue is that MELPA is it is hosted on Gitblob, that requires
> users to run non-free Javascript, that is another problem making it
> unsuitable to reference in any GNU website.

Using the MELPA archives doesn't require the users to run any 
JavaScript, as long as they add it as package repository in Emacs.

MELPA's JavaScript is entirely libre: https://melpa.org/jslicense.html

And its sources hosted on Github has no practical effect on the end users.

Re: What is GNU ELPA?

[Arthur Miller]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> The basic idea of the free software movement is that nonfree software
> is an injustice and it should not exist.  We tell people it is a problem
> we aim to eliminate.
>
> We take this issue very seriously, so we have a rule not to say
> anything that might suggest the use of nonfree software.  We don't
> mention the existence of a program if it can't be used in the Free
> World.  See the GNU Coding standards in gnu.org/prep/standards, node
> References.
>
> That rule is needed because if we ever recommended a nonfree program
> it would cast doubt on our commitment to our principles.

I definitely agree with you RMS. I have big respect for you, as a
programmer and a philosopher, and I have a question for you:
What if non-free software does not have an alternative? Or there is one,
but most people don't use it anyway. Or if the software itself is free,
but it is on some company servers, like say gitlab. 

People choose to use non-free services (programs, hardware,
prenumerations, etc) all the time. Yes, often they don't have choice
because market powers have understand that by limiting peoples choice,
they will force them to choose more expensive or worse alternative.
Sure.

But if we look at OS arena, nowdays there is very good free choice,
yet it has very small share of the market. Why? Faimilarity? Latency?
Laziness? I dont' know. Maybe this will change with milleinal generation
that is used to different guis and interaction models then Windows or
Mac, but there is still something more to it. Milleinals, at least here
in Sweden, love their Macs. At university I see almost every student
running around with a Mac. Why people wish to pay double the price for
the hardware with inferior OS that invades on their privacy? I have no
idea. Pretty looks, status symbol, fashion/conformancy to group? I have
no answer myself, it is just a reflection over what I see.

The world is as it is, not as it should be (in my eyes or your eyes). I
think one has to go where people are if one wish to engage them, not to
wait for people to come over.

Off topic:
Another scary scenario is when techonology is used to control people. If
Yuval Harari is correct in his analysis ("21 Lessons for 21 century",
"Homo Deus"), then there sure might be very good reasons to fear
any software that isn't open (open not free) and built by one self. But
openess and/or freeness of the softwre is not the guarantee from the
opression. When opression becomes legal, i.e. when opressive political
forces take over, they might force people to use whatever technoology
they find needed to exercise the control, regardless if it is for a good
cause like protecting people from getting Covid-19 or for hindering
people to express and exercise their free will.

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What if non-free software does not have an alternative? Or there is one,
  > but most people don't use it anyway. 

We never refer people to nonfree software -- regardless of the details
of the situation.  If we can't recommend people a free method to do
something, we recommend no method.

If you look at directory.fsf.org, you will see we recommend thousands
of free programs, and ZERO nonfree programs.

					 Or if the software itself is free,
  > but it is on some company servers, like say gitlab. 

That is no problem.  If people can download it from place X without
running nonfree software, we can suggest they download it from place
X.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Also, there are twoo many discussions going on here.  Please
understand that I don't want to start yet another.

The general question of how to promote free software, and what the
obstacles are, is off topic on emacs-devel.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I have an alternative approach. What if instead of removing
  > the text you explained the rationale?

We can explan more, but that will make it even more crucial to
practice what we preach.  This principle is fundamental:
we do not steer people towards nonfree software.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 15 May 2020 06:46:02 +0300
> Cc: joostkremers@fastmail.fm, ams@gnu.org, Emacs-devel@gnu.org,
>  monnier@iro.umontreal.ca, pcr910303@icloud.com, phillip.lord@russet.org.uk
> 
> the FAQ explanation is pretty good.

Except that they fly in the face of what Stefan said and what "the
majority of users" out there think.  Why is that?

Re: What is GNU ELPA?

[Dmitry Gutov]

On 15.05.2020 10:06, Eli Zaretskii wrote:
>> From: Dmitry Gutov<dgutov@yandex.ru>
>> Date: Fri, 15 May 2020 06:46:02 +0300
>> Cc:joostkremers@fastmail.fm,ams@gnu.org,Emacs-devel@gnu.org,
>>   monnier@iro.umontreal.ca,pcr910303@icloud.com,phillip.lord@russet.org.uk
>>
>> the FAQ explanation is pretty good.
> Except that they fly in the face of what Stefan said and what "the
> majority of users" out there think.  Why is that?

Could you clarify what you mean? Is it that passage?

   GNU packages that are available for use with Emacs, but are
distributed separately from Emacs itself, for reasons of space, etc

They are indeed "available to use" and "distributed separately".

Re: What is GNU ELPA?

[Eli Zaretskii]

> Cc: rms@gnu.org, joostkremers@fastmail.fm, ams@gnu.org, Emacs-devel@gnu.org,
>  monnier@iro.umontreal.ca, pcr910303@icloud.com, phillip.lord@russet.org.uk
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 15 May 2020 15:25:59 +0300
> 
> > Except that they fly in the face of what Stefan said and what "the
> > majority of users" out there think.  Why is that?
> 
> Could you clarify what you mean? Is it that passage?
> 
>    GNU packages that are available for use with Emacs, but are
> distributed separately from Emacs itself, for reasons of space, etc
> 
> They are indeed "available to use" and "distributed separately".

But only "for reasons of space, etc.".

Re: What is GNU ELPA?

[Dmitry Gutov]

On 15.05.2020 16:40, Eli Zaretskii wrote:
>> Cc:rms@gnu.org,joostkremers@fastmail.fm,ams@gnu.org,Emacs-devel@gnu.org,
>>   monnier@iro.umontreal.ca,pcr910303@icloud.com,phillip.lord@russet.org.uk
>> From: Dmitry Gutov<dgutov@yandex.ru>
>> Date: Fri, 15 May 2020 15:25:59 +0300
>>
>>> Except that they fly in the face of what Stefan said and what "the
>>> majority of users" out there think.  Why is that?
>> Could you clarify what you mean? Is it that passage?
>>
>>     GNU packages that are available for use with Emacs, but are
>> distributed separately from Emacs itself, for reasons of space, etc
>>
>> They are indeed "available to use" and "distributed separately".
> But only "for reasons of space, etc.".

There are multiple ways to interpret that, indeed.

So we should decide on what GNU ELPA means to us, a then clarify that "etc".

Re: What is GNU ELPA?

[Stefan Monnier]

> So we should decide on what GNU ELPA means to us, a then clarify that "etc".

AFAIK the defining feature of GNU ELPA is that it's in the default value
of `package-archives`.


        Stefan

Re: What is GNU ELPA?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  rms@gnu.org,  joostkremers@fastmail.fm,
>   ams@gnu.org,  Emacs-devel@gnu.org,  pcr910303@icloud.com,
>   phillip.lord@russet.org.uk
> Date: Fri, 15 May 2020 11:11:47 -0400
> 
> > So we should decide on what GNU ELPA means to us, a then clarify that "etc".
> 
> AFAIK the defining feature of GNU ELPA is that it's in the default value
> of `package-archives`.

IMO, that's the most extreme non-definition that we could come up
with.

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >    GNU packages that are available for use with Emacs, but are
  > > distributed separately from Emacs itself, for reasons of space, etc
  > > 
  > > They are indeed "available to use" and "distributed separately".

  > But only "for reasons of space, etc.".

As I understand it, space is the only reason that we don't
put all those packages in the core.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: dgutov@yandex.ru, joostkremers@fastmail.fm, ams@gnu.org,
> 	Emacs-devel@gnu.org, monnier@iro.umontreal.ca,
> 	pcr910303@icloud.com, phillip.lord@russet.org.uk
> Date: Sat, 16 May 2020 00:16:07 -0400
> 
>   > >    GNU packages that are available for use with Emacs, but are
>   > > distributed separately from Emacs itself, for reasons of space, etc
>   > > 
>   > > They are indeed "available to use" and "distributed separately".
> 
>   > But only "for reasons of space, etc.".
> 
> As I understand it, space is the only reason that we don't
> put all those packages in the core.

Perhaps not the only one, but all the others are of the same nature:
boring mechanical issues, not some fundamental incompatibilities, let
alone ideological problems.

Re: What is GNU ELPA?

[Dmitry Gutov]

On 16.05.2020 10:08, Eli Zaretskii wrote:
> Perhaps not the only one, but all the others are of the same nature:
> boring mechanical issues, not some fundamental incompatibilities, let
> alone ideological problems.

Nothing ideological, but there are organizational reasons for sure: 
maybe not everybody, but some authors prefer to develop outside of the 
core (and use other hosting platforms for that), and use a different bug 
tracker.

I, for one, am I no hurry to put all my packages into the core.

Re: What is GNU ELPA?

[Dmitry Gutov]

Sorry,

On 16.05.2020 12:05, Dmitry Gutov wrote:
> I, for one, am I no hurry to put all my packages into the core.

                  ^in

Re: What is GNU ELPA?

[Alfred M. Szmidt]

     > >    GNU packages that are available for use with Emacs, but are
     > > distributed separately from Emacs itself, for reasons of space, etc
     > > 
     > > They are indeed "available to use" and "distributed separately".

     > But only "for reasons of space, etc.".

   As I understand it, space is the only reason that we don't
   put all those packages in the core.

Wouldn't it mean that if is a mainly a space issue, then all the
things in ELPA should also be held to the same standards as the rest
of GNU Emacs?

Re: What is GNU ELPA?

[Stefan Monnier]

> As I understand it, space is the only reason that we don't
> put all those packages in the core.

AFAIK space was definitely not the motivation (GNU ELPA has grown a fair
bit since then, so maybe nowadays we'd think twice before adding those
packages to emacs.git, but it's still not the main motivation, AFAICT).

The reasons were rather to:
- Decouple those packages from Emacs's release cycle.
- Avoid the never ending problems with packages that are in emacs.git
  but whose "upstream" is elsewhere, where the two code bases require
  extra efforts to keep them in sync (enough efforts that sometimes this
  sync is lost, as is the case for CEDET IIRC).
- More generally give more freedom to the authors (many authors are
  *not* interested in having their code in emacs.git because of the
  extra constraints that it implies).

The last two problems *also* affect GNU ELPA, ironically, tho to
a lesser extent.


        Stefan

Re: What is GNU ELPA?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  dgutov@yandex.ru,
>   joostkremers@fastmail.fm,  ams@gnu.org,  Emacs-devel@gnu.org,
>   pcr910303@icloud.com,  phillip.lord@russet.org.uk
> Date: Sat, 16 May 2020 08:48:55 -0400
> 
> > As I understand it, space is the only reason that we don't
> > put all those packages in the core.
> 
> AFAIK space was definitely not the motivation

It says "space etc.", so not just space.

> The reasons were rather to:
> - Decouple those packages from Emacs's release cycle.

That'd fall under the "etc." part in my book, because the package
still logically is part of Emacs, just being released separately.

> - Avoid the never ending problems with packages that are in emacs.git
>   but whose "upstream" is elsewhere, where the two code bases require
>   extra efforts to keep them in sync (enough efforts that sometimes this
>   sync is lost, as is the case for CEDET IIRC).

For that, we could use the same technique as we use in ELPA for
"external" packages, could we not?

> - More generally give more freedom to the authors (many authors are
>   *not* interested in having their code in emacs.git because of the
>   extra constraints that it implies).
> 
> The last two problems *also* affect GNU ELPA, ironically, tho to
> a lesser extent.

I'm sorry to say, but I feel we are trying to retroactively invent
reasons to justify the actual situation and the goals that got shifted
from their original.  As the description of ELPA shows, the original
goal was to have an extension of Emacs that is separate for minor
technical reasons.

Re: What is GNU ELPA?

[Dmitry Gutov]

Hi Eli,

On 16.05.2020 17:43, Eli Zaretskii wrote:

>> - More generally give more freedom to the authors (many authors are
>>    *not* interested in having their code in emacs.git because of the
>>    extra constraints that it implies).
>>
>> The last two problems *also* affect GNU ELPA, ironically, tho to
>> a lesser extent.
> 
> I'm sorry to say, but I feel we are trying to retroactively invent
> reasons to justify the actual situation and the goals that got shifted
> from their original.  As the description of ELPA shows, the original
> goal was to have an extension of Emacs that is separate for minor
> technical reasons.

You seem to be trying to produce logical rules for governing ELPA based 
on two lines from the manual written by Glenn likely on a whim 4 years 
after GNU ELPA was created.

And does it really matter why it was created originally? Perhaps it was 
made to fight forest fires or raise awareness about the surveillance 
state. Does that really matter now, 10 years later?

GNU ELPA is where it is because it turned out useful enough for both 
sides (users and package authors) that the latter contributed a 
considerable number of packages, and a significant fraction of them 
continues to be updated. Making the situation worse for either side can 
break the balance, and there will be fewer updates, as a result it will 
be less useful for the users, and in the end we could simply be forced 
to write it off as a failure.

The current situation is, it is okay-ish for the authors (for reasons 
Stefan described), though probably not ideal for minor technical 
reasons. As well a copyright assignment reasons, but that acts as a filter.

This filter, however, makes it not great for the users, because the 
result is, we provide less functionality out-of-the-box. Hopefully I 
don't have to explain why this is a problem.

On the authors' side, however, if you decide to dispense with some of 
the reasons people contribute to GNU ELPA, that can also lead to fewer 
packages in there, and the result for the users is predictable.

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'm sorry to say, but I feel we are trying to retroactively invent
  > reasons to justify the actual situation and the goals that got shifted
  > from their original.

I recall now that we have discussed these maintenance reasons to
choose between core Emacs and GNU ELPA before, for certain packages.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > the FAQ explanation is pretty good.

  > Except that they fly in the face of what Stefan said and what "the
  > majority of users" out there think.  Why is that?

If Stefan is right about this -- I don't know whether he is -- then it
would imply that our explanation is not reaching the enough users.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > And efaq.texi:

  >    There are other, non-GNU, Emacs Lisp package servers, including:
  > @uref{https://melpa.org, MELPA}; and
  > @uref{https://marmalade-repo.org, Marmalade}.  To use additional
  > package servers, customize the @code{package-archives} variable.  Be
  > aware that installing a package can run arbitrary code, so only add
  > sources that you trust.

We should not be recommending those to the public.  Would someone
please delete that text?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Eli Zaretskii]

On May 17, 2020 5:48:53 AM GMT+03:00, Richard Stallman <rms@gnu.org> wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>   > And efaq.texi:
> 
>  >    There are other, non-GNU, Emacs Lisp package servers, including:
>   > @uref{https://melpa.org, MELPA}; and
>   > @uref{https://marmalade-repo.org, Marmalade}.  To use additional
> > package servers, customize the @code{package-archives} variable.  Be
>  > aware that installing a package can run arbitrary code, so only add
>   > sources that you trust.
> 
> We should not be recommending those to the public.  Would someone
> please delete that text?

This text is obsolete.  The current text says:

"There are other, non-GNU, Emacs Lisp package servers, including:
MELPA and Marmalade.  To use additional package servers, customize
the 'package-archives' variable.  Be aware that installing a package can run
arbitrary code, so only add sources that you trust.  Also, packages
hosted on non-GNU package servers may encourage or require you
to install and use non-free software; for example, MELPA is known
to host some packages that do this."

This explicitly warns about the MELPA problem, and so a reference to it there is no more a recommendation to use it than a reference to Microsoft or MS-Windowd is a promotion of those two.

It makes very little sense to me to remove any and all references to the M-word, because doing so doesn't inform our users about these dangers, and thus implicitly exposes them to those dangers.

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Will users see that explanation
  > > in the normal workflow of using packages from GNU ELPA?

  > I'm not sure they need to.

We want them to know this, and it seems we are not reaching most of
them with this information.  What can we do to reach them better?

The idea that occurs to me is to make M-x load-package tell them,
occasionally.  Perhaps the first time each user runs it.
Perhaps once a year after that.

Any other suggestions?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Dmitry Gutov]

On 17.05.2020 05:48, Richard Stallman wrote:

>    > > Will users see that explanation
>    > > in the normal workflow of using packages from GNU ELPA?
> 
>    > I'm not sure they need to.
> 
> We want them to know this, and it seems we are not reaching most of
> them with this information.  What can we do to reach them better?
> 
> The idea that occurs to me is to make M-x load-package tell them,
> occasionally.  Perhaps the first time each user runs it.
> Perhaps once a year after that.

The first time it is run sounds fine to me. "A years later" sounds more 
puzzling. Although it all depends on what we actually want to write there.

> Any other suggestions?

If there was a more prominent place where Emacs introduced the users to 
ELPA (say, Getting Started guide, the tutorial, or the startup screen), 
some extra explanation could also be there.

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > The idea that occurs to me is to make M-x load-package tell them,
  > > occasionally.  Perhaps the first time each user runs it.
  > > Perhaps once a year after that.

  > The first time it is run sounds fine to me. "A years later" sounds more 
  > puzzling. Although it all depends on what we actually want to write there.

The idea is that it would somehow record the date when it shows you
the explanation.  For the following year, it would not show the
explanation again.  After that, the next use would show the
explanation again.  For the following year, it would not show the
explanation again.

The idea is to remind you occasionally but avoid doing that so often
that you would be annoyed.

  > If there was a more prominent place where Emacs introduced the users to 
  > ELPA (say, Getting Started guide, the tutorial, or the startup screen), 
  > some extra explanation could also be there.

I agree, mostly.

The explanation does not need to be in the startup screen, where there
is no space for it.  Rather, the startup screen would tell you to run
load-package, and load-package the first time would show you this
explanation.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Dmitry Gutov]

On 18.05.2020 06:53, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>    > > The idea that occurs to me is to make M-x load-package tell them,
>    > > occasionally.  Perhaps the first time each user runs it.
>    > > Perhaps once a year after that.
> 
>    > The first time it is run sounds fine to me. "A years later" sounds more
>    > puzzling. Although it all depends on what we actually want to write there.
> 
> The idea is that it would somehow record the date when it shows you
> the explanation.  For the following year, it would not show the
> explanation again.  After that, the next use would show the
> explanation again.  For the following year, it would not show the
> explanation again.

I got that.

> The idea is to remind you occasionally but avoid doing that so often
> that you would be annoyed.

Probably not too annoying, yes. But the info has to be really important 
that we'd tell it our users every year.

>    > If there was a more prominent place where Emacs introduced the users to
>    > ELPA (say, Getting Started guide, the tutorial, or the startup screen),
>    > some extra explanation could also be there.
> 
> I agree, mostly.
> 
> The explanation does not need to be in the startup screen, where there
> is no space for it.  Rather, the startup screen would tell you to run
> load-package, and load-package the first time would show you this
> explanation.

'M-x list-packages', you mean.

The startup screen might be too cramped, indeed. The other options could 
be better, though.

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > The idea is to remind you occasionally but avoid doing that so often
  > > that you would be annoyed.

  > Probably not too annoying, yes. But the info has to be really important 
  > that we'd tell it our users every year.

To be worth mentioning just once a year, it only needs to be
_important_.  (More precisely, important to us that they know.)

If it were _really important_, we would tell them every six months ;-).

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Yoni Rabkin]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > >     Most people (somewhat wrongly) think of GNU ELPA as a *distribution*
>   > >     site for third party packages, like MELPA.
>   > > 
>   > > 
>   > > I just wanted to confirm that it's indeed the case for a lot of users.
>
> What do we say now to explain what GNU ELPA is?
> And where do we say it?
> Will users see that explanation
> in the normal workflow of using packages from GNU ELPA?

In case it is an interesting data point, ELPA may be invisible and
un-discoverable to many Emacs users. Here is why I think this may be the
case:

I've been using Emacs for decades, and I use it every day for my living
(read: Emacs is crucial to me and my family). I have a few packages
installed in an ~/elisp/ directory and I load those manually via my
~/.emacs.

Over all of these years I have never had Emacs ask, offer, or otherwise
point out ELPA. The only reason I know it even exists is because as a
GNU maintainer of some emacs extensions, people sometimes asked my why
those extensions are not in ELPA. At that point, I needed to try and
figure out what ELPA even was (not to mention what MELPA was, and what
the difference is between them.)

This is good for me because it means that I have never been bothered by
Emacs telling me that I need to change my setup and use ELPA instead of
what I'm currently using. I want that to continue being the case, as I
do not anticipate ELPA ever being relevant to me personally. Emacs isn't
a hobby for me; it is a tool I use to earn a salary.

However, I am guessing that this may be bad news for people who love and
support ELPA, since it means that it is effectively invisible to many
emacs users like me. I certainly can't speak on the behalf of the ELPA
people, but perhaps they should be thinking about how to explain to
emacs users what ELPA is, or why we should even care. If they can show
me how ELPA can make Emacs better for my work, then ELPA may become very
important to me. Otherwise, I have no incentive to break my current
setup, which is precious to me, in order to try out something that may
or may not be better.

Please don't take this as criticism of ELPA or the efforts of the people
who work on it, as it is not intended to be one. Instead, I wanted to
share my real-world use of Emacs in relation to ELPA, in case it would
be a valuable data-point for the people who are considering the question
in the subject of the thread: "What is GNU ELPA?"


-- 
   "Cut your own wood and it will warm you twice"

Re: What is GNU ELPA?

[Dmitry Gutov]

On 17.05.2020 17:49, Yoni Rabkin wrote:
> In case it is an interesting data point, ELPA may be invisible and
> un-discoverable to many Emacs users. Here is why I think this may be the
> case:
> 
> I've been using Emacs for decades, and I use it every day for my living
> (read: Emacs is crucial to me and my family). I have a few packages
> installed in an ~/elisp/  directory and I load those manually via my
> ~/.emacs.
> 
> Over all of these years I have never had Emacs ask, offer, or otherwise
> point out ELPA. The only reason I know it even exists is because as a
> GNU maintainer of some emacs extensions, people sometimes asked my why
> those extensions are not in ELPA. At that point, I needed to try and
> figure out what ELPA even was (not to mention what MELPA was, and what
> the difference is between them.)

I think that ties in to my suggestion that we should promote ELPA more, 
no less.

Probably feature it in the Getting Started guide. Maybe even on the 
startup screen.

The menu item "Manage Emacs Packages" could also use a more prominent 
position.

 > Please don't take this as criticism of ELPA or the efforts of the 
people who work on it, as it is not intended to be one. Instead, I 
wanted to share my real-world use of Emacs in relation to ELPA, in case 
it would be a valuable data-point for the people who are considering the 
question in the subject of the thread: "What is GNU ELPA?"

In your opinion, what would have been the best option to raise awareness 
of it?

Re: What is GNU ELPA?

[Yoni Rabkin]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 17.05.2020 17:49, Yoni Rabkin wrote:
>> In case it is an interesting data point, ELPA may be invisible and
>> un-discoverable to many Emacs users. Here is why I think this may be the
>> case:
>> I've been using Emacs for decades, and I use it every day for my
>> living
>> (read: Emacs is crucial to me and my family). I have a few packages
>> installed in an ~/elisp/  directory and I load those manually via my
>> ~/.emacs.
>> Over all of these years I have never had Emacs ask, offer, or
>> otherwise
>> point out ELPA. The only reason I know it even exists is because as a
>> GNU maintainer of some emacs extensions, people sometimes asked my why
>> those extensions are not in ELPA. At that point, I needed to try and
>> figure out what ELPA even was (not to mention what MELPA was, and what
>> the difference is between them.)
>
> I think that ties in to my suggestion that we should promote ELPA
> more, no less.
>
> Probably feature it in the Getting Started guide. Maybe even on the
> startup screen.
>
> The menu item "Manage Emacs Packages" could also use a more prominent
> position.
>
>> Please don't take this as criticism of ELPA or the efforts of the
>   people who work on it, as it is not intended to be one. Instead, I 
> wanted to share my real-world use of Emacs in relation to ELPA, in
> case it would be a valuable data-point for the people who are
> considering the question in the subject of the thread: "What is GNU
> ELPA?"
>
> In your opinion, what would have been the best option to raise
> awareness of it?

It's a bootstrapping problem, since I don't use ELPA and my Emacs is
already tuned to my needs. I guess that if I had a problem with Emacs,
and I did a web search for it, then it would be useful if the first few
results would be links to something that points at ELPA.

I also wouldn't mind a weekly or monthly mailing list posting somewhere
that presents a few choice packages on ELPA, explaining why its good and
how people are using it to be effective. If I read about something
really neat there, I might be tempted to go to ELPA to get it.

-- 
   "Cut your own wood and it will warm you twice"

Re: What is GNU ELPA?

[Dmitry Gutov]

On 18.05.2020 05:42, Yoni Rabkin wrote:
> I also wouldn't mind a weekly or monthly mailing list posting somewhere
> that presents a few choice packages on ELPA, explaining why its good and
> how people are using it to be effective. If I read about something
> really neat there, I might be tempted to go to ELPA to get it.

That's what Sacha Chua does! And posts it in two places.

https://sachachua.com/blog/
https://lists.gnu.org/mailman/listinfo/emacs-tangents

Re: What is GNU ELPA?

[Alfred M. Szmidt]

   I think that ties in to my suggestion that we should promote ELPA more, 
   no less.

Does ELPA need to be promoted at all?

Wouldn't it rather make more sense to promote that "Emacs also comes
the following that isn't part of core" (might need a better
explanation).

ELPA as it is, isn't that interesting from a user POV -- what is
interesting is what you get when you do M-x list-packages.  I think it
would be more interesting to say what ELPA contains, i.e. mention that
you can get foo-mode or bar-mode just by doing 'M-x package-install'
without needing to do anything, than ELPA specifically.

Re: What is GNU ELPA?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I think it would be useful to make sure we inform users about ELPA --
but certainly not pressure people to _use_ it.  People will use it
if they think it is useful, or else ignore it.  But at least they
will know it is there.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: What is GNU ELPA?

[Dmitry Gutov]

On 18.05.2020 06:53, Richard Stallman wrote:
> I think it would be useful to make sure we inform users about ELPA --
> but certainly not pressure people to_use_  it.  People will use it
> if they think it is useful, or else ignore it.  But at least they
> will know it is there.

I wonder how you imagine "pressuring" to look like.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We do *accept* bug reports to GNU ELPA packages in our debbugs instance,
  > and we occasionally invite people to submit them there, but in my
  > experience people do not expect us to accept bug reports and even less
  > to fix those bugs.

  > Most people (somewhat wrongly) think of GNU ELPA as a *distribution*
  > site for third party packages, like MELPA.

Maybe that changes things.  If we present ELPA as just a distribution site.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

On May 17, 2020 5:53:51 AM GMT+03:00, Richard Stallman <rms@gnu.org> wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
> > We do *accept* bug reports to GNU ELPA packages in our debbugs
> instance,
>   > and we occasionally invite people to submit them there, but in my
> > experience people do not expect us to accept bug reports and even
> less
>   > to fix those bugs.
> 
> > Most people (somewhat wrongly) think of GNU ELPA as a *distribution*
>   > site for third party packages, like MELPA.
> 
> Maybe that changes things.  If we present ELPA as just a distribution
> site.

If we regard ELPA just as distribution site, we shouldn't decide so lightly to leave important packages on ELPA, we should be tend more to adding them to core.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 17.05.2020 16:01, Eli Zaretskii wrote:
>>> Most people (somewhat wrongly) think of GNU ELPA as a*distribution*
>>    > site for third party packages, like MELPA.
>>
>> Maybe that changes things.  If we present ELPA as just a distribution
>> site.
> If we regard ELPA just as distribution site, we shouldn't decide so lightly to leave important packages on ELPA, we should be tend more to adding them to core.

I would like to point out, as an author of several packages, that in my 
experience having a package in ELPA is _better_ than having it in the core.

The exception to that rule is more or less cases where the package is 
not only added but also enabled by default.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

On May 17, 2020 4:38:40 PM GMT+03:00, Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 17.05.2020 16:01, Eli Zaretskii wrote:
> >>> Most people (somewhat wrongly) think of GNU ELPA as
> a*distribution*
> >>    > site for third party packages, like MELPA.
> >>
> >> Maybe that changes things.  If we present ELPA as just a
> distribution
> >> site.
> > If we regard ELPA just as distribution site, we shouldn't decide so
> lightly to leave important packages on ELPA, we should be tend more to
> adding them to core.
> 
> I would like to point out, as an author of several packages, that in
> my 
> experience having a package in ELPA is _better_ than having it in the
> core.
> 
> The exception to that rule is more or less cases where the package is 
> not only added but also enabled by default.

I'm saying that maybe we shouldn't agree so easily to put a package on ELPA if the author would like more than just distribution services.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 17.05.2020 17:24, Eli Zaretskii wrote:

> I'm saying that maybe we shouldn't agree so easily to put a package on ELPA if the author would like more than just distribution services.

I'd say it all depends. We probably aren't going to simply follow what 
the author will be asking for, either.

Do they want code review? We could do it once (couldn't we?), but if the 
author wants all the changes reviewed all the time, we would probably do 
that only for most important packages. Ones that will be enabled for 
default, maybe?

On the other hand, I don't see why we couldn't do code reviews on ELPA 
for select packages, if the authors ask for it. I don't see that 
happening a lot anyway.

Do they simply ask to be included? Perhaps we should ask why. Up until 
now, we often pointed at GNU ELPA and proposed that the package will 
live there. It seems to have worked out fine in those cases. Even aside 
organizational issues (different upstreams, etc), simply including a 
package in Emacs will hardly help it gain users (people don't often 
examine the contents of our distribution). Whereas being featured in 
'M-x list-packages' with a good summary *can* make it well-known.

Do they want to be included and turned on by default? This seems like a 
good reason, if we really like it.

On the flip side, not every discussion that starts with 'I propose to 
put pkg-X in GNU ELPA' should end there.

As I wrote in another thread, maple-minibuffer is something we should 
consider sooner or later for the default behavior, for friendlier 
positioning of the minibuffer on graphical displays.

Another example is elegant-emacs, suggested in yet another thread by 
Nicolas P. Rougier. There's nothing stopping us from featuring it in GNU 
ELPA (right?), but we would get the most value if we really examine it 
and look for pieces to put into the vanilla Emacs by default.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> Cc: joostkremers@fastmail.fm, ams@gnu.org, Emacs-devel@gnu.org,
>  pcr910303@icloud.com, phillip.lord@russet.org.uk
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 17 May 2020 21:27:51 +0300
> 
> On 17.05.2020 17:24, Eli Zaretskii wrote:
> 
> > I'm saying that maybe we shouldn't agree so easily to put a package on ELPA if the author would like more than just distribution services.
> 
> I'd say it all depends. We probably aren't going to simply follow what 
> the author will be asking for, either. [...]

This is all so very different from what was happening here lately (and
as far back as I remember).  So different, in fact, that I find myself
wondering whether I'm living in some parallel world.

What happened here was something completely different.  "Here's a new
package, it does this and that." -- "Not sure we should have it in
core, how about if you put it on ELPA for the time being?"

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 17.05.2020 21:38, Eli Zaretskii wrote:
> What happened here was something completely different.  "Here's a new
> package, it does this and that." -- "Not sure we should have it in
> core, how about if you put it on ELPA for the time being?"

Yes, we did say that.

And then, we never went back to reconsider, to put any of such packages 
into the core.

And we never got any complaints about that, either.

So I'm saying it's generally a good thing for most packages, to be put 
into ELPA instead. We didn't explain that outright (which is 
suboptimal), but it seems like most submitters realized this afterward.

Some of them probably haven't been aware of GNU ELPA initially, too.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 17 May 2020 22:21:33 +0300
> Cc: rms@gnu.org, joostkremers@fastmail.fm, Emacs-devel@gnu.org, ams@gnu.org,
>  monnier@iro.umontreal.ca, pcr910303@icloud.com, phillip.lord@russet.org.uk
> 
> On 17.05.2020 21:38, Eli Zaretskii wrote:
> > What happened here was something completely different.  "Here's a new
> > package, it does this and that." -- "Not sure we should have it in
> > core, how about if you put it on ELPA for the time being?"
> 
> Yes, we did say that.
> 
> And then, we never went back to reconsider, to put any of such packages 
> into the core.

That's not very relevant to the issue at hand.  Let's not change the
subject.

> And we never got any complaints about that, either.

Here I am, complaining.

> So I'm saying it's generally a good thing for most packages, to be put 
> into ELPA instead. We didn't explain that outright (which is 
> suboptimal), but it seems like most submitters realized this afterward.

Of course, you'd say that: you think packages should be on ELPA as a
matter of principle.  You also think we should take some of what is
already in Emacs and put it on ELPA instead.  I wouldn't expect a
balanced opinion from you on this matter.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 17.05.2020 22:30, Eli Zaretskii wrote:
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Sun, 17 May 2020 22:21:33 +0300
>> Cc: rms@gnu.org, joostkremers@fastmail.fm, Emacs-devel@gnu.org, ams@gnu.org,
>>   monnier@iro.umontreal.ca, pcr910303@icloud.com, phillip.lord@russet.org.uk
>>
>> On 17.05.2020 21:38, Eli Zaretskii wrote:
>>> What happened here was something completely different.  "Here's a new
>>> package, it does this and that." -- "Not sure we should have it in
>>> core, how about if you put it on ELPA for the time being?"
>>
>> Yes, we did say that.
>>
>> And then, we never went back to reconsider, to put any of such packages
>> into the core.
> 
> That's not very relevant to the issue at hand.  Let's not change the
> subject.

I don't understand what makes you say that.

>> And we never got any complaints about that, either.
> 
> Here I am, complaining.

You're not the author of any of these packages. Not an ELPA user either, 
IIUC.

>> So I'm saying it's generally a good thing for most packages, to be put
>> into ELPA instead. We didn't explain that outright (which is
>> suboptimal), but it seems like most submitters realized this afterward.
> 
> Of course, you'd say that: you think packages should be on ELPA as a
> matter of principle. You also think we should take some of what is
> already in Emacs and put it on ELPA instead.  I wouldn't expect a
> balanced opinion from you on this matter.

Way to dismiss the arguments without reading.

Since your experience with ELPA is basically zero, how are you going to 
determine what is a balanced opinion, and what isn't?

"Write a new package" culture instead of patches?

[Stefan Kangas]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Another example is elegant-emacs, suggested in yet another thread by
> Nicolas P. Rougier. There's nothing stopping us from featuring it in GNU
> ELPA (right?), but we would get the most value if we really examine it
> and look for pieces to put into the vanilla Emacs by default.

Yes, this is the correct approach in many cases.

This reminds me of something else:

There's a general problem that when package <foo> lacks small feature
<bar>, some users don't see this as a chance to write a patch for <foo>.
Instead, they write a new library <foo>-<bar> to add this feature.

Sometimes, of course, this is the correct choice.  But I've seen some
very small packages just to basically patch this or that annoyance in a
package, or in core.  For example:

    https://github.com/Fuco1/eshell-bookmark/issues/1

(FWIW, I think we should have a policy to reject such packages on
technical grounds (and ideally MELPA would do the same).)

Now, this is an extreme example, but many more could be found.  Why are
the authors of "helpful.el" not helping us mainline some of their great
innovation, for example?

Has anyone else thought about this?  Is it correct to say that such a
"package first" culture has developed?  If yes, why has it developed,
and is there anything we could do about it?

Best regards,
Stefan Kangas

Re: "Write a new package" culture instead of patches?

[Dmitry Gutov]

On 17.05.2020 21:52, Stefan Kangas wrote:
> Why are
> the authors of "helpful.el" not helping us mainline some of their great
> innovation, for example?

I think Wilfred worked on some patch or other, to upstream some of the 
improvements. But not the whole of it.

Maybe because it's a much bigger job: to port the code, to satisfy all 
the historically accumulated edge cases, and to spend a few weeks 
arguing with whoever thinks the previous behavior was better at least in 
some respect.

We don't really have a conceptual framework for assessing big breaking 
changes.

Re: "Write a new package" culture instead of patches?

[Yuan Fu]

> On May 17, 2020, at 3:42 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
> On 17.05.2020 21:52, Stefan Kangas wrote:
>> Why are
>> the authors of "helpful.el" not helping us mainline some of their great
>> innovation, for example?
> 
> I think Wilfred worked on some patch or other, to upstream some of the improvements. But not the whole of it.
> 
> Maybe because it's a much bigger job: to port the code, to satisfy all the historically accumulated edge cases, and to spend a few weeks arguing with whoever thinks the previous behavior was better at least in some respect.
> 
> We don't really have a conceptual framework for assessing big breaking changes.
> 

I think it’s just much easier to write helpful.el from scratch than read all the old code and understand it and try to patch it. I could have patched package.el to make it fetch from github repos, but instead I just wrote a quick small package to do that and moved on, which is much easier than reading and understanding package.el and convince people that such change is necessary.

Yuan

Re: "Write a new package" culture instead of patches?

[Arthur Miller]

Yuan Fu <casouri@gmail.com> writes:

>> On May 17, 2020, at 3:42 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
>> 
>> On 17.05.2020 21:52, Stefan Kangas wrote:
>>> Why are
>>> the authors of "helpful.el" not helping us mainline some of their great
>>> innovation, for example?
>> 
>> I think Wilfred worked on some patch or other, to upstream some of the improvements. But not the whole of it.
>> 
>> Maybe because it's a much bigger job: to port the code, to satisfy all the
>> historically accumulated edge cases, and to spend a few weeks arguing with
>> whoever thinks the previous behavior was better at least in some respect.
>> 
>> We don't really have a conceptual framework for assessing big breaking changes.
>> 
>
> I think it’s just much easier to write helpful.el from scratch than read all the
> old code and understand it and try to patch it. I could have patched package.el
> to make it fetch from github repos, but instead I just wrote a quick small
> package to do that and moved on, which is much easier than reading and
> understanding package.el and convince people that such change is necessary.
>
> Yuan
This "convincing people that such change is necessary" seems to be
actually an important reason :-).

Re: "Write a new package" culture instead of patches?

[chad]

[-- Attachment #1: Type: text/plain, Size: 3176 bytes --]

On Sun, May 17, 2020 at 3:38 PM Yuan Fu <casouri@gmail.com> wrote:

> > On May 17, 2020, at 3:42 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> >
> > On 17.05.2020 21:52, Stefan Kangas wrote:
> >> Why are
> >> the authors of "helpful.el" not helping us mainline some of their great
> >> innovation, for example?
> >
> > I think Wilfred worked on some patch or other, to upstream some of the
> improvements. But not the whole of it.
> >
> > Maybe because it's a much bigger job: to port the code, to satisfy all
> the historically accumulated edge cases, and to spend a few weeks arguing
> with whoever thinks the previous behavior was better at least in some
> respect.
> >
> > We don't really have a conceptual framework for assessing big breaking
> changes.
>
> I think it’s just much easier to write helpful.el from scratch than read
> all the old code and understand it and try to patch it. I could have
> patched package.el to make it fetch from github repos, but instead I just
> wrote a quick small package to do that and moved on, which is much easier
> than reading and understanding package.el and convince people that such
> change is necessary.


Extending this thought even further, I think that there is a perception
from people outside emacs-devel (and also some inside emacs-devel) that
"big" changes are often subjected to a "trial/pilot period" on GNU ELPA.
This seems entirely reasonable, but there are some important extra caveats:

1.) Almost everything that makes substantial user-visible changes in emacs
is very likely to generate (small-c) conservative resistance on emacs-devel.
2.) There doesn't seem to be any real process for taking things off of
their trial/pilot period.
3.) Developers seem most likely to fall into one of two buckets. Etiher my
code changes, and thus I'm happier with it in ELPA than inside emacs core,
or it doesn't, and there's basically no pressure for it not to just stay
inside ELPA, especially since it'll need to stay inside ELPA anyway for
older emacsen (and quite a lot of people are running older emacsen, for
performance and maintenance reasons).

These caveats combine to suggest very little benefit for moving code out of
ELPA and into core. (In fact, I think e.g. Dmitry's preference for moving
things into GNU ELPA is a natural expression of the same  factors.)

In the specific case of helpful.el: IIRC, it was brought up and immediately
ran into the tangle of these things: some interest, some conservatism, some
bikeshedding, and the realization that an ELPA package was
defintely required and, in practice, pretty close to sufficient.

For the future, my hope is that the recent interest in the user experience
of emacs for people who aren't deeply embedded in years (decades) of custom
and practice will result in more of these sorts of things getting included.
I've also noticed that some recent changes to emacs have pushed way more
people to upgrading emacs -- emacs 27's performance for code editing
compared to using LSP & LSP-like systems seems to have pushed upgrades, as
has the potential of testing native-comp. That's hopeful.

 ~Chad

[-- Attachment #2: Type: text/html, Size: 3803 bytes --]

Re: "Write a new package" culture instead of patches?

[Stefan Monnier]

Integration is hard.  So yes, people prefer to make new packages, this
way whoever likes it can install it and those who don't like it won't
be affected.

I don't think it's necessarily a problem.  It just means integration has
to be done separately.  Nowadays it's done at various places by
various people.  It's done here on emacs-devel, of course, but it's also
done in all the "Emacs distributions" like Doom, Spacemacs, ...

Having several distributions makes it easier, because the affected users
can go elsewhere when they're not happy, so while emacs-devel is quite
conservative, other distributions can be more daring.


        Stefan

Re: "Write a new package" culture instead of patches?

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 630 bytes --]

On Mon, May 18, 2020 at 12:43 AM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:
> Integration is hard.  So yes, people prefer to make new packages, this
>  way whoever likes it can install it and those who don't like it won't
> be affected.

Of course. And those are the "hoops" that need be jumped,
and have absolutely nothing to do with Emacs's coding practices
or culture, they're just plain old engineering challenges.  If one
has little time or is bored to solve them, it's easy to flee to
MELPA: a fresh start, no-one depends on the new package,
which usually has a user base of one (or close to it).

João

[-- Attachment #2: Type: text/html, Size: 1187 bytes --]

Re: "Write a new package" culture instead of patches?

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 3236 bytes --]

On Mon, 18 May 2020 at 09:51, Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> Integration is hard.  So yes, people prefer to make new packages, this
> way whoever likes it can install it and those who don't like it won't
> be affected.
>
> I don't think it's necessarily a problem.  It just means integration has
> to be done separately.  Nowadays it's done at various places by
> various people.  It's done here on emacs-devel, of course, but it's also
> done in all the "Emacs distributions" like Doom, Spacemacs, ...
>
> Having several distributions makes it easier, because the affected users
> can go elsewhere when they're not happy, so while emacs-devel is quite
> conservative, other distributions can be more daring.
>
>
> +1. I don't think this is a big issue. There are pros/cons on both sides
and neither has significant advantage over the other. Personally, I like
having ELPA and Emacs be a minimal core which I can extend and add via
packages to get the environment I want. This tends to keep the core stuff
smaller and less complex, leading to less bugs and easier maintenance. The
downside is that when things in core ELPA/Emacs are updated, add on
packages may be broken until the author/maintainer gets around to updating
them.

As someone who has/does maintain code with a free and/or open source
license, I know from experience that contributions, enhancements and
extensions can be a real problem. I have one library which has become much
harder to maintain because I accepted enhancements from others. While those
enhancements were well written, now that they are part of my package, I'm
left to maintain them even though they were not enhancements I needed or
wanted. They have made my package harder to maintain and consequently,
takes time I really don't have. However, because it is now part of my
package, I either have to maintain it or remove it and the associated
functionality, potentially frusrating some users. In hindsight, I wish
these enhancements had been added as separate packages that extended my
lib.  I am now much less willing to accept pull requests that add new
features or functionality.

I think org-mode is probably a good example. There are many extensions and
enhancements to org-mode and many of them are separate packages. There are
still some things in org-mode core which probably should be separate
packages. The majority of these separate extensions/enhancements are not
things I'm interested in, so I don't want them installed. I'm pleased all
these extensions have not been added into the core as if they did,
development of core functionality would slow down and would likely become
more complex and buggy. The downside is that when a new org-mode release
occurs some of the add on packages I use may be broken for a time. However,
perhaps that doesn't matter or perhaps I may choose to pin the version of
org-mode to one where the add-on does work. If an add-on becomes really
popular and used by a majority of org-mode users, then perhaps it becomes a
good candidate for inclusion into org-mode or the functionality it
represents can be added to org-mode (negating the need for the add-on, but
possibly with a different implementation).

-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 3919 bytes --]

Re: "Write a new package" culture instead of patches?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I don't think it's necessarily a problem.  It just means integration has
  > to be done separately.

I agree -- but let's be clear that "integration" means merging the change
(perhaps rewritten), not installing the patch package.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: "Write a new package" culture instead of patches?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I think it’s just much easier to write helpful.el from scratch
  > than read all the old code and understand it and try to patch
  > it. I could have patched package.el to make it fetch from github
  > repos, but instead I just wrote a quick small package to do that
  > and moved on, which is much easier than reading and understanding
  > package.el and convince people that such change is necessary.

When such patch packages accumulate, they will get in each others' way
because sometimes they will replace the same function with different
patched versions.  To make the unintegrated patches useful together
requires merging them.

Thus, the author of each patch package saves work by not following
through on the job.  Eventually the code in Emacs changes and the
patch package doesn't work any more.

We would like those people to help us merge their code (when it is
useful enough), but We can't tell them what to do.  What can we do?
What should we do?  Here is what I suggest.

* We should not distribute or refer people to patch packages.
If a repo includes more than a tiny fraction of patch packages,
we should consider it low-quality.

* If the job of merging is really easy we could do it straightaway.
Rewriting the change would avoid any need for an assignment.

* When users express interest in a patch package, we should say, "The
right thing to do here is merge that change.  Would you like to do
that work?  Then we could install the patch and support it."


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: "Write a new package" culture instead of patches?

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

> * When users express interest in a patch package, we should say, "The
> right thing to do here is merge that change.  Would you like to do
> that work?  Then we could install the patch and support it."

I agree with everything you say here.  It is a good summary.

Just to clarify that helpful.el goes much further than a "patch
package".  It has significant features.

I believe I opened up for confusion by mentioning helpful.el in the same
breath as another package.  That other package indeed could and should
rightly be described as a "patch package".

I felt it was in place for me to clarify this, in fairness to the
helpful.el developers.

Best regards,
Stefan Kangas

Re: "Write a new package" culture instead of patches?

[Alan Third]

On Sun, May 17, 2020 at 11:52:18AM -0700, Stefan Kangas wrote:
> 
> Has anyone else thought about this?  Is it correct to say that such a
> "package first" culture has developed?  If yes, why has it developed,
> and is there anything we could do about it?

I wonder if it's related to the way that a couple of years ago many of
the discussions on the Emacs reddit seemed to revolve around why the
Emacs maintainers hadn't yet fixed someone's pet bug, but nobody ever
thought to report it to us.
-- 
Alan Third

Re: "Write a new package" culture instead of patches?

[Arthur Miller]

Alan Third <alan@idiocy.org> writes:

> On Sun, May 17, 2020 at 11:52:18AM -0700, Stefan Kangas wrote:
>> 
>> Has anyone else thought about this?  Is it correct to say that such a
>> "package first" culture has developed?  If yes, why has it developed,
>> and is there anything we could do about it?
>
> I wonder if it's related to the way that a couple of years ago many of
> the discussions on the Emacs reddit seemed to revolve around why the
> Emacs maintainers hadn't yet fixed someone's pet bug, but nobody ever
> thought to report it to us.
Could it rather be that a "github" culture has evolved, together with
social media it makes + melpa it makes it relatively easy to fork
someone's work, change/fix what bother you and make your own package
under other name.

There was recent reddit thread where some guy posted new
search/completion framework. Reason was nothing suites him. On the
github page he went through all the different frameworks already
avialable (some of which I didn't even hear off) concluding that Helm
was the only "resonable" pne, but was so big so he prefer to write his
own.

Another factor is that maybe original author does not wish to bend
his/her package according to what someone wishes, and that someone forks
and patches the original according to own desire under different name. I
don't know, seems a little bit so. Github and flexible licensing made it
easy to fork other peoples work, social media like Reddit & Twitter made
it easy to spread the word about it and Melpa makes it easy to share (in
case of Emacs).

I don't think it has anything Emacs devs not fixing something, it is
just emerging social dev trend. Also more people are technically
skilled nowadays (millenials) and we more programmers or hobby
programmers then probably ever before.

Re: "Write a new package" culture instead of patches?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1856 bytes --]

On Mon, May 18, 2020 at 12:02:52AM +0200, Arthur Miller wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > On Sun, May 17, 2020 at 11:52:18AM -0700, Stefan Kangas wrote:
> >> 
> >> Has anyone else thought about this?  Is it correct to say that such a
> >> "package first" culture has developed?  If yes, why has it developed,
> >> and is there anything we could do about it?
> >
> > I wonder if it's related to the way that a couple of years ago many of
> > the discussions on the Emacs reddit seemed to revolve around why the
> > Emacs maintainers hadn't yet fixed someone's pet bug, but nobody ever
> > thought to report it to us.
> Could it rather be that a "github" culture has evolved, together with
> social media it makes + melpa it makes it relatively easy to fork
> someone's work, change/fix what bother you and make your own package
> under other name.

This rhymes with one observation I made: Git makes branching easy.
Still, Github strongly encourages forking. Why is this so? My hunch
is that for Github, the number of repositories they host is /currency/
(actually to the tune of $7.5B, as it turned out by 2018). So there's
a strong motivation to multiply the number of repos.

That is, I think, the same mechanism as Twitter or Facebook
tolerating bots as legit accounts (up to a certain point),
because they inflate their market value. And not much different
as Microsoft tolerating pirated versions of Windows (remember
the end-90s where everyone knew that you could generate a valid
Windows license key by making sure that the middle part of
the number was divisible by 7?).

These are, of course, mechanisms which are totally alien to the
Free Software world [1]. But I guess it's standard corporate
fare. Something-something-strategy, I guess.

Cheers
[1] Although we're catching up :-/
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: "Write a new package" culture instead of patches?

[Arthur Miller]

<tomas@tuxteam.de> writes:

> On Mon, May 18, 2020 at 12:02:52AM +0200, Arthur Miller wrote:
>> Alan Third <alan@idiocy.org> writes:
>> 
>> > On Sun, May 17, 2020 at 11:52:18AM -0700, Stefan Kangas wrote:
>> >> 
>> >> Has anyone else thought about this?  Is it correct to say that such a
>> >> "package first" culture has developed?  If yes, why has it developed,
>> >> and is there anything we could do about it?
>> >
>> > I wonder if it's related to the way that a couple of years ago many of
>> > the discussions on the Emacs reddit seemed to revolve around why the
>> > Emacs maintainers hadn't yet fixed someone's pet bug, but nobody ever
>> > thought to report it to us.
>> Could it rather be that a "github" culture has evolved, together with
>> social media it makes + melpa it makes it relatively easy to fork
>> someone's work, change/fix what bother you and make your own package
>> under other name.
>
> This rhymes with one observation I made: Git makes branching easy.
> Still, Github strongly encourages forking. Why is this so? My hunch
> is that for Github, the number of repositories they host is /currency/
> (actually to the tune of $7.5B, as it turned out by 2018). So there's
> a strong motivation to multiply the number of repos.
>
> That is, I think, the same mechanism as Twitter or Facebook
> tolerating bots as legit accounts (up to a certain point),
> because they inflate their market value. And not much different
> as Microsoft tolerating pirated versions of Windows (remember
> the end-90s where everyone knew that you could generate a valid
> Windows license key by making sure that the middle part of
> the number was divisible by 7?).
>
> These are, of course, mechanisms which are totally alien to the
> Free Software world [1]. But I guess it's standard corporate
> fare. Something-something-strategy, I guess.
>
> Cheers
> [1] Although we're catching up :-/
> -- t
That plays role definitely. Familiarity as well. Github is really easy
to work with and get started with git and as they let people do whatever
they want, like fork crap load of repos, it also makes people use github
and get familiar with its APIs etc. Once I setuped github api keys, I
don't feel for going over and setting up API keys for gitlab, I dont'
know is there a free service like github? I have very modest needs and
am too lazy for me to be worth going hoops around searching for better
alternative.

I think familiarity is also reason why major companies tolarated privacy
back as you say. I remember in early 2000, if you wrote word microsoft
or adobe in back than AltaVista or Google you would be bombed with pages
that offered keygens and pirated copies. They let people use it, it ment
people will learn it, and once they learn it, they stick with it. So
once they come to workplace the company will get them the software
because they are knowledgable with it.

Re: "Write a new package" culture instead of patches?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 782 bytes --]

On Mon, May 18, 2020 at 02:08:38PM +0200, Arthur Miller wrote:
> <tomas@tuxteam.de> writes:

[on branching vs contributing, Github culture]

> That plays role definitely. Familiarity as well. Github is really easy
> to work with [...]

Yes, the bribe of convenience.

> [...] I dont' know is there a free service like github? I have
> very modest needs [...]

If you are willing to learn a new web interface, there's Gitlab
(the server component is umm... somewhat free; more precisely
it's "open core", as they say) and there's Gitea. Savannah has
a git service too, I don't know very much about the interface
they offer.

It does take a bit of willpower to leave the plushy universe,
but believe me -- it's a great landscape out there!

C'm on. Take the red pill ;-D

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

RE: "Write a new package" culture instead of patches?

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 2029 bytes --]

Haha, yeah I know. I actually once  created a gitlab project for a customer on girlab, I just didn't know it was open source. Or I have just forgot . But even if gitlab is open source, what says that their web interface is? How do I know with my data and me, that I can't know? :-) Is it just "open source" or  "free" as in fsf free.

Anyway, convenience is just one part of equation. The big issue is convenience of group. Everyone is on github. One fork a repo, make a commit and create PR. PR is the new patch. People don't send patches in emails longer (ok kernel a d Emacs folks does), it is kind of getting out of fashion. And github makes that  very convenient.

Anyway, the forking culture has more to do with business then just for the service providers. Small companies create projects, and let people fork, the more people fork, the  better it looks in presentation for in estors: ohook, we ha e 5000 firks and 10 000 downloads, we are popular, grant us funding for next year and we can do this and that....




-------- Originalmeddelande --------
Från: tomas@tuxteam.de
Datum: 2020-05-18 14:33 (GMT+01:00)
Till: emacs-devel@gnu.org
Ämne: Re: "Write a new package" culture instead of patches?

On Mon, May 18, 2020 at 02:08:38PM +0200, Arthur Miller wrote:
> <tomas@tuxteam.de> writes:

[on branching vs contributing, Github culture]

> That plays role definitely. Familiarity as well. Github is really easy
> to work with [...]

Yes, the bribe of convenience.

> [...] I dont' know is there a free service like github? I have
> very modest needs [...]

If you are willing to learn a new web interface, there's Gitlab
(the server component is umm... somewhat free; more precisely
it's "open core", as they say) and there's Gitea. Savannah has
a git service too, I don't know very much about the interface
they offer.

It does take a bit of willpower to leave the plushy universe,
but believe me -- it's a great landscape out there!

C'm on. Take the red pill ;-D

Cheers
-- t

[-- Attachment #2: Type: text/html, Size: 2972 bytes --]

Re: "Write a new package" culture instead of patches?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

On Mon, May 18, 2020 at 11:07:23PM +0000, arthur miller wrote:
> Haha, yeah I know. I actually once  created a gitlab project for a customer on girlab, I just didn't know it was open source. Or I have just forgot . But even if gitlab is open source, what says that their web interface is? How do I know with my data and me, that I can't know? :-) Is it just "open source" or  "free" as in fsf free.

A service per se isn't "open source". The programs it is based on
can be... and Gitlab scores decently here (it's "open core").

> Anyway, convenience is just one part of equation. The big issue is convenience of group. Everyone is on github. One fork a repo, make a commit and create PR. PR is the new patch. People don't send patches in emails longer (ok kernel a d Emacs folks does), it is kind of getting out of fashion. And github makes that  very convenient.

This is called network effect. And yes, it's part of convenience.

> Anyway, the forking culture has more to do with business then just for the service providers. Small companies create projects, and let people fork, the more people fork, the  better it looks in presentation for in estors: ohook, we ha e 5000 firks and 10 000 downloads, we are popular, grant us funding for next year and we can do this and that....

That's my guess too: some shiny pseudo-metrics (that what made
Github 7.5B dollar worth in the first place).

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: "Write a new package" culture instead of patches?

[Matthias Meulien]

Stefan Kangas <stefankangas@gmail.com> writes:
 
> There's a general problem that when package <foo> lacks small 
> feature <bar>, some users don't see this as a chance to write a 
> patch for <foo>.  Instead, they write a new library <foo>-<bar> 
> to add this feature. 
> 
> Sometimes, of course, this is the correct choice.  But I've seen 
> some very small packages just to basically patch this or that 
> annoyance in a package, or in core.  For example: 
> 
>     https://github.com/Fuco1/eshell-bookmark/issues/1

Stefan, may be you'll like to have support for bookmark.el in VC 
dir buffers too (see bug #39722)? ;-)

> (FWIW, I think we should have a policy to reject such packages 
> on technical grounds (and ideally MELPA would do the same).) 
> 
> Now, this is an extreme example, but many more could be found. 
> Why are the authors of "helpful.el" not helping us mainline some 
> of their great innovation, for example? 
> 
> Has anyone else thought about this?  Is it correct to say that 
> such a "package first" culture has developed?  If yes, why has 
> it developed, and is there anything we could do about it?

I guess good reasons to prefer small packages to fix annoyances is 
that: 

1) The delay can be long between a patch is submitted and it is 
   commented or merged (already two months for the mentionned 
   one). OTOH packages are immediatly availables to all your 
   computers

2) It's not clear to me whether trivial patches are welcome or 
not; My 
   feeling is that they're wasting precious time of core Emacs 
   developers
-- 
Matthias

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'd say it all depends. We probably aren't going to simply follow what 
  > the author will be asking for, either.

  > Do they want code review? We could do it once (couldn't we?), but if the 
  > author wants all the changes reviewed all the time, we would probably do 
  > that only for most important packages. Ones that will be enabled for 
  > default, maybe?

If we are going to continue saying, of GNU ELPA, that "You can trust
it", I think that we need to do some code review for every package in
GNU ELPA.  We had better treat serious bugs in GNU ELPA the way we
treat serious bugs in Emacs.

If we want to have two different ways of treating those packages,
we need to show the users which category each package is in.

The reliable way to do that is to have two archives: one we say you
can trust, and one that provides only a place to distribute them.

Good names might be "GNU Emacs Exocore" for the ones we review, and
"GNU ELPA" for those we don't.  I suggest "Exocore" as meaning "like
the core, but hosted separately."

Or maybe, GNU ELPA for the ones we review, and Alt-ELPA for what we don't.

For now, let's call them reviewed and unreviewed.


MAYBE it will work well if we get papers for the reviewed packages
but not for the unreviewed.  Then the reviewed packages might be
merged into the core, and the unreviewed are ones we don't consider
moving into the core.  So if we think a package might be good to put
in the core, we should review it AND get papers for it.

Eli, do you think that makes sense?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 18.05.2020 06:49, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>    > I'd say it all depends. We probably aren't going to simply follow what
>    > the author will be asking for, either.
> 
>    > Do they want code review? We could do it once (couldn't we?), but if the
>    > author wants all the changes reviewed all the time, we would probably do
>    > that only for most important packages. Ones that will be enabled for
>    > default, maybe?
> 
> If we are going to continue saying, of GNU ELPA, that "You can trust
> it", I think that we need to do some code review for every package in
> GNU ELPA.  We had better treat serious bugs in GNU ELPA the way we
> treat serious bugs in Emacs.

I'm all for it, personally. As far as I know, this has only been limited 
by our developers' free time (or its deficit), and by their general 
disinterest (seems like).

I don't think any package author would really object to an external 
review, people pointing out serious bugs, etc.

(Enforcing commit message format and documentation standards are a 
somewhat different matter.)

> If we want to have two different ways of treating those packages,
> we need to show the users which category each package is in.
> 
> The reliable way to do that is to have two archives: one we say you
> can trust, and one that provides only a place to distribute them.

Personally, I don't know if we really need two archives, or packages we 
don't "trust". We could somehow annotate packages without copyright 
assignment, but its lack alone doesn't make a package untrustworthy.

> Good names might be "GNU Emacs Exocore" for the ones we review, and
> "GNU ELPA" for those we don't.  I suggest "Exocore" as meaning "like
> the core, but hosted separately."
> 
> Or maybe, GNU ELPA for the ones we review, and Alt-ELPA for what we don't.
> 
> For now, let's call them reviewed and unreviewed.

We still could do that, if we find that we don't have enough manpower 
for reviewing all.

> MAYBE it will work well if we get papers for the reviewed packages
> but not for the unreviewed.  Then the reviewed packages might be
> merged into the core, and the unreviewed are ones we don't consider
> moving into the core.  So if we think a package might be good to put
> in the core, we should review it AND get papers for it.

Not sure this would be best for the users, but it does make some 
economical sense.

Splitting GNU ELPA

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >  We could somehow annotate packages without copyright 
  > assignment, but its lack alone doesn't make a package untrustworthy.

You've changd the point a little, so as to disagree with something
that wasn't what I said.

Indeed, there is nothing to stop us from reviewing, and subsequently
maintaining, non-copyright-assigned packages too.  But that is extra
work, and we already have a too much to do.  So while we consider
adding non-copyright-assigned packages to GNU ELPA, perhaps many of
them, we should consider not adding them to the packages that get
our code review and maintenance.

We could instead check them only to make sure they don't lead people
to nonfree software.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> cc: Emacs-devel@gnu.org
> Date: Sun, 17 May 2020 23:49:39 -0400
> 
> The reliable way to do that is to have two archives: one we say you
> can trust, and one that provides only a place to distribute them.
> 
> Good names might be "GNU Emacs Exocore" for the ones we review, and
> "GNU ELPA" for those we don't.  I suggest "Exocore" as meaning "like
> the core, but hosted separately."
> 
> Or maybe, GNU ELPA for the ones we review, and Alt-ELPA for what we don't.
> 
> For now, let's call them reviewed and unreviewed.
> 
> 
> MAYBE it will work well if we get papers for the reviewed packages
> but not for the unreviewed.  Then the reviewed packages might be
> merged into the core, and the unreviewed are ones we don't consider
> moving into the core.  So if we think a package might be good to put
> in the core, we should review it AND get papers for it.
> 
> Eli, do you think that makes sense?

I do, and a similar suggestion was mentioned here more than once in
the recent days (including by myself).  The question is, do we have a
wide enough consensus about such an arrangement?

Splitting GNU ELPA

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I do, and a similar suggestion was mentioned here more than once in
  > the recent days (including by myself).  The question is, do we have a
  > wide enough consensus about such an arrangement?

I've changed the Subject to separate this thread from the others.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Splitting GNU ELPA

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > My opinion on these matters means very little, but FWIW I basically
  > agree with what Stefan wrote.

Your opinion matters, but Stefan has said many things and I don't
recall them all precisely.  I may not even have seen all those messages.

Could you restate the position you agree with?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Splitting GNU ELPA

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: Emacs-devel@gnu.org
> Date: Mon, 18 May 2020 23:55:51 -0400
> 
>   > My opinion on these matters means very little, but FWIW I basically
>   > agree with what Stefan wrote.
> 
> Your opinion matters, but Stefan has said many things and I don't
> recall them all precisely.  I may not even have seen all those messages.
> 
> Could you restate the position you agree with?

Here's what Stefan said about this:

  FWIW, I'm not too fond of such shorthand syntax.  The benefit is not
  very high and it makes the language that much more difficult to learn
  for newcomers.

  For a programming language like Closure, it might make sense, since
  most/all people writing Closure programs are actual programmers that
  have to be proficient in Clojure.  But Elisp lambdas are very common in
  .emacs files, so this additional complexity will be exposed to some of
  our users who aren't programmers or aren't proficient in Elisp.

  [ Yes, I know it may sound strange coming from me, since I'm to blame
    for a lot of complexity in Elisp :-(
    But to my defense, `pcase` and `cl-defmethod` aren't nearly as often
    needed in .emacs as lambdas.  ]

  I can already see the discussions in forums about "what's the difference
  between #(f % 1) and (lambda (x) (f x 1))", "when should I use which",
  ...

  If we want to shorten anonymous functions we could start with something
  like (λ (x) (f x 1)), which you can get with
  prettify-symbols-mode already.

Re: Splitting GNU ELPA

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I don't think that #(+ %1 %2) is very difficult to learn.  So I don't
see a significant downside in supporting this construct.

On the other hand, for that particular function, you could write '+.

    > I can already see the discussions in forums about "what's the difference
    > between #(f % 1) and (lambda (x) (f x 1))", "when should I use which",

But the answer is simple: they are equivalent, the choice is a matter
of taste.

    > If we want to shorten anonymous functions we could start with something
    > like (λ (x) (f x 1)), which you can get with
    > prettify-symbols-mode already.

That is not much of a simplification, and think it is a drawback
to display different text from what is really in the file.
#(f % 1) avoids that drawback.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I would like to point out, as an author of several packages, that in my 
  > experience having a package in ELPA is _better_ than having it in the core.

Better in which sense?  Do you mean, better for you in maintaining the
package?

Having the package in the core makes it possible to maintain it in
sync with Emacs releases.  Having it in GNU ELPA makes it possible to
release improvements in the package that work for old Emacs releases.
I think that either one can be valuable in some cases.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 18.05.2020 06:49, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>    > I would like to point out, as an author of several packages, that in my
>    > experience having a package in ELPA is _better_ than having it in the core.
> 
> Better in which sense?  Do you mean, better for you in maintaining the
> package?

As explained in another email: better discoverability. Better at helping 
the users notice the package.

Simply including a file in the Emacs distro doesn't do much.

> Having the package in the core makes it possible to maintain it in
> sync with Emacs releases.  Having it in GNU ELPA makes it possible to
> release improvements in the package that work for old Emacs releases.
> I think that either one can be valuable in some cases.

The latter is definitely useful. It's also a boon (with :core packages) 
when you want a release a new version, the last Emacs release was a 
month ago, and it's a year or two until the next one.

The former... there are certainly cases when a package is good to have 
in the core (the obvious ones are when we want to turn it on by default).

As far as maintaining them in sync, do you mean the advantage is the 
package can use new APIs, hooks, etc, that are added at the same time 
the change in the package is made? That can certainly be a boon, but the 
very same thing makes keeping the package work with older Emacs more 
difficult.

The packages that require new APIS are almost always new packages 
themselves. So it might help to let them "incubate" inside Emacs, but 
after that the advantages of staying in the core are less obvious (aside 
from keeping compatibility, etc). And such packages can still be 
developed externally, perhaps with somewhat higher overhead.

IME most packages depend on stable interfaces, though.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> Cc: eliz@gnu.org, monnier@iro.umontreal.ca, joostkremers@fastmail.fm,
>  ams@gnu.org, phillip.lord@russet.org.uk, pcr910303@icloud.com,
>  Emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 18 May 2020 14:24:56 +0300
> 
> Simply including a file in the Emacs distro doesn't do much.

Indeed, it doesn't.  Which is why we call out new packages in NEWS.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 18.05.2020 18:10, Eli Zaretskii wrote:
>> Simply including a file in the Emacs distro doesn't do much.
> Indeed, it doesn't.  Which is why we call out new packages in NEWS.

Okay. That's better than nothing, but not even close to being in the 
packages list.

An arbitrary NEWS file doesn't mention all packages we offer. It 
obviously doesn't cover past releases. People who just installed Emacs 
don't read NEWS.

Even experienced users can miss announcements in there. Yoni not being 
aware of package.el (even though we surely announced it in NEWS) is a 
recent example.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> Cc: rms@gnu.org, monnier@iro.umontreal.ca, joostkremers@fastmail.fm,
>  ams@gnu.org, phillip.lord@russet.org.uk, pcr910303@icloud.com,
>  Emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 18 May 2020 19:13:51 +0300
> 
> An arbitrary NEWS file doesn't mention all packages we offer.

It mentions the new ones.  For the rest, we have "C-h p".  Of course,
you already know all that, so I'm not sure I understand where does
this argument go.

> Even experienced users can miss announcements in there. Yoni not being 
> aware of package.el (even though we surely announced it in NEWS) is a 
> recent example.

An example is just that.  It doesn't necessarily prove a general
point, not by itself.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 18.05.2020 19:28, Eli Zaretskii wrote:
>> Cc: rms@gnu.org, monnier@iro.umontreal.ca, joostkremers@fastmail.fm,
>>   ams@gnu.org, phillip.lord@russet.org.uk, pcr910303@icloud.com,
>>   Emacs-devel@gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Mon, 18 May 2020 19:13:51 +0300
>>
>> An arbitrary NEWS file doesn't mention all packages we offer.
> 
> It mentions the new ones.  For the rest, we have "C-h p".  Of course,
> you already know all that,

No, I don't. I may have seen it mentioned a few years ago, tried it 
once, the result didn't help me with whatever I was doing at the time, 
so I promptly forgot the binding and what it does.

Does anybody here use 'C-h p' on a regular basis?

I use 'M-x list-packages' fairly often, however. So do many other users.

It's also easier to search right away with 'C-s'.

> so I'm not sure I understand where does
> this argument go.

Perhaps I'm just talking to Richard, given how quickly you dismiss most 
of my explanations.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

On Mon, May 18, 2020 at 7:00 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> No, I don't. I may have seen it mentioned a few years ago, tried it
> once, the result didn't help me with whatever I was doing at the time,
> so I promptly forgot the binding and what it does.
>
> Does anybody here use 'C-h p' on a regular basis?

Not me. And indeed, I must say I'm always (pleasantly) surprised
by the vastness of C-h commands.  But I don't use half of them,
because I don't know they exist.

But again, that's a question of making them more visible.
I probably know hundreds of keybindings by heart.  I learned
them because they were either very useful or cleverly
advertised.

Let's make a tip-of-the-day or something like that.  This
isn't exactly a new problem in UI design. Most importantly
let's fix these shortcomings before using them as an excuse
to do fundamental changes to the way we organize and
develop software.

João

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 19.05.2020 17:18, João Távora wrote:
> Let's make a tip-of-the-day or something like that.  This
> isn't exactly a new problem in UI design.

Splitting the users' attention between different ways to do similar 
things is a known anti-pattern in UI design, however.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Kangas]

João Távora <joaotavora@gmail.com> writes:

> Let's make a tip-of-the-day or something like that.  This
> isn't exactly a new problem in UI design. Most importantly
> let's fix these shortcomings before using them as an excuse
> to do fundamental changes to the way we organize and
> develop software.

I have no well-formed opinion on `C-h p', but I do think a "tip of the
day" feature could be a good idea.  It's obviously not a silver bullet,
but it can be useful within its limits.

There is on-going work to the splash screen, and I have had the loose
idea to maybe suggest placing a "tip of the day" there.

Is anyone aware of any drawbacks to having such a feature?  Are there
better or smarter alternatives?

I think the biggest challenge is to make it useful enough that users
will actively want to read it.  Otherwise, they will just ignore it.

Best regards,
Stefan Kangas

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 19.05.2020 18:14, Stefan Kangas wrote:
> I have no well-formed opinion on `C-h p', but I do think a "tip of the
> day" feature could be a good idea.  It's obviously not a silver bullet,
> but it can be useful within its limits.
> 
> There is on-going work to the splash screen, and I have had the loose
> idea to maybe suggest placing a "tip of the day" there.

I like it.

Though it seems most users customize the start screen to not be shown 
anymore. So we'd either need to change that, or the tips will have a 
markedly smaller audience.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Jean-Christophe Helary]

> On May 20, 2020, at 0:14, Stefan Kangas <stefankangas@gmail.com> wrote:
> 
> João Távora <joaotavora@gmail.com> writes:
> 
>> Let's make a tip-of-the-day or something like that.  This
>> isn't exactly a new problem in UI design. Most importantly
>> let's fix these shortcomings before using them as an excuse
>> to do fundamental changes to the way we organize and
>> develop software.
> 
> I have no well-formed opinion on `C-h p', but I do think a "tip of the
> day" feature could be a good idea.  It's obviously not a silver bullet,
> but it can be useful within its limits.
> 
> There is on-going work to the splash screen, and I have had the loose
> idea to maybe suggest placing a "tip of the day" there.

Why not in the echo area ?

> I think the biggest challenge is to make it useful enough that users
> will actively want to read it.  Otherwise, they will just ignore it.

Make it contextual. Like, display a command in the current mode that the user has not used yet.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > An arbitrary NEWS file doesn't mention all packages we offer.

  > It mentions the new ones.  For the rest, we have "C-h p".  Of course,
  > you already know all that, so I'm not sure I understand where does
  > this argument go.

If C-h p is not doing the job, let's look for ways to do it better.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 19.05.2020 06:53, Richard Stallman wrote:
>    > > An arbitrary NEWS file doesn't mention all packages we offer.
> 
>    > It mentions the new ones.  For the rest, we have "C-h p".  Of course,
>    > you already know all that, so I'm not sure I understand where does
>    > this argument go.
> 
> If C-h p is not doing the job, let's look for ways to do it better.

It's not like it's "not doing the job", or that Finder became worse now, 
3 decades after it was created (I just looked up its development history).

The new alternative simply has more traction. It's basic usability: once 
the user learns that they can install packages to add extra Emacs 
functionality, they will look for more packages if they need something else.

And the notion of package repositories is everywhere in the software 
world these days.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

On Tue, May 19, 2020 at 2:07 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 19.05.2020 06:53, Richard Stallman wrote:
> >    > > An arbitrary NEWS file doesn't mention all packages we offer.
> >
> >    > It mentions the new ones.  For the rest, we have "C-h p".  Of course,
> >    > you already know all that, so I'm not sure I understand where does
> >    > this argument go.
> >
> > If C-h p is not doing the job, let's look for ways to do it better.
>
> It's not like it's "not doing the job", or that Finder became worse now,
> 3 decades after it was created (I just looked up its development history).
>
> The new alternative simply has more traction. It's basic usability: once
> the user learns that they can install packages to add extra Emacs
> functionality, they will look for more packages if they need something else.

Let's hypothesize that's really true: then let's go heavy on the :core
packages and problem solved, right?

João

PS: but you hypothesis might not really be 100% true.  A few years
ago, a blogger called Mickey Peterson had a very nice
https://www.masteringemacs.org/ blog where he went patiently over
the NEWS items and commented on  each individual addition.  The
very pretty blog theme helped.  I learned a lot there. In the end it came
down to someone that sat down and took the time to write something
useful.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 19.05.2020 17:29, João Távora wrote:
> PS: but you hypothesis might not really be 100% true.  A few years
> ago, a blogger called Mickey Peterson had a very nice
> https://www.masteringemacs.org/  blog where he went patiently over
> the NEWS items and commented on  each individual addition.  The
> very pretty blog theme helped.  I learned a lot there. In the end it came
> down to someone that sat down and took the time to write something
> useful.

I wasn't arguing that nobody reads NEWS at all. I saw those articles, 
and read them with great interest.

That's still a long list of features, packed tightly together (meaning 
the info about a single package doesn't stand out much), that comes out 
only once every 2 years.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Kangas]

Dmitry Gutov <dgutov@yandex.ru> writes:

> It's not like it's "not doing the job", or that Finder became worse now,
> 3 decades after it was created (I just looked up its development history).
>
> The new alternative simply has more traction. It's basic usability: once
> the user learns that they can install packages to add extra Emacs
> functionality, they will look for more packages if they need something else.
>
> And the notion of package repositories is everywhere in the software
> world these days.

Maybe we should work on making `list-packages' better instead?  For
example, it could support package categories much like finder does, or
it could do more to make built-in packages visible.

Best regards,
Stefan Kangas

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 19.05.2020 18:20, Stefan Kangas wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>> It's not like it's "not doing the job", or that Finder became worse now,
>> 3 decades after it was created (I just looked up its development history).
>>
>> The new alternative simply has more traction. It's basic usability: once
>> the user learns that they can install packages to add extra Emacs
>> functionality, they will look for more packages if they need something else.
>>
>> And the notion of package repositories is everywhere in the software
>> world these days.
> 
> Maybe we should work on making `list-packages' better instead?  For
> example, it could support package categories much like finder does, or
> it could do more to make built-in packages visible.

But it does have similar features: it can filter by categories ('/ k'), 
for instance. And they're based on the same underlying library.

Some built-in packages show up in there too (woman, viper, epg).

I don't think it would make sense for 'list-packages' to start with a 
categories list, however.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It's not like it's "not doing the job", or that Finder became worse now, 
  > 3 decades after it was created (I just looked up its development history).

  > The new alternative simply has more traction. It's basic usability: once 
  > the user learns that they can install packages to add extra Emacs 
  > functionality, they will look for more packages if they need something else.

Any ideas for how to make the core packages more visible?

Make stub entries for them in GNU ELPA, perhaps?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Better in which sense?  Do you mean, better for you in maintaining the
  > > package?

  > As explained in another email: better discoverability. Better at helping 
  > the users notice the package.

  > Simply including a file in the Emacs distro doesn't do much.

Do people have any concrete suggestions for how to improve this?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 19.05.2020 06:59, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>    > > Better in which sense?  Do you mean, better for you in maintaining the
>    > > package?
> 
>    > As explained in another email: better discoverability. Better at helping
>    > the users notice the package.
> 
>    > Simply including a file in the Emacs distro doesn't do much.
> 
> Do people have any concrete suggestions for how to improve this?

We do have an existing solution, actually. We add a number of core 
packages to GNU ELPA with a special annotation that makes it build and 
distribute the versions currently in Emacs master branch. Then they also 
show up in 'M-x list-packages'. There's some organizational overhead 
associated with that choice, but in any case, as a result, the author 
reaps all the benefits associated with being in ELPA as well.

So, to take a step back, what I'm saying is, adding a package to the 
core doesn't automatically makes things _worse_ for the author, in terms 
of the eventual popularity of the package. It's just that, if they added 
the package to GNU ELPA, they already enjoy the vast majority of the 
benefits.

And if we create a "blessed" subsection of GNU ELPA which we pay more 
attention to, review and promote on the official Emacs website, the 
reasons to push into the core will diminish further, for most packages.

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 4401 bytes --]

On Tue, 19 May 2020 at 14:01, Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > Better in which sense?  Do you mean, better for you in maintaining
> the
>   > > package?
>
>   > As explained in another email: better discoverability. Better at
> helping
>   > the users notice the package.
>
>   > Simply including a file in the Emacs distro doesn't do much.
>
> Do people have any concrete suggestions for how to improve this?
>
>
Is there actually a discoverability problem for GNU ELPA packages? After
over 25 years of Emacs use, I think discovering and trying out new packages
and features has never been easier. I really don't miss the old days of
hunting around various ftp servers trying to find the latest and current
version of some library or package.

I know for me at least, as soon as I need some new functionality (setting
up for a new language, needing to do something new, looking for
alternatives etc),  or just check out what new feaqtures, packages etc are
available, one of the first things I do is M-x list-packages. This gives me
a list of all the available packages from all the package archives I have
listed. The source (archive) for each package is clearly identified and
since I added all the additional archives, I know what the differences is
between them. If a package is available in both GNU ELPA and another
archive, like MELPA, I will typically select the one from ELPA, because
rightly or wrongly, I expect it to be more stable and comply with GNU
philosophy. I might later change (as is the case for org because I want
both the latest org version and the additional packages from org contrib).

The only real issue I see is the apparent confusion within the GNU Emacs
project itself regarding ELPA, what role it should play and how it should
be maintained. This is based on some of the threads seen in this list.
Perhaps the below has already been done and if so, then we just need to
make that information more prominent or accessible and point people to this
information when questions regarding ELPA arise. Anyway, I think the
following might help

- Define the rationale and objectives that underpins ELPA. Forget about
historical context and focus on current context.

- Define what should be and what should not be included in ELPA. This
should be based on objective criteria

- Define an ELPA custodian responsible for deciding whether a package
should or should not be added/removed from ELPA. This custodian should be
able to appoint a team of people who can assist. There will probably need
to be some appeal process as well, but I'd keep it simple to begin with.

- Ensure the custodian is supported and is seen as the 'official voice'.
Any ideas, concerns or need for clarification re: policy etc should first
be discussed with the ELPA custodian (they may delegate some of the work,
but announcements, decisions etc should come from them). Note that this
does not mean they are the decision maker or benevolent dictator - they are
just the spokesperson and what they say can be take as the official
position.

- Develop guidelines on when a package or library should be included in
core Emacs i.e. is in the Emacs source code distribution and when it should
be in ELPA.

- Decide if we really need the ability to have some sort of category or
split i.e the 'blessed' and 'neutral' packages (I'm not convinced this is a
good idea).

Some of the above steps will be challenging and likely involve considerable
debate and things may evolve and change over time. However, it would at
least provide a more consistent base from which to develop and expand. Once
this is done, a review of existing ELPA packages and GNU Emacs
libraries/packages could be performed to decide if the current split is
correct. Some things may need to move from ELPA back to Emacs and vice
versa. More importantly, once we have this level of clarity, it should be
easier to decide whether there needs to be more done to inform and educate
the wider user community regarding emacs lisp archives, the roles they
fulfill, address identified discoverability issues, add infrastructure to
encourage/support underlying philosophy etc.


-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 5284 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Date: Wed, 20 May 2020 10:28:42 +1000
> Cc: Emacs developers <Emacs-devel@gnu.org>
> 
> I think the following might help 
> 
> - Define the rationale and objectives that underpins ELPA. Forget about historical context and focus on
> current context. 
> 
> - Define what should be and what should not be included in ELPA. This should be based on objective criteria
> 
> - Define an ELPA custodian responsible for deciding whether a package should or should not be
> added/removed from ELPA. This custodian should be able to appoint a team of people who can assist.
> There will probably need to be some appeal process as well, but I'd keep it simple to begin with. 
> 
> - Ensure the custodian is supported and is seen as the 'official voice'. Any ideas, concerns or need for
> clarification re: policy etc should first be discussed with the ELPA custodian (they may delegate some of the
> work, but announcements, decisions etc should come from them). Note that this does not mean they are the
> decision maker or benevolent dictator - they are just the spokesperson and what they say can be take as the
> official position. 
> 
> - Develop guidelines on when a package or library should be included in core Emacs i.e. is in the Emacs
> source code distribution and when it should be in ELPA. 
> 
> - Decide if we really need the ability to have some sort of category or split i.e the 'blessed' and 'neutral'
> packages (I'm not convinced this is a good idea). 

Sounds like a good plan, thanks.

RE: GNU ELPA package discoverability

[Drew Adams]

+1, all around.

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Is there actually a discoverability problem for GNU ELPA packages? After
  > over 25 years of Emacs use, I think discovering and trying out new packages
  > and features has never been easier. I really don't miss the old days of
  > hunting around various ftp servers trying to find the latest and current
  > version of some library or package.

It sounds like you are especially savvy about searching for packages.
Most users are not so knowledgeable; improving discoverability
wouldbe intended for their sake.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 2501 bytes --]

On Thu, 21 May 2020 at 13:42, Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Is there actually a discoverability problem for GNU ELPA packages?
> After
>   > over 25 years of Emacs use, I think discovering and trying out new
> packages
>   > and features has never been easier. I really don't miss the old days of
>   > hunting around various ftp servers trying to find the latest and
> current
>   > version of some library or package.
>
> It sounds like you are especially savvy about searching for packages.
> Most users are not so knowledgeable; improving discoverability
> wouldbe intended for their sake.
>
>
>
But that is my point. With the introduction of package.el and ELPA (as well
as other 3rd party archives), you don't need to be savvy about
searching for packages. In the old days, yes, this was a constant problem
and time consuming. Now, all you need to do is run M-x list-packages. Each
package is listed with a brief description and if you click on (or hit
enter on) a package, a new window pups up with additional details (longer
description, link to homepage and/or source repository etc). Emacs lisp
packages are more discoverable now than they have ever been. Furthermore,
if I understand correctly, core libs/packages in Emacs can be 'flagged' as
'built-in' which makes them discoverable via M-x list-packages as well.

This is why I think just clarifying the rationale, goals and maintenance
structure for ELPA, developing guidelines for when packages should be in
ELPA and when should not would go a long way to improving the situation.

As this has not been done, people are confused. Developers will turn to
MELPA instead of ELPA simply because the process is clear, fast and
relatively easy. Users will use MELPA because that is what the user
community knows as the 'main' Emacs package archive - it is where they can
find most of the things they need.

Clarify what ELPA is for, how it is managed and what the process is to get
a package into ELPA, then streamline this process and provide an efficient
workflow for package maintenance and many may begin to use ELPA more and
rely on MELPA less. Even more importantly, ELPA might become the definitive
archive for good quality, stable and GPL compliant Emacs packages.

-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 3169 bytes --]

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Clarify what ELPA is for, how it is managed and what the process is to get
  > a package into ELPA, then streamline this process and provide an efficient
  > workflow for package maintenance and many may begin to use ELPA more and
  > rely on MELPA less. Even more importantly, ELPA might become the definitive
  > archive for good quality, stable and GPL compliant Emacs packages.

We are looking at how to do this, within the limits of our basic
goals.  I'm thinking about making a second ELPA which won't require
copyright assignment.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 3062 bytes --]

> I'm thinking about making a second ELPA which won't require copyright
> assignment.

This sounds like an interesting proposal.  A similar system is used for the
package management system of the Arch Linux GNU/Linux distribution:

- core repository: Essential packages for setting up a base system.
- community repository: Packages the community considers essential, adopted
  by a "Trusted User" (TU) to ensure quality standards and ongoing
  maintenance.
- Arch User Repository (AUR): External repository allowing anyone to submit
  packages in source form, these can be installed using an AUR helper
  program (which themselves are part of the AUR).

There are several more repositories in this picture that I didn't mention
for simplicity's sake.  The idea is that new community packages are
submitted to the AUR; if they prove to be sufficiently popular, a user may
contact a TU for inclusion into the community repository.  If the package
doesn't prove to hold their ground, it can be removed from the community
repository and added back to the AUR again.  A TU typically maintains around
10-30 packages.

Here's how I imagine the model to be adapted for our purposes:

- Users familiar with commit access to that "second ELPA" identify important
  community packages and contact their maintainers.
- Packages are reviewed for their overall quality (license, release policy,
  code style, ...).
- Package releases are tested and included in that "second ELPA".
- Packages of exceptional quality may be promoted to GNU ELPA.
- Problematic packages (lack of upstream communication, ...) may be demoted and
  become a regular community package again.

Some crucial points with this proposal:

- The naming is important. GNU ELPA is a bit unfortunate as a name, it
  consists of a license identifier and a technology.  It's not uncommon to
  contract it to just ELPA which causes confusion with the technology.
  Perhaps it's late, but a rename to ensure consistency with the "second
  ELPA" would help with adoption of the proposal.
- There must be some obvious benefit to the "second ELPA" for users.  The
  most obvious one is that there will be a vetted collection of packages
  installable out of the box.  Perhaps more can be added, such as a
  guarantee of their overall quality.  A common issue with
  community-provided ELPA repositories is the lack of quality assurance,
  for example upgrading all installed packages is a gamble, with breakage
  being a common side effect.  If that "second ELPA" avoids this issue,
  that would make for a strong case.
- The exact relationship between GNU ELPA, the "second ELPA" and
  community-provided ELPA repositories needs to be clarified and clearly
  documented in the manuals to help users choose the appropriate
  repositories to install packages from.  For example users caring about
  maximal stability would be advised to stick to the official ELPA
  repositories, but users seeking to install any kind of packages should
  look into community repositories, with the appropriate disclaimers.

Vasilij

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This sounds like an interesting proposal.  A similar system is used for the
  > package management system of the Arch Linux GNU/Linux distribution:

  > - core repository: Essential packages for setting up a base system.
  > - community repository: Packages the community considers essential, adopted
  >   by a "Trusted User" (TU) to ensure quality standards and ongoing
  >   maintenance.
  > - Arch User Repository (AUR): External repository allowing anyone to submit
  >   packages in source form, these can be installed using an AUR helper
  >   program (which themselves are part of the AUR).

If this is simply a matter of three different levels of maintenance, I
see no reason we can't have all three for ELPA packages.  But it is
rather complicated, and managing that will take work.  Having just two
levels may be enough, and it is much simpler.

Arch GNU/Linux does not face the issue of making sure that packages
don't recommend nonfree software.  Indeed, it includes nonfree
programs.  But that issue is part of our core goal.
We cannot follow their example into working against our principles.

From that rough description of the Arch User Repository, I get the
impression it doesn't have any step at which that checking would be
done.  So we would have to add that.

  > - Users familiar with commit access to that "second ELPA" identify important
  >   community packages and contact their maintainers.
  > - Packages are reviewed for their overall quality (license, release policy,
  >   code style, ...).
  > - Package releases are tested and included in that "second ELPA".
  > - Packages of exceptional quality may be promoted to GNU ELPA.

GNU ELPA will require copyright assignment.  It will be only for
packages that, aside from practicalities, are like core Emacs.

  > - The naming is important. GNU ELPA is a bit unfortunate as a name, it
  >   consists of a license identifier and a technology.

GNU is an operating system, and the project to develop it.  I wrote
licenses for it, so we call them "GNU licenses", but those licenses
are not the meaning of the name "GNU".

GNU ELPA is the ELPA that the GNU Project supports.

Nonetheless, I take your point.

    > Perhaps it's late, but a rename to ensure consistency with the "second
    > ELPA" would help with adoption of the proposal.

It is somewhat late, but not impossible if we find a clearly better name.

  > - The exact relationship between GNU ELPA, the "second ELPA" and
  >   community-provided ELPA repositories needs to be clarified and clearly
  >   documented in the manuals to help users choose the appropriate
  >   repositories to install packages from.

If "community-provided ELPA repositories" means those that include
packages that depend on nonfree programs, we can't "work with" that.
The only thing we can legitimately say about them is "don't go there."

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 1639 bytes --]

> Arch GNU/Linux does not face the issue of making sure that packages
> don't recommend nonfree software.  Indeed, it includes nonfree
> programs.  But that issue is part of our core goal.
> We cannot follow their example into working against our principles.

My explanation was solely about the technical aspects of the repositories.
It goes without saying that non-free packages (that is, packages without a
GPL-compatible license) are not welcome.  Even MELPA switched to only
allowing inclusion of free packages and went through the effort of checking
each submitted package for GPL-compatibility, including removal of
non-compliant ones.  Additionally to this their website is fully functional
when accessed in a browser with the LibreJS extension.

> From that rough description of the Arch User Repository, I get the
> impression it doesn't have any step at which that checking would be
> done.  So we would have to add that.

The Arch User Repository would be the equivalent of MELPA.  It is not
something the FSF would be directly concerned with and merely serve as a way
to gauge candidates for inclusion into the "second ELPA".

> If "community-provided ELPA repositories" means those that include packages
> that depend on nonfree programs, we can't "work with" that.  The only thing
> we can legitimately say about them is "don't go there."

This point somewhat surprises me.  By that logic GNU ELPA itself would be
"don't go there" as it hosts the excorporate package [1] which depends on
Microsoft's Exchange service, a non-free Service as a Software Substitute.

Vasilij

[1]: http://elpa.gnu.org/packages/excorporate.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > If "community-provided ELPA repositories" means those that include packages
  > > that depend on nonfree programs, we can't "work with" that.  The only thing
  > > we can legitimately say about them is "don't go there."

  > This point somewhat surprises me.  By that logic GNU ELPA itself would be
  > "don't go there" as it hosts the excorporate package [1] which depends on
  > Microsoft's Exchange service, a non-free Service as a Software Substitute.

Exchange is a service, not a program, it is not well-defined to say it
is "nonfree".  See
https://www.gnu.org/philosophy/network-services-arent-free-or-nonfree.html.

Also, email service is not SaaSS.  See
https://www.gnu.org/philosophy/who-does-that-server-really-serve.html.

The criterion we apply to GNU ELPA is not to refer users (lead or
steer them) to a nonfree program.  A service is a different issue.  We
don't have a rule against packages that communicate with problematical
services.

Should we have one?  Maybe, but that question calls for careful
thought, not haste.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Vasilij Schneidermann]

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

> The criterion we apply to GNU ELPA is not to refer users (lead or
> steer them) to a nonfree program.  A service is a different issue.  We
> don't have a rule against packages that communicate with problematical
> services.

This is interesting.  Does this mean an ELPA package that communicates with
a service to, say, convert a document, is fine?  Would that mean that MELPA
could be made FSF-compatible by splitting it up into a free and non-free
section, similar how Debian uses split package repositories, with the
non-free being opt-in?  Or is that not an option either as soon as even the
existence of a non-free repository is mentioned?

If this is indeed an option, please let me know about the specific criteria
necessary to address and I'll initiate conversation with the MELPA
maintainers.  While their first and foremost idea is enabling greater
convenience for users, they've been receptive to enforcing free licenses and
the LibreJS restrictions for their website.

Vasilij

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 3230 bytes --]

On Fri, 22 May 2020 at 13:11, Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Clarify what ELPA is for, how it is managed and what the process is to
> get
>   > a package into ELPA, then streamline this process and provide an
> efficient
>   > workflow for package maintenance and many may begin to use ELPA more
> and
>   > rely on MELPA less. Even more importantly, ELPA might become the
> definitive
>   > archive for good quality, stable and GPL compliant Emacs packages.
>
> We are looking at how to do this, within the limits of our basic
> goals.  I'm thinking about making a second ELPA which won't require
> copyright assignment.
>
> Which is possibly a good idea. However, I think there are more basic
questions and issues which need addressing first.

You mention 'within the limits of our basic goals'. I don't think this has
been clearly defined and/or does not have consensus. Start by defining the
core rationale and goals for a GNU elisp archive. Whether we do or do not
need a second ELPA is just an implementation detail. First, have clarity
and agreement on what the goals are and then consider the implementation.

Decide whether we need just one or more GNU elisp archives (maybe called
ELPA and XXX, maybe not, maybe actually separate archives or maybe tagged
differently - again, implementation details). Something which will need to
be considered at this point is available maintenance resources. For
example, at a high level, having a stable and reliable archive of elisp
packages which are guaranteed to be GPL compliant and do not reference,
link-to or encourage the use of non-free software, but does not require
assignment of copyright to the FSF may be a worthy goal. However, if the
resources are not available to maintain such an archive and  or
adding/updating code in the archive becomes a slow process because
approvals take too long (because there are insufficient resources), it
simply won't work. A big reason MELPA has been so popular is the overhead
associated with getting code into that repository is low and once in,
maintenance overhead (updates, patching etc) is trivial.

Define guidelines to help developers decide where is the best location for
their library, module, extension etc. After reading these guidelines, a
developer should be confident they now know what is the appropriate
location for their code i.e. Emacs core, a GNU ELPA repository or some
external repository (which does not need to be specified or referenced in
the guidelines, it is just an external repository). These guidelines should
also articulate what the on-going responsibilities and expectations are
with regard to on-going maintenance of the code in each location (Emacs
core, GNU ELPA).

Document process workflow, the steps a developer needs to follow in order
to get their code into the appropriate place and how to manage bugs,
patches, updates and releases. Developers need to understand what they are
committing to before making the commitment.


-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 3920 bytes --]

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > You mention 'within the limits of our basic goals'. I don't think this has
  > been clearly defined 

The goal of the GNU Project is to develop the entirely-free operating
system GNU and rid the world of nonfree software.  It is a very
ambitious goal, but we will push as far as we can, and we will not
give up.

  >  Something which will need to
  > be considered at this point is available maintenance resources. For
  > example, at a high level, having a stable and reliable archive of elisp
  > packages which are guaranteed to be GPL compliant and do not reference,
  > link-to or encourage the use of non-free software, but does not require
  > assignment of copyright to the FSF may be a worthy goal. However, if the
  > resources are not available to maintain such an archive and  or
  > adding/updating code in the archive becomes a slow process because
  > approvals take too long (because there are insufficient resources), it
  > simply won't work.

Since we will put less effort into checking and maintaining these
packages than into GNU ELPA packages, we can surely handle more
packages in it.

How many more?  It isn't useful to worry about.  If this is the
right approach, we will try it and see how many.

  > Document process workflow, the steps a developer needs to follow in order
  > to get their code into the appropriate place and how to manage bugs,

Our usual approach is to try things, develop a practice that works,
then write it down.  Too much advance planning is a pain, and often it
turns out not to be useful.  Once we have real experience we can make
a better plan.

What we do need to write down are the criteria that would make a package
acceptable for each repositor.  We need that stated clearly so that
we make consistent decisions.

  > developer should be confident [perse] now know[s] what is the appropriate
  > location for [per] code

When would that question arise?  In what scenario?



-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 6718 bytes --]

On Sun, 24 May 2020 at 13:53, Richard Stallman <rms@gnu.org> wrote:

>
>   > You mention 'within the limits of our basic goals'. I don't think this
> has
>   > been clearly defined
>
> The goal of the GNU Project is to develop the entirely-free operating
> system GNU and rid the world of nonfree software.  It is a very
> ambitious goal, but we will push as far as we can, and we will not
> give up.
>

Yes, but more specifically, what are the goals of ELPA (and the proposed
ELPA without copyright assignment)? How are these to fit into the GNU Emacs
eco-system? What should go into ELPA, what should go into Emacs 'core'? How
will these ELPA archives work with GNU Emacs releases? (e.g. Do the
packages in these archives need to be compliant with new release, such as
not using functions flagged obsolete, using updated versions of
libs/modules in emacs core etc). Will an Emacs release be held up if there
is a package in ELPA that does not work with new version? When can packages
be updated and what backwards compatibility with older versions of Emacs
should they support? What about platforms - do packages need to support all
the same platforms that Emacs supports? for the archive where copyright is
assigned to the FSF, who is responsible for maintenance and updates? When
should packages be removed?



>
>   >  Something which will need to
>   > be considered at this point is available maintenance resources. For
>   > example, at a high level, having a stable and reliable archive of elisp
>   > packages which are guaranteed to be GPL compliant and do not reference,
>   > link-to or encourage the use of non-free software, but does not require
>   > assignment of copyright to the FSF may be a worthy goal. However, if
> the
>   > resources are not available to maintain such an archive and  or
>   > adding/updating code in the archive becomes a slow process because
>   > approvals take too long (because there are insufficient resources), it
>   > simply won't work.
>
> Since we will put less effort into checking and maintaining these
> packages than into GNU ELPA packages, we can surely handle more
> packages in it.
>
> How many more?  It isn't useful to worry about.  If this is the
> right approach, we will try it and see how many.
>

You seem to be focusing on just the new proposed non-copyright assignment
repository. There is much unclear about the existing repository, including
what should and should not be put there, who is responsible for making
decisions and managing that resource, maintenance and life cycle
maintenance of packages etc.

>
>   > Document process workflow, the steps a developer needs to follow in
> order
>   > to get their code into the appropriate place and how to manage bugs,
>
> Our usual approach is to try things, develop a practice that works,
> then write it down.  Too much advance planning is a pain, and often it
> turns out not to be useful.  Once we have real experience we can make
> a better plan.
>

Which is fine provided it does get done. Has this been done for the
existing ELPA repository? If so, can you point me to it as I'd like to read
it to improve my understanding?

>
>   > developer should be confident [perse] now know[s] what is the
> appropriate
>   > location for [per] code
>
> When would that question arise?  In what scenario?
>

>
To put it back into context, what I wrote was -

 Define guidelines to help developers decide where is the best location for
their library, module, extension etc. After reading these guidelines, a
developer should be confident they now know what is the appropriate
location for their code.

I think this is a common question developers would ask themselves. Simple
scenario -

I have an elisp package which I wrote, which is licensed under the GPL and
which I think others might find useful. I need to decide how to make it
available. The package.el packaging system seems to be a popular choice, I
now need to decide which repository to put it in. My choices are

- GNU ELPA (if I'm prepared to assign copyright to FSF)
- GNU Unassigned Copyright ELPA
- MELPA
- Public archive on my web site

All of these options have pros and cons. However, without adequate
documentation regarding the GNU repositories, workflows and guidelines on
using them and what the commitments and expectations are with respect to
having a package in one  of these two repositories, I'm in the dark. I
cannot assess whether they are appropriate for me. To get clarification,
all i can do is send a message to the list, which too often will end up
in a long thread going back and forth and with little definitive outcome. I
may get some clarification, but likely end up with more questions.

In the end, I decide to initially release through MELPA as it is simple and
quick. I can always change later should things become clearer.

Six months later, I'm still using MELPA. Most of my questions have been
answered, but now I'm use to MELPA and it is working fine and has achieved
my goal, which was simply to make my package available to others who may
find it useful. I'll just leave it as it is. Most Emacs users use MELPA
anyway and while I believe my package could be in an official GNU archive,
the work required to do that is hard to justify, plus I now have other
things to work on.

The above is not an uncommon scenario. I have asked a number of package
owners why they put their package in MELPA rather than GNU ELPA and the
answer generally came down to two things. Number one was a perception it
was just too hard - process was unclear, time it takes too long and there
were just too many unknowns. The second common response was about copyright
assignment - most of the time it wasn't so much about assigning copyright
so much as uncertainty about what that really meant (i.e. and education
issue).

Having a non-copyright assignment repository may help, but only if all the
rest becomes clearer and easier. A 'build it and they will come' approach
is unlikely to work - there is already a solution, MELPA. While that is not
a good solution from a philosophical point of view, it is a fine solution
for many who have released packages that are GPL'd and don't use or
encourage non-free software. Those authors have met their moral obligations
by release their software with a GPL and by avoiding use of or encourage
use of non-free software. The fact other software in the repository is not
compliant is not really their concern. You could argue it should be, but
then you also have to take into account individual capacity and freedoms
and the fact they probably would have used a more complaint repository if
one had been available that did not have too many barriers.


-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 8410 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Date: Sun, 24 May 2020 19:15:46 +1000
> Cc: Emacs developers <Emacs-devel@gnu.org>
> 
>  Define guidelines to help developers decide where is the best location for their library, module, extension
> etc. After reading these guidelines, a developer should be confident they now know what is the appropriate
> location for their code. 
> 
> I think this is a common question developers would ask themselves.

Aren't these questions answered in ELPA's README?

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 2482 bytes --]

On Mon, 25 May 2020 at 00:38, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Tim Cross <theophilusx@gmail.com>
> > Date: Sun, 24 May 2020 19:15:46 +1000
> > Cc: Emacs developers <Emacs-devel@gnu.org>
> >
> >  Define guidelines to help developers decide where is the best location
> for their library, module, extension
> > etc. After reading these guidelines, a developer should be confident
> they now know what is the appropriate
> > location for their code.
> >
> > I think this is a common question developers would ask themselves.
>
> Aren't these questions answered in ELPA's README?
>

No it doesn't. It does answer some of them, but not others (and is probably
not
the right place to answer some of them).

The ELPA README is certainly a start with respect to basic workflow.
Information
which it lacks that a new developer may want to know or which needs
clarification includes

- Who can get push permission to the ELPA git repository?
- How do you request push permission?
- The README is a little weak on process when you don't have push permission
- The instructions re: email to emacs-devel for package developers who do
not
  have push permission does not indicate how to provide the package
  sources/updates. It says that someone will push it for you, but if you
  don't have push permission to the git repo, how do you get your code in
there
  to begin with?

The README does provide reasonable details for someone who has push rights
to
the ELPA repository. However, there is a lack of detail for anyone who does
ot
have push rights and nothing about who can get push rights or the process to
obtain them. (there is also a problem of scale and security with this
model).

The README does state you can add an external repository, which is probably
one
model that could scale better than providing developers with push rights. I
was
quite surprised to see this given all the debate surrounding pull requests
given
that the same basic technique is how pull requests work. I was even more
surprised to see around 80 of these external packages are hosted on github
given
all the discussions about github being evil. Around 30% of the packages in
ELPA
list github as their homepage with the URL: tag.Response to Eli re ELPA
README

Questions about what can/should go into ELPA, what should be included in
Emacs
core and what cannot go into ELPA are not addressed at all (the README is
probably not the right place for this information)
-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 3259 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Date: Mon, 25 May 2020 10:21:47 +1000
> Cc: Richard Stallman <rms@gnu.org>, Emacs developers <Emacs-devel@gnu.org>
> 
>  Aren't these questions answered in ELPA's README?
> 
> No it doesn't. It does answer some of them, but not others (and is probably not
> the right place to answer some of them).
> 
> The ELPA README is certainly a start with respect to basic workflow. Information
> which it lacks that a new developer may want to know or which needs
> clarification includes
> 
> - Who can get push permission to the ELPA git repository?
> - How do you request push permission?
> - The README is a little weak on process when you don't have push permission
> - The instructions re: email to emacs-devel for package developers who do not
>   have push permission does not indicate how to provide the package
>   sources/updates. It says that someone will push it for you, but if you
>   don't have push permission to the git repo, how do you get your code in there
>   to begin with?

So basically what's missing is the write access issue.  That should be
a single sentence: we don't have ELPA write access, only access to the
entire Emacs repository, so they need to request membership in the
Emacs project.

> Questions about what can/should go into ELPA, what should be included in Emacs
> core and what cannot go into ELPA are not addressed at all (the README is
> probably not the right place for this information)

It's quite expected that this is not described, because we are still
arguing about that.

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 2817 bytes --]

On Tue, 26 May 2020 at 01:20, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Tim Cross <theophilusx@gmail.com>
> > Date: Mon, 25 May 2020 10:21:47 +1000
> > Cc: Richard Stallman <rms@gnu.org>, Emacs developers <
> Emacs-devel@gnu.org>
> >
> >  Aren't these questions answered in ELPA's README?
> >
> > No it doesn't. It does answer some of them, but not others (and is
> probably not
> > the right place to answer some of them).
> >
> > The ELPA README is certainly a start with respect to basic workflow.
> Information
> > which it lacks that a new developer may want to know or which needs
> > clarification includes
> >
> > - Who can get push permission to the ELPA git repository?
> > - How do you request push permission?
> > - The README is a little weak on process when you don't have push
> permission
> > - The instructions re: email to emacs-devel for package developers who
> do not
> >   have push permission does not indicate how to provide the package
> >   sources/updates. It says that someone will push it for you, but if you
> >   don't have push permission to the git repo, how do you get your code
> in there
> >   to begin with?
>
> So basically what's missing is the write access issue.  That should be
> a single sentence: we don't have ELPA write access, only access to the
> entire Emacs repository, so they need to request membership in the
> Emacs project.
>

That would certainly be a good start. However, is that a maintainable
approach.

Assume we are successful in getting more packages into ELPA, increasing the
discoverability of appropriate packages without the need to add
repositories like MELPA. Will all of those developers be entitled to write
access to the GNU git repository? What about Richard's proposed
non-copyrighted repository? Perhaps now is the right time to look at the
architecture and consider breaking off ELPA into a separate authentication
realm?

>
> > Questions about what can/should go into ELPA, what should be included in
> Emacs
> > core and what cannot go into ELPA are not addressed at all (the README is
> > probably not the right place for this information)
>
> It's quite expected that this is not described, because we are still
> arguing about that.
>

I think this is an important argument to resolve in order to address the
other issues that have been raised, such as improving package
discoverability or implementation of a non-copyrights assigned repository.
ELPA has existed for quite some time now and we still don't have a clear
definition of what should go into it. How long do we need to argue about it
before making a decision? Is anything being recorded regarding the various
arguments or is it just endless unconnected threads in the mail list? In
other words, how far have we moved towards a consensus?

-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 3877 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Date: Tue, 26 May 2020 10:24:01 +1000
> Cc: Richard Stallman <rms@gnu.org>, Emacs developers <Emacs-devel@gnu.org>
> 
>  So basically what's missing is the write access issue.  That should be
>  a single sentence: we don't have ELPA write access, only access to the
>  entire Emacs repository, so they need to request membership in the
>  Emacs project.
> 
> That would certainly be a good start. However, is that a maintainable approach. 

That's what we have now.  IMO information for contributors should
reflect the present state of affairs.

> Assume we are successful in getting more packages into ELPA, increasing the discoverability of appropriate
> packages without the need to add repositories like MELPA. Will all of those developers be entitled to write
> access to the GNU git repository? What about Richard's proposed non-copyrighted repository? Perhaps now
> is the right time to look at the architecture and consider breaking off ELPA into a separate authentication
> realm? 

If and when the situation changes, we will update the information.  It
is not useful to worry about issues that didn't yet materialize, and
are anybody's guess when they will.

>  > Questions about what can/should go into ELPA, what should be included in Emacs
>  > core and what cannot go into ELPA are not addressed at all (the README is
>  > probably not the right place for this information)
> 
>  It's quite expected that this is not described, because we are still
>  arguing about that.
> 
> I think this is an important argument to resolve in order to address the other issues that have been raised,
> such as improving package discoverability or implementation of a non-copyrights assigned repository. ELPA
> has existed for quite some time now and we still don't have a clear definition of what should go into it. How
> long do we need to argue about it before making a decision? Is anything being recorded regarding the
> various arguments or is it just endless unconnected threads in the mail list? In other words, how far have we
> moved towards a consensus?

I don't know.  A proposal was put on the table, but some of the
important stakeholders didn't yet respond to it.  If you know what
that means and can predict whether, when and how a decision will be
reached, maybe you can advise me which shares to buy to become rich.

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 4959 bytes --]

On Wed, 27 May 2020 at 00:42, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Tim Cross <theophilusx@gmail.com>
> > Date: Tue, 26 May 2020 10:24:01 +1000
> > Cc: Richard Stallman <rms@gnu.org>, Emacs developers <
> Emacs-devel@gnu.org>
> >
> >  So basically what's missing is the write access issue.  That should be
> >  a single sentence: we don't have ELPA write access, only access to the
> >  entire Emacs repository, so they need to request membership in the
> >  Emacs project.
> >
> > That would certainly be a good start. However, is that a maintainable
> approach.
>
> That's what we have now.  IMO information for contributors should
> reflect the present state of affairs.
>

Except it isn't. There is nothing in the README about how to obtain
write/push rights to the repository or that it is the same rights as you
need to add code to the Emacs repository. It also states that if you don't
have the necessary rights, someone will do it all for you, but with no
indication on how to find that someone or provide them with the code. In a
single file package, I guess sending the file to emacs-devel might be
sufficient, but for larger projects that won't work. Relying on 'someone'
to do the work is a vague proposition and will often result in a rather
slow process.

>
> > Assume we are successful in getting more packages into ELPA, increasing
> the discoverability of appropriate
> > packages without the need to add repositories like MELPA. Will all of
> those developers be entitled to write
> > access to the GNU git repository? What about Richard's proposed
> non-copyrighted repository? Perhaps now
> > is the right time to look at the architecture and consider breaking off
> ELPA into a separate authentication
> > realm?
>
> If and when the situation changes, we will update the information.  It
> is not useful to worry about issues that didn't yet materialize, and
> are anybody's guess when they will.
>

I disagree with this approach. All it does is maintain the status quo.

As demonstrated by this thread and others, the current situation is not
working. GNU ELPA has relatively few packages. A far larger number of
packages which could be in a GNU repository are being provided via MELPA,
which also includes packages that do not support the philosophy and goals
of the GNU project. Richard and others are talking about creating an
additional repository which has packages that are philosophically aligned
with the GNU project, but which do not assign copyright to the FSF.  All in
an attempt to reduce the need for users to add a repository which does not
promote core GNU project objectives and to make more appropriate packages
easily to discover without the need to add 3rd party repositories. To
achieve this goal, we need to actively change the situation. A passive 'we
will change it when it is needed' will never work.  If we want people to
consider ELPA as the repository to use, we need to facilitate their ability
to do that.  To make it maintainable, we need to design a solution which
minimises the time burden on those few volunteers prepared to put in the
effort. Improvements and change is something which needs to be driven.

>
> >  > Questions about what can/should go into ELPA, what should be included
> in Emacs
> >  > core and what cannot go into ELPA are not addressed at all (the
> README is
> >  > probably not the right place for this information)
> >
> >  It's quite expected that this is not described, because we are still
> >  arguing about that.
> >
> > I think this is an important argument to resolve in order to address the
> other issues that have been raised,
> > such as improving package discoverability or implementation of a
> non-copyrights assigned repository. ELPA
> > has existed for quite some time now and we still don't have a clear
> definition of what should go into it. How
> > long do we need to argue about it before making a decision? Is anything
> being recorded regarding the
> > various arguments or is it just endless unconnected threads in the mail
> list? In other words, how far have we
> > moved towards a consensus?
>
> I don't know.  A proposal was put on the table, but some of the
> important stakeholders didn't yet respond to it.


How long ago? Perhaps that proposal needs to be res erected? Any proposal
which is just thrown out into the ether is rarely going to achieve
anything. It usually needs a champion that will drive the proposal until an
eventual resolution (which may be for or against - either are valid and
provide a means to move forward).

Do you have a copy of this proposal and are you able to share it?


> If you know what
> that means and can predict whether, when and how a decision will be
> reached, maybe you can advise me which shares to buy to become rich.
>

Maybe I can. My portfolio has done nicely. In fact, it is because of it I
have the time to spend contributing to various open source projects.

-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 6402 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Date: Wed, 27 May 2020 11:06:37 +1000
> Cc: Richard Stallman <rms@gnu.org>, Emacs developers <Emacs-devel@gnu.org>
> 
>  > That would certainly be a good start. However, is that a maintainable approach. 
> 
>  That's what we have now.  IMO information for contributors should
>  reflect the present state of affairs.
> 
> Except it isn't. There is nothing in the README about how to obtain write/push rights to the repository or that
> it is the same rights as you need to add code to the Emacs repository.

Which I agreed should be added.  What is your point in reiterating
that?

The above was in response to a different argument: that describing how
to get write access now is in your opinion not TRT, because that might
change in the future.  Let's keep the argument going forward, not
backward or in circles, please.

>  If and when the situation changes, we will update the information.  It
>  is not useful to worry about issues that didn't yet materialize, and
>  are anybody's guess when they will.
> 
> I disagree with this approach. All it does is maintain the status quo. 

It's okay to disagree.  Status quo is what needs to be described,
because documentation needs to be accurate.

> As demonstrated by this thread and others, the current situation is not working. GNU ELPA has relatively few
> packages.

Because the procedure to get write access is not described?

> To make it maintainable, we need to design a solution which minimises the time burden on those
> few volunteers prepared to put in the effort. Improvements and change is something which needs to be
> driven. 

Talking about what "we need" to do doesn't get anything done,
unfortunately.  Frankly, that's my summary of everything you wrote on
the subject till now.  It helps nothing to lecture what needs to be
done when no one is doing it.

>  I don't know.  A proposal was put on the table, but some of the
>  important stakeholders didn't yet respond to it. 
> 
> How long ago? Perhaps that proposal needs to be res erected? Any proposal which is just thrown out into
> the ether is rarely going to achieve anything. It usually needs a champion that will drive the proposal until an
> eventual resolution (which may be for or against - either are valid and provide a means to move forward). 
> 
> Do you have a copy of this proposal and are you able to share it?

It's all in the archives, please look there.  It wasn't long ago, just
a couple of weeks, so it should be no problem to look it up.

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 4881 bytes --]

On Wed, 27 May 2020 at 12:40, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Tim Cross <theophilusx@gmail.com>
> > Date: Wed, 27 May 2020 11:06:37 +1000
> > Cc: Richard Stallman <rms@gnu.org>, Emacs developers <
> Emacs-devel@gnu.org>
> >
> >  > That would certainly be a good start. However, is that a maintainable
> approach.
> >
> >  That's what we have now.  IMO information for contributors should
> >  reflect the present state of affairs.
> >
> > Except it isn't. There is nothing in the README about how to obtain
> write/push rights to the repository or that
> > it is the same rights as you need to add code to the Emacs repository.
>
> Which I agreed should be added.  What is your point in reiterating
> that?
>

I apologise - I thought you were talking about what is in the README as in
the process is documented.

>
> The above was in response to a different argument: that describing how
> to get write access now is in your opinion not TRT, because that might
> change in the future.  Let's keep the argument going forward, not
> backward or in circles, please.
>

I wasn't saying that it wasn't the right thing or way to do it, merely
asking if it is a maintainable approach.

>
> >  If and when the situation changes, we will update the information.  It
> >  is not useful to worry about issues that didn't yet materialize, and
> >  are anybody's guess when they will.
> >
> > I disagree with this approach. All it does is maintain the status quo.
>
> It's okay to disagree.  Status quo is what needs to be described,
> because documentation needs to be accurate.
>

Yes, understanding the current status quo is important, but not necessarily
just for documentation purposes. The high level objective is to make ELPA a
more comprehensive repository of elisp packages that satisfy GNU project
goals and objectives, enabling easier discovery of said packages without
needing to add 3rd party repositories that may include packages which do
not further the GNU project's goals. Much of what I wrote regarding the
lack of documentation was about how this lack of documentation is an
impediment to developers wanting to add packages to ELPA. Documentation is
only part of the story.

>
> > As demonstrated by this thread and others, the current situation is not
> working. GNU ELPA has relatively few
> > packages.
>
> Because the procedure to get write access is not described?
>

No, this lack of information was simply one example of one aspect of
factors which discourage developers from submitting packages for inclusion
in ELPA. It was mentioned in response to a comment that said the
information developers needed was in the ELPA README file and I listed it
as an example of how the README failed to answer the sort of questions a
developer may have when trying to decide whether to publish a package in
ELPA or some other repository, like MELPA.

>
> > To make it maintainable, we need to design a solution which minimises
> the time burden on those
> > few volunteers prepared to put in the effort. Improvements and change is
> something which needs to be
> > driven.
>
> Talking about what "we need" to do doesn't get anything done,
> unfortunately.  Frankly, that's my summary of everything you wrote on
> the subject till now.  It helps nothing to lecture what needs to be
> done when no one is doing it.
>

Well then I think your summary is poor. I have made suggestions and offered
to do things to try and move matters forward and received no response.

As I wrote on the 25th may -

I'm willing to try and assist here if possible. As a starting point, would
> it be worthwhile starting some discussion threads in emacs-devel for each
> question. I could then try and collate the responses into a single document
> which could be a starting point. For example, I would probably start with
> the question "What criteria should we use to determine if a module,
> library, feature etc go into GNU Emacs core, or GNU ELPA?
> What do others think? I could also have a thread asking for questions
> which should be answered or things which should be documented with respect
> to ELPA.



> >  I don't know.  A proposal was put on the table, but some of the
> >  important stakeholders didn't yet respond to it.
> >
> > How long ago? Perhaps that proposal needs to be res erected? Any
> proposal which is just thrown out into
> > the ether is rarely going to achieve anything. It usually needs a
> champion that will drive the proposal until an
> > eventual resolution (which may be for or against - either are valid and
> provide a means to move forward).
> >
> > Do you have a copy of this proposal and are you able to share it?
>
> It's all in the archives, please look there.  It wasn't long ago, just
> a couple of weeks, so it should be no problem to look it up.
>

I will try to find it. Any clue as to the subject line?

-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 6879 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Date: Wed, 27 May 2020 14:40:08 +1000
> Cc: Richard Stallman <rms@gnu.org>, Emacs developers <Emacs-devel@gnu.org>
> 
>  It's all in the archives, please look there.  It wasn't long ago, just
>  a couple of weeks, so it should be no problem to look it up.
> 
> I will try to find it. Any clue as to the subject line?

Sorry, I don't remember.  They were buried in the longish threads
about ELPA.

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 781 bytes --]

On Wed, 27 May 2020 at 12:40, Eli Zaretskii <eliz@gnu.org> wrote:

> >
> > Do you have a copy of this proposal and are you able to share it?
>
> It's all in the archives, please look there.  It wasn't long ago, just
> a couple of weeks, so it should be no problem to look it up.
>

I have been back through the archives back to mid April and cannot find
anything that looks like a proposal. There are numerous threads discussing
why we have elpa, what conventions/rules should be enforced, rationale for
having elpa, etc, but nothing which comes to  any conclusion or anything
close to a proposal or call for decisions or any other actions, like
feedback or further suggestions. Either it has a non-obvious subject or is
much further back in time.

-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 1297 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Date: Wed, 27 May 2020 15:39:49 +1000
> Cc: Richard Stallman <rms@gnu.org>, Emacs developers <Emacs-devel@gnu.org>
> 
> I have been back through the archives back to mid April and cannot find anything that looks like a proposal.
> There are numerous threads discussing why we have elpa, what conventions/rules should be enforced,
> rationale for having elpa, etc, but nothing which comes to  any conclusion or anything close to a proposal or
> call for decisions or any other actions, like feedback or further suggestions. Either it has a non-obvious
> subject or is much further back in time. 

The proposal was to split ELPA into two, either insided the single
archive or make the second part be hosted on a different server.

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 1231 bytes --]

On Thu, 28 May 2020 at 01:45, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Tim Cross <theophilusx@gmail.com>
> > Date: Wed, 27 May 2020 15:39:49 +1000
> > Cc: Richard Stallman <rms@gnu.org>, Emacs developers <
> Emacs-devel@gnu.org>
> >
> > I have been back through the archives back to mid April and cannot find
> anything that looks like a proposal.
> > There are numerous threads discussing why we have elpa, what
> conventions/rules should be enforced,
> > rationale for having elpa, etc, but nothing which comes to  any
> conclusion or anything close to a proposal or
> > call for decisions or any other actions, like feedback or further
> suggestions. Either it has a non-obvious
> > subject or is much further back in time.
>
> The proposal was to split ELPA into two, either insided the single
> archive or make the second part be hosted on a different server.
>

OK, so it was really focusing on the lower level technical implementation
rather than clarifying what the purpose of GNU ELPA is, relationship with
GNU Emacs and what should be in either or both? It is probably too early to
consider such matters, especially when things like a non-copyrights
repository are being considered.
-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 1939 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Tim Cross <theophilusx@gmail.com>
> Date: Thu, 28 May 2020 05:41:06 +1000
> Cc: Richard Stallman <rms@gnu.org>, Emacs developers <Emacs-devel@gnu.org>
> 
>  The proposal was to split ELPA into two, either insided the single
>  archive or make the second part be hosted on a different server.
> 
> OK, so it was really focusing on the lower level technical implementation rather than clarifying what the
> purpose of GNU ELPA is, relationship with GNU Emacs and what should be in either or both? It is probably
> too early to consider such matters, especially when things like a non-copyrights repository are being
> considered. 

Are you kidding? we just had several longish threads discussing these
very matters.

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > OK, so it was really focusing on the lower level technical implementation
  > rather than clarifying what the purpose of GNU ELPA is, relationship with
  > GNU Emacs and what should be in either or both?

Yes, we should document that clearly -- but since we are reconsidering
it now, let's finish deciding and document that decision.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Except it isn't. There is nothing in the README about how to obtain
  > write/push rights to the repository or that it is the same rights as you
  > need to add code to the Emacs repository. It also states that if you don't
  > have the necessary rights, someone will do it all for you, but with no
  > indication on how to find that someone or provide them with the code. In a
  > single file package, I guess sending the file to emacs-devel might be
  > sufficient, but for larger projects that won't work. Relying on 'someone'
  > to do the work is a vague proposition and will often result in a rather
  > slow process.

I agree that we should document how to participate in developing
packages in ELPA.  But this is a long term need, not an urgent need.

Since we are considering a big change in how people would do that,
why document the current system?  We may as well change it, get
the new system working well, and document that one.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 2743 bytes --]

On Wed, 27 May 2020 at 13:19, Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Except it isn't. There is nothing in the README about how to obtain
>   > write/push rights to the repository or that it is the same rights as
> you
>   > need to add code to the Emacs repository. It also states that if you
> don't
>   > have the necessary rights, someone will do it all for you, but with no
>   > indication on how to find that someone or provide them with the code.
> In a
>   > single file package, I guess sending the file to emacs-devel might be
>   > sufficient, but for larger projects that won't work. Relying on
> 'someone'
>   > to do the work is a vague proposition and will often result in a rather
>   > slow process.
>
> I agree that we should document how to participate in developing
> packages in ELPA.  But this is a long term need, not an urgent need.
>
> Since we are considering a big change in how people would do that,
> why document the current system?  We may as well change it, get
> the new system working well, and document that one.
>
> --
> Dr Richard Stallman
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)
>
>
>
I totally agree. I think we should focus on what we want more than what we
have. We can then look at the gap between what is and what we want and
develop plans to get us from here to there.

I do think we need to clarify what should go into ELPA, what should go into
Emacs core and what is not compatible with either (I include in ELPA the
possible non-copyright assigned repository, even though it may be a
completely separate repository). Much of the 'higher level' criteria is
known (for example, GPL'd, not rely on or encourage use of non-free
software etc) and some has contention (i.e. ELPA v Emacs core). Eli has
made reference to a previous proposal, which might be a good staring point
if someone has a copy.  Once we know how ELPA will be used, we can
determine the best way to structure it so that developers can submit new
packages for inclusion and maintain these packages such that barriers to
developers are minimised while maintaining the level of control the GNU
project requires without imposing too much additional burden on those who
volunteer to help maintain the repository. Overall object would likely be
something along the lines of making GNU ELPA a viable first choice for
appropriate packages over MELPA.


-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 3633 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: eliz@gnu.org, Emacs-devel@gnu.org
> Date: Tue, 26 May 2020 23:19:51 -0400
> 
> Since we are considering a big change in how people would do that,
> why document the current system?  We may as well change it, get
> the new system working well, and document that one.

Since the process of changing might take some time, I see no reason
not to have accurate information in the meantime.

But I think arguing about this is a bit academic as long as no one
does anything to make the changes.

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]

On Thu, 28 May 2020 at 01:39, Eli Zaretskii <eliz@gnu.org> wrote:

>
> But I think arguing about this is a bit academic as long as no one
> does anything to make the changes.
>

True, but before you can make change you need to agree on what the changes
are. At this stage, we don't even seem to have agreement on what the
relationship is between GNU ELPA, GNU Emacs and what should be in Emacs
core, in ELPA or in both., whether some packages should be seen as
'blessed' by the GNU project, what level of compliance with GNU coding
styles and conventions is required, how and where bug reports should be
managed for ELPA packages etc. Many of these questions are related and
answering some of them will either answer others or limit the available
options.

I also wouldn't call what has been in this thread as arguing, but rather
discussing in order to gain understanding, identify issues and
opportunities and hopefully find appropriate courses of action which
benefit the GNU project.

regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 1550 bytes --]

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So basically what's missing is the write access issue.  That should be
  > a single sentence: we don't have ELPA write access, only access to the
  > entire Emacs repository, so they need to request membership in the
  > Emacs project.

That's how it is now, but we could change it.  For instance, we could
make a Savannah project for each package in GNU ELPA, give access to
the Emacs maintainers plus the developers of the package, and make
releases of GNU ELPA by collecting all those packages.

Would that be better?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Michael Albinus]

Richard Stallman <rms@gnu.org> writes:

> For instance, we could make a Savannah project for each package in GNU
> ELPA, give access to the Emacs maintainers plus the developers of the
> package, and make releases of GNU ELPA by collecting all those
> packages.

For Tramp, this would be a problem. It lives on savannah for 20+ years
as own project already. One year ago I've started to distribute it also
on GNU ELPA, and since there are some problems how the GNU ELPA archive
expects a project layout (bug#32544), I always do a transition from the
Tramp repo to the GNU ELPA repo, when releasing a new Tramp
version. This wouldn't be possible any longer, if GNU ELPA releases are
collected automatically from a project's repo on savannah.

I don't know how many other packages would run into the same problem;
maybe Tramp is just an exception.

Best regards, Michael.

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: theophilusx@gmail.com, Emacs-devel@gnu.org
> Date: Tue, 26 May 2020 00:15:02 -0400
> 
>   > So basically what's missing is the write access issue.  That should be
>   > a single sentence: we don't have ELPA write access, only access to the
>   > entire Emacs repository, so they need to request membership in the
>   > Emacs project.
> 
> That's how it is now, but we could change it.  For instance, we could
> make a Savannah project for each package in GNU ELPA, give access to
> the Emacs maintainers plus the developers of the package, and make
> releases of GNU ELPA by collecting all those packages.

If and when we change the situation, we will update the README
accordingly.  That file needs to be maintained.

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Yes, but more specifically, what are the goals of ELPA (and the proposed
  > ELPA without copyright assignment)? How are these to fit into the GNU Emacs
  > eco-system?

For us, the question is how they fit into the GNU Project and with its
goals of freedom.  However, most of the specific questions you brought
up are mostly practical.

You posed one question has an answer that comes from a nontechnical basis:

  > What about platforms - do packages need to support all
  > the same platforms that Emacs supports?

Each package, and each feature in each package, must support the GNU
system, and in particular GNU/Linux.  Support for any other systems,
including Windows and MacOS, is optional.

The rest of these questions you posed are purely practical.

  >  What should go into ELPA, what should go into Emacs 'core'? How
  > will these ELPA archives work with GNU Emacs releases? (e.g. Do the
  > packages in these archives need to be compliant with new release, such as
  > not using functions flagged obsolete, using updated versions of
  > libs/modules in emacs core etc). Will an Emacs release be held up if there
  > is a package in ELPA that does not work with new version? When can packages
  > be updated and what backwards compatibility with older versions of Emacs
  > should they support? ... for the archive where copyright is
  > assigned to the FSF, who is responsible for maintenance and updates? When
  > should packages be removed?

and the Emacs developers (talking with this list)
can work out good practical answers.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 3061 bytes --]

On Mon, 25 May 2020 at 14:36, Richard Stallman <rms@gnu.org> wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Yes, but more specifically, what are the goals of ELPA (and the
> proposed
>   > ELPA without copyright assignment)? How are these to fit into the GNU
> Emacs
>   > eco-system?
>
> For us, the question is how they fit into the GNU Project and with its
> goals of freedom.  However, most of the specific questions you brought
> up are mostly practical.
>
> You posed one question has an answer that comes from a nontechnical basis:
>
>   > What about platforms - do packages need to support all
>   > the same platforms that Emacs supports?
>
> Each package, and each feature in each package, must support the GNU
> system, and in particular GNU/Linux.  Support for any other systems,
> including Windows and MacOS, is optional.
>
> That seems clear and makes sense.


> The rest of these questions you posed are purely practical.
>
>   >  What should go into ELPA, what should go into Emacs 'core'? How
>   > will these ELPA archives work with GNU Emacs releases? (e.g. Do the
>   > packages in these archives need to be compliant with new release, such
> as
>   > not using functions flagged obsolete, using updated versions of
>   > libs/modules in emacs core etc). Will an Emacs release be held up if
> there
>   > is a package in ELPA that does not work with new version? When can
> packages
>   > be updated and what backwards compatibility with older versions of
> Emacs
>   > should they support? ... for the archive where copyright is
>   > assigned to the FSF, who is responsible for maintenance and updates?
> When
>   > should packages be removed?
>
> and the Emacs developers (talking with this list)
> can work out good practical answers.
>

I think getting some initial answers/position on some of these could help.
For the type of packages we would like under a GNU managed archive, we need
to make that processes as easy and straight-forward as possible for both
the developers and those who perform critical maintenance processes.

I'm willing to try and assist here if possible. As a starting point, would
it be worthwhile starting some discussion threads in emacs-devel for each
question. I could then try and collate the responses into a single document
which could be a starting point. For example, I would probably start with
the question "What criteria should we use to determine if a module,
library, feature etc go into GNU Emacs core, or GNU ELPA?

What do others think? I could also have a thread asking for questions which
should be answered or things which should be documented with respect to
ELPA.

>
> --
> Dr Richard Stallman
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)
>
>
>

-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 4453 bytes --]

Re: GNU ELPA package discoverability

[Bastien]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > Better in which sense?  Do you mean, better for you in maintaining the
>   > > package?
>
>   > As explained in another email: better discoverability. Better at helping 
>   > the users notice the package.
>
>   > Simply including a file in the Emacs distro doesn't do much.
>
> Do people have any concrete suggestions for how to improve this?

I think advertizing GNU ELPA on the Emacs splash screen would be a
good first step.

-- 
 Bastien

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Date: Sun, 24 May 2020 09:34:14 +0200
> Cc: Emacs-devel@gnu.org
> 
> I think advertizing GNU ELPA on the Emacs splash screen would be a
> good first step.

Any practical suggestions for how to do that in the little screen
estate we have there?

Thanks.

Re: GNU ELPA package discoverability

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 531 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Bastien <bzg@gnu.org>
>> Date: Sun, 24 May 2020 09:34:14 +0200
>> Cc: Emacs-devel@gnu.org
>> 
>> I think advertizing GNU ELPA on the Emacs splash screen would be a
>> good first step.
>
> Any practical suggestions for how to do that in the little screen
> estate we have there?

I attach a patch against lisp/startup.el.

I don't really know how "little" is the screen estate in other
configurations, so perhaps this suggestion is not okay, but it
fits quite well here.

-- 
 Bastien

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: startup.el.patch --]
[-- Type: text/x-diff, Size: 619 bytes --]

diff --git a/lisp/startup.el b/lisp/startup.el
index bff10003f8..2db774a4e1 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -1601,6 +1601,8 @@ fancy-startup-text
      "\tOverview of Emacs features at gnu.org\n"
      :link ("View Emacs Manual" ,(lambda (_button) (info-emacs-manual)))
      "\tView the Emacs manual using Info\n"
+     :link ("Install Emacs packages" ,(lambda (_button) (list-packages)))
+     "\tBrowse the Emacs Lisp Package Archive\n"
      :link ("Absence of Warranty" ,(lambda (_button) (describe-no-warranty)))
      "\tGNU Emacs comes with "
      :face (variable-pitch (:slant oblique))

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Cc: rms@gnu.org,  Emacs-devel@gnu.org
> Date: Sun, 24 May 2020 18:16:03 +0200
> 
> diff --git a/lisp/startup.el b/lisp/startup.el
> index bff10003f8..2db774a4e1 100644
> --- a/lisp/startup.el
> +++ b/lisp/startup.el
> @@ -1601,6 +1601,8 @@ fancy-startup-text
>       "\tOverview of Emacs features at gnu.org\n"
>       :link ("View Emacs Manual" ,(lambda (_button) (info-emacs-manual)))
>       "\tView the Emacs manual using Info\n"
> +     :link ("Install Emacs packages" ,(lambda (_button) (list-packages)))
> +     "\tBrowse the Emacs Lisp Package Archive\n"
>       :link ("Absence of Warranty" ,(lambda (_button) (describe-no-warranty)))
>       "\tGNU Emacs comes with "
>       :face (variable-pitch (:slant oblique))

Thanks, this will need an extra line.  Perhaps we should delete the
empty line below the image which we display at the beginning of the
splash screen.

Another question is whether installing extra packages is indeed
something to be shown on the splash screen, i.e. a newbie will see it
as long as he/she doesn't disable it.

Re: GNU ELPA package discoverability

[Yuri Khan]

On Sun, 24 May 2020 at 23:38, Eli Zaretskii <eliz@gnu.org> wrote:

> Another question is whether installing extra packages is indeed
> something to be shown on the splash screen, i.e. a newbie will see it
> as long as he/she doesn't disable it.

Another consideration is that

    list-packages is not on any key;
     its alias package-list-packages is on <menu-bar> <options> <package>

that is, the command is already made discoverable via the menu. As
long as the newbie does not disable it, that is.

Re: GNU ELPA package discoverability

[Bastien]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks, this will need an extra line.  Perhaps we should delete the
> empty line below the image which we display at the beginning of the
> splash screen.

A matter of taste, of course, but I don't mind the extra line below
the picture.

> Another question is whether installing extra packages is indeed
> something to be shown on the splash screen, i.e. a newbie will see it
> as long as he/she doesn't disable it.

The splash screen is a way to advertize the "World of Emacs" to users:
IMHO a resource like ELPA fits well with other external resources like
the GNU/Linux page.

To align the message with the menu entry, the link can say "Manage
Emacs packages" while the description would say "Install or uninstall
additional Emacs packages".

-- 
 Bastien

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Cc: rms@gnu.org,  Emacs-devel@gnu.org
> Date: Sun, 24 May 2020 19:01:41 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Thanks, this will need an extra line.  Perhaps we should delete the
> > empty line below the image which we display at the beginning of the
> > splash screen.
> 
> A matter of taste, of course, but I don't mind the extra line below
> the picture.

Not a matter of taste, but a matter of showing everything we do today,
plus the additional line you propose.  We don't want to have the last
line or two we currently show on the slash screen to wind up below the
window bottom, thus invisible.

Re: GNU ELPA package discoverability

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> Not a matter of taste, but a matter of showing everything we do today,
> plus the additional line you propose.  We don't want to have the last
> line or two we currently show on the slash screen to wind up below the
> window bottom, thus invisible.

Sure.  I am attaching a screenshot showing how emacs -q looks like on
my computer, with a lot of free space (but again, that may be just me).

-- 
 Bastien

[-- Attachment #2: 2020-05-25-082731_1920x1080_scrot.png --]
[-- Type: image/png, Size: 163936 bytes --]

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Cc: rms@gnu.org,  Emacs-devel@gnu.org
> Date: Mon, 25 May 2020 08:29:00 +0200
> 
> > Not a matter of taste, but a matter of showing everything we do today,
> > plus the additional line you propose.  We don't want to have the last
> > line or two we currently show on the slash screen to wind up below the
> > window bottom, thus invisible.
> 
> Sure.  I am attaching a screenshot showing how emacs -q looks like on
> my computer, with a lot of free space (but again, that may be just me).

Looks like you have an unusually large screen.  On my system, the
current text takes the entire window height.  This is why I proposed
to delete the empty line below the GNU Emacs image.

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 25.05.2020 17:37, Eli Zaretskii wrote:
> Looks like you have an unusually large screen.  On my system, the
> current text takes the entire window height.  This is why I proposed
> to delete the empty line below the GNU Emacs image.

Should we decide on and document the required maximum height in lines?

I have a big screen, but Emacs starts up in windowed mode by default 
(not fullscreen), and the height of that window is 33 lines, if both the 
menu bar and the toolbar are enabled.

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> Cc: rms@gnu.org, Emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 25 May 2020 17:42:19 +0300
> 
> On 25.05.2020 17:37, Eli Zaretskii wrote:
> > Looks like you have an unusually large screen.  On my system, the
> > current text takes the entire window height.  This is why I proposed
> > to delete the empty line below the GNU Emacs image.
> 
> Should we decide on and document the required maximum height in lines?

Required from whom or from what?

> I have a big screen, but Emacs starts up in windowed mode by default 
> (not fullscreen), and the height of that window is 33 lines, if both the 
> menu bar and the toolbar are enabled.

34 lines here, not counting the mode line.

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 25.05.2020 18:09, Eli Zaretskii wrote:
> Required from whom or from what?

Required by you (for example), for the splash screen not to exceed.

 > 34 lines here, not counting the mode line.

OK.

FWIW, the "Auto-save file lists were found" line is barely visible on my 
machine, and the "M-x recover-session RET" is fully concealed.

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> Cc: bzg@gnu.org, rms@gnu.org, Emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 25 May 2020 18:12:33 +0300
> 
> On 25.05.2020 18:09, Eli Zaretskii wrote:
> > Required from whom or from what?
> 
> Required by you (for example), for the splash screen not to exceed.

That's what I just did, I suppose.

>  > 34 lines here, not counting the mode line.
> 
> OK.
> 
> FWIW, the "Auto-save file lists were found" line is barely visible on my 
> machine, and the "M-x recover-session RET" is fully concealed.

That means we have a bug.

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 25.05.2020 19:06, Eli Zaretskii wrote:
>>   > 34 lines here, not counting the mode line.
>>
>> OK.
>>
>> FWIW, the "Auto-save file lists were found" line is barely visible on my
>> machine, and the "M-x recover-session RET" is fully concealed.
> That means we have a bug.

I guess.

But we might as well switch to the new layout before trying to fix it.

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 25.05.2020 19:06, Eli Zaretskii wrote:
>>   > 34 lines here, not counting the mode line.
>>
>> OK.
>>
>> FWIW, the "Auto-save file lists were found" line is barely visible on my
>> machine, and the "M-x recover-session RET" is fully concealed.
> That means we have a bug.

By the way, are those 34 lines on your machine restricted by the total 
height of the screen, or is it just in windowed mode, and we could make 
the initial size bigger on Emacs' side?

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> Cc: bzg@gnu.org, rms@gnu.org, Emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 27 May 2020 23:33:03 +0300
> 
> By the way, are those 34 lines on your machine restricted by the total 
> height of the screen, or is it just in windowed mode, and we could make 
> the initial size bigger on Emacs' side?

It's the size of the Emacs window.  The screen is much larger.

Re: GNU ELPA package discoverability

[Sacha Chua]

Bastien <bzg@gnu.org> writes:

> The splash screen is a way to advertize the "World of Emacs" to users:
> IMHO a resource like ELPA fits well with other external resources like
> the GNU/Linux page.
> To align the message with the menu entry, the link can say "Manage
> Emacs packages" while the description would say "Install or uninstall
> additional Emacs packages".

I imagine that a newbie getting into Emacs because of things like Magit
would be excited to see the splash screen or menu item for "Manage Emacs
packages," but might be confused by the short list of packages. They
might think: "Is that all there is? Is my Emacs broken? Is my setup
incompatible with those packages?" 

Here's a Reddit post from two years ago showing what that confusion
might look like:
https://www.reddit.com/r/emacs/comments/8eweus/installing_a_package_manually/

I wonder if there's a way we could make package-archives customization
more discoverable from the list-packages screen. Package Archives is in
the Package > Customize list as the third option, after two options that
would probably make newbies' eyes glaze over, and it doesn't suggest
values to add. It also doesn't make it clear that you should run
package-refresh-contents after changing the value, so that might trip up
a few people if they expect to see new changes right away.

It might be interesting to have something like the checkboxes Ubuntu
uses ( https://help.ubuntu.com/community/Repositories/Ubuntu ) so that
people can quickly select from among the main repositories, with a way
to add their own. I'm not sure how that fits into the Customize
interface, though...

Sacha Chua

Re: GNU ELPA package discoverability

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 1453 bytes --]

Hi Sacha,

thanks for the link to the discussion, it's always good to get such
feedback from users!

I don't know if we can make it easier to customize package-archives
from the view the user get after M-x list-packages RET.

I suggest to try enhancing the discoverability "upstream", by updating
the default Emacs menu bar.

For now, package is present in two menus: [Options] and [Help].

  [Options] contains "Manage Emacs Packages" which M-x list-packages,
  so it does not solve the problem you are insisting on.

  [Help] contains "Search Built-in Packages" and "Find extra packages"
  which leads to an Info page, not directly to the customize menu.

I suggest the following patch, which moves "Manage Emacs Packages"
from the [Options] menu to the [Tools] menu and add these subitems:

- List packages
- Customize package archives (<= the option you want to promote)
- Customize the package interface

I think package is more a "tool" than an "option", so it makes sense
to move it there.  Also, as you mention, just listing packages leaves
aside the possibility to customize the package sources, which is what
most users may be looking for.

I think "Search Built-in Packages" could also be moved to this new
menu, while "Find extra packages" could stay in the [Help] menu,
because -- but I don't have a strong opinion on those.

I attach a (rough) patch for this idea, let me know if it sounds
like an improvement to you!

Best,

-- 
 Bastien

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: menu-bar.el.patch --]
[-- Type: text/x-diff, Size: 2017 bytes --]

diff --git a/lisp/menu-bar.el b/lisp/menu-bar.el
index 9bc667acd6..0c26b415b6 100644
--- a/lisp/menu-bar.el
+++ b/lisp/menu-bar.el
@@ -1414,10 +1414,6 @@ menu-bar-options-menu
     (bindings--define-key menu [customize]
       `(menu-item "Customize Emacs" ,menu-bar-custom-menu))
 
-    (bindings--define-key menu [package]
-      '(menu-item "Manage Emacs Packages" package-list-packages
-        :help "Install or uninstall additional Emacs packages"))
-
     (bindings--define-key menu [save]
       '(menu-item "Save Options" menu-bar-options-save
                   :help "Save options set from the menu above"))
@@ -1682,6 +1678,22 @@ menu-bar-shell-commands-menu
 
     menu))
 
+(defvar menu-bar-package-menu
+  (let ((menu (make-sparse-keymap "Manage Emacs Packages")))
+    (bindings--define-key menu [customize-package]
+      '(menu-item "Customize the package tool"
+                  (lambda () (interactive) (customize-group "package"))
+                  :help "Customize the package interface"))
+    (bindings--define-key menu [customize-package-archives]
+      '(menu-item "Customize package archives"
+                  (lambda () (interactive) (customize-variable 'package-archives))
+                  :help "Set the list of Emacs package sources"))
+    (bindings--define-key menu [list-packages]
+      '(menu-item "List packages" list-packages
+                  :help "Browse a list of Emacs packages"))
+
+    menu))
+
 (defun menu-bar-read-mail ()
   "Read mail using `read-mail-command'."
   (interactive)
@@ -1716,6 +1728,13 @@ menu-bar-tools-menu
     (bindings--define-key menu [separator-net]
       menu-bar-separator)
 
+    (bindings--define-key menu [package]
+      `(menu-item "Manage Emacs Packages"
+                  ,menu-bar-package-menu))
+
+    (bindings--define-key menu [separator-package]
+      menu-bar-separator)
+
     (bindings--define-key menu [browse-web]
       '(menu-item "Browse the Web..." browse-web))
     (bindings--define-key menu [directory-search]

RE: GNU ELPA package discoverability

[Drew Adams]

> I suggest to try enhancing the discoverability "upstream", by updating
> the default Emacs menu bar.
> 
> For now, package is present in two menus: [Options] and [Help].
> 
>   [Options] contains "Manage Emacs Packages" which M-x
>   list-packages, so it does not solve the problem you
>   are insisting on.

IMO, `Manage Emacs Packages' should never have been
added to menu `Options'.  That menu is essentially for
setting user options (especially Boolean ones).  And
it shouldn't be called "Manage" anything (weasel word).

That command (`list-packages') is akin to commands
`list-faces-display' and `list-colors-display'.  Those
are in submenu `Edit > Text Properties', as `Display
Faces' and `Display Colors'.

Do they belong there?  Not really.  IMO, such lists
belong under `Help > Describe', just like `List Key
Bindings'.  Their first-order use is for listing stuff.
(Secondarily, you can get more info about the stuff
listed by clicking parts of the display.)

Similarly, `Edit > Text Properties > Display Properties'
is akin to the various items in `Help > Describe'.

I'd suggest moving all such to `Help > Describe'.  And
all that are about listing stuff should have similar
names: `List XYZ' or `Display XYZ', but not a mix of
`List' and `Display'. 

Menu `Help' should be used for obtaining information
about stuff, which can include exploring/discovering
stuff.  It's not primarily for accessing stuff, to
use it, or for changing stuff.

`Tools' is OK for accessing/using stuff that doesn't
fit elsewhere (e.g. under `Edit').  It's not the place
for finding out about packages etc.

(FWIW, in my `menu-bar+.el' there's a top-level
`Search' menu, which has the `Search' and `Replace'
submenus from `Edit'.  And `Edit' submenus `Go To' and
`Bookmarks' are instead on `Files' - they're not about
editing.)

`Help > Search Built-in Packages' is misnamed.  It's
not about searching.  The doc string for that command
is "Find packages matching a given keyword."

It's not obvious from the name that what gets listed
by it are not packages but categories of packages (by
keyword).  (And from there you can find lists of
packages per keyword.)  It should be called something
like `Packages By Category' or `Describe Packages By
Keyword'.

`Help > Describe' already has `Describe Package...'.
That submenu is also the place for `finder-by-keyword'
(`Describe Packages By Keyword').

`Help > Finding Extra Packages' should be called
something like `Getting More Packages', and it should
be placed in the same part of menu `Help' as `Getting
New Versions' (which should be called `Getting New
Emacs Versions').

Re: GNU ELPA package discoverability

[Sacha Chua]

Bastien <bzg@gnu.org> writes:

Hello, Bastien, all!

> I suggest the following patch, which moves "Manage Emacs Packages"
> from the [Options] menu to the [Tools] menu and add these subitems:
> - List packages
> - Customize package archives (<= the option you want to promote)
> - Customize the package interface

Oh, Tools makes sense too.

Hmm... What if the submenu has:

- Manage package sources     <-- move this one first?
- Choose packages          
- Customize the package interface
- ... Maybe that help link?

(I was going to suggest a second item that updates the package contents,
but I just realized that package-list-packages refreshes the list before
displaying, so that's something people don't actually have to worry
about. Here I've been refreshing it before listing, duh!)

Maybe in the customize help for package-archives, we can mention the
info node for "Finding extra packages" or directly list a few URLs so
that people don't have to go back and forth copying.

I'm sort of split on the wording of "Manage package sources". Package
archives is the name of the variable, so that's probably not going to
change, but an archive might also be conceptually equivalent to a single
package. "Package repositories" or "package sources" might be more
similar to the words used by other distributions. 

"Choose packages" is a little more action-oriented than "List packages",
but it moves away from what the command is called, so I'm not sure about
that either. Staying close to the existing names has lots of benefits,
after all.

Anyway, making package-archives a little closer to list-packages is
probably the main thing.

Sacha Chua

Re: GNU ELPA package discoverability

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 869 bytes --]

Hi Sacha,

thanks for the feedback!

Sacha Chua <sacha@sachachua.com> writes:

> Oh, Tools makes sense too.

Okay.

> Hmm... What if the submenu has:
>
> - Manage package sources     <-- move this one first?

I agree this is better (more explicit) than "Manage package archives".

> - Choose packages

I'd rather go for "List available packages" as "Choose packages" feels
like the user has to do something with the listed packages while "List
available packages" feels more factual.  But not a big deal, both are
good.

> - Customize the package interface

As Dmitry said, maybe this one is not really needed.
 
> - ... Maybe that help link?

I'm not sure here -- maybe we can modify the help menu in another
patch?

I've updated my patch to modify the menu according to the suggestion
above.

Eli, let me know if I can apply it against master.

Thanks,

-- 
 Bastien

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: menu-bar.el.patch --]
[-- Type: text/x-diff, Size: 1792 bytes --]

diff --git a/lisp/menu-bar.el b/lisp/menu-bar.el
index 9bc667acd6..6340af08e9 100644
--- a/lisp/menu-bar.el
+++ b/lisp/menu-bar.el
@@ -1414,10 +1414,6 @@ menu-bar-options-menu
     (bindings--define-key menu [customize]
       `(menu-item "Customize Emacs" ,menu-bar-custom-menu))
 
-    (bindings--define-key menu [package]
-      '(menu-item "Manage Emacs Packages" package-list-packages
-        :help "Install or uninstall additional Emacs packages"))
-
     (bindings--define-key menu [save]
       '(menu-item "Save Options" menu-bar-options-save
                   :help "Save options set from the menu above"))
@@ -1682,6 +1678,18 @@ menu-bar-shell-commands-menu
 
     menu))
 
+(defvar menu-bar-package-menu
+  (let ((menu (make-sparse-keymap "Manage Emacs Packages")))
+    (bindings--define-key menu [customize-package-archives]
+      '(menu-item "Customize package sources"
+                  (lambda () (interactive) (customize-variable 'package-archives))
+                  :help "Set the list of Emacs package sources"))
+    (bindings--define-key menu [list-packages]
+      '(menu-item "List available packages" list-packages
+                  :help "Browse a list of Emacs packages"))
+
+    menu))
+
 (defun menu-bar-read-mail ()
   "Read mail using `read-mail-command'."
   (interactive)
@@ -1716,6 +1724,13 @@ menu-bar-tools-menu
     (bindings--define-key menu [separator-net]
       menu-bar-separator)
 
+    (bindings--define-key menu [package]
+      `(menu-item "Manage Emacs Packages"
+                  ,menu-bar-package-menu))
+
+    (bindings--define-key menu [separator-package]
+      menu-bar-separator)
+
     (bindings--define-key menu [browse-web]
       '(menu-item "Browse the Web..." browse-web))
     (bindings--define-key menu [directory-search]

Re: GNU ELPA package discoverability

[Stefan Kangas]

Bastien <bzg@gnu.org> writes:

> +      '(menu-item "Customize package sources"
> +                  (lambda () (interactive) (customize-variable 'package-archives))
> +                  :help "Set the list of Emacs package sources"))

I propose to hold off on "Customize package sources" until we have a
final decision on the "Non-GNU ELPA" archive.  If we get a positive
decision to go ahead with that, IMHO this menu item is not necessary and
possibly even undesirable.

Just my two cents.

Best regards,
Stefan Kangas

Re: GNU ELPA package discoverability

[Bastien]

Hi Stefan,

Stefan Kangas <stefankangas@gmail.com> writes:

> Bastien <bzg@gnu.org> writes:
>
>> +      '(menu-item "Customize package sources"
>> +                  (lambda () (interactive) (customize-variable 'package-archives))
>> +                  :help "Set the list of Emacs package sources"))
>
> I propose to hold off on "Customize package sources" until we have a
> final decision on the "Non-GNU ELPA" archive.  If we get a positive
> decision to go ahead with that, IMHO this menu item is not necessary and
> possibly even undesirable.

Sorry, I must be missing something here, but how can it be undesirable
to advertize the fact that users can customize Emacs in a way that let
them install packages from non-official sources?

As long as the official sources are the default, I don't see a problem.

-- 
 Bastien

Re: GNU ELPA package discoverability

[Stefan Kangas]

Bastien <bzg@gnu.org> writes:

> Sorry, I must be missing something here, but how can it be undesirable
> to advertize the fact that users can customize Emacs in a way that let
> them install packages from non-official sources?

I'm not against advertising it, which we do in several places already.
It is only a question of how prominently we advertise it.

I would expect that once we can get "Non-GNU ELPA" going and configured
by default, there is less need to advertise this so prominently.
Perhaps the most important and/or popular packages will already be
installable by default, which means that this customization will take
place more infrequently.

My proposal is therefore simply to wait with adding the menu entry until
we know more about the way forward for "Non-GNU ELPA", its relation to
MELPA and so on.  There is ample time before Emacs 28.1 is released to
consider it again.  But this is just my personal opinion, of course.

Another thing, which I failed to bring up in my previous email.  If I
understand correctly, you propose to put "List available packages" after
"Customize package sources".  This seems to suggest that the package
system is not usable until after you customize it.  Am I missing
something here?  If not, should we perhaps change that order?

Best regards,
Stefan Kangas

Re: GNU ELPA package discoverability

[Bastien]

Hi Stefan,

Stefan Kangas <stefankangas@gmail.com> writes:

> My proposal is therefore simply to wait with adding the menu entry until
> we know more about the way forward for "Non-GNU ELPA", its relation to
> MELPA and so on.  There is ample time before Emacs 28.1 is released to
> consider it again.  But this is just my personal opinion, of course.

I'd rather detangle both issues: one is to promote the fact that users
can customize `package-archives' to what they need, another one is what
sources Emacs should promote.  I'll let Eli decide, of course.

> Another thing, which I failed to bring up in my previous email.  If I
> understand correctly, you propose to put "List available packages" after
> "Customize package sources".  This seems to suggest that the package
> system is not usable until after you customize it.  Am I missing
> something here?  If not, should we perhaps change that order?

No, you're right, "List available packages" should probably come first.

-- 
 Bastien

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  rms@gnu.org,  Emacs-devel@gnu.org
> Date: Mon, 01 Jun 2020 11:26:23 +0200
> 
> Eli, let me know if I can apply it against master.

See my other message: I'd rather we found a better name for these
items.  "Manage packages" sounds boring and uninteresting (who starts
using Emacs to "manage" something?).

Thanks.

Re: GNU ELPA package discoverability

[Jean-Christophe Helary]

> On Jun 2, 2020, at 0:08, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Bastien <bzg@gnu.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  rms@gnu.org,  Emacs-devel@gnu.org
>> Date: Mon, 01 Jun 2020 11:26:23 +0200
>> 
>> Eli, let me know if I can apply it against master.
> 
> See my other message: I'd rather we found a better name for these
> items.  "Manage packages" sounds boring and uninteresting (who starts
> using Emacs to "manage" something?).

New extensions


-- 
Jean-Christophe Helary @brandelune
http://mac4translators.blogspot.com

Re: GNU ELPA package discoverability

[Corwin Brust]

[-- Attachment #1: Type: text/plain, Size: 711 bytes --]

On Mon, Jun 1, 2020, 10:19 Jean-Christophe Helary <
jean.christophe.helary@traduction-libre.org> wrote:

>
>
> > On Jun 2, 2020, at 0:08, Eli Zaretskii <eliz@gnu.org> wrote:
> >
> >> From: Bastien <bzg@gnu.org>
> >> Cc: Eli Zaretskii <eliz@gnu.org>,  rms@gnu.org,  Emacs-devel@gnu.org
> >> Date: Mon, 01 Jun 2020 11:26:23 +0200
> >>
> >> Eli, let me know if I can apply it against master.
> >
> > See my other message: I'd rather we found a better name for these
> > items.  "Manage packages" sounds boring and uninteresting (who starts
> > using Emacs to "manage" something?).
>
> New extensions
>

Add/Remove Features

>
>
> --
> Jean-Christophe Helary @brandelune
> http://mac4translators.blogspot.com
>
>
>

[-- Attachment #2: Type: text/html, Size: 1825 bytes --]

Re: GNU ELPA package discoverability

[Bastien]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Bastien <bzg@gnu.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  rms@gnu.org,  Emacs-devel@gnu.org
>> Date: Mon, 01 Jun 2020 11:26:23 +0200
>> 
>> Eli, let me know if I can apply it against master.
>
> See my other message: I'd rather we found a better name for these
> items.  "Manage packages" sounds boring and uninteresting (who starts
> using Emacs to "manage" something?).

Fair enough.  What about this:

Tools
  Install Emacs packages
    List available packages
    Add new package* sources

* Not sure whether this should be package or packages.

-- 
 Bastien

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Cc: Emacs-devel@gnu.org,  sacha@sachachua.com,  rms@gnu.org
> Date: Tue, 02 Jun 2020 01:25:10 +0200
> 
> Tools
>   Install Emacs packages

I think "Install Add-On Packages" is better.

>     Add new package* sources
> 
> * Not sure whether this should be package or packages.

What would this item do?  Variable customization is not a good
candidate for a menu item.

Re: GNU ELPA package discoverability

[Bastien]

Hi Eli,

Eli Zaretskii <eliz@gnu.org> writes:

>>   Install Emacs packages
>
> I think "Install Add-On Packages" is better.

Add-On and Packages sound a bit redundant to me, but it's okay.

>>     Add new package* sources
>> 
>> * Not sure whether this should be package or packages.
>
> What would this item do?  Variable customization is not a good
> candidate for a menu item.

My motivation while pushing for a change here is to make it easier
for users to customize package-archives, something I've seen asked
very often.

If an item pointing to the variable customization is too "direct",
then I suggest to move the item "Find extra packages" from the Help
menu to this new "Install Add-On Packages" menu.

(If we go that way, then we should probably also move the "Search
built-in packages" item.)

-- 
 Bastien

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Cc: sacha@sachachua.com,  rms@gnu.org,  Emacs-devel@gnu.org
> Date: Wed, 03 Jun 2020 09:44:50 +0200
> 
> >>   Install Emacs packages
> >
> > I think "Install Add-On Packages" is better.
> 
> Add-On and Packages sound a bit redundant to me, but it's okay.

I don't think this will be redundant to newbies.  It also isn't
necessarily redundant in general, since "packages" exist in the core
as well.  E.g., "C-h p" finds "packages".

> >>     Add new package* sources
> >> 
> >> * Not sure whether this should be package or packages.
> >
> > What would this item do?  Variable customization is not a good
> > candidate for a menu item.
> 
> My motivation while pushing for a change here is to make it easier
> for users to customize package-archives, something I've seen asked
> very often.

My question was what will this menu item lead to.  IOW, what command
will it invoke, and what UI will that command present to users?

Re: GNU ELPA package discoverability

[Bastien]

Eli Zaretskii <eliz@gnu.org> writes:

>> > I think "Install Add-On Packages" is better.
>> 
>> Add-On and Packages sound a bit redundant to me, but it's okay.
>
> I don't think this will be redundant to newbies.  It also isn't
> necessarily redundant in general, since "packages" exist in the core
> as well.  E.g., "C-h p" finds "packages".

Okay.

>> My motivation while pushing for a change here is to make it easier
>> for users to customize package-archives, something I've seen asked
>> very often.
>
> My question was what will this menu item lead to.  IOW, what command
> will it invoke, and what UI will that command present to users?

My initial idea was for this item to invoke a command like
(lambda () (interactive) (customize-variable 'package-archives))
but you said that variable customization was not okay in a menu.

So my other suggestion is to move the Help item "Finding extra
packages" into this menu.

This would be:

Tools
  Install Add-On Packages
    List available packages  (aka M-x list-packages RET)
    Find extra packages      (aka C-h C-e)

And I'd also move "Search built-in packages" there:

Tools
  Install Add-On Packages
    List available packages  (aka M-x list-packages RET)
    Find extra packages      (aka C-h C-e)
    Search built-in packages (aka C-h p)

WDYT?

-- 
 Bastien

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Cc: Emacs-devel@gnu.org,  sacha@sachachua.com,  rms@gnu.org
> Date: Sun, 07 Jun 2020 11:18:31 +0200
> 
> My initial idea was for this item to invoke a command like
> (lambda () (interactive) (customize-variable 'package-archives))
> but you said that variable customization was not okay in a menu.

I think it would be strange.  We already have "Customize Emacs" in the
menu bar, so having another item that leads to customization of a
single variable would be weird.

> Tools
>   Install Add-On Packages
>     List available packages  (aka M-x list-packages RET)
>     Find extra packages      (aka C-h C-e)
> 
> And I'd also move "Search built-in packages" there:
> 
> Tools
>   Install Add-On Packages
>     List available packages  (aka M-x list-packages RET)
>     Find extra packages      (aka C-h C-e)
>     Search built-in packages (aka C-h p)
> 
> WDYT?

If we want to move "C-h p" there, the parent item should be "Add-On
Packages", without the "Install" part.

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 25.05.2020 10:51, Bastien wrote:
> - List packages
> - Customize package archives (<= the option you want to promote)
> - Customize the package interface

I wonder if we do want to promote that second option (the mention of 
MELPA was recently deleted from the manual).

And the last one... when did you last customize that group, aside from 
the list of archives?

I do agree that the first one could be made move prominent. Perhaps the 
first step would be rearranging the items in the Options menu?

Right now its bottom looks like:

   Save Options
   Customize Emacs
   Manage Emacs Packages

I suggest to make it:

   Customize Emacs
   Save Options (it makes sense for this one to go after, doesn't it?)
   ------- (a separator, probably)
   Manage Emacs Packages

Re: GNU ELPA package discoverability

[Bastien]

Hi Dmitry,

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 25.05.2020 10:51, Bastien wrote:
>> - List packages
>> - Customize package archives (<= the option you want to promote)
>> - Customize the package interface
>
> I wonder if we do want to promote that second option (the mention of
> MELPA was recently deleted from the manual).

My take on this is that yes, I'd want to promote the second option,
and I don't think we need to sort out all the issues with ELPA (GNU
and possibly non-GNU) before doing so.

The reason is newbies and old-timers both want to customize the list
of package archives.

> And the last one... when did you last customize that group, aside from
> the list of archives?

Agreed, I don't think it is necessary.

> I do agree that the first one could be made move prominent. Perhaps
> the first step would be rearranging the items in the Options menu?

I'd rather go ahead by moving Package information to the Tools menu,
where it really belongs.

-- 
 Bastien

Re: GNU ELPA package discoverability

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Cc: Sacha Chua <sacha@sachachua.com>,  Eli Zaretskii <eliz@gnu.org>,
>   rms@gnu.org,  Emacs-devel@gnu.org
> Date: Mon, 01 Jun 2020 11:16:12 +0200
> 
> I'd rather go ahead by moving Package information to the Tools menu,
> where it really belongs.

Why "Tools"? because some other applications have Tools->Add-ons menu
item?  But if so, then (a) not all of them do that (some have it in
"Options"), and (b) the proposed menu items are nowhere near
"Add-ons", so if someone wanted to alert newbies to something they are
maybe used to, I very much doubt that "Manage Emacs Packages" will do.

IOW, if this intends to make additional packages more easily
detectable, let's find a better name for this menu item, because
what's being suggested here is more a turn-off than a feature
promoting Emacs.

Or maybe I'm missing something.

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 25.05.2020 05:57, Sacha Chua wrote:
> I imagine that a newbie getting into Emacs because of things like Magit
> would be excited to see the splash screen or menu item for "Manage Emacs
> packages," but might be confused by the short list of packages. They
> might think: "Is that all there is? Is my Emacs broken? Is my setup
> incompatible with those packages?"

All other suggestions aside, I think that'd still be better than the 
current situation. A user who has questions is motivated to find 
answers. A user who doesn't know, doesn't know. :-)

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I wonder if there's a way we could make package-archives customization
  > more discoverable from the list-packages screen.

There are very few package archives that meet our moral criteria
for suggesting their use.  In fact, currently only one: GNU ELPA.

We are thinking of making another, which we can informally call NonGNU
ELPA.  Some of the packages now in other archives could be included
there.  The details remain to be worked out.

For moral reasons, we will not suggest an archive that contains
packages that steer or lead users towards installing or running
nonfree software.  That is a brief summary of the rule in node
References in https://www.gnu.org/prep/standards/.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Sacha Chua]

[-- Attachment #1: Type: text/plain, Size: 7746 bytes --]

Steve: Adding you because it's about MELPA and package archive
discoverability, which you've probably thought about waaaay more about than
I have. =) I think the newbie flow is that people usually go from MELPA ->
figure out package-archives rather than starting with package-archives and
finding MELPA. The code from MELPA's Getting Started guide might be one of
the first things a newbie wants to add to init.el. Would you consider
adding or linking to some kind of tutorial on editing init.el?

Other parts of the discussion follow:


> There are very few package archives that meet our moral criteria
> for suggesting their use.  In fact, currently only one: GNU ELPA.
> We are thinking of making another, which we can informally call NonGNU
> ELPA.  Some of the packages now in other archives could be included
> there.  The details remain to be worked out.
> For moral reasons, we will not suggest an archive that contains
> packages that steer or lead users towards installing or running
> nonfree software.  That is a brief summary of the rule in node
> References in https://www.gnu.org/prep/standards/.
>

That's cool, GNU Emacs is a GNU flagship project after all, and lots of
people learn about and become committed to free software philosophy because
of it. That's good. So if MELPA doesn't make the cut, maybe the NonGNU ELPA
could have enough popular packages so that people can install them easily
and get the hang of things. Let's see what the top MELPA packages are based
on downloads... Would things like Magit, Helm, and Flycheck be eligible for
inclusion in the new package archive?

Let's think through a bunch of scenarios where package management hiccups
might come up. How do people come to Emacs? From reading the newbie threads
in reddit.com/r/emacs, I get the sense that lots of people come in because
they've seen a cool video or blog post about programming, Spacemacs, or Org
Mode, or because they're Vim/VSCode users who are curious about the other
side.

Org Mode newbies tend to spend some time familiarizing themselves with
basic Org before digging into additional modules. The thing that usually
trips people up is probably upgrading Org from an Emacs that hasn't opened
an .org file before, but people seem to figure that out with some help.
There used to be some confusion due to major differences between versions,
but that seemed to have settled down after people got used to Emacs 26.1.

If newbies are coming in because of a Spacemacs programming video or
because they're coming from Vim, they'll probably go straight to Spacemacs,
which will solve the package management issue for them for a while. In
fact, Spacemacs wants people to set the dotspacemacs-additional-packages
variable and removes packages installed with package-install, so package
management improvements in vanilla Emacs are irrelevant. Some people
eventually leave Spacemacs for vanilla Emacs or something like Doom Emacs
because they want something lighter-weight, but they'll probably be a bit
more comfortable with Emacs by then.

So let's say they're starting with vanilla Emacs instead of Spacemacs. Most
people's questions tend to show that they already know about MELPA and are
running into problems setting it up, although there are a couple of
questions from people who want to install a package that isn't on ELPA
and they don't know about MELPA yet. I wonder what kind of hiccups people
usually run into... Here are snippets from an IRC conversation in Feb 2020:
"I want to install julia mode (for programming in julia) on my ubuntu 16.04
box. Apparently I should use the MELPA repository but looking at
https://melpa.org/#/getting-started is very intimidating" . The person
turned out to have a problem with having both .emacs and .emacs.d/init.el
and also having Emacs 24, and eventually got it working after upgrading.
Another conversation from 2019-07-10 turned out to be a problem with
editing .init.el instead of init.el (note initial .). Some people got
tripped up by using add-to-list 'package-archives before package.el is
loaded. The MELPA instructions for getting started have (require 'package)
at the beginning, but the one for melpa-stable might confuse some people
because it doesn't include that, and sometimes newbies don't notice the
(require 'package) when copying parts of other people's configs. Most
people seem to be okay with copying the right code, they just get tripped
up by where to put it. Other questions about package archives tend to be
related to servers being down, HTTP vs HTTPS, and so on.

Then of course there's figuring out which packages to install and how they
work together, how to configure them, how to load them... People often ask
for package/config recommendations on reddit.com/r/emacs and community
members share lots of resources, so I think that's something we can handle
on the community side.

We probably lose a bunch of potential users in that gap between "Okay, I've
installed Emacs" to "I've installed the packages I'm curious about" -
something along the lines of "Why is it so hard to install ____? Are things
going to be as frustrating as this?" Starter kits like Spacemacs do an
interesting job of bridging that, although I think adoption is still a bit
low. (Maybe 5-10%, as a totally wild guess?) Anyway, once they get past
that gap and start having their first wins, they have a bit more incentive
to stick around, as Emacs becomes more useful, more personalized, and more
fun.

I don't think MELPA would change the getting started instructions to use
Customize, so maybe tweaking the customization flow for package-archives is
actually less relevant. If more packages (especially the more popular ones)
could be installed without mucking around with package-archives or
~/.config/emacs/init.el (I need to get used to the new location!), that's
probably a bigger win than nudging people to customize package-archives
would be.

Hmm... Actually, come to think of it, where *do* we help people figure out
how to tweak their init.el? Emacs tends to recommend Customize because it
offers more guidance (and blindly copying Emacs Lisp into your config can
lead to Big Problems), but people who write about Emacs or share their
configs tend to post (setq ...)-type stuff (or use-package, which is
becoming quite popular), so newbies want to edit their init.el and then run
into all these problems finding the right file and figuring out what to do.
There's (emacs) Init File, which is under Advanced Features >
Customization. Should we think about init.el as something beginners might
want to know about early? Maybe a mention or info link in the tutorial?
Maybe a splash screen or Options menu shortcut to edit the init file so
that at least they're editing the right place, and maybe even some code to
check if the init file is getting shadowed? Maybe a M-x find-user-init-file
or edit-user-init-file? It seems a little dangerous to nudge people towards
init.el early, but newbies are already jumping into it and getting lost, so
maybe we can make it easier for them and frame it as a natural part of
working with Emacs. Come to think of it, tutorial writers are going to have
to figure out how to talk about editing either ~/.config/emacs/init.el or
~/.emacs.d/init.el . If there's going to be a transition period for that
anyway, it might make sense to promote something like M-x
find-user-init-file as well.

Along those lines, maybe there could be a hello-world sort of package that
people can install as part of the tutorial (or a tutorial part 2) that just
helps them confirm that they can do that, although we might need a link to
more troubleshooting info if they run into problems with proxies or other
stuff. Something for the info manual, maybe?

Sacha

[-- Attachment #2: Type: text/html, Size: 8701 bytes --]

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Any practical suggestions for how to do that in the little screen
  > > estate we have there?

  > I attach a patch against lisp/startup.el.

ISTR that someone proposed a clean-up of the splash screen a few weeks
ago.  We need to add back C-h C-c and C-h C-w, but even after that it
will still be shorter than now.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Bastien]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > Any practical suggestions for how to do that in the little screen
>   > > estate we have there?
>
>   > I attach a patch against lisp/startup.el.
>
> ISTR that someone proposed a clean-up of the splash screen a few weeks
> ago.

Good to know!  I searched the archive of this mailing list but could
not find any such proposal -- if someone can find the reference, I'd
love to rework my patch on top of the current proposal.

> We need to add back C-h C-c and C-h C-w, but even after that it
> will still be shorter than now.

C-h C-c and C-h C-w are present in the splash screen in the master
branch.

-- 
 Bastien

Re: GNU ELPA package discoverability

[Alan Third]

On Mon, May 25, 2020 at 09:41:12AM +0200, Bastien wrote:
> Richard Stallman <rms@gnu.org> writes:
> > ISTR that someone proposed a clean-up of the splash screen a few weeks
> > ago.
> 
> Good to know!  I searched the archive of this mailing list but could
> not find any such proposal -- if someone can find the reference, I'd
> love to rework my patch on top of the current proposal.

There's a reasonably long thread starting here:

    https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg01733.html

-- 
Alan Third

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 25.05.2020 23:48, Alan Third wrote:
> On Mon, May 25, 2020 at 09:41:12AM +0200, Bastien wrote:
>> Richard Stallman<rms@gnu.org>  writes:
>>> ISTR that someone proposed a clean-up of the splash screen a few weeks
>>> ago.
>> Good to know!  I searched the archive of this mailing list but could
>> not find any such proposal -- if someone can find the reference, I'd
>> love to rework my patch on top of the current proposal.
> There's a reasonably long thread starting here:
> 
>      https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg01733.html

And another here: 
https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg01946.html

Re: GNU ELPA package discoverability

[Bastien]

Hi Alan and Dmitry,

Alan Third <alan@idiocy.org> writes:

> There's a reasonably long thread starting here:
>
>     https://lists.gnu.org/archive/html/emacs-devel/2020-04/msg01733.html

Dmitry Gutov <dgutov@yandex.ru> writes:

> And another here:
> https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg01946.html

thanks, I've checked those links and they helped me understand the
situtation.

-- 
 Bastien

Re: GNU ELPA package discoverability

[Stefan Kangas]

Bastien <bzg@gnu.org> writes:

>> ISTR that someone proposed a clean-up of the splash screen a few weeks
>> ago.
>
> Good to know!  I searched the archive of this mailing list but could
> not find any such proposal -- if someone can find the reference, I'd
> love to rework my patch on top of the current proposal.

I started that work, but there has not been much progress since.  I
believe Dmitry and Alan linked the relevant threads.

I fully intend to get back to it, but I got distracted by other things
and the last week or so I've been too busy IRL.  Expect more when I find
some free time, hopefully soon.

There is no conflict with the changes you propose here, AFAICT.  The
idea is good, and we should easily be able to work that into the new
splash screen proposal.  I see no reason not to push the change.

Best regards,
Stefan Kangas

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > There is no conflict with the changes you propose here, AFAICT.  The
  > idea is good, and we should easily be able to work that into the new
  > splash screen proposal.  I see no reason not to push the change.

If someone does this, please add back the Copying Conditions and No
Warranty items.

I've been referring to them by their keyboard commands, which are C-h
C-c and C-h C-w, but there is no need to mention those keyboard
commands in the splash screen.  What the splash screen should show are
their names, which are Copying Conditions and No Warranty.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 27.05.2020 06:06, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>    > There is no conflict with the changes you propose here, AFAICT.  The
>    > idea is good, and we should easily be able to work that into the new
>    > splash screen proposal.  I see no reason not to push the change.
> 
> If someone does this, please add back the Copying Conditions and No
> Warranty items.

You're probably referring to the second design, by Nicolas P. Rougier,

> I've been referring to them by their keyboard commands, which are C-h
> C-c and C-h C-w, but there is no need to mention those keyboard
> commands in the splash screen.  What the splash screen should show are
> their names, which are Copying Conditions and No Warranty.

Could you explain why we absolutely need both of them there? Especially 
the second one (since COPYING includes the mention of No Warranty).

The space at the splash screen is at a premium; whatever additional 
things we show there will mean we won't be able to put another item on 
that page. Or mention a key binding, etc.

And going by some recent discussion, Nicolas's design might already be 
over the height limit.

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Could you explain why we absolutely need both of them there? Especially 
  > the second one (since COPYING includes the mention of No Warranty).

_We_ need to make sure users see this.  Having it available in case they
decide to look is not enough.  That is why it is there now.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: GNU ELPA package discoverability

[Dmitry Gutov]

On 29.05.2020 06:00, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>    > Could you explain why we absolutely need both of them there? Especially
>    > the second one (since COPYING includes the mention of No Warranty).
> 
> _We_ need to make sure users see this.  Having it available in case they
> decide to look is not enough.  That is why it is there now.

Could you explain why?

I understand we'd want to show the COPYING link to promote our license, 
but the warranty thing?

That sounds like a recommendation from legal, but a lot of other free 
software don't feature it so prominently.

Re: GNU ELPA package discoverability

[Bastien]

Hi Stefan,

I've read the threads -- great to see that Nicolas and you are both
working on this.  I have a small preference for Nicolas' approach, as
the dotted lines feel a bit more readable than the three columns.

I'll let you integrate my suggestion on adding the line about listing
ELPA packages when/where you see fit.

About the vertical space in the splash screen: here (emacs-28.0.50)
the number of text lines in the splash screen is 15 by default and 18
with the message about auto-saved files.

Perhaps we can agree to keep this number of lines below 20 and adapt
the height of the logo when (20 lines + current logo's height) does
not fit into Emacs startup height?

-- 
 Bastien

Re: GNU ELPA package discoverability

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I think advertizing GNU ELPA on the Emacs splash screen would be a
  > good first step.

+1

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

On Sun, May 17, 2020 at 2:38 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> I would like to point out, as an author of several packages, that in my
> experience having a package in ELPA is _better_ than having it in the core.

I know you like to develop on GitHub, and I'm not going to argue that:
it is a completely separate discussion.

I disagree on specific technical reasons that IMO don't get enough attention:

1. If Company were in the core, you could have major modes in
the core or elsewhere setting up the desired completion strategy or
strategies.  I.e. instead of  having central database of
`company-backends` that packages like Eglot have to override to behave
sanely, you could have a buffer-local  `company-backend` instead. You c
ould continue to develop the specific eclim, clang, xcode, etc backends
on GitHub, but you wouldn't "force" them on people that don't use them.

2. If Yasnippet were in the core, again, no need for the centralized
database of snippets: major modes define their snippets.  Other tools
can use the snippet-defining infrastructure without eating all that
João-the-ten-years-ago newbie cruft. But even with yasnippet.el, it
is possible (though not ideal: Stefan once convinced me to rewrite it
which I did to 90% in a so-called snippet.el -- but there's still the
10% missing).

3. If Eglot were in the core, again, no need for the centralized stuff
currently living in eglot-server-programs.  Also, major modes in the
core can affect Eglot's LSP interfaces via stronger coupling techniques,
such as adding methods to generic functions.  Besides major modes,
other tools than build on some LSP basics, such as an LSP-enabled
spell checker could be built.

In general, I'm intent on describing a pattern here: there's a very
good reason for functionality that can be seen as infrastructure
to be in the core.  At least as long as the users of such infrastructure
(which are often, but not always, major mdoes) also live in the core.

So the solution is to move the infrastructure bits to the core, or
move every major mode to GNU ELPA.

Also, I want to note, again, that even if Company, snippet.el and
Eglot  were in the core, they could still _also_ be available
for download via GNU ELPA.

> The exception to that rule is more or less cases where the package is
> not only added but also enabled by default.

I've argued that that's not the only exception.

João

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: João Távora <joaotavora@gmail.com>
> Date: Tue, 19 May 2020 15:11:23 +0100
> Cc: Eli Zaretskii <eliz@gnu.org>, Richard Stallman <rms@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	joostkremers@fastmail.fm, "Alfred M. Szmidt" <ams@gnu.org>, 
> 	Phillip Lord <phillip.lord@russet.org.uk>, 조성빈 <pcr910303@icloud.com>, 
> 	emacs-devel <Emacs-devel@gnu.org>
> 
> In general, I'm intent on describing a pattern here: there's a very
> good reason for functionality that can be seen as infrastructure
> to be in the core.

Definite 100% agreement.  And I think this is a general tendency we
should agree on regardless of the specific issues raised in this
particular discussion.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 19.05.2020 17:35, Eli Zaretskii wrote:
>> In general, I'm intent on describing a pattern here: there's a very
>> good reason for functionality that can be seen as infrastructure
>> to be in the core.
> Definite 100% agreement.  And I think this is a general tendency we
> should agree on regardless of the specific issues raised in this
> particular discussion.

We clearly have different ideas on what infrastructure is.

xref and project are infrastructure. completion-at-point-functions and 
flymake, and eldoc are infrastructure.

Infrastructure is extensible.

Eglot is some infrastructure and some implementation. And I'm still not 
sure what part of its "infrastructure" is generic enough to be used by 
other packages. Aside from jsonrpc, of course.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 19.05.2020 17:11, João Távora wrote:
> On Sun, May 17, 2020 at 2:38 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
>> I would like to point out, as an author of several packages, that in my
>> experience having a package in ELPA is _better_ than having it in the core.
> 
> I know you like to develop on GitHub, and I'm not going to argue that:
> it is a completely separate discussion.

Indeed. That's why I was arguing about different things, rather than how 
nice it is to be able to use a different hosting and bug tracker.

> I disagree on specific technical reasons that IMO don't get enough attention:

Below, you are arguing about two specific packages. I didn't say that 
_no_ packages should be in the core, however. Just that for most of them 
there is not much additional benefit from that.

But we can discuss these examples, too.

> 1. If Company were in the core, you could have major modes in
> the core or elsewhere setting up the desired completion strategy or
> strategies.  I.e. instead of  having central database of
> `company-backends` that packages like Eglot have to override to behave
> sanely, you could have a buffer-local  `company-backend` instead. You c
> ould continue to develop the specific eclim, clang, xcode, etc backends
> on GitHub, but you wouldn't "force" them on people that don't use them.

Major modes already define c-a-p-f, we have a frontend-agnostic API for 
that. On top of it, however, Company provides finer-grained controls for 
the users, which can't be decided by major modes.

I don't think 'M-x completion-at-point' can be realistically expected to 
go away in any near future, so it doesn't seem like some tighter 
integration with major modes is on the table (and I don't know what it 
would look like anyway).

As for Eglot, it seems to successfully do what you want here with no 
extra boilerplate. The only problem I see there is breaking some users' 
existing configurations.

> 2. If Yasnippet were in the core, again, no need for the centralized
> database of snippets: major modes define their snippets.  Other tools
> can use the snippet-defining infrastructure without eating all that
> João-the-ten-years-ago newbie cruft. But even with yasnippet.el, it
> is possible (though not ideal: Stefan once convinced me to rewrite it
> which I did to 90% in a so-called snippet.el -- but there's still the
> 10% missing).

Yasnippet has been wildly successful without all that. It's the #1 
standard snippet solution and format for Emacs. It's in GNU ELPA, 
everybody can use and depend on it.

About "newbie cruft", how will that go away without somebody rewriting 
yasnippet's code? And when they do, the result can sit in GNU ELPA as well.

Finally, the only snippet collections I still see maintained are being 
done by third-party developers. If the major mode authors (BTW, how many 
are actively maintaining major modes in the core?) wanted to also add 
snippet collections, they'd have already done so.

And even if they do the new snippets will only reach the users once 
every 2 years. (facepalm emoji)

> 3. If Eglot were in the core, again, no need for the centralized stuff
> currently living in eglot-server-programs.

Right. So CC Mode will define which completion server will be used by 
Eglot? Really?

And suppose major mode authors even decide to take up that 
responsibility, the LSP field moves much faster than Emacs these days. 
Six month after Emacs 27 is released, another LSP server for C++ falls 
out of fashion and stops being developed, and Emacs users will stay with 
outdated settings until the next release. Or until their distro switches 
to Emacs 28, actually, which can be another several years.

> Also, major modes in the
> core can affect Eglot's LSP interfaces via stronger coupling techniques,
> such as adding methods to generic functions.

You still never gave an example for that.

> Besides major modes,
> other tools than build on some LSP basics, such as an LSP-enabled
> spell checker could be built.

Yes it can. Anything stops it from being in ELPA?

> Also, I want to note, again, that even if Company, snippet.el and
> Eglot  were in the core, they could still _also_ be available
> for download via GNU ELPA.

I also noted that later.

>> The exception to that rule is more or less cases where the package is
>> not only added but also enabled by default.
> 
> I've argued that that's not the only exception.

I'd argue if we call something "infrastructure", then it should either 
be enabled by default, or used by other packages, or otherwise have a 
necessary stronger coupling to other code. This is a fairly high barrier.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

On Tue, May 19, 2020 at 3:59 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> I don't think 'M-x completion-at-point' can be realistically expected to
> go away in any near future, so it doesn't seem like some tighter
> integration with major modes is on the table (and I don't know what it
> would look like anyway).

I'm not arguing for it to go away, not at all.  I am arguing for a completion
tooltip in the core, much like icomplete, one that gets to integrate with
other pieces in the core.

> As for Eglot, it seems to successfully do what you want here with no
> extra boilerplate. The only problem I see there is breaking some users'
> existing configurations.

In the Clangd webpage, a warning was issued to Eglot users:

"**company-clang is enabled by default**, and will interfere with clangd.
Disable it in `M-x customize-variable RET company-backends RET`."

This is clearly counter to Eglot's out-of-the-box experience, so I had
to override company-backends. Again, if you had left that brutal default
configuration out of the company package, I wouldn't have had to.
Really, even if just on github, why don't you create a `company-nocruft`
package that only has capf support?

Anyway, after my fix, I've finally got them to update the webpage:

https://github.com/llvm/clangd-www/pull/1/commits/a328d8e92eb05b8e62cd0973b592d6b48278c23d

> Yasnippet has been wildly successful without all that. It's the #1
> standard snippet solution and format for Emacs. It's in GNU ELPA,
> everybody can use and depend on it.

What are you saying: major modes in the core _can't_.

And just because it was successful in GNU ELPA doesn't me it can't
be _more_ successful if developed in the core.  Below you say major
modes in the core aren't maintained: well, maybe _because_ they
can't tap into resources like Yasnippet.

> About "newbie cruft", how will that go away without somebody rewriting
> yasnippet's code? And when they do, the result can sit in GNU ELPA as well.

One part can be developed in GNU ELPA, the other in the core.  BOTH
parts can be downloadable from GNU ELPA, nothing against that.

> Finally, the only snippet collections I still see maintained are being
> done by third-party developers. If the major mode authors (BTW, how many
> are actively maintaining major modes in the core?) wanted to also add
> snippet collections, they'd have already done so.

The lack of maintainers is not an argument for changing the place
where these things are maintained.  That's not magically going to
bring more maintainers.

> And even if they do the new snippets will only reach the users once
> every 2 years. (facepalm emoji)

No. You keep saying this, but if the major-mode is GNU ELPA :core
it's no problem.

> Right. So CC Mode will define which completion server will be used by
> Eglot? Really?

Yes, supposing Alan wants to provide LSP features, yes. Eli has said
we wants features in CC mode that LSP is capable of supporting.

If Alan doesn't want to, then we'll make another C mode, maybe
resorting to LSP only or to TreeSitter or a mix of both. Or Alan
can come around.  Or make it opt-in.  Lots of options here.

> And suppose major mode authors even decide to take up that
> responsibility, the LSP field moves much faster than Emacs these days.
> Six month after Emacs 27 is released, another LSP server for C++ falls
> out of fashion and stops being developed, and Emacs users will stay with
> outdated settings until the next release. Or until their distro switches
> to Emacs 28, actually, which can be another several years.

Again, this fictional tragedy ignore the advent of :core GNU ELPA
packages.

> > Also, major modes in the
> > core can affect Eglot's LSP interfaces via stronger coupling techniques,
> > such as adding methods to generic functions.
> You still never gave an example for that.

Unless I missed something, you never asked.  Stefan asked off-list, and
I gave him examples.

Anyway, see eglot.el.  It has some, though I've been quite conservative,
again, precisely _because_ that code doesn't belong in eglot.el.
So you're describing the chicken and the egg.  Or look at the very large
amount of mode-specific code that lsp-mode.el.  It also doesn't belong
there.

And do I really have to download clangd and xcode-specific code
to my machine.

> > Besides major modes,
> > other tools than build on some LSP basics, such as an LSP-enabled
> > spell checker could be built.
> Yes it can. Anything stops it from being in ELPA?

But it can be developed in the core and be downloaded from
GNU ELPA, as I keep saying.  The two things are completely
unrelated, thankfully.

> I'd argue if we call something "infrastructure", then it should either
> be enabled by default, or used by other packages, or otherwise have a
> necessary stronger coupling to other code. This is a fairly high barrier.

This is the classic monolithic vs modular debate.  Emacs is mostly
monolithic, so unless it goes full modular, you're seriously
handicapping development of some stuff by demanding it sit
outside the core.  Both approaches are defensible, of course.
But a mixed approach will tend to limp.

Again, distribution and where the source code is kept is now mostly
orthogonal, thanks to :core packages.  The distribution is the thing
that's preventing the "discoverability" the supposed subject of
this debate.  Development is a different reality: we shouldn't
have distribution concerns hamper it.

João

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 19.05.2020 20:28, João Távora wrote:

>> I don't think 'M-x completion-at-point' can be realistically expected to
>> go away in any near future, so it doesn't seem like some tighter
>> integration with major modes is on the table (and I don't know what it
>> would look like anyway).
> 
> I'm not arguing for it to go away, not at all.  I am arguing for a completion
> tooltip in the core, much like icomplete, one that gets to integrate with
> other pieces in the core.

Maybe you could provide better arguments. Because I understand the idea, 
but the details of the argument don't convince me, so far.

If you just argued that it would be good to have a completion popup in 
the core, and _enabled by default_, I would have nothing to say against 
that.

>> As for Eglot, it seems to successfully do what you want here with no
>> extra boilerplate. The only problem I see there is breaking some users'
>> existing configurations.
> 
> In the Clangd webpage, a warning was issued to Eglot users:
> 
> "**company-clang is enabled by default**, and will interfere with clangd.
> Disable it in `M-x customize-variable RET company-backends RET`."

This is a different point than you originally made.

> This is clearly counter to Eglot's out-of-the-box experience, so I had
> to override company-backends. Again, if you had left that brutal default
> configuration out of the company package, I wouldn't have had to.
> Really, even if just on github, why don't you create a `company-nocruft`
> package that only has capf support?

Because I have other things on my list, and because supporting different 
versions of the same package is not as fun as someone might expect.

Also, you ask for things that would make life easier for you, but when I 
ask you to play by the rules of the framework, you reserve your choice 
to just go the most convenient route.

> Anyway, after my fix, I've finally got them to update the webpage:
> 
> https://github.com/llvm/clangd-www/pull/1/commits/a328d8e92eb05b8e62cd0973b592d6b48278c23d

That's a good news.

Anyway, I'm considering removing company-clang by default (and some 
other backends as well) after some upcoming release of company.

To do that, and to avoid confusing some of the users, I'd have to create 
some instructions on what C/C++ developers should use and in what cases.

If you have free time for some documentation writing, help welcome.

>> Yasnippet has been wildly successful without all that. It's the #1
>> standard snippet solution and format for Emacs. It's in GNU ELPA,
>> everybody can use and depend on it.
> 
> What are you saying: major modes in the core _can't_.

What would they use in it?

> And just because it was successful in GNU ELPA doesn't me it can't
> be _more_ successful if developed in the core.  Below you say major
> modes in the core aren't maintained: well, maybe _because_ they
> can't tap into resources like Yasnippet.

Um, that's unlikely to play a big role.

>> About "newbie cruft", how will that go away without somebody rewriting
>> yasnippet's code? And when they do, the result can sit in GNU ELPA as well.
> 
> One part can be developed in GNU ELPA, the other in the core.  BOTH
> parts can be downloadable from GNU ELPA, nothing against that.

If there is something existing code in Emacs can tap into in the "new 
part", I don't have anything against that.

>> Finally, the only snippet collections I still see maintained are being
>> done by third-party developers. If the major mode authors (BTW, how many
>> are actively maintaining major modes in the core?) wanted to also add
>> snippet collections, they'd have already done so.
> 
> The lack of maintainers is not an argument for changing the place
> where these things are maintained.  That's not magically going to
> bring more maintainers.

As far as Yasnippet is concerned, I'm not arguing to change things. You are.

I'm arguing the current situation is pretty good, and not without reasons.

>> And even if they do the new snippets will only reach the users once
>> every 2 years. (facepalm emoji)
> 
> No. You keep saying this, but if the major-mode is GNU ELPA :core
> it's no problem.

Okay then, we'll go ahead and make everything into GNU ELPA packages.

And /my plan/ will be two steps closer to fruition.

*evil laugh*

>> Right. So CC Mode will define which completion server will be used by
>> Eglot? Really?
> 
> Yes, supposing Alan wants to provide LSP features, yes. Eli has said
> we wants features in CC mode that LSP is capable of supporting.

Could you please avoid referring to general statements by Eli until you 
actually discuss some particulars?

"Let's provide features we can provide" as as sensible as it is 
noncommittal.

> If Alan doesn't want to, then we'll make another C mode,

*insert another kind of laugh here*

> maybe
> resorting to LSP only or to TreeSitter or a mix of both. Or Alan
> can come around.  Or make it opt-in.  Lots of options here.

Yes, there are many ways to try to skin this particular feline. Some of 
them more optimal than others.

>> And suppose major mode authors even decide to take up that
>> responsibility, the LSP field moves much faster than Emacs these days.
>> Six month after Emacs 27 is released, another LSP server for C++ falls
>> out of fashion and stops being developed, and Emacs users will stay with
>> outdated settings until the next release. Or until their distro switches
>> to Emacs 28, actually, which can be another several years.
> 
> Again, this fictional tragedy ignore the advent of :core GNU ELPA
> packages.

If we add every [important or fast-moving] core package to GNU ELPA, 
that could solve some problems indeed.

Mainly leaving organizational issues, like increasing the diffs mailing 
list traffic, the bug tracker load, and so on, the more packages we take 
in. And increasing the core's risks and commitments in unfortunate cases 
where the original authors leave.

>>> Also, major modes in the
>>> core can affect Eglot's LSP interfaces via stronger coupling techniques,
>>> such as adding methods to generic functions.
>> You still never gave an example for that.
> 
> Unless I missed something, you never asked.  Stefan asked off-list, and
> I gave him examples.

I asked, last time you mentioned this.

> Anyway, see eglot.el.  It has some, though I've been quite conservative,
> again, precisely _because_ that code doesn't belong in eglot.el.

Could you mention some symbol to search for, at least?

> So you're describing the chicken and the egg.  Or look at the very large
> amount of mode-specific code that lsp-mode.el.  It also doesn't belong
> there.

That could be true. But if the Eglot team are best-positioned to write 
that code, and to maintain it, you'll be the ones to do it anyway. Then 
why split it across files, move it to major modes, etc?

Furthermore, if some necessary setup will be performed in CC Mode, then 
alternative C modes, if they ever appear, will harder time making use of 
it. I'm not sure if we'll have this particular problem, but alternative 
major modes have been created for other languages. Rust, for instance.

> And do I really have to download clangd and xcode-specific code
> to my machine.

This pertaining to..?

>>> Besides major modes,
>>> other tools than build on some LSP basics, such as an LSP-enabled
>>> spell checker could be built.
>> Yes it can. Anything stops it from being in ELPA?
> 
> But it can be developed in the core and be downloaded from
> GNU ELPA, as I keep saying.  The two things are completely
> unrelated, thankfully.

I'm just saying it's not an argument in favor of either of the options.

>> I'd argue if we call something "infrastructure", then it should either
>> be enabled by default, or used by other packages, or otherwise have a
>> necessary stronger coupling to other code. This is a fairly high barrier.
> 
> This is the classic monolithic vs modular debate.  Emacs is mostly
> monolithic, so unless it goes full modular, you're seriously
> handicapping development of some stuff by demanding it sit
> outside the core.

In theory, your statement makes sense. But I don't see any solid 
examples so far, as pertaining to packages under discussion. And only by 
considering examples in detail we can derive good rules.

> Both approaches are defensible, of course.
> But a mixed approach will tend to limp.

This, however, is something good-sounding, but just as likely wrong as 
it is right. We could use some principles, however.

Yes, I want Emacs to be modular, and I'm not the only one. But the 
degree of modularity, and the division lines, all should be driven by 
practical considerations.

Do you like the current monolith? Are you sure the current architecture 
is good, and gives appropriate incentives to the authors?

Do you like message-mode (as essential feature) still being a part of 
Gnus, a big specialized application, after all these years, and 
depending on its other code?

Do you like that Org is just its own microcosm, and basically never 
extracts features?

Do you like all the references to tramp- functions outside of its 
implementation?

I don't want to criticize anyone in particular (we're all volunteers 
here), but some clearer distinction between infrastructure and 
applications could do good.

> Again, distribution and where the source code is kept is now mostly
> orthogonal, thanks to :core packages.  The distribution is the thing
> that's preventing the "discoverability" the supposed subject of
> this debate.  Development is a different reality: we shouldn't
> have distribution concerns hamper it.

Eli asked why we never followed up on our "promises" to consider some 
packages for the core which we put into GNU ELPA. I explained that those 
packages likely didn't need that anyway. This is where this line of 
discussion came from.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

On Tue, May 19, 2020 at 8:38 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> > "**company-clang is enabled by default**, and will interfere with clangd.
> > Disable it in `M-x customize-variable RET company-backends RET`."
> This is a different point than you originally made.

Maybe you misunderstood me. But that warning is exactly, ipsis verbis,
the reason I did what I had to do (override company-backends).

> > Really, even if just on github, why don't you create a `company-nocruft`
> > package that only has capf support?
> Because I have other things on my list, and because supporting different
> versions of the same package is not as fun as someone might expect.

But you're the one arguing for all-out modular and mini-package stuff for
the others!  Your Company has become a mini-Emacs :-)

> To do that, and to avoid confusing some of the users, I'd have to create
> some instructions on what C/C++ developers should use and in what cases.

Or just create those things as packages that depend on `company-nocruft`
(Yes I know "nocruft" is a cheeky name, just to make a point.  Call it
what you with, `company-base`.  Eglot would depend on that)

> If you have free time for some documentation writing, help welcome.

No thank you, but I _will_ help you split it into packages.

> >> Yasnippet has been wildly successful without all that. It's the #1
> >> standard snippet solution and format for Emacs. It's in GNU ELPA,
> >> everybody can use and depend on it.
> > What are you saying: major modes in the core _can't_.
> What would they use in it?

Hm? They would define snippets for the programming they support.
Snippets that are active by default once you activate the major mode.

> > And just because it was successful in GNU ELPA doesn't me it can't
> > be _more_ successful if developed in the core.  Below you say major
> > modes in the core aren't maintained: well, maybe _because_ they
> > can't tap into resources like Yasnippet.
> Um, that's unlikely to play a big role.

Well, we wouldn't know, would we?  A bit of history: I got seriously
interested in Yasnippet because of TextMate. At the time, I did some
Ruby and watched lots of screecasts, all had Textmate. Textmate had
all the things: syntax, snippets, commands, menus out of the box.  I
even tried to do that stuff with Yasnippet only, there's still some heroic
(but working!) code that will take a Textmate "bundle" and put everything
it has in the Yasnippet menu.

Many years after that, I'm still trying to achieve the same, but by
integrating with existing code, instead of trying to cram
Textmate stuff into Emacs. I know  you're going to say: "then
put the major mode in ELPA and you can have your bundle", and
I'll say  "OK, but let's develop it in the core".

> > The lack of maintainers is not an argument for changing the place
> > where these things are maintained.  That's not magically going to
> > bring more maintainers.
>
> As far as Yasnippet is concerned, I'm not arguing to change things. You are.

I thought you were talking about major-modes, which Yasnippet is not.

Yasnippet would presumably maintain its one-man maintainership, Noam,
for which  I am very grateful.  It's Noam's call if Yasnippet stays in GitHub,
mind you.  But it's my call to propose snippet.el to the core, and I can't
imagine Noam saying no to offloading a big part of the snippet expansion
and navigation engine to another much cleaner package.

> Okay then, we'll go ahead and make everything into GNU ELPA packages.
>
> And /my plan/ will be two steps closer to fruition.
>
> *evil laugh*

An evil laugh straight from dependency hell, evil indeed :-)

Now seriously: not everything, but the things that we think will be updated
often enough to benefit users between two major releases, sure, why not?

> Could you please avoid referring to general statements by Eli until you
> actually discuss some particulars?
> "Let's provide features we can provide" as as sensible as it is
> noncommittal.

That's not what I wrote.

Eli said that he wants CC mode to support syntax-highlighting
and indentation from other sources, such as TreeSitter or LSP.
LSP will soon support these things, along with many other things.

> > If Alan doesn't want to, then we'll make another C mode,
> *insert another kind of laugh here*

Don't be silly.  I'm not proposing we evict Alan.  But Emacs is about
alternatives.  There's already another C mode in ELPA, I think.  No
C++ mode though, that's much harder to parse (and _that's_ why we
should offload this work).

> > Again, this fictional tragedy ignore the advent of :core GNU ELPA
> > packages.
> If we add every [important or fast-moving] core package to GNU ELPA,
> that could solve some problems indeed.

Yes.  Important or fast-moving sounds good.

> Mainly leaving organizational issues, like increasing the diffs mailing
> list traffic, the bug tracker load, and so on, the more packages we take
> in. And increasing the core's risks and commitments in unfortunate cases
> where the original authors leave.

I told you I was not going to discuss these things ;-)  And you agreed,
so don't mix up the discussions.

> > Anyway, see eglot.el.  It has some, though I've been quite conservative,
> > again, precisely _because_ that code doesn't belong in eglot.el.
> Could you mention some symbol to search for, at least?

OK. Rust and java stuff down the end of the file.  But the bug tracker
is rife  with hacks that people use to get it working in other languages:
I've been telling people to put them in their .emacs. But they belong in
the major mode really.

See also https://github.com/joaotavora/eglot/issues/363, for example.

See also the `;;; API (WORK-IN-PROGRESS!)` section in eglot.el. Every
generic function that takes a server as an argument is a candidate
for major mode tweaking.

> > So you're describing the chicken and the egg.  Or look at the very large
> > amount of mode-specific code that lsp-mode.el.  It also doesn't belong
> > there.
> That could be true. But if the Eglot team are best-positioned to write
> that code, and to maintain it, you'll be the ones to do it anyway. Then
> why split it across files, move it to major modes, etc?

Why not? We do have your beautiful xref.el right?  Again Lisp is not
Java hehe.

Seriously, because that's where it belongs, at least the way that
major-modes have been though of throughout the aeons.  If you want
to change that fundamental property, you are fighting Emacs, in my
opinion.  I want to find a Python file and start working in the best
environment.

By the way this reminds me of a Medium thing I read this week
where one users was fed up with VsCode asking to install packages
for typing an Hello World in C++.

> > And do I really have to download clangd and xcode-specific code
> > to my machine.
> This pertaining to..?

Company.  Did I put it out of context? Sorry.

> > GNU ELPA, as I keep saying.  The two things are completely
> > unrelated, thankfully.
> I'm just saying it's not an argument in favor of either of the options.

It is if you want to rewrite or enhance flyspell, for example.
Exactly  how I did flymake, a package that was injustly ridiculed
to the point that someone made a "Flymake done right", tsk tsk

> In theory, your statement makes sense. But I don't see any solid
> examples so far, as pertaining to packages under discussion. And only by
> considering examples in detail we can derive good rules.
>
> > Both approaches are defensible, of course.
> > But a mixed approach will tend to limp.
>
> This, however, is something good-sounding, but just as likely wrong as
> it is right. We could use some principles, however.
>
> Yes, I want Emacs to be modular, and I'm not the only one. But the
> degree of modularity, and the division lines, all should be driven by
> practical considerations.
>
> Do you like the current monolith?

And this is precisely where the problem is.  My answer to this
question is irrelevant, because I'm a very small part in the
development of Emacs, due to multiple reasons. Skills, time available,
etc.  So I try to work in a way that doesn't actively preempt future
migrations  to other architectures, but I don't refrain from working
inside the current architecture because I'm not conceptually
satisfied with what it could be if I could magically convince everyone
to think like me and spend an almost unlimited about of collective brain
power.

These is my rule-of-thumb: scratch my itch. If possible scratch
other's, and do it in a way that doesn't induce new itches to
others still.

> Do you like that Org is just its own microcosm, and basically never
> extracts features?

I think you know my answer to that.  But again, irrelevant.  Stuff
doesn't happen by the truckload.  Look in your backyard :-) extract
that company-clang to a separate package :-)

> Eli asked why we never followed up on our "promises" to consider some
> packages for the core which we put into GNU ELPA. I explained that those
> packages likely didn't need that anyway. This is where this line of
> discussion came from.

Sorry, I might have missed some posts in this particular line

João

PS: this discussion is getting looong.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 19.05.2020 23:56, João Távora wrote:

>>> "**company-clang is enabled by default**, and will interfere with clangd.
>>> Disable it in `M-x customize-variable RET company-backends RET`."
>> This is a different point than you originally made.
> 
> Maybe you misunderstood me. But that warning is exactly, ipsis verbis,
> the reason I did what I had to do (override company-backends).

I'm saying that simply moving company to the core won't solve that 
problem (it will still need to support existing features).

I'm not sure how splitting out a "core" package would help if the "full" 
version depends on it, actually. Surely you are going to support the 
users of the "full" version as well? And they will have a longer list in 
`company-backends' by default.

>>> Really, even if just on github, why don't you create a `company-nocruft`
>>> package that only has capf support?
>> Because I have other things on my list, and because supporting different
>> versions of the same package is not as fun as someone might expect.
> 
> But you're the one arguing for all-out modular and mini-package stuff for
> the others!  Your Company has become a mini-Emacs :-)

Nice reductio at absurdum. I never said every single file should be a 
separate package.

A decent guideline, in the commercial world at least, is to split 
project into services along organizational lines (e.g. one team = one 
service, very roughly speaking). There's nobody else who's going to 
maintain company-clang but me. In other cases, I do recommend people 
create and maintain their own packages. You might want to count the 
number of dependents on MELPA.

>> If you have free time for some documentation writing, help welcome.
> 
> No thank you, but I _will_ help you split it into packages.

If it behaves sufficiently differently from company (as I recall you 
were planning), you might want to pick a different name.

For example, "solitude".

>>>> Yasnippet has been wildly successful without all that. It's the #1
>>>> standard snippet solution and format for Emacs. It's in GNU ELPA,
>>>> everybody can use and depend on it.
>>> What are you saying: major modes in the core _can't_.
>> What would they use in it?
> 
> Hm? They would define snippets for the programming they support.
> Snippets that are active by default once you activate the major mode.

Or maybe they simply won't.

You might want to create a separate thread here with a vote and count 
the major mode maintainers who are in. :-)

>>> And just because it was successful in GNU ELPA doesn't me it can't
>>> be _more_ successful if developed in the core.  Below you say major
>>> modes in the core aren't maintained: well, maybe _because_ they
>>> can't tap into resources like Yasnippet.
>> Um, that's unlikely to play a big role.
> 
> Well, we wouldn't know, would we?  A bit of history: I got seriously
> interested in Yasnippet because of TextMate. At the time, I did some
> Ruby and watched lots of screecasts, all had Textmate. Textmate had
> all the things: syntax, snippets, commands, menus out of the box.  I
> even tried to do that stuff with Yasnippet only, there's still some heroic
> (but working!) code that will take a Textmate "bundle" and put everything
> it has in the Yasnippet menu.

Good stuff.

> Many years after that, I'm still trying to achieve the same, but by
> integrating with existing code, instead of trying to cram
> Textmate stuff into Emacs. I know  you're going to say: "then
> put the major mode in ELPA and you can have your bundle", and
> I'll say  "OK, but let's develop it in the core".

I don't really mind the idea of putting snippet.el into the core. It 
sounds like a small, self-contained library, with limited scope. Perhaps 
some other core developers take interest, too.

But I don't see moving snippet definitions to major modes as inherently 
"right" or beneficial, because putting a lot of functionality in one 
place is not particularly flexible thing to do, it's like a huge 
function that does everything: when you need another function (e.g. 
major mode) to reuse some part of it, the design breaks.

I see your desire to offload some responsibility to major mode authors, 
and I can sympathize with that. But if the people on the other side 
don't pick it up (or pick it up but then forget), this won't work. 
Especially with eglot-server-programs.

>> Okay then, we'll go ahead and make everything into GNU ELPA packages.
>>
>> And /my plan/ will be two steps closer to fruition.
>>
>> *evil laugh*
> 
> An evil laugh straight from dependency hell, evil indeed :-)
> 
> Now seriously: not everything, but the things that we think will be updated
> often enough to benefit users between two major releases, sure, why not?

Indeed, why not. Were I supposed to argue against this? :-) The more 
developers think about ELPA, and take care to bump version numbers in 
their files, etc, the more acceptable the idea of "let's jump move to 
packages will become".

Not everybody might agree, though. You might notice that there's still 
no cc-mode package in there.

>> Could you please avoid referring to general statements by Eli until you
>> actually discuss some particulars?
>> "Let's provide features we can provide" as as sensible as it is
>> noncommittal.
> 
> That's not what I wrote.
> 
> Eli said that he wants CC mode to support syntax-highlighting
> and indentation from other sources, such as TreeSitter or LSP.
> LSP will soon support these things, along with many other things.

I'll have to see this first. And how reliable it is, and how much of it 
depends on VS Code's internals, etc. And actual language server support. 
But I'm repeating myself, from another thread.

>>> If Alan doesn't want to, then we'll make another C mode,
>> *insert another kind of laugh here*
> 
> Don't be silly.  I'm not proposing we evict Alan.  But Emacs is about
> alternatives.

It was more of a weak, exasperated laugh. We couldn't evict Alan even if 
we wanted: CC Mode is a beast, and C++ is difficult enough to support 
that there haven't been any real alternatives, I think, ever.

sm-c-mode in GNU ELPA is a cute experiment that supports only a fraction 
of its features. Even for C, IIRC.

>>> Anyway, see eglot.el.  It has some, though I've been quite conservative,
>>> again, precisely _because_ that code doesn't belong in eglot.el.
>> Could you mention some symbol to search for, at least?
> 
> OK. Rust and java stuff down the end of the file.  But the bug tracker
> is rife  with hacks that people use to get it working in other languages:
> I've been telling people to put them in their .emacs. But they belong in
> the major mode really.
> 
> See also https://github.com/joaotavora/eglot/issues/363, for example.

 From what I see it there, someone could/should make an RLS-specific 
plugin for Eglot. Not sure how major modes would help. There are several 
language servers for Rust, after all.

> See also the `;;; API (WORK-IN-PROGRESS!)` section in eglot.el. Every
> generic function that takes a server as an argument is a candidate
> for major mode tweaking.

Like generic functions. Suspicious on the idea of major mode specification.

>>> So you're describing the chicken and the egg.  Or look at the very large
>>> amount of mode-specific code that lsp-mode.el.  It also doesn't belong
>>> there.
>> That could be true. But if the Eglot team are best-positioned to write
>> that code, and to maintain it, you'll be the ones to do it anyway. Then
>> why split it across files, move it to major modes, etc?
> 
> Why not? We do have your beautiful xref.el right?  Again Lisp is not
> Java hehe.
> 
> Seriously, because that's where it belongs, at least the way that
> major-modes have been though of throughout the aeons.  If you want
> to change that fundamental property, you are fighting Emacs, in my
> opinion.  I want to find a Python file and start working in the best
> environment.

The only major mode that integrates with xref is emacs-lisp-mode, AFAIK 
(ok, and a couple of derivatives of it). All other integrations are in 
minor modes. That could probably tell you something.

> By the way this reminds me of a Medium thing I read this week
> where one users was fed up with VsCode asking to install packages
> for typing an Hello World in C++.

One user indeed.

>>> And do I really have to download clangd and xcode-specific code
>>> to my machine.
>> This pertaining to..?
> 
> Company.  Did I put it out of context? Sorry.

I still don't understand what this is about. I don't have clangd or any 
xcode stuff on my machine, and I develop Company.

>>> GNU ELPA, as I keep saying.  The two things are completely
>>> unrelated, thankfully.
>> I'm just saying it's not an argument in favor of either of the options.
> 
> It is if you want to rewrite or enhance flyspell, for example.
> Exactly  how I did flymake, a package that was injustly ridiculed
> to the point that someone made a "Flymake done right", tsk tsk

But you wouldn't rewrite Flyspell on rely on LSP only. It would still 
have to be extensible/flexible/open. So how would Eglot's presence help?

>> Do you like that Org is just its own microcosm, and basically never
>> extracts features?
> 
> I think you know my answer to that.  But again, irrelevant.  Stuff
> doesn't happen by the truckload.

So you wouldn't even agree that it would make a lot of sense to move Org 
to ELPA? It's not even developed here.

> PS: this discussion is getting looong.

Yup.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

> I'm not sure how splitting out a "core" package would help if the
> "full" version depends on it, actually. Surely you are going to
> support the users of the "full" version as well? And they will have a
> longer list in `company-backends' by default.

You're not following my idea.  I'm saying:

- make a company-base package that includes the basic company stuff like
  the nice tooltip code and what is now company-capf. (Maybe also move
  that to the core, why not?  :-)) Eglot would depend on just this.

- Make a company-clang, company-xcode, company-foo for users that want
  to use these things.

Then I would remove Eglot's overriding of company-backends, making you
happy in return.  If a user _also_ has company-clang installed and it
conflicts with company-capf, it's his problem, not Eglot's.

> If it behaves sufficiently differently from company (as I recall you
> were planning), you might want to pick a different name.

_That_ idea was a rewrite, something I've done in the past when I
haven't been able to convince the owner of the thing I want integrated.

> For example, "solitude".

Interesting, but doesn't really sell, does it? I prefer "freelance"
or "proletarian".

>> Hm? They would define snippets for the programming they support.
>> Snippets that are active by default once you activate the major mode.
> Or maybe they simply won't.
> You might want to create a separate thread here with a vote and count
> the major mode maintainers who are in. :-)

Major mode mantainers are all of us.

Also, there's a very good thing in Emacs called `define-derived-mode`.
And lots of other options.

> I see your desire to offload some responsibility to major mode
> authors, and I can sympathize with that. But if the people on the
> other side don't pick it up (or pick it up but then forget), this
> won't work. Especially with eglot-server-programs.

Again, major mode maintainers are all of us. I don't view code that
someone else authored as off-limits.  If there is a difference of
opinion, we bring it here and the matter usually resolved in the
interest of both parties.  If the major mode authors simply don't want
to have snippets or LSP or Flymake or treesitter-vapourware.el support,
then there's nothing we can do except making an alternate major mode.

> Not everybody might agree, though. You might notice that there's still
> no cc-mode package in there.

I detect a hint of obsession with cc-mode.  Not good, not good :-)

> It was more of a weak, exasperated laugh. We couldn't evict Alan even
> if we wanted: CC Mode is a beast, and C++ is difficult enough to
> support that there haven't been any real alternatives, I think, ever.
>
> sm-c-mode in GNU ELPA is a cute experiment that supports only a
> fraction of its features. Even for C, IIRC.

But there is TreeSitter.  And LSP.  And probably more parser solutions.
Even Semantic has a parser generator which I dont' think has been
sufficiently explored.

Anyway, C++ is just one language among many.

> From what I see it there, someone could/should make an RLS-specific
> plugin for Eglot.

No, that's not my view at all.  That's precisely what I don't like about
lsp-mode.

> Not sure how major modes would help. There are several language
> servers for Rust, after all.

So the major mode chooses the best one.  Or it can even decide on a
preference order between multiple servers.  Up to the
author/maintainers.

>> See also the `;;; API (WORK-IN-PROGRESS!)` section in eglot.el. Every
>> generic function that takes a server as an argument is a candidate
>> for major mode tweaking.
>
> Like generic functions. Suspicious on the idea of major mode specification.

¯\_(ツ)_/¯

>> to change that fundamental property, you are fighting Emacs, in my
>> opinion.  I want to find a Python file and start working in the best
>> environment.
> The only major mode that integrates with xref is emacs-lisp-mode,
> AFAIK (ok, and a couple of derivatives of it). All other integrations
> are in minor modes. That could probably tell you something.

It tells me those major modes don't have access to the tools that
provide code navigation, because they're not in the core.  Actually
CEDET is, but noone knows how that works.

>> By the way this reminds me of a Medium thing I read this week
>> where one users was fed up with VsCode asking to install packages
>> for typing an Hello World in C++.
> One user indeed.

With a meme-laden Medium account, and lots of fanboys, come on! That has
to count!

>>>> And do I really have to download clangd and xcode-specific code
>>>> to my machine.
>>> This pertaining to..?
>> Company.  Did I put it out of context? Sorry.
>
> I still don't understand what this is about. I don't have clangd or
> any xcode stuff on my machine, and I develop Company.

Sorry, not clangd, I meant clang.  I mean I _certainly_ don't have xcode
but I have a file called company-xcode.el because i installed Company
with code that I suppose is for xcode.  Not only that company-xcode is
in the default value of company-backends.

> But you wouldn't rewrite Flyspell on rely on LSP only. It would still
> have to be extensible/flexible/open. So how would Eglot's presence
> help?

Eglot contains much of the code that would make it possible to rewrite
Flyspell to rely on LSP, even if as an alternative.

> So you wouldn't even agree that it would make a lot of sense to move
> Org to ELPA? It's not even developed here.

If Org is somehow a library helpful to other things in the core,
no.  Otherwise yes, if indeed it is not developed here (in emacs.git)

>> PS: this discussion is getting looong.
> Yup.
Yup, yup, trying to wrap up.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 20.05.2020 03:59, João Távora wrote:

> You're not following my idea.  I'm saying:
> 
> - make a company-base package that includes the basic company stuff like
>    the nice tooltip code and what is now company-capf. (Maybe also move
>    that to the core, why not?  :-)) Eglot would depend on just this.
> 
> - Make a company-clang, company-xcode, company-foo for users that want
>    to use these things.
> 
> Then I would remove Eglot's overriding of company-backends, making you
> happy in return.  If a user _also_ has company-clang installed and it
> conflicts with company-capf, it's his problem, not Eglot's.

Again, note that none of this is predicated, or really affected, by 
company simply being in the core.

>> If it behaves sufficiently differently from company (as I recall you
>> were planning), you might want to pick a different name.
> 
> _That_ idea was a rewrite, something I've done in the past when I
> haven't been able to convince the owner of the thing I want integrated.

You might have more luck after following certain requests by the said owner.

>> For example, "solitude".
> 
> Interesting, but doesn't really sell, does it? I prefer "freelance"
> or "proletarian".

Proletarian sounds the best. Very solid and people-power'd.

>>> Hm? They would define snippets for the programming they support.
>>> Snippets that are active by default once you activate the major mode.
>> Or maybe they simply won't.
>> You might want to create a separate thread here with a vote and count
>> the major mode maintainers who are in. :-)
> 
> Major mode mantainers are all of us.

So you'll move information around but continue to keep it all up-to-date 
yourself, together with other Eglot developers?

>> I see your desire to offload some responsibility to major mode
>> authors, and I can sympathize with that. But if the people on the
>> other side don't pick it up (or pick it up but then forget), this
>> won't work. Especially with eglot-server-programs.
> 
> Again, major mode maintainers are all of us. I don't view code that
> someone else authored as off-limits.  If there is a difference of
> opinion, we bring it here and the matter usually resolved in the
> interest of both parties.  If the major mode authors simply don't want
> to have snippets or LSP or Flymake or treesitter-vapourware.el support,
> then there's nothing we can do except making an alternate major mode.

I'm not even talking about people rejecting such changes. Only about 
that someone would need to make them, and to maintain them thereafter.

>> Not everybody might agree, though. You might notice that there's still
>> no cc-mode package in there.
> 
> I detect a hint of obsession with cc-mode.  Not good, not good :-)

Just a friendly warning, from a fellow revolutionary to another.

>>> to change that fundamental property, you are fighting Emacs, in my
>>> opinion.  I want to find a Python file and start working in the best
>>> environment.
>> The only major mode that integrates with xref is emacs-lisp-mode,
>> AFAIK (ok, and a couple of derivatives of it). All other integrations
>> are in minor modes. That could probably tell you something.
> 
> It tells me those major modes don't have access to the tools that
> provide code navigation, because they're not in the core.

Or that the tooling is decoupled from the major mode definition.

> Actually
> CEDET is, but noone knows how that works.

There is one Indian guy who keeps posting videos of new improvements he 
made, but for some reason isn't hurrying to upstream the changes.

>>> By the way this reminds me of a Medium thing I read this week
>>> where one users was fed up with VsCode asking to install packages
>>> for typing an Hello World in C++.
>> One user indeed.
> 
> With a meme-laden Medium account, and lots of fanboys, come on! That has
> to count!

Then he speaks for all of us, naturally.

>> I still don't understand what this is about. I don't have clangd or
>> any xcode stuff on my machine, and I develop Company.
> 
> Sorry, not clangd, I meant clang.  I mean I _certainly_ don't have xcode
> but I have a file called company-xcode.el because i installed Company
> with code that I suppose is for xcode.  Not only that company-xcode is
> in the default value of company-backends.

Yeah, that one will also have to go.

>> But you wouldn't rewrite Flyspell on rely on LSP only. It would still
>> have to be extensible/flexible/open. So how would Eglot's presence
>> help?
> 
> Eglot contains much of the code that would make it possible to rewrite
> Flyspell to rely on LSP, even if as an alternative.

I don't recall: if Flyspell extensible? Like Flymake, for instance. If 
it is, this seems to call for an eglot-flyspell plugin.

>> So you wouldn't even agree that it would make a lot of sense to move
>> Org to ELPA? It's not even developed here.
> 
> If Org is somehow a library helpful to other things in the core,
> no.  Otherwise yes, if indeed it is not developed here (in emacs.git)

At least we're somewhat compatible on the basics.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1567 bytes --]

On Wed, May 20, 2020, 02:17 Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 20.05.2020 03:59, João Távora wrote:
>
> Again, note that none of this is predicated, or really affected, by
> company simply being in the core.
>

Right. That part is optional, as I wrote.

You might have more luck after following certain requests by the said owner.
>

Wot run errands? Buy milk? Sell my programming soul?

>
 not even talking about people rejecting such changes. Only about
> that someone would need to make them, and to maintain them thereafter.
>

Code doesn't really rot, especially when maintained in the same repository,
built together and tested together. And these things would be using a
stable API presumably. As would your non-major-mode plugin alternatives. So
this point of maintenance is misplaced.

Anyway, major modes in emacs are about loading one file, or  requiring one
feature and having everything set up, even if not enabled immediately. In
my opinion.

Just a friendly warning, from a fellow revolutionary to another.
>

This comrade thanks you.

> With a meme-laden Medium account, and lots of fanboys, come on! That has
> > to count!
>
> Then he speaks for all of us, naturally.
>

She, actually ;-)

I don't recall: if Flyspell extensible? Like Flymake, for instance. If
> it is, this seems to call for an eglot-flyspell plugin.
>

I don't think it is. And extensibility is not a binary thing, depends on
the API. Oftentimes rewriting is better. Flymake was extensible, before the
rewrite.

João

[-- Attachment #2: Type: text/html, Size: 3343 bytes --]

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 20.05.2020 04:37, João Távora wrote:
> On Wed, May 20, 2020, 02:17 Dmitry Gutov <dgutov@yandex.ru 
> <mailto:dgutov@yandex.ru>> wrote:
> 
>     On 20.05.2020 03:59, João Távora wrote:
> 
>     Again, note that none of this is predicated, or really affected, by
>     company simply being in the core.
> 
> 
> Right. That part is optional, as I wrote.

So it's actually off topic for this discussion.

>     You might have more luck after following certain requests by the
>     said owner.
> 
> 
> Wot run errands? Buy milk? Sell my programming soul?

Fix your completion function to honor the c-a-p-f contract (at least 
when it matters).

I wouldn't say no to a milk run, however.

>       not even talking about people rejecting such changes. Only about
>     that someone would need to make them, and to maintain them thereafter.
> 
> 
> Code doesn't really rot, especially when maintained in the same 
> repository, built together and tested together.

I already explained how these settings get outdated.

> Anyway, major modes in emacs are about loading one file, or  requiring 
> one feature and having everything set up, even if not enabled 
> immediately. In my opinion.

Another problem with that idea is that it hints that major modes might 
stop to work properly in the absence of LSP. Or a least some of their 
major functions.

That aside, going back to 
https://github.com/joaotavora/eglot/issues/363, it seems to mention some 
extra configuration that is needed to be done for integration for RLS. 
And it's fiddly, and the format how to specify it is non-obvious.

So IIUC you want to delegate that to both major modes that are available 
for Rust currently. And if the one installed by a user will fail to do 
that (or do it correctly), and the user files a report with us, we will 
shrug and forward them to deal with the authors of their major mode 
(helpfully explaining how to determine what major mode they are 
currently using).

Is that the idea?

>      > With a meme-laden Medium account, and lots of fanboys, come on!
>     That has
>      > to count!
> 
>     Then he speaks for all of us, naturally.
> 
> 
> She, actually ;-)

Touché. :)

>     I don't recall: if Flyspell extensible? Like Flymake, for instance. If
>     it is, this seems to call for an eglot-flyspell plugin.
> 
> 
> I don't think it is. And extensibility is not a binary thing, depends on 
> the API. Oftentimes rewriting is better. Flymake was extensible, before 
> the rewrite.

Sounds like it should be. If so, my point stands.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> Right. That part is optional, as I wrote.
> So it's actually off topic for this discussion.

AFAIU we've been talking about modularity for quite a while, so it's
really very much on topic that we discuss ways to modularize software
and the possibilities that opens.

> Fix your completion function to honor the c-a-p-f contract (at least
> when it matters).

It _is_ honouring it.  You want to revisit that, fine.  Open an issue in
Eglot's bug tracker.

Better yet, M-x report-emacs-bug, put "eglot" somewhere in the title,
and make sure you CC me in the receipt from debbugs (yes I know you know
how to do these things, just testing out a new section in the README.md
;-)  )

> I wouldn't say no to a milk run, however.

Skimmed, full-fat, goat's, what do you want?

>>       not even talking about people rejecting such changes. Only about
>>     that someone would need to make them, and to maintain them thereafter.
>> Code doesn't really rot, especially when maintained in the same
>> repository, built together and tested together.
> I already explained how these settings get outdated.

I missed it.  Can you point to it?

>> Anyway, major modes in emacs are about loading one file, or 
>> requiring one feature and having everything set up, even if not
>> enabled immediately. In my opinion.
>
> Another problem with that idea is that it hints that major modes might
> stop to work properly in the absence of LSP. Or a least some of their 
> major functions.

I mean, major modes can _leverage_ LSP: if it's not there they do
something best-effortish.  Modes can leverage multiple things: up to the
mode.

> That aside, going back to
> https://github.com/joaotavora/eglot/issues/363, it seems to mention
> some extra configuration that is needed to be done for integration for
> RLS. And it's fiddly, and the format how to specify it is non-obvious.

That's a question of interfaces and documentations.  The format to
configure servers in some parts is "free", or "server dependent".  It's
the major mode that must know what is reasonable to provide to the
server that it is configururing.

> So IIUC you want to delegate that to both major modes that are
> available for Rust currently. And if the one installed by a user will
> fail to do that (or do it correctly), and the user files a report with
> us, we will shrug and forward them to deal with the authors of their
> major mode (helpfully explaining how to determine what major mode they
> are currently using).

> Is that the idea?

More or less, yes. That's very much what I do already when the problem
is on the server side.  It's totally reasonable that the user comes to
Eglot first, because that is the user-facing side. But the problem very
often lies partially in the server, so we try to politely inform users
of that and help them debug and/or collect information from Eglot's
interfaces.

Come to think of it, there's another argument for making Eglot
"invisible" in the long run: the report would go directly to foo-mode's
developers.

>>     I don't recall: if Flyspell extensible? Like Flymake, for instance. If
>>     it is, this seems to call for an eglot-flyspell plugin.
>> I don't think it is. And extensibility is not a binary thing,
>> depends on the API. Oftentimes rewriting is better. Flymake was
>> extensible, before the rewrite.
> Sounds like it should be. If so, my point stands.

Doen't stand very strong though.  IIUC you're saying: first invest the
effort in making Flyspell extensible specifically to support LSP, then
develop its LSP backend in a different repo because modularity.  That's
not very enticing :-)

João

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 20.05.2020 19:41, João Távora wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>>> Right. That part is optional, as I wrote.
>> So it's actually off topic for this discussion.
> 
> AFAIU we've been talking about modularity for quite a while, so it's
> really very much on topic that we discuss ways to modularize software
> and the possibilities that opens.

The topic is core vs ELPA.

>> Fix your completion function to honor the c-a-p-f contract (at least
>> when it matters).
> 
> It _is_ honouring it.  You want to revisit that, fine.  Open an issue in
> Eglot's bug tracker.

We've been over this a dozen times, on the issue tracker, in the commit 
comments, etc. If you just want to give me the runaround, it's on you.

>> I wouldn't say no to a milk run, however.
> 
> Skimmed, full-fat, goat's, what do you want?

Goat milk is the best.

>>>        not even talking about people rejecting such changes. Only about
>>>      that someone would need to make them, and to maintain them thereafter.
>>> Code doesn't really rot, especially when maintained in the same
>>> repository, built together and tested together.
>> I already explained how these settings get outdated.
> 
> I missed it.  Can you point to it?

LSP servers come and go, settings get outdated. Like the one that was 
mentioned in the RLS issue, I imagine.

>> So IIUC you want to delegate that to both major modes that are
>> available for Rust currently. And if the one installed by a user will
>> fail to do that (or do it correctly), and the user files a report with
>> us, we will shrug and forward them to deal with the authors of their
>> major mode (helpfully explaining how to determine what major mode they
>> are currently using).
> 
>> Is that the idea?
> 
> More or less, yes. That's very much what I do already when the problem
> is on the server side.  It's totally reasonable that the user comes to
> Eglot first, because that is the user-facing side. But the problem very
> often lies partially in the server, so we try to politely inform users
> of that and help them debug and/or collect information from Eglot's
> interfaces.

Meanwhile, lsp-mode developers will address similar reports directly, 
without footballing the users.

You probably see what I'm getting at.

> Come to think of it, there's another argument for making Eglot
> "invisible" in the long run: the report would go directly to foo-mode's
> developers.

I'm not sure this is how it's going to work. But suppose it will. And 
suppose we only consider the major modes already in Emacs.

If you decide to only field Debbugs issues that have "eglot" or "lsp" in 
the name, you will miss said reports. Who's going to address them?

>>>      I don't recall: if Flyspell extensible? Like Flymake, for instance. If
>>>      it is, this seems to call for an eglot-flyspell plugin.
>>> I don't think it is. And extensibility is not a binary thing,
>>> depends on the API. Oftentimes rewriting is better. Flymake was
>>> extensible, before the rewrite.
>> Sounds like it should be. If so, my point stands.
> 
> Doen't stand very strong though.  IIUC you're saying: first invest the
> effort in making Flyspell extensible specifically to support LSP, then
> develop its LSP backend in a different repo because modularity.  That's
> not very enticing :-)

That's how it's going to have to work anyway.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

>>>> Right. That part is optional, as I wrote.
>>> So it's actually off topic for this discussion.
>> AFAIU we've been talking about modularity for quite a while, so it's
>> really very much on topic that we discuss ways to modularize software
>> and the possibilities that opens.
>
> The topic is core vs ELPA.

Sorry to have drifted the party line.  Anyway, my point, for your
consideration: "I wish Company could be modularized.  Among other
reasons so that a part of it could enter core and be more integrated
there".

>> It _is_ honouring it.  You want to revisit that, fine.  Open an
>> issue in Eglot's bug tracker.
> We've been over this a dozen times, on the issue tracker, in the
> commit comments, etc. If you just want to give me the runaround, it's
> on you.

No.  Burden of proof is on the accuser.  Open new issue, start a thread,
continue an existing one, etc.  Also: "The topic is core vs ELPA."

>>>>        not even talking about people rejecting such changes. Only about
>>>>      that someone would need to make them, and to maintain them thereafter.
>>>> Code doesn't really rot, especially when maintained in the same
>>>> repository, built together and tested together.
>>> I already explained how these settings get outdated.
>> I missed it.  Can you point to it?
> LSP servers come and go, settings get outdated. Like the one that was
> mentioned in the RLS issue, I imagine.

But that's the server's fault.  Hopefully, if major modes start relying
on things, they should choose wisely and rely on some stable stuff.
Just like ispell.el does to spelling programs.  I'm just saying
generalities, because these things are general, they have nothing to do
with LSP.  If one wants integration one has to integrate.

Seems like you don't want much integration.  Fine.  It's been said to be
hard, and that's true.  But there _is_ an I in IDE, and it doesn't stand
for "immense .emacs".

>> is on the server side.  It's totally reasonable that the user comes to
>> Eglot first, because that is the user-facing side. But the problem very
>> often lies partially in the server, so we try to politely inform users
>> of that and help them debug and/or collect information from Eglot's
>> interfaces.
> Meanwhile, lsp-mode developers will address similar reports directly,
> without footballing the users.

They regularly solve LSP server bugs in lsp-mode? Very bad idea. Relying
on features is one thing, relying on bugs is another thing entirely.

>> Come to think of it, there's another argument for making Eglot
>> "invisible" in the long run: the report would go directly to foo-mode's
>> developers.
>
> I'm not sure this is how it's going to work. But suppose it will. And
> suppose we only consider the major modes already in Emacs.
>
> If you decide to only field Debbugs issues that have "eglot" or "lsp"
> in the name, you will miss said reports. Who's going to address them?

I supose it'll work the same way we deal with a bug report about "my
colours are all garbled" and it turns out to be about font-lock, which
the user is oblivious to.  Again, nothing to do with LSP.  And again,
you're putting the cart of the apocalypse way in front of the horses of
the apocalypse.

>>> Sounds like it should be. If so, my point stands.
>> Doen't stand very strong though.  IIUC you're saying: first invest
>> the effort in making Flyspell extensible specifically to support LSP,
>> then develop its LSP backend in a different repo because modularity.
>> That's not very enticing :-)
> That's how it's going to have to work anyway.

No. If there is LSP support in emacs.git and Flyspell is in emacs.git,
you just develop in the same repo, emacs.git.

João

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 22.05.2020 13:49, João Távora wrote:

>>> It _is_ honouring it.  You want to revisit that, fine.  Open an
>>> issue in Eglot's bug tracker.
>> We've been over this a dozen times, on the issue tracker, in the
>> commit comments, etc. If you just want to give me the runaround, it's
>> on you.
> 
> No.  Burden of proof is on the accuser.  Open new issue, start a thread,
> continue an existing one, etc.  Also: "The topic is core vs ELPA."

You've made my point.

>>>>>         not even talking about people rejecting such changes. Only about
>>>>>       that someone would need to make them, and to maintain them thereafter.
>>>>> Code doesn't really rot, especially when maintained in the same
>>>>> repository, built together and tested together.
>>>> I already explained how these settings get outdated.
>>> I missed it.  Can you point to it?
>> LSP servers come and go, settings get outdated. Like the one that was
>> mentioned in the RLS issue, I imagine.
> 
> But that's the server's fault.> Hopefully, if major modes start relying
> on things, they should choose wisely and rely on some stable stuff.

Hopefully major modes don't.

> Just like ispell.el does to spelling programs.  I'm just saying
> generalities, because these things are general, they have nothing to do
> with LSP.  If one wants integration one has to integrate.

Did you just compare RLS to Aspell?

Relying on a "stable" program is fine until it doesn't do the job anymore.

> Seems like you don't want much integration.  Fine.  It's been said to be
> hard, and that's true.  But there _is_ an I in IDE, and it doesn't stand
> for "immense .emacs".

I never suggested to put the burden of configuring this on the user.

>>> is on the server side.  It's totally reasonable that the user comes to
>>> Eglot first, because that is the user-facing side. But the problem very
>>> often lies partially in the server, so we try to politely inform users
>>> of that and help them debug and/or collect information from Eglot's
>>> interfaces.
>> Meanwhile, lsp-mode developers will address similar reports directly,
>> without footballing the users.
> 
> They regularly solve LSP server bugs in lsp-mode? Very bad idea. Relying
> on features is one thing, relying on bugs is another thing entirely.

Did that report concern a bug in a language server? It didn't look that 
way to me.

>>> Come to think of it, there's another argument for making Eglot
>>> "invisible" in the long run: the report would go directly to foo-mode's
>>> developers.
>>
>> I'm not sure this is how it's going to work. But suppose it will. And
>> suppose we only consider the major modes already in Emacs.
>>
>> If you decide to only field Debbugs issues that have "eglot" or "lsp"
>> in the name, you will miss said reports. Who's going to address them?
> 
> I supose it'll work the same way we deal with a bug report about "my
> colours are all garbled" and it turns out to be about font-lock, which
> the user is oblivious to.

Offloading work to other contributors, I see.

>>>> Sounds like it should be. If so, my point stands.
>>> Doen't stand very strong though.  IIUC you're saying: first invest
>>> the effort in making Flyspell extensible specifically to support LSP,
>>> then develop its LSP backend in a different repo because modularity.
>>> That's not very enticing :-)
>> That's how it's going to have to work anyway.
> 
> No. If there is LSP support in emacs.git and Flyspell is in emacs.git,
> you just develop in the same repo, emacs.git.

So what? Implementation-wise, it will be a plugin either way. Which 
would be just as easy to put in ELPA.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

>>> Meanwhile, lsp-mode developers will address similar reports directly,
>>> without footballing the users.
>> They regularly solve LSP server bugs in lsp-mode? Very bad
>> idea. Relying
>> on features is one thing, relying on bugs is another thing entirely.
> Did that report concern a bug in a language server? It didn't look
> that way to me.

I wasn't speaking of any particular one. I was telling how we work with
Eglot users that report server bugs to us.  You talked about
"footballing users" and how lsp-mode adress them "directly".  I assumed
that means they hack lsp-mode.el of lsp-foo.el to work around server
bugs, but I really have no idea.  In fact I don't know what we're
talking about anymore, I have to admit.

>>> If you decide to only field Debbugs issues that have "eglot" or "lsp"
>>> in the name, you will miss said reports. Who's going to address them?
>> I supose it'll work the same way we deal with a bug report about "my
>> colours are all garbled" and it turns out to be about font-lock, which
>> the user is oblivious to.
> Offloading work to other contributors, I see.

Drats, you've uncovered my master plan.  And I would have gotten away
with it too.

>>> That's how it's going to have to work anyway.
>> No. If there is LSP support in emacs.git and Flyspell is in
>> emacs.git,
>> you just develop in the same repo, emacs.git.
> So what? Implementation-wise, it will be a plugin either way. Which
> would be just as easy to put in ELPA.

I wrote "develop".  And for the zillionth time: it _can_ be in ELPA too.

João

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 22.05.2020 17:32, João Távora wrote:
>>>> Meanwhile, lsp-mode developers will address similar reports directly,
>>>> without footballing the users.
>>> They regularly solve LSP server bugs in lsp-mode? Very bad
>>> idea. Relying
>>> on features is one thing, relying on bugs is another thing entirely.
>> Did that report concern a bug in a language server? It didn't look
>> that way to me.
> I wasn't speaking of any particular one. I was telling how we work with
> Eglot users that report server bugs to us.  You talked about
> "footballing users" and how lsp-mode adress them "directly".  I assumed
> that means they hack lsp-mode.el of lsp-foo.el to work around server
> bugs, but I really have no idea.  In fact I don't know what we're
> talking about anymore, I have to admit.

We're talking about this issue you mentioned: 
https://github.com/joaotavora/eglot/issues/363

The resolution there, it seems, is that the user must discover which 
data, and in which format, to add to eglot-workspace-configuration for 
stuff to work as expected.

In the meantime (as I have just found out by doing a search), lsp-rust 
both contains this setting at a reasonable default:

https://github.com/emacs-lsp/lsp-mode/blob/057e8789638a0bf493930637185694b6b09ea58e/lsp-rust.el#L267

...and exposes the possible values of this setting in a well-documented 
user option:

https://github.com/emacs-lsp/lsp-mode/blob/057e8789638a0bf493930637185694b6b09ea58e/lsp-rust.el#L185

So, which of these two approaches to development does look more 
"integrated" to you?

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 22.05.2020 17:32, João Távora wrote:
> We're talking about this issue you mentioned:
> https://github.com/joaotavora/eglot/issues/363

I was talking about user reports about server bugs, but no matter we can
talk bout what you want to talk.

> The resolution there, it seems, is that the user must discover which
> data, and in which format, to add to eglot-workspace-configuration for 
> stuff to work as expected.
>
> In the meantime (as I have just found out by doing a search), lsp-rust
> both contains this setting at a reasonable default:
>
> https://github.com/emacs-lsp/lsp-mode/blob/057e8789638a0bf493930637185694b6b09ea58e/lsp-rust.el#L267
>
> ...and exposes the possible values of this setting in a
> well-documented user option:
>
> https://github.com/emacs-lsp/lsp-mode/blob/057e8789638a0bf493930637185694b6b09ea58e/lsp-rust.el#L185
>
> So, which of these two approaches to development does look more
> "integrated" to you?

You know the answer: the one where one does the latter in rust-mode

This x-y.el way of working where x is the extension and y is the
language is an explicit anti-goal.  I know you prefer it, you have in
company, but I don't, I don't prefer it.

Of course, as you know, the whole point of LSP to start with is to make
these obsolete.  It's not 100% there, and thus you need these
customizations (which again, should IMHO live in the major-mdoe).  But
ideally they shouln't be needed.

João

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 22.05.2020 22:44, João Távora wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>> On 22.05.2020 17:32, João Távora wrote:
>> We're talking about this issue you mentioned:
>> https://github.com/joaotavora/eglot/issues/363
> 
> I was talking about user reports about server bugs, but no matter we can
> talk bout what you want to talk.

This is the bug report you showed, and I made some extra observations.

>> The resolution there, it seems, is that the user must discover which
>> data, and in which format, to add to eglot-workspace-configuration for
>> stuff to work as expected.
>>
>> In the meantime (as I have just found out by doing a search), lsp-rust
>> both contains this setting at a reasonable default:
>>
>> https://github.com/emacs-lsp/lsp-mode/blob/057e8789638a0bf493930637185694b6b09ea58e/lsp-rust.el#L267
>>
>> ...and exposes the possible values of this setting in a
>> well-documented user option:
>>
>> https://github.com/emacs-lsp/lsp-mode/blob/057e8789638a0bf493930637185694b6b09ea58e/lsp-rust.el#L185
>>
>> So, which of these two approaches to development does look more
>> "integrated" to you?
> 
> You know the answer: the one where one does the latter in rust-mode

I don't think that's what "integrated" means.

> This x-y.el way of working where x is the extension and y is the
> language is an explicit anti-goal.  I know you prefer it, you have in
> company, but I don't, I don't prefer it.

It's not like I prefer it. And I'm happy when delegating this stuff to 
major modes (via c-a-p-f) works. These particular responsibilities, 
however, the ones you want to delegate, are more technical, and require 
the knowledge of ecosystem more alien to Emacs, and the settings 
themselves will likely become outdated, or incomplete, on the regular basis.

OTOH, when a major mode provides a completion function that's outdated, 
users will install an extension that provides "better" completions (via 
company-backends or, again, c-a-p-f), and will consider it as a 
self-contained, distinct piece of improved functionality.

If the LSP settings are broken or outdated, however, the user are more 
likely to see that as Eglot being broken or obsolete.

> Of course, as you know, the whole point of LSP to start with is to make
> these obsolete.
Some basic idea was like that, maybe. But then the reality intervened. 
And people value features more than stability.

And I'm fairly sure language servers will continue to provide their 
ad-hoc extensions because different languages have different needs.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

> I don't think that's what "integrated" means.

Why not? You find the Rust file, activate Eglot and it starts working.
Some major modes might even automate the second step, if it's cheap.  I
know people really like `eglot-ensure`, which that starts up a server
automatically.

> If the LSP settings are broken or outdated, however, the user are more
> likely to see that as Eglot being broken or obsolete.

I've explained this has specific to do with LSP.  It's the fact that you
rely on an external program.  Emacs does that all the time, with aspell,
shell programs, so nothing different here.  Aren't you ruby-mode
maintainer?  Doesn't it call the rubocop program?  Is it rotting
frequently?

>> Of course, as you know, the whole point of LSP to start with is to make
>> these obsolete.
> Some basic idea was like that, maybe. But then the reality
> intervened. And people value features more than stability.

I don't think it's very bad, the fact that only minor customization is
needed for a small set of servers means the idea is mostly solid.  It's
a question of good defaults, mostly.  And growing the protocol.

> And I'm fairly sure language servers will continue to provide their
> ad-hoc extensions because different languages have different needs.

Yes, true, but from these two years I've been seeing less and less of
that.  I bet a good majority of the servers Eglot supports work out of
the box, no special customization.  pyls, clangd, the javascript one,
and probably many of the other simpler ones. 

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 23.05.2020 03:08, João Távora wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>> I don't think that's what "integrated" means.
> 
> Why not? You find the Rust file, activate Eglot and it starts working.
> Some major modes might even automate the second step, if it's cheap.  I
> know people really like `eglot-ensure`, which that starts up a server
> automatically.

It's less "integrated" than what that other mode is doing.

So worst case scenario, people will point out that Emacs core has 
released something not entirely complete while there is a package with 
all batteries included.

>> If the LSP settings are broken or outdated, however, the user are more
>> likely to see that as Eglot being broken or obsolete.
> 
> I've explained this has specific to do with LSP.  It's the fact that you
> rely on an external program.  Emacs does that all the time, with aspell,
> shell programs, so nothing different here.  Aren't you ruby-mode
> maintainer?  Doesn't it call the rubocop program?  Is it rotting
> frequently?

Again, examples of much simpler programs. And historically stable. And 
Rubocop has its own, well-documented configuration interface (text file) 
that's expected to be edited by users.

>>> Of course, as you know, the whole point of LSP to start with is to make
>>> these obsolete.
>> Some basic idea was like that, maybe. But then the reality
>> intervened. And people value features more than stability.
> 
> I don't think it's very bad, the fact that only minor customization is
> needed for a small set of servers means the idea is mostly solid.  It's
> a question of good defaults, mostly.  And growing the protocol.

And of every major mode author making the effort to familiarize 
themselves with this stuff, apparently.

>> And I'm fairly sure language servers will continue to provide their
>> ad-hoc extensions because different languages have different needs.
> 
> Yes, true, but from these two years I've been seeing less and less of
> that.  I bet a good majority of the servers Eglot supports work out of
> the box, no special customization.  pyls, clangd, the javascript one,
> and probably many of the other simpler ones.

You might be forgetting that Eglot doesn't exactly support _all_ the 
features that languages servers offer. At least, that's my impression of 
your answer to the question about refactoring actions.

And take the lsp-rust example again: 
https://github.com/emacs-lsp/lsp-mode/blob/057e8789638a0bf493930637185694b6b09ea58e/lsp-rust.el#L259-L283

There are 24 settings there. Perhaps half is outdated or unimportant. 
Who will be responsible for setting them and choosing their values?

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 23.05.2020 03:08, João Távora wrote:
>> Dmitry Gutov <dgutov@yandex.ru> writes:
>> 
>>> I don't think that's what "integrated" means.
>> Why not? You find the Rust file, activate Eglot and it starts
>> working.
>> Some major modes might even automate the second step, if it's cheap.  I
>> know people really like `eglot-ensure`, which that starts up a server
>> automatically.
>
> It's less "integrated" than what that other mode is doing.

I don't see how.  We must be talking about different things.  I'm
talking Emacs -Q, C-x C-f foo.xxx RET and xxx-mode kicks in, sets up and
maybe even starts Eglot.  For that to work, xxx-mdoe has to be in core,
otherwise you need M-x package-install xxx-mode

> So worst case scenario, people will point out that Emacs core has
> released something not entirely complete while there is a package with 
> all batteries included.

You're saying this becasue rust-mode isn't in the core and/or GNU ELPA?
Than it's not a problem of Eglot/LSP at all.  

> Again, examples of much simpler programs. And historically stable. And
> Rubocop has its own, well-documented configuration interface (text
> file) that's expected to be edited by users.

Don't see any reson why an LSP server would be some kind of second-grade
delinquent program.

> And of every major mode author making the effort to familiarize
> themselves with this stuff, apparently.

Same with font-lock and imenu and eldoc and forward-sexp-function.  Mode
authors needn't familizarize themselves with the protocol, only in some
cases with the server program they're setting up.  Like rubocop.

> You might be forgetting that Eglot doesn't exactly support _all_ the
> features that languages servers offer. At least, that's my impression
> of your answer to the question about refactoring actions.

Eglot supports refactoring actions.  Commands and renames and stuff . I
don't know what where you want to go with this.  Give an example.  If
Eglot doesn't support something we can add it.  LSP works by
"capability", what capability are you exactly referring to?  And not all
servers support the same capabilities.

> And take the lsp-rust example again:
> https://github.com/emacs-lsp/lsp-mode/blob/057e8789638a0bf493930637185694b6b09ea58e/lsp-rust.el#L259-L283
>
> There are 24 settings there.

Yuck.  Well they seem to be doing stuff that rust-mode could do in aone
of Eglots variables and a generic function.

I think you should acquaint yourself with Eglot and some of these
servers, and then bring specific problems, here or to the tracker.  I or
some other member of the team will be happy to answer.  Else we're just
speaking in the abstract, you trying to convince me that something
somewhere someday is not going to work.

João

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 23.05.2020 03:48, João Távora wrote:

>>> Some major modes might even automate the second step, if it's cheap.  I
>>> know people really like `eglot-ensure`, which that starts up a server
>>> automatically.
>>
>> It's less "integrated" than what that other mode is doing.
> 
> I don't see how.  We must be talking about different things.  I'm
> talking Emacs -Q, C-x C-f foo.xxx RET and xxx-mode kicks in, sets up and
> maybe even starts Eglot.  For that to work, xxx-mdoe has to be in core,
> otherwise you need M-x package-install xxx-mode

*And* you need the author of xxx-mode to participate in the scheme.

>> So worst case scenario, people will point out that Emacs core has
>> released something not entirely complete while there is a package with
>> all batteries included.
> 
> You're saying this becasue rust-mode isn't in the core and/or GNU ELPA?
> Than it's not a problem of Eglot/LSP at all.

I'm talking about user perception, you respond with "not a problem".

All right then.

>> Again, examples of much simpler programs. And historically stable. And
>> Rubocop has its own, well-documented configuration interface (text
>> file) that's expected to be edited by users.
> 
> Don't see any reson why an LSP server would be some kind of second-grade
> delinquent program.

I integrated Rubocop because I'm familiar with it, just like almost 
every Rubyist. I don't imagine we could say the same about all major 
mode authors and the LSP ecosystem.

>> And of every major mode author making the effort to familiarize
>> themselves with this stuff, apparently.
> 
> Same with font-lock and imenu and eldoc and forward-sexp-function.  Mode
> authors needn't familizarize themselves with the protocol, only in some
> cases with the server program they're setting up.  Like rubocop.

The font-lock, imenu, eldoc and f-s-f provide extension points where the 
extensions can basically indulge in unbounded levels of creativity, 
support various languages, frameworks, libraries, and so on.

There's literally no possibility to contain that all in a file shipped 
with Emacs. *And* a significant portion of these extensions out there 
requires knowledge and experience that the authors of these extension 
points do not possess. They only needed to create flexibility.

The situation with LSP is different: the total creativity is limited by 
the LSP protocol, and the extensions will be pertaining to LSP server 
settings, if I understand your examples so far.

>> You might be forgetting that Eglot doesn't exactly support _all_ the
>> features that languages servers offer. At least, that's my impression
>> of your answer to the question about refactoring actions.
> 
> Eglot supports refactoring actions.  Commands and renames and stuff . I
> don't know what where you want to go with this.  Give an example.  If
> Eglot doesn't support something we can add it.  LSP works by
> "capability", what capability are you exactly referring to?  And not all
> servers support the same capabilities.

What about Java development? Does Eglot provide all capabilities 
included in the "LSP Java commands" section here, in some shape or form?

https://github.com/emacs-lsp/lsp-java/#supported-commands

Here's some other stuff that seems unlikely to be in the spec.

FSharp-specific settings and notifications: 
https://github.com/emacs-lsp/lsp-mode/blob/master/lsp-fsharp.el

Metals specific notifications and commands: 
https://github.com/emacs-lsp/lsp-mode/blob/master/lsp-metals.el

 > Else we're just
speaking in the abstract, you trying to convince me that something
somewhere someday is not going to work.

I'm not saying it can't possibly work, just that it looks suboptimal, 
given existing development trends and areas of responsibility.

And I'm not crazy about the increased coupling to major modes.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 23.05.2020 03:48, João Távora wrote:
>
>>>> Some major modes might even automate the second step, if it's cheap.  I
>>>> know people really like `eglot-ensure`, which that starts up a server
>>>> automatically.
>>>
>>> It's less "integrated" than what that other mode is doing.
>> I don't see how.  We must be talking about different things.  I'm
>> talking Emacs -Q, C-x C-f foo.xxx RET and xxx-mode kicks in, sets up and
>> maybe even starts Eglot.  For that to work, xxx-mdoe has to be in core,
>> otherwise you need M-x package-install xxx-mode
>
> *And* you need the author of xxx-mode to participate in the scheme.

That has nothing to do with "integrated" as in IDE.  One presents a
patch to the author: if he doesn't want to participate in the scheme,
then he doesn't want to, period.  But it's very unlikely, since there's
no downside.  It's like not wanting to participate in, say, keybindings
or font-lock.  Even Alan agreed some lines of flymake stuff to go into
cc-mode.el after understanding they weren't going to explode.

>>> So worst case scenario, people will point out that Emacs core has
>>> released something not entirely complete while there is a package with
>>> all batteries included.
>> You're saying this becasue rust-mode isn't in the core and/or GNU
>> ELPA?
>> Than it's not a problem of Eglot/LSP at all.

> I'm talking about user perception, you respond with "not a problem".

Let's stop this silliness.  I wrote: _not a problem of Eglot/LSP_.  How
could it be? If Emacs doesn't have rust-mode out of the box, you can't
use rust-mode out of the box, with copious apologies to users and their
perceptions and everyone other reader having to endure my statement of
the very obvious.

>>> Again, examples of much simpler programs. And historically stable. And
>>> Rubocop has its own, well-documented configuration interface (text
>>> file) that's expected to be edited by users.
>> Don't see any reson why an LSP server would be some kind of
>> second-grade
>> delinquent program.
>
> I integrated Rubocop because I'm familiar with it, just like almost
> every Rubyist. I don't imagine we could say the same about all major 
> mode authors and the LSP ecosystem.

They don't have to be familiar with the "LSP ecosystem" just a
particular program and its invocation.  The newbiest of users can help
there, it's trivial to setup.  You're totally off in this: Eglot
supports more than 20 servers (yes, it supports more than are listed)
and the majority most of them work very nicely without any
customization.

>>> And of every major mode author making the effort to familiarize
>>> themselves with this stuff, apparently.
>> Same with font-lock and imenu and eldoc and forward-sexp-function.
>> Mode
>> authors needn't familizarize themselves with the protocol, only in some
>> cases with the server program they're setting up.  Like rubocop.
>
> The font-lock, imenu, eldoc and f-s-f provide extension points where
> the extensions can basically indulge in unbounded levels of
> creativity, support various languages, frameworks, libraries, and so
> on.
>
> There's literally no possibility to contain that all in a file shipped
> with Emacs. *And* a significant portion of these extensions out there 
> requires knowledge and experience that the authors of these extension
> points do not possess. They only needed to create flexibility.
>
> The situation with LSP is different: the total creativity is limited
> by the LSP protocol, and the extensions will be pertaining to LSP
> server settings, if I understand your examples so far.

Yes, so you're making my point.  The things that go into the major mode
are usually minimal, usually little more than setting the value of
`eglot-server-program`. But those minimal bits do belong there.

>>> You might be forgetting that Eglot doesn't exactly support _all_ the
>>> features that languages servers offer. At least, that's my impression
>>> of your answer to the question about refactoring actions.
>> Eglot supports refactoring actions.  Commands and renames and stuff
>> . I
>> don't know what where you want to go with this.  Give an example.  If
>> Eglot doesn't support something we can add it.  LSP works by
>> "capability", what capability are you exactly referring to?  And not all
>> servers support the same capabilities.
>
> What about Java development? Does Eglot provide all capabilities
> included in the "LSP Java commands" section here, in some shape or
> form?

Eglot supports LSP, the Language Server Protocol, a mechanism whereby
editors don't have to care about the specifics details of a language.
We don't support language specific extensions, but we do support an API
for major-modes or users to enter into that business if they can't
select a better server or work to improve the protocol.  If you have a
specific thing would like to be able to do, I'm more than open to hear
about it, but shoving some lsp-mode files here to make me hunt its bloat
for dubious language-specific features of servers that I don't use is
not my idea of fun.

Eglot was designed specifically with certain ideal of Emacs in mind, one
you're totally free not to agree with.  Goes like this: major modes
customize little bits of minor modes and other infrastrcuture to assist
them in their work.  I see Eglot as LSP infrastructure much as
font-lock, imenu, syntax, flymake, you name it.  I will only do the
<thing>-<language>.el dance as a last resort, (cf. flymake-cc.el), when
it's hard to communicate with the major mode author.  But my starting
position assumes it won't be difficult to do so, because that's how my
ideal of Emacs is: open, collaborative and proven good stuff goes where
it needs to go.

The fact that Eglot started of as a separate package was purely
intrumental: I needed to test some ideas beforehand and convince myself
and the world that such an approach was possible.  I'm sorry if you're
not convinced (or only half-convinced).  I think it would help if you
actually tried it a bit and collected some actual problems or ideas.

> I'm not saying it can't possibly work, just that it looks suboptimal,
> given existing development trends and areas of responsibility.
  
Sorry, but that sounds a bit like generic managerial talk, very little
juice there.  Bring real problems to the table.  Bring something of the
form: "I was trying to this and that with eglot.el, but then this dragon
appeared, how do I slay him?".  Bring that to the issue tracker.

I don't think your worries about major-mode authors being burdened, or
too many emacs-diffs emails, or confused users, hold much water,
frankly.  The idea is for Emacs 28 to have good LSP support in the core,
and I'm encouraged to see some support for that idea.  Incidentally,
there are much larger challenges with LSP such as arguing for a more
inclusive spec -- which is the whole point of LSP.  Perhaps you could
cast your energy and theoretical concerns in that direction.

I'm signing off this sub-thread, there's nothing more to add here, you
can have the last words/make your final arguments.

João

RE: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[arthur miller]

[-- Attachment #1: Type: text/plain, Size: 6362 bytes --]

Lately, you have talked so much about inclusion, Elpa, Melpa,  making Emacs easier, what new functionality would be useful, etc.

But nobody talks about excluding stuff. One of Emacs complexities is that there are so many different paths to accomplish same thing. Autocomplete - Company, Helm-Ivy, semantic-lsp, different spellcheckers, skeleton-yasnippet, etc. While I am all for democracy and freedom and Emacs extensible nature is perfect ground for growing diversity, it might not always be that too many choices equal freedom.

Maybe it could benefit if you clean up Emacs of some old cruft, and maybe replace some old for new. I mean be brave enough to make a choice and be vise enough to leave options for those who prefer differ path.

There will always be someone who disagrees with any choice of package you make and in general with any choice you made. If you wish to please everyone you will never please anyone. And whatever choices you make today will probably change in 40 years to come anyway, so it is not like change is written in stone forever.


Skickat från min Samsung Galaxy-smartphone.



-------- Originalmeddelande --------
Från: Dmitry Gutov <dgutov@yandex.ru>
Datum: 2020-05-19 17:25 (GMT+01:00)
Till: João Távora <joaotavora@gmail.com>
Kopia: Richard Stallman <rms@gnu.org>, joostkremers@fastmail.fm, emacs-devel <Emacs-devel@gnu.org>, "Alfred M. Szmidt" <ams@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>, 조성빈 <pcr910303@icloud.com>, Eli Zaretskii <eliz@gnu.org>, Phillip Lord <phillip.lord@russet.org.uk>
Ämne: Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

On 19.05.2020 17:11, João Távora wrote:
> On Sun, May 17, 2020 at 2:38 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
>> I would like to point out, as an author of several packages, that in my
>> experience having a package in ELPA is _better_ than having it in the core.
>
> I know you like to develop on GitHub, and I'm not going to argue that:
> it is a completely separate discussion.

Indeed. That's why I was arguing about different things, rather than how
nice it is to be able to use a different hosting and bug tracker.

> I disagree on specific technical reasons that IMO don't get enough attention:

Below, you are arguing about two specific packages. I didn't say that
_no_ packages should be in the core, however. Just that for most of them
there is not much additional benefit from that.

But we can discuss these examples, too.

> 1. If Company were in the core, you could have major modes in
> the core or elsewhere setting up the desired completion strategy or
> strategies.  I.e. instead of  having central database of
> `company-backends` that packages like Eglot have to override to behave
> sanely, you could have a buffer-local  `company-backend` instead. You c
> ould continue to develop the specific eclim, clang, xcode, etc backends
> on GitHub, but you wouldn't "force" them on people that don't use them.

Major modes already define c-a-p-f, we have a frontend-agnostic API for
that. On top of it, however, Company provides finer-grained controls for
the users, which can't be decided by major modes.

I don't think 'M-x completion-at-point' can be realistically expected to
go away in any near future, so it doesn't seem like some tighter
integration with major modes is on the table (and I don't know what it
would look like anyway).

As for Eglot, it seems to successfully do what you want here with no
extra boilerplate. The only problem I see there is breaking some users'
existing configurations.

> 2. If Yasnippet were in the core, again, no need for the centralized
> database of snippets: major modes define their snippets.  Other tools
> can use the snippet-defining infrastructure without eating all that
> João-the-ten-years-ago newbie cruft. But even with yasnippet.el, it
> is possible (though not ideal: Stefan once convinced me to rewrite it
> which I did to 90% in a so-called snippet.el -- but there's still the
> 10% missing).

Yasnippet has been wildly successful without all that. It's the #1
standard snippet solution and format for Emacs. It's in GNU ELPA,
everybody can use and depend on it.

About "newbie cruft", how will that go away without somebody rewriting
yasnippet's code? And when they do, the result can sit in GNU ELPA as well.

Finally, the only snippet collections I still see maintained are being
done by third-party developers. If the major mode authors (BTW, how many
are actively maintaining major modes in the core?) wanted to also add
snippet collections, they'd have already done so.

And even if they do the new snippets will only reach the users once
every 2 years. (facepalm emoji)

> 3. If Eglot were in the core, again, no need for the centralized stuff
> currently living in eglot-server-programs.

Right. So CC Mode will define which completion server will be used by
Eglot? Really?

And suppose major mode authors even decide to take up that
responsibility, the LSP field moves much faster than Emacs these days.
Six month after Emacs 27 is released, another LSP server for C++ falls
out of fashion and stops being developed, and Emacs users will stay with
outdated settings until the next release. Or until their distro switches
to Emacs 28, actually, which can be another several years.

> Also, major modes in the
> core can affect Eglot's LSP interfaces via stronger coupling techniques,
> such as adding methods to generic functions.

You still never gave an example for that.

> Besides major modes,
> other tools than build on some LSP basics, such as an LSP-enabled
> spell checker could be built.

Yes it can. Anything stops it from being in ELPA?

> Also, I want to note, again, that even if Company, snippet.el and
> Eglot  were in the core, they could still _also_ be available
> for download via GNU ELPA.

I also noted that later.

>> The exception to that rule is more or less cases where the package is
>> not only added but also enabled by default.
>
> I've argued that that's not the only exception.

I'd argue if we call something "infrastructure", then it should either
be enabled by default, or used by other packages, or otherwise have a
necessary stronger coupling to other code. This is a fairly high barrier.


[-- Attachment #2: Type: text/html, Size: 8016 bytes --]

Discarding superfluous old alternative packages

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Maybe it could benefit if you clean up Emacs of some old cruft,
  > and maybe replace some old for new. I mean be brave enough to make
  > a choice and be vise enough to leave options for those who prefer
  > differ path.

In principle, this might be a good thing to do.  It could be that newer options
B, C, D and E cover all needs so that it would be fine if we did not have A.

But there could be many existing users of A who would be unhappy if we
deleted it now.

We could perhaps move A to GNU ELPA, and ask users to switch to B, C,
D or E within two years.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If we regard ELPA just as distribution site, we shouldn't decide so lightly to leave important packages on ELPA, we should be tend more to adding them to core.

I don't see how that follows necessarily.  Can you explain what leads
you from the premise to that conclusion?
-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: monnier@iro.umontreal.ca, joostkremers@fastmail.fm, ams@gnu.org,
> 	Emacs-devel@gnu.org, pcr910303@icloud.com,
> 	phillip.lord@russet.org.uk
> Date: Sun, 17 May 2020 23:48:59 -0400
> 
>   > If we regard ELPA just as distribution site, we shouldn't decide so lightly to leave important packages on ELPA, we should be tend more to adding them to core.
> 
> I don't see how that follows necessarily.  Can you explain what leads
> you from the premise to that conclusion?

Basically, I believe that if ELPA is just a distribution site, putting
a package there sends a message that we aren't interested in a package
and are indifferent to its contents, coding standards, etc.  In
practice this means that there's nothing to allow us to recommend
those packages over those from other archives, except that we checked
the license and made sure it doesn't use non-free software (and even
the latter will probably become with time a superficial check that
will not reveal enough).

When someone comes with a package that implements important features,
I'd like them to understand that, and be prepared to cooperate and
coordinate with us, whether immediately or in the future.  That needs
better ties that just "here's your place; use it wisely".

Splitting GNU ELPA

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Basically, I believe that if ELPA is just a distribution site, putting
  > a package there sends a message that we aren't interested in a package
  > and are indifferent to its contents, coding standards, etc.  In
  > practice this means that there's nothing to allow us to recommend
  > those packages over those from other archives,

That is true, but we have no reason to compare one archive with
another archive.  We don't want to disparage MELPA, just avoid
referring people to it.  So we have no wish to compare anything with
it.

We would refer people to the no-assignments Exo-ELPA in the sense of
telling people "It is there", but that is all we would do.  We don't
wish to say to people "It is good", only "we don't morally object to
them."

						   except that we checked
  > the license and made sure it doesn't use non-free software (and even
  > the latter will probably become with time a superficial check that
  > will not reveal enough).

We should take care not to let that happen, and invite people to report
that sort of problem to us.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Splitting GNU ELPA

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > When someone comes with a package that implements important features,
  > I'd like them to understand that, and be prepared to cooperate and
  > coordinate with us, whether immediately or in the future.  That needs
  > better ties that just "here's your place; use it wisely".

That makes sense.  Could you make a more concrete recommendation?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Splitting GNU ELPA

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: joostkremers@fastmail.fm, Emacs-devel@gnu.org, ams@gnu.org,
> 	monnier@iro.umontreal.ca, pcr910303@icloud.com,
> 	phillip.lord@russet.org.uk
> Date: Mon, 18 May 2020 23:56:00 -0400
> 
>   > When someone comes with a package that implements important features,
>   > I'd like them to understand that, and be prepared to cooperate and
>   > coordinate with us, whether immediately or in the future.  That needs
>   > better ties that just "here's your place; use it wisely".
> 
> That makes sense.  Could you make a more concrete recommendation?

A more concrete recommendation would be to examine each proposed
package more closely and try to decide whether we'd like to have that
in Emacs or "close by", or just offer it for download to those who are
interested (a.k.a. "just distribute").  This should include some
meaningful discussion with arguments on both directions, and hopefully
some consensus about the decision.

We could also ask the package developer to state their POV on these
matters, and their plans for further development and maintenance.

Re: Splitting GNU ELPA

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > A more concrete recommendation would be to examine each proposed
  > package more closely and try to decide whether we'd like to have that
  > in Emacs or "close by", or just offer it for download to those who are
  > interested (a.k.a. "just distribute").  This should include some
  > meaningful discussion with arguments on both directions, and hopefully
  > some consensus about the decision.

  > We could also ask the package developer to state their POV on these
  > matters, and their plans for further development and maintenance.

It seems good to me.  Do you suggest making a list of questions
and sending that to each developer, or would you rather just talk
with them without a preplanned list?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Splitting GNU ELPA

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: joostkremers@fastmail.fm, Emacs-devel@gnu.org, ams@gnu.org,
> 	monnier@iro.umontreal.ca, pcr910303@icloud.com,
> 	phillip.lord@russet.org.uk
> Date: Wed, 20 May 2020 00:01:21 -0400
> 
>   > A more concrete recommendation would be to examine each proposed
>   > package more closely and try to decide whether we'd like to have that
>   > in Emacs or "close by", or just offer it for download to those who are
>   > interested (a.k.a. "just distribute").  This should include some
>   > meaningful discussion with arguments on both directions, and hopefully
>   > some consensus about the decision.
> 
>   > We could also ask the package developer to state their POV on these
>   > matters, and their plans for further development and maintenance.
> 
> It seems good to me.  Do you suggest making a list of questions
> and sending that to each developer, or would you rather just talk
> with them without a preplanned list?

It might be good to have this in the README file on ELPA.  I'd like to
hear Stefan's opinion about this.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> Would you like to suggest which parts of those libraries are nice in
> your opinon so that they could maybe be added to Emacs, following
> normal Emacs conventions?

I can see 2 main features:
1- A uniform naming that groups things nicely.
2- A short prefix to keep the names concise.

The longish recent discussion about namespace prefixes for core
functions shows solving (1) is *really hard* (tho see the
`prefixed-core` package I just started in elpa.git).

And this discussion shows that (2) is considered as a good enough reason
to try and prevent it not just from being incorporated into Emacs but
even prevent it from being added just GNU ELPA.

IOW, I'm not holding my breath.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > This is just a matter of following the good practises that already
  > > exist in Emacs.  It would be a bad idea to start making a mess, and
  > > then encouraging this mess to become larger.

  > Posited on s.el being a mess, which neither it, nor dash.el is. They are
  > both nice APIs that are nice to use.

I agree with you that s.el, in and of itself, is not a mess.
I can't speak for AMS, but maybe he would agree.

However, adding s.el to Emacs, or to GNU ELPA, would make a mess.  The
mess would be the juxtaposition of Emacs's own string functions, with
the s.el string functions -- two different interfaces for the same
job, similar in some places and disparate in others, in an
unpredictable way.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

     > > This is just a matter of following the good practises that already
     > > exist in Emacs.  It would be a bad idea to start making a mess, and
     > > then encouraging this mess to become larger.

     > Posited on s.el being a mess, which neither it, nor dash.el is. They are
     > both nice APIs that are nice to use.

   I agree with you that s.el, in and of itself, is not a mess.
   I can't speak for AMS, but maybe he would agree.

Indeed I do, and this sums it up well.

   However, adding s.el to Emacs, or to GNU ELPA, would make a mess.  The
   mess would be the juxtaposition of Emacs's own string functions, with
   the s.el string functions -- two different interfaces for the same
   job, similar in some places and disparate in others, in an
   unpredictable way.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Yuri Khan]

On Mon, 11 May 2020 at 09:43, Richard Stallman <rms@gnu.org> wrote:

>   > So "clostring-prepend"
>
> Rather, 'clos-prepend'.

It would be a bad idea to use the ‘clos-’ prefix for anything other
than “Common Lisp Object System” (whether Emacs ever implements that
or not).

RE: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Drew Adams]

> >   > So "clostring-prepend"
> > Rather, 'clos-prepend'.
> 
> It would be a bad idea to use the ‘clos-’ prefix for anything other
> than “Common Lisp Object System” (whether Emacs ever implements that
> or not).

+1.

If you want to suggest Clojure, then `cloj-' or `clj-'
is better than `clos-'.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If you want to suggest Clojure, then `cloj-' or `clj-'
  > is better than `clos-'.

The "s" is for "string" or "s.el".
"Clo" is for Clojure.
I don't see a shorter prefix to stand for "Clojure".
There are other ok choices which are longer.
For instance, "clo-s-".

If it is really more Ruby than Clojure, it could be 'rubs-'.
Or 'rubys-' or 'rus-'.  Or 'rub-s-'.

If the 'shorthand.el' approach works, maybe we won't need
this prefix to be short.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The s.el API is actually quite a bit richer than that of
  > clojure. Indeed, I found at least one non-standard clojure library that
  > looks like s.el. So I think the logic is flowing the other way.

I stand partially corrected, but I don't think this alters the
situation much.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> Are you saying that popularity and similarity of a package is the only
> criterion we should apply when deciding whether to add a package to
> ELPA?

Eli, I think you need to go take a look at the packages we have in
GNU ELPA.  Some are really good packages which we could or even should
distribute with Emacs.  Others are technically far from that bar.
Some are very popular.  Some are not popular at all.

My criterion for inclusion so far has mostly been:
- the maintainer is willing to include it into GNU ELPA.
- the technical obstacles are cleared (format of the package compatible
  with GNU ELPA, copyright paperwork, ...).
- it's sufficiently harmless in the sense that just having it
  *installed* won't unduly affect the user.

I see no benefit to adding any other obstacle, really.

Actual quality control is imposed for inclusion into Emacs,
not GNU ELPA.

> IOW, are you saying that the technical details of the package's
> implementation should not matter, for fear of sending the wrong
> message?

Pretty much, yes.  Not just "for fear", but because we do want to
encourage people to share their code (which can always be improved if
necessary).


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Philippe Vaucher <philippe.vaucher@gmail.com>,
>   joostkremers@fastmail.fm,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 10:11:00 -0400
> 
> > I as a Emacs user would think s.el is a bad addition to ELPA -- it
> > doesn't add anything special,
> 
> It is used by hundreds of packages.  So as long as it's not in GNU ELPA,
> none of those hundreds of packages can be even considered for inclusion
> into GNU ELPA.
> 
> That's the special that it adds.

So we are going to add s.el because it is used by hundreds of
packages.  And then we will add some of those hundreds, and then we
will add some more packages that use those other packages.  And so on
and so forth.

But to what end, may I ask?  Those packages already exist and are
available from MELPA, people are already using them, and installing a
package from MELPA is no more effort than from ELPA.  So what is the
purpose of adding them to ELPA as well, if all we are going to do is
mimic the same procedures and requirements as MELPA?

GNU ELPA is (or was?) supposed to be an extension of Emacs.  Being an
extension means the packages need to undergo almost the same quality
control as the code in core.  Deferring such quality control to some
imaginary future means simply that we decide not to do that: how can
we seriously expect that the package authors will agree to changes to
which they don't agree today, especially after so much time has passed
and so many other ELPA packages already depend on them?  Those
requirements are not arbitrary, they are supposed to keep Emacs easier
to use, develop, and maintain.  By waiving those requirements today we
in fact waive our ability to decide later whether or not to include a
package in Emacs.  By that we actually remove the main, if not the
only, reason to have ELPA at all.

> > IOW, are you saying that the technical details of the package's
> > implementation should not matter, for fear of sending the wrong
> > message?
> 
> Pretty much, yes.  Not just "for fear", but because we do want to
> encourage people to share their code (which can always be improved if
> necessary).

They already share their code, on MELPA, on GitHub, on GitLab, etc.
Why do we need to invest our efforts into one more repository "like
that"?  It makes no sense at all.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[조성빈]

> 2020. 5. 12. 오전 1:26, Eli Zaretskii <eliz@gnu.org> 작성:
> 
> 
>> 
>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: Philippe Vaucher <philippe.vaucher@gmail.com>,
>>  joostkremers@fastmail.fm,  rms@gnu.org,  emacs-devel@gnu.org
>> Date: Sat, 09 May 2020 10:11:00 -0400
>> 
>>> I as a Emacs user would think s.el is a bad addition to ELPA -- it
>>> doesn't add anything special,
>> 
>> It is used by hundreds of packages.  So as long as it's not in GNU ELPA,
>> none of those hundreds of packages can be even considered for inclusion
>> into GNU ELPA.
>> 
>> That's the special that it adds.
> 
> So we are going to add s.el because it is used by hundreds of
> packages.  And then we will add some of those hundreds, and then we
> will add some more packages that use those other packages.  And so on
> and so forth.
> 
> But to what end, may I ask?  Those packages already exist and are
> available from MELPA, people are already using them, and installing a
> package from MELPA is no more effort than from ELPA.  So what is the
> purpose of adding them to ELPA as well, if all we are going to do is
> mimic the same procedures and requirements as MELPA?
> 
> GNU ELPA is (or was?) supposed to be an extension of Emacs.  Being an
> extension means the packages need to undergo almost the same quality
> control as the code in core.  Deferring such quality control to some
> imaginary future means simply that we decide not to do that: how can
> we seriously expect that the package authors will agree to changes to
> which they don't agree today, especially after so much time has passed
> and so many other ELPA packages already depend on them?  Those
> requirements are not arbitrary, they are supposed to keep Emacs easier
> to use, develop, and maintain.  By waiving those requirements today we
> in fact waive our ability to decide later whether or not to include a
> package in Emacs.  By that we actually remove the main, if not the
> only, reason to have ELPA at all.

Looks like everybody has a different idea of ELPA is. I personally viewed it as a blessed package repo, nothing that different from MELPA except that it’s the default, official one from Emacs. In that case, I want these packages in ELPA because I want my friend to try out rjsx-mode (which AFAIK is only in MELPA) by M-x package-install rjsx-mode without explaining what the init file is, why the state isn’t persistent, what package-archives is, how lisp works, etc...

So the point for me is that it’s the default. (And as an additional point is that an official GNU manual for a ‘How to make Emacs an IDE’ will probably never recommend MELPA nor any MELPA packages.)

However, I can fully understand your preferences. In that case, would adding MELPA or at least recommending MELPA to add them be possible in Emacs? 

>>> IOW, are you saying that the technical details of the package's
>>> implementation should not matter, for fear of sending the wrong
>>> message?
>> 
>> Pretty much, yes.  Not just "for fear", but because we do want to
>> encourage people to share their code (which can always be improved if
>> necessary).
> 
> They already share their code, on MELPA, on GitHub, on GitLab, etc.
> Why do we need to invest our efforts into one more repository "like
> that"?  It makes no sense at all.
> 

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: 조성빈 <pcr910303@icloud.com>
> Date: Tue, 12 May 2020 02:19:56 +0900
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, joostkremers@fastmail.fm,
>  rms@gnu.org, Emacs-devel@gnu.org
> 
> would adding MELPA or at least recommending MELPA to add them be possible in Emacs? 

Why are we required to recommend MELPA?  MELPA is known to everyone,
it advertises itself just fine without our help, and how to configure
Emacs to install packages from there is hardly a secret.  It is also
very simple.  The Emacs user manual includes a description of how to
add archives beyond ELPA to sites where package.el looks for
packages.  Everything is described and visible in plain sight.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> Richard also proposed a compromise which AFAIU would allow it to be
> added.  For some reason, that proposal got no responses at all.

IIUC you're referring to:

    However, a small change would avoid these problems.  We could rename
    the file to clostring.el and rename the functions to start with clo
    also: clostring-prepend, clostring-format, etc.

I can't begin to understand how this can be considered a solution.
There's nothing in s.el that is needed.  If people want to use
`s-prepend` instead of `concat` or `s-join` instead of `mapconcat`, why
would anyone presume that they'd be willing to use `clostring-join` and
`clostring-prepend` instead?

We can't force people to write the code we like by withholding `s.el`
from GNU ELPA.  We could hope to get some traction by providing better
names in core (tho it will take many years before that will reduce the
need for `s.el` to be in GNU ELPA), but the long thread about namespace
prefixes has shown that we're unable to do that either anyway.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   >   > But you just described what dash does. ;-) It is just a collection
   >   > of list-handling functions such as they exist in modern functional
   >   > programming languages. If you're used to thinking in this paradigm
   >   > and then come (back) to Emacs Lisp, it feels like a hopelessly
   >   > clunky language. `dash.el` was written to remedy this.
   >
   > That sounds like they could be useful facilities.  Since they are real
   > features, not mere aliases and trivialities, they would not have the
   > main flaw of s.el.

   Richard, I think this is a good opportunity for you to actually go and
   really see what dash.el is and also what s.el is, maybe even code a
   little with them. Your comments really make it look like you vaguely
   understand what they are about. I'm sorry if that's not the case.

As someone who has gone through both, I still do not know what to use
dash.el for, I still don't see what s.el adds other than some more
Scheme like alises for the string functions in Emacs.

Insisting on everyone to just sit down and understand 100K of code
(that is the size of dash.el), and then on top of it trying to find
something to use it for won't move the discussion forward.

   Except if I missed it, here are the questions I didn't see an answer to:

   - As far as I know you are the only one who objected strongly against
   s.el in ELPA (others please voice your opinion if you think like
   Richard). 

I as a Emacs user would think s.el is a bad addition to ELPA -- it
doesn't add anything special, and makes code harder to read if people
use such constructs in the form that they are now, encouraging people
to use s-titleize instead of capitalize is going backwards, not
forwards. 

I haven't seen anyone objecting from adding some of the more
complicated functions and making them follow a more Emacs-y form, or
even on a case-by-case basis renaming some string functions to make
more sense. 

So why this forceful insistance of adding a s.el? Why not try to make
Emacs as a whole better by suggesting the better parts of s.el that
can be added to Emacs, or finding functions that could be renamed?

   - For most users, dash.el and s.el are very similar in nature. dash.el
   is already in ELPA. If we refuse s.el, isn't it inconsistent? What
   about the message we send?

If these two libraries had only added, and not tried to redefine basic
Emacs Lisp, I think the discussion would have been quite different.
Encouraging people to use `s-titalize' instead of `capitalize' is a
net loss for all Emacs Lisp programmers.

There is I think a big difference between s.el -- which adds mainly
trivial wrappers, and dash.el adds complicated control flow and
functions for working on trees, etc.  That is not to say that dash.el
too doesn't fall into the trap of adding many trival wrappers.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

>    Richard, I think this is a good opportunity for you to actually go and
>    really see what dash.el is and also what s.el is, maybe even code a
>    little with them. Your comments really make it look like you vaguely
>    understand what they are about. I'm sorry if that's not the case.
>
> As someone who has gone through both, I still do not know what to use
> dash.el for, I still don't see what s.el adds other than some more
> Scheme like alises for the string functions in Emacs.

I don't understand... is your point that dash.el/s.el would not be
useful to you? Yeah sure, not all packages are useful for everyone.

Or is it that you still don't know what dash function "maps" to what
elisp function?


> Insisting on everyone to just sit down and understand 100K of code
> (that is the size of dash.el), and then on top of it trying to find
> something to use it for won't move the discussion forward.

I'm not talking about reading the code, just the API. The API is
straightforward for a lot of users. Maybe that's the point you are
trying to make, that it is not straight forward to you.

I think it boils down to this:
https://en.wikipedia.org/wiki/Higher-order_programming

Some of us are exposed to a great deal of languages, and in most of
these we can do high order programming

Quoting the wikipedia article: Prominent examples of languages
supporting this are Wolfram Language, C#, Java, ECMAScript
(ActionScript, JavaScript, JScript), F#, Haskell, Lisp (Common Lisp,
Scheme, Clojure, others), Lua, Oz, Perl, PHP, Prolog,[1] Python, Ruby,
Smalltalk, Scala, ML, and Erlang.

I'd argue that even C++ supports it
http://www.cplusplus.com/reference/algorithm/

Thus I think we found why both camps find their perspective "obvious"
and has trouble understanding the other side.


>    Except if I missed it, here are the questions I didn't see an answer to:
>
>    - As far as I know you are the only one who objected strongly against
>    s.el in ELPA (others please voice your opinion if you think like
>    Richard).
>
> I as a Emacs user would think s.el is a bad addition to ELPA -- it
> doesn't add anything special, and makes code harder to read if people
> use such constructs in the form that they are now, encouraging people
> to use s-titleize instead of capitalize is going backwards, not
> forwards.

Well it'd be `s-capitalize` instead of `capitalize` but I think I
understand your point, you think in terms of what the name is now how
changing it is a burden. I think we disagree about what constitutes
going forward but that's ok.


> I haven't seen anyone objecting from adding some of the more
> complicated functions and making them follow a more Emacs-y form, or
> even on a case-by-case basis renaming some string functions to make
> more sense.

Yes, I'm glad we have at least that. There will be proposals in that
direction in the future.


> So why this forceful insistance of adding a s.el? Why not try to make
> Emacs as a whole better by suggesting the better parts of s.el that
> can be added to Emacs, or finding functions that could be renamed?

Well there isn't any forceful insistance of adding s.el to Emacs core
as far as I know. There is an incomprehension about why not add it to
ELPA, but it's more for ELPA's sake than mine. I use MELPA so I don't
really care.


>    - For most users, dash.el and s.el are very similar in nature. dash.el
>    is already in ELPA. If we refuse s.el, isn't it inconsistent? What
>    about the message we send?
>
> If these two libraries had only added, and not tried to redefine basic
> Emacs Lisp, I think the discussion would have been quite different.
> Encouraging people to use `s-titalize' instead of `capitalize' is a
> net loss for all Emacs Lisp programmers.

Again not the best example but ok. Please remember that what you see
as a net loss is a net gain for most people used to high order
programming discoverability.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   >    Richard, I think this is a good opportunity for you to actually go and
   >    really see what dash.el is and also what s.el is, maybe even code a
   >    little with them. Your comments really make it look like you vaguely
   >    understand what they are about. I'm sorry if that's not the case.
   >
   > As someone who has gone through both, I still do not know what to use
   > dash.el for, I still don't see what s.el adds other than some more
   > Scheme like alises for the string functions in Emacs.

   I don't understand... is your point that dash.el/s.el would not be
   useful to you? Yeah sure, not all packages are useful for everyone.

I cannot address both s.el and dash.el at the same time, they are two
extremely different libraries that have nothing incommon.  Lets please
stop mixing them up as if they are the same.

s.el has little point, it is thin wrappers around Emacs Lisp
functions, and it has been explained why it isn't suitable (as is).

dash.el is more interesting, but since that is not how you generally
write Lisp (in general), it is hard for me to find something I'd use
it for.  And so far, nobody is willing to provide concrete examples on
that topic and instead telling everyone to just go and read the code,
I did -- still no clue what or how to use these functions in the
context of Emacs Lisp.

Can you give some examples of using dash.el?

   > Insisting on everyone to just sit down and understand 100K of code
   > (that is the size of dash.el), and then on top of it trying to find
   > something to use it for won't move the discussion forward.

   I'm not talking about reading the code, just the API. 

That is a very naive view of the matter.

Lisp is one of the oldest high level programming languages that allow
for higher order functions, s.el has nothing to do with that.  

dash.el adds interesting constructs that are available in other
langugaes, and some might be interesting to add in Emacs Lisp.

   >    Except if I missed it, here are the questions I didn't see an answer to:
   >
   >    - As far as I know you are the only one who objected strongly against
   >    s.el in ELPA (others please voice your opinion if you think like
   >    Richard).
   >
   > I as a Emacs user would think s.el is a bad addition to ELPA -- it
   > doesn't add anything special, and makes code harder to read if people
   > use such constructs in the form that they are now, encouraging people
   > to use s-titleize instead of capitalize is going backwards, not
   > forwards.

   Well it'd be `s-capitalize` instead of `capitalize` but I think I
   understand your point, you think in terms of what the name is now how
   changing it is a burden. I think we disagree about what constitutes
   going forward but that's ok.

Adding s.el to ELPA isn't about changing any names, they are two
different topics.  Renaming existing functions in Emacs is, and should
be, a long process that shouldn't be done lightly.

s-capitalize is something different, and unrelated to the Emacs Lisp
function `capitalize'.  s-capitalize does a downcase on all words
except the first one, that is not the behaviour of `capitalize'.  The
corresponding function from s.el is s-titleize, which is a simple
alias for `capitalize'.

I picked this particular function because it is confusing to existing
Emacs Lisp users, since its behaviour is entierly different.

   > So why this forceful insistance of adding a s.el? Why not try to make
   > Emacs as a whole better by suggesting the better parts of s.el that
   > can be added to Emacs, or finding functions that could be renamed?

   Well there isn't any forceful insistance of adding s.el to Emacs core
   as far as I know. There is an incomprehension about why not add it to
   ELPA, but it's more for ELPA's sake than mine. 

That is not true, it is exactly the opposite.  See,

  https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg00850.html

Adding something to Emacs (be it Emacs or ELPA) has a bar, we
shouldn't blindly add anything and everything.

   >    - For most users, dash.el and s.el are very similar in nature. dash.el
   >    is already in ELPA. If we refuse s.el, isn't it inconsistent? What
   >    about the message we send?
   >
   > If these two libraries had only added, and not tried to redefine basic
   > Emacs Lisp, I think the discussion would have been quite different.
   > Encouraging people to use `s-titalize' instead of `capitalize' is a
   > net loss for all Emacs Lisp programmers.

   Again not the best example but ok. Please remember that what you see
   as a net loss is a net gain for most people used to high order
   programming discoverability.

This again mixes a different matters, one of higher order functions
and one of discoverability.  Neither s.el or dash.el make Emacs lisp
more discoverable.

Adding a well design higher order function library is I'm sure
something that would be welcome in Emacs.  

dash.el has some interesting functions for that, but could use better
function names for them, and removing the trivial aliases would be a
good start to maybe add them too Emacs (either via ELPA, or in Emacs
proper).

The objection here isn't what these libraries provide, but _how_ they
provide it.  As I mentioned before -- if these libraries would
concentrate on extending, not replacing, Emacs Lisp it would be a
different topic.    

Even the simple suggestion by RMS to call the library for clojure-lib
or similar for functions that follow the semantics from Clojure would
be vastly better.  That makes it very clear that these are not
functions that follow the Emacs Lisp style.  

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> s.el has little point, it is thin wrappers around Emacs Lisp
> functions, and it has been explained why it isn't suitable (as is).

To me it has the point of discoverability. I understand it's not your
position. I think both positions are reasonable.


> dash.el is more interesting, but since that is not how you generally
> write Lisp (in general), it is hard for me to find something I'd use
> it for.  And so far, nobody is willing to provide concrete examples on
> that topic and instead telling everyone to just go and read the code,
> I did -- still no clue what or how to use these functions in the
> context of Emacs Lisp.
>
> Can you give some examples of using dash.el?

Sure! I takes a bit of time to construct these so they'll come in a
separate email.


> That is a very naive view of the matter.
>
> Lisp is one of the oldest high level programming languages that allow
> for higher order functions, s.el has nothing to do with that.
>
> dash.el adds interesting constructs that are available in other
> langugaes, and some might be interesting to add in Emacs Lisp.

FTR, I'm not saying high order programming is possible thanks to
dash.el, I'm saying dash.el follows the "conventions" ones expect a
high order programming language to use. It's probably not so much
about the exact names but more about the grouping (namespaces) which
makes it easy to "guess" or "look for" a similar name that you already
know.


>    Well it'd be `s-capitalize` instead of `capitalize` but I think I
>    understand your point, you think in terms of what the name is now how
>    changing it is a burden. I think we disagree about what constitutes
>    going forward but that's ok.
>
> Adding s.el to ELPA isn't about changing any names, they are two
> different topics.  Renaming existing functions in Emacs is, and should
> be, a long process that shouldn't be done lightly.
>
> s-capitalize is something different, and unrelated to the Emacs Lisp
> function `capitalize'.  s-capitalize does a downcase on all words
> except the first one, that is not the behaviour of `capitalize'.  The
> corresponding function from s.el is s-titleize, which is a simple
> alias for `capitalize'.

Got me there! See, this is a good example: in clojure, ruby, python
and I'm sure other languages too "capitalize" means "downcase the
whole string but upcase the first character".
Emacs's `capitalize` is usually absent from other languages, but when
it exists it's named titleize (Ruby).

I can see how wanting to rename capitalize to titleize makes people
cry from the Emacs lisp perspective and how it feels silly from the
"rest of the languages" perspective (that you have "titleize" but not
"capitalize").


> This again mixes a different matters, one of higher order functions
> and one of discoverability.  Neither s.el or dash.el make Emacs lisp
> more discoverable.

Hum. Let's try to unify our vision by saying: it does not make the
current Emacs core lisp functions a lot more discoverable, but for a
significant number of people it'd allow them to quickly "understand"
how to do things and code packages and thus increase the overall
productivity ?

Also personnaly when I want to know how to do something in plain Emacs
lisp I go & look at the source of dash functions, so in a sense it
improves "plain Emacs lisp" discoverability for me.


> dash.el has some interesting functions for that, but could use better
> function names for them, and removing the trivial aliases would be a
> good start to maybe add them too Emacs (either via ELPA, or in Emacs
> proper).
>
> The objection here isn't what these libraries provide, but _how_ they
> provide it.  As I mentioned before -- if these libraries would
> concentrate on extending, not replacing, Emacs Lisp it would be a
> different topic.
>
> Even the simple suggestion by RMS to call the library for clojure-lib
> or similar for functions that follow the semantics from Clojure would
> be vastly better.  That makes it very clear that these are not
> functions that follow the Emacs Lisp style.

You made yourself very clear, thanks for the summary.

Philippe

discoveribility [Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Alfred M. Szmidt]

Since this is taking a turn of topic, into the realm of
discoverablity, I changed the subject.

   > s.el has little point, it is thin wrappers around Emacs Lisp
   > functions, and it has been explained why it isn't suitable (as is).

   To me it has the point of discoverability. I understand it's not
   your position. I think both positions are reasonable.

How does it make it more discoverable?  

E.g., how is s-ends-with? more discoverable than string-suffix-p? When
Emacs Lisp custom is to use -p for predicates, and to use the words
suffix/prefix.  To take a trivial example.

I understand that from the point of view of a different language
s-ends-with? might make more sense since you're assuming that it will
do the same as in Ruby since they have the same name.  And you will
also assume that since one such function exists, there will be more
similar functions that remind you of your previous experience in Ruby.

But we should view it from the lense of an Emacs Lisp hacker, where
discovering that one function for predicates is named with -p and some
with ? will be a big mess.  Where sometimes one uses ends/starts
instead of suffix/prefix.

   > dash.el is more interesting, but since that is not how you generally
   > write Lisp (in general), it is hard for me to find something I'd use
   > it for.  And so far, nobody is willing to provide concrete examples on
   > that topic and instead telling everyone to just go and read the code,
   > I did -- still no clue what or how to use these functions in the
   > context of Emacs Lisp.
   >
   > Can you give some examples of using dash.el?

   Sure! I takes a bit of time to construct these so they'll come in a
   separate email.

Thank you, I'm looking forward to that!

   Emacs's `capitalize` is usually absent from other languages, but when
   it exists it's named titleize (Ruby).

If you come from Ruby and assume that Emacs lisp is Ruby you will trip
-- but they are not the same languages, one cannot expect them to
behave the same.

   I can see how wanting to rename capitalize to titleize makes people
   cry from the Emacs lisp perspective and how it feels silly from the
   "rest of the languages" perspective (that you have "titleize" but not
   "capitalize").

I think it makes people cry from an English language perspective
too... One is an actual word, the other isn't!  

It also is misleading, since it will lower case all words follow the
first one, where in an actual title one would expect things like
subjects to be capitalize.  E.g, the chapter title "Strings and
Characters" -- we do not want it to be "Strings and characters"!

   Also personnaly when I want to know how to do something in plain Emacs
   lisp I go & look at the source of dash functions, so in a sense it
   improves "plain Emacs lisp" discoverability for me.

Why would you go to dash.el to understand what Emacs does?  Isn't it
just C-h f away to understand what any Emacs function is doing?  How
is the Emacs manual, or Emacs Lisp manuals not serving this purpose
for you?  Or any of the other features that Emacs provides to look up
things.

Re: discoveribility [Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Yuri Khan]

On Sun, 10 May 2020 at 00:36, Alfred M. Szmidt <ams@gnu.org> wrote:

>    I can see how wanting to rename capitalize to titleize makes people
>    cry from the Emacs lisp perspective and how it feels silly from the
>    "rest of the languages" perspective (that you have "titleize" but not
>    "capitalize").
>
> I think it makes people cry from an English language perspective
> too... One is an actual word, the other isn't!
>
> It also is misleading, since it will lower case all words follow the
> first one, where in an actual title one would expect things like
> subjects to be capitalize.  E.g, the chapter title "Strings and
> Characters" -- we do not want it to be "Strings and characters"!

On the other hand, capitalizing each word in a title is a peculiarity
of English and a few other languages, and even there the naïve
approach of capitalizing every word does not yield the grammatically
correct result.

Re: discoveribility [Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Philippe Vaucher]

>    > s.el has little point, it is thin wrappers around Emacs Lisp
>    > functions, and it has been explained why it isn't suitable (as is).
>
>    To me it has the point of discoverability. I understand it's not
>    your position. I think both positions are reasonable.
>
> How does it make it more discoverable?

It looks like you missed the previous discussion about namespaces.

I think it's simpler if you read the following posts and then continue
reading here:

Why I'd like namespace for discoverabilty (C-h f):
https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg00099.html
Proposed aliases:
https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg00379.html
Summary of counter-arguments:
https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg00508.html

So to still reply to your one-liner: because all the function using
the same prefix makes it easy for me to quickly get *the exact list*
of functions for a topic using "C-h f" or other tools. In the vast
majority of languages some sort of grouping is done, e.g
https://ruby-doc.org/core-2.6/String.html but also Python, Clojure,
Haskell, etc.

> E.g., how is s-ends-with? more discoverable than string-suffix-p? When
> Emacs Lisp custom is to use -p for predicates, and to use the words
> suffix/prefix.  To take a trivial example.
>
> I understand that from the point of view of a different language
> s-ends-with? might make more sense since you're assuming that it will
> do the same as in Ruby since they have the same name.  And you will
> also assume that since one such function exists, there will be more
> similar functions that remind you of your previous experience in Ruby.
>
> But we should view it from the lense of an Emacs Lisp hacker, where
> discovering that one function for predicates is named with -p and some
> with ? will be a big mess.  Where sometimes one uses ends/starts
> instead of suffix/prefix.

All true but "offtopic" ;-)

>    Emacs's `capitalize` is usually absent from other languages, but when
>    it exists it's named titleize (Ruby).
>
> If you come from Ruby and assume that Emacs lisp is Ruby you will trip
> -- but they are not the same languages, one cannot expect them to
> behave the same.

Well yes I kinda expected Emacs to give us high order programming in a
consistent and organized manner. If you look at the popularity of
dash/s.el/f.el we are many wanting that.

>    I can see how wanting to rename capitalize to titleize makes people
>    cry from the Emacs lisp perspective and how it feels silly from the
>    "rest of the languages" perspective (that you have "titleize" but not
>    "capitalize").
>
> I think it makes people cry from an English language perspective
> too... One is an actual word, the other isn't!
>
> It also is misleading, since it will lower case all words follow the
> first one, where in an actual title one would expect things like
> subjects to be capitalize.  E.g, the chapter title "Strings and
> Characters" -- we do not want it to be "Strings and characters"!

Well as I said already in every language "capitalize" means first
character uppercase all the rest lowercase:

https://clojuredocs.org/clojure.string/capitalize
https://apidock.com/ruby/String/capitalize
https://www.geeksforgeeks.org/string-capitalize-python/

Emacs lisp is the only language that names "capitalize" what is named
"titleize" in other languages (when it exists) and not have the
"capitalize" equivalent of other languages.

>    Also personnaly when I want to know how to do something in plain Emacs
>    lisp I go & look at the source of dash functions, so in a sense it
>    improves "plain Emacs lisp" discoverability for me.
>
> Why would you go to dash.el to understand what Emacs does?  Isn't it
> just C-h f away to understand what any Emacs function is doing?  How
> is the Emacs manual, or Emacs Lisp manuals not serving this purpose
> for you?  Or any of the other features that Emacs provides to look up
> things.

If you know what to look for yes, with dash I can just regexp search
for "^-.*\?" and find the *exact list* of all the predicates that
works on a list. Searching the equivalent "*-p" in emacs gives me many
wrong predicates not about list (note that all the dash `?` function
have aliases named `-p` to be consistent with emacs, so you can use
`-any-p` instead of `-any?`).

Re: discoveribility [Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Philippe Vaucher]

> Well as I said already in every language "capitalize" means first
> character uppercase all the rest lowercase:

s/every/almost every/

> Emacs lisp is the only language that names "capitalize" what is named
> "titleize" in other languages (when it exists) and not have the
> "capitalize" equivalent of other languages.

s/only language/only language I know of/

(sorry but if I don't correct myself people will :-)

Re: discoveribility [Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]]

[Alfred M. Szmidt]

   It looks like you missed the previous discussion about namespaces.

I didn't.

   >    Emacs's `capitalize` is usually absent from other languages, but when
   >    it exists it's named titleize (Ruby).
   >
   > If you come from Ruby and assume that Emacs lisp is Ruby you will trip
   > -- but they are not the same languages, one cannot expect them to
   > behave the same.

   Well yes I kinda expected Emacs to give us high order programming in a
   consistent and organized manner. 

And how does it not already to that?

   If you look at the popularity of dash/s.el/f.el we are many wanting
   that.

I don't know how to look at the popularity of s.el, dash.el which I
have never heard of before this discussion.  I've never heard of f.el
until you just mentioned it here, and I cannot even guess what it
does.  s.el has nothing to do with higher order functions, so why put
it into an already mixed bag of functions?

   > It also is misleading, since it will lower case all words follow the
   > first one, where in an actual title one would expect things like
   > subjects to be capitalize.  E.g, the chapter title "Strings and
   > Characters" -- we do not want it to be "Strings and characters"!

   Well as I said already in every language "capitalize" means first
   character uppercase all the rest lowercase:

You cannot assume that every language, be it spoken or programming,
will use the same word for the same meaning.  And three languages is
quite short of "every language"; these type of exaggerations are not
helpful...

   If you know what to look for yes, with dash I can just regexp
   search for "^-.*\?" and find the *exact list* of all the predicates
   that works on a list.

You are simply using the wrong tool for the job, nor will this regexp
work very well, since it will not list functions that are part of
Emacs Lisp.

If you wish to know what functions work on a list, one should look to
the Emacs Lisp manual, specifically the Lists chapter (even more
specifically, List-related Predicates).  Eli also had a very good run
down on how to use the different functions in Emacs to look up what is
what.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I can see how wanting to rename capitalize to titleize makes people
  > cry from the Emacs lisp perspective and how it feels silly from the
  > "rest of the languages" perspective (that you have "titleize" but not
  > "capitalize").

People who blame us for the fact that other, younger languages are not
compatible with Emacs Lisp are judging in a fundamentally unfair way.

The necessary response to that
is not to worry about it too much.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> Can you give some examples of using dash.el?

Let's start with a disclaimer: it's hard to come with examples on the
spot. Also because I usually code with dash or seq it's hard for me to
know the real "plain emacs lisp" equivalent.

(let ((lst '(1 2 3 4)))
  (pp (--map-when (= it 2) 17 lst))
  (pp (mapcar (lambda (it) (if (= it 2) 17 it)) lst)))

The point here is that the first one almost reads like english. The
intention is very clear (to me). The "pure lisp" one I have to put
more attention to it. I guess you'll disagree :-)

(let ((lst '("1" 2 "3" 4))
      (delme (make-symbol "delme")))
  (pp (--remove (and (numberp it) (= it 2)) lst))
  (pp (delete delme (mapcar (lambda (it) (if (and (numberp it) (= it
2)) delme it)) lst))))

Yes, I now I can use cl-remove-if or seq-remove, but I think it
further illustrates the point: I see both seq- and cl- as attempts at
fixing the Emacs api that was lacking. If I'm not mistaken seq.el was
even inspired by dash.

I was going to make an example with `-flatten` but I think this page
is better showing my point:
https://stackoverflow.com/questions/969067/name-of-this-function-in-built-in-emacs-lisp-library

In many languages, "flatten" is a basic concept. You'd expect
something as basic to be in the language... for me dash adresses these
"basics" you can't find easily in the manual. Same for other topics
(e.g set union/intersection/different), basically any topic related to
lists I can look on https://github.com/magnars/dash.el and quickly be
productive.

Now that I said all this, FWIW I'd agree that because there is seq-
and map- we don't really need dash- anymore in Emacs core for a lot of
use cases. Maybe the anaphoric/threading functions would be nice but I
doubt they'd interest you. Examples of threading:

(->> '(1 2 3) (-map 'square) (-remove 'even?)) ;; => '(1 9)
(--> "def" (concat "abc" it "ghi") (upcase it)) ;; => "ABCDEFGHI"

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   > Can you give some examples of using dash.el?

   Let's start with a disclaimer: it's hard to come with examples on the
   spot. Also because I usually code with dash or seq it's hard for me to
   know the real "plain emacs lisp" equivalent.

I understand, and I wasn't really expecting an example so soon, so
thank you for taking the time to do it.  I was hoping for examples of
the more complicated constructs, like the threading ones not the
anaphoric variants.

   (let ((lst '(1 2 3 4)))
     (pp (--map-when (= it 2) 17 lst))
     (pp (mapcar (lambda (it) (if (= it 2) 17 it)) lst)))

   The point here is that the first one almost reads like english. The
   intention is very clear (to me). The "pure lisp" one I have to put
   more attention to it. I guess you'll disagree :-)

I guess it is because you are more used to the abstraction level.  I
find the later to be much more natural to write.  It also lends it
self much easier (I think) if you wish to modify it at some later
point, e.g., you put the lambda in a function.  

I do not see what is won here won in this case, you write code only
once -- you read it much more.  If the lambda does something remotley
complicated, you would want to put it in a seperate function (either
defun, or flet).  

This leads to easier testing and experimenting/debugging too, since in
both cases (lambda or named function) you can use it as a normal form.
That isn't true for the first example.

   (let ((lst '("1" 2 "3" 4))
	 (delme (make-symbol "delme")))
     (pp (--remove (and (numberp it) (= it 2)) lst))
     (pp (delete delme (mapcar (lambda (it) (if (and (numberp it) (= it
   2)) delme it)) lst))))

   Yes, I now I can use cl-remove-if or seq-remove, but I think it
   further illustrates the point: I see both seq- and cl- as attempts at
   fixing the Emacs api that was lacking. If I'm not mistaken seq.el was
   even inspired by dash.

Why even do that! You could just use `delq`:

   (delq 2 '("1" 2 "3" 4))

The more I think about it, it seems that people aren't looking to
 learn Emacs Lisp -- but rather they want to learn
 Haskell/Clojure/...? 

   In many languages, "flatten" is a basic concept. You'd expect
   something as basic to be in the language... 

flatten-list?

   Now that I said all this, FWIW I'd agree that because there is seq-
   and map- we don't really need dash- anymore in Emacs core for a lot of
   use cases. Maybe the anaphoric/threading functions would be nice but I
   doubt they'd interest you. Examples of threading:

Do not assume that!  It is the threading ones that I was
_specifically_ interested in, since they are non-trivial in behaviour.

Anaphoric functions are boring, they are trivial and only muddle the
viewable code.  I personally dislike them immensly since they bind an
implicit variable that you cannot easily see as a reader of the code.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> I understand, and I wasn't really expecting an example so soon, so
> thank you for taking the time to do it.  I was hoping for examples of
> the more complicated constructs, like the threading ones not the
> anaphoric variants.

I think I need your help here. About the threading ones, can you look
at the few examples at
https://github.com/magnars/dash.el#--x-optional-form-rest-more and
tell me what you need more?

The threading macros are useful when you'll "cascade" arguments into a
bunch of function. See it as a better and more flexible `let*`, for
example:

(let* ((lst '(1 2 3))
         (lst-square (-map 'square lst))
         (lst-square-reverse (-reverse lst-square)))
  (pp lst-square-reverse))

>    (let ((lst '(1 2 3 4)))
>      (pp (--map-when (= it 2) 17 lst))
>      (pp (mapcar (lambda (it) (if (= it 2) 17 it)) lst)))
>
>    The point here is that the first one almost reads like english. The
>    intention is very clear (to me). The "pure lisp" one I have to put
>    more attention to it. I guess you'll disagree :-)
>
> I guess it is because you are more used to the abstraction level.  I
> find the later to be much more natural to write.  It also lends it
> self much easier (I think) if you wish to modify it at some later
> point, e.g., you put the lambda in a function.

Yes, it's hard ot make this example shine because it's too simple. It
was probably a poor example to begin with, still my point was that
"(map-when pred rep list)" speaks more to me than "use mapcar and pass
a lambda containing an if to it". Just by seeing the signature I know
what to do.

> I do not see what is won here won in this case, you write code only
> once -- you read it much more.  If the lambda does something remotley
> complicated, you would want to put it in a seperate function (either
> defun, or flet).

Well it's more of an example for the countless times you do simple
things in this lambda.

> This leads to easier testing and experimenting/debugging too, since in
> both cases (lambda or named function) you can use it as a normal form.
> That isn't true for the first example.

Agreed for complicated lambdas.

>    (let ((lst '("1" 2 "3" 4))
>          (delme (make-symbol "delme")))
>      (pp (--remove (and (numberp it) (= it 2)) lst))
>      (pp (delete delme (mapcar (lambda (it) (if (and (numberp it) (= it
>    2)) delme it)) lst))))
>
>    Yes, I now I can use cl-remove-if or seq-remove, but I think it
>    further illustrates the point: I see both seq- and cl- as attempts at
>    fixing the Emacs api that was lacking. If I'm not mistaken seq.el was
>    even inspired by dash.
>
> Why even do that! You could just use `delq`:
>
>    (delq 2 '("1" 2 "3" 4))
>

What if the list contains elements not comparable with `eq` or `=` ?
I'll show you the source of `seq-filter` as a reminder:

(cl-defgeneric seq-filter (pred sequence)
  "Return a list of all the elements for which (PRED element) is
non-nil in SEQUENCE."
  (let ((exclude (make-symbol "exclude")))
    (delq exclude (seq-map (lambda (elt)
                             (if (funcall pred elt)
                                 elt
                               exclude))
                           sequence))))

> The more I think about it, it seems that people aren't looking to
>  learn Emacs Lisp -- but rather they want to learn
>  Haskell/Clojure/...?
>
>    In many languages, "flatten" is a basic concept. You'd expect
>    something as basic to be in the language...
>
> flatten-list?

Nice. Why isn't this function findable using `C-h f` in Emacs 26.3? I
tested with emacs -Q, and I even tried to load `subr` and `subr-x`.

Anway, I hope you understand now that the point is not that Emacs
doesn't have such functions, is that there's no centralized place
where you have the exact short list of functions related to a topic.
Using prefixes like in dash or s.el allows for easy discoverability
using `C-h f`.

We'd be getting close to it by writing some mode that parses
https://www.gnu.org/software/emacs/manual/html_node/elisp/index.html
and grep for `-- function` and list these in a grouped manner like
dash/s.el does. I might get around to start doing this if time allows.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   > I understand, and I wasn't really expecting an example so soon, so
   > thank you for taking the time to do it.  I was hoping for examples of
   > the more complicated constructs, like the threading ones not the
   > anaphoric variants.

   I think I need your help here. About the threading ones, can you look
   at the few examples at
   https://github.com/magnars/dash.el#--x-optional-form-rest-more and
   tell me what you need more?

They are very trivial examples, they don't really show how people use
it in actual code which is I think where we could have an interesting
comparison.  I'd like to see something more "real world" -- something
that would be hard, or annoying to do in Emacs Lisp.

  (-> '(2 3 5)) ;; => '(2 3 5)

'(2 3 5)

  (-> '(2 3 5) (append '(8 13))) ;; => '(2 3 5 8 13)

(append '(2 3 5) '(8 13))

  (-> '(2 3 5) (append '(8 13)) (-slice 1 -1)) ;; => '(3 5 8)

(subseq (append '(2 3 5) '(8 13)) 1 -1)

   The threading macros are useful when you'll "cascade" arguments into a
   bunch of function. See it as a better and more flexible `let*`, for
   example:

   (let* ((lst '(1 2 3))
	    (lst-square (-map 'square lst))
	    (lst-square-reverse (-reverse lst-square)))
     (pp lst-square-reverse))

Why even use let*? (-reverse (-map 'square '(1 2 3)))

In these cases, I see absolutley no benefit of using ->, it just makes
it more "magical" for no particular reason.  

And you can individually extract a form (which is useful when
debugging), and it will have some meaning, take (-slice 1 -1).  The
function is described as (-slice LIST FROM &optional TO STEP) .. but 1
isn't a list!

   > I guess it is because you are more used to the abstraction level.  I
   > find the later to be much more natural to write.  It also lends it
   > self much easier (I think) if you wish to modify it at some later
   > point, e.g., you put the lambda in a function.

   Yes, it's hard ot make this example shine because it's too simple. It
   was probably a poor example to begin with, still my point was that
   "(map-when pred rep list)" speaks more to me than "use mapcar and pass
   a lambda containing an if to it". Just by seeing the signature I know
   what to do.

Polish speaks more to me than English, we all have our preferences and
have to adjust when we speak or write in a different language.  Which
is why it is important to see things through the intended lense -- in
this case, Emacs Lisp.  Your example of using -slice instead of subseq
is one such example -- I cannot assume that the word miasto will exist
in English and mean city.

   > Why even do that! You could just use `delq`:
   >
   >    (delq 2 '("1" 2 "3" 4))

   What if the list contains elements not comparable with `eq` or `=` ?

Everything is comparable using eq; but if you wish to strings content
or fixnums specifically then remove-if would be a better choice with
the correct predicate..

   > flatten-list?

   Nice. Why isn't this function findable using `C-h f` in Emacs 26.3? I
   tested with emacs -Q, and I even tried to load `subr` and `subr-x`.

It was added in 27.1.

   Anway, I hope you understand now that the point is not that Emacs
   doesn't have such functions, is that there's no centralized place
   where you have the exact short list of functions related to a topic.

That is the purpose of the Emacs Lisp manual.

   Using prefixes like in dash or s.el allows for easy discoverability
   using `C-h f`.

Prefixes can help in that, sometimes but not always, since sometimes
another prefix might make more sense.  But neither `-' nor `s' is a
good prefix for discoverability.

   We'd be getting close to it by writing some mode that parses
   https://www.gnu.org/software/emacs/manual/html_node/elisp/index.html
   and grep for `-- function` and list these in a grouped manner like
   dash/s.el does. I might get around to start doing this if time allows.

That would be great!

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Joost Kremers]

On Sun, May 10 2020, Alfred M. Szmidt wrote:
[threading macros]
> In these cases, I see absolutley no benefit of using ->, it just 
> makes
> it more "magical" for no particular reason.  

For sure, whether to use a threading macro is a large part 
personal preference, but IMHO they *can* make code more readable. 
Basically, you can convert:

   (print (eval (read)))

into:

    (-> (read)
      eval
      print)

The first form is backwards, in a way: `print` is written first, 
but it's what happens last. With `->` you can write the functions 
in the order in which they're executed.

For a longer discussion with more examples, see here: 
<https://yoo2080.wordpress.com/2014/02/23/threading-macros-from-dash-for-emacs-lisp/>

And I don't think it's been pointed out yet (though I may have 
missed some messages in this thread), that `->` and `->>` are part 
of Elisp since Emacs 25, except that they are called 
`thread-first` and `thread-last` (somewhat unfortunately, I 
believe, because they have nothing to do with CPU threads and are 
therefore unrelated to the other Elisp functions whose names start 
with `thread-`.)


-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

   And I don't think it's been pointed out yet (though I may have
   missed some messages in this thread), that `->` and `->>` are part
   of Elisp since Emacs 25, except that they are called `thread-first`
   and `thread-last` (somewhat unfortunately, I believe, because they
   have nothing to do with CPU threads and are therefore unrelated to
   the other Elisp functions whose names start with `thread-`.)

This was totally missed.  Thank you for that!

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Dmitry Gutov]

On 10.05.2020 03:02, Joost Kremers wrote:
> And I don't think it's been pointed out yet (though I may have missed 
> some messages in this thread), that `->` and `->>` are part of Elisp 
> since Emacs 25, except that they are called `thread-first` and 
> `thread-last` (somewhat unfortunately, I believe, because they have 
> nothing to do with CPU threads and are therefore unrelated to the other 
> Elisp functions whose names start with `thread-`.)

Prime candidates for a renaming, I'd say. And for documenting in the manual.

Clojure aficionados should appreciate it.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >    (print (eval (read)))

  > into:

  >     (-> (read)
  >       eval
  >       print)

The preference is a matter of taste.  If you have a taste for Lisp, the former
seems totally natural.  If you don't, maybe it seems strange.

->> is natural enough, because it adds the carried-over
argument at the end of the list.  But -> is unnatural,
since it adds the carried-over argument in the middle of the list
(after the first element).  I think that nested calls are
cleaner than ->.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Joost Kremers]

On Mon, May 11 2020, Richard Stallman wrote:
>   >    (print (eval (read)))
>
>   > into:
>
>   >     (-> (read)
>   >       eval
>   >       print)
>
> The preference is a matter of taste.  If you have a taste for 
> Lisp, the former
> seems totally natural.  If you don't, maybe it seems strange.

IMHO it's not a matter of having a taste for Lisp or finding 
`(print (eval (read)))` 'natural' or not. I find it perfectly 
natural, but there are cases where it becomes difficult to read, 
especially when you nest not two or three but five or six function 
calls and/or when each function call has additional arguments, 
which may be function calls themselves.

You can use `let*` to lay out such structures in a way that make 
them more readable. `->` and `->>` are even better than `let*` in 
this particular case, because you don't need the intermediate 
names.

> ->> is natural enough, because it adds the carried-over
> argument at the end of the list.  But -> is unnatural,
> since it adds the carried-over argument in the middle of the 
> list
> (after the first element).  I think that nested calls are
> cleaner than ->.

Well, let's agree to disagree on what 'natural' means when it 
comes to Lisp. :-) To me, Lisp is first and foremost a practical 
language and if there's more than one way to skin a cat, it's fine 
if there's more than one language construct to do so.

Unlike *some* languages where everything must be an object and you 
need to contort your mind to use an anonymous function, or where 
everything needs to be a pure function and doing basic file IO 
becomes a mountain to climb. (All IME, of course. :-/ )

-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 609 bytes --]

On Mon, May 11, 2020 at 11:08:25AM +0200, Joost Kremers wrote:

[...]

> You can use `let*` to lay out such structures in a way that make
> them more readable. `->` and `->>` are even better [...]

In *your* frame of reference, perhaps. Not in mine, by a far shot.
And I think /that/ is one of the shortcomings of this discussion.

As long as we don't accept all that there are different approaches
and give up on the "mine's better (or 'more modern' or 'more Lispy'
or whatnot) than yours" the discussion is bound to go in circles
(at best) or to hurt feelings (at worst).

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> In *your* frame of reference, perhaps. Not in mine, by a far shot.
> And I think /that/ is one of the shortcomings of this discussion.
>
> As long as we don't accept all that there are different approaches
> and give up on the "mine's better (or 'more modern' or 'more Lispy'
> or whatnot) than yours" the discussion is bound to go in circles
> (at best) or to hurt feelings (at worst).

For the record I think I've reach that point since a while now.

I'll happily continue to explain my way of thinking but I'm not
pushing anymore for mass aliases/renames, this is a dead end.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1143 bytes --]

On Mon, May 11, 2020 at 11:46:52AM +0200, Philippe Vaucher wrote:
> > In *your* frame of reference, perhaps. Not in mine, by a far shot.
> > And I think /that/ is one of the shortcomings of this discussion.
> >
> > As long as we don't accept all that there are different approaches
> > and give up on the "mine's better (or 'more modern' or 'more Lispy'
> > or whatnot) than yours" the discussion is bound to go in circles
> > (at best) or to hurt feelings (at worst).
> 
> For the record I think I've reach that point since a while now.

That's my feeling, too, and I'm very thankful to you for that.

I just follow this thread and try to point out those little
"linguistic antipatterns" I spot. Based on my past experience,
those can be very detrimental to otherwise strong communities
without any bad intentions.

BTW, I know this can happen to me, too: thus I'm happy whenever
someone tells me.

> I'll happily continue to explain my way of thinking but I'm not
> pushing anymore for mass aliases/renames, this is a dead end.

Yes, and thanks for this. That's definitely the more exciting
way :-)

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> I just follow this thread and try to point out those little
> "linguistic antipatterns" I spot. Based on my past experience,
> those can be very detrimental to otherwise strong communities
> without any bad intentions.

That's a very valuable service you are doing here, thanks a lot!

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[조성빈]

> 2020. 5. 11. 오후 6:48, Philippe Vaucher <philippe.vaucher@gmail.com> 작성:
> 
> 
>> 
>> In *your* frame of reference, perhaps. Not in mine, by a far shot.
>> And I think /that/ is one of the shortcomings of this discussion.
>> 
>> As long as we don't accept all that there are different approaches
>> and give up on the "mine's better (or 'more modern' or 'more Lispy'
>> or whatnot) than yours" the discussion is bound to go in circles
>> (at best) or to hurt feelings (at worst).
> 
> For the record I think I've reach that point since a while now.

Yeah, so true. 

> I'll happily continue to explain my way of thinking but I'm not
> pushing anymore for mass aliases/renames, this is a dead end.

Everybody in this mailing list that tried to push some more sane names got tired about why the current status quo isn’t good enogu, so... :-(

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Joost Kremers]

On Mon, May 11 2020, tomas@tuxteam.de wrote:
> On Mon, May 11, 2020 at 11:08:25AM +0200, Joost Kremers wrote:
>
> [...]
>
>> You can use `let*` to lay out such structures in a way that 
>> make
>> them more readable. `->` and `->>` are even better [...]
>
> In *your* frame of reference, perhaps.

Yes, obviously. Perhaps I could have made that clearer.

> As long as we don't accept all that there are different 
> approaches
> and give up on the "mine's better (or 'more modern' or 'more 
> Lispy'
> or whatnot) than yours" the discussion is bound to go in circles
> (at best) or to hurt feelings (at worst).

Well, as I said, I like Lisp for its practical nature and it's a 
good thing that it offers more than one way of doing things.

-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> > The preference is a matter of taste.  If you have a taste for
> > Lisp, the former
> > seems totally natural.  If you don't, maybe it seems strange.
>
> IMHO it's not a matter of having a taste for Lisp or finding
> `(print (eval (read)))` 'natural' or not. I find it perfectly
> natural, but there are cases where it becomes difficult to read,
> especially when you nest not two or three but five or six function
> calls and/or when each function call has additional arguments,
> which may be function calls themselves.
>
> You can use `let*` to lay out such structures in a way that make
> them more readable. `->` and `->>` are even better than `let*` in
> this particular case, because you don't need the intermediate
> names.

Thanks for explaining my example better than I did. I couldn't find a
way to make people understand it but I think you did.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Thanks for showing me examples of dash.el.  A lot of these facilities
seem useful.  I would very much like to see the _documentation_ for how
to use it.

First, these all seem to be macros, right?

Why do some names start with a single dash and some with a double dash?

About --map-when:

  > (let ((lst '(1 2 3 4)))
  >   (pp (--map-when (= it 2) 17 lst))
  >   (pp (mapcar (lambda (it) (if (= it 2) 17 it)) lst)))

The map-when version was not self-evident to me.
I had to read the mapcar version to understand its meaning.
Once I did that, it made sense, and I can see that it is useful.
I think we should add this to Emacs, but with a couple of changes.

* Call it 'map-when' -- why use dashes?

* Change the calling convention syntax to (map-when ((= it 2) 17) lst)
  simply so it easier to see what groups with what.
  That gives it the structure of the inside of a cond.

About --remove:

That is good as is, I would just call it 'remove-if'.

  > I was going to make an example with `-flatten` but I think this page
  > is better showing my point:
  > https://stackoverflow.com/questions/969067/name-of-this-function-in-built-in-emacs-lisp-library

I will fetch that page and look at it.

    (->> '(1 2 3) (-map 'square) (-remove 'even?)) ;; => '(1 9)

Is '-map' just another name for 'mapcar'?

After some effort I figured out what that example is doing, I think.
Indeed, it can be used.  At the same time, what advantage is that
supposed to provide over writing it the Lisp way?

   (remove-if 'evenp (mapcar 'square '(1 2 3)))

About -->:

    (--> "def" (concat "abc" it "ghi") (upcase it)) ;; => "ABCDEFGHI"

This sort of shorthand for binding a variable can make code simpler.
The name could be a good one, if there are only a few such constructs
and each one of them acts at the very general level of this one.

Do poeple see any practical problem that would result from the name -->?


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> Thanks for showing me examples of dash.el.  A lot of these facilities
> seem useful.  I would very much like to see the _documentation_ for how
> to use it.

Go to https://github.com/magnars/dash.el#functions for its
documentation. As soon as you load that page you'd see the
documentation, if not scroll a bit down until you read the text.

> First, these all seem to be macros, right?

Some of them are, some others not.

Usually the ones that have double dashes at the beginning are macros, yes.

> Why do some names start with a single dash and some with a double dash?

I think its simpler for me to copy paste the documentation:

All functions and constructs in the library are prefixed with a dash (-).

There are also anaphoric versions of functions where that makes sense,
prefixed with two dashes instead of one.

While -map takes a function to map over the list, you can also use the
anaphoric form with double dashes - which will then be executed with
it exposed as the list item. Here's an example:

(-map (lambda (n) (* n n)) '(1 2 3 4)) ;; normal version
(--map (* it it) '(1 2 3 4)) ;; anaphoric version

of course the original can also be written like

(defun square (n) (* n n))
(-map 'square '(1 2 3 4))

which demonstrates the usefulness of both versions.


> About --map-when:
>
>   > (let ((lst '(1 2 3 4)))
>   >   (pp (--map-when (= it 2) 17 lst))
>   >   (pp (mapcar (lambda (it) (if (= it 2) 17 it)) lst)))
>
> The map-when version was not self-evident to me.
> I had to read the mapcar version to understand its meaning.
> Once I did that, it made sense, and I can see that it is useful.
> I think we should add this to Emacs, but with a couple of changes.
>
> * Call it 'map-when' -- why use dashes?

Yes, if in Emacs it'll probably have another name. We could also put
in seq as seq-map-when.

>   > I was going to make an example with `-flatten` but I think this page
>   > is better showing my point:
>   > https://stackoverflow.com/questions/969067/name-of-this-function-in-built-in-emacs-lisp-library
>
> I will fetch that page and look at it.

The point was that until very recently (27.1) there was no flatten
function, which is something almost every language has.

>     (->> '(1 2 3) (-map 'square) (-remove 'even?)) ;; => '(1 9)
>
> Is '-map' just another name for 'mapcar'?

Yes

RE: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Drew Adams]

> The point was that until very recently (27.1)
> there was no flatten function, which is
> something almost every language has.
            ^^^^^^^^^^^^

Elisp survived for 35 years without a
predefined `flatten'?  And Common Lisp
still doesn't have it?  How come?

I'd say that `flatten' is something
"almost every" Lisp user learns to
write early on, at least as a simple
recursive function.  Good exercise.

I don't think it's a crying need that
`flatten' be predefined.  But OK, fine,
getting the leaves of a tree can be
useful.  Sometimes you need something
like that; mostly you don't.
___

Google/duck-duck: "flatten list lisp".

https://stackoverflow.com/questions/10465096/flatten-nests-function-in-lisp-need-help-understanding

http://lee-mac.com/flatten.html

https://coderwall.com/p/7uecya/flatten-a-multi-dimensional-list-in-lisp

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Philippe Vaucher]

> I'd say that `flatten' is something
> "almost every" Lisp user learns to
> write early on, at least as a simple
> recursive function.  Good exercise.

When you are learning programming, yeah sure. When you just want to
flatten a list when writing a package for Emacs (for which you get 0
rewards except people saying "thanks" from time to time and your own
satisfaction of making Emacs a better place, all of which is great
already but that's not the point), there I think I just want flatten
to exist already. It's about productivity. Actually you made me
realise most of my points are about productivity.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Alfred M. Szmidt]

How often do you flatten lists? In 30 years of programming, I've never
come across the situation where I needed to do so be it in Lisp, or
any other functional language.  flatten is a trivial, mostly useless
function, when it comes to Lisp in my experience.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

ams@gnu.org (Alfred M. Szmidt) writes:

> How often do you flatten lists? In 30 years of programming, I've never
> come across the situation where I needed to do so be it in Lisp, or
> any other functional language.  flatten is a trivial, mostly useless
> function, when it comes to Lisp in my experience.


Why ask, when you can find out?

There are about 350 usages of -flatten in MELPA, by grepping for
"(-flatten". Of these around 50 are in dash itself, the rest elsewhere.

So people do use -flatten but not that frequently; that is the way with
this kind of library. Most of the functions do not get used that often,
but it is often that many of them get used.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> there I think I just want flatten to exist already.

His point was: it's rare to need exactly `flatten` (which BTW is called
`flatten-tree` since Emacs-27 ;-).
AFAIK in most cases what you really want is something along the lines of
(apply #'append X), i.e. a "one-level flatten".

As its name implies, `flatten-tree` really operates on a tree, but it's
very rare to represent trees in Elisp in such a way that all non-cons
cells are elements of the tree, so `flatten-tree` rarely does something
meaningful on a tree.

But yes, there are several cases where you may happen to know that
`flatten-tree` will work because it won't have the opportunity to go
deeper than the first level.  I.e. you have a "list of lists of strings"
or a "list of either strings or lists of strings", so you won't see the
difference between `flatten-tree` and a hypothetical one-level flatten.


        Stefan

RE: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Drew Adams]

> As its name implies, `flatten-tree` really operates on a tree...

(I almost pointed out that `flatten' is the same as, and
could be called, a bit whimsically or perhaps haskelly,
`leaves', since that's what it returns.)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I don't have anything against a 'flatten-list' function.
If users want that enough, we should have it.

However, I don't see a problem for which it is a good solution -- only
problems for which it is a bad solution.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Thanks for the explanation.  Now I think I basically understand
the design of dash.el.

I wonder if there really need to be anaphoric versions and functional
versions of each construct.  I think it would be easy to define
a macro thaat can handle both cases.

FUN-NAME  for a function to call
(lambda ...) for an anonymous function to call
(apply EXP) to evaluate EXP to get the function to call
LIST-EXPRESSION for anaphora

WDYT?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> Thanks for the explanation.  Now I think I basically understand
> the design of dash.el.
>
> I wonder if there really need to be anaphoric versions and functional
> versions of each construct.  I think it would be easy to define
> a macro thaat can handle both cases.
>
> FUN-NAME  for a function to call
> (lambda ...) for an anonymous function to call
> (apply EXP) to evaluate EXP to get the function to call
> LIST-EXPRESSION for anaphora
>
> WDYT?


It would be a library rewrite because Magnar mostly defines the function
forms in terms of the anaphoric macros. The point of the anaphoric
macros is, I think, to stop you from having to write `(lambda() it)` to
replacing it with calls to `lambda` does not really make sense.

This probably is inspired by clojure which has a very short read macro
support functions ---

(map #(/ % 2) '(2 4 6 8))

It's quite neat (the % being the equivalent of dash.el's `it`).

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It would be a library rewrite because Magnar mostly defines the function
  > forms in terms of the anaphoric macros.

I did not suggest eliminating the anaphoric macros.  I think they are good.
Rather, I suggested extending them to support also the
function-based calling conventions.  Thus, the anaphoric macro would
support both forms of the construct.

  > (map #(/ % 2) '(2 4 6 8))

  > It's quite neat (the % being the equivalent of dash.el's `it`).

In Lisp we have always preferred to write things in the more general
form where you specify the arg names.  But a shorter construct
which avoid the need to specify the arg names might be ok.

What could that kind of abbreviated lambda-expression look like?


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Joost Kremers]

On Tue, May 12 2020, Richard Stallman wrote:
>   > (map #(/ % 2) '(2 4 6 8))
>
>   > It's quite neat (the % being the equivalent of dash.el's 
>   > `it`).
>
> In Lisp we have always preferred to write things in the more 
> general
> form where you specify the arg names.  But a shorter construct
> which avoid the need to specify the arg names might be ok.

In Clojure, they're called function literals and the Clojure docs 
say "[...] idiomatic use would be for very short one-off 
mapping/filter fns and the like." So basically just what you said.

> What could that kind of abbreviated lambda-expression look like?

In Clojure, a function literal is written with `#(...)`, and it 
supports `%n` for arguments, where n=1,2,3... (Actually, I don't 
know if Clojure supports `%10` and up or if it just goes up to 
`%9`, but I guess that doesn't matter much). A single `%` is 
synonymous with `%1`, which is used when there's only one 
argument. Also supported is `%&`, which is similar to `&rest 
<var>` in Elisp, being bound to a list containing the remaining 
arguments.

That's all there is to it (the official doc is ever shorter: 
fourth bullet point under 
<https://clojure.org/reference/reader#_dispatch>).

-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> In Clojure, a function literal is written with `#(...)`, and it supports
> `%n` for arguments, where n=1,2,3...

IIRC there's a macro somewhere providing a similar facility.
We can't use (# ...) because the reader doesn't like `#` on its own (it
expects it to be followed by something like a vector, a hex/binary/octal
number, etc...).


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> In Clojure, a function literal is written with `#(...)`, and it supports
>> `%n` for arguments, where n=1,2,3...
>
> IIRC there's a macro somewhere providing a similar facility.
> We can't use (# ...) because the reader doesn't like `#` on its own (it
> expects it to be followed by something like a vector, a hex/binary/octal
> number, etc...).

It's not hard to do in a cheesy way. Implemented with dash.el, entirely
from a sense of mischievousness.

(defmacro fn (&rest args)
  (let ((argsd
         (--tree-map
          (cl-case it
            ('% 'x)
            (t it))
          args)))
    `(lambda (x) ,@args)))

Obviously this is not fully functional, but it works for the trivial example.

(funcall (fn %) 10)
(funcall (fn (+ 10 (/ % 2))) 50)

I can't get it to work from first position though. So:

((lambda (x) x) 10)

works, while

((fn %) 10)

doesn't.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> I can't get it to work from first position though. So:

That's normal: first position does not allow a normal expressions.
"first position" accepts basically the same things as what follows #'

Elisp is not Scheme.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I can't get it to work from first position though. So:
>
> That's normal: first position does not allow a normal expressions.
> "first position" accepts basically the same things as what follows #'
>
> Elisp is not Scheme.

Yes, indeed, that is true. Actually I was more surprised that it worked
with lambda.

But, it *does* work with `lambda', so if we wanted a shortened version
of lambda with autonamed arguments, then, ideally, it should work in all
the places that lambda work?

Still, the `((lambda (x) x)` is pretty pointless in Emacs lisp, so
perhaps this bit is not important.

You said that there is a macro like this already in Emacs? If not, it
would be nice to add. Not sure `fn' is the greatest name though.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> You said that there is a macro like this already in Emacs?

I don't think it's in Emacs, but I remember seeing something like
that somewhere.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Jonas Bernoulli]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> You said that there is a macro like this already in Emacs?
>
> I don't think it's in Emacs, but I remember seeing something like
> that somewhere.

You were probably thinking of Oleh Krehel's short-lambda package.

https://github.com/abo-abo/short-lambda

;; This package allows to do this:
;;
;; (cl-mapcar #(concat %1 " are " %2)
;;            '("roses" "violets")
;;            '("red" "blue"))
;; ;; => '("roses are red" "violets are blue")
;;
;; Or this:
;;
;; (mapc #(put % 'disabled nil)
;;       '(upcase-region downcase-region narrow-to-region))
;;
;; This assumes that there is a reader macro in the Emacs C code that
;; translates #(STRUCTURE) to (short-lambda STRUCTURE), in the same
;; way as for the `backquote' macro.

If we ask nicely, he might still be able to dig up the C code.
Hey Oleh!

  Jonas

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Adrián Medraño Calvo]

> On 13. May 2020, at 23:03, Jonas Bernoulli <jonas@bernoul.li> wrote:
> 
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
>>> You said that there is a macro like this already in Emacs?
>> 
>> I don't think it's in Emacs, but I remember seeing something like
>> that somewhere.
> 
> You were probably thinking of Oleh Krehel's short-lambda package.
> 
> https://github.com/abo-abo/short-lambda

There’s also dollar.el, by Göktuğ Kayaalp (CC’d):

	https://github.com/cadadr/elisp/blob/devel/dollar.el

I find it very elegant: it’s just a macro.

;;; Commentary:
;; This package provides a macro named $ where in its body symbols in
;; the form $N where N is a positive integer are to stand for
;; positional arguments to the generated lambda.
;; If the car of the body is a vector though, that vector becomes the
;; argument list of the new lambda.

For example:

	($ (+ $1 $2))

expands to:

	(lambda
	  ($1 $2 &rest $_)
	  (let
	      (($*
	        (cons
	         (vector $1 $2)
	         $_)))
	    (+ $1 $2)))
	
		(funcall ($ + $1 $2) 4 6)
		10

Example usage:

	(seq-map ($ (* $1 $1)) '(1 2 3))
	(1 4 9)

Using named arguments:

	(seq-map ($ [n] (* n n)) '(1 2 3))
	(1 4 9)

At first glance, I’d do away with &rest $_ and collecting the arguments in the $* variable, for better performance.  The former could be left to the named arguments mechanism.

Göktuğ Kayaalp, would you be willing to include dollar.el in Emacs or GNU ELPA?

Thank you all and
best regards,
Adrián.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Göktuğ Kayaalp]

On 2020-05-14 10:26 +03, Adrián Medraño Calvo wrote:
>> On 13. May 2020, at 23:03, Jonas Bernoulli <jonas@bernoul.li> wrote:
>>
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>
>>>> You said that there is a macro like this already in Emacs?
>>>
>>> I don't think it's in Emacs, but I remember seeing something like
>>> that somewhere.
>>
>> You were probably thinking of Oleh Krehel's short-lambda package.
>>
>> https://github.com/abo-abo/short-lambda
>
> There’s also dollar.el, by Göktuğ Kayaalp (CC’d):
>
> 	https://github.com/cadadr/elisp/blob/devel/dollar.el

Hi, I’m the author of this package.  The parent poster has asked me
earlier today if I’d consider dollar.el being incorporated to Emacs. I’m
totally on board with it and can help to make it happen if you guys do
like it.

This whole thread is way too big for me to read in a timely manner, but
from what I get there’s an inclination towards the Clojure-like reader
macro syntax for this. Personally, I’m not that fond of it both
aesthetically and because it seems to be potentially a breaking change
in the parser. A macro that spits out a closure is better IMHO.

Below are some examples of usage from my init.el:

($ (with-current-buffer buf
     (hl-line-mode -1)))
---
;; If the explicit argument was omitted, this would throw because the
;; arglist of the generated closure would empty (no $... in body).
(let ((compilation-buffer-name-function ($ [_] "*Build Emacs Master*")))
  (compile "build-emacs-master.sh" t)))
---
(mapcar
 ($ (cons (buffer-name (window-buffer $1)) $1))
 (window-list))
---
;; A named argument
(add-to-list 'compilation-finish-functions
             ($ [_ status]
                (gk-send-desktop-notification "Compilation finished" status)))
---
;; Empty function (i.e. (lambda ())
(setf ring-bell-function ($))


The one trouble with dollar.el is that it currently doesn’t filter out
quoted expressions, so something like

($ ’($1))

expands to

(lambda
  ($1 &rest $_)
  (let
      (($*
	(cons
	 (vector $1)
	 $_)))
    '
    ($1)))

which is undesirable. I’ve been too lazy to fix it because it wouldn’t
really come up in practical use (WCGW...), but it shouldn’t be that
difficult to implement.

It also uses -flatten from dash.el, but adding a seq-flatten function
should be trivial.


> At first glance, I’d do away with &rest $_ and collecting the
> arguments in the $* variable, for better performance.  The former
> could be left to the named arguments mechanism.

It should be possible to get those arguments set up in the macroexpand
phase in order to minimise runtime impact.

We could’ve a simpler $ and a $$ which supports $* and $_ too.

> Göktuğ Kayaalp, would you be willing to include dollar.el in Emacs or
> GNU ELPA?

I’d love that to happen, and I’d prefer Emacs proper because IMHO this
is such an elementary and small feature that you wouldn’t want to need a
third party package for it.

If it’ll be ELPA nevertheless tho, could I use that version as the
upstream (i.e. push commits to the file on ELPA)? I don’t use package.el
so I’m not very up to date on how things are done on ELPA.


Cheers,
        -gk.

--
İ. Göktuğ Kayaalp / @cadadr / <https://www.gkayaalp.com/>
pgp:   024C 30DD 597D 142B 49AC 40EB 465C D949 B101 2427

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

The choice between the syntax ($ (+ $1 $2))
and the syntax #(+ %1 %2), I think it is a small difference.
If one is ok, the other is ok.

The # syntax has the small advantage of being a little shorter.

The $ syntax can be implemented now with a macro -- but if we want this
feature, I see no reason to hesitate to do implement it in the reader,
no need to use a macro.  It would not be a lot of work to do that.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> The choice between the syntax ($ (+ $1 $2))
> and the syntax #(+ %1 %2), I think it is a small difference.
> If one is ok, the other is ok.
>
> The # syntax has the small advantage of being a little shorter.
>
> The $ syntax can be implemented now with a macro -- but if we want this
> feature, I see no reason to hesitate to do implement it in the reader,
> no need to use a macro.  It would not be a lot of work to do that.


I would go for the macro syntax because is could be made backward
compatible with Emacs-27 (as seq.el was with Emacs-24), which the reader
macro cannot.

I also think that if this goes anywhere, Emacs core is the place to be.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I would go for the macro syntax because is could be made backward
  > compatible with Emacs-27 (as seq.el was with Emacs-24), which the reader
  > macro cannot.

I see your point, but I disagree with the conclusion because I don't
think we need to start _using_ this syntax in external packages right
away.  So I think we should choose the syntax which is better in the
long term rather than let backwards compatibility fix this forever.

However, we could implement both syntaxes, use the $ syntax in the
short term, and convert external packages to the # syntax in a few
years.

Let's see what Eli thinks about it.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> However, we could implement both syntaxes, use the $ syntax in the
> short term, and convert external packages to the # syntax in a few
> years.
>
> Let's see what Eli thinks about it.

FWIW, I'm not too fond of such shorthand syntax.  The benefit is not
very high and it makes the language that much more difficult to learn
for newcomers.

For a programming language like Closure, it might make sense, since
most/all people writing Closure programs are actual programmers that
have to be proficient in Clojure.  But Elisp lambdas are very common in
.emacs files, so this additional complexity will be exposed to some of
our users who aren't programmers or aren't proficient in Elisp.

[ Yes, I know it may sound strange coming from me, since I'm to blame
  for a lot of complexity in Elisp :-(
  But to my defense, `pcase` and `cl-defmethod` aren't nearly as often
  needed in .emacs as lambdas.  ]

I can already see the discussions in forums about "what's the difference
between #(f % 1) and (lambda (x) (f x 1))", "when should I use which",
...

If we want to shorten anonymous functions we could start with something
like (λ (x) (f x 1)), which you can get with
prettify-symbols-mode already.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Adrián Medraño Calvo]

> On 18. May 2020, at 06:42, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>> However, we could implement both syntaxes, use the $ syntax in the
>> short term, and convert external packages to the # syntax in a few
>> years.
>> 
>> Let's see what Eli thinks about it.
> 
> FWIW, I'm not too fond of such shorthand syntax.  The benefit is not
> very high and it makes the language that much more difficult to learn
> for newcomers.
> 
> For a programming language like Closure, it might make sense, since
> most/all people writing Closure programs are actual programmers that
> have to be proficient in Clojure.  But Elisp lambdas are very common in
> .emacs files, so this additional complexity will be exposed to some of
> our users who aren't programmers or aren't proficient in Elisp.

I understand your argument, particularly as it applies to the
reader syntax.  On the other hand, dollar.el is just a macro: it
does not complicate the syntax (for some definition of syntax);
although it does complicate understanding when first presented.
Being just a macro entails:

It’s highlighted in Emacs, just like lambda.
Its documentation can be queried on the spot with C-h f RET.
It can be macroexpanded, in order to inspect its effect.

In any case, I have no experience helping beginners, so I’m
not sure whether the above would help them or not.

> [ Yes, I know it may sound strange coming from me, since I'm to blame
>  for a lot of complexity in Elisp :-(
>  But to my defense, `pcase` and `cl-defmethod` aren't nearly as often
>  needed in .emacs as lambdas.  ]

(You’ve done much good, thank you.)

> I can already see the discussions in forums about "what's the difference
> between #(f % 1) and (lambda (x) (f x 1))", "when should I use which",
> ...
> 
> If we want to shorten anonymous functions we could start with something
> like (λ (x) (f x 1)), which you can get with
> prettify-symbols-mode already.

Something like this is supported by dollar.el, when using the named variables
syntax (substituting λ for $):

	($ [x] (f x 1))

though the usual way would be:

	($ (f $1 1))

Would prettifying lambda to λ lead to visually unexpected indentation?
(I think not, for the uses…)  Typing λ is a no-go, in my opinion.

I’m not entirely sold on the symbol $ for the macro name, but its use
for the variables is genius, in my opinion, which makes the combination
“click”.  Variable names starting with $ ($1, $2, etc.) are familiar for
people with experience with in Bash, Tcl or PHP.  For people familiar
with Haskell, $ is another form of function application, which would
remind of the $ macro.

Going with % and %1, %2, etc. would be fine, and would be intuitive
for people with Clojure backgrounds.  But % is taken.

Other symbol might be more intuitive or “lispy”, though I can’t
think of any, especially that is easy to type, “familiar” (subjective) and
intuitive (also subjective).

[… This alternative occurred to me meanwhile:

	(\ (f \1 1))

\ is used by Haskell for its lambda expressions (it’s supposed to remind
of λ).  \1, \2, etc is used in regular expressions to refer to capture
groups.  It’s somewhat ugly, but makes sense.]

Best regards,
Adrián.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> I understand your argument, particularly as it applies to the
> reader syntax.  On the other hand, dollar.el is just a macro: it
> does not complicate the syntax (for some definition of syntax);
> although it does complicate understanding when first presented.

Don't get me wrong: if I were to design Elisp from scratch, I'd most
probably choose a syntax for functions which has a handy "no explicit
parameter names" shorthand like `#(f % 1)` and also naturally
generalizes to the "fully explicit parameter names".

But I can't see a way to get a "shorthand" syntax sufficiently close to
the existing "longhand" syntax for this to work well.  I guess we could
try to do "the opposite" of what you do, and support

    (lambda [f $1 5])

which `prettify-symbols-mode` displays as

    (λ [f $1 5])

But even there, I'm not sure the benefit is worth the added complexity
for the occasional user trying to make sense of a few Elisp snippets.

> Would prettifying lambda to λ lead to visually unexpected indentation?

It does, occasionally, yes, as in:

    (lambda (x) (+ x
                   5))
vs
    (λ (x) (+ x
              5))

I know because I use `prettify-symbols-mode` ;-)


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Don't get me wrong: if I were to design Elisp from scratch, I'd most
> probably choose a syntax for functions which has a handy "no explicit
> parameter names" shorthand like `#(f % 1)` and also naturally
> generalizes to the "fully explicit parameter names".

Writing Clojure, I do find this stuff handy.  Reading Clojure, on the
other hand, I don't.  :-/

When I mentioned that (reduce + #(* %1 %2) things) was valid Clojure on
an IRC channel, people thought I was just joking.  When a programming
idiom is indistinguishable from satire, I think that may perhaps mean
that it's not a particularly good idiom.  On the other hand, Clojure is
really popular, so what do I know?

I have great sympathy for the idea that Emacs should cater to multiple
styles of programming.  I mean, it's Emacs: You should do whatever you
want.  If people like dash.el, then of course they should be allowed to
use that to their hearts content.  And people are voting with their feet
(I mean fingers): People really like s.el and dash.el.

On the other hand, Emacs has been around for 40 years, and will be
around for 40 more, and programming idioms come and go.  If Emacs had
succumbed to the "let's add more Perl idioms!" in the late 80s and
"let's make everything more Java-like!" in the 90s, then we'd be staring
at a much less cohesive code base.

Take things like Clojore-like threading: It's a sop to people who come
from Java and jQuery.  They can't read "inwards out", so:

(defn calculate []
   (reduce + (map #(* % %) (filter odd? (range 10)))))

becomes

(defn calculate* []
   (->> (range 10)
        (filter odd?)
        (map #(* % %))
        (reduce +)))

But while this syntax makes sense in jQuery, it's ambiguous where the
parameters are supposed to go in Lisp, so you have -> and ->> for two
common cases, and you have to write your functions with that in mind,
and then you have others where this doesn't work and you rewrite and
and and.

Some idioms translate well between languages, and some don't, but leave
a brittle, awkward, frustrating language surface, and I think that
that's where Clojure ended up.  If you read programming blogs, you'll
see an enormous enthusiasm for Clojure a few years back, and then less
and less, and there's a whole bunch of projects written in Clojure that
have been ported (back) to Java, because there just wasn't much of a
value proposition there anyway.  It's simply not a good Lisp.

(That's my impression from years of reading HackerNews (yeah, I know),
anyway.)

dash.el and s.el allows the masses of people coming from Clojure(script)
and JavaScript to acclimatise to Emacs Lisp a lot faster.  That's good.
But I'd rather not have to read the resulting un-Lispy code for the next
few decades in Emacs itself.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Bastien]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> dash.el and s.el allows the masses of people coming from Clojure(script)
> and JavaScript to acclimatise to Emacs Lisp a lot faster.  That's good.
> But I'd rather not have to read the resulting un-Lispy code for the next
> few decades in Emacs itself.

FWIW, as someone writing both Emacs Lisp and Clojure, the paragraph above
captures my opinion very well.

-- 
 Bastien

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Joost Kremers]

On Mon, May 18 2020, Adrián Medraño Calvo wrote:
> Typing λ is a no-go, in my opinion.

Well, on the assumption that Elisp is mostly written in Emacs 
using emacs-lisp-mode, one might introduce a command that makes it 
easier to type. But I tend to agree that it's a bad idea.

> Going with % and %1, %2, etc. would be fine, and would be 
> intuitive
> for people with Clojure backgrounds.  But % is taken.

Why is % taken? Because it's a function name? But Elisp is a 
Lisp-2.

> [… This alternative occurred to me meanwhile:
>
> 	(\ (f \1 1))
>
> \ is used by Haskell for its lambda expressions (it’s supposed 
> to remind
> of λ).  \1, \2, etc is used in regular expressions to refer to 
> capture
> groups.  It’s somewhat ugly, but makes sense.]

Not my personal favourite, but I'll get used to any syntax that's 
chosen. :-)

-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > [… This alternative occurred to me meanwhile:
  > >
  > > 	(\ (f \1 1))

In Emacs Lisp, \ quotes the next character, even a space.
That calls the function whose name is a space.  I don't think that
is a good syntax to choose.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Göktuğ Kayaalp]

On 2020-05-18 07:42 +03, Stefan Monnier wrote:
> FWIW, I'm not too fond of such shorthand syntax.  The benefit is not
> very high and it makes the language that much more difficult to learn
> for newcomers.
>
> For a programming language like Closure, it might make sense, since
> most/all people writing Closure programs are actual programmers that
> have to be proficient in Clojure.  But Elisp lambdas are very common in
> .emacs files, so this additional complexity will be exposed to some of
> our users who aren't programmers or aren't proficient in Elisp.

I don’t think it’s that much of additional complexity given many
popular programming languages have multiple ways to define one off
closures: Python (inline def vs. lambda), JavaScript, Ruby, even C++
these days AFAIU.

And, as someone else mentioned, the $ syntax has the nice effect that
C-h f can be enough.

> If we want to shorten anonymous functions we could start with something
> like (λ (x) (f x 1)), which you can get with
> prettify-symbols-mode already.

dollar.el and the reader modifications do more than just shortening the
keywords tho: it allows to define arguments whose names don’t really
matter.  Which is why these syntactic patterns go well with map-reduce
programs and when passing :key, :test, and simple callback arguments.

I’d say random one letter names or one off long names may create more
confusion than symbols that boil down to nth arg.

          -gk.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> I don’t think it’s that much of additional complexity given many
> popular programming languages have multiple ways to define one off
> closures: Python (inline def vs. lambda), JavaScript, Ruby, even C++
> these days AFAIU.

How many non-Python programmers have to touch such Python code?
[ Repeat that same question for your other examples.  ]

Anonymous functions are part of the subset of Elisp that random Emacs
users are exposed to in normal .emacs settings.  Any change that makes
this subset more complex is significantly more harmful than changes to
other parts of Elisp, so it needs to come with bigger upsides to pay
for itself.

Anyway, that's just my opinion.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 455 bytes --]

On Mon, May 18, 2020 at 11:34:49PM -0400, Stefan Monnier wrote:

[...]

> Anonymous functions are part of the subset of Elisp that random Emacs
> users are exposed to in normal .emacs settings.  Any change that makes
> this subset more complex is significantly more harmful than changes to
> other parts of Elisp, so it needs to come with bigger upsides to pay
> for itself.
> 
> Anyway, that's just my opinion.

Count me in.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Göktuğ Kayaalp]

On 2020-05-19 06:34 +03, Stefan Monnier wrote:
>> I don’t think it’s that much of additional complexity given many
>> popular programming languages have multiple ways to define one off
>> closures: Python (inline def vs. lambda), JavaScript, Ruby, even C++
>> these days AFAIU.
>
> How many non-Python programmers have to touch such Python code?
> [ Repeat that same question for your other examples.  ]

That’s fair.

> Anonymous functions are part of the subset of Elisp that random Emacs
> users are exposed to in normal .emacs settings.  Any change that makes
> this subset more complex is significantly more harmful than changes to
> other parts of Elisp, so it needs to come with bigger upsides to pay
> for itself.

One problem with this argument is that random, non-programming users are
only exposed to those things in their .emacs files which they put in
there in the first place.  So if there is some syntax X in there, they
either put it in there willingly, i.e. they knew what it does (to an
extent, at least), or they copied it from somewhere where the context
and contextual help would make it fairly easy to understand what it is
going on.

IMHO, we shouldn’t be that protective of non-programming users.  After
all, almost all concepts they’ll have to work with will be novel to
them, so I doubt a few shorthands would make that big of a difference.
I’d even say Elisp is a nicer environment to learn about the more
advanced concepts of programming because the unusual opportunities of
online inspection, introspection, and documentation Emacs provides.
Things like control structures or OOP are black boxes in even Ruby or
Python, and if you needed/wanted to look under the hood, you’d need to
fetch the source code (a non-trivial task) and study it to find what’s
where, whereas in Emacs if you want to see how ‘if’ is implemented all
you do is C-h f if RET & hit a link, and you have the C code before your
eyes.

        -gk.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> IMHO, we shouldn’t be that protective of non-programming users.  After
> all, almost all concepts they’ll have to work with will be novel to
> them, so I doubt a few shorthands would make that big of a difference.

Agreed, which is why I think it's perfectly fine to have anonymous
lambdas in there [even tho they are still "scary higher-order
programming" to many professional programmers, last I checked ;-) ]

We don't disagree on the advantages and disadvantages.  We just disagree
on the respective weight we put on them.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Göktuğ Kayaalp]

On 2020-05-19 18:52 +03, Stefan Monnier wrote:
> We don't disagree on the advantages and disadvantages.  We just disagree
> on the respective weight we put on them.

:thumbs-up:

                                   ⁂

I wonder what’s the final say on this.  AFAIU y’all are not keen on
merging this.  Or is it still up for discussion?  There’s a nice patch
by Adrián I’ll be merging soon which removes the dash dependency and
fixes a few problems.

I’m fine with dollar.el remaining as is: a nice little addition to your
.emacs.d that can make smaller lambdas a bit more terse, not really
intended for use in packages (which is FWIW why I didn’t put it on
Melpa).


Best,
        -gk.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Göktuğ Kayaalp <self@gkayaalp.com>
> Date: Wed, 20 May 2020 17:04:28 +0300
> Cc: jonas@bernoul.li, Richard Stallman <rms@gnu.org>, joostkremers@fastmail.fm,
>  adrian@medranocalvo.com, emacs-devel@gnu.org, ohwoeowho@gmail.com,
>  Göktuğ Kayaalp <self@gkayaalp.com>,
>  Phillip Lord <phillip.lord@russet.org.uk>
> 
> 
> On 2020-05-19 18:52 +03, Stefan Monnier wrote:
> > We don't disagree on the advantages and disadvantages.  We just disagree
> > on the respective weight we put on them.
> 
> :thumbs-up:
> 
>                                    ⁂
> 
> I wonder what’s the final say on this.  AFAIU y’all are not keen on
> merging this.  Or is it still up for discussion?  There’s a nice patch
> by Adrián I’ll be merging soon which removes the dash dependency and
> fixes a few problems.

I think the only proposal on the table that has real chances to be
acceptable by most of us is to split ELPA into two, whether internally
(i.e. separate directories or something similar) or externally (i.e. 2
different repositories).  If there's no significant opposition to
this, I hope we can start actually doing that.  Then s.el, dash.el,
and other similar packages can be put in the "non-core" part of ELPA
without any significant changes.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Sun, 17 May 2020 23:47:12 -0400
> Cc: jonas@bernoul.li, joostkremers@fastmail.fm, adrian@medranocalvo.com,
>  emacs-devel@gnu.org, ohwoeowho@gmail.com, self@gkayaalp.com,
>  monnier@iro.umontreal.ca
> 
> Let's see what Eli thinks about it.

My opinion on these matters means very little, but FWIW I basically
agree with what Stefan wrote.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Göktuğ Kayaalp]

On 2020-05-15 06:24 +03, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> The choice between the syntax ($ (+ $1 $2))
> and the syntax #(+ %1 %2), I think it is a small difference.
> If one is ok, the other is ok.

Sure.

> The # syntax has the small advantage of being a little shorter.
>
> The $ syntax can be implemented now with a macro -- but if we want this
> feature, I see no reason to hesitate to do implement it in the reader,
> no need to use a macro.  It would not be a lot of work to do that.

I think the main point which can help pick one over the other is whether
or not the reader implementation can reliably and easily be backported.
AFAIU Emacs can’t do reader macros, so in order to implement this in the
reader we’d have to change stuff in C which means the feature is limited
to future releases. A Lisp macro (with whatever syntax, I’m not really
partial to $, just that it’s short and, reserved as an operator, and is
mildly similar to how most Unix shells do it) OTOH can be backported
easily via an ‘fboundp’ check or (less desirably) a dependency on an
obsoleted package we might introduce to ELPA (would require an Emacs
version check to reliably control), making it easier to adopt especially
in packages.

This is especially relevant for those who use Emacs from their distro
repos who won’t be on board with latest releases possibly for years, and
for most popular packages who still support at least Emacs 24. A reader
implementation would confine this to the init.el’s of those who (like
me) follow the development branch for years to come, which is
unnecessarily limiting given we have the luxury of not needing to change
the parser.

--
İ. Göktuğ Kayaalp / @cadadr / <https://www.gkayaalp.com/>
pgp:   024C 30DD 597D 142B 49AC 40EB 465C D949 B101 2427

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

If we decide to adopt a brief syntax for anonymous functions, we can
make the reader handle it.  So don't worry about that.  What matters
is what syntax we want.

The real obstacle would be if a syntax has some other meaning already.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Joost Kremers]

On Tue, May 12 2020, Stefan Monnier wrote:
>> In Clojure, a function literal is written with `#(...)`, and it 
>> supports
>> `%n` for arguments, where n=1,2,3...
>
> IIRC there's a macro somewhere providing a similar facility.
> We can't use (# ...) because the reader doesn't like `#` on its 
> own (it
> expects it to be followed by something like a vector, a 
> hex/binary/octal
> number, etc...).

Doing it in the reader would be nicer, because `#(/ 3 %)` is more 
obviously a division than `(# / 3 %)`, I think.

-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

> Doing it in the reader would be nicer, because `#(/ 3 %)` is more obviously
> a division than `(# / 3 %)`, I think.

I think to the untrained eye, these two specific example both look
equally like line-noise.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Joost Kremers]

On Wed, May 13 2020, Stefan Monnier wrote:
>> Doing it in the reader would be nicer, because `#(/ 3 %)` is 
>> more obviously
>> a division than `(# / 3 %)`, I think.
>
> I think to the untrained eye, these two specific example both 
> look
> equally like line-noise.

That's definitely true. But some line noise is better than other 
line noise. :-) 

-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > In Clojure, a function literal is written with `#(...)`, and it 
  > supports `%n` for arguments, where n=1,2,3... (Actually, I don't 
  > know if Clojure supports `%10` and up or if it just goes up to 
  > `%9`, but I guess that doesn't matter much). A single `%` is 
  > synonymous with `%1`, which is used when there's only one 
  > argument. Also supported is `%&`, which is similar to `&rest 
  > <var>` in Elisp, being bound to a list containing the remaining 
  > arguments.

This is rather shell-like.  Can we find a nicer one?

It doesn't specify how many args the literal function should accept.
Maybe there is no need for that, but if we want to do that, how could we?

Is there ever a need for nested literal functions?  To implement them
fully, we would want to use different arg names at each level.

What Lisp syntax would people suggest for this construct?
What is available?

Does anyone think this is a bad idea?


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

literal functions [was: Re: dash.el]

[Joost Kremers]

On Wed, May 13 2020, Richard Stallman wrote:
>   > In Clojure, a function literal is written with `#(...)`, and 
>   > it 
>   > supports `%n` for arguments, where n=1,2,3... (Actually, I 
>   > don't 
>   > know if Clojure supports `%10` and up or if it just goes up 
>   > to 
>   > `%9`, but I guess that doesn't matter much). A single `%` is 
>   > synonymous with `%1`, which is used when there's only one 
>   > argument. Also supported is `%&`, which is similar to `&rest 
>   > <var>` in Elisp, being bound to a list containing the 
>   > remaining 
>   > arguments.
>
> This is rather shell-like.  Can we find a nicer one?

Not sure what a nicer syntax would look like. If nicer means more 
verbose,  function literals might become pointless, because their 
compactness is IMHO the main reason for having them.

Using Unicode would be nice:

    (seq-reduce λ(* 2 (+ α β)) (number-sequence 1 10) 0)

But that runs into obvious problems (font support, input method). 

Personally, I wouldn't be adverse to adopting the Clojure syntax 
for function literals wholesale, except that `#(...)` already has 
a meaning for the reader. Perhaps `#f(...)` can be used instead, 
since it follows the existing pattern (`#` plus a single 
character).

> It doesn't specify how many args the literal function should 
> accept.
> Maybe there is no need for that, but if we want to do that, how 
> could we?

Something like

    #f2(* 2 (+ %1 %2))

with the 2 after #f indicating the number of arguments? Doable, I 
guess. But I'm not sure it gets you anything, unless you also find 
a way to indicate the use of an `&rest` argument. If you don't, 
you'd need to go through the entire form anyway to search for it, 
and then you might as well collect the other arguments as well.

You could of course decide not to support `&rest` arguments. That 
would limit a function literal's usefulness somewhat, but that 
might be an acceptable compromise.

OTOH you might want to use gensyms to represent the arguments in 
the expanded lambda form. In that case you need to go through the 
entire form as well, so no need to indicate the number of 
arguments.

A related question is what to do about function literals such as 
`#f(* 2 (+ %1 %3))`. If the reader collects all %n arguments and 
puts them in the argument list in the right order, this doesn't 
have to be an error. OTOH if you use the #f2 syntax and let the 
reader create an argument list without checking, a form such as , 
`#f2(* 2 (+ %1 %3))` would yield a void variable error upon 
execution.

My personal preference would be to just use `#f(...)` and not 
indicate the number of arguments.

> Is there ever a need for nested literal functions?  To implement 
> them
> fully, we would want to use different arg names at each level.

Which is probably why Clojure doesn't allow nesting them. Function 
literals are not meant to replace lambdas in every context, they 
are mainly meant as short, one-off function arguments to 
higher-order predicates, so it's an acceptable compromise that 
they're not nestable. 

> Does anyone think this is a bad idea?

I obviously don't. :-)

-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Phillip Lord]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > In Clojure, a function literal is written with `#(...)`, and it 
>   > supports `%n` for arguments, where n=1,2,3... (Actually, I don't 
>   > know if Clojure supports `%10` and up or if it just goes up to 
>   > `%9`, but I guess that doesn't matter much). A single `%` is 
>   > synonymous with `%1`, which is used when there's only one 
>   > argument. Also supported is `%&`, which is similar to `&rest 
>   > <var>` in Elisp, being bound to a list containing the remaining 
>   > arguments.
>
> This is rather shell-like.  Can we find a nicer one?

Suggest away.


> It doesn't specify how many args the literal function should accept.
> Maybe there is no need for that, but if we want to do that, how could we?

You look for the maximum argument number

#(list %)    takes one argument
#(list %1 %2) takes two
#(list %1 %2 %3)  takes three.


> Is there ever a need for nested literal functions?  To implement them
> fully, we would want to use different arg names at each level.

You don't. They are for short functions that you pass to something
else, and for which you don't want to name parameters. For nesting, you
use lambda.


> What Lisp syntax would people suggest for this construct?
> What is available?
>
> Does anyone think this is a bad idea?

It's a small feature, and convenient for some things, but overuse makes
the code messy.

Phil

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > This is rather shell-like.  Can we find a nicer one?

  > Suggest away.

No ideas occur to me.  Does anyone else have an idea?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It appears that the syntax #(list %1 %2) is not in use in Emacs Lisp.
%1 is a symbol so it can be a variable.

So we could implement that same syntax.

One wrinkle is that read needs to distinguish this construct
from strings with text properties.
A string with text properties starts with #( followed by an ordinary string.
They can be distinguished, but it is somewhat ugly.

It appears that the Emacs Lisp Reference Manual doesn't describe
all of read syntax in any one concise place.  It would be good
to add an appendix with that.
-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Joost Kremers]

On Sat, May 16 2020, Richard Stallman wrote:
> It appears that the Emacs Lisp Reference Manual doesn't describe
> all of read syntax in any one concise place.  It would be good
> to add an appendix with that.

That seems to have already been taken care of: In Emacs 27, the 
Elisp manual has a node "Lisp Data Types > Special Read Syntax".

-- 
Joost Kremers
Life has its moments

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > That seems to have already been taken care of: In Emacs 27, the 
  > Elisp manual has a node "Lisp Data Types > Special Read Syntax".

That is nice.  The build I'm using is from Oct and this
seems to have been added since then.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Sat, 09 May 2020 22:33:43 -0400
> Cc: joostkremers@fastmail.fm, ams@gnu.org, emacs-devel@gnu.org
> 
> Thanks for showing me examples of dash.el.  A lot of these facilities
> seem useful.  I would very much like to see the _documentation_ for how
> to use it.

I suggest to contrast these facilities with what we have in seq.el and
map.el.  Dash was created when we didn't have these facilities in
Emacs, but now we do.

RE: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Drew Adams]

> they are two extremely different libraries that
> have nothing in common.

Sorry - can't resist finding some humor in imagining
pairs of things/people that are extremely different
and yet have everything in common. ;-)

Unity of opposites, and all.

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Stefan Monnier]

>    - As far as I know you are the only one who objected strongly against
>    s.el in ELPA (others please voice your opinion if you think like
>    Richard). 
>
> I as a Emacs user would think s.el is a bad addition to ELPA -- it
> doesn't add anything special,

It is used by hundreds of packages.  So as long as it's not in GNU ELPA,
none of those hundreds of packages can be even considered for inclusion
into GNU ELPA.

That's the special that it adds.

> and makes code harder to read if people use such constructs in the
> form that they are now, encouraging people to use s-titleize instead
> of capitalize is going backwards, not forwards.

I highly doubt including it in GNU ELPA will make much difference w.r.t
encouraging people to use it.

> I haven't seen anyone objecting from adding some of the more
> complicated functions and making them follow a more Emacs-y form, or
> even on a case-by-case basis renaming some string functions to make
> more sense. 

That's a completely different discussion (which affects Emacs's core
but doesn't help clear the obstacles for inclusion of other packages
into GNU ELPA).

> So why this forceful insistance of adding a s.el?  Why not try to make
> Emacs as a whole better by suggesting the better parts of s.el that
> can be added to Emacs, or finding functions that could be renamed?

Whether s.el is included into GNU ELPA has no bearing on whether or not
we'll want to include some of its ideas into Emacs's core string functions.


        Stefan

Re: dash.el [was: Re: Imports / inclusion of s.el into Emacs]

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > - For most users, dash.el and s.el are very similar in nature. dash.el
  > is already in ELPA. If we refuse s.el, isn't it inconsistent? 

Treating two different cases differently is never "inconsistent".
This is a software project, not a court of law.

There is a superficial similarity between dash.el and s.el:
they both make a mess in the Emacs namespace.

However, at the level of substance they are not similar.
s.el does not try to add substantive features -- it is
about names only.

By contrast, dash.el has real substance: it is useful.  Some of the
ccnstructs it defines -- the ones I have seen -- are useful and very
general.

I look forward to seeing the rest of the constructs in dash.el, but
based on what I've seen, I expect to see more good features.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 07.05.2020 05:44, Richard Stallman wrote:

>    > Richard, Eli: please decide wether you want s.el into ELPA or not.
> 
> In its current form, I think it would be a grave mistake to include
> s.el in ELPA.  Its purpose is to make Emacs Lisp mimic object-oriented
> languages which are alien to Emacs Lisp.

No, its purpose is to mimic Clojure, which is a modern Lisp and a 
functional language. Not any more object oriented than CL.

> See my message to Stefan for a change that would make s.sl ok to add.
> 
>    > I'm not sure why there's this sudden turnaround on this issue, maybe
>    > I'm missing something.
> 
> I don't think there was a turnaround.  We never decided to include
> s.el in GNU ELPA.
> 
> Before yesterday, we were talking about a different question: whether
> to adopt the s.el functions and their names in Emacs Lisp.  When I saw
> concretely what those actually were, I said no.

Are you saying that no decision, no matter how minor, should be 
considered "undecided" until you have personally weighed in?

Stefan has been responsible for GNU ELPA from the outset, and has been 
doing almost all the work on it. Both technical and social.

> Then yesterday I saw a message proposing to include s.el in ELPA, and
> _presuming_ that that was ok.  I responded no, saying that it would
> send Emacs Lisp down the same wrong path.  We should not have code in
> Emacs that uses string-prepend instead of concat.  We should fix that
> code to use concat.

I'm sorry, but you sound confused. No code in Emacs would use 
string-prepend because it's not in the cards for addition. *Some* code 
in GNU ELPA could start using s-prepend after s.el is added. But not in 
Emacs itself.

>    > This is a bit embarassing for me to have done the work of getting
>    > magnars to agree to put it there just to be refused at the last
>    > minute,
> 
> I am sorry for your disappointment, because I feel for your eagerness
> to make a change you consider an improvement.  But we have to make the
> decision that is right.
> 
> There is no need for you to feel embarrassed.  The people you talked
> with will understand that there is no reason to blame or criticize you
> for this.

I think this decision will do nothing to improve the community's 
reputation of emacs-devel (which has been improving in the recent years, 
but is still not stellar).

And overriding a fellow developer's decision in his area of 
responsibility *is* likely to affect his reputation. That seems to be 
not very ethical of you.

Note that I personally hold no love for s/f/dash.el, and use none of 
them in my projects.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I have the impression that you don't live in the same universe as mine:

If your universe is limited to Emacs Lisp packages outside Emacs, then
indeed I am living outside your universe.  I never come in contact
with them.  If you would like me to know about those, how about telling
me from time to time?

  > in my world, `s.el` is already used by the majority of new packages even

If that is so, it is quite unfortunate.  But things could get worse:
we could include s.el in GNU ELPA, or even in Emacs itself, and thus
spread its influence even wider.

We could tell ourselves that this was merely an extension users could
use or not.  But that would be closing our eyes to the situation.

For instance, people would complain that these "essential facilities
that so many packages use" were not in the Emacs Lisp Reference
Manual.  They would demand that we give priority to the usual
Clojure-based names and relegate the "irregular" 'concat' and 'format'
to an appendix.

If we did not comply, they would bombard us with verbal aggressions
such as, "We are the majority, what universe are you in?"

Likewise if we defined other Emacs Lisp functions with names starting
with 'string-', departing from Clojure's spec, they would accuse us of
risking a name conflict with future Clojure extensions.

Rather than let that happen, we should say no at the outset.

However, a small change would avoid these problems.  We could rename
the file to clostring.el and rename the functions to start with clo
also: clostring-prepend, clostring-format, etc.

This file would do the same job as s.el, but including it in GNU ELPA
or in Emacs itself would not have the negative effects I described
above.  It would truly be simply an extension that programmers could
use or not.  There would be no reason not to include it.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> If that is so, it is quite unfortunate.  But things could get worse:
> we could include s.el in GNU ELPA, or even in Emacs itself, and thus
> spread its influence even wider.

If that's your worry, I'm not impressed.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[조성빈]

Richard Stallman <rms@gnu.org> 작성:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>> I have the impression that you don't live in the same universe as mine:
>
> If your universe is limited to Emacs Lisp packages outside Emacs, then
> indeed I am living outside your universe.  I never come in contact
> with them.  If you would like me to know about those, how about telling
> me from time to time?
>
>> in my world, `s.el` is already used by the majority of new packages even
>
> If that is so, it is quite unfortunate.  But things could get worse:
> we could include s.el in GNU ELPA, or even in Emacs itself, and thus
> spread its influence even wider.

You know what? By these explicit decisions, ELPA is close to useless in the
Emacs community, MELPA is the biggest repo, and since MELPA isn’t ELPA, it
already has proprietary packages that a lot of people rely and use.

I personally dislike all of that free software movement, but by alienating  
99%
of Emacs users for the sake of elisp purity (which frankly doesn’t make sense
at all, I didn’t try to use strong words in the previous discussions but
Elisp’s API naming is just such a terrible mess and has zero consistency,  
some
new package that at least has some internal consistency doesn’t make it  
worse)
will push people to use MELPA much more, and people will find it much easier
to install proprietary Emacs plugins. I’m pretty sure you would hate that?

(For an example, check out the https://github.com/TommyX12/company-tabnine
  plugin that works with the tabnine GPT code completer that’s proprietary,
  which I already know quite a few people use it.)

> We could tell ourselves that this was merely an extension users could
> use or not.  But that would be closing our eyes to the situation.
>
> For instance, people would complain that these "essential facilities
> that so many packages use" were not in the Emacs Lisp Reference
> Manual.

That looks like a valid complaint, it’s just that essential facilities aren’t
in Emacs, not Eintr.

> They would demand that we give priority to the usual
> Clojure-based names

No, people will demand names with consistency, not Clojure-based names. And
the only reason s.el used the Clojure API as a base is that they actually
consider the users and design a proper API, they do not say that you can look
names up on the manuals.

> and relegate the "irregular" 'concat' and 'format'
> to an appendix.
>
> If we did not comply, they would bombard us with verbal aggressions
> such as, "We are the majority, what universe are you in?"

That’s quite true, I can guarantee that 99% of Emacs users don’t know what
mailing lists are, why there are crazy names like assq, and just think that
Emacs has this crazy API just because some old timers designed it. Emacs
package developers find the API strange and make packages like s.el, a usual
Emacs user gets alienated with the API. People are already in a different
universe with emacs-devel. And this whole discussion was about trying to  
merge
them, and everybody is like one can use C-h a, C-h d, and that’s how lisp
works.

> Likewise if we defined other Emacs Lisp functions with names starting
> with 'string-', departing from Clojure's spec, they would accuse us of
> risking a name conflict with future Clojure extensions.
>
> Rather than let that happen, we should say no at the outset.
>
> However, a small change would avoid these problems.  We could rename
> the file to clostring.el and rename the functions to start with clo
> also: clostring-prepend, clostring-format, etc.
>
> This file would do the same job as s.el, but including it in GNU ELPA
> or in Emacs itself would not have the negative effects I described
> above.  It would truly be simply an extension that programmers could
> use or not.  There would be no reason not to include it.
>
> -- 
> Dr Richard Stallman
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> You know what? By these explicit decisions, ELPA is close to useless in the
> Emacs community, MELPA is the biggest repo, and since MELPA isn’t ELPA, it
> already has proprietary packages that a lot of people rely and use.

I'm very surprised to hear that.  Can you point to examples of
proprietary packages?  It's virtually impossible to legally distribute
a proprietary Elisp package because it inevitably has to link to some
existing Elisp code and almost all of it is GLPv3.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

> > You know what? By these explicit decisions, ELPA is close to useless in the
> > Emacs community, MELPA is the biggest repo, and since MELPA isn’t ELPA, it
> > already has proprietary packages that a lot of people rely and use.
>
> I'm very surprised to hear that.  Can you point to examples of
> proprietary packages?  It's virtually impossible to legally distribute
> a proprietary Elisp package because it inevitably has to link to some
> existing Elisp code and almost all of it is GLPv3.

I'm pretty sure he meant "packages that use proprietary software".

In the example he cited (https://github.com/TommyX12/company-tabnine)
we see the license is MIT.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 07.05.2020 22:03, Philippe Vaucher wrote:
>>> You know what? By these explicit decisions, ELPA is close to useless in the
>>> Emacs community, MELPA is the biggest repo, and since MELPA isn’t ELPA, it
>>> already has proprietary packages that a lot of people rely and use.
>> I'm very surprised to hear that.  Can you point to examples of
>> proprietary packages?  It's virtually impossible to legally distribute
>> a proprietary Elisp package because it inevitably has to link to some
>> existing Elisp code and almost all of it is GLPv3.
> I'm pretty sure he meant "packages that use proprietary software".
> 
> In the example he cited (https://github.com/TommyX12/company-tabnine)
> we see the license is MIT.

Indeed. So it's not a problem from the licensing point of view (I think).

But his general point is probably valid: the more we alienate the 
third-party community, and the more we are dismissive of their needs and 
expectations, the more likely they are to discount any of our other 
recommendations, pay no attention whether a package is in GNU ELPA or 
not, and generally use whatever programs that *do* attend to their needs.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But his general point is probably valid: the more we alienate the 
  > third-party community, and the more we are dismissive of their needs and 
  > expectations, the more likely they are to discount any of our other 
  > recommendations, pay no attention whether a package is in GNU ELPA or 
  > not, and generally use whatever programs that *do* attend to their needs.

Don't worry, it won't make a big difference.
We must not suppose we desperately need someone to like us.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[조성빈]

2020. 5. 9. 오후 12:51, Richard Stallman <rms@gnu.org> 작성:
> 
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>> But his general point is probably valid: the more we alienate the 
>> third-party community, and the more we are dismissive of their needs and 
>> expectations, the more likely they are to discount any of our other 
>> recommendations, pay no attention whether a package is in GNU ELPA or 
>> not, and generally use whatever programs that *do* attend to their needs.
> 
> Don't worry, it won't make a big difference.
> We must not suppose we desperately need someone to like us.

You do realize that almost nobody uses ELPA only, do you? I thought Emacs was a program to lead people using ‘free software’, that’s why all of the stuff that enhances macOS gets rejected, right?

> -- 
> Dr Richard Stallman
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)
> 
> 

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 09.05.2020 06:50, Richard Stallman wrote:
>    > But his general point is probably valid: the more we alienate the
>    > third-party community, and the more we are dismissive of their needs and
>    > expectations, the more likely they are to discount any of our other
>    > recommendations, pay no attention whether a package is in GNU ELPA or
>    > not, and generally use whatever programs that*do*  attend to their needs.
> 
> Don't worry, it won't make a big difference.
> We must not suppose we desperately need someone to like us.

We must not create such dislike over relatively unimportant issues either.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We must not create such dislike over relatively unimportant issues either.

I'm concerned with goals I consider quite important, such as (1)
defending copyleft, (2) supporting the rest of the GNU Project, and
(3) the overall design of GNU Emacs.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 10.05.2020 05:31, Richard Stallman wrote:
>    > We must not create such dislike over relatively unimportant issues either.
> 
> I'm concerned with goals I consider quite important, such as (1)
> defending copyleft, (2) supporting the rest of the GNU Project, and
> (3) the overall design of GNU Emacs.

Right. And I'll posit that including s.el into GNU ELPA, by itself, will 
not hurt any of these three goals.

It won't be used inside Emacs. And as for projects "outside", it seems 
those that want to use it, would rather not be featured in GNU ELPA 
rather than stop using it.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I'm concerned with goals I consider quite important, such as (1)
  > > defending copyleft, (2) supporting the rest of the GNU Project, and
  > > (3) the overall design of GNU Emacs.

  > Right. And I'll posit that including s.el into GNU ELPA, by itself, will 
  > not hurt any of these three goals.

It would mess up the naming in GNU Emacs, making two incongruous
systems.  AMS made it clear:

  > E.g., how is s-ends-with? more discoverable than string-suffix-p? When
  > Emacs Lisp custom is to use -p for predicates, and to use the words
  > suffix/prefix.  To take a trivial example.

  ...

  > But we should view it from the lense of an Emacs Lisp hacker, where
  > discovering that one function for predicates is named with -p and some
  > with ? will be a big mess.  Where sometimes one uses ends/starts
  > instead of suffix/prefix.

In addition to that incongruity in naming, look at the way some would
have us create it.  Those who argue for incorporating s.el are saying
that we should abandon judging Emacs design decisions, and blindly
yield to the choices of a group of people that hardly talk with us,
value other traditions more than those of Emacs and Lisp, and would
like to paste the others over those two.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 11.05.2020 05:37, Richard Stallman wrote:

>    > Right. And I'll posit that including s.el into GNU ELPA, by itself, will
>    > not hurt any of these three goals.
> 
> It would mess up the naming in GNU Emacs, making two incongruous
> systems.

Not in GNU Emacs, though. In ELPA. I would agree that s.el is, in a 
sense, unnecessary (considered by itself), but there are clear benefits 
to including it as well, as a part of existing ecosystem.

> Those who argue for incorporating s.el are saying
> that we should abandon judging Emacs design decisions, and blindly
> yield to the choices of a group of people that hardly talk with us,
> value other traditions more than those of Emacs and Lisp, and would
> like to paste the others over those two.

This sounds like a strawman. We're not discussing doing that.

There was a separate discussion on improving some of the names of the 
functions in the core, but you effectively put a stop to that endeavor. 
I find that regrettable as well.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Cc: stefan@marxist.se, emacs-devel@gnu.org, joaotavora@gmail.com,
>  pcr910303@icloud.com, eliz@gnu.org, drew.adams@oracle.com,
>  monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 11 May 2020 05:54:30 +0300
> 
> > It would mess up the naming in GNU Emacs, making two incongruous
> > systems.
> 
> Not in GNU Emacs, though. In ELPA. I would agree that s.el is, in a 
> sense, unnecessary (considered by itself), but there are clear benefits 
> to including it as well, as a part of existing ecosystem.

AFAIU, that ecosystem doesn't require that packages be available on
the same site.  So there should be nothing wrong with having some of
them elsewhere.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 11.05.2020 18:02, Eli Zaretskii wrote:
> AFAIU, that ecosystem doesn't require that packages be available on
> the same site.  So there should be nothing wrong with having some of
> them elsewhere.

Only if we're going to start adding MELPA (or, rather, MELPA Stable) to 
the default list of package repositories.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Cc: rms@gnu.org, stefan@marxist.se, emacs-devel@gnu.org,
>  joaotavora@gmail.com, pcr910303@icloud.com, drew.adams@oracle.com,
>  monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 11 May 2020 19:24:03 +0300
> 
> On 11.05.2020 18:02, Eli Zaretskii wrote:
> > AFAIU, that ecosystem doesn't require that packages be available on
> > the same site.  So there should be nothing wrong with having some of
> > them elsewhere.
> 
> Only if we're going to start adding MELPA (or, rather, MELPA Stable) to 
> the default list of package repositories.

Why do we have to do that by default?  People who want those packages
can (and do) do that by themselves.

In any case, adding to ELPA packages just so we could let people
install them with absolutely no customization sounds like a tail
wagging the dog to me.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 11.05.2020 19:55, Eli Zaretskii wrote:
> In any case, adding to ELPA packages just so we could let people
> install them with absolutely no customization sounds like a tail
> wagging the dog to me.

I don't understand this.

Surely we want people to be able to edit more file formats 
out-of-the-box, for instance.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Cc: rms@gnu.org, stefan@marxist.se, emacs-devel@gnu.org,
>  joaotavora@gmail.com, pcr910303@icloud.com, drew.adams@oracle.com,
>  monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 11 May 2020 20:01:50 +0300
> 
> Surely we want people to be able to edit more file formats 
> out-of-the-box, for instance.

Not at any cost.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Today I came across a message remaining from May 11 and concluded I had
better respond to it.  I regret taking so long.

  > >    > Right. And I'll posit that including s.el into GNU ELPA, by itself, will
  > >    > not hurt any of these three goals.
  > > 
  > > It would mess up the naming in GNU Emacs, making two incongruous
  > > systems.

  > Not in GNU Emacs, though.

Yes, in GNU Emacs.  If we have s.el in GNU ELPA under current
conditions, that will put the s- namespace of GNU Emacs outside our
control.  (Indeed, someone claimed it is already de facto outside our
control.)

It is a big problem that other people, without even consulting us, can
take major parts of the Emacs Lisp namespace away from us.

Maybe we can correct this general problem in a general way with a
system of namespaces.  I hope so.  But denying the problem is not
solving it.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 03.06.2020 07:24, Richard Stallman wrote:
> Today I came across a message remaining from May 11 and concluded I had
> better respond to it.  I regret taking so long.
> 
>    > >    > Right. And I'll posit that including s.el into GNU ELPA, by itself, will
>    > >    > not hurt any of these three goals.
>    > >
>    > > It would mess up the naming in GNU Emacs, making two incongruous
>    > > systems.
> 
>    > Not in GNU Emacs, though.
> 
> Yes, in GNU Emacs.  If we have s.el in GNU ELPA under current
> conditions, that will put the s- namespace of GNU Emacs outside our
> control.  (Indeed, someone claimed it is already de facto outside our
> control.)

That "someone" was me as well.

> It is a big problem that other people, without even consulting us, can
> take major parts of the Emacs Lisp namespace away from us.
> 
> Maybe we can correct this general problem in a general way with a
> system of namespaces.  I hope so.  But denying the problem is not
> solving it.

I agree it's a problem, but my (and not only mine) point was that adding 
the package to ELPA won't really change anything in this regard.

A namespace system would be a welcome change, of course.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I agree it's a problem, but my (and not only mine) point was that adding 
  > the package to ELPA won't really change anything in this regard.

I understand your point of view.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Basically I focus more on the advantages of putting the
  > discoverability/consistency inside the language itself instead of its
  > tools,

Changing the language for the sake of discoverability or consistency
is an incompatibility that will make life difficult for other
programmers.  If a name is not bad, we should not change it.

By contrast, we can add all sorts of tools, or alternate modes for
existing tools, without making anything difficult for anyone.  Those
who don't like the new tools and modes can simply not use them.

We can get the same results with either method.  For instance,
grouping 'concat' and 'format' with 'string-' functions can be
achieved either way.  Therefore, we must choose to the method of
tools, which doesn't inconvenience others.


As a separate matter, there may be SOME names that are so unnatural
and ugly that renaming them would be a clear improvement.  Not
'concat' or 'format' -- those names are clean and good -- but perhaps
for some of the regexp functions.

Please treat the string functions and the regexp functions as two
different issues.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 05.05.2020 13:14, João Távora wrote:
> You'll always
> get a group of people that really expected it to be multibyte-string and
> group of people that expect string-multibyte.

If there is an actual naming standard, the people can learn.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 1:22 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 05.05.2020 13:14, João Távora wrote:
> > You'll always
> > get a group of people that really expected it to be multibyte-string and
> > group of people that expect string-multibyte.
>
> If there is an actual naming standard, the people can learn.

Sorry, but that sounds either naive or arrogant, not sound.

Surely you know https://xkcd.com/927/

João

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 05.05.2020 15:53, João Távora wrote:
> Surely you knowhttps://xkcd.com/927/

That comic is about multiplication of standards. The present discussion 
is actually is about applying one of the standards we already have more 
consistently.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 2:03 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 05.05.2020 15:53, João Távora wrote:
> > Surely you knowhttps://xkcd.com/927/
>
> That comic is about multiplication of standards. The present discussion
> is actually is about applying one of the standards we already have more
> consistently.

Hah, what an exquisitely contorted way to say "standardize".

João

Re: Imports / inclusion of s.el into Emacs

[조성빈]

Dmitry Gutov <dgutov@yandex.ru> 작성:

> On 05.05.2020 13:14, João Távora wrote:
>> You'll always
>> get a group of people that really expected it to be multibyte-string and
>> group of people that expect string-multibyte.
>
> If there is an actual naming standard, the people can learn.

I’ve proposed here:  
https://lists.gnu.org/archive/html/emacs-devel/2020-05/msg00671.html
to create a preliminary Elisp API standard:

I think some people will be interested here.

Below is the full mail contents:

> 조성빈 <pcr910303@icloud.com> 작성:
>
>> Philippe Vaucher <philippe.vaucher@gmail.com> 작성:
>>
>>>>> Given this is more or less the position held by Alan, Eli, Richard,
>>>>> Drew and João I think the chances of seeing new aliases is close to 0.
>>>>
>>>> This is not my conclusion.  I've seen several calls to move away from
>>>> from discussing in the abstract to discuss specific, concrete
>>>> examples.  I think this is a good idea, since IMHO the abstract
>>>> discussion is likely exhausted.
>>>>
>>>> There is always the chance that some of the proposals will be voted
>>>> down.  But also consider that some who have disagreed with you in the
>>>> abstract might be more convinced by specific, concrete proposals.
>>>
>>> So far the string- proposal got shot down entirely. The regexp one was
>>> initially a no-go from Alan but I then Richard kinda liked it and
>>> proposed adaptations.
>>>
>>> @Stefan Monnier: I see that you talked about `multibyte-string-p`
>>> already (and iirc that didn't went well. You talked earlier about
>>> `process-`, maybe you'd like to propose some changes there?
>>
>> I think for people to propose changes and get them adapted, we first  
>> have to
>> have some proper goals to target.
>>
>> So there are a few people here who think renaming some functions is  
>> beneficial
>> - but everybody’s reasoning is different here. Some people who are  
>> opposed to
>> renaming are a bit confused.
>>
>> I think the two big goals are consistency and discoverability. And then  
>> there
>> are various small arguments like it’s easier to use prefix based  
>> completion and
>> function search, it’s easier to guess, namespace means less function name
>> clashing, etc…
>>
>> I think consistency is important, and if the language itself wants naming
>> things the ‘lisp-way’, I’m fine with a consistent naming scheme. I’m not  
>> sure
>> if you’d agree or not, but maybe trying to find a consistent naming  
>> scheme and
>> documenting them (which was called as the ‘lisp-way’ by some) might be  
>> first.
>> And then we can rename the ones that don’t follow them.
>>
>>> I mean I'm willing to propose concrete changes but if it's not obvious
>>> for string- and regexp- why would it be for other topics? Let's try
>>> another topic just to see:
>>>
>>> rename-file -> file-rename
>>> delete-file -> file-delete
>>> copy-file -> file-copy
>>> expand-file-name -> file-expand-name
>>>
>>> Do you think people will be ok with that?
>>
>> The reason why I said about finding the naming scheme was because both the
>> function name rename-file and file-truename makes sense to me.
>>
>> I think some preliminary conventions that Elisp already follows is that  
>> the
>> <action>-<object> scheme is for actions on the object like rename-file or
>> clear-string, and <object>-<property> scheme is for getting properties  
>> of the
>> object like string-width and file-name-extension. (I’m not considering
>> polymorphic ones.)
>>
>> But then there are exceptions, like string-trim (which should then be
>> trim-string) or string-join (which should then be join-string).
>>
>> What does everybody think about this? I think it would be less  
>> disruptive and
>> controversial if some Elisp core API guidelines are decided, written, and
>> followed in the future. Then, if it turns out it’s useful enough, we can  
>> start
>> aliasing functions that don’t follow them to names that do follow them (if
>> it’s desirable to do so.)
>
> I’ve CCed some Elisp users that I think would have some knowledge and  
> opinions
> about this - can people dial in and codify the ‘lisp-way’ or the  
> ‘elisp-way’
> so that we can have some starting point of the API guidelines?
>
> (If someone is opposed to making an API guideline, can somebody explain the
>  reasons too?)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

>> You'll always
>> get a group of people that really expected it to be multibyte-string and
>> group of people that expect string-multibyte.
> If there is an actual naming standard, the people can learn.

There is a naming standard that every Elisp programmer must learn, it's
the use of a namespace prefix to group elements.  Sadly, this standard
is only taken half-seriously for non-core Elisp definitions.

Of course that standard doesn't say on what basis to group.

But currently, if you look at the "string-" and "multibyte-" prefixes,
it's immediately clear that "multibyte-" has not been really used as
a namespace prefix whereas "string-" has been used as a namespace prefix
for several function (for some function it's been used as a namespace
prefix but not all, e.g. `string-rectangle`).

Which is why I propose to rename `multibyte-string-p` to
`string-multibyte-p`, since it would feel at home inside the
"string" group.  Then Elisp programmers would benefit from the
naming convention for that function as well.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 6:23 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:>

>>> You'll always
>>> get a group of people that really expected it to be multibyte-string and
>>> group of people that expect string-multibyte.
> There is a naming standard that every Elisp programmer must learn, it's
> the use of a namespace prefix to group elements.  Sadly, this standard
> is only taken half-seriously for non-core Elisp definitions.

Hmmm. You are implying that "core" definitions are exempted from
this right?  That's why we don't want to have list-car or number-+.
I think we agree there. So if I read you correctly you're also saying
the string type isn't a core thing.  Well, it is, isn't it? Basically I
want string stuff to be in the "safe" group. BUT a good regexp library
that lives somewhere neat and tidy, say a "rex.el", wouldn't bother
me at all. But I want this regexp library to be built as a proper library
not (necessarily) using aliases which might constrain the design.

That said, if you want to alias that one function because you think
it would bring great benefits, fine I guess.  But keep this minimal,
I beg you. Or give flex a go, or substring. What do you use, really,
in your day-to-day? How exactly is "multibyte-string" ruining
your day?

   João "who thinks this renaming has a bit of good ol' OCD in it"

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

>>>> You'll always get a group of people that really expected it to be
>>>> multibyte-string and group of people that expect string-multibyte.
>> There is a naming standard that every Elisp programmer must learn, it's
>> the use of a namespace prefix to group elements.  Sadly, this standard
>> is only taken half-seriously for non-core Elisp definitions.
>
> Hmmm. You are implying that "core" definitions are exempted from
> this right?

No, I mean that we  take this convention half-seriously for non-core
code, and we mostly disregard it for core code.

And all this discussion is about some of us thinking that we *should*
take it quarter-seriously for the some of the core as well.

There can be good reasons to have exceptions, exemptions, etc... but
they should be understood as that.  From this point of view, I think
it's hard to justify an exception for `multibyte-string-p` (the only
argument so far would be that it returns nil rather than signaling an
error when called with a non-string, making it marginally more like
a type-test than about extracting a 1bit-field from a string data
object).

>    João "who thinks this renaming has a bit of good ol' OCD in it"

Could be.  But I think it's also just good practice.
OCD would be to force the use of such namespacing *everywhere* and we're
pretty far from that.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 7:39 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> And all this discussion is about some of us thinking that we *should*
> take it quarter-seriously for the some of the core as well.

Heh, a quarter-serious dose of consistency for a fuzzily defined "core"

Sure that's worth it? Wouldn't you rather work on namespaces? The two
issues intersect!

> There can be good reasons to have exceptions, exemptions, etc... but
> they should be understood as that.  From this point of view, I think
> it's hard to justify an exception for `multibyte-string-p`

Other than: it's there already: it's ingrained in code and programmers
minds.  (And also reads better: tho I would be equally opposed to renaming
it to multibyte if there were a multibyte.el)

> OCD would be to force the use of such namespacing *everywhere* and we're
> pretty far from that.

OK, I trust you (as usual ;-) ) to consider that this aliasing thing has
some negative side-effects such as noise in people's completion
lists (again, not to mention minds). I suppose there will be a double
entry in the corresponding info node, right? Otherwise I could use
my "boost from the manual" hack to fix some of that.

   João "who instantly regrets mentioning the manual cause I just say
unibyte-string there, too"

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 05.05.2020 21:56, João Távora wrote:
>> There can be good reasons to have exceptions, exemptions, etc... but
>> they should be understood as that.  From this point of view, I think
>> it's hard to justify an exception for `multibyte-string-p`
> Other than: it's there already: it's ingrained in code and programmers
> minds.

I seriously doubt it's so ingrained in the minds: like Stefan said in 
the thread it was proposed in, this name is not so easy to recall, and 
one tends to try the `string-multibyte-p' variant first.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 8:01 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> >> There can be good reasons to have exceptions, exemptions, etc... but
> >> they should be understood as that.  From this point of view, I think
> >> it's hard to justify an exception for `multibyte-string-p`
> > Other than: it's there already: it's ingrained in code and programmers
> > minds.
>
> I seriously doubt it's so ingrained in the minds: like Stefan said in
> the thread it was proposed in, this name is not so easy to recall, and
> one tends to try the `string-multibyte-p' variant first.

Time to pull out the ingrainometer :-)

João

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 05.05.2020 22:04, João Távora wrote:
> Time to pull out the ingrainometer:-)

Sure, why not.

Want to poll the users? :-)

Hope you can cover the people outside of the mailing lists too, though.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 8:06 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 05.05.2020 22:04, João Távora wrote:
> > Time to pull out the ingrainometer:-)
> Want to poll the users? :-)

I was hoping you'd have M-x ingrainometer

Oh no, sorry! M-x ingrainometer-meterize-ingrainometrions

João :-)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Heh, a quarter-serious dose of consistency for a fuzzily defined "core"
> Sure that's worth it? Wouldn't you rather work on namespaces?

I expect that adding lexical scoping to Elisp was significantly easier
than adding namespaces will/would be.  And I'm not nearly as often
frustrated by the lack of namespaces as I was by the lack of lexical
scoping (or as I am by the lack of namespacing).  So my estimation of
the ratio gain/pain is not looking good (note that the "pain" factor
there refers to the pain of the designer/implementor rather than the
pain inflected on the rest of Emacs coders).

But maybe someone else has an idea for how to add namespacing much more
easily than what I expect.

> The two issues intersect!

But it could be in a destructive rather than constructive way: having
a separate "ELISP" package would reduce the impetus to impose a good
naming structure within this package, which is what this thread is
all about.

>> There can be good reasons to have exceptions, exemptions, etc... but
>> they should be understood as that.  From this point of view, I think
>> it's hard to justify an exception for `multibyte-string-p`
> Other than: it's there already: it's ingrained in code and programmers
> minds.

I see exactly 1 occurrence of `multibyte-string-p` in GNU ELPA, so
I expect that the majority of Elisp programmers have never used that
function in their code.  There are some renamings I'd like to see happen
which would definitely go against the "ingrained" habits (including
mine), but I don't think `multibyte-string-p` is affected by this aspect.

>    João "who instantly regrets mentioning the manual cause I just say
> unibyte-string there, too"

Yup: `multybyte-string-p` is not even mentioned in the manual.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1458 bytes --]

On Tue, May 5, 2020 at 8:29 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> But it could be in a destructive rather than constructive way: having
> a separate "ELISP" package would reduce the impetus to impose a good
> naming structure within this package, which is what this thread is
> all about.

You can split it up, slowly. Or quickly if you pick a namespacing
technique that is backward compatible with the current naming
scheme. Tom Tromey's idea sounds nice, and it could be
enhanced, IMO.  Let's at least admit that if namespaces were here
right now it would be No Bad Thing. The s.el request that started
this whole thing would be trivially attended to, at least.

> >    João "who instantly regrets mentioning the manual cause I just say
> > unibyte-string there, too"
>
> Yup: `multybyte-string-p` is not even mentioned in the manual.

multybyte-string-p isn't :-) , but multibyte-string-p is:

in (elisp) Text Representations

 -- Function: multibyte-string-p string
     Return ‘t’ if STRING is a multibyte string, ‘nil’ otherwise.  This
     function also returns ‘nil’ if STRING is some object other than a
     string.

But OK. I've already said this one wouldn't bother me. It's just I've
seen big lists (10-20 long) of renames/aliases and those new names
would burden my mental namespace -- and my completion-based
discovery techniques (and Eli's C-h a techniques, to a lesser degree).

[-- Attachment #2: Type: text/html, Size: 2176 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

>> >    João "who instantly regrets mentioning the manual cause I just say
>> > unibyte-string there, too"
>> Yup: `multybyte-string-p` is not even mentioned in the manual.
> multybyte-string-p isn't :-) , but multibyte-string-p is:

Ah, that's what it was ;-)


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Alfred M. Szmidt]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 484 bytes --]

   >    João "who instantly regrets mentioning the manual cause I just say
   > unibyte-string there, too"

   Yup: `multybyte-string-p` is not even mentioned in the manual.

That is true.  But that isn't the same as multibyte-string-p. :-)

(elisp) Text Representations:

 -- Function: multibyte-string-p string
     Return ‘t’ if STRING is a multibyte string, ‘nil’ otherwise.  This
     function also returns ‘nil’ if STRING is some object other than a
     string.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  Philippe Vaucher
>  <philippe.vaucher@gmail.com>,  Drew Adams <drew.adams@oracle.com>,  Eli
>  Zaretskii <eliz@gnu.org>,  Richard Stallman <rms@gnu.org>,
>   조성빈
>  <pcr910303@icloud.com>,  Emacs developers <emacs-devel@gnu.org>
> Date: Tue, 05 May 2020 15:29:40 -0400
> 
> Yup: `multybyte-string-p` is not even mentioned in the manual.

And for a good reason: there's no such function.

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

>> Yup: `multybyte-string-p` is not even mentioned in the manual.
> And for a good reason: there's no such function.

Maybe it's time to rename `multibyte-string-p` to `multybyte-string-p`?


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 05 May 2020 22:44:43 -0400
> Cc: rms@gnu.org, emacs-devel@gnu.org, joaotavora@gmail.com,
>  pcr910303@icloud.com, dgutov@yandex.ru, drew.adams@oracle.com
> 
> >> Yup: `multybyte-string-p` is not even mentioned in the manual.
> > And for a good reason: there's no such function.
> 
> Maybe it's time to rename `multibyte-string-p` to `multybyte-string-p`?

Something to consider seriously, I guess.  For emacs-27, right?

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

>> >> Yup: `multybyte-string-p` is not even mentioned in the manual.
>> > And for a good reason: there's no such function.
>> Maybe it's time to rename `multibyte-string-p` to `multybyte-string-p`?
> Something to consider seriously, I guess.  For emacs-27, right?

OK, I'll prepare the patch for it,


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rms@gnu.org,  emacs-devel@gnu.org,  joaotavora@gmail.com,
>   pcr910303@icloud.com,  dgutov@yandex.ru,  drew.adams@oracle.com
> Date: Wed, 06 May 2020 10:03:47 -0400
> 
> >> >> Yup: `multybyte-string-p` is not even mentioned in the manual.
> >> > And for a good reason: there's no such function.
> >> Maybe it's time to rename `multibyte-string-p` to `multybyte-string-p`?
> > Something to consider seriously, I guess.  For emacs-27, right?
> 
> OK, I'll prepare the patch for it,

Please do, and thanks.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Wed, May 6, 2020 at 3:09 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Stefan Monnier <monnier@iro.umontreal.ca>
> > Cc: rms@gnu.org,  emacs-devel@gnu.org,  joaotavora@gmail.com,
> >   pcr910303@icloud.com,  dgutov@yandex.ru,  drew.adams@oracle.com
> > Date: Wed, 06 May 2020 10:03:47 -0400
> >
> > >> >> Yup: `multybyte-string-p` is not even mentioned in the manual.
> > >> > And for a good reason: there's no such function.
> > >> Maybe it's time to rename `multibyte-string-p` to `multybyte-string-p`?
> > > Something to consider seriously, I guess.  For emacs-27, right?
> > OK, I'll prepare the patch for it,
> Please do, and thanks.

Are we having fun yet?

João

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Please do, and thanks.

I'll fix the other occurrences of "multi" to "multy" while I'm at it.
Should I keep that for `master` or can it go to `emacs-27` as well?


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Alan Mackenzie]

Hello, Stefan and Eli.

On Wed, May 06, 2020 at 11:48:26 -0400, Stefan Monnier wrote:
> > Please do, and thanks.

> I'll fix the other occurrences of "multi" to "multy" while I'm at it.
> Should I keep that for `master` or can it go to `emacs-27` as well?

You're actually being serious here and not having a big joke?

Don't do it.  The correct spelling in English of "multi" is "multi",
regardless of which side of the Atlantic you're sitting in.

"Multiple" is correct, as is "multiplication".  I don't think the
combination "multy" occurs in any English word.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Don't do it.  The correct spelling in English of "multi" is "multi",
> regardless of which side of the Atlantic you're sitting in.

Right, but that's not what we're talking about.  The correct spelling of
"multy" is, well, "multy".  On all pages of the Atlantic, AFAIK (and
even in Der Spiegel, last I checked).


        Stefan

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> > Don't do it.  The correct spelling in English of "multi" is "multi",
> > regardless of which side of the Atlantic you're sitting in.
> 
> Right, but that's not what we're talking about.  The correct spelling
> of "multy" is, well, "multy".  On all pages of the Atlantic, AFAIK (and
> even in Der Spiegel, last I checked).

There's a rumor that there are multyple correct spellings of "multy".

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rms@gnu.org,  emacs-devel@gnu.org,  joaotavora@gmail.com,
>   pcr910303@icloud.com,  dgutov@yandex.ru,  drew.adams@oracle.com
> Date: Wed, 06 May 2020 11:48:26 -0400
> 
> > Please do, and thanks.
> 
> I'll fix the other occurrences of "multi" to "multy" while I'm at it.
> Should I keep that for `master` or can it go to `emacs-27` as well?

Why wait till Emacs 28?

Maybe we should released Emacs 26.4 with these fixes.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

> Sure that's worth it? Wouldn't you rather work on namespaces? The two
> issues intersect!

Out of curiosity, say "real" namespaces land in Emacs, do you reckon
we'd be able to agree on reasonably well-defined topics?

Re: Imports / inclusion of s.el into Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1531 bytes --]

On Tue, May 5, 2020 at 8:59 PM Philippe Vaucher <philippe.vaucher@gmail.com>
wrote:

> > Sure that's worth it? Wouldn't you rather work on namespaces? The two
> > issues intersect!
>
> Out of curiosity, say "real" namespaces land in Emacs, do you reckon
> we'd be able to agree on reasonably well-defined topics?
>

Sure, I think so.  If they were here today, you could have
the s.el library under  the "modern-string" namespace
or "magnar-string" namespace or something like that. I don't
see it'd be contentious. In your library, you could then somehow
indicate you'd like to use the "magnar-string" namespace
and have access to what is now "magnar-string-empty-p"
under just "empty-p". Or maybe you'd prefer to indicate
you want to use the "magnar-string" namespace under
the "s-" local nickname.  Then you can type "s-empty-p"
as you're used to. Same thing with dash.el that so many
people like.

Some parts of what is not yet in a namespace can be
moved to a proper namespace with no breakage.  So
someone working a lot with a properly named lib
today, say, "package.el" could -- locally -- access all
those functions without the "package-" suffix.

You could request completion for symbols in a space
or maybe only "external" symbols in a space.

The problem is rather agree on the form that this
namespace system should take.  There are many ways,
all have advantages and disadvantages. From a purely
cosmetic one to full-featured CL packages. See the other
thread.

-- 
João Távora

[-- Attachment #2: Type: text/html, Size: 2294 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 05.05.2020 23:42, João Távora wrote:
> Sure, I think so.  If they were here today, you could have
> the s.el library under  the "modern-string" namespace
> or "magnar-string" namespace or something like that. I don't
> see it'd be contentious.

What about the "string" namespace?

Re: Imports / inclusion of s.el into Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

On Tue, May 5, 2020 at 10:13 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 05.05.2020 23:42, João Távora wrote:
> > Sure, I think so.  If they were here today, you could have
> > the s.el library under  the "modern-string" namespace
> > or "magnar-string" namespace or something like that. I don't
> > see it'd be contentious.
>
> What about the "string" namespace?
>

It's only a namespace if/when we decide it.  Maybe never.
Until then, its the beginning of a symbol name in the main
nameless space (which is what it has always been -- we
just squint really hard to pretend it's not).

João

[-- Attachment #2: Type: text/html, Size: 1022 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

>> Out of curiosity, say "real" namespaces land in Emacs, do you reckon
>> we'd be able to agree on reasonably well-defined topics?
>
> Sure, I think so.  If they were here today, you could have
> the s.el library under  the "modern-string" namespace
> or "magnar-string" namespace or something like that. I don't
> see it'd be contentious. In your library, you could then somehow
> indicate you'd like to use the "magnar-string" namespace
> and have access to what is now "magnar-string-empty-p"
> under just "empty-p". Or maybe you'd prefer to indicate
> you want to use the "magnar-string" namespace under
> the "s-" local nickname.  Then you can type "s-empty-p"
> as you're used to. Same thing with dash.el that so many
> people like.

Very interesting. To me it looks like we cannot agree on prefixes
already... thus I infer that if namespaces would work, then the
current disagreeing is about "any renames" more than disagreeing with
the "prefix" (namespace) chosen. Putting things in namespace would not
be seen as renames, because they can still get the "untouched" names
experience.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Wed, May 6, 2020 at 10:20 AM Philippe Vaucher
<philippe.vaucher@gmail.com> wrote:
>
> >> Out of curiosity, say "real" namespaces land in Emacs, do you reckon
> >> we'd be able to agree on reasonably well-defined topics?
> >
> > Sure, I think so.  If they were here today, you could have
> > the s.el library under  the "modern-string" namespace
> > or "magnar-string" namespace or something like that. I don't
> > see it'd be contentious. In your library, you could then somehow
> > indicate you'd like to use the "magnar-string" namespace
> > and have access to what is now "magnar-string-empty-p"
> > under just "empty-p". Or maybe you'd prefer to indicate
> > you want to use the "magnar-string" namespace under
> > the "s-" local nickname.  Then you can type "s-empty-p"
> > as you're used to. Same thing with dash.el that so many
> > people like.
>
> Very interesting. To me it looks like we cannot agree on prefixes
> already... thus I infer that if namespaces would work, then the
> current disagreeing is about "any renames" more than disagreeing with
> the "prefix" (namespace) chosen. Putting things in namespace would not
> be seen as renames, because they can still get the "untouched" names
> experience.

Yes, I think so.  Today we can agree on a prefix by
choosing a long or unique enough name that everyone agrees
won't impact them.  That happens all the time.  But it's clear that
s.el and dash.el want to be (very) short-prefixed, by their very
nature, otherwise they lose their charm.  So namespaces are
a way to get that opt-in, instead of wholesale or indiscriminately.

My problem with renames or aliases  on "core" symbols
is like Richard's, I think.  It burdens my mental model and it
burdens my discovery techniques, which include C-h f and C-h a. So
my logic is: if we're going to do that in the name of a good and
discoverable library that also wants to be readily accessible,
we might as well work on ways to get to that objective in ways
that don't have those drawbacks _and_ kill the possibility
of that library.

My personal opinion on the s.el library, which isn't very positive,
doesn't matter here. It should not be excluded because of a
namespacing issue.

João

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> it's clear that s.el and dash.el want to be (very)
> short-prefixed, by their very nature, otherwise
> they lose their charm.

What is that "very nature"?  Something specific
about them?

Wouldn't a short prefix bestow the same charm
on any library?

It's a serious question, not rhetorical or
whimsical.  Is there something special about
these libraries that calls for, or fits
particularly well, with a short prefix?

Or is it just that they're already out there
with short prefixes (and the charm they bestow)?

Re: Imports / inclusion of s.el into Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 805 bytes --]

On Wed, May 6, 2020, 22:42 Drew Adams <drew.adams@oracle.com> wrote:

> > it's clear that s.el and dash.el want to be (very)
> > short-prefixed, by their very nature, otherwise
> > they lose their charm.
>
> What is that "very nature"?  Something specific
> about them?
>
> Wouldn't a short prefix bestow the same charm
> on any library?
>
> It's a serious question, not rhetorical or
> whimsical.  Is there something special about
> these libraries that calls for, or fits
> particularly well, with a short prefix?
>
> Or is it just that they're already out there
> with short prefixes (and the charm they bestow)?
>

Yep, you're right. A short prefix would be good for any library. It's just
that these two were bold enough to try it and (not yet) get away with it.

João

>

[-- Attachment #2: Type: text/html, Size: 1405 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > No, I mean that we  take this convention half-seriously for non-core
  > code, and we mostly disregard it for core code.

This seems like a clash of different assumptions.

What we have is a convention for names, in non-core packages and
users's packages, which is to include the name of the package (perhaps
in shortened form) in the functions and variables it defines.

multibyte-string-p is not an exception.  It contains the word string,
so it follows this convention.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> > Don't forget that completion can now use substring
> > and other kinds of matching.
> 
> You understand that substring completion fails as soon as the term is
> too generic? e.g "string", "regexp" or "list", the list is just a lot
> of noise.

No, I don't "understand" that.  I mean I don't agree.

My position is that there is no one-size-fits-all
answer wrt completion.  Completion can serve many,
many, many different purposes.  And different
matching methods offer different advantages for
different purposes.

I use substring matching (regexp actually, but
often the pattern is, in effect, a substring)
very, very often.  And I can change among
different match methods instantly - nothing ties
me down to using substring matching, at all.

The domain (search space) for any given completion
is a set of completion candidates.  This set is
defined before you input anything.  The definition
can come from anything/anywhere, in any way.

For completion of buffer text (e.g. code) it
typically comes from text immediately before point.
For a help command (e.g. `C-h f') it might come
from the symbols that fit certain criteria (e.g.
have a function definition).

So already there is "matching" in some sense.

`completing-read', for example, can define a set
of candidates using a PREDICATE argument.  All
built-in functions that accept exactly one arg?
This predicate should do that job:

(defun unary-subr-p (f)
  (let* ((fun  (symbol-function f))
         (ar   (and fun
                    (ignore-errors (subr-arity fun)))))
    (and ar  (eql 1 (car ar))  (eql 1 (cdr ar)))))

(completing-read "Unary subr: " obarray 'unary-subr-p)

A particular set of candidates is defined, and you
can use completion just to examine that set, if
you want.  (There are 345 of them in my session.)

And if you have an interesting completion UI you
can perhaps selectively act on one or more of
those candidates.  You can perhaps show some info
(e.g. help/definition) about specific candidates.

E.g., perhaps cycling among function-name
candidates shows you the first line of their doc
strings in the mode-line of buffer `*Completions',
one by one.  Or maybe it shows you their complete
doc strings in `*Help*'.  Or if the candidates
are file names, maybe it shows you info about the
individual files as you cycle among them.

Or perhaps your UI lets you act on the set as a
whole in some way, e.g. print it, check out all
of the files, ... whatever.

This is all still without any matching of input
that you provide.  Completion is about using a
_set_ of things (names, but behind names, things
they name).

Then, you can of course provide input to _match_
against those domain candidates, to narrow the
field, giving a new, smaller set to play with.

But you can do the same kinds of things with
that new set: examine members, get info about
them, maybe act on them, individually or as a
set, in specific or arbitrary ways.

Provide a regexp or substring pattern to match,
et voila!  You've got "apropos" filtering of
candidate names.  And if your completion UI
gives you help/info about individual candidates
then you get the "apropos" doc as well.

Some completion UIs let you can perform
general set operations: intersect the current
set of matches with one or more other candidate
sets, get the union of sets, complement (e.g.
wrt the domain or a previous filtering or a
previously saved candidates set),...

And yes,...(drum roll) typically you can also
just choose one element of the set, act on it
in some way, and exit completion.  A quick
in-&-out-burger.

That last action is what many people think of
as completion - nothing more.  And if that's
their only goal then they can easily think that
what's most important is always only speed of
action, small number of choices, etc.

IMO, that's just one small, but important, use
case for completion.

In sum, yes, I might want to see all of the
functions with `string' in their name.  And
seeing them, I might want to get more info
about some of them.  Or act on some of them
in other ways.  No matter how many there are.

Heck, I might even just want to see how many
functions have `string' or `regexp' in their
name - e.g., to consider a question about
possible renamings... ;-)

In my Emacs session right now, I use `C-h f',
type `string' and `S-TAB', to (1) see all
that have `string' as substring and (2) see
that there are 459 of them.  And plain `TAB'
tells me there are 47 with `string' as prefix,
and matching `string-' instead of `string'
tells me there are 42 with that prefix (it
filters out candidates `string', `string<',
`string=', `string>', and `stringp').

In general, it's the completion _process_
that's important.  And that can serve several
goals.

Instead of focusing on just the goal of
choosing one candidate, consider the overall
process:

There's a set (domain) of possible candidates,
which you can you filter with your input.
You can do anything you want with a set of
candidates or any of its members, at any point
in the process.

> Also it doesn't quite work when the order is
> "reversed", e.g C-h f "string TAB multibyte"
> would not return the desired function
> because it has to be "multibyte TAB string".

For orderless matching I match the 2 patterns
separately.  With my UI that's `string S-SPC
multibyte' or `multibyte S-SPC string' - same
result: all matches, any order.

And of course all I really need to type are
some chars from those patterns. And I can
limit matches to candidates that start or
end with one of those patterns, not just
candidates that contain them.

Is what I'm saying limited to having a UI
that lets me do such non-vanilla stuff?  No.
With vanilla Emacs, out of the box, you can
use completion to _examine_ a set of matches.
That can tell you something.  It's not only
about choosing a candidate and acting on it
in only one way.  Completion is about sets.
___

https://www.emacswiki.org/emacs/Icicles_-_Nutshell_View#completionprocessandgoals

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > You understand that substring completion fails as soon as the term is
  > too generic? e.g "string", "regexp" or "list", the list is just a lot
  > of noise.

How about implementing the kind of completion (or, more generally,
function discovery) that you would like to use?  People can
tweak each others' attempts and produce something that a lot of
users would like.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1641 bytes --]

Hello,

Following our discussion about namespace I think it's time to start with
something concrete where most people are agreeing already.

I started an issue at https://github.com/magnars/s.el/issues/136 asking the
author how he feels about inclusion/importing into Emacs.

Past experiences showed me that inclusion of such public library is always
complicated due to copyright assignments, but I believe we can skip that
painful part as explained later on.

I propose the following plan:

   1.  Check wether there are existing emacs `*-string-* functions that'd
   be aliased to start with `string-` and make a patch for that. That is,
   already unify the existing Emacs string library if needed.
   2. Decide of a set of function that we'd "import" from s.el and
   namespace them under the `string-` namespace. For example, `s-prepend`
   would become `string-prepend`. Maybe that can be the whole API of s.el but
   probably that some will object to that while agreeing on a smaller set.
   3. Decide how we import these, either by copyright assignments or by
   importing the function signature and "simply" reimplementing these. Their
   implementation is usually trivial.

On a side note, I'm curious about 3: given the code is already GPL v3, are
we allowed to simply implement the Emacs string-* function using s.el? or
copy-paste from it? WARNING I know this is a very touchy subject and my
goal is not to offend anyone, especially not the author of this wonderful
library, I am just genuinely curious of what is allowed and what is not in
this case. I'd not like to do that without the author consent.

Kind regards,
Philippe

[-- Attachment #2: Type: text/html, Size: 1910 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Fri, 1 May 2020 16:56:13 +0200
> 
> 3 Decide how we import these, either by copyright assignments or by importing the function signature and
>  "simply" reimplementing these. Their implementation is usually trivial.
> 
> On a side note, I'm curious about 3: given the code is already GPL v3, are we allowed to simply implement
> the Emacs string-* function using s.el? or copy-paste from it? WARNING I know this is a very touchy
> subject and my goal is not to offend anyone, especially not the author of this wonderful library, I am just
> genuinely curious of what is allowed and what is not in this case. I'd not like to do that without the author
> consent.

If you copy someone's code, the code is still theirs, and we need both
a permission from them to use it and a copyright assignment to include
that in Emacs.

If we are thinking about reimplementing those function, then the
person who is reimplementing is well advised not to look at all at the
original code, but just work from requirements or from an idea of the
implementation described by someone else.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1092 bytes --]

>
> > On a side note, I'm curious about 3: given the code is already GPL v3,
> are we allowed to simply implement
> > the Emacs string-* function using s.el? or copy-paste from it? WARNING I
> know this is a very touchy
> > subject and my goal is not to offend anyone, especially not the author
> of this wonderful library, I am just
> > genuinely curious of what is allowed and what is not in this case. I'd
> not like to do that without the author
> > consent.
>
> If you copy someone's code, the code is still theirs, and we need both
> a permission from them to use it and a copyright assignment to include
> that in Emacs.
>
> If we are thinking about reimplementing those function, then the
> person who is reimplementing is well advised not to look at all at the
> original code, but just work from requirements or from an idea of the
> implementation described by someone else.
>

Got it. And implementing those `string-*` functions using `s.el` is not
possible, because that's require `s.el` being distributed with Emacs which
would require copyright assignments, correct?

Philippe

[-- Attachment #2: Type: text/html, Size: 1391 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Fri, 1 May 2020 17:56:14 +0200
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
>  If you copy someone's code, the code is still theirs, and we need both
>  a permission from them to use it and a copyright assignment to include
>  that in Emacs.
> 
>  If we are thinking about reimplementing those function, then the
>  person who is reimplementing is well advised not to look at all at the
>  original code, but just work from requirements or from an idea of the
>  implementation described by someone else.
> 
> Got it. And implementing those `string-*` functions using `s.el` is not possible, because that's require `s.el`
> being distributed with Emacs which would require copyright assignments, correct?

Yes.

Re: Imports / inclusion of s.el into Emacs

[Stefan Kangas]

Philippe Vaucher <philippe.vaucher@gmail.com> writes:

> Decide how we import these, either by copyright assignments or by importing the function signature and "simply" reimplementing these. Their implementation is usually trivial.

How many people have contributed to s.el?  And how many lines of code
is it?  I haven't looked at the code.

Best regards,
Stefan Kangas

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 949 bytes --]

>
> > Decide how we import these, either by copyright assignments or by
> importing the function signature and "simply" reimplementing these. Their
> implementation is usually trivial.
>
> How many people have contributed to s.el?  And how many lines of code
> is it?  I haven't looked at the code.


31 contributors, but > 90% of the commits is done by the author (
https://github.com/magnars/s.el/graphs/contributors).

I have good news, basically he's 100% behind the idea, is already on the
assignment file and gave us permission to use his code wholesale.

Some other contributors (if not most) also already gave their assignments,
so I expect them to be ok as well.

I think we can move on with my proposal. You can look at
https://github.com/magnars/s.el/blame/master/s.el and see which line was
written by who. As you can see there's no other dependency than Emacs.

Before we get going, is there something else I should wait for?

Philippe

[-- Attachment #2: Type: text/html, Size: 1442 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 01.05.2020 17:56, Philippe Vaucher wrote:
> Following our discussion about namespace I think it's time to start with 
> something concrete where most people are agreeing already

FWIW, I made a couple of other suggestions.

As far as s.el goes, I suppose it might be a good thing to add it to 
ELPA for those who like it.

I'm not completely sold on its contents for the core, however: a lot of 
it looks like a compatibility layer for Clojure's familiarity's sake, 
with very thin wrappers (which basically just add the cost of function 
invocation).

Examples:

1. s-prepend/s-append: trivially replaced by 'concat'.

2. s-trim: string-trim is already in subr-x.

3. s-split: basically delegates to split-string, but wraps it in 
save-match-data (which is generally against our guidelines for its use).

Re: Imports / inclusion of s.el into Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1122 bytes --]

Just wanted to voice my support for exactly the points that Dmitry makes.

And if we do add it to Elpa, can we give it a slightly longer prefix? Like
'cljstr`?

Thanks,
João


On Fri, May 1, 2020, 18:21 Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 01.05.2020 17:56, Philippe Vaucher wrote:
> > Following our discussion about namespace I think it's time to start with
> > something concrete where most people are agreeing already
>
> FWIW, I made a couple of other suggestions.
>
> As far as s.el goes, I suppose it might be a good thing to add it to
> ELPA for those who like it.
>
> I'm not completely sold on its contents for the core, however: a lot of
> it looks like a compatibility layer for Clojure's familiarity's sake,
> with very thin wrappers (which basically just add the cost of function
> invocation).
>
> Examples:
>
> 1. s-prepend/s-append: trivially replaced by 'concat'.
>
> 2. s-trim: string-trim is already in subr-x.
>
> 3. s-split: basically delegates to split-string, but wraps it in
> save-match-data (which is generally against our guidelines for its use).
>
>

[-- Attachment #2: Type: text/html, Size: 1602 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Just wanted to voice my support for exactly the points that Dmitry makes.
> And if we do add it to Elpa, can we give it a slightly longer prefix? Like
> 'cljstr`?

I don't think renaming it will fly.  We should definitely add it to GNU
ELPA because it's used by many other Elisp packages.  But renaming would
be self-defeating some all those packages won't want to go through
the renaming.

But for Emacs itself, we could probably rename some existing
functions to use the `string-` prefix and maybe incorporate some ideas
from s.el (I haven't looked very much into it, so I have no idea how
much there is to incorporate, but I assume there are some good ideas).


        Stefan


> On Fri, May 1, 2020, 18:21 Dmitry Gutov <dgutov@yandex.ru> wrote:
>
>> On 01.05.2020 17:56, Philippe Vaucher wrote:
>> > Following our discussion about namespace I think it's time to start with
>> > something concrete where most people are agreeing already
>>
>> FWIW, I made a couple of other suggestions.
>>
>> As far as s.el goes, I suppose it might be a good thing to add it to
>> ELPA for those who like it.
>>
>> I'm not completely sold on its contents for the core, however: a lot of
>> it looks like a compatibility layer for Clojure's familiarity's sake,
>> with very thin wrappers (which basically just add the cost of function
>> invocation).
>>
>> Examples:
>>
>> 1. s-prepend/s-append: trivially replaced by 'concat'.
>>
>> 2. s-trim: string-trim is already in subr-x.
>>
>> 3. s-split: basically delegates to split-string, but wraps it in
>> save-match-data (which is generally against our guidelines for its use).
>>
>>

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 01.05.2020 21:09, Stefan Monnier wrote:
> we could probably rename some existing
> functions to use the `string-` prefix

We have both 'match-string' and 'string-match', which do different 
things. Any particular ideas for renaming?

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 288 bytes --]

>
> > we could probably rename some existing
> > functions to use the `string-` prefix
>
> We have both 'match-string' and 'string-match', which do different
> things. Any particular ideas for renaming?
>

`match-string` could be aliased to `string-match-result` or
`string-match-value`.

[-- Attachment #2: Type: text/html, Size: 537 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 01.05.2020 21:19, Philippe Vaucher wrote:
> `match-string` could be aliased to `string-match-result` or 
> `string-match-value`.

Wouldn't 'string-match' match any of these names better?

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 449 bytes --]

On Fri, May 1, 2020 at 8:30 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 01.05.2020 21:19, Philippe Vaucher wrote:
> > `match-string` could be aliased to `string-match-result` or
> > `string-match-value`.
>
> Wouldn't 'string-match' match any of these names better?
>

But `string-match` is already taken?!

string-match: apply regexp (function untouched)
string-match-value: extract value from previous match (instead of current
`match-string`)

[-- Attachment #2: Type: text/html, Size: 842 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Joost Kremers]

On Fri, May 01 2020, Philippe Vaucher wrote:
> On Fri, May 1, 2020 at 8:30 PM Dmitry Gutov <dgutov@yandex.ru> 
> wrote:
>
>> On 01.05.2020 21:19, Philippe Vaucher wrote:
>> > `match-string` could be aliased to `string-match-result` or
>> > `string-match-value`.
>>
>> Wouldn't 'string-match' match any of these names better?
>>
>
> But `string-match` is already taken?!
>
> string-match: apply regexp (function untouched)
> string-match-value: extract value from previous match (instead 
> of current
> `match-string`)

string-match and string-matched?

-- 
Joost Kremers
Life has its moments

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

>> we could probably rename some existing
>> functions to use the `string-` prefix
> We have both 'match-string' and 'string-match', which do different
> things. Any particular ideas for renaming?

I'm not sure `match-string` qualifies as
a string-manipulating function.  Maybe it belongs in a "regexp-matching"
category instead?

In any case, if there's no good choice, we should it alone: we don't
have to rename everything.


        Stefan "puts on his asbestos suit and then blurts out
                «I actually like Scheme's `->` so I'd suggest
                `match->string»"

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 555 bytes --]

> >> we could probably rename some existing
> >> functions to use the `string-` prefix
> > We have both 'match-string' and 'string-match', which do different
> > things. Any particular ideas for renaming?
>
> I'm not sure `match-string` qualifies as
> a string-manipulating function.  Maybe it belongs in a "regexp-matching"
> category instead?


I agree. What would be the most logical is to move the regexp related
functions to the regexp- namespace no?

string-match would be aliased as regexp-match, match-string as
regexp-match-value, etc.

Philippe

[-- Attachment #2: Type: text/html, Size: 909 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 01.05.2020 21:32, Stefan Monnier wrote:
> I'm not sure `match-string` qualifies as
> a string-manipulating function.  Maybe it belongs in a "regexp-matching"
> category instead?

Um. That's a good point.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'm not sure `match-string` qualifies as
  > a string-manipulating function.  Maybe it belongs in a "regexp-matching"
  > category instead?

match-string never directly operates on a regexp.  It returns the string
which was matched by a match or search operation.

The match or search operation typically matches or searches for a regexp.
However, (match-string 0) can be useful after a non-regexp search.

So, does it belong in the "regexp" category?  Perhaps 90% yes and 10% no.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Yuan Fu]

> On May 1, 2020, at 2:16 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
> On 01.05.2020 21:09, Stefan Monnier wrote:
>> we could probably rename some existing
>> functions to use the `string-` prefix
> 
> We have both 'match-string' and 'string-match', which do different things. Any particular ideas for renaming?
> 

That reminded me that regex-related functions are not very well-named, namely

re-search-forward
re-search-backward
replace-regexp-in-string
replace-match
string-match
match-string

These are what I can come think of on top of my head. String-match is especially bad. Every times I search with string with re/regexp and just couldn’t find the function that matches regexp in string… I suggest give every re-related functions a re- prefix. 

Yuan

Re: Imports / inclusion of s.el into Emacs

[Yuan Fu]

> On May 1, 2020, at 6:53 PM, Yuan Fu <casouri@gmail.com> wrote:
> 
> 
> 
>> On May 1, 2020, at 2:16 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
>> 
>> On 01.05.2020 21:09, Stefan Monnier wrote:
>>> we could probably rename some existing
>>> functions to use the `string-` prefix
>> 
>> We have both 'match-string' and 'string-match', which do different things. Any particular ideas for renaming?
>> 
> 
> That reminded me that regex-related functions are not very well-named, namely
> 
> re-search-forward
> re-search-backward
> replace-regexp-in-string
> replace-match
> string-match
> match-string
> 
> These are what I can come think of on top of my head. String-match is especially bad. Every times I search with string with re/regexp and just couldn’t find the function that matches regexp in string… I suggest give every re-related functions a re- prefix. 
> 
> Yuan

What about

re-search-forward		re-search-forward
re-search-backward		re-search-backward
replace-regexp-in-string	re-replace-in-string
replace-match			re-replace-match
string-match			re-search-in-string
match-string			re-matched-string

Or something along that line?

Yuan

Rename regex functions to use prefix re-

[Stefan Kangas]

Yuan Fu <casouri@gmail.com> writes:

> What about
>
> re-search-forward               re-search-forward
> re-search-backward              re-search-backward
> replace-regexp-in-string        re-replace-in-string
> replace-match                   re-replace-match
> string-match                    re-search-in-string
> match-string                    re-matched-string
>
> Or something along that line?

We could think about the exact renames, but already the above looks
like a big improvement to me.  I would definitely support regularizing
the regex functions to use the "re-" prefix.

Best regards,
Stefan Kangas

Re: Rename regex functions to use prefix re-

[Phillip Lord]

Stefan Kangas <stefan@marxist.se> writes:

> Yuan Fu <casouri@gmail.com> writes:
>
>> What about
>>
>> re-search-forward               re-search-forward
>> re-search-backward              re-search-backward
>> replace-regexp-in-string        re-replace-in-string
>> replace-match                   re-replace-match
>> string-match                    re-search-in-string
>> match-string                    re-matched-string
>>
>> Or something along that line?
>
> We could think about the exact renames, but already the above looks
> like a big improvement to me.  I would definitely support regularizing
> the regex functions to use the "re-" prefix.


Worth looking at Wilfred Hughes re.el. Or my own "m-buffer" which
obviates the need for re-search-forward and replace-match in many
circumstances.

There are a pile of these different libraries, with clean,
well-thoughtout and regular APIs. They are a big improvement over Emacs
organically grown set.

Phil

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 518 bytes --]

>
> What about
>
> re-search-forward               re-search-forward
> re-search-backward              re-search-backward
> replace-regexp-in-string        re-replace-in-string
> replace-match                   re-replace-match
> string-match                    re-search-in-string
> match-string                    re-matched-string
>

This sounds good, but I think `regexp-` is a better namespace given there
is `regexp-quote`, `regexp-opt` etc. That'd also probably mean alias
`replace-regexp` as `regexp-replace`.

[-- Attachment #2: Type: text/html, Size: 806 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Stefan Kangas]

Philippe Vaucher <philippe.vaucher@gmail.com> writes:

> This sounds good, but I think `regexp-` is a better namespace given there is `regexp-quote`, `regexp-opt` etc. That'd also probably mean alias `replace-regexp` as `regexp-replace`.

FWIW, I think the regex case is important enough to warrant a short
prefix, to save us some typing.  (I count six functions using the
"re-" prefix and five using the "regexp-" prefix.)

Best regards,
Stefan Kangas

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > re-search-forward		re-search-forward
  > re-search-backward		re-search-backward
  > replace-regexp-in-string	re-replace-in-string
  > replace-match			re-replace-match
  > string-match			re-search-in-string
  > match-string			re-matched-string

I think this is mostly good.  However, string-match does not search
the string.  It tries to match only at the start of the string.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> I think this is mostly good.  However, string-match does not search
> the string.  It tries to match only at the start of the string.

No, it searches.

    M-: (and (string-match "a." "Richard") (match-string 0 "Richard"))
    "ar"


-- Stefan

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 636 bytes --]

>   > re-search-forward           re-search-forward
>   > re-search-backward          re-search-backward
>   > replace-regexp-in-string    re-replace-in-string
>   > replace-match                       re-replace-match
>   > string-match                        re-search-in-string
>   > match-string                        re-matched-string
>
> I think this is mostly good.  However, string-match does not search
> the string.  It tries to match only at the start of the string.

You said in a later mail that adding such aliases would pollute the
namespace. I infer that some aliases would be ok but many aliases not, is
that correct?

[-- Attachment #2: Type: text/html, Size: 804 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > You said in a later mail that adding such aliases would pollute the
  > namespace. I infer that some aliases would be ok but many aliases not, is
  > that correct?

I don't exactly remember my words, but I think I said that the s.el aliases
would pollute the name space.

These renamings for regexp-related functions might be ok as a renaming:
a plan to obsolete the current names, perhaps 5 or 10 years from now,
and switch to the new cleaner names.

That is why I am working on the question of what renamings to do.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Oops, I misremembered.  You were right about string-match.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Fri, May 1, 2020 at 7:09 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> > Just wanted to voice my support for exactly the points that Dmitry makes.
> > And if we do add it to Elpa, can we give it a slightly longer prefix? Like
> > 'cljstr`?
>
> I don't think renaming it will fly.  We should definitely add it to GNU
> ELPA because it's used by many other Elisp packages.  But renaming would
> be self-defeating some all those packages won't want to go through
> the renaming.

Well none of those users of s.el are in GNU Elpa, right? They could
trivially rename to use cjstring.el, or string.el.

To be clear, I don't oppose the inclusion of those functions/macros
especially if they really are advantageous (but are they really?).  What
I really don't like about this and the dash.el library is that it takes up
such a prominent place in the shared namespace, because of the
very short  prefix.  Perhaps this argument is a bit reactionary, but then
again s.el seems a bit /arriviste/ in that regard (likely unintentionally).
If we had a package system, ideally one with local nicknames,
this would all be moot, I think.

João

Re: Imports / inclusion of s.el into Emacs

[Stefan Kangas]

João Távora <joaotavora@gmail.com> writes:

> Well none of those users of s.el are in GNU Elpa, right? They could
> trivially rename to use cjstring.el, or string.el.

I suppose they're all in MELPA, but there are a lot of them.
See: https://melpa.org/#/s

Best regards,
Stefan Kangas

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sat, May 2, 2020 at 12:33 AM Stefan Kangas <stefan@marxist.se> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> > Well none of those users of s.el are in GNU Elpa, right? They could
> > trivially rename to use cjstring.el, or string.el.
>
> I suppose they're all in MELPA, but there are a lot of them.
> See: https://melpa.org/#/s

Yes.  As they join the GNU Elpa flock, we cleanse them of the evil, brother!

João

Re: Imports / inclusion of s.el into Emacs

[Stefan Kangas]

João Távora <joaotavora@gmail.com> writes:

> Yes.  As they join the GNU Elpa flock, we cleanse them of the evil, brother!

;-)

Personally, I'm not convinced there's a strong need to rename it.  But
I think it's important that any rename would be agreed upon by the
current module author.  The worst possible situation would be to have
a "better" name but a maintainer that feels alienated.

It's my impression that we've had a number of cases where we
successfully got a package into GNU ELPA, only to have it fall out of
sync with development within a couple of years.  We've recently
discussed that our copy dash.el is out of sync with recent
development, and we have coffee-mode.el before that.  Stefan Monnier
said in relation to this that: "Other work to do is to make sure the
GNU ELPA packages don't become stale.  E.g. `dash` is out of date,
IIRC."

Is there something we could do to avoid such a situation before it
arises?  To my mind, we really need package authors to push
(figuratively speaking) their changes to us, rather than us having to
pull their changes.  We seem to lack the necessary manpower to
constantly "chase down" package authors or do the work to merge
changes ourselves on a consistent basis.

And as we are working to get even more packages into GNU ELPA, which I
think we should, this problem will only tend to increase over time.

Best regards,
Stefan Kangas

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   We've recently
  > discussed that our copy dash.el is out of sync with recent
  > development, and we have coffee-mode.el before that.

Would someone with good social skills like to ask the developers of
dash to move their development into GNU ELPA, or ask why they have not
done so?  I am sure they can see the advantage of having their latest
version available through Emacs.

  > Is there something we could do to avoid such a situation before it
  > arises?  To my mind, we really need package authors to push
  > (figuratively speaking) their changes to us, rather than us having to
  > pull their changes.

Could we make explicit arrangements with the package developers, about
continued maintainance of the package within GNU ELPA, as part of
arranging to include the package in GNU ELPA?  Do we already do this?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Kangas]

Richard Stallman <rms@gnu.org> writes:

>   >   We've recently
>   > discussed that our copy dash.el is out of sync with recent
>   > development, and we have coffee-mode.el before that.
>
> Would someone with good social skills like to ask the developers of
> dash to move their development into GNU ELPA, or ask why they have not
> done so?  I am sure they can see the advantage of having their latest
> version available through Emacs.

It seems no one has volunteered yet.

But I'm not sure what exactly is involved here.  All I can see in
elpa.git is that there is a pointer in externals-list to the dash
repository?  So why isn't it updated exactly?

Sorry, I don't know how this is all supposed to work.

Hmm, looking closer.  Is it enough for the package maintainer to just
push the latest changes to the 'externals/dash' branch in addition to
the original repository?

>   > Is there something we could do to avoid such a situation before it
>   > arises?  To my mind, we really need package authors to push
>   > (figuratively speaking) their changes to us, rather than us having to
>   > pull their changes.
>
> Could we make explicit arrangements with the package developers, about
> continued maintainance of the package within GNU ELPA, as part of
> arranging to include the package in GNU ELPA?  Do we already do this?

Does anyone know the answer to this?

Best regards,
Stefan Kangas

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 06.05.2020 01:35, Stefan Kangas wrote:
> Hmm, looking closer.  Is it enough for the package maintainer to just
> push the latest changes to the 'externals/dash' branch in addition to
> the original repository?

Yep.

>> Could we make explicit arrangements with the package developers, about
>> continued maintainance of the package within GNU ELPA, as part of
>> arranging to include the package in GNU ELPA?  Do we already do this?
> Does anyone know the answer to this?

I suppose we already do, but then some developers stop (because of some 
inconveniences, I suppose?), or do it very sporadically.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

> >   >   We've recently
> >   > discussed that our copy dash.el is out of sync with recent
> >   > development, and we have coffee-mode.el before that.
> >
> > Would someone with good social skills like to ask the developers of
> > dash to move their development into GNU ELPA, or ask why they have not
> > done so?  I am sure they can see the advantage of having their latest
> > version available through Emacs.
>
> It seems no one has volunteered yet.

Not true anymore! \o/

I just spoke with the author of s.el and I'll quote him directly:

"I would be happy to add s.el to ELPA as is, especially if it does not
include any (or very little) work on my part. Let me know what is
needed."

I think the author does not know what is required to add s.el in ELPA,
and thus dash.el was added by someone else, maybe someone knows about
it?
Please tell me the required steps and hopefully it's not too much work for him.


> But I'm not sure what exactly is involved here.  All I can see in
> elpa.git is that there is a pointer in externals-list to the dash
> repository?  So why isn't it updated exactly?

If that's the case then I have the same question. It'd be
best/simplest if ELPA just mirrors the github repository.

> Hmm, looking closer.  Is it enough for the package maintainer to just
> push the latest changes to the 'externals/dash' branch in addition to
> the original repository?

If so I think it's reasonable. Out of curiosity can we also do this
with the master branch?

Re: Imports / inclusion of s.el into Emacs

[Phillip Lord]

Philippe Vaucher <philippe.vaucher@gmail.com> writes:

>> >   >   We've recently
>> >   > discussed that our copy dash.el is out of sync with recent
>> >   > development, and we have coffee-mode.el before that.
>> >
>> > Would someone with good social skills like to ask the developers of
>> > dash to move their development into GNU ELPA, or ask why they have not
>> > done so?  I am sure they can see the advantage of having their latest
>> > version available through Emacs.
>>
>> It seems no one has volunteered yet.
>
> Not true anymore! \o/
>
> I just spoke with the author of s.el and I'll quote him directly:
>
> "I would be happy to add s.el to ELPA as is, especially if it does not
> include any (or very little) work on my part. Let me know what is
> needed."
>
> I think the author does not know what is required to add s.el in ELPA,
> and thus dash.el was added by someone else, maybe someone knows about
> it?
> Please tell me the required steps and hopefully it's not too much work for him.


It was added by me.

Essentially, you add something to the readme saying "copyright
assignment", update the header of "s.el" to say "this is part of
Emacs". Then you push the github master branch to a new branch
externals/s on ELPA, then update externals-list which on master.

IIRC, ELPA will sit on this till there is a version bump, as described
in the header, so you would need to do that as well. Then wait, because
ELPA builds every 24 hrs I think.

>
>
>> But I'm not sure what exactly is involved here.  All I can see in
>> elpa.git is that there is a pointer in externals-list to the dash
>> repository?  So why isn't it updated exactly?
>
> If that's the case then I have the same question. It'd be
> best/simplest if ELPA just mirrors the github repository.


That link is for information. To get later versions up you need to pull
from github and push to the ELPA branch.



>> Hmm, looking closer.  Is it enough for the package maintainer to just
>> push the latest changes to the 'externals/dash' branch in addition to
>> the original repository?
>
> If so I think it's reasonable. Out of curiosity can we also do this
> with the master branch?

And version bump!

Phil

Re: Imports / inclusion of s.el into Emacs

[Stefan Kangas]

Phillip Lord <phillip.lord@russet.org.uk> writes:

>>> Hmm, looking closer.  Is it enough for the package maintainer to just
>>> push the latest changes to the 'externals/dash' branch in addition to
>>> the original repository?
>>
>> If so I think it's reasonable. Out of curiosity can we also do this
>> with the master branch?
>
> And version bump!

Maybe someone should have a look at the elpa.git README to see if
there's anything missing, or if this could be made even clearer.  I'd
volunteer, but I'm not sure I understand the full process.

It would also be good if all this information was easily accessible
from the https://elpa.gnu.org website.  I know there's a link to the
README, but that fails the "easily accessible" criterion, I think.

Best regards,
Stefan Kangas

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Essentially, you add something to the readme saying "copyright
> assignment",

That's good.

> update the header of "s.el" to say "this is part of Emacs".

This is not necessary (GNU ELPA packages are neither part nor not part
of Emacs.  They live in a happy in-between).  The better option is to
remove the "this is (not) part of" altogether.

> Then you push the github master branch to a new branch
> externals/s on ELPA, then update externals-list which on master.

That's about right.  Tho you first need to check whether all the
contributors of non-trivial amounts of code&doc have signed the paperwork.

> IIRC, ELPA will sit on this till there is a version bump, as described
> in the header, so you would need to do that as well.

The appearance of the package counts as a version bump, so there's no
need to change the "Version:" before the first package is built.

> Then wait, because ELPA builds every 24 hrs I think.

Actually, it should be every 12h, usually.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 02:36, João Távora wrote:
> Yes.  As they join the GNU Elpa flock, we cleanse them of the evil, brother!

And then wait for 5-10 years for even 80% of all users to update?

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > > Well none of those users of s.el are in GNU Elpa, right? They could
  > > > trivially rename to use cjstring.el, or string.el.
  > >
  > > I suppose they're all in MELPA, but there are a lot of them.
  > > See: https://melpa.org/#/s

If packages not in Emacs use s.el which is not in Emacs, it is not
directly an issue for us.  However, it could become an issue for us if
we want to import those packages into Emacs or GNU ELPA.  We can't
import something into GNU ELPA if it depends on something outside of
that.  So at the time of importing it we would have to fix up those
names one way or another.

It depends partly on whether some of those s.el names are a good idea,
and on which ones a given package uses.  Anyway, we can cope.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Well none of those users of s.el are in GNU Elpa, right?  They could
> trivially rename to use cjstring.el, or string.el.

I think you're confused: I think you say "trivially" because you only
consider the technical part of it.  In practice, most developers see
little benefit (if any) to having their package in GNU ELPA rather than
in MELPA, so forcing a renaming (and one with longer names) will
definitely put off a fair chunk of them.

You may not like the `s-` prefix, but there's a reason why this library
is so popular and it's not only because it puts everything under
a single prefix where things are easier to find (tho, I do think it's
also a significant part of its attractiveness).


        Stefan

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sat, May 2, 2020 at 1:59 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> > Well none of those users of s.el are in GNU Elpa, right?  They could
> > trivially rename to use cjstring.el, or string.el.
>
> You may not like the `s-` prefix, but there's a reason why this library
> is so popular and it's not only because it puts everything under
> a single prefix where things are easier to find (tho, I do think it's
> also a significant part of its attractiveness).

Sure. In that case, if it's _not only_ for that reason, then I
would presume it to be mainly because it encapsulates useful
behaviour.  Logically, those developers wouldn't mind
keeping that useful behaviour and finding it under a different
prefix/names. We could even offer to do the renaming for them.

If OTOH they really are in love with the short prefix,
either we convince them to fall out of love with it, or we
implement a package system. Maybe it doesn't have to be a
full-fledged CL package system, but something that allows,
say, file-local nicknames.

That said, looking at the library. I wonder how much of it
is just odd to elisp and wouldn't need renames anyway.
Shouldn't 's-uppercase?' be string-uppercase-p?

João

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Sure. In that case, if it's _not only_ for that reason, then I
> would presume it to be mainly because it encapsulates useful
> behaviour.

It can also be "just because" (i.e. a question of habit).  In any case,
the point is that renaming the `s` library will go against the grain so
the old `s` will likely stick around anyway.  So I think we're better
off either switching to `s` or coming up with our own `string.el` which
we can design without having to worry about compatibility with `s`.

> Logically, those developers wouldn't mind keeping that useful
> behaviour and finding it under a different prefix/names.

I wouldn't bet on it.  It might have been true before `s` existed, tho.


        Stefan "who favors to `string-` prefix"

Re: Imports / inclusion of s.el into Emacs

[Lars Ingebrigtsen]

João Távora <joaotavora@gmail.com> writes:

> That said, looking at the library. I wonder how much of it
> is just odd to elisp and wouldn't need renames anyway.
> Shouldn't 's-uppercase?' be string-uppercase-p?

People with Clojure backgrounds are used to having predicates with ? at
the end, so -p looks very odd to them.  

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If we had a package system, ideally one with local nicknames,
  > this would all be moot, I think.

Multiple namespaces with a list of namespaces to search has generally
worked badly in Lisp.  In order for that sort of feature to work well,
the search needs to be done when a symbol is used; but Lisp's nature
forces searching when the symbol is looked up.  Common Lisp is an example
of how this loses.  Name prefixes, which we use in Emacs Lisp, work more
clearly and reliably.

I would strongly oppose adding any sort of read-time package searching
in Emacs Lisp.  If you can manage to design one that does it at run time,
it might be a big advance, but don't expect that task to be easy.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Phillip Lord]

João Távora <joaotavora@gmail.com> writes:

> On Fri, May 1, 2020 at 7:09 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>
>> > Just wanted to voice my support for exactly the points that Dmitry makes.
>> > And if we do add it to Elpa, can we give it a slightly longer prefix? Like
>> > 'cljstr`?
>>
>> I don't think renaming it will fly.  We should definitely add it to GNU
>> ELPA because it's used by many other Elisp packages.  But renaming would
>> be self-defeating some all those packages won't want to go through
>> the renaming.
>
> Well none of those users of s.el are in GNU Elpa, right? They could
> trivially rename to use cjstring.el, or string.el.
>
> To be clear, I don't oppose the inclusion of those functions/macros
> especially if they really are advantageous (but are they really?).  What
> I really don't like about this and the dash.el library is that it takes up
> such a prominent place in the shared namespace, because of the
> very short  prefix.  Perhaps this argument is a bit reactionary, but then
> again s.el seems a bit /arriviste/ in that regard (likely unintentionally).
> If we had a package system, ideally one with local nicknames,
> this would all be moot, I think.

s.el is a big short, and dash even more so. But, to be honest, I prefer
dash over seq.el because it is short. It's double dash macros are also
really nice.

Obviously, there is a potential for disaster here, but I would argue
that if this was going to happen it would have already. Of the top of my
head, there is dash, s.el, f.el (for files), kv (for key value).

If emacs had a namespace prefix policy, which it doesn't, I would
reserve one letter and maybe two letter prefixes for nice APIs for
common things.

We'd still have a problem if Emacs had full namespaces right? Packages
always hog the high value namespace.

Phil

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Tue, May 5, 2020 at 11:05 PM Phillip Lord <phillip.lord@russet.org.uk> wrote:

> We'd still have a problem if Emacs had full namespaces right? Packages
> always hog the high value namespace.

Not necessarily.  You can declare locally you want to
use all the symbols of a namespace with no qualification
or very terse qualification. Like python's "import as"

Most CL packages systems have that, by the way. It's
called package-local nicknames. If we implement
namespaces, we want something like that.

João







-- 
João Távora

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Most CL packages systems have that, by the way.  It's called
> package-local nicknames.  If we implement namespaces, we want
> something like that.

Definitely.  If we want namespaces, we want to be able to locally
rename them.  Even better if the system can seamlessly handle the case
where two authors happen to choose the same name for their
respective packages.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1318 bytes --]

>
> As far as s.el goes, I suppose it might be a good thing to add it to
>
ELPA for those who like it.
>
> I'm not completely sold on its contents for the core, however: a lot of
> it looks like a compatibility layer for Clojure's familiarity's sake,
> with very thin wrappers (which basically just add the cost of function
> invocation).
>

Okay, I picked s.el because I felt it was one of the low hanging fruits,
but maybe I'd have picked something else. Now I started let's finish this
one and then I'll move to `file-*`.


> 1. s-prepend/s-append: trivially replaced by 'concat'.
>

Okay, that falls in the part 2 of my plan (decide which functions to
import). I note that you are against such helpers, do people generally have
the same opinion as you on this topic around here?



> 2. s-trim: string-trim is already in subr-x.
>

Well as I said we would not import what already exists int he `string-`
namespace


3. s-split: basically delegates to split-string, but wraps it in
> save-match-data (which is generally against our guidelines for its use).
>

That falls in part 1 of my plan (alias existing misnamespaced functions),
so we'd simply alias split-string as string-split.

Now this point raises an interesting question about save-match-data, are
these guidelines available somewhere?

Thanks,
Philippe

[-- Attachment #2: Type: text/html, Size: 2322 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 01.05.2020 20:36, Philippe Vaucher wrote:

>     1. s-prepend/s-append: trivially replaced by 'concat'.
> 
> 
> Okay, that falls in the part 2 of my plan (decide which functions to 
> import). I note that you are against such helpers, do people generally 
> have the same opinion as you on this topic around here?

I imagine so.

And in general, Clojure tends toward more high level programming, with 
its protocols and optimizing VM. Emacs is more low level, so that shows 
in the API. And, well, it's good to encourage faster code (within the 
limitations of our VM).

>     3. s-split: basically delegates to split-string, but wraps it in
>     save-match-data (which is generally against our guidelines for its use).
> 
> 
> That falls in part 1 of my plan (alias existing 
> misnamespaced functions), so we'd simply alias split-string as string-split.

That's why I suggested to propose a list of changes instead of 
continuing to debate intentions.

> Now this point raises an interesting question about save-match-data, are 
> these guidelines available somewhere?

Hmm. I'm not sure. There is this passage in searching.text, though:

   Notice that all functions are allowed to overwrite the match data
unless they're explicitly documented not to do so.  A consequence is
that functions that are run implicitly in the background
(@pxref{Timers}, and @ref{Idle Timers}) should likely save and restore
the match data explicitly.

And since save-match-data is not free, it makes sense to omit it. 
Especially it functions that are supposed to be used in a lot of code.

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

>> Now this point raises an interesting question about save-match-data, are
>> these guidelines available somewhere?
>
> Hmm. I'm not sure. There is this passage in searching.text, though:

Maybe the doc could be improved to clarify that `save-match-data` should
not be used to preserve someone else's match data but rather to protect
your own match data in the rare case where you execute non-trivial code
between the match and the use of the match-data.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 2223 bytes --]

> I'm not completely sold on its contents for the core, however: a lot of
> it looks like a compatibility layer for Clojure's familiarity's sake,
> with very thin wrappers (which basically just add the cost of function
> invocation).
>
> 1. s-prepend/s-append: trivially replaced by 'concat'.


I'd like to get more insights about this point. I think it's a point that
comes back fairly often here, and I'm not sure I fully understand why.

I'll try to explain my side first so you see where I'm coming from. So, I
guess a lot of s.el users think it's a massive improvement over what emacs
has to offer reguarding string manipulations. This is visible in issues
like https://github.com/magnars/s.el/issues/88. I never coded with clojure,
but my argument for these helpers is that in code we do string
manipulations all the time, they are trivial to write, so why not have
trivial helpers. I understand there's a limit to such helpers because you
could write thousands of them, so good taste should be used to introduce
"the minimal set". I usually don't care about these extra function calls,
what matters more to me is how beautiful the code looks and how readable it
is. If performance becomes a concern, then I usually first improve the
algorithm or switch to C++, and only at last measure do I resort to these
"micro optimisations" like avoiding a function call. Because string
manipulation is so trivial, I'd like them to be grouped together so they
are easy to find. In other languages I can usually go to one page and have
the list of all strings related functions. In Emacs the manual at
https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html
while
helpful makes it hard to "read it all" and find what you are looking for.
For example I couldn't find concat on that page.

From your side, I understand that your definition of the minimal set is
much thiner, and maybe what would help is to give the complete list of the
functions you find superfluous and why (list at
https://github.com/magnars/s.el so far you mention `s-prepend` and
`s-append`). For example I'd disagree if you said `s-left` and `s-right`
are not useful to import because it can be done with `substring`.

Philippe

[-- Attachment #2: Type: text/html, Size: 2847 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 01.05.2020 21:05, Philippe Vaucher wrote:

> I'll try to explain my side first so you see where I'm coming from. So, 
> I guess a lot of s.el users think it's a massive improvement over what 
> emacs has to offer reguarding string manipulations. This is visible in 
> issues like https://github.com/magnars/s.el/issues/88.

I imagine this impression might come purely from seeing a set of these 
functions collected in one place. And not necessarily from including all 
the helpers as well.

> I never coded 
> with clojure, but my argument for these helpers is that in code we do 
> string manipulations all the time, they are trivial to write, so why not 
> have trivial helpers.

The criticism would be exactly the same: if they're trivial, why have 
them in the library at all.

> I understand there's a limit to such helpers 
> because you could write thousands of them, so good taste should be used 
> to introduce "the minimal set".

Exactly. We do have some of them in subr-x already. We can add some 
more, though I'd only pick the non-trivial ones, among those that are 
likely to be used.

> I usually don't care about these extra 
> function calls, what matters more to me is how beautiful the code looks 
> and how readable it is.

Often, I'd agree. But the tradeoffs when creating the standard library 
of the language are somewhat different, IMHO.

> Because string manipulation is so trivial, I'd like them to be 
> grouped together so they are easy to find. In other languages I can 
> usually go to one page and have the list of all strings related 
> functions. In Emacs the manual at 
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html while 
> helpful makes it hard to "read it all" and find what you are looking 
> for. For example I couldn't find concat on that page.

I fully support your criticism here. The old-timers will tell you to use 
the manual (from Emacs), but we both know that a lot of user are still 
much more likely to resort to a web search, and there the manual is not 
so great (no index, at the very least).

>  From your side, I understand that your definition of the minimal set is 
> much thiner, and maybe what would help is to give the complete list of 
> the functions you find superfluous and why (list at 
> https://github.com/magnars/s.el so far you mention `s-prepend` and 
> `s-append`). For example I'd disagree if you said `s-left` and `s-right` 
> are not useful to import because it can be done with `substring`.
I will respectfully refuse this invitation: I have other work to do.

As for 'concat' being hard to find: adding a 'string-concat' alias 
(without any plans to remove the original) could satisfy both sides, maybe.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1530 bytes --]

>
> > I never coded
> > with clojure, but my argument for these helpers is that in code we do
> > string manipulations all the time, they are trivial to write, so why not
> > have trivial helpers.
>
> The criticism would be exactly the same: if they're trivial, why have
> them in the library at all.
>

For me the reason would be: to focus on actual code instead of silly tasks
like computing indices and making code more verbose than necessary also
while avoiding silly bugs.



> > I usually don't care about these extra
> > function calls, what matters more to me is how beautiful the code looks
> > and how readable it is.
>
> Often, I'd agree. But the tradeoffs when creating the standard library
> of the language are somewhat different, IMHO.
>

Interesting point. I want to believe that we can have convenience without
sacrificing performance, maybe by coding these in C? Just a thought, not an
actual proposal.


> >  From your side, I understand that your definition of the minimal set is
> > much thiner, and maybe what would help is to give the complete list of
> > the functions you find superfluous and why (list at
> > https://github.com/magnars/s.el so far you mention `s-prepend` and
> > `s-append`). For example I'd disagree if you said `s-left` and `s-right`
> > are not useful to import because it can be done with `substring`.
>
>
> I will respectfully refuse this invitation: I have other work to do.

No worries. I think I now have enough information to propose something
concrete.

Thanks for your input!

[-- Attachment #2: Type: text/html, Size: 2242 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

You're proposing that we adopt a policy of adding functions to Emacs's
standard name space as if that cost nothing.  Any function that anyone
thinks provides the tiniest simplification, we would add.

Adding so many functions would be detrimental in many ways.

Perhaps nowadays the increased computer memory size would not matter.
It would not matter for five or ten new functions; for hundreds,
perhaps it would.  But computer memory size is the smallest part of
the problems they would cause.

There is also human memory size.  That policy would mean more names
that every Emacs Lisp programmer would need to remember.

It would mean more names to document in the Emacs Lisp Reference Manual.

It would mean more pages to print the Emacs Lisp Reference Manual,
making it cost more.

It would mean more text to maintain when something changes.

With such a weak thresold for adding a functionb name, we would have
these costs over and over.

The cl library caused these problems.  It was not gratuitous -- it
provided useful features.  But eventually we decided that no package
included in Emacs can use the cl library an run time.  We fix packages
to use those facilities in other ways that don't pollute the namespace.

I suggest we adopt the same policy towards s.el, which is entirely
gratuitous.

We can also improve the standard names for string functions.  They
were invented a few at a time, and there is room to make them more
systematic.

This way we would get the improvement that is actually useful, while
paying much lower costs in incompatibility and bloat.  The s.el
approach is designed to maximize the costs.




-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 897 bytes --]

> You're proposing that we adopt a policy of adding functions to Emacs's
> standard name space as if that cost nothing.  Any function that anyone
> thinks provides the tiniest simplification, we would add.

> Adding so many functions would be detrimental in many ways.

> It would mean more names to document in the Emacs Lisp Reference Manual.

> It would mean more pages to print the Emacs Lisp Reference Manual, making
it cost more.

> It would mean more text to maintain when something changes.

These are all very valid points. I'm a bit surprised they come that late in
the debate. I even had to formulate some of these myself as examples of
refutations that addressed the central part of my argument.

So, does that mean that once something "ships" with Emacs it's written in
stone forever? What could be good rules of thumb to be able to rename or
alias something?

Kind regards,
Philippe

[-- Attachment #2: Type: text/html, Size: 1054 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So, does that mean that once something "ships" with Emacs it's written in
  > stone forever? What could be good rules of thumb to be able to rename or
  > alias something?

To replace an established function name is something we do only with
great caution.  First we define the new name.  Later we deprecate the
old name.  Later we make a warning when it is used.  A few years after
that, we could delete it, perhaps.  It takes many years.

We can do this when the new name is clearly an improvement.

We can also add new aliases without replacing the old name.  But we
should not do that merely because someone asks for another name.
-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Phillip Lord]

I would set this against the cognitive overload of the current API which
is difficult and complex to remember, because of its lack of
consistency, and low level nature. This is in addition to the it's
tendency to side-effect, particularly with moving point, hence the
necessity for regular use of save-excursion.

s.el (like dash, f.el) provides a clean API, documented at a single
point. Of course there is a maintaince cost, but people have already
written these packages and are maintaining them. They also help to
provide a breath of modernity, and familiarity for newer developers.

There is a reason that s.el is currently registering 2 million downloads
on MELPA -- part of that will come from downloads in continuous
integration of course, but this is a large number. Perhaps you think it
is gratuitous, but many other people do not.

Phil


Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> You're proposing that we adopt a policy of adding functions to Emacs's
> standard name space as if that cost nothing.  Any function that anyone
> thinks provides the tiniest simplification, we would add.
>
> Adding so many functions would be detrimental in many ways.
>
> Perhaps nowadays the increased computer memory size would not matter.
> It would not matter for five or ten new functions; for hundreds,
> perhaps it would.  But computer memory size is the smallest part of
> the problems they would cause.
>
> There is also human memory size.  That policy would mean more names
> that every Emacs Lisp programmer would need to remember.
>
> It would mean more names to document in the Emacs Lisp Reference Manual.
>
> It would mean more pages to print the Emacs Lisp Reference Manual,
> making it cost more.
>
> It would mean more text to maintain when something changes.
>
> With such a weak thresold for adding a functionb name, we would have
> these costs over and over.
>
> The cl library caused these problems.  It was not gratuitous -- it
> provided useful features.  But eventually we decided that no package
> included in Emacs can use the cl library an run time.  We fix packages
> to use those facilities in other ways that don't pollute the namespace.
>
> I suggest we adopt the same policy towards s.el, which is entirely
> gratuitous.
>
> We can also improve the standard names for string functions.  They
> were invented a few at a time, and there is room to make them more
> systematic.
>
> This way we would get the improvement that is actually useful, while
> paying much lower costs in incompatibility and bloat.  The s.el
> approach is designed to maximize the costs.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

     > 1.  Check wether there are existing emacs `*-string-* functions that'd
     > be aliased to start with `string-` and make a patch for that. That is,
     > already unify the existing Emacs string library if needed.

To define some new 'string-' functions to aid discoverability could be
good in some cases.  But we should judge properly which cases could
be improved this way.

I would be against renaming 'substring' to start with 'string'.
Renaming 'concat' seems also like spurious inconvenience in the name
of rigidity -- the Lisp equivalent of bureaucratese.

In general, I think that very basic and commonly used functions should
be left alone.  Beginners will learn their names anyway.  The more
obscure the function, the more attractive renaming becomes.

(I understand that we might never deprecate the old names, but I still
consider it in spirit a kind of renaming.)

     > 2. Decide of a set of function that we'd "import" from s.el and
     > namespace them under the `string-` namespace. For example, `s-prepend`
     > would become `string-prepend`.

I am not familiar with s.el.  I gather it defines some functions
to do generally useful things on strings.  Is that right>

What does 's-prepend' do?  And how would we do it in Emacs Lisp now?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Fri, 01 May 2020 22:23:44 -0400
> Cc: emacs-devel@gnu.org
> 
> Renaming 'concat' seems also like spurious inconvenience in the name
> of rigidity -- the Lisp equivalent of bureaucratese.

We all do keep in mind that 'concat' is about much more than strings,
yes?  Such polymorphic APIs are IMO a problem when a simplistic
re/naming is considered.

> What does 's-prepend' do?

  (defun s-prepend (prefix s)
  "Concatenate PREFIX and S."

> And how would we do it in Emacs Lisp now?

Trivially, use 'concat'.

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Renaming 'concat' seems also like spurious inconvenience in the name
> of rigidity -- the Lisp equivalent of bureaucratese.

Funny, I find `concat` a perfect example of a bad name because it
doesn't say what it is that it's concatenating nor how.

If I try to imagine myself a "fresh new Elisp programmer" I can't
intuitively know which of `append` or `concat` does what I want.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sat, 02 May 2020 09:03:49 -0400
> Cc: emacs-devel@gnu.org
> 
> > Renaming 'concat' seems also like spurious inconvenience in the name
> > of rigidity -- the Lisp equivalent of bureaucratese.
> 
> Funny, I find `concat` a perfect example of a bad name because it
> doesn't say what it is that it's concatenating nor how.

You expect the _name_ to tell you all that??  Then what do you think
of C library functions like realpath or stpcpy or strpbrk or rpmatch?

Let's not exaggerate, it is only useful if you want to present an
argument ad absurdum.  The world of programming is full of obscure
acronyms, and there's nothing we can do about that.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 16:23, Eli Zaretskii wrote:
> You expect the_name_  to tell you all that??  Then what do you think
> of C library functions like realpath or stpcpy or strpbrk or rpmatch?

Let's please not use the C library as the example of good naming practice.

> Let's not exaggerate, it is only useful if you want to present an
> argument ad absurdum.  The world of programming is full of obscure
> acronyms, and there's nothing we can do about that.

There are more recent popular languages where the situation is much 
better. The author of s.el came from one of them.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sat, May 2, 2020 at 2:29 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 02.05.2020 16:23, Eli Zaretskii wrote:
> > You expect the_name_  to tell you all that??  Then what do you think
> > of C library functions like realpath or stpcpy or strpbrk or rpmatch?
>
> Let's please not use the C library as the example of good naming practice.

I bet you'll find people that think it has the best naming. I kind of like it.

My point this: this is all quite subjective. "recent" and "popular" doesn't
necessarily equal "good".

João

Re: Imports / inclusion of s.el into Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 996 bytes --]

On Sat, May 02, 2020 at 02:34:11PM +0100, João Távora wrote:
> On Sat, May 2, 2020 at 2:29 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> >
> > On 02.05.2020 16:23, Eli Zaretskii wrote:
> > > You expect the_name_  to tell you all that??  Then what do you think
> > > of C library functions like realpath or stpcpy or strpbrk or rpmatch?
> >
> > Let's please not use the C library as the example of good naming practice.
> 
> I bet you'll find people that think it has the best naming. I kind of like it.
> 
> My point this: this is all quite subjective. "recent" and "popular" doesn't
> necessarily equal "good".

...or perhaps it is, but it's a different culture, so it's somewhat
irrelevant. I'm all for putting thought when developing new interfaces,
and correcting obvious mistakes in old ones, but for the latter, one
would first have to gather some consensus that it's broken, then on
which direction to take, and then on doing the change. That's the art!

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sat, May 2, 2020 at 2:43 PM <tomas@tuxteam.de> wrote:
>
> On Sat, May 02, 2020 at 02:34:11PM +0100, João Távora wrote:
> > On Sat, May 2, 2020 at 2:29 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> > >
> > > On 02.05.2020 16:23, Eli Zaretskii wrote:
> > > > You expect the_name_  to tell you all that??  Then what do you think
> > > > of C library functions like realpath or stpcpy or strpbrk or rpmatch?
> > >
> > > Let's please not use the C library as the example of good naming practice.
> >
> > I bet you'll find people that think it has the best naming. I kind of like it.
> >
> > My point this: this is all quite subjective. "recent" and "popular" doesn't
> > necessarily equal "good".
>
> ...or perhaps it is, but it's a different culture, so it's somewhat
> irrelevant. I'm all for putting thought when developing new interfaces,
> and correcting obvious mistakes in old ones, but for the latter, one
> would first have to gather some consensus that it's broken, then on
> which direction to take, and then on doing the change. That's the art!

I agree!

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

I'm not sure what people expect a potential renaming to look like, but
the way I see it, it would be a bunch of individual renamings, not a big
all-or-nothing deal.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 16:34, João Távora wrote:
> I bet you'll find people that think it has the best naming. I kind of like it.

If the name strpbrk speaks to your heart, more power to you.

But let's not use it as an argument that function names don't need to 
convey information.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sat, May 2, 2020 at 2:47 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 02.05.2020 16:34, João Távora wrote:
> > I bet you'll find people that think it has the best naming. I kind of like it.
>
> If the name strpbrk speaks to your heart, more power to you.
>
> But let's not use it as an argument that function names don't need to
> convey information.

As Philippe himself has pointed out, there's the trade-off between
convey information and a long-a$$ name, for example. So we
_can_ use that as argument, in the opposite part of the spectrum.
I for one think that expecting names to tell us everything, or
even a lot, is naive, a losing battle.  And, yes, naming _is_ hard
it's one of the 2 hard ones along with cache invalidation and
off-by-one.

João

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 17:18, João Távora wrote:
> As Philippe himself has pointed out, there's the trade-off between
> convey information and a long-a$$ name, for example. So we
> _can_  use that as argument, in the opposite part of the spectrum.
> I for one think that expecting names to tell us everything, or
> even a lot, is naive, a losing battle.  And, yes, naming_is_  hard
> it's one of the 2 hard ones along with cache invalidation and
> off-by-one.

Can we agree, though, that 'concat' and 'append' are too far from the 
"long-a🐍🐍" end of the spectrum?

I hesitate to propose a renaming because there's a lot of history, but 
just having a string-concat alias could improve the situation.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1117 bytes --]

On Sat, May 2, 2020 at 4:04 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 02.05.2020 17:18, João Távora wrote:
> > As Philippe himself has pointed out, there's the trade-off between
> > convey information and a long-a$$ name, for example. So we
> > _can_  use that as argument, in the opposite part of the spectrum.
> > I for one think that expecting names to tell us everything, or
> > even a lot, is naive, a losing battle.  And, yes, naming_is_  hard
> > it's one of the 2 hard ones along with cache invalidation and
> > off-by-one.
>
> Can we agree, though, that 'concat' and 'append' are too far from the
> "long-a🐍🐍" end of the spectrum?
>
> I hesitate to propose a renaming because there's a lot of history, but
> just having a string-concat alias could improve the situation.
>

Sure I'd agree to that.  (though maybe its rather concat-to-string hehehe).
That's quite different from adding a zillion new s- words because clojure,
tho.

My position is: work on the manual.  Make it prettier, better organized,
etc.
Parsimoniously add new names if that really helps.

João

[-- Attachment #2: Type: text/html, Size: 1621 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 18:10, João Távora wrote:
> (though maybe its rather concat-to-string hehehe)

Which would defeat the point of the original request.

BTW, does 'flex' not care about whether the match is in the beginning of 
the string? Only about the gaps and the string's length?

> My position is: work on the manual.  Make it prettier, better organized, 
> etc.
> Parsimoniously add new names if that really helps.

It might take another 5-10 years before I start using the manual regularly.

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sat, May 2, 2020 at 4:48 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 02.05.2020 18:10, João Távora wrote:
> > (though maybe its rather concat-to-string hehehe)
>
> Which would defeat the point of the original request.

Yeah, but things are what they are! And still discoverable though, IMO.

But the more I think about it the more I think it's a bad name.  I would
never use it, it's so long. concat is concat. It's not His Excellence
Concatenatious the Third. Learning names of variables and functions
is the easiest and most fun part of programming anyway. You use format
until you stumble upon it. And I hope you don't want to rename it
string-format.

> BTW, does 'flex' not care about whether the match is in the beginning of
> the string? Only about the gaps and the string's length?

I think so, yes.  I remember thinking about that and come to the conclusion
that the beginning is no more important than the end.

> > My position is: work on the manual.  Make it prettier, better organized,
> > etc.
> > Parsimoniously add new names if that really helps.
> It might take another 5-10 years before I start using the manual regularly.

We should start by reading the time-traveller's handbook, then.

No really, I'm curious. Why don't _you_ use the manual.  Do you read
any manuals? I'm like 50-50%, honestly I'll open the Elisp info node
and just C-s from there, then search around. But sometimes I'll
use google and C-h f.

João

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 19:04, João Távora wrote:

> But the more I think about it the more I think it's a bad name.  I would
> never use it, it's so long. concat is concat. It's not His Excellence
> Concatenatious the Third. Learning names of variables and functions
> is the easiest and most fun part of programming anyway. You use format
> until you stumble upon it. And I hope you don't want to rename it
> string-format.

'format' is a well-known name on its own, across many languages.

>> BTW, does 'flex' not care about whether the match is in the beginning of
>> the string? Only about the gaps and the string's length?
> 
> I think so, yes.  I remember thinking about that and come to the conclusion
> that the beginning is no more important than the end.

Consider completions for C-h f file-name-.

Don't you think it would be better to sort file-name-directory before 
backup-file-name-p?

Or file-name-sans-extension before help-C-file-name.

>>> My position is: work on the manual.  Make it prettier, better organized,
>>> etc.
>>> Parsimoniously add new names if that really helps.
>> It might take another 5-10 years before I start using the manual regularly.
> 
> We should start by reading the time-traveller's handbook, then.

No need. Just ask a time traveller's really.

> No really, I'm curious. Why don't _you_ use the manual.  Do you read
> any manuals?

Very rarely. It's like a separate skill which I never put effort into 
mastering to an automatic level. The fact that we've had multiple 
disagreements about how the documentation should be written doesn't help 
either.

> I'm like 50-50%, honestly I'll open the Elisp info node
> and just C-s from there, then search around.

That's still slower than using completion.

> But sometimes I'll
> use google and C-h f.

Yup. Most users will use Google anyway, because that's #1 thing they know.

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> Consider completions for C-h f file-name-.
> 
> Don't you think it would be better to sort file-name-directory before
> backup-file-name-p?  Or file-name-sans-extension before help-C-file-name.

Not necessarily, no.  There are several ways
one might want function names sorted, including
for the input `file-name-'.

Some of them:

 alphabetically
 alphabetically and case insensitively
 by a fuzzy-match score
 by last use as input 
 by time of previous use, or alphabetically if not used
 ...

And of course the reverse of any order.  Sorting
candidates is an area with lots of room for improvement.

Re: Imports / inclusion of s.el into Emacs

[chad]

[-- Attachment #1: Type: text/plain, Size: 3314 bytes --]

There is a long-standing school of computer programmers who learn to code
(not algorithms, not group communication, not conveying intention clearly,
but the mechanical "coding" parts) from reading manuals and practicing with
projects (real or "learning" or both). I'm from that school myself.
People from this school tend (large generalization admitted) to learn a
small core from a tutorial, and then glean the rest of the practice of
coding (often, "how to use libraries") by reading manuals (a large part of
the unix `man' tradition, for example).

There is also a very large group of professional, prolific programmers who
learn coding through interactive exploration of APIs. This isn't just
limited to interpreted vs. compiled languages (although I believed that it
comes in part from the rise of teaching via interpreted languages); it also
comes out of the practice of learning via auto-complete and live
documentation. It also has roots in the rise of problem solving via
searching on google/stack exchange/etc.

Like most communities, there are extremes: the grognard who cares why
creat() lost the final `e' and thinks that X is a waste of resources might
be at one end, while the front-end developer who cargo-cults together
javascript frameworks might be at the other. The interesting part for the
discussion here is that each represents a different tendency towards
"learning coding" (which even super-experienced programmers do when
approaching a new system). That some people will prefer to learn by groking
a manual before opening an editor and others by skimming an overview and
relying on completion and searching should not surprise us. Cruically, I
would go further and say that Emacs will be poorer over-all if it insists
on supporting only one part of this spectrum.  I think that it's fair to
say that it currently leans away from the method that a large number of new
coders are demonstrating that they prefer. The question is if and how far
emacs is willing to change to adapt.

I hope that helps,
~Chad

On Sat, May 2, 2020 at 8:11 AM João Távora <joaotavora@gmail.com> wrote:

>
>
> On Sat, May 2, 2020 at 4:04 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
>> On 02.05.2020 17:18, João Távora wrote:
>> > As Philippe himself has pointed out, there's the trade-off between
>> > convey information and a long-a$$ name, for example. So we
>> > _can_  use that as argument, in the opposite part of the spectrum.
>> > I for one think that expecting names to tell us everything, or
>> > even a lot, is naive, a losing battle.  And, yes, naming_is_  hard
>> > it's one of the 2 hard ones along with cache invalidation and
>> > off-by-one.
>>
>> Can we agree, though, that 'concat' and 'append' are too far from the
>> "long-a🐍🐍" end of the spectrum?
>>
>> I hesitate to propose a renaming because there's a lot of history, but
>> just having a string-concat alias could improve the situation.
>>
>
> Sure I'd agree to that.  (though maybe its rather concat-to-string
> hehehe).
> That's quite different from adding a zillion new s- words because clojure,
> tho.
>
> My position is: work on the manual.  Make it prettier, better organized,
> etc.
> Parsimoniously add new names if that really helps.
>
> João
>
>
>

[-- Attachment #2: Type: text/html, Size: 4222 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: chad <yandros@gmail.com>
> Date: Sun, 3 May 2020 17:12:00 -0700
> Cc: Dmitry Gutov <dgutov@yandex.ru>, emacs-devel <emacs-devel@gnu.org>, Eli Zaretskii <eliz@gnu.org>, 
> 	Richard Stallman <rms@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>
> I think that it's fair to say that it currently leans away from the method that a large
> number of new coders are demonstrating that they prefer. The question is if and how far emacs is willing to
> change to adapt.  

IMNSHO, it's inaccurate, and even somewhat unfair, to claim that Emacs
doesn't adapt.  Just read through the latest NEWS file, and you will
see hundreds of new features and quite a few significant new
technologies that we acquired just recently.  Against this backdrop,
to take a single UI feature, or a couple of features, and blame the
entire project that it turns its back on progress and the community at
large, is to blow the issue out of proportion.

The truth is that slow adoption of most significant new features is
almost always caused by absence of someone who'd sit down and do the
job of design, implementation, and testing.  The tab bar is a good
recent example, but there are many others.  Once such a person is
present, and he or she does a clean job, we "adapt" very quickly.
'Nough said.

Re: Imports / inclusion of s.el into Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 482 bytes --]

On Mon, May 04, 2020 at 05:16:46PM +0300, Eli Zaretskii wrote:

[...]

> IMNSHO, it's inaccurate, and even somewhat unfair, to claim that Emacs
> doesn't adapt [...]

Seconded. I think it'll be difficult to find another software
project with 30 years of history. It'd be impossible to achieve
that without adapting.

Hitting a sweet spot between stability and steady-state change is
not an easy task, and I think Emacs maintainers have done an
outstanding job at that.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 04.05.2020 17:16, Eli Zaretskii wrote:
> IMNSHO, it's inaccurate, and even somewhat unfair, to claim that Emacs
> doesn't adapt.  Just read through the latest NEWS file, and you will
> see hundreds of new features and quite a few significant new
> technologies that we acquired just recently.

It adds features where there were none, sure.

But changing something that exists, especially when it had existed for a 
while, is a long, arduous journey with rare chance of success, IME.

It's the difference between adaptation and accretion.

Emacs has survived for decades, sure, but that's because it's based on 
some fundamentally good ideas, and that it managed (probably as a 
consequence) to attract a few very smart people along the way that keep 
it running.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Cc: joaotavora@gmail.com, emacs-devel@gnu.org, rms@gnu.org,
>  monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 4 May 2020 20:23:59 +0300
> 
> On 04.05.2020 17:16, Eli Zaretskii wrote:
> > IMNSHO, it's inaccurate, and even somewhat unfair, to claim that Emacs
> > doesn't adapt.  Just read through the latest NEWS file, and you will
> > see hundreds of new features and quite a few significant new
> > technologies that we acquired just recently.
> 
> It adds features where there were none, sure.
> 
> But changing something that exists, especially when it had existed for a 
> while, is a long, arduous journey with rare chance of success, IME.

It's longer, but not by a long shot.  A recent example: the :extend
face attribute.

> Emacs has survived for decades, sure, but that's because it's based on 
> some fundamentally good ideas, and that it managed (probably as a 
> consequence) to attract a few very smart people along the way that keep 
> it running.

That too, but it isn't enough.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 04.05.2020 20:37, Eli Zaretskii wrote:

>> But changing something that exists, especially when it had existed for a
>> while, is a long, arduous journey with rare chance of success, IME.
> 
> It's longer, but not by a long shot.

Of many discussion about changing default behavior that I've seen, the 
vast majority have failed.

> A recent example: the :extend
> face attribute.

It's an interesting counter-example. First, it did little to change the 
actual functionality, just the looks. Second, there were no noticeable 
consensus, inside or outside the core, that the new behavior is better 
(I've mostly seen dissenting feedback, but the split is probably more 
like 50/50). But apparently you liked it well enough because it made 
Emacs's behavior more compatible with other software _you_ were familiar 
with, that even breaking the expectations of a lot of our users, or 
having to force all theme authors to update their themes (until I came 
along with a fix) wasn't price too high.

>> Emacs has survived for decades, sure, but that's because it's based on
>> some fundamentally good ideas, and that it managed (probably as a
>> consequence) to attract a few very smart people along the way that keep
>> it running.
> 
> That too, but it isn't enough.

Indeed, it isn't. Every such person is a stroke of luck, and people have 
a tendency to get old/change careers/retire/burn out/etc.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Cc: yandros@gmail.com, joaotavora@gmail.com, emacs-devel@gnu.org,
>  rms@gnu.org, monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 4 May 2020 20:52:15 +0300
> 
> On 04.05.2020 20:37, Eli Zaretskii wrote:
> 
> >> But changing something that exists, especially when it had existed for a
> >> while, is a long, arduous journey with rare chance of success, IME.
> > 
> > It's longer, but not by a long shot.
> 
> Of many discussion about changing default behavior that I've seen, the 
> vast majority have failed.

Then you only remember the ones that led to nothing because we didn't
do anything but talk.

> > A recent example: the :extend
> > face attribute.
> 
> It's an interesting counter-example. First, it did little to change the 
> actual functionality, just the looks.

The reaction to this "little change" could have fooled me.

> Second, there were no noticeable consensus, inside or outside the
> core, that the new behavior is better

Not true.  There was complete consensus among those who discussed the
feature before it went into implementation.

> (I've mostly seen dissenting feedback, but the split is probably more 
> like 50/50). But apparently you liked it well enough because it made 
> Emacs's behavior more compatible with other software _you_ were familiar 
> with, that even breaking the expectations of a lot of our users, or 
> having to force all theme authors to update their themes (until I came 
> along with a fix) wasn't price too high.

So what is the lesson you suggest to take out of this example, in the
context of "adapting"?

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 04.05.2020 21:11, Eli Zaretskii wrote:

>> Of many discussion about changing default behavior that I've seen, the
>> vast majority have failed.
> 
> Then you only remember the ones that led to nothing because we didn't
> do anything but talk.

Others, where some change did arrive, contained a lot of talk as well.

>>> A recent example: the :extend
>>> face attribute.
>>
>> It's an interesting counter-example. First, it did little to change the
>> actual functionality, just the looks.
> 
> The reaction to this "little change" could have fooled me.

In your opinion at the time it was little, IIRC. One of the reasons it 
got in, I guess.

>> Second, there were no noticeable consensus, inside or outside the
>> core, that the new behavior is better
> 
> Not true.  There was complete consensus among those who discussed the
> feature before it went into implementation.

Among the 2-3 people who participated in the highly technical 
bug-report/discussion about the display engine? Please be serious. That 
doesn't reflect the opinions even across emacs-devel, much less the 
community at large.

And once we found out the backward compatibility problem, and all the 
associated details, there was still one solution available: revert. 
That's what we usually do when we don't manage to fix a regression 
before a release, don't we?

>> (I've mostly seen dissenting feedback, but the split is probably more
>> like 50/50). But apparently you liked it well enough because it made
>> Emacs's behavior more compatible with other software _you_ were familiar
>> with, that even breaking the expectations of a lot of our users, or
>> having to force all theme authors to update their themes (until I came
>> along with a fix) wasn't price too high.
> 
> So what is the lesson you suggest to take out of this example, in the
> context of "adapting"?

Maybe that backward compatibility is not as important as some people 
like to claim? Other lessons would be less kind to type out.

In any case, I don't see that particular change affecting the experience 
of new Emacs users much one way or another (as long as all the themes 
work). Thus it's not a great example of Emacs "adapting" to contemporary 
user expectations.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Cc: yandros@gmail.com, joaotavora@gmail.com, emacs-devel@gnu.org,
>  rms@gnu.org, monnier@iro.umontreal.ca
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 4 May 2020 21:44:58 +0300
> 
> > The reaction to this "little change" could have fooled me.
> 
> In your opinion at the time it was little, IIRC. One of the reasons it 
> got in, I guess.

No, I didn't think it was little.

> >> Second, there were no noticeable consensus, inside or outside the
> >> core, that the new behavior is better
> > 
> > Not true.  There was complete consensus among those who discussed the
> > feature before it went into implementation.
> 
> Among the 2-3 people who participated in the highly technical 
> bug-report/discussion about the display engine? Please be serious. That 
> doesn't reflect the opinions even across emacs-devel, much less the 
> community at large.

Those were all the people who participated in the discussion.
Everyone who cared.

> And once we found out the backward compatibility problem, and all the 
> associated details, there was still one solution available: revert. 
> That's what we usually do when we don't manage to fix a regression 
> before a release, don't we?

Except that wasn't a regression.

> > So what is the lesson you suggest to take out of this example, in the
> > context of "adapting"?
> 
> Maybe that backward compatibility is not as important as some people 
> like to claim? Other lessons would be less kind to type out.
> 
> In any case, I don't see that particular change affecting the experience 
> of new Emacs users much one way or another (as long as all the themes 
> work). Thus it's not a great example of Emacs "adapting" to contemporary 
> user expectations.

Then I'm sorry to say it, but you have learned nothing from that
event.  Nothing useful, anyway.  Too bad.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 04.05.2020 21:57, Eli Zaretskii wrote:
>> Among the 2-3 people who participated in the highly technical
>> bug-report/discussion about the display engine? Please be serious. That
>> doesn't reflect the opinions even across emacs-devel, much less the
>> community at large.
> Those were all the people who participated in the discussion.
> Everyone who cared.

Obviously not.

>> In any case, I don't see that particular change affecting the experience
>> of new Emacs users much one way or another (as long as all the themes
>> work). Thus it's not a great example of Emacs "adapting" to contemporary
>> user expectations.
> Then I'm sorry to say it, but you have learned nothing from that
> event.  Nothing useful, anyway.  Too bad.

Are you the final authority on who learned what, and on things that are 
there to learn?

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It adds features where there were none, sure.

  > But changing something that exists, especially when it had existed for a 
  > while, is a long, arduous journey with rare chance of success, IME.

That is how it should be.  Stability in Emacs interfaces is very important.

  > It's the difference between adaptation and accretion.

Call it what you wish.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 05.05.2020 05:53, Richard Stallman wrote:
>    > It adds features where there were none, sure.
> 
>    > But changing something that exists, especially when it had existed for a
>    > while, is a long, arduous journey with rare chance of success, IME.
> 
> That is how it should be.  Stability in Emacs interfaces is very important.

It should be harder to change than something new and non-established, 
for sure.

But other factors should play into this as well: the actual impact on 
the users, on one hand. And whether the change is an objective improvement.

If we hold on to the past too strongly every single time, the future 
will zip right by us.

Re: Imports / inclusion of s.el into Emacs

[chad]

[-- Attachment #1: Type: text/plain, Size: 1604 bytes --]

On Mon, May 4, 2020 at 7:16 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: chad <yandros@gmail.com>
> > Date: Sun, 3 May 2020 17:12:00 -0700
> > Cc: Dmitry Gutov <dgutov@yandex.ru>, emacs-devel <emacs-devel@gnu.org>,
> Eli Zaretskii <eliz@gnu.org>,
> >       Richard Stallman <rms@gnu.org>, Stefan Monnier <
> monnier@iro.umontreal.ca>
> > I think that it's fair to say that it currently leans away from the
> method that a large
> > number of new coders are demonstrating that they prefer. The question is
> if and how far emacs is willing to
> > change to adapt.
>
> IMNSHO, it's inaccurate, and even somewhat unfair, to claim that Emacs
> doesn't adapt.


Nowhere did I "claim that Emacs doesn't adapt". I asked, in a very specific
context that you quoted, if and how far emacs is willing to change to adapt
(to "the method that a large number of new coders are demonstrating that
they prefer"), since it is already adapting (in many, many ways). I will
also claim that emacs-devel, as evidenced by this discussion, at least
somewhat resistant to change.

Resistance to change is a very common attribute, especially of older
systems, systems that value stability, and systems that value history --
all of which, I think you and I agree, is true of Emacs/elisp/emacs-devel.
It is not a fault, although it can be pushed to an extreme that becomes a
fault. In many contexts, it is a strong virtue. For one example, search for
Jamie Zawinski's comments on "The CADT Model".

I hope that makes my point clear. And, because I haven't said it in a
while, thanks again for all that you do for emacs!
~Chad

[-- Attachment #2: Type: text/html, Size: 2476 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Cruically, I
  > would go further and say that Emacs will be poorer over-all if it insists
  > on supporting only one part of this spectrum.  I think that it's fair to
  > say that it currently leans away from the method that a large number of new
  > coders are demonstrating that they prefer.

If that refers implicitly to API-based help screens, I don't think
anyone is against better support for those.  There is no need to argue
for the general idea of better API-based help screens.

					       The question is if and how far
  > emacs is willing to change to adapt.

No, that is NOT the question.

People have not asked for "better API-based help screens".  What they
have asked for is specific ways of implementing such API-based help
screens.  Ways that, for the most part, would have other painful
consequences.  People are arguing that we should suffer those painful
consequences.

However, discussion has revealed that one of the things some of those
people really want is better API-based help screens.  We can implement
that without any downside.  I've presented a few ways.  There are
surely others.

Some of these API-based help screens would require attaching data to
some function names -- for instance, to say that 'concat' is a
function for operating on strings.  But that is not hard.  When we see
what data is useful, we can install it.  So don't worry about
installing it -- figure out what data is useful for this.

Fitting these better API-based help screens into the contexts where
they would be most useful would require changes in various places,
including apropos.c.  But those changes are not hard.  When code for
those screens is implemented and ready to install, we will put in the
changes to invoke the code from the best places.

Would people like to write code to display these help screens as they
would like them to appear, based on the data that is available or that
we could add?


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>,
>  Richard Stallman <rms@gnu.org>, emacs-devel <emacs-devel@gnu.org>
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2020 18:03:59 +0300
> 
> just having a string-concat alias could improve the situation.

But concat is not only about strings.  Will we have 3 different
aliases?  And what ab out the case where each argument is of a
different type?

More generally, there's a similar problem with APIs that can accept
arguments of different types.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 363 bytes --]

>
> > just having a string-concat alias could improve the situation.
>
> But concat is not only about strings.  Will we have 3 different
> aliases?  And what ab out the case where each argument is of a
> different type?
>

I think "seq-concat-to-string" is a better choice here. We can remove the
"to-string".

That or "seq-join" with a default separator to "" ?

[-- Attachment #2: Type: text/html, Size: 654 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 2 May 2020 17:47:43 +0200
> Cc: Dmitry Gutov <dgutov@yandex.ru>, João Távora <joaotavora@gmail.com>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Richard Stallman <rms@gnu.org>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
>  But concat is not only about strings.  Will we have 3 different
>  aliases?  And what ab out the case where each argument is of a
>  different type?
> 
> I think "seq-concat-to-string" is a better choice here.

What is a "seq"? asks a newbie.

> We can remove the "to-string".

But then it won't be found when looking for APIs that are related to
strings...

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

>> But concat is not only about strings.  Will
>> we have 3 different aliases?  And what about the
>> case where each argument is of a different type?

Exactly.

> I think "seq-concat-to-string" is a better choice here.
> We can remove the "to-string".
>
> That or "seq-join" with a default separator to "" ?

A character is not a sequence.  `concat' is not
about concatenating sequences to return a string.

`append' is about concatenating sequences to
return a list sequence.

Oops.  Except that it's not.  Although it always
returns a list, in the general sense, it's not
always a sequence (a true list):

(append '(x y) [z]) => (x y . [z])

The devil is in the details.  Lisp is in many
ways an ad hoc language.  Some things are ugly.
Some things are elegant.  Some things are ugly
AND elegant.  Complicated - and simple.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 18:43, Eli Zaretskii wrote:
> But concat is not only about strings.  Will we have 3 different
> aliases?  And what ab out the case where each argument is of a
> different type?

It's another argument for keeping the original names as well.

> More generally, there's a similar problem with APIs that can accept
> arguments of different types.

Like with length or copy-sequence, yes.

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> But concat is not only about strings.  Will we have 3 different
> aliases?  And what ab out the case where each argument is of a
> different type?

And here I am regretting I jumped on `concat`.  I do think the name is
not ideal, but I don't think we can do much better given what it does
and how often it's used in Emacs.  So, while I don't think it's a good
example of naming, it's a particularly poor example to choose to argue for
a renaming.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 18:43, Eli Zaretskii wrote:
>> just having a string-concat alias could improve the situation.
> But concat is not only about strings.  Will we have 3 different
> aliases?  And what ab out the case where each argument is of a
> different type?

BTW, speaking solely of concat, the return value is always a string.

So a 'string-' prefix can make sense from that perspective anyway.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Cc: joaotavora@gmail.com, philippe.vaucher@gmail.com,
>  monnier@iro.umontreal.ca, rms@gnu.org, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2020 21:25:26 +0300
> 
> On 02.05.2020 18:43, Eli Zaretskii wrote:
> >> just having a string-concat alias could improve the situation.
> > But concat is not only about strings.  Will we have 3 different
> > aliases?  And what ab out the case where each argument is of a
> > different type?
> 
> BTW, speaking solely of concat, the return value is always a string.
> 
> So a 'string-' prefix can make sense from that perspective anyway.

That's a slippery slope.  What to do with the likes of
shell-command-to-string? or buffer-string? or even string-match?

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 21:35, Eli Zaretskii wrote:
> That's a slippery slope.  What to do with the likes of
> shell-command-to-string? or buffer-string?

Keep them. One is about shell, another is about buffers.

> or even string-match?

Maybe the same. Maybe rename it to have 'regexp' in the name, as per 
other discussion.

Re: Imports / inclusion of s.el into Emacs

[조성빈]

Dmitry Gutov <dgutov@yandex.ru> 작성:

> On 02.05.2020 18:43, Eli Zaretskii wrote:
>>> just having a string-concat alias could improve the situation.
>> But concat is not only about strings.  Will we have 3 different
>> aliases?  And what ab out the case where each argument is of a
>> different type?
>
> BTW, speaking solely of concat, the return value is always a string.
>
> So a 'string-' prefix can make sense from that perspective anyway.

IMO a type prefix mostly makes sense when

* the function has a central argument(s) both with the types
* constructor/factory/init functions

And a type suffix makes sense when the function returns the type.

So, if I would rename concat, it would be concat-to-string.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: 조성빈 <pcr910303@icloud.com>
> Date: Sun, 3 May 2020 03:40:35 +0900
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org, joaotavora@gmail.com,
>  rms@gnu.org, monnier@iro.umontreal.ca
> 
> So, if I would rename concat, it would be concat-to-string.

But then people who need to concatenate strings will not find it,
because they will type string- TAB.

Re: Imports / inclusion of s.el into Emacs

[조성빈]

Eli Zaretskii <eliz@gnu.org> 작성:

>> From: 조성빈 <pcr910303@icloud.com>
>> Date: Sun, 3 May 2020 03:40:35 +0900
>> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org,  
>> joaotavora@gmail.com,
>>  rms@gnu.org, monnier@iro.umontreal.ca
>>
>> So, if I would rename concat, it would be concat-to-string.
>
> But then people who need to concatenate strings will not find it,
> because they will type string- TAB.

I can’t understand the sarcasm here.

If that becomes a serious problem, we can alias string-concat,
list-concat, and vector-concat to concat. But even if we don’t,
and prefix only some other functions, nothing changes for concat.
The situation isn't becoming worser.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: 조성빈 <pcr910303@icloud.com>
> Date: Sun, 3 May 2020 03:53:37 +0900
> Cc: Emacs developers <emacs-devel@gnu.org>,
>  monnier@iro.umontreal.ca,
>  joaotavora@gmail.com,
>  rms@gnu.org,
>  dgutov@yandex.ru
> 
> >> So, if I would rename concat, it would be concat-to-string.
> >
> > But then people who need to concatenate strings will not find it,
> > because they will type string- TAB.
> 
> I can’t understand the sarcasm here.

There is no sarcasm.

Re: Imports / inclusion of s.el into Emacs

[조성빈]

> 2020. 5. 3. 오전 4:15, Eli Zaretskii <eliz@gnu.org> 작성:
> 
> 
>> From: 조성빈 <pcr910303@icloud.com>
>> Date: Sun, 3 May 2020 03:53:37 +0900
>> Cc: Emacs developers <emacs-devel@gnu.org>,
>> monnier@iro.umontreal.ca,
>> joaotavora@gmail.com,
>> rms@gnu.org,
>> dgutov@yandex.ru
>>>> So, if I would rename concat, it would be concat-to-string.
>>> But then people who need to concatenate strings will not find it,
>>> because they will type string- TAB.
>> I can’t understand the sarcasm here.
> 
> There is no sarcasm.

I’m sorry about the misunderstanding there then.
If it becomes a serious problem, then (as I have mentioned it in the last mail,) we can alias it as string-concat, list-concat, and so on.
But prefixing the functions that aren’t polymorphic (and isn’t as controversial as concat) would be first.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > So, if I would rename concat, it would be concat-to-string.

  > But then people who need to concatenate strings will not find it,
  > because they will type string- TAB.

My solution, extending apropos and its variants so it "finds" 'string-'
in the name 'concat', would deal with that.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: pcr910303@icloud.com, emacs-devel@gnu.org,
> 	monnier@iro.umontreal.ca, joaotavora@gmail.com,
> 	dgutov@yandex.ru
> Date: Sun, 03 May 2020 23:04:43 -0400
> 
>  > > So, if I would rename concat, it would be concat-to-string.
> 
>   > But then people who need to concatenate strings will not find it,
>   > because they will type string- TAB.
> 
> My solution, extending apropos and its variants so it "finds" 'string-'
> in the name 'concat', would deal with that.

The main motivation for "renaming" was to have completion find those
names.  People who advance that proposal don't want to use apropos
instead of completion.  So we will need to extend the completion to do
the same trick you had in mind for apropos.

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> IMO a type prefix mostly makes sense when

Using a naming based on the type of the arguments is a different
convention than the one proposed here.  We're proposing to use
a *prefix* to group functions that are "related" by that prefix.
It doesn't have to do with the type of the return value or the type of
an argument.  E.g. `match-string` doesn't take any regexp as argument
but I'd be happy to put it under the `regexp-` or `re-` prefix
(e.g. `regexp-submatch`).


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> BTW, speaking solely of concat, the return value is always a string.

And there's a good to make that in 99% of the cases, the arguments are
strings as well.  Its main use is to concatenate strings.

> So a 'string-' prefix can make sense from that perspective anyway.

Agreed.  But I'm not sure it's a strong enough case to justify
renaming it.  At least not in the current context.  Maybe some years in
the future when most of the other string manipulation functions have
been brought into the `string-` prefix and people got used to expect all
string manipulations to appear there, the case for renaming `concat`
will be stronger.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 03.05.2020 00:05, Stefan Monnier wrote:
>> BTW, speaking solely of concat, the return value is always a string.
> 
> And there's a good to make that in 99% of the cases, the arguments are
> strings as well.  Its main use is to concatenate strings.

Yup.

>> So a 'string-' prefix can make sense from that perspective anyway.
> 
> Agreed.  But I'm not sure it's a strong enough case to justify
> renaming it.  At least not in the current context.  Maybe some years in
> the future when most of the other string manipulation functions have
> been brought into the `string-` prefix and people got used to expect all
> string manipulations to appear there, the case for renaming `concat`
> will be stronger.

Note I didn't say 'rename', but only to create an alias. And 'some years 
in the future' we could delete the original (though more likely 1-2 
decades in the future). I'm not saying we have to do this, but I do 
think it makes sense.

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> Maybe some years in the future when most of the other string
> manipulation functions have been brought into the `string-`
> prefix and people got used to expect all string manipulations
> to appear there, the case for renaming `concat` will be stronger.

If all string manipulations are to have prefix
`string-' then I guess you're arguing for
that for a function such as `format' too, right?

And does string manipulation for this include
strings that have particular formats or meanings?
Filename-string manipulations (strings, in the end)?
Date-string manipulations (strings, in the end)?

I'm guessing you'd say "no" for the questions in
the last paragraph.

(There are a lot of string-manipulation functions.
The devil is in the details.)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> If all string manipulations are to have prefix `string-'

No we will never put *all* string manipulation under the "string-"
prefix, simply because there's no way to precisely define what is "all
string manipulation".

We know that we can't classify everything correctly.  So we feel free to
use "all" to mean something closer to "most" or "those where it's clear".


        Stefan

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> > If all string manipulations are to have prefix `string-'
> 
> No we will never put *all* string manipulation under the "string-"
> prefix, simply because there's no way to precisely define what is "all
> string manipulation".

Good.  You did speak of a time when "people got
used to expect all string manipulations to appear
there."  They'll expect them all to be there, but
they won't all be there.  Then we can have another
round of this topic, responding to queries about
those that don't appear there... ;-)
 
> We know that we can't classify everything correctly.

You're right about that.  Starting with no agreement
about what "correctly" is for any given case, and
continuing through there not being any single
"correctly" for many cases.

To be clear and more serious: In general I'm in favor
of the most relevant terms appearing in function etc.
names.

Usually that's already the case, but there are no
doubt lots of poorly named functions.

Sometimes there's an order that's more natural or
expected - word order similar to what's spoken, for
example.  And sometimes (often) the order really
doesn't matter - _especially_ for discovery.

But sometimes, sure, there's no reason not to use
a more standardized order.  Whether and when that
standardization should privilege the type of objects
involved or the action involved is another question.

I do expect that the proposed initiative (no, it's
not formulated yet, but the impetus is there) will
likely be a waste of time and doesn't represent a
big problem to be solved.  And I suspect the cure
might be as bad, in the end, as the problem it
tries to solve, or at best, it'll be a wash.

To me it makes more sense for specific problematic
names to be reported - as bugs or enhancement
requests.  Patch this or that real hole in the
bikeshed roof, rather than repaint it.  Even that
(fixing this or that problematic name) has proved
difficult to get done in the past.

Re: Imports / inclusion of s.el into Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 545 bytes --]

On Sat, May 02, 2020 at 04:10:03PM -0700, Drew Adams wrote:
> > > If all string manipulations are to have prefix `string-'
> > 
> > No we will never put *all* string manipulation under the "string-"
> > prefix, simply because there's no way to precisely define what is "all
> > string manipulation".
> 
> Good.

If you pick up a graph by a random node and shake vigorously, it'll
tend to look like a tree. But it won't be one :-)

That's the problem with namespacing. It tries to force the world
into being a tree.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> And there's a good to make that in 99% of the cases, the arguments are

[ Wow, that was quit badly miswritten.  Let me try to untangle it: ]

And there's a good chance that in 99% of the cases, the arguments are


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But concat is not only about strings.

Its arguments can be other types, but the value is always a string.
So _in one sense_ it is firmly tied with strings.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Can we agree, though, that 'concat' and 'append' are too far from the 
  > "long-a🐍🐍" end of the spectrum?

Could you restate the point in plain English?
I don't get it.

'append' is an ancient, fundamental Lisp function
from before 1960.

'concat' is not quite so old, but the Lisp Machine had it in the late
70s and maybe other Lisp implementations had it earlier.
That is too old a tradition to mess with.
'substring' is equally old.
As soon as there were strings, there would have been these two functions.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 03.05.2020 06:43, Richard Stallman wrote:
>    > Can we agree, though, that 'concat' and 'append' are too far from the
>    > "long-a🐍🐍" end of the spectrum?
> 
> Could you restate the point in plain English?
> I don't get it.

The names sound very close and offer little to remind of the difference 
between their purposes.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The names sound very close and offer little to remind of the difference 
  > between their purposes.

It is out of the question to change old traditional names such as
append and concat.  However, you will soon learn what they do, if you
keep programming in Emacs Lisp, because they are used often.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> If the name strpbrk speaks to your heart, more power to you.
> 
> But let's not use it as an argument that function names don't need to
> convey information.

Let's not exaggerate.  Has anyone argued against
function names generally needing to convey info?

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1204 bytes --]

>
> > Let's please not use the C library as the example of good naming
> practice.
>
> I bet you'll find people that think it has the best naming. I kind of like
> it.
>

Of course everything is subjective. But if we take "clarity" as one factor,
then "string-copy" is better than "strcpy". Of course we can argue that
"string-copy-from-one-to-another" is more clear, but then another factor
comes in which is "prefer shorter names because they are less verbose".

It's all a balance between clarity, usability, discoverabilty, etc. I tend
to be on the side of prefering code that looks and read easily while also
being not overly verbose. Code you can read in 3 years and still understand
it straight away. I understand others don't.

I come from C/C++ and in the beginning I liked these cryptic APIs, with all
these corner cases, complexity and microoptimisations. When something
simpler came along I was pesting that kids these days have it too easy. To
me this is just a form of stockholm syndrome :-) Nowadays I'm much more
seduced by elegancy, clarity of intentions and maintainability... reading
code where I instantly know what topic we are talking about seems like an
attractive property to me.

[-- Attachment #2: Type: text/html, Size: 1506 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> Cc: rms@gnu.org, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 2 May 2020 16:29:08 +0300
> 
> On 02.05.2020 16:23, Eli Zaretskii wrote:
> > You expect the_name_  to tell you all that??  Then what do you think
> > of C library functions like realpath or stpcpy or strpbrk or rpmatch?
> 
> Let's please not use the C library as the example of good naming practice.

I wasn't the first to introduce that into this discussion.  People
keep going back to that as an example we should follow.

> > Let's not exaggerate, it is only useful if you want to present an
> > argument ad absurdum.  The world of programming is full of obscure
> > acronyms, and there's nothing we can do about that.
> 
> There are more recent popular languages where the situation is much 
> better. The author of s.el came from one of them.

I can only talk about the stuff I'm familiar with.  Let others bring
up counter-arguments from other places.  But while doing that, let's
remember to compare the sizes of the languages, because a small enough
language can definitely use an exhaustive list of candidates to the
benefit of the users.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 994 bytes --]

>
> > There are more recent popular languages where the situation is much
> > better. The author of s.el came from one of them.
>
> I can only talk about the stuff I'm familiar with.  Let others bring
> up counter-arguments from other places.  But while doing that, let's
> remember to compare the sizes of the languages, because a small enough
> language can definitely use an exhaustive list of candidates to the
> benefit of the users.
>

I don't understand why we still need to come up with examples of other
languages (in other languages they usually have namespaces as the norm),
but here's a list of examples:

https://ruby-doc.org/core-2.2.0/File.html
https://clojure.github.io/clojure/clojure.string-api.html
https://docs.python.org/3/library/filesys.html
https://doc.rust-lang.org/std/thread/index.html
https://docs.racket-lang.org/reference/vectors.html

I understand that just because this technique is popular doens't mean it's
good or right or that it applies to elisp.

Philippe

[-- Attachment #2: Type: text/html, Size: 1707 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 2 May 2020 16:05:29 +0200
> Cc: Dmitry Gutov <dgutov@yandex.ru>, Emacs developers <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Richard Stallman <rms@gnu.org>
>  I can only talk about the stuff I'm familiar with.  Let others bring
>  up counter-arguments from other places.  But while doing that, let's
>  remember to compare the sizes of the languages, because a small enough
>  language can definitely use an exhaustive list of candidates to the
>  benefit of the users.
> 
> I don't understand why we still need to come up with examples of other languages (in other languages they
> usually have namespaces as the norm), but here's a list of examples:
> 
> https://ruby-doc.org/core-2.2.0/File.html

Thank you.  This shows 60 file-related functions.  Does anyone want to
look through a list that long to find what they need?  I don't.

(Funnily enough, "C-u C-h a file-name RET" also shows 63 functions
and commands.)

Re: Imports / inclusion of s.el into Emacs

[조성빈]

Eli Zaretskii <eliz@gnu.org> 작성:

>> From: Philippe Vaucher <philippe.vaucher@gmail.com>
>> Date: Sat, 2 May 2020 16:05:29 +0200
>> Cc: Dmitry Gutov <dgutov@yandex.ru>, Emacs developers  
>> <emacs-devel@gnu.org>,
>> 	Stefan Monnier <monnier@iro.umontreal.ca>, Richard Stallman <rms@gnu.org>
>>  I can only talk about the stuff I'm familiar with.  Let others bring
>>  up counter-arguments from other places.  But while doing that, let's
>>  remember to compare the sizes of the languages, because a small enough
>>  language can definitely use an exhaustive list of candidates to the
>>  benefit of the users.
>>
>> I don't understand why we still need to come up with examples of other  
>> languages (in other languages they
>> usually have namespaces as the norm), but here's a list of examples:
>>
>> https://ruby-doc.org/core-2.2.0/File.html
>
> Thank you.  This shows 60 file-related functions.

Which… are all file-related. For example, if I want to know about whether a
file exists or not, I can go to the File:: and find out there’s a  
File::exist? method.

>  Does anyone want to
> look through a list that long to find what they need?  I don't.
>
> (Funnily enough, "C-u C-h a file-name RET" also shows 63 functions
> and commands.)

"C-u C-h a file exist RET” on the other hand, gives me 9 functions in my  
current session
(with functions totally unrelated, like ones from company-mode).

This particular function is easy to find, since it employs the prefix scheme,
so I can just guess it starts with file and type file-exi and let  
autocomplete help me,
but it still illustrates that C-u C-h a isn’t great enough.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: 조성빈 <pcr910303@icloud.com>
> Date: Sat, 2 May 2020 23:36:48 +0900
> Cc: Philippe Vaucher <philippe.vaucher@gmail.com>,
>  Emacs developers <emacs-devel@gnu.org>,
>  Stefan Monnier <monnier@iro.umontreal.ca>,
>  Richard Stallman <rms@gnu.org>,
>  dgutov@yandex.ru
> 
> "C-u C-h a file exist RET” on the other hand, gives me 9 functions in my  
> current session

Which is quite small and reasonable.

> This particular function is easy to find, since it employs the prefix scheme,
> so I can just guess it starts with file and type file-exi and let  
> autocomplete help me,
> but it still illustrates that C-u C-h a isn’t great enough.

What's wrong with showing 9 candidates?  Btw, in "emacs -Q" the above
show only 5 hits.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1169 bytes --]

>
> > https://ruby-doc.org/core-2.2.0/File.html
>
> Thank you.  This shows 60 file-related functions.  Does anyone want to
> look through a list that long to find what they need?  I don't.


Does anyone? Yes, me and I suspect thousands of people.

This is the crux of the issue. You never need to do that, and I to that
often. When I search for a single function I'm like you, I don't really
need that list. But when I'm discovering the API, it's a gold mine. I can
just read that, quickly understand what it offers and what it does not.
Also everytime I fall on that list it reminds me of almost the whole API
because it's so condensed. Because it's all grouped in one place I can also
curiously click on the names and quickly read what they're about, which
cannot happen in the Emacs manual's "paragraph form".

Hope I was able to give you some insights about how I & other function. I
totally understand your point of view tho, you just always search exactly
for what you want, you think viewing this overview is overrated, and you
don't mind the occasional confusion when you have to search 3 times before
finding what you want (correct me if I'm wrong).

Philippe

[-- Attachment #2: Type: text/html, Size: 1611 bytes --]

Re: Imports / inclusion of s.el into Emacs

[João Távora]

> But when I'm discovering the API, it's a gold mine.

Some of us are trying to tell you is that the Elisp manual is a
gold mine, too. If you see parts where it is not, report it.
Lisp languages are traditionally organized differently.

If you like some "overview by topic" tool, you yourself have
shown it can be built easily (in that "keep-lines" example).
Let's do that first and then think it it's still worth including
many new foreign words, not the reverse.

João

On Sat, May 2, 2020 at 3:43 PM Philippe Vaucher
<philippe.vaucher@gmail.com> wrote:
>>
>> > https://ruby-doc.org/core-2.2.0/File.html
>>
>> Thank you.  This shows 60 file-related functions.  Does anyone want to
>> look through a list that long to find what they need?  I don't.
>
>
> Does anyone? Yes, me and I suspect thousands of people.
>
> This is the crux of the issue. You never need to do that, and I to that often. When I search for a single function I'm like you, I don't really need that list. But when I'm discovering the API, it's a gold mine. I can just read that, quickly understand what it offers and what it does not. Also everytime I fall on that list it reminds me of almost the whole API because it's so condensed. Because it's all grouped in one place I can also curiously click on the names and quickly read what they're about, which cannot happen in the Emacs manual's "paragraph form".
>
> Hope I was able to give you some insights about how I & other function. I totally understand your point of view tho, you just always search exactly for what you want, you think viewing this overview is overrated, and you don't mind the occasional confusion when you have to search 3 times before finding what you want (correct me if I'm wrong).
>
> Philippe



-- 
João Távora

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1905 bytes --]

>
> > But when I'm discovering the API, it's a gold mine.
>
> Some of us are trying to tell you is that the Elisp manual is a
> gold mine, too. If you see parts where it is not, report it.
> Lisp languages are traditionally organized differently.
>

Thanks for the perspective. The parts where it is not is a gold mine is the
amount of hops we have to do:

Say I want to quickly remember how strings work in Emacs, how you
manipulate them, etc. With a few keystrokes I end up on
https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html.
From there I have to select a topics, read it partially, go back, read
another topic, all that while skipping paragraphs most of the time (but
that can be improved with my keep-lines trick).

There is no manual entry where I see all these classified string functions
at once, and "C-h d string" makes my emacs so laggy that I cannot use it,
also most of the entries are irrelevant.

Now compare that to https://ruby-doc.org/core-2.6/String.html. Do you see
how faster that is or is just my lack of habit of using the manual? Or with
https://ruby-doc.org/core-2.6/Thread.html, see how you directly have an
example of common usage?

I guess you could argue that I'm not used to having to read big chunks of
text to get the information I'm looking for, that's I think a valid
criticism.


> If you like some "overview by topic" tool, you yourself have
> shown it can be built easily (in that "keep-lines" example).
> Let's do that first and then think it it's still worth including
> many new foreign words, not the reverse.


That's probably the path of least changes, I agree.

Design-wise I still think that Elisp would be a better language if it was
properly namespaced, given the language itself does not support namespaces.
But that's arguable even tho I think there are more strong points in favor
of using namespaces than against it.

[-- Attachment #2: Type: text/html, Size: 2705 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 2 May 2020 17:20:16 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, Emacs developers <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Richard Stallman <rms@gnu.org>, Dmitry Gutov <dgutov@yandex.ru>
> 
> Say I want to quickly remember how strings work in Emacs, how you manipulate them, etc. With a few
> keystrokes I end up on
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html. From there I
> have to select a topics, read it partially, go back, read another topic, all that while skipping paragraphs most
> of the time (but that can be improved with my keep-lines trick).
> 
> There is no manual entry where I see all these classified string functions at once, and "C-h d string" makes
> my emacs so laggy that I cannot use it, also most of the entries are irrelevant.

It sounds very strange to me that the method of learning about strings
is by looking at the list of string-related APIs.  If you want to
learn about strings in Emacs, you should read the entire chapter
"Strings and Characters", including all of its sections and
subsections.  This is how a certain topic should be learned for the
first time, or after a long break when you no longer sure you remember
enough of the basics.

> Now compare that to https://ruby-doc.org/core-2.6/String.html. Do you see how faster that is or is just my
> lack of habit of using the manual?

What should I look at there?  I see a very long list of functions,
each one followed by 5 to 10 lines of description.  How is it
different from what we have in the ELisp manual?

> Or with https://ruby-doc.org/core-2.6/Thread.html, see how you directly
> have an example of common usage?

How can a single example of "typical usage" help you understand a
complex topic such as threads?  And what is "typical usage" of
threads, anyway?  I could use threads in umpteen different ways, all
of them "typical".  What am I missing?

> I guess you could argue that I'm not used to having to read big chunks of text to get the information I'm
> looking for, that's I think a valid criticism.

What big chunks of text are you alluding to?  Are you saying that the
smaller is the documentation of a function, the better?

In the ELisp manual we describe each function with as many words as
required to describe its functionality.  (There are people who think
we need to tell more.)  We also provide "continuity" text to serve as
a "glue" which is supposed to help the reader understand the topic
better and see each API in its context and relation to others.

"Manuals" that are just lists of APIs with minimum explanatory text,
a-la JavaDoc, are _bad_ manuals.  They don't tell you enough about the
topics for you to understand when use one class of APIs and when to
use another.  If you want to see a representative of such bad manuals,
look at the GTK docs.  Is this what you'd like to see in the ELisp
manual?

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 3157 bytes --]

>
> > There is no manual entry where I see all these classified string
> functions at once, and "C-h d string" makes
> > my emacs so laggy that I cannot use it, also most of the entries are
> irrelevant.
>
> It sounds very strange to me that the method of learning about strings
> is by looking at the list of string-related APIs.  If you want to
> learn about strings in Emacs, you should read the entire chapter
> "Strings and Characters", including all of its sections and
> subsections.  This is how a certain topic should be learned for the
> first time, or after a long break when you no longer sure you remember
> enough of the basics.
>

I understand that's how you function. That's not how I & some
others function. My way is arguably bad, but it just works in other
languages.



> > Now compare that to https://ruby-doc.org/core-2.6/String.html. Do you
> see how faster that is or is just my
> > lack of habit of using the manual?
>
> What should I look at there?  I see a very long list of functions,
> each one followed by 5 to 10 lines of description.  How is it
> different from what we have in the ELisp manual?
>

Look at the list of methods on the left, which you can click and it makes
you jump to the complete description. I miss that list in Emacs.



> > Or with https://ruby-doc.org/core-2.6/Thread.html, see how you directly
> > have an example of common usage?
>
> How can a single example of "typical usage" help you understand a
> complex topic such as threads?  And what is "typical usage" of
> threads, anyway?  I could use threads in umpteen different ways, all
> of them "typical".  What am I missing?
>

I'm sorry but I give up. You'd be able to understand on your own why basic
examples are helpful. Try to look at sites like stackoverflow and try to
understand why people like it.



> > I guess you could argue that I'm not used to having to read big chunks
> of text to get the information I'm
> > looking for, that's I think a valid criticism.
>
> What big chunks of text are you alluding to?  Are you saying that the
> smaller is the documentation of a function, the better?
>

Well, do you really believe this is what I'm trying to say?

I'm saying the full-blown documentation is all that is available, and I'd
like a summarized view like in other languages.



> In the ELisp manual we describe each function with as many words as
> required to describe its functionality.  (There are people who think
> we need to tell more.)  We also provide "continuity" text to serve as
> a "glue" which is supposed to help the reader understand the topic
> better and see each API in its context and relation to others.
>

Yes, the Emacs manual is good in that regard.



> "Manuals" that are just lists of APIs with minimum explanatory text,
> a-la JavaDoc, are _bad_ manuals.  They don't tell you enough about the
> topics for you to understand when use one class of APIs and when to
> use another.  If you want to see a representative of such bad manuals,
> look at the GTK docs.  Is this what you'd like to see in the ELisp
> manual?
>

I think you need boths. Do you think the Ruby documentation I linked is bad?

Philippe

[-- Attachment #2: Type: text/html, Size: 4770 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 2 May 2020 18:31:27 +0200
> Cc: João Távora <joaotavora@gmail.com>, 
> 	Emacs developers <emacs-devel@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	Richard Stallman <rms@gnu.org>, Dmitry Gutov <dgutov@yandex.ru>
> 
>  > Now compare that to https://ruby-doc.org/core-2.6/String.html. Do you see how faster that is or is
>  just my
>  > lack of habit of using the manual?
> 
>  What should I look at there?  I see a very long list of functions,
>  each one followed by 5 to 10 lines of description.  How is it
>  different from what we have in the ELisp manual?
> 
> Look at the list of methods on the left, which you can click and it makes you jump to the complete
> description. I miss that list in Emacs.

Why?  If you already know what method you want to look up, just use
"C-h f" or "i" in Info.  If you do NOT know the name of the method,
how do you know which method to click on?

>  > Or with https://ruby-doc.org/core-2.6/Thread.html, see how you directly
>  > have an example of common usage?
> 
>  How can a single example of "typical usage" help you understand a
>  complex topic such as threads?  And what is "typical usage" of
>  threads, anyway?  I could use threads in umpteen different ways, all
>  of them "typical".  What am I missing?
> 
> I'm sorry but I give up. You'd be able to understand on your own why basic examples are helpful. Try to look
> at sites like stackoverflow and try to understand why people like it.

I guess I'm not smart enough to understand that.

>  "Manuals" that are just lists of APIs with minimum explanatory text,
>  a-la JavaDoc, are _bad_ manuals.  They don't tell you enough about the
>  topics for you to understand when use one class of APIs and when to
>  use another.  If you want to see a representative of such bad manuals,
>  look at the GTK docs.  Is this what you'd like to see in the ELisp
>  manual?
> 
> I think you need boths. Do you think the Ruby documentation I linked is bad?

It's worse than the ELisp manual, yes.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 19:40, Eli Zaretskii wrote:
>> Look at the list of methods on the left, which you can click and it makes you jump to the complete
>> description. I miss that list in Emacs.
> Why?  If you already know what method you want to look up, just use
> "C-h f" or "i" in Info.  If you do NOT know the name of the method,
> how do you know which method to click on?

You look over the list, and either notice the word you wanted inside one 
of the function names, or look at the names one by one, in the worst 
case, and recall that one of them is what you need. It might be a 
different skill, but it's one we already have (by the necessity of 
dealing with a number of other languages).

Looking over a list of 50-60 entries like that takes only a few seconds, 
and that's longer than dealing with the manual.

Re: Imports / inclusion of s.el into Emacs

[Dmitry Gutov]

On 02.05.2020 19:53, Dmitry Gutov wrote:
> On 02.05.2020 19:40, Eli Zaretskii wrote:
>>> Look at the list of methods on the left, which you can click and it 
>>> makes you jump to the complete
>>> description. I miss that list in Emacs.
>> Why?  If you already know what method you want to look up, just use
>> "C-h f" or "i" in Info.  If you do NOT know the name of the method,
>> how do you know which method to click on?
> 
> You look over the list, and either notice the word you wanted inside one 
> of the function names, or look at the names one by one, in the worst 
> case, and recall that one of them is what you need. It might be a 
> different skill, but it's one we already have (by the necessity of 
> dealing with a number of other languages).
> 
> Looking over a list of 50-60 entries like that takes only a few seconds, 
> and that's longer than dealing with the manual.
              ^ faster

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1563 bytes --]

>
> >  > Or with https://ruby-doc.org/core-2.6/Thread.html, see how you
> directly
> >  > have an example of common usage?
> >
> >  How can a single example of "typical usage" help you understand a
> >  complex topic such as threads?  And what is "typical usage" of
> >  threads, anyway?  I could use threads in umpteen different ways, all
> >  of them "typical".  What am I missing?
> >
> > I'm sorry but I give up. You'd be able to understand on your own why
> basic examples are helpful. Try to look
> > at sites like stackoverflow and try to understand why people like it.
>
> I guess I'm not smart enough to understand that.


I'm sorry. Whenever I explain something and it looks so evident to me and
someone still does not understand my point, I tend to think he has to be
trolling me and get a bit upset. I shouldn't have answered that way.

Maybe you are just unfamiliar with the "tutorial" way of learning things.

I'll try to make an example. It is not perfect. Please try to bear with me.

For quite a lot of people, this page
http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler to
use & learn from than this page
https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html

It does not mean that the later page is bad, it is actually more complete,
better documented, already grouped in topics... and yet, a lot of people
prefer the first page... straight to the point, simple examples.

Please tell me if you are able to understand this, if you cannot I'll try
to explain further.

Kind regards,
Philippe

[-- Attachment #2: Type: text/html, Size: 2335 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sat, 2 May 2020 21:54:42 +0200
> Cc: João Távora <joaotavora@gmail.com>, 
> 	Emacs developers <emacs-devel@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	Richard Stallman <rms@gnu.org>, Dmitry Gutov <dgutov@yandex.ru>
> 
> Maybe you are just unfamiliar with the "tutorial" way of learning things. 

I am familiar with tutorials.  They have their limits: you can only
learn this much by following a tutorial.

The ELisp manual is not a tutorial, it is (or at least is supposed to
be) a complete and comprehensive reference manual to the Emacs Lisp
language and core facilities.

> For quite a lot of people, this page http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler
> to use & learn from than this page
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html
> 
> It does not mean that the later page is bad, it is actually more complete, better documented, already grouped
> in topics... and yet, a lot of people prefer the first page... straight to the point, simple examples.
> 
> Please tell me if you are able to understand this, if you cannot I'll try to explain further.

I understand this very well, I'm just astonished to hear that this is
all the documentation you and some others want to see or have.  It's
completely against all I have ever learned and practiced in the way of
teaching people various skills and topics (and I did that more than
once or twice).

Emacs has a wealth of documentation at your fingertips, arranged,
indexed, and enriched with a large collection of commands designed to
make our self-documenting editor easy to study.  To give up all of
this and insist on having instead a flat list of APIs that match some
textual pattern is something that comes as a shock to me.

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 995 bytes --]

>
> I understand this very well, I'm just astonished to hear that this is
> all the documentation you and some others want to see or have.  It's
> completely against all I have ever learned and practiced in the way of
> teaching people various skills and topics (and I did that more than
> once or twice).
>

No this is not what we are saying. Tutorials are not the only documentation
I and others want to see.

We like to switch between tutorials and full-blown documentation at
different times depending on various factors. In Emacs there's only the
full-blown documentation.



> Emacs has a wealth of documentation at your fingertips, arranged,
> indexed, and enriched with a large collection of commands designed to
> make our self-documenting editor easy to study.  To give up all of
> this and insist on having instead a flat list of APIs that match some
> textual pattern is something that comes as a shock to me.
>

We are not giving up on it. Some might but at least not me.

Philippe

[-- Attachment #2: Type: text/html, Size: 1462 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Philippe Vaucher <philippe.vaucher@gmail.com>
> Date: Sun, 3 May 2020 16:18:43 +0200
> Cc: João Távora <joaotavora@gmail.com>, 
> 	Emacs developers <emacs-devel@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	Richard Stallman <rms@gnu.org>, Dmitry Gutov <dgutov@yandex.ru>
> 
>  I understand this very well, I'm just astonished to hear that this is
>  all the documentation you and some others want to see or have.  It's
>  completely against all I have ever learned and practiced in the way of
>  teaching people various skills and topics (and I did that more than
>  once or twice).
> 
> No this is not what we are saying. Tutorials are not the only documentation I and others want to see.
> 
> We like to switch between tutorials and full-blown documentation at different times depending on various
> factors. In Emacs there's only the full-blown documentation.

Then I really don't get it.  Tutorials are generally read only once,
when you first familiarize yourself with a topic and generally want
just the "taste" of it.  Once you are past that step, there's no
reason to go back to the tutorial, and the reference manual is the
main tool for learning and doing.

And yet you insist on seeing "tutorial-style" lists each time you need
some API, but don't really know which.  Why? this doesn't sound like
what people use tutorials for.

>  Emacs has a wealth of documentation at your fingertips, arranged,
>  indexed, and enriched with a large collection of commands designed to
>  make our self-documenting editor easy to study.  To give up all of
>  this and insist on having instead a flat list of APIs that match some
>  textual pattern is something that comes as a shock to me.
> 
> We are not giving up on it. Some might but at least not me.

Well, you could have fooled me!  You started by saying that C-h
commands are virtually useless (maybe not in these words, but that's
the cruz of what you said back then).

Re: Imports / inclusion of s.el into Emacs

[Yuri Khan]

On Sun, 3 May 2020 at 23:13, Eli Zaretskii <eliz@gnu.org> wrote:

> Tutorials are generally read only once,
> when you first familiarize yourself with a topic and generally want
> just the "taste" of it.  Once you are past that step, there's no
> reason to go back to the tutorial, and the reference manual is the
> main tool for learning and doing.
>
> And yet you insist on seeing "tutorial-style" lists each time you need
> some API, but don't really know which.  Why? this doesn't sound like
> what people use tutorials for.

API lists are not tutorials. They are the reference manual, just
presented in a very condensed form. A user who is moderately
experienced with an API but has not committed it into memory will
sometimes forget exact names of functions, and a concise API list will
refresh his/her memory just enough to continue coding.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sun, 3 May 2020 23:32:01 +0700
> Cc: Philippe Vaucher <philippe.vaucher@gmail.com>, Dmitry Gutov <dgutov@yandex.ru>, 
> 	Richard Stallman <rms@gnu.org>, João Távora <joaotavora@gmail.com>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, Emacs developers <emacs-devel@gnu.org>
> 
> On Sun, 3 May 2020 at 23:13, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > Tutorials are generally read only once,
> > when you first familiarize yourself with a topic and generally want
> > just the "taste" of it.  Once you are past that step, there's no
> > reason to go back to the tutorial, and the reference manual is the
> > main tool for learning and doing.
> >
> > And yet you insist on seeing "tutorial-style" lists each time you need
> > some API, but don't really know which.  Why? this doesn't sound like
> > what people use tutorials for.
> 
> API lists are not tutorials.

Are you saying this to me or to Philippe?  It was him who brought up
the tutorial analogy.  I just responded to that.

> They are the reference manual, just presented in a very condensed
> form. A user who is moderately experienced with an API but has not
> committed it into memory will sometimes forget exact names of
> functions, and a concise API list will refresh his/her memory just
> enough to continue coding.

We've been there, let's not make another round of the same arguments.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > API lists are not tutorials. They are the reference manual, just
  > presented in a very condensed form. A user who is moderately
  > experienced with an API but has not committed it into memory will
  > sometimes forget exact names of functions, and a concise API list will
  > refresh his/her memory just enough to continue coding.

That's a valid point.

I think we could generate API lists like these
automatically from the text of the Emacs Lisp Ref Manual.
Just search through the chosen node (and its subnodes) for
definition constructs, and process them to get calling sequences.

Would someone like to try writing that?
-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Jean-Christophe Helary]

> On May 4, 2020, at 12:09, Richard Stallman <rms@gnu.org> wrote:

>> API lists are not tutorials. They are the reference manual, just
>> presented in a very condensed form. A user who is moderately
>> experienced with an API but has not committed it into memory will
>> sometimes forget exact names of functions, and a concise API list will
>> refresh his/her memory just enough to continue coding.
> 
> That's a valid point.
> 
> I think we could generate API lists like these
> automatically from the text of the Emacs Lisp Ref Manual.
> Just search through the chosen node (and its subnodes) for
> definition constructs, and process them to get calling sequences.
> 
> Would someone like to try writing that?

There was a similar proposal in June 2017 in a thread titled "docstrings and elisp reference" where I suggested that the docstrings could be used for that purpose.

Considering that docstrings describe more functions than the manual does, would you think that working on the dosctrings for what you describe above would be better to expose emacs' APIs ?


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
> Date: Mon, 4 May 2020 16:35:53 +0900
> Cc: Yuri Khan <yuri.v.khan@gmail.com>,
>  Eli Zaretskii <eliz@gnu.org>,
>  Dmitry Gutov <dgutov@yandex.ru>,
>  joaotavora@gmail.com,
>  Stefan Monnier <monnier@iro.umontreal.ca>,
>  emacs-devel@gnu.org
> 
> Considering that docstrings describe more functions than the manual does, would you think that working on the dosctrings for what you describe above would be better to expose emacs' APIs ?

The doc strings cannot be used for this purpose, because they lack the
information about the groups to which each function belongs.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Considering that docstrings describe more functions than the
  > > manual does, would you think that working on the dosctrings for
  > > what you describe above would be better to expose emacs' APIs ?

  > The doc strings cannot be used for this purpose, because they lack the
  > information about the groups to which each function belongs.

Don't worry about what information Emacs does or doesn't have.  For
providing better help, we can add whatever classification data is
useful for that.  Adding that data is just some work -- it is not an
incompatibility.  People who like some new help interface will add the
data it needs to work.


-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Lars Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

>   > The doc strings cannot be used for this purpose, because they lack the
>   > information about the groups to which each function belongs.
>
> Don't worry about what information Emacs does or doesn't have.  For
> providing better help, we can add whatever classification data is
> useful for that.  Adding that data is just some work -- it is not an
> incompatibility.  People who like some new help interface will add the
> data it needs to work.

Indeed.

My suggestion was to add a grouping operator to functions (like we have
with variables (i.e., defcustoms) that could be used to do these simple
overview pages a la the s.el documentation.

For instance something like

(define-function-group concat
  :group string
  :example "(concat \"abc\" \"def\" \"ghi\") => \"abcdefghi\"")

or whatever.  The overview list would then consist of the first line of
the doc string and the example(s).

The normal help buffer for `concat' would link to this list of string
functions, and the entries in this new overview buffer would link to the
manual entry.

I mean, this is Emacs, the self-documenting editor.  

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Considering that docstrings describe more functions than the
  > manual does, would you think that working on the dosctrings for
  > what you describe above would be better to expose emacs' APIs ?

If you think it might work well, please try implementing it.
We don't need to argue theoretically about which of these ideas is better.
Try them and decide.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: eliz@gnu.org, emacs-devel@gnu.org, joaotavora@gmail.com,
> 	dgutov@yandex.ru, monnier@iro.umontreal.ca
> Date: Sun, 03 May 2020 23:09:21 -0400
> 
> I think we could generate API lists like these
> automatically from the text of the Emacs Lisp Ref Manual.
> Just search through the chosen node (and its subnodes) for
> definition constructs, and process them to get calling sequences.

What would you suggest to do with the generated list?  How would users
access it?

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I think we could generate API lists like these
  > > automatically from the text of the Emacs Lisp Ref Manual.
  > > Just search through the chosen node (and its subnodes) for
  > > definition constructs, and process them to get calling sequences.

  > What would you suggest to do with the generated list?  How would users
  > access it?

There is no need for us to try to answer that.  The people who want
that kind of interface are the people who should work out the details.

They can try implementing it in various ways, decide what they prefer,
and offer us the code for it.

  > The main motivation for "renaming" was to have completion find those
  > names.  People who advance that proposal don't want to use apropos
  > instead of completion.  So we will need to extend the completion to do
  > the same trick you had in mind for apropos.

That won't be painful.  It isn't a fundamental or incompatible change.
It would take some work to implement -- but the people who want it can
do that work.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Jean-Christophe Helary]

> On May 3, 2020, at 23:13, Eli Zaretskii <eliz@gnu.org> wrote:

>> For quite a lot of people, this page http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler
>> to use & learn from than this page
>> https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html
>> 
>> It does not mean that the later page is bad, it is actually more complete, better documented, already grouped
>> in topics... and yet, a lot of people prefer the first page... straight to the point, simple examples.
>> 
>> Please tell me if you are able to understand this, if you cannot I'll try to explain further.
> 
> I understand this very well, I'm just astonished to hear that this is
> all the documentation you and some others want to see or have.

Nobody claimed that it was *all* we want to see or have. Quite the contrary.

What ergoemacs shows represents one step on a pretty steep learning curve.

That's all "we" (Philippe and I as far as I can tell from this thread, but all the people who use ergoemacs as a reference too, I guess) claim.

There is a need for such a summary *that also links at the current documentation*, as I also replied.

That "summary" (regardless of how it is implemented) does not exist today and that is partly what this whole thread is about.

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
> Date: Mon, 4 May 2020 16:47:11 +0900
> Cc: Philippe Vaucher <philippe.vaucher@gmail.com>,
>  dgutov@yandex.ru,
>  rms@gnu.org,
>  joaotavora@gmail.com,
>  monnier@iro.umontreal.ca,
>  emacs-devel@gnu.org
> 
> >> For quite a lot of people, this page 
> >> http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler
> >> to use & learn from than this page
> >> https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html
> >> 
> >> It does not mean that the later page is bad, it is actually more complete, 
> >> better documented, already grouped
> >> in topics... and yet, a lot of people prefer the first page... straight to 
> >> the point, simple examples.
> >> 
> >> Please tell me if you are able to understand this, if you cannot I'll try to 
> >> explain further.
> > 
> > I understand this very well, I'm just astonished to hear that this is
> > all the documentation you and some others want to see or have.
> 
> Nobody claimed that it was *all* we want to see or have. Quite the contrary.

Maybe we are in two different discussions, then.  Because the above
clearly says: "this page (ergoemacs) is much simpler to use & learn
from than this page (ELisp)".  IOW, it sees the ergoemacs page as a
better _replacement_ for the ELisp manual.  If this doesn't mean that
the ELisp manual is not needed, then maybe I have basic
misunderstanding of written English.

Or maybe your opinion is different from that of Philippe, but then I
was responding to Philippe, and in any case "nobody claimed" is then
inaccurate, isn't it?

> There is a need for such a summary *that also links at the current 
> documentation*, as I also replied.
> 
> That "summary" (regardless of how it is implemented) does not exist today and 
> that is partly what this whole thread is about.

AFAIU, the request for such a summary is quite new in this discussion,
and is definitely not "what this whole thread is about.".  Most of
this thread is about something entirely different: changing the names
of APIs so that completion could be more useful for discovery.

Re: Imports / inclusion of s.el into Emacs

[조성빈]

Eli Zaretskii <eliz@gnu.org> 작성:

>> From: Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
>> Date: Mon, 4 May 2020 16:47:11 +0900
>> Cc: Philippe Vaucher <philippe.vaucher@gmail.com>,
>>  dgutov@yandex.ru,
>>  rms@gnu.org,
>>  joaotavora@gmail.com,
>>  monnier@iro.umontreal.ca,
>>  emacs-devel@gnu.org
>>
>>>> For quite a lot of people, this page
>>>> http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler
>>>> to use & learn from than this page
>>>> https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html
>>>>
>>>> It does not mean that the later page is bad, it is actually more  
>>>> complete,
>>>> better documented, already grouped
>>>> in topics... and yet, a lot of people prefer the first page...  
>>>> straight to
>>>> the point, simple examples.
>>>>
>>>> Please tell me if you are able to understand this, if you cannot I'll  
>>>> try to
>>>> explain further.
>>>
>>> I understand this very well, I'm just astonished to hear that this is
>>> all the documentation you and some others want to see or have.
>>
>> Nobody claimed that it was *all* we want to see or have. Quite the  
>> contrary.
>
> Maybe we are in two different discussions, then.  Because the above
> clearly says: "this page (ergoemacs) is much simpler to use & learn
> from than this page (ELisp)".  IOW, it sees the ergoemacs page as a
> better _replacement_ for the ELisp manual.  If this doesn't mean that
> the ELisp manual is not needed, then maybe I have basic
> misunderstanding of written English.

So to me, the ergoemacs page is useful for various situations like…
* One knows basic Elisp knowledge but have not used any string APIs. So one
   wants to know various operations that Emacs already provides.
* One have already written Elisp to be familiar enough about the string APIs,
   knows that there are APIs that allow replacing string using a regex, but
   doesn’t remember the replace-regexp-in-string name.

The Elisp manual in the other hand, is useful for situations like…
* One wants to know exactly how strings work in Elisp - what happens if the
   argument of string-to-number is a garbage string?
* Would there be some options to number-to-string, like bases? Or should I
   implement them myself?

> Or maybe your opinion is different from that of Philippe, but then I
> was responding to Philippe, and in any case "nobody claimed" is then
> inaccurate, isn't it?
>
>> There is a need for such a summary *that also links at the current
>> documentation*, as I also replied.
>>
>> That "summary" (regardless of how it is implemented) does not exist  
>> today and
>> that is partly what this whole thread is about.
>
> AFAIU, the request for such a summary is quite new in this discussion,
> and is definitely not "what this whole thread is about.".  Most of
> this thread is about something entirely different: changing the names
> of APIs so that completion could be more useful for discovery.

Re: Imports / inclusion of s.el into Emacs

[Jean-Christophe Helary]

> On May 4, 2020, at 23:36, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
>> Date: Mon, 4 May 2020 16:47:11 +0900
>> Cc: Philippe Vaucher <philippe.vaucher@gmail.com>,
>> dgutov@yandex.ru,
>> rms@gnu.org,
>> joaotavora@gmail.com,
>> monnier@iro.umontreal.ca,
>> emacs-devel@gnu.org
>> 
>>>> For quite a lot of people, this page 
>>>> http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler
>>>> to use & learn from than this page
>>>> https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html
>>>> 
>>>> It does not mean that the later page is bad, it is actually more complete, 
>>>> better documented, already grouped
>>>> in topics... and yet, a lot of people prefer the first page... straight to 
>>>> the point, simple examples.
>>>> 
>>>> Please tell me if you are able to understand this, if you cannot I'll try to 
>>>> explain further.
>>> 
>>> I understand this very well, I'm just astonished to hear that this is
>>> all the documentation you and some others want to see or have.
>> 
>> Nobody claimed that it was *all* we want to see or have. Quite the contrary.
> 
> Maybe we are in two different discussions, then.  Because the above
> clearly says: "this page (ergoemacs) is much simpler to use & learn
> from than this page (ELisp)".  IOW, it sees the ergoemacs page as a
> better _replacement_ for the ELisp manual.

No. This information comes in the context of *adding* some discoverability to current emacs. What *this* thread is all about. Nobody is discussing the lack of usefulness of the emacs manual or its replacement by another data set.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
> Date: Tue, 5 May 2020 10:37:50 +0900
> Cc: Richard Stallman <rms@gnu.org>,
>  Emacs developers <emacs-devel@gnu.org>,
>  joaotavora@gmail.com,
>  Dmitry Gutov <dgutov@yandex.ru>,
>  monnier@iro.umontreal.ca
> 
> >> Nobody claimed that it was *all* we want to see or have. Quite the contrary.
> > 
> > Maybe we are in two different discussions, then.  Because the above
> > clearly says: "this page (ergoemacs) is much simpler to use & learn
> > from than this page (ELisp)".  IOW, it sees the ergoemacs page as a
> > better _replacement_ for the ELisp manual.
> 
> No. This information comes in the context of *adding* some discoverability to current emacs. What *this* thread is all about. Nobody is discussing the lack of usefulness of the emacs manual or its replacement by another data set.

That's not my interpretation of the discussion, sorry.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > For quite a lot of people, this page
  > http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler to
  > use & learn from than this page
  > https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html

The ergoemacs page is exactly what you need if (1) you already know how to
operate on strings in some other programming language that is typical of today,
(2) you don't care about the edge cases and details, on which the page
can be wrong.

There are topics for which this approach is adequate for many users,
but that is not true of all topics.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Jean-Christophe Helary]

> On May 3, 2020, at 1:40, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Philippe Vaucher <philippe.vaucher@gmail.com>

>> I think you need boths. Do you think the Ruby documentation I linked is bad?
> 
> It's worse than the ELisp manual, yes.

In terms of visual contrast/cues, it is not.

On a separate note, I've tried to work on syntax highlighting for the manual but there are issues with texinfo where Gavin prefers not tagging all the symbols and I can't help much with the solution he has in mind.

Also:

> On May 3, 2020, at 4:54, Philippe Vaucher <philippe.vaucher@gmail.com> wrote:
> 
> For quite a lot of people, this page http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler to use & learn from than this page https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html
> 
> It does not mean that the later page is bad, it is actually more complete, better documented, already grouped in topics... and yet, a lot of people prefer the first page... straight to the point, simple examples.

I, for one, prefer the ergoemacs approach.

• I can read the page in a reasonable amount of time,
• I can figure out what functions do and how to use them
• I can quickly imagine new ways of doing things

• and I can refer to the manual when I want deeper knowledge

In fact, when I can't make sense of the manual or find my way in it, I usually check the ergoemacs pages.

As mentioned above, I think the ergoemacs pages work well because of the way they use visual cues. We don't have many visual cues in the manual. That makes the manual uselessly more difficult to read.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 1307 bytes --]

>>> I think you need boths. Do you think the Ruby documentation I linked is
bad?
>> It's worse than the ELisp manual, yes.
> In terms of visual contrast/cues, it is not.

>>> For quite a lot of people, this page
http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler to
use & learn from than this page
https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html
>>> It does not mean that the later page is bad, it is actually more
complete, better documented, already grouped in topics... and yet, a lot of
people prefer the first page... straight to the point, simple examples.

> • I can read the page in a reasonable amount of time,
> • I can figure out what functions do and how to use them
> • I can quickly imagine new ways of doing things
> • and I can refer to the manual when I want deeper knowledge
> As mentioned above, I think the ergoemacs pages work well because of the
way they use visual cues. We don't have many visual cues in the manual.
That makes the manual uselessly more difficult to read.

Thanks for explaining my example. I find it very interesting/surprising
that many people here do not think that way. Our position is probably due
to exposure to many different workflows/languages, but I'm guessing here.

[-- Attachment #2: Type: text/html, Size: 1592 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Jean-Christophe Helary]

> On May 3, 2020, at 17:10, Philippe Vaucher <philippe.vaucher@gmail.com> wrote:
> 
> >>> I think you need boths. Do you think the Ruby documentation I linked is bad?
> >> It's worse than the ELisp manual, yes.
> > In terms of visual contrast/cues, it is not.
> 
> >>> For quite a lot of people, this page http://ergoemacs.org/emacs/elisp_string_functions.html is much simpler to use & learn from than this page https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html
> >>> It does not mean that the later page is bad, it is actually more complete, better documented, already grouped in topics... and yet, a lot of people prefer the first page... straight to the point, simple examples.
> 
> > • I can read the page in a reasonable amount of time,
> > • I can figure out what functions do and how to use them
> > • I can quickly imagine new ways of doing things
> > • and I can refer to the manual when I want deeper knowledge
> > As mentioned above, I think the ergoemacs pages work well because of the way they use visual cues. We don't have many visual cues in the manual. That makes the manual uselessly more difficult to read.
> 
> Thanks for explaining my example. I find it very interesting/surprising that many people here do not think that way.

I'm not sure about that.

It is natural to question propositions especially when such propositions do not relate directly to programming but to ergonomic/cognitive issues, which are highly (personal/social/historical/technological) context dependent.

> Our position is probably due to exposure to many different workflows/languages, but I'm guessing here.

Not so much in my case, but I would certainly remove the "many" in your sentence because it does read a bit condescending :)


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 367 bytes --]

> > Our position is probably due to exposure to many different
workflows/languages, but I'm guessing here.
>
> Not so much in my case, but I would certainly remove the "many" in your
sentence because it does read a bit condescending :)

Right, sorry again.

I have trouble finding non-offensive formulations. Can you help me
reformulate? Or maybe skip it altogether?

[-- Attachment #2: Type: text/html, Size: 464 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Jean-Christophe Helary]

> On May 3, 2020, at 17:28, Philippe Vaucher <philippe.vaucher@gmail.com> wrote:
> 
> > > Our position is probably due to exposure to many different workflows/languages, but I'm guessing here.
> >
> > Not so much in my case, but I would certainly remove the "many" in your sentence because it does read a bit condescending :)
> 
> Right, sorry again.
> 
> I have trouble finding non-offensive formulations.

You seem to be doing alright, just like anybody else :)

If you worry about that, just don't send your mails right away. Take 5mn off the keyboard and read them once more before you send them.


> Can you help me reformulate? Or maybe skip it altogether?

Just remove the "many":

→ Our position is probably due to exposure to different workflows/languages



Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 512 bytes --]

> > I have trouble finding non-offensive formulations.
>
> You seem to be doing alright, just like anybody else :)
>
> If you worry about that, just don't send your mails right away. Take 5mn
off the keyboard and read them once more before you send them.

Very good advice!

> > Can you help me reformulate? Or maybe skip it altogether?
>
> Just remove the "many":
>
> → Our position is probably due to exposure to different
workflows/languages

True, that sounds better.

Thanks!
Philippe

[-- Attachment #2: Type: text/html, Size: 691 bytes --]

Re: Imports / inclusion of s.el into Emacs

[조성빈]

Eli Zaretskii <eliz@gnu.org> 작성:

>> From: Philippe Vaucher <philippe.vaucher@gmail.com>
>> Date: Sat, 2 May 2020 17:20:16 +0200
>> Cc: Eli Zaretskii <eliz@gnu.org>, Emacs developers <emacs-devel@gnu.org>,
>> 	Stefan Monnier <monnier@iro.umontreal.ca>, Richard Stallman <rms@gnu.org>, Dmitry Gutov <dgutov@yandex.ru>
>>
>> Say I want to quickly remember how strings work in Emacs, how you  
>> manipulate them, etc. With a few
>> keystrokes I end up on
>> https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html.  
>> From there I
>> have to select a topics, read it partially, go back, read another topic,  
>> all that while skipping paragraphs most
>> of the time (but that can be improved with my keep-lines trick).
>>
>> There is no manual entry where I see all these classified string  
>> functions at once, and "C-h d string" makes
>> my emacs so laggy that I cannot use it, also most of the entries are  
>> irrelevant.
>
> It sounds very strange to me that the method of learning about strings
> is by looking at the list of string-related APIs.

Unless one doesn’t have any programming experience, IMO one can learn
(or refresh old memory) of how strings work from the API list.

(And if that doesn’t work, that’s a design failure. Function names exist  
for meanings.)

> If you want to
> learn about strings in Emacs, you should read the entire chapter
> "Strings and Characters", including all of its sections and
> subsections.  This is how a certain topic should be learned for the
> first time, or after a long break when you no longer sure you remember
> enough of the basics.

Does every Emacs user have to go through the long manual to write some  
short functions?

>> Now compare that to https://ruby-doc.org/core-2.6/String.html. Do you  
>> see how faster that is or is just my
>> lack of habit of using the manual?
>
> What should I look at there?  I see a very long list of functions,
> each one followed by 5 to 10 lines of description.  How is it
> different from what we have in the ELisp manual?
>
>> Or with https://ruby-doc.org/core-2.6/Thread.html, see how you directly
>> have an example of common usage?
>
> How can a single example of "typical usage" help you understand a
> complex topic such as threads?

A single example is of course not for understanding the whole complex
topic, but it serves as a reminder.

>  And what is "typical usage" of
> threads, anyway?  I could use threads in umpteen different ways, all
> of them "typical".  What am I missing?
>
>> I guess you could argue that I'm not used to having to read big chunks  
>> of text to get the information I'm
>> looking for, that's I think a valid criticism.
>
> What big chunks of text are you alluding to?  Are you saying that the
> smaller is the documentation of a function, the better?
>
> In the ELisp manual we describe each function with as many words as
> required to describe its functionality.  (There are people who think
> we need to tell more.)  We also provide "continuity" text to serve as
> a "glue" which is supposed to help the reader understand the topic
> better and see each API in its context and relation to others.

Yes, that’s great when one doesn’t know about programming in general or
how Emacs work. It’s not great when you forgot what string API Emacs
provides, and trying to find out what operations exist, to write my
custom interactive function in the cleanest way possible.

> "Manuals" that are just lists of APIs with minimum explanatory text,
> a-la JavaDoc, are _bad_ manuals.  They don't tell you enough about the
> topics for you to understand when use one class of APIs and when to
> use another.  If you want to see a representative of such bad manuals,
> look at the GTK docs.  Is this what you'd like to see in the ELisp
> manual?

No, of course not. I’m pretty sure nobody wants a manual like that.
Just that it would be better if there’s a good way to view all of
the string operations that Emacs supports without text.

I think this is going too far from the original discussion.
The original discussion was adding prefixes to *some* functions
that aren’t that controversial. (Like make-string, for example.)

One of the arguments was that it’s easier to find, remind & predict,
which aids in discoverability.

The prefix-scheme means that, when one tries to write code, it’s easy
to guess based on faint memory or just from pure speculation. And
that’s important, especially in Emacs, because one shouldn’t need to
read through the manual just to write some interactive functions for
a personal configuration.

I think a lot of people here are misunderstanding why this is a
virtue -  understandable since most here are already Elisp masters,
can guess make-string or split-string based on memory, write out
code without searching. But that’s not true for a lot of Emacs users,
including a lot of users who have used Emacs for a long time without
writing lots of Elisp or written any packages, and for those people,
it’s very useful to guess and write code.

Re: Imports / inclusion of s.el into Emacs

[Eli Zaretskii]

> From: 조성빈 <pcr910303@icloud.com>
> Date: Sun, 3 May 2020 03:01:03 +0900
> Cc: Philippe Vaucher <philippe.vaucher@gmail.com>,
>  Dmitry Gutov <dgutov@yandex.ru>,
>  Richard Stallman <rms@gnu.org>,
>  joaotavora@gmail.com,
>  monnier@iro.umontreal.ca,
>  emacs-devel@gnu.org
> 
> > It sounds very strange to me that the method of learning about strings
> > is by looking at the list of string-related APIs.
> 
> Unless one doesn’t have any programming experience, IMO one can learn
> (or refresh old memory) of how strings work from the API list.

I doubt that.

> > If you want to
> > learn about strings in Emacs, you should read the entire chapter
> > "Strings and Characters", including all of its sections and
> > subsections.  This is how a certain topic should be learned for the
> > first time, or after a long break when you no longer sure you remember
> > enough of the basics.
> 
> Does every Emacs user have to go through the long manual to write some  
> short functions?

Of course not.

The Emacs manuals are designed to support 2 general use cases:

  . reading on a subject (like strings) for the first time, i.e. using
    the manual as a textbook
  . finding quickly the details about some small topic, i.e. using the
    manual as a reference

In the first use case you are supposed to read the chapter, in the
second you are supposed to use index-search for that topic you have in
mind (the topic could also be the name of a function or a key
sequence).  That second use case would be the case of writing a short
function.

> > How can a single example of "typical usage" help you understand a
> > complex topic such as threads?
> 
> A single example is of course not for understanding the whole complex
> topic, but it serves as a reminder.

A reminder of a single example can only be useful if that example
happens to be what you want.  For a subject as broad as strings or
threads, this is unlikely to be the case.

The ELisp manual includes quite a few examples, each one where we want
to explain some topic by showing concrete code.

> > In the ELisp manual we describe each function with as many words as
> > required to describe its functionality.  (There are people who think
> > we need to tell more.)  We also provide "continuity" text to serve as
> > a "glue" which is supposed to help the reader understand the topic
> > better and see each API in its context and relation to others.
> 
> Yes, that’s great when one doesn’t know about programming in general or
> how Emacs work. It’s not great when you forgot what string API Emacs
> provides, and trying to find out what operations exist, to write my
> custom interactive function in the cleanest way possible.

I invite you to read the chapter about strings and characters in the
ELisp manual, and also examine the index entries in that chapter.
Then we could talk specifics.  Otherwise I have no way of connecting
what you say with what is actually in the manual.

> > "Manuals" that are just lists of APIs with minimum explanatory text,
> > a-la JavaDoc, are _bad_ manuals.  They don't tell you enough about the
> > topics for you to understand when use one class of APIs and when to
> > use another.  If you want to see a representative of such bad manuals,
> > look at the GTK docs.  Is this what you'd like to see in the ELisp
> > manual?
> 
> No, of course not. I’m pretty sure nobody wants a manual like that.
> Just that it would be better if there’s a good way to view all of
> the string operations that Emacs supports without text.

And I'm sure such a list will be almost useless.  Nevertheless, I'm
not opposed to having such a list somewhere.  I just don't think it's
an efficient way of finding information quickly and efficiently --
except if one already knows the name of the function.

But I already said that several times, so it's time to stop repeating
myself.

> I think a lot of people here are misunderstanding why this is a
> virtue -  understandable since most here are already Elisp masters,
> can guess make-string or split-string based on memory, write out
> code without searching.

You are mistaken: I'm not such a master.  Like Stefan, I can never
remember whether its string-multibyte-p or multibyte-string-p.  I use
the manual and the doc strings all the time because I don't remember
all those details.  So what I wrote is based on personal experience of
looking up and finding this information quickly and efficiently.

Re: Imports / inclusion of s.el into Emacs

[Arthur Miller]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: 조성빈 <pcr910303@icloud.com>
>> Date: Sun, 3 May 2020 03:01:03 +0900
>> Cc: Philippe Vaucher <philippe.vaucher@gmail.com>,
>>  Dmitry Gutov <dgutov@yandex.ru>,
>>  Richard Stallman <rms@gnu.org>,
>>  joaotavora@gmail.com,
>>  monnier@iro.umontreal.ca,
>>  emacs-devel@gnu.org
>> 
>> > It sounds very strange to me that the method of learning about strings
>> > is by looking at the list of string-related APIs.
>> 
>> Unless one doesn’t have any programming experience, IMO one can learn
>> (or refresh old memory) of how strings work from the API list.
>
> I doubt that.

Why? If I know what I want to do, but I don't know or remember how stuff is
named organized in an API, I could just fuzzy search, or just look at
completion list and by seeing the names of available stuff I can
guess at which one I want. Unless it is called make-process &
call-process in which case I have to look it up the manual and see what is
which one I want. But if names reflect what stuff does, i.e.
self-documenting code, then by just looking at the list can save some time.

In such context docs like Java docs are not so bad, but context-aware
completion or search like Helm would probably be even better.

> You are mistaken: I'm not such a master.  Like Stefan, I can never
> remember whether its string-multibyte-p or multibyte-string-p.  I use
> the manual and the doc strings all the time because I don't remember
> all those details.  So what I wrote is based on personal experience of
> looking up and finding this information quickly and efficiently.

I try delibarately to remember as little as possible and to search as
much as I can. Not just APIs, but pretty much anything nowdays. There is
so much information around, and everything (in software community) is
developing fast and contionously, so remembering APIs seems as a
waste of energy.

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > There is
  > so much information around, and everything (in software community) is
  > developing fast and contionously, so remembering APIs seems as a
  > waste of energy.

We must not change Emacs that fast.  It would be a disaster:
it would break users' code.  Incompatible changes must occur
at a snail's pace.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

>> Unless one doesn’t have any programming experience, IMO one can learn
>> (or refresh old memory) of how strings work from the API list.
> I doubt that.

While it doesn't hold for all kinds of data types (where you may have
to read a bit more of the doc to understand it all), I do expect this to
be true for strings.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sat, May 2, 2020 at 10:02 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> >> Unless one doesn’t have any programming experience, IMO one can learn
> >> (or refresh old memory) of how strings work from the API list.
> > I doubt that.
>
> While it doesn't hold for all kinds of data types (where you may have
> to read a bit more of the doc to understand it all), I do expect this to
> be true for strings.

Depends where which language you're coming from. C strings
are quite different than other languages.

I think that unfortunately, people expect to be able to learn a
language on the fly from little snippets, say, the way we all
"learned" to code really bad javascript.  I think Emacs's manual
is a useful tug in the other direction.  Those "large blocks of
text" that someone complained about, are exactly what makes
it good, it tries to give context, to hold your hand.  Of course
everyone likes the feeling of impatiently "learning" a language by
autocomplete of object methods. They expect to type "foo"<dot>
and lo, here are the methods of a string. But in Lisp, function
is king, so that doesn't work. I suppose whoever designed elisp
or its ancestors didn't want to type  "string-" all the time, a good
decision IMO. That person probably didn't have internet and
read a great deal more manuals.

João

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What should I look at there?  I see a very long list of functions,
  > each one followed by 5 to 10 lines of description.  How is it
  > different from what we have in the ELisp manual?

Maybe we are not succeeding in informing new Lisp programmers
that we have good manuals, or that it is worth trying them first.



-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > With a few keystrokes I end up on
  > https://www.gnu.org/software/emacs/manual/html_node/elisp/Strings-and-Characters.html.
  > >From there I have to select a topics, read it partially, go back, read
  > another topic,

Could you please give a concrete example of what you mean by this, and
why you find it inconvenient?  I can imagine several possible meanings
for your words, referring to different conceptual levels.

Do you know that if you just start typing SPC SPC SPC... in Info
you will go through the whole contents of that chapter in linear order?

		   all that while skipping paragraphs most of the time (but
  > that can be improved with my keep-lines trick).

It would take mind-reading for documenation to exclude all the parts
of a general topic that you're not interested in at the moment.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Stefan Monnier]

> Some of us are trying to tell you is that the Elisp manual is a
> gold mine, too. If you see parts where it is not, report it.
> Lisp languages are traditionally organized differently.

Why choose?
Is `multibyte-string-p` really a better name than `string-multibyte-p`?

Structuring the name space does not have to come at the cost of
something else, AFAICT.  Sometimes/often we won't be able to choose
a good structuring, but even in those cases we're no worse than now.


        Stefan

Re: Imports / inclusion of s.el into Emacs

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Renaming 'concat' seems also like spurious inconvenience in the name
>> of rigidity -- the Lisp equivalent of bureaucratese.
>
> Funny, I find `concat` a perfect example of a bad name because it
> doesn't say what it is that it's concatenating nor how.
>
> If I try to imagine myself a "fresh new Elisp programmer" I can't
> intuitively know which of `append` or `concat` does what I want.

After giving s.el a cursory look, it seems like it either comes from a
Javascript or a Clojure background, and provides names for common Emacs
Lisp functions that are easier to remember for people with those
backgrounds.

There's nothing wrong with that -- like most people, I hate learning
things, too.  And the number of Javascript/Clojure programmers eclipses
the number of Emacs Lisp programmers by several orders of magnitude, so
it's no wonder that it's a popular library.

But it leads to non-idiomatic Emacs Lisp code.  For instance, variadic
functions aren't common in those languages, so you end up with oddities
like s-prepend and s-append, and if that's what the user thinks is
available, then you end up with code like

  (s-append (s-append (s-append "foo" "bar") "zot") "gazonk")

instead of

  (concat "foo" "bar" "zot" "gazonk")

That is, s.el leads to bad Emacs Lisp code, and I don't think it's a
good idea to have something like that (even with less lodash-like name)
in Emacs.

And I think that, since Lisp is obviously better than Javascript, we
should just tell people to avoid writing Javascript-in-Lisp and instead,
unfortunately, learn some Lisp.  It's better in the long run for everybody.

As for the common "but just add aliases" to everything, I'm agin it.
Programming is 95% reading old code, and having to remember a number of
different names for common functions (and verify that they are really
equivalent) is not a good language culture to strive for.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 815 bytes --]

>
> But it leads to non-idiomatic Emacs Lisp code.  For instance, variadic
> functions aren't common in those languages, so you end up with oddities
> like s-prepend and s-append, and if that's what the user thinks is
> available, then you end up with code like
>
>   (s-append (s-append (s-append "foo" "bar") "zot") "gazonk")
>
> instead of
>
>   (concat "foo" "bar" "zot" "gazonk")
>
> That is, s.el leads to bad Emacs Lisp code, and I don't think it's a
> good idea to have something like that (even with less lodash-like name)
> in Emacs.
>

No, there is `s-concat`. People would write

(s-concat "foo" "bar" "zot" "gazonk")

That said it was already said that `s-prepend` and `s-append` were likely
to not make it to Emacs core, which is fine by me. It's the other functions
I'm more interested in.

Philippe

[-- Attachment #2: Type: text/html, Size: 1288 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Lars Ingebrigtsen]

Philippe Vaucher <philippe.vaucher@gmail.com> writes:

> No, there is `s-concat`. People would write
>
> (s-concat "foo" "bar" "zot" "gazonk")

Probably not if they'd learned the (nonsensical) prepend/append versions
of the functions first.

This design is all over the s.el library -- for instance, there's a
s-chop-suffix function, and a s-chop-suffixes function, where a natural
Lisp design would be to have the suffixes a &rest in a single function.

It's understandable for people to want to port over idioms they are
familiar with from other programming languages, but it would (usually)
be a mistake to incorporate them into the language.  s.el is "let's
pretend Emacs Lisp is Javascript and/or Clojure"; in a couple of years
there'll be an s.el that is "let's pretend Emacs Lisp is Rust or Go" and
we'll have the same aliasing discussion all over again.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It's understandable for people to want to port over idioms they are
  > familiar with from other programming languages, but it would (usually)
  > be a mistake to incorporate them into the language.  s.el is "let's
  > pretend Emacs Lisp is Javascript and/or Clojure";

Emacs Lisp can be improved, and we may have found some good renamings of
string and regexp functions in this discussion.

However, mimicking Javascript or Clojure is a misguided goal.  We don't
want new functions or new names for that.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Lars Ingebrigtsen]

(The one thing s.el has, though, is an excellent overview of the
functions: Just a list of all the functions, and examples that show how
they work.  This is what most people who are trying to use a new
language wants -- very few people are interested in reading a manual.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Imports / inclusion of s.el into Emacs

[Jean-Christophe Helary]

> On May 3, 2020, at 17:17, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> 
> (The one thing s.el has, though, is an excellent overview of the
> functions: Just a list of all the functions, and examples that show how
> they work.  This is what most people who are trying to use a new
> language wants -- very few people are interested in reading a manual.)

And that is *very much* one of the things that is being discussed here :)


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Imports / inclusion of s.el into Emacs

[Lars Ingebrigtsen]

Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
writes:

>> On May 3, 2020, at 17:17, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>> 
>> (The one thing s.el has, though, is an excellent overview of the
>> functions: Just a list of all the functions, and examples that show how
>> they work.  This is what most people who are trying to use a new
>> language wants -- very few people are interested in reading a manual.)
>
> And that is *very much* one of the things that is being discussed here :)

Perhaps we should just have an in-Emacs solution for the discoverability
problem.

For instance, the doc strings of all the string related functions could
have a link "For more string functions, click here" and then you're
taken to a (info?) buffer that lists all the functions in the style of
s.el's overview.  That is, just function signatures and examples, no
in-depth talk at all.

(The functions could link to the "real" manual pages, of course.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

For instance, the doc strings of all the string related functions could
> have a link "For more string functions, click here" and then you're
> taken to a (info?) buffer that lists all the functions in the style of
> s.el's overview.  That is, just function signatures and examples, no
> in-depth talk at all.
>
> (The functions could link to the "real" manual pages, of course.)


This would already be a huge improvement, given there's also a
completing-read version of that page that'd act like "C-h f" but first you
select a topic.

I'd like to point out that we'd get this behavior for free out of the
current "C-h f" it there was aliases :-P Ok I guess I said this enough time
now :-)

Philippe

[-- Attachment #2: Type: text/html, Size: 1076 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Jean-Christophe Helary]

> On May 3, 2020, at 18:39, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> 
> Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
> writes:
> 
>>> On May 3, 2020, at 17:17, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>>> 
>>> (The one thing s.el has, though, is an excellent overview of the
>>> functions: Just a list of all the functions, and examples that show how
>>> they work.  This is what most people who are trying to use a new
>>> language wants -- very few people are interested in reading a manual.)
>> 
>> And that is *very much* one of the things that is being discussed here :)
> 
> Perhaps we should just have an in-Emacs solution for the discoverability
> problem.
> 
> For instance, the doc strings of all the string related functions could
> have a link "For more string functions, click here" and then you're
> taken to a (info?) buffer that lists all the functions in the style of
> s.el's overview.  That is, just function signatures and examples, no
> in-depth talk at all.
> 
> (The functions could link to the "real" manual pages, of course.)

The helpful package does a lot of extra linking and formatting. Have you checked it ?


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Imports / inclusion of s.el into Emacs

[Lars Ingebrigtsen]

Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
writes:

> The helpful package does a lot of extra linking and formatting. Have
> you checked it ?

Nope.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Imports / inclusion of s.el into Emacs

[Jean-Christophe Helary]

> On May 5, 2020, at 16:59, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> 
> Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
> writes:
> 
>> The helpful package does a lot of extra linking and formatting. Have
>> you checked it ?
> 
> Nope.

I currently use helpful and which-key to discover more information than what is already available.

If you have any other package to suggest, please do, I am very interested.


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

RE: Imports / inclusion of s.el into Emacs

[Drew Adams]

> >> The helpful package does a lot of extra linking
> >> and formatting. Have you checked it ?
> > Nope.
> 
> I currently use helpful and which-key to discover more
> information than what is already available.
> 
> If you have any other package to suggest, please do,
> I am very interested.

Not sure what the overall question is that you're
asking.  But if you're asking about other packages
that provide info similar to what helpful and
which-key provide:

* Icicles key completion provides some that's
  similar to what which-key provides.

* info+.el and help-fns+.el provide some that's
  similar to what helpful.el provides

https://www.emacswiki.org/emacs/Icicles_-_Key_Completion

https://www.emacswiki.org/emacs/InfoPlus

https://www.emacswiki.org/emacs/HelpPlus

Re: Imports / inclusion of s.el into Emacs

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 317 bytes --]

On Sun, May 3, 2020, 08:42 Philippe Vaucher <philippe.vaucher@gmail.com>
wrote:

>
>>
> That said it was already said that `s-prepend` and `s-append` were likely
> to not make it to Emacs core, which is fine by me. It's the other functions
> I'm more interested in
>

Can you make a top-ten?

João

>

[-- Attachment #2: Type: text/html, Size: 1037 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 746 bytes --]

> Can you make a top-ten?

Aliases:

split-string aliased as string-split
downcase aliased as string-downcase
upcase aliased as string-upcase
format aliased as string-format
concat aliased as string-concat

New functions (with string- instead of s-):

s-truncate (len s)
s-chomp (s)
s-left (len s)
s-right (len s)
s-repeat (num s)
s-capitalize (s)
s-reverse (s)
s-contains-p (needle s &optional ignore-case)
s-blank-p (s)

Nice to have, but do not really care:

s-index-of (needle s &optional ignore-case)
s-center (len s)
s-pad-left (len padding s)
s-pad-right (len padding s)
s-chop-suffixes (suffixes s)
s-chop-prefixes (prefixes s)
s-shared-start (s1 s2)
s-shared-end (s1 s2)
s-prepend (prefix s)
s-append (suffix s)

Hope it helps,
Philippe

[-- Attachment #2: Type: text/html, Size: 1265 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 257 bytes --]

> > Can you make a top-ten?
>
> Aliases:

I forgot:

substring aliased as string-substring

> New functions (with string- instead of s-):
>
> s-truncate (len s)

This one can be moved to "nice to have" given substring takes negative
arguments.

Philippe

>

[-- Attachment #2: Type: text/html, Size: 641 bytes --]

Re: Imports / inclusion of s.el into Emacs

[Stefan Kangas]

Philippe Vaucher <philippe.vaucher@gmail.com> writes:

> Aliases:
>
> split-string aliased as string-split

This is a better name.  FWIW, I think the old one should be deprecated too.

> downcase aliased as string-downcase
> upcase aliased as string-upcase
> format aliased as string-format
> concat aliased as string-concat

I think the first two are used rarely enough to warrant a prefix.  No
opinion about the other two, since they're used often enough that
you'll just learn them.

> New functions (with string- instead of s-):
[...]
> Nice to have, but do not really care:

Could you please summarize what the quoted functions functions do?

Best regards,
Stefan Kangas

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sun, May 3, 2020 at 11:55 AM Stefan Kangas <stefan@marxist.se> wrote:
> Philippe Vaucher <philippe.vaucher@gmail.com> writes:
>
> > Aliases:
> >
> > split-string aliased as string-split
>
> This is a better name.  FWIW, I think the old one should be deprecated too.

People coming from Common Lisp and not Clojure will find 'split-string'
more appropriate.

Now really, I don't think this "prefix with the type name" makes
any sense.  When I learned Lisp coming from Java, it bothered me,
yes. But I got over it, quickly. Functions in lisp are generic, or at least
duck-typed. If in Haskell or Ruby the name of the operated-on type
always  matches the namespace qualifier, good for them I guess. I know
0 Haskell, but last I checked those Ruby functions aren't top-level
functions, they're methods on a String object. So that just reflects a
different way to look at OO.  I often use it when in those languages, but
in Lisp there's a different take to it.  So the namespace qualifier doesn't
necessarily bear any direct relation to type of the object being operated
on [1]. Sometimes it does, yes, but it's accidental. Shoving this foreign
convention down Lisp's throat is cruel. And ignorant, sorry. (ignorance
isn't a defect, it's a default). The point of doing it, I suppose, is that it
would bring in lots of newcomers onboard. But shouldn't we rather be
welcoming them by efficiently showing them them the joys of
Javalessness instead?  Do we need a manual with pretty pictures
and fancy fonts? Let's do it then.

OK, ranting over. In CL-land, there is the very good split-sequence[3],
which is a generalized split-string (and compatible with the "split the
empty string" case). I propose we get that instead.  If we're lucky, that
CL code is directly translatable to Elisp (seems to be from a cursory
inspection). Yeah call it seq-split if you really must drink the kool-aid.

[1]: The lack of a package system in Elisp (not package.el, mind you
that's a different thing) adds to the confusion, but it's really the same.
[2]: http://quickdocs.org/split-sequence/

João

Re: Imports / inclusion of s.el into Emacs

[조성빈]

João Távora <joaotavora@gmail.com> 작성:

> On Sun, May 3, 2020 at 11:55 AM Stefan Kangas <stefan@marxist.se> wrote:
>> Philippe Vaucher <philippe.vaucher@gmail.com> writes:
>>
>>> Aliases:
>>>
>>> split-string aliased as string-split
>>
>> This is a better name.  FWIW, I think the old one should be deprecated  
>> too.
>
> People coming from Common Lisp and not Clojure will find 'split-string'
> more appropriate.
>
> Now really, I don't think this "prefix with the type name" makes
> any sense.  When I learned Lisp coming from Java, it bothered me,
> yes. But I got over it, quickly. Functions in lisp are generic, or at least
> duck-typed. If in Haskell or Ruby the name of the operated-on type
> always  matches the namespace qualifier, good for them I guess. I know
> 0 Haskell, but last I checked those Ruby functions aren't top-level
> functions, they're methods on a String object. So that just reflects a
> different way to look at OO.  I often use it when in those languages, but
> in Lisp there's a different take to it.  So the namespace qualifier doesn't
> necessarily bear any direct relation to type of the object being operated
> on [1]. Sometimes it does, yes, but it's accidental. Shoving this foreign
> convention down Lisp's throat is cruel. And ignorant, sorry. (ignorance
> isn't a defect, it's a default).

OK, so nobody is really forcing that ’split-string’ should be aliased
as ’string-split’. What people want, is *not* a OO-based naming scheme
- it just needs to be consistent. If you think ‘traditional’ Lisps should
follow a verb-noun scheme for non-polymorphic functions and only verb for
polymorphic functions, then that’s fine. Then the user can predict that…
when one wants to split a string, one can search it with ones that start
with ’split’.

The problem here is current Elisp isn’t predictable enough. Is it
string-trim or  is it trim-string? Is it string-join or is it join-string?
Is it string-split or is it split-string? And that’s the problem this
proposal is trying to solve.

BTW, a quick check from sly shows me that cl:split-string doesn’t exist
- while cl:string-trim, cl:string-left-trim, cl:string-right-trim exists.

Which probably means that it’s not really a foreign convention in Lisp.
(And Elisp already adopted the convention of prefixing the module name
  in the front, which was the reason this discussion started.)

> The point of doing it, I suppose, is that it
> would bring in lots of newcomers onboard. But shouldn't we rather be
> welcoming them by efficiently showing them them the joys of
> Javalessness instead?

Seriously, a consistent std is *not* Java. See almost every programming
language - consistency is something that people value. It’s not always
the answer, but it’s something that people value & appreciate.

> Do we need a manual with pretty pictures
> and fancy fonts? Let's do it then.
>
> OK, ranting over. In CL-land, there is the very good split-sequence[3],
> which is a generalized split-string (and compatible with the "split the
> empty string" case). I propose we get that instead.  If we're lucky, that
> CL code is directly translatable to Elisp (seems to be from a cursory
> inspection). Yeah call it seq-split if you really must drink the kool-aid.
>
> [1]: The lack of a package system in Elisp (not package.el, mind you
> that's a different thing) adds to the confusion, but it's really the same.
> [2]: http://quickdocs.org/split-sequence/
>
> João

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sun, May 3, 2020 at 3:52 PM 조성빈 <pcr910303@icloud.com> wrote:
>
> João Távora <joaotavora@gmail.com> 작성:
>

> The problem here is current Elisp isn’t predictable enough. Is it
> string-trim or  is it trim-string? Is it string-join or is it join-string?
> Is it string-split or is it split-string? And that’s the problem this
> proposal is trying to solve.

OK I see. And I agree to a point. It's just Lisp doesn't care so
much about it.  For a newcomer to Perl, there is _no way_,
unless super lucky, to know that 'chomp' will remove the boring
parts of a string.  Predictability from names is not useless, but
is overrated, because it will only tell you so much and it works
better in some languages versus others.

And introducing new conventions that clash with the existing
idiosyncrasy is _not_ the best way to solve it.

And people have demonstrated here over and over how it fails.
Say you don't like an hypothetical "frob" and you decide to make
a "string-frob" or "frob-string".  Now can you predict from that name
if it's destructive or not? And can you use return value? What if
you can "frob" more things than a string? Sequences, multi-dimensional
arrays, characters?  What if frob already exists and can be both
a verb or a property (like "match"). It's a losing game.  Yes, some
people have played it (and mostly lost) and these names are burned
in the namespace. Because culture.  Nothing to do about those.
If you start renaming things for predictability it'll:

1. be quite hard to get right, if at all possible.  And doing it on the
type is a bad idea, because polymorphism;
2. make many new symbols that will confuse existing users.
Even new users ask: why does this have two/three names?;
3. encourage you to deprecate and eventually remove names,
which breaks programs and is a shame.

> BTW, a quick check from sly shows me that cl:split-string doesn’t exist
> - while cl:string-trim, cl:string-left-trim, cl:string-right-trim exists.
> Which probably means that it’s not really a foreign convention in Lisp.
> (And Elisp already adopted the convention of prefixing the module name
>   in the front, which was the reason this discussion started.)

CL is foreign to Elisp, whether you like it or not (I don't, of course
but hey...). It's just not _as_ foreign. If we're going to import things
from CL I'd rather bring in specific CL features like restarts, packages,
streams, multiple values.  Definitely _not_ function names. I'm not
in love with CL's names: I just know them and don't question them
so much, like names of people. Lisp is a language of symbols and
symbols have names you learn. The function of a symbol is only
_one_ of its many facets anyway. So when bringing new symbols
into the world you shouldn't try to be too clever about the name.
Be just moderately clever.

BTW Aren't there more interesting parts of Clojure other than the
symbol names?

And "module name" is a broken convention. It's the best we have
so we might as well, but what is the module for 'car'?  Again, for
the sixteenth time, if we had packages, it would be solved: nobody
touches the :EL package but everyone can make their own fancy
"predictable" utilities package.

_THE_ way to solve is is to transform s.el into magnar-string.el
_BUT_ because of the lack of a package system, that means we
have to (magnar-string-truncate 42 s) which is a pain in the s, I'll readily
admit that. You should be able to say: "I want to use magnar-string.el
_here_ because great Clojure god".  And then type (truncate 42 s),
(left s 3), and be done with it. Or use the local "s" nickname and type
(s:truncate 42 s) and (s:left s 3). But you can't, so you want to shove
it, figuratively ;-), in the main library but that's really hard and
controversial and for every compromise you make you start
doing the original s.el library less and less justice.

> Seriously, a consistent std is *not* Java. See almost every programming
> language - consistency is something that people value. It’s not always
> the answer, but it’s something that people value & appreciate.

A std that changes according to the language-du-jour is not a std
by definition.

João

Re: Imports / inclusion of s.el into Emacs

[조성빈]

João Távora <joaotavora@gmail.com> 작성:

> On Sun, May 3, 2020 at 3:52 PM 조성빈 <pcr910303@icloud.com> wrote:
>> João Távora <joaotavora@gmail.com> 작성:
>
>> The problem here is current Elisp isn’t predictable enough. Is it
>> string-trim or  is it trim-string? Is it string-join or is it join-string?
>> Is it string-split or is it split-string? And that’s the problem this
>> proposal is trying to solve.
>
> OK I see. And I agree to a point. It's just Lisp doesn't care so
> much about it.

Are you calling Elisp Lisp, or are you calling the Lisp family?
I personally think that both Scheme and Clojure has a pretty consistent
std naming scheme.

> For a newcomer to Perl, there is _no way_,
> unless super lucky, to know that 'chomp' will remove the boring
> parts of a string.

… which is definitely one reason why Perl is called ‘line noise’.
(Disclaimer: I have never used Perl enough to know it’s syntax, so
  I might be wrong about it.)

> Predictability from names is not useless, but
> is overrated, because it will only tell you so much and it works
> better in some languages versus others.
>
> And introducing new conventions that clash with the existing
> idiosyncrasy is _not_ the best way to solve it.

If you’re saying that new conventions should not clash with existing
conventions, that I can agree - I can’t agree that new conventions
shouldn’t clash with idiosyncrasies. That’s the point of conventions, to
remove idiosyncrasies!

> And people have demonstrated here over and over how it fails.
> Say you don't like an hypothetical "frob" and you decide to make
> a "string-frob" or "frob-string".  Now can you predict from that name
> if it's destructive or not?

Maybe it’s not a literal question, but in my ideal world, functions that
aren’t destructive should read as nouns (e.g. trim-string or string-trim),
and ones that are destructive should read as verbs (e.g. trimmed-string or
string-trimmed).

> And can you use return value? What if
> you can "frob" more things than a string? Sequences, multi-dimensional
> arrays, characters?  What if frob already exists and can be both
> a verb or a property (like "match").

Well, those bike shedding can happen in this mailing list.
(FYI, I don’t think that polymorphic functions like concat should be renamed
  now.)

> It's a losing game.  Yes, some
> people have played it (and mostly lost) and these names are burned
> in the namespace. Because culture.  Nothing to do about those.
> If you start renaming things for predictability it'll:
>
> 1. be quite hard to get right, if at all possible.

Yes, it’s hard to be perfect - but that’s not a reason to be better.

> And doing it on the
> type is a bad idea, because polymorphism;

I agree that prefixing types on polymorphic functions aren’t a good idea.

> 2. make many new symbols that will confuse existing users.
> Even new users ask: why does this have two/three names?;

Which has a plausible explanation: it’s been renamed. There are a lot of
functions/variables that are already in similar state, IIRC.

> 3. encourage you to deprecate and eventually remove names,

If the old name’s use reduces and eventually becomes close to zero, yes one
might deprecate & remove names.

> which breaks programs and is a shame.

My personal view on this is that if one hates breaking things, one should
just not upgrade. However, considering that Emacs don’t really remove much,
I think just marking them obsolete will work, without breaking anything.

>> BTW, a quick check from sly shows me that cl:split-string doesn’t exist
>> - while cl:string-trim, cl:string-left-trim, cl:string-right-trim exists.
>> Which probably means that it’s not really a foreign convention in Lisp.
>> (And Elisp already adopted the convention of prefixing the module name
>>   in the front, which was the reason this discussion started.)
>
> CL is foreign to Elisp, whether you like it or not (I don't, of course
> but hey...). It's just not _as_ foreign. If we're going to import things
> from CL I'd rather bring in specific CL features like restarts, packages,
> streams, multiple values.  Definitely _not_ function names.

I’m not saying that we should bring string-trim, string-left-trim, etc… It
was a response to this:

> Shoving this foreign
> convention down Lisp's throat is cruel. And ignorant, sorry.

It’s definitely not a foreign convention to Lisp.

> I'm not
> in love with CL's names: I just know them and don't question them
> so much, like names of people. Lisp is a language of symbols and
> symbols have names you learn. The function of a symbol is only
> _one_ of its many facets anyway. So when bringing new symbols
> into the world you shouldn't try to be too clever about the name.
> Be just moderately clever.

I think you don’t need to question them much is mostly because you’re
already familiar with the CL names. I actually feel this same problem when I
try to write some CL - I like the language, but the std is too inconsistent.

Cl21’s author Fukamachi explains this well on cl21.org:

> Is Common Lisp sufficiently well-designed? I don't think so. You use
> different functions to do the same thing to different data types (elt,  
> aref,
> nth). You have long names for commonly used macros (destructuring-bind,
> multiple-value-bind). There is no consistency in argument order (getf and
> gethash). To put it simply, the language is time-consuming to learn.

I think this applies similar to Elisp.

> BTW Aren't there more interesting parts of Clojure other than the
> symbol names?

Nobody is trying to bring Clojure inside Elisp, nor CL. This is not something
like make Elisp look like an another language project.

> And "module name" is a broken convention. It's the best we have
> so we might as well, but what is the module for 'car'?  Again, for
> the sixteenth time, if we had packages, it would be solved: nobody
> touches the :EL package but everyone can make their own fancy
> "predictable" utilities package.
>
> _THE_ way to solve is is to transform s.el into magnar-string.el
> _BUT_ because of the lack of a package system, that means we
> have to (magnar-string-truncate 42 s) which is a pain in the s, I'll  
> readily
> admit that. You should be able to say: "I want to use magnar-string.el
> _here_ because great Clojure god".  And then type (truncate 42 s),
> (left s 3), and be done with it. Or use the local "s" nickname and type
> (s:truncate 42 s) and (s:left s 3). But you can't, so you want to shove
> it, figuratively ;-), in the main library but that's really hard and
> controversial and for every compromise you make you start
> doing the original s.el library less and less justice.
>
>> Seriously, a consistent std is *not* Java. See almost every programming
>> language - consistency is something that people value. It’s not always
>> the answer, but it’s something that people value & appreciate.
>
> A std that changes according to the language-du-jour is not a std
> by definition.
>
> João

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sun, May 3, 2020 at 7:12 PM 조성빈 <pcr910303@icloud.com> wrote:

> Are you calling Elisp Lisp, or are you calling the Lisp family?
> I personally think that both Scheme and Clojure has a pretty consistent
> std naming scheme.

As far as I know, Scheme is minimalist, and has modules.  Common
Lisp and Elisp are not minimalist.  And Elisp doesn't have modules,
possibly because no-one ever thought it would be one of the most
popular Lisps. And Clojure as far as I understand, doesn't have
to deal with a class of problems such as mutable version of each
function, and crucially, has namespaces. So you have multiple
reverse's apparently.  One in clojure.string, another in clojure.core.
You _don't_ have string-reverse and seq-reverse (again, I may
be wrong).

> … which is definitely one reason why Perl is called ‘line noise’.
> (Disclaimer: I have never used Perl enough to know it’s syntax, so
>   I might be wrong about it.)

Perl has lots of flaws, but its terse names are not it, IMO.

> > And introducing new conventions that clash with the existing
> > idiosyncrasy is _not_ the best way to solve it.
>
> If you’re saying that new conventions should not clash with existing
> conventions, that I can agree - I can’t agree that new conventions
> shouldn’t clash with idiosyncrasies. That’s the point of conventions, to
> remove idiosyncrasies!

But the one you're proposing is too ambitious and flawed.

> Maybe it’s not a literal question, but in my ideal world, functions that
> aren’t destructive should read as nouns (e.g. trim-string or string-trim),
> and ones that are destructive should read as verbs (e.g. trimmed-string or
> string-trimmed).

Yes, I've used that convention in the past. C.f sort and sorted.
But it doesn't always work, because English is irregular.

> > And can you use return value? What if
> > you can "frob" more things than a string? Sequences, multi-dimensional
> > arrays, characters?  What if frob already exists and can be both
> > a verb or a property (like "match").
>
> Well, those bike shedding can happen in this mailing list.

It already does to a point.  But we tend to leave old stuff alone.

> (FYI, I don’t think that polymorphic functions like concat should be renamed
>   now.)

I would think the value of s.el comes from the consistency between
its members.  I would really just like for Elispers to be able to use it
without encroaching on the legacy namespace.

> > It's a losing game.  Yes, some
> > people have played it (and mostly lost) and these names are burned
> > in the namespace. Because culture.  Nothing to do about those.
> > If you start renaming things for predictability it'll:
> >
> > 1. be quite hard to get right, if at all possible.
>
> Yes, it’s hard to be perfect - but that’s not a reason to be better.

[To _not_ want to be better, you mean. ]

Yes, than I will turn that argument around.  We're not that
far away from perfection that it warrants new names.

> > And doing it on the
> > type is a bad idea, because polymorphism;
>
> I agree that prefixing types on polymorphic functions aren’t a good idea.

Once you start thinking of by operations rather than
by types, many functions become generic.

A big flaw in CL is not having gone far enough with generic
functions. i.e. most people agree that generic arithmetic
+, -, would be a tremendous gain.  I guess the money ran out :-),
so now +, - etc only work with numbers.  Should we rename them
number-+ and number-- in Elisp, for predictability? :-)

> > 2. make many new symbols that will confuse existing users.
> > Even new users ask: why does this have two/three names?;
>
> Which has a plausible explanation: it’s been renamed. There are a lot of
> functions/variables that are already in similar state, IIRC.

In specialized domains, but not in the base general purpose
library.

> > 3. encourage you to deprecate and eventually remove names,
>
> If the old name’s use reduces and eventually becomes close to zero, yes one
> might deprecate & remove names.

How can it become close to 0? What about old programs?

> My personal view on this is that if one hates breaking things, one should
> just not upgrade. However, considering that Emacs don’t really remove much,
> I think just marking them obsolete will work, without breaking anything.

Couldn't disagree more.  Old programs are useful.  I want them and their
dependencies to load cleanly, so I can study them. The real-world
equivalent to your idea is just throwing away old books.

> I’m not saying that we should bring string-trim, string-left-trim, etc… It
> was a response to this:
>
> > Shoving this foreign
> > convention down Lisp's throat is cruel. And ignorant, sorry.
>
> It’s definitely not a foreign convention to Lisp.

Doing it in the scale that you originally proposed would be. And
string-trim etc, already exist in Elisp.  In the "foreign" purgatory
of subr-x.el.  Curiously, there is this comment there:

;; Less commonly used functions that complement basic APIs, often implemented in
;; C code (like hash-tables and strings), and are not eligible for inclusion
;; in subr.el.
;; Do not document these functions in the lispref.
;; https://lists.gnu.org/r/emacs-devel/2014-01/msg01006.html

> I think you don’t need to question them much is mostly because you’re
> already familiar with the CL names. I actually feel this same problem when I
> try to write some CL - I like the language, but the std is too inconsistent.

It's the weight of history.  But in CL there are multiple attempts (curiously
not particularly successful) to write a modernized library. You can use
one of those, and they fit snugly in a package. You can make that
package your default package, if you wish and enjoy its benefits.

> Cl21’s author Fukamachi explains this well on cl21.org:
>
> > Is Common Lisp sufficiently well-designed? I don't think so. You use
> > different functions to do the same thing to different data types (elt,
> > aref,
> > nth). You have long names for commonly used macros (destructuring-bind,
> > multiple-value-bind). There is no consistency in argument order (getf and
> > gethash). To put it simply, the language is time-consuming to learn.

Oh, so you did find it. That is one of those attempts. But there have
been more, I think.

But I disagree with Fukamachi, or at least I think it is very naive. CL21 is an
interesting project, but CL is time-consuming because it is big, not because
of function names or signatures., but the many programming paradigms in
one language: functional, imperative, OO, meta, exceptions, systems
programming...  It's enormous. It's not CL21 that's going to "fix" that.
Really, the "learning names" problem is vastly overstated.  It's only for people
who use the language once in a while.  But that also happens for other
languages. I just quickly skim a manual before I get back to them. Of course
CL21's authors are trying to appeal to that new impatient blood that's flocking
to Clojure.  But they're flocking to Clojure mainly for the libraries, Java
interop, concurrency features, and the real-life jobs, not really the names.

By comparison, the only language as ambitious as CL that I know is C++.
Same hunger for general-purpose, multi-paradigm, meta-functional-system,
imperative-type-hungry language . All backward-compatible. A work of art.
 _That's_ a time-consuming programming language. And they're adding
and adding to it, to make it sexy for the cool kids.  Are they succeeding?
Open question.

> > BTW Aren't there more interesting parts of Clojure other than the
> > symbol names?
> Nobody is trying to bring Clojure inside Elisp, nor CL. This is not something
> like make Elisp look like an another language project.

Oh but I wouldn't think it that to be a bad thing, on the contrary.  I was
asking honestly: is the best thing in Clojure the names of the functions?
I find that hard to believe.

João

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

> Could you please summarize what the quoted functions functions do?

Do you mind taking a bit of time to look at https://github.com/magnars/s.el and
click on the names it's unclear?

I mean, I could copy paste it all here but that'd take me 10 minutes, while
I guess looking at that page takes about 1 minute.

Except if maybe you think it'd also help others? To be honest I'm surprised
that you think those names are not self-evident about what they do. Maybe
the fine details are not evident, but what it does in a rough sense should
be able to be deduced from their name.

Philippe

[-- Attachment #2: Type: text/html, Size: 746 bytes --]

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sun, May 3, 2020 at 11:06 AM Philippe Vaucher
<philippe.vaucher@gmail.com> wrote:
>
> > Can you make a top-ten?
>
> Aliases:
>
> split-string aliased as string-split
> downcase aliased as string-downcase
> upcase aliased as string-upcase
> format aliased as string-format
> concat aliased as string-concat

Don't do this.  Just learn these 5 names, or use apropos, as
has been suggested.  Though curiously, downcase is
string-downcase in CL, but that only really does work for
strings.

(BTW, a tangent, what do you think of CL's hyperspec reference?

Check out

http://www.lispworks.com/documentation/HyperSpec/Body/f_stg_up.htm

and

http://www.lispworks.com/documentation/HyperSpec/Body/c_string.htm

Do you think this organization would be useful? What if it had a cool
font and some cool logos? :-)  I ask because that put me off at first, it
looked mouldy.)

> New functions (with string- instead of s-):
>
> s-truncate (len s)

truncate-string-to-width (I got there with apropos truncate.*string btw).

> s-chomp (s)

Let's make a 'chomp' function.  I'm always looking for that. Though there
is a string-trim already (sigh).

> s-left (len s)

subseq

> s-right (len s)

Add an "from-end" arg to subseq?

> s-repeat (num s)

(cl-loop repeat num concat s)

> s-capitalize (s)

capitalize,  also works for chars.

> s-reverse (s)

reverse, also works for sequences

> s-contains-p (needle s &optional ignore-case)

string-match-p

> s-blank-p (s)

ugh. nil is a string?

Hope it helps,
João

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 2210 bytes --]

>> split-string aliased as string-split
>> downcase aliased as string-downcase
>> upcase aliased as string-upcase
>> format aliased as string-format
>> concat aliased as string-concat

>Don't do this.  Just learn these 5 names, or use apropos, as
> has been suggested.

Just curious, if these were to be introduced today, would you also name
them that way? What possible genericity is there about downcase/upcase?
Okay they work on strings and on chars, but it's pretty clear their topic
is "characters" (string) no?


> Check out
http://www.lispworks.com/documentation/HyperSpec/Body/f_stg_up.htm and
http://www.lispworks.com/documentation/HyperSpec/Body/c_string.htm

> Do you think this organization would be useful? What if it had a cool
> font and some cool logos? :-)  I ask because that put me off at first, it
> looked mouldy.)

Yes, that'd be very useful and it's more or less what we are asking to be
able to do. Again curious, do you think they made a mistake when they named
it `string-downcase` there and not `downcase`?


>> s-truncate (len s)
> truncate-string-to-width (I got there with apropos truncate.*string btw).

Okay then let's alias that to string-truncate.


>> s-left (len s)
>subseq
>> s-right (len s)
> Add an "from-end" arg to subseq?

Why is there substring if there is subseq then?


>> s-repeat (num s)
> (cl-loop repeat num concat s)

What if I don't know cl (like I a significant number of package authors) ?


>> s-capitalize (s)
> capitalize,  also works for chars.

Alias it to string-capitalize :-)


>> s-reverse (s)
> reverse, also works for sequences

Ah, this one I didn't think worked on strings. Thought you have to split,
reverse then join. I agree this can be left as a generic algorithm.


>> s-contains-p (needle s &optional ignore-case)
> string-match-p

That can work, tho the mental path of having to do a regexp match while
regexp-quoting the string just to do a plain text search feels very
inneficient.


>> s-blank-p (s)
> ugh. nil is a string?

"is the string blank?" is a concept in many languages, but I understand it
can be surprising at first. The idea is that very often when validating
user input you filter for nil, empy string or trimmed string.

[-- Attachment #2: Type: text/html, Size: 3047 bytes --]

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sun, May 3, 2020 at 2:47 PM Philippe Vaucher
<philippe.vaucher@gmail.com> wrote:
>
> >> split-string aliased as string-split
> >> downcase aliased as string-downcase
> >> upcase aliased as string-upcase
> >> format aliased as string-format
> >> concat aliased as string-concat
>
> >Don't do this.  Just learn these 5 names, or use apropos, as
> > has been suggested.
>
> Just curious, if these were to be introduced today, would you also name them that way?

I don't know, honestly.  But that is not relevant. We are not starting
an universe from scratch.

> What possible genericity is there about downcase/upcase? Okay they work on strings and on chars,
> but it's pretty clear their topic is "characters" (string) no?

Always and forever? Are strings, one-dimensional sequences of
characters by definition, the only thing you can think about that
involves characters whose case can be mutated?

> > Check out http://www.lispworks.com/documentation/HyperSpec/Body/f_stg_up.htm and http://www.lispworks.com/documentation/HyperSpec/Body/c_string.htm
>
> > Do you think this organization would be useful? What if it had a cool
> > font and some cool logos? :-)  I ask because that put me off at first, it
> > looked mouldy.)
>
> Yes, that'd be very useful and it's more or less what we are asking
> to be able to do. Again curious, do you think they made a mistake
> when they named it `string-downcase` there and not `downcase`?

It only works on strings in CL, but characters are defined to be "singleton
string designators", a common concept in CL. Maybe they couldn't
quite decide on a generic version. You see CL didn't  come from
nothing, it was a giant and very expensive effort to reconcile a
_lot_ of competing Lisps.

> >> s-truncate (len s)
> > truncate-string-to-width (I got there with apropos truncate.*string btw).
>
> Okay then let's alias that to string-truncate.

To achieve what? I would only bring confusion.  Also to the s.el
cool kids, they would be equally confused, since the arguments
are reversed.

> >> s-left (len s)
> >subseq
> >> s-right (len s)
> > Add an "from-end" arg to subseq?
>
> Why is there substring if there is subseq then?

¯\_(ツ)_/¯ History?  substring has both optional args.
Use substring if you absolutely know your sequence
is a string, subseq otherwise.

> >> s-repeat (num s)
> > (cl-loop repeat num concat s)
>
> What if I don't know cl (like I a significant number of package authors) ?

I think you might want to learn how to loop if you're going to program.
You can use while, do, etc. cl-loop is just my favourite.
I don't think this function needs to be in a library, but maybe an example
of how do this can be in one of those boring paragraphs of the manual.

> >> s-capitalize (s)
> > capitalize,  also works for chars.
> Alias it to string-capitalize :-)

I guess if I was starting from scratch I would do that.  We're not.  But maybe.

> >> s-reverse (s)
> > reverse, also works for sequences
>
> Ah, this one I didn't think worked on strings. Thought you have to split,
> reverse then join. I agree this can be left as a generic algorithm.

Good.

> >> s-contains-p (needle s &optional ignore-case)
> > string-match-p
> That can work, tho the mental path of having to do a regexp match
> while regexp-quoting the string just to do a plain text search feels very inneficient.

Well, I honestly had to look up the doc if "needle" was a string
or a character.  So that mental path is quite worse for me.

> >> s-blank-p (s)
> > ugh. nil is a string?
> "is the string blank?" is a concept in many languages,
> but I understand it can be surprising at first.

Yes, things are suprising at first, then you learn them.
Get the hint? ;-)

João Távora

Re: Imports / inclusion of s.el into Emacs

[Philippe Vaucher]

[-- Attachment #1: Type: text/plain, Size: 3993 bytes --]

>
> > What possible genericity is there about downcase/upcase? Okay they work
> on strings and on chars,
> > but it's pretty clear their topic is "characters" (string) no?
>
> Always and forever? Are strings, one-dimensional sequences of
> characters by definition, the only thing you can think about that
> involves characters whose case can be mutated?
>

Oh, it's designed for possible future structure in mind? While an
interesting point I think it's a bit far fetched.



> > > Do you think this organization would be useful? What if it had a cool
> > > font and some cool logos? :-)  I ask because that put me off at first,
> it
> > > looked mouldy.)
> >
> > Yes, that'd be very useful and it's more or less what we are asking
> > to be able to do. Again curious, do you think they made a mistake
> > when they named it `string-downcase` there and not `downcase`?
>
> It only works on strings in CL, but characters are defined to be "singleton
> string designators", a common concept in CL. Maybe they couldn't
> quite decide on a generic version. You see CL didn't  come from
> nothing, it was a giant and very expensive effort to reconcile a
> _lot_ of competing Lisps.
>

Interesting.


> > >> s-truncate (len s)
> > > truncate-string-to-width (I got there with apropos truncate.*string
> btw).
> >
> > Okay then let's alias that to string-truncate.
>
> To achieve what? I would only bring confusion.  Also to the s.el
> cool kids, they would be equally confused, since the arguments
> are reversed.
>

Can you stop with the "cool kids" labelling? I mean we are all old farts
here :-P I just happen to have learnt both ways (readings lots of C++ books
then eating lots of Ruby tutorials) and both are useful depending on what
you do.



> > >> s-left (len s)
> > >subseq
> > >> s-right (len s)
> > > Add an "from-end" arg to subseq?
> >
> > Why is there substring if there is subseq then?
>
> ¯\_(ツ)_/¯ History?  substring has both optional args.
> Use substring if you absolutely know your sequence
> is a string, subseq otherwise.
>

Thanks for the clarification.



> > >> s-repeat (num s)
> > > (cl-loop repeat num concat s)
> >
> > What if I don't know cl (like I a significant number of package authors)
> ?
>
> I think you might want to learn how to loop if you're going to program.
> You can use while, do, etc. cl-loop is just my favourite.
> I don't think this function needs to be in a library, but maybe an example
> of how do this can be in one of those boring paragraphs of the manual.
>

The implication that I don't know how to do loops is a bit mean, but I'll
let that pass. I don't know `cl` (tho I copy-pasted some in the past), but
I do know loops. I was always somewhat taught that `cl` was not _really_
emacs lisp so you should not use it. Maybe that's wrong.



> > >> s-contains-p (needle s &optional ignore-case)
> > > string-match-p
> > That can work, tho the mental path of having to do a regexp match
> > while regexp-quoting the string just to do a plain text search feels
> very inneficient.
>
> Well, I honestly had to look up the doc if "needle" was a string
> or a character.  So that mental path is quite worse for me.
>

Again very interesting: needle and haystack is also a metaphor used in many
languages. It's another point where I'd have thought you were trolling if
you didn't understand!



> > >> s-blank-p (s)
> > > ugh. nil is a string?
> > "is the string blank?" is a concept in many languages,
> > but I understand it can be surprising at first.
>
> Yes, things are suprising at first, then you learn them.
> Get the hint? ;-)


:-)

Maybe a bit of information that can be useful for you is that I code
packages since a while now, for example https://github.com/Silex/docker.el

It's been years since I use Emacs. I obviously never took the proper time
to learn it in full depth but I'm not a "newbie".

Philippe

[-- Attachment #2: Type: text/html, Size: 5751 bytes --]

Re: Imports / inclusion of s.el into Emacs

[João Távora]

On Sun, May 3, 2020 at 3:28 PM Philippe Vaucher
<philippe.vaucher@gmail.com> wrote:
>> Always and forever? Are strings, one-dimensional sequences of
>> characters by definition, the only thing you can think about that
>> involves characters whose case can be mutated?
> Oh, it's designed for possible future structure in mind?
> While an interesting point I think it's a bit far fetched.

Don't know if it was intentional, but I see it rather as not
unnecessarily under-designed.  Don't burn types in the name
for no good reason.

And a matrix of characters doesn't sound far-fetched to me.

>> It only works on strings in CL, but characters are defined to be "singleton
>> string designators", a common concept in CL. Maybe they couldn't
>> quite decide on a generic version. You see CL didn't  come from
>> nothing, it was a giant and very expensive effort to reconcile a
>> _lot_ of competing Lisps.
> Interesting.

Indeed you should check it out.
>
>>
>> > >> s-truncate (len s)
>> > > truncate-string-to-width (I got there with apropos truncate.*string btw).
>> >
>> > Okay then let's alias that to string-truncate.
>>
>> To achieve what? I would only bring confusion.  Also to the s.el
>> cool kids, they would be equally confused, since the arguments
>> are reversed.
>
>
> Can you stop with the "cool kids" labelling? I mean we are all old farts here :-P I just happen to have learnt both ways (readings lots of C++ books then eating lots of Ruby tutorials) and both are useful depending on what you do.

Oh I'm exactly the same, I'm a C++ Ruby cool kid too, kinda.
They shot that at me a lot when I was beginning CL. I don't
think it's offensive, and laugh it off (remember it _is_ cool to
be cool) I'll stop since it bothers you tho.

>> > >> s-repeat (num s)
>> > > (cl-loop repeat num concat s)
>> >
>> > What if I don't know cl (like I a significant number of package authors) ?
>>
>> I think you might want to learn how to loop if you're going to program.
>> You can use while, do, etc. cl-loop is just my favourite.
>> I don't think this function needs to be in a library, but maybe an example
>> of how do this can be in one of those boring paragraphs of the manual.
> The implication that I don't know how to do loops is a bit mean, but I'll let that pass. I don't know `cl` (tho I copy-pasted some in the past), but I do know loops. I was always somewhat taught that `cl` was not _really_ emacs lisp so you should not use it. Maybe that's wrong.

cl-lib is a utility library. I didn't mean you didn't know how to
loop.  But you literally suggested the possibility that one
might not know how to loop in Elisp.  So I stated that
learning to do so is inevitable, and showed you alternative
ways to loop.

>> Well, I honestly had to look up the doc if "needle" was a string
>> or a character.  So that mental path is quite worse for me.
> Again very interesting: needle and haystack is also a metaphor used in many languages. It's another point where I'd have thought you were trolling if you didn't understand!

I know strstr() of course.  But I have to `man strtr` anyway, or
check the signature and spot the char*.  There's no
universal definition of "needle", except for the pointy thing.
And strstr is haystack first needle second by the way.
So let's change the libc because clojure!

> It's been years since I use Emacs. I obviously never took the proper
> time to learn it in full depth but I'm not a "newbie".

We're all newbies, thankfully, the world is a big place, with lots
of new stuff to discover.

I've not learned it in "full depth" either. Actually I find this
conversation engaging because it sounds like I'm speaking to
myself from some years ago.  I _don't_ want to sound paternalistic
and it's not to make it sound like I'm more "evolved", but I would
have liked a me to explain the things that confused me when
I was struggling.   Obviously, you're not me, so sorry if there's
some cognitive dissonance.

Anyway, how's that API-from-doc sweeper thing coming along?
You know, the thing you started with `keep-lines`. With a few lines
of code, you can make a bigger difference than all of us with a
zillion lines of bikeshedding.

-- 
João Távora

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Always and forever? Are strings, one-dimensional sequences of
  > characters by definition, the only thing you can think about that
  > involves characters whose case can be mutated?

That is a can of worms with unclear boundaries.  How about if we
discuss it some other day?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >> split-string aliased as string-split
  > >> downcase aliased as string-downcase
  > >> upcase aliased as string-upcase
  > >> format aliased as string-format
  > >> concat aliased as string-concat

  > >Don't do this.  Just learn these 5 names, or use apropos, as
  > > has been suggested.

  > Just curious, if these were to be introduced today, would you also name
  > them that way? What possible genericity is there about downcase/upcase?

I would certainly use the names 'format' and 'concat'.

For 'upcase', 'downcase' and 'capitalize', I might limit them
to strings (not characters) and give them names with 'string' in them.
But I am not sure.

I think that names of the form VERB-TYPE are natural when VERB is short
and refers to a transformation of the object itself.  For instance,
'split-string', 'trim-string'.

However, if we want to say something more complex about the action
than just a single verb, TYPE-ACTIONPHRASE is cleaner.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Imports / inclusion of s.el into Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > New functions (with string- instead of s-):

  > s-truncate (len s)
  > s-chomp (s)
  > s-left (len s)
  > s-right (len s)
  > s-repeat (num s)
  > s-capitalize (s)
  > s-reverse (s)
  > s-contains-p (needle s &optional ignore-case)
  > s-blank-p (s)

Would you please say what each of these does,
and why in your opinion it would be an improvement to
Emacs Lisp?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Proposed new core library: pl.el

[John Wiegley]

pl.el (standing for "parser library") is a combinator parsing library for
Emacs, similar to Haskell's Parsec. You can see how it works at the following
README:

    https://github.com/jwiegley/emacs-pl

PL offers a way to write very compact parsers of structured text. For example
(from the README):

  (pl-parse
    (delete-region (pl-str "<xml>" :beg)
                   (pl-until
                     (pl-str "</xml>" :end))))

The idea being applicative parsers is that the result of `pl-parse' is the
FORM you pass in, where every sub-parser becomes a value of the type you
intended to parse. If a sub-parse fails, either the whole parse fails, or it
returns nil if you wrap the parser in `pl-try'.

There is room for improving performance, but the API is complete enough to
start using it. Giving the unproven status, though, perhaps it should start
out in ELPA, and move to core after it has solidified and gained some users?

John

Re: Proposed new core library: pl.el

[Dmitry Gutov]

On 11/05/2015 04:14 AM, John Wiegley wrote:

> There is room for improving performance, but the API is complete enough to
> start using it. Giving the unproven status, though, perhaps it should start
> out in ELPA, and move to core after it has solidified and gained some users?

I think the rule for moving new stuff out of ELPA should be whether it's 
used by the core. That's for libraries.

For utility packages (ui/commands/etc), the question should be asked: if 
it works fine in ELPA, why move? Popularizing ELPA would be better, IMHO.

ELPA policy (was: Proposed new core library: pl.el)

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> I think the rule for moving new stuff out of ELPA should be whether it's
> used by the core. That's for libraries.

An exception to this rule is when a certain service (say, streams) should
always be available, without requiring further installation of libraries.
Emacs acts as a sort of "standard library" for Emacs Lisp, so the same kinds
of things we'd like to have in such a meta-library, should be in core.

I do think that applications using such libraries should almost always go in
ELPA, except for those that have been grandfathered in, like Emacs Calc. There
are some things you should always be able to reach for, no matter whose
machine you are on, or if the Internet is currently available.

John

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 04:41 AM, John Wiegley wrote:

> An exception to this rule is when a certain service (say, streams) should
> always be available, without requiring further installation of libraries.
> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same kinds
> of things we'd like to have in such a meta-library, should be in core.

Why not consider ELPA a part of the "standard library", too?

As long as all code that uses streams.el is not in Emacs core, I don't 
see any harm in requiring the user to install streams from ELPA (which 
would probably happen automatically, via a declared dependency).

When the core starts using it, sure, we can move it there.

> I do think that applications using such libraries should almost always go in
> ELPA, except for those that have been grandfathered in, like Emacs Calc. There
> are some things you should always be able to reach for, no matter whose
> machine you are on, or if the Internet is currently available.

Yes, some pieces of functionality that we consider a part of the 
standard set (Calc is not that, for me, but apparently it's popular 
enough). The criteria for applications to include them might be that 
Emacs has dedicated menu items, or uses the application's commands in 
some global bindings (that's why xref, for example, can't be moved to ELPA).

Re: ELPA policy

[Artur Malabarba]

2015-11-05 3:00 GMT+00:00 Dmitry Gutov <dgutov@yandex.ru>:
> On 11/05/2015 04:41 AM, John Wiegley wrote:
>
>> An exception to this rule is when a certain service (say, streams) should
>> always be available, without requiring further installation of libraries.
>> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same
>> kinds
>> of things we'd like to have in such a meta-library, should be in core.
>
>
> Why not consider ELPA a part of the "standard library", too?
>
>[...]
> When the core starts using it, sure, we can move it there.

I agree in a general sense. But case-by-case discretion will have to be applied.

> The
> criteria for applications to include them might be that Emacs has dedicated
> menu items, or uses the application's commands in some global bindings
> (that's why xref, for example, can't be moved to ELPA).

IMO, xref can't be moved to Gelpa because users expect this "find
definition" functionality out-of-the box. By which I don't mean "Emacs
users have come to expect it", I mean "users coming to Emacs will
expect it".

Re: ELPA policy

[Michael Welsh Duggan]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/05/2015 04:41 AM, John Wiegley wrote:
>
>> An exception to this rule is when a certain service (say, streams) should
>> always be available, without requiring further installation of libraries.
>> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same kinds
>> of things we'd like to have in such a meta-library, should be in core.
>
> Why not consider ELPA a part of the "standard library", too?

Because it is not.  I have often had to use Emacs on computers that did
not have access to an Internet connection.  To consider something part
of the "standard library," I believe you should have access to it when
Emacs is installed.  This is why I dislike the move towards systems that
only provide easy installation via network channels.  (Python, this
means you!)

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 02:51 PM, Michael Welsh Duggan wrote:

>> Why not consider ELPA a part of the "standard library", too?
>
> Because it is not.  I have often had to use Emacs on computers that did
> not have access to an Internet connection.  To consider something part
> of the "standard library," I believe you should have access to it when
> Emacs is installed.

We do intend to bundle some packages from ELPA to be distributed with 
Emacs releases.

Anyway, if you don't have Internet access, you also can't install any 
package that depends on the library in question. Right?

Re: ELPA policy

[Michael Welsh Duggan]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/05/2015 02:51 PM, Michael Welsh Duggan wrote:
>
>>> Why not consider ELPA a part of the "standard library", too?
>>
>> Because it is not.  I have often had to use Emacs on computers that did
>> not have access to an Internet connection.  To consider something part
>> of the "standard library," I believe you should have access to it when
>> Emacs is installed.
>
> We do intend to bundle some packages from ELPA to be distributed with
> Emacs releases.

In this case, I can consider these part of a standard library.

> Anyway, if you don't have Internet access, you also can't install any
> package that depends on the library in question. Right?

Not so.  Packages can still be installed via sneakernet, and elisp can
be written that wants to take advantage of a "standard library."  I have
no objection to ELPA in general.  I just have an issue with the idea
that all of ELPA (non-bundled) can be considered "standard."

-- 
Michael Welsh Duggan
(mwd@cert.org)

Re: ELPA policy

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> We do intend to bundle some packages from ELPA to be distributed with Emacs
> releases.

I'd like to know more about this. How does the user access these bundled
packages; by installing them with M-x package-install?

John

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 05:09 PM, John Wiegley wrote:

> I'd like to know more about this. How does the user access these bundled
> packages; by installing them with M-x package-install?

By having them in the default load-path somewhere already, I imagine.

The user would be able to M-x package-install, however, to install newer 
a version of the same package from ELPA, that was released in the meantime.

I think Fabian was looking into making that happen, but I can't find the 
respective thread right now.

Re: ELPA policy

[Artur Malabarba]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/05/2015 05:09 PM, John Wiegley wrote:
>
>> I'd like to know more about this. How does the user access these bundled
>> packages; by installing them with M-x package-install?
>
> By having them in the default load-path somewhere already, I imagine.

There are a few possible levels this could go.
But yes, the basic idea is that these bundled packages would be pulled
from Gelpa into some subdir of emacs/lisp when making the tarball and
autoloading/byte-compilation would be done. So they'd be available like
any other built-in package.

The advantage over just keeping the packages there in the first place is
that packages offered on GElpa can be updated more frequently.

A second level would be to do this in such a way that allows Emacs
packages to depend on these pulled-in Gelpa packages. This would require
changes to the build process, not just the tarball.

> I think Fabian was looking into making that happen, but I can't find
> the respective thread right now.

I think I asked about this a couple of months ago, and Stefan (?) wrote
a very nice explanation of what his thoughts were on this possible
future.

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 06:58 PM, Artur Malabarba wrote:

>> I think Fabian was looking into making that happen, but I can't find
>> the respective thread right now.
>
> I think I asked about this a couple of months ago, and Stefan (?) wrote
> a very nice explanation of what his thoughts were on this possible
> future.

Stefan, most likely.  Too bad it wasn't in a dedicated bug discussion.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'd like to know more about this. How does the user access these bundled
  > packages; by installing them with M-x package-install?

I think they should appear, to the user, like any other standard part of Emacs.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

>> I'd like to know more about this. How does the user access these bundled
>> packages; by installing them with M-x package-install?

> I think they should appear, to the user, like any other standard part of
> Emacs.

If that's the case, it changes my thoughts on what needs to be in core, and
what should be in ELPA. Until now I was thinking ELPA required Internet
access; but if there are parts of ELPA that "come in the box", then I'd like
to see more packages there.

John

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Sun, 08 Nov 2015 08:30:17 -0800
> Cc: mwd@md5i.com, emacs-devel@gnu.org, dgutov@yandex.ru
> 
> If that's the case, it changes my thoughts on what needs to be in core, and
> what should be in ELPA. Until now I was thinking ELPA required Internet
> access; but if there are parts of ELPA that "come in the box", then I'd like
> to see more packages there.

ELPA requires Internet access if you are building from the repository.

My greatest fears are that packages that move from core to ELPA will
get much less attention from the core maintenance team, which will
eventually degrade their quality.

Re: ELPA policy

[Wolfgang Jenkner]

On Sun, Nov 08 2015, Eli Zaretskii wrote:

> My greatest fears are that packages that move from core to ELPA will
> get much less attention from the core maintenance team, which will
> eventually degrade their quality.

But this is not what actually happens with completely separately
developed complex packages like SLIME, Evil...

On the other hand, IIUC, moving CEDET to emacs core has not been
completely without problems for the CEDET project and its users.

Re: ELPA policy

[Eli Zaretskii]

> From: Wolfgang Jenkner <wjenkner@inode.at>
> Cc: John Wiegley <jwiegley@gmail.com>,  mwd@md5i.com,  emacs-devel@gnu.org,  rms@gnu.org,  dgutov@yandex.ru
> Date: Sun, 08 Nov 2015 19:00:06 +0100
> 
> On Sun, Nov 08 2015, Eli Zaretskii wrote:
> 
> > My greatest fears are that packages that move from core to ELPA will
> > get much less attention from the core maintenance team, which will
> > eventually degrade their quality.
> 
> But this is not what actually happens with completely separately
> developed complex packages like SLIME, Evil...

Most packages in core are not of this kind.

> On the other hand, IIUC, moving CEDET to emacs core has not been
> completely without problems for the CEDET project and its users.

Such moves are never without problems, which is exactly my point.  In
this case, I think the benefits far outweighed the problems: until
CEDET was incorporated into Emacs core, we couldn't even dream about
using it as infrastructure for IDEs and other similar features.
AFAIR, there was also lots of cleanup during the move.

But moving ELPA packages into the core is not being discussed or
suggested.  So what happened with CEDET, even if you disagree with my
assessment above, has little relevance to the issue at hand.

Re: ELPA policy

[Wolfgang Jenkner]

On Sun, Nov 08 2015, Eli Zaretskii wrote:

>> On the other hand, IIUC, moving CEDET to emacs core has not been
>> completely without problems for the CEDET project and its users.
>
> Such moves are never without problems, which is exactly my point.

I was using the wrong word here as it's more a fork than a move:

http://sourceforge.net/p/cedet/git/ci/master/tree/INSTALL

> In
> this case, I think the benefits far outweighed the problems: until
> CEDET was incorporated into Emacs core, we couldn't even dream about
> using it as infrastructure for IDEs and other similar features.

CEDET being a separate project did not stop various people elsewhere
from realizing their dream about an "IDE" according to their own ideas:

https://www.youtube.com/results?search_query=emacs+cedet

(sorry for giving a link to a google service...)

> AFAIR, there was also lots of cleanup during the move.

> But moving ELPA packages into the core is not being discussed or
> suggested.  So what happened with CEDET, even if you disagree with my
> assessment above, has little relevance to the issue at hand.

To be clear: FWIW, I agree that CEDET is great and that using real
grammars for doing various stuff in emacs is an exciting idea.  I

Re: ELPA policy

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> If that's the case, it changes my thoughts on what needs to be in core, and
>> what should be in ELPA. Until now I was thinking ELPA required Internet
>> access; but if there are parts of ELPA that "come in the box", then I'd like
>> to see more packages there.
>
> ELPA requires Internet access if you are building from the repository.
>
> My greatest fears are that packages that move from core to ELPA will
> get much less attention from the core maintenance team, which will
> eventually degrade their quality.

Org-mode, for instance, gets regular bug fix releases on GNU ELPA while
it stagnates on core. I would venture that the contribution from the
"core maintenance team" to org-mode amounts to a negative effect,
causing more hassle than it saves.

For projects with their own development team and VCS repository, having
the project copied into the Emacs core VCS makes no sense, moreover when
they have to support multiple Emacs releases.

Re: ELPA policy

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sun, 08 Nov 2015 19:26:20 +0100
> 
> For projects with their own development team and VCS repository, having
> the project copied into the Emacs core VCS makes no sense, moreover when
> they have to support multiple Emacs releases.

I could probably agree, as soon as we find a good way of getting the
"right" version of those when an Emacs tarball is being prepared (or
maybe even when Emacs is built from the repository).

But most packages in core are not of this variety, and it's them that
I worry about, not Org or Gnus.

Re: ELPA policy

[Artur Malabarba]

Eli Zaretskii <eliz@gnu.org> writes:

> My greatest fears are that packages that move from core to ELPA will
> get much less attention from the core maintenance team, which will
> eventually degrade their quality.

We could bring the Elpa and Emacs repositories closer together somehow.
Maybe as a submodule or something like that.

This way the core maintenance team would also have the Elpa sources
"right there" to do bug fixes.

Re: ELPA policy

[Eli Zaretskii]

> From: Artur Malabarba <bruce.connor.am@gmail.com>
> Cc: John Wiegley <jwiegley@gmail.com>,  emacs-devel@gnu.org,
> Date: Sun, 08 Nov 2015 19:27:08 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > My greatest fears are that packages that move from core to ELPA will
> > get much less attention from the core maintenance team, which will
> > eventually degrade their quality.
> 
> We could bring the Elpa and Emacs repositories closer together somehow.
> Maybe as a submodule or something like that.

How about just using a single repository? ;-)

Seriously, why should they be separate?  Submodules still complicate
things, compared to just having the stuff in the repo.

Re: ELPA policy

[Artur Malabarba]

Eli Zaretskii <eliz@gnu.org> writes:

> How about just using a single repository? ;-)
>
> Seriously, why should they be separate?  Submodules still complicate
> things, compared to just having the stuff in the repo.

I'm not strongly against that. But there are a couple of things that
might be annoying with that:

- Elpa commits would occasionally flood the git log

- Git operations in both repositories would become even slower than they
  are now (as they'd both increase in size).

Re: ELPA policy

[Eli Zaretskii]

> From: Artur Malabarba <bruce.connor.am@gmail.com>
> Cc: jwiegley@gmail.com,  emacs-devel@gnu.org
> Date: Sun, 08 Nov 2015 20:04:56 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > How about just using a single repository? ;-)
> >
> > Seriously, why should they be separate?  Submodules still complicate
> > things, compared to just having the stuff in the repo.
> 
> I'm not strongly against that. But there are a couple of things that
> might be annoying with that:
> 
> - Elpa commits would occasionally flood the git log

I'm sure no one actually looks at too many commits at once, only those
that are of interest.

> - Git operations in both repositories would become even slower than they
>   are now (as they'd both increase in size).

GCC is migrating to Git as we speak, even though it has something like
3-4 times more commits in its history than we do.  If they aren't
afraid, I don't think we should be.

Re: ELPA policy

[Dmitry Gutov]

On 11/08/2015 09:58 PM, Eli Zaretskii wrote:

> I'm sure no one actually looks at too many commits at once, only those
> that are of interest.

Filtering out the non-interesting commits is a hassle in itself, and I 
saying this as a emacs-diffs subscriber. With everyone committing to the 
same repo, it will become worse.

Re: ELPA policy

[Eli Zaretskii]

> Cc: jwiegley@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 8 Nov 2015 22:10:29 +0200
> 
> On 11/08/2015 09:58 PM, Eli Zaretskii wrote:
> 
> > I'm sure no one actually looks at too many commits at once, only those
> > that are of interest.
> 
> Filtering out the non-interesting commits is a hassle in itself, and I 
> saying this as a emacs-diffs subscriber. With everyone committing to the 
> same repo, it will become worse.

ELPA gets about 4 commits per day on average, so I don't think we
should worry about this aspect.

Re: ELPA policy

[Dmitry Gutov]

On 11/08/2015 10:26 PM, Eli Zaretskii wrote:

> ELPA gets about 4 commits per day on average, so I don't think we
> should worry about this aspect.

Emacs master has had ~82 commits in November, according to 'git log', 
which comes out to ~10 a day. With ELPA, this will be something like 40% 
increase?

And this is only while ELPA is relatively unpopular. If it ever grows to 
anything comparable to the size of MELPA (which we probably should 
aspire to), we will drown.

Re: ELPA policy

[Eli Zaretskii]

> Cc: bruce.connor.am@gmail.com, jwiegley@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 8 Nov 2015 22:36:00 +0200
> 
> On 11/08/2015 10:26 PM, Eli Zaretskii wrote:
> 
> > ELPA gets about 4 commits per day on average, so I don't think we
> > should worry about this aspect.
> 
> Emacs master has had ~82 commits in November, according to 'git log', 
> which comes out to ~10 a day. With ELPA, this will be something like 40% 
> increase?
> 
> And this is only while ELPA is relatively unpopular. If it ever grows to 
> anything comparable to the size of MELPA (which we probably should 
> aspire to), we will drown.

OK, then let's drop this idea.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Filtering out the non-interesting commits is a hassle in itself, and I 
  > saying this as a emacs-diffs subscriber. With everyone committing to the 
  > same repo, it will become worse.

I am sure we could set up multiple mailing lists and send each commit
to a list according to what directory it is in.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 01:16 AM, Richard Stallman wrote:

> I am sure we could set up multiple mailing lists and send each commit
> to a list according to what directory it is in.

You have a point there, and if someone were to actually do that, it 
would help with making the notifications manageable.

Still, the more happens in the repository, the harder it is to grasp or 
manage. Right now you can call 'git log' and see the most recent things 
that happened to Emacs master. Or do 'git pull', and call 'git diff' on 
the revision range that was printed out, and see all changes since your 
last update.

Unless the Emacs sources move into a subdirectory, even listing all 
commits that were made to Emacs recently, would involve the contributor 
learning magic like this: 
http://stackoverflow.com/questions/5685007/making-git-log-ignore-changes-for-certain-paths

Re: ELPA policy

[Yuri Khan]

On Mon, Nov 9, 2015 at 7:45 AM, Dmitry Gutov <dgutov@yandex.ru> wrote:

> Still, the more happens in the repository, the harder it is to grasp or
> manage. Right now you can call 'git log' and see the most recent things that
> happened to Emacs master. Or do 'git pull', and call 'git diff' on the
> revision range that was printed out, and see all changes since your last
> update.

I almost wanted to mention that you can restrict “git log” and “git
diff” to a particular path in the repository, but realized it might
start a new git subthread.

(FWIW I also think separate repositories are better, as long as
development is mostly independent across the divide.)

Re: ELPA policy

[Artur Malabarba]

John Wiegley <jwiegley@gmail.com> writes:

> If that's the case, it changes my thoughts on what needs to be in core, and
> what should be in ELPA. Until now I was thinking ELPA required Internet
> access; but if there are parts of ELPA that "come in the box", then I'd like
> to see more packages there.

I think we ought to clarify what we're talking about here.

- There are packages that are offered only on Elpa. For these you need
  an internet connection and you need to `M-x package-install' them.

- There are packages that are offered both on Elpa and Emacs. These
  are just like any other core package (no connection and no
  package-install necessary).

  The difference is that users can get access to new versions from Elpa
  without having to wait for the next Emacs release. The disadvantage
  here is code duplication, as someone needs to make sure that any bug
  fixes applied on the Emacs side also get applied on the Elpa side (and
  vice versa).

  Current examples of this are seq.el and let-alist.

- Then there's a third option. I only found this out recently while
  browsing the `externals-list' file on Elpa, and it was apparently
  implemented by Fabián in September.

  You can configure Elpa to (while building) copy the most recent
  version of an packages from the Emacs repository. IIUC, this would
  have the same effect as the previous option, except no code
  duplication. The thing is: no packages seem to be using this, and it's
  only documented in the comments of that file (though I've verified
  that there's indeed code to implement this).

Fabián, is this feature complete?

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Richard Stallman <rms@gnu.org> writes:
>
>>> I'd like to know more about this. How does the user access these bundled
>>> packages; by installing them with M-x package-install?
>
>> I think they should appear, to the user, like any other standard part of
>> Emacs.
>
> If that's the case, it changes my thoughts on what needs to be in core, and
> what should be in ELPA. Until now I was thinking ELPA required Internet
> access; but if there are parts of ELPA that "come in the box", then I'd like
> to see more packages there.

It depends on what git repository the package is in, and how the ELPA
packages are included in core.

My impression is that the latest proposal for doing this is to keep
"core ELPA" packages in the emacs repository (not the Gnu ELPA repository),
and fix the Gnu ELPA server scripts to work with that.

Then all core Emacs developers see the core ELPA packages in their
current workflow, and they are released along with Emacs.

They are also released separately to Gnu ELPA, and they appear in
`list-packages' (as other core packages do now); that's what makes them
"ELPA core packages" as opposed to "core code".

That's an elegant solution.


The alternative is to figure out how to merge a partial checkout of the
Gnu ELPA repository into an Emacs git checkout without confusing git and
make; not at all easy.

-- 
-- Stephe

Re: ELPA policy

[David Kastrup]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> I think the rule for moving new stuff out of ELPA should be whether it's
>> used by the core. That's for libraries.
>
> An exception to this rule is when a certain service (say, streams)
> should always be available, without requiring further installation of
> libraries.

What does "service" and "should always be available" mean?  If it is not
end-user functionality useful for ad-hoc code entered with M-: or its
ilk (it would appear that streams is moving there), then I don't see a
necessity for including in the core: it makes more sense to put it as a
dependency in ELPA.  That way, other ELPA packages may easily require it
without depending on an upgrade of the core Emacs.

> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same
> kinds of things we'd like to have in such a meta-library, should be in
> core.

Emacs cannot be upgraded using the package manager, so any dependencies
on Emacs core code are harder to cater for than a dependency on another
ELPA package.

-- 
David Kastrup

streams are cool, you could stream virtually anything!

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 708 bytes --]

Hi guys,

Aren't you tired of writing the same for loops in various contexts?
I know I was, and there's a better way!

I just added a function `stream-regexp' to stream.el to stream the
search of a regexp in a buffer:


    (let ((str (stream-regexp buf ".*foo\\(.*\\)$")))
      (seq-map (lambda (match-data)
                 ...)
               str))

(I still have to document it and test it properly)

Because streams are lazy and immutable, you can reuse such a stream
forever or keep it for later, move around in a buffer or modify the
buffer and re-evaluate the stream (you'll get new match data), or map
the result, etc.

All the credit goes to Damien Cassou for giving me the idea :-)

Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: streams are cool, you could stream virtually anything!

[John Wiegley]

>>>>> Nicolas Petton <nicolas@petton.fr> writes:

> I just added a function `stream-regexp' to stream.el to stream the search of
> a regexp in a buffer:

This is amazing, Nicolas! I've wanted this exact construction so many times!

Do you know the memory behavior of doing it this way? That is, for 1000 hits,
how does the number of cons's allocated compare to the iterative approach?

John

Re: streams are cool, you could stream virtually anything!

[Michael Heerdegen]

John Wiegley <jwiegley@gmail.com> writes:

> Do you know the memory behavior of doing it this way? That is, for
> 1000 hits, how does the number of cons's allocated compare to the
> iterative approach?

That depends on whether you want/need to keep the matches so far.  If
you don't need to save prior matches, you can just

  (setq stream (stream-cdr stream))

after generating an element, and memory usage will be O(1).  (Note to
Nicolas: a `stream-pop' function would be nice in this context).

In that case, you can alternatively use generators (generator.el) or the
derived iterators.el.

If you keep the complete stream in memory, you get O(nbr-matches) conses
(of course).  Not much a problem IME, if you keep an eye on what you
want to keep and what you can throw away when using streams.


Regards,

Michael.

Re: streams are cool, you could stream virtually anything!

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 443 bytes --]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> That depends on whether you want/need to keep the matches so far.  If
> you don't need to save prior matches, you can just
>
>   (setq stream (stream-cdr stream))
>
> after generating an element, and memory usage will be O(1).  (Note to
> Nicolas: a `stream-pop' function would be nice in this context).

Indeed!  Could I by any chance get another pull request from you on
GitHub?

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: streams are cool, you could stream virtually anything!

[T.V Raman]

As more of these patterns emerge, I'm even more convinced that  the
stream package belongs alongside thunk.el in core emacs.
-- 

Re: streams are cool, you could stream virtually anything!

[John Wiegley]

>>>>> T V Raman <raman@google.com> writes:

> As more of these patterns emerge, I'm even more convinced that the stream
> package belongs alongside thunk.el in core emacs.

Let's see it develop during these earlier stages on ELPA, so people can easily
update the package as it develops. Once it stabilizes, I agree we should move
it into core.

John

Re: streams are cool, you could stream virtually anything!

[T.V Raman]

SGTM.

While on the topic -- should we move things out of core into elpa --
there are a lot of elisp packages that are presently bundled -- mostly
because "there was no elpa around when they were bundled".

John Wiegley writes:
 > >>>>> T V Raman <raman@google.com> writes:
 > 
 > > As more of these patterns emerge, I'm even more convinced that the stream
 > > package belongs alongside thunk.el in core emacs.
 > 
 > Let's see it develop during these earlier stages on ELPA, so people can easily
 > update the package as it develops. Once it stabilizes, I agree we should move
 > it into core.
 > 
 > John

-- 

-- 

ELPA policy (Was: streams are cool, you could stream virtually anything!)

[John Wiegley]

>>>>> T V Raman <raman@google.com> writes:

> While on the topic -- should we move things out of core into elpa -- there
> are a lot of elisp packages that are presently bundled -- mostly because
> "there was no elpa around when they were bundled".

I would in fact like to see more packages move from core into ELPA, although
I'm not eager to do so willy nilly. Some packages now in Core are just fun, or
should be there because people are used to them being there, even if today we
wouldn't necessarily add that package to core.

If someone would be willing to go through every core package, and produce a
list of what should stay in core, what should move to ELPA, and why they think
so, that would be a great starting point.

I know from past experience that reading every core file is highly interesting
and educational, and I guarantee whoever volunteers for this will discover
many cool and interesting things. :)

John

Re: ELPA policy

[Achim Gratz]

John Wiegley writes:
> I would in fact like to see more packages move from core into ELPA, although
> I'm not eager to do so willy nilly. Some packages now in Core are just fun, or
> should be there because people are used to them being there, even if today we
> wouldn't necessarily add that package to core.

I've proposed this before, but I guess I should run it by you again (and
warn you it wasn't favorably received):

The current definition of "core" is that the sources live inside the
Emacs repository.  Several of the larger core packages like Org, CEDET
and Gnus are already largely developed outside Emacs, for instance
because the developers want to keep them compatible with different/older
Emacsen and need to be synced into Emacs sources to make them "core"
anyway.

I posit that the only thing that actually matters for something to be
considered "core" is that authors of other packages can rely on the
(stable) API provided by these packages to be available in an Emacs as
it gets distributed and no installation of further packages or software
is necessary, neither by the sysadmin nor the user.  If so, instead of
keeping the "core" sources all in Emacs, they could equally well be
living in ELPA and be pre-installed into the distribution, or installed
into the Emacs build tree as submodules or subtrees.  The most radical
(and likely most controversial) thing to do would be to move everything
to ELPA that isn't needed to bootstrap Emacs.

Doing this would need some as of yet non-existing infrastructure to get
the chosen ELPA version of each package built into the distribution, and
facilities for sysadmins and users to update (but not disable) the
"core" packages at the system level or in their private directories.
Emacs may eventually distribute some "non-core" packages also that
sysadmins or users could disable if they chose to not use them.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: ELPA policy

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sun, 08 Nov 2015 18:18:26 +0100
> 
> I posit that the only thing that actually matters for something to be
> considered "core" is that authors of other packages can rely on the
> (stable) API provided by these packages to be available in an Emacs as
> it gets distributed and no installation of further packages or software
> is necessary, neither by the sysadmin nor the user.  If so, instead of
> keeping the "core" sources all in Emacs, they could equally well be
> living in ELPA and be pre-installed into the distribution, or installed
> into the Emacs build tree as submodules or subtrees.

IMO, no serious move such as this one should be argued for, let alone
attempted, without some minimal analysis of advantages and
disadvantages.  In particular, such an analysis cannot be limited to
the POV of maintainers of packages tat are currently not bundled, it
must first and foremost look at this from the POV of the Emacs
maintenance, definitely if the argument is to leave in the Emacs
repository only what's needed for bootstrap.

And any change in maintenance routine, small or large, should have
enough advantages to justify the energy that will certainly go into
the move itself and into cleaning up the resulting fallout.

I don't see how we can seriously discuss such suggestions when they
are not accompanied by anything like the analysis they need.

> The most radical (and likely most controversial) thing to do would
> be to move everything to ELPA that isn't needed to bootstrap Emacs.

Most such packages don't have any active maintainers, i.e. they are
maintained by the "FSF", which means us the core developers.  IMO, it
makes very little sense to spread the stuff we maintain between 2
separate repositories, because all this does is add overhead and
complexity without any clear benefits that I could see.

Another important aspect that this suggestion seems to overlook is
that Emacs packages rely on others not only via APIs, but also by
inheritance, like all the modes that derive from Text mode etc.

> Doing this would need some as of yet non-existing infrastructure to get
> the chosen ELPA version of each package built into the distribution, and
> facilities for sysadmins and users to update (but not disable) the
> "core" packages at the system level or in their private directories.

Yes, and the effort this will require is squarely in the disadvantages
camp.

Let me turn the table and ask: Are there any _advantages_ in moving
stuff like Dired, CC Mode, Shell Mode, Speedbar, and ps-print, to name
just a random few?

Re: ELPA policy

[Aaron Ecay]

Hi Eli,

2015ko azaroak 8an, Eli Zaretskii-ek idatzi zuen:

[...]

> 
> Let me turn the table and ask: Are there any _advantages_ in moving
> stuff like Dired, CC Mode, Shell Mode, Speedbar, and ps-print, to name
> just a random few?

Imagine someone implements an awesome new feature for dired.  Emacs
users the world over are amazed by this, and fill their blogs, twitter,
etc. with the news.  If dired is an ELPA package, everyone who hears
this news can get the new feature in their emacs instantly by upgrading
their ELPA packages.  No need to wait N months for a new release of
emacs, or compile a non-release version of emacs from git.

Whether this counts as an advantage or not is complicated, and partially
depends on one’s point of view.  It would spell the end of an era where
each emacs release’s features are explicated through a year of excited
blogposts (for example
<http://endlessparentheses.com/tags-expanded.html#emacs-25>).  Users
will be able to choose a new model where features trickle in rather than
coming in sudden major chunks.  This might make it more complicated to
advocate adoption of new emacs versions.  But I don’t think so actually:
we regularly get big features at the language level too, which are very
exciting (for emacs 24 lexical binding, now dynamic modules and
xwidgets, and one dares to hope for the concurrency branch in emacs26).

I also think that discussion around this topic is distorted by a
long-tailed distribution.  Most people probably have in mind the big
emacs packages with dynamic developer communities and independent
release cycles.  Org, Gnus, CEDET, and maybe a few others.  On the other
hand, it seems to me that you have in mind (in addition to these) all
the tiny packages that see very few commits and perhaps no new features
(in the extreme case take kermit.el, which has seen no substantive
changes since at least 1992, but still gets its copyright header
lovingly updated every year).  I don’t know how to reconcile these
viewpoints, or even if it’s necessary.  Just something to consider.

Speaking for myself, my primary interest in emacs development is working
on org-mode, and I heavily use org as well as third-party packages for
python (elpy), clojure (cider), R (ESS), and a few other random things.
I build emacs from git every month or so in order to pick up little
quality of life improvements, like with-eval-afer-load, some of the
subr-x stuff, seq.el, the overhauled package menu, etc.  But I have to
make sure to keep a couple months’ worth of old emacsen around, in case
my monthly pull happens to land on a buggy commit – one that causes
regular segfaults, which I have had happen more than once.  I use emacs
for work so I don’t have the luxury of just going without for a few days
until the problem is fixed.  I could go back to a previous released
version of course, but then my init.el breaks because it relies on new
features since the release.

I’d be much happier running released versions of emacs for stability, if
it were easier to pull in new versions of core elisp libraries as they
are developed.  I suspect there are some others out there like me, and
many more who don’t currently build emacs from source for whatever reason
but would enjoy more regular access to new features in core elisp
libraries.

My 2 cents FWIW,
Aaron

PS I also have the impression that life would be easier for org’s
maintainers under a system that freed them from merging code back and
forth from org’s repo to emacs.  This would have indirect positive
effects on me as I work on org development.  However, I’m sure the
maintainers themselves – and maintainers of other projects in the same
situation like CEDET and Gnus – can speak to the question better than I
can.

-- 
Aaron Ecay

Re: ELPA policy

[Eli Zaretskii]

> From: Aaron Ecay <aaronecay@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Sun, 08 Nov 2015 20:52:12 +0000
> 
> Imagine someone implements an awesome new feature for dired.  Emacs
> users the world over are amazed by this, and fill their blogs, twitter,
> etc. with the news.  If dired is an ELPA package, everyone who hears
> this news can get the new feature in their emacs instantly by upgrading
> their ELPA packages.  No need to wait N months for a new release of
> emacs, or compile a non-release version of emacs from git.

How is this different when Dired is in the Emacs repository?  The
Emacs repository is a public one, so anyone and everyone can get the
latest version from there and use it, if they want.

> I also think that discussion around this topic is distorted by a
> long-tailed distribution.  Most people probably have in mind the big
> emacs packages with dynamic developer communities and independent
> release cycles.  Org, Gnus, CEDET, and maybe a few others.  On the other
> hand, it seems to me that you have in mind (in addition to these) all
> the tiny packages that see very few commits and perhaps no new features
> (in the extreme case take kermit.el, which has seen no substantive
> changes since at least 1992, but still gets its copyright header
> lovingly updated every year).  I don’t know how to reconcile these
> viewpoints, or even if it’s necessary.  Just something to consider.

The suggestion was to move _all_ of them, except the few that are
needed for bootstrap, out of the Emacs repository.  Most of the
packages in that category are neither like Org nor like kermit.  They
are relatively small, but get quite a significant number of changes.

> Speaking for myself, my primary interest in emacs development is working
> on org-mode, and I heavily use org as well as third-party packages for
> python (elpy), clojure (cider), R (ESS), and a few other random things.
> I build emacs from git every month or so in order to pick up little
> quality of life improvements, like with-eval-afer-load, some of the
> subr-x stuff, seq.el, the overhauled package menu, etc.  But I have to
> make sure to keep a couple months’ worth of old emacsen around, in case
> my monthly pull happens to land on a buggy commit – one that causes
> regular segfaults, which I have had happen more than once.  I use emacs
> for work so I don’t have the luxury of just going without for a few days
> until the problem is fixed.  I could go back to a previous released
> version of course, but then my init.el breaks because it relies on new
> features since the release.

I don't see how the issue at hand can affect you, then.  Your usage is
very similar to mine, FWIW.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 05:42 AM, Eli Zaretskii wrote:

>> Imagine someone implements an awesome new feature for dired.  Emacs
>> users the world over are amazed by this, and fill their blogs, twitter,
>> etc. with the news.  If dired is an ELPA package, everyone who hears
>> this news can get the new feature in their emacs instantly by upgrading
>> their ELPA packages.  No need to wait N months for a new release of
>> emacs, or compile a non-release version of emacs from git.
>
> How is this different when Dired is in the Emacs repository?  The
> Emacs repository is a public one, so anyone and everyone can get the
> latest version from there and use it, if they want.

a) That's a more involved endeavor than installing a package from ELPA. 
And then you don't get the same conveniences, such as automatic updates.

b) There's a much higher probability that Dired depends on something 
only the current development version of Emacs has. ELPA packages declare 
their version requirements explicitly, and try not to break 
compatibility with earlier versions without sufficient reasons.

> The suggestion was to move _all_ of them, except the few that are
> needed for bootstrap, out of the Emacs repository.  Most of the
> packages in that category are neither like Org nor like kermit.  They
> are relatively small, but get quite a significant number of changes.

There were different suggestions, with different degrees between "let's 
move Org and Gnus out" and "let's move everything out".

Re: ELPA policy

[Eli Zaretskii]

> Cc: Stromeko@nexgo.de, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 9 Nov 2015 05:51:16 +0200
> 
> On 11/09/2015 05:42 AM, Eli Zaretskii wrote:
> 
> >> Imagine someone implements an awesome new feature for dired.  Emacs
> >> users the world over are amazed by this, and fill their blogs, twitter,
> >> etc. with the news.  If dired is an ELPA package, everyone who hears
> >> this news can get the new feature in their emacs instantly by upgrading
> >> their ELPA packages.  No need to wait N months for a new release of
> >> emacs, or compile a non-release version of emacs from git.
> >
> > How is this different when Dired is in the Emacs repository?  The
> > Emacs repository is a public one, so anyone and everyone can get the
> > latest version from there and use it, if they want.
> 
> a) That's a more involved endeavor than installing a package from ELPA. 
> And then you don't get the same conveniences, such as automatic updates.

Nothing prevents us from making the same arrangements in the Emacs
repository as we have in ELPA, and then users could add the Emacs
repository to their list of sites that package.el knows about.

Besides, most core packages don't need any elaborate setup, you just
drop them in and restart Emacs.

> b) There's a much higher probability that Dired depends on something 
> only the current development version of Emacs has. ELPA packages declare 
> their version requirements explicitly, and try not to break 
> compatibility with earlier versions without sufficient reasons.

How will that change if we move them outside Emacs?  It's not like
there's some magic at ELPA that just putting a package there magically
resolves all these issues.  And if someone will have to work on that,
she can do that without moving the package aside.

> > The suggestion was to move _all_ of them, except the few that are
> > needed for bootstrap, out of the Emacs repository.  Most of the
> > packages in that category are neither like Org nor like kermit.  They
> > are relatively small, but get quite a significant number of changes.
> 
> There were different suggestions, with different degrees between "let's 
> move Org and Gnus out" and "let's move everything out".

I didn't see any.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 06:05 PM, Eli Zaretskii wrote:

> Nothing prevents us from making the same arrangements in the Emacs
> repository as we have in ELPA, and then users could add the Emacs
> repository to their list of sites that package.el knows about.

Create yet another package archive? With a separate publishing mechanism?

> Besides, most core packages don't need any elaborate setup, you just
> drop them in and restart Emacs.

"Just drop them in" doesn't address the issue of package updates, not in 
the slightest.

> How will that change if we move them outside Emacs?  It's not like
> there's some magic at ELPA that just putting a package there magically
> resolves all these issues.  And if someone will have to work on that,
> she can do that without moving the package aside.

They will be forced to declare the required Emacs version, for one. And 
abide by it.

But yes, that's also possible to do inside the Emacs repo.

>> There were different suggestions, with different degrees between "let's
>> move Org and Gnus out" and "let's move everything out".
>
> I didn't see any.

All right, then. Let me make one right now: let's move out Org, Gnus and 
CEDET. And consider doing that with any other big codebase that's also 
maintained separately (Calc comes to mind).

There have been objections to doing that for each of these packages, but 
there you go. Suggestion made.

Re: ELPA policy

[Eli Zaretskii]

> Cc: aaronecay@gmail.com, Stromeko@nexgo.de, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 9 Nov 2015 18:15:28 +0200
> 
> On 11/09/2015 06:05 PM, Eli Zaretskii wrote:
> 
> > Nothing prevents us from making the same arrangements in the Emacs
> > repository as we have in ELPA, and then users could add the Emacs
> > repository to their list of sites that package.el knows about.
> 
> Create yet another package archive? With a separate publishing mechanism?

Why not?  There are several already, no?

> > Besides, most core packages don't need any elaborate setup, you just
> > drop them in and restart Emacs.
> 
> "Just drop them in" doesn't address the issue of package updates, not in 
> the slightest.

It does, for most of the core.  That's what you do each time you say
"git pull", I believe.

> >> There were different suggestions, with different degrees between "let's
> >> move Org and Gnus out" and "let's move everything out".
> >
> > I didn't see any.
> 
> All right, then. Let me make one right now: let's move out Org, Gnus and 
> CEDET. And consider doing that with any other big codebase that's also 
> maintained separately (Calc comes to mind).
> 
> There have been objections to doing that for each of these packages, but 
> there you go. Suggestion made.

It was made before -- that's the "let's move Org and Gnus out"
variant, to which I said I could easily agree.  And then there was the
"move everything" one, to which I object for the reasons stated.  What
I meant was that there was no 3rd variant, AFAIR.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 06:20 PM, Eli Zaretskii wrote:

>> Create yet another package archive? With a separate publishing mechanism?
>
> Why not?  There are several already, no?

So far, it seems we're short of manpower to manage even the current one. 
To manage the infrastructure, improve the scripts, etc.

> It does, for most of the core.  That's what you do each time you say
> "git pull", I believe.

I cannot "git pull" a new version of Dired into my stable installation 
of Emacs 24.5.

> It was made before -- that's the "let's move Org and Gnus out"
> variant, to which I said I could easily agree.  And then there was the
> "move everything" one, to which I object for the reasons stated.  What
> I meant was that there was no 3rd variant, AFAIR.

Well, I enumerated 4 packages instead of 2, didn't I? :)

I agree that "move everything" is a non-starter (or very hard, at 
least). So it seems more like a red herring, there's no real use arguing 
against it.

Re: ELPA policy

[Eli Zaretskii]

> Cc: aaronecay@gmail.com, Stromeko@nexgo.de, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 9 Nov 2015 18:26:58 +0200
> 
> On 11/09/2015 06:20 PM, Eli Zaretskii wrote:
> 
> >> Create yet another package archive? With a separate publishing mechanism?
> >
> > Why not?  There are several already, no?
> 
> So far, it seems we're short of manpower to manage even the current one. 
> To manage the infrastructure, improve the scripts, etc.

Is it really such a significant effort?  I wasn't aware of that.

> > It does, for most of the core.  That's what you do each time you say
> > "git pull", I believe.
> 
> I cannot "git pull" a new version of Dired into my stable installation 
> of Emacs 24.5.

Of course you can: just copy the file over.

> > It was made before -- that's the "let's move Org and Gnus out"
> > variant, to which I said I could easily agree.  And then there was the
> > "move everything" one, to which I object for the reasons stated.  What
> > I meant was that there was no 3rd variant, AFAIR.
> 
> Well, I enumerated 4 packages instead of 2, didn't I? :)

We could probably find a few more.  MH-E, for example.  (Of course,
their developers will have to agree to move.)

> I agree that "move everything" is a non-starter (or very hard, at 
> least). So it seems more like a red herring, there's no real use arguing 
> against it.

Agreed.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 06:44 PM, Eli Zaretskii wrote:

>> So far, it seems we're short of manpower to manage even the current one.
>> To manage the infrastructure, improve the scripts, etc.
>
> Is it really such a significant effort?  I wasn't aware of that.

There are things to be done, and just a couple months ago nobody was 
doing them. I see some movement now, e.g. the last commit by Thomas 
Fitzsimmons.

>> I cannot "git pull" a new version of Dired into my stable installation
>> of Emacs 24.5.
>
> Of course you can: just copy the file over.

Automatic update is a really handy feature when you have 50+ packages 
installed. See some user statistics at 
https://www.reddit.com/r/emacs/comments/3s4k03/how_many_packages_do_you_use/

> We could probably find a few more.  MH-E, for example.  (Of course,
> their developers will have to agree to move.)

Yeah, I bet the majority of our users are unaware of what MH-E is.

Re: ELPA policy

[Artur Malabarba]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Nothing prevents us from making the same arrangements in the Emacs
>> > repository as we have in ELPA, and then users could add the Emacs
>> > repository to their list of sites that package.el knows about.
>> 
>> Create yet another package archive? With a separate publishing mechanism?
>
> Why not?  There are several already, no?

This sounds already quite similar to the system that Fabian implemented
a couple months ago (which I hear isn't quite perfect yet, but should be
almost there).

With this system, the daily script that Gelpa runs would copy some
packages straight from Emacs core. This way you could very well
configure Gelpa to always offer the lastest Dired (or maybe only when we
bump a version number, but that's a detail).

Re: ELPA policy

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> It was made before -- that's the "let's move Org and Gnus out" variant, to
> which I said I could easily agree. And then there was the "move everything"
> one, to which I object for the reasons stated. What I meant was that there
> was no 3rd variant, AFAIR.

Wow, a lot of discussion about ELPA policy, this is great. We definitely have
an opportunity here to bring clarity to an area that is on people's minds.

I agree with a lot of what I read, although it was a too spread out for me to
add specific quotes here. Let me just summarize a possible path forward:

 1. Things that are maintained by the core Emacs developers should be in core,
    and in the Emacs.git repository. This makes it easy for them to access and
    build, search history, read emacs-diffs, etc.

 2. Things that are maintained outside of Emacs, but contributed back for
    inclusion, should be in ELPA, and in the Elpa.git repository. This
    includes Gnus, Org, CEDET, and a few more. (We don't want to go crazy,
    so let's start with the big ones).

 3. There should be a defined subset of packages that get copied from ELPA
    into the Emacs tarball during release, and an easy way to manage this list
    for the core developers. That way, certain packages like seq.el and
    stream.el can feel like "core" packages to users, when they are really
    "external" packages from the point of view of the core developers.

 4. Everything else in ELPA is Internet-installation based, as it is today.

I would very much like for ELPA to be a way to:

  - give the core developers a smaller surface area to focus on;

  - allow ELPA package maintainers to "have their package in Emacs", while
    retaining nimble in their own development process and in delivering
    updates to users;

  - allow the core maintainers to decide what exactly is going into "Emacs":
    For example, I wouldn't want to pull Org releases into the distribution
    from a submodule; I really do want a version of that source code to be in
    ELPA, so we can separately patch if there are last minute problems.

ELPA should thus benefit core developers by reducing load, and benefit package
maintainers by being an easier platform for their contributions and their
users. If it causes extra work for anyone, that's something I'd want to
change.

There are three actions I'd like to take from here:

  a. That we clearly document an ELPA policy we all agree on;

  b. That we move "external" packages from core into ELPA, starting with the
     really big ones;

  c. That we assess any points of friction after doing so, and adjust as
     necessary.

John

Re: ELPA policy

[Achim Gratz]

John Wiegley writes:
>  3. There should be a defined subset of packages that get copied from ELPA
>     into the Emacs tarball during release, and an easy way to manage this list
>     for the core developers. That way, certain packages like seq.el and
>     stream.el can feel like "core" packages to users, when they are really
>     "external" packages from the point of view of the core developers.

They could (and probably should) effectively be in the build environment
as well.  In other words, I would try to not pull them into the release
tarball at the last possible moment.  One thing that would make
difficult is to test interactions between such "external core" packages
and here I have to agree with Eli that this has potential for
degradation of quality.

>   - allow the core maintainers to decide what exactly is going into "Emacs":
>     For example, I wouldn't want to pull Org releases into the distribution
>     from a submodule; I really do want a version of that source code to be in
>     ELPA, so we can separately patch if there are last minute problems.

You'd do that with branches if it really becomes necessary.  I don't
really see why submodules could not be used, except for the extra rope
they give you to hang yourself with.  Any other solution is going to
have the same basic complexity beneath and the potential to not work on
some platform or other.

> ELPA should thus benefit core developers by reducing load, and benefit package
> maintainers by being an easier platform for their contributions and their
> users. If it causes extra work for anyone, that's something I'd want to
> change.

I maintain that ELPA should benefit Emacs users first.

>   b. That we move "external" packages from core into ELPA, starting with the
>      really big ones;

...after the necessary changes to Emacs' build system and package.el are
architected and implemented.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: ELPA policy

[John Wiegley]

>>>>> Achim Gratz <Stromeko@nexgo.de> writes:

> They could (and probably should) effectively be in the build environment as
> well. In other words, I would try to not pull them into the release tarball
> at the last possible moment. One thing that would make difficult is to test
> interactions between such "external core" packages and here I have to agree
> with Eli that this has potential for degradation of quality.

Could this be solved by branching ELPA leading up to a release -- this being
the "frozen ELPA" that we do integration against before cutting the tarball --
while cherry-picking fixes in from master? I think the way we proceed here
would be similar to what we do for Emacs itself.

> You'd do that with branches if it really becomes necessary. I don't really
> see why submodules could not be used, except for the extra rope they give
> you to hang yourself with. Any other solution is going to have the same
> basic complexity beneath and the potential to not work on some platform or
> other.

A submodule doesn't allow us to have "Org + some-patch-not-yet-in-Org".

> I maintain that ELPA should benefit Emacs users first.

I'm intrigued; could you clarify this a bit more? I saw what you wrote in
response to Eli, but that didn't really help. Specifically, do you have a
scenario in mind that is better for users and worse for developers?

>> b. That we move "external" packages from core into ELPA, starting with the
>> really big ones;

> ...after the necessary changes to Emacs' build system and package.el are
> architected and implemented.

Agreed.

John

Re: ELPA policy

[Achim Gratz]

John Wiegley writes:
>> You'd do that with branches if it really becomes necessary. I don't really
>> see why submodules could not be used, except for the extra rope they give
>> you to hang yourself with. Any other solution is going to have the same
>> basic complexity beneath and the potential to not work on some platform or
>> other.
>
> A submodule doesn't allow us to have "Org + some-patch-not-yet-in-Org".

You'd branch on ELPA to something like emacs-25.0.50, patch away and
point the submodule there if you really can't wait for the patch to be
committed in Org upstream and mirrored back to ELPA.

>> I maintain that ELPA should benefit Emacs users first.
>
> I'm intrigued; could you clarify this a bit more? I saw what you wrote in
> response to Eli, but that didn't really help. Specifically, do you have a
> scenario in mind that is better for users and worse for developers?

It was a long day and I'm afraid I'd not be too coherent, so I'll punt
for now.  But in short, look at CPAN, CRAN and CTAN for NIH examples of
what ELPA might possibly become.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I agree with a lot of what I read, although it was a too spread out for me to
  > add specific quotes here. Let me just summarize a possible path forward:

This plan seems good, overall, But in the specific case of stream.el
and seq.el, shouldn't they be maintained in the core?  I don't see
stream.el in the sources I fetched, but seq.el is small, in emacs-lisp,
and it says Maintainer: emacs-devel.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> This plan seems good, overall, But in the specific case of stream.el and
> seq.el, shouldn't they be maintained in the core? I don't see stream.el in
> the sources I fetched, but seq.el is small, in emacs-lisp, and it says
> Maintainer: emacs-devel.

If that's the case, then yes, it should be in core. I'm personally a bit
surprised that a library of this nature is in ELPA, unless it was being
provided and actively maintained by a non-core developer.

John

Re: ELPA policy

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 748 bytes --]

On 9 Nov 2015 10:06 pm, "John Wiegley" <jwiegley@gmail.com> wrote:
>
> >>>>> Richard Stallman <rms@gnu.org> writes:
>
> > This plan seems good, overall, But in the specific case of stream.el and
> > seq.el, shouldn't they be maintained in the core? I don't see stream.el
in
> > the sources I fetched, but seq.el is small, in emacs-lisp, and it says
> > Maintainer: emacs-devel.
>
> If that's the case, then yes, it should be in core. I'm personally a bit
> surprised that a library of this nature is in ELPA, unless it was being
> provided and actively maintained by a non-core developer.

Seq.el was put on Gelpa so that it can be used by emacsen < 25. I'm very
grateful for that as I use it as a dependency in quite a few of my packages
already.

[-- Attachment #2: Type: text/html, Size: 1009 bytes --]

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Seq.el was put on Gelpa so that it can be used by emacsen < 25.

It is fine to have it there, but this suggests we should maintain
it inside Emacs.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 419 bytes --]

Richard Stallman <rms@gnu.org> writes:

> It is fine to have it there, but this suggests we should maintain
> it inside Emacs.

I'm not sure I understand what you mean Richard.  seq.el is maintained
inside Emacs (in lisp/emacs-lisp/seq.el and
test/automated/seq-tests.el).  The version published in ELPA is there
for backward-compatibility, and does not have all the features (some
features do require Emacs 25).

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > It is fine to have it there, but this suggests we should maintain
  > > it inside Emacs.

  > I'm not sure I understand what you mean Richard.

I mean that we should maintain the file inside Emacs, 
not inside ELPA.

  >   seq.el is maintained
  > inside Emacs (in lisp/emacs-lisp/seq.el and
  > test/automated/seq-tests.el).

That means people are already doing what I think we should do.
I didn't have this information when I wrote my previous message.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

John Wiegley <jwiegley@gmail.com> writes:

> If that's the case, then yes, it should be in core. I'm personally a bit
> surprised that a library of this nature is in ELPA,

In the case of seq.el, it's for backward-compatibility.  It made it
possible for instance for CIDER devs to start using it while Emacs 25 is
still in progress.

> unless it was being
> provided and actively maintained by a non-core developer.

IIRC, Stefan thought that having stream.el in the ELPA meant more sense
at least until people start using it more.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: ELPA policy

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 424 bytes --]

Richard Stallman <rms@gnu.org> writes:

> This plan seems good, overall, But in the specific case of stream.el
> and seq.el, shouldn't they be maintained in the core?  I don't see
> stream.el in the sources I fetched, but seq.el is small, in emacs-lisp,
> and it says Maintainer: emacs-devel.

stream.el is currently in ELPA, which can make some sense, but regarding
seq.el, I think it should really stay in the core.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: ELPA policy

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]

John Wiegley <jwiegley@gmail.com> writes:

>  3. There should be a defined subset of packages that get copied from ELPA
>     into the Emacs tarball during release, and an easy way to manage this list
>     for the core developers. That way, certain packages like seq.el and
>     stream.el can feel like "core" packages to users, when they are really
>     "external" packages from the point of view of the core developers.

I only added seq.el to ELPA for backward compatibility with Emacs 24.5.
The development of seq.el (and map.el for that matter) happen only in
Emacs trunk.

Moving it out of lisp/emacs-lisp/, and only having it copied in the
release tarballs would be a step backward IMHO, it would make it hard to
use it in core packages, and would basically mean demoting the libraries
to external ones while I'm really working on seq.el to provide a good,
consistent and comprehensive *built-in* sequence library to
Emacs-Lisp. That's also why all the functions are documented in the
Sequences chapter of the Elisp documentation.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: ELPA policy

[Aaron Ecay]

Hi John,

Thanks for setting this out very concretely.

2015ko azaroak 9an, John Wiegley-ek idatzi zuen:
> 
>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
>> It was made before -- that's the "let's move Org and Gnus out" variant, to
>> which I said I could easily agree. And then there was the "move everything"
>> one, to which I object for the reasons stated. What I meant was that there
>> was no 3rd variant, AFAIR.
> 
> Wow, a lot of discussion about ELPA policy, this is great. We definitely have
> an opportunity here to bring clarity to an area that is on people's minds.
> 
> I agree with a lot of what I read, although it was a too spread out for me to
> add specific quotes here. Let me just summarize a possible path forward:
> 
>  1. Things that are maintained by the core Emacs developers should be in core,
>     and in the Emacs.git repository. This makes it easy for them to access and
>     build, search history, read emacs-diffs, etc.
> 
>  2. Things that are maintained outside of Emacs, but contributed back for
>     inclusion, should be in ELPA, and in the Elpa.git repository. This
>     includes Gnus, Org, CEDET, and a few more. (We don't want to go crazy,
>     so let's start with the big ones).
> 
>  3. There should be a defined subset of packages that get copied from ELPA
>     into the Emacs tarball during release, and an easy way to manage this list
>     for the core developers. That way, certain packages like seq.el and
>     stream.el can feel like "core" packages to users, when they are really
>     "external" packages from the point of view of the core developers.
> 
>  4. Everything else in ELPA is Internet-installation based, as it is
>     today.

The one thing that I see missing from this list is a way to take
packages that are developed in emacs.git and distribute them on an
ELPA.  (Either GNU ELPA, or a (g)new ELPA as was suggested by Eli.)
This would be the inverse of your 3: packages that feel like core
packages to emacs devs, but feel like external packages to users (at
least in the sense of receiving updates outside of the emacs release
cycle).

I’m not sure if that omission was intentional or not.  I hope it wasn’t:
this model of distribution has been chosen for several important
“low-level” packages, like seq, cl-lib, and let-alist.

If this addition is desired then:

[...]

> 
> There are three actions I'd like to take from here:
> 
>   a. That we clearly document an ELPA policy we all agree on;
> 
>   b. That we move "external" packages from core into ELPA, starting with the
>      really big ones;
> 
>   c. That we assess any points of friction after doing so, and adjust as
>      necessary.

This list needs an additional item, namely the development of the
script that publishes elisp from emacs.git to an ELPA.  I think people
have said that Fabian is already working on this.

Thanks,

-- 
Aaron Ecay

Re: ELPA policy

[John Wiegley]

>>>>> Aaron Ecay <aaronecay@gmail.com> writes:

> I’m not sure if that omission was intentional or not. I hope it wasn’t: this
> model of distribution has been chosen for several important “low-level”
> packages, like seq, cl-lib, and let-alist.

[...]

> This list needs an additional item, namely the development of the script
> that publishes elisp from emacs.git to an ELPA. I think people have said
> that Fabian is already working on this.

It was an unintentional omission, and I agree that we should support that
usage pattern as well. So let it be 4 modes of looking at "Emacs packages",
then.

John

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>
>> It was made before -- that's the "let's move Org and Gnus out" variant, to
>> which I said I could easily agree. And then there was the "move everything"
>> one, to which I object for the reasons stated. What I meant was that there
>> was no 3rd variant, AFAIR.
>
> Wow, a lot of discussion about ELPA policy, this is great. We definitely have
> an opportunity here to bring clarity to an area that is on people's minds.
>
> I agree with a lot of what I read, although it was a too spread out for me to
> add specific quotes here. Let me just summarize a possible path forward:
>
>  1. Things that are maintained by the core Emacs developers should be in core,
>     and in the Emacs.git repository. This makes it easy for them to access and
>     build, search history, read emacs-diffs, etc.
>
>  2. Things that are maintained outside of Emacs, but contributed back for
>     inclusion, should be in ELPA, and in the Elpa.git repository. This
>     includes Gnus, Org, CEDET, and a few more. (We don't want to go crazy,
>     so let's start with the big ones).
>
>  3. There should be a defined subset of packages that get copied from ELPA
>     into the Emacs tarball during release, and an easy way to manage this list
>     for the core developers. That way, certain packages like seq.el and
>     stream.el can feel like "core" packages to users, when they are really
>     "external" packages from the point of view of the core developers.

ELPA packages that other core code depends on (like CEDET; xref uses it
- called "core ELPA packages" hereinafter) have to be in every
developer's build environment everyday; the other core code has to see
the current version of the package, and it has to be the same for every
developer.

If core ELPA packages are in the ELPA git repository, you would copy
some subtrees of the ELPA git workspace into the Emacs git workspace.
People have looked into doing this, but it gets messy and confusing.
"git fetch" is dealing with two upstreams for one workspace. There has
been talk about git submodules, etc, but nothing concrete.

The alternative, which is being implemented by the :core external
package feature in Gnu ELPA, and has been shown to work (but has some
details to work out), is to keep core ELPA packages in the emacs
git repository. Then they are part of the developer's build environment
using the current git workflow, and they are included in both the emacs
tarball and the ELPA package server with the current release workflows.


There are two rationales for moving a package to ELPA:

1) Allow a release schedule independent of the core Emacs release
   schedule.

2) Reduce the size of the Emacs git workspace, which simplifies managing
   it.

But a _core_ ELPA package must remain in the Emacs git workspace, so the
only rationale for moving a core package to ELPA is for an independent
release schedule.

> There are three actions I'd like to take from here:
>
>   a. That we clearly document an ELPA policy we all agree on;

In admin/notes/elpa ?

>   b. That we move "external" packages from core into ELPA, starting with the
>      really big ones;

It's not clear that "really big" implies "independent release schedule".
"externally managed" implies that.

"not used by other core code" is also a good reason to move a package to
ELPA. 

>   c. That we assess any points of friction after doing so, and adjust as
>      necessary.

If we use the approach of keeping core ELPA packages in the Emacs git
repository, there is _zero_ friction for the current core Emacs
developers; the only change is in the ELPA config scripts. 

-- 
-- Stephe

Re: ELPA policy

[John Wiegley]

>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:

> ELPA packages that other core code depends on (like CEDET; xref uses it -
> called "core ELPA packages" hereinafter) have to be in every developer's
> build environment everyday; the other core code has to see the current
> version of the package, and it has to be the same for every developer.
> 
> If core ELPA packages are in the ELPA git repository, you would copy some
> subtrees of the ELPA git workspace into the Emacs git workspace. People have
> looked into doing this, but it gets messy and confusing. "git fetch" is
> dealing with two upstreams for one workspace. There has been talk about git
> submodules, etc, but nothing concrete.

Elpa.git should be a submodule referenced from within Emacs.git (under "elpa").
This allows us to expressly state which "version" of Elpa is expected to work
with the current Emacs.git.

Large packages like CEDET should move outside of Emacs.git and into Elpa.git.
If xref.el depends on CEDET, it would move to Elpa.git as well.

> If we use the approach of keeping core ELPA packages in the Emacs git
> repository, there is _zero_ friction for the current core Emacs developers;
> the only change is in the ELPA config scripts.

I do not want this code replication. We will promote ELPA as an active
development space alongside core Emacs. If we use a submodule, and improve our
tooling slightly, it should become a seamless thing.

John

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:28 PM, John Wiegley wrote:

> If xref.el depends on CEDET, it would move to Elpa.git as well.

I can cut out xref.el's dependency on CEDET easily enough.

Moving it into ELPA would be kinda problematic, because guess what M-. 
and M-, and bound to by default now.

Re: ELPA policy

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> I can cut out xref.el's dependency on CEDET easily enough.

Great, let's do that. I'd prefer to have the functionality of xref.el in core.

Thanks,
  John

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> I can cut out xref.el's dependency on CEDET easily enough.
>
> Great, let's do that. I'd prefer to have the functionality of xref.el in core.

So that means xref won't support CEDET out of the box? What is gained by
that, exactly?

-David

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> So that means xref won't support CEDET out of the box? What is gained by
> that, exactly?

It could always support it with customization hooks.

John

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:44 PM, David Engster wrote:

> So that means xref won't support CEDET out of the box? What is gained by
> that, exactly?

It doesn't support CEDET even now. Nobody wrote the Semantic 
implementation of a xref backend, and it might be tool late now for the 
25.1 release. If CEDET were moved to ELPA, however, you can add that 
support at your leisure later.

And that's an orthogonal issue. xref has no need to support CEDET: CEDET 
should support xref (the support code will be in CEDET's tree).

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Tue, 10 Nov 2015 10:35:13 -0800
> Cc: aaronecay@gmail.com, Eli Zaretskii <eliz@gnu.org>, Stromeko@nexgo.de,
> 	Stephen Leake <stephen_leake@stephe-leake.org>, emacs-devel@gnu.org
> 
> >>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
> > I can cut out xref.el's dependency on CEDET easily enough.
> 
> Great, let's do that.

Sounds like a step backward to me.  CEDET allows xref to be more
versatile and support more back-ends.

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:35 PM, John Wiegley wrote:
>>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> I can cut out xref.el's dependency on CEDET easily enough.
>
> Great, let's do that. I'd prefer to have the functionality of xref.el in core.

Let me know when it becomes a blocker, and I'll do it in the simplest 
way. But I'd like to defer that for now, because it comes with making 
choices (see the other messages in this thread), and I'm running low on 
decision-making juice these days.

Re: ELPA policy

[David Engster]

Dmitry Gutov writes:
> On 11/10/2015 08:28 PM, John Wiegley wrote:
>
>> If xref.el depends on CEDET, it would move to Elpa.git as well.
>
> I can cut out xref.el's dependency on CEDET easily enough.

Which illustrates nicely the downside of moving packages to ELPA.

-David

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:37 PM, David Engster wrote:

>> I can cut out xref.el's dependency on CEDET easily enough.
>
> Which illustrates nicely the downside of moving packages to ELPA.

Nope. It will be beneficial to the code in Emacs, because xref is 
currently depending on a minor feature of CEDET (the 
semantic-symref-tool infrastructure), which could use a rewrite, to be 
available for a more general usage.

Re: ELPA policy

[Eli Zaretskii]

> Cc: Stephen Leake <stephen_leake@stephe-leake.org>,
>  Eli Zaretskii <eliz@gnu.org>, aaronecay@gmail.com, Stromeko@nexgo.de,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 10 Nov 2015 21:57:25 +0200
> 
> On 11/10/2015 08:37 PM, David Engster wrote:
> 
> >> I can cut out xref.el's dependency on CEDET easily enough.
> >
> > Which illustrates nicely the downside of moving packages to ELPA.
> 
> Nope. It will be beneficial to the code in Emacs, because xref is 
> currently depending on a minor feature of CEDET (the 
> semantic-symref-tool infrastructure), which could use a rewrite, to be 
> available for a more general usage.

I think a point we should take from this is that the prerequisites (in
this case, the said rewrite) should be in place _before_ we seriously
consider such moves (pun intended ;-).

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 10:01 PM, Eli Zaretskii wrote:

> I think a point we should take from this is that the prerequisites (in
> this case, the said rewrite) should be in place _before_ we seriously
> consider such moves (pun intended ;-).

The only prerequisite is to have this functionality (which is fairly 
self-contained) in the core. It would benefit from a rewrite (and moving 
away from CEDET would encourage that), but as the first step, we can 
just copy it and do some renaming.

However, like mentioned in the "project.el semantics" thread:

   I'm seriously considering to get away from it and always use Grep,
   because the other tools, while they can be faster, they're also
   entirely opaque to the user, can have outdated databases, non-indexed
   relevant directories, unsupported languages, and so on.

So this feature, which xref is depending on for the default 
xref-find-references implementation, requires certain amount of 
hand-holding from the user. And I don't know how to approach documenting 
it, and otherwise make it apparent to the user.

So someone else should probably start on taking care of that. Otherwise, 
I'd rather fall back on Grep, until such point that Emacs starts to 
manage the Global/id-utils/CScope datbases for the user, and learns to 
make sure that the database is up-to-date.

Re: ELPA policy

[Eli Zaretskii]

> Cc: deng@randomsample.de, stephen_leake@stephe-leake.org,
>  aaronecay@gmail.com, Stromeko@nexgo.de, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 10 Nov 2015 22:19:46 +0200
> 
> So someone else should probably start on taking care of that. Otherwise, 
> I'd rather fall back on Grep, until such point that Emacs starts to 
> manage the Global/id-utils/CScope datbases for the user, and learns to 
> make sure that the database is up-to-date.

Grep doesn't scale well to large projects, IME.  You get too many
false positives.

Outdated databases are easy to avoid with the likes of cron jobs.
Yes, that's hand-holding, but when you have to quickly find stuff in a
project with 3 million lines of code and thousands of classes, there
really is no other alternative.

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 10:34 PM, Eli Zaretskii wrote:

> Grep doesn't scale well to large projects, IME.  You get too many
> false positives.

We can also check the "inside string or comment" status, as well as use 
the Emacs syntax tables to make sure that the match begins and ends at 
symbol boundaries appropriate for the file's language. Everything 
necessary for that is already written (and you can try it out by calling 
project-find-regexp and using something like \_<tool_bar_items\_> as the 
regexp). I don't think that id-utils does anything more to weed out 
false positives.

Grep is still probably going to be slower than at least some of the 
tools in question. Could you test, on a large project of your choice?

> Outdated databases are easy to avoid with the likes of cron jobs.
> Yes, that's hand-holding, but when you have to quickly find stuff in a
> project with 3 million lines of code and thousands of classes, there
> really is no other alternative.

Yes. But the xref commands should be easy to use. Even if the above is 
not rocket science, the user would still have to know what they need to 
do to get up-to-date results. (Believe it or not, I haven't created a 
single cron job for code writing purposes in my life, and I don't know 
its syntax for the time intervals. I'm likely not alone in that.)

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 11:16 PM, Dmitry Gutov wrote:

> project-find-regexp and using something like \_<tool_bar_items\_> as the

Sorry, using the Emacs-only syntax there doesn't exactly work yet. But 
you can compare by calling xref-find-references when the project has an 
id-utils index file, and doing the same without it.

Re: ELPA policy

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> However, like mentioned in the "project.el semantics" thread:

please change the thread subject.

-- 
-- Stephe

Re: ELPA policy

[David Engster]

Dmitry Gutov writes:
> On 11/10/2015 08:37 PM, David Engster wrote:
>
>>> I can cut out xref.el's dependency on CEDET easily enough.
>>
>> Which illustrates nicely the downside of moving packages to ELPA.
>
> Nope. It will be beneficial to the code in Emacs, because xref is
> currently depending on a minor feature of CEDET (the
> semantic-symref-tool infrastructure), which could use a rewrite, to be
> available for a more general usage.

As you know, we will drop our upstream repository. During the last
merge, I will also have to discuss with Eric what we can do to narrow
CEDET's scope. I agree with you that semantic-symref is probably one of
the packages that should be replaced with something more general. But as
long as it's there, I'd appreciate if you could leave whatever support
you have coded for it.

I'd also be happy to drop the completion GUI stuff, BTW. I've repeatedly
said that company should be in core for that.

-David

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 10:45 PM, David Engster wrote:

> As you know, we will drop our upstream repository. During the last
> merge, I will also have to discuss with Eric what we can do to narrow
> CEDET's scope.
> I agree with you that semantic-symref is probably one of
> the packages that should be replaced with something more general.

In the end, I would leave only the "backend" part of semantic-symref 
(calling the tools + piping the results through Semantic), and use it in 
the Semantic xref backend. So the results will be displayed by xref.

Alas, xref is not up to feature parity yet (cannot group hits by 
containing functions, no checkboxes, simplistic "rename" command), and 
I'm having difficulties deciding on the changes to the xref data 
structures that would make that possible, as well as keep allowing 
certain other uses of xref, such as project-find-regexp.

I'll write that up in a separate thread later this week.

> But as
> long as it's there, I'd appreciate if you could leave whatever support
> you have coded for it.

Just to be clear, there's no "support" for semantic-symref in xref yet. 
We just use some low-level pieces of it as a "fast Grep".

> I'd also be happy to drop the completion GUI stuff, BTW. I've repeatedly
> said that company should be in core for that.

Also not exactly necessary. Semantic should make its 
completion-at-point-functions element the main entry point for 
completion, and Company will pick it up. You can even have argument list 
annotations, if you add the :annotation-function property.

I do want to have Company more widely available, but probably not in 
25.1. There are quite a few things I wanted to do with it first, and 
most are still untouched.

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>> ELPA packages that other core code depends on (like CEDET; xref uses it -
>> called "core ELPA packages" hereinafter) have to be in every developer's
>> build environment everyday; the other core code has to see the current
>> version of the package, and it has to be the same for every developer.
>> 
>> If core ELPA packages are in the ELPA git repository, you would copy some
>> subtrees of the ELPA git workspace into the Emacs git workspace. People have
>> looked into doing this, but it gets messy and confusing. "git fetch" is
>> dealing with two upstreams for one workspace. There has been talk about git
>> submodules, etc, but nothing concrete.
>
> Elpa.git should be a submodule referenced from within Emacs.git (under "elpa").
> This allows us to expressly state which "version" of Elpa is expected to work
> with the current Emacs.git.

Since ELPA comprises many packages, that simply cannot work.

> Large packages like CEDET should move outside of Emacs.git and into Elpa.git.

Why?

> I do not want this code replication. We will promote ELPA as an active
> development space alongside core Emacs. If we use a submodule, and improve our
> tooling slightly, it should become a seamless thing.

From my experience, git submodules are never seamless to work with.

-David

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

>> Elpa.git should be a submodule referenced from within Emacs.git (under "elpa").
>> This allows us to expressly state which "version" of Elpa is expected to work
>> with the current Emacs.git.

> Since ELPA comprises many packages, that simply cannot work.

Perhaps I should say, "The version of Elpa.git, a subset of which will appear
in the release tarball of this commit".

>> Large packages like CEDET should move outside of Emacs.git and into
>> Elpa.git.

> Why?

There will never be 100% agreement on whether they should be in ELPA, or be in
Core, so I'm making the decision that they belong in ELPA. The big three that
will be moved first are: Gnus, Org and CEDET.

> From my experience, git submodules are never seamless to work with.

Even still, they work perfectly fine for this purpose.

John

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>> Large packages like CEDET should move outside of Emacs.git and into
>>> Elpa.git.
>
>> Why?
>
> There will never be 100% agreement on whether they should be in ELPA, or be in
> Core, so I'm making the decision that they belong in ELPA.

So you simply don't want to give any reason?

-David

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> So you simply don't want to give any reason?

David, although I work by consensus as much as possible, I won't explain
everything I choose to do. My reason is that I think this will improve Emacs
development.

Part of my goal is to make the boundary between Emacs and ELPA thinner, so it
will actually be *easier* to move CEDET and friends back into core later, if
my reasoning is wrong.

John

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>>>>> David Engster <deng@randomsample.de> writes:
>
>> So you simply don't want to give any reason?
>
> David, although I work by consensus as much as possible, I won't explain
> everything I choose to do. My reason is that I think this will improve Emacs
> development.

This is not about reaching a consensus. This is about you giving proper
reasons for such a big change.

-David

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> This is not about reaching a consensus. This is about you giving proper
> reasons for such a big change.

To be clear, I would also put Eshell (though not pcomplete) into the category
of "big things that should be in ELPA" -- albeit, the subset of ELPA that will
be in the release tarball.

It's hard to pin down why in a manner that fits in an e-mail. If Eshell were
in ELPA today, and we were talking about moving it into core, I would have
just as much trouble justifying that move too. Perhaps this simply underscores
the fact that we don't have an agreed upon ELPA policy we can all refer to.

OK, David, I won't decide this by fiat. Let us decide what our ELPA policy
will be, until it becomes clear, based on that document, what should go where,
and why. We'll defer discussions of package movement until we have that in
place.

John

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>>>>> David Engster <deng@randomsample.de> writes:
>
>> This is not about reaching a consensus. This is about you giving proper
>> reasons for such a big change.
>
> To be clear, I would also put Eshell (though not pcomplete) into the category
> of "big things that should be in ELPA" -- albeit, the subset of ELPA that will
> be in the release tarball.
>
> It's hard to pin down why in a manner that fits in an e-mail. If Eshell were
> in ELPA today, and we were talking about moving it into core, I would have
> just as much trouble justifying that move too. Perhaps this simply underscores
> the fact that we don't have an agreed upon ELPA policy we can all refer to.

In my opinion, the main question is whether something provides
infrastructure for other packages to use. This is precisely what CEDET
tries to do. I wouldn't have much trouble with putting parts of CEDET in
ELPA, namely those parts that do not directly provide infrastructure,
like support for certain languages, project types, indexing tools, etc.

> OK, David, I won't decide this by fiat. Let us decide what our ELPA policy
> will be, until it becomes clear, based on that document, what should go where,
> and why. We'll defer discussions of package movement until we have that in
> place.

It is still not clear to me what exactly is gained by moving core
packages to ELPA. In my opinion, packages like Org, Gnus or CEDET didn't
create significant problems for Emacs development in the past. On the
contrary, it made sure that those packages were adapted quickly when
larger changes in core were made. Those changes got synced back upstream
by people like Bastien, Katsumi and me. So in my opinion, you are trying
to fix something that is not broken.

What actually *does* create problems are packages that don't have an
active maintainer. The 'big three' however are not among those.

-David

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> the main question is whether something provides infrastructure for other
> packages to use.

Sounds like a good sentence for an ELPA policy. :)

> It is still not clear to me what exactly is gained by moving core packages
> to ELPA.

Agility. What is appropriate. Knowing when a thing goes into core, and when in
ELPA.

Org is an application, it's not infrastructure; the same with Gnus. *Parts* of
Gnus might rightly be considered infrastructure, but the whole of Gnus just
doesn't belong there. Parts of CEDET probably do belong in Emacs core, but as
an application, I don't think the whole of it does.

Since we don't have an agreed upon basis by which to draw such lines, we're
both talking about what we feel is right. Let's agree to revisit after having
the ELPA discussion.

John

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> David Engster <deng@randomsample.de> writes:
>
>> the main question is whether something provides infrastructure for other
>> packages to use.
>
> Sounds like a good sentence for an ELPA policy. :)
>
>> It is still not clear to me what exactly is gained by moving core packages
>> to ELPA.

Since no other core code depends on ELPA, CEDET is _not_ a "core package".

Rather, CEDET is a "tarball package"; it is in Emacs git solely to
ensure that it is included in the Emacs release tarball.

A better example of a possible "core ELPA package" is the "seq" package.

> Agility. 

I hear that as "easier to make small/frequent changes". That is what the
ELPA release policy gives you, yes. So this is one rationale for moving
packages to ELPA.

> What is appropriate. 

That's what we are trying to figure out :)

> Knowing when a thing goes into core, and when in ELPA.

Ditto.

> Org is an application, it's not infrastructure; the same with Gnus. *Parts* of
> Gnus might rightly be considered infrastructure, but the whole of Gnus just
> doesn't belong there. Parts of CEDET probably do belong in Emacs core, but as
> an application, I don't think the whole of it does.

I think the distinction between "tarball package" and "core package" is
helpful here.

I'm guessing that the main motivation for including Org and Gnus in
Emacs git (well, CVS back then?) was to include them in the release
tarball. If we have a mechanism to allow that for ELPA packages, moving
them to ELPA makes sense.

-- 
-- Stephe

Re: ELPA policy

[John Wiegley]

>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:

> I think the distinction between "tarball package" and "core package" is
> helpful here.
> 
> I'm guessing that the main motivation for including Org and Gnus in Emacs
> git (well, CVS back then?) was to include them in the release tarball. If we
> have a mechanism to allow that for ELPA packages, moving them to ELPA makes
> sense.

I like this. I think we have a good striation:

  core
  tarball ELPA
  net ELPA

To the user, core and tarball ELPA should be indistinguishable. I think also
that some tarball ELPA packages should come "pre-installed", if that is not
already done. This would makes them accessible via autoload, rather than
requiring the package interface to opt-in.

If core == tarball ELPA for everyone but us, this makes the decision easy
whether something should be in core or not a lot: We put it in core if core
requires it. If nothing at all in core uses the package -- and if that package
doesn't define "essential" functionality, like isearch.el or grep.el -- it can
shift to tarball ELPA.

This still leaves open the meaning of "essential", but it does make the
picture clearer.

John

RE: ELPA policy

[Drew Adams]

> > I think the distinction between "tarball package" and "core package" is
> > helpful here.
> >
> > I'm guessing that the main motivation for including Org and Gnus in
> > Emacs git (well, CVS back then?) was to include them in the release tarball.
> > If we have a mechanism to allow that for ELPA packages, moving them to ELPA
> > makes sense.
> 
> I like this. I think we have a good striation:
>   core
>   tarball ELPA
>   net ELPA
> To the user, core and tarball ELPA should be indistinguishable.

I haven't yet received an answer to my question whether anything
will change for users, depending on where you happen to manage
the code wrt ELPA etc.

But it sounds like the answer is yes.  If some stuff that has
traditionally been part of the distribution gets moved to (net)
ELPA, it will no longer be distributed to users.  They will
need to pull it down using the package interface.  Is that right?

If so, I'm not crazy about that.  I don't particularly want to
go fishing in (net) ELPA for stuff that I've always been able to
simply grep from within the distribution `lisp' directory. 
Especially, but not only, when I am not on the Internet.

I hope you will continue to (also) distribute Emacs with all
of its (traditional) source code, and not just make users
request it from (net) ELPA.

Re: ELPA policy

[John Wiegley]

>>>>> Drew Adams <drew.adams@oracle.com> writes:

> I haven't yet received an answer to my question whether anything will change
> for users, depending on where you happen to manage the code wrt ELPA etc.
> 
> But it sounds like the answer is yes. If some stuff that has traditionally
> been part of the distribution gets moved to (net) ELPA, it will no longer be
> distributed to users. They will need to pull it down using the package
> interface. Is that right?

Correct. Sorry I assumed that the discussion had answered your question in
passing.

> If so, I'm not crazy about that. I don't particularly want to go fishing in
> (net) ELPA for stuff that I've always been able to simply grep from within
> the distribution `lisp' directory. Especially, but not only, when I am not
> on the Internet.

Those things could then be in tarball ELPA. The distinction between tarball
ELPA and net ELPA is much more plastic than core and tarball ELPA. In fact, I
have a feeling that, initially, tarball ELPA and net ELPA will just be the
same thing, until we have a reason to tease them apart.

> I hope you will continue to (also) distribute Emacs with all of its
> (traditional) source code, and not just make users request it from (net)
> ELPA.

This is my preference as well.

John

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

> The distinction between tarball
> ELPA and net ELPA is much more plastic than core and tarball ELPA. In fact, I
> have a feeling that, initially, tarball ELPA and net ELPA will just be the
> same thing, until we have a reason to tease them apart.

Including a package in tarball ELPA means that the Emacs project has
decided that the package is sufficiently important to be made
immediately available to all Emacs users with no effort on their part
beyond installing Emacs from the tarball.

That implies a certain level of maturity and quality, and implies that
it should be favored over other similar packages.

On the other hand, we have in several cases recommended that someone put
a package in Gnu ELPA in order to give it more exposure, so that it
might eventually become higher quality and more mature.

In general, ELPA packages have far less oversight than core packages.
tarball ELPA packages should have the same oversight as core, or at
least more than normal ELPA packages.

So I disagree that all of Gnu ELPA should be included in the tarball.

Another choice would be to say "use MELPA for experimental packages",
but it is already too late for that.

-- 
-- Stephe

Re: ELPA policy

[Filipp Gunbin]

I do not consider myself competent enough in this area, but I'd like
to share some thoughts here:

For each package version there is a range (possibly sparse) of core
versions on which the package is supported (or just should work).

While the intent to move as much as possible in ELPA can be understood,
it leads to potentially more incompatibilities between important parts
which can provide API by themselves.

So, there should be some compromise between "latency" of new features
before they generally become available for use in packages and overall
core stability.  To me, it seems that the latter is more important and
it's better to keep infrastructure & library things in core, while
moving everything that uses them for a concrete purpose to ELPA.

If that in turn provides some API which others use, then we have
package interdependencies and that is probably OK (but can lead to
conflicts).  If sufficient amount of important packages use some API
and that API is rather mature, then the maintainer can decide to move
that in core to simplify dependency management.

So, I'd suggest that:

- Language features go straight into core (and people working on them
  / using them will have to use git version of Emacs until next
  release)

- New libraries which are not supposed to be in very common use go to
  ELPA first

- Then they are probably promoted to core as they get mature and more
  widely used - just to simplify their usage ("they will always be
  available").

- Major applications (like Gnus) and smaller ones always go to ELPA
  (most probably we should also bundle them in a tarball, but keep
  outside of the core).  A user can then decide whether to use git
  version of e.g. Gnus (from Elpa or private package repository) if she
  likes, or update to a released package version from Elpa (if core
  supports it), or just keep using the Elpa version she uses at the
  moment (which probably came together with current core).

- If such an application provides things useful for other
  applications, then it probably should extract that into a library to
  go through the cycle oulined above (elpa --maturity--> core).

The API / SPI notion can also be used to provide implementations
(backends) for different features which may have default simple
implementations in core and more advanced ones in packages.  A package
must somehow "register" itself as a candidate for being a service for
a concrete feature during installation.

Filipp

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> For each package version there is a range (possibly sparse) of core
> versions on which the package is supported (or just should work).

For normal ELPA packages, that's true.

For core and tarball ELPA packages, they must work in each current Emacs
release (since they are in the release tarball). They may also support
some set of previous Emacs releases.

> While the intent to move as much as possible in ELPA can be understood,
> it leads to potentially more incompatibilities between important parts
> which can provide API by themselves.

Yes.

> So, there should be some compromise between "latency" of new features
> before they generally become available for use in packages and overall
> core stability.  To me, it seems that the latter is more important and
> it's better to keep infrastructure & library things in core, while
> moving everything that uses them for a concrete purpose to ELPA.

For normal ELPA packages, that's true. But this is partly why core and
tarball ELPA packages are being considered.

> If that in turn provides some API which others use, then we have
> package interdependencies and that is probably OK (but can lead to
> conflicts). 

A more flexible requirements spec may  be needed. For example, Debian
packages allow specifying a range of versions in the dependency; that
can reflect actual testing, and not rely on implicit promises of future
backward compatibility. Of course, it can also lead to false failures.

> If sufficient amount of important packages use some API and that API
> is rather mature, then the maintainer can decide to move that in core
> to simplify dependency management.

or to a tarball ELPA package, or to a core ELPA package.

Perhaps that is too much choice?

> So, I'd suggest that:
>
> - Language features go straight into core (and people working on them
>   / using them will have to use git version of Emacs until next
>   release)

By "language features" do you mean things like ada-mode.el (primary mode
for editing Ada source files)? If so, I strongly disagree; ada-mode.el
is very happy as a normal ELPA package. Similar modes for other
languages with a small audience could be in ELPA as well; I have not
looked to see what's actually there.

cc-mode is the core for a lot of similar C-like languages, so it
probably makes sense to keep that in core. But it would be interesting
to see what the consequences of moving it to a normal or tarball ELPA
package would be.

But if you mean things like parsers and xref support, then I agree.
Although ada-mode also provides its own parser ...

> - New libraries which are not supposed to be in very common use go to
>   ELPA first

> - Then they are probably promoted to core as they get mature and more
>   widely used - just to simplify their usage ("they will always be
>   available").

That's what tarball ELPA packages are for.

The only reason to move a package to actual core (ie emacs git, as
opposed to elpa git) is if some other core code wants to depend in it.

> - Major applications (like Gnus) and smaller ones always go to ELPA
>   (most probably we should also bundle them in a tarball, but keep
>   outside of the core).

Right; that's a tarball ELPA package.

> - If such an application provides things useful for other
>   applications, then it probably should extract that into a library to
>   go through the cycle oulined above (elpa --maturity--> core).

Yes.

> The API / SPI notion can also be used to provide implementations
> (backends) for different features which may have default simple
> implementations in core and more advanced ones in packages.  A package
> must somehow "register" itself as a candidate for being a service for
> a concrete feature during installation.

If we use cl-generic to provide dispatching, the cl-defmethod form does
the registration.

-- 
-- Stephe

Re: ELPA policy

[Filipp Gunbin]

Stephen,

On 11/11/2015 15:22 -0600, Stephen Leake wrote:

>> If sufficient amount of important packages use some API and that API
>> is rather mature, then the maintainer can decide to move that in core
>> to simplify dependency management.
>
> or to a tarball ELPA package, or to a core ELPA package.
>
> Perhaps that is too much choice?

Mm yes, that's one of my fears, that it will become too complex for a
package maintaner.  Why not just treat each ELPA package just as an ELPA
package and leave the option of bundling it to core maintainers which
are better in this area (they'll do it in agreement with package
maintainer, of course)?

A "tarball" ELPA package is a one thing (that's what I call "bundle into
tarball", if I understood right), but "core" ELPA package is another -
here I have another fear, that normal and tarball ELPA packages
depending on such "core" packages, will have to accurately define
versions of the core package they support, and then we have to check all
this during the installation.  We'll probably have to calculate and
offer to user which set of the multiple core packages' multiple versions
suits for multiple normal and tarball (perhaps overriden by version from
Internet package archive) packages, at the same time probably giving the
user to opportunity to specify her preferred version of each requested
package.  Is it worth the trouble?  Or do I understand something wrong?

That's why I wrote about the feature latency - maybe it would be enough
if a person willing to use latest core features in her package will have
to develop it on git emacs and wait for the next official release to
make her package available to the public.  This will be the same as with
new C level features, which we cannot "push quicky" as we can with ELPA
package.

>> So, I'd suggest that:
>>
>> - Language features go straight into core (and people working on them
>>   / using them will have to use git version of Emacs until next
>>   release)
>
> By "language features" do you mean things like ada-mode.el (primary mode
> for editing Ada source files)? If so, I strongly disagree; ada-mode.el
> is very happy as a normal ELPA package. Similar modes for other
> languages with a small audience could be in ELPA as well; I have not
> looked to see what's actually there.

No-no, what I meant were Emacs Lisp libraries extending language, like
seq.el.

>> The API / SPI notion can also be used to provide implementations
>> (backends) for different features which may have default simple
>> implementations in core and more advanced ones in packages.  A package
>> must somehow "register" itself as a candidate for being a service for
>> a concrete feature during installation.
>
> If we use cl-generic to provide dispatching, the cl-defmethod form does
> the registration.

Thanks, that's one more reason for me to become acquainted with Emacs CL
features.

Filipp

Re: ELPA policy

[John Wiegley]

>>>>> Filipp Gunbin <fgunbin@fastmail.fm> writes:

> A "tarball" ELPA package is a one thing (that's what I call "bundle into
> tarball", if I understood right), but "core" ELPA package is another

I must have missed this distinction. What is the difference between "tarball
ELPA" and "core ELPA"?  I was thinking of "core" as anything that is in
Emacs.git.

John

Re: ELPA policy

[Filipp Gunbin]

On 12/11/2015 09:11 -0800, John Wiegley wrote:

>>>>>> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>
>> A "tarball" ELPA package is a one thing (that's what I call "bundle into
>> tarball", if I understood right), but "core" ELPA package is another
>
> I must have missed this distinction. What is the difference between "tarball
> ELPA" and "core ELPA"?  I was thinking of "core" as anything that is in
> Emacs.git.

If I got it right, a "tarball" package is copied from elpa.git into the
tarball at release time.  This is to bundle package so that Internet is
not required to install it (whether or not it is installed by default).

"core" package is developed in emacs.git (because core code depends on
it and we have to have full environment in the repo), but then can be
updated from elpa if new version is available.  This is to allow
frequent updates of such packages.

If I didn't get it right, then probably other package maintainers would
feel a bit lost, too, unless we provide concise explanation of this
separation.

Filipp

Re: ELPA policy

[John Wiegley]

>>>>> Filipp Gunbin <fgunbin@fastmail.fm> writes:

> "core" package is developed in emacs.git (because core code depends on it
> and we have to have full environment in the repo), but then can be updated
> from elpa if new version is available. This is to allow frequent updates of
> such packages.

Ah, that makes perfect sense, thanks.

John

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> On 12/11/2015 09:11 -0800, John Wiegley wrote:
>
>>>>>>> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>>
>>> A "tarball" ELPA package is a one thing (that's what I call "bundle into
>>> tarball", if I understood right), but "core" ELPA package is another
>>
>> I must have missed this distinction. What is the difference between "tarball
>> ELPA" and "core ELPA"?  I was thinking of "core" as anything that is in
>> Emacs.git.
>
> If I got it right, a "tarball" package is copied from elpa.git into the
> tarball at release time.  This is to bundle package so that Internet is
> not required to install it (whether or not it is installed by default).
>
> "core" package is developed in emacs.git (because core code depends on
> it and we have to have full environment in the repo), but then can be
> updated from elpa if new version is available.  This is to allow
> frequent updates of such packages.

Yes.

I'm working on writing this up for admin/notes/elpa.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> Stephen,
>
> On 11/11/2015 15:22 -0600, Stephen Leake wrote:
>
>>> If sufficient amount of important packages use some API and that API
>>> is rather mature, then the maintainer can decide to move that in core
>>> to simplify dependency management.
>>
>> or to a tarball ELPA package, or to a core ELPA package.
>>
>> Perhaps that is too much choice?
>
> Mm yes, that's one of my fears, that it will become too complex for a
> package maintaner.  Why not just treat each ELPA package just as an ELPA
> package and leave the option of bundling it to core maintainers which
> are better in this area (they'll do it in agreement with package
> maintainer, of course)?
>
> A "tarball" ELPA package is a one thing (that's what I call "bundle into
> tarball", if I understood right), 

Yes. We have not implemented this yet, but I'm imagining there is a list
of these in the Gnu Emacs Makefile somewhere (probably in a separate
file read by the Makefile). I don't think there will need to be any
metadata in the package files for this.

Declaring an ELPA package to be a tarball package does impose some
restrictions on the package maintainer; they have to synchronize with
each Emacs release, and accept the same review/oversight as core code.

> but "core" ELPA package is another -
> here I have another fear, that normal and tarball ELPA packages
> depending on such "core" packages, will have to accurately define
> versions of the core package they support, and then we have to check all
> this during the installation.  

I don't see why that is different from depending on a normal ELPA
package. 

> We'll probably have to calculate and offer to user which set of the
> multiple core packages' multiple versions suits for multiple normal
> and tarball (perhaps overriden by version from Internet package
> archive) packages, at the same time probably giving the user to
> opportunity to specify her preferred version of each requested
> package. Is it worth the trouble? Or do I understand something wrong?

Emacs does not support multiple versions of packages available
simultaneously; there is only one instance of a package that is first in
load-path.

You can end up with one copy in the installed Emacs distribution, and
one in ~/.emacs.d/elpa. But the elpa one will take precedence; that is
the only one that other packages have to worry about. It either meets
their dependency requirement, or not.

> That's why I wrote about the feature latency - maybe it would be enough
> if a person willing to use latest core features in her package will have
> to develop it on git emacs and wait for the next official release to
> make her package available to the public.  This will be the same as with
> new C level features, which we cannot "push quicky" as we can with ELPA
> package.

That's the main reason to make a package available in both core and ELPA:
users of the released version of Emacs can use the latest version of the
core ELPA package from ELPA, overriding the copy in their installation.

>>> So, I'd suggest that:
>>>
>>> - Language features go straight into core (and people working on them
>>>   / using them will have to use git version of Emacs until next
>>>   release)
>
> No-no, what I meant were Emacs Lisp libraries extending language, like
> seq.el.

That's a good candidate for a core ELPA package.

-- 
-- Stephe

Re: ELPA policy

[Filipp Gunbin]

On 12/11/2015 13:52 -0600, Stephen Leake wrote:

>> A "tarball" ELPA package is a one thing (that's what I call "bundle into
>> tarball", if I understood right), 
>
> Yes. We have not implemented this yet, but I'm imagining there is a list
> of these in the Gnu Emacs Makefile somewhere (probably in a separate
> file read by the Makefile). I don't think there will need to be any
> metadata in the package files for this.
>
> Declaring an ELPA package to be a tarball package does impose some
> restrictions on the package maintainer; they have to synchronize with
> each Emacs release, and accept the same review/oversight as core code.

They'll just have to make sure that a single version (which is going to
be bundled in a tarball) works good with an Emacs release being
prepared.

>> but "core" ELPA package is another -
>> here I have another fear, that normal and tarball ELPA packages
>> depending on such "core" packages, will have to accurately define
>> versions of the core package they support, and then we have to check all
>> this during the installation.  
>
> I don't see why that is different from depending on a normal ELPA
> package.

I think it's their dependants what make them different from tarball and
normal packages.

Emacs core provides API, which others use.  A normal package declares
which versions of this API it is supposed to work against.

With core packages, Emacs still provides API, but it now consists of a
static part (C level & core) and dynamic part (core packages, which can
later be updated from ELPA - correct me if I'm wrong).  So, a package
should independently define which core version it works agains and which
core package(s) version(s) it works against.

Here's where I see the complexity with multiple packages installed on
user request with package manager, which I tried to describe below.

>> We'll probably have to calculate and offer to user which set of the
>> multiple core packages' multiple versions suits for multiple normal
>> and tarball (perhaps overriden by version from Internet package
>> archive) packages, at the same time probably giving the user to
>> opportunity to specify her preferred version of each requested
>> package. Is it worth the trouble? Or do I understand something wrong?
>
> Emacs does not support multiple versions of packages available
> simultaneously; there is only one instance of a package that is first in
> load-path.
>
> You can end up with one copy in the installed Emacs distribution, and
> one in ~/.emacs.d/elpa. But the elpa one will take precedence; that is
> the only one that other packages have to worry about. It either meets
> their dependency requirement, or not.

Of course, I was talking about the set of available versions which could
be installed and from which a user or a package manager should choose.

>> That's why I wrote about the feature latency - maybe it would be enough
>> if a person willing to use latest core features in her package will have
>> to develop it on git emacs and wait for the next official release to
>> make her package available to the public.  This will be the same as with
>> new C level features, which we cannot "push quicky" as we can with ELPA
>> package.
>
> That's the main reason to make a package available in both core and ELPA:
> users of the released version of Emacs can use the latest version of the
> core ELPA package from ELPA, overriding the copy in their installation.

Yes, but it can introduce complexities I wrote of above.

Filipp

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> On 12/11/2015 13:52 -0600, Stephen Leake wrote:
>
>> I don't see why that is different from depending on a normal ELPA
>> package.
>
> I think it's their dependants what make them different from tarball and
> normal packages.
>
> Emacs core provides API, which others use.  A normal package declares
> which versions of this API it is supposed to work against.
>
> With core packages, Emacs still provides API, but it now consists of a
> static part (C level & core) and dynamic part (core packages, which can
> later be updated from ELPA - correct me if I'm wrong).  So, a package
> should independently define which core version it works agains and which
> core package(s) version(s) it works against.

package.el dependencies can only specify a minimum version, not a
maximum. there is no way that a normal ELPA package can declare that it
works with seq.el 1.0 but not seq.el 1.1

So if emacs 25 contains a core ELPA package seq.el 1.0, the declaration

;; Package-Requires: ((emacs "25.1"))

is equivalent to:

;; Package-Requires: ((emacs "25.1") (seq "1.0))

If seq.el 1.1 is later released via the Gnu ELPA server, it will be used
in either case.

Listing a core ELPA package explicitly is only necessary when the emacs
version is less than the first version that included the core ELPA
package version.

On the other hand, it doesn't hurt, so it's probably best to recommend
listing all ELPA packages explicitly.

>>> We'll probably have to calculate and offer to user which set of the
>>> multiple core packages' multiple versions suits for multiple normal
>>> and tarball (perhaps overriden by version from Internet package
>>> archive) packages, at the same time probably giving the user to
>>> opportunity to specify her preferred version of each requested
>>> package. Is it worth the trouble? Or do I understand something wrong?
>>
>> Emacs does not support multiple versions of packages available
>> simultaneously; there is only one instance of a package that is first in
>> load-path.
>>
>> You can end up with one copy in the installed Emacs distribution, and
>> one in ~/.emacs.d/elpa. But the elpa one will take precedence; that is
>> the only one that other packages have to worry about. It either meets
>> their dependency requirement, or not.
>
> Of course, I was talking about the set of available versions which could
> be installed and from which a user or a package manager should choose.

I still don't see a problem.

We have an example of a core ELPA package now; ada-mode. version 4.3 is
in emacs git; version 5.1.8 is in ELPA (if I ever get 5.x to be fast
enough on huge files, I'll delete 4.3 from core).

package.el has no issues with this.

Similar things can occur when there are different versions of the same
package in multiple repositories. In that sense, emacs git is just
another package repository.

Can you be more explicit about what problem you foresee? Give an example
of a package that would cause a problem.

-- 
-- Stephe

Re: ELPA policy

[Filipp Gunbin]

Stephen,

On 14/11/2015 04:30 -0600, Stephen Leake wrote:

> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>
>> On 12/11/2015 13:52 -0600, Stephen Leake wrote:
>>
>>> I don't see why that is different from depending on a normal ELPA
>>> package.
>>
>> I think it's their dependants what make them different from tarball and
>> normal packages.
>>
>> Emacs core provides API, which others use.  A normal package declares
>> which versions of this API it is supposed to work against.
>>
>> With core packages, Emacs still provides API, but it now consists of a
>> static part (C level & core) and dynamic part (core packages, which can
>> later be updated from ELPA - correct me if I'm wrong).  So, a package
>> should independently define which core version it works agains and which
>> core package(s) version(s) it works against.
>
> package.el dependencies can only specify a minimum version, not a
> maximum. there is no way that a normal ELPA package can declare that it
> works with seq.el 1.0 but not seq.el 1.1

My words were more of a theoretical speculation rather than discussion
of the current state of package.el.

Some parts of API may be deprecated and removed, so it may be the case
that a current package is not updated instantly and need to use some
previous version of core package.  While having a "single snapshot" of
Emacs + core packages does not cause such issues (even if package does
not specify maximum version which author of course does not know in
advance, the package just doesn’t work and so the user can downgrade to
previous "snapshot", that is, previous Emacs version), separate core
packages update after installation of such a "snapshot" might move
forward and thus break some normal package (and downgrade will not help
- the user will not know what to downgrade - what package? or core
emacs?  And how can I downgrade a single (core) package?).  Just theory,
again.  Ignore if it is irrelevant and I’m complicating things too much.

> So if emacs 25 contains a core ELPA package seq.el 1.0, the declaration
>
> ;; Package-Requires: ((emacs "25.1"))
>
> is equivalent to:
>
> ;; Package-Requires: ((emacs "25.1") (seq "1.0))
>
> If seq.el 1.1 is later released via the Gnu ELPA server, it will be used
> in either case.

Will the user have the option to continue to use the tarball version
instead of newer ELPA version?  Or choosing which ELPA version to use?
This may be needed when she uses previous major Emacs release, and
current ELPA package version requires newer core APIs.

>>>> We'll probably have to calculate and offer to user which set of the
>>>> multiple core packages' multiple versions suits for multiple normal
>>>> and tarball (perhaps overriden by version from Internet package
>>>> archive) packages, at the same time probably giving the user to
>>>> opportunity to specify her preferred version of each requested
>>>> package. Is it worth the trouble? Or do I understand something wrong?
>>>
>>> Emacs does not support multiple versions of packages available
>>> simultaneously; there is only one instance of a package that is first in
>>> load-path.
>>>
>>> You can end up with one copy in the installed Emacs distribution, and
>>> one in ~/.emacs.d/elpa. But the elpa one will take precedence; that is
>>> the only one that other packages have to worry about. It either meets
>>> their dependency requirement, or not.
>>
>> Of course, I was talking about the set of available versions which could
>> be installed and from which a user or a package manager should choose.
>
> I still don't see a problem.
>
> We have an example of a core ELPA package now; ada-mode. version 4.3 is
> in emacs git; version 5.1.8 is in ELPA (if I ever get 5.x to be fast
> enough on huge files, I'll delete 4.3 from core).
>
> package.el has no issues with this.
>
> Similar things can occur when there are different versions of the same
> package in multiple repositories. In that sense, emacs git is just
> another package repository.
>
> Can you be more explicit about what problem you foresee? Give an example
> of a package that would cause a problem.

For now, I don’t know of any practical examples.  But thanks for your
time on this.

Filipp

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

>> So if emacs 25 contains a core ELPA package seq.el 1.0, the declaration
>>
>> ;; Package-Requires: ((emacs "25.1"))
>>
>> is equivalent to:
>>
>> ;; Package-Requires: ((emacs "25.1") (seq "1.0))
>>
>> If seq.el 1.1 is later released via the Gnu ELPA server, it will be used
>> in either case.
>
> Will the user have the option to continue to use the tarball version
> instead of newer ELPA version?  

Yes, they may install it or not; my statement above assumed the ELPA
package was installed and updated.

> Or choosing which ELPA version to use?

Technically, there is only one ELPA version; the one shown by
`list-packages'. However, after user installs one version, they may opt
to not install an update.

If they delete the installed version, they can only install the latest
version.

That's all just using the commands in package.el; users can of course
save code and copy it into and out of `load-path' manually.

> This may be needed when she uses previous major Emacs release, and
> current ELPA package version requires newer core APIs.

If there is a compatible ELPA package version that is more recent than
the version bundled with that Emacs version, but older than the latest
ELPA version, yes.

But that's not supported by the current ELPA server and package.el.

So at the moment, users of older Emacsen are stuck with either the
bundled package, or the latest package.

The maintainer of the ELPA package can provide compatibility code so it
is useful with older Emacsen. But that gets harder the farther back you
go; my current limit is 24.3.

-- 
-- Stephe

Re: ELPA policy

[Tom Tromey]

Stephen> package.el dependencies can only specify a minimum version, not a
Stephen> maximum. there is no way that a normal ELPA package can declare that it
Stephen> works with seq.el 1.0 but not seq.el 1.1

If I had to do it all over, I'd require semver for Emacs packages.
Maybe this could be done optionally somehow.

Tom

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Another choice would be to say "use MELPA for experimental packages",
  > but it is already too late for that.

If my memory serves, MELPA contains packages we would like to include
in Emacs, but we can't get legal papers for them.
We are seeking people to write replacements for those, though we wish
it were not necessary.

We would like developers of future useful packages to want to put
their packages into the GNU ELPA.

To recommend putting some packages -- whatever they might be -- into
MELPA would work against these goals.  It would have been a bad
approach.

  > Which raises the question; what is the rationale for moving from MELPA
  > to Gnu ELPA?

If a package is useful to recommend, and we CAN get papers for it,
we definitely want to move it to GNU ELPA.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> If a package is useful to recommend, and we CAN get papers for it, we
> definitely want to move it to GNU ELPA.

Yes, this is why I want to clearly define ELPA policy, and streamline things
as much as possible for developers and users: so that it becomes a more
attractive means for distributing Emacs packages. I think many people may be
largely ignoring it right now, and so they reach for MELPA. Since so many
people contribute to MELPA, they consider it a more attractive distribution
platform.

John

Re: ELPA policy

[Phillip Lord]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Richard Stallman <rms@gnu.org> writes:
>
>> If a package is useful to recommend, and we CAN get papers for it, we
>> definitely want to move it to GNU ELPA.
>
> Yes, this is why I want to clearly define ELPA policy, and streamline things
> as much as possible for developers and users: so that it becomes a more
> attractive means for distributing Emacs packages. I think many people may be
> largely ignoring it right now, and so they reach for MELPA. Since so many
> people contribute to MELPA, they consider it a more attractive distribution
> platform.


MELPA is also *easier* to contribute to. Aside from copyright issues, it
involves writing something that looks like lisp, testing on your local
fork, then a PR.

With GNU ELPA, it involves some fairly obscure git cleverness. GNU ELPA
could be made easier by allowing recipes, and by accepting PRs. This
avoids the necessity to have commit access to GNU ELPA to contribute.

Finally, I think that core devs should contribute to individual packages
by PR to their repos.

Phil

Re: ELPA policy

[John Wiegley]

>>>>> Phillip Lord <phillip.lord@russet.org.uk> writes:

> With GNU ELPA, it involves some fairly obscure git cleverness. GNU ELPA
> could be made easier by allowing recipes, and by accepting PRs. This avoids
> the necessity to have commit access to GNU ELPA to contribute.

We can always use the power of Git here: Keep a fork of ELPA.git on GitHub,
accept PRs there, and periodically merge back in.

John

Re: ELPA policy

[Stephen Leake]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> John Wiegley <jwiegley@gmail.com> writes:
>
>>>>>>> Richard Stallman <rms@gnu.org> writes:
>>
>>> If a package is useful to recommend, and we CAN get papers for it, we
>>> definitely want to move it to GNU ELPA.
>>
>> Yes, this is why I want to clearly define ELPA policy, and streamline things
>> as much as possible for developers and users: so that it becomes a more
>> attractive means for distributing Emacs packages. I think many people may be
>> largely ignoring it right now, and so they reach for MELPA. Since so many
>> people contribute to MELPA, they consider it a more attractive distribution
>> platform.
>
>
> MELPA is also *easier* to contribute to. Aside from copyright issues, it
> involves writing something that looks like lisp, testing on your local
> fork, then a PR.
>
> With GNU ELPA, it involves some fairly obscure git cleverness. 

It's only normal git cleverness; there's nothing special about the Gnu
ELPA git. Unless you are using an "external" package, perhaps.

I can understand treating all of git as "obscure", if that's what you
meant. I much prefer monotone.

> GNU ELPA could be made easier by allowing recipes, and by accepting
> PRs. This avoids the necessity to have commit access to GNU ELPA to
> contribute.

"PR" is "Pull Request"? Doesn't that mean "pull from my git repository"?
Or is it more general than that?

> Finally, I think that core devs should contribute to individual packages
> by PR to their repos.

I'm guessing "their repos" is _not_ Gnu ELPA git? So this only applies
to packages that have a primary repo that is not Gnu ELPA git.

I agree that any one that edits a Gnu ELPA package in Gnu ELPA git
should also notify the upstream project if there is one. But they do not
have to wait for approval if it's a critical bug.

Normal ettiquette should apply of course.

This is more important for tarball and core ELPA packages, since they
are part of the blessed Emacs standard library; Emacs developers can
edit those as if they were in core.

-- 
-- Stephe

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > MELPA is also *easier* to contribute to. Aside from copyright issues, it
  > involves writing something that looks like lisp, testing on your local
  > fork, then a PR.

What is a PR?

  > With GNU ELPA, it involves some fairly obscure git cleverness. GNU ELPA
  > could be made easier by allowing recipes, and by accepting PRs. This
  > avoids the necessity to have commit access to GNU ELPA to contribute.

Maybe we should do this (but I don't know what a PR is).

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[JJ Asghar]

On 11/13/15 3:58 PM, Richard Stallman wrote:
>   > MELPA is also *easier* to contribute to. Aside from copyright issues, it
>   > involves writing something that looks like lisp, testing on your local
>   > fork, then a PR.
> 
> What is a PR?

A great detailed explanation of a PR is here[1]. A short explanation is
a request to put in a patch to a repository.

[1]: https://help.github.com/articles/using-pull-requests/

-JJ

Re: ELPA policy

[Jorge A. Alfaro-Murillo]

Stephen Leake writes:
 
> Including a package in tarball ELPA means that the Emacs project 
> has decided that the package is sufficiently important to be 
> made immediately available to all Emacs users with no effort on 
> their part beyond installing Emacs from the tarball. 
> 
> That implies a certain level of maturity and quality, and 
> implies that it should be favored over other similar packages.

Would that just be packages that are currently in core? Because 
AUCTeX definitely fits that description. I have always been 
curious as why it is not in core currently.

Best,
-- 
Jorge.

Re: ELPA policy

[John Wiegley]

>>>>> Jorge A Alfaro-Murillo <jorge.alfaro-murillo@yale.edu> writes:

>> Including a package in tarball ELPA means that the Emacs project has
>> decided that the package is sufficiently important to be made immediately
>> available to all Emacs users with no effort on their part beyond installing
>> Emacs from the tarball.
>> 
>> That implies a certain level of maturity and quality, and implies that it
>> should be favored over other similar packages.

> Would that just be packages that are currently in core? Because AUCTeX
> definitely fits that description. I have always been curious as why it is
> not in core currently.

No, it wouldn't be just packages currently in core, we could consider others
for inclusion as well. AucTeX might be an excellent candidate.

I like that ELPA could give us better flexibility about what goes in the
tarball, without having to make decisions that directly affect core.

John

RE: ELPA policy

[Drew Adams]

> We would like developers of future useful packages to want to put
> their packages into the GNU ELPA.
> 
> To recommend putting some packages -- whatever they might be -- into
> MELPA would work against these goals.  It would have been a bad
> approach.

FWIW -

I will continue to use MELPA for my libraries, and not GNU ELPA,
for one reason:

I post my code to Emacs Wiki (to locked pages).  MELPA pulls
the code from the wiki (daily).  I do not need to use GIT etc.

This is the way I prefer to work.  Call it old-fashioned or
bizarre; I don't care.

So you might recommend that packages not be put in MELPA, but
some will continue to be put there, including perhaps some that
you might someday want to include in Emacs.

>   > Which raises the question; what is the rationale for moving from
>   > MELPA to Gnu ELPA?
> 
> If a package is useful to recommend, and we CAN get papers for it,
> we definitely want to move it to GNU ELPA.

This was refused by Stefan for my libraries (for which FSF has papers),
based on the fact that I do not use GIT, so any updates I make to them
would not be done directly in the repository.  It was not acceptable to
update elsewhere (e.g. the wiki) and then have someone or a program
pull from there to the repository when appropriate.  So be it.

Re: ELPA policy

[John Wiegley]

>>>>> Drew Adams <drew.adams@oracle.com> writes:

> I will continue to use MELPA for my libraries, and not GNU ELPA, for one
> reason:
>
> I post my code to Emacs Wiki (to locked pages). MELPA pulls the code from
> the wiki (daily). I do not need to use GIT etc.
>
> This is the way I prefer to work. Call it old-fashioned or bizarre; I don't
> care.

This is a legitimate complaint, Drew. One of the things I want to avoid with
ELPA is making it harder for people to contribute to than other alternatives.

Since pulling in data from the Web indiscriminately is not possible for this
project (we have to make sure we're not unwittingly including code without a
proper copyright assignment), perhaps we need as an "integrator": someone
willing to guide the update of ELPA from various sources on the Web, keeping
an eye out for changes that might affect copyright.

It wouldn't be too hard to setup a process that updates a local clone of ELPA
with all your latest versions from GitHub and EmacsWiki, and then to check
over a diff of those changes before pushing them to ELPA. One might even
suggest that you could do this yourself, as part of your own release work.
Would that be something you'd find worth doing?

John

RE: ELPA policy

[Drew Adams]

> This is a legitimate complaint, Drew. One of the things I want to avoid
> with ELPA is making it harder for people to contribute to than other
> alternatives.
> 
> Since pulling in data from the Web indiscriminately is not possible for
> this project (we have to make sure we're not unwittingly including code without
> a proper copyright assignment), perhaps we need as an "integrator": someone
> willing to guide the update of ELPA from various sources on the Web,
> keeping an eye out for changes that might affect copyright.

I'm glad to hear that; all of it.  And thanks for replying.

> It wouldn't be too hard to setup a process that updates a local clone of
> ELPA with all your latest versions from GitHub and EmacsWiki, and then to check
> over a diff of those changes before pushing them to ELPA. One might even
> suggest that you could do this yourself, as part of your own release work.
> Would that be something you'd find worth doing?

I don't want to develop anything like that; sorry.  I don't want to fiddle
with a local GIT clone.  I would be willing to identify versions of my
libraries to be pulled at specific times ("releases") from the wiki (or
from MELPA) to be put (updated) in GNU ELPA.

I doubt that my case is typical, however.  I'm not saying that Emacs Dev
needs to cater to such a case.  I'm just explaining that I update my
libraries locally, without GIT etc., and I upload updates to the wiki.
They are automatically pulled to MELPA.

What do I need to know, currently?  Only how to write some Lisp (a
library) and how to copy and paste it into a Web page.  Simple.  Maybe
too simple to accommodate by Emacs Dev.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Since pulling in data from the Web indiscriminately is not
  > possible for this project (we have to make sure we're not
  > unwittingly including code without a proper copyright assignment),
  > perhaps we need as an "integrator": someone willing to guide the
  > update of ELPA from various sources on the Web, keeping an eye out
  > for changes that might affect copyright.

What we need to solve Drew's problem is to merge changes into a few
specific files from a specific place, and commit them using git.
Isn't that easy enough to do?

There could be one page where he provides the commit log info.
Whenever that page changes (we could check every 5 minutes), our demon
could merge in all the files which have changed.  They it would email
the diffs to Drew so he could confirm what he installed.

Thus, when Drew wants to install a new version, he would change the
other pages first, then write the change log text into the log page.
Then wait 5 minutes and it's done.

This is assuming there are no other obstacles aside from the
mechanics of installing changes into our repository.



-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 787 bytes --]

On 11 Nov 2015 11:27 pm, "Richard Stallman" <rms@gnu.org> wrote:
>
> There could be one page where he provides the commit log info.
> Whenever that page changes (we could check every 5 minutes), our demon
> could merge in all the files which have changed.  They it would email
> the diffs to Drew so he could confirm what he installed.

Richard, do think such as system would be viable copyright-wise?
Of course, we would only do this when the author assigns copyright, but
there's a very real possibility that a careless maintainer might add
someone else's code to this page. And then this (non-copyright-assigned)
code would be automatically pulled and committed to Elpa.

I ask this because if we are OK with this risk then I'm perfectly willing
to write such a script (eager, even).

[-- Attachment #2: Type: text/html, Size: 949 bytes --]

Re: ELPA policy

[Dmitry Gutov]

On 11/12/2015 02:35 AM, Artur Malabarba wrote:

> Richard, do think such as system would be viable copyright-wise?
> Of course, we would only do this when the author assigns copyright, but
> there's a very real possibility that a careless maintainer might add
> someone else's code to this page. And then this (non-copyright-assigned)
> code would be automatically pulled and committed to Elpa.

We should also consider whether this is sound from the security 
standpoint. I mean, pushing the code to ELPA without any review from the 
core developers.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Richard, do think such as system would be viable copyright-wise?

In terms of copyright, there is no difference between the system I
proposed (installing what Drew puts in certain web pages) and the
method Drew doesn't like, which is for him to install the same changes
via Git.

Whichever method he uses, he technically could install code written by
others whom we don't have papers for.  If we permit him to use Git to
install changes, that entails trusting him to handle this
responsibility.  If we can trust him to do it with one method, we can
trust him to do it with the other method.

  > We should also consider whether this is sound from the security 
  > standpoint. I mean, pushing the code to ELPA without any review from the 
  > core developers.

The same would occur if we invite him to install changes in his packages
through Git.

If we can check enough one way, we can check enough the other way.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

On 12 Nov 2015 10:34 pm, "Richard Stallman" <rms@gnu.org> wrote:
>   > Richard, do think such as system would be viable copyright-wise?
>
> In terms of copyright, there is no difference between the system I
> proposed (installing what Drew puts in certain web pages) and the
> method Drew doesn't like, which is for him to install the same changes
> via Git.
>
> Whichever method he uses, he technically could install code written by
> others whom we don't have papers for.  If we permit him to use Git to
> install changes, that entails trusting him to handle this
> responsibility.  If we can trust him to do it with one method, we can
> trust him to do it with the other method.

I'm trying to determine whether this method can be employed as a general
elpa mechanism. Of course we trust Drew, but would it be acceptable to use
such a script with other people who submit packages to Elpa? (assuming the
author has assigned copyright)

[-- Attachment #2: Type: text/html, Size: 1135 bytes --]

Re: ELPA policy

[Stephen Leake]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Since pulling in data from the Web indiscriminately is not
>   > possible for this project (we have to make sure we're not
>   > unwittingly including code without a proper copyright assignment),
>   > perhaps we need as an "integrator": someone willing to guide the
>   > update of ELPA from various sources on the Web, keeping an eye out
>   > for changes that might affect copyright.
>
> What we need to solve Drew's problem is to merge changes into a few
> specific files from a specific place, and commit them using git.
> Isn't that easy enough to do?
>
> There could be one page where he provides the commit log info.
> Whenever that page changes (we could check every 5 minutes), our demon
> could merge in all the files which have changed.  They it would email
> the diffs to Drew so he could confirm what he installed.
>
> Thus, when Drew wants to install a new version, he would change the
> other pages first, then write the change log text into the log page.
> Then wait 5 minutes and it's done.
>
> This is assuming there are no other obstacles aside from the
> mechanics of installing changes into our repository.

Any malicious hacker can drop completely different code in that web
page, and thus get it into Gnu ELPA.

We will have replaced the security of private machines with whatever web
login that web page requires; that's a huge step backwards.

-- 
-- Stephe

RE: ELPA policy

[Drew Adams]

> Any malicious hacker can drop completely different code in that web
> page, and thus get it into Gnu ELPA.  We will have replaced the
> security of private machines with whatever web login that web page
> requires; that's a huge step backwards.

Again: "I post my code to Emacs Wiki (to locked pages)."

But of course that doesn't mean it is uncrackable.

Re: ELPA policy

[Alex Schröder]

Drew Adams <drew.adams@oracle.com> writes:

> Again: "I post my code to Emacs Wiki (to locked pages)."
>
> But of course that doesn't mean it is uncrackable.

Given that Emacs Wiki currently does not use HTTPS, we should consider
Emacs Wiki insecure. Even then, the admin passwords are shared and there
are no other efforts made to identify editors once they present the
admin password. The current setup only protects against the simplest of
threat models, e.g. spammers, vandals, well-meaning contributors that
want to edit wiki pages instead of informing maintainers, and so on. I
think that's the threat model wiki pages need to defend against, but I
suspect that's not good enough for an inclusion into Emacs Core.

-- 
Public Key Fingerprint = DF94 46EB 7B78 4638 7CCC  018B C78C A29B ACEC FEAE

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Any malicious hacker can drop completely different code in that web
  > page, and thus get it into Gnu ELPA.

Drew said the pages were locked.
Doesn't that mean that only he has access to change them?

  > We will have replaced the security of private machines with whatever web
  > login that web page requires; that's a huge step backwards.

I think you are concerned that someone might break the security on that other
server and then install changes on it using Drew's account.

In general, someone who breaks the security on a machine used by
an Emacs contributor might be able to insert changes in Emacs
by pretending to be that contributor.  I don't think this is
fundamentally different.  But maybe the web site's security is
not quite as good.

We can make the security tighter.  Drew, are you willing to GPG-sign
your new versions?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Stephen Leake]

Richard Stallman <rms@gnu.org> writes:

>   > We will have replaced the security of private machines with whatever web
>   > login that web page requires; that's a huge step backwards.
>
> I think you are concerned that someone might break the security on that other
> server and then install changes on it using Drew's account.
>
> In general, someone who breaks the security on a machine used by
> an Emacs contributor might be able to insert changes in Emacs
> by pretending to be that contributor.  I don't think this is
> fundamentally different.  But maybe the web site's security is
> not quite as good.

Yes, my concern is about the strength of the web security, as opposed to
the strength of Drew's private machine.

I suppose there's no a priori reason to assume one is more secure than
the other; maybe I'm just reacting to all the news reports about hacking.

-- 
-- Stephe

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I suppose there's no a priori reason to assume one is more secure than
  > the other; maybe I'm just reacting to all the news reports about hacking.

Would you please call that "cracking"?  Hacking is what we do as Emacs
developers.  See http://stallman.org/articles/on-hacking.html.

Anyway, now I understand your concern, and I think it's a valid one.
That's why I've proposed another solution.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

RE: ELPA policy

[Drew Adams]

>   > Any malicious hacker can drop completely different code in that web
>   > page, and thus get it into Gnu ELPA.
> 
> Drew said the pages were locked.
> Doesn't that mean that only he has access to change them?

No, anyone with admin privileges for the wiki has access to do so.
There are a few people in this category.  And see Alex Schroeder's
clarification of what this means.  This is not watertight security,
by any means.

Perhaps one way to look at it is similar to submitting something
by email (which would be another possibility, for me).

>   > We will have replaced the security of private machines with whatever
>   > web login that web page requires; that's a huge step backwards.
> 
> I think you are concerned that someone might break the security on that
> other server and then install changes on it using Drew's account.

See above.

> In general, someone who breaks the security on a machine used by
> an Emacs contributor might be able to insert changes in Emacs
> by pretending to be that contributor.  I don't think this is
> fundamentally different.  But maybe the web site's security is
> not quite as good.
> 
> We can make the security tighter.  Drew, are you willing to GPG-sign
> your new versions?

I don't really know what that entails.

Dunno whether you really want to discuss my case in particular in
detail here.  Again, I doubt that it is typical.  The reason for
my initial message about this was to suggest that some people do
use MELPA, and that perhaps some way to accommodate them could be
devised.  But maybe not.

To repeat the summary of my original point:

  So you might recommend that packages not be put in MELPA, but
  some will continue to be put there, including perhaps some that
  you might someday want to include in Emacs.

Regarding my own case, this was the point:

  I do not use GIT, so any updates I make to them
  would not be done directly in the repository.  It was not
  acceptable to update elsewhere (e.g. the wiki) and then have
  someone or a program pull from there to the repository when
  appropriate.

In sum, some people will post code to MELPA, including some
that you might someday want in Emacs.  And some input to MELPA
comes from the wiki, not from GIT - but this is probably a small
portion of what is in MELPA.

Re: ELPA policy

[Eli Zaretskii]

> Date: Thu, 12 Nov 2015 15:05:50 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: jwiegley@gmail.com, drew.adams@oracle.com, emacs-devel@gnu.org
> 
> Regarding my own case, this was the point:
> 
>   I do not use GIT, so any updates I make to them
>   would not be done directly in the repository.  It was not
>   acceptable to update elsewhere (e.g. the wiki) and then have
>   someone or a program pull from there to the repository when
>   appropriate.
> 
> In sum, some people will post code to MELPA, including some
> that you might someday want in Emacs.  And some input to MELPA
> comes from the wiki, not from GIT - but this is probably a small
> portion of what is in MELPA.

FWIW, I very much doubt that many people nowadays would share Drew's
issue with using Git.  Using Git is a no-brainer nowadays.  I actually
think this is a very unique situation, and anyone else who insists on
using MELPA doesn't do that for such simple technical reasons.  I
think the legal paperwork is the main issue, and also perhaps not
everyone would like to be accountable for their code practices before
the Emacs maintainers and their peculiar requirements and
expectations.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > We can make the security tighter.  Drew, are you willing to GPG-sign
  > > your new versions?

  > I don't really know what that entails.

It means you would make diffs to show the change to install,
plus a log entry, sign that whole thing with GnuPG, and mail
it to a certain address.  The signature would demonstrate it came
from you.

Is that ok for you?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

RE: ELPA policy

[Drew Adams]

>   > > We can make the security tighter.  Drew, are you willing
>   > > to GPG-sign your new versions?
> 
>   > I don't really know what that entails.
> 
> It means you would make diffs to show the change to install,
> plus a log entry, sign that whole thing with GnuPG, and mail
> it to a certain address.  The signature would demonstrate it
> came from you.
> 
> Is that ok for you?

I have no problem creating diffs & log entries, and emailing
them in.  I've mailed in plenty of patches that way.

Dunno what "sign that whole thing with GnuPG" entails.  What
is needed for that (e.g., on MS Windows)?

Looking at the doc at https://www.gnupg.org/documentation/howtos.html
and http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto-2.html,
it seems I would need to install GnuPG from source code,
using a C compiler etc.  I don't plan to install and use tools
such as a C compiler on my laptop, so I guess the answer would
be no; sorry.

If there were an easy-to-use Windows binary for such signing,
then I would probably do as you propose.

(I don't do software development anymore.  I use MS Windows
binaries of Emacs, and I write some Emacs-Lisp code.  I do
have (an old version of) Cygwin installed, so that I can use
a few simple commands from within Emacs.  That's about it.)

I don't mean to make a big deal about this.  I'm not saying
no definitively.  But AFAIK there is not necessarily even
any interest in adding any of the code I've written to Emacs.
I don't feel like going through a lot of preparation for
nothing.

My libraries are there, and have been for quite a while.
If someone is really interested in something, let me know
and maybe we can work something out.  Otherwise, we are
likely wasting each other's time here.  I don't mean to
waste your time.  (But I do appreciate your having
considered this.)

Again, my original point was about MELPA and updates to
it from the wiki.  And again, my use of MELPA is likely
not typical.  Perhaps I should have said nothing at all
about it.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Dunno what "sign that whole thing with GnuPG" entails.  What
  > is needed for that (e.g., on MS Windows)?

GnuPG works on Windows.  I expect we can find someone to help
you install it.  Then we need you to send us your public key
so we can verify your signatures.  We can find a way to do that.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Michael Heerdegen]

Richard Stallman <rms@gnu.org> writes:

> GnuPG works on Windows.  I expect we can find someone to help
> you install it.  Then we need you to send us your public key
> so we can verify your signatures.  We can find a way to do that.

I think that's not much easier for Drew than to use git.

I know Drew doesn't want to use git in his workflow (which he doesn't
have to), he does not dislike git for political reasons or so AFAIR.

Once he has set up things correctly (we can help him with that), he just
has to change the file in his repo clone, pull, add and push.  That's not
more complicated than what he does now when uploading to the wiki.  He
doesn't have to use git for anything else, or while developing.

Does that sound acceptable to you, Drew?  I can help you with any
problems you might encounter.  After everything has been set up
properly, uploading to Gnu Elpa is trivial, you don't have to learn git
for that.


Regards,

Michael.

RE: ELPA policy

[Drew Adams]

> Once he has set up things correctly (we can help him with that), he
> just has to change the file in his repo clone, pull, add and push.  
> That's not more complicated than what he does now when uploading to
> the wiki.  He doesn't have to use git for anything else, or while 
> developing.
> 
> Does that sound acceptable to you, Drew?  I can help you with any
> problems you might encounter.  After everything has been set up
> properly, uploading to Gnu Elpa is trivial, you don't have to
> learn git for that.

I will take a look.  Please follow up with me off-list.

(I've tried to make clear that my reply to this thread was
not about getting my libraries into Emacs, but was about
"MELPA and updates to it from the wiki."  And as I said,
"AFAIK there is not necessarily even any interest in adding
any of the code I've written to Emacs."  This has largely
been much ado about nothing, I think.  I'm sorry for the
resulting waste of people's time.  But I'm glad that the
question of MELPA and wiki code was at least considered
to some extent.)

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I don't think this is "much ado about nothing".
The reasons that affect one person may affect others.

We want people to arrange to contribute their Emacs add-ons
to Emacs, and that suggests that whenever someone says a reason
why he didn't want to put his code in GNU ELPA, we should try
to correct it.



-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> We want people to arrange to contribute their Emacs add-ons to Emacs, and
> that suggests that whenever someone says a reason why he didn't want to put
> his code in GNU ELPA, we should try to correct it.

Agreed. ELPA should become a more positive experience, both for developers,
contributors and users.

John

Re: ELPA policy

[Artur Malabarba]

2015-11-17 23:17 GMT+00:00 John Wiegley <jwiegley@gmail.com>:
>>>>>> Richard Stallman <rms@gnu.org> writes:
>
>> We want people to arrange to contribute their Emacs add-ons to Emacs, and
>> that suggests that whenever someone says a reason why he didn't want to put
>> his code in GNU ELPA, we should try to correct it.
>
> Agreed. ELPA should become a more positive experience, both for developers,
> contributors and users.

I'm deeply in favor of making it easier for people to contribute code
to Elpa, and (like I said), I'm willing to help write the code for
that.
But I think the hardest part of the problem is not being discussed here.

Whatever this method is, it needs to meet two criteria simultaneously:
1. Not commit any code to Elpa until someone with push access has OK'd
it. This is just to ensure nothing malicious is being done.
2. Be fairly automated (not completely), so that the Elpa maintainers
don't have to manually commit+push all code that gets sent to them.
This is to preserve the sanity of the maintainers. The current model
where everything is commited by us just isn't scalable.

While those two points are not mutually exclusive, they are quite
difficult to reconcile. It needs to show the maintainers a diff of the
changes and then only proceed with the commit after receiving some
very minimal interaction (a reply to an email, or the click of a
button).

Re: ELPA policy

[David Kastrup]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

> 2015-11-17 23:17 GMT+00:00 John Wiegley <jwiegley@gmail.com>:
>>>>>>> Richard Stallman <rms@gnu.org> writes:
>>
>>> We want people to arrange to contribute their Emacs add-ons to Emacs, and
>>> that suggests that whenever someone says a reason why he didn't want to put
>>> his code in GNU ELPA, we should try to correct it.
>>
>> Agreed. ELPA should become a more positive experience, both for developers,
>> contributors and users.
>
> I'm deeply in favor of making it easier for people to contribute code
> to Elpa, and (like I said), I'm willing to help write the code for
> that.
> But I think the hardest part of the problem is not being discussed here.
>
> Whatever this method is, it needs to meet two criteria simultaneously:
> 1. Not commit any code to Elpa until someone with push access has OK'd
> it. This is just to ensure nothing malicious is being done.
> 2. Be fairly automated (not completely), so that the Elpa maintainers
> don't have to manually commit+push all code that gets sent to them.
> This is to preserve the sanity of the maintainers. The current model
> where everything is commited by us just isn't scalable.
>
> While those two points are not mutually exclusive, they are quite
> difficult to reconcile. It needs to show the maintainers a diff of the
> changes and then only proceed with the commit after receiving some
> very minimal interaction (a reply to an email, or the click of a
> button).

With the GNU ELPA requiring copyright assignments, I doubt that we are
going to hit scalability issues of the kind you fear.  But of course
there is nothing wrong with trying to create a contributing process that
is minimal effort for the sake of such archives where this is not the
case.

-- 
David Kastrup

Re: ELPA policy

[Achim Gratz]

Drew Adams writes:
> Dunno what "sign that whole thing with GnuPG" entails.  What
> is needed for that (e.g., on MS Windows)?

http://www.gpg4win.org/

> (I don't do software development anymore.  I use MS Windows
> binaries of Emacs, and I write some Emacs-Lisp code.  I do
> have (an old version of) Cygwin installed, so that I can use
> a few simple commands from within Emacs.  That's about it.)

You'd really be off easier if you updated Cygwin so that you could use
Git and SSH and GPG from that.  Perhaps you'll find that using Emacs
from Cygwin is a lot easier now, too -- you can use emacs-w32 if you
don't care about running X11.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

RE: ELPA policy

[Drew Adams]

>   > Since pulling in data from the Web indiscriminately is not
>   > possible for this project (we have to make sure we're not
>   > unwittingly including code without a proper copyright assignment),
>   > perhaps we need as an "integrator": someone willing to guide the
>   > update of ELPA from various sources on the Web, keeping an eye out
>   > for changes that might affect copyright.
> 
> What we need to solve Drew's problem is to merge changes into a few
> specific files from a specific place, and commit them using git.
> Isn't that easy enough to do?
> 
> There could be one page where he provides the commit log info.
> Whenever that page changes (we could check every 5 minutes), our demon
> could merge in all the files which have changed.  They it would email
> the diffs to Drew so he could confirm what he installed.
> 
> Thus, when Drew wants to install a new version, he would change the
> other pages first, then write the change log text into the log page.
> Then wait 5 minutes and it's done.
> 
> This is assuming there are no other obstacles aside from the
> mechanics of installing changes into our repository.

I will again say that I don't expect that my situation is typical.
There may be no one else out there who ever prefers something like it.

But FWIW I think that what you describe would work for me.

Re: ELPA policy

[Stephen Leake]

David Engster <deng@randomsample.de> writes:

> John Wiegley writes:
>>>>>>> David Engster <deng@randomsample.de> writes:
>>
>>> This is not about reaching a consensus. This is about you giving proper
>>> reasons for such a big change.
>>
>> To be clear, I would also put Eshell (though not pcomplete) into the category
>> of "big things that should be in ELPA" -- albeit, the subset of ELPA that will
>> be in the release tarball.
>>
>> It's hard to pin down why in a manner that fits in an e-mail. If Eshell were
>> in ELPA today, and we were talking about moving it into core, I would have
>> just as much trouble justifying that move too. Perhaps this simply underscores
>> the fact that we don't have an agreed upon ELPA policy we can all refer to.
>
> In my opinion, the main question is whether something provides
> infrastructure for other packages to use. 

I think it is a reasonable goal to allow ELPA packages to serve in that
role, as long as the "other packages" are also normal or tarball ELPA
packages.

> This is precisely what CEDET tries to do.

Yes.

However, now the shoe is on the other foot: why must infrastructure
packages be in Emacs core?

I think the trigger is "some other _core_ code wants to depend on it".
That would trigger a move to Emacs git, as a core ELPA package (the
package could still have intermediate released via the Gnu ELPA server).

> I wouldn't have much trouble with putting parts of CEDET in ELPA,
> namely those parts that do not directly provide infrastructure, like
> support for certain languages, project types, indexing tools, etc.

Good, but let's not try to do that for Emacs 25; since we are trying to
get to feature freeze, it's too much.

> It is still not clear to me what exactly is gained by moving core
> packages to ELPA. 

One gain is making it clear that other core code is not allowed to depend
on it. This is in turn to ensure that it doesn't creep into the dumped
code. But I'm not sure that's an important reason.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> David Engster <deng@randomsample.de> writes:
>
>> This is not about reaching a consensus. This is about you giving proper
>> reasons for such a big change.
>
> To be clear, I would also put Eshell (though not pcomplete) into the category
> of "big things that should be in ELPA" -- albeit, the subset of ELPA that will
> be in the release tarball.
>
> It's hard to pin down why in a manner that fits in an e-mail. If Eshell were
> in ELPA today, and we were talking about moving it into core, I would have
> just as much trouble justifying that move too. Perhaps this simply underscores
> the fact that we don't have an agreed upon ELPA policy we can all
> refer to.

Yes.

> OK, David, I won't decide this by fiat. Let us decide what our ELPA policy
> will be, until it becomes clear, based on that document, what should go where,
> and why. We'll defer discussions of package movement until we have that in
> place.

Thank you.


-- 
-- Stephe

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Stephen Leake <stephen_leake@stephe-leake.org>,  aaronecay@gmail.com,  Eli Zaretskii <eliz@gnu.org>,  Stromeko@nexgo.de,  Dmitry Gutov <dgutov@yandex.ru>,  emacs-devel@gnu.org
> Date: Tue, 10 Nov 2015 10:52:41 -0800
> 
> > Why?
> 
> There will never be 100% agreement on whether they should be in ELPA, or be in
> Core, so I'm making the decision that they belong in ELPA.

IMO, it's a mistake to move CEDET.

Re: ELPA policy

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: John Wiegley <jwiegley@gmail.com>
>> Cc: Stephen Leake <stephen_leake@stephe-leake.org>,
>> aaronecay@gmail.com, Eli Zaretskii <eliz@gnu.org>,
>> Stromeko@nexgo.de, Dmitry Gutov <dgutov@yandex.ru>,
>> emacs-devel@gnu.org
>> Date: Tue, 10 Nov 2015 10:52:41 -0800
>> 
>> > Why?
>> 
>> There will never be 100% agreement on whether they should be in ELPA, or be in
>> Core, so I'm making the decision that they belong in ELPA.
>
> IMO, it's a mistake to move CEDET.

Bald statements of preference, without rationale, do not help move this
discussion forward.

We are trying to establish a general ELPA policy, not just vote on one
particular package.

What is it about CEDET that makes it a poor candidate?

How would your workflow suffer if it was moved?

How would users suffer?

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> David Engster <deng@randomsample.de> writes:
>
>>> Large packages like CEDET should move outside of Emacs.git and into
>>> Elpa.git.
>
>> Why?
>
> There will never be 100% agreement on whether they should be in ELPA, or be in
> Core, so I'm making the decision that they belong in ELPA. 

I'm happy to have someone making decisions, but there needs to be some
rationale given, so we understand the tradeoffs.

I believe the rationale here is:

- CEDET is not used by other core Emacs code (xref.el will be fixed)

- code that is not used by other core Emacs code clutters the workspace,
  producing extra hits in greps, etc.

- CEDET can still be included in the Emacs tarball, so users do
  not have to use list-packages to install it (this requires a to be
  implemented mechanism).

>> From my experience, git submodules are never seamless to work with.
>
> Even still, they work perfectly fine for this purpose.

I would like to see an actual demonstration, please; there are lots of
devils in the details.

Which means:

Please post a clear recipe for setting up an Emacs + ELPA developer
workspace, starting from git clone, so I can try this on my machine.


The test is then that some core Emacs code depend on some ELPA package;
we'd need branches in both to make that happen, since it doesn't now.

-- 
-- Stephe

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  aaronecay@gmail.com,  Stromeko@nexgo.de,  emacs-devel@gnu.org,  Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 10 Nov 2015 10:28:26 -0800
> 
> Large packages like CEDET should move outside of Emacs.git and into Elpa.git.

"Should" based on what? just the fact that it's large?  I think we
should decide this stuff on a case by case basis.  For example:

> If xref.el depends on CEDET, it would move to Elpa.git as well.

IMO, the exact opposite: if there are core features that we want to be
in Emacs no matter what, and those features depend on a package which
could be a candidate to move to ELPA, that package should NOT move to
ELPA.

Re: ELPA policy

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

>> Large packages like CEDET should move outside of Emacs.git and into
>> Elpa.git.
> "Should" based on what? just the fact that it's large? I think we should
> decide this stuff on a case by case basis. For example:

I'm surprised you say this, Eli, because in another thread you agree that
packages like this should be in Elpa, didn't you?

>> If xref.el depends on CEDET, it would move to Elpa.git as well.

> IMO, the exact opposite: if there are core features that we want to be in
> Emacs no matter what, and those features depend on a package which could be
> a candidate to move to ELPA, that package should NOT move to ELPA.

Core should provide functionality along the lines of a "standard library" and
a "standard environment", where having them in core is as much a statement
about consistency of interface, as it is about universal availability of the
functionality.

Since xref.el does not need to depend on CEDET, I don't see a reason why it
should, causing CEDET to remain in core.

John

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: stephen_leake@stephe-leake.org,  aaronecay@gmail.com,  Stromeko@nexgo.de,  emacs-devel@gnu.org,  dgutov@yandex.ru
> Date: Tue, 10 Nov 2015 10:54:51 -0800
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Large packages like CEDET should move outside of Emacs.git and into
> >> Elpa.git.
> > "Should" based on what? just the fact that it's large? I think we should
> > decide this stuff on a case by case basis. For example:
> 
> I'm surprised you say this, Eli, because in another thread you agree that
> packages like this should be in Elpa, didn't you?

I said I _could_agree.  But not unconditionally.  We should discuss
each case.

> >> If xref.el depends on CEDET, it would move to Elpa.git as well.
> 
> > IMO, the exact opposite: if there are core features that we want to be in
> > Emacs no matter what, and those features depend on a package which could be
> > a candidate to move to ELPA, that package should NOT move to ELPA.
> 
> Core should provide functionality along the lines of a "standard library" and
> a "standard environment", where having them in core is as much a statement
> about consistency of interface, as it is about universal availability of the
> functionality.

Sorry, I don't understand what this means in practice.

> Since xref.el does not need to depend on CEDET, I don't see a reason why it
> should, causing CEDET to remain in core.

xref is as useful as its backends.  If you take away backends, it
becomes less useful, or supports less programming languages, or both.

Re: ELPA policy

[Eli Zaretskii]

How about if we leave this issue alone for a while?  Let's concentrate
on releasing Emacs 25.1 first, and return to this issue later.  Rome
wasn't built in one day, we don't have to rush.

WDYT?

Re: ELPA policy

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> How about if we leave this issue alone for a while? Let's concentrate on
> releasing Emacs 25.1 first, and return to this issue later. Rome wasn't
> built in one day, we don't have to rush.
> 
> WDYT?

Agreed. Let's table for now and revisit. If the suggested moves really trouble
core developers, we'll talk about it again before making a final decision.

John

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 09:21 PM, Eli Zaretskii wrote:

> xref is as useful as its backends.  If you take away backends, it
> becomes less useful, or supports less programming languages, or both.

None of the xref backends use Semantic. There are still only two of 
them: elisp and etags.

When CEDET grows a backend, it can just as well add it when 
semantic-mode is enabled. No need to have it in the core for that.

Re: ELPA policy

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> Since xref.el does not need to depend on CEDET, I don't see a reason why it
>> should, causing CEDET to remain in core.
>
> xref is as useful as its backends.  If you take away backends, it
> becomes less useful, or supports less programming languages, or both.

Yes.

That's an argument for making the CEDET xref backend available
somewhere; I've committed to ensuring that.

But this discussion is about ELPA vs Emacs core:

Does it matter whether that backend is in ELPA or Emacs core?

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> Large packages like CEDET should move outside of Emacs.git and into
>>> Elpa.git.
>> "Should" based on what? just the fact that it's large? I think we should
>> decide this stuff on a case by case basis. For example:
>
> I'm surprised you say this, Eli, because in another thread you agree that
> packages like this should be in Elpa, didn't you?
>
>>> If xref.el depends on CEDET, it would move to Elpa.git as well.
>
>> IMO, the exact opposite: if there are core features that we want to be in
>> Emacs no matter what, and those features depend on a package which could be
>> a candidate to move to ELPA, that package should NOT move to ELPA.
>
> Core should provide functionality along the lines of a "standard library" and
> a "standard environment", where having them in core is as much a statement
> about consistency of interface, as it is about universal availability of the
> functionality.

Right. Since CEDET provides lots of interesting infrastrucure, I assumed
it was part of the Emacs standard library.

But I also see no reason not to have two levels of "standard
infrastructure"; one in core Emacs, on in ELPA.

That said, moving CEDET out of core to ELPA can be seen as a demotion.
For example, I was assuming that the EDE package system should be
prefered over any package system that is in ELPA, precisely because it
is in Emacs git. So I was spending my time improving EDE, rather than
some other ELPA package.

That aspect should be considered.

In other emails, I've said "CEDET is a tarball package; in
Emacs git solely to be included in the tarball".

Is it fair to add "also because it is approved as part of the standard
Emacs library"?

> Since xref.el does not need to depend on CEDET, I don't see a reason why it
> should, causing CEDET to remain in core.

I would object to moving CEDET to ELPA git if it were not easy to
refactor the current dependency to still be provided by an ELPA package.

-- 
-- Stephe

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:49 PM, Eli Zaretskii wrote:

> IMO, the exact opposite: if there are core features that we want to be
> in Emacs no matter what, and those features depend on a package which
> could be a candidate to move to ELPA, that package should NOT move to
> ELPA.

xref only depends on a minor feature of CEDET (the code indexing tools 
integration), and I've done that in a quick-and-dirty way, as a proof of 
concept.

It would be better to have functionality like that in the core, 
independent of CEDET.

Re: ELPA policy

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: John Wiegley <jwiegley@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>, aaronecay@gmail.com,
>> Stromeko@nexgo.de, emacs-devel@gnu.org, Dmitry Gutov
>> <dgutov@yandex.ru>
>> Date: Tue, 10 Nov 2015 10:28:26 -0800
>> 
>> Large packages like CEDET should move outside of Emacs.git and into Elpa.git.
>
> "Should" based on what? just the fact that it's large?  I think we
> should decide this stuff on a case by case basis.  For example:
>
>> If xref.el depends on CEDET, it would move to Elpa.git as well.
>
> IMO, the exact opposite: if there are core features that we want to be
> in Emacs no matter what, and those features depend on a package which
> could be a candidate to move to ELPA, that package should NOT move to
> ELPA.

In general, I agree.

However, there are two _independent_ aspects of "move to ELPA":

1) Move to ELPA git.

2) Be distributed by the ELPA package server.


If other core code depends on a package, it should not move to ELPA git
(unless the dependency is easily refactored, as in the CEDET/xgit case;
there will always be corner cases).

However, if there is a desire to do package releases between Emacs
releases, making the package available via the ELPA package server makes
sense.

I think this discussion is mainly focused on "move to ELPA git", but it
would help to be clear.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>> ELPA packages that other core code depends on (like CEDET; xref uses it -
>> called "core ELPA packages" hereinafter) have to be in every developer's
>> build environment everyday; the other core code has to see the current
>> version of the package, and it has to be the same for every developer.
>> 
>> If core ELPA packages are in the ELPA git repository, you would copy some
>> subtrees of the ELPA git workspace into the Emacs git workspace. People have
>> looked into doing this, but it gets messy and confusing. "git fetch" is
>> dealing with two upstreams for one workspace. There has been talk about git
>> submodules, etc, but nothing concrete.
>
> Elpa.git should be a submodule referenced from within Emacs.git (under
> "elpa").

That's overkill; there are many packages in Gnu ELPA that core code
should _not_ depend on.

We could try to have a list of approved packages in a file somewhere,
but I don't think we can enforce that; better to use git to enforce it.

We need a way to explicitly label and enforce which ELPA packages core
code can depend on. The only demonstrated way to do that is to keep them
in Emacs git.

> This allows us to expressly state which "version" of Elpa is expected
> to work with the current Emacs.git.

That's only required for the ELPA packages that core code can depend on.
For the other ELPA packages, we explicitly don't want to know if they
work with Emacs master; that's up to the individual package maintainers.
That's one of the primary purposes of ELPA.

> Large packages like CEDET should move outside of Emacs.git and into
> Elpa.git.

If this said "large packages that the rest of Emacs core does not depend
on ...", I could agree.

I just checked; the only other core code that depends on CEDET is
xref.el, which can be fixed. So I agree, CEDET should move to elpa git.

However, I assume the reason CEDET is in Emacs git is because there is a
desire to distribute it in the tarball; a sufficient number of users
expect it to be there without going thru package install. So the
mechanism to put ELPA packages in the tarball (but _not_ in the emacs
development workspace) must also be implemented. That should not be too
hard; no git submodules, just directory tree copy.

So this introduces a third kind of ELPA package: "distributed in Emacs
tarball". A "tarball package" for short?

Except that Eric maintains that there is more to "CEDET" in Emacs core
than just lisp/cedet/*. So to be clear, we are proposing to move only
lisp/cedet/* to elpa git.

> If xref.el depends on CEDET, it would move to Elpa.git as well.

As Dmitry points out, better to remove the dependency from the core
xref.el. Then we can add it back either in the ELPA CEDET package or in
a new xref-cedet ELPA package.

>> If we use the approach of keeping core ELPA packages in the Emacs git
>> repository, there is _zero_ friction for the current core Emacs developers;
>> the only change is in the ELPA config scripts.
>
> I do not want this code replication. 

How is it replication?

There are two styles of ELPA packages currently:

1) The only repository for the code is Gnu ELPA git

2) There is an external repository that the package maintainer uses;
   releases are pushed to Gnu ELPA git

The choice is up to the ELPA package maintainer.

The proposal is to change both to say:

1a) The only repository for the code is Gnu ELPA git or Gnu Emacs git.

2a) There is an external repository that the package maintainer uses;
    releases are pushed to Gnu ELPA git or Gnu Emacs git.

The ELPA packages in Emacs git are "core ELPA packages"; core Emacs code
may rely on them. 

There is no additional replication by keeping the core ELPA packages in
Emacs git.

The only reason the core Emacs packages are in the ELPA server is to
allow package releases in addition to Emacs releases. The package
maintainers have to accept maintaining backward compatibility, so a core
package in an Emacs 25.1 binary install can work with a more recent
version of the core ELPA package.

In summary, I'm proposing that there are three kinds of ELPA packages:

1) "normal ELPA package". 

    - Stored in Gnu ELPA git, and possibly in the package maintainer's
      separate repository.

    - Releases only occur via the ELPA package server, whenever the
      version number increments in the ELPA git main package file.
    
    - Core Emacs code may not depend on these.

2) "tarball ELPA package". 

    - Stored in Gnu ELPA git, and possibly in the package maintainer's
      separate repository.

    - Each Emacs release includes a released version of the package in
      the Emacs tarball, via copy from an ELPA workspace at Emacs
      tarball build time. Other releases also occur, via the ELPA
      package server, whenever the version number increments in the ELPA
      git main package file.
    
    - Core Emacs code may not depend on these.

3) "core ELPA package".  

    - Stored in Gnu Emacs git, and possibly in the package maintainer's
      separate repository.

    - Each Emacs release includes a version of the package in the
      tarball, because it is in the Emacs git workspace. Other releases
      also occur, via the ELPA package server, whenever the version
      number increments in the Emacs git main package file.
    
    - Core Emacs code may depend on these.

-- 
-- Stephe

Re: ELPA policy

[John Wiegley]

>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:

>> Elpa.git should be a submodule referenced from within Emacs.git (under
>> "elpa").

> That's overkill; there are many packages in Gnu ELPA that core code should
> _not_ depend on.

I believe that no core code should depend on any ELPA packages. Such a
dependency would be a reason to move that package into core, no? If that's
really the case, no submodule reference is necessary after all.

> If this said "large packages that the rest of Emacs core does not depend on
> ...", I could agree.

Yes, I meant that.

> So this introduces a third kind of ELPA package: "distributed in Emacs
> tarball". A "tarball package" for short?

Yes, for ELPA packages, there would be two flavors: in the tarball, and not.
Some have suggested just putting everything in the tarball for now. So we
could defer this question until the size of the tarball mattered, or if a
particular package was so dynamic it didn't want a version frozen into the
tarball.

Note that going down this road starts to make the line between "core" and
"ELPA" very thin, with easy migration between the two when desired. For
developers it allows Git to focus better, but for users, the difference is
largely invisible.

> Except that Eric maintains that there is more to "CEDET" in Emacs core than
> just lisp/cedet/*. So to be clear, we are proposing to move only
> lisp/cedet/* to elpa git.

Yes.

> In summary, I'm proposing that there are three kinds of ELPA packages:

I think we have a lot of agreement here, but I'd still like a clearer policy
than just "if core depends on it".

John

RE: ELPA policy

[Drew Adams]

Here's a naive question regarding where you place Lisp code
that is distributed with Emacs (part of the tarball): Will
anything change for users, depending on where you happen to
manage the code wrt ELPA etc.?

I'm guessing (and hoping) not, so that the distributed
directory structure will remain the same.  Is that correct?

Re: ELPA policy

[Stephen Leake]

Drew Adams <drew.adams@oracle.com> writes:

> Here's a naive question regarding where you place Lisp code
> that is distributed with Emacs (part of the tarball): Will
> anything change for users, depending on where you happen to
> manage the code wrt ELPA etc.?
>
> I'm guessing (and hoping) not, so that the distributed
> directory structure will remain the same.  Is that correct?

That's a good point. But I'm not clear what previous state you are
comparing to.

There are two cases;

1) A package that is currently in Emacs git moves to Gnu ELPA git but
   remains in the Emacs tarball.

2) A package that is currently in Gnu ELPA git stays there, and is added
   to the Emacs tarball.

I would argue that tarball ELPA packages should be installed in some
system level elpa directory (similar to the user's .emacs.d/elpa), not
in the main emacs/lisp install directory.

I don't think the history of a package should determine where it is
installed. So in case 1), I expect the package to be in a different
installation directory from the previous Emacs release. But it's still
in the default load-path, so this change is transparent to the user.

But it's not too important; the important distinction is that tarball
ELPA packages are _not_ in the developer Emacs workspace, so core
code cannot accidently depend on them.

-- 
-- Stephe

RE: ELPA policy

[Drew Adams]

> > Here's a naive question regarding where you place Lisp code
> > that is distributed with Emacs (part of the tarball): Will
> > anything change for users, depending on where you happen to
> > manage the code wrt ELPA etc.?
> >
> > I'm guessing (and hoping) not, so that the distributed
> > directory structure will remain the same.  Is that correct?
> 
> That's a good point. But I'm not clear what previous state you are
> comparing to.

Previous delivery of Emacs (back to ~Day One), including its
source code, especially Lisp, under directory lisp/.  What's
unclear about that?

> There are two cases;
> 1) A package that is currently in Emacs git moves to Gnu ELPA git but
>    remains in the Emacs tarball.
> 2) A package that is currently in Gnu ELPA git stays there, and is added
>    to the Emacs tarball.
> 
> I would argue that tarball ELPA packages should be installed in some
> system level elpa directory (similar to the user's .emacs.d/elpa), not
> in the main emacs/lisp install directory.

You say you would argue that, but where's the argument?  How about
a reason?

As one user, I would prefer that all distributed Lisp code be under
lisp/.  Makes things simpler for users (who might even have existing
scripts etc. that expect this).

Is there a good reason not to keep this behavior?

I don't really care where you stick code, or how you access it, for
development purposes.  Where an Emacs distribution puts it for
_users_ should remain what it has been, IMO, if possible - somewhere
under lisp/, at least.

Why should changes in where you keep code for development affect
where users of Emacs find it?

> I don't think the history of a package should determine where it is
> installed. So in case 1), I expect the package to be in a different
> installation directory from the previous Emacs release. But it's still
> in the default load-path, so this change is transparent to the user.

`load-path' is not everything, so no, it is _not_ transparent to
users, depending on what they do and how they use Emacs (including
its source code).

Emacs should be able to present its source code to users in the
same way as in the past, I would think.  They certainly should
not be _required_ to use the package system or git or ... to
access it.

`package.el' should be a nice-to-have.  It should not become an
obligatory hoop for users to jump through.  Most users will find
it handy, just as some users might find a menu handy.  But it
should not become a necessary way to access Emacs code.

> But it's not too important; the important distinction is that tarball
> ELPA packages are _not_ in the developer Emacs workspace, so core
> code cannot accidently depend on them.

That might be important to developers, but it is not the issue I
asked about.

Re: ELPA policy

[Achim Gratz]

Eli Zaretskii writes:
> The suggestion was to move _all_ of them, except the few that are
> needed for bootstrap, out of the Emacs repository.

You keep arguing about things that realistically aren't even on the
table yet (I don't want to prevent that discussion, mind you).  I said
it would be possible to do this, given some not-yet-existing
infrastructure and the only reason I even mentioned it was because
that's the other end of the spectrum from moving everything that looks
halfway useful into core.  Even if that infrastructure develops (which I
have serious doubts about given how this discussion goes), nobody in
their right mind would move all packages big-bang style to ELPA.
There'd certainly be a transition period with some pilot packages, I'd
probably pick Org, CEDET and Tramp based on the list traffic.  You'd
then see how that works and make some fixes that will certainly be
necessary before you get to the point to move more packages over to
ELPA.  This being Emacs, for better or worse, I'd expect that at this
point we'd be looking at _years_ before the extreme end you seem to be
so worried about is even within reach.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: ELPA policy

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Mon, 09 Nov 2015 19:22:50 +0100
> 
> Eli Zaretskii writes:
> > The suggestion was to move _all_ of them, except the few that are
> > needed for bootstrap, out of the Emacs repository.
> 
> You keep arguing about things that realistically aren't even on the
> table yet (I don't want to prevent that discussion, mind you).

If that's not on the table, then please tell what in your opinion _is_
on the table.  You did want to propose something practical that could
be done soon, right?  Because this discussion was about what we want
to do from now on with ELPA, not what we might want considering in
some distant future that might never come.

So please, let's be practical and consider alternatives that
realistically can be taken soon.  Like tomorrow or the next month.
Okay?

> There'd certainly be a transition period with some pilot packages, I'd
> probably pick Org, CEDET and Tramp based on the list traffic.

Moving out a few large packages that are developed outside Emacs
anyway is a no-brainer, provided that their developers agree.  I
already said I'll probably agree, and no one else objected.  The only
question is what is the opinion of the respective developers: we
cannot move the packages if they disagree.

Is there some other alternative on the table?  If not, we can finally
agree about something, I think.

Thanks.

Re: ELPA policy

[David Engster]

Eli Zaretskii writes:
>> From: Achim Gratz <Stromeko@nexgo.de>
>> Date: Mon, 09 Nov 2015 19:22:50 +0100

>> There'd certainly be a transition period with some pilot packages, I'd
>> probably pick Org, CEDET and Tramp based on the list traffic.
>
> Moving out a few large packages that are developed outside Emacs
> anyway is a no-brainer, provided that their developers agree.  I
> already said I'll probably agree, and no one else objected.  The only
> question is what is the opinion of the respective developers: we
> cannot move the packages if they disagree.

As long as Emacs core cannot depend on packages that are bundled from
ELPA, I would definitely not agree to move CEDET to ELPA.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Date: Mon, 09 Nov 2015 19:58:29 +0100
> Cc: Achim Gratz <Stromeko@nexgo.de>, emacs-devel@gnu.org
> 
> As long as Emacs core cannot depend on packages that are bundled from
> ELPA, I would definitely not agree to move CEDET to ELPA.

Of course.  I don't think we should move anything before such a
dependency is possible.

Re: ELPA policy

[David Engster]

Eli Zaretskii writes:
>> From: David Engster <deng@randomsample.de>
>> Date: Mon, 09 Nov 2015 19:58:29 +0100
>> Cc: Achim Gratz <Stromeko@nexgo.de>, emacs-devel@gnu.org
>> 
>> As long as Emacs core cannot depend on packages that are bundled from
>> ELPA, I would definitely not agree to move CEDET to ELPA.
>
> Of course.  I don't think we should move anything before such a
> dependency is possible.

But if you have such dependencies, what's the point? IIRC, Stefan
decidedly did not want such dependencies, since the whole point was to
add those bundles only when creating releases.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: Stromeko@nexgo.de,  emacs-devel@gnu.org
> Date: Mon, 09 Nov 2015 20:35:04 +0100
> 
> Eli Zaretskii writes:
> >> From: David Engster <deng@randomsample.de>
> >> Date: Mon, 09 Nov 2015 19:58:29 +0100
> >> Cc: Achim Gratz <Stromeko@nexgo.de>, emacs-devel@gnu.org
> >> 
> >> As long as Emacs core cannot depend on packages that are bundled from
> >> ELPA, I would definitely not agree to move CEDET to ELPA.
> >
> > Of course.  I don't think we should move anything before such a
> > dependency is possible.
> 
> But if you have such dependencies, what's the point? IIRC, Stefan
> decidedly did not want such dependencies, since the whole point was to
> add those bundles only when creating releases.

I'm not pushing for the move, I just said that if the technical
problems are successfully resolved, I won't object.

Re: ELPA policy

[Rasmus]

Eli Zaretskii <eliz@gnu.org> writes:

> Moving out a few large packages that are developed outside Emacs
> anyway is a no-brainer, provided that their developers agree.  I
> already said I'll probably agree, and no one else objected.  The only
> question is what is the opinion of the respective developers: we
> cannot move the packages if they disagree.

I'm pretty sure Org would be happy to be distributed this way.  In fact,
we have got the 8.3 release now.  It really *must* be in 25.1, but I think
it would be fine to try out Fabian’s system to allow it to be upgraded
later.  Otherwise, I will hopefully be able to merge it soon the
"copy-paste" way.

Rasmus


-- 
However beautiful the theory, you should occasionally look at the evidence

Re: ELPA policy

[Achim Gratz]

Eli Zaretskii writes:
> If that's not on the table, then please tell what in your opinion _is_
> on the table.  You did want to propose something practical that could
> be done soon, right?  Because this discussion was about what we want
> to do from now on with ELPA, not what we might want considering in
> some distant future that might never come.

I did want to propose something that could become a vision for some time
in the foreseeable, yet somewhat distant future that we can then take
steps towards.  ELPA is foremost for _users_ and any decisions taken
should put the users first and developers second.  As a user I don't
have the luxury of chosing which Emacs version I get to use and what
packages are already installed on the system (indeed in my day job that
changes when I switch between project workspaces).  Wouldn't it be
splendid if ELPA helped me making those differences fade into the
background, if not disappear entirely?

> So please, let's be practical and consider alternatives that
> realistically can be taken soon.  Like tomorrow or the next month.
> Okay?

No, not okay.  There's still no consensus of even roughly where we want
to end up, so starting a random walk to find out who's feeling well or
not with whichever direction is certainly not my idea of progress.

> Moving out a few large packages that are developed outside Emacs
> anyway is a no-brainer, provided that their developers agree.

Again, it's not.  It simply doesn't work at this moment, not for Emacs,
not for the package developers and not for the users.  That doesn't have
anything to do with the size of those packages best I can tell.  To
change that (without moving the sources anywhere, just to not go down
that rabbit hole again) we'd need to devise a way to have an Emacs core
package whose autoload definitions can be separated and that can be
overridden (maybe only with a newer version) on the installation level
or by the user.  That would be the first step before any others can
follow, provided you even want to go that direction.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

ELPA policy

[Julien Danjou]

Hi,

This topic has been recently brought by Lars, Ted and me on the gnus
mailing list. I've been asked to move this discussion here for
clarification.

I am currently worried by the Emacs ELPA archive and what will be that
policy. As Lars pointed out, until now, packages have been added over
the years into Emacs trunk, maintained by a extended set of skilled
hands, assuring good quality in (almost :-)) every place of every
packages furnished with Emacs.

Now, with the introduction of ELPA I'm worried about a lot of things.

- Who will be able to upload to ELPA?
- Who will fix the packages' bugs?
- Who will assure there's no regression for Emacs 24.1 user when people
  will starts uploading packages using 24.2 only function?
- Who will assure there's no regression at all?
- Who will assure there's no really bad things uploaded?
- Will all new packages go to ELPA, or with some still go to Emacs
  trunk? And if some can go to the trunk, upon which rules?
- Will all/most of the current packages be moved to ELPA?

There's probably a lot more question outstanding. I've the feeling ELPA
is adding a second class citizenship for packages, and I really do not
like that, at least in its current form.

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

Re: ELPA policy

[Chong Yidong]

Julien Danjou <julien@danjou.info> writes:

> I am currently worried by the Emacs ELPA archive and what will be that
> policy. As Lars pointed out, until now, packages have been added over
> the years into Emacs trunk, maintained by a extended set of skilled
> hands, assuring good quality in (almost :-)) every place of every
> packages furnished with Emacs.

Emacs contains many packages that are maintained "externally".  While
Emacs developers might make small changes to the in-tree version, most
development is done upstream and periodically merged in.  Those upstream
maintainers handle bugfixes and ensure compatibility among Emacs
versions.

One good reason to put a package in elpa.gnu.org, rather than in the
Emacs tarball, is if it is likely to benefit only a small segment of
Emacs users (even if it's of tremendous usefulness to that segment).
Especially, but not necessarily, if a package is large and complex, like
Auctex and Muse.

There are other good reasons too, e.g. packages that we want to merge
into Emacs core in the future, but not yet (for whatever reason).

(Incidentally, there are also some packages in Emacs that might be
usefully moved out into elpa.gnu.org, e.g. since they are so rarely
used.  We could also move some of the files in obsolete/ into a
subrepository, e.g. elpa.gnu.org/packages/obsolete)

(Also, as stated before, the FSF's policy is that for a package to be
listed on elpa.gnu.org its copyright must be assigned, in the exact same
way as if it's included in Emacs core.)

> - Who will be able to upload to ELPA?
> - Who will assure there's no really bad things uploaded?

The way it's currently set up is that only a couple of people can upload
to ELPA; package maintainers, when they release a new version, should
email me (or Ted) to get the package uploaded.  The system is still a
work in progress; we might set up a more elaborate "staging area" system
in the future.  (Such a system would still involve a human component, of
course, to reduce the possibility of malicious uploads.)  I will add a
page to the elpa.gnu.org webpage explaining this.

Re: ELPA policy

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> One good reason to put a package in elpa.gnu.org, rather than in the
> Emacs tarball, is if it is likely to benefit only a small segment of
> Emacs users (even if it's of tremendous usefulness to that segment).
> Especially, but not necessarily, if a package is large and complex, like
> Auctex and Muse.
>
> There are other good reasons too, e.g. packages that we want to merge
> into Emacs core in the future, but not yet (for whatever reason).

The reason that I raised the question now was that I was trying to
figure out how to do reasonable colour distance calculations for
rendering foreground colours in shr.el.  rainbow-mode (for colourising
colour specs in, for instance, CSS files) was brought up, that had
similar needs, and its calculations in the same area might possibly be
reusable by shr.el.

But apparently rainbow-mode went to ELPA instead of into Emacs, even
though it's small, it's clearly written, and it seems to be generally
useful for anybody editing stuff like CSS or .Xdefaults files or
anything.  So I wondered why that was...

> The way it's currently set up is that only a couple of people can upload
> to ELPA; package maintainers, when they release a new version, should
> email me (or Ted) to get the package uploaded.

If ELPA is supposed to be a repository of really special-needs code, I
think it's a good idea.  It would be nice to have an automatic way to
download, say, one of the Emacs-based mp3 players.  But if it's going to
carry stuff that people do really want to have in the Emacs
distribution, then I think it's counter-productive.

Stuff that is in Emacs gets loving care.  When url.el needs tweaking
(for instance), people step up to the plate and makes sure that it
works.  Things that live in a package repository bit-rot at an alarming
rate.  One of the great things about Emacs is that when you "apt-get
install emacs", you get a fully-featured system where you can be
reasonably sure that things work.  Over-modularisation can be costly in
the long run.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: ELPA policy

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> figure out how to do reasonable colour distance calculations for
> rendering foreground colours in shr.el.  rainbow-mode (for colourising
> colour specs in, for instance, CSS files) was brought up, that had
> similar needs, and its calculations in the same area might possibly be
> reusable by shr.el.
>
> But apparently rainbow-mode went to ELPA instead of into Emacs, even
> though it's small, it's clearly written, and it seems to be generally
> useful for anybody editing stuff like CSS or .Xdefaults files or
> anything.  So I wondered why that was...

Rainbow-mode, as presented, is a tool for colorizing strings in Emacs
buffers, the kind of neat hack that doesn't really need to be in Emacs.
From your description, it sounds like parts of it can be usefully
refactored into a general color-manipulation library, in which case we
could include that part in Emacs (preferably, using a better prefix than
`rainbow').

> If ELPA is supposed to be a repository of really special-needs code, I
> think it's a good idea.  It would be nice to have an automatic way to
> download, say, one of the Emacs-based mp3 players.  But if it's going
> to carry stuff that people do really want to have in the Emacs
> distribution, then I think it's counter-productive.  Stuff that is in
> Emacs gets loving care.  Things that live in a package repository
> bit-rot at an alarming rate.

Well, obviously it's going to be a judgement call, analogous to the
debates about what goes into distribution DVDs (with programs like Emacs
nowadays frequently not making the cut :-P).  The level of bit-rot is
dependent on the upstream package maintainers; I don't think you can
argue that packages like Auctex (or SLIME or ESS, which are not
packaged) have bit-rotted.

Some packages integrated into Emacs are no longer active upstream but
are still tinkered with by Emacs developers, but what's really happening
here is that we have effectively forked the project.  If a package on
elpa.gnu.org ends up being abandoned, there's little difference in how
we could solve the problem, except the fork would be more explicit than
implicit.

Re: ELPA policy

[Edward O'Connor]

> The reason that I raised the question now was that I was trying to
> figure out how to do reasonable colour distance calculations for
> rendering foreground colours in shr.el.  rainbow-mode (for colourising
> colour specs in, for instance, CSS files) was brought up, that had
> similar needs, and its calculations in the same area might possibly be
> reusable by shr.el.

Maybe generalizing `tty-color-approximate' is the right thing to do?


Ted

Re: ELPA policy

[Glenn Morris]

Chong Yidong wrote:

> (Incidentally, there are also some packages in Emacs that might be
> usefully moved out into elpa.gnu.org, e.g. since they are so rarely
> used.  We could also move some of the files in obsolete/ into a
> subrepository, e.g. elpa.gnu.org/packages/obsolete)

I don't see how that would work. If, as soon as something becomes
obsolete, you move it to elpa, you've defeated the entire point of
obsolescence. Emacs goes from "you can assume we have function foo" to
"you can't rely on us having function foo" (unless your users have
added an optional package).

If on the other hand you move it to elpa at the point it which it
would normally be deleted altogether, then you're just prolonging its
life, when the whole point of making things obsolete is to migrate
people away from them.

And to move it at some intermediate stage seems a little pointless.

> (Also, as stated before, the FSF's policy is that for a package to be
> listed on elpa.gnu.org its copyright must be assigned, in the exact same
> way as if it's included in Emacs core.)

I was surprised to see AUCTeX there, since I thought its copyright was
notoriously hard to assign. At a quick glance, tex-fptex.el, tex-jp.el
have non-FSF copyrights in the headers, and there is no information
for the images/ directory. The texinfo source of the manuals and pdfs
also appears to be missing (maybe this is by design).

Re: ELPA policy

[Tom Tromey]

>>>>> "Julien" == Julien Danjou <julien@danjou.info> writes:

Julien> - Who will assure there's no regression for Emacs 24.1 user when people
Julien>   will starts uploading packages using 24.2 only function?

You could try making it so uploads require a warning-free byte-compile
on various Emacs versions first.

Packages can require a minimum Emacs version, if the author knows that
this is needed.

Julien> - Who will assure there's no regression at all?

You can't catch all the possible bugs.
I've had ok results on my site just forwarding reports to the package
maintainer for fixing.

Julien> There's probably a lot more question outstanding. I've the feeling ELPA
Julien> is adding a second class citizenship for packages, and I really do not
Julien> like that, at least in its current form.

My view leans more towards package inclusion maximalism -- put
everything in Emacs, but also do more frequent releases of some included
packages via ELPA.

Tom

Re: ELPA policy

[Glenn Morris]

Tom Tromey wrote:

> You could try making it so uploads require a warning-free byte-compile
> on various Emacs versions first.

Hah, we can't even get this for Emacs proper...

Re: ELPA policy

[Stefan Monnier]

> This topic has been recently brought by Lars, Ted and me on the gnus
> mailing list. I've been asked to move this discussion here for
> clarification.

As mentioned Chong, it's still "work in progress" from this point
of view.  But since your questions use the future tense, I'll try to
reply based on what I think should happen in the future.

> - Who will be able to upload to ELPA?

Depends: new packages or updates to existing packages?
For new packages, I'd expect only a few people to have such rights, but
for updates, I'd expect something like "anybody with access to the Bzr
repository".  After all, if they can screw with the main Emacs codebase,
why not with the ELPA packages.

> - Who will fix the packages' bugs?

The upstream maintainers, I'd expect.  That's one of the reasons to have
packages outside of Emacs's tree.

> - Who will assure there's no regression for Emacs 24.1 user when people
>   will starts uploading packages using 24.2 only function?

The upstream maintainer, for the same reason.

> - Who will assure there's no regression at all?

In which sense?  I'd expect the upstream maintainer to take some role
here, but admittedly, since those packages are easily available in
a central place, it's more likely that Emacs maintainers will pay
attention to them when introducing changes to the core.

> - Who will assure there's no really bad things uploaded?

The same people who currently assure that there's no really bad things
installed in the Emacs trunk.

> - Will all new packages go to ELPA, or with some still go to Emacs
>   trunk? And if some can go to the trunk, upon which rules?

We don't have rules for it, right now.  Here are some considerations, on
top of what Chong mentioned:
- in order to keep Emacs maintainable, we'd rather have packages in
  ELPA, all things being equal.
- packages of general usefulness (e.g. libraries) are good candidates
  for inclusion in Emacs, to reduce dependencies among ELPA packages.
  
> - Will all/most of the current packages be moved to ELPA?

Definitely not (in the foreseeable future).

> There's probably a lot more question outstanding. I've the feeling ELPA
> is adding a second class citizenship for packages, and I really do not
> like that, at least in its current form.

We currently have two kinds of packages: bundled and unbundled.  So ELPA
is about adding a third kind in-between, with properties such as
"smooth integration", "ease of installation", ... to be halfway between
the other two.


        Stefan

Re: ELPA policy

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> For new packages, I'd expect only a few people to have such rights,
> but for updates, I'd expect something like "anybody with access to the
> Bzr repository".  After all, if they can screw with the main Emacs
> codebase, why not with the ELPA packages.

One difference, though, is that screwing with the main Emacs codebase
affects only those using the development version of Emacs, and we have
mechanisms like the diffs mailing list for problems to be easily
spotted.  After Emacs 24 is released, by screwing with elpa.gnu.org you
can immediately affect users of deployed stable Emacs versions.

So, we need to be a bit more paranoid for elpa.gnu.org than for our main
repository.

I agree, though, that it would be nice for Emacs developers to easily
edit packages in elpa.gnu.org without going through an onerous package
upload process.  I'm not sure how to set this up, though maybe the way
the Org daily builds are handled can be used as a starting point.

Re: ELPA policy

[Ted Zlatanov]

On Mon, 15 Nov 2010 15:12:15 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> For new packages, I'd expect only a few people to have such rights,
>> but for updates, I'd expect something like "anybody with access to the
>> Bzr repository".  After all, if they can screw with the main Emacs
>> codebase, why not with the ELPA packages.

CY> One difference, though, is that screwing with the main Emacs codebase
CY> affects only those using the development version of Emacs, and we have
CY> mechanisms like the diffs mailing list for problems to be easily
CY> spotted.  After Emacs 24 is released, by screwing with elpa.gnu.org you
CY> can immediately affect users of deployed stable Emacs versions.

CY> So, we need to be a bit more paranoid for elpa.gnu.org than for our main
CY> repository.

This is one good reason to keep a separate repository per major Emacs
version, at least.  Then you can fix packages without so much fear.

CY> I agree, though, that it would be nice for Emacs developers to easily
CY> edit packages in elpa.gnu.org without going through an onerous package
CY> upload process.  I'm not sure how to set this up, though maybe the way
CY> the Org daily builds are handled can be used as a starting point.

I hope we make it easy to submit changes but harder to publish them.
See below for my staging suggestion.

CY> Maintaining a separate repository for every Emacs release sounds like a
CY> lot of work.  Currently, it's up to maintainers of upstream/external
CY> packages ensure backward compatibility with older Emacs versions,
CY> e.g. by introducing compatibility functions.  Obviously this system
CY> doesn't work perfectly, but it doesn't seem like we have the manpower
CY> for anything more elaborate.

Setting it up is really not hard, even if we don't use it.  Just add
"packages-MAJOR" and "packages-MAJOR.MINOR" to the default "packages"
repository on startup.  Those can be empty but when we need something in
them (and we will, I promise you), they'll Just Work.

The dev checkout version should also have "packages-dev" or some such
that's only enabled if you make a dev build.

All this is not hard if we use VCS branches.  It's a 1-to-1 mapping to
the Emacs repo's branches.  In fact the elpa.gnu.org repo could mirror a
subdirectory of the Emacs repo, which would solve the staging problem
(developers would just stage to the Emacs repo and the elpa.gnu.org
maintainers would publish from there).

I think the alternative approach, keeping it really simple now, will
require more manpower and drain more time long-term.

Ted

Re: ELPA policy

[Richard Stallman]

    - Who will assure there's no really bad things uploaded?

We should treat inclusion in ELPA like inclusion in Emacs.

-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org