Proposed new core library: pl.el

Proposed new core library: pl.el

[John Wiegley]

pl.el (standing for "parser library") is a combinator parsing library for
Emacs, similar to Haskell's Parsec. You can see how it works at the following
README:

    https://github.com/jwiegley/emacs-pl

PL offers a way to write very compact parsers of structured text. For example
(from the README):

  (pl-parse
    (delete-region (pl-str "<xml>" :beg)
                   (pl-until
                     (pl-str "</xml>" :end))))

The idea being applicative parsers is that the result of `pl-parse' is the
FORM you pass in, where every sub-parser becomes a value of the type you
intended to parse. If a sub-parse fails, either the whole parse fails, or it
returns nil if you wrap the parser in `pl-try'.

There is room for improving performance, but the API is complete enough to
start using it. Giving the unproven status, though, perhaps it should start
out in ELPA, and move to core after it has solidified and gained some users?

John

Re: Proposed new core library: pl.el

[Dmitry Gutov]

On 11/05/2015 04:14 AM, John Wiegley wrote:

> There is room for improving performance, but the API is complete enough to
> start using it. Giving the unproven status, though, perhaps it should start
> out in ELPA, and move to core after it has solidified and gained some users?

I think the rule for moving new stuff out of ELPA should be whether it's 
used by the core. That's for libraries.

For utility packages (ui/commands/etc), the question should be asked: if 
it works fine in ELPA, why move? Popularizing ELPA would be better, IMHO.

ELPA policy (was: Proposed new core library: pl.el)

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> I think the rule for moving new stuff out of ELPA should be whether it's
> used by the core. That's for libraries.

An exception to this rule is when a certain service (say, streams) should
always be available, without requiring further installation of libraries.
Emacs acts as a sort of "standard library" for Emacs Lisp, so the same kinds
of things we'd like to have in such a meta-library, should be in core.

I do think that applications using such libraries should almost always go in
ELPA, except for those that have been grandfathered in, like Emacs Calc. There
are some things you should always be able to reach for, no matter whose
machine you are on, or if the Internet is currently available.

John

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 04:41 AM, John Wiegley wrote:

> An exception to this rule is when a certain service (say, streams) should
> always be available, without requiring further installation of libraries.
> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same kinds
> of things we'd like to have in such a meta-library, should be in core.

Why not consider ELPA a part of the "standard library", too?

As long as all code that uses streams.el is not in Emacs core, I don't 
see any harm in requiring the user to install streams from ELPA (which 
would probably happen automatically, via a declared dependency).

When the core starts using it, sure, we can move it there.

> I do think that applications using such libraries should almost always go in
> ELPA, except for those that have been grandfathered in, like Emacs Calc. There
> are some things you should always be able to reach for, no matter whose
> machine you are on, or if the Internet is currently available.

Yes, some pieces of functionality that we consider a part of the 
standard set (Calc is not that, for me, but apparently it's popular 
enough). The criteria for applications to include them might be that 
Emacs has dedicated menu items, or uses the application's commands in 
some global bindings (that's why xref, for example, can't be moved to ELPA).

Re: ELPA policy

[Artur Malabarba]

2015-11-05 3:00 GMT+00:00 Dmitry Gutov <dgutov@yandex.ru>:
> On 11/05/2015 04:41 AM, John Wiegley wrote:
>
>> An exception to this rule is when a certain service (say, streams) should
>> always be available, without requiring further installation of libraries.
>> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same
>> kinds
>> of things we'd like to have in such a meta-library, should be in core.
>
>
> Why not consider ELPA a part of the "standard library", too?
>
>[...]
> When the core starts using it, sure, we can move it there.

I agree in a general sense. But case-by-case discretion will have to be applied.

> The
> criteria for applications to include them might be that Emacs has dedicated
> menu items, or uses the application's commands in some global bindings
> (that's why xref, for example, can't be moved to ELPA).

IMO, xref can't be moved to Gelpa because users expect this "find
definition" functionality out-of-the box. By which I don't mean "Emacs
users have come to expect it", I mean "users coming to Emacs will
expect it".

Re: ELPA policy

[Michael Welsh Duggan]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/05/2015 04:41 AM, John Wiegley wrote:
>
>> An exception to this rule is when a certain service (say, streams) should
>> always be available, without requiring further installation of libraries.
>> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same kinds
>> of things we'd like to have in such a meta-library, should be in core.
>
> Why not consider ELPA a part of the "standard library", too?

Because it is not.  I have often had to use Emacs on computers that did
not have access to an Internet connection.  To consider something part
of the "standard library," I believe you should have access to it when
Emacs is installed.  This is why I dislike the move towards systems that
only provide easy installation via network channels.  (Python, this
means you!)

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 02:51 PM, Michael Welsh Duggan wrote:

>> Why not consider ELPA a part of the "standard library", too?
>
> Because it is not.  I have often had to use Emacs on computers that did
> not have access to an Internet connection.  To consider something part
> of the "standard library," I believe you should have access to it when
> Emacs is installed.

We do intend to bundle some packages from ELPA to be distributed with 
Emacs releases.

Anyway, if you don't have Internet access, you also can't install any 
package that depends on the library in question. Right?

Re: ELPA policy

[Michael Welsh Duggan]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/05/2015 02:51 PM, Michael Welsh Duggan wrote:
>
>>> Why not consider ELPA a part of the "standard library", too?
>>
>> Because it is not.  I have often had to use Emacs on computers that did
>> not have access to an Internet connection.  To consider something part
>> of the "standard library," I believe you should have access to it when
>> Emacs is installed.
>
> We do intend to bundle some packages from ELPA to be distributed with
> Emacs releases.

In this case, I can consider these part of a standard library.

> Anyway, if you don't have Internet access, you also can't install any
> package that depends on the library in question. Right?

Not so.  Packages can still be installed via sneakernet, and elisp can
be written that wants to take advantage of a "standard library."  I have
no objection to ELPA in general.  I just have an issue with the idea
that all of ELPA (non-bundled) can be considered "standard."

-- 
Michael Welsh Duggan
(mwd@cert.org)

Re: ELPA policy

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> We do intend to bundle some packages from ELPA to be distributed with Emacs
> releases.

I'd like to know more about this. How does the user access these bundled
packages; by installing them with M-x package-install?

John

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 05:09 PM, John Wiegley wrote:

> I'd like to know more about this. How does the user access these bundled
> packages; by installing them with M-x package-install?

By having them in the default load-path somewhere already, I imagine.

The user would be able to M-x package-install, however, to install newer 
a version of the same package from ELPA, that was released in the meantime.

I think Fabian was looking into making that happen, but I can't find the 
respective thread right now.

Re: ELPA policy

[Artur Malabarba]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/05/2015 05:09 PM, John Wiegley wrote:
>
>> I'd like to know more about this. How does the user access these bundled
>> packages; by installing them with M-x package-install?
>
> By having them in the default load-path somewhere already, I imagine.

There are a few possible levels this could go.
But yes, the basic idea is that these bundled packages would be pulled
from Gelpa into some subdir of emacs/lisp when making the tarball and
autoloading/byte-compilation would be done. So they'd be available like
any other built-in package.

The advantage over just keeping the packages there in the first place is
that packages offered on GElpa can be updated more frequently.

A second level would be to do this in such a way that allows Emacs
packages to depend on these pulled-in Gelpa packages. This would require
changes to the build process, not just the tarball.

> I think Fabian was looking into making that happen, but I can't find
> the respective thread right now.

I think I asked about this a couple of months ago, and Stefan (?) wrote
a very nice explanation of what his thoughts were on this possible
future.

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 06:58 PM, Artur Malabarba wrote:

>> I think Fabian was looking into making that happen, but I can't find
>> the respective thread right now.
>
> I think I asked about this a couple of months ago, and Stefan (?) wrote
> a very nice explanation of what his thoughts were on this possible
> future.

Stefan, most likely.  Too bad it wasn't in a dedicated bug discussion.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'd like to know more about this. How does the user access these bundled
  > packages; by installing them with M-x package-install?

I think they should appear, to the user, like any other standard part of Emacs.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

>> I'd like to know more about this. How does the user access these bundled
>> packages; by installing them with M-x package-install?

> I think they should appear, to the user, like any other standard part of
> Emacs.

If that's the case, it changes my thoughts on what needs to be in core, and
what should be in ELPA. Until now I was thinking ELPA required Internet
access; but if there are parts of ELPA that "come in the box", then I'd like
to see more packages there.

John

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Sun, 08 Nov 2015 08:30:17 -0800
> Cc: mwd@md5i.com, emacs-devel@gnu.org, dgutov@yandex.ru
> 
> If that's the case, it changes my thoughts on what needs to be in core, and
> what should be in ELPA. Until now I was thinking ELPA required Internet
> access; but if there are parts of ELPA that "come in the box", then I'd like
> to see more packages there.

ELPA requires Internet access if you are building from the repository.

My greatest fears are that packages that move from core to ELPA will
get much less attention from the core maintenance team, which will
eventually degrade their quality.

Re: ELPA policy

[Wolfgang Jenkner]

On Sun, Nov 08 2015, Eli Zaretskii wrote:

> My greatest fears are that packages that move from core to ELPA will
> get much less attention from the core maintenance team, which will
> eventually degrade their quality.

But this is not what actually happens with completely separately
developed complex packages like SLIME, Evil...

On the other hand, IIUC, moving CEDET to emacs core has not been
completely without problems for the CEDET project and its users.

Re: ELPA policy

[Eli Zaretskii]

> From: Wolfgang Jenkner <wjenkner@inode.at>
> Cc: John Wiegley <jwiegley@gmail.com>,  mwd@md5i.com,  emacs-devel@gnu.org,  rms@gnu.org,  dgutov@yandex.ru
> Date: Sun, 08 Nov 2015 19:00:06 +0100
> 
> On Sun, Nov 08 2015, Eli Zaretskii wrote:
> 
> > My greatest fears are that packages that move from core to ELPA will
> > get much less attention from the core maintenance team, which will
> > eventually degrade their quality.
> 
> But this is not what actually happens with completely separately
> developed complex packages like SLIME, Evil...

Most packages in core are not of this kind.

> On the other hand, IIUC, moving CEDET to emacs core has not been
> completely without problems for the CEDET project and its users.

Such moves are never without problems, which is exactly my point.  In
this case, I think the benefits far outweighed the problems: until
CEDET was incorporated into Emacs core, we couldn't even dream about
using it as infrastructure for IDEs and other similar features.
AFAIR, there was also lots of cleanup during the move.

But moving ELPA packages into the core is not being discussed or
suggested.  So what happened with CEDET, even if you disagree with my
assessment above, has little relevance to the issue at hand.

Re: ELPA policy

[Wolfgang Jenkner]

On Sun, Nov 08 2015, Eli Zaretskii wrote:

>> On the other hand, IIUC, moving CEDET to emacs core has not been
>> completely without problems for the CEDET project and its users.
>
> Such moves are never without problems, which is exactly my point.

I was using the wrong word here as it's more a fork than a move:

http://sourceforge.net/p/cedet/git/ci/master/tree/INSTALL

> In
> this case, I think the benefits far outweighed the problems: until
> CEDET was incorporated into Emacs core, we couldn't even dream about
> using it as infrastructure for IDEs and other similar features.

CEDET being a separate project did not stop various people elsewhere
from realizing their dream about an "IDE" according to their own ideas:

https://www.youtube.com/results?search_query=emacs+cedet

(sorry for giving a link to a google service...)

> AFAIR, there was also lots of cleanup during the move.

> But moving ELPA packages into the core is not being discussed or
> suggested.  So what happened with CEDET, even if you disagree with my
> assessment above, has little relevance to the issue at hand.

To be clear: FWIW, I agree that CEDET is great and that using real
grammars for doing various stuff in emacs is an exciting idea.  I

Re: ELPA policy

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> If that's the case, it changes my thoughts on what needs to be in core, and
>> what should be in ELPA. Until now I was thinking ELPA required Internet
>> access; but if there are parts of ELPA that "come in the box", then I'd like
>> to see more packages there.
>
> ELPA requires Internet access if you are building from the repository.
>
> My greatest fears are that packages that move from core to ELPA will
> get much less attention from the core maintenance team, which will
> eventually degrade their quality.

Org-mode, for instance, gets regular bug fix releases on GNU ELPA while
it stagnates on core. I would venture that the contribution from the
"core maintenance team" to org-mode amounts to a negative effect,
causing more hassle than it saves.

For projects with their own development team and VCS repository, having
the project copied into the Emacs core VCS makes no sense, moreover when
they have to support multiple Emacs releases.

Re: ELPA policy

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sun, 08 Nov 2015 19:26:20 +0100
> 
> For projects with their own development team and VCS repository, having
> the project copied into the Emacs core VCS makes no sense, moreover when
> they have to support multiple Emacs releases.

I could probably agree, as soon as we find a good way of getting the
"right" version of those when an Emacs tarball is being prepared (or
maybe even when Emacs is built from the repository).

But most packages in core are not of this variety, and it's them that
I worry about, not Org or Gnus.

Re: ELPA policy

[Artur Malabarba]

Eli Zaretskii <eliz@gnu.org> writes:

> My greatest fears are that packages that move from core to ELPA will
> get much less attention from the core maintenance team, which will
> eventually degrade their quality.

We could bring the Elpa and Emacs repositories closer together somehow.
Maybe as a submodule or something like that.

This way the core maintenance team would also have the Elpa sources
"right there" to do bug fixes.

Re: ELPA policy

[Eli Zaretskii]

> From: Artur Malabarba <bruce.connor.am@gmail.com>
> Cc: John Wiegley <jwiegley@gmail.com>,  emacs-devel@gnu.org,
> Date: Sun, 08 Nov 2015 19:27:08 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > My greatest fears are that packages that move from core to ELPA will
> > get much less attention from the core maintenance team, which will
> > eventually degrade their quality.
> 
> We could bring the Elpa and Emacs repositories closer together somehow.
> Maybe as a submodule or something like that.

How about just using a single repository? ;-)

Seriously, why should they be separate?  Submodules still complicate
things, compared to just having the stuff in the repo.

Re: ELPA policy

[Artur Malabarba]

Eli Zaretskii <eliz@gnu.org> writes:

> How about just using a single repository? ;-)
>
> Seriously, why should they be separate?  Submodules still complicate
> things, compared to just having the stuff in the repo.

I'm not strongly against that. But there are a couple of things that
might be annoying with that:

- Elpa commits would occasionally flood the git log

- Git operations in both repositories would become even slower than they
  are now (as they'd both increase in size).

Re: ELPA policy

[Eli Zaretskii]

> From: Artur Malabarba <bruce.connor.am@gmail.com>
> Cc: jwiegley@gmail.com,  emacs-devel@gnu.org
> Date: Sun, 08 Nov 2015 20:04:56 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > How about just using a single repository? ;-)
> >
> > Seriously, why should they be separate?  Submodules still complicate
> > things, compared to just having the stuff in the repo.
> 
> I'm not strongly against that. But there are a couple of things that
> might be annoying with that:
> 
> - Elpa commits would occasionally flood the git log

I'm sure no one actually looks at too many commits at once, only those
that are of interest.

> - Git operations in both repositories would become even slower than they
>   are now (as they'd both increase in size).

GCC is migrating to Git as we speak, even though it has something like
3-4 times more commits in its history than we do.  If they aren't
afraid, I don't think we should be.

Re: ELPA policy

[Dmitry Gutov]

On 11/08/2015 09:58 PM, Eli Zaretskii wrote:

> I'm sure no one actually looks at too many commits at once, only those
> that are of interest.

Filtering out the non-interesting commits is a hassle in itself, and I 
saying this as a emacs-diffs subscriber. With everyone committing to the 
same repo, it will become worse.

Re: ELPA policy

[Eli Zaretskii]

> Cc: jwiegley@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 8 Nov 2015 22:10:29 +0200
> 
> On 11/08/2015 09:58 PM, Eli Zaretskii wrote:
> 
> > I'm sure no one actually looks at too many commits at once, only those
> > that are of interest.
> 
> Filtering out the non-interesting commits is a hassle in itself, and I 
> saying this as a emacs-diffs subscriber. With everyone committing to the 
> same repo, it will become worse.

ELPA gets about 4 commits per day on average, so I don't think we
should worry about this aspect.

Re: ELPA policy

[Dmitry Gutov]

On 11/08/2015 10:26 PM, Eli Zaretskii wrote:

> ELPA gets about 4 commits per day on average, so I don't think we
> should worry about this aspect.

Emacs master has had ~82 commits in November, according to 'git log', 
which comes out to ~10 a day. With ELPA, this will be something like 40% 
increase?

And this is only while ELPA is relatively unpopular. If it ever grows to 
anything comparable to the size of MELPA (which we probably should 
aspire to), we will drown.

Re: ELPA policy

[Eli Zaretskii]

> Cc: bruce.connor.am@gmail.com, jwiegley@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 8 Nov 2015 22:36:00 +0200
> 
> On 11/08/2015 10:26 PM, Eli Zaretskii wrote:
> 
> > ELPA gets about 4 commits per day on average, so I don't think we
> > should worry about this aspect.
> 
> Emacs master has had ~82 commits in November, according to 'git log', 
> which comes out to ~10 a day. With ELPA, this will be something like 40% 
> increase?
> 
> And this is only while ELPA is relatively unpopular. If it ever grows to 
> anything comparable to the size of MELPA (which we probably should 
> aspire to), we will drown.

OK, then let's drop this idea.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Filtering out the non-interesting commits is a hassle in itself, and I 
  > saying this as a emacs-diffs subscriber. With everyone committing to the 
  > same repo, it will become worse.

I am sure we could set up multiple mailing lists and send each commit
to a list according to what directory it is in.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 01:16 AM, Richard Stallman wrote:

> I am sure we could set up multiple mailing lists and send each commit
> to a list according to what directory it is in.

You have a point there, and if someone were to actually do that, it 
would help with making the notifications manageable.

Still, the more happens in the repository, the harder it is to grasp or 
manage. Right now you can call 'git log' and see the most recent things 
that happened to Emacs master. Or do 'git pull', and call 'git diff' on 
the revision range that was printed out, and see all changes since your 
last update.

Unless the Emacs sources move into a subdirectory, even listing all 
commits that were made to Emacs recently, would involve the contributor 
learning magic like this: 
http://stackoverflow.com/questions/5685007/making-git-log-ignore-changes-for-certain-paths

Re: ELPA policy

[Yuri Khan]

On Mon, Nov 9, 2015 at 7:45 AM, Dmitry Gutov <dgutov@yandex.ru> wrote:

> Still, the more happens in the repository, the harder it is to grasp or
> manage. Right now you can call 'git log' and see the most recent things that
> happened to Emacs master. Or do 'git pull', and call 'git diff' on the
> revision range that was printed out, and see all changes since your last
> update.

I almost wanted to mention that you can restrict “git log” and “git
diff” to a particular path in the repository, but realized it might
start a new git subthread.

(FWIW I also think separate repositories are better, as long as
development is mostly independent across the divide.)

Re: ELPA policy

[Artur Malabarba]

John Wiegley <jwiegley@gmail.com> writes:

> If that's the case, it changes my thoughts on what needs to be in core, and
> what should be in ELPA. Until now I was thinking ELPA required Internet
> access; but if there are parts of ELPA that "come in the box", then I'd like
> to see more packages there.

I think we ought to clarify what we're talking about here.

- There are packages that are offered only on Elpa. For these you need
  an internet connection and you need to `M-x package-install' them.

- There are packages that are offered both on Elpa and Emacs. These
  are just like any other core package (no connection and no
  package-install necessary).

  The difference is that users can get access to new versions from Elpa
  without having to wait for the next Emacs release. The disadvantage
  here is code duplication, as someone needs to make sure that any bug
  fixes applied on the Emacs side also get applied on the Elpa side (and
  vice versa).

  Current examples of this are seq.el and let-alist.

- Then there's a third option. I only found this out recently while
  browsing the `externals-list' file on Elpa, and it was apparently
  implemented by Fabián in September.

  You can configure Elpa to (while building) copy the most recent
  version of an packages from the Emacs repository. IIUC, this would
  have the same effect as the previous option, except no code
  duplication. The thing is: no packages seem to be using this, and it's
  only documented in the comments of that file (though I've verified
  that there's indeed code to implement this).

Fabián, is this feature complete?

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Richard Stallman <rms@gnu.org> writes:
>
>>> I'd like to know more about this. How does the user access these bundled
>>> packages; by installing them with M-x package-install?
>
>> I think they should appear, to the user, like any other standard part of
>> Emacs.
>
> If that's the case, it changes my thoughts on what needs to be in core, and
> what should be in ELPA. Until now I was thinking ELPA required Internet
> access; but if there are parts of ELPA that "come in the box", then I'd like
> to see more packages there.

It depends on what git repository the package is in, and how the ELPA
packages are included in core.

My impression is that the latest proposal for doing this is to keep
"core ELPA" packages in the emacs repository (not the Gnu ELPA repository),
and fix the Gnu ELPA server scripts to work with that.

Then all core Emacs developers see the core ELPA packages in their
current workflow, and they are released along with Emacs.

They are also released separately to Gnu ELPA, and they appear in
`list-packages' (as other core packages do now); that's what makes them
"ELPA core packages" as opposed to "core code".

That's an elegant solution.


The alternative is to figure out how to merge a partial checkout of the
Gnu ELPA repository into an Emacs git checkout without confusing git and
make; not at all easy.

-- 
-- Stephe

Re: ELPA policy

[David Kastrup]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> I think the rule for moving new stuff out of ELPA should be whether it's
>> used by the core. That's for libraries.
>
> An exception to this rule is when a certain service (say, streams)
> should always be available, without requiring further installation of
> libraries.

What does "service" and "should always be available" mean?  If it is not
end-user functionality useful for ad-hoc code entered with M-: or its
ilk (it would appear that streams is moving there), then I don't see a
necessity for including in the core: it makes more sense to put it as a
dependency in ELPA.  That way, other ELPA packages may easily require it
without depending on an upgrade of the core Emacs.

> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same
> kinds of things we'd like to have in such a meta-library, should be in
> core.

Emacs cannot be upgraded using the package manager, so any dependencies
on Emacs core code are harder to cater for than a dependency on another
ELPA package.

-- 
David Kastrup

Re: Proposed new core library: pl.el

[Artur Malabarba]

2015-11-05 2:22 GMT+00:00 Dmitry Gutov <dgutov@yandex.ru>:
> On 11/05/2015 04:14 AM, John Wiegley wrote:
>
>> There is room for improving performance, but the API is complete enough to
>> start using it. Giving the unproven status, though, perhaps it should
>> start
>> out in ELPA, and move to core after it has solidified and gained some
>> users?
>
> I think the rule for moving new stuff out of ELPA should be whether it's
> used by the core. That's for libraries.
>
> For utility packages (ui/commands/etc), the question should be asked: if it
> works fine in ELPA, why move? Popularizing ELPA would be better, IMHO.

I think pl.el would be fine on Gelpa. I've personally never felt the
need for this in core (though I've definitely missed it when writing
some external packages).

Re: Proposed new core library: pl.el

[Ted Zlatanov]

On Wed, 04 Nov 2015 21:14:27 -0500 John Wiegley <jwiegley@gmail.com> wrote: 

JW> pl.el (standing for "parser library") is a combinator parsing library for
JW> Emacs, similar to Haskell's Parsec. You can see how it works at the following
JW> README:

JW>     https://github.com/jwiegley/emacs-pl
...
JW> There is room for improving performance, but the API is complete enough to
JW> start using it. Giving the unproven status, though, perhaps it should start
JW> out in ELPA, and move to core after it has solidified and gained some users?

My vote, after thinking about it, is to move it to the core.  That would
turn it into an Emacs facility, rather than an external package.  The
closest analogue is SMIE, which also lives in the core.

PL is a library for building other packages, so I think users don't
really care where it lives.

It's also unlikely existing packages will switch to use it, so new
packages can require it *and* a new Emacs. If a compatibility version is
needed, I would put the compatibility PL with version 24.999 in the GNU
ELPA or elsewhere, so the built-in PL with version 25.x or newer wins.
But I'd rather not maintain two versions.

Ted

Re: Proposed new core library: pl.el

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

On 5 Nov 2015 8:19 pm, "Ted Zlatanov" <tzz@lifelogs.com> wrote:
> My vote, after thinking about it, is to move it to the core.  That would
> turn it into an Emacs facility, rather than an external package.  The
> closest analogue is SMIE, which also lives in the core.
>
> PL is a library for building other packages, so I think users don't
> really care where it lives.

IMO, that's a reason to put it on Gelpa.
That way it can be used by such other packages that want to support Emacs <
25 without having to maintain duplicate code.

[-- Attachment #2: Type: text/html, Size: 692 bytes --]

Re: Proposed new core library: pl.el

[Ted Zlatanov]

On Thu, 5 Nov 2015 23:54:21 +0000 Artur Malabarba <bruce.connor.am@gmail.com> wrote: 

AM> On 5 Nov 2015 8:19 pm, "Ted Zlatanov" <tzz@lifelogs.com> wrote:
>> My vote, after thinking about it, is to move it to the core.  That would
>> turn it into an Emacs facility, rather than an external package.  The
>> closest analogue is SMIE, which also lives in the core.
>> 
>> PL is a library for building other packages, so I think users don't
>> really care where it lives.

AM> IMO, that's a reason to put it on Gelpa.
AM> That way it can be used by such other packages that want to support Emacs <
AM> 25 without having to maintain duplicate code.

I'm not convinced.

Ted

Re: Proposed new core library: pl.el

[Ted Zlatanov]

On Fri, 06 Nov 2015 10:35:47 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Thu, 5 Nov 2015 23:54:21 +0000 Artur Malabarba <bruce.connor.am@gmail.com> wrote: 
AM> On 5 Nov 2015 8:19 pm, "Ted Zlatanov" <tzz@lifelogs.com> wrote:
>>> My vote, after thinking about it, is to move it to the core.  That would
>>> turn it into an Emacs facility, rather than an external package.  The
>>> closest analogue is SMIE, which also lives in the core.
>>> 
>>> PL is a library for building other packages, so I think users don't
>>> really care where it lives.

AM> IMO, that's a reason to put it on Gelpa.
AM> That way it can be used by such other packages that want to support Emacs <
AM> 25 without having to maintain duplicate code.

TZ> I'm not convinced.

Artur, since there have been no further comments, maybe it would help if
I explained why I'm not convinced: because parsing libraries tend to be
very performance-sensitive and could take advantage of the core in ways
that most other libraries don't. They are also rare, so it makes sense
to treat them with special care instead of as just another library.

The only other example I know is SMIE, which again lives in the core.
So there are two things that would convince me in combination:

1) examples of other parsing libraries in ELPAs (GNU or otherwise)

2) examples of packages that would use PL *and* want to support Emacs 24
or older (please, let's not invent them, I want actual examples)

Thanks!
Ted

Re: Proposed new core library: pl.el

[Artur Malabarba]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Artur, since there have been no further comments, maybe it would help if
> I explained why I'm not convinced: because parsing libraries tend to be
> very performance-sensitive and could take advantage of the core in ways
> that most other libraries don't. They are also rare, so it makes sense
> to treat them with special care instead of as just another library.

Yes, this makes sense.

My comment was more about dependency libs in general, not really about parsing libs.

> The only other example I know is SMIE, which again lives in the core.
> So there are two things that would convince me in combination:
>
> 1) examples of other parsing libraries in ELPAs (GNU or otherwise)

That I'm aware there's edn.el: https://github.com/expez/edn.el

> 2) examples of packages that would use PL *and* want to support Emacs 24
> or older (please, let's not invent them, I want actual examples)

The only examples that come to mind are the ones I was involved in.
ham-mode and SX (both from Melpa) both manually parse html and would
benefit from a lib for that. (Am I correct in understanding pl helps
with html parsing?)

I'd be fine with dropping 24 support on ham-mode because it's rather
niche, but I wouldn't to drop it on SX (but then, I'm not even sure I'd
use `pl' on SX because its needs are very specific and fine-tuned).

Anyway, you've convinced me that this lib might be good in core. So I'm
OK with this now.

Re: Proposed new core library: pl.el

[John Wiegley]

>>>>> Ted Zlatanov <tzz@lifelogs.com> writes:

> Artur, since there have been no further comments, maybe it would help if I
> explained why I'm not convinced: because parsing libraries tend to be very
> performance-sensitive and could take advantage of the core in ways that most
> other libraries don't. They are also rare, so it makes sense to treat them
> with special care instead of as just another library.

Ted, what if pl.el is one of those "EPLA packages bundled with the tarball",
the way that seq.el is?

John

Re: Proposed new core library: pl.el

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 915 bytes --]

On 9 Nov 2015 10:02 pm, "John Wiegley" <jwiegley@gmail.com> wrote:
>
> >>>>> Ted Zlatanov <tzz@lifelogs.com> writes:
>
> > Artur, since there have been no further comments, maybe it would help
if I
> > explained why I'm not convinced: because parsing libraries tend to be
very
> > performance-sensitive and could take advantage of the core in ways that
most
> > other libraries don't. They are also rare, so it makes sense to treat
them
> > with special care instead of as just another library.
>
> Ted, what if pl.el is one of those "EPLA packages bundled with the
tarball",
> the way that seq.el is?

I'm fine either way now.
One thing to remember about these double packages is that they kinda need
someone who keeps an eye on both sides to ensure that any change made to
one is reflected on the other.
If a package is just going to be "maintained by emacs-devel", then it's
probably safer to keep it one-sided.

[-- Attachment #2: Type: text/html, Size: 1202 bytes --]

Re: Proposed new core library: pl.el

[John Wiegley]

>>>>> Artur Malabarba <bruce.connor.am@gmail.com> writes:

> If a package is just going to be "maintained by emacs-devel", then it's
> probably safer to keep it one-sided.

Fair enough. I think Ted was interested in helping to maintain this package,
and if so, then whichever is easiest for him. For me, having it in core is of
course easier.

John

Re: Proposed new core library: pl.el

[Ted Zlatanov]

On Mon, 09 Nov 2015 15:18:31 -0800 John Wiegley <jwiegley@gmail.com> wrote: 

>>>>>> Artur Malabarba <bruce.connor.am@gmail.com> writes:
>> If a package is just going to be "maintained by emacs-devel", then it's
>> probably safer to keep it one-sided.

JW> Fair enough. I think Ted was interested in helping to maintain this package,
JW> and if so, then whichever is easiest for him. For me, having it in core is of
JW> course easier.

I'll put it in the core when I have a chance to catch up. Before the
feature freeze, I hope.

Ted

Re: Proposed new core library: pl.el

[Ted Zlatanov]

On Mon, 09 Nov 2015 20:45:10 -0500 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Mon, 09 Nov 2015 15:18:31 -0800 John Wiegley <jwiegley@gmail.com> wrote: 
>>>>>>> Artur Malabarba <bruce.connor.am@gmail.com> writes:
>>> If a package is just going to be "maintained by emacs-devel", then it's
>>> probably safer to keep it one-sided.

JW> Fair enough. I think Ted was interested in helping to maintain this package,
JW> and if so, then whichever is easiest for him. For me, having it in core is of
JW> course easier.

TZ> I'll put it in the core when I have a chance to catch up. Before the
TZ> feature freeze, I hope.

John and others, can you please review the branch scratch/tzz/import-pl?

I've never imported a whole library before.  I did:

* split README.md into Commentary and the `pl-parse' docstring
* set Author to John Wiegley
* changed Keywords
* removed Version
* commit message is very brief, just "import library"

If that's OK, I'll push it.  Otherwise, please let me know what needs
adjustment.

Thanks
Ted

Re: Proposed new core library: pl.el

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 159 bytes --]

Ted Zlatanov <tzz@lifelogs.com> writes:

> * removed Version

I don't think you had to remove the version, many packages in Emacs have
a version number.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: Proposed new core library: pl.el

[Ted Zlatanov]

On Wed, 11 Nov 2015 16:08:24 +0100 Nicolas Petton <nicolas@petton.fr> wrote: 

NP> Ted Zlatanov <tzz@lifelogs.com> writes:
>> * removed Version

NP> I don't think you had to remove the version, many packages in Emacs have
NP> a version number.

I want the version in the core to be pinned to the Emacs version, which
will be automatic this way IIUC (e.g. 25.1).

Meanwhile the version in Github can stay at 1.x in MELPA or GNU ELPA for
pre-25 users, until it's killed off.  Did I plan this correctly?

Ted

Re: Proposed new core library: pl.el

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 899 bytes --]

On 11 Nov 2015 3:28 pm, "Ted Zlatanov" <tzz@lifelogs.com> wrote:
>
> On Wed, 11 Nov 2015 16:08:24 +0100 Nicolas Petton <nicolas@petton.fr>
wrote:
>
> NP> Ted Zlatanov <tzz@lifelogs.com> writes:
> >> * removed Version
>
> NP> I don't think you had to remove the version, many packages in Emacs
have
> NP> a version number.
>
> I want the version in the core to be pinned to the Emacs version, which
> will be automatic this way IIUC (e.g. 25.1).
>
> Meanwhile the version in Github can stay at 1.x in MELPA or GNU ELPA for
> pre-25 users, until it's killed off.  Did I plan this correctly?

Not quite. If a file inside emacs has no version number it is not
considered a package at all. So a user of emacs 25 will be able to use pl
just fine, but package.el will not see it and therfore will gladly shadow
it with pl installed from *elpa.

This is only relevant if pl is actually on one of the elpas.

[-- Attachment #2: Type: text/html, Size: 1244 bytes --]

Re: Proposed new core library: pl.el

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

On 11 Nov 2015 5:27 pm, "Artur Malabarba" <bruce.connor.am@gmail.com> wrote:
>
>
> On 11 Nov 2015 3:28 pm, "Ted Zlatanov" <tzz@lifelogs.com> wrote:
> >
> > On Wed, 11 Nov 2015 16:08:24 +0100 Nicolas Petton <nicolas@petton.fr>
wrote:
> >
> > NP> Ted Zlatanov <tzz@lifelogs.com> writes:
> > >> * removed Version
> >
> > NP> I don't think you had to remove the version, many packages in Emacs
have
> > NP> a version number.
> >
> > I want the version in the core to be pinned to the Emacs version, which
> > will be automatic this way IIUC (e.g. 25.1).
> >
> > Meanwhile the version in Github can stay at 1.x in MELPA or GNU ELPA for
> > pre-25 users, until it's killed off.  Did I plan this correctly?

> This is only relevant if pl is actually on one of the elpas.

I just checked and it's not on Melpa, so it's fine to remove the number.

[-- Attachment #2: Type: text/html, Size: 1317 bytes --]

Re: Proposed new core library: pl.el

[Ted Zlatanov]

After discussing with John, we'll put pl.el in the "tarball ELPA."
Please disregard my request for review, and sorry for the noise.

Ted

Re: Proposed new core library: pl.el

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It was good to remove the version number.

Most of the Lisp files in Emacs don't need version numbers of their
own, and when there is one, people feel obliged to worry about when to
increment it.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.