ELPA policy

ELPA policy

[Julien Danjou]

Hi,

This topic has been recently brought by Lars, Ted and me on the gnus
mailing list. I've been asked to move this discussion here for
clarification.

I am currently worried by the Emacs ELPA archive and what will be that
policy. As Lars pointed out, until now, packages have been added over
the years into Emacs trunk, maintained by a extended set of skilled
hands, assuring good quality in (almost :-)) every place of every
packages furnished with Emacs.

Now, with the introduction of ELPA I'm worried about a lot of things.

- Who will be able to upload to ELPA?
- Who will fix the packages' bugs?
- Who will assure there's no regression for Emacs 24.1 user when people
  will starts uploading packages using 24.2 only function?
- Who will assure there's no regression at all?
- Who will assure there's no really bad things uploaded?
- Will all new packages go to ELPA, or with some still go to Emacs
  trunk? And if some can go to the trunk, upon which rules?
- Will all/most of the current packages be moved to ELPA?

There's probably a lot more question outstanding. I've the feeling ELPA
is adding a second class citizenship for packages, and I really do not
like that, at least in its current form.

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

Re: ELPA policy

[Chong Yidong]

Julien Danjou <julien@danjou.info> writes:

> I am currently worried by the Emacs ELPA archive and what will be that
> policy. As Lars pointed out, until now, packages have been added over
> the years into Emacs trunk, maintained by a extended set of skilled
> hands, assuring good quality in (almost :-)) every place of every
> packages furnished with Emacs.

Emacs contains many packages that are maintained "externally".  While
Emacs developers might make small changes to the in-tree version, most
development is done upstream and periodically merged in.  Those upstream
maintainers handle bugfixes and ensure compatibility among Emacs
versions.

One good reason to put a package in elpa.gnu.org, rather than in the
Emacs tarball, is if it is likely to benefit only a small segment of
Emacs users (even if it's of tremendous usefulness to that segment).
Especially, but not necessarily, if a package is large and complex, like
Auctex and Muse.

There are other good reasons too, e.g. packages that we want to merge
into Emacs core in the future, but not yet (for whatever reason).

(Incidentally, there are also some packages in Emacs that might be
usefully moved out into elpa.gnu.org, e.g. since they are so rarely
used.  We could also move some of the files in obsolete/ into a
subrepository, e.g. elpa.gnu.org/packages/obsolete)

(Also, as stated before, the FSF's policy is that for a package to be
listed on elpa.gnu.org its copyright must be assigned, in the exact same
way as if it's included in Emacs core.)

> - Who will be able to upload to ELPA?
> - Who will assure there's no really bad things uploaded?

The way it's currently set up is that only a couple of people can upload
to ELPA; package maintainers, when they release a new version, should
email me (or Ted) to get the package uploaded.  The system is still a
work in progress; we might set up a more elaborate "staging area" system
in the future.  (Such a system would still involve a human component, of
course, to reduce the possibility of malicious uploads.)  I will add a
page to the elpa.gnu.org webpage explaining this.

elpa.gnu.org policy (was: ELPA policy)

[Ted Zlatanov]

On Mon, 15 Nov 2010 16:40:39 +0100 Julien Danjou <julien@danjou.info> wrote: 

JD> - Who will assure there's no regression for Emacs 24.1 user when people
JD>   will starts uploading packages using 24.2 only function?

JD> - Who will assure there's no regression at all?

(I've realized ELPA is ambiguous because there's an ELPA archive at
http://tromey.com/elpa/, so I clarified the topic)

Maybe elpa.gnu.org should publish a repository for each Emacs release,
enabled by default in that release:

All 24.x releases:
elpa.gnu.org/packages-24

24.1 only:
elpa.gnu.org/packages-24.1

24.2 only:
elpa.gnu.org/packages-24.2

...

All releases:
elpa.gnu.org/packages

I think we can handle that with VCS branches.

Through consistent versioning, the newest package in any of the enabled
repositories should win.  So consistent versioning should probably be an
absolute requirement for elpa.gnu.org inclusion, together with the
copyright assignment.

Ted

Re: elpa.gnu.org policy

[Lluís]

Ted Zlatanov writes:

> On Mon, 15 Nov 2010 16:40:39 +0100 Julien Danjou <julien@danjou.info> wrote: 
JD> - Who will assure there's no regression for Emacs 24.1 user when people
JD> will starts uploading packages using 24.2 only function?

JD> - Who will assure there's no regression at all?

[...]
> Maybe elpa.gnu.org should publish a repository for each Emacs release,
> enabled by default in that release:

> All 24.x releases:
> elpa.gnu.org/packages-24

> 24.1 only:
> elpa.gnu.org/packages-24.1

> 24.2 only:
> elpa.gnu.org/packages-24.2

> ...

> All releases:
> elpa.gnu.org/packages

But package.el supports version dependencies, right? Then package
developers should set that accordingly (both lower and upper version
bounds, if necessary).

Using version-specific repositories requires just that same work of
checking compatibility.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: elpa.gnu.org policy

[Ted Zlatanov]

On Mon, 15 Nov 2010 19:01:14 +0100 Lluís <xscript@gmx.net> wrote: 

L> Ted Zlatanov writes:
>> Maybe elpa.gnu.org should publish a repository for each Emacs release,
>> enabled by default in that release:

>> All 24.x releases:
>> elpa.gnu.org/packages-24

>> 24.1 only:
>> elpa.gnu.org/packages-24.1

>> 24.2 only:
>> elpa.gnu.org/packages-24.2

>> ...

>> All releases:
>> elpa.gnu.org/packages

L> But package.el supports version dependencies, right? Then package
L> developers should set that accordingly (both lower and upper version
L> bounds, if necessary).

L> Using version-specific repositories requires just that same work of
L> checking compatibility.

(these are just my ideas, not necessarily the solution)

Package developers tell us some of this, but elpa.gnu.org needs to have
a way of freezing package versions.  Otherwise any new package version
has the potential to break everyone with an older Emacs version.  Just
as with the regular Emacs, we can backport fixes but at least we're
guaranteed that we won't have surprises due to an upgrade.

So packages will start in the global repository.  But when Emacs changes
or the maintainer tells us or we get bug reports, we'll look for
affected packages and branch them accordingly.

Ted

Re: ELPA policy

[Tom Tromey]

>>>>> "Julien" == Julien Danjou <julien@danjou.info> writes:

Julien> - Who will assure there's no regression for Emacs 24.1 user when people
Julien>   will starts uploading packages using 24.2 only function?

You could try making it so uploads require a warning-free byte-compile
on various Emacs versions first.

Packages can require a minimum Emacs version, if the author knows that
this is needed.

Julien> - Who will assure there's no regression at all?

You can't catch all the possible bugs.
I've had ok results on my site just forwarding reports to the package
maintainer for fixing.

Julien> There's probably a lot more question outstanding. I've the feeling ELPA
Julien> is adding a second class citizenship for packages, and I really do not
Julien> like that, at least in its current form.

My view leans more towards package inclusion maximalism -- put
everything in Emacs, but also do more frequent releases of some included
packages via ELPA.

Tom

Re: ELPA policy

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> One good reason to put a package in elpa.gnu.org, rather than in the
> Emacs tarball, is if it is likely to benefit only a small segment of
> Emacs users (even if it's of tremendous usefulness to that segment).
> Especially, but not necessarily, if a package is large and complex, like
> Auctex and Muse.
>
> There are other good reasons too, e.g. packages that we want to merge
> into Emacs core in the future, but not yet (for whatever reason).

The reason that I raised the question now was that I was trying to
figure out how to do reasonable colour distance calculations for
rendering foreground colours in shr.el.  rainbow-mode (for colourising
colour specs in, for instance, CSS files) was brought up, that had
similar needs, and its calculations in the same area might possibly be
reusable by shr.el.

But apparently rainbow-mode went to ELPA instead of into Emacs, even
though it's small, it's clearly written, and it seems to be generally
useful for anybody editing stuff like CSS or .Xdefaults files or
anything.  So I wondered why that was...

> The way it's currently set up is that only a couple of people can upload
> to ELPA; package maintainers, when they release a new version, should
> email me (or Ted) to get the package uploaded.

If ELPA is supposed to be a repository of really special-needs code, I
think it's a good idea.  It would be nice to have an automatic way to
download, say, one of the Emacs-based mp3 players.  But if it's going to
carry stuff that people do really want to have in the Emacs
distribution, then I think it's counter-productive.

Stuff that is in Emacs gets loving care.  When url.el needs tweaking
(for instance), people step up to the plate and makes sure that it
works.  Things that live in a package repository bit-rot at an alarming
rate.  One of the great things about Emacs is that when you "apt-get
install emacs", you get a fully-featured system where you can be
reasonably sure that things work.  Over-modularisation can be costly in
the long run.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: ELPA policy

[Stefan Monnier]

> This topic has been recently brought by Lars, Ted and me on the gnus
> mailing list. I've been asked to move this discussion here for
> clarification.

As mentioned Chong, it's still "work in progress" from this point
of view.  But since your questions use the future tense, I'll try to
reply based on what I think should happen in the future.

> - Who will be able to upload to ELPA?

Depends: new packages or updates to existing packages?
For new packages, I'd expect only a few people to have such rights, but
for updates, I'd expect something like "anybody with access to the Bzr
repository".  After all, if they can screw with the main Emacs codebase,
why not with the ELPA packages.

> - Who will fix the packages' bugs?

The upstream maintainers, I'd expect.  That's one of the reasons to have
packages outside of Emacs's tree.

> - Who will assure there's no regression for Emacs 24.1 user when people
>   will starts uploading packages using 24.2 only function?

The upstream maintainer, for the same reason.

> - Who will assure there's no regression at all?

In which sense?  I'd expect the upstream maintainer to take some role
here, but admittedly, since those packages are easily available in
a central place, it's more likely that Emacs maintainers will pay
attention to them when introducing changes to the core.

> - Who will assure there's no really bad things uploaded?

The same people who currently assure that there's no really bad things
installed in the Emacs trunk.

> - Will all new packages go to ELPA, or with some still go to Emacs
>   trunk? And if some can go to the trunk, upon which rules?

We don't have rules for it, right now.  Here are some considerations, on
top of what Chong mentioned:
- in order to keep Emacs maintainable, we'd rather have packages in
  ELPA, all things being equal.
- packages of general usefulness (e.g. libraries) are good candidates
  for inclusion in Emacs, to reduce dependencies among ELPA packages.
  
> - Will all/most of the current packages be moved to ELPA?

Definitely not (in the foreseeable future).

> There's probably a lot more question outstanding. I've the feeling ELPA
> is adding a second class citizenship for packages, and I really do not
> like that, at least in its current form.

We currently have two kinds of packages: bundled and unbundled.  So ELPA
is about adding a third kind in-between, with properties such as
"smooth integration", "ease of installation", ... to be halfway between
the other two.


        Stefan

Re: ELPA policy

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> For new packages, I'd expect only a few people to have such rights,
> but for updates, I'd expect something like "anybody with access to the
> Bzr repository".  After all, if they can screw with the main Emacs
> codebase, why not with the ELPA packages.

One difference, though, is that screwing with the main Emacs codebase
affects only those using the development version of Emacs, and we have
mechanisms like the diffs mailing list for problems to be easily
spotted.  After Emacs 24 is released, by screwing with elpa.gnu.org you
can immediately affect users of deployed stable Emacs versions.

So, we need to be a bit more paranoid for elpa.gnu.org than for our main
repository.

I agree, though, that it would be nice for Emacs developers to easily
edit packages in elpa.gnu.org without going through an onerous package
upload process.  I'm not sure how to set this up, though maybe the way
the Org daily builds are handled can be used as a starting point.

Re: elpa.gnu.org policy

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Package developers tell us some of this, but elpa.gnu.org needs to have
> a way of freezing package versions.  Otherwise any new package version
> has the potential to break everyone with an older Emacs version.  Just
> as with the regular Emacs, we can backport fixes but at least we're
> guaranteed that we won't have surprises due to an upgrade.
>
> So packages will start in the global repository.  But when Emacs changes
> or the maintainer tells us or we get bug reports, we'll look for
> affected packages and branch them accordingly.

Maintaining a separate repository for every Emacs release sounds like a
lot of work.  Currently, it's up to maintainers of upstream/external
packages ensure backward compatibility with older Emacs versions,
e.g. by introducing compatibility functions.  Obviously this system
doesn't work perfectly, but it doesn't seem like we have the manpower
for anything more elaborate.

Re: ELPA policy

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> figure out how to do reasonable colour distance calculations for
> rendering foreground colours in shr.el.  rainbow-mode (for colourising
> colour specs in, for instance, CSS files) was brought up, that had
> similar needs, and its calculations in the same area might possibly be
> reusable by shr.el.
>
> But apparently rainbow-mode went to ELPA instead of into Emacs, even
> though it's small, it's clearly written, and it seems to be generally
> useful for anybody editing stuff like CSS or .Xdefaults files or
> anything.  So I wondered why that was...

Rainbow-mode, as presented, is a tool for colorizing strings in Emacs
buffers, the kind of neat hack that doesn't really need to be in Emacs.
From your description, it sounds like parts of it can be usefully
refactored into a general color-manipulation library, in which case we
could include that part in Emacs (preferably, using a better prefix than
`rainbow').

> If ELPA is supposed to be a repository of really special-needs code, I
> think it's a good idea.  It would be nice to have an automatic way to
> download, say, one of the Emacs-based mp3 players.  But if it's going
> to carry stuff that people do really want to have in the Emacs
> distribution, then I think it's counter-productive.  Stuff that is in
> Emacs gets loving care.  Things that live in a package repository
> bit-rot at an alarming rate.

Well, obviously it's going to be a judgement call, analogous to the
debates about what goes into distribution DVDs (with programs like Emacs
nowadays frequently not making the cut :-P).  The level of bit-rot is
dependent on the upstream package maintainers; I don't think you can
argue that packages like Auctex (or SLIME or ESS, which are not
packaged) have bit-rotted.

Some packages integrated into Emacs are no longer active upstream but
are still tinkered with by Emacs developers, but what's really happening
here is that we have effectively forked the project.  If a package on
elpa.gnu.org ends up being abandoned, there's little difference in how
we could solve the problem, except the fork would be more explicit than
implicit.

Re: ELPA policy

[Edward O'Connor]

> The reason that I raised the question now was that I was trying to
> figure out how to do reasonable colour distance calculations for
> rendering foreground colours in shr.el.  rainbow-mode (for colourising
> colour specs in, for instance, CSS files) was brought up, that had
> similar needs, and its calculations in the same area might possibly be
> reusable by shr.el.

Maybe generalizing `tty-color-approximate' is the right thing to do?


Ted

rainbow-mode (was: ELPA policy)

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Rainbow-mode, as presented, is a tool for colorizing strings in Emacs
> buffers, the kind of neat hack that doesn't really need to be in Emacs.

So it was rejected because you didn't think it was generally useful
enough, basically?

That's a judgement call, of course, but I think it sounds quite useful.
If you're fiddling with colours in CSS files, for instance, it would
speed up the editing/reload process -- at least the way I do it, which
is "er, this should be a bluish colour, so I'll try #404090", *save*,
*reload*, "no, darker", etc.

Just seeing what the colour would be at once would be really neat.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org policy

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Maintaining a separate repository for every Emacs release sounds like a
> lot of work.  Currently, it's up to maintainers of upstream/external
> packages ensure backward compatibility with older Emacs versions,
> e.g. by introducing compatibility functions.  Obviously this system
> doesn't work perfectly, but it doesn't seem like we have the manpower
> for anything more elaborate.

Maybe I'm misunderstanding something, but it sounds to me like you're
saying that if a package is in ELPA, then the maintainer has
(implicitly) committed themselves to support older versions of Emacs in,
sort of, perpetuity.  (Well, for a long time.)

Otherwise people using older versions of Emacs, and who are relying on
ELPA for some of the packages, will suddenly have partially
non-functioning Emacsen.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

RE: rainbow-mode (was: ELPA policy)

[Drew Adams]

> So it was rejected because you didn't think it was generally useful
> enough, basically? That's a judgement call, of course, but I think
> it sounds quite useful.
>
> If you're fiddling with colours in CSS files, for instance, it would
> speed up the editing/reload process -- at least the way I do it, which
> is "er, this should be a bluish colour, so I'll try #404090", *save*,
> *reload*, "no, darker", etc.
> 
> Just seeing what the colour would be at once would be really neat.

http://www.emacswiki.org/emacs/hexrgb.el

http://www.emacswiki.org/emacs/palette.el

Wrt "no, darker", etc., there are several WYSIWYG ways to incrementally adjust
colors.

http://www.emacswiki.org/emacs/SetColor

http://www.emacswiki.org/emacs/ColorPalette

http://www.emacswiki.org/emacs/DoReMi

Re: rainbow-mode

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> So it was rejected because you didn't think it was generally useful
> enough, basically?

"Rejected" is obviously the wrong word; as Stefan says, all things being
equal, it is preferable to put new non-infrastructure packages into
elpa.gnu.org.

> That's a judgement call, of course, but I think it sounds quite useful.
> If you're fiddling with colours in CSS files, for instance, it would
> speed up the editing/reload process -- at least the way I do it, which
> is "er, this should be a bluish colour, so I'll try #404090", *save*,
> *reload*, "no, darker", etc.
>
> Just seeing what the colour would be at once would be really neat.

You could use the same argument for including all the other packages in
elpa.gnu.org in the tarball.

By the way, if there is real demand for a proper color selection tool,
it ought to be trivial to write an Elisp binding for the GTK color
selector.

Re: ELPA policy

[Ted Zlatanov]

On Mon, 15 Nov 2010 15:12:15 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> For new packages, I'd expect only a few people to have such rights,
>> but for updates, I'd expect something like "anybody with access to the
>> Bzr repository".  After all, if they can screw with the main Emacs
>> codebase, why not with the ELPA packages.

CY> One difference, though, is that screwing with the main Emacs codebase
CY> affects only those using the development version of Emacs, and we have
CY> mechanisms like the diffs mailing list for problems to be easily
CY> spotted.  After Emacs 24 is released, by screwing with elpa.gnu.org you
CY> can immediately affect users of deployed stable Emacs versions.

CY> So, we need to be a bit more paranoid for elpa.gnu.org than for our main
CY> repository.

This is one good reason to keep a separate repository per major Emacs
version, at least.  Then you can fix packages without so much fear.

CY> I agree, though, that it would be nice for Emacs developers to easily
CY> edit packages in elpa.gnu.org without going through an onerous package
CY> upload process.  I'm not sure how to set this up, though maybe the way
CY> the Org daily builds are handled can be used as a starting point.

I hope we make it easy to submit changes but harder to publish them.
See below for my staging suggestion.

CY> Maintaining a separate repository for every Emacs release sounds like a
CY> lot of work.  Currently, it's up to maintainers of upstream/external
CY> packages ensure backward compatibility with older Emacs versions,
CY> e.g. by introducing compatibility functions.  Obviously this system
CY> doesn't work perfectly, but it doesn't seem like we have the manpower
CY> for anything more elaborate.

Setting it up is really not hard, even if we don't use it.  Just add
"packages-MAJOR" and "packages-MAJOR.MINOR" to the default "packages"
repository on startup.  Those can be empty but when we need something in
them (and we will, I promise you), they'll Just Work.

The dev checkout version should also have "packages-dev" or some such
that's only enabled if you make a dev build.

All this is not hard if we use VCS branches.  It's a 1-to-1 mapping to
the Emacs repo's branches.  In fact the elpa.gnu.org repo could mirror a
subdirectory of the Emacs repo, which would solve the staging problem
(developers would just stage to the Emacs repo and the elpa.gnu.org
maintainers would publish from there).

I think the alternative approach, keeping it really simple now, will
require more manpower and drain more time long-term.

Ted

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> You could use the same argument for including all the other packages in
> elpa.gnu.org in the tarball.

Yes, that's true.  "This would be a good package to have in Emacs" is an
argument you could use for everything.  So would "this package is not
necessary to have in Emacs".

> By the way, if there is real demand for a proper color selection tool,
> it ought to be trivial to write an Elisp binding for the GTK color
> selector.

That would be neat, too, but I'd prefer just to see and edit the colour
specs directly in the CSS file.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org policy

[Tom Tromey]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

Lars> Maybe I'm misunderstanding something, but it sounds to me like you're
Lars> saying that if a package is in ELPA, then the maintainer has
Lars> (implicitly) committed themselves to support older versions of Emacs in,
Lars> sort of, perpetuity.  (Well, for a long time.)

A package can have a minimum emacs version dependency.
Users with an older emacs won't be able to install that package.
I think this will be ok in practice as long as most maintainers don't
constantly bump their minimum requirement to the absolute newest.

Tom

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

"Drew Adams" <drew.adams@oracle.com> writes:

> http://www.emacswiki.org/emacs/hexrgb.el
>
> http://www.emacswiki.org/emacs/palette.el
>
> Wrt "no, darker", etc., there are several WYSIWYG ways to incrementally adjust
> colors.
>
> http://www.emacswiki.org/emacs/SetColor
>
> http://www.emacswiki.org/emacs/ColorPalette
>
> http://www.emacswiki.org/emacs/DoReMi

I'm not quite sure what you mean to say here.  Is it "this is something
that so many people want in Emacs that it's been partially reimplemented
half a dozen times"?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: ELPA policy

[Glenn Morris]

Tom Tromey wrote:

> You could try making it so uploads require a warning-free byte-compile
> on various Emacs versions first.

Hah, we can't even get this for Emacs proper...

Re: elpa.gnu.org policy

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Maybe I'm misunderstanding something, but it sounds to me like you're
> saying that if a package is in ELPA, then the maintainer has
> (implicitly) committed themselves to support older versions of Emacs in,
> sort of, perpetuity.  (Well, for a long time.)

You mean, like Gnus? ;-)

Re: elpa.gnu.org policy

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> You mean, like Gnus? ;-)

If I had my way, I'd only target bzr Emacs.  :-)  It'd cut the amount of
work in half.  

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: rainbow-mode

[Glenn Morris]

Chong Yidong wrote:

> as Stefan says, all things being equal, it is preferable to put new
> non-infrastructure packages into elpa.gnu.org.

This is a pretty big change in policy, IMO. Who's going to maintain
these things? Where do the bug reports go? If it were in Emacs proper,
I could and would (maybe) be fixing any bugs in it, improving docs,
etc. If it's in elpa.gnu.org, I'm unlikely to ever even look at the
code. And if I did and wanted to change something, all I have is a
tarfile (no?), the repository lives somewhere else, and I'm unlikely
to bother seeking it out, submitting a patch, etc.

I tried the elpa interface and it seems very nifty, but I do share the
concerns expressed about what it all means in practice for the
maintenance of the associated packages. As an "easy way to get a
package that would otherwise not be in Emacs" (eg AUCTeX), it seems
really nice; for farming out things that otherwise _would_ be in
Emacs proper, not so much.

Some of these issues would go away if these "non-infrastructure"
packages (not AUCTeX etc, but rather the things that would have been
added to Emacs before elpa) were hosted in a central Savannah bzr
repository (or possible as a subdirectory in the existing Emacs
repository that is excluded from the tarfiles; I'm not sure if the
number of files in the Bzr version of Emacs matters very much), so
that the Emacs developers can and do feel able to help maintain them.
From my point of, the ideal thing would be something like a separate
top-level "elpa/" directory in the normal repo, not compiled by
default, that I could compile with `make elpa' or somesuch.

Then putting something on elpa doesn't mean relegating it to being a
second-class citizen.

Re: ELPA policy

[Glenn Morris]

Chong Yidong wrote:

> (Incidentally, there are also some packages in Emacs that might be
> usefully moved out into elpa.gnu.org, e.g. since they are so rarely
> used.  We could also move some of the files in obsolete/ into a
> subrepository, e.g. elpa.gnu.org/packages/obsolete)

I don't see how that would work. If, as soon as something becomes
obsolete, you move it to elpa, you've defeated the entire point of
obsolescence. Emacs goes from "you can assume we have function foo" to
"you can't rely on us having function foo" (unless your users have
added an optional package).

If on the other hand you move it to elpa at the point it which it
would normally be deleted altogether, then you're just prolonging its
life, when the whole point of making things obsolete is to migrate
people away from them.

And to move it at some intermediate stage seems a little pointless.

> (Also, as stated before, the FSF's policy is that for a package to be
> listed on elpa.gnu.org its copyright must be assigned, in the exact same
> way as if it's included in Emacs core.)

I was surprised to see AUCTeX there, since I thought its copyright was
notoriously hard to assign. At a quick glance, tex-fptex.el, tex-jp.el
have non-FSF copyrights in the headers, and there is no information
for the images/ directory. The texinfo source of the manuals and pdfs
also appears to be missing (maybe this is by design).

Re: elpa.gnu.org policy

[Glenn Morris]

Lars Magne Ingebrigtsen wrote:

> If I had my way, I'd only target bzr Emacs.  :-)  It'd cut the amount of
> work in half.  

Maybe try offering people the choice between the current situation;
and one in which you only support the latest Emacs, and have twice as
much time to develop new stuff...

(I'm glad Gnus finally dropped Emacs 21, at least.)

Re: rainbow-mode

[Chong Yidong]

Glenn Morris <rgm@gnu.org> writes:

> I tried the elpa interface and it seems very nifty, but I do share the
> concerns expressed about what it all means in practice for the
> maintenance of the associated packages. As an "easy way to get a
> package that would otherwise not be in Emacs" (eg AUCTeX), it seems
> really nice; for farming out things that otherwise _would_ be in
> Emacs proper, not so much.

I hear you.  Let's try to clarify the discussion.  There are two classes
of packages that could end up on elpa.gnu.org:

  (i)  Small packages.  Individually, each could be included in the
       tarball, but when the number of such small packages goes into the
       hundreds (*), this eventually becomes problematic.

  (ii) Big packages, like AUCTeX, big enough that including it in Emacs
       would be a momentous decision.

Based on the discussion so far, I think we should give all Emacs hackers
some kind of access to modify the packages.  (For instance, we could set
up a separate "dev-only" version of the package repository, which is
occassionally merged into the "public usage" repository.)

The problem is that if we Emacs hackers start tinkering with everything,
the situation could be headache-inducing.  At first glance, it seems
like we should only feel free to modify the small packages, while
leaving the big ones alone.  But there have made many tweaks made to big
packages built into Emacs, such as CEDET.  These tweaks have to be dealt
with when upstream is merged in, but that's life; they have been very
important in making sure that Emacs works as a coherent whole.

So my question is, do we really want to deal the same issues for AUCTeX
and Muse and Company mode and whatever other big package ends up on
elpa.gnu.org?  This is not a rhetorical question; I don't know the
answer.

Re: rainbow-mode

[Stephen J. Turnbull]

Lars Magne Ingebrigtsen writes:

 > I'm not quite sure what you mean to say here.  Is it "this is something
 > that so many people want in Emacs that it's been partially reimplemented
 > half a dozen times"?

IIRC, the most recent discussion concluded that none of the half-
implemented modes is good enough, especially given the nice features
in the other half-implemented modes, and nobody stepped up to the
plate to merge them, document the result, and make sure the result
conforms to coding standards etc.

Re: rainbow-mode

[Stefan Monnier]

> these things? Where do the bug reports go? If it were in Emacs proper,
> I could and would (maybe) be fixing any bugs in it, improving docs,
> etc. If it's in elpa.gnu.org, I'm unlikely to ever even look at the
> code. And if I did and wanted to change something, all I have is a
> tarfile (no?), the repository lives somewhere else, and I'm unlikely
> to bother seeking it out, submitting a patch, etc.

That's a problem with the current setup, indeed.  I think we should be
moving towards a Bzr "packages" branch which we could checkout alongside
Emacs and edit easily to fix bugs.

There's a fair bit of work left to do to get to that point, tho.
Part of the problem here is how to decouple edits from uploads, how to
sync local changes with the upstream maintainer, etc...


        Stefan

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> IIRC, the most recent discussion concluded that none of the half-
> implemented modes is good enough, especially given the nice features
> in the other half-implemented modes, and nobody stepped up to the
> plate to merge them, document the result, and make sure the result
> conforms to coding standards etc.

Again, I think that illustrates my point.

I think experience shows that packages that are in Emacs do get hacked
on to add features that people feel are missing.  Small packages that
live outside of Emacs don't get that treatment, as a norm.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org policy

[Lars Magne Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

> Maybe try offering people the choice between the current situation;
> and one in which you only support the latest Emacs, and have twice as
> much time to develop new stuff...

Yeah...  well, back in the 90s, it made sense to support all the
different Emacs versions, because the development direction of the main
Emacs was so, er, interesting.  Ahem.  So there really were reasons for
people not to just use the latest Emacs.  But I don't really see that as
much of an issue any more...

> (I'm glad Gnus finally dropped Emacs 21, at least.)

Me too.  I wish it was possible to just write towards Emacs 24 and
pretend that nothing else existed, but just providing compat functions
for older/different Emacs versions, but then you'd have competing compat
functions for, well, everything.  So it's not doable.  *sigh*

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

elpa.gnu.org repository sync with Emacs (was: rainbow-mode)

[Ted Zlatanov]

On Mon, 15 Nov 2010 22:15:33 -0500 Glenn Morris <rgm@gnu.org> wrote: 

GM> I tried the elpa interface and it seems very nifty, but I do share the
GM> concerns expressed about what it all means in practice for the
GM> maintenance of the associated packages. As an "easy way to get a
GM> package that would otherwise not be in Emacs" (eg AUCTeX), it seems
GM> really nice; for farming out things that otherwise _would_ be in
GM> Emacs proper, not so much.

GM> Some of these issues would go away if these "non-infrastructure"
GM> packages (not AUCTeX etc, but rather the things that would have been
GM> added to Emacs before elpa) were hosted in a central Savannah bzr
GM> repository (or possible as a subdirectory in the existing Emacs
GM> repository that is excluded from the tarfiles; I'm not sure if the
GM> number of files in the Bzr version of Emacs matters very much), so
GM> that the Emacs developers can and do feel able to help maintain them.
GM> From my point of, the ideal thing would be something like a separate
GM> top-level "elpa/" directory in the normal repo, not compiled by
GM> default, that I could compile with `make elpa' or somesuch.

GM> Then putting something on elpa doesn't mean relegating it to being a
GM> second-class citizen.

On Tue, 16 Nov 2010 00:55:16 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> That's a problem with the current setup, indeed.  I think we should be
SM> moving towards a Bzr "packages" branch which we could checkout alongside
SM> Emacs and edit easily to fix bugs.

SM> There's a fair bit of work left to do to get to that point, tho.
SM> Part of the problem here is how to decouple edits from uploads, how to
SM> sync local changes with the upstream maintainer, etc...

I'll try to merge Glenn's comments, your comments, and mine into a plan:

1) add elpa/ directory to main Emacs repo (as a branch or subdirectory;
my vote is for a subdirectory that's not bundled or compiled because it
will get branched together with Emacs itself).  Make it available to a
dev checkout of Emacs as a file:/// URL (so it can be tested easily).

2) let the usual Emacs hackers access elpa/* normally

3) mirror elpa/ to a repo on elpa.gnu.org daily after reviewing the
changes (so it's a supervised pull, not automatic).  Allow admins to
trigger this from the web site.

4) Set up a deploy process of the elpa.gnu.org repo to the HTML tree
(with one package repository per major version as I suggested, plus a
"dev" package repository and an "all" package repository).  This can be
deployed automatically and manually.  Allow admins to trigger
deployments from the web site.

5) set up specific third-party packages to be fetched into the "all"
package repository daily on top of the deployment.  AUCTeX, BBDB,
etc. would be good candidates for this.

This lets us bundle Emacs-local changes to third-party packages into
clear diffs we can send back upstream.  It also separates "commit
something into the package repository" from "deploy the package
repository to the world."

Ted

Re: rainbow-mode

[Ted Zlatanov]

On Tue, 16 Nov 2010 14:25:08 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> "Stephen J. Turnbull" <stephen@xemacs.org> writes:
>> IIRC, the most recent discussion concluded that none of the half-
>> implemented modes is good enough, especially given the nice features
>> in the other half-implemented modes, and nobody stepped up to the
>> plate to merge them, document the result, and make sure the result
>> conforms to coding standards etc.

LMI> Again, I think that illustrates my point.

LMI> I think experience shows that packages that are in Emacs do get hacked
LMI> on to add features that people feel are missing.  Small packages that
LMI> live outside of Emacs don't get that treatment, as a norm.

So let's check everything on the Emacs Wiki into Emacs.  By your theory
that will get the Emacs developers to work on all of it.

I'm trying to say that:

- there's plenty of bit rot in Emacs itself

- this is a manpower problem, not a segregation problem

- new packages don't add complexity linearly

Ted

compat unification (was: elpa.gnu.org policy)

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Me too.  I wish it was possible to just write towards Emacs 24 and
> pretend that nothing else existed, but just providing compat functions
> for older/different Emacs versions, but then you'd have competing compat
> functions for, well, everything.  So it's not doable.  *sigh*

Or is it?

I'm just brainstorming here.  (Or barnstorming.  Freebasing.
Something.)

What if...  there was an ems-compat function in bzr Emacs.  It would be
a, sort of, "official" current-Emacs-to-everything compat layer.  It
would probably be totally crufty, but it'd basically be along the lines
of

(unless (fboundp 'put-image)
  (cond ((featurep 'xemacs)
         (defun put-image (...) ...))
        (t
         (defalias 'put-image 'ignore))))

Any packages that are external/internal (like Gnus) would include a copy
of this file in its distribution, and would load it.  Emacs 24 itself
would never load it, but would just have it in its distribution as a
central clearinghouse for the compat functions, and any external
packages could just pull it in as needed.

Does it sound really moronic?
         
-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> So let's check everything on the Emacs Wiki into Emacs.  By your theory
> that will get the Emacs developers to work on all of it.

No, that's not my theory.

Look, in each separate case, it's obviously just a matter of taste.  Is
a mode for highlighting Clojure code something that Emacs should have
centrally?  In ELPA?  Totally separately?  Is a PVR interface?  Is
gnutls?

Each of these, of course, need individual consideration.

However, in general I don't think offloading things that would
traditionally be in Emacs (say, a Ruby mode) to ELPA will help.  I think
it's quite clear that things that are in ELPA will bit rot faster than
code that's in Emacs.  As Glenn pointed out, he's cleaning up and
working on the Emacs distribution, and that's my inclination, too.  If
we want to say "well, Emacs has a supported Ruby mode", then having it
in ELPA will only hinder that, not help.

To get back to the example at hand, rainbow-mode, it's something that I
would use, and the repeated reimplementation of the same functionality
shows that it's something that other people would use, too.  That's my
argument for including it in Emacs.  What are the counter-arguments?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: rainbow-mode

[Ted Zlatanov]

On Tue, 16 Nov 2010 15:22:09 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> To get back to the example at hand, rainbow-mode, it's something that I
LMI> would use, and the repeated reimplementation of the same functionality
LMI> shows that it's something that other people would use, too.  That's my
LMI> argument for including it in Emacs.  What are the counter-arguments?

Discussion is archived here:

http://thread.gmane.org/gmane.emacs.devel/127873

You saw some of the arguments:

> "Stephen J. Turnbull" <stephen@xemacs.org> writes:

> > IIRC, the most recent discussion concluded that none of the half-
> > implemented modes is good enough, especially given the nice features
> > in the other half-implemented modes, and nobody stepped up to the
> > plate to merge them, document the result, and make sure the result
> > conforms to coding standards etc.

Plus Chong Yidong explained in this thread and in
http://thread.gmane.org/gmane.emacs.devel/127873/focus=128245

Basically none of the authors wanted to merge their mode with the
others.  Julien added enough to rainbow-mode.el to make it the best
choice but it's still not superior to css-color-mode.el in every way.

FWIW my vote is still for rainbow-mode.el as follows: I think it's most
useful in its current form out of all the contenders.  I would put it in
the core Emacs because editing CSS and HTML (plus other languages with
inlined colors) is so common.  The color transformation functions it has
are generally useful.  I would also give it first-class customization
variables so it's really easy to turn it on.  Finally, it's pretty
unlikely to undergo big changes at this point.

Maybe it could get renamed, I don't care too much.  The current name is
quite memorable though.

Ted

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Plus Chong Yidong explained in this thread and in
> http://thread.gmane.org/gmane.emacs.devel/127873/focus=128245

I've read the thread.  There was discussion, Julien added stuff to
rainbow-mode as suggested, and then Chong said "I've added rainbow-mode
to the Emacs 24 package repository.", and that ended the discussion.
Is that an unfair summation of that thread?

> FWIW my vote is still for rainbow-mode.el as follows: I think it's most
> useful in its current form out of all the contenders.  I would put it in
> the core Emacs because editing CSS and HTML (plus other languages with
> inlined colors) is so common.  The color transformation functions it has
> are generally useful.  I would also give it first-class customization
> variables so it's really easy to turn it on.  Finally, it's pretty
> unlikely to undergo big changes at this point.

I agree 100%.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org repository sync with Emacs

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> 1) add elpa/ directory to main Emacs repo (as a branch or
> subdirectory; my vote is for a subdirectory that's not bundled or
> compiled because it will get branched together with Emacs itself).
> Make it available to a dev checkout of Emacs as a file:/// URL (so it
> can be tested easily).

I think it makes more sense to add elpa/ as a branch.  Then we can just
`bzr merge' from savannah into the bzr repository used for elpa.gnu.org,
in order to grab the changes made by Emacs devs.  (I don't think you can
`bzr merge' a subdirectory, or can you?)

First, we should make a change to the elpa.gnu.org repository:
currently, the .tar packages live in the repositories as tarballs.  We
should instead leave them as untarred directories, and put the script on
elpa.gnu.org in charge of tarring them up after `bzr export'.  That way,
changes made to the contents of packages can be viewed with bzr diff.

Still unresolved is the problem of which packages should be considered
"OK" to tweak, and which should not.  I don't relish the idea of having
to do a big merge every time a package is released upstream.  I think we
should have a clear policy that only packages that are developed
principally inside the package bzr repository, or whose maintainer is
explicitly in charge of merging, should be hacked on.

Re: rainbow-mode

[Ted Zlatanov]

On Tue, 16 Nov 2010 15:50:25 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Plus Chong Yidong explained in this thread and in
>> http://thread.gmane.org/gmane.emacs.devel/127873/focus=128245

LMI> I've read the thread.  There was discussion, Julien added stuff to
LMI> rainbow-mode as suggested, and then Chong said "I've added rainbow-mode
LMI> to the Emacs 24 package repository.", and that ended the discussion.
LMI> Is that an unfair summation of that thread?

He also said he would add the others if proposed, meaning (I think) none
of them were ready.  I simply added rainbow-mode.el from the package
repository at that point because it was enough for me.

>> FWIW my vote is still for rainbow-mode.el as follows: I think it's most
>> useful in its current form out of all the contenders.  I would put it in
>> the core Emacs because editing CSS and HTML (plus other languages with
>> inlined colors) is so common.  The color transformation functions it has
>> are generally useful.  I would also give it first-class customization
>> variables so it's really easy to turn it on.  Finally, it's pretty
>> unlikely to undergo big changes at this point.

LMI> I agree 100%.

If the maintainers agree too, I'll work with Julien to make this
happen.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 10:01:03 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> Ted Zlatanov <tzz@lifelogs.com> writes:
>> 1) add elpa/ directory to main Emacs repo (as a branch or
>> subdirectory; my vote is for a subdirectory that's not bundled or
>> compiled because it will get branched together with Emacs itself).
>> Make it available to a dev checkout of Emacs as a file:/// URL (so it
>> can be tested easily).

CY> I think it makes more sense to add elpa/ as a branch.  Then we can just
CY> `bzr merge' from savannah into the bzr repository used for elpa.gnu.org,
CY> in order to grab the changes made by Emacs devs.  (I don't think you can
CY> `bzr merge' a subdirectory, or can you?)

We would merge the whole Emacs repository into a repo on elpa.gnu.org.
But only the elpa/ subdirectory would get deployed out of that repo, the
rest would be there just to make the merge simpler.  I think that's the
easiest way.

As I said, I think a subdirectory will allow branching which in turn
will freeze the elpa/* packages for a particular version of Emacs
(instead of making special branches for that purpose).  That alone is
worth the price of admission.

CY> First, we should make a change to the elpa.gnu.org repository:
CY> currently, the .tar packages live in the repositories as tarballs.  We
CY> should instead leave them as untarred directories, and put the script on
CY> elpa.gnu.org in charge of tarring them up after `bzr export'.  That way,
CY> changes made to the contents of packages can be viewed with bzr
CY> diff.

No problem.  That would just be a "make elpa" command in each directory
and at the top.  I agree tarballs are a pain for tracking so this works
best.

CY> Still unresolved is the problem of which packages should be considered
CY> "OK" to tweak, and which should not.  I don't relish the idea of having
CY> to do a big merge every time a package is released upstream.  I think we
CY> should have a clear policy that only packages that are developed
CY> principally inside the package bzr repository, or whose maintainer is
CY> explicitly in charge of merging, should be hacked on.

How about a "third-party" package repository, disabled by default but
easily enabled, which would host externally hosted packages?

We would fetch truly external packages such as org-mode into it with
scripts, disconnected from the Emacs repo and the elpa.gnu.org
deployment.  And it would not be versioned: whatever is fetched today is
what you get.

That separates things nicely between "internal" and "external"
elpa.gnu.org packages, with "internal" packages coming from the Emacs
repo, versioned, and hacked regularly; "external" packages independent,
fetched regularly, and maintained outside of Emacs.

Ted

Re: compat unification

[Stefan Monnier]

> What if...  there was an ems-compat function in bzr Emacs.  It would be
> a, sort of, "official" current-Emacs-to-everything compat layer.

It couldn't be part of Emacs: Emacs-22's ems-compat wouldn't be able to
come with Emacs-24's compatibility functions, would it?
So it'd have to be an ELPA package.

You can go some way with such a package, but it has significant limits.
E.g. you'll still often want to know whether a feature is supported or
not before using it, because you'll want to use some other code if
it's not.  Also such compatibility definitions may confuse other
packages who'd check (fboundp 'foo) to see if a particular feature
is present.


        Stefan

RE: rainbow-mode

[Drew Adams]

>  > I'm not quite sure what you mean to say here.  Is it "this 
>  > is something that so many people want in Emacs that it's
>  > been partially reimplemented half a dozen times"?
> 
> IIRC, the most recent discussion concluded that none of the half-
> implemented modes is good enough, especially given the nice features
> in the other half-implemented modes, and nobody stepped up to the
> plate to merge them, document the result, and make sure the result
> conforms to coding standards etc.

YRW.  Or you are inventing.  What "half-implemented" modes are you talking
about?  What do either of you think has been "partially reimplemented"?  What
discussion and conclusions are you talking about?

None of the libraries I mentioned (hexrgb.el, palette.el, doremi.el) have been
discussed here for inclusion in Emacs.  They are not half-implemented, and
they've been in use for years.

And none of them reimplement anything.  hexrgb.el and rainbow-mode.el both use
standard HSV/RGB conversion formulas.  That's all they have in common.  Using a
standard conversion formula, whether inches to centimeters or RGB to HSV, is not
"partially reimplementing" anything.

* hexrgb.el is a general-purpose library for color hex code conversion.  It is
used by several other libraries (including DoReMi and Palette and 4 other
libraries I wrote) to do just that.

* rainbow-mode.el is a library for showing color names/codes using the color as
background.  That is one specific use of color.

Similarly, DoReMi, Palette, and rainbow-mode all have different uses.  Yes, you
can use any of them to see what a color's hex code looks like.  But that is not
what they are about.  None of them "partially reimplements" another.

I mentioned DoReMi and Palette in the context of Lars's description of
attempting to tweak colors the hard way ("try #404090, *save*, *reload*, 'no,
darker', etc.").  The point was that there are "WYSIWYG ways to incrementally
adjust colors".

Re: compat unification

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> It couldn't be part of Emacs: Emacs-22's ems-compat wouldn't be able to
> come with Emacs-24's compatibility functions, would it?
> So it'd have to be an ELPA package.

The ems-compat library would support *all* previous Emacs and XEmacs
instances -- so there would only be one, was my thought.

But having it in ELPA would work, too.

> You can go some way with such a package, but it has significant limits.
> E.g. you'll still often want to know whether a feature is supported or
> not before using it, because you'll want to use some other code if
> it's not.

Yes, and it doesn't really help with functions that do exist, but take
different parameters, for instance.  (Those are thankfully somewhat
rare.)  And variables that you bind to achieve an effect are difficult
to do anything with...

So it's not exactly a panacea...  

> Also such compatibility definitions may confuse other packages who'd
> check (fboundp 'foo) to see if a particular feature is present.

Yes, that's been the main objection to doing this in the past.
However -- packages really should check for features, and not the
presence of functions, so we could declare doing that as an error.  I'm
not sure what the impact would be, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: rainbow-mode

[Julien Danjou]

On Tue, Nov 16 2010, Ted Zlatanov wrote:

> Basically none of the authors wanted to merge their mode with the
> others.  Julien added enough to rainbow-mode.el to make it the best
> choice but it's still not superior to css-color-mode.el in every way.

Just to summarize.

css-color-mode features:
- displays colors for CSS/HTML.
- allow colors edition like augmenting hue, or RGB value with
  interactive functions

rainbow-mode features:
- displays colors for CSS, HTML, LaTeX, and anything that can has
  hexadecimal or X colors (C code, .Xresources, whatever).

So for now rainbow-mode is really about displaying and not editing at
all.

At least for now. :)

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

RE: elpa.gnu.org policy

[Drew Adams]

> Yeah...  well, back in the 90s, it made sense to support all the
> different Emacs versions, because the development direction 
> of the main Emacs was so, er, interesting.  Ahem.

Revisionism.

Emacs development today is not as robust and inventive as it was in the 90s
(including Epoch, Zmacs,...), in spite of far better communication today and a
larger number of software developers and Emacs users around the planet.

Far from being some horrible Frankenstein experiment gone bad, that was a
fertile period for Emacs development.

(In my view.)

> So there really were reasons for people not to just use
> the latest Emacs.  But I don't really see that as
> much of an issue any more...

Get real.  There are lots of people _today_ who use other than the latest Emacs.
That you don't understand that or the different reasons they do so is too bad.

> Me too.  I wish it was possible to just write towards Emacs 24 and
> pretend that nothing else existed

Right, who cares about users?

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

>> 1) add elpa/ directory to main Emacs repo (as a branch or
>> subdirectory; my vote is for a subdirectory that's not bundled or
>> compiled because it will get branched together with Emacs itself).
>> Make it available to a dev checkout of Emacs as a file:/// URL (so it
>> can be tested easily).

> I think it makes more sense to add elpa/ as a branch.  Then we can just
> `bzr merge' from savannah into the bzr repository used for elpa.gnu.org,
> in order to grab the changes made by Emacs devs.

200% agreement.

> (I don't think you can `bzr merge' a subdirectory, or can you?)

If you want to you can, but it's poorly supported by Bazaar, and in any
case we don't want to do that (if for nothing else, so as to avoid
imposing the slowness of Emacs's repository to the elpa repository).

> First, we should make a change to the elpa.gnu.org repository:
> currently, the .tar packages live in the repositories as tarballs.  We
> should instead leave them as untarred directories, and put the script on
> elpa.gnu.org in charge of tarring them up after `bzr export'.  That way,
> changes made to the contents of packages can be viewed with bzr diff.

Yes, obviously.

> Still unresolved is the problem of which packages should be considered
> "OK" to tweak, and which should not.  I don't relish the idea of having
> to do a big merge every time a package is released upstream.  I think we
> should have a clear policy that only packages that are developed
> principally inside the package bzr repository, or whose maintainer is
> explicitly in charge of merging, should be hacked on.

All packages should be OK to hack on.  And all commits should be
reflected on emacs-diffs (or maybe emacs-packages-diffs) *as well as*
sent to the respective upstream maintainer.
And all maintainers will be in charge of merging (which may include
rejecting a change we've made, of course).

What's the difference between this and packages that are bundled, then?
The difference is that we know for sure that the upstream maintainer
automatically gets an email for each one of our changes (i.e. he is
proactively warned of such changes), and also we know that the package
needs to pay attention to backward compatibility, so we have to be more
careful with the changes we install there.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

> That separates things nicely between "internal" and "external"
> elpa.gnu.org packages,

I think such a neat separation doesn't actually exist, so I'd rather
treat them all the same, except that some will get more
frequent/significant updates from the upstream maintainers.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> What's the difference between this and packages that are bundled, then?
> The difference is that we know for sure that the upstream maintainer
> automatically gets an email for each one of our changes (i.e. he is
> proactively warned of such changes),

Surely that could be hacked up with a commit script (or the like) that
would Cc the ";; Author:" line in the files, if that's a thing that we
want to do.

> and also we know that the package needs to pay attention to backward
> compatibility, so we have to be more careful with the changes we
> install there.

So...  ELPA would be for packages that we want to have readily available
in Emacs, but that we can't ever easily cut any cruft out of as Emacs
progresses?  But since the FSF-copyright assignment regime would still
apply to ELPA, we'd wouldn't be able to accept big packages like auctex
anyway?

I don't honestly see how this helps either Emacs users or people who hack
on Emacs.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 19:00:10 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> So...  ELPA would be for packages that we want to have readily available
LMI> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
LMI> progresses?  But since the FSF-copyright assignment regime would still
LMI> apply to ELPA, we'd wouldn't be able to accept big packages like auctex
LMI> anyway?

You know, AUCTeX is in there...  http://elpa.gnu.org/packages/

Chong Yidong and the AUCTeX maintainers spent a lot of time getting the
copyrights done and putting the tarball together.

IMHO big packages are exactly where elpa.gnu.org will be most useful.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 12:28:41 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> That separates things nicely between "internal" and "external"
>> elpa.gnu.org packages,

SM> I think such a neat separation doesn't actually exist, so I'd rather
SM> treat them all the same, except that some will get more
SM> frequent/significant updates from the upstream maintainers.

OK, whatever works (we can change our mind before 24 is out so I want to
get things going in any form).  You and Chong can let me know how you
decide to lay things out and I will start getting the deployment and
sync scripts done.  I hope you'll reconsider my proposal to keep the
packages in an elpa/ subdirectory instead of a separate branch.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> You know, AUCTeX is in there...  http://elpa.gnu.org/packages/

Oh, OK.  Somebody else mentioned possible AUCTeX copyright problems...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org repository sync with Emacs

[Eric Schulte]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> 
> But since the FSF-copyright assignment regime would still apply to
> ELPA, we'd wouldn't be able to accept big packages like auctex anyway?
>

If this is true and ELPA has the same strict copyright restrictions as
the Emacs core, then IMO we're losing a significant portion of the
benefit of ELPA.

There are many packages which will never secure sufficient copyright
permissions to be included into Emacs.  Such packages were previously
available through ELPA.  Where can these packages live now?

Would it make sense to setup a second ELPA repository for non-FSF
software?

Forgive me if this issue has already been settled in a previous
conversation which I've missed.

-- Eric

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 11:29:12 -0700 "Eric Schulte" <schulte.eric@gmail.com> wrote: 

ES> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>> 
>> But since the FSF-copyright assignment regime would still apply to
>> ELPA, we'd wouldn't be able to accept big packages like auctex anyway?

ES> If this is true and ELPA has the same strict copyright restrictions as
ES> the Emacs core, then IMO we're losing a significant portion of the
ES> benefit of ELPA.

It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs ELPA.  I make the
distinction again because it matters:

ES> There are many packages which will never secure sufficient copyright
ES> permissions to be included into Emacs.  Such packages were previously
ES> available through ELPA.  Where can these packages live now?

Tom Tromey's ELPA (http://tromey.com/elpa/) will not go away.  You can
also host your own ELPA-style package repository.  All you need is a web
site with a package directory.

The only thing that makes elpa.gnu.org special is that some of the
package repositories it hosts will be enabled by default with Emacs 24.
But this requires that elpa.gnu.org only host assigned packages, unlike
Tom Tromey's ELPA.

ES> Would it make sense to setup a second ELPA repository for non-FSF
ES> software?

Yes and I suggested it to the Emacs maintainers a while ago, but Tom
Tromey's ELPA may be sufficient (and also see ELL at
http://www.damtp.cam.ac.uk/user/sje30/emacs/ell.html and
http://www.emacswiki.org/ElispArea).  I think we should concentrate on
getting elpa.gnu.org operational first and then cast a wider net.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

> If this is true and ELPA has the same strict copyright restrictions as
> the Emacs core, then IMO we're losing a significant portion of the
> benefit of ELPA.

It doesn't cut any of the benefits of ELPA.  It will only restrict them
to the cases of packages whose copyright has been transferred.
Note that transferring copyright is not nearly as much of a hassle as
many people think.  Yes, it's a big hassle for old packages that have
accumulated many contributions by many people over the years (AUCTeX and
BBDB come to mind), but most Elisp packages are smaller and mostly
written by a single author.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

> sync scripts done.  I hope you'll reconsider my proposal to keep the
> packages in an elpa/ subdirectory instead of a separate branch.

No, I definitely do not want it within the trunk branch.  It has to have
a separate branch.  Actually, there's a good case to be made for having
a separate branch per package, tho I'd rather not go there because it
would make it too hard for hackers to just always have a checkout of
"all packages".


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

>> What's the difference between this and packages that are bundled, then?
>> The difference is that we know for sure that the upstream maintainer
>> automatically gets an email for each one of our changes (i.e. he is
>> proactively warned of such changes),
> Surely that could be hacked up with a commit script (or the like) that
> would Cc the ";; Author:" line in the files, if that's a thing that we
> want to do.

Yes, that's indeed what I want (I also want it for the trunk, FWIW).

>> and also we know that the package needs to pay attention to backward
>> compatibility, so we have to be more careful with the changes we
>> install there.
> So...  ELPA would be for packages that we want to have readily available
> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
> progresses?

No, not at all.  Just that those packages will often want to support
older Emacsen just like they already do currently when they're not
in ELPA.

> I don't honestly see how this helps either Emacs users

It helps Emacs users because those packages are easier to install than
having to go to emacswiki, download the file(s), tweak their .emacs, ...

> or people who hack on Emacs.

It reduces the pressure to bundle each and every package with Emacs.
It provides a place to keep "copyright ready code", that we can bundle in
Emacs if/when we decide it's a good idea.
A few other such reasons.

Regarding moving stuff from ELPA into Emacs, there's also the idea that
some packages could be moved out of Emacs into ELPA, but come bundled
into the Emacs release tarball.  Gnus and Org would seem like good
candidates for that, although I'm still not sure whether that's
a good idea.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 14:24:15 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

SM> Regarding moving stuff from ELPA into Emacs, there's also the idea that
SM> some packages could be moved out of Emacs into ELPA, but come bundled
SM> into the Emacs release tarball.  Gnus and Org would seem like good
SM> candidates for that, although I'm still not sure whether that's
SM> a good idea.

IMO we should enable and supply a file:// repository with a new Emacs
release and with every trunk build (as part of `make install').  That
repository can be augmented by elpa.gnu.org, but if there's no network
connection to it users will at least be able to install the packages in
the latest Emacs checkout and hack on them.

In fact the repository can be generated at install time by the same
scripts and Makefiles that will generate the elpa.gnu.org package
repositories.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Tom Tromey]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

Lars> I don't honestly see how this helps either Emacs users or people who hack
Lars> on Emacs.

I can name a few benefits.

One is that a big package like gnus, with its own release schedule, can
push updates to elpa.gnu.org between Emacs releases.  This helps gnus
users.  (And, due to the way package activation is done, when such a
user upgrades Emacs, the right thing will still happen.)

Another benefit is that by being in Emacs, it provides elisp authors
some simple packaging guidelines.  My hope is that over time, most 3rd
party elisp will follow these rules (meaning it will be trivial for
users to install the code, regardless of how they get it).  I also hope
the distros move to using package.el for elisp activation.

Tom

Re: elpa.gnu.org repository sync with Emacs

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> IMO we should enable and supply a file:// repository with a new Emacs
> release and with every trunk build (as part of `make install').  That
> repository can be augmented by elpa.gnu.org, but if there's no network
> connection to it users will at least be able to install the packages in
> the latest Emacs checkout and hack on them.

Conceptually neat, but I agree with Stefan that the practical
difficulties of putting the elpa packages in the trunk are too great.

Those Emacs hackers interested in the elpa packages can always check out
the branch separately, and we could even add a Makefile option that
installs into a local directory.  It doesn't have to be in the trunk.

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Tom Tromey <tromey@redhat.com> writes:

> I also hope the distros move to using package.el for elisp activation.

The distros have perfectly good packaging solutions already, and they
have the QA in place to make sure that whatever you're downloading will
play nice with what you already have.

Which we don't really have, and it sounds from the discussion that the
Emacs maintainers think it would be too much work to have branches for
each Emacs, for instance.

So we'd basically have the same situation that Perl has with CTAN
vs. the packages that (say) Debian has.  You can say "apt-get install
libemail-mime-createhtml-perl" and get a version of that library that
you know will work on your machine, or you can root around on CTAN,
download the latest version there, and it'll probably work -- except for
the times it doesn't, because it relies on something that's slightly too
new for your machine.

Decoupled packaging over the long haul is hard.  Making sure that
separate packages work together over a decade is difficult.
Dependencies change and stuff break.

I want to be able to install Emacs in 2015 and just have it work without
getting into a morass of packages that I have to install from here and
there.  I want to say "apt-get install emacs", and possibly "apt-get
install slime" (if that's not in Emacs by then), and know that I have
something that has been tested to work on the machine I have then.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org repository sync with Emacs

[Eric Schulte]

Hi Ted,

Thanks for the thorough reply, this clears everything up.

Best -- Eric

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Tue, 16 Nov 2010 11:29:12 -0700 "Eric Schulte"
> <schulte.eric@gmail.com> wrote:
>
> ES> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>> 
>>> But since the FSF-copyright assignment regime would still apply to
>>> ELPA, we'd wouldn't be able to accept big packages like auctex anyway?
>
> ES> If this is true and ELPA has the same strict copyright restrictions as
> ES> the Emacs core, then IMO we're losing a significant portion of the
> ES> benefit of ELPA.
>
> It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs ELPA.  I make the
> distinction again because it matters:
>
> ES> There are many packages which will never secure sufficient copyright
> ES> permissions to be included into Emacs.  Such packages were previously
> ES> available through ELPA.  Where can these packages live now?
>
> Tom Tromey's ELPA (http://tromey.com/elpa/) will not go away.  You can
> also host your own ELPA-style package repository.  All you need is a web
> site with a package directory.
>
> The only thing that makes elpa.gnu.org special is that some of the
> package repositories it hosts will be enabled by default with Emacs 24.
> But this requires that elpa.gnu.org only host assigned packages, unlike
> Tom Tromey's ELPA.
>
> ES> Would it make sense to setup a second ELPA repository for non-FSF
> ES> software?
>
> Yes and I suggested it to the Emacs maintainers a while ago, but Tom
> Tromey's ELPA may be sufficient (and also see ELL at
> http://www.damtp.cam.ac.uk/user/sje30/emacs/ell.html and
> http://www.emacswiki.org/ElispArea).  I think we should concentrate on
> getting elpa.gnu.org operational first and then cast a wider net.
>
> Ted

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 15:02:57 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> Ted Zlatanov <tzz@lifelogs.com> writes:
>> IMO we should enable and supply a file:// repository with a new Emacs
>> release and with every trunk build (as part of `make install').  That
>> repository can be augmented by elpa.gnu.org, but if there's no network
>> connection to it users will at least be able to install the packages in
>> the latest Emacs checkout and hack on them.

CY> Conceptually neat, but I agree with Stefan that the practical
CY> difficulties of putting the elpa packages in the trunk are too great.

They don't have to live in the trunk; sorry I didn't say that clearly.

CY> Those Emacs hackers interested in the elpa packages can always check out
CY> the branch separately, and we could even add a Makefile option that
CY> installs into a local directory.  It doesn't have to be in the trunk.

That's exactly what I meant, with the Makefile option :)

Ted

Re: ELPA policy

[Richard Stallman]

    - Who will assure there's no really bad things uploaded?

We should treat inclusion in ELPA like inclusion in Emacs.

-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 21:21:40 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Tom Tromey <tromey@redhat.com> writes:
>> I also hope the distros move to using package.el for elisp activation.

LMI> The distros have perfectly good packaging solutions already, and they
LMI> have the QA in place to make sure that whatever you're downloading will
LMI> play nice with what you already have.

The problem is there are lots of distros and most don't have good QA in
place.  Debian and Ubuntu are hardly the norm.  The ELPA-style repos are
a way to work around the lack of standard packaging.

LMI> So we'd basically have the same situation that Perl has with CTAN
LMI> vs. the packages that (say) Debian has.  You can say "apt-get install
LMI> libemail-mime-createhtml-perl" and get a version of that library that
LMI> you know will work on your machine, or you can root around on CTAN,
LMI> download the latest version there, and it'll probably work -- except for
LMI> the times it doesn't, because it relies on something that's slightly too
LMI> new for your machine.

s/CTAN/CPAN/g :)

This problem is not solvable.  You have competing interests (distro
vs. users vs. developers) that are pulling in different directions.
ELPA-style repos for Emacs give control back to the developers and the
users in a distro-neutral way (much like CPAN, in fact).  In addition,
distros can take advantage of that facility and add their own ELPA-style
repos, but they probably won't.  They have a strong interest in keeping
the packaging in their own format, which is perfectly sensible to keep
QA and other costs down.

LMI> Decoupled packaging over the long haul is hard.  Making sure that
LMI> separate packages work together over a decade is difficult.
LMI> Dependencies change and stuff break.

That's the worst haiku *ever* :)

LMI> I want to be able to install Emacs in 2015 and just have it work without
LMI> getting into a morass of packages that I have to install from here and
LMI> there.  I want to say "apt-get install emacs", and possibly "apt-get
LMI> install slime" (if that's not in Emacs by then), and know that I have
LMI> something that has been tested to work on the machine I have then.

If you want QA from the distro you have to install their packages.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> That's the worst haiku *ever* :)

Heh heh heh.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: compat unification

[Glenn Morris]

Lars Magne Ingebrigtsen wrote:

> The ems-compat library would support *all* previous Emacs and XEmacs
> instances

Eeesh, what a horrible thought. I don't think supporting old releases
should be encouraged.

RE: rainbow-mode

[Stephen J. Turnbull]

Drew Adams writes:
 > >  > I'm not quite sure what you mean to say here.  Is it "this 
 > >  > is something that so many people want in Emacs that it's
 > >  > been partially reimplemented half a dozen times"?
 > > 
 > > IIRC, the most recent discussion concluded that none of the half-
 > > implemented modes is good enough, especially given the nice features
 > > in the other half-implemented modes, and nobody stepped up to the
 > > plate to merge them, document the result, and make sure the result
 > > conforms to coding standards etc.
 > 
 > YRW.  Or you are inventing.

Nope.  See Ted Z's post; others understood what I was talking about
even if my description was imprecise or even inaccurate (as signaled
"IIRC", in any case).  I'm just not writing for

    (setq audience (list "Drew Adams"))

Re: elpa.gnu.org repository sync with Emacs

[Stephen J. Turnbull]

Stefan Monnier writes:

 > 200% agreement.

Rounding error here. ;-)

AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Dan Nicolaescu]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Tue, 16 Nov 2010 19:00:10 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>
> LMI> So...  ELPA would be for packages that we want to have readily available
> LMI> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
> LMI> progresses?  But since the FSF-copyright assignment regime would still
> LMI> apply to ELPA, we'd wouldn't be able to accept big packages like auctex
> LMI> anyway?
>
> You know, AUCTeX is in there...  http://elpa.gnu.org/packages/
>
> Chong Yidong and the AUCTeX maintainers spent a lot of time getting the
> copyrights done and putting the tarball together.

Is the any reason not to include AUCTeX in emacs proper instead of elpa?
It's a very useful package to have working by default without having
to deal with setting up packages. 

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Ted Zlatanov]

On Wed, 17 Nov 2010 03:01:09 -0500 Dan Nicolaescu <dann@gnu.org> wrote: 

DN> Ted Zlatanov <tzz@lifelogs.com> writes:
>> On Tue, 16 Nov 2010 19:00:10 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>> 
LMI> So...  ELPA would be for packages that we want to have readily available
LMI> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
LMI> progresses?  But since the FSF-copyright assignment regime would still
LMI> apply to ELPA, we'd wouldn't be able to accept big packages like auctex
LMI> anyway?
>> 
>> You know, AUCTeX is in there...  http://elpa.gnu.org/packages/
>> 
>> Chong Yidong and the AUCTeX maintainers spent a lot of time getting the
>> copyrights done and putting the tarball together.

DN> Is the any reason not to include AUCTeX in emacs proper instead of elpa?

We just went over this several times, didn't we?

DN> It's a very useful package to have working by default without having
DN> to deal with setting up packages. 

You say that like it's a hard task.

M-x package-list-packages
`i' on whatever you want to install, then `x'

Ted

Re: elpa.gnu.org repository sync with Emacs

[Richard Stallman]

    It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs ELPA.  I make the
    distinction again because it matters:

To avoid this confusion, how about renaming the server?
We could call it emacs-pkg.gnu.org.

-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

RE: elpa.gnu.org repository sync with Emacs

[Drew Adams]

>     It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs 
>     ELPA.  I make the distinction again because it matters:
> 
> To avoid this confusion, how about renaming the server?
> We could call it emacs-pkg.gnu.org.

1+

And in discussions (e.g. here) call it something other than ELPA.

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Wed, 17 Nov 2010 11:45:42 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 

>> It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs 
>> ELPA.  I make the distinction again because it matters:
>> 
>> To avoid this confusion, how about renaming the server?
>> We could call it emacs-pkg.gnu.org.

DA> 1+

DA> And in discussions (e.g. here) call it something other than ELPA.

epkg is already used.  emacs-pkg is long and hard to type.

How about ENE (ENE is Not ELPA)?

Ted

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> How about ENE (ENE is Not ELPA)?

ELLE, for ELLE is a Lot Like ELPA.

Stylish!

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org repository sync with Emacs

[Jambunathan K]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 17 Nov 2010 11:45:42 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 
>
>>> It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs 
>>> ELPA.  I make the distinction again because it matters:
>>> 
>>> To avoid this confusion, how about renaming the server?
>>> We could call it emacs-pkg.gnu.org.
>
> DA> 1+
>
> DA> And in discussions (e.g. here) call it something other than ELPA.
>
> epkg is already used.  emacs-pkg is long and hard to type.
>

From a user-perspective the length of the string shouldn't matter. Emacs
will ship with the default archives (whatever it is called) and for
existing installations it is one time re-configuration of
`package-archives'. May be for maintainers the story is different ...

> How about ENE (ENE is Not ELPA)?

ELPA seems like a generic term to me. It is an archive that stores emacs
lisp packages. There could be multiple instantiantions of the archive
(let's call them repositories) and GNU archives is one of it. Needless
to say, each repo can enforce different gating policies or cater to a
niche audience say like windows users.

I think the term ELPA has become 'popular' only in recent years and it
has not been around for so long that it is etched in people's
memory. Furthermore there are other implementations that users use (like
el-get) which more or less provides same functionality as the current
package manager at their core.

I would hesistate to call anything 'not something' unless 'something'
happens to 'a very big thing' or 'a long standing thing' that it gets in
the way of people thinking about it.

Whatever be the name that is adopted, it should capture the 'uniqueness'
of the underlying repo. ie., the name has to speak for itself or it
could be something that is neutral (think 'savannah' for example). 

Just thinking as I type, something 'rooted' in savannah.gnu.org would
create the right association. Or one could get play with names and end
with one of of veldt, puszta, pampas, steppe, prairie ...

Jambunathan K.

>
> Ted

Re: elpa.gnu.org repository sync with Emacs

[Jambunathan K]

Jambunathan K <kjambunathan@gmail.com> writes:

> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> On Wed, 17 Nov 2010 11:45:42 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 
>>
>>>> It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs 
>>>> ELPA.  I make the distinction again because it matters:
>>>> 
>>>> To avoid this confusion, how about renaming the server?
>>>> We could call it emacs-pkg.gnu.org.
>>
>> DA> 1+
>>
>> DA> And in discussions (e.g. here) call it something other than ELPA.
>>

How about http://emacsforge.gnu.org/packages.

These packages actually help one forging of GNU Emacs - these are
privileged packages that are either venerable or beatified and on their
way to sainthood.

Very importantly it is `gnu' and *not* `nongnu'.

> Just thinking as I type, something 'rooted' in savannah.gnu.org would
> create the right association. Or one could get play with names and end
> with one of of veldt, puszta, pampas, steppe, prairie ...
>

Jambunathan K.

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

>     It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs ELPA.  I make the
>     distinction again because it matters:
> To avoid this confusion, how about renaming the server?
> We could call it emacs-pkg.gnu.org.

I had suggested emacs.gnu.org, simply (after all, it's the name of a VM
which we can use for other Emacs-related services).


        Stefan

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Dan Nicolaescu]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 17 Nov 2010 03:01:09 -0500 Dan Nicolaescu <dann@gnu.org> wrote: 
>
> DN> Ted Zlatanov <tzz@lifelogs.com> writes:
>>> On Tue, 16 Nov 2010 19:00:10 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>>> 
> LMI> So...  ELPA would be for packages that we want to have readily available
> LMI> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
> LMI> progresses?  But since the FSF-copyright assignment regime would still
> LMI> apply to ELPA, we'd wouldn't be able to accept big packages like auctex
> LMI> anyway?
>>> 
>>> You know, AUCTeX is in there...  http://elpa.gnu.org/packages/
>>> 
>>> Chong Yidong and the AUCTeX maintainers spent a lot of time getting the
>>> copyrights done and putting the tarball together.
>
> DN> Is the any reason not to include AUCTeX in emacs proper instead of elpa?
>
> We just went over this several times, didn't we?

Sorry, I must have missed that discussion.  Can you please point me to it?

> DN> It's a very useful package to have working by default without having
> DN> to deal with setting up packages. 
>
> You say that like it's a hard task.
>
> M-x package-list-packages
> `i' on whatever you want to install, then `x'

That's a lot more complicated than have things just work by default.
AUCTeX is better than the currently integrated latex mode, so it's
better to just integrated it in emacs instead of having people fiddle
with installing packages.

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Thu, 18 Nov 2010 04:47:49 +0530 Jambunathan K <kjambunathan@gmail.com> wrote: 

>> How about ENE (ENE is Not ELPA)?

JK> I would hesistate to call anything 'not something' unless 'something'
JK> happens to 'a very big thing' or 'a long standing thing' that it gets in
JK> the way of people thinking about it.

That's the joke I was trying to make.  I guess it wasn't very funny.

JK> How about http://emacsforge.gnu.org/packages.

JK> These packages actually help one forging of GNU Emacs - these are
JK> privileged packages that are either venerable or beatified and on their
JK> way to sainthood.

Whoa.  No.

On Wed, 17 Nov 2010 23:19:45 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:

>> How about ENE (ENE is Not ELPA)?

LMI> ELLE, for ELLE is a Lot Like ELPA.

Works for me.

Ted

RE: elpa.gnu.org repository sync with Emacs

[Drew Adams]

> I had suggested emacs.gnu.org, simply (after all, it's the 
> name of a VM which we can use for other Emacs-related services).

Whatever name/address is chosen for the server, we still also need a name for
talking about this thing - a name that is different from "ELPA" (if that name
refer's to Tom's thing/place).

I suggest that the server name/address reflect whatever name you choose for the
thing itself (the Gnu "elpa").  Keep it simple.

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Richard Stallman]

    Is the any reason not to include AUCTeX in emacs proper instead of elpa?

The benefit of the Emacs package archive (let's not call it elpa!) is
to make the Emacs tar file smaller by offloading a lot of Lisp code
that many people don't want.  I think AUCTeX is a good candidate for
that.

-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

Re: elpa.gnu.org repository sync with Emacs

[Richard Stallman]

    ELLE, for ELLE is a Lot Like ELPA.

Better, ELLE = Elle Looks Like ELPA.

-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Dan Nicolaescu]

Richard Stallman <rms@gnu.org> writes:

>     Is the any reason not to include AUCTeX in emacs proper instead of elpa?
>
> The benefit of the Emacs package archive (let's not call it elpa!) is
> to make the Emacs tar file smaller by offloading a lot of Lisp code
> that many people don't want.  I think AUCTeX is a good candidate for
> that.

Is the tar file size worth worrying about?  It seems hat tar file size
is immaterial for most users nowadays, they install packages from
various distributions.
The advantage of having things included in emacs proper is that one
just has to make sure that has the emacs package installed, and
everything else works.
These days users work on many machines: work, home, maybe a work
laptop, a personal laptop, etc.  Having to configure each one of them
is not ideal, it's better when things just work by default.
AUCTeX is much better that the currently bundled latex/tex modes.

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[David Kastrup]

Dan Nicolaescu <dann@gnu.org> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>>     Is the any reason not to include AUCTeX in emacs proper instead of elpa?
>>
>> The benefit of the Emacs package archive (let's not call it elpa!) is
>> to make the Emacs tar file smaller by offloading a lot of Lisp code
>> that many people don't want.  I think AUCTeX is a good candidate for
>> that.
>
> Is the tar file size worth worrying about?  It seems hat tar file size
> is immaterial for most users nowadays, they install packages from
> various distributions.
> The advantage of having things included in emacs proper is that one
> just has to make sure that has the emacs package installed, and
> everything else works.
> These days users work on many machines: work, home, maybe a work
> laptop, a personal laptop, etc.  Having to configure each one of them
> is not ideal, it's better when things just work by default.
> AUCTeX is much better that the currently bundled latex/tex modes.

Correction: AUCTeX is much better than the currently bundled LaTeX mode.
The other TeX-based modes, however, are of comparable quality to the
existing ones while offering a similar interface to AUCTeX's LaTeX mode.
In particular people preferring to work with plain TeX will often find
the small code footprint and simplicity of Emacs' bundled TeX mode more
suited to their way of thinking.

-- 
David Kastrup

Re: elpa.gnu.org repository sync with Emacs

[Byung-Hee HWANG]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

Ted Zlatanov <tzz@lifelogs.com> writes:
> [...]
> epkg is already used.  emacs-pkg is long and hard to type.

Well, i use more mouse than keyboard. So, at least, there is no problem
to me. Also, by the way, i'd like to give you other name if you're OK: 

    GNU-Emacs-Lisp-Package-Archive.GNU.ORG

Sincerely,

-- 
소여물 황병희(黃炳熙) | .. 출항 15분전..

"We can talk business after dinner."
		-- Jack Woltz, "Chapter 1", page 59

[-- Attachment #2: Type: application/pgp-signature, Size: 195 bytes --]

RE: elpa.gnu.org repository sync with Emacs

[Drew Adams]

> i'd like to give you other name if you're OK: 
> 
>     GNU-Emacs-Lisp-Package-Archive.GNU.ORG

1+

(Those so inclined can call it GELPA.  Or not.)

The name should say what it is.  If the abbreviation shows some relation to
ELPA, that's not a bad thing.

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Sat, 20 Nov 2010 19:05:58 +0900 Byung-Hee HWANG <bh@izb.knu.ac.kr> wrote: 

BH> Also, by the way, i'd like to give you other name if you're OK:

BH>     GNU-Emacs-Lisp-Package-Archive.GNU.ORG

That's a very long CNAME.  I think it would annoy everyone.

On Sat, 20 Nov 2010 07:26:17 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 

DA> (Those so inclined can call it GELPA.  Or not.)

DA> The name should say what it is.  If the abbreviation shows some relation to
DA> ELPA, that's not a bad thing.

I think the confusion between "ELPA-like" and "ELPA" will only get worse
as ELLE (or whatever we call it) becomes more popular.  So I'm against
anything that includes "ELPA" in the name.  ELLE is short, memorable,
and slightly amusing, though I'm sure the joke will wear thin if we
discuss it to death.

I think we have a majority in favor of ELLE (Lars, RMS, and me).  Can we
rely on Chong Yidong and Stefan Monnier to make the final decision?

Ted

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Ted Zlatanov]

On Sat, 20 Nov 2010 02:45:30 -0500 Dan Nicolaescu <dann@gnu.org> wrote: 

DN> The advantage of having things included in emacs proper is that one
DN> just has to make sure that has the emacs package installed, and
DN> everything else works.

I really don't think using package.el is any different.  Remember I'm
advocating to make a local install of the packages in the "elpa" branch
as part of the Emacs install from a fresh checkout or in a distro.  If
we do that, network access is unnecessary but users can still get an
updated package if they can contact the elpa.gnu.org server.  So if we
never release an updated package there's no difference, but if we do
have updates then the users don't have to update all of Emacs.

DN> These days users work on many machines: work, home, maybe a work
DN> laptop, a personal laptop, etc.  Having to configure each one of them
DN> is not ideal, it's better when things just work by default.
DN> AUCTeX is much better that the currently bundled latex/tex modes.

It's not hard to add wrappers that will trigger a package install if
necessary and will at least check if the version is the latest
available.  That would come really close to "work by default," probably
more so than the current situation.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I think the confusion between "ELPA-like" and "ELPA" will only get
> worse as ELLE (or whatever we call it) becomes more popular.  So I'm
> against anything that includes "ELPA" in the name.  ELLE is short,
> memorable, and slightly amusing, though I'm sure the joke will wear
> thin if we discuss it to death.
>
> I think we have a majority in favor of ELLE (Lars, RMS, and me).  Can
> we rely on Chong Yidong and Stefan Monnier to make the final decision?

I think we should stick to ELPA.  It stands for "Emacs Lisp Package
Archive", which is much more logical and easier to remember than an
unrelated self-referential acronym.  There need not be one single ELPA;
if one needs to draw a distinction, we can refer to GNU ELPA versus Tom
Tromey's ELPA, etc.

Stefan's suggestion to use emacs.gnu.org/packages is fine by me too, but
even in that case we'd still refer to the package archive as the GNU
Emacs package archive, so there's hardly much difference.

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

> I think we have a majority in favor of ELLE (Lars, RMS, and me).  Can we
> rely on Chong Yidong and Stefan Monnier to make the final decision?

I stand by my suggestion to name the machine emacs.gnu.org (in short,
EGO) and use the name "packages" for the Bzr branch (and URLs to
emacs.gnu.org would also use "packages" somewhere) that holds the
unbundled packages.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Mon, 22 Nov 2010 11:47:37 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> I think we should stick to ELPA.  It stands for "Emacs Lisp Package
CY> Archive", which is much more logical and easier to remember than an
CY> unrelated self-referential acronym.  There need not be one single ELPA;
CY> if one needs to draw a distinction, we can refer to GNU ELPA versus Tom
CY> Tromey's ELPA, etc.

CY> Stefan's suggestion to use emacs.gnu.org/packages is fine by me too, but
CY> even in that case we'd still refer to the package archive as the GNU
CY> Emacs package archive, so there's hardly much difference.

On Mon, 22 Nov 2010 11:48:10 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

SM> I stand by my suggestion to name the machine emacs.gnu.org (in short,
SM> EGO) and use the name "packages" for the Bzr branch (and URLs to
SM> emacs.gnu.org would also use "packages" somewhere) that holds the
SM> unbundled packages.

OK, so the "GNU ELPA" will be hosted on elpa.gnu.org/packages AKA
emacs.gnu.org/packages.  I'll make sure to use "GNU ELPA" in messages
from now on to denote this package repository.  If we set up multiples
we may need to refine the terminology further but I think this is a good
start.

Thanks
Ted

Re: elpa.gnu.org repository sync with Emacs

[Richard Stallman]

    I think we should stick to ELPA.  It stands for "Emacs Lisp Package
    Archive", which is much more logical and easier to remember than an
    unrelated self-referential acronym.

That is the worst possible name.
We already know that this leads to confusion.

      There need not be one single ELPA;
    if one needs to draw a distinction, we can refer to GNU ELPA versus Tom
    Tromey's ELPA, etc.

But not everyone will know this is necessary.

    Stefan's suggestion to use emacs.gnu.org/packages is fine by me too, but
    even in that case we'd still refer to the package archive as the GNU
    Emacs package archive, so there's hardly much difference.

emacs.gnu.org is fine as a host name, but if the URL doesn't provide
some suitable name to refer to this collection of packages, it won't
avoid the confusion.  emacs.gnu.org/packages doesn't solve the
problem.

So we want to replace either `emacs' or `packages' with a name people
could use conversationally for this particular collection of packages.

Another idea might be to replace the word "archive" with something
else -- "collection"?



-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

RE: elpa.gnu.org repository sync with Emacs

[Drew Adams]

> So we want to replace either `emacs' or `packages' with a name people
> could use conversationally for this particular collection of packages.

Yes.  The official (i.e. possibly longer) name and the URL should be
unambiguous, but there should also be a unique short name for use in discussion.
None of the three should be easily confusable with ELPA, but any of them could
suggest some relation to it.

For those reasons I supported Byung-Hee's suggestion of
`gnu-emacs-lisp-package-archive.gnu.org' and the acronym GELPA (which recalls
ELPA but is qualified).

`Lisp' could be dropped, if this is the only Emacs package archive.  And
`archive' could be dropped by using `packages'.  For the URL, as opposed to the
long name, the prefix `gnu-' could be dropped if length is a problem (but why
would it be?):

URL:    emacs-packages.gnu.org
Name:   GNU Emacs Packages
Abbrev: GEP

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 04:41 AM, John Wiegley wrote:

> An exception to this rule is when a certain service (say, streams) should
> always be available, without requiring further installation of libraries.
> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same kinds
> of things we'd like to have in such a meta-library, should be in core.

Why not consider ELPA a part of the "standard library", too?

As long as all code that uses streams.el is not in Emacs core, I don't 
see any harm in requiring the user to install streams from ELPA (which 
would probably happen automatically, via a declared dependency).

When the core starts using it, sure, we can move it there.

> I do think that applications using such libraries should almost always go in
> ELPA, except for those that have been grandfathered in, like Emacs Calc. There
> are some things you should always be able to reach for, no matter whose
> machine you are on, or if the Internet is currently available.

Yes, some pieces of functionality that we consider a part of the 
standard set (Calc is not that, for me, but apparently it's popular 
enough). The criteria for applications to include them might be that 
Emacs has dedicated menu items, or uses the application's commands in 
some global bindings (that's why xref, for example, can't be moved to ELPA).

Re: ELPA policy

[David Kastrup]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> I think the rule for moving new stuff out of ELPA should be whether it's
>> used by the core. That's for libraries.
>
> An exception to this rule is when a certain service (say, streams)
> should always be available, without requiring further installation of
> libraries.

What does "service" and "should always be available" mean?  If it is not
end-user functionality useful for ad-hoc code entered with M-: or its
ilk (it would appear that streams is moving there), then I don't see a
necessity for including in the core: it makes more sense to put it as a
dependency in ELPA.  That way, other ELPA packages may easily require it
without depending on an upgrade of the core Emacs.

> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same
> kinds of things we'd like to have in such a meta-library, should be in
> core.

Emacs cannot be upgraded using the package manager, so any dependencies
on Emacs core code are harder to cater for than a dependency on another
ELPA package.

-- 
David Kastrup

Re: ELPA policy

[Artur Malabarba]

2015-11-05 3:00 GMT+00:00 Dmitry Gutov <dgutov@yandex.ru>:
> On 11/05/2015 04:41 AM, John Wiegley wrote:
>
>> An exception to this rule is when a certain service (say, streams) should
>> always be available, without requiring further installation of libraries.
>> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same
>> kinds
>> of things we'd like to have in such a meta-library, should be in core.
>
>
> Why not consider ELPA a part of the "standard library", too?
>
>[...]
> When the core starts using it, sure, we can move it there.

I agree in a general sense. But case-by-case discretion will have to be applied.

> The
> criteria for applications to include them might be that Emacs has dedicated
> menu items, or uses the application's commands in some global bindings
> (that's why xref, for example, can't be moved to ELPA).

IMO, xref can't be moved to Gelpa because users expect this "find
definition" functionality out-of-the box. By which I don't mean "Emacs
users have come to expect it", I mean "users coming to Emacs will
expect it".

Re: ELPA policy

[Michael Welsh Duggan]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/05/2015 04:41 AM, John Wiegley wrote:
>
>> An exception to this rule is when a certain service (say, streams) should
>> always be available, without requiring further installation of libraries.
>> Emacs acts as a sort of "standard library" for Emacs Lisp, so the same kinds
>> of things we'd like to have in such a meta-library, should be in core.
>
> Why not consider ELPA a part of the "standard library", too?

Because it is not.  I have often had to use Emacs on computers that did
not have access to an Internet connection.  To consider something part
of the "standard library," I believe you should have access to it when
Emacs is installed.  This is why I dislike the move towards systems that
only provide easy installation via network channels.  (Python, this
means you!)

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 02:51 PM, Michael Welsh Duggan wrote:

>> Why not consider ELPA a part of the "standard library", too?
>
> Because it is not.  I have often had to use Emacs on computers that did
> not have access to an Internet connection.  To consider something part
> of the "standard library," I believe you should have access to it when
> Emacs is installed.

We do intend to bundle some packages from ELPA to be distributed with 
Emacs releases.

Anyway, if you don't have Internet access, you also can't install any 
package that depends on the library in question. Right?

Re: ELPA policy

[Michael Welsh Duggan]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/05/2015 02:51 PM, Michael Welsh Duggan wrote:
>
>>> Why not consider ELPA a part of the "standard library", too?
>>
>> Because it is not.  I have often had to use Emacs on computers that did
>> not have access to an Internet connection.  To consider something part
>> of the "standard library," I believe you should have access to it when
>> Emacs is installed.
>
> We do intend to bundle some packages from ELPA to be distributed with
> Emacs releases.

In this case, I can consider these part of a standard library.

> Anyway, if you don't have Internet access, you also can't install any
> package that depends on the library in question. Right?

Not so.  Packages can still be installed via sneakernet, and elisp can
be written that wants to take advantage of a "standard library."  I have
no objection to ELPA in general.  I just have an issue with the idea
that all of ELPA (non-bundled) can be considered "standard."

-- 
Michael Welsh Duggan
(mwd@cert.org)

Re: ELPA policy

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> We do intend to bundle some packages from ELPA to be distributed with Emacs
> releases.

I'd like to know more about this. How does the user access these bundled
packages; by installing them with M-x package-install?

John

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 05:09 PM, John Wiegley wrote:

> I'd like to know more about this. How does the user access these bundled
> packages; by installing them with M-x package-install?

By having them in the default load-path somewhere already, I imagine.

The user would be able to M-x package-install, however, to install newer 
a version of the same package from ELPA, that was released in the meantime.

I think Fabian was looking into making that happen, but I can't find the 
respective thread right now.

Re: ELPA policy

[Artur Malabarba]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/05/2015 05:09 PM, John Wiegley wrote:
>
>> I'd like to know more about this. How does the user access these bundled
>> packages; by installing them with M-x package-install?
>
> By having them in the default load-path somewhere already, I imagine.

There are a few possible levels this could go.
But yes, the basic idea is that these bundled packages would be pulled
from Gelpa into some subdir of emacs/lisp when making the tarball and
autoloading/byte-compilation would be done. So they'd be available like
any other built-in package.

The advantage over just keeping the packages there in the first place is
that packages offered on GElpa can be updated more frequently.

A second level would be to do this in such a way that allows Emacs
packages to depend on these pulled-in Gelpa packages. This would require
changes to the build process, not just the tarball.

> I think Fabian was looking into making that happen, but I can't find
> the respective thread right now.

I think I asked about this a couple of months ago, and Stefan (?) wrote
a very nice explanation of what his thoughts were on this possible
future.

Re: ELPA policy

[Dmitry Gutov]

On 11/05/2015 06:58 PM, Artur Malabarba wrote:

>> I think Fabian was looking into making that happen, but I can't find
>> the respective thread right now.
>
> I think I asked about this a couple of months ago, and Stefan (?) wrote
> a very nice explanation of what his thoughts were on this possible
> future.

Stefan, most likely.  Too bad it wasn't in a dedicated bug discussion.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'd like to know more about this. How does the user access these bundled
  > packages; by installing them with M-x package-install?

I think they should appear, to the user, like any other standard part of Emacs.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

>> I'd like to know more about this. How does the user access these bundled
>> packages; by installing them with M-x package-install?

> I think they should appear, to the user, like any other standard part of
> Emacs.

If that's the case, it changes my thoughts on what needs to be in core, and
what should be in ELPA. Until now I was thinking ELPA required Internet
access; but if there are parts of ELPA that "come in the box", then I'd like
to see more packages there.

John

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Sun, 08 Nov 2015 08:30:17 -0800
> Cc: mwd@md5i.com, emacs-devel@gnu.org, dgutov@yandex.ru
> 
> If that's the case, it changes my thoughts on what needs to be in core, and
> what should be in ELPA. Until now I was thinking ELPA required Internet
> access; but if there are parts of ELPA that "come in the box", then I'd like
> to see more packages there.

ELPA requires Internet access if you are building from the repository.

My greatest fears are that packages that move from core to ELPA will
get much less attention from the core maintenance team, which will
eventually degrade their quality.

Re: ELPA policy

[Achim Gratz]

John Wiegley writes:
> I would in fact like to see more packages move from core into ELPA, although
> I'm not eager to do so willy nilly. Some packages now in Core are just fun, or
> should be there because people are used to them being there, even if today we
> wouldn't necessarily add that package to core.

I've proposed this before, but I guess I should run it by you again (and
warn you it wasn't favorably received):

The current definition of "core" is that the sources live inside the
Emacs repository.  Several of the larger core packages like Org, CEDET
and Gnus are already largely developed outside Emacs, for instance
because the developers want to keep them compatible with different/older
Emacsen and need to be synced into Emacs sources to make them "core"
anyway.

I posit that the only thing that actually matters for something to be
considered "core" is that authors of other packages can rely on the
(stable) API provided by these packages to be available in an Emacs as
it gets distributed and no installation of further packages or software
is necessary, neither by the sysadmin nor the user.  If so, instead of
keeping the "core" sources all in Emacs, they could equally well be
living in ELPA and be pre-installed into the distribution, or installed
into the Emacs build tree as submodules or subtrees.  The most radical
(and likely most controversial) thing to do would be to move everything
to ELPA that isn't needed to bootstrap Emacs.

Doing this would need some as of yet non-existing infrastructure to get
the chosen ELPA version of each package built into the distribution, and
facilities for sysadmins and users to update (but not disable) the
"core" packages at the system level or in their private directories.
Emacs may eventually distribute some "non-core" packages also that
sysadmins or users could disable if they chose to not use them.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: ELPA policy

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sun, 08 Nov 2015 18:18:26 +0100
> 
> I posit that the only thing that actually matters for something to be
> considered "core" is that authors of other packages can rely on the
> (stable) API provided by these packages to be available in an Emacs as
> it gets distributed and no installation of further packages or software
> is necessary, neither by the sysadmin nor the user.  If so, instead of
> keeping the "core" sources all in Emacs, they could equally well be
> living in ELPA and be pre-installed into the distribution, or installed
> into the Emacs build tree as submodules or subtrees.

IMO, no serious move such as this one should be argued for, let alone
attempted, without some minimal analysis of advantages and
disadvantages.  In particular, such an analysis cannot be limited to
the POV of maintainers of packages tat are currently not bundled, it
must first and foremost look at this from the POV of the Emacs
maintenance, definitely if the argument is to leave in the Emacs
repository only what's needed for bootstrap.

And any change in maintenance routine, small or large, should have
enough advantages to justify the energy that will certainly go into
the move itself and into cleaning up the resulting fallout.

I don't see how we can seriously discuss such suggestions when they
are not accompanied by anything like the analysis they need.

> The most radical (and likely most controversial) thing to do would
> be to move everything to ELPA that isn't needed to bootstrap Emacs.

Most such packages don't have any active maintainers, i.e. they are
maintained by the "FSF", which means us the core developers.  IMO, it
makes very little sense to spread the stuff we maintain between 2
separate repositories, because all this does is add overhead and
complexity without any clear benefits that I could see.

Another important aspect that this suggestion seems to overlook is
that Emacs packages rely on others not only via APIs, but also by
inheritance, like all the modes that derive from Text mode etc.

> Doing this would need some as of yet non-existing infrastructure to get
> the chosen ELPA version of each package built into the distribution, and
> facilities for sysadmins and users to update (but not disable) the
> "core" packages at the system level or in their private directories.

Yes, and the effort this will require is squarely in the disadvantages
camp.

Let me turn the table and ask: Are there any _advantages_ in moving
stuff like Dired, CC Mode, Shell Mode, Speedbar, and ps-print, to name
just a random few?

Re: ELPA policy

[Wolfgang Jenkner]

On Sun, Nov 08 2015, Eli Zaretskii wrote:

> My greatest fears are that packages that move from core to ELPA will
> get much less attention from the core maintenance team, which will
> eventually degrade their quality.

But this is not what actually happens with completely separately
developed complex packages like SLIME, Evil...

On the other hand, IIUC, moving CEDET to emacs core has not been
completely without problems for the CEDET project and its users.

Re: ELPA policy

[Eli Zaretskii]

> From: Wolfgang Jenkner <wjenkner@inode.at>
> Cc: John Wiegley <jwiegley@gmail.com>,  mwd@md5i.com,  emacs-devel@gnu.org,  rms@gnu.org,  dgutov@yandex.ru
> Date: Sun, 08 Nov 2015 19:00:06 +0100
> 
> On Sun, Nov 08 2015, Eli Zaretskii wrote:
> 
> > My greatest fears are that packages that move from core to ELPA will
> > get much less attention from the core maintenance team, which will
> > eventually degrade their quality.
> 
> But this is not what actually happens with completely separately
> developed complex packages like SLIME, Evil...

Most packages in core are not of this kind.

> On the other hand, IIUC, moving CEDET to emacs core has not been
> completely without problems for the CEDET project and its users.

Such moves are never without problems, which is exactly my point.  In
this case, I think the benefits far outweighed the problems: until
CEDET was incorporated into Emacs core, we couldn't even dream about
using it as infrastructure for IDEs and other similar features.
AFAIR, there was also lots of cleanup during the move.

But moving ELPA packages into the core is not being discussed or
suggested.  So what happened with CEDET, even if you disagree with my
assessment above, has little relevance to the issue at hand.

Re: ELPA policy

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> If that's the case, it changes my thoughts on what needs to be in core, and
>> what should be in ELPA. Until now I was thinking ELPA required Internet
>> access; but if there are parts of ELPA that "come in the box", then I'd like
>> to see more packages there.
>
> ELPA requires Internet access if you are building from the repository.
>
> My greatest fears are that packages that move from core to ELPA will
> get much less attention from the core maintenance team, which will
> eventually degrade their quality.

Org-mode, for instance, gets regular bug fix releases on GNU ELPA while
it stagnates on core. I would venture that the contribution from the
"core maintenance team" to org-mode amounts to a negative effect,
causing more hassle than it saves.

For projects with their own development team and VCS repository, having
the project copied into the Emacs core VCS makes no sense, moreover when
they have to support multiple Emacs releases.

Re: ELPA policy

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sun, 08 Nov 2015 19:26:20 +0100
> 
> For projects with their own development team and VCS repository, having
> the project copied into the Emacs core VCS makes no sense, moreover when
> they have to support multiple Emacs releases.

I could probably agree, as soon as we find a good way of getting the
"right" version of those when an Emacs tarball is being prepared (or
maybe even when Emacs is built from the repository).

But most packages in core are not of this variety, and it's them that
I worry about, not Org or Gnus.

Re: ELPA policy

[Eli Zaretskii]

> From: Artur Malabarba <bruce.connor.am@gmail.com>
> Cc: John Wiegley <jwiegley@gmail.com>,  emacs-devel@gnu.org,
> Date: Sun, 08 Nov 2015 19:27:08 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > My greatest fears are that packages that move from core to ELPA will
> > get much less attention from the core maintenance team, which will
> > eventually degrade their quality.
> 
> We could bring the Elpa and Emacs repositories closer together somehow.
> Maybe as a submodule or something like that.

How about just using a single repository? ;-)

Seriously, why should they be separate?  Submodules still complicate
things, compared to just having the stuff in the repo.

Re: ELPA policy

[Artur Malabarba]

Eli Zaretskii <eliz@gnu.org> writes:

> My greatest fears are that packages that move from core to ELPA will
> get much less attention from the core maintenance team, which will
> eventually degrade their quality.

We could bring the Elpa and Emacs repositories closer together somehow.
Maybe as a submodule or something like that.

This way the core maintenance team would also have the Elpa sources
"right there" to do bug fixes.

Re: ELPA policy

[Artur Malabarba]

John Wiegley <jwiegley@gmail.com> writes:

> If that's the case, it changes my thoughts on what needs to be in core, and
> what should be in ELPA. Until now I was thinking ELPA required Internet
> access; but if there are parts of ELPA that "come in the box", then I'd like
> to see more packages there.

I think we ought to clarify what we're talking about here.

- There are packages that are offered only on Elpa. For these you need
  an internet connection and you need to `M-x package-install' them.

- There are packages that are offered both on Elpa and Emacs. These
  are just like any other core package (no connection and no
  package-install necessary).

  The difference is that users can get access to new versions from Elpa
  without having to wait for the next Emacs release. The disadvantage
  here is code duplication, as someone needs to make sure that any bug
  fixes applied on the Emacs side also get applied on the Elpa side (and
  vice versa).

  Current examples of this are seq.el and let-alist.

- Then there's a third option. I only found this out recently while
  browsing the `externals-list' file on Elpa, and it was apparently
  implemented by Fabián in September.

  You can configure Elpa to (while building) copy the most recent
  version of an packages from the Emacs repository. IIUC, this would
  have the same effect as the previous option, except no code
  duplication. The thing is: no packages seem to be using this, and it's
  only documented in the comments of that file (though I've verified
  that there's indeed code to implement this).

Fabián, is this feature complete?

Re: ELPA policy

[Eli Zaretskii]

> From: Artur Malabarba <bruce.connor.am@gmail.com>
> Cc: jwiegley@gmail.com,  emacs-devel@gnu.org
> Date: Sun, 08 Nov 2015 20:04:56 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > How about just using a single repository? ;-)
> >
> > Seriously, why should they be separate?  Submodules still complicate
> > things, compared to just having the stuff in the repo.
> 
> I'm not strongly against that. But there are a couple of things that
> might be annoying with that:
> 
> - Elpa commits would occasionally flood the git log

I'm sure no one actually looks at too many commits at once, only those
that are of interest.

> - Git operations in both repositories would become even slower than they
>   are now (as they'd both increase in size).

GCC is migrating to Git as we speak, even though it has something like
3-4 times more commits in its history than we do.  If they aren't
afraid, I don't think we should be.

Re: ELPA policy

[Artur Malabarba]

Eli Zaretskii <eliz@gnu.org> writes:

> How about just using a single repository? ;-)
>
> Seriously, why should they be separate?  Submodules still complicate
> things, compared to just having the stuff in the repo.

I'm not strongly against that. But there are a couple of things that
might be annoying with that:

- Elpa commits would occasionally flood the git log

- Git operations in both repositories would become even slower than they
  are now (as they'd both increase in size).

Re: ELPA policy

[Dmitry Gutov]

On 11/08/2015 09:58 PM, Eli Zaretskii wrote:

> I'm sure no one actually looks at too many commits at once, only those
> that are of interest.

Filtering out the non-interesting commits is a hassle in itself, and I 
saying this as a emacs-diffs subscriber. With everyone committing to the 
same repo, it will become worse.

Re: ELPA policy

[Eli Zaretskii]

> Cc: jwiegley@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 8 Nov 2015 22:10:29 +0200
> 
> On 11/08/2015 09:58 PM, Eli Zaretskii wrote:
> 
> > I'm sure no one actually looks at too many commits at once, only those
> > that are of interest.
> 
> Filtering out the non-interesting commits is a hassle in itself, and I 
> saying this as a emacs-diffs subscriber. With everyone committing to the 
> same repo, it will become worse.

ELPA gets about 4 commits per day on average, so I don't think we
should worry about this aspect.

Re: ELPA policy

[Dmitry Gutov]

On 11/08/2015 10:26 PM, Eli Zaretskii wrote:

> ELPA gets about 4 commits per day on average, so I don't think we
> should worry about this aspect.

Emacs master has had ~82 commits in November, according to 'git log', 
which comes out to ~10 a day. With ELPA, this will be something like 40% 
increase?

And this is only while ELPA is relatively unpopular. If it ever grows to 
anything comparable to the size of MELPA (which we probably should 
aspire to), we will drown.

Re: ELPA policy

[Eli Zaretskii]

> Cc: bruce.connor.am@gmail.com, jwiegley@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 8 Nov 2015 22:36:00 +0200
> 
> On 11/08/2015 10:26 PM, Eli Zaretskii wrote:
> 
> > ELPA gets about 4 commits per day on average, so I don't think we
> > should worry about this aspect.
> 
> Emacs master has had ~82 commits in November, according to 'git log', 
> which comes out to ~10 a day. With ELPA, this will be something like 40% 
> increase?
> 
> And this is only while ELPA is relatively unpopular. If it ever grows to 
> anything comparable to the size of MELPA (which we probably should 
> aspire to), we will drown.

OK, then let's drop this idea.

Re: ELPA policy

[Aaron Ecay]

Hi Eli,

2015ko azaroak 8an, Eli Zaretskii-ek idatzi zuen:

[...]

> 
> Let me turn the table and ask: Are there any _advantages_ in moving
> stuff like Dired, CC Mode, Shell Mode, Speedbar, and ps-print, to name
> just a random few?

Imagine someone implements an awesome new feature for dired.  Emacs
users the world over are amazed by this, and fill their blogs, twitter,
etc. with the news.  If dired is an ELPA package, everyone who hears
this news can get the new feature in their emacs instantly by upgrading
their ELPA packages.  No need to wait N months for a new release of
emacs, or compile a non-release version of emacs from git.

Whether this counts as an advantage or not is complicated, and partially
depends on one’s point of view.  It would spell the end of an era where
each emacs release’s features are explicated through a year of excited
blogposts (for example
<http://endlessparentheses.com/tags-expanded.html#emacs-25>).  Users
will be able to choose a new model where features trickle in rather than
coming in sudden major chunks.  This might make it more complicated to
advocate adoption of new emacs versions.  But I don’t think so actually:
we regularly get big features at the language level too, which are very
exciting (for emacs 24 lexical binding, now dynamic modules and
xwidgets, and one dares to hope for the concurrency branch in emacs26).

I also think that discussion around this topic is distorted by a
long-tailed distribution.  Most people probably have in mind the big
emacs packages with dynamic developer communities and independent
release cycles.  Org, Gnus, CEDET, and maybe a few others.  On the other
hand, it seems to me that you have in mind (in addition to these) all
the tiny packages that see very few commits and perhaps no new features
(in the extreme case take kermit.el, which has seen no substantive
changes since at least 1992, but still gets its copyright header
lovingly updated every year).  I don’t know how to reconcile these
viewpoints, or even if it’s necessary.  Just something to consider.

Speaking for myself, my primary interest in emacs development is working
on org-mode, and I heavily use org as well as third-party packages for
python (elpy), clojure (cider), R (ESS), and a few other random things.
I build emacs from git every month or so in order to pick up little
quality of life improvements, like with-eval-afer-load, some of the
subr-x stuff, seq.el, the overhauled package menu, etc.  But I have to
make sure to keep a couple months’ worth of old emacsen around, in case
my monthly pull happens to land on a buggy commit – one that causes
regular segfaults, which I have had happen more than once.  I use emacs
for work so I don’t have the luxury of just going without for a few days
until the problem is fixed.  I could go back to a previous released
version of course, but then my init.el breaks because it relies on new
features since the release.

I’d be much happier running released versions of emacs for stability, if
it were easier to pull in new versions of core elisp libraries as they
are developed.  I suspect there are some others out there like me, and
many more who don’t currently build emacs from source for whatever reason
but would enjoy more regular access to new features in core elisp
libraries.

My 2 cents FWIW,
Aaron

PS I also have the impression that life would be easier for org’s
maintainers under a system that freed them from merging code back and
forth from org’s repo to emacs.  This would have indirect positive
effects on me as I work on org development.  However, I’m sure the
maintainers themselves – and maintainers of other projects in the same
situation like CEDET and Gnus – can speak to the question better than I
can.

-- 
Aaron Ecay

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Filtering out the non-interesting commits is a hassle in itself, and I 
  > saying this as a emacs-diffs subscriber. With everyone committing to the 
  > same repo, it will become worse.

I am sure we could set up multiple mailing lists and send each commit
to a list according to what directory it is in.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Wolfgang Jenkner]

On Sun, Nov 08 2015, Eli Zaretskii wrote:

>> On the other hand, IIUC, moving CEDET to emacs core has not been
>> completely without problems for the CEDET project and its users.
>
> Such moves are never without problems, which is exactly my point.

I was using the wrong word here as it's more a fork than a move:

http://sourceforge.net/p/cedet/git/ci/master/tree/INSTALL

> In
> this case, I think the benefits far outweighed the problems: until
> CEDET was incorporated into Emacs core, we couldn't even dream about
> using it as infrastructure for IDEs and other similar features.

CEDET being a separate project did not stop various people elsewhere
from realizing their dream about an "IDE" according to their own ideas:

https://www.youtube.com/results?search_query=emacs+cedet

(sorry for giving a link to a google service...)

> AFAIR, there was also lots of cleanup during the move.

> But moving ELPA packages into the core is not being discussed or
> suggested.  So what happened with CEDET, even if you disagree with my
> assessment above, has little relevance to the issue at hand.

To be clear: FWIW, I agree that CEDET is great and that using real
grammars for doing various stuff in emacs is an exciting idea.  I

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 01:16 AM, Richard Stallman wrote:

> I am sure we could set up multiple mailing lists and send each commit
> to a list according to what directory it is in.

You have a point there, and if someone were to actually do that, it 
would help with making the notifications manageable.

Still, the more happens in the repository, the harder it is to grasp or 
manage. Right now you can call 'git log' and see the most recent things 
that happened to Emacs master. Or do 'git pull', and call 'git diff' on 
the revision range that was printed out, and see all changes since your 
last update.

Unless the Emacs sources move into a subdirectory, even listing all 
commits that were made to Emacs recently, would involve the contributor 
learning magic like this: 
http://stackoverflow.com/questions/5685007/making-git-log-ignore-changes-for-certain-paths

Re: ELPA policy

[Yuri Khan]

On Mon, Nov 9, 2015 at 7:45 AM, Dmitry Gutov <dgutov@yandex.ru> wrote:

> Still, the more happens in the repository, the harder it is to grasp or
> manage. Right now you can call 'git log' and see the most recent things that
> happened to Emacs master. Or do 'git pull', and call 'git diff' on the
> revision range that was printed out, and see all changes since your last
> update.

I almost wanted to mention that you can restrict “git log” and “git
diff” to a particular path in the repository, but realized it might
start a new git subthread.

(FWIW I also think separate repositories are better, as long as
development is mostly independent across the divide.)

Re: ELPA policy

[Eli Zaretskii]

> From: Aaron Ecay <aaronecay@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Sun, 08 Nov 2015 20:52:12 +0000
> 
> Imagine someone implements an awesome new feature for dired.  Emacs
> users the world over are amazed by this, and fill their blogs, twitter,
> etc. with the news.  If dired is an ELPA package, everyone who hears
> this news can get the new feature in their emacs instantly by upgrading
> their ELPA packages.  No need to wait N months for a new release of
> emacs, or compile a non-release version of emacs from git.

How is this different when Dired is in the Emacs repository?  The
Emacs repository is a public one, so anyone and everyone can get the
latest version from there and use it, if they want.

> I also think that discussion around this topic is distorted by a
> long-tailed distribution.  Most people probably have in mind the big
> emacs packages with dynamic developer communities and independent
> release cycles.  Org, Gnus, CEDET, and maybe a few others.  On the other
> hand, it seems to me that you have in mind (in addition to these) all
> the tiny packages that see very few commits and perhaps no new features
> (in the extreme case take kermit.el, which has seen no substantive
> changes since at least 1992, but still gets its copyright header
> lovingly updated every year).  I don’t know how to reconcile these
> viewpoints, or even if it’s necessary.  Just something to consider.

The suggestion was to move _all_ of them, except the few that are
needed for bootstrap, out of the Emacs repository.  Most of the
packages in that category are neither like Org nor like kermit.  They
are relatively small, but get quite a significant number of changes.

> Speaking for myself, my primary interest in emacs development is working
> on org-mode, and I heavily use org as well as third-party packages for
> python (elpy), clojure (cider), R (ESS), and a few other random things.
> I build emacs from git every month or so in order to pick up little
> quality of life improvements, like with-eval-afer-load, some of the
> subr-x stuff, seq.el, the overhauled package menu, etc.  But I have to
> make sure to keep a couple months’ worth of old emacsen around, in case
> my monthly pull happens to land on a buggy commit – one that causes
> regular segfaults, which I have had happen more than once.  I use emacs
> for work so I don’t have the luxury of just going without for a few days
> until the problem is fixed.  I could go back to a previous released
> version of course, but then my init.el breaks because it relies on new
> features since the release.

I don't see how the issue at hand can affect you, then.  Your usage is
very similar to mine, FWIW.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 05:42 AM, Eli Zaretskii wrote:

>> Imagine someone implements an awesome new feature for dired.  Emacs
>> users the world over are amazed by this, and fill their blogs, twitter,
>> etc. with the news.  If dired is an ELPA package, everyone who hears
>> this news can get the new feature in their emacs instantly by upgrading
>> their ELPA packages.  No need to wait N months for a new release of
>> emacs, or compile a non-release version of emacs from git.
>
> How is this different when Dired is in the Emacs repository?  The
> Emacs repository is a public one, so anyone and everyone can get the
> latest version from there and use it, if they want.

a) That's a more involved endeavor than installing a package from ELPA. 
And then you don't get the same conveniences, such as automatic updates.

b) There's a much higher probability that Dired depends on something 
only the current development version of Emacs has. ELPA packages declare 
their version requirements explicitly, and try not to break 
compatibility with earlier versions without sufficient reasons.

> The suggestion was to move _all_ of them, except the few that are
> needed for bootstrap, out of the Emacs repository.  Most of the
> packages in that category are neither like Org nor like kermit.  They
> are relatively small, but get quite a significant number of changes.

There were different suggestions, with different degrees between "let's 
move Org and Gnus out" and "let's move everything out".

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Richard Stallman <rms@gnu.org> writes:
>
>>> I'd like to know more about this. How does the user access these bundled
>>> packages; by installing them with M-x package-install?
>
>> I think they should appear, to the user, like any other standard part of
>> Emacs.
>
> If that's the case, it changes my thoughts on what needs to be in core, and
> what should be in ELPA. Until now I was thinking ELPA required Internet
> access; but if there are parts of ELPA that "come in the box", then I'd like
> to see more packages there.

It depends on what git repository the package is in, and how the ELPA
packages are included in core.

My impression is that the latest proposal for doing this is to keep
"core ELPA" packages in the emacs repository (not the Gnu ELPA repository),
and fix the Gnu ELPA server scripts to work with that.

Then all core Emacs developers see the core ELPA packages in their
current workflow, and they are released along with Emacs.

They are also released separately to Gnu ELPA, and they appear in
`list-packages' (as other core packages do now); that's what makes them
"ELPA core packages" as opposed to "core code".

That's an elegant solution.


The alternative is to figure out how to merge a partial checkout of the
Gnu ELPA repository into an Emacs git checkout without confusing git and
make; not at all easy.

-- 
-- Stephe

Re: ELPA policy

[Eli Zaretskii]

> Cc: Stromeko@nexgo.de, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 9 Nov 2015 05:51:16 +0200
> 
> On 11/09/2015 05:42 AM, Eli Zaretskii wrote:
> 
> >> Imagine someone implements an awesome new feature for dired.  Emacs
> >> users the world over are amazed by this, and fill their blogs, twitter,
> >> etc. with the news.  If dired is an ELPA package, everyone who hears
> >> this news can get the new feature in their emacs instantly by upgrading
> >> their ELPA packages.  No need to wait N months for a new release of
> >> emacs, or compile a non-release version of emacs from git.
> >
> > How is this different when Dired is in the Emacs repository?  The
> > Emacs repository is a public one, so anyone and everyone can get the
> > latest version from there and use it, if they want.
> 
> a) That's a more involved endeavor than installing a package from ELPA. 
> And then you don't get the same conveniences, such as automatic updates.

Nothing prevents us from making the same arrangements in the Emacs
repository as we have in ELPA, and then users could add the Emacs
repository to their list of sites that package.el knows about.

Besides, most core packages don't need any elaborate setup, you just
drop them in and restart Emacs.

> b) There's a much higher probability that Dired depends on something 
> only the current development version of Emacs has. ELPA packages declare 
> their version requirements explicitly, and try not to break 
> compatibility with earlier versions without sufficient reasons.

How will that change if we move them outside Emacs?  It's not like
there's some magic at ELPA that just putting a package there magically
resolves all these issues.  And if someone will have to work on that,
she can do that without moving the package aside.

> > The suggestion was to move _all_ of them, except the few that are
> > needed for bootstrap, out of the Emacs repository.  Most of the
> > packages in that category are neither like Org nor like kermit.  They
> > are relatively small, but get quite a significant number of changes.
> 
> There were different suggestions, with different degrees between "let's 
> move Org and Gnus out" and "let's move everything out".

I didn't see any.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 06:05 PM, Eli Zaretskii wrote:

> Nothing prevents us from making the same arrangements in the Emacs
> repository as we have in ELPA, and then users could add the Emacs
> repository to their list of sites that package.el knows about.

Create yet another package archive? With a separate publishing mechanism?

> Besides, most core packages don't need any elaborate setup, you just
> drop them in and restart Emacs.

"Just drop them in" doesn't address the issue of package updates, not in 
the slightest.

> How will that change if we move them outside Emacs?  It's not like
> there's some magic at ELPA that just putting a package there magically
> resolves all these issues.  And if someone will have to work on that,
> she can do that without moving the package aside.

They will be forced to declare the required Emacs version, for one. And 
abide by it.

But yes, that's also possible to do inside the Emacs repo.

>> There were different suggestions, with different degrees between "let's
>> move Org and Gnus out" and "let's move everything out".
>
> I didn't see any.

All right, then. Let me make one right now: let's move out Org, Gnus and 
CEDET. And consider doing that with any other big codebase that's also 
maintained separately (Calc comes to mind).

There have been objections to doing that for each of these packages, but 
there you go. Suggestion made.

Re: ELPA policy

[Eli Zaretskii]

> Cc: aaronecay@gmail.com, Stromeko@nexgo.de, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 9 Nov 2015 18:15:28 +0200
> 
> On 11/09/2015 06:05 PM, Eli Zaretskii wrote:
> 
> > Nothing prevents us from making the same arrangements in the Emacs
> > repository as we have in ELPA, and then users could add the Emacs
> > repository to their list of sites that package.el knows about.
> 
> Create yet another package archive? With a separate publishing mechanism?

Why not?  There are several already, no?

> > Besides, most core packages don't need any elaborate setup, you just
> > drop them in and restart Emacs.
> 
> "Just drop them in" doesn't address the issue of package updates, not in 
> the slightest.

It does, for most of the core.  That's what you do each time you say
"git pull", I believe.

> >> There were different suggestions, with different degrees between "let's
> >> move Org and Gnus out" and "let's move everything out".
> >
> > I didn't see any.
> 
> All right, then. Let me make one right now: let's move out Org, Gnus and 
> CEDET. And consider doing that with any other big codebase that's also 
> maintained separately (Calc comes to mind).
> 
> There have been objections to doing that for each of these packages, but 
> there you go. Suggestion made.

It was made before -- that's the "let's move Org and Gnus out"
variant, to which I said I could easily agree.  And then there was the
"move everything" one, to which I object for the reasons stated.  What
I meant was that there was no 3rd variant, AFAIR.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 06:20 PM, Eli Zaretskii wrote:

>> Create yet another package archive? With a separate publishing mechanism?
>
> Why not?  There are several already, no?

So far, it seems we're short of manpower to manage even the current one. 
To manage the infrastructure, improve the scripts, etc.

> It does, for most of the core.  That's what you do each time you say
> "git pull", I believe.

I cannot "git pull" a new version of Dired into my stable installation 
of Emacs 24.5.

> It was made before -- that's the "let's move Org and Gnus out"
> variant, to which I said I could easily agree.  And then there was the
> "move everything" one, to which I object for the reasons stated.  What
> I meant was that there was no 3rd variant, AFAIR.

Well, I enumerated 4 packages instead of 2, didn't I? :)

I agree that "move everything" is a non-starter (or very hard, at 
least). So it seems more like a red herring, there's no real use arguing 
against it.

Re: ELPA policy

[Artur Malabarba]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Nothing prevents us from making the same arrangements in the Emacs
>> > repository as we have in ELPA, and then users could add the Emacs
>> > repository to their list of sites that package.el knows about.
>> 
>> Create yet another package archive? With a separate publishing mechanism?
>
> Why not?  There are several already, no?

This sounds already quite similar to the system that Fabian implemented
a couple months ago (which I hear isn't quite perfect yet, but should be
almost there).

With this system, the daily script that Gelpa runs would copy some
packages straight from Emacs core. This way you could very well
configure Gelpa to always offer the lastest Dired (or maybe only when we
bump a version number, but that's a detail).

Re: ELPA policy

[Eli Zaretskii]

> Cc: aaronecay@gmail.com, Stromeko@nexgo.de, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 9 Nov 2015 18:26:58 +0200
> 
> On 11/09/2015 06:20 PM, Eli Zaretskii wrote:
> 
> >> Create yet another package archive? With a separate publishing mechanism?
> >
> > Why not?  There are several already, no?
> 
> So far, it seems we're short of manpower to manage even the current one. 
> To manage the infrastructure, improve the scripts, etc.

Is it really such a significant effort?  I wasn't aware of that.

> > It does, for most of the core.  That's what you do each time you say
> > "git pull", I believe.
> 
> I cannot "git pull" a new version of Dired into my stable installation 
> of Emacs 24.5.

Of course you can: just copy the file over.

> > It was made before -- that's the "let's move Org and Gnus out"
> > variant, to which I said I could easily agree.  And then there was the
> > "move everything" one, to which I object for the reasons stated.  What
> > I meant was that there was no 3rd variant, AFAIR.
> 
> Well, I enumerated 4 packages instead of 2, didn't I? :)

We could probably find a few more.  MH-E, for example.  (Of course,
their developers will have to agree to move.)

> I agree that "move everything" is a non-starter (or very hard, at 
> least). So it seems more like a red herring, there's no real use arguing 
> against it.

Agreed.

Re: ELPA policy

[Dmitry Gutov]

On 11/09/2015 06:44 PM, Eli Zaretskii wrote:

>> So far, it seems we're short of manpower to manage even the current one.
>> To manage the infrastructure, improve the scripts, etc.
>
> Is it really such a significant effort?  I wasn't aware of that.

There are things to be done, and just a couple months ago nobody was 
doing them. I see some movement now, e.g. the last commit by Thomas 
Fitzsimmons.

>> I cannot "git pull" a new version of Dired into my stable installation
>> of Emacs 24.5.
>
> Of course you can: just copy the file over.

Automatic update is a really handy feature when you have 50+ packages 
installed. See some user statistics at 
https://www.reddit.com/r/emacs/comments/3s4k03/how_many_packages_do_you_use/

> We could probably find a few more.  MH-E, for example.  (Of course,
> their developers will have to agree to move.)

Yeah, I bet the majority of our users are unaware of what MH-E is.

Re: ELPA policy

[Achim Gratz]

Eli Zaretskii writes:
> The suggestion was to move _all_ of them, except the few that are
> needed for bootstrap, out of the Emacs repository.

You keep arguing about things that realistically aren't even on the
table yet (I don't want to prevent that discussion, mind you).  I said
it would be possible to do this, given some not-yet-existing
infrastructure and the only reason I even mentioned it was because
that's the other end of the spectrum from moving everything that looks
halfway useful into core.  Even if that infrastructure develops (which I
have serious doubts about given how this discussion goes), nobody in
their right mind would move all packages big-bang style to ELPA.
There'd certainly be a transition period with some pilot packages, I'd
probably pick Org, CEDET and Tramp based on the list traffic.  You'd
then see how that works and make some fixes that will certainly be
necessary before you get to the point to move more packages over to
ELPA.  This being Emacs, for better or worse, I'd expect that at this
point we'd be looking at _years_ before the extreme end you seem to be
so worried about is even within reach.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: ELPA policy

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Mon, 09 Nov 2015 19:22:50 +0100
> 
> Eli Zaretskii writes:
> > The suggestion was to move _all_ of them, except the few that are
> > needed for bootstrap, out of the Emacs repository.
> 
> You keep arguing about things that realistically aren't even on the
> table yet (I don't want to prevent that discussion, mind you).

If that's not on the table, then please tell what in your opinion _is_
on the table.  You did want to propose something practical that could
be done soon, right?  Because this discussion was about what we want
to do from now on with ELPA, not what we might want considering in
some distant future that might never come.

So please, let's be practical and consider alternatives that
realistically can be taken soon.  Like tomorrow or the next month.
Okay?

> There'd certainly be a transition period with some pilot packages, I'd
> probably pick Org, CEDET and Tramp based on the list traffic.

Moving out a few large packages that are developed outside Emacs
anyway is a no-brainer, provided that their developers agree.  I
already said I'll probably agree, and no one else objected.  The only
question is what is the opinion of the respective developers: we
cannot move the packages if they disagree.

Is there some other alternative on the table?  If not, we can finally
agree about something, I think.

Thanks.

Re: ELPA policy

[David Engster]

Eli Zaretskii writes:
>> From: Achim Gratz <Stromeko@nexgo.de>
>> Date: Mon, 09 Nov 2015 19:22:50 +0100

>> There'd certainly be a transition period with some pilot packages, I'd
>> probably pick Org, CEDET and Tramp based on the list traffic.
>
> Moving out a few large packages that are developed outside Emacs
> anyway is a no-brainer, provided that their developers agree.  I
> already said I'll probably agree, and no one else objected.  The only
> question is what is the opinion of the respective developers: we
> cannot move the packages if they disagree.

As long as Emacs core cannot depend on packages that are bundled from
ELPA, I would definitely not agree to move CEDET to ELPA.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Date: Mon, 09 Nov 2015 19:58:29 +0100
> Cc: Achim Gratz <Stromeko@nexgo.de>, emacs-devel@gnu.org
> 
> As long as Emacs core cannot depend on packages that are bundled from
> ELPA, I would definitely not agree to move CEDET to ELPA.

Of course.  I don't think we should move anything before such a
dependency is possible.

Re: ELPA policy

[David Engster]

Eli Zaretskii writes:
>> From: David Engster <deng@randomsample.de>
>> Date: Mon, 09 Nov 2015 19:58:29 +0100
>> Cc: Achim Gratz <Stromeko@nexgo.de>, emacs-devel@gnu.org
>> 
>> As long as Emacs core cannot depend on packages that are bundled from
>> ELPA, I would definitely not agree to move CEDET to ELPA.
>
> Of course.  I don't think we should move anything before such a
> dependency is possible.

But if you have such dependencies, what's the point? IIRC, Stefan
decidedly did not want such dependencies, since the whole point was to
add those bundles only when creating releases.

-David

Re: ELPA policy

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> It was made before -- that's the "let's move Org and Gnus out" variant, to
> which I said I could easily agree. And then there was the "move everything"
> one, to which I object for the reasons stated. What I meant was that there
> was no 3rd variant, AFAIR.

Wow, a lot of discussion about ELPA policy, this is great. We definitely have
an opportunity here to bring clarity to an area that is on people's minds.

I agree with a lot of what I read, although it was a too spread out for me to
add specific quotes here. Let me just summarize a possible path forward:

 1. Things that are maintained by the core Emacs developers should be in core,
    and in the Emacs.git repository. This makes it easy for them to access and
    build, search history, read emacs-diffs, etc.

 2. Things that are maintained outside of Emacs, but contributed back for
    inclusion, should be in ELPA, and in the Elpa.git repository. This
    includes Gnus, Org, CEDET, and a few more. (We don't want to go crazy,
    so let's start with the big ones).

 3. There should be a defined subset of packages that get copied from ELPA
    into the Emacs tarball during release, and an easy way to manage this list
    for the core developers. That way, certain packages like seq.el and
    stream.el can feel like "core" packages to users, when they are really
    "external" packages from the point of view of the core developers.

 4. Everything else in ELPA is Internet-installation based, as it is today.

I would very much like for ELPA to be a way to:

  - give the core developers a smaller surface area to focus on;

  - allow ELPA package maintainers to "have their package in Emacs", while
    retaining nimble in their own development process and in delivering
    updates to users;

  - allow the core maintainers to decide what exactly is going into "Emacs":
    For example, I wouldn't want to pull Org releases into the distribution
    from a submodule; I really do want a version of that source code to be in
    ELPA, so we can separately patch if there are last minute problems.

ELPA should thus benefit core developers by reducing load, and benefit package
maintainers by being an easier platform for their contributions and their
users. If it causes extra work for anyone, that's something I'd want to
change.

There are three actions I'd like to take from here:

  a. That we clearly document an ELPA policy we all agree on;

  b. That we move "external" packages from core into ELPA, starting with the
     really big ones;

  c. That we assess any points of friction after doing so, and adjust as
     necessary.

John

Re: ELPA policy

[Rasmus]

Eli Zaretskii <eliz@gnu.org> writes:

> Moving out a few large packages that are developed outside Emacs
> anyway is a no-brainer, provided that their developers agree.  I
> already said I'll probably agree, and no one else objected.  The only
> question is what is the opinion of the respective developers: we
> cannot move the packages if they disagree.

I'm pretty sure Org would be happy to be distributed this way.  In fact,
we have got the 8.3 release now.  It really *must* be in 25.1, but I think
it would be fine to try out Fabian’s system to allow it to be upgraded
later.  Otherwise, I will hopefully be able to merge it soon the
"copy-paste" way.

Rasmus


-- 
However beautiful the theory, you should occasionally look at the evidence

Re: ELPA policy

[Achim Gratz]

Eli Zaretskii writes:
> If that's not on the table, then please tell what in your opinion _is_
> on the table.  You did want to propose something practical that could
> be done soon, right?  Because this discussion was about what we want
> to do from now on with ELPA, not what we might want considering in
> some distant future that might never come.

I did want to propose something that could become a vision for some time
in the foreseeable, yet somewhat distant future that we can then take
steps towards.  ELPA is foremost for _users_ and any decisions taken
should put the users first and developers second.  As a user I don't
have the luxury of chosing which Emacs version I get to use and what
packages are already installed on the system (indeed in my day job that
changes when I switch between project workspaces).  Wouldn't it be
splendid if ELPA helped me making those differences fade into the
background, if not disappear entirely?

> So please, let's be practical and consider alternatives that
> realistically can be taken soon.  Like tomorrow or the next month.
> Okay?

No, not okay.  There's still no consensus of even roughly where we want
to end up, so starting a random walk to find out who's feeling well or
not with whichever direction is certainly not my idea of progress.

> Moving out a few large packages that are developed outside Emacs
> anyway is a no-brainer, provided that their developers agree.

Again, it's not.  It simply doesn't work at this moment, not for Emacs,
not for the package developers and not for the users.  That doesn't have
anything to do with the size of those packages best I can tell.  To
change that (without moving the sources anywhere, just to not go down
that rabbit hole again) we'd need to devise a way to have an Emacs core
package whose autoload definitions can be separated and that can be
overridden (maybe only with a newer version) on the installation level
or by the user.  That would be the first step before any others can
follow, provided you even want to go that direction.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

DIY Stuff:
http://Synth.Stromeko.net/DIY.html

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: Stromeko@nexgo.de,  emacs-devel@gnu.org
> Date: Mon, 09 Nov 2015 20:35:04 +0100
> 
> Eli Zaretskii writes:
> >> From: David Engster <deng@randomsample.de>
> >> Date: Mon, 09 Nov 2015 19:58:29 +0100
> >> Cc: Achim Gratz <Stromeko@nexgo.de>, emacs-devel@gnu.org
> >> 
> >> As long as Emacs core cannot depend on packages that are bundled from
> >> ELPA, I would definitely not agree to move CEDET to ELPA.
> >
> > Of course.  I don't think we should move anything before such a
> > dependency is possible.
> 
> But if you have such dependencies, what's the point? IIRC, Stefan
> decidedly did not want such dependencies, since the whole point was to
> add those bundles only when creating releases.

I'm not pushing for the move, I just said that if the technical
problems are successfully resolved, I won't object.

Re: ELPA policy

[Achim Gratz]

John Wiegley writes:
>  3. There should be a defined subset of packages that get copied from ELPA
>     into the Emacs tarball during release, and an easy way to manage this list
>     for the core developers. That way, certain packages like seq.el and
>     stream.el can feel like "core" packages to users, when they are really
>     "external" packages from the point of view of the core developers.

They could (and probably should) effectively be in the build environment
as well.  In other words, I would try to not pull them into the release
tarball at the last possible moment.  One thing that would make
difficult is to test interactions between such "external core" packages
and here I have to agree with Eli that this has potential for
degradation of quality.

>   - allow the core maintainers to decide what exactly is going into "Emacs":
>     For example, I wouldn't want to pull Org releases into the distribution
>     from a submodule; I really do want a version of that source code to be in
>     ELPA, so we can separately patch if there are last minute problems.

You'd do that with branches if it really becomes necessary.  I don't
really see why submodules could not be used, except for the extra rope
they give you to hang yourself with.  Any other solution is going to
have the same basic complexity beneath and the potential to not work on
some platform or other.

> ELPA should thus benefit core developers by reducing load, and benefit package
> maintainers by being an easier platform for their contributions and their
> users. If it causes extra work for anyone, that's something I'd want to
> change.

I maintain that ELPA should benefit Emacs users first.

>   b. That we move "external" packages from core into ELPA, starting with the
>      really big ones;

...after the necessary changes to Emacs' build system and package.el are
architected and implemented.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Re: ELPA policy

[John Wiegley]

>>>>> Achim Gratz <Stromeko@nexgo.de> writes:

> They could (and probably should) effectively be in the build environment as
> well. In other words, I would try to not pull them into the release tarball
> at the last possible moment. One thing that would make difficult is to test
> interactions between such "external core" packages and here I have to agree
> with Eli that this has potential for degradation of quality.

Could this be solved by branching ELPA leading up to a release -- this being
the "frozen ELPA" that we do integration against before cutting the tarball --
while cherry-picking fixes in from master? I think the way we proceed here
would be similar to what we do for Emacs itself.

> You'd do that with branches if it really becomes necessary. I don't really
> see why submodules could not be used, except for the extra rope they give
> you to hang yourself with. Any other solution is going to have the same
> basic complexity beneath and the potential to not work on some platform or
> other.

A submodule doesn't allow us to have "Org + some-patch-not-yet-in-Org".

> I maintain that ELPA should benefit Emacs users first.

I'm intrigued; could you clarify this a bit more? I saw what you wrote in
response to Eli, but that didn't really help. Specifically, do you have a
scenario in mind that is better for users and worse for developers?

>> b. That we move "external" packages from core into ELPA, starting with the
>> really big ones;

> ...after the necessary changes to Emacs' build system and package.el are
> architected and implemented.

Agreed.

John

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I agree with a lot of what I read, although it was a too spread out for me to
  > add specific quotes here. Let me just summarize a possible path forward:

This plan seems good, overall, But in the specific case of stream.el
and seq.el, shouldn't they be maintained in the core?  I don't see
stream.el in the sources I fetched, but seq.el is small, in emacs-lisp,
and it says Maintainer: emacs-devel.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> This plan seems good, overall, But in the specific case of stream.el and
> seq.el, shouldn't they be maintained in the core? I don't see stream.el in
> the sources I fetched, but seq.el is small, in emacs-lisp, and it says
> Maintainer: emacs-devel.

If that's the case, then yes, it should be in core. I'm personally a bit
surprised that a library of this nature is in ELPA, unless it was being
provided and actively maintained by a non-core developer.

John

Re: ELPA policy

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 748 bytes --]

On 9 Nov 2015 10:06 pm, "John Wiegley" <jwiegley@gmail.com> wrote:
>
> >>>>> Richard Stallman <rms@gnu.org> writes:
>
> > This plan seems good, overall, But in the specific case of stream.el and
> > seq.el, shouldn't they be maintained in the core? I don't see stream.el
in
> > the sources I fetched, but seq.el is small, in emacs-lisp, and it says
> > Maintainer: emacs-devel.
>
> If that's the case, then yes, it should be in core. I'm personally a bit
> surprised that a library of this nature is in ELPA, unless it was being
> provided and actively maintained by a non-core developer.

Seq.el was put on Gelpa so that it can be used by emacsen < 25. I'm very
grateful for that as I use it as a dependency in quite a few of my packages
already.

[-- Attachment #2: Type: text/html, Size: 1009 bytes --]

Re: ELPA policy

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]

John Wiegley <jwiegley@gmail.com> writes:

>  3. There should be a defined subset of packages that get copied from ELPA
>     into the Emacs tarball during release, and an easy way to manage this list
>     for the core developers. That way, certain packages like seq.el and
>     stream.el can feel like "core" packages to users, when they are really
>     "external" packages from the point of view of the core developers.

I only added seq.el to ELPA for backward compatibility with Emacs 24.5.
The development of seq.el (and map.el for that matter) happen only in
Emacs trunk.

Moving it out of lisp/emacs-lisp/, and only having it copied in the
release tarballs would be a step backward IMHO, it would make it hard to
use it in core packages, and would basically mean demoting the libraries
to external ones while I'm really working on seq.el to provide a good,
consistent and comprehensive *built-in* sequence library to
Emacs-Lisp. That's also why all the functions are documented in the
Sequences chapter of the Elisp documentation.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: ELPA policy

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 424 bytes --]

Richard Stallman <rms@gnu.org> writes:

> This plan seems good, overall, But in the specific case of stream.el
> and seq.el, shouldn't they be maintained in the core?  I don't see
> stream.el in the sources I fetched, but seq.el is small, in emacs-lisp,
> and it says Maintainer: emacs-devel.

stream.el is currently in ELPA, which can make some sense, but regarding
seq.el, I think it should really stay in the core.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: ELPA policy

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 538 bytes --]

John Wiegley <jwiegley@gmail.com> writes:

> If that's the case, then yes, it should be in core. I'm personally a bit
> surprised that a library of this nature is in ELPA,

In the case of seq.el, it's for backward-compatibility.  It made it
possible for instance for CIDER devs to start using it while Emacs 25 is
still in progress.

> unless it was being
> provided and actively maintained by a non-core developer.

IIRC, Stefan thought that having stream.el in the ELPA meant more sense
at least until people start using it more.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: ELPA policy

[Aaron Ecay]

Hi John,

Thanks for setting this out very concretely.

2015ko azaroak 9an, John Wiegley-ek idatzi zuen:
> 
>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
>> It was made before -- that's the "let's move Org and Gnus out" variant, to
>> which I said I could easily agree. And then there was the "move everything"
>> one, to which I object for the reasons stated. What I meant was that there
>> was no 3rd variant, AFAIR.
> 
> Wow, a lot of discussion about ELPA policy, this is great. We definitely have
> an opportunity here to bring clarity to an area that is on people's minds.
> 
> I agree with a lot of what I read, although it was a too spread out for me to
> add specific quotes here. Let me just summarize a possible path forward:
> 
>  1. Things that are maintained by the core Emacs developers should be in core,
>     and in the Emacs.git repository. This makes it easy for them to access and
>     build, search history, read emacs-diffs, etc.
> 
>  2. Things that are maintained outside of Emacs, but contributed back for
>     inclusion, should be in ELPA, and in the Elpa.git repository. This
>     includes Gnus, Org, CEDET, and a few more. (We don't want to go crazy,
>     so let's start with the big ones).
> 
>  3. There should be a defined subset of packages that get copied from ELPA
>     into the Emacs tarball during release, and an easy way to manage this list
>     for the core developers. That way, certain packages like seq.el and
>     stream.el can feel like "core" packages to users, when they are really
>     "external" packages from the point of view of the core developers.
> 
>  4. Everything else in ELPA is Internet-installation based, as it is
>     today.

The one thing that I see missing from this list is a way to take
packages that are developed in emacs.git and distribute them on an
ELPA.  (Either GNU ELPA, or a (g)new ELPA as was suggested by Eli.)
This would be the inverse of your 3: packages that feel like core
packages to emacs devs, but feel like external packages to users (at
least in the sense of receiving updates outside of the emacs release
cycle).

I’m not sure if that omission was intentional or not.  I hope it wasn’t:
this model of distribution has been chosen for several important
“low-level” packages, like seq, cl-lib, and let-alist.

If this addition is desired then:

[...]

> 
> There are three actions I'd like to take from here:
> 
>   a. That we clearly document an ELPA policy we all agree on;
> 
>   b. That we move "external" packages from core into ELPA, starting with the
>      really big ones;
> 
>   c. That we assess any points of friction after doing so, and adjust as
>      necessary.

This list needs an additional item, namely the development of the
script that publishes elisp from emacs.git to an ELPA.  I think people
have said that Fabian is already working on this.

Thanks,

-- 
Aaron Ecay

Re: ELPA policy

[John Wiegley]

>>>>> Aaron Ecay <aaronecay@gmail.com> writes:

> I’m not sure if that omission was intentional or not. I hope it wasn’t: this
> model of distribution has been chosen for several important “low-level”
> packages, like seq, cl-lib, and let-alist.

[...]

> This list needs an additional item, namely the development of the script
> that publishes elisp from emacs.git to an ELPA. I think people have said
> that Fabian is already working on this.

It was an unintentional omission, and I agree that we should support that
usage pattern as well. So let it be 4 modes of looking at "Emacs packages",
then.

John

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>
>> It was made before -- that's the "let's move Org and Gnus out" variant, to
>> which I said I could easily agree. And then there was the "move everything"
>> one, to which I object for the reasons stated. What I meant was that there
>> was no 3rd variant, AFAIR.
>
> Wow, a lot of discussion about ELPA policy, this is great. We definitely have
> an opportunity here to bring clarity to an area that is on people's minds.
>
> I agree with a lot of what I read, although it was a too spread out for me to
> add specific quotes here. Let me just summarize a possible path forward:
>
>  1. Things that are maintained by the core Emacs developers should be in core,
>     and in the Emacs.git repository. This makes it easy for them to access and
>     build, search history, read emacs-diffs, etc.
>
>  2. Things that are maintained outside of Emacs, but contributed back for
>     inclusion, should be in ELPA, and in the Elpa.git repository. This
>     includes Gnus, Org, CEDET, and a few more. (We don't want to go crazy,
>     so let's start with the big ones).
>
>  3. There should be a defined subset of packages that get copied from ELPA
>     into the Emacs tarball during release, and an easy way to manage this list
>     for the core developers. That way, certain packages like seq.el and
>     stream.el can feel like "core" packages to users, when they are really
>     "external" packages from the point of view of the core developers.

ELPA packages that other core code depends on (like CEDET; xref uses it
- called "core ELPA packages" hereinafter) have to be in every
developer's build environment everyday; the other core code has to see
the current version of the package, and it has to be the same for every
developer.

If core ELPA packages are in the ELPA git repository, you would copy
some subtrees of the ELPA git workspace into the Emacs git workspace.
People have looked into doing this, but it gets messy and confusing.
"git fetch" is dealing with two upstreams for one workspace. There has
been talk about git submodules, etc, but nothing concrete.

The alternative, which is being implemented by the :core external
package feature in Gnu ELPA, and has been shown to work (but has some
details to work out), is to keep core ELPA packages in the emacs
git repository. Then they are part of the developer's build environment
using the current git workflow, and they are included in both the emacs
tarball and the ELPA package server with the current release workflows.


There are two rationales for moving a package to ELPA:

1) Allow a release schedule independent of the core Emacs release
   schedule.

2) Reduce the size of the Emacs git workspace, which simplifies managing
   it.

But a _core_ ELPA package must remain in the Emacs git workspace, so the
only rationale for moving a core package to ELPA is for an independent
release schedule.

> There are three actions I'd like to take from here:
>
>   a. That we clearly document an ELPA policy we all agree on;

In admin/notes/elpa ?

>   b. That we move "external" packages from core into ELPA, starting with the
>      really big ones;

It's not clear that "really big" implies "independent release schedule".
"externally managed" implies that.

"not used by other core code" is also a good reason to move a package to
ELPA. 

>   c. That we assess any points of friction after doing so, and adjust as
>      necessary.

If we use the approach of keeping core ELPA packages in the Emacs git
repository, there is _zero_ friction for the current core Emacs
developers; the only change is in the ELPA config scripts. 

-- 
-- Stephe

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Seq.el was put on Gelpa so that it can be used by emacsen < 25.

It is fine to have it there, but this suggests we should maintain
it inside Emacs.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:

> ELPA packages that other core code depends on (like CEDET; xref uses it -
> called "core ELPA packages" hereinafter) have to be in every developer's
> build environment everyday; the other core code has to see the current
> version of the package, and it has to be the same for every developer.
> 
> If core ELPA packages are in the ELPA git repository, you would copy some
> subtrees of the ELPA git workspace into the Emacs git workspace. People have
> looked into doing this, but it gets messy and confusing. "git fetch" is
> dealing with two upstreams for one workspace. There has been talk about git
> submodules, etc, but nothing concrete.

Elpa.git should be a submodule referenced from within Emacs.git (under "elpa").
This allows us to expressly state which "version" of Elpa is expected to work
with the current Emacs.git.

Large packages like CEDET should move outside of Emacs.git and into Elpa.git.
If xref.el depends on CEDET, it would move to Elpa.git as well.

> If we use the approach of keeping core ELPA packages in the Emacs git
> repository, there is _zero_ friction for the current core Emacs developers;
> the only change is in the ELPA config scripts.

I do not want this code replication. We will promote ELPA as an active
development space alongside core Emacs. If we use a submodule, and improve our
tooling slightly, it should become a seamless thing.

John

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:28 PM, John Wiegley wrote:

> If xref.el depends on CEDET, it would move to Elpa.git as well.

I can cut out xref.el's dependency on CEDET easily enough.

Moving it into ELPA would be kinda problematic, because guess what M-. 
and M-, and bound to by default now.

Re: ELPA policy

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> I can cut out xref.el's dependency on CEDET easily enough.

Great, let's do that. I'd prefer to have the functionality of xref.el in core.

Thanks,
  John

Re: ELPA policy

[David Engster]

Dmitry Gutov writes:
> On 11/10/2015 08:28 PM, John Wiegley wrote:
>
>> If xref.el depends on CEDET, it would move to Elpa.git as well.
>
> I can cut out xref.el's dependency on CEDET easily enough.

Which illustrates nicely the downside of moving packages to ELPA.

-David

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>> ELPA packages that other core code depends on (like CEDET; xref uses it -
>> called "core ELPA packages" hereinafter) have to be in every developer's
>> build environment everyday; the other core code has to see the current
>> version of the package, and it has to be the same for every developer.
>> 
>> If core ELPA packages are in the ELPA git repository, you would copy some
>> subtrees of the ELPA git workspace into the Emacs git workspace. People have
>> looked into doing this, but it gets messy and confusing. "git fetch" is
>> dealing with two upstreams for one workspace. There has been talk about git
>> submodules, etc, but nothing concrete.
>
> Elpa.git should be a submodule referenced from within Emacs.git (under "elpa").
> This allows us to expressly state which "version" of Elpa is expected to work
> with the current Emacs.git.

Since ELPA comprises many packages, that simply cannot work.

> Large packages like CEDET should move outside of Emacs.git and into Elpa.git.

Why?

> I do not want this code replication. We will promote ELPA as an active
> development space alongside core Emacs. If we use a submodule, and improve our
> tooling slightly, it should become a seamless thing.

From my experience, git submodules are never seamless to work with.

-David

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> I can cut out xref.el's dependency on CEDET easily enough.
>
> Great, let's do that. I'd prefer to have the functionality of xref.el in core.

So that means xref won't support CEDET out of the box? What is gained by
that, exactly?

-David

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  aaronecay@gmail.com,  Stromeko@nexgo.de,  emacs-devel@gnu.org,  Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 10 Nov 2015 10:28:26 -0800
> 
> Large packages like CEDET should move outside of Emacs.git and into Elpa.git.

"Should" based on what? just the fact that it's large?  I think we
should decide this stuff on a case by case basis.  For example:

> If xref.el depends on CEDET, it would move to Elpa.git as well.

IMO, the exact opposite: if there are core features that we want to be
in Emacs no matter what, and those features depend on a package which
could be a candidate to move to ELPA, that package should NOT move to
ELPA.

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> So that means xref won't support CEDET out of the box? What is gained by
> that, exactly?

It could always support it with customization hooks.

John

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

>> Elpa.git should be a submodule referenced from within Emacs.git (under "elpa").
>> This allows us to expressly state which "version" of Elpa is expected to work
>> with the current Emacs.git.

> Since ELPA comprises many packages, that simply cannot work.

Perhaps I should say, "The version of Elpa.git, a subset of which will appear
in the release tarball of this commit".

>> Large packages like CEDET should move outside of Emacs.git and into
>> Elpa.git.

> Why?

There will never be 100% agreement on whether they should be in ELPA, or be in
Core, so I'm making the decision that they belong in ELPA. The big three that
will be moved first are: Gnus, Org and CEDET.

> From my experience, git submodules are never seamless to work with.

Even still, they work perfectly fine for this purpose.

John

Re: ELPA policy

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

>> Large packages like CEDET should move outside of Emacs.git and into
>> Elpa.git.
> "Should" based on what? just the fact that it's large? I think we should
> decide this stuff on a case by case basis. For example:

I'm surprised you say this, Eli, because in another thread you agree that
packages like this should be in Elpa, didn't you?

>> If xref.el depends on CEDET, it would move to Elpa.git as well.

> IMO, the exact opposite: if there are core features that we want to be in
> Emacs no matter what, and those features depend on a package which could be
> a candidate to move to ELPA, that package should NOT move to ELPA.

Core should provide functionality along the lines of a "standard library" and
a "standard environment", where having them in core is as much a statement
about consistency of interface, as it is about universal availability of the
functionality.

Since xref.el does not need to depend on CEDET, I don't see a reason why it
should, causing CEDET to remain in core.

John

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>> Large packages like CEDET should move outside of Emacs.git and into
>>> Elpa.git.
>
>> Why?
>
> There will never be 100% agreement on whether they should be in ELPA, or be in
> Core, so I'm making the decision that they belong in ELPA.

So you simply don't want to give any reason?

-David

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> So you simply don't want to give any reason?

David, although I work by consensus as much as possible, I won't explain
everything I choose to do. My reason is that I think this will improve Emacs
development.

Part of my goal is to make the boundary between Emacs and ELPA thinner, so it
will actually be *easier* to move CEDET and friends back into core later, if
my reasoning is wrong.

John

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Tue, 10 Nov 2015 10:35:13 -0800
> Cc: aaronecay@gmail.com, Eli Zaretskii <eliz@gnu.org>, Stromeko@nexgo.de,
> 	Stephen Leake <stephen_leake@stephe-leake.org>, emacs-devel@gnu.org
> 
> >>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
> > I can cut out xref.el's dependency on CEDET easily enough.
> 
> Great, let's do that.

Sounds like a step backward to me.  CEDET allows xref to be more
versatile and support more back-ends.

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Stephen Leake <stephen_leake@stephe-leake.org>,  aaronecay@gmail.com,  Eli Zaretskii <eliz@gnu.org>,  Stromeko@nexgo.de,  Dmitry Gutov <dgutov@yandex.ru>,  emacs-devel@gnu.org
> Date: Tue, 10 Nov 2015 10:52:41 -0800
> 
> > Why?
> 
> There will never be 100% agreement on whether they should be in ELPA, or be in
> Core, so I'm making the decision that they belong in ELPA.

IMO, it's a mistake to move CEDET.

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>>>>> David Engster <deng@randomsample.de> writes:
>
>> So you simply don't want to give any reason?
>
> David, although I work by consensus as much as possible, I won't explain
> everything I choose to do. My reason is that I think this will improve Emacs
> development.

This is not about reaching a consensus. This is about you giving proper
reasons for such a big change.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: stephen_leake@stephe-leake.org,  aaronecay@gmail.com,  Stromeko@nexgo.de,  emacs-devel@gnu.org,  dgutov@yandex.ru
> Date: Tue, 10 Nov 2015 10:54:51 -0800
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Large packages like CEDET should move outside of Emacs.git and into
> >> Elpa.git.
> > "Should" based on what? just the fact that it's large? I think we should
> > decide this stuff on a case by case basis. For example:
> 
> I'm surprised you say this, Eli, because in another thread you agree that
> packages like this should be in Elpa, didn't you?

I said I _could_agree.  But not unconditionally.  We should discuss
each case.

> >> If xref.el depends on CEDET, it would move to Elpa.git as well.
> 
> > IMO, the exact opposite: if there are core features that we want to be in
> > Emacs no matter what, and those features depend on a package which could be
> > a candidate to move to ELPA, that package should NOT move to ELPA.
> 
> Core should provide functionality along the lines of a "standard library" and
> a "standard environment", where having them in core is as much a statement
> about consistency of interface, as it is about universal availability of the
> functionality.

Sorry, I don't understand what this means in practice.

> Since xref.el does not need to depend on CEDET, I don't see a reason why it
> should, causing CEDET to remain in core.

xref is as useful as its backends.  If you take away backends, it
becomes less useful, or supports less programming languages, or both.

Re: ELPA policy

[Eli Zaretskii]

How about if we leave this issue alone for a while?  Let's concentrate
on releasing Emacs 25.1 first, and return to this issue later.  Rome
wasn't built in one day, we don't have to rush.

WDYT?

Re: ELPA policy

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> How about if we leave this issue alone for a while? Let's concentrate on
> releasing Emacs 25.1 first, and return to this issue later. Rome wasn't
> built in one day, we don't have to rush.
> 
> WDYT?

Agreed. Let's table for now and revisit. If the suggested moves really trouble
core developers, we'll talk about it again before making a final decision.

John

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> This is not about reaching a consensus. This is about you giving proper
> reasons for such a big change.

To be clear, I would also put Eshell (though not pcomplete) into the category
of "big things that should be in ELPA" -- albeit, the subset of ELPA that will
be in the release tarball.

It's hard to pin down why in a manner that fits in an e-mail. If Eshell were
in ELPA today, and we were talking about moving it into core, I would have
just as much trouble justifying that move too. Perhaps this simply underscores
the fact that we don't have an agreed upon ELPA policy we can all refer to.

OK, David, I won't decide this by fiat. Let us decide what our ELPA policy
will be, until it becomes clear, based on that document, what should go where,
and why. We'll defer discussions of package movement until we have that in
place.

John

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:37 PM, David Engster wrote:

>> I can cut out xref.el's dependency on CEDET easily enough.
>
> Which illustrates nicely the downside of moving packages to ELPA.

Nope. It will be beneficial to the code in Emacs, because xref is 
currently depending on a minor feature of CEDET (the 
semantic-symref-tool infrastructure), which could use a rewrite, to be 
available for a more general usage.

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:44 PM, David Engster wrote:

> So that means xref won't support CEDET out of the box? What is gained by
> that, exactly?

It doesn't support CEDET even now. Nobody wrote the Semantic 
implementation of a xref backend, and it might be tool late now for the 
25.1 release. If CEDET were moved to ELPA, however, you can add that 
support at your leisure later.

And that's an orthogonal issue. xref has no need to support CEDET: CEDET 
should support xref (the support code will be in CEDET's tree).

Re: ELPA policy

[Eli Zaretskii]

> Cc: Stephen Leake <stephen_leake@stephe-leake.org>,
>  Eli Zaretskii <eliz@gnu.org>, aaronecay@gmail.com, Stromeko@nexgo.de,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 10 Nov 2015 21:57:25 +0200
> 
> On 11/10/2015 08:37 PM, David Engster wrote:
> 
> >> I can cut out xref.el's dependency on CEDET easily enough.
> >
> > Which illustrates nicely the downside of moving packages to ELPA.
> 
> Nope. It will be beneficial to the code in Emacs, because xref is 
> currently depending on a minor feature of CEDET (the 
> semantic-symref-tool infrastructure), which could use a rewrite, to be 
> available for a more general usage.

I think a point we should take from this is that the prerequisites (in
this case, the said rewrite) should be in place _before_ we seriously
consider such moves (pun intended ;-).

Re: ELPA policy

[David Engster]

John Wiegley writes:
>>>>>> David Engster <deng@randomsample.de> writes:
>
>> This is not about reaching a consensus. This is about you giving proper
>> reasons for such a big change.
>
> To be clear, I would also put Eshell (though not pcomplete) into the category
> of "big things that should be in ELPA" -- albeit, the subset of ELPA that will
> be in the release tarball.
>
> It's hard to pin down why in a manner that fits in an e-mail. If Eshell were
> in ELPA today, and we were talking about moving it into core, I would have
> just as much trouble justifying that move too. Perhaps this simply underscores
> the fact that we don't have an agreed upon ELPA policy we can all refer to.

In my opinion, the main question is whether something provides
infrastructure for other packages to use. This is precisely what CEDET
tries to do. I wouldn't have much trouble with putting parts of CEDET in
ELPA, namely those parts that do not directly provide infrastructure,
like support for certain languages, project types, indexing tools, etc.

> OK, David, I won't decide this by fiat. Let us decide what our ELPA policy
> will be, until it becomes clear, based on that document, what should go where,
> and why. We'll defer discussions of package movement until we have that in
> place.

It is still not clear to me what exactly is gained by moving core
packages to ELPA. In my opinion, packages like Org, Gnus or CEDET didn't
create significant problems for Emacs development in the past. On the
contrary, it made sure that those packages were adapted quickly when
larger changes in core were made. Those changes got synced back upstream
by people like Bastien, Katsumi and me. So in my opinion, you are trying
to fix something that is not broken.

What actually *does* create problems are packages that don't have an
active maintainer. The 'big three' however are not among those.

-David

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:49 PM, Eli Zaretskii wrote:

> IMO, the exact opposite: if there are core features that we want to be
> in Emacs no matter what, and those features depend on a package which
> could be a candidate to move to ELPA, that package should NOT move to
> ELPA.

xref only depends on a minor feature of CEDET (the code indexing tools 
integration), and I've done that in a quick-and-dirty way, as a proof of 
concept.

It would be better to have functionality like that in the core, 
independent of CEDET.

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 09:21 PM, Eli Zaretskii wrote:

> xref is as useful as its backends.  If you take away backends, it
> becomes less useful, or supports less programming languages, or both.

None of the xref backends use Semantic. There are still only two of 
them: elisp and etags.

When CEDET grows a backend, it can just as well add it when 
semantic-mode is enabled. No need to have it in the core for that.

Re: ELPA policy

[Achim Gratz]

John Wiegley writes:
>> You'd do that with branches if it really becomes necessary. I don't really
>> see why submodules could not be used, except for the extra rope they give
>> you to hang yourself with. Any other solution is going to have the same
>> basic complexity beneath and the potential to not work on some platform or
>> other.
>
> A submodule doesn't allow us to have "Org + some-patch-not-yet-in-Org".

You'd branch on ELPA to something like emacs-25.0.50, patch away and
point the submodule there if you really can't wait for the patch to be
committed in Org upstream and mirrored back to ELPA.

>> I maintain that ELPA should benefit Emacs users first.
>
> I'm intrigued; could you clarify this a bit more? I saw what you wrote in
> response to Eli, but that didn't really help. Specifically, do you have a
> scenario in mind that is better for users and worse for developers?

It was a long day and I'm afraid I'd not be too coherent, so I'll punt
for now.  But in short, look at CPAN, CRAN and CTAN for NIH examples of
what ELPA might possibly become.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 10:01 PM, Eli Zaretskii wrote:

> I think a point we should take from this is that the prerequisites (in
> this case, the said rewrite) should be in place _before_ we seriously
> consider such moves (pun intended ;-).

The only prerequisite is to have this functionality (which is fairly 
self-contained) in the core. It would benefit from a rewrite (and moving 
away from CEDET would encourage that), but as the first step, we can 
just copy it and do some renaming.

However, like mentioned in the "project.el semantics" thread:

   I'm seriously considering to get away from it and always use Grep,
   because the other tools, while they can be faster, they're also
   entirely opaque to the user, can have outdated databases, non-indexed
   relevant directories, unsupported languages, and so on.

So this feature, which xref is depending on for the default 
xref-find-references implementation, requires certain amount of 
hand-holding from the user. And I don't know how to approach documenting 
it, and otherwise make it apparent to the user.

So someone else should probably start on taking care of that. Otherwise, 
I'd rather fall back on Grep, until such point that Emacs starts to 
manage the Global/id-utils/CScope datbases for the user, and learns to 
make sure that the database is up-to-date.

Re: ELPA policy

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> the main question is whether something provides infrastructure for other
> packages to use.

Sounds like a good sentence for an ELPA policy. :)

> It is still not clear to me what exactly is gained by moving core packages
> to ELPA.

Agility. What is appropriate. Knowing when a thing goes into core, and when in
ELPA.

Org is an application, it's not infrastructure; the same with Gnus. *Parts* of
Gnus might rightly be considered infrastructure, but the whole of Gnus just
doesn't belong there. Parts of CEDET probably do belong in Emacs core, but as
an application, I don't think the whole of it does.

Since we don't have an agreed upon basis by which to draw such lines, we're
both talking about what we feel is right. Let's agree to revisit after having
the ELPA discussion.

John

Re: ELPA policy

[Eli Zaretskii]

> Cc: deng@randomsample.de, stephen_leake@stephe-leake.org,
>  aaronecay@gmail.com, Stromeko@nexgo.de, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 10 Nov 2015 22:19:46 +0200
> 
> So someone else should probably start on taking care of that. Otherwise, 
> I'd rather fall back on Grep, until such point that Emacs starts to 
> manage the Global/id-utils/CScope datbases for the user, and learns to 
> make sure that the database is up-to-date.

Grep doesn't scale well to large projects, IME.  You get too many
false positives.

Outdated databases are easy to avoid with the likes of cron jobs.
Yes, that's hand-holding, but when you have to quickly find stuff in a
project with 3 million lines of code and thousands of classes, there
really is no other alternative.

Re: ELPA policy

[David Engster]

Dmitry Gutov writes:
> On 11/10/2015 08:37 PM, David Engster wrote:
>
>>> I can cut out xref.el's dependency on CEDET easily enough.
>>
>> Which illustrates nicely the downside of moving packages to ELPA.
>
> Nope. It will be beneficial to the code in Emacs, because xref is
> currently depending on a minor feature of CEDET (the
> semantic-symref-tool infrastructure), which could use a rewrite, to be
> available for a more general usage.

As you know, we will drop our upstream repository. During the last
merge, I will also have to discuss with Eric what we can do to narrow
CEDET's scope. I agree with you that semantic-symref is probably one of
the packages that should be replaced with something more general. But as
long as it's there, I'd appreciate if you could leave whatever support
you have coded for it.

I'd also be happy to drop the completion GUI stuff, BTW. I've repeatedly
said that company should be in core for that.

-David

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 10:45 PM, David Engster wrote:

> As you know, we will drop our upstream repository. During the last
> merge, I will also have to discuss with Eric what we can do to narrow
> CEDET's scope.
> I agree with you that semantic-symref is probably one of
> the packages that should be replaced with something more general.

In the end, I would leave only the "backend" part of semantic-symref 
(calling the tools + piping the results through Semantic), and use it in 
the Semantic xref backend. So the results will be displayed by xref.

Alas, xref is not up to feature parity yet (cannot group hits by 
containing functions, no checkboxes, simplistic "rename" command), and 
I'm having difficulties deciding on the changes to the xref data 
structures that would make that possible, as well as keep allowing 
certain other uses of xref, such as project-find-regexp.

I'll write that up in a separate thread later this week.

> But as
> long as it's there, I'd appreciate if you could leave whatever support
> you have coded for it.

Just to be clear, there's no "support" for semantic-symref in xref yet. 
We just use some low-level pieces of it as a "fast Grep".

> I'd also be happy to drop the completion GUI stuff, BTW. I've repeatedly
> said that company should be in core for that.

Also not exactly necessary. Semantic should make its 
completion-at-point-functions element the main entry point for 
completion, and Company will pick it up. You can even have argument list 
annotations, if you add the :annotation-function property.

I do want to have Company more widely available, but probably not in 
25.1. There are quite a few things I wanted to do with it first, and 
most are still untouched.

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 10:34 PM, Eli Zaretskii wrote:

> Grep doesn't scale well to large projects, IME.  You get too many
> false positives.

We can also check the "inside string or comment" status, as well as use 
the Emacs syntax tables to make sure that the match begins and ends at 
symbol boundaries appropriate for the file's language. Everything 
necessary for that is already written (and you can try it out by calling 
project-find-regexp and using something like \_<tool_bar_items\_> as the 
regexp). I don't think that id-utils does anything more to weed out 
false positives.

Grep is still probably going to be slower than at least some of the 
tools in question. Could you test, on a large project of your choice?

> Outdated databases are easy to avoid with the likes of cron jobs.
> Yes, that's hand-holding, but when you have to quickly find stuff in a
> project with 3 million lines of code and thousands of classes, there
> really is no other alternative.

Yes. But the xref commands should be easy to use. Even if the above is 
not rocket science, the user would still have to know what they need to 
do to get up-to-date results. (Believe it or not, I haven't created a 
single cron job for code writing purposes in my life, and I don't know 
its syntax for the time intervals. I'm likely not alone in that.)

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 11:16 PM, Dmitry Gutov wrote:

> project-find-regexp and using something like \_<tool_bar_items\_> as the

Sorry, using the Emacs-only syntax there doesn't exactly work yet. But 
you can compare by calling xref-find-references when the project has an 
id-utils index file, and doing the same without it.

Re: ELPA policy

[Dmitry Gutov]

On 11/10/2015 08:35 PM, John Wiegley wrote:
>>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> I can cut out xref.el's dependency on CEDET easily enough.
>
> Great, let's do that. I'd prefer to have the functionality of xref.el in core.

Let me know when it becomes a blocker, and I'll do it in the simplest 
way. But I'd like to defer that for now, because it comes with making 
choices (see the other messages in this thread), and I'm running low on 
decision-making juice these days.

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:
>
>> ELPA packages that other core code depends on (like CEDET; xref uses it -
>> called "core ELPA packages" hereinafter) have to be in every developer's
>> build environment everyday; the other core code has to see the current
>> version of the package, and it has to be the same for every developer.
>> 
>> If core ELPA packages are in the ELPA git repository, you would copy some
>> subtrees of the ELPA git workspace into the Emacs git workspace. People have
>> looked into doing this, but it gets messy and confusing. "git fetch" is
>> dealing with two upstreams for one workspace. There has been talk about git
>> submodules, etc, but nothing concrete.
>
> Elpa.git should be a submodule referenced from within Emacs.git (under
> "elpa").

That's overkill; there are many packages in Gnu ELPA that core code
should _not_ depend on.

We could try to have a list of approved packages in a file somewhere,
but I don't think we can enforce that; better to use git to enforce it.

We need a way to explicitly label and enforce which ELPA packages core
code can depend on. The only demonstrated way to do that is to keep them
in Emacs git.

> This allows us to expressly state which "version" of Elpa is expected
> to work with the current Emacs.git.

That's only required for the ELPA packages that core code can depend on.
For the other ELPA packages, we explicitly don't want to know if they
work with Emacs master; that's up to the individual package maintainers.
That's one of the primary purposes of ELPA.

> Large packages like CEDET should move outside of Emacs.git and into
> Elpa.git.

If this said "large packages that the rest of Emacs core does not depend
on ...", I could agree.

I just checked; the only other core code that depends on CEDET is
xref.el, which can be fixed. So I agree, CEDET should move to elpa git.

However, I assume the reason CEDET is in Emacs git is because there is a
desire to distribute it in the tarball; a sufficient number of users
expect it to be there without going thru package install. So the
mechanism to put ELPA packages in the tarball (but _not_ in the emacs
development workspace) must also be implemented. That should not be too
hard; no git submodules, just directory tree copy.

So this introduces a third kind of ELPA package: "distributed in Emacs
tarball". A "tarball package" for short?

Except that Eric maintains that there is more to "CEDET" in Emacs core
than just lisp/cedet/*. So to be clear, we are proposing to move only
lisp/cedet/* to elpa git.

> If xref.el depends on CEDET, it would move to Elpa.git as well.

As Dmitry points out, better to remove the dependency from the core
xref.el. Then we can add it back either in the ELPA CEDET package or in
a new xref-cedet ELPA package.

>> If we use the approach of keeping core ELPA packages in the Emacs git
>> repository, there is _zero_ friction for the current core Emacs developers;
>> the only change is in the ELPA config scripts.
>
> I do not want this code replication. 

How is it replication?

There are two styles of ELPA packages currently:

1) The only repository for the code is Gnu ELPA git

2) There is an external repository that the package maintainer uses;
   releases are pushed to Gnu ELPA git

The choice is up to the ELPA package maintainer.

The proposal is to change both to say:

1a) The only repository for the code is Gnu ELPA git or Gnu Emacs git.

2a) There is an external repository that the package maintainer uses;
    releases are pushed to Gnu ELPA git or Gnu Emacs git.

The ELPA packages in Emacs git are "core ELPA packages"; core Emacs code
may rely on them. 

There is no additional replication by keeping the core ELPA packages in
Emacs git.

The only reason the core Emacs packages are in the ELPA server is to
allow package releases in addition to Emacs releases. The package
maintainers have to accept maintaining backward compatibility, so a core
package in an Emacs 25.1 binary install can work with a more recent
version of the core ELPA package.

In summary, I'm proposing that there are three kinds of ELPA packages:

1) "normal ELPA package". 

    - Stored in Gnu ELPA git, and possibly in the package maintainer's
      separate repository.

    - Releases only occur via the ELPA package server, whenever the
      version number increments in the ELPA git main package file.
    
    - Core Emacs code may not depend on these.

2) "tarball ELPA package". 

    - Stored in Gnu ELPA git, and possibly in the package maintainer's
      separate repository.

    - Each Emacs release includes a released version of the package in
      the Emacs tarball, via copy from an ELPA workspace at Emacs
      tarball build time. Other releases also occur, via the ELPA
      package server, whenever the version number increments in the ELPA
      git main package file.
    
    - Core Emacs code may not depend on these.

3) "core ELPA package".  

    - Stored in Gnu Emacs git, and possibly in the package maintainer's
      separate repository.

    - Each Emacs release includes a version of the package in the
      tarball, because it is in the Emacs git workspace. Other releases
      also occur, via the ELPA package server, whenever the version
      number increments in the Emacs git main package file.
    
    - Core Emacs code may depend on these.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> However, like mentioned in the "project.el semantics" thread:

please change the thread subject.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> David Engster <deng@randomsample.de> writes:
>
>>> Large packages like CEDET should move outside of Emacs.git and into
>>> Elpa.git.
>
>> Why?
>
> There will never be 100% agreement on whether they should be in ELPA, or be in
> Core, so I'm making the decision that they belong in ELPA. 

I'm happy to have someone making decisions, but there needs to be some
rationale given, so we understand the tradeoffs.

I believe the rationale here is:

- CEDET is not used by other core Emacs code (xref.el will be fixed)

- code that is not used by other core Emacs code clutters the workspace,
  producing extra hits in greps, etc.

- CEDET can still be included in the Emacs tarball, so users do
  not have to use list-packages to install it (this requires a to be
  implemented mechanism).

>> From my experience, git submodules are never seamless to work with.
>
> Even still, they work perfectly fine for this purpose.

I would like to see an actual demonstration, please; there are lots of
devils in the details.

Which means:

Please post a clear recipe for setting up an Emacs + ELPA developer
workspace, starting from git clone, so I can try this on my machine.


The test is then that some core Emacs code depend on some ELPA package;
we'd need branches in both to make that happen, since it doesn't now.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> David Engster <deng@randomsample.de> writes:
>
>> This is not about reaching a consensus. This is about you giving proper
>> reasons for such a big change.
>
> To be clear, I would also put Eshell (though not pcomplete) into the category
> of "big things that should be in ELPA" -- albeit, the subset of ELPA that will
> be in the release tarball.
>
> It's hard to pin down why in a manner that fits in an e-mail. If Eshell were
> in ELPA today, and we were talking about moving it into core, I would have
> just as much trouble justifying that move too. Perhaps this simply underscores
> the fact that we don't have an agreed upon ELPA policy we can all
> refer to.

Yes.

> OK, David, I won't decide this by fiat. Let us decide what our ELPA policy
> will be, until it becomes clear, based on that document, what should go where,
> and why. We'll defer discussions of package movement until we have that in
> place.

Thank you.


-- 
-- Stephe

Re: ELPA policy

[John Wiegley]

>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:

>> Elpa.git should be a submodule referenced from within Emacs.git (under
>> "elpa").

> That's overkill; there are many packages in Gnu ELPA that core code should
> _not_ depend on.

I believe that no core code should depend on any ELPA packages. Such a
dependency would be a reason to move that package into core, no? If that's
really the case, no submodule reference is necessary after all.

> If this said "large packages that the rest of Emacs core does not depend on
> ...", I could agree.

Yes, I meant that.

> So this introduces a third kind of ELPA package: "distributed in Emacs
> tarball". A "tarball package" for short?

Yes, for ELPA packages, there would be two flavors: in the tarball, and not.
Some have suggested just putting everything in the tarball for now. So we
could defer this question until the size of the tarball mattered, or if a
particular package was so dynamic it didn't want a version frozen into the
tarball.

Note that going down this road starts to make the line between "core" and
"ELPA" very thin, with easy migration between the two when desired. For
developers it allows Git to focus better, but for users, the difference is
largely invisible.

> Except that Eric maintains that there is more to "CEDET" in Emacs core than
> just lisp/cedet/*. So to be clear, we are proposing to move only
> lisp/cedet/* to elpa git.

Yes.

> In summary, I'm proposing that there are three kinds of ELPA packages:

I think we have a lot of agreement here, but I'd still like a clearer policy
than just "if core depends on it".

John

Re: ELPA policy

[Stephen Leake]

David Engster <deng@randomsample.de> writes:

> John Wiegley writes:
>>>>>>> David Engster <deng@randomsample.de> writes:
>>
>>> This is not about reaching a consensus. This is about you giving proper
>>> reasons for such a big change.
>>
>> To be clear, I would also put Eshell (though not pcomplete) into the category
>> of "big things that should be in ELPA" -- albeit, the subset of ELPA that will
>> be in the release tarball.
>>
>> It's hard to pin down why in a manner that fits in an e-mail. If Eshell were
>> in ELPA today, and we were talking about moving it into core, I would have
>> just as much trouble justifying that move too. Perhaps this simply underscores
>> the fact that we don't have an agreed upon ELPA policy we can all refer to.
>
> In my opinion, the main question is whether something provides
> infrastructure for other packages to use. 

I think it is a reasonable goal to allow ELPA packages to serve in that
role, as long as the "other packages" are also normal or tarball ELPA
packages.

> This is precisely what CEDET tries to do.

Yes.

However, now the shoe is on the other foot: why must infrastructure
packages be in Emacs core?

I think the trigger is "some other _core_ code wants to depend on it".
That would trigger a move to Emacs git, as a core ELPA package (the
package could still have intermediate released via the Gnu ELPA server).

> I wouldn't have much trouble with putting parts of CEDET in ELPA,
> namely those parts that do not directly provide infrastructure, like
> support for certain languages, project types, indexing tools, etc.

Good, but let's not try to do that for Emacs 25; since we are trying to
get to feature freeze, it's too much.

> It is still not clear to me what exactly is gained by moving core
> packages to ELPA. 

One gain is making it clear that other core code is not allowed to depend
on it. This is in turn to ensure that it doesn't creep into the dumped
code. But I'm not sure that's an important reason.

-- 
-- Stephe

RE: ELPA policy

[Drew Adams]

Here's a naive question regarding where you place Lisp code
that is distributed with Emacs (part of the tarball): Will
anything change for users, depending on where you happen to
manage the code wrt ELPA etc.?

I'm guessing (and hoping) not, so that the distributed
directory structure will remain the same.  Is that correct?

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> David Engster <deng@randomsample.de> writes:
>
>> the main question is whether something provides infrastructure for other
>> packages to use.
>
> Sounds like a good sentence for an ELPA policy. :)
>
>> It is still not clear to me what exactly is gained by moving core packages
>> to ELPA.

Since no other core code depends on ELPA, CEDET is _not_ a "core package".

Rather, CEDET is a "tarball package"; it is in Emacs git solely to
ensure that it is included in the Emacs release tarball.

A better example of a possible "core ELPA package" is the "seq" package.

> Agility. 

I hear that as "easier to make small/frequent changes". That is what the
ELPA release policy gives you, yes. So this is one rationale for moving
packages to ELPA.

> What is appropriate. 

That's what we are trying to figure out :)

> Knowing when a thing goes into core, and when in ELPA.

Ditto.

> Org is an application, it's not infrastructure; the same with Gnus. *Parts* of
> Gnus might rightly be considered infrastructure, but the whole of Gnus just
> doesn't belong there. Parts of CEDET probably do belong in Emacs core, but as
> an application, I don't think the whole of it does.

I think the distinction between "tarball package" and "core package" is
helpful here.

I'm guessing that the main motivation for including Org and Gnus in
Emacs git (well, CVS back then?) was to include them in the release
tarball. If we have a mechanism to allow that for ELPA packages, moving
them to ELPA makes sense.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: John Wiegley <jwiegley@gmail.com>
>> Cc: Stephen Leake <stephen_leake@stephe-leake.org>,
>> aaronecay@gmail.com, Eli Zaretskii <eliz@gnu.org>,
>> Stromeko@nexgo.de, Dmitry Gutov <dgutov@yandex.ru>,
>> emacs-devel@gnu.org
>> Date: Tue, 10 Nov 2015 10:52:41 -0800
>> 
>> > Why?
>> 
>> There will never be 100% agreement on whether they should be in ELPA, or be in
>> Core, so I'm making the decision that they belong in ELPA.
>
> IMO, it's a mistake to move CEDET.

Bald statements of preference, without rationale, do not help move this
discussion forward.

We are trying to establish a general ELPA policy, not just vote on one
particular package.

What is it about CEDET that makes it a poor candidate?

How would your workflow suffer if it was moved?

How would users suffer?

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: John Wiegley <jwiegley@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>, aaronecay@gmail.com,
>> Stromeko@nexgo.de, emacs-devel@gnu.org, Dmitry Gutov
>> <dgutov@yandex.ru>
>> Date: Tue, 10 Nov 2015 10:28:26 -0800
>> 
>> Large packages like CEDET should move outside of Emacs.git and into Elpa.git.
>
> "Should" based on what? just the fact that it's large?  I think we
> should decide this stuff on a case by case basis.  For example:
>
>> If xref.el depends on CEDET, it would move to Elpa.git as well.
>
> IMO, the exact opposite: if there are core features that we want to be
> in Emacs no matter what, and those features depend on a package which
> could be a candidate to move to ELPA, that package should NOT move to
> ELPA.

In general, I agree.

However, there are two _independent_ aspects of "move to ELPA":

1) Move to ELPA git.

2) Be distributed by the ELPA package server.


If other core code depends on a package, it should not move to ELPA git
(unless the dependency is easily refactored, as in the CEDET/xgit case;
there will always be corner cases).

However, if there is a desire to do package releases between Emacs
releases, making the package available via the ELPA package server makes
sense.

I think this discussion is mainly focused on "move to ELPA git", but it
would help to be clear.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> Large packages like CEDET should move outside of Emacs.git and into
>>> Elpa.git.
>> "Should" based on what? just the fact that it's large? I think we should
>> decide this stuff on a case by case basis. For example:
>
> I'm surprised you say this, Eli, because in another thread you agree that
> packages like this should be in Elpa, didn't you?
>
>>> If xref.el depends on CEDET, it would move to Elpa.git as well.
>
>> IMO, the exact opposite: if there are core features that we want to be in
>> Emacs no matter what, and those features depend on a package which could be
>> a candidate to move to ELPA, that package should NOT move to ELPA.
>
> Core should provide functionality along the lines of a "standard library" and
> a "standard environment", where having them in core is as much a statement
> about consistency of interface, as it is about universal availability of the
> functionality.

Right. Since CEDET provides lots of interesting infrastrucure, I assumed
it was part of the Emacs standard library.

But I also see no reason not to have two levels of "standard
infrastructure"; one in core Emacs, on in ELPA.

That said, moving CEDET out of core to ELPA can be seen as a demotion.
For example, I was assuming that the EDE package system should be
prefered over any package system that is in ELPA, precisely because it
is in Emacs git. So I was spending my time improving EDE, rather than
some other ELPA package.

That aspect should be considered.

In other emails, I've said "CEDET is a tarball package; in
Emacs git solely to be included in the tarball".

Is it fair to add "also because it is approved as part of the standard
Emacs library"?

> Since xref.el does not need to depend on CEDET, I don't see a reason why it
> should, causing CEDET to remain in core.

I would object to moving CEDET to ELPA git if it were not easy to
refactor the current dependency to still be provided by an ELPA package.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

Eli Zaretskii <eliz@gnu.org> writes:

>> Since xref.el does not need to depend on CEDET, I don't see a reason why it
>> should, causing CEDET to remain in core.
>
> xref is as useful as its backends.  If you take away backends, it
> becomes less useful, or supports less programming languages, or both.

Yes.

That's an argument for making the CEDET xref backend available
somewhere; I've committed to ensuring that.

But this discussion is about ELPA vs Emacs core:

Does it matter whether that backend is in ELPA or Emacs core?

-- 
-- Stephe

Re: ELPA policy

[John Wiegley]

>>>>> Stephen Leake <stephen_leake@stephe-leake.org> writes:

> I think the distinction between "tarball package" and "core package" is
> helpful here.
> 
> I'm guessing that the main motivation for including Org and Gnus in Emacs
> git (well, CVS back then?) was to include them in the release tarball. If we
> have a mechanism to allow that for ELPA packages, moving them to ELPA makes
> sense.

I like this. I think we have a good striation:

  core
  tarball ELPA
  net ELPA

To the user, core and tarball ELPA should be indistinguishable. I think also
that some tarball ELPA packages should come "pre-installed", if that is not
already done. This would makes them accessible via autoload, rather than
requiring the package interface to opt-in.

If core == tarball ELPA for everyone but us, this makes the decision easy
whether something should be in core or not a lot: We put it in core if core
requires it. If nothing at all in core uses the package -- and if that package
doesn't define "essential" functionality, like isearch.el or grep.el -- it can
shift to tarball ELPA.

This still leaves open the meaning of "essential", but it does make the
picture clearer.

John

RE: ELPA policy

[Drew Adams]

> > I think the distinction between "tarball package" and "core package" is
> > helpful here.
> >
> > I'm guessing that the main motivation for including Org and Gnus in
> > Emacs git (well, CVS back then?) was to include them in the release tarball.
> > If we have a mechanism to allow that for ELPA packages, moving them to ELPA
> > makes sense.
> 
> I like this. I think we have a good striation:
>   core
>   tarball ELPA
>   net ELPA
> To the user, core and tarball ELPA should be indistinguishable.

I haven't yet received an answer to my question whether anything
will change for users, depending on where you happen to manage
the code wrt ELPA etc.

But it sounds like the answer is yes.  If some stuff that has
traditionally been part of the distribution gets moved to (net)
ELPA, it will no longer be distributed to users.  They will
need to pull it down using the package interface.  Is that right?

If so, I'm not crazy about that.  I don't particularly want to
go fishing in (net) ELPA for stuff that I've always been able to
simply grep from within the distribution `lisp' directory. 
Especially, but not only, when I am not on the Internet.

I hope you will continue to (also) distribute Emacs with all
of its (traditional) source code, and not just make users
request it from (net) ELPA.

Re: ELPA policy

[John Wiegley]

>>>>> Drew Adams <drew.adams@oracle.com> writes:

> I haven't yet received an answer to my question whether anything will change
> for users, depending on where you happen to manage the code wrt ELPA etc.
> 
> But it sounds like the answer is yes. If some stuff that has traditionally
> been part of the distribution gets moved to (net) ELPA, it will no longer be
> distributed to users. They will need to pull it down using the package
> interface. Is that right?

Correct. Sorry I assumed that the discussion had answered your question in
passing.

> If so, I'm not crazy about that. I don't particularly want to go fishing in
> (net) ELPA for stuff that I've always been able to simply grep from within
> the distribution `lisp' directory. Especially, but not only, when I am not
> on the Internet.

Those things could then be in tarball ELPA. The distinction between tarball
ELPA and net ELPA is much more plastic than core and tarball ELPA. In fact, I
have a feeling that, initially, tarball ELPA and net ELPA will just be the
same thing, until we have a reason to tease them apart.

> I hope you will continue to (also) distribute Emacs with all of its
> (traditional) source code, and not just make users request it from (net)
> ELPA.

This is my preference as well.

John

Re: ELPA policy

[Stephen Leake]

Drew Adams <drew.adams@oracle.com> writes:

> Here's a naive question regarding where you place Lisp code
> that is distributed with Emacs (part of the tarball): Will
> anything change for users, depending on where you happen to
> manage the code wrt ELPA etc.?
>
> I'm guessing (and hoping) not, so that the distributed
> directory structure will remain the same.  Is that correct?

That's a good point. But I'm not clear what previous state you are
comparing to.

There are two cases;

1) A package that is currently in Emacs git moves to Gnu ELPA git but
   remains in the Emacs tarball.

2) A package that is currently in Gnu ELPA git stays there, and is added
   to the Emacs tarball.

I would argue that tarball ELPA packages should be installed in some
system level elpa directory (similar to the user's .emacs.d/elpa), not
in the main emacs/lisp install directory.

I don't think the history of a package should determine where it is
installed. So in case 1), I expect the package to be in a different
installation directory from the previous Emacs release. But it's still
in the default load-path, so this change is transparent to the user.

But it's not too important; the important distinction is that tarball
ELPA packages are _not_ in the developer Emacs workspace, so core
code cannot accidently depend on them.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

John Wiegley <jwiegley@gmail.com> writes:

> The distinction between tarball
> ELPA and net ELPA is much more plastic than core and tarball ELPA. In fact, I
> have a feeling that, initially, tarball ELPA and net ELPA will just be the
> same thing, until we have a reason to tease them apart.

Including a package in tarball ELPA means that the Emacs project has
decided that the package is sufficiently important to be made
immediately available to all Emacs users with no effort on their part
beyond installing Emacs from the tarball.

That implies a certain level of maturity and quality, and implies that
it should be favored over other similar packages.

On the other hand, we have in several cases recommended that someone put
a package in Gnu ELPA in order to give it more exposure, so that it
might eventually become higher quality and more mature.

In general, ELPA packages have far less oversight than core packages.
tarball ELPA packages should have the same oversight as core, or at
least more than normal ELPA packages.

So I disagree that all of Gnu ELPA should be included in the tarball.

Another choice would be to say "use MELPA for experimental packages",
but it is already too late for that.

-- 
-- Stephe

Re: ELPA policy

[Filipp Gunbin]

I do not consider myself competent enough in this area, but I'd like
to share some thoughts here:

For each package version there is a range (possibly sparse) of core
versions on which the package is supported (or just should work).

While the intent to move as much as possible in ELPA can be understood,
it leads to potentially more incompatibilities between important parts
which can provide API by themselves.

So, there should be some compromise between "latency" of new features
before they generally become available for use in packages and overall
core stability.  To me, it seems that the latter is more important and
it's better to keep infrastructure & library things in core, while
moving everything that uses them for a concrete purpose to ELPA.

If that in turn provides some API which others use, then we have
package interdependencies and that is probably OK (but can lead to
conflicts).  If sufficient amount of important packages use some API
and that API is rather mature, then the maintainer can decide to move
that in core to simplify dependency management.

So, I'd suggest that:

- Language features go straight into core (and people working on them
  / using them will have to use git version of Emacs until next
  release)

- New libraries which are not supposed to be in very common use go to
  ELPA first

- Then they are probably promoted to core as they get mature and more
  widely used - just to simplify their usage ("they will always be
  available").

- Major applications (like Gnus) and smaller ones always go to ELPA
  (most probably we should also bundle them in a tarball, but keep
  outside of the core).  A user can then decide whether to use git
  version of e.g. Gnus (from Elpa or private package repository) if she
  likes, or update to a released package version from Elpa (if core
  supports it), or just keep using the Elpa version she uses at the
  moment (which probably came together with current core).

- If such an application provides things useful for other
  applications, then it probably should extract that into a library to
  go through the cycle oulined above (elpa --maturity--> core).

The API / SPI notion can also be used to provide implementations
(backends) for different features which may have default simple
implementations in core and more advanced ones in packages.  A package
must somehow "register" itself as a candidate for being a service for
a concrete feature during installation.

Filipp

RE: ELPA policy

[Drew Adams]

> > Here's a naive question regarding where you place Lisp code
> > that is distributed with Emacs (part of the tarball): Will
> > anything change for users, depending on where you happen to
> > manage the code wrt ELPA etc.?
> >
> > I'm guessing (and hoping) not, so that the distributed
> > directory structure will remain the same.  Is that correct?
> 
> That's a good point. But I'm not clear what previous state you are
> comparing to.

Previous delivery of Emacs (back to ~Day One), including its
source code, especially Lisp, under directory lisp/.  What's
unclear about that?

> There are two cases;
> 1) A package that is currently in Emacs git moves to Gnu ELPA git but
>    remains in the Emacs tarball.
> 2) A package that is currently in Gnu ELPA git stays there, and is added
>    to the Emacs tarball.
> 
> I would argue that tarball ELPA packages should be installed in some
> system level elpa directory (similar to the user's .emacs.d/elpa), not
> in the main emacs/lisp install directory.

You say you would argue that, but where's the argument?  How about
a reason?

As one user, I would prefer that all distributed Lisp code be under
lisp/.  Makes things simpler for users (who might even have existing
scripts etc. that expect this).

Is there a good reason not to keep this behavior?

I don't really care where you stick code, or how you access it, for
development purposes.  Where an Emacs distribution puts it for
_users_ should remain what it has been, IMO, if possible - somewhere
under lisp/, at least.

Why should changes in where you keep code for development affect
where users of Emacs find it?

> I don't think the history of a package should determine where it is
> installed. So in case 1), I expect the package to be in a different
> installation directory from the previous Emacs release. But it's still
> in the default load-path, so this change is transparent to the user.

`load-path' is not everything, so no, it is _not_ transparent to
users, depending on what they do and how they use Emacs (including
its source code).

Emacs should be able to present its source code to users in the
same way as in the past, I would think.  They certainly should
not be _required_ to use the package system or git or ... to
access it.

`package.el' should be a nice-to-have.  It should not become an
obligatory hoop for users to jump through.  Most users will find
it handy, just as some users might find a menu handy.  But it
should not become a necessary way to access Emacs code.

> But it's not too important; the important distinction is that tarball
> ELPA packages are _not_ in the developer Emacs workspace, so core
> code cannot accidently depend on them.

That might be important to developers, but it is not the issue I
asked about.

Re: ELPA policy

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 419 bytes --]

Richard Stallman <rms@gnu.org> writes:

> It is fine to have it there, but this suggests we should maintain
> it inside Emacs.

I'm not sure I understand what you mean Richard.  seq.el is maintained
inside Emacs (in lisp/emacs-lisp/seq.el and
test/automated/seq-tests.el).  The version published in ELPA is there
for backward-compatibility, and does not have all the features (some
features do require Emacs 25).

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Another choice would be to say "use MELPA for experimental packages",
  > but it is already too late for that.

If my memory serves, MELPA contains packages we would like to include
in Emacs, but we can't get legal papers for them.
We are seeking people to write replacements for those, though we wish
it were not necessary.

We would like developers of future useful packages to want to put
their packages into the GNU ELPA.

To recommend putting some packages -- whatever they might be -- into
MELPA would work against these goals.  It would have been a bad
approach.

  > Which raises the question; what is the rationale for moving from MELPA
  > to Gnu ELPA?

If a package is useful to recommend, and we CAN get papers for it,
we definitely want to move it to GNU ELPA.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > It is fine to have it there, but this suggests we should maintain
  > > it inside Emacs.

  > I'm not sure I understand what you mean Richard.

I mean that we should maintain the file inside Emacs, 
not inside ELPA.

  >   seq.el is maintained
  > inside Emacs (in lisp/emacs-lisp/seq.el and
  > test/automated/seq-tests.el).

That means people are already doing what I think we should do.
I didn't have this information when I wrote my previous message.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> If a package is useful to recommend, and we CAN get papers for it, we
> definitely want to move it to GNU ELPA.

Yes, this is why I want to clearly define ELPA policy, and streamline things
as much as possible for developers and users: so that it becomes a more
attractive means for distributing Emacs packages. I think many people may be
largely ignoring it right now, and so they reach for MELPA. Since so many
people contribute to MELPA, they consider it a more attractive distribution
platform.

John

RE: ELPA policy

[Drew Adams]

> We would like developers of future useful packages to want to put
> their packages into the GNU ELPA.
> 
> To recommend putting some packages -- whatever they might be -- into
> MELPA would work against these goals.  It would have been a bad
> approach.

FWIW -

I will continue to use MELPA for my libraries, and not GNU ELPA,
for one reason:

I post my code to Emacs Wiki (to locked pages).  MELPA pulls
the code from the wiki (daily).  I do not need to use GIT etc.

This is the way I prefer to work.  Call it old-fashioned or
bizarre; I don't care.

So you might recommend that packages not be put in MELPA, but
some will continue to be put there, including perhaps some that
you might someday want to include in Emacs.

>   > Which raises the question; what is the rationale for moving from
>   > MELPA to Gnu ELPA?
> 
> If a package is useful to recommend, and we CAN get papers for it,
> we definitely want to move it to GNU ELPA.

This was refused by Stefan for my libraries (for which FSF has papers),
based on the fact that I do not use GIT, so any updates I make to them
would not be done directly in the repository.  It was not acceptable to
update elsewhere (e.g. the wiki) and then have someone or a program
pull from there to the repository when appropriate.  So be it.

Re: ELPA policy

[John Wiegley]

>>>>> Drew Adams <drew.adams@oracle.com> writes:

> I will continue to use MELPA for my libraries, and not GNU ELPA, for one
> reason:
>
> I post my code to Emacs Wiki (to locked pages). MELPA pulls the code from
> the wiki (daily). I do not need to use GIT etc.
>
> This is the way I prefer to work. Call it old-fashioned or bizarre; I don't
> care.

This is a legitimate complaint, Drew. One of the things I want to avoid with
ELPA is making it harder for people to contribute to than other alternatives.

Since pulling in data from the Web indiscriminately is not possible for this
project (we have to make sure we're not unwittingly including code without a
proper copyright assignment), perhaps we need as an "integrator": someone
willing to guide the update of ELPA from various sources on the Web, keeping
an eye out for changes that might affect copyright.

It wouldn't be too hard to setup a process that updates a local clone of ELPA
with all your latest versions from GitHub and EmacsWiki, and then to check
over a diff of those changes before pushing them to ELPA. One might even
suggest that you could do this yourself, as part of your own release work.
Would that be something you'd find worth doing?

John

RE: ELPA policy

[Drew Adams]

> This is a legitimate complaint, Drew. One of the things I want to avoid
> with ELPA is making it harder for people to contribute to than other
> alternatives.
> 
> Since pulling in data from the Web indiscriminately is not possible for
> this project (we have to make sure we're not unwittingly including code without
> a proper copyright assignment), perhaps we need as an "integrator": someone
> willing to guide the update of ELPA from various sources on the Web,
> keeping an eye out for changes that might affect copyright.

I'm glad to hear that; all of it.  And thanks for replying.

> It wouldn't be too hard to setup a process that updates a local clone of
> ELPA with all your latest versions from GitHub and EmacsWiki, and then to check
> over a diff of those changes before pushing them to ELPA. One might even
> suggest that you could do this yourself, as part of your own release work.
> Would that be something you'd find worth doing?

I don't want to develop anything like that; sorry.  I don't want to fiddle
with a local GIT clone.  I would be willing to identify versions of my
libraries to be pulled at specific times ("releases") from the wiki (or
from MELPA) to be put (updated) in GNU ELPA.

I doubt that my case is typical, however.  I'm not saying that Emacs Dev
needs to cater to such a case.  I'm just explaining that I update my
libraries locally, without GIT etc., and I upload updates to the wiki.
They are automatically pulled to MELPA.

What do I need to know, currently?  Only how to write some Lisp (a
library) and how to copy and paste it into a Web page.  Simple.  Maybe
too simple to accommodate by Emacs Dev.

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> For each package version there is a range (possibly sparse) of core
> versions on which the package is supported (or just should work).

For normal ELPA packages, that's true.

For core and tarball ELPA packages, they must work in each current Emacs
release (since they are in the release tarball). They may also support
some set of previous Emacs releases.

> While the intent to move as much as possible in ELPA can be understood,
> it leads to potentially more incompatibilities between important parts
> which can provide API by themselves.

Yes.

> So, there should be some compromise between "latency" of new features
> before they generally become available for use in packages and overall
> core stability.  To me, it seems that the latter is more important and
> it's better to keep infrastructure & library things in core, while
> moving everything that uses them for a concrete purpose to ELPA.

For normal ELPA packages, that's true. But this is partly why core and
tarball ELPA packages are being considered.

> If that in turn provides some API which others use, then we have
> package interdependencies and that is probably OK (but can lead to
> conflicts). 

A more flexible requirements spec may  be needed. For example, Debian
packages allow specifying a range of versions in the dependency; that
can reflect actual testing, and not rely on implicit promises of future
backward compatibility. Of course, it can also lead to false failures.

> If sufficient amount of important packages use some API and that API
> is rather mature, then the maintainer can decide to move that in core
> to simplify dependency management.

or to a tarball ELPA package, or to a core ELPA package.

Perhaps that is too much choice?

> So, I'd suggest that:
>
> - Language features go straight into core (and people working on them
>   / using them will have to use git version of Emacs until next
>   release)

By "language features" do you mean things like ada-mode.el (primary mode
for editing Ada source files)? If so, I strongly disagree; ada-mode.el
is very happy as a normal ELPA package. Similar modes for other
languages with a small audience could be in ELPA as well; I have not
looked to see what's actually there.

cc-mode is the core for a lot of similar C-like languages, so it
probably makes sense to keep that in core. But it would be interesting
to see what the consequences of moving it to a normal or tarball ELPA
package would be.

But if you mean things like parsers and xref support, then I agree.
Although ada-mode also provides its own parser ...

> - New libraries which are not supposed to be in very common use go to
>   ELPA first

> - Then they are probably promoted to core as they get mature and more
>   widely used - just to simplify their usage ("they will always be
>   available").

That's what tarball ELPA packages are for.

The only reason to move a package to actual core (ie emacs git, as
opposed to elpa git) is if some other core code wants to depend in it.

> - Major applications (like Gnus) and smaller ones always go to ELPA
>   (most probably we should also bundle them in a tarball, but keep
>   outside of the core).

Right; that's a tarball ELPA package.

> - If such an application provides things useful for other
>   applications, then it probably should extract that into a library to
>   go through the cycle oulined above (elpa --maturity--> core).

Yes.

> The API / SPI notion can also be used to provide implementations
> (backends) for different features which may have default simple
> implementations in core and more advanced ones in packages.  A package
> must somehow "register" itself as a candidate for being a service for
> a concrete feature during installation.

If we use cl-generic to provide dispatching, the cl-defmethod form does
the registration.

-- 
-- Stephe

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Since pulling in data from the Web indiscriminately is not
  > possible for this project (we have to make sure we're not
  > unwittingly including code without a proper copyright assignment),
  > perhaps we need as an "integrator": someone willing to guide the
  > update of ELPA from various sources on the Web, keeping an eye out
  > for changes that might affect copyright.

What we need to solve Drew's problem is to merge changes into a few
specific files from a specific place, and commit them using git.
Isn't that easy enough to do?

There could be one page where he provides the commit log info.
Whenever that page changes (we could check every 5 minutes), our demon
could merge in all the files which have changed.  They it would email
the diffs to Drew so he could confirm what he installed.

Thus, when Drew wants to install a new version, he would change the
other pages first, then write the change log text into the log page.
Then wait 5 minutes and it's done.

This is assuming there are no other obstacles aside from the
mechanics of installing changes into our repository.



-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

RE: ELPA policy

[Drew Adams]

>   > Since pulling in data from the Web indiscriminately is not
>   > possible for this project (we have to make sure we're not
>   > unwittingly including code without a proper copyright assignment),
>   > perhaps we need as an "integrator": someone willing to guide the
>   > update of ELPA from various sources on the Web, keeping an eye out
>   > for changes that might affect copyright.
> 
> What we need to solve Drew's problem is to merge changes into a few
> specific files from a specific place, and commit them using git.
> Isn't that easy enough to do?
> 
> There could be one page where he provides the commit log info.
> Whenever that page changes (we could check every 5 minutes), our demon
> could merge in all the files which have changed.  They it would email
> the diffs to Drew so he could confirm what he installed.
> 
> Thus, when Drew wants to install a new version, he would change the
> other pages first, then write the change log text into the log page.
> Then wait 5 minutes and it's done.
> 
> This is assuming there are no other obstacles aside from the
> mechanics of installing changes into our repository.

I will again say that I don't expect that my situation is typical.
There may be no one else out there who ever prefers something like it.

But FWIW I think that what you describe would work for me.

Re: ELPA policy

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 787 bytes --]

On 11 Nov 2015 11:27 pm, "Richard Stallman" <rms@gnu.org> wrote:
>
> There could be one page where he provides the commit log info.
> Whenever that page changes (we could check every 5 minutes), our demon
> could merge in all the files which have changed.  They it would email
> the diffs to Drew so he could confirm what he installed.

Richard, do think such as system would be viable copyright-wise?
Of course, we would only do this when the author assigns copyright, but
there's a very real possibility that a careless maintainer might add
someone else's code to this page. And then this (non-copyright-assigned)
code would be automatically pulled and committed to Elpa.

I ask this because if we are OK with this risk then I'm perfectly willing
to write such a script (eager, even).

[-- Attachment #2: Type: text/html, Size: 949 bytes --]

Re: ELPA policy

[Dmitry Gutov]

On 11/12/2015 02:35 AM, Artur Malabarba wrote:

> Richard, do think such as system would be viable copyright-wise?
> Of course, we would only do this when the author assigns copyright, but
> there's a very real possibility that a careless maintainer might add
> someone else's code to this page. And then this (non-copyright-assigned)
> code would be automatically pulled and committed to Elpa.

We should also consider whether this is sound from the security 
standpoint. I mean, pushing the code to ELPA without any review from the 
core developers.

Re: ELPA policy

[Stephen Leake]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Since pulling in data from the Web indiscriminately is not
>   > possible for this project (we have to make sure we're not
>   > unwittingly including code without a proper copyright assignment),
>   > perhaps we need as an "integrator": someone willing to guide the
>   > update of ELPA from various sources on the Web, keeping an eye out
>   > for changes that might affect copyright.
>
> What we need to solve Drew's problem is to merge changes into a few
> specific files from a specific place, and commit them using git.
> Isn't that easy enough to do?
>
> There could be one page where he provides the commit log info.
> Whenever that page changes (we could check every 5 minutes), our demon
> could merge in all the files which have changed.  They it would email
> the diffs to Drew so he could confirm what he installed.
>
> Thus, when Drew wants to install a new version, he would change the
> other pages first, then write the change log text into the log page.
> Then wait 5 minutes and it's done.
>
> This is assuming there are no other obstacles aside from the
> mechanics of installing changes into our repository.

Any malicious hacker can drop completely different code in that web
page, and thus get it into Gnu ELPA.

We will have replaced the security of private machines with whatever web
login that web page requires; that's a huge step backwards.

-- 
-- Stephe

Re: ELPA policy

[Filipp Gunbin]

Stephen,

On 11/11/2015 15:22 -0600, Stephen Leake wrote:

>> If sufficient amount of important packages use some API and that API
>> is rather mature, then the maintainer can decide to move that in core
>> to simplify dependency management.
>
> or to a tarball ELPA package, or to a core ELPA package.
>
> Perhaps that is too much choice?

Mm yes, that's one of my fears, that it will become too complex for a
package maintaner.  Why not just treat each ELPA package just as an ELPA
package and leave the option of bundling it to core maintainers which
are better in this area (they'll do it in agreement with package
maintainer, of course)?

A "tarball" ELPA package is a one thing (that's what I call "bundle into
tarball", if I understood right), but "core" ELPA package is another -
here I have another fear, that normal and tarball ELPA packages
depending on such "core" packages, will have to accurately define
versions of the core package they support, and then we have to check all
this during the installation.  We'll probably have to calculate and
offer to user which set of the multiple core packages' multiple versions
suits for multiple normal and tarball (perhaps overriden by version from
Internet package archive) packages, at the same time probably giving the
user to opportunity to specify her preferred version of each requested
package.  Is it worth the trouble?  Or do I understand something wrong?

That's why I wrote about the feature latency - maybe it would be enough
if a person willing to use latest core features in her package will have
to develop it on git emacs and wait for the next official release to
make her package available to the public.  This will be the same as with
new C level features, which we cannot "push quicky" as we can with ELPA
package.

>> So, I'd suggest that:
>>
>> - Language features go straight into core (and people working on them
>>   / using them will have to use git version of Emacs until next
>>   release)
>
> By "language features" do you mean things like ada-mode.el (primary mode
> for editing Ada source files)? If so, I strongly disagree; ada-mode.el
> is very happy as a normal ELPA package. Similar modes for other
> languages with a small audience could be in ELPA as well; I have not
> looked to see what's actually there.

No-no, what I meant were Emacs Lisp libraries extending language, like
seq.el.

>> The API / SPI notion can also be used to provide implementations
>> (backends) for different features which may have default simple
>> implementations in core and more advanced ones in packages.  A package
>> must somehow "register" itself as a candidate for being a service for
>> a concrete feature during installation.
>
> If we use cl-generic to provide dispatching, the cl-defmethod form does
> the registration.

Thanks, that's one more reason for me to become acquainted with Emacs CL
features.

Filipp

Re: ELPA policy

[Phillip Lord]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Richard Stallman <rms@gnu.org> writes:
>
>> If a package is useful to recommend, and we CAN get papers for it, we
>> definitely want to move it to GNU ELPA.
>
> Yes, this is why I want to clearly define ELPA policy, and streamline things
> as much as possible for developers and users: so that it becomes a more
> attractive means for distributing Emacs packages. I think many people may be
> largely ignoring it right now, and so they reach for MELPA. Since so many
> people contribute to MELPA, they consider it a more attractive distribution
> platform.


MELPA is also *easier* to contribute to. Aside from copyright issues, it
involves writing something that looks like lisp, testing on your local
fork, then a PR.

With GNU ELPA, it involves some fairly obscure git cleverness. GNU ELPA
could be made easier by allowing recipes, and by accepting PRs. This
avoids the necessity to have commit access to GNU ELPA to contribute.

Finally, I think that core devs should contribute to individual packages
by PR to their repos.

Phil

RE: ELPA policy

[Drew Adams]

> Any malicious hacker can drop completely different code in that web
> page, and thus get it into Gnu ELPA.  We will have replaced the
> security of private machines with whatever web login that web page
> requires; that's a huge step backwards.

Again: "I post my code to Emacs Wiki (to locked pages)."

But of course that doesn't mean it is uncrackable.

Re: ELPA policy

[John Wiegley]

>>>>> Phillip Lord <phillip.lord@russet.org.uk> writes:

> With GNU ELPA, it involves some fairly obscure git cleverness. GNU ELPA
> could be made easier by allowing recipes, and by accepting PRs. This avoids
> the necessity to have commit access to GNU ELPA to contribute.

We can always use the power of Git here: Keep a fork of ELPA.git on GitHub,
accept PRs there, and periodically merge back in.

John

Re: ELPA policy

[John Wiegley]

>>>>> Filipp Gunbin <fgunbin@fastmail.fm> writes:

> A "tarball" ELPA package is a one thing (that's what I call "bundle into
> tarball", if I understood right), but "core" ELPA package is another

I must have missed this distinction. What is the difference between "tarball
ELPA" and "core ELPA"?  I was thinking of "core" as anything that is in
Emacs.git.

John

Re: ELPA policy

[Alex Schröder]

Drew Adams <drew.adams@oracle.com> writes:

> Again: "I post my code to Emacs Wiki (to locked pages)."
>
> But of course that doesn't mean it is uncrackable.

Given that Emacs Wiki currently does not use HTTPS, we should consider
Emacs Wiki insecure. Even then, the admin passwords are shared and there
are no other efforts made to identify editors once they present the
admin password. The current setup only protects against the simplest of
threat models, e.g. spammers, vandals, well-meaning contributors that
want to edit wiki pages instead of informing maintainers, and so on. I
think that's the threat model wiki pages need to defend against, but I
suspect that's not good enough for an inclusion into Emacs Core.

-- 
Public Key Fingerprint = DF94 46EB 7B78 4638 7CCC  018B C78C A29B ACEC FEAE

Re: ELPA policy

[Filipp Gunbin]

On 12/11/2015 09:11 -0800, John Wiegley wrote:

>>>>>> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>
>> A "tarball" ELPA package is a one thing (that's what I call "bundle into
>> tarball", if I understood right), but "core" ELPA package is another
>
> I must have missed this distinction. What is the difference between "tarball
> ELPA" and "core ELPA"?  I was thinking of "core" as anything that is in
> Emacs.git.

If I got it right, a "tarball" package is copied from elpa.git into the
tarball at release time.  This is to bundle package so that Internet is
not required to install it (whether or not it is installed by default).

"core" package is developed in emacs.git (because core code depends on
it and we have to have full environment in the repo), but then can be
updated from elpa if new version is available.  This is to allow
frequent updates of such packages.

If I didn't get it right, then probably other package maintainers would
feel a bit lost, too, unless we provide concise explanation of this
separation.

Filipp

Re: ELPA policy

[John Wiegley]

>>>>> Filipp Gunbin <fgunbin@fastmail.fm> writes:

> "core" package is developed in emacs.git (because core code depends on it
> and we have to have full environment in the repo), but then can be updated
> from elpa if new version is available. This is to allow frequent updates of
> such packages.

Ah, that makes perfect sense, thanks.

John

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> Stephen,
>
> On 11/11/2015 15:22 -0600, Stephen Leake wrote:
>
>>> If sufficient amount of important packages use some API and that API
>>> is rather mature, then the maintainer can decide to move that in core
>>> to simplify dependency management.
>>
>> or to a tarball ELPA package, or to a core ELPA package.
>>
>> Perhaps that is too much choice?
>
> Mm yes, that's one of my fears, that it will become too complex for a
> package maintaner.  Why not just treat each ELPA package just as an ELPA
> package and leave the option of bundling it to core maintainers which
> are better in this area (they'll do it in agreement with package
> maintainer, of course)?
>
> A "tarball" ELPA package is a one thing (that's what I call "bundle into
> tarball", if I understood right), 

Yes. We have not implemented this yet, but I'm imagining there is a list
of these in the Gnu Emacs Makefile somewhere (probably in a separate
file read by the Makefile). I don't think there will need to be any
metadata in the package files for this.

Declaring an ELPA package to be a tarball package does impose some
restrictions on the package maintainer; they have to synchronize with
each Emacs release, and accept the same review/oversight as core code.

> but "core" ELPA package is another -
> here I have another fear, that normal and tarball ELPA packages
> depending on such "core" packages, will have to accurately define
> versions of the core package they support, and then we have to check all
> this during the installation.  

I don't see why that is different from depending on a normal ELPA
package. 

> We'll probably have to calculate and offer to user which set of the
> multiple core packages' multiple versions suits for multiple normal
> and tarball (perhaps overriden by version from Internet package
> archive) packages, at the same time probably giving the user to
> opportunity to specify her preferred version of each requested
> package. Is it worth the trouble? Or do I understand something wrong?

Emacs does not support multiple versions of packages available
simultaneously; there is only one instance of a package that is first in
load-path.

You can end up with one copy in the installed Emacs distribution, and
one in ~/.emacs.d/elpa. But the elpa one will take precedence; that is
the only one that other packages have to worry about. It either meets
their dependency requirement, or not.

> That's why I wrote about the feature latency - maybe it would be enough
> if a person willing to use latest core features in her package will have
> to develop it on git emacs and wait for the next official release to
> make her package available to the public.  This will be the same as with
> new C level features, which we cannot "push quicky" as we can with ELPA
> package.

That's the main reason to make a package available in both core and ELPA:
users of the released version of Emacs can use the latest version of the
core ELPA package from ELPA, overriding the copy in their installation.

>>> So, I'd suggest that:
>>>
>>> - Language features go straight into core (and people working on them
>>>   / using them will have to use git version of Emacs until next
>>>   release)
>
> No-no, what I meant were Emacs Lisp libraries extending language, like
> seq.el.

That's a good candidate for a core ELPA package.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> John Wiegley <jwiegley@gmail.com> writes:
>
>>>>>>> Richard Stallman <rms@gnu.org> writes:
>>
>>> If a package is useful to recommend, and we CAN get papers for it, we
>>> definitely want to move it to GNU ELPA.
>>
>> Yes, this is why I want to clearly define ELPA policy, and streamline things
>> as much as possible for developers and users: so that it becomes a more
>> attractive means for distributing Emacs packages. I think many people may be
>> largely ignoring it right now, and so they reach for MELPA. Since so many
>> people contribute to MELPA, they consider it a more attractive distribution
>> platform.
>
>
> MELPA is also *easier* to contribute to. Aside from copyright issues, it
> involves writing something that looks like lisp, testing on your local
> fork, then a PR.
>
> With GNU ELPA, it involves some fairly obscure git cleverness. 

It's only normal git cleverness; there's nothing special about the Gnu
ELPA git. Unless you are using an "external" package, perhaps.

I can understand treating all of git as "obscure", if that's what you
meant. I much prefer monotone.

> GNU ELPA could be made easier by allowing recipes, and by accepting
> PRs. This avoids the necessity to have commit access to GNU ELPA to
> contribute.

"PR" is "Pull Request"? Doesn't that mean "pull from my git repository"?
Or is it more general than that?

> Finally, I think that core devs should contribute to individual packages
> by PR to their repos.

I'm guessing "their repos" is _not_ Gnu ELPA git? So this only applies
to packages that have a primary repo that is not Gnu ELPA git.

I agree that any one that edits a Gnu ELPA package in Gnu ELPA git
should also notify the upstream project if there is one. But they do not
have to wait for approval if it's a critical bug.

Normal ettiquette should apply of course.

This is more important for tarball and core ELPA packages, since they
are part of the blessed Emacs standard library; Emacs developers can
edit those as if they were in core.

-- 
-- Stephe

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Any malicious hacker can drop completely different code in that web
  > page, and thus get it into Gnu ELPA.

Drew said the pages were locked.
Doesn't that mean that only he has access to change them?

  > We will have replaced the security of private machines with whatever web
  > login that web page requires; that's a huge step backwards.

I think you are concerned that someone might break the security on that other
server and then install changes on it using Drew's account.

In general, someone who breaks the security on a machine used by
an Emacs contributor might be able to insert changes in Emacs
by pretending to be that contributor.  I don't think this is
fundamentally different.  But maybe the web site's security is
not quite as good.

We can make the security tighter.  Drew, are you willing to GPG-sign
your new versions?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Richard, do think such as system would be viable copyright-wise?

In terms of copyright, there is no difference between the system I
proposed (installing what Drew puts in certain web pages) and the
method Drew doesn't like, which is for him to install the same changes
via Git.

Whichever method he uses, he technically could install code written by
others whom we don't have papers for.  If we permit him to use Git to
install changes, that entails trusting him to handle this
responsibility.  If we can trust him to do it with one method, we can
trust him to do it with the other method.

  > We should also consider whether this is sound from the security 
  > standpoint. I mean, pushing the code to ELPA without any review from the 
  > core developers.

The same would occur if we invite him to install changes in his packages
through Git.

If we can check enough one way, we can check enough the other way.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

RE: ELPA policy

[Drew Adams]

>   > Any malicious hacker can drop completely different code in that web
>   > page, and thus get it into Gnu ELPA.
> 
> Drew said the pages were locked.
> Doesn't that mean that only he has access to change them?

No, anyone with admin privileges for the wiki has access to do so.
There are a few people in this category.  And see Alex Schroeder's
clarification of what this means.  This is not watertight security,
by any means.

Perhaps one way to look at it is similar to submitting something
by email (which would be another possibility, for me).

>   > We will have replaced the security of private machines with whatever
>   > web login that web page requires; that's a huge step backwards.
> 
> I think you are concerned that someone might break the security on that
> other server and then install changes on it using Drew's account.

See above.

> In general, someone who breaks the security on a machine used by
> an Emacs contributor might be able to insert changes in Emacs
> by pretending to be that contributor.  I don't think this is
> fundamentally different.  But maybe the web site's security is
> not quite as good.
> 
> We can make the security tighter.  Drew, are you willing to GPG-sign
> your new versions?

I don't really know what that entails.

Dunno whether you really want to discuss my case in particular in
detail here.  Again, I doubt that it is typical.  The reason for
my initial message about this was to suggest that some people do
use MELPA, and that perhaps some way to accommodate them could be
devised.  But maybe not.

To repeat the summary of my original point:

  So you might recommend that packages not be put in MELPA, but
  some will continue to be put there, including perhaps some that
  you might someday want to include in Emacs.

Regarding my own case, this was the point:

  I do not use GIT, so any updates I make to them
  would not be done directly in the repository.  It was not
  acceptable to update elsewhere (e.g. the wiki) and then have
  someone or a program pull from there to the repository when
  appropriate.

In sum, some people will post code to MELPA, including some
that you might someday want in Emacs.  And some input to MELPA
comes from the wiki, not from GIT - but this is probably a small
portion of what is in MELPA.

Re: ELPA policy

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

On 12 Nov 2015 10:34 pm, "Richard Stallman" <rms@gnu.org> wrote:
>   > Richard, do think such as system would be viable copyright-wise?
>
> In terms of copyright, there is no difference between the system I
> proposed (installing what Drew puts in certain web pages) and the
> method Drew doesn't like, which is for him to install the same changes
> via Git.
>
> Whichever method he uses, he technically could install code written by
> others whom we don't have papers for.  If we permit him to use Git to
> install changes, that entails trusting him to handle this
> responsibility.  If we can trust him to do it with one method, we can
> trust him to do it with the other method.

I'm trying to determine whether this method can be employed as a general
elpa mechanism. Of course we trust Drew, but would it be acceptable to use
such a script with other people who submit packages to Elpa? (assuming the
author has assigned copyright)

[-- Attachment #2: Type: text/html, Size: 1135 bytes --]

Re: ELPA policy

[Eli Zaretskii]

> Date: Thu, 12 Nov 2015 15:05:50 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: jwiegley@gmail.com, drew.adams@oracle.com, emacs-devel@gnu.org
> 
> Regarding my own case, this was the point:
> 
>   I do not use GIT, so any updates I make to them
>   would not be done directly in the repository.  It was not
>   acceptable to update elsewhere (e.g. the wiki) and then have
>   someone or a program pull from there to the repository when
>   appropriate.
> 
> In sum, some people will post code to MELPA, including some
> that you might someday want in Emacs.  And some input to MELPA
> comes from the wiki, not from GIT - but this is probably a small
> portion of what is in MELPA.

FWIW, I very much doubt that many people nowadays would share Drew's
issue with using Git.  Using Git is a no-brainer nowadays.  I actually
think this is a very unique situation, and anyone else who insists on
using MELPA doesn't do that for such simple technical reasons.  I
think the legal paperwork is the main issue, and also perhaps not
everyone would like to be accountable for their code practices before
the Emacs maintainers and their peculiar requirements and
expectations.

Re: ELPA policy

[Filipp Gunbin]

On 12/11/2015 13:52 -0600, Stephen Leake wrote:

>> A "tarball" ELPA package is a one thing (that's what I call "bundle into
>> tarball", if I understood right), 
>
> Yes. We have not implemented this yet, but I'm imagining there is a list
> of these in the Gnu Emacs Makefile somewhere (probably in a separate
> file read by the Makefile). I don't think there will need to be any
> metadata in the package files for this.
>
> Declaring an ELPA package to be a tarball package does impose some
> restrictions on the package maintainer; they have to synchronize with
> each Emacs release, and accept the same review/oversight as core code.

They'll just have to make sure that a single version (which is going to
be bundled in a tarball) works good with an Emacs release being
prepared.

>> but "core" ELPA package is another -
>> here I have another fear, that normal and tarball ELPA packages
>> depending on such "core" packages, will have to accurately define
>> versions of the core package they support, and then we have to check all
>> this during the installation.  
>
> I don't see why that is different from depending on a normal ELPA
> package.

I think it's their dependants what make them different from tarball and
normal packages.

Emacs core provides API, which others use.  A normal package declares
which versions of this API it is supposed to work against.

With core packages, Emacs still provides API, but it now consists of a
static part (C level & core) and dynamic part (core packages, which can
later be updated from ELPA - correct me if I'm wrong).  So, a package
should independently define which core version it works agains and which
core package(s) version(s) it works against.

Here's where I see the complexity with multiple packages installed on
user request with package manager, which I tried to describe below.

>> We'll probably have to calculate and offer to user which set of the
>> multiple core packages' multiple versions suits for multiple normal
>> and tarball (perhaps overriden by version from Internet package
>> archive) packages, at the same time probably giving the user to
>> opportunity to specify her preferred version of each requested
>> package. Is it worth the trouble? Or do I understand something wrong?
>
> Emacs does not support multiple versions of packages available
> simultaneously; there is only one instance of a package that is first in
> load-path.
>
> You can end up with one copy in the installed Emacs distribution, and
> one in ~/.emacs.d/elpa. But the elpa one will take precedence; that is
> the only one that other packages have to worry about. It either meets
> their dependency requirement, or not.

Of course, I was talking about the set of available versions which could
be installed and from which a user or a package manager should choose.

>> That's why I wrote about the feature latency - maybe it would be enough
>> if a person willing to use latest core features in her package will have
>> to develop it on git emacs and wait for the next official release to
>> make her package available to the public.  This will be the same as with
>> new C level features, which we cannot "push quicky" as we can with ELPA
>> package.
>
> That's the main reason to make a package available in both core and ELPA:
> users of the released version of Emacs can use the latest version of the
> core ELPA package from ELPA, overriding the copy in their installation.

Yes, but it can introduce complexities I wrote of above.

Filipp

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > MELPA is also *easier* to contribute to. Aside from copyright issues, it
  > involves writing something that looks like lisp, testing on your local
  > fork, then a PR.

What is a PR?

  > With GNU ELPA, it involves some fairly obscure git cleverness. GNU ELPA
  > could be made easier by allowing recipes, and by accepting PRs. This
  > avoids the necessity to have commit access to GNU ELPA to contribute.

Maybe we should do this (but I don't know what a PR is).

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > We can make the security tighter.  Drew, are you willing to GPG-sign
  > > your new versions?

  > I don't really know what that entails.

It means you would make diffs to show the change to install,
plus a log entry, sign that whole thing with GnuPG, and mail
it to a certain address.  The signature would demonstrate it came
from you.

Is that ok for you?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

RE: ELPA policy

[Drew Adams]

>   > > We can make the security tighter.  Drew, are you willing
>   > > to GPG-sign your new versions?
> 
>   > I don't really know what that entails.
> 
> It means you would make diffs to show the change to install,
> plus a log entry, sign that whole thing with GnuPG, and mail
> it to a certain address.  The signature would demonstrate it
> came from you.
> 
> Is that ok for you?

I have no problem creating diffs & log entries, and emailing
them in.  I've mailed in plenty of patches that way.

Dunno what "sign that whole thing with GnuPG" entails.  What
is needed for that (e.g., on MS Windows)?

Looking at the doc at https://www.gnupg.org/documentation/howtos.html
and http://www.dewinter.com/gnupg_howto/english/GPGMiniHowto-2.html,
it seems I would need to install GnuPG from source code,
using a C compiler etc.  I don't plan to install and use tools
such as a C compiler on my laptop, so I guess the answer would
be no; sorry.

If there were an easy-to-use Windows binary for such signing,
then I would probably do as you propose.

(I don't do software development anymore.  I use MS Windows
binaries of Emacs, and I write some Emacs-Lisp code.  I do
have (an old version of) Cygwin installed, so that I can use
a few simple commands from within Emacs.  That's about it.)

I don't mean to make a big deal about this.  I'm not saying
no definitively.  But AFAIK there is not necessarily even
any interest in adding any of the code I've written to Emacs.
I don't feel like going through a lot of preparation for
nothing.

My libraries are there, and have been for quite a while.
If someone is really interested in something, let me know
and maybe we can work something out.  Otherwise, we are
likely wasting each other's time here.  I don't mean to
waste your time.  (But I do appreciate your having
considered this.)

Again, my original point was about MELPA and updates to
it from the wiki.  And again, my use of MELPA is likely
not typical.  Perhaps I should have said nothing at all
about it.

Re: ELPA policy

[JJ Asghar]

On 11/13/15 3:58 PM, Richard Stallman wrote:
>   > MELPA is also *easier* to contribute to. Aside from copyright issues, it
>   > involves writing something that looks like lisp, testing on your local
>   > fork, then a PR.
> 
> What is a PR?

A great detailed explanation of a PR is here[1]. A short explanation is
a request to put in a patch to a repository.

[1]: https://help.github.com/articles/using-pull-requests/

-JJ

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Dunno what "sign that whole thing with GnuPG" entails.  What
  > is needed for that (e.g., on MS Windows)?

GnuPG works on Windows.  I expect we can find someone to help
you install it.  Then we need you to send us your public key
so we can verify your signatures.  We can find a way to do that.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Achim Gratz]

Drew Adams writes:
> Dunno what "sign that whole thing with GnuPG" entails.  What
> is needed for that (e.g., on MS Windows)?

http://www.gpg4win.org/

> (I don't do software development anymore.  I use MS Windows
> binaries of Emacs, and I write some Emacs-Lisp code.  I do
> have (an old version of) Cygwin installed, so that I can use
> a few simple commands from within Emacs.  That's about it.)

You'd really be off easier if you updated Cygwin so that you could use
Git and SSH and GPG from that.  Perhaps you'll find that using Emacs
from Cygwin is a lot easier now, too -- you can use emacs-w32 if you
don't care about running X11.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> On 12/11/2015 09:11 -0800, John Wiegley wrote:
>
>>>>>>> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>>
>>> A "tarball" ELPA package is a one thing (that's what I call "bundle into
>>> tarball", if I understood right), but "core" ELPA package is another
>>
>> I must have missed this distinction. What is the difference between "tarball
>> ELPA" and "core ELPA"?  I was thinking of "core" as anything that is in
>> Emacs.git.
>
> If I got it right, a "tarball" package is copied from elpa.git into the
> tarball at release time.  This is to bundle package so that Internet is
> not required to install it (whether or not it is installed by default).
>
> "core" package is developed in emacs.git (because core code depends on
> it and we have to have full environment in the repo), but then can be
> updated from elpa if new version is available.  This is to allow
> frequent updates of such packages.

Yes.

I'm working on writing this up for admin/notes/elpa.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> On 12/11/2015 13:52 -0600, Stephen Leake wrote:
>
>> I don't see why that is different from depending on a normal ELPA
>> package.
>
> I think it's their dependants what make them different from tarball and
> normal packages.
>
> Emacs core provides API, which others use.  A normal package declares
> which versions of this API it is supposed to work against.
>
> With core packages, Emacs still provides API, but it now consists of a
> static part (C level & core) and dynamic part (core packages, which can
> later be updated from ELPA - correct me if I'm wrong).  So, a package
> should independently define which core version it works agains and which
> core package(s) version(s) it works against.

package.el dependencies can only specify a minimum version, not a
maximum. there is no way that a normal ELPA package can declare that it
works with seq.el 1.0 but not seq.el 1.1

So if emacs 25 contains a core ELPA package seq.el 1.0, the declaration

;; Package-Requires: ((emacs "25.1"))

is equivalent to:

;; Package-Requires: ((emacs "25.1") (seq "1.0))

If seq.el 1.1 is later released via the Gnu ELPA server, it will be used
in either case.

Listing a core ELPA package explicitly is only necessary when the emacs
version is less than the first version that included the core ELPA
package version.

On the other hand, it doesn't hurt, so it's probably best to recommend
listing all ELPA packages explicitly.

>>> We'll probably have to calculate and offer to user which set of the
>>> multiple core packages' multiple versions suits for multiple normal
>>> and tarball (perhaps overriden by version from Internet package
>>> archive) packages, at the same time probably giving the user to
>>> opportunity to specify her preferred version of each requested
>>> package. Is it worth the trouble? Or do I understand something wrong?
>>
>> Emacs does not support multiple versions of packages available
>> simultaneously; there is only one instance of a package that is first in
>> load-path.
>>
>> You can end up with one copy in the installed Emacs distribution, and
>> one in ~/.emacs.d/elpa. But the elpa one will take precedence; that is
>> the only one that other packages have to worry about. It either meets
>> their dependency requirement, or not.
>
> Of course, I was talking about the set of available versions which could
> be installed and from which a user or a package manager should choose.

I still don't see a problem.

We have an example of a core ELPA package now; ada-mode. version 4.3 is
in emacs git; version 5.1.8 is in ELPA (if I ever get 5.x to be fast
enough on huge files, I'll delete 4.3 from core).

package.el has no issues with this.

Similar things can occur when there are different versions of the same
package in multiple repositories. In that sense, emacs git is just
another package repository.

Can you be more explicit about what problem you foresee? Give an example
of a package that would cause a problem.

-- 
-- Stephe

Re: ELPA policy

[Stephen Leake]

Richard Stallman <rms@gnu.org> writes:

>   > We will have replaced the security of private machines with whatever web
>   > login that web page requires; that's a huge step backwards.
>
> I think you are concerned that someone might break the security on that other
> server and then install changes on it using Drew's account.
>
> In general, someone who breaks the security on a machine used by
> an Emacs contributor might be able to insert changes in Emacs
> by pretending to be that contributor.  I don't think this is
> fundamentally different.  But maybe the web site's security is
> not quite as good.

Yes, my concern is about the strength of the web security, as opposed to
the strength of Drew's private machine.

I suppose there's no a priori reason to assume one is more secure than
the other; maybe I'm just reacting to all the news reports about hacking.

-- 
-- Stephe

Re: ELPA policy

[Jorge A. Alfaro-Murillo]

Stephen Leake writes:
 
> Including a package in tarball ELPA means that the Emacs project 
> has decided that the package is sufficiently important to be 
> made immediately available to all Emacs users with no effort on 
> their part beyond installing Emacs from the tarball. 
> 
> That implies a certain level of maturity and quality, and 
> implies that it should be favored over other similar packages.

Would that just be packages that are currently in core? Because 
AUCTeX definitely fits that description. I have always been 
curious as why it is not in core currently.

Best,
-- 
Jorge.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I suppose there's no a priori reason to assume one is more secure than
  > the other; maybe I'm just reacting to all the news reports about hacking.

Would you please call that "cracking"?  Hacking is what we do as Emacs
developers.  See http://stallman.org/articles/on-hacking.html.

Anyway, now I understand your concern, and I think it's a valid one.
That's why I've proposed another solution.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[Michael Heerdegen]

Richard Stallman <rms@gnu.org> writes:

> GnuPG works on Windows.  I expect we can find someone to help
> you install it.  Then we need you to send us your public key
> so we can verify your signatures.  We can find a way to do that.

I think that's not much easier for Drew than to use git.

I know Drew doesn't want to use git in his workflow (which he doesn't
have to), he does not dislike git for political reasons or so AFAIR.

Once he has set up things correctly (we can help him with that), he just
has to change the file in his repo clone, pull, add and push.  That's not
more complicated than what he does now when uploading to the wiki.  He
doesn't have to use git for anything else, or while developing.

Does that sound acceptable to you, Drew?  I can help you with any
problems you might encounter.  After everything has been set up
properly, uploading to Gnu Elpa is trivial, you don't have to learn git
for that.


Regards,

Michael.

RE: ELPA policy

[Drew Adams]

> Once he has set up things correctly (we can help him with that), he
> just has to change the file in his repo clone, pull, add and push.  
> That's not more complicated than what he does now when uploading to
> the wiki.  He doesn't have to use git for anything else, or while 
> developing.
> 
> Does that sound acceptable to you, Drew?  I can help you with any
> problems you might encounter.  After everything has been set up
> properly, uploading to Gnu Elpa is trivial, you don't have to
> learn git for that.

I will take a look.  Please follow up with me off-list.

(I've tried to make clear that my reply to this thread was
not about getting my libraries into Emacs, but was about
"MELPA and updates to it from the wiki."  And as I said,
"AFAIK there is not necessarily even any interest in adding
any of the code I've written to Emacs."  This has largely
been much ado about nothing, I think.  I'm sorry for the
resulting waste of people's time.  But I'm glad that the
question of MELPA and wiki code was at least considered
to some extent.)

Re: ELPA policy

[John Wiegley]

>>>>> Jorge A Alfaro-Murillo <jorge.alfaro-murillo@yale.edu> writes:

>> Including a package in tarball ELPA means that the Emacs project has
>> decided that the package is sufficiently important to be made immediately
>> available to all Emacs users with no effort on their part beyond installing
>> Emacs from the tarball.
>> 
>> That implies a certain level of maturity and quality, and implies that it
>> should be favored over other similar packages.

> Would that just be packages that are currently in core? Because AUCTeX
> definitely fits that description. I have always been curious as why it is
> not in core currently.

No, it wouldn't be just packages currently in core, we could consider others
for inclusion as well. AucTeX might be an excellent candidate.

I like that ELPA could give us better flexibility about what goes in the
tarball, without having to make decisions that directly affect core.

John

Re: ELPA policy

[Filipp Gunbin]

Stephen,

On 14/11/2015 04:30 -0600, Stephen Leake wrote:

> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>
>> On 12/11/2015 13:52 -0600, Stephen Leake wrote:
>>
>>> I don't see why that is different from depending on a normal ELPA
>>> package.
>>
>> I think it's their dependants what make them different from tarball and
>> normal packages.
>>
>> Emacs core provides API, which others use.  A normal package declares
>> which versions of this API it is supposed to work against.
>>
>> With core packages, Emacs still provides API, but it now consists of a
>> static part (C level & core) and dynamic part (core packages, which can
>> later be updated from ELPA - correct me if I'm wrong).  So, a package
>> should independently define which core version it works agains and which
>> core package(s) version(s) it works against.
>
> package.el dependencies can only specify a minimum version, not a
> maximum. there is no way that a normal ELPA package can declare that it
> works with seq.el 1.0 but not seq.el 1.1

My words were more of a theoretical speculation rather than discussion
of the current state of package.el.

Some parts of API may be deprecated and removed, so it may be the case
that a current package is not updated instantly and need to use some
previous version of core package.  While having a "single snapshot" of
Emacs + core packages does not cause such issues (even if package does
not specify maximum version which author of course does not know in
advance, the package just doesn’t work and so the user can downgrade to
previous "snapshot", that is, previous Emacs version), separate core
packages update after installation of such a "snapshot" might move
forward and thus break some normal package (and downgrade will not help
- the user will not know what to downgrade - what package? or core
emacs?  And how can I downgrade a single (core) package?).  Just theory,
again.  Ignore if it is irrelevant and I’m complicating things too much.

> So if emacs 25 contains a core ELPA package seq.el 1.0, the declaration
>
> ;; Package-Requires: ((emacs "25.1"))
>
> is equivalent to:
>
> ;; Package-Requires: ((emacs "25.1") (seq "1.0))
>
> If seq.el 1.1 is later released via the Gnu ELPA server, it will be used
> in either case.

Will the user have the option to continue to use the tarball version
instead of newer ELPA version?  Or choosing which ELPA version to use?
This may be needed when she uses previous major Emacs release, and
current ELPA package version requires newer core APIs.

>>>> We'll probably have to calculate and offer to user which set of the
>>>> multiple core packages' multiple versions suits for multiple normal
>>>> and tarball (perhaps overriden by version from Internet package
>>>> archive) packages, at the same time probably giving the user to
>>>> opportunity to specify her preferred version of each requested
>>>> package. Is it worth the trouble? Or do I understand something wrong?
>>>
>>> Emacs does not support multiple versions of packages available
>>> simultaneously; there is only one instance of a package that is first in
>>> load-path.
>>>
>>> You can end up with one copy in the installed Emacs distribution, and
>>> one in ~/.emacs.d/elpa. But the elpa one will take precedence; that is
>>> the only one that other packages have to worry about. It either meets
>>> their dependency requirement, or not.
>>
>> Of course, I was talking about the set of available versions which could
>> be installed and from which a user or a package manager should choose.
>
> I still don't see a problem.
>
> We have an example of a core ELPA package now; ada-mode. version 4.3 is
> in emacs git; version 5.1.8 is in ELPA (if I ever get 5.x to be fast
> enough on huge files, I'll delete 4.3 from core).
>
> package.el has no issues with this.
>
> Similar things can occur when there are different versions of the same
> package in multiple repositories. In that sense, emacs git is just
> another package repository.
>
> Can you be more explicit about what problem you foresee? Give an example
> of a package that would cause a problem.

For now, I don’t know of any practical examples.  But thanks for your
time on this.

Filipp

Re: ELPA policy

[Tom Tromey]

Stephen> package.el dependencies can only specify a minimum version, not a
Stephen> maximum. there is no way that a normal ELPA package can declare that it
Stephen> works with seq.el 1.0 but not seq.el 1.1

If I had to do it all over, I'd require semver for Emacs packages.
Maybe this could be done optionally somehow.

Tom

Re: ELPA policy

[Stephen Leake]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

>> So if emacs 25 contains a core ELPA package seq.el 1.0, the declaration
>>
>> ;; Package-Requires: ((emacs "25.1"))
>>
>> is equivalent to:
>>
>> ;; Package-Requires: ((emacs "25.1") (seq "1.0))
>>
>> If seq.el 1.1 is later released via the Gnu ELPA server, it will be used
>> in either case.
>
> Will the user have the option to continue to use the tarball version
> instead of newer ELPA version?  

Yes, they may install it or not; my statement above assumed the ELPA
package was installed and updated.

> Or choosing which ELPA version to use?

Technically, there is only one ELPA version; the one shown by
`list-packages'. However, after user installs one version, they may opt
to not install an update.

If they delete the installed version, they can only install the latest
version.

That's all just using the commands in package.el; users can of course
save code and copy it into and out of `load-path' manually.

> This may be needed when she uses previous major Emacs release, and
> current ELPA package version requires newer core APIs.

If there is a compatible ELPA package version that is more recent than
the version bundled with that Emacs version, but older than the latest
ELPA version, yes.

But that's not supported by the current ELPA server and package.el.

So at the moment, users of older Emacsen are stuck with either the
bundled package, or the latest package.

The maintainer of the ELPA package can provide compatibility code so it
is useful with older Emacsen. But that gets harder the farther back you
go; my current limit is 24.3.

-- 
-- Stephe

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I don't think this is "much ado about nothing".
The reasons that affect one person may affect others.

We want people to arrange to contribute their Emacs add-ons
to Emacs, and that suggests that whenever someone says a reason
why he didn't want to put his code in GNU ELPA, we should try
to correct it.



-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: ELPA policy

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> We want people to arrange to contribute their Emacs add-ons to Emacs, and
> that suggests that whenever someone says a reason why he didn't want to put
> his code in GNU ELPA, we should try to correct it.

Agreed. ELPA should become a more positive experience, both for developers,
contributors and users.

John

Re: ELPA policy

[Artur Malabarba]

2015-11-17 23:17 GMT+00:00 John Wiegley <jwiegley@gmail.com>:
>>>>>> Richard Stallman <rms@gnu.org> writes:
>
>> We want people to arrange to contribute their Emacs add-ons to Emacs, and
>> that suggests that whenever someone says a reason why he didn't want to put
>> his code in GNU ELPA, we should try to correct it.
>
> Agreed. ELPA should become a more positive experience, both for developers,
> contributors and users.

I'm deeply in favor of making it easier for people to contribute code
to Elpa, and (like I said), I'm willing to help write the code for
that.
But I think the hardest part of the problem is not being discussed here.

Whatever this method is, it needs to meet two criteria simultaneously:
1. Not commit any code to Elpa until someone with push access has OK'd
it. This is just to ensure nothing malicious is being done.
2. Be fairly automated (not completely), so that the Elpa maintainers
don't have to manually commit+push all code that gets sent to them.
This is to preserve the sanity of the maintainers. The current model
where everything is commited by us just isn't scalable.

While those two points are not mutually exclusive, they are quite
difficult to reconcile. It needs to show the maintainers a diff of the
changes and then only proceed with the commit after receiving some
very minimal interaction (a reply to an email, or the click of a
button).

Re: ELPA policy

[David Kastrup]

Artur Malabarba <bruce.connor.am@gmail.com> writes:

> 2015-11-17 23:17 GMT+00:00 John Wiegley <jwiegley@gmail.com>:
>>>>>>> Richard Stallman <rms@gnu.org> writes:
>>
>>> We want people to arrange to contribute their Emacs add-ons to Emacs, and
>>> that suggests that whenever someone says a reason why he didn't want to put
>>> his code in GNU ELPA, we should try to correct it.
>>
>> Agreed. ELPA should become a more positive experience, both for developers,
>> contributors and users.
>
> I'm deeply in favor of making it easier for people to contribute code
> to Elpa, and (like I said), I'm willing to help write the code for
> that.
> But I think the hardest part of the problem is not being discussed here.
>
> Whatever this method is, it needs to meet two criteria simultaneously:
> 1. Not commit any code to Elpa until someone with push access has OK'd
> it. This is just to ensure nothing malicious is being done.
> 2. Be fairly automated (not completely), so that the Elpa maintainers
> don't have to manually commit+push all code that gets sent to them.
> This is to preserve the sanity of the maintainers. The current model
> where everything is commited by us just isn't scalable.
>
> While those two points are not mutually exclusive, they are quite
> difficult to reconcile. It needs to show the maintainers a diff of the
> changes and then only proceed with the commit after receiving some
> very minimal interaction (a reply to an email, or the click of a
> button).

With the GNU ELPA requiring copyright assignments, I doubt that we are
going to hit scalability issues of the kind you fear.  But of course
there is nothing wrong with trying to create a contributing process that
is minimal effort for the sake of such archives where this is not the
case.

-- 
David Kastrup

Re: ELPA policy

[Stefan Monnier]

> FWIW, I took a cursory look at dash.el in ELPA, and was surprised to
> see that its doc strings are not according to our coding conventions,
> and functions/macros that clearly aren't internal don't follow the
> naming conventions.

Then I suggest you file a bug report with the author.

> So maybe it's time we defined the minimum requirements for packages to
> be included in ELPA?

What good would it do?


        Stefan

Re: ELPA policy

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Fri, 08 May 2020 10:57:58 -0400
> 
> > FWIW, I took a cursory look at dash.el in ELPA, and was surprised to
> > see that its doc strings are not according to our coding conventions,
> > and functions/macros that clearly aren't internal don't follow the
> > naming conventions.
> 
> Then I suggest you file a bug report with the author.

Aren't ELPA bugs reported via our bug tracker?

> > So maybe it's time we defined the minimum requirements for packages to
> > be included in ELPA?
> 
> What good would it do?

<Shrug> It will get us on the same page when we are reviewing packages
and patches for ELPA.

Re: ELPA policy

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: philippe.vaucher@gmail.com,  stefan@marxist.se,  emacs-devel@gnu.org,
>>   joaotavora@gmail.com,  pcr910303@icloud.com,  dgutov@yandex.ru,
>>   eliz@gnu.org,  drew.adams@oracle.com
>> Date: Thu, 07 May 2020 23:38:23 -0400
>> 
>> I made the first efforts to get it integrated into GNU ELPA (and Phil
>> did the heavy lifting), so no I definitely don't think it's problematic.
>
> FWIW, I took a cursory look at dash.el in ELPA, and was surprised to
> see that its doc strings are not according to our coding conventions,
> and functions/macros that clearly aren't internal don't follow the
> naming conventions.
>
> So maybe it's time we defined the minimum requirements for packages to
> be included in ELPA?


dash.el has an extensive README with a very large number of
examples. The README is produced by combining the examples from a lisp
file which also generates a test file. I think Magnar values doc tests
more than he values Emacs doc string conventions.

Phil

Re: ELPA policy

[Stefan Monnier]

>> > FWIW, I took a cursory look at dash.el in ELPA, and was surprised to
>> > see that its doc strings are not according to our coding conventions,
>> > and functions/macros that clearly aren't internal don't follow the
>> > naming conventions.
>> Then I suggest you file a bug report with the author.
> Aren't ELPA bugs reported via our bug tracker?

We tell users that they can submit bug reports via our bug tracker.
And maintainers are free to use other bug trackers.
Ideally they should keep an eye on our tracker, or else someone needs to
forward our bugs to their tracker when such things happen.

IOW, as a user of dash.el you can submit your bug report via `M-x
report-emacs-bug` (and if needed, hopefully someone will forward it to
where the maintainer will see it).

>> > So maybe it's time we defined the minimum requirements for packages to
>> > be included in ELPA?
>> What good would it do?
> <Shrug> It will get us on the same page when we are reviewing packages
> and patches for ELPA.

That's not a very high concern for me, compared to the concern of keeping
GNU ELPA relevant and vaguely up-to-date.


        Stefan

Re: ELPA policy

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: emacs-devel@gnu.org
> Date: Fri, 08 May 2020 19:16:03 -0400
> 
> >> > So maybe it's time we defined the minimum requirements for packages to
> >> > be included in ELPA?
> >> What good would it do?
> > <Shrug> It will get us on the same page when we are reviewing packages
> > and patches for ELPA.
> 
> That's not a very high concern for me, compared to the concern of keeping
> GNU ELPA relevant and vaguely up-to-date.

Excuse me, but what do you mean by "for me"?  Isn't GNU ELPA an
important part of the Emacs project?  I'd expect something like "for
us" or "for Emacs", in which case not just you personally have a say.

And why do I have to submit bug reports against an ELPA package for
violation of our coding conventions?  I'd expect the maintainer of the
package to be asked to fix those as a precondition for accepting the
package in GNU ELPA, or at least as a long-term plan to which the
maintainer agreed in advance (in which case no bug report would have
been necessary).

What am I missing here?

Re: ELPA policy

[David Engster]

> And why do I have to submit bug reports against an ELPA package for
> violation of our coding conventions?  I'd expect the maintainer of the
> package to be asked to fix those as a precondition for accepting the
> package in GNU ELPA, or at least as a long-term plan to which the
> maintainer agreed in advance (in which case no bug report would have
> been necessary).
>
> What am I missing here?

The README for GNU ELPA states:

We do not impose a particular coding style on GNU ELPA packages, but of
course we recommend the coding style used in Emacs's own source code.
Furthermore we recommend the following:
- Use `cl-lib` rather than `cl` if it all possible.
- Use lexical-binding if it all possible.
- Try and fix the warnings emitted when compiling the package with a
  recent Emacs.

From: http://git.savannah.gnu.org/cgit/emacs/elpa.git/plain/README

So from my understanding: Following Emacs coding guidelines is a
recommendation, but not a precondition for getting packages into
GNU ELPA.

If we start bundling certain ELPA packages with Emacs proper, then of
course these special "core packages" would need to adhere to the Emacs
coding style. I don't see any difficulty in making this distinction
between core packages and the rest. And I also don't see any problem to
put s.el in ELPA and say: note that using this package is against the
Emacs coding style, so as long as you depend on this packages, it cannot
become a "core package" in the future (same for dash.el).

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 09:35:49 +0200
> 
> From: http://git.savannah.gnu.org/cgit/emacs/elpa.git/plain/README
> 
> So from my understanding: Following Emacs coding guidelines is a
> recommendation, but not a precondition for getting packages into
> GNU ELPA.
> 
> If we start bundling certain ELPA packages with Emacs proper, then of
> course these special "core packages" would need to adhere to the Emacs
> coding style. I don't see any difficulty in making this distinction
> between core packages and the rest. And I also don't see any problem to
> put s.el in ELPA and say: note that using this package is against the
> Emacs coding style, so as long as you depend on this packages, it cannot
> become a "core package" in the future (same for dash.el).

If we mark some ELPA packages up front as unsuitable for inclusion in
core, then I guess this could be acceptable.  It does mean that such a
decision would make it hard to change our minds later, though.  But at
least we should have an indication in each package whether it is or
isn't exempt from our conventions, something we don't have now.

Moreover, our conventions not being an absolute requirement, I think
_some_ requirements should still be non-negotiable, because we cannot
just accept anything.  Otherwise any request for a change could be met
with an argument like "but I'm not under any obligation to comply" or
somesuch.  I don't see any such requirements described in README, so
my question still stands, I think.

Re: ELPA policy

[David Engster]

>> From: David Engster <deng@randomsample.de>
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
>> Date: Sat, 09 May 2020 09:35:49 +0200
>
>> 
>> From: http://git.savannah.gnu.org/cgit/emacs/elpa.git/plain/README
>> 
>> So from my understanding: Following Emacs coding guidelines is a
>> recommendation, but not a precondition for getting packages into
>> GNU ELPA.
>> 
>> If we start bundling certain ELPA packages with Emacs proper, then of
>> course these special "core packages" would need to adhere to the Emacs
>> coding style. I don't see any difficulty in making this distinction
>> between core packages and the rest. And I also don't see any problem to
>> put s.el in ELPA and say: note that using this package is against the
>> Emacs coding style, so as long as you depend on this packages, it cannot
>> become a "core package" in the future (same for dash.el).
>
> If we mark some ELPA packages up front as unsuitable for inclusion in
> core, then I guess this could be acceptable.  It does mean that such a
> decision would make it hard to change our minds later, though.  But at
> least we should have an indication in each package whether it is or
> isn't exempt from our conventions, something we don't have now.

If you look here

http://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/externals-list

you'll see that there are already packages marked as ":core" which all
currently refer to a path in Emacs proper. This way, they could be
overriden through GNU ELPA, and they of course need to adhere to the
Emacs coding style.

You are right that it will be difficult if we decide a package should
become "core" and currently does not adhere to the Emacs coding style,
but frankly, we should be more worried that this ship has already sailed
far away. Many packages which are absolutely essential for a modern
programmer's editor are already only available through MELPA. If we want
to make GNU ELPA more relevant, we need to lower the hurdle for
inclusion. I also fully agree with Stefan that we should make it
possible for packages that have non-FSF copyright to be included. Of
course these packages could never become "core", but having them
installable throught GNU ELPA would be the next best thing.

> Moreover, our conventions not being an absolute requirement, I think
> _some_ requirements should still be non-negotiable, because we cannot
> just accept anything.  Otherwise any request for a change could be met
> with an argument like "but I'm not under any obligation to comply" or
> somesuch.  I don't see any such requirements described in README, so
> my question still stands, I think.

Yes. For instance, it is clear that a package in GNU ELPA must not
recommend or even depend on non-free software.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 10:16:38 +0200
> 
> You are right that it will be difficult if we decide a package should
> become "core" and currently does not adhere to the Emacs coding style,
> but frankly, we should be more worried that this ship has already sailed
> far away. Many packages which are absolutely essential for a modern
> programmer's editor are already only available through MELPA.

Why is that a problem?  We can never do anything to prevent people
from concocting whatever packages they want and making them available
for others.  Nor do I think we should: this is, after all, Free
Software: people should be free to choose whatever software they like
that does the job for them.  We can never control that, and we
shouldn't try.

> I also fully agree with Stefan that we should make it possible for
> packages that have non-FSF copyright to be included. Of course these
> packages could never become "core", but having them installable
> throught GNU ELPA would be the next best thing.

If we are going to drop requirements, then what will distinguish ELPA
from MELPA?  And what's the problem with having non-core packages
available through MELPA, anyway? why do we need to have them in ELPA?

This all goes back to my confusion about the relation between ELPA and
Emacs, something I thought I understood, but now I conclude that I
don't.

Re: ELPA policy

[David Engster]

>> From: David Engster <deng@randomsample.de>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Sat, 09 May 2020 10:16:38 +0200
>> 
>> You are right that it will be difficult if we decide a package should
>> become "core" and currently does not adhere to the Emacs coding style,
>> but frankly, we should be more worried that this ship has already sailed
>> far away. Many packages which are absolutely essential for a modern
>> programmer's editor are already only available through MELPA.
>
> Why is that a problem?  We can never do anything to prevent people
> from concocting whatever packages they want and making them available
> for others.  Nor do I think we should: this is, after all, Free
> Software: people should be free to choose whatever software they like
> that does the job for them.  We can never control that, and we
> shouldn't try.
>
>> I also fully agree with Stefan that we should make it possible for
>> packages that have non-FSF copyright to be included. Of course these
>> packages could never become "core", but having them installable
>> throught GNU ELPA would be the next best thing.
>
> If we are going to drop requirements, then what will distinguish ELPA
> from MELPA?  And what's the problem with having non-core packages
> available through MELPA, anyway? why do we need to have them in ELPA?

In principal, I agree with you. The problem is mainly Richard's stance
on this issue, which says that we must not recommend packages which are
not in Emacs or GNU ELPA, but that we should rather re-implement them. I
think that's a terrible waste.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 10:43:55 +0200
> 
> > If we are going to drop requirements, then what will distinguish ELPA
> > from MELPA?  And what's the problem with having non-core packages
> > available through MELPA, anyway? why do we need to have them in ELPA?
> 
> In principal, I agree with you. The problem is mainly Richard's stance
> on this issue, which says that we must not recommend packages which are
> not in Emacs or GNU ELPA, but that we should rather re-implement them. I
> think that's a terrible waste.

Is this only about "recommending" or not "recommending" a package?  Is
this why we created GNU ELPA and invest non-trivial amount of effort
in maintaining and developing it?  I very much hope there's more to
it than just that.

I could understand if you'd say "use" instead of "recommend",
i.e. have code in Emacs, which, if a package is installed, would use
it.  That'd actually have the package's name in our sources, and would
constitute some kind of "endorsement".  But as long as we don't use
any of those packages, why should we care what other people like or
don't like?

Re: ELPA policy

[David Engster]

>> From: David Engster <deng@randomsample.de>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Sat, 09 May 2020 10:43:55 +0200
>
>> 
>> > If we are going to drop requirements, then what will distinguish ELPA
>> > from MELPA?  And what's the problem with having non-core packages
>> > available through MELPA, anyway? why do we need to have them in ELPA?
>> 
>> In principal, I agree with you. The problem is mainly Richard's stance
>> on this issue, which says that we must not recommend packages which are
>> not in Emacs or GNU ELPA, but that we should rather re-implement them. I
>> think that's a terrible waste.
>
> Is this only about "recommending" or not "recommending" a package?  Is
> this why we created GNU ELPA and invest non-trivial amount of effort
> in maintaining and developing it?  I very much hope there's more to
> it than just that.

We created GNU ELPA because we wanted a package system in Emacs. It was
simply the first, then things evolved to the state we have now, where
MELPA is the dominant package repository.

> I could understand if you'd say "use" instead of "recommend",
> i.e. have code in Emacs, which, if a package is installed, would use
> it.  That'd actually have the package's name in our sources, and would
> constitute some kind of "endorsement".  But as long as we don't use
> any of those packages, why should we care what other people like or
> don't like?

Sorry, but you lost me there. All I'm saying is that there's a whole lot
of terrific packages out there but which we must not recommend to users,
although they are free software and often vastly superior to the things
that are built into Emacs. For instance, there's a discussion going on
about making a video showing Emacs' capabilities, but I assume we'd not
be allowed to show Magit. That's a huge loss.

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 12:13:01 +0200
> 
> > I could understand if you'd say "use" instead of "recommend",
> > i.e. have code in Emacs, which, if a package is installed, would use
> > it.  That'd actually have the package's name in our sources, and would
> > constitute some kind of "endorsement".  But as long as we don't use
> > any of those packages, why should we care what other people like or
> > don't like?
> 
> Sorry, but you lost me there. All I'm saying is that there's a whole lot
> of terrific packages out there but which we must not recommend to users,
> although they are free software and often vastly superior to the things
> that are built into Emacs. For instance, there's a discussion going on
> about making a video showing Emacs' capabilities, but I assume we'd not
> be allowed to show Magit. That's a huge loss.

So package maintainers are supposed to want to be on ELPA so that they
could appear in a video, or in someone's message on a GNU mailing
list?  Really?

Once again, I wasn't asking whether it was okay or not to show off
Magit in a video or promote it on this list, I was asking why do we
need ELPA when MELPA is out there and has many more packages (and
always will)?

Re: ELPA policy

[David Engster]

>> From: David Engster <deng@randomsample.de>
>> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Sat, 09 May 2020 12:13:01 +0200
>
>> 
>> > I could understand if you'd say "use" instead of "recommend",
>> > i.e. have code in Emacs, which, if a package is installed, would use
>> > it.  That'd actually have the package's name in our sources, and would
>> > constitute some kind of "endorsement".  But as long as we don't use
>> > any of those packages, why should we care what other people like or
>> > don't like?
>> 
>> Sorry, but you lost me there. All I'm saying is that there's a whole lot
>> of terrific packages out there but which we must not recommend to users,
>> although they are free software and often vastly superior to the things
>> that are built into Emacs. For instance, there's a discussion going on
>> about making a video showing Emacs' capabilities, but I assume we'd not
>> be allowed to show Magit. That's a huge loss.
>
> So package maintainers are supposed to want to be on ELPA so that they
> could appear in a video, or in someone's message on a GNU mailing
> list?  Really?

No, package maintainers usually don't care. I think this should be in
*our* interest.

> Once again, I wasn't asking whether it was okay or not to show off
> Magit in a video or promote it on this list, I was asking why do we
> need ELPA when MELPA is out there and has many more packages (and
> always will)?

Indeed, one possibility would be to simply close GNU ELPA for everything
but core or maybe-in-the-future-core packages. Is that what you're
proposing?

-David

Re: ELPA policy

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Sat, 09 May 2020 12:29:22 +0200
> 
> > So package maintainers are supposed to want to be on ELPA so that they
> > could appear in a video, or in someone's message on a GNU mailing
> > list?  Really?
> 
> No, package maintainers usually don't care. I think this should be in
> *our* interest.

If it's in our interest, then why do we accept packages in exactly the
same form as they are on MELPA?  What do we gain from this?

> > Once again, I wasn't asking whether it was okay or not to show off
> > Magit in a video or promote it on this list, I was asking why do we
> > need ELPA when MELPA is out there and has many more packages (and
> > always will)?
> 
> Indeed, one possibility would be to simply close GNU ELPA for everything
> but core or maybe-in-the-future-core packages. Is that what you're
> proposing?

I don't have any proposals yet, because I still don't have a clear
idea of what is (or should be) the relation between Emacs and ELPA,
nor even what are the goals of adding packages to ELPA which clearly
couldn't be added to core even if we wanted.  And with Stefan's
proposal to drop the copyright assignment requirement, bringing them
into core will even harder.  But OTOH Stefan says that requesting the
assignment is harmful to the Emacs project.  So I'm utterly confused
regarding our goals in this regard.

Re: ELPA policy

[Alfred M. Szmidt]

I think one of the reasons is simply that the GNU project doesn't want
to be in a situation where a third party (i.e., MELPA) is recommending
non-free software, or where the license is unclear, etc.

So ELPA was setup so that this cannot happen, since that is part
(AFAIK) of Emacs.

Re: ELPA policy

[David Engster]

>> Indeed, one possibility would be to simply close GNU ELPA for everything
>> but core or maybe-in-the-future-core packages. Is that what you're
>> proposing?
>
> I don't have any proposals yet, because I still don't have a clear
> idea of what is (or should be) the relation between Emacs and ELPA,
> nor even what are the goals of adding packages to ELPA which clearly
> couldn't be added to core even if we wanted.  And with Stefan's
> proposal to drop the copyright assignment requirement, bringing them
> into core will even harder.  But OTOH Stefan says that requesting the
> assignment is harmful to the Emacs project.  So I'm utterly confused
> regarding our goals in this regard.

Even if we drop the FSF copyright requirement, we still would have
requirements that MELPA does not have, like not supporting/promoting
non-free software, GPLv3-or-later license (MELPA only requires
GPL-compatible), and probably (and unfortunately, IMHO) also the
requirement to not depend on free software that is seen critical by the
FSF, like LLVM/clang.

My hope is that if we could drop the FSF copyright requirement for
non-core packages, we at least could try to get popular and useful
packages into GNU ELPA, we could make them prominently visible and easy
to install without the user having to put anything into her dotemacs,
and we could show the power of Emacs that is visible in these packages
on our main web site, in videos, etc.

-David

Re: ELPA policy

[Dmitry Gutov]

On 09.05.2020 09:22, Eli Zaretskii wrote:
> And why do I have to submit bug reports against an ELPA package for
> violation of our coding conventions?

Because it's the only way how things can work well. Once we have a bug 
tracker that most maintainers like to use, that could change.

> I'd expect the maintainer of the
> package to be asked to fix those as a precondition for accepting the
> package in GNU ELPA, or at least as a long-term plan to which the
> maintainer agreed in advance (in which case no bug report would have
> been necessary).
> 
> What am I missing here?

You're missing that the maintainers don't have any contractual 
obligations. The more conditions we add, and the less effort we want to 
expend ourselves [forwarding bug reports], the higher the odds are that 
we end up with fewer contributions/stale code/untended bug reports.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Indeed, one possibility would be to simply close GNU ELPA for everything
  > but core or maybe-in-the-future-core packages. Is that what you're
  > proposing?

I think that is unnecessarily strict.  It presumes that when adding a
package to GNU ELPA we always know, already, whether we will ever want
to put it in the core.  I expect there are lots of cases  where
that decision will depend on circumstances.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: ELPA policy

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 9 May 2020 18:06:05 +0300
> 
> On 09.05.2020 09:22, Eli Zaretskii wrote:
> > And why do I have to submit bug reports against an ELPA package for
> > violation of our coding conventions?
> 
> Because it's the only way how things can work well. Once we have a bug 
> tracker that most maintainers like to use, that could change.

There's no chance in the world someone will be serious about such bug
reports, given that ELPA's README basically tells that these
requirements are "recommendations".  Why would someone invest any
effort if they can avoid doing that?

> You're missing that the maintainers don't have any contractual 
> obligations. The more conditions we add, and the less effort we want to 
> expend ourselves [forwarding bug reports], the higher the odds are that 
> we end up with fewer contributions/stale code/untended bug reports.

And what's the problem with that?  These packages are available from
elsewhere anyway.

Our coding standards aren't arbitrary.  Either we think they are solid
and should apply to every code that could end up in Emacs, or we
should stop requiring adherence to them.  Anything else makes no
sense.

Re: ELPA policy

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Our coding standards aren't arbitrary.  Either we think they are solid
  > and should apply to every code that could end up in Emacs, or we
  > should stop requiring adherence to them.  Anything else makes no
  > sense.

In principle, they should apply to ELPA packages.  But it is ok, I think,
if we don't rush to bring all of them completely up to snuff.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: ELPA policy

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: dgutov@yandex.ru, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Date: Mon, 11 May 2020 23:16:20 -0400
> 
>   > Our coding standards aren't arbitrary.  Either we think they are solid
>   > and should apply to every code that could end up in Emacs, or we
>   > should stop requiring adherence to them.  Anything else makes no
>   > sense.
> 
> In principle, they should apply to ELPA packages.  But it is ok, I think,
> if we don't rush to bring all of them completely up to snuff.

I could agree to that, but I wouldn't agree to drop all of the
requirements.  And even if we loosen some of them, it's already
problematic: the contributors of code for the core, which are
generally required to adhere to all of the requirements, could
rightfully feel they are treated unfairly, even though developing the
core benefits Emacs more than an unbundled package.