ELPA policy

ELPA policy

[Julien Danjou]

Hi,

This topic has been recently brought by Lars, Ted and me on the gnus
mailing list. I've been asked to move this discussion here for
clarification.

I am currently worried by the Emacs ELPA archive and what will be that
policy. As Lars pointed out, until now, packages have been added over
the years into Emacs trunk, maintained by a extended set of skilled
hands, assuring good quality in (almost :-)) every place of every
packages furnished with Emacs.

Now, with the introduction of ELPA I'm worried about a lot of things.

- Who will be able to upload to ELPA?
- Who will fix the packages' bugs?
- Who will assure there's no regression for Emacs 24.1 user when people
  will starts uploading packages using 24.2 only function?
- Who will assure there's no regression at all?
- Who will assure there's no really bad things uploaded?
- Will all new packages go to ELPA, or with some still go to Emacs
  trunk? And if some can go to the trunk, upon which rules?
- Will all/most of the current packages be moved to ELPA?

There's probably a lot more question outstanding. I've the feeling ELPA
is adding a second class citizenship for packages, and I really do not
like that, at least in its current form.

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

Re: ELPA policy

[Chong Yidong]

Julien Danjou <julien@danjou.info> writes:

> I am currently worried by the Emacs ELPA archive and what will be that
> policy. As Lars pointed out, until now, packages have been added over
> the years into Emacs trunk, maintained by a extended set of skilled
> hands, assuring good quality in (almost :-)) every place of every
> packages furnished with Emacs.

Emacs contains many packages that are maintained "externally".  While
Emacs developers might make small changes to the in-tree version, most
development is done upstream and periodically merged in.  Those upstream
maintainers handle bugfixes and ensure compatibility among Emacs
versions.

One good reason to put a package in elpa.gnu.org, rather than in the
Emacs tarball, is if it is likely to benefit only a small segment of
Emacs users (even if it's of tremendous usefulness to that segment).
Especially, but not necessarily, if a package is large and complex, like
Auctex and Muse.

There are other good reasons too, e.g. packages that we want to merge
into Emacs core in the future, but not yet (for whatever reason).

(Incidentally, there are also some packages in Emacs that might be
usefully moved out into elpa.gnu.org, e.g. since they are so rarely
used.  We could also move some of the files in obsolete/ into a
subrepository, e.g. elpa.gnu.org/packages/obsolete)

(Also, as stated before, the FSF's policy is that for a package to be
listed on elpa.gnu.org its copyright must be assigned, in the exact same
way as if it's included in Emacs core.)

> - Who will be able to upload to ELPA?
> - Who will assure there's no really bad things uploaded?

The way it's currently set up is that only a couple of people can upload
to ELPA; package maintainers, when they release a new version, should
email me (or Ted) to get the package uploaded.  The system is still a
work in progress; we might set up a more elaborate "staging area" system
in the future.  (Such a system would still involve a human component, of
course, to reduce the possibility of malicious uploads.)  I will add a
page to the elpa.gnu.org webpage explaining this.

Re: ELPA policy

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> One good reason to put a package in elpa.gnu.org, rather than in the
> Emacs tarball, is if it is likely to benefit only a small segment of
> Emacs users (even if it's of tremendous usefulness to that segment).
> Especially, but not necessarily, if a package is large and complex, like
> Auctex and Muse.
>
> There are other good reasons too, e.g. packages that we want to merge
> into Emacs core in the future, but not yet (for whatever reason).

The reason that I raised the question now was that I was trying to
figure out how to do reasonable colour distance calculations for
rendering foreground colours in shr.el.  rainbow-mode (for colourising
colour specs in, for instance, CSS files) was brought up, that had
similar needs, and its calculations in the same area might possibly be
reusable by shr.el.

But apparently rainbow-mode went to ELPA instead of into Emacs, even
though it's small, it's clearly written, and it seems to be generally
useful for anybody editing stuff like CSS or .Xdefaults files or
anything.  So I wondered why that was...

> The way it's currently set up is that only a couple of people can upload
> to ELPA; package maintainers, when they release a new version, should
> email me (or Ted) to get the package uploaded.

If ELPA is supposed to be a repository of really special-needs code, I
think it's a good idea.  It would be nice to have an automatic way to
download, say, one of the Emacs-based mp3 players.  But if it's going to
carry stuff that people do really want to have in the Emacs
distribution, then I think it's counter-productive.

Stuff that is in Emacs gets loving care.  When url.el needs tweaking
(for instance), people step up to the plate and makes sure that it
works.  Things that live in a package repository bit-rot at an alarming
rate.  One of the great things about Emacs is that when you "apt-get
install emacs", you get a fully-featured system where you can be
reasonably sure that things work.  Over-modularisation can be costly in
the long run.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: ELPA policy

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> figure out how to do reasonable colour distance calculations for
> rendering foreground colours in shr.el.  rainbow-mode (for colourising
> colour specs in, for instance, CSS files) was brought up, that had
> similar needs, and its calculations in the same area might possibly be
> reusable by shr.el.
>
> But apparently rainbow-mode went to ELPA instead of into Emacs, even
> though it's small, it's clearly written, and it seems to be generally
> useful for anybody editing stuff like CSS or .Xdefaults files or
> anything.  So I wondered why that was...

Rainbow-mode, as presented, is a tool for colorizing strings in Emacs
buffers, the kind of neat hack that doesn't really need to be in Emacs.
From your description, it sounds like parts of it can be usefully
refactored into a general color-manipulation library, in which case we
could include that part in Emacs (preferably, using a better prefix than
`rainbow').

> If ELPA is supposed to be a repository of really special-needs code, I
> think it's a good idea.  It would be nice to have an automatic way to
> download, say, one of the Emacs-based mp3 players.  But if it's going
> to carry stuff that people do really want to have in the Emacs
> distribution, then I think it's counter-productive.  Stuff that is in
> Emacs gets loving care.  Things that live in a package repository
> bit-rot at an alarming rate.

Well, obviously it's going to be a judgement call, analogous to the
debates about what goes into distribution DVDs (with programs like Emacs
nowadays frequently not making the cut :-P).  The level of bit-rot is
dependent on the upstream package maintainers; I don't think you can
argue that packages like Auctex (or SLIME or ESS, which are not
packaged) have bit-rotted.

Some packages integrated into Emacs are no longer active upstream but
are still tinkered with by Emacs developers, but what's really happening
here is that we have effectively forked the project.  If a package on
elpa.gnu.org ends up being abandoned, there's little difference in how
we could solve the problem, except the fork would be more explicit than
implicit.

rainbow-mode (was: ELPA policy)

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Rainbow-mode, as presented, is a tool for colorizing strings in Emacs
> buffers, the kind of neat hack that doesn't really need to be in Emacs.

So it was rejected because you didn't think it was generally useful
enough, basically?

That's a judgement call, of course, but I think it sounds quite useful.
If you're fiddling with colours in CSS files, for instance, it would
speed up the editing/reload process -- at least the way I do it, which
is "er, this should be a bluish colour, so I'll try #404090", *save*,
*reload*, "no, darker", etc.

Just seeing what the colour would be at once would be really neat.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

RE: rainbow-mode (was: ELPA policy)

[Drew Adams]

> So it was rejected because you didn't think it was generally useful
> enough, basically? That's a judgement call, of course, but I think
> it sounds quite useful.
>
> If you're fiddling with colours in CSS files, for instance, it would
> speed up the editing/reload process -- at least the way I do it, which
> is "er, this should be a bluish colour, so I'll try #404090", *save*,
> *reload*, "no, darker", etc.
> 
> Just seeing what the colour would be at once would be really neat.

http://www.emacswiki.org/emacs/hexrgb.el

http://www.emacswiki.org/emacs/palette.el

Wrt "no, darker", etc., there are several WYSIWYG ways to incrementally adjust
colors.

http://www.emacswiki.org/emacs/SetColor

http://www.emacswiki.org/emacs/ColorPalette

http://www.emacswiki.org/emacs/DoReMi

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

"Drew Adams" <drew.adams@oracle.com> writes:

> http://www.emacswiki.org/emacs/hexrgb.el
>
> http://www.emacswiki.org/emacs/palette.el
>
> Wrt "no, darker", etc., there are several WYSIWYG ways to incrementally adjust
> colors.
>
> http://www.emacswiki.org/emacs/SetColor
>
> http://www.emacswiki.org/emacs/ColorPalette
>
> http://www.emacswiki.org/emacs/DoReMi

I'm not quite sure what you mean to say here.  Is it "this is something
that so many people want in Emacs that it's been partially reimplemented
half a dozen times"?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: rainbow-mode

[Stephen J. Turnbull]

Lars Magne Ingebrigtsen writes:

 > I'm not quite sure what you mean to say here.  Is it "this is something
 > that so many people want in Emacs that it's been partially reimplemented
 > half a dozen times"?

IIRC, the most recent discussion concluded that none of the half-
implemented modes is good enough, especially given the nice features
in the other half-implemented modes, and nobody stepped up to the
plate to merge them, document the result, and make sure the result
conforms to coding standards etc.

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> IIRC, the most recent discussion concluded that none of the half-
> implemented modes is good enough, especially given the nice features
> in the other half-implemented modes, and nobody stepped up to the
> plate to merge them, document the result, and make sure the result
> conforms to coding standards etc.

Again, I think that illustrates my point.

I think experience shows that packages that are in Emacs do get hacked
on to add features that people feel are missing.  Small packages that
live outside of Emacs don't get that treatment, as a norm.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: rainbow-mode

[Ted Zlatanov]

On Tue, 16 Nov 2010 14:25:08 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> "Stephen J. Turnbull" <stephen@xemacs.org> writes:
>> IIRC, the most recent discussion concluded that none of the half-
>> implemented modes is good enough, especially given the nice features
>> in the other half-implemented modes, and nobody stepped up to the
>> plate to merge them, document the result, and make sure the result
>> conforms to coding standards etc.

LMI> Again, I think that illustrates my point.

LMI> I think experience shows that packages that are in Emacs do get hacked
LMI> on to add features that people feel are missing.  Small packages that
LMI> live outside of Emacs don't get that treatment, as a norm.

So let's check everything on the Emacs Wiki into Emacs.  By your theory
that will get the Emacs developers to work on all of it.

I'm trying to say that:

- there's plenty of bit rot in Emacs itself

- this is a manpower problem, not a segregation problem

- new packages don't add complexity linearly

Ted

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> So let's check everything on the Emacs Wiki into Emacs.  By your theory
> that will get the Emacs developers to work on all of it.

No, that's not my theory.

Look, in each separate case, it's obviously just a matter of taste.  Is
a mode for highlighting Clojure code something that Emacs should have
centrally?  In ELPA?  Totally separately?  Is a PVR interface?  Is
gnutls?

Each of these, of course, need individual consideration.

However, in general I don't think offloading things that would
traditionally be in Emacs (say, a Ruby mode) to ELPA will help.  I think
it's quite clear that things that are in ELPA will bit rot faster than
code that's in Emacs.  As Glenn pointed out, he's cleaning up and
working on the Emacs distribution, and that's my inclination, too.  If
we want to say "well, Emacs has a supported Ruby mode", then having it
in ELPA will only hinder that, not help.

To get back to the example at hand, rainbow-mode, it's something that I
would use, and the repeated reimplementation of the same functionality
shows that it's something that other people would use, too.  That's my
argument for including it in Emacs.  What are the counter-arguments?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: rainbow-mode

[Ted Zlatanov]

On Tue, 16 Nov 2010 15:22:09 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> To get back to the example at hand, rainbow-mode, it's something that I
LMI> would use, and the repeated reimplementation of the same functionality
LMI> shows that it's something that other people would use, too.  That's my
LMI> argument for including it in Emacs.  What are the counter-arguments?

Discussion is archived here:

http://thread.gmane.org/gmane.emacs.devel/127873

You saw some of the arguments:

> "Stephen J. Turnbull" <stephen@xemacs.org> writes:

> > IIRC, the most recent discussion concluded that none of the half-
> > implemented modes is good enough, especially given the nice features
> > in the other half-implemented modes, and nobody stepped up to the
> > plate to merge them, document the result, and make sure the result
> > conforms to coding standards etc.

Plus Chong Yidong explained in this thread and in
http://thread.gmane.org/gmane.emacs.devel/127873/focus=128245

Basically none of the authors wanted to merge their mode with the
others.  Julien added enough to rainbow-mode.el to make it the best
choice but it's still not superior to css-color-mode.el in every way.

FWIW my vote is still for rainbow-mode.el as follows: I think it's most
useful in its current form out of all the contenders.  I would put it in
the core Emacs because editing CSS and HTML (plus other languages with
inlined colors) is so common.  The color transformation functions it has
are generally useful.  I would also give it first-class customization
variables so it's really easy to turn it on.  Finally, it's pretty
unlikely to undergo big changes at this point.

Maybe it could get renamed, I don't care too much.  The current name is
quite memorable though.

Ted

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Plus Chong Yidong explained in this thread and in
> http://thread.gmane.org/gmane.emacs.devel/127873/focus=128245

I've read the thread.  There was discussion, Julien added stuff to
rainbow-mode as suggested, and then Chong said "I've added rainbow-mode
to the Emacs 24 package repository.", and that ended the discussion.
Is that an unfair summation of that thread?

> FWIW my vote is still for rainbow-mode.el as follows: I think it's most
> useful in its current form out of all the contenders.  I would put it in
> the core Emacs because editing CSS and HTML (plus other languages with
> inlined colors) is so common.  The color transformation functions it has
> are generally useful.  I would also give it first-class customization
> variables so it's really easy to turn it on.  Finally, it's pretty
> unlikely to undergo big changes at this point.

I agree 100%.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: rainbow-mode

[Ted Zlatanov]

On Tue, 16 Nov 2010 15:50:25 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Plus Chong Yidong explained in this thread and in
>> http://thread.gmane.org/gmane.emacs.devel/127873/focus=128245

LMI> I've read the thread.  There was discussion, Julien added stuff to
LMI> rainbow-mode as suggested, and then Chong said "I've added rainbow-mode
LMI> to the Emacs 24 package repository.", and that ended the discussion.
LMI> Is that an unfair summation of that thread?

He also said he would add the others if proposed, meaning (I think) none
of them were ready.  I simply added rainbow-mode.el from the package
repository at that point because it was enough for me.

>> FWIW my vote is still for rainbow-mode.el as follows: I think it's most
>> useful in its current form out of all the contenders.  I would put it in
>> the core Emacs because editing CSS and HTML (plus other languages with
>> inlined colors) is so common.  The color transformation functions it has
>> are generally useful.  I would also give it first-class customization
>> variables so it's really easy to turn it on.  Finally, it's pretty
>> unlikely to undergo big changes at this point.

LMI> I agree 100%.

If the maintainers agree too, I'll work with Julien to make this
happen.

Ted

Re: rainbow-mode

[Julien Danjou]

On Tue, Nov 16 2010, Ted Zlatanov wrote:

> Basically none of the authors wanted to merge their mode with the
> others.  Julien added enough to rainbow-mode.el to make it the best
> choice but it's still not superior to css-color-mode.el in every way.

Just to summarize.

css-color-mode features:
- displays colors for CSS/HTML.
- allow colors edition like augmenting hue, or RGB value with
  interactive functions

rainbow-mode features:
- displays colors for CSS, HTML, LaTeX, and anything that can has
  hexadecimal or X colors (C code, .Xresources, whatever).

So for now rainbow-mode is really about displaying and not editing at
all.

At least for now. :)

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

RE: rainbow-mode

[Drew Adams]

>  > I'm not quite sure what you mean to say here.  Is it "this 
>  > is something that so many people want in Emacs that it's
>  > been partially reimplemented half a dozen times"?
> 
> IIRC, the most recent discussion concluded that none of the half-
> implemented modes is good enough, especially given the nice features
> in the other half-implemented modes, and nobody stepped up to the
> plate to merge them, document the result, and make sure the result
> conforms to coding standards etc.

YRW.  Or you are inventing.  What "half-implemented" modes are you talking
about?  What do either of you think has been "partially reimplemented"?  What
discussion and conclusions are you talking about?

None of the libraries I mentioned (hexrgb.el, palette.el, doremi.el) have been
discussed here for inclusion in Emacs.  They are not half-implemented, and
they've been in use for years.

And none of them reimplement anything.  hexrgb.el and rainbow-mode.el both use
standard HSV/RGB conversion formulas.  That's all they have in common.  Using a
standard conversion formula, whether inches to centimeters or RGB to HSV, is not
"partially reimplementing" anything.

* hexrgb.el is a general-purpose library for color hex code conversion.  It is
used by several other libraries (including DoReMi and Palette and 4 other
libraries I wrote) to do just that.

* rainbow-mode.el is a library for showing color names/codes using the color as
background.  That is one specific use of color.

Similarly, DoReMi, Palette, and rainbow-mode all have different uses.  Yes, you
can use any of them to see what a color's hex code looks like.  But that is not
what they are about.  None of them "partially reimplements" another.

I mentioned DoReMi and Palette in the context of Lars's description of
attempting to tweak colors the hard way ("try #404090, *save*, *reload*, 'no,
darker', etc.").  The point was that there are "WYSIWYG ways to incrementally
adjust colors".

RE: rainbow-mode

[Stephen J. Turnbull]

Drew Adams writes:
 > >  > I'm not quite sure what you mean to say here.  Is it "this 
 > >  > is something that so many people want in Emacs that it's
 > >  > been partially reimplemented half a dozen times"?
 > > 
 > > IIRC, the most recent discussion concluded that none of the half-
 > > implemented modes is good enough, especially given the nice features
 > > in the other half-implemented modes, and nobody stepped up to the
 > > plate to merge them, document the result, and make sure the result
 > > conforms to coding standards etc.
 > 
 > YRW.  Or you are inventing.

Nope.  See Ted Z's post; others understood what I was talking about
even if my description was imprecise or even inaccurate (as signaled
"IIRC", in any case).  I'm just not writing for

    (setq audience (list "Drew Adams"))

Re: rainbow-mode

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> So it was rejected because you didn't think it was generally useful
> enough, basically?

"Rejected" is obviously the wrong word; as Stefan says, all things being
equal, it is preferable to put new non-infrastructure packages into
elpa.gnu.org.

> That's a judgement call, of course, but I think it sounds quite useful.
> If you're fiddling with colours in CSS files, for instance, it would
> speed up the editing/reload process -- at least the way I do it, which
> is "er, this should be a bluish colour, so I'll try #404090", *save*,
> *reload*, "no, darker", etc.
>
> Just seeing what the colour would be at once would be really neat.

You could use the same argument for including all the other packages in
elpa.gnu.org in the tarball.

By the way, if there is real demand for a proper color selection tool,
it ought to be trivial to write an Elisp binding for the GTK color
selector.

Re: rainbow-mode

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> You could use the same argument for including all the other packages in
> elpa.gnu.org in the tarball.

Yes, that's true.  "This would be a good package to have in Emacs" is an
argument you could use for everything.  So would "this package is not
necessary to have in Emacs".

> By the way, if there is real demand for a proper color selection tool,
> it ought to be trivial to write an Elisp binding for the GTK color
> selector.

That would be neat, too, but I'd prefer just to see and edit the colour
specs directly in the CSS file.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: rainbow-mode

[Glenn Morris]

Chong Yidong wrote:

> as Stefan says, all things being equal, it is preferable to put new
> non-infrastructure packages into elpa.gnu.org.

This is a pretty big change in policy, IMO. Who's going to maintain
these things? Where do the bug reports go? If it were in Emacs proper,
I could and would (maybe) be fixing any bugs in it, improving docs,
etc. If it's in elpa.gnu.org, I'm unlikely to ever even look at the
code. And if I did and wanted to change something, all I have is a
tarfile (no?), the repository lives somewhere else, and I'm unlikely
to bother seeking it out, submitting a patch, etc.

I tried the elpa interface and it seems very nifty, but I do share the
concerns expressed about what it all means in practice for the
maintenance of the associated packages. As an "easy way to get a
package that would otherwise not be in Emacs" (eg AUCTeX), it seems
really nice; for farming out things that otherwise _would_ be in
Emacs proper, not so much.

Some of these issues would go away if these "non-infrastructure"
packages (not AUCTeX etc, but rather the things that would have been
added to Emacs before elpa) were hosted in a central Savannah bzr
repository (or possible as a subdirectory in the existing Emacs
repository that is excluded from the tarfiles; I'm not sure if the
number of files in the Bzr version of Emacs matters very much), so
that the Emacs developers can and do feel able to help maintain them.
From my point of, the ideal thing would be something like a separate
top-level "elpa/" directory in the normal repo, not compiled by
default, that I could compile with `make elpa' or somesuch.

Then putting something on elpa doesn't mean relegating it to being a
second-class citizen.

Re: rainbow-mode

[Chong Yidong]

Glenn Morris <rgm@gnu.org> writes:

> I tried the elpa interface and it seems very nifty, but I do share the
> concerns expressed about what it all means in practice for the
> maintenance of the associated packages. As an "easy way to get a
> package that would otherwise not be in Emacs" (eg AUCTeX), it seems
> really nice; for farming out things that otherwise _would_ be in
> Emacs proper, not so much.

I hear you.  Let's try to clarify the discussion.  There are two classes
of packages that could end up on elpa.gnu.org:

  (i)  Small packages.  Individually, each could be included in the
       tarball, but when the number of such small packages goes into the
       hundreds (*), this eventually becomes problematic.

  (ii) Big packages, like AUCTeX, big enough that including it in Emacs
       would be a momentous decision.

Based on the discussion so far, I think we should give all Emacs hackers
some kind of access to modify the packages.  (For instance, we could set
up a separate "dev-only" version of the package repository, which is
occassionally merged into the "public usage" repository.)

The problem is that if we Emacs hackers start tinkering with everything,
the situation could be headache-inducing.  At first glance, it seems
like we should only feel free to modify the small packages, while
leaving the big ones alone.  But there have made many tweaks made to big
packages built into Emacs, such as CEDET.  These tweaks have to be dealt
with when upstream is merged in, but that's life; they have been very
important in making sure that Emacs works as a coherent whole.

So my question is, do we really want to deal the same issues for AUCTeX
and Muse and Company mode and whatever other big package ends up on
elpa.gnu.org?  This is not a rhetorical question; I don't know the
answer.

Re: rainbow-mode

[Stefan Monnier]

> these things? Where do the bug reports go? If it were in Emacs proper,
> I could and would (maybe) be fixing any bugs in it, improving docs,
> etc. If it's in elpa.gnu.org, I'm unlikely to ever even look at the
> code. And if I did and wanted to change something, all I have is a
> tarfile (no?), the repository lives somewhere else, and I'm unlikely
> to bother seeking it out, submitting a patch, etc.

That's a problem with the current setup, indeed.  I think we should be
moving towards a Bzr "packages" branch which we could checkout alongside
Emacs and edit easily to fix bugs.

There's a fair bit of work left to do to get to that point, tho.
Part of the problem here is how to decouple edits from uploads, how to
sync local changes with the upstream maintainer, etc...


        Stefan

elpa.gnu.org repository sync with Emacs (was: rainbow-mode)

[Ted Zlatanov]

On Mon, 15 Nov 2010 22:15:33 -0500 Glenn Morris <rgm@gnu.org> wrote: 

GM> I tried the elpa interface and it seems very nifty, but I do share the
GM> concerns expressed about what it all means in practice for the
GM> maintenance of the associated packages. As an "easy way to get a
GM> package that would otherwise not be in Emacs" (eg AUCTeX), it seems
GM> really nice; for farming out things that otherwise _would_ be in
GM> Emacs proper, not so much.

GM> Some of these issues would go away if these "non-infrastructure"
GM> packages (not AUCTeX etc, but rather the things that would have been
GM> added to Emacs before elpa) were hosted in a central Savannah bzr
GM> repository (or possible as a subdirectory in the existing Emacs
GM> repository that is excluded from the tarfiles; I'm not sure if the
GM> number of files in the Bzr version of Emacs matters very much), so
GM> that the Emacs developers can and do feel able to help maintain them.
GM> From my point of, the ideal thing would be something like a separate
GM> top-level "elpa/" directory in the normal repo, not compiled by
GM> default, that I could compile with `make elpa' or somesuch.

GM> Then putting something on elpa doesn't mean relegating it to being a
GM> second-class citizen.

On Tue, 16 Nov 2010 00:55:16 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> That's a problem with the current setup, indeed.  I think we should be
SM> moving towards a Bzr "packages" branch which we could checkout alongside
SM> Emacs and edit easily to fix bugs.

SM> There's a fair bit of work left to do to get to that point, tho.
SM> Part of the problem here is how to decouple edits from uploads, how to
SM> sync local changes with the upstream maintainer, etc...

I'll try to merge Glenn's comments, your comments, and mine into a plan:

1) add elpa/ directory to main Emacs repo (as a branch or subdirectory;
my vote is for a subdirectory that's not bundled or compiled because it
will get branched together with Emacs itself).  Make it available to a
dev checkout of Emacs as a file:/// URL (so it can be tested easily).

2) let the usual Emacs hackers access elpa/* normally

3) mirror elpa/ to a repo on elpa.gnu.org daily after reviewing the
changes (so it's a supervised pull, not automatic).  Allow admins to
trigger this from the web site.

4) Set up a deploy process of the elpa.gnu.org repo to the HTML tree
(with one package repository per major version as I suggested, plus a
"dev" package repository and an "all" package repository).  This can be
deployed automatically and manually.  Allow admins to trigger
deployments from the web site.

5) set up specific third-party packages to be fetched into the "all"
package repository daily on top of the deployment.  AUCTeX, BBDB,
etc. would be good candidates for this.

This lets us bundle Emacs-local changes to third-party packages into
clear diffs we can send back upstream.  It also separates "commit
something into the package repository" from "deploy the package
repository to the world."

Ted

Re: elpa.gnu.org repository sync with Emacs

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> 1) add elpa/ directory to main Emacs repo (as a branch or
> subdirectory; my vote is for a subdirectory that's not bundled or
> compiled because it will get branched together with Emacs itself).
> Make it available to a dev checkout of Emacs as a file:/// URL (so it
> can be tested easily).

I think it makes more sense to add elpa/ as a branch.  Then we can just
`bzr merge' from savannah into the bzr repository used for elpa.gnu.org,
in order to grab the changes made by Emacs devs.  (I don't think you can
`bzr merge' a subdirectory, or can you?)

First, we should make a change to the elpa.gnu.org repository:
currently, the .tar packages live in the repositories as tarballs.  We
should instead leave them as untarred directories, and put the script on
elpa.gnu.org in charge of tarring them up after `bzr export'.  That way,
changes made to the contents of packages can be viewed with bzr diff.

Still unresolved is the problem of which packages should be considered
"OK" to tweak, and which should not.  I don't relish the idea of having
to do a big merge every time a package is released upstream.  I think we
should have a clear policy that only packages that are developed
principally inside the package bzr repository, or whose maintainer is
explicitly in charge of merging, should be hacked on.

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 10:01:03 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> Ted Zlatanov <tzz@lifelogs.com> writes:
>> 1) add elpa/ directory to main Emacs repo (as a branch or
>> subdirectory; my vote is for a subdirectory that's not bundled or
>> compiled because it will get branched together with Emacs itself).
>> Make it available to a dev checkout of Emacs as a file:/// URL (so it
>> can be tested easily).

CY> I think it makes more sense to add elpa/ as a branch.  Then we can just
CY> `bzr merge' from savannah into the bzr repository used for elpa.gnu.org,
CY> in order to grab the changes made by Emacs devs.  (I don't think you can
CY> `bzr merge' a subdirectory, or can you?)

We would merge the whole Emacs repository into a repo on elpa.gnu.org.
But only the elpa/ subdirectory would get deployed out of that repo, the
rest would be there just to make the merge simpler.  I think that's the
easiest way.

As I said, I think a subdirectory will allow branching which in turn
will freeze the elpa/* packages for a particular version of Emacs
(instead of making special branches for that purpose).  That alone is
worth the price of admission.

CY> First, we should make a change to the elpa.gnu.org repository:
CY> currently, the .tar packages live in the repositories as tarballs.  We
CY> should instead leave them as untarred directories, and put the script on
CY> elpa.gnu.org in charge of tarring them up after `bzr export'.  That way,
CY> changes made to the contents of packages can be viewed with bzr
CY> diff.

No problem.  That would just be a "make elpa" command in each directory
and at the top.  I agree tarballs are a pain for tracking so this works
best.

CY> Still unresolved is the problem of which packages should be considered
CY> "OK" to tweak, and which should not.  I don't relish the idea of having
CY> to do a big merge every time a package is released upstream.  I think we
CY> should have a clear policy that only packages that are developed
CY> principally inside the package bzr repository, or whose maintainer is
CY> explicitly in charge of merging, should be hacked on.

How about a "third-party" package repository, disabled by default but
easily enabled, which would host externally hosted packages?

We would fetch truly external packages such as org-mode into it with
scripts, disconnected from the Emacs repo and the elpa.gnu.org
deployment.  And it would not be versioned: whatever is fetched today is
what you get.

That separates things nicely between "internal" and "external"
elpa.gnu.org packages, with "internal" packages coming from the Emacs
repo, versioned, and hacked regularly; "external" packages independent,
fetched regularly, and maintained outside of Emacs.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

> That separates things nicely between "internal" and "external"
> elpa.gnu.org packages,

I think such a neat separation doesn't actually exist, so I'd rather
treat them all the same, except that some will get more
frequent/significant updates from the upstream maintainers.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 12:28:41 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

>> That separates things nicely between "internal" and "external"
>> elpa.gnu.org packages,

SM> I think such a neat separation doesn't actually exist, so I'd rather
SM> treat them all the same, except that some will get more
SM> frequent/significant updates from the upstream maintainers.

OK, whatever works (we can change our mind before 24 is out so I want to
get things going in any form).  You and Chong can let me know how you
decide to lay things out and I will start getting the deployment and
sync scripts done.  I hope you'll reconsider my proposal to keep the
packages in an elpa/ subdirectory instead of a separate branch.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

> sync scripts done.  I hope you'll reconsider my proposal to keep the
> packages in an elpa/ subdirectory instead of a separate branch.

No, I definitely do not want it within the trunk branch.  It has to have
a separate branch.  Actually, there's a good case to be made for having
a separate branch per package, tho I'd rather not go there because it
would make it too hard for hackers to just always have a checkout of
"all packages".


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 14:24:15 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

SM> Regarding moving stuff from ELPA into Emacs, there's also the idea that
SM> some packages could be moved out of Emacs into ELPA, but come bundled
SM> into the Emacs release tarball.  Gnus and Org would seem like good
SM> candidates for that, although I'm still not sure whether that's
SM> a good idea.

IMO we should enable and supply a file:// repository with a new Emacs
release and with every trunk build (as part of `make install').  That
repository can be augmented by elpa.gnu.org, but if there's no network
connection to it users will at least be able to install the packages in
the latest Emacs checkout and hack on them.

In fact the repository can be generated at install time by the same
scripts and Makefiles that will generate the elpa.gnu.org package
repositories.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> IMO we should enable and supply a file:// repository with a new Emacs
> release and with every trunk build (as part of `make install').  That
> repository can be augmented by elpa.gnu.org, but if there's no network
> connection to it users will at least be able to install the packages in
> the latest Emacs checkout and hack on them.

Conceptually neat, but I agree with Stefan that the practical
difficulties of putting the elpa packages in the trunk are too great.

Those Emacs hackers interested in the elpa packages can always check out
the branch separately, and we could even add a Makefile option that
installs into a local directory.  It doesn't have to be in the trunk.

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 15:02:57 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> Ted Zlatanov <tzz@lifelogs.com> writes:
>> IMO we should enable and supply a file:// repository with a new Emacs
>> release and with every trunk build (as part of `make install').  That
>> repository can be augmented by elpa.gnu.org, but if there's no network
>> connection to it users will at least be able to install the packages in
>> the latest Emacs checkout and hack on them.

CY> Conceptually neat, but I agree with Stefan that the practical
CY> difficulties of putting the elpa packages in the trunk are too great.

They don't have to live in the trunk; sorry I didn't say that clearly.

CY> Those Emacs hackers interested in the elpa packages can always check out
CY> the branch separately, and we could even add a Makefile option that
CY> installs into a local directory.  It doesn't have to be in the trunk.

That's exactly what I meant, with the Makefile option :)

Ted

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

>> 1) add elpa/ directory to main Emacs repo (as a branch or
>> subdirectory; my vote is for a subdirectory that's not bundled or
>> compiled because it will get branched together with Emacs itself).
>> Make it available to a dev checkout of Emacs as a file:/// URL (so it
>> can be tested easily).

> I think it makes more sense to add elpa/ as a branch.  Then we can just
> `bzr merge' from savannah into the bzr repository used for elpa.gnu.org,
> in order to grab the changes made by Emacs devs.

200% agreement.

> (I don't think you can `bzr merge' a subdirectory, or can you?)

If you want to you can, but it's poorly supported by Bazaar, and in any
case we don't want to do that (if for nothing else, so as to avoid
imposing the slowness of Emacs's repository to the elpa repository).

> First, we should make a change to the elpa.gnu.org repository:
> currently, the .tar packages live in the repositories as tarballs.  We
> should instead leave them as untarred directories, and put the script on
> elpa.gnu.org in charge of tarring them up after `bzr export'.  That way,
> changes made to the contents of packages can be viewed with bzr diff.

Yes, obviously.

> Still unresolved is the problem of which packages should be considered
> "OK" to tweak, and which should not.  I don't relish the idea of having
> to do a big merge every time a package is released upstream.  I think we
> should have a clear policy that only packages that are developed
> principally inside the package bzr repository, or whose maintainer is
> explicitly in charge of merging, should be hacked on.

All packages should be OK to hack on.  And all commits should be
reflected on emacs-diffs (or maybe emacs-packages-diffs) *as well as*
sent to the respective upstream maintainer.
And all maintainers will be in charge of merging (which may include
rejecting a change we've made, of course).

What's the difference between this and packages that are bundled, then?
The difference is that we know for sure that the upstream maintainer
automatically gets an email for each one of our changes (i.e. he is
proactively warned of such changes), and also we know that the package
needs to pay attention to backward compatibility, so we have to be more
careful with the changes we install there.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> What's the difference between this and packages that are bundled, then?
> The difference is that we know for sure that the upstream maintainer
> automatically gets an email for each one of our changes (i.e. he is
> proactively warned of such changes),

Surely that could be hacked up with a commit script (or the like) that
would Cc the ";; Author:" line in the files, if that's a thing that we
want to do.

> and also we know that the package needs to pay attention to backward
> compatibility, so we have to be more careful with the changes we
> install there.

So...  ELPA would be for packages that we want to have readily available
in Emacs, but that we can't ever easily cut any cruft out of as Emacs
progresses?  But since the FSF-copyright assignment regime would still
apply to ELPA, we'd wouldn't be able to accept big packages like auctex
anyway?

I don't honestly see how this helps either Emacs users or people who hack
on Emacs.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 19:00:10 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> So...  ELPA would be for packages that we want to have readily available
LMI> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
LMI> progresses?  But since the FSF-copyright assignment regime would still
LMI> apply to ELPA, we'd wouldn't be able to accept big packages like auctex
LMI> anyway?

You know, AUCTeX is in there...  http://elpa.gnu.org/packages/

Chong Yidong and the AUCTeX maintainers spent a lot of time getting the
copyrights done and putting the tarball together.

IMHO big packages are exactly where elpa.gnu.org will be most useful.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> You know, AUCTeX is in there...  http://elpa.gnu.org/packages/

Oh, OK.  Somebody else mentioned possible AUCTeX copyright problems...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Dan Nicolaescu]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Tue, 16 Nov 2010 19:00:10 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>
> LMI> So...  ELPA would be for packages that we want to have readily available
> LMI> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
> LMI> progresses?  But since the FSF-copyright assignment regime would still
> LMI> apply to ELPA, we'd wouldn't be able to accept big packages like auctex
> LMI> anyway?
>
> You know, AUCTeX is in there...  http://elpa.gnu.org/packages/
>
> Chong Yidong and the AUCTeX maintainers spent a lot of time getting the
> copyrights done and putting the tarball together.

Is the any reason not to include AUCTeX in emacs proper instead of elpa?
It's a very useful package to have working by default without having
to deal with setting up packages. 

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Ted Zlatanov]

On Wed, 17 Nov 2010 03:01:09 -0500 Dan Nicolaescu <dann@gnu.org> wrote: 

DN> Ted Zlatanov <tzz@lifelogs.com> writes:
>> On Tue, 16 Nov 2010 19:00:10 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>> 
LMI> So...  ELPA would be for packages that we want to have readily available
LMI> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
LMI> progresses?  But since the FSF-copyright assignment regime would still
LMI> apply to ELPA, we'd wouldn't be able to accept big packages like auctex
LMI> anyway?
>> 
>> You know, AUCTeX is in there...  http://elpa.gnu.org/packages/
>> 
>> Chong Yidong and the AUCTeX maintainers spent a lot of time getting the
>> copyrights done and putting the tarball together.

DN> Is the any reason not to include AUCTeX in emacs proper instead of elpa?

We just went over this several times, didn't we?

DN> It's a very useful package to have working by default without having
DN> to deal with setting up packages. 

You say that like it's a hard task.

M-x package-list-packages
`i' on whatever you want to install, then `x'

Ted

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Dan Nicolaescu]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 17 Nov 2010 03:01:09 -0500 Dan Nicolaescu <dann@gnu.org> wrote: 
>
> DN> Ted Zlatanov <tzz@lifelogs.com> writes:
>>> On Tue, 16 Nov 2010 19:00:10 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>>> 
> LMI> So...  ELPA would be for packages that we want to have readily available
> LMI> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
> LMI> progresses?  But since the FSF-copyright assignment regime would still
> LMI> apply to ELPA, we'd wouldn't be able to accept big packages like auctex
> LMI> anyway?
>>> 
>>> You know, AUCTeX is in there...  http://elpa.gnu.org/packages/
>>> 
>>> Chong Yidong and the AUCTeX maintainers spent a lot of time getting the
>>> copyrights done and putting the tarball together.
>
> DN> Is the any reason not to include AUCTeX in emacs proper instead of elpa?
>
> We just went over this several times, didn't we?

Sorry, I must have missed that discussion.  Can you please point me to it?

> DN> It's a very useful package to have working by default without having
> DN> to deal with setting up packages. 
>
> You say that like it's a hard task.
>
> M-x package-list-packages
> `i' on whatever you want to install, then `x'

That's a lot more complicated than have things just work by default.
AUCTeX is better than the currently integrated latex mode, so it's
better to just integrated it in emacs instead of having people fiddle
with installing packages.

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Richard Stallman]

    Is the any reason not to include AUCTeX in emacs proper instead of elpa?

The benefit of the Emacs package archive (let's not call it elpa!) is
to make the Emacs tar file smaller by offloading a lot of Lisp code
that many people don't want.  I think AUCTeX is a good candidate for
that.

-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Dan Nicolaescu]

Richard Stallman <rms@gnu.org> writes:

>     Is the any reason not to include AUCTeX in emacs proper instead of elpa?
>
> The benefit of the Emacs package archive (let's not call it elpa!) is
> to make the Emacs tar file smaller by offloading a lot of Lisp code
> that many people don't want.  I think AUCTeX is a good candidate for
> that.

Is the tar file size worth worrying about?  It seems hat tar file size
is immaterial for most users nowadays, they install packages from
various distributions.
The advantage of having things included in emacs proper is that one
just has to make sure that has the emacs package installed, and
everything else works.
These days users work on many machines: work, home, maybe a work
laptop, a personal laptop, etc.  Having to configure each one of them
is not ideal, it's better when things just work by default.
AUCTeX is much better that the currently bundled latex/tex modes.

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[David Kastrup]

Dan Nicolaescu <dann@gnu.org> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>>     Is the any reason not to include AUCTeX in emacs proper instead of elpa?
>>
>> The benefit of the Emacs package archive (let's not call it elpa!) is
>> to make the Emacs tar file smaller by offloading a lot of Lisp code
>> that many people don't want.  I think AUCTeX is a good candidate for
>> that.
>
> Is the tar file size worth worrying about?  It seems hat tar file size
> is immaterial for most users nowadays, they install packages from
> various distributions.
> The advantage of having things included in emacs proper is that one
> just has to make sure that has the emacs package installed, and
> everything else works.
> These days users work on many machines: work, home, maybe a work
> laptop, a personal laptop, etc.  Having to configure each one of them
> is not ideal, it's better when things just work by default.
> AUCTeX is much better that the currently bundled latex/tex modes.

Correction: AUCTeX is much better than the currently bundled LaTeX mode.
The other TeX-based modes, however, are of comparable quality to the
existing ones while offering a similar interface to AUCTeX's LaTeX mode.
In particular people preferring to work with plain TeX will often find
the small code footprint and simplicity of Emacs' bundled TeX mode more
suited to their way of thinking.

-- 
David Kastrup

Re: AUCTeX inclusion [Re: elpa.gnu.org repository sync with Emacs]

[Ted Zlatanov]

On Sat, 20 Nov 2010 02:45:30 -0500 Dan Nicolaescu <dann@gnu.org> wrote: 

DN> The advantage of having things included in emacs proper is that one
DN> just has to make sure that has the emacs package installed, and
DN> everything else works.

I really don't think using package.el is any different.  Remember I'm
advocating to make a local install of the packages in the "elpa" branch
as part of the Emacs install from a fresh checkout or in a distro.  If
we do that, network access is unnecessary but users can still get an
updated package if they can contact the elpa.gnu.org server.  So if we
never release an updated package there's no difference, but if we do
have updates then the users don't have to update all of Emacs.

DN> These days users work on many machines: work, home, maybe a work
DN> laptop, a personal laptop, etc.  Having to configure each one of them
DN> is not ideal, it's better when things just work by default.
DN> AUCTeX is much better that the currently bundled latex/tex modes.

It's not hard to add wrappers that will trigger a package install if
necessary and will at least check if the version is the latest
available.  That would come really close to "work by default," probably
more so than the current situation.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Eric Schulte]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> 
> But since the FSF-copyright assignment regime would still apply to
> ELPA, we'd wouldn't be able to accept big packages like auctex anyway?
>

If this is true and ELPA has the same strict copyright restrictions as
the Emacs core, then IMO we're losing a significant portion of the
benefit of ELPA.

There are many packages which will never secure sufficient copyright
permissions to be included into Emacs.  Such packages were previously
available through ELPA.  Where can these packages live now?

Would it make sense to setup a second ELPA repository for non-FSF
software?

Forgive me if this issue has already been settled in a previous
conversation which I've missed.

-- Eric

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 11:29:12 -0700 "Eric Schulte" <schulte.eric@gmail.com> wrote: 

ES> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>> 
>> But since the FSF-copyright assignment regime would still apply to
>> ELPA, we'd wouldn't be able to accept big packages like auctex anyway?

ES> If this is true and ELPA has the same strict copyright restrictions as
ES> the Emacs core, then IMO we're losing a significant portion of the
ES> benefit of ELPA.

It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs ELPA.  I make the
distinction again because it matters:

ES> There are many packages which will never secure sufficient copyright
ES> permissions to be included into Emacs.  Such packages were previously
ES> available through ELPA.  Where can these packages live now?

Tom Tromey's ELPA (http://tromey.com/elpa/) will not go away.  You can
also host your own ELPA-style package repository.  All you need is a web
site with a package directory.

The only thing that makes elpa.gnu.org special is that some of the
package repositories it hosts will be enabled by default with Emacs 24.
But this requires that elpa.gnu.org only host assigned packages, unlike
Tom Tromey's ELPA.

ES> Would it make sense to setup a second ELPA repository for non-FSF
ES> software?

Yes and I suggested it to the Emacs maintainers a while ago, but Tom
Tromey's ELPA may be sufficient (and also see ELL at
http://www.damtp.cam.ac.uk/user/sje30/emacs/ell.html and
http://www.emacswiki.org/ElispArea).  I think we should concentrate on
getting elpa.gnu.org operational first and then cast a wider net.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Eric Schulte]

Hi Ted,

Thanks for the thorough reply, this clears everything up.

Best -- Eric

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Tue, 16 Nov 2010 11:29:12 -0700 "Eric Schulte"
> <schulte.eric@gmail.com> wrote:
>
> ES> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>> 
>>> But since the FSF-copyright assignment regime would still apply to
>>> ELPA, we'd wouldn't be able to accept big packages like auctex anyway?
>
> ES> If this is true and ELPA has the same strict copyright restrictions as
> ES> the Emacs core, then IMO we're losing a significant portion of the
> ES> benefit of ELPA.
>
> It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs ELPA.  I make the
> distinction again because it matters:
>
> ES> There are many packages which will never secure sufficient copyright
> ES> permissions to be included into Emacs.  Such packages were previously
> ES> available through ELPA.  Where can these packages live now?
>
> Tom Tromey's ELPA (http://tromey.com/elpa/) will not go away.  You can
> also host your own ELPA-style package repository.  All you need is a web
> site with a package directory.
>
> The only thing that makes elpa.gnu.org special is that some of the
> package repositories it hosts will be enabled by default with Emacs 24.
> But this requires that elpa.gnu.org only host assigned packages, unlike
> Tom Tromey's ELPA.
>
> ES> Would it make sense to setup a second ELPA repository for non-FSF
> ES> software?
>
> Yes and I suggested it to the Emacs maintainers a while ago, but Tom
> Tromey's ELPA may be sufficient (and also see ELL at
> http://www.damtp.cam.ac.uk/user/sje30/emacs/ell.html and
> http://www.emacswiki.org/ElispArea).  I think we should concentrate on
> getting elpa.gnu.org operational first and then cast a wider net.
>
> Ted

Re: elpa.gnu.org repository sync with Emacs

[Richard Stallman]

    It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs ELPA.  I make the
    distinction again because it matters:

To avoid this confusion, how about renaming the server?
We could call it emacs-pkg.gnu.org.

-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

RE: elpa.gnu.org repository sync with Emacs

[Drew Adams]

>     It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs 
>     ELPA.  I make the distinction again because it matters:
> 
> To avoid this confusion, how about renaming the server?
> We could call it emacs-pkg.gnu.org.

1+

And in discussions (e.g. here) call it something other than ELPA.

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Wed, 17 Nov 2010 11:45:42 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 

>> It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs 
>> ELPA.  I make the distinction again because it matters:
>> 
>> To avoid this confusion, how about renaming the server?
>> We could call it emacs-pkg.gnu.org.

DA> 1+

DA> And in discussions (e.g. here) call it something other than ELPA.

epkg is already used.  emacs-pkg is long and hard to type.

How about ENE (ENE is Not ELPA)?

Ted

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> How about ENE (ENE is Not ELPA)?

ELLE, for ELLE is a Lot Like ELPA.

Stylish!

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org repository sync with Emacs

[Richard Stallman]

    ELLE, for ELLE is a Lot Like ELPA.

Better, ELLE = Elle Looks Like ELPA.

-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

Re: elpa.gnu.org repository sync with Emacs

[Jambunathan K]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 17 Nov 2010 11:45:42 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 
>
>>> It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs 
>>> ELPA.  I make the distinction again because it matters:
>>> 
>>> To avoid this confusion, how about renaming the server?
>>> We could call it emacs-pkg.gnu.org.
>
> DA> 1+
>
> DA> And in discussions (e.g. here) call it something other than ELPA.
>
> epkg is already used.  emacs-pkg is long and hard to type.
>

From a user-perspective the length of the string shouldn't matter. Emacs
will ship with the default archives (whatever it is called) and for
existing installations it is one time re-configuration of
`package-archives'. May be for maintainers the story is different ...

> How about ENE (ENE is Not ELPA)?

ELPA seems like a generic term to me. It is an archive that stores emacs
lisp packages. There could be multiple instantiantions of the archive
(let's call them repositories) and GNU archives is one of it. Needless
to say, each repo can enforce different gating policies or cater to a
niche audience say like windows users.

I think the term ELPA has become 'popular' only in recent years and it
has not been around for so long that it is etched in people's
memory. Furthermore there are other implementations that users use (like
el-get) which more or less provides same functionality as the current
package manager at their core.

I would hesistate to call anything 'not something' unless 'something'
happens to 'a very big thing' or 'a long standing thing' that it gets in
the way of people thinking about it.

Whatever be the name that is adopted, it should capture the 'uniqueness'
of the underlying repo. ie., the name has to speak for itself or it
could be something that is neutral (think 'savannah' for example). 

Just thinking as I type, something 'rooted' in savannah.gnu.org would
create the right association. Or one could get play with names and end
with one of of veldt, puszta, pampas, steppe, prairie ...

Jambunathan K.

>
> Ted

Re: elpa.gnu.org repository sync with Emacs

[Jambunathan K]

Jambunathan K <kjambunathan@gmail.com> writes:

> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> On Wed, 17 Nov 2010 11:45:42 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 
>>
>>>> It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs 
>>>> ELPA.  I make the distinction again because it matters:
>>>> 
>>>> To avoid this confusion, how about renaming the server?
>>>> We could call it emacs-pkg.gnu.org.
>>
>> DA> 1+
>>
>> DA> And in discussions (e.g. here) call it something other than ELPA.
>>

How about http://emacsforge.gnu.org/packages.

These packages actually help one forging of GNU Emacs - these are
privileged packages that are either venerable or beatified and on their
way to sainthood.

Very importantly it is `gnu' and *not* `nongnu'.

> Just thinking as I type, something 'rooted' in savannah.gnu.org would
> create the right association. Or one could get play with names and end
> with one of of veldt, puszta, pampas, steppe, prairie ...
>

Jambunathan K.

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Thu, 18 Nov 2010 04:47:49 +0530 Jambunathan K <kjambunathan@gmail.com> wrote: 

>> How about ENE (ENE is Not ELPA)?

JK> I would hesistate to call anything 'not something' unless 'something'
JK> happens to 'a very big thing' or 'a long standing thing' that it gets in
JK> the way of people thinking about it.

That's the joke I was trying to make.  I guess it wasn't very funny.

JK> How about http://emacsforge.gnu.org/packages.

JK> These packages actually help one forging of GNU Emacs - these are
JK> privileged packages that are either venerable or beatified and on their
JK> way to sainthood.

Whoa.  No.

On Wed, 17 Nov 2010 23:19:45 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:

>> How about ENE (ENE is Not ELPA)?

LMI> ELLE, for ELLE is a Lot Like ELPA.

Works for me.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Byung-Hee HWANG]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

Ted Zlatanov <tzz@lifelogs.com> writes:
> [...]
> epkg is already used.  emacs-pkg is long and hard to type.

Well, i use more mouse than keyboard. So, at least, there is no problem
to me. Also, by the way, i'd like to give you other name if you're OK: 

    GNU-Emacs-Lisp-Package-Archive.GNU.ORG

Sincerely,

-- 
소여물 황병희(黃炳熙) | .. 출항 15분전..

"We can talk business after dinner."
		-- Jack Woltz, "Chapter 1", page 59

[-- Attachment #2: Type: application/pgp-signature, Size: 195 bytes --]

RE: elpa.gnu.org repository sync with Emacs

[Drew Adams]

> i'd like to give you other name if you're OK: 
> 
>     GNU-Emacs-Lisp-Package-Archive.GNU.ORG

1+

(Those so inclined can call it GELPA.  Or not.)

The name should say what it is.  If the abbreviation shows some relation to
ELPA, that's not a bad thing.

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Sat, 20 Nov 2010 19:05:58 +0900 Byung-Hee HWANG <bh@izb.knu.ac.kr> wrote: 

BH> Also, by the way, i'd like to give you other name if you're OK:

BH>     GNU-Emacs-Lisp-Package-Archive.GNU.ORG

That's a very long CNAME.  I think it would annoy everyone.

On Sat, 20 Nov 2010 07:26:17 -0800 "Drew Adams" <drew.adams@oracle.com> wrote: 

DA> (Those so inclined can call it GELPA.  Or not.)

DA> The name should say what it is.  If the abbreviation shows some relation to
DA> ELPA, that's not a bad thing.

I think the confusion between "ELPA-like" and "ELPA" will only get worse
as ELLE (or whatever we call it) becomes more popular.  So I'm against
anything that includes "ELPA" in the name.  ELLE is short, memorable,
and slightly amusing, though I'm sure the joke will wear thin if we
discuss it to death.

I think we have a majority in favor of ELLE (Lars, RMS, and me).  Can we
rely on Chong Yidong and Stefan Monnier to make the final decision?

Ted

Re: elpa.gnu.org repository sync with Emacs

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I think the confusion between "ELPA-like" and "ELPA" will only get
> worse as ELLE (or whatever we call it) becomes more popular.  So I'm
> against anything that includes "ELPA" in the name.  ELLE is short,
> memorable, and slightly amusing, though I'm sure the joke will wear
> thin if we discuss it to death.
>
> I think we have a majority in favor of ELLE (Lars, RMS, and me).  Can
> we rely on Chong Yidong and Stefan Monnier to make the final decision?

I think we should stick to ELPA.  It stands for "Emacs Lisp Package
Archive", which is much more logical and easier to remember than an
unrelated self-referential acronym.  There need not be one single ELPA;
if one needs to draw a distinction, we can refer to GNU ELPA versus Tom
Tromey's ELPA, etc.

Stefan's suggestion to use emacs.gnu.org/packages is fine by me too, but
even in that case we'd still refer to the package archive as the GNU
Emacs package archive, so there's hardly much difference.

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Mon, 22 Nov 2010 11:47:37 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> I think we should stick to ELPA.  It stands for "Emacs Lisp Package
CY> Archive", which is much more logical and easier to remember than an
CY> unrelated self-referential acronym.  There need not be one single ELPA;
CY> if one needs to draw a distinction, we can refer to GNU ELPA versus Tom
CY> Tromey's ELPA, etc.

CY> Stefan's suggestion to use emacs.gnu.org/packages is fine by me too, but
CY> even in that case we'd still refer to the package archive as the GNU
CY> Emacs package archive, so there's hardly much difference.

On Mon, 22 Nov 2010 11:48:10 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

SM> I stand by my suggestion to name the machine emacs.gnu.org (in short,
SM> EGO) and use the name "packages" for the Bzr branch (and URLs to
SM> emacs.gnu.org would also use "packages" somewhere) that holds the
SM> unbundled packages.

OK, so the "GNU ELPA" will be hosted on elpa.gnu.org/packages AKA
emacs.gnu.org/packages.  I'll make sure to use "GNU ELPA" in messages
from now on to denote this package repository.  If we set up multiples
we may need to refine the terminology further but I think this is a good
start.

Thanks
Ted

Re: elpa.gnu.org repository sync with Emacs

[Richard Stallman]

    I think we should stick to ELPA.  It stands for "Emacs Lisp Package
    Archive", which is much more logical and easier to remember than an
    unrelated self-referential acronym.

That is the worst possible name.
We already know that this leads to confusion.

      There need not be one single ELPA;
    if one needs to draw a distinction, we can refer to GNU ELPA versus Tom
    Tromey's ELPA, etc.

But not everyone will know this is necessary.

    Stefan's suggestion to use emacs.gnu.org/packages is fine by me too, but
    even in that case we'd still refer to the package archive as the GNU
    Emacs package archive, so there's hardly much difference.

emacs.gnu.org is fine as a host name, but if the URL doesn't provide
some suitable name to refer to this collection of packages, it won't
avoid the confusion.  emacs.gnu.org/packages doesn't solve the
problem.

So we want to replace either `emacs' or `packages' with a name people
could use conversationally for this particular collection of packages.

Another idea might be to replace the word "archive" with something
else -- "collection"?



-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org

RE: elpa.gnu.org repository sync with Emacs

[Drew Adams]

> So we want to replace either `emacs' or `packages' with a name people
> could use conversationally for this particular collection of packages.

Yes.  The official (i.e. possibly longer) name and the URL should be
unambiguous, but there should also be a unique short name for use in discussion.
None of the three should be easily confusable with ELPA, but any of them could
suggest some relation to it.

For those reasons I supported Byung-Hee's suggestion of
`gnu-emacs-lisp-package-archive.gnu.org' and the acronym GELPA (which recalls
ELPA but is qualified).

`Lisp' could be dropped, if this is the only Emacs package archive.  And
`archive' could be dropped by using `packages'.  For the URL, as opposed to the
long name, the prefix `gnu-' could be dropped if length is a problem (but why
would it be?):

URL:    emacs-packages.gnu.org
Name:   GNU Emacs Packages
Abbrev: GEP

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

> I think we have a majority in favor of ELLE (Lars, RMS, and me).  Can we
> rely on Chong Yidong and Stefan Monnier to make the final decision?

I stand by my suggestion to name the machine emacs.gnu.org (in short,
EGO) and use the name "packages" for the Bzr branch (and URLs to
emacs.gnu.org would also use "packages" somewhere) that holds the
unbundled packages.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

>     It's *not* ELPA, it's elpa.gnu.org.  Tom Tromey runs ELPA.  I make the
>     distinction again because it matters:
> To avoid this confusion, how about renaming the server?
> We could call it emacs-pkg.gnu.org.

I had suggested emacs.gnu.org, simply (after all, it's the name of a VM
which we can use for other Emacs-related services).


        Stefan

RE: elpa.gnu.org repository sync with Emacs

[Drew Adams]

> I had suggested emacs.gnu.org, simply (after all, it's the 
> name of a VM which we can use for other Emacs-related services).

Whatever name/address is chosen for the server, we still also need a name for
talking about this thing - a name that is different from "ELPA" (if that name
refer's to Tom's thing/place).

I suggest that the server name/address reflect whatever name you choose for the
thing itself (the Gnu "elpa").  Keep it simple.

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

> If this is true and ELPA has the same strict copyright restrictions as
> the Emacs core, then IMO we're losing a significant portion of the
> benefit of ELPA.

It doesn't cut any of the benefits of ELPA.  It will only restrict them
to the cases of packages whose copyright has been transferred.
Note that transferring copyright is not nearly as much of a hassle as
many people think.  Yes, it's a big hassle for old packages that have
accumulated many contributions by many people over the years (AUCTeX and
BBDB come to mind), but most Elisp packages are smaller and mostly
written by a single author.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Stefan Monnier]

>> What's the difference between this and packages that are bundled, then?
>> The difference is that we know for sure that the upstream maintainer
>> automatically gets an email for each one of our changes (i.e. he is
>> proactively warned of such changes),
> Surely that could be hacked up with a commit script (or the like) that
> would Cc the ";; Author:" line in the files, if that's a thing that we
> want to do.

Yes, that's indeed what I want (I also want it for the trunk, FWIW).

>> and also we know that the package needs to pay attention to backward
>> compatibility, so we have to be more careful with the changes we
>> install there.
> So...  ELPA would be for packages that we want to have readily available
> in Emacs, but that we can't ever easily cut any cruft out of as Emacs
> progresses?

No, not at all.  Just that those packages will often want to support
older Emacsen just like they already do currently when they're not
in ELPA.

> I don't honestly see how this helps either Emacs users

It helps Emacs users because those packages are easier to install than
having to go to emacswiki, download the file(s), tweak their .emacs, ...

> or people who hack on Emacs.

It reduces the pressure to bundle each and every package with Emacs.
It provides a place to keep "copyright ready code", that we can bundle in
Emacs if/when we decide it's a good idea.
A few other such reasons.

Regarding moving stuff from ELPA into Emacs, there's also the idea that
some packages could be moved out of Emacs into ELPA, but come bundled
into the Emacs release tarball.  Gnus and Org would seem like good
candidates for that, although I'm still not sure whether that's
a good idea.


        Stefan

Re: elpa.gnu.org repository sync with Emacs

[Tom Tromey]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

Lars> I don't honestly see how this helps either Emacs users or people who hack
Lars> on Emacs.

I can name a few benefits.

One is that a big package like gnus, with its own release schedule, can
push updates to elpa.gnu.org between Emacs releases.  This helps gnus
users.  (And, due to the way package activation is done, when such a
user upgrades Emacs, the right thing will still happen.)

Another benefit is that by being in Emacs, it provides elisp authors
some simple packaging guidelines.  My hope is that over time, most 3rd
party elisp will follow these rules (meaning it will be trivial for
users to install the code, regardless of how they get it).  I also hope
the distros move to using package.el for elisp activation.

Tom

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Tom Tromey <tromey@redhat.com> writes:

> I also hope the distros move to using package.el for elisp activation.

The distros have perfectly good packaging solutions already, and they
have the QA in place to make sure that whatever you're downloading will
play nice with what you already have.

Which we don't really have, and it sounds from the discussion that the
Emacs maintainers think it would be too much work to have branches for
each Emacs, for instance.

So we'd basically have the same situation that Perl has with CTAN
vs. the packages that (say) Debian has.  You can say "apt-get install
libemail-mime-createhtml-perl" and get a version of that library that
you know will work on your machine, or you can root around on CTAN,
download the latest version there, and it'll probably work -- except for
the times it doesn't, because it relies on something that's slightly too
new for your machine.

Decoupled packaging over the long haul is hard.  Making sure that
separate packages work together over a decade is difficult.
Dependencies change and stuff break.

I want to be able to install Emacs in 2015 and just have it work without
getting into a morass of packages that I have to install from here and
there.  I want to say "apt-get install emacs", and possibly "apt-get
install slime" (if that's not in Emacs by then), and know that I have
something that has been tested to work on the machine I have then.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org repository sync with Emacs

[Ted Zlatanov]

On Tue, 16 Nov 2010 21:21:40 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Tom Tromey <tromey@redhat.com> writes:
>> I also hope the distros move to using package.el for elisp activation.

LMI> The distros have perfectly good packaging solutions already, and they
LMI> have the QA in place to make sure that whatever you're downloading will
LMI> play nice with what you already have.

The problem is there are lots of distros and most don't have good QA in
place.  Debian and Ubuntu are hardly the norm.  The ELPA-style repos are
a way to work around the lack of standard packaging.

LMI> So we'd basically have the same situation that Perl has with CTAN
LMI> vs. the packages that (say) Debian has.  You can say "apt-get install
LMI> libemail-mime-createhtml-perl" and get a version of that library that
LMI> you know will work on your machine, or you can root around on CTAN,
LMI> download the latest version there, and it'll probably work -- except for
LMI> the times it doesn't, because it relies on something that's slightly too
LMI> new for your machine.

s/CTAN/CPAN/g :)

This problem is not solvable.  You have competing interests (distro
vs. users vs. developers) that are pulling in different directions.
ELPA-style repos for Emacs give control back to the developers and the
users in a distro-neutral way (much like CPAN, in fact).  In addition,
distros can take advantage of that facility and add their own ELPA-style
repos, but they probably won't.  They have a strong interest in keeping
the packaging in their own format, which is perfectly sensible to keep
QA and other costs down.

LMI> Decoupled packaging over the long haul is hard.  Making sure that
LMI> separate packages work together over a decade is difficult.
LMI> Dependencies change and stuff break.

That's the worst haiku *ever* :)

LMI> I want to be able to install Emacs in 2015 and just have it work without
LMI> getting into a morass of packages that I have to install from here and
LMI> there.  I want to say "apt-get install emacs", and possibly "apt-get
LMI> install slime" (if that's not in Emacs by then), and know that I have
LMI> something that has been tested to work on the machine I have then.

If you want QA from the distro you have to install their packages.

Ted

Re: elpa.gnu.org repository sync with Emacs

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> That's the worst haiku *ever* :)

Heh heh heh.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org repository sync with Emacs

[Stephen J. Turnbull]

Stefan Monnier writes:

 > 200% agreement.

Rounding error here. ;-)

Re: ELPA policy

[Edward O'Connor]

> The reason that I raised the question now was that I was trying to
> figure out how to do reasonable colour distance calculations for
> rendering foreground colours in shr.el.  rainbow-mode (for colourising
> colour specs in, for instance, CSS files) was brought up, that had
> similar needs, and its calculations in the same area might possibly be
> reusable by shr.el.

Maybe generalizing `tty-color-approximate' is the right thing to do?


Ted

Re: ELPA policy

[Glenn Morris]

Chong Yidong wrote:

> (Incidentally, there are also some packages in Emacs that might be
> usefully moved out into elpa.gnu.org, e.g. since they are so rarely
> used.  We could also move some of the files in obsolete/ into a
> subrepository, e.g. elpa.gnu.org/packages/obsolete)

I don't see how that would work. If, as soon as something becomes
obsolete, you move it to elpa, you've defeated the entire point of
obsolescence. Emacs goes from "you can assume we have function foo" to
"you can't rely on us having function foo" (unless your users have
added an optional package).

If on the other hand you move it to elpa at the point it which it
would normally be deleted altogether, then you're just prolonging its
life, when the whole point of making things obsolete is to migrate
people away from them.

And to move it at some intermediate stage seems a little pointless.

> (Also, as stated before, the FSF's policy is that for a package to be
> listed on elpa.gnu.org its copyright must be assigned, in the exact same
> way as if it's included in Emacs core.)

I was surprised to see AUCTeX there, since I thought its copyright was
notoriously hard to assign. At a quick glance, tex-fptex.el, tex-jp.el
have non-FSF copyrights in the headers, and there is no information
for the images/ directory. The texinfo source of the manuals and pdfs
also appears to be missing (maybe this is by design).

elpa.gnu.org policy (was: ELPA policy)

[Ted Zlatanov]

On Mon, 15 Nov 2010 16:40:39 +0100 Julien Danjou <julien@danjou.info> wrote: 

JD> - Who will assure there's no regression for Emacs 24.1 user when people
JD>   will starts uploading packages using 24.2 only function?

JD> - Who will assure there's no regression at all?

(I've realized ELPA is ambiguous because there's an ELPA archive at
http://tromey.com/elpa/, so I clarified the topic)

Maybe elpa.gnu.org should publish a repository for each Emacs release,
enabled by default in that release:

All 24.x releases:
elpa.gnu.org/packages-24

24.1 only:
elpa.gnu.org/packages-24.1

24.2 only:
elpa.gnu.org/packages-24.2

...

All releases:
elpa.gnu.org/packages

I think we can handle that with VCS branches.

Through consistent versioning, the newest package in any of the enabled
repositories should win.  So consistent versioning should probably be an
absolute requirement for elpa.gnu.org inclusion, together with the
copyright assignment.

Ted

Re: elpa.gnu.org policy

[Lluís]

Ted Zlatanov writes:

> On Mon, 15 Nov 2010 16:40:39 +0100 Julien Danjou <julien@danjou.info> wrote: 
JD> - Who will assure there's no regression for Emacs 24.1 user when people
JD> will starts uploading packages using 24.2 only function?

JD> - Who will assure there's no regression at all?

[...]
> Maybe elpa.gnu.org should publish a repository for each Emacs release,
> enabled by default in that release:

> All 24.x releases:
> elpa.gnu.org/packages-24

> 24.1 only:
> elpa.gnu.org/packages-24.1

> 24.2 only:
> elpa.gnu.org/packages-24.2

> ...

> All releases:
> elpa.gnu.org/packages

But package.el supports version dependencies, right? Then package
developers should set that accordingly (both lower and upper version
bounds, if necessary).

Using version-specific repositories requires just that same work of
checking compatibility.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: elpa.gnu.org policy

[Ted Zlatanov]

On Mon, 15 Nov 2010 19:01:14 +0100 Lluís <xscript@gmx.net> wrote: 

L> Ted Zlatanov writes:
>> Maybe elpa.gnu.org should publish a repository for each Emacs release,
>> enabled by default in that release:

>> All 24.x releases:
>> elpa.gnu.org/packages-24

>> 24.1 only:
>> elpa.gnu.org/packages-24.1

>> 24.2 only:
>> elpa.gnu.org/packages-24.2

>> ...

>> All releases:
>> elpa.gnu.org/packages

L> But package.el supports version dependencies, right? Then package
L> developers should set that accordingly (both lower and upper version
L> bounds, if necessary).

L> Using version-specific repositories requires just that same work of
L> checking compatibility.

(these are just my ideas, not necessarily the solution)

Package developers tell us some of this, but elpa.gnu.org needs to have
a way of freezing package versions.  Otherwise any new package version
has the potential to break everyone with an older Emacs version.  Just
as with the regular Emacs, we can backport fixes but at least we're
guaranteed that we won't have surprises due to an upgrade.

So packages will start in the global repository.  But when Emacs changes
or the maintainer tells us or we get bug reports, we'll look for
affected packages and branch them accordingly.

Ted

Re: elpa.gnu.org policy

[Chong Yidong]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Package developers tell us some of this, but elpa.gnu.org needs to have
> a way of freezing package versions.  Otherwise any new package version
> has the potential to break everyone with an older Emacs version.  Just
> as with the regular Emacs, we can backport fixes but at least we're
> guaranteed that we won't have surprises due to an upgrade.
>
> So packages will start in the global repository.  But when Emacs changes
> or the maintainer tells us or we get bug reports, we'll look for
> affected packages and branch them accordingly.

Maintaining a separate repository for every Emacs release sounds like a
lot of work.  Currently, it's up to maintainers of upstream/external
packages ensure backward compatibility with older Emacs versions,
e.g. by introducing compatibility functions.  Obviously this system
doesn't work perfectly, but it doesn't seem like we have the manpower
for anything more elaborate.

Re: elpa.gnu.org policy

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Maintaining a separate repository for every Emacs release sounds like a
> lot of work.  Currently, it's up to maintainers of upstream/external
> packages ensure backward compatibility with older Emacs versions,
> e.g. by introducing compatibility functions.  Obviously this system
> doesn't work perfectly, but it doesn't seem like we have the manpower
> for anything more elaborate.

Maybe I'm misunderstanding something, but it sounds to me like you're
saying that if a package is in ELPA, then the maintainer has
(implicitly) committed themselves to support older versions of Emacs in,
sort of, perpetuity.  (Well, for a long time.)

Otherwise people using older versions of Emacs, and who are relying on
ELPA for some of the packages, will suddenly have partially
non-functioning Emacsen.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org policy

[Tom Tromey]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

Lars> Maybe I'm misunderstanding something, but it sounds to me like you're
Lars> saying that if a package is in ELPA, then the maintainer has
Lars> (implicitly) committed themselves to support older versions of Emacs in,
Lars> sort of, perpetuity.  (Well, for a long time.)

A package can have a minimum emacs version dependency.
Users with an older emacs won't be able to install that package.
I think this will be ok in practice as long as most maintainers don't
constantly bump their minimum requirement to the absolute newest.

Tom

Re: elpa.gnu.org policy

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Maybe I'm misunderstanding something, but it sounds to me like you're
> saying that if a package is in ELPA, then the maintainer has
> (implicitly) committed themselves to support older versions of Emacs in,
> sort of, perpetuity.  (Well, for a long time.)

You mean, like Gnus? ;-)

Re: elpa.gnu.org policy

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> You mean, like Gnus? ;-)

If I had my way, I'd only target bzr Emacs.  :-)  It'd cut the amount of
work in half.  

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: elpa.gnu.org policy

[Glenn Morris]

Lars Magne Ingebrigtsen wrote:

> If I had my way, I'd only target bzr Emacs.  :-)  It'd cut the amount of
> work in half.  

Maybe try offering people the choice between the current situation;
and one in which you only support the latest Emacs, and have twice as
much time to develop new stuff...

(I'm glad Gnus finally dropped Emacs 21, at least.)

Re: elpa.gnu.org policy

[Lars Magne Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

> Maybe try offering people the choice between the current situation;
> and one in which you only support the latest Emacs, and have twice as
> much time to develop new stuff...

Yeah...  well, back in the 90s, it made sense to support all the
different Emacs versions, because the development direction of the main
Emacs was so, er, interesting.  Ahem.  So there really were reasons for
people not to just use the latest Emacs.  But I don't really see that as
much of an issue any more...

> (I'm glad Gnus finally dropped Emacs 21, at least.)

Me too.  I wish it was possible to just write towards Emacs 24 and
pretend that nothing else existed, but just providing compat functions
for older/different Emacs versions, but then you'd have competing compat
functions for, well, everything.  So it's not doable.  *sigh*

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

compat unification (was: elpa.gnu.org policy)

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Me too.  I wish it was possible to just write towards Emacs 24 and
> pretend that nothing else existed, but just providing compat functions
> for older/different Emacs versions, but then you'd have competing compat
> functions for, well, everything.  So it's not doable.  *sigh*

Or is it?

I'm just brainstorming here.  (Or barnstorming.  Freebasing.
Something.)

What if...  there was an ems-compat function in bzr Emacs.  It would be
a, sort of, "official" current-Emacs-to-everything compat layer.  It
would probably be totally crufty, but it'd basically be along the lines
of

(unless (fboundp 'put-image)
  (cond ((featurep 'xemacs)
         (defun put-image (...) ...))
        (t
         (defalias 'put-image 'ignore))))

Any packages that are external/internal (like Gnus) would include a copy
of this file in its distribution, and would load it.  Emacs 24 itself
would never load it, but would just have it in its distribution as a
central clearinghouse for the compat functions, and any external
packages could just pull it in as needed.

Does it sound really moronic?
         
-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: compat unification

[Stefan Monnier]

> What if...  there was an ems-compat function in bzr Emacs.  It would be
> a, sort of, "official" current-Emacs-to-everything compat layer.

It couldn't be part of Emacs: Emacs-22's ems-compat wouldn't be able to
come with Emacs-24's compatibility functions, would it?
So it'd have to be an ELPA package.

You can go some way with such a package, but it has significant limits.
E.g. you'll still often want to know whether a feature is supported or
not before using it, because you'll want to use some other code if
it's not.  Also such compatibility definitions may confuse other
packages who'd check (fboundp 'foo) to see if a particular feature
is present.


        Stefan

Re: compat unification

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> It couldn't be part of Emacs: Emacs-22's ems-compat wouldn't be able to
> come with Emacs-24's compatibility functions, would it?
> So it'd have to be an ELPA package.

The ems-compat library would support *all* previous Emacs and XEmacs
instances -- so there would only be one, was my thought.

But having it in ELPA would work, too.

> You can go some way with such a package, but it has significant limits.
> E.g. you'll still often want to know whether a feature is supported or
> not before using it, because you'll want to use some other code if
> it's not.

Yes, and it doesn't really help with functions that do exist, but take
different parameters, for instance.  (Those are thankfully somewhat
rare.)  And variables that you bind to achieve an effect are difficult
to do anything with...

So it's not exactly a panacea...  

> Also such compatibility definitions may confuse other packages who'd
> check (fboundp 'foo) to see if a particular feature is present.

Yes, that's been the main objection to doing this in the past.
However -- packages really should check for features, and not the
presence of functions, so we could declare doing that as an error.  I'm
not sure what the impact would be, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: compat unification

[Glenn Morris]

Lars Magne Ingebrigtsen wrote:

> The ems-compat library would support *all* previous Emacs and XEmacs
> instances

Eeesh, what a horrible thought. I don't think supporting old releases
should be encouraged.

RE: elpa.gnu.org policy

[Drew Adams]

> Yeah...  well, back in the 90s, it made sense to support all the
> different Emacs versions, because the development direction 
> of the main Emacs was so, er, interesting.  Ahem.

Revisionism.

Emacs development today is not as robust and inventive as it was in the 90s
(including Epoch, Zmacs,...), in spite of far better communication today and a
larger number of software developers and Emacs users around the planet.

Far from being some horrible Frankenstein experiment gone bad, that was a
fertile period for Emacs development.

(In my view.)

> So there really were reasons for people not to just use
> the latest Emacs.  But I don't really see that as
> much of an issue any more...

Get real.  There are lots of people _today_ who use other than the latest Emacs.
That you don't understand that or the different reasons they do so is too bad.

> Me too.  I wish it was possible to just write towards Emacs 24 and
> pretend that nothing else existed

Right, who cares about users?

Re: ELPA policy

[Tom Tromey]

>>>>> "Julien" == Julien Danjou <julien@danjou.info> writes:

Julien> - Who will assure there's no regression for Emacs 24.1 user when people
Julien>   will starts uploading packages using 24.2 only function?

You could try making it so uploads require a warning-free byte-compile
on various Emacs versions first.

Packages can require a minimum Emacs version, if the author knows that
this is needed.

Julien> - Who will assure there's no regression at all?

You can't catch all the possible bugs.
I've had ok results on my site just forwarding reports to the package
maintainer for fixing.

Julien> There's probably a lot more question outstanding. I've the feeling ELPA
Julien> is adding a second class citizenship for packages, and I really do not
Julien> like that, at least in its current form.

My view leans more towards package inclusion maximalism -- put
everything in Emacs, but also do more frequent releases of some included
packages via ELPA.

Tom

Re: ELPA policy

[Glenn Morris]

Tom Tromey wrote:

> You could try making it so uploads require a warning-free byte-compile
> on various Emacs versions first.

Hah, we can't even get this for Emacs proper...

Re: ELPA policy

[Stefan Monnier]

> This topic has been recently brought by Lars, Ted and me on the gnus
> mailing list. I've been asked to move this discussion here for
> clarification.

As mentioned Chong, it's still "work in progress" from this point
of view.  But since your questions use the future tense, I'll try to
reply based on what I think should happen in the future.

> - Who will be able to upload to ELPA?

Depends: new packages or updates to existing packages?
For new packages, I'd expect only a few people to have such rights, but
for updates, I'd expect something like "anybody with access to the Bzr
repository".  After all, if they can screw with the main Emacs codebase,
why not with the ELPA packages.

> - Who will fix the packages' bugs?

The upstream maintainers, I'd expect.  That's one of the reasons to have
packages outside of Emacs's tree.

> - Who will assure there's no regression for Emacs 24.1 user when people
>   will starts uploading packages using 24.2 only function?

The upstream maintainer, for the same reason.

> - Who will assure there's no regression at all?

In which sense?  I'd expect the upstream maintainer to take some role
here, but admittedly, since those packages are easily available in
a central place, it's more likely that Emacs maintainers will pay
attention to them when introducing changes to the core.

> - Who will assure there's no really bad things uploaded?

The same people who currently assure that there's no really bad things
installed in the Emacs trunk.

> - Will all new packages go to ELPA, or with some still go to Emacs
>   trunk? And if some can go to the trunk, upon which rules?

We don't have rules for it, right now.  Here are some considerations, on
top of what Chong mentioned:
- in order to keep Emacs maintainable, we'd rather have packages in
  ELPA, all things being equal.
- packages of general usefulness (e.g. libraries) are good candidates
  for inclusion in Emacs, to reduce dependencies among ELPA packages.
  
> - Will all/most of the current packages be moved to ELPA?

Definitely not (in the foreseeable future).

> There's probably a lot more question outstanding. I've the feeling ELPA
> is adding a second class citizenship for packages, and I really do not
> like that, at least in its current form.

We currently have two kinds of packages: bundled and unbundled.  So ELPA
is about adding a third kind in-between, with properties such as
"smooth integration", "ease of installation", ... to be halfway between
the other two.


        Stefan

Re: ELPA policy

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> For new packages, I'd expect only a few people to have such rights,
> but for updates, I'd expect something like "anybody with access to the
> Bzr repository".  After all, if they can screw with the main Emacs
> codebase, why not with the ELPA packages.

One difference, though, is that screwing with the main Emacs codebase
affects only those using the development version of Emacs, and we have
mechanisms like the diffs mailing list for problems to be easily
spotted.  After Emacs 24 is released, by screwing with elpa.gnu.org you
can immediately affect users of deployed stable Emacs versions.

So, we need to be a bit more paranoid for elpa.gnu.org than for our main
repository.

I agree, though, that it would be nice for Emacs developers to easily
edit packages in elpa.gnu.org without going through an onerous package
upload process.  I'm not sure how to set this up, though maybe the way
the Org daily builds are handled can be used as a starting point.

Re: ELPA policy

[Ted Zlatanov]

On Mon, 15 Nov 2010 15:12:15 -0500 Chong Yidong <cyd@stupidchicken.com> wrote: 

CY> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> For new packages, I'd expect only a few people to have such rights,
>> but for updates, I'd expect something like "anybody with access to the
>> Bzr repository".  After all, if they can screw with the main Emacs
>> codebase, why not with the ELPA packages.

CY> One difference, though, is that screwing with the main Emacs codebase
CY> affects only those using the development version of Emacs, and we have
CY> mechanisms like the diffs mailing list for problems to be easily
CY> spotted.  After Emacs 24 is released, by screwing with elpa.gnu.org you
CY> can immediately affect users of deployed stable Emacs versions.

CY> So, we need to be a bit more paranoid for elpa.gnu.org than for our main
CY> repository.

This is one good reason to keep a separate repository per major Emacs
version, at least.  Then you can fix packages without so much fear.

CY> I agree, though, that it would be nice for Emacs developers to easily
CY> edit packages in elpa.gnu.org without going through an onerous package
CY> upload process.  I'm not sure how to set this up, though maybe the way
CY> the Org daily builds are handled can be used as a starting point.

I hope we make it easy to submit changes but harder to publish them.
See below for my staging suggestion.

CY> Maintaining a separate repository for every Emacs release sounds like a
CY> lot of work.  Currently, it's up to maintainers of upstream/external
CY> packages ensure backward compatibility with older Emacs versions,
CY> e.g. by introducing compatibility functions.  Obviously this system
CY> doesn't work perfectly, but it doesn't seem like we have the manpower
CY> for anything more elaborate.

Setting it up is really not hard, even if we don't use it.  Just add
"packages-MAJOR" and "packages-MAJOR.MINOR" to the default "packages"
repository on startup.  Those can be empty but when we need something in
them (and we will, I promise you), they'll Just Work.

The dev checkout version should also have "packages-dev" or some such
that's only enabled if you make a dev build.

All this is not hard if we use VCS branches.  It's a 1-to-1 mapping to
the Emacs repo's branches.  In fact the elpa.gnu.org repo could mirror a
subdirectory of the Emacs repo, which would solve the staging problem
(developers would just stage to the Emacs repo and the elpa.gnu.org
maintainers would publish from there).

I think the alternative approach, keeping it really simple now, will
require more manpower and drain more time long-term.

Ted

Re: ELPA policy

[Richard Stallman]

    - Who will assure there's no really bad things uploaded?

We should treat inclusion in ELPA like inclusion in Emacs.

-- 
Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org