Re: Git version of ELPA

[Stefan Monnier <monnier@iro.umontreal.ca>]

> I'm making this emphasis because git subtree has been mentioned several
> times, and its apparent advantages include being able to merge in both
> directions. If you don't have external repos' histories, apparently
> splitting commits and pushing them back won't work:

I'm mostly concerned about keeping the `elpa' code up-to-date,
i.e. merging from external repositories into elpa, not the other
way around.
Git's subtrees handle this well.

To merge elpa changes back into the external repository, the best option
is to cherry-pick patches.  I don't think there's any VCS that can
handle this side of the sync semi-automatically, because I don't know of
any VCS that handles two-way syncs (except maybe for the special case where
both sides are meant to be exact mirrors, but even those can be
tricky).  So subtrees don't make any difference in this respect, AFAIK.

>> That would be nice.  I did not insist on it, because my tests showed
>> that you can "subtree merge" an external branch after-the-fact: with
>> Bazaar this resulted in serious problems, whereas with Git this is
>> handled sanely (not as well as having the external branch merged
>> right from the start, of course, but good enough).

> With Bazaar, I always just copied the most recent versions of files
> in a package, more or less manually, then marked them in `vc-dir' buffer and
> committed the result as the new revision. I don't think that doing
> only a little better than that is good enough.

With subtrees, I can do

   bzr merge --subtree <external-branch>

and it will work whatever has changed since the last merge of
that branch.  But that was already true to some extent with Bzr.
The difference is that this now works without "bzr join" and its bugs
(and also that most external branches are using Git, so we also avoid
the bugs of bzr-git).

I.e. instead of "bzr join" which can only add a new tree, I can do

   bzr merge --subtree <external-branch>

for a branch that I've never merged before: for lack of a common
ancestor it can't know what's new and what isn't, but it does
the sane thing: a 2-way "merge".
   
>> The `elpa' code will always tend to include local changes.  Less is
>> better, but they're a fact of life.
> Inside package directories?

Yes.

> How necessary is that?

Indispensible.

> If you mean bugfixes touching several packages at once,

Not only.  The difference can have to do with packaging, or with
non-copyright-assigned code, ...

In any case it's the responsibility of the package's maintainer to feed
elpa changes back to the external branch, if any.

> they should be prime candidates for splitting and pushing upstream (I
> think git subtree supports that, not 100% sure).

Sure.

> On the subject of extra files, more specifically:
> 1) Can we do without README and -pkg.el files?

The -pkg.el is the one that holds the metadata, so it's pretty
important.  We could get by without it if <pkg>/<pkg>.el contains the
corresponding info in the usual single-file format, but currently the
presence of <pkg>-pkg.el is used to indicate that this is a multifile
package rather than a singlefile package, so we'd need that
info elsewhere.

Mostly same story for README, except that it's a bit easier (the code
I have might already fallback to reading the Commentary section of
<pkg>/<pkg>.el if the README file is missing).

> 2) Can we handle having README.md (and probably other formats),

Yes.  Patches welcome (tho you might like to wait before the new Git
repository is in place, since the code is being rewritten as we speak).

> .travis.yml, Makefile, extra dirs and .el files not meant for end
> users? Like `doc', extras', `yasnippet-debug.el' and
> `yasnippet-tests.el' in the yasnippet repository on GitHub.

Currently, you can have extra (ignored) files only for singlefile
packages.  Multifile packages will package up whatever is present.
But it should be easy to add some way to list files that should be
skipped.  IOW, same as above "patch is welcome, tho you might like to
wait a bit".

> Probably not, but Melpa does this okay. Aside from not having to pull from
> hundreds of external repositories, I think ELPA should work the same way
> (but also retain and use the Version header values).

That's partly the way it's heading.

> If implementing it is going to be a lot of work, are we willing to take
> package-build.el from the Melpa project and adapt it, or reuse some of its
> pieces?

Hmm... I didn't think about that.  Kinda stupid, eh?
It's probably a bit late, tho (I have most of the old code adjusted
already).  In any case, for enhancements, it's probably a good idea to
look there.

> If maybe, do we care about copyright assignments for its code?

We would, of course.

> My intention here was to illustrate the difference between the file trees,
> not between their contents.

You'd have to ask Yidong, but I have the impression that yasnippet was
not installed in GNU ELPA in a straightforward way to start with
(usually I try to limit changes to copyright-fix up and things like
that); but sadly I can't remember details about that.


        Stefan