Merging CEDET

Merging CEDET

[Chong Yidong]

Hello CEDET folks,

Is the CEDET file-rename branch ready?  If so, now is a good time to
start merging into Emacs.

Re: Merging CEDET

[David Engster]

[Eric forgot to include emacs-devel in CC, which is why I'm quoting his
message fully at the end.]

Eric M. Ludlam writes:
> On 06/02/2012 03:47 AM, Chong Yidong wrote:
>> Hello CEDET folks,
>>
>> Is the CEDET file-rename branch ready?  If so, now is a good time to
>> start merging into Emacs.

>   From the perspective of content, the current trunk in CEDET bzr
> matches up with CEDET 1.1 closely.

To make things clear: I have merged our 'newtrunk' branch (which
included the changes in 'file-rename') into our development trunk. This
means we are now finally working directly with the new file and
directory structure from Emacs, and most changes from Emacs trunk are
now incorporated into CEDET.

As I've already written some time ago, I've started to write a special
package for merging Emacs<->CEDET. The most important goal is to keep
the granularity of commits in both repositories. For this to work, we
should get our work-flow straight before we start.

My idea is the following and I'd like to hear opinions from the VCS
gurus around here if this is the right approach:

  We use three branches:

    - CEDET trunk (in the following: 'cedet')
    - Emacs trunk (in the following: 'emacs'). Of course, we do not
      care for the full Emacs trunk, but only for the CEDET-related
      files (essentially: lisp/cedet and lisp/emacs-lisp/eieio*).
    - a special branch inside of CEDET upstream (in the following: 'merge')

  The 'merge' branch is special in that it follows Emacs *and* CEDET
  development. It is derived from the old 'file-rename' branch and thus
  has a common history with 'cedet', so that we can do proper merges
  from and to 'cedet'.

The main concept is this:

  - 'merge' follows 'emacs' as close as possible. That means:
    - It must *not* have any files from 'cedet' which should not end
      up in 'emacs'. Most importantly, this means that *everything*
      that is in 'merge' must have signed papers.
    - It follows exactly the Emacs directory structure, meaning that
      EIEIO is in lisp/emacs-lisp.

  - Syncing with 'cedet' happens through *merges*.

  - Syncing with 'emacs' happens through *cherry picking*, which with
    bzr just boils down to applying patches from "bzr diff -c
    <revision>". Alternatively, Lluís "bzr transfer" plugin can be used,
    but I couldn't get it to work. Either way, the special
    CEDET-Emacs-merge package I've written is used to track which
    commits have been merged and which not (and for what reason).

I could think of two alternatives to this approach:

  - Drop the special 'merge' branch and directly cherry-pick between the
    two repositories, hence essentially do what org and Gnus are
    currently doing. However, I think this can only work well if both
    repositories are very similar, and CEDET upstream still contains
    lots of stuff which isn't in Emacs.

  - Use *two* merge branches 'to-emacs' and 'from-emacs', that means one
    for each direction and both with a common history with 'cedet'.
    This was actually the initial idea, but by now I think this approach
    is just over-complicating things and could easily get pretty messy.

-David



Eric's full mail:


>  From a quality perspective it is pretty good in that it passes all
> the unit tests and the key interactive tests.  I am familiar with a
> couple typo type bugs I need to check in from the translation to the
> new file format still.  I'll do that today. From a copyright
> assignment perspective, all is good, though my last employer release
> for changes ends on July 3, so I can help out on any big changes for a
> month, and get another release soon if needed.
>
>   From the perspective of transplanting changes between our branches,
> David has merged many changes from Emacs into CEDET, and Lluis was
> working on a script to make it easy to do, so I added him to the CC
> list.  Since CEDET includes a 'contrib' area that doesn't have
> copyright assignments, you will still need to avoid that.  We have
> dropped explicit support for older Emacs versions so many previous
> conflicts have since been removed.  The test suites have all been
> converted to the new file system, so Emacs can use that if you'd like
> to enable the complete suite in Emacs core also.
>
> All-in-all I think you will be in good shape unless David or Lluis is
> familiar with something I am not.  We may need to do a second merge
> later, since our conversion to the new file name format is still quite
> fresh.
>
> Thanks!
> Eric

Re: Merging CEDET

[Lluís]

David Engster writes:

> [Eric forgot to include emacs-devel in CC, which is why I'm quoting his
> message fully at the end.]

> Eric M. Ludlam writes:
>> On 06/02/2012 03:47 AM, Chong Yidong wrote:
>>> Hello CEDET folks,
>>> 
>>> Is the CEDET file-rename branch ready?  If so, now is a good time to
>>> start merging into Emacs.

>> From the perspective of content, the current trunk in CEDET bzr
>> matches up with CEDET 1.1 closely.

> To make things clear: I have merged our 'newtrunk' branch (which
> included the changes in 'file-rename') into our development trunk. This
> means we are now finally working directly with the new file and
> directory structure from Emacs, and most changes from Emacs trunk are
> now incorporated into CEDET.

The original idea was that 2 steps are still missing:

- Check which changes in emacs are still missing in cedet's 'file-rename'
  branch. When done, set that emacs revision in stone, as it will be the base
  for future merges from emacs to cedet.

- Emacs people should check the 'file-rename' branch to manually select any
  desired changes to integrate into Emacs. When done, set that cedet revision in
  stone, as it will be the base for future merges from cedet to emacs.

Now, David finished stabilizing 'file-rename' in 'newtrunk' (as it contained
lots of other changes that were also necessary but not interesting to emacs), so
I'm no longer sure if 'newtrunk' should be used instead, as it might contain
some changes from emacs that are not present in 'file-rename'.



> As I've already written some time ago, I've started to write a special
> package for merging Emacs<->CEDET. The most important goal is to keep
> the granularity of commits in both repositories. For this to work, we
> should get our work-flow straight before we start.

> My idea is the following and I'd like to hear opinions from the VCS
> gurus around here if this is the right approach:

>   We use three branches:

>     - CEDET trunk (in the following: 'cedet')
>     - Emacs trunk (in the following: 'emacs'). Of course, we do not
>       care for the full Emacs trunk, but only for the CEDET-related
>       files (essentially: lisp/cedet and lisp/emacs-lisp/eieio*).
>     - a special branch inside of CEDET upstream (in the following: 'merge')

>   The 'merge' branch is special in that it follows Emacs *and* CEDET
>   development. It is derived from the old 'file-rename' branch and thus
>   has a common history with 'cedet', so that we can do proper merges
>   from and to 'cedet'.

> The main concept is this:

>   - 'merge' follows 'emacs' as close as possible. That means:
>     - It must *not* have any files from 'cedet' which should not end
>       up in 'emacs'. Most importantly, this means that *everything*
>       that is in 'merge' must have signed papers.
>     - It follows exactly the Emacs directory structure, meaning that
>       EIEIO is in lisp/emacs-lisp.

>   - Syncing with 'cedet' happens through *merges*.

>   - Syncing with 'emacs' happens through *cherry picking*, which with
>     bzr just boils down to applying patches from "bzr diff -c
>     <revision>". Alternatively, Lluís "bzr transfer" plugin can be used,
>     but I couldn't get it to work. Either way, the special
>     CEDET-Emacs-merge package I've written is used to track which
>     commits have been merged and which not (and for what reason).

> I could think of two alternatives to this approach:

>   - Drop the special 'merge' branch and directly cherry-pick between the
>     two repositories, hence essentially do what org and Gnus are
>     currently doing. However, I think this can only work well if both
>     repositories are very similar, and CEDET upstream still contains
>     lots of stuff which isn't in Emacs.

>   - Use *two* merge branches 'to-emacs' and 'from-emacs', that means one
>     for each direction and both with a common history with 'cedet'.
>     This was actually the initial idea, but by now I think this approach
>     is just over-complicating things and could easily get pretty messy.

The initial workflow was (=> manual/scripted/whatever operation; -> regular
operation):

emacs/trunk => cedet/from-emacs

  This requires renaming some files in the process (I wrote "bzr transplant" for
  this specific case).

cedet/from-emacs -> cedet/trunk

  In principle, almost no changes should happen in emacs, so this would require
  almost no reviews.

cedet/trunk -> cedet/to-emacs

  Not all changes in cedet should immediately go to the next emacs version, so
  this would probably be based on cherry-picking.

cedet/to-emacs => emacs/trunk

  This requires renaming some files in the process, too (again, can be
  automated).

For the sake of completeness, if 'file-rename' were used to establish the
current synchronization point, under this scheme it would be branched twice as
cedet/{from,to}-emacs.


If I understood it correctly, you're proposing to unify branches
cedet/{from,to}-emacs. I just think it's less messier to have them separate.


Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth

Re: Merging CEDET

[Stefan Monnier]

>   We use three branches:

>     - CEDET trunk (in the following: 'cedet')
>     - Emacs trunk (in the following: 'emacs'). Of course, we do not
>       care for the full Emacs trunk, but only for the CEDET-related
>       files (essentially: lisp/cedet and lisp/emacs-lisp/eieio*).
>     - a special branch inside of CEDET upstream (in the following: 'merge')

Sounds OK.  Note that I do not have a strong opinion on how things are
done as long as the two are kept in sync reliably (ideally
automatically, but most importantly without relying critically on
a single person).


        Stefan

Re: Merging CEDET

[Chong Yidong]

David Engster <deng@randomsample.de> writes:

> As I've already written some time ago, I've started to write a special
> package for merging Emacs<->CEDET.  The most important goal is to keep
> the granularity of commits in both repositories. 

Any progress on this?  It would be a shame for the CEDET merge to miss
the 24.2 merge window.  Or are you still waiting for more input?

Emacs 24.2 merge window (was: Merging CEDET)

[Bastien]

Chong Yidong <cyd@gnu.org> writes:

> David Engster <deng@randomsample.de> writes:
>
>> As I've already written some time ago, I've started to write a special
>> package for merging Emacs<->CEDET.  The most important goal is to keep
>> the granularity of commits in both repositories. 
>
> Any progress on this?  It would be a shame for the CEDET merge to miss
> the 24.2 merge window.

Btw, any hint on when the Emacs 24.2 merge window ends?

I will release Org 7.9 somewhere next week and I plan to
merge it to Emacs trunk one week after that.  Hopefully
this will make it.

Thanks,

-- 
 Bastien

Re: Merging CEDET

[David Engster]

Chong Yidong writes:
> David Engster <deng@randomsample.de> writes:
>
>> As I've already written some time ago, I've started to write a special
>> package for merging Emacs<->CEDET.  The most important goal is to keep
>> the granularity of commits in both repositories. 
>
> Any progress on this?  It would be a shame for the CEDET merge to miss
> the 24.2 merge window.  Or are you still waiting for more input?

It would help to know more about that window. Like, when it ends. :-)

There's one thing I need help with, and that's the final step of pushing
commits into Emacs trunk. I will prepare a branch 'to-emacs' in CEDET
where commits can be cleanly applied to Emacs, but I need someone to
double check and do the final push to Emacs. After that grammar file
disaster, I'm not willing to step into this mine field alone.

-David

Re: Merging CEDET

[Chong Yidong]

David Engster <deng@randomsample.de> writes:

> It would help to know more about that window. Like, when it ends. :-)

I'm thinking around October, but the actual date not been discussed or
decided yet.

> There's one thing I need help with, and that's the final step of
> pushing commits into Emacs trunk. I will prepare a branch 'to-emacs'
> in CEDET where commits can be cleanly applied to Emacs, but I need
> someone to double check and do the final push to Emacs. After that
> grammar file disaster, I'm not willing to step into this mine field
> alone.

OK, let me know where to get the to-emacs branch.