Merging Org 7.8.02 before Emacs 24.1

Merging Org 7.8.02 before Emacs 24.1

[Bastien]

Dear all,

Emacs trunk now contains Org 7.7 and I'd like to merge Org 7.8.02 -- I
have prepared a patch for this.

Org 7.8.02 comes with two major changes, both worth the merge to happen
before the 24.1 release.

1. New ODT exporter: Jambunathan has been working very hard on this.  I
   didn't merge his work into Org 7.7 because I wanted to clear up some
   copyright issues that took long to understand and to sort out.  This
   is now okay, and not having this exporter in 24.1 would be a major
   frustration for Jambunathan, me and many others.

2. New Babel syntax: Eric Schulte went through a rich discussion with
   the Org Mode folks to simplify the syntax of code blocks.  Releasing
   Emacs 24.1 with the old syntax would expose many users to bad habits
   and force us to strongly suggest using latest Org instead of the one
   that comes with Emacs.  No one would benefit from this.

   Also note that a scientific paper illustrating the use of Emacs Org
   and Babel for scientific writing will be published soon, and relies
   on the new syntax.

Both these features have been heavily tested and are mature enough: the
ODT exporter is now one year old, and discussions about the new syntax
are several weeks old.  It is safe to merge them and it would be really
problematic to leave them behind.

Can I proceed with the merge?  

Btw, is there a fixed date for the release?

Thanks!

-- 
 Bastien

Re: Merging Org 7.8.02 before Emacs 24.1

[Jambunathan K]

Bastien

> Org 7.8.02 comes with two major changes, both worth the merge to happen
> before the 24.1 release.
>
> 1. New ODT exporter: Jambunathan has been working very hard on this.  I
>    didn't merge his work into Org 7.7 because I wanted to clear up some
>    copyright issues that took long to understand and to sort out.  This
>    is now okay, and not having this exporter in 24.1 would be a major
>    frustration for Jambunathan, me and many others.

Where will the odt styles files go? It will definitely go under etc/
hierarchy but where exactly?

Jambunathan K.

Re: Merging Org 7.8.02 before Emacs 24.1

[Leo]

On 2011-12-23 21:54 +0800, Bastien wrote:
> Both these features have been heavily tested and are mature enough: the
> ODT exporter is now one year old, and discussions about the new syntax
> are several weeks old.  It is safe to merge them and it would be really
> problematic to leave them behind.
>
> Can I proceed with the merge? 

Note the included org-mode always has more bugs than upstream. All
reported bugs are not fixed in the included one. I'd suggest the merge
to go ahead.

Leo

Re: Merging Org 7.8.02 before Emacs 24.1

[Jambunathan K]

Jambunathan K <kjambunathan@gmail.com> writes:

> Bastien
>
>> Org 7.8.02 comes with two major changes, both worth the merge to happen
>> before the 24.1 release.
>>
>> 1. New ODT exporter: Jambunathan has been working very hard on this.  I
>>    didn't merge his work into Org 7.7 because I wanted to clear up some
>>    copyright issues that took long to understand and to sort out.  This
>>    is now okay, and not having this exporter in 24.1 would be a major
>>    frustration for Jambunathan, me and many others.
>
> Where will the odt styles files go? It will definitely go under etc/
> hierarchy but where exactly?

I propose that under the Emacs system hierarchy, these files go under
etc/org/. With this, the above files will have the following names.

(i) etc/org/OrgOdtStyles.xml  
(ii) etc/org/OrgOdtContentTemplate.xml

If some other location is chosen, the following var has to be updated
suitably in the bzr repo. Otherwise org-odt will refuse to load.

Please see the following commit in Org's repo.

,----
| commit f237ef8b3766c7fcf4213d8c334ef403ede1c6e5
| Author: Jambunathan K <kjambunathan@gmail.com>
| Date:   Sat Dec 24 11:33:54 2011 +0530
| 
|     org-odt.el (org-odt-styles-dir): It is etc/org/ under vanilla Emacs
|     
|     * org-odt.el (org-odt-styles-dir): Assume that the styles
|     files are located under `data-directory' of Emacs distribution
|     as etc/org/OrgOdtStyles.xml and
|     etc/org/OrgOdtContentTemplate.xml.  Also update docstring.
|     (org-export-odt-schema-dir): Update docstring.
`----








> Jambunathan K.
>
>

-- 

Re: Merging Org 7.8.02 before Emacs 24.1

[Bastien]

Hi Jambunathan,

Jambunathan K <kjambunathan@gmail.com> writes:

> I propose that under the Emacs system hierarchy, these files go under
> etc/org/. With this, the above files will have the following names.

Yes, that was the plan, precisely because `org-odt-styles-dir' is
looking there.

-- 
 Bastien

Re: Merging Org 7.8.02 before Emacs 24.1

[Chong Yidong]

Bastien <bzg@altern.org> writes:

> 2. New Babel syntax: Eric Schulte went through a rich discussion with
>    the Org Mode folks to simplify the syntax of code blocks.  Releasing
>    Emacs 24.1 with the old syntax would expose many users to bad habits
>    and force us to strongly suggest using latest Org instead of the one
>    that comes with Emacs.  No one would benefit from this.

Please provide some more information about the plan for Org development.
Suppose we merge 7.8.02 into trunk; what assurance would we have that,
within months, the Org team wouldn't switch tack and "strongly suggest"
the latest Org containing yet more features, rather than 7.8.02?  That
would defeat the purpose of a feature freeze.

Re: Merging Org 7.8.02 before Emacs 24.1

[Bastien]

Chong Yidong <cyd@gnu.org> writes:

> Please provide some more information about the plan for Org
> development.

I expect 7.9 to happen in January or February and 8.0 in March or
April.  I don't think 7.9 will contain major changes, only small new
features.  The most significant change in Org 8.0 will be a more
systematic use of the new export engine.

In a word: 7.8 contains more new features over 7.7 than 8.0 will 
contain over 7.8.

> Suppose we merge 7.8.02 into trunk; what assurance would we have that,
> within months, the Org team wouldn't switch tack and "strongly suggest"
> the latest Org containing yet more features, rather than 7.8.02?  

No assurance at all.  There will be releases with more features in 
the next coming months.  But I believe none of these features will be 
as important as the two I described in my previous email.

> That would defeat the purpose of a feature freeze.

IMHO the feature freeze would make more sense if there was a deadline
for the Emacs release.  

Is there one?

Planning Org releases and merges would be easier then.

-- 
 Bastien

Re: Merging Org 7.8.02 before Emacs 24.1

[Eric Schulte]

Chong Yidong <cyd@gnu.org> writes:

> Bastien <bzg@altern.org> writes:
>
>> 2. New Babel syntax: Eric Schulte went through a rich discussion with
>>    the Org Mode folks to simplify the syntax of code blocks.  Releasing
>>    Emacs 24.1 with the old syntax would expose many users to bad habits
>>    and force us to strongly suggest using latest Org instead of the one
>>    that comes with Emacs.  No one would benefit from this.
>
> Please provide some more information about the plan for Org development.
> Suppose we merge 7.8.02 into trunk; what assurance would we have that,
> within months, the Org team wouldn't switch tack and "strongly suggest"
> the latest Org containing yet more features, rather than 7.8.02?  That
> would defeat the purpose of a feature freeze.

From the perspective of the Org-mode development mailing list this
latest 7.8 release was seen as "the last chance" to make changes before
the Emacs24 merge.  Hence, many of the pending updates (e.g., the syntax
standardization mentioned above) were specifically completed with the
Emacs24 merge in mind.

I can assure you that there will not be another patch between now and
the Emacs24 release which we (the Org-mode devs) will recommend as
strongly as this patch is recommended.  However (as Bastien mentioned)
development is ongoing and if the release of Emacs24 is truly many
months down the line there is a possibility that another merge will be
suggested.  In either case I think merging 7.8.02 now would be for the
best.

Thanks,

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Merging Org 7.8.02 before Emacs 24.1

[Jambunathan K]

Bastien <bzg@altern.org> writes:

> Chong Yidong <cyd@gnu.org> writes:
>
>> Please provide some more information about the plan for Org
>> development.
>
> I expect 7.9 to happen in January or February and 8.0 in March or
> April.  I don't think 7.9 will contain major changes, only small new
> features.  The most significant change in Org 8.0 will be a more
> systematic use of the new export engine.
>
> In a word: 7.8 contains more new features over 7.7 than 8.0 will 
> contain over 7.8.

ODT exporter has been part of the Org's staging directory since rel-7.6
(early Jul, 2011) [1]. I publicly announced the availability of the
exporter around end of Jan, 2011 [2]. So the exporter has been in the
wild for an year or so.

org-odt is not only complete in terms of features it implements but has
considerably mellowed in the last year.

If stability of org-odt is a concern, I urge the maintainers to set
aside that concern. 

As Bastien notes, I have no "disruptive" features queued up for next
quarter or so. If org-odt gets accepted in to the Emacs-24.1 tree, I can
assure restraint on my part so that org-odt *never* regresses but *only*
improves.

Needless to say, I am for inclusion of org-odt in Emacs-24.1.

Jambunathan K.

Footnotes: 
[1]  http://lists.gnu.org/archive/html/emacs-orgmode/2011-07/msg00294.html
[2]  http://lists.gnu.org/archive/html/emacs-orgmode/2011-01/msg01210.html
-- 

Re: Merging Org 7.8.02 before Emacs 24.1

[Lennart Borgman]

On Tue, Dec 27, 2011 at 08:19, Jambunathan K <kjambunathan@gmail.com> wrote:
> Bastien <bzg@altern.org> writes:
>
>
> Needless to say, I am for inclusion of org-odt in Emacs-24.1.

I agree. I would guess that those using the exporter in org-mode would
like to have it included. And I can't see why not to include it since
it is much better structured and will probably give much fewer errors
to take care of.

Re: Merging Org 7.8.02 before Emacs 24.1

[Chong Yidong]

OK, please go ahead and merge Org 7.8 to trunk.

Re: Merging Org 7.8.02 before Emacs 24.1

[Chong Yidong]

Bastien <bzg@altern.org> writes:

> IMHO the feature freeze would make more sense if there was a deadline
> for the Emacs release.  Is there one?

There is no deadline.  It will take at least a few more months, but we
are close enough that a further Org merge after this one will most
likely be unacceptable, unless it is purely bug fixes.

If you guys are continuing to add features upstream, I recommend setting
up a bugfix-only branch or some other setup for channeling upstream bug
fixes into the Emacs trunk, for the duration of the Emacs pretest.

Re: Merging Org 7.8.02 before Emacs 24.1

[Bastien]

Chong Yidong <cyd@gnu.org> writes:

> OK, please go ahead and merge Org 7.8 to trunk.

Thanks a lot.  

I am currently away from a stable Internet connection,
I will merge Org 7.8.02 when I'm back early next week.

-- 
 Bastien

Re: Merging Org 7.8.02 before Emacs 24.1

[Bastien]

Chong Yidong <cyd@gnu.org> writes:

> Bastien <bzg@altern.org> writes:
>
>> IMHO the feature freeze would make more sense if there was a deadline
>> for the Emacs release.  Is there one?
>
> There is no deadline.  It will take at least a few more months, but we
> are close enough that a further Org merge after this one will most
> likely be unacceptable, unless it is purely bug fixes.

I'll leave it to you and the Emacs team to judge whether those new
features are acceptable or not during the feature freeze.  I guess it
all depends on the features themselves (probably small ones for 7.9) 
and the deadline.

> If you guys are continuing to add features upstream, I recommend setting
> up a bugfix-only branch or some other setup for channeling upstream bug
> fixes into the Emacs trunk, for the duration of the Emacs pretest.

This is what I will try to do from now on: the Org master branch is for
development and the maint branch is for bug fixes.

But there is a problem: each time there is a new Org major release, the
master branch is merged on the maint branch.  If new bug fixes are done
after the major release, how should I merge them into Emacs?

I can think of two solutions: (1) have an emacs-maint branch containing
bug fixes for the last Org version that has been merged into Emacs; the
master branch will be merged on that branch only when the major release
of Org is merged into Emacs; (2) don't merge bug fixes that happen after
a new major release during the feature freeze.

Unless someone come up with a better (simpler) proposal, I will perhaps
try (1) -- if that's too much work, I will fall back on (2).

Best,

-- 
 Bastien

Re: Merging Org 7.8.02 before Emacs 24.1

[Stefan Monnier]

> This is what I will try to do from now on: the Org master branch is for
> development and the maint branch is for bug fixes.

I'd suggest you setup a regular "merge from Emacs trunk".  So your
"maint" branch can simply be the Emacs trunk.


        Stefan

Re: Merging Org 7.8.02 before Emacs 24.1

[Bastien]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I'd suggest you setup a regular "merge from Emacs trunk".  So your
> "maint" branch can simply be the Emacs trunk.

Not sure I can parse this correctly.

Your suggestion is to create a branch on org-mode.git that contains 
the latest org* files from the Emacs trunk?  Then to patch the Emacs
trunk with patches that go into that branch?

Thanks for further explanations!

-- 
 Bastien

Re: Merging Org 7.8.02 before Emacs 24.1

[Eric Schulte]

Bastien <bzg@altern.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> I'd suggest you setup a regular "merge from Emacs trunk".  So your
>> "maint" branch can simply be the Emacs trunk.
>
> Not sure I can parse this correctly.
>
> Your suggestion is to create a branch on org-mode.git that contains 
> the latest org* files from the Emacs trunk?  Then to patch the Emacs
> trunk with patches that go into that branch?
>
> Thanks for further explanations!

I believe the suggestion is to increase the frequency of merges with the
Emacs trunk s.t. every Org-mode major release is also a merge with the
Emacs trunk.  Then we have no need for a release-specific maintenance
branch *and* a branch tracking the Emacs trunk as they will be one in
the same.

I do think that (after the feature freeze) a tighter coupling between
the Org-mode and Emacs development repositories with more frequent and
less dramatic merges would be a good thing.  Although I don't know how
much manual work is required to accomplish each merge.

-- 
Eric Schulte
http://cs.unm.edu/~eschulte/

Re: Merging Org 7.8.02 before Emacs 24.1

[Bastien]

Chong Yidong <cyd@gnu.org> writes:

> OK, please go ahead and merge Org 7.8 to trunk.

Done.

-- 
 Bastien