org-merger questions

org-merger questions

[Rasmus]

Hi,

As discussed, Org urgently needs to be merged into Emacs.  I have started
a scratch branch with v9.0.8, although I haven’t pushed it yet.

So far, I do the following changes from the Org release tarball:

- Necessary white-space cleanup, required by probably some git hook to let
  me commit.  We will have to do this on the Org repo as well.
- Hardcode version in org.texi to remove dependency of "org-version.inc".
- Include org-version.el (normally generated at compile-time)
- Include org-loaddefs.el (normally generated at compile-time)

I have a couple of questions.

* Schemas

ox-odt.el needs to be able to find schemas files.  By default it looks
into <Emacs>/etc/schemas and some folders computed from
‘org-odt-data-dir’, which is normally computed at compile-time.  That is
not so nice, and it anyway doesn’t work with some packaging systems such
as "makepkg".

In principal, I’d be happy to just throw the 3 Org schemas files into
<Emacs>/etc/schemas but it already contain an schemas.xml file.  Does
anyone know if the two schemas files can be merged?  Here’s the two files:

    http://orgmode.org/cgit.cgi/org-mode.git/tree/etc/schema/schemas.xml
    http://git.savannah.gnu.org/cgit/emacs.git/tree/etc/schema/schemas.xml


I could also add a quick "fix" to ox-odt.el to look into
<Emacs>/etc/org/schemas for now.

* Compiling

Org seems to mostly "just work" when compiling without the specific Org
Makefiles.

Achim:
Are there any reasons to think that Emacs-core would need to include the
Org Makefiles assuming org-version.el and org-loaddefs.el is "statically"
included, as currently are for Org 8.3?  The "mk" folder is not included
in Emacs-core now.  I assume there is still no reason to do so.

Thanks,
Rasmus

-- 
Send from my Emacs

Re: org-merger questions

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 526 bytes --]

This doesn't have to do with your email directly.

But on the topic of tests, org 9.0.8 will fail 2 tests on emacs master. The
fix is already in org maint branch, post 9.0.8:
http://orgmode.org/cgit.cgi/org-mode.git/commit/?h=maint&id=3bcfed071deca0e9c5e8dd298b28bfd5903c6f9a

Just saying in case someone comes across those 2 test failures:

   FAILED  test-org/custom-properties
   FAILED  test-org/forward-paragraph

So do not worry about those 2 as they will be fixed after the next minor
release of org.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 886 bytes --]

Re: org-merger questions

[Bastien Guerry]

Kaushal Modi <kaushal.modi@gmail.com> writes:

> So do not worry about those 2 as they will be fixed after the next
> minor release of org.

The idea would be to release 9.0.9 from maint just before the merge
to merge a clean version.

-- 
 Bastien

Re: org-merger questions

[Rasmus]

> The idea would be to release 9.0.9 from maint just before the merge
> to merge a clean version.

I have pushed to scratch/org-mode-merge on Emacs.git.

I had to do a few whitespace cleanup to be able to commit.  We should
backport those, I guess.  Other than that, it seemed to compile and work
without issues on my system (GNU/Linux).

Re: schemas.xml files, the Org one contained <?xml version="1.0"?>, which
I have left out.  I added the Org location rules to bottom.

Note that http://thaiopensource.com/ns/locating-rules/1.0 isn’t
accessible.

It does not include tests for now.

Rasmus

Re: org-merger questions

[Yuri Khan]

On Thu, Jun 22, 2017 at 5:31 PM, Rasmus <rasmus@gmx.us> wrote:

> Note that http://thaiopensource.com/ns/locating-rules/1.0 isn’t
> accessible.

XML namespace URIs are not supposed to be accessible. They are used as
opaque identifiers, guaranteed to be unique because presumably people
won’t allocate XML namespaces in domains they don’t own, and will
ensure uniqueness within domains they do own.

Re: org-merger questions

[Kyle Meyer]

Rasmus <rasmus@gmx.us> writes:

>> The idea would be to release 9.0.9 from maint just before the merge
>> to merge a clean version.
>
> I have pushed to scratch/org-mode-merge on Emacs.git.

Thanks.

> I had to do a few whitespace cleanup to be able to commit.  We should
> backport those, I guess.

Hmm, I was hoping I had already caught most of them, but maybe the
issues you encountered were in the few files changed in
scratch/org-mode-merge that I didn't look at.  I only checked
lisp/org/*.el, lisp/doc/misc/org.texi, and etc/refcards/orgcard.tex.

A few problems in scratch/org-mode-merge:

  * scratch/org-mode-merge reverts Emacs-specific changes in
    etc/refcards/orgcard.tex.  This same mistake was made in a previous
    sync and fixed with Emacs's e90dec2be1.  Since then, there have been
    additional Emacs-specific changes added to orgcard.tex, and these
    are all included in Org's emacs-sync branch.

  * I made the mistake of not looking at ORG-NEWS for backports, so
    scratch/org-mode-merge is reverting a few Emacs commits here.  I'll
    backport these, and I can also add your ORG-NEWS formatting changes
    to the Org repo.

  * The copyright years in Org's etc/styles/README have not been kept up
    to date, so scratch/org-mode-merge is overwriting the updated years
    in Emacs's etc/org/README.

And a question:

  * Should we include the "@set DATE ..." when we replace "@include
    org-version.inc" in Emacs's org.texi?  It doesn't seem like we have
    in the past (e.g., in Emacs's 6f66f53f65).

-- 
Kyle

Re: org-merger questions

[Kyle Meyer]

Hi Rasmus,

Kyle Meyer <kyle@kyleam.com> writes:

> Rasmus <rasmus@gmx.us> writes:
>
>> I had to do a few whitespace cleanup to be able to commit.  We should
>> backport those, I guess.
>
> Hmm, I was hoping I had already caught most of them, but maybe the
> issues you encountered were in the few files changed in
> scratch/org-mode-merge that I didn't look at.  I only checked
> lisp/org/*.el, lisp/doc/misc/org.texi, and etc/refcards/orgcard.tex.

It seems that etc/ORG-NEWS, od-manifest-schema-v1.2-os.rnc, and
od-schema-v1.2-os.rnc all required whitespace cleanups.  These cleanups
are now included in Org's maint (and cherry-picked to the emacs-sync branch).

> A few problems in scratch/org-mode-merge:

[...]

>   * I made the mistake of not looking at ORG-NEWS for backports, so
>     scratch/org-mode-merge is reverting a few Emacs commits here.  I'll
>     backport these

I've backported all the changes to ORG-NEWS in maint (and cherry-picked
them to the emacs-sync branch).

Please consider using Org files from the emacs-sync branch instead of
the release_9.0.9 tag.  The emacs-sync branch is a superset of the last
release, but it contains a few more changes that are specific for Emacs.

These are changes that should not be in the maint branch but should be
in the Emacs repo.  In other words, these are the changes that we should
make sure not to overwrite in a sync.  For example, Org org.texi has
"@include org-version.inc", but in the Emacs repo this should be "@set
VERSION 9.0.9" instead.

You can see these changes in the Org repo with

    git diff release_9.0.9..origin/emacs-sync

(This diff also includes the whitespace cleanup and ORG-NEWS backports
that were cherry-picked to emacs-sync because they are in the maint
branch after release_9.0.9 but should be in the 9.0.9 sync.)

> [...] and I can also add your ORG-NEWS formatting changes to the Org repo.

I don't understand the formatting changes made in 5cfdf8dd17 (; Fix
ORG-NEWS formatting in previous commit, 2017-06-22).  Aside from fixing
the alignment of a table that already seems correct in Org's ORG-NEWS,
all the changes seem to be space to tab conversions.  Are these
necessary?

-- 
Kyle

Re: org-merger questions

[Rasmus]

Hi Kyle,

Thanks again.  I synced your suggested changes.

Kyle Meyer <kyle@kyleam.com> writes:

> It seems that etc/ORG-NEWS, od-manifest-schema-v1.2-os.rnc, and
> od-schema-v1.2-os.rnc all required whitespace cleanups.  These cleanups
> are now included in Org's maint (and cherry-picked to the emacs-sync branch).

Indeed.  Those were the ones it complained about.

>> A few problems in scratch/org-mode-merge:
>
> [...]
>
>>   * I made the mistake of not looking at ORG-NEWS for backports, so
>>     scratch/org-mode-merge is reverting a few Emacs commits here.  I'll
>>     backport these
>
> I've backported all the changes to ORG-NEWS in maint (and cherry-picked
> them to the emacs-sync branch).
>
> Please consider using Org files from the emacs-sync branch instead of
> the release_9.0.9 tag.  The emacs-sync branch is a superset of the last
> release, but it contains a few more changes that are specific for Emacs.

Cool.  Thanks a lot.  As said in the previous post, I didn’t use this
branch.

> You can see these changes in the Org repo with
>
>     git diff release_9.0.9..origin/emacs-sync
>
> (This diff also includes the whitespace cleanup and ORG-NEWS backports
> that were cherry-picked to emacs-sync because they are in the maint
> branch after release_9.0.9 but should be in the 9.0.9 sync.)

I updated the files in question to the "emacs-sync" version.

I had to change the version in orgcard.tex as it wasn’t 9.0.9.

>> [...] and I can also add your ORG-NEWS formatting changes to the Org repo.
>
> I don't understand the formatting changes made in 5cfdf8dd17 (; Fix
> ORG-NEWS formatting in previous commit, 2017-06-22).  Aside from fixing
> the alignment of a table that already seems correct in Org's ORG-NEWS,
> all the changes seem to be space to tab conversions.  Are these
> necessary?

I must have done a mistake.  ORG-NEWS seems to always change when I open
with my normal Emacs configuration.  So I redid the whitespace cleanup
with "emacs -q" but apparently that also induced unnecessary changes.

Thanks for fixing this!

Rasmus

-- 
Enough with the blah blah!

Re: org-merger questions

[Kyle Meyer]

Rasmus <rasmus@gmx.us> writes:

> I updated the files in question to the "emacs-sync" version.

Thanks.  Checking the diffs again, things look good to me.

> I had to change the version in orgcard.tex as it wasn’t 9.0.9.

Yes, thanks.  I think this is the only place in the Org repo where the
Org version is hardcoded into a file, and it hasn't been getting bumped
at release time.  Since we automatically generate version numbers
everywhere else, it'd be nice if we did it here too (though of course
we'd still have to manually add the version number to the synced file,
like we do for org.texi).

-- 
Kyle

Re: org-merger questions

[Rasmus]

Hi Kyle,

Thanks for carefully looking into this.

>> I had to do a few whitespace cleanup to be able to commit.  We should
>> backport those, I guess.
>
> Hmm, I was hoping I had already caught most of them, but maybe the
> issues you encountered were in the few files changed in
> scratch/org-mode-merge that I didn't look at.  I only checked
> lisp/org/*.el, lisp/doc/misc/org.texi, and etc/refcards/orgcard.tex.

Perhaps I did something wrong.  I used release 9.0.9, not the emacs-sync
branch.  So if there were additional changes on the emacs-sync branch on
top of 9.0.9 I won’t have picked them up.  This is the version I used:

    http://orgmode.org/cgit.cgi/org-mode.git/tag/?h=emacs-sync&id=release_9.0.9

> A few problems in scratch/org-mode-merge:
>
>   * scratch/org-mode-merge reverts Emacs-specific changes in
>     etc/refcards/orgcard.tex.  This same mistake was made in a previous
>     sync and fixed with Emacs's e90dec2be1.  Since then, there have been
>     additional Emacs-specific changes added to orgcard.tex, and these
>     are all included in Org's emacs-sync branch.

See above.

So do you think it’s best to just cherry-pick these changes back?

>   * I made the mistake of not looking at ORG-NEWS for backports, so
>     scratch/org-mode-merge is reverting a few Emacs commits here.  I'll
>     backport these, and I can also add your ORG-NEWS formatting changes
>     to the Org repo.

Note that the formatting itself is a bit weird in ORG-NEWS.  At least on
my normal Emacs setup it’s always changed when opened.  The only issue was
some dangling whitespace at the end of a couple of lines, which the Emacs
commit hook didn’t like.

>   * The copyright years in Org's etc/styles/README have not been kept up
>     to date, so scratch/org-mode-merge is overwriting the updated years
>     in Emacs's etc/org/README.

OK

> And a question:
>
>   * Should we include the "@set DATE ..." when we replace "@include
>     org-version.inc" in Emacs's org.texi?  It doesn't seem like we have
>     in the past (e.g., in Emacs's 6f66f53f65).

I don’t know.  I think we should be bound by the Emacs practice, not what
was done in the last sync aeon ago.  Principally, I think documents should
be dated.  I don’t know if it’s right in an Emacs context, as it carries
its own date.  idlwave.texi and elisp.texi are the only other dated texi
files.  

Rasmus

-- 
Together we will make the possible totalllly impossible!

Re: org-merger questions

[Kyle Meyer]

Hi Rasmus,

Rasmus <rasmus@gmx.us> writes:

[I see that most of this message is addressed by the next post, but ...]

>> And a question:
>>
>>   * Should we include the "@set DATE ..." when we replace "@include
>>     org-version.inc" in Emacs's org.texi?  It doesn't seem like we have
>>     in the past (e.g., in Emacs's 6f66f53f65).
>
> I don’t know.  I think we should be bound by the Emacs practice, not what
> was done in the last sync aeon ago.  Principally, I think documents should
> be dated.  I don’t know if it’s right in an Emacs context, as it carries
> its own date.  idlwave.texi and elisp.texi are the only other dated texi
> files.  

OK, I don't have a strong preference either way.  In the future, I'll
add the date line when I replace the "@include org-version.inc" line in
the emacs-sync branch.

-- 
Kyle

Re: org-merger questions

[Rasmus]

Hi,

>> I don’t know.  I think we should be bound by the Emacs practice, not what
>> was done in the last sync aeon ago.  Principally, I think documents should
>> be dated.  I don’t know if it’s right in an Emacs context, as it carries
>> its own date.  idlwave.texi and elisp.texi are the only other dated texi
>> files.  
>
> OK, I don't have a strong preference either way.  In the future, I'll
> add the date line when I replace the "@include org-version.inc" line in
> the emacs-sync branch.

Neither do I.  If there's an Emacs policy we should follow that.
Otherwise, I think the best thing is to use the date of the release.  But
other people may think differently.  In any case, it’s a minor issue.

-- 
Lasciate ogni speranza o voi che entrate: siete nella mani di'machellaio

Re: org-merger questions

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 334 bytes --]

On Thu, Jun 22, 2017 at 6:32 AM Rasmus <rasmus@gmx.us> wrote:

> > The idea would be to release 9.0.9 from maint just before the merge
> > to merge a clean version.
>
> I have pushed to scratch/org-mode-merge on Emacs.git.
>

Hi Rasmus,

I have been using the org-mode-merge branch and it looks good to me.

Thanks.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 739 bytes --]

Re: org-merger questions

[Rasmus]

Hi,

Kaushal Modi <kaushal.modi@gmail.com> writes:

> I have been using the org-mode-merge branch and it looks good to me.

Thank you for testing it, Kaushal.  The branch also seems to be working
fine for me.

Rasmus
 
-- 
Enough with the blah blah!

Re: org-merger questions

[Bastien Guerry]

Hi all,

I will merge the org-mode-merger branch into Emacs
master this sunday, around 12am (Paris time).

Let's continue to review this branch for a few days.

John and Eli, if there is any concern, let us know.

Thanks,

-- 
 Bastien

Re: org-merger questions

[John Wiegley]

>>>>> Bastien Guerry <bzg@gnu.org> writes:

> John and Eli, if there is any concern, let us know.

No concerns here, thanks for the heads up, Bastien.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: org-merger questions

[Bastien]

Hi John,

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Bastien Guerry <bzg@gnu.org> writes:
>
>> John and Eli, if there is any concern, let us know.
>
> No concerns here, thanks for the heads up, Bastien.

Done, thanks!  Emacs master now features Org 9.0.9.

-- 
 Bastien

Emacs master now updated to Org 9.0.9 (Was: org-merger questions)

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 700 bytes --]

Yay!

emacs -Q on emacs master will now feature the new Org 9.0.9.

Thanks to Rasmus, Kyle, Bastien in making this merge happen. This was a
mega-merger and it happened in a seamless fashion.

Thanks to Nicolas and everyone involved in the Org development that enabled
us getting Org updates with new features and fixes on timely basis.

On Mon, Jul 3, 2017 at 3:25 AM Bastien <bzg@gnu.org> wrote:

> Hi John,
>
> John Wiegley <jwiegley@gmail.com> writes:
>
> >>>>>> Bastien Guerry <bzg@gnu.org> writes:
> >
> >> John and Eli, if there is any concern, let us know.
> >
> > No concerns here, thanks for the heads up, Bastien.
>
> Done, thanks!  Emacs master now features Org 9.0.9.
>
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1297 bytes --]

Re: Emacs master now updated to Org 9.0.9

[Bastien Guerry]

Indeed, thanks to you and to everyone involved.

I'm very grateful everyone has been patiently baring
with me for this task.

-- 
 Bastien

Re: Emacs master now updated to Org 9.0.9

[numbchild]

[-- Attachment #1: Type: text/plain, Size: 490 bytes --]

Thanks for your work.

[stardiviner]           <Hack this world!>      GPG key ID: 47C32433
IRC(freeenode): stardiviner                     Twitter:  @numbchild
Key fingerprint = 9BAA 92BC CDDD B9EF 3B36  CB99 B8C4 B8E5 47C3 2433
Blog: http://stardiviner.github.io/

On Tue, Jul 4, 2017 at 3:54 PM, Bastien Guerry <bzg@gnu.org> wrote:

> Indeed, thanks to you and to everyone involved.
>
> I'm very grateful everyone has been patiently baring
> with me for this task.
>
> --
>  Bastien
>
>

[-- Attachment #2: Type: text/html, Size: 1203 bytes --]

Re: org-merger questions

[Bastien Guerry]

Hi all,

I’m planning to release Org 9.1.2 tomorrow, and according to Nicolas,
Emacs 26.1 is currently being tested.

Can we move forward with merging latest Org stable version into Emacs?

Maybe by updating the scratch/org-mode-merge branch?

Let me know if there is anything I can do to help here.

Best,

-- 
 Bastien

Re: org-merger questions

[Rasmus]

Bastien Guerry <bzg@gnu.org> writes:

> I’m planning to release Org 9.1.2 tomorrow, and according to Nicolas,
> Emacs 26.1 is currently being tested.
>
> Can we move forward with merging latest Org stable version into Emacs?
>
> Maybe by updating the scratch/org-mode-merge branch?

AFAIK 9.1.1 is already merged to the emacs-26 branch (via
scratch/org-mode-merge).

    http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/org/org-version.el?h=emacs-26

Rasmus

-- 
ツ

Re: org-merger questions

[Rasmus]

Rasmus <rasmus@gmx.us> writes:

> Bastien Guerry <bzg@gnu.org> writes:
>
>> I’m planning to release Org 9.1.2 tomorrow, and according to Nicolas,
>> Emacs 26.1 is currently being tested.
>>
>> Can we move forward with merging latest Org stable version into Emacs?
>>
>> Maybe by updating the scratch/org-mode-merge branch?
>
> AFAIK 9.1.1 is already merged to the emacs-26 branch (via
> scratch/org-mode-merge).
>
>     http://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/org/org-version.el?h=emacs-26

What I mean is, v9.1.2 is a bugfix release of the v9.1 series, so it
should not be an issue to merge it.

Rasmus

-- 
Sådan en god dansk lagereddike kan man slet ikke bruge mere

Re: org-merger questions

[Eli Zaretskii]

> From: Rasmus <rasmus@gmx.us>
> Date: Mon, 02 Oct 2017 11:46:10 +0200
> 
> What I mean is, v9.1.2 is a bugfix release of the v9.1 series, so it
> should not be an issue to merge it.

Indeed, there's no issue with that.

Org mode update missing in NEWS (Was: Re: org-merger questions)

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

Hello all,

I just realized this when the pre-test got announced. As an Org mode user,
having the latest Org version in the emacs core is a big plus. Should there
be a NEWS entry for that, that mentions that now Org mode version has been
updated to version 9.1.2?

On Mon, Oct 2, 2017 at 12:06 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Rasmus <rasmus@gmx.us>
> > Date: Mon, 02 Oct 2017 11:46:10 +0200
> >
> > What I mean is, v9.1.2 is a bugfix release of the v9.1 series, so it
> > should not be an issue to merge it.
>
> Indeed, there's no issue with that.
>
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1053 bytes --]

Re: Org mode update missing in NEWS (Was: Re: org-merger questions)

[Eli Zaretskii]

> From: Kaushal Modi <kaushal.modi@gmail.com>
> Date: Wed, 11 Oct 2017 22:31:09 +0000
> Cc: emacs-devel@gnu.org
> 
> I just realized this when the pre-test got announced. As an Org mode user, having the latest Org version in the
> emacs core is a big plus. Should there be a NEWS entry for that, that mentions that now Org mode version
> has been updated to version 9.1.2?

Done, thanks.

in general, such a note should be added to NEWS whenever a new Org
version is merged.

Re: Org mode update missing in NEWS

[Rasmus]

Eli Zaretskii <eliz@gnu.org> writes:

> in general, such a note should be added to NEWS whenever a new Org
> version is merged.

OK, thanks for the pointer.  I will do that in the future.

Rasmus

-- 
It was you, Jezebel, it was you

Re: org-merger questions

[Kyle Meyer]

Rasmus <rasmus@gmx.us> writes:

> As discussed, Org urgently needs to be merged into Emacs.  I have started
> a scratch branch with v9.0.8, although I haven’t pushed it yet.
>
> So far, I do the following changes from the Org release tarball:
>
> - Necessary white-space cleanup, required by probably some git hook to let
>   me commit.  We will have to do this on the Org repo as well.
> - Hardcode version in org.texi to remove dependency of "org-version.inc".

The emacs-sync branch in the Org repo contains the changes above, as
well as a few other Emacs-specific changes that should not be in Org's
maint branch but should be in the synced files.

Even if you don't sync from emacs-sync rather than release_9.0.8,
running

    git diff release_9.0.8..origin/emacs-sync

in the Org repo can help you see the other Emacs-specific changes that
you shouldn't overwrite.

-- 
Kyle