Commit 49c0ff2 "Don't bind org-agenda .."

Commit 49c0ff2 "Don't bind org-agenda .."

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 536 bytes --]

Hello Mark,

I recently saw this commit:
http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=49c0ff29c2e0243ba35ec17e3e3af49369be43db
on
emacs master branch.

It caught my attention because I haven't seen any commits for org-mode
functional changes in emacs master. The org-related commits in emacs master
are usually ones for compatibility with emacs master.

Not knowing the "right way", I am just confirming if commits like these
should go to org master ( http://orgmode.org/cgit.cgi/org-mode.git/log/ )
instead.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 914 bytes --]

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Hi Kaushal and Mark,

> I recently saw this commit:
> http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=
> 49c0ff29c2e0243ba35ec17e3e3af49369be43db on emacs master branch.
>
> It caught my attention because I haven't seen any commits for
> org-mode functional changes in emacs master. The org-related commits
> in emacs master are usually ones for compatibility with emacs master.
>
> Not knowing the "right way", I am just confirming if commits like
> these should go to org master ( http://orgmode.org/cgit.cgi/
> org-mode.git/log/ ) instead.

For *every* org commit, committers should first consider committing
(1) to the Org maint branch, then (2) to the Org master branch, then
(3) to the Emacs repo.

We don't discourage commits made directly in the Emacs repo, because
we are glad people fix things there too, but life would be a little
easier if all commits would go to upstream first.

In this particular case, the commit should go on Org master.  (No
need to delete it from Emacs.)

About the functionality itself, I just gave a quick look, but it
feels like (org-agenda-redo t) should be responsible for doing what
`org-agenda-redo-all' is doing, and we need to carefully check that
checking all buffers is a good idea.

Thanks,

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Kyle Meyer]

Bastien Guerry <bzg@gnu.org> writes:

> Hi Kaushal and Mark,
>
>> I recently saw this commit:
>> http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=
>> 49c0ff29c2e0243ba35ec17e3e3af49369be43db on emacs master branch.
>>
>> It caught my attention because I haven't seen any commits for
>> org-mode functional changes in emacs master. The org-related commits
>> in emacs master are usually ones for compatibility with emacs master.
>>
>> Not knowing the "right way", I am just confirming if commits like
>> these should go to org master ( http://orgmode.org/cgit.cgi/
>> org-mode.git/log/ ) instead.
>
> For *every* org commit, committers should first consider committing
> (1) to the Org maint branch, then (2) to the Org master branch, then
> (3) to the Emacs repo.
>
> We don't discourage commits made directly in the Emacs repo, because
> we are glad people fix things there too, but life would be a little
> easier if all commits would go to upstream first.

Right, but I think it's worth emphasizing that commits that are more
than just fixes are much harder to deal with.  Under the current system
for syncing Emacs and Org [*], bug fixes are (usually) trivial to
backport to Org's maint branch, but changes that are appropriate for
Org's master branch are more problematic because it's a release from the
Org maint branch that will be synced with the Emacs master branch.

[*] As mentioned several times on this list, the Emacs repo is very
    overdue for a sync with Org.  Aside from this commit, all
    Org-related commits on Emacs's side have been backported (or
    otherwise resolved) to Org's maint branch.  On the Org list, Kaushal
    expressed interest in moving along the final step of syncing, so
    hopefully a sync isn't too far off.
      
> In this particular case, the commit should go on Org master.  (No
> need to delete it from Emacs.)
>
> About the functionality itself, I just gave a quick look, but it
> feels like (org-agenda-redo t) should be responsible for doing what
> `org-agenda-redo-all' is doing, and we need to carefully check that
> checking all buffers is a good idea.

I think the options are

 1. Revert this commit in Emacs and make the (org-agenda-redo t) change
    you propose (in either maint or master).

 2. Revert this commit in Emacs.

 3. Revert this commit in Emacs and apply it to Org's master branch.

 4. Despite this commit being more appropriate for Org's master branch,
    backport it to Org's maint branch because it seems like it would be
    backward-compatible.

Other options will result in Emacs-specific Org code that isn't included
in a released version of Org and that will be lost when syncing.

-- 
Kyle

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 2135 bytes --]

On Mon, Jun 19, 2017 at 3:12 PM Kyle Meyer <kyle@kyleam.com> wrote:

>
> Right, but I think it's worth emphasizing that commits that are more
> than just fixes are much harder to deal with.  Under the current system
> for syncing Emacs and Org [*], bug fixes are (usually) trivial to
> backport to Org's maint branch, but changes that are appropriate for
> Org's master branch are more problematic because it's a release from the
> Org maint branch that will be synced with the Emacs master branch.
>

I believe that in that case, the user just uses the org master branch..
Just as people wanting the latest and greatest emacs use the emacs master
branch :)


> [*] As mentioned several times on this list, the Emacs repo is very
>     overdue for a sync with Org.  Aside from this commit, all
>     Org-related commits on Emacs's side have been backported (or
>     otherwise resolved) to Org's maint branch.  On the Org list, Kaushal
>     expressed interest in moving along the final step of syncing,


Yes!


> so
>     hopefully a sync isn't too far off.
>

It would need someone with Org authority to do the actual sync.. Bastein?
Nicolas? Looks like Bastein did the last Org sync (v8.2.10) onto Emacs
master in Oct 2014. We need to happen again; sync the latest release (9.0.8
as of now), and then keep on syncing the next Org release with Emacs master
each time.

I use emacs master + org master as my daily driver and they work in
harmony. Recently a commit in emacs master broke 'make test' on Org, but
that has been fixed. I think that the best time to sync Org 9.0.8 with
emacs master is *now*. Eli? John?


> > In this particular case, the commit should go on Org master.  (No
> > need to delete it from Emacs.)
> >
> > About the functionality itself, I just gave a quick look, but it
> > feels like (org-agenda-redo t) should be responsible for doing what
> > `org-agenda-redo-all' is doing, and we need to carefully check that
> > checking all buffers is a good idea.
>
> I think the options are
>
>  3. Revert this commit in Emacs and apply it to Org's master branch.
>

I would vote for this option.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 3094 bytes --]

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Eli Zaretskii]

> From: Kaushal Modi <kaushal.modi@gmail.com>
> Date: Mon, 19 Jun 2017 19:36:15 +0000
> Cc: Mark Oteiza <mvoteiza@udel.edu>, Nicolas Goaziou <mail@nicolasgoaziou.fr>,
> 	Emacs developers <emacs-devel@gnu.org>
> 
> I use emacs master + org master as my daily driver and they work in harmony. Recently a commit in emacs
> master broke 'make test' on Org, but that has been fixed. I think that the best time to sync Org 9.0.8 with
> emacs master is *now*. Eli? John? 

What exactly is the decision to be made here?  What are the pro's and
the con's?  For that matter, are there any con's?

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 611 bytes --]

On Mon, Jun 19, 2017 at 3:49 PM Eli Zaretskii <eliz@gnu.org> wrote:

> What exactly is the decision to be made here?


It's just good to know that there is a clear green signal from the emacs
maintainers.


> What are the pro's and
> the con's?  For that matter, are there any con's?
>

From what I understand, people who are still using org 8.2.10 from emacs
master (and not using the latest release from Org Elpa) might need to make
few backward incompatible changes to their org files (
http://orgmode.org/Changes.html ) and/or config.

Pros: A lot of bug fixes and many many new features.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1198 bytes --]

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Hi Kaushal,

Kaushal Modi <kaushal.modi@gmail.com> writes:

> From what I understand, people who are still using org 8.2.10 from
> emacs master (and not using the latest release from Org Elpa) might
> need to make few backward incompatible changes to their org files (
> http://orgmode.org/Changes.html) and/or config.

We have a helper function for the properties drawers syntax change.

Are you aware of other helper functions that we could point at for
the new syntax of export blocks and the new constraint on the Babel
header properties?  Maybe people have been sending recipes on the
mailing list, it would be useful to gather that knowledge in the
etc/ORG-NEWS file.

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 1016 bytes --]

On Mon, Jun 19, 2017 at 4:41 PM Bastien Guerry <bzg@gnu.org> wrote:

> Hi Kaushal,
>
> Kaushal Modi <kaushal.modi@gmail.com> writes:
>
> > From what I understand, people who are still using org 8.2.10 from
> > emacs master (and not using the latest release from Org Elpa) might
> > need to make few backward incompatible changes to their org files (
> > http://orgmode.org/Changes.html) and/or config.
>
> We have a helper function for the properties drawers syntax change.
>
> Are you aware of other helper functions that we could point at for
> the new syntax of export blocks and the new constraint on the Babel
> header properties?  Maybe people have been sending recipes on the
> mailing list, it would be useful to gather that knowledge in the
> etc/ORG-NEWS file.
>

Not that I am aware of. Also I have never needed to use any of the helper
functions because by chance, none of my existing org files or config were
that backward incompatible that I would need helper functions to fix those.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1562 bytes --]

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Kaushal Modi <kaushal.modi@gmail.com> writes:

> Not that I am aware of.

Okay, thanks for checking.

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Eli Zaretskii <eliz@gnu.org> writes:

> What exactly is the decision to be made here?  What are the pro's and
> the con's?  For that matter, are there any con's?

There are no con's on my side but John had some concern about changes
made in Org 8.3.

The document I just sent is something I promised John a long time ago,
so I'm responsible for the delay.  I hope the document clears the path
for a new sync soon.

Thanks!

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Kyle Meyer]

Kaushal Modi <kaushal.modi@gmail.com> writes:

> On Mon, Jun 19, 2017 at 3:12 PM Kyle Meyer <kyle@kyleam.com> wrote:
>
>> Right, but I think it's worth emphasizing that commits that are more
>> than just fixes are much harder to deal with.  Under the current system
>> for syncing Emacs and Org [*], bug fixes are (usually) trivial to
>> backport to Org's maint branch, but changes that are appropriate for
>> Org's master branch are more problematic because it's a release from the
>> Org maint branch that will be synced with the Emacs master branch.
>
> I believe that in that case, the user just uses the org master branch..
> Just as people wanting the latest and greatest emacs use the emacs master
> branch :)

The issue is that, if we backport to Org's master branch, the changes in
Emacs's master branch will be reverted when synced with the release from
Org's maint branch.

-- 
Kyle

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Kyle Meyer <kyle@kyleam.com> writes:

> The issue is that, if we backport to Org's master branch, the changes in
> Emacs's master branch will be reverted when synced with the release from
> Org's maint branch.

I'm not entirely sure I understand here.

In general we should backport changes from Emacs master to Org maint
branch, because changes in Emacs master are done against the latest
sync'ed Org maint release.

But for this particular case, applying the change in Org master is
the way to go, because the change is not a bug fix.

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Glenn Morris]

Kyle Meyer wrote:

> The issue is that, if we backport to Org's master branch, the changes in
> Emacs's master branch will be reverted when synced with the release from
> Org's maint branch.

That's only true if an Org "sync" is yet again going to be just a dump
of the files from Org on top of the ones in Emacs. As has been explained
before, this is not how to do it. In relevant branch of the Org repo,
construct the diff between the current revision and the last revision
that was synced with Emacs. Then apply that diff to Emacs.

It really seems to me that keeping code in two repositories does not
benefit either users or developers (witness the apparent problems caused
by one single, simple commit). Did the project to include elpa packages
in Emacs releases stall?

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Hi Glenn,

Glenn Morris <rgm@gnu.org> writes:

> That's only true if an Org "sync" is yet again going to be just a dump
> of the files from Org on top of the ones in Emacs. As has been explained
> before, this is not how to do it. In relevant branch of the Org repo,
> construct the diff between the current revision and the last revision
> that was synced with Emacs. Then apply that diff to Emacs.

One problem is that the set of files in Org's repo is not the same
than the set of files in Emacs org/ directory.

So we cannot simply use ~$ git diff release_8.2.0 release_9.0.8 to
produce a patch that we would apply in the Emacs master branch.

We can produce individual patches for all the relevant files, though,
then carefully apply each of them, just to check that we are not
overwriting Emacs-only changes.

If there is a better recipe that does not involve maintaining another
branch in the Org repo, please let me know.

> It really seems to me that keeping code in two repositories does not
> benefit either users or developers (witness the apparent problems caused
> by one single, simple commit). Did the project to include elpa packages
> in Emacs releases stall?

I don't know about this, but I think Org should be used instead of
outline-mode for etc/TODO, etc/*NEWS, and some other files, and if
we go that way, Org needs to be part of Emacs core.

If such a move requires to trim Org's core from some non-essential
features, I'd still be in favor of it.

But I agree we should only use one repo, either Emacs or Org.

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Kyle Meyer]

Glenn Morris <rgm@gnu.org> writes:

> Kyle Meyer wrote:
>
>> The issue is that, if we backport to Org's master branch, the changes in
>> Emacs's master branch will be reverted when synced with the release from
>> Org's maint branch.
>
> That's only true if an Org "sync" is yet again going to be just a dump
> of the files from Org on top of the ones in Emacs. As has been explained
> before, this is not how to do it. In relevant branch of the Org repo,
> construct the diff between the current revision and the last revision
> that was synced with Emacs. Then apply that diff to Emacs.

No, you face the same problem either way.  You have commit X in Emacs's
master branch and in Org's master branch, but not Org's maint branch.
Because the goal is to update Emacs to the latest Org release from Org's
maint [*], commit X would be reverted in the sync.  Sure, you could
selectively ignore the hunk that reverts back to maint's state
(i.e. keep the changes from commit X), but then you create a situation
where Emacs's Org files are a weird mix between the released version of
Org (maint) and the developmental version (Org's master).

-- 
Kyle

Org contribution flow (Was: Commit 49c0ff2 "Don't bind org-agenda ..")

[Kaushal Modi]

[-- Attachment #1.1: Type: text/plain, Size: 2188 bytes --]

Hello all,

I thought it would be better to spawn off a different thread to discuss
this.

Based on my observation, the Org contribution flow is something like this:

[image: image.png]


On Mon, Jun 19, 2017 at 7:16 PM Kyle Meyer <kyle@kyleam.com> wrote:

> Glenn Morris <rgm@gnu.org> writes:
>
> > Kyle Meyer wrote:
> >
> >> The issue is that, if we backport to Org's master branch, the changes in
> >> Emacs's master branch will be reverted when synced with the release from
> >> Org's maint branch.
>

I believe that that reverting is OK as long as that is a new-feature
commit. That revert will be "reverted" the next time Org major release (cut
from org master) is merged into emacs master. I made the above diagram to
clarify this.

> That's only true if an Org "sync" is yet again going to be just a dump
> > of the files from Org on top of the ones in Emacs. As has been explained
> > before, this is not how to do it. In relevant branch of the Org repo,
> > construct the diff between the current revision and the last revision
> > that was synced with Emacs. Then apply that diff to Emacs.
>
> No, you face the same problem either way.  You have commit X in Emacs's
> master branch and in Org's master branch, but not Org's maint branch.
> Because the goal is to update Emacs to the latest Org release from Org's
> maint [*], commit X would be reverted in the sync.  Sure, you could
> selectively ignore the hunk that reverts back to maint's state
> (i.e. keep the changes from commit X),




> but then you create a situation
> where Emacs's Org files are a weird mix between the released version of
> Org (maint) and the developmental version (Org's master).
>

And thus to prevent this mix, Org related commits should not happen
directly to emacs master. They must go to either org maint or org master.
If they go to org maint (bug/doc fix), emacs master will see those commits
merged sooner. If they go to org master (new feature), emacs master will
eventually see them but only when an org major release is cut out.

Nutshell: Use org master to get bug/doc fixes + new org features at the
soonest.

PS: Please point out any error with the above diagram.
-- 

Kaushal Modi

[-- Attachment #1.2: Type: text/html, Size: 3259 bytes --]

[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 83305 bytes --]

Re: Org contribution flow

[Bastien Guerry]

Hi Kaushal,

the diagram looks neat, thanks! It would be useful to have it, with
some contextual explanations on Worg.

Modi <kaushal.modi@gmail.com> writes:

> And thus to prevent this mix, Org related commits should not happen
> directly to emacs master. They must go to either org maint or org
> master.

I don't want to ask Emacs developers not to commit Org-related changes
to the Emacs repo.  All those commits are very useful since ~10 years,
and we cannot force developers to use another repo.

We need to make it easier for Org maintainers to know when a commit
hit the org/ directory in the Emacs repo though.  An email would be
nice.

But it's not a big issue.

Something I do before copying Org files into Emacs master branch: in
org-mode/, I do a "git checkout release_8.2.10" (using the tag of the
latest Org version sync'd), then I copy Org .el files to Emacs, then I
check the diff from Emacs to see what changed.

In general, most of these changes are already backported to Org, but
this helped me spotting a few mistakes in the past.

Anyway -- I have some time for the merge today and tomorrow.
The next time window for me is next monday.

Best,

-- 
 Bastien

Re: Org contribution flow

[Mark Oteiza]

[-- Attachment #1: Type: text/plain, Size: 757 bytes --]

On Jun 21, 2017 2:57 AM, "Bastien Guerry" <bzg@gnu.org> wrote:

Modi <kaushal.modi@gmail.com> writes:

> And thus to prevent this mix, Org related commits should not happen
> directly to emacs master. They must go to either org maint or org
> master.

I don't want to ask Emacs developers not to commit Org-related changes
to the Emacs repo.  All those commits are very useful since ~10 years,
and we cannot force developers to use another repo.

We need to make it easier for Org maintainers to know when a commit
hit the org/ directory in the Emacs repo though.  An email would be
nice.


With cgit you can subscribe to atom feeds of paths within the repository.
For instance, the org directory:

https://git.savannah.gnu.org/cgit/emacs.git/atom/lisp/org

[-- Attachment #2: Type: text/html, Size: 1373 bytes --]

Re: Org contribution flow

[Bastien Guerry]

Mark Oteiza <mvoteiza@udel.edu> writes:

> With cgit you can subscribe to atom feeds of paths within the
> repository. For instance, the org directory:
>
> https://git.savannah.gnu.org/cgit/emacs.git/atom/lisp/org

Neat!  I've created the group

 gwene.org.gnu.savannah.git.cgit.emacs.git.atom.lisp.org

on gwene.org, I will read the feed from there.

Thanks for sharing this,

-- 
 Bastien

Re: Org contribution flow

[Kyle Meyer]

Bastien Guerry <bzg@gnu.org> writes:

[...]

> We need to make it easier for Org maintainers to know when a commit
> hit the org/ directory in the Emacs repo though.  An email would be
> nice.

When I check for backports each week, I run a command in the Emacs repo
that is essentially

  git log $rev..origin/emacs-25 -- lisp/org doc/misc/org.texi etc/refcards/orgcard.tex

where $rev is the last commit in the emacs-25 branch that I've
backported to the Org repo.  I do the same for the Emacs master branch.

-- 
Kyle

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Kyle Meyer]

Kaushal Modi <kaushal.modi@gmail.com> writes:

> On Mon, Jun 19, 2017 at 3:12 PM Kyle Meyer <kyle@kyleam.com> wrote:

[...]

>> [*] As mentioned several times on this list, the Emacs repo is very
>>     overdue for a sync with Org.  Aside from this commit, all
>>     Org-related commits on Emacs's side have been backported (or
>>     otherwise resolved) to Org's maint branch.  On the Org list, Kaushal
>>     expressed interest in moving along the final step of syncing,
>
> Yes!
>
>> so
>>     hopefully a sync isn't too far off.
>
> It would need someone with Org authority to do the actual sync.. Bastein?
> Nicolas? Looks like Bastein did the last Org sync (v8.2.10) onto Emacs
> master in Oct 2014.

As I mentioned to you in a message in May [*], Rasmus [+cc] was
previously working on doing a sync.  I think it'd be good to check with
him.

[*] https://lists.gnu.org/archive/html/emacs-orgmode/2017-05/msg00354.html

-- 
Kyle

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Kyle Meyer <kyle@kyleam.com> writes:

> As I mentioned to you in a message in May [*], Rasmus [+cc] was
> previously working on doing a sync.  I think it'd be good to check with
> him.

Sure--sorry Rasmus!  Let me know if you want to do the merge
from Org latest stable release to Emacs master branch.

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Rasmus Pank Roulund]

Bastien Guerry <bzg@gnu.org> writes:

> Kyle Meyer <kyle@kyleam.com> writes:
>
>> As I mentioned to you in a message in May [*], Rasmus [+cc] was
>> previously working on doing a sync.  I think it'd be good to check with
>> him.
>
> Sure--sorry Rasmus!  Let me know if you want to do the merge
> from Org latest stable release to Emacs master branch.

I started doing it, but got distracted with trying to make the org unit
tests and the Emacs unit tests work together.  ATM I’m rather busy.  If
someone wants to do this sync up feel free.  Maybe for now we can leave
out the unit tests?

Rasmus

-- 
Sådan en god dansk lagereddike kan man slet ikke bruge mere

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Michael Albinus]

Rasmus Pank Roulund <rasmus@gmx.us> writes:

Hi Rasmus,

> I started doing it, but got distracted with trying to make the org unit
> tests and the Emacs unit tests work together.  ATM I’m rather busy.  If
> someone wants to do this sync up feel free.  Maybe for now we can leave
> out the unit tests?

Maybe we could sync the org unit tests to Emacs, but disable them for
the time being in the Makefile? This would give Emacs developers the
chance to work on them case by case, in order to enable them in the
Emacs environment.

As a starter, all org unit tests could carry

:tags '(:org :not-synced)

in the ert-deftest heading. This allows a simple Makefile rule to filter
them out by suppressing tests with the tag :not-synced. Test cases,
which work in the Emacs environment, will loose the :not-synced tag.

Later on, the :expensive-test tag could be added to the heading of, hmm,
expensive org unit tests.

All of this does not matter in the org environment, which does not care
the ert tags yet.

Background: a while ago, I've started to migrate org tests into the
Emacs environment. It wasn't straight forward, and due to other
priorities I've stopped it. But if org tests would be in the Emacs
environment, I could restart my work (or any other Emacs developer), and
we could enable test by test via proper ert tags. This might work better
than an all-or-nothing approach.

> Rasmus

Best regards, Michael.

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Hi Rasmus and Michael,

I am in favor of leaving tests aside for this sync.

Since the exact same Org version that resides in Emacs is already
tested from Org's repository, I do not see what is the additional
value of having Org tests in Emacs.

Except for when an Emacs contributor commits a change in Emacs
lisp/org/, but we explicitely discourage this and this does not
happen often.

In the long term, when Org's core will be completely merged into
Emacs and when Org's development will happen there only, we would
need to migrate the numerous tests, for sure.

Thanks,

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Rasmus]

Hi,

(I have not read the whole thread, only the messages that appeared in my
 inbox; sorry)

Bastien Guerry <bzg@gnu.org> writes:

> I am in favor of leaving tests aside for this sync.

OK.

> Since the exact same Org version that resides in Emacs is already
> tested from Org's repository, I do not see what is the additional
> value of having Org tests in Emacs.

I guess it would also be useful in order to detect changes in other parts
of Emacs that Org might depend on.

As Michael said, perhaps someone might be encouraged to port one
organizational method to the other.  OTOH, the initial change to the
core-format for tests might as well happen in the org repo first...

Rasmus

-- 
Er du tosset for noge' lårt!

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Rasmus <rasmus@gmx.us> writes:

> I guess it would also be useful in order to detect changes in other parts
> of Emacs that Org might depend on.

Indeed.

> As Michael said, perhaps someone might be encouraged to port one
> organizational method to the other.  OTOH, the initial change to the
> core-format for tests might as well happen in the org repo first...

Sure.  I've nothing against changes that could enhance the situation
here, first in Org's repo, then in Emacs, I'm just saying this is not
critical at this point IMO - and I know you don't disagree :)

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Michael Albinus]

Bastien Guerry <bzg@gnu.org> writes:

>> As Michael said, perhaps someone might be encouraged to port one
>> organizational method to the other.  OTOH, the initial change to the
>> core-format for tests might as well happen in the org repo first...
>
> Sure.  I've nothing against changes that could enhance the situation
> here, first in Org's repo, then in Emacs, I'm just saying this is not
> critical at this point IMO - and I know you don't disagree :)

Many Emacs developers won't work in the Org repo, they live in the Emacs
repo. That's all.

Best regards, Michael.

PS: They same is true for the Tramp repo. I know the situation ...

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Rasmus]

Bastien Guerry <bzg@gnu.org> writes:

>> As Michael said, perhaps someone might be encouraged to port one
>> organizational method to the other.  OTOH, the initial change to the
>> core-format for tests might as well happen in the org repo first...
>
> Sure.  I've nothing against changes that could enhance the situation
> here, first in Org's repo, then in Emacs, I'm just saying this is not
> critical at this point IMO - and I know you don't disagree :)

Well, for now I will focus on getting Org 9.0.8 into Emacs.  It has
already been too long and for that I apologize.

Perhaps I will find time to try to figure out how to convert tests after
that.  It seems test/lisp/calendar/todo-mode-tests.el might serve as an
inspiration as it also include external files...

Rasmus

-- 
Lasciate ogni speranza, voi che leggete questo.

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Eli Zaretskii]

> From: Bastien Guerry <bzg@gnu.org>
> Cc: Rasmus Pank Roulund <rasmus@gmx.us>,  johnw@gnu.org,  emacs-devel@gnu.org,  mvoteiza@udel.edu,  mail@nicolasgoaziou.fr,  kaushal.modi@gmail.com,  kyle@kyleam.com,  eliz@gnu.org
> Date: Wed, 21 Jun 2017 11:58:25 +0200
> 
> Since the exact same Org version that resides in Emacs is already
> tested from Org's repository, I do not see what is the additional
> value of having Org tests in Emacs.

The value is that Emacs end-users who build a new tarball could run
tests if they want to make sure their build is working correctly.

Whether this is a reason good enough to justify the effort of adding
the tests to the Emacs repository, is your call, of course.

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Eli Zaretskii <eliz@gnu.org> writes:

> The value is that Emacs end-users who build a new tarball could run
> tests if they want to make sure their build is working correctly.

Fair enough.

> Whether this is a reason good enough to justify the effort of adding
> the tests to the Emacs repository, is your call, of course.

It is a good reason to justify the effort of adding the test, but
let's not make it a blocker for this merge, we can do it later.

Thanks,

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

[-- Attachment #1: Type: text/plain, Size: 975 bytes --]

Hi Kyle and all,

For this particular case, I suggest (1) reverting the commit in Emacs
master branch, (2) discussing it on the orgmode list and (3) appling
it to Org master branch.

IIRC this is the first problematic commit we have in Emacs, so I don't
want to make a rule--let's just say that such important changes should
go upstream first.

As for sync'ing Org latest stable release (9.0.8) with Emacs master
branch, I'm ready to do it now (thanks Kyle and Kaushal for your work
on keeping Org ready for this!)

For Emacs maintainers, here is a document with an overview of critical
Org changes since Org >=8.3.

I speak under the control of Nicolas, who did the bulk of the work,
but to my knowledge, none of these six changes causes any trouble now.
Only the first two were problematic when they were first proposed, but
they are good changes, and users are happy with them now.

Once Eli and John sends the green light I'm fine to send a patch.

Thanks!

-- 
 Bastien

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: org-mode-critical-changes-19062017.org --]
[-- Type: text/x-org, Size: 13795 bytes --]

#+TITLE: Org critical incompatible changes between >=8.3 and <=9.0.8

* Overview
** Properties drawers syntax changes

: Properties drawers are now required to be located right after a
: headline and its planning line, when applicable.

This change has caused many questions when it happened, it is widely
accepted now and I fully support it.

** New syntax for export blocks

: Export blocks are explicitly marked as such at the syntax level to
: disambiguate their parsing from special blocks.  The new syntax is
: 
:  #+BEGIN_SRC org
:  ,#+BEGIN_EXPORT backend
:  ...
:  ,#+END_EXPORT
:  #+END_SRC
: 
:  instead of
: 
:  #+BEGIN_SRC org
:  ,#+BEGIN_backend
:  ...
:  ,#+END_backend
:  #+END_SRC

This change caused some headaches because it involves semi-manual
migration.  Below is `org-repair-export-blocks', a function to ease
the transition.

The rationale behind this change is to make the parser more robust.

** Old Babel header properties are no longer supported

: Using header arguments as property names is no longer possible.  As
: such, the following
: 
: #+BEGIN_EXAMPLE
: ,* Headline
: :PROPERTIES:
: :exports: code
: :var: a=1 b=2
: :var+: c=3
: :END:
: #+END_EXAMPLE
: 
: should be written instead
: 
: #+BEGIN_EXAMPLE
: ,* Headline
: :PROPERTIES:
: :header-args: :exports code
: :header-args: :var a=1 b=2
: :header-args+: :var c=3
: :END:
: #+END_EXAMPLE

This makes the parser more robust by clearly separating custom
property names from Babel header arguments.

** Footnotes : [1]-like constructs are not valid footnotes

Here again, Org parser is more robust with a dedicated syntax for
footnotes.  This change has not cause any trouble, since footnotes are
generally added through an Org command.

** Using "COMMENT" is now equivalent to commenting with "#"

: If you used "COMMENT" in headlines to prevent a subtree from being
: exported, you can still do it but all information within the subtree
: is now commented out, i.e. no #+OPTIONS line will be parsed or taken
: into account when exporting.
: 
: If you want to exclude a headline from export while using its contents
: for setting options, use =:noexport:= (see =org-export-exclude-tags=.)

Org used to allow both methods to prevent a subtree from being
exported but one method is enough, and this change also makes the
behavior of the COMMENT special keyword more consistent.

** =#+CATEGORY= keywords no longer apply partially to document

: It was possible to use several such keywords and have them apply to
: the text below until the next one, but strongly deprecated since Org
: 5.14 (2008).

This change brought no problem at all and is a welcome enhancement.

* Details

** Version 9.1

*** Incompatible changes

**** Variables relative to clocksum duration are obsolete

 ~org-time-clocksum-format~, ~org-time-clocksum-use-fractional~ and
 ~org-time-clocksum-fractional-format~ are obsolete.  If you changed
 them, consider modifying ~org-duration-format~ instead.

 Variable ~org-time-clocksum-use-effort-durations~ is also obsolete.
 Consider setting ~org-duration-units~ instead.

**** ~org-at-timestamp-p~ optional argument accepts different values

 See docustrings for the allowed values.  For backward compatibility,
 ~(org-at-timestamp-p t)~ is still supported, but should be updated
 accordingly.

**** ~org-capture-templates~ no longer accepts S-expressions as file names

 Since functions are allowed there, a straightforward way to migrate
 is to turn, e.g.,

 : (file (sexp))

 into

 : (file (lambda () (sexp)))

**** Removal of ~:sitemap-sans-extension~

 The publishing property is no longer recognized, as a consequence of
 changes to site-map generation.

 You can get the same functionality by setting ~:sitemap-format-entry~
 to the following

 #+BEGIN_SRC elisp
 (lambda (entry style project)
   (cond ((not (directory-name-p entry))
	  (format "[[file:%s][%s]]"
		  (file-name-sans-extension entry)
		  (org-publish-find-title entry project)))
	 ((eq style 'tree) (file-name-nondirectory (directory-file-name entry)))
	 (t entry)))
 #+END_SRC

**** Change signature for ~:sitemap-function~

 ~:sitemap-function~ now expects to be called with two arguments.  See
 ~org-publish-project-alist~ for details.

**** Change signature for some properties in ~org-list-to-generic~

 ~:istart~, ~:icount~, ~:iend~ and ~:isep~ now expect the type of the
 list as their first argument.

**** Change signature for ~org-get-repeater~

 The optional argument is now a string to extract the repeater from.
 See docstring for details.

** Version 9.0

*** Incompatible changes

**** Emacs 23 support has been dropped

 From now on, Org expects at least Emacs 24.3, although Emacs 24.4 or
 above is suggested.

**** XEmacs support has been dropped

 Incomplete compatibility layer with XEmacs has been removed.  If you
 want to take over maintainance of this compatibility, please contact
 our mailing list.

**** New syntax for export blocks

Export blocks are explicitly marked as such at the syntax level to
disambiguate their parsing from special blocks.  The new syntax is

 #+BEGIN_SRC org
 ,#+BEGIN_EXPORT backend
 ...
 ,#+END_EXPORT
 #+END_SRC

 instead of

 #+BEGIN_SRC org
 ,#+BEGIN_backend
 ...
 ,#+END_backend
 #+END_SRC

 As a consequence, =INCLUDE= keywords syntax is modified, e.g.,

 #+BEGIN_SRC org
 ,#+INCLUDE: "file.org" HTML
 #+END_SRC

 becomes

 #+BEGIN_SRC org
 ,#+INCLUDE: "file.org" export html
 #+END_SRC

 The following function repairs export blocks and =INCLUDE= keywords
 using previous syntax:

 #+BEGIN_SRC emacs-lisp
 (defun org-repair-export-blocks ()
   "Repair export blocks and INCLUDE keywords in current buffer."
   (interactive)
   (when (eq major-mode 'org-mode)
     (let ((case-fold-search t)
           (back-end-re (regexp-opt
                         '("HTML" "ASCII" "LATEX" "ODT" "MARKDOWN" "MD" "ORG"
                           "MAN" "BEAMER" "TEXINFO" "GROFF" "KOMA-LETTER")
                         t)))
       (org-with-wide-buffer
	(goto-char (point-min))
	(let ((block-re (concat "^[ \t]*#\\+BEGIN_" back-end-re)))
          (save-excursion
            (while (re-search-forward block-re nil t)
              (let ((element (save-match-data (org-element-at-point))))
		(when (eq (org-element-type element) 'special-block)
                  (save-excursion
                    (goto-char (org-element-property :end element))
                    (save-match-data (search-backward "_"))
                    (forward-char)
                    (insert "EXPORT")
                    (delete-region (point) (line-end-position)))
                  (replace-match "EXPORT \\1" nil nil nil 1))))))
	(let ((include-re
               (format "^[ \t]*#\\+INCLUDE: .*?%s[ \t]*$" back-end-re)))
          (while (re-search-forward include-re nil t)
            (let ((element (save-match-data (org-element-at-point))))
              (when (and (eq (org-element-type element) 'keyword)
                         (string= (org-element-property :key element) "INCLUDE"))
		(replace-match "EXPORT \\1" nil nil nil 1)))))))))
 #+END_SRC

 Moreover, ~:export-block~ keyword used in ~org-export-define-backend~ and
 ~org-export-define-derived-backend~ is no longer used and needs to be
 removed.

**** Footnotes

***** [1]-like constructs are not valid footnotes

 Using =[1]= as a footnote was already discouraged in the manual, since
 it introduced too many false-positives in many Org documents.  These
 constructs are now unsupported.

 If you used =[N]= in some of your documents, consider turning them into
 =[fn:N]=.

***** /Org Footnote/ library doesn't handle non-Org buffers

 Commands for footnotes in an Org document no longer try to do
 something in non-Org ones.  If you need to have footnotes there,
 consider using the =footnote.el= library, shipped with Emacs.

 In particular, ~org-footnote-tag-for-non-org-mode-files~ no longer
 exists.

**** ~org-file-apps~ no longer accepts S-expressions as commands

 The variable now accepts functions of two arguments instead of plain
 S-expressions.  Replacing a S-expresion with an appropriate function
 is straightforward.  For example

 : ("pdf" . (foo))

 becomes

 : ("pdf" . (lambda (file link) (foo)))

**** The ~{{{modification-time}}}~ macro can get time via =vc=

 The modification time will be determined via =vc.el= if the second
 argument is non-nil.  See the manual for details.

**** Preparation and completion functions in publishing projects change signature

 Preparation and completion functions are now called with an argument,
 which is the project property list.  It used to be dynamically scoped
 through the ~project-plist~ variable.

**** Old Babel header properties are no longer supported

 Using header arguments as property names is no longer possible.  As
 such, the following

 #+BEGIN_EXAMPLE
 ,* Headline
 :PROPERTIES:
 :exports: code
 :var: a=1 b=2
 :var+: c=3
 :END:
 #+END_EXAMPLE

 should be written instead

 #+BEGIN_EXAMPLE
 ,* Headline
 :PROPERTIES:
 :header-args: :exports code
 :header-args: :var a=1 b=2
 :header-args+: :var c=3
 :END:
 #+END_EXAMPLE

 Please note that, however, old properties were defined at the source
 block definition.  Current ones are defined where the block is called.

** Version 8.3

*** Incompatible changes

**** Properties drawers syntax changes

 Properties drawers are now required to be located right after a
 headline and its planning line, when applicable.

 It will break some documents as TODO states changes were sometimes
 logged before the property drawer.

 The following function will repair them:

 #+BEGIN_SRC emacs-lisp
 (defun org-repair-property-drawers ()
   "Fix properties drawers in current buffer.
 Ignore non Org buffers."
   (when (eq major-mode 'org-mode)
     (org-with-wide-buffer
      (goto-char (point-min))
      (let ((case-fold-search t)
            (inline-re (and (featurep 'org-inlinetask)
                            (concat (org-inlinetask-outline-regexp)
                                    "END[ \t]*$"))))
	(org-map-entries
         (lambda ()
           (unless (and inline-re (org-looking-at-p inline-re))
             (save-excursion
               (let ((end (save-excursion (outline-next-heading) (point))))
                 (forward-line)
                 (when (org-looking-at-p org-planning-line-re) (forward-line))
                 (when (and (< (point) end)
                            (not (org-looking-at-p org-property-drawer-re))
                            (save-excursion
                              (and (re-search-forward org-property-drawer-re end t)
                                   (eq (org-element-type
					(save-match-data (org-element-at-point)))
                                       'drawer))))
                   (insert (delete-and-extract-region
                            (match-beginning 0)
                            (min (1+ (match-end 0)) end)))
                   (unless (bolp) (insert "\n"))))))))))))
 #+END_SRC

**** Using "COMMENT" is now equivalent to commenting with "#"

 If you used "COMMENT" in headlines to prevent a subtree from being
 exported, you can still do it but all information within the subtree
 is now commented out, i.e. no #+OPTIONS line will be parsed or taken
 into account when exporting.

 If you want to exclude a headline from export while using its contents
 for setting options, use =:noexport:= (see =org-export-exclude-tags=.)

**** =#+CATEGORY= keywords no longer apply partially to document

 It was possible to use several such keywords and have them apply to
 the text below until the next one, but strongly deprecated since Org
 5.14 (2008).

 =#+CATEGORY= keywords are now global to the document.  You can use node
 properties to set category for a subtree, e.g.,

 #+BEGIN_SRC org
 ,* Headline
    :PROPERTIES:
    :CATEGORY: some category
    :END:
 #+END_SRC

**** New variable to control visibility when revealing a location

 ~org-show-following-heading~, ~org-show-siblings~, ~org-show-entry-below~
 and ~org-show-hierarchy-above~ no longer exist.  Instead, visibility is
 controlled through a single variable: ~org-show-context-detail~, which
 see.

**** Replace disputed keys again when reading a date

 ~org-replace-disputed-keys~ has been ignored when reading date since
 version 8.1, but the former behavior is restored again.

 Keybinding for reading date can be customized with a new variable
 ~org-read-date-minibuffer-local-map~.

**** No default title is provided when =TITLE= keyword is missing

 Skipping =TITLE= keyword no longer provides the current file name, or
 buffer name, as the title.  Instead, simply ignore the title.

**** Default bindings of =C-c C-n= and =C-c C-p= changed

 The key sequences =C-c C-n= and =C-c C-p= are now bound to
 ~org-next-visible-heading~ and ~org-previous-visible-heading~
 respectively, rather than the =outline-mode= versions of these
 functions.  The Org version of these functions skips over inline tasks
 (and even-level headlines when ~org-odd-levels-only~ is set).

**** ~org-element-context~ no longer return objects in keywords

 ~org-element-context~ used to return objects on some keywords, i.e.,
 =TITLE=, =DATE= and =AUTHOR=.  It now returns only the keyword.

**** ~org-timer-default-timer~ type changed from number to string

 If you have, in your configuration, something like =(setq
 org-timer-default-timer 10)= replace it with =(setq
 org-timer-default-timer "10")=.

**** Functions signature changes

 The following functions require an additional argument.  See their
 docstring for more information.

 - ~org-export-collect-footnote-definitions~
 - ~org-html-format-headline-function~
 - ~org-html-format-inlinetask-function~
 - ~org-latex-format-headline-function~
 - ~org-latex-format-inlinetask-function~
 - ~org-link-search~

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Kyle Meyer]

Bastien Guerry <bzg@gnu.org> writes:

> Hi Kyle and all,
>
> For this particular case, I suggest (1) reverting the commit in Emacs
> master branch, (2) discussing it on the orgmode list and (3) appling
> it to Org master branch.

Thanks, Bastien.  Sounds good to me.  Mark?

-- 
Kyle

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Mark Oteiza]

On 19/06/17 at 05:44pm, Kyle Meyer wrote:
> Bastien Guerry <bzg@gnu.org> writes:
> > For this particular case, I suggest (1) reverting the commit in Emacs
> > master branch, (2) discussing it on the orgmode list and (3) appling
> > it to Org master branch.
> 
> Thanks, Bastien.  Sounds good to me.  Mark?

If (1) this key shall no longer bound to a lambda, and (2) the
functionality I implemented in some capacity shall exist, then please
make it happen in a way that works for you.

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Hi Mark,

Mark Oteiza <mvoteiza@udel.edu> writes:

> On 19/06/17 at 05:44pm, Kyle Meyer wrote:
>> Bastien Guerry <bzg@gnu.org> writes:
>> > For this particular case, I suggest (1) reverting the commit in Emacs
>> > master branch, (2) discussing it on the orgmode list and (3) appling
>> > it to Org master branch.
>> 
>> Thanks, Bastien.  Sounds good to me.  Mark?
>
> If (1) this key shall no longer bound to a lambda, and (2) the
> functionality I implemented in some capacity shall exist, then please
> make it happen in a way that works for you.

Done, thanks a lot.  I've revert your commit and pushed it under your
name in Org's master branch.

-- 
 Bastien

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Eli Zaretskii]

> From: Bastien Guerry <bzg@gnu.org>
> Date: Mon, 19 Jun 2017 22:33:52 +0200
> Cc: Mark Oteiza <mvoteiza@udel.edu>, Emacs developers <emacs-devel@gnu.org>,
> 	Nicolas Goaziou <mail@nicolasgoaziou.fr>,
> 	Kaushal Modi <kaushal.modi@gmail.com>
> 
> As for sync'ing Org latest stable release (9.0.8) with Emacs master
> branch, I'm ready to do it now (thanks Kyle and Kaushal for your work
> on keeping Org ready for this!)
> 
> For Emacs maintainers, here is a document with an overview of critical
> Org changes since Org >=8.3.

Thanks.  Since this is Org's _stable_ release, which AFAIU you
recommend to all users, the decision to merge it with Emacs master is
really a no-brainer.  Any incompatibilities that this merge could
bring are already out in the open, and you still decided to recommend
this version to your users, right?  So I see no reasons whatsoever why
we would want to keep this out of the Emacs master branch.

Am I missing something?

Re: Commit 49c0ff2 "Don't bind org-agenda .."

[Bastien Guerry]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks.  Since this is Org's _stable_ release, which AFAIU you
> recommend to all users, the decision to merge it with Emacs master is
> really a no-brainer.  Any incompatibilities that this merge could
> bring are already out in the open, and you still decided to recommend
> this version to your users, right?  So I see no reasons whatsoever why
> we would want to keep this out of the Emacs master branch.
>
> Am I missing something?

No.  But John has been waiting (for ages) for the review I sent
yesterday, so I'd rather wait for his feedback.

John, just cc'ing you, in case you missed the thread.

-- 
 Bastien