next pretest

next pretest

[Glenn Morris]

Let's have the next pretest this Friday.
If you need more time, please ask.

(Slightly surprised to have seen no Org changes since April.)

Re: next pretest

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Mon, 11 Aug 2014 11:43:23 -0400
> 
> (Slightly surprised to have seen no Org changes since April.)

They save them for the last moment of the last pretest.

Re: next pretest

[Bastien]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Glenn Morris <rgm@gnu.org>
>> Date: Mon, 11 Aug 2014 11:43:23 -0400
>> 
>> (Slightly surprised to have seen no Org changes since April.)
>
> They save them for the last moment of the last pretest.

That may be true, since I don't know when is "the last pretest".

Merging regularily is less a priority than fixing bugs in the
stable branch of Org, which gets a lot of users.

Also, I would appreciate help on merging Org into Emacs.

Anyone?

PS: I'm right now on holiday until August, 17th.

-- 
 Bastien

Re: next pretest

[Eli Zaretskii]

> From: Bastien <bzg@gnu.org>
> Cc: Glenn Morris <rgm@gnu.org>,  emacs-devel@gnu.org
> Date: Tue, 12 Aug 2014 10:49:44 +0200
> 
> Also, I would appreciate help on merging Org into Emacs.

Please elaborate: what kind of help do you need?

Re: next pretest

[Bastien]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Bastien <bzg@gnu.org>
>> Cc: Glenn Morris <rgm@gnu.org>,  emacs-devel@gnu.org
>> Date: Tue, 12 Aug 2014 10:49:44 +0200
>> 
>> Also, I would appreciate help on merging Org into Emacs.
>
> Please elaborate: what kind of help do you need?

I need someone to take care of the merge.

-- 
 Bastien

Re: next pretest

[Glenn Morris]

Bastien wrote:

> That may be true, since I don't know when is "the last pretest".

It's several months after various announcements of increasing severity.
A week after the announcement of a release candidate.
Pretty standard procedure.

> Also, I would appreciate help on merging Org into Emacs.

I think I've suggested before you that you ask for help with this on the
Org mailing list, which seems to have an extensive development community.

Re: next pretest

[Bastien]

Glenn Morris <rgm@gnu.org> writes:

> Bastien wrote:
>
>> That may be true, since I don't know when is "the last pretest".
>
> It's several months after various announcements of increasing severity.
> A week after the announcement of a release candidate.
> Pretty standard procedure.

Well, it's a matter of timing.  Fixing bugs in the stable Org branch
seems preferrable than merging Org with Emacs, because I suspect the
stable Org branch has more testers than emacs-24.

>> Also, I would appreciate help on merging Org into Emacs.
>
> I think I've suggested before you that you ask for help with this on the
> Org mailing list, which seems to have an extensive development community.

I did that already, I will do it again.

-- 
 Bastien

Org merging [was Re: next pretest]

[Glenn Morris]

Bastien wrote:

> Well, it's a matter of timing. 

Org already gets special treatment. I don't see what else can be done.

>  Fixing bugs in the stable Org branch seems preferrable than merging
> Org with Emacs, because I suspect the stable Org branch has more
> testers than emacs-24.

I suggest that after this release we stop keeping Org in the Emacs
*repository*. Motivation:

1) It's usually months out-of-date.
2) Merging has been a frequent problem, wastes resources, is not a
priority for you, and no-one else seems interested in doing it.
3) As you say, probably it has relatively few users anyway. Anyone
capable of getting Emacs from VCS can get Org from elpa or wherever;
I suspect most who want to use it do.
4) Almost no-one changes Org in the Emacs repo.
5) Org bugs reported to bug-gnu-emacs get little attention. I don't
think I ever see any of the other Org people apart from you comment on
bug-gnu-emacs reports. The reports usually refer to versions of Org not
even in Emacs.
6) I know you have frequent problems with people wanting to use a
different Org version than the one in Emacs.
7) It doesn't get any benefit from Emacs pretests, owing to point 1.

We can still include it future release *tarballs*, if you like, using a
mechanism to be determined before the next release. But keeping it in
the *repository* seems pointless.

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

While talking about Org: I think the current way we have Org in GNU ELPA
is problematic and incompatible with the way GNU ELPA is working
nowadays.  It'd be better to have in GNU ELPA the *released* versions of
Org rather than the bleeding edge (which can be obtained from Org's own
ELPA repository).

So I suggest:
- we remove the hack that adds bleeding-edge versions of Org to GNU ELPA.
- we move the Org code from the Emacs's bzr repository to
  an `externals/org-mode' Git branch in the `elpa' repository.  At the same
  time, this will cause GNU ELPA to provide packages for Org releases
  with Org's version numbers.
- we work on the long-standing need to include GNU ELPA packages in
  the tarball.

Moving the code to the `elpa' branch means that there'll still be a need
to synchronize, but the "orgmode.org => elpa" direction can be done with
a simple "git pull", and the other direction is not supposed to have
very many changes to sync, so it should be much easier.


        Stefan

Re: Org merging [was Re: next pretest]

[Achim Gratz]

Stefan Monnier writes:
> While talking about Org: I think the current way we have Org in GNU ELPA
> is problematic and incompatible with the way GNU ELPA is working
> nowadays.  It'd be better to have in GNU ELPA the *released* versions of
> Org rather than the bleeding edge (which can be obtained from Org's own
> ELPA repository).

The version of Org in ELPA is "maint", not master.  The "maint" branch
tracks the released version plus any bugfixes between releases and it's
currently 39 commits ahead of release_8.2.7c (the version in ELPA
actually is 25 versions ahead since it's two days old).  The bleeding
edge version of Org is "master" and is currently 1342 commits ahead of
release_8.2.7c.

> So I suggest:
> - we remove the hack that adds bleeding-edge versions of Org to GNU
> ELPA.

To the best of my knowledge there is no such thing (yes I did look it up
again just now to be sure), Org does not provide packages for bleeding
edge at all; not on GNU ELPA and not on orgmode.org.  Bleeding edge is
primarily provided through Git, but there's a daily snapshot tarball of
the "master" tree for those who can't use Git.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

>> - we remove the hack that adds bleeding-edge versions of Org to GNU ELPA.
> To the best of my knowledge there is no such thing (yes I did look it up

That's just because your definition of "bleeding edge" is different.
I don't know the specifics, so I just used it to refer to the tarballs
that we get from orgmode.org, which have no release version but
a date instead.


        Stefan

Re: Org merging [was Re: next pretest]

[Achim Gratz]

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
> That's just because your definition of "bleeding edge" is different.

Preventively, I'm claiming Humtpy-Dumpty privileges.  I thought the
development projects get to decide what their bleeding edge is.

> I don't know the specifics, so I just used it to refer to the tarballs
> that we get from orgmode.org, which have no release version but
> a date instead.

If that's your problem then it can be changed easily enough.  The current
naming scheme is only in place to provide a continuing stream of increasing
version numbers to fit in with earlier releases of Org as a package. 
Changing that would require manual intervention from the user to force the
"downgrade" of Org once and we wanted to avoid that.

I'd still have to check if package.el deals correctly with a package name of

orgmode-8.2.7c-39-ge112f3c

though.


Regards,
Achim.

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

> If that's your problem then it can be changed easily enough.

That's not the only problem: org-mode is an oddball case nowadays, since
all other packages are "built locally from source".

> The current naming scheme is only in place to provide a continuing
> stream of increasing version numbers to fit in with earlier releases
> of Org as a package.  Changing that would require manual intervention
> from the user to force the "downgrade" of Org once and we wanted to
> avoid that.

I think for the user a version like org-8.2.7.<something>
would be much better than what we have now.

The manual intervention is indeed a problem, but I think we can't really
avoid it (tho package.el should probably be changed to recognize
version numbers of the form 20yymmdd and treat them as "smaller than
they appear").

> I'd still have to check if package.el deals correctly with a package name of
> orgmode-8.2.7c-39-ge112f3c

M-: (version-to-list "8.2.7c-39-ge112f3c")  ==>  error
M-: (version-to-list "8.2.7c.39")  ==>  error
M-: (version-to-list "8.2.7c")  ==>  (8 2 7 3)
M-: (version-to-list "8.2.7.39")  ==>  (8 2 7 39)

So this suggests version numbers shouldn't end in "c", in order to be
able to tack on a "bugfix counter" after it.


        Stefan

Re: Org merging [was Re: next pretest]

[Achim Gratz]

Stefan Monnier writes:
>> I'd still have to check if package.el deals correctly with a package name of
>> orgmode-8.2.7c-39-ge112f3c
>
> M-: (version-to-list "8.2.7c-39-ge112f3c")  ==>  error
> M-: (version-to-list "8.2.7c.39")  ==>  error
> M-: (version-to-list "8.2.7c")  ==>  (8 2 7 3)
> M-: (version-to-list "8.2.7.39")  ==>  (8 2 7 39)
>
> So this suggests version numbers shouldn't end in "c", in order to be
> able to tack on a "bugfix counter" after it.

In the end it's Bastiens decision how to name the releases and so far
he's preferred suffixes instead of going past 9.  But would you be
interested in an extended version-to-list to properly deal with the
general format of "git describe"?  It would probably help a lot with
getting rid of the nonsense that MELPA is doing to package versions.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf rackAttack V1.04R1:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

> But would you be interested in an extended version-to-list to properly
> deal with the general format of "git describe"?

Not sure it would help.  E.g. I think that allowing "-" in version names
could be problematic for package.el (tho I have removed many such
assumptions in the 24.4 version, I'd be surprised if there aren't still
some lurking).

But it should be pretty easy to pass git-describe's output through sed
to throw away the "-gXXXX" and to turn the "-" into a ".".


        Stefan

Re: Org merging [was Re: next pretest]

[Achim Gratz]

Stefan Monnier writes:
> Not sure it would help.  E.g. I think that allowing "-" in version names
> could be problematic for package.el (tho I have removed many such
> assumptions in the 24.4 version, I'd be surprised if there aren't still
> some lurking).

I'd call that a bug then…

> But it should be pretty easy to pass git-describe's output through sed
> to throw away the "-gXXXX" and to turn the "-" into a ".".

Yes you can do that, although I'd use the GNU make builtins rather than
requiring sed.  But the question was to support it as-is since this gets
ubiquitous by now and inventing yet another format for the same thing
surely creates more confusion than less.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

> I'd call that a bug then…

Partly.  The issue is simply that if you have a package
foo-bar-1.2-3.4.el, is that "foo-bar" version "1.2-3.4" or is it
"foo-bar-1.2" version "3.4"?

I'm pretty sure Emacs-24.[123] will end up burping on such package names.


        Stefan

Re: Org merging [was Re: next pretest]

[Bastien]

Hi Stefan,

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> While talking about Org: I think the current way we have Org in GNU ELPA
> is problematic and incompatible with the way GNU ELPA is working
> nowadays.  It'd be better to have in GNU ELPA the *released* versions of
> Org rather than the bleeding edge (which can be obtained from Org's own
> ELPA repository).
>
> So I suggest:
> - we remove the hack that adds bleeding-edge versions of Org to GNU ELPA.

Yes -- this can be done anytime.

> - we move the Org code from the Emacs's bzr repository to
>   an `externals/org-mode' Git branch in the `elpa' repository.  At the same
>   time, this will cause GNU ELPA to provide packages for Org releases
>   with Org's version numbers.

Okay.  Do you really want to move the bzr history there or just import
the Org Git stable branch into an externals/org-mode Git branch?

> - we work on the long-standing need to include GNU ELPA packages in
>   the tarball.
>
> Moving the code to the `elpa' branch means that there'll still be a need
> to synchronize, but the "orgmode.org => elpa" direction can be done with
> a simple "git pull", and the other direction is not supposed to have
> very many changes to sync, so it should be much easier.

Agreed.

FWIW, my plan for Org-mode development is (1) to let core development
happen on the GNU ELPA Git branch and (2) to have a separate Org ELPA
for contributions.  Both goals are separate and (2) has priority over
(1), which is not really necessary, but desirable IMHO.

-- 
 Bastien

Re: Org merging [was Re: next pretest]

[Achim Gratz]

Bastien <bzg <at> gnu.org> writes:
> > - we remove the hack that adds bleeding-edge versions of Org to GNU ELPA.
> 
> Yes -- this can be done anytime.

Not so fast.  To the best of my knowledge ELPA doesn't run make on the
projects there, so the Org package you're producing would be non-functional
-- which was the raison d'être for the current solution (or "hack"), IIRC.

> Okay.  Do you really want to move the bzr history there or just import
> the Org Git stable branch into an externals/org-mode Git branch?

You can't import the Git stable branch either, but you'd need to cut a
specific ELPA branch with contrib removed and mirror that.  Unless ELPA lets
us run make, that elpa branch would also need to contain auto-generated
files.  All things considered a direct link between ELPA and the development
repo isn't going to fly in the immediate future, IMHO.  An intermediate repo
on the Org server that we push to ELPA or ELPA pulls from would work nicely,
on the other hand.

> FWIW, my plan for Org-mode development is (1) to let core development
> happen on the GNU ELPA Git branch and (2) to have a separate Org ELPA
> for contributions.  Both goals are separate and (2) has priority over
> (1), which is not really necessary, but desirable IMHO.

Since this list wasn't involved in the discussion, let me state again that
such a split wouldn't quite work the way you envision it to do at the
moment.  For one, it would quite likely increase the maintenance workload
rather than reduce it.  Further discussion of this topic should be moved to
the orgmode mailing list, though.


Regards,
Achim.

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

>>>>> "Achim" == Achim Gratz <Stromeko@NexGo.DE> writes:
> Preventively, I'm claiming Humtpy-Dumpty privileges.  I thought the
> development projects get to decide what their bleeding edge is.

I'm more important, so I get to decide.  And by "important" I mean
exactly what I mean.  Don't put worth in my mouth, please!

> Not so fast.  To the best of my knowledge ELPA doesn't run make on the
> projects there, so the Org package you're producing would be non-functional
> -- which was the raison d'être for the current solution (or "hack"), IIRC.

Very good point.  The GNU ELPA infrastructure might need some beefing up
before it can produce a good package from the Org source code.  I don't
know the specific, but a few other packages require such extra "build"
stage (AUCTeX comes to mind, as does the recent discussion about Texinfo
docs), so it's something that needs to happen anyway.

> Since this list wasn't involved in the discussion, let me state again that
> such a split wouldn't quite work the way you envision it to do at the
> moment.  For one, it would quite likely increase the maintenance workload
> rather than reduce it.  Further discussion of this topic should be moved to
> the orgmode mailing list, though.

I'll happily bow out of this part of the discussion ;-)


        Stefan

Re: Org merging [was Re: next pretest]

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>>>> "Achim" == Achim Gratz <Stromeko@NexGo.DE> writes:
>> Preventively, I'm claiming Humtpy-Dumpty privileges.  I thought the
>> development projects get to decide what their bleeding edge is.
>
> I'm more important, so I get to decide.  And by "important" I mean
> exactly what I mean.  Don't put worth in my mouth, please!
>
>> Not so fast.  To the best of my knowledge ELPA doesn't run make on the
>> projects there, so the Org package you're producing would be non-functional
>> -- which was the raison d'être for the current solution (or "hack"), IIRC.
>
> Very good point.  The GNU ELPA infrastructure might need some beefing up
> before it can produce a good package from the Org source code.  I don't
> know the specific, but a few other packages require such extra "build"
> stage (AUCTeX comes to mind, as does the recent discussion about Texinfo
> docs), so it's something that needs to happen anyway.

As someone essentially unqualified to comment, I'll keep this short: how
about checking for an "elpa" target to make? Run it if it exists, do
nothing if it doesn't.

Done!

Eric

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

> As someone essentially unqualified to comment, I'll keep this short: how
> about checking for an "elpa" target to make? Run it if it exists, do
> nothing if it doesn't.

Of course, but that's not the issue.  The problems are rather:
- The machine on which the GNU ELPA archive is built is a small virtual
  machine, with fairly few software packages installed, currently.
  We can add more, of course.
- Since the GNU ELPA archive is auto-built nightly from the `elpa'
  branch and (inevitably) many people have write access to the `elpa'
  branch, there's a security issue if we run just arbitrary code from
  the `elpa' branch.


        Stefan

Re: Org merging [was Re: next pretest]

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> As someone essentially unqualified to comment, I'll keep this short: how
>> about checking for an "elpa" target to make? Run it if it exists, do
>> nothing if it doesn't.
>
> Of course, but that's not the issue.  The problems are rather:
> - The machine on which the GNU ELPA archive is built is a small virtual
>   machine, with fairly few software packages installed, currently.
>   We can add more, of course.
> - Since the GNU ELPA archive is auto-built nightly from the `elpa'
>   branch and (inevitably) many people have write access to the `elpa'
>   branch, there's a security issue if we run just arbitrary code from
>   the `elpa' branch.

Yup, went above my paygrade even faster than I'd expected. Oh well..

Re: Org merging [was Re: next pretest]

[Achim Gratz]

Stefan Monnier writes:
>> As someone essentially unqualified to comment, I'll keep this short: how
>> about checking for an "elpa" target to make? Run it if it exists, do
>> nothing if it doesn't.
>
> Of course, but that's not the issue.  The problems are rather:
> - The machine on which the GNU ELPA archive is built is a small virtual
>   machine, with fairly few software packages installed, currently.
>   We can add more, of course.

Org needs these programs for bootstrapping a release into a
package (the GNU versions if there are alternatives):

 emacs makeinfo make rm ln install tar echo find git

The use of make implies sh (any POSIXy shell will do) and makeinfo and
git have (extensive) further dependencies.  For the reference card and
manual you'd need PDFLaTeX as well.

> - Since the GNU ELPA archive is auto-built nightly from the `elpa'
>   branch and (inevitably) many people have write access to the `elpa'
>   branch, there's a security issue if we run just arbitrary code from
>   the `elpa' branch.

Those builds should run in their own VM obviously, but that may not be
easily set up.  Bouncing a work tree over Git with the info and autoload
files pre-built really isn't all that different than just handing you
the tarball to distribute, so I think that'd be a toss-up.  You just
have that build done on orgmode.org instead of elpa.gnu.org in this
case.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

> Org needs these programs for bootstrapping a release into a
> package (the GNU versions if there are alternatives):
>  emacs makeinfo make rm ln install tar echo find git

So, IIUC the main work is to build the manual from the Texinfo source?
That's a common need, indeed.

Currently, we do have all the other tools installed on that VM.

> The use of make implies sh (any POSIXy shell will do) and makeinfo and
> git have (extensive) further dependencies.  For the reference card and
> manual you'd need PDFLaTeX as well.

I tend to think these can be built by the end user instead of including
it in the tarball.


        Stefan

Re: Org merging [was Re: next pretest]

[Achim Gratz]

Stefan Monnier writes:
>> For the reference card and manual you'd need PDFLaTeX as well.
>
> I tend to think these can be built by the end user instead of including
> it in the tarball.

Try to think like a user, especially a Windows or OSX one.  You wouldn't
want to install TeXlive just for the refence card.  Yes there are other
venues to get these, but it seems anything they can't get with a single
click is forever out of reach.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

>>> For the reference card and manual you'd need PDFLaTeX as well.
>> I tend to think these can be built by the end user instead of including
>> it in the tarball.
> Try to think like a user, especially a Windows or OSX one.

I don't use either of those, but I think a Windows user wouldn't want
a PDF (nor PS) reference card in any case.


        Stefan

Re: Org merging [was Re: next pretest]

[Stefan Monnier]

>> - we remove the hack that adds bleeding-edge versions of Org to GNU ELPA.
> Yes -- this can be done anytime.

I think we do want to have Org in GNU ELPA, so I wouldn't do it "any
time" but only after Org is provided some other way in GNU ELPA.

> Okay.  Do you really want to move the bzr history there or just import
> the Org Git stable branch into an externals/org-mode Git branch?

I think the Git history is preferable (not only because it should
involve less work).

> FWIW, my plan for Org-mode development is (1) to let core development
> happen on the GNU ELPA Git branch

I guess it's not as simple as it sounds, but I'm all for it.

> and (2) to have a separate Org ELPA for contributions.

You mean for the org-plus-contrib?  IIUC it can't go into GNU ELPA for
copyright reasons, so indeed you'd still have to host it elsewhere.


        Stefan

Re: Org merging [was Re: next pretest]

[Bastien]

Glenn Morris <rgm@gnu.org> writes:

> We can still include it future release *tarballs*, if you like, using a
> mechanism to be determined before the next release. But keeping it in
> the *repository* seems pointless.

Agreed.  And yes, I'd still like Org to be packaged in Emacs tarballs,
I guess that's what many users expect now.

-- 
 Bastien

Re: next pretest

[Bastien]

Glenn Morris <rgm@gnu.org> writes:

> (Slightly surprised to have seen no Org changes since April.)

I took over maintainership (again) in May and I have been busy
fixing longstanding bugs since then.

-- 
 Bastien