When will we have our next release?

When will we have our next release?

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 2949 bytes --]

On Fri, 03 Jun 2011 14:39:13 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> Can we set a target date for 0.6 release?  So we'll all start feeling
> really bad if we miss it?

Frankly, I wouldn't mind doing strict time-based releases with something
like the following:

	* We schedule a release period (once per month?)

	* We schedule a "safety period" before the release (one week?)

	* At the beginning of the safety period, package up the head
          of the notmuch tree and upload to Debian experimental and
          anywhere else similar.

	* During the safety period we hopefully get wider testing than
          we normally have from just the git master branch.

	  If any bugs are found and fixed during this time we can create
	  a branch for them. New feature work can continue to land on
	  master.

	* At the end of the safety period we package up the same bits,
          or the new bits from the cherry-picked fixes on the branch,
          and upload them to Debian unstable and anywhere else similar.

I'd even be happy for someone else (other than me) to run that process.

That might be beneficial for a couple of reasons:

	* It would simply take one thing off my plate.

	* I'm inclined to do feature work myself---and when I start
          doing that again, I might get tempted to delay the release
          "just until I finish this next feature...".

I think that's the problematic state we've been in for the past several
months. Right after 0.5 I thought "I should do some database changes to
support indexing/searching of arbitrary headers and to capture complete
email addresses". I've not actually gotten around to doing that work
yet, but I was a bit stuck mentally in "the next release will have those
features" so there was never any threat of a release actually happening.

So switching to a more strictly time-based cycle can definitely help,
(so many other projects use time-based releases very successfully).

Now, in order to hand the release process over to someone else, we need
a really simple "just push this button" mechanism for the release. I
think we've got that pretty-well in place right now, with the large
exception of writing the NEWS file.

So the fix for that is to start rejecting patches that add features or
fix user-visible bugs (other than regressions since the past release)
without also updating the NEWS file. I'll commit myself to doing that
for my own bug fixes and features as well.

I also think it's possible for me to be successful as the release
manager as long as we decide on a schedule as a community and that way
you all keep me to it.

The current state of keep-coding-until-we-have-a-state-good-enough-to-
call-the-next-release does have the potential to keep running on
forever.

I'd be glad to get any feedback or additional ideas from anyone,

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: When will we have our next release?

[David Bremner]

Overall I think Carl's time based release proposal is a reasonable
plan. I think one problem we've been having is that we seem to have lost
track of 

    # Releases of notmuch have a two-digit version (0.1, 0.2, etc.). We
    # increment the second digit for each release and increment the first
    # digit when we reach particularly major milestones of usability.

In short, I think we are make too big of a deal out of releases. Looking
at the log between 0.5 and now, there are features enough to justify
several minor releases.

On Fri, 03 Jun 2011 15:56:42 -0700, Carl Worth <cworth@cworth.org> wrote:
> 
> Frankly, I wouldn't mind doing strict time-based releases with something
> like the following:
> 
> 	* We schedule a release period (once per month?)

I think every two months might be a bit more comfortable, but then
again, 1 month would keep us from "making a big deal out of releases."

> 	* We schedule a "safety period" before the release (one week?)
> 	* At the beginning of the safety period, package up the head
>           of the notmuch tree and upload to Debian experimental and
>           anywhere else similar.

Sure. I don't mind doing that part, at least for Debian.  I'm going to
try to do at roughly weekly uploads to Debian experimental. Hopefully
this will get some critical mass of users testing those versions.

d

Re: When will we have our next release?

[Xavier Maillard]

Hi,

On Sat, 04 Jun 2011 10:21:00 -0300, David Bremner <bremner@unb.ca> wrote:

> Overall I think Carl's time based release proposal is a reasonable
> plan. I think one problem we've been having is that we seem to have lost
> track of 
> 
>     # Releases of notmuch have a two-digit version (0.1, 0.2, etc.). We
>     # increment the second digit for each release and increment the first
>     # digit when we reach particularly major milestones of usability.
> 
> In short, I think we are make too big of a deal out of releases. Looking
> at the log between 0.5 and now, there are features enough to justify
> several minor releases.

Or even major ! Frankly, this project has grew up quite quickly and
features are implemented at a really good rythm. The sole problem is
that it is really hard to see how far we are from a release and what
exactly has been cooked up since latest release (from my point of view).
 
> On Fri, 03 Jun 2011 15:56:42 -0700, Carl Worth <cworth@cworth.org> wrote:
> > 
> > Frankly, I wouldn't mind doing strict time-based releases with something
> > like the following:
> > 
> > 	* We schedule a release period (once per month?)
> 
> I think every two months might be a bit more comfortable, but then
> again, 1 month would keep us from "making a big deal out of releases."

Best before choosing the frequency is probably to try doing this a few
times and be comfortable with the process. If after a few releases
-i.e. say 3- the more we can do is release every trimester so do it.

The process should be simple (and will be I guess) and the most
difficult part is probably to document every aspect of every changes in
the NEWS file (with eventually a good shaped manual ;)).

> > 	* We schedule a "safety period" before the release (one week?)
> > 	* At the beginning of the safety period, package up the head
> >           of the notmuch tree and upload to Debian experimental and
> >           anywhere else similar.
> 
> Sure. I don't mind doing that part, at least for Debian.  I'm going to
> try to do at roughly weekly uploads to Debian experimental. Hopefully
> this will get some critical mass of users testing those versions.

I know it is a bit off topic here but just a question: how will you deal
with dependencies ? I mean, when we need GMime vX.Y.Z and Debian has
already vX.V.W ?

/Xavier

Re: When will we have our next release?

[David Bremner]

On Sat, 04 Jun 2011 16:27:33 +0200, Xavier Maillard <xavier@maillard.im> wrote:

> I know it is a bit off topic here but just a question: how will you deal
> with dependencies ? I mean, when we need GMime vX.Y.Z and Debian has
> already vX.V.W ?

The same as every other Debian package? Try to persuade the maintainer
followed by uploading the new version ourselves if we run out of
patience.

To bring it back on topic (softof) I don't think anyone is suggesting
that the Debian packaging be any kind of gatekeeper in the release
process. I believe it is just that several of the people involved are
also involved with Debian, and tend to think Debian uploads as a natural
outcome of releasing software.

d

Re: When will we have our next release?

[Xavier Maillard]

Hi Carl,

On Fri, 03 Jun 2011 15:56:42 -0700, Carl Worth <cworth@cworth.org> wrote:
> On Fri, 03 Jun 2011 14:39:13 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > Can we set a target date for 0.6 release?  So we'll all start feeling
> > really bad if we miss it?
> 
> Frankly, I wouldn't mind doing strict time-based releases with something
> like the following:
> 
> 	* We schedule a release period (once per month?)

That sounds reasonable to me. You could even do it less frequently
-i.e. every 2 month.
 
> 	* At the beginning of the safety period, package up the head
>           of the notmuch tree and upload to Debian experimental and
>           anywhere else similar.

I like this idea. I already have slackware packages for notmuch and
that's the way I prefer testing notmuch since I am not really
comfortable with the git machinery (I wish I could do something for
notmuch though).
  
> So switching to a more strictly time-based cycle can definitely help,
> (so many other projects use time-based releases very successfully).

+1. THat's pretty frustating to contemplate all those patches/source
code sent to this list and not being able to "test" them :)
 
Whatever we choose, keep up the good work guys !

/Xavier

Re: When will we have our next release?

[Jameson Graef Rollins]

[-- Attachment #1: Type: text/plain, Size: 3366 bytes --]

On Fri, 03 Jun 2011 15:56:42 -0700, Carl Worth <cworth@cworth.org> wrote:
> Frankly, I wouldn't mind doing strict time-based releases with something
> like the following:

Hi, Carl.  I think this is a fine idea, and we (not you) can definitely
run this process.  I'm quite sure that at least bremner and I can
completely handle this together, and I'm sure we can get others to help.

But the mechanics of the actual release are not the problem.  The
problem is the current one-person bottleneck for all patches: your
review and merge of all patches into the notmuch/master branch.  This
would not necessarily be a problem if you could get to reviewing and
merging patches more frequently.  But as it is now, if there are no new
patches on notmuch/master for longer than the release period, there will
be nothing to package and upload.

This is *not* meant to be an indictment of you at all.  I know it's
incredibly hard to keep up with the incoming patch flow.  It takes a lot
of time and work to review every patch.  I also really like your
reviews.  They are incredibly thorough and insightful, and I always
learn from them.  Your intimate knowledge of the code base also means
that you can frequently come up with cleaner solutions to the proposed
patches (as with the reworked part handling).

However, the bottleneck presents a big problem when delays are
introduced.  We can't really do anything until you can get to the
review.  Furthermore, even if we do push ahead to put together a release
candidate branch (as we did with 0.6), if your review severely alters
patches early in that branch we have to do a lot of work to rework their
decedents (as we did with 0.6).  This leads to a lot of inefficiencies.

So we need to figure out a way to break the bottleneck.

I would really like to continue to get your review of patches.  I think
they're just too valuable.  So it would be really nice if one of the
solutions was a way to just "grease" the bottleneck, so to speak.  For
instance, if you could commit to reviewing just 1 patch series a week we
would be way ahead of where we have been.

Another thing that would help would be to delegate responsibility of
certain components to others, as you have with the python binding to
spaetz.  For instance, we have at least a couple of elisp experts
hanging around.  Maybe you could cede handling of all emacs patches to
someone like jkr or dme, and to felipe for vim, etc. (if they're willing
to take on those rolls).  That would help reduce your burden a bit.

We could also formalize some sort of tiered review system.  amdragon has
been doing a really good job of frequently providing good review of
patches on list.  I think that any proposed patch that gets a thumbs up
From someone like amdragon should immediately be elevated in your queue,
or just applied out-right.  If the review of others explicitly helped
get patches merged faster, I'm quite sure it would encourage more folks
to submit their reviews as well.

I would love to hear any other ideas people have on this front.

Notmuch is an incredible project, with an absolutely incredible
development community.  It's an absolute joy to work on.  If we can just
grease the wheels a little bit to get releases out the door a little
quicker, I think we'll all be a lot happier.

jamie.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: When will we have our next release?

[Sebastian Spaeth]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

On Tue, 07 Jun 2011 09:44:40 -0700, Jameson Graef Rollins wrote:

> I would love to hear any other ideas people have on this front.

I agree that delegation of bindings (and potentially the emacs code) is
a good thing. I believe both areas are separate enough to be
delegated. Perhaps have a similar tiered sytem with a different
"lieutenant" for the elisp code would be acceptable?

As for notmuch core, I do like the careful reviews and would prefer if
cworth still nodded off patch series, but a tiered review system, where
amdragon's nod off is enough to get patches at the top of cworth queue
would be great too.

> Notmuch is an incredible project, with an absolutely incredible
> development community.  It's an absolute joy to work on.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de> :)

spaetz

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: When will we have our next release?

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 5729 bytes --]

On Tue, 07 Jun 2011 09:44:40 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > Frankly, I wouldn't mind doing strict time-based releases with something
> > like the following:
> 
> Hi, Carl.  I think this is a fine idea, and we (not you) can definitely
> run this process.  I'm quite sure that at least bremner and I can
> completely handle this together, and I'm sure we can get others to
> help.

Excellent!

> But the mechanics of the actual release are not the problem.  The
> problem is the current one-person bottleneck for all patches:

This is a problem if master isn't moving, I agree. And that has been a
problem in the past.

More recently, master has been moving just fine, and I have proven to
get too caught up in "oh, just a few more bug fixes to merge..." to just
sit down and make a release. So that's what I want to fix now.

To that end, I've just nominated David Bremner as our release
manager. He's already been pushing packages of recent notmuch master to
Debian experimental, so he's effectively already in the role of choosing
particular code states that the "masses" can easily get their hands on.

I've asked him to just take over the release process from here and I've
pretty much left all decisions in his hands. He'll probably make a
separate branch for the release at some point, (in the primary
repository). I'll let him talk more about plans if he needs to.

> This is *not* meant to be an indictment of you at all.  I know it's
> incredibly hard to keep up with the incoming patch flow.  It takes a lot
> of time and work to review every patch.  I also really like your
> reviews.  They are incredibly thorough and insightful, and I always
> learn from them.

Thanks. That's very kind of you to say so.

> I would really like to continue to get your review of patches.  I think
> they're just too valuable.  So it would be really nice if one of the
> solutions was a way to just "grease" the bottleneck, so to speak.  For
> instance, if you could commit to reviewing just 1 patch series a week we
> would be way ahead of where we have been.

I can't really promise anything other than doing the best I can to stay
on top of things. Lately, I am at least using notmuch itself more
effectively to manage outstanding patches and this is helping I thinl.

> Another thing that would help would be to delegate responsibility of
> certain components to others, as you have with the python binding to
> spaetz.  For instance, we have at least a couple of elisp experts
> hanging around.  Maybe you could cede handling of all emacs patches to
> someone like jkr or dme, and to felipe for vim, etc. (if they're willing
> to take on those rolls).  That would help reduce your burden a bit.

Yes! For people that are already effectively maintaining isolated
portions of the code, I'm more than happy to delegate full editorial
control over those pieces, (and allow direct pushes, etc.). This has
actually already been happening with python and vim pieces. And I plan
to add some new mutt code soon with its own maintainer.

And the delegation of release management that I'm announcing here will
help too.

> We could also formalize some sort of tiered review system.  amdragon has
> been doing a really good job of frequently providing good review of
> patches on list.  I think that any proposed patch that gets a thumbs up
> From someone like amdragon should immediately be elevated in your queue,
> or just applied out-right.  If the review of others explicitly helped
> get patches merged faster, I'm quite sure it would encourage more folks
> to submit their reviews as well.

I would love to have more review from anyone. I don't think there's any
need to formalize anything.

Much of it can be as simple as watching things you've seen me do and
then doing them yourself. For example, there are a lot of recent patches
that I merged only after I wrote a test case for the bug fix. It's quite
a bottleneck for me to write new tests like that. If other people could
review patches and ask for test cases, or write test cases themselves,
then that would definitely relieve a burden on my part.

Similarly, I think the most regular coders on the project have come to
understand what I expect from commit messages. So that's something else
that's pretty easy for anyone to review.

So, yes, please help in the review process, everybody! I don't think I
have any particularly exclusive talent with regard to judging code to be
clean and in good taste. I definitely appreciate the good sense of taste
that many on this list are demonstrating regularly.

> Notmuch is an incredible project, with an absolutely incredible
> development community.  It's an absolute joy to work on.

I absolutely agree. I haven't taken the opportunity often enough to say
thank you to everyone who has contributed!

So, thanks!

It's such a wonderful thing to me to see so many good people working
hard and having fun to collectively make such a fun tool![*]

> If we can just grease the wheels a little bit to get releases out the
> door a little quicker, I think we'll all be a lot happier.

Hopefully, we're doing that. Thanks for all your efforts, Jamie. I
really appreciate them. And I'm happier at least!

-Carl

[*] For "fun tool" I always have to hesitate a bit—notmuch itself is
fun, but it's funny that email itself is often more annoying to me than
anything else. I suppose what makes notmuch fun is that it helps me more
quickly eliminate so much of the annoyance of email from my life so that
I can focus more on the things that I really want to.

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]