Merging release branch

Merging release branch

[Glenn Morris]

I've had enough of merging the release branch to master,
so someone else will have to do it from now on.

I use admin/automerge from cron with a wrapper that:
1) first checks the release branch passed make check
2) passes "-n 1" if the last merge was more than 1 week ago

The final straw was a triple failure in the automatic merge today due to:
an easily avoidable build breakage in the release branch
a non-repeatable check failure due to network-stream-tests timing out
  (such classes of test failure are common)
3 separate merge conflicts due to the same code being modified on both branches

Re: Merging release branch

[Lars Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

> The final straw was a triple failure in the automatic merge today due to:
> an easily avoidable build breakage in the release branch

Sorry about that -- I should have noticed that earlier...

> 3 separate merge conflicts due to the same code being modified on both branches

... and one of those were due to me trying to fix up that breakage in a
hurry and doing it on the wrong branch.  Sorry again.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging release branch

[Stefan Kangas]

Glenn Morris <rgm@gnu.org> writes:

> I've had enough of merging the release branch to master,
> so someone else will have to do it from now on.

If that is your decision, I respect it.  I can only salute and thank you
for carrying out this thankless task for so long.

> a non-repeatable check failure due to network-stream-tests timing out
>   (such classes of test failure are common)

IMO, we should aggressively mark such tests as :unstable when we detect
that they are, in fact, unstable.  Because they can and will break
automatic processes like merging, CI, etc.  That seems worse than
whatever regression the test is intended to avoid.

Another idea is that other people could proactively merge, if they
suspect that their changes might cause conflicts.  (I haven't done so,
for fear of stepping on your toes, but I would be happy to start.)

Re: Merging release branch

[Stefan Monnier]

> Another idea is that other people could proactively merge, if they
> suspect that their changes might cause conflicts.  (I haven't done so,
> for fear of stepping on your toes, but I would be happy to start.)

I think we should get in the habit of merging proactively, indeed.
If `emacs-28` was already merged before your commit(s), then it's
a small matter of going back to the main branch and doing `git merge
emacs-28` or `git merge -s ours emacs-28`.


        Stefan

Re: Merging release branch

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> If `emacs-28` was already merged before your commit(s), then it's
> a small matter of going back to the main branch and doing `git merge
> emacs-28` or `git merge -s ours emacs-28`.

That work flow sounds fine to me in general -- but what about those
changes to emacs-28 that shouldn't be merged?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging release branch

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> That work flow sounds fine to me in general -- but what about those
> changes to emacs-28 that shouldn't be merged?

I know there's other projects that don't do merging at all between
long-lived branches like this.

In Emacs, that would mean that things that are supposed to be
emacs-28-only are developed there, and are pushed as normal.  Any things
that are supposed to go to both master and emacs-28 are committed to
master, and then cherry-picked for emacs-28.  (Or cherry-picked the
other way, but that risks "losing" changes if you forget.)

Doing it that way would involve much fewer special rules (about commit
message formats) and less magic.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging release branch

[Daniel Martín]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
> I know there's other projects that don't do merging at all between
> long-lived branches like this.
>
> In Emacs, that would mean that things that are supposed to be
> emacs-28-only are developed there, and are pushed as normal.  Any things
> that are supposed to go to both master and emacs-28 are committed to
> master, and then cherry-picked for emacs-28.  (Or cherry-picked the
> other way, but that risks "losing" changes if you forget.)

An approach many of those projects follow is that every change is always
developed on master.  Only if a change is considered safe for the
release, it is cherry-picked into the release branch.  Always developing
on master greatly reduces the risk you mentioned about losing changes.

But this workflow is not exent of problems, though, because at the time
you cherry-pick it's possible that the change doesn't apply cleanly to
the emacs-28 branch.  In that case, you'll need to adapt the patch, and
possibly a maintainer would potentially need to review it again to see
if it's still safe enough for the release branch.

The main benefit this approach has is that there is not a single person
doing all the merges periodically, fixing conflicts affecting code he
has not written.  The release branch is never merged back and that means
there's no need to answer the non-intuitive question "should this commit
NOT be merged into master?", for example.  The merging work is
distributed across the people that cherry-pick their commits if they are
deemed safe for the next release.

My two cents.

Re: Merging release branch

[Lars Ingebrigtsen]

Daniel Martín <mardani29@yahoo.es> writes:

> But this workflow is not exent of problems, though, because at the time
> you cherry-pick it's possible that the change doesn't apply cleanly to
> the emacs-28 branch.  In that case, you'll need to adapt the patch, and
> possibly a maintainer would potentially need to review it again to see
> if it's still safe enough for the release branch.

Well, we depend upon people making that decision already -- when they're
putting things into emacs-28 or master.  

> The merging work is distributed across the people that cherry-pick
> their commits if they are deemed safe for the next release.

That's one of the things I find really attractive about the
cherry-picking work flow: Offloading the (boring) merging work on Glenn
has never felt very fair -- the person doing the commit should also do
the merges (not least because they're the ones that know best what the
change is meant to achieve).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging release branch

[Daniel Martín]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Daniel Martín <mardani29@yahoo.es> writes:
>
>> But this workflow is not exent of problems, though, because at the time
>> you cherry-pick it's possible that the change doesn't apply cleanly to
>> the emacs-28 branch.  In that case, you'll need to adapt the patch, and
>> possibly a maintainer would potentially need to review it again to see
>> if it's still safe enough for the release branch.
>
> Well, we depend upon people making that decision already -- when they're
> putting things into emacs-28 or master.

TBH, I don't think there's a "silver bullet" workflow, it depends on the
project, the participants, etc.  I presented some downsides of mine as
well, so you can decide if the trade-offs would make it less
time-consuming than the current one.  I suspect some of those trade-offs
are "invisible" until the workflow is put into practice and given some
time.

Re: Merging release branch

[Tassilo Horn]

Daniel Martín <mardani29@yahoo.es> writes:

> TBH, I don't think there's a "silver bullet" workflow, it depends on
> the project, the participants, etc.  I presented some downsides of
> mine as well, so you can decide if the trade-offs would make it less
> time-consuming than the current one.  I suspect some of those
> trade-offs are "invisible" until the workflow is put into practice and
> given some time.

That's true.  As an anecdote: we've tried a cherry-pick-based workflow
at my dayjob for some period.  Basically, (for each released and still
supported version) we've had a "pending", and a "release" branch.
Developers committed their fixes in the "pending" branch where the QA
would test them.  If the test was successful, the commit(s) where
cherry-picked into the "release" branch.

I can report that it didn't work out well.  Things that happend quite
often were:

- Some ticket had N commits but only N-M (M > 0) commits were
  cherry-picked.  And once you notice something's wrong, you have to go
  and diff diffs to find out which one is missing since the question has
  turned from "is commit 18a9ef4 reachable from this branch's HEAD" to
  "is there some commit with the same message and same diff"?
- The compile/CI broke because the cherry-picked commit X required stuff
  of commit Y which has not yet been cherry-picked because Y's test took
  longer than X's (or X has been prioritized over Y).
- Same as the last bullet point but worse: the compile still worked but
  the runtime behavior was wrong.

So in general, my gut feeling is that merging is preferable over
cherry-picking.  Even more so if the time between initial commit and
cherry-pick can become longer.

A policy which makes sense would be (as Stefan M. suggested, I think):
if you commit in emacs-<major>, immediately do a merge into master
afterwards (with -s ours if the fix is not appropriate for master).

Bye,
Tassilo

Re: Merging release branch

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Fri, 29 Oct 2021 20:52:22 +0200
> Cc: Glenn Morris <rgm@gnu.org>, Stefan Kangas <stefan@marxist.se>,
>  emacs-devel@gnu.org
> 
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
> > That work flow sounds fine to me in general -- but what about those
> > changes to emacs-28 that shouldn't be merged?
> 
> I know there's other projects that don't do merging at all between
> long-lived branches like this.
> 
> In Emacs, that would mean that things that are supposed to be
> emacs-28-only are developed there, and are pushed as normal.  Any things
> that are supposed to go to both master and emacs-28 are committed to
> master, and then cherry-picked for emacs-28.  (Or cherry-picked the
> other way, but that risks "losing" changes if you forget.)
> 
> Doing it that way would involve much fewer special rules (about commit
> message formats) and less magic.

Who'd do the cherry-picking in this scenario?  And why do you assume
people won't forget doing that, like they forget to merge and/or mark
the changes "not to be merged to master"?

I fear that what you propose will completely demote the release branch
to the status of second-rate citizen, because it then becomes a
burden.  That'd be a regression.  Cherry-picking makes much more sense
when branches are short-lived, which is not the case here.

Re: Merging release branch

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Who'd do the cherry-picking in this scenario?  And why do you assume
> people won't forget doing that, like they forget to merge and/or mark
> the changes "not to be merged to master"?

The people who commit things to master would also be tasked with
deciding whether to cherry-pick things for the release branch.  And,
yes, they'll forget to do this as they forget to put "don't merge" into
the commit messages, but there's one major advantage to the
cherry-picking here: Even if they forget, it's not then too late (as it
is with the commit message thing).  They can just cherry-pick the next
day when somebody reminds them.

> I fear that what you propose will completely demote the release branch
> to the status of second-rate citizen, because it then becomes a
> burden.  That'd be a regression. 

Speaking for myself, a cherry-picking work flow would be less work, not
more work in general, because I normally use an Emacs from the master
branch.  Being able to apply commits to master would be faster for me,
and cherry-picking to the release branch can be largely automated.

And I don't see any demotion of status here.  If anything, I see an
elevation -- we're deeming that branch so important that we're being
careful with it.

> Cherry-picking makes much more sense when branches are short-lived,
> which is not the case here.

I think it's rather the opposite?  The projects I'm familiar with that
use cherry-picking have very long-lived release branches (several of
them), and they keep on cherry-picking bug/documention fixes from the
trunk for years and years.  (Which we don't do, because of our
merge-from-release strategy.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging release branch

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: monnier@iro.umontreal.ca,  rgm@gnu.org,  stefan@marxist.se,
>   emacs-devel@gnu.org
> Date: Sat, 30 Oct 2021 13:51:59 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Who'd do the cherry-picking in this scenario?  And why do you assume
> > people won't forget doing that, like they forget to merge and/or mark
> > the changes "not to be merged to master"?
> 
> The people who commit things to master would also be tasked with
> deciding whether to cherry-pick things for the release branch.

That assumes they will want to, and will do a good job.

> > I fear that what you propose will completely demote the release branch
> > to the status of second-rate citizen, because it then becomes a
> > burden.  That'd be a regression. 
> 
> Speaking for myself, a cherry-picking work flow would be less work, not
> more work in general, because I normally use an Emacs from the master
> branch.

Then it's good for you, but not for me: when a release is in progress,
I work mainly on the release branch.

And of course, cherry-picking doesn't remove merge conflicts, they
will still need to be resolved.

We've been using the current workflow for years without any major
problems.  Changing that now radically doesn't make sense to me.  It
will most probably need several procedures to be modified that were
stable for several releases, and I don't see the gains which would
justify that.  We have enough real work on our hands.  So please let's
not do that.

Re: Merging release branch

[Dmitry Gutov]

On 30.10.2021 15:11, Eli Zaretskii wrote:
>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Cc: monnier@iro.umontreal.ca,  rgm@gnu.org,  stefan@marxist.se,
>>    emacs-devel@gnu.org
>> Date: Sat, 30 Oct 2021 13:51:59 +0200
>>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>>> Who'd do the cherry-picking in this scenario?  And why do you assume
>>> people won't forget doing that, like they forget to merge and/or mark
>>> the changes "not to be merged to master"?
>>
>> The people who commit things to master would also be tasked with
>> deciding whether to cherry-pick things for the release branch.
> 
> That assumes they will want to, and will do a good job.

People seem to generally want to push changes to the release branch. If 
only to get them to the users sooner.

>>> I fear that what you propose will completely demote the release branch
>>> to the status of second-rate citizen, because it then becomes a
>>> burden.  That'd be a regression.
>>
>> Speaking for myself, a cherry-picking work flow would be less work, not
>> more work in general, because I normally use an Emacs from the master
>> branch.
> 
> Then it's good for you, but not for me: when a release is in progress,
> I work mainly on the release branch.
> 
> And of course, cherry-picking doesn't remove merge conflicts, they
> will still need to be resolved.

By the author of the change, that's the point.

So I also think it might justify the additional overhead of having to 
cherry-pick every change.

> We've been using the current workflow for years without any major
> problems.  Changing that now radically doesn't make sense to me.  It
> will most probably need several procedures to be modified that were
> stable for several releases, and I don't see the gains which would
> justify that.  We have enough real work on our hands.  So please let's
> not do that.

Do we have a volunteer who will take over from Glenn?

Re: Merging release branch

[Eli Zaretskii]

> Cc: rgm@gnu.org, stefan@marxist.se, monnier@iro.umontreal.ca,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 30 Oct 2021 15:30:30 +0300
> 
> > And of course, cherry-picking doesn't remove merge conflicts, they
> > will still need to be resolved.
> 
> By the author of the change, that's the point.
> 
> So I also think it might justify the additional overhead of having to 
> cherry-pick every change.

Many authors don't have write access, though.

> > We've been using the current workflow for years without any major
> > problems.  Changing that now radically doesn't make sense to me.  It
> > will most probably need several procedures to be modified that were
> > stable for several releases, and I don't see the gains which would
> > justify that.  We have enough real work on our hands.  So please let's
> > not do that.
> 
> Do we have a volunteer who will take over from Glenn?

I hope we will; if not, then the usual suspect(s) will have to do it.

Re: Merging release branch

[dick]

> I hope we will; if not, then the usual suspect(s) will have to do it.

Cue Borscht Belt rejoinder "What am I, chopped liver?"

https://lists.gnu.org/archive/html/emacs-devel/2021-10/msg02270.html

Re: Merging release branch

[Dmitry Gutov]

On 30.10.2021 15:32, Eli Zaretskii wrote:
>> Cc:rgm@gnu.org,stefan@marxist.se,monnier@iro.umontreal.ca,
>>   emacs-devel@gnu.org
>> From: Dmitry Gutov<dgutov@yandex.ru>
>> Date: Sat, 30 Oct 2021 15:30:30 +0300
>>
>>> And of course, cherry-picking doesn't remove merge conflicts, they
>>> will still need to be resolved.
>> By the author of the change, that's the point.
>>
>> So I also think it might justify the additional overhead of having to
>> cherry-pick every change.
> Many authors don't have write access, though.
> 

The responsibility with fall on those who install the change, to no 
one's surprise.

Re: Merging release branch

[Eli Zaretskii]

> Cc: larsi@gnus.org, rgm@gnu.org, stefan@marxist.se, monnier@iro.umontreal.ca,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 31 Oct 2021 14:38:26 +0300
> 
> On 30.10.2021 15:32, Eli Zaretskii wrote:
> >> Cc:rgm@gnu.org,stefan@marxist.se,monnier@iro.umontreal.ca,
> >>   emacs-devel@gnu.org
> >> From: Dmitry Gutov<dgutov@yandex.ru>
> >> Date: Sat, 30 Oct 2021 15:30:30 +0300
> >>
> >>> And of course, cherry-picking doesn't remove merge conflicts, they
> >>> will still need to be resolved.
> >> By the author of the change, that's the point.
> >>
> >> So I also think it might justify the additional overhead of having to
> >> cherry-pick every change.
> > Many authors don't have write access, though.
> > 
> 
> The responsibility with fall on those who install the change, to no 
> one's surprise.

My point is that in those cases resolving merge conflicts won't be
easier.

Re: Merging release branch

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 30 Oct 2021 15:30:30 +0300
> Cc: rgm@gnu.org, stefan@marxist.se, monnier@iro.umontreal.ca,
>  emacs-devel@gnu.org
> 
> > And of course, cherry-picking doesn't remove merge conflicts, they
> > will still need to be resolved.
> 
> By the author of the change, that's the point.

Btw, if this is what will make the situation better, we could merge ti
master immediately after pushing to the release branch, by the same
committer.

Re: Merging release branch

[Dmitry Gutov]

On 30.10.2021 19:33, Eli Zaretskii wrote:
>> From: Dmitry Gutov<dgutov@yandex.ru>
>> Date: Sat, 30 Oct 2021 15:30:30 +0300
>> Cc:rgm@gnu.org,stefan@marxist.se,monnier@iro.umontreal.ca,
>>   emacs-devel@gnu.org
>>
>>> And of course, cherry-picking doesn't remove merge conflicts, they
>>> will still need to be resolved.
>> By the author of the change, that's the point.
> Btw, if this is what will make the situation better, we could merge ti
> master immediately after pushing to the release branch, by the same
> committer.

We can try making it a rule.

I think it has always been good form (esp. after making conflicting 
changes), but people are a little afraid on gitmerge.el.

Re: Merging release branch

[Eli Zaretskii]

> Cc: larsi@gnus.org, rgm@gnu.org, stefan@marxist.se, monnier@iro.umontreal.ca,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 31 Oct 2021 00:36:51 +0300
> 
> >> By the author of the change, that's the point.
> > Btw, if this is what will make the situation better, we could merge ti
> > master immediately after pushing to the release branch, by the same
> > committer.
> 
> We can try making it a rule.
> 
> I think it has always been good form (esp. after making conflicting 
> changes), but people are a little afraid on gitmerge.el.

I see no reason to be afraid of gitmerge.el.  If there are no
conflicts (and most of the time there aren't), it "just works".  If
there are conflicts, then their resolution is outside of gitmerge.el's
scope anyway.

I guess those people should try using gitmerge.el once or twice in
their local clones, to convince themselves that there's no real
problem here.

Re: Merging release branch

[Gregory Heytings]

>
> Btw, if this is what will make the situation better, we could merge ti 
> master immediately after pushing to the release branch, by the same 
> committer.
>

This could be automated with a server-side hook, whose execution would 
depend on some tag in the commit message which would indicate whether the 
commit should or should not be merged into the trunk.

Re: Merging release branch

[Dmitry Gutov]

On 31.10.2021 02:14, Gregory Heytings wrote:
>>
>> Btw, if this is what will make the situation better, we could merge ti 
>> master immediately after pushing to the release branch, by the same 
>> committer.
>>
> 
> This could be automated with a server-side hook, whose execution would 
> depend on some tag in the commit message which would indicate whether 
> the commit should or should not be merged into the trunk.

If we had a functional and blessed CI service, one of the checks could 
verify that the release branch is merge-able to master (using 'git 
merge' or even 'gitmerge.el').

And if not, ask the author of the corresponding push to do the merge.

Re: Merging release branch

[Eli Zaretskii]

> Date: Sat, 30 Oct 2021 23:14:00 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: Dmitry Gutov <dgutov@yandex.ru>, rgm@gnu.org, larsi@gnus.org, 
>     stefan@marxist.se, monnier@iro.umontreal.ca, emacs-devel@gnu.org
> 
> > Btw, if this is what will make the situation better, we could merge ti 
> > master immediately after pushing to the release branch, by the same 
> > committer.
> 
> This could be automated with a server-side hook, whose execution would 
> depend on some tag in the commit message which would indicate whether the 
> commit should or should not be merged into the trunk.

The problematic part of merging is when there are conflict, and that
cannot be done by the server.  If there are no conflicts, using
gitmerge.el is so simple it doesn't deserve any discussion.

Re: Merging release branch

[Gregory Heytings]

>>> Btw, if this is what will make the situation better, we could merge ti 
>>> master immediately after pushing to the release branch, by the same 
>>> committer.
>>
>> This could be automated with a server-side hook, whose execution would 
>> depend on some tag in the commit message which would indicate whether 
>> the commit should or should not be merged into the trunk.
>
> The problematic part of merging is when there are conflict, and that 
> cannot be done by the server.  If there are no conflicts, using 
> gitmerge.el is so simple it doesn't deserve any discussion.
>

It's simple, but IMO everything that can be automated should be automated. 
What I have in mind is this:

if push is not on <release branch> return
if tag is missing raise an error and return
if tag is "escape" do nothing (in case something must be done without running that server-hook script) and return
if tag is "do not merge" do a git merge -s ours <release branch> and return (this cannot fail)
if tag is "merge" try a git merge <release branch>; if that fails (which as you said happens rarely), print a message to ask the committer to use gitmerge.el; return

Re: Merging release branch

[Dmitry Gutov]

On 31.10.2021 10:20, Eli Zaretskii wrote:
>> Date: Sat, 30 Oct 2021 23:14:00 +0000
>> From: Gregory Heytings<gregory@heytings.org>
>> cc: Dmitry Gutov<dgutov@yandex.ru>,rgm@gnu.org,larsi@gnus.org,
>>      stefan@marxist.se,monnier@iro.umontreal.ca,emacs-devel@gnu.org
>>
>>> Btw, if this is what will make the situation better, we could merge ti
>>> master immediately after pushing to the release branch, by the same
>>> committer.
>> This could be automated with a server-side hook, whose execution would
>> depend on some tag in the commit message which would indicate whether the
>> commit should or should not be merged into the trunk.
> The problematic part of merging is when there are conflict, and that
> cannot be done by the server.  If there are no conflicts, using
> gitmerge.el is so simple it doesn't deserve any discussion.

A server-side check could be used to do the check "lazily" and only do 
one merge commit per day (or week, etc), rather than filling the master 
history with merge commits (where their ration to "real" commits could 
be approaching 100% if we ask everyone to merge right away).

Re: Merging release branch

[Gregory Heytings]

>
> A server-side check could be used to do the check "lazily" and only do 
> one merge commit per day (or week, etc), rather than filling the master 
> history with merge commits (where their ration to "real" commits could 
> be approaching 100% if we ask everyone to merge right away).
>

And why would that be a problem?  It's a commit to two branches, one 
branch has a merge commit for each commit on the other branch.

Re: Merging release branch

[Dmitry Gutov]

On 31.10.2021 15:58, Gregory Heytings wrote:
> 
>>
>> A server-side check could be used to do the check "lazily" and only do 
>> one merge commit per day (or week, etc), rather than filling the 
>> master history with merge commits (where their ration to "real" 
>> commits could be approaching 100% if we ask everyone to merge right 
>> away).
>>
> 
> And why would that be a problem?  It's a commit to two branches, one 
> branch has a merge commit for each commit on the other branch.

More merge commits = more complex history = more effort one has to spent 
(sometimes) when reading it (busier 'git log --graph').

It's not a grave problem, but best to be avoided when we can.

Re: Merging release branch

[Lars Ingebrigtsen]

Dmitry Gutov <dgutov@yandex.ru> writes:

> A server-side check could be used to do the check "lazily" and only do
> one merge commit per day (or week, etc), rather than filling the
> master history with merge commits (where their ration to "real"
> commits could be approaching 100% if we ask everyone to merge right
> away).

This reminds me -- another advantage to the cherry-picking approach is
that the "master" VC history remains clean and easy to understand, while
doing all these merges makes it full of noise.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging release branch

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sun, 31 Oct 2021 15:46:21 +0100
> Cc: rgm@gnu.org, stefan@marxist.se, emacs-devel@gnu.org,
>  Gregory Heytings <gregory@heytings.org>, monnier@iro.umontreal.ca,
>  Eli Zaretskii <eliz@gnu.org>
> 
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
> > A server-side check could be used to do the check "lazily" and only do
> > one merge commit per day (or week, etc), rather than filling the
> > master history with merge commits (where their ration to "real"
> > commits could be approaching 100% if we ask everyone to merge right
> > away).
> 
> This reminds me -- another advantage to the cherry-picking approach is
> that the "master" VC history remains clean and easy to understand, while
> doing all these merges makes it full of noise.

We've discussed this long ago and decided we didn't care.  The master
history cannot be clean anyway, because when there's a race between
two committers pushing at the same time, one of them will look like a
merge from a branch.

Re: Merging release branch

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> We've discussed this long ago and decided we didn't care.  The master
> history cannot be clean anyway, because when there's a race between
> two committers pushing at the same time, one of them will look like a
> merge from a branch.

Not if the committers rebasing (which I do), but we've already been over
that.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Dealing with merge noise (was: Merging release branch)

[Kévin Le Gouguec]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> We've discussed this long ago and decided we didn't care.  The master
>> history cannot be clean anyway, because when there's a race between
>> two committers pushing at the same time, one of them will look like a
>> merge from a branch.
>
> Not if the committers rebasing (which I do), but we've already been over
> that.  :-)

(Not sure if this was suggested in past discussions, but maybe sticking
"--no-merges" in vc-git-log-switches can reduce the pain somewhat?)

Re: Dealing with merge noise (was: Merging release branch)

[Stefan Kangas]

Kévin Le Gouguec <kevin.legouguec@gmail.com> writes:

> (Not sure if this was suggested in past discussions, but maybe sticking
> "--no-merges" in vc-git-log-switches can reduce the pain somewhat?)

You still want to know when emacs-28 was merged into master though.

The better solution is for everyone to just use "rebase" when they want
to push their own local work to master, and "merge" when merging from
emacs-28 or a feature branch.

Re: Dealing with merge noise (was: Merging release branch)

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Sun, 31 Oct 2021 11:19:43 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> 
> Kévin Le Gouguec <kevin.legouguec@gmail.com> writes:
> 
> > (Not sure if this was suggested in past discussions, but maybe sticking
> > "--no-merges" in vc-git-log-switches can reduce the pain somewhat?)
> 
> You still want to know when emacs-28 was merged into master though.
> 
> The better solution is for everyone to just use "rebase" when they want
> to push their own local work to master, and "merge" when merging from
> emacs-28 or a feature branch.

We have decided long ago that we won't impose any restrictions on the
workflows people use, and allow both merges and rebases.  Whether or
not to rebase is a matter of personal preferences, many times
associated with strong feelings, and there's no need for us to bring
the disagreements about that into the project.  Allowing both styles
worked well for us since we switched to Git, so I see no reason to
deviate from that now.

Re: Merging release branch

[Dmitry Gutov]

On 31.10.2021 17:46, Lars Ingebrigtsen wrote:
> This reminds me -- another advantage to the cherry-picking approach is
> that the "master" VC history remains clean and easy to understand, while
> doing all these merges makes it full of noise.

Good point, actually. While cherry-picking is an inherently "dirty" 
activity, neither the release branch nor the master branch will carry 
duplicate commits, so they aren't carried over to the future version.

Re: Merging release branch

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> The people who commit things to master would also be tasked with
>> deciding whether to cherry-pick things for the release branch.
>
> That assumes they will want to, and will do a good job.

That's an assumption that doesn't change -- we assume that now, and we
continue to assume so in the future.

>> Speaking for myself, a cherry-picking work flow would be less work, not
>> more work in general, because I normally use an Emacs from the master
>> branch.
>
> Then it's good for you, but not for me: when a release is in progress,
> I work mainly on the release branch.

And you can continue to do so -- and cherry-pick commits to master.

> And of course, cherry-picking doesn't remove merge conflicts, they
> will still need to be resolved.

So that's neither more than less work than the merge based work flow --
but with the advantage that you're resolving one pick at a time instead
of the more overwhelming thing that can happen when merging and you get
conflicts from several commits at once.

> We've been using the current workflow for years without any major
> problems.  Changing that now radically doesn't make sense to me.  It
> will most probably need several procedures to be modified that were
> stable for several releases, and I don't see the gains which would
> justify that.  We have enough real work on our hands.  So please let's
> not do that.

What's changed is that Glenn is (understandably) no longer willing to do
the merges, so something has to change.  (Unless somebody else picks up
the slack, and we can continue as before.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging release branch

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: monnier@iro.umontreal.ca,  rgm@gnu.org,  stefan@marxist.se,
>   emacs-devel@gnu.org
> Date: Sat, 30 Oct 2021 14:31:36 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The people who commit things to master would also be tasked with
> >> deciding whether to cherry-pick things for the release branch.
> >
> > That assumes they will want to, and will do a good job.
> 
> That's an assumption that doesn't change -- we assume that now, and we
> continue to assume so in the future.

No, until now committers didn't have to merge to master, they just
committed to a single branch.

> >> Speaking for myself, a cherry-picking work flow would be less work, not
> >> more work in general, because I normally use an Emacs from the master
> >> branch.
> >
> > Then it's good for you, but not for me: when a release is in progress,
> > I work mainly on the release branch.
> 
> And you can continue to do so -- and cherry-pick commits to master.

That's a disadvantage, because Git will not help us track which
changes from the branch are on master (and vice versa).

> > And of course, cherry-picking doesn't remove merge conflicts, they
> > will still need to be resolved.
> 
> So that's neither more than less work than the merge based work flow --
> but with the advantage that you're resolving one pick at a time instead
> of the more overwhelming thing that can happen when merging and you get
> conflicts from several commits at once.

Conflicts are generally rare with Git, though.

> > We've been using the current workflow for years without any major
> > problems.  Changing that now radically doesn't make sense to me.  It
> > will most probably need several procedures to be modified that were
> > stable for several releases, and I don't see the gains which would
> > justify that.  We have enough real work on our hands.  So please let's
> > not do that.
> 
> What's changed is that Glenn is (understandably) no longer willing to do
> the merges, so something has to change.  (Unless somebody else picks up
> the slack, and we can continue as before.)

We haven't even tried to find a new volunteer yet.  You suggest that
we give up before we even try.

Re: Merging release branch

[Stefan Monnier]

Lars Ingebrigtsen [2021-10-29 20:26:46] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> If `emacs-28` was already merged before your commit(s), then it's
>> a small matter of going back to the main branch and doing `git merge
>> emacs-28` or `git merge -s ours emacs-28`.
> That work flow sounds fine to me in general -- but what about those
> changes to emacs-28 that shouldn't be merged?

That's what `git merge -s ours emacs-28` does (it marks the commit as
merged but without bringing any of the corresponding code changes).


        Stefan

Re: Merging release branch

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> That's what `git merge -s ours emacs-28` does (it marks the commit as
> merged but without bringing any of the corresponding code changes).

Oh, I see.  But what if there was some other commit in emacs-28 that
that committer had forgotten to merge?  Then it would also be kept out
of master?

Your assumption here was that emacs-28 had already been merged, but I
think people would be likely to forget.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Merging release branch

[Stefan Monnier]

Lars Ingebrigtsen [2021-10-29 21:05:07] wrote:
> Your assumption here was that emacs-28 had already been merged, but I
> think people would be likely to forget.

Agreed.


        Stefan

Re: Merging release branch

[David Engster]

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> That's what `git merge -s ours emacs-28` does (it marks the commit as
>> merged but without bringing any of the corresponding code changes).
>
> Oh, I see.  But what if there was some other commit in emacs-28 that
> that committer had forgotten to merge?  Then it would also be kept out
> of master?

Yup. In that case, you need to do two merges with different merge
strategies. That's what admin/gitmerge.el is for, and I think it's
pretty easy to use (but I'm biased).

> Your assumption here was that emacs-28 had already been merged, but I
> think people would be likely to forget.

For sure.

I agree that doing cherry-picks from 'master' to 'emacs-28' would be
easier for people, simply because they are familiar with it, but keeping
track of what has been cherry-picked and what not can be pretty tedious,
and git cannot help you much with that.

-David

Re: Merging release branch

[dick]

GM> The final straw was a triple failure in the automatic merge today due to:

Good stuff.  I could take this over but I would need the right privileges.

Re: Merging release branch

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Fri, 29 Oct 2021 12:35:22 -0400
> 
> 
> I've had enough of merging the release branch to master,
> so someone else will have to do it from now on.
> 
> I use admin/automerge from cron with a wrapper that:
> 1) first checks the release branch passed make check
> 2) passes "-n 1" if the last merge was more than 1 week ago

What's the procedure with merging commits that change etc/NEWS on the
release branch?  gitmerge.el insists on trying to merge those in
etc/NEWS for the next release, which is not TRT.  One could skip the
commits which change only etc/NEWS and then "merge" manually by
copying branch/etc/NEWS into trunk/etc/NEWS.28, but isn't there a
better way?  And why doesn't gitmerge.el do that automatically?

Re: Merging release branch

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> What's the procedure with merging commits that change etc/NEWS on the
> release branch?  gitmerge.el insists on trying to merge those in
> etc/NEWS for the next release, which is not TRT.  One could skip the
> commits which change only etc/NEWS and then "merge" manually by
> copying branch/etc/NEWS into trunk/etc/NEWS.28, but isn't there a
> better way?  And why doesn't gitmerge.el do that automatically?

(I don't know how Glenn has done it until now.)

AFAIK, there is no strong need to name that file etc/NEWS until we
create a tarball or run "make install".  I think we should just keep the
file in etc/NEWS.NN even on the release branch.  That would avoid the
merging headache, and the file would also preserve its full history
between versions (no more moving the file).

If we want to do this, our build-scripts depend on there being an
etc/NEWS file, so they would need to be updated.

Re: Merging release branch

[Glenn Morris]

Stefan Kangas wrote:

> (I don't know how Glenn has done it until now.)

Using the tools in the Emacs repository, and documented in eg
admin/notes/git-workflow

> AFAIK, there is no strong need to name that file etc/NEWS until we
> create a tarball or run "make install".  I think we should just keep the
> file in etc/NEWS.NN even on the release branch.  That would avoid the
> merging headache, and the file would also preserve its full history
> between versions (no more moving the file).

See previous discussions in eg
https://lists.gnu.org/r/emacs-devel/2017-12/msg00340.html
https://debbugs.gnu.org/29366

Re: Merging release branch

[Glenn Morris]

Eli Zaretskii wrote:

> What's the procedure with merging commits that change etc/NEWS on the
> release branch?

gitmerge.el does have special handling for NEWS (I wrote it).
https://debbugs.gnu.org/29366

git reset --hard 5a01383
emacs -Q -nw -l ./admin/gitmerge.el -f gitmerge
RET
m
y # "Try to fix NEWS conflict?"
y
y

works for me.

Surely you weren't saying "n" to "Try to fix NEWS conflict?"...

Re: Merging release branch

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 06 Nov 2021 12:11:12 -0400
> 
> Eli Zaretskii wrote:
> 
> > What's the procedure with merging commits that change etc/NEWS on the
> > release branch?
> 
> gitmerge.el does have special handling for NEWS (I wrote it).
> https://debbugs.gnu.org/29366
> 
> git reset --hard 5a01383
> emacs -Q -nw -l ./admin/gitmerge.el -f gitmerge
> RET
> m
> y # "Try to fix NEWS conflict?"
> y
> y
> 
> works for me.
> 
> Surely you weren't saying "n" to "Try to fix NEWS conflict?"...

No, I was saying "y" (I think), and then it would put me in a buffer
with conflicts, all of which were from trying to merge the branch's
NEWS into the master's.

I guess some debugging is in order.