getting emacs-25-to-master merges working again

getting emacs-25-to-master merges working again

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1497 bytes --]

Today I got burned again by a bug in Emacs master that has been fixed in 
emacs-25.  Right now, my impression is that the master branch has serious 
unfixed bugs; I wouldn't recommend it for routine use. At some point we need to 
get emacs-25 merged with master, to fix the bugs. I drafted a patch to help do 
this (attached). It still needs work but I thought I'd send what I have, to help 
get the ball rolling.

The main problem in getting merging to work again is that changes to ChangeLog.2 
in emacs-25 will collide with changes to ChangeLog.2 in the master. The attached 
patch attempts to fix this by renaming ChangeLog.2 to ChangeLog-25.0 in 
emacs-25, and to create a new file ChangeLog-25.1 in master, such that 'make 
change-history' updates ChangeLog-25.0 in emacs-25 and updates ChangeLog-25.1 in 
master.  That way, the two sets of histories ordinarily shouldn't overlap and 
changes to one shouldn't collide with the other.

Undoubtedly there will be kinks in this process that need to be ironed out, 
e.g., when backporting changes from master to emacs-25. One way to iron them out 
will be to review and/or improve this patch. Another is to install this patch, 
do a merge, and deal with the resulting fallout.  Either will take nontrivial work.

I'm thinking, though, that there should be a better way, by using the --cherry 
option of 'git log', and by generating ChangeLog.2 and/or ChangeLog-25.1 
specially during a merge. I haven't had time to investigate this yet, though.

[-- Attachment #2: emacs-25-merge.patch --]
[-- Type: text/x-diff, Size: 15201 bytes --]

From f80213336f066b6b60e6eab33553bc5bc8953479 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sun, 29 Nov 2015 15:32:46 -0800
Subject: [PATCH] Rename ChangeLog.2 to ChangeLog-25.0, etc.

The idea is to use ChangeLog-25.0 for the emacs-25 branch,
and ChangeLog-25.1 for the master branch, to make it easier
to merge from emacs-25 into master.
* Makefile.in (CHANGELOG_VERSION): Rename from
CHANGELOG_HISTORY_INDEX_MAX, and change it to be a traditional
Emacs version number.  All uses changed.
(CHANGELOG_N): Use "-" to separate "ChangeLog" from version number.
(current-branch-25.0): New rule.
(current-branch-25.1): Rename from master-branch-is-current.
All uses changed.
* admin/notes/git-workflow, admin/notes/repo: Document new procedure.
* admin/update_autogen (CHANGELOG_VERSION): New var, replacing
changelog_n.  Only use changed.
* build-aux/gitlog-to-emacslog (nmax): If unset or empty (default),
infer it from CHANGELOG_VERSION in Makefile.in.
(ChangeLog_nmax): New var.  All uses of ChangeLog.$nmax
replaced with $ChangeLog_nmax.
Check nmax for proper chars.
Substitute ChangeLog-* too, and versions with "." and "-".
* make-dist: Also distribute ChangeLog-25.0 etc.
---
 ChangeLog-25.0               | 17980 +++++++++++++++++++++++++++++++++++++++++
 ChangeLog.2                  | 17980 -----------------------------------------
 Makefile.in                  |    28 +-
 admin/notes/git-workflow     |    53 +-
 admin/notes/repo             |    27 +-
 admin/update_autogen         |     6 +-
 build-aux/gitlog-to-emacslog |    26 +-
 make-dist                    |     2 +-
 8 files changed, 18076 insertions(+), 18026 deletions(-)
 create mode 100644 ChangeLog-25.0
 delete mode 100644 ChangeLog.2

[Boring long diff omitted, which basically says "rename ChangeLog-2 to ChangeLog-25.0."]

diff --git a/Makefile.in b/Makefile.in
index 1245f76..2a1eb46 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -1093,20 +1093,31 @@ bootstrap: bootstrap-clean

 CHANGELOG = ChangeLog
 emacslog = build-aux/gitlog-to-emacslog
-# The ChangeLog history files are called ChangeLog.1, ChangeLog.2, ...,
-# ChangeLog.$(CHANGELOG_HISTORY_INDEX_MAX).  $(CHANGELOG_N) stands for
-# the newest (highest-numbered) ChangeLog history file.
-CHANGELOG_HISTORY_INDEX_MAX = 2
-CHANGELOG_N = ChangeLog.$(CHANGELOG_HISTORY_INDEX_MAX)
+
+# CHANGELOG_VERSION is the version number for the next (or this) Emacs version.
+# ChangeLog-25.0 is the ChangeLog history file for Emacs 25.0,
+# and similarly for ChangeLog-25.1, ChangeLog-25.2, etc.
+# ChangeLog.1 records the history of top-level changes for Emacs
+# before the unified ChangeLog reporting mechanism was instituted
+# during the development of Emacs 25.0.  Other ChangeLog history
+# files (e.g., lisp/ChangeLog.17) follow this older naming convention,
+# and represent older changes to their respective subdirectories only.
+# Increment CHANGELOG_VERSION when you want to start development for
+# a later version.  It should be OK for CHANGELOG_VERSION to have one value
+# in a bugfix branch, and a later value in the main development branch.
+CHANGELOG_VERSION = 25.0
+CHANGELOG_N = ChangeLog-$(CHANGELOG_VERSION)

 # Convert git commit log to ChangeLog file.  make-dist uses this.
 # I guess this is PHONY so it always updates?
 ChangeLog:
 	$(AM_V_GEN)cd $(srcdir) && \
-	  ./$(emacslog) -o $(CHANGELOG) -n $(CHANGELOG_HISTORY_INDEX_MAX)
+	  ./$(emacslog) -o $(CHANGELOG) -n $(CHANGELOG_VERSION)

 # Check that we are in a good state for changing history.
-master-branch-is-current:
+current-branch-25.0:
+	git branch | grep -q '^\* emacs-25$$'
+current-branch-25.1:
 	git branch | grep -q '^\* master$$'
 unchanged-history-files:
 	x=$$(git diff-files --name-only $(CHANGELOG_N) $(emacslog)) && \
@@ -1117,7 +1128,8 @@ new_commit_regexp = ^commit [0123456789abcdef]* (inclusive)

 # Copy newer commit messages to the start of the ChangeLog history file,
 # and consider them to be older.
-change-history-nocommit: master-branch-is-current unchanged-history-files
+change-history-nocommit: \
+  current-branch-$(CHANGELOG_VERSION) unchanged-history-files
 	-rm -f ChangeLog.tmp
 	$(MAKE) ChangeLog CHANGELOG=ChangeLog.tmp
 	sed '/^This file records repository revisions/,$$d' \
diff --git a/admin/notes/git-workflow b/admin/notes/git-workflow
index 3c2c76c..2c3be9d 100644
--- a/admin/notes/git-workflow
+++ b/admin/notes/git-workflow
@@ -1,6 +1,5 @@
 (This is a draft.  The method here won't actually work yet, because
-neither git-new-workdir nor merge-changelog are in the Emacs
-distribution yet.)
+git-new-workdir is not in the Emacs distribution yet.)

 Setting up and using git for normal, simple bugfixing
 =====================================================
@@ -14,16 +13,16 @@ Initial setup
 =============

 Then we want to clone the repository.  We normally want to have both
-the current trunk and the emacs-24 branch.
+the current trunk and the emacs-25 branch.

 mkdir ~/emacs
 cd ~/emacs
 git clone <membername>@git.sv.gnu.org:/srv/git/emacs.git
 mv emacs trunk
 (cd trunk; git config push.default current)
-./trunk/admin/git-new-workdir trunk emacs-24
-cd emacs-24
-git checkout emacs-24
+./trunk/admin/git-new-workdir trunk emacs-25
+cd emacs-25
+git checkout emacs-25
 git config push.default current

 You now have both branches conveniently accessible, and you can do
@@ -54,11 +53,11 @@ you commit your change locally and then send a patch file as a bug report
 as described in ../../CONTRIBUTE.


-Backporting to emacs-24
+Backporting to emacs-25
 =======================

 If you have applied a fix to the trunk, but then decide that it should
-be applied to the emacs-24 branch, too, then
+be applied to the emacs-25 branch, too, then

 cd ~/emacs/trunk
 git log
@@ -68,7 +67,7 @@ which will look like

 commit 958b768a6534ae6e77a8547a56fc31b46b63710b

-cd ~/emacs/emacs-24
+cd ~/emacs/emacs-25
 git cherry-pick -xe 958b768a6534ae6e77a8547a56fc31b46b63710b

 and add "Backport:" to the commit string.  Then
@@ -76,17 +75,17 @@ and add "Backport:" to the commit string.  Then
 git push


-Merging emacs-24 to trunk/master
+Merging emacs-25 to trunk/master
 ================================

 It is recommended to use the file gitmerge.el in the admin directory
-for merging 'emacs-24' into 'master'.  It will take care of many
+for merging 'emacs-25' into 'master'.  It will take care of many
 things which would otherwise have to be done manually, like ignoring
 commits that should not land in master, fixing up ChangeLogs and
 automatically dealing with certain types of conflicts.  If you really
 want to, you can do the merge manually, but then you're on your own.
 If you still choose to do that, make absolutely sure that you *always*
-use the 'merge' command to transport commits from 'emacs-24' to
+use the 'merge' command to transport commits from 'emacs-25' to
 'master'.  *Never* use 'cherry-pick'!  If you don't know why, then you
 shouldn't manually do the merge in the first place; just use
 gitmerge.el instead.
@@ -94,16 +93,24 @@ gitmerge.el instead.
 How to use gitmerge.el:

 Enter the Emacs repository, checkout 'master' and make sure it's
-up-to-date by doing a pull.  Then start Emacs with
+up-to-date by doing:
+
+  git pull
+
+Then bring the file ChangeLog-25.1 up-to-date by doing:
+
+  make change-history
+
+Then start Emacs with

   emacs -l admin/gitmerge.el -f gitmerge

 You'll be asked for the branch to merge, which will default to
-'origin/emacs-24', which you should accept.  Merging a local tracking
+'origin/emacs-25', which you should accept.  Merging a local tracking
 branch is discouraged, since it might not be up-to-date, or worse,
 contain commits from you which are not yet pushed upstream.

-You will now see the list of commits from 'emacs-24' which are not yet
+You will now see the list of commits from 'emacs-25' which are not yet
 merged to 'master'.  You might also see commits that are already
 marked for "skipping", which means that they will be merged with a
 different merge strategy ('ours'), which will effectively ignore the
@@ -126,7 +133,21 @@ current state will be saved to disk.

 When everything's done, look hard at the resulting merge.  Skipping
 commits requires separate merges, so don't be surprised to see more
-than one merge commit.  If you're happy, push.
+than one merge commit.
+
+The above steps should have updated ChangeLog-25.0 without altering
+ChangeLog-25.1.  If you were to run 'make change-history' now, this
+would mess up ChangeLog-25.1 by adding a redundant copy of the newly
+merged ChangeLog entries.  Prevent this by editing ChangeLog-25.1 to
+say that it records revisions to the current master commit
+(inclusive), by doing the following shell commands:
+
+  commitid=$(git log -1 --format=%H)
+  sed "s/^commit .* (inclusive)\\.\$/commit $commitid (inclusive)./" ChangeLog-25.1 >ChangeLog-25.1.tmp
+  cp ChangeLog-25.1.tmp ChangeLog-25.1
+  git commit -m"Omit ChangeLog entries from emacs-25 merge" ChangeLog-25.1
+
+If you're happy, push.

 Warnings about X11 forwarding
 =============================
diff --git a/admin/notes/repo b/admin/notes/repo
index 3ab3da7..7daffee 100644
--- a/admin/notes/repo
+++ b/admin/notes/repo
@@ -11,9 +11,9 @@ instructions.

 In particular, install bug-fixes only on the release branch (if there
 is one) and let them get synced to the master; do not install them by
-hand on the master as well.  E.g. if there is an active "emacs-24" branch
-and you have a bug-fix appropriate for the next emacs-24.x release,
-install it only on the emacs-24 branch, not on the master as well.
+hand on the master as well.  E.g. if there is an active "emacs-25" branch
+and you have a bug-fix appropriate for the next emacs-25.x release,
+install it only on the emacs-25 branch, not on the master as well.

 Installing things manually into more than one branch makes merges more
 difficult.
@@ -66,9 +66,10 @@ variable in admin/merge-gnulib before running it.
 If you remove a gnulib module, or if a gnulib module
 removes a file, then remove the corresponding files by hand.

-* How to merge changes from emacs-24 to master
+* How to merge changes from emacs-25 to master

-[The section on git merge procedure has not yet been written.]
+[The section on git merge procedure has not yet been written.
+See git-workflow for a draft.]

 You may see conflicts in autoload md5sums in comments.  Strictly
 speaking, the right thing to do is merge everything else, resolve the
@@ -119,14 +120,20 @@ Browse 'git help bisect' for technical instructions.

 * Maintaining ChangeLog history

-Older ChangeLog entries are kept in history files named ChangeLog.1,
-ChangeLog.2, etc., and can be edited just as any other source files
-can.  Newer ChangeLog entries are stored in the repository as commit
-messages, which cannot be edited directly.
+Newer ChangeLog entries are stored in the repository as commit
+messages, which cannot be edited directly.  Older ChangeLog entries
+are kept in top-level history files named ChangeLog-25.0,
+ChangeLog-25.1, etc., and can be edited just as any other source files
+can; their version numbers correspond to Emacs release numbers.
+Ancient ChangeLog entries are kept in directory-local history files
+named ChangeLog.1, ChangeLog.2, etc., and can also be edited like
+other source files.

 'make ChangeLog' copies newer ChangeLog entries into a file
 'ChangeLog' that is intended to be put into the distribution tarball.
-This ChangeLog file is not put into the repository.
+This ChangeLog file is not put into the repository.  This means
+the distribution tarball for (say) Emacs 25.0 has a file
+ChangeLog.25.0

 'make change-history' copies all newer ChangeLog entries into the
 start of the newest ChangeLog history file.  These ChangeLog entries
diff --git a/admin/update_autogen b/admin/update_autogen
index 23e1d40..21b7247 100755
--- a/admin/update_autogen
+++ b/admin/update_autogen
@@ -69,7 +69,7 @@ Options:
     commit them (caution).
 -q: be quiet; only give error messages, not status messages.
 -A: only update autotools files, copying into specified dir.
--H: also update ChangeLog.${changelog_n}
+-H: also update ChangeLog-${CHANGELOG_VERSION}
 -I: also update info/dir.
 -L: also update ldefs-boot.el.
 -C: start from a clean state.  Slower, but more correct.
@@ -93,8 +93,8 @@ changelog_flag=
 ## Parameters.
 ldefs_in=lisp/loaddefs.el
 ldefs_out=lisp/ldefs-boot.el
-changelog_n=$(sed -n 's/CHANGELOG_HISTORY_INDEX_MAX *= *//p' Makefile.in)
-changelog_files="ChangeLog.$changelog_n"
+CHANGELOG_VERSION=$(sed -n 's/CHANGELOG_VERSION *= *//p' Makefile.in) || die
+changelog_files="ChangeLog-$CHANGELOG_VERSION"
 sources="configure.ac lib/Makefile.am"
 ## Files to copy into autogendir.
 ## Everything:
diff --git a/build-aux/gitlog-to-emacslog b/build-aux/gitlog-to-emacslog
index 3d61ba1..f32cbe5 100755
--- a/build-aux/gitlog-to-emacslog
+++ b/build-aux/gitlog-to-emacslog
@@ -27,7 +27,7 @@ gen_origin=

 force=
 output=ChangeLog
-nmax=2
+nmax=

 while [ $# -gt 0 ]; do
   case "$1" in
@@ -40,8 +40,18 @@ while [ $# -gt 0 ]; do
   shift
 done

-if [ ! -f ChangeLog.$nmax ]; then
-    printf '%s\n' "Can't find ChangeLog.$nmax" >&2
+[ "$nmax" ] || nmax=$(sed -n 's/CHANGELOG_VERSION *= *//p' Makefile.in) || exit
+
+ChangeLog_nmax=ChangeLog-$nmax
+
+case $nmax in
+    *[!-0-9.]*)
+	printf '%s\n' "Invalid ChangeLog file name: $ChangeLog_nmax" >&2
+	exit 1;;
+esac
+
+if [ ! -f "$ChangeLog_nmax" ]; then
+    printf '%s\n' "Can't find $ChangeLog_nmax" >&2
     printf '%s\n' "Must be run from the top source directory" >&2
     exit 1
 fi
@@ -50,9 +60,9 @@ fi
 # ChangeLog file.
 [ "$gen_origin" ] || {
     gen_origin_line=`
-      grep -E '^commit [0-9a-f]+ [(]inclusive[)]' ChangeLog.$nmax
+      grep -E '^commit [0-9a-f]+ [(]inclusive[)]' "$ChangeLog_nmax"
     ` || {
-	printf '%s\n' "ChangeLog.$nmax lacks a 'commit ... (inclusive)' line" >&2
+	printf '%s\n' "$ChangeLog_nmax lacks a 'commit ... (inclusive)' line" >&2
 	exit 1
     }
     set $gen_origin_line
@@ -117,12 +127,12 @@ if test -s "ChangeLog.tmp"; then
     /^This file records repository revisions/p
     s/^commit [0-9a-f]* (exclusive)/commit '"$gen_origin"' (exclusive)/p
     s/^commit [0-9a-f]* (inclusive)/commit '"$new_origin"' (inclusive)/p
-    /^See ChangeLog.[0-9]* for earlier/,${
-       s/ChangeLog\.[0-9]*/ChangeLog.'$nmax'/
+    /^See ChangeLog[-.][0-9.]* for earlier/,${
+       s/ChangeLog[-.][-0-9.]*/ChangeLog-'"$nmax"'/
        s/\(Copyright[ (C)]*\)[0-9]*-[0-9]*/\1'"$year_range"'/
        p
     }
-  ' <ChangeLog.$nmax >>"ChangeLog.tmp" || exit
+  ' <"$ChangeLog_nmax" >>"ChangeLog.tmp" || exit
 fi

 # Install the generated ChangeLog.
diff --git a/make-dist b/make-dist
index 48a2836..3d01bcf 100755
--- a/make-dist
+++ b/make-dist
@@ -299,7 +299,7 @@ fi
 ### README while the rest of the tar file is still unpacking.  Whoopee.
 echo "Making links to top-level files"
 ln INSTALL README BUGS ${tempdir}
-ln ChangeLog.*[0-9] Makefile.in autogen.sh configure configure.ac ${tempdir}
+ln ChangeLog[-.]*[0-9] Makefile.in autogen.sh configure configure.ac ${tempdir}
 ln config.bat make-dist .dir-locals.el ${tempdir}
 ln aclocal.m4 ${tempdir}

--
2.1.0