git replace --edit for updating commit messages (was: Is it time to drop ChangeLogs?)

git replace --edit for updating commit messages (was: Is it time to drop ChangeLogs?)

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 2119 bytes --]

On Wed, Jul 13, 2016 at 1:47 PM, Sven Axelsson <sven.axelsson@gmail.com> wrote:
>> Yes, this is a common need and Git should be changed to allow editing
>> of commit log entries even though this could be said to change the
>> state of the prior releases.
>
>
> A couple of months ago it was suggested to use `git replace --edit` for
> that.
> Did anyone try it out to see if it could be used in the Emacs workflow?

There were some questions raised, the 2 main ones were

1. How to deal with the fact git replace is powerful enough to replace
the actual content of the commit.

This is easy enough to stop with a hook, see attached pre-receive.

2. Performance implications of having many replacements.

This appears to be a more serious problem. The git replace command
creates a ref per replacement. Since we expect our number of
replacements to be some fraction of the commits, we could end up with
quite a few. To get a sense of the worst case timing I wrote a script
to replace the message of every commit in the repository with an
upcased SHOUTY version. After running it on an Emacs repo clone, my
.git/refs/replace directory was 8.5M in size. I tried git pack-refs
--all which emptied the the replace directory, and resulted in a 12M
packed-refs file.

Doing git log became dramatically slower (packing the refs did not
make much difference).

Original repository:

~/src/emacs/emacs-master$ time git log --oneline | wc -l
126614

real    0m5.435s
user    0m3.157s
sys    0m2.780s

~/src/emacs/emacs-master$ time git log --oneline | head >/dev/null

real    0m0.009s
user    0m0.003s
sys    0m0.003s


Repository with all commit messages replaced: (almost all, there were
a few messages that lacked letters, so my upcasing script had no
effect)

~/src/emacs/emacs-clone$ time git log --oneline | wc -l
126614

real    0m49.337s
user    0m8.270s
sys    0m39.680s

~/src/emacs/emacs-clone$ time git log --oneline | head >/dev/null

real    0m0.520s
user    0m0.237s
sys    0m0.250s


I attached the scripts used (upcase-commit-message.sh and
git-replace-all.sh), in case anyone wants to replicate my experiments.

[-- Attachment #2: pre-receive --]
[-- Type: application/octet-stream, Size: 470 bytes --]

#!/bin/sh

# Print hashes of tree and parent.  This is the data we don't want to
# allow replacements for.
commit_data () {
    git --no-replace-objects show --no-patch --format='%T:%P' "$@"
}

while read old new name ; do
    case $name in
        (refs/replace/*)
            if [ "$(commit_data $(basename $name))" != "$(commit_data $new)" ] ; then
                echo "$name changes commit diff/parentage!" >&2
                exit 1
            fi;;
    esac
done

[-- Attachment #3: upcase-commit-message.sh --]
[-- Type: application/x-sh, Size: 94 bytes --]

[-- Attachment #4: git-replace-all.sh --]
[-- Type: application/x-sh, Size: 171 bytes --]

Re: git replace --edit for updating commit messages

[Andreas Schwab]

On Do, Jul 28 2016, Noam Postavsky <npostavs@users.sourceforge.net> wrote:

> Doing git log became dramatically slower (packing the refs did not
> make much difference).

FWIW, there is ongoing work to add a refs API that allows plugging in a
different backend to store references, to scale better with the number
of refs.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: git replace --edit for updating commit messages (was: Is it time to drop ChangeLogs?)

[Eli Zaretskii]

> From: Noam Postavsky <npostavs@users.sourceforge.net>
> Date: Wed, 27 Jul 2016 21:35:34 -0400
> Cc: Robert Weiner <rswgnu@gmail.com>, Stefan Monnier <monnier@iro.umontreal.ca>,
> 	emacs-devel <emacs-devel@gnu.org>
> 
> Doing git log became dramatically slower (packing the refs did not
> make much difference).
> 
> Original repository:
> 
> ~/src/emacs/emacs-master$ time git log --oneline | wc -l
> 126614
> 
> real    0m5.435s
> user    0m3.157s
> sys    0m2.780s
> 
> ~/src/emacs/emacs-master$ time git log --oneline | head >/dev/null
> 
> real    0m0.009s
> user    0m0.003s
> sys    0m0.003s
> 
> 
> Repository with all commit messages replaced: (almost all, there were
> a few messages that lacked letters, so my upcasing script had no
> effect)
> 
> ~/src/emacs/emacs-clone$ time git log --oneline | wc -l
> 126614
> 
> real    0m49.337s
> user    0m8.270s
> sys    0m39.680s
> 
> ~/src/emacs/emacs-clone$ time git log --oneline | head >/dev/null
> 
> real    0m0.520s
> user    0m0.237s
> sys    0m0.250s

Thanks.  But I think a more relevant timing would be one for something
like 1% to 10% of replaced messages, because we will never have all of
them replaced.

Re: git replace --edit for updating commit messages (was: Is it time to drop ChangeLogs?)

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

On Thu, Jul 28, 2016 at 10:43 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>
> Thanks.  But I think a more relevant timing would be one for something
> like 1% to 10% of replaced messages, because we will never have all of
> them replaced.

I made a script to collect some more timings at different replacement
levels. Script and timings attached. The time needed grows linearly
per number of replacements. Looking at the time for git log | head
(which I believe is representative of the how long it takes to get
something onscreen from a normal git log with pager),  I'd say it's
big enough to be annoying (10th of a second) starting from around 4000
replacements (that's roughly 3%).

[-- Attachment #2: full-log-times.txt --]
[-- Type: text/plain, Size: 1155 bytes --]

#	real	user	sys
0	4.41	2.85	1.53
1000	4.76	2.96	1.76
2000	5.43	3.15	2.24
3000	6.11	3.31	2.76
4000	7.28	3.52	3.73
5000	8.33	3.57	4.73
6000	9.60	3.62	5.94
7000	11.25	3.86	7.35
8000	12.27	3.86	8.36
9000	13.07	4.02	8.98
10000	13.65	4.20	9.40
11000	14.11	4.37	9.69
12000	14.04	4.14	9.85
13000	14.28	4.28	9.95
14000	14.51	4.23	10.23
15000	15.22	4.26	10.90
16000	15.46	4.31	11.08
17000	16.12	4.30	11.74
18000	16.89	4.59	12.25
19000	17.39	4.66	12.68
20000	18.41	4.69	13.65
21000	18.96	4.70	14.20
22000	19.67	4.99	14.62
23000	20.29	4.92	15.31
24000	20.81	5.06	15.67
25000	21.68	5.20	16.38
26000	22.17	5.17	16.93
27000	22.71	5.56	17.05
28000	23.69	5.43	18.19
29000	24.22	5.54	18.56
30000	24.82	5.51	19.21
31000	25.47	5.70	19.68
32000	26.21	5.82	20.28
33000	26.50	5.53	20.88
34000	27.17	5.76	21.31
35000	27.66	5.96	21.62
36000	28.33	6.09	22.16
37000	29.13	5.71	23.32
38000	30.10	6.43	23.57
39000	30.32	6.10	24.13
40000	30.97	6.16	24.68
41000	31.65	6.27	25.27
42000	32.36	6.46	25.78
43000	32.91	6.57	26.24
44000	33.59	6.50	26.95
45000	34.30	6.65	27.51
46000	34.92	6.73	28.07
47000	35.52	6.65	28.75
48000	36.12	6.90	29.08
49000	36.75	7.37	29.25
50000	37.42	6.96	30.33

[-- Attachment #3: git-replace-some.sh --]
[-- Type: application/x-sh, Size: 631 bytes --]

[-- Attachment #4: log-times.txt --]
[-- Type: text/plain, Size: 1074 bytes --]

#	real	user	sys
0	0.00	0.00	0.00
1000	0.01	0.00	0.01
2000	0.04	0.02	0.02
3000	0.06	0.03	0.03
4000	0.10	0.05	0.05
5000	0.15	0.04	0.10
6000	0.19	0.08	0.10
7000	0.25	0.06	0.17
8000	0.30	0.11	0.17
9000	0.34	0.12	0.19
10000	0.35	0.11	0.22
11000	0.36	0.16	0.19
12000	0.37	0.14	0.21
13000	0.37	0.13	0.22
14000	0.39	0.13	0.24
15000	0.44	0.19	0.22
16000	0.46	0.17	0.24
17000	0.48	0.20	0.26
18000	0.50	0.22	0.24
19000	0.54	0.18	0.33
20000	0.58	0.22	0.33
21000	0.60	0.25	0.30
22000	0.63	0.22	0.37
23000	0.66	0.25	0.38
24000	0.70	0.27	0.37
25000	0.78	0.24	0.43
26000	0.78	0.27	0.43
27000	0.76	0.25	0.48
28000	0.85	0.31	0.46
29000	0.94	0.34	0.45
30000	0.93	0.31	0.52
31000	0.90	0.36	0.48
32000	1.01	0.32	0.55
33000	1.03	0.39	0.52
34000	0.97	0.40	0.53
35000	1.00	0.37	0.57
36000	1.02	0.41	0.57
37000	1.06	0.45	0.55
38000	1.07	0.43	0.59
39000	1.12	0.42	0.63
40000	1.14	0.45	0.61
41000	1.14	0.38	0.70
42000	1.21	0.53	0.62
43000	1.20	0.45	0.68
44000	1.26	0.45	0.73
45000	1.26	0.47	0.72
46000	1.30	0.55	0.67
47000	1.34	0.57	0.69
48000	1.34	0.53	0.73
49000	1.41	0.56	0.78
50000	1.43	0.59	0.76

Re: git replace --edit for updating commit messages

[John Wiegley]

>>>>> "AS" == Andreas Schwab <schwab@suse.de> writes:

AS> FWIW, there is ongoing work to add a refs API that allows plugging in a
AS> different backend to store references, to scale better with the number of
AS> refs.

Also, wouldn't it be possible to exclude these refs from being pulled, so that
only those who actually need to see the updated commit messages pay any cost?

If the cost is neglible for a small number of revised commits (for now), it
doesn't seem too scary.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: git replace --edit for updating commit messages

[Andreas Schwab]

John Wiegley <jwiegley@gmail.com> writes:

> Also, wouldn't it be possible to exclude these refs from being pulled, so that
> only those who actually need to see the updated commit messages pay any cost?

The refs/replace namespace isn't fetched by default.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."