Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 839 bytes --]

 On Sun, Mar 06 2016 13:07 (-0800), John Wiegley wrote:
> 
> I have always wanted to drop the ChangeLogs, so if the other developers agree,
> I'm all for it. Keeping ChangeLog style in the commit entry is not terribly
> useful either, since the diff output of log -p lets you know which function or
> variable is being modified. I've never missed not having that ChangeLog data
> in other projects, of any size. But that's up to the other developers and what
> makes their lives easier.

I'd like to open this up to discussion on emacs-devel, so that we hear from
our other developers. What do you all think about ChangeLogs, and their value
to you in your work on Emacs?

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Eric S. Raymond]

[-- Attachment #1: Type: text/plain, Size: 916 bytes --]

John Wiegley <jwiegley@gmail.com>:
>  On Sun, Mar 06 2016 13:07 (-0800), John Wiegley wrote:
> > 
> > I have always wanted to drop the ChangeLogs, so if the other developers agree,
> > I'm all for it. Keeping ChangeLog style in the commit entry is not terribly
> > useful either, since the diff output of log -p lets you know which function or
> > variable is being modified. I've never missed not having that ChangeLog data
> > in other projects, of any size. But that's up to the other developers and what
> > makes their lives easier.
> 
> I'd like to open this up to discussion on emacs-devel, so that we hear from
> our other developers. What do you all think about ChangeLogs, and their value
> to you in your work on Emacs?

I advocated dropping ChangeLogs at the time I did the repository conversion.
That is still my position.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Paul Eggert]

John Wiegley wrote:
> What do you all think about ChangeLogs, and their value
> to you in your work on Emacs?

I regularly do the equivalent of:

grep PATTERN $(git ls-files | grep -v ChangeLog)

That is, in searches I typically ignore ChangeLog* files because of their 
less-useful chatter. I'd be happy if we stopped maintaining these files in the 
master branch (we should keep them for emacs-25). I'd be happy if we removed the 
existing ChangeLog* files from the master branch.

Even if we remove ChangeLogs, we should still have guidelines for commit message 
format, as saying "anything goes" would make it harder to read the output of 
'git log'. I don't mind using ChangeLog format in commit messages, as it's a 
well-understood format and switching to some other format could well be more 
trouble than it's worth.

RE: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Drew Adams]

Maybe see this thread?
http://lists.gnu.org/archive/html/emacs-devel/2013-03/msg00904.html

(FWIW, I have no opinion about where Emacs Dev puts change-log info.)

Re: Is it time to drop ChangeLogs?

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 551 bytes --]

>>>>> Drew Adams <drew.adams@oracle.com> writes:

> Maybe see this thread?
> http://lists.gnu.org/archive/html/emacs-devel/2013-03/msg00904.html

I respect that Richard may use them in a constructive way, but he's no longer
doing the majority of the work on Emacs, and I want to choose the path that
works for the core developers, and will make Emacs more welcoming to new
contributors.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

Re: Is it time to drop ChangeLogs?

[Mathieu Lirzin]

Hi,

John Wiegley <jwiegley@gmail.com> writes:

>  On Sun, Mar 06 2016 13:07 (-0800), John Wiegley wrote:
>> 
>> I have always wanted to drop the ChangeLogs, so if the other developers agree,
>> I'm all for it. Keeping ChangeLog style in the commit entry is not terribly
>> useful either, since the diff output of log -p lets you know which function or
>> variable is being modified. I've never missed not having that ChangeLog data
>> in other projects, of any size. But that's up to the other developers and what
>> makes their lives easier.
>
> I'd like to open this up to discussion on emacs-devel, so that we hear from
> our other developers. What do you all think about ChangeLogs, and their value
> to you in your work on Emacs?

Discussing such thing seems reasonable.  However I don't think any
decision should be made by Emacs developpers on their own.  Change Logs
are part of the GCS so relaxing their requirement should be made at a
GNU level instead.

In my short experience, Change Logs has generally been useful both when
reading and composing them.  When writing them it helps me structure
large changes in logical commits that are modelled by the Change Log
format.  Finally It helps me being precise in my wordings which is not
trivial for non-native english speakers.

On a more spiritual side, I think they belong to the zen of contributing
to a GNU project.  :)

-- 
Mathieu Lirzin

RE: Is it time to drop ChangeLogs?

[Drew Adams]

> > Maybe see this thread?
> > http://lists.gnu.org/archive/html/emacs-devel/2013-
> > 03/msg00904.html
> 
> I respect that Richard may use them in a constructive way, but he's
> no longer doing the majority of the work on Emacs, and I want to
> choose the path that works for the core developers, and will make
> Emacs more welcoming to new contributors.

I meant the thread, not that message.
I meant to link to the first, not the third, message of the thread:
http://lists.gnu.org/archive/html/emacs-devel/2013-03/msg00879.html

The point is that the subject was the same, so it might serve as
a good starting point now, to see what was said before about it.

bug#21998: Is it time to drop ChangeLogs?

[Eric S. Raymond]

Mathieu Lirzin <mthl@gnu.org>:
> On a more spiritual side, I think they belong to the zen of contributing
> to a GNU project.  :)

Dear Goddess, I hope you're joking.

Speaking as a very, *very* long-time GNU contributor (I'm pretty sure
my earliest Emacs patches predated the formation of the FSF), I
consider ChangeLogs a relic of a bygone era.

Changelogs made some sense as a way of grouping together what we now
call a changeset back in the days of file-oriented version-control
systems.  Nowadays, set against the actual changeset comments in
version-control histories, Changelogs are a pointless and duplicative
ritual.

I think we should have dispensed with this practice during the CVS-to-bzr
transition. As it is, the sooner gone the better.  It's not as if retaining
arbitrary hoops for new developers to have to jump through is a *good* idea.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Is it time to drop ChangeLogs?

[Alan Mackenzie]

Hello, John.

On Sun, Mar 06, 2016 at 01:52:04PM -0800, John Wiegley wrote:
>  On Sun, Mar 06 2016 13:07 (-0800), John Wiegley wrote:

> > I have always wanted to drop the ChangeLogs, so if the other
> > developers agree, I'm all for it. Keeping ChangeLog style in the
> > commit entry is not terribly useful either, ....

I find that a strange thing to say.  I frequently search the ChangeLog
for the name of a function, to find out when it was last changed.  I
think the relatively rigid format of the ChangeLog/git commit messages
very helpful.

> > ...., since the diff output of log -p lets you know which function
> > or variable is being modified. I've never missed not having that
> > ChangeLog data in other projects, of any size. But that's up to the
> > other developers and what makes their lives easier.

The ChangeLog is easy to use.  git is difficult to use (more precisely,
is difficult to find out how to use).  I've no idea what git log -p
does, for example (though I'll be looking it up after I've posted this
post :-).  If we drop the ChangeLog we're cutting off its contents from
those for whom discovering the appropriate git commands is too much
work.

> I'd like to open this up to discussion on emacs-devel, so that we hear from
> our other developers. What do you all think about ChangeLogs, and their value
> to you in your work on Emacs?

The ChangeLog is useful, enabling things to be done that can't be done
without it.  If I want to see whether some change was made in Emacs
23.2, for example, I can either inspect 23.2's ChangeLog or spend
several hours working out how to get the information out of git.  No
contest.

The ChangeLog is distributed with the release, enabling our users to
access its information.  They typically don't have and/or don't know how
to use git repositories.

I'm decidedly in favour of keeping the ChangeLog.  But surely an
important consideration is whether the person/people who put in the work
to generate it are prepared to carry on doing this.

> -- 
> John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
> http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Eli Zaretskii]

[I removed the bug address from the CC list, as we shouldn't cross-post.]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Sun, 06 Mar 2016 13:52:04 -0800
> Cc: 21998@debbugs.gnu.org, Lars Magne Ingebrigtsen <larsi@gnus.org>
> 
> Keeping ChangeLog style in the commit entry is not terribly
> useful either, since the diff output of log -p lets you know which function or
> variable is being modified.

"git log -p" cannot do the job for changes in many types of files.
For example, try it on Lisp or Texinfo files.

More generally, there's no way Git could replace ChangeLog style
entries, because they frequently include information that is not in
the diffs.  To say nothing of the fact that understanding the change
from reading Diff hunks is much harder, and therefore much less
efficient, than from reading a log entry which describes the change in
plain English.

> I've never missed not having that ChangeLog data in other projects,
> of any size.

Maybe you rarely need to do any forensics.  Me, I do it all the time
in Emacs, and ChangeLog files are a valuable tool in the chest.

bug#21998: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Sun, 6 Mar 2016 14:34:12 -0800
> Cc: 21998@debbugs.gnu.org
> 
> John Wiegley wrote:
> > What do you all think about ChangeLogs, and their value
> > to you in your work on Emacs?
> 
> I regularly do the equivalent of:
> 
> grep PATTERN $(git ls-files | grep -v ChangeLog)
> 
> That is, in searches I typically ignore ChangeLog* files because of their 
> less-useful chatter.

It is a truism that when you know ChangeLog files don't included the
information you are after, you'd like to exclude them.  Likewise, when
I'm looking for information that can only be found in ChangeLog files,
I will filter out everything else as "less-useful chatter".

> Even if we remove ChangeLogs, we should still have guidelines for commit message 
> format

We won't be able to enforce that rule for long.  It's a slippery
slope.  Very slippery.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Sun, 06 Mar 2016 16:15:14 -0800
> Cc: 21998@debbugs.gnu.org, Lars Magne Ingebrigtsen <larsi@gnus.org>,
> 	Richard Stallman <rms@gnu.org>, Emacs developers <emacs-devel@gnu.org>
> 
> I respect that Richard may use them in a constructive way, but he's no longer
> doing the majority of the work on Emacs

Neither are some of those who expressed their views here.  Unlike
Richard, those others cannot present a history of contributions and of
leading the development that is anywhere near what Richard has on his
record.

> I want to choose the path that works for the core developers, and
> will make Emacs more welcoming to new contributors.

It is IMO a grave mistake to remove parts of our development process
just because they need to be learned by new contributors.  The result
will be increased burden on the shoulders of those who review the
submissions, due to the need to coach them, catch their mistakes, ask
for re-submissions, etc.  It will eventually be a net loss, because
those contributors will not enjoy the need to produce several versions
of the patch before it is admitted, and the reviewers will not enjoy
the extra burden.

We should consider each part of the development process on its own
merit, first and foremost.  If a part is useful, and its removal will
make the quality of the code or the documentation lower, then it
should not be removed, and new contributors will have to learn it and
to live by it.

There's no useful way into Emacs development that doesn't require
negotiating a few barriers.  No free lunch here (or anywhere).  We
should recognize this fact instead of trying to bury our head in the
sand.

Re: Is it time to drop ChangeLogs?

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> More generally, there's no way Git could replace ChangeLog style entries,
> because they frequently include information that is not in the diffs. To say
> nothing of the fact that understanding the change from reading Diff hunks is
> much harder, and therefore much less efficient, than from reading a log
> entry which describes the change in plain English.

I'm not saying that commit log entries are not necessary. Having a richly
verbose description of a change is always good. I'm just saying that enforcing
ChangeLog structure is orthogonal.

>> I've never missed not having that ChangeLog data in other projects, of any
>> size.

> Maybe you rarely need to do any forensics. Me, I do it all the time in
> Emacs, and ChangeLog files are a valuable tool in the chest.

I do deep forensics in other projects all the time. I've never noticed having
or not having a ChangeLog as bearing much on the ease of doing so. What
matters more is how well documented a commit is, and how well each commit
sticks to doing just one thing.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Mathieu Lirzin <mthl@gnu.org>
> Date: Mon, 07 Mar 2016 01:22:11 +0100
> Cc: 21998@debbugs.gnu.org, Lars Magne Ingebrigtsen <larsi@gnus.org>
> 
> In my short experience, Change Logs has generally been useful both when
> reading and composing them.  When writing them it helps me structure
> large changes in logical commits that are modelled by the Change Log
> format.  Finally It helps me being precise in my wordings which is not
> trivial for non-native english speakers.
> 
> On a more spiritual side, I think they belong to the zen of contributing
> to a GNU project.  :)

I agree completely.  Writing a log entry in ChangeLog format is an
excellent opportunity for reflecting on the changeset, for summarizing
its intent and final shape, and for making sure nothing was left out.
Writing those entries teaches one discipline, the ability to describe
your changes in just enough detail, and facilitates communications
between members of the development team.  And in loose teams such as
ours, good communications are everything.

As told many times in past discussions, ChangeLog files are also an
excellent first tool for forensics, easy to search with many available
tools.  It is invaluable when you don't have access to the repository,
and a good asset even if you do: traversing the history of a complex
Git repo without missing commits on branches is not a trivial task, it
requires using non-default switches and some rarely used commands and
options.  Even when you do use Git tools, ChangeLog frequently
provides additional important evidence.

So removing ChangeLog files will be a bad blow to our ability to
easily and conveniently research the past, something that is extremely
important in a project with such a rich history, where it's all too
easy to reintroduce a bug if you don't look hard enough at the history
of some code fragment.

People are saying it's an extra barrier to contributing.  That is
true, but so is understanding the Emacs internals, code conventions,
organization of the documentation system, its auxiliary files (like
CONTRIBUTE, NEWS, DEBUG, PROBLEMS, etc.), the release process, and a
few more things.  Having to write ChangeLog entries is an
insignificant addition to the body of knowledge a contributor needs to
master, there's no way around that.  Nor should there be: without
knowing this stuff, you cannot be a useful contributor anyway, as your
contributions will need too much attention from the veterans, who then
will be unable to make more significant contributions due to lack of
time.  We need here contributors who know enough to work on their own
with minimal guidance, who can be trusted to do a good job that
doesn't need to be reviewed too deeply, whose design can be trusted to
be in line with the Emacsy way of doing things.  How can one raise to
this position without learning a lot of project-specific stuff?  You
can't.

Writing ChangeLog entries is just one small part of that.  It's no
accident that people who don't want ChangeLog files more often than
not don't want to write detailed commit log messages, either, and many
times don't know how to write good documentation.  Do we want to
dispense with these as well?  If we drop the ChangeLog files, there's
no way we can explain why we ask for commit log messages in ChangeLog
format, so the next logical step is to drop that as well, and we will
then lose valuable information.  We already are firmly on that path.

Other prominent GNU projects that maintain ChangeLog files in the
repository include GCC, Binutils, GDB, glibc, and Texinfo.  XEmacs
also has it.  Why should Emacs be the first one to plunge into this
adventure?  Why not let others try that first, so that we could later
learn from their mistakes?  We have more important things to do than
waste our scarce resources on side issues, and too few people to do
them.

Let's reinstate the ChangeLog files.  Maintaining them is a negligible
cost; many other projects do that and don't have any trouble.  Unlike
what some people say, merge conflicts in ChangeLog files are very
rare, once you install git-merge-changelog.  We have some important
infrastructure based on ChangeLog files that will become extinct
without them, something that people tend to forget.  We tried to live
without these files for a year; that experiment failed miserably.
It's time to admit that, and fix the mistake we made.

Re: Is it time to drop ChangeLogs?

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> It is IMO a grave mistake to remove parts of our development process just
> because they need to be learned by new contributors. The result will be
> increased burden on the shoulders of those who review the submissions, due
> to the need to coach them, catch their mistakes, ask for re-submissions,
> etc. It will eventually be a net loss, because those contributors will not
> enjoy the need to produce several versions of the patch before it is
> admitted, and the reviewers will not enjoy the extra burden.

I agree with you, Eli. The question: What is the role of _ChangeLog_ in this.
Most other projects don't have them, and I don't hear them complaining. So
what is the exact merit of the format, and why should we continue to require
it in spite of its costs?

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Is it time to drop ChangeLogs?

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> So removing ChangeLog files will be a bad blow to our ability to easily and
> conveniently research the past, something that is extremely important in a
> project with such a rich history, where it's all too easy to reintroduce a
> bug if you don't look hard enough at the history of some code fragment.

Since you are a primary contributor, Eli, your opinion on this does count very
much. If you genuinely think that ChangeLogs makes your life easier, then
maybe we should just table this discussion for another few years.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Mon, 07 Mar 2016 08:29:44 -0800
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > More generally, there's no way Git could replace ChangeLog style entries,
> > because they frequently include information that is not in the diffs. To say
> > nothing of the fact that understanding the change from reading Diff hunks is
> > much harder, and therefore much less efficient, than from reading a log
> > entry which describes the change in plain English.
> 
> I'm not saying that commit log entries are not necessary. Having a richly
> verbose description of a change is always good. I'm just saying that enforcing
> ChangeLog structure is orthogonal.

So you are saying we should come up with a completely new structure?
And you are sure we will do it correctly the first time?  I doubt
that.

ChangeLog structure is a well-known one, it has good support in Emacs
itself, including some quite exotic features (like invoking "C-x 4 a"
from diffs).  Developers that come from other GNU projects are
familiar with this format, and will have no difficulties to fit in.
None of this will be true for any new format we invent.  Sounds like
waste of energy to me.

> I do deep forensics in other projects all the time. I've never noticed having
> or not having a ChangeLog as bearing much on the ease of doing so. What
> matters more is how well documented a commit is, and how well each commit
> sticks to doing just one thing.

What can I say? we have very different experiences.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: emacs-devel@gnu.org,  rms@gnu.org
> Date: Mon, 07 Mar 2016 08:31:56 -0800
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It is IMO a grave mistake to remove parts of our development process just
> > because they need to be learned by new contributors. The result will be
> > increased burden on the shoulders of those who review the submissions, due
> > to the need to coach them, catch their mistakes, ask for re-submissions,
> > etc. It will eventually be a net loss, because those contributors will not
> > enjoy the need to produce several versions of the patch before it is
> > admitted, and the reviewers will not enjoy the extra burden.
> 
> I agree with you, Eli. The question: What is the role of _ChangeLog_ in this.
> Most other projects don't have them, and I don't hear them complaining.

I disagree with this assessment.  I've just provided a list of
prominent GNU projects that do have them, _as_files_in_the_repo_.  The
list of GNU projects that don't have them in the repo, but do adhere
to the ChangeLog format, is much longer.

> So what is the exact merit of the format, and why should we continue
> to require it in spite of its costs?

I think I just answered that in my other mails in this thread.

As for the costs, I think they are negligible, certainly much smaller
than what we had to pay for this experiment during the year since we
decided to try.  And some of the fallout is not yet solved, and we
don't even have a clear idea how to solve it, nor motivation to do
that.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Mathieu Lirzin <mthl@gnu.org>,  emacs-devel@gnu.org
> Date: Mon, 07 Mar 2016 08:33:38 -0800
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > So removing ChangeLog files will be a bad blow to our ability to easily and
> > conveniently research the past, something that is extremely important in a
> > project with such a rich history, where it's all too easy to reintroduce a
> > bug if you don't look hard enough at the history of some code fragment.
> 
> Since you are a primary contributor, Eli, your opinion on this does count very
> much. If you genuinely think that ChangeLogs makes your life easier, then
> maybe we should just table this discussion for another few years.

Thanks.

But the discussion is not the main issue.  We should actually go back
to having an actively maintained ChangeLog file in the repository,
something we stopped doing a year ago.  If there's agreement to that,
I rest my case.

Should we restore manually maintained ChangeLogs (was: Is it time to drop ChangeLogs?)

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> But the discussion is not the main issue. We should actually go back to
> having an actively maintained ChangeLog file in the repository, something we
> stopped doing a year ago. If there's agreement to that, I rest my case.

OK, let's shift this discussion in that direction again.

Given that there are active developers who appreciate and use the ChangeLog
format, I don't think we are going to remove them just yet. Instead, the
question has been raised as to whether we should go back to maintaining
ChangeLog files manually, or continue to generate them from version control as
we do now.

My vote is to continue generating from version control, and Eli would like to
go back to direct maintenance. What do others think?

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Should we restore manually maintained ChangeLogs

[Karl Fogel]

John Wiegley <jwiegley@gmail.com> writes:
>> But the discussion is not the main issue. We should actually go back to
>> having an actively maintained ChangeLog file in the repository, something we
>> stopped doing a year ago. If there's agreement to that, I rest my case.
>
>OK, let's shift this discussion in that direction again.
>
>Given that there are active developers who appreciate and use the ChangeLog
>format, I don't think we are going to remove them just yet. Instead, the
>question has been raised as to whether we should go back to maintaining
>ChangeLog files manually, or continue to generate them from version control as
>we do now.
>
>My vote is to continue generating from version control, and Eli would like to
>go back to direct maintenance. What do others think?

It's no great burden to maintain them manually, once you get used to it; if the developers who are doing the most work would prefer it, then +1 to switching back to the old way of manually maintaining ChangeLogs.

I'll admit, I'm not clear on the advantage myself :-).  The other free software projects I work in use the version control system's logs, without separate ChangeLog files, and it's fine.  Most of those projects use the same commit message conventions (see http://chris.beams.io/posts/git-commit/), and I can't remember any log search I ever wanted to do that I could have done with ChangeLogs but was unable to do with the vc-system logs -- it's the same information, after all, and the Emacs project can enforce the ChangeLog conventions on git commit logs.

I guess one advantage of manually-maintained ChangeLogs is that the entries can still be fixed or improved after being committed, which isn't in practice possible with git commit logs, once the commits have been pushed upstream.

In any case, I don't mean any of the above as a sour-grapes statement.  I really do believe that if the people doing the most work would prefer manually-maintained ChangeLogs, we should just do that.  It's not that hard for developers.  Heck, all I used to do is take my commit message and put it in the ChangeLog too; that's pretty simple.

-Karl

P.S. For those who use 'git log' rather than ChangeLog when exploring history, the helper `kf-git-show-change' in http://svn.red-bean.com/repos/kfogel/trunk/.emacs may be useful.  With point anywhere inside a commit message from 'git log' output, run that command to display the associated change.

Re: Should we restore manually maintained ChangeLogs (was: Is it time to drop ChangeLogs?)

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Mon, 07 Mar 2016 09:16:27 -0800
> Cc: mthl@gnu.org, emacs-devel@gnu.org
> 
> Given that there are active developers who appreciate and use the ChangeLog
> format, I don't think we are going to remove them just yet. Instead, the
> question has been raised as to whether we should go back to maintaining
> ChangeLog files manually, or continue to generate them from version control as
> we do now.
> 
> My vote is to continue generating from version control, and Eli would like to
> go back to direct maintenance. What do others think?

My vote is to go back to having ChangeLog files (could be a single
file, though) in the repository.  My reasons:

  . The current system is clearly not working well: the mistakes in
    the log messages are not corrected, and there's an unsolved
    problem of merging from the release branch to master.

  . Other projects maintain ChangeLog files in the repository: GCC,
    Binutils, GDB, glibc, Texinfo, XEmacs, to name just those that I
    know about.

  . We have maintained ChangeLog files in the repo for years, and I
    don't remember this ever being a problem, provided that a proper
    merge tool (git-merge-changelog for Git) is installed.

Re: Should we restore manually maintained ChangeLogs

[Andy Moreton]

On Mon 07 Mar 2016, John Wiegley wrote:

>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>
>> But the discussion is not the main issue. We should actually go back to
>> having an actively maintained ChangeLog file in the repository, something we
>> stopped doing a year ago. If there's agreement to that, I rest my case.
>
> OK, let's shift this discussion in that direction again.
>
> Given that there are active developers who appreciate and use the ChangeLog
> format, I don't think we are going to remove them just yet. Instead, the
> question has been raised as to whether we should go back to maintaining
> ChangeLog files manually, or continue to generate them from version control as
> we do now.
>
> My vote is to continue generating from version control, and Eli would like to
> go back to direct maintenance. What do others think?

I have no opinion on whether the ChangeLog file should exist in the
repo, but having git commit messages look like ChangeLog entries is
actively harmful.

Most of the content duplicates what the version control system can show
you with greater accuracy, and describes only what was changed. Far too
little attention is given in this format as to *why* a changeset was
committed.

Commit messages should show both what motivates the need for a patch
(bugfix, new feature etc), and why the approach chosen is better than
other possible designs.

The Linux kernel documentation has a good description of what is needed
in a commit message in section 2 of Documentation/SubmittingPatches.

    AndyM

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Just adding my opinion...

Eli Zaretskii <eliz@gnu.org> writes:

> I agree completely. Writing a log entry in ChangeLog format is an
> excellent opportunity for reflecting on the changeset, for summarizing
> its intent and final shape, and for making sure nothing was left out.
> Writing those entries teaches one discipline, the ability to describe
> your changes in just enough detail, and facilitates communications
> between members of the development team. And in loose teams such as
> ours, good communications are everything.

The same applies to writing a good commit log. I have the impression
that the ChangeLog was acting as an excuse for poor commit logs, while
the ChangeLog's contents could be categorized as `lame' on the best of
cases. There is nothing on the ChangeLog that I can't get from the VC
history, on steroids. However, having poor commit logs really hurts.
Emacs has, probably, the worst record on commit log quality of all the
major projects I contributed to.

Writing ChangeLogs feels so pointless (and, hence, irritant) to me that
on the past I refrained from submitting things to Emacs and keep them on
my local branch.

[snip]

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Óscar Fuentes <ofv@wanadoo.es> writes:

> There is nothing on the ChangeLog that I can't get from the VC
> history, on steroids.

Forgot to mention that we could try with adding to VC functions for
easily obtaining the information people get from the ChangeLogs, to the
benefit of those who are not familiarized with Git.

We could also emphasize the tools we already have. `vc-region-history',
for instance. Try to beat that with the ChangeLog.

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

Let's not go back. Maintaining ChangeLog files by hand with each commit 
makes it harder to merge changes due to the inevitable collisions with 
ChangeLog files. Although other projects that continue to use this older 
style (e.g., glibc) work around the problem by requiring contributors to 
deal with the hassles, this is one more barrier to contributions and 
there are better alternatives.

On 03/07/2016 09:50 AM, Eli Zaretskii wrote:
> mistakes in the log messages are not corrected

This is a problem regardless of whether ChangeLog files are updated by 
each commit. Under either approach, contributors often make mistakes in 
their ChangeLog entries, and don't bother to fix them because (let's 
face it) ChangeLog entries are low priority.

> there's an unsolved problem of merging from the release branch to master.

We can solve that problem. (It hasn't been high priority to fix.)

Here's one simple way to fix it: run the automated ChangeLog generator 
only on the master branch, as was originally intended, and use manual 
updates to ChangeLog files in other branches.  (The hassle of manually 
maintaining ChangeLog files on emacs-25 will serve to discourage further 
changes to the emacs-25 branch, which is arguably a good thing. :-) We 
can come up with a better approach later.

> Other projects maintain ChangeLog files in the repository: GCC, 
> Binutils, GDB, glibc, Texinfo, XEmacs, to name just those that I know 
> about.

These are all longstanding projects that haven't changed their 
procedures for ages. New projects don't do this, by and large. (XEmacs 
has gone quiescent, as I understand it, so it's not a good example here.)

> We have maintained ChangeLog files in the repo for years, and I don't 
> remember this ever being a problem, provided that a proper merge tool 
> (git-merge-changelog for Git) is installed. 

I often ran into problems. Yes, git-merge-changelog should reduce the 
number of merge conflicts, but it doesn't eliminate them, and things can 
be even more confusing when there are conflicts anyway. Also, requiring 
git-merge-changelog means that many contributors would have to worry 
about installing and configuring git-merge-changelog, which would be 
more of a hassle for recruiting contributors. In practice under the old 
approach many contributors didn't bother, dealt with the merge conflicts 
by hand, and all too often messed up.

I should mention that git-merge-changelog effectively has no maintainer 
now; if we run into a problem with it, we'll have to maintain it ourselves.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 07 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> I want to choose the path that works for the core developers, and
>> will make Emacs more welcoming to new contributors.
>
> It is IMO a grave mistake to remove parts of our development process
> just because they need to be learned by new contributors.  The result
> will be increased burden on the shoulders of those who review the
> submissions, due to the need to coach them, catch their mistakes, ask
> for re-submissions, etc.

When I submitted my first Emacs patch, I was astonished when I was asked
to re-submit with my commit message essentially duplicated in the
ChangeLog.

How can it be an increased burden if reviewers have to review just one
thing (the commit message) instead of two (commit message and
ChangeLog)?


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 07 Mar 2016 19:07:14 +0100
> 
> > I agree completely. Writing a log entry in ChangeLog format is an
> > excellent opportunity for reflecting on the changeset, for summarizing
> > its intent and final shape, and for making sure nothing was left out.
> > Writing those entries teaches one discipline, the ability to describe
> > your changes in just enough detail, and facilitates communications
> > between members of the development team. And in loose teams such as
> > ours, good communications are everything.
> 
> The same applies to writing a good commit log. I have the impression
> that the ChangeLog was acting as an excuse for poor commit logs, while
> the ChangeLog's contents could be categorized as `lame' on the best of
> cases. There is nothing on the ChangeLog that I can't get from the VC
> history, on steroids.

We can have a practice of copying the same log entries to both
places.  In fact, most people already do.

> Emacs has, probably, the worst record on commit log quality of all the
> major projects I contributed to.

That's far from truth, IME.

> Writing ChangeLogs feels so pointless (and, hence, irritant) to me that
> on the past I refrained from submitting things to Emacs and keep them on
> my local branch.

If you want to write commit log messages, you won't need any
additional effort for ChangeLogs.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 07 Mar 2016 19:13:25 +0100
> 
> Óscar Fuentes <ofv@wanadoo.es> writes:
> 
> > There is nothing on the ChangeLog that I can't get from the VC
> > history, on steroids.
> 
> Forgot to mention that we could try with adding to VC functions for
> easily obtaining the information people get from the ChangeLogs, to the
> benefit of those who are not familiarized with Git.
> 
> We could also emphasize the tools we already have. `vc-region-history',
> for instance. Try to beat that with the ChangeLog.

You are not addressing any of the problems we have with the existing
practice, which attempted to do exactly that.

The current situation is unacceptable.

Re: Is it time to drop ChangeLogs?

[Eric S. Raymond]

Nikolaus Rath <Nikolaus@rath.org>:
> On Mar 07 2016, Eli Zaretskii <eliz@gnu.org> wrote:
> >> I want to choose the path that works for the core developers, and
> >> will make Emacs more welcoming to new contributors.
> >
> > It is IMO a grave mistake to remove parts of our development process
> > just because they need to be learned by new contributors.  The result
> > will be increased burden on the shoulders of those who review the
> > submissions, due to the need to coach them, catch their mistakes, ask
> > for re-submissions, etc.
> 
> When I submitted my first Emacs patch, I was astonished when I was asked
> to re-submit with my commit message essentially duplicated in the
> ChangeLog.
> 
> How can it be an increased burden if reviewers have to review just one
> thing (the commit message) instead of two (commit message and
> ChangeLog)?

This +1000

ChangeLogs are *absurd* under present-day conditions.  They are functional
only as a kind of tribal commitment signal.
-- 
		<a href="http://www.catb.org/~esr/">Eric S. Raymond</a>

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: mthl@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Mon, 7 Mar 2016 11:02:02 -0800
> 
> Maintaining ChangeLog files by hand with each commit makes it harder
> to merge changes due to the inevitable collisions with ChangeLog
> files.

Incorrect!  And the current situation creates _more_ collisions, not
less!

> Although other projects that continue to use this older style (e.g.,
> glibc) work around the problem by requiring contributors to deal
> with the hassles, this is one more barrier to contributions and
> there are better alternatives.

We don't have to be better than the other prominent GNU projects.

> > mistakes in the log messages are not corrected
> 
> This is a problem regardless of whether ChangeLog files are updated by 
> each commit. Under either approach, contributors often make mistakes in 
> their ChangeLog entries, and don't bother to fix them because (let's 
> face it) ChangeLog entries are low priority.

But ChangeLog mistakes can be easily fixed.

> > there's an unsolved problem of merging from the release branch to master.
> 
> We can solve that problem. (It hasn't been high priority to fix.)

We don't know how, and we don't have anyone who is motivated enough to
do that.  And even if and when we do have some solution, it is likely
to be inconvenient and unreliable.

> Here's one simple way to fix it: run the automated ChangeLog generator 
> only on the master branch, as was originally intended, and use manual 
> updates to ChangeLog files in other branches.  (The hassle of manually 
> maintaining ChangeLog files on emacs-25 will serve to discourage further 
> changes to the emacs-25 branch, which is arguably a good thing. :-) We 
> can come up with a better approach later.

Let other projects invent those schemes and test-drive them.  Enough
with these experiments!  They draw the last drops of energy from us,
and they avert the few last veteran contributors we have left.

> > Other projects maintain ChangeLog files in the repository: GCC, 
> > Binutils, GDB, glibc, Texinfo, XEmacs, to name just those that I know 
> > about.
> 
> These are all longstanding projects that haven't changed their 
> procedures for ages.

That's not a bad thing in itself.  The point is, these procedures
work, and all those projects are alive and kicking, and actually make
more frequent releases than we do.

> > We have maintained ChangeLog files in the repo for years, and I don't 
> > remember this ever being a problem, provided that a proper merge tool 
> > (git-merge-changelog for Git) is installed. 
> 
> I often ran into problems. Yes, git-merge-changelog should reduce the 
> number of merge conflicts, but it doesn't eliminate them

Oh, yes, it does.

> requiring git-merge-changelog means that many contributors would
> have to worry about installing and configuring git-merge-changelog,
> which would be more of a hassle for recruiting contributors.

It's a 5-sec configuration, let's not make a mountain out of a
molehill.

> I should mention that git-merge-changelog effectively has no maintainer 
> now; if we run into a problem with it, we'll have to maintain it ourselves.

There's no need for a maintainer, as there are no problems.  I'm using
that program all the time.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Date: Mon, 07 Mar 2016 12:46:30 -0800
> 
> When I submitted my first Emacs patch, I was astonished when I was asked
> to re-submit with my commit message essentially duplicated in the
> ChangeLog.

One is just a copy of the other, so I fail to see a problem, or a
reason for astonishment.

> How can it be an increased burden if reviewers have to review just one
> thing (the commit message) instead of two (commit message and
> ChangeLog)?

No one reviews the same text twice, so doing this will not affect the
review directly.  Indirectly, it will make sure your patches are
cleaner, because summarizing what you did will frequently reveal
subtle blunders and things you forgot.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Forgot to mention that we could try with adding to VC functions for
>> easily obtaining the information people get from the ChangeLogs, to the
>> benefit of those who are not familiarized with Git.
>> 
>> We could also emphasize the tools we already have. `vc-region-history',
>> for instance. Try to beat that with the ChangeLog.
>
> You are not addressing any of the problems we have with the existing
> practice, which attempted to do exactly that.
>
> The current situation is unacceptable.

Of course I'm addressing the problem: ditch changelogs, require quality
log messages and put to good use our VC tool.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> Date: Mon, 7 Mar 2016 16:04:26 -0500
> From: "Eric S. Raymond" <esr@thyrsus.com>
> 
> ChangeLogs are *absurd* under present-day conditions.  They are functional
> only as a kind of tribal commitment signal.

Sorry, that's not a relevant response.  You are not addressing any of
the problems in the current system that I brought up.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 07 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Date: Mon, 07 Mar 2016 12:46:30 -0800
>> 
>> When I submitted my first Emacs patch, I was astonished when I was asked
>> to re-submit with my commit message essentially duplicated in the
>> ChangeLog.
>
> One is just a copy of the other, so I fail to see a problem, or a
> reason for astonishment.

One just being a copy of the other is the reason for astonishment.

>> How can it be an increased burden if reviewers have to review just one
>> thing (the commit message) instead of two (commit message and
>> ChangeLog)?
>
> No one reviews the same text twice, so doing this will not affect the
> review directly.

Well, you are the one who claimed that it makes a difference.

> Indirectly, it will make sure your patches are cleaner, because
> summarizing what you did will frequently reveal subtle blunders and
> things you forgot.

We are talking about the advantages of making a copy of that summary,
after it has been written.

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 07 Mar 2016 22:10:42 +0100
> 
> > You are not addressing any of the problems we have with the existing
> > practice, which attempted to do exactly that.
> >
> > The current situation is unacceptable.
> 
> Of course I'm addressing the problem: ditch changelogs, require quality
> log messages and put to good use our VC tool.

Requiring  something that is not enforceable, with mistakes that
cannot be corrected, is exactly what we were doing for the past year.
That's what isn't working.

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 03/07/2016 10:46 PM, Nikolaus Rath wrote:

> When I submitted my first Emacs patch, I was astonished when I was asked
> to re-submit with my commit message essentially duplicated in the
> ChangeLog.

Was your commit message already written in ChangeLog format?

> How can it be an increased burden if reviewers have to review just one
> thing (the commit message) instead of two (commit message and
> ChangeLog)?

Eli was stating something more general, but to get into specifics: 
often, I only need to read the patch's introduction (high-level 
description), and its ChangeLog entry, to understand it well enough.

A diff contains the same information, but it's usually longer, could be 
harder to read, and often you don't see right away which function a 
given change is being changed, especially if the function is long, and 
the change is right in the middle of it (though that can be alleviated 
with language-specific diff options).

And then, I can look through the diff, compare it against the ChangeLog, 
and see if there are any discrepancies. So the odds of getting some 
unrelated changes (or missing some related ones) is lower.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 07 Mar 2016 13:15:16 -0800
> 
> On Mar 07 2016, Eli Zaretskii <eliz@gnu.org> wrote:
> >> From: Nikolaus Rath <Nikolaus@rath.org>
> >> Date: Mon, 07 Mar 2016 12:46:30 -0800
> >> 
> >> When I submitted my first Emacs patch, I was astonished when I was asked
> >> to re-submit with my commit message essentially duplicated in the
> >> ChangeLog.
> >
> > One is just a copy of the other, so I fail to see a problem, or a
> > reason for astonishment.
> 
> One just being a copy of the other is the reason for astonishment.

I still don't see why.  It's a simple request that is easily
accomplished.  I mentioned several other GNU projects that do the
same.  It's accepted practice.

> >> How can it be an increased burden if reviewers have to review just one
> >> thing (the commit message) instead of two (commit message and
> >> ChangeLog)?
> >
> > No one reviews the same text twice, so doing this will not affect the
> > review directly.
> 
> Well, you are the one who claimed that it makes a difference.

No, you've misunderstood what I wrote.

> > Indirectly, it will make sure your patches are cleaner, because
> > summarizing what you did will frequently reveal subtle blunders and
> > things you forgot.
> 
> We are talking about the advantages of making a copy of that summary,
> after it has been written.

You didn't read what I wrote about the current system.  If we stop
producing ChangeLog files, there will be no reason for having detailed
enough commit log messages, and soon enough there will be no
summaries.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 07 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Cc: emacs-devel@gnu.org
>> Date: Mon, 07 Mar 2016 13:15:16 -0800
>> 
>> On Mar 07 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> >> From: Nikolaus Rath <Nikolaus@rath.org>
>> >> Date: Mon, 07 Mar 2016 12:46:30 -0800
>> >> 
>> >> When I submitted my first Emacs patch, I was astonished when I was asked
>> >> to re-submit with my commit message essentially duplicated in the
>> >> ChangeLog.
>> >
>> > One is just a copy of the other, so I fail to see a problem, or a
>> > reason for astonishment.
>> 
>> One just being a copy of the other is the reason for astonishment.
>
> I still don't see why.  It's a simple request that is easily
> accomplished.  I mentioned several other GNU projects that do the
> same.  It's accepted practice.

Asking me to walk three times around my chair before making the commit
is also a simple request that's easily accomplished. That doesn't make
it any less astonishing. There is just no apparent good reason for this
duplication. Thus it astonishes.

>> >> How can it be an increased burden if reviewers have to review just one
>> >> thing (the commit message) instead of two (commit message and
>> >> ChangeLog)?
>> >
>> > No one reviews the same text twice, so doing this will not affect the
>> > review directly.
>> 
>> Well, you are the one who claimed that it makes a difference.
>
> No, you've misunderstood what I wrote.
>
>> > Indirectly, it will make sure your patches are cleaner, because
>> > summarizing what you did will frequently reveal subtle blunders and
>> > things you forgot.
>> 
>> We are talking about the advantages of making a copy of that summary,
>> after it has been written.
>
> You didn't read what I wrote about the current system.  If we stop
> producing ChangeLog files, there will be no reason for having detailed
> enough commit log messages, and soon enough there will be no
> summaries.

I don't see how one follows from the other. Why isn't whatever measure
you are currently take to ensure detailed ChangeLog files equally
suitable to ensure detailed commit messages?


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Of course I'm addressing the problem: ditch changelogs, require quality
>> log messages and put to good use our VC tool.
>
> Requiring  something that is not enforceable, with mistakes that
> cannot be corrected,

Introduce code reviews. Don't give commit access to the "golden"
branches to everyone, just to a few top contributors and reviewers.

[snip]

Re: Is it time to drop ChangeLogs?

[Lars Magne Ingebrigtsen]

Nikolaus Rath <Nikolaus@rath.org> writes:

> When I submitted my first Emacs patch, I was astonished when I was asked
> to re-submit with my commit message essentially duplicated in the
> ChangeLog.

And I think this is essential input.  We are alienating contributors by
insisting on our weird (ok, "unusual") format of the meta data of the
patches.

The ChangeLog is useful for some of us.  But I would contend that its
usefulness doesn't trump all the contributors we have discouraged by
insisting on our quite singular submission format.

`M-x vc-region-history' is much, much more useful, in my opinion, than
perusing ChangeLogs.  Especially if the text people write is more than a
reiteration of what the VC logs already tell us.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

Eli Zaretskii <eliz@gnu.org> writes:
>You didn't read what I wrote about the current system.  If we stop
>producing ChangeLog files, there will be no reason for having detailed
>enough commit log messages, and soon enough there will be no
>summaries.

This has not been the case in the other projects I work in (none of which use ChangeLog files).  Log message discipline is maintained socially for commit messages just as it would be for ChangeLog entries.

Re: Should we restore manually maintained ChangeLogs

[Dmitry Gutov]

On 03/07/2016 11:06 PM, Eli Zaretskii wrote:

>> Maintaining ChangeLog files by hand with each commit makes it harder
>> to merge changes due to the inevitable collisions with ChangeLog
>> files.
>
> Incorrect!  And the current situation creates _more_ collisions, not
> less!

Only when merging between branches.

Other than that, only a select group of people needs to bother: those 
who make mistakes, and those who feel a general need to clean up.

As a relatively careful committer, I've only had to correct the entries 
a few times, and I've been enjoying the lack of collisions quite a bit.

>>> mistakes in the log messages are not corrected
>>
>> This is a problem regardless of whether ChangeLog files are updated by
>> each commit. Under either approach, contributors often make mistakes in
>> their ChangeLog entries, and don't bother to fix them because (let's
>> face it) ChangeLog entries are low priority.
>
> But ChangeLog mistakes can be easily fixed.

In the current approach, as well.

>> We can solve that problem. (It hasn't been high priority to fix.)
>
> We don't know how, and we don't have anyone who is motivated enough to
> do that.  And even if and when we do have some solution, it is likely
> to be inconvenient and unreliable.

I think we should wait and see until the work really transitions back to 
master. The motivation must rise.

> Let other projects invent those schemes and test-drive them.  Enough
> with these experiments!  They draw the last drops of energy from us,
> and they avert the few last veteran contributors we have left.

Has the current experiment really sucked too much energy from anyone, 
aside from the implementors? Yes, there is a problem on master, but 
we're still mostly expected to use, and polish, emacs-25.

> That's not a bad thing in itself.  The point is, these procedures
> work, and all those projects are alive and kicking, and actually make
> more frequent releases than we do.

For all we know, they might be thriving despite this practice.

>>> We have maintained ChangeLog files in the repo for years, and I don't
>>> remember this ever being a problem, provided that a proper merge tool
>>> (git-merge-changelog for Git) is installed.
>>
>> I often ran into problems. Yes, git-merge-changelog should reduce the
>> number of merge conflicts, but it doesn't eliminate them
>
> Oh, yes, it does.

Not in my experience either. I've still had collisions, and even when 
git-merge-changelog resolved them, it often put my entry in the middle 
of the file, whereas I usually needed it to be at the top. Leading to 
extra manual labor.

>> requiring git-merge-changelog means that many contributors would
>> have to worry about installing and configuring git-merge-changelog,
>> which would be more of a hassle for recruiting contributors.
>
> It's a 5-sec configuration, let's not make a mountain out of a
> molehill.

It was longer for me. But either way, it's more hassle for a random 
contributor than the current system. If it can be fixed, Someone should.

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

Dmitry Gutov <dgutov@yandex.ru> writes:
>On 03/07/2016 10:46 PM, Nikolaus Rath wrote:
>> How can it be an increased burden if reviewers have to review just one
>> thing (the commit message) instead of two (commit message and
>> ChangeLog)?
>
>Eli was stating something more general, but to get into specifics:
>often, I only need to read the patch's introduction (high-level
>description), and its ChangeLog entry, to understand it well enough.
>
>A diff contains the same information, but it's usually longer, could
>be harder to read, and often you don't see right away which function a
>given change is being changed, especially if the function is long, and
>the change is right in the middle of it (though that can be alleviated
>with language-specific diff options).
>
>And then, I can look through the diff, compare it against the
>ChangeLog, and see if there are any discrepancies. So the odds of
>getting some unrelated changes (or missing some related ones) is
>lower.

So much about this conversation is baffling to me :-).

Let's please take it for granted that everyone who's suggesting to get rid of ChangeLog files is also assuming that we would, of course, use the same conventions for writing git commit messages that we would use for writing ChangeLog entries.  (As far as I can tell, everyone who wants to get rid of ChangeLog files has either expressed that assumption explicitly or implied it pretty clearly.  )

Of course, one can view the diff any time, and a good log entry should prepare the reader's mind for comprehending the diff.  Whether that entry's text is stored in just the VC system commit logs, or in the commit log *and* in a ChangeLog entry, shouldn't affect one's ability to use the text to understand the change.  Dmitry, you seem to be saying that a "patch's introduction (high-level description), and its ChangeLog entry" are two different things.  That's confusing, to me at least.  The commit message should be the introduction to the patch *and* should be the same as the ChangeLog entry (if one is keeping separate ChangeLog files).  At least, that's what those of us on the no-ChangeLogs side are saying.

Again, if the majority of people like Eli who do tons of work around here would prefer to keep ChangeLogs, then I'm in favor of doing so.  But it would be nice to be de-confused about the arguments for it.  Many of those arguments seem to be based on the idea that the ChangeLog entry is somehow *different* from the VC system commit message -- which no one is proposing, as far as I can tell.

Best regards,
-Karl

Re: Is it time to drop ChangeLogs?

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 596 bytes --]

On 03/07/2016 04:29 PM, Óscar Fuentes wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
>>> >> Of course I'm addressing the problem: ditch changelogs, require quality
>>> >> log messages and put to good use our VC tool.
>> >
>> > Requiring  something that is not enforceable, with mistakes that
>> > cannot be corrected,
> Introduce code reviews. Don't give commit access to the "golden"
> branches to everyone, just to a few top contributors and reviewers.

Wouldn't that be a very big change in the way Emacs development works, much bigger than deciding for or against Changelogs?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 03/07/2016 11:43 PM, Karl Fogel wrote:

> Let's please take it for granted that everyone who's suggesting to get rid of ChangeLog files is also assuming that we would, of course, use the same conventions for writing git commit messages that we would use for writing ChangeLog entries.  (As far as I can tell, everyone who wants to get rid of ChangeLog files has either expressed that assumption explicitly or implied it pretty clearly.  )

Not so. Many messages in this thread are saying different.

> Dmitry, you seem to be saying that a "patch's introduction (high-level description), and its ChangeLog entry" are two different things.

A ChangeLog entry has a well-defined format. It can also start with a 
free-form description. Whether the latter is a part of it or not, is not 
100% important. Point is, it is a distinct entity, and I can refer to it.

The high-level description may also come separately, in a message to the 
bug tracker or the mailing list. That one may go into more detail.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Clément Pit--Claudel <clement.pit@gmail.com> writes:

>> Introduce code reviews. Don't give commit access to the "golden"
>> branches to everyone, just to a few top contributors and reviewers.
>
> Wouldn't that be a very big change in the way Emacs development works,
> much bigger than deciding for or against Changelogs?

Not really. Emacs always accepted patches from casual contributors.

Another option for correcting errors on commit logs is `git notes', but
I don't know about its implications.

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

Dmitry Gutov <dgutov@yandex.ru> writes:
>On 03/07/2016 11:43 PM, Karl Fogel wrote:
>> Let's please take it for granted that everyone who's suggesting to
>> get rid of ChangeLog files is also assuming that we would, of
>> course, use the same conventions for writing git commit messages
>> that we would use for writing ChangeLog entries.  (As far as I can
>> tell, everyone who wants to get rid of ChangeLog files has either
>> expressed that assumption explicitly or implied it pretty clearly.
>> )
>
>Not so. Many messages in this thread are saying different.

I looked again -- who are these "many"?  I don't see any messages from someone who both advocates that we get rid of ChangeLogs *and* suggests that we should not use the same format for commit messages as we currently do for ChangeLogs.  (Andy Moreton suggested something about how we could have improved standards about what we put in a change description in general, but that's different and orthogonal to this discussion.)

Please let me know if I missed a post, though.

>A ChangeLog entry has a well-defined format. It can also start with a
>free-form description. Whether the latter is a part of it or not, is
>not 100% important. Point is, it is a distinct entity, and I can refer
>to it.

A commit message is a distinct entity, and you can refer to it.

>The high-level description may also come separately, in a message to
>the bug tracker or the mailing list. That one may go into more detail.

Well, one can just include that in the commit message.  My point is just that if it's appropriate for a ChangeLog entry, then it's appropriate for a commit message, and vice versa.  I mean, at least for my commits, the former have always just been a literal copy of the latter, even when doing it manually :-).

Best regards,
-Karl

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 03/08/2016 12:24 AM, Karl Fogel wrote:

> I looked again -- who are these "many"?

ESR, Óscar, Andy. Maybe others.

> (Andy Moreton suggested something about how we could have improved standards about what we put in a change description in general, but that's different and orthogonal to this discussion.)

Not necessarily improved, but different standards (see "actively 
harmful" in his message).

> Well, one can just include that in the commit message.  My point is just that if it's appropriate for a ChangeLog entry, then it's appropriate for a commit message, and vice versa.  I mean, at least for my commits, the former have always just been a literal copy of the latter, even when doing it manually :-).

How is this relevant? I was describing how the Change Log entries 
submitted together with a patch help reviewing it now. So of course I'm 
referring to the current practice, where the commit message includes the 
Change Log entry.

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 03/08/2016 12:45 AM, Dmitry Gutov wrote:
> On 03/08/2016 12:24 AM, Karl Fogel wrote:
>
>> I looked again -- who are these "many"?
>
> ESR, Óscar, Andy. Maybe others.

Also John, of course. The Emacs Maintainer one.

Re: Should we restore manually maintained ChangeLogs

[Dmitry Gutov]

On 03/07/2016 08:05 PM, Andy Moreton wrote:

> Commit messages should show both what motivates the need for a patch
> (bugfix, new feature etc), and why the approach chosen is better than
> other possible designs.

Not to say there's no place for improvement in our commit messages, but 
keeping the actual explanation separately (in a bug report, in in a 
mailing list message), and linking to it from the commit message is 
entirely fine in my book. In certain cases, it's hard to avoid that 
anyway (the explanation is too verbose/hard to summarize/etc).

> The Linux kernel documentation has a good description of what is needed
> in a commit message in section 2 of Documentation/SubmittingPatches.

It's a fine description, but it's huge compared to the Change Log format 
description (thus raising the barrier of entry if we decide to use it 
officially), and ultimately, it leaves a lot up to the patch author's 
judgment anyway.

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

Dmitry Gutov <dgutov@yandex.ru> writes:
>On 03/08/2016 12:24 AM, Karl Fogel wrote:
>> I looked again -- who are these "many"?
>
>ESR, Óscar, Andy. Maybe others.
>Also John, of course. The Emacs Maintainer one.

Uh, hmm.  We are having some sort of miscommunication.  I can't figure out what it is, because I expressed myself as precisely as I can -- and when I look at their messages, in this thread and the related thread, what they're saying is compatible with what I described them as saying.

<shrug> :-)

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/07/2016 01:06 PM, Eli Zaretskii wrote:
>> Maintaining ChangeLog files by hand with each commit makes it harder
>> to merge changes due to the inevitable collisions with ChangeLog
>> files.
> Incorrect!  And the current situation creates _more_ collisions, not
> less!

My own experience is otherwise. For the kinds of development I do, I 
rarely see ChangeLog screwups now, whereas I used to see them routinely. 
I far prefer the current approach. Of course our approach can be 
improved (in particular merging from emacs-25 to master doesn't work 
well now), but let's not throw out the baby with the bathwater.

> We don't have to be better than the other prominent GNU projects.

I'd be happy if we were merely as good as the other prominent GNU 
projects that generate ChangeLog entries automatically. As things stand, 
due to our attempt to cater to all sides of this disagreement, we have 
an approach that satisfies nobody.

> ChangeLog mistakes can be easily fixed. 

That's true under both the old and the new regimes.

> Let other projects invent those schemes and test-drive them. Enough 
> with these experiments!

I'd rather just do what coreutils, grep, tar, etc. use. It's just as 
simple for ordinary committers and it would not involve duplication in 
patches or experimentation with maintenance procedures. I could fairly 
easily change the master branch to do that.

Even simpler would be to do what Guile does: it dispenses with 
ChangeLogs entirely. With Guile if you want something like a ChangeLog 
you run "git log".

If neither of the above two approaches suffice, we can always fall back 
on my previous email's proposal. It's not that experiemental, as it says 
to use the new produre on master and the old procedure on emacs-25. The 
new procedure works well enough within a single master branch.

Of all the above, though, Guile's approach is the simplest and is 
probably the best for us overall.

> these procedures work, and all those projects are alive and kicking, 
> and actually make more frequent releases than we do.

XEmacs is not really alive and kicking.

Projects like GCC and glibc have more resources than we do, and can 
afford to insist on more-expert contributions that involve 
harder-to-generate patch formats. We do not have that luxury.

>
>> I often ran into problems. Yes, git-merge-changelog should reduce the
>> number of merge conflicts, but it doesn't eliminate them
> Oh, yes, it does.

Not in my experience, or in Dmitry's. It's a fine program, but it 
sometimes makes mistakes and they can be a pain to fix.

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 03/08/2016 01:24 AM, Karl Fogel wrote:
> Dmitry Gutov <dgutov@yandex.ru> writes:
>> On 03/08/2016 12:24 AM, Karl Fogel wrote:
>>> I looked again -- who are these "many"?
>>
>> ESR, Óscar, Andy. Maybe others.
>> Also John, of course. The Emacs Maintainer one.
>
> Uh, hmm.  We are having some sort of miscommunication.  I can't figure out what it is, because I expressed myself as precisely as I can -- and when I look at their messages, in this thread and the related thread, what they're saying is compatible with what I described them as saying.

You're saying everyone agrees to at least continue including Change Log 
entries in commit messages. The persons above are counter-examples.

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> I'd like to open this up to discussion on emacs-devel, so that we hear from
> our other developers. What do you all think about ChangeLogs, and their value
> to you in your work on Emacs?

I couldn't care less about ChangeLog *files*, really.

I think it makes sense to provide them in tarballs (to replace the Git
metadata), but I've personally stopped using them years ago when "bzr
log" started being fast enough.

But OTOH I strongly oppose getting rid of the ChangeLog format in commit
messages.  My experience with commit messages is that it's very
difficult for people to know what should go in a commit message
(especially so for less experienced coders, but even for experienced
coders it's difficult to guess what information will turn out to be
important).

The ChangeLog format's rules gives a very good baseline for that.
Even if some of that info seems redundant, the act of describing all the
parts that are changed, encourages the coder to pay attention to every
part of the diff she's about to commit.

The original ChangeLog format lacked a high-level description of the
intention behind the changes, so the rules we currently use in Emacs is
to use something of the form:

    <One line summary>

    <High-level description of the purpose of the change>

    <ChangeLog-style details>

If we were to drop the "ChangeLog-style details", we'd end up with very
little guidance as to what to put in the commit message, so we'd have
even more trouble getting contributors to start providing acceptable
commit messages.

My experience (when trying to figure out, years later, why such and such
line of code was changed in that particular way) is that ChangeLog-style
details don't always give you the info you want, but at least they tend
to say *something* about your particular line (because they have to talk
about every part of the diff), whereas the high level description is
too often too high-level and hence not detailed enough for that.


        Stefan

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> I find that a strange thing to say.  I frequently search the ChangeLog
> for the name of a function, to find out when it was last changed.

I use vc-region-history for that.  I find it addictive.


        Stefan

Code reviews (was: Is it time to drop ChangeLogs?)

[Stefan Monnier]

> Introduce code reviews.  Don't give commit access to the "golden"
> branches to everyone, just to a few top contributors and reviewers.

We already have a fair bit of patches submitted and lingering in limbo
forever until someone (almost always the same someone, BTW) finally
loses hope that some of the other contributors take care of it.

If we could switch to a system where every patch is reviewed before
commit, that'd be great.  My own impression is that it will kill the
development pace because too few people are willing to spend the
corresponding efforts.

That's why I've followed a practice of giving out write access very
liberally, with "post-commit spot-check reviews" instead.  Indeed, it
means that errors in commit messages can't be fixed (we can fix them in
the ChangeLog files, admittedly, but since I don't use them it doesn't
help me).

Maybe we could have a half-way system, where commits are pushed to
a branch that is "not fast-forward-only", and this branch is then
auto-merged to the real (fast-forward-only) master branch after a delay
(one day, maybe?) to give time to fix mess ups before they're cast
in stone.


        Stefan

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> When I submitted my first Emacs patch, I was astonished when I was asked
> to re-submit with my commit message essentially duplicated in the
> ChangeLog.

I can't remember ever asking this kind of duplication.  We used to
duplicate the ChangeLog-info in the ChangeLog file and in the commit
message (tho we stopped doing it a year ago), but we never asked for it
to be duplicated in the email messages that submits a patch.

So I have the impression that I'm misunderstanding you.  Could you
describe more precisely the kind of duplication you're talking about?


        Stefan

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 07 Mar 2016 13:28:21 -0800
> 
> Asking me to walk three times around my chair before making the commit
> is also a simple request that's easily accomplished. That doesn't make
> it any less astonishing. There is just no apparent good reason for this
> duplication. Thus it astonishes.

The reasons exist, and has been described.  You may not agree with
them, perhaps due to different experience, but that doesn't mean they
aren't valid.

> > You didn't read what I wrote about the current system.  If we stop
> > producing ChangeLog files, there will be no reason for having detailed
> > enough commit log messages, and soon enough there will be no
> > summaries.
> 
> I don't see how one follows from the other. Why isn't whatever measure
> you are currently take to ensure detailed ChangeLog files equally
> suitable to ensure detailed commit messages?

Because if we continue going the way we do, we are sending a very
specific message to the future contributors.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 07 Mar 2016 22:29:27 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Of course I'm addressing the problem: ditch changelogs, require quality
> >> log messages and put to good use our VC tool.
> >
> > Requiring  something that is not enforceable, with mistakes that
> > cannot be corrected,
> 
> Introduce code reviews. Don't give commit access to the "golden"
> branches to everyone, just to a few top contributors and reviewers.

We don't have manpower for that.

Re: Is it time to drop ChangeLogs?

[Christopher Allan Webber]

Stefan Monnier writes:

>> I'd like to open this up to discussion on emacs-devel, so that we hear from
>> our other developers. What do you all think about ChangeLogs, and their value
>> to you in your work on Emacs?
>
> The ChangeLog format's rules gives a very good baseline for that.
> Even if some of that info seems redundant, the act of describing all the
> parts that are changed, encourages the coder to pay attention to every
> part of the diff she's about to commit.
>

[...]

> My experience (when trying to figure out, years later, why such and such
> line of code was changed in that particular way) is that ChangeLog-style
> details don't always give you the info you want, but at least they tend
> to say *something* about your particular line (because they have to talk
> about every part of the diff), whereas the high level description is
> too often too high-level and hence not detailed enough for that.
>
>
>         Stefan

Unlike most others to this list, I am relatively new to ChangeLog style
messages.  I only started making commits with them over the last year,
because of contributing to Guix (and less so to Guile).

I have mixed feelings about the whole thing, probably learning towards
positive, but not entirely.  Here's maybe a summary of a lot of the
points I'm seeing raised (and my own conflicted thoughts too):

 - It does seem like there's duplication of data between vc commits and
   ChangeLog files.  So for the most part, just having that data be *in*
   the commit message makes most sense, and thus so does extracting it
   after to produce the ChangeLog.
 - Unless you make a mistake in the commit message of course!  Then I
   don't know what you do.  Nobody in Guix seems too worried about it,
   so...
 - ChangeLog style commits did take me a bit to do right, and I did not
   find the GNU Coding Standards easy for me to understand clearly.  So
   initially I mimed the look and feel from other committers in Guix,
   and eventually it began to make sense.  This does seem like an extra
   barrier, but it does seem like the kind of workflow I go through for
   most skills which eventually become intuitive anyway.
 - That said I do think I developed better, cleaner commit message
   hygeine after adopting ChangeLog style commit messages.
 - Though some of it still feels redundant!
 - Prior to this, I mostly used "git log" and vc-annotate to find
   things.  (vc-annotate is awesome, btw.)  Usually it worked just
   great.  Though... when it didn't, it could be a big headache.
   Refactoring of files into other files, or large reindenting of Python
   files for instance, are hard to reason about after those kinds of
   commits.

So is it worth it?  I don't know!  But those seem like the main points
to me, from my own (relatively newcomer to the format) experiences.

 - Chris

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 07 Mar 2016 22:30:04 +0100
> 
> Nikolaus Rath <Nikolaus@rath.org> writes:
> 
> > When I submitted my first Emacs patch, I was astonished when I was asked
> > to re-submit with my commit message essentially duplicated in the
> > ChangeLog.
> 
> And I think this is essential input.  We are alienating contributors by
> insisting on our weird (ok, "unusual") format of the meta data of the
> patches.

I don't think we are alienating them.  The difference wrt what we
request now is almost nonexistent.  And it's not unusual to request
ChangeLog entries, many projects do that.

So the change I propose is not as dramatic as it sounds.  It does fix
a grave problem we have lately, and it will allow us to switch to more
elaborate workflows in the future.

> The ChangeLog is useful for some of us.  But I would contend that its
> usefulness doesn't trump all the contributors we have discouraged by
> insisting on our quite singular submission format.

It's not singular.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Mon, 7 Mar 2016 16:56:17 -0500
> 
> > Introduce code reviews. Don't give commit access to the "golden"
> > branches to everyone, just to a few top contributors and reviewers.
> 
> Wouldn't that be a very big change in the way Emacs development works, much bigger than deciding for or against Changelogs?

Of course, it will.  More importantly, we simply don't have the
resources for that.  The result will be a total slowdown of
development, a never-ending patch queue, and burn-out of lead
developers.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 07 Mar 2016 23:19:03 +0100
> 
> Another option for correcting errors on commit logs is `git notes', but
> I don't know about its implications.

Nobody does.  We shouldn't consider any more experiments in this area
until someone else uses these methods and proves they are viable in
our case.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 07 Mar 2016 21:59:02 -0500
> 
> > I'd like to open this up to discussion on emacs-devel, so that we hear from
> > our other developers. What do you all think about ChangeLogs, and their value
> > to you in your work on Emacs?
> 
> I couldn't care less about ChangeLog *files*, really.
> 
> I think it makes sense to provide them in tarballs (to replace the Git
> metadata), but I've personally stopped using them years ago when "bzr
> log" started being fast enough.

We tried that, for a full year.  If it worked, we could continue using
it.  But it doesn't work, even when we have only one active branch.
So continuing using it is out of the question if we want to have the
files available in the tarballs.

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

>> I think it makes sense to provide them in tarballs (to replace the Git
>> metadata), but I've personally stopped using them years ago when "bzr
>> log" started being fast enough.

> We tried that, for a full year.  If it worked, we could continue using
> it.  But it doesn't work, even when we have only one active branch.
> So continuing using it is out of the question if we want to have the
> files available in the tarballs.

No, what we do currently is different: we auto-generate the ChangeLog
and then commit it into the Git.  What I think we should do is to never
commit it into the Git, and only auto-generate it into the tarball.
Like what we do for GNU ELPA packages (where the log is extracted via
"git log" when building the package and added to the package's content).

That makes "fixing" ChangeLog entries harder/impossible.  But it gets us
rid of the merge-mess once and for all.


        Stefan

Re: Code reviews

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Introduce code reviews.  Don't give commit access to the "golden"
>> branches to everyone, just to a few top contributors and reviewers.
>
> We already have a fair bit of patches submitted and lingering in limbo
> forever until someone (almost always the same someone, BTW) finally
> loses hope that some of the other contributors take care of it.
>
> If we could switch to a system where every patch is reviewed before
> commit, that'd be great.  My own impression is that it will kill the
> development pace because too few people are willing to spend the
> corresponding efforts.
>
> That's why I've followed a practice of giving out write access very
> liberally, with "post-commit spot-check reviews" instead.  Indeed, it
> means that errors in commit messages can't be fixed (we can fix them in
> the ChangeLog files, admittedly, but since I don't use them it doesn't
> help me).
>
> Maybe we could have a half-way system, where commits are pushed to
> a branch that is "not fast-forward-only",

Something like this is what I meant with "Don't give commit access to
the golden branches to everyone". Anyways, you can't expect having high
quality commit logs (or VC history at all, take a look at the DAG to see
what I mean) and give write access liberally. Having ChangeLogs as a
compromise is absurd: you have to proof-read and fix them anyway.
Correct or reject the real thing (the commit) instead.

I guess that asking for a review queue is out of the question, although
I'm afraid that it would not turn to be a good thing since some people
here tend to be quite picky when reviewing foreign contributions (with
the best of the intentions, but I can attest from personal experience
that being the subject of one of those reviews can be disheartening.)

[snip]

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Introduce code reviews. Don't give commit access to the "golden"
>> branches to everyone, just to a few top contributors and reviewers.
>
> We don't have manpower for that.

I find perplexing that we have manpower for reviewing ChangeLogs but not
for reviewing commit logs *instead*.

And is it so super-important to have typo-free commit logs? We are not
even using `git notes'.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Another option for correcting errors on commit logs is `git notes', but
>> I don't know about its implications.
>
> Nobody does.  We shouldn't consider any more experiments in this area
> until someone else uses these methods and proves they are viable in
> our case.

How could *someone else* prove the viability of something in *our* case?
You always can argue that his project is not Emacs... This is like your
argument that Emacs should keep using ChangeLogs because gcc uses
ChangeLogs... and gcc should keep using ChangeLogs because Emacs uses
ChangeLogs, right?.

`git notes' is a documented feature available since a long time ago. It
can turn the immutability of the commit log into a non-issue if you wish
to produce a proofread ChangeLog to include in the tarball.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 08 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Cc: emacs-devel@gnu.org
>> Date: Mon, 07 Mar 2016 13:28:21 -0800
>> 
>> Asking me to walk three times around my chair before making the commit
>> is also a simple request that's easily accomplished. That doesn't make
>> it any less astonishing. There is just no apparent good reason for this
>> duplication. Thus it astonishes.
>
> The reasons exist, and has been described.  You may not agree with
> them, perhaps due to different experience, but that doesn't mean they
> aren't valid.

Yeah, but what exactly is your point here? I was pointing out that this
practice is astonishing for new contributors. You didn't understand why,
so I explained that it's astonishing because to a new contributor it
seems pointless. Now you assert that there are good reasons for it, but
that really doesn't change that, to new contributors, it appears
pointless and is astonishing.

The only way to change that is probably to come up with a well-written,
concise summary of these reasons and include it whenever a submitted
patch has to be refused due to a missing changelog entry.

>>> You didn't read what I wrote about the current system.  If we stop
>>> producing ChangeLog files, there will be no reason for having detailed
>>> enough commit log messages, and soon enough there will be no
>>> summaries.
>> 
>> I don't see how one follows from the other. Why isn't whatever measure
>> you are currently take to ensure detailed ChangeLog files equally
>> suitable to ensure detailed commit messages?
>
> Because if we continue going the way we do, we are sending a very
> specific message to the future contributors.

Sorry, I still don't follow your reasoning. I still consider myself a
potential contributor (I didn't contribute much beyond that first little
foray), and not requiring me to copy my commit message into the
ChangeLog is not going to affect the quality of the commit measures in
any way. The only "message" that I get from this is "Great, sending
patches for Emacs doesn't require me to do a copy & paste anymore".

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Code reviews

[Stefan Monnier]

> Something like this is what I meant with "Don't give commit access to
> the golden branches to everyone". Anyways, you can't expect having high
> quality commit logs (or VC history at all, take a look at the DAG to see
> what I mean) and give write access liberally.

Indeed (and yes, the DAG shape is another one of those problems, made
worse by the fact that Git doesn't help us).

> Having ChangeLogs as a compromise is absurd: you have to proof-read
> and fix them anyway.  Correct or reject the real thing (the
> commit) instead.

Agreed.  My choice has been to live with unfixed (and hence
lower-quality) commit messages [even if some people here do make the
effort to fix them in the ChangeLog, I don't benefit from it because
I don't use the ChangeLog].

> I guess that asking for a review queue is out of the question, although
> I'm afraid that it would not turn to be a good thing since some people
> here tend to be quite picky when reviewing foreign contributions

True enough on both counts.  A code review system should allow both to
"reject a commit and ask the author to provide a new patch" and to "fix
the commit ourselves" since in many cases it's easier both for the
contributors and the reviewer.


        Stefan

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 07 2016, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> When I submitted my first Emacs patch, I was astonished when I was asked
>> to re-submit with my commit message essentially duplicated in the
>> ChangeLog.
>
> I can't remember ever asking this kind of duplication.  We used to
> duplicate the ChangeLog-info in the ChangeLog file and in the commit
> message (tho we stopped doing it a year ago), but we never asked for it
> to be duplicated in the email messages that submits a patch.
>
> So I have the impression that I'm misunderstanding you.  Could you
> describe more precisely the kind of duplication you're talking about?

No, you understood me correctly. However, I've just looked this up again
and it turns out I was actually submitting a patch to Gnus, not
Emacs. It was eventually committed by Eric Abrahamsen to the Gnus git
repo, who first asked me to re-submit my patch with a commit message
that was formatted like a Changelog entry, and to duplicate the message
in the ChangeLog itself. It seemed to me that he considered that a
questionable practice himself, but said it was obligatory because Gnus
closely followed the Emacs standards here.


Unfortunately that discussion never made it into the bug
(http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20956), so I'm
paraphrasing from memory. But I'm pretty sure I'm remembering correctly,
because I was so surprised by this.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[David Engster]

Óscar Fuentes writes:
> `git notes' is a documented feature available since a long time ago. It
> can turn the immutability of the commit log into a non-issue if you wish
> to produce a proofread ChangeLog to include in the tarball.

I can't imagine that you have ever really used 'git notes'. It's a hack
with terrible usability and no merge support.

-David

Re: Is it time to drop ChangeLogs?

[Eric Abrahamsen]

Nikolaus Rath <Nikolaus@rath.org> writes:

> On Mar 07 2016, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>> When I submitted my first Emacs patch, I was astonished when I was asked
>>> to re-submit with my commit message essentially duplicated in the
>>> ChangeLog.
>>
>> I can't remember ever asking this kind of duplication.  We used to
>> duplicate the ChangeLog-info in the ChangeLog file and in the commit
>> message (tho we stopped doing it a year ago), but we never asked for it
>> to be duplicated in the email messages that submits a patch.
>>
>> So I have the impression that I'm misunderstanding you.  Could you
>> describe more precisely the kind of duplication you're talking about?
>
> No, you understood me correctly. However, I've just looked this up again
> and it turns out I was actually submitting a patch to Gnus, not
> Emacs. It was eventually committed by Eric Abrahamsen to the Gnus git
> repo, who first asked me to re-submit my patch with a commit message
> that was formatted like a Changelog entry, and to duplicate the message
> in the ChangeLog itself. It seemed to me that he considered that a
> questionable practice himself, but said it was obligatory because Gnus
> closely followed the Emacs standards here.
>
>
> Unfortunately that discussion never made it into the bug
> (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20956), so I'm
> paraphrasing from memory. But I'm pretty sure I'm remembering correctly,
> because I was so surprised by this.

Not duplicating it in the email, just commit message + ChangeLog
duplication. This may be getting conflated with me asking for the commit
to be emailed as a git-format-patch attachment, which was just because I
was lazy (and it also preserves the author/committer distinction).

But yes, Gnus was doing the ChangeLog dance right up until the repo
merged with Emacs -- I pushed a couple of changes there just before
that, neglecting to include the ChangeLog, and someone went and cleaned
up after me. Embarrassing.

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

Stefan Monnier wrote:
> What I think we should do is to never
> commit it into the Git, and only auto-generate it into the tarball.

That's easy to do -- it's what coreutils etc. do -- and is what I implemented at 
first for Emacs. It should be easy to change Emacs to do things that way; the 
change would mostly consist of removing gorp from the top-level Makefile.

Re: Code reviews

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Introduce code reviews.  Don't give commit access to the "golden"
>> branches to everyone, just to a few top contributors and reviewers.
>
> We already have a fair bit of patches submitted and lingering in limbo
> forever until someone (almost always the same someone, BTW) finally
> loses hope that some of the other contributors take care of it.
>
> If we could switch to a system where every patch is reviewed before
> commit, that'd be great.  My own impression is that it will kill the
> development pace because too few people are willing to spend the
> corresponding efforts.

I think that there is a half-way house here. You extensively reviewed
the first change that I made to the core of Emacs -- as someone who
didn't know the internals or, indeed, even C, there is no way on earth
that I could have made that commit without. Other commits are less of
problem.

Giving people different permissions on different branches seems to make
sense. Overtime, people move toward master/emacs-25.

Having said that, while I found your feedback and the overall experience
very positive, I did feel the lack of a nicer pull/merge request system.
Tools such as gitlab or gerrit would have helped.



> That's why I've followed a practice of giving out write access very
> liberally, with "post-commit spot-check reviews" instead.  Indeed, it
> means that errors in commit messages can't be fixed (we can fix them in
> the ChangeLog files, admittedly, but since I don't use them it doesn't
> help me).
>
> Maybe we could have a half-way system, where commits are pushed to
> a branch that is "not fast-forward-only", and this branch is then
> auto-merged to the real (fast-forward-only) master branch after a delay
> (one day, maybe?) to give time to fix mess ups before they're cast
> in stone.

I think that this would require some considerable co-ordination. If I
push a broken commit to devel branch, and then you fix this commit
through rebase, my copy of the branch (and everyone elses) is now
broken. I'm pretty sure that this kind of fix up can only happen on a
feature branch in a sane way.

Phil

Re: Is it time to drop ChangeLogs?

[Uwe Brauer]

    > John Wiegley <jwiegley@gmail.com>:

    > I advocated dropping ChangeLogs at the time I did the repository conversion.
    > That is still my position.

There is one feature, add-change-log-entry has while vc-next-command has
not:

In the case of lisp files: a  reference to the function or variable, you
created or modified as in

2016-03-08  Uwe Brauer  <oub@mat.ucm.es>

	* gnus-init.el (gnus-user-format-function-G): 

Is inserted in the ChangeLog file.

In the case of latex files the section and or equation are taken.

I wish vc-next-command had such a functionality (either for git or hg at least).

Re: Code reviews

[Yuri Khan]

On Tue, Mar 8, 2016 at 2:48 PM, Phillip Lord <phillip.lord@russet.org.uk> wrote:

> I think that this would require some considerable co-ordination. If I
> push a broken commit to devel branch, and then you fix this commit
> through rebase, my copy of the branch (and everyone elses) is now
> broken. I'm pretty sure that this kind of fix up can only happen on a
> feature branch in a sane way.

Fixups by rebase work if everybody agrees to never pull+merge and
always does a pull+rebase instead.

Re: Code reviews

[Stefan Monnier]

>> Maybe we could have a half-way system, where commits are pushed to
>> a branch that is "not fast-forward-only", and this branch is then
>> auto-merged to the real (fast-forward-only) master branch after a delay
>> (one day, maybe?) to give time to fix mess ups before they're cast
>> in stone.
> I think that this would require some considerable co-ordination.  If I
> push a broken commit to devel branch, and then you fix this commit
> through rebase, my copy of the branch (and everyone elses) is now
> broken.  I'm pretty sure that this kind of fix up can only happen on a
> feature branch in a sane way.

I think you misunderstood me: what I was thinking about in the quoted
text is a system where you submit a pull request to a review queue (not
to some kind of shared "proto-master" branch), and those requests are
automatically accepted after some timeout.


        Stefan

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

David Engster <deng@randomsample.de> writes:

> Óscar Fuentes writes:
>> `git notes' is a documented feature available since a long time ago. It
>> can turn the immutability of the commit log into a non-issue if you wish
>> to produce a proofread ChangeLog to include in the tarball.
>
> I can't imagine that you have ever really used 'git notes'.

I didn't.

> It's a hack with terrible usability

This is very vague. Is it good enough for solving our problem?
(providing a fixed commit message in case the original one is not
correct.)

> and no merge support.

What's wrong with `git notes merge' ?

Re: Should we restore manually maintained ChangeLogs

[Mathieu Lirzin]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 03/07/2016 01:06 PM, Eli Zaretskii wrote:
>
>> Let other projects invent those schemes and test-drive them. Enough
>> with these experiments!
>
> I'd rather just do what coreutils, grep, tar, etc. use. It's just as
> simple for ordinary committers and it would not involve duplication in
> patches or experimentation with maintenance procedures. I could fairly
> easily change the master branch to do that.

I like what is done by those projects.  However they don't have a merge
workflow.  I agree with you that for Emacs release branches, maintaining
ChangeLog manually seems reasonable.  However how will it work for the
feature branches which are meant to be merge sooner or later?

> Even simpler would be to do what Guile does: it dispenses with
> ChangeLogs entirely. With Guile if you want something like a ChangeLog
> you run "git log".

This approach seems wrong to me.  I think it is still desirable to be
able to partially analyze history directly from a tarball (even if it is
not optimal) without requiring a web browser, or cloning a full git
repository.

Thanks,

-- 
Mathieu Lirzin

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> No, you understood me correctly. However, I've just looked this up again
> and it turns out I was actually submitting a patch to Gnus, not
> Emacs.

I see.  Not sure how the Gnus guys did it, but in Emacs we asked for
"the diff (not including the change to the ChangeLog file)" plus "the
text to add to the ChangeLog file".  Then the guy who committed this
change would copy the ChangeLog text into the ChangeLog file *and* into
the commit message.

So the submitter did not see the duplication.  This was done not really
for the submitter's sake (tho she also benefitted) but because it's
easier to copy the ChangeLog text than to have to deal with the problem
of a patch that doesn't apply because the ChangeLog file has changed in
the mean time.

IOW, I understand your surprise, and Emacs submitters AFAIK haven't had
to deal with such situations, so it's not really relevant to this
whole discussion.


        Stefan

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> 2016-03-08  Uwe Brauer  <oub@mat.ucm.es>
> 	* gnus-init.el (gnus-user-format-function-G): 

For this reasons, I have an (untracked) ChangeLog file in all
my repositories.  This way I can use C-x 4 a to write a ChangeLog entry
in the "good old way" and then VC will automatically copy that data into
the *VC-Log* buffer for me to commit it.

I had a hack which made C-x 4 a insert that text directly into *VC-Log*
but in the end I found it didn't work so well: it didn't let me save the
intermediate commit message and return to it later so it worked OK for
small simple changes, but for larger commits with larger diffs I found it
rather inconvenient.


        Stefan

Re: Is it time to drop ChangeLogs?

[Uwe Brauer]

>>> "Stefan" == Stefan Monnier <monnier@iro.umontreal.ca> writes:

   >> 2016-03-08  Uwe Brauer  <oub@mat.ucm.es>
   >> * gnus-init.el (gnus-user-format-function-G): 

   > For this reasons, I have an (untracked) ChangeLog file in all
   > my repositories.  This way I can use C-x 4 a to write a ChangeLog entry
   > in the "good old way" and then VC will automatically copy that data into
   > the *VC-Log* buffer for me to commit it.


That's what I do also.


   > I had a hack which made C-x 4 a insert that text directly into *VC-Log*
   > but in the end I found it didn't work so well: it didn't let me save the
   > intermediate commit message and return to it later so it worked OK for
   > small simple changes, but for larger commits with larger diffs I found it
   > rather inconvenient.

In any case could you send me that hack please?

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

>> I'd rather just do what coreutils, grep, tar, etc. use. It's just as
> I like what is done by those projects.  However they don't have a merge
> workflow.  I agree with you that for Emacs release branches, maintaining
> ChangeLog manually seems reasonable.  However how will it work for the
> feature branches which are meant to be merge sooner or later?

The ChangeLog would only be generated into tarballs (i.e. only when
leaving the realm of the VCS), so they just never appear on any branch.


        Stefan

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

>> I had a hack which made C-x 4 a insert that text directly into *VC-Log*
>> but in the end I found it didn't work so well: it didn't let me save the
>> intermediate commit message and return to it later so it worked OK for
>> small simple changes, but for larger commits with larger diffs I found it
>> rather inconvenient.
> In any case could you send me that hack please?

Sorry, it died a while ago and I'm not sure how to resurrect it.


        Stefan

Re: Is it time to drop ChangeLogs?

[Andy Moreton]

On Mon 07 Mar 2016, Karl Fogel wrote:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>>On 03/07/2016 11:43 PM, Karl Fogel wrote:
>>> Let's please take it for granted that everyone who's suggesting to
>>> get rid of ChangeLog files is also assuming that we would, of
>>> course, use the same conventions for writing git commit messages
>>> that we would use for writing ChangeLog entries.  (As far as I can
>>> tell, everyone who wants to get rid of ChangeLog files has either
>>> expressed that assumption explicitly or implied it pretty clearly.
>>> )
>>
>>Not so. Many messages in this thread are saying different.
>
> I looked again -- who are these "many"? I don't see any messages from someone
> who both advocates that we get rid of ChangeLogs *and* suggests that we should
> not use the same format for commit messages as we currently do for ChangeLogs.
> (Andy Moreton suggested something about how we could have improved standards
> about what we put in a change description in general, but that's different and
> orthogonal to this discussion.)
>
> Please let me know if I missed a post, though.

For the avoidance of doubt, I think that the changelog format describing
the contents of the diff should not be included in the git commit
message, as it is a pointless duplication of information that the diff
already provides. Let the version control system handle what changed,
and put much more emphasis on describing why the patch exists.

    AndyM

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: mthl@gnu.org, johnw@gnu.org, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 7 Mar 2016 23:42:33 +0200
> 
>         Maintaining ChangeLog files by hand with each commit makes it harder
>         to merge changes due to the inevitable collisions with ChangeLog
>         files.
> 
>     Incorrect!  And the current situation creates _more_ collisions, not
>     less!
> 
> Only when merging between branches.

It doesn't work very well even on a single branch.  With two branches,
it completely breaks down.

We do want to work on multiple branches, don't we?  We discussed how
to make more frequent releases, and perhaps have 3 branches for even
more flexible development and release schedule.  All of that would be
impossible because of this tiny but annoying PITA.  Do we really want
to continue wasting energy on fixing something that wasn't such a big
problem to begin with?

> Other than that, only a select group of people needs to bother: those who make mistakes, and those who feel a general need to clean up.

See, that's the crux of the issue: do we or don't we care about the
need to clean up our ChangeLogs?  If we don't, then why bother
maintaining them?

You (and some others) say the format and the content in the log
messages are important, and I agree.  But if we do care about them,
how can we NOT clean them up?  Having them in their current state
means they cannot be trusted, which is worse than not having them at
all.

> As a relatively careful committer, I've only had to correct the entries a few times, and I've been enjoying the lack of collisions quite a bit.

Yes, a few of us don't need any corrections.  But enough of us do, and
that's where the problem lies.  It is that problem that we need to
fix.  Leaving it unfixed makes your accurate work unreliable as well.

>     But ChangeLog mistakes can be easily fixed.
> 
> In the current approach, as well.

Only on a single branch, and even there this is not done enough.  We
all relied on Glenn to do that behind the scenes.  Now Glenn gave up
in despair, so things will degrade from now on.

>     We don't know how, and we don't have anyone who is motivated enough to
>     do that.  And even if and when we do have some solution, it is likely
>     to be inconvenient and unreliable.
> 
> I think we should wait and see until the work really transitions back to master. The motivation must rise.

The release branch will remain active for quite some time: we have
just started a new major version.  And if we want more frequent
releases, we should strive to have an active release branch at all
times.  So I don't think we can wait; waiting will not solve anything,
it will just make the current problems bigger and harder to solve.

>     Let other projects invent those schemes and test-drive them.  Enough
>     with these experiments!  They draw the last drops of energy from us,
>     and they avert the few last veteran contributors we have left.
> 
> Has the current experiment really sucked too much energy from anyone, aside from the implementors?

Why do you think Glenn gave up?

>     That's not a bad thing in itself.  The point is, these procedures
>     work, and all those projects are alive and kicking, and actually make
>     more frequent releases than we do.
> 
> For all we know, they might be thriving despite this practice.

Nothing wrong with that.

>         I often ran into problems. Yes, git-merge-changelog should reduce the
>         number of merge conflicts, but it doesn't eliminate them
> 
>     Oh, yes, it does.
> 
> Not in my experience either. I've still had collisions, and even when git-merge-changelog resolved them, it often put my entry in the middle of the file, whereas I usually needed it to be at the top. Leading to extra manual labor.

That extra manual labor is very small, and it's still a rare case to
have that.  A small price to pay for a clean and reliable solution.

>         requiring git-merge-changelog means that many contributors would
>         have to worry about installing and configuring git-merge-changelog,
>         which would be more of a hassle for recruiting contributors.
> 
>     It's a 5-sec configuration, let's not make a mountain out of a
>     molehill.
> 
> It was longer for me. But either way, it's more hassle for a random contributor than the current system.

The current system is much more hassle for non-random contributors, so
much so that we risk losing them, something we cannot afford.

> If it can be fixed, Someone should.

Let that Someone step forward, then, and speak up.  We have been
waiting for that for the past year, so I'm not holding my breath now.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Date: Mon, 07 Mar 2016 15:43:29 -0600
> 
> Again, if the majority of people like Eli who do tons of work around here would prefer to keep ChangeLogs, then I'm in favor of doing so.  But it would be nice to be de-confused about the arguments for it.

I've posted my arguments.  In a nutshell, maintaining the logs in the
VCS doesn't allow us to fix mistakes there (which then get propagated
to the generated ChangeLog), and the way we designed to work around
this breaks down when we have more than one active branch.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: johnw@gnu.org, mthl@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Mon, 7 Mar 2016 15:31:54 -0800
> 
> On 03/07/2016 01:06 PM, Eli Zaretskii wrote:
> 
>         Maintaining ChangeLog files by hand with each commit makes it harder
>         to merge changes due to the inevitable collisions with ChangeLog
>         files.
> 
>     Incorrect!  And the current situation creates _more_ collisions, not
>     less!
>
> My own experience is otherwise. For the kinds of development I do, I rarely see ChangeLog screwups now, whereas I used to see them routinely.

With or without git-merge-changelog?

If you have git-merge-changelog installed, what kind of screwups did
you see with ChangeLogs?  (I didn't see even one, in all the years I
use Git.)

>  I far prefer the current approach. Of course our approach can be improved (in particular merging from emacs-25 to master doesn't work well now), but let's not throw out the baby with the bathwater.

The current approach completely breaks down when more than one branch
is active.  So there's no baby in that bathwater.

>     We don't have to be better than the other prominent GNU projects.
> 
> I'd be happy if we were merely as good as the other prominent GNU projects that generate ChangeLog entries automatically. As things stand, due to our attempt to cater to all sides of this disagreement, we have an approach that satisfies nobody.

Not sure what you mean here.  What alternatives that don't "cater to
all sides" would you suggest?  The only one I see is to stop producing
ChangeLog files for the releases.

>     ChangeLog mistakes can be easily fixed. 
> 
> That's true under both the old and the new regimes.

No, it isn't.  Definitely not with two branches.

>     Let other projects invent those schemes and test-drive them. Enough with these experiments! 
> 
> I'd rather just do what coreutils, grep, tar, etc. use.

Don't know what hides behind "etc", but the rest are much smaller
projects, which are all in maintenance mode, and have a very small
number of active committers (of which you personally are a significant
fraction ;-).  They also don't use branches, at least not as much as
we do.  So I don't see how the experience of these projects is more
relevant to us than that of GCC, GDB, Binutils, and glibc.

> I could fairly easily change the master branch to do that.

Please describe the details of your proposal.  Also, please tell what
do you suggest doing on the release branches.

> Even simpler would be to do what Guile does: it dispenses with ChangeLogs entirely. With Guile if you want something like a ChangeLog you run "git log".

As I said, tossing ChangeLog files entirely would indeed solve the
problems, but it's a sure path to further deterioration in the quality
of log messages.  It is easy to keep the quality in a project that has
a small number of committers who are veteran GNU developers (Guile has
1 frequent committer on master, and 3 on stable).  That's not our
case.

> If neither of the above two approaches suffice, we can always fall back on my previous email's proposal. It's not that experiemental, as it says to use the new produre on master and the old procedure on emacs-25. The new procedure works well enough within a single master branch.

Any suggested approach should support not only the current emacs-25
branch, but also the future release branches, i.e. it should continue
working when we cut the emacs-26 branch in the future.  It should also
be reliable, and not require manual labor for fixing mistakes in the
log messages, beyond the fix itself.

>     these procedures work, and all those projects are alive and kicking, and actually make more frequent releases than we do. 
> 
> XEmacs is not really alive and kicking.

XEmacs is not that different from Grep or Sed.  Sed, for example, saw
just 30 commits during all of the last year.

> Projects like GCC and glibc have more resources than we do, and can afford to insist on more-expert contributions that involve harder-to-generate patch formats. We do not have that luxury.

It's the other way around, actually: the current situation requires
more labor from us to get it right, so our lack of resources should
lead us to the opposite conclusion.

>         I often ran into problems. Yes, git-merge-changelog should reduce the
>         number of merge conflicts, but it doesn't eliminate them
> 
>     Oh, yes, it does.
> 
> Not in my experience, or in Dmitry's. It's a fine program, but it sometimes makes mistakes and they can be a pain to fix. 

Fixing its mistakes involves moving an entry to a different place,
that's all.  Easy done, and even if not done, it's not a disaster, as
the information is there anyway.

Re: Code reviews (was: Is it time to drop ChangeLogs?)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 07 Mar 2016 22:25:12 -0500
> 
> If we could switch to a system where every patch is reviewed before
> commit, that'd be great.  My own impression is that it will kill the
> development pace because too few people are willing to spend the
> corresponding efforts.

Agreed.  For my part, I can say that I simply don't have enough free
time to review more patches than I do (which is an abysmal little).

> That's why I've followed a practice of giving out write access very
> liberally, with "post-commit spot-check reviews" instead.  Indeed, it
> means that errors in commit messages can't be fixed (we can fix them in
> the ChangeLog files, admittedly, but since I don't use them it doesn't
> help me).

Post-commit reviews also take time.  Do you have an estimation of how
much time per day this takes?

> Maybe we could have a half-way system, where commits are pushed to
> a branch that is "not fast-forward-only", and this branch is then
> auto-merged to the real (fast-forward-only) master branch after a delay
> (one day, maybe?) to give time to fix mess ups before they're cast
> in stone.

A day is nowhere near enough.  IME, a bad commit pushed to master
takes up to a week to be discovered.

More generally, the problem with such a branch is that it won't be
much different from pushing to master, except in rare cases that it
breaks the build, and even that can only be avoided if we set up some
kind of CI system that continuously builds that branch on the main
supported platforms.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 08 Mar 2016 05:23:46 +0100
> 
> >> Introduce code reviews. Don't give commit access to the "golden"
> >> branches to everyone, just to a few top contributors and reviewers.
> >
> > We don't have manpower for that.
> 
> I find perplexing that we have manpower for reviewing ChangeLogs but not
> for reviewing commit logs *instead*.

I'm surprised that you find it perplexing.

Log messages are much shorter than code diffs, and much simpler to
read and understand.  It takes a few seconds to read them; reading a
code patch takes much more, typically needs to consult the surrounding
and related code, etc.  So code review takes orders of magnitude more
time than reviewing the corresponding log entries.

> And is it so super-important to have typo-free commit logs?

Typos is the least of our problems.  But yes, it's important: if you
start by forgiving typos, you end with unhelpful and uninformative log
messages.

> We are not even using `git notes'.

No one came up with a detailed procedure for doing that.  If you can
propose something that works, please do.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 08 Mar 2016 05:34:10 +0100
> 
> >> Another option for correcting errors on commit logs is `git notes', but
> >> I don't know about its implications.
> >
> > Nobody does.  We shouldn't consider any more experiments in this area
> > until someone else uses these methods and proves they are viable in
> > our case.
> 
> How could *someone else* prove the viability of something in *our* case?

By coming up with a procedure which we can study and decide whether
it fits us.

> You always can argue that his project is not Emacs...

I can, but why would I want to?  The opinions are not pre-determined,
they are based on study and in this case also on a year-long
experience of actually using a procedure.

> `git notes' is a documented feature available since a long time ago. It
> can turn the immutability of the commit log into a non-issue if you wish
> to produce a proofread ChangeLog to include in the tarball.

We need a more detailed proposal than this, TIA.  David says "git
notes" is not a solution; if notes don't support merging, then I tend
to think he is right.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 07 Mar 2016 21:00:50 -0800
> 
> > The reasons exist, and has been described.  You may not agree with
> > them, perhaps due to different experience, but that doesn't mean they
> > aren't valid.
> 
> Yeah, but what exactly is your point here? I was pointing out that this
> practice is astonishing for new contributors. You didn't understand why,
> so I explained that it's astonishing because to a new contributor it
> seems pointless. Now you assert that there are good reasons for it, but
> that really doesn't change that, to new contributors, it appears
> pointless and is astonishing.

My point is that the initial surprise will subside with time, by way
of getting used to it.  It is even possible that you will come to
value this, as someone else who reported here.

> Sorry, I still don't follow your reasoning. I still consider myself a
> potential contributor (I didn't contribute much beyond that first little
> foray), and not requiring me to copy my commit message into the
> ChangeLog is not going to affect the quality of the commit measures in
> any way. The only "message" that I get from this is "Great, sending
> patches for Emacs doesn't require me to do a copy & paste anymore".

As long as you don't have write access to the repository, you are not
required to provide the log entries more than once, and so this issue
doesn't really affect you.  This issue is only relevant to those who
actually push changes, because they might need to copy/paste into the
ChangeLog file.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 08 2016, Eric Abrahamsen <eric@ericabrahamsen.net> wrote:
> Nikolaus Rath <Nikolaus@rath.org> writes:
>
>> On Mar 07 2016, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>>> When I submitted my first Emacs patch, I was astonished when I was asked
>>>> to re-submit with my commit message essentially duplicated in the
>>>> ChangeLog.
>>>
>>> I can't remember ever asking this kind of duplication.  We used to
>>> duplicate the ChangeLog-info in the ChangeLog file and in the commit
>>> message (tho we stopped doing it a year ago), but we never asked for it
>>> to be duplicated in the email messages that submits a patch.
>>>
>>> So I have the impression that I'm misunderstanding you.  Could you
>>> describe more precisely the kind of duplication you're talking about?
>>
>> No, you understood me correctly. However, I've just looked this up again
>> and it turns out I was actually submitting a patch to Gnus, not
>> Emacs. It was eventually committed by Eric Abrahamsen to the Gnus git
>> repo, who first asked me to re-submit my patch with a commit message
>> that was formatted like a Changelog entry, and to duplicate the message
>> in the ChangeLog itself. It seemed to me that he considered that a
>> questionable practice himself, but said it was obligatory because Gnus
>> closely followed the Emacs standards here.
>>
>>
>> Unfortunately that discussion never made it into the bug
>> (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20956), so I'm
>> paraphrasing from memory. But I'm pretty sure I'm remembering correctly,
>> because I was so surprised by this.
>
> Not duplicating it in the email, just commit message + ChangeLog
> duplication. This may be getting conflated with me asking for the commit
> to be emailed as a git-format-patch attachment, which was just because I
> was lazy (and it also preserves the author/committer distinction).

Actually, I was quite happy about that. I very much prefer if patches
that I submit also have me as the author of the commit.

(Eric, I think my last email sounded a bit like telling on you. I'm
sorry about that, I think I should have written it differently (or asked
you first). I'm very grateful for the time you took to help me with
those patches, and for patiently explaining the rationale for the
format).

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Code reviews

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Tue, 08 Mar 2016 08:48:56 +0000
> Cc: emacs-devel@gnu.org
> 
> Giving people different permissions on different branches seems to make
> sense. Overtime, people move toward master/emacs-25.

AFAIK, Git (or is it Savannah?) doesn't support such granularity.

> > Maybe we could have a half-way system, where commits are pushed to
> > a branch that is "not fast-forward-only", and this branch is then
> > auto-merged to the real (fast-forward-only) master branch after a delay
> > (one day, maybe?) to give time to fix mess ups before they're cast
> > in stone.
> 
> I think that this would require some considerable co-ordination. If I
> push a broken commit to devel branch, and then you fix this commit
> through rebase, my copy of the branch (and everyone elses) is now
> broken.

Stefan was talking about merging, not rebasing.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 08 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> Sorry, I still don't follow your reasoning. I still consider myself a
>> potential contributor (I didn't contribute much beyond that first little
>> foray), and not requiring me to copy my commit message into the
>> ChangeLog is not going to affect the quality of the commit measures in
>> any way. The only "message" that I get from this is "Great, sending
>> patches for Emacs doesn't require me to do a copy & paste anymore".
>
> As long as you don't have write access to the repository, you are not
> required to provide the log entries more than once, and so this issue
> doesn't really affect you.

In that case I agree. However, this practice has the distinct
disadvantage that a patch's submitter does not show up as the author of
the commit. But that's a different discussion.

(As I said in the other mail, I was actually working with the Gnus repo
rather than the Emacs repo - thus the confusion about what the submitter
needs to do).

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

Uwe Brauer wrote:
>     > For this reasons, I have an (untracked) ChangeLog file in all
>     > my repositories.  This way I can use C-x 4 a to write a ChangeLog entry
>     > in the "good old way" and then VC will automatically copy that data into
>     > the*VC-Log*  buffer for me to commit it.
>
>
> That's what I do also.

Me too, except I track the ChangeLog file in a *separate* git repository (i.e., 
not in the git repository that holds emacs). This separate repository is 
automatically managed by vc-dwim.

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

> You (and some others) say the format and the content in the log
> messages are important, and I agree.  But if we do care about them,
> how can we NOT clean them up?  Having them in their current state
> means they cannot be trusted, which is worse than not having them at
> all.

For people like me who solely rely on the VCS commit messages when doing
forensics (because I find it tremendously more powerful, e.g. via
vc-region-history or vc-annotate), it's already the case that mistakes
can't be corrected, no matter what we do with ChangeLog files.

So for my own personal self, the issue is not "should we keep ChangeLog"
but "should we try and fix mistakes in commit messages":
- either we do care about fixing the commit messages.  In this case
  either we need to switch to a system that lets us review changes
  before they get to master, or we figure out a way to update commit
  messages after the fact (this is clearly a missing feature in Git and
  there's no reason it can't be implemented).
- or we don't care enough, so we keep doing as we've been doing so far,
  i.e. relying on "best effort and education" to try and keep the
  mistakes to a tolerable level.

So far we've been following a halfway path where we tolerate mistakes in
the commit messages but not in the ChangeLog file, probably because
there's a fairly strong correlation between those who care about
mistakes in those messages and those who cares about the
ChangeLog files.

But really, fixing the ChangeLog while leaving the VCS commit messages
unfixed is just not a good solution (to me, it's kind of like installing
security fixes in the tarball rather than in the VCS repository).

For those reasons, I think ChangeLog files should be 100%
auto-generated, and never committed into the VCS.

> Yes, a few of us don't need any corrections.  But enough of us do, and
> that's where the problem lies.  It is that problem that we need to
> fix.  Leaving it unfixed makes your accurate work unreliable as well.

Right.  But if we need/want to fix it, we should do it *at the source*
(i.e. in the VCS metadata) rather than only in some of the derived data
(the ChangeLog files).


        Stefan

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: Mathieu Lirzin <mthl@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  johnw@gnu.org,  emacs-devel@gnu.org
> Date: Tue, 08 Mar 2016 15:03:28 +0100
> 
> > Even simpler would be to do what Guile does: it dispenses with
> > ChangeLogs entirely. With Guile if you want something like a ChangeLog
> > you run "git log".
> 
> This approach seems wrong to me.  I think it is still desirable to be
> able to partially analyze history directly from a tarball (even if it is
> not optimal) without requiring a web browser, or cloning a full git
> repository.

Yes, I very much agree.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 08 Mar 2016 09:43:55 -0500
> 
> > 2016-03-08  Uwe Brauer  <oub@mat.ucm.es>
> > 	* gnus-init.el (gnus-user-format-function-G): 
> 
> For this reasons, I have an (untracked) ChangeLog file in all
> my repositories.  This way I can use C-x 4 a to write a ChangeLog entry
> in the "good old way" and then VC will automatically copy that data into
> the *VC-Log* buffer for me to commit it.

Same here.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 08 Mar 2016 10:15:35 -0500
> 
> >> I'd rather just do what coreutils, grep, tar, etc. use. It's just as
> > I like what is done by those projects.  However they don't have a merge
> > workflow.  I agree with you that for Emacs release branches, maintaining
> > ChangeLog manually seems reasonable.  However how will it work for the
> > feature branches which are meant to be merge sooner or later?
> 
> The ChangeLog would only be generated into tarballs (i.e. only when
> leaving the realm of the VCS), so they just never appear on any branch.

Which means they will either have all the uncorrected mistakes (like
missing "tiny change" etc.), or you expect from Someone to do all the
fixes as part of tarring a release.  I think both alternatives are
unacceptable.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Tue, 08 Mar 2016 15:21:07 +0000
> 
> For the avoidance of doubt, I think that the changelog format describing
> the contents of the diff should not be included in the git commit
> message, as it is a pointless duplication of information that the diff
> already provides. Let the version control system handle what changed,
> and put much more emphasis on describing why the patch exists.

I don't think anyone is arguing with that.  We don't tell contributors
to describe the changes literally.  For example, if the change is

  i = i + 1;

then saying "add 1 to i" is a very bad log entry.

Re: Code reviews

[Stefan Monnier]

>> That's why I've followed a practice of giving out write access very
>> liberally, with "post-commit spot-check reviews" instead.  Indeed, it
> Post-commit reviews also take time.

Indeed.  But notice the "spot-check" part: only some of the commits are
actually (post-commit) reviewed.

>> Maybe we could have a half-way system, where commits are pushed to
>> a branch that is "not fast-forward-only", and this branch is then
>> auto-merged to the real (fast-forward-only) master branch after a delay
>> (one day, maybe?) to give time to fix mess ups before they're cast
>> in stone.
> A day is nowhere near enough.  IME, a bad commit pushed to master
> takes up to a week to be discovered.

Maybe the delay should depend on the submitter.  An infinite delay for
first-submitters and a 0 delay for those submitters we trust to
carefully review they commit messages before pushing (clearly,
I wouldn't be one of them), and all kinds of other values in-between.

> More generally, the problem with such a branch is that it won't be
> much different from pushing to master, except in rare cases that it
> breaks the build, and even that can only be avoided if we set up some
> kind of CI system that continuously builds that branch on the main
> supported platforms.

The review queue could be used for code-quality reviews indeed, but
I was thinking here about focusing on the commit-message reviewing
(since we can fix the code after the fact with additional commits,
whereas we can't fix commit messages after the fact, thanks to Git
implementors's short-sightedness).


        Stefan

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> In that case I agree. However, this practice has the distinct
> disadvantage that a patch's submitter does not show up as the author of
> the commit. But that's a different discussion.

Normally the committer should setup the "Author:" info properly (and if
the commit message is auto-extracted from the ChangeLog file, this
should happen automatically by extracting the author's name from the
ChangeLog), so the author is properly recorded in the VCS (even if he's
not the committer).


        Stefan

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

> Which means they will either have all the uncorrected mistakes (like
> missing "tiny change" etc.), or you expect from Someone to do all the
> fixes as part of tarring a release.  I think both alternatives are
> unacceptable.

I've been living with those uncorrected mistakes ever since bzr became
fast enough that I started to rely on "bzr log" rather than on
ChangeLog files.

I'd prefer if we could fix those mistakes, indeed, but fixing them in
the ChangeLog is (for me) a complete waste of time.


        Stefan

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

Eli Zaretskii wrote:
> You (and some others) say the format and the content in the log
> messages are important, and I agree.  But if we do care about them,
> how can we NOT clean them up?

We can care about them, but agree to fix them up only until they become part of 
history. Once they're history we don't worry about trying to change history; 
they're just old mistakes that are part of the log but are not otherwise part of 
the current Emacs. If managed well, this can help motivate contributors to write 
good commit messages the first time. This approach is not perfect, but it works 
reasonably well in other projects and it is way easier to explain and to 
maintain than what we're doing now, or what we did a year ago.

With version control systems our natural perfectionist inclinations can cause us 
to want to rewrite history to make ourselves look more error-free than we 
actually were.  In extreme cases (e.g., massive copyright infringement committed 
by a rogue developer) we would indeed need to rewrite history, despite all the 
hassles that would ensue with Git (hassle that would not be limited to 
commit-message contents!). However, in ordinary use we should resist the 
temptation to change history; at best it's makework.

So, for example, we should strive to get the "tiny change" stuff right the first 
time in commit messages; but if we make mistakes in that area it OK -- the sky 
will not fall down, and software archaeologists of the future will still be able 
to figure things out well enough.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 08 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Tue, 08 Mar 2016 05:23:46 +0100
>> 
>> >> Introduce code reviews. Don't give commit access to the "golden"
>> >> branches to everyone, just to a few top contributors and reviewers.
>> >
>> > We don't have manpower for that.
>> 
>> I find perplexing that we have manpower for reviewing ChangeLogs but not
>> for reviewing commit logs *instead*.
>
> I'm surprised that you find it perplexing.
>
> Log messages are much shorter than code diffs, and much simpler to
> read and understand.

Óscar was talking about reviewing the *commit message* instead of the
changelog, not about reviewing the entire commit (including the diff)
instead.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Date: Tue, 08 Mar 2016 08:08:54 -0800
> 
> I very much prefer if patches that I submit also have me as the
> author of the commit.

Of course!  We cannot do it any other way, if only for legal reasons.

If you don't use git-format-patch, then the committer should use
"git commit --author" to achieve the same effect.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Date: Tue, 08 Mar 2016 08:12:22 -0800
> 
> > As long as you don't have write access to the repository, you are not
> > required to provide the log entries more than once, and so this issue
> > doesn't really affect you.
> 
> In that case I agree. However, this practice has the distinct
> disadvantage that a patch's submitter does not show up as the author of
> the commit.

No, this is a misunderstanding: the submitter _always_ appears as the
author, just not as the committer.  We use the --author switch for
that.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 08 Mar 2016 11:14:14 -0500
> 
> > Yes, a few of us don't need any corrections.  But enough of us do, and
> > that's where the problem lies.  It is that problem that we need to
> > fix.  Leaving it unfixed makes your accurate work unreliable as well.
> 
> Right.  But if we need/want to fix it, we should do it *at the source*
> (i.e. in the VCS metadata) rather than only in some of the derived data
> (the ChangeLog files).

If possible, that's the preferred way, sure.

Re: Code reviews

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 08 Mar 2016 11:21:42 -0500
> 
> I was thinking here about focusing on the commit-message reviewing

That would better be served by a separate mailing list where patches
are posted for such a review.

Re: Is it time to drop ChangeLogs?

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 4338 bytes --]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> And I think this is essential input. We are alienating contributors by
> insisting on our weird (ok, "unusual") format of the meta data of the
> patches.
>
> The ChangeLog is useful for some of us. But I would contend that its
> usefulness doesn't trump all the contributors we have discouraged by
> insisting on our quite singular submission format.

Thank you, Lars, you have pretty much summed up my own concerns.

To summarize several of the points in this thread so far:

  - First, and most importantly, we have significant contributors who rely on
    and enjoy the ChangeLog file -- even if there are others, and potential
    contributors, who dislike it.

    If people doing the "real work" are using this file, we should keep it. If
    they want to maintain it as a separate file in Git, we should let them.
    Even if it costs time to maintain it, or explain its necessity to
    newcomers, it's worth it to support those who do the lion's share of the
    work.

    However: once this situation changes, we should revisit the issue. I am
    prepared to drop separate ChangeLog files (generated or not) the moment
    our primary developers say they no longer need it.

  - The ChangeLog format helps to structure commit messages.

    I like this point, and agree that it sets a "bar" for commit log content.
    Whatever helps us raise the calibre of our commit messages makes sense to
    me. However, if we ever find that we're not reading that data while using
    vc-region-history or some other tool, we should revisit this point.

  - Commit messages cannot be edited.

    This is an area where "git notes" could, in fact, help. The idea is not
    that we use git notes for every commit, to provide ChangeLog data, but use
    them to "amend" commit log messages. That way, when generating the
    ChangeLog for release, a git note can override a commit message,
    correcting what appears in the final ChangeLog.

    Using git notes in this way should almost entirely eliminate its merge
    issues, and we'd be using it for its intended purpose: notes on specific
    commits.

In summary, I see two viable paths before us:

  1. Go back to the old way of doing things, using a separately maintained
     ChangeLog file. As long as that's what our main contributors want, I
     believe it's what we should do. Even if we lose some new contributors,
     we should show support for those devoting the most time to our project.

  2. At some future date, should the main contributors no longer rely on or
     want a separate ChangeLog file, we should:

     - Remove ChangeLogs from version control.
     - Require ChangeLog format entries in the commit message, as long as it
       helps us and improves the quality of contributed commit messages.
     - Employ some mechanism, such as git notes, to amend incorrect log
       messages when generating metadata for the release tarball.

Our present mechanism, of maintaining a generated file under version control,
is currently agitating some of our core contributors, nor does it free us from
ChangeLogs. Rather, it's a compromise alternative that includes some of the
worst aspects of both options (e.g., inability to edit commit messages; merge
problems with ChangeLogs; annoying primary contributors; needs tooling to
handle the generation).

That said, I want to make sure that we're actually hearing from all our "core
contributors" in this discussion. Can I hear from some of the other large-
scale contributors? For the sake of argument, here is everyone who has made
more than 500 commits in the past five years (git shortlog --since 2011):

    4467    Glenn Morris
    3582    Paul Eggert                   #2
    2543    Eli Zaretskii                 #1
    1822    Stefan Monnier                #2
    1062    Chong Yidong
    759     Michael Albinus
    675     Lars Magne Ingebrigtsen       #2
    653     Dmitry Antipov
    639     Juanma Barranquero

If there are others who have only recently become highly active, please pitch
in also.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 8 Mar 2016 08:34:04 -0800
> 
> Eli Zaretskii wrote:
> > You (and some others) say the format and the content in the log
> > messages are important, and I agree.  But if we do care about them,
> > how can we NOT clean them up?
> 
> We can care about them, but agree to fix them up only until they become part of 
> history. Once they're history we don't worry about trying to change history; 
> they're just old mistakes that are part of the log but are not otherwise part of 
> the current Emacs. If managed well, this can help motivate contributors to write 
> good commit messages the first time. This approach is not perfect, but it works 
> reasonably well in other projects and it is way easier to explain and to 
> maintain than what we're doing now, or what we did a year ago.

AFAIK, those other projects get commits from veteran GNU developers
such as yourself, which have the ChangeLog format and good log
messages burnt into their finger memories.

Emacs is different.  We have to educate the newcomers to become such
veterans.  And education simply doesn't work without the need to fix
your beginner's mistakes.  By giving up on the need to fix them, we
inadvertently send a very loud and clear message to the newcomers
saying that good, correct, and accurate log messages are not
important.  As result, they will never acquire the skills that you and
other veterans here have since long ago.

When I was a newcomer, I had the privilege of getting comments and
requests to fix my log messages from Richard and others.  Had they not
insist on making my errors clear to me, had they not asked for me to
fix them, I wouldn't be able to write the kind of log messages and
other short descriptions I can do today.

So let's stop thinking about ourselves -- we don't need these fixes
anyway.  Let's think about the new generation, the ones we must
educate as long as we are around.  They _need_ us to point them to
their flops, and they need to learn to fix them.  This cannot be
learned in theory, only by doing.  It is an important part of their
apprenticeship.  This isn't about perfectionism, this is about
teaching the newcomers to be better Emacs developers.

> So, for example, we should strive to get the "tiny change" stuff right the first 
> time in commit messages; but if we make mistakes in that area it OK -- the sky 
> will not fall down, and software archaeologists of the future will still be able 
> to figure things out well enough.

The skies will not fall down if Emacs ceases to exist, either, or
becomes a much less clean and orderly project than it is now.  But
we'd like to avoid that, one hopes.

Re: Is it time to drop ChangeLogs?

[David Engster]

Óscar Fuentes writes:
> David Engster <deng@randomsample.de> writes:
>
>> Óscar Fuentes writes:
>>> `git notes' is a documented feature available since a long time ago. It
>>> can turn the immutability of the commit log into a non-issue if you wish
>>> to produce a proofread ChangeLog to include in the tarball.
>>
>> I can't imagine that you have ever really used 'git notes'.
>
> I didn't.

Then please do. Just take one of your private repositories and add a
note to the current commit. And then, just for fun, read the git-notes
man page and figure out how to push that note to your server (seriously:
the word 'push' is not mentioned once, but note that they *do* tell you
how to create a binary note blob, which sure is nifty, even if you'll
never figure out how to push it).

>> It's a hack with terrible usability
>
> This is very vague. Is it good enough for solving our problem?
> (providing a fixed commit message in case the original one is not
> correct.)

https://git-scm.com/blog/2010/08/25/notes.html

See paragraphs "Sharing Notes" (which also includes the answer to the
above), "Getting Notes" and "Collaborating on Notes", and decide for
yourself.

Yes, that page is from over 5 years ago, but note that there is still
no 'git push --notes'. If you want to know why, see for instance

http://permalink.gmane.org/gmane.comp.version-control.git/235597

>> and no merge support.
>
> What's wrong with `git notes merge' ?

See the git-notes man page and read section "Notes on merge strategies"
(the one that begins with 'The default notes merge strategy is
"manual"'). The above gmane-link also has a script attached which shows
how to resolve a conflict with 'git notes merge'. I strongly suggest to
once go through it and see what it involves.

'git notes' is a quick hack that somehow survived until today, and now
it's too late to remove it. It is barely OK for things like attaching
test results from your CI server. It's pretty much useless for
everything else. For correcting commit messages, it'd be much easier to
cook up something for ourselves (like simply a file
SHA1:NEW_COMMIT_MESSAGE).

-David

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]

John Wiegley <jwiegley@gmail.com> writes:
> 2. At some future date, should the main contributors no longer rely on or
>    want a separate ChangeLog file, we should:
>
>    - Remove ChangeLogs from version control.
>    - Require ChangeLog format entries in the commit message, as long as it
>      helps us and improves the quality of contributed commit messages.
>    - Employ some mechanism, such as git notes, to amend incorrect log
>      messages when generating metadata for the release tarball.

Aren't we doing that middle item already?  As I read the CONTRIBUTE file, the section on "commit messages" basically says to use ChangeLog style.

Which is good.  Since a ChangeLog entry is always suitable as a commit message (i.e., there is no good ChangeLog entry which wouldn't work fine as the commit message for the associated commit too), then there's no time like the present to start having a commit message convention we can all agree on.  Since we've already agreed on ChangeLog-style entries for ChangeLogs, I think we would also agree on it for commit messages... and CONTRIBUTE suggests that we have already done so.

This is orthogonal to whether we keep ChangeLog *files* in the tree, of course.  It just settles that we have one unified convention for log messages, and that convention applies to both of the places where log messages can be stored: in commit messages, and in ChangeLog files.

So the only remaining question is about where information should be stored (and, if there is duplication, whether that duplication should be maintained manually or automatically).  The question of what the information looks like is, I hope, already settled.

-Karl

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: Is it time to drop ChangeLogs?

[John Wiegley]

>>>>> Karl Fogel <kfogel@red-bean.com> writes:

> So the only remaining question is about where information should be stored
> (and, if there is duplication, whether that duplication should be maintained
> manually or automatically). The question of what the information looks like
> is, I hope, already settled.

Yes, I was only mentioning it for the sake of completeness.  We have already
accomplished the second bullet of option #2, and have no intention of changing
that, as it does have a positive impact on our commit message quality.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

John Wiegley <jwiegley@gmail.com> writes:
>>>>>> Karl Fogel <kfogel@red-bean.com> writes:
>> So the only remaining question is about where information should be stored
>> (and, if there is duplication, whether that duplication should be maintained
>> manually or automatically). The question of what the information looks like
>> is, I hope, already settled.
>
>Yes, I was only mentioning it for the sake of completeness.  We have already
>accomplished the second bullet of option #2, and have no intention of changing
>that, as it does have a positive impact on our commit message quality.

100% agreed, and glad to hear it!

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Tue, 08 Mar 2016 09:21:20 -0800
> Cc: Lars Magne Ingebrigtsen <larsi@gnus.org>, emacs-devel@gnu.org
> 
> >>>>> Karl Fogel <kfogel@red-bean.com> writes:
> 
> > So the only remaining question is about where information should be stored
> > (and, if there is duplication, whether that duplication should be maintained
> > manually or automatically). The question of what the information looks like
> > is, I hope, already settled.
> 
> Yes, I was only mentioning it for the sake of completeness.  We have already
> accomplished the second bullet of option #2, and have no intention of changing
> that, as it does have a positive impact on our commit message quality.

Just to put the record straight: we've been using the ChangeLog format
in the commit log message since forever, certainly during the CVS
days.  It's not a recent invention in any way.

Re: Should we restore manually maintained ChangeLogs

[Dmitry Gutov]

On 03/08/2016 05:45 PM, Eli Zaretskii wrote:

> You (and some others) say the format and the content in the log
> messages are important, and I agree.  But if we do care about them,
> how can we NOT clean them up?  Having them in their current state
> means they cannot be trusted, which is worse than not having them at
> all.

That's true. But my motivation for using ChangeLogs stems from having 
people describe their changes in a strict format. If someone writes a 
wrong name or omits the "copyright-exempt" header, I could live with that.

We should find out how much it is of a problem, though, legally speaking.

>> Has the current experiment really sucked too much energy from anyone, aside from the implementors?
>
> Why do you think Glenn gave up?

My bad. All right, Glenn gave up fixing errors. Isn't that because 
people made too much mistakes, and didn't bother to fix them?

Even if we transition to the previous system, it will need the same 
people to fix their errors.

>> Not in my experience either. I've still had collisions, and even when git-merge-changelog resolved them, it often put my entry in the middle of the file, whereas I usually needed it to be at the top. Leading to extra manual labor.
>
> That extra manual labor is very small, and it's still a rare case to
> have that.  A small price to pay for a clean and reliable solution.

It's a bit hard to remember now, but I think I had to move my entry to 
the top more often than not. So, not a rare case.

>> It was longer for me. But either way, it's more hassle for a random contributor than the current system.
>
> The current system is much more hassle for non-random contributors, so
> much so that we risk losing them, something we cannot afford.

Will someone decide to stop contributing to Emacs because our Change Log 
entries contain mistakes? That doesn't sound very plausible.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Tue, 8 Mar 2016 19:44:47 +0200
> Cc: mthl@gnu.org, eggert@cs.ucla.edu, johnw@gnu.org, emacs-devel@gnu.org
> 
> On 03/08/2016 05:45 PM, Eli Zaretskii wrote:
> 
> > You (and some others) say the format and the content in the log
> > messages are important, and I agree.  But if we do care about them,
> > how can we NOT clean them up?  Having them in their current state
> > means they cannot be trusted, which is worse than not having them at
> > all.
> 
> That's true. But my motivation for using ChangeLogs stems from having 
> people describe their changes in a strict format. If someone writes a 
> wrong name or omits the "copyright-exempt" header, I could live with that.
> 
> We should find out how much it is of a problem, though, legally speaking.

Not just legal aspect are at stake.  I tried to explain that in one of
my previous messages.

> >> Has the current experiment really sucked too much energy from anyone, aside from the implementors?
> >
> > Why do you think Glenn gave up?
> 
> My bad. All right, Glenn gave up fixing errors. Isn't that because 
> people made too much mistakes, and didn't bother to fix them?
> 
> Even if we transition to the previous system, it will need the same 
> people to fix their errors.

It is easy to ask someone to fix a mistake in a file and push the
change.  With the current system, fixing mistakes requires a much more
complex procedure, and also screws up merges to master.

> > The current system is much more hassle for non-random contributors, so
> > much so that we risk losing them, something we cannot afford.
> 
> Will someone decide to stop contributing to Emacs because our Change Log 
> entries contain mistakes? That doesn't sound very plausible.

When people like Glenn give up in despair, I think the danger is real.
IME, it's hard to be part of a project that ignores repeated requests
to fix something you believe must be fixed.

Re: Should we restore manually maintained ChangeLogs

[Dmitry Gutov]

On 03/08/2016 08:02 PM, Eli Zaretskii wrote:

> It is easy to ask someone to fix a mistake in a file and push the
> change.  With the current system, fixing mistakes requires a much more
> complex procedure, and also screws up merges to master.

And who's going to be that person who carefully reviews every ChangeLog 
entry? Fixing them yourself right away might actually be easier than 
messaging the commit authors every time, for each mistake. Even if that 
involves a "more complex" upfront procedure for each correction session.

> IME, it's hard to be part of a project that ignores repeated requests
> to fix something you believe must be fixed.

Many of us some have such things (ignored, or closed as wontfix, bug 
reports, for instance).

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

Eli Zaretskii <eliz@gnu.org> writes:
>> Yes, I was only mentioning it for the sake of completeness.  We have already
>> accomplished the second bullet of option #2, and have no intention of changing
>> that, as it does have a positive impact on our commit message quality.
>
>Just to put the record straight: we've been using the ChangeLog format
>in the commit log message since forever, certainly during the CVS
>days.  It's not a recent invention in any way.

I think it's a good requirement, though a quick trawl through the output of 'git log --name-status' shows that not everyone has always obeyed it.  (However, the standard does seem be more adhered to the closer we get to the present, so things have been moving in the right direction.)

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

>   - Commit messages cannot be edited.

>     This is an area where "git notes" could, in fact, help.  The idea
>     is not that we use git notes for every commit, to provide
>     ChangeLog data, but use them to "amend" commit log messages.
>     That way, when generating the ChangeLog for release, a git note
>     can override a commit message, correcting what appears in the
>     final ChangeLog.

I agree that this problem can be solved.  I'm far from convinced that
"git notes" can be part of such a solution, but if it works: fine by me.

But I don't want to only fix the generated ChangeLog files.  I'd like
the fixed text to appear at least in `vc-region-history' and
`vc-print-log'.
Otherwise, it's a lot of work that doesn't help me.


        Stefan

ChangeLog to *VC-log* (was: Is it time to drop ChangeLogs?)

[Stephen Berman]

On Tue, 08 Mar 2016 09:43:55 -0500 Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> 2016-03-08  Uwe Brauer  <oub@mat.ucm.es>
>> 	* gnus-init.el (gnus-user-format-function-G): 
>
> For this reasons, I have an (untracked) ChangeLog file in all
> my repositories.  This way I can use C-x 4 a to write a ChangeLog entry
> in the "good old way" and then VC will automatically copy that data into
> the *VC-Log* buffer for me to commit it.

This hasn't worked for me for some time, nor has `C-c C-a' in the
*VC-Log* buffer (though it used to work).  I just tried again: I edited
lisp/calendar/todo-mode.el, wrote a ChangeLog entry with `C-x 4 a',
typed `C-x v d' to get the VC-Dired buffer, typed v, got a *VC-Log*
buffer with no ChangeLog entry, typed `C-c C-a', the *VC-Log* buffer
remained empty.  Same thing after saving the ChangeLog buffer and
repeated the preceding steps.  Same thing when running VC-dired in the
Emacs source tree root.  Same thing with emacs -Q.  What am I doing
wrong?

Steve Berman

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Tue, 08 Mar 2016 05:23:46 +0100
>> 
>> >> Introduce code reviews. Don't give commit access to the "golden"
>> >> branches to everyone, just to a few top contributors and reviewers.
>> >
>> > We don't have manpower for that.
>> 
>> I find perplexing that we have manpower for reviewing ChangeLogs but not
>> for reviewing commit logs *instead*.
>
> I'm surprised that you find it perplexing.
>
> Log messages are much shorter than code diffs,

It is not necessary to review the diff (except for newcomers, possibly,
but that's an added bonus.)

[snip]

>> And is it so super-important to have typo-free commit logs?
>
> Typos is the least of our problems.  But yes, it's important: if you
> start by forgiving typos, you end with unhelpful and uninformative log
> messages.

Please note that typos are usually corrected by someone else, not by the
committer. AFAIR so far nobody received a warning for too much typos,
but everybody will see as a reasonable procedure to warn a contributor
for poor log messages. Which, as I said elsewhere, Emacs *already* has
plenty. I can't imagine what can be worse than what we have now except
for leaving a blank commit log or filling it with M-x cookie.

I'll reiterate my impression about ChangeLogs being a strong factor on
the lack of useful content of Emacs' commit messages.

>> We are not even using `git notes'.
>
> No one came up with a detailed procedure for doing that.  If you can
> propose something that works, please do.

It's simple: put the corrected commit log on a note. When the ChangeLogs
are generated, if a commit has a note, use the note instead of the
commit log message.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> `git notes' is a documented feature available since a long time ago. It
>> can turn the immutability of the commit log into a non-issue if you wish
>> to produce a proofread ChangeLog to include in the tarball.
>
> We need a more detailed proposal than this, TIA.  David says "git
> notes" is not a solution; if notes don't support merging, then I tend
> to think he is right.

But they do support merging.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

David Engster <deng@randomsample.de> writes:

[about git notes]

Thanks David for the informative post. Rigth now I have no time to read
the references you listed, but I'll do ASAP. This picked my curiosity.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

John Wiegley <jwiegley@gmail.com> writes:

>> The ChangeLog is useful for some of us. But I would contend that its
>> usefulness doesn't trump all the contributors we have discouraged by
>> insisting on our quite singular submission format.

ChangeLogs also have a big impact when long-lived branches need to be
merged, and IIRC are a major pain for external projects that
periodically merge with Emacs (org).

> Thank you, Lars, you have pretty much summed up my own concerns.
>
> To summarize several of the points in this thread so far:
>
>   - First, and most importantly, we have significant contributors who rely on
>     and enjoy the ChangeLog file -- even if there are others, and potential
>     contributors, who dislike it.
>
>     If people doing the "real work" are using this file, we should
>     keep it.

Why not explore the possibility of giving the information they want from
the VC system? As I see this issue, the ChangeLog is seen as
indispensable by those who don't know how to use Git. We can ask them
about their requirements and try to provide the necessary Emacs
commands.

[snip]

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 08 Mar 2016 21:28:24 +0100
> 
> John Wiegley <jwiegley@gmail.com> writes:
> 
> >> The ChangeLog is useful for some of us. But I would contend that its
> >> usefulness doesn't trump all the contributors we have discouraged by
> >> insisting on our quite singular submission format.
> 
> ChangeLogs also have a big impact when long-lived branches need to be
> merged, and IIRC are a major pain for external projects that
> periodically merge with Emacs (org).

With or without git-merge-changelog installed?  With it the problems
should be minimal or non-existent, IME.

> >   - First, and most importantly, we have significant contributors who rely on
> >     and enjoy the ChangeLog file -- even if there are others, and potential
> >     contributors, who dislike it.
> >
> >     If people doing the "real work" are using this file, we should
> >     keep it.
> 
> Why not explore the possibility of giving the information they want from
> the VC system? As I see this issue, the ChangeLog is seen as
> indispensable by those who don't know how to use Git. We can ask them
> about their requirements and try to provide the necessary Emacs
> commands.

Using Git is not a problem for me.  The problem is that the
information in Git log is unreliable.  The other problem is that will
never succeed in teaching new contributors how to make good log
messages unless we have an easy way of fixing mistakes there.

Re: Is it time to drop ChangeLogs?

[David Caldwell]

[-- Attachment #1: Type: text/plain, Size: 682 bytes --]

On 3/8/16 11:57 AM, Óscar Fuentes wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
>>> From: Óscar Fuentes <ofv@wanadoo.es>

>>> We are not even using `git notes'.
>>
>> No one came up with a detailed procedure for doing that.  If you can
>> propose something that works, please do.
> 
> It's simple: put the corrected commit log on a note. When the ChangeLogs
> are generated, if a commit has a note, use the note instead of the
> commit log message.

Since notes seem like a no-go, what about taking the same approach but
using an empty commit to do it (`git commit --allow-empty`)? That way it
gets pushed and merged between branches just like normal.

-David


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3819 bytes --]

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 03/08/2016 10:56 PM, David Caldwell wrote:

> Since notes seem like a no-go, what about taking the same approach but
> using an empty commit to do it (`git commit --allow-empty`)? That way it
> gets pushed and merged between branches just like normal.

How would such commit indicate a relation to an existing commit? And 
after making it, you can't easily edit the result, you can only redo it 
fully.

How about putting all corrections as plain files in a subdirectory? Each 
file will be named after a commit whose message it's "changing". IIRC, 
I've seen such idea mentioned before, and it seems like it should work.

We could even implement integration with vc-print-log without too much 
difficulty. The main thing to solve is the cherrypick commits (does the 
correction apply to whose? always?); but as long as cherrypicks include 
the references to their parents in the message, it should be workable, 
one way or another.

Re: Is it time to drop ChangeLogs?

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> On 03/08/2016 10:56 PM, David Caldwell wrote:
>> Since notes seem like a no-go, what about taking the same approach but
>> using an empty commit to do it (`git commit --allow-empty`)? That way it
>> gets pushed and merged between branches just like normal.

> How would such commit indicate a relation to an existing commit? And after
> making it, you can't easily edit the result, you can only redo it fully.

What if we had a ADDENDUM file with the following contents:

    <COMMIT ID><NEWLINE>
    <REPLACEMENT LOG MESSAGE><NEWLINE>
    <NEWLINE>

Then we could teach vc-region-history and the ChangeLog generator to pay
attention to it.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Is it time to drop ChangeLogs?

[Ingo Lohmar]

On Tue, Mar 08 2016 22:37 (+0200), Eli Zaretskii wrote:
> Using Git is not a problem for me.  The problem is that the
> information in Git log is unreliable.  The other problem is that will
> never succeed in teaching new contributors how to make good log
> messages unless we have an easy way of fixing mistakes there.


Some arguments in this thread are repeated ad infinitum although they
don't seem to stand a little scrutiny.  "git log" messages cannot
technically be both immutable and unreliable: At least there is some
severely imprecise use of language going on.

As to the teaching argument: I have read every single message in this
thread, and nobody has argued for lower (but several people for higher)
commit message standards.

In contrast to your opinion, it seems to me that fixing mistakes in the
Changelogs teaches a contributor who has committed with a flawed commit
message that it's not really important.  They, or somebody else, can
clean up their (incl. possibly my) mess.  As Oscar has argued, having
the original commit rejected (by means to be discussed, and only until
people have shown good judgment and discipline) teaches them that commit
messages matter.

The whole argument for Changelogs comes down to a) being an established
band-aid to clean up spilt milk, or b) providing a fixed-form summary of
things that can be obtained using the VCS (provided the humans or tools
wirting the Changelog are as "reliable" as the VCS).

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 03/08/2016 11:23 PM, John Wiegley wrote:

> What if we had a ADDENDUM file with the following contents:
>
>     <COMMIT ID><NEWLINE>
>     <REPLACEMENT LOG MESSAGE><NEWLINE>
>     <NEWLINE>
>
> Then we could teach vc-region-history and the ChangeLog generator to pay
> attention to it.

That's close to my suggestion, except it's using one file instead of 
many. I guess the question is what will work better: many small files in 
a directory, or a huge one.

Also note that a log message usually contains multiple newlines already.

Re: Is it time to drop ChangeLogs?

[David Caldwell]

[-- Attachment #1: Type: text/plain, Size: 1474 bytes --]

On 3/8/16 1:16 PM, Dmitry Gutov wrote:
> On 03/08/2016 10:56 PM, David Caldwell wrote:
> 
>> Since notes seem like a no-go, what about taking the same approach but
>> using an empty commit to do it (`git commit --allow-empty`)? That way it
>> gets pushed and merged between branches just like normal.
> 
> How would such commit indicate a relation to an existing commit?

I was thinking something along the lines of "reword: <HASH>" in the message.

> And after making it, you can't easily edit the result, you can only
> redo it fully.

Correct. Newest one wins.

> How about putting all corrections as plain files in a subdirectory? Each
> file will be named after a commit whose message it's "changing". IIRC,
> I've seen such idea mentioned before, and it seems like it should work.

Yeah, It would work as well. I just thought it would be nice to keep
meta-data corrections in the meta-data itself.

> We could even implement integration with vc-print-log without too much
> difficulty. The main thing to solve is the cherrypick commits (does the
> correction apply to whose? always?);

Cherry-picks would definitely require more effort.

> but as long as cherrypicks include the references to their parents in
> the message, it should be workable, one way or another.

I don't believe they do, by default. And if they need to be amended to
include that, you might as well amend to include the fixed commit-log
instead.

-David



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3819 bytes --]

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

David Caldwell <david@porkrind.org> writes:

>>>> We are not even using `git notes'.
>>>
>>> No one came up with a detailed procedure for doing that.  If you can
>>> propose something that works, please do.
>> 
>> It's simple: put the corrected commit log on a note. When the ChangeLogs
>> are generated, if a commit has a note, use the note instead of the
>> commit log message.
>
> Since notes seem like a no-go,

After reading the first of the references listed by David, I see nothing
that indicates that `git notes' is inadequate for *this* use case. I
still need to study the rest of his message.

Re: Is it time to drop ChangeLogs?

[Alan Mackenzie]

Hello, Ingo.

On Tue, Mar 08, 2016 at 10:25:35PM +0100, Ingo Lohmar wrote:
> On Tue, Mar 08 2016 22:37 (+0200), Eli Zaretskii wrote:
> > Using Git is not a problem for me.  The problem is that the
> > information in Git log is unreliable.  The other problem is that will
> > never succeed in teaching new contributors how to make good log
> > messages unless we have an easy way of fixing mistakes there.


> Some arguments in this thread are repeated ad infinitum although they
> don't seem to stand a little scrutiny.  "git log" messages cannot
> technically be both immutable and unreliable: At least there is some
> severely imprecise use of language going on.

If a git log message starts off life unreliable (i.e. there are mistakes
in it) it can never be corrected.  It stays unreliable for the lifetime
of the repository.

> As to the teaching argument: I have read every single message in this
> thread, and nobody has argued for lower (but several people for higher)
> commit message standards.

> In contrast to your opinion, it seems to me that fixing mistakes in the
> Changelogs teaches a contributor who has committed with a flawed commit
> message that it's not really important.  They, or somebody else, can
> clean up their (incl. possibly my) mess.  As Oscar has argued, having
> the original commit rejected (by means to be discussed, and only until
> people have shown good judgment and discipline) teaches them that commit
> messages matter.

I am an experienced Emacs contributor, and I have, even recently, made
mistakes in my commit messages.  I resent the fact that it is so
difficult to correct them, even before pushing to savannah.  I don't
think I need any teaching on the importance of good commit messages.

> The whole argument for Changelogs comes down to a) being an established
> band-aid to clean up spilt milk, or b) providing a fixed-form summary of
> things that can be obtained using the VCS (provided the humans or tools
> wirting the Changelog are as "reliable" as the VCS).

Not everybody has access to the git repository, and not everybody who
has is capable of using it effectively.  ChangeLogs remain a useful,
easy to use summary of Emacs's progress.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 03/08/2016 11:27 PM, David Caldwell wrote:

>> How would such commit indicate a relation to an existing commit?
>
> I was thinking something along the lines of "reword: <HASH>" in the message.

That would work.

>> How about putting all corrections as plain files in a subdirectory? Each
>> file will be named after a commit whose message it's "changing". IIRC,
>> I've seen such idea mentioned before, and it seems like it should work.
>
> Yeah, It would work as well. I just thought it would be nice to keep
> meta-data corrections in the meta-data itself.

I figured it would be nice to be able to quickly find it. In 
vc-print-log integration code, for example.

> I don't believe they do, by default. And if they need to be amended to
> include that, you might as well amend to include the fixed commit-log
> instead.

The do when produced like we recommend in admin/notes/git-workflow (with 
-x argument). If it doesn't, it's not the end of the world. Someone will 
just have to issue a correction for the new hash as well.

Re: Is it time to drop ChangeLogs?

[Giuseppe Scrivano]

John Wiegley <jwiegley@gmail.com> writes:

>>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> On 03/08/2016 10:56 PM, David Caldwell wrote:
>>> Since notes seem like a no-go, what about taking the same approach but
>>> using an empty commit to do it (`git commit --allow-empty`)? That way it
>>> gets pushed and merged between branches just like normal.
>
>> How would such commit indicate a relation to an existing commit? And after
>> making it, you can't easily edit the result, you can only redo it fully.
>
> What if we had a ADDENDUM file with the following contents:
>
>     <COMMIT ID><NEWLINE>
>     <REPLACEMENT LOG MESSAGE><NEWLINE>
>     <NEWLINE>
>
> Then we could teach vc-region-history and the ChangeLog generator to pay
> attention to it.

the git-to-changelog gnulib module already supports something similar
via --amend=FILE.

For example coreutils is keeping a list of corrections to make to the
git commit messages before the ChangeLog is generated:
http://git.savannah.gnu.org/cgit/coreutils.git/tree/build-aux/git-log-fix

Regards,
Giuseppe

Re: ChangeLog to *VC-log*

[Dmitry Gutov]

On 03/08/2016 09:21 PM, Stephen Berman wrote:

> This hasn't worked for me for some time, nor has `C-c C-a' in the
> *VC-Log* buffer (though it used to work).  I just tried again: I edited
> lisp/calendar/todo-mode.el, wrote a ChangeLog entry with `C-x 4 a',

Where does the ChangeLog file reside, in which you make that entry? If 
it's not at the top level, try deleting that one.

Re: ChangeLog to *VC-log*

[Stephen Berman]

On Wed, 9 Mar 2016 00:26:10 +0200 Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 03/08/2016 09:21 PM, Stephen Berman wrote:
>
>> This hasn't worked for me for some time, nor has `C-c C-a' in the
>> *VC-Log* buffer (though it used to work).  I just tried again: I edited
>> lisp/calendar/todo-mode.el, wrote a ChangeLog entry with `C-x 4 a',
>
> Where does the ChangeLog file reside, in which you make that entry? If it's
> not at the top level, try deleting that one.

Ah, thanks.  I'm pretty sure it used to be the case that `C-x 4 a' would
automatically find the right ChangeLog file, but I suppose that changed
when manually maintaining ChangeLogs was abandoned.  So I guess Emacs
developers should now set `change-log-default-name' to top level.

Steve Berman

Re: ChangeLog to *VC-log*

[Dmitry Gutov]

On 03/09/2016 01:07 AM, Stephen Berman wrote:

> Ah, thanks.  I'm pretty sure it used to be the case that `C-x 4 a' would
> automatically find the right ChangeLog file, but I suppose that changed
> when manually maintaining ChangeLogs was abandoned.  So I guess Emacs
> developers should now set `change-log-default-name' to top level.

Probably. If you have a solid reproduction scenario, please make a bug 
report.

Re: Is it time to drop ChangeLogs?

[Eric Abrahamsen]

Nikolaus Rath <Nikolaus@rath.org> writes:

> On Mar 08 2016, Eric Abrahamsen <eric@ericabrahamsen.net> wrote:
>> Nikolaus Rath <Nikolaus@rath.org> writes:
>>
>>> On Mar 07 2016, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>>>> When I submitted my first Emacs patch, I was astonished when I was asked
>>>>> to re-submit with my commit message essentially duplicated in the
>>>>> ChangeLog.
>>>>
>>>> I can't remember ever asking this kind of duplication.  We used to
>>>> duplicate the ChangeLog-info in the ChangeLog file and in the commit
>>>> message (tho we stopped doing it a year ago), but we never asked for it
>>>> to be duplicated in the email messages that submits a patch.
>>>>
>>>> So I have the impression that I'm misunderstanding you.  Could you
>>>> describe more precisely the kind of duplication you're talking about?
>>>
>>> No, you understood me correctly. However, I've just looked this up again
>>> and it turns out I was actually submitting a patch to Gnus, not
>>> Emacs. It was eventually committed by Eric Abrahamsen to the Gnus git
>>> repo, who first asked me to re-submit my patch with a commit message
>>> that was formatted like a Changelog entry, and to duplicate the message
>>> in the ChangeLog itself. It seemed to me that he considered that a
>>> questionable practice himself, but said it was obligatory because Gnus
>>> closely followed the Emacs standards here.
>>>
>>>
>>> Unfortunately that discussion never made it into the bug
>>> (http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20956), so I'm
>>> paraphrasing from memory. But I'm pretty sure I'm remembering correctly,
>>> because I was so surprised by this.
>>
>> Not duplicating it in the email, just commit message + ChangeLog
>> duplication. This may be getting conflated with me asking for the commit
>> to be emailed as a git-format-patch attachment, which was just because I
>> was lazy (and it also preserves the author/committer distinction).
>
> Actually, I was quite happy about that. I very much prefer if patches
> that I submit also have me as the author of the commit.

That was my thinking, though as Eli and others point out, it's usually
done differently.

> (Eric, I think my last email sounded a bit like telling on you. I'm
> sorry about that, I think I should have written it differently (or asked
> you first). I'm very grateful for the time you took to help me with
> those patches, and for patiently explaining the rationale for the
> format).

No worries, I didn't read it that way! I'm very much not a maintainer,
just someone who occasionally patches Gnus when Lars is on vacation, so
it's been a learning experience for me, too.

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/08/2016 09:05 AM, Eli Zaretskii wrote:
> By giving up on the need to fix them, we
> inadvertently send a very loud and clear message to the newcomers
> saying that good, correct, and accurate log messages are not
> important.

We also send that message by giving up on the need to write good commit 
messages in the first place. If we were to tell newcomers something like 
"Thanks, that was a nice patch, and could you please add a commit 
message following the guidelines so that we can install it for you?" 
then they will figure things out. But if we instead keep installing such 
patches as-is and maybe fixing the commit messages ourselves later, we 
are in effect telling contributors not to bother writing good commit 
messages.

Re: ChangeLog to *VC-log*

[Stefan Monnier]

> automatically find the right ChangeLog file, but I suppose that changed
> when manually maintaining ChangeLogs was abandoned.  So I guess Emacs
> developers should now set `change-log-default-name' to top level.

AFAIK the C-x 4 a code hasn't been changed, so if there's a change in
behavior it's most likely an accident/bug.  Please report it via M-x
report-emacs-bug with a detailed recipe.


        Stefan

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ingo Lohmar <i.lohmar@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 08 Mar 2016 22:25:35 +0100
> 
> On Tue, Mar 08 2016 22:37 (+0200), Eli Zaretskii wrote:
> > Using Git is not a problem for me.  The problem is that the
> > information in Git log is unreliable.  The other problem is that will
> > never succeed in teaching new contributors how to make good log
> > messages unless we have an easy way of fixing mistakes there.
> 
> 
> Some arguments in this thread are repeated ad infinitum although they
> don't seem to stand a little scrutiny.

That's because some people don't seem to read the thread, and keep
coming up with the same incorrect arguments time and again.

> "git log" messages cannot technically be both immutable and
> unreliable: At least there is some severely imprecise use of
> language going on.

You need to read the thread to understand what is being alluded to.
"Unreliable" in the sense that its text includes mistakes and cannot
be trusted.

> In contrast to your opinion, it seems to me that fixing mistakes in the
> Changelogs teaches a contributor who has committed with a flawed commit
> message that it's not really important.  They, or somebody else, can
> clean up their (incl. possibly my) mess.  As Oscar has argued, having
> the original commit rejected (by means to be discussed, and only until
> people have shown good judgment and discipline) teaches them that commit
> messages matter.

Emacs development doesn't work by requiring each commit be posted for
review as prerequisite for committing, so what Oscar suggests is not
possible.  (Please don't ask why, it was explained many times
already.)

> The whole argument for Changelogs comes down to a) being an established
> band-aid to clean up spilt milk, or b) providing a fixed-form summary of
> things that can be obtained using the VCS (provided the humans or tools
> wirting the Changelog are as "reliable" as the VCS).

Find a better and more reliable way of dealing with the problems
described here, and I'll be the first to agree not to reintroduce
ChangeLogs.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 8 Mar 2016 17:08:39 -0800
> 
> On 03/08/2016 09:05 AM, Eli Zaretskii wrote:
> > By giving up on the need to fix them, we
> > inadvertently send a very loud and clear message to the newcomers
> > saying that good, correct, and accurate log messages are not
> > important.
> 
> We also send that message by giving up on the need to write good commit 
> messages in the first place. If we were to tell newcomers something like 
> "Thanks, that was a nice patch, and could you please add a commit 
> message following the guidelines so that we can install it for you?" 
> then they will figure things out. But if we instead keep installing such 
> patches as-is and maybe fixing the commit messages ourselves later, we 
> are in effect telling contributors not to bother writing good commit 
> messages.

The problems are with those who have commit rights.

Re: Is it time to drop ChangeLogs?

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> Find a better and more reliable way of dealing with the problems described
> here, and I'll be the first to agree not to reintroduce ChangeLogs.

As far as I understand so far, we have only a few needs. If I've understood
incorrectly, please let me know, as this thread has wandered in a few places.

 1. We need a way to ensure quality commit messages from contributors.
 2. We'd like to be able to correct commit messages after the fact.
 3. We need a convenient way to both create and access this information.
 4. We want to publish the history of these commit messages in the tarball.

The ChangeLog file and its format are one implementation of these needs that
has worked over the years. Are you saying that if we move to something else,
and it fulfills these criteria, you'd be just as happy with it?

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1475 bytes --]

Eli Zaretskii wrote:
>> My own experience is otherwise. For the kinds of development I do, I rarely see ChangeLog screwups now, whereas I used to see them routinely.
>
> With or without git-merge-changelog?

Without.  That program is not normally installed.  And I rarely do merges so I 
don't see why it would help.  I recall trying to use it a while ago and had 
trouble (sorry, do not recall details).

> What alternatives that don't "cater to
> all sides" would you suggest?  The only one I see is to stop producing
> ChangeLog files for the releases.

That's what Guile does and it works OK.

If we want to be more traditional and keep ChangeLog files in releases, we can 
do what coreutils etc. do. They autogenerate ChangeLog files for releases, but 
do not put these ChangeLog files in their repositories. They have a way to fix 
typos in the autogenerated ChangeLog files. It works well enough, as long as 
typo fixes are rare enough (which they should be). This is all a bit more 
complicated than what Guile does, but it's simpler than what Emacs does now, and 
it preserves most of the advantages of what Emacs does now.

> Please describe the details of your proposal.

For the more-traditional approach, apply the attached patch to emacs-25, and 
merge it to master.  Other branches can pick it up as needed.

We can easily implement the Guile approach too (it's even simpler), though it 
sounds like you prefer the more-traditional approach, at least for now.

[-- Attachment #2: 0001-Simplify-autogeneration-of-top-level-ChangeLog.patch --]
[-- Type: text/x-diff, Size: 11777 bytes --]

From c49feeacc33772212ebf4c53a05adb169d8b1d9b Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Tue, 8 Mar 2016 23:49:05 -0800
Subject: [PATCH] Simplify autogeneration of top-level ChangeLog

Use the simpler approach of coreutils, etc.  Do not maintain
ChangeLog files in the repository (except for files already
present, which are grandfathered in).  Insted, generate a
ChangeLog file when making a distribution tarball.  Any typos
in the generated ChangeLog file can be fixed by editing the
new file build-aux/git-log-fix.
* CONTRIBUTE (Commit messages):
* admin/make-tarball.txt:
* admin/notes/repo (Maintaining ChangeLog history):
Adjust documentation accordingly.
* Makefile.in (emacslog, CHANGELOG_N, emacs-25-branch-is-current)
(unchanged-history-files, new_commit_regexp)
(change-history-nocommit, change-history, change-history-commit):
Remove; no longer needed.
(gen-ChangeLog): Use an approach like Coreutils.  Rename from
ChangeLog, for consistency with coreutils.
* admin/update_autogen (usage, changelog_flag):
Remove -H option, since update_autogen no longer updates ChangeLog.
* build-aux/git-log-fix: New file, with format copied from coreutils.
* build-aux/gitlog-to-emacslog: New option --amend=FILE.
* make-dist: Adjust to ChangeLog -> gen-ChangeLog renaming.
---
 CONTRIBUTE                   |  5 ++--
 Makefile.in                  | 61 ++++++++++++--------------------------------
 admin/make-tarball.txt       |  4 +--
 admin/notes/repo             | 12 +++------
 admin/update_autogen         | 14 +---------
 build-aux/git-log-fix        | 32 +++++++++++++++++++++++
 build-aux/gitlog-to-emacslog |  6 ++++-
 make-dist                    |  2 +-
 8 files changed, 63 insertions(+), 73 deletions(-)
 create mode 100644 build-aux/git-log-fix

diff --git a/CONTRIBUTE b/CONTRIBUTE
index 5102b4f..878ac8e 100644
--- a/CONTRIBUTE
+++ b/CONTRIBUTE
@@ -56,8 +56,9 @@ Here is an example commit message (indented):
 	* src/frame.c (Fhandle_switch_frame, Fselected_frame):
 	Deactivate the mark.
 
-Occasionally, commit messages are collected and prepended to a
-ChangeLog file, where they can be corrected.  It saves time to get
+When generating a release, commit messages are collected into a
+ChangeLog file for the release tarball.  Although errors in these commit
+messages can be fixed (see build-aux/git-log-fix), it saves time to get
 them right the first time, so here are guidelines for formatting them:
 
 - Start with a single unindented summary line explaining the change;
diff --git a/Makefile.in b/Makefile.in
index b212c91..49c133c 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -1092,54 +1092,25 @@ bootstrap: bootstrap-clean
 	$(MAKE) MAKEFILE_NAME=force-Makefile force-Makefile
 	$(MAKE) all
 
-.PHONY: ChangeLog change-history change-history-commit change-history-nocommit
-.PHONY: emacs-25-branch-is-current unchanged-history-files
-
 CHANGELOG = ChangeLog
-emacslog = build-aux/gitlog-to-emacslog
 # The ChangeLog history files are called ChangeLog.1, ChangeLog.2, ...,
-# ChangeLog.$(CHANGELOG_HISTORY_INDEX_MAX).  $(CHANGELOG_N) stands for
-# the newest (highest-numbered) ChangeLog history file.
+# ChangeLog.$(CHANGELOG_HISTORY_INDEX_MAX).  These files are left over
+# from the old way, where ChangeLogs were kept in the repository.
 CHANGELOG_HISTORY_INDEX_MAX = 2
-CHANGELOG_N = ChangeLog.$(CHANGELOG_HISTORY_INDEX_MAX)
-
-# Convert git commit log to ChangeLog file.  make-dist uses this.
-# I guess this is PHONY so it always updates?
-ChangeLog:
-	$(AM_V_GEN)cd $(srcdir) && \
-	  ./$(emacslog) -o $(CHANGELOG) -n $(CHANGELOG_HISTORY_INDEX_MAX)
-
-# Check that we are in a good state for changing history.
-emacs-25-branch-is-current:
-	git branch | grep -q '^\* emacs-25$$'
-unchanged-history-files:
-	x=$$(git diff-files --name-only $(CHANGELOG_N) $(emacslog)) && \
-	  test -z "$$x"
-
-# Regular expression that matches the newest commit covered by a ChangeLog.
-new_commit_regexp = ^commit [0123456789abcdef]* (inclusive)
-
-# Copy newer commit messages to the start of the ChangeLog history file,
-# and consider them to be older.
-change-history-nocommit: emacs-25-branch-is-current unchanged-history-files
-	-rm -f ChangeLog.tmp
-	$(MAKE) ChangeLog CHANGELOG=ChangeLog.tmp
-	sed '/^This file records repository revisions/,$$d' \
-	  ChangeLog.tmp >$(CHANGELOG_N).tmp
-	new_commit_line=`grep '$(new_commit_regexp)' ChangeLog.tmp` && \
-	sed 's/$(new_commit_regexp).*/'"$$new_commit_line/" \
-	  $(CHANGELOG_N) >>$(CHANGELOG_N).tmp
-	rm ChangeLog.tmp
-	mv $(CHANGELOG_N).tmp $(CHANGELOG_N)
-
-change-history: change-history-nocommit
-	$(MAKE) $@-commit
-
-# If 'make change-history' fails because the newest ChangeLog history
-# file contains invalid text, fix the file by hand and then run
-# 'make change-history-commit'.
-change-history-commit:
-	git commit -m'; make $@' $(CHANGELOG_N) $(emacslog)
+
+# Convert more-recent git commit messages to a ChangeLog file.
+# make-dist uses this.
+.PHONY: gen-ChangeLog
+gen-ChangeLog:
+	$(AM_V_GEN)if test -d .git; then				\
+	  log_fix="$(srcdir)/build-aux/git-log-fix";			\
+	  test -e "$$log_fix"						\
+	    && amend_git_log="--amend=$$log_fix"			\
+	    || amend_git_log=;						\
+	  $(srcdir)/build-aux/gitlog-to-emacslog			\
+	    -f -n $(CHANGELOG_HISTORY_INDEX_MAX) -o $(CHANGELOG)	\
+	    $$amend_git_log;						\
+	fi
 
 .PHONY: check-declare
 
diff --git a/admin/make-tarball.txt b/admin/make-tarball.txt
index 030ad4c..68d0209 100644
--- a/admin/make-tarball.txt
+++ b/admin/make-tarball.txt
@@ -38,8 +38,8 @@ General steps (for each step, check for possible errors):
       M-x authors RET
 
     If there is an "*Authors Errors*" buffer, address the issues.
-    If there was a ChangeLog typo, run "make change-history" and then
-    fix the newest ChangeLog history file.  If a file was deleted or
+    If there was an important ChangeLog typo, edit
+    build-aux/git-log-fix to fix it.  If a file was deleted or
     renamed, consider adding an appropriate entry to
     authors-ignored-files, authors-valid-file-names, or
     authors-renamed-files-alist.
diff --git a/admin/notes/repo b/admin/notes/repo
index 3ab3da7..bd0d15a 100644
--- a/admin/notes/repo
+++ b/admin/notes/repo
@@ -124,13 +124,7 @@ ChangeLog.2, etc., and can be edited just as any other source files
 can.  Newer ChangeLog entries are stored in the repository as commit
 messages, which cannot be edited directly.
 
-'make ChangeLog' copies newer ChangeLog entries into a file
+'make gen-ChangeLog' copies newer ChangeLog entries into a file
 'ChangeLog' that is intended to be put into the distribution tarball.
-This ChangeLog file is not put into the repository.
-
-'make change-history' copies all newer ChangeLog entries into the
-start of the newest ChangeLog history file.  These ChangeLog entries
-are thereafter considered to be old, so later uses of 'make ChangeLog'
-and/or 'make change-history' will no longer copy the entries.  To
-alter ChangeLog history, run 'make change-history', then edit
-the ChangeLog history files manually and commit your changes.
+This ChangeLog file is not put into the repository.  To correct an
+error in a newer ChangeLog entry, edit build-aux/git-log-fix.
diff --git a/admin/update_autogen b/admin/update_autogen
index 199a3aa..7065e22 100755
--- a/admin/update_autogen
+++ b/admin/update_autogen
@@ -69,7 +69,6 @@ Options:
     commit them (caution).
 -q: be quiet; only give error messages, not status messages.
 -A: only update autotools files, copying into specified dir.
--H: also update ChangeLog.${changelog_n}
 -I: also update info/dir.
 -L: also update ldefs-boot.el.
 -C: start from a clean state.  Slower, but more correct.
@@ -88,7 +87,6 @@ autogendir=                     # was "autogen"
 ldefs_flag=1
 lboot_flag=
 info_flag=
-changelog_flag=
 
 ## Parameters.
 ldefs_in=lisp/loaddefs.el
@@ -117,7 +115,7 @@ tempfile=/tmp/$PN.$$
 trap "rm -f $tempfile 2> /dev/null" EXIT
 
 
-while getopts ":hcfqA:HCIL" option ; do
+while getopts ":hcfqA:CIL" option ; do
     case $option in
         (h) usage ;;
 
@@ -133,8 +131,6 @@ while getopts ":hcfqA:HCIL" option ; do
 
         (C) clean=1 ;;
 
-        (H) changelog_flag=1 ;;
-
         (I) info_flag=1 ;;
 
         (L) lboot_flag=1 ;;
@@ -388,14 +384,6 @@ modified=$(status $genfiles $ldefs_out) || die
 commit "loaddefs" $modified || die "commit error"
 
 
-## Less important than the other stuff, so do it last.
-[ ! "$changelog_flag" ] || {
-    make change-history-nocommit || die "make change-history error"
-    modified=$(status $changelog_files) || die
-    commit "ChangeLog" $modified || die "commit error"
-}
-
-
 exit 0
 
 ### update_autogen ends here
diff --git a/build-aux/git-log-fix b/build-aux/git-log-fix
new file mode 100644
index 0000000..8976682
--- /dev/null
+++ b/build-aux/git-log-fix
@@ -0,0 +1,32 @@
+# Changes to automatically-generated ChangeLog
+
+# Copyright 2016 Free Software Foundation, Inc.
+
+# This file is part of GNU Emacs.
+
+# GNU Emacs is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# GNU Emacs is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+
+# You should have received a copy of the GNU General Public License
+# along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.
+
+
+# This file is expected to be used via gitlog-to-changelog's --amend=FILE
+# option.  It specifies what changes to make to each given SHA1's commit
+# log and metadata, using Perl-eval'able expressions.
+
+# For examples of what you can put into this file, see what Coreutils does:
+# http://git.savannah.gnu.org/cgit/coreutils.git/tree/build-aux/git-log-fix
+
+
+b1abce1a30c66a22766e3d4b8b4ff9ae852f150c
+# This example uniformly replaces one phrase by another, as a test.
+# We can remove this example once this file contains real corrections.
+s/leading space/leading ' '/g
diff --git a/build-aux/gitlog-to-emacslog b/build-aux/gitlog-to-emacslog
index bcc47b1..8d12245 100755
--- a/build-aux/gitlog-to-emacslog
+++ b/build-aux/gitlog-to-emacslog
@@ -25,12 +25,16 @@ export LC_ALL
 # The newest revision that should not appear in the generated ChangeLog.
 gen_origin=
 
+# Whether to amend the git log.  The default is no amendments.
+amend_git_log=--amend=/dev/null
+
 force=
 output=ChangeLog
 nmax=2
 
 while [ $# -gt 0 ]; do
   case "$1" in
+      --amend=*) amend_git_log=$1 ;;
       -g|--gen-origin) gen_origin="$2" ; shift ;;
       -f|--force) force=1 ;;
       -n|--nmax) nmax="$2"; shift ;;
@@ -78,7 +82,7 @@ test -d .git || {
 # See eg the cairo-related ones.
 ./build-aux/gitlog-to-changelog \
     --ignore-matching="^; |^Merge branch '(master|emacs-[0-9][0-9])' of git\.(savannah|sv)\.gnu\.org:/srv/git/emacs$|^Merge remote-tracking branch '.*'$" \
-  --ignore-line='^; ' --format='%B' \
+  --ignore-line='^; ' --format='%B' "$amend_git_log" \
   "$gen_origin..$new_origin" >"ChangeLog.tmp" || exit
 
 if test -s "ChangeLog.tmp"; then
diff --git a/make-dist b/make-dist
index 1cd1a50..9fa20c4 100755
--- a/make-dist
+++ b/make-dist
@@ -286,7 +286,7 @@ mkdir ${tempdir}
 if [ "$changelog" = yes ]; then
   if test -d .git; then
     echo "Making top-level ChangeLog"
-    make ChangeLog CHANGELOG=${tempdir}/ChangeLog || \
+    make CHANGELOG=${tempdir}/ChangeLog gen-ChangeLog || \
       { x=$?; echo "make ChangeLog FAILED (try --no-changelog?)" >&2; exit $x; }
   else
     echo "No repository, so omitting top-level ChangeLog"
-- 
2.5.0

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

> can do what coreutils etc. do.  They autogenerate ChangeLog files for
> releases, but do not put these ChangeLog files in their repositories.
> They have a way to fix typos in the autogenerated ChangeLog files.

So they go though the trouble of fixing typos, but only in the ChangeLog
files in the release tarballs (where *very* few people will ever look)?


        Stefan

Re: Code reviews

[Phillip Lord]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> Maybe we could have a half-way system, where commits are pushed to
>>> a branch that is "not fast-forward-only", and this branch is then
>>> auto-merged to the real (fast-forward-only) master branch after a delay
>>> (one day, maybe?) to give time to fix mess ups before they're cast
>>> in stone.
>> I think that this would require some considerable co-ordination.  If I
>> push a broken commit to devel branch, and then you fix this commit
>> through rebase, my copy of the branch (and everyone elses) is now
>> broken.  I'm pretty sure that this kind of fix up can only happen on a
>> feature branch in a sane way.
>
> I think you misunderstood me: what I was thinking about in the quoted
> text is a system where you submit a pull request to a review queue (not
> to some kind of shared "proto-master" branch), and those requests are
> automatically accepted after some timeout.


Oh. You mean "where commits are pushed to a branch" as in "some branch"
rather than "all commits are pushed to a specific branch".

So, essentially a PR system with a time out.

That would be quite nice.

Phil

Re: Code reviews

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: phillip.lord@russet.org.uk (Phillip Lord)
>> Date: Tue, 08 Mar 2016 08:48:56 +0000
>> Cc: emacs-devel@gnu.org
>> 
>> Giving people different permissions on different branches seems to make
>> sense. Overtime, people move toward master/emacs-25.
>
> AFAIK, Git (or is it Savannah?) doesn't support such granularity.

Yes, it does, through hooks. I think, most people just use gitolite for
the purpose.


>> > Maybe we could have a half-way system, where commits are pushed to
>> > a branch that is "not fast-forward-only", and this branch is then
>> > auto-merged to the real (fast-forward-only) master branch after a delay
>> > (one day, maybe?) to give time to fix mess ups before they're cast
>> > in stone.
>> 
>> I think that this would require some considerable co-ordination. If I
>> push a broken commit to devel branch, and then you fix this commit
>> through rebase, my copy of the branch (and everyone elses) is now
>> broken.
>
> Stefan was talking about merging, not rebasing.

I think he was talking about both. "fix mess ups" sounds like a rebase
to me. But I was confused about his overall workflow.

Phil

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Ingo Lohmar <i.lohmar@gmail.com>,  ofv@wanadoo.es,  emacs-devel@gnu.org
> Date: Tue, 08 Mar 2016 22:41:49 -0800
> 
> Keeping ChangeLog style in the commit entry is not terribly
> useful either, since the diff output of log -p lets you know which function or
> variable is being modified.

"git log -p" cannot do the job for changes in many types of files.
For example, try it on Lisp or Texinfo files.

More generally, there's no way Git could replace ChangeLog style
entries, because they frequently include information that is not in
the diffs.  To say nothing of the fact that understanding the change
from reading Diff hunks is much harder, and therefore much less
efficient, than from reading a log entry which describes the change in
plain English.

> I've never missed not having that ChangeLog data in other projects,
> of any size.

Maybe you rarely need to do any forensics.  Me, I do it all the time
in Emacs, and ChangeLog files are a valuable tool in the chest.

======================================================================

(Why is this topic cross-posted to bug-gnu-Emacs??)

> In my short experience, Change Logs has generally been useful both when
> reading and composing them.  When writing them it helps me structure
> large changes in logical commits that are modelled by the Change Log
> format.  Finally It helps me being precise in my wordings which is not
> trivial for non-native english speakers.
> 
> On a more spiritual side, I think they belong to the zen of contributing
> to a GNU project.  :)

I agree completely.  Writing a log entry in ChangeLog format is an
excellent opportunity for reflecting on the changeset, for summarizing
its intent and final shape, and for making sure nothing was left out.
Writing those entries teaches one discipline, the ability to describe
your changes in just enough detail, and facilitates communications
between members of the development team.  And in loose teams such as
ours, good communications are everything.

As told many times in past discussions, ChangeLog files are also an
excellent first tool for forensics, easy to search with many available
tools.  It is invaluable when you don't have access to the repository,
and a good asset even if you do: traversing the history of a complex
Git repo without losing commits on branches is not a trivial task, it
requires using non-default switches and some rarely used commands and
options.  Even when you do use Git tools, ChangeLog frequently
provides additional important evidence.

So removing ChangeLog files will be a bad blow to our ability to
easily and conveniently research the past, something that is extremely
important in a project with such a rich history, where it's all too
easy to reintroduce a bug if you don't look hard enough at the history
of some code fragment.

People are saying it's an extra barrier to contributing.  That is
true, but so is understanding the Emacs internals, code conventions,
organization of the documentation system, its auxiliary files (like
CONTRIBUTE, NEWS, DEBUG, PROBLEMS, etc.), the release process, and a
few more things.  Having to write ChangeLog entries is an
insignificant addition to the body of knowledge a contributor needs to
master, there's no way around that.  Nor should there be: without
knowing this stuff, you cannot be a useful contributor anyway, as your
contributions will need too much attention from the veterans, who then
will be unable to make more significant contributions due to lack of
time.  We need here contributors who know enough to work on their own
with minimal guidance, who can be trusted to do a good job that
doesn't need to be reviewed too deeply, whose design can be trusted to
be in line with the Emacsy way of doing things.  How can one raise to
this position without learning a lot of project-specific stuff?  You
can't.

Writing ChangeLog entries is just one small part of that.  It's no
accident that people who don't want ChangeLog files more often than
not don't want to write detailed commit log messages, either, and many
times don't know how to write good documentation.  Do we want to
dispense with these as well?  If we drop the ChangeLog files, there's
no way we can explain why we ask for commit log messages in ChangeLog
format, so the next logical step is to drop that as well, and we will
then lose valuable information.  We already are firmly on that path.

Other prominent GNU projects that maintain ChangeLog files in the
repository include GCC, Binutils, GDB, glibc, and Texinfo.  XEmacs
also has it.  Why should Emacs be the first one to plunge into this
adventure?  Why not let others try that first, and then we could learn
from their mistakes?

Let's reinstate the ChangeLog files.  Maintaining them is a negligible
cost; many other projects do that and don't have any trouble.  Unlike
what some people say, merge conflicts in ChangeLog files are very
rare, once you install git-merge-changelog.  We have some important
infrastructure based on ChangeLog files that will become extinct
without them, something that people tend to forget.  We tried to live
without these files for a year; that experiment failed miserably.
It's time to admit that, and fix the mistake we made.

======================================================================

> > On a more spiritual side, I think they belong to the zen of contributing
> > to a GNU project.  :)
> 
> Dear Goddess, I hope you're joking.

Actually, I'm sure he isn't.  Maybe you are.

> Speaking as a very, *very* long-time GNU contributor (I'm pretty sure
> my earliest Emacs patches predated the formation of the FSF), I
> consider ChangeLogs a relic of a bygone era.

You are entitled to your opinions; others are entitled to theirs.
IMO, active involvement in Emacs development during the recent years
brings more weight to an opinion about the current subject than
contributions made 30 years ago.

> Changelogs made some sense as a way of grouping together what we now
> call a changeset back in the days of file-oriented version-control
> systems.  Nowadays, set against the actual changeset comments in
> version-control histories, Changelogs are a pointless and duplicative
> ritual.

They are not duplications.  They are work-tree expressions of the VCS
log, exactly like any source file is the work-tree expression of the
VCS repo objects which hold that file's data.  You won't lobby for
removing the source files, and use the likes of "git show :foo.c"
instead, would you?  It should go without saying that having a file is
more convenient than having to regenerate it every time from the
repository.

And please note that many people who want to drop Changelogs also want
to drop the policy of making detailed commit log messages.  We are
experiencing the adverse effects of that for the past year.  If we
continue on that disastrous path, things will keep deteriorating, and
we will lose important information that cannot be restored.

> I think we should have dispensed with this practice during the CVS-to-bzr
> transition. As it is, the sooner gone the better.

You are a year late to the party, I think: we already did what you
describe.  We no longer maintain Changelogs directly, we produce them
from Git logs.  The result is problematic even when there's only one
active branch of development; when there are 2 branches, as we have
now, the result is an unimaginable mess, with _more_ merge conflicts
and more work to fix the fallout, not less.  I cannot even imagine
what will happen if we will ever want to start a 3rd branch.

From this past year's experience, it is clear to me that having
ChangeLog files as we did before is a fraction of the price we had to
pay for removing them.  Removing them is also a serious impediment on
our way to more complex workflows, so it's actually a regression, not
progress.  It's high time we recognized the mistake and fixed it.

======================================================================

> I respect that Richard may use them in a constructive way, but he's no longer
> doing the majority of the work on Emacs

Neither are some of those who expressed their views here.  Unlike
Richard, those others cannot present a history of contributions and of
leading the development that is anywhere near what Richard has on his
record.

> I want to choose the path that works for the core developers, and
> will make Emacs more welcoming to new contributors.

It is IMO a grave mistake to remove parts of our development process
just because they need to be learned by new contributors.  The result
will be increased burden on the shoulders of those who review the
submissions, due to the need to coach them, catch their mistakes, ask
for re-submissions, etc.  It will eventually be a net loss, because
those contributors will not enjoy the need to produce several versions
of the patch before it is admitted, and the reviewers will not enjoy
the extra burden.

We should consider each part of the development process on its own
merit, first and foremost.  If a part is useful, and its removal will
make the quality of the code or the documentation lower, then it
should not be removed, and new contributors will have to learn it and
to live by it.

There's no useful way into Emacs development that doesn't require
negotiating a few barriers.  No free lunch here (or anywhere).  We
should recognize this fact instead of trying to bury our head in the
sand.

======================================================================

> > At this point, I give up, since it seems fairly clear that maintaining
>   > an accurate ChangeLog just isn't of interest. Even the bare minimum
>   > legally relevant mistakes (missing "tiny change") don't seem to be being
>   > corrected.
> 
> In concrete terms, what is the problem with these mistakes?

The mistakes are not being corrected.  Experience shows that
correcting them is enough of an annoyance to discourage people.

> Where is the master record of this information now?

In the Git commit messages.

> Is it being maintained correctly there?

When a mistake is made there, it cannot be corrected, because Git
commit log is immutable.  Corrections must be made manually to a
ChangeLog file produced from the Git log by a script.  That proved not
to work well, see above.  It also proved to be a non-trivial problem
when merging changes from the release branch to master -- for this
latter issue we still don't have any idea for how to solve it reliably
and without requiring a lot of manual labor.

>                Probably just deleting it from the repo would be more honest.
> 
> What exactly are you proposing as the new practice
> for handling ChangeLog files?

There are 3 possibilities:

  . Keep the current system, where ChangeLog is produced from Git log
    and mistakes made in Git log should be corrected manually after
    producing ChangeLog

  . Give up on having ChangeLog files, either produced from Git log or
    maintained in the repository -- meaning a tarball will not include
    any ChangeLog at all

  . Go back to previous practice where we maintained ChangeLog files
    in the repository, and Git log messages were just copies of the
    ChangeLog entries

======================================================================

> No, what we do currently is different: we auto-generate the ChangeLog
> and then commit it into the Git.  What I think we should do is to never
> commit it into the Git, and only auto-generate it into the tarball.
> Like what we do for GNU ELPA packages (where the log is extracted via
> "git log" when building the package and added to the package's content).
> 
> That makes "fixing" ChangeLog entries harder/impossible.  But it gets us
> rid of the merge-mess once and for all.

This would mean we don't care at all about what's in the ChangeLog.
You already said you didn't care, so it's not really surprising you'd
consider this as a viable option.  But for someone like me, who does
care, this is not really a solution at all.

======================================================================

> If we could switch to a system where every patch is reviewed before
> commit, that'd be great.  My own impression is that it will kill the
> development pace because too few people are willing to spend the
> corresponding efforts.

Agreed.  For my part, I can say that I simply don't have enough free
time to review more patches than I do (which is an abysmal little).

> That's why I've followed a practice of giving out write access very
> liberally, with "post-commit spot-check reviews" instead.  Indeed, it
> means that errors in commit messages can't be fixed (we can fix them in
> the ChangeLog files, admittedly, but since I don't use them it doesn't
> help me).

Post-commit reviews also take time.  Do you have an estimation of how
much time per day this takes?

> Maybe we could have a half-way system, where commits are pushed to
> a branch that is "not fast-forward-only", and this branch is then
> auto-merged to the real (fast-forward-only) master branch after a delay
> (one day, maybe?) to give time to fix mess ups before they're cast
> in stone.

A day is nowhere near enough.  IME, a bad commit pushed to master
takes up to a week to be discovered.

More generally, the problem with such a branch is that it won't be
much different from pushing to master, except in rare cases that it
breaks the build, and even that can only be avoided if we set up some
kind of CI system that continuously builds that branch on the main
supported platforms.

======================================================================

> On 03/07/2016 01:06 PM, Eli Zaretskii wrote:
> 
>         Maintaining ChangeLog files by hand with each commit makes it harder
>         to merge changes due to the inevitable collisions with ChangeLog
>         files.
> 
>     Incorrect!  And the current situation creates _more_ collisions, not
>     less!
>
> My own experience is otherwise. For the kinds of development I do, I rarely see ChangeLog screwups now, whereas I used to see them routinely.

With or without git-merge-changelog?

If you have git-merge-changelog installed, what kind of screwups did
you see with ChangeLogs?  (I didn't see even one, in all the years I
use Git.)

>  I far prefer the current approach. Of course our approach can be improved (in particular merging from emacs-25 to master doesn't work well now), but let's not throw out the baby with the bathwater.

The current approach completely breaks down when more than one branch
is active.  So there's no baby in that bathwater.

>     We don't have to be better than the other prominent GNU projects.
> 
> I'd be happy if we were merely as good as the other prominent GNU projects that generate ChangeLog entries automatically. As things stand, due to our attempt to cater to all sides of this disagreement, we have an approach that satisfies nobody.

Not sure what you mean here.  What alternatives that don't "cater to
all sides" would you suggest?  The only one I see is to stop producing
ChangeLog files for the releases.

>     ChangeLog mistakes can be easily fixed. 
> 
> That's true under both the old and the new regimes.

No, it isn't.  Definitely not with two branches.

>     Let other projects invent those schemes and test-drive them. Enough with these experiments! 
> 
> I'd rather just do what coreutils, grep, tar, etc. use.

Don't know what hides behind "etc", but the rest are much smaller
projects, which are all in maintenance mode, and have a very small
number of active committers (of which you personally are a significant
fraction ;-).  They also don't use branches, at least not as much as
we do.  So I don't see how the experience of these projects is more
relevant to us than that of GCC, GDB, Binutils, and glibc.

> I could fairly easily change the master branch to do that.

To do what?  Please describe the details of your proposal.  Also,
please tell what do you suggest doing on the release branches.

> Even simpler would be to do what Guile does: it dispenses with ChangeLogs entirely. With Guile if you want something like a ChangeLog you run "git log".

As I said, tossing ChangeLog files entirely would indeed solve the
problems, but it's a sure path to further deterioration in the quality
of log messages.  It is easy to keep the quality in a project that has
a small number of committers who are veteran GNU developers (Guile has
1 frequent committer on master, and 3 on stable).  That's not our
case.

> If neither of the above two approaches suffice, we can always fall back on my previous email's proposal. It's not that experiemental, as it says to use the new produre on master and the old procedure on emacs-25. The new procedure works well enough within a single master branch.

Any suggested approach should support not only the current emacs-25
branch, but also the future release branches, i.e. it should continue
working when we cut the emacs-26 branch in the future.  It should also
be reliable, and not require manual labor for fixing mistakes in the
log messages, beyond the fix itself.

>     these procedures work, and all those projects are alive and kicking, and actually make more frequent releases than we do. 
> 
> XEmacs is not really alive and kicking.

XEmacs is not that different from Grep or Sed.  Sed, for example, saw
just 30 commits during the last year.

> Projects like GCC and glibc have more resources than we do, and can afford to insist on more-expert contributions that involve harder-to-generate patch formats. We do not have that luxury.

It's the other way around, actually: the current situation requires
more labor from us to get it right, so our lack of resources should
lead us to the opposite conclusion.

>         I often ran into problems. Yes, git-merge-changelog should reduce the
>         number of merge conflicts, but it doesn't eliminate them
> 
>     Oh, yes, it does.
> 
> Not in my experience, or in Dmitry's. It's a fine program, but it sometimes makes mistakes and they can be a pain to fix. 

Fixing its mistakes involves moving an entry to a different place,
that's all.  Easy done, and even if not done, it's not a disaster, as
the information is there anyway.

======================================================================

>         Maintaining ChangeLog files by hand with each commit makes it harder
>         to merge changes due to the inevitable collisions with ChangeLog
>         files.
> 
>     Incorrect!  And the current situation creates _more_ collisions, not
>     less!
> 
> Only when merging between branches.

It doesn't work very well even on a single branch.  With two branches,
it completely breaks down.

We do want to work on multiple branches, don't we?  We discussed how
to make more frequent releases, and perhaps have 3 branches for even
more flexible development and release schedule.  All of that would be
impossible because of this tiny but annoying PITA.  Do we really want
to continue wasting energy on fixing something that wasn't such a big
problem to begin with?

> Other than that, only a select group of people needs to bother: those who make mistakes, and those who feel a general need to clean up.

See, that's the crux of the issue: do we or don't we care about the
need to clean up our ChangeLogs?  If we don't, then why bother
maintaining them?

You (and some others) say the format and the content in the log
messages are important, and I agree.  But if we do care about them,
how can we NOT clean them up?  Having them in their current state
means they cannot be trusted, which is worse than not having them at
all.

> As a relatively careful committer, I've only had to correct the entries a few times, and I've been enjoying the lack of collisions quite a bit.

Yes, a few of us don't need any corrections.  But enough of us do, and
that's where the problem lies.  It is that problem that we need to
fix.  Leaving it unfixed makes your accurate work unreliable as well.

>     But ChangeLog mistakes can be easily fixed.
> 
> In the current approach, as well.

Only on a single branch, and even there this is not done enough.  We
all relied on Glenn to do that behind the scenes.  Now Glenn gave up
in despair, so things will degrade from now on.

>     We don't know how, and we don't have anyone who is motivated enough to
>     do that.  And even if and when we do have some solution, it is likely
>     to be inconvenient and unreliable.
> 
> I think we should wait and see until the work really transitions back to master. The motivation must rise.

The release branch will remain active for quite some time: we have
just started a new major version.  And if we want more frequent
releases, we should strive to have an active release branch at all
times.  So I don't think we can wait; waiting will not solve anything,
it will just make the current problems bigger and harder to solve.

>     Let other projects invent those schemes and test-drive them.  Enough
>     with these experiments!  They draw the last drops of energy from us,
>     and they avert the few last veteran contributors we have left.
> 
> Has the current experiment really sucked too much energy from anyone, aside from the implementors?

Why do you think Glenn gave up?

>     That's not a bad thing in itself.  The point is, these procedures
>     work, and all those projects are alive and kicking, and actually make
>     more frequent releases than we do.
> 
> For all we know, they might be thriving despite this practice.

Nothing wrong with that.

>         I often ran into problems. Yes, git-merge-changelog should reduce the
>         number of merge conflicts, but it doesn't eliminate them
> 
>     Oh, yes, it does.
> 
> Not in my experience either. I've still had collisions, and even when git-merge-changelog resolved them, it often put my entry in the middle of the file, whereas I usually needed it to be at the top. Leading to extra manual labor.

That extra manual labor is very small, and it's still a rare case to
have that.  A small price to pay for a clean and reliable solution.

>         requiring git-merge-changelog means that many contributors would
>         have to worry about installing and configuring git-merge-changelog,
>         which would be more of a hassle for recruiting contributors.
> 
>     It's a 5-sec configuration, let's not make a mountain out of a
>     molehill.
> 
> It was longer for me. But either way, it's more hassle for a random contributor than the current system.

The current system is much more hassle for non-random contributors, so
much so that we risk losing them, something we cannot afford.

> If it can be fixed, Someone should.

Let that Someone step forward, then, and speak up.  We have been
waiting for that for the past year, so I'm not holding my breath now.

======================================================================

> >> Introduce code reviews. Don't give commit access to the "golden"
> >> branches to everyone, just to a few top contributors and reviewers.
> >
> > We don't have manpower for that.
> 
> I find perplexing that we have manpower for reviewing ChangeLogs but not
> for reviewing commit logs *instead*.

Log messages are much shorter than code diffs, and much simpler to
read and understand.  It takes a few seconds to read them; reading a
code patch takes much more, typically needs to consult the surrounding
and related code, etc.  So code review takes orders of magnitude more
time than reviewing the corresponding log entries.

> And is it so super-important to have typo-free commit logs?

Typos is the least of our problems.  But yes, it's important: if you
start by forgiving typos, you end with unhelpful and uninformative log
messages.

> We are not even using `git notes'.

No one came up with a detailed procedure for doing that.  If you can
propose something that works, please do.

======================================================================

> >> Another option for correcting errors on commit logs is `git notes', but
> >> I don't know about its implications.
> >
> > Nobody does.  We shouldn't consider any more experiments in this area
> > until someone else uses these methods and proves they are viable in
> > our case.
> 
> How could *someone else* prove the viability of something in *our* case?

By coming up with a procedure which we can study and decide whether
it fits us.

> You always can argue that his project is not Emacs...

I can, but why would I want to?  The opinions are not pre-determined,
they are based on study and in this case also on a year-long
experience of actually using a procedure.

> `git notes' is a documented feature available since a long time ago. It
> can turn the immutability of the commit log into a non-issue if you wish
> to produce a proofread ChangeLog to include in the tarball.

We need a more detailed proposal than this, TIA.  David says "git
notes" is not a solution; if notes don't support merging, then I tend
to think he is right.

======================================================================

> > The reasons exist, and has been described.  You may not agree with
> > them, perhaps due to different experience, but that doesn't mean they
> > aren't valid.
> 
> Yeah, but what exactly is your point here? I was pointing out that this
> practice is astonishing for new contributors. You didn't understand why,
> so I explained that it's astonishing because to a new contributor it
> seems pointless. Now you assert that there are good reasons for it, but
> that really doesn't change that, to new contributors, it appears
> pointless and is astonishing.

My point is that the initial surprise will subside with time, by way
of getting used to it.  It is even possible that you will come to
value this, as someone else who reported here.

> Sorry, I still don't follow your reasoning. I still consider myself a
> potential contributor (I didn't contribute much beyond that first little
> foray), and not requiring me to copy my commit message into the
> ChangeLog is not going to affect the quality of the commit measures in
> any way. The only "message" that I get from this is "Great, sending
> patches for Emacs doesn't require me to do a copy & paste anymore".

As long as you don't have write access to the repository, you are not
required to provide the log entries more than once, and so this issue
doesn't really affect you.  This issue is only relevant to those who
actually push changes, because they might need to copy/paste into the
ChangeLog file.

======================================================================

> > I think if we care at all about having ChangeLog in the releases, we
> > should simply reinstate the file and maintain it in the repository.
> 
> FWIW, that's not what I was hoping would come from this, and I think
> that would be a retrograde step.

Can you tell why?  It solves all the problems you mention in your
mail, at negligible costs.  So it seems to be a clear winner.

> For 1) and 2), experience shows that few people will bother to make
> corrections.

Is it an important drawback that few people bother to make
corrections?  If it is, then any solution that involves such
corrections is not what we want.

Also, is the situation with corrections worse or better than what it
was when we maintained ChangeLog files in the repository?

> PS For the record, to explain the actual merging issue as I see it:
> 
> Suppose emacs-25 and master are synced at revision aaa and ChangeLog.2
> is up-to-date. emacs-25 advances to rev xxx, and independently master to
> rev xxx'. emacs-25 gets ChangeLog.2 updated, and merged to master. Now
> the footer of master ChangeLog.2 reports that it is up-to-date to rev
> xxx. What does this mean for the changes on master between aaa and xxx'?
> Because xxx on master is "after" xxx', I suspect it means they end up
> missing from ChangeLog.2 forever, which is bad.

No, they don't end up missing.  They will in general be in the "wrong"
order, time-wise, though.  But what is the "right" order for this
situation, where changes are made in parallel on several branches?  Do
we want them interwoven, in strict order of their commit times?  Or do
we want all the entries of a merge from the branch be kept together
with the time stamp of the merge?  If we want to continue keeping a
single generated ChangeLog file on master and on the branch, we need
to decide what is the desired result.

> But maybe there's no such issue, or it is fixable with some git
> trivia.

The only "git trivia" that's possible is a custom merge driver (which
AFAIK doesn't exist).  Git itself is not aware of the special meaning
of the time stamps in ChangeLog entries.

We could also have some Lisp rearranging the entries in whatever order
we decide we want it, after git-merge-changelog puts them in the order
it thinks is right.

> I don't know. That's why I made this bug report. AFAICS, the adopted
> "solution" was simply to ignore the issue, which means
> master/ChangeLog.2 is (probably) messed up at the moment.

It is not messed, but it isn't in chronological order, either.  And it
looks like no one ran "make change-history" on master for several
moons, so problems might become worse when they do.

======================================================================

> Giving people different permissions on different branches seems to make
> sense. Overtime, people move toward master/emacs-25.

AFAIK, Git (or is it Savannah?) doesn't support such granularity.

> > Maybe we could have a half-way system, where commits are pushed to
> > a branch that is "not fast-forward-only", and this branch is then
> > auto-merged to the real (fast-forward-only) master branch after a delay
> > (one day, maybe?) to give time to fix mess ups before they're cast
> > in stone.
> 
> I think that this would require some considerable co-ordination. If I
> push a broken commit to devel branch, and then you fix this commit
> through rebase, my copy of the branch (and everyone elses) is now
> broken.

Stefan was talking about merging, not rebasing.

======================================================================

> > Find a better and more reliable way of dealing with the problems described
> > here, and I'll be the first to agree not to reintroduce ChangeLogs.
> 
> As far as I understand so far, we have only a few needs. If I've understood
> incorrectly, please let me know, as this thread has wandered in a few places.
> 
>  1. We need a way to ensure quality commit messages from contributors.
>  2. We'd like to be able to correct commit messages after the fact.
>  3. We need a convenient way to both create and access this information.
>  4. We want to publish the history of these commit messages in the tarball.
> 
> The ChangeLog file and its format are one implementation of these needs that
> has worked over the years. Are you saying that if we move to something else,
> and it fulfills these criteria, you'd be just as happy with it?

Yes.

However, putting on my system engineer hat, let me make sure the list
of requirements covers all the aspects:

 5. The "history of commit messages" produced in 4 above should
    resemble a ChangeLog file (ideally, it should use the same
    format).
 6. The solution should allow manual generation of the "history" file
    mentioned in 4, on demand, in reasonably short time, even if the
    repo clone does not include a built Emacs.  (It is okay to use the
    installed Emacs binary, if necessary.)
 7. The solution should support Git workflows we allow: merging
    between branches, rebasing local changes on upstream,
    cherry-picking from another branch, reverting a commit, tagging a
    commit, and amending an unpushed commit.  "Support" here means
    that the solution should not impose limitations on these
    workflows, and the steps for updating whatever databases the
    solution will use should not require additional manual work as
    result of using these workflows.

(I'm not saying these additions are something non-obvious, I just
want to make sure we have everything covered.)

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: johnw@gnu.org, mthl@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 8 Mar 2016 23:58:07 -0800
> 
> >> My own experience is otherwise. For the kinds of development I do, I rarely see ChangeLog screwups now, whereas I used to see them routinely.
> >
> > With or without git-merge-changelog?
> 
> Without.

Then I understand why your experience is so negative.

> > Please describe the details of your proposal.
> 
> For the more-traditional approach, apply the attached patch to emacs-25, and 
> merge it to master.  Other branches can pick it up as needed.

Didn't we consider this approach, and decided that having ChangeLog.2
was better?

Re: Should we restore manually maintained ChangeLogs

[Dmitry Gutov]

On 03/09/2016 03:19 PM, Stefan Monnier wrote:
>> can do what coreutils etc. do.  They autogenerate ChangeLog files for
>> releases, but do not put these ChangeLog files in their repositories.
>> They have a way to fix typos in the autogenerated ChangeLog files.
>
> So they go though the trouble of fixing typos, but only in the ChangeLog
> files in the release tarballs (where *very* few people will ever look)?

Making vc-git-expanded-log-entry look for and show a corresponding entry 
from the fixes file shouldn't be particularly hard.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> Date: Wed, 09 Mar 2016 17:53:06 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > From: John Wiegley <jwiegley@gmail.com>
> > Cc: Ingo Lohmar <i.lohmar@gmail.com>,  ofv@wanadoo.es,  emacs-devel@gnu.org
> > Date: Tue, 08 Mar 2016 22:41:49 -0800
> > 
> > Keeping ChangeLog style in the commit entry is not terribly
> > useful either, since the diff output of log -p lets you know which function or
> > variable is being modified.
> 
> "git log -p" cannot do the job for changes in many types of files.
> For example, try it on Lisp or Texinfo files.

Sorry, please disregard this mess.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Tue, 08 Mar 2016 22:41:49 -0800
> Cc: ofv@wanadoo.es, Ingo Lohmar <i.lohmar@gmail.com>, emacs-devel@gnu.org
> 
> > Find a better and more reliable way of dealing with the problems described
> > here, and I'll be the first to agree not to reintroduce ChangeLogs.
> 
> As far as I understand so far, we have only a few needs. If I've understood
> incorrectly, please let me know, as this thread has wandered in a few places.
> 
>  1. We need a way to ensure quality commit messages from contributors.
>  2. We'd like to be able to correct commit messages after the fact.
>  3. We need a convenient way to both create and access this information.
>  4. We want to publish the history of these commit messages in the tarball.
> 
> The ChangeLog file and its format are one implementation of these needs that
> has worked over the years. Are you saying that if we move to something else,
> and it fulfills these criteria, you'd be just as happy with it?

Yes.

However, putting on my system engineer hat, let me make sure the list
of requirements covers all the aspects:

 5. The "history of commit messages" produced in 4 above should
    resemble a ChangeLog file (ideally, it should use the same
    format).
 6. The solution should allow manual generation of the "history" file
    mentioned in 4, on demand, in reasonably short time, even if the
    repo clone does not include a built Emacs.  (It is okay to use the
    installed Emacs binary, if necessary.)
 7. The solution should support Git workflows we allow: merging
    between branches, rebasing local changes on upstream,
    cherry-picking from another branch, reverting a commit, tagging a
    commit, and amending an unpushed commit.  "Support" here means
    that the solution should not impose limitations on these
    workflows, and the steps for updating whatever databases the
    solution will use should not require additional manual work as
    result of using these workflows.

(I'm not saying these additions are something non-obvious, I just
want to make sure we have everything covered.)

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/09/2016 05:19 AM, Stefan Monnier wrote:
> So they go though the trouble of fixing typos, but only in the ChangeLog
> files in the release tarballs (where*very*  few people will ever look)?
One can easily generate such a ChangeLog before a release, with a 
coreutils-like approach. Just type "make gen-ChangeLog".

In a larger sense you're right, though. I typically don't look at 
ChangeLogs. and as I understand it you don't either. As time goes on I 
expect more developers will gravitate to the new system where if you 
want to look at the change logs you run 'git log' or type 'C-x v l' or 
whatever floats your boat. I wouldn't be surprised if most developers 
are there already.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 09:42:43 -0800
> 
> As time goes on I expect more developers will gravitate to the new
> system where if you want to look at the change logs you run 'git
> log' or type 'C-x v l' or whatever floats your boat.

Which will bring them the same text with mistakes and omissions that
we see now.  E.g., I see someone's contribution without the
paperwork-exempt tag -- how do I know if that person has an
assignment, and I'm just missing it somehow, or she doesn't and I;m
looking at someone's omission?  Same with other blunders.

Who needs a history record one cannot trust?  It's worse than having
no record at all.

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

On 03/09/2016 07:53 AM, Eli Zaretskii wrote:
> Writing a log entry in ChangeLog format is an excellent opportunity 
> for reflecting on the changeset

Yes, the ChangeLog format is useful. I use it myself for commits I make 
to GNU projects (as well as some non-GNU projects, e.g., 
https://github.com/eggert/tz). Perhaps the format could be improved, but 
that should be a different thread.

> So removing ChangeLog files will be a bad blow to our ability to 
> easily and conveniently research the past,

No, this doesn't follow. If we use ChangeLog formats in commit messages, 
we can still research the past easily and conveniently.

> If we drop the ChangeLog files, there's no way we can explain why we 
> ask for commit log messages in ChangeLog format, so the next logical 
> step is to drop that as well, and we will then lose valuable information.

It's not a logical step at all, and we already have an explanation of 
why we ask for ChangeLog format in CONTRIBUTE. Perhaps the explanation 
can be improved, but that's true no matter what approach we take 
(assuming we continue to prefer ChangeLog format).

> Other prominent GNU projects that maintain ChangeLog files in the 
> repository include GCC, Binutils, GDB, glibc, and Texinfo. XEmacs also 
> has it. Why should Emacs be the first one to plunge into this adventure?

Emacs is not the first at all. All the projects I've mentioned (Guile, 
coreutils, tar, etc.) used to maintain ChangeLog files in the 
repository. They've all moved away from that approach, in large part 
because of the hassle and confusion it entails.

>
> There are 3 possibilities:
>
>    . Keep the current system, where ChangeLog is produced from Git log
>      and mistakes made in Git log should be corrected manually after
>      producing ChangeLog
>
>    . Give up on having ChangeLog files, either produced from Git log or
>      maintained in the repository -- meaning a tarball will not include
>      any ChangeLog at all
>
>    . Go back to previous practice where we maintained ChangeLog files
>      in the repository, and Git log messages were just copies of the
>      ChangeLog entries

There is a 4th possibility: switch to what coreutils etc. do.

> This would mean we don't care at all about what's in the ChangeLog.

No, we can still make corrections with the 4th possibility. And even if 
we couldn't make corrections, it wouldn't mean we don't care about 
what's in the ChangeLog.

Re: Should we restore manually maintained ChangeLogs

[Alan Mackenzie]

Hello, Eli.

On Wed, Mar 09, 2016 at 08:01:24PM +0200, Eli Zaretskii wrote:
> > From: Paul Eggert <eggert@cs.ucla.edu>
> > Date: Wed, 9 Mar 2016 09:42:43 -0800

> > As time goes on I expect more developers will gravitate to the new
> > system where if you want to look at the change logs you run 'git
> > log' or type 'C-x v l' or whatever floats your boat.

> Which will bring them the same text with mistakes and omissions that
> we see now.  E.g., I see someone's contribution without the
> paperwork-exempt tag -- how do I know if that person has an
> assignment, and I'm just missing it somehow, or she doesn't and I;m
> looking at someone's omission?  Same with other blunders.

> Who needs a history record one cannot trust?  It's worse than having
> no record at all.

Maybe we should move to a version control system which permits commit
messages to be amended.  (Only half joking.)

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Should we restore manually maintained ChangeLogs

[Dmitry Gutov]

On 03/09/2016 08:01 PM, Eli Zaretskii wrote:

> Which will bring them the same text with mistakes and omissions that
> we see now.  E.g., I see someone's contribution without the
> paperwork-exempt tag -- how do I know if that person has an
> assignment, and I'm just missing it somehow, or she doesn't and I;m
> looking at someone's omission?  Same with other blunders.

Again, we can integrate the fixes file with VC, and show its contents 
together with the original commit message.

Although, to make that it consistent, we'll need to switch to the 
"short" format in vc-print-log, just like vc-print-root-log is already 
displayed.

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/09/2016 08:04 AM, Eli Zaretskii wrote:
>> Cc: johnw@gnu.org, mthl@gnu.org, emacs-devel@gnu.org
>> From: Paul Eggert <eggert@cs.ucla.edu>
>> Date: Tue, 8 Mar 2016 23:58:07 -0800
>>
>>>> My own experience is otherwise. For the kinds of development I do, I rarely see ChangeLog screwups now, whereas I used to see them routinely.
>>> With or without git-merge-changelog?
>> Without.
> Then I understand why your experience is so negative.

This appears to be implying that if I had installed git-merge-changelog 
and configured Git to use it, I would have had a better experience. I 
don't see how this would be so. As I said, I rarely do merges, so making 
merges work better wouldn't help me.

>>> Please describe the details of your proposal.
>> For the more-traditional approach, apply the attached patch to emacs-25, and
>> merge it to master.  Other branches can pick it up as needed.
> Didn't we consider this approach, and decided that having ChangeLog.2
> was better?

Yes and no. We came up with Emacs's current approach as a compromise. I 
would have preferred the approach taken by Guile (no ChangeLogs at all; 
just use 'git log' or whatever). Second-best would have been Coreutils 
(ChangeLog files in tarball, mainly to keep ChangeLog-file fans happy; 
they're not used by people doing active development). The approach 
currently taken by Emacs is a further compromise, which I think nobody 
likes.

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

Paul Eggert <eggert@cs.ucla.edu> writes:
>Yes, the ChangeLog format is useful. I use it myself for commits I
>make to GNU projects (as well as some non-GNU projects, e.g.,
>https://github.com/eggert/tz). Perhaps the format could be improved,
>but that should be a different thread.
>
>On 03/09/2016 07:53 AM, Eli Zaretskii wrote:
>> So removing ChangeLog files will be a bad blow to our ability to
>> easily and conveniently research the past,
>
>No, this doesn't follow. If we use ChangeLog formats in commit
>messages, we can still research the past easily and conveniently.
>
>> If we drop the ChangeLog files, there's no way we can explain why we
>> ask for commit log messages in ChangeLog format, so the next logical
>> step is to drop that as well, and we will then lose valuable
>> information.
>
>It's not a logical step at all, and we already have an explanation of
>why we ask for ChangeLog format in CONTRIBUTE. Perhaps the explanation
>can be improved, but that's true no matter what approach we take
>(assuming we continue to prefer ChangeLog format).

What Paul said.

The conflation of "ChangeLog-style entries" with "ChangeLog files" has been a persistent anti-pattern in this discussion.  It just causes confusion.  It would help if we remained clear about the distinction, since the former does not imply the latter.

This isn't merely a theoretical distinction.  I've been writing all my commit messages in the style of a ChangeLog entry for as long as I can remember (not just in this project), and 'git log' shows that other developers have been doing that in this project too.  Which is good, since that's exactly what the CONTRIBUTE guidelines currently recommend.

-K

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

On 03/09/2016 08:41 AM, Eli Zaretskii wrote:
>> From: John Wiegley <jwiegley@gmail.com>
>> ... we have only a few needs. If I've understood
>> incorrectly, please let me know, as this thread has wandered in a few places.
>>
>>   1. We need a way to ensure quality commit messages from contributors.

This boils down to applying social pressure, one way or another, 
regardless of the technical details about how ChangeLog entries are 
recorded.

>>   2. We'd like to be able to correct commit messages after the fact.
>>   3. We need a convenient way to both create and access this information.
>>   4. We want to publish the history of these commit messages in the tarball.
>>
>> The ChangeLog file and its format are one implementation of these needs that
>> has worked over the years. Are you saying that if we move to something else,
>> and it fulfills these criteria, you'd be just as happy with it?
> Yes.
>
> However, putting on my system engineer hat, let me make sure the list
> of requirements covers all the aspects:
>
>   5. The "history of commit messages" produced in 4 above should
>      resemble a ChangeLog file (ideally, it should use the same
>      format).
>   6. The solution should allow manual generation of the "history" file
>      mentioned in 4, on demand, in reasonably short time, even if the
>      repo clone does not include a built Emacs.  (It is okay to use the
>      installed Emacs binary, if necessary.)
>   7. The solution should support Git workflows we allow: merging
>      between branches, rebasing local changes on upstream,
>      cherry-picking from another branch, reverting a commit, tagging a
>      commit, and amending an unpushed commit.  "Support" here means
>      that the solution should not impose limitations on these
>      workflows, and the steps for updating whatever databases the
>      solution will use should not require additional manual work as
>      result of using these workflows.
>

The coreutils-like approach that I recently proposed should satisfy 
requirements (2) through (7).  See:

https://lists.gnu.org/archive/html/emacs-devel/2016-03/msg00391.html

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: johnw@gnu.org, mthl@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 10:16:29 -0800
> 
> On 03/09/2016 08:04 AM, Eli Zaretskii wrote:
> >> Cc: johnw@gnu.org, mthl@gnu.org, emacs-devel@gnu.org
> >> From: Paul Eggert <eggert@cs.ucla.edu>
> >> Date: Tue, 8 Mar 2016 23:58:07 -0800
> >>
> >>>> My own experience is otherwise. For the kinds of development I do, I rarely see ChangeLog screwups now, whereas I used to see them routinely.
> >>> With or without git-merge-changelog?
> >> Without.
> > Then I understand why your experience is so negative.
> 
> This appears to be implying that if I had installed git-merge-changelog 
> and configured Git to use it, I would have had a better experience. I 
> don't see how this would be so. As I said, I rarely do merges, so making 
> merges work better wouldn't help me.

If you don't do merges, you won't ever see merge conflicts.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  John Wiegley <johnw@gnu.org>,  emacs-devel@gnu.org
> Date: Wed, 09 Mar 2016 12:18:19 -0600
> 
> Paul Eggert <eggert@cs.ucla.edu> writes:
> >Yes, the ChangeLog format is useful. I use it myself for commits I
> >make to GNU projects (as well as some non-GNU projects, e.g.,
> >https://github.com/eggert/tz). Perhaps the format could be improved,
> >but that should be a different thread.
> >
> >On 03/09/2016 07:53 AM, Eli Zaretskii wrote:
> >> So removing ChangeLog files will be a bad blow to our ability to
> >> easily and conveniently research the past,
> >
> >No, this doesn't follow. If we use ChangeLog formats in commit
> >messages, we can still research the past easily and conveniently.
> >
> >> If we drop the ChangeLog files, there's no way we can explain why we
> >> ask for commit log messages in ChangeLog format, so the next logical
> >> step is to drop that as well, and we will then lose valuable
> >> information.
> >
> >It's not a logical step at all, and we already have an explanation of
> >why we ask for ChangeLog format in CONTRIBUTE. Perhaps the explanation
> >can be improved, but that's true no matter what approach we take
> >(assuming we continue to prefer ChangeLog format).
> 
> What Paul said.
> 
> The conflation of "ChangeLog-style entries" with "ChangeLog files" has been a persistent anti-pattern in this discussion.  It just causes confusion.  It would help if we remained clear about the distinction, since the former does not imply the latter.
> 
> This isn't merely a theoretical distinction.  I've been writing all my commit messages in the style of a ChangeLog entry for as long as I can remember (not just in this project), and 'git log' shows that other developers have been doing that in this project too.  Which is good, since that's exactly what the CONTRIBUTE guidelines currently recommend.

Paul omitted an important part of what I said, which of course made
the point I was trying to make incoherent.

And you are just repeating his omission.  And then you claim that
others cause confusion, whereas in fact you confuse yourself (and
perhaps others) by reading selectively what I and others say.  How
does this make any sense?

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/09/2016 10:27 AM, Eli Zaretskii wrote:
> If you don't do merges, you won't ever see merge conflicts.

I saw conflicts all the time when rebasing or applying patches. Perhaps 
these weren't "merge conflicts" in some technical sense, but they were 
just as much a hassle.

Re: Should we restore manually maintained ChangeLogs

[Dmitry Gutov]

On 03/09/2016 08:34 PM, Paul Eggert wrote:
> On 03/09/2016 10:27 AM, Eli Zaretskii wrote:
>> If you don't do merges, you won't ever see merge conflicts.
>
> I saw conflicts all the time when rebasing or applying patches. Perhaps
> these weren't "merge conflicts" in some technical sense, but they were
> just as much a hassle.

IIRC, git-merge-changelog is supposed to handle those cases, too.

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

>> So they go though the trouble of fixing typos, but only in the ChangeLog
>> files in the release tarballs (where *very* few people will ever look)?
> Making vc-git-expanded-log-entry look for and show a corresponding entry
> from the fixes file shouldn't be particularly hard.

But what about vc-print-log and vc-region-history?
Doable in Elisp, yes, but how hard, and at what performance cost?


        Stefan

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> Cc: ofv@wanadoo.es, i.lohmar@gmail.com, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 10:24:33 -0800
> 
> The coreutils-like approach that I recently proposed should satisfy 
> requirements (2) through (7).  See:
> 
> https://lists.gnu.org/archive/html/emacs-devel/2016-03/msg00391.html

It doesn't satisfy (3).

More generally, it won't work because it requires the amendments to be
in a form that is tedious to produce and error-prone.  I don't see how
we can have more motivation for using this than we have for the
current method.

This method is fine for a small group of veteran hackers who never
make mistakes writing Perl editing commands (but then those hackers
don't need this facility anyway) and when the number of amendments is
small.

So: thanks, but no, thanks.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: johnw@gnu.org, mthl@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 10:34:46 -0800
> 
> On 03/09/2016 10:27 AM, Eli Zaretskii wrote:
> > If you don't do merges, you won't ever see merge conflicts.
> 
> I saw conflicts all the time when rebasing or applying patches. Perhaps 
> these weren't "merge conflicts" in some technical sense, but they were 
> just as much a hassle.

git-merge-changelog supports rebase and cherry-picks as well.  So your
assumption that it won't help you is based on a mistake.

As for applying patches, just don't apply the ChangeLog hunks.

Re: Should we restore manually maintained ChangeLogs

[Dmitry Gutov]

On 03/09/2016 08:39 PM, Stefan Monnier wrote:

>> Making vc-git-expanded-log-entry look for and show a corresponding entry
>> from the fixes file shouldn't be particularly hard.
>
> But what about vc-print-log and vc-region-history?

vc-print-log - only if we make it use the same format as 
vc-print-root-log, which isn't not a terrible solution (it'll improve UI 
consistency).

vc-region-history - probably not, if we can't use the same approach for 
messages as above. Although we could hack something with jit-lock...

> Doable in Elisp, yes, but how hard, and at what performance cost?

With the above caveats, not too hard, and shouldn't be too slow either.

Although if we could co-opt git notes after all, it would be even easier 
to just ask Git to show them together with commit messages. But that's 
something Someone should investigate.

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Yuri Khan]

On Mon, Mar 7, 2016 at 10:24 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> "git log -p" cannot do the job for changes in many types of files.
> For example, try it on Lisp or Texinfo files.

I assume you are talking about the function header that is displayed
in the hunk headers, and by default contains the nearest preceding
line that starts in the zeroth column.

Git can in fact be configured to recognize “functions” differently,
per file type. The gitattributes(5) manual page describes this:

   Defining a custom hunk-header
       […]

       First, in .gitattributes, you would assign the diff attribute for
       paths.

           *.tex   diff=tex

       Then, you would define a "diff.tex.xfuncname" configuration to
       specify a regular expression that matches a line that you would
       want to appear as the hunk header "TEXT". Add a section to your
       $GIT_DIR/config file (or $HOME/.gitconfig file) like this:

           [diff "tex"]
               xfuncname = "^(\\\\(sub)*section\\{.*)$"

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/09/2016 10:39 AM, Dmitry Gutov wrote:
> IIRC, git-merge-changelog is supposed to handle those cases, too. 

I don't see how it can handle the case where I use GNU 'patch' instead 
of Git. I do that regularly, since GNU 'patch' sometimes works better 
for me than Git does.

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/09/2016 10:01 AM, Eli Zaretskii wrote:
> Who needs a history record one cannot trust?  It's worse than having
> no record at all.

Any historian will tell you that you cannot trust historical records. 
Caesar's commentaries, Churchill's speeches, the Open Group Rationale, 
Emacs ChangeLog entries -- they're all riddled with errors, and 
sometimes have outright fabrications, and anybody studying them must 
take this into account. That's just life.

(Though I do hope our ChangeLogs are more trustworthy than Caesar was....)

There is a reasonable question about how much of our development effort 
should be devoted to sprucing up ChangeLogs after they're committed. I 
think this should be low priority, whereas as I understand it you would 
prefer that we boost its priority. Neither side is advocating 
untrustworthy ChangeLogs, or perfect ChangeLogs for that matter; it's 
mainly a question of where to allocate our scarce development resources.

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

> Maybe we should move to a version control system which permits commit
> messages to be amended.  (Only half joking.)

These seem to be in rather short supply, sadly.  Any suggestion?


        Stefan

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Thu, 10 Mar 2016 00:52:53 +0600
> Cc: John Wiegley <johnw@gnu.org>, Emacs developers <emacs-devel@gnu.org>
> 
> On Mon, Mar 7, 2016 at 10:24 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > "git log -p" cannot do the job for changes in many types of files.
> > For example, try it on Lisp or Texinfo files.
> 
> I assume you are talking about the function header that is displayed
> in the hunk headers

John was.

> Git can in fact be configured to recognize “functions” differently,
> per file type.

Yes, I know.  But how many people have done that, and how many of them
will be able to cough up the right regex for whatever language they
need?

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

>   3. We need a convenient way to both create and access this information. 

On 03/09/2016 10:41 AM, Eli Zaretskii wrote:
>> From: Paul Eggert
>>
>> The coreutils-like approach that I recently proposed should satisfy
>> requirements (2) through (7).  See:
>>
>> https://lists.gnu.org/archive/html/emacs-devel/2016-03/msg00391.html
> It doesn't satisfy (3).

Sure it does. There are lots of ways to look at Git logs, ranging from 
plain "git log" through VC commands through running "make gen-ChangeLog" 
and looking at the resulting ChangeLog file.

One old-fashioned example: on my 6-year-old desktop it takes 2.3 seconds 
to generate the entire log of all 125,294 commits to the Emacs 
repository's main branch, using plain "git log". This generates a 37 MB 
file that Emacs can easily load and I can quickly search through 
interactively, or apply 'grep' to, or whatever. It's not a problem.

When Emacs was first developed, this sort of approach would have been 
prohibitively expensive. Nowadays it's no big deal.

As for convenient way to create the information, that's trivial: just 
put it into the commit message. This has long been common practice with 
Emacs.

> it requires the amendments to be
> in a form that is tedious to produce and error-prone.

It's not that bad, and it's good enough. True, it will discourage minor 
tidying-up of old entries (reindenting and the like), but that's a good 
thing. We should be spending our scarce development resources in more 
important areas.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 10:55:39 -0800
> 
> On 03/09/2016 10:01 AM, Eli Zaretskii wrote:
> > Who needs a history record one cannot trust?  It's worse than having
> > no record at all.
> 
> Any historian will tell you that you cannot trust historical records. 
> Caesar's commentaries, Churchill's speeches, the Open Group Rationale, 
> Emacs ChangeLog entries -- they're all riddled with errors, and 
> sometimes have outright fabrications, and anybody studying them must 
> take this into account. That's just life.

If I wanted to be a historian, I wouldn't be here.  This community is
about software development, not about historical research.  When I'm
looking up a commit, I want accurate information about it.  I don't
want to embark on a history research project to find out which words
are truthful and which are a lie.

IOW, this analogy is not helpful, and I suspect you know that.

> There is a reasonable question about how much of our development effort 
> should be devoted to sprucing up ChangeLogs after they're committed. I 
> think this should be low priority, whereas as I understand it you would 
> prefer that we boost its priority. Neither side is advocating 
> untrustworthy ChangeLogs, or perfect ChangeLogs for that matter; it's 
> mainly a question of where to allocate our scarce development resources.

I'm arguing that we shouldn't _need_ to allocate resources to it.
Reinstating ChangeLog files solves that with minimal costs, and its
only disadvantage seems to be that it sounds "regression" to some.

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Paul Eggert]

On 03/09/2016 11:10 AM, Eli Zaretskii wrote:
>> >Git can in fact be configured to recognize “functions” differently,
>> >per file type.
> Yes, I know.  But how many people have done that, and how many of them
> will be able to cough up the right regex for whatever language they
> need?

We don't need to do it for all languages, just for the languages Emacs 
is written in. And people shouldn't need to cough up the right regex; 
all they need to do is run "./autogen.sh all" or "./autogen.sh git". We 
should get people into the habit of running autogen.sh that way, when 
they clone from the repository.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> Cc: ofv@wanadoo.es, i.lohmar@gmail.com, johnw@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 11:10:50 -0800
> 
> >   3. We need a convenient way to both create and access this information. 
> 
> On 03/09/2016 10:41 AM, Eli Zaretskii wrote:
> >> From: Paul Eggert
> >>
> >> The coreutils-like approach that I recently proposed should satisfy
> >> requirements (2) through (7).  See:
> >>
> >> https://lists.gnu.org/archive/html/emacs-devel/2016-03/msg00391.html
> > It doesn't satisfy (3).
> 
> Sure it does. There are lots of ways to look at Git logs, ranging from 
> plain "git log" through VC commands through running "make gen-ChangeLog" 
> and looking at the resulting ChangeLog file.

The point is the "convenient" part.

> > it requires the amendments to be
> > in a form that is tedious to produce and error-prone.
> 
> It's not that bad, and it's good enough. True, it will discourage minor 
> tidying-up of old entries (reindenting and the like), but that's a good 
> thing. We should be spending our scarce development resources in more 
> important areas.

It simply won't work, certainly not better than what we have now.

Corrections should be made of plain text, otherwise no one except
yourself will do that (or maybe not even that, since you see this as a
waste).

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/09/2016 11:14 AM, Eli Zaretskii wrote:
> This community is about software development, not about historical 
> research. When I'm looking up a commit, I want accurate information 
> about it.

Like it or not, that is a form of historical research. One cannot escape 
the basic principles of history, and in particular one cannot insist 
that the historical record must be error-free.


>
>> There is a reasonable question about how much of our development effort
>> should be devoted to sprucing up ChangeLogs after they're committed. I
>> think this should be low priority, whereas as I understand it you would
>> prefer that we boost its priority. Neither side is advocating
>> untrustworthy ChangeLogs, or perfect ChangeLogs for that matter; it's
>> mainly a question of where to allocate our scarce development resources.
> I'm arguing that we shouldn't _need_ to allocate resources to it.
>

There is no free lunch here. There is a real cost to the old-fashioned 
approach of keeping commit messages as files in the repository. This 
cost is borne by every contributor, and the hassles of dealing with it 
was a primary motivation for Emacs (and other projects) moving away from 
that approach. Regardless of the approach taken, there is also a cost to 
sprucing up the historical record, a cost borne by the developers who do 
the sprucing-up. This sort of approach does work unless we devote real 
resources to it.

Re: ChangeLog to *VC-log*

[Stephen Berman]

On Wed, 9 Mar 2016 01:11:01 +0200 Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 03/09/2016 01:07 AM, Stephen Berman wrote:
>
>> Ah, thanks.  I'm pretty sure it used to be the case that `C-x 4 a' would
>> automatically find the right ChangeLog file, but I suppose that changed
>> when manually maintaining ChangeLogs was abandoned.  So I guess Emacs
>> developers should now set `change-log-default-name' to top level.
>
> Probably. If you have a solid reproduction scenario, please make a bug report.
>

On Tue, 08 Mar 2016 21:01:40 -0500 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

> AFAIK the C-x 4 a code hasn't been changed, so if there's a change in
> behavior it's most likely an accident/bug.  Please report it via M-x
> report-emacs-bug with a detailed recipe.

I found the cause of the change in behavior I observed: it's due to the
elimination of files named "ChangeLog" from the Emacs sources (there are
now only files named "ChangeLog.<n>").  In consequence, find-change-log
creates an empty file named "ChangeLog" in the immediate directory of
the file in which `C-x 4 a' was invoked, e.g. in lisp/calendar/, when
I'm editing todo-mode.el; previously, `C-x 4 a' found lisp/ChangeLog.  I
guess those who don't see this keep an non-VCS-tracked file "ChangeLog"
at top level.

If we don't return to maintaining versioned ChangeLog files, I think it
would be desirable for `C-x 4 a' to always create (or find) the file
"ChangeLog" at top level when editing Emacs sources, since this would
add the required directory levels to the commit message.  (Of course,
people could just always use `C-u C-x 4 a', but that's not as
convenient.)  Since other projects that could use `C-x 4 a' may have
different requirements on its effect, this should be conditioned by a
user option (or maybe just a plain variable), which can (and probably
should) be set in .dir-locals.el.  What do others think?

(Should I still file a bug (though it's more like a feature request), or
should we just continue the discussion here?)

Steve Berman

Re: Is it time to drop ChangeLogs?

[Ingo Lohmar]

Hi Alan,

On Tue, Mar 08 2016 21:36 (+0000), Alan Mackenzie wrote:

> If a git log message starts off life unreliable (i.e. there are mistakes
> in it) it can never be corrected.  It stays unreliable for the lifetime
> of the repository.

That's pretty much what I meant.  It may not contain the information you
(and I) would like to see in it, but it is highly reliable in that it is
the message that was used to describe the commit.  I would like to
apologize if this comes about as nitpicking.

> I am an experienced Emacs contributor, and I have, even recently, made
> mistakes in my commit messages.  I resent the fact that it is so
> difficult to correct them, even before pushing to savannah.  I don't
> think I need any teaching on the importance of good commit messages.

I make a lot of mistakes in commit messages, and I do not agree that it
is difficult to fix them before pushing.  You surely don't need any
teaching about commit messages, but Eli has repeatedly referred to those
who do, and I try to counter the argument that Changelog fixing teaches
them a *good* lesson.

> Not everybody has access to the git repository, and not everybody who
> has is capable of using it effectively.  ChangeLogs remain a useful,
> easy to use summary of Emacs's progress.

I cannot speak for anybody but myself.  When I am a user of Emacs,
trying to get an idea of what has changed on the level of detail that
the Changelog provides, I would rather go to the authoritative source,
if online (without any repository) at
http://git.savannah.gnu.org/cgit/emacs.git.

Re: ChangeLog to *VC-log*

[Lars Magne Ingebrigtsen]

Stephen Berman <stephen.berman@gmx.net> writes:

> (Should I still file a bug (though it's more like a feature request), or
> should we just continue the discussion here?)

Please file a bug report.  This behaviour is really annoying, and should
be fixed one way or another.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Should we restore manually maintained ChangeLogs

[Alan Mackenzie]

Hello, Stefan.

On Wed, Mar 09, 2016 at 02:09:53PM -0500, Stefan Monnier wrote:
> > Maybe we should move to a version control system which permits commit
> > messages to be amended.  (Only half joking.)

> These seem to be in rather short supply, sadly.  Any suggestion?

Mercurial.  It has the hg histedit command.

Or there's always CVS.  :-)

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

Eli Zaretskii <eliz@gnu.org> writes:
>Paul omitted an important part of what I said, which of course made
>the point I was trying to make incoherent.

I read the original message (it was https://lists.gnu.org/archive/html/emacs-devel/2016-03/msg00405.html, right?).

>And you are just repeating his omission.  And then you claim that
>others cause confusion, whereas in fact you confuse yourself (and
>perhaps others) by reading selectively what I and others say.  How
>does this make any sense?

I'm not sure what omission you're referring to.  You've referred to it twice now, but not specified what it is, and re-reading your original post I am unable to figure it out.

For example, your original post wrote:

> Writing ChangeLog entries is just one small part of that.  It's no
> accident that people who don't want ChangeLog files more often than
> not don't want to write detailed commit log messages, either, and many
> times don't know how to write good documentation.  Do we want to
> dispense with these as well?  If we drop the ChangeLog files, there's
> no way we can explain why we ask for commit log messages in ChangeLog
> format, so the next logical step is to drop that as well, and we will
> then lose valuable information.  We already are firmly on that path.

How does dropping ChangeLog files cause us to not be able to ask for ChangeLog-style entries?

(Also, I'm not sure your assertion about what kinds of behaviors correlate is true, but that's a separate issue.)

You wrote of the importance of "ChangeLog files" to forensics, but having that information in git commit messages is exactly equivalent (I use it for forensics all the time, in lots of projects).  Maybe you meant something like preserving the *existing* ChangeLog files, even if we don't maintain them in the future, to make it easier to do forensics involving changes that happened before CONTRIBUTE started asking for ChangeLog-style entries in commit messages?  If so, that wasn't at all clear from your post.  That's an ambiguity of the word "drop", perhaps: dropping ChangeLog files could mean removing them from the tree altogether, or it could mean leaving them preserved in amber but not adding material to them anymore.

When I read your post, I concluded that you were not proposing the "preserve in amber" approach, partly because of this:

> So removing ChangeLog files will be a bad blow to our ability to
> easily and conveniently research the past, something that is extremely
> important in a project with such a rich history, where it's all too
> easy to reintroduce a bug if you don't look hard enough at the history
> of some code fragment.
> 
> People are saying it's an extra barrier to contributing. ...

If the "removing ChangeLog files" in the first part had just been about removing the existing ones (in a hypothetical universe in which we didn't plan to add material to them in the future), then the first sentence of the next paragraph, "People are saying it's an extra barrier to contributing.", would have been a non sequitur, because a ChangeLog file that one isn't obligated to add to is not much of a barrier to contributing.  Anyway, a few sentences later in that paragraph you wrote "Having to write ChangeLog entries is an insignificant addition to the body of knowledge a contributor needs to master, there's no way around that." ... so now you're talking about writing ChangeLog-style *entries* (which could be done in commit messages), not really about ChangeLog files, as the immediately
  previous two paragraphs had been talking about, with no indication in between that some kind of conceptual transition had been made.

Do you see now why it at least looks like you're conflating these two different things?  Maybe you didn't mean to do so, but Paul's interpretation was reasonable; he was not quoting you in a misleading way.  When I read your original post, I already got confused by the conflation.  It wasn't Paul's followup that made me think that.  Paul and I came to the same conclusion for a reason.

Best regards,
-Karl

Re: Should we restore manually maintained ChangeLogs

[Andreas Schwab]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Stefan.
>
> On Wed, Mar 09, 2016 at 02:09:53PM -0500, Stefan Monnier wrote:
>> > Maybe we should move to a version control system which permits commit
>> > messages to be amended.  (Only half joking.)
>
>> These seem to be in rather short supply, sadly.  Any suggestion?
>
> Mercurial.  It has the hg histedit command.

Git.  It has the git rebase command.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Is it time to drop ChangeLogs?

[Ingo Lohmar]

Hi Eli,

On Wed, Mar 09 2016 05:46 (+0200), Eli Zaretskii wrote:

> That's because some people don't seem to read the thread, and keep
> coming up with the same incorrect arguments time and again.

Ok, so now we have established that we both think the other is not
reading the read "correctly" :)

>> "git log" messages cannot technically be both immutable and
>> unreliable: At least there is some severely imprecise use of
>> language going on.
>
> You need to read the thread to understand what is being alluded to.
> "Unreliable" in the sense that its text includes mistakes and cannot
> be trusted.

I realized that this may seem like nitpicking and apologized in my other
response.

> Emacs development doesn't work by requiring each commit be posted for
> review as prerequisite for committing, so what Oscar suggests is not
> possible.  (Please don't ask why, it was explained many times
> already.)

Do you mean in this thread or elsewhere?  I would honestly appreciate it
if you could point me to such a discussion, as I am not aware of the
arguments.  In fact, in this very thread, the previous Emacs maintainer
has contemplated an automatic queueing system, so it surprises me that
such a scheme should be totally out of the question.  (Note that I have
not advocated any specific means, like posting each patch for review.  I
realize there is a lot of work and little time, and I greatly appreciate
your constant work on Emacs.)

>
>> The whole argument for Changelogs comes down to a) being an established
>> band-aid to clean up spilt milk, or b) providing a fixed-form summary of
>> things that can be obtained using the VCS (provided the humans or tools
>> wirting the Changelog are as "reliable" as the VCS).
>
> Find a better and more reliable way of dealing with the problems
> described here, and I'll be the first to agree not to reintroduce
> ChangeLogs.

Several solutions to both a) and b) were proposed in this thread.

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 11:18:04 -0800
> Cc: johnw@gnu.org, emacs-devel@gnu.org
> 
> On 03/09/2016 11:10 AM, Eli Zaretskii wrote:
> >> >Git can in fact be configured to recognize “functions” differently,
> >> >per file type.
> > Yes, I know.  But how many people have done that, and how many of them
> > will be able to cough up the right regex for whatever language they
> > need?
> 
> We don't need to do it for all languages, just for the languages Emacs 
> is written in.

Which are quite a few.

> And people shouldn't need to cough up the right regex; all they need
> to do is run "./autogen.sh all" or "./autogen.sh git".

That still misses a couple that add-change-log-entry doesn't.

> We should get people into the habit of running autogen.sh that way,
> when they clone from the repository.

Some of them didn't like the extra baggage that brings, though.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 11:26:19 -0800
> 
> On 03/09/2016 11:14 AM, Eli Zaretskii wrote:
> > This community is about software development, not about historical 
> > research. When I'm looking up a commit, I want accurate information 
> > about it.
> 
> Like it or not, that is a form of historical research.

No, it's a very far cry from historical research.

> >> There is a reasonable question about how much of our development effort
> >> should be devoted to sprucing up ChangeLogs after they're committed. I
> >> think this should be low priority, whereas as I understand it you would
> >> prefer that we boost its priority. Neither side is advocating
> >> untrustworthy ChangeLogs, or perfect ChangeLogs for that matter; it's
> >> mainly a question of where to allocate our scarce development resources.
> > I'm arguing that we shouldn't _need_ to allocate resources to it.
> 
> There is no free lunch here.

No free lunch, but some lunches are cheaper than others.

> There is a real cost to the old-fashioned approach of keeping commit
> messages as files in the repository. This cost is borne by every
> contributor, and the hassles of dealing with it was a primary
> motivation for Emacs (and other projects) moving away from that
> approach.

That cost is much lower than any of the alternatives proposed so far,
including the current arrangement with ChangeLog.2.  It worked for
years.

If someone has a better proposal, let's hear it.

> Regardless of the approach taken, there is also a cost to 
> sprucing up the historical record

Since this is regardless of the approach, it shouldn't affect the
decision in this matter.

Re: ChangeLog to *VC-log*

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Date: Wed, 09 Mar 2016 20:30:32 +0100
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> I found the cause of the change in behavior I observed: it's due to the
> elimination of files named "ChangeLog" from the Emacs sources (there are
> now only files named "ChangeLog.<n>").  In consequence, find-change-log
> creates an empty file named "ChangeLog" in the immediate directory of
> the file in which `C-x 4 a' was invoked, e.g. in lisp/calendar/, when
> I'm editing todo-mode.el; previously, `C-x 4 a' found lisp/ChangeLog.  I
> guess those who don't see this keep an non-VCS-tracked file "ChangeLog"
> at top level.
> 
> If we don't return to maintaining versioned ChangeLog files, I think it
> would be desirable for `C-x 4 a' to always create (or find) the file
> "ChangeLog" at top level when editing Emacs sources, since this would
> add the required directory levels to the commit message.

No, _you_ should create a file at top level, and then everything will
continue working.  That's why the top-level file is called
ChangeLog.2: to keep it out of your way.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ingo Lohmar <i.lohmar@gmail.com>
> Date: Wed, 09 Mar 2016 20:32:28 +0100
> 
> > If a git log message starts off life unreliable (i.e. there are mistakes
> > in it) it can never be corrected.  It stays unreliable for the lifetime
> > of the repository.
> 
> That's pretty much what I meant.  It may not contain the information you
> (and I) would like to see in it, but it is highly reliable in that it is
> the message that was used to describe the commit.

Yeah, it's a reliable lie.  How helpful is that?

> > I am an experienced Emacs contributor, and I have, even recently, made
> > mistakes in my commit messages.  I resent the fact that it is so
> > difficult to correct them, even before pushing to savannah.  I don't
> > think I need any teaching on the importance of good commit messages.
> 
> I make a lot of mistakes in commit messages, and I do not agree that it
> is difficult to fix them before pushing.

Indeed, it isn't difficult.  But since mistakes do get pushed, I guess
people are not careful enough to review their commits before pushing,
or else we wouldn't be having this conversation, and there would be no
need for the procedures to correct pushed commits with mistaken log
messages.

> You surely don't need any teaching about commit messages, but Eli
> has repeatedly referred to those who do, and I try to counter the
> argument that Changelog fixing teaches them a *good* lesson.

Telling the wrongdoer to clean up their mess surely does teach them a
lesson.  It could well be the lesson we both agree is the best one:
review your commits before pushing.

I hope you agree that NOT fixing the mistakes teaches them an entirely
different lesson, the one we think is undesirable: that mistakes don't
matter, and eventually that the log messages don't matter, as long as
they say _something_.

> > Not everybody has access to the git repository, and not everybody who
> > has is capable of using it effectively.  ChangeLogs remain a useful,
> > easy to use summary of Emacs's progress.
> 
> I cannot speak for anybody but myself.  When I am a user of Emacs,
> trying to get an idea of what has changed on the level of detail that
> the Changelog provides, I would rather go to the authoritative source,
> if online (without any repository) at
> http://git.savannah.gnu.org/cgit/emacs.git.

I guess you never had to work in segregated networks, where access to
the outside world is not available.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Cc: eggert@cs.ucla.edu,  johnw@gnu.org,  emacs-devel@gnu.org
> Date: Wed, 09 Mar 2016 13:36:07 -0600
> 
> > Writing ChangeLog entries is just one small part of that.  It's no
> > accident that people who don't want ChangeLog files more often than
> > not don't want to write detailed commit log messages, either, and many
> > times don't know how to write good documentation.  Do we want to
> > dispense with these as well?  If we drop the ChangeLog files, there's
> > no way we can explain why we ask for commit log messages in ChangeLog
> > format, so the next logical step is to drop that as well, and we will
> > then lose valuable information.  We already are firmly on that path.
> 
> How does dropping ChangeLog files cause us to not be able to ask for ChangeLog-style entries?

I don't even need to explain that: there are already people in this
discussion who said they'd like to use a less formal format.  (You
already asked who those people were, and Dmitry already pointed out
who they were.)  To me, the connection is very clear.  If you don't
see it, just trust the facts -- it is already happening.

> You wrote of the importance of "ChangeLog files" to forensics, but having that information in git commit messages is exactly equivalent (I use it for forensics all the time, in lots of projects).

Once again, you take a part of the argument and try to analyze it
separately.

The point was that if we start on the slippery slope of saying the
mistakes in the log messages are unimportant, we will eventually lose
both the information in the Git log records and the ability to produce
ChangeLog files.

IOW, it's fortress -- if you dismantle one of its bastions, the others
will soon fall as well.  The only way to avoid that is to hold all of
them.

> Do you see now why it at least looks like you're conflating these two different things?

No.  There are different aspects to this issue, and I described them
one after the other.  Sometimes they are only loosely related,
sometimes they are more tightly coupled.  This is the best way I could
express my thoughts.  Since English is not the first language for
either of us, let's just agree that I failed to write in a style that
would allow you to better understand.

Once again, if there's a way to have the information in Git such that
a correct log could be generated from it, it would be fine with me,
and keeping the ChangeLog files in the repository won't be necessary,
as far as I'm concerned.  But so far, keeping the files looks like the
cheapest way of satisfying all the requirements.  I understand the
psychological effect of "going the old ways", but this is a practical
matter for me, not a religious one, and I'm looking for a practical
solution that would allow us to continue keeping an accurate accord of
the development history with as little overhead as possible.  If the
best alternative is those "old ways", I have no problem with that.

Re: Is it time to drop ChangeLogs?

[Ingo Lohmar]

On Wed, Mar 09 2016 22:06 (+0200), Eli Zaretskii wrote:
>> That's pretty much what I meant.  It may not contain the information you
>> (and I) would like to see in it, but it is highly reliable in that it is
>> the message that was used to describe the commit.
>
> Yeah, it's a reliable lie.  How helpful is that?

This discussion is about the content, not the technical merits of
immutability.  As I said, I am happy to drop the nitpicking.

>> I make a lot of mistakes in commit messages, and I do not agree that it
>> is difficult to fix them before pushing.
>
> Indeed, it isn't difficult.  But since mistakes do get pushed, I guess
> people are not careful enough to review their commits before pushing,
> or else we wouldn't be having this conversation, and there would be no
> need for the procedures to correct pushed commits with mistaken log
> messages.

It happens to the most careful people, correct.

> Telling the wrongdoer to clean up their mess surely does teach them a
> lesson.  It could well be the lesson we both agree is the best one:
> review your commits before pushing.
>
> I hope you agree that NOT fixing the mistakes teaches them an entirely
> different lesson, the one we think is undesirable: that mistakes don't
> matter, and eventually that the log messages don't matter, as long as
> they say _something_.

We completely agree on the best lesson and the importance of fixing
mistakes.  We probably disagree on what is a good way to actually teach
them the lesson.  We also seem to disagree on good ways of "fixing", or
ways to avoid the need for after-the-fact fixing.

>> I cannot speak for anybody but myself.  When I am a user of Emacs,
>> trying to get an idea of what has changed on the level of detail that
>> the Changelog provides, I would rather go to the authoritative source,
>> if online (without any repository) at
>> http://git.savannah.gnu.org/cgit/emacs.git.
>
> I guess you never had to work in segregated networks, where access to
> the outside world is not available.

In any case, I was speaking mostly hypothetically.  I am either
interested in a high-level description of some changes that might affect
me, or cool new things to try, in which case I look at NEWS.  Or I am
interested in details of what a function does differently now, or a
changed call signature or whatever --- in that case, I have never found
the Changelog to be sufficiently detailed.  Instead I will look (on the
next possible occasion) at the (commits that changed the) source code.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ingo Lohmar <i.lohmar@gmail.com>
> Cc: ofv@wanadoo.es, emacs-devel@gnu.org
> Date: Wed, 09 Mar 2016 20:51:15 +0100
> 
> > Emacs development doesn't work by requiring each commit be posted for
> > review as prerequisite for committing, so what Oscar suggests is not
> > possible.  (Please don't ask why, it was explained many times
> > already.)
> 
> Do you mean in this thread or elsewhere?

Both.  (This is not the first time these issues are discussed.)

> I would honestly appreciate it if you could point me to such a
> discussion, as I am not aware of the arguments.

In one word: manpower.  In 3 words: lack of manpower.

> In fact, in this very thread, the previous Emacs maintainer
> has contemplated an automatic queueing system, so it surprises me that
> such a scheme should be totally out of the question.

It's not out of the question.  But we are just contemplating it.
Before we could rely on it enough to make significant decisions like
the one discussed here, we'd need stop contemplating, install the
procedures to implement such a queuing system, run it for a while,
perhaps augment it some, until it's reliable enough and brings
consistently good results.  _Then_, and no earlier, we can base
changes in development process on such a system.  We are not there
yet.  Until we are, we need to find a workable solution with what we
have now.

> > Find a better and more reliable way of dealing with the problems
> > described here, and I'll be the first to agree not to reintroduce
> > ChangeLogs.
> 
> Several solutions to both a) and b) were proposed in this thread.

Unfortunately, they don't fit the bill.  See the rest of the thread
for why.

Re: Is it time to drop ChangeLogs?

[Karl Fogel]

Eli Zaretskii <eliz@gnu.org> writes:
>I don't even need to explain that: there are already people in this
>discussion who said they'd like to use a less formal format.  (You
>already asked who those people were, and Dmitry already pointed out
>who they were.)  To me, the connection is very clear.  If you don't
>see it, just trust the facts -- it is already happening.

I've worked in projects where we successfully enforce ChangeLog-style commit message conventions without having actual ChangeLog files.  If the Emacs project is having a social problem enforcing conventions, that's certainly something to be solved.  But blaming it on the absence of ChangeLog files does not hold up empirically.

I do not see any connection between the fact that some people advocate using a less formal format, and the question of whether or not we should have ChangeLog *files*.  One is about commit message format (other projects enforce a format successfully, and some even enforce this particular format), the other is about whether certain files exist in the tree.   It's surprising to me that you think this connection is so obvious that you don't even need to explain it :-), whereas to me there is no strong connection at all.

>> You wrote of the importance of "ChangeLog files" to forensics, but
>> having that information in git commit messages is exactly equivalent
>> (I use it for forensics all the time, in lots of projects).
>
>Once again, you take a part of the argument and try to analyze it
>separately.
>
>The point was that if we start on the slippery slope of saying the
>mistakes in the log messages are unimportant, we will eventually lose
>both the information in the Git log records and the ability to produce
>ChangeLog files.
>
>IOW, it's fortress -- if you dismantle one of its bastions, the others
>will soon fall as well.  The only way to avoid that is to hold all of
>them.

Okay, I understand your argument (I don't think your previous messages made it clear, but you've clarified it now).  However, it just does not hold true in other projects.  Why would we expect Emacs developers to be more slovenly than developers elsewhere?

>> Do you see now why it at least looks like you're conflating these two different things?
>
>No.  There are different aspects to this issue, and I described them
>one after the other.  Sometimes they are only loosely related,
>sometimes they are more tightly coupled.  This is the best way I could
>express my thoughts.  Since English is not the first language for
>either of us, let's just agree that I failed to write in a style that
>would allow you to better understand.

English is my native language, by the way; I'm American, not German.  FWIW, I couldn't tell from your writing that English is not your native language.

>Once again, if there's a way to have the information in Git such that
>a correct log could be generated from it, it would be fine with me,
>and keeping the ChangeLog files in the repository won't be necessary,
>as far as I'm concerned.  But so far, keeping the files looks like the
>cheapest way of satisfying all the requirements.  I understand the
>psychological effect of "going the old ways", but this is a practical
>matter for me, not a religious one, and I'm looking for a practical
>solution that would allow us to continue keeping an accurate accord of
>the development history with as little overhead as possible.  If the
>best alternative is those "old ways", I have no problem with that.

I think this project can successfully enforce ChangeLog-style entries in commit messages if we really want to, without much difficulty.  There are a number of techniques for doing so, mostly social techniques, with a few technical helpers if we want.  Other projects with similarly cohesive developer communities enforce such conventions all the time.  I don't think it has anything to do with whether or not there are actual ChangeLog files in the repository; the fact that some of the people who don't want to keep those files are *also* people who don't want to use that style of entry generally, even in commit messages, is merely correlative -- getting rid of the files doesn't mean that everything else they advocate must happen too.

I believe this is also what Paul was saying, though I don't want to put words in his mouth.

Best regards,
-Karl

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ingo Lohmar <i.lohmar@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Wed, 09 Mar 2016 21:22:11 +0100
> 
> In any case, I was speaking mostly hypothetically.  I am either
> interested in a high-level description of some changes that might affect
> me, or cool new things to try, in which case I look at NEWS.  Or I am
> interested in details of what a function does differently now, or a
> changed call signature or whatever --- in that case, I have never found
> the Changelog to be sufficiently detailed.  Instead I will look (on the
> next possible occasion) at the (commits that changed the) source code.

My use cases are very different, because more often than not they have
to do with fixing some more-or-less subtle bug in code written long
ago and modified many times since.  I use "git log -L" a lot (it even
ran out of memory  few times on me), but that often is not enough,
because you can see the code changes, but not many explanations why
they were made.  And since people who made those changes are generally
very bright, concluding that they just made stupid mistakes somehow
doesn't seem appropriate.

So I need every piece of evidence I can dig up, in order to figure out
the reasons for the change.  I frequently search mailing list
discussions around the dates of the changes, including mailing lists
that are now long gone, and bug reports.  An accurate log entry is
sometimes a jewel, in that it suddenly makes all the pieces to fall in
place.  A bad log entry, by contrast, is a stub in the back.

Re: Is it time to drop ChangeLogs?

[Ingo Lohmar]

On Wed, Mar 09 2016 22:30 (+0200), Eli Zaretskii wrote:

>> > Emacs development doesn't work by requiring each commit be posted for
>> > review as prerequisite for committing, so what Oscar suggests is not
>> > possible.  (Please don't ask why, it was explained many times
>> > already.)
>> 
>> Do you mean in this thread or elsewhere?
>
> Both.  (This is not the first time these issues are discussed.)
>
>> I would honestly appreciate it if you could point me to such a
>> discussion, as I am not aware of the arguments.
>
> In one word: manpower.  In 3 words: lack of manpower.

You have omitted the part where I said that I do not advocate a
mandatory review.  Several other schemes (some of them highly, perhaps
too highly, refined) have been proposed.

>
>> In fact, in this very thread, the previous Emacs maintainer
>> has contemplated an automatic queueing system, so it surprises me that
>> such a scheme should be totally out of the question.
>
> It's not out of the question.  But we are just contemplating it.
> Before we could rely on it enough to make significant decisions like
> the one discussed here, we'd need stop contemplating, install the
> procedures to implement such a queuing system, run it for a while,
> perhaps augment it some, until it's reliable enough and brings
> consistently good results.  _Then_, and no earlier, we can base
> changes in development process on such a system.  We are not there
> yet.  Until we are, we need to find a workable solution with what we
> have now.

No disagreement here.  I am nowhere near an anti-Changelog zealot, BTW.
It's just that I think there were/are a lot of smoke-and-mirrors
arguments flying around.

In fact, I think that your above paragraph is a much better argument to
keep Changelogs around for the time being than any arguments that have
been made in this thread alluding to their importance in the development
process.

>
>> > Find a better and more reliable way of dealing with the problems
>> > described here, and I'll be the first to agree not to reintroduce
>> > ChangeLogs.
>> 
>> Several solutions to both a) and b) were proposed in this thread.
>
> Unfortunately, they don't fit the bill.  See the rest of the thread
> for why.

This blanket statement is exactly as helpful as saying that they would
work perfectly fine.  We obviously disagree on this.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Cc: eggert@cs.ucla.edu,  johnw@gnu.org,  emacs-devel@gnu.org
> Date: Wed, 09 Mar 2016 14:42:30 -0600
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >I don't even need to explain that: there are already people in this
> >discussion who said they'd like to use a less formal format.  (You
> >already asked who those people were, and Dmitry already pointed out
> >who they were.)  To me, the connection is very clear.  If you don't
> >see it, just trust the facts -- it is already happening.
> 
> I've worked in projects where we successfully enforce ChangeLog-style commit message conventions without having actual ChangeLog files.  If the Emacs project is having a social problem enforcing conventions, that's certainly something to be solved.  But blaming it on the absence of ChangeLog files does not hold up empirically.

I'm not blaming anything.  I'm just saying that having ChangeLog files
seems so far to be the cheapest solution to a conundrum.

I don't know what project you have in mind, but IME the only way to
_force_ something in a Free Software project is to have a very
charismatic leader to whose opinions all the rest heed.  That's a rare
case to see, and it certainly isn't our situation here.

> >IOW, it's fortress -- if you dismantle one of its bastions, the others
> >will soon fall as well.  The only way to avoid that is to hold all of
> >them.
> 
> Okay, I understand your argument (I don't think your previous messages made it clear, but you've clarified it now).  However, it just does not hold true in other projects.  Why would we expect Emacs developers to be more slovenly than developers elsewhere?

Because they in fact are?  What can I say? just look around.

> English is my native language, by the way; I'm American, not German.

Apologies.  Then the problem is mine alone.

> FWIW, I couldn't tell from your writing that English is not your native language.

Yes, my disguise is good.  But it's just a disguise, believe me.

> I think this project can successfully enforce ChangeLog-style entries in commit messages if we really want to, without much difficulty.

I don't know why you think so, since we are already failing to do
that.  I agree that it should be possible in theory, but we don't seem
to live up to that.

> I believe this is also what Paul was saying, though I don't want to put words in his mouth.

Paul actually said that it was not a disaster if the log messages were
less than perfect.  (His own are, of course, very good, I wish
everyone would write similar log messages.)

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ingo Lohmar <i.lohmar@gmail.com>
> Cc: ofv@wanadoo.es, emacs-devel@gnu.org
> Date: Wed, 09 Mar 2016 21:50:34 +0100
> 
> In fact, I think that your above paragraph is a much better argument to
> keep Changelogs around for the time being than any arguments that have
> been made in this thread alluding to their importance in the development
> process.

Each one of us has their own knobs that need to be pushed to drive
point home.  It's only natural.

> >> > Find a better and more reliable way of dealing with the problems
> >> > described here, and I'll be the first to agree not to reintroduce
> >> > ChangeLogs.
> >> 
> >> Several solutions to both a) and b) were proposed in this thread.
> >
> > Unfortunately, they don't fit the bill.  See the rest of the thread
> > for why.
> 
> This blanket statement is exactly as helpful as saying that they would
> work perfectly fine.

I explained at length elsewhere, so the above is just a summary, not a
blanket rejection.

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

>> > Maybe we should move to a version control system which permits commit
>> > messages to be amended.  (Only half joking.)
>> These seem to be in rather short supply, sadly.  Any suggestion?
> Mercurial.  It has the hg histedit command.

AFAIK this is the same as Git's rebase, i.e. it leads to a different
commit id, so it can't really be used on branches like "master" and
"emacs-25" where we do merges and things like that.


        Stefan

Re: ChangeLog to *VC-log*

[Stephen Berman]

On Wed, 09 Mar 2016 22:00:05 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Date: Wed, 09 Mar 2016 20:30:32 +0100
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
>> 
>> I found the cause of the change in behavior I observed: it's due to the
>> elimination of files named "ChangeLog" from the Emacs sources (there are
>> now only files named "ChangeLog.<n>").  In consequence, find-change-log
>> creates an empty file named "ChangeLog" in the immediate directory of
>> the file in which `C-x 4 a' was invoked, e.g. in lisp/calendar/, when
>> I'm editing todo-mode.el; previously, `C-x 4 a' found lisp/ChangeLog.  I
>> guess those who don't see this keep an non-VCS-tracked file "ChangeLog"
>> at top level.
>> 
>> If we don't return to maintaining versioned ChangeLog files, I think it
>> would be desirable for `C-x 4 a' to always create (or find) the file
>> "ChangeLog" at top level when editing Emacs sources, since this would
>> add the required directory levels to the commit message.
>
> No, _you_ should create a file at top level, and then everything will
> continue working.  That's why the top-level file is called
> ChangeLog.2: to keep it out of your way.

But if Emacs can do that for me (and in so doing automatically comply
with its commit message format specification), don't you agree that's
more user-friendly?  Do you see a down side to having that automatism?
(Feel free to reply to bug#22968, if you prefer.)

Steve Berman

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 09 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> How does dropping ChangeLog files cause us to not be able to ask for
>> ChangeLog-style entries?
>
> I don't even need to explain that: there are already people in this
> discussion who said they'd like to use a less formal format.

I don't see any causal connection between these two statements. Some
people have preferences that differ from current project standards. That
will always be the case, no matter if ChangeLog files are dropped or
not.

> IOW, it's fortress -- if you dismantle one of its bastions, the others
> will soon fall as well.  The only way to avoid that is to hold all of
> them.

I think other people simply don't see it that way.

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Should we restore manually maintained ChangeLogs

[Nikolaus Rath]

On Mar 09 2016, Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:
>> Maybe we should move to a version control system which permits commit
>> messages to be amended.  (Only half joking.)
>
> These seem to be in rather short supply, sadly.  Any suggestion?

Mercurial with the "evolve" extension implements non-destructive history
editing. In short, commits can have metadata that marks them as
"superseding" a previous commit. Both superseded and new commit remain
part of the repository, but most commands be default work on the
superseding commit.

Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Paul Eggert]

On 03/09/2016 11:53 AM, Eli Zaretskii wrote:

>
>> And people shouldn't need to cough up the right regex; all they need
>> to do is run "./autogen.sh all" or "./autogen.sh git".
> That still misses a couple that add-change-log-entry doesn't.

Which two languages? It should be easy to fix and I can volunteer to do 
that.

>
>> We should get people into the habit of running autogen.sh that way,
>> when they clone from the repository.
> Some of them didn't like the extra baggage that brings, though.

Yes, I remember. :-)  Experts who want detailed control over every Git 
setting can of course configure this stuff any way they like. I just 
want the default to be reasonable, that's all.

Re: Should we restore manually maintained ChangeLogs

[Noam Postavsky]

On Wed, Mar 9, 2016 at 2:09 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> Maybe we should move to a version control system which permits commit
>> messages to be amended.  (Only half joking.)
>
> These seem to be in rather short supply, sadly.  Any suggestion?
>
>
>         Stefan
>

fossil allows commit message editing. It works by adding a new
commit-editing commit, e.g:
https://www.fossil-scm.org/index.html/info/d1438b400b85dfe8 had its
comment changed by
https://www.fossil-scm.org/index.html/info/680b5619ff723562

Re: Should we restore manually maintained ChangeLogs

[Christopher Allan Webber]

Andreas Schwab writes:

> Alan Mackenzie <acm@muc.de> writes:
>
>> Hello, Stefan.
>>
>> On Wed, Mar 09, 2016 at 02:09:53PM -0500, Stefan Monnier wrote:
>>> > Maybe we should move to a version control system which permits commit
>>> > messages to be amended.  (Only half joking.)
>>
>>> These seem to be in rather short supply, sadly.  Any suggestion?
>>
>> Mercurial.  It has the hg histedit command.
>
> Git.  It has the git rebase command.
>
> Andreas.

That requires throwing out your previous history... that's not good
practice for code already pushed to a master branch, unless it's an
emergency.  It creates huge headaches.

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

On 03/09/2016 12:21 PM, Eli Zaretskii wrote:
> it's fortress -- if you dismantle one of its bastions, the others
> will soon fall as well.

It's not a fortress, as is demonstrated in other GNU projects. For 
example, here are the last three commit messages for coreutils, which 
has been using the proposed approach since 2008. These commit messages 
are just as good as what we typically see for Emacs. Maybe better. I'm 
sure we can improve the quality of Emacs commit messages, but putting 
them into ChangeLog files is not necessary or even all that helpful for 
doing so.

   commit e735d417fb2e5c7427b3622f2a78e65e450b49a8
   Author: Eric Blake <eblake@redhat.com>
   Date:   Tue Mar 8 16:01:34 2016 -0700

       build: update gnulib submodule to latest

       Mainly for:
       *bdb72bc6 set-permissions: fix compilation on Cygwin

       * bootstrap: Sync with gnulib.
       * gl/lib/regcomp.c.diff: Regenerate against latest gnulib.

   commit 68ede201e8890b757aa37d72ce4dca872155f7db
   Author: Jim Meyering <meyering@fb.com>
   Date:   Sun Mar 6 16:38:01 2016 -0800

       tests: avoid false-failure of split/filter.sh on XFS

       * tests/split/filter.sh: Use OFF_T_MAX-1 rather than OFF_T_MAX
       as the size of a test file, to avoid false failure on an XFS file
       system (or any file system permitting a file of size OFF_T_MAX).
       Reported as http://bugs.gnu.org/22931

   commit 6475c514550bc1fbdb3410485312211726765798
   Author: Eric Blake <eblake@redhat.com>
   Date:   Fri Mar 4 09:47:33 2016 -0700

       test: Document that -a and -o are undesirable

       POSIX recommends avoiding -a and -o, for good reason.
       src/test.c (usage): Mention that inherent ambiguities exist with
       binary -a and -o.
       Problem reported by Martin Gebert in: http://bugs.gnu.org/22909

Re: Should we restore manually maintained ChangeLogs

[Andreas Schwab]

Nikolaus Rath <Nikolaus@rath.org> writes:

> On Mar 09 2016, Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:
>>> Maybe we should move to a version control system which permits commit
>>> messages to be amended.  (Only half joking.)
>>
>> These seem to be in rather short supply, sadly.  Any suggestion?
>
> Mercurial with the "evolve" extension implements non-destructive history
> editing. In short, commits can have metadata that marks them as
> "superseding" a previous commit. Both superseded and new commit remain
> part of the repository, but most commands be default work on the
> superseding commit.

Like git replace.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/09/2016 11:57 AM, Eli Zaretskii wrote:
>> Like it or not, that is a form of historical research. 
> No, it's a very far cry from historical research.
>

I won't insist on calling it "historical research", admittedly a phrase 
that suggests bigger things than finding out why Emacs doesn't use 
setenv on "TZ". Still, the point remains that primary sources are 
unreliable in Emacs development, just as they are in any form of 
historical research. If someone ever gets around to writing a definitive 
history of Emacs (I'm looking at you, ESR!), then they'll have to take 
ChangeLogs with a grain of salt, just as you and I do in routine code 
spelunking.

> That cost is much lower than any of the alternatives proposed so far, 
> including the current arrangement with ChangeLog.2. It worked for years.

I'm afraid we'll have to disagree on costs. The old way of doing things 
was a constant irritation to me and to others.

>> Regardless of the approach taken, there is also a cost to
>> sprucing up the historical record
> Since this is regardless of the approach, it shouldn't affect the
> decision in this matter.

No, they're still related. If sprucing up ChangeLogs is low-priority 
work that distracts us from other things, then it's not advantageous to 
adopt a technical approach merely on the grounds that the approach makes 
it easier to spruce up ChangeLogs.

Re: ChangeLog to *VC-log*

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: dgutov@yandex.ru,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Thu, 10 Mar 2016 00:01:00 +0100
> 
> > No, _you_ should create a file at top level, and then everything will
> > continue working.  That's why the top-level file is called
> > ChangeLog.2: to keep it out of your way.
> 
> But if Emacs can do that for me (and in so doing automatically comply
> with its commit message format specification), don't you agree that's
> more user-friendly?  Do you see a down side to having that automatism?

We are trying to bend features so that they do what we think we want
when we work on Emacs.  But Emacs is not only for working on Emacs,
and creating a top-level ChangeLog file is not necessarily TRT for all
those other projects.

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Eli Zaretskii]

> Cc: yuri.v.khan@gmail.com, johnw@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 15:22:31 -0800
> 
> On 03/09/2016 11:53 AM, Eli Zaretskii wrote:
> 
> >
> >> And people shouldn't need to cough up the right regex; all they need
> >> to do is run "./autogen.sh all" or "./autogen.sh git".
> > That still misses a couple that add-change-log-entry doesn't.
> 
> Which two languages? It should be easy to fix and I can volunteer to do 
> that.

Shell scripts and Makefiles are the two I thought of immediately.
Maybe there are more, I don't know.  How about TeX and PS, for
example?  We have a few of those.

Basically, every major mode that has support for add-change-log-entry
and for which we have files in the repository is a candidate.

Btw, the best solution would be to have these tricks added to standard
Git installation, or maybe to GNU Diff (which I think Git uses?).  Why
should users need to reinvent the same wheel all the time?

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> Cc: johnw@gnu.org, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 15:39:25 -0800
> 
> On 03/09/2016 12:21 PM, Eli Zaretskii wrote:
> > it's fortress -- if you dismantle one of its bastions, the others
> > will soon fall as well.
> 
> It's not a fortress, as is demonstrated in other GNU projects. For 
> example, here are the last three commit messages for coreutils, which 
> has been using the proposed approach since 2008. These commit messages 
> are just as good as what we typically see for Emacs. Maybe better. I'm 
> sure we can improve the quality of Emacs commit messages, but putting 
> them into ChangeLog files is not necessary or even all that helpful for 
> doing so.
> 
>    commit e735d417fb2e5c7427b3622f2a78e65e450b49a8
>    Author: Eric Blake <eblake@redhat.com>
>    Date:   Tue Mar 8 16:01:34 2016 -0700

If I could by some magic wand make a few distinguished individuals
become part of the Emacs team, and never give write access to anyone
else, then those two would certainly be on the list.  And then we
could simply forget about the problems we are arguing ab out, because
they would have become nonexistent.

But as long as we are trying to make Emacs development be more
crowdware (which I think is a correct direction, given that Emacs is a
very large collection of loosely-related packages), we cannot rely on
the high quality of log messages produced by the individual
contributors, because many of them will only casually contribute, and
won't have the kind of experience and habits that Eric and Jim have.

We must have a convenient way of correcting the log messages.  The
alternative is to give push rights to a very few individuals, and have
a patch review process through which the log messages could be
corrected before pushing, but that requires more manpower than we
currently have, so it cannot be used for now.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Thu, 10 Mar 2016 00:43:57 +0100
> 
> > Mercurial with the "evolve" extension implements non-destructive history
> > editing. In short, commits can have metadata that marks them as
> > "superseding" a previous commit. Both superseded and new commit remain
> > part of the repository, but most commands be default work on the
> > superseding commit.
> 
> Like git replace.

Do you know how does "git replace" interact with merging, rebasing,
and cherry-picking?  The man page doesn't seem to mention these.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 9 Mar 2016 17:32:44 -0800
> 
> > That cost is much lower than any of the alternatives proposed so far, 
> > including the current arrangement with ChangeLog.2. It worked for years.
> 
> I'm afraid we'll have to disagree on costs. The old way of doing things 
> was a constant irritation to me and to others.

Your opinion on this needs to be taken with a grain of salt, since you
never tried to install git-merge-changelog.  Without it, I agree with
you that ChangeLog merge conflicts are very irritating.  But that's
exactly why git-merge-changelog was written.

> >> Regardless of the approach taken, there is also a cost to
> >> sprucing up the historical record
> > Since this is regardless of the approach, it shouldn't affect the
> > decision in this matter.
> 
> No, they're still related. If sprucing up ChangeLogs is low-priority 
> work that distracts us from other things, then it's not advantageous to 
> adopt a technical approach merely on the grounds that the approach makes 
> it easier to spruce up ChangeLogs.

It's not low-priority work when I need an accurate accord of what
happened.  Then it's very high priority for me.

Re: ChangeLog to *VC-log*

[Xue Fuqiao]

On Thu, Mar 10, 2016 at 3:30 AM, Stephen Berman <stephen.berman@gmx.net> wrote:

Hi Stephen,

> I found the cause of the change in behavior I observed: it's due to the
> elimination of files named "ChangeLog" from the Emacs sources (there are
> now only files named "ChangeLog.<n>").  In consequence, find-change-log
> creates an empty file named "ChangeLog" in the immediate directory of
> the file in which `C-x 4 a' was invoked, e.g. in lisp/calendar/, when
> I'm editing todo-mode.el; previously, `C-x 4 a' found lisp/ChangeLog.  I
> guess those who don't see this keep an non-VCS-tracked file "ChangeLog"
> at top level.
>
> If we don't return to maintaining versioned ChangeLog files, I think it
> would be desirable for `C-x 4 a' to always create (or find) the file
> "ChangeLog" at top level when editing Emacs sources, since this would
> add the required directory levels to the commit message.  (Of course,
> people could just always use `C-u C-x 4 a', but that's not as
> convenient.)  Since other projects that could use `C-x 4 a' may have
> different requirements on its effect, this should be conditioned by a
> user option (or maybe just a plain variable), which can (and probably
> should) be set in .dir-locals.el.  What do others think?

Does commit dc435af152e6df3d85b0c21eaf9ff39dce0091bb fix this issue?

(I'm not sure if I understand the issue at hand correctly, because I
don't use 'C-x 4 a' very often.)

Re: Should we restore manually maintained ChangeLogs

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Date: Thu, 10 Mar 2016 00:43:57 +0100
>> 
>> > Mercurial with the "evolve" extension implements non-destructive history
>> > editing. In short, commits can have metadata that marks them as
>> > "superseding" a previous commit. Both superseded and new commit remain
>> > part of the repository, but most commands be default work on the
>> > superseding commit.
>> 
>> Like git replace.
>
> Do you know how does "git replace" interact with merging, rebasing,
> and cherry-picking?  The man page doesn't seem to mention these.

git replace should be transparent most of the time (except for some
plumbing like git cat-file, and you can ignore replacements with
--no-replace-objects).  But note that they are stored in a different
namespace that is not pulled by default.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Yuri Khan]

On Thu, Mar 10, 2016 at 12:36 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> Btw, the best solution would be to have these tricks added to standard
> Git installation, or maybe to GNU Diff (which I think Git uses?).  Why
> should users need to reinvent the same wheel all the time?

Git in fact predefines header regexps for many languages predefined
(see list below). It just does not apply them out-of-the-box; the user
has to add the “*.tex diff=tex” and similar mappings to one of their
gitattributes file. Actually, mappings that are useful for all
contributors could be stored in the version-controlled .gitattributes
file; no need for magic autogen.sh machinery.

    The following built in patterns are available:

       ·   ada suitable for source code in the Ada language.
       ·   bibtex suitable for files with BibTeX coded references.
       ·   cpp suitable for source code in the C and C++ languages.
       ·   csharp suitable for source code in the C# language.
       ·   fortran suitable for source code in the Fortran language.
       ·   fountain suitable for Fountain documents.
       ·   html suitable for HTML/XHTML documents.
       ·   java suitable for source code in the Java language.
       ·   matlab suitable for source code in the MATLAB language.
       ·   objc suitable for source code in the Objective-C language.
       ·   pascal suitable for source code in the Pascal/Delphi language.
       ·   perl suitable for source code in the Perl language.
       ·   php suitable for source code in the PHP language.
       ·   python suitable for source code in the Python language.
       ·   ruby suitable for source code in the Ruby language.
       ·   tex suitable for source code for LaTeX documents.

(from gitattributes(5) as of Git 2.7.1)

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: emacs-devel@gnu.org
> Date: Thu, 10 Mar 2016 10:05:06 +0100
> 
> git replace should be transparent most of the time (except for some
> plumbing like git cat-file, and you can ignore replacements with
> --no-replace-objects).  But note that they are stored in a different
> namespace that is not pulled by default.

Does "pull --all" solve that?

More generally, can we put something in .git/config that would cause
that namespace to be pulled by default?

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Thu, 10 Mar 2016 15:14:13 +0600
> Cc: Paul Eggert <eggert@cs.ucla.edu>, John Wiegley <johnw@gnu.org>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> On Thu, Mar 10, 2016 at 12:36 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > Btw, the best solution would be to have these tricks added to standard
> > Git installation, or maybe to GNU Diff (which I think Git uses?).  Why
> > should users need to reinvent the same wheel all the time?
> 
> Git in fact predefines header regexps for many languages predefined
> (see list below). It just does not apply them out-of-the-box; the user
> has to add the “*.tex diff=tex” and similar mappings to one of their
> gitattributes file.

Then I guess my suggestion above is that these _are_ applied out of
the box.

>     The following built in patterns are available:
> 
>        ·   ada suitable for source code in the Ada language.
>        ·   bibtex suitable for files with BibTeX coded references.
>        ·   cpp suitable for source code in the C and C++ languages.
>        ·   csharp suitable for source code in the C# language.
>        ·   fortran suitable for source code in the Fortran language.
>        ·   fountain suitable for Fountain documents.
>        ·   html suitable for HTML/XHTML documents.
>        ·   java suitable for source code in the Java language.
>        ·   matlab suitable for source code in the MATLAB language.
>        ·   objc suitable for source code in the Objective-C language.
>        ·   pascal suitable for source code in the Pascal/Delphi language.
>        ·   perl suitable for source code in the Perl language.
>        ·   php suitable for source code in the PHP language.
>        ·   python suitable for source code in the Python language.
>        ·   ruby suitable for source code in the Ruby language.
>        ·   tex suitable for source code for LaTeX documents.

We need some more, which are not in this list.

Thanks.

Re: Should we restore manually maintained ChangeLogs

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 10 Mar 2016 10:05:06 +0100
>> 
>> git replace should be transparent most of the time (except for some
>> plumbing like git cat-file, and you can ignore replacements with
>> --no-replace-objects).  But note that they are stored in a different
>> namespace that is not pulled by default.
>
> Does "pull --all" solve that?

No, you need to add a fetch spec for it.  pull --all is only about
multiple remotes.

> More generally, can we put something in .git/config that would cause
> that namespace to be pulled by default?

git config --add remote.origin.fetch +refs/replace/*:refs/replace/*

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Code reviews

[Andreas Röhler]

Here just my two cents:

WRT logs: code itself as basic reference, i.e. using rather the diffs 
than second sources like log-messages. In favor of dropping the changelog.

WRT code-review: More important seems a rigid test-policy, i.e. no 
commit without tests passed. Focus on analyze why things got broken, how 
the bug could pass the tests.

Cheers,

Andreas

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: emacs-devel@gnu.org
> Date: Thu, 10 Mar 2016 11:00:47 +0100
> 
> > More generally, can we put something in .git/config that would cause
> > that namespace to be pulled by default?
> 
> git config --add remote.origin.fetch +refs/replace/*:refs/replace/*

OK, thanks.

So how does one go about creating a replacement commit for a commit
that was pushed upstream?  The man page of "git replace" mentions "git
rebase", but I cannot find anything pertinent on the rebase man page.
The procedure described here:

  https://git-scm.com/book/en/v2/Git-Tools-Replace

sounds quite complicated.  If we need to do all that every time we
want to fix a single commit, I'd expect a lot of mistakes and snafus.
But maybe there's an easier way, in which case we could consider this
feature as part of the solution for fixing mistakes in log messages.

Re: Should we restore manually maintained ChangeLogs

[Sven Axelsson]

On 10 March 2016 at 11:34, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Cc: emacs-devel@gnu.org
>> Date: Thu, 10 Mar 2016 11:00:47 +0100
>>
>> > More generally, can we put something in .git/config that would cause
>> > that namespace to be pulled by default?
>>
>> git config --add remote.origin.fetch +refs/replace/*:refs/replace/*
>
> OK, thanks.
>
> So how does one go about creating a replacement commit for a commit
> that was pushed upstream?  The man page of "git replace" mentions "git
> rebase", but I cannot find anything pertinent on the rebase man page.
> The procedure described here:
>
>   https://git-scm.com/book/en/v2/Git-Tools-Replace
>
> sounds quite complicated.  If we need to do all that every time we
> want to fix a single commit, I'd expect a lot of mistakes and snafus.
> But maybe there's an easier way, in which case we could consider this
> feature as part of the solution for fixing mistakes in log messages.
>

When you only want to deal with changing the commit message and not
the actual commit, it is as simple as calling `git replace --edit
<SHA>'. This will bring up your system editor where you can change the
commit message.

Doing it this way will not change history.

-- 
Sven Axelsson
++++++++++[>++++++++++>+++++++++++>++++++++++>++++++
>++++<<<<<-]>++++.+.++++.>+++++.>+.<<-.>>+.>++++.<<.
+++.>-.<<++.>>----.<++.>>>++++++.<<<<.>>++++.<----.

Re: ChangeLog to *VC-log*

[Stephen Berman]

On Thu, 10 Mar 2016 08:26:56 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: dgutov@yandex.ru,  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
>> Date: Thu, 10 Mar 2016 00:01:00 +0100
>> 
>> > No, _you_ should create a file at top level, and then everything will
>> > continue working.  That's why the top-level file is called
>> > ChangeLog.2: to keep it out of your way.
>> 
>> But if Emacs can do that for me (and in so doing automatically comply
>> with its commit message format specification), don't you agree that's
>> more user-friendly?  Do you see a down side to having that automatism?
>
> We are trying to bend features so that they do what we think we want
> when we work on Emacs.  But Emacs is not only for working on Emacs,
> and creating a top-level ChangeLog file is not necessarily TRT for all
> those other projects.

That's why I wrote this in the post you replied to: "Since other
projects that could use `C-x 4 a' may have different requirements on its
effect, this should be conditioned by a user option (or maybe just a
plain variable), which can (and probably should) be set in
.dir-locals.el."  But anyway, Glenn Morris said he already implemented
something like this on master, which I have not yet tested or even seen,
so I don't know how flexible it is.

Steve Berman

Re: ChangeLog to *VC-log*

[Stephen Berman]

On Thu, 10 Mar 2016 15:16:01 +0800 Xue Fuqiao <xfq.free@gmail.com> wrote:

> On Thu, Mar 10, 2016 at 3:30 AM, Stephen Berman <stephen.berman@gmx.net> wrote:
>
> Hi Stephen,
>
>> I found the cause of the change in behavior I observed: it's due to the
>> elimination of files named "ChangeLog" from the Emacs sources (there are
>> now only files named "ChangeLog.<n>").  In consequence, find-change-log
>> creates an empty file named "ChangeLog" in the immediate directory of
>> the file in which `C-x 4 a' was invoked, e.g. in lisp/calendar/, when
>> I'm editing todo-mode.el; previously, `C-x 4 a' found lisp/ChangeLog.  I
>> guess those who don't see this keep an non-VCS-tracked file "ChangeLog"
>> at top level.
>>
>> If we don't return to maintaining versioned ChangeLog files, I think it
>> would be desirable for `C-x 4 a' to always create (or find) the file
>> "ChangeLog" at top level when editing Emacs sources, since this would
>> add the required directory levels to the commit message.  (Of course,
>> people could just always use `C-u C-x 4 a', but that's not as
>> convenient.)  Since other projects that could use `C-x 4 a' may have
>> different requirements on its effect, this should be conditioned by a
>> user option (or maybe just a plain variable), which can (and probably
>> should) be set in .dir-locals.el.  What do others think?
>
> Does commit dc435af152e6df3d85b0c21eaf9ff39dce0091bb fix this issue?

Glenn Morris mentioned this in bug#22968; since I'm not currently using
master, I haven't tried it yet, but intend to as soon as I can.

Steve Berman

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

> fossil allows commit message editing. It works by adding a new
> commit-editing commit, e.g:

Cool, thanks, I haven't looked seriously at fossil yet, so I'm glad to
hear that they took this issue seriously.


        Stefan

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Date: Thu, 10 Mar 2016 12:49:34 +0100
> From: Sven Axelsson <sven.axelsson@gmail.com>
> Cc: Andreas Schwab <schwab@linux-m68k.org>, emacs <emacs-devel@gnu.org>
> 
> When you only want to deal with changing the commit message and not
> the actual commit, it is as simple as calling `git replace --edit
> <SHA>'. This will bring up your system editor where you can change the
> commit message.
> 
> Doing it this way will not change history.

How can it not change history, when (AFAIK) the log message is part of
the commit object, and it included in the stuff that determines the
SHA1 checksum of the commit?

Re: ChangeLog to *VC-log*

[Stefan Monnier]

> and creating a top-level ChangeLog file is not necessarily TRT for all
> those other projects.

Creating a ChangeLog right along side the changed file is not
necessarily TRT either.  IOW what we've done in the past was just
a heuristic where the really wasn't many other choices, but if you
consider the "VC root" information that's available nowadays it's
actually a very valid fallback option as well.

From where I stand "<VCroot>/ChangeLog" is a better fallback option than
"<cwd>/ChangeLog", but in any case either choice is just a "best guess"
and will not always be right (which is why we only make this choice
when there's no pre-existing ChangeLog file).


        Stefan

Re: Should we restore manually maintained ChangeLogs

[Evgeny Panasyuk]

10.03.2016 16:08, Eli Zaretskii:
>> Doing it this way will not change history.
>
> How can it not change history, when (AFAIK) the log message is part of
> the commit object, and it included in the stuff that determines the
> SHA1 checksum of the commit?

As I understand it does not change original commit object, but rather 
creates kind of "replacement rule". In a similar manner as git notes 
attaches additional info to commit message without changing original one.
If you would not fetch replacement refs then you will not see these 
"amendments".

-- 
Evgeny Panasyuk

Re: Should we restore manually maintained ChangeLogs

[Sven Axelsson]

On 10 March 2016 at 14:14, Evgeny Panasyuk <evgeny.panasyuk@gmail.com> wrote:
> 10.03.2016 16:08, Eli Zaretskii:
>>>
>>> Doing it this way will not change history.
>>
>>
>> How can it not change history, when (AFAIK) the log message is part of
>> the commit object, and it included in the stuff that determines the
>> SHA1 checksum of the commit?
>
>
> As I understand it does not change original commit object, but rather
> creates kind of "replacement rule". In a similar manner as git notes
> attaches additional info to commit message without changing original one.
> If you would not fetch replacement refs then you will not see these
> "amendments".

That's right. As mentioned earlier in the thread, it creates a
replacement object stored in a special refs hierarchy. But for normal
use, e.g. git log, git grep and such it appears as if the commit
objects have been changed in-place.

-- 
Sven Axelsson
++++++++++[>++++++++++>+++++++++++>++++++++++>++++++
>++++<<<<<-]>++++.+.++++.>+++++.>+.<<-.>>+.>++++.<<.
+++.>-.<<++.>>----.<++.>>>++++++.<<<<.>>++++.<----.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> Date: Thu, 10 Mar 2016 14:25:49 +0100
> From: Sven Axelsson <sven.axelsson@gmail.com>
> Cc: emacs <emacs-devel@gnu.org>

If using "git replace" to fix a mistake in a log message is as simple
as saying "git replace --edit SHA1" followed by "git push", then
perhaps someone would like to try this for a while in a test
repository, and if that gives good results, we could start using that.

Thanks.

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

> When you only want to deal with changing the commit message and not
> the actual commit, it is as simple as calling `git replace --edit
> <SHA>'. This will bring up your system editor where you can change the
> commit message.

Ha!  Finally!

So "git replace" seems like it might do the trick.
At least it leads to the right "git log" output.  Now the question is:
how do we distribute those "replacements"?

According to "git replace --help" they're added to the refs/replace/
namespace, so what do we need to do for push/pull to propagate them?


        Stefan

Re: Should we restore manually maintained ChangeLogs

[Sven Axelsson]

On 10 March 2016 at 16:03, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> When you only want to deal with changing the commit message and not
>> the actual commit, it is as simple as calling `git replace --edit
>> <SHA>'. This will bring up your system editor where you can change the
>> commit message.
>
> Ha!  Finally!
>
> So "git replace" seems like it might do the trick.
> At least it leads to the right "git log" output.  Now the question is:
> how do we distribute those "replacements"?
>
> According to "git replace --help" they're added to the refs/replace/
> namespace, so what do we need to do for push/pull to propagate them?

Andreas Schwab answered that in an earlier message.

> More generally, can we put something in .git/config that would cause
> that namespace to be pulled by default?

git config --add remote.origin.fetch +refs/replace/*:refs/replace/*

-- 
Sven Axelsson
++++++++++[>++++++++++>+++++++++++>++++++++++>++++++
>++++<<<<<-]>++++.+.++++.>+++++.>+.<<-.>>+.>++++.<<.
+++.>-.<<++.>>----.<++.>>>++++++.<<<<.>>++++.<----.

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/10/2016 07:03 AM, Stefan Monnier wrote:
> So "git replace" seems like it might do the trick.

I'm not a fan of 'git replace' for the same reason I'm not a fan of 
changing history in general. I don't plan to use it and won't recommend 
it to others. However, if allowing its usage is the price we need to pay 
for Eli's agreement then I'm willing to pay that price.

As I understand it, the patch I proposed should work just fine when 
combined with 'git-replace'.

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/09/2016 10:55 PM, Eli Zaretskii wrote:
> Your opinion on this needs to be taken with a grain of salt, since you 
> never tried to install git-merge-changelog. Without it, I agree with 
> you that ChangeLog merge conflicts are very irritating. But that's 
> exactly why git-merge-changelog was written.

This doesn't address the point that the old system had real problems 
even when I wasn't doing merges (which is my normal mode of operation). 
And Dmitry reported problems even when using git-merge-changelog.

> It's not low-priority work when I need an accurate accord of what 
> happened. Then it's very high priority for me. 

When I want an accurate record of what happened, I want to see what was 
actually committed, warts and all. If people start using git-replace to 
change old commit messages, I'll probably start using 
--no-replace-objects to see what the original commit looked like. I 
realize your style differs. Still, it's not clear that it's worth 
inflicting significant pain on other developers in order to support that 
style.

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

> I'm not a fan of 'git replace' for the same reason I'm not a fan of changing
> history in general.

Indeed, "git replace" is a lot more powerful than I'd like.

I don't consider commit messages as being part of "what happened" but
rather as a description of what happened (i.e. what historiographers
write).  So, I'm perfectly fine with "git replace" used on
commit messages.

[ IOW If it was for me, the computation of the SHA1 would not include
  the commit message.  ]

> I don't plan to use it and won't recommend it to others.  However, if
> allowing its usage is the price we need to pay for Eli's agreement
> then I'm willing to pay that price.

Is there some way to control its usage?


        Stefan

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

> When I want an accurate record of what happened, I want to see what was
> actually committed, warts and all. If people start using git-replace to
> change old commit messages, I'll probably start using --no-replace-objects
> to see what the original commit looked like. I realize your style
> differs. Still, it's not clear that it's worth inflicting significant pain
> on other developers in order to support that style.

Maybe the Git guys would accept a patch which lets you provide
a "--full-history" option to "git log" such that for every commit whose
message has a "replace"ment, it will show you not only the replacement,
but the original message as well (and maybe even the sequence of
replacements, if there can be such a thing).


        Stefan

Re: Should we restore manually maintained ChangeLogs

[Yuri Khan]

On Thu, Mar 10, 2016 at 11:17 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:

> Indeed, "git replace" is a lot more powerful than I'd like.
>
> Is there some way to control its usage?

Perhaps an update hook on the central repository could abort the push
if the ref name starts with refs/replace/ and the diff between old and
new ref states is non-empty.

Re: Should we restore manually maintained ChangeLogs

[Paul Eggert]

On 03/10/2016 09:17 AM, Stefan Monnier wrote:
> Indeed, "git replace" is a lot more powerful than I'd like....
> Is there some way to control its usage?

As far as I know, the only control is to ignore all replacements, by 
using git's --no-replace-objects option or setting the 
GIT_NO_REPLACE_OBJECTS environment variable.

As I understand it, 'git replace' was motivated by the need to do 
fancier things, e.g., link a repository with another old repository. It 
is indeed a powerful mechanism.

Re: Should we restore manually maintained ChangeLogs

[Wolfgang Jenkner]

On Thu, Mar 10 2016, Paul Eggert wrote:

> When I want an accurate record of what happened, I want to see what
> was actually committed, warts and all.

Wouldn't git-notes be a better option then (IIUC), so that instead of
the original commit message being superseded there would be a list of
errata (or other notes), which git-log still shows by default?

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 10 Mar 2016 12:17:06 -0500
> Cc: emacs-devel@gnu.org
> 
> > I'm not a fan of 'git replace' for the same reason I'm not a fan of changing
> > history in general.
> 
> Indeed, "git replace" is a lot more powerful than I'd like.
> 
> I don't consider commit messages as being part of "what happened" but
> rather as a description of what happened (i.e. what historiographers
> write).  So, I'm perfectly fine with "git replace" used on
> commit messages.

That's my view as well, although having heard about this command just
today admittedly doesn't give my view too much weight.

> > I don't plan to use it and won't recommend it to others.  However, if
> > allowing its usage is the price we need to pay for Eli's agreement
> > then I'm willing to pay that price.
> 
> Is there some way to control its usage?

I think we also need to test-drive it before we decide to use it.
I hope someone will try that and report back.

Re: Should we restore manually maintained ChangeLogs

[Stefan Monnier]

> I think we also need to test-drive it before we decide to use it.
> I hope someone will try that and report back.

Indeed.  E.g. if there are ten thousand replacements, does it affect
performance noticeably?


        Stefan

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > That's pretty much what I meant.  It may not contain the information you
  > > (and I) would like to see in it, but it is highly reliable in that it is
  > > the message that was used to describe the commit.

  > Yeah, it's a reliable lie.  How helpful is that?

"Lie" means a false statement intentionally made as an attempt to
deceive.  If we are talking about errors, it is harsh (and incorrect)
to call them "lies".  Let's just say "errors".

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > So removing ChangeLog files will be a bad blow to our ability to 
  > > easily and conveniently research the past,

  > No, this doesn't follow. If we use ChangeLog formats in commit messages, 
  > we can still research the past easily and conveniently.

I agree, as long as we include a compilation as a ChangeLog file
in tar releases.

Even though Git does not permit editing a commit message _as such_, we
can provide, at a higher level, a way to correct old commit messages.
Maintaining a ChangeLog file in the repository is one way to do this,
but if it has flaws in the case of merging, can we design another mechanism
for the job that solves the problems with merging?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

On 03/10/2016 01:23 PM, Richard Stallman wrote:
> Maintaining a ChangeLog file in the repository is one way to do this,
> but if it has flaws in the case of merging, can we design another mechanism
> for the job that solves the problems with merging?

Yes, one such approach (used by coreutils) was proposed a couple of days 
ago, in the patch archived here:

http://lists.gnu.org/archive/html/emacs-devel/2016-03/msg00391.html

Eli dislikes the way this supports patching old commit messages, and has 
proposed to instead use 'git replace'; we'd still use the abovementioned 
patch, but would patch old commit messages in a different way. We 
haven't yet tested this suggestion.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: eliz@gnu.org, johnw@gnu.org, emacs-devel@gnu.org
> Date: Thu, 10 Mar 2016 16:23:48 -0500
> 
> Even though Git does not permit editing a commit message _as such_, we
> can provide, at a higher level, a way to correct old commit messages.
> Maintaining a ChangeLog file in the repository is one way to do this,
> but if it has flaws in the case of merging, can we design another mechanism
> for the job that solves the problems with merging?

The known mechanisms are user-unfriendly, in that they require to
write the corrections as editing commands for Perl or some other
Sed-like facility.  This is too error prone for a feature that should
be usable by non-experts.

A facility that could be considered is one that allows to type the
correction as plain text which should replace the existing text in the
Git log.  We haven't found such a facility yet.  "git replace" sounds
like a promising alternative, though.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 11 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Richard Stallman <rms@gnu.org>
>> CC: eliz@gnu.org, johnw@gnu.org, emacs-devel@gnu.org
>> Date: Thu, 10 Mar 2016 16:23:48 -0500
>> 
>> Even though Git does not permit editing a commit message _as such_, we
>> can provide, at a higher level, a way to correct old commit messages.
>> Maintaining a ChangeLog file in the repository is one way to do this,
>> but if it has flaws in the case of merging, can we design another mechanism
>> for the job that solves the problems with merging?
>
> The known mechanisms are user-unfriendly, in that they require to
> write the corrections as editing commands for Perl or some other
> Sed-like facility.  This is too error prone for a feature that should
> be usable by non-experts.
>
> A facility that could be considered is one that allows to type the
> correction as plain text which should replace the existing text in the
> Git log.  We haven't found such a facility yet.  "git replace" sounds
> like a promising alternative, though.

I think the low-tech option of a "commit-msg-override" directory with
one (updated) commit message per file (and the file name being the hash
of the commit being updated) is also still on the table.

This directory could be trivially checked by whatever tool will be used
to turn the git log into a ChangeLog file.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

Nikolaus Rath wrote:
> I think the low-tech option of a "commit-msg-override" directory with
> one (updated) commit message per file (and the file name being the hash
> of the commit being updated) is also still on the table.

Sure, I could add that fairly easily too. That would avoid the potential 
problems with 'git replace'.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Date: Fri, 11 Mar 2016 09:17:02 -0800
> 
> I think the low-tech option of a "commit-msg-override" directory with
> one (updated) commit message per file (and the file name being the hash
> of the commit being updated) is also still on the table.
> 
> This directory could be trivially checked by whatever tool will be used
> to turn the git log into a ChangeLog file.

That could work, provided that Someone™ will post the changes to the
gitlog-to-changelog script (or maybe an entirely different script) to
produce ChangeLog from the log messages.

This solution should also support merging from the release branch to
master and cherry-picks in the opposite direction, where (AFAIK) SHA1
checksum is not necessarily left intact.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 11 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Date: Fri, 11 Mar 2016 09:17:02 -0800
>> 
>> I think the low-tech option of a "commit-msg-override" directory with
>> one (updated) commit message per file (and the file name being the hash
>> of the commit being updated) is also still on the table.
>> 
>> This directory could be trivially checked by whatever tool will be used
>> to turn the git log into a ChangeLog file.
>
> That could work, provided that Someone™ will post the changes to the
> gitlog-to-changelog script (or maybe an entirely different script) to
> produce ChangeLog from the log messages.
>
> This solution should also support merging from the release branch to
> master and cherry-picks in the opposite direction, where (AFAIK) SHA1
> checksum is not necessarily left intact.

I don't think *any* solution can support cherry-picks without extra
manual work. Even if you keep the ChangeLog file under version control
and edit it by hand, you have to cherry-pick the original commit plus
the later commit that updated the ChangeLog.

The same applies here, you have to cherry-pick both the original commit
and the later one that created the "overwrite file" in the
"commit-msg-override" directory.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Date: Fri, 11 Mar 2016 11:53:32 -0800
> 
> > This solution should also support merging from the release branch to
> > master and cherry-picks in the opposite direction, where (AFAIK) SHA1
> > checksum is not necessarily left intact.
> 
> I don't think *any* solution can support cherry-picks without extra
> manual work. Even if you keep the ChangeLog file under version control
> and edit it by hand, you have to cherry-pick the original commit plus
> the later commit that updated the ChangeLog.

ChangeLog is updated in the same commit as the code, so cherry-picking
brings them both to the other branch.  Or maybe I misunderstand what
you mean by "the later commit"?

> The same applies here, you have to cherry-pick both the original commit
> and the later one that created the "overwrite file" in the
> "commit-msg-override" directory.

I hoped for some post-commit hook that could do that automatically, or
at least suggest that another commit needs to be picked.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

Hi Eli,

Please do not Cc me on replies, I am reading the list.


On Mar 11 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Date: Fri, 11 Mar 2016 11:53:32 -0800
>> 
>> > This solution should also support merging from the release branch to
>> > master and cherry-picks in the opposite direction, where (AFAIK) SHA1
>> > checksum is not necessarily left intact.
>> 
>> I don't think *any* solution can support cherry-picks without extra
>> manual work. Even if you keep the ChangeLog file under version control
>> and edit it by hand, you have to cherry-pick the original commit plus
>> the later commit that updated the ChangeLog.
>
> ChangeLog is updated in the same commit as the code, so cherry-picking
> brings them both to the other branch.  Or maybe I misunderstand what
> you mean by "the later commit"?

We are talking about the case were the ChangeLog entry / original commit
message was incorrect and need to be corrected later (in the "later
commit").

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Date: Fri, 11 Mar 2016 12:08:09 -0800
> 
> Please do not Cc me on replies, I am reading the list.

That's the standard way of replying to a message: to the From address
and to the CC address.  I edited out yours this time, but I cannot
afford doing that every time; if you want not to receive direct
replies, you will have to configure your MUA to insert the necessary
headers (like Mail-followups-to etc.).

> > ChangeLog is updated in the same commit as the code, so cherry-picking
> > brings them both to the other branch.  Or maybe I misunderstand what
> > you mean by "the later commit"?
> 
> We are talking about the case were the ChangeLog entry / original commit
> message was incorrect and need to be corrected later (in the "later
> commit").

Apologies for my misunderstanding.

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> This directory could be trivially checked by whatever tool will be used
> to turn the git log into a ChangeLog file.

For me a solution which doesn't work (efficiently enough) with
vc-region-history and vc-print-log is just pointless.


        Stefan

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 11 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Date: Fri, 11 Mar 2016 12:08:09 -0800
>> 
>> Please do not Cc me on replies, I am reading the list.
>
> That's the standard way of replying to a message: to the From address
> and to the CC address.  I edited out yours this time, but I cannot
> afford doing that every time; if you want not to receive direct
> replies, you will have to configure your MUA to insert the necessary
> headers (like Mail-followups-to etc.).

Well, my messages all contain both:

Mail-Copies-To: never
Mail-Followup-To: emacs-devel@gnu.org

..is there any other convention that I'm not yet following?


Thanks,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 11 2016, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> This directory could be trivially checked by whatever tool will be used
>> to turn the git log into a ChangeLog file.
>
> For me a solution which doesn't work (efficiently enough) with
> vc-region-history and vc-print-log is just pointless.

Ah, yeah, there was that. Sorry, forgot about that part. So it needs to
be properly integrated in git then.


Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Date: Fri, 11 Mar 2016 19:32:49 -0800
> 
> On Mar 11 2016, Eli Zaretskii <eliz@gnu.org> wrote:
> >> From: Nikolaus Rath <Nikolaus@rath.org>
> >> Date: Fri, 11 Mar 2016 12:08:09 -0800
> >> 
> >> Please do not Cc me on replies, I am reading the list.
> >
> > That's the standard way of replying to a message: to the From address
> > and to the CC address.  I edited out yours this time, but I cannot
> > afford doing that every time; if you want not to receive direct
> > replies, you will have to configure your MUA to insert the necessary
> > headers (like Mail-followups-to etc.).
> 
> Well, my messages all contain both:
> 
> Mail-Copies-To: never
> Mail-Followup-To: emacs-devel@gnu.org
> 
> ..is there any other convention that I'm not yet following?

I think you need "Reply-to: emacs-devel@gnu.org" as well.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > For me a solution which doesn't work (efficiently enough) with
  > vc-region-history and vc-print-log is just pointless.

We could modify them to check whatever mechanism, when working with Git.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

>> For me a solution which doesn't work (efficiently enough) with
>> vc-region-history and vc-print-log is just pointless.
> We could modify them to check whatever mechanism, when working with Git.

In theory it's possible, yes.  But Someone™ needs to figure out how to
do it efficiently enough, and then Someone™ needs to actually do it.


        Stefan

Re: Is it time to drop ChangeLogs?

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 1880 bytes --]

>>>>> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> In theory it's possible, yes. But Someone™ needs to figure out how to do it
> efficiently enough, and then Someone™ needs to actually do it.

I'm willing work on tooling for a satisfactory mechanism to allow us to
eliminate the need for a separate ChangeLog file. I'm volunteering to be that
Someone™, and to afterwards maintain it and field bugs in its use. I would
also like to work directly with Eli and others to ensure it satisfies their
needs and concerns.

We know we need the following:

  1. That commit messages be written in ChangeLog form.  Regardless.

  2. A file-based mechanism to amend commit messages, by allowing substitute
     commit messages. I prefer a single file, or set of files, rather than one
     file per commit.

  3. That the mechanism in #2 work with merges, cherry-picking, rebasing, etc.

     (One way this could be achieved: base amendments on an SHA256 of the
     commit message+date being replaced, rather than the Git hash).

  4. A way to generate a proper ChangeLog from #1 and #2, without taking too
     long, and without requiring Emacs to be built before-hand.

  5. That "proper" history presented in #4 must also be available to other
     tools, such as `vc-region-history', etc.

  6. An easy way for contributors to craft ChangeLog entries for commit
     messages. I mention this because I saw issues mentioned with the current
     C-x 4 a.

I'll also take a look at what coreutils is already doing, but I share Eli's
reservation about a Perl-based process for correcting messages. I prefer
something dead simple, as a plain text file that anyone can edit and commit.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

>   3. That the mechanism in #2 work with merges, cherry-picking, rebasing, etc.

AFAIK if the mechanism is file-based it will automatically work with
merges (except for occasional conflicts to resolve, of course).

Personally, I think that handling cherry-picking and rebasing is
very secondary.


        Stefan

Re: Should we restore manually maintained ChangeLogs

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> If using "git replace" to fix a mistake in a log message is as simple as
> saying "git replace --edit SHA1" followed by "git push", then perhaps
> someone would like to try this for a while in a test repository, and if that
> gives good results, we could start using that.

I'll give it a try and see how well it fits with the other proposal I made
today. If 'git replace' is better/easier than maintaining our own separate
amendments file, then it sounds nice.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Should we restore manually maintained ChangeLogs

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 620 bytes --]

>>>>> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> Maybe we should move to a version control system which permits commit
>> messages to be amended. (Only half joking.)

> These seem to be in rather short supply, sadly. Any suggestion?

Please avoid questions like this. We are not going to be changing our version
control system, and so opening a discussion to discuss alternatives all over
again is both unnecessary and distracting.

Thank you,
-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Date: Sat, 12 Mar 2016 17:33:16 -0500
> Cc: emacs-devel@gnu.org
> 
> Personally, I think that handling cherry-picking and rebasing is
> very secondary.

We use the former for backporting changes from master.  As for the
latter, I think quite a few people who track the development and
contribute patches do that, so I think it's important for their
workflows to be undisturbed.

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 03/13/2016 05:53 PM, Eli Zaretskii wrote:
> As for the
> latter, I think quite a few people who track the development and
> contribute patches do that, so I think it's important for their
> workflows to be undisturbed.

If a person can rebase, they can fix the commit message while doing 
that. There's no point is supporting an alternative mechanism there.

Re: Should we restore manually maintained ChangeLogs

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Cc: Sven Axelsson <sven.axelsson@gmail.com>,  evgeny.panasyuk@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 12 Mar 2016 20:04:32 -0800
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > If using "git replace" to fix a mistake in a log message is as simple as
> > saying "git replace --edit SHA1" followed by "git push", then perhaps
> > someone would like to try this for a while in a test repository, and if that
> > gives good results, we could start using that.
> 
> I'll give it a try and see how well it fits with the other proposal I made
> today. If 'git replace' is better/easier than maintaining our own separate
> amendments file, then it sounds nice.

Thanks.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> Cc: rms@gnu.org, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 13 Mar 2016 18:05:47 +0200
> 
> On 03/13/2016 05:53 PM, Eli Zaretskii wrote:
> > As for the
> > latter, I think quite a few people who track the development and
> > contribute patches do that, so I think it's important for their
> > workflows to be undisturbed.
> 
> If a person can rebase, they can fix the commit message while doing 
> that. There's no point is supporting an alternative mechanism there.

No, I mean the messages corrected in commits by others.

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> We use the former for backporting changes from master.  As for the

I know we use cherry-picking, but I still think it's secondary to make
sure the "git log fixes" are propagated correctly in those cases.

I'm not opposed to it.  I just think it's of much lesser importance than
everything else we've considered in this thread.

> If a person can rebase, they can fix the commit message while doing
> that. There's no point is supporting an alternative mechanism there.

Exactly.


        Stefan

Re: Is it time to drop ChangeLogs?

[Yuri Khan]

On Sun, Mar 13, 2016 at 11:16 PM, Eli Zaretskii <eliz@gnu.org> wrote:

>> On 03/13/2016 05:53 PM, Eli Zaretskii wrote:
>> > As for the
>> > latter, I think quite a few people who track the development and
>> > contribute patches do that, so I think it's important for their
>> > workflows to be undisturbed.
>>
>> If a person can rebase, they can fix the commit message while doing
>> that. There's no point is supporting an alternative mechanism there.
>
> No, I mean the messages corrected in commits by others.

Commits by others do not typically occur on branches which are
socially accepted as rebasable. The only case I can think of is when
you cherry-pick someone’s commit into your feature branch while that
commit had been corrected. Is that the case you have in mind?

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sun, 13 Mar 2016 23:22:40 +0600
> Cc: Dmitry Gutov <dgutov@yandex.ru>, Emacs developers <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, "rms@gnu.org" <rms@gnu.org>
> 
> > No, I mean the messages corrected in commits by others.
> 
> Commits by others do not typically occur on branches which are
> socially accepted as rebasable. The only case I can think of is when
> you cherry-pick someone’s commit into your feature branch while that
> commit had been corrected. Is that the case you have in mind?

No, I mean "git pull --rebase" and its variants.  AFAIK, there's
nothing anti-social in such a workflow, when you have local unpushed
commits that you intend to push later.

Re: Is it time to drop ChangeLogs?

[Yuri Khan]

On Sun, Mar 13, 2016 at 11:30 PM, Eli Zaretskii <eliz@gnu.org> wrote:

>> > No, I mean the messages corrected in commits by others.
>>
>> Commits by others do not typically occur on branches which are
>> socially accepted as rebasable. The only case I can think of is when
>> you cherry-pick someone’s commit into your feature branch while that
>> commit had been corrected. Is that the case you have in mind?
>
> No, I mean "git pull --rebase" and its variants.  AFAIK, there's
> nothing anti-social in such a workflow, when you have local unpushed
> commits that you intend to push later.

If you do a git pull --rebase, then your branch now starts after the
other’s commit. I assume any corrections that apply to that commit on
master (or whatever other development branch it is on), also apply to
that same commit as seen from your feature branch.

Re: Is it time to drop ChangeLogs?

[David Caldwell]

[-- Attachment #1: Type: text/plain, Size: 2032 bytes --]

On 3/13/16 8:53 AM, Eli Zaretskii wrote:
>> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
>> Date: Sat, 12 Mar 2016 17:33:16 -0500
>> Cc: emacs-devel@gnu.org
>>
>> Personally, I think that handling cherry-picking and rebasing is
>> very secondary.
> 
> We use the former for backporting changes from master.

Cherry-picking a replaced commit works, as long as you use the
replacement SHA. This is the hash that shows up when you `git log`, so
it should just work without any thought on the part of the cherry-picker.

> As for the latter, I think quite a few people who track the
> development and contribute patches do that, so I think it's important
> for their workflows to be undisturbed.

I don't see why people who are tracking Emacs are rebasing commits which
have been pushed to the master or branch. Nonetheless, if you rebase and
one of the commits you are rebasing has been replaced, the replacement
gets used instead of the original.

In both of these cases, the new commit doesn't generate new replacement
commits. This means, for the use case we are thinking of here, that the
old (presumably incorrect) log message will be lost when you rebase or
cherry-pick.

-David


Here's my test log if you want to recreate my results, or poke around a
.git repo with replacements:

  mkdir -p /tmp/test/a
  cd /tmp/test/a
  git init
  echo a > a
  git add a
  git commit -m "a"
  echo b > a
  git add a
  git commit -m "changes"
  git replace --edit HEAD # changed the message in emacs to "b"
  git log # verified message was changed
  git checkout HEAD^
  git checkout -b c
  echo c > c
  git add c
  git commit -m "c"
  git log master
  git cherry-pick e3291a2f81bad7b24ba18e28e60378667ff78acd # The hash
that the previous `git log master` showed
  git log # verified cherry-picked commit has correct message
  find .git/refs/ # no new replacement refs created
  git checkout master
  git rebase c
  git log
  find .git/refs/ # no new replacement refs created



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3819 bytes --]

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sun, 13 Mar 2016 23:42:29 +0600
> Cc: Brief Busters <dgutov@yandex.ru>, Emacs developers <emacs-devel@gnu.org>, 
> 	Stefan Monnier <monnier@iro.umontreal.ca>, "rms@gnu.org" <rms@gnu.org>
> 
> On Sun, Mar 13, 2016 at 11:30 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> If you do a git pull --rebase, then your branch now starts after the
> other’s commit. I assume any corrections that apply to that commit on
> master (or whatever other development branch it is on), also apply to
> that same commit as seen from your feature branch.

It probably will be, yes.  But since the details of the mechanism are
not yet finalized, I think it's important to have this requirement on
the table, together with others.

Re: Is it time to drop ChangeLogs? (Was: bug#21998: Run 'make change-history' on release branch)

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

>> Which two languages? It should be easy to fix and I can volunteer to do
>> >that.
> Shell scripts and Makefiles are the two I thought of immediately.
> Maybe there are more, I don't know.  How about TeX and PS, for
> example?  We have a few of those.

.ps files are hardly ever edited any more, so they're low priority. .m4 files 
are more important.  I installed into master the attached patch, which adds 
support for shell, makefiles, m4, and other formats that git already supports 
well. Run './autogen.sh git' to use it.

> Btw, the best solution would be to have these tricks added to standard
> Git installation, or maybe to GNU Diff (which I think Git uses?).  Why
> should users need to reinvent the same wheel all the time?

shell scripts and makefiles are hard to automate in a portable way, as projects 
use different styles.  Perhaps that's why the Git developers have omitted those.

[-- Attachment #2: 0001-Improve-diff-hunk-headers-when-maintaining-Emacs.patch --]
[-- Type: text/x-diff, Size: 2740 bytes --]

From 571b427753f8e469f777fbc1f7d1cbe247f48840 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sun, 13 Mar 2016 12:20:01 -0700
Subject: [PATCH] Improve diff hunk headers when maintaining Emacs

* .gitattributes: Improve diff hunk header support for makefiles,
shell scripts, Ada, C, C++, Objective C, HTML, SHTML, XML, Java,
Perl, PHP, Python, Ruby, and TeX, all of which are used in Emacs
somewhere (sometimes just in test cases).
* autogen.sh: Add regexes for makefiles and shell scripts.
---
 .gitattributes | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 autogen.sh     |  5 +++++
 2 files changed, 51 insertions(+)

diff --git a/.gitattributes b/.gitattributes
index 5ccf9a5..13e58a8 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -47,6 +47,52 @@ doc/misc/texinfo.tex -whitespace=blank-at-eol
 *.tiff binary
 etc/e/eterm-color binary
 
+# Git's builtin diff hunk header styles.
+*.ada diff=ada
+*.[ch] diff=cpp
+*.cc diff=cpp
+*.cpp diff=cpp
+*.hh diff=cpp
+*.for diff=fortran
+*.html diff=html
+*.shtml diff=html
+*.xml diff=html
+*.java diff=java
+*.m diff=objc
+*.perl diff=perl
+*.pl diff=perl
+*.php diff=php
+*.py diff=python
+*.rb diff=ruby
+*.ruby diff=ruby
+*.tex diff=tex
+
 # Hooks for non-default diff hunk headers; see autogen.sh.
 *.el diff=elisp
+*.ac diff=m4
+*.m4 diff=m4
+*.mk diff=make
+*[Mm]akefile diff=make
+Makefile.in diff=make
+*.sh diff=shell
 *.texi diff=texinfo
+#
+# Diff hunk header special-case file names.
+admin/build-configs diff=perl
+admin/charsets/mapconv diff=shell
+admin/diff-tar-files diff=shell
+admin/make-emacs diff=perl
+admin/merge-gnulib diff=shell
+admin/merge-pkg-config diff=shell
+admin/quick-install-emacs diff=shell
+admin/update-copyright diff=shell
+admin/update_autogen diff=shell
+build-aux/git-hooks/commit-msg diff=shell
+build-aux/git-hooks/pre-commit diff=shell
+build-aux/gitlog-to-emacslog diff=shell
+build-aux/make-info-dir diff=shell
+build-aux/move-if-change diff=shell
+build-aux/msys-to-w32 diff=shell
+build-aux/update-subdirs diff=shell
+lib-src/rcs2log diff=shell
+/make-dist diff=shell
diff --git a/autogen.sh b/autogen.sh
index ac728cc..9042465 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -281,6 +281,11 @@ git_config ()
 
 git_config diff.elisp.xfuncname \
 	   '^\(def[^[:space:]]+[[:space:]]+([^()[:space:]]+)'
+git_config 'diff.m4.xfuncname' '^((m4_)?define|A._DEFUN(_ONCE)?)\([^),]*'
+git_config 'diff.make.xfuncname' \
+	   '^([$.[:alnum:]_].*:|[[:alnum:]_]+[[:space:]]*([*:+]?[:?]?|!?)=|define .*)'
+git_config 'diff.shell.xfuncname' \
+	   '^([[:space:]]*[[:alpha:]_][[:alnum:]_]*[[:space:]]*\(\)|[[:alpha:]_][[:alnum:]_]*=)'
 git_config diff.texinfo.xfuncname \
 	   '^@node[[:space:]]+([^,[:space:]][^,]+)'
 
-- 
2.5.0

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 12 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Date: Fri, 11 Mar 2016 19:32:49 -0800
>> 
>> On Mar 11 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> >> From: Nikolaus Rath <Nikolaus@rath.org>
>> >> Date: Fri, 11 Mar 2016 12:08:09 -0800
>> >> 
>> >> Please do not Cc me on replies, I am reading the list.
>> >
>> > That's the standard way of replying to a message: to the From address
>> > and to the CC address.  I edited out yours this time, but I cannot
>> > afford doing that every time; if you want not to receive direct
>> > replies, you will have to configure your MUA to insert the necessary
>> > headers (like Mail-followups-to etc.).
>> 
>> Well, my messages all contain both:
>> 
>> Mail-Copies-To: never
>> Mail-Followup-To: emacs-devel@gnu.org
>> 
>> ..is there any other convention that I'm not yet following?
>
> I think you need "Reply-to: emacs-devel@gnu.org" as well.

That's not a good idea, because it robs any reader of the choice of
sending a personal or public reply.

See http://cr.yp.to/proto/replyto.html for a more detailed discussion.


Best,
-Nikolaus


-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Cc: emacs-devel@gnu.org
> Date: Sun, 13 Mar 2016 14:43:27 -0700
> 
> > I think you need "Reply-to: emacs-devel@gnu.org" as well.
> 
> That's not a good idea, because it robs any reader of the choice of
> sending a personal or public reply.
> 
> See http://cr.yp.to/proto/replyto.html for a more detailed discussion.

The problems mentioned there don't exist in our case.

Re: Is it time to drop ChangeLogs?

[Nikolaus Rath]

On Mar 14 2016, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Cc: emacs-devel@gnu.org
>> Date: Sun, 13 Mar 2016 14:43:27 -0700
>> 
>> > I think you need "Reply-to: emacs-devel@gnu.org" as well.
>> 
>> That's not a good idea, because it robs any reader of the choice of
>> sending a personal or public reply.
>> 
>> See http://cr.yp.to/proto/replyto.html for a more detailed discussion.
>
> The problems mentioned there don't exist in our case.

Why not?

If I send Reply-To to my address, but the reader wants to reply to the
list, he/she has to manually edit the recipient address. If I sent
Reply-To to the list, the reader has to manually edit the recipient
address if he/she wants to a reply just to me.

The Mail-Copies-To header solves this properly.

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 14 Mar 2016 07:56:29 -0700
> 
> If I send Reply-To to my address, but the reader wants to reply to the
> list, he/she has to manually edit the recipient address. If I sent
> Reply-To to the list, the reader has to manually edit the recipient
> address if he/she wants to a reply just to me.
> 
> The Mail-Copies-To header solves this properly.

IME, you are assuming too much about the MUA on the other side --
quite a few will behave the same with Mail-Copies-To as with Reply-To,
or even ignore the former entirely.

Anyway, if Reply-To doesn't work for you, I'm out of ideas (and I'm
not an expert on this to begin with, just a long-time user).

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Sun, 06 Mar 2016 13:52:04 -0800 John Wiegley <jwiegley@gmail.com> wrote: 

JW> I'd like to open this up to discussion on emacs-devel, so that we hear from
JW> our other developers. What do you all think about ChangeLogs, and their value
JW> to you in your work on Emacs?

Currently, I think ChangeLogs are a barrier to contribution. The vast
majority of other software projects don't use them.

But Emacs doesn't have a pull request contribution system, which makes
it hard to review things before they go in, so contributors must know
and follow the right format at all times. It's a pain.

So I would suggest moving to a pull request system, where code review
from a second contributor is required to merge any non-trivial code
(exceptions should be granted based on years contributing to Emacs).
That also gives *everyone* the opportunity to comment on the code before
it's merged, instead of post-facto. Clearly services such as Github and
BitBucket and many others have been offering this functionality for a
while with good results.

A big advantage of pull requests is that they can group commits, so each
commit doesn't need the level of detail it does today, and so the
evolution of the work is visible to a reviewer.

Then ChangeLogs become simply documentation for the merged code,
together with actual docs and other notes that are needed. The pull
request system can later provide *everything* that a ChangeLog could,
and more (such as better searching and cross-referencing) so in the long
term the ChangeLog can go away.

Ted

Re: Is it time to drop ChangeLogs?

[Robert Weiner]

On Wed, Jul 6, 2016 at 10:20 AM, Ted Zlatanov <tzz@lifelogs.com> wrote:
> A big advantage of pull requests is that they can group commits, so each
> commit doesn't need the level of detail it does today, and so the
> evolution of the work is visible to a reviewer.
>
> Then ChangeLogs become simply documentation for the merged code,
> together with actual docs and other notes that are needed. The pull
> request system can later provide *everything* that a ChangeLog could,
> and more (such as better searching and cross-referencing) so in the long
> term the ChangeLog can go away.

ChangeLogs can group commits of code changes quite simply (just group
the change comments together) and have the advantage that no special
tools are needed to review them, just browse them linearly or search
for changes within a specific release.  I would be in favor of
grouping changes spanning multiple days together and using the final
date rather than having them split into separate entries by day.

I agree that having to provide secondary commentary in a commit log is
a pain and duplicative of effort but I don't have any suggestion for
solution on that.

Bob

Re: Is it time to drop ChangeLogs?

[Noam Postavsky]

On Wed, Jul 6, 2016 at 11:08 AM, Robert Weiner <rsw@gnu.org> wrote:
> I agree that having to provide secondary commentary in a commit log is
> a pain and duplicative of effort but I don't have any suggestion for
> solution on that.

It's already been solved a while ago, now ChangeLogs are
auto-generated from the commit message.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Wed, 06 Jul 2016 10:20:07 -0400
> 
> Currently, I think ChangeLogs are a barrier to contribution. The vast
> majority of other software projects don't use them.

We don't have ChangeLogs anymore.  So I'm not sure what is this about,
please clarify.

> So I would suggest moving to a pull request system, where code review
> from a second contributor is required to merge any non-trivial code
> (exceptions should be granted based on years contributing to Emacs).
> That also gives *everyone* the opportunity to comment on the code before
> it's merged, instead of post-facto. Clearly services such as Github and
> BitBucket and many others have been offering this functionality for a
> while with good results.

Everyone can comment on patches now, but almost no one does.  How can
we be sure this arrangement will work?  It could be a tremendous
impediment to development if everyone must wait for comments that
never come.

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Wed, 06 Jul 2016 18:36:38 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Wed, 06 Jul 2016 10:20:07 -0400
>> 
>> Currently, I think ChangeLogs are a barrier to contribution. The vast
>> majority of other software projects don't use them.

EZ> We don't have ChangeLogs anymore.  So I'm not sure what is this about,
EZ> please clarify.

ChangeLog-style formatting. Sorry. I was talking about the benefits and
reasons for having that specific format and the problems from when the
format is not followed (bad formatting, misspellings, etc.)

I was trying to say that using a pull request system would reduce the
need for writing every commit message as if it was going into a
ChangeLog, and would give better assurance that mistakes are caught
before they get merged. It has many other benefits (code review,
community feedback) but I was commenting specifically for ChangeLogs,
following up to an old message. I hope that's more clear.

Ted

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Wed, 06 Jul 2016 12:06:28 -0400
> 
> ChangeLog-style formatting. Sorry. I was talking about the benefits and
> reasons for having that specific format and the problems from when the
> format is not followed (bad formatting, misspellings, etc.)

What format do you suggest instead?

The advantage of the current formatting is that we have Emacs features
which support it very well.  Any other formatting will have to grow a
similar support first.  And then we all need to re-learn.

IOW, isn't this suggestion going to favor casual contributors over
those who push commits day in and day out?

> I was trying to say that using a pull request system would reduce the
> need for writing every commit message as if it was going into a
> ChangeLog, and would give better assurance that mistakes are caught
> before they get merged. It has many other benefits (code review,
> community feedback) but I was commenting specifically for ChangeLogs,
> following up to an old message.

As I wrote, I have doubts that we will be able to sustain such a
system.  Maybe I'm wrong.

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Wed, 06 Jul 2016 19:36:30 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Wed, 06 Jul 2016 12:06:28 -0400
>> 
>> ChangeLog-style formatting. Sorry. I was talking about the benefits and
>> reasons for having that specific format and the problems from when the
>> format is not followed (bad formatting, misspellings, etc.)

EZ> What format do you suggest instead?

"Summary line.

Details..."

Where the Details can be in the current or another format or whatever
works for the contributor. The per-file summary would be available from
the pull request system, regardless of the formatting of the commit message.

EZ> The advantage of the current formatting is that we have Emacs features
EZ> which support it very well.  Any other formatting will have to grow a
EZ> similar support first.  And then we all need to re-learn.

Good points. I'm trying to say all the support code can move to the pull
request system, so it won't go away, and the transition can be gradual.

EZ> IOW, isn't this suggestion going to favor casual contributors over
EZ> those who push commits day in and day out?

Yes, exceptions should be made for those regular contributors. But you
know that some features merit discussion, even if a very senior
contributor wants them.

>> I was trying to say that using a pull request system would reduce the
>> need for writing every commit message as if it was going into a
>> ChangeLog, and would give better assurance that mistakes are caught
>> before they get merged. It has many other benefits (code review,
>> community feedback) but I was commenting specifically for ChangeLogs,
>> following up to an old message.

EZ> As I wrote, I have doubts that we will be able to sustain such a
EZ> system.  Maybe I'm wrong.

If I understand correctly, your concern is about logistics rather than
about the fundamental benefits of such a system. I share your concerns
and think the risk is worth the benefit.

Ted

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Wed, 06 Jul 2016 13:03:07 -0400
> 
> On Wed, 06 Jul 2016 19:36:30 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 
> 
> EZ> What format do you suggest instead?
> 
> "Summary line.
> 
> Details..."
> 
> Where the Details can be in the current or another format or whatever
> works for the contributor. The per-file summary would be available from
> the pull request system, regardless of the formatting of the commit message.

That's already so.  The number of times I had to reformat log messages
for casual contributors is too large to remember.  You just suggest to
codify that, so they won't feel obliged to even try to format the
messages as we want them.

> EZ> The advantage of the current formatting is that we have Emacs features
> EZ> which support it very well.  Any other formatting will have to grow a
> EZ> similar support first.  And then we all need to re-learn.
> 
> Good points. I'm trying to say all the support code can move to the pull
> request system, so it won't go away, and the transition can be gradual.

Sorry, I don't understand how this will work.  Can you explain?  Let's
say I received a pull request, and the commit's log message needs
reformatting.  What next?

> EZ> IOW, isn't this suggestion going to favor casual contributors over
> EZ> those who push commits day in and day out?
> 
> Yes, exceptions should be made for those regular contributors. But you
> know that some features merit discussion, even if a very senior
> contributor wants them.

Our current practice is to discuss when the commit is already pushed.

> If I understand correctly, your concern is about logistics rather than
> about the fundamental benefits of such a system. I share your concerns
> and think the risk is worth the benefit.

My concern is about the manpower.  If we don't have enough, this is
going to be an exercise in futility.

So I'd suggest first to have enough patch reviewers step forward and
volunteer, before we start thinking about such a process seriously.

Re: Is it time to drop ChangeLogs?

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 1189 bytes --]

Hi Ted,

I may be a strange one here, but I like the ChangeLog format for commit
messages. It's not unreasonable to ask a submitter to take some care and pride
in their description of the changes they've made -- after all, they've spent a
fair bit of time preparing the change itself.

A little discipline never hurt anyone, and pays dividends in the long run by
making commit messages easier to scan through and read for us. Even if we
didn't have the ChangeLog format, I'm sure we'd want *some* kind of consistent
uniformity, so why not settle on the one we've had for decades?

I've always been against the ChangeLog file, but we've moved past that, even
if there are still some technical challenges to resolve.

If writing a ChangeLog entry to describe a change increases the time spent
crafting a patch by 10%, and if this loses a contributor who would have
otherwise sent us code, I'm forced to wonder in what other ways have they've
been hasty, such that working on the commit message was too much for them.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

On 07/06/2016 06:06 PM, Ted Zlatanov wrote:
> using a pull request system would reduce the
> need for writing every commit message as if it was going into a
> ChangeLog
I don't see why; if we're going to keep the current format, pull 
requests should use the format too.

I have the sense that pull requests work better in projects with a large 
number of occasional committers and a small number of full-time 
developers who triage and review. Emacs development doesn't work that 
way: among other things, there are no full-time developers, and we don't 
have enough reviewer time. So it may not be a good fit for the 
pull-request model. (It might make sense to change Emacs's development 
model but that's a larger topic....)

Re: Is it time to drop ChangeLogs?

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 228 bytes --]

On 2016-07-06 13:23, Eli Zaretskii wrote:
> So I'd suggest first to have enough patch reviewers step forward and
> volunteer, before we start thinking about such a process seriously.

I'm happy to look at lisp patches :)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Wed, 6 Jul 2016 13:44:46 -0400
> 
> I'm happy to look at lisp patches :)

Thank you, and please go ahead.

Re: Is it time to drop ChangeLogs?

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 268 bytes --]

On 2016-07-06 14:32, Eli Zaretskii wrote:
>> From: Clément Pit--Claudel <clement.pit@gmail.com>
>> > Date: Wed, 6 Jul 2016 13:44:46 -0400
>> > 
>> > I'm happy to look at lisp patches :)
> Thank you, and please go ahead.

Should I subscribe to the bugs list?


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Wed, 06 Jul 2016 10:39:35 -0700 John Wiegley <jwiegley@gmail.com> wrote: 

JW> I may be a strange one here, but I like the ChangeLog format for commit
JW> messages. It's not unreasonable to ask a submitter to take some care and pride
JW> in their description of the changes they've made -- after all, they've spent a
JW> fair bit of time preparing the change itself.

All right, that's very reasonable.

JW> If writing a ChangeLog entry to describe a change increases the time spent
JW> crafting a patch by 10%, and if this loses a contributor who would have
JW> otherwise sent us code, I'm forced to wonder in what other ways have they've
JW> been hasty, such that working on the commit message was too much for them.

I disagree with this. Processes such as the one here are learned
behavior, not something that identifies good vs. bad programmers.

On Wed, 06 Jul 2016 20:23:32 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> That's already so.  The number of times I had to reformat log messages
EZ> for casual contributors is too large to remember.  You just suggest to
EZ> codify that, so they won't feel obliged to even try to format the
EZ> messages as we want them.
...
EZ> Sorry, I don't understand how this will work.  Can you explain?  Let's
EZ> say I received a pull request, and the commit's log message needs
EZ> reformatting.  What next?

The code doesn't get merged until the commit is up to our standards. So
even if the ChangeLog format doesn't change, this is still a good way to
prevent bad code or bad commit messages from making it into Emacs. The
fixes are up to the contributor; the reviewer will almost never have to
rewrite things on their own.

EZ> My concern is about the manpower.  If we don't have enough, this is
EZ> going to be an exercise in futility.

EZ> So I'd suggest first to have enough patch reviewers step forward and
EZ> volunteer, before we start thinking about such a process seriously.

I think we already do this to some degree in emacs-bugs, but without a
process (so casual contributors often don't know where to start).

I agree manpower is a concern. Perhaps a pull request process would
actually take work off our shoulders by simplifying the steps so more
people can be reviewers.

On Wed, 6 Jul 2016 19:41:15 +0200 Paul Eggert <eggert@cs.ucla.edu> wrote: 

PE> I have the sense that pull requests work better in projects with a large number
PE> of occasional committers and a small number of full-time developers who triage
PE> and review. Emacs development doesn't work that way: among other things, there
PE> are no full-time developers, and we don't have enough reviewer time. So it may
PE> not be a good fit for the pull-request model. (It might make sense to change
PE> Emacs's development model but that's a larger topic....)

Good points; Eli noted that as well. I don't have proof one way or the
other, but my feeling is that it would be a change for the better.

Ted

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Wed, 6 Jul 2016 15:16:31 -0400
> 
> >> > I'm happy to look at lisp patches :)
> > Thank you, and please go ahead.
> 
> Should I subscribe to the bugs list?

Yes, please.

Re: Is it time to drop ChangeLogs?

[John Wiegley]

>>>>> Ted Zlatanov <tzz@lifelogs.com> writes:

JW> If writing a ChangeLog entry to describe a change increases the time spent
JW> crafting a patch by 10%, and if this loses a contributor who would have
JW> otherwise sent us code, I'm forced to wonder in what other ways have
JW> they've been hasty, such that working on the commit message was too much
JW> for them.

> I disagree with this. Processes such as the one here are learned behavior,
> not something that identifies good vs. bad programmers.

You are right, of course, Ted, I'll remove that from my thinking as an
indicator of sloppiness. :)

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

On 07/07/2016 04:44 AM, John Wiegley wrote:
>> I disagree with this. Processes such as the one here are learned behavior,
>> >not something that identifies good vs. bad programmers.
> You are right, of course, Ted

Yes and no. Good software engineers are adept at software processes, and 
this is mainly what distinguishes them from mere good programmers. If we 
want high-quality software then we need software engineering, not mere 
programming.

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 07/06/2016 06:06 PM, Ted Zlatanov wrote:
>> using a pull request system would reduce the
>> need for writing every commit message as if it was going into a
>> ChangeLog
> I don't see why; if we're going to keep the current format, pull requests
> should use the format too.
>
> I have the sense that pull requests work better in projects with a large
> number of occasional committers and a small number of full-time developers who
> triage and review. Emacs development doesn't work that way: among other
> things, there are no full-time developers, and we don't have enough reviewer
> time. So it may not be a good fit for the pull-request model. (It might make
> sense to change Emacs's development model but that's a larger topic....)

I think that this is not true. I have recently sent patches into, for
example, ensime which is a fairly small development. Also, Cask uses an
always PR model which means that all changes get looked at by at least
two people.

Where PRs work better over direct commits is when ever someone wants
comments and feedback. There are two main reasons for this. One is where
commits need to be checked by someone else before going in; the emacs-25
branch is in this state at the moment.

The other is where someone wants feedback on their work, because they
are new, or because they are fiddling with parts of Emacs which they
understand poorly. I've found this extremely useful, for instance, with
my changes to the undo system, which involved extensive communications
mostly with Stefan. With Cask, also, I have found this very useful,
since I've had to make changes to in three different languages.

At the moment, we have a poor workflow for supporting this.

 - I can push a branch onto the Emacs git. But, this is not squashable,
   so the final state before the merge is hard to do
 - There is no system for queuing pull requests, so sometimes things get
   forgotten
 - I can send patches, but this is clunk compared to pushing a branch
   within version control.
 - There is no system for viewing feedback about an individual patch.
 - There is no system for adding inline comments to patches

Installing something like gerrit or kallithea would be nice (I have no
direct experience of using either, but they are similar to other
systems). However, this would be considerable work.

Perhaps, as a half way house, we could use the resources that we have.
PRs could go to the bug reporting system. This will, at least, keep all
the conversations in one place. If we can tag these with "has patch"
here as well, it will give an queue also. We would still need to do
something about the Emacs git, in terms of squashability; in practice,
this would probably require something like gitolite as allowing non-FF
pushes on all branches would be a bad thing.

This would not give a nice web interface, nor inline comments, but it's
a start.

Phil

Re: Is it time to drop ChangeLogs?

[Noam Postavsky]

On Thu, Jul 7, 2016 at 7:29 AM, Phillip Lord <phillip.lord@russet.org.uk> wrote:
> At the moment, we have a poor workflow for supporting this.
>
>  - I can push a branch onto the Emacs git. But, this is not squashable,
>    so the final state before the merge is hard to do
>  - There is no system for queuing pull requests, so sometimes things get
>    forgotten
>  - I can send patches, but this is clunk compared to pushing a branch
>    within version control.
>  - There is no system for viewing feedback about an individual patch.
>  - There is no system for adding inline comments to patches

So maybe we just need some more support in vc-git to make sending
patches less clunky? I've been sending patches to bug threads, and
often getting useful feedback on them (and since it's by email, the
comments can easily be inline). Personally, I don't find it more
clunky than pushing to a branch, and then opening a PR in a web
browser. Yes, some patches are forgotten, but I don't see how a PR
"system" makes that less likely to happen, e.g., cask has a bunch of
open PRs sitting around: https://github.com/cask/cask/pulls.

> Perhaps, as a half way house, we could use the resources that we have.
> PRs could go to the bug reporting system. This will, at least, keep all
> the conversations in one place. If we can tag these with "has patch"
> here as well, it will give an queue also.

I thought that was the current system. Here is the queue:
http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs;include=tags%3Apatch;bug-rev=on

Re: Is it time to drop ChangeLogs?

[Alan Mackenzie]

Hello, Ted.

On Wed, Jul 06, 2016 at 10:20:07AM -0400, Ted Zlatanov wrote:
> On Sun, 06 Mar 2016 13:52:04 -0800 John Wiegley <jwiegley@gmail.com> wrote: 

[ .... ]

> But Emacs doesn't have a pull request contribution system, which makes
> it hard to review things before they go in, so contributors must know
> and follow the right format at all times. It's a pain.

> So I would suggest moving to a pull request system, where code review
> from a second contributor is required to merge any non-trivial code
> (exceptions should be granted based on years contributing to Emacs).
> That also gives *everyone* the opportunity to comment on the code before
> it's merged, instead of post-facto. Clearly services such as Github and
> BitBucket and many others have been offering this functionality for a
> while with good results.

> A big advantage of pull requests is that they can group commits, so each
> commit doesn't need the level of detail it does today, and so the
> evolution of the work is visible to a reviewer.

I don't know exactly what is meant by "pull request" and "pull request
system".  I don't think they are established terms.

The term seems to imply that instead of a contributor pushing a change
from his machine to a central repository, some specially authorised
authority would pull the change from the contributor's machine.  This
would seem to imply every contributor needing to set up an scp daemon on
his local machine, which doesn't feel like a Good Thing.

Please explain "pull request\( system\)?" more precisely.

Thanks!

> Then ChangeLogs become simply documentation for the merged code,
> together with actual docs and other notes that are needed. The pull
> request system can later provide *everything* that a ChangeLog could,
> and more (such as better searching and cross-referencing) so in the long
> term the ChangeLog can go away.

> Ted

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Noam Postavsky <npostavs@users.sourceforge.net> writes:

> On Thu, Jul 7, 2016 at 7:29 AM, Phillip Lord <phillip.lord@russet.org.uk> wrote:
>> At the moment, we have a poor workflow for supporting this.
>>
>>  - I can push a branch onto the Emacs git. But, this is not squashable,
>>    so the final state before the merge is hard to do
>>  - There is no system for queuing pull requests, so sometimes things get
>>    forgotten
>>  - I can send patches, but this is clunk compared to pushing a branch
>>    within version control.
>>  - There is no system for viewing feedback about an individual patch.
>>  - There is no system for adding inline comments to patches
>
> So maybe we just need some more support in vc-git to make sending
> patches less clunky?

Yes, that would help, assuming that it's not there already. But, again,
I still feel that patches are fairly "old school".


> I've been sending patches to bug threads, and often getting useful
> feedback on them (and since it's by email, the comments can easily be
> inline). Personally, I don't find it more clunky than pushing to a
> branch, and then opening a PR in a web browser.

This is true for the first patch, but not true for additional commits.


> Yes, some patches are forgotten, but I don't see how a PR "system"
> makes that less likely to happen, e.g., cask has a bunch of open PRs
> sitting around: https://github.com/cask/cask/pulls.

I think you have just demonstrated my point. You found out all the
outstanding ones also.


>> Perhaps, as a half way house, we could use the resources that we have.
>> PRs could go to the bug reporting system. This will, at least, keep all
>> the conversations in one place. If we can tag these with "has patch"
>> here as well, it will give an queue also.
>
> I thought that was the current system. Here is the queue:
> http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs;include=tags%3Apatch;bug-rev=on

Again, also my point.

Phil

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Ted.
>
> On Wed, Jul 06, 2016 at 10:20:07AM -0400, Ted Zlatanov wrote:
>> On Sun, 06 Mar 2016 13:52:04 -0800 John Wiegley <jwiegley@gmail.com> wrote: 
>
> [ .... ]
>
>> But Emacs doesn't have a pull request contribution system, which makes
>> it hard to review things before they go in, so contributors must know
>> and follow the right format at all times. It's a pain.
>
>> So I would suggest moving to a pull request system, where code review
>> from a second contributor is required to merge any non-trivial code
>> (exceptions should be granted based on years contributing to Emacs).
>> That also gives *everyone* the opportunity to comment on the code before
>> it's merged, instead of post-facto. Clearly services such as Github and
>> BitBucket and many others have been offering this functionality for a
>> while with good results.
>
>> A big advantage of pull requests is that they can group commits, so each
>> commit doesn't need the level of detail it does today, and so the
>> evolution of the work is visible to a reviewer.
>
> I don't know exactly what is meant by "pull request" and "pull request
> system".  I don't think they are established terms.

https://en.wikipedia.org/wiki/Distributed_version_control#Pull_requests

You send an email saying "here are the changes that I want to
incorporate".


> The term seems to imply that instead of a contributor pushing a change
> from his machine to a central repository, some specially authorised
> authority would pull the change from the contributor's machine.  This
> would seem to imply every contributor needing to set up an scp daemon on
> his local machine, which doesn't feel like a Good Thing.

On *some* machine, yes. That can be their own server, or a hosted git
repository, or a branch on the Emacs git repository.


> Please explain "pull request\( system\)?" more precisely.

https://en.wikipedia.org/wiki/List_of_tools_for_code_review

It keeps a list of all the pull requests coming in. They provide things
like inline comments over diffs, threaded conversation, integration with
continuous integration. Many of them, once the PR is complete, will
automate the merge to master.

Phil

Re: Is it time to drop ChangeLogs?

[Andreas Schwab]

phillip.lord@russet.org.uk (Phillip Lord) writes:

> I still feel that patches are fairly "old school".

linux-kernel@ is full of patches.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: Is it time to drop ChangeLogs?

[Noam Postavsky]

On Thu, Jul 7, 2016 at 8:55 AM, Phillip Lord <phillip.lord@russet.org.uk> wrote:
>
>> I've been sending patches to bug threads, and often getting useful
>> feedback on them (and since it's by email, the comments can easily be
>> inline). Personally, I don't find it more clunky than pushing to a
>> branch, and then opening a PR in a web browser.
>
> This is true for the first patch, but not true for additional commits.

Yeah, you're right about that. Most of my Emacs patches so far have
been small enough that it didn't really bother me too much.

>
>
>> Yes, some patches are forgotten, but I don't see how a PR "system"
>> makes that less likely to happen, e.g., cask has a bunch of open PRs
>> sitting around: https://github.com/cask/cask/pulls.
>
> I think you have just demonstrated my point. You found out all the
> outstanding ones also.
>
>
>>> Perhaps, as a half way house, we could use the resources that we have.
>>> PRs could go to the bug reporting system. This will, at least, keep all
>>> the conversations in one place. If we can tag these with "has patch"
>>> here as well, it will give an queue also.
>>
>> I thought that was the current system. Here is the queue:
>> http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs;include=tags%3Apatch;bug-rev=on
>
> Again, also my point.

If your point is that there are more open patches in Emacs than cask,
IMO that's mostly because Emacs is a larger and older project.

>
> Phil

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Thu, 7 Jul 2016 09:31:55 +0200 Paul Eggert <eggert@cs.ucla.edu> wrote: 

PE> On 07/07/2016 04:44 AM, John Wiegley wrote:
>>> I disagree with this. Processes such as the one here are learned behavior,
>>> >not something that identifies good vs. bad programmers.
>> You are right, of course, Ted

PE> Yes and no. Good software engineers are adept at software processes, and this is
PE> mainly what distinguishes them from mere good programmers. If we want
PE> high-quality software then we need software engineering, not mere
PE> programming.

You're right, Paul. My point, however, is that writing ChangeLog-style
commits is definitely not something you'll learn in school or in
industry, and we shouldn't turn it into a shibboleth by itself.

Working with GNU software is fun, but it's a tiny community compared to
the world-wide developer community. It's hard to grow it if contributors
are judged on the ChangeLog-style formatting on their first contributions.

On Thu, 07 Jul 2016 12:29:23 +0100 phillip.lord@russet.org.uk (Phillip Lord) wrote: 

PL> Where PRs work better over direct commits is when ever someone wants
PL> comments and feedback. There are two main reasons for this. One is where
PL> commits need to be checked by someone else before going in; the emacs-25
PL> branch is in this state at the moment.

PL> The other is where someone wants feedback on their work, because they
PL> are new, or because they are fiddling with parts of Emacs which they
PL> understand poorly.

Exactly! What I'm suggesting would benefit those two cases the most.

PL> Installing something like gerrit or kallithea would be nice (I have no
PL> direct experience of using either, but they are similar to other
PL> systems). However, this would be considerable work.

PL> Perhaps, as a half way house, we could use the resources that we have.
PL> PRs could go to the bug reporting system. This will, at least, keep all
PL> the conversations in one place. If we can tag these with "has patch"
PL> here as well, it will give an queue also. We would still need to do
PL> something about the Emacs git, in terms of squashability; in practice,
PL> this would probably require something like gitolite as allowing non-FF
PL> pushes on all branches would be a bad thing.

PL> This would not give a nice web interface, nor inline comments, but it's
PL> a start.

So maybe the workflow can be:

1) propose a patch on emacs-bug with tag `has patch'. A Git branch
pointer is also acceptable?

2) a reviewer comments; patch/branch is reworked

3) reviewer signs off and commits patch / merges branch

This is similar to how the Git project does it. It can be a bit chaotic,
but seems to work for their scale (which is similar to Emacs' scale). It
won't require new software.

As Noah mentioned, we already have (1) with
http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs;include=tags%3Apatch;bug-rev=on
but (2) and (3) are not "expected" or in any way documented. So the
contributor docs would be adjusted to mention these steps. That's a
pretty small amount of voluntary effort, and no one has to follow these
steps that doesn't want to. I sincerely hate the current bug tracker,
but am willing to work with it if that's what it takes to get this
moving. As a first attempt, I'll post my gnus-cloud work when it's ready
for review.

On Thu, 7 Jul 2016 08:43:30 -0400 Noam Postavsky <npostavs@users.sourceforge.net> wrote: 

NP> So maybe we just need some more support in vc-git to make sending
NP> patches less clunky? I've been sending patches to bug threads, and
NP> often getting useful feedback on them (and since it's by email, the
NP> comments can easily be inline). Personally, I don't find it more
NP> clunky than pushing to a branch, and then opening a PR in a web
NP> browser.

For a single patch, e-mail works. For multiple commits, rebasing,
adjusting, it can be overwhelming.

NP> Yes, some patches are forgotten, but I don't see how a PR
NP> "system" makes that less likely to happen, e.g., cask has a bunch of
NP> open PRs sitting around: https://github.com/cask/cask/pulls.

A PR system makes it easier to find forgotten submissions. Usually it
can assign a reviewer, who then should get regular reminders. But, of
course, it won't fix lack of manpower or lack of interest. It's just a
virtual clerk.

On Thu, 7 Jul 2016 12:46:07 +0000 Alan Mackenzie <acm@muc.de> wrote: 

AM> I don't know exactly what is meant by "pull request" and "pull request
AM> system".  I don't think they are established terms.

AM> The term seems to imply that instead of a contributor pushing a change
AM> from his machine to a central repository, some specially authorised
AM> authority would pull the change from the contributor's machine.  This
AM> would seem to imply every contributor needing to set up an scp daemon on
AM> his local machine, which doesn't feel like a Good Thing.

They are well known today, and have existed (in the form of patch
reviews) for a long time. Wikipedia doesn't have much, but
https://help.github.com/articles/using-pull-requests/ has a pretty good
overview. It shows the Github UI, but the steps are the same without it.

The Git book has some good workflow suggestions as well https://git-scm.com/book/ch5-2.html

Ted

Re: Is it time to drop ChangeLogs?

[Alan Mackenzie]

Hello, Phillip.

On Thu, Jul 07, 2016 at 02:01:34PM +0100, Phillip Lord wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > I don't know exactly what is meant by "pull request" and "pull request
> > system".  I don't think they are established terms.

> https://en.wikipedia.org/wiki/Distributed_version_control#Pull_requests

> You send an email saying "here are the changes that I want to
> incorporate".

Thanks.

> > The term seems to imply that instead of a contributor pushing a change
> > from his machine to a central repository, some specially authorised
> > authority would pull the change from the contributor's machine.  This
> > would seem to imply every contributor needing to set up an scp daemon on
> > his local machine, which doesn't feel like a Good Thing.

> On *some* machine, yes. That can be their own server, or a hosted git
> repository, or a branch on the Emacs git repository.

The only one of these usable by me would be the last one.  I can foresee
this branch would be open to anybody to commit anything, and could
quickly fill up with questionable changes.  Anybody using this method
would need to maintain their own copy of this "pull" branch.  This could
lead to quite a few logistical problems, could it not?

Also, the email containing the patch is not the source for what gets
merged?  This seems inefficient.

> > Please explain "pull request\( system\)?" more precisely.

> https://en.wikipedia.org/wiki/List_of_tools_for_code_review

> It keeps a list of all the pull requests coming in. They provide things
> like inline comments over diffs, threaded conversation, integration with
> continuous integration. Many of them, once the PR is complete, will
> automate the merge to master.

OK.

> Phil

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Is it time to drop ChangeLogs?

[Paul Eggert]

On 07/07/2016 03:18 PM, Ted Zlatanov wrote:
> writing ChangeLog-style
> commits is definitely not something you'll learn in school or in industry

Not true. For example, the following school assignment has students 
writing ChangeLog-format entries:

Change management. UCLA Computer Science 35L,Software Construction 
Laboratory, Spring 2016, Assignment 4. 
http://web.cs.ucla.edu/classes/winter16/cs35L/assign/assign4.html

More generally, any good undergraduate software-engineering curriculum 
should cover change management and should have exercises where students 
describe, review and commit patches, merge branches, etc. There should 
be some well-defined procedure that students actually do (as opposed to 
merely reading about it).This stuff is basic nowadays.

Of course we can't expect every new Emacs developer to know ChangeLog 
format, but that'd be true of any format. It's not too much to expect 
people to look at recent commits and use a similar format (this is 
standard practice pretty much everywhere). The format from a new user 
doesn't have to be perfect, after all.

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Thu, 7 Jul 2016 16:26:31 +0200 Paul Eggert <eggert@cs.ucla.edu> wrote: 

PE> On 07/07/2016 03:18 PM, Ted Zlatanov wrote:
>> writing ChangeLog-style
>> commits is definitely not something you'll learn in school or in industry

PE> Not true. For example, the following school assignment has students writing
PE> ChangeLog-format entries:

PE> Change management. UCLA Computer Science 35L,Software Construction Laboratory,
PE> Spring 2016, Assignment 4.
PE> http://web.cs.ucla.edu/classes/winter16/cs35L/assign/assign4.html

That is pretty cool :) I hope they can handle a few million students per semester!

Ted

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Date: Thu, 07 Jul 2016 12:29:23 +0100
> Cc: emacs-devel@gnu.org
> 
>  - I can push a branch onto the Emacs git. But, this is not squashable,
>    so the final state before the merge is hard to do
>  - There is no system for queuing pull requests, so sometimes things get
>    forgotten
>  - I can send patches, but this is clunk compared to pushing a branch
>    within version control.
>  - There is no system for viewing feedback about an individual patch.
>  - There is no system for adding inline comments to patches

This will continue to be like that as long as we don't have enough
core developers who understand enough of Emacs and can be trusted to
approve changes.  When (if) we do have enough of them, patches
submitted to our 2 lists will be reviewed in time, and most of the
above will just go away without a trace.

Starting a PR system in the current conditions is just going to
_increase_ the workload of the few overloaded people.  E.g., I don't
want to push changes for anyone else, ever: it takes too much of the
little time I have to work on Emacs.  I want this burden off-loaded to
someone else.  Guess what? patches that I said were okay can rot for
many days without anyone with write access doing anything.  I can only
conclude they all are as busy as I am or busier.  As long as this is
the situation, how could a PR system help, when it requires me to do
_more_ than I have to now?

> Installing something like gerrit or kallithea would be nice (I have no
> direct experience of using either, but they are similar to other
> systems). However, this would be considerable work.

Exactly!

> Perhaps, as a half way house, we could use the resources that we have.
> PRs could go to the bug reporting system. This will, at least, keep all
> the conversations in one place. If we can tag these with "has patch"
> here as well, it will give an queue also.

We already try doing that, as much as we can.  Fortunately, a few
people lately started actively reviewing bug reports, both old and
new, and post analyzes, tag them, propose patches, etc.  That's great,
but we need more of them, and we need to wait for them to gain enough
knowledge and experience to be able to overlook larger portions of
Emacs (including the parts written in C, btw).  This should be the
main vector of our process improvement, at least for some time to
come.  Because nothing more substantial can happen until we have a
critical mass of active maintainers.

> We would still need to do something about the Emacs git, in terms of
> squashability; in practice, this would probably require something
> like gitolite as allowing non-FF pushes on all branches would be a
> bad thing.

I don't really see a problem.  Why doesn't a feature branch or a
scratch branch solve all of this nicely and easily?

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> More generally, any good undergraduate software-engineering curriculum
> should cover change management and should have exercises where
> students describe, review and commit patches, merge branches, etc.
> There should be some well-defined procedure that students actually do
> (as opposed to merely reading about it).This stuff is basic nowadays.

GNU ChangeLogs, as used in practice, are so lame and thin on content
that they certainly can't be taken seriously as a method of documenting
changes.

GNU ChangeLogs always looked to me as cargo cult and an excuse for not
writing proper commit messages.

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Andreas Schwab <schwab@suse.de> writes:

> phillip.lord@russet.org.uk (Phillip Lord) writes:
>
>> I still feel that patches are fairly "old school".
>
> linux-kernel@ is full of patches.


Yes, this is true. However, I would say that linux-kernel also has some 
significant issues in terms of user unfriendliness. Of course not all of
this is due to software.

I believe that they use patchwork, though, which provides threaded
discussion of patches.

Phil

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Noam Postavsky <npostavs@users.sourceforge.net> writes:
>>> I've been sending patches to bug threads, and often getting useful
>>> feedback on them (and since it's by email, the comments can easily be
>>> inline). Personally, I don't find it more clunky than pushing to a
>>> branch, and then opening a PR in a web browser.
>>
>> This is true for the first patch, but not true for additional commits.
>
> Yeah, you're right about that. Most of my Emacs patches so far have
> been small enough that it didn't really bother me too much.

Indeed. And there are many patches to emacs which affect one small part
of Emacs. No worries. I don't think it's necessarily the case that all
changes should go through PR. It depends on the experience of the
developer and the criticality of the code.

From my own perspective, I like code review, and would always be happy
to have feedback on my code, but that many not be possible.


>>> Yes, some patches are forgotten, but I don't see how a PR "system"
>>> makes that less likely to happen, e.g., cask has a bunch of open PRs
>>> sitting around: https://github.com/cask/cask/pulls.
>>
>> I think you have just demonstrated my point. You found out all the
>> outstanding ones also.
>>
>>
>>>> Perhaps, as a half way house, we could use the resources that we have.
>>>> PRs could go to the bug reporting system. This will, at least, keep all
>>>> the conversations in one place. If we can tag these with "has patch"
>>>> here as well, it will give an queue also.
>>>
>>> I thought that was the current system. Here is the queue:
>>> http://debbugs.gnu.org/cgi/pkgreport.cgi?package=emacs;include=tags%3Apatch;bug-rev=on
>>
>> Again, also my point.
>
> If your point is that there are more open patches in Emacs than cask,
> IMO that's mostly because Emacs is a larger and older project.


No, sorry, I was being opaque. Having a nice queue system is a good
thing, and we could use debbugs to do pull requests also, was my point.

Phil

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Phillip.
>
>> > The term seems to imply that instead of a contributor pushing a change
>> > from his machine to a central repository, some specially authorised
>> > authority would pull the change from the contributor's machine.  This
>> > would seem to imply every contributor needing to set up an scp daemon on
>> > his local machine, which doesn't feel like a Good Thing.
>
>> On *some* machine, yes. That can be their own server, or a hosted git
>> repository, or a branch on the Emacs git repository.
>
> The only one of these usable by me would be the last one. I can
> foresee this branch would be open to anybody to commit anything, and
> could quickly fill up with questionable changes. Anybody using this
> method would need to maintain their own copy of this "pull" branch.
> This could lead to quite a few logistical problems, could it not?

It doesn't in my experience. In generally, you create a feature branch
for each thing you want included. Once it is merged to mainline, you
throw it away again. If you are worried about permissions the repo could
be created with individualized namespaces (so phil/* branches would be
readable by all, but writeable only by me).


> Also, the email containing the patch is not the source for what gets
> merged?  This seems inefficient.

There is no patch. Discussion happens on each PR, when everyone is
happy, you rebase trunk against the feature branch, merge back to trunk,
and kill the feature branch.

At the moment, many of the patches that get discussion here do not
contain all the information anyway; I think patches with commit message
are the minority. So in many ways this is better.

Phil

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:
> Starting a PR system in the current conditions is just going to
> _increase_ the workload of the few overloaded people.  E.g., I don't
> want to push changes for anyone else, ever: it takes too much of the
> little time I have to work on Emacs. I want this burden off-loaded to
> someone else.  Guess what? patches that I said were okay can rot for
> many days without anyone with write access doing anything.  I can only
> conclude they all are as busy as I am or busier.  As long as this is
> the situation, how could a PR system help, when it requires me to do
> _more_ than I have to now?

I think in two ways. First, we do not have a system for managing
PRs/patches in the queue. So, it's possible that people with outstanding
patches are not busier than you, they just missed things.

Secondly, in terms of pushing patches for someone else, this doesn't
need to be harder than reviewing the patch and signaling that you are
happy. Many PR management systems will do the merge after "LGTM".


>> Perhaps, as a half way house, we could use the resources that we have.
>> PRs could go to the bug reporting system. This will, at least, keep all
>> the conversations in one place. If we can tag these with "has patch"
>> here as well, it will give an queue also.
>
> We already try doing that, as much as we can.  Fortunately, a few
> people lately started actively reviewing bug reports, both old and
> new, and post analyzes, tag them, propose patches, etc.  That's great,
> but we need more of them, and we need to wait for them to gain enough
> knowledge and experience to be able to overlook larger portions of
> Emacs (including the parts written in C, btw).  This should be the
> main vector of our process improvement, at least for some time to
> come.  Because nothing more substantial can happen until we have a
> critical mass of active maintainers.

I look at this the other way around. I think we are likely to get more
developers, if it is easier to contribute. A project where you can send
in a patch, and have some one give you feedback on it, will end up with
more developers.


>> We would still need to do something about the Emacs git, in terms of
>> squashability; in practice, this would probably require something
>> like gitolite as allowing non-FF pushes on all branches would be a
>> bad thing.
>
> I don't really see a problem.  Why doesn't a feature branch or a
> scratch branch solve all of this nicely and easily?

I think I have discussed this with you before. You have to create
multiple feature branches, or do strange things, rather than just
rebase, force push.

Phil

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: eggert@cs.ucla.edu,  emacs-devel@gnu.org
> Date: Thu, 07 Jul 2016 18:24:10 +0100
> 
> First, we do not have a system for managing PRs/patches in the
> queue. So, it's possible that people with outstanding patches are
> not busier than you, they just missed things.

I just read the bug list and manage my email queue.  How hard can that
be?

> Secondly, in terms of pushing patches for someone else, this doesn't
> need to be harder than reviewing the patch and signaling that you are
> happy. Many PR management systems will do the merge after "LGTM".

Problem is, I don't find the subtle fine points that need addressing
until I actually apply the patch: compiler warnings, code not
according to our conventions, sometimes patch won't apply, etc.

> I look at this the other way around. I think we are likely to get more
> developers, if it is easier to contribute.

I think we've already done a lot in that direction, and I don't see
how can we be expected to do more.  All the other projects I
participate in make it harder, and yet no one complains or thinks they
are hard on contributors.

> A project where you can send in a patch, and have some one give you
> feedback on it, will end up with more developers.

Which is exactly what I said, and you objected.  So now I'm confused
what we are arguing about.

> >> We would still need to do something about the Emacs git, in terms of
> >> squashability; in practice, this would probably require something
> >> like gitolite as allowing non-FF pushes on all branches would be a
> >> bad thing.
> >
> > I don't really see a problem.  Why doesn't a feature branch or a
> > scratch branch solve all of this nicely and easily?
> 
> I think I have discussed this with you before. You have to create
> multiple feature branches, or do strange things, rather than just
> rebase, force push.

What's the problem with multiple branches?  It's a very easy
technique, and also very safe.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Currently, I think ChangeLogs are a barrier to contribution.

We no longer directly maintain ChangeLog files.  Are you talking about
writing the commit log entries?

								 The vast
  > majority of other software projects don't use them.

Are you saying that most projects do not keep track of which functions
are changed in each commit?

How can maintainers figure out how to solve problems without detailed
log records to show them which previous changes they need to study?

That "vast majority" -- how long have those projects been going?  GNU
Emacs was first released over 30 years ago.

  > So I would suggest moving to a pull request system, where code review
  > from a second contributor is required to merge any non-trivial code
  > (exceptions should be granted based on years contributing to Emacs).
  > That also gives *everyone* the opportunity to comment on the code before
  > it's merged, instead of post-facto. Clearly services such as Github and
  > BitBucket and many others have been offering this functionality for a
  > while with good results.

I don't know exactly how this works, but it seems like a good idea.

  > A big advantage of pull requests is that they can group commits, so each
  > commit doesn't need the level of detail it does today, and so the
  > evolution of the work is visible to a reviewer.

I don't see how the one relates to the other.  Better review of the
changes before they installed will not eliminate the need for support
_finding_ pertinent changes for a problem found 5 or 10 years from
now.

  > The pull
  > request system can later provide *everything* that a ChangeLog could,

I am skeptical of this claim.  How precisely will the pull request system
provide what we now get from the detailed lists of objects changed
in the log entries?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   I would be in favor of
  > grouping changes spanning multiple days together and using the final
  > date rather than having them split into separate entries by day.

I'm in favor of that, too

  > I agree that having to provide secondary commentary in a commit log is
  > a pain and duplicative of effort but I don't have any suggestion for
  > solution on that.

What does "secondary commentary" mean?  A concise overall description
of what that change does?  That shouldn't be hard to write.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > EZ> What format do you suggest instead?

  > "Summary line.

  > Details..."

  > Where the Details can be in the current or another format or whatever
  > works for the contributor.

That's exactly the opposite of what we need.
These records will be read by hundreds of people.
We need them to be kept in a uniform way to make them reliable.

			       The per-file summary would be available from
  > the pull request system, regardless of the formatting of the commit message.

What exactly would be "available from" that system?
What does the material look like?
Is it hand-written or auto-generated?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > JW> I may be a strange one here, but I like the ChangeLog format for commit
  > JW> messages. It's not unreasonable to ask a submitter to take some care and pride
  > JW> in their description of the changes they've made -- after all, they've spent a
  > JW> fair bit of time preparing the change itself.

  > All right, that's very reasonable.

I think so too.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 07/07/2016 10:57 PM, Eli Zaretskii wrote:

> I just read the bug list and manage my email queue.  How hard can that
> be?

A code review system will manage the patch queue for you, organize the 
discussion, hide the messages related to the code that has changed in 
the latest revision of the patch while still showing the rest. Maybe 
your email setup does all that already; mine doesn't.

> I don't find the subtle fine points that need addressing
> until I actually apply the patch: compiler warnings, code not
> according to our conventions, sometimes patch won't apply, etc.

The mature modern code review systems integrate with the VCS, and can 
check whether the patch applies to the current master (essentially, they 
check for merge conflicts, since submitting a patchset involves pushing 
it to a branch), lets you look at the code with the patch applied, 
review the conventions, run the CI build (which could include a check 
for compiler warnings, though I admit it's a difficult area), and then 
allow you to merge the patch with a click of a mouse (or whichever way 
we choose, in a new package that interacts with the review system's API).

That said, before continuing this discussion, I'd rather we switch to a 
modern-ish bug tracker first.

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> First, we do not have a system for managing PRs/patches in the
>> queue. So, it's possible that people with outstanding patches are
>> not busier than you, they just missed things.
>
> I just read the bug list and manage my email queue. How hard can that
> be?

It is okay, once you are used to it.


>> Secondly, in terms of pushing patches for someone else, this doesn't
>> need to be harder than reviewing the patch and signaling that you are
>> happy. Many PR management systems will do the merge after "LGTM".
>
> Problem is, I don't find the subtle fine points that need addressing
> until I actually apply the patch: compiler warnings, code not
> according to our conventions, sometimes patch won't apply, etc.

With a PR, it always applies. Compiler warnings obviously require an
integrated continuous integration system (i.e. the PR is pre-built,
tested and the potential merge is checked for conflicts).


>> I look at this the other way around. I think we are likely to get more
>> developers, if it is easier to contribute.
>
> I think we've already done a lot in that direction, and I don't see
> how can we be expected to do more.  All the other projects I
> participate in make it harder, and yet no one complains or thinks they
> are hard on contributors.

The fundamental complexity is contributing is the software engineering.
Tools do not really change that, of course, but that is not really a
reason for not using good tools.


>> I think I have discussed this with you before. You have to create
>> multiple feature branches, or do strange things, rather than just
>> rebase, force push.
>
> What's the problem with multiple branches?  It's a very easy
> technique, and also very safe.


I think a system where you start off with

feature/my-new-feature
feature/my-new-feature-1
feature/my-new-feature-2

with consecutive squashes is not a good system.

I think that the discussion is bottoming out here. If there is interest,
I will be happy to investigate some of the options and produce a report
of pros and cons. Alternatively, if the general feeling is, it's all
fine, then no worries, I'll leave it.

Phil

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Richard Stallman <rms@gnu.org> writes:
> 								 The vast
>   > majority of other software projects don't use them.
>
> Are you saying that most projects do not keep track of which functions
> are changed in each commit?

Yes, or at least not directly in the commit log.

> How can maintainers figure out how to solve problems without detailed
> log records to show them which previous changes they need to study?

They use the version control system. This provides different information
from the changelogs, of course. Worse in some cases, but better in
others. A function that is renamed, for example, but stays in the same
place in the file is better tracked by VC than a changelog.

> That "vast majority" -- how long have those projects been going?  GNU
> Emacs was first released over 30 years ago.

The vast majority are younger than Emacs, given that it's older than
most.



Phil

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Thu, 07 Jul 2016 22:57:46 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> Problem is, I don't find the subtle fine points that need addressing
EZ> until I actually apply the patch: compiler warnings, code not
EZ> according to our conventions, sometimes patch won't apply, etc.

Typically this is addressed with a build system that takes every pull
request and builds it, logging problems right into the pull request.

EZ> All the other projects I
EZ> participate in make it harder, and yet no one complains or thinks they
EZ> are hard on contributors.

I think there is a confirmation bias there: people tend to believe the
feedback that confirms their existing views and discount the opposite.

On Fri, 08 Jul 2016 12:28:32 +0100 phillip.lord@russet.org.uk (Phillip Lord) wrote: 

PL> I think that the discussion is bottoming out here. If there is interest,
PL> I will be happy to investigate some of the options and produce a report
PL> of pros and cons. Alternatively, if the general feeling is, it's all
PL> fine, then no worries, I'll leave it.

On Fri, 8 Jul 2016 02:50:31 +0300 Dmitry Gutov <dgutov@yandex.ru> wrote: 

DG> That said, before continuing this discussion, I'd rather we switch to a
DG> modern-ish bug tracker first.

As I said, I'll try using the (painful) bug tracker to do code reviews.
Please consider me strongly in favor of a better code review system and
a better bug tracker. Ideally they would be one and the same.

On Thu, 07 Jul 2016 17:56:01 -0400 Richard Stallman <rms@gnu.org> wrote: 

RS> Are you saying that most projects do not keep track of which functions
RS> are changed in each commit?

RS> How can maintainers figure out how to solve problems without detailed
RS> log records to show them which previous changes they need to study?

The code review (pull request handling) system has all that information.

For instance, you can search for error messages, not just the commits
that fixed them. The real value emerges when the entire development
workflow is captured in the same place, indexed for searching, and open
for world-wide review and contribution.

>> The pull request system can later provide *everything* that a
>> ChangeLog could,

RS> I am skeptical of this claim.  How precisely will the pull request system
RS> provide what we now get from the detailed lists of objects changed
RS> in the log entries?

The ChangeLog structure is something like this:

top-level-message
  file1: message1
    symbol1: message11
    symbol2: message12
  file2: message2
    ...

This can easily be encoded in a database table so it's searchable,
indexed by symbol name or file name (to build a history), etc. as part
of the pull request system. It can be semi-automatic: the system figures
out the file and symbol changes, then the developer adds the rest (and
the review doesn't end until this is done).

The database can be part of the Emacs Git repository, it doesn't have to
be something magical outside it. But that's an architectural decision.

The end result is a lot like the ChangeLog format we have today...
except it's structured data, linked to a *series* of commits *and* the
code review on them.

That piece doesn't exist, but I'm willing to bet that developing it is
*much less* work than the man-hours that will be spent crafting the
ChangeLog format over the coming years. Consider as well that it will be
useful to the whole GNU project, not just Emacs.

Ted

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > GNU ChangeLogs, as used in practice, are so lame and thin on content
  > that they certainly can't be taken seriously as a method of documenting
  > changes.

They are good for their purpose, which is to summarize which functions
or objects were changed, when, and by whom.  That's useful when
you want to see which changes to look at to figure out when a bug
got introduced.

You haven't said what "content" you think is missing, so it is hard
for me to respond further than that.  However, if you're talking about
explanations of the code, those are supposed to go in comments in the
code.

  > GNU ChangeLogs always looked to me as cargo cult and an excuse for not
  > writing proper commit messages.

There was no such thing as a commit message in 1985.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: eggert@cs.ucla.edu,  emacs-devel@gnu.org
> Date: Fri, 08 Jul 2016 12:28:32 +0100
> 
> >> I think I have discussed this with you before. You have to create
> >> multiple feature branches, or do strange things, rather than just
> >> rebase, force push.
> >
> > What's the problem with multiple branches?  It's a very easy
> > technique, and also very safe.
> 
> I think a system where you start off with
> 
> feature/my-new-feature
> feature/my-new-feature-1
> feature/my-new-feature-2
> 
> with consecutive squashes is not a good system.

That's because you decided to squash.  You don't have to.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Richard Stallman <rms@gnu.org> writes:

>   > GNU ChangeLogs, as used in practice, are so lame and thin on content
>   > that they certainly can't be taken seriously as a method of documenting
>   > changes.
>
> They are good for their purpose, which is to summarize which functions
> or objects were changed, when, and by whom.  That's useful when
> you want to see which changes to look at to figure out when a bug
> got introduced.

That information is already available from the VCS, on steroids. You can
query the history of a function, ask which changes introduced or deleted
a call to certain function...

But the real damage ChangeLogs cause is their formulaic, almost mechanic
aspect: people write a lot of minutiae instead of writing a commit
message intended to help the reader, the same way good code comments are
written. It is true that some hackers write good commit messages *and*
ChangeLogs, but I'm under the impression that most people just write the
changelog and consider that their job is done (after all, current policy
for commit messages is to write a summary line and the 

> You haven't said what "content" you think is missing, so it is hard
> for me to respond further than that.

Just a few minutes ago I responded to another developer that asked on a
private email about details about what a "proper commit message" means
to me. Here is my response:

In a nutshell, a proper commit message shall contain, either explicitly
or implicitly, the what, why, where and how of the change. It must be
informative and provide the info that the reader is interested on,
without entering on unnecessary detail (the diff is readily available,
there is no need to name each function touched by the change). The
author must write it with the same mindset he chooses function and
variable names, decides when it is a good idea to put a code comment,
etc.

> However, if you're talking about
> explanations of the code, those are supposed to go in comments in the
> code.

Absolutely agreed. That's about putting each piece where it belongs.
Some times, however, the code is not the best place to put some
explanation (think of an scattered change or why some alternative that,
at first, seems more adequate was discarded) and the commit message
offers an alternative. The commit message is a good place to direct the
reader to the key areas for understanding the change, inform the reader
about current status and future steps and any other info that the author
thinks is relevant for current and/or future hackers with an interest on
the affected area.

>   > GNU ChangeLogs always looked to me as cargo cult and an excuse for not
>   > writing proper commit messages.
>
> There was no such thing as a commit message in 1985.

Exactly. I'm pretty sure that if you had a good VCS at that time
ChangeLogs would never came into being.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Fri, 08 Jul 2016 09:27:55 -0400
> 
> EZ> All the other projects I
> EZ> participate in make it harder, and yet no one complains or thinks they
> EZ> are hard on contributors.
> 
> I think there is a confirmation bias there: people tend to believe the
> feedback that confirms their existing views and discount the opposite.

I think you underestimate my abilities to avoid from such fallacies.
It's almost an offense to say that.

> That piece doesn't exist, but I'm willing to bet that developing it is
> *much less* work than the man-hours that will be spent crafting the
> ChangeLog format over the coming years.

Actually, writing ChangeLog entries takes a negligible fraction of my
time.  I can almost do that while taking a nap.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Ted Zlatanov <tzz@lifelogs.com> writes:

> EZ> All the other projects I
> EZ> participate in make it harder, and yet no one complains or thinks they
> EZ> are hard on contributors.
>
> I think there is a confirmation bias there: people tend to believe the
> feedback that confirms their existing views and discount the opposite.

It is worse: those that think that the process makes things
unnecessarily difficult probably will never contribute and hence you
will never know about their existence, except for some passing comment
on some other forum.

That said, IMO writing (and learning to write) ChangeLogs on itself is
not a big deterrent to contributing (for small contributions!) Its
impact on the quality of commit messages is more concerning.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Óscar Fuentes <ofv@wanadoo.es> writes:

Sorry, sent too early.

> But the real damage ChangeLogs cause is their formulaic, almost mechanic
> aspect: people write a lot of minutiae instead of writing a commit
> message intended to help the reader, the same way good code comments are
> written. It is true that some hackers write good commit messages *and*
> ChangeLogs, but I'm under the impression that most people just write the
> changelog and consider that their job is done (after all, current policy
> for commit messages is to write a summary line and the 

... Changelog just below.)

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Fri, 08 Jul 2016 17:06:36 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Fri, 08 Jul 2016 09:27:55 -0400
>> 
EZ> All the other projects I
EZ> participate in make it harder, and yet no one complains or thinks they
EZ> are hard on contributors.
>> 
>> I think there is a confirmation bias there: people tend to believe the
>> feedback that confirms their existing views and discount the opposite.

EZ> I think you underestimate my abilities to avoid from such fallacies.
EZ> It's almost an offense to say that.

I was talking about all the people in those projects that are not
complaining.

Ted

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Fri, 08 Jul 2016 16:02:29 +0200
> Cc: emacs-devel@gnu.org
> 
> Richard Stallman <rms@gnu.org> writes:
> 
> >   > GNU ChangeLogs, as used in practice, are so lame and thin on content
> >   > that they certainly can't be taken seriously as a method of documenting
> >   > changes.
> >
> > They are good for their purpose, which is to summarize which functions
> > or objects were changed, when, and by whom.  That's useful when
> > you want to see which changes to look at to figure out when a bug
> > got introduced.
> 
> That information is already available from the VCS, on steroids. You can
> query the history of a function, ask which changes introduced or deleted
> a call to certain function...

Yes, but doing so on a large file that saw many changes is relatively
slow.  Scanning a ChangeLog is much faster, so I always use it as the
first approximation, and only go to the likes of "git log -L" and
"git annotate" if I have to.

> But the real damage ChangeLogs cause is their formulaic, almost mechanic
> aspect: people write a lot of minutiae instead of writing a commit
> message intended to help the reader, the same way good code comments are
> written. It is true that some hackers write good commit messages *and*
> ChangeLogs, but I'm under the impression that most people just write the
> changelog and consider that their job is done (after all, current policy
> for commit messages is to write a summary line and the 

You are invited to look at our recent commit log messages, which are
in the ChangeLog format, and point out the ones that fit the above
description.  I don't think you will find a lot of them, but maybe I'm
mistaken.

> In a nutshell, a proper commit message shall contain, either explicitly
> or implicitly, the what, why, where and how of the change. It must be
> informative and provide the info that the reader is interested on,
> without entering on unnecessary detail (the diff is readily available,
> there is no need to name each function touched by the change). The
> author must write it with the same mindset he chooses function and
> variable names, decides when it is a good idea to put a code comment,
> etc.

This conflates several distinct places where this information is held,
making it sound that all of it should be in the commit log message.
That is false, or at least I couldn't disagree more.  First and
foremost, the code should speak for itself, so the most detailed
explanations should be in comments there.  We didn't just replace the
code by the VCS!

Second, many times the information is in a prolonged discussion, in
particular on the bug tracker.  By including a reference to the bug
report, we implicitly include all of it in the log message.

As for "the diff is readily available, there is no need to name each
function touched by the change" part, it is inaccurate: the VCS diff
tool doesn't always show the name of the function or macro that is
being changed.  Moreover, if several functions/macros are changed in a
single hunk, at most one of them will be named in the hunk header, the
rest need to be deduced by painfully reading the diffs.  I don't think
we can say in good faith that this information is as readily available
as in ChangeLog-style log messages.

> Some times, however, the code is not the best place to put some
> explanation (think of an scattered change or why some alternative that,
> at first, seems more adequate was discarded) and the commit message
> offers an alternative. The commit message is a good place to direct the
> reader to the key areas for understanding the change, inform the reader
> about current status and future steps and any other info that the author
> thinks is relevant for current and/or future hackers with an interest on
> the affected area.

I find comments to be a better place for these situations as well (and
made many comments of this kind over the years myself).  The most
important reason is discoverability: when one reads code, they usually
won't look at the Git history of the code they go through, but
comments they will see.  Scattered changes are not a problem, you can
always say "see such-and-such function".

> >   > GNU ChangeLogs always looked to me as cargo cult and an excuse for not
> >   > writing proper commit messages.
> >
> > There was no such thing as a commit message in 1985.
> 
> Exactly. I'm pretty sure that if you had a good VCS at that time
> ChangeLogs would never came into being.

If history went another way, perhaps we wouldn't have Emacs as well.

More to the point, fact is that no better methodology was invented.
So history is not the only reason why we are still using this format.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Fri, 08 Jul 2016 16:13:21 +0200
> 
> Ted Zlatanov <tzz@lifelogs.com> writes:
> 
> > EZ> All the other projects I
> > EZ> participate in make it harder, and yet no one complains or thinks they
> > EZ> are hard on contributors.
> >
> > I think there is a confirmation bias there: people tend to believe the
> > feedback that confirms their existing views and discount the opposite.
> 
> It is worse: those that think that the process makes things
> unnecessarily difficult probably will never contribute and hence you
> will never know about their existence, except for some passing comment
> on some other forum.

Someone will always think that, no matter how low the bar is.  And I
don't think this is "worse": I'm not sure we want such people on
board.

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Fri, 08 Jul 2016 09:27:55 -0400
>> 
>> EZ> All the other projects I
>> EZ> participate in make it harder, and yet no one complains or thinks they
>> EZ> are hard on contributors.
>> 
>> I think there is a confirmation bias there: people tend to believe the
>> feedback that confirms their existing views and discount the opposite.
>
> I think you underestimate my abilities to avoid from such fallacies.
> It's almost an offense to say that.

Wow, Eli! One thing modern psychology taught us is that human thinking
is anything but rational and objective. It is true that different
individuals suffer on different levels of biased views, after-the-fact
reasoning for previous actions, etc. but I'm skeptical of the existence
of individuals who are able to be completely objective and rational for
more than a few minutes.

Since I subscribed Ted's point on a previous message, I'll ask for your
forgiveness for considering you a human being.

>> That piece doesn't exist, but I'm willing to bet that developing it is
>> *much less* work than the man-hours that will be spent crafting the
>> ChangeLog format over the coming years.
>
> Actually, writing ChangeLog entries takes a negligible fraction of my
> time.  I can almost do that while taking a nap.

Good for you. You do a terrific job at documenting things in general,
which includes writing good commit messages on addition of the Changelog
entry. And you are the top-hacker of Emacs. You are anything but the
typical contributor Ted is talking about.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Fri, 08 Jul 2016 10:22:43 -0400
> 
> On Fri, 08 Jul 2016 17:06:36 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 
> 
> >> From: Ted Zlatanov <tzz@lifelogs.com>
> >> Date: Fri, 08 Jul 2016 09:27:55 -0400
> >> 
> EZ> All the other projects I
> EZ> participate in make it harder, and yet no one complains or thinks they
> EZ> are hard on contributors.
> >> 
> >> I think there is a confirmation bias there: people tend to believe the
> >> feedback that confirms their existing views and discount the opposite.
> 
> EZ> I think you underestimate my abilities to avoid from such fallacies.
> EZ> It's almost an offense to say that.
> 
> I was talking about all the people in those projects that are not
> complaining.

That includes me.

Besides, why would people in those other projects suffer from this
bias, while people here don't?

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Fri, 08 Jul 2016 17:29:48 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Fri, 08 Jul 2016 10:22:43 -0400
>> 
>> On Fri, 08 Jul 2016 17:06:36 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 
>> 
>> >> From: Ted Zlatanov <tzz@lifelogs.com>
>> >> Date: Fri, 08 Jul 2016 09:27:55 -0400
>> >> 
EZ> All the other projects I
EZ> participate in make it harder, and yet no one complains or thinks they
EZ> are hard on contributors.
>> >> 
>> >> I think there is a confirmation bias there: people tend to believe the
>> >> feedback that confirms their existing views and discount the opposite.
>> 
EZ> I think you underestimate my abilities to avoid from such fallacies.
EZ> It's almost an offense to say that.
>> 
>> I was talking about all the people in those projects that are not
>> complaining.

EZ> That includes me.

I'm hereby explicitly excluding you from my statement :)

Please consider the wider implication, not the personal one.

EZ> Besides, why would people in those other projects suffer from this
EZ> bias, while people here don't?

All people have this bias, see for example:

Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in
many guises. Review of General Psychology, 2(2), 175-220.

Ted

Re: Is it time to drop ChangeLogs?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> That information is already available from the VCS, on steroids. You can
>> query the history of a function, ask which changes introduced or deleted
>> a call to certain function...
>
> Yes, but doing so on a large file that saw many changes is relatively
> slow.  Scanning a ChangeLog is much faster, so I always use it as the
> first approximation, and only go to the likes of "git log -L" and
> "git annotate" if I have to.

You rarely are interested on the whole file. vc-region-history makes
wonders when you care about part of the file (a function, for instance.)
Using the `annotate' interface has the advantage of bringing in the rest
of VC: one key and you see the patch that introduced the change, another
key and you see the commit message, yet another key and you can annotate
the function as it was before the change, and so on. Tools such as
git-timemachine.el makes the process even more effective.

>> But the real damage ChangeLogs cause is their formulaic, almost mechanic
>> aspect: people write a lot of minutiae instead of writing a commit
>> message intended to help the reader, the same way good code comments are
>> written. It is true that some hackers write good commit messages *and*
>> ChangeLogs, but I'm under the impression that most people just write the
>> changelog and consider that their job is done (after all, current policy
>> for commit messages is to write a summary line and the 
>
> You are invited to look at our recent commit log messages, which are
> in the ChangeLog format, and point out the ones that fit the above
> description.  I don't think you will find a lot of them, but maybe I'm
> mistaken.

Just the second commit on the log output:

  Remove just input mark

  * lisp/ibuffer.el (ibuffer-unmark-all): When MARK is not ?\r remove
  just MARK.

no hint about why it was necessary.

About 3 commits below:

  Disable App Nap (bug#22993)

  * nextstep/templates/Info.plist.in: Insert AppNap disable code.

No hint about why it was necessary, except for the bug# (I hope the
reason for the change is documented there, along with its implications;
the bug is still open). Besides, having to jump to the bug is a nuisance
(some people mentioned that having the ChangeLogs readily available on
the tarballs is an advantage.) It is not necessary to duplicate the
whole discussion, but briefly mentioning what the problem was and why it
was decided to solve it this way would be helpful.

Yet a two or three commits below:

 Fix an error in Tramp for rsync

 * lisp/net/tramp-sh.el (tramp-do-copy-or-rename-file-out-of-band):
 Make it work for "rsync".
 (tramp-make-copy-program-file-name): Apply `directory-file-name'.

There is no even bug # here.

And so on.

>> In a nutshell, a proper commit message shall contain, either explicitly
>> or implicitly, the what, why, where and how of the change. It must be
>> informative and provide the info that the reader is interested on,
>> without entering on unnecessary detail (the diff is readily available,
>> there is no need to name each function touched by the change). The
>> author must write it with the same mindset he chooses function and
>> variable names, decides when it is a good idea to put a code comment,
>> etc.
>
> This conflates several distinct places where this information is held,
> making it sound that all of it should be in the commit log message.
> That is false, or at least I couldn't disagree more.  First and
> foremost, the code should speak for itself, so the most detailed
> explanations should be in comments there.  We didn't just replace the
> code by the VCS!

I never said otherwise. I talked about the "right place" for putting
information on the next paragraph.

> Second, many times the information is in a prolonged discussion, in
> particular on the bug tracker.  By including a reference to the bug
> report, we implicitly include all of it in the log message.

It is not helpful to point the reader to a long discussion if the
rationale for the change can be summarized (mentioning the bug# is a
good thing, of course.)

> As for "the diff is readily available, there is no need to name each
> function touched by the change" part, it is inaccurate: the VCS diff
> tool doesn't always show the name of the function or macro that is
> being changed.  Moreover, if several functions/macros are changed in a
> single hunk, at most one of them will be named in the hunk header, the
> rest need to be deduced by painfully reading the diffs.  I don't think
> we can say in good faith that this information is as readily available
> as in ChangeLog-style log messages.

Possibly not as readily available but, to be fair, how many times you
are interested on the functions affected on a trivial way by a change?

Usually you are interested on either how a given function evolved or on
how the introduction of a fix/feature affected the code base. For the
first case you have vc-region-history and for the second a combination
of VC features and text search.

>> Some times, however, the code is not the best place to put some
>> explanation (think of an scattered change or why some alternative that,
>> at first, seems more adequate was discarded) and the commit message
>> offers an alternative. The commit message is a good place to direct the
>> reader to the key areas for understanding the change, inform the reader
>> about current status and future steps and any other info that the author
>> thinks is relevant for current and/or future hackers with an interest on
>> the affected area.
>
> I find comments to be a better place for these situations as well (and
> made many comments of this kind over the years myself).  The most
> important reason is discoverability: when one reads code, they usually
> won't look at the Git history of the code they go through, but
> comments they will see.  Scattered changes are not a problem, you can
> always say "see such-and-such function".

I see this on a different way: commit messages add to, and complement,
code comments.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Fri, 08 Jul 2016 10:49:50 -0400
> 
> EZ> Besides, why would people in those other projects suffer from this
> EZ> bias, while people here don't?
> 
> All people have this bias, see for example:
> 
> Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in
> many guises. Review of General Psychology, 2(2), 175-220.

Great, we are well on our way to explain any fact that contradicts our
beliefs by quoting from papers on psychology!

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Fri, 08 Jul 2016 17:07:05 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> That information is already available from the VCS, on steroids. You can
> >> query the history of a function, ask which changes introduced or deleted
> >> a call to certain function...
> >
> > Yes, but doing so on a large file that saw many changes is relatively
> > slow.  Scanning a ChangeLog is much faster, so I always use it as the
> > first approximation, and only go to the likes of "git log -L" and
> > "git annotate" if I have to.
> 
> You rarely are interested on the whole file. vc-region-history makes
> wonders when you care about part of the file (a function, for instance.)

That's "git log -L" that I mentioned.  It's startup time is not
negligible.

>   Remove just input mark
> 
>   * lisp/ibuffer.el (ibuffer-unmark-all): When MARK is not ?\r remove
>   just MARK.
> 
> no hint about why it was necessary.

Did you look at the comments?

> About 3 commits below:
> 
>   Disable App Nap (bug#22993)
> 
>   * nextstep/templates/Info.plist.in: Insert AppNap disable code.
> 
> No hint about why it was necessary, except for the bug#

Which is more than enough.

> Besides, having to jump to the bug is a nuisance
> (some people mentioned that having the ChangeLogs readily available on
> the tarballs is an advantage.) It is not necessary to duplicate the
> whole discussion, but briefly mentioning what the problem was and why it
> was decided to solve it this way would be helpful.

This request is unreasonable.  If nothing else, it will make the bar
for contributing higher, not lower.  The information is recorded in
the bug discussion, and there's no need to reproduce it in the log
message.

> Yet a two or three commits below:
> 
>  Fix an error in Tramp for rsync
> 
>  * lisp/net/tramp-sh.el (tramp-do-copy-or-rename-file-out-of-band):
>  Make it work for "rsync".
>  (tramp-make-copy-program-file-name): Apply `directory-file-name'.
> 
> There is no even bug # here.
> 
> And so on.

I see no problems in these log messages.

> > This conflates several distinct places where this information is held,
> > making it sound that all of it should be in the commit log message.
> > That is false, or at least I couldn't disagree more.  First and
> > foremost, the code should speak for itself, so the most detailed
> > explanations should be in comments there.  We didn't just replace the
> > code by the VCS!
> 
> I never said otherwise. I talked about the "right place" for putting
> information on the next paragraph.

Which we already do.  Improvements are always possible, but by and
large our commit log messages are good.

> > Second, many times the information is in a prolonged discussion, in
> > particular on the bug tracker.  By including a reference to the bug
> > report, we implicitly include all of it in the log message.
> 
> It is not helpful to point the reader to a long discussion if the
> rationale for the change can be summarized (mentioning the bug# is a
> good thing, of course.)

There's no other practical way.  It's unreasonable to ask people to
summarize such discussions in a log message.

> > As for "the diff is readily available, there is no need to name each
> > function touched by the change" part, it is inaccurate: the VCS diff
> > tool doesn't always show the name of the function or macro that is
> > being changed.  Moreover, if several functions/macros are changed in a
> > single hunk, at most one of them will be named in the hunk header, the
> > rest need to be deduced by painfully reading the diffs.  I don't think
> > we can say in good faith that this information is as readily available
> > as in ChangeLog-style log messages.
> 
> Possibly not as readily available but, to be fair, how many times you
> are interested on the functions affected on a trivial way by a change?

Always.  For me, digging into an issue usually begins with a function
I know is misbehaving.  The very next question is: when was it last
changed, how, and why.

> Usually you are interested on either how a given function evolved or on
> how the introduction of a fix/feature affected the code base.

No, that's rarely the case.  It only happens when I find a very old
bug, and am curious who and when introduced it.

> > I find comments to be a better place for these situations as well (and
> > made many comments of this kind over the years myself).  The most
> > important reason is discoverability: when one reads code, they usually
> > won't look at the Git history of the code they go through, but
> > comments they will see.  Scattered changes are not a problem, you can
> > always say "see such-and-such function".
> 
> I see this on a different way: commit messages add to, and complement,
> code comments.

They shouldn't.

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Fri, 08 Jul 2016 18:18:03 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Fri, 08 Jul 2016 10:49:50 -0400
>> 
>> Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in
>> many guises. Review of General Psychology, 2(2), 175-220.

EZ> Great, we are well on our way to explain any fact that contradicts our
EZ> beliefs by quoting from papers on psychology!

That was a citation, not a quote :)

You asked for a plausible explanation for why no one in several projects
complains about high barriers to contribution and I provided it. The
citation was a shortcut so I didn't have to explain it in detail. The
paper was actually a review of studies, showing significant empirical
evidence, and not the original work on that topic by Peter Wason[1].

One way to prove me wrong is to look at the number of new contributors
and churn (how many leave and don't come back) as a function of time,
both in GNU projects and outside them. Is it correlated to the
difficulty of contributing to a project? Is there any statistical
evidence that high barriers discourage contribution? Is there any
evidence of a correlation between code quality or bug counts with high
barriers? That would be an interesting study that would benefit FSF/GNU
as a whole, and would be much more convincing than our personal
opinions.

Ted

[1] Wason, Peter C. (1960), "On the failure to eliminate hypotheses in a conceptual task", Quarterly Journal of Experimental Psychology (Psychology Press) 12 (3): 129–140.

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 07/08/2016 06:25 PM, Eli Zaretskii wrote:
>> From: Óscar Fuentes <ofv@wanadoo.es>

>> You rarely are interested on the whole file. vc-region-history makes
>> wonders when you care about part of the file (a function, for instance.)
>
> That's "git log -L" that I mentioned.  It's startup time is not
> negligible.

Personally, I've never found vc-region-history too useful (as opposed to 
e.g. vc-annotate).

>> Besides, having to jump to the bug is a nuisance
>> (some people mentioned that having the ChangeLogs readily available on
>> the tarballs is an advantage.) It is not necessary to duplicate the
>> whole discussion, but briefly mentioning what the problem was and why it
>> was decided to solve it this way would be helpful.
>
> This request is unreasonable.  If nothing else, it will make the bar
> for contributing higher, not lower.  The information is recorded in
> the bug discussion, and there's no need to reproduce it in the log
> message.

We do encourage that, actually, though not require (bar for 
contributing, etc). This can be especially useful when the issue is 
not-to-complex, but the bug discussion is long.

Mentioned near "rationale for a change" in CONTRIBUTE.

Re: Is it time to drop ChangeLogs?

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> Personally, I've never found vc-region-history too useful (as opposed to
> e.g. vc-annotate).

Aren't they quite different?  vc-annotate shows me the last time any line was
modified, while vc-region-history show me the history of that region,
including all the times it was modified.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

vc-region-history, was: Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 07/09/2016 02:29 AM, John Wiegley wrote:

>> Personally, I've never found vc-region-history too useful (as opposed to
>> e.g. vc-annotate).
>
> Aren't they quite different?  vc-annotate shows me the last time any line was
> modified, while vc-region-history show me the history of that region,
> including all the times it was modified.

Yes, but both are tools that can be used for examining how a given piece 
of code changed over time and why.

Inside vc-annotate buffer, I use vc-annotate-revision-previous-to-line a 
lot (bound to `a').

vc-region-history, on the other hand, gives a narrow view of each 
changeset (but the full commit messages, which is understandable, but 
can be confusing), and it seems basically impossible for it to correctly 
follow the history of a given piece of code, as long as its complex enough.

Granted, I've only ever tried it out in complex cases so far.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 9 Jul 2016 01:58:48 +0300
> 
> On 07/08/2016 06:25 PM, Eli Zaretskii wrote:
> >> From: Óscar Fuentes <ofv@wanadoo.es>
> 
> >> You rarely are interested on the whole file. vc-region-history makes
> >> wonders when you care about part of the file (a function, for instance.)
> >
> > That's "git log -L" that I mentioned.  It's startup time is not
> > negligible.
> 
> Personally, I've never found vc-region-history too useful (as opposed to 
> e.g. vc-annotate).

I do use "git annotate" at first, but it will only show the last
commit that changed the code you are investigating, and I find this
too frequently to be some reformatting or other similar cleanup that
is not really what I want.  Then going back in history with "git
annotate" is inconvenient, so "git log -L" is better.

In any case, the startup times of 'annotate' and 'log -L' are
comparable, and both are non-negligible.

> >> Besides, having to jump to the bug is a nuisance
> >> (some people mentioned that having the ChangeLogs readily available on
> >> the tarballs is an advantage.) It is not necessary to duplicate the
> >> whole discussion, but briefly mentioning what the problem was and why it
> >> was decided to solve it this way would be helpful.
> >
> > This request is unreasonable.  If nothing else, it will make the bar
> > for contributing higher, not lower.  The information is recorded in
> > the bug discussion, and there's no need to reproduce it in the log
> > message.
> 
> We do encourage that, actually, though not require (bar for 
> contributing, etc). This can be especially useful when the issue is 
> not-to-complex, but the bug discussion is long.
> 
> Mentioned near "rationale for a change" in CONTRIBUTE.

Right, but this discussion is about the requirements, not about the
stuff for which one gets extra bonus points.

Re: vc-region-history, was: Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 9 Jul 2016 02:39:01 +0300
> 
> vc-region-history, on the other hand, gives a narrow view of each 
> changeset (but the full commit messages, which is understandable, but 
> can be confusing), and it seems basically impossible for it to correctly 
> follow the history of a given piece of code, as long as its complex enough.

That's very far from my experience.  I frequently need to investigate
fragments of code which have very long histories, and "log -L" always
gives me the full picture, even when the code in question changed
significantly, right to the commit that first introduced the code.

Perhaps the secret is in choosing the appropriate region.  I normally
try to submit the entire function where the code lives, and it works
very well.

Re: vc-region-history, was: Re: Is it time to drop ChangeLogs?

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 939 bytes --]

If the discussion is on the usefulness of vc-region-history, then it has
been very useful for me.

Coincidentally, just yesterday, I bound that command to "C-x v H" in my
config, after I realized that I had been doing M-x vc-region-history for a
few dozen times.

I always wondered why this command was not bound to the "C-x v" map, and
have also created a little patch for master branch to add that binding.

On Sat, Jul 9, 2016, 3:04 AM Eli Zaretskii <eliz@gnu.org> wrote:

> That's very far from my experience.  I frequently need to investigate
> fragments of code which have very long histories, and "log -L" always
> gives me the full picture, even when the code in question changed
> significantly, right to the commit that first introduced the code.
>
> Perhaps the secret is in choosing the appropriate region.  I normally
> try to submit the entire function where the code lives, and it works
> very well.
>
-- 

-- 
Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1357 bytes --]

Re: Is it time to drop ChangeLogs?

[Tino Calancha]

[-- Attachment #1: Type: text/plain, Size: 1903 bytes --]


On Fri, 8 Jul 2016, Óscar Fuentes wrote:

> Just the second commit on the log output:
>
>  Remove just input mark
>
>  * lisp/ibuffer.el (ibuffer-unmark-all): When MARK is not ?\r remove
>  just MARK.
>
> no hint about why it was necessary.

Sorry for not being more clear in my commit, and thanks for mentioned it,
so i can try do it better next time.

IMO, from this commit one emacs developper should conclude:

1) Its a change in file: lisp/ibuffer.el
2) Affects a func. to unmark buffers.
3) Affects the behaviour when an input arg (MARK) satisfy one condition
   (not equal ?\r): it will remove (unmark) just buffers with mark being 
MARK.
*) Fix? Enhancement? Unclear.

After that, a reader interested in the commit could check the diffs: i 
agree that in this case the diffs are not helpful to understand the 
necessity of the change.

Then, a really interested reader could easily put the point in the func.
name between parens and do: F1-f RET
(That automatically confirms 2).

After my commit the function spread 29 lines: it fully matches
within my laptop screen (with average screen size).
The first 3 lines help to understand the thing:
'(defun ibuffer-unmark-all (mark)
   "Unmark all buffers with mark MARK."
   (interactive "cRemove marks (RET means all):")
'
Basically, a command unmarking buffers marked with MARK; if MARK equals 
?\r
then all marked buffers are unmarked.  This is what is written in the 
commit, so the reader should conclude that this is a fix instead of an 
enhancement.

I don't know how much average time could take this for an experience emacs
developer; negligible i would say: probably Eli can do this almost while
taking a nap.

I got the write permissions in this project last month and i am still
a newby in git.  For my lack of experience my commits are far to be
perfect.  Sorry.   I am sure they will be better and better just with the 
practice.

Tino

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This can easily be encoded in a database table so it's searchable,
  > indexed by symbol name or file name (to build a history), etc. as part
  > of the pull request system. It can be semi-automatic: the system figures
  > out the file and symbol changes, then the developer adds the rest (and
  > the review doesn't end until this is done).

Given a program that can determine which function or entity name
to associate with each line, it would not be hard to make a list of
which entities are changed in a given patch.

Do we have such a program?  I don't know that we do.

However, determining what kind of change was made to a certain entity
in a certain patch is much harder.  Consider this:

* file.c (create_swimming_pool): New function
(modernize_building): Call it.

How would a program decide whether "Call it." is enough to say about
the change in modernize_building, or whether it is necessary to say
more?

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > They use the version control system. This provides different information
  > from the changelogs, of course. Worse in some cases, but better in
  > others. A function that is renamed, for example, but stays in the same
  > place in the file is better tracked by VC than a changelog.

With all due respect, I think that is not true, not even close to
being true.  The change log is tremendously better in all cases than
any other method I've seen.  It is hard work to recover the change log
from a diff.

A diff doesn't generally show the names of the entities that it
changes.  It takes human effort have to see which entities are changed
in any given checkin; thus, it takes lots and lots of human effort to
find which checkins to the file affected any given entity.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   Remove just input mark

  >   * lisp/ibuffer.el (ibuffer-unmark-all): When MARK is not ?\r remove
  >   just MARK.

  > no hint about why it was necessary.

That explanation should be in comments in the code.

  > About 3 commits below:

  >   Disable App Nap (bug#22993)

  >   * nextstep/templates/Info.plist.in: Insert AppNap disable code.

  > No hint about why it was necessary, except for the bug#

Likewise.

The point is, you should not have to go to any sort of log
to see the explanations for why the code needs to work this way.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Sat, 09 Jul 2016 12:54:48 -0400
> 
> Given a program that can determine which function or entity name
> to associate with each line, it would not be hard to make a list of
> which entities are changed in a given patch.
> 
> Do we have such a program?  I don't know that we do.

GDB can do that.

Re: Is it time to drop ChangeLogs?

[Ted Zlatanov]

On Sat, 09 Jul 2016 12:54:48 -0400 Richard Stallman <rms@gnu.org> wrote: 

>> This can easily be encoded in a database table so it's searchable,
>> indexed by symbol name or file name (to build a history), etc. as part
>> of the pull request system. It can be semi-automatic: the system figures
>> out the file and symbol changes, then the developer adds the rest (and
>> the review doesn't end until this is done).

RS> Given a program that can determine which function or entity name
RS> to associate with each line, it would not be hard to make a list of
RS> which entities are changed in a given patch.

RS> Do we have such a program?  I don't know that we do.

RS> However, determining what kind of change was made to a certain entity
RS> in a certain patch is much harder.  Consider this:

RS> * file.c (create_swimming_pool): New function
RS> (modernize_building): Call it.

RS> How would a program decide whether "Call it." is enough to say about
RS> the change in modernize_building, or whether it is necessary to say
RS> more?

You're right, of course. But I wasn't saying the program would do the
entire job. It will know *what* has changed and provide a place to enter
the *why* messages. Then the code reviewer will require those messages
before approving the merge. That's essentially what ChangeLogs do, but
this will be in a structured format so it's easier to index and analyze
the data.

A good first iteration of this would be a mechanism to generate the
structured data from a patch/diff, which would then hook into the
current mechanism to generate ChangeLog-style entries in the commit
message. (Currently this is a mostly manual process, I believe.)

That would benefit everyone right away by speeding up the ChangeLog
format generation, and can then be used for the more elaborate system
when and if we decide.

Ted

Re: Is it time to drop ChangeLogs?

[Dmitry Gutov]

On 07/09/2016 10:00 AM, Eli Zaretskii wrote:

> I do use "git annotate" at first, but it will only show the last
> commit that changed the code you are investigating, and I find this
> too frequently to be some reformatting or other similar cleanup that
> is not really what I want.

On the other hand, region-history will show you all commits that touched 
a given region. And you'll have to page through all of them if the 
interesting change happened long ago.

> Then going back in history with "git
> annotate" is inconvenient, so "git log -L" is better.

I find `a' very convenient.

> In any case, the startup times of 'annotate' and 'log -L' are
> comparable, and both are non-negligible.

Sure, but grepping through all NEWS or ChangeLog files doesn't happen 
instantly either.

>>> This request is unreasonable.  If nothing else, it will make the bar
>>> for contributing higher, not lower.  The information is recorded in
>>> the bug discussion, and there's no need to reproduce it in the log
>>> message.
>>
>> We do encourage that, actually, though not require (bar for
>> contributing, etc). This can be especially useful when the issue is
>> not-to-complex, but the bug discussion is long.
>>
>> Mentioned near "rationale for a change" in CONTRIBUTE.
>
> Right, but this discussion is about the requirements, not about the
> stuff for which one gets extra bonus points.

The arguments started with "ChangeLogs take a long time to write", but 
somehow mutated into "committers don't put the information I'd want into 
the entries".

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> Cc: ofv@wanadoo.es, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 10 Jul 2016 02:04:55 +0300
> 
> > In any case, the startup times of 'annotate' and 'log -L' are
> > comparable, and both are non-negligible.
> 
> Sure, but grepping through all NEWS or ChangeLog files doesn't happen 
> instantly either.

Its startup time is virtually zero, and then it's just browsing
through results, like with the other methods.  A failure to find
anything is very fast, unlike with the other methods.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It is true that some hackers write good commit messages *and*
  > ChangeLogs, but I'm under the impression that most people just write the
  > changelog and consider that their job is done (after all, current policy
  > for commit messages is to write a summary line and the 

There must be some misunderstanding.  Nowadays, in Emacs, there is
only one log entry.  It is supposed to contain an overall summary,
followed by the details of which entities were changed and how.

  > Absolutely agreed. That's about putting each piece where it belongs.
  > Some times, however, the code is not the best place to put some
  > explanation (think of an scattered change or why some alternative that,
  > at first, seems more adequate was discarded) and the commit message
  > offers an alternative.

When I wanted to put an explanation in comments in the code,
I could always find a natural place to put it.

  > > There was no such thing as a commit message in 1985.

  > Exactly. I'm pretty sure that if you had a good VCS at that time
  > ChangeLogs would never came into being.

We can't tell what _would have_ happened, but all the time I have used
a VCS I still wanted changelogs just as much as ever.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > You're right, of course. But I wasn't saying the program would do the
  > entire job. It will know *what* has changed and provide a place to enter
  > the *why* messages. Then the code reviewer will require those messages
  > before approving the merge. That's essentially what ChangeLogs do, but
  > this will be in a structured format so it's easier to index and analyze
  > the data.

That sounds like an idea worth trying to implement.
If it works well, it could indeed be helpful.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > This can easily be encoded in a database table so it's searchable,
>   > indexed by symbol name or file name (to build a history), etc. as part
>   > of the pull request system. It can be semi-automatic: the system figures
>   > out the file and symbol changes, then the developer adds the rest (and
>   > the review doesn't end until this is done).
>
> Given a program that can determine which function or entity name
> to associate with each line, it would not be hard to make a list of
> which entities are changed in a given patch.
>
> Do we have such a program?  I don't know that we do.

I use magit to create the structure for change log from the diff,
interactively during the commit process. It's a little clunky, but it
works okay.


Phil

auto-generating skeleton ChangeLogs (was: Is it time to drop ChangeLogs?)

[Ted Zlatanov]

On Sun, 10 Jul 2016 10:25:31 -0400 Richard Stallman <rms@gnu.org> wrote: 

RS> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
RS> [[[ whether defending the US Constitution against all enemies,     ]]]
RS> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]

>> You're right, of course. But I wasn't saying the program would do the
>> entire job. It will know *what* has changed and provide a place to enter
>> the *why* messages. Then the code reviewer will require those messages
>> before approving the merge. That's essentially what ChangeLogs do, but
>> this will be in a structured format so it's easier to index and analyze
>> the data.

RS> That sounds like an idea worth trying to implement.
RS> If it works well, it could indeed be helpful.

On Mon, 11 Jul 2016 12:29:38 +0100 phillip.lord@russet.org.uk (Phillip Lord) wrote: 

PL> I use magit to create the structure for change log from the diff,
PL> interactively during the commit process. It's a little clunky, but it
PL> works okay.

That's what I do as well. It's very visual and I sometimes miss things.

Can we do better? What are the things we can catch, given a patch or VCS
commit and the VCS-controlled file?

* additions of variables: defvar, defcustom, ???
* additions of functions: defun, defun*, ???
* removals of variables and functions
* changes to defcustoms or defvars
* changes to functions
* other top-level macros such as `define-minor-mode'
* new files, deleted files

If there is code already written for this in Emacs or other packages,
great. If not, I guess it should be part of the VC mode?

Ted

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I use magit to create the structure for change log from the diff,
  > interactively during the commit process. It's a little clunky, but it
  > works okay.

You can use Magit if you want to, but we shouldn't
recommend Magit as part of our recommended procedures
for GNU, or for Emacs, unless/until we can include it
in Emacs.

If we are considering development of programs other than Emacs,
a recommendation that only works if the person uses Emacs
has a drawback.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: auto-generating skeleton ChangeLogs

[Ted Zlatanov]

On Mon, 11 Jul 2016 09:28:52 -0400 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> Can we do better? What are the things we can catch, given a patch or VCS
TZ> commit and the VCS-controlled file?

TZ> * additions of variables: defvar, defcustom, ???
TZ> * additions of functions: defun, defun*, ???
TZ> * removals of variables and functions
TZ> * changes to defcustoms or defvars
TZ> * changes to functions
TZ> * other top-level macros such as `define-minor-mode'
TZ> * new files, deleted files

TZ> If there is code already written for this in Emacs or other packages,
TZ> great. If not, I guess it should be part of the VC mode?

...another approach I thought of: step through every - or + line of the
patch and do `add-change-log-entry'. The functionality to detect the
current function or symbol changed will have to be factored out, so we
can use a "change" data structure instead of inserting the ChangeLog entry.

Then walk through the list of changes and consolidate them into a tree
structure. That would be the minimum effort implementation...

Would that cover enough of the cases above?

Ted

Re: auto-generating skeleton ChangeLogs

[Stefan Monnier]

FWIW there is diff-add-change-log-entries-other-window.
I don't think it's very good, but it's a starting point.


        Stefan

Re: Is it time to drop ChangeLogs?

[Andreas Röhler]

On 09.07.2016 18:55, Richard Stallman wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>    > They use the version control system. This provides different information
>    > from the changelogs, of course. Worse in some cases, but better in
>    > others. A function that is renamed, for example, but stays in the same
>    > place in the file is better tracked by VC than a changelog.
>
> With all due respect, I think that is not true, not even close to
> being true.  The change log is tremendously better in all cases than
> any other method I've seen.  It is hard work to recover the change log
> from a diff.
>
> A diff doesn't generally show the names of the entities that it
> changes.  It takes human effort have to see which entities are changed
> in any given checkin; thus, it takes lots and lots of human effort to
> find which checkins to the file affected any given entity.
>

Agree as a person who changed his opinion several times in this point:

pure diff-logs don't provide a comparable information.
Given an environment like Emacs, where communication is an important 
issue, a changelog seems worth its effort.

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> With all due respect, I think that is not true, not even close to
> being true.  The change log is tremendously better in all cases than
> any other method I've seen.  It is hard work to recover the change log
> from a diff.

But it's not like the VCS only has diffs to offer.
"git log" gives pretty much the same info as the ChangeLog.


        Stefan

Re: Is it time to drop ChangeLogs?

[Robert Weiner]

On Wed, Jul 13, 2016 at 8:37 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
> "git log" gives pretty much the same info as the ChangeLog.

Hi Stefan:

This seems very hard to believe, as a ChangeLog should include
comments about the rationale for changes and should separate out each
change.  If I make 3 functional changes in one commit which I think
would be common in many projects, each change across multiple files,
the ChangeLog reflects 3 separate changes and can point out the files
and identifiers involved in each change.  How can an automated system
do this (it would have to understand the semantics of the code)?

Here is a short excerpt from Hyperbole's ChangeLog as an example.  Can
you show a git log example with similar functionality?

* hypb.el (hypb:maximize-window-height): Added.
  hib-debbugs.el (debbugs-gnu-mode, smart-debbugs-gnu,
debbugs-gnu-mode:help): Added to support
    the Smart Keys in Gnu Debbugs listing buffers.

* hversion.el (id-info, id-info-item): Generalized and improved file handling.

* hypb.el (hypb:format-quote): Added.
  hmouse-drv.el (hkey-debug): Called above function to protect
existing % fields in ButLabel
    and Actions from affecting the message call.

* DEMO (Hyperbole Menus): Updated with these new menu control features.
  hui-mouse.el (hkey-alist): Same changes for Smart Mouse Key as below
in hui-mini.el.
  hui-mini.el (hui:menu-select): Changed to make a press of RET within
a menu prefix (before a '>')
    return to the top-level menu and a press at the end of the menu,
quit the menu.

Bob

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> This seems very hard to believe, as a ChangeLog should include
> comments about the rationale for changes and should separate out each
> change.

The ChangeLog should contain what we decide it should contain.
The "git log" should contain what we decide it should contain.

So it's actually easy to make sure "git log" gives the same info: just
decide that it should be so (and try to enforce it).

In Emacs we indeed decided that it should be so and we make an effort to
enforce it.  The only thing really missing to get rid of the remaining
differences is a way to fix "git log" errors (typo and omissions,
typically) after the fact.


        Stefan

Re: Is it time to drop ChangeLogs?

[Robert Weiner]

On Wed, Jul 13, 2016 at 9:43 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
> The ChangeLog should contain what we decide it should contain.
> The "git log" should contain what we decide it should contain.

Ok, I see.  You are talking about simply having ChangeLog entries as
part of the version control system, not having the system generate
ChangeLog entries from the code changes.  It seems at some point the
latter was being discussed.

Personally, I like having a single file that I can reference for major
changes to a program and another file for news about such changes
without needing a specific tool to examine these changes.  But if the
version control system can generate such a file when distributions are
made, then we are pretty much talking about the same thing, with the
added bonus of being able to extract different snapshots of focused
changes based on the tool and the code versions.

Bob

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 13 Jul 2016 08:37:36 -0400
> 
> > With all due respect, I think that is not true, not even close to
> > being true.  The change log is tremendously better in all cases than
> > any other method I've seen.  It is hard work to recover the change log
> > from a diff.
> 
> But it's not like the VCS only has diffs to offer.
> "git log" gives pretty much the same info as the ChangeLog.

Only because we format commit log messages as we did with ChangeLog.
The issue at hand was to stop requiring that.

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> Ok, I see.  You are talking about simply having ChangeLog entries as
> part of the version control system, not having the system generate
> ChangeLog entries from the code changes.

No, I don't think anybody would argue to drop commit messages.
While I do like it when as much as possible of the info is placed
directly in the code, the commit message provides information which
seems pretty hard to auto-generate since it generally requires a much
higher-level understanding of the intention behind the code.


        Stefan

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

> Only because we format commit log messages as we did with ChangeLog.
> The issue at hand was to stop requiring that.

Our commit messages aren't great.
I don't think dropping the only requirement we have for them is going
to help.

So, it seems any discussion about such a topic should start by
discussing what we'd use *instead*.


        Stefan

Re: Is it time to drop ChangeLogs?

[Robert Weiner]

On Wed, Jul 13, 2016 at 11:33 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
> Our commit messages aren't great.

What do you think are the main reasons for this?

Bob

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

>> Our commit messages aren't great.
> What do you think are the main reasons for this?

The main reason I can see is that we can't fix them after-the fact
because Git doesn't support that feature.

We could try and be more strict on "write access", so as to try and
check the commit message quality before the commit is pushed, which
would help reduce the need to fix them after the fact, but there will
always be cases where we'd want to improve them after the fact.


        Stefan

Re: Is it time to drop ChangeLogs?

[Robert Weiner]

On Wed, Jul 13, 2016 at 11:47 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
> The main reason I can see is that we can't fix them after-the fact
> because Git doesn't support that feature.

Yes, this is a common need and Git should be changed to allow editing
of commit log entries even though this could be said to change the
state of the prior releases.

Bob

Re: Is it time to drop ChangeLogs?

[Stefan Monnier]

>> The main reason I can see is that we can't fix them after-the fact
>> because Git doesn't support that feature.
> Yes, this is a common need and Git should be changed to allow editing
> of commit log entries even though this could be said to change the
> state of the prior releases.

This has been hashed and rehashed: no, you don't need to change history
for that.  You only need to make "git log" pay attention to erratas
added later on.

IOW, if you "git checkout emacs-24.1; git log" you'd get the commit
entries as they were when the emacs-24.1 commit-id was committed, which
may be different from the entries you see if you do "get checkout master;
git log" since some of the old entries may have been fixed in the mean time.


        Stefan

Re: Is it time to drop ChangeLogs?

[Sven Axelsson]

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]

On 13 July 2016 at 18:54, Robert Weiner <rsw@gnu.org> wrote:

> On Wed, Jul 13, 2016 at 11:47 AM, Stefan Monnier
> <monnier@iro.umontreal.ca> wrote:
> > The main reason I can see is that we can't fix them after-the fact
> > because Git doesn't support that feature.
>
> Yes, this is a common need and Git should be changed to allow editing
> of commit log entries even though this could be said to change the
> state of the prior releases.
>

A couple of months ago it was suggested to use `git replace --edit` for
that.
Did anyone try it out to see if it could be used in the Emacs workflow?

-- 
Sven Axelsson
++++++++++[>++++++++++>+++++++++++>++++++++++>++++++
>++++<<<<<-]>++++.+.++++.>+++++.>+.<<-.>>+.>++++.<<.
+++.>-.<<++.>>----.<++.>>>++++++.<<<<.>>++++.<----.

[-- Attachment #2: Type: text/html, Size: 1397 bytes --]

Re: Is it time to drop ChangeLogs?

[Phillip Lord]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Date: Wed, 13 Jul 2016 08:37:36 -0400
>> 
>> > With all due respect, I think that is not true, not even close to
>> > being true.  The change log is tremendously better in all cases than
>> > any other method I've seen.  It is hard work to recover the change log
>> > from a diff.
>> 
>> But it's not like the VCS only has diffs to offer.
>> "git log" gives pretty much the same info as the ChangeLog.
>
> Only because we format commit log messages as we did with ChangeLog.
> The issue at hand was to stop requiring that.

I would suggest not the idea was not to no longer require changelog
format, but to replace it with something else.

There are two main problems with ChangeLog format -- first, it's just a
bit of a pain splitting the information into changes associated with
individual functions. The second, more substantive issue, is that it
focuses the mind too much on how we have changed things rather than why,
at least in my experience.

For myself, it's a pretty minor issue compared to having some pull
request queue software, so I will comment no further.


Phil

Re: Is it time to drop ChangeLogs?

[Eli Zaretskii]

> From: phillip.lord@russet.org.uk (Phillip Lord)
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
> Date: Thu, 14 Jul 2016 12:56:34 +0100
> 
> I would suggest not the idea was not to no longer require changelog
> format, but to replace it with something else.

No one suggested any particular format, the only suggestion was to
relax the current requirements, AFAIR.

> There are two main problems with ChangeLog format -- first, it's just a
> bit of a pain splitting the information into changes associated with
> individual functions.

Stefan mention the feature of diff-mode, which IMO is very convenient
if you run it on the output of "git diff" or something similar.

> The second, more substantive issue, is that it focuses the mind too
> much on how we have changed things rather than why, at least in my
> experience.

It's up to us.  Just to show you a random example to the contrary:

  commit fbfd4787fbec11fe69f5e661e7c5ab4436f7cb72
  Author: Eli Zaretskii <eliz@gnu.org>
  Date:   Tue May 24 19:04:16 2016 +0300

      Avoid aborting due to errors in arguments of 'set-face-attribute'

      * src/xfaces.c (Finternal_set_lisp_face_attribute): Check the
      FRAME argument before using it.  This avoids gratuitous aborts in
      Emacs built with --enable-checking when the luser was unlucky
      enough to get the argument list in wrong order.

You will find quite a few similar examples in Paul's commits, for
example.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > With all due respect, I think that is not true, not even close to
  > > being true.  The change log is tremendously better in all cases than
  > > any other method I've seen.  It is hard work to recover the change log
  > > from a diff.

  > But it's not like the VCS only has diffs to offer.
  > "git log" gives pretty much the same info as the ChangeLog.

I think we are miscommunicating.  The data we formerly put in the
ChangeLog file, we now put into the git commit log, and 'git log' is
now the way to look at it.  It is the same data, and we can use it the
same way.  But that's not the issue now.

Unless I have misunderstood, someone is now arguing that we shouldn't
write that information _at all, anywhere_.

That is what I oppose.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Is it time to drop ChangeLogs?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > There are two main problems with ChangeLog format -- first, it's just a
  > bit of a pain splitting the information into changes associated with
  > individual functions. The second, more substantive issue, is that it
  > focuses the mind too much on how we have changed things rather than why,
  > at least in my experience.

We could ask people to give bigger overall explanations in the log
entry.  However, those should be in addition to, not instead of,
explaining in comments in the code.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

debbugs tracker builds character (was: Is it time to drop ChangeLogs?)

[Ted Zlatanov]

On Fri, 08 Jul 2016 09:27:55 -0400 Ted Zlatanov <tzz@lifelogs.com> wrote: 

TZ> On Fri, 8 Jul 2016 02:50:31 +0300 Dmitry Gutov <dgutov@yandex.ru> wrote: 

DG> That said, before continuing this discussion, I'd rather we switch to a
DG> modern-ish bug tracker first.

TZ> I'll try using the (painful) bug tracker to do code reviews.

Unsurprisingly, it was not a good experience.

TZ> Please consider me strongly in favor of a better code review system and
TZ> a better bug tracker. Ideally they would be one and the same.

I wish I knew why people find the current bug tracker acceptable. It
reminds me of the old ftpmail/DEC gatekeeper where you sent e-mails
requesting files:

from ftp://ftp.math.utah.edu/pub/dec-alpha/dec-docs.txt

"Instructions for using ftpmail to copy files:

    1. Address the mail message to ftpmail@gatekeeper.dec.com.

    2. Ignore the subject line.

    3. Include in the first line of the body the word "connect".

    4. Include get commands for each document required.
	(for example: get /pub/DEC/DECinfo/infosheet/decnsr-v2p0.ps)

    5. The body of a typical request might look like:

	connect
	get /pub/DEC/DECinfo/SPD/index
	get /pub/DEC/DECinfo/infosheet/decnsr-v2p0.ps
	get /pub/DEC/DECinfo/brochure/unix-from-digital.txt

    Note that results will be mailed to you later in the day as requests
    are queued up and processed every 30 minutes.  For more timely access,
    consider using anonymous ftp."

That was cool in 1992, yeah... and it certainly builds character.

Ted

Re: debbugs tracker builds character

[Lars Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I wish I knew why people find the current bug tracker acceptable.

The web interface?  No, that's not a very pleasant interface, but I
think `M-x debbugs-gnu' is a better bug tracker interface than most.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: debbugs tracker builds character

[Dmitry Gutov]

On 07/20/2016 04:44 PM, Lars Ingebrigtsen wrote:

> The web interface?  No, that's not a very pleasant interface, but I
> think `M-x debbugs-gnu' is a better bug tracker interface than most.  :-)

I think it's worse than some.

It does makes some things easier once you know how to do them, but the 
UI takes getting used to, and it does not compensate for debbugs's 
sluggishness.

Nor does it add many missing features, some of which many developers 
have come to consider essential (such as VCS integration, for example).

Re: debbugs tracker builds character

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> TZ> Please consider me strongly in favor of a better code review system and
> TZ> a better bug tracker. Ideally they would be one and the same.
>
> I wish I knew why people find the current bug tracker acceptable. It
> reminds me of the old ftpmail/DEC gatekeeper where you sent e-mails
> requesting files:

One of the requirements for a bug tracker for Emacs was, that it must be
possible to control everything via email. That's one of the reasons
debbugs was chosen.

> Ted

Best regards, Michael.

Re: debbugs tracker builds character

[Robert Weiner]

[-- Attachment #1: Type: text/plain, Size: 457 bytes --]

On Wed, Jul 20, 2016 at 10:53 AM, Michael Albinus <michael.albinus@gmx.de>
wrote:

> One of the requirements for a bug tracker for Emacs was, that it must be
> possible to control everything via email. That's one of the reasons
> debbugs was chosen.
>

​Just curious if anyone looked at Roundup, the Python-based bug tracker
when choosing debbugs.  I always found it small, fast and easy to use with
both email and web-based control.​

Bob

[-- Attachment #2: Type: text/html, Size: 1105 bytes --]

Re: debbugs tracker builds character

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 07/20/2016 04:44 PM, Lars Ingebrigtsen wrote:
>
>> The web interface?  No, that's not a very pleasant interface, but I
>> think `M-x debbugs-gnu' is a better bug tracker interface than most.  :-)
>
> I think it's worse than some.
>
> It does makes some things easier once you know how to do them, but the
> UI takes getting used to, and it does not compensate for debbugs's
> sluggishness.
>
> Nor does it add many missing features, some of which many developers
> have come to consider essential (such as VCS integration, for
> example).

Patches welcome! Or at least precise proposals what you would like to
get improved. "VCS integration" sounds too vague to me.

Best regards, Michael.

Re: debbugs tracker builds character

[Ted Zlatanov]

On Wed, 20 Jul 2016 16:53:45 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:
>> I wish I knew why people find the current bug tracker acceptable. It
>> reminds me of the old ftpmail/DEC gatekeeper where you sent e-mails
>> requesting files:

MA> One of the requirements for a bug tracker for Emacs was, that it must be
MA> possible to control everything via email. That's one of the reasons
MA> debbugs was chosen.

IMHO that's not an actual requirement. It's an implementation detail.

The actual requirement was probably about disconnected operation or
something like that. It must accomplish something from the user's
viewpoint (and sending e-mail, in itself, does not accomplish anything).

On Wed, 20 Jul 2016 15:44:03 +0200 Lars Ingebrigtsen <larsi@gnus.org> wrote: 

LI> The web interface?  No, that's not a very pleasant interface, but I
LI> think `M-x debbugs-gnu' is a better bug tracker interface than most.  :-)

There's a separation between the UI and the underlying functionality.

IMHO, the underlying bug tracker is extremely limited.

IMVHO, the `M-x debbugs-gnu' UI is decent but only because of a lot of
hacking, and the same UI could be applied to other bug trackers. In
other words, the reasons why it's good are not due to the bug tracker. I
could be wrong on this, though, I don't know the internals well, I've
just looked through the code.

On Wed, 20 Jul 2016 16:58:24 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Patches welcome! Or at least precise proposals what you would like to
MA> get improved. "VCS integration" sounds too vague to me.

The bug tracker should be aware of repositories, branches, commits,
contributors, and ticket links or mentions in commit messages.
Contributors should be able to tag and notify each other. Markdown etc.
should be well supported. Inline code comments should be easy, and
linked to a commit (so an updated commit can resolve the comment). This
is just the essential stuff.

I think this is not something you can solve with patches or good UI. It
requires a tool architected correctly from the start. Such tools exist aplenty.

Ted

Re: debbugs tracker builds character

[Stefan Monnier]

> IMHO that's not an actual requirement. It's an implementation detail.

I think at the time, working similarly to the old bug-gnu-emacs
mailing-list was a clear desire.

And I think it's probably still the case that we'd want our bug-tracker to
be usable via email for those who want to (tho it's probably OK if only
the main functions are available via this UI).

> The actual requirement was probably about disconnected operation or
> something like that.

Yes, I agree that this is the crucial point.  That's my motivation for
developing BugIt (https://gitlab.com/monnier/bugit).

I haven't had much time to devote to it lately, tho, so it's stuck at
the stage of "proof of concept" right now.

> The bug tracker should be aware of repositories, branches, commits,
> contributors, and ticket links or mentions in commit messages.

I've never seen a bug-tracker do anything really useful with those
(other than what you can get by embedding URLs in the bug
description/discussion), so I'd be interested to hear more (tho it could
be difficult to retro-fit it into BugIt since BugIt is designed to be
fundamentally an issue/ticket-tracking system not necessarily related to
"code" or to any kind of VCS repository).

> Contributors should be able to tag and notify each other.

You mean to (re)assign bugs to particular persons and things like that?

> Markdown etc. should be well supported.

Right.  In BugIt I decided to skip the "etc." part, tho.

> Inline code comments should be easy, and linked to a commit (so an
> updated commit can resolve the comment).

How do you "update a commit"?  What does "resolve a comment" mean?

> I think this is not something you can solve with patches or good UI.
> It requires a tool architected correctly from the start.  Such tools
> exist aplenty.

Do you have a recommendation of something you consider well-designed
(not necessarily for Emacs's use, so I could look at it)?


        Stefan

Re: debbugs tracker builds character

[Ted Zlatanov]

On Wed, 20 Jul 2016 12:56:54 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> IMHO that's not an actual requirement. It's an implementation detail.
SM> I think at the time, working similarly to the old bug-gnu-emacs
SM> mailing-list was a clear desire.

SM> And I think it's probably still the case that we'd want our bug-tracker to
SM> be usable via email for those who want to (tho it's probably OK if only
SM> the main functions are available via this UI).

>> The actual requirement was probably about disconnected operation or
>> something like that.

SM> Yes, I agree that this is the crucial point.  That's my motivation for
SM> developing BugIt (https://gitlab.com/monnier/bugit).

SM> I haven't had much time to devote to it lately, tho, so it's stuck at
SM> the stage of "proof of concept" right now.

I'm not against a custom solution, but it's hard to justify the cost and
effort compared to something more standard. Do you think BugIt can
provide the features we've mentioned at some point?

>> The bug tracker should be aware of repositories, branches, commits,
>> contributors, and ticket links or mentions in commit messages.

SM> I've never seen a bug-tracker do anything really useful with those
SM> (other than what you can get by embedding URLs in the bug
SM> description/discussion), so I'd be interested to hear more (tho it could
SM> be difficult to retro-fit it into BugIt since BugIt is designed to be
SM> fundamentally an issue/ticket-tracking system not necessarily related to
SM> "code" or to any kind of VCS repository).

Well, clearly the level of integration can vary. JIRA for instance
doesn't have the deep Github integration that Github's issue tracker
offers, but it's also more flexible. People have written thick books on
these topics.

For me personally, if I can *see* the specific code that fixes a ticket
inside the ticket as a commit, and click my way to the wider commit and
then diff from before that commit against today's state of that code,
I've built a mental map of the code that would otherwise take me a lot
of work. That's one common workflow. Another is to view several commits
that fix a single ticket in one place. So it's not revolutionary, just
simpler and more straightforward for the user.

>> Contributors should be able to tag and notify each other.

SM> You mean to (re)assign bugs to particular persons and things like that?

Yes, plus ping someone or a team specifically: "hey, maybe the @gnus
team should look at this" in a comment.

>> Inline code comments should be easy, and linked to a commit (so an
>> updated commit can resolve the comment).

SM> How do you "update a commit"?  What does "resolve a comment" mean?

Rebase or amend+force push would update a branch destructively, which in
a pull request context should show you that a comment was for a commit
that's no longer in the branch. Furthermore some trackers allow you to
mark a comment as resolved (e.g. Github recently added reactions, which
can be used as ad-hoc markup).

The next link in the chain are CI/CD hooks. You can set up a Github
repo, for instance, to build every pull request before the reviewer ever
looks, which saves a lot of time with compiled languages. It will run
tests and so on, but most important is that it keeps the context inside
the pull request, you don't have to go elsewhere.

>> I think this is not something you can solve with patches or good UI.
>> It requires a tool architected correctly from the start.  Such tools
>> exist aplenty.

SM> Do you have a recommendation of something you consider well-designed
SM> (not necessarily for Emacs's use, so I could look at it)?

Github, Gitlab, BitBucket come immediately to mind as highly integrated
environments that could host Emacs development. But they all have
drawbacks you know. Gitlab seems closest to "acceptable" for the Emacs
team but it's not GPL:
https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE

Ted

Re: debbugs tracker builds character

[Michael Albinus]

Robert Weiner <rsw@gnu.org> writes:

Hi Bob,

> ​Just curious if anyone looked at Roundup, the Python-based bug tracker
> when choosing debbugs.

Not that I'm aware of.

> I always found it small, fast and easy to use with both email and
> web-based control.​

Beside using email for bug handling, there were are criteria for
decision. One essential requirement is always the underlying license;
don't know whether ZPL 2.0 used by Roundup is sufficient for the
recommend bug tracker of Gnu projects.

Anyway, I believe this ship has sailed years ago. If somebody wants to
replace debbugs, (s)he must do heavy lobbying for it. Don't know whether
it could succeed, and whether it is worth the trouble. The better
approach seems to be to exploit debbugs to its best. IMO.

> Bob

Best regards, Michael.

Re: debbugs tracker builds character

[Dmitry Gutov]

On 07/20/2016 08:54 PM, Ted Zlatanov wrote:

> For me personally, if I can *see* the specific code that fixes a ticket
> inside the ticket as a commit, and click my way to the wider commit and
> then diff from before that commit against today's state of that code,
> I've built a mental map of the code that would otherwise take me a lot
> of work. That's one common workflow. Another is to view several commits
> that fix a single ticket in one place. So it's not revolutionary, just
> simpler and more straightforward for the user.

Being able to close a bug just by mentioning it in a certain way in the 
commit message and pushing that commit is also handy. You don't have to 
switch to the bug discussion and duplicate that info there manually.

> SM> How do you "update a commit"?  What does "resolve a comment" mean?
>
> Rebase or amend+force push would update a branch destructively, which in
> a pull request context should show you that a comment was for a commit
> that's no longer in the branch. Furthermore some trackers allow you to
> mark a comment as resolved (e.g. Github recently added reactions, which
> can be used as ad-hoc markup).

Even if you don't rebase, but just push a new commit to the branch upon 
review, IIRC both Github and Gitlab can see that the changes that 
started a particular discussion are no longer there (and collapse the 
comment sub-thread a no longer relevant, while allowing the user to 
expand it again if they so wish).

And on the more basic level, compared to flat discussions in mailing 
lists, having separate subthread for each part of the patch the reviewer 
commented on, is great. You can have discussion sub-threads in the 
mailing list too, but people never split their emails in pieces that small.

Re: debbugs tracker builds character

[Dmitry Gutov]

On 07/20/2016 05:58 PM, Michael Albinus wrote:

>> Nor does it add many missing features, some of which many developers
>> have come to consider essential (such as VCS integration, for
>> example).
>
> Patches welcome! Or at least precise proposals what you would like to
> get improved. "VCS integration" sounds too vague to me.

Here's one example:

https://help.github.com/articles/closing-issues-via-commit-messages/

I don't think you can do anything like this using debbugs.

Re: debbugs tracker builds character

[Dmitry Gutov]

On 07/20/2016 07:56 PM, Stefan Monnier wrote:

> And I think it's probably still the case that we'd want our bug-tracker to
> be usable via email for those who want to (tho it's probably OK if only
> the main functions are available via this UI).

You usually can receive bug comments over email, and reply to them, but 
not create or close an issue. There are HTTP APIs for that, though.

>> The actual requirement was probably about disconnected operation or
>> something like that.
>
> Yes, I agree that this is the crucial point.  That's my motivation for
> developing BugIt (https://gitlab.com/monnier/bugit).

IMHO it's not that important a goal. The old-timers have lived without 
it for a while, and the youngsters have no idea that it's something to 
be desired.

Or, if you want to say that it's already available with debbugs because 
one can delay the delivery of their emails, an Emacs package for 
interacting with the new bug tracker could implement something like a 
queue of delayed actions inside it.

Being able to close bugs from commits (which you can push with a delay) 
would also lower the demand for such a feature.

Re: debbugs tracker builds character

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

Hi Ted,

>>> I wish I knew why people find the current bug tracker acceptable. It
>>> reminds me of the old ftpmail/DEC gatekeeper where you sent e-mails
>>> requesting files:
>
> MA> One of the requirements for a bug tracker for Emacs was, that it must be
> MA> possible to control everything via email. That's one of the reasons
> MA> debbugs was chosen.
>
> IMHO that's not an actual requirement. It's an implementation detail.

<http://thread.gmane.org/gmane.emacs.devel/60833/focus=60976>

That thread is the 10 years old discussion on emacs-devel, which ended
up in deciding for debbugs. And I find the reasoning for using email
still convincing.

> On Wed, 20 Jul 2016 15:44:03 +0200 Lars Ingebrigtsen <larsi@gnus.org> wrote:
>
> LI> The web interface?  No, that's not a very pleasant interface, but I
> LI> think `M-x debbugs-gnu' is a better bug tracker interface than most.  :-)
>
> There's a separation between the UI and the underlying functionality.
>
> IMHO, the underlying bug tracker is extremely limited.
>
> IMVHO, the `M-x debbugs-gnu' UI is decent but only because of a lot of
> hacking, and the same UI could be applied to other bug trackers. In
> other words, the reasons why it's good are not due to the bug tracker. I
> could be wrong on this, though, I don't know the internals well, I've
> just looked through the code.

In general, you are right. First there was debbugs.el, which is a
backend talking to the debbugs server (of Gnu or of Debian, both
works). debbugs-gnu.el (and debbugs-org.el) were written years later,
being just the UI, and being directed to the Gnu debbugs server. So
there exist already a separation, although the interface between
debbugs.el and debbugs-gnu.el is mainly driven by functions offered from
debbugs.el.

> On Wed, 20 Jul 2016 16:58:24 +0200 Michael Albinus
> <michael.albinus@gmx.de> wrote:
>
> MA> Patches welcome! Or at least precise proposals what you would like to
> MA> get improved. "VCS integration" sounds too vague to me.
>
> The bug tracker should be aware of repositories, branches, commits,
> contributors, and ticket links or mentions in commit messages.
> Contributors should be able to tag and notify each other. Markdown etc.
> should be well supported. Inline code comments should be easy, and
> linked to a commit (so an updated commit can resolve the comment). This
> is just the essential stuff.

Some of this exist already (tags), maybe underdocumented. For the rest I
must at least sleep one night :-)

> I think this is not something you can solve with patches or good UI. It
> requires a tool architected correctly from the start. Such tools exist aplenty.

A practical counter-argument: do you believe that debbugs is *such* bad
that the vast majority of Emacs developers will follow you for a new bug
tracker? I'm not a debbugs missionary, but in my daily work I found it
sufficient. Somehow.

> Ted

Best regards, Michael.

Re: debbugs tracker builds character

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Here's one example:
>
> https://help.github.com/articles/closing-issues-via-commit-messages/
>
> I don't think you can do anything like this using debbugs.

Why not? This seems to be rather a feature of the repository. If you
configure the (Emacs) git repository that it detects "Fixes #45"
patterns in the commit message, then it needs just an email to
control@debbugs.gnu.org with "close 45" in the body. Something, which
could be done by the commit-msg or post-commit scripts.

I'm not arguing to install this just now. But debbugs wouldn't be a
barrier to do it. No special debbugs configuration needed.

Best regards, Michael.

Re: debbugs tracker builds character

[Robert Weiner]

[-- Attachment #1: Type: text/plain, Size: 709 bytes --]

On Wed, Jul 20, 2016 at 3:48 PM, Michael Albinus <michael.albinus@gmx.de>
wrote:

> Why not? This seems to be rather a feature of the repository. If you
> configure the (Emacs) git repository that it detects "Fixes #45"
> patterns in the commit message, then it needs just an email to
> control@debbugs.gnu.org with "close 45" in the body. Something, which
> could be done by the commit-msg or post-commit scripts.
>

​If you implement anything like this, please use "Fixes bug#45." rather
than "Fixes #45", then a press of a key with point somewhere within
`bug#45' with the Hyperbole package installed will display the discussion
of that bug number or display its status, as desired.

Bob

[-- Attachment #2: Type: text/html, Size: 1445 bytes --]

Re: debbugs tracker builds character

[Michael Albinus]

Robert Weiner <rsw@gnu.org> writes:

> ​If you implement anything like this, please use "Fixes bug#45." rather
> than "Fixes #45", then a press of a key with point somewhere within
> `bug#45' with the Hyperbole package installed will display the
> discussion of that bug number or display its status, as desired.

Don't panic :-)

I've just quoted the example given on the Github page, and they've used
"Fixes #45". *If* we introduce something like this, I would be in favor
of the established bug#45 format.

Btw, do you know `debbugs-browse-mode'? (Shameless advertisement, I'm
the author of).

> Bob

Best regards, Michael.

Re: debbugs tracker builds character

[Dmitry Gutov]

On 07/20/2016 10:48 PM, Michael Albinus wrote:

>> https://help.github.com/articles/closing-issues-via-commit-messages/
>>
>> I don't think you can do anything like this using debbugs.
>
> Why not? This seems to be rather a feature of the repository. If you
> configure the (Emacs) git repository that it detects "Fixes #45"
> patterns in the commit message, then it needs just an email to
> control@debbugs.gnu.org with "close 45" in the body. Something, which
> could be done by the commit-msg or post-commit scripts.

Oh, ok. But it would have to email to the bug address because the person 
who filed it would surely want to see why and how it was closed. 
Preferably with a link to the closing commit.

A commit can also reference a bug (and thus show in the bug's 
discussion) without closing it.

> I'm not arguing to install this just now. But debbugs wouldn't be a
> barrier to do it. No special debbugs configuration needed.

I concede that this particular feature is indeed possible to add while 
still using debbugs.

Re: debbugs tracker builds character

[Stefan Monnier]

> Rebase or amend+force push would update a branch destructively, which in
> a pull request context should show you that a comment was for a commit
> that's no longer in the branch. Furthermore some trackers allow you to
> mark a comment as resolved (e.g. Github recently added reactions, which
> can be used as ad-hoc markup).

I think I'm starting to see what you mean.  You're talking about a tight
integration where a pull-request is also itself an issue, so the comments
can be directly on the patch itself.  As opposed to having issues and
pull-request be two separate things that can refer to each other via
an indirection.

So this is particularly useful/meaningful when reviewing a proposed
patch from another developer, rather than when interacting with an
end-user trying to track down some bugs here's experiencing (which is
the kind of use-case I've had in mind when working on BugIt).

But indeed, the two use-cases would best be served by the same tool
since after the bug is tracked a patch might show up to fix it, after
which a review process will come up.

Food for thought, thank you,


        Stefan

Re: debbugs tracker builds character

[Stefan Monnier]

>> Yes, I agree that this is the crucial point.  That's my motivation for
>> developing BugIt (https://gitlab.com/monnier/bugit).
> IMHO it's not that important a goal. The old-timers have lived without it
> for a while, and the youngsters have no idea that it's something to
> be desired.

The more I use BugIt, the more I find it important to be able to query
the bug database while offline.  Being able to queue actions so they're
executed when I get online is not too terrible, but it's not really
good enough.


        Stefan


PS: Of course, I'm not representative because I avoid cell-phone
technology, so I'm often offline.

Re: debbugs tracker builds character

[Eric Abrahamsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> TZ> Please consider me strongly in favor of a better code review system and
>> TZ> a better bug tracker. Ideally they would be one and the same.
>>
>> I wish I knew why people find the current bug tracker acceptable. It
>> reminds me of the old ftpmail/DEC gatekeeper where you sent e-mails
>> requesting files:
>
> One of the requirements for a bug tracker for Emacs was, that it must be
> possible to control everything via email. That's one of the reasons
> debbugs was chosen.

A suggestion from a part-time package maintainer: make
`report-emacs-bug' prompt for a package, and offer the results of
`featurep' as completion. If the user picks a package that only exists
in ELPA, email the package maintainer with the bug report.

I have no opinion about bug-tracking backends, the only reason I start
packages off in github is that the issue tracker is mature. But I would
prefer to be emailed via the debbugs interface. In fact, I would prefer
to add a debbugs server to Gnus, which I would ignore 99% of the time,
except when a bug came down that was reported against one of my
packages.

How far away from that are we?

Re: debbugs tracker builds character

[Christian Kruse]

[-- Attachment #1: Type: text/plain, Size: 391 bytes --]


Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Markdown etc. should be well supported.
>
> Right.  In BugIt I decided to skip the "etc." part, tho.

Github uses text from emails as-is and doesn't parse it as
Markdown/CommonMark. I don't say that this is the right way to go, but
it is worth a consideration.

Best regards,
-- 
Christian Kruse
https://wwwtech.de/about

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: debbugs tracker builds character

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

Hi Dmitry,

> Oh, ok. But it would have to email to the bug address because the
> person who filed it would surely want to see why and how it was
> closed. Preferably with a link to the closing commit.

No, sending the "close 45" action to control@debbugs.gnu.org triggers
also an email to the submitter of the bug. Adding the commit reference +
message shall also be possible.

Sending a commit link might be more tricky. The bug could belong to
another pakage but "emacs", debbugs cannot know the base of the package
repository.

> A commit can also reference a bug (and thus show in the bug's
> discussion) without closing it.

Sure. There must be an obvious indication that the bug shall be
closed. "Fixes #45" was just the example I've taken from the Github page
you've referenced to. It could be any other keyword.

Re: debbugs tracker builds character

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The more I use BugIt, the more I find it important to be able to query
> the bug database while offline.  Being able to queue actions so they're
> executed when I get online is not too terrible, but it's not really
> good enough.

debbugs-gnu has some half-assed support for offline usage which I've
used a couple of times.  It lacks hooks for just updating the bug
reports that have been changed lately, though, so you get out of sync
pretty quickly.  I couldn't find any way to ask debbugs "what bugs have
changed since time <foo>"...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: debbugs tracker builds character

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> In fact, I would prefer to add a debbugs server to Gnus, which I would
> ignore 99% of the time, except when a bug came down that was reported
> against one of my packages.
>
> How far away from that are we?

I don't think that's a massive amount of work, but it might be slow
because of the limited query capabilities on the debbugs side...  Hm...

I mean, querying per package is fast-ish (well, it's pretty slow, but
it's not unbearable), but seeing whether there's any new articles in all
the bug reports you're following would require polling each bug report,
and that's just s-l-o-w.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: debbugs tracker builds character

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> I've just quoted the example given on the Github page, and they've used
> "Fixes #45". *If* we introduce something like this, I would be in favor
> of the established bug#45 format.

It'd be nice if the language would be something along the lines of

bug#45 done
bug#45 tags 25.1 fixed

That is, that you could drop arbitrary debbugs commands into commit
messages.

Or something.  Perhaps that's over-engineering.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: debbugs tracker builds character

[Dmitry Gutov]

On 07/21/2016 09:27 AM, Michael Albinus wrote:

> Sending a commit link might be more tricky. The bug could belong to
> another pakage but "emacs", debbugs cannot know the base of the package
> repository.

Send a full URL?

>> A commit can also reference a bug (and thus show in the bug's
>> discussion) without closing it.
>
> Sure. There must be an obvious indication that the bug shall be
> closed. "Fixes #45" was just the example I've taken from the Github page
> you've referenced to. It could be any other keyword.

I mean that in this case the notification about the commit would also 
need to be added to the bug discussion, but the control server is not 
the way to do this, at least in this case.

Re: debbugs tracker builds character

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> The more I use BugIt, the more I find it important to be able to query
>> the bug database while offline.  Being able to queue actions so they're
>> executed when I get online is not too terrible, but it's not really
>> good enough.
>
> debbugs-gnu has some half-assed support for offline usage which I've
> used a couple of times.  It lacks hooks for just updating the bug
> reports that have been changed lately, though, so you get out of sync
> pretty quickly.  I couldn't find any way to ask debbugs "what bugs have
> changed since time <foo>"...

This might be possible by extending the SOAP interface. Not that I like
it, but we've changed the SOAP interface already, so we aren't
compatible with Debian's debbugs server anymore, for some operations.

Best regards, Michael.

Re: debbugs tracker builds character

[Eric Abrahamsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> In fact, I would prefer to add a debbugs server to Gnus, which I would
>> ignore 99% of the time, except when a bug came down that was reported
>> against one of my packages.
>>
>> How far away from that are we?
>
> I don't think that's a massive amount of work, but it might be slow
> because of the limited query capabilities on the debbugs side...  Hm...
>
> I mean, querying per package is fast-ish (well, it's pretty slow, but
> it's not unbearable), but seeing whether there's any new articles in all
> the bug reports you're following would require polling each bug report,
> and that's just s-l-o-w.

I guess it doesn't make too much sense to do it this way. If we're
getting emails for bug reports we're subscribed to, then the problem's
already solved.

The main thing would be the prompting for package, and automatic emails
to maintainers!

Re: debbugs tracker builds character

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I mean, querying per package is fast-ish (well, it's pretty slow, but
> it's not unbearable),

Packages in Emacs and packages in debbugs are not the same. Most of the
packages in Emacs are tracked via the debbugs "emacs" package. There are
exceptions, like "auctex", "gnus", "hyperbole" and "org-mode", which
exist also as debbugs packages.

Best regards, Michael.

Re: debbugs tracker builds character

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> This might be possible by extending the SOAP interface. Not that I like
> it, but we've changed the SOAP interface already, so we aren't
> compatible with Debian's debbugs server anymore, for some operations.

If you add such a function, I'll finish the offline implementation (and
document it :-)).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: debbugs tracker builds character

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> Sending a commit link might be more tricky. The bug could belong to
>> another pakage but "emacs", debbugs cannot know the base of the package
>> repository.
>
> Send a full URL?

I don't know the repositories of the different packages supported by the
debbugs server. <http://debbugs.gnu.org/Packages.html> gives you an idea
which packages are hosted.

>> Sure. There must be an obvious indication that the bug shall be
>> closed. "Fixes #45" was just the example I've taken from the Github page
>> you've referenced to. It could be any other keyword.
>
> I mean that in this case the notification about the commit would also
> need to be added to the bug discussion, but the control server is not
> the way to do this, at least in this case.

The control message is added to the bug. You simply don't see it in the
bug-gnu-emacs ML.

Best regards, Michael.

Re: debbugs tracker builds character

[Dmitry Gutov]

On 07/21/2016 03:42 PM, Michael Albinus wrote:

> I don't know the repositories of the different packages supported by the
> debbugs server. <http://debbugs.gnu.org/Packages.html> gives you an idea
> which packages are hosted.

I'm sure the repository knows its URL, or can be configured to. You 
suggested to implement this via a Git hook, right?

>> I mean that in this case the notification about the commit would also
>> need to be added to the bug discussion, but the control server is not
>> the way to do this, at least in this case.
>
> The control message is added to the bug. You simply don't see it in the
> bug-gnu-emacs ML.

If you don't see it in the web interface, the users can't track the 
progress, or get an overview of the commits later, just looking at the 
bug. Which is the whole point.

Re: debbugs tracker builds character

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Michael Albinus <michael.albinus@gmx.de> writes:
>
>> This might be possible by extending the SOAP interface. Not that I like
>> it, but we've changed the SOAP interface already, so we aren't
>> compatible with Debian's debbugs server anymore, for some operations.
>
> If you add such a function, I'll finish the offline implementation (and
> document it :-)).

I've feared that response ... not so much time. I'll check whether it
could be implemented easily. No promise, tho.

Best regards, Michael.

Re: debbugs tracker builds character

[Michael Albinus]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> I don't know the repositories of the different packages supported by the
>> debbugs server. <http://debbugs.gnu.org/Packages.html> gives you an idea
>> which packages are hosted.
>
> I'm sure the repository knows its URL, or can be configured to. You
> suggested to implement this via a Git hook, right?


Yes.

>>> I mean that in this case the notification about the commit would also
>>> need to be added to the bug discussion, but the control server is not
>>> the way to do this, at least in this case.
>>
>> The control message is added to the bug. You simply don't see it in the
>> bug-gnu-emacs ML.
>
> If you don't see it in the web interface, the users can't track the
> progress, or get an overview of the commits later, just looking at the
> bug. Which is the whole point.

You see control messages also in the web interface. See the line

<Toggle> the display of automated, internal messages from the tracker

in a bug's page. Not very convenient, I agree.

Re: debbugs tracker builds character

[Ted Zlatanov]

On Wed, 20 Jul 2016 20:48:07 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Anyway, I believe this ship has sailed years ago. If somebody wants to
MA> replace debbugs, (s)he must do heavy lobbying for it. Don't know whether
MA> it could succeed, and whether it is worth the trouble. The better
MA> approach seems to be to exploit debbugs to its best. IMO.
...
MA> <http://thread.gmane.org/gmane.emacs.devel/60833/focus=60976>

MA> That thread is the 10 years old discussion on emacs-devel, which ended
MA> up in deciding for debbugs. And I find the reasoning for using email
MA> still convincing.

I don't care about the underlying bug tracker as much as I care about
getting things done. This is my underlying reason to start a discussion.

(Right now, I can set up an Emacs mirror on Github, do a pull request
there and accept comments there. It would work great and I can merge the
branch back into the Emacs Git repo. The only reason I haven't is
political: I believe the maintainers should guide the tool choices.)

>> The bug tracker should be aware of repositories, branches, commits,
>> contributors, and ticket links or mentions in commit messages.
>> Contributors should be able to tag and notify each other. Markdown etc.
>> should be well supported. Inline code comments should be easy, and
>> linked to a commit (so an updated commit can resolve the comment). This
>> is just the essential stuff.

MA> Some of this exist already (tags), maybe underdocumented. For the rest I
MA> must at least sleep one night :-)

It's hard to do the things I listed (especially VCS integration) with
the current Emacs bug tracker. These things were not so important 10
years ago. I think as a team, we have to be willing to reassess our tool
choices at least every 10 years (e.g. Bazaar and then Git were major
tool changes but didn't change the Emacs project).

As a practical choice, I'd rather lobby to use better tools than to
write them ourselves: it wastes resources, and I don't think bug and
issue tracking is our strongest area. I'd rather have you (Michael and
others) get some sleep or work on Emacs itself.

MA> A practical counter-argument: do you believe that debbugs is *such* bad
MA> that the vast majority of Emacs developers will follow you for a new bug
MA> tracker? I'm not a debbugs missionary, but in my daily work I found it
MA> sufficient. Somehow.

Well, I'm partly basing my view on the vitality of ELPA packages on
Github. There are hundreds (some end up on the GNU ELPA, some hosted
elsewhere). I've contributed to several like
https://github.com/Silex/docker.el and
https://github.com/flycheck/flycheck and the process is better for the
reasons I've described.

Will the Emacs developers follow me? I'd love to see them use a better
process. I'm happy to concede I'm wrong and the current process works
fine, but the evidence leads me to believe it can be improved with
better tools (similar to the evidence in favor of Git when we were using
Bazaar). I'm not going to spend the next 6 months talking about it; I'd
rather know if the maintainers (Eli and John) and other core developers
are willing to consider Gitlab or something similar. I think I've stated
the case clearly, and quite a few of us have experience with more
powerful bug/issue trackers so we know what we're missing.

If the decision is "no, we'll move on with whatever marginal
improvements to debbugs we can make" then I'll come back to the topic in
another 10 years :)

On Wed, 20 Jul 2016 16:43:08 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>>> Yes, I agree that this is the crucial point.  That's my motivation for
>>> developing BugIt (https://gitlab.com/monnier/bugit).
>> IMHO it's not that important a goal. The old-timers have lived without it
>> for a while, and the youngsters have no idea that it's something to
>> be desired.

SM> The more I use BugIt, the more I find it important to be able to query
SM> the bug database while offline.  Being able to queue actions so they're
SM> executed when I get online is not too terrible, but it's not really
SM> good enough.

I don't disagree entirely, but in 2016 disconnected operation is much
less common than in 2006. There are tools that can synchronize database
replicas seamlessly. Fundamentally, though, I think the focus of bug and
issue tracking has shifted from individuals working on fixes, to teams
collaborating and reviewing each other's work, with the ability to
include and notify the entire community. IMHO this is the true value in
Github and its competitors, and the biggest reason why they've seen so
much adoption.

On Wed, 20 Jul 2016 16:40:01 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> I think I'm starting to see what you mean.  You're talking about a tight
SM> integration where a pull-request is also itself an issue, so the comments
SM> can be directly on the patch itself.  As opposed to having issues and
SM> pull-request be two separate things that can refer to each other via
SM> an indirection.

Yes. Actually Github has both of these right now. When pull requests are
not true issues, they tend to be associated with issues: sometimes
1-to-N and sometimes N-to-1. People have different workflows, so it's
good to have flexibility built into the model. Sorry if this seems
ambivalent; it's really a degree of creative freedom that users
appreciate.

On Wed, 20 Jul 2016 17:25:33 -0400 Eric Abrahamsen <eric@ericabrahamsen.net> wrote: 

EA> A suggestion from a part-time package maintainer: make
EA> `report-emacs-bug' prompt for a package, and offer the results of
EA> `featurep' as completion. If the user picks a package that only exists
EA> in ELPA, email the package maintainer with the bug report.

That's a really good suggestion, regardless of the underlying bug tracker.

Thanks for the thoughtful comments
Ted

Re: debbugs tracker builds character

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Thu, 21 Jul 2016 10:13:50 -0400
> 
> Will the Emacs developers follow me? I'd love to see them use a better
> process. I'm happy to concede I'm wrong and the current process works
> fine, but the evidence leads me to believe it can be improved with
> better tools (similar to the evidence in favor of Git when we were using
> Bazaar). I'm not going to spend the next 6 months talking about it; I'd
> rather know if the maintainers (Eli and John) and other core developers
> are willing to consider Gitlab or something similar.

If you expect an answer from me, then I don't have enough information
to do that.  I never worked with Gitlab, have only a casual
familiarity with Github, and don't have any clear idea of what would
that mean for the process.  Someone™ will have to set up a test
repository which can be freely played with, and provide enough
instructions for the relevant frequent ops to allow some first-hand
experience with Gitlab (or whatever).  Only then I will be able to say
something intelligent.

FWIW, most of the time I use up on Emacs-related work is spent
debugging and reviewing patches, not on stuff related to debbugs and
the VCS; the latter take negligible time.  Not sure what that means in
the context of this discussion.

Thanks.

Re: debbugs tracker builds character

[Ted Zlatanov]

On Thu, 21 Jul 2016 17:50:01 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Thu, 21 Jul 2016 10:13:50 -0400
>> 
>> Will the Emacs developers follow me? I'd love to see them use a better
>> process. I'm happy to concede I'm wrong and the current process works
>> fine, but the evidence leads me to believe it can be improved with
>> better tools (similar to the evidence in favor of Git when we were using
>> Bazaar). I'm not going to spend the next 6 months talking about it; I'd
>> rather know if the maintainers (Eli and John) and other core developers
>> are willing to consider Gitlab or something similar.

EZ> If you expect an answer from me, then I don't have enough information
EZ> to do that.  I never worked with Gitlab, have only a casual
EZ> familiarity with Github, and don't have any clear idea of what would
EZ> that mean for the process.  Someone™ will have to set up a test
EZ> repository which can be freely played with, and provide enough
EZ> instructions for the relevant frequent ops to allow some first-hand
EZ> experience with Gitlab (or whatever).  Only then I will be able to say
EZ> something intelligent.

The license is the first issue, before we do that work. Gitlab is not
GPL: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE

Is that acceptable? If yes, we can set up an instance against a mirror
of Emacs. If not, we'll need to keep looking.

EZ> FWIW, most of the time I use up on Emacs-related work is spent
EZ> debugging and reviewing patches, not on stuff related to debbugs and
EZ> the VCS; the latter take negligible time.  Not sure what that means in
EZ> the context of this discussion.

In the process I proposed, they are all the same thing, so I think you
would find it useful as well. A particularly useful CI feature is that
you could review a patch and see that it breaks the C build, or emits
some warnings, or breaks some tests... all done before your attention
was focused on it.

Ted

Re: debbugs tracker builds character

[Stefan Monnier]

SM> I think I'm starting to see what you mean.  You're talking about a tight
SM> integration where a pull-request is also itself an issue, so the comments
SM> can be directly on the patch itself.  As opposed to having issues and
SM> pull-request be two separate things that can refer to each other via
SM> an indirection.

> Yes. Actually Github has both of these right now.  When pull requests are
> not true issues, they tend to be associated with issues: sometimes
> 1-to-N and sometimes N-to-1.  People have different workflows, so it's
> good to have flexibility built into the model.  Sorry if this seems
> ambivalent; it's really a degree of creative freedom that users
> appreciate.

So, we could start by setting up a Gitlab or Kallithea instance for
Emacs and use it for pull-requests, while we keep using Debbugs for
issue tracking.

This lets us move forward on the pull-request and code-review front
without having to solve the "issue" of switching to another bug-tracker.

EA> A suggestion from a part-time package maintainer: make
EA> `report-emacs-bug' prompt for a package, and offer the results of
EA> `featurep' as completion. If the user picks a package that only exists
EA> in ELPA, email the package maintainer with the bug report.
> That's a really good suggestion, regardless of the underlying bug tracker.

I also like this idea.  Patches welcome,


        Stefan

Re: debbugs tracker builds character

[David Engster]

Ted Zlatanov writes:
> On Thu, 21 Jul 2016 17:50:01 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 
> EZ> If you expect an answer from me, then I don't have enough information
> EZ> to do that.  I never worked with Gitlab, have only a casual
> EZ> familiarity with Github, and don't have any clear idea of what would
> EZ> that mean for the process.  Someone™ will have to set up a test
> EZ> repository which can be freely played with, and provide enough
> EZ> instructions for the relevant frequent ops to allow some first-hand
> EZ> experience with Gitlab (or whatever).  Only then I will be able to say
> EZ> something intelligent.
>
> The license is the first issue, before we do that work. Gitlab is not
> GPL: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE
>
> Is that acceptable?

According to the "GNU ethical repository criteria" GitLab currently has
grade C, meaning "Acceptable hosting for a GNU package", and is actually
pretty close to grade B ("Good enough to recommend"). See

https://www.gnu.org/software/repo-criteria-evaluation.html

-David

Re: debbugs tracker builds character

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Thu, 21 Jul 2016 11:17:03 -0400
> 
> The license is the first issue, before we do that work. Gitlab is not
> GPL: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE

That question is not for me to answer.

Re: debbugs tracker builds character

[Ted Zlatanov]

On Thu, 21 Jul 2016 11:38:09 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> So, we could start by setting up a Gitlab or Kallithea instance for
SM> Emacs and use it for pull-requests, while we keep using Debbugs for
SM> issue tracking.

SM> This lets us move forward on the pull-request and code-review front
SM> without having to solve the "issue" of switching to another bug-tracker.

Works for me. We'll try it out for a few contributions (so Eli and
others can get a feel for it), then make it the recommended way to
submit patches.

If that works well, we can consider using it for bug tracking as well.

On Thu, 21 Jul 2016 18:58:40 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Thu, 21 Jul 2016 11:17:03 -0400
>> 
>> The license is the first issue, before we do that work. Gitlab is not
>> GPL: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE

EZ> That question is not for me to answer.

On Thu, 21 Jul 2016 17:41:15 +0200 David Engster <deng@randomsample.de> wrote: 

DE> According to the "GNU ethical repository criteria" GitLab currently has
DE> grade C, meaning "Acceptable hosting for a GNU package", and is actually
DE> pretty close to grade B ("Good enough to recommend"). See

DE> https://www.gnu.org/software/repo-criteria-evaluation.html

Eli and John, I'm happy to wait for a decision as long as someone is
making it eventually. I think the Emacs maintainers are the right people
to communicate the decision back to the developers.

Stefan brought up Kallithea https://kallithea-scm.org/ as an alternative
to Gitlab. It's much less mature but GPLv3. We can try both side by side.

Ted

Re: debbugs tracker builds character

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 1914 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> SM> I think I'm starting to see what you mean.  You're talking about a tight
> SM> integration where a pull-request is also itself an issue, so the comments
> SM> can be directly on the patch itself.  As opposed to having issues and
> SM> pull-request be two separate things that can refer to each other via
> SM> an indirection.
>
>> Yes. Actually Github has both of these right now.  When pull requests are
>> not true issues, they tend to be associated with issues: sometimes
>> 1-to-N and sometimes N-to-1.  People have different workflows, so it's
>> good to have flexibility built into the model.  Sorry if this seems
>> ambivalent; it's really a degree of creative freedom that users
>> appreciate.
>
> So, we could start by setting up a Gitlab or Kallithea instance for
> Emacs and use it for pull-requests, while we keep using Debbugs for
> issue tracking.
>
> This lets us move forward on the pull-request and code-review front
> without having to solve the "issue" of switching to another bug-tracker.
>
> EA> A suggestion from a part-time package maintainer: make
> EA> `report-emacs-bug' prompt for a package, and offer the results of
> EA> `featurep' as completion. If the user picks a package that only exists
> EA> in ELPA, email the package maintainer with the bug report.
>> That's a really good suggestion, regardless of the underlying bug tracker.
>
> I also like this idea.  Patches welcome,

Here's a patch. I hope I've used all the proper utility functions for
finding libraries and extracting headers.

So far as I can tell, debbugs doesn't keep track of "bug subscriptions",
and putting the package maintainer in the Cc header is all we need to
do, right?

Anyhow, I'm sure this will require tweaking, but take a look and let me
know.

Also, I've included a second patch to `lm-maintainer', which is (I hope)
a fairly minor fix.

Eric


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Cc-package-maintainers-when-reporting-emacs-bugs.patch --]
[-- Type: text/x-diff, Size: 2433 bytes --]

From 347d5a0935a671e4a1401925ecff0ca2c4c7a277 Mon Sep 17 00:00:00 2001
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Date: Thu, 21 Jul 2016 15:58:47 -0400
Subject: [PATCH 1/2] Cc package maintainers when reporting emacs bugs

* lisp/mail/emacsbug.el (report-emacs-bug): Prompt for a package to
  report a bug against, and cc the package maintainer, if any, on the
  bug report.
---
 lisp/mail/emacsbug.el | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/lisp/mail/emacsbug.el b/lisp/mail/emacsbug.el
index 18eaa22..ce3b60c 100644
--- a/lisp/mail/emacsbug.el
+++ b/lisp/mail/emacsbug.el
@@ -144,11 +144,14 @@ message-send-mail-function
 (defvar message-sendmail-envelope-from)
 
 ;;;###autoload
-(defun report-emacs-bug (topic &optional unused)
+(defun report-emacs-bug (topic package &optional unused)
   "Report a bug in GNU Emacs.
 Prompts for bug subject.  Leaves you in a mail buffer."
   (declare (advertised-calling-convention (topic) "24.5"))
-  (interactive "sBug Subject: ")
+  (interactive (list
+                (read-string "Bug Subject: ")
+                (when (bound-and-true-p package--initialized)
+                  (completing-read "Package: " package-alist))))
   ;; The syntax `version;' is preferred to `[version]' because the
   ;; latter could be mistakenly stripped by mailing software.
   (if (eq system-type 'ms-dos)
@@ -158,6 +161,9 @@ report-emacs-bug
   (let ((from-buffer (current-buffer))
 	(can-insert-mail (or (report-emacs-bug-can-use-xdg-email)
 			     (report-emacs-bug-can-use-osx-open)))
+        (cc (when (and package (null (string= package "emacs")))
+              (require 'finder)
+              (lm-maintainer (find-library-name package))))
         user-point message-end-point)
     (setq message-end-point
 	  (with-current-buffer (messages-buffer)
@@ -184,6 +190,13 @@ report-emacs-bug
       (when (and (not message-sendmail-envelope-from)
 		 (message-bogus-recipient-p (message-make-address)))
 	(set (make-local-variable 'message-sendmail-envelope-from) 'header)))
+    (when (cdr cc) ;; The cdr is the email address.
+      (if (eq major-mode 'message-mode)
+          (message-goto-cc)
+        (mail-cc))
+      (if (car cc)
+          (insert (format "%s <%s>" (car cc) (cdr cc)))
+        (insert (format "%s" (cdr cc)))))
     (rfc822-goto-eoh)
     (forward-line 1)
     ;; Move the mail signature to the proper place.
-- 
2.9.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-lm-maintainer-should-pass-file-arg-to-lm-authors.patch --]
[-- Type: text/x-diff, Size: 954 bytes --]

From fd9fec872ba4799dae4718cd5716f96c97d4b823 Mon Sep 17 00:00:00 2001
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Date: Thu, 21 Jul 2016 16:00:20 -0400
Subject: [PATCH 2/2] lm-maintainer should pass file arg to lm-authors

* lisp/emacs-lisp/lisp-mnt.el (lm-maintainer): If `lm-maintainer' ends
  up calling `lm-authors', it should pass the optional FILE arg to that
  call.
---
 lisp/emacs-lisp/lisp-mnt.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/lisp-mnt.el b/lisp/emacs-lisp/lisp-mnt.el
index 46373da..604f472 100644
--- a/lisp/emacs-lisp/lisp-mnt.el
+++ b/lisp/emacs-lisp/lisp-mnt.el
@@ -393,7 +393,7 @@ lm-maintainer
     (let ((maint (lm-header "maintainer")))
       (if maint
 	  (lm-crack-address maint)
-	(car (lm-authors))))))
+	(car (lm-authors file))))))
 
 (defun lm-creation-date (&optional file)
   "Return the created date given in file FILE, or current buffer if FILE is nil."
-- 
2.9.0

Re: debbugs tracker builds character

[Dmitry Gutov]

On 07/20/2016 11:43 PM, Stefan Monnier wrote:

> The more I use BugIt, the more I find it important to be able to query
> the bug database while offline.  Being able to queue actions so they're
> executed when I get online is not too terrible, but it's not really
> good enough.

An Emacs interface could cache some information about open issues 
locally, too. So you could read them, queue some actions, and fire them 
off when online.

Admittedly, that would require a significant amount of work, but this 
way we could use an existing issue tracker, rather than a new one 
developed just for this purpose.

Re: debbugs tracker builds character

[Stefan Monnier]

> Admittedly, that would require a significant amount of work, but this way we
> could use an existing issue tracker, rather than a new one developed just
> for this purpose.

Of course.  My interest in BugIt is just a personal interest in
distributed bug tracking.  I think Emacs should use something that works
well and has a lot of existing maintainers, so we can focus on
maintaining Emacs instead.

I do think that my vision of what BugIt could be would work great for
Emacs, but currently it's just a vision, and based on recent progress
there's no reason to assume it will turn into anything else in the
foreseeable future.


        Stefan

Re: debbugs tracker builds character

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Response to Ted Zlatanov

  > The license is the first issue, before we do that work. Gitlab is not
  > GPL: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE

What is the proposed activity here?  Is this about storing some
software on the gitlab.com repository?  Or about doing something with
some of the programs that run on the gitlab.com servers?  Those
two questions have very little in common.  Or about something else?

For using a repository, see http://gnu.org/software/repo-criteria.

For running a program, the program needs to be free software.
See http://gnu.org/philosophy/free-sw.html for the definition
and http://gnu.org/licenses/license-list.html for whether various
licenses are free or not.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: debbugs tracker builds character

[Dmitry Gutov]

On 07/22/2016 02:49 AM, Richard Stallman wrote:

> Response to Ted Zlatanov
>
>   > The license is the first issue, before we do that work. Gitlab is not
>   > GPL: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE
>
> What is the proposed activity here?  Is this about storing some
> software on the gitlab.com repository?  Or about doing something with
> some of the programs that run on the gitlab.com servers?

Not gitlab.com. I imagine we will install the self-hosted version of 
Gitlab, on one of FSF's servers.

Re: debbugs tracker builds character

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Michael Albinus <michael.albinus@gmx.de> writes:
>
>> This might be possible by extending the SOAP interface. Not that I like
>> it, but we've changed the SOAP interface already, so we aren't
>> compatible with Debian's debbugs server anymore, for some operations.
>
> If you add such a function, I'll finish the offline implementation (and
> document it :-)).

Looks like we have it already. `debbugs-search-est' supports the
attribute :@cdate, which means modification date of a message. If you
want to see all bugs of package "emacs" with a message sent between
July 20th and July 30th 2016, you'll apply

(apply
 'debbugs-gnu-bugs 
 (delete-dups
  (mapcar
   (lambda (x) (cdr (assoc "id" x)))
   (debbugs-search-est
    '(:max 1000)
    `(:@cdate
      ,(floor (float-time (encode-time 0 0 0 20 07 2016)))
      ,(floor (float-time (encode-time 0 0 0 30 07 2016)))
      :operator "NUMBT")
    '(:package "emacs")))))

Since status changes are also triggered by emails, you should get all
modified bugs of that time frame. That's more than you want (just bugs
with changed attributes), but it might be sufficient, if the time frame
is not too large.

This still needs some work. `debbugs-search-est' does not retrieve the
results in seval hunks asynchronously, as `debbugs-get-status' does
already. The attribute :@cdate is not exposed (yet) in
`debbugs-gnu-search', and `debbugs-search-est' is also not documented
yet in the Debbugs Programmer Guide. Will do all of this.

Best regards, Michael.

Re: debbugs tracker builds character

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> This still needs some work. `debbugs-search-est' does not retrieve the
> results in seval hunks asynchronously, as `debbugs-get-status' does
> already. The attribute :@cdate is not exposed (yet) in
> `debbugs-gnu-search', and `debbugs-search-est' is also not documented
> yet in the Debbugs Programmer Guide. Will do all of this.

Great!  Looking forward to it...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: debbugs tracker builds character

[Eric Abrahamsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> SM> I think I'm starting to see what you mean.  You're talking about a tight
>> SM> integration where a pull-request is also itself an issue, so the comments
>> SM> can be directly on the patch itself.  As opposed to having issues and
>> SM> pull-request be two separate things that can refer to each other via
>> SM> an indirection.
>>
>>> Yes. Actually Github has both of these right now.  When pull requests are
>>> not true issues, they tend to be associated with issues: sometimes
>>> 1-to-N and sometimes N-to-1.  People have different workflows, so it's
>>> good to have flexibility built into the model.  Sorry if this seems
>>> ambivalent; it's really a degree of creative freedom that users
>>> appreciate.
>>
>> So, we could start by setting up a Gitlab or Kallithea instance for
>> Emacs and use it for pull-requests, while we keep using Debbugs for
>> issue tracking.
>>
>> This lets us move forward on the pull-request and code-review front
>> without having to solve the "issue" of switching to another bug-tracker.
>>
>> EA> A suggestion from a part-time package maintainer: make
>> EA> `report-emacs-bug' prompt for a package, and offer the results of
>> EA> `featurep' as completion. If the user picks a package that only exists
>> EA> in ELPA, email the package maintainer with the bug report.
>>> That's a really good suggestion, regardless of the underlying bug tracker.
>>
>> I also like this idea.  Patches welcome,
>
> Here's a patch. I hope I've used all the proper utility functions for
> finding libraries and extracting headers.
>
> So far as I can tell, debbugs doesn't keep track of "bug subscriptions",
> and putting the package maintainer in the Cc header is all we need to
> do, right?
>
> Anyhow, I'm sure this will require tweaking, but take a look and let me
> know.
>
> Also, I've included a second patch to `lm-maintainer', which is (I hope)
> a fairly minor fix.

Please ignore this second patch, this was a brain fart.

Re: debbugs tracker builds character

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> I've just quoted the example given on the Github page, and they've used
> "Fixes #45". *If* we introduce something like this, I would be in favor
> of the established bug#45 format.

I guess the reason nobody has made a vc-message-to-debbugs bridge by now
is that nobody with interest in this have access to, or are comfortable
working on the FSF servers.  But it just occurred to me that this could
be set up anywhere.  Just subscribe the VC log list to an email address
and have a .forward script on that host parse the messages and send out
debbugs control messages.

(This script could also subscribe to the bug mailing list and send out
the previously discussed "tags patch" for reports containing patches...)

It sounds like a very small and simple ... Python script to me.  And
free software, of course, so that it can be moved around if the external
hosting proves to be unreliable.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: debbugs tracker builds character

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

> Looks like we have it already. `debbugs-search-est' supports the
> attribute :@cdate, which means modification date of a message. If you
> want to see all bugs of package "emacs" with a message sent between
> July 20th and July 30th 2016, you'll apply

For the records, I forgot the operator for the :package attribute. The
correct call is

(apply
 'debbugs-gnu-bugs 
 (delete-dups
  (mapcar
   (lambda (x) (cdr (assoc "id" x)))
   (debbugs-search-est
    '(:skip 0 :max 1000)
    `(:@cdate
      ,(floor (float-time (encode-time 0 0 0 20 07 2016)))
      ,(floor (float-time (encode-time 0 0 0 30 07 2016)))
      :operator "NUMBT")
    '(:package "emacs" :operator "STRINC")))))

Best regards, Michael.

Re: debbugs tracker builds character

[Ted Zlatanov]

On Thu, 21 Jul 2016 19:49:21 -0400 Richard Stallman <rms@gnu.org> wrote: 

>> The license is the first issue, before we do that work. Gitlab is not
>> GPL: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/LICENSE

RS> What is the proposed activity here?  Is this about storing some
RS> software on the gitlab.com repository?  Or about doing something with
RS> some of the programs that run on the gitlab.com servers?  Those
RS> two questions have very little in common.  Or about something else?

Gitlab is software to manage collaboration against a Git repository.
It's similar to the functionality offered by Github and BitBucket, but
is free (as pointed out earlier in the thread) and can be self-hosted
(community edition).

Gitlab.com/gitlab.org host an instance of the Gitlab software and are
not going to be used.

RS> For using a repository, see http://gnu.org/software/repo-criteria.

RS> For running a program, the program needs to be free software.
RS> See http://gnu.org/philosophy/free-sw.html for the definition
RS> and http://gnu.org/licenses/license-list.html for whether various
RS> licenses are free or not.

We believe Gitlab Community Edition is OK in this regard and would like
to do a trial run of using it to submit and review patches to Emacs
(pull requests). Later it may also be considered as the bug/issue
tracker but pull requests are the primary need.

Ted

Re: debbugs tracker builds character

[Stefan Monnier]

> +                (when (bound-and-true-p package--initialized)
> +                  (completing-read "Package: " package-alist))))

package-alist may not exist.  Wouldn't (mapcar #'symbol-name features)
be preferable?


        Stefan

Re: debbugs tracker builds character

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> +                (when (bound-and-true-p package--initialized)
>> +                  (completing-read "Package: " package-alist))))
>
> package-alist may not exist.  Wouldn't (mapcar #'symbol-name features)
> be preferable?

I figured if package--initialized was bound and true, then package-alist
should exist.

I did this at the package level (rather than the deeper features level)
just because it seemed cleaner. Using features, you get all the
individual files of multi-file packages, which seems messy (and prone to
missing the maintainer). We'd also be offering all the built-in Emacs
libraries, for which there's no sensible "maintainer", and could end up
cc'ing people who wrote libraries decades ago.

That's just what I was thinking at the time, though -- obviously I'd be
happy to rework it to use features.

Eric

Re: debbugs tracker builds character

[Stefan Monnier]

> I did this at the package level (rather than the deeper features level)
> just because it seemed cleaner. Using features, you get all the
> individual files of multi-file packages, which seems messy (and prone to
> missing the maintainer). We'd also be offering all the built-in Emacs
> libraries, for which there's no sensible "maintainer", and could end up
> cc'ing people who wrote libraries decades ago.

Hmm... how 'bout

    (if (bound-and-true-p package--initialized)
        (seq-intersection (mapcar #'car package-alist) features)
      features)

then?


        Stefan

Re: debbugs tracker builds character

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Gitlab is software to manage collaboration against a Git repository.
  > It's similar to the functionality offered by Github and BitBucket, but
  > is free (as pointed out earlier in the thread) and can be self-hosted
  > (community edition).

Since it is free software, we can use it when it is useful.

On occasions when we can use a GNU package or a non-GNU package, we
should choose the GNU package, out of loyalty.  On occasions when we
can use a GPL'd package (version N or later) or a free package with
some other license we should choose former package, out of loyalty.
But that's only about choosing _between_ free programs.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: debbugs tracker builds character

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I did this at the package level (rather than the deeper features level)
>> just because it seemed cleaner. Using features, you get all the
>> individual files of multi-file packages, which seems messy (and prone to
>> missing the maintainer). We'd also be offering all the built-in Emacs
>> libraries, for which there's no sensible "maintainer", and could end up
>> cc'ing people who wrote libraries decades ago.
>
> Hmm... how 'bout
>
>     (if (bound-and-true-p package--initialized)
>         (seq-intersection (mapcar #'car package-alist) features)
>       features)
>
> then?

But the `seq-intersection' call will filter out packages that have been
installed (are in package-alist), but not loaded (are not in features).
Wouldn't it make more sense to use:

(seq-uniq (append features (mapcar #'car package-alist)))

Also, what was the outcome of the seq.el pre-loading/autoloading
discussion? Are seq-* functions guaranteed to be available?

Eric

Re: debbugs tracker builds character

[Stefan Monnier]

> But the `seq-intersection' call will filter out packages that have been
> installed (are in package-alist), but not loaded (are not in features).

That was the intention.  But indeed, if the user submits the report from
another Emacs session than the one where the bug occurs, this may filter
out the relevant package.

> Also, what was the outcome of the seq.el pre-loading/autoloading
> discussion? Are seq-* functions guaranteed to be available?

We'd (require 'seq), that's not a problem.
But indeed, by that measure we could also package-initialize
unconditionally and just use package-alist.


        Stefan

Re: debbugs tracker builds character

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> But the `seq-intersection' call will filter out packages that have been
>> installed (are in package-alist), but not loaded (are not in features).
>
> That was the intention.  But indeed, if the user submits the report from
> another Emacs session than the one where the bug occurs, this may filter
> out the relevant package.
>
>> Also, what was the outcome of the seq.el pre-loading/autoloading
>> discussion? Are seq-* functions guaranteed to be available?
>
> We'd (require 'seq), that's not a problem.
> But indeed, by that measure we could also package-initialize
> unconditionally and just use package-alist.

Okay, understood on both counts.

I can't say I have much of an opinion on the finer points -- I'm just
pleased to have the basics in place.

We can either "go big" (use features), or "go small" (manually
package-initialize, then use package-alist). Would you make an executive
decision, and I'll patch?

Eric

Re: debbugs tracker builds character

[Stefan Monnier]

> We can either "go big" (use features), or "go small" (manually
> package-initialize, then use package-alist). Would you make an executive
> decision, and I'll patch?

Your guess is as good as mine,


        Stefan

Re: debbugs tracker builds character

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> We can either "go big" (use features), or "go small" (manually
>> package-initialize, then use package-alist). Would you make an executive
>> decision, and I'll patch?
>
> Your guess is as good as mine,

Fair enough! I choose the conservative route: here's a patch that just
works on package-alist. If anyone complains, it will be easy enough to
expand later.

Eric


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Cc-package-maintainers-when-reporting-emacs-bugs.patch --]
[-- Type: text/x-diff, Size: 2482 bytes --]

From 542d7b342d0ce86933d8c6998ca7a817952b0b3f Mon Sep 17 00:00:00 2001
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Date: Thu, 21 Jul 2016 15:58:47 -0400
Subject: [PATCH] Cc package maintainers when reporting emacs bugs

* lisp/mail/emacsbug.el (report-emacs-bug): Prompt for a package to
  report a bug against, and cc the package maintainer, if any, on the
  bug report.
---
 lisp/mail/emacsbug.el | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/lisp/mail/emacsbug.el b/lisp/mail/emacsbug.el
index 18eaa22..87ac234 100644
--- a/lisp/mail/emacsbug.el
+++ b/lisp/mail/emacsbug.el
@@ -144,11 +144,17 @@ message-send-mail-function
 (defvar message-sendmail-envelope-from)
 
 ;;;###autoload
-(defun report-emacs-bug (topic &optional unused)
+(defun report-emacs-bug (topic package &optional unused)
   "Report a bug in GNU Emacs.
 Prompts for bug subject.  Leaves you in a mail buffer."
   (declare (advertised-calling-convention (topic) "24.5"))
-  (interactive "sBug Subject: ")
+  (interactive (list
+                (read-string "Bug Subject: ")
+                (completing-read
+                 "Package: "
+                 (progn
+                   (package-initialize)
+                   package-alist))))
   ;; The syntax `version;' is preferred to `[version]' because the
   ;; latter could be mistakenly stripped by mailing software.
   (if (eq system-type 'ms-dos)
@@ -158,6 +164,9 @@ report-emacs-bug
   (let ((from-buffer (current-buffer))
 	(can-insert-mail (or (report-emacs-bug-can-use-xdg-email)
 			     (report-emacs-bug-can-use-osx-open)))
+        (cc (unless (or (string-empty-p package) (string= package "emacs"))
+              (require 'finder)
+              (lm-maintainer (find-library-name package))))
         user-point message-end-point)
     (setq message-end-point
 	  (with-current-buffer (messages-buffer)
@@ -184,6 +193,13 @@ report-emacs-bug
       (when (and (not message-sendmail-envelope-from)
 		 (message-bogus-recipient-p (message-make-address)))
 	(set (make-local-variable 'message-sendmail-envelope-from) 'header)))
+    (when (cdr cc) ;; The cdr is the email address.
+      (if (eq major-mode 'message-mode)
+          (message-goto-cc)
+        (mail-cc))
+      (if (car cc)
+          (insert (format "%s <%s>" (car cc) (cdr cc)))
+        (insert (format "%s" (cdr cc)))))
     (rfc822-goto-eoh)
     (forward-line 1)
     ;; Move the mail signature to the proper place.
-- 
2.9.0

Re: debbugs tracker builds character

[Ted Zlatanov]

On Sat, 23 Jul 2016 15:54:01 -0400 Richard Stallman <rms@gnu.org> wrote: 

>> Gitlab is software to manage collaboration against a Git repository.
>> It's similar to the functionality offered by Github and BitBucket, but
>> is free (as pointed out earlier in the thread) and can be self-hosted
>> (community edition).

RS> Since it is free software, we can use it when it is useful.

RS> On occasions when we can use a GNU package or a non-GNU package, we
RS> should choose the GNU package, out of loyalty.  On occasions when we
RS> can use a GPL'd package (version N or later) or a free package with
RS> some other license we should choose former package, out of loyalty.
RS> But that's only about choosing _between_ free programs.

Thanks for the explanation, Richard.

Maintainers: can we run a Gitlab installation on FSF/GNU hardware? If
not, I'll bring an instance up in AWS. It should have a recent copy of
the Emacs Git repo, but doesn't need to be kept up to date, since we'll
just run it as a demo of the pull request workflow. It will be
temporary.

Thanks
Ted

Re: debbugs tracker builds character

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Mon, 25 Jul 2016 08:52:33 -0400
> 
> Maintainers: can we run a Gitlab installation on FSF/GNU hardware?

I think you need to ask this on savannah-hackers-public@gnu.org, not
here.

Re: debbugs tracker builds character

[Ted Zlatanov]

On Mon, 25 Jul 2016 19:51:00 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> From: Ted Zlatanov <tzz@lifelogs.com>
>> Date: Mon, 25 Jul 2016 08:52:33 -0400
>> 
>> Maintainers: can we run a Gitlab installation on FSF/GNU hardware?

EZ> I think you need to ask this on savannah-hackers-public@gnu.org, not
EZ> here.

OK; done.

Ted

Re: debbugs tracker builds character

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

> Looks like we have it already. `debbugs-search-est' supports the
> attribute :@cdate, which means modification date of a message. If you
> want to see all bugs of package "emacs" with a message sent between
> July 20th and July 30th 2016, you'll apply

[...]

Unfortunately, it doesn't work completely as expected. The debbugs
search index is updated once a day only, so you don't get recent chages
by such a search :-(

I'll go back to the first idea, extending the SOAP interface, somehow.

Best regards, Michael.

[ELPA] I regret using subtree (was: debbugs tracker builds character)

[Eric Abrahamsen]

Okay, I officially regret having used the subtree approach to squash
Gnorb into Elpa, and if at all possible I would like to abandon this and
move to just working in the Elpa tree.

Do I have to do anything magic? My understanding of the `git subtree'
command is that it isn't actually a "real" command, it's just some
behind-the-scenes jiggery-pokery with merge and cherry-pick and
read-tree. Ie, I should just be able to declare "I'm done with this" and
carry on making commits directly into Elpa. Is that correct?

The only thing I'll regret is the github issue tracker, and for that
reason I'm tacking this on to a thread from last year, about making
`report-emacs-bug' collect package information before sending a bug
report, and cc'ing maintainers. I still think this is a good idea, and I
think my patch, if not ideal, is good enough, so I'm reviving it. What
say you all?

Eric


> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> We can either "go big" (use features), or "go small" (manually
>>> package-initialize, then use package-alist). Would you make an executive
>>> decision, and I'll patch?
>>
>> Your guess is as good as mine,
>
> Fair enough! I choose the conservative route: here's a patch that just
> works on package-alist. If anyone complains, it will be easy enough to
> expand later.

Re: [ELPA] I regret using subtree

[Stefan Monnier]

> Okay, I officially regret having used the subtree approach to squash
> Gnorb into Elpa, and if at all possible I would like to abandon this and
> move to just working in the Elpa tree.
> Do I have to do anything magic? My understanding of the `git subtree'
> command is that it isn't actually a "real" command, it's just some
> behind-the-scenes jiggery-pokery with merge and cherry-pick and
> read-tree. Ie, I should just be able to declare "I'm done with this" and
> carry on making commits directly into Elpa. Is that correct?

That is correct.
[ Of course, if you want, you can also "git rm -r packages/gnorb" and
  then add an externals/gnorb branch.  ]

> The only thing I'll regret is the github issue tracker, and for that
> reason I'm tacking this on to a thread from last year, about making
> `report-emacs-bug' collect package information before sending a bug
> report, and cc'ing maintainers. I still think this is a good idea, and I
> think my patch, if not ideal, is good enough, so I'm reviving it. What
> say you all?

The patch seems OK, except it should put the Cc in the "X-Debbugs-Cc:"
header (so that the maintainer gets the email with the bug-number
rather than email before assigned a bug-number).


        Stefan

Re: [ELPA] I regret using subtree

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 1377 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Okay, I officially regret having used the subtree approach to squash
>> Gnorb into Elpa, and if at all possible I would like to abandon this and
>> move to just working in the Elpa tree.
>> Do I have to do anything magic? My understanding of the `git subtree'
>> command is that it isn't actually a "real" command, it's just some
>> behind-the-scenes jiggery-pokery with merge and cherry-pick and
>> read-tree. Ie, I should just be able to declare "I'm done with this" and
>> carry on making commits directly into Elpa. Is that correct?
>
> That is correct.
> [ Of course, if you want, you can also "git rm -r packages/gnorb" and
>   then add an externals/gnorb branch.  ]

No, I think I'll go with the simpler option.

>> The only thing I'll regret is the github issue tracker, and for that
>> reason I'm tacking this on to a thread from last year, about making
>> `report-emacs-bug' collect package information before sending a bug
>> report, and cc'ing maintainers. I still think this is a good idea, and I
>> think my patch, if not ideal, is good enough, so I'm reviving it. What
>> say you all?
>
> The patch seems OK, except it should put the Cc in the "X-Debbugs-Cc:"
> header (so that the maintainer gets the email with the bug-number
> rather than email before assigned a bug-number).

Here's an updated version.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Prompt-for-package-when-reporting-emacs-bug.patch --]
[-- Type: text/x-patch, Size: 2461 bytes --]

From 51cd6dd13b965453b56526e6f323fd563b7ed026 Mon Sep 17 00:00:00 2001
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Date: Sun, 12 Mar 2017 18:41:42 -0700
Subject: [PATCH] Prompt for package when reporting emacs bug

* lisp/mail/emacsbug.el (report-emacs-bug): Prompt for a package, and
  put the package maintainer in the X-Debbugs-Cc header.
---
 lisp/mail/emacsbug.el | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/lisp/mail/emacsbug.el b/lisp/mail/emacsbug.el
index c1aec6923f..656eeabe0f 100644
--- a/lisp/mail/emacsbug.el
+++ b/lisp/mail/emacsbug.el
@@ -123,17 +123,26 @@ message-send-mail-function
 (defvar message-sendmail-envelope-from)
 
 ;;;###autoload
-(defun report-emacs-bug (topic &optional unused)
+(defun report-emacs-bug (topic package &optional unused)
   "Report a bug in GNU Emacs.
 Prompts for bug subject.  Leaves you in a mail buffer."
   (declare (advertised-calling-convention (topic) "24.5"))
-  (interactive "sBug Subject: ")
+  (interactive (list
+                (read-string "Bug Subject: ")
+                (completing-read
+                 "Package: "
+                 (progn
+                   (package-initialize)
+                   package-alist))))
   ;; The syntax `version;' is preferred to `[version]' because the
   ;; latter could be mistakenly stripped by mailing software.
   (setq topic (concat emacs-version "; " topic))
   (let ((from-buffer (current-buffer))
 	(can-insert-mail (or (report-emacs-bug-can-use-xdg-email)
 			     (report-emacs-bug-can-use-osx-open)))
+        (cc (unless (or (string-empty-p package) (string= package "emacs"))
+              (require 'finder)
+              (lm-maintainer (find-library-name package))))
         user-point message-end-point)
     (setq message-end-point
 	  (with-current-buffer (messages-buffer)
@@ -160,6 +169,13 @@ report-emacs-bug
       (when (and (not message-sendmail-envelope-from)
 		 (message-bogus-recipient-p (message-make-address)))
 	(set (make-local-variable 'message-sendmail-envelope-from) 'header)))
+    (when (cdr cc) ;; cdr is the email address.
+      (if (eq major-mode 'message-mode)
+          (message-position-on-field "X-Debbugs-Cc")
+        (mail-cc))
+      (if (car cc)
+          (insert (format "%s <%s>" (car cc) (cdr cc)))
+        (insert (format "%s" (cdr cc)))))
     (rfc822-goto-eoh)
     (forward-line 1)
     ;; Move the mail signature to the proper place.
-- 
2.12.0

Re: [ELPA] I regret using subtree

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]


() Eric Abrahamsen <eric@ericabrahamsen.net>
() Sun, 12 Mar 2017 17:22:37 -0700

   regret having used the subtree approach to squash
   Gnorb into Elpa

Just curious: What is it you regret about it?  TIA.

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: [ELPA] I regret using subtree

[Eric Abrahamsen]

Thien-Thi Nguyen <ttn@gnu.org> writes:

> () Eric Abrahamsen <eric@ericabrahamsen.net>
> () Sun, 12 Mar 2017 17:22:37 -0700
>
>    regret having used the subtree approach to squash
>    Gnorb into Elpa
>
> Just curious: What is it you regret about it?  TIA.

I don't have a strong enough grasp of git to feel confident with the
relevant invocations -- I get it, but I don't grok it. There was a
period of a year and a half when I did not update gnorb because I lost
my saved "git subtree pull --squash" incantation, and just couldn't be
bothered to read the blog posts and reconstruct it. I still don't know
why it makes me do a merge commit afterwards.

I use git daily, but at a pretty shallow level. I hate deployment
friction, and that's what this turned out to be. I once laughed at
Python's fabric, because it does almost nothing, yet now I use "fab
deploy" (with test integration) as a matter of course, and bugs get
fixed.

If I knew git better it would be no problem, I'm sure, but I'm not
interested in becoming a git expert. I like to keep it to feature
branches, rebasing to master, and pushing.

Eric

Re: [ELPA] I regret using subtree

[Tino Calancha]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> The patch seems OK, except it should put the Cc in the "X-Debbugs-Cc:"
>> header (so that the maintainer gets the email with the bug-number
>> rather than email before assigned a bug-number).
>
> Here's an updated version.

I)
There are files with more than 1 maintainer/author.  Just pick up the first one
may not suffice.
E.g.,
(lm-authors (find-library-name "tramp"))
=> (("Kai Großjohann" . "kai.grossjohann@gmx.net")
    ("Michael Albinus" . "michael.albinus@gmx.de"))

(lm-maintainer (find-library-name "tramp"))
=> ("Kai Großjohann" . "kai.grossjohann@gmx.net")
;; Your patch won't send CC to Michael.
I suggest to CC all of them.  In order to do that `lm-maintainer' must
be updated.  See patches below.

II) There are files with plural versions of 'Author', 'Maintainer' headers
For instance,
lisp/net/rcirc.el
In this case `lm-maintainer' fall back in `lm-authors' so that Liu wouldn't
be in CC.

Another example,
lisp/color.el
Just 'Authors' header, no 'Maintainer' one.  There wouldn't be CC in this case.

III)
The e-mail format might confuse these tools.
E.g.
lisp/org/org-protocol.el
;; Maintainer: Sebastian Rose <sebastian_rose AT gmx DOT de>
or in
lisp/progmodes/prolog.el
;; Maintainer: Stefan Bruda <stefan(at)bruda(dot)ca>

--8<-----------------------------cut here---------------start------------->8---
From b0a51241e4393ee62673c181c03317649e9652fe Mon Sep 17 00:00:00 2001
From: Tino Calancha <tino.calancha@gmail.com>
Date: Mon, 13 Mar 2017 18:56:45 +0900
Subject: [PATCH 1/3] lm-maintainer: Allow return more than 1 maintainer

* lisp/emacs-lisp/lisp-mnt.el (lm-authors, lm-maintainer): Pluralize regexp.
(lm-header-multiline): Fix regexp.
(lm-maintainer): Add second argument ALL.
---
 lisp/emacs-lisp/lisp-mnt.el | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/lisp/emacs-lisp/lisp-mnt.el b/lisp/emacs-lisp/lisp-mnt.el
index fc3caf3359..c96d224679 100644
--- a/lisp/emacs-lisp/lisp-mnt.el
+++ b/lisp/emacs-lisp/lisp-mnt.el
@@ -118,6 +118,8 @@
 
 ;;; Variables:
 
+(eval-when-compile (require 'subr-x))
+
 (defgroup lisp-mnt nil
   "Utility functions for Emacs Lisp maintainers."
   :prefix "lm-"
@@ -285,7 +287,7 @@ lm-header-multiline
       (when res
 	(setq res (list res))
 	(forward-line 1)
-	(while (looking-at "^;+\\(\t\\|[\t\s]\\{2,\\}\\)\\(.+\\)")
+	(while (looking-at "^;+\\(\t+\s?+\\|[[:blank:]]\\{2,\\}\\)\\(.+\\)")
 	  (push (match-string-no-properties 2) res)
 	  (forward-line 1)))
       (nreverse res))))
@@ -383,17 +385,20 @@ lm-authors
 Each element of the list is a cons; the car is the full name,
 the cdr is an email address."
   (lm-with-file file
-    (let ((authorlist (lm-header-multiline "author")))
+    (let ((authorlist (lm-header-multiline "author[s]?")))
       (mapcar 'lm-crack-address authorlist))))
 
-(defun lm-maintainer (&optional file)
+(defun lm-maintainer (&optional file all)
   "Return the maintainer of file FILE, or current buffer if FILE is nil.
-The return value has the form (NAME . ADDRESS)."
+If optional arg ALL is non-nil, then return all maintainers.  Otherwise,
+return just the first one.  In the former case the return value has
+the form ((NAME1 . ADDRESS1) (NAME2 . ADDRESS2) ...); in the latter case,
+the return value is just (NAME1 . ADDRESS1)."
   (lm-with-file file
-    (let ((maint (lm-header "maintainer")))
-      (if maint
-	  (lm-crack-address maint)
-	(car (lm-authors))))))
+    (let ((people (if-let (maint (lm-header-multiline "maintainer[s]?"))
+                      (mapcar 'lm-crack-address maint)
+                    (lm-authors))))
+      (if all people (car people)))))
 
 (defun lm-creation-date (&optional file)
   "Return the created date given in file FILE, or current buffer if FILE is nil."
-- 
2.11.0

From 26be86a780cc9c5f497a6e99bd41fc81a2be8ad6 Mon Sep 17 00:00:00 2001
From: Tino Calancha <tino.calancha@gmail.com>
Date: Mon, 13 Mar 2017 18:56:45 +0900
Subject: [PATCH 2/3] * lisp/mail/emacsbug.el (report-emacs-bug): Apply Eric
 patch.

---
 lisp/mail/emacsbug.el | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/lisp/mail/emacsbug.el b/lisp/mail/emacsbug.el
index c1aec6923f..656eeabe0f 100644
--- a/lisp/mail/emacsbug.el
+++ b/lisp/mail/emacsbug.el
@@ -123,17 +123,26 @@ message-send-mail-function
 (defvar message-sendmail-envelope-from)
 
 ;;;###autoload
-(defun report-emacs-bug (topic &optional unused)
+(defun report-emacs-bug (topic package &optional unused)
   "Report a bug in GNU Emacs.
 Prompts for bug subject.  Leaves you in a mail buffer."
   (declare (advertised-calling-convention (topic) "24.5"))
-  (interactive "sBug Subject: ")
+  (interactive (list
+                (read-string "Bug Subject: ")
+                (completing-read
+                 "Package: "
+                 (progn
+                   (package-initialize)
+                   package-alist))))
   ;; The syntax `version;' is preferred to `[version]' because the
   ;; latter could be mistakenly stripped by mailing software.
   (setq topic (concat emacs-version "; " topic))
   (let ((from-buffer (current-buffer))
 	(can-insert-mail (or (report-emacs-bug-can-use-xdg-email)
 			     (report-emacs-bug-can-use-osx-open)))
+        (cc (unless (or (string-empty-p package) (string= package "emacs"))
+              (require 'finder)
+              (lm-maintainer (find-library-name package))))
         user-point message-end-point)
     (setq message-end-point
 	  (with-current-buffer (messages-buffer)
@@ -160,6 +169,13 @@ report-emacs-bug
       (when (and (not message-sendmail-envelope-from)
 		 (message-bogus-recipient-p (message-make-address)))
 	(set (make-local-variable 'message-sendmail-envelope-from) 'header)))
+    (when (cdr cc) ;; cdr is the email address.
+      (if (eq major-mode 'message-mode)
+          (message-position-on-field "X-Debbugs-Cc")
+        (mail-cc))
+      (if (car cc)
+          (insert (format "%s <%s>" (car cc) (cdr cc)))
+        (insert (format "%s" (cdr cc)))))
     (rfc822-goto-eoh)
     (forward-line 1)
     ;; Move the mail signature to the proper place.
-- 
2.11.0

From 2114a448ad145923980dbe4d09ae5feff18eb49d Mon Sep 17 00:00:00 2001
From: Tino Calancha <tino.calancha@gmail.com>
Date: Mon, 13 Mar 2017 18:56:45 +0900
Subject: [PATCH 3/3] * lisp/mail/emacsbug.el (report-emacs-bug): Send Cc to
 all maintainers.

---
 lisp/mail/emacsbug.el | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/lisp/mail/emacsbug.el b/lisp/mail/emacsbug.el
index 656eeabe0f..018d2ea6a7 100644
--- a/lisp/mail/emacsbug.el
+++ b/lisp/mail/emacsbug.el
@@ -142,7 +142,7 @@ report-emacs-bug
 			     (report-emacs-bug-can-use-osx-open)))
         (cc (unless (or (string-empty-p package) (string= package "emacs"))
               (require 'finder)
-              (lm-maintainer (find-library-name package))))
+              (lm-maintainer (find-library-name package) 'all)))
         user-point message-end-point)
     (setq message-end-point
 	  (with-current-buffer (messages-buffer)
@@ -169,13 +169,19 @@ report-emacs-bug
       (when (and (not message-sendmail-envelope-from)
 		 (message-bogus-recipient-p (message-make-address)))
 	(set (make-local-variable 'message-sendmail-envelope-from) 'header)))
-    (when (cdr cc) ;; cdr is the email address.
-      (if (eq major-mode 'message-mode)
-          (message-position-on-field "X-Debbugs-Cc")
-        (mail-cc))
-      (if (car cc)
-          (insert (format "%s <%s>" (car cc) (cdr cc)))
-        (insert (format "%s" (cdr cc)))))
+    (dolist (maint cc)
+      (when (cdr maint) ;; cdr is the email address.
+        (if (eq major-mode 'message-mode)
+            (message-position-on-field "X-Debbugs-Cc")
+          (mail-cc))
+        (cond ((car maint)
+               (insert (format "%s%s <%s>"
+                               (if (looking-back ":[[:blank:]]+?" (point-at-bol)) "" ", ")
+                               (car maint) (cdr maint))))
+              (t
+               (insert (format "%s%s"
+                               (if (looking-back ":[[:blank:]]+?" (point-at-bol)) "" ", ")
+                               (cdr maint)))))))
     (rfc822-goto-eoh)
     (forward-line 1)
     ;; Move the mail signature to the proper place.
-- 
2.11.0

--8<-----------------------------cut here---------------end--------------->8---
In GNU Emacs 26.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.8)
 of 2017-03-13
Repository revision: 94b59f7dd1e8611495ff0f4596dc6dec20e268af

Re: [ELPA] I regret using subtree

[Stefan Monnier]

> There are files with more than 1 maintainer/author.  Just pick up the
> first one may not suffice.

I disagree here.  E.g. for Tramp, Kai is not involved any more, so the
right fix is to add a "Maintainer:" to tramp.el.

> II) There are files with plural versions of 'Author', 'Maintainer' headers
> For instance, lisp/net/rcirc.el In this case `lm-maintainer' fall back
> in `lm-authors' so that Liu wouldn't be in CC.

Again, these are errors in the files.

> III)
> The e-mail format might confuse these tools.
> E.g.
> lisp/org/org-protocol.el
> ;; Maintainer: Sebastian Rose <sebastian_rose AT gmx DOT de>
> or in
> lisp/progmodes/prolog.el
> ;; Maintainer: Stefan Bruda <stefan(at)bruda(dot)ca>

And here as well, these are errors in the file.

I don't think we want to make lisp-mnt.el into some kind of DWIM
guesswork when we can very easily fix the files.


        Stefan

Re: [ELPA] I regret using subtree

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> There are files with more than 1 maintainer/author.  Just pick up the
>> first one may not suffice.
>
> I disagree here.  E.g. for Tramp, Kai is not involved any more, so the
> right fix is to add a "Maintainer:" to tramp.el.

Done. Will appear in the repo next time I synchronize.

>         Stefan

Best regards, Michael.

Re: [ELPA] I regret using subtree

[Tino Calancha]

On Mon, 13 Mar 2017, Michael Albinus wrote:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>>> There are files with more than 1 maintainer/author.  Just pick up the
>>> first one may not suffice.
>>
>> I disagree here.  E.g. for Tramp, Kai is not involved any more, so the
>> right fix is to add a "Maintainer:" to tramp.el.
>
> Done. Will appear in the repo next time I synchronize.
Wow, that what really fast! without being in CC ;-)