Recommend these .gitconfig settings for git integrity.

Recommend these .gitconfig settings for git integrity.

[Karl Fogel]

I've just added these settings to my ~/.gitconfig.  Based on some recent reports, it might be a good idea for all of us to do the same:

  [transfer]
          fsckObjects = true
  [fetch]
          fsckObjects = true
  [receive]
          fsckObjects = true

Summary:

Although git communicates object ID by content-addressable hashes (thus in theory ensuring integrity), git apparently doesn't always bother to actually *check* the hashes, e.g., when receiving objects from remote repositories.  Enabling the above settings causes git to notice if someone ships you a bogus object, which seems like, er, a win.  You might be worried about a slowdown in some git operations, since content would now be checked against a hash, but according to those who've enabled the settings there's no noticeable slowdown in practice.

So, based on the discussion in the thread below, there are good reasons for everyone to enable these settings, and no reason not to.  (I was kind of surprised they weren't turned on by default in git, actually.)

See this post & thread for more details:

  From: Eric Myhre
  Subject: git integrity
  To: binary-transparency (Google Group)
  Message-ID: <56ABBA5B.6020703@exultant.us>
  Date: Fri, 29 Jan 2016 11:15:39 -0800

  https://groups.google.com/forum/#!topic/binary-transparency/f-BI4o8HZW0

See also these bug tickets mentioned by DKG in the thread:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=813157
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=743227

If we have consensus here, I could add this recommendation to the 'CONTRIBUTE' file in the Emacs tree; I'll wait to see what the followup is before doing that, however.

Best regards,
-Karl

Re: Recommend these .gitconfig settings for git integrity.

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Date: Sun, 31 Jan 2016 14:22:02 -0600
> 
> If we have consensus here, I could add this recommendation to the 'CONTRIBUTE' file in the Emacs tree; I'll wait to see what the followup is before doing that, however.

I think a better place is admin/notes/git-workflow and
http://www.emacswiki.org/emacs/GitForEmacsDevs.  CONTRIBUTE's topics
are different.

Thanks.

Re: Recommend these .gitconfig settings for git integrity.

[Karl Fogel]

Eli Zaretskii <eliz@gnu.org> writes:
>I think a better place is admin/notes/git-workflow and
>http://www.emacswiki.org/emacs/GitForEmacsDevs.  CONTRIBUTE's topics
>are different.

Ah, thank you -- I didn't think of that.  Agreed.  (I hope when I went to look in detail at CONTRIBUTE, I would have spotted the pointers to the above two locations, but anyway now the dereferencing is already done.)

-K

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 470 bytes --]

Karl Fogel wrote:
> Eli Zaretskii<eliz@gnu.org>  writes:
>> >I think a better place is admin/notes/git-workflow and
>> >http://www.emacswiki.org/emacs/GitForEmacsDevs.  CONTRIBUTE's topics
>> >are different.
> Ah, thank you -- I didn't think of that.  Agreed.

We shouldn't need to configure fetch.fsckObjects and receive.fsckObjects, as 
they default to transfer.fsckObjects.

I installed the attached patch to try to automate this; I think this'll be more 
effective.

[-- Attachment #2: 0001-autogen.sh-now-arranges-for-git-to-check-hashes.patch --]
[-- Type: text/x-diff, Size: 906 bytes --]

From b1ccd481738b1427ac0e294f2405a9ebe86c8561 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sun, 31 Jan 2016 13:43:13 -0800
Subject: [PATCH] autogen.sh now arranges for git to check hashes

Suggested by Karl Fogel in:
http://lists.gnu.org/archive/html/emacs-devel/2016-01/msg01802.html
* autogen.sh: Do "git config transfer.fsckObjects true".
---
 autogen.sh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/autogen.sh b/autogen.sh
index a63c53c..877bb84 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -220,6 +220,11 @@ echo timestamp > src/stamp-h.in || exit
 ## Configure Git, if using Git.
 if test -d .git && (git status -s) >/dev/null 2>&1; then
 
+    # Check hashes when transferring objects among repositories.
+
+    git config transfer.fsckObjects true || exit
+
+
     # Configure 'git diff' hunk header format.
 
     git config 'diff.elisp.xfuncname' \
-- 
2.5.0

Re: Recommend these .gitconfig settings for git integrity.

[Karl Fogel]

Quoting Paul Eggert <eggert@cs.ucla.edu> out of order:
>I installed the attached patch to try to automate this; I think
>this'll be more effective.

*nod*  That has autogen.sh install the setting in the repository-specific git config.  That's a good a companion to also including the advice in admin/notes/git-workflow and http://www.emacswiki.org/emacs/GitForEmacsDevs, but by itself it's not enough, I think.  It only takes effect after the user runs autogen.sh, so she might have pulled an arbitrary number of times before the setting gets put in place by autogen.sh.

>We shouldn't need to configure fetch.fsckObjects and
>receive.fsckObjects, as they default to transfer.fsckObjects.

Yeah... Hmmm.  I think Eric Myhre's post recommending all three setting was his too-simple-to-fail way of making sure that users didn't have a combination whereby transfer.fsckObjects=true was overridden by one or both of fetch.fsckObjects=false or receive.fsckObjects=false.

Anyway, we seem to have consensus that the intended resultant setting is a good one.  So I'll add the advice to the above file and wiki page, and will find some way to express the desired result without asking the user to make redundant settings.

Best regards,
-Karl

>>From b1ccd481738b1427ac0e294f2405a9ebe86c8561 Mon Sep 17 00:00:00 2001
>From: Paul Eggert <eggert@cs.ucla.edu>
>Date: Sun, 31 Jan 2016 13:43:13 -0800
>Subject: [PATCH] autogen.sh now arranges for git to check hashes
>
>Suggested by Karl Fogel in:
>http://lists.gnu.org/archive/html/emacs-devel/2016-01/msg01802.html
>* autogen.sh: Do "git config transfer.fsckObjects true".
>---
> autogen.sh | 5 +++++
> 1 file changed, 5 insertions(+)
>
>diff --git a/autogen.sh b/autogen.sh
>index a63c53c..877bb84 100755
>--- a/autogen.sh
>+++ b/autogen.sh
>@@ -220,6 +220,11 @@ echo timestamp > src/stamp-h.in || exit
> ## Configure Git, if using Git.
> if test -d .git && (git status -s) >/dev/null 2>&1; then
> 
>+    # Check hashes when transferring objects among repositories.
>+
>+    git config transfer.fsckObjects true || exit
>+
>+
>     # Configure 'git diff' hunk header format.
> 
>     git config 'diff.elisp.xfuncname' \

Re: Recommend these .gitconfig settings for git integrity.

[Óscar Fuentes]

Karl Fogel <kfogel@red-bean.com> writes:

> Quoting Paul Eggert <eggert@cs.ucla.edu> out of order:
>>I installed the attached patch to try to automate this; I think
>>this'll be more effective.
>
> *nod* That has autogen.sh install the setting in the
> repository-specific git config. That's a good a companion to also
> including the advice in admin/notes/git-workflow and
> http://www.emacswiki.org/emacs/GitForEmacsDevs, but by itself it's not
> enough, I think. It only takes effect after the user runs autogen.sh,
> so she might have pulled an arbitrary number of times before the
> setting gets put in place by autogen.sh.

Why is the Emacs build process underhandedly changing my git
configuration? How doing that improves or benefits the Emacs project?

If those config settings are so good, why don't you spend your energies
on convincing the git maintainers about enabling them by default instead
of pretending that you know better than your users and the git
maintainers themselves?

The Emacs build process has no bussiness on sneaking config changes that
affects how tools work on my system, unless those config changes are
strictly required for the proper functioning of the development process,
and even then a prominent notice should be given while making the config
changes, preferably asking for confirmation.

Re: Recommend these .gitconfig settings for git integrity.

[Stefan Monnier]

> Why is the Emacs build process underhandedly changing my git
> configuration? How doing that improves or benefits the Emacs project?
> If those config settings are so good, why don't you spend your energies
> on convincing the git maintainers about enabling them by default instead
> of pretending that you know better than your users and the git
> maintainers themselves?

I would have put it a bit more softly (I don't really care about those
settings either way), but I tend to agree with the sentiment.
It seems odd.


        Stefan

Re: Recommend these .gitconfig settings for git integrity.

[Karl Fogel]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> Why is the Emacs build process underhandedly changing my git
>> configuration? How doing that improves or benefits the Emacs project?
>> If those config settings are so good, why don't you spend your energies
>> on convincing the git maintainers about enabling them by default instead
>> of pretending that you know better than your users and the git
>> maintainers themselves?
>
>I would have put it a bit more softly (I don't really care about those
>settings either way), but I tend to agree with the sentiment.
>It seems odd.

Ah -- Stefan and Óscar, you might be misunderstanding?

This only changes the configuration in the current repository (your local Emacs repository).  It's not changing your ~/.gitconfig -- that is, running Emacs's autogen.sh won't suddenly affect git's behavior in other repositories on your box.  You would have to pass '--global' to cause it to take effect in your ~/.gitconfig.

Also, note that autogen.sh was *already* modifying the repository-local git config.  Search for "Configure 'git diff' hunk header format" in autogen.sh.

Best regards,
-Karl

Re: Recommend these .gitconfig settings for git integrity.

[Stefan Monnier]

> Ah -- Stefan and Óscar, you might be misunderstanding?
> This only changes the configuration in the current repository (your local
> Emacs repository).

I understand this part, but it's still odd to override Git's defaults here,
since it doesn't seem needed.

> Also, note that autogen.sh was *already* modifying the repository-local git
> config.

I may not have noticed all the things it used to do, but the hooks it
added were in a different category (in my book), because they did make
a change that was "required" by Emacs's coding conventions.


        Stefan

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/01/2016 11:12 AM, Stefan Monnier wrote:
> I may not have noticed all the things it used to do, but the hooks it
> added were in a different category (in my book), because they did make
> a change that was "required" by Emacs's coding conventions.

autogen.sh already configured Git to do things that are not required by 
Emacs's coding conventions. For example, it configures 
diff.texinfo.xfuncname as a matter of convenience, to avoid the sorts of 
hassles Alan Mackenzie noted on this list in September, not because 
Emacs coding conventions require it.

autogen.sh should limit itself to Git settings suitable for Emacs 
developers that are not the Git default. My impression is that 
transfer.fsckObjects falls into this category, but if this is not the 
consensus of course we should change autogen.sh to stop configuring that 
particular setting. I haven't heard any argument against that particular 
setting, though.

Re: Recommend these .gitconfig settings for git integrity.

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Mon, 1 Feb 2016 11:56:25 -0800
> Cc: emacs-devel@gnu.org
> 
> I haven't heard any argument against that particular setting,
> though.

It's a general security-related issue that Emacs shouldn't force on
its users.

Re: Recommend these .gitconfig settings for git integrity.

[Stefan Monnier]

>> I haven't heard any argument against that particular setting, though.

FWIW, I like this setting and added it to my ~/.gitconfig.
I just find it odd to put it in Emacs's autogen.sh.

> It's a general security-related issue that Emacs shouldn't force on
> its users.

At least not before we lobby the Git crowd to turn it on by default.


        Stefan

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/01/2016 12:28 PM, Eli Zaretskii wrote:
> It's a general security-related issue

As I understand things it's more general than that. Setting transfer.fsckObjects 
also catches things like transmission and storage errors.

Should we pay the checking overhead to catch these reliability and security 
problems? That's the question. Emacs developers who are using this setting 
report that the overhead is small enough as to not be a significant issue, which 
suggests that the extra checking is a good idea.

> that Emacs shouldn't force on its users.

The script is intended to be run only by Emacs developers, and is optional even 
for developers. It's certainly not forcing anything on Emacs users.


On 02/01/2016 01:40 PM, Stefan Monnier wrote:
> At least not before we lobby the Git crowd to turn it on by default.

Any such lobbying effort could well fail, as the checking overhead can be 
substantial in some projects, which means that changing the default for all 
Git-using projects could well be a bad idea.

Besides, if the idea is a good one for Emacs development, then even if we 
successfully lobby for it to be the Git default it will take quite some time for 
the change to propagate, and in the meantime we win by setting the Git 
configuration appropriately for Emacs. Conversely, if our lobbying effort fails, 
we still win by setting the Git configuration appropriately for Emacs now.


On 02/01/2016 12:34 PM, Óscar Fuentes wrote:
> The script I use for building Emacs contains
>
> git clean -f -d -x
> ./autogen.sh

Surely this script is unaffected by the benign git config settings that 
autogen.sh installs. However, as you're worried about the possibility, I added 
support for a new option to autogen.sh, so if you change that script's autogen 
line to:

    ./autogen.sh --git-config=false

then autogen.sh won't use "git config" and will not install the setting that 
started this thread.

> How about mentioning (not applying) the recommended settings on autogen.sh

That would make autogen.sh less convenient to use. I largely ignore autogen.sh's 
output, and I expect it to work without my having to fiddle with Git 
configuration or hooks by hand afterwards. Besides, autogen.sh has been 
configuring Git for quite some time with no practical problems reported; why 
change something that isn't broken?

> How those settings benefit Emacs development?

Assuming you're asking about the recent change, there is some justification for 
it in the first part of this email.

Re: Recommend these .gitconfig settings for git integrity.

[John Wiegley]

>>>>> Paul Eggert <eggert@cs.ucla.edu> writes:

> Should we pay the checking overhead to catch these reliability and security
> problems? That's the question. Emacs developers who are using this setting
> report that the overhead is small enough as to not be a significant issue,
> which suggests that the extra checking is a good idea.

Even still, this is purely a Git matter, and not an Emacs matter. This choice
shouldn't be automatically made on user's machine, however beneficial. It's
been recommended now in git-workflow, but should not be applied automatically.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Recommend these .gitconfig settings for git integrity.

[Stefan Monnier]

>> At least not before we lobby the Git crowd to turn it on by default.
> Any such lobbying effort could well fail,

The outcome of the lobbying is of no importance, really.

My point is that if someone thinks she needs/wants to a change some
setting for reasons which would seem to apply to most people if not
everybody, then she should lobby for the defaults to change.

That's what I want Emacs users to do w.r.t Emacs defaults.  And I'd
expect that it's also what Git developer want Git users to do.


        Stefan

Re: Recommend these .gitconfig settings for git integrity.

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

[snip]

>> How those settings benefit Emacs development?
>
> Assuming you're asking about the recent change, there is some
> justification for it in the first part of this email.

That justification relies purely on personal preferences.

Have you asked the git maintainers why the check is not activated by
default? I mean, it look like it does a good thing, if you read the
description. So why it is not activated by default? Maybe it has some
drawbacks. Anyone cared about that possibility, apart from the "I
activated it and so far, so good" testimony from *one* individual?

Anyways, what we have here is two Emacs hackers (one, at this point)
which use their commit powers for setting tool preferences on other's
repos just because "it seems a good thing". Not because the setting is
required by the Emacs development process, neither because it helps
hackers on doing their chores. Someone puts a message on a forum and
bam! two days later the Emacs build process is changed to sneak the
setting on everyone's repo, as if it fixed some super-critical problem.
That's worrying.

What's even more disturbing is that those settings are still there, no
matter the disapproval of other Emacs hackers, including the head
maintainer. And at this point the setting was already forcefully
installed on who-knows-many Emacs clones, and growing. Let's hope it
doesn't backfire.

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/02/2016 07:49 AM, Óscar Fuentes wrote:

> I mean, it look like it does a good thing, if you read the
> description. So why it is not activated by default? Maybe it has some
> drawbacks.
Yes, it has a drawback. I mentioned it in my previous message. On some 
larger projects, it slows down some git operations. However, this does 
not seem to be a significant problem with Emacs development.

In contrast, omitting the option can cause significant problems later, 
problems that can be hard to discover and even harder to repair. For an 
example of this, please see the following bugreport on GitHub:

https://github.com/wp-cli/php-cli-tools/issues/70

Apparently the wp-cli project has a corrupted Git history that could 
have been prevented by the new setting. It is so much trouble to fix 
this that the project's maintainer and the GitHub staff think that 
fixing it is more trouble than it's worth, and will just live with the 
corrupted repository. I'd hate to see the same thing happen to the Emacs 
source code repository.

> Anyone cared about that possibility, apart from the "I
> activated it and so far, so good" testimony from*one*  individual?

I've been using it. So has Stefan, and Karl. Nobody who has used it has 
reported problems. If there were real problems with this option we 
should not make it the default.

I will look into modifying autogen.sh to make this flag optional, since 
there seems to be significant opposition to it. I still don't understand 
why there's so much opposition, though. If the Emacs repository becomes 
corrupted because this option was omitted, quite possibly we won't 
notice, and even if we notice quite possibly we won't fix it. I would 
hate to see that happen.

Re: Recommend these .gitconfig settings for git integrity.

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 02/02/2016 07:49 AM, Óscar Fuentes wrote:
>
>> I mean, it look like it does a good thing, if you read the
>> description. So why it is not activated by default? Maybe it has some
>> drawbacks.
> Yes, it has a drawback. I mentioned it in my previous message. On some
> larger projects, it slows down some git operations. However, this does
> not seem to be a significant problem with Emacs development.

Care to explain this in more detail? Emacs is no small project. If the
slow down affects large transfers ("large" meaning either many objects
or big objects) what happens if an Emacs hackers pauses his activity for
several months and then pulls? (after monitoring emacs-diffs for several
years, I can attest that this scenario is quite frequent.)

> In contrast, omitting the option can cause significant problems later,
> problems that can be hard to discover and even harder to repair. For
> an example of this, please see the following bugreport on GitHub:
>
> https://github.com/wp-cli/php-cli-tools/issues/70
>
> Apparently the wp-cli project has a corrupted Git history that could
> have been prevented by the new setting. It is so much trouble to fix
> this that the project's maintainer and the GitHub staff think that
> fixing it is more trouble than it's worth, and will just live with the
> corrupted repository. I'd hate to see the same thing happen to the
> Emacs source code repository.

So the GitHub folks, on their early days, had a bug on their Git
re-implementation and caused some hosted project to become "infected".
This looks like a far-fetched case from the Emacs POV. If we wish to
avoid tainted objects created by whatever cause, the check can be
enabled on the Savannah's repo, hence limiting the problem to the
"infected" user. The problem seems to be so rare that a single Emacs
hacker experiencing it every decade or so (if it ever happens) doesn't
warrant the risks and inconveniences associated with using the setting
(see my previous messages.)

>> Anyone cared about that possibility, apart from the "I
>> activated it and so far, so good" testimony from*one*  individual?
>
> I've been using it. So has Stefan, and Karl. Nobody who has used it
> has reported problems. If there were real problems with this option we
> should not make it the default.

So you, Karl and Stefan used the feature... for how much time? Two days?
Are you a reprensetative sample of the overwhelming majority of the
Emacs devel populationt? (No, you aren't). Sorry but imposing the config
change to everybody's repo so soon and without discussion looks like a
mindlessly hurried-up move, to say it bluntly.

> I will look into modifying autogen.sh to make this flag optional,
> since there seems to be significant opposition to it.

Thanks, although it is now a bit late. It is already installed on
several repos, for sure. And anybody that stumbles on the revisions that
contained your change (by bisecting some bug, for example) will have his
.git/config file modified. I'm pretty sure that you didn't think of this
issues when you made the change (neither I did at first.)

> I still don't
> understand why there's so much opposition, though. If the Emacs
> repository becomes corrupted because this option was omitted, quite
> possibly we won't notice, and even if we notice quite possibly we
> won't fix it. I would hate to see that happen.

I appreciate your concern but that has an easy solution: enable it on
the server (and on your own machine, to be extra sure.) See, problem
solved. Now that you are at it, convince the Savannah admins to enable
the check on all git repos they host. That is much more reasonable than
automatically changing the configuration of each clone out there.

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/02/2016 10:48 AM, Óscar Fuentes wrote:
> Emacs is no small project.

Emacs is considerably smaller than other projects that Git regularly deals with. 
The Emacs master branch has 125,000 commits; the Linux kernel has 574,000 
commits in its master branch (this omits history before 2005). I've seen reports 
of git repositories at Facebook where the .git subdirectory contains 50 GB. By 
comparison, my repository for Emacs master has 0.25 GB and for the Linux 
kernelhas 1.7 GB. These numbers can vary quite a bit depending on packing and so 
forth; still, the point remains that Emacs is not nearly as large as other 
projects that use git.

> If the slow down affects large transfers ("large" meaning either many objects 
> or big objects) what happens if an Emacs hackers pauses his activity for 
> several months and then pulls? (after monitoring emacs-diffs for several 
> years, I can attest that this scenario is quite frequent.) 

I doubt whether it will slow down such pulls significantly for typical Emacs 
development. I just now did a simple benchmark that cloned Emacs master from 
savannah to my desktop at UCLA, and the overhead from the transfer.fsckObjects 
setting was swamped by noise due to the network being slower or faster.Here are 
a few details:

     average times
     real      user+system   fsckObjects value
     212.4       202.6           true
     217.1       195.7           false

     The command used was:
     git clone --config transfer.fsckObjects=VALUE git://git.sv.gnu.org/emacs.git

     I warmed up with a clone that I discarded. I then tried three of
     each command, interleaved, and took the averages.

This is just one benchmark of course, but it's suggestive.

> If we wish to avoid tainted objects created by whatever cause, the check can 
> be enabled on the Savannah's repo, hence limiting the problem to the 
> "infected" user.

The problem would not be limited to the infected user, if that user pushes 
commits. Git on the client could remove the taint without actually fixing the 
problem. Although the pushed commits would have checksums that match their data, 
the pushed data would be corrupt.

A possible source of problems in this area is an attack on the integrity of the 
Emacs repository by a determined outsider. Any such attacks cannot be discounted 
by appealing to estimates based purely on counts of random hardware or software 
or configuration errors, as the attacks would not be random.

> The problem seems to be so rare that a single Emacs hacker experiencing it 
> every decade or so

Perhaps you're right, but perhaps not. Really, we do not know how common this 
particular problem is. In my experience strange problems with git occur more 
often than once per decade.

> (doesn't warrant the risks and inconveniences associated with using the 
> setting (see my previous messages.)

Although I recall concerns based on hypothetical scenarios, I don't recall 
descriptions of real inconveniences. Perhaps I missed an email or two.

> Are you a reprensetative sample of the overwhelming majority of the Emacs 
> devel populationt? (No, you aren't).

True. I expect I use git more than the average Emacs developer does. Stefan 
too.So we will probably be more affected than usual by this change.

> Thanks, although it is now a bit late. It is already installed on
> several repos, for sure. And anybody that stumbles on the revisions that
> contained your change (by bisecting some bug, for example) will have his
> .git/config file modified. I'm pretty sure that you didn't think of this
> issues when you made the change (neither I did at first.)

No, I anticipated these issues. There's no evidence that they are significant 
problems.git bisect will still work.

> I appreciate your concern but that has an easy solution: enable it on the 
> server (and on your own machine, to be extra sure.) See, problem solved

That would not solve the problem. True, it would help against some random 
errors, but it won't work in general even there, much less against a determined 
attack.

I take your point that there's no rush in making this change, and it will be 
helpful to gain experience on it among developers who prefer a more 
bleeding-edge environment, so I have reverted the change on emacs-25 and 
installed a considerably more conservative approach on master to help us get 
started. The new approach does not alter git configuration if a developer 
invokes plain './autogen.sh'. Instead a developer can invoke the script with a 
newly-introduced extension: either './autogen.sh git' to configure just git, or 
'./autogen.sh all' to configure both autoconf and git. I hope this helps address 
the concerns raised in this thread.

Re: Recommend these .gitconfig settings for git integrity.

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 02/02/2016 10:48 AM, Óscar Fuentes wrote:
>> Emacs is no small project.
>
> Emacs is considerably smaller than other projects that Git regularly
> deals with. The Emacs master branch has 125,000 commits; the Linux
> kernel has 574,000 commits in its master branch (this omits history
> before 2005). I've seen reports of git repositories at Facebook where
> the .git subdirectory contains 50 GB. By comparison, my repository for
> Emacs master has 0.25 GB and for the Linux kernelhas 1.7 GB. These
> numbers can vary quite a bit depending on packing and so forth; still,
> the point remains that Emacs is not nearly as large as other projects
> that use git.

The fact that there are projects with a larger repo than Emacs doesn't
mean that Emacs is not large.

>> If the slow down affects large transfers ("large" meaning either
>> many objects or big objects) what happens if an Emacs hackers pauses
>> his activity for several months and then pulls? (after monitoring
>> emacs-diffs for several years, I can attest that this scenario is
>> quite frequent.) 
>
> I doubt whether it will slow down such pulls significantly for typical
> Emacs development. I just now did a simple benchmark that cloned Emacs
> master from savannah to my desktop at UCLA, and the overhead from the
> transfer.fsckObjects setting was swamped by noise due to the network
> being slower or faster.Here are a few details:
>
>     average times
>     real      user+system   fsckObjects value
>     212.4       202.6           true
>     217.1       195.7           false
>
>     The command used was:
>     git clone --config transfer.fsckObjects=VALUE git://git.sv.gnu.org/emacs.git
>
>     I warmed up with a clone that I discarded. I then tried three of
>     each command, interleaved, and took the averages.
>
> This is just one benchmark of course, but it's suggestive.

You are hypothesizing about the reasons why fsckObjects is not the
default. Instead of making up reasons and proving yourself right, the
reasonable thing to do is to ask the git maintainers. If they respond
citing performance concerns, you prove *them* wrong with some
experiments, so they get convinced about enabling the option.

I woulnd't trust any experiment about fsckObjects without asking the git
maintainers about its specific drawbacks (not just "performance", but
"performance on such and such scenario.")

>> If we wish to avoid tainted objects created by whatever cause, the
>> check can be enabled on the Savannah's repo, hence limiting the
>> problem to the "infected" user.
>
> The problem would not be limited to the infected user, if that user
> pushes commits. Git on the client could remove the taint without
> actually fixing the problem. Although the pushed commits would have
> checksums that match their data, the pushed data would be corrupt.
>
> A possible source of problems in this area is an attack on the
> integrity of the Emacs repository by a determined outsider. Any such
> attacks cannot be discounted by appealing to estimates based purely on
> counts of random hardware or software or configuration errors, as the
> attacks would not be random.

So enabling the check on the server doesn't protect it, but for
protecting the server the check must be enabled on the user's clone...
That doesn't make sense. It either works or doesn't. If you are able to
push tainted objects to a server that has the check enabled, you also
can put them to the user's repo.

>> The problem seems to be so rare that a single Emacs hacker
>> experiencing it every decade or so
>
> Perhaps you're right, but perhaps not. Really, we do not know how
> common this particular problem is. In my experience strange problems
> with git occur more often than once per decade.

Yes, and it is reasonable to expect that as more features you activate,
more problems can arise.

>> (doesn't warrant the risks and inconveniences associated with using
>> the setting (see my previous messages.)
>
> Although I recall concerns based on hypothetical scenarios, I don't
> recall descriptions of real inconveniences. Perhaps I missed an email
> or two.

See above.

[snip]

>> Thanks, although it is now a bit late. It is already installed on
>> several repos, for sure. And anybody that stumbles on the revisions that
>> contained your change (by bisecting some bug, for example) will have his
>> .git/config file modified. I'm pretty sure that you didn't think of this
>> issues when you made the change (neither I did at first.)
>
> No, I anticipated these issues. There's no evidence that they are
> significant problems.git bisect will still work.

Of course bisect will work. My point is that running autogen.sh on any
of those commits will put the setting on .git/config, no matter that you
reverted autogen.sh on a subsequent commit. So people like me, who often
use bisect, will end with the setting on .git/config again and again.

And we are talking about a 4 day timeframe. Too early for talking about
evidences.

>> I appreciate your concern but that has an easy solution: enable it
>> on the server (and on your own machine, to be extra sure.) See,
>> problem solved
>
> That would not solve the problem. True, it would help against some
> random errors, but it won't work in general even there, much less
> against a determined attack.
>
> I take your point that there's no rush in making this change, and it
> will be helpful to gain experience on it among developers who prefer a
> more bleeding-edge environment, so I have reverted the change on
> emacs-25 and installed a considerably more conservative approach on
> master to help us get started. The new approach does not alter git
> configuration if a developer invokes plain './autogen.sh'. Instead a
> developer can invoke the script with a newly-introduced extension:
> either './autogen.sh git' to configure just git, or './autogen.sh all'
> to configure both autoconf and git. I hope this helps address the
> concerns raised in this thread.

Thank you. But in the future, I would appreciate if you discuss a change
like this before rushing to commit it and cause irreversible
consequences.

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/03/2016 08:20 AM, Óscar Fuentes wrote:
> Instead of making up reasons and proving yourself right, the
> reasonable thing to do is to ask the git maintainers.

I'd rather not waste their time on this unless the concerns are 
warranted. We would ill-serve Emacs development by giving ourselves a 
reputation for cluelessness. Let's wait for real problems to turn up 
before ringing alarm bells elsewhere.

> It either works or doesn't.

It does both, and we're concerned about the part that doesn't work. A 
client can operate for a while with tainted objects, and do (say) a 
rebase that computes new checksums for the bad data, and then ship the 
tainted data off to the server with the newly computed checksums. The 
server can't detect the problem because the recomputed checksums match 
the tainted data.

> in the future, I would appreciate if you discuss a change like this 
> before rushing to commit it and cause irreversible consequences. 

The consequences are largely benign and entirely reversible.

I often install changes to Emacs, and obviously can't clear each change 
before installing it, as that would slow us all down for little benefit. 
So I use my best judgment to decide which changes to install in master, 
which in emacs-25, and which to discuss first. I sometimes make mistakes 
in that area, for which I apologize, but what can I say? The only way to 
prevent all mistaken changes is to not make any changes. This change 
seemed like a no-brainer at the time, but we've decided to take a more 
cautious approach for now. Such things happen, and life moves on.

Re: Recommend these .gitconfig settings for git integrity.

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

[snip]

I give up. I already wasted too much energy on this issue. As far as I'm
concerned, do as you please.

Re: Recommend these .gitconfig settings for git integrity.

[Stefan Monnier]

> I give up. I already wasted too much energy on this issue. As far as I'm
> concerned, do as you please.

Not sure what could be done at this point, really.


        Stefan

Re: Recommend these .gitconfig settings for git integrity.

[Karl Fogel]

Paul Eggert <eggert@cs.ucla.edu> writes:
>I will look into modifying autogen.sh to make this flag optional,
>since there seems to be significant opposition to it. I still don't
>understand why there's so much opposition, though. If the Emacs
>repository becomes corrupted because this option was omitted, quite
>possibly we won't notice, and even if we notice quite possibly we
>won't fix it. I would hate to see that happen.

I agree, but there's a clear majority (at least in this thread, which is all the information we have) in opposition to the setting being placed by autogen.sh, even with chatter.  I think the right thing to do would be to revert it now.

I'm not sure what it would mean to make the flag optional in autogen.sh.  Does it mean autogen.sh would interactively query the user or something?  If autogen.sh isn't going to do this automatically, then probably autogen.sh shouldn't do it at all, and we just have to rely on the advice in git-workflow and other places.  Again, I personally agree with your reasoning & preference, Paul, but we're a minority and I think it's clear that reverting is the right step now.  Just because one doesn't understand the opposition doesn't mean one can ignore it, when it's the majority.  (The results last night out of Iowa here in the U.S. reminded me of this :-) .)

Best regards,
-Karl

Re: Recommend these .gitconfig settings for git integrity.

[Lars Ingebrigtsen]

Karl Fogel <kfogel@red-bean.com> writes:

> I agree, but there's a clear majority (at least in this thread, which
> is all the information we have) in opposition to the setting being
> placed by autogen.sh, even with chatter.  I think the right thing to
> do would be to revert it now.

Well, that may be because people who think that it's a sensible to thing
to don't really care that much.  :-)

I don't see any reason why Emacs shouldn't come with sensible defaults
for Emacs development.  We provide a .gdbinit file that makes debugging
more pleasant.  We determine what gcc parameters we should use by
default, even if some developers like other parameters.  Etc etc etc.

Why is git different?  Is there something magical and mystical about git
that means that every developer should and must type these incantations
by hand?

There's this tendency amongst programmers to provide a blank framework
that has to be configured before you start using it, because cleanliness
etc.  This rant is somewhat appropriate here, I think:

http://robotlolita.me/2016/01/09/no-i-dont-want-to-configure-your-app.html

Emacs is pretty opinionated, and I think Emacs can continue to opine,
even when it comes to sacred stuff like git.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Recommend these .gitconfig settings for git integrity.

[John Wiegley]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

>>>>> Karl Fogel <kfogel@red-bean.com> writes:

> I agree, but there's a clear majority (at least in this thread, which is all
> the information we have) in opposition to the setting being placed by
> autogen.sh, even with chatter. I think the right thing to do would be to
> revert it now.

Yes, please revert this change.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 629 bytes --]

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/02/2016 06:16 PM, John Wiegley wrote:
> Yes, please revert this change.

I did that, about a dozen minutes before you sent your email.

Although I have some further changes planned, (1) I plan to install them 
on the master branch and leave emacs-25 alone, and (2) I plan to arrange 
things so that running the command './autogen.sh' does not change the 
git configuration. That is, git configuration changes will be done by 
running autogen.sh with an explicit argument, with the default being to 
leave git configuration alone. This should be enough to allay the 
reasonable concerns expressed in this area.

Re: Recommend these .gitconfig settings for git integrity.

[John Wiegley]

>>>>> Paul Eggert <eggert@cs.ucla.edu> writes:

> Although I have some further changes planned, (1) I plan to install them on
> the master branch and leave emacs-25 alone, and (2) I plan to arrange things
> so that running the command './autogen.sh' does not change the git
> configuration. That is, git configuration changes will be done by running
> autogen.sh with an explicit argument, with the default being to leave git
> configuration alone. This should be enough to allay the reasonable concerns
> expressed in this area.

Thank you, Paul!

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Recommend these .gitconfig settings for git integrity.

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Tue, 2 Feb 2016 18:26:38 -0800
> 
> I plan to arrange things so that running the command './autogen.sh'
> does not change the git configuration. That is, git configuration
> changes will be done by running autogen.sh with an explicit
> argument, with the default being to leave git configuration alone.

No objections here, but I wonder: how is that different from just
telling people who track the development to run the corresponding
"git config" command?  Both require a deliberate action, and neither
of the commands is significantly more complex than the other one.

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/03/2016 07:47 AM, Eli Zaretskii wrote:
> No objections here, but I wonder: how is that different from just
> telling people who track the development to run the corresponding
> "git config" command?  Both require a deliberate action, and neither
> of the commands is significantly more complex than the other one.
I'm afraid it's more complicated than that. './autogen.sh all' currently 
configures three Git variables with values like 
'^\(def[^[:space:]]+[[:space:]]+([^()[:space:]]+)', avoids touching the 
configuration if it's already correct for other reasons, and on 
GNU/Linux hosts makes backups of all its configuration-file changes. I'd 
hate to cut and paste all that stuff out of a text file.

Re: Recommend these .gitconfig settings for git integrity.

[Eli Zaretskii]

> Cc: kfogel@red-bean.com, ofv@wanadoo.es, emacs-devel@gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Wed, 3 Feb 2016 09:40:22 -0800
> 
> On 02/03/2016 07:47 AM, Eli Zaretskii wrote:
> > No objections here, but I wonder: how is that different from just
> > telling people who track the development to run the corresponding
> > "git config" command?  Both require a deliberate action, and neither
> > of the commands is significantly more complex than the other one.
> I'm afraid it's more complicated than that. './autogen.sh all' currently 
> configures three Git variables with values like 
> '^\(def[^[:space:]]+[[:space:]]+([^()[:space:]]+)', avoids touching the 
> configuration if it's already correct for other reasons, and on 
> GNU/Linux hosts makes backups of all its configuration-file changes. I'd 
> hate to cut and paste all that stuff out of a text file.

No, I was talking only about the transfer.fsckObjects setting, not
about the others.  Sorry for being unclear.

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/03/2016 09:52 AM, Eli Zaretskii wrote:
> I was talking only about the transfer.fsckObjects setting, not
> about the others.

Ah, in that case I don't see why we should single out this Git setting 
from all the others, in a document intended for people starting to 
develop Emacs. We should strive for simple instructions, something like 
"git clone URL; cd emacs; make" rather than a treatise on Git 
configuration that will be confusing and off-putting. If anything, even 
the current instructions are too complicated.

If this setting is generally advisable for Emacs developers, it should 
be done as part of the default simple setup. If not, not.

Re: Recommend these .gitconfig settings for git integrity.

[Lars Ingebrigtsen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 02/03/2016 09:52 AM, Eli Zaretskii wrote:
>> I was talking only about the transfer.fsckObjects setting, not
>> about the others.

[...]

> If this setting is generally advisable for Emacs developers, it should
> be done as part of the default simple setup. If not, not.

I agree 100%.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Recommend these .gitconfig settings for git integrity.

[Eli Zaretskii]

> From: Paul Eggert <eggert@cs.ucla.edu>
> Cc: Eli Zaretskii <eliz@gnu.org>, Stefan Monnier <monnier@IRO.UMontreal.CA>,
>  Karl Fogel <kfogel@red-bean.com>, Óscar Fuentes
>  <ofv@wanadoo.es>
> Date: Tue, 2 Feb 2016 00:02:10 -0800
> 
> > that Emacs shouldn't force on its users.
> 
> The script is intended to be run only by Emacs developers, and is optional even 
> for developers. It's certainly not forcing anything on Emacs users.

Granted, that sentence alluded not to users of Emacs the editor, but
of the Emacs repository.  Myself, you, Óscar, and all the rest of us
here.

Re: Recommend these .gitconfig settings for git integrity.

[John Wiegley]

>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Why is the Emacs build process underhandedly changing my git configuration?
>> How doing that improves or benefits the Emacs project? If those config
>> settings are so good, why don't you spend your energies on convincing the
>> git maintainers about enabling them by default instead of pretending that
>> you know better than your users and the git maintainers themselves?

> I would have put it a bit more softly (I don't really care about those
> settings either way), but I tend to agree with the sentiment. It seems odd.

I second that agreement.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 738 bytes --]

On 02/01/2016 08:01 AM, Óscar Fuentes wrote:
> If those config settings are so good, why don't you spend your energies
> on convincing the git maintainers

Emacs has its own development style and processes, which we can't expect 
the rest of the world to adopt. The Git maintainers support git 
configuration settings and hooks like that in Git, precisely because no 
single configuration will work for every project.

> The Emacs build process has no bussiness on sneaking config changes that
> affects how tools work on my system

Yes, the developer should be told. This was already done for Git hooks, 
but I forgot to do it for Git configuration settings. I fixed that by 
installing the attached patch into emacs-25.

[-- Attachment #2: 0001-Chatter-when-autogen.sh-changes-Git-configuration.patch --]
[-- Type: application/x-patch, Size: 1746 bytes --]

Re: Recommend these .gitconfig settings for git integrity.

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 02/01/2016 08:01 AM, Óscar Fuentes wrote:
>> If those config settings are so good, why don't you spend your energies
>> on convincing the git maintainers
>
> Emacs has its own development style and processes, which we can't
> expect the rest of the world to adopt.

There is nothing Emacs-specific on those settings, so this argument is
not valid.

Paul and Karl: I know you are doing this on good faith and I appreciate
your efforts, but this approach of changing the .git/config file (and
the emacs' .git/config file is also on my system, Karl) is a bit
worrying. If the change produces some side-effect, it's the user who is
in charge of investigating what's going on and imagine his surprise
when, after looking at .git/config in desperation, he sees it edited.

>> The Emacs build process has no bussiness on sneaking config changes that
>> affects how tools work on my system
>
> Yes, the developer should be told. This was already done for Git
> hooks, but I forgot to do it for Git configuration settings. I fixed
> that by installing the attached patch into emacs-25.

Thanks, but I would prefer a `make' target or script and a mention on
CONTRIBUTING about all the good it will cause if the user ever decides
to execute it.

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/01/2016 08:51 AM, Óscar Fuentes wrote:
> I would prefer a `make' target or script

It is already in a script, and the script plainly notifies developers of 
the changes it makes to files under ./.git. Yes, in theory the process 
could be abused by an overenthusiastic script, but that is true for 
everything else the script does. In practice it's a win to run Git in 
way suitable for Emacs development, just as it's a win to run Autoconf 
and Automake in a way suitable for Emacs development.

> There is nothing Emacs-specific on those settings

Each of those settings is there to support Emacs development. It is true 
that other projects could use similar settings. For example, another 
project could also check that commit messages be in UTF-8. However, such 
a check is not appropriate for Git in general, as Git supports 
developers who use non-UTF-8 encodings for their commit messages. So we 
can't rely on vanilla Git to check that commit messages use UTF-8.

Re: Recommend these .gitconfig settings for git integrity.

[Óscar Fuentes]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 02/01/2016 08:51 AM, Óscar Fuentes wrote:
>> I would prefer a `make' target or script
>
> It is already in a script,

Oh, please...

> and the script plainly notifies developers
> of the changes it makes to files under ./.git.

That's a good thing, but still not enough. For starters, if I edit
.git/config, next time autogen.sh is run it will revert the edits,
right? The script I use for building Emacs contains

git clean -f -d -x
./autogen.sh

... to ensure that the build is not tainted by previous builds.

How about mentioning (not applying) the recommended settings on
autogen.sh, with instructions for running a script that makes the
changes? That ensures that the user is aware of the changes, and agrees
with them.

[snip]

>> There is nothing Emacs-specific on those settings
>
> Each of those settings is there to support Emacs development.

How those settings benefit Emacs development? (In contrast to any other
project that uses git.) I asked that on my first message, but there is no
answer yet.

> It is true that other projects could use similar settings. For
> example, another project could also check that commit messages be in
> UTF-8. However, such a check is not appropriate for Git in general, as
> Git supports developers who use non-UTF-8 encodings for their commit
> messages. So we can't rely on vanilla Git to check that commit
> messages use UTF-8.

Okay, you explained why Emacs checks that UTF-8 is used on the commit
messages of their repo (I thought that git records the text encoding and
then converts it before display; that's basic functionality for a
distributed VCS, but if you say that an explicit setting is required for
avoiding making a mess of the VC history, I believe you.)

Now, what about the reasoning for the recent changes, in the context of
Emacs development? (Other than "they look like a good thing to me, so I
will put them on everyone's repo.")

Re: Recommend these .gitconfig settings for git integrity.

[Karl Fogel]

Óscar Fuentes <ofv@wanadoo.es> writes:
>There is nothing Emacs-specific on those settings, so this argument is
>not valid.
>
>Paul and Karl: I know you are doing this on good faith and I appreciate
>your efforts, but this approach of changing the .git/config file (and
>the emacs' .git/config file is also on my system, Karl) is a bit
>worrying. If the change produces some side-effect, it's the user who is
>in charge of investigating what's going on and imagine his surprise
>when, after looking at .git/config in desperation, he sees it edited.
>
>>> The Emacs build process has no bussiness on sneaking config changes that
>>> affects how tools work on my system
>>
>> Yes, the developer should be told. This was already done for Git
>> hooks, but I forgot to do it for Git configuration settings. I fixed
>> that by installing the attached patch into emacs-25.
>
>Thanks, but I would prefer a `make' target or script and a mention on
>CONTRIBUTING about all the good it will cause if the user ever decides
>to execute it.

Hmmm.  I know of no way to resolve what is essentially a difference in taste.  It would bother me if emacs/autogen.sh made changes to my global ~/.gitconfig, but it doesn't bother me if it makes changes to emacs/.git/config.  Whereas you are bothered in both cases.

Emacs's autogen.sh already does things that cause a permanent difference in the behavior of, say, the 'configure' and 'make' tools when run in the Emacs tree.  Why should autogen.sh not do the same for the 'git' tool when run in the Emacs tree?  Is there some reason why the Emacs developers as a group can't decide that integrity-checking should be turned on for git data transfers?  After all, if the developer group decided that some sort of integrity checking should be turned on by default for the build process itself, we wouldn't have any problem with that.  In other words, how is this really different?

However, I'm pretty sure you thought of all those arguments already, and are just not convinced by them.  It's Paul's commits in question, so I don't feel I'm in the hot seat for deciding whether or not to revert them :-).  But FWIW I think they are a good idea, and are consistent with principles we'd use for deciding how any other tool should behave when invoked on the Emacs source tree.

(In any case, the documentation changes I'm making are not affected by this.)

Best regards,
-Karl

Re: Recommend these .gitconfig settings for git integrity.

[Óscar Fuentes]

Karl Fogel <kfogel@red-bean.com> writes:

> Hmmm. I know of no way to resolve what is essentially a difference in
> taste. It would bother me if emacs/autogen.sh made changes to my
> global ~/.gitconfig, but it doesn't bother me if it makes changes to
> emacs/.git/config. Whereas you are bothered in both cases.

emacs/.git/config is in my machine, I work with that repo and if
something breaks, I must deal with it, not you. Even if the changes
themselves does not cause any breakage, as soon as I have some other
issue and look at .git/config I'll see those changes; finding those
settings there will cause suspicion and distract me from fixing the real
problem.

> Emacs's autogen.sh already does things that cause a permanent
> difference in the behavior of, say, the 'configure' and 'make' tools
> when run in the Emacs tree. Why should autogen.sh not do the same for
> the 'git' tool when run in the Emacs tree?

*If* the config changes are *required*, I'm all for it, giving the
developer a prominent notice. But what is at stake here is something
very different. You found something that looks like a "something
sensible for the cautious" feature type, and Paul is imposing it on
everyone's Emacs repo. The Emacs developement doesn't need that setting
at all, nor it affects the regular Emacs hacker, who only communicates
with the upstream Emacs repo. And if you worry about the possibility of
the upstream Emacs repo being tainted, just enable the checking on some
build bots (and/or on your machine).

> Is there some reason why
> the Emacs developers as a group can't decide that integrity-checking
> should be turned on for git data transfers?

The Emacs developers as a group are entitled to have a say on what is
accepted on the upstream repo, but never on unilaterally changing how my
Emacs repo is configured.

> After all, if the
> developer group decided that some sort of integrity checking should be
> turned on by default for the build process itself, we wouldn't have
> any problem with that. In other words, how is this really different?

See above.

> However, I'm pretty sure you thought of all those arguments already,
> and are just not convinced by them. It's Paul's commits in question,
> so I don't feel I'm in the hot seat for deciding whether or not to
> revert them :-). But FWIW I think they are a good idea, and are
> consistent with principles we'd use for deciding how any other tool
> should behave when invoked on the Emacs source tree.

I think that, given how it was implemented, this change is bad on
itself, and establishes a bad precedent. See my reply to Paul for some
ideas about how to improve the implementation. Requiring from the user
an informed consent is the right thing here. After all, if you convince
the user about the convenience of those settings, maybe he will apply
them on his other repos, which is a much better outcome from your POV, I
guess.

[snip]

Re: Recommend these .gitconfig settings for git integrity.

[Karl Fogel]

Óscar Fuentes <ofv@wanadoo.es> writes:
>*If* the config changes are *required*, I'm all for it, giving the
>developer a prominent notice. But what is at stake here is something
>very different. You found something that looks like a "something
>sensible for the cautious" feature type, and Paul is imposing it on
>everyone's Emacs repo. The Emacs developement doesn't need that setting
>at all, nor it affects the regular Emacs hacker, who only communicates
>with the upstream Emacs repo. And if you worry about the possibility of
>the upstream Emacs repo being tainted, just enable the checking on some
>build bots (and/or on your machine).

Yup.  Those are all reasonable arguments.

>I think that, given how it was implemented, this change is bad on
>itself, and establishes a bad precedent. See my reply to Paul for some
>ideas about how to improve the implementation. Requiring from the user
>an informed consent is the right thing here. After all, if you convince
>the user about the convenience of those settings, maybe he will apply
>them on his other repos, which is a much better outcome from your POV, I
>guess.

Understood.

I don't have time to find the appropriate git maintainers' forum myself, but if I did, I would post a link to this thread there and say "See, if you folks don't set this to the obvious default in upstream git, you're just going to cause this discussion to happen on the dev list of every git-using project out there.  Please do the needful!" :-)

Re: Recommend these .gitconfig settings for git integrity.

[Tom]

Karl Fogel <kfogel <at> red-bean.com> writes:

> Is there some reason why the Emacs developers as a group can't decide that
> integrity-checking should be turned on for git data transfers?  

Mandatory checks should happen on the server side, in checkin hooks,
so it's not up to the clients if they perform the checks or not.

No invalid data should be accepted by the server if the check can
be done on the server side.

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/02/2016 02:30 AM, Tom wrote:
> No invalid data should be accepted by the server if the check can
> be done on the server side.
That makes sense. However, as I understand it the checks that started 
this thread cannot be done just on the server side, because they're 
checking for things like transmission errors from the server to the client.

Re: Recommend these .gitconfig settings for git integrity.

[Tom]

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> 
> That makes sense. However, as I understand it the checks that started 
> this thread cannot be done just on the server side, because they're 
> checking for things like transmission errors from the server to the client.
> 

In order to check it the client compares it with something, so it knows
it received the right thing. I guess the server can also do such a
comparison, since it also knows the same baseline which the client uses 
for comparison.

Re: Recommend these .gitconfig settings for git integrity.

[Paul Eggert]

On 02/02/2016 09:24 AM, Tom wrote:
> In order to check it the client compares it with something, so it knows
> it received the right thing. I guess the server can also do such a
> comparison,

By then it's too late, if the client omitted the check earlier. The 
client can generate commits with valid checksums, commits that contain 
data from the unchecked and incorrect commits. It can then send the 
commits-with-valid-checksums to the server.

Re: Recommend these .gitconfig settings for git integrity.

[Karl Fogel]

I don't have very strong feelings on the autogen.sh git config change, but I think we had consensus about the documentation change anyway, and that is now committed:

  https://lists.gnu.org/archive/html/emacs-diffs/2016-02/msg00008.html

(Paul, sorry for getting the date wrong by one day, where my commit message refers to the earlier of your two autogen.sh commits; I believe the reference will be easy to follow for anyone who wants to, though, given all the context.)

I've updated the relevant wiki pages as well.