Git refuses to commit files with DOS EOL

Git refuses to commit files with DOS EOL

[Eli Zaretskii]

The pre-commit hook complains about trailing whitespace in such
files.  I needed to use --no-verify to get past it.

Why is it doing that?

Re: Git refuses to commit files with DOS EOL

[Paul Eggert]

Eli Zaretskii wrote:
> The pre-commit hook complains about trailing whitespace in such
> files.  I needed to use --no-verify to get past it.
>
> Why is it doing that?

Git keeps track of text files with CRLF vs LF endings, and autoconverts from one 
format to another when it is run on platforms that prefer one or the other.  It 
also knows about binary formats, where there's no conversion.  It needs to do 
all this stuff in order for merging to work reliably, since merging often works 
on a line-by-line basis.

To get this all to work, you sometimes have to tell Git about files that don't 
follow the ordinary rules: e.g., they are text files that should use CRLF 
endings even on GNUish and POSIX platforms.  This is done in the .gitattributes 
file.  There's a man page for .gitattributes, as well as one for the 'git 
check-attr' command.  The actual set of features is more complicated than what 
I've mentioned here, and the GNU Emacs .gitattributes file doesn't use all the 
features.

I patched .gitattributes to mention the CRLF-oriented files you recently added 
to the test cases.

Re: Git refuses to commit files with DOS EOL

[Eli Zaretskii]

> Date: Sat, 30 May 2015 09:35:49 -0700
> From: Paul Eggert <eggert@cs.ucla.edu>
> 
> Eli Zaretskii wrote:
> > The pre-commit hook complains about trailing whitespace in such
> > files.  I needed to use --no-verify to get past it.
> >
> > Why is it doing that?
> 
> Git keeps track of text files with CRLF vs LF endings, and autoconverts from one 
> format to another when it is run on platforms that prefer one or the other.  It 
> also knows about binary formats, where there's no conversion.  It needs to do 
> all this stuff in order for merging to work reliably, since merging often works 
> on a line-by-line basis.

I've set up Git to not perform any EOL conversions.

> To get this all to work, you sometimes have to tell Git about files that don't 
> follow the ordinary rules: e.g., they are text files that should use CRLF 
> endings even on GNUish and POSIX platforms.  This is done in the .gitattributes 
> file.  There's a man page for .gitattributes, as well as one for the 'git 
> check-attr' command.  The actual set of features is more complicated than what 
> I've mentioned here, and the GNU Emacs .gitattributes file doesn't use all the 
> features.
> 
> I patched .gitattributes to mention the CRLF-oriented files you recently added 
> to the test cases.

Thanks.

So you are saying that adding a file with DOS EOLs requires such
additions to .gitattributes?

Is that because of our pre-commit hook, or is this something general
with Git?

Re: Git refuses to commit files with DOS EOL

[Paul Eggert]

Eli Zaretskii wrote:
> So you are saying that adding a file with DOS EOLs requires such
> additions to .gitattributes?

Yes, basically.

> Is that because of our pre-commit hook, or is this something general
> with Git?

Both.  It's in our pre-commit hook, and our pre-commit hook uses a Git general 
facility to check this.

Re: Git refuses to commit files with DOS EOL

[Eli Zaretskii]

> Date: Sat, 30 May 2015 11:52:10 -0700
> From: Paul Eggert <eggert@cs.ucla.edu>
> Cc: emacs-devel@gnu.org
> 
> > Is that because of our pre-commit hook, or is this something general
> > with Git?
> 
> Both.  It's in our pre-commit hook, and our pre-commit hook uses a Git general 
> facility to check this.

I mean, if some other project using Git doesn't have a pre-commit
hook, do I still need to make these changes in .gitattributes for
files with DOS EOLs?

Re: Git refuses to commit files with DOS EOL

[Paul Eggert]

Eli Zaretskii wrote:
> if some other project using Git doesn't have a pre-commit
> hook, do I still need to make these changes in .gitattributes for
> files with DOS EOLs?

It depends on the maintenance style the other project wants to use, and which 
versions of Git they want to require (as this stuff has changed since older Git 
versions).  It's a long story.  For an intro, please see:

http://adaptivepatchwork.com/2012/03/01/mind-the-end-of-your-line/

Re: Git refuses to commit files with DOS EOL

[Eli Zaretskii]

> Date: Sat, 30 May 2015 12:40:21 -0700
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: emacs-devel@gnu.org
> 
>     if some other project using Git doesn't have a pre-commit
>     hook, do I still need to make these changes in .gitattributes for
>     files with DOS EOLs?
> 
> It depends on the maintenance style the other project wants to use, and which versions of Git they want to require (as this stuff has changed since older Git versions). It's a long story. For an intro, please see:
> 
> http://adaptivepatchwork.com/2012/03/01/mind-the-end-of-your-line/

Thanks, but that doesn't even mention 'whitespace'.  It talks only
about EOL conversion, which, as I've said, is disabled in my Git
installation (core.autcrlf is unset, which means it's false by
default).  I hope everybody else here does the same, although that
page (and numerous others I've seen) all advise to turn on EOL
conversion, which is simply wrong.

After reading all that stuff, including the gitattributes man page, I
have only one question: why files like dostorture.c are marked
"-whitespace" and not "whitespace=cr-at-eol"?  The latter seems to be
more specific, and still allows detection of trailing whitespace in
files with DOS-style EOL.

Re: Git refuses to commit files with DOS EOL

[Paul Eggert]

Eli Zaretskii wrote:
> that
> page (and numerous others I've seen) all advise to turn on EOL
> conversion, which is simply wrong.

Shrug.  It's right for many projects, probably most.  But that's not a topic we 
need to worry about here.

> After reading all that stuff, including the gitattributes man page, I
> have only one question: why files like dostorture.c are marked
> "-whitespace" and not "whitespace=cr-at-eol"?  The latter seems to be
> more specific, and still allows detection of trailing whitespace in
> files with DOS-style EOL

When I wrote that, I was concerned about portability to older versions of git, 
where the crlf conversion algorithms were different.  I didn't want to worry 
about the hassle of testing this stuff on older Git versions, so I used a 
conservative (i.e., loose) setting for the CRLF files.  If you can verify that a 
stricter setting based on cr-at-eol is useful on older Git versions on GNUish 
platforms, feel free to change to the stricter setting.

I suggest testing Git 1.7.1 for this, as per 
<https://lists.gnu.org/archive/html/emacs-devel/2014-12/msg00037.html>.

Re: Git refuses to commit files with DOS EOL

[Eli Zaretskii]

> Date: Sun, 31 May 2015 12:54:47 -0700
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: emacs-devel@gnu.org
> 
>     that
>     page (and numerous others I've seen) all advise to turn on EOL
>     conversion, which is simply wrong.
> 
> Shrug. It's right for many projects, probably most. But that's not a topic we need to worry about here.

If it is not right for Emacs (and I do hope we agree on that), I don't
see why it would be right for many other projects, let alone most.

But I agree it's not a topic to worry about here.

>     After reading all that stuff, including the gitattributes man page, I
>     have only one question: why files like dostorture.c are marked
>     "-whitespace" and not "whitespace=cr-at-eol"?  The latter seems to be
>     more specific, and still allows detection of trailing whitespace in
>     files with DOS-style EOL
> 
> When I wrote that, I was concerned about portability to older versions of git, where the crlf conversion algorithms were different. I didn't want to worry about the hassle of testing this stuff on older Git versions, so I used a conservative (i.e., loose) setting for the CRLF files. If you can verify that a stricter setting based on cr-at-eol is useful on older Git versions on GNUish platforms, feel free to change to the stricter setting.
> 
> I suggest testing Git 1.7.1 for this, as per <https://lists.gnu.org/archive/html/emacs-devel/2014-12/msg00037.html>.

I don't have access to a system with Git 1.7.1, and I cannot find this
setting mentioned in the Release Notes (is there a better resource for
looking up a Git release number that introduced some feature?).

However, according to Git's own git repository, this setting was added
in Jan 2008, in a commit about which "git describe" says:

  $ git describe b2979ff5
  v1.5.4-35-gb2979ff

So I think it is safe for us to use it.

I've pushed a change to that effect.

Thanks.