delete-trailing-whitespace and binary files

delete-trailing-whitespace and binary files

[Kim F. Storm]

I have delete-trailing-whitespace in before-save-hook, but that
was a real disaster editing etc/spook.lines ... as soon as
I saved the file, all the trailing NUL-characters were deleted.

What can be done about this?

1) If file is binary (how to check that?), don't delete whitespace
   when run from a hook (how to check that?).

Is there a "file-binary-p" function which tests if
a file contains any non-printable characters?

2) Don't consider NUL to be whitespace.

3) ?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: delete-trailing-whitespace and binary files

[Eli Zaretskii]

> From: storm@cua.dk (Kim F. Storm)
> Date: Sun, 21 May 2006 20:46:11 +0200
> 
> 1) If file is binary (how to check that?)

Look for null characters.  (That's what Grep and a few other programs
do.)

Re: delete-trailing-whitespace and binary files

[Richard Stallman]

    1) If file is binary (how to check that?), don't delete whitespace
       when run from a hook (how to check that?).

That is so complex that I think it would be undesirable.

    2) Don't consider NUL to be whitespace.

That seems like a good idea.  Is there EVER a reason for
delete-trailing-whitespace to delete NUL?

Re: delete-trailing-whitespace and binary files

[Kim F. Storm]

Richard Stallman <rms@gnu.org> writes:

>     1) If file is binary (how to check that?), don't delete whitespace
>        when run from a hook (how to check that?).
>
> That is so complex that I think it would be undesirable.
>
>     2) Don't consider NUL to be whitespace.
>
> That seems like a good idea.  Is there EVER a reason for
> delete-trailing-whitespace to delete NUL?

If the syntax table says that NUL is whitespace, I guess it should do it...


I looked a little further at this.

Opening file spook.lines selects text-mode.

text-mode is derived from Fundamental-mode, which has
a somewhat peculiar (IMO) interpretation of "whitespace":

C-@ .. SPC	  	which means: whitespace
DEL .. ÿ	  	which means: whitespace
   		  	which means: whitespace
      .. ​	  	which means: whitespace
<<default>>	  	which means: whitespace


So delete-trailing-whitespace just does what it's supposed to do.

Is there a better major-mode to choose for spook.lines (e.g. via a
file local variable)?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: delete-trailing-whitespace and binary files

[Stefan Monnier]

> text-mode is derived from Fundamental-mode, which has
> a somewhat peculiar (IMO) interpretation of "whitespace":

> C-@ .. SPC	  	which means: whitespace
> DEL .. ÿ	  	which means: whitespace
>    		  	which means: whitespace
>       .. ​	  	which means: whitespace
> <<default>>	  	which means: whitespace

This looks wrong.  Why should all the control chars be considered whitespace
in text-mode?  Rather than try to use another major mode I think we should
fix the above.


        Stefan

Re: delete-trailing-whitespace and binary files

[Kevin Rodgers]

Stefan Monnier wrote:
>> text-mode is derived from Fundamental-mode, which has
>> a somewhat peculiar (IMO) interpretation of "whitespace":
> 
>> C-@ .. SPC	  	which means: whitespace
>> DEL .. ÿ	  	which means: whitespace
>>    		  	which means: whitespace
>>       .. ​	  	which means: whitespace
>> <<default>>	  	which means: whitespace
> 
> This looks wrong.  Why should all the control chars be considered whitespace
> in text-mode?  Rather than try to use another major mode I think we should
> fix the above.

That's what I thought, but I don't see a more appropriate choice in the
Syntax Class Table node of the Emacs Lisp manual, which says this:

| "Whitespace characters" (designated by ` ' or `-') separate symbols
| and words from each other.

Thanks,
-- 
Kevin

Re: delete-trailing-whitespace and binary files

[Kim F. Storm]

Kevin Rodgers <ihs_4664@yahoo.com> writes:

> Stefan Monnier wrote:
>>> text-mode is derived from Fundamental-mode, which has
>>> a somewhat peculiar (IMO) interpretation of "whitespace":
>>
>>> C-@ .. SPC	  	which means: whitespace
>>> DEL .. ÿ	  	which means: whitespace
>>>    		  	which means: whitespace
>>>       .. ​	  	which means: whitespace
>>> <<default>>	  	which means: whitespace
>>
>> This looks wrong.  Why should all the control chars be considered whitespace
>> in text-mode?  Rather than try to use another major mode I think we should
>> fix the above.
>
> That's what I thought, but I don't see a more appropriate choice in the
> Syntax Class Table node of the Emacs Lisp manual, which says this:
>
> | "Whitespace characters" (designated by ` ' or `-') separate symbols
> | and words from each other.

Maybe the solution for delete-trailing-whitespace is to explicitly
only consider "normal" whitespace characters, i.e. don't use the
syntax table ?

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: delete-trailing-whitespace and binary files

[Richard Stallman]

    >> This looks wrong.  Why should all the control chars be considered whitespace
    >> in text-mode?  Rather than try to use another major mode I think we should
    >> fix the above.
    >
    > That's what I thought, but I don't see a more appropriate choice in the
    > Syntax Class Table node of the Emacs Lisp manual, which says this:

We could give them Word syntax.

Since they don't normally appear in proper English text, it is hard to
argue that any choice of syntax is fundamentally wrong.  It is just a
matter of what seems marginally better.  This problem is an argument
that Word is marginally better than Whitespace.

Re: delete-trailing-whitespace and binary files

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     >> This looks wrong.  Why should all the control chars be
>     >> considered whitespace in text-mode?  Rather than try to use
>     >> another major mode I think we should fix the above.
>     >
>     > That's what I thought, but I don't see a more appropriate
>     > choice in the Syntax Class Table node of the Emacs Lisp
>     > manual, which says this:
>
> We could give them Word syntax.
>
> Since they don't normally appear in proper English text, it is hard to
> argue that any choice of syntax is fundamentally wrong.  It is just a
> matter of what seems marginally better.  This problem is an argument
> that Word is marginally better than Whitespace.

I think "punctuation" would make more sense.  This designates stuff
that is neither a component of words nor of symbols, and not
whitespace.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: delete-trailing-whitespace and binary files

[Richard Stallman]

    I think "punctuation" would make more sense.  This designates stuff
    that is neither a component of words nor of symbols, and not
    whitespace.

Ok with me.  Would you like to do it?

Re: delete-trailing-whitespace and binary files

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     I think "punctuation" would make more sense.  This designates stuff
>     that is neither a component of words nor of symbols, and not
>     whitespace.
>
> Ok with me.  Would you like to do it?

I am not at all familiar with the code in question and would not be
comfortable trying to find and change it.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: delete-trailing-whitespace and binary files

[Richard Stallman]

Does this patch do the job?

*** syntax.c	01 May 2006 16:16:58 -0400	1.189
--- syntax.c	03 Jun 2006 18:43:27 -0400	
***************
*** 3122,3127 ****
--- 3122,3133 ----
  
    Vstandard_syntax_table = Fmake_char_table (Qsyntax_table, temp);
  
+   /* Control characters should not be whitespace.  */
+   temp = XVECTOR (Vsyntax_code_object)->contents[(int) Spunct];
+   for (i = 1; i <= ' ' - 1; i++)
+     SET_RAW_SYNTAX_ENTRY (Vstandard_syntax_table, i, temp);
+   SET_RAW_SYNTAX_ENTRY (Vstandard_syntax_table, 0177, temp);
+ 
    temp = XVECTOR (Vsyntax_code_object)->contents[(int) Sword];
    for (i = 'a'; i <= 'z'; i++)
      SET_RAW_SYNTAX_ENTRY (Vstandard_syntax_table, i, temp);

Re: delete-trailing-whitespace and binary files

[Stuart D. Herring]

> +   /* Control characters should not be whitespace.  */
> +   temp = XVECTOR (Vsyntax_code_object)->contents[(int) Spunct];
> +   for (i = 1; i <= ' ' - 1; i++)
> +     SET_RAW_SYNTAX_ENTRY (Vstandard_syntax_table, i, temp);
> +   SET_RAW_SYNTAX_ENTRY (Vstandard_syntax_table, 0177, temp);

This seems wrong to me in two respects.  It doesn't do anything to NUL,
which was the original problem character; instead, it marks as punctuation
^I (TAB), ^J (LFD), ^L (FFD), and ^M (CR), which should probably be left
as whitespace.  One could also make a case that such characters as ^?
(VTB) should remain whitespace, although I've never seen anyone use that
one.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: delete-trailing-whitespace and binary files

[Stuart D. Herring]

I wrote:

> This seems wrong to me in two respects.  It doesn't do anything to NUL,
> which was the original problem character; instead, it marks as punctuation
> ^I (TAB), ^J (LFD), ^L (FFD), and ^M (CR), which should probably be left
> as whitespace.  One could also make a case that such characters as ^?
> (VTB) should remain whitespace, although I've never seen anyone use that
> one.

I didn't mean ^? (DEL); I just hadn't looked up which character was
vertical tab and forgot to fill it in before sending.  I should have
written ^I (HT: horizontal tab, or TAB), ^J (LF: line feed, or newline),
^L (FF: form feed), ^M (CR: carriage return), and ^K (VT: vertical tab);
one out of five ASCII abbreviations isn't bad, is it?

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: delete-trailing-whitespace and binary files

[Richard Stallman]

    I didn't mean ^? (DEL); I just hadn't looked up which character was
    vertical tab and forgot to fill it in before sending.  I should have
    written ^I (HT: horizontal tab, or TAB), ^J (LF: line feed, or newline),
    ^L (FF: form feed), ^M (CR: carriage return), and ^K (VT: vertical tab);
    one out of five ASCII abbreviations isn't bad, is it?

Yes, you're right (though I don't see a need to include vertical tab;
nobody really uses that).