bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

[Eli Zaretskii]

This bug report will be sent to the Bug-GNU-Emacs mailing list
and the GNU bug tracker at debbugs.gnu.org.  Please check that
the From: line contains a valid email address.  After a delay of up
to one day, you should receive an acknowledgement at that address.

Please write in English if possible, as the Emacs maintainers
usually do not have translators for other languages.

Please describe exactly what actions triggered the bug, and
the precise symptoms of the bug.  If you can, give a recipe
starting from `emacs -Q':

 emacs -Q
 C-x C-f guile-2.0.3.tar.gz RET

Instead of the tarball contents, you will be presented with the
literal binary contents of the archive, and the mode line indicates
that Emacs thinks it's a shell script ("Shell-script[sh]").

The same happens if you gunzip the compressed tarball and visit the
resulting tar file.

I tried with a couple of other tarballs, and they work as expected.
So there's something in this specific tarball that triggers the bug.

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
d:/gnu/bzr/emacs/trunk/etc/DEBUG.


In GNU Emacs 24.0.92.1 (i386-mingw-nt5.1.2600)
 of 2012-01-14 on HOME-C4E4A596F7
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4) --no-opt'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  value of $XMODIFIERS: nil
  locale-coding-system: cp1255
  default enable-multibyte-characters: t

Major mode: Shell-script

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<help-echo> C-x C-f <M-backspace> <M-backspace> <M-backspace> 
<M-backspace> u s <tab> a r <tab> g u i l <tab> . g 
z <return> M-x r e p o r t - e m a c <tab> <return
>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
uncompressing guile-2.0.3.tar.gz...done
Setting up indent for shell type sh
setting up indent stuff
Indentation variables are now local.
Indentation setup for shell type sh

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr message format-spec rfc822 mml easymenu
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mailabbrev mail-utils gmm-utils mailheader
emacsbug regexp-opt sh-script executable jka-compr time-date tooltip
ediff-hook vc-hooks lisp-float-type mwheel dos-w32 disp-table ls-lisp
w32-win w32-vars tool-bar dnd fontset image fringe lisp-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer loaddefs
button faces cus-face files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process multi-tty emacs)

bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

[Glenn Morris]

Eli Zaretskii wrote:

> I tried with a couple of other tarballs, and they work as expected.
> So there's something in this specific tarball that triggers the bug.

It happens to contain a shell-script (depcomp) near the end of the
tarfile, and that shell-script happens to have a footer that starts
fewer than 3000 characters from the end of the tarfile, and happens to
contain:

    # Local Variables:
    # mode: shell-script
    # sh-indentation: 2
    # eval: (add-hook 'write-file-hooks 'time-stamp)
    # time-stamp-start: "scriptversion="
    # time-stamp-format: "%:y-%02m-%02d.%02H"
    # time-stamp-time-zone: "UTC"
    # time-stamp-end: "; # UTC"
    # End:

(The mode setting is superfluous since it begins with #!/bin/sh anyway.)

So hack-local-variables decides the tarfile should be in sh-mode.

bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: 10506@debbugs.gnu.org
> Date: Sat, 14 Jan 2012 17:16:21 -0500
> 
> It happens to contain a shell-script (depcomp) near the end of the
> tarfile, and that shell-script happens to have a footer that starts
> fewer than 3000 characters from the end of the tarfile, and happens to
> contain:
> 
>     # Local Variables:
>     # mode: shell-script
>     # sh-indentation: 2
>     # eval: (add-hook 'write-file-hooks 'time-stamp)
>     # time-stamp-start: "scriptversion="
>     # time-stamp-format: "%:y-%02m-%02d.%02H"
>     # time-stamp-time-zone: "UTC"
>     # time-stamp-end: "; # UTC"
>     # End:
> 
> (The mode setting is superfluous since it begins with #!/bin/sh anyway.)
> 
> So hack-local-variables decides the tarfile should be in sh-mode.

Thanks.  But this is a regression from Emacs 23.3, which doesn't have
this problem.  So I think we ought to fix it.

bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

[Glenn Morris]

Eli Zaretskii wrote:

> Thanks.  But this is a regression from Emacs 23.3, which doesn't have
> this problem.  So I think we ought to fix it.

Sure. We shouldn't be looking for local variable sections in tar files
and other such things.

bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

[Glenn Morris]

Eli Zaretskii wrote:

>> It happens to contain a shell-script (depcomp) near the end of the
>> tarfile, and that shell-script happens to have a footer that starts
>> fewer than 3000 characters from the end of the tarfile, and happens to
>> contain:
>> 
>>     # Local Variables:
>>     # mode: shell-script
[...]
> But this is a regression from Emacs 23.3, which doesn't have this
> problem.

The change is because set-auto-mode now checks for "mode:" at the end of
the file earlier than it did in 23.3 (specifically, before
auto-mode-alist). This is for correct handling of some dir-local
variables (bug#8586), but also seems like the right thing to do in
general. It is however wrong in this specific instance, and in other
instances of binary files.

It works in 23.3 because auto-mode-alist is consulted first, chooses
tar-mode, and this then binds local-enable-local-variables to nil.

I was going to say that I think it would still go wrong in 23.3 if you
could somehow construct a tar file with a -*- mode -*- entry on the
first line, but then I discovered inhibit-first-line-modes-regexps.

I think the right solution is to extend the meaning of
inhibit-first-line-modes-regexps, so that we ignore mode: entries
(indeed, all file local variables) in matching files wherever they
appear (not just the first line). IMO, this is obviously the sense in
which one would want to use such a variable - I can't think of a case
where one would want to ignore mode in the first line, but respect it at
the end, or ignore all mode: settings but still respect other file local
variables.

This variable is not documented in the manual, so I would not feel too
bad about extending it in this way; it's just a shame that it has the
name it does.


A final comment: it seems to me that anything in auto-coding-alist with
a no-conversion* coding should be in inhibit-first-line-modes-regexps as
well, since the meaning in both cases is basically "this is a binary
file". Are there any exceptions?


Thoughts?

bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

[Glenn Morris]

Glenn Morris wrote:

> A final comment: it seems to me that anything in auto-coding-alist with
> a no-conversion* coding should be in inhibit-first-line-modes-regexps as
> well, since the meaning in both cases is basically "this is a binary
> file". Are there any exceptions?

Perhaps the compression ones (gz etc) are exceptions, eg in case one
visits foo.txt.gz (though I haven't looked how jka-compr actually
handles such things).

bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

[Stefan Monnier]

> I was going to say that I think it would still go wrong in 23.3 if you
> could somehow construct a tar file with a -*- mode -*- entry on the
> first line, but then I discovered inhibit-first-line-modes-regexps.

Thanks for the footwork.
I agree that inhibit-first-line-modes-regexps should apply to all forms
of file-local variables.  I.e. please rename it to
inhibit-file-variables-regexps (with a define-obsolete-variable-alias,
of course).

> A final comment: it seems to me that anything in auto-coding-alist with
> a no-conversion* coding should be in inhibit-first-line-modes-regexps as
> well, since the meaning in both cases is basically "this is a binary
> file". Are there any exceptions?

There could be exceptions, I guess.  I think the way to reduce such
redundancy (without removing the possibility to handle things
differently) is to add indirections: e.g. allow not just regexps but
also major-mode symbols (so auto-mode-alist maps the file name to
a major mode and than this mode can be consulted in
inhibit-file-variables-regexps and auto-coding-alist, and we (w|c)ould
ideally take major-mode inheritance into account so we could make
tar-mode and image-mode inherit from a binary-mode parent and only have
binary-mode listed in those two lists).


        Stefan

bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

[Glenn Morris]

Stefan Monnier wrote:

> I agree that inhibit-first-line-modes-regexps should apply to all forms
> of file-local variables.  I.e. please rename it to
> inhibit-file-variables-regexps (with a define-obsolete-variable-alias,
> of course).

I believe this is implemented now.

This area seems quite messy. Some more comments:

1) inhibit-local-variables-regexps - should it be explicitly
case-insensitive?

2) local-enable-local-variables - a strange variable. AFAICS (see my
comments in files.el), its only real use is as a minor convenience
feature for visiting RMAIL BABYL files with M-x find-file. Should it be
declared obsolete? Should it affect directory-local variables (it
currently does, I think)?

> There could be exceptions, I guess.  I think the way to reduce such
> redundancy (without removing the possibility to handle things
> differently) is to add indirections: e.g. allow not just regexps but
> also major-mode symbols (so auto-mode-alist maps the file name to
> a major mode and than this mode can be consulted in
> inhibit-file-variables-regexps and auto-coding-alist, and we (w|c)ould
> ideally take major-mode inheritance into account so we could make
> tar-mode and image-mode inherit from a binary-mode parent and only have
> binary-mode listed in those two lists).

Still to be done after 24.1.

bug#10506: 24.0.92; Visiting the Guile v2.0.3 tarball shows binary garbage

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  10506@debbugs.gnu.org
> Date: Fri, 20 Jan 2012 19:49:03 -0500
> 
> Stefan Monnier wrote:
> 
> > I agree that inhibit-first-line-modes-regexps should apply to all forms
> > of file-local variables.  I.e. please rename it to
> > inhibit-file-variables-regexps (with a define-obsolete-variable-alias,
> > of course).
> 
> I believe this is implemented now.

Thanks, it works for me now.