bug#1853: Trouble with gzipped info files on Windows

bug#1853: Trouble with gzipped info files on Windows

[Juanma Barranquero]

Package: emacs,w32
Version: 23.0.60

Gzipping normal CRLF info files on Windows, there's currently two problems:

  1.- For all info files:

    cd info
    gzip efaq
    emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
\"(efaq)\")"

    The info pages are decoded with a -unix coding system, so lines
contain spurious ^M characters.

    It does depend on setting the language environment (on the command
line or .emacs). For example, with my default "Spanish" environment,
it does not happen; but if I pass "Spanish" in the command above, the
info pages are erroneously decoded as info-latin-1-unix.

    The presence of NUL characters (see fixed bug#876) is irrelevant.
efaq does contain NUL, but the same bug happens with gnus, which does
not.

 2.- Additionally, for info nodes that do NOT contain a Top node:

    cd info
    emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
\"(ccmode)\")"
    ;; works OK.

    gzip ccmode*
    emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
\"(ccmode)\")"
    ;;  "No such node or anchor: Top"

bug#1853: marked as done (Trouble with gzipped info files on Windows)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]


Your message dated Sat, 17 Jan 2009 15:11:14 +0200
with message-id <utz7y2i31.fsf@gnu.org>
and subject line Re: bug#1853: Trouble with gzipped info files on Windows
has caused the Emacs bug report #1853,
regarding Trouble with gzipped info files on Windows
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@emacsbugs.donarmstrong.com
immediately.)


-- 
1853: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=1853
Emacs Bug Tracking System
Contact owner@emacsbugs.donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 3387 bytes --]

From: "Juanma Barranquero" <lekktu@gmail.com>
To: "Emacs Bug Tracker" <submit@emacsbugs.donarmstrong.com>
Subject: Trouble with gzipped info files on Windows
Date: Sat, 10 Jan 2009 22:49:05 +0100
Message-ID: <f7ccd24b0901101349s605a5909p2f42abb51ac957df@mail.gmail.com>

Package: emacs,w32
Version: 23.0.60

Gzipping normal CRLF info files on Windows, there's currently two problems:

  1.- For all info files:

    cd info
    gzip efaq
    emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
\"(efaq)\")"

    The info pages are decoded with a -unix coding system, so lines
contain spurious ^M characters.

    It does depend on setting the language environment (on the command
line or .emacs). For example, with my default "Spanish" environment,
it does not happen; but if I pass "Spanish" in the command above, the
info pages are erroneously decoded as info-latin-1-unix.

    The presence of NUL characters (see fixed bug#876) is irrelevant.
efaq does contain NUL, but the same bug happens with gnus, which does
not.

 2.- Additionally, for info nodes that do NOT contain a Top node:

    cd info
    emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
\"(ccmode)\")"
    ;; works OK.

    gzip ccmode*
    emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
\"(ccmode)\")"
    ;;  "No such node or anchor: Top"



[-- Attachment #3: Type: message/rfc822, Size: 3188 bytes --]

From: Eli Zaretskii <eliz@gnu.org>
To: Juanma Barranquero <lekktu@gmail.com>, 1853-done@emacsbugs.donarmstrong.com
Cc: emacs-devel@gnu.org
Subject: Re: bug#1853: Trouble with gzipped info files on Windows
Date: Sat, 17 Jan 2009 15:11:14 +0200
Message-ID: <utz7y2i31.fsf@gnu.org>

> Date: Sat, 10 Jan 2009 22:49:05 +0100
> From: "Juanma Barranquero" <lekktu@gmail.com>
> Cc: 
> 
> Gzipping normal CRLF info files on Windows, there's currently two problems:
> 
>   1.- For all info files:
> 
>     cd info
>     gzip efaq
>     emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info \"(efaq)\"))"
> 
>     The info pages are decoded with a -unix coding system, so lines
> contain spurious ^M characters.
> 
>     It does depend on setting the language environment (on the command
> line or .emacs). For example, with my default "Spanish" environment,
> it does not happen; but if I pass "Spanish" in the command above, the
> info pages are erroneously decoded as info-latin-1-unix.

This happened because, on DOS and Windows, file-coding-system-alist
includes the association `("" . find-buffer-file-type-coding-system)',
and find-buffer-file-type-coding-system was not ready to see an
argument whose `car' does not appear to exist because jka-compr
removed the .gz extension from its name.

I fixed find-buffer-file-type-coding-system to work correctly in this
case.

>  2.- Additionally, for info nodes that do NOT contain a Top node:
> 
>     cd info
>     emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
> \"(ccmode)\")"
>     ;; works OK.
> 
>     gzip ccmode*
>     emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
> \"(ccmode)\")"
>     ;;  "No such node or anchor: Top"

This is yet another separate bug: set-language-environment always sets
default-buffer-file-coding-system to *-unix.  See my other message a
few minutes ago.

bug#1853: Trouble with gzipped info files on Windows

[Eli Zaretskii]

> Date: Sat, 17 Jan 2009 15:11:14 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> This happened because, on DOS and Windows, file-coding-system-alist
> includes the association `("" . find-buffer-file-type-coding-system)',
> and find-buffer-file-type-coding-system was not ready to see an
> argument whose `car' does not appear to exist because jka-compr
> removed the .gz extension from its name.

Actually, this explains why _visiting_ compressed files might produce
a buffer with ^M characters in it.  The change I made cannot help with
Info files because the buffer name is "*info*" in that case, not
FOO.gz.

After the change, visiting, e.g., emacs-1.gz produces a buffer whose
buffer-file-coding-system is -dos, and there are no ^M characters in
it.  Visiting ccmode-1.gz still produces ^M's (because ccmode-1
includes null characters in the Index node).

> >  2.- Additionally, for info nodes that do NOT contain a Top node:
> > 
> >     cd info
> >     emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
> > \"(ccmode)\")"
> >     ;; works OK.
> > 
> >     gzip ccmode*
> >     emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
> > \"(ccmode)\")"
> >     ;;  "No such node or anchor: Top"
> 
> This is yet another separate bug: set-language-environment always sets
> default-buffer-file-coding-system to *-unix.  See my other message a
> few minutes ago.

Specifically, I propose the change below for this latter problem.
After that change, visiting compressed files, both regular and Info
files, work correctly for me, both before or after setting a
non-default language environment.

Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.353
diff -u -r1.353 mule-cmds.el
--- lisp/international/mule-cmds.el	9 Jan 2009 05:01:02 -0000	1.353
+++ lisp/international/mule-cmds.el	17 Jan 2009 14:01:33 -0000
@@ -1936,7 +1936,11 @@
 	 (eol-type (coding-system-eol-type default-buffer-file-coding-system)))
     (when priority
       (set-default-coding-systems
-       (if (memq eol-type '(0 1 2 unix dos mac))
+       ;; Don't use eol-type if default-buffer-file-coding-system is
+       ;; nil, because coding-system-eol-type treats nil as
+       ;; `no-conversion'.
+       (if (and default-buffer-file-coding-system
+		(memq eol-type '(0 1 2 unix dos mac)))
 	   (coding-system-change-eol-conversion default-coding eol-type)
 	 default-coding))
       (setq default-sendmail-coding-system default-coding)

bug#1853: Trouble with gzipped info files on Windows

[Eli Zaretskii]

> Date: Sat, 17 Jan 2009 16:05:36 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: lekktu@gmail.com, emacs-devel@gnu.org
> 
> --- lisp/international/mule-cmds.el	9 Jan 2009 05:01:02 -0000	1.353
> +++ lisp/international/mule-cmds.el	17 Jan 2009 14:01:33 -0000
> @@ -1936,7 +1936,11 @@
>  	 (eol-type (coding-system-eol-type default-buffer-file-coding-system)))
>      (when priority
>        (set-default-coding-systems
> -       (if (memq eol-type '(0 1 2 unix dos mac))
> +       ;; Don't use eol-type if default-buffer-file-coding-system is
> +       ;; nil, because coding-system-eol-type treats nil as
> +       ;; `no-conversion'.
> +       (if (and default-buffer-file-coding-system
> +		(memq eol-type '(0 1 2 unix dos mac)))
>  	   (coding-system-change-eol-conversion default-coding eol-type)
>  	 default-coding))
>        (setq default-sendmail-coding-system default-coding)

Upon further thought, perhaps we want the default coding-systems to
have an explicit -dos EOL type on DOS and Windows, for consistency
with how we set up things on startup.  This would involve explicitly
changing eol-type of default-coding before the last line above.

WDYT?

bug#1853: Trouble with gzipped info files on Windows

[Stefan Monnier]

> Upon further thought, perhaps we want the default coding-systems to
> have an explicit -dos EOL type on DOS and Windows, for consistency
> with how we set up things on startup.  This would involve explicitly
> changing eol-type of default-coding before the last line above.

Ideally, it should not only do the same as done on startup, but also do
it by running the same code.


        Stefan

bug#1853: Trouble with gzipped info files on Windows

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: 1853@emacsbugs.donarmstrong.com,  handa@m17n.org,  lekktu@gmail.com,  emacs-devel@gnu.org
> Date: Sun, 18 Jan 2009 16:01:40 -0500
> 
> > Upon further thought, perhaps we want the default coding-systems to
> > have an explicit -dos EOL type on DOS and Windows, for consistency
> > with how we set up things on startup.  This would involve explicitly
> > changing eol-type of default-coding before the last line above.
> 
> Ideally, it should not only do the same as done on startup, but also do
> it by running the same code.

Right, I had that in mind, too.  The problem is, it's not trivial.
The way it works on startup now is a bit kludgey: dos-w32.el says

  (setq-default buffer-file-coding-system 'undecided-dos)

and then set-language-environment-coding-systems does

 (let* ((priority (get-language-info language-name 'coding-priority))
        (default-coding (car priority))
        (eol-type (coding-system-eol-type default-buffer-file-coding-system)))
  ...
    (if (memq eol-type '(0 1 2 unix dos mac))
      (coding-system-change-eol-conversion default-coding eol-type)))

the kludge being that this only works because of the specific order
files are loaded at dump time and the order of function invocation
during startup.  Are we okay with making non-trivial changes in that
at this time?

bug#1853: Trouble with gzipped info files on Windows

[Stefan Monnier]

> the kludge being that this only works because of the specific order
> files are loaded at dump time and the order of function invocation
> during startup.  Are we okay with making non-trivial changes in that
> at this time?

I guess it's better to make a simpler local change, and add a big fat
comment about what should be done instead.


        Stefan

bug#1853: Trouble with gzipped info files on Windows

[Eli Zaretskii]

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: 1853@emacsbugs.donarmstrong.com, handa@m17n.org, lekktu@gmail.com,
>         emacs-devel@gnu.org
> Date: Tue, 20 Jan 2009 00:10:04 -0500
> 
> > the kludge being that this only works because of the specific order
> > files are loaded at dump time and the order of function invocation
> > during startup.  Are we okay with making non-trivial changes in that
> > at this time?
> 
> I guess it's better to make a simpler local change, and add a big fat
> comment about what should be done instead.

Done, with the following change:

Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.353
retrieving revision 1.354
diff -u -r1.353 -r1.354
--- lisp/international/mule-cmds.el	9 Jan 2009 05:01:02 -0000	1.353
+++ lisp/international/mule-cmds.el	24 Jan 2009 15:31:09 -0000	1.354
@@ -1933,7 +1933,25 @@
   "Do various coding system setups for language environment LANGUAGE-NAME."
   (let* ((priority (get-language-info language-name 'coding-priority))
 	 (default-coding (car priority))
-	 (eol-type (coding-system-eol-type default-buffer-file-coding-system)))
+	 ;; If default-buffer-file-coding-system is nil, don't use
+	 ;; coding-system-eol-type, because it treats nil as
+	 ;; `no-conversion'.  default-buffer-file-coding-system is set
+	 ;; to nil by reset-language-environment, and in that case we
+	 ;; want to have here the native EOL type for each platform.
+	 ;; FIXME: there should be a common code that runs both on
+	 ;; startup and here to set the default EOL type correctly.
+	 ;; Right now, DOS/Windows platforms set this on dos-w32.el,
+	 ;; which works only as long as the order of loading files at
+	 ;; dump time and calling functions at startup is not modified
+	 ;; significantly, i.e. as long as this function is called
+	 ;; _after_ default-buffer-file-coding-system was set by
+	 ;; dos-w32.el.
+	 (eol-type
+	  (if (null default-buffer-file-coding-system)
+	      (cond ((memq system-type '(windows-nt ms-dos)) 1)
+		    ((eq system-type 'macos) 2)
+		    (t 0))
+	    (coding-system-eol-type default-buffer-file-coding-system))))
     (when priority
       (set-default-coding-systems
        (if (memq eol-type '(0 1 2 unix dos mac))

bug#1853: marked as done (Trouble with gzipped info files on Windows)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]


Your message dated Sat, 24 Jan 2009 17:39:35 +0200
with message-id <uab9gu30o.fsf@gnu.org>
and subject line Re: bug#1853: Trouble with gzipped info files on Windows
has caused the Emacs bug report #1853,
regarding Trouble with gzipped info files on Windows
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@emacsbugs.donarmstrong.com
immediately.)


-- 
1853: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=1853
Emacs Bug Tracking System
Contact owner@emacsbugs.donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 3387 bytes --]

From: "Juanma Barranquero" <lekktu@gmail.com>
To: "Emacs Bug Tracker" <submit@emacsbugs.donarmstrong.com>
Subject: Trouble with gzipped info files on Windows
Date: Sat, 10 Jan 2009 22:49:05 +0100
Message-ID: <f7ccd24b0901101349s605a5909p2f42abb51ac957df@mail.gmail.com>

Package: emacs,w32
Version: 23.0.60

Gzipping normal CRLF info files on Windows, there's currently two problems:

  1.- For all info files:

    cd info
    gzip efaq
    emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
\"(efaq)\")"

    The info pages are decoded with a -unix coding system, so lines
contain spurious ^M characters.

    It does depend on setting the language environment (on the command
line or .emacs). For example, with my default "Spanish" environment,
it does not happen; but if I pass "Spanish" in the command above, the
info pages are erroneously decoded as info-latin-1-unix.

    The presence of NUL characters (see fixed bug#876) is irrelevant.
efaq does contain NUL, but the same bug happens with gnus, which does
not.

 2.- Additionally, for info nodes that do NOT contain a Top node:

    cd info
    emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
\"(ccmode)\")"
    ;; works OK.

    gzip ccmode*
    emacs -Q --eval "(progn (set-language-environment \"UTF-8\") (info
\"(ccmode)\")"
    ;;  "No such node or anchor: Top"



[-- Attachment #3: Type: message/rfc822, Size: 1453 bytes --]

From: Eli Zaretskii <eliz@gnu.org>
To: 1853-done@emacsbugs.donarmstrong.com
Subject: Re: bug#1853: Trouble with gzipped info files on Windows
Date: Sat, 24 Jan 2009 17:39:35 +0200
Message-ID: <uab9gu30o.fsf@gnu.org>

Fixed with the change in my latest message on this subject.