bug#20309: 25.0.50; Coding conversion error during clojure-mode package upgrade

bug#20309: 25.0.50; Coding conversion error during clojure-mode package upgrade

[Dmitry Gutov]

[-- Attachment #1: Type: text/plain, Size: 283 bytes --]

The package is being installed from MELPA, the file can be viewed at
http://melpa.org/packages/clojure-mode-20150410.2319.el.

Viewing it in the browser (or downloading and opening in `less')
displays those copyright characters just fine.

Not so here, see the attached screenshot.


[-- Attachment #2: Select coding system... --]
[-- Type: image/png, Size: 136453 bytes --]

[-- Attachment #3: Type: text/plain, Size: 265 bytes --]


In GNU Emacs 25.0.50.2 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2)
 of 2015-04-12 on axl
Repository revision: d0fcb21254394e22542dbc350220db3bafe0cc13
Windowing system distributor `The X.Org Foundation', version 11.0.11601901
System Description:	Ubuntu 14.10

bug#20309: 25.0.50; Coding conversion error during clojure-mode package upgrade

[Stefan Kangas]

Dmitry Gutov <dgutov@yandex.ru> writes:

> The package is being installed from MELPA, the file can be viewed at
> http://melpa.org/packages/clojure-mode-20150410.2319.el.

The latest version seems to be available here:
https://raw.githubusercontent.com/clojure-emacs/clojure-mode/master/clojure-mode.el

> Viewing it in the browser (or downloading and opening in `less')
> displays those copyright characters just fine.
>
> Not so here, see the attached screenshot.

I can't reproduce this on current master.  I believe that it has been
fixed by the following commit (if not before):

commit 5f9671e57ee99cfe4653b2cb6aca16d52f9a5c53
Author: Stefan Monnier <monnier@iro.umontreal.ca>
Date:   Sat May 18 18:32:47 2019 -0400

    * lisp/emacs-lisp/package.el: Fix decoding of downloaded files

    This is a different fix for bug#34909, which should also fix bug#35739.

    Our downloading code used to automatically decode the result according
    to the usual heuristics for files.  This caused problems when we later
    needed to save the data in a file that needed to be byte-for-byte
    equal to the original in order to pass the signature verification,
    especially because we didn't keep track of which coding-system was
    used to decode the data.

There is also a separate fix installed on the now released Emacs 26.3.

Could you please test this on Emacs 26.3 or the current master branch
and see if it works for you?

If I don't hear back from you within a couple of weeks, I'll just close
this as unreproducible.

Best regards,
Stefan Kangas

bug#20309: 25.0.50; Coding conversion error during clojure-mode package upgrade

[Dmitry Gutov]

On 22.10.2019 3:57, Stefan Kangas wrote:
> I can't reproduce this on current master.  I believe that it has been
> fixed by the following commit (if not before)

Thank you for the reminder.

I haven't seen this lately, so: closing.

bug#20309: bug#21319: `package-install-file' fails with CRLF (Windows) line endings

[Stefan Kangas]

steckerhalter <emacs@postmomentum.ch> writes:

> package.el contains this code:
>
> (defun package-install-file (file)
>   "Install a package from a file.
> The file can either be a tar file or an Emacs Lisp file."
>   (interactive "fPackage file name: ")
>   (with-temp-buffer
>     (insert-file-contents-literally file)
>     (when (string-match "\\.tar\\'" file) (tar-mode))
>     (package-install-from-buffer)))
>
> Since `insert-file-contents-literally' does not do any conversion
> `package-buffer-info' fails to parse the info if the file is .el and
> contains CRLF line endings. It then errors out with:
>
> Debugger entered--Lisp error: (error "Package lacks a \"Version\" or
> \"Package-Version\" header")
>   signal(error ("Package lacks a \"Version\" or \"Package-Version\" header"))
>   error("Package lacks a \"Version\" or \"Package-Version\" header")
>   package-buffer-info()
>
> This affects all recent Emacs versions (including trunk).

This was reported 4 years ago, but unfortunately never got a reply at
the time.

Are you still seeing this on a recent version of Emacs?  If yes, could
you please provide a minimal example file which fails to install?

Also, why does your file have these line endings in the first place?
Was this file written in Emacs?  I'm trying to understand if this is
an important use case or the common scenario for users of MS-Windows,
since I don't use it myself.

Best regards,
Stefan Kangas

bug#21319: `package-install-file' fails with CRLF (Windows) line endings

[Stefan Kangas]

Stefan Kangas <stefan@marxist.se> writes:

> steckerhalter <emacs@postmomentum.ch> writes:
>
> > package.el contains this code:
> >
> > (defun package-install-file (file)
> >   "Install a package from a file.
> > The file can either be a tar file or an Emacs Lisp file."
> >   (interactive "fPackage file name: ")
> >   (with-temp-buffer
> >     (insert-file-contents-literally file)
> >     (when (string-match "\\.tar\\'" file) (tar-mode))
> >     (package-install-from-buffer)))
> >
> > Since `insert-file-contents-literally' does not do any conversion
> > `package-buffer-info' fails to parse the info if the file is .el and
> > contains CRLF line endings. It then errors out with:
> >
> > Debugger entered--Lisp error: (error "Package lacks a \"Version\" or
> > \"Package-Version\" header")
> >   signal(error ("Package lacks a \"Version\" or \"Package-Version\" header"))
> >   error("Package lacks a \"Version\" or \"Package-Version\" header")
> >   package-buffer-info()
> >
> > This affects all recent Emacs versions (including trunk).
>
> This was reported 4 years ago, but unfortunately never got a reply at
> the time.
>
> Are you still seeing this on a recent version of Emacs?  If yes, could
> you please provide a minimal example file which fails to install?
>
> Also, why does your file have these line endings in the first place?
> Was this file written in Emacs?  I'm trying to understand if this is
> an important use case or the common scenario for users of MS-Windows,
> since I don't use it myself.

This was merged with Bug#20309, which was closed.  I suspect that the
bugs are most probably different, so I've unmerged this bug and
reopened it.  Please disregard any message saying that this bug is
closed for now.

Best regards,
Stefan Kangas

bug#21319: `package-install-file' fails with CRLF (Windows) line endings

[Stefan Kangas]

Stefan Kangas <stefan@marxist.se> writes:

> steckerhalter <emacs@postmomentum.ch> writes:
>
>> package.el contains this code:
>>
>> (defun package-install-file (file)
>>   "Install a package from a file.
>> The file can either be a tar file or an Emacs Lisp file."
>>   (interactive "fPackage file name: ")
>>   (with-temp-buffer
>>     (insert-file-contents-literally file)
>>     (when (string-match "\\.tar\\'" file) (tar-mode))
>>     (package-install-from-buffer)))
>>
>> Since `insert-file-contents-literally' does not do any conversion
>> `package-buffer-info' fails to parse the info if the file is .el and
>> contains CRLF line endings. It then errors out with:
>>
>> Debugger entered--Lisp error: (error "Package lacks a \"Version\" or
>> \"Package-Version\" header")
>>   signal(error ("Package lacks a \"Version\" or \"Package-Version\" header"))
>>   error("Package lacks a \"Version\" or \"Package-Version\" header")
>>   package-buffer-info()
>>
>> This affects all recent Emacs versions (including trunk).
>
> This was reported 4 years ago, but unfortunately never got a reply at
> the time.
>
> Are you still seeing this on a recent version of Emacs?  If yes, could
> you please provide a minimal example file which fails to install?
>
> Also, why does your file have these line endings in the first place?
> Was this file written in Emacs?  I'm trying to understand if this is
> an important use case or the common scenario for users of MS-Windows,
> since I don't use it myself.

We have gotten no more information within five weeks here.

Does anyone think it's important to support CRLF line endings in
package.el?  I don't use Windows, so I'm not sure that I understand if
this is an important use case or not.  But AFAIK, and this might be
completely wrong, you should be fine as long as you edit the file in
Emacs?

I'll give this a couple of weeks more, and if I don't hear anything
else within that time, I'll just go ahead and assume that what I'm
saying is correct and therefore close this bug report.

Please speak up if any of the above is incorrect and this bug should
remain open.

Best regards,
Stefan Kangas

bug#21319: `package-install-file' fails with CRLF (Windows) line endings

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Fri, 29 Nov 2019 13:58:15 +0100
> Cc: 21319@debbugs.gnu.org
> 
> >> (defun package-install-file (file)
> >>   "Install a package from a file.
> >> The file can either be a tar file or an Emacs Lisp file."
> >>   (interactive "fPackage file name: ")
> >>   (with-temp-buffer
> >>     (insert-file-contents-literally file)
> >>     (when (string-match "\\.tar\\'" file) (tar-mode))
> >>     (package-install-from-buffer)))
> >>
> >> Since `insert-file-contents-literally' does not do any conversion
> >> `package-buffer-info' fails to parse the info if the file is .el and
> >> contains CRLF line endings. It then errors out with:
> >>
> >> Debugger entered--Lisp error: (error "Package lacks a \"Version\" or
> >> \"Package-Version\" header")
> >>   signal(error ("Package lacks a \"Version\" or \"Package-Version\" header"))
> >>   error("Package lacks a \"Version\" or \"Package-Version\" header")
> >>   package-buffer-info()
> >>
> >> This affects all recent Emacs versions (including trunk).
> >
> > This was reported 4 years ago, but unfortunately never got a reply at
> > the time.
> >
> > Are you still seeing this on a recent version of Emacs?  If yes, could
> > you please provide a minimal example file which fails to install?

The code is still there.  One can see if the problem happens by making
a package that is a single .el file with CRLF EOL format.

> > Also, why does your file have these line endings in the first place?
> > Was this file written in Emacs?  I'm trying to understand if this is
> > an important use case or the common scenario for users of MS-Windows,
> > since I don't use it myself.

When a file is created on MS-Windows, even if it's a .el file, its EOL
format is by default identical to that of the underlying platform's
defaults.  On MS-Windows, you will get CRLF, yes.

> Does anyone think it's important to support CRLF line endings in
> package.el?

I don't know if this is important, but then fix is trivial: use
insert-file-contents after binding coding-system-for-read to raw-text.

> I don't use Windows, so I'm not sure that I understand if
> this is an important use case or not.  But AFAIK, and this might be
> completely wrong, you should be fine as long as you edit the file in
> Emacs?

AFAIU, the problem is that one cannot install such a package, because
the current code signals an error.

bug#21319: `package-install-file' fails with CRLF (Windows) line endings

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> Does anyone think it's important to support CRLF line endings in
>> package.el?
>
> I don't know if this is important, but then fix is trivial: use
> insert-file-contents after binding coding-system-for-read to raw-text.

I think I remember this being fixed recently?  Anybody remember?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#21319: `package-install-file' fails with CRLF (Windows) line endings

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Stefan Kangas <stefan@marxist.se>,  emacs@postmomentum.ch,
>   21319@debbugs.gnu.org
> Date: Sat, 29 Jan 2022 18:03:18 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Does anyone think it's important to support CRLF line endings in
> >> package.el?
> >
> > I don't know if this is important, but then fix is trivial: use
> > insert-file-contents after binding coding-system-for-read to raw-text.
> 
> I think I remember this being fixed recently?  Anybody remember?

Yes, it should be fixed.

bug#21319: `package-install-file' fails with CRLF (Windows) line endings

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> I think I remember this being fixed recently?  Anybody remember?
>
> Yes, it should be fixed.

Thanks; closing this bug report, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#21319: `package-install-file' fails with CRLF (Windows) line endings

[steckerhalter]

package.el contains this code:

(defun package-install-file (file)
  "Install a package from a file.
The file can either be a tar file or an Emacs Lisp file."
  (interactive "fPackage file name: ")
  (with-temp-buffer
    (insert-file-contents-literally file)
    (when (string-match "\\.tar\\'" file) (tar-mode))
    (package-install-from-buffer)))

Since `insert-file-contents-literally' does not do any conversion
`package-buffer-info' fails to parse the info if the file is .el and
contains CRLF line endings. It then errors out with:

Debugger entered--Lisp error: (error "Package lacks a \"Version\" or
\"Package-Version\" header")
  signal(error ("Package lacks a \"Version\" or \"Package-Version\" header"))
  error("Package lacks a \"Version\" or \"Package-Version\" header")
  package-buffer-info()

This affects all recent Emacs versions (including trunk).

What we did now is to replace `insert-file-contents-literally' with
`insert-file-contents' dynamically for non-tar files and that seems to
work, although I'm not sure if that's a good solution since I don't
know if there might be side-effects somewhere that can cause other
problems.

The other solution I can think of might be to adapt the regexp in
`package-buffer-info` so that in can deal with CRLF.

cheers,
steckerhalter