jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

Platform: amd64
          debian 11.6 "bullseye"
          GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, cairo version 1.16.0) of 2023-02-23, modified by Debian
          unzip 6.0-26+deb11u1
          libbz2 1.0.8-4

I tried opening some jar files yesterday. I expected them to open in
unzip mode and show the "directory listing" of the jar file contents
(jar files are zip files).

But of these files:
  -rw-r--r--  1 sb sb  52055 Apr  1 20:41 modelstore.backend-1.0.0-SNAPSHOT.jar
  -rw-r--r--  1 sb sb 455179 Apr  1 20:41 modelstore.backend-1.0.0-SNAPSHOT-javadoc.jar
  -rw-r--r--  1 sb sb  28639 Apr  1 20:41 modelstore.backend-1.0.0-SNAPSHOT-sources.jar

The modelstore.backend-1.0.0-SNAPSHOT.jar and the
modelstore.backend-1.0.0-SNAPSHOT-sources.jar file opened showing just
the following contents:
M Filemode      Length  Date         Time      File
- ----------  --------  -----------  --------  --------
- ----------  --------  -----------  --------  --------
                     0                         0 files

The modelstore.backend-1.0.0-SNAPSHOT-javadoc.jar files just gives the
following error message, and doesn't open:
File mode specification error: (args-out-of-range modelstore.backend-1.0.0-SNAPSHOT-javadoc.jar 445526 445530)

Any idea of how to debug this?
Alternatively, does someone recognize this behaviour and have a workaround?

Does the zip support of emacs 27.1 come in the form of running unzip in
a sub-process? Or does it use a linked in library? (libbz2...?)

Thanks!


- Steinar

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Sun, 02 Apr 2023 08:15:27 +0200
> 
> Platform: amd64
>           debian 11.6 "bullseye"
>           GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, cairo version 1.16.0) of 2023-02-23, modified by Debian
>           unzip 6.0-26+deb11u1
>           libbz2 1.0.8-4
> 
> I tried opening some jar files yesterday. I expected them to open in
> unzip mode and show the "directory listing" of the jar file contents
> (jar files are zip files).
> 
> But of these files:
>   -rw-r--r--  1 sb sb  52055 Apr  1 20:41 modelstore.backend-1.0.0-SNAPSHOT.jar
>   -rw-r--r--  1 sb sb 455179 Apr  1 20:41 modelstore.backend-1.0.0-SNAPSHOT-javadoc.jar
>   -rw-r--r--  1 sb sb  28639 Apr  1 20:41 modelstore.backend-1.0.0-SNAPSHOT-sources.jar
> 
> The modelstore.backend-1.0.0-SNAPSHOT.jar and the
> modelstore.backend-1.0.0-SNAPSHOT-sources.jar file opened showing just
> the following contents:
> M Filemode      Length  Date         Time      File
> - ----------  --------  -----------  --------  --------
> - ----------  --------  -----------  --------  --------
>                      0                         0 files
> 
> The modelstore.backend-1.0.0-SNAPSHOT-javadoc.jar files just gives the
> following error message, and doesn't open:
> File mode specification error: (args-out-of-range modelstore.backend-1.0.0-SNAPSHOT-javadoc.jar 445526 445530)

I cannot reproduce this with a random .jar file I have here and with
Emacs 27.1.  So I guess this is something specific to that particular
jar file.

> Any idea of how to debug this?

Step with Edebug through arc-mode.el and see why it signals the error?
It is possible that the jar file you tried uses some extensions of the
ZIP format which were not yet supported in Emacs 27, so perhaps try
with a later version of Emacs.

> Does the zip support of emacs 27.1 come in the form of running unzip in
> a sub-process? Or does it use a linked in library? (libbz2...?)

Neither.  Just visiting a zip file doesn't require any of that, at
least not by default (I have no idea whether Ubuntu made any changes
in the upstream sources or in site-wide customizations).

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Michael Albinus]

Steinar Bang <sb@dod.no> writes:

Hi Steinar,

> But of these files:
>   -rw-r--r--  1 sb sb  52055 Apr  1 20:41 modelstore.backend-1.0.0-SNAPSHOT.jar
>   -rw-r--r--  1 sb sb 455179 Apr  1 20:41 modelstore.backend-1.0.0-SNAPSHOT-javadoc.jar
>   -rw-r--r--  1 sb sb  28639 Apr  1 20:41 modelstore.backend-1.0.0-SNAPSHOT-sources.jar
>
> Alternatively, does someone recognize this behaviour and have a workaround?

You might try tramp-archive. Just open the file as directory with a
trailing slash, like "C-x C-f modelstore.backend-1.0.0-SNAPSHOT.jar/"

> Thanks!
>
> - Steinar

Best regards, Michael.

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

>>>>> Michael Albinus <michael.albinus@gmx.de>:

> You might try tramp-archive. Just open the file as directory with a
> trailing slash, like "C-x C-f modelstore.backend-1.0.0-SNAPSHOT.jar/"

Heh! That worked:

  /home/sb/workspaces/modelstore/modelstore.services/target/modelstore.services-1.0.0-SNAPSHOT-javadoc.jar/:
  total used in directory 150 available 377.3 GiB
  dr-x------  0 sb sb     0 1970-01-01  .
  dr-x------  0 sb sb     0 1970-01-01  ..
  -r--------  0 sb sb  2677 04-02 14:41 allclasses.html
  -r--------  0 sb sb  8928 04-02 14:41 allclasses-index.html
  -r--------  0 sb sb  5270 04-02 14:41 allpackages-index.html
  -r--------  0 sb sb  4785 04-02 14:41 constant-values.html
  -r--------  0 sb sb  4714 04-02 14:41 deprecated-list.html
  -r--------  0 sb sb    42 04-02 14:41 element-list
  -r--------  0 sb sb 10089 04-02 14:41 help-doc.html
  -r--------  0 sb sb 51340 04-02 14:41 index-all.html
  -r--------  0 sb sb   955 04-02 14:41 index.html
  dr-x------  0 sb sb     0 04-02 14:41 jquery
  -r--------  0 sb sb  1498 04-02 14:41 jquery-ui.overrides.css
  -r--------  0 sb sb 11040 04-02 14:41 member-search-index.js
  -r--------  0 sb sb  1204 04-02 14:41 member-search-index.zip
  dr-x------  0 sb sb     0 04-02 14:41 META-INF
  dr-x------  0 sb sb     0 04-02 14:41 no
  -r--------  0 sb sb  7924 04-02 14:41 overview-tree.html
  -r--------  0 sb sb   124 04-02 14:41 package-search-index.js
  -r--------  0 sb sb   250 04-02 14:41 package-search-index.zip
  dr-x------  0 sb sb     0 04-02 14:41 resources
  -r--------  0 sb sb  6040 04-02 14:41 script.js
  -r--------  0 sb sb 13309 04-02 14:41 search.js
  -r--------  0 sb sb 22360 04-02 14:41 stylesheet.css
  -r--------  0 sb sb   606 04-02 14:41 type-search-index.js
  -r--------  0 sb sb   316 04-02 14:41 type-search-index.zip

Thanks!

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

> I cannot reproduce this with a random .jar file I have here and with
> Emacs 27.1.  So I guess this is something specific to that particular
> jar file.

Indeed. I walked down into ~/.m2/repository/ and tried the first jar
there, which happened to be antlr-2.7.7.jar and that opened fine.

I've been doing this with jar files for years (decades, even,
maybe...?), on multiple platforms (various GNU/linux distros and
versions, mac OSX, various windows versions) and over many emacs
versions.

And I've never seen this behaviour before.

Hm... seems to be a problem on everything I've build myself...?
At least: everything I've recently built myself on this computer.

FWIW I'm using java11 and maven-jar-plugin 3.2.0 (if that rings a bell
for someone out there somewhere...?)

>> Any idea of how to debug this?

> Step with Edebug through arc-mode.el and see why it signals the error?
> It is possible that the jar file you tried uses some extensions of the
> ZIP format which were not yet supported in Emacs 27, so perhaps try
> with a later version of Emacs.

no luck with edebug so far.

I first did (setq debug-on-error t) in a fresh emacs and got the
following stack trace from attempting to open the javadoc file:
 https://gist.github.com/steinarb/234cf172c214b26b33ee92a4e5b5afd1
and found archive-zip-summarize which looks promising (and is in
arc-mode.el) 

Then I opened a new fresh emacs and openet /usr/share/emacs/27.1/lisp/arc-mode.el.gz
and but the cursor over archive-zip-summarize and did 'C-u C-M-x'

Then I tried opening the javadoc.jar but, unfortunately it didn't stop
in archive-zip-summarize

Not sure what more to do to make the code stop in the debugger (I have
skimmed the edebug docs to get this far).

>> Does the zip support of emacs 27.1 come in the form of running unzip in
>> a sub-process? Or does it use a linked in library? (libbz2...?)

> Neither.  Just visiting a zip file doesn't require any of that, at
> least not by default (I have no idea whether Ubuntu made any changes
> in the upstream sources or in site-wide customizations).

Thanks for the info.

FWIW From what you describe above, I doubt there are any modifications
in ubuntu/debian on the lines I was asking about (but I haven't actually
checked).

Thanks!


- Steinar

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Sun, 02 Apr 2023 17:03:20 +0200
> 
> > Step with Edebug through arc-mode.el and see why it signals the error?
> > It is possible that the jar file you tried uses some extensions of the
> > ZIP format which were not yet supported in Emacs 27, so perhaps try
> > with a later version of Emacs.
> 
> no luck with edebug so far.
> 
> I first did (setq debug-on-error t) in a fresh emacs and got the
> following stack trace from attempting to open the javadoc file:
>  https://gist.github.com/steinarb/234cf172c214b26b33ee92a4e5b5afd1
> and found archive-zip-summarize which looks promising (and is in
> arc-mode.el) 
> 
> Then I opened a new fresh emacs and openet /usr/share/emacs/27.1/lisp/arc-mode.el.gz
> and but the cursor over archive-zip-summarize and did 'C-u C-M-x'
> 
> Then I tried opening the javadoc.jar but, unfortunately it didn't stop
> in archive-zip-summarize
> 
> Not sure what more to do to make the code stop in the debugger (I have
> skimmed the edebug docs to get this far).

You need to "M-x load-file RET arc-mode.el.gz RET" before "C-u M-x" of
the function.  The Emacs will stop in the function.

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

> You need to "M-x load-file RET arc-mode.el.gz RET" before "C-u M-x" of
> the function.  The Emacs will stop in the function.

Ok, archive-zip-summarize looks for the magic sequence "PK\005\006" or
"PK\006\007" or "PK\006\006" or "PK\001\002".

But what this jar file starts with, is "PK^c^d", which I think is
"PK\003\004", which is non of the above.

I.e. archive-zip-summarize doesn't recognize the magic number of these
jar files.

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Sun, 02 Apr 2023 20:59:36 +0200
> 
> >>>>> Eli Zaretskii <eliz@gnu.org>:
> 
> > You need to "M-x load-file RET arc-mode.el.gz RET" before "C-u M-x" of
> > the function.  The Emacs will stop in the function.
> 
> Ok, archive-zip-summarize looks for the magic sequence "PK\005\006" or
> "PK\006\007" or "PK\006\006" or "PK\001\002".
> 
> But what this jar file starts with, is "PK^c^d", which I think is
> "PK\003\004", which is non of the above.
> 
> I.e. archive-zip-summarize doesn't recognize the magic number of these
> jar files.

Please report this as a bug and include such a jar file.

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Yuri Khan]

On Mon, 3 Apr 2023 at 02:00, Steinar Bang <sb@dod.no> wrote:

> Ok, archive-zip-summarize looks for the magic sequence "PK\005\006" or
> "PK\006\007" or "PK\006\006" or "PK\001\002".
>
> But what this jar file starts with, is "PK^c^d", which I think is
> "PK\003\004", which is non of the above.
>
> I.e. archive-zip-summarize doesn't recognize the magic number of these
> jar files.

ZIP archives are not defined by the magic number in the starting
bytes. It is valid to prepend anything to a ZIP archive. This is used
by self-extracting archives where the starting part is an executable.

The defining part of a ZIP archive is the End of central directory
record, magic number PK\x05\x06; it is optionally preceded by Zip64
extensions (PK\x06\x07 and PK\x06\x06), further preceded by Central
directory headers (PK\x01\x02). These are sufficient for displaying a
directory listing of the archive.

PK\x03\x04 is the signature for Local file header, which marks the
start of an actual compressed file’s data. So, indeed, a plain
unadorned non-empty ZIP archive will start with PK\x03\x04, but it is
not the magic number for ZIP archives and a conforming archive
processor should not scan specifically for that, at least until it is
asked to actually extract enclosed files.

See <https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT> for details.

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

>> From: Steinar Bang <sb@dod.no>
>> Date: Sun, 02 Apr 2023 20:59:36 +0200
>> 
>> >>>>> Eli Zaretskii <eliz@gnu.org>:
>> 
>> > You need to "M-x load-file RET arc-mode.el.gz RET" before "C-u M-x" of
>> > the function.  The Emacs will stop in the function.
>> 
>> Ok, archive-zip-summarize looks for the magic sequence "PK\005\006" or
>> "PK\006\007" or "PK\006\006" or "PK\001\002".
>> 
>> But what this jar file starts with, is "PK^c^d", which I think is
>> "PK\003\004", which is non of the above.
>> 
>> I.e. archive-zip-summarize doesn't recognize the magic number of these
>> jar files.

> Please report this as a bug and include such a jar file.

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62630

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

>>>>> Yuri Khan <yuri.v.khan@gmail.com>:

> ZIP archives are not defined by the magic number in the starting
> bytes. It is valid to prepend anything to a ZIP archive. This is used
> by self-extracting archives where the starting part is an executable.

> The defining part of a ZIP archive is the End of central directory
> record, magic number PK\x05\x06; it is optionally preceded by Zip64
> extensions (PK\x06\x07 and PK\x06\x06), further preceded by Central
> directory headers (PK\x01\x02). These are sufficient for displaying a
> directory listing of the archive.

> PK\x03\x04 is the signature for Local file header, which marks the
> start of an actual compressed file’s data. So, indeed, a plain
> unadorned non-empty ZIP archive will start with PK\x03\x04, but it is
> not the magic number for ZIP archives and a conforming archive
> processor should not scan specifically for that, at least until it is
> asked to actually extract enclosed files.

Not saying it should. 

Just saying it looks like the code doesn't find what it looks for in
that file and that's why it gets the Args out of range error.

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:
>>>>> Eli Zaretskii <eliz@gnu.org>:
>>> From: Steinar Bang <sb@dod.no>

>>> >>>>> Eli Zaretskii <eliz@gnu.org>:
>>> 
>>> > You need to "M-x load-file RET arc-mode.el.gz RET" before "C-u M-x" of
>>> > the function.  The Emacs will stop in the function.
>>> 
>>> Ok, archive-zip-summarize looks for the magic sequence "PK\005\006" or
>>> "PK\006\007" or "PK\006\006" or "PK\001\002".
>>> 
>>> But what this jar file starts with, is "PK^c^d", which I think is
>>> "PK\003\004", which is non of the above.
>>> 
>>> I.e. archive-zip-summarize doesn't recognize the magic number of these
>>> jar files.

>> Please report this as a bug and include such a jar file.

> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62630

This was not a bug in GNU Emacs.  This was a problem in my config.

I determined this by first running "emacs -Q" and when that opened the
offending .jar file without problems, I tried "emacs -q".

Since "emacs -q" also opened the .jar file without problems, I started
bisecting my ~/.emacs file.

The offending config was the following in my ~/.emacs:

;; editorconfig formatting support
;;;================================
(when (locate-library "editorconfig")
  (editorconfig-mode 1))

editorconfig is a minor mode that makes emacs respect formatting
settings found in .editorconfig files at the top of projects, making
emacs respect the same formatting settings as e.g. IntelliJ IDEA and
Eclipse.
 https://github.com/editorconfig/editorconfig-emacs

And all of the projects that contains .jar files that have failed to
open correctly have .editorconfig files on the top.

Removing the .editorconfig file on the top of the project with the
failing jar files removed the problem.

I bisected the .editorconfig file and found that the offending line was: https://github.com/steinarb/modelstore/blob/master/.editorconfig#L7

I.e:
charset = utf-8

I will look into the .editorconfig settings to see if there is a way to
exclude .jar files and/or the target directories from what editorconfig
affects, and if there isn't, open an issue on editorconfig-emacs.

Thanks!


- Steinar

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> I will look into the .editorconfig settings to see if there is a way
> to exclude .jar files and/or the target directories from what
> editorconfig affects, and if there isn't, open an issue on
> editorconfig-emacs.

I couldn't find any config for excluding files from editorconfig-emacs,
so I opened the following issue on editorconfig-emacs:
 https://github.com/editorconfig/editorconfig-emacs/issues/294

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

>> I will look into the .editorconfig settings to see if there is a way
>> to exclude .jar files and/or the target directories from what
>> editorconfig affects, and if there isn't, open an issue on
>> editorconfig-emacs.

> I couldn't find any config for excluding files from editorconfig-emacs,
> so I opened the following issue on editorconfig-emacs:
>  https://github.com/editorconfig/editorconfig-emacs/issues/294

The editorconfig-emacs mode does indeed have config for excluding modes and buffers:
 https://github.com/editorconfig/editorconfig-emacs/blob/master/editorconfig.el#L338

I tried excluding the Zip-Archive major mode, but that had no effect:
jar files in projects with .editorconfig files, still failed to open
correctly.
 https://github.com/editorconfig/editorconfig-emacs/issues/294#issuecomment-1493814857

But the editorconfig-exclude-regexps which matches buffer names, worked
for me:
 https://github.com/editorconfig/editorconfig-emacs/issues/294#issuecomment-1493824449

I replaced the customizations with hand-written config in my .emacs and
the following also works:

;; editorconfig formatting support
;;;================================
(when (locate-library "editorconfig")
  (editorconfig-mode 1)
  (setq editorconfig-exclude-regexps '("\\.jar$")))

Re: jar files won't open in emacs 27.1 on debian 11.6 "bullseye"

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

>> I will look into the .editorconfig settings to see if there is a way
>> to exclude .jar files and/or the target directories from what
>> editorconfig affects, and if there isn't, open an issue on
>> editorconfig-emacs.

> I couldn't find any config for excluding files from editorconfig-emacs,
> so I opened the following issue on editorconfig-emacs:
>  https://github.com/editorconfig/editorconfig-emacs/issues/294

Note: editorconfig issue 294 has been fixed, in the way outlined in
comments on the emacs bugs: editorconfig won't touch files where
mode-class is special:
 https://github.com/editorconfig/editorconfig-emacs/issues/294#event-9183783067

Here are the comments on the emacs bug:
 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62630#26
 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=62630#29