unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
From: Richard Hopkins <emacs@unbit.co.uk>
To: 58367@debbugs.gnu.org
Subject: bug#58367: `package-install-file' rejects some .tar files (tentative patch)
Date: Fri, 07 Oct 2022 22:15:56 +0100	[thread overview]
Message-ID: <263dae3ce307236c2694a2bc0baa3f63@unbit.co.uk> (raw)

[-- Attachment #1: Type: text/plain, Size: 3979 bytes --]

`package-install-file' signals an error for some .tar packages based
on their structure - both file layout and tar format.

This is specific to the package routines for .tar files as Emacs can
understand them fine, and can also install the packages if they have
already been extracted.

There's 2 different issues that cause this:

1) An old style .tar file (autoconf tar-v7, `tar cof ...`) vs a new
style file (autoconf tar-ustar, `tar cf ...`)

2) A package where all files are in a single directory vs having
child directories.  A package with child directories currently only
installs successfully if it's already extracted; the manual also says
they are allowed.

Combining these issues the current state of package installation using
.tar files is:

* ustar-nosub - error
* ustar-withsub - error
* v7-nosub - success
* v7-withsub - error

The main issue is how the package functions try to calculate the root
of the package directory and from there it can locate the "-pkg.el"
file etc.  From this we get two different errors:

1) Cannot handle a plain directory being first and missing a path
separator.  e.g. This happens for ustar files with or without a sub
directory.

$ tar tf ustar-nosub-0.1.tar
ustar-nosub-0.1
ustar-nosub-0.1/ustar-nosub.el
ustar-nosub-0.1/ustar-nosub-pkg.el

M-: (package-install-file "ustar-nosub-0.1.tar")
Debugger entered--Lisp error: (wrong-type-argument stringp nil)
   directory-file-name(nil)
   package--description-file(nil)
   package-tar-file-info()
   package-install-from-buffer()
   package-install-file("~/tmp/pkgtest/tars/ustar-nosub-0.1.tar")
   funcall-interactively(package-install-file 
"~/tmp/pkgtest/tars/ustar-nosub-0.1.tar")
   call-interactively(package-install-file record nil)
   command-execute(package-install-file record)
   execute-extended-command(nil "package-install-file" 
"package-install-file")
   funcall-interactively(execute-extended-command nil 
"package-install-file" "package-install-file")
   call-interactively(execute-extended-command nil nil)
   command-execute(execute-extended-command)

2) Looks for "-pkg.el" in a sub directory when it won't be there.
This happens for a v7 file but may also happen for a ustar if it
wasn't for error (1).

$ tar tf v7-withsub-0.1.tar
v7-withsub-0.1/v7-sub/foo.txt
v7-withsub-0.1/v7-withsub.el
v7-withsub-0.1/v7-withsub-pkg.el

M-: (package-install-file "tar tf v7-withsub-0.1.tar")
Debugger entered--Lisp error: (wrong-type-argument arrayp nil)
   tar--describe-as-link(nil)
   tar--check-descriptor(nil)
   tar-get-file-descriptor("v7-withsub-0.1/v7-sub/v7-sub-pkg.el")
   package-tar-file-info()
   package-install-from-buffer()
   package-install-file("~/tmp/pkgtest/tars/v7-withsub-0.1.tar")
   funcall-interactively(package-install-file 
"~/tmp/pkgtest/tars/v7-withsub-0.1.tar")
   call-interactively(package-install-file record nil)
   command-execute(package-install-file record)
   execute-extended-command(nil "package-install-file" 
"package-install-file")
   funcall-interactively(execute-extended-command nil 
"package-install-file" "package-install-file")
   call-interactively(execute-extended-command nil nil)
   command-execute(execute-extended-command)

For comparison the only .tar file that works is an old style v7 file
with all files in a single directory

$ tar tf v7-nosub-0.1.tar
v7-nosub-0.1/v7-nosub.el
v7-nosub-0.1/v7-nosub-pkg.el

M-: (package-install-file "v7-nosub-0.1.tar")
success

I've attached the example files to help you investigate.

There is also a proof of concept patch as it fixes ustar files with or
without sub directories, but it still fails on v7-withsub making the
new summary:

* ustar-nosub - success
* ustar-withsub - success
* v7-nosub - success
* v7-withsub - error

The modified functions were `package-untar-buffer' and
`package-tar-file-info'.

p.s. These results are from OpenBSD 7.1 with their `tar` and
Emacs 28.2 and master.

A related issue #13136 - 
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=13136

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-tar-package-install-file.patch --]
[-- Type: text/x-diff; name=0001-tar-package-install-file.patch, Size: 2436 bytes --]

Teach `package-install-file' to install previously rejected tarballs

Whilst Emacs can open the varieties of tarballs with no problem, the
package management functions depended on a particular table layout in
the tarball to calculate the directory for the package.

For example, with BSD tar: `tar cf ...` would generate a rejected
tarball, and `tar cof ...` would generate an accepted one.

* lisp/emacs-lisp/package.el (package-untar-buffer): Compare against
the header name from the tarball rather than the filesystem
expanded form.
(package-tar-file-info): Also allow the first entry to be a plain
directory name without trailing path separator.
---
 lisp/emacs-lisp/package.el | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index 4268f7d27a..cc371fbc50 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -926,11 +926,12 @@ package-untar-buffer
   (let ((regexp (concat "\\`" (regexp-quote (expand-file-name dir)) "/"))
         (case-fold-search (file-name-case-insensitive-p dir)))
     (dolist (tar-data tar-parse-info)
-      (let ((name (expand-file-name (tar-header-name tar-data))))
+      (let* ((header-name (tar-header-name tar-data))
+             (name (expand-file-name header-name)))
         (or (string-match regexp name)
             ;; Tarballs created by some utilities don't list
             ;; directories with a trailing slash (Bug#13136).
-            (and (string-equal dir name)
+            (and (string-equal dir header-name)
                  (eq (tar-header-link-type tar-data) 5))
             (error "Package does not untar cleanly into directory %s/" dir)))))
   (tar-untar-buffer))
@@ -1192,8 +1193,10 @@ package-tar-file-info
   "Find package information for a tar file.
 The return result is a `package-desc'."
   (cl-assert (derived-mode-p 'tar-mode))
-  (let* ((dir-name (file-name-directory
-                    (tar-header-name (car tar-parse-info))))
+  (let* ((header-name (tar-header-name (car tar-parse-info)))
+	 (dir-name (or (file-name-directory header-name)
+                       ;; Some tar utilities generate a plain directory name first
+                       (file-name-as-directory header-name)))
          (desc-file (package--description-file dir-name))
          (tar-desc (tar-get-file-descriptor (concat dir-name desc-file))))
     (unless tar-desc
2.38.0


[-- Attachment #3: tar-package-install-file.tar.gz --]
[-- Type: application/gzip, Size: 2342 bytes --]

             reply	other threads:[~2022-10-07 21:15 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-07 21:15 Richard Hopkins [this message]
2022-10-08 13:28 ` bug#58367: `package-install-file' rejects some .tar files (tentative patch) Lars Ingebrigtsen
2022-10-08 13:55   ` Richard Hopkins
2022-10-08 16:25 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2022-10-08 22:45   ` Richard Hopkins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/emacs/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=263dae3ce307236c2694a2bc0baa3f63@unbit.co.uk \
    --to=emacs@unbit.co.uk \
    --cc=58367@debbugs.gnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).