bug#8968: arc-mode 7z writing support

bug#8968: arc-mode 7z writing support

[Juri Linkov]

I'd like to install a patch that implements update/delete operations
for 7z archives in arc-mode.el:

=== modified file 'lisp/arc-mode.el'
--- lisp/arc-mode.el	2011-04-19 13:44:55 +0000
+++ lisp/arc-mode.el	2011-06-30 21:39:45 +0000
@@ -55,9 +55,9 @@
 ;;			--------------------------------------------
 ;; View listing		Intern	Intern	Intern	Intern	Y	Y
 ;; Extract member	Y	Y	Y	Y	Y	Y
-;; Save changed member	Y	Y	Y	Y	N	N
+;; Save changed member	Y	Y	Y	Y	N	Y
 ;; Add new member	N	N	N	N	N	N
-;; Delete member	Y	Y	Y	Y	N	N
+;; Delete member	Y	Y	Y	Y	N	Y
 ;; Rename member	Y	Y	N	N	N	N
 ;; Chmod		-	Y	Y	-	N	N
 ;; Chown		-	Y	-	-	N	N
@@ -328,6 +328,27 @@ (defcustom archive-7z-extract
 			(string :format "%v")))
   :group 'archive-7z)
 
+(defcustom archive-7z-expunge
+  '("7z" "d")
+  "Program and its options to run in order to delete 7z file members.
+Archive and member names will be added."
+  :type '(list (string :tag "Program")
+	       (repeat :tag "Options"
+		       :inline t
+		       (string :format "%v")))
+  :group 'archive-7z)
+
+(defcustom archive-7z-update
+  '("7z" "u")
+  "Program and its options to run in order to update a 7z file member.
+Options should ensure that specified directory will be put into the 7z
+file.  Archive and member name will be added."
+  :type '(list (string :tag "Program")
+	       (repeat :tag "Options"
+		       :inline t
+		       (string :format "%v")))
+  :group 'archive-7z)
+
 ;; -------------------------------------------------------------------------
 ;;; Section: Variables
 
@@ -2037,7 +2058,9 @@ (defun archive-7z-summarize ()
     (with-temp-buffer
       (call-process "7z" nil t nil "l" "-slt" file)
       (goto-char (point-min))
-      (re-search-forward "^-+\n")
+      ;; Four dashes start the meta info section that should be skipped.
+      ;; Archive members start with more than four dashes.
+      (re-search-forward "^-----+\n")
       (while (re-search-forward "^Path = \\(.*\\)\n" nil t)
         (goto-char (match-end 0))
         (let ((name (match-string 1))
@@ -2084,6 +2107,12 @@ (defun archive-7z-extract (archive name)
 	  (message "%s" (buffer-string)))
 	(delete-file tmpfile)))))
 
+(defun archive-7z-write-file-member (archive descr)
+  (archive-*-write-file-member
+   archive
+   descr
+   archive-7z-update))
+
 ;; -------------------------------------------------------------------------
 ;;; Section `ar' archives.
 

bug#8968: arc-mode 7z writing support

[Stefan Monnier]

> I'd like to install a patch that implements update/delete operations
> for 7z archives in arc-mode.el:

Go ahead,


        Stefan

bug#8968: arc-mode 7z writing support

[Juri Linkov]

>> I'd like to install a patch that implements update/delete operations
>> for 7z archives in arc-mode.el:
>
> Go ahead,

Done.

Some users prefer using 7z even for operations on zip archives.

This patch provides the right default values for them in case
7z is installed on the system instead of zip/unzip.

For searching the available programs it uses exactly the same logic
as already is implemented in `archive-zip-extract' (i.e. first try
to find zip/unzip, then 7z, and finally pkzip/pkunzip):

=== modified file 'lisp/arc-mode.el'
--- lisp/arc-mode.el	2011-04-19 13:44:55 +0000
+++ lisp/arc-mode.el	2011-07-04 21:44:48 +0000
@@ -235,10 +235,10 @@ (defcustom archive-zip-extract
 ;; names.
 
 (defcustom archive-zip-expunge
-  (if (and (not (executable-find "zip"))
-           (executable-find "pkzip"))
-      '("pkzip" "-d")
-    '("zip" "-d" "-q"))
+  (cond ((executable-find "zip")     '("zip" "-d" "-q"))
+	((executable-find "7z")      '("7z" "d"))
+	((executable-find "pkzip")   '("pkzip" "-d"))
+	(t                           '("zip" "-d" "-q")))
   "Program and its options to run in order to delete zip file members.
 Archive and member names will be added."
   :type '(list (string :tag "Program")
@@ -248,10 +248,10 @@ (defcustom archive-zip-expunge
   :group 'archive-zip)
 
 (defcustom archive-zip-update
-  (if (and (not (executable-find "zip"))
-           (executable-find "pkzip"))
-      '("pkzip" "-u" "-P")
-    '("zip" "-q"))
+  (cond ((executable-find "zip")     '("zip" "-q"))
+	((executable-find "7z")      '("7z" "u"))
+	((executable-find "pkzip")   '("pkzip" "-u" "-P"))
+	(t                           '("zip" "-q")))
   "Program and its options to run in order to update a zip file member.
 Options should ensure that specified directory will be put into the zip
 file.  Archive and member name will be added."
@@ -262,10 +262,10 @@ (defcustom archive-zip-update
   :group 'archive-zip)
 
 (defcustom archive-zip-update-case
-  (if (and (not (executable-find "zip"))
-           (executable-find "pkzip"))
-      '("pkzip" "-u" "-P")
-    '("zip" "-q" "-k"))
+  (cond ((executable-find "zip")     '("zip" "-q" "-k"))
+	((executable-find "7z")      '("7z" "u"))
+	((executable-find "pkzip")   '("pkzip" "-u" "-P"))
+	(t                           '("zip" "-q" "-k")))
   "Program and its options to run in order to update a case fiddled zip member.
 Options should ensure that specified directory will be put into the zip file.
 Archive and member name will be added."

bug#8968: arc-mode 7z writing support

[Juri Linkov]

> Some users prefer using 7z even for operations on zip archives.
>
> This patch provides the right default values for them in case
> 7z is installed on the system instead of zip/unzip.
>
> For searching the available programs it uses exactly the same logic
> as already is implemented in `archive-zip-extract' (i.e. first try
> to find zip/unzip, then 7z, and finally pkzip/pkunzip):

This is installed as well.

As reported in http://thread.gmane.org/gmane.emacs.devel/136578/focus=136670
on cygwin the program name is "p7zip".  But I can't confirm this fact,
so I'm closing this feature request at its current state with the
program name "7z".

bug#8968: arc-mode 7z writing support

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Date: Fri, 08 Jul 2011 03:09:51 +0300
> 
> As reported in http://thread.gmane.org/gmane.emacs.devel/136578/focus=136670
> on cygwin the program name is "p7zip".  But I can't confirm this fact,
> so I'm closing this feature request at its current state with the
> program name "7z".

In fact, I think any GNU/Linux system that cares about avoiding
non-free software will have only p7zip, not 7z.  That's because
current releases of 7z are not Free Software, whereas p7zip uses the
free-software subset of the 7z algorithms.

So by supporting only 7z, let alone making that the default, Emacs is
in fact preferring a non-free software program to a free one.  We have
the same situation with zip/unzip vs pkzip/pkunzip, and in that case
we do it the other way around: the default is the free software
variant, with the non-free one available as an option.

bug#8968: arc-mode 7z writing support

[Stefan Monnier]

> So by supporting only 7z, let alone making that the default, Emacs is
> in fact preferring a non-free software program to a free one.  We have

Oh, I didn't realize 7z was non-Free.  We need to fix that, then.


        Stefan

bug#8968: arc-mode 7z writing support

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Juri Linkov <juri@jurta.org>
>> Date: Fri, 08 Jul 2011 03:09:51 +0300
>> 
>> As reported in http://thread.gmane.org/gmane.emacs.devel/136578/focus=136670
>> on cygwin the program name is "p7zip".  But I can't confirm this fact,
>> so I'm closing this feature request at its current state with the
>> program name "7z".
>
> In fact, I think any GNU/Linux system that cares about avoiding
> non-free software will have only p7zip, not 7z.  That's because
> current releases of 7z are not Free Software, whereas p7zip uses the
> free-software subset of the 7z algorithms.

All GNU/Linux systems will have only p7zip, as that is the ported
version of 7-zip. 7z is only available for Windows, and comes with two
command line binaries - 7za.exe, which is Free Software, and 7z.exe
which is not (due to unRAR support).

bug#8968: arc-mode 7z writing support

[Juri Linkov]

>> So by supporting only 7z, let alone making that the default, Emacs is
>> in fact preferring a non-free software program to a free one.  We have
>
> Oh, I didn't realize 7z was non-Free.  We need to fix that, then.

AFAIU, 7z as a program is Free Software on GNU/Linux systems.
However, some packages can process formats of non-Free archives.

I see the following Debian packages:

* p7zip-rar.deb - non-free rar module for p7zip

  p7zip-rar provides a module for p7zip-full to make 7z able to
  extract RAR files.

* p7zip-full.deb provides:

 - 7za - a standalone version of the 7-zip tool that handles 7z archives
   and other free compression formats.

 - 7z - handles 7z and other free compression formats.

* p7zip.deb provides:

 - 7zr - a standalone minimal version that only handles 7z archives.

 - p7zip - a gzip like wrapper around 7zr.

There is some confusion: the same name `p7zip' is used for the package
and for the program, and the same name `7z' is used for the package,
for the program and for the format name.

The program `p7zip' (a gzip like wrapper) is not suitable for using in
arc-mode.el on GNU/Linux.  That's why I omitted it from searching for
available programs.  On GNU/Linux it will find the wrong program.
But I don't know the situation on cygwin.

The package `p7zip-full' uses the Free Software subset of the algorithms
supported by 7z.  It contains Free Software that has program names `7z',
`7za'.  So it seems correct to use `7z' for search a program to handle
zip archives.

The difference between programs `7z' and `7za' is not essential:
one of them is just a standalone version of another.

p7zip-rar.deb is a separate non-free module for rar archives.

bug#8968: arc-mode 7z writing support

[Juri Linkov]

> All GNU/Linux systems will have only p7zip, as that is the ported
> version of 7-zip. 7z is only available for Windows, and comes with two
> command line binaries - 7za.exe, which is Free Software, and 7z.exe
> which is not (due to unRAR support).

Does this mean that we should use the binary "7z" on GNU/Linux systems
and "7za" on Windows?

bug#8968: arc-mode 7z writing support

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  8968@debbugs.gnu.org
> Date: Fri, 08 Jul 2011 21:38:17 +0300
> 
> >> So by supporting only 7z, let alone making that the default, Emacs is
> >> in fact preferring a non-free software program to a free one.  We have
> >
> > Oh, I didn't realize 7z was non-Free.  We need to fix that, then.
> 
> AFAIU, 7z as a program is Free Software on GNU/Linux systems.

Not AFAIK.  At least fencepost.gnu.org doesn't have 7z, but does have
p7zip, which is a shell script.  That shell script invokes another
shell script, called 7zr, which in turn invokes a program
/usr/lib/p7zip/7zr.

>  - 7zr - a standalone minimal version that only handles 7z archives.
> 
>  - p7zip - a gzip like wrapper around 7zr.

That's what fencepost has.

> The program `p7zip' (a gzip like wrapper) is not suitable for using in
> arc-mode.el on GNU/Linux.

Why not?

Is 7zr suitable?  If so, we should use it in preference to 7z.

> The difference between programs `7z' and `7za' is not essential:
> one of them is just a standalone version of another.

That's not what Jason said.

bug#8968: arc-mode 7z writing support

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  8968@debbugs.gnu.org
> Date: Fri, 08 Jul 2011 21:39:27 +0300
> 
> > All GNU/Linux systems will have only p7zip, as that is the ported
> > version of 7-zip. 7z is only available for Windows, and comes with two
> > command line binaries - 7za.exe, which is Free Software, and 7z.exe
> > which is not (due to unRAR support).
> 
> Does this mean that we should use the binary "7z" on GNU/Linux systems
> and "7za" on Windows?

I actually think that we should not use 7z of any kind at all, but if
we must, then use 7zr on GNU/Linux and 7za on Windows, and offer 7z as
a second option.

bug#8968: arc-mode 7z writing support

[Juri Linkov]

>> AFAIU, 7z as a program is Free Software on GNU/Linux systems.
>
> Not AFAIK.  At least fencepost.gnu.org doesn't have 7z, but does have
> p7zip, which is a shell script.  That shell script invokes another
> shell script, called 7zr, which in turn invokes a program
> /usr/lib/p7zip/7zr.

I'd like to find the definitive answer.  On fencepost.gnu.org
7z is not installed, but for what reason?

I see that p7zip-full.deb contains 7z that handles free archives,
and there is a separate p7zip-rar.deb to handle non-free archives.

>> The program `p7zip' (a gzip like wrapper) is not suitable for using in
>> arc-mode.el on GNU/Linux.
>
> Why not?

`p7zip' is just a gzip like wrapper that just calls `7zr a -si' and
`7zr x -so' on stdin/stdout.  Such interface is not used in arc-mode.el.

> Is 7zr suitable?  If so, we should use it in preference to 7z.

There are two kinds of functionality where 7z can be used
in arc-mode.el:

1. Browse and update archives in the 7z format.

I agree that `7zr' should be preferable, and `7z' should be
used as a second option when `7zr' is not available.

2. Browse and update zip and other free archives.

Only `7z' can be used for this functionality, not `7zr'.

>> The difference between programs `7z' and `7za' is not essential:
>> one of them is just a standalone version of another.
>
> That's not what Jason said.

Jason said this about Windows.  What I see on GNU/Linux is different.

bug#8968: arc-mode 7z writing support

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Cc: monnier@iro.umontreal.ca,  8968@debbugs.gnu.org
> Date: Fri, 08 Jul 2011 23:20:43 +0300
> 
> >> AFAIU, 7z as a program is Free Software on GNU/Linux systems.
> >
> > Not AFAIK.  At least fencepost.gnu.org doesn't have 7z, but does have
> > p7zip, which is a shell script.  That shell script invokes another
> > shell script, called 7zr, which in turn invokes a program
> > /usr/lib/p7zip/7zr.
> 
> I'd like to find the definitive answer.  On fencepost.gnu.org
> 7z is not installed, but for what reason?

I don't have a definitive answer.  All I know is that arc-mode could
not handle certain archives that I could handle with a 7z.exe
downloaded for comparison.  I then researched this issue and found out
that those archives used a newer format that the Free Software
variants of 7z did not support.  In particular, the variant installed
on fencepost didn't support it.  And the reason was that the newer
formats used code that wasn't Free Software and whose sources were not
available even under non-Free licenses.

> 2. Browse and update zip and other free archives.
> 
> Only `7z' can be used for this functionality, not `7zr'.

Then perhaps we shouldn't offer 7z at all for this purpose.  If we
must, it should be a distant second or third alternative to `zip'.

bug#8968: arc-mode 7z writing support

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Juri Linkov <juri@jurta.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  8968@debbugs.gnu.org
>> Date: Fri, 08 Jul 2011 21:39:27 +0300
>> 
>> > All GNU/Linux systems will have only p7zip, as that is the ported
>> > version of 7-zip. 7z is only available for Windows, and comes with two
>> > command line binaries - 7za.exe, which is Free Software, and 7z.exe
>> > which is not (due to unRAR support).

Seems I jumped the gun here. p7zip may be the name of the port, but it
also provides 7z and 7za binaries.

>> Does this mean that we should use the binary "7z" on GNU/Linux systems
>> and "7za" on Windows?
>
> I actually think that we should not use 7z of any kind at all, but if
> we must, then use 7zr on GNU/Linux and 7za on Windows, and offer 7z as
> a second option.

I suggest to list in the order 7za, 7zr, 7z.  7za is also available as
part of the p7zip-full package on Debian derived distributions (at
least), so 7zr is only required for systems that only have the minimal
p7zip package installed.

bug#8968: arc-mode 7z writing support

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

>> The difference between programs `7z' and `7za' is not essential:
>> one of them is just a standalone version of another.
>
> That's not what Jason said.

The non standalone version can load additional external libraries. That
is where the non-Free part comes in.  Aside from that additional
capability, Juri is essentially correct.

bug#8968: arc-mode 7z writing support

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

> I don't have a definitive answer.  All I know is that arc-mode could
> not handle certain archives that I could handle with a 7z.exe
> downloaded for comparison.  I then researched this issue and found out
> that those archives used a newer format that the Free Software
> variants of 7z did not support.

Were these archives .rar archives?  If so, it is an older format, not a
newer one, and not the core format of 7z, just supported for
completeness.  If not, then this is an issue that I cannot find any
information about - perhaps you could point us to the information so we
can stop recommending 7-Zip to others.

> on fencepost didn't support it.  And the reason was that the newer
> formats used code that wasn't Free Software and whose sources were not
> available even under non-Free licenses.

Even the non-Free unRAR source is available, it just comes with
unacceptable restrictions on its use. So I'd be interested in what this
new format is.

bug#8968: arc-mode 7z writing support

[Eli Zaretskii]

> From: Jason Rumney <jasonr@gnu.org>
> Cc: Juri Linkov <juri@jurta.org>,  8968@debbugs.gnu.org
> Date: Sat, 09 Jul 2011 19:34:59 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I don't have a definitive answer.  All I know is that arc-mode could
> > not handle certain archives that I could handle with a 7z.exe
> > downloaded for comparison.  I then researched this issue and found out
> > that those archives used a newer format that the Free Software
> > variants of 7z did not support.
> 
> Were these archives .rar archives?

No, they were "*.7z".

bug#8968: arc-mode 7z writing support

[Juri Linkov]

> I suggest to list in the order 7za, 7zr, 7z.  7za is also available as
> part of the p7zip-full package on Debian derived distributions (at
> least), so 7zr is only required for systems that only have the minimal
> p7zip package installed.

This is my understanding as well.  It seems this is what
we should do in arc-mode.el now to handle 7z and zip
until someone proves that it can be used for non-free formats.

BTW, error handling of the non-free RAR format should be more graceful.
Currently, trying to browse an incompatible RAR archive with unrar-free
displays an uninformative error message:

  Removing old name: no such file or directory

This message is shown by `delete-file' in `archive-rar-extract'.
Maybe it should check the presence of the extracted file
before trying to delete and display more clear message,
e.g. "Your RAR archive format is non-free".

Also I don't understand why the command recently added to dired-x.el
by http://thread.gmane.org/gmane.emacs.devel/136578
is "unrar x", but in `archive-rar-extract' it's still
"unrar-free --extract"?