bug#33880: [PATCH] Compress past NEWS files during installation

bug#33880: [PATCH] Compress past NEWS files during installation

[agrambot]

[-- Attachment #1: Type: text/plain, Size: 468 bytes --]

This shaves off just over a MB off of the installation using gzip -9.
While I was at it, I put in the compression of etc/refcards/*.ps, which
covers part of a TODO.

I wanted to compress etc/refcards/*.tex as well, but I noticed that it
would affect the Makefile that is also installed. It appears to be the
only Makefile that's installed; if there's little or no reason for it,
would it be okay to remove it during installation and compress the *.tex
files as well?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Compress-past-NEWS-files-and-etc-refcards-.ps.patch --]
[-- Type: text/x-patch, Size: 4420 bytes --]

From 40bd26ccbbae7c486dc2be39fe21147694889af4 Mon Sep 17 00:00:00 2001
From: Alexander Gramiak <agrambot@gmail.com>
Date: Mon, 24 Dec 2018 18:06:30 -0600
Subject: [PATCH] Compress past NEWS files and etc/refcards/*.ps

Do not compress NEWS or ORG-NEWS as those are actively updated and
shouldn't rely on decompression support to be viewed.

* Makefile.in (install-arch-indep): Do it.

* lisp/help.el (view-emacs-news): Refactor slightly, and support
compressed NEWS files when `auto-compression-mode' is enabled.
---
 Makefile.in  |  9 ++++++---
 etc/NEWS     |  3 +++
 lisp/help.el | 28 +++++++++++++++++-----------
 3 files changed, 26 insertions(+), 14 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index f0b2b66c88..bbb028a74a 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -541,8 +541,7 @@ set_installuser=
 ## install-etcdoc installs the relevant DOC.
 
 ## Note that we install etc/refcards/*.ps if present.
-## TODO we should compress these if GZIP_PROG is set.
-## It would be simpler to have a separate install rule for etc/refcards
+## TODO It would be simpler to have a separate install rule for etc/refcards
 ## (maybe move it to doc/refcards?).
 
 ## Note that the Makefiles in the etc directory are potentially useful
@@ -616,7 +615,11 @@ install-arch-indep:
 	  for f in `find . -name "*.elc" -print | sed 's/.elc$$/.el/'`; do \
 	    ${GZIP_PROG} -9n "$$f"; \
 	  done; \
-	  ${GZIP_PROG} -9n "../etc/publicsuffix.txt"; \
+	  cd "$(DESTDIR)${etcdir}" && \
+	  for f in `find . -maxdepth 1 -name "*NEWS*" -not -name NEWS \
+	-not -name ORG-NEWS` `find refcards -name "*.ps"` "publicsuffix.txt"; do \
+	    ${GZIP_PROG} -9n "$$f"; \
+	  done; \
 	}
 	-chmod -R a+r "$(DESTDIR)${datadir}/emacs/${version}" ${COPYDESTS}
 
diff --git a/etc/NEWS b/etc/NEWS
index e427e84780..586e7065d1 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -37,6 +37,9 @@ functions 'json-serialize', 'json-insert', 'json-parse-string', and
 'json-parse-buffer' are typically much faster than their Lisp
 counterparts from json.el.
 
+** NEWS files for past Emacs versions and packages are now compressed
+by default.
+
 ** The etags program now uses the C library's regular expression matcher
 when possible, and a compatible regex substitute otherwise.  This will
 let developers maintain Emacs's own regex code without having to also
diff --git a/lisp/help.el b/lisp/help.el
index ad782f74ca..e0738da485 100644
--- a/lisp/help.el
+++ b/lisp/help.el
@@ -350,16 +350,18 @@ view-emacs-news
 	      (mapc
 	       (lambda (file)
 		 (with-temp-buffer
-		   (insert-file-contents
-		    (expand-file-name file data-directory))
-		   (while (re-search-forward
-			   (if (member file '("NEWS.18" "NEWS.1-17"))
-			       "Changes in \\(?:Emacs\\|version\\)?[ \t]*\\([0-9]+\\(?:\\.[0-9]+\\)?\\)"
-			     "^\\* [^0-9\n]*\\([0-9]+\\.[0-9]+\\)") nil t)
-		     (setq res (cons (match-string-no-properties 1) res)))))
-	       (cons "NEWS"
-		     (directory-files data-directory nil
-				      "^NEWS\\.[0-9][-0-9]*$" nil)))
+		   (insert-file-contents file)
+                   (let ((section (if (string-match "\\`NEWS\\.\\(1-17\\|18\\)"
+                                                    (file-name-nondirectory file))
+                                      "Changes in \\(?:Emacs\\|version\\)?[ \t]*\\([0-9]+\\(?:\\.[0-9]+\\)?\\)"
+			            "^\\* [^0-9\n]*\\([0-9]+\\.[0-9]+\\)")))
+                     (while (re-search-forward section nil t)
+                       (setq res (cons (match-string-no-properties 1) res))))))
+	       (cons (expand-file-name "NEWS" data-directory)
+		     (directory-files data-directory
+                                      t
+				      (format "^NEWS\\.[0-9][-0-9]*%s\\'"
+                                              (regexp-opt load-file-rep-suffixes)))))
 	      (sort (delete-dups res) #'string>)))
 	   (current (car all-versions)))
       (setq version (completing-read
@@ -385,7 +387,11 @@ view-emacs-news
 		((< vn 18) "NEWS.1-17")
 		(t (format "NEWS.%d" vn))))
 	 res)
-    (view-file (expand-file-name file data-directory))
+    (view-file (or (locate-file file
+                                (list data-directory)
+                                load-file-rep-suffixes)
+                   (error "Requested NEWS file is compressed; please enable \
+`auto-compression-mode' and try again")))
     (widen)
     (goto-char (point-min))
     (when (stringp version)
-- 
2.20.1

bug#33880: [PATCH] Compress past NEWS files during installation

[Alex]

agrambot@gmail.com writes:

> This shaves off just over a MB off of the installation using gzip -9.
> While I was at it, I put in the compression of etc/refcards/*.ps, which
> covers part of a TODO.
>
> I wanted to compress etc/refcards/*.tex as well, but I noticed that it
> would affect the Makefile that is also installed. It appears to be the
> only Makefile that's installed; if there's little or no reason for it,
> would it be okay to remove it during installation and compress the *.tex
> files as well?

Ping.  Is this okay to apply to master?

bug#33880: [PATCH] Compress past NEWS files during installation

[Eli Zaretskii]

> From: Alex <agrambot@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>
> Date: Thu, 14 Mar 2019 08:57:13 -0600
> 
> agrambot@gmail.com writes:
> 
> > This shaves off just over a MB off of the installation using gzip -9.
> > While I was at it, I put in the compression of etc/refcards/*.ps, which
> > covers part of a TODO.
> >
> > I wanted to compress etc/refcards/*.tex as well, but I noticed that it
> > would affect the Makefile that is also installed. It appears to be the
> > only Makefile that's installed; if there's little or no reason for it,
> > would it be okay to remove it during installation and compress the *.tex
> > files as well?
> 
> Ping.  Is this okay to apply to master?

Sorry for not responding earlier.

I don't think compressing *.ps files is a good idea, because then they
cannot be printed without manually uncompressing them.

As for NEWS, I don't have a strong opinion, but it sounds like a very
small gain for the trouble.  What do others think?  Perhaps this
should be discussed on emacs-devel first.

Thanks.

bug#33880: [PATCH] Compress past NEWS files during installation

[Alex]

Eli Zaretskii <eliz@gnu.org> writes:

> Sorry for not responding earlier.

No worries.

> I don't think compressing *.ps files is a good idea, because then they
> cannot be printed without manually uncompressing them.

That makes sense; I just saw the TODO and figured that I might as well
do it while I was there.

> As for NEWS, I don't have a strong opinion, but it sounds like a very
> small gain for the trouble.  What do others think?  Perhaps this
> should be discussed on emacs-devel first.
>
> Thanks.

I don't think it's much trouble (though I admit that a MB isn't much of
a deal these days), especially after the patch has already been made.

I'll make a thread over there.

bug#33880: [PATCH] Compress past NEWS files during installation

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> As for NEWS, I don't have a strong opinion, but it sounds like a very
> small gain for the trouble.  What do others think?  Perhaps this
> should be discussed on emacs-devel first.

IIRC, Stefan wrote a small code snippet which scans NEWS files and tries
to determine, when a Lisp object was added to Emacs. I don't know
whether this would still work with compressed NEWS files.

Given, that disk space of 1 MB is nothing today, I'm not in favour of
such a compression. I don't know what it is good for, honestly.

> Thanks.

Best regards, Michael.

bug#33880: [PATCH] Compress past NEWS files during installation

[Alex]

Michael Albinus <michael.albinus@gmx.de> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> As for NEWS, I don't have a strong opinion, but it sounds like a very
>> small gain for the trouble.  What do others think?  Perhaps this
>> should be discussed on emacs-devel first.
>
> IIRC, Stefan wrote a small code snippet which scans NEWS files and tries
> to determine, when a Lisp object was added to Emacs. I don't know
> whether this would still work with compressed NEWS files.

It should work similarly to scanning *.el.gz files. Depending on how the
snippet was made (e.g., with a regex that accepts *.gz NEWS files), it
might even work with no changes.

> Given, that disk space of 1 MB is nothing today, I'm not in favour of
> such a compression. I don't know what it is good for, honestly.

It's indeed barely anything, but I also don't see much of a reason to
not do it. If we're already compressing a bunch of files, why not
compress the stale NEWS files at the same time?

bug#33880: [PATCH] Compress past NEWS files during installation

[Eli Zaretskii]

> From: Alex <agrambot@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  33880@debbugs.gnu.org
> Date: Thu, 14 Mar 2019 10:23:16 -0600
> 
> It's indeed barely anything, but I also don't see much of a reason to
> not do it. If we're already compressing a bunch of files, why not
> compress the stale NEWS files at the same time?

FWIW, I use those "stale" NEWS files all the time, because I
frequently need to establish what was a version of Emacs where some
feature was first introduced or announced.

bug#33880: [PATCH] Compress past NEWS files during installation

[Alex]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Alex <agrambot@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  33880@debbugs.gnu.org
>> Date: Thu, 14 Mar 2019 10:23:16 -0600
>> 
>> It's indeed barely anything, but I also don't see much of a reason to
>> not do it. If we're already compressing a bunch of files, why not
>> compress the stale NEWS files at the same time?
>
> FWIW, I use those "stale" NEWS files all the time, because I
> frequently need to establish what was a version of Emacs where some
> feature was first introduced or announced.

Oh, I didn't mean that "no one uses these dusty old files", but rather
"no one often uses these files outside of Emacs". If you view them in
Emacs, then `auto-compression-mode' decompresses the files on the fly. I
think compression here is less of an issue than it is for Emacs Lisp
files, which are already compressed and used for source code lookup all
the time.

bug#33880: [PATCH] Compress past NEWS files during installation

[Glenn Morris]

FWIW, I think files like NEWS belong in /usr/share/doc
(bug#25781, I haven't found any time to work on it yet).
I think compressing the files installed there would be very natural.
(Eg Debian seems to compress text files in /usr/share/doc by default.
Looks like Red Hat does not, though.)

bug#33880: [PATCH] Compress past NEWS files during installation

[Eli Zaretskii]

> From: Alex <agrambot@gmail.com>
> Cc: michael.albinus@gmx.de,  33880@debbugs.gnu.org
> Date: Thu, 14 Mar 2019 15:33:24 -0600
> 
> If you view them in Emacs, then `auto-compression-mode' decompresses
> the files on the fly.

I usually grep them outside Emacs.

bug#33880: [PATCH] Compress past NEWS files during installation

[Alex]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Alex <agrambot@gmail.com>
>> Cc: michael.albinus@gmx.de,  33880@debbugs.gnu.org
>> Date: Thu, 14 Mar 2019 15:33:24 -0600
>> 
>> If you view them in Emacs, then `auto-compression-mode' decompresses
>> the files on the fly.
>
> I usually grep them outside Emacs.

Any chance you have zgrep on your system?

If not, then do you not have an uncompressed git repository directory to
grep?

bug#33880: [PATCH] Compress past NEWS files during installation

[Eli Zaretskii]

> From: Alex <agrambot@gmail.com>
> Cc: 33880@debbugs.gnu.org
> Date: Sat, 16 Mar 2019 12:28:16 -0600
> 
> > I usually grep them outside Emacs.
> 
> Any chance you have zgrep on your system?

It's a Posix shell script, so not very useful on MS-Windows.

> If not, then do you not have an uncompressed git repository directory to
> grep?

Not on every machine I work on.  Besides, sometimes I need to look in
a released Emacs, whose sources are in a separate tree.

bug#33880: [PATCH] Compress past NEWS files during installation

[Michael Albinus]

Alex <agrambot@gmail.com> writes:

Hi Alex,

>>> If you view them in Emacs, then `auto-compression-mode' decompresses
>>> the files on the fly.
>>
>> I usually grep them outside Emacs.
>
> Any chance you have zgrep on your system?

I believe we could find a solution for all these problems. But I still
don't understand why we should.

Maybe you said it already, and I didn't care enough: could you pls say,
what is the advantage of compressed NEWS files, that we must do this.

Best regards, Michael

bug#33880: [PATCH] Compress past NEWS files during installation

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> Alex <agrambot@gmail.com> writes:
>
> Hi Alex,
>
>>>> If you view them in Emacs, then `auto-compression-mode' decompresses
>>>> the files on the fly.
>>>
>>> I usually grep them outside Emacs.
>>
>> Any chance you have zgrep on your system?
>
> I believe we could find a solution for all these problems. But I still
> don't understand why we should.
>
> Maybe you said it already, and I didn't care enough: could you pls say,
> what is the advantage of compressed NEWS files, that we must do this.

The space gain here was quite small, and there are drawbacks to
compressing the NEWS files, and nobody seemed really enthusiastic about
it, so I think the decision is not to do this, and I'm closing this bug
report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no