[PATCH 0/3] Update 'emacs-w3m' and improve loading emacs autoloads

[PATCH 0/3] Update 'emacs-w3m' and improve loading emacs autoloads

[Alex Kost]

During updating 'emacs-w3m', I found that its *.el files are compressed
now, so there is no "w3m-load.el" anymore, but there are "w3m-load.elc"
and "w3m-load.el.gz".  So this patchset also contains a patch to load
any "autoloads" file (el, elc or el.gz).

[PATCH 1/3] emacs: Handle compiled and compressed autoloads.
[PATCH 2/3] gnu: emacs-w3m: Use 'modify-syntax'.
[PATCH 3/3] gnu: emacs-w3m: Update to 1.4.538+0.20141022.

[PATCH 1/3] emacs: Handle compiled and compressed autoloads.

[Alex Kost]

* emacs/guix-emacs.el (guix-emacs-autoloads-regexp): New variable.
  (guix-emacs-find-autoloads-in-directory): Return a list of autoloads
  without extensions.
---
 emacs/guix-emacs.el | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/emacs/guix-emacs.el b/emacs/guix-emacs.el
index 6515b3d..948d2f2 100644
--- a/emacs/guix-emacs.el
+++ b/emacs/guix-emacs.el
@@ -37,6 +37,12 @@ they are successfully installed."
 (defvar guix-emacs-autoloads nil
   "List of the last loaded Emacs autoloads.")
 
+(defvar guix-emacs-autoloads-regexp
+  (rx (group (* any) "-autoloads")
+      ".el" (zero-or-one "c") (zero-or-one ".gz")
+      string-end)
+  "Regexp to match Emacs 'autoloads' file.")
+
 (defun guix-emacs-directory (&optional profile)
   "Return directory with Emacs packages installed in PROFILE.
 If PROFILE is nil, use `guix-user-profile'."
@@ -44,8 +50,15 @@ If PROFILE is nil, use `guix-user-profile'."
                     (or profile guix-user-profile)))
 
 (defun guix-emacs-find-autoloads-in-directory (directory)
-  "Return list of Emacs 'autoloads' files in DIRECTORY."
-  (directory-files directory 'full-name "-autoloads\\.el\\'" 'no-sort))
+  "Return a list of Emacs 'autoloads' files in DIRECTORY.
+The files in the list do not have extensions (.el, .elc)."
+  (cl-remove-duplicates
+   (delq nil
+        (mapcar (lambda (file)
+                  (when (string-match guix-emacs-autoloads-regexp file)
+                    (match-string 1 file)))
+                (directory-files directory 'full-name nil 'no-sort)))
+   :test #'string=))
 
 (defun guix-emacs-subdirs (directory)
   "Return list of DIRECTORY subdirectories."
-- 
2.6.3

Re: [PATCH 1/3] emacs: Handle compiled and compressed autoloads.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> * emacs/guix-emacs.el (guix-emacs-autoloads-regexp): New variable.
>   (guix-emacs-find-autoloads-in-directory): Return a list of autoloads
>   without extensions.

LGTM.

However, note that autoloads may only be compressed when we know they do
not embed store file names (which I think is usually the case.)
Otherwise there’s a risk of fooling the GC, as in
<http://bugs.gnu.org/20765>.

Ludo’.

Re: [PATCH 1/3] emacs: Handle compiled and compressed autoloads.

[Alex Kost]

Ludovic Courtès (2015-12-08 02:00 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> * emacs/guix-emacs.el (guix-emacs-autoloads-regexp): New variable.
>>   (guix-emacs-find-autoloads-in-directory): Return a list of autoloads
>>   without extensions.
>
> LGTM.
>
> However, note that autoloads may only be compressed when we know they do
> not embed store file names (which I think is usually the case.)

Thanks for the info, currently none of the emacs packages provides a
compressed 'autoloads' file.

> Otherwise there’s a risk of fooling the GC, as in
> <http://bugs.gnu.org/20765>.

Hm, OK.  I don't fully understand the issue though.  So *.el files of
'emacs-w3m-1.4.538+0.20141022' will be compressed, and some of these
*.el.gz files contain store file names.  Do these files fool GC?
If so, I think 'emacs-w3m' should be configured with
'--without-compress-install', right?

-- 
Alex

Re: [PATCH 1/3] emacs: Handle compiled and compressed autoloads.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-12-08 02:00 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
>>> * emacs/guix-emacs.el (guix-emacs-autoloads-regexp): New variable.
>>>   (guix-emacs-find-autoloads-in-directory): Return a list of autoloads
>>>   without extensions.
>>
>> LGTM.
>>
>> However, note that autoloads may only be compressed when we know they do
>> not embed store file names (which I think is usually the case.)
>
> Thanks for the info, currently none of the emacs packages provides a
> compressed 'autoloads' file.

Perfect.

>> Otherwise there’s a risk of fooling the GC, as in
>> <http://bugs.gnu.org/20765>.
>
> Hm, OK.  I don't fully understand the issue though.  So *.el files of
> 'emacs-w3m-1.4.538+0.20141022' will be compressed, and some of these
> *.el.gz files contain store file names.  Do these files fool GC?

Yes.  When a derivation build completes, the daemon scans all the files
in the derivation outputs for /gnu/store/… patterns, and then records
those patterns in the ‘References’ table of /var/guix/db/db.sqlite.

This is what makes sure that if /gnu/store/A refers to /gnu/store/B, and
if /gnu/store/A is live, then /gnu/store/B will not be GC’.d

> If so, I think 'emacs-w3m' should be configured with
> '--without-compress-install', right?

Sounds like it!

Thanks,
Ludo’.

Re: [PATCH 1/3] emacs: Handle compiled and compressed autoloads.

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 1321 bytes --]

Ludovic Courtès (2015-12-09 16:58 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2015-12-08 02:00 +0300) wrote:
>>> Otherwise there’s a risk of fooling the GC, as in
>>> <http://bugs.gnu.org/20765>.
>>
>> Hm, OK.  I don't fully understand the issue though.  So *.el files of
>> 'emacs-w3m-1.4.538+0.20141022' will be compressed, and some of these
>> *.el.gz files contain store file names.  Do these files fool GC?
>
> Yes.  When a derivation build completes, the daemon scans all the files
> in the derivation outputs for /gnu/store/… patterns, and then records
> those patterns in the ‘References’ table of /var/guix/db/db.sqlite.
>
> This is what makes sure that if /gnu/store/A refers to /gnu/store/B, and
> if /gnu/store/A is live, then /gnu/store/B will not be GC’.d

Thank you for the descriptive explanation!  I confess I'm almost
ignorant in these internals.

>> If so, I think 'emacs-w3m' should be configured with
>> '--without-compress-install', right?
>
> Sounds like it!

Great, I have updated and pushed 'emacs-w3m' patch.

Now since it is clear that it is potentially problematic to have
compressed *.el files, I have changed this patch to handle only compiled
autoloads as the compressed ones are unlikely to appear ever.  OK?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-emacs-Handle-compiled-autoloads.patch --]
[-- Type: text/x-patch, Size: 1858 bytes --]

From baa1950d823d0de07fa083f5f2770d3d6b681510 Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Thu, 3 Dec 2015 13:03:19 +0300
Subject: [PATCH] emacs: Handle compiled autoloads.

* emacs/guix-emacs.el (guix-emacs-autoloads-regexp): New variable.
  (guix-emacs-find-autoloads-in-directory): Return a list of autoloads
  without extensions.
---
 emacs/guix-emacs.el | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/emacs/guix-emacs.el b/emacs/guix-emacs.el
index 6515b3d..2f809ed 100644
--- a/emacs/guix-emacs.el
+++ b/emacs/guix-emacs.el
@@ -37,6 +37,11 @@ they are successfully installed."
 (defvar guix-emacs-autoloads nil
   "List of the last loaded Emacs autoloads.")
 
+(defvar guix-emacs-autoloads-regexp
+  (rx (group (* any) "-autoloads")
+      ".el" (zero-or-one "c") string-end)
+  "Regexp to match Emacs 'autoloads' file.")
+
 (defun guix-emacs-directory (&optional profile)
   "Return directory with Emacs packages installed in PROFILE.
 If PROFILE is nil, use `guix-user-profile'."
@@ -44,8 +49,15 @@ If PROFILE is nil, use `guix-user-profile'."
                     (or profile guix-user-profile)))
 
 (defun guix-emacs-find-autoloads-in-directory (directory)
-  "Return list of Emacs 'autoloads' files in DIRECTORY."
-  (directory-files directory 'full-name "-autoloads\\.el\\'" 'no-sort))
+  "Return a list of Emacs 'autoloads' files in DIRECTORY.
+The files in the list do not have extensions (.el, .elc)."
+  (cl-remove-duplicates
+   (delq nil
+        (mapcar (lambda (file)
+                  (when (string-match guix-emacs-autoloads-regexp file)
+                    (match-string 1 file)))
+                (directory-files directory 'full-name nil 'no-sort)))
+   :test #'string=))
 
 (defun guix-emacs-subdirs (directory)
   "Return list of DIRECTORY subdirectories."
-- 
2.6.3

Re: [PATCH 1/3] emacs: Handle compiled and compressed autoloads.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Great, I have updated and pushed 'emacs-w3m' patch.

Perfect.

> Now since it is clear that it is potentially problematic to have
> compressed *.el files, I have changed this patch to handle only compiled
> autoloads as the compressed ones are unlikely to appear ever.  OK?

Sounds good.

> From baa1950d823d0de07fa083f5f2770d3d6b681510 Mon Sep 17 00:00:00 2001
> From: Alex Kost <alezost@gmail.com>
> Date: Thu, 3 Dec 2015 13:03:19 +0300
> Subject: [PATCH] emacs: Handle compiled autoloads.
>
> * emacs/guix-emacs.el (guix-emacs-autoloads-regexp): New variable.
>   (guix-emacs-find-autoloads-in-directory): Return a list of autoloads
>   without extensions.

[...]

>  (defun guix-emacs-find-autoloads-in-directory (directory)
> -  "Return list of Emacs 'autoloads' files in DIRECTORY."
> -  (directory-files directory 'full-name "-autoloads\\.el\\'" 'no-sort))
> +  "Return a list of Emacs 'autoloads' files in DIRECTORY.
> +The files in the list do not have extensions (.el, .elc)."

Callers won’t have any problems with the fact that the extension is
stripped, right?

Otherwise LGTM.

Thanks,
Ludo’.

Re: [PATCH 1/3] emacs: Handle compiled and compressed autoloads.

[Alex Kost]

Ludovic Courtès (2015-12-11 12:51 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
[...]
>> From baa1950d823d0de07fa083f5f2770d3d6b681510 Mon Sep 17 00:00:00 2001
>> From: Alex Kost <alezost@gmail.com>
>> Date: Thu, 3 Dec 2015 13:03:19 +0300
>> Subject: [PATCH] emacs: Handle compiled autoloads.
>>
>> * emacs/guix-emacs.el (guix-emacs-autoloads-regexp): New variable.
>>   (guix-emacs-find-autoloads-in-directory): Return a list of autoloads
>>   without extensions.
>
> [...]
>
>>  (defun guix-emacs-find-autoloads-in-directory (directory)
>> -  "Return list of Emacs 'autoloads' files in DIRECTORY."
>> -  (directory-files directory 'full-name "-autoloads\\.el\\'" 'no-sort))
>> +  "Return a list of Emacs 'autoloads' files in DIRECTORY.
>> +The files in the list do not have extensions (.el, .elc)."
>
> Callers won’t have any problems with the fact that the extension is
> stripped, right?

Right, "C-h f load" tells that it picks a proper .elc/.el file.

> Otherwise LGTM.

Thanks, pushed.

-- 
Alex

[PATCH 2/3] gnu: emacs-w3m: Use 'modify-syntax'.

[Alex Kost]

* gnu/packages/emacs.scm (emacs-w3m)[arguments]: Use 'modify-syntax'.
---
 gnu/packages/emacs.scm | 67 +++++++++++++++++++++++++-------------------------
 1 file changed, 33 insertions(+), 34 deletions(-)

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 947ddcb..e0c986e 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -600,40 +600,39 @@ provides an optional IDE-like error list.")
                               out "/share/images/emacs-w3m")))
        #:tests? #f  ; no check target
        #:phases
-       (alist-cons-after
-        'unpack 'autoconf
-        (lambda _
-          (zero? (system* "autoconf")))
-        (alist-cons-before
-         'build 'patch-exec-paths
-         (lambda* (#:key inputs outputs #:allow-other-keys)
-          (let ((out (assoc-ref outputs "out"))
-                (w3m (assoc-ref inputs "w3m"))
-                (imagemagick (assoc-ref inputs "imagemagick"))
-                (coreutils (assoc-ref inputs "coreutils")))
-            (emacs-substitute-variables "w3m.el"
-              ("w3m-command" (string-append w3m "/bin/w3m"))
-              ("w3m-touch-command" (string-append coreutils "/bin/touch"))
-              ("w3m-image-viewer" (string-append imagemagick "/bin/display"))
-              ("w3m-icon-directory" (string-append out
-                                                   "/share/images/emacs-w3m")))
-            (emacs-substitute-variables "w3m-image.el"
-              ("w3m-imagick-convert-program" (string-append imagemagick
-                                                            "/bin/convert"))
-              ("w3m-imagick-identify-program" (string-append imagemagick
-                                                             "/bin/identify")))
-            #t))
-         (alist-replace
-          'install
-          (lambda* (#:key outputs #:allow-other-keys)
-            (and (zero? (system* "make" "install" "install-icons"))
-                 (with-directory-excursion
-                     (string-append (assoc-ref outputs "out")
-                                    "/share/emacs/site-lisp")
-                   (for-each delete-file '("ChangeLog" "ChangeLog.1"))
-                   (symlink "w3m-load.el" "w3m-autoloads.el")
-                   #t)))
-          %standard-phases)))))
+       (modify-phases %standard-phases
+         (add-after 'unpack 'autoconf
+           (lambda _
+             (zero? (system* "autoconf"))))
+         (add-before 'build 'patch-exec-paths
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out"))
+                   (w3m (assoc-ref inputs "w3m"))
+                   (imagemagick (assoc-ref inputs "imagemagick"))
+                   (coreutils (assoc-ref inputs "coreutils")))
+               (emacs-substitute-variables "w3m.el"
+                 ("w3m-command" (string-append w3m "/bin/w3m"))
+                 ("w3m-touch-command"
+                  (string-append coreutils "/bin/touch"))
+                 ("w3m-image-viewer"
+                  (string-append imagemagick "/bin/display"))
+                 ("w3m-icon-directory"
+                  (string-append out "/share/images/emacs-w3m")))
+               (emacs-substitute-variables "w3m-image.el"
+                 ("w3m-imagick-convert-program"
+                  (string-append imagemagick "/bin/convert"))
+                 ("w3m-imagick-identify-program"
+                  (string-append imagemagick "/bin/identify")))
+               #t)))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (and (zero? (system* "make" "install" "install-icons"))
+                  (with-directory-excursion
+                      (string-append (assoc-ref outputs "out")
+                                     "/share/emacs/site-lisp")
+                    (for-each delete-file '("ChangeLog" "ChangeLog.1"))
+                    (symlink "w3m-load.el" "w3m-autoloads.el")
+                    #t)))))))
     (home-page "http://emacs-w3m.namazu.org/")
     (synopsis "Simple Web browser for Emacs based on w3m")
     (description
-- 
2.6.3

Re: [PATCH 2/3] gnu: emacs-w3m: Use 'modify-syntax'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> * gnu/packages/emacs.scm (emacs-w3m)[arguments]: Use 'modify-syntax'.

OK!

Ludo'.

[PATCH 3/3] gnu: emacs-w3m: Update to 1.4.538+0.20141022.

[Alex Kost]

* gnu/packages/emacs.scm (emacs-w3m): Update to 1.4.538+0.20141022.
  [arguments]: Symlink 'autoloads' to a compiled "w3m-load.elc".
---
 gnu/packages/emacs.scm | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index e0c986e..89dd5d4 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -574,13 +574,14 @@ provides an optional IDE-like error list.")
 (define-public emacs-w3m
   (package
     (name "emacs-w3m")
-    (version "1.4.483+0.20120614")
+    (version "1.4.538+0.20141022")
     (source (origin
              (method url-fetch)
              (uri (string-append "mirror://debian/pool/main/w/w3m-el/w3m-el_"
                                  version ".orig.tar.gz"))
              (sha256
-              (base32 "0ms181gjavnfk79hhv5xl9llik4c6kj0w3c04kgyif8lcy2sxljx"))))
+              (base32
+               "0zfxmq86pwk64yv0426gnjrvhjrgrjqn08sdcdhmmjmfpmqvm79y"))))
     (build-system gnu-build-system)
     (native-inputs `(("autoconf" ,autoconf)))
     (inputs `(("w3m" ,w3m)
@@ -631,7 +632,7 @@ provides an optional IDE-like error list.")
                       (string-append (assoc-ref outputs "out")
                                      "/share/emacs/site-lisp")
                     (for-each delete-file '("ChangeLog" "ChangeLog.1"))
-                    (symlink "w3m-load.el" "w3m-autoloads.el")
+                    (symlink "w3m-load.elc" "w3m-autoloads.elc")
                     #t)))))))
     (home-page "http://emacs-w3m.namazu.org/")
     (synopsis "Simple Web browser for Emacs based on w3m")
-- 
2.6.3

Re: [PATCH 3/3] gnu: emacs-w3m: Update to 1.4.538+0.20141022.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> * gnu/packages/emacs.scm (emacs-w3m): Update to 1.4.538+0.20141022.
>   [arguments]: Symlink 'autoloads' to a compiled "w3m-load.elc".

LGTM!

Ludo'.