bug#51258: emacs-next/git-master broken due to pdmp file naming changes

bug#51258: emacs-next/git-master broken due to pdmp file naming changes

[Paxton Evans]

As of Emacs commit e81f1faca4382ed5c8f15fec84fb7c900a5468f9, building
emacs-next
with the git-master branch will cause some problems on Guix. The
"emacs-next"
build itself will work without any errors, but it will complain whenever
you try
run it:

> emacs: could not load dump file
> "/gnu/store/a5bx5v96snvgiv9r1b7i1im5ccn5mz54-emacs-next-git.master/libexec/emacs/29.0.50/x86_64-pc-linux-gnu/emacs-66e6890a5565f2bed1ee56075d21e0051d891a59200cdd092c0946403fb84ac2.pdmp":
> not a dump file

This will also cause new builds of packages that use
emacs-next/git-master as an
input to fail as well, with a version of the same error. Here's a
(failed) build
log for emacs-pdf-tools:

> [top part snipped]
> 
> make[1]: Leaving directory '/tmp/guix-build-emacs-pdf-tools-0.90-2.5f77dae.drv-0/source/server'
> phase `install' succeeded after 0.3 seconds
> starting phase `patch-shebangs'
> phase `patch-shebangs' succeeded after 0.0 seconds
> starting phase `strip'
> stripping binaries in "/gnu/store/swfjir05k1bg22l50yawhibbv0y5qhr9-emacs-pdf-tools-0.90-2.5f77dae/bin" with "strip" and flags ("--strip-debug" "--enable-deterministic-archives")
> phase `strip' succeeded after 0.0 seconds
> starting phase `validate-runpath'
> validating RUNPATH of 1 binaries in "/gnu/store/swfjir05k1bg22l50yawhibbv0y5qhr9-emacs-pdf-tools-0.90-2.5f77dae/bin"...
> phase `validate-runpath' succeeded after 0.0 seconds
> starting phase `validate-documentation-location'
> phase `validate-documentation-location' succeeded after 0.0 seconds
> starting phase `delete-info-dir-file'
> phase `delete-info-dir-file' succeeded after 0.0 seconds
> starting phase `patch-dot-desktop-files'
> phase `patch-dot-desktop-files' succeeded after 0.0 seconds
> starting phase `install-license-files'
> installing 0 license files from '.'
> phase `install-license-files' succeeded after 0.0 seconds
> starting phase `reset-gzip-timestamps'
> phase `reset-gzip-timestamps' succeeded after 0.0 seconds
> starting phase `compress-documentation'
> phase `compress-documentation' succeeded after 0.0 seconds
> starting phase `enter-lisp-dir'
> phase `enter-lisp-dir' succeeded after 0.0 seconds
> starting phase `emacs-patch-variables'
> emacs: could not load dump file
> "/gnu/store/rv72rvqa3vh2vw7jpkm3d9ww0xb4ibxv-emacs-next-git.master/libexec/emacs/29.0.50/x86_64-pc-linux-gnu/emacs-4588128eef9937d195927b0cccee280697619db43041dd08b620788b8dd59b77.pdmp":
> not a dump file

I think this is because of some recent changes made to the naming of the
Emacs
pdmp file in the master branch. Previously, it just made a file called
"emacs.pdmp" but now it appends the version or commit hash, i.e.
"emacs-4588128eef9937d195927b0cccee280697619db43041dd08b620788b8dd59b77.pdmp".
This new naming scheme started with Emacs commit
e81f1faca4382ed5c8f15fec84fb7c900a5468f9.

This might be confusing Guix's Emacs package recipe, which has a regexp
that
simply looks for "emacs.pdmp" when it does some cleanup after building
Emacs.

guix/gnu/packages/emacs.scm:182 and following:

           > (lambda* (#:key outputs target #:allow-other-keys)
           >   (let* ((libexec (string-append (assoc-ref outputs "out")
           >                                  "/libexec"))
           >          ;; each of these find-files should return one file
           >          (pdmp (find-files libexec "^emacs\\.pdmp$"))
           >          (pdmp-real (find-files libexec
           >                                 "^\\.emacs\\.pdmp-real$")))

So I think that regexp might need to be changed a little to accommodate
those
recent Emacs changes. That's as far as I'm able to get with it, though,
as I'm
not very good at Guile or regexps.

-paxton

bug#51258: [PATCH] gnu: emacs: Handle pdump filenames that contain a fingerprint

[Morgan.J.Smith]

From: Morgan Smith <Morgan.J.Smith@outlook.com>

* gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Add regex to handle
filenames that have a fingerprint in them
---

Fixed your problem.  Well it was also my problem.  The regex is pretty simple but probably a little intimidating if you've never done it before.  I highly recommend reading up on ERE's and BRE's from the POSIX specification if you have time.  I still don't exactly know how guile does regex but knowing POSIX regex has served me quite well.

I've tested this patch and it works great for me on the latest emacs commit.  It should also be backwards compatible so that's nice.


 gnu/packages/emacs.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 6d9950d068..6ebcfecbc7 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -183,9 +183,10 @@ (define* (emacs-byte-compile-directory dir)
              (let* ((libexec (string-append (assoc-ref outputs "out")
                                             "/libexec"))
                     ;; each of these find-files should return one file
-                    (pdmp (find-files libexec "^emacs\\.pdmp$"))
+                    ;; there might be a fingerprint in the name
+                    (pdmp (find-files libexec "^emacs(-[[:xdigit:]]+)?\\.pdmp$"))
                     (pdmp-real (find-files libexec
-                                           "^\\.emacs\\.pdmp-real$")))
+                                           "^\\.emacs(-[[:xdigit:]]+)?\\.pdmp-real$")))
                (for-each (lambda (wrapper real)
                            (delete-file wrapper)
                            (rename-file real wrapper))
-- 
2.33.0

bug#51258: [PATCH] gnu: emacs: Handle pdump filenames that contain a fingerprint

[Liliana Marie Prikler]

Hi,

Am Montag, den 18.10.2021, 19:50 -0400 schrieb
Morgan.J.Smith@outlook.com:
> From: Morgan Smith <Morgan.J.Smith@outlook.com>
> 
> * gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Add regex to
> handle
> filenames that have a fingerprint in them
> ---
> 
> Fixed your problem.  Well it was also my problem.  The regex is
> pretty simple but probably a little intimidating if you've never done
> it before.  I highly recommend reading up on ERE's and BRE's from the
> POSIX specification if you have time.  I still don't exactly know how
> guile does regex but knowing POSIX regex has served me quite well.
> 
> I've tested this patch and it works great for me on the latest emacs
> commit.  It should also be backwards compatible so that's nice.
> 
> 
>  gnu/packages/emacs.scm | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 6d9950d068..6ebcfecbc7 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -183,9 +183,10 @@ (define* (emacs-byte-compile-directory dir)
>               (let* ((libexec (string-append (assoc-ref outputs
> "out")
>                                              "/libexec"))
>                      ;; each of these find-files should return one
> file
> -                    (pdmp (find-files libexec "^emacs\\.pdmp$"))
> +                    ;; there might be a fingerprint in the name
> +                    (pdmp (find-files libexec "^emacs(-
> [[:xdigit:]]+)?\\.pdmp$"))
>                      (pdmp-real (find-files libexec
> -                                           "^\\.emacs\\.pdmp-
> real$")))
> +                                           "^\\.emacs(-
> [[:xdigit:]]+)?\\.pdmp-real$")))
>                 (for-each (lambda (wrapper real)
>                             (delete-file wrapper)
>                             (rename-file real wrapper))
Hi, this might sound somewhat odd, but I think this solution could be a
little over-engineered (also it does not match the case where the
version is an actual version rather than a git hash).  Perhaps instead
we ought to simply look for the .pdmp-real suffix, construct the old
file name and move the file back.  WDYT?

bug#51258: [PATCH v2] gnu: emacs: Handle pdump filenames that contain a fingerprint

[Morgan.J.Smith]

From: Morgan Smith <Morgan.J.Smith@outlook.com>

* gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Change regex to handle
filenames that have a fingerprint in them. Also remove #t at the end of the phase.
---

I'm a fan of your idea. I tested this patch and it seems to work as expected.

I do think that the fingerprint will never be a version and my original patch would've been fine. However, I'm not certain of that and there isn't a good reason to have a super precise regex here anyways.


 gnu/packages/emacs.scm | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 6d9950d068..2aae21be6d 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -183,14 +183,13 @@ (define* (emacs-byte-compile-directory dir)
              (let* ((libexec (string-append (assoc-ref outputs "out")
                                             "/libexec"))
                     ;; each of these find-files should return one file
-                    (pdmp (find-files libexec "^emacs\\.pdmp$"))
+                    (pdmp (find-files libexec "\\.pdmp$"))
                     (pdmp-real (find-files libexec
-                                           "^\\.emacs\\.pdmp-real$")))
+                                           "\\.pdmp-real$")))
                (for-each (lambda (wrapper real)
                            (delete-file wrapper)
                            (rename-file real wrapper))
-                         pdmp pdmp-real))
-             #t))
+                         pdmp pdmp-real))))
          (add-after 'glib-or-gtk-wrap 'strip-double-wrap
            (lambda* (#:key outputs #:allow-other-keys)
              ;; Directly copy emacs-X.Y to emacs, so that it is not wrapped
-- 
2.33.1

bug#51258: [PATCH v2] gnu: emacs: Handle pdump filenames that contain a fingerprint

[Liliana Marie Prikler]

Am Dienstag, den 19.10.2021, 15:10 -0400 schrieb
Morgan.J.Smith@outlook.com:
> From: Morgan Smith <Morgan.J.Smith@outlook.com>
> 
> * gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Change regex
> to handle filenames that have a fingerprint in them. Also remove #t
> at the end of the phase.
> ---
> 
> I'm a fan of your idea. I tested this patch and it seems to work as
> expected.
> 
> I do think that the fingerprint will never be a version and my
> original patch would've been fine. However, I'm not certain of that
> and there isn't a good reason to have a super precise regex here
> anyways.
Mostly LGTM, but keep in mind that your new pdmp and pdmp-real could
potentially match more than one file even if realistically there's only
one to be matched.  Hence why I've said we ought to construct the old
file name from the pdmp-real, because find-files need not be sorted.

> 
>  gnu/packages/emacs.scm | 7 +++----
>  1 file changed, 3 insertions(+), 4 deletions(-)
> 
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 6d9950d068..2aae21be6d 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -183,14 +183,13 @@ (define* (emacs-byte-compile-directory dir)
>               (let* ((libexec (string-append (assoc-ref outputs
> "out")
>                                              "/libexec"))
>                      ;; each of these find-files should return one
> file
> -                    (pdmp (find-files libexec "^emacs\\.pdmp$"))
> +                    (pdmp (find-files libexec "\\.pdmp$"))
>                      (pdmp-real (find-files libexec
> -                                           "^\\.emacs\\.pdmp-
> real$")))
> +                                           "\\.pdmp-real$")))
>                 (for-each (lambda (wrapper real)
>                             (delete-file wrapper)
>                             (rename-file real wrapper))
> -                         pdmp pdmp-real))
> -             #t))
> +                         pdmp pdmp-real))))
>           (add-after 'glib-or-gtk-wrap 'strip-double-wrap
>             (lambda* (#:key outputs #:allow-other-keys)
>               ;; Directly copy emacs-X.Y to emacs, so that it is not
> wrapped

Regards,
Liliana

bug#51258: [PATCH v3] gnu: emacs: Handle pdump filenames that contain a fingerprint

[Morgan.J.Smith]

From: Morgan Smith <Morgan.J.Smith@outlook.com>

* gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Change regex to handle
filenames that have a fingerprint in them. Remove #t at the end of the
phase. Don't bother deleting the old files because rename will overwrite them
anyways.
---

find-files does sort its return so my previous 2 patches probably work great
regardless of the number of pdmps we encounter (which will only ever be 1
anyways).

Also the fingerprint is a 32 char hexidecimal number used to uniquely[1]
identify the Emacs binary (well more specifically an earlier binary generated
during the build process).  If you put the version in there it would
completely invalidate the entire purpose of this fingerprint.  Different
distros will have different fingerprints for the same version and that's the
point.

Here is another over engineered solution even though the previous 2 still work
great.  I'm not sure if I'm supposed to just throw a '(use-modules ...)' in the
phase but if we want to use the #:modules argument then it will break all of
the other Emacsen that inherit from Emacs since they use different build
systems.


1. Hash collisions are a thing so maybe not actually uniquely


 gnu/packages/emacs.scm | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 6d9950d068..e25b79b832 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -179,18 +179,18 @@ (define* (emacs-byte-compile-directory dir)
          (add-after 'glib-or-gtk-wrap 'restore-emacs-pdmp
            ;; restore the dump file that Emacs installs somewhere in
            ;; libexec/ to its original state
-           (lambda* (#:key outputs target #:allow-other-keys)
-             (let* ((libexec (string-append (assoc-ref outputs "out")
+           (lambda* (#:key outputs #:allow-other-keys)
+             (use-modules (ice-9 regex))
+             (let* ((pdmp-regex ".(emacs(-[[:xdigit:]]+)?.pdmp)-real")
+                    (libexec (string-append (assoc-ref outputs "out")
                                             "/libexec"))
-                    ;; each of these find-files should return one file
-                    (pdmp (find-files libexec "^emacs\\.pdmp$"))
-                    (pdmp-real (find-files libexec
-                                           "^\\.emacs\\.pdmp-real$")))
-               (for-each (lambda (wrapper real)
-                           (delete-file wrapper)
-                           (rename-file real wrapper))
-                         pdmp pdmp-real))
-             #t))
+                    (pdmp (find-files libexec pdmp-regex)))
+               (map
+                (lambda (pdmp)
+                  (rename-file
+                   pdmp
+                   (regexp-substitute #f (string-match pdmp-regex pdmp) 'pre 1)))
+                pdmp))))
          (add-after 'glib-or-gtk-wrap 'strip-double-wrap
            (lambda* (#:key outputs #:allow-other-keys)
              ;; Directly copy emacs-X.Y to emacs, so that it is not wrapped
-- 
2.33.1

bug#51258: [PATCH v3] gnu: emacs: Handle pdump filenames that contain a fingerprint

[Liliana Marie Prikler]

Hi,

Am Samstag, den 23.10.2021, 21:00 -0400 schrieb
Morgan.J.Smith@outlook.com:
> From: Morgan Smith <Morgan.J.Smith@outlook.com>
> 
> * gnu/packages/emacs.scm (emacs) [restore-emacs-pdmp]: Change regex
> to handle
> filenames that have a fingerprint in them. Remove #t at the end of
> the
> phase. Don't bother deleting the old files because rename will
> overwrite them
> anyways.
> ---
> 
> find-files does sort its return so my previous 2 patches probably
> work great regardless of the number of pdmps we encounter (which will
> only ever be 1 anyways).
You are completely right, thanks for pointing that out.  I've pushed an
adjusted v2 with updated comments and everything.  I also verified that
emacs-next could be updated with a little hacking around patches that
don't apply.

Thanks!