[bug#49517] [PATCH] gnu: txr: Build documentation and update to 265.

["Paul A. Patience" <paul@apatience.com>]

[-- Attachment #1: Type: text/plain, Size: 3413 bytes --]

Hi Guillaume,

On Sunday, July 18th, 2021 at 06:36, Guillaume Le Vaillant <glv@posteo.net> wrote:

> Hi Kaz,
>
> I tried your patch and it doesn't fix all the timestamps in the
> environment used to build Guix packages:

I had sent an email last night but accidentally only to Kaz. Here it is below:

On Saturday, July 17th, 2021 at 18:51, Kaz Kylheku <kaz@kylheku.com> wrote:
> On 2021-07-17 02:57, Guillaume Le Vaillant wrote:
>> When testing the patch to build the HTML and PDF documentation,
>> I noticed that the 'share/doc/txr-263/txr-manpage.pdf' file is not
>> reproducible. There are some timestamps and UUIDs in it that change at
>> each build (diffoscope output attached).

I've updated the first patch to fix this by setting GS_GENERATE_UUIDS
to 0, which seems to be the standard Guix way to patch groff's use of
Ghostscript.
It removes most of the date (i.e., the hours, minutes and seconds) and
the UUID, but leaves the year, month and day:

  $ xxd /gnu/store/h94iilsa2xsp2ymn3k9x3ckmvfjha731-txr-266/share/doc/txr-266/txr-manpage.pdf | grep -C 1 Date
  00231430: 702f 312e 302f 273e 3c78 6d70 3a4d 6f64  p/1.0/'><xmp:Mod
  00231440: 6966 7944 6174 653e 3230 3231 2d30 372d  ifyDate>2021-07-
  00231450: 3138 3c2f 786d 703a 4d6f 6469 6679 4461  18</xmp:ModifyDa
  --
  00231470: 6174 653e 3230 3231 2d30 372d 3138 3c2f  ate>2021-07-18</
  00231480: 786d 703a 4372 6561 7465 4461 7465 3e0a  xmp:CreateDate>.
  00231490: 3c78 6d70 3a43 7265 6174 6f72 546f 6f6c  <xmp:CreatorTool

Is this acceptable?
Otherwise we may have to resort to a variation of the method Kaz
mentioned, though it's probably better to fix the Ghostscript patches
implementing GS_GENERATE_UUIDS, because otherwise any package relying on
groff to make PDFs will suffer from this very problem.

> Thank you for your report. I don't see anything in the pdfroff
> documentation about getting rid of this.

The problem is in fact with Ghostscript [1].
Ghostscript is the program adding the metadata.

> 2. Is there some recommended practice with regard to some
>     ./configure option or environment/make variable to react to
>     for ensuring reproducible builds? So that is to say, suppose
>     I don't wish to do the above embedded XML cleaning, except
>     when building for a distro that strives for reproducibility.
>
>     For opting in to reproducibilty, should I again rely on
>     SOURCE_DATE_EPOCH and have the build react to it?

I think the goal of SOURCE_DATE_EPOCH is for projects such as TXR to
need do nothing, and rather have Guix arrange for the "builder"
applications (i.e., Ghostscript here) to produce reproducible outputs.
In this case with GS_GENERATE_UUIDS=0.
So I don't think TXR need change anything.

Since I had to make a change in one of the patches, I have added a third
patch (squeezed in between the other two) adjusting the installation of
the license files.
The three patches are attached.

(Kaz, if there's anything TXR should change, perhaps it is the target
directory of the license files, i.e., $(datadir) -> $(docdir).
I think it's more common in general to install license files into
/usr/share/doc/APP rather than /usr/share/APP -- at least, that's where
Guix installs them.
This would render the second attached patch unnecessary.)

Best regards,
Paul

[1]: https://bugs.ghostscript.com/show_bug.cgi?id=696765

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-txr-Build-documentation.patch --]
[-- Type: text/x-patch; name=0001-gnu-txr-Build-documentation.patch, Size: 7049 bytes --]

From 693c520e8cc15fb3a3e2e85d561bbed9d2277841 Mon Sep 17 00:00:00 2001
From: "Paul A. Patience" <paul@apatience.com>
Date: Sat, 10 Jul 2021 20:23:49 -0400
Subject: [PATCH 1/3] gnu: txr: Build documentation.

* gnu/packages/lisp.scm (man-for-txr): New variable.
(txr)[native-inputs]: New field.
[inputs]: Move above the arguments field.
[arguments]<#:phases>{inhibit-doc-syms-generation, build-doc}
{install-doc}: New phases.
{fix-tests}: Add after the unpack phase rather than after the configure
phase so that all file-modifying phases are performed after unpacking.
---
 gnu/packages/lisp.scm | 96 ++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 90 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/lisp.scm b/gnu/packages/lisp.scm
index 382f48beb1..7625c57b60 100644
--- a/gnu/packages/lisp.scm
+++ b/gnu/packages/lisp.scm
@@ -18,6 +18,7 @@
 ;;; Copyright © 2020 Marius Bakke <mbakke@fastmail.com>
 ;;; Copyright © 2020 Zhu Zihao <all_but_last@163.com>
 ;;; Copyright © 2021 Sharlatan Hellseher <sharlatanus@gmail.com>
+;;; Copyright © 2021 Paul A. Patience <paul@apatience.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -59,8 +60,10 @@
   #:use-module (gnu packages fontutils)
   #:use-module (gnu packages gcc)
   #:use-module (gnu packages gettext)
+  #:use-module (gnu packages ghostscript)
   #:use-module (gnu packages gl)
   #:use-module (gnu packages glib)
+  #:use-module (gnu packages groff)
   #:use-module (gnu packages m4)
   #:use-module (gnu packages maths)
   #:use-module (gnu packages multiprecision)
@@ -846,6 +849,60 @@ enough to play the original mainframe Zork all the way through.")
       (home-page "http://www.russotto.net/git/mrussotto/confusion/src/master/src/README")
       (license license:gpl3+))))
 
+(define man-for-txr
+  (let ((commit "dfbf19b9a96474b8c1bacac85e43605e5691ceb2")
+        ;; Number of additional commits since the last tag (see the output of
+        ;; "git describe --tags").
+        (revision "41"))
+    (package
+      (name "man-for-txr")
+      (version (git-version "1.6g" revision commit))
+      (source
+       (origin
+         (method git-fetch)
+         (uri (git-reference
+               (url "http://www.kylheku.com/git/man/")
+               (commit commit)))
+         (file-name (git-file-name name version))
+         (sha256
+          (base32 "1zy0g8fj9nsfwzvg88hyaiy94r8j14xhs8vy2ln2niqdm6x2lvy2"))))
+      (build-system gnu-build-system)
+      (arguments
+       `(#:tests? #f                    ; There are no tests.
+         #:phases
+         (modify-phases %standard-phases
+           (add-after 'unpack 'fix-man2html-makefile
+             (lambda _
+               (substitute* "man2html/Makefile.in"
+                 ;; It inadvertently ignores @bindir@.
+                 (("^(bindir = \\$\\(DESTDIR\\)\\$\\(PREFIX\\)).*" _ prefix)
+                  (string-append prefix "@bindir@\n")))
+               #t))
+           (add-after 'unpack 'delete-generated-files
+             (lambda _
+               (for-each delete-file
+                         (append
+                          (list "conf_script")
+                          (map (lambda (d) (string-append d "/Makefile"))
+                               '("." "man" "man2html" "src"))
+                          (map (lambda (f) (string-append "src/" f))
+                               '("makewhatis.in" "man.conf"
+                                 "paths.h" "version.h"))))
+               #t))
+           (replace 'configure
+             (lambda* (#:key outputs #:allow-other-keys)
+               (setenv "CC" ,(cc-for-target))
+               ;; Humor the manually written configure script.
+               (invoke "./configure" "+lang" "en" "+fhs"
+                       (string-append "-prefix=" (assoc-ref outputs "out")))
+               #t)))))
+      (home-page "http://www.kylheku.com/cgit/man/")
+      (synopsis "Modifications to the man utilities, specifically man2html")
+      (description
+       "This is a fork of the man utilities intended specifically for building
+the HTML documentation of TXR.")
+      (license license:gpl2))))
+
 (define-public txr
   (package
     (name "txr")
@@ -860,6 +917,13 @@ enough to play the original mainframe Zork all the way through.")
        (sha256
         (base32 "14zaziymnbr2ld79x4h7sf88bzzzj82w3xpavmcx7mhwannb2swh"))))
     (build-system gnu-build-system)
+    (native-inputs
+     ;; Required to build the documentation.
+     `(("ghostscript" ,ghostscript)
+       ("groff" ,groff)
+       ("man2html" ,man-for-txr)))
+    (inputs
+     `(("libffi" ,libffi)))
     (arguments
      `(#:configure-flags
        (list ,(string-append "cc=" (cc-for-target))
@@ -867,6 +931,22 @@ enough to play the original mainframe Zork all the way through.")
        #:test-target "tests"
        #:phases
        (modify-phases %standard-phases
+         (add-after 'unpack 'inhibit-doc-syms-generation
+           (lambda _
+             (substitute* "genman.txr"
+               ;; Exit from genman.txr before it tries to write to
+               ;; stdlib/doc-syms.tl, which is anyway kept up to date with
+               ;; each release (and is already compiled to stdlib/doc-syms.tlo
+               ;; when genman.txr is run).
+               (("^@\\(output \"share/txr/stdlib/doc-syms\\.tl\"\\).*" line)
+                (string-append "@(do (exit))\n" line)))
+             #t))
+         (add-after 'unpack 'fix-tests
+           (lambda _
+             (substitute* (list "tests/017/realpath.tl"
+                                "tests/017/realpath.expected")
+               (("/usr/bin") "/"))
+             #t))
          (replace 'configure
            ;; ./configure is a hand-written script that can't handle standard
            ;; autotools arguments like CONFIG_SHELL.
@@ -874,14 +954,18 @@ enough to play the original mainframe Zork all the way through.")
              (setenv "txr_shell" (which "bash"))
              (apply invoke "./configure" configure-flags)
              #t))
-         (add-after 'configure 'fix-tests
+         (add-after 'build 'build-doc
            (lambda _
-             (substitute* (list "tests/017/realpath.tl"
-                                "tests/017/realpath.expected")
-               (("/usr/bin") "/"))
+             (setenv "GS_GENERATE_UUIDS" "0")
+             (invoke "make" "txr-manpage.html" "txr-manpage.pdf")
+             #t))
+         (add-after 'install 'install-doc
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let ((doc (string-append (assoc-ref outputs "out")
+                                       "/share/doc/" ,name "-" ,version)))
+               (for-each (lambda (f) (install-file f doc))
+                         '("txr-manpage.html" "txr-manpage.pdf")))
              #t)))))
-    (inputs
-     `(("libffi" ,libffi)))
     (synopsis "General-purpose, multi-paradigm programming language")
     (description
      "TXR is a general-purpose, multi-paradigm programming language.  It
-- 
2.32.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-gnu-txr-Fix-license-installation.patch --]
[-- Type: text/x-patch; name=0002-gnu-txr-Fix-license-installation.patch, Size: 1702 bytes --]

From e19c5b58ec8cd829f3f2bfb900f079fa801fda31 Mon Sep 17 00:00:00 2001
From: "Paul A. Patience" <paul@apatience.com>
Date: Sat, 17 Jul 2021 23:19:31 -0400
Subject: [PATCH 2/3] gnu: txr: Fix license installation.

The install-license-files phase was installing LICENSE-CYG when it
shouldn't have, because that license applies only to TXR builds for
Microsoft Windows; and it was also ignoring the METALICENSE file.
However, TXR's Makefile was already installing the LICENSE and
METALICENSE files into share/txr (the datadir), so redirect them into
the appropriate doc directory and delete the install-license-files
phase.

* gnu/packages/lisp.scm (txr)[arguments]<#:phases>
{fix-license-installation}: New phase.
{install-license-files}: Delete phase.
---
 gnu/packages/lisp.scm | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/gnu/packages/lisp.scm b/gnu/packages/lisp.scm
index 7625c57b60..6a1cc4dcfd 100644
--- a/gnu/packages/lisp.scm
+++ b/gnu/packages/lisp.scm
@@ -931,6 +931,15 @@ the HTML documentation of TXR.")
        #:test-target "tests"
        #:phases
        (modify-phases %standard-phases
+         (add-after 'unpack 'fix-license-installation
+           (lambda* (#:key outputs #:allow-other-keys)
+             (substitute* "Makefile"
+               (("INSTALL(,.*LICENSE,.*)\\$\\(datadir\\)" _ match)
+                (string-append "INSTALL" match
+                               (assoc-ref outputs "out")
+                               "/share/doc/" ,name "-" ,version)))
+             #t))
+         (delete 'install-license-files)
          (add-after 'unpack 'inhibit-doc-syms-generation
            (lambda _
              (substitute* "genman.txr"
-- 
2.32.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0003-gnu-txr-Update-to-266.patch --]
[-- Type: text/x-patch; name=0003-gnu-txr-Update-to-266.patch, Size: 3033 bytes --]

From dfd4147f5992b561ac1a72bbc32c738724c855f9 Mon Sep 17 00:00:00 2001
From: "Paul A. Patience" <paul@apatience.com>
Date: Tue, 13 Jul 2021 19:38:12 -0400
Subject: [PATCH 3/3] gnu: txr: Update to 266.

* gnu/packages/lisp.scm (txr)[version]: Update to 266.
[arguments]<#:phases>{inhibit-doc-syms-generation}: Correct the path to
doc-syms.tl to its new location.
{fix-tests}: Fix another test.
{disable-failing-tests}: New phase.
---
 gnu/packages/lisp.scm | 22 +++++++++++++++++++---
 1 file changed, 19 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/lisp.scm b/gnu/packages/lisp.scm
index 6a1cc4dcfd..6ec59f5231 100644
--- a/gnu/packages/lisp.scm
+++ b/gnu/packages/lisp.scm
@@ -906,7 +906,7 @@ the HTML documentation of TXR.")
 (define-public txr
   (package
     (name "txr")
-    (version "263")
+    (version "266")
     (source
      (origin
        (method git-fetch)
@@ -915,7 +915,7 @@ the HTML documentation of TXR.")
              (commit (string-append "txr-" version))))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "14zaziymnbr2ld79x4h7sf88bzzzj82w3xpavmcx7mhwannb2swh"))))
+        (base32 "1k9mj3zaxdsylgnb8g6cq0cmp6420x7fp3nnsqdmds4gh8ib95wn"))))
     (build-system gnu-build-system)
     (native-inputs
      ;; Required to build the documentation.
@@ -947,7 +947,7 @@ the HTML documentation of TXR.")
                ;; stdlib/doc-syms.tl, which is anyway kept up to date with
                ;; each release (and is already compiled to stdlib/doc-syms.tlo
                ;; when genman.txr is run).
-               (("^@\\(output \"share/txr/stdlib/doc-syms\\.tl\"\\).*" line)
+               (("^@\\(output \"stdlib/doc-syms\\.tl\"\\).*" line)
                 (string-append "@(do (exit))\n" line)))
              #t))
          (add-after 'unpack 'fix-tests
@@ -955,6 +955,22 @@ the HTML documentation of TXR.")
              (substitute* (list "tests/017/realpath.tl"
                                 "tests/017/realpath.expected")
                (("/usr/bin") "/"))
+             (substitute* "tests/018/path-test.tl"
+               (("/bin") (dirname (which "sh"))))
+             #t))
+         (add-after 'unpack 'disable-failing-tests
+           (lambda _
+             (for-each delete-file
+                       ;; These tests fail because they call external commands
+                       ;; (like ls and cat) that don't run because the forked
+                       ;; child processes' execvp calls cannot find the
+                       ;; hardcoded /bin/sh.
+                       ;; (The tests pass when run manually in the directory
+                       ;; left by --keep-failed.)
+                       (map (lambda (f) (string-append "tests/" f))
+                            '("002/query-1.txr"
+                              "010/json.tl"
+                              "018/process.tl")))
              #t))
          (replace 'configure
            ;; ./configure is a hand-written script that can't handle standard
-- 
2.32.0