[bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0

[bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0

[Thiago Jung Bauermann via Guix-patches via]

Hello,

These patches fix a few issues with the current public-inbox package, add a
new package which is needed by version 1.8.0, and finally updates
public-inbox.

A few notes regarding the new package perl-socket-msghdr:

- It's based on the package generated by the CPAN importer. As such, the
  origin URL is on CPAN. There's an upstream repo on GitHub¹, but it
  appears inactive. The contents there are identical to the CPAN tarball,
  modulo the files META.json, META.yml and MANIFEST (the former two exist
  only on CPAN, and the latter is changed to include them) so I left the
  origin unchanged.

- There's a file called ‘ppport.h’ with 7k lines that was automatically
  generated by some tool called “Devel::PPPort”. Ideally we should delete
  it and regenerate it, but I haven't done that.

And regarding public-inbox 1.8.0:

- The diff from current version 1.6.1 has about 57k lines (including
  context). I didn't do even a cursory review of the changes.

- The new tool ‘lei’ depends on curl to work. I tried substituting its path
  directly in the source code, but for some reason that didn't work.
  Therefore I settled for adding curl to the PATH with ‘program-wrapper’.

- The tests are disabled because more changes are needed to enable them.
  I have them working in a local branch, but there's one patch to the
  public-inbox testsuite which I still need to cleanup and post upstream.
  If anyone is curious about it, I pushed the branch ‘public-inbox-updates’
  to my GitLab repo².

Thiago Jung Bauermann (3):
  gnu: public-inbox: Some packaging fixes
  gnu: Add perl-socket-msghdr
  gnu: public-inbox: Update to version 1.8.0

 gnu/packages/mail.scm | 38 +++++++++++++++++++++-----------------
 gnu/packages/perl.scm | 24 ++++++++++++++++++++++++
 2 files changed, 45 insertions(+), 17 deletions(-)

base-commit: 527952945c59f69665e40b6b416141554084e057

-- 

¹ https://github.com/FGasper/p5-Socket-MsgHdr
² https://gitlab.com/bauermann/guix/

[bug#55436] [PATCH 1/3] gnu: public-inbox: Some packaging fixes

[Thiago Jung Bauermann via Guix-patches via]

The file ‘script/public-inbox-compact’ doesn't call ‘xapian-compact’ directly
anymore, so the substitution in the ‘qualify-paths’ phase doesn't do anything.

Now there's a reference to ‘xapian-compact’ in ‘lib/PublicInbox/Xapcmd.pm’,
so substitute the path there.

And move xapian from native-inputs to inputs and use its input path in the
substitution, since ‘xapian-compact’ is called during normal execution of
public-inbox.

Also add ‘bash-minimal’ to the package's inputs, since it uses ‘wrap-program’.

Finally, take the opportunity to change build phases to not return #t
unconditionally, since that isn't necessary anymore.

* gnu/packages/mail.scm (public-inbox)[arguments]<#:phases>: Don't return #t
from each phase.
{qualify-paths}: Substitute ‘xapian-compact’ path in the correct file.  Use
‘search-input-file’ rather than ‘which’ to find it.
[native-inputs]: Move ‘xapian’ from here ...
[inputs]: ... to here. Add ‘bash-minimal’.
---
 gnu/packages/mail.scm | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/gnu/packages/mail.scm b/gnu/packages/mail.scm
index c4ccc53e84ef..0728d65d1d10 100644
--- a/gnu/packages/mail.scm
+++ b/gnu/packages/mail.scm
@@ -47,6 +47,7 @@
 ;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;; Copyright © 2022 Andrew Tropin <andrew@trop.in>
 ;;; Copyright © 2022 Justin Veilleux <terramorpha@cock.li>
+;;; Copyright © 2022 Thiago Jung Bauermann <bauermann@kolabnow.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -4101,12 +4102,12 @@ (define-public public-inbox
      '(#:phases
        (modify-phases %standard-phases
          (add-before 'configure 'qualify-paths
-           (lambda _
+           (lambda* (#:key inputs #:allow-other-keys)
              ;; Use absolute paths for 'xapian-compact'.
-             (let ((xapian-compact (which "xapian-compact")))
-               (substitute* "script/public-inbox-compact"
-                 (("xapian-compact") xapian-compact)))
-             #t))
+             (substitute* "lib/PublicInbox/Xapcmd.pm"
+               (("'xapian-compact'")
+                (format #f "'~a'" (search-input-file inputs
+                                                     "/bin/xapian-compact"))))))
          (add-before 'check 'pre-check
            (lambda _
              (substitute* "t/spawn.t"
@@ -4117,8 +4118,7 @@ (define-public public-inbox
              ;; XXX: This test fails due to zombie process is not reaped by
              ;; the builder.
              (substitute* "t/httpd-unix.t"
-               (("^SKIP: \\{") "SKIP: { skip('Guix');"))
-             #t))
+               (("^SKIP: \\{") "SKIP: { skip('Guix');"))))
          (add-after 'install 'wrap-programs
            (lambda* (#:key inputs outputs #:allow-other-keys)
              (let ((out (assoc-ref outputs "out")))
@@ -4133,14 +4133,13 @@ (define-public public-inbox
                     ;; perl module.
                     `("PATH" ":" prefix
                       (,(dirname (search-input-file inputs "/bin/git"))))))
-                (find-files (string-append out "/bin"))))
-             #t)))))
+                (find-files (string-append out "/bin")))))))))
     (native-inputs
-     (list xapian
-           ;; For testing.
+     (list ;; For testing.
            lsof openssl))
     (inputs
-     (list git
+     (list bash-minimal
+           git
            perl-dbd-sqlite
            perl-dbi
            perl-email-address-xs
@@ -4162,7 +4161,8 @@ (define-public public-inbox
            ;; ("highlight" ,highlight)
            ;; For testing.
            perl-ipc-run
-           perl-xml-feed))
+           perl-xml-feed
+           xapian))
     (home-page "https://public-inbox.org/README.html")
     (synopsis "Archive mailing lists in Git repositories")
     (description

[bug#55436] [PATCH 2/3] gnu: Add perl-socket-msghdr

[Thiago Jung Bauermann via Guix-patches via]

* gnu/packages/perl.scm (perl-socket-msghdr): New variable.
---
 gnu/packages/perl.scm | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/gnu/packages/perl.scm b/gnu/packages/perl.scm
index ba65cc6e6d8e..5fc46452440f 100644
--- a/gnu/packages/perl.scm
+++ b/gnu/packages/perl.scm
@@ -11849,3 +11849,27 @@ (define-public perl-data-sexpression
     (description "Data::SExpression parses Lisp S-Expressions into Perl data
 structures.")
     (license license:perl-license)))
+
+(define-public perl-socket-msghdr
+  (package
+    (name "perl-socket-msghdr")
+    (version "0.05")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append
+             "mirror://cpan/authors/id/F/FE/FELIPE/Socket-MsgHdr-"
+             version
+             ".tar.gz"))
+       (sha256
+        (base32 "0g3qa7xn0aqn417jfvnc0i3ksyqa7bnvws0wihldir6ywcaiql4n"))))
+    (build-system perl-build-system)
+    (home-page "https://metacpan.org/release/Socket-MsgHdr")
+    (synopsis "Perform advanced operations via sendmsg and recvmsg")
+    (description "Socket::MsgHdr provides advanced socket messaging operations
+via sendmsg and recvmsg.
+
+It also allows manipulating ancillary data or so-called control
+information (cmsghdr).  This ancillary data may be used for file descriptor
+passing, IPv6 operations, and a host of implemenation-specific extensions.")
+    (license license:perl-license)))

[bug#55436] [PATCH 3/3] gnu: public-inbox: Update to version 1.8.0

[Thiago Jung Bauermann via Guix-patches via]

Some fixes are needed in both the testsuite and the package itself before
tests can pass so disable them for now.

Also, the lei command needs curl to work, so add it to the PATH in
‘wrap-program’.

* gnu/packages/mail.scm (public-inbox)[version]: Change to 1.8.0.
[source]: Update hash.
[arguments]<#:tests?>: Set to #f.
[arguments]<#:phases>{wrap-programs}: Add curl to PATH.
[inputs]: Add curl and perl-socket-msghdr.
---
 gnu/packages/mail.scm | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/mail.scm b/gnu/packages/mail.scm
index 0728d65d1d10..781c3bd028ff 100644
--- a/gnu/packages/mail.scm
+++ b/gnu/packages/mail.scm
@@ -4087,7 +4087,7 @@ (define-public l2md
 (define-public public-inbox
   (package
     (name "public-inbox")
-    (version "1.6.1")
+    (version "1.8.0")
     (source
      (origin (method git-fetch)
              (uri (git-reference
@@ -4095,11 +4095,12 @@ (define-public public-inbox
                    (commit (string-append "v" version))))
              (sha256
               (base32
-               "0mlwnp5knr7rk9kv8grlh342wsq2193m22zs83cjn9p7x9r2x5f9"))
+               "0xni1l54v1z3p0zb52807maay0yqabp8jgf5iras5zmhgjyk3swz"))
              (file-name (git-file-name name version))))
     (build-system perl-build-system)
     (arguments
-     '(#:phases
+     '(#:tests? #f
+       #:phases
        (modify-phases %standard-phases
          (add-before 'configure 'qualify-paths
            (lambda* (#:key inputs #:allow-other-keys)
@@ -4132,13 +4133,15 @@ (define-public public-inbox
                     ;; 'git' is invoked in various files of the PublicInbox
                     ;; perl module.
                     `("PATH" ":" prefix
-                      (,(dirname (search-input-file inputs "/bin/git"))))))
+                      (,(dirname (search-input-file inputs "/bin/git"))
+                       ,(dirname (search-input-file inputs "/bin/curl"))))))
                 (find-files (string-append out "/bin")))))))))
     (native-inputs
      (list ;; For testing.
            lsof openssl))
     (inputs
      (list bash-minimal
+           curl
            git
            perl-dbd-sqlite
            perl-dbi
@@ -4151,6 +4154,7 @@ (define-public public-inbox
            perl-plack-middleware-reverseproxy
            perl-plack
            perl-search-xapian
+           perl-socket-msghdr
            perl-timedate
            perl-uri-escape
            perl-inline-c

[bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0

[Kyle Meyer]

Thiago Jung Bauermann via Guix-patches via writes:

> Hello,
>
> These patches fix a few issues with the current public-inbox package, add a
> new package which is needed by version 1.8.0, and finally updates
> public-inbox.

Thanks for working on this.

> [...]
> - The tests are disabled because more changes are needed to enable them.
>   I have them working in a local branch, but there's one patch to the
>   public-inbox testsuite which I still need to cleanup and post upstream.
>   If anyone is curious about it, I pushed the branch ‘public-inbox-updates’
>   to my GitLab repo².

Fwiw I applied your patches and tested it a bit with my local inboxes.
I didn't spot any issues.

--8<---------------cut here---------------start------------->8---

[guix env]% realpath $(which lei)
/gnu/store/564p55ajr8wlbml1rafs4pggjzpvmqaf-public-inbox-1.8.0/bin/lei

[guix env]% lei daemon-pid
199095

[guix env]% lei q -f ldjson -O guix-patches nq:lei d:30.days.ago.. | jq '.s' | sort | uniq
"[bug#54997] [PATCH 04/12] Add (guix least-authority)."
"[bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0"
"[bug#55436] [PATCH 3/3] gnu: public-inbox: Update to version 1.8.0"

[guix env]% lei daemon-kill
[guix env]% lei daemon-pid
199200

[guix env]% lei q -O https://lore.kernel.org/git -o mboxcl2:/tmp/t.mbox nq:pack d:20.days.ago..
# /gnu/store/i11i4xrx3gibdm012bqqhgrjs69qyia3-curl-7.79.1/bin/curl -Sf -s -d '' https://lore.kernel.org/git/?x=m&q=nq%3Apack+dt%3A20220426015402..
# https://lore.kernel.org/git/ 57/57
# 57 written to /tmp/t.mbox (57 matches)

[guix env]% public-inbox-httpd &
[1] 199585
# browsed around http://127.0.0.1:8080/guix-patches/ a bit
--8<---------------cut here---------------end--------------->8---

[bug#55436] [PATCH 3/3] gnu: public-inbox: Update to version 1.8.0

[Kyle Meyer]

Thiago Jung Bauermann via Guix-patches via writes:

> [...]
> [inputs]: Add curl and perl-socket-msghdr.
> ---
> [...]
>      (inputs
>       (list bash-minimal
> +           curl
>             git
>             perl-dbd-sqlite
>             perl-dbi

I think it'd be good to add libgit2 as well.

[bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0

[Thiago Jung Bauermann via Guix-patches via]

Kyle Meyer <kyle@kyleam.com> writes:

> Thiago Jung Bauermann via Guix-patches via writes:
>
>> Hello,
>>
>> These patches fix a few issues with the current public-inbox package, add a
>> new package which is needed by version 1.8.0, and finally updates
>> public-inbox.
>
> Thanks for working on this.

No problem. Sorry to take so long. There were a surprising number of
little issues along the way…

>> [...]
>> - The tests are disabled because more changes are needed to enable them.
>>   I have them working in a local branch, but there's one patch to the
>>   public-inbox testsuite which I still need to cleanup and post upstream.
>>   If anyone is curious about it, I pushed the branch ‘public-inbox-updates’
>>   to my GitLab repo².
>
> Fwiw I applied your patches and tested it a bit with my local inboxes.
> I didn't spot any issues.

<snip>

Awesome! Thank you for testing. You certainly use a lot more features of
the package than I do.

-- 
Thanks
Thiago

[bug#55436] [PATCH 3/3] gnu: public-inbox: Update to version 1.8.0

[Thiago Jung Bauermann via Guix-patches via]

Hello Kyle,

Thank you for the feedback.

Kyle Meyer <kyle@kyleam.com> writes:

> Thiago Jung Bauermann via Guix-patches via writes:
>
>> [...]
>> [inputs]: Add curl and perl-socket-msghdr.
>> ---
>> [...]
>>      (inputs
>>       (list bash-minimal
>> +           curl
>>             git
>>             perl-dbd-sqlite
>>             perl-dbi
>
> I think it'd be good to add libgit2 as well.

Ah, I wasn't aware of that. I'll wait a bit to see if there are more
comments and spin a v2 with the change.

-- 
Thanks
Thiago

[bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0

[Ludovic Courtès]

Hi,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

> Ah, I wasn't aware of that. I'll wait a bit to see if there are more
> comments and spin a v2 with the change.

I haven’t tested it but at first sight it LGTM.

Ludo’.

[bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0

[Thiago Jung Bauermann via Guix-patches via]

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

> Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
>
>> Ah, I wasn't aware of that. I'll wait a bit to see if there are more
>> comments and spin a v2 with the change.
>
> I haven’t tested it but at first sight it LGTM.

Thank you for your review!

I tried adding both libgit2 and pkg-config (which public-inbox uses to
find the library) to the package inputs, but unfortunately the resulting
package doesn't retain a reference to libgit2, so I don't think it
worked:

--8<---------------cut here---------------start------------->8---
popigai ⸤env⸥: ./pre-inst-env guix gc --references \
  $(./pre-inst-env guix build --no-grafts public-inbox) \
  | grep git
/gnu/store/afr6sh2vap4vdvj09s1q0rmcc3gl51rc-git-2.36.0
popigai ⸤env⸥: 
--8<---------------cut here---------------end--------------->8---

There's more work to be done to make Perl's Inline::C mechanism work in
Guix's environment. To be honest, I won't be able to work on that.

Would it be possible to commit these patches as they are now, and then
if someone can make public-inbox use libgit2 then that change can be
submitted as a separate patch?

-- 
Thanks
Thiago

bug#55436: [PATCH 0/3] Update public-inbox to 1.8.0

[Ludovic Courtès]

Hi Thiago,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

> Would it be possible to commit these patches as they are now,

Definitely, done!

> and then if someone can make public-inbox use libgit2 then that change
> can be submitted as a separate patch?

Sure, we can keep that as future work.  :-)

Thanks,
Ludo’.

[bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0

[Thiago Jung Bauermann via Guix-patches via]

Hello Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

> Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
>
>> Would it be possible to commit these patches as they are now,
>
> Definitely, done!

Awesome, thank you!

-- 
Thanks
Thiago