[PATCH] doc: INSTALL: note OpenBSD xapian-bindings-perl package

[PATCH] doc: INSTALL: note OpenBSD xapian-bindings-perl package

[Eric Wong]

The `xapian-bindings-perl' package contains the Xapian.pm
SWIG bindings, but doesn't adhere to the existing convention
of naming system packages after the Perl package name itself
using: "p5-${\($Perl_package_name =~ s/::/-/gr)}".

Reported-by: Štěpán Němec <stepnem@smrk.net>
Link: https://public-inbox.org/meta/20230824132323+0200.507058-stepnem@smrk.net/
---
 INSTALL | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/INSTALL b/INSTALL
index 65032d11..91e590ce 100644
--- a/INSTALL
+++ b/INSTALL
@@ -68,7 +68,8 @@ Numerous optional modules are likely to be useful as well:
                                    (for v2, IMAP, NNTP, or gzipped mboxes)
 
 - Search::Xapian or Xapian(.pm)    deb: libsearch-xapian-perl
-                                   pkg: p5-Search-Xapian OR p5-Xapian
+                                   pkg: p5-Search-Xapian OR p5-Xapian (FreeBSD)
+                                        xapian-bindings-perl (OpenBSD)
                                    rpm: perl-Search-Xapian
                                    (HTTP and IMAP search)
 

Re: [PATCH] doc: INSTALL: note OpenBSD xapian-bindings-perl package

[Štěpán Němec]

On Thu, 24 Aug 2023 19:35:09 +0000
Eric Wong wrote:

> The `xapian-bindings-perl' package contains the Xapian.pm
> SWIG bindings, but doesn't adhere to the existing convention
> of naming system packages after the Perl package name itself

I'm not familiar with the history of Xapian upstream or OpenBSD
packaging, but I'm guessing the main reason for the lack of "adherence"
here is that the package is being built together with the other language
bindings from the Xapian tarball (as opposed to, say, CPAN):

https://github.com/openbsd/ports/blob/master/databases/xapian-bindings/Makefile

Tangentially related: I've taken this opportunity to review my (OpenBSD)
install script and found one other package I needed to install that
isn't mentioned in INSTALL, namely, p5-IO-Socket-SSL.

I didn't report it or give it too much thought at the time, as I was
under the impression that the INSTALL file wasn't quite up-to-date or
definitive, anyway (that impression being corroborated by my previous
experience installing public-inbox on Arch Linux: IIRC there were some
dependency surprises there, too, unfortunately I didn't keep any notes
then), but seeing you touching these parts now I figured I'd mention it.

-- 
Štěpán

Re: [PATCH] doc: INSTALL: note OpenBSD xapian-bindings-perl package

[Eric Wong]

Štěpán Němec <stepnem@smrk.net> wrote:
> Tangentially related: I've taken this opportunity to review my (OpenBSD)
> install script and found one other package I needed to install that
> isn't mentioned in INSTALL, namely, p5-IO-Socket-SSL.

Is there a test which fails without it?  It should be optional
(warnings emitted, functionality degrading gracefully and tests
automatically skipped).

It's pulled in by Mail-IMAPClient and a bunch of other modules
I have installed, so I can't easily test it atm, but I'll put
a note for it in INSTALL.

On a side note, running OpenBSD i386 via QEMU on amd64 Linux is
very slow.  Hoping OpenBSD amd64 will be faster...

> I didn't report it or give it too much thought at the time, as I was
> under the impression that the INSTALL file wasn't quite up-to-date or
> definitive, anyway (that impression being corroborated by my previous
> experience installing public-inbox on Arch Linux: IIRC there were some
> dependency surprises there, too, unfortunately I didn't keep any notes
> then), but seeing you touching these parts now I figured I'd mention it.

INSTALL ought to be definitive and up-to-date as there's
optional and non-Perl dependencies which (AFAIK) can't be
specified in Makefile.PL

Any help you can offer would be appreciated, since I'm usually
in a Debian bubble :)

Re: [PATCH] doc: INSTALL: note OpenBSD xapian-bindings-perl package

[Štěpán Němec]

On Thu, 24 Aug 2023 21:51:03 +0000
Eric Wong wrote:

> Štěpán Němec <stepnem@smrk.net> wrote:
>> Tangentially related: I've taken this opportunity to review my (OpenBSD)
>> install script and found one other package I needed to install that
>> isn't mentioned in INSTALL, namely, p5-IO-Socket-SSL.
>
> Is there a test which fails without it?  It should be optional
> (warnings emitted, functionality degrading gracefully and tests
> automatically skipped).

Yeah, false alarm, sorry.  I now tried a clean install, and it (what I
want, i.e., NNTP and HTTP) does work without it.  I guess I only needed
it because I initially tested public-inbox handling TLS itself?  I later
thought better of it anyway and put it behind a proxy.

As for the tests, I have never tried running any (was too scared, sorry,
there seems to be way too many of them for me to audit ATM, and I didn't
feel like digging through the makefile maze to understand what the test
targets really do, either), but now that I tried in the throwaway
install, it only got this far:

t/address.t .................. ok
t/admin.t .................... ok
t/altid.t .................... ok
t/altid_v2.t ................. ok
t/cgi.t ...................... ok
t/cindex.t ................... 37/?

Has been stuck there for a Real Long Time now (half an hour or so).

(That's "make test" with latest master.)

Both the HTTP and NNTP servers do work, though, so I guess it's
something with the test or the cindex functionality (which I know next
to nothing about).

All info I have for you is a stuck perl process after ^Cing out of the
test run:

USER       PID %CPU %MEM   VSZ   RSS TT  STAT   STARTED       TIME COMMAND
pi       18600  0.0  6.5 53884 66800 p1  I       3:51PM    0:04.99 perl: -cindex -q --prune -d /tmp/pi-cindex-18600-9QG_/ext (perl)

Attaching to the process didn't provide anything (nothing from ktrace,
continuing after attaching with gdb got stuck, too, killing the gdb
process finally got rid of the perl one as well).

> On a side note, running OpenBSD i386 via QEMU on amd64 Linux is
> very slow.  Hoping OpenBSD amd64 will be faster...

My experience with QEMU is limited, but running amd64 OpenBSD on an
amd64 Linux host seemed OK last time I tried (2 years ago?).  Running
aarch64 Linux on amd64 Linux was unusably slow, even on a recentish
laptop.

BTW, if you're up for a slightly Kafkaesque experience, you can get a
few free VPSes at Oracle Cloud, {amd,arm}64.  That's where the test
install described above is running.  I've heard varying experiences and
wouldn't put there anything I'm not prepared to lose, but I have to
admit I've also been running a Tor relay and a DNS server there (both
recursive and authoritative backup) for almost a year now, and, if
anything, it's been more stable than my paid VPS at Vultr.  I could also
give you access to the test install at least (OpenBSD 7.3).

>> I didn't report it or give it too much thought at the time, as I was
>> under the impression that the INSTALL file wasn't quite up-to-date or
>> definitive, anyway (that impression being corroborated by my previous
>> experience installing public-inbox on Arch Linux: IIRC there were some
>> dependency surprises there, too, unfortunately I didn't keep any notes
>> then), but seeing you touching these parts now I figured I'd mention it.
>
> INSTALL ought to be definitive and up-to-date as there's
> optional and non-Perl dependencies which (AFAIK) can't be
> specified in Makefile.PL

Noted.  (In my defense, there _is_ "TODO: this still needs to be
documented better" in the INSTALL file.)

-- 
Štěpán

Re: [PATCH] doc: INSTALL: note OpenBSD xapian-bindings-perl package

[Eric Wong]

Štěpán Němec <stepnem@smrk.net> wrote:
> On Thu, 24 Aug 2023 21:51:03 +0000
> Eric Wong wrote:
> > Štěpán Němec <stepnem@smrk.net> wrote:
> >> Tangentially related: I've taken this opportunity to review my (OpenBSD)
> >> install script and found one other package I needed to install that
> >> isn't mentioned in INSTALL, namely, p5-IO-Socket-SSL.
> >
> > Is there a test which fails without it?  It should be optional
> > (warnings emitted, functionality degrading gracefully and tests
> > automatically skipped).
> 
> Yeah, false alarm, sorry.  I now tried a clean install, and it (what I
> want, i.e., NNTP and HTTP) does work without it.  I guess I only needed
> it because I initially tested public-inbox handling TLS itself?  I later
> thought better of it anyway and put it behind a proxy.

Yeah, though I'm not comfortable recommending the popular open core
reverse proxy so I intend to implement one in Perl5 at some point...
(I currently use a GPL-3+ Ruby reverse proxy, but Ruby introduces
new incompatibilities every year so I'm moving away from it).

> t/cindex.t ................... 37/?
> 
> Has been stuck there for a Real Long Time now (half an hour or so).
> 
> (That's "make test" with latest master.)
> 
> Both the HTTP and NNTP servers do work, though, so I guess it's
> something with the test or the cindex functionality (which I know next
> to nothing about).

-cindex is a new feature and still under development and the
good news is I'm able to reproduce it.  I will try to fix it soon.

> All info I have for you is a stuck perl process after ^Cing out of the
> test run:
> 
> USER       PID %CPU %MEM   VSZ   RSS TT  STAT   STARTED       TIME COMMAND
> pi       18600  0.0  6.5 53884 66800 p1  I       3:51PM    0:04.99 perl: -cindex -q --prune -d /tmp/pi-cindex-18600-9QG_/ext (perl)
> 
> Attaching to the process didn't provide anything (nothing from ktrace,
> continuing after attaching with gdb got stuck, too, killing the gdb
> process finally got rid of the perl one as well).

Yeah, ktrace doesn't seem to tell the actual syscall it's stuck on
(IIRC I had the same experience on FreeBSD)

> > On a side note, running OpenBSD i386 via QEMU on amd64 Linux is
> > very slow.  Hoping OpenBSD amd64 will be faster...
> 
> My experience with QEMU is limited, but running amd64 OpenBSD on an
> amd64 Linux host seemed OK last time I tried (2 years ago?).  Running
> aarch64 Linux on amd64 Linux was unusably slow, even on a recentish
> laptop.

Yeah, I just setup OpenBSD amd64 on a amd64 Linux host and it seems fine.
I'll try to do more work on it (as I do in FreeBSD) to iron out
portability problems.

<snip>  Thanks for the VPS note though I avoid doing anything which
has even slightest possibility of being interpreted as a commercial
endorsement (even if it's "free as in beer").

Re: [PATCH] doc: INSTALL: note OpenBSD xapian-bindings-perl package

[Eric Wong]

Eric Wong <e@80x24.org> wrote:
> Štěpán Němec <stepnem@smrk.net> wrote:
> > t/cindex.t ................... 37/?
> > 
> > Has been stuck there for a Real Long Time now (half an hour or so).
> > 
> -cindex is a new feature and still under development and the
> good news is I'm able to reproduce it.  I will try to fix it soon.

One problem is the lack of p5-Inline/p5-Inline-C, nor Socket::MsgHdr
being installed (none are needed on Linux); so I'll have to make
that part work with pure Perl...

The other thing is if Inline/Inline::C/Socket::MsgHdr is installed
on OpenBSD; AF_UNIX + SOCK_SEQPACKET + MSG_EOR behaves different on
OpenBSD than it does on FreeBSD or Linux.  So maybe it's a bug in
OpenBSD, I'm misunderstanding MSG_EOR, or both:

https://marc.info/?i=20230826020759.M335788@dcvr

In any case, my connectivity to my OpenBSD VM is spotty but
maybe s/MSG_EOR/0/g for all send*() calls is the way to go...