[PATCH 0/4] lei: finish wiring up pure-Perl stuff for Linux

[PATCH 0/4] lei: finish wiring up pure-Perl stuff for Linux

[Eric Wong]

Some cleanups here, too; but this hopefully makes lei more
accessible to users who don't have space/bandwidth to use
Inline::C.

I thought I sent these out weeks ago, but Real-Life, power outages
and other lei problems got in the way, I guess :x

Eric Wong (4):
  lei: clobber recvmsg buffer on errors
  syscall: more idiomatic cmsghdr space allocation
  lei: wire up pure Perl sendmsg/recvmsg for Linux users
  syscall: golf + more idiomatic buffer initialization

 lib/PublicInbox/IPC.pm           |  4 ++++
 lib/PublicInbox/LEI.pm           |  4 ++++
 lib/PublicInbox/LeiSelfSocket.pm |  1 +
 lib/PublicInbox/Syscall.pm       | 13 ++++++-------
 lib/PublicInbox/TestCommon.pm    | 16 ++++++++++++----
 t/lei-daemon.t                   |  6 +++++-
 t/lei-externals.t                |  1 +
 7 files changed, 33 insertions(+), 12 deletions(-)

[PATCH 1/4] lei: clobber recvmsg buffer on errors

[Eric Wong]

It will be necessary when we drop the Inline::C requirement
since the pure Perl Linux syscall recvmsg implementation.

This likely would've caused errors for Socket::MsgHdr users
without Inline::C, but I haven't tested it since it's a rare
configuration.
---
 lib/PublicInbox/LEI.pm           | 1 +
 lib/PublicInbox/LeiSelfSocket.pm | 1 +
 2 files changed, 2 insertions(+)

diff --git a/lib/PublicInbox/LEI.pm b/lib/PublicInbox/LEI.pm
index a7ddc21f..9ab91714 100644
--- a/lib/PublicInbox/LEI.pm
+++ b/lib/PublicInbox/LEI.pm
@@ -1149,6 +1149,7 @@ sub event_step {
 		if (scalar(@fds) == 1 && !defined($fds[0])) {
 			return if $! == EAGAIN;
 			die "recvmsg: $!" if $! != ECONNRESET;
+			$buf = '';
 			@fds = (); # for open loop below:
 		}
 		for (@fds) { open my $rfh, '+<&=', $_ }
diff --git a/lib/PublicInbox/LeiSelfSocket.pm b/lib/PublicInbox/LeiSelfSocket.pm
index dd64b6cf..860020cb 100644
--- a/lib/PublicInbox/LeiSelfSocket.pm
+++ b/lib/PublicInbox/LeiSelfSocket.pm
@@ -29,6 +29,7 @@ sub event_step {
 	if (scalar(@fds) == 1 && !defined($fds[0])) {
 		return if $!{EAGAIN};
 		die "recvmsg: $!" unless $!{ECONNRESET};
+		$buf = '';
 	} else { # just in case open so perl can auto-close them:
 		for (@fds) { open my $fh, '+<&=', $_ };
 	}

[PATCH 2/4] syscall: more idiomatic cmsghdr space allocation

[Eric Wong]

Since we know the space required under Linux, we can use the
same initialization as the Inline::C version instead of
hard-coding 256 as we do for Socket::MsgHdr.
---
 lib/PublicInbox/Syscall.pm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/Syscall.pm b/lib/PublicInbox/Syscall.pm
index e972aa41..cc282f9f 100644
--- a/lib/PublicInbox/Syscall.pm
+++ b/lib/PublicInbox/Syscall.pm
@@ -418,6 +418,7 @@ sub CMSG_ALIGN ($) { ($_[0] + SIZEOF_size_t - 1) & ~(SIZEOF_size_t - 1) }
 use constant CMSG_ALIGN_SIZEOF_cmsghdr => CMSG_ALIGN(SIZEOF_cmsghdr);
 sub CMSG_SPACE ($) { CMSG_ALIGN($_[0]) + CMSG_ALIGN_SIZEOF_cmsghdr }
 sub CMSG_LEN ($) { CMSG_ALIGN_SIZEOF_cmsghdr + $_[0] }
+use constant msg_controllen => CMSG_SPACE(10 * SIZEOF_int) + 16; # 10 FDs
 
 if (defined($SYS_sendmsg) && defined($SYS_recvmsg)) {
 no warnings 'once';
@@ -457,7 +458,7 @@ no warnings 'once';
 *recv_cmd4 = sub ($$$) {
 	my ($sock, undef, $len) = @_;
 	vec($_[1], ($len + 1) * 8, 1) = 0;
-	vec(my $cmsghdr = '', 256 * 8 - 1, 1) = 1;
+	my $cmsghdr = "\0" x msg_controllen; # 10 * sizeof(int)
 	my $iov = pack('P'.TMPL_size_t, $_[1], $len);
 	my $mh = pack('PL' . # msg_name, msg_namelen (socklen_t (U32))
 			BYTES_4_hole . # 4-byte padding on 64-bit
@@ -468,7 +469,7 @@ no warnings 'once';
 			@BYTES_4_hole,
 			$iov, 1, # msg_iov, msg_iovlen
 			$cmsghdr, # msg_control
-			256, # msg_controllen
+			msg_controllen,
 			0); # msg_flags
 	my $r = syscall($SYS_recvmsg, fileno($sock), $mh, 0);
 	return (undef) if $r < 0; # $! set

[PATCH 3/4] lei: wire up pure Perl sendmsg/recvmsg for Linux users

[Eric Wong]

This enables lei-daemon to work without Inline::C nor
Socket::MsgHdr installed.  Prior to this, only the `lei' client
was using the pure Perl implementation.  Either C implementation
is still marginally faster, however.
---
 lib/PublicInbox/IPC.pm        |  4 ++++
 lib/PublicInbox/LEI.pm        |  3 +++
 lib/PublicInbox/TestCommon.pm | 16 ++++++++++++----
 t/lei-daemon.t                |  6 +++++-
 t/lei-externals.t             |  1 +
 5 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/lib/PublicInbox/IPC.pm b/lib/PublicInbox/IPC.pm
index 8376275e..67e86a43 100644
--- a/lib/PublicInbox/IPC.pm
+++ b/lib/PublicInbox/IPC.pm
@@ -47,6 +47,10 @@ my $send_cmd = PublicInbox::Spawn->can('send_cmd4') // do {
 	require PublicInbox::CmdIPC4;
 	$recv_cmd //= PublicInbox::CmdIPC4->can('recv_cmd4');
 	PublicInbox::CmdIPC4->can('send_cmd4');
+} // do {
+	require PublicInbox::Syscall;
+	$recv_cmd //= PublicInbox::Syscall->can('recv_cmd4');
+	PublicInbox::Syscall->can('send_cmd4');
 };
 
 sub _get_rec ($) {
diff --git a/lib/PublicInbox/LEI.pm b/lib/PublicInbox/LEI.pm
index 9ab91714..4bd9183e 100644
--- a/lib/PublicInbox/LEI.pm
+++ b/lib/PublicInbox/LEI.pm
@@ -1277,6 +1277,9 @@ sub lazy_start {
 			require PublicInbox::CmdIPC4;
 			$send_cmd = PublicInbox::CmdIPC4->can('send_cmd4');
 			PublicInbox::CmdIPC4->can('recv_cmd4');
+		} // do {
+			$send_cmd = PublicInbox::Syscall->can('send_cmd4');
+			PublicInbox::Syscall->can('recv_cmd4');
 		};
 	}
 	$recv_cmd or die <<"";
diff --git a/lib/PublicInbox/TestCommon.pm b/lib/PublicInbox/TestCommon.pm
index ca732811..943dd2fa 100644
--- a/lib/PublicInbox/TestCommon.pm
+++ b/lib/PublicInbox/TestCommon.pm
@@ -1,4 +1,4 @@
-# Copyright (C) 2015-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 
 # internal APIs used only for tests
@@ -19,7 +19,8 @@ BEGIN {
 		run_script start_script key2sub xsys xsys_e xqx eml_load tick
 		have_xapian_compact json_utf8 setup_public_inboxes create_inbox
 		tcp_host_port test_lei lei lei_ok $lei_out $lei_err $lei_opt
-		test_httpd xbail require_cmd is_xdeeply tail_f);
+		test_httpd xbail require_cmd is_xdeeply tail_f
+		ignore_inline_c_missing);
 	require Test::More;
 	my @methods = grep(!/\W/, @Test::More::EXPORT);
 	eval(join('', map { "*$_=\\&Test::More::$_;" } @methods));
@@ -547,6 +548,11 @@ sub is_xdeeply ($$$) {
 	$ok;
 }
 
+sub ignore_inline_c_missing {
+	$_[0] = join('', grep(/\S/, grep(!/compilation aborted/,
+		grep(!/\bInline\b/, split(/^/m, $_[0])))));
+}
+
 sub test_lei {
 SKIP: {
 	my ($cb) = pop @_;
@@ -571,8 +577,10 @@ SKIP: {
 	$ENV{LANG} = $ENV{LC_ALL} = 'C';
 	my (undef, $fn, $lineno) = caller(0);
 	my $t = "$fn:$lineno";
-	state $lei_daemon = PublicInbox::Spawn->can('send_cmd4') ||
-				eval { require Socket::MsgHdr; 1 };
+	state $lei_daemon = PublicInbox::Spawn->can('send_cmd4') || do {
+			require PublicInbox::Syscall;
+			PublicInbox::Syscall->can('send_cmd4');
+		} || eval { require Socket::MsgHdr; 1 };
 	unless ($lei_daemon) {
 		skip('Inline::C unconfigured/missing '.
 '(mkdir -p ~/.cache/public-inbox/inline-c) OR Socket::MsgHdr missing',
diff --git a/t/lei-daemon.t b/t/lei-daemon.t
index b60c7ce6..e11105bc 100644
--- a/t/lei-daemon.t
+++ b/t/lei-daemon.t
@@ -1,5 +1,5 @@
 #!perl -w
-# Copyright (C) 2020-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 use strict; use v5.10.1; use PublicInbox::TestCommon;
 use Socket qw(AF_UNIX SOCK_SEQPACKET MSG_EOR pack_sockaddr_un);
@@ -8,12 +8,16 @@ test_lei({ daemon_only => 1 }, sub {
 	my $send_cmd = PublicInbox::Spawn->can('send_cmd4') // do {
 		require PublicInbox::CmdIPC4;
 		PublicInbox::CmdIPC4->can('send_cmd4');
+	} // do {
+		require PublicInbox::Syscall;
+		PublicInbox::Syscall->can('send_cmd4');
 	};
 	$send_cmd or BAIL_OUT 'started testing lei-daemon w/o send_cmd4!';
 
 	my $sock = "$ENV{XDG_RUNTIME_DIR}/lei/5.seq.sock";
 	my $err_log = "$ENV{XDG_RUNTIME_DIR}/lei/errors.log";
 	lei_ok('daemon-pid');
+	ignore_inline_c_missing($lei_err);
 	is($lei_err, '', 'no error from daemon-pid');
 	like($lei_out, qr/\A[0-9]+\n\z/s, 'pid returned') or BAIL_OUT;
 	chomp(my $pid = $lei_out);
diff --git a/t/lei-externals.t b/t/lei-externals.t
index fed57789..284be1b9 100644
--- a/t/lei-externals.t
+++ b/t/lei-externals.t
@@ -76,6 +76,7 @@ test_lei(sub {
 	my $config_file = "$home/.config/lei/config";
 	my $store_dir = "$home/.local/share/lei";
 	lei_ok 'ls-external', \'ls-external on fresh install';
+	ignore_inline_c_missing($lei_err);
 	is($lei_out.$lei_err, '', 'ls-external no output, yet');
 	ok(!-e $config_file && !-e $store_dir,
 		'nothing created by ls-external');

[PATCH 4/4] syscall: golf + more idiomatic buffer initialization

[Eric Wong]

While `vec' is useful for user-supplied buffers to avoid excess
memory traffic, but provides no benefit when we need to allocate
our own buffers as we do in nodatacow_fh, since Perl can't elide
memset(ptr, 0, len).  So just use the idiomatic `"\0" x $LEN' here.
---
 lib/PublicInbox/Syscall.pm | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/lib/PublicInbox/Syscall.pm b/lib/PublicInbox/Syscall.pm
index cc282f9f..22b779ad 100644
--- a/lib/PublicInbox/Syscall.pm
+++ b/lib/PublicInbox/Syscall.pm
@@ -390,12 +390,10 @@ sub rename_noreplace ($$) {
 	}
 }
 
-sub nodatacow_fh {
-	return if !defined($SYS_fstatfs);
-	my $buf = '';
-	vec($buf, 120 * 8 - 1, 1) = 0;
+sub nodatacow_fh ($) {
 	my ($fh) = @_;
-	syscall($SYS_fstatfs, fileno($fh), $buf) == 0 or
+	my $buf = "\0" x 120;
+	syscall($SYS_fstatfs // return, fileno($fh), $buf) == 0 or
 		return warn("fstatfs: $!\n");
 	my $f_type = unpack('l!', $buf); # statfs.f_type is a signed word
 	return if $f_type != 0x9123683E; # BTRFS_SUPER_MAGIC