unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 09/13] ipc: simplify partial sendmsg fallback
Date: Thu,  9 Nov 2023 10:09:42 +0000	[thread overview]
Message-ID: <20231109100946.1440611-10-e@80x24.org> (raw)
In-Reply-To: <20231109100946.1440611-1-e@80x24.org>

In the rare case sendmsg(2) isn't able to send the full amount
(due to buffers >=2GB on Linux), use print + (autodie)close
to send the remainder and retry on EINTR.  `substr' should be
able to avoid a large malloc via offsets and CoW on modern Perl.
---
 lib/PublicInbox/IPC.pm | 13 +++----------
 t/ipc.t                |  7 +++++++
 2 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/lib/PublicInbox/IPC.pm b/lib/PublicInbox/IPC.pm
index 3292d960..a5cae6f2 100644
--- a/lib/PublicInbox/IPC.pm
+++ b/lib/PublicInbox/IPC.pm
@@ -10,7 +10,7 @@
 package PublicInbox::IPC;
 use v5.12;
 use parent qw(Exporter);
-use autodie qw(fork pipe read socketpair sysread);
+use autodie qw(close fork pipe read socketpair sysread);
 use Carp qw(croak);
 use PublicInbox::DS qw(awaitpid);
 use PublicInbox::Spawn;
@@ -266,15 +266,8 @@ sub stream_in_full ($$$) {
 			0) // croak "sendmsg: $!";
 	undef $r;
 	$n = $send_cmd->($w, $fds, $buf, 0) // croak "sendmsg: $!";
-	while ($n < length($buf)) {
-		my $x = syswrite($w, $buf, length($buf) - $n, $n);
-		if (!defined($n)) {
-			next if $!{EINTR};
-			croak "syswrite: $!";
-		}
-		$x or croak "syswrite wrote 0 bytes";
-		$n += $x;
-	}
+	print $w substr($buf, $n) if $n < length($buf); # need > 2G on Linux
+	close $w; # autodies
 }
 
 sub wq_io_do { # always async
diff --git a/t/ipc.t b/t/ipc.t
index 519ef089..23ae2e7b 100644
--- a/t/ipc.t
+++ b/t/ipc.t
@@ -132,6 +132,13 @@ for my $t ('worker', 'worker again') {
 		$exp = sha1_hex($bigger)."\n";
 		is(readline($rb), $exp, "SHA WQWorker limit ($t)");
 	}
+	SKIP: {
+		$ENV{TEST_EXPENSIVE} or skip 'TEST_EXPENSIVE not set', 1;
+		my $bigger = $big x 75000; # over 2G to trigger partial sendmsg
+		$ipc->wq_io_do('test_sha', [ $wa, $wb ], $bigger);
+		my $exp = sha1_hex($bigger)."\n";
+		is(readline($rb), $exp, "SHA WQWorker sendmsg limit ($t)");
+	}
 }
 
 # wq_io_do works across fork (siblings can feed)

  parent reply	other threads:[~2023-11-09 10:09 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-09 10:09 [PATCH 00/13] misc error handling stuff and simplifications Eric Wong
2023-11-09 10:09 ` [PATCH 01/13] lei_xsearch: put query in process title for debugging Eric Wong
2023-11-09 10:09 ` [PATCH 02/13] lei: use cached $daemon_pid when possible Eric Wong
2023-11-09 10:09 ` [PATCH 03/13] lei: reuse FDs atfork and close explicitly Eric Wong
2023-11-09 10:09 ` [PATCH 04/13] lei_up: use v5.12 Eric Wong
2023-11-09 10:09 ` [PATCH 05/13] net_nntp_socks: more comments around how it works Eric Wong
2023-11-09 10:09 ` [PATCH 06/13] lei ls-mail-source: gracefully handle network failures Eric Wong
2023-11-09 10:09 ` [PATCH 07/13] net: retry on EINTR and check for {quit} flag Eric Wong
2023-11-09 10:09 ` [PATCH 08/13] lei_mirror: note missing local manifests are non-fatal Eric Wong
2023-11-09 10:09 ` Eric Wong [this message]
2023-11-09 10:09 ` [PATCH 10/13] lei_input: always close single `eml' inputs Eric Wong
2023-11-09 10:09 ` [PATCH 11/13] xapcmd: get rid of scalar wantarray popen_rd Eric Wong
2023-11-09 10:09 ` [PATCH 12/13] lei: get rid of autoreap usage Eric Wong
2023-11-09 10:09 ` [PATCH 13/13] spawn: get rid of wantarray popen_rd/popen_wr Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231109100946.1440611-10-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).