[PATCH] git: cap MAX_INFLIGHT value to POSIX minimum

[Eric Wong <e@80x24.org>]

Chris Brannon <chris@the-brannons.com> wrote:
> Eric Wong <e@80x24.org> writes:
> 
> > The following changes MAX_INFLIGHT to 23 for all platforms (and
> > explains things better).  I also wonder if you can find a
> > threshold for when things start failing (e.g. hardcoding 24
> > should fail on FreeBSD)
> 
> I did a binary search.  On my Linux system with musl, deadlocks start at
> MAX_INFLIGHT >= 115.

Thanks.  I'll push the following out and intend to keep it at <= 23
going forward.  However, it still worries me greatly that I have
no explanation to why MAX_INFLIGHT >= 115 fails.  This stuff is
my bread and butter, really...

I'll wait on a cfarm admins to test.

Also, can I assume just running 'public-inbox-index' on any
freshly-cloned inbox also fails for you independently of
-convert?

------------8<----------
Subject: [PATCH] git: cap MAX_INFLIGHT value to POSIX minimum

This ensures we get consistent pipelining behavior across
platforms.  Furthermore, a smaller value is probably more
reasonable since "git cat-file" can usually outpace indexing and
lower values allow us to react to user interaction (e.g. Ctrl-C)
more quickly.

The previous value based on Linux PIPE_BUF (4096) allowed a
value of 189 which worked fine on non-musl Linux systems, but
failed on musl-based Void and Alpine Linux.  Mysteriously, this
works on musl up to a value of 114 and starts locking up at 115.
The reason for this failure is currently unexplained and will
hopefully be discovered soon.

Regardless, capping the value to 23 based on the universal
PIPE_BUF minimum (512) seems reasonable, anyways.

Reported-by: Chris Brannon <chris@the-brannons.com>
Tested-by: Chris Brannon <chris@the-brannons.com>
Link: https://public-inbox.org/meta/87edssl7u0.fsf@the-brannons.com/T/
---
 lib/PublicInbox/Git.pm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 882a9a4a..a1af776b 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -28,8 +28,10 @@ our $in_cleanup;
 our $RDTIMEO = 60_000; # milliseconds
 our $async_warn; # true in read-only daemons
 
-use constant MAX_INFLIGHT => (POSIX::PIPE_BUF * 3) /
-	65; # SHA-256 hex size + "\n" in preparation for git using non-SHA1
+# 512: POSIX PIPE_BUF minimum (see pipe(7))
+# 3: @$inflight is flattened [ $OID, $cb, $arg ]
+# 65: SHA-256 hex size + "\n" in preparation for git using non-SHA1
+use constant MAX_INFLIGHT => 512 * 3 / 65;
 
 my %GIT_ESC = (
 	a => "\a",