From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-ASN:  
X-Spam-Status: No, score=-4.1 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no
	autolearn=ham autolearn_force=no version=3.4.2
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id 7DFA81F852;
	Wed, 21 Dec 2022 21:11:14 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org;
	s=selector1; t=1671657074;
	bh=gAVDIVupvGm2WP0WlAACpZWMp0rygPMEP9a9eAe1a0k=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=DbLYMt2uWKsINPjHjNAo5DyE9cZfeamtltRP97IVUt0n60rwLWdY2E1dgnDEAjIhn
	 4N9x5DF7gWqq37Zsxxep6dL/qVN9+nSsuh06URneKSSpeUHqJ4u0mwl+mZR5Y3N5Eg
	 FL9qRr2VodWjke2YOy+ZidCNMqlhWdmF28kDNwTg=
Date: Wed, 21 Dec 2022 21:11:14 +0000
From: Eric Wong <e@80x24.org>
To: Chris Brannon <chris@the-brannons.com>
Cc: meta@public-inbox.org
Subject: Re: public-inbox-convert hangs on systems using musl libc
Message-ID: <20221221211114.M412849@dcvr>
References: <875ye5m1wo.fsf@the-brannons.com>
 <20221221122102.M600156@dcvr>
 <871qosna30.fsf@the-brannons.com>
 <20221221194855.GA5179@dcvr>
 <87mt7glc23.fsf@the-brannons.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <87mt7glc23.fsf@the-brannons.com>
List-Id: <meta.public-inbox.org>

Chris Brannon <chris@the-brannons.com> wrote:
> Eric Wong <e@80x24.org> writes:
> 
> > Are you able to isolate whether $PIPE_BUFSIZ or MAX_INFLIGHT on
> > it's own fixes it?
> 
> MAX_INFLIGHT is sufficient.  In fact, $PIPE_BUFSIZ had no effect at all
> when I tried just now.

OK, good to know.  Thanks so far for your other testing and
reports.

> > Capping MAX_INFLIGHT to a smaller value is probably fine (maybe
> > 10 can work).
> 
> I tried with 10 just now, and that was fine also.

Oops, so POSIX::PIPE_BUF is actually 4096 on Linux.
(`perl -MPOSIX -E 'say POSIX::PIPE_BUF'` shows 4096 on the
Alpine cfarm machine)

But I'm still confused as to why the original code didn't
work on Linux since the pipe can't be smaller than 4096...

The following changes MAX_INFLIGHT to 23 for all platforms (and
explains things better).  I also wonder if you can find a
threshold for when things start failing (e.g. hardcoding 24
should fail on FreeBSD)

diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 882a9a4a..a1af776b 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -28,8 +28,10 @@ our $in_cleanup;
 our $RDTIMEO = 60_000; # milliseconds
 our $async_warn; # true in read-only daemons
 
-use constant MAX_INFLIGHT => (POSIX::PIPE_BUF * 3) /
-	65; # SHA-256 hex size + "\n" in preparation for git using non-SHA1
+# 512: POSIX PIPE_BUF minimum (see pipe(7))
+# 3: @$inflight is flattened [ $OID, $cb, $arg ]
+# 65: SHA-256 hex size + "\n" in preparation for git using non-SHA1
+use constant MAX_INFLIGHT => 512 * 3 / 65;
 
 my %GIT_ESC = (
 	a => "\a",