From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS6939 216.252.162.0/24 X-Spam-Status: No, score=-1.5 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,PP_MIME_FAKE_ASCII_TEXT, SPF_HELO_PASS,SPF_PASS,URIBL_BLACK shortcircuit=no autolearn=no autolearn_force=no version=3.4.2 Received: from hurricane.the-brannons.com (hurricane.the-brannons.com [216.252.162.32]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id DADCE1F852 for ; Wed, 21 Dec 2022 13:46:29 +0000 (UTC) Authentication-Results: dcvr.yhbt.net; dkim=pass (1024-bit key; unprotected) header.d=the-brannons.com header.i=@the-brannons.com header.b="HhumdYIl"; dkim-atps=neutral DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; s=hurricane; bh=2u2CkcvJoI tu1y/vwJTd4MYvi7bsG5J3Nw1/8AKdiwo=; h=in-reply-to:date:references: subject:cc:to:from; d=the-brannons.com; b=HhumdYIlIaQnnGVzyo0VngS//ogh cD7TTxMQKtouykkR9S068pqTo2M77TrAB/J2ejJWGTiXDdHV9/Yoq5BIrl42tH6cZYif3R hBzg2jRsMGYnMku9gaCF0q3qTT72S+0f1N3CHOJbndvmIkqHbE6PxiwK5McajYb62xmF90 djw= Received: by hurricane.the-brannons.com (OpenSMTPD) with ESMTPSA id 88199b8e (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO) auth=yes user=chris; Wed, 21 Dec 2022 05:46:28 -0800 (PST) From: Chris Brannon To: Eric Wong Cc: meta@public-inbox.org Subject: Re: public-inbox-convert hangs on systems using musl libc References: <875ye5m1wo.fsf@the-brannons.com> <20221221122102.M600156@dcvr> Autocrypt: addr=chris@the-brannons.com; prefer-encrypt=mutual; keydata= mDMEX37pCBYJKwYBBAHaRw8BAQdASkFrVm/MHGNpj7Oa+nq4b7eBg5LX5qnvS6ojH04ZE9+0LENo cmlzdG9waGVyIEJyYW5ub24gPGNocmlzQHRoZS1icmFubm9ucy5jb20+iJAEExYIADgWIQTFfzWT HzF7+QWPg09oJ1i8YNj4egUCX37pCAIbAwULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRBoJ1i8 YNj4esVsAQCJXDxhj5gZjcrNMmSH6Z+X5RDdbvsTRvS5wwQXM3g4kAEAjOrdx/fvZDiKL8dylDhU YUVVSkwWeOD2UtmYjm/QtwG4OARffukIEgorBgEEAZdVAQUBAQdAOMaQorDR8QUdM34/Q9mUyBvS 9WFK0Mb76dHyNkERx2kDAQgHiHgEGBYIACAWIQTFfzWTHzF7+QWPg09oJ1i8YNj4egUCX37pCAIb DAAKCRBoJ1i8YNj4eo/8AQC4AX6NMv65KQQU7s95T/1Y4E+Z7KTbvhKJeSjdDfj88AEA/qzPnDvl dLmqqNTqsfZCK81teOxSoNhRlhWcQ8QQ/wY= Date: Wed, 21 Dec 2022 05:46:27 -0800 In-Reply-To: <20221221122102.M600156@dcvr> (Eric Wong's message of "Wed, 21 Dec 2022 12:21:02 +0000") Message-ID: <871qosna30.fsf@the-brannons.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" List-Id: --=-=-= Content-Type: text/plain Eric Wong writes: > Do you know which pipes are which? "lsof -p $PID +E" can help > with connectivity checking, as can script/dtas-graph in > https://80x24.org/dtas.git if you have Graph::Easy Yes. I'm attaching my lsof output and a typescript. The processes of interest here are 4849 public-inbox-convert and 4879 git cat-file. PID 4849's FD 11 is the write end of a pipe, with 4879's stdin as the read end. PID 4849's FD 12 is the read end of a pipe, with 4879's stdout as the write end. At the point of the hang, 4849 is trying to write a SHA1 to FD 11, while 4879 is writing an email message to its stdout. > Some shots in the dark: > > 2. Tweak $PIPE_BUFSIZ and/or MAX_INFLIGHT to smaller values. e.g. > > diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm > index 882a9a4a..ec40edd7 100644 > --- a/lib/PublicInbox/Git.pm > +++ b/lib/PublicInbox/Git.pm > @@ -23,13 +23,12 @@ use Carp qw(croak carp); > use Digest::SHA (); > use PublicInbox::DS qw(dwaitpid); > our @EXPORT_OK = qw(git_unquote git_quote); > -our $PIPE_BUFSIZ = 65536; # Linux default > +our $PIPE_BUFSIZ = 4096; # Linux default > our $in_cleanup; > our $RDTIMEO = 60_000; # milliseconds > our $async_warn; # true in read-only daemons > > -use constant MAX_INFLIGHT => (POSIX::PIPE_BUF * 3) / > - 65; # SHA-256 hex size + "\n" in preparation for git using non-SHA1 > +use constant MAX_INFLIGHT => 4; > > my %GIT_ESC = ( > a => "\a", > This right here seems to have fixed it, when testing locally. PS. Thank you for that lsof command. I've never used lsof in that way; I'll have to add that to my *nix debugging toolbelt. -- Chris --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=lsofout.txt COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME screen 4028 chris 11u CHR 5,2 0t0 86 /dev/ptmx ->/dev/pts/7 2214,mksh,0u 2214,mksh,1u 2214,mksh,2u 4849,public-in,0u 4849,public-in,1u 4849,public-in,2u 4874,shard[0],0u 4874,shard[0],1u 4874,shard[0],2u 4879,git,2u public-in 4849 chris cwd DIR 0,42 412 17211 /files/cmb-files/repos public-in 4849 chris rtd DIR 0,26 28 34 / public-in 4849 chris txt REG 0,26 14144 239824 /usr/bin/perl public-in 4849 chris mem REG 0,26 2969912 227085 /usr/lib/libstdc++.so.6.0.30 public-in 4849 chris mem REG 0,26 100328 564900 /usr/lib/libz.so.1.2.13 public-in 4849 chris mem REG 0,26 30760 399826 /usr/lib/libuuid.so.1.3.0 public-in 4849 chris mem REG 0,26 121208 226756 /usr/lib/libgcc_s.so.1 public-in 4849 chris mem REG 0,26 2234600 563318 /usr/lib/libxapian.so.30.12.2 public-in 4849 chris mem REG 0,26 500184 219769 /usr/lib/perl5/vendor_perl/auto/Search/Xapian/Xapian.so public-in 4849 chris mem REG 0,26 132928 239703 /usr/lib/perl5/core_perl/auto/Compress/Raw/Zlib/Zlib.so public-in 4849 chris mem REG 0,26 1351488 347561 /usr/lib/libsqlite3.so.0.8.6 public-in 4849 chris mem REG 0,26 180112 239928 /usr/lib/perl5/vendor_perl/auto/DBD/SQLite/SQLite.so public-in 4849 chris mem REG 0,26 138264 240017 /usr/lib/perl5/vendor_perl/auto/DBI/DBI.so public-in 4849 chris mem REG 0,26 2310 4758 /usr/share/zoneinfo/PST8PDT public-in 4849 chris mem REG 0,26 18168 239686 /usr/lib/perl5/core_perl/auto/MIME/Base64/Base64.so public-in 4849 chris mem REG 0,26 657216 239665 /usr/lib/perl5/core_perl/auto/re/re.so public-in 4849 chris mem REG 0,26 116640 239674 /usr/lib/perl5/core_perl/auto/Storable/Storable.so public-in 4849 chris mem REG 0,26 46920 239702 /usr/lib/perl5/core_perl/auto/Encode/Encode.so public-in 4849 chris mem REG 0,26 39608 218906 /usr/lib/perl5/vendor_perl/auto/Email/Address/XS/XS.so public-in 4849 chris mem REG 0,26 46920 239675 /usr/lib/perl5/core_perl/auto/Socket/Socket.so public-in 4849 chris mem REG 0,26 59128 239687 /usr/lib/perl5/core_perl/auto/List/Util/Util.so public-in 4849 chris mem REG 0,26 46840 238715 /usr/lib/perl5/core_perl/auto/Digest/SHA/SHA.so public-in 4849 chris mem REG 0,26 30456 239671 /usr/lib/perl5/core_perl/auto/Time/HiRes/HiRes.so public-in 4849 chris mem REG 0,26 18168 239744 /usr/lib/perl5/core_perl/auto/Cwd/Cwd.so public-in 4849 chris mem REG 0,26 30536 239695 /usr/lib/perl5/core_perl/auto/File/Glob/Glob.so public-in 4849 chris mem REG 0,26 22264 239689 /usr/lib/perl5/core_perl/auto/IO/IO.so public-in 4849 chris mem REG 0,26 108360 239682 /usr/lib/perl5/core_perl/auto/POSIX/POSIX.so public-in 4849 chris mem REG 0,26 22344 239697 /usr/lib/perl5/core_perl/auto/Fcntl/Fcntl.so public-in 4849 chris mem REG 0,26 3789544 239902 /usr/lib/libperl.so.5.36.0 public-in 4849 chris mem REG 0,26 4251720 228062 /usr/lib/libc.so public-in 4849 chris 0u CHR 136,7 0t0 10 /dev/pts/7 4028,screen,11u public-in 4849 chris 1u CHR 136,7 0t0 10 /dev/pts/7 4028,screen,11u public-in 4849 chris 2u CHR 136,7 0t0 10 /dev/pts/7 4028,screen,11u public-in 4849 chris 3w CHR 1,3 0t0 4 /dev/null public-in 4849 chris 4uW REG 0,42 0 2074958 /files/cmb-files/repos/metanew/inbox.lock public-in 4849 chris 5ur REG 0,42 24576 2074742 /files/cmb-files/repos/metanew/msgmap.sqlite3 public-in 4849 chris 6w FIFO 0,12 0t0 26905684 pipe 4874,shard[0],5r public-in 4849 chris 7r FIFO 0,12 0t0 26905685 pipe 4874,shard[0],8w public-in 4849 chris 8ur REG 0,42 69632 2074960 /files/cmb-files/repos/metanew/xap15/over.sqlite3 public-in 4849 chris 9u REG 0,49 461301 410306 /tmp/PerlIO_LaapMm (deleted) public-in 4849 chris 10u REG 0,42 24576 2074986 /files/cmb-files/repos/metanew/mm_tmp-4849-i60n public-in 4849 chris 11w FIFO 0,12 0t0 26905179 pipe 4879,git,0r public-in 4849 chris 12r FIFO 0,12 0t0 26905180 pipe 4879,git,1w public-in 4849 chris 13u REG 0,42 16928 2074743 /files/cmb-files/repos/metanew/msgmap.sqlite3-journal public-in 4849 chris 14u REG 0,42 66176 2074961 /files/cmb-files/repos/metanew/xap15/over.sqlite3-journal shard[0] 4874 chris 5r FIFO 0,12 0t0 26905684 pipe 4849,public-in,6w shard[0] 4874 chris 8w FIFO 0,12 0t0 26905685 pipe 4849,public-in,7r git 4879 chris 0r FIFO 0,12 0t0 26905179 pipe 4849,public-in,11w git 4879 chris 1w FIFO 0,12 0t0 26905180 pipe 4849,public-in,12r --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=typescript Content-Transfer-Encoding: 8bit Script started on 2022-12-21 05:12:15-08:00 [TERM="screen.linux" TTY="/dev/pts/10" COLUMNS="80" LINES="30"] chris@beast:/home/chris $ doas strace -p 4849 doas (chris@beast) password: strace: Process 4849 attached write(11, "4187293ba385ec204df84742b7197f4d"..., 41^Cstrace: Process 4849 detached Interrupt chris@beast:/home/chris $ doas strace -p 4879 strace: Process 4879 attached write(1, "Return-Path: \nX-Spa"..., 3793^Cstrace: Process 4879 detached Interrupt chris@beast:/home/chris $ exit Script done on 2022-12-21 05:12:43-08:00 [COMMAND_EXIT_CODE="130"] --=-=-=--