From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-ASN:  
X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00
	shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id 369C31F463;
	Wed, 11 Sep 2019 17:12:40 +0000 (UTC)
Date: Wed, 11 Sep 2019 17:12:40 +0000
From: Eric Wong <e@80x24.org>
To: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Cc: meta@public-inbox.org
Subject: Re: trying to figure out 100% CPU usage in nntpd...
Message-ID: <20190911171239.xkclzeszibtmlxl3@dcvr>
References: <20190908104518.11919-1-e@80x24.org>
 <20190908105243.GA15983@dcvr>
 <20190909100500.GA9452@pure.paranoia.local>
 <20190909175340.u5aq4ztfzukko7zb@dcvr>
 <20190910083820.GA8018@pure.paranoia.local>
 <20190910181224.urhyoo6av7mhjs67@dcvr>
 <20190911094435.GB3548@pure.paranoia.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20190911094435.GB3548@pure.paranoia.local>
List-Id: <meta.public-inbox.org>

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Tue, Sep 10, 2019 at 06:12:24PM +0000, Eric Wong wrote:
> > Any git-http-backend stuck from people fetching/cloning?
> 
> No, all git processes seem to exit cleanly on both ends.
> 
> > This is -httpd writing to varnish, still, right?
> 
> We bypass varnish for git requests, since this is not generally useful.
> Nginx goes straight to public-inbox-httpd for those.
> 
> I did run some updates on lore.kernel.org on Thursday, including kernel
> (3.10.0-957.27.2), nginx (1.16.1) and public-inbox updates. For the
> latter, it went from f4f0a3be to what was latest master at the time
> (d327141c).

I also noticed "tcp: remove empty skb from write queue in error cases"
(64a2a93ba3856d16765e5c8a00c133fa8750879a) which is likely to affect
public-inbox using edge-triggered epoll.  Not sure if
Linux 3.10.0-957.27.2 has that backported...

Also, what nginx buffer settings are you using?  In particular,
is proxy_buffering off or on?  Also, does that version of nginx
do persistent connections to backends (and if so, is it enabled)?

It doesn't seem pipe-user-pages related, but what I wrote could
definitely be a problem in the future with the number if inboxes
you have and I should be able to mitigate that by alternates
and sharing a cat-file process.