From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-ASN:  
X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00
	shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id 971461F463;
	Sat, 14 Sep 2019 05:25:09 +0000 (UTC)
Date: Sat, 14 Sep 2019 05:25:09 +0000
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Cc: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Subject: Re: httpd 502s [was: trying to figure out 100% CPU usage in nntpd...]
Message-ID: <20190914052509.5quscvb5x42pu6co@dcvr>
References: <20190911022215.GA309@dcvr>
 <20190911102436.GA21959@pure.paranoia.local>
 <20190911171250.vqqpaeb7sn34hv3s@dcvr>
 <20190911173628.GA14147@pure.paranoia.local>
 <20190912000541.gikjoimbdeahh7lx@whir>
 <20190912024942.7dejzf47swpzkdux@dcvr>
 <20190912083503.GA10657@dcvr>
 <20190912113758.GB29277@pure.paranoia.local>
 <20190913031212.dkbtvj3ij6qqm6im@dcvr>
 <20190913180732.GA3654@pure.paranoia.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20190913180732.GA3654@pure.paranoia.local>
List-Id: <meta.public-inbox.org>

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Fri, Sep 13, 2019 at 03:12:12AM +0000, Eric Wong wrote:
> > > No, it's just vanilla what comes with the source.
> > 
> > OK, and Perl 5.16.3 from CentOS 7?  (4:5.16.3-294.el7_6 RPM)
> 
> Correct, though another thing that may help is the git version -- we
> install git2u from IUS, so the git version is not what is in standard
> CentOS7 (it's too ancient).
> 
> # rpm -q git2u
> git2u-2.16.5-1.ius.centos7.x86_64
> 
> See if you have more luck reproducing the problem when you get that in
> place. You should be able to find what you need by searching for "git2u
> ius centos".

Tried git2u, didn't help.  I wouldn't expect external processes
such as nginx or git-http-backend having any impact on leaks
unless it's slower/faster and triggers a race in public-inbox.

At this point I suspect it's timing-related in our code but
maybe not something my hardware/VM is capable of reproducing.

What level of parallelism is grokmirror using?

I am very curious if anybody else sees it, too...

> My access over the next 10 days is going to be spotty due to travel, but
> I'll try to catch up at least once a day.

The /proc fdinfo of the eventpoll descriptor+lsof at the same
time would be helpful if you get a moment.  But it's understandable
if you don't :>

> Thanks again for your help!

No problem.  I've gone through the code again but nothing jumped
out at me.  Will add some comments and dig into it again after
I do some other stuff.

Enjoy your travels!