Re: Occasional public-inbox-httpd flakiness

[Eric Wong <e@80x24.org>]

Jonathan Corbet <corbet@lwn.net> wrote:
> Eric Wong <e@80x24.org> writes:
> > Jonathan Corbet <corbet@lwn.net> wrote:
> >> Eric Wong <e@80x24.org> writes:
> >> > Jonathan Corbet <corbet@lwn.net> wrote:
> >> >> Just to add a data point...the problem just recurred, and there are
> >> >> definitely cat-file processes running:
> >> >> 
> >> >> $  ps ax | fgrep git
> >> >> 2640024 ?        S      0:40 /usr/bin/git --git-dir=repos/ALL.git -c core.abbrev=40 cat-file --batch
> >> >> 2735080 ?        S      0:18 /usr/bin/git --git-dir=repos/ALL.git -c core.abbrev=40 cat-file --batch
> >> >> 3184082 ?        S      0:03 /usr/bin/git --git-dir=repos/ALL.git -c core.abbrev=40 cat-file --batch
> >> >> 3723223 ?        Z      0:00 [git] <defunct>
> >> >> 3723227 ?        Z      0:00 [git] <defunct>
> >> >  
> >> > Can you see if the worker process causing warnings is connected to defunct gits?
> >> > Should've been fixed in master a while ago, but there's a lot of changes :x
> >> > master should be fine as long as you're not using -cindex + coderepos yet.
> >> 
> >> By "connected to" you mean "is the parent of"?
> >
> > Yes.  lsof +E should show how pipes are connecting processes.
> > Just wondering, those git zombies lingered until the restart, right?
> >
> > IOW, they didn't disappear after a few seconds if the -httpd
> > worker was busy with other things.  During heavy traffic you'll
> > inevitably see short-lived zombies as the -httpd may not reap
> > fast enough, but zombies shouldn't linger indefinitely.
> 
> Looking back through the terminal history, it looks like the zombies
> hung out for a bit, but then went away.  There were a couple of zombies
> every time I looked, but the PIDs eventually changed.

OK.  Any idea how long the zombies lingered?

How much load the LWN -httpd instance see?

AFAIK, the main cause of zombies I've seen was from the blob solver,
but that requires coderepos, which AFAIK nobody else ever used...

Also, it would likely be useful if you got an strace on the worker PID
that was causing warnings