From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 8170C1F4C4; Tue, 5 Nov 2024 23:24:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1730849085; bh=hAASE9LQU0VWw+hlfti52fmeqNm5xJeYcxUIfSazK4A=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=FiqjRkn19BKhXT3f0W51lHDaF18czRy8bAIliopCtpNULlCwh0Edbd9wBHG6diDJw 7vn9PwZry0MQ4CihfQj7SHk5FJndLa4zl6cT3JkmxXRvwsuQmwJEcvlhU8clMo7y0O lzDvinqxCc3I3TYckLeZtOJKf1EuOw8n0Q16BS3A= Date: Tue, 5 Nov 2024 23:24:45 +0000 From: Eric Wong To: Jonathan Corbet Cc: meta@public-inbox.org Subject: Re: Occasional public-inbox-httpd flakiness Message-ID: <20241105232445.M291444@dcvr> References: <875xp15n3o.fsf@trenco.lwn.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <875xp15n3o.fsf@trenco.lwn.net> List-Id: Jonathan Corbet wrote: > The LWN archive server is running Debian's 1.9.0 public-inbox package. > Every now and then, usually after at least a week of operation, HTTP > requests will start returning empty messages; I find stuff like this in > the log: > > Nov 05 15:24:37 archive2.lwn.net public-inbox-httpd[1267166]: Use of uninitialized value in index at /usr/share/perl5/PublicInbox/Eml.pm line 109. > Nov 05 15:24:37 archive2.lwn.net public-inbox-httpd[1267166]: Use of uninitialized value in index at /usr/share/perl5/PublicInbox/Eml.pm line 109. > Nov 05 15:24:37 archive2.lwn.net public-inbox-httpd[1267166]: Use of uninitialized value in pattern match (m//) at /usr/share/perl5/PublicInbox/Eml.pm line 115. > Nov 05 15:24:37 archive2.lwn.net public-inbox-httpd[1267166]: Use of uninitialized value in pattern match (m//) at /usr/share/perl5/PublicInbox/Eml.pm line 120. > Nov 05 15:24:37 archive2.lwn.net public-inbox-httpd[1267166]: Use of uninitialized value in pattern match (m//) at /usr/share/perl5/PublicInbox/Eml.pm line 127. > Nov 05 15:24:37 archive2.lwn.net public-inbox-httpd[1267166]: Use of uninitialized value in substitution (s///) at /usr/share/perl5/PublicInbox/Mbox.pm line 115. Definitely something that's popped up in my recollection; but hasn't happened in a while for Eml.pm and Mbox.pm (yeah, it's been a while since v1.9 :x). Are the git cat-file (or Gcf2) processes still running? Are any successful responses returned for requests to mail messages? Error handling should be improved in .git nowadays but I'm still struggling to get a release out due to coderepo <=> inbox mapping messiness :< > The pattern is pretty much always the same. Restarting > public-inbox-httpd makes the problem go away again. Usually, uninitialized value errors are isolated to a single request (e.g. broken emails) and there shouldn't be a need to restart unless every request is failing. > Is this a problem that anybody else has seen, or am I especially > lucky...? I've had some similar problems from inboxes/coderepos getting removed; also there's OOMs on my HW causing git processes to fail.