From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 80F631F464 for ; Thu, 26 Sep 2019 01:50:38 +0000 (UTC) From: Eric Wong To: meta@public-inbox.org Subject: [PATCH 0/2] leak workarounds for Perl 5.16 on CentOS/RHEL 7 Date: Thu, 26 Sep 2019 01:50:36 +0000 Message-Id: <20190926015038.29770-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: After many hours of reviewing our code in PublicInbox::Qspawn, PublicInbox::GitHTTPBackend, and PublicInbox::HTTP and finding nothing but cleanups and documentation improvements; it seems the leaks affecting lore is down to bugs in Perl 5.16.3. After removing the warning for Deflater being missing in d883d4a93b23be134038e28f421eafca70c3d838 ("httpd: get rid of Deflater warning"), I missed that my own CentOS 7 VM was missing that module so was unable to reproduce the FD leaks :x The first patch is a straightforward workaround that I was able to test without Plack::Middleware::Deflater being installed. The second patch stops loading Deflater in httpd on Perls earlier than 5.18. (But I haven't built or tested 5.18 myself). Enabling gzip in varnish will be needed for 5.16 users. Independently of this fix, I've long been considering replacing Deflater with a buffer-to-gzip layer which would reduce memory pressure from intermediate uncompressed strings. This might serve as impetus to move that idea along (and of course I'd test it heavily under CentOS 7 :>). Anybody using custom .psgi files on Perl 5.16.x will need to make adjustments to disable the Deflater in that. Eric Wong (2): ds: workaround a memory leak in Perl 5.16.x httpd: disable Deflater middleware by default on Perl <5.18 lib/PublicInbox/DS.pm | 9 ++++++--- script/public-inbox-httpd | 8 +++++++- 2 files changed, 13 insertions(+), 4 deletions(-) -- EW