From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 1EB5D6DE0A89 for ; Sun, 22 Sep 2019 12:03:43 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -5 X-Spam-Level: X-Spam-Status: No, score=-5 tagged_above=-999 required=5 tests=[RCVD_IN_DNSWL_HI=-5] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BDnICZP3w3bM for ; Sun, 22 Sep 2019 12:03:42 -0700 (PDT) X-Greylist: delayed 905 seconds by postgrey-1.36 at arlo; Sun, 22 Sep 2019 12:03:41 PDT Received: from youngberry.canonical.com (youngberry.canonical.com [91.189.89.112]) by arlo.cworth.org (Postfix) with ESMTPS id CD0CC6DE0232 for ; Sun, 22 Sep 2019 12:03:41 -0700 (PDT) Received: from 1.is.james.uk.vpn ([10.172.254.24] helo=malefic) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1iC6uQ-0002kP-JW for notmuch@notmuchmail.org; Sun, 22 Sep 2019 18:48:34 +0000 Received: from james by malefic with local (Exim 4.92 #3 (Debian)) id 1iC6uQ-0003rg-Ex for ; Sun, 22 Sep 2019 19:48:34 +0100 From: James Troup To: notmuch@notmuchmail.org Subject: [PATCH] fix file descriptor leak with compressed email Mail-Copies-To: never Date: Sun, 22 Sep 2019 19:48:34 +0100 Message-ID: <87pnjsf9q5.fsf@canonical.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2019 19:03:43 -0000 Hi, I'm trying notmuch for the first time using notmuch 0.29.1-2ubuntu1 backported to Ubuntu 19.04. My email is stored in Gnus nnml format and is approximately 410K mails with 60% compressed. While trying to run 'notmuch new' I very quickly ran into issues with 'Too many open files', even after adjusting the maximum number of open file descriptor up to 65K. The patch below solved the problem for me. diff -ur orig/notmuch-0.29.1/util/gmime-extra.c notmuch-0.29.1/util/gmime-extra.c --- orig/notmuch-0.29.1/util/gmime-extra.c 2019-06-11 23:16:48.000000000 +0000 +++ notmuch-0.29.1/util/gmime-extra.c 2019-09-22 18:25:28.629546076 +0000 @@ -28,6 +28,8 @@ /* ignore filter id */ (void)g_mime_stream_filter_add ((GMimeStreamFilter *)gzstream, gzfilter); + g_object_unref(gzfilter); + g_object_unref(file_stream); return gzstream; } else { return file_stream; -- James