From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id cGrEH+UAll4segAA0tVLHw (envelope-from ) for ; Tue, 14 Apr 2020 18:28:53 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id UDy/GOgAll5fWwAAB5/wlQ (envelope-from ) for ; Tue, 14 Apr 2020 18:28:56 +0000 Received: from arlo.cworth.org (arlo.cworth.org [50.126.95.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5963894219B for ; Tue, 14 Apr 2020 18:28:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 5D0B46DE0F43; Tue, 14 Apr 2020 11:28:48 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YN_bK1-ygo73; Tue, 14 Apr 2020 11:28:47 -0700 (PDT) Received: from arlo.cworth.org (localhost [IPv6:::1]) by arlo.cworth.org (Postfix) with ESMTP id 7E6F86DE0BEF; Tue, 14 Apr 2020 11:28:41 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id DA4556DE0948 for ; Tue, 14 Apr 2020 10:38:42 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zMyH2RVN-6Ky for ; Tue, 14 Apr 2020 10:38:42 -0700 (PDT) Received: from olitb.net (olitb.net [37.187.104.224]) by arlo.cworth.org (Postfix) with ESMTP id 376FA6DE0943 for ; Tue, 14 Apr 2020 10:38:42 -0700 (PDT) Received: from localhost (unknown [IPv6:2a00:5881:4008:6c00:607b:1d71:9451:79b4]) by olitb.net (Postfix) with ESMTPSA id C95A2DF61B for ; Tue, 14 Apr 2020 19:39:48 +0200 (CEST) Date: Tue, 14 Apr 2020 19:38:40 +0200 From: Olivier =?utf-8?B?VGHDr2Jp?= To: notmuch@notmuchmail.org Subject: [PATCH] after gzgets(), Z_STREAM_END means EOF, not error Message-ID: <20200414173840.nwsaemepavnbc2za@siegel.lan> MIME-Version: 1.0 Content-Disposition: inline X-Mailman-Approved-At: Tue, 14 Apr 2020 11:28:39 -0700 X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: notmuch-bounces@notmuchmail.org Sender: "notmuch" X-Scanner: scn0 X-Spam-Score: -1.01 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of notmuch-bounces@notmuchmail.org designates 50.126.95.6 as permitted sender) smtp.mailfrom=notmuch-bounces@notmuchmail.org X-Scan-Result: default: False [-1.01 / 13.00]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; GENERIC_REPUTATION(0.00)[-0.45304610647521]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; SPF_REPUTATION_HAM(0.00)[-0.45157941738515]; R_SPF_ALLOW(-0.20)[+a]; IP_REPUTATION_HAM(0.00)[asn: 27017(-0.18), country: US(-0.01), ip: 50.126.95.6(-0.45)]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[notmuch@notmuchmail.org]; HAS_LIST_UNSUB(-0.01)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[olitb.net]; MX_GOOD(-0.50)[cached: notmuchmail.org]; FORGED_SENDER_MAILLIST(0.00)[]; MAILLIST(-0.20)[mailman]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:27017, ipnet:50.126.64.0/18, country:US]; RCVD_COUNT_SEVEN(0.00)[8]; FROM_NEQ_ENVFROM(0.00)[oli@olitb.net,notmuch-bounces@notmuchmail.org] X-TUID: U5bU+0Fid10S As suggested by David Bremner in https://notmuchmail.org/pipermail/notmuch/2020/029288.html here is the patch for bug #3: after gzgets() returns NULL (meaning EOF or error), the error code Z_STREAM_END means EOF and not error. Context: I am compiling notmuch on OpenBSD which has a rather old zlib 1.2.3. It seems that the behaviour of gzgets() changed slightly between this version and more recent versions, but the manual does not reflect that change. Note that zlib's manual: - does not specify which error code (Z_OK or Z_STREAM_END) is set when EOF is reached, - does not indicate the meaning of Z_STREAM_END after gzgets(), but based on its meaning as a possible return value of inflate(), I would guess that it means EOF. PS: out of curiosity, why bother with the --gzip feature in notmuch dump and restore when the user can simply pipe to/from a gzip/bzip2/xz/... command? It seems that this feature adds complication, some of this complication being due to the fact that the behaviour of zlib is not as well-defined as stdio. Moreover using pipes allows to check that the compressed dump decompresses as it should, e.g.: $ mkfifo pipe $ dump.gz & $ notmuch dump | tee pipe | cksum $ wait && zcat dump.gz | cksum ; rm pipe This could be simplified with process substitution for shells that have this feature, and the checksum comparison can certainly be made automatic in a backup script. Disclosure: I am biased because I am currently patching notmuch-dump.c to use stdio instead of zlib in order to port notmuch to OpenBSD, since OpenBSD's older zlib does not have the "T" mode for gzopen(), so with zlib the only choice would to compress the output. Perhaps zlib will be updated in OpenBSD in the future, but this is a short-term solution that seems to not be too much trouble to maintain downstream. PPS: apart from dump and restore (and the indirect use of xapian), it seems that the only other use of zlib in notmuch is in format_part_mbox() in notmuch-show.c, which is able to read a compressed email (it seems that dovecot has an option to write emails in maildir format in this way to save space). Do I understand correctly that notmuch does not support indexing compressed email, and if so what is the point of using zlib in format_part_mbox()? diff --git a/util/zlib-extra.c b/util/zlib-extra.c index 623f6d62..2d2d2414 100644 --- a/util/zlib-extra.c +++ b/util/zlib-extra.c @@ -47,6 +47,7 @@ gz_getline (void *talloc_ctx, char **bufptr, ssize_t *bytes_read, gzFile stream) int zlib_status = 0; (void) gzerror (stream, &zlib_status); switch (zlib_status) { + case Z_STREAM_END: case Z_OK: /* no data read before EOF */ if (offset == 0)