From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Amritpal Singh Newsgroups: gmane.emacs.bugs Subject: bug#63831: [PATCH] Elaboration on the bugfix for failed inflation of gz archives Date: Sat, 3 Jun 2023 08:38:18 +0530 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="36650"; mail-complaints-to="usenet@ciao.gmane.io" To: 63831@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Sat Jun 03 09:15:11 2023 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1q5LTj-0009Ob-2d for geb-bug-gnu-emacs@m.gmane-mx.org; Sat, 03 Jun 2023 09:15:11 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q5LTe-0003hD-5c; Sat, 03 Jun 2023 03:15:06 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q5LTb-0003gV-RH for bug-gnu-emacs@gnu.org; Sat, 03 Jun 2023 03:15:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q5LTa-00082z-GS for bug-gnu-emacs@gnu.org; Sat, 03 Jun 2023 03:15:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1q5LTa-00043N-BB for bug-gnu-emacs@gnu.org; Sat, 03 Jun 2023 03:15:02 -0400 X-Loop: help-debbugs@gnu.org In-Reply-To: Resent-From: Amritpal Singh Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 03 Jun 2023 07:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 63831 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch Original-Received: via spool by 63831-submit@debbugs.gnu.org id=B63831.168577645015486 (code B ref 63831); Sat, 03 Jun 2023 07:15:02 +0000 Original-Received: (at 63831) by debbugs.gnu.org; 3 Jun 2023 07:14:10 +0000 Original-Received: from localhost ([127.0.0.1]:41447 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5LSj-00041c-96 for submit@debbugs.gnu.org; Sat, 03 Jun 2023 03:14:10 -0400 Original-Received: from mail-lj1-f177.google.com ([209.85.208.177]:55609) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5Hd2-0000ZL-PS for 63831@debbugs.gnu.org; Fri, 02 Jun 2023 23:08:34 -0400 Original-Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2af2db78b38so37052601fa.3 for <63831@debbugs.gnu.org>; Fri, 02 Jun 2023 20:08:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685761706; x=1688353706; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=7tPJbejxTyM9fxJVEzi49jWgxQTRKa6tzNO5QaaSnI8=; b=m10u6YMDCuftWj5YzSxjpc3kcvFKXFc5mgaEw5ghpENjoor6G+aFh1MdkUnTahh5rF qK5O8rxoeSSzKuYuD8/ESXKTgWrfAmiuyMRf7s9Mq9QN6AThsy6SvYOLGrC0kQ7ih5re bMhAgU1f4KM9xv5z/V4puhWWZr6JvfeuFid+G2vLBBlpDqZJ2ULg9yXDzoKG009y+/rx DPKaRbxI9Mq37idh5LMiFdKXoVLDkL9wdjIFrUBgUcujdsreOYi1P2bjcv7X4jFICvqZ 2rkfZ4RJVeARVrDl9uRtx2y1aIXAW7xBaMmb8+xqYozjyJP8/w9kQVn0TSEB5UZ3vbNP r4IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685761706; x=1688353706; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=7tPJbejxTyM9fxJVEzi49jWgxQTRKa6tzNO5QaaSnI8=; b=k1nb+F51jMjduVbMM4JsiLlaaX/ewRYxJxicuwSbNq2GGstr9XYMPH7sUEpFkLtZy8 92DjqXsSQnyLezzEwz8izVTqD76TOWrUKnL6NTVwGtzQB5KVVxqKGITV6K0O+c16AXte MIiDlpVgeiwqiuk3AyjugH6YebAsZyoJ6sIoM21EeOzMdxZed+89PArPC9wzFdeSePtU lqxev14fkuwwMgcknjpOe4ehgjK+PgNyc5fIyUoMS6IXG449BbPR5LJPdx0nlTpfa9FX 8X3vy0PN48g32Z0I4Ow4MryZi/3ncXlJAvW4NJz4MpLXCr8LiVZyjKpGqsENXh1eo1pH v4KQ== X-Gm-Message-State: AC+VfDySxiCFx/IlCm5EHBA2C4jdnbX70sYBDzl6x5LLYDtqYgLVS5tH 113r+Jw1xRopuG2rKt+hts09ELd2jv6U7X4RqLN3etk4osKYkA== X-Google-Smtp-Source: ACHHUZ7Cf+xgjLZZtbEwTrbmT89YMzwct+ShPVfqFkiw9+Guw0qiLkxDpm7WTVLwvXD1WcthMHHbABiBq9eJe01GQTQ= X-Received: by 2002:a2e:7804:0:b0:2af:21d3:a4ca with SMTP id t4-20020a2e7804000000b002af21d3a4camr912946ljc.47.1685761706203; Fri, 02 Jun 2023 20:08:26 -0700 (PDT) X-Mailman-Approved-At: Sat, 03 Jun 2023 03:14:08 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:262803 Archived-At: > Now after logging some code in $EMACS_REPO/src/decompress.c, it was > learned that in the pigz specific case, `inflate()` was returning > Z_BUF_ERROR(-5) which is an indicator for zstream's either `avail_in` > or `avail_out` fields are 0. > > Observing the code in `$EMACS_REPO/src/decompress.c` > L154: > } while (!stream.avail_out); > only checks stream.avail_out and not stream.avail_in which also might > have been set to 0. A special case here can be constructed where > `avail_in` is 0, and the code keeps looping even though our input > buffer is empty and thus causing a Z_BUF_ERROR. Placing a simple check > for it fixes the bug in pigz's gz archives case and does not cause any > issue with gzip archives. I'd like to elaborate a bit further on this part mentioned in the bug report-cum-patch email. The code in question where the bug occurs is as follows: > do { > stream.avail_in = fread (in, 1, MD5_BLOCKSIZE, source); > if (ferror (source)) { > inflateEnd (&stream); > return -1; > } > if (stream.avail_in == 0) > break; > stream.next_in = in; > > do { > stream.avail_out = MD5_BLOCKSIZE; > stream.next_out = out; > res = inflate (&stream, Z_NO_FLUSH); > > if (res != Z_OK && res != Z_STREAM_END) > return -1; > > accumulate_and_process_md5 (out, MD5_BLOCKSIZE - stream.avail_out, &ctx); L154: > } while (!stream.avail_out); L155: > > } while (res != Z_STREAM_END); >From here, it is a bit more clear what the originally mentioned `L154` was referring to. The archives compressed using pigz set as the systems' gzip program either via symlink as `gzip` in $PATH or using $GZIP_PROG, fail to hash in this specific instance which is quite strange, nonetheless this patch (provided in the original bug report) fixes the issue of calling inflate() repeatedly (in the inner do-while() loop) when we do not have any more input to process. This happens so, as mentioned before, because `infalte()` can set `stream.avail_in` to 0 which is the available number of bytes to process. And since we're only checking if stream.avail_out hasn't been set to 0 in the inner loop, the hashing fails even though our archive is correctly made and decompresses successfully externally. ps. The patch is in the original bug report.