From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Noam Postavsky Newsgroups: gmane.emacs.bugs Subject: bug#33133: 26.1.50; zlib-decompress-region too rigid Date: Sat, 27 Oct 2018 17:48:26 -0400 Message-ID: <87efcbjc2d.fsf@gmail.com> References: <87a7n4mbos.fsf@gmail.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: blaine.gmane.org 1540676829 24197 195.159.176.226 (27 Oct 2018 21:47:09 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 27 Oct 2018 21:47:09 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) Cc: Kevin Ryde , 33133@debbugs.gnu.org To: Katsumi Yamaoka Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Oct 27 23:47:05 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gGWQB-0006BT-63 for geb-bug-gnu-emacs@m.gmane.org; Sat, 27 Oct 2018 23:47:03 +0200 Original-Received: from localhost ([::1]:37904 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gGWSH-0004FB-6p for geb-bug-gnu-emacs@m.gmane.org; Sat, 27 Oct 2018 17:49:13 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39725) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gGWS9-0004E9-VS for bug-gnu-emacs@gnu.org; Sat, 27 Oct 2018 17:49:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gGWS6-0006jk-Ot for bug-gnu-emacs@gnu.org; Sat, 27 Oct 2018 17:49:05 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:41611) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gGWS6-0006je-Ie for bug-gnu-emacs@gnu.org; Sat, 27 Oct 2018 17:49:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gGWS6-0004GN-AQ for bug-gnu-emacs@gnu.org; Sat, 27 Oct 2018 17:49:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Noam Postavsky Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 27 Oct 2018 21:49:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33133 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 33133-submit@debbugs.gnu.org id=B33133.154067691616356 (code B ref 33133); Sat, 27 Oct 2018 21:49:02 +0000 Original-Received: (at 33133) by debbugs.gnu.org; 27 Oct 2018 21:48:36 +0000 Original-Received: from localhost ([127.0.0.1]:45869 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gGWRf-0004Fe-Tm for submit@debbugs.gnu.org; Sat, 27 Oct 2018 17:48:36 -0400 Original-Received: from mail-it1-f180.google.com ([209.85.166.180]:55239) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gGWRe-0004FM-08; Sat, 27 Oct 2018 17:48:34 -0400 Original-Received: by mail-it1-f180.google.com with SMTP id l191-v6so5460625ita.4; Sat, 27 Oct 2018 14:48:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=QCHdACLKeS8jzzr8o1vAVC4Bn4t/JajO+czw+Mm5V/8=; b=M24ashCPLlIc7mhvtcSck0NouLi1e358wZLglW/wWnLrKuoE3TGOl62k7mLvtwTmqJ /pu7fUFLgmGyWgoCsgVQLzoRE12IQGLlxW5KVv8RtV4udAV6HlBkvR4SONv78bCjwufB Ic8N2pjPjgi9m0RgLuU/ySufmMtCJ4kscKSXwpRynXZk/3aEATfIGBqaNDNpXFAv5GDb GcZuk3+cZHelO8iZe1K/CykmVRSDHXb3/MLM/NRWmknRfYe2ZelZogRXFi2fFVffXXrz rS3cGdtxj9k9nyL+enI6fAfvaLOkwm/nfTn96DEVQXNfIhOQBLArpY7XJzJwOJKVPkWu 29BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=QCHdACLKeS8jzzr8o1vAVC4Bn4t/JajO+czw+Mm5V/8=; b=ons+D0oalNnVQ8T3r7go9gCa+JZE49aAPBgNuXEgMtRrxVxXfuha/H8SW2AUV8mz5i lQkEyLRbDchws++u+VOZBhC5CFGjZYldOTnUeWZ9l1+LIrTkcndUbcEpH4tAKpDHsDjA hgK1eYRkDq3WGWRFTSpur5eE7jxSxJglvoH6Q/2TaT314g3azedc0jMHSYGlUnhmiizb Abx0W5PZOGhKhusCVx+3JrnRntfMMU9fTTQwmh+pEoqjo0CFC1LNie8SXbsk/pA5KEl5 dLDeGrCbAq+MR9WB8K97TbwHEJEcjdU/eySuRSjJpGez3zKde2Qo/MWxTWqfvRcyUnVe mc7Q== X-Gm-Message-State: AGRZ1gLhUGInG1KWaKdFGkJfLCQi12ueHyEnfPp2KaosQ+ZpyFr2leNO KAVF1vmhlfcxYbCnoKMMi5lFutK7 X-Google-Smtp-Source: AJdET5exkDXcHLoyvuEhC7FACp0w6QmcIjdVj2m3EM8WTgKIqmEM7Ll59NueDo2On+vO/YBitmV4mA== X-Received: by 2002:a02:c18:: with SMTP id g24-v6mr6457762jad.131.1540676908126; Sat, 27 Oct 2018 14:48:28 -0700 (PDT) Original-Received: from zebian (cbl-45-2-119-34.yyz.frontiernetworks.ca. [45.2.119.34]) by smtp.googlemail.com with ESMTPSA id u68-v6sm3208552itd.1.2018.10.27.14.48.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 27 Oct 2018 14:48:27 -0700 (PDT) In-Reply-To: (Katsumi Yamaoka's message of "Wed, 24 Oct 2018 10:16:16 +0900") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:151715 Archived-At: --=-=-= Content-Type: text/plain tags 33133 + patch quit Katsumi Yamaoka writes: > On Tue, 23 Oct 2018 20:26:59 -0400, Noam Postavsky wrote: >> --- i/src/decompress.c >> +++ w/src/decompress.c >> @@ -206,7 +206,7 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, > >> while (inflate_status == Z_OK); > >> - if (inflate_status != Z_STREAM_END) >> + if (inflate_status != Z_STREAM_END && inflate_status != Z_BUF_ERROR) >> return unbind_to (count, Qnil); > >> unwind_data.start = 0; > > I confirmed that it makes it work for the corrupted web site in > question. Thank you! Here's a proper patch. --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=v1-0001-Allow-partial-decompression-Bug-33133.patch Content-Description: patch >From 430ebd936b0bc41bd3e33e171938161846597196 Mon Sep 17 00:00:00 2001 From: Noam Postavsky Date: Sat, 27 Oct 2018 17:45:52 -0400 Subject: [PATCH v1] Allow partial decompression (Bug#33133) * src/decompress.c (Fzlib_decompress_region): Add optional ALLOW-PARTIAL parameter. * lisp/url/url-http.el (url-handle-content-transfer-encoding): Use it. * doc/lispref/text.texi (Decompression): Document it. * etc/NEWS: Announce it. --- doc/lispref/text.texi | 10 ++++++---- etc/NEWS | 6 ++++++ lisp/url/url-http.el | 5 +++-- src/decompress.c | 22 +++++++++++++++++----- 4 files changed, 32 insertions(+), 11 deletions(-) diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi index 6c38d8eed0..e39ba6a192 100644 --- a/doc/lispref/text.texi +++ b/doc/lispref/text.texi @@ -4462,14 +4462,16 @@ Decompression available. @end defun -@defun zlib-decompress-region start end +@defun zlib-decompress-region start end &optional allow-partial This function decompresses the region between @var{start} and @var{end}, using built-in zlib decompression. The region should contain data that were compressed with gzip or zlib. On success, the function replaces the contents of the region with the decompressed -data. On failure, the function leaves the region unchanged and -returns @code{nil}. This function can be called only in unibyte -buffers. +data. If @var{allow-partial} is @code{nil}, on failure, the function +leaves the region unchanged and returns @code{nil}. Otherwise, it +returns the number of bytes that were not decompressed and replaces +the region text by whatever data was successfully decompressed. This +function can be called only in unibyte buffers. @end defun diff --git a/etc/NEWS b/etc/NEWS index 3f86195695..395169253d 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -1159,6 +1159,12 @@ to mean that it is not known whether DST is in effect. 'json-insert', 'json-parse-string', and 'json-parse-buffer'. These are implemented in C using the Jansson library. ++++ +** 'zlib-decompress-region' can partially decompress corrupted data. +If the new optional ALLOW-PARTIAL argument is passed, then the data +that was decompressed successfully before failing will be inserted +into the buffer. + ** Mailcap --- diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 6b5749e1bc..94ac660fcf 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -939,7 +939,8 @@ url-http-parse-headers (goto-char (point-min)) success)) -(declare-function zlib-decompress-region "decompress.c" (start end)) +(declare-function zlib-decompress-region "decompress.c" + (start end &optional allow-partial)) (defun url-handle-content-transfer-encoding () (let ((encoding (mail-fetch-field "content-encoding"))) @@ -951,7 +952,7 @@ url-handle-content-transfer-encoding (widen) (goto-char (point-min)) (when (search-forward "\n\n") - (zlib-decompress-region (point) (point-max))))))) + (zlib-decompress-region (point) (point-max) t)))))) ;; Miscellaneous (defun url-http-activate-callback () diff --git a/src/decompress.c b/src/decompress.c index 2836338216..3872014739 100644 --- a/src/decompress.c +++ b/src/decompress.c @@ -120,12 +120,18 @@ DEFUN ("zlib-available-p", Fzlib_available_p, Szlib_available_p, 0, 0, 0, DEFUN ("zlib-decompress-region", Fzlib_decompress_region, Szlib_decompress_region, - 2, 2, 0, + 2, 3, 0, doc: /* Decompress a gzip- or zlib-compressed region. Replace the text in the region by the decompressed data. -On failure, return nil and leave the data in place. + +If optional parameter ALLOW-PARTIAL is nil or omitted, on failure, +return nil and leave the data in place. Otherwise, return the number +of bytes that were not decompressed and replace the region text by +whatever data was successfully decompressed. If decompression is +completely successful return t. + This function can be called only in unibyte buffers. */) - (Lisp_Object start, Lisp_Object end) + (Lisp_Object start, Lisp_Object end, Lisp_Object allow_partial) { ptrdiff_t istart, iend, pos_byte; z_stream stream; @@ -206,8 +212,14 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, } while (inflate_status == Z_OK); + Lisp_Object ret = Qt; if (inflate_status != Z_STREAM_END) - return unbind_to (count, Qnil); + { + if (!NILP (allow_partial)) + ret = make_int (iend - pos_byte); + else + return unbind_to (count, Qnil); + } unwind_data.start = 0; @@ -218,7 +230,7 @@ DEFUN ("zlib-decompress-region", Fzlib_decompress_region, signal_after_change (istart, iend - istart, unwind_data.nbytes); update_compositions (istart, istart, CHECK_HEAD); - return unbind_to (count, Qt); + return unbind_to (count, ret); } -- 2.11.0 --=-=-=--