From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Lars Magne Ingebrigtsen Newsgroups: gmane.emacs.devel Subject: Re: Mysterious gzipped images Date: Thu, 08 Aug 2013 16:06:01 +0200 Message-ID: References: <87mwoul8sz.fsf@igel.home> <52039B84.4070603@cs.ucla.edu> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1375970792 31165 80.91.229.3 (8 Aug 2013 14:06:32 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 8 Aug 2013 14:06:32 +0000 (UTC) Cc: emacs-devel@gnu.org To: Paul Eggert Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Aug 08 16:06:35 2013 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1V7QrK-0004ei-00 for ged-emacs-devel@m.gmane.org; Thu, 08 Aug 2013 16:06:34 +0200 Original-Received: from localhost ([::1]:48420 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V7QrJ-0005pF-Gx for ged-emacs-devel@m.gmane.org; Thu, 08 Aug 2013 10:06:33 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:57720) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V7Qr7-0005bo-OB for emacs-devel@gnu.org; Thu, 08 Aug 2013 10:06:28 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1V7Qr0-0002Ym-Vf for emacs-devel@gnu.org; Thu, 08 Aug 2013 10:06:21 -0400 Original-Received: from hermes.netfonds.no ([80.91.224.195]:48858) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1V7Qr0-0002YS-Me for emacs-devel@gnu.org; Thu, 08 Aug 2013 10:06:14 -0400 Original-Received: from cm-84.215.51.58.getinternet.no ([84.215.51.58] helo=stories.gnus.org) by hermes.netfonds.no with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1V7Qqo-0001oL-2v; Thu, 08 Aug 2013 16:06:02 +0200 Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEUAAAADAwMBAQEZGxQC AQMDAgMCAgKrfQ++AAACXklEQVQ4jVVUQY4bQQhEffC9e9acp1GWe2Ipdyv7gIwj+2yIxf+fkIIZ ZbVt2ZZgKKoKemiIRlNVjm6dWbnvh1zDT0Ia4RyhjvxbJf5ePk/n4DDUXPfEH8Qqn1FUGEAtEwti vzIxXE3ZAHT+CsUsVXLusSd+HBUd3QPcOrBoqcTPSmyh7GieTb5CuYmi+Ui68bGf+0ffmDlSSGol J1pFRZOqdY1h/RFZsRJRE4EpEXjAdXK5k2EcETcw8uliYAyNqGjNYWQYYPIHMMC80uV8D2miDzeo FsvSZEyQdQ4hGTI9LKJwsnnb+vkhU+cETcjOL8DQXO53beIyEUOD6GFbshJdEUUXsBmFZTARUE9w XTXpgiaXgwUG5S0ZyxMusWrPKq3E2lI8JMMXQZSzExIM1VKehHdPkNjp+jvJDTkxlRFTjfdGtMlJ ZTZKgRLlMpKxEAiB7wpakJGmTfwDjVQfqiO7BHYPBTObIAG4AeXPdSIuMNokck3JWNqzATDE4Yvk orJdO/0msA08JeY5wxQIvgvV9CKVYU7R0shaeAInHFd8IrWn8ZUocc1zSMpTJbeQE0pOkgbW+qCv 2X5zFhpthLwrvNt39rhRF+rb9gawXGc093jZkeDVJ5RF5O6b8x7v36ljlaKWowdjbXLXd9tBEdOI occNPJrAbsfMoxYWG/Ia16MHEtAVdy5P4/Xi/4koFYKIGYTGAYUH4EWiV2/fb+1lIe8dKkr5xCBs X9HlQo+mTpgFe05Ovt0i9SMBJEnFSSnS/s+XjOeGSr4t0uBa9ZrHu6xr01tOClcpOVXRP4sa5ALL aZtNAAAAAElFTkSuQmCC X-Now-Playing: Talking Heads's _Fear of Music_: "Memories Can't Wait" X-Hashcash: 1:23:130808:emacs-devel@gnu.org::hSxE2tGFTqSKGTwm:000000000000000000000000000000000000000000Uxu+ X-Hashcash: 1:23:130808:eggert@cs.ucla.edu::9zYXGFyWVN2A+X55:0000000000000000000000000000000000000000000bX2+ In-Reply-To: <52039B84.4070603@cs.ucla.edu> (Paul Eggert's message of "Thu, 08 Aug 2013 06:22:12 -0700") User-Agent: Gnus/5.130008 (Ma Gnus v0.8) Emacs/24.3.50 (gnu/linux) X-MailScanner-ID: 1V7Qqo-0001oL-2v MailScanner-NULL-Check: 1376575562.25399@YDe4RQMhEQfMHwluJ4tgIg X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 80.91.224.195 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:162506 Archived-At: Paul Eggert writes: >> out = (char *) malloc (BUFFER_SIZE); > > BUFFER_SIZE is so small that you should just put the buffer on the > stack "char out[BUFFER_SIZE];". That way, you don't need to worry > about freeing it later (e.g., on error). Ah, I thought having (largish) arrays on the stack was frowned upon, but perhaps 16K isn't big these days? > Or better yet, why not use the gap as the output buffer? > That should avoid some unnecessary copying. The buffer gap? Ah, I see. That's very clever. I'll rework the code to do that. >> stream.avail_in = iend - istart; > > On 64-bit platforms, this won't work on buffers larger than 4 GiB. > I suggest 'stream.avail_in = min (iend - istart, UINT_MAX);' > and then put the whole thing inside a loop that repeats > until the input buffer is exhausted. Right. > Each time through the inner loop, it should QUIT so that the user can > interrupt. You need a record_unwind_protect around the whole thing; > the unwind-protect should be the code that deletes any uncompressed > data already inserted and restores point. Hm... is that necessary? gunzipping is quite fast. And if you have a 8GB compressed file somehow in your buffer, I would assume that we're in the year 2025, and your machine is fast enough to make that not matter. >"? On the other hand, if your machine starts thrashing because it's running out of memory, having `C-g' work would be nice. In which case, having the outer loop be smaller might be a good idea. Say, only decompress a few hundred megs at a time, max, for instance. >> case Z_STREAM_ERROR: >> case Z_NEED_DICT: >> case Z_DATA_ERROR: >> case Z_MEM_ERROR: > > Shouldn't an error be signaled for this sort of thing? > That would fit in with the unwind-protect. Yeah, I debated with myself whether signalling an error or returning nil would be the best interface here. We have lots of decoding functions that won't signal errors on invalid inputs (like `rfc2047-decode-encoded-words'), because we're typically calling them in tight loops, and slapping `condition-case' around those calls are annoying. I.e., they're not anywhere close to the user level, so making them signal an error is almost never what we want. Decompressing the region, on the other hand, won't usually be called that way (in a loop), but it's not a user-level command, either. So I don't know. -- (domestic pets only, the antidote for overdose, milk.) No Gnus T-Shirt for sale: http://ingebrigtsen.no/no.php and http://lars.ingebrigtsen.no/2013/08/twenty-years-of-september.html