Re: Mysterious gzipped images

[Lars Magne Ingebrigtsen <larsi@gnus.org>]

Paul Eggert <eggert@cs.ucla.edu> writes:

>>   out = (char *) malloc (BUFFER_SIZE);
>
> BUFFER_SIZE is so small that you should just put the buffer on the
> stack "char out[BUFFER_SIZE];".  That way, you don't need to worry
> about freeing it later (e.g., on error).

Ah, I thought having (largish) arrays on the stack was frowned upon, but
perhaps 16K isn't big these days?

> Or better yet, why not use the gap as the output buffer?
> That should avoid some unnecessary copying.

The buffer gap?  Ah, I see.  That's very clever.  I'll rework the code
to do that.

>>   stream.avail_in = iend - istart;
>
> On 64-bit platforms, this won't work on buffers larger than 4 GiB.
> I suggest 'stream.avail_in = min (iend - istart, UINT_MAX);'
> and then put the whole thing inside a loop that repeats
> until the input buffer is exhausted.

Right.

> Each time through the inner loop, it should QUIT so that the user can
> interrupt.  You need a record_unwind_protect around the whole thing;
> the unwind-protect should be the code that deletes any uncompressed
> data already inserted and restores point.

Hm...  is that necessary?  gunzipping is quite fast.  And if you have a
8GB compressed file somehow in your buffer, I would assume that we're in
the year 2025, and your machine is fast enough to make that not matter. >"?

On the other hand, if your machine starts thrashing because it's running
out of memory, having `C-g' work would be nice.  In which case, having
the outer loop be smaller might be a good idea.  Say, only decompress a
few hundred megs at a time, max, for instance.

>>     case Z_STREAM_ERROR:
>>     case Z_NEED_DICT:
>>     case Z_DATA_ERROR:
>>     case Z_MEM_ERROR:
>
> Shouldn't an error be signaled for this sort of thing?
> That would fit in with the unwind-protect.

Yeah, I debated with myself whether signalling an error or returning nil
would be the best interface here.  We have lots of decoding functions
that won't signal errors on invalid inputs (like
`rfc2047-decode-encoded-words'), because we're typically calling them in
tight loops, and slapping `condition-case' around those calls are
annoying.

I.e., they're not anywhere close to the user level, so making them
signal an error is almost never what we want.

Decompressing the region, on the other hand, won't usually be called
that way (in a loop), but it's not a user-level command, either.  So I
don't know.

-- 
(domestic pets only, the antidote for overdose, milk.)
  No Gnus T-Shirt for sale: http://ingebrigtsen.no/no.php
  and http://lars.ingebrigtsen.no/2013/08/twenty-years-of-september.html