Re: Huge file adventure (+patch)

[Stefan Monnier <monnier@iro.umontreal.ca>]

> And, of course, there is a patch to address an issues described above.
> Comments are very welcome because I'm not hooked too much in coding
> machinery (yet).

Sounds good, in general.

>        else
>  	{
> -	  coding->dst_object
> -	    = make_unibyte_string ((char *) coding->destination,
> -				   coding->produced);
> -	  xfree (coding->destination);
> +	  /* This is used to avoid creating huge Lisp string.
> +	     NOTE: caller who set `raw_destination' is also
> +	     responsible to free `destination' buffer.  */
> +	  if (coding->raw_destination)
> +	    coding->dst_object = Qnil;
> +	  else
> +	    {
> +	      coding->dst_object
> +		= make_unibyte_string ((char *) coding->destination,
> +				       coding->produced);
> +	      xfree (coding->destination);
> +	    }
>  	}
>      }
 
You can remove the { between "else" and "if" (with the side-benefit that
the unchanged code will stay at the same indentation level).

>  	  if (CODING_REQUIRE_ENCODING (coding))
>  	    {
> -	      encode_coding_object (coding, string,
> -				    start, string_char_to_byte (string, start),
> -				    end, string_char_to_byte (string, end), Qt);
> +	      ptrdiff_t nchars = min (end - start, E_WRITE_MAX);
> +
> +	      /* Avoid creating huge Lisp string in encode_coding_object.  */
> +	      if (nchars == E_WRITE_MAX)
> +		coding->raw_destination = 1;
> +
> +	      encode_coding_object
> +		(coding, string, start, string_char_to_byte (string, start),
> +		 start + nchars, string_char_to_byte (string, start + nchars),
> +		 Qt);
>  	    }

Where/how do you make sure we loop back here for the rest
(when end-start > E_WRITE_MAX) ?


        Stefan