From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.devel Subject: Re: Huge file adventure (+patch) Date: Mon, 07 Oct 2013 11:15:41 -0400 Message-ID: References: <5252832E.5060804@yandex.ru> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1381158957 14385 80.91.229.3 (7 Oct 2013 15:15:57 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 7 Oct 2013 15:15:57 +0000 (UTC) Cc: Emacs development discussions To: Dmitry Antipov Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Oct 07 17:16:00 2013 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1VTCXQ-0005jC-Gp for ged-emacs-devel@m.gmane.org; Mon, 07 Oct 2013 17:16:00 +0200 Original-Received: from localhost ([::1]:60237 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VTCXP-0007Ll-Nv for ged-emacs-devel@m.gmane.org; Mon, 07 Oct 2013 11:15:59 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:44079) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VTCXG-0007Lb-Ad for emacs-devel@gnu.org; Mon, 07 Oct 2013 11:15:57 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VTCX9-0002bE-0h for emacs-devel@gnu.org; Mon, 07 Oct 2013 11:15:50 -0400 Original-Received: from ironport2-out.teksavvy.com ([206.248.154.182]:46614) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VTCX8-0002b5-Et for emacs-devel@gnu.org; Mon, 07 Oct 2013 11:15:42 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av4EABK/CFHO+K8t/2dsb2JhbABEvw4Xc4IeAQEEAScvIwULCw4mEhQYDSSIHgbBLZEKA6R6gV6DEw X-IPAS-Result: Av4EABK/CFHO+K8t/2dsb2JhbABEvw4Xc4IeAQEEAScvIwULCw4mEhQYDSSIHgbBLZEKA6R6gV6DEw X-IronPort-AV: E=Sophos;i="4.84,565,1355115600"; d="scan'208";a="34930561" Original-Received: from 206-248-175-45.dsl.teksavvy.com (HELO pastel.home) ([206.248.175.45]) by ironport2-out.teksavvy.com with ESMTP/TLS/ADH-AES256-SHA; 07 Oct 2013 11:12:04 -0400 Original-Received: by pastel.home (Postfix, from userid 20848) id 8ED7560C2E; Mon, 7 Oct 2013 11:15:41 -0400 (EDT) In-Reply-To: <5252832E.5060804@yandex.ru> (Dmitry Antipov's message of "Mon, 07 Oct 2013 13:47:26 +0400") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 206.248.154.182 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:163946 Archived-At: > And, of course, there is a patch to address an issues described above. > Comments are very welcome because I'm not hooked too much in coding > machinery (yet). Sounds good, in general. > else > { > - coding->dst_object > - = make_unibyte_string ((char *) coding->destination, > - coding->produced); > - xfree (coding->destination); > + /* This is used to avoid creating huge Lisp string. > + NOTE: caller who set `raw_destination' is also > + responsible to free `destination' buffer. */ > + if (coding->raw_destination) > + coding->dst_object = Qnil; > + else > + { > + coding->dst_object > + = make_unibyte_string ((char *) coding->destination, > + coding->produced); > + xfree (coding->destination); > + } > } > } You can remove the { between "else" and "if" (with the side-benefit that the unchanged code will stay at the same indentation level). > if (CODING_REQUIRE_ENCODING (coding)) > { > - encode_coding_object (coding, string, > - start, string_char_to_byte (string, start), > - end, string_char_to_byte (string, end), Qt); > + ptrdiff_t nchars = min (end - start, E_WRITE_MAX); > + > + /* Avoid creating huge Lisp string in encode_coding_object. */ > + if (nchars == E_WRITE_MAX) > + coding->raw_destination = 1; > + > + encode_coding_object > + (coding, string, start, string_char_to_byte (string, start), > + start + nchars, string_char_to_byte (string, start + nchars), > + Qt); > } Where/how do you make sure we loop back here for the rest (when end-start > E_WRITE_MAX) ? Stefan