From mboxrd@z Thu Jan 1 00:00:00 1970 Path: main.gmane.org!not-for-mail From: Kevin Ryde Newsgroups: gmane.lisp.guile.devel Subject: string port slow output on big string Date: Mon, 14 Feb 2005 11:22:04 +1100 Message-ID: <874qggvvmb.fsf@zip.com.au> NNTP-Posting-Host: main.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: sea.gmane.org 1108342047 9971 80.91.229.2 (14 Feb 2005 00:47:27 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Mon, 14 Feb 2005 00:47:27 +0000 (UTC) Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Mon Feb 14 01:47:27 2005 Original-Received: from lists.gnu.org ([199.232.76.165]) by ciao.gmane.org with esmtp (Exim 4.43) id 1D0UOV-0004f5-4Y for guile-devel@m.gmane.org; Mon, 14 Feb 2005 01:47:19 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1D0Udw-0005oa-8i for guile-devel@m.gmane.org; Sun, 13 Feb 2005 20:03:16 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1D0UZr-0003hO-CA for guile-devel@gnu.org; Sun, 13 Feb 2005 19:59:03 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1D0UZn-0003fZ-Vb for guile-devel@gnu.org; Sun, 13 Feb 2005 19:59:01 -0500 Original-Received: from [199.232.76.173] (helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1D0UXR-00030G-Ri for guile-devel@gnu.org; Sun, 13 Feb 2005 19:56:34 -0500 Original-Received: from [61.8.0.84] (helo=mailout1.pacific.net.au) by monty-python.gnu.org with esmtp (Exim 4.34) id 1D0U0F-0002fD-GV for guile-devel@gnu.org; Sun, 13 Feb 2005 19:22:16 -0500 Original-Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.0.87]) by mailout1.pacific.net.au (8.12.3/8.12.3/Debian-7.1) with ESMTP id j1E0MBA6019749 for ; Mon, 14 Feb 2005 11:22:11 +1100 Original-Received: from localhost (ppp2FFC.dyn.pacific.net.au [61.8.47.252]) by mailproxy2.pacific.net.au (8.12.3/8.12.3/Debian-7.1) with ESMTP id j1E0MAGf009353 for ; Mon, 14 Feb 2005 11:22:10 +1100 Original-Received: from gg by localhost with local (Exim 3.36 #1 (Debian)) id 1D0U04-0000Vw-00; Mon, 14 Feb 2005 11:22:04 +1100 Original-To: guile-devel@gnu.org Mail-Copies-To: never User-Agent: Gnus/5.110003 (No Gnus v0.3) Emacs/21.3 (gnu/linux) X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org X-MailScanner-To: guile-devel@m.gmane.org Xref: main.gmane.org gmane.lisp.guile.devel:4779 X-Report-Spam: http://spam.gmane.org/gmane.lisp.guile.devel:4779 --=-=-= I tried writing a biggish string to a string port, and it was very slow. Eg. (use-modules (ice-9 time)) (let ((str (make-string 100000 #\x))) (call-with-output-string (lambda (port) (time (display str port)))) #f) gives on my poor 333mhz clock utime stime cutime cstime gctime 7.63 7.58 0.05 0.00 0.00 4.17 I struck this trying to use regexp-substitute/global on a file slurped into memory. It was 130k, which is a decent size, but it's well within the realm of reason. I think strports.c st_write ends up doing a realloc and copy every 80 bytes of the block it's writing. It knows the size, but it lets st_flush just grow by 80 bytes at a time. The change below speeds it up from 7 seconds to 10 ms for me. But I don't know if the read side bits of this change are right. Is it supposed to update read_pos, read_end and read_buf_size to be the end of the string, or something? (Of course what would be even nicer would be to avoid big reallocing altogether, like keep a list of chunks and only join them when a get-string call wants the entire block. But that can wait.) --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=strports.c.write.diff --- strports.c.~1.105.~ 2005-01-28 08:25:34.000000000 +1100 +++ strports.c 2005-02-14 11:20:05.000000000 +1100 @@ -142,18 +142,14 @@ scm_t_port *pt = SCM_PTAB_ENTRY (port); const char *input = (char *) data; - while (size > 0) - { - int space = pt->write_end - pt->write_pos; - int write_len = (size > space) ? space : size; - - memcpy ((char *) pt->write_pos, input, write_len); - pt->write_pos += write_len; - size -= write_len; - input += write_len; - if (write_len == space) - st_flush (port); - } + /* if not enough room for "size" then make that amount and an additional + SCM_WRITE_BLOCK */ + if (size > pt->write_end - pt->write_pos) + st_resize_port (pt, pt->write_pos - pt->write_buf + + size + SCM_WRITE_BLOCK); + + memcpy ((char *) pt->write_pos, input, size); + pt->write_pos += size; } static void --=-=-= Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Guile-devel mailing list Guile-devel@gnu.org http://lists.gnu.org/mailman/listinfo/guile-devel --=-=-=--