From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#15368: HTTP client is slow [2.0.9] Date: Fri, 13 Sep 2013 23:14:07 +0200 Message-ID: <87wqmkwhk0.fsf@gnu.org> References: <87d2oc260o.fsf@gnu.org> <8738p8ln8a.fsf@tines.lan> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: ger.gmane.org 1379106915 24380 80.91.229.3 (13 Sep 2013 21:15:15 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 13 Sep 2013 21:15:15 +0000 (UTC) Cc: 15368@debbugs.gnu.org To: Mark H Weaver Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Fri Sep 13 23:15:18 2013 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1VKahx-0002Rh-Rx for guile-bugs@m.gmane.org; Fri, 13 Sep 2013 23:15:18 +0200 Original-Received: from localhost ([::1]:51276 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VKahx-00068o-1C for guile-bugs@m.gmane.org; Fri, 13 Sep 2013 17:15:17 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:43182) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VKaho-00068O-O2 for bug-guile@gnu.org; Fri, 13 Sep 2013 17:15:14 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VKahj-0002qA-EG for bug-guile@gnu.org; Fri, 13 Sep 2013 17:15:08 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:54683) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VKahj-0002pc-9x for bug-guile@gnu.org; Fri, 13 Sep 2013 17:15:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1VKahi-0003sW-RP for bug-guile@gnu.org; Fri, 13 Sep 2013 17:15:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Fri, 13 Sep 2013 21:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 15368 X-GNU-PR-Package: guile X-GNU-PR-Keywords: Original-Received: via spool by 15368-submit@debbugs.gnu.org id=B15368.137910685514834 (code B ref 15368); Fri, 13 Sep 2013 21:15:02 +0000 Original-Received: (at 15368) by debbugs.gnu.org; 13 Sep 2013 21:14:15 +0000 Original-Received: from localhost ([127.0.0.1]:34742 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VKagv-0003rA-Gs for submit@debbugs.gnu.org; Fri, 13 Sep 2013 17:14:15 -0400 Original-Received: from hera.aquilenet.fr ([141.255.128.1]:40095) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VKags-0003qx-Bw for 15368@debbugs.gnu.org; Fri, 13 Sep 2013 17:14:11 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 8E52D151B; Fri, 13 Sep 2013 23:14:08 +0200 (CEST) Original-Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qWG7kENTsnkt; Fri, 13 Sep 2013 23:14:08 +0200 (CEST) Original-Received: from pluto (reverse-83.fdn.fr [80.67.176.83]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 225341512; Fri, 13 Sep 2013 23:14:07 +0200 (CEST) X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 27 Fructidor an 221 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0xEA52ECF4 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 83C4 F8E5 10A3 3B4C 5BEA D15D 77DD 95E2 EA52 ECF4 X-OS: x86_64-unknown-linux-gnu In-Reply-To: <8738p8ln8a.fsf@tines.lan> (Mark H. Weaver's message of "Fri, 13 Sep 2013 12:07:01 -0400") User-Agent: Gnus/5.130007 (Ma Gnus v0.7) Emacs/24.3 (gnu/linux) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:7283 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mark H Weaver skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> I just noticed that our HTTP client is very slow. Consider this: >> >> (use-modules (web client) >> (rnrs io ports) >> (rnrs bytevectors) >> (srfi srfi-11) >> (ice-9 format)) >> >> (define %uri >> "http://ftp.gnu.org/gnu/idutils/idutils-4.6.tar.xz") >> >> (with-fluids ((%default-port-encoding #f)) >> (let*-values (((start) >> (gettimeofday)) >> ((p) >> (let ((s (open-socket-for-uri %uri))) >> (setvbuf s _IONBF) > > Why are you using an unbuffered port? On my system, changing this to > _IOFBF increases throughput from 326 KiB/s to 489.0 KiB/s. Arf, that=E2=80=99s because I was also forcing the =E2=80=98scm_c_read=E2= =80=99 hack (which is currently never used, and this is a bug): --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/libguile/ports.c b/libguile/ports.c index 9068c5c..c217712 100644 --- a/libguile/ports.c +++ b/libguile/ports.c @@ -1657,7 +1657,8 @@ scm_c_read (SCM port, void *buffer, size_t size) requested number of bytes. (Note that a single scm_i_fill_input call does not guarantee to fill the whole of the port's read buffer.) */ - if (pt->read_buf_size <= 1 && pt->encoding == NULL) + if (pt->read_buf_size <= 1 + && (pt->encoding == NULL || strcmp (pt->encoding, "ISO-8859-1") == 0)) { /* The port that we are reading from is unbuffered - i.e. does not have its own persistent buffer - but we have a buffer, --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable So in practice it was reading several KiB at a time, doing zero-copy. > Also, the fact that my throughput is so much higher than yours (on a > several-year-old computer) is interesting. Obviously I have a faster > net connection (wget reports 1.19M/s), So for you wget is ~2.5 times faster than Guile, right? [...] >> Looking at the strace output reveals no real difference: they all make >> one syscall for each chunk of 1410 bytes. >> >> =E2=80=98time=E2=80=99 reports that Guile spends 0.2 s. in user and 0.8 = s. in system, >> both of which are an order of magnitude higher than wget/curl. > > If they make essentially the same syscalls, then why would the system > time be an order of magnitude higher? Something doesn't sound right > here. I concur. I=E2=80=99ve tried Linux perf and OProfile but failed to get useful info. Ludo=E2=80=99. --=-=-=--