From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Newsgroups: gmane.lisp.guile.bugs Subject: bug#15368: HTTP client is slow [2.0.9] Date: Fri, 13 Sep 2013 15:41:27 +0200 Message-ID: <87d2oc260o.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1379080040 22298 80.91.229.3 (13 Sep 2013 13:47:20 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Fri, 13 Sep 2013 13:47:20 +0000 (UTC) To: 15368@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Fri Sep 13 15:47:22 2013 Return-path: Envelope-to: guile-bugs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1VKTiU-00076U-7Z for guile-bugs@m.gmane.org; Fri, 13 Sep 2013 15:47:22 +0200 Original-Received: from localhost ([::1]:47108 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VKTiT-0000V4-QI for guile-bugs@m.gmane.org; Fri, 13 Sep 2013 09:47:21 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:40259) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VKTiJ-0000T3-JZ for bug-guile@gnu.org; Fri, 13 Sep 2013 09:47:18 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VKTiA-0003J4-Ba for bug-guile@gnu.org; Fri, 13 Sep 2013 09:47:11 -0400 Original-Received: from debbugs.gnu.org ([140.186.70.43]:53071) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VKTiA-0003J0-8L for bug-guile@gnu.org; Fri, 13 Sep 2013 09:47:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1VKTiA-000880-17 for bug-guile@gnu.org; Fri, 13 Sep 2013 09:47:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Fri, 13 Sep 2013 13:47:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 15368 X-GNU-PR-Package: guile X-GNU-PR-Keywords: X-Debbugs-Original-To: bug-guile@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.137908001731232 (code B ref -1); Fri, 13 Sep 2013 13:47:01 +0000 Original-Received: (at submit) by debbugs.gnu.org; 13 Sep 2013 13:46:57 +0000 Original-Received: from localhost ([127.0.0.1]:33129 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VKTi4-00087f-KL for submit@debbugs.gnu.org; Fri, 13 Sep 2013 09:46:57 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:38450) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1VKTi2-000874-KT for submit@debbugs.gnu.org; Fri, 13 Sep 2013 09:46:54 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VKThr-0003Fh-0v for submit@debbugs.gnu.org; Fri, 13 Sep 2013 09:46:49 -0400 Original-Received: from lists.gnu.org ([2001:4830:134:3::11]:59640) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VKThq-0003Fb-Ue for submit@debbugs.gnu.org; Fri, 13 Sep 2013 09:46:42 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:40123) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VKThl-0000SY-1z for bug-guile@gnu.org; Fri, 13 Sep 2013 09:46:42 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VKThf-0003E6-39 for bug-guile@gnu.org; Fri, 13 Sep 2013 09:46:36 -0400 Original-Received: from hera.aquilenet.fr ([141.255.128.1]:55191) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VKThe-0003E2-Sk for bug-guile@gnu.org; Fri, 13 Sep 2013 09:46:31 -0400 Original-Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 81F5F15BD for ; Fri, 13 Sep 2013 15:41:28 +0200 (CEST) Original-Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3XntG1Atvijz for ; Fri, 13 Sep 2013 15:41:28 +0200 (CEST) Original-Received: from pluto (reverse-83.fdn.fr [80.67.176.83]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 2106415BB for ; Fri, 13 Sep 2013 15:41:28 +0200 (CEST) User-Agent: Gnus/5.130007 (Ma Gnus v0.7) Emacs/24.3 (gnu/linux) X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 27 Fructidor an 221 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0xEA52ECF4 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 83C4 F8E5 10A3 3B4C 5BEA D15D 77DD 95E2 EA52 ECF4 X-OS: x86_64-unknown-linux-gnu X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy] X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Original-Sender: bug-guile-bounces+guile-bugs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.bugs:7281 Archived-At: I just noticed that our HTTP client is very slow. Consider this: --8<---------------cut here---------------start------------->8--- (use-modules (web client) (rnrs io ports) (rnrs bytevectors) (srfi srfi-11) (ice-9 format)) (define %uri "http://ftp.gnu.org/gnu/idutils/idutils-4.6.tar.xz") (with-fluids ((%default-port-encoding #f)) (let*-values (((start) (gettimeofday)) ((p) (let ((s (open-socket-for-uri %uri))) (setvbuf s _IONBF) s)) ((r h) (http-get %uri #:port p #:streaming? #t #:decode-body? #f)) ((d len) (let ((b (get-bytevector-all h))) (values b (bytevector-length b))) ;; (let ((b (make-bytevector (* 5 (expt 2 20))))) ;; (values b ;; (get-bytevector-n! h b 0 (bytevector-length b= )))) ) ((end) (gettimeofday)) ((throughput) (let ((duration (- (car end) (car start)))) (/ (/ len 1024.) duration 1.0)))) (format #t "~5,1f KiB/s (total: ~5,1f KiB)~%" throughput (/ len 1024.)))) --8<---------------cut here---------------end--------------->8--- Here I get a throughput of ~60 KiB/s, vs. ~400 KiB/s for wget and curl. Looking at the strace output reveals no real difference: they all make one syscall for each chunk of 1410 bytes. =E2=80=98time=E2=80=99 reports that Guile spends 0.2 s. in user and 0.8 s. = in system, both of which are an order of magnitude higher than wget/curl. Bypassing the custom binary input ports from http.scm and response.scm doesn=E2=80=99t make any big difference. Forcing the zero-copy path in =E2=80=98scm_c_read=E2=80=99 doesn=E2=80=99t help much either. Ideas? Thanks, Ludo=E2=80=99.