From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: =?UTF-8?Q?Linus_Bj=C3=B6rnstam?= Newsgroups: gmane.lisp.guile.devel,gmane.lisp.guile.user Subject: Re: Guile's time execution issues Date: Wed, 22 Apr 2020 15:47:36 +0200 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="50113"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Cyrus-JMAP/3.3.0-dev0-351-g9981f4f-fmstable-20200421v1 To: =?UTF-8?Q?Aleix_Conchillo_Flaqu=C3=A9?= , guile-user , guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Wed Apr 22 15:48:45 2020 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jRFk5-000CwI-4b for guile-devel@m.gmane-mx.org; Wed, 22 Apr 2020 15:48:45 +0200 Original-Received: from localhost ([::1]:51034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jRFk4-0008Ro-7R for guile-devel@m.gmane-mx.org; Wed, 22 Apr 2020 09:48:44 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:39350) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jRFjU-0008Qu-Sl for guile-devel@gnu.org; Wed, 22 Apr 2020 09:48:09 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.90_1) (envelope-from ) id 1jRFjQ-0007J5-AG for guile-devel@gnu.org; Wed, 22 Apr 2020 09:48:08 -0400 Original-Received: from out4-smtp.messagingengine.com ([66.111.4.28]:56489) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jRFjP-00076b-MI; Wed, 22 Apr 2020 09:48:03 -0400 Original-Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id E9F4F5C01C4; Wed, 22 Apr 2020 09:47:59 -0400 (EDT) Original-Received: from imap24 ([10.202.2.74]) by compute7.internal (MEProxy); Wed, 22 Apr 2020 09:47:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=veryfast.biz; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type:content-transfer-encoding; s=fm2; bh=k6MXx wyiIoDZH2tEeZTY2sDWYNWP604Ncjo7c7WpHgs=; b=M8bAgTFrLh7eWRNermwgN jsqjU6dD30NbnKfa2Xg4AsAxkACQs+Hvr38oSj93XcAyACPFeiGBnJR/dh5A6Lgs UA9Z6nsKUnu8AtYpgHgNYFeMQ+i+6Wr1JDxEqhD7kgL8Pw5dJvmETTIwMqezVx17 psXf/7bGV/vh4IRBzQIR8ZH6wj94JbrBHzpiWc96JVfe7m74PXx4IVNV3QLzkZOG D3FWbuzF9DMAarqndmeqqHftiY0AyXF/18cVFdsVlOhHrN1RBva3DE2Lud0BJrAM WdswtEIZCedcmwyJdx4nhxwuCC2GjMoiiTo/bw+VHptvVkiQaI7u7GFQJHVLgDfP A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm2; bh=k6MXxwyiIoDZH2tEeZTY2sDWYNWP604Ncjo7c7WpH gs=; b=bA3Q6xRS8QwfXLPI1YqjiIFmDpDe4I9KfR7CbjnUoCmDt+jBrUo3vvVGA 3+H//+01d46CvK5Y6mjfASNm5Rfy4kQBd+JV286E7r6YQHsoLmiZJkG0EmDQwqnN DWgFgAW14cLoGcC2K/4UPJcVCaTwsO0oHIibpAThqKSjg3Nr6Y2p5LHMXY2vbxpL Cu+sJPTugEiAPhPdj1J3SwueRmyXNKsnu6VmbST5ZEj5kZxpyrzYKeEPnvcqJgLP JwkjCJHs3btGZrWlWxypMD0ZB1+Awkyv7Zyfvq4g1qq0Lc+G1yxEGqnR36yDrLgb ROq4odrfXPJzz2mOckIWJzkMk7oQA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrgeejgdeigecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefofgggkfgjfhffhffvufgtgfesthhqredtreerjeenucfhrhhomhepnfhinhhu shgpuehjnphrnhhsthgrmhcuoehlihhnuhhsrdgsjhhorhhnshhtrghmsehvvghrhihfrg hsthdrsghiiieqnecuffhomhgrihhnpehgihhthhhusgdrtghomhenucevlhhushhtvghr ufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehlihhnuhhsrdgsjhhorhhnsh htrghmsehvvghrhihfrghsthdrsghiii X-ME-Proxy: Original-Received: by mailuser.nyi.internal (Postfix, from userid 501) id 7BC8F2000A3; Wed, 22 Apr 2020 09:47:59 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface In-Reply-To: Received-SPF: pass client-ip=66.111.4.28; envelope-from=linus.bjornstam@veryfast.biz; helo=out4-smtp.messagingengine.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/04/22 08:19:23 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 66.111.4.28 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.io gmane.lisp.guile.devel:20491 gmane.lisp.guile.user:16398 Archived-At: Hi Aleix Try throwing the result of json->SCM away: (let ((a (call-with...))) 1).= Then you can run a (gc) between runs. If the GC time increases then, ma= ybe guile cannot release the memory?? I don't know, but there is some sh= ared substring thing going on in guile, even though I think string-appen= d doesn't do that. Nice to see that my suggestions worked! --=20 Linus Bj=C3=B6rnstam On Wed, 22 Apr 2020, at 00:03, Aleix Conchillo Flaqu=C3=A9 wrote: > Hi, >=20 > I was trying to get some guile-json performance times loading large=20= > JSON file. However, I'm getting increasing numbers at each run, so I'm= =20 > wondering if I'm doing something wrong. Below you can see how the firs= t=20 > run took 19.95s and then running the same command kept increasing. >=20 > I'm running Guile 2.2.7 on macOS Catalina 10.15.3. >=20 > scheme@(guile-user)> (use-modules (json)) > scheme@(guile-user)> ,t (define a (call-with-input-file=20 > "/Users/aleix/Downloads/large-file.json" (lambda (port) (json->scm=20 > port)))) > ;; 19.956429s real time, 87.100982s run time. 75.270202s spent in GC. > ;; 26.173179s real time, 143.645265s run time. 131.022631s spent in GC= . > ;; 28.193926s real time, 154.758375s run time. 141.697236s spent in GC= . > ;; 29.044218s real time, 160.745984s run time. 147.449073s spent in GC= . > ;; 30.480873s real time, 170.855527s run time. 157.332793s spent in GC= . > ;; 30.555700s real time, 172.938278s run time. 159.468737s spent in GC= . > ;; 32.190478s real time, 172.807551s run time. 158.905645s spent in GC= . >=20 > The good news is that I have applied a suggestion from Linus Bj=C3=B6r= nstam=20 > that reduces times by half (considering the first run above). The=20 > suggestion was to use string ports instead of string-append. And=20 > interestingly this time, time executions remain around the same value:= >=20 > scheme@(guile-user)> ,t (define a (call-with-input-file=20 > "/Users/aleix/Downloads/large-file.json" (lambda (port) (json->scm=20 > port)))) > ;; 9.099597s real time, 17.239176s run time. 9.309819s spent in GC. > ;; 9.927868s real time, 20.855859s run time. 12.528727s spent in GC. > ;; 8.981248s real time, 15.043991s run time. 7.050269s spent in GC. > ;; 9.055893s real time, 15.400981s run time. 7.383187s spent in GC. > ;; 9.055850s real time, 15.261824s run time. 7.239188s spent in GC. > ;; 9.315239s real time, 15.231889s run time. 7.005570s spent in GC. > ;; 9.106168s real time, 14.987678s run time. 6.915028s spent in GC. > ;; 9.175142s real time, 14.471324s run time. 6.302749s spent in GC. > ;; 8.966537s real time, 14.400318s run time. 6.412858s spent in GC. >=20 > JSON test data obtained from: >=20 > https://github.com/json-iterator/test-data/blob/master/large-file.json= >=20 > Best, >=20 > Aleix