From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: "Philip K." Newsgroups: gmane.emacs.devel Subject: Re: Emacs HTTP libraries [was: Re: How to contribute new package to GNU ELPA?] Date: Tue, 22 Dec 2020 11:38:31 +0100 Message-ID: <87v9cuf7mg.fsf@posteo.net> References: <831rfisebv.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="24093"; mail-complaints-to="usenet@ciao.gmane.io" Cc: adam@alphapapa.net, rms@gnu.org, arthur.miller@live.com, emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Dec 22 11:41:11 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1krf6N-000693-Ia for ged-emacs-devel@m.gmane-mx.org; Tue, 22 Dec 2020 11:41:11 +0100 Original-Received: from localhost ([::1]:46576 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1krf6M-0002ww-GT for ged-emacs-devel@m.gmane-mx.org; Tue, 22 Dec 2020 05:41:10 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:37000) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1krf3u-0001Uu-FY for emacs-devel@gnu.org; Tue, 22 Dec 2020 05:38:38 -0500 Original-Received: from mout02.posteo.de ([185.67.36.66]:53523) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1krf3r-0004bb-QF for emacs-devel@gnu.org; Tue, 22 Dec 2020 05:38:38 -0500 Original-Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 455992400FD for ; Tue, 22 Dec 2020 11:38:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1608633513; bh=7TLZfQyfp5Qiwr57NzpilEddNMZ4O10MGIJdDj8kexk=; h=From:To:Cc:Subject:Date:From; b=Yeg3ZM8Kegh/eb2UtfIwwbuoRN3yOBI9Y06IMPY7EJ7JMdmNzd7hx3foGHqI8BQ4W w+jlNglFuHALBXLYcxf5wPQQAK/Dh32tFjP+IloG1fjVeckKUs0VwGP4GBTKPf9ty5 RRhwMNG/RMGbtty5XJJEMiVC0AumzBc2nXXsuGTNN08hggk5esTp6NEfIWdtuaIB6P 4JQBHgphBtJVtuVnG3b88GzaXlgl6Bn2Ir/5tlxwbEW+6zBVFBFXNUCRLjghVB1gqW 9XRyJMDH0V3zGirytv2BykNy1Toi3BY/GDFcsk8Y4dlTmZeUXzhAeYAp1lLDWrmKif b+7+zFxQBCByQ== Original-Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4D0Xqr3Tb1z6tmG; Tue, 22 Dec 2020 11:38:32 +0100 (CET) In-Reply-To: <831rfisebv.fsf@gnu.org> (message from Eli Zaretskii on Tue, 22 Dec 2020 05:35:16 +0200) Received-SPF: pass client-ip=185.67.36.66; envelope-from=philipk@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:261472 Archived-At: Eli Zaretskii writes: >> From: "Philip K." >> Cc: arthur.miller@live.com, adam@alphapapa.net, rms@gnu.org, >> emacs-devel@gnu.org >> Date: Tue, 22 Dec 2020 00:51:08 +0100 >>=20 >> >> > Before this is taken as an indication that using one of these >> >> > libraries will automagically make Emacs as fast as the applications >> >> > like curl, we should carefully profile url.el and find out which >> >> > part(s) of it cause the slowness. Because it could well be that wh= at >> >> > makes url.el slow will also make Emacs using libcurl slow. >> >>=20 >> >> Maybe this library can be of help while testing/profiling >> > >> > We have a built-in profiler in Emacs. >>=20 >> In that case, gnutls-negotiate seems to be the most suspicious function, >> using over 50%-60% of the CPU time, at least on my machine. This also >> makes sense, as TLS sites seem to take longer to load than regular, >> non-encrypted ones. > > Please show the code you profiled and the fully expanded profile. I sadly coudln't reproduce it, but this time the critical section looked something like this: - url-retrieve-synchronously 212 52% - url-retrieve 149 36% - url-retrieve-internal 149 36% - url-http 136 33% - url-http-find-free-connection 135 33% - url-open-stream 135 33% open-network-stream 134 33% when evaluating=20 (url-retrieve-synchronously "http://textboard.org/sexp/prog/index") in the *scratch* buffer. I used emacs -Q (GNU Emacs 27.1 (build 1, x86_64-redhat-linux-gnu, GTK+ Version 3.24.22, cairo version 1.16.0) of 2020-08-21), but I don't know why that should make any difference. I repeated the same test on the master branch and the results were basically the same (=C2=B15%). Either way, this simple request took over 2.5 minutes, whereas curl requires a quarter of a second. Note that this is even unencrypted, so this is not even taking the encryption overhead into account. (FYI: I took textboard.org as an example, because I wrote a client for this site in Elisp, that often takes a while to load, so I could count on it for this test. An interesting observation is that the first request might take a while, but sometimes the connection time suddenly drops to basically nothing. I have yet to figure out what the reason is. :/) --=20 Philip K.