From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: =?UTF-8?Q?Cl=c3=a9ment_Pit--Claudel?= Newsgroups: gmane.emacs.devel Subject: Help understanding the URL code Date: Tue, 5 Apr 2016 16:37:22 +0100 Message-ID: <5703DBB2.1030903@gmail.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xDVOXN3LIiqmCwVS5vvDnQM9Ac6DDxHEu" X-Trace: ger.gmane.org 1459870872 20813 80.91.229.3 (5 Apr 2016 15:41:12 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 5 Apr 2016 15:41:12 +0000 (UTC) To: Emacs developers Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Apr 05 17:41:04 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1anT4y-0000jn-BH for ged-emacs-devel@m.gmane.org; Tue, 05 Apr 2016 17:39:44 +0200 Original-Received: from localhost ([::1]:37869 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anT4x-0002Kp-Oj for ged-emacs-devel@m.gmane.org; Tue, 05 Apr 2016 11:39:43 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:56194) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anT2n-0006vB-My for emacs-devel@gnu.org; Tue, 05 Apr 2016 11:37:31 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1anT2j-0000e7-7N for emacs-devel@gnu.org; Tue, 05 Apr 2016 11:37:29 -0400 Original-Received: from mout.kundenserver.de ([212.227.126.135]:53013) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1anT2i-0000dq-TE for emacs-devel@gnu.org; Tue, 05 Apr 2016 11:37:25 -0400 Original-Received: from [172.30.34.146] ([194.81.217.65]) by mrelayeu.kundenserver.de (mreue004) with ESMTPSA (Nemesis) id 0Lg4q7-1bb6eu40ip-00pavu for ; Tue, 05 Apr 2016 17:37:24 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 X-Provags-ID: V03:K0:KfHyK+iaZ5YQSfgHtvpJBzgvfJdLkpZ2yet/iud3myjqPVIlV5M f8CbRHHAPgn4jqesffFSRly02SN/+AzfLN8ZwnWpjrU7/5+mhCdrbR13HJMVTDgSKjpuuyY 4rPumEWvkHsVpgAuxjCpsvfhsdkqpr+AdE6eiB+ylN3TNQIgVsVC6xno/GrFAtqkIJY2gaB aNfud2l3qUC6SWA5GqRfg== X-UI-Out-Filterresults: notjunk:1;V01:K0:IIk67o1TikE=:aCRtlsLLCzqJUfHj/gj5ZG fzQ0eWU9/5VG8gyFQQs9x2qtdRnPjvm+rdLfrLQx2n03+wZfGx7RwTWpVSWN04NFLgiJyYI14 CLinyxSCsJYqoAIf+ZnSjoJobuWOsCtno/YG4gBZvrvMXfUd/SnleWLLZKWKklEKc2WO5UBRR b3+pNZ4jHw7avyK2FGpFNIPaLz1VXrwFMmbGcM9PqtACFfxbgbcy8TFNHR6UFFtRMNaSNMgs8 VXLRb3AXrqCUGnHApi34KmifbauPrNkIoZ1OzVn1A4qBVlbuin7kO3baaFtCGTAZs/LRM4zQk gcRdvKphptiepfFBoah8WXphVll72xog7VggFfZ+Qr0LONfgwRN0TA3J+42bY5xTFywjaSiw6 +IwlwWdeKU5TT9mmWR4rgXE2hmLHQ/aDTfffFPKp1iWe+lDWDtoJSKWUueUJBVbMFHMEbmDmE PuF2FzmA+wMLLySG7auVIS5dMVHbdmB+mw1hSLWNw060V7XEnWkW1taSxm9uC7eDfDXp2daqz eyPI0lCdf+YccU3hk1d6QvrOlgCRvXecVSEP66ULVLwOvfIoHh6e9uVryjTBNZ2t0jCxJfz5R yPVHIeH5QnNwKnSOwtFQUNYhQ2ctdRLQcPFMNn7VVrApUMMHBSR3arIDg5E47DgJSNUl2xGZg MQhHsRcfWMbaf+WTcE3nOblq782EMMAoRuDGfFHhzAbHBdb53jI94oqTPPWwKNsCNzsU= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 212.227.126.135 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:202744 Archived-At: This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --xDVOXN3LIiqmCwVS5vvDnQM9Ac6DDxHEu Content-Type: multipart/mixed; boundary="K0cSTl7Whjr8GfEmNnulH5AtLf0wwubpI" From: =?UTF-8?Q?Cl=c3=a9ment_Pit--Claudel?= To: Emacs developers Message-ID: <5703DBB2.1030903@gmail.com> Subject: Help understanding the URL code --K0cSTl7Whjr8GfEmNnulH5AtLf0wwubpI Content-Type: multipart/mixed; boundary="------------020708040604070809000202" This is a multi-part message in MIME format. --------------020708040604070809000202 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi emacs-devel, I'm looking into producing a patch to make it easier to customize the use= r-agent string that URL uses, but I'm hitting a number of walls. For exam= ple, what does this piece of code achieve? (defcustom url-user-agent (format "User-Agent: %sURL/%s\r\n" (if url-package-name (concat url-package-name "/" url-package-version " ") "") url-version) "User Agent used by the URL package for HTTP/HTTPS requests Should be a string or a function of no arguments returning a string."= :type '(choice (string :tag "A static User-Agent string") (function :tag "Call a function to get the User-Agen= t string")) :version "25.1" :group 'url) This defcustom is evaluated the first time that url-vars.el is loaded, ri= ght? In that case, what's the point of the url-package-name and url-packa= ge-version variables? It seems that rebinding them won't ever do anything= , since the definition is evaluated once and for all. Thus IIUC this bit = of code will return different values depending on whether the url package= had already been loaded before or whether it was loaded through url-retr= ieve-synchronously being autoloaded. Is that right? (let ((url-package-name "MyPackage")) (url-retrieve-synchronously "http://example.com") url-user-agent) If that's correct, then the attached (draft) patch might help in fixing t= his issue. I would also welcome help on other issues, such as #23140 (about being un= able to let-bind url-mime-accept-string). Am I missing something in how t= hese new parts of url.el are supposed to be used?=20 Cl=C3=A9ment. --------------020708040604070809000202 Content-Type: text/x-diff; name="user-agent.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="user-agent.patch" diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el index 5832e92..f775e72 100644 --- a/lisp/url/url-http.el +++ b/lisp/url/url-http.el @@ -211,15 +211,33 @@ url-http-find-free-connection (if connection (url-http-mark-connection-as-busy host port connection)))) =20 +(defun url-http--user-agent-default-string () + "Compute a default User-Agent string based on `url-privacy-level'." + (let ((package-info (when url-package-name + (format "%s/%s" url-package-name url-package-ver= sion))) + (emacs-info (unless (and (listp url-privacy-level) + (memq 'emacs url-privacy-level)) + (format "GNUEmacs/%s" emacs-version))) + (os-info (unless (and (listp url-privacy-level) + (memq 'os url-privacy-level)) + (format "(%s; %s)" url-system-type url-os-type))) + (url-info (format "URL/%s" url-version))) + (string-join (delq nil (list package-info url-info + emacs-info os-info)) + " "))) + ;; Building an HTTP request (defun url-http-user-agent-string () (if (or (eq url-privacy-level 'paranoid) (and (listp url-privacy-level) (memq 'agent url-privacy-level))) "" - (if (functionp url-user-agent) - (funcall url-user-agent) - url-user-agent))) + (format "User-Agent: %s\r\n" + (string-trim + (cond + ((functionp url-user-agent) (funcall url-user-agent)) + ((stringp url-user-agent) url-user-agent) + (t (url-http--user-agent-default-string))))))) =20 (defun url-http-create-request (&optional ref-url) "Create an HTTP request for `url-http-target-url', referred to by REF-= URL." diff --git a/lisp/url/url-vars.el b/lisp/url/url-vars.el index 960a04a..48a4cbe 100644 --- a/lisp/url/url-vars.el +++ b/lisp/url/url-vars.el @@ -116,6 +116,7 @@ url-privacy-level Valid symbols are: email -- the email address os -- the operating system info +emacs -- the version of Emacs lastloc -- the last location agent -- do not send the User-Agent string cookies -- never accept HTTP cookies @@ -143,6 +144,7 @@ url-privacy-level (checklist :tag "Custom" (const :tag "Email address" :value email) (const :tag "Operating system" :value os) + (const :tag "Emacs version" :value emacs) (const :tag "Last location" :value lastloc) (const :tag "Browser identification" :value agent) (const :tag "No cookies" :value cookie))) @@ -357,13 +359,11 @@ url-gateway-method (const :tag "Direct connection" :value native)) :group 'url-hairy) =20 -(defcustom url-user-agent (format "User-Agent: %sURL/%s\r\n" - (if url-package-name - (concat url-package-name "/" - url-package-version " ") - "") url-version) - "User Agent used by the URL package for HTTP/HTTPS requests -Should be a string or a function of no arguments returning a string." +(defcustom url-user-agent nil + "User Agent used by the URL package for HTTP/HTTPS requests. +Should be a string not including the \"User-Agent:\", or a +function of no arguments returning a such string. +If nil, the value is calculated based on `url-privacy-level'." :type '(choice (string :tag "A static User-Agent string") (function :tag "Call a function to get the User-Agent s= tring")) :version "25.1" --------------020708040604070809000202-- --K0cSTl7Whjr8GfEmNnulH5AtLf0wwubpI-- --xDVOXN3LIiqmCwVS5vvDnQM9Ac6DDxHEu Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJXA9uyAAoJEPqg+cTm90wj1bMP/i/ZF31ssUQg/3eJCRAnMXed yDT/8MqMiRYQpqxMMLYeAmdzpXYjhClYKs8kVy9H6CvbSigY9JN11m7N9NFyYq/q Nung0Up+X+4QPlbjAYtCj1qbMIEE30DxzFYFYVNtTcMZnJomJHQN28z/+CGUl3dJ zUe+ZBRrIsBI/OEBjlg4+PdBdzVvDpKklBT9uFqDvGvARhWaOY0gm47gbrH8vHDS IO+OH6e1etK74KLLraQ5FiaUpWDss1sPF+NhcmrIQweA4Um8w066g8U8hIZQf/om Zc2/h/nwTaJMFuKEjtuxZi6oAB57WwVyxAmaz+lmBxjFkXz3DPRF5z+8GRAg2fn/ e2/XDScQTPhUo7OYwkH19gbASX8Gf7s9NI6lVY2FwDx+kMzMMyms3+rQE+Mk0dKw ysb04GTbaRSGTK4tFyLaeJpKUQJTNYwedAsykOHlWfI6UcXKLelDprdJ0DMHjTfG v9LO1RMWOkuXezAZi1A80a0GFGS+Ov39WUbLj3CNkFfuv4J6KuWZlhCIKteUWpXH sslScEifTos+5WOnAFXuQSTcamK5CpWO20Xu6rCcQguuUpuIFSKNt83db1FAsJfU a5Xa8G8SZEMnmf1bS9XGSzWjh7s7QIoePIWtG6KpRrrBZ/O5HhiIPBbpZX3GY7Q8 kyfzy93RaDnpTaUfD3I4 =MJIP -----END PGP SIGNATURE----- --xDVOXN3LIiqmCwVS5vvDnQM9Ac6DDxHEu--