From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: David Ringo Newsgroups: gmane.emacs.bugs Subject: bug#37044: 27.0.50; url-http breaks behind proxy when reusing existing connection Date: Thu, 15 Aug 2019 19:36:44 -0600 Message-ID: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="0000000000008cbf9b05903206a3" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="115277"; mail-complaints-to="usenet@blaine.gmane.org" To: 37044@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Aug 16 06:50:22 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hyUBx-000ToX-Mv for geb-bug-gnu-emacs@m.gmane.org; Fri, 16 Aug 2019 06:50:21 +0200 Original-Received: from localhost ([::1]:49186 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyUBu-0002Ty-Pm for geb-bug-gnu-emacs@m.gmane.org; Fri, 16 Aug 2019 00:50:19 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:46358) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyUBg-0002Pi-U9 for bug-gnu-emacs@gnu.org; Fri, 16 Aug 2019 00:50:06 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hyUBe-0003OP-Sg for bug-gnu-emacs@gnu.org; Fri, 16 Aug 2019 00:50:04 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:44091) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hyUBe-0003OD-PT for bug-gnu-emacs@gnu.org; Fri, 16 Aug 2019 00:50:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hyUBe-0003Nv-K8 for bug-gnu-emacs@gnu.org; Fri, 16 Aug 2019 00:50:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: David Ringo Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 16 Aug 2019 04:50:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 37044 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.156593096912952 (code B ref -1); Fri, 16 Aug 2019 04:50:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 16 Aug 2019 04:49:29 +0000 Original-Received: from localhost ([127.0.0.1]:52910 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hyUB6-0003Mk-DG for submit@debbugs.gnu.org; Fri, 16 Aug 2019 00:49:29 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:59315) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hyRAs-00053T-UB for submit@debbugs.gnu.org; Thu, 15 Aug 2019 21:37:05 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:56726) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1hyRAq-0008MR-Oy for bug-gnu-emacs@gnu.org; Thu, 15 Aug 2019 21:37:02 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hyRAo-0004cB-Ef for bug-gnu-emacs@gnu.org; Thu, 15 Aug 2019 21:37:00 -0400 Original-Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]:54196) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hyRAo-0004bb-6R for bug-gnu-emacs@gnu.org; Thu, 15 Aug 2019 21:36:58 -0400 Original-Received: by mail-wm1-x341.google.com with SMTP id 10so2773628wmp.3 for ; Thu, 15 Aug 2019 18:36:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=g/Ksl+ztci0yZ+aI/sKlHZUxlwjnUUApgwXSNBjzs5w=; b=Fp+wh2r+wCTfViBP/n6hXOooEIhnqCHH2FuAS+cR24CHLOCYM5pq+5L+EC92bwjOdw 5Zzpu/RpmCrDYlaGKdbinTOwZMArR+aBmGKmqC2jTP4FKU+xyd8LTd4wjTa2VtHG2+Nz No5C0flpp2UZLGk1UjAiTwKpLnLrQ5WieZtK4pYigXUdJIQHfPmm34Kl9AO1khpYhy+O 8wRS8KiIvYHo8g+gTTCZB8n+lg8wm87ZyzKdjEEPbHuzBqdNP1KkxDM79FGA4nCrlo4m tiYAGj9/LBjGhfERoMkZIGgvRmC77Dv5I/yJADfIT5GcTZVOSer+kouKutwhNdQClC0D g5RQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=g/Ksl+ztci0yZ+aI/sKlHZUxlwjnUUApgwXSNBjzs5w=; b=lyH7u0JiRhvYuet8cZTqHXTwwBFAmJTH3n5pmrY6Vg/hxt1fTogW5Kk/vr3HFvWSeX y7XLQI/CUdfjqycbhhxZ2jHDJikDMKtCZXIEZi1Dc8iT7P0o/f/qtmflIaZY+l7Wz7g6 VBvqOmM5sZUkjFH+0dSlTcqfjZ/4yXUm0povTRi3b7R+KDX+hY1bVjW3mQKMhgbUlO8Q SUmpaDndoM5KokNzZHkTznCm3CF06k+O20d+Vjtgf8awZV0Ki3lQV2bni8rpiMNOs9uB 76req63TTZvLifUWsjGxe++2PaHGc/I5Rjkb7B2mLoLZVGPXHl61ioEdtI9ckZpwXd2a IpZA== X-Gm-Message-State: APjAAAXsdFJ3oGzZTQdC0zPFns5rbFXb1Fw7M1WICzdbES/WJlShNeDD 152JclikgfFkOFfQW2Qr9RId2AMciPRSplLeb0foTxMX0/U= X-Google-Smtp-Source: APXvYqwpE62Nz4vFJVUhtn0jMVDBKkWvO+gwkA6GiKFDHeZuQPuCTxGK5QMsIfXNgJ5eEW2X4hFyh0L43ku2mAXcibA= X-Received: by 2002:a7b:cf2d:: with SMTP id m13mr4774399wmg.120.1565919416203; Thu, 15 Aug 2019 18:36:56 -0700 (PDT) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Mailman-Approved-At: Fri, 16 Aug 2019 00:49:27 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:165181 Archived-At: --0000000000008cbf9b05903206a3 Content-Type: text/plain; charset="UTF-8" When trying to make more than one HTTPS request to the same host via `url-retrieve-synchronously' behind my work proxy, only the first request succeeds. Depending on the host, subsequent requests result in 400/404/302 and perhaps a couple other response codes. There's some minimal code below that can be run with emacs -Q, but I don't know how to replicate the proxy. It seems that commit 84613dae5c3 is at least partially responsible for this. It's a small change to `url-https-proxy-after-change-function' that binds `url-http-proxy' around the call to `url-http-create-request'. Without that binding, I see no errors. I tested this both in 26.2.90 which does not include that commit, and the revision in this report by simply redefining the function to the older version. This happens both when `url-proxy-services' is set manually and when it is set from environment variables. I'm not sure if this is a problem with Emacs, the proxy I'm behind, or if there is some configuration I should be aware of that already addresses this. Please let me know if there's any other useful information I can provide. Sample code and the corresponding url-http buffer headers follow. The specific URLs don't seem to matter as long they are HTTPS. The optional arguments also don't seem to make a difference. - David Code: (url-retrieve-synchronously "https://xkcd.com/1/") (url-retrieve-synchronously "https://xkcd.com/2/") First request (success): HTTP/1.1 200 OK Server: nginx Content-Type: text/html; charset=UTF-8 Last-Modified: Thu, 15 Aug 2019 11:24:39 GMT Expires: Thu, 15 Aug 2019 21:22:11 GMT Cache-Control: max-age=300 Content-Length: 3012 Accept-Ranges: bytes Date: Fri, 16 Aug 2019 01:32:01 GMT Via: 1.1 varnish Age: 142 Connection: keep-alive X-Served-By: cache-mdw17348-MDW X-Cache: HIT X-Cache-Hits: 1 X-Timer: S1565919121.298692,VS0,VE0 Vary: Accept-Encoding Second request (failuer): HTTP/1.1 400 Bad Request Server: nginx Content-Type: text/html; charset=UTF-8 Content-Length: 166 Accept-Ranges: bytes Date: Fri, 16 Aug 2019 01:32:01 GMT Via: 1.1 varnish Connection: keep-alive X-Served-By: cache-mdw17348-MDW X-Cache: MISS X-Cache-Hits: 0 X-Timer: S1565919121.349321,VS0,VE26 In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu) of 2019-08-07 built on slartibartfast Repository revision: d5622eb6fff94714c5d5a64c98c5e02bc1be478c Repository branch: master Windowing system distributor 'Fedora Project', version 11.0.12005000 System Description: Fedora 30 (Workstation Edition) Recent messages: Making completion list... Quit Note: file is write protected View mode: type C-h for help, h for commands, q to quit. previous-line: Beginning of buffer [2 times] Mark saved where search started Note: file is write protected View mode: type C-h for help, C-h m for commands, M-x View-quit to quit. Quit [3 times] Mark saved where search started Quit Configured using: 'configure --with-imagemagick --with-modules --with-x-toolkit=no' Configured features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB OLDXMENU X11 XDBE XIM MODULES THREADS PDUMPER LCMS2 GMP Important settings: value of $LANG: en_US.UTF-8 value of $XMODIFIERS: @im=ibus locale-coding-system: utf-8-unix Major mode: Info Minor modes in effect: tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t buffer-read-only: t line-number-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message dired dired-loaddefs format-spec rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail rmail-loaddefs text-property-search time-date mm-decode mm-bodies mm-encode mailabbrev gmm-utils mailheader sendmail goto-addr thingatpt mule-util jka-compr info noutline outline easy-mmode view help-mode easymenu misearch multi-isearch mail-utils gnutls network-stream url-cache url-http url-auth mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr url-gw nsm rmc puny url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib url-vars mailcap tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify lcms2 dynamic-setting system-font-setting font-render-setting x multi-tty make-network-process emacs) Memory information: ((conses 16 76232 12727) (symbols 48 8198 1) (strings 32 28196 1967) (string-bytes 1 800056) (vectors 16 13427) (vector-slots 8 162016 12678) (floats 8 43 388) (intervals 56 1044 0) (buffers 992 19)) --0000000000008cbf9b05903206a3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
When trying to make more than one HTTPS request to th= e same host via `url-retrieve-synchronously' behind my work proxy, only= the first request succeeds.=C2=A0 Depending on the host, subsequent reques= ts result in 400/404/302 and perhaps a couple other response codes.=C2=A0 T= here's some minimal code below that can be run with emacs -Q, but I don= 't know how to replicate the proxy.

It seems t= hat commit=C2=A084613dae5c3 is at least partially responsible for this.=C2= =A0 It's a small change to `url-https-proxy-after-change-function' = that binds `url-http-proxy' around the call to `url-http-create-request= '.=C2=A0 Without that binding, I see no errors.=C2=A0 I tested this bot= h in 26.2.90 which does not include that commit, and the revision in this r= eport by simply redefining the function to the older version.
This happens both when `url-proxy-services' is set manually= and when it is set from environment variables.

I&= #39;m not sure if this is a problem with Emacs, the proxy I'm behind, o= r if there is some configuration I should be aware of that already addresse= s this.=C2=A0 Please let me know if there's any other useful informatio= n I can provide.

Sample code and the corresponding= url-http buffer headers follow.=C2=A0 The specific URLs don't seem to = matter as long they are HTTPS.=C2=A0 The optional arguments also don't = seem to make a difference.

- David

<= /div>
Code:
(url-retrieve-synchronously "https://xkcd.com/1/")
(url-retrieve= -synchronously "https://xkcd.com/2/")



In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu)
=C2=A0of 20= 19-08-07 built on slartibartfast
Repository revision: d5622eb6fff94714c5= d5a64c98c5e02bc1be478c
Repository branch: master
Windowing system dis= tributor 'Fedora Project', version 11.0.12005000
System Descript= ion: Fedora 30 (Workstation Edition)

Recent messages:
Making comp= letion list...
Quit
Note: file is write protected
View mode: type = C-h for help, h for commands, q to quit.
previous-line: Beginning of buf= fer [2 times]
Mark saved where search started
Note: file is write pro= tected
View mode: type C-h for help, C-h m for commands, M-x View-quit t= o quit.
Quit [3 times]
Mark saved where search started
Quit
Con= figured using:
=C2=A0'configure --with-imagemagick --with-modules --= with-x-toolkit=3Dno'

Configured features:
XPM JPEG TIFF GIF P= NG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS GLIB
NOTIFY INOTIFY ACL LIB= SELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT
LIBOTF XFT ZLIB OLDXME= NU X11 XDBE XIM MODULES THREADS PDUMPER LCMS2 GMP

Important settings= :
=C2=A0 value of $LANG: en_US.UTF-8
=C2=A0 value of $XMODIFIERS: @im= =3Dibus
=C2=A0 locale-coding-system: utf-8-unix

Major mode: Info<= br>
Minor modes in effect:
=C2=A0 tooltip-mode: t
=C2=A0 global-el= doc-mode: t
=C2=A0 electric-indent-mode: t
=C2=A0 mouse-wheel-mode: t=
=C2=A0 tool-bar-mode: t
=C2=A0 menu-bar-mode: t
=C2=A0 file-name-= shadow-mode: t
=C2=A0 global-font-lock-mode: t
=C2=A0 font-lock-mode:= t
=C2=A0 blink-cursor-mode: t
=C2=A0 auto-composition-mode: t
=C2= =A0 auto-encryption-mode: t
=C2=A0 auto-compression-mode: t
=C2=A0 bu= ffer-read-only: t
=C2=A0 line-number-mode: t
=C2=A0 transient-mark-mo= de: t

Load-path shadows:
None found.

Features:
(shadow = sort mail-extr emacsbug message dired dired-loaddefs format-spec
rfc822 = mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs te= xt-property-search time-date mm-decode mm-bodies
mm-encode mailabbrev gm= m-utils mailheader sendmail goto-addr thingatpt
mule-util jka-compr info= noutline outline easy-mmode view help-mode
easymenu misearch multi-isea= rch mail-utils gnutls network-stream
url-cache url-http url-auth mail-pa= rse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr url-gw nsm rmc= puny url url-proxy url-privacy
url-expand url-methods url-history url-c= ookie url-domsuf url-util
url-parse auth-source cl-seq eieio eieio-core = cl-macs eieio-loaddefs
password-cache json subr-x map seq byte-opt gv by= tecomp byte-compile
cconv cl-loaddefs cl-lib url-vars mailcap tooltip el= doc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-= win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fr= inge
tabulated-list replace newcomment text-mode elisp-mode lisp-modeprog-mode register page menu-bar rfn-eshadow isearch timer select
scrol= l-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors= frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan = thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian= slovak czech european ethiopic indian cyrillic
chinese composite charsc= ript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarr= ay minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroe= xp files text-properties overlay sha1 md5
base64 format env code-pages m= ule custom widget hashtable-print-readable
backquote threads dbusbind in= otify lcms2 dynamic-setting
system-font-setting font-render-setting x mu= lti-tty make-network-process
emacs)

Memory information:
((cons= es 16 76232 12727)
=C2=A0(symbols 48 8198 1)
=C2=A0(strings 32 28196 = 1967)
=C2=A0(string-bytes 1 800056)
=C2=A0(vectors 16 13427)
=C2= =A0(vector-slots 8 162016 12678)
=C2=A0(floats 8 43 388)
=C2=A0(inter= vals 56 1044 0)
=C2=A0(buffers 992 19))
--0000000000008cbf9b05903206a3--