From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Richard Copley Newsgroups: gmane.emacs.bugs Subject: bug#27658: 26.0.50; response parsing hangs when using tls.el and gnutls-cli.exe on Windows Date: Tue, 11 Jul 2017 23:18:39 +0100 Message-ID: NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: blaine.gmane.org 1499811679 12875 195.159.176.226 (11 Jul 2017 22:21:19 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Tue, 11 Jul 2017 22:21:19 +0000 (UTC) To: 27658@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Jul 12 00:21:12 2017 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dV3Wn-0002ry-Br for geb-bug-gnu-emacs@m.gmane.org; Wed, 12 Jul 2017 00:21:09 +0200 Original-Received: from localhost ([::1]:49174 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dV3Ws-0004fV-TL for geb-bug-gnu-emacs@m.gmane.org; Tue, 11 Jul 2017 18:21:14 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:32846) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dV3Wk-0004fK-AA for bug-gnu-emacs@gnu.org; Tue, 11 Jul 2017 18:21:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dV3Wg-00070j-9l for bug-gnu-emacs@gnu.org; Tue, 11 Jul 2017 18:21:06 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:59988) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dV3Wg-00070Y-5h for bug-gnu-emacs@gnu.org; Tue, 11 Jul 2017 18:21:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1dV3Wf-0007Jg-Tm for bug-gnu-emacs@gnu.org; Tue, 11 Jul 2017 18:21:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Richard Copley Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 11 Jul 2017 22:21:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 27658 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: X-Debbugs-Original-To: "bug-gnu-emacs@gnu.org" Original-Received: via spool by submit@debbugs.gnu.org id=B.149981162628062 (code B ref -1); Tue, 11 Jul 2017 22:21:01 +0000 Original-Received: (at submit) by debbugs.gnu.org; 11 Jul 2017 22:20:26 +0000 Original-Received: from localhost ([127.0.0.1]:34432 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dV3W6-0007IY-GW for submit@debbugs.gnu.org; Tue, 11 Jul 2017 18:20:26 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:41760) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dV3W5-0007IC-Jf for submit@debbugs.gnu.org; Tue, 11 Jul 2017 18:20:25 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dV3Vz-0006iI-44 for submit@debbugs.gnu.org; Tue, 11 Jul 2017 18:20:20 -0400 Original-Received: from lists.gnu.org ([2001:4830:134:3::11]:59535) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dV3Vz-0006iA-0W for submit@debbugs.gnu.org; Tue, 11 Jul 2017 18:20:19 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:60934) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dV3Vx-0004bg-43 for bug-gnu-emacs@gnu.org; Tue, 11 Jul 2017 18:20:18 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dV3Vt-0006fJ-27 for bug-gnu-emacs@gnu.org; Tue, 11 Jul 2017 18:20:17 -0400 Original-Received: from mail-vk0-f44.google.com ([209.85.213.44]:33493) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dV3Vs-0006eW-Rv for bug-gnu-emacs@gnu.org; Tue, 11 Jul 2017 18:20:12 -0400 Original-Received: by mail-vk0-f44.google.com with SMTP id r126so3398416vkg.0 for ; Tue, 11 Jul 2017 15:20:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Ia9Go3Scb0Ai9BMBrhluEpDtGmwZpUsOaTLUOYcOeHc=; b=LJM6VBKIiu88+sgaMRjEHHHFgLFQkma0gx+4QlUWVvdUoqOdvHNt5eS/OOGeVvgSTY zWfg9jI5g3ILvRgq8e1j2AzO8IroJAXd0F50Xlo4SKtbwUvEdKjgxzOd6nBn1+DpiqwZ dNab87EONirtp/jthP7APor+WvvyxwnhzoyTRaqffmiI5v7yP1PnY13nXjScfWo2sswg hWHKFyEnkTyC0C3L3Gf/tx5mo1EZlTYo0Dcabuj+dAokzebZFm6cfdcYheSNfVoDe8Eu EL1ATSpinRlPV9VtO8Ab9w1ZpfSiCk28Nmb01Q9FnZ0KC3HN2nXJoXGmpcRy2ud/+Xuf C3+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Ia9Go3Scb0Ai9BMBrhluEpDtGmwZpUsOaTLUOYcOeHc=; b=litNl8o/RsZXl0dgJygjJyvEB9aJ5xf0RavHnxVpmoQYiV3qWqTlum474ooLb0iFlW MUDtqgcGSCjiUR3PiAfI5wtWwTkPYJKZq9mC6Vpv6odBXOFMmzndsEFwvoY7yeD68Q5C aoHx/aogogMhOrYLeoJ5ZYu5GMx2OB3MhlqjvRaAKZDFNv8xL0TJf1eMCjYL6cyYSQJn ZzUArTnn3FiI8MDZ6OUT93AfvoGTeyTp2/Dtg0lEWxpnVypDQe1kQr5BHmmCL01mrVVu b1po2eRabHFpB8T/4zx9KGa4kTJIVFQLSIJ0ixXTZuPtJ6FUc46pRrlLduYM4CQSEv+d QFIA== X-Gm-Message-State: AIVw113spyvvm1c3GgNUhm2TR7d+fFGeJbfUS/XIy8ZP9GMbIJw+6Db8 72aDcgYVHCAk2zQfK8nsuxOJn7YwFfiL X-Received: by 10.31.205.2 with SMTP id d2mr10150vkg.77.1499811549956; Tue, 11 Jul 2017 15:19:09 -0700 (PDT) Original-Received: by 10.176.78.1 with HTTP; Tue, 11 Jul 2017 15:18:39 -0700 (PDT) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:134441 Archived-At: Install GNUTLS. Obtain certificates and set `tls-program' as described in Step 3 of Glyph's blog post at "https://glyph.twistedmatrix.com/2015/11/editor-malware.html". Ensure Emacs is using "gnutls-cli.exe" instead of "libgnutls-30.dll" by (a) removing the DLL from your path or (b) configuring Emacs with the "--without-gnutls" configure option. In emacs -Q: (pop-to-buffer (url-retrieve "https://elpa.gnu.org/packages/archive-contents" #'ignore)) Expected result: a buffer is displayed and, soon afterwards, filled with the HTTP response (with the network CRLF line-endings in the raw response translated to ordinary newlines, without CR, in the buffer). Actual result: "Opening TLS connection [...]...done" is shown in the minibuffer and the function never returns. (Type C-g to quit). (This is probably the cause of #22929.) ---- Observing the process filter in the debugger, there is an extra carriage return before every newline in the process buffer of the gnutls-cli.exe process. By "extra", I mean the informational text from "gnutls-cli.exe" has one carriage return at the end of each line (where there should be none) and the HTTP headers have two carriage returns at the end of each line (where there should be one, because this is the raw payload and has network line endings). The extra newlines in the informational output cause `open-tls-stream' in "tls.el" to hang indefinitely because the regexp `tls-end-of-info' never matches. Fixing that regexp is not a solution. When I try that, the HTTP header parsing in "url-http.el" fails in a similar way. Possible fix: The other obvious experiment I tried was to add "(set-process-coding-system process 'raw-text-dos)" to `open-tls-stream', just after the call to `start-process'. (Line 238.) Having done that, I do get a reasonable HTML document (and no error) from `url-retrieve'. That's a data point for information. I'm not sure if it's a good fix. In the blog post I referred to above, Glyph says: "[...] for some reason, Emacs could not parse gnutls-cli.exe s output no matter what I did." Indeed, with the change(s) above, "package.el" still doesn't succeed in parsing package lists retrieved over HTTPS; but that might be a separate bug. ---- In GNU Emacs 26.0.50 (build 1, x86_64-w64-mingw32) of 2017-07-11 built on MACHINE Repository revision: 7168bd2957a567c235d1b8b57acb4325187bd542 Windowing system distributor 'Microsoft Corp.', version 10.0.15063 Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Configured using: 'configure --prefix=/mingw64 --config-cache --with-modules --without-gnutls --without-pop --enable-locallisppath=/c/emacs-lisp CFLAGS=-O3' Configured features: XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL LIBXML2 ZLIB TOOLKIT_SCROLL_BARS MODULES Important settings: value of $EMACSLOADPATH: c:\emacs-lisp; value of $LANG: ENG locale-coding-system: cp1252 Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message subr-x puny seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote w32notify w32 multi-tty make-network-process emacs) Memory information: ((conses 16 96670 10234) (symbols 56 20064 1) (miscs 48 39 87) (strings 32 29950 1608) (string-bytes 1 773498) (vectors 16 13875) (vector-slots 8 480519 8489) (floats 8 50 151) (intervals 56 240 3) (buffers 976 11))