From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Nic Ferrier Newsgroups: gmane.emacs.devel Subject: https and emacs and package archives Date: Mon, 27 Oct 2014 17:07:42 +0000 Message-ID: <874muph25d.fsf@ferrier.me.uk> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1414429702 7424 80.91.229.3 (27 Oct 2014 17:08:22 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 27 Oct 2014 17:08:22 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Oct 27 18:08:17 2014 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Xinm6-00079O-GW for ged-emacs-devel@m.gmane.org; Mon, 27 Oct 2014 18:08:10 +0100 Original-Received: from localhost ([::1]:34963 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xinm6-0001F9-1y for ged-emacs-devel@m.gmane.org; Mon, 27 Oct 2014 13:08:10 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51699) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xinlo-0001C2-9T for emacs-devel@gnu.org; Mon, 27 Oct 2014 13:07:57 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Xinlj-0001CZ-7O for emacs-devel@gnu.org; Mon, 27 Oct 2014 13:07:52 -0400 Original-Received: from static.17.66.46.78.clients.your-server.de ([78.46.66.17]:48136 helo=po1.ferrier.me.uk) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Xinli-0001CB-Ui for emacs-devel@gnu.org; Mon, 27 Oct 2014 13:07:47 -0400 Original-Received: from nicferrier-dell-xps (140.35.155.90.in-addr.arpa [90.155.35.140]) by po1.ferrier.me.uk (Postfix) with ESMTPA id 537C9AC0498; Mon, 27 Oct 2014 18:25:09 +0100 (CET) Original-Received: from nicferrier-XPS13-9333 (localhost [127.0.0.1]) by nicferrier-dell-xps (Postfix) with ESMTPS id 4EA0060352; Mon, 27 Oct 2014 17:07:42 +0000 (GMT) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 78.46.66.17 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:175897 Archived-At: I moved marmalade-repo to HTTPS. Ever since a few people with 24.4 have been having trouble. And today I experienced a really strange thing with 24.4. I wrote this code to be used to automatically download packages: (when (member "elpakit-run.el" (mapcar 'file-name-nondirectory command-line-args)) (let ((package-user-dir (make-temp-name "elpakit-run"))) (package-initialize) (add-to-list 'package-archives '("marmalade" . "https://marmalade-repo.org/packages/")) (package-refresh-contents) (package-install 'elpakit))) And here's what happened when I ran it: $ ~/emacs-24-4/bin/emacs -batch -l ~/work/elpakit/elpakit-run.el ("/home/nicferrier/emacs-24-4/bin/emacs" "-l" "/home/nicferrier/work/elpakit/elpakit-run.el") Importing package-keyring.gpg... Importing package-keyring.gpg...done Contacting host: marmalade-repo.org:443 Contacting host: elpa.gnu.org:80 Making version-control local to dash-autoloads.el while let-bound! Generating autoloads for dash.el... Generating autoloads for dash.el...done Saving file /home/nicferrier/scratch/elpakit-run15942qZn/dash-2.9.0/dash-autoloads.el... Wrote /home/nicferrier/scratch/elpakit-run15942qZn/dash-2.9.0/dash-autoloads.el Checking /home/nicferrier/scratch/elpakit-run15942qZn/dash-2.9.0... Compiling /home/nicferrier/scratch/elpakit-run15942qZn/dash-2.9.0/dash-autoloads.el... Compiling /home/nicferrier/scratch/elpakit-run15942qZn/dash-2.9.0/dash-pkg.el... Wrote /home/nicferrier/scratch/elpakit-run15942qZn/dash-2.9.0/dash-pkg.elc Compiling /home/nicferrier/scratch/elpakit-run15942qZn/dash-2.9.0/dash.el... Wrote /home/nicferrier/scratch/elpakit-run15942qZn/dash-2.9.0/dash.elc Done (Total of 2 files compiled, 1 skipped) Contacting host: marmalade-repo.org:443 Contacting host: marmalade-repo.org:443 Generating autoloads for anaphora.el... Generating autoloads for anaphora.el...done Saving file /home/nicferrier/scratch/elpakit-run15942qZn/anaphora-1.0.0/anaphora-autoloads.el... Wrote /home/nicferrier/scratch/elpakit-run15942qZn/anaphora-1.0.0/anaphora-autoloads.el (No changes need to be saved) Checking /home/nicferrier/scratch/elpakit-run15942qZn/anaphora-1.0.0... Compiling /home/nicferrier/scratch/elpakit-run15942qZn/anaphora-1.0.0/anaphora-autoloads.el... Compiling /home/nicferrier/scratch/elpakit-run15942qZn/anaphora-1.0.0/anaphora-pkg.el... Wrote /home/nicferrier/scratch/elpakit-run15942qZn/anaphora-1.0.0/anaphora-pkg.elc Compiling /home/nicferrier/scratch/elpakit-run15942qZn/anaphora-1.0.0/anaphora.el... Wrote /home/nicferrier/scratch/elpakit-run15942qZn/anaphora-1.0.0/anaphora.elc Done (Total of 2 files compiled, 1 skipped) https://marmalade-repo.org/packages/elpakit-1.1.1.el: Bad Request In other words downloading that file over https did not work. But then I tried changing the package-archive reference to: http://marmalade-repo.org/packages/ in other words, dropping the HTTPS. Guess what? It worked. That is NOT because marmalade-repo.org is serving HTTP, it isn't. It simply redirects every HTTP request to an equivalent HTTPS request with a 301 redirect. If you doubt that, you're not alone. I was so bemused by the behaviour I checked it and here's the packet log: http://nic.ferrier.me.uk/pastes/OTU2N2QxNTQ3ZTEwNTQ3MGU0ZTUzYzE1NjhmMDdhMjA5ZTgyYTE2MQ== So that definitely shows marmalade is working properly. So switching back to HTTPS, what is going wrong? The depends of a package are all downloaded with HTTPS fine. But then: GET /packages/elpakit-1.1.1.el HTTP/1.1.. MIME-Version: 1.0.. Connection: keep-alive.. Extension: Security/Digest Security/SSL.. Host: marmalade-repo.org.. Accept-encoding: gzip..Accept: */*.. User-Agent: URL/Emacs.... ## T 80.69.77.43:443 -> 172.30.1.18:48975 [AP] HTTP/1.1 400 Bad Request.. Server: nginx/1.7.1.. Date: Mon, 27 Oct 2014 16:51:04 GMT.. Content-Type: text/html.. Content-Length: 270.. Connection: close.... .. 400 The plain HTTP request was sent to HTTPS port ..... That looks to me like the packaging system is forgetting that the package source is HTTPS when it downloads the target package and is sending the request as HTTP. Is this a regression? Yes. Doing exactly the same thing with my 24.3 install works fine. The packaging system had extensive changes of course, related to the internals of it's API. You may remember I remarked at the time about how frustrating it was. Why is this a big deal? I can just tell people to use HTTP can't I? Why yes. Of course I can. In fact, now I have to. But that's very sad. We should be encouraging users (and package archives) to use HTTPS where possible, shouldn't we? Probably the response to this will be "implement package signing". Nic