bug#10223: 24.0.91; URL (url-http-parse-reponse) errors on Timeout

bug#10223: 24.0.91; URL (url-http-parse-reponse) errors on Timeout

[Donald Curtis]

I have tracked down a bug to the url package, but let me explain how I
get this error.

I have my own package.el repository that is hosted on a shared hosting
service.  When I install a number of packages using the following
command (or from the package-list-packages interface),

(mapc 'package-install '(markdown-mode deft hl-sentence magit evil))

error occurs after the `magit` package (which takes a while to
compile).  I tracked it down to URL using persistent connections.  If I
disable keepalives,

(setq url-http-attempt-keepalives nil)

then it works.  It appears that when url-http-parse-response is called,
there is no data, since the timeout has occurred.  I realize this is
actually because my hosting service has decided that the timeout should
be low.  But it's reasonable to assume that url should be able to handle
this situation OR package.el should handle it.

I'd be willing to take a stab at a patch but I'm not familiar enough
with URL to know if that is where the fix should go or if it should go
in package.el.


--- TAIL OF *URL-DEBUG* Buffer
http -> Marking connection as free: melpa.milkbox.net:80 #<process melpa.milkbox.net>
http -> url-http-parse-headers called in ( *http melpa.milkbox.net:80*<2>)
http -> url-http-parse-response called in ( *http melpa.milkbox.net:80*<2>)
http -> Parsed HTTP headers: class=2 status=200
http -> Finished parsing HTTP headers: t
http -> Marking connection as free: melpa.milkbox.net:80 #<process melpa.milkbox.net>
http -> Activating callback in buffer ( *http melpa.milkbox.net:80*<2>)
retrieval -> Synchronous fetching done (#<buffer  *http melpa.milkbox.net:80*<2>>)
http -> url-http-parse-response called in ( *http melpa.milkbox.net:80*<2>)
http -> Found existing connection: melpa.milkbox.net:80 #<process melpa.milkbox.net>
http -> Reusing existing connection: melpa.milkbox.net:80
http -> Marking connection as busy: melpa.milkbox.net:80 #<process melpa.milkbox.net>
http -> Request is:
GET /packages/evil-20111205.tar HTTP/1.1
MIME-Version: 1.0
Connection: keep-alive
Extension: Security/Digest Security/SSL
Host: melpa.milkbox.net
Accept: */*
User-Agent: URL/Emacs (i386-apple-darwin11.2.0; OpenStep)


retrieval -> Spinning in url-retrieve-synchronously: nil (#<buffer  *http melpa.milkbox.net:80*<2>>)
http -> url-http-end-of-document-sentinel in buffer ( *http melpa.milkbox.net:80*<2>)
http -> Marking connection as free: melpa.milkbox.net:80 #<process melpa.milkbox.net>
http -> Activating callback in buffer ( *http melpa.milkbox.net:80*<2>)
retrieval -> Synchronous fetching done (#<buffer  *http melpa.milkbox.net:80*<2>>)

--- ERROR OCCURS ---

Debugger entered--Lisp error: (error "Trying to parse HTTP response code in odd buffer:  *http melpa.milkbox.net:80*<2>")
  signal(error ("Trying to parse HTTP response code in odd buffer:  *http melpa.milkbox.net:80*<2>"))
  error("Trying to parse HTTP response code in odd buffer: %s" " *http melpa.milkbox.net:80*<2>")
  url-http-parse-response()
  package-handle-response()
  package-download-tar(evil "20111205")
  package-download-transaction((evil))
  package-install(evil)
  mapc(package-install (markdown-mode deft hl-sentence magit evil))
  eval((mapc (quote package-install) (quote (markdown-mode deft hl-sentence magit evil))) nil)
  eval-expression((mapc (quote package-install) (quote (markdown-mode deft hl-sentence magit evil))) nil)
  call-interactively(eval-expression nil nil)
  recursive-edit()
  debug(error (error "Trying to parse HTTP response code in odd buffer:  *http melpa.milkbox.net:80*"))
  signal(error ("Trying to parse HTTP response code in odd buffer:  *http melpa.milkbox.net:80*"))
  error("Trying to parse HTTP response code in odd buffer: %s" " *http melpa.milkbox.net:80*")
  url-http-parse-response()
  package-handle-response()
  package-download-tar(deft "20111102")
  package-download-transaction((deft))
  package-install(deft)
  mapc(package-install (markdown-mode deft hl-sentence magit evil))
  eval((mapc (quote package-install) (quote (markdown-mode deft hl-sentence magit evil))) nil)
  eval-expression((mapc (quote package-install) (quote (markdown-mode deft hl-sentence magit evil))) nil)
  call-interactively(eval-expression nil nil)


--- END OF BUG REPORT ---


T


In GNU Emacs 24.0.91.1 (i386-apple-darwin11.2.0, NS apple-appkit-1138.23)
of 2011-11-16 on hendrix.local
Windowing system distributor `Apple', version 10.3.1138
configured using `configure  '--prefix=/usr/local/Cellar/emacs/HEAD' '--without-dbus' '--enable-locallisppath=/usr/local/share/emacs/site-lisp' '--infodir=/usr/local/Cellar/emacs/HEAD/share/info/emacs' '--with-ns' '--disable-ns-self-contained' 'CC=/usr/bin/llvm-gcc' 'CFLAGS=-O3 -march=core2 -w -pipe''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: Debugger

Minor modes in effect:
  savehist-mode: t
  recentf-mode: t
  shell-dirtrack-mode: t
  ido-ubiquitous: t
  ido-everywhere: t
  show-paren-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-c h v p a c k a g e i n s t a l l C-w C-g C-c h f
p a c k a g e i n s t a l l C-n C-n C-n C-n C-n <return>
C-x o <tab> <return> C-x 1 C-n C-n C-n C-n C-n C-n
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-s u r l C-s C-s
C-s C-s C-s C-s C-s C-s C-s C-s C-s C-s C-g C-x b i
n i <return> C-p C-e C-b C-w t C-x C-s M-: M-p M-p
M-n <return> <down-mouse-1> <mouse-1> C-p C-p C-p C-p
C-p C-p C-x o C-n C-n C-n C-n C-n C-n C-p C-p C-p C-p
C-p C-p C-p C-n C-n C-n C-n C-s C-g C-x b <return>
C-s h a n d l e - r e s p o n C-s C-s <return> C-n
C-n C-n C-n C-n C-n C-n C-n C-r C-r C-r C-r C-r C-r
C-x o C-x RET <return>

Recent messages:
Wrote /Users/dcurtis/.emacs.d/elpa/markdown-mode-20111203/markdown-mode-pkg.elc
Checking /Users/dcurtis/.emacs.d/elpa/markdown-mode-20111203...
Compiling /Users/dcurtis/.emacs.d/elpa/markdown-mode-20111203/markdown-mode.el...done
Wrote /Users/dcurtis/.emacs.d/elpa/markdown-mode-20111203/markdown-mode.elc
Checking /Users/dcurtis/.emacs.d/elpa/markdown-mode-20111203...
Done (Total of 2 files compiled, 1 skipped)
Contacting host: melpa.milkbox.net:80
Entering debugger...
Quit
Mark saved where search started [2 times]

Load-path shadows:
~/.emacs.d/custom hides /usr/local/Cellar/emacs/HEAD/share/emacs/24.0.91/lisp/custom
/Users/dcurtis/.emacs.d/elpa/magit-20111205/.dir-locals hides /usr/local/Cellar/emacs/HEAD/share/emacs/24.0.91/lisp/gnus/.dir-locals

Features:
(shadow sort flyspell ispell mail-extr message rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mailabbrev gmm-utils mailheader emacsbug
debug derived noutline outline easy-mmode compile autoload tar-mode
mail-utils url-cache network-stream starttls url-http tls mail-parse
rfc2231 rfc2047 rfc2045 ietf-drums url-gw url-auth url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-util
url-parse url-vars mailcap time-stamp hi-lock multi-isearch vc-git
jka-compr find-func eldoc elisp-slime-nav etags paredit hl-line
saveplace idle-highlight-mode dropdown-list yasnippet help-mode view
whitespace tango-dark-theme savehist recentf tree-widget wid-edit
autorevert cua-base cus-start cus-load warnings server misc midnight
auctex-autoloads tex-site info browse-kill-ring-autoloads deft-autoloads
ess-autoloads tramp tramp-compat auth-source eieio byte-opt bytecomp
byte-compile cconv macroexp gnus-util mm-util mail-prsvr password-cache
format-spec tramp-loaddefs ess-toolbar ess-mouse mouseme thingatpt
browse-url ess-menu speedbar sb-image ezimage dframe ess-swv ess-noweb
noweb-font-lock-mode ess-bugs-l essd-els ess-sas-d ess-sas-l ess-sas-a
executable shell pcomplete ess-arc-d ess-vst-d ess-xls-d ess-lsp-l
ess-sta-d ess-sta-l cc-vars cc-defs make-regexp ess-sp6-d ess-sp5-d
ess-sp3-d ess-r-d ess-r-args assoc ess-s-l ess-inf ess-utils comint ring
ess-mode noweb-mode ess ess-custom regexp-opt easymenu ess-compat
ess-site evil-autoloads haskell-mode-autoloads hl-sentence-autoloads
markdown-mode+-autoloads markdown-mode-autoloads
starter-kit-bindings-autoloads windmove starter-kit-lisp-autoloads
elisp-slime-nav-autoloads starter-kit-autoloads edmacro kmacro smex
starter-kit-misc ido-ubiquitous ido paren starter-kit-defuns uniquify
advice help-fns advice-preload magit-autoloads ido-ubiquitous-autoloads
smex-autoloads find-file-in-project-autoloads
idle-highlight-mode-autoloads paredit-autoloads undo-tree-autoloads
yasnippet-autoloads package tabulated-list defun cl time-date tooltip
ediff-hook vc-hooks lisp-float-type mwheel ns-win tool-bar dnd fontset
image fringe lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer loaddefs button faces cus-face files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process ns multi-tty
emacs)

bug#10223: 24.0.91; URL (url-http-parse-reponse) errors on Timeout

[Chong Yidong]

Donald Curtis <dcurtis@gmail.com> writes:

> It appears that when url-http-parse-response is called, there is no
> data, since the timeout has occurred.  I realize this is actually
> because my hosting service has decided that the timeout should be low.
> But it's reasonable to assume that url should be able to handle this
> situation OR package.el should handle it.

I've committed a fix to url-http.el.  Thanks for the bug report.