bug#36350: [2.2.5] ‘read-headers’ blocks, thereby breaking web servers

[Dan Frumin <dfrumin@cs.ru.nl>]

I believe that `(lookahead-char port)` really blocks when the client has finished sending the request and there is no more data from `port` to consume.
If I understand it correctly, then per HTTP/1.1 [1] the request ends with CRLF at the last line, and then comes the message. So I we have read an 
empty string, then we shouldn't proceed with further lookaheads.

Specifically, the following code works out for me:


(define (read-header-line port)
   "Read an HTTP header line, including any continuation lines, and
return the combined string without its final CRLF or LF.  Raise a
'bad-header' exception if the line does not end in CRLF or LF, or if EOF
is reached."
   (format #t "Reading header line now: ")
   (match (%read-line port)
     (((? string? line) . #\newline)
      ;; '%read-line' does not consider #\return a delimiter; so if it's
      ;; there, remove it.  We are more tolerant than the RFC in that we
      ;; tolerate LF-only endings.
      (let ((line (if (string-suffix? "\r" line)
                      (string-drop-right line 1)
                      line)))
        ;; If the next character is a space or tab, then there's at least
        ;; one continuation line.  Read the continuation lines by calling
        ;; 'read-header-line' recursively, and append them to this header
        ;; line, folding the leading spaces and tabs to a single space.
        (if (and (not (string-null? line))
                 (space-or-tab? (lookahead-char port)))
            (string-append line " " (string-trim (read-header-line port)
                                                 spaces-and-tabs))
            line)))
     ((line . _)                                ;EOF or missing delimiter
      (bad-header 'read-header-line line))))

Moreover, the continuation lines in general have been deprecated: [2].
I have to say I would be in favor of removing support for continuation lines in general.

Best regards,
-Dan


[1]: https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html
[2]: https://tools.ietf.org/html/rfc7230#section-3.2.4