bug#15227: Possible bug in (web server)

bug#15227: Possible bug in (web server)

[Shane Celis]

Hi all,

I found a bug in (web server) on Mac OS X, and when I compared against GNU/Linux I seem to have found a bug in (web client) there.  I've included a script and further details below.

#!/usr/local/bin/guile \
-e main -s
!#
;; guile-web-server-osx-bug.scm
;;
;; This script demonstrates a possible bug in Guile's web server on
;; Mac OS X.  And it demonstrates a possible bug in Guile's web client
;; on GNU/Linux.
;;
;; Problem
;; =======
;;
;; Using Guile's (web server) with an example program, I ran into the
;; following issue on Mac OS X: If I ran
;; "./guile-web-server-osx-bug.scm server" and pointed my browser at
;; http://localhost:8080, the first request would work and be
;; displayed in my browser.  The second request would timeout.  I
;; compared this with Guile on GNU/Linux and found that the (web
;; server) worked fine, but the (web client) did not seem to work.
;;
;;
;; Reproduction
;; ============
;;
;; The script takes an argument "server" or "client [hostname/ip]".
;; When run as a server, it will say hello with an incremented
;; counter.  When run as a client, it will grab a couple of known
;; sites gnu.org, fsf.org, and then try localhost or the given
;; hostname on port 8080 twice, because on Mac OS X I saw the web
;; server work on the first request but not the second. The
;; hostname/ip the client is contacting is presumed to be a Guile web
;; server on the other end but doesn't have to be.
;;
;; If there were no bugs, this script should work like this:
;; $ ./guile-web-server-osx-bug.scm server &
;; Web server replied 'Hello World! 0'.
;; Web server replied 'Hello World! 1'.
;;
;; $ ./guile-web-server-osx-bug.scm client
;; Getting http://gnu.org... response #<<response> [...]
;;
;; Getting http://fsf.org... response #<<response> [...]
;;
;; Getting http://localhost:8080... response #<<response> [...]
;;
;; Getting http://localhost:8080... response #<<response> [...]
;;
;; Mac OS X bug
;; ------------
;;
;; Info
;; ----
;; 
;; $ uname -a
;; Darwin mbp13.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May  1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
;;
;; $ bash ./build-aux/config.guess
;; x86_64-apple-darwin12.4.0
;;
;; $ guile --version
;; guile (GNU Guile) 2.0.9 [...]
;;
;; $ ./config.status --config
;; 'LDFLAGS=-L/opt/local/lib' 'CPPFLAGS=-I/opt/local/include'
;;
;; Running
;; -------
;; 
;; $ ./guile-web-server-osx-bug.scm server 
;; Web server replied 'Hello World! 0'.
;;
;; --
;;
;; $ ./guile-web-server-osx-bug.scm client
;; Getting http://gnu.org... response #<<response> [...]
;;
;; Getting http://fsf.org... response #<<response> [...]
;;
;; Getting http://localhost:8080...SIGALRM called; timedout.
;; 
;; If I try to connect to http://localhost:8080 through a web browser,
;; it does not connect; however, the web server does report that it
;; has replied.
;;
;; GNU/Linux bug
;; ------------
;; 
;; Info
;; ----
;;
;; $ uname -a
;; Linux debian 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux
;;
;; $ guile --version
;; guile (GNU Guile) 2.0.9 [...]
;;
;; $ bash ./build-aux/config.guess
;; i686-pc-linux-gnu
;;
;; $ ./config.status --config
;;
;; Running
;; -------
;;
;; $ ./guile-web-server-osx-bug.scm server 
;; Web server replied 'Hello World! 0'.
;;
;; ---
;;
;; $ ./guile-web-server-osx-bug.scm client
;; Getting http://gnu.org... response #<<response> [...]
;;
;; Getting http://fsf.org... response #<<response> [...]
;;
;; Getting http://localhost:8080...SIGALRM called; timedout.
;;
;; If I try to connect to http://localhost:8080 through a web browser,
;; it works fine; so it seems like the (web client) is not working on
;; GNU/Linux and the (web server) is not working on Mac OS X.
;; 

(use-modules (web server)
             (web client)
             (ice-9 optargs))

(define count 0)

(define (hello-world-handler request request-body)
  (let ((reply (format #f "Hello World! ~a" count)))
    (set! count (1+ count))
    (format #t "Web server replied '~a'.~%" reply)
    (values '((content-type . (text/plain)))
            reply)))

(define (start-server)
   (run-server hello-world-handler 'http '(#:port 8080)))

(define (check-url url)
  (format #t "Getting ~a..." url)
  (format #t " response ~a~%~%" (http-get url)))

(define* (run-client #:optional (ip "localhost"))
  (alarm 10)
  ;; Test some URLs that should work.
  (check-url "http://gnu.org")
  (check-url "http://fsf.org")
  ;; Try the web server's URL now twice.
  (check-url (format #f "http://~a:8080" ip))
  (check-url (format #f "http://~a:8080" ip)))

(sigaction SIGALRM (lambda (sig)
                     (format #t "SIGALRM called; timedout.~%")
                     (exit 1)))


(define (main args)
  (unless (or (= 2 (length args))
              (= 3 (length args)))
    (format (current-error-port) "Usage: ~a <server|client [ip]>~%" (car args))
    (exit 2))
  (cond
   ((string= "server" (cadr args))
    (start-server))
   ((string= "client" (cadr args))
    (apply run-client (cddr args)))))

bug#15227: Possible bug in (web server)

[Ian Price]

Shane Celis <shane.celis@gmail.com> writes:

> ;; GNU/Linux bug
> ;; ------------
> ;; 
> ;; Info
> ;; ----
> ;;
> ;; $ uname -a
> ;; Linux debian 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux
> ;;
> ;; $ guile --version
> ;; guile (GNU Guile) 2.0.9 [...]
> ;;
> ;; $ bash ./build-aux/config.guess
> ;; i686-pc-linux-gnu
> ;;
> ;; $ ./config.status --config
> ;;
> ;; Running
> ;; -------
> ;;
> ;; $ ./guile-web-server-osx-bug.scm server 
> ;; Web server replied 'Hello World! 0'.
> ;;
> ;; ---
> ;;
> ;; $ ./guile-web-server-osx-bug.scm client
> ;; Getting http://gnu.org... response #<<response> [...]
> ;;
> ;; Getting http://fsf.org... response #<<response> [...]
> ;;
> ;; Getting http://localhost:8080...SIGALRM called; timedout.
> ;;
> ;; If I try to connect to http://localhost:8080 through a web browser,
> ;; it works fine; so it seems like the (web client) is not working on
> ;; GNU/Linux and the (web server) is not working on Mac OS X.
> ;; 
If I run the client under strace I get

read(13, "HTTP/1.1 200 OK\r\nContent-Length: 15\r\nContent-Type: text/plain;charset=utf-8\r\n\r\nHello World! 22", 4096) = 94
read(13, 0xa1ec000, 4096)               = ? ERESTARTSYS (To be restarted if SA_RESTART is set)

so even though we've read all the data that's needed, we're still making
another call to read and blocking.

One problem was that read-response-body was making a call to
get-bytevector-all, but even after changing it to

(define (read-response-body r)
  "Reads the response body from R, as a bytevector.  Returns
‘#f’ if there was no response body."
  (let ((body (and=> (response-body-port r #:decode? #f)
                     (lambda (p)
                       (cond ((response-content-length r) =>
                              (lambda (n)
                                (get-bytevector-n p n)))
                             (else (get-bytevector-all p)))))))
    ;; Reading a body of length 0 will result in get-bytevector-all
    ;; returning the EOF object.
    (if (eof-object? body)
        #vu8()
        body)))

I'm still seeing the same behaviour. (I did check and the
get-bytevector-n case is definitely being ran)

So, I'm guessing the problem lies in the C code for handling ports,
and/or for handling sockets.

No idea why it doesn't happen on web calls but only localhost.

Aside:
The above change to read-response-body has not been committed. I'm
undecided as to whether or not read-response-body should handle checking
the size of the content body. Currently it doesn't, but we have separate
code for doing that in (web client), so maybe not. But it might be an
idea to document that read-response-body may return less bytes than the
content length.

-- 
Ian Price -- shift-reset.com

"Programming is like pinball. The reward for doing it well is
the opportunity to do it again" - from "The Wizardy Compiled"

bug#15227: Possible bug in (web server)

[Andy Wingo]

I don't really understand this bug.  Ian's investigation isn't right, I
don't think; the "delimited" port stuff from (web response) should make
it correct to call get-bytevector-all on the response port.  So I think
there is not a client bug, but there may be a server bug on macOS.  Can
you retry on macOS with 2.0.11 and with 2.1.3 and see if the problem
still exists?

Thanks,

Andy

On Sat 31 Aug 2013 09:11, Shane Celis <shane.celis@gmail.com> writes:

> Hi all,
>
> I found a bug in (web server) on Mac OS X, and when I compared against GNU/Linux I seem to have found a bug in (web client) there.  I've included a script and further details below.
>
> #!/usr/local/bin/guile \
> -e main -s
> !#
> ;; guile-web-server-osx-bug.scm
> ;;
> ;; This script demonstrates a possible bug in Guile's web server on
> ;; Mac OS X.  And it demonstrates a possible bug in Guile's web client
> ;; on GNU/Linux.
> ;;
> ;; Problem
> ;; =======
> ;;
> ;; Using Guile's (web server) with an example program, I ran into the
> ;; following issue on Mac OS X: If I ran
> ;; "./guile-web-server-osx-bug.scm server" and pointed my browser at
> ;; http://localhost:8080, the first request would work and be
> ;; displayed in my browser.  The second request would timeout.  I
> ;; compared this with Guile on GNU/Linux and found that the (web
> ;; server) worked fine, but the (web client) did not seem to work.
> ;;
> ;;
> ;; Reproduction
> ;; ============
> ;;
> ;; The script takes an argument "server" or "client [hostname/ip]".
> ;; When run as a server, it will say hello with an incremented
> ;; counter.  When run as a client, it will grab a couple of known
> ;; sites gnu.org, fsf.org, and then try localhost or the given
> ;; hostname on port 8080 twice, because on Mac OS X I saw the web
> ;; server work on the first request but not the second. The
> ;; hostname/ip the client is contacting is presumed to be a Guile web
> ;; server on the other end but doesn't have to be.
> ;;
> ;; If there were no bugs, this script should work like this:
> ;; $ ./guile-web-server-osx-bug.scm server &
> ;; Web server replied 'Hello World! 0'.
> ;; Web server replied 'Hello World! 1'.
> ;;
> ;; $ ./guile-web-server-osx-bug.scm client
> ;; Getting http://gnu.org... response #<<response> [...]
> ;;
> ;; Getting http://fsf.org... response #<<response> [...]
> ;;
> ;; Getting http://localhost:8080... response #<<response> [...]
> ;;
> ;; Getting http://localhost:8080... response #<<response> [...]
> ;;
> ;; Mac OS X bug
> ;; ------------
> ;;
> ;; Info
> ;; ----
> ;; 
> ;; $ uname -a
> ;; Darwin mbp13.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May  1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
> ;;
> ;; $ bash ./build-aux/config.guess
> ;; x86_64-apple-darwin12.4.0
> ;;
> ;; $ guile --version
> ;; guile (GNU Guile) 2.0.9 [...]
> ;;
> ;; $ ./config.status --config
> ;; 'LDFLAGS=-L/opt/local/lib' 'CPPFLAGS=-I/opt/local/include'
> ;;
> ;; Running
> ;; -------
> ;; 
> ;; $ ./guile-web-server-osx-bug.scm server 
> ;; Web server replied 'Hello World! 0'.
> ;;
> ;; --
> ;;
> ;; $ ./guile-web-server-osx-bug.scm client
> ;; Getting http://gnu.org... response #<<response> [...]
> ;;
> ;; Getting http://fsf.org... response #<<response> [...]
> ;;
> ;; Getting http://localhost:8080...SIGALRM called; timedout.
> ;; 
> ;; If I try to connect to http://localhost:8080 through a web browser,
> ;; it does not connect; however, the web server does report that it
> ;; has replied.
> ;;
> ;; GNU/Linux bug
> ;; ------------
> ;; 
> ;; Info
> ;; ----
> ;;
> ;; $ uname -a
> ;; Linux debian 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux
> ;;
> ;; $ guile --version
> ;; guile (GNU Guile) 2.0.9 [...]
> ;;
> ;; $ bash ./build-aux/config.guess
> ;; i686-pc-linux-gnu
> ;;
> ;; $ ./config.status --config
> ;;
> ;; Running
> ;; -------
> ;;
> ;; $ ./guile-web-server-osx-bug.scm server 
> ;; Web server replied 'Hello World! 0'.
> ;;
> ;; ---
> ;;
> ;; $ ./guile-web-server-osx-bug.scm client
> ;; Getting http://gnu.org... response #<<response> [...]
> ;;
> ;; Getting http://fsf.org... response #<<response> [...]
> ;;
> ;; Getting http://localhost:8080...SIGALRM called; timedout.
> ;;
> ;; If I try to connect to http://localhost:8080 through a web browser,
> ;; it works fine; so it seems like the (web client) is not working on
> ;; GNU/Linux and the (web server) is not working on Mac OS X.
> ;; 
>
> (use-modules (web server)
>              (web client)
>              (ice-9 optargs))
>
> (define count 0)
>
> (define (hello-world-handler request request-body)
>   (let ((reply (format #f "Hello World! ~a" count)))
>     (set! count (1+ count))
>     (format #t "Web server replied '~a'.~%" reply)
>     (values '((content-type . (text/plain)))
>             reply)))
>
> (define (start-server)
>    (run-server hello-world-handler 'http '(#:port 8080)))
>
> (define (check-url url)
>   (format #t "Getting ~a..." url)
>   (format #t " response ~a~%~%" (http-get url)))
>
> (define* (run-client #:optional (ip "localhost"))
>   (alarm 10)
>   ;; Test some URLs that should work.
>   (check-url "http://gnu.org")
>   (check-url "http://fsf.org")
>   ;; Try the web server's URL now twice.
>   (check-url (format #f "http://~a:8080" ip))
>   (check-url (format #f "http://~a:8080" ip)))
>
> (sigaction SIGALRM (lambda (sig)
>                      (format #t "SIGALRM called; timedout.~%")
>                      (exit 1)))
>
>
> (define (main args)
>   (unless (or (= 2 (length args))
>               (= 3 (length args)))
>     (format (current-error-port) "Usage: ~a <server|client [ip]>~%" (car args))
>     (exit 2))
>   (cond
>    ((string= "server" (cadr args))
>     (start-server))
>    ((string= "client" (cadr args))
>     (apply run-client (cddr args)))))

bug#15227: Possible bug in (web server)

[Matt Wette]

[-- Attachment #1: Type: text/plain, Size: 4090 bytes --]


I tried this on guile 2.2.0 and the localhost queries seem to work OK. — Matt



mwette$ ./15227 server
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /Users/mwette/proj/scheme/guile/bugs-guile/./15227
;;; compiled /Users/mwette/.cache/guile/ccache/2.2-LE-8-3.9/Users/mwette/proj/scheme/guile/bugs-guile/15227.go
Web server replied 'Hello World! 0'.
Web server replied 'Hello World! 1'.
Web server replied 'Hello World! 2'.
Web server replied 'Hello World! 3'.



mwette$ ./15227 client
Getting http://gnu.org... response #<<response> version: (1 . 1) code: 301 reason-phrase: "Moved Permanently" headers: ((date . #<date nanosecond: 0 second: 12 minute: 33 hour: 20 day: 29 month: 4 year: 2017 zone-offset: 0>) (server . "Apache/2.4.7") (location . #<<uri> scheme: http userinfo: #f host: "www.gnu.org" port: #f path: "/" query: #f fragment: #f>) (content-length . 290) (connection close) (content-type text/html (charset . "iso-8859-1"))) port: #<closed: file 1021b0690>>

Getting http://fsf.org... response #<<response> version: (1 . 0) code: 301 reason-phrase: "Moved Permanently" headers: ((server . "nginx/1.1.19") (date . #<date nanosecond: 0 second: 42 minute: 36 hour: 20 day: 29 month: 4 year: 2017 zone-offset: 0>) (content-type text/html) (content-length . 185) (location . #<<uri> scheme: http userinfo: #f host: "www.fsf.org" port: #f path: "/" query: #f fragment: #f>) (x-cache . "MISS from www.fsforg") (x-cache-lookup . "MISS from www.fsforg:80") (via "1.0 www.fsforg (squid/3.1.19)") (connection close)) port: #<closed: file 1021b03f0>>

Getting http://localhost:8080... response #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((content-length . 14) (content-type text/plain (charset . "utf-8"))) port: #<closed: file 1021b0000>>

Getting http://localhost:8080... response #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((content-length . 14) (content-type text/plain (charset . "utf-8"))) port: #<closed: file 1022d1c40>>

mwette$ ./15227 client
Getting http://gnu.org... response #<<response> version: (1 . 1) code: 301 reason-phrase: "Moved Permanently" headers: ((date . #<date nanosecond: 0 second: 28 minute: 33 hour: 20 day: 29 month: 4 year: 2017 zone-offset: 0>) (server . "Apache/2.4.7") (location . #<<uri> scheme: http userinfo: #f host: "www.gnu.org" port: #f path: "/" query: #f fragment: #f>) (content-length . 290) (connection close) (content-type text/html (charset . "iso-8859-1"))) port: #<closed: file 107eb9690>>

Getting http://fsf.org... response #<<response> version: (1 . 0) code: 301 reason-phrase: "Moved Permanently" headers: ((server . "nginx/1.1.19") (date . #<date nanosecond: 0 second: 58 minute: 36 hour: 20 day: 29 month: 4 year: 2017 zone-offset: 0>) (content-type text/html) (content-length . 185) (location . #<<uri> scheme: http userinfo: #f host: "www.fsf.org" port: #f path: "/" query: #f fragment: #f>) (x-cache . "MISS from www.fsforg") (x-cache-lookup . "MISS from www.fsforg:80") (via "1.0 www.fsforg (squid/3.1.19)") (connection close)) port: #<closed: file 107eb93f0>>

Getting http://localhost:8080... response #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((content-length . 14) (content-type text/plain (charset . "utf-8"))) port: #<closed: file 107eb9000>>

Getting http://localhost:8080... response #<<response> version: (1 . 1) code: 200 reason-phrase: "OK" headers: ((content-length . 14) (content-type text/plain (charset . "utf-8"))) port: #<closed: file 107fd9c40>>

mwette$ uname -a
Darwin nautilus 16.5.0 Darwin Kernel Version 16.5.0: Fri Mar  3 16:52:33 PST 2017; root:xnu-3789.51.2~3/RELEASE_X86_64 x86_64

mwette$ guile22 --version
guile (GNU Guile) 2.2.0
Copyright (C) 2017 Free Software Foundation, Inc.

License LGPLv3+: GNU LGPL 3 or later <http://gnu.org/licenses/lgpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.


[-- Attachment #2: Type: text/html, Size: 13685 bytes --]