Non-blocking web server

Non-blocking web server

[Ryan Raymond]

[-- Attachment #1: Type: text/plain, Size: 919 bytes --]

Hello, all.
I was able to build a non-blocking web-server using network sockets.
However, the existing guile web/server.scm implementation is
single-threaded and therefore blocking, which is sub-optimal for some
use-cases.

I suggest we slightly modify the server logic to have an optional
#:blocking? [bool] parameter which would enable behavior in line with the
following excerpt from my own code. Obviously some changes would be made to
methodology and code style, but you get the picture.

(define (listen sock)
  (let* (
    (client-connection (accept sock))
    (client-details (cdr client-connection))
    (client (car client-connection)))
    (begin-thread
      (sigaction SIGPIPE SIG_IGN)
      (handle client)
      (close client))
    ))

This shouldn't cause any backwards-compatibility issues since it's
optional, but the specifics of web/server.scm might cause problems.
Let me know what you think!
Ryan

[-- Attachment #2: Type: text/html, Size: 1099 bytes --]

Re: Non-blocking web server

[tomas]

[-- Attachment #1: Type: text/plain, Size: 563 bytes --]

On Sun, Mar 24, 2024 at 03:41:32PM -0400, Ryan Raymond wrote:
> Hello, all.
> I was able to build a non-blocking web-server using network sockets.
> However, the existing guile web/server.scm implementation is
> single-threaded and therefore blocking [...]

How does "single-threaded" imply "blocking"? I mean: multithreading
does have its uses, no question (especially if you want to leverage
more than one CPU core).

As one example, Nginx, arguably the fastest of the web servers
out there isn't based on threads for concurrency.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Non-blocking web server

[Nala Ginrut]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]

Hi Ryan!
For the single-threaded non-blocking server in Guile, you have to make sure
at least 3 steps:
1. set socket to non-blocking
2. enable suspended ports to prepare delimited-continuation powered
coroutine
3. designed your own scheduler

Even in the single thread, you can handle requests concurrently because of
the coroutine based on delimited-continuation.

Of course you may enhance it with thread pool.


Best regards.


On Mon, Mar 25, 2024 at 4:04 AM Ryan Raymond <rjraymond@oakland.edu> wrote:

> Hello, all.
> I was able to build a non-blocking web-server using network sockets.
> However, the existing guile web/server.scm implementation is
> single-threaded and therefore blocking, which is sub-optimal for some
> use-cases.
>
> I suggest we slightly modify the server logic to have an optional
> #:blocking? [bool] parameter which would enable behavior in line with the
> following excerpt from my own code. Obviously some changes would be made to
> methodology and code style, but you get the picture.
>
> (define (listen sock)
>   (let* (
>     (client-connection (accept sock))
>     (client-details (cdr client-connection))
>     (client (car client-connection)))
>     (begin-thread
>       (sigaction SIGPIPE SIG_IGN)
>       (handle client)
>       (close client))
>     ))
>
> This shouldn't cause any backwards-compatibility issues since it's
> optional, but the specifics of web/server.scm might cause problems.
> Let me know what you think!
> Ryan
>

[-- Attachment #2: Type: text/html, Size: 2143 bytes --]

RE: Non-blocking web server

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1625 bytes --]

Your flag #:blocking? #true would do a per-connection thread, which while implementing lack of blocking, is a stronger property than non-blocking. Also, (kernel-level) threads have some overhead, which is sub-optimal;

Instead of a flag, I propose defining a new ‘server-impl’

Web Server (Guile Reference Manual) (gnu.org)

This impl could then spawn threads (or, more likely, do some thread pool things, presumably with some mitigations against ‘read one byte, wait N minutes’ DDOS attacks).

Also, single-threaded does not imply blocking (assuming the implementation uses ‘select’ / ‘epoll’ / etc. -- I don’t know if Guile does that yet, might be worth checking).  

If the respond code writes to ports, some ‘(parameterize ((current-read-waiter ...) (current-write-waiter ...)) ...)’ + delimited continuations + O_NONBLOCK will be needed if the respond code writes to port.

This, by the way, is already implemented by Guile-Fibers. I recommend using Guile-Fibers HTTP implementation or something based upon this (it has (relatively?) lightweight M:N user-level threads).

I don’t recommend the default Fibers implementation, because it does ‘run-fibers’ and create threads on its own, which is really not its responsibility and as such makes it inconvenient to use when Fibers is used for other things as well.

Instead, I am currently using

fiberized.scm « server « web - gnunet-scheme.git - GNUnet client implementation in (Guile) Scheme

(I don’t recall which changes I made). I’ve only used it for demo purposes so far, though.

Best regards,
Maxime Devos.

[-- Attachment #2: Type: text/html, Size: 4271 bytes --]