Serving files with guile web server

Serving files with guile web server

[romel]

Hi,

I will like to serve plain text files and images with run-server from (web
server) module.

After reading the documentation I think in something similar to:

-----------------
(define (serve-file request body)
  (let* ((path (request-path-components request))
	 (file-path (public-file-path path)))
    (if (and file-path (file-exists? file-path))
	(values '((content-type . (text/plain)))
		(open-input-file file-path))
	(not-found request))))

(define (file-server)
  (run-server serve-file))
-----------------

But I got:

-----------------
ERROR: In procedure scm-error:
ERROR: unexpected body type

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt

In module/web/server.scm:
   291:29  1 (#<procedure 22d9480 at module/web/server.scm:283:3 ()>)
In unknown file:
           0 (scm-error misc-error #f "~A" ("unexpected body type") #f)
-----------------

Anybody knows how to serve text and image files from guile web server?

Thanks,
Romel Sandoval

Re: Serving files with guile web server

[romel]

I gave a second read to the manual and found:

"The handler should return two values: the response, as a <response>
record from (web response), and the response body as a string, bytevector,
or #f if not present."

If this is correct then I guess I should check the file type first and
then read the file to either create a string or a bytevector from it.

Any recommendation will be highly appreciated.

Thanks,
Romel Sandoval

> Hi,
>
> I will like to serve plain text files and images with run-server from (web
> server) module.
>
> After reading the documentation I think in something similar to:
>
> -----------------
> (define (serve-file request body)
>   (let* ((path (request-path-components request))
> 	 (file-path (public-file-path path)))
>     (if (and file-path (file-exists? file-path))
> 	(values '((content-type . (text/plain)))
> 		(open-input-file file-path))
> 	(not-found request))))
>
> (define (file-server)
>   (run-server serve-file))
> -----------------
>
> But I got:
>
> -----------------
> ERROR: In procedure scm-error:
> ERROR: unexpected body type
>
> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
> scheme@(guile-user) [1]> ,bt
>
> In module/web/server.scm:
>    291:29  1 (#<procedure 22d9480 at module/web/server.scm:283:3 ()>)
> In unknown file:
>            0 (scm-error misc-error #f "~A" ("unexpected body type") #f)
> -----------------
>
> Anybody knows how to serve text and image files from guile web server?
>
> Thanks,
> Romel Sandoval
>
>
>
>
>

Re: Serving files with guile web server

[Neil Jerram]

romel@lavabit.com writes:

> I gave a second read to the manual and found:
>
> "The handler should return two values: the response, as a <response>
> record from (web response), and the response body as a string, bytevector,
> or #f if not present."
>
> If this is correct then I guess I should check the file type first and
> then read the file to either create a string or a bytevector from it.
>
> Any recommendation will be highly appreciated.

Well, I suppose I'd recommend considering not using the Guile server to
serve static files.  Using Guile makes more sense when you want to
generate dynamic content, because then you're really using the
capabilities of the language.

In my web server experiments so far I've used Apache, modlisp and Guile
together, and this combination seems to work well.  Then Apache can
handle all the static content (I guess using the efficient sendfile)
without bothering modlisp or Guile, but dynamic stuff can be routed via
modlisp to Guile.

Regards,
      Neil

Re: Serving files with guile web server

[romel]

Please see comments below.

Neil Jerram writes:
> romel@lavabit.com writes:
>
>> I gave a second read to the manual and found:
>>
>> "The handler should return two values: the response, as a <response>
>> record from (web response), and the response body as a string,
>> bytevector,
>> or #f if not present."

This is true but I found that when it comes about text is better to use a
lambda that receive a port and then write text to that port.

>>
>> If this is correct then I guess I should check the file type first and
>> then read the file to either create a string or a bytevector from it.

Is important to check the file type, if you need to serve an image you
must use the correct mime-type and a bytevertor.

>>
>> Any recommendation will be highly appreciated.
>
> Well, I suppose I'd recommend considering not using the Guile server to
> serve static files.  Using Guile makes more sense when you want to
> generate dynamic content, because then you're really using the
> capabilities of the language.
>
> In my web server experiments so far I've used Apache, modlisp and Guile
> together, and this combination seems to work well.  Then Apache can
> handle all the static content (I guess using the efficient sendfile)
> without bothering modlisp or Guile, but dynamic stuff can be routed via
> modlisp to Guile.
>

I think that's a good idea but while I was trying to setup my Apache I
found it complex compared to setting up something for python or java using
a web framework (maybe because of the lack of tutorials, examples,
frameworks, etc.), so I think that it would be great to have a standalone
server so you can instant hack something, or even for production, I think
it could be scalable using clusters.

So far I have been able to serve text files and images, it should work
with any mime-type now, but I have not test it yet.

Here is the piece of code that handles files for the curious:

	(let ((file-path (public-file-path path)))
	  (if (file-exists? file-path)
	      (let* ((mime-type (mime-type-ref file-path))
		     (mime-type-symbol (mime-type-symbol mime-type)))
		(if (text-mime-type? mime-type)
		    (values
		     `((content-type . (,mime-type-symbol)))
		     (lambda (out-port)
		       (call-with-input-file file-path
			 (lambda (in-port)
			   (display (read-delimited "" in-port)
                                                    out-port)))))
		    (values
		     `((content-type . (,mime-type-symbol)))
		     (call-with-input-file file-path
			 (lambda (in-port)
			   (get-bytevector-all in-port))))))
	      (not-found request)))

I hope it's readable :-)

Guile web module it's great!

Thanks,
Romel Sandoval

Re: Serving files with guile web server

[Neil Jerram]

romel@lavabit.com writes:

> So far I have been able to serve text files and images, it should work
> with any mime-type now, but I have not test it yet.
>
> Here is the piece of code that handles files for the curious:
>
> 	(let ((file-path (public-file-path path)))
> 	  (if (file-exists? file-path)
> 	      (let* ((mime-type (mime-type-ref file-path))
> 		     (mime-type-symbol (mime-type-symbol mime-type)))
> 		(if (text-mime-type? mime-type)
> 		    (values
> 		     `((content-type . (,mime-type-symbol)))
> 		     (lambda (out-port)
> 		       (call-with-input-file file-path
> 			 (lambda (in-port)
> 			   (display (read-delimited "" in-port)
>                                                     out-port)))))
> 		    (values
> 		     `((content-type . (,mime-type-symbol)))
> 		     (call-with-input-file file-path
> 			 (lambda (in-port)
> 			   (get-bytevector-all in-port))))))
> 	      (not-found request)))
>
> I hope it's readable :-)

Certainly, it looks good.  Where does mime-type-ref come from, though?

Perhaps that could be wrapped up as `static-file-response' (or a better
name if you can think of one) and added to (web server).

Regards,
     Neil

Re: Serving files with guile web server

[romel]

Neil Jerram writes:
> Certainly, it looks good.  Where does mime-type-ref come from, though?
>

Thanks, mime-type-ref comes from (glider mime-types) it's a little module
I wrote specifically for this purpose, see code below

(define-module (glider mime-types)
  :export (mime-type-ref text-mime-type? mime-type-symbol))

(define *mime-types* (make-hash-table 31))
(hash-set! *mime-types* "css" '("text" . "css"))
(hash-set! *mime-types* "txt" '("text" . "plain"))
(hash-set! *mime-types* "png" '("image" . "png"))
(hash-set! *mime-types* "jpg" '("image" . "jpeg"))
(hash-set! *mime-types* "jpeg" '("image" . "jpeg"))
(hash-set! *mime-types* "gif" '("image" . "gif"))

(define (mime-type-ref file-name)
  (let* ((dot-position (string-rindex file-name #\.))
	 (extension (and dot-position
			 (string-copy file-name (+ dot-position 1))))
	 (mime-type (and dot-position
			 (hash-ref *mime-types* extension))))
    (if mime-type mime-type '("application" . "octet-stream"))))

(define (mime-type-symbol mime-type)
  (string->symbol (string-append (car mime-type) "/" (cdr mime-type))))

(define (text-mime-type? mime-type)
  (if (equal? (car mime-type) "text") #t #f))

> Perhaps that could be wrapped up as `static-file-response' (or a better
> name if you can think of one) and added to (web server).

It would be great to have static file serving available from (web server),
I think that static-file-response is a good name.

Thanks,
Romel Sandoval

Re: Serving files with guile web server

[Andy Wingo]

On Sat 19 Mar 2011 17:47, romel@lavabit.com writes:

>> romel@lavabit.com writes:
>>
>>> I gave a second read to the manual and found:
>>>
>>> "The handler should return two values: the response, as a <response>
>>> record from (web response), and the response body as a string,
>>> bytevector,
>>> or #f if not present."
>
> This is true but I found that when it comes about text is better to use a
> lambda that receive a port and then write text to that port.

I have added info about returning procedures as response bodies to the
doc.  Thanks.

It would be good for sanitize-response to support ports as well, I
think.

> I think that it would be great to have a standalone server so you can
> instant hack something, or even for production, I think it could be
> scalable using clusters.

I think more standalone capabilities is a good thing.  It's nice to be
able to throw up a server for testing.  For production though you really
need to not do blocking reads or writes.

> Guile web module it's great!

Thanks!  Have fun and let us know how it goes :)

Andy
-- 
http://wingolog.org/

Re: Serving files with guile web server

[Thien-Thi Nguyen]

() Neil Jerram <neil@ossau.uklinux.net>
() Sat, 19 Mar 2011 02:20:47 +0000

   Well, I suppose I'd recommend considering not using
   the Guile server to serve static files.

If efficiency is a concern, another approach is to wrap
sendfile(2) or something like it.

 http://www.gnuvola.org/software/ttn-do/ttn-do.html.gz#index-sendfile-140

This is what "ttn-do sizzweb", "ttn-do serve-debiso" and the
serveez packages do, although for the latter the wrapping is
admittedly very thin (C only) at present.  That may change...

 http://www.gnuvola.org/software/ttn-do/ttn-do.html.gz#sizzweb
 http://www.gnuvola.org/software/ttn-do/ttn-do.html.gz#serve-debiso
 http://git.savannah.gnu.org/cgit/serveez.git?h=next
 https://github.com/spk121/serveez-mg/

Probably a good exercise for Guile 2 FFI.  If sendfile (or ilk) is
too Linux- (or ilk) specific, a nice fallback would be to wrap the
POSIX scatter/gather support, i.e., writev(2)/readv(2).

 http://www.gnuvola.org/software/ttn-do/ttn-do.html.gz#index-iovec-142
 http://www.gnuvola.org/software/ttn-do/ttn-do.html.gz#index-writev-144
 http://www.gnuvola.org/software/ttn-do/ttn-do.html.gz#index-readv-143

This is what (ttn-do zzz x-protocol) uses, albeit sub-optimally for now.

 http://www.gnuvola.org/software/ttn-do/ttn-do.html.gz#zzz-x_002dprotocol

Re: Serving files with guile web server

[Andy Wingo]

On Tue 22 Mar 2011 09:51, Thien-Thi Nguyen <ttn@gnuvola.org> writes:

> If efficiency is a concern, another approach is to wrap
> sendfile(2) or something like it.

IMHO we should bind sendfile(2) in Guile itself.

Andy
-- 
http://wingolog.org/

Re: Serving files with guile web server

[Andy Wingo]

On Thu 31 Mar 2011 16:23, Andy Wingo <wingo@pobox.com> writes:

> On Tue 22 Mar 2011 09:51, Thien-Thi Nguyen <ttn@gnuvola.org> writes:
>
>> If efficiency is a concern, another approach is to wrap
>> sendfile(2) or something like it.
>
> IMHO we should bind sendfile(2) in Guile itself.

Well, I guess I take that back:

  http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/25012

But we'll still be here whenever gnulib figures out what it is they want
to name it :)

Andy
-- 
http://wingolog.org/