read-all ?

read-all ?

[Andy Wingo]

I find myself writing (read-delimited "" p) to slurp in a file as a
string, but it's not a very straightforward way to say that.

What about `read-all'?  We could add it to `(ice-9 rdelim)', I guess.
R6RS calls this `read-string-all'.

Thoughts?

Andy
-- 
http://wingolog.org/

Re: read-all ?

[Ludovic Courtès]

Hi!

Andy Wingo <wingo@pobox.com> skribis:

> I find myself writing (read-delimited "" p) to slurp in a file as a
> string, but it's not a very straightforward way to say that.
>
> What about `read-all'?  We could add it to `(ice-9 rdelim)', I guess.
> R6RS calls this `read-string-all'.

Sounds like a good idea.

(Believe it or not, in light of the recent RnRS discussion: I always use
‘get-string-all’ for that.  :-))

Ludo’.

Re: read-all ?

[Andy Wingo]

Hi,

On Sat 12 Jan 2013 22:22, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> skribis:
>
>> I find myself writing (read-delimited "" p) to slurp in a file as a
>> string, but it's not a very straightforward way to say that.
>>
>> What about `read-all'?  We could add it to `(ice-9 rdelim)', I guess.
>> R6RS calls this `read-string-all'.
>
> Sounds like a good idea.

Patch attached.  I didn't update the docs because it wasn't clear to me
that (ice-9 rdelim) is actually the right place to put it.

What do you think?  Should we perhaps put it in a new (ice-9 ports)?
Are the names right?

I started by writing it in C but I noticed the C was doing a very stupid
get-and-set algorithm, so I figured it got no advantage and we should
just write it in Scheme from the get-go.

Andy
-- 
http://wingolog.org/

Re: read-all ?

[Andy Wingo]

[-- Attachment #1: Type: text/plain, Size: 934 bytes --]

On Tue 22 Jan 2013 10:15, Andy Wingo <wingo@pobox.com> writes:

> Hi,
>
> On Sat 12 Jan 2013 22:22, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Andy Wingo <wingo@pobox.com> skribis:
>>
>>> I find myself writing (read-delimited "" p) to slurp in a file as a
>>> string, but it's not a very straightforward way to say that.
>>>
>>> What about `read-all'?  We could add it to `(ice-9 rdelim)', I guess.
>>> R6RS calls this `read-string-all'.
>>
>> Sounds like a good idea.
>
> Patch attached.  I didn't update the docs because it wasn't clear to me
> that (ice-9 rdelim) is actually the right place to put it.
>
> What do you think?  Should we perhaps put it in a new (ice-9 ports)?
> Are the names right?
>
> I started by writing it in C but I noticed the C was doing a very stupid
> get-and-set algorithm, so I figured it got no advantage and we should
> just write it in Scheme from the get-go.
>
> Andy


[-- Attachment #2: 0001-add-read-all-and-read-all-to-ice-9-rdelim.patch --]
[-- Type: text/x-diff, Size: 6340 bytes --]

From 056c69dee301f346d172293b71dfdc66ddfa0282 Mon Sep 17 00:00:00 2001
From: Andy Wingo <wingo@pobox.com>
Date: Tue, 22 Jan 2013 10:12:59 +0100
Subject: [PATCH] add read-all and read-all! to (ice-9 rdelim)

* module/ice-9/rdelim.scm (read-all!, read-all): New functions.
* test-suite/tests/rdelim.test: Add tests.
---
 module/ice-9/rdelim.scm      |   52 +++++++++++++++++++++++++++++++++++++-
 test-suite/tests/rdelim.test |   57 ++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 106 insertions(+), 3 deletions(-)

diff --git a/module/ice-9/rdelim.scm b/module/ice-9/rdelim.scm
index c6ab2ba..9938942 100644
--- a/module/ice-9/rdelim.scm
+++ b/module/ice-9/rdelim.scm
@@ -1,6 +1,6 @@
 ;;; installed-scm-file
 
-;;;; Copyright (C) 1997, 1999, 2000, 2001, 2006, 2010 Free Software Foundation, Inc.
+;;;; Copyright (C) 1997, 1999, 2000, 2001, 2006, 2010, 2013 Free Software Foundation, Inc.
 ;;;; 
 ;;;; This library is free software; you can redistribute it and/or
 ;;;; modify it under the terms of the GNU Lesser General Public
@@ -26,6 +26,8 @@
             read-line!
             read-delimited
             read-delimited!
+            read-all
+            read-all!
             %read-delimited!
             %read-line
             write-line))
@@ -114,6 +116,54 @@
             (else (error "unexpected handle-delim value: "
                          handle-delim)))))))))
 
+(define-syntax-rule (check-arg arg exp message)
+  (unless exp
+    (error message arg)))
+
+(define (index? n)
+  (and (integer? n) (exact? n) (>= n 0)))
+
+(define* (read-all! buf #:optional
+                    (port (current-input-port))
+                    (start 0) (end (string-length buf)))
+  "Read all of the characters out of PORT and write them to BUF.
+Returns the number of characters read.
+
+This function only reads out characters from PORT if it will be able to
+write them to BUF.  That is to say, if BUF is smaller than the number of
+available characters, then BUF will be filled, and characters will be
+left in the port."
+  (check-arg buf (string? buf) "not a string")
+  (check-arg start (index? start) "bad index")
+  (check-arg end (index? end) "bad index")
+  (check-arg start (<= start end) "start beyond end")
+  (check-arg end (<= end (string-length buf)) "end beyond string length")
+  (let lp ((n start))
+    (if (< n end)
+        (let ((c (read-char port)))
+          (if (eof-object? c)
+              (- n start)
+              (begin
+                (string-set! buf n c)
+                (lp (1+ n)))))
+        (- n start))))
+
+(define* (read-all #:optional (port (current-input-port)))
+  "Read all of the characters out of PORT and return them as a string."
+  (let loop ((substrings '())
+             (total-chars 0)
+             (buf-size 100))		; doubled each time through.
+    (let* ((buf (make-string buf-size))
+           (nchars (read-all! buf port))
+           (new-total (+ total-chars nchars)))
+      (cond
+       ((= nchars buf-size)
+        ;; buffer filled.
+        (loop (cons buf substrings) new-total (* buf-size 2)))
+       (else
+        (string-concatenate-reverse
+         (cons (substring buf 0 nchars) substrings)))))))
+
 ;;; read-line [PORT [HANDLE-DELIM]] reads a newline-terminated string
 ;;; from PORT.  The return value depends on the value of HANDLE-DELIM,
 ;;; which may be one of the symbols `trim', `concat', `peek' and
diff --git a/test-suite/tests/rdelim.test b/test-suite/tests/rdelim.test
index e61fc92..a102df6 100644
--- a/test-suite/tests/rdelim.test
+++ b/test-suite/tests/rdelim.test
@@ -1,7 +1,7 @@
 ;;;; rdelim.test --- Delimited I/O.      -*- mode: scheme; coding: utf-8; -*-
 ;;;; Ludovic Courtès <ludo@gnu.org>
 ;;;;
-;;;; 	Copyright (C) 2011 Free Software Foundation, Inc.
+;;;; 	Copyright (C) 2011, 2013 Free Software Foundation, Inc.
 ;;;;
 ;;;; This library is free software; you can redistribute it and/or
 ;;;; modify it under the terms of the GNU Lesser General Public
@@ -189,7 +189,60 @@
 
     (pass-if "eof, split"
       (eof-object? (read-delimited! ":" (make-string 7)
-                                    (open-input-string ""))))))
+                                    (open-input-string "")))))
+
+  (with-test-prefix "read-all"
+
+    (pass-if "short string"
+      (let* ((s "hello, world!")
+             (p (open-input-string s)))
+        (and (string=? (read-all p) s)
+             (string=? (read-all p) ""))))
+
+    (pass-if "100 chars"
+      (let* ((s (make-string 100 #\space))
+             (p (open-input-string s)))
+        (and (string=? (read-all p) s)
+             (string=? (read-all p) ""))))
+
+    (pass-if "longer than 100 chars"
+      (let* ((s (string-concatenate (make-list 20 "hello, world!")))
+             (p (open-input-string s)))
+        (and (string=? (read-all p) s)
+             (string=? (read-all p) "")))))
+
+  (with-test-prefix "read-all!"
+
+    (pass-if "buf smaller"
+      (let* ((s "hello, world!")
+             (len (1- (string-length s)))
+             (buf (make-string len #\.))
+             (p (open-input-string s)))
+        (and (= (read-all! buf p) len)
+             (string=? buf (substring s 0 len))
+             (= (read-all! buf p) 1)
+             (string=? (substring buf 0 1) (substring s len)))))
+
+    (pass-if "buf right size"
+      (let* ((s "hello, world!")
+             (len (string-length s))
+             (buf (make-string len #\.))
+             (p (open-input-string s)))
+        (and (= (read-all! buf p) len)
+             (string=? buf (substring s 0 len))
+             (= (read-all! buf p) 0)
+             (string=? buf (substring s 0 len)))))
+
+    (pass-if "buf bigger"
+      (let* ((s "hello, world!")
+             (len (string-length s))
+             (buf (make-string (1+ len) #\.))
+             (p (open-input-string s)))
+        (and (= (read-all! buf p) len)
+             (string=? (substring buf 0 len) s)
+             (= (read-all! buf p) 0)
+             (string=? (substring buf 0 len) s)
+             (string=? (substring buf len) "."))))))
 
 ;;; Local Variables:
 ;;; eval: (put 'with-test-prefix 'scheme-indent-function 1)
-- 
1.7.10.4


[-- Attachment #3: Type: text/plain, Size: 26 bytes --]


-- 
http://wingolog.org/

Re: read-all ?

[Ludovic Courtès]

Hi!

Andy Wingo <wingo@pobox.com> skribis:

> On Tue 22 Jan 2013 10:15, Andy Wingo <wingo@pobox.com> writes:

[...]

>> Patch attached.  I didn't update the docs because it wasn't clear to me
>> that (ice-9 rdelim) is actually the right place to put it.
>>
>> What do you think?  Should we perhaps put it in a new (ice-9 ports)?

I would avoid adding a new module, unless there are other things to put
in there.

>> Are the names right?

‘read-all’ doesn’t convey the idea that it’s textual (unlike the R6RS
names).

Perhaps ‘port-contents-as-string’, or ‘read-all-string’, or...?

> +(define* (read-all! buf #:optional
> +                    (port (current-input-port))
> +                    (start 0) (end (string-length buf)))
> +  "Read all of the characters out of PORT and write them to BUF.
> +Returns the number of characters read.
> +
> +This function only reads out characters from PORT if it will be able to
> +write them to BUF.  That is to say, if BUF is smaller than the number of
> +available characters, then BUF will be filled, and characters will be
> +left in the port."
> +  (check-arg buf (string? buf) "not a string")
> +  (check-arg start (index? start) "bad index")
> +  (check-arg end (index? end) "bad index")
> +  (check-arg start (<= start end) "start beyond end")
> +  (check-arg end (<= end (string-length buf)) "end beyond string length")
> +  (let lp ((n start))
> +    (if (< n end)
> +        (let ((c (read-char port)))
> +          (if (eof-object? c)
> +              (- n start)
> +              (begin
> +                (string-set! buf n c)
> +                (lp (1+ n)))))
> +        (- n start))))

As you note, this is fairly inefficient, like ‘get-string-n!’.

Given that ‘string-set!’ is (unduly) costly, I wonder if consing all the
chars and then calling ‘list->string’ wouldn’t be more efficient in time
(it’d be less efficient in space.)

Thanks!

Ludo’.

Re: read-all ?

[Andy Wingo]

On Tue 22 Jan 2013 11:01, ludo@gnu.org (Ludovic Courtès) writes:

>>> Are the names right?
>
> ‘read-all’ doesn’t convey the idea that it’s textual (unlike the R6RS
> names).
>
> Perhaps ‘port-contents-as-string’, or ‘read-all-string’, or...?

What about read-string with an optional #:count argument ?

>> +  (let lp ((n start))
>> +    (if (< n end)
>> +        (let ((c (read-char port)))
>> +          (if (eof-object? c)
>> +              (- n start)
>> +              (begin
>> +                (string-set! buf n c)
>> +                (lp (1+ n)))))
>> +        (- n start))))
>
> As you note, this is fairly inefficient, like ‘get-string-n!’.

It is exactly what %read-delimited does, though, and that's the most
efficient thing we have.  read-string/partial! does something more
clever, but I don't understand it fully.  I was thinking (ice-9 ports)
could be appropriate if we exposed port buffers to scheme, because that
way we could read characters in bulk.  Dunno.

> Given that ‘string-set!’ is (unduly) costly, I wonder if consing all the
> chars and then calling ‘list->string’ wouldn’t be more efficient in time
> (it’d be less efficient in space.)

Probably not, I would say that the interface is more important than the
implementation.

So, proposal: read-all{,!} to read-string{,!} and add optional count
argument to read-string, and leave it in ice-9 rdelim.  WDYT?

Andy
-- 
http://wingolog.org/

Re: read-all ?

[Ludovic Courtès]

Andy Wingo <wingo@pobox.com> skribis:

> So, proposal: read-all{,!} to read-string{,!} and add optional count
> argument to read-string, and leave it in ice-9 rdelim.  WDYT?

Go for it!

Ludo’.

Re: read-all ?

[Andy Wingo]

On Tue 22 Jan 2013 14:32, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> skribis:
>
>> So, proposal: read-all{,!} to read-string{,!} and add optional count
>> argument to read-string, and leave it in ice-9 rdelim.  WDYT?
>
> Go for it!

Done, thanks for the review :)

Andy
-- 
http://wingolog.org/