I/O, modules

I/O, modules

[thorsopia]

Hello,

Could you show me some trivial programs related to I/O (e.g. read from
file, convert to uppercase, write to another file)?

This page [0] doesn't list a function that can be used to read the whole
file at once. Is there a "read-string" function? What about "read-port"?

Should I import some module to get the above functions? Which one?

BTW, I have a problem with modules. How to check what can be imported?

For example:

I created two files in the same dir:

test-scm.scm:

(define-module (test-scm)
  #:export (test-func))

(define (test-func x)
  (+ x 1))

import-test.scm:

(define-module (import-test)
  #:use-module (test-scm))

(display (test-func 2))

Why does "guile import-test.scm" raise "ERROR: no code for module
(test-scm)"?

[0] https://gnu.org/software/guile/manual/guile.html#Reading

Re: I/O, modules

[Ludovic Courtès]

Hi,

thorsopia@lavabit.com skribis:

> Could you show me some trivial programs related to I/O (e.g. read from
> file, convert to uppercase, write to another file)?
>
> This page [0] doesn't list a function that can be used to read the whole
> file at once. Is there a "read-string" function? What about "read-port"?

For textual I/O, search the manual for (ice-9 rdelim).  This module
provides a ‘read-line’ function, which reads a line from a port.

For binary I/O, search for (rnrs io ports).  You’ll get a variety of
procedures, including ‘get-bytevector-all’, which reads all of a ports
contents in memory.

> I created two files in the same dir:
>
> test-scm.scm:
>
> (define-module (test-scm)
>   #:export (test-func))
>
> (define (test-func x)
>   (+ x 1))
>
> import-test.scm:
>
> (define-module (import-test)
>   #:use-module (test-scm))
>
> (display (test-func 2))
>
> Why does "guile import-test.scm" raise "ERROR: no code for module
> (test-scm)"?

Because ‘test-scm.scm’ is not in the search path.

The fix is to run:

  guile -L . import-test.scm

Thanks,
Ludo’.

Re: I/O, modules

[Thien-Thi Nguyen]

[-- Attachment #1.1: Type: text/plain, Size: 204 bytes --]

() thorsopia@lavabit.com
() Tue, 13 Nov 2012 00:57:11 -0500 (EST)

   [...] to read the whole file at once.

   Should I import some module to get the above functions? Which one?

Personally, i would do:

[-- Attachment #1.2: upcase.out --]
[-- Type: text/plain, Size: 856 bytes --]

-*- mode: compilation; default-directory: "/tmp/" -*-
Compilation started at Tue Nov 13 10:04:09

cd /tmp ; cat upcase ; guile -s upcase upcase to ; echo =/= ; cat to
;;; upcase

(use-modules (ttn-do mogrify))

(define (upcase-file i o)
  (call-with-output-file o
    (lambda (o)
      (display (string-upcase
                (editing-buffer (find-file-read-only i)
                  (buffer-string)))
               o))))

(apply upcase-file (cdr (command-line)))

;;; upcase ends here
=/=
;;; UPCASE

(USE-MODULES (TTN-DO MOGRIFY))

(DEFINE (UPCASE-FILE I O)
  (CALL-WITH-OUTPUT-FILE O
    (LAMBDA (O)
      (DISPLAY (STRING-UPCASE
                (EDITING-BUFFER (FIND-FILE-READ-ONLY I)
                  (BUFFER-STRING)))
               O))))

(APPLY UPCASE-FILE (CDR (COMMAND-LINE)))

;;; UPCASE ENDS HERE

Compilation finished at Tue Nov 13 10:04:09

[-- Attachment #1.3: Type: text/plain, Size: 1020 bytes --]


which (trivially) uses module (ttn-do mogrify) for input:
<http://www.glug.org/software/ttn-do/ttn-do.html.gz#mogrify>

If you grep ttn-do, you'll see other examples, most of which do more
inside ‘editing-buffer’ than outside.  In various versions of Guile,
there is module ‘(ice-9 gap-buffer)’, which is "related technology".

   [...] problem with modules. How to check what can be imported?

You can ‘(set! %load-verbosely #t)’ to check.  On a deeper level, Guile
resolves a module name to a filename using var ‘%load-path’, which is
initialized from env var ‘GUILE_LOAD_PATH’ and command-line arg ‘-L DIR’
(one or more).  By default, cwd is excluded from these, for security.
Try to specify it explicitly, e.g.: "guile -L . -s import-test.scm".

At least, this is my understanding of Guile 1.x -- maybe 2.0 differs.

[clickety click]

Hmm, i see you've successfully tempted me into passing a half hour
playing w/ this stuff.  Well done!  Might as well share the fruit.

[-- Attachment #1.4: upcase --]
[-- Type: text/plain, Size: 1288 bytes --]

;;; upcase
;;;
;;; Usage: guile -s upcase INFILE OUTSTEM
;;;
;;; Description: Upcase INFILE to both OUTSTEM-unsafe, OUTSTEM-safe{1,2}.

(use-modules (ttn-do mogrify))

(define (upcase-file i o)

  ;; unsafe: Output fails silently if ‘display’ does short ‘write’.
  ;;         This occurs for some Guile versions, sometimes.
  (let ((o (string-append o "-unsafe")))
    (call-with-output-file o
      (lambda (o)
        (display (string-upcase
                  (editing-buffer (find-file-read-only i)
                    (buffer-string)))
                 o))))

  ;; safe1: As long as you have the memory.  :-D
  (let ((o (string-append o "-safe1")))
    (editing-buffer (find-file i)
      (let ((new (string-upcase (buffer-string))))
        (delete-region (point-min) (point-max))
        (insert new))
      (write-to-port (open-output-file o))))

  ;; safe2: Like safe1, but w/ a more functional style.
  (let ((o (string-append o "-safe2")))
    (editing-buffer (string-upcase
                     (editing-buffer (find-file-read-only i)
                       (buffer-string)))
      (write-to-port (open-output-file o))))

  ;; Add more shouted mumblings here.
  )

(apply upcase-file (cdr (command-line)))

;;; upcase ends here

[-- Attachment #1.5: Type: text/plain, Size: 56 bytes --]


Poking around in ~/codebits/scheme, i also found this:

[-- Attachment #1.6: upcase.scm --]
[-- Type: text/plain, Size: 466 bytes --]

#!/usr/local/bin/guile \
-e main -s
!#

;;; ID: $Id: upcase.scm,v 1.3 1999/05/15 10:15:31 ttn Exp $

(define (toupper s)
  (list->string
   (map (lambda (c)
	  (if (char-lower-case? c)
	      (char-upcase c)
	      c))
	(string->list s))))

(define main
  (lambda args
    (let r ((form (read-line)))
      (or (eof-object? form)
	  (begin
	    (write-line (toupper form))
	    (r (read-line)))))
    (quit)))

;;; $RCSfile: upcase.scm,v $$Revision: 1.3 $ ends here

[-- Attachment #1.7: Type: text/plain, Size: 453 bytes --]


It operates line at a time, which is nice for pipes, etc...  OK, enough!

-- 
Thien-Thi Nguyen ..................................... GPG key: 4C807502
.                  NB: ttn at glug dot org is not me                   .
.                 (and has not been since 2007 or so)                  .
.                        ACCEPT NO SUBSTITUTES                         .
........... please send technical questions to mailing lists ...........

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: I/O, modules

[Mark H Weaver]

thorsopia@lavabit.com writes:
> Could you show me some trivial programs related to I/O (e.g. read from
> file, convert to uppercase, write to another file)?

There are many ways to do this, but here's a simple upcase program that
reads the entire file into memory and does not depend on any external
modules:

--8<---------------cut here---------------start------------->8---
#!/usr/bin/guile \
-e main -s
!#

(use-modules (ice-9 match)
             (ice-9 rdelim))

(define (usage)
  (display "Usage: upcase <in-filename> <out-filename>\n")
  (exit 1))

(define (upcase-file in-filename out-filename)
  (call-with-input-file in-filename
    (lambda (in-port)
      (call-with-output-file out-filename
        (lambda (out-port)
          (display (string-upcase (read-delimited "" in-port))
                   out-port))))))

(define (main args)
  (setlocale LC_ALL "")
  (match args
    ((program in-filename out-filename)
     (upcase-file in-filename out-filename))
    (_ (usage))))
--8<---------------cut here---------------end--------------->8---

> This page [0] doesn't list a function that can be used to read the whole
> file at once.

As shown in the upcase program above, (read-delimited "" port) will read
the whole file as a string.  You need to (use-modules (ice-9 rdelim)) to
import this procedure.

Most of these procedures allow you to omit the 'port' argument, in which
case they will use (current-input-port) or (current-output-port).  Those
can be temporarily set within a block using 'with-input-from-file' and
'with-output-to-file'.  For example, 'upcase-file' above could be
written as follows:

   (define (upcase-file in-filename out-filename)
     (with-input-from-file in-filename
       (lambda ()
         (with-output-to-file out-filename
           (lambda ()
             (display (string-upcase (read-delimited ""))))))))

To read one line from a file, use (read-line port), which is also from
(ice-9 rdelim).  For example, we can use this to modify the upcase
program to read and write one line at a time, so that it will use less
memory for large files (as long as the lines aren't too long):

   (define (upcase-file in-filename out-filename)
     (call-with-input-file in-filename
       (lambda (in-port)
         (call-with-output-file out-filename
           (lambda (out-port)
             (define (loop)
               (let ((line (read-line in-port 'concat)))
                 (unless (eof-object? line)
                   (display (string-upcase line) out-port)
                   (loop))))
             (loop))))))

The 'concat argument to 'read-line' means that it should include the
end-of-line character(s), if any, in the returned string.  By default
they are stripped.

  Happy hacking!
      Mark

Re: I/O, modules

[Mike Gran]

> From: Mark H Weaver <mhw@netris.org>

> 
> As shown in the upcase program above, (read-delimited "" port) will 
> read
> the whole file as a string.  You need to (use-modules (ice-9 rdelim)) to
> import this procedure.

Nice.  For reading entire files, I always ended up doing something like
the following (which does reveal how I think of everything in low-level C
constructs, I suppose.)

(use-modules (ice-9 rw))

(define (read-all port) (let loop ((block (make-string 32000))
(text ""))
    (let ((count (read-string!/partial block port)))
      (if count
	  (loop block 
(string-append text (substring block 0 count)))
	  text))))-Mike

Re: I/O, modules

[Ian Price]

Mark H Weaver <mhw@netris.org> writes:

> As shown in the upcase program above, (read-delimited "" port) will read
> the whole file as a string.  You need to (use-modules (ice-9 rdelim)) to
> import this procedure.

A perhaps more lucid way is to use get-string-all from (rnrs io ports)

-- 
Ian Price -- shift-reset.com

"Programming is like pinball. The reward for doing it well is
the opportunity to do it again" - from "The Wizardy Compiled"

Re: I/O, modules

[tantalum]

Hi

The (ice-9 streams) module could be interesting, too.
http://www.gnu.org/software/guile/manual/html_node/Streams.html

Example:
-------------
(import
  (ice-9 streams)
  (ice-9 rdelim))

(define (port->line-stream port handle-delim)
  (port->stream port (lambda (port) (read-line port handle-delim))))

(define (file-upcase source-path target-path)
  (call-with-input-file source-path
    (lambda (source-file)
      (call-with-output-file target-path
        (lambda (target-file)
          (stream-for-each
            (lambda (line) (display (string-upcase line) target-file))
            (port->line-stream source-file (quote concat))))))))


;test
(system "echo test >/tmp/file-upcase-test")
(file-upcase "/tmp/file-upcase-test" "/tmp/file-upcase-test-out")
--------------
Where
  - Handle-delim specifies what should happen with the newline character. Default is to remove it
  - And display is Schemes standard string write procedure

I don't know of any existing generic file to string procedure.
Something similar to what I have used is:
---------------
(import
  (ice-9 streams)
  (ice-9 rdelim))

(define (port->line-stream port handle-delim)
  (port->stream port (lambda (port) (read-line port handle-delim))))

(define (file->string file)
  (apply string-append
    (stream->list (port->line-stream file (quote concat)))))

(define (file-from-path->string path)
  (call-with-input-file path file->string))

(display (file-from-path->string "/home/jkalbhenn/temp/temp.scm"))
-------------

Julian

Re: I/O, modules

[Ian Price]

tantalum@gmx.net writes:

> The (ice-9 streams) module could be interesting, too.
> http://www.gnu.org/software/guile/manual/html_node/Streams.html

When it gets merged into master/stable-2.0, I'd suggest using (srfi
srfi-41) rather than (ice-9 streams). It cures the off-by-1 problems
that frequently occur with "odd" streams.

Another approach to stream based IO is called "iteratees", as proposed
by Oleg Kiselyov.[0] I had started working on an implementation[1], but
it quite incomplete at the moment. Maybe one day...

0. http://okmij.org/ftp/Streams.html
1. https://github.com/ijp/iteratees/blob/master/iteratees.sls
-- 
Ian Price -- shift-reset.com

"Programming is like pinball. The reward for doing it well is
the opportunity to do it again" - from "The Wizardy Compiled"