unofficial mirror of guile-user@gnu.org 
 help / color / mirror / Atom feed
* Nonblocking get-bytevector-n bug?
@ 2015-12-06 20:38 Marko Rauhamaa
  2015-12-07  9:50 ` Ludovic Courtès
                   ` (2 more replies)
  0 siblings, 3 replies; 16+ messages in thread
From: Marko Rauhamaa @ 2015-12-06 20:38 UTC (permalink / raw)
  To: guile-user


Guile 2.0.11 provides:

   get-bytevector-n
   get-bytevector-n!
   get-bytevector-some

Of these, only get-bytevector-some seems to behave as expected when the
port is nonblocking:

========================================================================
(use-modules (rnrs io ports)
             (rnrs bytevectors))

(let ((port (fdes->inport 0)))
  (fcntl port F_SETFL (logior O_NONBLOCK (fcntl port F_GETFL)))
  (let loop ()
    (catch 'system-error
      (lambda ()
        (format #t "~s\n" (get-bytevector-some port)))
      (lambda syserr
        (format #t "err ~s\n" (system-error-errno syserr))))
    (sleep 1)
    (loop)))
========================================================================

If you replace (get-bytevector-some port) with
(get-bytevector-n port 8192) or (get-bytevector-n! port bv 0 8192),
no partial data is returned.

The problem with get-bytevector-some is that there is no limit to how
many bytes might be returned. In practice, I see that the amount is
capped at 4096 bytes, but the documentation does not guarantee any
limit.


Marko



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-06 20:38 Nonblocking get-bytevector-n bug? Marko Rauhamaa
@ 2015-12-07  9:50 ` Ludovic Courtès
  2015-12-07 10:21   ` Taylan Ulrich Bayırlı/Kammer
  2015-12-07 10:28   ` Marko Rauhamaa
  2015-12-07 12:44 ` Amirouche Boubekki
  2015-12-08 15:39 ` Mark H Weaver
  2 siblings, 2 replies; 16+ messages in thread
From: Ludovic Courtès @ 2015-12-07  9:50 UTC (permalink / raw)
  To: guile-user

Marko Rauhamaa <marko@pacujo.net> skribis:

> Of these, only get-bytevector-some seems to behave as expected when the
> port is nonblocking:
>
> ========================================================================
> (use-modules (rnrs io ports)
>              (rnrs bytevectors))
>
> (let ((port (fdes->inport 0)))
>   (fcntl port F_SETFL (logior O_NONBLOCK (fcntl port F_GETFL)))
>   (let loop ()
>     (catch 'system-error
>       (lambda ()
>         (format #t "~s\n" (get-bytevector-some port)))
>       (lambda syserr
>         (format #t "err ~s\n" (system-error-errno syserr))))
>     (sleep 1)
>     (loop)))
> ========================================================================
>
> If you replace (get-bytevector-some port) with
> (get-bytevector-n port 8192) or (get-bytevector-n! port bv 0 8192),
> no partial data is returned.

That’s because get-bytevector-n{,!} are specified as blocking if
necessary until N bytes are read or EOF is reached.

> The problem with get-bytevector-some is that there is no limit to how
> many bytes might be returned. In practice, I see that the amount is
> capped at 4096 bytes, but the documentation does not guarantee any
> limit.

Right, the R6RS specification of that procedure doesn’t tell anything
more either.

It was criticized back in the day partly for that reason:

  http://www.r6rs.org/formal-comments/comment-224.txt

I think a ‘get-bytevector-some!’ procedure could address the issue.

Not sure what R7 does here.

Thoughts?

Ludo’.




^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-07  9:50 ` Ludovic Courtès
@ 2015-12-07 10:21   ` Taylan Ulrich Bayırlı/Kammer
  2015-12-07 10:58     ` Taylan Ulrich Bayırlı/Kammer
  2015-12-07 10:28   ` Marko Rauhamaa
  1 sibling, 1 reply; 16+ messages in thread
From: Taylan Ulrich Bayırlı/Kammer @ 2015-12-07 10:21 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guile-user

ludo@gnu.org (Ludovic Courtès) writes:

> Not sure what R7 does here.

R7RS section 6.13.2 Input, page 58, defines:

    (read-bytevector! bytevector [port [start [end]]])

    Reads the next END - START bytes, or as many as are available before
    the end of file, from the binary input PORT into BYTEVECTOR in
    left-to-right order beginning at the START position.  If END is not
    supplied, reads until the end of BYTEVECTOR has been reached.  If
    START is not supplied, reads beginning at position 0.  Returns the
    number of bytes read.  If no bytes are available, an end-of-file
    object is returned.

Maybe the r7rs-wip branch has an implementation of it.

Taylan



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-07  9:50 ` Ludovic Courtès
  2015-12-07 10:21   ` Taylan Ulrich Bayırlı/Kammer
@ 2015-12-07 10:28   ` Marko Rauhamaa
  2015-12-07 11:15     ` Marko Rauhamaa
  1 sibling, 1 reply; 16+ messages in thread
From: Marko Rauhamaa @ 2015-12-07 10:28 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guile-user

ludo@gnu.org (Ludovic Courtès):

>> If you replace (get-bytevector-some port) with
>> (get-bytevector-n port 8192) or (get-bytevector-n! port bv 0 8192),
>> no partial data is returned.
>
> That’s because get-bytevector-n{,!} are specified as blocking if
> necessary until N bytes are read or EOF is reached.

Ok, the Guile manual description leaves it a ambiguous. Even the R6RS
wording could be graciously bent slightly to support the age-old
nonblocking semantics. It boils down to what bytes are considered
"available." I don't think any nonblocking application would mind those
functions returning immediately with what they managed to read.

Thanks for the answer.


Marko



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-07 10:21   ` Taylan Ulrich Bayırlı/Kammer
@ 2015-12-07 10:58     ` Taylan Ulrich Bayırlı/Kammer
  2015-12-07 14:42       ` Ludovic Courtès
  0 siblings, 1 reply; 16+ messages in thread
From: Taylan Ulrich Bayırlı/Kammer @ 2015-12-07 10:58 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guile-user

taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Not sure what R7 does here.
>
> R7RS section 6.13.2 Input, page 58, defines:
>
>     (read-bytevector! bytevector [port [start [end]]])
>
>     Reads the next END - START bytes, or as many as are available before
>     the end of file, from the binary input PORT into BYTEVECTOR in
>     left-to-right order beginning at the START position.  If END is not
>     supplied, reads until the end of BYTEVECTOR has been reached.  If
>     START is not supplied, reads beginning at position 0.  Returns the
>     number of bytes read.  If no bytes are available, an end-of-file
>     object is returned.
>
> Maybe the r7rs-wip branch has an implementation of it.

Eh, never mind, this seems to imply blocking ("as many as are available
before the end of file").

Taylan



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-07 10:28   ` Marko Rauhamaa
@ 2015-12-07 11:15     ` Marko Rauhamaa
  0 siblings, 0 replies; 16+ messages in thread
From: Marko Rauhamaa @ 2015-12-07 11:15 UTC (permalink / raw)
  To: guile-user

Marko Rauhamaa <marko@pacujo.net>:

> ludo@gnu.org (Ludovic Courtès):
>> That’s because get-bytevector-n{,!} are specified as blocking if
>> necessary until N bytes are read or EOF is reached.
>
> Ok, the Guile manual description leaves it a ambiguous. Even the R6RS
> wording could be graciously bent slightly to support the age-old
> nonblocking semantics. It boils down to what bytes are considered
> "available." I don't think any nonblocking application would mind those
> functions returning immediately with what they managed to read.

In fact, I think Guile definitely *should* bend the R6RS specification
because:

  1. Guile 2.0's get-bytevector-n(!) does not comply with the strict
     R6RS reading anyway. The Guile implementation comes out with a
     system-error (EAGAIN) exception in case a nonblocking port does not
     have any bytes available at the moment. A strict reading would
     entail spinning in a tight loop, which would be silly, of course.

  2. Partial reads throw a system-error (EAGAIN) and lose the already
     read bytes, which makes Guile's get-bytevector-n(!) completely
     unusable for nonblocking ports.

  3. The R6RS wording suggests nonblocking ports were simply overlooked.
     A looser interpretation could be considered compliant.

So, I think get-bytevector-n(!) could become useful if:

  A. If count bytes are available at the moment, follow the spec (no
     change to Guile 2.0.11).

  B. If eof is encountered, follow the spec (no change to Guile 2.0.11).

  C. If no bytes are available at the moment, a system-error (EGAIN)
     exception is thrown (no change to Guile 2.0.11).

  D. If some but fewer than count bytes are available at the moment,
     transfer those bytes and return the number of bytes transferred.

Only item D is new. Wouldn't implementing it have only positives and no
negatives? In fact, without it, how could nonblocking I/O be carried out
effectively?

Note also get-u8 and get-char, which *are* usefully implemented by Guile
for nonblocking ports but are similarly in violation of R6RS (they don't
block until a byte or complete character is available or eof is
reached).


Marko



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-06 20:38 Nonblocking get-bytevector-n bug? Marko Rauhamaa
  2015-12-07  9:50 ` Ludovic Courtès
@ 2015-12-07 12:44 ` Amirouche Boubekki
  2015-12-07 12:52   ` Marko Rauhamaa
  2015-12-08 15:39 ` Mark H Weaver
  2 siblings, 1 reply; 16+ messages in thread
From: Amirouche Boubekki @ 2015-12-07 12:44 UTC (permalink / raw)
  To: Marko Rauhamaa; +Cc: guile-user, guile-user-bounces+amirouche=hypermove.net

[-- Attachment #1: Type: text/plain, Size: 1315 bytes --]

Le 2015-12-06 21:38, Marko Rauhamaa a écrit :
> Guile 2.0.11 provides:
> 
>    get-bytevector-n
>    get-bytevector-n!
>    get-bytevector-some
> 
> Of these, only get-bytevector-some seems to behave as expected when the
> port is nonblocking:

First thanks for pointing the correct way to setup a non-blocking port. 
I was missing that bit in my code.

I can't make work `get-bytevector-some` with my code. It only returns a 
single cell bv.
Instead I use `char-ready?`. Here is the definition of `recv-some`:

```
(define (recv-some port)
   (let next ((out '()))
     (if (char-ready? port)
       (let ((byte (get-u8 port)))
         (if (eof-object? byte)
             (u8-list->bytevector (reverse out))
             (next (cons byte out))))
       (u8-list->bytevector (reverse out)))))
```

FWIW, i also have a send-all procedure that works around the fact that 
the socket is non blocking:

```
(define (send-all socket message cc)
   (let loop ((message message))
     (let* ((count (send socket message))
            (message (bytevector-drop message count)))
       (if (eq? (bytevector-length message) 0)
           (cc)
           (loop-add-writer socket (lambda () (loop message)))))))
```

`loop-add-writer` register a thunk to be run when the socket is ready 
for writes.

I attached tests files.

[-- Attachment #2: async.scm --]
[-- Type: text/plain, Size: 6222 bytes --]

;; Copyright (C) 2015 Amirouche Boubekki <amirouche@hypermove.net>

;; This library is free software; you can redistribute it and/or
;; modify it under the terms of the GNU Lesser General Public
;; License as published by the Free Software Foundation; either
;; version 3 of the License, or (at your option) any later version.
;;
;; This library is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
;; Lesser General Public License for more details.
;;
;; You should have received a copy of the GNU Lesser General Public
;; License along with this library; if not, write to the Free Software
;; Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
;; 02110-1301 USA
(define-module (async))

(use-modules (srfi srfi-69)) ;; hash-table

(use-modules (ice-9 q))
(use-modules (ice-9 match))
(use-modules (ice-9 rdelim))

(use-modules (srfi srfi-9))


;; srfi-queue :o)
(define make-queue make-q)
(define push! enq!)
(define pop! deq!)
(define empty? q-empty?)


(define-record-type <loop>
  (make-loop running tasks readers writers)
  loop?
  (running loop-running loop-running!)
  (tasks loop-tasks loop-tasks!)
  (readers loop-readers loop-readers!)
  (writers loop-writers loop-writers!))


(define loop (make-loop #false (make-queue) (make-hash-table) (make-hash-table)))

(define-public *loop* (make-fluid loop))

(define-public (loop-call-later thunk) ;; support delay
  (let ((loop (fluid-ref *loop*)))
    (push! (loop-tasks loop) thunk)))

(define (loop-add-reader port callback)
  (let* ((loop (fluid-ref *loop*))
         (readers (loop-readers loop))
         (queue (hash-table-ref readers port make-queue)))
    (push! queue callback)
    (hash-table-set! readers port queue)))

(define (loop-add-writer port callback)
  (let* ((loop (fluid-ref *loop*))
         (writers (loop-writers loop))
         (queue (hash-table-ref writers port make-queue)))
    (push! queue callback)
    (hash-table-set! writers port queue)))

(define (call-read-callback port)
  (let* ((loop (fluid-ref *loop*))
         (queue (hash-table-ref (loop-readers loop) port)))
    (when (and queue (not (empty? queue)))
      (let ((callback (pop! queue)))
      (when (empty? queue)
        (hash-table-delete! (loop-readers loop) port))
      (callback)))))

(define (call-write-callback port)
  (let* ((loop (fluid-ref *loop*))
         (queue (hash-table-ref (loop-writers loop) port)))
    (when (and queue (not (empty? queue)))
      (let ((callback (pop! queue)))
        (when (empty? queue)
          (hash-table-delete! (loop-writers loop) port))
        (callback)))))

(define (loop-run-once)
  (let ((loop (fluid-ref *loop*))
        (readers (hash-table-keys (loop-readers loop)))
        (writers (hash-table-keys (loop-writers loop))))
    ;; first poll ready ports
    (match (select readers writers '() 0) ;; FIXME: replace 0 with time for next task
      ((to-read to-write _)
       (for-each call-read-callback to-read)
       (for-each call-write-callback to-write)))
    ;; execute tasks
    (while (not (empty? (loop-tasks loop)))
      ((pop! (loop-tasks loop))))))

(define-public (loop-run-forever)
  (let* ((loop (fluid-ref *loop*)))
    (loop-running! loop #true)
    (while (loop-running loop)
      (call-with-prompt 'loop
        loop-run-once
        (lambda (cc callback) (callback cc))))))

(define-public (loop-running?)
  (let ((loop (fluid-ref *loop*)))
    (loop-running loop)))

(define-public (loop-stop!)
  (let ((loop (fluid-ref *loop*)))
    (loop-running! loop #false)))


;; non-blocking replacement for read/write procedures

(use-modules (ice-9 binary-ports))

(define (recv-some port)
  (let next ((out '()))
    (if (char-ready? port)
      (let ((byte (get-u8 port)))
        (if (eof-object? byte)
            (pk 'eof (u8-list->bytevector (reverse out)))
            (next (cons byte out))))
      (pk '!ready (u8-list->bytevector (reverse out))))))
             
(define-public (recv-some/ socket)
  (abort-to-prompt 'loop (lambda (cc) (loop-add-reader socket (lambda () (cc (recv-some socket)))))))

(define-public (get-bytevector-some/ socket)
  (abort-to-prompt 'loop (lambda (cc) (loop-add-reader socket (lambda () (cc (get-bytevector-some socket)))))))

(define-public (accept/ socket)
  (abort-to-prompt 'loop (lambda (cc) (loop-add-reader socket (lambda () (cc (accept socket)))))))

(use-modules (rnrs bytevectors))

(define (bytevector-drop bv count)
  (u8-list->bytevector (list-tail (bytevector->u8-list bv) count)))


(define (send-all socket message cc)
  (let loop ((message message))
    (let* ((count (send socket message))
           (message (bytevector-drop message count)))
      (if (eq? (bytevector-length message) 0)
          (cc)
          (loop-add-writer socket (lambda () (loop message)))))))

(define-public (send-all/ socket message)
  (abort-to-prompt 'loop (lambda (cc) (loop-add-writer socket (lambda () (send-all socket message cc))))))


;; XXX: those are blocking anyway

;; (define-public (read/ socket)
;;   (abort-to-prompt 'loop (lambda (cc) (loop-add-reader socket (lambda () (cc (read socket)))))))

;; (define-public (write/ message socket)
;;   (abort-to-prompt 'loop (lambda (cc) (loop-add-writer socket (lambda () (cc (write message socket)))))))

;; (define-public (display/ message socket)
;;   (abort-to-prompt 'loop (lambda (cc) (loop-add-writer socket (lambda () (cc (display message socket)))))))

;; (define-public (read-line/ socket)
;;   (abort-to-prompt 'loop (lambda (cc) (loop-add-writer socket (lambda () (cc (read-line socket)))))))

;;; basic TCP sockets

(define make-socket socket)

(define-public (make-client-socket port)
  (let ((socket (make-socket PF_INET SOCK_STREAM 0)))
    (connect socket AF_INET INADDR_LOOPBACK port)
    (fcntl socket F_SETFL (logior O_NONBLOCK (fcntl socket F_GETFL)))
    socket))

(define-public (make-server-socket port)
  (let ((socket (make-socket PF_INET SOCK_STREAM 0)))
    (bind socket (make-socket-address AF_INET INADDR_ANY port))
    (listen socket 128)
    (fcntl socket F_SETFL (logior O_NONBLOCK (fcntl socket F_GETFL)))
    socket))

[-- Attachment #3: client.scm --]
[-- Type: text/plain, Size: 562 bytes --]

(define-module (client))

(use-modules (rnrs bytevectors))

(use-modules (async))


(setlocale LC_ALL "")


(define (scm->string scm)
  (call-with-output-string
    (lambda (port)
      (write scm port))))

(define scm->bv (compose string->utf8 scm->string))  

(define (client)
  (define socket (make-client-socket 12345))
  (send-all/ socket (string->utf8 "héllo world"))
  (send-all/ socket #vu8(0))
  (send-all/ socket (string->utf8 "you are free"))
  (send-all/ socket #vu8(0))
  (close socket)
  (loop-stop!))

(loop-call-later client)
(loop-run-forever)

[-- Attachment #4: server.scm --]
[-- Type: text/plain, Size: 417 bytes --]

(define-module (server))

(use-modules (rnrs bytevectors))

(use-modules (async))


(setlocale LC_ALL "")

;; very simple server socket helper

(define (server)
  (let* ((sock (make-server-socket 12345))
         (client (car (accept/ sock))))
    (pk (utf8->string (recv-some/ client)))
    (pk (utf8->string (recv-some/ client)))
    (close client)
    (loop-stop!)))


(loop-call-later server)

(loop-run-forever)

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-07 12:44 ` Amirouche Boubekki
@ 2015-12-07 12:52   ` Marko Rauhamaa
  0 siblings, 0 replies; 16+ messages in thread
From: Marko Rauhamaa @ 2015-12-07 12:52 UTC (permalink / raw)
  To: Amirouche Boubekki; +Cc: guile-user, guile-user-bounces+amirouche=hypermove.net

Amirouche Boubekki <amirouche@hypermove.net>:

> I can't make work `get-bytevector-some` with my code. It only returns
> a single cell bv.

Run my test code as follows:

   cat | guile test.scm

That way, you get the natural pipe semantics for stdin.

> Instead I use `char-ready?`. Here is the definition of `recv-some`:
>
> ```
> (define (recv-some port)
>   (let next ((out '()))
>     (if (char-ready? port)
>       (let ((byte (get-u8 port)))
>         (if (eof-object? byte)
>             (u8-list->bytevector (reverse out))
>             (next (cons byte out))))
>       (u8-list->bytevector (reverse out)))))
> ```

Your code is ok but very micromanagerial. I really would like the
bytevector variants to do I/O in bulk.


Marko



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-07 10:58     ` Taylan Ulrich Bayırlı/Kammer
@ 2015-12-07 14:42       ` Ludovic Courtès
  2015-12-08 10:53         ` tomas
  0 siblings, 1 reply; 16+ messages in thread
From: Ludovic Courtès @ 2015-12-07 14:42 UTC (permalink / raw)
  To: Taylan Ulrich "Bayırlı/Kammer"; +Cc: guile-user

taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:

> taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> Not sure what R7 does here.
>>
>> R7RS section 6.13.2 Input, page 58, defines:
>>
>>     (read-bytevector! bytevector [port [start [end]]])
>>
>>     Reads the next END - START bytes, or as many as are available before
>>     the end of file, from the binary input PORT into BYTEVECTOR in
>>     left-to-right order beginning at the START position.  If END is not
>>     supplied, reads until the end of BYTEVECTOR has been reached.  If
>>     START is not supplied, reads beginning at position 0.  Returns the
>>     number of bytes read.  If no bytes are available, an end-of-file
>>     object is returned.
>>
>> Maybe the r7rs-wip branch has an implementation of it.
>
> Eh, never mind, this seems to imply blocking ("as many as are available
> before the end of file").

Yes, but only “until the end of BYTEVECTOR has been reached.”

Ludo’.



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-07 14:42       ` Ludovic Courtès
@ 2015-12-08 10:53         ` tomas
  0 siblings, 0 replies; 16+ messages in thread
From: tomas @ 2015-12-08 10:53 UTC (permalink / raw)
  To: guile-user

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, Dec 07, 2015 at 03:42:28PM +0100, Ludovic Courtès wrote:
> taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:
> 
> > taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes:
> >
> >> ludo@gnu.org (Ludovic Courtès) writes:
> >>
> >>> Not sure what R7 does here.
> >>
> >> R7RS section 6.13.2 Input, page 58, defines:
> >>
> >>     (read-bytevector! bytevector [port [start [end]]])
> >>
> >>     Reads the next END - START bytes, or as many as are available before
> >>     the end of file [...]

[...]

> Yes, but only “until the end of BYTEVECTOR has been reached.”

All else would lead to... "interesting results", as every C hacker knows ;-)

But still the original problem remains: how to get "up to" N octets from
a stream without blocking, even if the stream hasn't N octets available
yet (the UNIX read() semantics with O_NONBLOCK, more or less)

Symetrically, there's the same thing with write().

Now the R7RS comes very close to that, but alas, not quite.

Marko's proposal seems extremely tempting, I must say...

regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlZmtpMACgkQBcgs9XrR2kYnMgCfUj8F1pWXdEunzEyXQg3tj3a6
wOUAn2srffROBjnUiRi17RXiLHOe8NK1
=5qrh
-----END PGP SIGNATURE-----



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-06 20:38 Nonblocking get-bytevector-n bug? Marko Rauhamaa
  2015-12-07  9:50 ` Ludovic Courtès
  2015-12-07 12:44 ` Amirouche Boubekki
@ 2015-12-08 15:39 ` Mark H Weaver
  2015-12-08 17:34   ` Marko Rauhamaa
  2 siblings, 1 reply; 16+ messages in thread
From: Mark H Weaver @ 2015-12-08 15:39 UTC (permalink / raw)
  To: Marko Rauhamaa; +Cc: guile-user

Marko Rauhamaa <marko@pacujo.net> writes:

> The problem with get-bytevector-some is that there is no limit to how
> many bytes might be returned. In practice, I see that the amount is
> capped at 4096 bytes, but the documentation does not guarantee any
> limit.

While I'm reluctant to guarantee any fixed limit, I can offer this: the
returned bytevector will always be of a manageable size, assuming that
you have not "unread" a huge amount of data, and that you haven't set a
huge buffer size via 'setvbuf'.  In the current implementation, the size
is limited by the read buffer size of the port, or the size of the
"putback" buffer.

If 'get-bytevector-some' returns more data than you want, I would
suggest using 'unget-bytevector' to "unread" the unwanted suffix.  This
procedure is quite efficient.  'unget-bytevector' was introduced in
Guile 2.0.9.

Would this work for you?

     Thanks,
       Mark



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-08 15:39 ` Mark H Weaver
@ 2015-12-08 17:34   ` Marko Rauhamaa
  2015-12-08 19:28     ` Chris Vine
  0 siblings, 1 reply; 16+ messages in thread
From: Marko Rauhamaa @ 2015-12-08 17:34 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: guile-user

Mark H Weaver <mhw@netris.org>:

> While I'm reluctant to guarantee any fixed limit, I can offer this: the
> returned bytevector will always be of a manageable size,
>
> [...]
>
> Would this work for you?

It suits my immediate needs, thank you.

However, I think the whole slew of bytevector I/O could take a more
relaxed reading of the RnRS spec, which probably was simply worded
carelessly.


Marko



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-08 17:34   ` Marko Rauhamaa
@ 2015-12-08 19:28     ` Chris Vine
  2015-12-08 21:51       ` Marko Rauhamaa
  0 siblings, 1 reply; 16+ messages in thread
From: Chris Vine @ 2015-12-08 19:28 UTC (permalink / raw)
  To: guile-user

On Tue, 08 Dec 2015 19:34:39 +0200
Marko Rauhamaa <marko@pacujo.net> wrote:
> Mark H Weaver <mhw@netris.org>:
> 
> > While I'm reluctant to guarantee any fixed limit, I can offer this:
> > the returned bytevector will always be of a manageable size,
> >
> > [...]
> >
> > Would this work for you?  
> 
> It suits my immediate needs, thank you.
> 
> However, I think the whole slew of bytevector I/O could take a more
> relaxed reading of the RnRS spec, which probably was simply worded
> carelessly.

I think better non-blocking RnRS input procedures would be advantageous
for the reasons you have given, but R6RS and R7RS seem to me to be clear
on any reasonable reading: apart from get-bytevector-some they require
blocking behaviour if the request has not been met and end-of-file has
not been reached (as do other comparable things, such as fread())[1].
Otherwise, get-bytevector-some, for all its inadequacies, would not
have been necessary.

I would be very surprised if it was a result of careless wording.

Chris

[1] With the caveat that read-bytevector! will not overrun the
bytevector if no end argument is provided (the end position is
inferred).



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-08 19:28     ` Chris Vine
@ 2015-12-08 21:51       ` Marko Rauhamaa
  2015-12-08 22:02         ` Marko Rauhamaa
  0 siblings, 1 reply; 16+ messages in thread
From: Marko Rauhamaa @ 2015-12-08 21:51 UTC (permalink / raw)
  To: Chris Vine; +Cc: guile-user

Chris Vine <chris@cvine.freeserve.co.uk>:

> I think better non-blocking RnRS input procedures would be advantageous
> for the reasons you have given, but R6RS and R7RS seem to me to be clear
> on any reasonable reading: apart from get-bytevector-some they require
> blocking behaviour if the request has not been met and end-of-file has
> not been reached (as do other comparable things, such as fread())[1].
> Otherwise, get-bytevector-some, for all its inadequacies, would not
> have been necessary.

Well, none of Guile's blocking I/O routines block when the port is
nonblocking. Thus, they are in dire violation of the strict
interpretation.

> I would be very surprised if it was a result of careless wording.

It seems to me whoever wrote the spec wasn't thinking of nonblocking
ports at all. Are nonblocking ports recognized by RnRS?


Marko



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-08 21:51       ` Marko Rauhamaa
@ 2015-12-08 22:02         ` Marko Rauhamaa
  2015-12-09  0:52           ` Chris Vine
  0 siblings, 1 reply; 16+ messages in thread
From: Marko Rauhamaa @ 2015-12-08 22:02 UTC (permalink / raw)
  To: Chris Vine; +Cc: guile-user

Marko Rauhamaa <marko@pacujo.net>:

> It seems to me whoever wrote the spec wasn't thinking of nonblocking
> ports at all. Are nonblocking ports recognized by RnRS?

Took a quick glance at the R6RS spec. It would appear nonblocking ports
are *not* mentioned or thought of in it.

Thus, *any* nonblocking behavior would be an extension to R6RS, and
Guile can do whatever it wishes.


Marko



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: Nonblocking get-bytevector-n bug?
  2015-12-08 22:02         ` Marko Rauhamaa
@ 2015-12-09  0:52           ` Chris Vine
  0 siblings, 0 replies; 16+ messages in thread
From: Chris Vine @ 2015-12-09  0:52 UTC (permalink / raw)
  To: guile-user

On Wed, 09 Dec 2015 00:02:53 +0200
Marko Rauhamaa <marko@pacujo.net> wrote:
> Marko Rauhamaa <marko@pacujo.net>:
> 
> > It seems to me whoever wrote the spec wasn't thinking of nonblocking
> > ports at all. Are nonblocking ports recognized by RnRS?  
> 
> Took a quick glance at the R6RS spec. It would appear nonblocking
> ports are *not* mentioned or thought of in it.
> 
> Thus, *any* nonblocking behavior would be an extension to R6RS, and
> Guile can do whatever it wishes.

That depends on what you (and other posters) mean.  Non-blocking in the
O_NONBLOCK POSIX sense is not recognised, probably because that kind of
non-blocking does not often make sense in the case of buffered input.
Partial satisfaction of a read request, in the mode of POSIX read() on a
blocking file descriptor, is recognised to the extent that R6RS
get-bytevector-some does something similar to read() for R6RS ports.

If you are using select then it is the second of those which is
important - you know something is available, but not how much.  What is
lacking for this is something like a get-bytevector-some! procedure.

If you are using guile's POSIX wrapper of fcntl on a scheme port I am
not sure what it does.  It would be consisent with POSIX's fread() if
it threw a EAGAIN exception if nothing is available.  As you say,
because such ports are not recognised by RnRS they can do what they
want.

Chris



^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2015-12-09  0:52 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-12-06 20:38 Nonblocking get-bytevector-n bug? Marko Rauhamaa
2015-12-07  9:50 ` Ludovic Courtès
2015-12-07 10:21   ` Taylan Ulrich Bayırlı/Kammer
2015-12-07 10:58     ` Taylan Ulrich Bayırlı/Kammer
2015-12-07 14:42       ` Ludovic Courtès
2015-12-08 10:53         ` tomas
2015-12-07 10:28   ` Marko Rauhamaa
2015-12-07 11:15     ` Marko Rauhamaa
2015-12-07 12:44 ` Amirouche Boubekki
2015-12-07 12:52   ` Marko Rauhamaa
2015-12-08 15:39 ` Mark H Weaver
2015-12-08 17:34   ` Marko Rauhamaa
2015-12-08 19:28     ` Chris Vine
2015-12-08 21:51       ` Marko Rauhamaa
2015-12-08 22:02         ` Marko Rauhamaa
2015-12-09  0:52           ` Chris Vine

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).