Guile 2.0.9, reader: Cannot 'read' an '*unspecified*' value

[Artyom Poptsov <poptsov.artyom@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 3170 bytes --]

Hello Guilers,

it seems that currently there's no way to 'read' back an '*unspecified*'
value, but in some cases such a feature might be handy.  Here's the
description of the problem; a patch is attached as well.

To be more specific, this expression fails in GNU Guile 2.0.9 and
2.1.0.455-73f61-dirty (which I compiled from the master branch):

--8<---------------cut here---------------start------------->8---
(read (open-input-string (object->string *unspecified*)))
-| ERROR: In procedure read:
-| ERROR: In procedure scm_lreadr: #<unknown port>:1:3: Unknown # object: #\<
-|
-| Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
--8<---------------cut here---------------end--------------->8---

I faced with this problem when tried to 'read' back a vector returned by
'make-vector' with the default filling (that are '*unspecified*'
values).

Looking through the Guile sources I found that one could fix a problem
with reading of a (custom) object using 'read-hash-extend' procedure.
But alas -- the problem, again, that I cannot return an unspecified
value from my Guile reader callback because 'scm_read_sharp' from
'libguile/read.c' considers such a value as an indication that the
procedure is unable to read an object.

With that said, I think the fix could be pretty simple -- we could
return a multiple values object from the Guile reader callback in the
case when we need to 'read' an unspecified value where the 2nd value
could indicate whether the returned unspecified value is *the* value or
an indication that we could not read the value.  As far as I understand,
this solution is backward compatible so current callbacks will work as
usual.

As an example:

--8<---------------cut here---------------start------------->8---
(read-hash-extend #\<
                  (lambda (c port)
                    (let ((str     "")
                          (pending 0)
                          (c       (read-char port)))
                      (while (not (and (char=? c #\>) (= pending 0)))
                        (and (char=? c #\<)
                             (set! pending (1+ pending)))
                        (and (char=? c #\>)
                             (set! pending (1- pending)))
                        (set! str (string-append str (string c)))
                        (set! c (read-char port)))
                      (if (string=? str "unspecified")
                          (values *unspecified* #t)
                          *unspecified*))))

(define (read-string str)
  (read (open-input-string str)))

(write (read-string (object->string (make-vector 2))))
;; => #(#<unspecified> #<unspecified>)

(write (read-string (object->string *unspecified*)))
;; => #<unspecified>
--8<---------------cut here---------------end--------------->8---

As I said, the patch is attached.  I'd love to hear any comments on the
patch, especially given that this patch is my first attempt to make a
contribution to GNU Guile.

Thanks,

- Artyom

-- 
Artyom V. Poptsov <poptsov.artyom@gmail.com>;  GPG Key: 0898A02F
Home page: http://poptsov-artyom.narod.ru/

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: [PATCH] Provide means to 'read' an unspecified object --]
[-- Type: text/x-diff, Size: 5648 bytes --]

From c593bdd8b7c62ea857fb196ceeb003bb87caf9fa Mon Sep 17 00:00:00 2001
From: "Artyom V. Poptsov" <poptsov.artyom@gmail.com>
Date: Tue, 6 Oct 2015 10:20:57 +0300
Subject: [PATCH] Provide means to 'read' an #<unspecified> object

Currently there's no way to 'read' back an unspecified value, but in
some cases such a feature might be handy (eg. in case of reading a
vector filled with unspecified objects.)  This patch changes the way the
reader treats the returned value of a reader extension callback set by
'read-hash-extend': now when the callback returns a multiple values
object and the second value is '#t', the 1st value returned as is, even
if it is an unspecified object.

If a callback returns an unspecified object alone, it is considered as
an indication that the callback could not read an object -- in other
words, the solution should be backward compatible with existing reader
extenstions.

* libguile/read.c (scm_read_sharp): Check whether the value returned by
  'scm_read_sharp_extension' is a multiple values object and handle it
  appropriately if so.
  (read_inner_expression): Handle multiple values object returned by
  'scm_read_sharp'.
* test-suite/tests/reader.test: Add a test for 'read-hash-extend' to
  ensure that the callback can return an unspecified object.  Add a test
  for the same procedure that ensures that the callback can indicate an
  unknown object by returning an unspecified value alone.
* doc/ref/api-evaluation.texi: Update description of 'read-hash-extend'.
---
 doc/ref/api-evaluation.texi  |  8 ++++++++
 libguile/read.c              | 19 +++++++++++++------
 test-suite/tests/reader.test | 11 +++++++++++
 3 files changed, 32 insertions(+), 6 deletions(-)

diff --git a/doc/ref/api-evaluation.texi b/doc/ref/api-evaluation.texi
index 296f1da..8278ab9 100644
--- a/doc/ref/api-evaluation.texi
+++ b/doc/ref/api-evaluation.texi
@@ -298,6 +298,14 @@ starting with the character sequence @code{#} and @var{chr}.
 @var{proc} will be called with two arguments:  the character
 @var{chr} and the port to read further data from. The object
 returned will be the return value of @code{read}. 
+
+Returning of an unspecified value (value of the @code{*unspecified*}
+constant) alone from @var{proc} means that the procedure cannot parse
+the character sequence.  Despite of this, in some cases you may want to
+read and return an unspecified value as the correct value.  This can be
+achieved by returning of two values: an unspecified value as the first
+value and @code{#t} as the second, see (@pxref{Multiple Values}).
+
 Passing @code{#f} for @var{proc} will remove a previous setting. 
 
 @end deffn
diff --git a/libguile/read.c b/libguile/read.c
index ecf27ff..e9bb6f0 100644
--- a/libguile/read.c
+++ b/libguile/read.c
@@ -1656,8 +1656,15 @@ scm_read_sharp (scm_t_wchar chr, SCM port, scm_t_read_opts *opts,
   chr = scm_getc_unlocked (port);
 
   result = scm_read_sharp_extension (chr, port, opts);
-  if (!scm_is_eq (result, SCM_UNSPECIFIED))
-    return result;
+  if (! SCM_VALUESP (result))
+    {
+      if (! scm_is_eq (result, SCM_UNSPECIFIED))
+        return result;
+    }
+  else if (scm_is_true (scm_c_value_ref (result, 1)))
+    {
+      return result;
+    }
 
   switch (chr)
     {
@@ -1713,7 +1720,7 @@ scm_read_sharp (scm_t_wchar chr, SCM port, scm_t_read_opts *opts,
       return (scm_read_nil (chr, port, opts));
     default:
       result = scm_read_sharp_extension (chr, port, opts);
-      if (scm_is_eq (result, SCM_UNSPECIFIED))
+      if ((! SCM_VALUESP (result)) && scm_is_eq (result, SCM_UNSPECIFIED))
 	{
 	  /* To remain compatible with 1.8 and earlier, the following
 	     characters have lower precedence than `read-hash-extend'
@@ -1728,7 +1735,7 @@ scm_read_sharp (scm_t_wchar chr, SCM port, scm_t_read_opts *opts,
 	    }
 	}
       else
-	return result;
+	return scm_c_value_ref (result, 0);
     }
 
   return SCM_UNSPECIFIED;
@@ -1807,11 +1814,11 @@ read_inner_expression (SCM port, scm_t_read_opts *opts)
             long line  = SCM_LINUM (port);
             int column = SCM_COL (port) - 1;
 	    SCM result = scm_read_sharp (chr, port, opts, line, column);
-	    if (scm_is_eq (result, SCM_UNSPECIFIED))
+	    if ((! SCM_VALUESP (result)) && scm_is_eq (result, SCM_UNSPECIFIED))
 	      /* We read a comment or some such.  */
 	      break;
 	    else
-	      return result;
+	      return scm_c_value_ref (result, 0);
 	  }
 	case ')':
 	  scm_i_input_error (FUNC_NAME, port, "unexpected \")\"", SCM_EOL);
diff --git a/test-suite/tests/reader.test b/test-suite/tests/reader.test
index 5eb368d..b860abe 100644
--- a/test-suite/tests/reader.test
+++ b/test-suite/tests/reader.test
@@ -169,6 +169,17 @@
   (pass-if "square brackets are parens"
     (equal? '() (read-string "[]")))
 
+  (pass-if "reader callback can read an unspecified value"
+    (with-fluids ((%read-hash-procedures (fluid-ref %read-hash-procedures)))
+      (read-hash-extend #\< (lambda (c port) (values *unspecified* #t)))
+      (eq? *unspecified* (read-string (object->string *unspecified*)))))
+
+  (pass-if-exception "reader callback can indicate an unknown object"
+      exception:unknown-sharp-object
+    (with-fluids ((%read-hash-procedures (fluid-ref %read-hash-procedures)))
+      (read-hash-extend #\< (lambda (c port) *unspecified*))
+      (eq? *unspecified* (read-string (object->string *unspecified*)))))
+
   (pass-if-exception "paren mismatch" exception:unexpected-rparen
                      (read-string "'[)"))
 
-- 
2.4.6


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]