problem with trailing comment in repl

problem with trailing comment in repl

[Jose A. Ortega Ruiz]

hi,

currently, when entering a sexp in the repl with a trailing comment,
there's no prompt in response (although the sexp is evaluated):

    [/home/jao]$ guile -q
    GNU Guile 1.9.15.16-9970c
    Copyright (C) 1995-2010 Free Software Foundation, Inc.

    Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
    This program is free software, and you are welcome to redistribute it
    under certain conditions; type `,show c' for details.

    Enter `,help' for help.
    scheme@(guile-user)> (define a 3) ;; foo
    <- cursor stays here; no prompt


is that intended? i'm hoping it is not, because it confuses geiser, who
is waiting for a new prompt to mark the end of the transaction.

cheers,
jao

Re: problem with trailing comment in repl

[Andy Wingo]

On Fri 11 Feb 2011 22:15, "Jose A. Ortega Ruiz" <jao@gnu.org> writes:

>     scheme@(guile-user)> (define a 3) ;; foo
>     <- cursor stays here; no prompt
>
> is that intended? i'm hoping it is not, because it confuses geiser, who
> is waiting for a new prompt to mark the end of the transaction.

It would be nice if there were a prompt in this case.  However, it is
difficult to do.  Because this case is very much like:

   guile> (define a 3) (
   <- no prompt, waiting for you to finish the expression

Or indeed, like:
   guile> ;; foo
   <- no prompt

After reading an expression, the REPL reader flushes any available
whitespace, then goes into a new read.  If there are characters waiting
on the current input port, as they are if there is more than one
expression on one line, no prompt is printed.  In this case we flush the
whitespace, and start reading at the ";", and keep on reading in the
next line.

Regards,

Andy
-- 
http://wingolog.org/

Re: problem with trailing comment in repl

[Jose A. Ortega Ruiz]

On Sat, Feb 12 2011, Andy Wingo wrote:

> On Fri 11 Feb 2011 22:15, "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>
>>     scheme@(guile-user)> (define a 3) ;; foo
>>     <- cursor stays here; no prompt
>>
>> is that intended? i'm hoping it is not, because it confuses geiser, who
>> is waiting for a new prompt to mark the end of the transaction.
>
> It would be nice if there were a prompt in this case.  However, it is
> difficult to do.  Because this case is very much like:
>
>    guile> (define a 3) (
>    <- no prompt, waiting for you to finish the expression
>
> Or indeed, like:
>    guile> ;; foo
>    <- no prompt
>
> After reading an expression, the REPL reader flushes any available
> whitespace, then goes into a new read.  If there are characters waiting
> on the current input port, as they are if there is more than one
> expression on one line, no prompt is printed.  In this case we flush the
> whitespace, and start reading at the ";", and keep on reading in the
> next line.

I'm not sure i understand this reasoning, because when there are two (or
more) complete sexps in a line they're accepted, and a new prompt
appears; and the same happens for a sexp with trailing whitespace, and i
was expecting a comment to be equivalent to whitespace. But, anyway, you
know better than me how the reader works and i'm sure there are good
reasons for that behaviour. (And of course it is true that the case of
_only_ whitespace behaves as the one of only a comment.)

However, i think there's a problem with metacommands at the
"non-prompt":

    scheme@(guile-user)> ;; foo
    ,error
    While compiling expression:
    Syntax error:
    standard input:2:0: unquote: expression not valid outside of quasiquote
    in form (unquote error)
    scheme@(guile-user)> 

This happens with all metacommands i've tried, not just ,error.  Compare
to:

    scheme@(guile-user)>     
    ,error
    Nothing to debug.
    scheme@(guile-user)>


Cheers,
jao
-- 
Convictions are more dangerous enemies of truth than lies.
 -Friedrich Wilhelm Nietzsche, philosopher (1844-1900)

Re: problem with trailing comment in repl

[Andy Wingo]

On Sat 12 Feb 2011 14:21, "Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> I'm not sure i understand this reasoning, because when there are two (or
> more) complete sexps in a line they're accepted, and a new prompt
> appears;

Of course.

  1. Guile: No input available, print prompt:
     guile>

  2. User: one line of input:
     (exp-1) (exp-2)

  3. Guile: read one sexp

  4. Guile: flush whitespace; input state is:
     (exp-2)

  5. Guile: Input is ready, so don't print prompt, just read

  6. Guile: Flush whitespace, no input available, so print prompt:
     guile

> i was expecting a comment to be equivalent to whitespace.

We could add some hacks in that regard, but it wouldn't work for
ecmascript...  If there is input, Guile calls `read', not some
hypothetical `read-syntax' that could return a comment, and `read'
doesn't return until it has read an entire expression.

> However, i think there's a problem with metacommands at the
> "non-prompt":
>
>     scheme@(guile-user)> ;; foo
>     ,error
>     While compiling expression:
>     Syntax error:

Meta-commands have to be the first thing on the prompt.  It's a special
case to look for the comma character; once we've seen non-whitespace --
";" in this case -- we're in the scheme reader, not the repl reader.

Regards,

Andy
-- 
http://wingolog.org/

Re: problem with trailing comment in repl

[Noah Lavine]

Hello all,

I was thinking about how this might be solved. I can see two ways of doing it:

- The nicer way: add a new read function (or a keyword argument to the
current read function) that can tell it to stop without returning
anything if it hits a newline and there's no more input ready on its
port. You'd probably have to have it return a special "no expression"
token, but you could make that with an uninterned symbol.

- The quicker way: if the language in use is scheme, change the
next-char function (repl.scm line 205) to know about comments, so that
if it saw a semicolon, it would snarf everything until the end of the
line.

Either way is language-specific, but I don't think that's really
avoidable since different languages have different comment syntaxes.
Either way could also lead to a situation where some languages (i.e.
Scheme) had the nice prompt-printing functionality and others didn't,
but again, I think that's to be expected if this has to be implemented
once per language.

What do you all think of these ideas?

Noah

On Sat, Feb 12, 2011 at 9:36 AM, Andy Wingo <wingo@pobox.com> wrote:
> On Sat 12 Feb 2011 14:21, "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>
>> I'm not sure i understand this reasoning, because when there are two (or
>> more) complete sexps in a line they're accepted, and a new prompt
>> appears;
>
> Of course.
>
>  1. Guile: No input available, print prompt:
>     guile>
>
>  2. User: one line of input:
>     (exp-1) (exp-2)
>
>  3. Guile: read one sexp
>
>  4. Guile: flush whitespace; input state is:
>     (exp-2)
>
>  5. Guile: Input is ready, so don't print prompt, just read
>
>  6. Guile: Flush whitespace, no input available, so print prompt:
>     guile
>
>> i was expecting a comment to be equivalent to whitespace.
>
> We could add some hacks in that regard, but it wouldn't work for
> ecmascript...  If there is input, Guile calls `read', not some
> hypothetical `read-syntax' that could return a comment, and `read'
> doesn't return until it has read an entire expression.
>
>> However, i think there's a problem with metacommands at the
>> "non-prompt":
>>
>>     scheme@(guile-user)> ;; foo
>>     ,error
>>     While compiling expression:
>>     Syntax error:
>
> Meta-commands have to be the first thing on the prompt.  It's a special
> case to look for the comma character; once we've seen non-whitespace --
> ";" in this case -- we're in the scheme reader, not the repl reader.
>
> Regards,
>
> Andy
> --
> http://wingolog.org/
>
>

Re: problem with trailing comment in repl

[Andy Wingo]

Hi Noah,

I think it makes sense to have a reader that actually returns comments.
That, to me, is the general solution: the REPL reader just treats a
comment as whitespace.

On Sun 13 Feb 2011 16:22, Noah Lavine <noah.b.lavine@gmail.com> writes:

> - The quicker way: if the language in use is scheme, change the
> next-char function (repl.scm line 205) to know about comments, so that
> if it saw a semicolon, it would snarf everything until the end of the
> line.

Nasty, but a good idea I think!  Care to submit a patch? :)

Andy
-- 
http://wingolog.org/

Re: problem with trailing comment in repl

[Noah Lavine]

[-- Attachment #1: Type: text/plain, Size: 957 bytes --]

The attached patch does it. I almost hate to commit it because it's
such a hack, but this is from my last Guile session:

scheme@(guile-user)> 'foo
$2 = foo
scheme@(guile-user)> 'foo ; hi there!
$3 = foo
scheme@(guile-user)> ; why, hello!
scheme@(guile-user)> ,q

Noah

On Sun, Feb 13, 2011 at 2:19 PM, Andy Wingo <wingo@pobox.com> wrote:
> Hi Noah,
>
> I think it makes sense to have a reader that actually returns comments.
> That, to me, is the general solution: the REPL reader just treats a
> comment as whitespace.
>
> On Sun 13 Feb 2011 16:22, Noah Lavine <noah.b.lavine@gmail.com> writes:
>
>> - The quicker way: if the language in use is scheme, change the
>> next-char function (repl.scm line 205) to know about comments, so that
>> if it saw a semicolon, it would snarf everything until the end of the
>> line.
>
> Nasty, but a good idea I think!  Care to submit a patch? :)
>
> Andy
> --
> http://wingolog.org/
>

[-- Attachment #2: 0001-Show-prompts-after-a-comment-in-the-REPL.patch --]
[-- Type: application/octet-stream, Size: 3785 bytes --]

From 805a1813420c480559c849bc1984f65db3880937 Mon Sep 17 00:00:00 2001
From: Noah Lavine <nlavine@haverford.edu>
Date: Sun, 13 Feb 2011 16:17:11 -0500
Subject: [PATCH] Show prompts after a comment in the REPL

 * module/system/repl/repl.scm (next-char, run-repl, meta-reader): if the
    current language is Scheme, ignore any characters from a semicolon to
    the end of the line, then print a new prompt.
---
 module/system/repl/repl.scm |   29 +++++++++++++++++++++--------
 1 files changed, 21 insertions(+), 8 deletions(-)

diff --git a/module/system/repl/repl.scm b/module/system/repl/repl.scm
index b135dbb..47e5771 100644
--- a/module/system/repl/repl.scm
+++ b/module/system/repl/repl.scm
@@ -29,6 +29,7 @@
   #:use-module (system repl common)
   #:use-module (system repl command)
   #:use-module (ice-9 control)
+  #:use-module (ice-9 rdelim)
   #:export (start-repl run-repl))
 
 \f
@@ -62,11 +63,11 @@
 
 (define meta-command-token (cons 'meta 'command))
 
-(define (meta-reader read env)
+(define (meta-reader read env semicolon-comments)
   (lambda* (#:optional (port (current-input-port)))
     (with-input-from-port port
       (lambda ()
-        (let ((ch (next-char #t)))
+        (let ((ch (next-char #t semicolon-comments)))
           (cond ((eof-object? ch)
                  ;; EOF objects are not buffered. It's quite possible
                  ;; to peek an EOF then read something else. It's
@@ -75,6 +76,7 @@
                 ((eqv? ch #\,)
                  (read-char port)
                  meta-command-token)
+                ((not ch) *unspecified*)
                 (else (read port env))))))))
         
 ;; repl-reader is a function defined in boot-9.scm, and is replaced by
@@ -82,12 +84,13 @@
 ;; to be able to re-use the existing readline machinery.
 ;;
 ;; Catches read errors, returning *unspecified* in that case.
-(define (prompting-meta-read repl)
+(define (prompting-meta-read repl semicolon-comments)
   (catch #t
     (lambda ()
       (repl-reader (lambda () (repl-prompt repl))
                    (meta-reader (language-reader (repl-language repl))
-                                (current-module))))
+                                (current-module)
+                                semicolon-comments)))
     (lambda (key . args)
       (case key
         ((quit)
@@ -146,7 +149,10 @@
        (if (null? (cdr (fluid-ref *repl-stack*)))
            (repl-welcome repl))
        (let prompt-loop ()
-         (let ((exp (prompting-meta-read repl)))
+         (let ((exp (prompting-meta-read
+                     repl
+                     (eq? (repl-language (car (fluid-ref *repl-stack*)))
+                          scheme-lang))))
            (cond
             ((eqv? exp *unspecified*))  ; read error, pass
             ((eq? exp meta-command-token)
@@ -197,16 +203,23 @@
                   (lambda (k . args)
                     (abort args))))
               #:trap-handler 'disabled)))
-           (next-char #f) ;; consume trailing whitespace
+           (next-char #f (eq? (repl-language (car (fluid-ref *repl-stack*)))
+                              scheme-lang)) ;; consume trailing whitespace
            (prompt-loop))))
      (lambda (k status)
        status)))
 
-(define (next-char wait)
+(define scheme-lang (lookup-language 'scheme))
+
+(define (next-char wait semicolon-comments)
   (if (or wait (char-ready?))
       (let ((ch (peek-char)))
 	(cond ((eof-object? ch) ch)
-	      ((char-whitespace? ch) (read-char) (next-char wait))
+              ((and (char=? ch #\;) semicolon-comments)
+               (read-line) #f) ; ignore wait after a newline
+	      ((char-whitespace? ch)
+               (read-char)
+               (next-char wait semicolon-comments))
 	      (else ch)))
       #f))
 
-- 
1.7.4

Re: problem with trailing comment in repl

[Ludovic Courtès]

Hi!

Noah Lavine <noah.b.lavine@gmail.com> writes:

> The attached patch does it. I almost hate to commit it because it's
> such a hack, but this is from my last Guile session:
>
> scheme@(guile-user)> 'foo
> $2 = foo
> scheme@(guile-user)> 'foo ; hi there!
> $3 = foo
> scheme@(guile-user)> ; why, hello!
> scheme@(guile-user)> ,q

Nice!  Can you add a test case?

Thanks,
Ludo’.

Re: problem with trailing comment in repl

[Noah Lavine]

Hello,

> Nice!  Can you add a test case?

I've thought about it, and I'm not sure how to do it well. The trouble
is that this only applies to the REPL, not scripts. So a test would
have to wrap the REPL in something and make sure its output is right.

I might be able to do that, but if I just compared the output to the
string I expected, the test would fail if the output format changed at
all, which seems excessively fragile to me. And I'm not sure how to
test it otherwise, because the REPL catches exceptions, so I can't
just run the REPL and see if it throws one.

Could someone help me?

Thanks,
Noah

Re: problem with trailing comment in repl

[Andy Wingo]

On Sun 13 Feb 2011 22:27, Noah Lavine <noah.b.lavine@gmail.com> writes:

> The attached patch does it. I almost hate to commit it because it's
> such a hack, but this is from my last Guile session:
>
> scheme@(guile-user)> 'foo
> $2 = foo
> scheme@(guile-user)> 'foo ; hi there!
> $3 = foo
> scheme@(guile-user)> ; why, hello!
> scheme@(guile-user)> ,q

It was a hack, in both the good and bad senses of the word ;-)

I ended up pushing something similar.  Let me know how it goes.

Regards,

Andy
-- 
http://wingolog.org/