Proposal for a new (ice-9 history)

Proposal for a new (ice-9 history)

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 961 bytes --]

I'd like to rewrite (ice-9 history) to conform to the full GDB value
history syntax. This is because I find that I miss being able to refer to
"the last value" etc.

Currently we have:

$<N> the N:th value from the start

The extension would add bindings for:

$$<N> the N:th value from the end

$ the last value (= $$0)

$$ the value just prior to the last value (= $$1)

Implementation:

Currently, every step in the REPL defines a $<N> in the module
(value-history) the interface of which is appended to the list of used
interfaces for the (current-module).

The new implementation would just add a new result value to a list, not
doing any definition.

The interface of (value-history) would instead have a lazy-binder which
provides a syntax transformer for every $... actually being used. The $...
identifier would expand into a list-ref into the value history.

Please evaluate this suggestion and give comments or an OK.

Best regards,
Mikael Djurfeldt

[-- Attachment #2: Type: text/html, Size: 1343 bytes --]

Re: Proposal for a new (ice-9 history)

[Mark H Weaver]

Hi Mikael,

Mikael Djurfeldt <mikael@djurfeldt.com> writes:

> I'd like to rewrite (ice-9 history) to conform to the full GDB value
> history syntax. This is because I find that I miss being able to refer
> to "the last value" etc.

Yes, I've also missed this!

> Currently we have:
>
> $<N> the N:th value from the start
>
> The extension would add bindings for:
>
> $$<N> the N:th value from the end
>
> $ the last value (= $$0)
>
> $$ the value just prior to the last value (= $$1)
>
> Implementation:
>
> Currently, every step in the REPL defines a $<N> in the module
> (value-history) the interface of which is appended to the list of used
> interfaces for the (current-module).
>
> The new implementation would just add a new result value to a list,
> not doing any definition.
>
> The interface of (value-history) would instead have a lazy-binder
> which provides a syntax transformer for every $... actually being
> used. The $... identifier would expand into a list-ref into the value
> history.
>
> Please evaluate this suggestion and give comments or an OK.

This strategy sounds good to me.  I'd also like to hear what Andy and
Ludovic think.

However, there's a complication with using '$' in this way.  '$' is
already widely used as part of the syntax for (ice-9 match), to specify
patterns that match record objects.  More precisely, it is a literal
identifier recognized by 'match' and related macros, in the same sense
that 'else' and '=>' are literal identifiers recognized by the 'cond'
macro.

R5RS section 4.3.2 (Pattern language) specifies how these literal
identifiers are to be compared with identifiers found in each macro use:

     Identifiers that appear in <literals> are interpreted as literal
     identifiers to be matched against corresponding subforms of the
     input.  A subform in the input matches a literal identifier if and
     only if it is an identifier and either both its occurrence in the
     macro expression and its occurrence in the macro definition have
     the same lexical binding, or the two identifiers are equal and both
     have no lexical binding.

The implication is that these literal identifiers such as 'else', '=>'
and '$' lose their special meaning in any environment where they are
bound, unless the same binding is visible in the corresponding macro
definition environment.  R6RS and R7RS also specify this behavior.

For example:

--8<---------------cut here---------------start------------->8---
mhw@jojen ~$ guile
GNU Guile 2.2.3
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,use (ice-9 match)
scheme@(guile-user)> ,use (srfi srfi-9)
scheme@(guile-user)> (define-record-type <foo> (make-foo a b) foo? (a foo-a) (b foo-b))
scheme@(guile-user)> (match (make-foo 1 2) (($ <foo> a b) (+ a b)))
$1 = 3
scheme@(guile-user)> (define $ 'blah)
scheme@(guile-user)> (match (make-foo 1 2) (($ <foo> a b) (+ a b)))
<unnamed port>:6:0: Throw to key `match-error' with args `("match" "no matching pattern" #<<foo> a: 1 b: 2>)'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]>
--8<---------------cut here---------------end--------------->8---

To avoid colliding with the popular 'match' syntax, how about making
'$$' the last value ($$0), and omitting the alias for '$$1'?

What do you think?

    Regards,
      Mark

Re: Proposal for a new (ice-9 history)

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 2949 bytes --]

On Tue, Oct 30, 2018 at 12:55 AM Mark H Weaver <mhw@netris.org> wrote:

> However, there's a complication with using '$' in this way.  '$' is
> already widely used as part of the syntax for (ice-9 match), to specify
> patterns that match record objects.


Yes, I actually looked at this, but thought that $ would be interpreted as
a literal inside the match expression, but was probably wrong according to
what you write below:


>   More precisely, it is a literal
> identifier recognized by 'match' and related macros, in the same sense
> that 'else' and '=>' are literal identifiers recognized by the 'cond'
> macro.
>
> R5RS section 4.3.2 (Pattern language) specifies how these literal
> identifiers are to be compared with identifiers found in each macro use:
>
>      Identifiers that appear in <literals> are interpreted as literal
>      identifiers to be matched against corresponding subforms of the
>      input.  A subform in the input matches a literal identifier if and
>      only if it is an identifier and either both its occurrence in the
>      macro expression and its occurrence in the macro definition have
>      the same lexical binding, or the two identifiers are equal and both
>      have no lexical binding.
>
> The implication is that these literal identifiers such as 'else', '=>'
> and '$' lose their special meaning in any environment where they are
> bound, unless the same binding is visible in the corresponding macro
> definition environment.  R6RS and R7RS also specify this behavior.
>
> For example:
>
> --8<---------------cut here---------------start------------->8---
> mhw@jojen ~$ guile
> GNU Guile 2.2.3
> Copyright (C) 1995-2017 Free Software Foundation, Inc.
>
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
>
> Enter `,help' for help.
> scheme@(guile-user)> ,use (ice-9 match)
> scheme@(guile-user)> ,use (srfi srfi-9)
> scheme@(guile-user)> (define-record-type <foo> (make-foo a b) foo? (a
> foo-a) (b foo-b))
> scheme@(guile-user)> (match (make-foo 1 2) (($ <foo> a b) (+ a b)))
> $1 = 3
> scheme@(guile-user)> (define $ 'blah)
> scheme@(guile-user)> (match (make-foo 1 2) (($ <foo> a b) (+ a b)))
> <unnamed port>:6:0: Throw to key `match-error' with args `("match" "no
> matching pattern" #<<foo> a: 1 b: 2>)'.
>
> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
> scheme@(guile-user) [1]>
> --8<---------------cut here---------------end--------------->8---
>

Incidentally, this does *not* throw an error in master (unless I made some
mistake in this late hour), which then is a bug!


>
> To avoid colliding with the popular 'match' syntax, how about making
> '$$' the last value ($$0), and omitting the alias for '$$1'?
>
> What do you think?
>

Not sure. This might be confusing for GDB users... Let's think about it.

[-- Attachment #2: Type: text/html, Size: 3913 bytes --]

REPL and load deifferences (was Re: Proposal for a new (ice-9 history))

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 3610 bytes --]

On Tue, Oct 30, 2018 at 1:55 AM Mikael Djurfeldt <mikael@djurfeldt.com>
wrote:

> On Tue, Oct 30, 2018 at 12:55 AM Mark H Weaver <mhw@netris.org> wrote:
>
>>   More precisely, it is a literal
>> identifier recognized by 'match' and related macros, in the same sense
>> that 'else' and '=>' are literal identifiers recognized by the 'cond'
>> macro.
>>
>> R5RS section 4.3.2 (Pattern language) specifies how these literal
>> identifiers are to be compared with identifiers found in each macro use:
>>
>>      Identifiers that appear in <literals> are interpreted as literal
>>      identifiers to be matched against corresponding subforms of the
>>      input.  A subform in the input matches a literal identifier if and
>>      only if it is an identifier and either both its occurrence in the
>>      macro expression and its occurrence in the macro definition have
>>      the same lexical binding, or the two identifiers are equal and both
>>      have no lexical binding.
>>
>> The implication is that these literal identifiers such as 'else', '=>'
>> and '$' lose their special meaning in any environment where they are
>> bound, unless the same binding is visible in the corresponding macro
>> definition environment.  R6RS and R7RS also specify this behavior.
>>
>> For example:
>>
>> --8<---------------cut here---------------start------------->8---
>> mhw@jojen ~$ guile
>> GNU Guile 2.2.3
>> Copyright (C) 1995-2017 Free Software Foundation, Inc.
>>
>> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
>> This program is free software, and you are welcome to redistribute it
>> under certain conditions; type `,show c' for details.
>>
>> Enter `,help' for help.
>> scheme@(guile-user)> ,use (ice-9 match)
>> scheme@(guile-user)> ,use (srfi srfi-9)
>> scheme@(guile-user)> (define-record-type <foo> (make-foo a b) foo? (a
>> foo-a) (b foo-b))
>> scheme@(guile-user)> (match (make-foo 1 2) (($ <foo> a b) (+ a b)))
>> $1 = 3
>> scheme@(guile-user)> (define $ 'blah)
>> scheme@(guile-user)> (match (make-foo 1 2) (($ <foo> a b) (+ a b)))
>> <unnamed port>:6:0: Throw to key `match-error' with args `("match" "no
>> matching pattern" #<<foo> a: 1 b: 2>)'.
>>
>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
>> scheme@(guile-user) [1]>
>> --8<---------------cut here---------------end--------------->8---
>>
>
> Incidentally, this does *not* throw an error in master (unless I made some
> mistake in this late hour), which then is a bug!
>

I now looked at this a bit more. It turns out that the difference is not
between stable-2.2 and master, but between REPL and load. While I can
reproduce the above also in master, if I instead load it (attached file
matchcoll.scm), I get no error!

Also, the following file (attached as "elsetest.scm"):
------------------------------
(display (cond (else #t)))
(newline)

(define else #f)

(display (cond (else #t)))
(newline)
------------------------------

gives the results #t and #<unspecified>, as expected, in the REPL, but if I
load the file, I instead get:

scheme@(guile-user)> (load "elsetest.scm")
/home/mdj/guile/elsetest.scm:7:0: Unbound variable: else

If I load it into Chez Scheme, I get:

#t
#<void>

as expected.

Maybe someone more knowledgeable than myself could sort out what out of
this is a bug?

Also, I have to rant a bit about R5RS section 4.3.2. What a mess this is!
To have the literals influenced by bindings outside goes against the spirit
of lexical binding, in my opinion, where the idea is to be able to judge
the outcome of the code from looking at it locally.

Best regards,
Mikael

[-- Attachment #2: Type: text/html, Size: 4831 bytes --]

Re: REPL and load deifferences (was Re: Proposal for a new (ice-9 history))

[Mikael Djurfeldt]

[-- Attachment #1.1: Type: text/plain, Size: 3847 bytes --]

And, the attachments...

On Tue, Oct 30, 2018 at 11:21 AM Mikael Djurfeldt <mikael@djurfeldt.com>
wrote:

> On Tue, Oct 30, 2018 at 1:55 AM Mikael Djurfeldt <mikael@djurfeldt.com>
> wrote:
>
>> On Tue, Oct 30, 2018 at 12:55 AM Mark H Weaver <mhw@netris.org> wrote:
>>
>>>   More precisely, it is a literal
>>> identifier recognized by 'match' and related macros, in the same sense
>>> that 'else' and '=>' are literal identifiers recognized by the 'cond'
>>> macro.
>>>
>>> R5RS section 4.3.2 (Pattern language) specifies how these literal
>>> identifiers are to be compared with identifiers found in each macro use:
>>>
>>>      Identifiers that appear in <literals> are interpreted as literal
>>>      identifiers to be matched against corresponding subforms of the
>>>      input.  A subform in the input matches a literal identifier if and
>>>      only if it is an identifier and either both its occurrence in the
>>>      macro expression and its occurrence in the macro definition have
>>>      the same lexical binding, or the two identifiers are equal and both
>>>      have no lexical binding.
>>>
>>> The implication is that these literal identifiers such as 'else', '=>'
>>> and '$' lose their special meaning in any environment where they are
>>> bound, unless the same binding is visible in the corresponding macro
>>> definition environment.  R6RS and R7RS also specify this behavior.
>>>
>>> For example:
>>>
>>> --8<---------------cut here---------------start------------->8---
>>> mhw@jojen ~$ guile
>>> GNU Guile 2.2.3
>>> Copyright (C) 1995-2017 Free Software Foundation, Inc.
>>>
>>> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
>>> This program is free software, and you are welcome to redistribute it
>>> under certain conditions; type `,show c' for details.
>>>
>>> Enter `,help' for help.
>>> scheme@(guile-user)> ,use (ice-9 match)
>>> scheme@(guile-user)> ,use (srfi srfi-9)
>>> scheme@(guile-user)> (define-record-type <foo> (make-foo a b) foo? (a
>>> foo-a) (b foo-b))
>>> scheme@(guile-user)> (match (make-foo 1 2) (($ <foo> a b) (+ a b)))
>>> $1 = 3
>>> scheme@(guile-user)> (define $ 'blah)
>>> scheme@(guile-user)> (match (make-foo 1 2) (($ <foo> a b) (+ a b)))
>>> <unnamed port>:6:0: Throw to key `match-error' with args `("match" "no
>>> matching pattern" #<<foo> a: 1 b: 2>)'.
>>>
>>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
>>> scheme@(guile-user) [1]>
>>> --8<---------------cut here---------------end--------------->8---
>>>
>>
>> Incidentally, this does *not* throw an error in master (unless I made
>> some mistake in this late hour), which then is a bug!
>>
>
> I now looked at this a bit more. It turns out that the difference is not
> between stable-2.2 and master, but between REPL and load. While I can
> reproduce the above also in master, if I instead load it (attached file
> matchcoll.scm), I get no error!
>
> Also, the following file (attached as "elsetest.scm"):
> ------------------------------
> (display (cond (else #t)))
> (newline)
>
> (define else #f)
>
> (display (cond (else #t)))
> (newline)
> ------------------------------
>
> gives the results #t and #<unspecified>, as expected, in the REPL, but if
> I load the file, I instead get:
>
> scheme@(guile-user)> (load "elsetest.scm")
> /home/mdj/guile/elsetest.scm:7:0: Unbound variable: else
>
> If I load it into Chez Scheme, I get:
>
> #t
> #<void>
>
> as expected.
>
> Maybe someone more knowledgeable than myself could sort out what out of
> this is a bug?
>
> Also, I have to rant a bit about R5RS section 4.3.2. What a mess this is!
> To have the literals influenced by bindings outside goes against the spirit
> of lexical binding, in my opinion, where the idea is to be able to judge
> the outcome of the code from looking at it locally.
>
> Best regards,
> Mikael
>
>

[-- Attachment #1.2: Type: text/html, Size: 5199 bytes --]

[-- Attachment #2: matchcoll.scm --]
[-- Type: text/x-scheme, Size: 235 bytes --]

(use-modules (ice-9 match))
(use-modules (srfi srfi-9))
(define-record-type <foo> (make-foo a b) foo? (a foo-a) (b foo-b))
(match (make-foo 1 2) (($ <foo> a b) (+ a b)))

(define $ 'blah)
(match (make-foo 1 2) (($ <foo> a b) (+ a b)))

[-- Attachment #3: elsetest.scm --]
[-- Type: text/x-scheme, Size: 93 bytes --]

(display (cond (else #t)))
(newline)

(define else #f)

(display (cond (else #t)))
(newline)

Re: REPL and load deifferences (was Re: Proposal for a new (ice-9 history))

[Göran Weinholt]

Mikael Djurfeldt <mikael@djurfeldt.com> writes:

> And, the attachments...
>
> On Tue, Oct 30, 2018 at 11:21 AM Mikael Djurfeldt <mikael@djurfeldt.com> wrote:
>
>  On Tue, Oct 30, 2018 at 1:55 AM Mikael Djurfeldt <mikael@djurfeldt.com> wrote:
>
>  On Tue, Oct 30, 2018 at 12:55 AM Mark H Weaver <mhw@netris.org> wrote: 
>
>> More precisely, it is a literal
>> identifier recognized by 'match' and related macros, in the same sense
>> that 'else' and '=>' are literal identifiers recognized by the 'cond'
>> macro.

>> R5RS section 4.3.2 (Pattern language) specifies how these literal
>> identifiers are to be compared with identifiers found in each macro use:

>> Identifiers that appear in <literals> are interpreted as literal
>> identifiers to be matched against corresponding subforms of the
>> input. A subform in the input matches a literal identifier if and
>> only if it is an identifier and either both its occurrence in the
>> macro expression and its occurrence in the macro definition have
>> the same lexical binding, or the two identifiers are equal and both
>> have no lexical binding.

Guile's documentation says something else: "A literal matches an input
expression if the input expression is an identifier with the same name
as the literal, and both are unbound(1)." What happened to the case
where they have the same binding?

> [...]
> Incidentally, this does *not* throw an error in master (unless I made
> some mistake in this late hour), which then is a bug!

There is a catch for R6RS users of Guile with the standard literals
'else', '=>', etc. They are not bound in Guile's standard library, so if
a library exports 'else' then the built-in 'cond' no longer recognizes
'else' for users of that library.

This is not how other R6RS implementations behave, but IIRC it is
somehow expected that Guile behaves this way. It leads to annoying and
hard to find bugs.

-- 
Göran Weinholt
Debian developer
73 de SA6CJK

Re: Proposal for a new (ice-9 history)

[Mark H Weaver]

Mikael Djurfeldt <mikael@djurfeldt.com> writes:

> The interface of (value-history) would instead have a lazy-binder
> which provides a syntax transformer for every $... actually being
> used. The $... identifier would expand into a list-ref into the value
> history.

A few more suggestions:

If I write (define (foo x) (+ $$0 x)) at the repl, then I expect 'foo'
to continue to refer to the same entry in the value history, even after
the value history is later extended.

I'm also a bit concerned about the efficiency implications of expanding
these variable references into 'list-ref' calls when the history grows
large.  If I write a loop that evaluates $$0 a million times, I'd prefer
to avoid a million 'list-ref' calls.

To address these concerns, I'd like to suggest a slightly different
approach:

* $0, $1, ... would continue to be ordinary variable bindings in
  (value-history), as they are now.

* The 'count' in 'save-value-history' would be made into a top-level
  variable in (ice-9 history).

* $$0, $$1, $$2, ... would be handled by a lazy-binder, providing a
  syntax transformer that looks at the value of 'count' at macro
  expansion time, and expands into the appropriate variable
  reference $N.

For example, if $5 is the most recent value, $$0 would expand into $5
instead of (list-ref ...).  This would eliminate my concerns over
efficiency.

What do you think?

      Mark

Re: Proposal for a new (ice-9 history)

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 2463 bytes --]

On Tue, Oct 30, 2018 at 1:26 AM Mark H Weaver <mhw@netris.org> wrote:

> Mikael Djurfeldt <mikael@djurfeldt.com> writes:
>
> > The interface of (value-history) would instead have a lazy-binder
> > which provides a syntax transformer for every $... actually being
> > used. The $... identifier would expand into a list-ref into the value
> > history.
>
> A few more suggestions:
>
> If I write (define (foo x) (+ $$0 x)) at the repl, then I expect 'foo'
> to continue to refer to the same entry in the value history, even after
> the value history is later extended.
>

Well, this could be interpreted in two ways. What I expect is that $$0
always refers to the last entry of the value history, even if it has been
extended, such that $$0 will evaluate to new values as new values are
pushed onto value history.

This is also the effect we get if $$0 expands to (list-ref value-history 0).

>
> I'm also a bit concerned about the efficiency implications of expanding
> these variable references into 'list-ref' calls when the history grows
> large.  If I write a loop that evaluates $$0 a million times, I'd prefer
> to avoid a million 'list-ref' calls.
>

Maybe this is a Microsoft-style argument, but do we really expect users to
use value history in that way? If so, I guess value-history could be stored
in a dynamically enlarged vector.


> To address these concerns, I'd like to suggest a slightly different
> approach:
>
> * $0, $1, ... would continue to be ordinary variable bindings in
>   (value-history), as they are now.
>
> * The 'count' in 'save-value-history' would be made into a top-level
>   variable in (ice-9 history).
>

(This (count) is what I had in mind for $<N>: $<N> -> (list-ref
value-history (- count <N>)) )


> * $$0, $$1, $$2, ... would be handled by a lazy-binder, providing a
>   syntax transformer that looks at the value of 'count' at macro
>   expansion time, and expands into the appropriate variable
>   reference $N.
>

> For example, if $5 is the most recent value, $$0 would expand into $5
> instead of (list-ref ...).  This would eliminate my concerns over
> efficiency.
>
> What do you think?
>

This would then have the problem that $$0 would get a more complex meaning:
It would mean "the most recent result at the time of macro expansion"
rather than "the most recent result".

If efficiency really is a concern, I would expect that vector references
would be rather efficient after compilation.

Best regards,
Mikael

[-- Attachment #2: Type: text/html, Size: 3692 bytes --]

Re: Proposal for a new (ice-9 history)

[Mark H Weaver]

Hi Mikael,

Mikael Djurfeldt <mikael@djurfeldt.com> writes:

> On Tue, Oct 30, 2018 at 1:26 AM Mark H Weaver <mhw@netris.org> wrote:
>
>  Mikael Djurfeldt <mikael@djurfeldt.com> writes:
>
>  > The interface of (value-history) would instead have a lazy-binder
>  > which provides a syntax transformer for every $... actually being
>  > used. The $... identifier would expand into a list-ref into the value
>  > history.
>
>  A few more suggestions:
>
>  If I write (define (foo x) (+ $$0 x)) at the repl, then I expect 'foo'
>  to continue to refer to the same entry in the value history, even after
>  the value history is later extended.
>
> Well, this could be interpreted in two ways. What I expect is that $$0
> always refers to the last entry of the value history, even if it has
> been extended, such that $$0 will evaluate to new values as new values
> are pushed onto value history.

I can see why it's a natural interpretation, but in practice this seems
far less useful to me.  I very often write procedures that reference
values from the value history.  In almost every case I can think of, I
want those references to continue to refer to the same value in the
future.  If $$N has your preferred semantics, then it would almost
always be a mistake to refer to $$N from within a procedure body.

What use cases do you have in mind that would benefit from your
preferred semantics for $$N?

I can think of one case: it would enable writing procedures that
magically operate on the most recent REPL result, or possibly the last
two REPL results, to avoid having to pass them in explicitly as
arguments.  To support this use case, we could export a procedure from
(ice-9 history) to fetch the Nth most recent value.

Are there other realistic use cases that you know about?

If you think that this is a sufficiently common use case to justify a
special set of abbreviations, perhaps we could have just one or two
magic variables to fetch the most recent values at run time?

What do other people think?

> This is also the effect we get if $$0 expands to (list-ref value-history 0).
>
>  I'm also a bit concerned about the efficiency implications of expanding
>  these variable references into 'list-ref' calls when the history grows
>  large.  If I write a loop that evaluates $$0 a million times, I'd prefer
>  to avoid a million 'list-ref' calls.
>
> Maybe this is a Microsoft-style argument, but do we really expect
> users to use value history in that way?

I do, FWIW.  It's not uncommon for me to compute a value at the REPL,
write a procedure that uses the value I just computed, possibly build
more procedures on top of that, and then evaluate expressions that call
the procedures repeatedly, e.g. from within loops.

That said, I don't want to overstate the concerns about efficiency.  I
acknowledge that it's not very important either way.

> If so, I guess value-history could be stored in a dynamically enlarged
> vector.

It would certainly help for efficiency, but it raises another issue,
namely that we would need to think about thread safety issues.  If a
procedure that refers to $$N is entered at the REPL and then evaluated
in another thread, the $$N could evaluate to garbage shortly after the
value-history vector is enlarged, unless all accesses to $$N are
serialized using a mutex.

There's also another issue that just came to mind: multiple concurrent
REPLs.  Each REPL should have its own history, I think.  Modern Guile
offers REPL servers, which listen for network connections and spawn a
new REPL thread for every incoming connection.  We also have cooperative
REPL servers that enable multiple REPLs within a single thread using
cooperative threading, to avoid thread safety issues.  Those require a
procedure to be called periodically, and are intended for
single-threaded programs based on event loops.  It would be good to
think about how to fix (ice-9 history) to properly support multiple
concurrent REPLs in the same process.

What do you think?

      Mark

Re: Proposal for a new (ice-9 history)

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 5255 bytes --]

On Tue, Oct 30, 2018 at 7:21 AM Mark H Weaver <mhw@netris.org> wrote:

> Hi Mikael,
>
> Mikael Djurfeldt <mikael@djurfeldt.com> writes:
>
> > On Tue, Oct 30, 2018 at 1:26 AM Mark H Weaver <mhw@netris.org> wrote:
> >
> >  Mikael Djurfeldt <mikael@djurfeldt.com> writes:
> >
> >  > The interface of (value-history) would instead have a lazy-binder
> >  > which provides a syntax transformer for every $... actually being
> >  > used. The $... identifier would expand into a list-ref into the value
> >  > history.
> >
> >  A few more suggestions:
> >
> >  If I write (define (foo x) (+ $$0 x)) at the repl, then I expect 'foo'
> >  to continue to refer to the same entry in the value history, even after
> >  the value history is later extended.
> >
> > Well, this could be interpreted in two ways. What I expect is that $$0
> > always refers to the last entry of the value history, even if it has
> > been extended, such that $$0 will evaluate to new values as new values
> > are pushed onto value history.
>
> I can see why it's a natural interpretation, but in practice this seems
> far less useful to me.  I very often write procedures that reference
> values from the value history.  In almost every case I can think of, I
> want those references to continue to refer to the same value in the
> future.  If $$N has your preferred semantics, then it would almost
> always be a mistake to refer to $$N from within a procedure body.
>
> What use cases do you have in mind that would benefit from your
> preferred semantics for $$N?
>

> I can think of one case: it would enable writing procedures that
> magically operate on the most recent REPL result, or possibly the last
> two REPL results, to avoid having to pass them in explicitly as
> arguments.  To support this use case, we could export a procedure from
> (ice-9 history) to fetch the Nth most recent value.
>
> Are there other realistic use cases that you know about?
>

I actually don't have other use-cases other than using value-history in
expressions on the command line. My only concern is about the complexity of
the semantics. I don't have very strong objections to the semantics you
suggest, though. But note, again, that your semantics depends on macro
expansion time, such that, e.g., if one would be crazy enough to put this
in a file and load it in, then it would behave entirely differently.

However, my feeling is that if one really wanted to do real programming
against value-history, then the (ice-9 history) module should export
suitable selectors with well-defined semantics.

Your argument that you find it useful that the value referred to becomes
fixed at macro expansion time is sufficient to me to accept it, but let's
hear what other people think about it.

Note that this issue is independent of whether we use bindings in a module
or a list. The list-ref version would simply be a reference relative to
"count", and your value would be fixed as per your requested semantics.


> If you think that this is a sufficiently common use case to justify a
> special set of abbreviations, perhaps we could have just one or two
> magic variables to fetch the most recent values at run time?
>

No, I think that if we implement backward references, then we should be
able to pick any value. One or two would be frustrating in many situations.

Another possible syntax would be that $-1 refers to the most recent value,
and then we could have $-2, $-3, etc. It would all be consistent if value
history started out with $0 = ... . Dunno.


> > If so, I guess value-history could be stored in a dynamically enlarged
> > vector.
>
> It would certainly help for efficiency, but it raises another issue,
> namely that we would need to think about thread safety issues.  If a
> procedure that refers to $$N is entered at the REPL and then evaluated
> in another thread, the $$N could evaluate to garbage shortly after the
> value-history vector is enlarged, unless all accesses to $$N are
> serialized using a mutex.
>
> There's also another issue that just came to mind: multiple concurrent
> REPLs.  Each REPL should have its own history, I think.  Modern Guile
> offers REPL servers, which listen for network connections and spawn a
> new REPL thread for every incoming connection.  We also have cooperative
> REPL servers that enable multiple REPLs within a single thread using
> cooperative threading, to avoid thread safety issues.  Those require a
> procedure to be called periodically, and are intended for
> single-threaded programs based on event loops.  It would be good to
> think about how to fix (ice-9 history) to properly support multiple
> concurrent REPLs in the same process.
>
> What do you think?
>

You raise an important issue.

Since every REPL runs with its own dynamic state, perhaps a par of fluids
would help?

Ive noticed Ludovic's (ice-9 vlist) which, as a functional datastructure,
would mesh well with a fluid. At the same time it at least partially
addresses your concern regarding efficiency since at least the most recent
history values would be accessed at O(1).

So, history would be stored as a vlist in a fluid and $... references would
expand to vlist-ref.

What do you think?

Is it a problem that this would drag in many modules at start-up?

[-- Attachment #2: Type: text/html, Size: 6679 bytes --]

Re: Proposal for a new (ice-9 history)

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 291 bytes --]

On Tue, Oct 30, 2018 at 2:59 PM Mikael Djurfeldt <mikael@djurfeldt.com>
wrote:

>
> Is it a problem that this would drag in many modules at start-up?
>

I checked. It turns out that all modules used by (ice-9 vlist) are loaded
anyway. (Otherwise, (language cps intmap) is an option too. :-)

[-- Attachment #2: Type: text/html, Size: 603 bytes --]

Re: Proposal for a new (ice-9 history)

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 4933 bytes --]

I've thought some more about this.


What this is about is a need to refer to previous values in value history
in a relative way rather than referring specifically to some historic
value. First a simple (and perhaps not so useful) example, where we use
Heron's method to compute the square root of 9 (where $ refers to the last
value):

scheme@(guile-user)> 1
$1 = 1
scheme@(guile-user)> (/ (+ $ (/ 9 $)) 2.0)
$2 = 5.0
scheme@(guile-user)> (/ (+ $ (/ 9 $)) 2.0)
$3 = 3.4
scheme@(guile-user)> (/ (+ $ (/ 9 $)) 2.0)
$4 = 3.023529411764706
scheme@(guile-user)> (/ (+ $ (/ 9 $)) 2.0)
$5 = 3.00009155413138
scheme@(guile-user)> (/ (+ $ (/ 9 $)) 2.0)
$6 = 3.000000001396984
scheme@(guile-user)> (/ (+ $ (/ 9 $)) 2.0)
$7 = 3.0

We also have the more common case that we are debugging a program and need
to inspect the output, e.g (where $$0 = $ and $$1 is the value before $).:

scheme@(guile-user)> (foo 1)
$1 = #<some-object>
scheme@(guile-user)> (get-bar $$0)
$2 = ...
scheme@(guile-user)> (get-baz $$1)
$3 = ...
scheme@(guile-user)> (foo 2)
$4 = #<some-object>
scheme@(guile-user)> (get-bar $$0)
$5 = ...
scheme@(guile-user)> (get-baz $$1)
$6 = ...

The point is that we can use readline's value history to pick earlier lines
with minor or no editing, i.e. we don't need to say $4 in the last two
selectors. Maybe even more importantly, using relative value history
references is conceptually easier, since we don't have to pay attention to
the index of the value in value history.

Mark also mentioned a use case where procedures are successively built up
using values from value history.


We have now discussed three different problems associated with such an
extension:

1. Naming and name collisions

We currently have a GDB-compatible naming scheme where values are named $1,
$2 etc. It is then natural to extend this to the full GDB value history
naming scheme, introducing $, $$ and $$<N>.

$ collides with the auxilliary syntactic keyword $ in (ice-9 match).
Unfortunately, the interpretation of literals in syntax-rules and
syntax-case macros are influenced by bindings in the environment where
macros are used (according to R5RS 4.3.2) such that value history $ will
disrupt $ matching in (ice-9 match).

This can be solved by using different naming. Mark suggested that $ could
be renamed to $$. I suggested a naming scheme where $-1, $-2 could be used
for relative refrences. Chicken scheme uses #[N] and this could also be
extended by letting negative numbers be relative references.

However, I'm thinking that it is a bit awkward to have to consider all
possible uses of names when selecting this naming scheme. I would very much
prefer to let a GDB user feel at home. Is there a way for names to coexist?

Well, we have the module system. Ideally, I think that value history
lookups should be done in a special top level environment which is
associated with the REPL. Top level bindings in the Scheme environment
should have precedence. Now, instead, (ice-9 history) has a hack which
patches (value-history) into every environment we visit.

Given this situation, is there still a way to use the module system to give
(ice-9 match) $ precedence? There is. Göran Weinholt has pointed out that
other Scheme implementations tend to export their auxilliary keywords. If
we export $ like this:

--8<---------------cut here---------------start------------->8---
(define-module (ice-9 match)
  #:export (match
            match-lambda
            match-lambda*
            match-let
            match-let*
            match-letrec)
  #:replace ($))

[...]

(define misplaced
  (let ((m (lambda (x)
             (format #f "Misplaced aux keyword ~A" x))))
    (lambda (x)
      (syntax-case x ()
        (_ (identifier? x) (syntax-violation #f (m (syntax->datum x)) x))
        ((_ ...) (syntax-violation #f (m (car (syntax->datum x))) x))))))

(define-syntax $ misplaced)

[...]

(include-from-path "ice-9/match.upstream.scm")
--8<---------------cut here---------------start------------->8---

then (ice-9 match) will gracefully replace $ in (value-history) and match
will work as expected. A good idea would be to define *all* auxilliary
keywords to `misplaced' above, according to what Göran has said. That is
independent of the issue of name collisions.

If $ is replaced this way, the user can still write $$0 for that value.

2. Threading and multiple REPL:s

I have suggested to store value history in a fluid and use a functional
data type for the value history. That way we avoid race conditions.

3. Efficiency

If value history is stored in a vlist (a module which is used in the core
of Guile), much of it, particularly the most recent values, will be
accessed at O(1). If we care less about efficiency and want something lean
and simple, then ordinary lists will do.


Comments?

[-- Attachment #2: Type: text/html, Size: 6056 bytes --]

Re: Proposal for a new (ice-9 history)

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 1458 bytes --]

On Fri, Nov 2, 2018 at 2:35 PM Mikael Djurfeldt <mikael@djurfeldt.com>
wrote:

> Given this situation, is there still a way to use the module system to
> give (ice-9 match) $ precedence? There is. Göran Weinholt has pointed out
> that other Scheme implementations tend to export their auxilliary keywords.
> If we export $ like this:
>
> --8<---------------cut here---------------start------------->8---
> (define-module (ice-9 match)
>   #:export (match
>                [...]
>
              match-letrec)
>   #:replace ($))
> [...]
> --8<---------------cut here---------------start------------->8---
>
> then (ice-9 match) will gracefully replace $ in (value-history) and match
> will work as expected. A good idea would be to define *all* auxilliary
> keywords to `misplaced' above, according to what Göran has said. That is
> independent of the issue of name collisions.
>

Just to avoid misunderstanding. Of course, to avoid the hard-to-find bugs
which Göran talked about, auxilliary keywords should be exported the normal
way (through export). That way name collisions will be reported. Possibly
that is what we should do here as well, also for $, such that there is a
warning when (ice-9 match) is imported.

Also, what we really wanted to say is not that (ice-9 match) $ is
"replacing". We rather wanted to say that (value-history) $ is "deferring",
but there is no way to do that in the module system right now.

[-- Attachment #2: Type: text/html, Size: 2167 bytes --]