bug#48318: (ice-9 match) does not allow distinguishing between () and #nil

bug#48318: (ice-9 match) does not allow distinguishing between () and #nil

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1185 bytes --]

Hi guilers,

I've found the following surprising behaviour:

(use-modules (ice-9 match))
(match (identity #nil) (() 'scheme-eol) (#nil 'elisp-eol))
--> scheme-eol, expected elisp-eol

(match '() (#nil 'elisp-eol) (() 'elisp-eol))
--> elisp-eol, expected scheme-eol

Treating () and #nil as equivalent makes sense, but should be
documented.

My suspicion, currently untested: the following code in
ice-9/match.upstream.scm ...

(define-syntax match-two
  (syntax-rules (_ ___ ..1 *** quote quasiquote ? $ = and or not set! get!)
    ((match-two v () g+s (sk ...) fk i)
     (if (null? v) (sk ... i) fk))
    [..]

should be:

(define-syntax match-two
  (syntax-rules (_ ___ ..1 *** quote quasiquote ? $ = and or not set! get!)
    ((match-two v () g+s (sk ...) fk i)
     (if (eq? v '()) (sk ... i) fk))
    ((match-two v #nil g+s
(sk ...) fk i)
     (if (eq? v #nil) (sk ... i) fk))
    [...]

And the following might need similar adjustment:

    ((match-two v (p) g+s sk fk i)
     (if (and (pair? v) (null? (cdr v)))
         (let ((w (car v)))
           (match-one w p ((car v) (set-car! v)) sk fk i))
         fk))

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#48318: (ice-9 match) does not allow distinguishing between () and #nil

[Taylan Kammer]

Hi Maxime,

I believe that match conflating () and #nil is the right thing,
even if the current implementation does it unintentionally.

Those two values should be considered "the same" in most situations
even though (eqv? #nil '()) is false.

In fact I think they should be equal? to each other.  It feels
wrong that (equal? '(foo . #nil) '(foo . ())) evaluates to false,
even though both arguments represent the list '(foo).

Please note that #nil is not ever supposed to be used intentionally.
It's there purely as an Elisp compatibility trick, and the only time
Scheme could should receive it is when receiving data generated by
Elisp code.  For instance when Elisp code generates a list, it would
be terminated by #nil.  (Which is why I think it should equal? '().)

Does that make sense to you?  I'm not sure what the Guile maintainers
would say about (equal? #nil '()).


- Taylan

bug#48318: (ice-9 match) does not allow distinguishing between () and #nil

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2186 bytes --]

Taylan Kammer schreef op do 13-05-2021 om 21:14 [+0200]:
> Hi Maxime,
> 
> I believe that match conflating () and #nil is the right thing,
> even if the current implementation does it unintentionally.
> 
> Those two values should be considered "the same" in most situations
> even though (eqv? #nil '()) is false.

Conflating #nil and () is reasonable for my use case,
though this conflation should be documented.

> In fact I think they should be equal? to each other.  It feels
> wrong that (equal? '(foo . #nil) '(foo . ())) evaluates to false,
> even though both arguments represent the list '(foo).

The guile manual has some information on this.
(6.24.2.1 Nil, under 6.24.2 Emacs Lisp).

> Please note that #nil is not ever supposed to be used intentionally.
I know, but ...
> It's there purely as an Elisp compatibility trick, and the only time
> Scheme could should receive it is when receiving data generated by
> Elisp code.  For instance when Elisp code generates a list, it would
> be terminated by #nil.  (Which is why I think it should equal? '().)

I have been porting some common lisp code to Guile Scheme. I replaced
'() with #nil, which allows me to largely ignore whether Lisp nil is used
as end-of-list or as boolean for now (I'm in the process of replacing it
with '() or #f where appropriate).

Being able to directly refer to #nil, to perform equality checks like
(eq? #f #nil) --> #f, (eq? '() #nil) --> #f, ... can be useful for writing
Scheme code that could be called from both elisp and Scheme when the
compatibility isn't transparent.  For example, suppose (ice-9 match) actually
did not conflate #nil and () (which is what I initially thought; I expected
(ice-9 match) to compare atoms with eqv?), then the
following code ...

;; Somewhat contrived (untested), but based on some real code
(define 
  (match-lambda
    ((_ . stuff) stuff)
    (() 0)))

... would need to be rewritten to something like ...

;; Somewhat contrived (untested), but based on some real code
(define 
  (match-lambda
    ((_ . stuff) stuff)
    (() 0)
    (#nil 0)))

Also, consider the 'case' syntax.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#48318: (ice-9 match) does not allow distinguishing between () and #nil

[Taylan Kammer]

On 13.05.2021 22:39, Maxime Devos wrote:
> Taylan Kammer schreef op do 13-05-2021 om 21:14 [+0200]:
>> Hi Maxime,
>>
>> I believe that match conflating () and #nil is the right thing,
>> even if the current implementation does it unintentionally.
>>
>> Those two values should be considered "the same" in most situations
>> even though (eqv? #nil '()) is false.
> 
> Conflating #nil and () is reasonable for my use case,
> though this conflation should be documented.

Agree, it should definitely be documented.

>> In fact I think they should be equal? to each other.  It feels
>> wrong that (equal? '(foo . #nil) '(foo . ())) evaluates to false,
>> even though both arguments represent the list '(foo).
> 
> The guile manual has some information on this.
> (6.24.2.1 Nil, under 6.24.2 Emacs Lisp).

Good catch.  I see it mentions that equal? is expected to be
transitive, so if (equal? #nil '()) and (equal? #nil #f) were
both true, (equal? '() #f) would have to be too, which would
be wrong for a Scheme implementation.

We could however make it equal? to just one of the two without
making equal? non-transitive.  And if we're going to do that,
I think the empty list would be the better choice, because of
the role it plays in the structure of lists.  Without any data
to verify this, I'd say that situations where #nil surprises
programmers by not being equal? to '() are likely to come up
much more often than cases where it surprises programmers by
not being equal? to #f.

The parallel between equal?-ness and external representation
also comes to mind.  I think it's not a concrete rule, but
there's the general expectation that two objects are equal?
if their external representation is the same, which is the
case for (foo . #nil) and (foo . ()), which would both be
canonically represented as (foo).

We could also go in the other direction and make Scheme's
write procedure output (foo . #nil) as (foo . #nil), but I
think that would be less desirable.

>> Please note that #nil is not ever supposed to be used intentionally.
> I know, but ...
>> It's there purely as an Elisp compatibility trick, and the only time
>> Scheme could should receive it is when receiving data generated by
>> Elisp code.  For instance when Elisp code generates a list, it would
>> be terminated by #nil.  (Which is why I think it should equal? '().)
> 
> I have been porting some common lisp code to Guile Scheme. I replaced
> '() with #nil, which allows me to largely ignore whether Lisp nil is used
> as end-of-list or as boolean for now (I'm in the process of replacing it
> with '() or #f where appropriate).

Exciting!  I guess that's one feasible extra use-case for #nil,
but as you noted it yourself, it's probably best to rewrite the
code to eliminate all the assumptions that the end-of-list object
is false as a Boolean.

> Being able to directly refer to #nil, to perform equality checks like
> (eq? #f #nil) --> #f, (eq? '() #nil) --> #f, ... can be useful for writing
> Scheme code that could be called from both elisp and Scheme when the
> compatibility isn't transparent.  For example, suppose (ice-9 match) actually
> did not conflate #nil and () (which is what I initially thought; I expected
> (ice-9 match) to compare atoms with eqv?), then the
> following code ...
> 
> ;; Somewhat contrived (untested), but based on some real code
> (define 
>   (match-lambda
>     ((_ . stuff) stuff)
>     (() 0)))
> 
> ... would need to be rewritten to something like ...
> 
> ;; Somewhat contrived (untested), but based on some real code
> (define 
>   (match-lambda
>     ((_ . stuff) stuff)
>     (() 0)
>     (#nil 0)))

Indeed.

> Also, consider the 'case' syntax.

Case is defined in terms of eqv? in the standards so I guess
it should differentiate between #nil and ().  Unlike match,
which does pattern matching on lists.

> Greetings,
> Maxime.
> 

- Taylan