srfi-64 tests passing when they should not

srfi-64 tests passing when they should not

[Zelphir Kaltstahl]

Hello Guile users!

I noticed something strange about SRFI-64 unit tests, when using test-equal with
an expected result of false (#f):

~~~~
(import
  ;; unit tests
  (srfi srfi-64)
  ;; module to test
  (list-utils))


(test-begin "list-utils-test")

(test-group
 "list-prefixes-test"

 (test-equal "gives all list prefixes -- 00"
   '(("static") ("static" "img"))
   (list-prefixes '("static" "img" "logo.png")))

 (test-equal "gives false for one element lists -- 00"
   #f
   (list-prefixes '("static")))

 (test-assert "gives false for zero element lists -- 00"
   (equal? #f (list-prefixes '()))))

(test-end "list-utils-test")
~~~~

This code is for a still empty module (not even a module defined in that
(list-utils) file. So I expect all tests to fail, due to the error of unbound
variables.

However, the result is surprisingly 2 passing tests, despite errors happening
during the test:

~~~~
%%%% Starting test list-utils-test
Group begin: list-utils-test
Group begin: list-prefixes-test
Test begin:
  test-name: "gives all list prefixes -- 00"
  source-file: "test/test-list-utils.scm"
  source-line: 13
  source-form: (test-equal "gives all list prefixes -- 00" (quote (("static") ("static" "img"))) (list-prefixes (quote ("static" "img" "logo.png"))))
Test end:
  result-kind: fail
  actual-value: #f
  actual-error: (unbound-variable #f "Unbound variable: ~S" (list-prefixes) #f)
  expected-value: (("static") ("static" "img"))
Test begin:
  test-name: "gives false for one element lists -- 00"
  source-file: "test/test-list-utils.scm"
  source-line: 17
  source-form: (test-equal "gives false for one element lists -- 00" #f (list-prefixes (quote ("static"))))
Test end:
  result-kind: pass
  actual-value: #f
  actual-error: (unbound-variable #f "Unbound variable: ~S" (list-prefixes) #f)
  expected-value: #f
Test begin:
  test-name: "gives false for zero element lists -- 00"
  source-file: "test/test-list-utils.scm"
  source-line: 21
  source-form: (test-equal "gives false for zero element lists -- 00" #f (list-prefixes (quote ())))
Test end:
  result-kind: pass
  actual-value: #f
  actual-error: (unbound-variable #f "Unbound variable: ~S" (list-prefixes) #f)
  expected-value: #f
Group end: list-prefixes-test
Group end: list-utils-test
# of expected passes      2
# of unexpected failures  1
~~~~

The first 2 tests are surprisingly passing. This is also the reason, why I used
test-assert and manually wrote the (equal? ...) in the last test, to see,
whether it makes any difference. Indeed it does.

Why does die implementation behave like this?

Best regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: srfi-64 tests passing when they should not

[Jérémy Korwin-Zmijowski]

Hey Zelphir !

I think it is related to something I noticed few monthes ago : 
https://lists.gnu.org/archive/html/guile-user/2021-02/msg00061.html

I cross posted the message over here : 
https://srfi-email.schemers.org/srfi-64/msg/16101182/

I got no responses from SRFI-64 mailing list.

I think, when an expression throws an exception, it also returns #f.
Unfortunately, you expect #f as well in your test.

Jérémy

Re: srfi-64 tests passing when they should not

[Zelphir Kaltstahl]

Hi Jérémy!

Thanks for letting me know!

I guess I can work around the issue like I did for now.

Best wishes,
Zelphir

On 5/4/21 12:15 PM, Jérémy Korwin-Zmijowski wrote:
> Hey Zelphir !
>
> I think it is related to something I noticed few monthes ago : 
> https://lists.gnu.org/archive/html/guile-user/2021-02/msg00061.html
>
> I cross posted the message over here : 
> https://srfi-email.schemers.org/srfi-64/msg/16101182/
>
> I got no responses from SRFI-64 mailing list.
>
> I think, when an expression throws an exception, it also returns #f.
> Unfortunately, you expect #f as well in your test.
>
> Jérémy
>
-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: srfi-64 tests passing when they should not

[Taylan Kammer]

On 04.05.2021 10:31, Zelphir Kaltstahl wrote:
>
> The first 2 tests are surprisingly passing. This is also the reason, why I used
> test-assert and manually wrote the (equal? ...) in the last test, to see,
> whether it makes any difference. Indeed it does.
> 

The reference implementation of SRFI-64 (which is what Guile ships)
doesn't seem to be written very well.

I have an alternative implementation here, if you're interested:

  https://github.com/TaylanUB/scheme-srfis

I'm not sure if the newest Guile is able to run it out of the box
though.  You might have to create some .scm symlinks to the .sld files.


- Taylan

Re: srfi-64 tests passing when they should not

[Luis Felipe]

Hi Taylan,

On Wednesday, May 5, 2021 6:39 AM, Taylan Kammer <taylan.kammer@gmail.com> wrote:

> On 04.05.2021 10:31, Zelphir Kaltstahl wrote:
>
> > The first 2 tests are surprisingly passing. This is also the reason, why I used
> > test-assert and manually wrote the (equal? ...) in the last test, to see,
> > whether it makes any difference. Indeed it does.
>
> The reference implementation of SRFI-64 (which is what Guile ships)
> doesn't seem to be written very well.
>
> I have an alternative implementation here, if you're interested:
>
> https://github.com/TaylanUB/scheme-srfis
>
> I'm not sure if the newest Guile is able to run it out of the box
> though. You might have to create some .scm symlinks to the .sld files.

For what it's worth, I know about your implementation for a long time, but I've never tried to use it because I don't know where to start. Is it not possible to package these libraries so that users can simply install them as any other guile library? Say:

$ guix install r7rs-srfi-64

I see that Guile can be run with the "--r7rs" option "to better support R7RS"...

Re: srfi-64 tests passing when they should not

[Taylan Kammer]

On 05.05.2021 15:47, Luis Felipe wrote:
> Hi Taylan,
> 
> On Wednesday, May 5, 2021 6:39 AM, Taylan Kammer <taylan.kammer@gmail.com> wrote:
> 
>> On 04.05.2021 10:31, Zelphir Kaltstahl wrote:
>>
>>> The first 2 tests are surprisingly passing. This is also the reason, why I used
>>> test-assert and manually wrote the (equal? ...) in the last test, to see,
>>> whether it makes any difference. Indeed it does.
>>
>> The reference implementation of SRFI-64 (which is what Guile ships)
>> doesn't seem to be written very well.
>>
>> I have an alternative implementation here, if you're interested:
>>
>> https://github.com/TaylanUB/scheme-srfis
>>
>> I'm not sure if the newest Guile is able to run it out of the box
>> though. You might have to create some .scm symlinks to the .sld files.
> 
> For what it's worth, I know about your implementation for a long time, but I've never tried to use it because I don't know where to start. Is it not possible to package these libraries so that users can simply install them as any other guile library? Say:
> 
> $ guix install r7rs-srfi-64
> 
> I see that Guile can be run with the "--r7rs" option "to better support R7RS"...
> 

Hmm, I had hoped that with the newest Guile, simply adding the repo's
root directory to the load path would work, at least when invoked with
the --r7rs switch, but it seems that Guile still chokes on library name
parts that are integers.  That's an incompatibility with r7rs that's not
mentioned in the manual.

I guess the only way to make the modules work is to rename all the files
and change the library names to not use integer parts.

Maybe I'll make a guile-compatible standalone package for the SRFI-64
implementation, since that's the most fancy thing in that repo.

I might do it in the following days since I'm on a vacation, but... the
vacation is supposed to be a vacation. :-)  Work has been really burning
me out in the last year.

If someone else feels like trying: all you have to do is rename .sld
files to .scm, change the integer parts of the module names to symbols
(e.g. s64 instead of 64), and rename the directory '64' accordingly.


- Taylan

Re: srfi-64 tests passing when they should not

[Luis Felipe]

On Thursday, May 6, 2021 11:16 AM, Taylan Kammer <taylan.kammer@gmail.com> wrote:

> On 05.05.2021 15:47, Luis Felipe wrote:
>
> > Hi Taylan,
> > On Wednesday, May 5, 2021 6:39 AM, Taylan Kammer taylan.kammer@gmail.com wrote:
> >
> > > On 04.05.2021 10:31, Zelphir Kaltstahl wrote:
> > >
> > > > The first 2 tests are surprisingly passing. This is also the reason, why I used
> > > > test-assert and manually wrote the (equal? ...) in the last test, to see,
> > > > whether it makes any difference. Indeed it does.
> > >
> > > The reference implementation of SRFI-64 (which is what Guile ships)
> > > doesn't seem to be written very well.
> > > I have an alternative implementation here, if you're interested:
> > > https://github.com/TaylanUB/scheme-srfis
> > > I'm not sure if the newest Guile is able to run it out of the box
> > > though. You might have to create some .scm symlinks to the .sld files.
> >
> > For what it's worth, I know about your implementation for a long time, but I've never tried to use it because I don't know where to start. Is it not possible to package these libraries so that users can simply install them as any other guile library? Say:
> > $ guix install r7rs-srfi-64
> > I see that Guile can be run with the "--r7rs" option "to better support R7RS"...
>
> Hmm, I had hoped that with the newest Guile, simply adding the repo's
> root directory to the load path would work, at least when invoked with
> the --r7rs switch, but it seems that Guile still chokes on library name
> parts that are integers. That's an incompatibility with r7rs that's not
> mentioned in the manual.

Ah, good to know.


> I guess the only way to make the modules work is to rename all the files
> and change the library names to not use integer parts.
>
> Maybe I'll make a guile-compatible standalone package for the SRFI-64
> implementation, since that's the most fancy thing in that repo.
>
> I might do it in the following days since I'm on a vacation, but... the
> vacation is supposed to be a vacation. :-) Work has been really burning
> me out in the last year.

Yeah, you should not work on vacation.


> If someone else feels like trying: all you have to do is rename .sld
> files to .scm, change the integer parts of the module names to symbols
> (e.g. s64 instead of 64), and rename the directory '64' accordingly.

That helps. I might even try it myself, thanks.

(But, really, at least for me, it would be ideal to have these libraries packaged.)