bug#48114: Disarchive occasionally fails tests

[Timothy Sample <samplet@ngyro.com>]

Hey,

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Disarchive 0.2.0 occasionally fails two tests:
>
>   FAIL: tests/kinds/octal.scm - [prop] Writing is reversible
>   FAIL: tests/kinds/octal.scm - [prop] Serializing is reversible

These two tests have a bit of a problem.  They occasionally fail by
“giving up”, which is when too many test cases are discarded rather than
used.  (This happens because you might write a generator for a superset
of the values you’re interested in, and then filter out some values with
“test-when”.)  I don’t think this is happening here, though.  You would
see something like “Gave up! Passed only 0 ests [sic].”

> I added ‘pk’ calls like so:
>
> (test-assert "[prop] Writing is reversible"
>   (quickcheck
>    (property ((octal $octal))
>      (test-when (valid-octal? octal)
>        (begin
>          (equal? (pk 'oct octal) (pk 'decode (decode-octal (encode-octal octal)))))))))
>
> (test-assert "[prop] Serializing is reversible"
>   (quickcheck
>    (property ((octal $octal))
>      (test-when (valid-octal? octal)
>        (equal? (pk 'OCT octal) (pk 'DECODE (serdeser -octal- octal)))))))
>
>
> and got this output:
>
> ;;; (oct #<<unstructured-octal> value: 0 source: #<<zero-string> value: "\U0f94a4\u0912\U025627\U10e96a\u9576\u2077\u048f\U0f2f60\U0f744b" trailer: #vu8(172 156 23 48 25 29 159 226 210)>>)
>
> ;;; (decode #<<unstructured-octal> value: 0 source: #<<zero-string> value: "\U0f94a4\u0912\U025627\U10e96a\u9576\u2077\u048f\U0f2f60\U0f744b" trailer: #vu8(172 156 23 48 25 29 159 226 210)>>)
> actual-value: #f
> actual-error:
> + (out-of-range
> +   #f
> +   "Value out of range ~S to ~S: ~S"
> +   (8 9 10)
> +   (10))
> result: FAIL
>
> […]
>
> ;;; (OCT #<<unstructured-octal> value: 0 source: #<<zero-string> value: "\U0f94a4\u0912\U025627\U10e96a\u9576\u2077\u048f\U0f2f60\U0f744b" trailer: #vu8(172 156 23 48 25 29 159 226 210)>>)
>
> ;;; (DECODE #<<unstructured-octal> value: 0 source: #<<zero-string> value: "\U0f94a4\u0912\U025627\U10e96a\u9576\u2077\u048f\U0f2f60\U0f744b" trailer: #vu8(172 156 23 48 25 29 159 226 210)>>)
> actual-value: #f
> actual-error:
> + (out-of-range
> +   #f
> +   "Value out of range ~S to ~S: ~S"
> +   (8 9 10)
> +   (10))
> result: FAIL
>
> I’m not sure where the exception comes from though.

I can’t seem to reproduce this.  I’ve run the test suite many, many
times, but I also tried:

    ,use (disarchive kinds octal)
    ,use (disarchive kinds zero-string)
    ,use (disarchive serialization)
    (define the-zero-string
      (make-zero-string
       "\U0f94a4\u0912\U025627\U10e96a\u9576\u2077\u048f\U0f2f60\U0f744b"
       #vu8(172 156 23 48 25 29 159 226 210)))
    (define the-octal
      (make-unstructured-octal 0 the-zero-string))
    (equal? the-octal (decode-octal (encode-octal the-octal)))
    (equal? the-octal (serdeser -octal- the-octal))

Which works fine.  (Does it work for you?)

However, isn’t it possible that these values aren’t the culprits?  With
the “pk” calls you added, isn’t it printing the last OK value without
telling us the value causing the issue?

What if you run it with the following?

    (test-assert "[prop] Writing is reversible"
      (quickcheck
       (property ((octal $octal))
         (test-when (valid-octal? octal)
           (false-if-exception  ; <-- changed!
             (equal? octal (decode-octal (encode-octal octal))))))))

This way, Guile-QuickCheck should print the offending value and the seed
used for the tests, which could be helpful for reproducing.  (The fact
that it doesn’t handle exceptions well is a known bug!)


-- Tim