bug#27476: libguile/memoize.c is not thread safe, so syntax parameter expansion is not thread-safe

[ludo@gnu.org (Ludovic Courtès)]

ludo@gnu.org (Ludovic Courtès) skribis:

> With Guile 2.2.3 a similar program triggers a crash very quickly:

Even simpler:

--8<---------------cut here---------------start------------->8---
$ guile ../guile-debugging/syntax-parms.scm 
;;; note: source file /home/ludo/src/guix/../guile-debugging/syntax-parms.scm
;;;       newer than compiled /home/ludo/.cache/guile/ccache/2.2-LE-8-3.A/home/ludo/src/guile-debugging/syntax-parms.scm.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-auto-compile argument to disable.
;;; compiling /home/ludo/src/guix/../guile-debugging/syntax-parms.scm
;;; compiled /home/ludo/.cache/guile/ccache/2.2-LE-8-3.A/home/ludo/src/guile-debugging/syntax-parms.scm.go
In /home/ludo/src/guix/../guile-debugging/syntax-parms.scm:
    37:17 13 (_)
In system/base/compile.scm:
    255:6 12 (compile _ #:from _ #:to _ #:env _ #:opts _)
   183:32 11 (compile-fold (#f syntax-violation (#<tree-il (const #f)> #<tree-il (const "source expression failed to match any pattern")> #<tree-il (lexical tmp tmp-1dff1b…>)) # …)
In ice-9/boot-9.scm:
   2312:4 10 (save-module-excursion #<procedure 57dd920 at ice-9/psyntax.scm:1774:12 (a b)>)
In language/scheme/compile-tree-il.scm:
    31:15  9 (_)
In ice-9/psyntax.scm:
  1234:22  8 (expand-top-sequence (#<tree-il (primcall syntax-violation (const #f) (const "source expression failed to match any pattern") (lexical tmp tmp-1dff1b83541ce327-2…>) …)
In ice-9/boot-9.scm:
   222:29  7 (map1 ((10 (3 #((0 . 0)) 2 (1 (11 0 . 0) (7 (3 #() 2 (10 (13 15 5 (guile) list . #f) (5 . #<syntax ->) (11 0 . 0) (11 0 . 1)) () 2) (11 0 . 0)) 10 (. #) …) …) #) …))
   222:17  6 (map1 (10 (3 #((0 . 0)) 2 (1 (11 0 . 0) (7 (3 #() 2 (10 (13 15 5 (guile) list . #f) (5 . #<syntax ->) (11 0 . 0) (11 0 . 1)) () 2) (11 0 . 0)) 10 (13 # . #) …) …) …))
In ice-9/psyntax.scm:
  2054:19  5 (_ _ (#<tree-il (const (_ any any))>) (param 1 2) ())
In ice-9/boot-9.scm:
   222:17  4 (map1 (#<tree-il (const (_ any any))>))
In ice-9/psyntax.scm:
  2057:27  3 (_ _)
   289:10  2 (eval-local-transformer _ _)
In ice-9/eval.scm:
   718:15  1 (primitive-eval _)
In unknown file:
           0 (memoize-expression #<tree-il (lambda () (lambda-case (((() 1) #f #f #f () ((11 0 . 0))) (call (lambda () (lambda-case ((((#f . #(tmp-1dff1b83541ce327-248119)) (#t…>)
ERROR: In procedure memoize-expression:
In procedure vector: Wrong type argument in position 1: #(#<directory (guile-user) cd8140>)
  C-c C-c
$ cat ../guile-debugging/syntax-parms.scm 
(use-modules (ice-9 threads)
             (srfi srfi-1)
             (system base compile))

(compile #f)                                      ;load modules

(define-syntax-parameter param
  (lambda (s)
    (syntax-case s ()
      ((_ a b) #'(+ a b)))))

(define threads
  (unfold (lambda (x) (> x 100))
          (lambda (x)
            (call-with-new-thread
             (lambda ()
               (while #t
                 (compile '(begin
                             (param 1 2)
                             (syntax-parameterize ((param (lambda (s)
                                                            (syntax-case s ()
                                                              ((_ a b)
                                                               #'(- a b))))))
                               (param 1 2)))
                          #:env (current-module)
                          #:from 'scheme
                          #:to 'tree-il)))))
          1+
          0))

(for-each join-thread threads)
--8<---------------cut here---------------end--------------->8---

So the problem, AIUI, is that psyntax evaluates syntax parameters using
‘primitive-eval’ (via ‘eval-local-transformer’), but memoization in
(ice-9 eval) is not thread-safe, hence the random crashes.

Not sure what can be done.  Thoughts?

Ludo’.