bug#3824: 23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems

bug#3824: 23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems

[Taylor R Campbell]

(Apologies for duplicates: apparently sending mail from this machine
using Emacs is non-trivial.  Grmble.)

Consider the following line of Scheme code:

   (foo bar #;(baz (quux #;() zot) mumble) frotz)

If the point is at the beginning, hitting C-M-f causes Emacs to barf
on imbalanced parentheses.  This is because Emacs goes to excessive
effort to handle S-expression comments in Scheme Mode, which causes
more problems than it solves.

Emacs should treat `#;' as whitespace, nothing more.  The text
following `#;' must be a valid S-expression anyway, so treating it as
if it were a comment, which can contain any unstructured text except
for a comment ender, leads to trouble.  For example, because Emacs's
`parse-partial-sexp' says that the point is in a comment if it is in
an S-expression comment, paredit fails to preserve structure there.
This is an extreme example; Emacs's S-expression motion commands in
general should work inside S-expression comments, but they don't.  For
example, if `|' denotes the point in

   (foo bar #;(baz |(quux #;() zot) mumble) frotz),

then C-M-f should cause the point to turn up at

   (foo bar #;(baz (quux #;() zot)| mumble) frotz),

but instead it barfs on imbalanced parentheses.

As far as I can imagine, the only legitimate context for treating `#;'
as more than whitespace is Font Lock, but since Font Lock works with
regular expressions, it, too, will do the wrong thing.  (In the above
example, Emacs fontifies everything after the initial `#;' as a
comment, including the text `frotz)', which is outside the comment.)

If it is too hard to make Emacs treat `#;' as whitespace and nothing
more, then Emacs shouldn't treat the octothorpe specially, and simply
read the rest of the line as a line comment.  Scheme programmers can
then just break the line after the semicolon.  Emacs's current attempt
to be clever causes trouble even for this simple workaround.  For
example, if `|' denotes the point in

   (foo bar
	#;
       |(baz (quux #;
		   () zot) mumble) frotz),

then C-M-f causes the point to move to the very end of the whole text,
rather than to

   (foo bar
	#;
	(baz (quux #;
		   () zot) mumble)| frotz),

where it should go.

In GNU Emacs 23.1.50.1 (i386-apple-darwin9.7.0, GTK+ Version 2.12.9)
 of 2009-07-11 on Oberon.local
Windowing system distributor `The X.Org Foundation', version 11.0.10402000
Important settings:
  value of $LC_ALL: en_US.UTF-8
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Scheme

Minor modes in effect:
  show-paren-mode: t
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
S-SPC f o o <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> # | f o o <backspace> <backspace> 
<backspace> SPC f o o SPC | # C-b C-b C-b C-b M-: M-p 
<return> <help-echo> <help-echo> C-b C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b b <backspace> C-b 
C-b SPC ( f r o b b l e ) S-SPC C-M-u C-M-f C-b C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-M-f C-M-b C-M-f C-M-b C-M-f C-M-b 
C-M-f C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b 
[ ] C-a C-M-f C-M-b C-M-f C-M-b C-M-f C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-f SPC 
# ; C-f C-f SPC C-a C-M-f C-M-b C-M-f C-M-f C-M-f C-M-f 
C-M-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f 
C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-d 
C-d ( ) C-a C-M-f C-M-f C-M-f C-M-b C-M-f C-M-b C-M-f 
C-M-b C-M-f C-M-b C-M-f C-M-b C-M-f C-M-b C-M-f C-M-b 
C-M-f C-M-b C-M-f C-M-b C-M-f C-M-b <help-echo> C-a 
C-n C-p C-n C-p C-f C-e C-b C-b C-b C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-k C-n C-a C-p M-f M-f M-f z M-f 
<M-backspace> q u u x M-f <M-backspace> z o t M-f <M-backspace> 
m u m b l e M-f <M-backspace> f r o t z C-n C-x o C-x 
4 0 M-x r e p r o t SPC e m a <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> o r 
t SPC e m a c s SPC b u g <help-echo> <help-echo> <help-echo> 
<return>

Recent messages:
(1 1 18 nil nil nil 0 nil nil (1))
(1 1 6 nil nil nil 0 nil nil (1))
(1 1 6 nil t nil 0 nil 10 (1))
(1 1 6 nil nil nil 0 nil nil (1))
let: End of file during parsing
(1 1 6 nil t nil 0 nil 10 (1))
Undo!
(0 nil 1 nil t nil 0 nil 26 nil)
(0 nil 1 nil 1 nil 0 t 26 nil)
forward-sexp: Scan error: "Unbalanced parentheses", 1, 53 [18 times]

bug#3824: 23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems

[Stefan Monnier]

> on imbalanced parentheses.  This is because Emacs goes to excessive
> effort to handle S-expression comments in Scheme Mode, which causes
> more problems than it solves.

Thank you for appreciating my efforts.  The first part of the problem is
known (quoting from the code:)

                   ;; FIXME: this doesn't handle the case where the sexp
                   ;; itself contains a #; comment.

The second part is indeed new: by marking the closing-paren as
a "comment end marker", we prevent the paren from being counted by
forward-sexp, so it can't stop there.

> Emacs should treat `#;' as whitespace, nothing more.

Of course, that's also incorrect with its own set of problems (mostly
indentation), but admittedly, those problems are easier for the user to
understand/predict.

> As far as I can imagine, the only legitimate context for treating `#;'
> as more than whitespace is Font Lock, but since Font Lock works with
> regular expressions, it, too, will do the wrong thing.  (In the above

Font lock can work with arbitrary Elisp code (tho it mostly relies on
regexps).

Can you try the patch below?


        Stefan


=== modified file 'lisp/progmodes/scheme.el'
--- lisp/progmodes/scheme.el	2009-06-06 04:49:20 +0000
+++ lisp/progmodes/scheme.el	2009-07-14 19:50:40 +0000
@@ -99,7 +99,8 @@
     (modify-syntax-entry ?\( "()  " st)
     (modify-syntax-entry ?\) ")(  " st)
     ;; It's used for single-line comments as well as for #;(...) sexp-comments.
-    (modify-syntax-entry ?\; "< 2 " st)
+    ;; It's too difficult to make sexp-comment work right with syntax-tables.
+    (modify-syntax-entry ?\; "<   " st)
     (modify-syntax-entry ?\" "\"   " st)
     (modify-syntax-entry ?' "'   " st)
     (modify-syntax-entry ?` "'   " st)
@@ -170,8 +171,9 @@
          nil t (("+-*/.<>=!?$%_&~^:" . "w") (?#. "w 14"))
          beginning-of-defun
          (font-lock-mark-block-function . mark-defun)
-         (font-lock-syntactic-face-function
-          . scheme-font-lock-syntactic-face-function)
+         (font-lock-syntactic-keywords . scheme-font-lock-syntactic-keywords)
+         ;; (font-lock-syntactic-face-function
+         ;;  . scheme-font-lock-syntactic-face-function)
          (parse-sexp-lookup-properties . t)
          (font-lock-extra-managed-props syntax-table)))
   (set (make-local-variable 'lisp-doc-string-elt-property)
@@ -290,6 +292,13 @@
      "^(declare\\(-\\sw+\\)+\\>\\s-+\\(\\sw+\\)" 2))
   "Imenu generic expression for DSSSL mode.  See `imenu-generic-expression'.")
 
+(defconst scheme-font-lock-syntactic-keywords
+  ;; Treat sexp-comment markers as "whitespace".
+  '(("#\\(;\\)"
+     (1 (if (nth 8 (save-excursion (syntax-ppss (match-beginning 0))))
+            ;; Check parser state to avoid problem with #|comment1|#;comment2
+            nil '(6))))))
+
 (defconst scheme-font-lock-keywords-1
   (eval-when-compile
     (list
@@ -317,8 +326,21 @@
      ))
   "Subdued expressions to highlight in Scheme modes.")
 
+(defun scheme-font-lock-sexp-comment (limit)
+  (when (search-forward "#;" limit t)
+    (let ((beg (match-beginning 0)))
+      (if (nth 8 (save-excursion (syntax-ppss beg)))
+          ;; Not a sexp-comment: keep looking.
+          (scheme-font-lock-sexp-comment limit)
+        (ignore-errors
+          (forward-sexp 1)
+          (set-match-data (list beg (point)))
+          (point))))))
+
 (defconst scheme-font-lock-keywords-2
-  (append scheme-font-lock-keywords-1
+  (append
+   '((scheme-font-lock-sexp-comment (0 font-lock-comment-face)))
+   scheme-font-lock-keywords-1
    (eval-when-compile
      (list
       ;;
@@ -388,6 +410,10 @@
         (when (< pos (- end 2))
           (put-text-property pos (- end 2)
                              'syntax-table scheme-sexp-comment-syntax-table))
+        ;; FIXME: This marks the closing paren as a comment-close
+        ;; instead, which means that forward-sexp won't stop here.
+        ;; I.e. it is fundamentally flawed: we should instead use a zero-width
+        ;; "comment-close" right after the closing paren.
         (put-text-property (- end 1) end 'syntax-table '(12)))))
   ;; Choose the face to use.
   (lisp-font-lock-syntactic-face-function state))

bug#3824: 23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems

[Taylor R Campbell]

Gosh, this must have gotten buried in my inbox a long time ago, and I
completely forgot about it...  Sorry about that!

   Date: Tue, 14 Jul 2009 15:52:52 -0400
   From: Stefan Monnier <monnier@iro.umontreal.ca>

   > Emacs should treat `#;' as whitespace, nothing more.

   Of course, that's also incorrect with its own set of problems (mostly
   indentation), but admittedly, those problems are easier for the user to
   understand/predict.

What problems?  I think it would be perfectly reasonable for #; not to
affect indentation except inasmuch as it moves code following it on
the same line by two columns just like two spaces would.  For example,
I think the following is reasonable indentation:

(cond ((foo? bar) baz)
  #;
      ((quux? zot) mumble)  #;
      ((grog? swat) zoogle)
      #;((blort? gronk)
         (frobnozzle))
   #; (else (frotz)))

I wouldn't use the second two examples myself, because it takes zero
effort to make an editor treat #; as the start of a line comment
(e.g., Edwin does this) and that works perfectly well as long as you
don't put anything interesting on the line.

   Can you try the patch below?

I tried it.  Seems to work on a couple simple cases I threw at it, but
I haven't tried it in anger yet.

bug#3824: 23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems

[Glenn Morris]

For the record: please send any future correspondence on this to 3824 AT
debbugs.gnu.org. emacsbugs.donarmstrong.com has been retired (although
it still works as an alias, so no need to resend anything). Also, do not
include emacs-pretest-bug and bug-gnu-emacs as well, since all these
addresses are essentially the same, and it means we get 3 copies of
everything.

bug#3824: This problem persists

[J. Ian Johnson]

I use #; comments extensively in my Racket code, and have been bitten by emacs's weird handling of it. Taylor pointed me to this bug to follow up.
The following is a snippet from one of my projects, with a single #; comment in it.
When you copy and paste it into emacs, it will likely match the parens correctly. If you save it into a file and reopen it, it will not. If you M-> C-M-b, then it will mark them matching.
My delay in reporting this is because the problem with #; really only manifests in large (more than a screen) sexps. Once I navigate it /enough/, then things match and I can keep working. I don't have a good qualification for "enough," i.e., what navigation is necessary for the parens to be marked matching; I only know that this should be seen as incorrect/buggy behavior.
I do hope that this can be fixed for later releases of emacs23/24.
Thanks,
-Ian

(define (a/equal? d₀ d₁ store-spaces μ)
  (define/match (egal-equal? a₀ a₁)
    [((Address-Egal space a) (Address-Egal space a))
     (match (hash-ref μ a₀ 'ω)
       [1 #t]
       ['ω 'b.⊤]
       [0 (error 'a/match "Live address with count 0: ~a (Counts ~a) (Store ~a)" a₀ μ store-spaces)])]
    [(_ _) #f])

  (define (ffun-equal? f₀ f₁)
    (if abs?
        (b∧ (ffun-⊑? f₀ f₁)
            (ffun-⊑? f₁ f₀))
        (concrete-ffun-equal? f₀ f₁)))

  ;; Slow path: linearly look for a key "equal" to k with "equal" values.
  (define (slow-equal k v f)
    (for/b∨ ([(k₁ v₁) (in-dict f)])
            (b∧ (a/equal? k k₁)
                (a/equal? v v₁))))

  (define (ffun-⊑? dom f₀ f₁)
    (for/b∧ ([(k v) (in-dict f₀)])
            (match (dict-ref f₁ k -unmapped)
              [(== -unmapped eq?) (slow-equal k v f₁)]
              [v₁ ;; fast path: check the structurally equal key
               (b∨ (a/equal? v₀ v₁)
                   (slow-equal k v f₁))])))

  (define (concrete-ffun-equal? m₀ m₁)
    (and (= (dict-count m₀) (dict-count m₁))
         (for/b∧ ([(k₀ v₀) (in-dict m₀)])
                 (match (dict-ref m₁ k₀ -unmapped)
                   ;; Concrete domains w/o structural equality are actually abstract.
                   ;; Note this is different from the concrete semantics.
                   [(== -unmapped eq?) #f]
                   ;; Note we don't use b∨ with the slow path
                   [v₁ (a/equal? v₀ v₁)]))))

  (define (discrete-ffun-equal? m₀ m₁)
    (and (= (dict-count m₀) (dict-count m₁))
         (for/b∧ ([(k₀ v₀) (in-dict m₀)])
                 (match (dict-ref m₁ k₀ -unmapped)
                   [(== -unmapped eq?) #f]
                   [v₁ (b∧
                        ;; Discrete maps get structural equality on keys, but can only be 
                        ;; truly equal if the key has cardinality 1.
                        (if (∣γ∣>1 k₀ μ) 'b.⊤ #t)
                        (a/equal? v₀ v₁))]))))

  (define (equal-step d₀ d₁)
    (match* (d₀ d₁)
      [((variant v ds₀) (variant v ds₁))
       (for/b∧ ([d₀ (in-vector ds₀)]
                [d₁ (in-vector ds₁)])
               (a/equal? d₀ d₁))]

      ;; Addresses are the same if they have cardinality 1. Distinct addresses don't overlap.
      [((? Address-Egal?) (? Address-Egal?))
       (egal-equal? d₀ d₁)]

      [((? Address-Structural? a₀) (? Address-Structural? a₁))
       (if (eq? (egal-equal? a₀ a₁) #t)
           #t
           ;; INVARIANT: not possible to be -unmapped since there must be
           ;; at least one value mapped in a store's address.
           (for*/bδ ([d₀ (in-set (store-ref store-spaces a₀))]
                     [d₁ (in-set (store-ref store-spaces a₁))])
                    (a/equal? d₀ d₁)))]

      [((? dict? m₀) (? dict? m₁)) (concrete-ffun-equal? m₀ m₁)]

      ;; If at least one map has qualification, we can check the other with the expectation of the same.
      ;; We log the incident for future debugging, since it seems like we shouldn't get this far.
      [((? dict? m₀) (abstract-ffun m₁))
       (log-info (format "Qualified/unqualified dictionary equality check ~a ~a" d₀ d₁))
       (ffun-equal? m₀ m₁)]
      [((abstract-ffun m₀) (? dict? m₁))
       (log-info (format "Qualified/unqualified dictionary equality check ~a ~a" d₀ d₁))
       (ffun-equal? m₀ m₁)]
      [((abstract-ffun m₀) (abstract-ffun m₁)) (ffun-equal? m₀ m₁)]
      ;; Discrete cases
      [((discrete-ffun m₀) (? dict? m₁))
       (log-info (format "Qualified/unqualified (discrete) dictionary equality check ~a ~a" d₀ d₁))
       (discrete-ffun-equal? m₀ m₁)]
      [((? dict? m₀) (discrete-ffun m₁))
       (log-info (format "Qualified/unqualified (discrete) dictionary equality check ~a ~a" d₀ d₁))
       (discrete-ffun-equal? m₀ m₁)]
      [((discrete-ffun m₀) (discrete-ffun m₁))
       (discrete-ffun-equal? m₀ m₁)]

      ;; OPT-OP: This has no information on discrete abstractions, thus n²logn instead of sometimes nlogn
      [((? set? s₀) (? set? s₁))
       (define (⊆? s₀ s₁)
         (for/b∧ ([v (in-set s₀)])
                 (for/b∨ ([v* (in-set s₁)])
                         (a/equal? v v*))))
       (b∧ (⊆? s₀ s₁) (⊆? s₁ s₀))]

      [(atom atom) #t]

      [((external ex v₀) (external ex v₁))
       (match-define (External-Space _ card precision special-equality) ex)
       (if special-equality
           (special-equality v₀ v₁ μ #;a/equal?) 
           (match precision
             ['concrete (equal? v₀ v₁)]
             ['discrete-abstraction (b∧ (equal? v₀ v₁) (implies (eq? (card v₀ μ)) 'b.⊤))]
             ['abstract (error 'a/match "Cannot have non-discrete abstraction of external values without a custom equality relation ~a" d₀)]))]
      [(_ _) #f]))

  ;; Circular addresses are possible
  ;; OPT-OP?: Racket impl of equal? uses union-find instead of Map[_,Set[_]].
  ;;          Is that applicable here?
  (define seen (make-hasheq))
  (define (a/equal? d₀ d₁)
    (define checked-against (hash-ref! seen d₀ mutable-seteq))
    ;; already checked ⇒ assume equal
    ;; XXX: should this be #t or 'b.⊤?
    (or (set-member? checked-against d₁)
        (begin (set-add! checked-against d₁)
               (equal-step d₀ d₁))))

  (a/equal? d₀ d₁))

bug#3824: This problem persists

[Stefan Monnier]

> I use #; comments extensively in my Racket code, and have been bitten
> by emacs's weird handling of it. Taylor pointed me to this bug to
> follow up.

Does the patch below fix it for you?


        Stefan


=== modified file 'lisp/progmodes/scheme.el'
--- lisp/progmodes/scheme.el	2014-03-17 06:22:58 +0000
+++ lisp/progmodes/scheme.el	2014-04-15 20:53:34 +0000
@@ -99,7 +99,7 @@
     (modify-syntax-entry ?\( "()  " st)
     (modify-syntax-entry ?\) ")(  " st)
     ;; It's used for single-line comments as well as for #;(...) sexp-comments.
-    (modify-syntax-entry ?\; "< 2 " st)
+    (modify-syntax-entry ?\; "<"    st)
     (modify-syntax-entry ?\" "\"   " st)
     (modify-syntax-entry ?' "'   " st)
     (modify-syntax-entry ?` "'   " st)
@@ -147,19 +147,15 @@
   (setq-local lisp-indent-function 'scheme-indent-function)
   (setq mode-line-process '("" scheme-mode-line-process))
   (setq-local imenu-case-fold-search t)
-  (setq imenu-generic-expression scheme-imenu-generic-expression)
-  (setq-local imenu-syntax-alist
-	'(("+-*/.<>=?!$%_&~^:" . "w")))
+  (setq-local imenu-generic-expression scheme-imenu-generic-expression)
+  (setq-local imenu-syntax-alist '(("+-*/.<>=?!$%_&~^:" . "w")))
+  (setq-local syntax-propertize-function #'scheme-syntax-propertize)
   (setq font-lock-defaults
 	'((scheme-font-lock-keywords
 	   scheme-font-lock-keywords-1 scheme-font-lock-keywords-2)
 	  nil t (("+-*/.<>=!?$%_&~^:" . "w") (?#. "w 14"))
 	  beginning-of-defun
-	  (font-lock-mark-block-function . mark-defun)
-	  (font-lock-syntactic-face-function
-	   . scheme-font-lock-syntactic-face-function)
-	  (parse-sexp-lookup-properties . t)
-	  (font-lock-extra-managed-props syntax-table)))
+	  (font-lock-mark-block-function . mark-defun)))
   (setq-local lisp-doc-string-elt-property 'scheme-doc-string-elt))
 
 (defvar scheme-mode-line-process "")
@@ -352,28 +348,28 @@
        (forward-comment (point-max))
        (if (eq (char-after) ?\() 2 0)))
 
-(defun scheme-font-lock-syntactic-face-function (state)
-  (when (and (null (nth 3 state))
-             (eq (char-after (nth 8 state)) ?#)
-             (eq (char-after (1+ (nth 8 state))) ?\;))
+(defun scheme-syntax-propertize (beg end)
+  (goto-char beg)
+  (scheme-syntax-propertize-sexp-comment (point) end)
+  (funcall
+   (syntax-propertize-rules
+    ("\\(#\\);" (1 (prog1 "< cn"
+                     (scheme-syntax-propertize-sexp-comment (point) end)))))
+   (point) end))
+
+(defun scheme-syntax-propertize-sexp-comment (_ end)
+  (let ((state (syntax-ppss)))
+    (when (eq 2 (nth 7 state))
     ;; It's a sexp-comment.  Tell parse-partial-sexp where it ends.
-    (save-excursion
-      (let ((pos (point))
-            (end
-             (condition-case err
-                 (let ((parse-sexp-lookup-properties nil))
+      (condition-case nil
+          (progn
                    (goto-char (+ 2 (nth 8 state)))
                    ;; FIXME: this doesn't handle the case where the sexp
                    ;; itself contains a #; comment.
                    (forward-sexp 1)
-                   (point))
-               (scan-error (nth 2 err)))))
-        (when (< pos (- end 2))
-          (put-text-property pos (- end 2)
-                             'syntax-table scheme-sexp-comment-syntax-table))
-        (put-text-property (- end 1) end 'syntax-table '(12)))))
-  ;; Choose the face to use.
-  (lisp-font-lock-syntactic-face-function state))
+            (put-text-property (1- (point)) (point)
+                               'syntax-table (string-to-syntax "> cn")))
+        (scan-error (goto-char end))))))
 
 ;;;###autoload
 (define-derived-mode dsssl-mode scheme-mode "DSSSL"

bug#3824: 23.1.50;

[Alex]

Is there any update on this? I don't mind so much the C-M-f behaviour
(I agree that ideally it should be fixed), but the nested #; not being
font-locked correctly is a bit sad.

I was recently wondering why Emacs Lisp didn't have sexp comments, but I
suppose I now get why that is.

bug#3824: This problem persists

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I use #; comments extensively in my Racket code, and have been bitten
>> by emacs's weird handling of it. Taylor pointed me to this bug to
>> follow up.
>
> Does the patch below fix it for you?

Two patches were proposed in this bug report, and this patch (looking
quite similar) was applied a month later.

commit 0a5cfeeecb9e1038f9df3b34b61b797e56213a7b
Author:     Stefan Monnier <monnier@iro.umontreal.ca>
AuthorDate: Tue May 20 16:12:30 2014 -0400

    * lisp/progmodes/scheme.el (scheme-mode-syntax-table): Remove hack for
    #; comments.
    (scheme-syntax-propertize, scheme-syntax-propertize-sexp-comment):
    New functions.
    (scheme-mode-variables): Set syntax-propertize-function instead of
    font-lock-syntactic-face-function.
    (scheme-font-lock-syntactic-face-function): Delete.

However, the original test case still fails:

> Consider the following line of Scheme code:
>
>    (foo bar #;(baz (quux #;() zot) mumble) frotz)
>
> If the point is at the beginning, hitting C-M-f causes Emacs to barf
> on imbalanced parentheses. 

(Or perhaps it fails again?)  At least, in Emacs 28, putting that into
/tmp/foo.sch and hitting `C-M-f' gives me an error.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no