Ert batch-mode error

Ert batch-mode error

[Andreas Röhler]

Hi all,

running occasionally into a strange error when testing with Emacs Ert 
test-suite.
Consider the test below from following source:

https://github.com/emacs-berlin/syntactic-close/blob/master/test/syntactic-close-python-tests.el

(ert-deftest syntactic-close-python-tqs-sq-test-eS7oPB ()
   (syntactic-close-test-with-python-buffer
       "'''asdf"
     (goto-char (point-max))
     (syntactic-close)
     (should
      ;; (char-equal (char-before (- (point) 2)) ?')
      (char-equal (char-before) ?')
      )))

It's intended to test for triple-quoted strings. Also the commented
line should do the test, but fails in batch-mode.

M-x ert RET syntactic-close-python-tqs-sq-test-eS7oPB RET

however succeeds even if this line is uncommented and the following
line commented instead. So the error occurs only in batch-mode.

Any explanation?

Current syntactic-close is provided for example by: git clone 
https://github.com/emacs-berlin/syntactic-close.git

Thanks,
Andreas

Re: Ert batch-mode error

[Noam Postavsky]

On Mon, 6 Apr 2020 at 05:05, Andreas Röhler
<andreas.roehler@easy-emacs.de> wrote:

>    (syntactic-close-test-with-python-buffer

> So the error occurs only in batch-mode.

This macro tries calls font-lock-fontity-region, perhaps the problem
is that font-lock mode doesn't activate in batch mode (although it's
not clear to me why your should depend on font-lock). I wrote
yas--with-font-locked-temp-buffer for some yasnippet tests to deal
with similar problems.

https://github.com/joaotavora/yasnippet/blob/0.14.0/yasnippet-tests.el#L856-L874

Re: Ert batch-mode error

[Stefan Monnier]

> This macro tries calls font-lock-fontity-region, perhaps the problem
> is that font-lock mode doesn't activate in batch mode (although it's
> not clear to me why your should depend on font-lock). I wrote
> yas--with-font-locked-temp-buffer for some yasnippet tests to deal
> with similar problems.
>
> https://github.com/joaotavora/yasnippet/blob/0.14.0/yasnippet-tests.el#L856-L874

Yuck!  This should use `font-lock-ensure` which *should* work in this
case (if it doesn't, please file a bug report; it's been introduced in
large part to solve these kinds of use cases).

But looking at yasnippet-tests.el, I'm not sure what's going on:
the only case where font-lock seems to be relevant is
`do-yas-org-native-tab-in-source-block` (and it already uses
`font-lock-ensure` when available).  So (barring compatibility with
Emacs<24 for which you still need the non-font-lock-ensure version),
would the patch below still perform the tests correctly?


        Stefan


diff --git a/packages/yasnippet/yasnippet-tests.el b/packages/yasnippet/yasnippet-tests.el
index f7ca2bb41..5d64085c6 100644
--- a/packages/yasnippet/yasnippet-tests.el
+++ b/packages/yasnippet/yasnippet-tests.el
@@ -2,7 +2,7 @@
 
 ;; Copyright (C) 2012-2015, 2017-2018  Free Software Foundation, Inc.
 
-;; Author: Joo Tvora <joaot@siscog.pt>
+;; Author: João Távora <joaot@siscog.pt>
 ;; Keywords: emulations, convenience
 
 ;; This program is free software; you can redistribute it and/or modify
@@ -853,28 +853,8 @@ mapconcat #'(lambda (arg)
     (yas-expand-snippet "\\`\\`\\`foo\n\n\\`\\`\\`")
     (should (equal (buffer-string) "```foo\n\n```"))))
 
-(defmacro yas--with-font-locked-temp-buffer (&rest body)
-  "Like `with-temp-buffer', but ensure `font-lock-mode'."
-  (declare (indent 0) (debug t))
-  (let ((temp-buffer (make-symbol "temp-buffer")))
-    ;; NOTE: buffer name must not start with a space, otherwise
-    ;; `font-lock-mode' doesn't turn on.
-    `(let ((,temp-buffer (generate-new-buffer "*yas-temp*")))
-       (with-current-buffer ,temp-buffer
-         ;; pretend we're interactive so `font-lock-mode' turns on
-         (let ((noninteractive nil)
-               ;; turn on font locking after major mode change
-               (change-major-mode-after-body-hook #'font-lock-mode))
-           (unwind-protect
-               (progn (require 'font-lock)
-                      ;; turn on font locking before major mode change
-                      (font-lock-mode +1)
-                      ,@body)
-             (and (buffer-name ,temp-buffer)
-                  (kill-buffer ,temp-buffer))))))))
-
 (ert-deftest example-for-issue-474 ()
-  (yas--with-font-locked-temp-buffer
+  (with-temp-buffer
     (c-mode)
     (yas-minor-mode 1)
     (insert "#include <foo>\n")
@@ -882,7 +862,7 @@ mapconcat #'(lambda (arg)
     (should (string= (yas--buffer-contents) "#include <foo>\nTODO: "))))
 
 (ert-deftest example-for-issue-404 ()
-  (yas--with-font-locked-temp-buffer
+  (with-temp-buffer
     (c++-mode)
     (yas-minor-mode 1)
     (insert "#include <foo>\n")
@@ -890,7 +870,7 @@ mapconcat #'(lambda (arg)
     (should (string= (yas--buffer-contents) "#include <foo>\nmain"))))
 
 (ert-deftest example-for-issue-404-c-mode ()
-  (yas--with-font-locked-temp-buffer
+  (with-temp-buffer
     (c-mode)
     (yas-minor-mode 1)
     (insert "#include <foo>\n")
@@ -1686,14 +1666,12 @@ TODO: be meaner"
           ;; `org-src-tab-acts-natively' to have effect.
           (org-src-fontify-natively t))
       (yas-reload-all)
-      ;; Org relies on font-lock to identify source blocks.
-      (yas--with-font-locked-temp-buffer
+      (with-temp-buffer
        (org-mode)
        (yas-minor-mode 1)
        (insert "#+BEGIN_SRC " mode "\nT\n#+END_SRC")
-       (if (fboundp 'font-lock-ensure)
-           (font-lock-ensure)
-         (jit-lock-fontify-now))
+       ;; Org relies on font-lock to identify source blocks.
+       (font-lock-ensure)
        (re-search-backward "^T$") (goto-char (match-end 0))
        (should (org-in-src-block-p))
        (ert-simulate-command `(,(key-binding (kbd "TAB"))))

Re: Ert batch-mode error

[Andreas Röhler]

Thanks all,

Andreas

Re: Ert batch-mode error

[Noam Postavsky]

On Mon, 6 Apr 2020 at 23:50, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> But looking at yasnippet-tests.el, I'm not sure what's going on:
> the only case where font-lock seems to be relevant is
> `do-yas-org-native-tab-in-source-block` (and it already uses
> `font-lock-ensure` when available).  So (barring compatibility with
> Emacs<24 for which you still need the non-font-lock-ensure version),
> would the patch below still perform the tests correctly?

As far as I can tell, the problem actually seems to be 24.3 specific,
so yes, assuming you meant <=24 rather than <24. In 24.3, the
following triggers an error when run interactively, but there is no
error in --batch mode, or when using with-temp-buffer (this is a
reduction from yasnippet's example-for-issue-474 test). That was the
motivation for adding yas--with-font-locked-temp-buffer (and I just
threw it in for all cc-mode related tests, just in case).

    (setq debug-on-error t)
    (with-current-buffer (get-buffer-create "foo") ;with-temp-buffer
      (c-mode)
      (insert "#include <foo>\n")
      (narrow-to-region (point) (point))
      (insert "`\"TODO: \"`")
      (delete-region (point-min) (point-max))
      (insert "TODO: "))

Re: Ert batch-mode error

[Stefan Monnier]

>> But looking at yasnippet-tests.el, I'm not sure what's going on:
>> the only case where font-lock seems to be relevant is
>> `do-yas-org-native-tab-in-source-block` (and it already uses
>> `font-lock-ensure` when available).  So (barring compatibility with
>> Emacs<24 for which you still need the non-font-lock-ensure version),
>> would the patch below still perform the tests correctly?
>
> As far as I can tell, the problem actually seems to be 24.3 specific,
> so yes, assuming you meant <=24 rather than <24. In 24.3, the
> following triggers an error when run interactively, but there is no
> error in --batch mode, or when using with-temp-buffer (this is a
> reduction from yasnippet's example-for-issue-474 test). That was the
> motivation for adding yas--with-font-locked-temp-buffer (and I just
> threw it in for all cc-mode related tests, just in case).
>
>     (setq debug-on-error t)
>     (with-current-buffer (get-buffer-create "foo") ;with-temp-buffer
>       (c-mode)
>       (insert "#include <foo>\n")
>       (narrow-to-region (point) (point))
>       (insert "`\"TODO: \"`")
>       (delete-region (point-min) (point-max))
>       (insert "TODO: "))

So, IIUC example-for-issue-474 failed in Emacs-24.3, because of a bug in
CC-mode.  Not sure why yasnippet cares, but in any case, I'd recommend
you put a comment there to clarify it.


        Stefan