Re: master 7ce721b: Migrate MH-E functional tests from SourceForge

Re: master 7ce721b: Migrate MH-E functional tests from SourceForge

[Lars Ingebrigtsen]

stepheng+savannah@gildea.com (Stephen Gildea) writes:

>     Migrate MH-E functional tests from SourceForge
>
>     * test/lisp/mh-e/mh-limit-tests.el:
>     * test/lisp/mh-e/mh-utils-tests.el:
>     * test/lisp/mh-e/mh-xface-tests.el:
>     Import the functional tests from src/mh-unit.el.

This leads to the following test failure -- but only when
native-compilation is enabled.  And the backtrace looks pretty odd: It's
failing when trying to make a trampoline for file-directory-p?

I've added Andrea to the CCs; perhaps he has some insights here.

Test mh-sub-folders condition:
    (native-compiler-error
     (lambda
       (arg852 &optional)
       (let
	   ((f ...))
	 (funcall f arg852)))
     "")
   FAILED  15/16  mh-sub-folders (0.002489 sec)
call-process mock unexpected arglist (emacs --batch -l /tmp/emacs-int-comp-subr--trampoline-66696c652d6469726563746f72792d70_file_directory_p_0-vEOZpy.el)
call-process mock unexpected arglist (emacs --batch -l /tmp/emacs-int-comp-subr--trampoline-66696c652d6469726563746f72792d70_file_directory_p_0-fiD9dz.el)
Test mh-sub-folders-actual backtrace:
  signal(native-compiler-error ((lambda (arg883 &optional) (let ((f #'
  comp--native-compile((lambda (arg883 &optional) (let ((f #'file-dire
  comp-trampoline-compile(file-directory-p)
  comp-subr-trampoline-install(file-directory-p)
  fset(file-directory-p #<subr file-directory-p>)
  (unwind-protect (progn (if mh-test-variant-logged-already nil (mh-va
  (let* ((vnew (make-hash-table :test #'equal)) (vnew (getenv "MH")) (
  (closure (t) nil (let* ((vnew (make-hash-table :test #'equal)) (vnew
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name mh-sub-folders-actual :documentation 
  ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
  ert-run-tests((not (tag :unstable)) #f(compiled-function (event-type
  ert-run-tests-batch((not (tag :unstable)))
  ert-run-tests-batch-and-exit((not (tag :unstable)))
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/mh-e/mh-utils-tests.
  command-line()
 

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: master 7ce721b: Migrate MH-E functional tests from SourceForge

[Stephen Gildea]

>   This leads to the following test failure -- but only when
>   native-compilation is enabled.  And the backtrace looks pretty odd: It's
>   failing when trying to make a trampoline for file-directory-p?

This test mocks out both call-process and file-directory-p.  Apparently
changing file-directory-p causes call-process to be called to run a
separate Emacs process to natively compile the new file-directory-p.
This test does not expect that, and the mocked-out functions aren't
prepared to handle this case.

I can change mh-utils-tests.el to tell the native compiler to ignore it
(perhaps by binding native-comp-never-optimize-functions) but I'm
surprised that I have to.  Why should fset of a locally-bound variable
cause a separate Emacs process to be run to compile it?  That doesn't
seem to be a useful optimization, on the face of it.

Interestingly, I get a different failure when I run the tests with native
compilation enabled.  But I think the fix will be the same for both.

 < Stephen

Re: master 7ce721b: Migrate MH-E functional tests from SourceForge

[Eli Zaretskii]

> From: Stephen Gildea <stepheng+emacs@gildea.com>
> Date: Thu, 23 Sep 2021 22:49:06 -0700
> Cc: emacs-devel@gnu.org
> 
> >   This leads to the following test failure -- but only when
> >   native-compilation is enabled.  And the backtrace looks pretty odd: It's
> >   failing when trying to make a trampoline for file-directory-p?
> 
> This test mocks out both call-process and file-directory-p.  Apparently
> changing file-directory-p causes call-process to be called to run a
> separate Emacs process to natively compile the new file-directory-p.
> This test does not expect that, and the mocked-out functions aren't
> prepared to handle this case.
> 
> I can change mh-utils-tests.el to tell the native compiler to ignore it
> (perhaps by binding native-comp-never-optimize-functions) but I'm
> surprised that I have to.  Why should fset of a locally-bound variable
> cause a separate Emacs process to be run to compile it?

Note that it compiles a trampoline, not the file itself.

That said, AFAIK we shouldn't automatically invoke native-compilation
in batch mode.  Lars, are by chance running the test suite in an
interactive session?  If not, we aren't supposed to natively-compile
code without an explicit request.

I hope Andrea could help us understand what is TRT in this case.

> Interestingly, I get a different failure when I run the tests with native
> compilation enabled.

"Enabled" or "disabled"?  If the former, how is that different from
the default?

Re: master 7ce721b: Migrate MH-E functional tests from SourceForge

[Stephen Gildea]

>   > Interestingly, I get a different failure when I run the tests with native
>   > compilation enabled.
>   
>   "Enabled" or "disabled"?  If the former, how is that different from
>   the default?

Before submitting, I had failed to test in an Emacs configured with
native compilation.  But now that I do that, I can induce different
errors from the one Lars reported.

But I think all the errors come from the same root, that the tests don't
expect to be interrupted by a call-out to have a trampoline compiled.

 < Stephen

Re: master 7ce721b: Migrate MH-E functional tests from SourceForge

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> That said, AFAIK we shouldn't automatically invoke native-compilation
> in batch mode.  Lars, are by chance running the test suite in an
> interactive session?  If not, we aren't supposed to natively-compile
> code without an explicit request.

I'm just doing a normal "make mh-utils-tests" -- but perhaps ert does
something to make Emacs think that it's interactive, and native-comp
picks up on that?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: master 7ce721b: Migrate MH-E functional tests from SourceForge

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sat, 25 Sep 2021 02:50:08 +0200
> Cc: emacs-devel@gnu.org, Stephen Gildea <stepheng+emacs@gildea.com>,
>  akrl@sdf.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > That said, AFAIK we shouldn't automatically invoke native-compilation
> > in batch mode.  Lars, are by chance running the test suite in an
> > interactive session?  If not, we aren't supposed to natively-compile
> > code without an explicit request.
> 
> I'm just doing a normal "make mh-utils-tests" -- but perhaps ert does
> something to make Emacs think that it's interactive, and native-comp
> picks up on that?

Then I think Andrea should look into this, AFAIU it shouldn't happen.
Maybe the trampoline case is somehow special?