From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: phillip.lord@russet.org.uk (Phillip Lord) Newsgroups: gmane.emacs.devel Subject: Re: [Emacs-diffs] emacs-25 f5c762c: Additional changes for "make check-expensive" Date: Thu, 14 Jan 2016 23:04:39 +0000 Message-ID: <871t9jyd88.fsf@russet.org.uk> References: <20160105084751.30334.99051@vcs.savannah.gnu.org> <87io2y66dl.fsf@russet.org.uk> <87lh7uwrg1.fsf@gmx.de> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1452812724 22590 80.91.229.3 (14 Jan 2016 23:05:24 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 14 Jan 2016 23:05:24 +0000 (UTC) Cc: emacs-devel@gnu.org To: Michael Albinus Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Jan 15 00:05:10 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aJqx3-0004RY-8i for ged-emacs-devel@m.gmane.org; Fri, 15 Jan 2016 00:05:09 +0100 Original-Received: from localhost ([::1]:44828 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aJqx2-0004ir-FK for ged-emacs-devel@m.gmane.org; Thu, 14 Jan 2016 18:05:08 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33073) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aJqwo-0004ie-SJ for emacs-devel@gnu.org; Thu, 14 Jan 2016 18:04:55 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aJqwj-00006i-MW for emacs-devel@gnu.org; Thu, 14 Jan 2016 18:04:54 -0500 Original-Received: from cheviot22.ncl.ac.uk ([128.240.234.22]:50666) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aJqwj-00006e-DJ for emacs-devel@gnu.org; Thu, 14 Jan 2016 18:04:49 -0500 Original-Received: from smtpauth-vm.ncl.ac.uk ([10.8.233.129] helo=smtpauth.ncl.ac.uk) by cheviot22.ncl.ac.uk with esmtp (Exim 4.63) (envelope-from ) id 1aJqwi-0008Ng-Ee; Thu, 14 Jan 2016 23:04:48 +0000 Original-Received: from cpc1-benw10-2-0-cust373.gate.cable.virginm.net ([77.98.219.118] helo=localhost) by smtpauth.ncl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63) (envelope-from ) id 1aJqwi-00083d-Ab; Thu, 14 Jan 2016 23:04:48 +0000 In-Reply-To: <87lh7uwrg1.fsf@gmx.de> (Michael Albinus's message of "Tue, 12 Jan 2016 20:03:42 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 128.240.234.22 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:198161 Archived-At: Michael Albinus writes: > phillip.lord@russet.org.uk (Phillip Lord) writes: > >> Michael > > Hi Philip, > >> I think that there is a problem with this commit, in that the default >> selector is only used for "make check". By default "make check-maybe" >> runs all tests (including the expensive ones). So, you have to do >> >> make check-maybe SELECTOR="(quote (not (tag :expensive-test)))" >> >> I think it makes more sense for check-maybe to skip expensive tests, >> unless told otherwise, as "make check-maybe" is a good candidate for use >> pre-commit. > > I see. Before touching the Makefile, we shall agree how all the targets > shall behave. I would say, that "check" and "check-maybe" shall skip the > expensive tests. "check-expensive", "", and ".log" shall run > all tests. Sorry for slow reply! I would say that, yes, both check and check-maybe should skip expensive tests by default. I think this actually contradicts the GNU coding standards, but possibly it's these that need updating for very slow tests. I'd also agree about , .log (on master .log is lisp/, and lisp/ also exists). However, I also think that these should also respond to setting SELECTOR on the command line. The problem with this patch: +SELECTOR_DEFAULT = (quote (not (tag :expensive-test))) +SELECTOR_EXPENSIVE = nil +SELECTOR = +check-maybe: + @${MAKE} check-doit SELECTOR="${SELECTOR_DEFAULT}" + is that "make check-maybe SELECTOR=nil" doesn't actually run the expensive tests as it should. That was the reason for my original, rather more complicated, suggestion. I've attached a complete patch below (actually tested this time!), which I think works. Would this make sense to you? Phil diff --git a/test/automated/Makefile.in b/test/automated/Makefile.in index 152e601..2534a65 100644 --- a/test/automated/Makefile.in +++ b/test/automated/Makefile.in @@ -89,7 +89,13 @@ WRITE_LOG = > $@ 2>&1 || { stat=ERROR; cat $@; }; echo $$stat: $@ ## Beware: it approximates 'no-byte-compile', so watch out for false-positives! SELECTOR_DEFAULT = (quote (not (tag :expensive-test))) SELECTOR_EXPENSIVE = nil -SELECTOR = +ifndef SELECTOR +SELECTOR_ACTUAL=$(SELECTOR_DEFAULT) +else +SELECTOR_ACTUAL=$(SELECTOR) +endif + + %.log: ${srcdir}/%.el @if grep '^;.*no-byte-compile: t' $< > /dev/null; then \ loadfile=$<; \ @@ -100,7 +106,7 @@ SELECTOR = echo Testing $$loadfile; \ stat=OK ; \ $(emacs) -l ert -l $$loadfile \ - --eval "(ert-run-tests-batch-and-exit ${SELECTOR})" ${WRITE_LOG} + --eval "(ert-run-tests-batch-and-exit ${SELECTOR_ACTUAL})" ${WRITE_LOG} ELFILES = $(sort $(wildcard ${srcdir}/*.el)) LOGFILES = $(patsubst %.el,%.log,$(notdir ${ELFILES})) @@ -123,7 +129,7 @@ $(foreach test,${TESTS},$(eval $(call test_template,${test}))) ## Rerun all default tests. check: mostlyclean - @${MAKE} check-doit SELECTOR="${SELECTOR_DEFAULT}" + @${MAKE} check-doit SELECTOR="${SELECTOR_ACTUAL}" ## Rerun all default and expensive tests. .PHONY: check-expensive @@ -133,7 +139,7 @@ check-expensive: mostlyclean ## Only re-run default tests whose .log is older than the test. .PHONY: check-maybe check-maybe: - @${MAKE} check-doit SELECTOR="${SELECTOR_DEFAULT}" + @${MAKE} check-doit SELECTOR="${SELECTOR_ACTUAL}" ## Run the tests. .PHONY: check-doit