From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Michael Albinus Newsgroups: gmane.emacs.devel Subject: Re: mark expensive tests Date: Mon, 04 Jan 2016 09:16:28 +0100 Message-ID: <87mvslzrmr.fsf@gmx.de> References: <8337vab7nx.fsf@gnu.org> <0d7fkmdxj1.fsf@fencepost.gnu.org> <566D0BEB.4010707@cs.ucla.edu> <52wpsif21j.fsf@fencepost.gnu.org> <6tpoy9aorv.fsf@fencepost.gnu.org> <83fuz54sfk.fsf@gnu.org> <566F5371.7010506@cs.ucla.edu> <87wprqzb0y.fsf_-_@gmx.de> <56898729.3070401@cs.ucla.edu> <568995FA.6060404@cs.ucla.edu> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1451895424 5861 80.91.229.3 (4 Jan 2016 08:17:04 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 4 Jan 2016 08:17:04 +0000 (UTC) Cc: emacs-devel@gnu.org To: Paul Eggert Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Jan 04 09:16:55 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aG0Jx-0005GJ-SD for ged-emacs-devel@m.gmane.org; Mon, 04 Jan 2016 09:16:54 +0100 Original-Received: from localhost ([::1]:44148 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aG0Jt-0000vn-U2 for ged-emacs-devel@m.gmane.org; Mon, 04 Jan 2016 03:16:49 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:58853) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aG0Jq-0000vi-Ug for emacs-devel@gnu.org; Mon, 04 Jan 2016 03:16:47 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aG0Jn-00057y-Na for emacs-devel@gnu.org; Mon, 04 Jan 2016 03:16:46 -0500 Original-Received: from mout.gmx.net ([212.227.15.19]:57515) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aG0Jn-00057p-Ds for emacs-devel@gnu.org; Mon, 04 Jan 2016 03:16:43 -0500 Original-Received: from detlef.gmx.de ([87.146.62.57]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0Lbdl5-1ZnE0q38Zf-00lAkO; Mon, 04 Jan 2016 09:16:31 +0100 In-Reply-To: (John Wiegley's message of "Sun, 03 Jan 2016 13:45:58 -0800") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1.50 (gnu/linux) X-Provags-ID: V03:K0:xX4Bf95sLamVutII6g7fQsKZQshg3xluZHVRwxDwj5lbFJqgnx6 6LydK0htpcoxtZ52dNC2xRpMd9lyAgU8p8vWjzFiNiLiXsoiCFHNyyxn9wIHJuZVwMfMK6P X57uPwadG7jMdr8a8K+99IbC7ATFMcVXWqkNb/62DTI5I1yENJt8gSIGoBCrFzmsAdWdUd4 KQViTcP3hir0z69qmGL8w== X-UI-Out-Filterresults: notjunk:1;V01:K0:SmEbKSBcqIk=:j8WjHfs1YdkTLRifg2BFSV 7H4qpCJK1AdV7hogzUPAPG8qWLdSpXBJGux3GQtETbYM4eeuhWHWH4IFASUwY+XljF56bvH/q UqPdiBKfLRirzIyTbtSANtgS5mc25R30YosFdv26O5IWuc52ddAKKaEgyomAqImUKqGRyGZvJ OEgDrqu0mU/nzFzrqQGeFvxAac9jeGoRoXY+gCo0lWBxXYkNoDklog/4iYSlJ9i/NtsJuyno1 dcgx4sV1V7Ucgk/5wYhOP/7x5CGd8z616pgKjM70FWZgkSF5DDInaBndGaM3aW8hhK4qBGDA3 pYE+lAoTxKLmJS1GlJFOSVt+ljvaS/l+Mx8i6aoAkSHT5yym5uElpGNUARfO7JrWnPEOYE5k1 SbcGqyFUAhBtY7osQmvtT6GwuI/sBWtZFW/5SR7JzcnnYe3JYY1v0yQW33DxeGjqRB/oFd4+1 ClyXQdoe9o0YZeJ7EJhvxWGKe0fglEK3aVzxkLiQIiYB4CNiPAPIbw3sIjniu42mx+PT+OemI FMJY+0QgcIXq9B1OFChVF1S5K409TofBr5Ff7TPSsxvJPG7ZuqW0plmKwk/deSSeSxE08jp5m rxp97+LT/9rG+SAWfepZccEDgxUijJvu904xlx38QRLM444UvZeZ8r8vhjIx5ZsCS/ssTpjbF OHIH42b2W0aw0rhUofh4JZCjKG0hizPySlUXGFps/D81rbktccywsCXcoSX/mhmVJ9ml54QTa UHzbSikXZRuJxB+PXATQO6gAr1V2KA+JTXfA5H5cq75poW4/9enTQpYKMVvFJ0M8n/Rk07a+ X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 212.227.15.19 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:197576 Archived-At: John Wiegley writes: >> make check >> make check-expensive >> make check-very-expensive > > Prior art from an existing GNU project is even better. Now, who wants to make > it so? I could start with it, in the emacs-25 branch. Those targets do cover #2, #3 and #4 tests. Which make target shall we use for #1? "make check-smoke"? Also, we need to decide how to distinguish such test categories. I don't believe we shall do it on test file basis, except maybe for the smoke tests. For the other tests, we could request something like this in the test functions: #3 tests: (skip-unless (or (getenv "RUN_EXPENSIVE_TESTS") (getenv "RUN_VERY_EXPENSIVE_TESTS"))) #4 tests: (skip-unless (getenv "RUN_VERY_EXPENSIVE_TESTS")) Alternatively, we could extend ert.el by a mechanism, marking tests optionally as "smoke", "normal", "expensive", "very-expensive". "normal" would be the default category. This shall be in accordance to the GNU Coding Standards, as Richard proposes. Best regards, Michael.