From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#15991: 24.3.50; Wishlist: Make test suite user-friendlier Date: Thu, 28 Nov 2013 22:04:26 +0200 Message-ID: <83pppk8f91.fsf@gnu.org> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1385669185 15565 80.91.229.3 (28 Nov 2013 20:06:25 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 28 Nov 2013 20:06:25 +0000 (UTC) To: 15991@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Thu Nov 28 21:06:26 2013 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1Vm7qz-0005xq-Oj for geb-bug-gnu-emacs@m.gmane.org; Thu, 28 Nov 2013 21:06:25 +0100 Original-Received: from localhost ([::1]:43873 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vm7qy-0005Z8-TX for geb-bug-gnu-emacs@m.gmane.org; Thu, 28 Nov 2013 15:06:24 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:36048) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vm7qm-0005YD-2y for bug-gnu-emacs@gnu.org; Thu, 28 Nov 2013 15:06:21 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Vm7qc-0004jr-Ly for bug-gnu-emacs@gnu.org; Thu, 28 Nov 2013 15:06:12 -0500 Original-Received: from debbugs.gnu.org ([140.186.70.43]:35174) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vm7qc-0004jl-Ix for bug-gnu-emacs@gnu.org; Thu, 28 Nov 2013 15:06:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.80) (envelope-from ) id 1Vm7qc-0002jU-A3 for bug-gnu-emacs@gnu.org; Thu, 28 Nov 2013 15:06:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 28 Nov 2013 20:06:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 15991 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.138566911710438 (code B ref -1); Thu, 28 Nov 2013 20:06:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 28 Nov 2013 20:05:17 +0000 Original-Received: from localhost ([127.0.0.1]:49193 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vm7pt-0002iH-7A for submit@debbugs.gnu.org; Thu, 28 Nov 2013 15:05:17 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:34146) by debbugs.gnu.org with esmtp (Exim 4.80) (envelope-from ) id 1Vm7pq-0002i1-Ob for submit@debbugs.gnu.org; Thu, 28 Nov 2013 15:05:15 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Vm7pc-0004LO-1V for submit@debbugs.gnu.org; Thu, 28 Nov 2013 15:05:09 -0500 Original-Received: from lists.gnu.org ([2001:4830:134:3::11]:36316) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vm7pb-0004LK-VA for submit@debbugs.gnu.org; Thu, 28 Nov 2013 15:04:59 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:35843) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vm7pU-0005Dh-Jj for bug-gnu-emacs@gnu.org; Thu, 28 Nov 2013 15:04:59 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Vm7pK-0004K8-Dx for bug-gnu-emacs@gnu.org; Thu, 28 Nov 2013 15:04:52 -0500 Original-Received: from mtaout23.012.net.il ([80.179.55.175]:60264) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Vm7pK-0004K2-5Y for bug-gnu-emacs@gnu.org; Thu, 28 Nov 2013 15:04:42 -0500 Original-Received: from conversion-daemon.a-mtaout23.012.net.il by a-mtaout23.012.net.il (HyperSendmail v2007.08) id <0MWZ00600P0LJ000@a-mtaout23.012.net.il> for bug-gnu-emacs@gnu.org; Thu, 28 Nov 2013 22:04:40 +0200 (IST) Original-Received: from HOME-C4E4A596F7 ([87.69.4.28]) by a-mtaout23.012.net.il (HyperSendmail v2007.08) with ESMTPA id <0MWZ006WTP3QHC40@a-mtaout23.012.net.il> for bug-gnu-emacs@gnu.org; Thu, 28 Nov 2013 22:04:40 +0200 (IST) X-012-Sender: halo1@inter.net.il X-detected-operating-system: by eggs.gnu.org: Solaris 10 X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address (bad octet value). X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.15 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 140.186.70.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:81054 Archived-At: This is a wishlist feature request: make the test suite easier to use when some tests fail, and there's a need to investigate why and debug them. When you see any failures, the first thing you want to do is run only one test that failed at a time. But there's no way to do that, so it seems, except by copying a (quite complex) command line from the Makefile, while manually expanding every variable that's involved in that. Also, suppose one needs to run Emacs under a debugger for running a test -- there doesn't seem to be a convenient way of doing that, either. Once again, start Emacs manually in GDB, then copy/paste the command line there, and be sure to set the environment variables, like EMACS_TEST_DIRECTORY before starting GDB. Wait, and what if the test forks, and you need to attach GDB to the correct Emacs process? Another possible improvement would be for each test to automatically save the log of its run, including any error messages, to a file. As things are now, one needs to do that manually as well, using shell magic and/or 'tee', as the full log is extremely voluminous. I wish all this could be easier, and I wish there would be a README in that directory with instructions of how to do all that. A good example of both the facilities and their README documentation can be found in the Cairo distribution (look in the test/ subdirectory). Thanks in advance. In GNU Emacs 24.3.50.81 (i686-pc-mingw32) of 2013-11-28 on HOME-C4E4A596F7 Bzr revision: 115269 eliz@gnu.org-20131128194015-2eu4ceuanrqt0qwr Windowing system distributor `Microsoft Corp.', version 5.1.2600 Configured using: `configure --prefix=/d/usr --enable-checking=yes,glyphs 'CFLAGS=-O0 -gdwarf-2 -g3'' Important settings: value of $LANG: ENU locale-coding-system: cp1255 default enable-multibyte-characters: t Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Recent input: M-x r e p o r t - e m Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Load-path shadows: None found. Features: (shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp w32-common-fns disp-table w32-win w32-vars tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote make-network-process w32notify w32 multi-tty emacs)