Re: make check fails due to missing test directory

[Paul Eggert <eggert@cs.ucla.edu>]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

On 5/3/19 12:13 PM, Eli Zaretskii wrote:
> You need to download the test suite.  It is not part of the release
> tarball.  You can find it in the Emacs development repository.

Sure, but as his email suggests, it's better for 'make check' to succeed
when the set of tests is empty. That way, standardized build procedures
(like his) are more likely to do the right thing. I installed the patch
0001 (attached) into the master branch to do that.

We've talked before about distributing tests in the Emacs tarball -
which is what the GNU Codings Standards recommend and what pretty much
every other GNU package does - and this email suggests that now's a good
time to do that. Distributing tests adds about 2.5% to the size of the
compressed tarball, and nowadays that is a small price to pay for making
tests available to people building from a release tarball rather than a
Git clone. Proposed patch attached as patch 0002, and comments welcome.


[-- Attachment #2: 0001-Skip-tests-if-test-subdir-is-missing.patch --]
[-- Type: text/x-patch, Size: 1464 bytes --]

From 965372cd55c9a9ad5a9e17614dc98502c79ab1e3 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 3 May 2019 14:19:26 -0700
Subject: [PATCH 1/2] Skip tests if test subdir is missing

Problem reported by Jeffrey Walton in:
https://lists.gnu.org/r/emacs-devel/2019-05/msg00041.html
* Makefile.in (CHECK_TARGETS): New macro; use it
to simplify 'check' and similar rules.
($(CHECK_TARGETS)): If tests are missing, do not fail
after issuing a diagnostic.  Just skip the tests.
---
 Makefile.in | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/Makefile.in b/Makefile.in
index 06da415a4a..21362a9196 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -950,17 +950,15 @@ .PHONY:
 TAGS tags: lib lib-src # src
 	$(MAKE) -C src tags
 
-.PHONY: have-tests
-have-tests:
-	@if test ! -d test; then \
-	  echo "You do not seem to have the test/ directory."; \
-	  echo "Maybe you are using a release tarfile, rather than a repository checkout."; \
-	 exit 1; \
-	fi
-
-.PHONY: check check-maybe check-expensive check-all
-check check-maybe check-expensive check-all: have-tests all
+CHECK_TARGETS = check check-maybe check-expensive check-all
+.PHONY: $(CHECK_TARGETS)
+$(CHECK_TARGETS): all
+ifeq ($(wildcard test),test)
 	$(MAKE) -C test $@
+else
+	@echo "You do not seem to have the test/ directory."
+	@echo "Maybe you used a release tarfile that lacks tests."
+endif
 
 dist:
 	cd ${srcdir}; ./make-dist
-- 
2.21.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-Distribute-test-cases-in-tarballs-by-default.patch --]
[-- Type: text/x-patch; name="0002-Distribute-test-cases-in-tarballs-by-default.patch", Size: 4699 bytes --]

From c0f5ddc7b376976634fb4708519a12bc147ba312 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 3 May 2019 14:53:58 -0700
Subject: [PATCH 2/2] Distribute test cases in tarballs by default
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* INSTALL, INSTALL.REPO, admin/make-tarball.txt:
Mention ‘make check’.
* configure.ac: Update comment.
* etc/NEWS: Say that tarballs have a test directory.
* make-dist (with_tests): Default to "yes".
Add an option --no-tests to make it "no".
---
 INSTALL                |  4 ++++
 INSTALL.REPO           |  4 ++--
 admin/make-tarball.txt |  2 +-
 configure.ac           |  3 +--
 etc/NEWS               |  3 +++
 make-dist              | 11 ++++++++---
 6 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/INSTALL b/INSTALL
index e38635c60b..0d5d8aa1cb 100644
--- a/INSTALL
+++ b/INSTALL
@@ -82,6 +82,10 @@ sections if you need to.
 
 		 src/emacs -Q
 
+     To test Emacs further:
+
+		 make check
+
   6. Assuming that the program 'src/emacs' starts and displays its
      opening screen, you can install the program and its auxiliary
      files into their installation directories:
diff --git a/INSTALL.REPO b/INSTALL.REPO
index 6dca9dd714..b7433856c6 100644
--- a/INSTALL.REPO
+++ b/INSTALL.REPO
@@ -32,8 +32,8 @@ can invoke './configure -C'.  After configuring, build Emacs as follows:
 
   $ make
 
-If you want to install Emacs, type 'make install' instead of 'make' in
-the last command.
+You can also type 'make check' to test and 'make install' to install
+Emacs.
 
 Occasionally the file 'lisp/loaddefs.el' (and similar automatically
 generated files, such as 'esh-groups.el', and '*-loaddefs.el' in some
diff --git a/admin/make-tarball.txt b/admin/make-tarball.txt
index 47b60173f8..43992a0bb2 100644
--- a/admin/make-tarball.txt
+++ b/admin/make-tarball.txt
@@ -115,7 +115,7 @@ General steps (for each step, check for possible errors):
     results against the new tar contents.
 
 7.   tar -xf emacs-NEW.tar; cd emacs-NEW
-     ./configure --prefix=/tmp/emacs && make && make install
+     ./configure --prefix=/tmp/emacs && make check && make install
     Use 'script' or M-x compile to save the compilation log in
     compile-NEW.log and compare it against an old one.  The easiest way
     to do that is to visit the old log in Emacs, change the version
diff --git a/configure.ac b/configure.ac
index 79fe0c98c6..ecce0223e0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -5726,9 +5726,8 @@ m4_define
 SUBDIR_MAKEFILES="subdir_makefiles"
 AC_CONFIG_FILES(subdir_makefiles)
 
-dnl test/ is not present in release tarfiles.
+dnl The test/ directory is missing if './make-dist --no-tests' was used.
 opt_makefile=test/Makefile
-
 if test -f "$srcdir/$opt_makefile.in"; then
   SUBDIR_MAKEFILES="$SUBDIR_MAKEFILES $opt_makefile"
   dnl Again, it's best not to use a variable.  Though you can add
diff --git a/etc/NEWS b/etc/NEWS
index 9e3559d27e..f76000b956 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -97,6 +97,9 @@ option was useful with modern debugging tools such as AddressSanitizer.
 (See etc/DEBUG for the details of using the modern replacements of the
 removed configure options.)
 
++++
+** The distribution tarball now has test cases; 'make check' runs them.
+
 ---
 ** Emacs now requires GTK 2.24 and GTK 3.10 for the GTK 2 and GTK 3
 builds respectively.
diff --git a/make-dist b/make-dist
index 74660032e9..e4e6d40d98 100755
--- a/make-dist
+++ b/make-dist
@@ -52,7 +52,7 @@ make_tar=
 default_gzip=gzip
 newer=""
 with_info=yes
-with_tests=no
+with_tests=yes
 changelog=yes
 verbose=no
 
@@ -110,11 +110,16 @@ verbose=
      ;;
 
     ## Include the test/ directory.
-    ## This option is mainly for the hydra build server.
+    ## This is for backward compability to when --no-tests was the default.
     "--tests")
       with_tests=yes
      ;;
 
+    ## Exclude the test/ directory.
+    "--no-tests")
+      with_tests=no
+     ;;
+
     "--verbose")
       verbose=yes
      ;;
@@ -131,9 +136,9 @@ verbose=
       echo "  --no-update	don't recompile or do analogous things"
       echo "  --no-changelog	don't generate the top-level ChangeLog"
       echo "  --no-info		don't include info files"
+      echo "  --no-tests	don't include the test/ directory"
       echo "  --snapshot	same as --clean-up --no-update --tar --no-check"
       echo "  --tar		make a tar file"
-      echo "  --tests	include the test/ directory"
       echo "  --verbose		noisier output"
       echo ""
       exit 0
-- 
2.21.0