From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: ludo@gnu.org (Ludovic =?iso-8859-1?Q?Court=E8s?=) Newsgroups: gmane.lisp.guile.devel Subject: On white-box tests Date: Wed, 19 Aug 2009 10:38:07 +0200 Message-ID: <87d46sci6o.fsf@gnu.org> References: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: ger.gmane.org 1250671139 11851 80.91.229.12 (19 Aug 2009 08:38:59 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 19 Aug 2009 08:38:59 +0000 (UTC) Cc: guile-devel@gnu.org To: "Michael Gran" Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Wed Aug 19 10:38:52 2009 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1Mdgga-0006aD-QU for guile-devel@m.gmane.org; Wed, 19 Aug 2009 10:38:25 +0200 Original-Received: from localhost ([127.0.0.1]:50415 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Mdgga-0001qA-5A for guile-devel@m.gmane.org; Wed, 19 Aug 2009 04:38:24 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1MdggT-0001pr-0E for guile-devel@gnu.org; Wed, 19 Aug 2009 04:38:17 -0400 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1MdggO-0001pN-Cq for guile-devel@gnu.org; Wed, 19 Aug 2009 04:38:16 -0400 Original-Received: from [199.232.76.173] (port=54660 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1MdggO-0001pK-7u for guile-devel@gnu.org; Wed, 19 Aug 2009 04:38:12 -0400 Original-Received: from mail4-relais-sop.national.inria.fr ([192.134.164.105]:17671) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.60) (envelope-from ) id 1MdggN-0007qp-Gd for guile-devel@gnu.org; Wed, 19 Aug 2009 04:38:11 -0400 X-IronPort-AV: E=Sophos;i="4.43,407,1246831200"; d="scan'208";a="44826473" Original-Received: from laptop-147-210-128-170.labri.fr (HELO nixey) ([147.210.128.170]) by mail4-relais-sop.national.inria.fr with ESMTP/TLS/DHE-RSA-AES128-SHA; 19 Aug 2009 10:38:09 +0200 X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 2 Fructidor an 217 de la =?iso-8859-1?Q?R=E9volution?= X-PGP-Key-ID: 0xEA52ECF4 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 821D 815D 902A 7EAB 5CEE D120 7FBA 3D4F EB1F 5364 X-OS: x86_64-unknown-linux-gnu In-Reply-To: (Michael Gran's message of "Tue, 11 Aug 2009 05:23:42 +0000") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) X-detected-operating-system: by monty-python.gnu.org: Genre and OS details not recognized. X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:9154 Archived-At: Hello! Just a note that I've been meaning to send for some time. "Michael Gran" writes: > http://git.savannah.gnu.org/cgit/guile.git/commit/?id=9b0c25f6d18d5be318ea3a47fd87cf7e63b689e1 [...] > --- a/test-suite/tests/strings.test > +++ b/test-suite/tests/strings.test [...] > +;; > +;; string internals > +;; > + > +;; Some abbreviations > +;; BMP - Basic Multilingual Plane (codepoints below U+FFFF) > +;; SMP - Suplementary Multilingual Plane (codebpoints from U+10000 to U+1FFFF) > + > +(with-test-prefix "string internals" > + > + (pass-if "new string starts at 1st char in stringbuf" > + (let ((s "abc")) > + (= 0 (assq-ref (%string-dump s) 'start)))) > + > + (pass-if "length of new string same as stringbuf" > + (let ((s "def")) > + (= (string-length s) (assq-ref (%string-dump s) 'stringbuf-length)))) > + > + (pass-if "contents of new string same as stringbuf" > + (let ((s "ghi")) > + (string=? s (assq-ref (%string-dump s) 'stringbuf-chars)))) > + > + (pass-if "writable strings are not read-only" > + (let ((s "zyx")) > + (not (assq-ref (%string-dump s) 'read-only)))) > + > + (pass-if "read-only strings are read-only" > + (let ((s (substring/read-only "zyx" 0))) > + (assq-ref (%string-dump s) 'read-only))) > + > + (pass-if "null strings are inlined" > + (let ((s "")) > + (assq-ref (%string-dump s) 'stringbuf-inline))) First of all, thanks for taking the time to write unit tests! I'm not fully convinced by some of these "string internals" tests, though. These are "white box tests". I believe these tests are mostly useful when written by someone different than the one who implemented the code under test, both of whom following a given specification. When someone writes both the code and the white box tests, I'm afraid they end up writing the same code twice, just differently. For example: SCM scm_i_substring_read_only (SCM str, size_t start, size_t end) { [...] return scm_double_cell (RO_STRING_TAG, /* <--- read-only */ SCM_UNPACK(buf), (scm_t_bits)str_start + start, (scm_t_bits) end - start); } and: (pass-if "read-only strings are read-only" (let ((s (substring/read-only "zyx" 0))) (assq-ref (%string-dump s) 'read-only))) Thus, I think such tests don't provide much information. Conversely, for this example, a black-box test of the public API would have helped catch regressions and non-conformance issues: (pass-if-exception exception:read-only-string (string-set! (substring/read-only "zyx" 0) 1 #\x)) Another issue is that these tests expose a lot of the implementation details, e.g.: (pass-if "short Latin-1 encoded strings are inlined" (let ((s "m")) (assq-ref (%string-dump s) 'stringbuf-inline))) The inline/outline distinction is an implementation detail. If we change it (that's something we could get rid of in the BDW-GC branch, for instance) then the tests will have to be rewritten or removed. Conversely, black box tests can give confidence that a change in implementation details did have any user-visible impact. There *are* situations where the gap between the public API and the internals is too important, and white box tests can come in handy here (that's why `%string-dump' et al. are nice tools). However, often, I think it's better to have good coverage of the public API than a wealth of "obvious" white-box tests. What do you think? Thanks, Ludo'.