From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: =?UTF-8?Q?Cl=c3=a9ment_Pit--Claudel?= Newsgroups: gmane.emacs.devel Subject: Re: Display test suite Date: Thu, 7 Jul 2016 15:40:42 -0400 Message-ID: <577EB03A.6030703@gmail.com> References: <83h9c1lbso.fsf@gnu.org> <577E79AA.2060208@gmail.com> <83ziptjtsk.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fK9tE9aBX4aLgXa1WN62TVP9VwiNxj6La" X-Trace: ger.gmane.org 1467920518 569 80.91.229.3 (7 Jul 2016 19:41:58 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 7 Jul 2016 19:41:58 +0000 (UTC) Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Jul 07 21:41:50 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bLFBF-0004LF-JC for ged-emacs-devel@m.gmane.org; Thu, 07 Jul 2016 21:41:49 +0200 Original-Received: from localhost ([::1]:41951 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bLFBE-0005K7-SE for ged-emacs-devel@m.gmane.org; Thu, 07 Jul 2016 15:41:48 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:44838) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bLFAL-000572-Ek for emacs-devel@gnu.org; Thu, 07 Jul 2016 15:40:54 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bLFAJ-0007t4-Pv for emacs-devel@gnu.org; Thu, 07 Jul 2016 15:40:52 -0400 Original-Received: from mout.kundenserver.de ([212.227.17.13]:61803) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bLFAD-0007rB-QO; Thu, 07 Jul 2016 15:40:46 -0400 Original-Received: from [18.189.62.212] ([18.189.62.212]) by mrelayeu.kundenserver.de (mreue101) with ESMTPSA (Nemesis) id 0MMF5h-1bDodV3iZB-0081Vq; Thu, 07 Jul 2016 21:40:45 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.8.0 In-Reply-To: <83ziptjtsk.fsf@gnu.org> X-Provags-ID: V03:K0:4TMT3Itz6qoAP4Dt7w0wi34JzLV2HWkDIHopUukJ+HjKWN/HblL k8Rg4KrbZJhJt0kRwu2caMSyrmH+L1lHdIpK/EJW6yoVhExBasaKWWfReFvYo0CGJxaBBMQ tJCXea2BcrdeCf0nmU0XMa3AY9hwmaON0C4PpPTpNAFJo0iTluG/3q9RNN+vsoGDEZQWrtm bkkx4amauTN4GELCz9VWA== X-UI-Out-Filterresults: notjunk:1;V01:K0:3u1IJjJdvDU=:+tgsEGGaTRcbKgh78iHUGM BNui7OEGPdbSFXuJlmLxmqfztCCnbrdCnrummEz7H9SQmb8E8gCveREV6nHtR5YBdQDomvSEb lE3nvhkH8WW9QZH6OqCh6HvwWwlqHIr0hmnwZeZBHM5onHBSk3ga6h6wUymSPbD5PQN7txfD0 bf6BHFEdjmpKq7JblPLcq1pQja5wn//8gjwcSpxAHWnUaW7ozqR79MDdhD/7Z5dnhKGLjaELA xJVbd17Rl6q9FY9z1f0gaOdAOiSyYv6dGuXcuhnhfVtyZeCruM5IpcpUoX8lACw+HqPFtRdRR zxcsSZ4tB+L/AYFHZuyau5NQ5Gx/0zBCfhcbvowlpOgYRwS4mIVc1CQcZHVOrOT6xm/LVmptO XFaeT0cR+Y37o3nr1ZffE8mTsUgg9F5AdtHU568WO1dgix6Eff39K0oTDFD3Y6gy6DLwcBq5I fTAhX1DBGsvpg1TSRfqjm9x5GBvb6YNGWQwG9auAH+FkrafQv9CKNAOEz2RNkhn12MN80FBq2 MBYBgZOBecUhPYndPT37vWlInBxLQql+1g51bug/AitIox/TjD1x8VxDlXHTPOP9dOTnvnX7j xLQKAN+9VvPDm0x4U5nMEFeLOG79JlxBtrj+Mi+jJPhl2KSYQt+TqlrtW6n/FMtitmSIOFiro uiMq3KEQPP8QBNbhne0bjRS+AmjMjlPHwV6YJla981p28OchB2WQX06de7ENdQdxFihw= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 212.227.17.13 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:205370 Archived-At: This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --fK9tE9aBX4aLgXa1WN62TVP9VwiNxj6La Content-Type: multipart/mixed; boundary="Up2EnBb09vsRitkvKbcBVXoNQaxppxMVV" From: =?UTF-8?Q?Cl=c3=a9ment_Pit--Claudel?= To: Eli Zaretskii Cc: emacs-devel@gnu.org Message-ID: <577EB03A.6030703@gmail.com> Subject: Re: Display test suite References: <83h9c1lbso.fsf@gnu.org> <577E79AA.2060208@gmail.com> <83ziptjtsk.fsf@gnu.org> In-Reply-To: <83ziptjtsk.fsf@gnu.org> --Up2EnBb09vsRitkvKbcBVXoNQaxppxMVV Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2016-07-07 12:29, Eli Zaretskii wrote: >> From: Cl=E9ment Pit--Claudel Date: Thu, 7 Jul >> 2016 11:47:54 -0400 >>=20 >> Richard recently suggested giving Emacs the ability to take a >> screenshot of itself. Could we imagine using this to run regression >> tests of the display engine? >=20 > Maybe I'm missing something, but I don't see how. A test suite > needs to include the expected results for each test. Even if you > include an image file for each test that shows what should be > displayed, AFAIU the image will be specific to the display > resolution, number of colors, dimensions, etc., so inappropriate for > automatic comparison with the screenshot on the target machine. Am I > right? You're right; that's why I can only run the company-coq tests on my machi= ne. Still, one might imagine setting up Emacs in a particular way to make= them reproducible. Along these lines, I run the PNG through an optimizer= and strip the metadata to make sure that I get the same output every tim= e. > Also, comparing images that could have a small number of pixels=20 > different in some corner is error-prone. There are very nice tools to do image diffs; they will highlight the offe= nding pixels in a different color. > IMO, a test suite for the display engine needs a way to describe=20 > precisely what's on the screen, in some language, and then compare=20 > those descriptions. Yes, that would be nice (we have something like that for font-lock, right= ?) >> Of course, it could be hard to get screeenshots to looks exactly >> the same for everyone. Instead, we could have a makefile target >> that builds a large collection of these screenshots, and another >> one that rebuilds it and compares against a previously generated >> one. >=20 > But then each time I start using a new machine, my reference point > is reset to a newer date, and I lose the ability for comparing > against older versions. Absolutely. I think such tests would serve as a nice way to check your ow= n commits (for before/after tests). Also, maybe the variability is a good point: each people regularly run th= ese tests, then changes that affect only one configuration would be notic= ed. If we have tests that produce the same output for every one, changes = that affect only one particular set of configurations would not be notice= d. In any case, I've found the graphical tests that I do for company-coq ver= y useful; whether they could generalize to all of Emacs is a non-trivial = question. Cheers, Cl=E9ment. --Up2EnBb09vsRitkvKbcBVXoNQaxppxMVV-- --fK9tE9aBX4aLgXa1WN62TVP9VwiNxj6La Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJXfrA6AAoJEPqg+cTm90wjI4QP/05Vuvt768i8bdxOXb/XnCLl f7rUSYqukgqblzJjwTUvLdKjGxU1eS9qKiZclndyynSIG6zdo0S5fJgDER+vPrkr epv7Q7RpZ4QGihIHtmIQtSmoSFUalDcfbkvtfIzm7aKgGFr3LsVOp+m/Zsr/dgIN n7RapaKIxcC4ZCKs70a1z2sjz529E4dL4SZQa3/Uttz91+1x5d+Iy/HAHfEkbPdC rAQ3LiT8K1s4eUiQb3IsUxJTe8tgL3LQ9dFvxDR0FocUs9CZDj+i+MVPOReviu8b X8KbXIix0ed86ZRr+zhY66Vh8KiVnSFA1GTlimGJylutD8p5n1WomQAeTvgEX6WQ 746Ea2F2YxXFIWQQhfixeiv//uE9SheCfBqyHsNTJe2nTzpC98DrItnMPDdAkoya 75pSKCNxC+OD3I9URLSU6o5npexufK9I+vTt3lJQf1fQOuYnMojSc1ap7pQ1NiMW ESnkT/6NVrAzJa4ykreK9FfBeWiSAW6SLvn4XSjzL9QBlVrzKRcHdQa3LcPDo0vT WN1vU7ADi65KKKQdv+Zm9i40+ggSyNCjVASaKndMBbi1L0z9vgTPr9coFs3QcsJK wZPAR7+BOxEujU/W5RBw40yeO1PX+RBGSGBo3qtjaGU/+XMbPXkCAiqUgeiyKSZA b1IkHbBxUKJjGJij1cmd =EQTj -----END PGP SIGNATURE----- --fK9tE9aBX4aLgXa1WN62TVP9VwiNxj6La--