From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: joakim@verona.se Newsgroups: gmane.emacs.devel Subject: Re: Automate Emacs UI testing? Date: Wed, 11 Nov 2015 18:18:44 +0100 Message-ID: References: <831tbxn5id.fsf@gnu.org> <87oaf1bvsb.fsf@wanadoo.es> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1447262705 22110 80.91.229.3 (11 Nov 2015 17:25:05 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 11 Nov 2015 17:25:05 +0000 (UTC) Cc: ofv@wanadoo.es, emacs-devel@gnu.org To: Richard Stallman Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Wed Nov 11 18:24:53 2015 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ZwZ8c-0000Xn-7Q for ged-emacs-devel@m.gmane.org; Wed, 11 Nov 2015 18:24:50 +0100 Original-Received: from localhost ([::1]:42049 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZwZ8b-000834-P2 for ged-emacs-devel@m.gmane.org; Wed, 11 Nov 2015 12:24:49 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:55795) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZwZ5j-0004uY-Iu for emacs-devel@gnu.org; Wed, 11 Nov 2015 12:24:47 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZwZ3B-0001rj-C4 for emacs-devel@gnu.org; Wed, 11 Nov 2015 12:21:51 -0500 Original-Received: from mx6.bahnhof.se ([213.80.101.16]:42487) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZwZ3B-0001r1-1u; Wed, 11 Nov 2015 12:19:13 -0500 Original-Received: from localhost (mf.bahnhof.se [213.80.101.20]) by mx6-reinject (Postfix) with ESMTP id 64FC246BD1; Wed, 11 Nov 2015 18:19:11 +0100 (CET) X-Virus-Scanned: by amavisd-new using ClamAV at bahnhof.se (MF4) Original-Received: from mf4.bahnhof.se ([127.0.0.1]) by localhost (mf4.bahnhof.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Pox2VJyhvADf; Wed, 11 Nov 2015 18:19:04 +0100 (CET) Original-Received: from mta.verona.se (h-235-62.a149.priv.bahnhof.se [85.24.235.62]) by mf4.bahnhof.se (Postfix) with ESMTP id 2CAD33D78B4; Wed, 11 Nov 2015 18:19:03 +0100 (CET) Original-Received: from localhost (unknown [127.0.0.1]) by mta.verona.se (Postfix) with ESMTP id C35EF4EE61C; Wed, 11 Nov 2015 17:19:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at verona.se Original-Received: from mta.verona.se ([127.0.0.1]) by localhost (exodia.verona.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5zU7QFkmuTA9; Wed, 11 Nov 2015 18:18:44 +0100 (CET) Original-Received: from exodia.verona.se (www.verona.se [192.168.200.15]) by mta.verona.se (Postfix) with ESMTP id DEC9F4EE61B; Wed, 11 Nov 2015 18:18:44 +0100 (CET) In-Reply-To: (Richard Stallman's message of "Wed, 11 Nov 2015 11:59:09 -0500") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 213.80.101.16 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:194105 Archived-At: Richard Stallman writes: > [[[ To any NSA and FBI agents reading my email: please consider ]]] > [[[ whether defending the US Constitution against all enemies, ]]] > [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > > > - Isolate an Emacs build and environment in a Docker container > > environment. > > Is Docker free software? Yes, it is free software. Apache v2. > > > - Make sikuli tests that test things like Helm and Yasnippets. > > What is a sikuli test? Can they be done without using any nonfree > software? Yes, Sikuli if free software, MIT license. A Sikuli test simulates what a user would do in terms of keyboard and mouse input. Sikuli can then verify that the correct thing happened in the application window, by analyzing what happened in the screen. The novelty Sikuli adds over older such test systems is that Sikuli is better at analyzing the screen bitmaps, because it uses a computer vision library rather than just verifying on a pixel by pixel basis. > > > - Do these tests on each commit in a Jenkins build server that I > > have. It could be any build server with the right configuration though. > > It would not be good for our tests to depend on running on one particular site. > They should run on any GNU/Linux system, however it could be ok for some tests to require loading certain well-known free software packages. The tests should run on any environment that support Docker, which is only free operating system AFAIK. (You can run parts of Docker on non-free operating systems, but that is of no real concern for us I would say) An important thing Docker helps with is packaging software so that it is easy to build and run the software. Dependencies can be clearly declared so building a container is portable. One way to look at it is that Docker is like a standardized chroot build and run environment. > > > Here is a container > > with Sikuli that I havent tested: https://hub.docker.com/r/kkochubey1/sikuli-chrome-x11vnc/ > > Maybe I could find out some of those answers by looking around at that, > but I have a feeling it would take hours to find them that way. I hope this helps. -- Joakim Verona