From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 0Lp5FJC4cmEorQAAgWs5BA (envelope-from ) for ; Fri, 22 Oct 2021 15:11:44 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id UKwMEJC4cmFFGAAAB5/wlQ (envelope-from ) for ; Fri, 22 Oct 2021 13:11:44 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id F0A142BDF8 for ; Fri, 22 Oct 2021 15:11:43 +0200 (CEST) Received: from localhost ([::1]:40262 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mduKl-000354-30 for larch@yhetil.org; Fri, 22 Oct 2021 09:11:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51850) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mduKH-00033M-NP for guix-devel@gnu.org; Fri, 22 Oct 2021 09:11:15 -0400 Received: from mail-io1-xd36.google.com ([2607:f8b0:4864:20::d36]:40548) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mduKD-00004u-LH for guix-devel@gnu.org; Fri, 22 Oct 2021 09:11:12 -0400 Received: by mail-io1-xd36.google.com with SMTP id q6so1163089iod.7 for ; Fri, 22 Oct 2021 06:11:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=greghogan-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=SCxdCRfMWLZ7qQG5NgJ2grli5b8hPa7kWKP9xBS8Qd8=; b=uMAwYpcPCEsa9MU37Mq+NJ4BOUXxngWWqld8hLFkPfgr5E+8ifqU78+Q2iVEsUyOvA pZ+zcaT0QflE35iAsXMbFp2inrcu9xKfIBMRpQ76CjQ1yiXq4jZVoiDu2pkJYgdGXRyL sJPhiQKcvzJZ8F88NbtkNnVrAMDExsn5KQuGTR2w5EooNUys/3ldRfp050AUlqh3izbh b6fX06lKlujlU+VzDBB3xaM38Wfza1v4uXNNynD/xtBFeZRqQLMbSYvdDaSa18iHR0NR S0kyUZMN6ss3SQ4njVs6YUCu3wARbZ1mooHR3UWDi0RbMHlQVcWpKCok70kTR0v2Cgbx wxmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=SCxdCRfMWLZ7qQG5NgJ2grli5b8hPa7kWKP9xBS8Qd8=; b=eI+v89uz6c9YfR+8NB+DZ0CZ5VE8w0fkTflIonaOWAlhkGxMPyNbZq/GsXru5ACYiH rjidVC/k6IEPKbAb5LF5aFimgEIbHpdmrPqkezbg6xpMBiLdvrD9tns+zR5DQWvxsrFt KinmBsjuqbKVYmJNx1EOEjTqqGq7919m+oZh+GmOYddX+wTgHji8V1KTnWB8wsRop/xa t83/yvSvG1p/hc0jtRhmzhVVMRqCy1VpyMpMDCKUqtXbBiAUOZ9O7Psss1E0W2sJie1V h6JqSaEbsZ3t+Dnh2fObUoIAB5cmM4MXabC+K0KCg0MiPfuwsc+/zX+KLyS3S+kmnwIG p6AQ== X-Gm-Message-State: AOAM530oneL9AN4FErtAwkeJ+O3z3EVT4bcyhII+p1c0sLzvCOq+g0J0 5Op89/mT8RCmq/XOe4HYiwjaOChm7GlB1BrHigSogw== X-Google-Smtp-Source: ABdhPJzpYzbjYUL1WjMyyMI6USNIEZEelJmJ+aPFR77wTou367PU4QbGFCHdpw5HMuAfK1StnJFf5hcx82ZqafIpqcw= X-Received: by 2002:a02:8808:: with SMTP id r8mr8565497jai.105.1634908267665; Fri, 22 Oct 2021 06:11:07 -0700 (PDT) MIME-Version: 1.0 References: <3906845baca1424432c1fd5c3ab23c1400695b66.camel@gmail.com> In-Reply-To: <3906845baca1424432c1fd5c3ab23c1400695b66.camel@gmail.com> From: Greg Hogan Date: Fri, 22 Oct 2021 09:10:56 -0400 Message-ID: Subject: Re: Test parallelism with CMake To: Liliana Marie Prikler , =?UTF-8?Q?Ludovic_Court=C3=A8s?= , guix-devel@gnu.org Content-Type: multipart/alternative; boundary="00000000000088ef3105cef0befc" Received-SPF: none client-ip=2607:f8b0:4864:20::d36; envelope-from=code@greghogan.com; helo=mail-io1-xd36.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1634908304; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=SCxdCRfMWLZ7qQG5NgJ2grli5b8hPa7kWKP9xBS8Qd8=; b=IGOS5HKFR5U/7oRQiFAZZ4YfByHHL6+CDL587WDBXMPkY8vX83JNocMkn4HcSAKkzkGnAy 0U1MzBd2wRSXmHNOV2vOxSeC6/PMnII09x2mxQsRrep+nyJD+B1YTO00uy7Smjbzv0O8E+ ypTuW4FPHmYv0CifziyvKFr6ymSJdwgXLrWdnjz4hCvP9vTDX4L3j9dskYa5MclWPEc65l cXSjovkE78lukoZTITJwn7547+4p6ZrkvN5gmVYJWRpV5GWWAPZa5JrVDlgjEJFyqzObhC dL+nCzU3my4KoeWX2UMh5+AH9gU/61sC+bp/gXMoWr5c5awuSwtmx99N9Js1uw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1634908304; a=rsa-sha256; cv=none; b=qyq7mrsIcpyHvEm/5ny/aTGZ02QtTRS5kt2XrknTivMSvojrL8cviimgqGeYVPxnZx8QE3 ELleD2Efp8gJVAm/04n+iOoIrahL+vyCroPY4DEIMItj+umy58W/3xcclC/3ighNq1cNug QpSBduONjeQv7Bnz4PdMZH4hDY/letZsRqoHC4/za1IEP4fgzl2OGERlxj3S45P4I0EeSb /WwkhJEFoT9SmqxD6ieQmNG03qfrzZ48d63pKrfyEKh6ZwNNwt9WIanLCg0UdJt+RUNgnt 1q5ObbiEoRbjYQADIsJ+JPbVoGecnnT9SSDD1KBZCgPf9jO64xpgFvyKAHjZ3A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=greghogan-com.20210112.gappssmtp.com header.s=20210112 header.b=uMAwYpcP; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -3.53 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=greghogan-com.20210112.gappssmtp.com header.s=20210112 header.b=uMAwYpcP; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: F0A142BDF8 X-Spam-Score: -3.53 X-Migadu-Scanner: scn1.migadu.com X-TUID: sNKDwC6C2kxn --00000000000088ef3105cef0befc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Oct 9, 2021 at 3:56 AM Liliana Marie Prikler < liliana.prikler@gmail.com> wrote: > for the purposes of GNU Guix, #:parallel-build? and #:parallel-tests? > are distinct flags and the latter (if implemented) would apply to the > check phase. Our cmake-build-system in this case defers to gnu-build- > system, which ought to insert -jN as you described. > On Thu, Oct 21, 2021 at 3:30 PM Ludovic Court=C3=A8s wrote: > Does CMake generate makefiles targets that would allow tests to run in > parallel? How does that even work in CMake? > Hi Liliana and Ludo=E2=80=99, The cmake-build-system does defer to gnu-build-system, which calls `make test -jN`; however, CMake generated Makefile specifies 'test' as a single target (the Ninja generator suffers from the same issue) so `ctest` is run without parallelism. To run CMake tests with parallelism the cmake-build-system should directly call `ctest` with the configured parallelism (same '-jN' argument). The cmake-build-system's check method is essentially untouched from March, 2013 (commit c6bded8a) so this issue has existed from the beginning. I made an inelegant patch to my local guix repo, essentially replacing cmake-build-system:check with a copy of gnu-build-system:check and changing 'apply invoke "make" test-target' to 'apply invoke "ctest"'. This works, although the package I was working on requires parallel tests to be disabled. Once the expectation is set that code will be run serially it is difficult to change the default to parallel execution. It seems that we should at a minimum document the issue in cmake-build-system:check. We could patch cmake-build-system to enable test parallelism and explicitly disable that setting for packages which succeed before but fail after making the change. What do you think? Greg --00000000000088ef3105cef0befc Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Sat, Oct 9, 2021 at 3:56 AM Liliana Ma= rie Prikler <liliana.prikler@gmail.com> wrote:
for the purposes of= GNU Guix, #:parallel-build? and #:parallel-tests?
are distinct flags and the latter (if implemented) would apply to the
check phase.=C2=A0 Our cmake-build-system in this case defers to gnu-build-=
system, which ought to insert -jN as you described.
= =C2=A0
On Thu, Oct 21, 2021= at 3:30 PM Ludovic Court=C3=A8s <ludo@g= nu.org> wrote:
Does CMake generate makefiles targets that would allow tests to run i= n
parallel?=C2=A0 How does that even work in CMake?

Hi Liliana and Ludo=E2=80=99,

The cmake-build-system does defer to gnu-build-system, which calls `make = test -jN`; however, CMake generated Makefile specifies 'test' as a = single target (the Ninja generator suffers from the same issue) so `ctest` = is run without parallelism.

To run CMake tests wit= h parallelism the cmake-build-system should directly call `ctest` with the = configured parallelism (same '-jN' argument). The cmake-build-syste= m's check method is essentially untouched from March, 2013 (commit c6bd= ed8a) so this issue has existed from the beginning.

I made an inelegant patch to my local guix repo, essentially replacing cm= ake-build-system:check with a copy of gnu-build-system:check and changing &= #39;apply invoke "make" test-target' to 'apply invoke &qu= ot;ctest"'. This works, although the package I was working on requ= ires parallel tests to be disabled. Once the expectation is set that code w= ill be run serially it is difficult to change the default to parallel execu= tion.

It seems that we should at a minimum documen= t the issue in cmake-build-system:check. We could patch cmake-build-system = to enable test parallelism and explicitly disable that setting for packages= which succeed before but fail after making the change. What do you think?<= /div>

Greg
--00000000000088ef3105cef0befc--