From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id GCAQJg0uRGAYaAAA0tVLHw (envelope-from ) for ; Sun, 07 Mar 2021 01:36:13 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 33vKIQ0uRGAlBAAAbx9fmQ (envelope-from ) for ; Sun, 07 Mar 2021 01:36:13 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1FFB61B52C for ; Sun, 7 Mar 2021 02:36:13 +0100 (CET) Received: from localhost ([::1]:53512 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lIiL5-0002Ja-MX for larch@yhetil.org; Sat, 06 Mar 2021 20:36:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:54352) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lIiKs-0002JR-Tk for guix-devel@gnu.org; Sat, 06 Mar 2021 20:35:59 -0500 Received: from world.peace.net ([64.112.178.59]:35700) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lIiKq-00052u-4O for guix-devel@gnu.org; Sat, 06 Mar 2021 20:35:58 -0500 Received: from mhw by world.peace.net with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lIiKo-0006Zg-5r; Sat, 06 Mar 2021 20:35:54 -0500 From: Mark H Weaver To: Christopher Baines , guix-devel@gnu.org Subject: Re: Implications of QEMU binfmt transparent emulation for builds In-Reply-To: <87eegsnexv.fsf@cbaines.net> References: <87eegsnexv.fsf@cbaines.net> Date: Sat, 06 Mar 2021 20:34:28 -0500 Message-ID: <87wnujzqyo.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=64.112.178.59; envelope-from=mhw@netris.org; helo=world.peace.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1615080973; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=LXr2MQFARIVgQvqTwdFV30jr/k5/5suOp+S8PJ/yOXo=; b=LDtE6a9v7cJ1aA9HG3RX1NwmaqWtrnSvijQ07+3e6x7UZW8UaZuqkZBVp2uwclC6r76l9f ywTgjfOG0Hqr7n+QCjXoVu0kjBTkJ7XxUeatyW082aVsE0tJ+7xx+dIyialbgn5jdQhBwY th7wIcY/l+bXVB/u5myNVLnHZcouNAcZeDSfm/5nXqiCdVtdqSZ9U+AU9R5ZkzKAmsWZRu oMkkRUye/C4gR/oGvci6r3APJyvRtXMxtZMiHmwKGkbEZwwcNDoiN0QENEfkzvmHm6IFeU 1uH8vdjdzCU6BQ8iZ+Rf/snrNp59OCrKyzDyuhM+6QUkLOhj5HAytDVpcKQzRg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1615080973; a=rsa-sha256; cv=none; b=tGoV7UrfMEIPBb2QKEmiybM8ffoJKxq0qiSFcuDkmMagxduPpDGBH/cdapZIjJvAtSeYZL zt37raHMbqlmg3iIZuhYITVPlmHHiJHKdxWPeWjf6RYtlIkS3XSrOMmHxxTVTvU/zKcP+u lWMHINSGUGk2wGZyQ+LsBg/1f8RO1jnm3b05LBui1qwVEX5yeMmBnPb1QlA+BbLcMGyX67 9P+XSDyyMBFTo5fhe1zEmmEoKiF/FNZzn72pmBRpNAwUssRpk+3Gcn2Z7pallQ2pVZBh4N Ix/0R3AEX9YbAS4yuqtua0t8WpQEv0loyiNbHCN0tiExvXP60rwgTauEflAS2g== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -2.37 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 1FFB61B52C X-Spam-Score: -2.37 X-Migadu-Scanner: scn0.migadu.com X-TUID: w/Pg5zJq7NXO Hi Christopher, Christopher Baines writes: > I'm starting to play with mixing native and emulated builds with the > Guix Build Coordinator again. I did do this many months ago, but at that > time, there wasn't support for targeting retries across a range of > machines, to help avoid blockages due to QEMU issues. > > Anyway, something that's been on my mind regarding QEMU and builds is > how well this matches up with building natively. Here's one significant problem with emulated builds, although perhaps not the kind of problem you were asking about: Due to differences in the memory models of Intel and other systems such as ARM, it is easy to write multithreaded code that works robustly on Intel systems but sporadically fails on other architectures such as ARM. However, an ARM emulator running on an Intel system will effectively use an Intel memory model, because it would be prohibitively expensive to faithfully simulate the ARM memory model on Intel hardware, and the only benefit would be to demonstrate bugs that can't happen on Intel systems. Therefore, test suites run under an emulator will almost certainly fail to find architecture-specific thread safety problems, which I expect are among the most important tests to be done, and among the most likely to otherwise go unnoticed. For this reason, I suggest that we should avoid emulated builds in our build farm. Regards, Mark