From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:1008:1e59::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id uCKMKpdeXWa7OwEAA41jLg (envelope-from ) for ; Mon, 03 Jun 2024 08:11:35 +0200 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id qJNbJ5deXWbhUQAA62LTzQ (envelope-from ) for ; Mon, 03 Jun 2024 08:11:35 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1717395095; a=rsa-sha256; cv=none; b=Jj0sZuTbHyvYRVI8/uZd5tQ4DIYxBIBGdjLu6aSiOnC6FHgky7k03AB8IYMaeyh1ryxIp+ dHeW2wxfHnZ9kyqQdv9sibyTl37IArKOxoAgjT7mal6/ZpLdJHZxzZXSyP8unCZssI2xiH RpAUzYe4RZpBDPlXY29zp8Ggtuj4C/UgiXvr/BV4KaPad8Urujs9wQ5vbvuXABsR2Z+93o L42ArR3VqjZ/Z2/PY44cBQnLofqAbKzLiY43u7fkLJ5OlLYCUR4Z09b64OOdo3lOWB9+oc TfEog5DKCKEr6r4EOggkUdcXTqvAvrL3eSl+fkMAPRlfiHzmVAAHCe/vr/WPnQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1717395095; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=GyS0AHO/fgNHEN6xvog7UWv4gAEc4x6rbjaTPtt59zg=; b=d+Nq7X0B6rxwSQFnlZ/bsKXBBFqsyAGC0bmJ9FW6fl8SQqlllwAyt6WlQ0336WpcZzfB1d JS5MfoWG2v9Nc21uDxoNsWqh9JJMxhadJSrkjHBDDYKRrVccjcgs1Lt7GsKAJaKcbYgOOH aLLlyUk3fdXsoNNzV5n/B0Xl5EXN9vCLUFRCzb/Nl2epN1VAVp167XQ16ntIuisd+bMj+0 gy0698dl81fpboYGNt0d20bRKl6SdOGR8C4ctoESq5N+xRqWoPfMlsXAdm4ghCzKzG+8Ua EnvYuI8k7Cqo6DDTkzQ200QLH3OF61qI1dp6M5nV5RS/2ucaEY/L9C4PUGZNIA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 820286E263 for ; Mon, 3 Jun 2024 08:11:35 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sE0uU-0002oD-1d; Mon, 03 Jun 2024 02:11:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sDzpp-0004RG-CM for guix-devel@gnu.org; Mon, 03 Jun 2024 01:02:17 -0400 Received: from thyrsus.com ([71.162.243.5] helo=snark.thyrsus.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sDsvP-0006gL-3P; Sun, 02 Jun 2024 17:39:37 -0400 Received: by snark.thyrsus.com (Postfix, from userid 1000) id 9373618A07A9; Sun, 2 Jun 2024 17:39:31 -0400 (EDT) Date: Sun, 2 Jun 2024 17:39:31 -0400 From: "Eric S. Raymond" To: jbranso@dismail.de Cc: guix-devel@gnu.org, rms@gnu.org, Ludovic =?iso-8859-1?Q?Court=E8s?= Subject: Re: Autodafe is "production" ready Message-ID: References: <9101aa92b331d50f3e70be1e676f7cafa1c1851f@dismail.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <9101aa92b331d50f3e70be1e676f7cafa1c1851f@dismail.de> Organization: Eric Conspiracy Secret Labs X-Eric-Conspiracy: There is no conspiracy Received-SPF: pass client-ip=71.162.243.5; envelope-from=esr@thyrsus.com; helo=snark.thyrsus.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Mon, 03 Jun 2024 02:11:09 -0400 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: esr@thyrsus.com Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Queue-Id: 820286E263 X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -6.34 X-Spam-Score: -6.34 X-TUID: tWMG/LgTVYqN jbranso@dismail.de : > Hey Guix! > > Have you heard about autodafe? I just did today. > > So it looks like autodafe (1) converts autotools into > simple makefiles. Quoting from it's hacker's guide (2):   > "The proximate goal of this project is to eliminate autotools's > piles of intermediate products and scripts that create a > jungle in which exploits like the xz crack of 2024 can hide. >  The wider goal is to abolish the festering complexity sink > that autotools recipes have become."  Sounds like a fairly cool > goal eh? I should emphasize that the project was *directly* motivated by the xz Trojan. That's what, in my mind, moved autotools from "chronic irritation" to "Someone should kill it with fire, *now*, and I guess that will be me". Took me eight weeks of balls-to-the-wall hacking. The demand for some solution to supply-chain attacks through autotools is urgent. I've been getting a particular earful about it from a friend who is on the Gentoo packager lists; they want to be able to throw out autotools *yesterday*. I have no doubt similar conversations are going on within other packaging groups. > I know that guix has wanted to create a "guile based build > system" to replace autotools (autodafe is written in > python 3).  It sounds like Eric's work is moving to > supplant and/or fix or repace autotools.  Eric is there > anything Guix can do for you to help you with your goal? > We do love our guile, so we will probably encourage you > to use some scheme code.  :) I looked into using Guile embedded in Make as an engine; I'm a fan of Scheme myself. There are two reasons I didn't go that route: 1. Practical: Guile support is normally disablled in the packaged versions of gmake on Debian, Ubuntu, Red Hat, etc. That means that for my purpose of fielding a prompt replacement for autotools, Guile might as well not be there at all. The demand for a usable drop-in replacement for autotools won't wait for the number of packaging cycles it would take to get all the distributions on board with enabling embedded Guile-in-make and then deploying it. 2. Architectural: I have a promary goal of making the build flow in the replacement for autotools more auditable, with fewer places an attacker can hide exploit components. As much as I personally like Scheme, it would be a move in the wrong direction from an audiability perspective even if it were available. Too few people can read Scheme, or want to be able to. What Scheme might have done if it were actually available I have managed to cover with 661 lines of standalone Python 3. Under the requirements, that is in every way a better solution, even if not having an excuse to hack some Lisp makes me a little wistful. > Richard do you have any advice for Eric in how to "fix or > replace" autotools? > > Ludo, does guix need to do anything to get ready for some > software wanting to use autodafe  ?  Is there a way we > could use autodafe to help us get started on a guile based > build tool? You'll make a better decision about that if you understand what I've actually done. There are three tools in the autodafe suite. Two of them are one-time transition aids; the third replaces autotools configure and is meant to be checked into project repositories and shipped with them. deconfig analyzes a code tree, removes guards abound POSIX features, and reports a minimal set of guard symbols for non-POSIX extensions. On a typical large old project, assuming POSIX reduces the complexity of the configuration about an order of magnitude - you go from hundreds of goard symbols to dozens. Yes, this does mean you risk losing back-portability to big-iron Unixes from the last century. One of the design premises of autodafe is that in 2024 this is an acceptable trade for a huge reduction in the complexity and obscurity of the build recipe. makemake takes a Makefuke generated by autotools and tries to rurn it into a standalone Makefile that can be read and modifies by humans - removing unused boilerplate and rearranging parts so the control variables of the build are easy to modify. These are two transion aids. They aren't needed onece you have your nuild conversion done. autodafe configure replaces the autotools generated configure script, but is much, *much* smaller and easier to read than those are. It reads magic comments in your Makefile (which you add during the conversion process - deconfig will typically generates most of them for you) to know which testrs to do to discover the capabilities of your host ststem. Here's what the example from the documentation looks like: # CHECK_ENABLE(ipv6) # CHECK_DECL(sys/socket.h, netinet/in.h, netinet/tcp.h, TCP_KEEPCNT) # CHECK_DECL(sys/socket.h, netinet/in.h, netinet/tcp.h, TCP_KEEPIDLE) # CHECK_DECL(sys/socket.h, netinet/in.h, netinet/tcp.h, TCP_KEEPINTVL) # CHECK_HAVE(getopt.h, getopt_long) # CHECK_HAVE(_GNU_SOURCE, unistd.h, getresuid) # CHECK_LIB(m) REQUIRED # CHECK_LIB(nsl) # CHECK_LIB(resolv) # CHECK_LIB(rt) # CHECK_HAVE(linux/if/tun.h) # CHECK_HAVE(linux/ipv6.h) # CHECK_HAVE(_GNU_SOURCE, string.h, memrchr) # CHECK_HAVE(_GNU_SOURCE, unistd.h, setresuid) # CHECK_SIZEOF(int) # CHECK_SIZEOF(int *) # CHECK_SIZEOF(long) # CHECK_SIZEOF(long long) # CHECK_SIZEOF(sys/types.h, off_t) # CHECK_SIZEOF(pthread.h, pthread_t) # CHECK_SIZEOF(stddef.h, size_t) The effect is that the Makefile becomes the single point of truth for the build, repl;acing configure.ac. The entire transition process is described in detail here: https://gitlab.com/esr/autodafe/-/blob/master/configure.adoc?ref_type=heads > Thanks, > > Joshua > > > 1) https://www.phoronix.com/news/Autodafe-1.0-Released > > 2) https://gitlab.com/esr/autodafe/-/blob/master/hacking.adoc -- Eric S. Raymond