From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id YFKVHGaWNl/+ZQAA0tVLHw (envelope-from ) for ; Fri, 14 Aug 2020 13:49:26 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id mFCQGGaWNl+4BQAA1q6Kng (envelope-from ) for ; Fri, 14 Aug 2020 13:49:26 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BBDFC940223 for ; Fri, 14 Aug 2020 13:49:25 +0000 (UTC) Received: from localhost ([::1]:49078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6a5E-00025a-Hb for larch@yhetil.org; Fri, 14 Aug 2020 09:49:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54646) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k6a52-00025Q-No for guix-devel@gnu.org; Fri, 14 Aug 2020 09:49:12 -0400 Received: from fsfla.org ([2001:aa8:ffed:f5f3::140]:40930) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k6a4z-0003aE-Jw for guix-devel@gnu.org; Fri, 14 Aug 2020 09:49:12 -0400 Received: from free.home (unknown [201.82.48.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by fsfla.org (Postfix) with ESMTPSA id B1F311D1D4D; Fri, 14 Aug 2020 13:49:06 +0000 (UTC) Received: from livre.home (livre.home [172.31.160.2]) by free.home (8.15.2/8.15.2) with ESMTPS id 07EDlmLw136727 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 14 Aug 2020 10:47:49 -0300 From: Alexandre Oliva To: Mark H Weaver Subject: Re: Linux-libre 5.8 and beyond Organization: Free thinker, not speaking for FSF Latin America References: <87d03vv0nm.fsf@netris.org> Date: Fri, 14 Aug 2020 10:47:48 -0300 In-Reply-To: <87d03vv0nm.fsf@netris.org> (Mark H. Weaver's message of "Wed, 12 Aug 2020 20:39:46 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 2.84 Received-SPF: none client-ip=2001:aa8:ffed:f5f3::140; envelope-from=lxoliva@fsfla.org; helo=fsfla.org X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org, Jason Self Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -1.01 X-TUID: qhiaDBS2FU5R Hello, Mark, On Aug 12, 2020, Mark H Weaver wrote: > Mark H Weaver wrote: >>> the linux-libre project periodically deletes most of its older >>> tarballs, even if there are no accidents. > Jason Self responded: >> Just FYI that git://linux-libre.fsfla.org/releases.git was created >> mainly to solve that problem. Versions are now pretty much permanent. > That's helpful, thanks. I didn't know about this. Out of curiosity, is > this git repository advertised anywhere? Not much. It was mentioned back in the announcements of 5.7-gnu and a few subsequent ones on social media; in the 5.7-gnu news entry in the Linux-libre web site, and in the documentation we wrote for Guix developers, that was sent to some of you not long ago. Though it was announced sort of widely, since this move was directed primarily at satisfying a Guix pain point, I figured I'd add it to downloads only after making sure it did address Guix's needs, so that, should it require significant changes, there wouldn't have to be much concern about backward compatibility with the current status quo. > One question: Would it solve the problem that I mentioned in my earlier > email, namely the problem of how to determine which precise commit > introduced a regression between two stable kernel releases? No. There are much better (faster and less risky) ways to tend to that requirement, see #bisecting below. >>> It may be useful for users with newer hardware devices, which are >>> not yet well supported by the latest stable release, to use an >>> arbitrary commit from either Linus' mainline git repository or some >>> other subsystem tree. >> >> The cleaning up scripts are version-specific and won't work on an >> "arbitrary commit from Linus's mainline git repository" (i.e., someone >> wanting to get today's most recent commit going into 5.9.) The scripts >> would fall over and die in such a scenario, > Okay, perhaps this was wishful thinking on my part. Yup. If you ran a deblob-check in verify mode on the resulting tarballs, you'd see how error-prone this is. You'd at least stop non-Free code from silently sneaking in and finding its way into running on users' machines. That's the *least* someone who runs the deblob-scripts on their own should do to smoke-test the result WRT *known* freedom issues. > I had hoped that the deblob scripts would typically mostly work, even > if they weren't able to do a comprehensive cleaning. I'd honestly hope for a much higher standard than that for a FSDG-compliant distro, especially one that carries the GNU mark. > I would oppose adding such a partly-cleaned kernel to Guix itself, But you don't! That's what you get when you jump the gun and use outdated cleaning up scripts, without waiting for us to verify, update and release them for a newer version. > but I wanted to enable users who need to use some other branch of > Linux on their own systems to make a best-effort cleaning. Besides the likelihood of something going wrong, that seems like a backwards goal for a distro that is not expected to as much as point users at a non-Free package. I'm sure that's not what you intend, but this arrangement, plus your mention of hurriedly getting releases out, adds up to an incentive to disable the deblobbing so as to get a faster build. I hope you'll agree that this is undesirable. As for how to speed up builds without sacrificing freedom, see below. >>> It allows us to update to a new point version (which usually >>> includes security fixes) more quickly, before the linux-libre >>> project reacts. >> >> Any attempt outrun the Linux-libre project and get updates out sooner >> is unwise. While major new kernel releases will definitely require >> updates to the cleanup scripts, even minor patched versions >> occasionally require changes too. Updating to a new version prior to >> the Linux-libre project having had time to review that new version and >> determine if any updates are needed to the scripts risks introducing >> freedom problems in the corresponding Guix version. > In my experience, the deblob scripts are very rarely changed after the > first few point releases of a stable release series. My personal experience tells me otherwise. 5.7 had only one update at .8; 5.6, at .6 and .16; 5.5, at .3, .11 and .19; 5.4, at .14, .18, .27, .34 and .44; 5.3, at .4 and .11; 5.2 at .1, .3 and .11; 5.1 at .2, .18 and .20; 5.0 at .7 and .16. What you describe was true only of 4.17, 4.10, 4.3, 3.13, 3.5, and 3.2, i.e. 6 out of the 50 major releases starting at 3.0. > I know this because I always check for updates to the deblob scripts > whenever I update linux-libre in Guix. In practice, the deblob scripts used by > Guix are never more than 1 or 2 micro versions behind the version of > Linux they are applied to. There have been 61 script updates for the 1274 4.*.*-gnu* and 5.*.*-gnu* stable releases, so Guix has shipped potentially non-FSDG code, that *would* have been flagged by deblob-check on the tarballs, at between 5% and 10% of these releases. Does that sound like a good standard for a freedom-first distro to aim for? >> The moment that the Linux-libre project determines that scripts are >> suitable is the moment that the new cleaned-up release is ready to >> publish in git and the appropriate tags will then appear in git. The >> compressed tarballs come some time later. > I prefer to avoid unnecessary delays when applying micro kernel updates, Sorry, but it doesn't look like you do. If you did, you would be taking a cleaned up tree instead of re-deblobbing it. You skip even the automated verification we do, which saves you some time, but at what price? If you waited another 30 minutes for our cleaned-up and verified tree to be available from git, you'd save yourself the 20 minutes of cleaning-up and another 20 minutes of deblob-checking the tarball for known or likely freedom issues. That sounds like a net win to me. Now, if your build machines clean up and verify much faster than ours, I'd be pretty glad to use them to get the verified commits in place so that you could use them faster. > I also consider it unwise for all of us, as a matter of habit or policy, > to trust the integrity of the computer systems used by the Linux-libre > project to perform the deblobbing. I welcome double-checking of our cleaning up at all levels, but why are you setting a higher trust standard for us than for a project known to be at odds with our shared goals, such as Linux? You don't apply the patches that went into it since the last known good release to double-check their releases, do you? For most projects, you just take their tarballs or tags and build it. For Linux-libre, you start from (untrustworthy) Linux, run the (presumed untrustworthy) cleaning up scripts, and blindly trust the result. There's no self-verification run with deblob-check, no compare with our release, nothing. If you were to test the integrity of our releases, you'd think you'd at least look at them. Starting from a known-good Linux release and applying patches to double-check the results is expensive, so it makes sense to do that only occasionally, rather than as part of every build. Deblobbing and checking the result is also expensive, so it also makes sense for you to do so only occasionally, rather than as part of every build. But the point stands that, for someone who'd rather trust no one, you're blindly trusting both Linux and Linux-libre. The former when it comes to base releases you don't check; the latter when it comes to scripts whose results you hardly even look at. Why not reduce your trust base to just Linux-libre, and treat is as a citizen of the same class as nearly every other project you build, and satisfy your trust-but-verify needs looking into what changes between one of our releases and another? #bisecting You can even take one of our releases and apply the patches that went into the next upstream stable release, and check that what you get matches our own corresponding release. Some 98% of the time, they will be exact matches. Occasionally, there will be a difference, and then you'll likely find a corresponding change in the deblobbing scripts, or a preexisting pattern that caused the change. We do this for every release, as part of our pre-release checks, and you're welcome to do so as well, and to use the resulting tree to bisect problems. You'll see the builds are much faster if you don't have to deblob every build. Now, we've long had plans to publish a cleaned-up repo with Linux git history. It would take a massive amount of work to get it started, but after that, getting new releases out might be about as fast as running a git merge. Ok, not that fast because there'd be some checking, but you get the idea. Other ideas to speed up our release process are: - enabling cleaning up in multiple concurrent processes, like 'make -j' - ditto for the deblob-check tarball verification - use faster machines for the above - monitor upstream git and fire up cleaning up automatically, and move the various manual signatures of commits, tarballs, and logs to the very end, after manual checking These would enable the git commits to be out sooner. Currently, our best case is to push a release to the git release archive about one hour after the upstream release, but we can only get that realistically when there's only one release to do. Most often, there are four to seven releases at once, and then, since we don't always react immediately (one needs to sleep occasionally ;-) and we don't get early warnings, especially about major security issues, and our main workhorse has limited capacity, we end up at about one hour per release anyway, having them all ready at about the same time. (Compression of tarballs then takes another half hour or more per release, but the releases are pushed to the git release archive long before compression is completed) -- Alexandre Oliva, happy hacker https://FSFLA.org/blogs/lxo/ Free Software Activist GNU Toolchain Engineer