From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id wF8uG/g3Q19jcwAA0tVLHw (envelope-from ) for ; Mon, 24 Aug 2020 03:46:00 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id CL32Fvg3Q182IAAAbx9fmQ (envelope-from ) for ; Mon, 24 Aug 2020 03:46:00 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C949F9408DA for ; Mon, 24 Aug 2020 03:45:59 +0000 (UTC) Received: from localhost ([::1]:46000 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kA3Qk-000757-Kx for larch@yhetil.org; Sun, 23 Aug 2020 23:45:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59768) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kA3QR-00070S-Kq for guix-devel@gnu.org; Sun, 23 Aug 2020 23:45:39 -0400 Received: from fsfla.org ([217.69.89.140]:48172) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kA3QP-0005s0-CC for guix-devel@gnu.org; Sun, 23 Aug 2020 23:45:39 -0400 Received: from free.home (unknown [201.82.51.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by fsfla.org (Postfix) with ESMTPSA id 59BB31DEEDF; Mon, 24 Aug 2020 03:45:31 +0000 (UTC) Received: from livre.home (livre.home [172.31.160.2]) by free.home (8.15.2/8.15.2) with ESMTPS id 07O3jFWH117435 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 24 Aug 2020 00:45:15 -0300 From: Alexandre Oliva To: Mark H Weaver Subject: Re: Linux-libre 5.8 and beyond Organization: Free thinker, not speaking for FSF Latin America References: <87d03vv0nm.fsf@netris.org> <875z9kv41h.fsf@netris.org> Date: Mon, 24 Aug 2020 00:45:15 -0300 In-Reply-To: <875z9kv41h.fsf@netris.org> (Mark H. Weaver's message of "Sat, 15 Aug 2020 02:03:27 -0400") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 2.84 Received-SPF: none client-ip=217.69.89.140; envelope-from=lxoliva@fsfla.org; helo=fsfla.org X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/23 23:45:31 X-ACL-Warn: Detected OS = Linux 3.11 and newer X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org, Jason Self Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -1.01 X-TUID: +dYpfaib8OF+ Hello, Mark, Apologies for the delay in responding. It's been an "interesting" week. I'm breaking up what turned out to be a very very long reply into multiple posts, so as to address the various issues in separate posts, that might very well turn into separate subthreads. On Aug 15, 2020, Mark H Weaver wrote: > Alexandre Oliva wrote: >> No. There are much better (faster and less risky) ways to tend to that >> requirement, see #bisecting below. > [...] >> #bisecting >> >> You can even take one of our releases and apply the patches that went >> into the next upstream stable release, and check that what you get >> matches our own corresponding release. Some 98% of the time, they will >> be exact matches. Occasionally, there will be a difference, and then >> you'll likely find a corresponding change in the deblobbing scripts, or >> a preexisting pattern that caused the change. We do this for every >> release, as part of our pre-release checks, and you're welcome to do so >> as well, and to use the resulting tree to bisect problems. > I agree that this would be faster, but I fail to see how it's "less > risky" than running the deblob scripts meant for Linux-libre X.Y.Z on a > git checkout of the upstream stable git repo between X.Y.(Z-1) and > X.Y.Z. You're right. My mistake was failing to mention the need to compare between X.Y.Z-gnu* and the result of rebasing the X.Y.(Z-1)..X.Y.Z patches onto X.Y.(Z-1)-gnu* to enjoy the safety I had in mind. If the changes made to both ends are the same, then it's not entirely unreasonable to assume that all intermediate commits would also be properly cleaned up with the same set of changes, assuming the history in between is linear (i.e., only cherry-picks, not merges that could take a bisect to much earlier commits). Even with linear history, you might admittedly still be surprised if an intervening patch introduces something undesirable and a subsequent one reverses it. Running through deblob-check each version of each modified file that matches neither boundary would mechanically avoid nearly all such surprises. > More importantly, it's a much less straightforward thing to implement. It really isn't. Once you remove the deblobbing and point the recipe directly at the linux-libre git repo, users will be able to do the above rebasing on a local repo, and build reasonably quickly any of the commits that the local git bisect tells them to try. > In the current implementation, we get the ability to deblob arbitrary > git commits from the same stable branch essentially for free. Taking arbitrary commits from a known non-Free repo is really not something to be encouraged, given the odds of hitting freedom problems. When using the latest scripts for a stable series, odds are the procedure you suggest would work more or less reliably within that stable series, but we've recently had an example in the 5.7-gnu series in which it wouldn't. > I guess you're suggesting that I should implement a radically different > mechanism specifically for this purpose, that extracts the individual > patches from the upstream stable git repository, attempt to apply them > to the base Linux-libre release, compare that to the next Linux-libre > release, and then implement my own bisection functionality. git rebase --onto libre/vX.Y.(Z-1)-gnu nonfree/vX.Y.(Z-1) nonfree/vX.Y.Z git diff libre/vX.Y.Z-gnu git bisect start HEAD libre/vX.Y.(Z-1)-gnu > If I were to implement this, what would you suggest I do if the patches > fail to apply Look at the conflict presented by the rebase, and resolve the likely freedom issue introduced at that point. > if the result fails to match the next Linux-libre release? Identify the intervening commit where code that got cleaned up differently was introduced or removed and make the change to the deblobbing at that point; rinse and repeat. If differences remain that are not caused by the patches, it's something that changed in the scripts, possibly improving or correcting an earlier deblobbing error, e.g., something cleaned up that was found to be Free, or something that was missed in deblobbing, or a different way to clean it up. Such differences will likely be noticeable in the scripts. It's probably best to turn the difference in cleaning up into a separate commit for the purposes of the bisection, just in case it is the source of the issue being investigated. -- Alexandre Oliva, happy hacker https://FSFLA.org/blogs/lxo/ Free Software Activist GNU Toolchain Engineer