From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id UICEGbhgdF8zKgAA0tVLHw (envelope-from ) for ; Wed, 30 Sep 2020 10:40:56 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 8DNYFbhgdF+tOAAAB5/wlQ (envelope-from ) for ; Wed, 30 Sep 2020 10:40:56 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id C623994014A for ; Wed, 30 Sep 2020 10:40:55 +0000 (UTC) Received: from localhost ([::1]:35480 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kNZXa-0004Yh-1I for larch@yhetil.org; Wed, 30 Sep 2020 06:40:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47830) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kNZM6-0007NO-1h for guix-patches@gnu.org; Wed, 30 Sep 2020 06:29:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46783) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kNZM5-0007Df-OC for guix-patches@gnu.org; Wed, 30 Sep 2020 06:29:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kNZM5-0001dF-LC for guix-patches@gnu.org; Wed, 30 Sep 2020 06:29:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43591] [PATCH core-updates] gnu: glibc-final: Catch all cases of a glibc user not requesting 64-bit offsets and then using readdir. Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 30 Sep 2020 10:29:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43591 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43591@debbugs.gnu.org, Marius Bakke Received: via spool by 43591-submit@debbugs.gnu.org id=B43591.16014617356261 (code B ref 43591); Wed, 30 Sep 2020 10:29:01 +0000 Received: (at 43591) by debbugs.gnu.org; 30 Sep 2020 10:28:55 +0000 Received: from localhost ([127.0.0.1]:58329 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kNZLz-0001cu-6c for submit@debbugs.gnu.org; Wed, 30 Sep 2020 06:28:55 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:42794) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kNZLw-0001ck-LF for 43591@debbugs.gnu.org; Wed, 30 Sep 2020 06:28:53 -0400 Received: from localhost (80-110-126-103.cgn.dynamic.surfer.at [80.110.126.103]) by dd26836.kasserver.com (Postfix) with ESMTPSA id E32923363688; Wed, 30 Sep 2020 12:28:49 +0200 (CEST) Date: Wed, 30 Sep 2020 12:28:21 +0200 From: Danny Milosavljevic Message-ID: <20200930122821.1471d155@scratchpost.org> In-Reply-To: <87tuvf1uet.fsf@gnu.org> References: <20200924141211.21649-1-dannym@scratchpost.org> <87363759at.fsf@gnu.org> <20200924222711.2f22281a@scratchpost.org> <87tuvm4vop.fsf@gnu.org> <87h7rg4879.fsf@gnu.org> <20200930000934.6812b7c8@scratchpost.org> <87tuvf1uet.fsf@gnu.org> X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/Hl9kSUfR.zZukas52q5QGRt"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.7 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -1.11 X-TUID: kqOnJIxr46jg --Sig_/Hl9kSUfR.zZukas52q5QGRt Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, On Wed, 30 Sep 2020 11:32:58 +0200 Ludovic Court=C3=A8s wrote: > Dropping emulated builds, or at least 32-bit emulated builds. We just > need to remove build machines from the file above. Oh. Do we have real armhf machines? (as in not aarch64) *Looks at guix-maintenance* We do. Awesome. Then sure. But just to be clear, WE MUST NOT USE aarch64 to build armhf as long as this problem isn't fixed. This problem has nothing to do with emulation. > A change in gnu-build-system would change all the derivations. I don=E2= =80=99t > think the Data Service can help us here. I want to know what actually changes in the final binaries. Surely that wo= rks somehow--guix data services or not. Basically, for each package in Guix, diff -r `~/broken-guix/pre-inst-env guix build $package` `~/fixed-guix/pr= e-inst-env guix build $package` || echo "affected: $package" but after replacing references by deduplicated content addressed references (for example if derivation A refers to files in derivation B, but derivatio= n B only changed the directory name it's in and not the content of the derivati= on, then that should not count as a diff in A. That should happen recursively). Basically ignore the directory names in /gnu/store and make new directory names that are the hash of each directory's (recursive) CONTENTS (after fixing references in that content, too, obviously)--as opposed to sources. Then diff those. If necessary, I can run that on my laptop--it will just take several weeks and miss derivations I don't have in the first place. > I have mixed feelings: fixing packages one by one doesn=E2=80=99t sound g= reat, > but OTOH setting the =E2=80=98CFLAGS=E2=80=99 environment variable global= ly can have > unexpected side effects in some cases (overriding package-specific > CFLAGS) and zero effects in other cases (for non-Autoconf packages or > badly-written =E2=80=98configure.ac=E2=80=99 files), both of which would = be hard to > detect. The latter is easy to detect since I patched dirent.h in glibc exactly for = that reason. That way, glibc WON'T let you use it wrong (except if you explicit= ly ask for it). On Guix systems, there is no legitimate reason to use it wrong in the first place. In my opinion, not having an automated way to tell us when a package is usi= ng it wrong would be not doing our due diligence--how would you know we had actually fixed the problem for good? You wouldn't know. And you wouldn't have fixed the problem for good--I can tell you that much = now. There already was an essential package, rhash, which didn't pick up the CFLAGS--and that's how I found it. It's easy. About the unexpected side effects--yes, that's right. That's why we should= get a list of diff results (see above for the command) and then manually look at the source code of those packages and their dependencies. > If we take a step back: what=E2=80=99s the problem? It means we have no trustworthy i686 packages, which means we do not have a trustworthy full source bootstrap using Mes (since that uses i686 executable to bootstrap). In practice, this problem is not so bad since the kernel on i686 has a comp= at layer that hasn't been telling us the truth for d_off, so we should be "goo= d". But philosophically, we are doing it dead wrong. Also, this won't work on armhf or any other 32 bit architecture--so there, we would be both philosophically and practically wrong. Also, the "not telling us the truth for d_off on i686" is a leaky compat la= yer. It totally DOES wind up telling us the truth sometimes (see my earlier test table)--and then we have a problem. > Then we have packages that do not > support large files; it=E2=80=99s not great but evidently we can live wit= h it. > :-) Ideally, we=E2=80=99d report it upstream when we encounter it. I really don't care about large file support. That's mostly a bonus we get while fixing this whole ordeal the right way. That said, maybe users care they actually can store a 5 GiB file on their 4000 GiB drive on armhf = :P > So to me that hints at targeted fixes: fixing key packages like CMake > (roughly what you already did) where lack of large file support can be > problematic. As long as we patch glibc's dirent.h so it tells us when we are doing stupid stuff, we can't go wrong much. So sure. As I said, the main problem is FINDING the affected packages. It's not like they'll fail building (in general). They'll just do weird stuff at runtime instead. Case in point: cmake DID NOT fail building. And it totally is broken. Everything after that is easy. --Sig_/Hl9kSUfR.zZukas52q5QGRt Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl90XcUACgkQ5xo1VCww uqXKxgf/foOlL/5z3NtiynHPzBn2e/EqziNnVXtSjZaPMdvmK4N09rvWBFLDZ3Rl dOBfMTuwhiY63BbhmmSXngYdO8USLxRoEn1GCgcYrjLt2YzfxGPaQ5xyrXV0FhDR l5UNQPNcqwk2qfK6kmmzg2qCoRG0C8+/MSt6QPGQLZ4U1JNfMuKqDFVoGR7sJW15 sLOH62x+WvIpOq8sXmHzeeguN1Aj01VaNvWWg5g0inBkqM5FcB1Q3JOxIAFnFsuf hH2YGqOfqlz/o9hIYHYOSQt9FcwtuBVurz4fac0QmKnNlVQ8Qj2UKuD26LO5BBT3 YR3MH6yIPKc8tMSU2XnnmhnE2hPS6Q== =/PXl -----END PGP SIGNATURE----- --Sig_/Hl9kSUfR.zZukas52q5QGRt--