From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id EFYYE3fUdl+gGQAA0tVLHw (envelope-from ) for ; Fri, 02 Oct 2020 07:19:19 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id oPXFDnfUdl9mTgAAB5/wlQ (envelope-from ) for ; Fri, 02 Oct 2020 07:19:19 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 83B599402AC for ; Fri, 2 Oct 2020 07:19:18 +0000 (UTC) Received: from localhost ([::1]:51158 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kOFLZ-0002XO-Bf for larch@yhetil.org; Fri, 02 Oct 2020 03:19:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54126) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kOFLK-0002We-Kd for guix-patches@gnu.org; Fri, 02 Oct 2020 03:19:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:55949) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kOFLK-0001tM-BF for guix-patches@gnu.org; Fri, 02 Oct 2020 03:19:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kOFLK-0008In-6b for guix-patches@gnu.org; Fri, 02 Oct 2020 03:19:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43591] [PATCH core-updates] gnu: glibc-final: Catch all cases of a glibc user not requesting 64-bit offsets and then using readdir. Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 02 Oct 2020 07:19:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43591 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 43591@debbugs.gnu.org, Marius Bakke Received: via spool by 43591-submit@debbugs.gnu.org id=B43591.160162311931881 (code B ref 43591); Fri, 02 Oct 2020 07:19:02 +0000 Received: (at 43591) by debbugs.gnu.org; 2 Oct 2020 07:18:39 +0000 Received: from localhost ([127.0.0.1]:39262 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kOFKw-0008I9-HK for submit@debbugs.gnu.org; Fri, 02 Oct 2020 03:18:38 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:38776) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kOFKt-0008Hy-MM for 43591@debbugs.gnu.org; Fri, 02 Oct 2020 03:18:37 -0400 Received: from localhost (80-110-126-103.cgn.dynamic.surfer.at [80.110.126.103]) by dd26836.kasserver.com (Postfix) with ESMTPSA id A9E13336064C; Fri, 2 Oct 2020 09:18:33 +0200 (CEST) Date: Fri, 2 Oct 2020 09:18:05 +0200 From: Danny Milosavljevic Message-ID: <20201001120944.5059c650@scratchpost.org> In-Reply-To: <87o8lmv2nx.fsf@gnu.org> References: <20200924141211.21649-1-dannym@scratchpost.org> <87363759at.fsf@gnu.org> <20200924222711.2f22281a@scratchpost.org> <87tuvm4vop.fsf@gnu.org> <87h7rg4879.fsf@gnu.org> <20200930000934.6812b7c8@scratchpost.org> <87tuvf1uet.fsf@gnu.org> <20200930122821.1471d155@scratchpost.org> <87o8lmv2nx.fsf@gnu.org> X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/ZzHzVZUMIMpot5OdtPvFfu+"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.7 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -1.11 X-TUID: VFl2L21G3mXQ --Sig_/ZzHzVZUMIMpot5OdtPvFfu+ Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo, On Thu, 01 Oct 2020 09:14:10 +0200 Ludovic Court=C3=A8s wrote: > > This problem has nothing to do with emulation. =20 >=20 > Now I=E2=80=99m lost; I thought this had to do with qemu-user. I had thought so, too, a few weeks ago. But that's not the case. It's not at all related to qemu. The problem is a fundamental problem: a 64 bit value does NOT fit into a 32= bit slot. glibc uses getdents64 to get 64 bit dents and then acts all surprised and errory when it gets 64 bit dents. (also on 32 bit glibc) The same happens natively when using armhf on aarch64, without qemu-user. That's what the table I sent was all about. Calling getdents64 is not the problem--glibc has to do that, otherwise a 32= bit glibc won't work RELIABLY on a 64 bit kernel anyway. And emitting an error it does because we do not enable large file support. This table (updated after I finally compiled a guix gcc-toolchain-10 with t= he (unpatched) glibc in question on armhf): system _FILE_OFFSET_BITS off_t d_off-sizeof d_off-values --------------------------------------------------------------- x86_64 - 8 Byte 8 Byte 8 Byte i686 - 4 Byte 4 Byte 4 Byte i686 64 8 Byte 8 Byte FAIL* i686 32 4 Byte 4 Byte FAIL* i686 7 4 Byte 4 Byte 4 Byte armhf - 4 Byte 4 Byte FAIL* armhf 64 8 Byte 8 Byte 8 Byte armhf 32 4 Byte 4 Byte FAIL* armhf 7 4 Byte 4 Byte FAIL* a64armhf - 4 Byte 4 Byte FAIL* a64armhf 64 8 Byte 8 Byte 8 Byte a64armhf 32 4 Byte 4 Byte FAIL* a64armhf 7 4 Byte 4 Byte FAIL*=20 aarch64 - 8 Byte 8 Byte 8 Byte *: Using FUSE filesystem with big d_off value. None of those tests were done with qemu. They were all native. That's why I wanted access to real aarch64 machines--otherwise I could have done it with qemu on my x86_64 computer :P > I=E2=80=99m very reluctant to patching public libc headers. Well, I don't like it either--that's why it's very very careful. My patch doesn't change anything that users experience at runtime and basically just prevents developers from compiling something that is using readdir without thinking about large files first (because they HAVE TO if their programs run on a host kernel that has bigger d_off--there's no sane way around it). If they absolutely want to, they can set _FILE_OFFSET_BITS=3D32 and it will let them do it (the argument by Marius is that people might want to do that on embedded. But that means they'll sometimes have readdir fail--depending on their setup (also on 32 bit kernels). Embedded is not specially exempt from ths bug ;) ). I think that this patch is guix-specific in the sense that it happens pretty often that we do "-s i686-linux" on x86_64, "-s armhf-linux" on aarch64 and expect that to work. And there's no qemu we could even patch in those cases, because no qemu is used in the first place. > Also, it=E2=80=99s not just > =E2=80=9Cour=E2=80=9D problem, we should definitely discuss it with upstr= eam and perhaps > propose your dirent.h patch. Sure. I think 15 years of migration path to 64 bit off_t was more than eno= ugh. Now, I'd prefer if glibc made people choose _FILE_OFFSET_BITS explicitly on 32 bit. Everything else is a hack that WILL break unexpectedly. Users sti= ll can choose _FILE_OFFSET_BITS=3D32, if they want. > I=E2=80=99m also not sure what you mean by =E2=80=9Cusing it wrong=E2=80= =9D, what is =E2=80=9Cit=E2=80=9D? "it" is users calling readdir() without defining _FILE_OFFSET_BITS=3D64 in = their source file / Makefile. This causes glibc to call getdents64 and then act all surprised when it gets a 64 bit result back. > > Also, this won't work on armhf or any other 32 bit architecture--so the= re, > > we would be both philosophically and practically wrong. > > > > Also, the "not telling us the truth for d_off on i686" is a leaky compa= t layer. > > It totally DOES wind up telling us the truth sometimes (see my earlier = test > > table)--and then we have a problem. =20 >=20 > Hmm I guess I need to re-read all that, I=E2=80=99m overwhelmed. Yeah--it's understandable. I'm working on understanding and fixing this pr= oblem for a hundred hours now--it took forever for me to get to the bottom of thi= s, too. And in the beginning I, too, suspected qemu. But it's totally blameless. Nothing should be changed in qemu-user or in our qemu binfmt service. The fundamental problem is that POSIX specifies that telldir and seekdir mu= st exist, and return and take a LONG, respectively. That means that glibc has to preserve d_off it got from getdents64 (size is 64 bits), otherwise how would seekdir work? But the offset parameter of seekdir is standardized as LONG, which means th= at it won't work in the first place on 32 bit when there is either a 64 bit kerne= l or a filesystem that just happens to store bigger stuff. So glibc chose to check whether the getdents64 d_off just happens to fit in= to the LONG this time around it was called. I argue that that is insane. It would be better to always fail, or never fail--not only fail on the first d= _off that is > 2**32. When that happens is a filesystem implementation detail :P I think the assumption was that the kernel would store an actual offset into d_off. But it doesn't--it stores a hash in the case of ext4 (and probably in other cases). And in any case, even if it was an offset, that is still an unsafe way to f= ix the problem. First, someone needs to fix the POSIX standard to say "off_t", not "long". Then, distributions who want to use 32 bit userland on 64 bit kernel need to enable large files globally. That is a choice a distribution has to mak= e. Not making a choice is a choice too--the behavior will be random, and if my research in wip-file-offset-bits-64 is any indication then very fundamental things will be broken at unexpected places, and usually it DOES NOT result in a build failure (without my glibc patch). That basically means that using 32 bit native on 64 bit kernel cannot be supported in Guix if no choice is made. If choice "yes" is made, one needs to have a way to find these non-build-breaking using-readdir-wrong packages. How would an alternative way to do this look? --Sig_/ZzHzVZUMIMpot5OdtPvFfu+ Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl921C0ACgkQ5xo1VCww uqVaPgf/fIxl8BxRKuGGhS5XYcC2VjATPoqeCdJwh+ewtIhNUd//GAPVz8GLvLlW kdyReYbVfqYTeQMxvRfurh7ASV3O64fiDMrAMkv1aYmjQXLZrviOI1NgTxDs4GcL XgSv1sKFLJwuLYqXAlpE9msOkrFX6IPhnprqFfr7OIDQtM6OgBOkWoJ7yW+fU3b0 G+2P52cknDtqLBMFMDH0bGCbILNb+mrQPxYXkJzKYEydWKuErlKnclvLUI88siWX BAg1LQ88jUjc86PmaJnwHU8nBPj2HrsfabcRNOIOMniSGRRYbFqr4slodVQ7nTzk tCGBCx9D/Iuvdvll6Lmq7lFuL9c1rQ== =559R -----END PGP SIGNATURE----- --Sig_/ZzHzVZUMIMpot5OdtPvFfu+--