Hi Ludo, On Wed, 30 Sep 2020 11:32:58 +0200 Ludovic Courtès wrote: > Dropping emulated builds, or at least 32-bit emulated builds. We just > need to remove build machines from the file above. Oh. Do we have real armhf machines? (as in not aarch64) *Looks at guix-maintenance* We do. Awesome. Then sure. But just to be clear, WE MUST NOT USE aarch64 to build armhf as long as this problem isn't fixed. This problem has nothing to do with emulation. > A change in gnu-build-system would change all the derivations. I don’t > think the Data Service can help us here. I want to know what actually changes in the final binaries. Surely that works somehow--guix data services or not. Basically, for each package in Guix, diff -r `~/broken-guix/pre-inst-env guix build $package` `~/fixed-guix/pre-inst-env guix build $package` || echo "affected: $package" but after replacing references by deduplicated content addressed references (for example if derivation A refers to files in derivation B, but derivation B only changed the directory name it's in and not the content of the derivation, then that should not count as a diff in A. That should happen recursively). Basically ignore the directory names in /gnu/store and make new directory names that are the hash of each directory's (recursive) CONTENTS (after fixing references in that content, too, obviously)--as opposed to sources. Then diff those. If necessary, I can run that on my laptop--it will just take several weeks and miss derivations I don't have in the first place. > I have mixed feelings: fixing packages one by one doesn’t sound great, > but OTOH setting the ‘CFLAGS’ environment variable globally can have > unexpected side effects in some cases (overriding package-specific > CFLAGS) and zero effects in other cases (for non-Autoconf packages or > badly-written ‘configure.ac’ files), both of which would be hard to > detect. The latter is easy to detect since I patched dirent.h in glibc exactly for that reason. That way, glibc WON'T let you use it wrong (except if you explicitly ask for it). On Guix systems, there is no legitimate reason to use it wrong in the first place. In my opinion, not having an automated way to tell us when a package is using it wrong would be not doing our due diligence--how would you know we had actually fixed the problem for good? You wouldn't know. And you wouldn't have fixed the problem for good--I can tell you that much now. There already was an essential package, rhash, which didn't pick up the CFLAGS--and that's how I found it. It's easy. About the unexpected side effects--yes, that's right. That's why we should get a list of diff results (see above for the command) and then manually look at the source code of those packages and their dependencies. > If we take a step back: what’s the problem? It means we have no trustworthy i686 packages, which means we do not have a trustworthy full source bootstrap using Mes (since that uses i686 executable to bootstrap). In practice, this problem is not so bad since the kernel on i686 has a compat layer that hasn't been telling us the truth for d_off, so we should be "good". But philosophically, we are doing it dead wrong. Also, this won't work on armhf or any other 32 bit architecture--so there, we would be both philosophically and practically wrong. Also, the "not telling us the truth for d_off on i686" is a leaky compat layer. It totally DOES wind up telling us the truth sometimes (see my earlier test table)--and then we have a problem. > Then we have packages that do not > support large files; it’s not great but evidently we can live with it. > :-) Ideally, we’d report it upstream when we encounter it. I really don't care about large file support. That's mostly a bonus we get while fixing this whole ordeal the right way. That said, maybe users care they actually can store a 5 GiB file on their 4000 GiB drive on armhf :P > So to me that hints at targeted fixes: fixing key packages like CMake > (roughly what you already did) where lack of large file support can be > problematic. As long as we patch glibc's dirent.h so it tells us when we are doing stupid stuff, we can't go wrong much. So sure. As I said, the main problem is FINDING the affected packages. It's not like they'll fail building (in general). They'll just do weird stuff at runtime instead. Case in point: cmake DID NOT fail building. And it totally is broken. Everything after that is easy.