From mboxrd@z Thu Jan 1 00:00:00 1970 From: Carl Dong Subject: bug#37801: Possible insight into issue #30756 #include_next bug Date: Thu, 17 Oct 2019 21:56:44 +0000 Message-ID: Reply-To: Carl Dong Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:33396) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iLDmX-0001v1-37 for bug-guix@gnu.org; Thu, 17 Oct 2019 17:58:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iLDmU-0003UK-RL for bug-guix@gnu.org; Thu, 17 Oct 2019 17:58:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:40216) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iLDmU-0003UD-NM for bug-guix@gnu.org; Thu, 17 Oct 2019 17:58:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1iLDmU-0003Nh-KM for bug-guix@gnu.org; Thu, 17 Oct 2019 17:58:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: Received: from eggs.gnu.org ([2001:470:142:3::10]:33220) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iLDlU-0000cC-8n for bug-guix@gnu.org; Thu, 17 Oct 2019 17:57:03 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iLDlR-00038Z-Kp for bug-guix@gnu.org; Thu, 17 Oct 2019 17:56:59 -0400 Received: from mail4.protonmail.ch ([185.70.40.27]:19652) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1iLDlQ-00035j-Vi for bug-guix@gnu.org; Thu, 17 Oct 2019 17:56:57 -0400 List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: 37801@debbugs.gnu.org Hi all! TLDR if you just want to know what's the final patch that worked, grep for "final patch which works is the following" This is a long email, but it is only long because I want to be very clear a= nd I include a lot of snippets of code and logs. I originally wrote my NSIS package (introduced in e214a22007) on top of 7150743522, which was before the GREAT CORE-UPDATES MERGE. However, after rebasing it over the GREAT CORE-UPDATES MERGE, the package stopped working.= My investigations into this failure has revealed some insights which might be useful for other cross-compiling packages and for resolving the long-standi= ng "important" issue #30756. For some context, the NSIS package is somewhat special. It requires that bo= th native compilers and mingw-w64 cross-compilers be available, as NSIS is a program used to create installers meant to be run on Windows. My original method of achieving this in pre-GREAT CORE-UPDATES MERGE Guix c= an be see in e214a22007, whereby I added a 'fix-env phase. In that 'fix-env phase= , I would remove mingw-w64 paths from CPLUS_INCLUDE_PATH, C_INCLUDE_PATH, and LIBRARY_PATH, and put them in CROSS_CPLUS_INCLUDE_PATH, CROSS_C_INCLUDE_PAT= H, and CROSS_LIBRARY_PATH respectively. This way, the native compilers won't t= hink that the mingw-w64 paths were something they were supposed to use and get confused. As part of the GREAT CORE-UPDATES MERGE, it would seem that CPLUS_INCLUDE_P= ATH and C_INCLUDE_PATH are no longer set in the build environment. Rather, only CPATH is set. Now, you would think that the solution here is just to replace CPLUS_INCLUDE_PATH and C_INCLUDE_PATH in my 'fix-env phase with CPATH like = so: diff --git a/gnu/packages/installers.scm b/gnu/packages/installers.scm index c987254d61..9d29e32485 100644 --- a/gnu/packages/installers.scm +++ b/gnu/packages/installers.scm @@ -92,7 +92,7 @@ ;; CROSS_-prefixed version of env vars (setenv (string-append "CROSS_" env-name) (filter-delimited-string env-val ming= w-path?)))) - '("CPLUS_INCLUDE_PATH" "LIBRARY_PATH" "C_INCLUDE_= PATH")))) + '("LIBRARY_PATH" "CPATH")))) (add-before 'build 'fix-target-detection (lambda _ ;; NSIS target detection is screwed up, manually However, that does not work. Trying to build this fix results in the curiou= s error that we encounter in the long-standing "important" issue #30756 (to b= e clear, this is an error while in the `build' phase of the nsis-x86_64 packa= ge, not mingw-w64): In file included from /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross= -x86_64-w64-mingw32-7.4.0/include/c++/bits/stl_algo.h:59:0, from /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross= -x86_64-w64-mingw32-7.4.0/include/c++/algorithm:62, from Contrib/InstallOptions/InstallerOptions.cpp:23: /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.= 4.0/include/c++/cstdlib:75:15: fatal error: stdlib.h: No such file or direc= tory #include_next ^~~~~~~~~~ compilation terminated. For some context, here is the full environment-variables file for that fail= ed build: export CPATH=3D"/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_6= 4-w64-mingw32-7.4.0/include:/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zli= b-1.2.11/include:/gnu/store/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/in= clude:/gnu/store/lbip9isk25isymvnb159l115xnacb5j8-xz-5.2.4/include:/gnu/sto= re/6jdshxwdrad9mlhcqc9k0g24yw45rqf1-file-5.33/include:/gnu/store/2z9hsww76a= ag37p40671l9niq5pvvasx-gawk-5.0.1/include:/gnu/store/b5vpfzkr59bpgcsg1k9vva= d9h5rwvpgk-make-4.2.1/include:/gnu/store/3hkdiscs4910r75njbrql10znxxn7chk-b= inutils-2.32/include:/gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/= include:/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/include:/gnu= /store/7czrqpi0kwazras6pgyx0bhdn89pg0jl-linux-libre-headers-4.19.56/include= " export CROSS_CPATH=3D"/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64= -x86_64-6.0.0/include" export CROSS_LIBRARY_PATH=3D"/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mi= ngw-w64-x86_64-6.0.0/lib" export GUIX_LD_WRAPPER_ALLOW_IMPURITIES=3D"no" export GUIX_LOCPATH=3D"/gnu/store/mmqp1xqffn6qw6v88i627c2bpbq36fcy-glibc-ut= f8-locales-2.29/lib/locale" export HOME=3D"/homeless-shelter" export LC_ALL=3D"en_US.utf8" export LIBRARY_PATH=3D"/gnu/store/xiiafbq3c78kxkgv5zr54zpb8r4jz5ii-scons-py= thon2-3.0.4/lib:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_6= 4-w64-mingw32-7.4.0/lib:/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.= 2.11/lib:/gnu/store/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/lib:/gnu/s= tore/lbip9isk25isymvnb159l115xnacb5j8-xz-5.2.4/lib:/gnu/store/6jdshxwdrad9m= lhcqc9k0g24yw45rqf1-file-5.33/lib:/gnu/store/2z9hsww76aag37p40671l9niq5pvva= sx-gawk-5.0.1/lib:/gnu/store/3hkdiscs4910r75njbrql10znxxn7chk-binutils-2.32= /lib:/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib:/gnu/store/= qky1x5bb2jygy58bn6y95ygfsmpakf52-glibc-2.29-static/lib:/gnu/store/mmqp1xqff= n6qw6v88i627c2bpbq36fcy-glibc-utf8-locales-2.29/lib" export NIX_BUILD_CORES=3D"0" export NIX_BUILD_TOP=3D"/tmp/guix-build-nsis-x86_64-3.04.drv-0" export NIX_STORE=3D"/gnu/store" export OLDPWD export PATH=3D"/gnu/store/xiiafbq3c78kxkgv5zr54zpb8r4jz5ii-scons-python2-3.= 0.4/bin:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mi= ngw32-7.4.0/bin:/gnu/store/9nspnmi8prly39p3xx4plvbk9dywhf4y-binutils-cross-= x86_64-w64-mingw32-2.32/bin:/gnu/store/cnqpra8vr2l5fz00rr4yj4bp3hr00cfw-tar= -1.32/bin:/gnu/store/py3k9zla9fj3z7430v4crqj5pyrsd3qj-gzip-1.10/bin:/gnu/st= ore/l86azr7r3p5631wj3kk329jl1y1mpjgy-bzip2-1.0.6/bin:/gnu/store/lbip9isk25i= symvnb159l115xnacb5j8-xz-5.2.4/bin:/gnu/store/6jdshxwdrad9mlhcqc9k0g24yw45r= qf1-file-5.33/bin:/gnu/store/58sq8iabw3jkv0fvf95hd7sq2g4xcsnz-diffutils-3.7= /bin:/gnu/store/v76scv4n63ip08g119rczh2mrw31zwpd-patch-2.7.6/bin:/gnu/store= /g9d3wv1d68iflx57yp3mcp3k3sv8spsl-findutils-4.6.0/bin:/gnu/store/2z9hsww76a= ag37p40671l9niq5pvvasx-gawk-5.0.1/bin:/gnu/store/afmvfw1yhfal48n1kjq6bk6kcw= 8sc3db-sed-4.7/bin:/gnu/store/7iyvxhp2g3v3655zqwr6biz2h0lqv7pr-grep-3.3/bin= :/gnu/store/9kzrrccpzl6i1sfwb0drb00gi2gwk0x0-coreutils-8.31/bin:/gnu/store/= b5vpfzkr59bpgcsg1k9vvad9h5rwvpgk-make-4.2.1/bin:/gnu/store/29jhbbg1hf557x8j= 53f9sxd9imlmf02a-bash-minimal-5.0.7/bin:/gnu/store/nc5vlidpxbvngalng30nif8n= b3j7gfy2-ld-wrapper-0/bin:/gnu/store/3hkdiscs4910r75njbrql10znxxn7chk-binut= ils-2.32/bin:/gnu/store/x3jx25cd3q363mr7nbgzrhrv1vza6cf7-gcc-7.4.0/bin:/gnu= /store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/bin:/gnu/store/ahqgl4h89= xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/sbin" export PWD=3D"/tmp/guix-build-nsis-x86_64-3.04.drv-0/nsis-3.04-src" export SHLVL=3D"1" export SOURCE_DATE_EPOCH=3D"1" export TEMP=3D"/tmp/guix-build-nsis-x86_64-3.04.drv-0" export TEMPDIR=3D"/tmp/guix-build-nsis-x86_64-3.04.drv-0" export TMP=3D"/tmp/guix-build-nsis-x86_64-3.04.drv-0" export TMPDIR=3D"/tmp/guix-build-nsis-x86_64-3.04.drv-0" export out=3D"/gnu/store/wnjkbggyv1jgc2ld9yrissdsjqjvnim8-nsis-x86_64-3.04" So, to get a better understanding of what was going on, I went into the environment, and ran: $ x86_64-w64-mingw32-g++ -E -v -xc++ - < /dev/null > /dev/null Which yielded: Using built-in specs. COLLECT_GCC=3Dx86_64-w64-mingw32-g++ Target: x86_64-w64-mingw32 Configured with: Thread model: win32 gcc version 7.4.0 (GCC) COLLECT_GCC_OPTIONS=3D'-E' '-v' '-shared-libgcc' '-mtune=3Dgeneric' '-march= =3Dx86-64' /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/libexec/gcc/x86_64-w64-mingw32/7.4.0/cc1plus -E -quiet -v -U_REENTRANT= - -mtune=3Dgeneric -march=3Dx86-64 ignoring nonexistent directory "/no-gcc-local-prefix/include" ignoring nonexistent directory "/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj= -gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/../../= ../../x86_64-w64-mingw32/include" ignoring nonexistent directory "/mingw/include" #include "..." search starts here: #include <...> search starts here: /gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/include /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/include/c++ /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/include/c++/x86_64-w64-mingw32 /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/include/c++/backward /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include-fixed End of search list. COMPILER_PATH=3D/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_6= 4-w64-mingw32-7.4.0/libexec/gcc/x86_64-w64-mingw32/7.4.0/:/gnu/store/4ahrjq= b268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/libexec/gcc/x= 86_64-w64-mingw32/7.4.0/:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cr= oss-x86_64-w64-mingw32-7.4.0/libexec/gcc/x86_64-w64-mingw32/:/gnu/store/4ah= rjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x8= 6_64-w64-mingw32/7.4.0/:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cro= ss-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/ CROSS_LIBRARY_PATH=3D/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-= x86_64-6.0.0/lib/:/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86= _64-w64-mingw32-7.4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/:/gnu/store/4ahrjqb2= 68ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.4.0/lib/gcc/x86_64-= w64-mingw32/7.4.0/../../../../x86_64-w64-mingw32/lib/ COLLECT_GCC_OPTIONS=3D'-E' '-v' '-shared-libgcc' '-mtune=3Dgeneric' '-march= =3Dx86-64' So if we were to look back at the error from before, it seems that gcc-cross-x86_64-w64-mingw32-7.4.0/include/c++/cstdlib.h was trying to incl= ude the file stdlib.h in directories listed AFTER it in the list of search path= s, BUT stdlib.h actually resides in mingw-w64-x86_64-6.0.0/include, which is b= efore it in the effective list of search paths. Note that the reason mingw-w64-x86_64-6.0.0/include is in this list at all = is because of the 'fix-env phase I added, which plucked it from CPATH and plop= ped it into CROSS_CPATH. I also took a look at Ubuntu's x86_64-w64-mingw32-g++ to see how they do it= , and it would seem that mingw-w64-x86_64-6.0.0/include should be the last thing = on that list: ignoring nonexistent directory "/usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/.= ./../../../x86_64-w64-mingw32/sys-include" #include "..." search starts here: #include <...> search starts here: /usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include/c++ /usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include/c++/x86_64-w64-mingw32 /usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include/c++/backward /usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include /usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/include-fixed /usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/../../../../x86_64-w64-mingw32/i= nclude End of search list. So now we know what to fix, but how? My naive first thought was to list all= of the search paths in the order I wanted them in CROSS_CPATH, like so (preten= d the newlines didn't exist): CROSS_CPATH=3D /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.= 4.0/include/c++ :/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/include/c++/x86_64-w64-mingw32 :/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/include/c++/backward :/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include :/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include-fixed :/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/include However, that doesn't work, what DID work was instead listing the order in CROSS_CPLUS_INCLUDE_PATH and leaving CROSS_CPATH alone, resulting in someth= ing like: CROSS_CPLUS_INCLUDE_PATH=3D /gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7.= 4.0/include/c++ :/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/include/c++/x86_64-w64-mingw32 :/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/include/c++/backward :/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include :/gnu/store/4ahrjqb268ip0j0ic0l1phmixic8zfzj-gcc-cross-x86_64-w64-mingw32-7= .4.0/lib/gcc/x86_64-w64-mingw32/7.4.0/include-fixed :/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-6.0.0/include CROSS_CPATH=3D/gnu/store/rabx878fijwgg0yl2bfsx0wvw7d7isnj-mingw-w64-x86_64-= 6.0.0/include Note that I did try unsetting CROSS_CPATH, which did not work either. Ultimately, this means that the final patch which works is the following: diff --git a/gnu/packages/installers.scm b/gnu/packages/installers.scm index c987254d61..794e9a758f 100644 --- a/gnu/packages/installers.scm +++ b/gnu/packages/installers.scm @@ -92,7 +92,20 @@ ;; CROSS_-prefixed version of env vars (setenv (string-append "CROSS_" env-name) (filter-delimited-string env-val ming= w-path?)))) - '("CPLUS_INCLUDE_PATH" "LIBRARY_PATH" "C_INCLUDE_= PATH")))) + '("CPATH" "LIBRARY_PATH")) + (setenv "CROSS_CPLUS_INCLUDE_PATH" + (string-append + (string-join + (map + (lambda (x) (string-append (assoc-ref %= build-inputs "xgcc") x)) + `("/include/c++" + ,(string-append "/include/c++" "/= " ,triplet) + "/include/c++/backward" + "/lib/gcc/x86_64-w64-mingw32/7.4.= 0/include" + "/lib/gcc/x86_64-w64-mingw32/7.4.= 0/include-fixed")) + ":") + ":" + (getenv "CROSS_CPATH"))))) (add-before 'build 'fix-target-detection (lambda _ ;; NSIS target detection is screwed up, manually I'd like some feedback on 1. Whether this hack is sane or not 2. Does this apply to other packages suffering from the long-standing "important" issue #30756 3. Does this reveal something more fundamentally wrong with how we build ou= r search paths in the first place that should be addressed 4. How I might improve the readability of my final patch Cheers, Carl Dong contact@carldong.me "I fight for the users"