From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:470:142:3::10]:48688) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j4dsa-0007SE-NU for guix-patches@gnu.org; Wed, 19 Feb 2020 23:56:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j4dsZ-00018F-BO for guix-patches@gnu.org; Wed, 19 Feb 2020 23:56:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:37733) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j4dsZ-000184-5m for guix-patches@gnu.org; Wed, 19 Feb 2020 23:56:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j4dsZ-0004WA-53 for guix-patches@gnu.org; Wed, 19 Feb 2020 23:56:03 -0500 Subject: [bug#39309] [PATCH WIP] gnu: add stack. Resent-Message-ID: From: Timothy Sample References: <87k14vou0u.fsf@ngyro.com> <87y2t6ayax.fsf@ngyro.com> <875zg78fcm.fsf@ngyro.com> Date: Wed, 19 Feb 2020 23:55:48 -0500 In-Reply-To: (John Soo's message of "Wed, 19 Feb 2020 14:56:23 +0000") Message-ID: <8736b57t7v.fsf@ngyro.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: John Soo Cc: 39309@debbugs.gnu.org Hi John, John Soo writes: >> At this point, it might be worth joining the Nix folks on that issue, >> and suggesting a patch to Cabal to call =E2=80=9Chsc2hs=E2=80=9D with de= duplicated >> flags. It will help that the other deduplication code was written by >> the same person who filed the bug you linked to earlier. Once the patch >> lands, we can apply it to our Cabal and then Stack will build. Yay! > > Sorry I'm a bit confused by all this. I guess if we can rely on a > patch into cabal that would be the least effort, right? Exactly. Also, sorry if I=E2=80=99m being a little unclear. I=E2=80=99ve = been kind of thinking out loud with these messages, as you=E2=80=99ll see below. ;) > Is there already a patch to deduplicate flags passed to hsc2hs? Do we > need to comment anywhere or express support? I couldn=E2=80=99t find a patch. Expressing support was definitely what I = was suggesting. I managed to find a note saying what I just said: https://github.com/NixOS/nixpkgs/issues/49206#issuecomment-470324743 I don=E2=80=99t think anything has become of it since then, though. There is one other thing we could do, but I=E2=80=99m still thinking it thr= ough. The reason we hit the limit is because of the way we use the =E2=80=9Cextra-lib-dirs=E2=80=9D and =E2=80=9Cextra-include-dirs=E2=80=9D f= lags when configuring. Every time we pass Cabal a list of directories via these flags, Cabal notes the directories in the package DB, and uses them when calling GCC. It doesn=E2=80=99t just use the directories specified when configuring a given package. It uses all the directories specified for all of that package=E2=80=99s dependencies, too. Right now, we pass in every =E2=80=9Clib=E2=80=9D and =E2=80=9Cinclude=E2= =80=9D directory from every input, including the =E2=80=9Cstandard-packages=E2=80=9D from the GNU build= system. This means that Gawk=E2=80=99s =E2=80=9Clib=E2=80=9D directory (for example= ) is included every time we configure any package. In turn, when GHC calls GCC as part of compiling a Haskell package, Gawk gets included on the command line once for every node on that package=E2=80=99s dependency tree. Packages with a = large dependency tree hit the 128K limit imposed by the kernel Linux. What=E2=80=99s funny is that not a single Haskell package even uses shared libraries from Gawk! The other thing that=E2=80=99s funny is that most Haskell packages build ju= st fine without =E2=80=9Cextra-lib-dirs=E2=80=9D and =E2=80=9Cextra-include-di= rs=E2=80=9D. Even those that call out to C code work, because Guix sets =E2=80=9CLIBRARY_PATH=E2=80=9D f= or us. Unfortunately, packages that depend on packages that call out to C code fail. This is because Guix doesn=E2=80=99t set =E2=80=9CLIBRARY_PATH=E2=80= =9D for the transitive input, but GHC still wants to make use of the shared library. This can be solved by adding back the =E2=80=9Cextra-lib-dirs=E2=80=9D and =E2=80=9Cextra-include-dirs=E2=80=9D flags as-needed. With this approach, I was able to compile Stack (the tests failed, though). The part that I=E2=80=99m still thinking through is how to make this whole system work. I would really like something that works automatically, but I don=E2=80=99t see a way to do it. This means we would have to add the flags manually to the handful of packages that need them. The part that I really don=E2=80=99t like is that we don=E2=80=99t know a package needs t= he flags until a package that depends on it fails to build. Having everything one step removed like that is not ideal. On the other hand, there are probably less than a dozen Haskell packages that need flags =E2=80=93 it=E2= =80=99s not that common to mix Haskell and C code. I will make a patch that does this and see what the damages are. This feels like a step in the right direction, because we end up with a cleaner package graph as a result. The added work and delayed feedback will be tricky, but it should only come up infrequently. What do you think? -- Tim