Help needed: Updating GHC to 8.4.3

Help needed: Updating GHC to 8.4.3

[Ricardo Wurmus]

[-- Attachment #1: Type: text/plain, Size: 9471 bytes --]

Hi Guix,

We need to update to GHC 8.4.3 because that’s the current version that
LTS Haskell depends on.  (Some packages I’m working on already depend on
packages that only come with GHC 8.4.3.)

Attached is a WIP patch to update GHC to 8.4.3.  (The patch renames the
existing “ghc-8” package to “ghc-8.0” and sets the default GHC at
“ghc-8.0” so that we can switch the default once it has been fully built
by the build farm.)

The problem is: it does not work, although the build phase succeeds.
Something with linking with other libraries fails.  Here’s an excerpt
from the end of the build phase:

--8<---------------cut here---------------start------------->8---
…
Warning: Distribution.Simple.Program: could not find link destinations for:
    findExecutable
Warning: Distribution.Simple.Compiler: could not find link destinations for:
    extensionToFlag
Warning: Distribution.Simple.Command: could not find link destinations for:
    FieldDescr
Warning: Distribution.Simple.Setup: could not find link destinations for:
    programDbPaths dispModSubstEntry
Warning: Distribution.PackageDescription.Configuration: could not find link destinations for:
    PDTagged PDNull
Warning: Distribution.PackageDescription.Parsec: could not find link destinations for:
    sectionizeFields
Warning: Distribution.Types.InstalledPackageInfo.FieldGrammar: could not find link destinations for:
    Basic
Warning: Distribution.InstalledPackageInfo: could not find link destinations for:
    LineNo
Warning: Distribution.Simple.PackageIndex: could not find link destinations for:
    DepUniqueKey
Warning: Distribution.Simple.BuildTarget: could not find link destinations for:
    matchInexactly findMatch None Unambiguous Ambiguous
Warning: Distribution.Simple.Build.Macros: could not find link destinations for:
    generateToolVersionMacros
Warning: Distribution.Simple.GHC: could not find link destinations for:
    gbuild GBuildMode
Warning: Distribution.Backpack.Configure: could not find link destinations for:
    PreExistingComponent
…
Warning: HscMain: could not find link destinations for:
    RenamedStuff
Warning: TcSplice: could not find link destinations for:
    annotThType
Warning: InteractiveEval: could not find link destinations for:
    RttiType
Warning: GhcMake: could not find link destinations for:
    NodeMap BuildModule LogQueue upsweep CompilationGraph
Warning: GHC: could not find link destinations for:
    OnOff LogOutput SseVersion BmiVersion modSummary tm_internals DesugaredMod WhetherHasFamInst holeUnitId
Warning: DriverBkp: could not find link destinations for:
    BkpEnv BkpM backpackProgressMsg
cd libraries && sh gen_contents_index --intree
phase `build' succeeded after 2940.0 seconds
--8<---------------cut here---------------end--------------->8---

Here’s how the check phase fails:

--8<---------------cut here---------------start------------->8---
starting phase `check'
make -C testsuite/tests CLEANUP=1 SUMMARY_FILE=../../testsuite_summary.txt
make[1]: Entering directory '/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/testsuite/tests'
"/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/inplace/bin/ghc-stage2" --make -o ../mk/ghc-config ../mk/ghc-config.hs
[1 of 1] Compiling Main             ( ../mk/ghc-config.hs, ../mk/ghc-config.o )
Linking ../mk/ghc-config ...
../mk/ghc-config "/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/inplace/bin/ghc-stage2" >"../mk/ghcconfig_tmp_guix-build-ghc-8.4.3.drv-0_ghc-8.4.3_inplace_bin_ghc-stage2.mk"; if [ $? != 0 ]; then rm -f "../mk/ghcconfig_tmp_guix-build-ghc-8.4.3.drv-0_ghc-8.4.3_inplace_bin_ghc-stage2.mk"; exit 1; fi
../mk/ghc-config: error while loading shared libraries: libgmp.so.10: cannot open shared object file: No such file or directory
Looks like you don't have timeout, building it first...
make -C ../timeout all
make[2]: Entering directory '/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/testsuite/timeout'
../mk/ghc-config "/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/inplace/bin/ghc-stage2" >"../mk/ghcconfig_tmp_guix-build-ghc-8.4.3.drv-0_ghc-8.4.3_inplace_bin_ghc-stage2.mk"; if [ $? != 0 ]; then rm -f "../mk/ghcconfig_tmp_guix-build-ghc-8.4.3.drv-0_ghc-8.4.3_inplace_bin_ghc-stage2.mk"; exit 1; fi
../mk/ghc-config: error while loading shared libraries: libgmp.so.10: cannot open shared object file: No such file or directory
rm -f -f TimeMe.o TimeMe.hi TimeMe TimeMe.exe
python3 calibrate '/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/inplace/bin/ghc-stage1' > calibrate.out
TimeMe: error while loading shared libraries: libgmp.so.10: cannot open shared object file: No such file or directory
rm -rf install-inplace
mkdir install-inplace
mkdir install-inplace/bin
cp timeout.py install-inplace/bin/timeout.py
echo '#!/gnu/store/rbrandv7anzjxqkr40d7fkanzssslk4b-bash-minimal-4.4.19/bin/sh' > install-inplace/bin/timeout
echo 'exec "python3" $0.py "$@"' >> install-inplace/bin/timeout
chmod +x install-inplace/bin/timeout
make[2]: Leaving directory '/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/testsuite/timeout'
PYTHON="python3" "python3" ../driver/runtests.py  -e "ghc_compiler_always_flags='-dcore-lint -dcmm-lint -no-user- -rtsopts  -dno-debug-output'" -e config.compiler_debugged= -e ghc_with_native_codegen=0 -e config.have_vanilla=True -e config.have_dynamic=True -e config.have_profiling=True -e ghc_with_threaded_rts=0 -e ghc_with_dynamic_rts=0 -e config.have_interp=True -e config.unregisterised=False -e config.ghc_dynamic_by_default=False -e config.ghc_dynamic=False -e ghc_with_smp=0 -e ghc_with_llvm=0 -e windows=False -e darwin=False -e config.in_tree_compiler=True -e config.cleanup=True -e config.local=True --rootdir=. --config-file=../config/ghc -e 'config.confdir="../config"' -e 'config.platform=""' -e 'config.os=""' -e 'config.arch=""' -e 'config.wordsize=""' -e 'config.timeout=int() or config.timeout' -e 'config.exeext=""' -e 'config.top="/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/testsuite"' --config 'compiler="/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/inplace/bin/ghc-stage2"' --config 'ghc_pkg="/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/inplace/bin/ghc-pkg"' --config 'haddock="/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/inplace/bin/haddock"' --config 'hp2ps="/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/inplace/bin/hp2ps"' --config 'hpc="/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/inplace/bin/hpc"' --config 'gs="gs"' --config 'timeout_prog="../timeout/install-inplace/bin/timeout"' -e "config.stage=" --summary-file "../../testsuite_summary.txt"   --rootdir=../../libraries/array/tests  --rootdir=../../libraries/base/tests  --rootdir=../../libraries/binary/tests  --rootdir=../../libraries/bytestring/tests  --rootdir=../../libraries/containers/tests  --rootdir=../../libraries/deepseq/tests  --rootdir=../../libraries/directory/tests  --rootdir=../../libraries/filepath/tests  --rootdir=../../libraries/ghc-compact/tests  --rootdir=../../libraries/ghc-prim/tests  --rootdir=../../libraries/haskeline/tests  --rootdir=../../libraries/hpc/tests  --rootdir=../../libraries/pretty/tests  --rootdir=../../libraries/process/tests  --rootdir=../../libraries/stm/tests  --rootdir=../../libraries/template-haskell/tests  --rootdir=../../libraries/text/tests  --rootdir=../../libraries/unix/tests \
         \
         \
         \
         \
         \
         \

Traceback (most recent call last):
  File "../driver/runtests.py", line 65, in <module>
    exec(e)
  File "<string>", line 1
    config.compiler_debugged=
                            ^
SyntaxError: invalid syntax
make[1]: *** [../mk/test.mk:303: test] Error 1
make[1]: Leaving directory '/tmp/guix-build-ghc-8.4.3.drv-0/ghc-8.4.3/testsuite/tests'
make: *** [Makefile:223: test] Error 2
--8<---------------cut here---------------end--------------->8---

It says “invalid syntax” because the value for “$(GhcDebugged)” is
unavailable, because the ghc-config stuff above failed.

GHC 8.0 had been patched with
"ghc-dont-pass-linker-flags-via-response-files.patch" to avoid using
response files with the linker, because our ld-wrapper doesn’t seem to
behave right in some edge case that GHC depends on.  I tried porting the
patch to GHC 8.4.3 by applying this snippet:

--8<---------------cut here---------------start------------->8---
      ;; FIXME: Our ld-wrapper does not seem to fully support response files,
      ;; thus breaking the build.  We use "runSomethingFiltered" instead of
      ;; "runSomethingResponseFile" to work around this problem.  Note the
      ;; slightly different type.
      (snippet
       '(begin
          (substitute* "compiler/main/SysTools/Tasks.hs"
            (("runSomethingResponseFile dflags cc_filter \"C Compiler\" p args2 mb_env")
             "runSomethingFiltered dflags cc_filter \"C Compiler\" p args2 Nothing mb_env")
            (("runSomethingResponseFile dflags ld_filter \"Linker\" p args2 mb_env")
             "runSomethingFiltered dflags ld_filter \"Linker\" p args2 Nothing mb_env"))
          #t))
--8<---------------cut here---------------end--------------->8---

We are using “runSomethingFiltered” instead of
“runSomethingResponseFile”, but I suppose something about the flags
still isn’t quite right.

Any ideas?

Unfortunately, Nixpkgs wasn’t of much help in this case.  AFAIU they
modified their ld wrapper so that no patch of GHC is required.

--
Ricardo


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-WIP-Add-ghc-8.4.3.patch --]
[-- Type: text/x-patch, Size: 5787 bytes --]

From 7485bad6a8b8d826c8c8e8f818027c6a51433971 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
Date: Fri, 17 Aug 2018 16:55:43 +0200
Subject: [PATCH] WIP Add ghc 8.4.3

---
 gnu/packages/haskell.scm | 113 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 111 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/haskell.scm b/gnu/packages/haskell.scm
index 7347c8753..d9e65ee4a 100644
--- a/gnu/packages/haskell.scm
+++ b/gnu/packages/haskell.scm
@@ -320,7 +320,7 @@ top of CLISP.")
 interactive environment for the functional language Haskell.")
     (license license:bsd-3)))
 
-(define-public ghc-8
+(define-public ghc-8.0
   (package
     (name "ghc")
     (version "8.0.2")
@@ -432,7 +432,116 @@ interactive environment for the functional language Haskell.")
 interactive environment for the functional language Haskell.")
     (license license:bsd-3)))
 
-(define-public ghc ghc-8)
+(define-public ghc-8
+  (package (inherit ghc-8.0)
+    (name "ghc")
+    (version "8.4.3")
+    (source
+     (origin
+      (method url-fetch)
+      (uri (string-append "https://www.haskell.org/ghc/dist/"
+                          version "/" name "-" version "-src.tar.xz"))
+      (sha256
+       (base32 "1mk046vb561j75saz05rghhbkps46ym5aci4264dwc2qk3dayixf"))
+      (modules '((guix build utils)))
+      ;; FIXME: Our ld-wrapper does not seem to fully support response files,
+      ;; thus breaking the build.  We use "runSomethingFiltered" instead of
+      ;; "runSomethingResponseFile" to work around this problem.  Note the
+      ;; slightly different type.
+      (snippet
+       '(begin
+          (substitute* "compiler/main/SysTools/Tasks.hs"
+            (("runSomethingResponseFile dflags cc_filter \"C Compiler\" p args2 mb_env")
+             "runSomethingFiltered dflags cc_filter \"C Compiler\" p args2 Nothing mb_env")
+            (("runSomethingResponseFile dflags ld_filter \"Linker\" p args2 mb_env")
+             "runSomethingFiltered dflags ld_filter \"Linker\" p args2 Nothing mb_env"))
+          #t))))
+    (inputs
+     `(("gmp" ,gmp)
+       ("ncurses" ,ncurses)
+       ("libffi" ,libffi)))
+    (native-inputs
+     `(("perl" ,perl)
+       ("python" ,python)               ; for tests
+       ("ghostscript" ,ghostscript)     ; for tests
+       ;; GHC 8.4.3 is built with GHC 8.
+       ("ghc-bootstrap" ,ghc-8.0)
+       ("ghc-testsuite"
+        ,(origin
+           (method url-fetch)
+           (uri (string-append
+                 "https://www.haskell.org/ghc/dist/"
+                 version "/" name "-" version "-testsuite.tar.xz"))
+           (sha256
+            (base32 "1z55b1z0m3plqd2d1ks6w5wvx7igm7zsk3i4v7cms003z0as0hzz"))))))
+    (arguments
+     `(#:test-target "test"
+       ;; We get a smaller number of test failures by disabling parallel test
+       ;; execution.
+       #:parallel-tests? #f
+
+       ;; The DSOs use $ORIGIN to refer to each other, but (guix build
+       ;; gremlin) doesn't support it yet, so skip this phase.
+       #:validate-runpath? #f
+
+       ;; Don't pass --build=<triplet>, because the configure script
+       ;; auto-detects slightly different triplets for --host and --target and
+       ;; then complains that they don't match.
+       #:build #f
+
+       #:configure-flags
+       (list
+        (string-append "--with-gmp-libraries="
+                       (assoc-ref %build-inputs "gmp") "/lib")
+        (string-append "--with-gmp-includes="
+                       (assoc-ref %build-inputs "gmp") "/include")
+        "--with-system-libffi"
+        (string-append "--with-ffi-libraries="
+                       (assoc-ref %build-inputs "libffi") "/lib")
+        (string-append "--with-ffi-includes="
+                       (assoc-ref %build-inputs "libffi") "/include")
+        (string-append "--with-curses-libraries="
+                       (assoc-ref %build-inputs "ncurses") "/lib")
+        (string-append "--with-curses-includes="
+                       (assoc-ref %build-inputs "ncurses") "/include"))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'unpack-testsuite
+           (lambda* (#:key inputs #:allow-other-keys)
+             (invoke "tar" "xvf"
+                     (assoc-ref inputs "ghc-testsuite")
+                     "--strip-components=1")
+             #t))
+         (add-after 'unpack-testsuite 'fix-shell-wrappers
+           (lambda _
+             (substitute* '("driver/ghci/ghc.mk"
+                            "utils/mkdirhier/ghc.mk"
+                            "rules/shell-wrapper.mk")
+               (("echo '#!/bin/sh'")
+                (format #f "echo '#!~a'" (which "sh"))))
+             #t))
+         (add-before 'build 'fix-references
+           (lambda _
+             (substitute* '("testsuite/timeout/Makefile"
+                            "testsuite/timeout/timeout.py"
+                            "testsuite/timeout/timeout.hs"
+                            "testsuite/tests/programs/life_space_leak/life.test"
+                            ;; libraries
+                            "libraries/process/System/Process/Posix.hs"
+                            "libraries/process/tests/process001.hs"
+                            "libraries/process/tests/process002.hs"
+                            "libraries/unix/cbits/execvpe.c")
+               (("/bin/sh") (which "sh"))
+               (("/bin/ls") (which "ls"))
+               (("/bin/rm") "rm"))
+             #t))
+         (add-before 'build 'fix-environment
+           (lambda _
+             (unsetenv "GHC_PACKAGE_PATH")
+             (setenv "CONFIG_SHELL" (which "bash"))
+             #t)))))))
+
+(define-public ghc ghc-8.0)
 
 (define-public ghc-hostname
   (package
-- 
2.18.0

Re: Help needed: Updating GHC to 8.4.3

[Ludovic Courtès]

Hello,

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> GHC 8.0 had been patched with
> "ghc-dont-pass-linker-flags-via-response-files.patch" to avoid using
> response files with the linker, because our ld-wrapper doesn’t seem to
> behave right in some edge case that GHC depends on.  I tried porting the
> patch to GHC 8.4.3 by applying this snippet:

I think this patch predates the addition of support for response files
in ld-wrapper.  That support is not entirely faithful (see the FIXME in
ld-wrapper.in), but I think it’s good enough for GHC.

Could you maybe try removing the patch and see if it works better?

HTH,
Ludo’.

Re: Help needed: Updating GHC to 8.4.3

[Ricardo Wurmus]

Hi Ludo,

> Hello,
>
> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>
>> GHC 8.0 had been patched with
>> "ghc-dont-pass-linker-flags-via-response-files.patch" to avoid using
>> response files with the linker, because our ld-wrapper doesn’t seem to
>> behave right in some edge case that GHC depends on.  I tried porting the
>> patch to GHC 8.4.3 by applying this snippet:
>
> I think this patch predates the addition of support for response files
> in ld-wrapper.  That support is not entirely faithful (see the FIXME in
> ld-wrapper.in), but I think it’s good enough for GHC.
>
> Could you maybe try removing the patch and see if it works better?

I did try to build GHC 8.4.3 without any patches first, but this failed
(with the same errors and warnings when trying to set up the tests).
Only then did I try to port the patch from 8.0.x to 8.4.3.

-- 
Ricardo

Re: Help needed: Updating GHC to 8.4.3

[Timothy Sample]

Hi Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi Ludo,
>
>> Hello,
>>
>> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>>
>>> GHC 8.0 had been patched with
>>> "ghc-dont-pass-linker-flags-via-response-files.patch" to avoid using
>>> response files with the linker, because our ld-wrapper doesn’t seem to
>>> behave right in some edge case that GHC depends on.  I tried porting the
>>> patch to GHC 8.4.3 by applying this snippet:
>>
>> I think this patch predates the addition of support for response files
>> in ld-wrapper.  That support is not entirely faithful (see the FIXME in
>> ld-wrapper.in), but I think it’s good enough for GHC.
>>
>> Could you maybe try removing the patch and see if it works better?
>
> I did try to build GHC 8.4.3 without any patches first, but this failed
> (with the same errors and warnings when trying to set up the tests).
> Only then did I try to port the patch from 8.0.x to 8.4.3.

I had some success with GHC 8.4.3.  It turns out that the warnings exist
for 8.0.2 as well.  I’m not sure what they are about, but they are not a
new problem.

The new problem is that the stage-2 GHC compiler is not using the
ld-wrapper.  This is due to some changes to their Autoconf scripts which
add “-fuse-ld=bfd” to the later-stage compilers’ “gcc” invocations by
default.  Since “ld.bfd” does not point to the ld-wrapper, the wrapper
gets bypassed.

There are two easy solutions to this:

  1. Use the “--disable-ld-override” configure flag.
  2. Set the “LD” environment variable.

I got the package to build using option 1.  However, it didn’t work
because it expected all the build tools (“gcc”, “ld”, etc.) to be in
“PATH”.  They traded “AC_PATH_” for “AC_CHECK_”, setting the variables
to just the command names.  Nix uses option 2 and sets explicit paths
for all the tools via environment variables.  I think we have to do the
same.

Also, I didn’t need the response files patch, and the
“native-search-paths” field needs to be updated.

Hope that helps!


-- Tim

Re: Help needed: Updating GHC to 8.4.3

[Timothy Sample]

[-- Attachment #1: Type: text/plain, Size: 3011 bytes --]

Hi Again,

I’ve attached an updated patch that builds a working GHC 8.4.3 (that is,
it compiles “hello.hs” in a pure environment).  It might need a bit more
work yet.  See notes below.

Timothy Sample <samplet@ngyro.com> writes:

> Hi Ricardo,
>
> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> Hi Ludo,
>>
>>> Hello,
>>>
>>> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:
>>>
>>>> GHC 8.0 had been patched with
>>>> "ghc-dont-pass-linker-flags-via-response-files.patch" to avoid using
>>>> response files with the linker, because our ld-wrapper doesn’t seem to
>>>> behave right in some edge case that GHC depends on.  I tried porting the
>>>> patch to GHC 8.4.3 by applying this snippet:
>>>
>>> I think this patch predates the addition of support for response files
>>> in ld-wrapper.  That support is not entirely faithful (see the FIXME in
>>> ld-wrapper.in), but I think it’s good enough for GHC.
>>>
>>> Could you maybe try removing the patch and see if it works better?
>>
>> I did try to build GHC 8.4.3 without any patches first, but this failed
>> (with the same errors and warnings when trying to set up the tests).
>> Only then did I try to port the patch from 8.0.x to 8.4.3.
>
> I had some success with GHC 8.4.3.  It turns out that the warnings exist
> for 8.0.2 as well.  I’m not sure what they are about, but they are not a
> new problem.
>
> The new problem is that the stage-2 GHC compiler is not using the
> ld-wrapper.  This is due to some changes to their Autoconf scripts which
> add “-fuse-ld=bfd” to the later-stage compilers’ “gcc” invocations by
> default.  Since “ld.bfd” does not point to the ld-wrapper, the wrapper
> gets bypassed.
>
> There are two easy solutions to this:
>
>   1. Use the “--disable-ld-override” configure flag.
>   2. Set the “LD” environment variable.
>
> I got the package to build using option 1.  However, it didn’t work
> because it expected all the build tools (“gcc”, “ld”, etc.) to be in
> “PATH”.  They traded “AC_PATH_” for “AC_CHECK_”, setting the variables
> to just the command names.  Nix uses option 2 and sets explicit paths
> for all the tools via environment variables.  I think we have to do the
> same.

I followed Nix’s example, and set environment variables.  I didn’t set
the same ones as Nix, since some of them are Windows- or Clang-specific,
and one of them (“ar”) didn’t work.

These tools need to be used by the resulting compiler, so I made sure
that they are “inputs” rather than “native-inputs”.  I don’t know if GHC
can be cross compiled, but I think something like this is necessary for
it.  To be honest, I’m not sure I did this correctly, so do take a
careful look.

> Also, I didn’t need the response files patch, and the
> “native-search-paths” field needs to be updated.

I added the “native-search-paths” field.

> Hope that helps!
>
>
> -- Tim


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: GHC 8.4.3 --]
[-- Type: text/x-patch, Size: 6372 bytes --]

diff --git a/gnu/packages/haskell.scm b/gnu/packages/haskell.scm
index 7347c8753..2dd1d0c66 100644
--- a/gnu/packages/haskell.scm
+++ b/gnu/packages/haskell.scm
@@ -33,6 +33,7 @@
 
 (define-module (gnu packages haskell)
   #:use-module (gnu packages)
+  #:use-module (gnu packages base)
   #:use-module (gnu packages bootstrap)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
@@ -320,7 +321,7 @@ top of CLISP.")
 interactive environment for the functional language Haskell.")
     (license license:bsd-3)))
 
-(define-public ghc-8
+(define-public ghc-8.0
   (package
     (name "ghc")
     (version "8.0.2")
@@ -432,7 +433,129 @@ interactive environment for the functional language Haskell.")
 interactive environment for the functional language Haskell.")
     (license license:bsd-3)))
 
-(define-public ghc ghc-8)
+(define-public ghc-8
+  (package
+    (inherit ghc-8.0)
+    (name "ghc")
+    (version "8.4.3")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://www.haskell.org/ghc/dist/"
+                           version "/" name "-" version "-src.tar.xz"))
+       (sha256
+        (base32
+         "1mk046vb561j75saz05rghhbkps46ym5aci4264dwc2qk3dayixf"))))
+    (inputs
+     `(("gmp" ,gmp)
+       ("ncurses" ,ncurses)
+       ("libffi" ,libffi)
+       ("target-binutils" ,binutils)
+       ("target-gcc" ,gcc)
+       ("target-ld-wrapper" ,(make-ld-wrapper "ld-wrapper"
+                                              #:binutils binutils))))
+    (native-inputs
+     `(("perl" ,perl)
+       ("python" ,python)            ; for tests
+       ("ghostscript" ,ghostscript)  ; for tests
+       ;; GHC 8.4.3 is built with GHC 8.
+       ("ghc-bootstrap" ,ghc-8.0)
+       ("ghc-testsuite"
+        ,(origin
+           (method url-fetch)
+           (uri (string-append
+                 "https://www.haskell.org/ghc/dist/"
+                 version "/" name "-" version "-testsuite.tar.xz"))
+           (sha256
+            (base32
+             "1z55b1z0m3plqd2d1ks6w5wvx7igm7zsk3i4v7cms003z0as0hzz"))))))
+    (arguments
+     `(#:test-target "test"
+       ;; We get a smaller number of test failures by disabling parallel test
+       ;; execution.
+       #:parallel-tests? #f
+
+       ;; The DSOs use $ORIGIN to refer to each other, but (guix build
+       ;; gremlin) doesn't support it yet, so skip this phase.
+       #:validate-runpath? #f
+
+       ;; Don't pass --build=<triplet>, because the configure script
+       ;; auto-detects slightly different triplets for --host and --target and
+       ;; then complains that they don't match.
+       #:build #f
+
+       #:configure-flags
+       (list
+        (string-append "--with-gmp-libraries="
+                       (assoc-ref %build-inputs "gmp") "/lib")
+        (string-append "--with-gmp-includes="
+                       (assoc-ref %build-inputs "gmp") "/include")
+        "--with-system-libffi"
+        (string-append "--with-ffi-libraries="
+                       (assoc-ref %build-inputs "libffi") "/lib")
+        (string-append "--with-ffi-includes="
+                       (assoc-ref %build-inputs "libffi") "/include")
+        (string-append "--with-curses-libraries="
+                       (assoc-ref %build-inputs "ncurses") "/lib")
+        (string-append "--with-curses-includes="
+                       (assoc-ref %build-inputs "ncurses") "/include"))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'unpack-testsuite
+           (lambda* (#:key inputs #:allow-other-keys)
+             (invoke "tar" "xvf"
+                     (assoc-ref inputs "ghc-testsuite")
+                     "--strip-components=1")
+             #t))
+         (add-after 'unpack-testsuite 'fix-shell-wrappers
+           (lambda _
+             (substitute* '("driver/ghci/ghc.mk"
+                            "utils/mkdirhier/ghc.mk"
+                            "rules/shell-wrapper.mk")
+               (("echo '#!/bin/sh'")
+                (format #f "echo '#!~a'" (which "sh"))))
+             #t))
+         (add-before 'configure 'set-target-programs
+           (lambda* (#:key inputs #:allow-other-keys)
+             (let ((binutils (assoc-ref inputs "target-binutils"))
+                   (gcc (assoc-ref inputs "target-gcc"))
+                   (ld-wrapper (assoc-ref inputs "target-ld-wrapper")))
+               (setenv "CC" (string-append gcc "/bin/gcc"))
+               (setenv "CXX" (string-append gcc "/bin/g++"))
+               (setenv "LD" (string-append ld-wrapper "/bin/ld"))
+               (setenv "NM" (string-append binutils "/bin/nm"))
+               (setenv "RANLIB" (string-append binutils "/bin/ranlib"))
+               (setenv "STRIP" (string-append binutils "/bin/strip"))
+               ;; The 'ar' command does not follow the same pattern.
+               (setenv "fp_prog_ar" (string-append binutils "/bin/ar")))))
+         (add-before 'build 'fix-references
+           (lambda _
+             (substitute* '("testsuite/timeout/Makefile"
+                            "testsuite/timeout/timeout.py"
+                            "testsuite/timeout/timeout.hs"
+                            "testsuite/tests/programs/life_space_leak/life.test"
+                            ;; libraries
+                            "libraries/process/System/Process/Posix.hs"
+                            "libraries/process/tests/process001.hs"
+                            "libraries/process/tests/process002.hs"
+                            "libraries/unix/cbits/execvpe.c")
+               (("/bin/sh") (which "sh"))
+               (("/bin/ls") (which "ls"))
+               (("/bin/rm") "rm"))
+             #t))
+         (add-before 'build 'fix-environment
+           (lambda _
+             (unsetenv "GHC_PACKAGE_PATH")
+             (setenv "CONFIG_SHELL" (which "bash"))
+             #t)))))
+    (native-search-paths (list (search-path-specification
+                                (variable "GHC_PACKAGE_PATH")
+                                (files (list
+                                        (string-append "lib/ghc-" version)))
+                                (file-pattern ".*\\.conf\\.d$")
+                                (file-type 'directory))))))
+
+(define-public ghc ghc-8.0)
 
 (define-public ghc-hostname
   (package

Re: Help needed: Updating GHC to 8.4.3

[Timothy Sample]

Hello,

Timothy Sample <samplet@ngyro.com> writes:

> Hi Again,
>
> I’ve attached an updated patch that builds a working GHC 8.4.3 (that is,
> it compiles “hello.hs” in a pure environment).  It might need a bit more
> work yet.  See notes below.

It turns out that it wasn’t a pure environment.  I was running Guix on
top of Debian, and GCC was presumably finding “as” and its other
run-time dependencies in hard-coded locations.  It does not work in a
pure environment on top of GuixSD.  (Next time I will use “-C” instead
of “--pure”!)

> Timothy Sample <samplet@ngyro.com> writes:
>
> [...]
>
>> I had some success with GHC 8.4.3.  It turns out that the warnings exist
>> for 8.0.2 as well.  I’m not sure what they are about, but they are not a
>> new problem.
>>
>> The new problem is that the stage-2 GHC compiler is not using the
>> ld-wrapper.  This is due to some changes to their Autoconf scripts which
>> add “-fuse-ld=bfd” to the later-stage compilers’ “gcc” invocations by
>> default.  Since “ld.bfd” does not point to the ld-wrapper, the wrapper
>> gets bypassed.
>>
>> There are two easy solutions to this:
>>
>>   1. Use the “--disable-ld-override” configure flag.
>>   2. Set the “LD” environment variable.
>>
>> I got the package to build using option 1.  However, it didn’t work
>> because it expected all the build tools (“gcc”, “ld”, etc.) to be in
>> “PATH”.  They traded “AC_PATH_” for “AC_CHECK_”, setting the variables
>> to just the command names.  Nix uses option 2 and sets explicit paths
>> for all the tools via environment variables.  I think we have to do the
>> same.
>
> I followed Nix’s example, and set environment variables.  I didn’t set
> the same ones as Nix, since some of them are Windows- or Clang-specific,
> and one of them (“ar”) didn’t work.
>
> These tools need to be used by the resulting compiler, so I made sure
> that they are “inputs” rather than “native-inputs”.  I don’t know if GHC
> can be cross compiled, but I think something like this is necessary for
> it.  To be honest, I’m not sure I did this correctly, so do take a
> careful look.

I think I may have gone overboard here.  It turns out GHC 8.0.2 does not
work in a pure environment, either.  It finds all of its immediate
dependencies (like GCC), but the transitive dependencies (like GCC
calling “as”) do not work.  After getting into the weeds with GCC’s “-B”
flag, I realized that even our GCC package doesn’t work in a pure
environment.  It doesn’t know how to find “as” in the store.  That’s why
the “gcc-toolchain” package exists.

To summarize, there are three problems with the original patch:

  1. it uses “ld.bfd” instead of “ld” (skipping the wrapper);
  2. it references native tools instead of target tools; and
  3. it requires certain tools to be available from “PATH”.

Problems 2 and 3 affect GHC 8.0.2 too.  Problem 1 is new.  At this
point, I’m thinking that using either “--disable-ld-override” or “LD=ld”
might be the most reasonable approach (they do the same thing, but the
second one is clearer IMO).  It means that GHC will look for GCC and
friends in “PATH”, but this is how GCC itself works currently with
“binutils”.  It has the added benefit that these will likely be the
tools for the target platform.

WDYT?


-- Tim

> [...]

Re: Help needed: Updating GHC to 8.4.3

[Ricardo Wurmus]

Hey Tim,

thank you so much for helping with GHC!

> To summarize, there are three problems with the original patch:
>
>   1. it uses “ld.bfd” instead of “ld” (skipping the wrapper);
>   2. it references native tools instead of target tools; and
>   3. it requires certain tools to be available from “PATH”.
>
> Problems 2 and 3 affect GHC 8.0.2 too.  Problem 1 is new.  At this
> point, I’m thinking that using either “--disable-ld-override” or “LD=ld”
> might be the most reasonable approach (they do the same thing, but the
> second one is clearer IMO).  It means that GHC will look for GCC and
> friends in “PATH”, but this is how GCC itself works currently with
> “binutils”.  It has the added benefit that these will likely be the
> tools for the target platform.
>
> WDYT?

I set the LD environment variable as suggested and the package build is
getting much further.  Thank you!

I’m not too concerned about the 2nd and 3rd problems, because at least
GHC 8.4.3 will likely be good enough to build all of our Haskell
packages.

We should record the other problems as bugs on bug-guix@gnu.org.

--
Ricardo