From mboxrd@z Thu Jan 1 00:00:00 1970 From: Danny Milosavljevic Subject: Re: heads-up: Haskell updates Date: Wed, 14 Feb 2018 23:47:21 +0100 Message-ID: <20180214234721.4e9fe198@scratchpost.org> References: <87r2ppjbst.fsf@elephly.net> <873723pfya.fsf@netris.org> <871shn8jm5.fsf@elephly.net> <87zi4b744f.fsf@elephly.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45641) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1em5pt-00065U-UI for guix-devel@gnu.org; Wed, 14 Feb 2018 17:47:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1em5pq-0004g3-Qa for guix-devel@gnu.org; Wed, 14 Feb 2018 17:47:33 -0500 Received: from dd26836.kasserver.com ([85.13.145.193]:55924) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1em5pq-0004eK-FV for guix-devel@gnu.org; Wed, 14 Feb 2018 17:47:30 -0500 In-Reply-To: <87zi4b744f.fsf@elephly.net> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ricardo Wurmus , Mark H Weaver Cc: guix-devel Hi Mark, Hi Ricardo, On Wed, 14 Feb 2018 20:39:12 +0100 Ricardo Wurmus wrote: > Nor do I see this message: >=20 > ghc-pkg: unable to decommit memory: Invalid argument Which Linux kernel version does this run on? > I don=E2=80=99t know what this message means, but the messages about requ= iring a If there's large address space support [1], ghc 8 does its own allocation i= n a 1 TB address space. That means it has to tell the kernel when it doesn't need some chunk anymore - otherwise you're gonna run out of memory. It does that using madvise(2). There's two ways it tries to do that: (1) MADV_FREE: Signals that "I don't need that range at all anymore". It usually means Linux will mark those pages free. (2) MADV_DONTNEED: Signals that "I don't need that range in the NEAR FUTURE= ". It usually means Linux will swap those pages out. MADV_FREE was added in Linux 4.5. Haskell uses a #ifdef to detect it. Newer glibc (such as the one in core-updates) unconditionally define MADV_F= REE to prevent programs from depending on a specific Linux kernel in this way [= 2]. There's a patch to ghc that falls back to (2) if (1) doesn't work: https://git.haskell.org/ghc.git/commitdiff/6576bf83cdf4eac05eb88a24aa934a73= 6c91e3da ... but ghc 8.0.2 which we have on core-updates doesn't use it. It uses either MADV_FREE or MADV_DONTNEED, determined at compile time. So if the Linux kernel is < 4.5 that's gonna end very badly. For the record: https://ghc.haskell.org/trac/ghc/ticket/12865 Also https://github.com/NixOS/nixpkgs/issues/18118 mmap has a flag MAP_HUGETLB which would cause it to use a mounted hugetlbfs (the cgroup of which I advised to remove from GuixSD from the time being). ghc 8 does not use it so we are safe there. [1] use_large_address_space=3Dno if test "$ac_cv_sizeof_void_p" -eq 8 ; then if test "x$EnableLargeAddressSpace" =3D "xyes" ; then if test "$ghc_host_os" =3D "darwin" ; then use_large_address_space=3Dyes elif test "$ghc_host_os" =3D "openbsd" ; then # as of OpenBSD 5.8 (2015), OpenBSD does not support mmap with = MAP_NORESERVE. # The flag MAP_NORESERVE is supported for source compatibility = reasons, # but is completely ignored by OS mmap use_large_address_space=3Dno else AC_CHECK_DECLS([MAP_NORESERVE, MADV_FREE, MADV_DONTNEED],[],[], [ #include #include #include #include ]) if test "$ac_cv_have_decl_MAP_NORESERVE" =3D "yes" && test "$ac_cv_have_decl_MADV_FREE" =3D "yes" || test "$ac_cv_have_decl_MADV_DONTNEED" =3D "yes" ; then use_large_address_space=3Dyes fi fi fi fi if test "$use_large_address_space" =3D "yes" ; then AC_DEFINE([USE_LARGE_ADDRESS_SPACE], [1], [Enable single heap address sp= ace support]) fi madvise: EINVAL addr is not page-aligned or length is negative. EINVAL advice is not a valid. EINVAL advice is MADV_DONTNEED or MADV_REMOVE and the specified addr= ess range includes locked, Huge TLB pages, or VM_PFNMAP pages. EINVAL advice is MADV_MERGEABLE or MADV_UNMERGEABLE, but the kernel = was not configured with CONFIG_KSM. EINVAL advice is MADV_FREE or MADV_WIPEONFORK but the specified addr= ess range includes file, Huge TLB, MAP_SHARED, or VM_PFNMAP range= s. [2] https://sourceware.org/git/?p=3Dglibc.git;a=3Dcommitdiff;h=3D981569c74c= bb6bafa2ddcefa6dd9dbdc938ff1c8