From mboxrd@z Thu Jan 1 00:00:00 1970 From: Mark H Weaver Subject: Re: heads-up: Haskell updates Date: Thu, 15 Feb 2018 03:41:33 -0500 Message-ID: <87a7waodaa.fsf@netris.org> References: <87r2ppjbst.fsf@elephly.net> <873723pfya.fsf@netris.org> <871shn8jm5.fsf@elephly.net> <87zi4b744f.fsf@elephly.net> <20180214234721.4e9fe198@scratchpost.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:49585) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1emF7N-0002ty-Sd for guix-devel@gnu.org; Thu, 15 Feb 2018 03:42:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1emF7J-0006tk-7c for guix-devel@gnu.org; Thu, 15 Feb 2018 03:42:13 -0500 Received: from world.peace.net ([50.252.239.5]:40172) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1emF7J-0006sK-1l for guix-devel@gnu.org; Thu, 15 Feb 2018 03:42:09 -0500 In-Reply-To: <20180214234721.4e9fe198@scratchpost.org> (Danny Milosavljevic's message of "Wed, 14 Feb 2018 23:47:21 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Danny Milosavljevic Cc: guix-devel Hi Danny, Danny Milosavljevic writes: > Hi Mark, > Hi Ricardo, > > On Wed, 14 Feb 2018 20:39:12 +0100 > Ricardo Wurmus wrote: > >> Nor do I see this message: >>=20 >> ghc-pkg: unable to decommit memory: Invalid argument > > Which Linux kernel version does this run on? Last I knew, Hydra's x86 build slaves were all running kernels older than 4.5, but I'm not sure if that's still the case. >> I don=E2=80=99t know what this message means, but the messages about req= uiring a > > If there's large address space support [1], ghc 8 does its own allocation= in a > 1 TB address space. That means it has to tell the kernel when it doesn't > need some chunk anymore - otherwise you're gonna run out of memory. > > It does that using madvise(2). There's two ways it tries to do that: > > (1) MADV_FREE: Signals that "I don't need that range at all anymore". > It usually means Linux will mark those pages free. > > (2) MADV_DONTNEED: Signals that "I don't need that range in the NEAR FUTU= RE". > It usually means Linux will swap those pages out. > > MADV_FREE was added in Linux 4.5. Haskell uses a #ifdef to detect it. Given that it's using #ifdef to detect this, I'd expect the result to depend only on the _headers_ being used to compile, and not the actual kernel running the build. Is that right? If so, this doesn't explain why it works for Ricardo but not for Andreas and Hydra. Note that on our 'master' branch we are using Linux-libre-4.4 kernel headers, but on 'core-updates' we're using 4.9 kernel headers. The recently created evaluation 109913 on Hydra is for core-updates with the recent Haskell changes included. It'll be interesting to see if the same failure happens there. I gave the 'ghc-resourcet' jobs a high priority. Here are the 'ghc-resourcet' jobs for evaluation 109913: https://hydra.gnu.org/build/2497604 (x86_64) https://hydra.gnu.org/build/2498874 (i686) > Newer glibc (such as the one in core-updates) unconditionally define MADV= _FREE > to prevent programs from depending on a specific Linux kernel in this way= [2]. > > There's a patch to ghc that falls back to (2) if (1) doesn't work: > > https://git.haskell.org/ghc.git/commitdiff/6576bf83cdf4eac05eb88a24aa934a= 736c91e3da > > ... but ghc 8.0.2 which we have on core-updates doesn't use it. > It uses either MADV_FREE or MADV_DONTNEED, determined at compile time. > > So if the Linux kernel is < 4.5 that's gonna end very badly. Thanks for the detailed analysis! I suppose if one reads the error message literally: Control/Monad/Trans/Resource/Internal.hs:302:10: error: =E2=80=A2 Could not deduce (MonadBase IO (Strict.StateT s m)) arising from the superclasses of an instance declaration from the context: MonadResource m bound by the instance declaration at Control/Monad/Trans/Resource/Internal.hs:302:10-63 =E2=80=A2 In the instance declaration for =E2=80=98MonadResource (Strict.StateT s m)=E2=80=99 "Could not deduce" might be interpreted to include the case where we failed for lack of sufficient memory. They might have written the code to catch any error at all, including out of memory errors, and report the failure with this error message. What do you think? Mark