From mboxrd@z Thu Jan 1 00:00:00 1970 From: Troy Sankey Subject: Re: updating many haskell packages Date: Sat, 18 Feb 2017 12:40:15 -0500 Message-ID: <148743961548.2035.9177897023989128359@what> References: <148639459210.5455.4004512896538726304@what> <148641482905.2874.1459989202091260937@what> <148718709098.32672.6688568200387118544@what> <148735363629.2068.10173874369279597347@what> Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="pgp-sha256"; boundary="===============1464262690==" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:35540) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cf8zc-0001Vh-3Q for guix-devel@gnu.org; Sat, 18 Feb 2017 12:40:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cf8za-0006ZH-Me for guix-devel@gnu.org; Sat, 18 Feb 2017 12:40:20 -0500 Received: from mail-qk0-x22e.google.com ([2607:f8b0:400d:c09::22e]:35029) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cf8za-0006ZA-IM for guix-devel@gnu.org; Sat, 18 Feb 2017 12:40:18 -0500 Received: by mail-qk0-x22e.google.com with SMTP id u25so68012430qki.2 for ; Sat, 18 Feb 2017 09:40:18 -0800 (PST) Content-Disposition: inline In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Federico Beffa Cc: Guix-devel --===============1464262690== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Quoting Federico Beffa (2017-02-18 04:43:51) > On Fri, Feb 17, 2017 at 6:47 PM, Troy Sankey wrote: > > Forgive me if my understanding of build systems in Guix is flawed, but > > let me explain my idea with more detail: > > > > First, make a data-only package called "ghc-all-cabal-files" containing > > the checkout of a specific commit of the all-cabal-files repository from > > github. We can periodically update this package, but there is no > > traditional "release"---we just keep pulling the HEAD of the hackage > > branch. This package would then act as a helper package for the haskell > > build system---every haskell package should implicitly use this package > > as input. Then we can write a post-unpack phase for the > > haskell-build-system which updates the unpacked .cabal file iff it finds > > a newer .cabal file in ghc-all-cabal-files (we know how to determine if > > the cabal file is newer: it will have a higher "x-revision" value, or > > that key will merely exist). > > > > One problem I have not fully solved is the technical debt associated > > with keeping the proposed ghc-all-cabal-files package up-to-date. I > > believe updating it would require all haskell packages to be rebuilt. > > We could create a build system argument called use-newest-cabal-file to > > toggle the feature, in which case we would only switch it to #t if we > > already know the .cabal file to be stale. Then only a small subset of > > packages would need to be rebuilt, and there is less technical debt than > > the current solution which involves monkey-patching every cabal file > > that needs it. > = > My worry with this approach is that every time that a single cabal > file in 'ghc-all-cabal-files' is updated all packages will fail to > build (the implicit input will fail). Given that there are several > thousands of cabal files, I suspect that this could occur quite often. Can you elaborate on how all packages will fail to build? Also, my hope is that with the toggle switch 'use-newest-cabal-file' we do not have to make ghc-all-cabal-files an implicit input for all haskell packages, only a few. > Maybe another approach could be: when we update the GHC packages we > follow a specific Stackage release. If a package tar includes an old > cabal file, we enable a 'jailbreak' flag such that the build system > clears version bounds. This of course relies on the fact that Stackage > verified that this is fine. What you describe is almost what I mean. It is a fine alternative solution, and I would be open to implementing this for the immediate term. My approach implies that we use Stackage LTS, but instead of "jailbreak" use "use-newest-cabal-file" since the newer cabal file often will include those relaxed version bounds, so the effect is almost the same. The newer cabal file may also contain other changes unrelated to dependency version bounds [0] and those changes are implicitly verified by Stackage LTS which also uses newer cabal files. In fact, by not using the updated cabal files, we are essentially diverging from the Stackage LTS. Remember, all-cabal-files tracks cabal revisions for every package version. If Stackage LTS references an old version of a haskell package, then we can still find the "latest" cabal file for that old version of the package in all-cabal-files. There is one more problem, and it is not a minor problem... ------------------------------- cut here ------------------------------- $ du -hs tmp/all-cabal-files 862M tmp/all-cabal-files $ du -hs tmp/all-cabal-files/.git 140M tmp/all-cabal-files/.git $ echo '862 - 140' | bc 722 ------------------------------- cut here ------------------------------- The filesize of the package output will be 722 MiB. We could prune it of all the unrelated versions according to the stackage LTS, but this starts to get complicated and error-prone... Troy [0] https://www.haskell.org/cabal/users-guide/developing-packages.html#buil= d-information --===============1464262690== MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Description: signature Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE0zLJ6STd4Cp+CgbIgs677ofYV8IFAliohvoACgkQgs677ofY V8KH4BAAo3Qletdl0B19BuJfXD3RIZy9L3nLOoq4ucrq01YKLyxgmaQF2ZcGjIS8 azdpVlnGlaS+PXgI/LxOYezIxx9JD7+crptvYzQEbU//nKhYG3I00E09QYyOVlDR lgxKRL2bIVWuNBjKNlCBjXywWOpxYMPRyOjsi/bTTTbjIxZ2tZgzcK4v2/49VmSn +Udw160t7ZkZ0M9eTo1hMIniyBUggD9U8krmmoJr1V2TYRCYO45cwo9fXzQyjsMt xVPbFAsMsi8VtlN0ZpM6m0u3DO9kQdqFbpovHi0rgXqlx3ERsx1ZJSIPNIAjeh0L CmmGh8ygcUcMToBLojourlMT8P001RXXZnSNyDe4SLW6ycm7ohSrUV6LWFcXm1EB ArCu/3J1LcDORjbpwze20JF51nfS7zSA+qKT8pPcectFPShHx7d/U2o+x8HHfa0V oOwnaJQaTDKUIv7Kvkmh9kVozwijiNATins7rlZXU9YpDmbjoG+mn3T+vmqxGeAl 1qkex4+XsJ+3pfJR5Ckj6oBayCOf9kQqPJIy/B0qGEi3GSXnG37Wsbf1ke2I4SQ7 xrND4bkBGA5qFg5ZM/4xKqGzX+ur7qE+R1yC5fmrRsUVteZ1Th7XFQMDUt8cQC9w /jlw55+917X/MwuCZxGvOUOLTPjMXPY6prphpEIV6ub2JjGzb8Q= =gKvM -----END PGP SIGNATURE----- --===============1464262690==--