From mboxrd@z Thu Jan 1 00:00:00 1970 From: Troy Sankey Subject: Re: updating many haskell packages Date: Fri, 17 Feb 2017 12:47:16 -0500 Message-ID: <148735363629.2068.10173874369279597347@what> References: <148639459210.5455.4004512896538726304@what> <148641482905.2874.1459989202091260937@what> <148718709098.32672.6688568200387118544@what> Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="pgp-sha256"; boundary="===============0647694371==" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:45126) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cemd1-0000fy-Sj for guix-devel@gnu.org; Fri, 17 Feb 2017 12:47:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cemd0-0006Kp-LM for guix-devel@gnu.org; Fri, 17 Feb 2017 12:47:31 -0500 Received: from mail-qk0-x230.google.com ([2607:f8b0:400d:c09::230]:33922) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cemd0-0006Kb-G4 for guix-devel@gnu.org; Fri, 17 Feb 2017 12:47:30 -0500 Received: by mail-qk0-x230.google.com with SMTP id s186so50903446qkb.1 for ; Fri, 17 Feb 2017 09:47:30 -0800 (PST) Content-Disposition: inline In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Federico Beffa Cc: Guix-devel --===============0647694371== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Quoting Federico Beffa (2017-02-17 03:00:04) > On Wed, Feb 15, 2017 at 8:31 PM, Troy Sankey wrote: > > Quoting Troy Sankey (2017-02-06 16:00:29) > >> Quoting Federico Beffa (2017-02-06 15:34:47) > >> > I would consider a discrepancy between a cabal file on Hackage and t= he > >> > actual cabal file included in a tar archive a bug. It may be helpful > >> > to report it to the author. > >> > >> I found this issue on the hackage github: > >> > >> https://github.com/haskell/hackage-server/issues/503 > >> > >> It looks like it's a feature of hackage and/or cabal to allow multiple > >> revisions of cabal files per tarball release. IIUC, the `cabal get` > >> utility takes care of automatically updating the cabal file to the > >> latest revision. > >> > >> I'll have to look more into this later. > > > > Indeed the .cabal files can be revised between releases. Somehow the > > new revisions get pushed to the all-cabal-files repository [0] with an > > additional "x-revision" key [1], and Hackage picks up the new cabal > > file. The `cabal` and `stack` utilities substitute the .cabal file from > > the release tarball with the revised .cabal file from Hackage if it is > > newer. > > > > In theory, anything in the .cabal file could be modified, not just > > dependency bounds. I think guix should use the updated .cabal files for > > better consistency with the rest of the haskell ecosystem, and also for > > less per-package maintenance cost. > > > > How can guix adapt? Should we somehow incorporate the all-cabal-files > > repository in the haskell build system, adding a post-unpack phase to > > substitute the entire .cabal file? > = > Packages are build in isolated environments without network. > Therefore it's not possible to access Hackage with a phase of the > build system. Even if it were possible it would be undesirable > because it would make things non reproducible. > = > Files not included in the tar can be added as origin inputs to a > package definition (see the testsuite for GHC in the ghc package). > However, they are verified with hashes. Any change to those files will > break the package. > = > Fede Forgive me if my understanding of build systems in Guix is flawed, but let me explain my idea with more detail: First, make a data-only package called "ghc-all-cabal-files" containing the checkout of a specific commit of the all-cabal-files repository from github. We can periodically update this package, but there is no traditional "release"---we just keep pulling the HEAD of the hackage branch. This package would then act as a helper package for the haskell build system---every haskell package should implicitly use this package as input. Then we can write a post-unpack phase for the haskell-build-system which updates the unpacked .cabal file iff it finds a newer .cabal file in ghc-all-cabal-files (we know how to determine if the cabal file is newer: it will have a higher "x-revision" value, or that key will merely exist). One problem I have not fully solved is the technical debt associated with keeping the proposed ghc-all-cabal-files package up-to-date. I believe updating it would require all haskell packages to be rebuilt. We could create a build system argument called use-newest-cabal-file to toggle the feature, in which case we would only switch it to #t if we already know the .cabal file to be stale. Then only a small subset of packages would need to be rebuilt, and there is less technical debt than the current solution which involves monkey-patching every cabal file that needs it. Troy --===============0647694371== MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Description: signature Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE0zLJ6STd4Cp+CgbIgs677ofYV8IFAlinNyEACgkQgs677ofY V8IZEw/8CaSba8UOqeuDqYIjm2fWbKpvR+6XKqtbh6ENvglAtUkU1bEO0M5NWJX5 8gTqytGy/C50HPDiiHZhv6ruhl/s4J1oRJJgPolNA+PsQHqHTmuRvKeVrbRkLgz0 FTnwKpkFC83lbtbD5w/j66iE604VNLmcRGMN8uhS3TZJara0FaLYZnpEj3/zuHIq Qid+V4I/z5/gc2/dSlqJ0z/1rwTKLSDjbSfB5Qsfj4D873VeH87XG40XOHhqQypj hQvFDIS7G0o9ZibA6MVcp2HXM4YqgUQ+4+GTtgMJmJMkoB7kS7DVyUi+N0FibLtV uFIsbIcY2kPByYhPqXLc32BQA6hF3v8e8kXC+SMYLGE7svoKnegFaLDwuGUlhEhY RtspzKdRPhOsC8Y7s2UawnhzUGJBwc4sV+s3dEFHf/Jus4LCG5bNkBsqj/lEiHQZ C7bl4tMt4bVqBzNSfi1AfLUVdhN0b6mXr3u5sqj1voyB+mH3tvv3I+HqaGSoevtr LY/KDq1lV9/YgKK/FYgFD42Y43Pazcf1J6UEJbs8MWU+5AAlz/4e7VswQT+FkZ6e EQpUQRxJMERK4cxsZaoQ/ff0Wiuatx1T+nhAP/FmuOkKn9bD0dP2+Ntf18z3AMBm KOzkot0XjjbU7J34ug1f3t8zN1MSGvvYWCt4ZupwDMUtp4eR8YQ= =Wysj -----END PGP SIGNATURE----- --===============0647694371==--