Re: updating many haskell packages

[Troy Sankey <sankeytms@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3658 bytes --]

Quoting Federico Beffa (2017-02-17 03:00:04)
> On Wed, Feb 15, 2017 at 8:31 PM, Troy Sankey <sankeytms@gmail.com> wrote:
> > Quoting Troy Sankey (2017-02-06 16:00:29)
> >> Quoting Federico Beffa (2017-02-06 15:34:47)
> >> > I would consider a discrepancy between a cabal file on Hackage and the
> >> > actual cabal file included in a tar archive a bug.  It may be helpful
> >> > to report it to the author.
> >>
> >> I found this issue on the hackage github:
> >>
> >> https://github.com/haskell/hackage-server/issues/503
> >>
> >> It looks like it's a feature of hackage and/or cabal to allow multiple
> >> revisions of cabal files per tarball release.  IIUC, the `cabal get`
> >> utility takes care of automatically updating the cabal file to the
> >> latest revision.
> >>
> >> I'll have to look more into this later.
> >
> > Indeed the .cabal files can be revised between releases.  Somehow the
> > new revisions get pushed to the all-cabal-files repository [0] with an
> > additional "x-revision" key [1], and Hackage picks up the new cabal
> > file.  The `cabal` and `stack` utilities substitute the .cabal file from
> > the release tarball with the revised .cabal file from Hackage if it is
> > newer.
> >
> > In theory, anything in the .cabal file could be modified, not just
> > dependency bounds.  I think guix should use the updated .cabal files for
> > better consistency with the rest of the haskell ecosystem, and also for
> > less per-package maintenance cost.
> >
> > How can guix adapt?  Should we somehow incorporate the all-cabal-files
> > repository in the haskell build system, adding a post-unpack phase to
> > substitute the entire .cabal file?
> 
> Packages are build in isolated environments without network.
> Therefore it's not possible to access Hackage with a phase of the
> build system.  Even if it were possible it would be undesirable
> because it would make things non reproducible.
> 
> Files not included in the tar can be added as origin inputs to a
> package definition (see the testsuite for GHC in the ghc package).
> However, they are verified with hashes. Any change to those files will
> break the package.
> 
> Fede

Forgive me if my understanding of build systems in Guix is flawed, but
let me explain my idea with more detail:

First, make a data-only package called "ghc-all-cabal-files" containing
the checkout of a specific commit of the all-cabal-files repository from
github.  We can periodically update this package, but there is no
traditional "release"---we just keep pulling the HEAD of the hackage
branch.  This package would then act as a helper package for the haskell
build system---every haskell package should implicitly use this package
as input.  Then we can write a post-unpack phase for the
haskell-build-system which updates the unpacked .cabal file iff it finds
a newer .cabal file in ghc-all-cabal-files (we know how to determine if
the cabal file is newer: it will have a higher "x-revision" value, or
that key will merely exist).

One problem I have not fully solved is the technical debt associated
with keeping the proposed ghc-all-cabal-files package up-to-date.  I
believe updating it would require all haskell packages to be rebuilt.
We could create a build system argument called use-newest-cabal-file to
toggle the feature, in which case we would only switch it to #t if we
already know the .cabal file to be stale.  Then only a small subset of
packages would need to be rebuilt, and there is less technical debt than
the current solution which involves monkey-patching every cabal file
that needs it.

Troy

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEE0zLJ6STd4Cp+CgbIgs677ofYV8IFAlinNyEACgkQgs677ofY
V8IZEw/8CaSba8UOqeuDqYIjm2fWbKpvR+6XKqtbh6ENvglAtUkU1bEO0M5NWJX5
8gTqytGy/C50HPDiiHZhv6ruhl/s4J1oRJJgPolNA+PsQHqHTmuRvKeVrbRkLgz0
FTnwKpkFC83lbtbD5w/j66iE604VNLmcRGMN8uhS3TZJara0FaLYZnpEj3/zuHIq
Qid+V4I/z5/gc2/dSlqJ0z/1rwTKLSDjbSfB5Qsfj4D873VeH87XG40XOHhqQypj
hQvFDIS7G0o9ZibA6MVcp2HXM4YqgUQ+4+GTtgMJmJMkoB7kS7DVyUi+N0FibLtV
uFIsbIcY2kPByYhPqXLc32BQA6hF3v8e8kXC+SMYLGE7svoKnegFaLDwuGUlhEhY
RtspzKdRPhOsC8Y7s2UawnhzUGJBwc4sV+s3dEFHf/Jus4LCG5bNkBsqj/lEiHQZ
C7bl4tMt4bVqBzNSfi1AfLUVdhN0b6mXr3u5sqj1voyB+mH3tvv3I+HqaGSoevtr
LY/KDq1lV9/YgKK/FYgFD42Y43Pazcf1J6UEJbs8MWU+5AAlz/4e7VswQT+FkZ6e
EQpUQRxJMERK4cxsZaoQ/ff0Wiuatx1T+nhAP/FmuOkKn9bD0dP2+Ntf18z3AMBm
KOzkot0XjjbU7J34ug1f3t8zN1MSGvvYWCt4ZupwDMUtp4eR8YQ=
=Wysj
-----END PGP SIGNATURE-----