Create branch for Haskell build changes and updates?

Create branch for Haskell build changes and updates?

[John Kehayias]

Hello,

There have been several recent bug reports and patches (list below) that could be addressed with some changes to haskell-build-system, the Hackage importer, and some package updates. Given the number of packages affected, it was discussed on #guix that it would be good to have a branch for these changes and to have Cuirass build the Haskell packages to look for breakage.

How do we feel about this? I'm happy to help with some patches I've submitted and update some packages, though it would also be good to have someone experienced with Haskell packaging and/or Guix too. (I'm new to both, but I've had success with my patches at least, locally.)

Here is a (partial) list of the most recent bugs and patches that would belong on this branch:

- https://issues.guix.gnu.org/48944 (build failure for new package, addressed by next patch)

- https://issues.guix.gnu.org/49199 (patch to add package-db to runhaskell to help with non-trivial configure stage; worked around in existing packages with a TODO marked to make this change actually)

- https://issues.guix.gnu.org/49418 (metadata revisions not imported, so e.g. dependency requirements out of date)

- https://issues.guix.gnu.org/49326 (bug with specifying ghc version for building)

- https://issues.guix.gnu.org/49320 (importer doesn't support some stages)

- https://issues.guix.gnu.org/48999 (local source for Haskell packages)

I'm not sure what we would want to include or not in such a branch, but I think it would make sense to put together the build-system and related changes at least. Some package updates (like a ghc version) would also affect a lot of packages, so might be good to do that together as well.

Thanks,
John

Re: Create branch for Haskell build changes and updates?

[Xinglu Chen]

[-- Attachment #1: Type: text/plain, Size: 2255 bytes --]

On Mon, Jul 05 2021, John Kehayias wrote:

> Hello,
>
> There have been several recent bug reports and patches (list below)
> that could be addressed with some changes to haskell-build-system, the
> Hackage importer, and some package updates. Given the number of
> packages affected, it was discussed on #guix that it would be good to
> have a branch for these changes and to have Cuirass build the Haskell
> packages to look for breakage.
>
> How do we feel about this? I'm happy to help with some patches I've
> submitted and update some packages, though it would also be good to
> have someone experienced with Haskell packaging and/or Guix too. (I'm
> new to both, but I've had success with my patches at least, locally.)
>
> Here is a (partial) list of the most recent bugs and patches that
> would belong on this branch:
>
> - https://issues.guix.gnu.org/48944 (build failure for new package,
>   addressed by next patch)
>
> - https://issues.guix.gnu.org/49199 (patch to add package-db to
>   runhaskell to help with non-trivial configure stage; worked around in
>   existing packages with a TODO marked to make this change actually)
>
> - https://issues.guix.gnu.org/49418 (metadata revisions not imported,
>   so e.g. dependency requirements out of date)
>
> - https://issues.guix.gnu.org/49326 (bug with specifying ghc version
>   for building)
>
> - https://issues.guix.gnu.org/49320 (importer doesn't support some
>   stages)
>
> - https://issues.guix.gnu.org/48999 (local source for Haskell
>   packages)
>
> I'm not sure what we would want to include or not in such a branch,
> but I think it would make sense to put together the build-system and
> related changes at least. Some package updates (like a ghc version)
> would also affect a lot of packages, so might be good to do that
> together as well.

Thank you for bringing this up!  I posted a WIP patch for adding GHC
8.10 a while ago, but nobody showed any interest[1].

I don’t have any experience with updating the Haskell toolchain in Guix,
but I think it would be great to have a separate branch for updating the
haskell-build-system and Haskell packages.  :)

[1]: https://yhetil.org/guix-devel/87o8cgfbg1.fsf@yoctocell.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Create branch for Haskell build changes and updates?

[John Kehayias]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Wednesday, July 7th, 2021 at 1:47 AM, Xinglu Chen wrote:

> On Mon, Jul 05 2021, John Kehayias wrote:
>
> > Hello,
> >
> > There have been several recent bug reports and patches (list below)
> >
> > that could be addressed with some changes to haskell-build-system, the
> >
> > Hackage importer, and some package updates. Given the number of
> >
> > packages affected, it was discussed on #guix that it would be good to
> >
> > have a branch for these changes and to have Cuirass build the Haskell
> >
> > packages to look for breakage.
> >
> > How do we feel about this? I'm happy to help with some patches I've
> >
> > submitted and update some packages, though it would also be good to
> >
> > have someone experienced with Haskell packaging and/or Guix too. (I'm
> >
> > new to both, but I've had success with my patches at least, locally.)
> >
> > Here is a (partial) list of the most recent bugs and patches that
> >
> > would belong on this branch:
> >
> > -   https://issues.guix.gnu.org/48944 (build failure for new package,
> >
> >     addressed by next patch)
> >
> > -   https://issues.guix.gnu.org/49199 (patch to add package-db to
> >
> >     runhaskell to help with non-trivial configure stage; worked around in
> >
> >     existing packages with a TODO marked to make this change actually)
> >
> > -   https://issues.guix.gnu.org/49418 (metadata revisions not imported,
> >
> >     so e.g. dependency requirements out of date)
> >
> > -   https://issues.guix.gnu.org/49326 (bug with specifying ghc version
> >
> >     for building)
> >
> > -   https://issues.guix.gnu.org/49320 (importer doesn't support some
> >
> >     stages)
> >
> > -   https://issues.guix.gnu.org/48999 (local source for Haskell
> >
> >     packages)
> >
> >
> > I'm not sure what we would want to include or not in such a branch,
> >
> > but I think it would make sense to put together the build-system and
> >
> > related changes at least. Some package updates (like a ghc version)
> >
> > would also affect a lot of packages, so might be good to do that
> >
> > together as well.
>
> Thank you for bringing this up! I posted a WIP patch for adding GHC
>
> 8.10 a while ago, but nobody showed any interest[1].
>
> I don’t have any experience with updating the Haskell toolchain in Guix,
>
> but I think it would be great to have a separate branch for updating the
>
> haskell-build-system and Haskell packages. :)
>
> [1]: https://yhetil.org/guix-devel/87o8cgfbg1.fsf@yoctocell.xyz/

I'm no expert in Haskell packages (every time I dive in anywhere it always seems a bit of a mess), but I'm happy to help out. Given the number of interdependent packages, probably makes sense to do this anytime needing to update a core package, GHC, etc.

Does any maintainer or someone with repository access want to set this up? Perhaps someone that has already handled Haskell updates?

We have several patches we can push right away and see how the updates go in the CI. Given the scope, I think it is difficult to do without a CI, though I've recompiled many a package with the patches I've submitted. I can help take charge (or do whatever) once a branch is set up, just let me know.

Re: Create branch for Haskell build changes and updates?

[John Soo]

[-- Attachment #1: Type: text/plain, Size: 522 bytes --]

Hey Guix,

I’m very happy to see some movement on haskell support.

We try to track the stack package set for a given ghc version. Updating to ghc 8.10.4, then, means updating to stackage lts 18.1.

One other issue that should be addressed is that cabal-install will need to be updated to 3.x. I have tried but I don’t know how to build it.

I also nominate updating the ghc-profile-hook. It needs some work to be fully functional. Perhaps a search-path would be the way to go there?

Kindly,

John Soo


[-- Attachment #2: Type: text/html, Size: 1018 bytes --]

Re: Create branch for Haskell build changes and updates?

[Xinglu Chen]

[-- Attachment #1: Type: text/plain, Size: 1117 bytes --]

On Wed, Jul 07 2021, John Kehayias wrote:


>> Thank you for bringing this up! I posted a WIP patch for adding GHC
>> 8.10 a while ago, but nobody showed any interest[1].
>>
>> I don’t have any experience with updating the Haskell toolchain in Guix,
>> but I think it would be great to have a separate branch for updating the
>>
>> haskell-build-system and Haskell packages. :)
>>
>> [1]: https://yhetil.org/guix-devel/87o8cgfbg1.fsf@yoctocell.xyz/
>
> I'm no expert in Haskell packages (every time I dive in anywhere it
> always seems a bit of a mess), but I'm happy to help out. Given the
> number of interdependent packages, probably makes sense to do this
> anytime needing to update a core package, GHC, etc.
>
> Does any maintainer or someone with repository access want to set this
> up? Perhaps someone that has already handled Haskell updates?

Browsing through older threads on guix-devel, Ricardo seems to have some
experience with Haskell stuff in Guix[1].  Any chance that you are
interested in helping out?  :)

[1]: https://yhetil.org/guix-devel/87mu386vk5.fsf@elephly.net/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Create branch for Haskell build changes and updates?

[Ricardo Wurmus]

Xinglu Chen <public@yoctocell.xyz> writes:

> On Wed, Jul 07 2021, John Kehayias wrote:
>
>
>>> Thank you for bringing this up! I posted a WIP patch for 
>>> adding GHC
>>> 8.10 a while ago, but nobody showed any interest[1].
>>>
>>> I don’t have any experience with updating the Haskell 
>>> toolchain in Guix,
>>> but I think it would be great to have a separate branch for 
>>> updating the
>>>
>>> haskell-build-system and Haskell packages. :)
>>>
>>> [1]: 
>>> https://yhetil.org/guix-devel/87o8cgfbg1.fsf@yoctocell.xyz/
>>
>> I'm no expert in Haskell packages (every time I dive in 
>> anywhere it
>> always seems a bit of a mess), but I'm happy to help out. Given 
>> the
>> number of interdependent packages, probably makes sense to do 
>> this
>> anytime needing to update a core package, GHC, etc.
>>
>> Does any maintainer or someone with repository access want to 
>> set this
>> up? Perhaps someone that has already handled Haskell updates?
>
> Browsing through older threads on guix-devel, Ricardo seems to 
> have some
> experience with Haskell stuff in Guix[1].  Any chance that you 
> are
> interested in helping out?  :)

Sorry, not this time :)

I suggest just starting a new branch.  It doesn’t really need to 
be hosted in the main repo at this point, but if you have some 
initial patches that you’d like me to push to wip-haskell feel 
free to send them.

-- 
Ricardo

Re: Create branch for Haskell build changes and updates?

[John Kehayias]

On Friday, July 9th, 2021 at 10:06 AM, Ricardo Wurmus wrote:
> Sorry, not this time :)
>
> I suggest just starting a new branch. It doesn’t really need to
>
> be hosted in the main repo at this point, but if you have some
>
> initial patches that you’d like me to push to wip-haskell feel
>
> free to send them.
>
Hi Ricardo,

Yes, we could have a repo anywhere, but what (at least) I was getting at was having some CI support to build Haskell packages to find any breakage. Even without being on the main repo could we do that? Changes I've made locally and packages I've tried to add (which might need version bumps of others) lead to a ton of compile time.

I'm pretty confident my build system patch https://issues.guix.gnu.org/49199 works well as far as I've tested it. However, for things like GHC, that would require a lot of compile time and package updates everywhere I think, and I personally don't have the resources/time to do that.

We could at least get a mirror from core-updates(?) to base off of and add some patches for individual testing. Maybe anyone who wants to lead a GHC update can weigh in on resources needed to make it doable (as that I would imagine needs the most time to do)?

John

Re: Create branch for Haskell build changes and updates?

[Ricardo Wurmus]

Hi John,

> I'm pretty confident my build system patch 
> https://issues.guix.gnu.org/49199 works well as far as I've 
> tested it. However, for things like GHC, that would require a 
> lot of compile time and package updates everywhere I think, and 
> I personally don't have the resources/time to do that.

I could branch wip-haskell off of core-updates and apply that 
patch, then add a build farm job for wip-haskell.

Would that help?

-- 
Ricardo

Re: Create branch for Haskell build changes and updates?

[John Kehayias]

Hi Ricardo,

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Friday, July 9th, 2021 at 12:44 PM, Ricardo Wurmus wrote:

> Hi John,
>
> > I'm pretty confident my build system patch
> >
> > https://issues.guix.gnu.org/49199 works well as far as I've
> >
> > tested it. However, for things like GHC, that would require a
> >
> > lot of compile time and package updates everywhere I think, and
> >
> > I personally don't have the resources/time to do that.
>
> I could branch wip-haskell off of core-updates and apply that
>
> patch, then add a build farm job for wip-haskell.
>
> Would that help?
>
At least for me that would be very nice as it would confirm if that patch is ready for core-updates. And that would help with including/updating some packages that have non-trivial configure.

As for the other large updates, like GHC and updating core packages, I will let others speak to. But with a separate branch we could start with getting GHC 8.10.x built and moving packages to newer stackage resolvers (lts 17 or 18?).

Thanks,
John

Re: Create branch for Haskell build changes and updates?

[Xinglu Chen]

[-- Attachment #1: Type: text/plain, Size: 901 bytes --]

On Fri, Jul 09 2021, John Kehayias wrote:

> On Friday, July 9th, 2021 at 10:06 AM, Ricardo Wurmus wrote:
>> Sorry, not this time :)
>>
>> I suggest just starting a new branch. It doesn’t really need to
>>
>> be hosted in the main repo at this point, but if you have some
>>
>> initial patches that you’d like me to push to wip-haskell feel
>>
>> free to send them.
>>
> Hi Ricardo,
>
> Yes, we could have a repo anywhere, but what (at least) I was getting
> at was having some CI support to build Haskell packages to find any
> breakage. Even without being on the main repo could we do that?
> Changes I've made locally and packages I've tried to add (which might
> need version bumps of others) lead to a ton of compile time.

Agreed, I only have one machine, and compiling GHC on it takes 3-4
hours.  Having CI compiling that and all the haskell packages would be
nice.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Create branch for Haskell build changes and updates?

[Philip Munksgaard]

Any updates on this? Who do we talk to to get this in place?

Best,
Philip

Re: Create branch for Haskell build changes and updates?

[Ricardo Wurmus]

Philip Munksgaard <philip@munksgaard.me> writes:

> Who do we talk to to get this in place?

Me, for example.

Sorry, I got sick and haven’t been doing anything towards getting 
wip-haskell started.

I’ll try to do this tonight:

- branch off wip-haskell from core-updates
- apply https://issues.guix.gnu.org/49199
- create a new job set for wip-haskell

-- 
Ricardo

Re: Create branch for Haskell build changes and updates?

[Philip Munksgaard]

On Tue, 20 Jul 2021, at 18:49, Ricardo Wurmus wrote:
> 
> Philip Munksgaard <philip@munksgaard.me> writes:
> 
> > Who do we talk to to get this in place?
> 
> Me, for example.
> 
> Sorry, I got sick and haven’t been doing anything towards getting 
> wip-haskell started.
> 
> I’ll try to do this tonight:
> 
> - branch off wip-haskell from core-updates
> - apply https://issues.guix.gnu.org/49199
> - create a new job set for wip-haskell
> 

Sounds great! Who gets commit access to that branch?

I'm new to guix (and mail-based development in general), so I'm sorry if that's a stupid question.

Re: Create branch for Haskell build changes and updates?

[Ricardo Wurmus]

Xinglu Chen <public@yoctocell.xyz> writes:

> Thank you for bringing this up!  I posted a WIP patch for adding 
> GHC
> 8.10 a while ago, but nobody showed any interest[1].
>
> I don’t have any experience with updating the Haskell toolchain 
> in Guix,
> but I think it would be great to have a separate branch for 
> updating the
> haskell-build-system and Haskell packages.  :)
>
> [1]: https://yhetil.org/guix-devel/87o8cgfbg1.fsf@yoctocell.xyz/

It would be good to send this to guix-patches@gnu.org instead of 
guix-devel.  Please also see the Contributing section in the 
manual.

-- 
Ricardo

Re: Create branch for Haskell build changes and updates?

[Ricardo Wurmus]

Ricardo Wurmus <rekado@elephly.net> writes:

> Philip Munksgaard <philip@munksgaard.me> writes:
>
>> Who do we talk to to get this in place?
>
> Me, for example.
>
> Sorry, I got sick and haven’t been doing anything towards 
> getting
> wip-haskell started.
>
> I’ll try to do this tonight:
>
> - branch off wip-haskell from core-updates
> - apply https://issues.guix.gnu.org/49199
> - create a new job set for wip-haskell

wip-haskell is being built now:

    https://ci.guix.gnu.org/eval/65335/dashboard


-- 
Ricardo

Re: Create branch for Haskell build changes and updates?

[Xinglu Chen]

[-- Attachment #1: Type: text/plain, Size: 1205 bytes --]

On Wed, Jul 21 2021, Ricardo Wurmus wrote:

> Xinglu Chen <public@yoctocell.xyz> writes:
>
>> Thank you for bringing this up!  I posted a WIP patch for adding 
>> GHC
>> 8.10 a while ago, but nobody showed any interest[1].
>>
>> I don’t have any experience with updating the Haskell toolchain 
>> in Guix,
>> but I think it would be great to have a separate branch for 
>> updating the
>> haskell-build-system and Haskell packages.  :)
>>
>> [1]: https://yhetil.org/guix-devel/87o8cgfbg1.fsf@yoctocell.xyz/
>
> It would be good to send this to guix-patches@gnu.org instead of 
> guix-devel.  Please also see the Contributing section in the 
> manual.

I asked on guix-devel to see if anybody was interested in an Haskell
update; I just attached a WIP patch to see if anyone would be interested
in helping out, since I didn’t really know where to start.  Sending it
to guix-patches would make it more likely to be forgotten since it
is a higher-volume list.

Now that more people have showed interest, I recently sent a patch for
it to guix-patches[1].  I still have to address some failing tests
before it’s ready to merge.  :)

[1]: https://issues.guix.gnu.org/49606

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

Re: Create branch for Haskell build changes and updates?

[Ricardo Wurmus]

Xinglu Chen <public@yoctocell.xyz> writes:

> Sending it
> to guix-patches would make it more likely to be forgotten since 
> it
> is a higher-volume list.

The opposite.  guix-patches is backed by debbugs / 
issues.guix.gnu.org, so we can query it and show forgotten 
patches.  Sending patches to guix-devel is how things get lost.

> Now that more people have showed interest, I recently sent a 
> patch for
> it to guix-patches[1].  I still have to address some failing 
> tests
> before it’s ready to merge.  :)
>
> [1]: https://issues.guix.gnu.org/49606

Thanks.

-- 
Ricardo

Re: Create branch for Haskell build changes and updates?

[John Kehayias]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Wednesday, July 21st, 2021 at 5:10 PM, Ricardo Wurmus wrote:

> Ricardo Wurmus rekado@elephly.net writes:
>
> > Philip Munksgaard philip@munksgaard.me writes:
> >
> > > Who do we talk to to get this in place?
> >
> > Me, for example.
> >
> > Sorry, I got sick and haven’t been doing anything towards
> >
> > getting
> >
> > wip-haskell started.
> >
> > I’ll try to do this tonight:
> >
> > -   branch off wip-haskell from core-updates
> > -   apply https://issues.guix.gnu.org/49199
> > -   create a new job set for wip-haskell
>
> wip-haskell is being built now:
>
> https://ci.guix.gnu.org/eval/65335/dashboard
>

Great, thanks! I was away for a bit and catching up now.

Some failing builds are lacking a proper backtrace, e.g. https://ci.guix.gnu.org/build/680388/details Perhaps it is what is referred to as fixed in this more recent commit in core-updates: https://git.savannah.gnu.org/cgit/guix.git/commit/?h=core-updates&id=a0f880afea3b29e817866de0170ed579738db2d1

In any event, going forward we can apply any Haskell patches here, and probably the big one is to update GHC and Stackage resolver? Is there any automated way to updating a large number of packages to get started? Some will require manual help, but at least we can start with getting all versions to whichever Stackage resolver we update to.

John

Re: Create branch for Haskell build changes and updates?

[Ricardo Wurmus]

John Kehayias <john.kehayias@protonmail.com> writes:

> In any event, going forward we can apply any Haskell patches 
> here, and probably the big one is to update GHC and Stackage 
> resolver?

Yes, please base all Haskell-related updates on that branch.

> Is there any automated way to updating a large number of 
> packages to get started?

Perhaps “guix refresh -u” can be of help here.  For making a large 
number of commits automatically consider using etc/committer.scm.

-- 
Ricardo