unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* merge wip-haskell?
@ 2020-08-06  8:13 Ricardo Wurmus
  2020-08-06 13:45 ` John Soo
  2020-08-07 15:03 ` Jakub Kądziołka
  0 siblings, 2 replies; 24+ messages in thread
From: Ricardo Wurmus @ 2020-08-06  8:13 UTC (permalink / raw)
  To: guix-devel

Hey there,

wip-haskell contains commits that do a number of things aimed to reduce
the closure of packages:

1) make the “out” (and “lib”) output independent from “doc”
2) add a “doc” output to more packages
3) add a “static” output for all Haskell packages containing the “.a” files
4) change ghc-pandoc (and ghc-pandoc-citeproc) to use static linking

Number 1 required some smelly hackery: the generated configuration file
is edited to remove the “haddock-html” field.  One unfortunate effect of
doing this and moving the .haddock files is that there are now
complaints about unresolvable links in generated documentation.  I don’t
know if we can avoid this, but it seems like a small price to pay for
independent “doc” outputs.  (Otherwise we’d have to download huge “doc”
outputs even if we don’t want them.)

Number 3 required circumventing bug 41569.

Number 4 is by far the ugliest change of them all.  In order to
statically link packages we need to add all the “static” outputs of all
Haskell inputs *and* the “static” outputs of *their* Haskell inputs.
This is not easily accomplished, so I ended up using “package-closure”
on all direct inputs, and then filtered the result to packages with
names starting with “ghc-”.  If there was a more appropriate tool I’d
use it, but I don’t think it exists.

The result is a much reduced closure for ghc-pandoc and all packages
using it (such as R markdown).  We should probably rename “ghc-pandoc”
to “pandoc”, while we’re at it, because now the package contains the
executable.

I suppose we could change this so that “ghc-pandoc” is the usual library
package with a new “pandoc” package inheriting from “ghc-pandoc”.  I’ll
give that a try soon.

I’d be happy to hear your comments about all of this, and I’m looking
forward to merging this branch into “master” soon.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-06  8:13 merge wip-haskell? Ricardo Wurmus
@ 2020-08-06 13:45 ` John Soo
  2020-08-07 17:46   ` Timothy Sample
  2020-08-07 15:03 ` Jakub Kądziołka
  1 sibling, 1 reply; 24+ messages in thread
From: John Soo @ 2020-08-06 13:45 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

Hi Ricardo,


Nice! Sounds good to me. There are a couple other bits of work I’d like to see make it in.

I believe there was also some work being done to de-duplicate flags sent to gcc sent by ghc (this was the only thing keeping stack from building). 
I hope that can make it in, too!

If there is any way we can bump the default ghc to 8.8.x, that would be excellent, too.  I think something like that makes sense on staging though.

Thanks for working on it!

- John


On Aug 6, 2020, at 1:14 AM, Ricardo Wurmus <rekado@elephly.net> wrote:

Hey there,

wip-haskell contains commits that do a number of things aimed to reduce
the closure of packages:

1) make the “out” (and “lib”) output independent from “doc”
2) add a “doc” output to more packages
3) add a “static” output for all Haskell packages containing the “.a” files
4) change ghc-pandoc (and ghc-pandoc-citeproc) to use static linking

Number 1 required some smelly hackery: the generated configuration file
is edited to remove the “haddock-html” field.  One unfortunate effect of
doing this and moving the .haddock files is that there are now
complaints about unresolvable links in generated documentation.  I don’t
know if we can avoid this, but it seems like a small price to pay for
independent “doc” outputs.  (Otherwise we’d have to download huge “doc”
outputs even if we don’t want them.)

Number 3 required circumventing bug 41569.

Number 4 is by far the ugliest change of them all.  In order to
statically link packages we need to add all the “static” outputs of all
Haskell inputs *and* the “static” outputs of *their* Haskell inputs.
This is not easily accomplished, so I ended up using “package-closure”
on all direct inputs, and then filtered the result to packages with
names starting with “ghc-”.  If there was a more appropriate tool I’d
use it, but I don’t think it exists.

The result is a much reduced closure for ghc-pandoc and all packages
using it (such as R markdown).  We should probably rename “ghc-pandoc”
to “pandoc”, while we’re at it, because now the package contains the
executable.

I suppose we could change this so that “ghc-pandoc” is the usual library
package with a new “pandoc” package inheriting from “ghc-pandoc”.  I’ll
give that a try soon.

I’d be happy to hear your comments about all of this, and I’m looking
forward to merging this branch into “master” soon.

-- 
Ricardo



^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-06  8:13 merge wip-haskell? Ricardo Wurmus
  2020-08-06 13:45 ` John Soo
@ 2020-08-07 15:03 ` Jakub Kądziołka
  2020-08-07 15:12   ` John Soo
  2020-08-07 16:27   ` Ricardo Wurmus
  1 sibling, 2 replies; 24+ messages in thread
From: Jakub Kądziołka @ 2020-08-07 15:03 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 993 bytes --]

On Thu, Aug 06, 2020 at 10:13:46AM +0200, Ricardo Wurmus wrote:
> Number 4 is by far the ugliest change of them all.  In order to
> statically link packages we need to add all the “static” outputs of all
> Haskell inputs *and* the “static” outputs of *their* Haskell inputs.
> This is not easily accomplished, so I ended up using “package-closure”
> on all direct inputs, and then filtered the result to packages with
> names starting with “ghc-”.  If there was a more appropriate tool I’d
> use it, but I don’t think it exists.

Perhaps we should work on making propagated-inputs per-output? That way,
:static could propagate the :static output of the dependencies.

This would also be useful in other situations. For example, a package
might contain both a binary and a library, and the library must
propagate its dependencies to make the header files work.

I don't know what a good syntax for this would be.

Thoughts?

Regards,
Jakub Kądziołka

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-07 15:03 ` Jakub Kądziołka
@ 2020-08-07 15:12   ` John Soo
  2020-08-07 15:46     ` Jakub Kądziołka
  2020-08-07 16:27   ` Ricardo Wurmus
  1 sibling, 1 reply; 24+ messages in thread
From: John Soo @ 2020-08-07 15:12 UTC (permalink / raw)
  To: Jakub Kądziołka; +Cc: guix-devel

Hi Jakub,

I could see splitting the static output being useful but I would rather wait until some evidence that the closure size would be too large. Also I’m not sure propagation is necessary for dependents to find libraries or use paths from an input.

Thoughts?

John

On Aug 7, 2020, at 8:04 AM, Jakub Kądziołka <kuba@kadziolka.net> wrote:

On Thu, Aug 06, 2020 at 10:13:46AM +0200, Ricardo Wurmus wrote:
> Number 4 is by far the ugliest change of them all.  In order to
> statically link packages we need to add all the “static” outputs of all
> Haskell inputs *and* the “static” outputs of *their* Haskell inputs.
> This is not easily accomplished, so I ended up using “package-closure”
> on all direct inputs, and then filtered the result to packages with
> names starting with “ghc-”.  If there was a more appropriate tool I’d
> use it, but I don’t think it exists.

Perhaps we should work on making propagated-inputs per-output? That way,
:static could propagate the :static output of the dependencies.

This would also be useful in other situations. For example, a package
might contain both a binary and a library, and the library must
propagate its dependencies to make the header files work.

I don't know what a good syntax for this would be.

Thoughts?

Regards,
Jakub Kądziołka


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-07 15:12   ` John Soo
@ 2020-08-07 15:46     ` Jakub Kądziołka
  2020-08-07 15:59       ` Ricardo Wurmus
  0 siblings, 1 reply; 24+ messages in thread
From: Jakub Kądziołka @ 2020-08-07 15:46 UTC (permalink / raw)
  To: John Soo; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 309 bytes --]

On Fri, Aug 07, 2020 at 08:12:36AM -0700, John Soo wrote:
> I would rather wait until some evidence that the closure size would be too large. Also I’m not sure propagation is necessary for dependents to find libraries or use paths from an input.

Ricardo already explained that this is indeed the case.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-07 15:46     ` Jakub Kądziołka
@ 2020-08-07 15:59       ` Ricardo Wurmus
  2020-08-07 16:08         ` John Soo
  2020-08-24 15:44         ` Ludovic Courtès
  0 siblings, 2 replies; 24+ messages in thread
From: Ricardo Wurmus @ 2020-08-07 15:59 UTC (permalink / raw)
  To: Jakub Kądziołka; +Cc: guix-devel


Jakub Kądziołka <kuba@kadziolka.net> writes:

> On Fri, Aug 07, 2020 at 08:12:36AM -0700, John Soo wrote:
>> I would rather wait until some evidence that the closure size would be too large. Also I’m not sure propagation is necessary for dependents to find libraries or use paths from an input.
>
> Ricardo already explained that this is indeed the case.

Yes, and for the case of pandoc it’s significant.  The closure of
ghc-pandoc is >3GiB right now and with the changes it’s <200MiB.  This
affects lots of R packages that need Rmarkdown, and lots of bioinfo
packages.

There's no doubt that moving the static libs to their own output has a
significant impact on closure size.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-07 15:59       ` Ricardo Wurmus
@ 2020-08-07 16:08         ` John Soo
  2020-08-24 15:44         ` Ludovic Courtès
  1 sibling, 0 replies; 24+ messages in thread
From: John Soo @ 2020-08-07 16:08 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: Jakub Kądziołka, guix-devel

Hi Ricardo and Jakub,

Ah ok. Sorry I had forgotten the point of the thread.  Sounds like a plan!

- John

On Aug 7, 2020, at 8:59 AM, Ricardo Wurmus <rekado@elephly.net> wrote:


Jakub Kądziołka <kuba@kadziolka.net> writes:

>> On Fri, Aug 07, 2020 at 08:12:36AM -0700, John Soo wrote:
>> I would rather wait until some evidence that the closure size would be too large. Also I’m not sure propagation is necessary for dependents to find libraries or use paths from an input.
> 
> Ricardo already explained that this is indeed the case.

Yes, and for the case of pandoc it’s significant.  The closure of
ghc-pandoc is >3GiB right now and with the changes it’s <200MiB.  This
affects lots of R packages that need Rmarkdown, and lots of bioinfo
packages.

There's no doubt that moving the static libs to their own output has a
significant impact on closure size.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-07 15:03 ` Jakub Kądziołka
  2020-08-07 15:12   ` John Soo
@ 2020-08-07 16:27   ` Ricardo Wurmus
  2020-08-07 16:55     ` Jakub Kądziołka
  1 sibling, 1 reply; 24+ messages in thread
From: Ricardo Wurmus @ 2020-08-07 16:27 UTC (permalink / raw)
  To: Jakub Kądziołka; +Cc: guix-devel


Jakub Kądziołka <kuba@kadziolka.net> writes:

> On Thu, Aug 06, 2020 at 10:13:46AM +0200, Ricardo Wurmus wrote:
>> Number 4 is by far the ugliest change of them all.  In order to
>> statically link packages we need to add all the “static” outputs of all
>> Haskell inputs *and* the “static” outputs of *their* Haskell inputs.
>> This is not easily accomplished, so I ended up using “package-closure”
>> on all direct inputs, and then filtered the result to packages with
>> names starting with “ghc-”.  If there was a more appropriate tool I’d
>> use it, but I don’t think it exists.
>
> Perhaps we should work on making propagated-inputs per-output? That way,
> :static could propagate the :static output of the dependencies.
>
> This would also be useful in other situations. For example, a package
> might contain both a binary and a library, and the library must
> propagate its dependencies to make the header files work.
>
> I don't know what a good syntax for this would be.

The binary / library problem is particularly interesting to me as we
have a few Python bioinfo tools that also work as libraries.  When used
as a library we need propagation, but when uses as command line tools
propagation is not needed (and can be harmful due to the effects on
PYTHONPATH).

Syntax seems less important than verifying that this actually works,
though.  Alas, as someone who has committed and encouraged syntactic
crimes with excessive reader macros, I think my vote on syntax carries
no weight :)

-- 
Ricardo


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-07 16:27   ` Ricardo Wurmus
@ 2020-08-07 16:55     ` Jakub Kądziołka
  0 siblings, 0 replies; 24+ messages in thread
From: Jakub Kądziołka @ 2020-08-07 16:55 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 2996 bytes --]

On Fri, Aug 07, 2020 at 06:27:21PM +0200, Ricardo Wurmus wrote:
> 
> Jakub Kądziołka <kuba@kadziolka.net> writes:
> 
> > On Thu, Aug 06, 2020 at 10:13:46AM +0200, Ricardo Wurmus wrote:
> >> Number 4 is by far the ugliest change of them all.  In order to
> >> statically link packages we need to add all the “static” outputs of all
> >> Haskell inputs *and* the “static” outputs of *their* Haskell inputs.
> >> This is not easily accomplished, so I ended up using “package-closure”
> >> on all direct inputs, and then filtered the result to packages with
> >> names starting with “ghc-”.  If there was a more appropriate tool I’d
> >> use it, but I don’t think it exists.
> >
> > Perhaps we should work on making propagated-inputs per-output? That way,
> > :static could propagate the :static output of the dependencies.
> >
> > This would also be useful in other situations. For example, a package
> > might contain both a binary and a library, and the library must
> > propagate its dependencies to make the header files work.
> >
> > I don't know what a good syntax for this would be.
> 
> The binary / library problem is particularly interesting to me as we
> have a few Python bioinfo tools that also work as libraries.  When used
> as a library we need propagation, but when uses as command line tools
> propagation is not needed (and can be harmful due to the effects on
> PYTHONPATH).
> 
> Syntax seems less important than verifying that this actually works,
> though.

One still needs to pick a variant, even at random, to begin
implementation, which is hard when no options have been suggested :)

I suppose that we could add a list of outputs each propagated-input
applies to, putting it after the package and its output?

(this is unambiguous as we can check whether the third element of the
list is a string, or a list)

    (propagated-inputs
     `(("foo-static" ,foo "static" ("static"))
       ("bar" ,bar ("static"))))

However, this has the issue of being somewhat ad-hoc, and other
extensions to the list of inputs might become troublesome. This
motivates my second proposal:

    (propagated-inputs
     `(("foo-static" ,foo "static" #:propagate-to ("static"))
       ("bar" ,bar #:propagate-to ("static"))))

On the other hand, this variant feels somewhat over-engineered. After
all, we can always change the syntax if it becomes actually necessary.

There's also the option of adding another field to the package record:

    (propagated-inputs-for-outputs ;; name up to bikeshedding :)
     `(("static"
        (("foo-static" ,foo "static")
         ("bar ,bar)))))

> Alas, as someone who has committed and encouraged syntactic
> crimes with excessive reader macros, I think my vote on syntax carries
> no weight :)

Well, the number of reader macro crimes encouraged seems to correlate
with experience, so I would still like to ask for your opinion.

Regards,
Jakub Kądziołka

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-06 13:45 ` John Soo
@ 2020-08-07 17:46   ` Timothy Sample
  2020-08-07 19:53     ` Ricardo Wurmus
  0 siblings, 1 reply; 24+ messages in thread
From: Timothy Sample @ 2020-08-07 17:46 UTC (permalink / raw)
  To: John Soo, Ricardo Wurmus; +Cc: guix-devel

Hi John and Ricardo,

John Soo <jsoo1@asu.edu> writes:

> I believe there was also some work being done to de-duplicate flags
> sent to gcc sent by ghc (this was the only thing keeping stack from
> building).
> I hope that can make it in, too!

I agree, and sorry to you John for the delay.  I’m not using much
Haskell these days, so I’m having trouble keeping Haskell stuff on my
radar.  The last patch I sent to <https://issues.guix.gnu.org/39309> is
about as good as I can come up with, so I will apply it to the branch
before the merge.

Ricardo, I will try and do this tonight or tomorrow, and will look over
your changes then as well.

> If there is any way we can bump the default ghc to 8.8.x, that would
> be excellent, too.  I think something like that makes sense on staging
> though.

I would love for this to happen, but I think it would needlessly delay
Ricardo’s changes.  Fortunately, one can build all of our Haskell
packages overnight on a modest computer, so it’s no big deal to merge
the branch now and then merge the GHC 8.8.x updates whenever they’re
ready.

Also, it looks like “wip-haskell-updates” is no longer being built by
the CI infrastructure.  Since the branch triggers a rebuild of all the
Haskell packages, it should be built before merging, right?


-- Tim


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-07 17:46   ` Timothy Sample
@ 2020-08-07 19:53     ` Ricardo Wurmus
  2020-08-09  4:29       ` Timothy Sample
  0 siblings, 1 reply; 24+ messages in thread
From: Ricardo Wurmus @ 2020-08-07 19:53 UTC (permalink / raw)
  To: Timothy Sample; +Cc: guix-devel


Timothy Sample <samplet@ngyro.com> writes:

> Also, it looks like “wip-haskell-updates” is no longer being built by
> the CI infrastructure.  Since the branch triggers a rebuild of all the
> Haskell packages, it should be built before merging, right?

Yes, I’ll rebase it on top of “master” and add the specification to
ci.guix.gnu.org well before the merge.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-07 19:53     ` Ricardo Wurmus
@ 2020-08-09  4:29       ` Timothy Sample
  2020-08-12 12:21         ` Ricardo Wurmus
  2020-08-13 15:37         ` Ricardo Wurmus
  0 siblings, 2 replies; 24+ messages in thread
From: Timothy Sample @ 2020-08-09  4:29 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

Hi Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> Timothy Sample <samplet@ngyro.com> writes:
>
>> Also, it looks like “wip-haskell-updates” is no longer being built by
>> the CI infrastructure.  Since the branch triggers a rebuild of all the
>> Haskell packages, it should be built before merging, right?
>
> Yes, I’ll rebase it on top of “master” and add the specification to
> ci.guix.gnu.org well before the merge.

Excellent.

I just pushed “wip-haskell-updates-2” which integrates my work from
<https://issues.guix.gnu.org/39309>.  I left the original branch intact
to make it easy to compare.

Basically, where you remove the “--extra-include-dirs” and
“--extra-lib-dirs” arguments in configure, I preserve them and hide them
behind a build system argument.  To do this, I split up the commit where
you remove them into a refactor commit (where you remove “append” and
just use quasiquoting), and a commit that removes “--bindir”.  My commit
goes in the middle.

Then, I remove the commits that fix up ghc-hslua, ghc-libyaml, and
ghc-zlib, as that’s handled in my commit.

The only other thing I did was move the shared libraries commit sooner,
since it needs to be in place for the static output commit to work (at
least nothing would build for me without it).

With respect to the substance of your changes, I think the results are
worth the ugliness!  Keeping “ghc-pandoc” as the “normal” package and
using “pandoc” for the statically linked one makes sense if feasible.
Unfortunately, I don’t know of a better way to get all the static
libraries in place.

It would be nice if there was a way to get similar improvements without
static linking, but I imagine it would be tough.  I’m not suggesting
anything for now, but maybe we could split the GHC package so that other
packages could reference the “base” library (125M) without referencing
the “ghc” and “Cabal” libraries (818M).  Ultimately it would be nice to
have a more general solution.

In the meantime, I think that this is fine.


-- Tim


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-09  4:29       ` Timothy Sample
@ 2020-08-12 12:21         ` Ricardo Wurmus
  2020-08-13 15:37         ` Ricardo Wurmus
  1 sibling, 0 replies; 24+ messages in thread
From: Ricardo Wurmus @ 2020-08-12 12:21 UTC (permalink / raw)
  To: Timothy Sample; +Cc: guix-devel


Timothy Sample <samplet@ngyro.com> writes:

> I just pushed “wip-haskell-updates-2” which integrates my work from
> <https://issues.guix.gnu.org/39309>.  I left the original branch intact
> to make it easy to compare.

Thank you!  I’ll work off of wip-haskell-updates-2 now.

> Basically, where you remove the “--extra-include-dirs” and
> “--extra-lib-dirs” arguments in configure, I preserve them and hide them
> behind a build system argument.  To do this, I split up the commit where
> you remove them into a refactor commit (where you remove “append” and
> just use quasiquoting), and a commit that removes “--bindir”.  My commit
> goes in the middle.
>
> Then, I remove the commits that fix up ghc-hslua, ghc-libyaml, and
> ghc-zlib, as that’s handled in my commit.
>
> The only other thing I did was move the shared libraries commit sooner,
> since it needs to be in place for the static output commit to work (at
> least nothing would build for me without it).

Sounds good.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-09  4:29       ` Timothy Sample
  2020-08-12 12:21         ` Ricardo Wurmus
@ 2020-08-13 15:37         ` Ricardo Wurmus
  2020-08-15  8:19           ` Ricardo Wurmus
  1 sibling, 1 reply; 24+ messages in thread
From: Ricardo Wurmus @ 2020-08-13 15:37 UTC (permalink / raw)
  To: Timothy Sample; +Cc: guix-devel


Timothy Sample <samplet@ngyro.com> writes:

> I just pushed “wip-haskell-updates-2” which integrates my work from
> <https://issues.guix.gnu.org/39309>.  I left the original branch intact
> to make it easy to compare.

I rebased this on top of “master” and pushed it as “wip-haskell”.  I
updated my changes to pandoc and pandoc-citeproc so that the statically
linked variants are now separate packages.

The evaluations on ci.guix.gnu.org failed due to a few minor bugs in the
extra-directories code (missing “match” case for inputs with declared
outputs, and a missing list wrapper in one package).  I fixed up that
commit and now wait for ci.guix.gnu.org to start building things:

    https://ci.guix.gnu.org/jobset/wip-haskell

-- 
Ricardo


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-13 15:37         ` Ricardo Wurmus
@ 2020-08-15  8:19           ` Ricardo Wurmus
  2020-08-17 15:22             ` Timothy Sample
  2020-08-24 15:38             ` Ludovic Courtès
  0 siblings, 2 replies; 24+ messages in thread
From: Ricardo Wurmus @ 2020-08-15  8:19 UTC (permalink / raw)
  To: Timothy Sample; +Cc: guix-devel


Ricardo Wurmus <rekado@elephly.net> writes:

> Timothy Sample <samplet@ngyro.com> writes:
>
>> I just pushed “wip-haskell-updates-2” which integrates my work from
>> <https://issues.guix.gnu.org/39309>.  I left the original branch intact
>> to make it easy to compare.
>
> I rebased this on top of “master” and pushed it as “wip-haskell”.  I
> updated my changes to pandoc and pandoc-citeproc so that the statically
> linked variants are now separate packages.
>
> The evaluations on ci.guix.gnu.org failed due to a few minor bugs in the
> extra-directories code (missing “match” case for inputs with declared
> outputs, and a missing list wrapper in one package).  I fixed up that
> commit and now wait for ci.guix.gnu.org to start building things:
>
>     https://ci.guix.gnu.org/jobset/wip-haskell

I just merged “wip-haskell” into “master” (after merging “master” into
“wip-haskell”).

-- 
Ricardo


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-15  8:19           ` Ricardo Wurmus
@ 2020-08-17 15:22             ` Timothy Sample
  2020-08-24 15:38             ` Ludovic Courtès
  1 sibling, 0 replies; 24+ messages in thread
From: Timothy Sample @ 2020-08-17 15:22 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

Hey Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> Timothy Sample <samplet@ngyro.com> writes:
>>
>>> I just pushed “wip-haskell-updates-2” which integrates my work from
>>> <https://issues.guix.gnu.org/39309>.  I left the original branch intact
>>> to make it easy to compare.
>>
>> I rebased this on top of “master” and pushed it as “wip-haskell”.  I
>> updated my changes to pandoc and pandoc-citeproc so that the statically
>> linked variants are now separate packages.
>>
>> The evaluations on ci.guix.gnu.org failed due to a few minor bugs in the
>> extra-directories code (missing “match” case for inputs with declared
>> outputs, and a missing list wrapper in one package).  I fixed up that
>> commit [...].

Sorry for the mistakes.  Thanks for fixing them up.  :)

> I just merged “wip-haskell” into “master” (after merging “master” into
> “wip-haskell”).

Fantastic!

I noticed a few surprising failures like Agda and git-annex.  With
git-annex, I had to create a fake "static" output to appease the daemon
(cf. e13fefbcffa50144d9301d67b61928ac3c964fb3).  It’s a little funny,
but I suppose can’t be helped until there is progress on
<https://issues.guix.gnu.org/41569>.

Is it true that now that “wip-haskell” is removed from Cuirass there’s
no longer any way to see the build failures?  I can’t find them, but my
Cuirass skills are rather mediocre.  :)  Either way, I have fast-enough
machine I can use to track down some of the other failures.  Maybe I
will try and do it along with the Stackage update.


-- Tim


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-15  8:19           ` Ricardo Wurmus
  2020-08-17 15:22             ` Timothy Sample
@ 2020-08-24 15:38             ` Ludovic Courtès
  1 sibling, 0 replies; 24+ messages in thread
From: Ludovic Courtès @ 2020-08-24 15:38 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: guix-devel

Hi!

Ricardo Wurmus <rekado@elephly.net> skribis:

> I just merged “wip-haskell” into “master” (after merging “master” into
> “wip-haskell”).

Awesome work, thank you!

Ludo’.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-07 15:59       ` Ricardo Wurmus
  2020-08-07 16:08         ` John Soo
@ 2020-08-24 15:44         ` Ludovic Courtès
  2020-08-24 17:54           ` Ricardo Wurmus
  1 sibling, 1 reply; 24+ messages in thread
From: Ludovic Courtès @ 2020-08-24 15:44 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: Jakub Kądziołka, guix-devel

Ricardo Wurmus <rekado@elephly.net> skribis:

> Yes, and for the case of pandoc it’s significant.  The closure of
> ghc-pandoc is >3GiB right now and with the changes it’s <200MiB.  This
> affects lots of R packages that need Rmarkdown, and lots of bioinfo
> packages.

Are those changes part of what was merged?  I get:

--8<---------------cut here---------------start------------->8---
$ guix describe
Generacio 153	Aug 24 2020 10:53:47	(nuna)
  guix c687299
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: c6872990b51971922f3064cba54ab752fcdc1559
$ guix size ghc-pandoc | tail -1
total: 2033.0 MiB
--8<---------------cut here---------------end--------------->8---

Or am I missing something?

Ludo’.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-24 15:44         ` Ludovic Courtès
@ 2020-08-24 17:54           ` Ricardo Wurmus
  2020-08-28 13:53             ` Ludovic Courtès
  0 siblings, 1 reply; 24+ messages in thread
From: Ricardo Wurmus @ 2020-08-24 17:54 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Jakub Kądziołka, guix-devel


Ludovic Courtès <ludo@gnu.org> writes:

> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Yes, and for the case of pandoc it’s significant.  The closure of
>> ghc-pandoc is >3GiB right now and with the changes it’s <200MiB.  This
>> affects lots of R packages that need Rmarkdown, and lots of bioinfo
>> packages.
>
> Are those changes part of what was merged?  I get:
>
> --8<---------------cut here---------------start------------->8---
> $ guix describe
> Generacio 153	Aug 24 2020 10:53:47	(nuna)
>   guix c687299
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     branch: master
>     commit: c6872990b51971922f3064cba54ab752fcdc1559
> $ guix size ghc-pandoc | tail -1
> total: 2033.0 MiB
> --8<---------------cut here---------------end--------------->8---
>
> Or am I missing something?

Try the “pandoc” package instead.  There are now two packages:
“ghc-pandoc” for the regular old shared library thing and “pandoc” for
this static linking abomination.

(Similarly for “ghc-pandoc-citeproc” and “pandoc-citeproc”.)

I should note that it doesn’t help with one case I had in mind: “guix
pack” still includes way too much, unfortunately, so there is no impact
on application bundle size yet.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-24 17:54           ` Ricardo Wurmus
@ 2020-08-28 13:53             ` Ludovic Courtès
  2020-08-28 14:05               ` John Soo
  0 siblings, 1 reply; 24+ messages in thread
From: Ludovic Courtès @ 2020-08-28 13:53 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: Jakub Kądziołka, guix-devel

Hi!

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Ricardo Wurmus <rekado@elephly.net> skribis:
>>
>>> Yes, and for the case of pandoc it’s significant.  The closure of
>>> ghc-pandoc is >3GiB right now and with the changes it’s <200MiB.  This
>>> affects lots of R packages that need Rmarkdown, and lots of bioinfo
>>> packages.
>>
>> Are those changes part of what was merged?  I get:
>>
>> --8<---------------cut here---------------start------------->8---
>> $ guix describe
>> Generacio 153	Aug 24 2020 10:53:47	(nuna)
>>   guix c687299
>>     repository URL: https://git.savannah.gnu.org/git/guix.git
>>     branch: master
>>     commit: c6872990b51971922f3064cba54ab752fcdc1559
>> $ guix size ghc-pandoc | tail -1
>> total: 2033.0 MiB
>> --8<---------------cut here---------------end--------------->8---
>>
>> Or am I missing something?
>
> Try the “pandoc” package instead.  There are now two packages:
> “ghc-pandoc” for the regular old shared library thing and “pandoc” for
> this static linking abomination.

Oh, the abomination is much nicer, thanks!  (Do we actually still need
‘ghc-pandoc’?)

> (Similarly for “ghc-pandoc-citeproc” and “pandoc-citeproc”.)
>
> I should note that it doesn’t help with one case I had in mind: “guix
> pack” still includes way too much, unfortunately, so there is no impact
> on application bundle size yet.

How come?  It should include exactly what ‘guix size pandoc’ shows, no?

Thanks!

Ludo’.


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-28 13:53             ` Ludovic Courtès
@ 2020-08-28 14:05               ` John Soo
  2020-08-28 17:29                 ` Timothy Sample
  0 siblings, 1 reply; 24+ messages in thread
From: John Soo @ 2020-08-28 14:05 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Jakub Kądziołka, guix-devel

On another note:

Does anyone know why idris, agda, and purescript are failing?

I have only been able to do very little recently to look at them.

- John


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-28 14:05               ` John Soo
@ 2020-08-28 17:29                 ` Timothy Sample
  2020-08-29 15:34                   ` Timothy Sample
  0 siblings, 1 reply; 24+ messages in thread
From: Timothy Sample @ 2020-08-28 17:29 UTC (permalink / raw)
  To: John Soo; +Cc: Jakub Kądziołka, guix-devel

Hi John,

John Soo <jsoo1@asu.edu> writes:

> Does anyone know why idris, agda, and purescript are failing?

The really short version is that we now use shared libraries for Haskell
dependencies, which mostly works, except for some of the more
complicated builds.  I did fix Agda a little while ago, so I think it’s
okay now.  The issue was it couldn’t run the Agda compiler because it
didn’t know where it’s own shared libraries were before installing.  I
fixed it by setting “LD_LIBRARY_PATH” during the build.  Idris has a
similar problem, but not similar enough that the same solution works.  I
imagine PureScript is in the same boat, but I didn’t know about it until
now.  This makes me wonder about Elm, too.

> I have only been able to do very little recently to look at them.

Me too.  I‘ll take a look at PureScript and check Elm, since maybe the
Agda fix will work for them.  I’m not sure about Idris – it needs a bit
more effort.


-- Tim


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-28 17:29                 ` Timothy Sample
@ 2020-08-29 15:34                   ` Timothy Sample
  2020-08-29 15:54                     ` John Soo
  0 siblings, 1 reply; 24+ messages in thread
From: Timothy Sample @ 2020-08-29 15:34 UTC (permalink / raw)
  To: John Soo; +Cc: Jakub Kądziołka, guix-devel

Hi again,

Timothy Sample <samplet@ngyro.com> writes:

> I‘ll take a look at PureScript and check Elm, since maybe the Agda fix
> will work for them.  I’m not sure about Idris – it needs a bit more
> effort.

Good news!  Elm and PureScript are fine, and I just pushed a fix for
Idris: 5aaa1995922285d53a97824f2b83bdb71ec9fdd0.


-- Tim


^ permalink raw reply	[flat|nested] 24+ messages in thread

* Re: merge wip-haskell?
  2020-08-29 15:34                   ` Timothy Sample
@ 2020-08-29 15:54                     ` John Soo
  0 siblings, 0 replies; 24+ messages in thread
From: John Soo @ 2020-08-29 15:54 UTC (permalink / raw)
  To: Timothy Sample; +Cc: Jakub Kądziołka, guix-devel

Nice!  Thanks Tim!


^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2020-08-29 15:54 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-06  8:13 merge wip-haskell? Ricardo Wurmus
2020-08-06 13:45 ` John Soo
2020-08-07 17:46   ` Timothy Sample
2020-08-07 19:53     ` Ricardo Wurmus
2020-08-09  4:29       ` Timothy Sample
2020-08-12 12:21         ` Ricardo Wurmus
2020-08-13 15:37         ` Ricardo Wurmus
2020-08-15  8:19           ` Ricardo Wurmus
2020-08-17 15:22             ` Timothy Sample
2020-08-24 15:38             ` Ludovic Courtès
2020-08-07 15:03 ` Jakub Kądziołka
2020-08-07 15:12   ` John Soo
2020-08-07 15:46     ` Jakub Kądziołka
2020-08-07 15:59       ` Ricardo Wurmus
2020-08-07 16:08         ` John Soo
2020-08-24 15:44         ` Ludovic Courtès
2020-08-24 17:54           ` Ricardo Wurmus
2020-08-28 13:53             ` Ludovic Courtès
2020-08-28 14:05               ` John Soo
2020-08-28 17:29                 ` Timothy Sample
2020-08-29 15:34                   ` Timothy Sample
2020-08-29 15:54                     ` John Soo
2020-08-07 16:27   ` Ricardo Wurmus
2020-08-07 16:55     ` Jakub Kądziołka

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).