* merge wip-haskell? @ 2020-08-06 8:13 Ricardo Wurmus 2020-08-06 13:45 ` John Soo 2020-08-07 15:03 ` Jakub Kądziołka 0 siblings, 2 replies; 24+ messages in thread From: Ricardo Wurmus @ 2020-08-06 8:13 UTC (permalink / raw) To: guix-devel Hey there, wip-haskell contains commits that do a number of things aimed to reduce the closure of packages: 1) make the “out” (and “lib”) output independent from “doc” 2) add a “doc” output to more packages 3) add a “static” output for all Haskell packages containing the “.a” files 4) change ghc-pandoc (and ghc-pandoc-citeproc) to use static linking Number 1 required some smelly hackery: the generated configuration file is edited to remove the “haddock-html” field. One unfortunate effect of doing this and moving the .haddock files is that there are now complaints about unresolvable links in generated documentation. I don’t know if we can avoid this, but it seems like a small price to pay for independent “doc” outputs. (Otherwise we’d have to download huge “doc” outputs even if we don’t want them.) Number 3 required circumventing bug 41569. Number 4 is by far the ugliest change of them all. In order to statically link packages we need to add all the “static” outputs of all Haskell inputs *and* the “static” outputs of *their* Haskell inputs. This is not easily accomplished, so I ended up using “package-closure” on all direct inputs, and then filtered the result to packages with names starting with “ghc-”. If there was a more appropriate tool I’d use it, but I don’t think it exists. The result is a much reduced closure for ghc-pandoc and all packages using it (such as R markdown). We should probably rename “ghc-pandoc” to “pandoc”, while we’re at it, because now the package contains the executable. I suppose we could change this so that “ghc-pandoc” is the usual library package with a new “pandoc” package inheriting from “ghc-pandoc”. I’ll give that a try soon. I’d be happy to hear your comments about all of this, and I’m looking forward to merging this branch into “master” soon. -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-06 8:13 merge wip-haskell? Ricardo Wurmus @ 2020-08-06 13:45 ` John Soo 2020-08-07 17:46 ` Timothy Sample 2020-08-07 15:03 ` Jakub Kądziołka 1 sibling, 1 reply; 24+ messages in thread From: John Soo @ 2020-08-06 13:45 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: guix-devel Hi Ricardo, Nice! Sounds good to me. There are a couple other bits of work I’d like to see make it in. I believe there was also some work being done to de-duplicate flags sent to gcc sent by ghc (this was the only thing keeping stack from building). I hope that can make it in, too! If there is any way we can bump the default ghc to 8.8.x, that would be excellent, too. I think something like that makes sense on staging though. Thanks for working on it! - John On Aug 6, 2020, at 1:14 AM, Ricardo Wurmus <rekado@elephly.net> wrote: Hey there, wip-haskell contains commits that do a number of things aimed to reduce the closure of packages: 1) make the “out” (and “lib”) output independent from “doc” 2) add a “doc” output to more packages 3) add a “static” output for all Haskell packages containing the “.a” files 4) change ghc-pandoc (and ghc-pandoc-citeproc) to use static linking Number 1 required some smelly hackery: the generated configuration file is edited to remove the “haddock-html” field. One unfortunate effect of doing this and moving the .haddock files is that there are now complaints about unresolvable links in generated documentation. I don’t know if we can avoid this, but it seems like a small price to pay for independent “doc” outputs. (Otherwise we’d have to download huge “doc” outputs even if we don’t want them.) Number 3 required circumventing bug 41569. Number 4 is by far the ugliest change of them all. In order to statically link packages we need to add all the “static” outputs of all Haskell inputs *and* the “static” outputs of *their* Haskell inputs. This is not easily accomplished, so I ended up using “package-closure” on all direct inputs, and then filtered the result to packages with names starting with “ghc-”. If there was a more appropriate tool I’d use it, but I don’t think it exists. The result is a much reduced closure for ghc-pandoc and all packages using it (such as R markdown). We should probably rename “ghc-pandoc” to “pandoc”, while we’re at it, because now the package contains the executable. I suppose we could change this so that “ghc-pandoc” is the usual library package with a new “pandoc” package inheriting from “ghc-pandoc”. I’ll give that a try soon. I’d be happy to hear your comments about all of this, and I’m looking forward to merging this branch into “master” soon. -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-06 13:45 ` John Soo @ 2020-08-07 17:46 ` Timothy Sample 2020-08-07 19:53 ` Ricardo Wurmus 0 siblings, 1 reply; 24+ messages in thread From: Timothy Sample @ 2020-08-07 17:46 UTC (permalink / raw) To: John Soo, Ricardo Wurmus; +Cc: guix-devel Hi John and Ricardo, John Soo <jsoo1@asu.edu> writes: > I believe there was also some work being done to de-duplicate flags > sent to gcc sent by ghc (this was the only thing keeping stack from > building). > I hope that can make it in, too! I agree, and sorry to you John for the delay. I’m not using much Haskell these days, so I’m having trouble keeping Haskell stuff on my radar. The last patch I sent to <https://issues.guix.gnu.org/39309> is about as good as I can come up with, so I will apply it to the branch before the merge. Ricardo, I will try and do this tonight or tomorrow, and will look over your changes then as well. > If there is any way we can bump the default ghc to 8.8.x, that would > be excellent, too. I think something like that makes sense on staging > though. I would love for this to happen, but I think it would needlessly delay Ricardo’s changes. Fortunately, one can build all of our Haskell packages overnight on a modest computer, so it’s no big deal to merge the branch now and then merge the GHC 8.8.x updates whenever they’re ready. Also, it looks like “wip-haskell-updates” is no longer being built by the CI infrastructure. Since the branch triggers a rebuild of all the Haskell packages, it should be built before merging, right? -- Tim ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-07 17:46 ` Timothy Sample @ 2020-08-07 19:53 ` Ricardo Wurmus 2020-08-09 4:29 ` Timothy Sample 0 siblings, 1 reply; 24+ messages in thread From: Ricardo Wurmus @ 2020-08-07 19:53 UTC (permalink / raw) To: Timothy Sample; +Cc: guix-devel Timothy Sample <samplet@ngyro.com> writes: > Also, it looks like “wip-haskell-updates” is no longer being built by > the CI infrastructure. Since the branch triggers a rebuild of all the > Haskell packages, it should be built before merging, right? Yes, I’ll rebase it on top of “master” and add the specification to ci.guix.gnu.org well before the merge. -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-07 19:53 ` Ricardo Wurmus @ 2020-08-09 4:29 ` Timothy Sample 2020-08-12 12:21 ` Ricardo Wurmus 2020-08-13 15:37 ` Ricardo Wurmus 0 siblings, 2 replies; 24+ messages in thread From: Timothy Sample @ 2020-08-09 4:29 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: guix-devel Hi Ricardo, Ricardo Wurmus <rekado@elephly.net> writes: > Timothy Sample <samplet@ngyro.com> writes: > >> Also, it looks like “wip-haskell-updates” is no longer being built by >> the CI infrastructure. Since the branch triggers a rebuild of all the >> Haskell packages, it should be built before merging, right? > > Yes, I’ll rebase it on top of “master” and add the specification to > ci.guix.gnu.org well before the merge. Excellent. I just pushed “wip-haskell-updates-2” which integrates my work from <https://issues.guix.gnu.org/39309>. I left the original branch intact to make it easy to compare. Basically, where you remove the “--extra-include-dirs” and “--extra-lib-dirs” arguments in configure, I preserve them and hide them behind a build system argument. To do this, I split up the commit where you remove them into a refactor commit (where you remove “append” and just use quasiquoting), and a commit that removes “--bindir”. My commit goes in the middle. Then, I remove the commits that fix up ghc-hslua, ghc-libyaml, and ghc-zlib, as that’s handled in my commit. The only other thing I did was move the shared libraries commit sooner, since it needs to be in place for the static output commit to work (at least nothing would build for me without it). With respect to the substance of your changes, I think the results are worth the ugliness! Keeping “ghc-pandoc” as the “normal” package and using “pandoc” for the statically linked one makes sense if feasible. Unfortunately, I don’t know of a better way to get all the static libraries in place. It would be nice if there was a way to get similar improvements without static linking, but I imagine it would be tough. I’m not suggesting anything for now, but maybe we could split the GHC package so that other packages could reference the “base” library (125M) without referencing the “ghc” and “Cabal” libraries (818M). Ultimately it would be nice to have a more general solution. In the meantime, I think that this is fine. -- Tim ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-09 4:29 ` Timothy Sample @ 2020-08-12 12:21 ` Ricardo Wurmus 2020-08-13 15:37 ` Ricardo Wurmus 1 sibling, 0 replies; 24+ messages in thread From: Ricardo Wurmus @ 2020-08-12 12:21 UTC (permalink / raw) To: Timothy Sample; +Cc: guix-devel Timothy Sample <samplet@ngyro.com> writes: > I just pushed “wip-haskell-updates-2” which integrates my work from > <https://issues.guix.gnu.org/39309>. I left the original branch intact > to make it easy to compare. Thank you! I’ll work off of wip-haskell-updates-2 now. > Basically, where you remove the “--extra-include-dirs” and > “--extra-lib-dirs” arguments in configure, I preserve them and hide them > behind a build system argument. To do this, I split up the commit where > you remove them into a refactor commit (where you remove “append” and > just use quasiquoting), and a commit that removes “--bindir”. My commit > goes in the middle. > > Then, I remove the commits that fix up ghc-hslua, ghc-libyaml, and > ghc-zlib, as that’s handled in my commit. > > The only other thing I did was move the shared libraries commit sooner, > since it needs to be in place for the static output commit to work (at > least nothing would build for me without it). Sounds good. -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-09 4:29 ` Timothy Sample 2020-08-12 12:21 ` Ricardo Wurmus @ 2020-08-13 15:37 ` Ricardo Wurmus 2020-08-15 8:19 ` Ricardo Wurmus 1 sibling, 1 reply; 24+ messages in thread From: Ricardo Wurmus @ 2020-08-13 15:37 UTC (permalink / raw) To: Timothy Sample; +Cc: guix-devel Timothy Sample <samplet@ngyro.com> writes: > I just pushed “wip-haskell-updates-2” which integrates my work from > <https://issues.guix.gnu.org/39309>. I left the original branch intact > to make it easy to compare. I rebased this on top of “master” and pushed it as “wip-haskell”. I updated my changes to pandoc and pandoc-citeproc so that the statically linked variants are now separate packages. The evaluations on ci.guix.gnu.org failed due to a few minor bugs in the extra-directories code (missing “match” case for inputs with declared outputs, and a missing list wrapper in one package). I fixed up that commit and now wait for ci.guix.gnu.org to start building things: https://ci.guix.gnu.org/jobset/wip-haskell -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-13 15:37 ` Ricardo Wurmus @ 2020-08-15 8:19 ` Ricardo Wurmus 2020-08-17 15:22 ` Timothy Sample 2020-08-24 15:38 ` Ludovic Courtès 0 siblings, 2 replies; 24+ messages in thread From: Ricardo Wurmus @ 2020-08-15 8:19 UTC (permalink / raw) To: Timothy Sample; +Cc: guix-devel Ricardo Wurmus <rekado@elephly.net> writes: > Timothy Sample <samplet@ngyro.com> writes: > >> I just pushed “wip-haskell-updates-2” which integrates my work from >> <https://issues.guix.gnu.org/39309>. I left the original branch intact >> to make it easy to compare. > > I rebased this on top of “master” and pushed it as “wip-haskell”. I > updated my changes to pandoc and pandoc-citeproc so that the statically > linked variants are now separate packages. > > The evaluations on ci.guix.gnu.org failed due to a few minor bugs in the > extra-directories code (missing “match” case for inputs with declared > outputs, and a missing list wrapper in one package). I fixed up that > commit and now wait for ci.guix.gnu.org to start building things: > > https://ci.guix.gnu.org/jobset/wip-haskell I just merged “wip-haskell” into “master” (after merging “master” into “wip-haskell”). -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-15 8:19 ` Ricardo Wurmus @ 2020-08-17 15:22 ` Timothy Sample 2020-08-24 15:38 ` Ludovic Courtès 1 sibling, 0 replies; 24+ messages in thread From: Timothy Sample @ 2020-08-17 15:22 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: guix-devel Hey Ricardo, Ricardo Wurmus <rekado@elephly.net> writes: > Ricardo Wurmus <rekado@elephly.net> writes: > >> Timothy Sample <samplet@ngyro.com> writes: >> >>> I just pushed “wip-haskell-updates-2” which integrates my work from >>> <https://issues.guix.gnu.org/39309>. I left the original branch intact >>> to make it easy to compare. >> >> I rebased this on top of “master” and pushed it as “wip-haskell”. I >> updated my changes to pandoc and pandoc-citeproc so that the statically >> linked variants are now separate packages. >> >> The evaluations on ci.guix.gnu.org failed due to a few minor bugs in the >> extra-directories code (missing “match” case for inputs with declared >> outputs, and a missing list wrapper in one package). I fixed up that >> commit [...]. Sorry for the mistakes. Thanks for fixing them up. :) > I just merged “wip-haskell” into “master” (after merging “master” into > “wip-haskell”). Fantastic! I noticed a few surprising failures like Agda and git-annex. With git-annex, I had to create a fake "static" output to appease the daemon (cf. e13fefbcffa50144d9301d67b61928ac3c964fb3). It’s a little funny, but I suppose can’t be helped until there is progress on <https://issues.guix.gnu.org/41569>. Is it true that now that “wip-haskell” is removed from Cuirass there’s no longer any way to see the build failures? I can’t find them, but my Cuirass skills are rather mediocre. :) Either way, I have fast-enough machine I can use to track down some of the other failures. Maybe I will try and do it along with the Stackage update. -- Tim ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-15 8:19 ` Ricardo Wurmus 2020-08-17 15:22 ` Timothy Sample @ 2020-08-24 15:38 ` Ludovic Courtès 1 sibling, 0 replies; 24+ messages in thread From: Ludovic Courtès @ 2020-08-24 15:38 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: guix-devel Hi! Ricardo Wurmus <rekado@elephly.net> skribis: > I just merged “wip-haskell” into “master” (after merging “master” into > “wip-haskell”). Awesome work, thank you! Ludo’. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-06 8:13 merge wip-haskell? Ricardo Wurmus 2020-08-06 13:45 ` John Soo @ 2020-08-07 15:03 ` Jakub Kądziołka 2020-08-07 15:12 ` John Soo 2020-08-07 16:27 ` Ricardo Wurmus 1 sibling, 2 replies; 24+ messages in thread From: Jakub Kądziołka @ 2020-08-07 15:03 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 993 bytes --] On Thu, Aug 06, 2020 at 10:13:46AM +0200, Ricardo Wurmus wrote: > Number 4 is by far the ugliest change of them all. In order to > statically link packages we need to add all the “static” outputs of all > Haskell inputs *and* the “static” outputs of *their* Haskell inputs. > This is not easily accomplished, so I ended up using “package-closure” > on all direct inputs, and then filtered the result to packages with > names starting with “ghc-”. If there was a more appropriate tool I’d > use it, but I don’t think it exists. Perhaps we should work on making propagated-inputs per-output? That way, :static could propagate the :static output of the dependencies. This would also be useful in other situations. For example, a package might contain both a binary and a library, and the library must propagate its dependencies to make the header files work. I don't know what a good syntax for this would be. Thoughts? Regards, Jakub Kądziołka [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-07 15:03 ` Jakub Kądziołka @ 2020-08-07 15:12 ` John Soo 2020-08-07 15:46 ` Jakub Kądziołka 2020-08-07 16:27 ` Ricardo Wurmus 1 sibling, 1 reply; 24+ messages in thread From: John Soo @ 2020-08-07 15:12 UTC (permalink / raw) To: Jakub Kądziołka; +Cc: guix-devel Hi Jakub, I could see splitting the static output being useful but I would rather wait until some evidence that the closure size would be too large. Also I’m not sure propagation is necessary for dependents to find libraries or use paths from an input. Thoughts? John On Aug 7, 2020, at 8:04 AM, Jakub Kądziołka <kuba@kadziolka.net> wrote: On Thu, Aug 06, 2020 at 10:13:46AM +0200, Ricardo Wurmus wrote: > Number 4 is by far the ugliest change of them all. In order to > statically link packages we need to add all the “static” outputs of all > Haskell inputs *and* the “static” outputs of *their* Haskell inputs. > This is not easily accomplished, so I ended up using “package-closure” > on all direct inputs, and then filtered the result to packages with > names starting with “ghc-”. If there was a more appropriate tool I’d > use it, but I don’t think it exists. Perhaps we should work on making propagated-inputs per-output? That way, :static could propagate the :static output of the dependencies. This would also be useful in other situations. For example, a package might contain both a binary and a library, and the library must propagate its dependencies to make the header files work. I don't know what a good syntax for this would be. Thoughts? Regards, Jakub Kądziołka ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-07 15:12 ` John Soo @ 2020-08-07 15:46 ` Jakub Kądziołka 2020-08-07 15:59 ` Ricardo Wurmus 0 siblings, 1 reply; 24+ messages in thread From: Jakub Kądziołka @ 2020-08-07 15:46 UTC (permalink / raw) To: John Soo; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 309 bytes --] On Fri, Aug 07, 2020 at 08:12:36AM -0700, John Soo wrote: > I would rather wait until some evidence that the closure size would be too large. Also I’m not sure propagation is necessary for dependents to find libraries or use paths from an input. Ricardo already explained that this is indeed the case. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-07 15:46 ` Jakub Kądziołka @ 2020-08-07 15:59 ` Ricardo Wurmus 2020-08-07 16:08 ` John Soo 2020-08-24 15:44 ` Ludovic Courtès 0 siblings, 2 replies; 24+ messages in thread From: Ricardo Wurmus @ 2020-08-07 15:59 UTC (permalink / raw) To: Jakub Kądziołka; +Cc: guix-devel Jakub Kądziołka <kuba@kadziolka.net> writes: > On Fri, Aug 07, 2020 at 08:12:36AM -0700, John Soo wrote: >> I would rather wait until some evidence that the closure size would be too large. Also I’m not sure propagation is necessary for dependents to find libraries or use paths from an input. > > Ricardo already explained that this is indeed the case. Yes, and for the case of pandoc it’s significant. The closure of ghc-pandoc is >3GiB right now and with the changes it’s <200MiB. This affects lots of R packages that need Rmarkdown, and lots of bioinfo packages. There's no doubt that moving the static libs to their own output has a significant impact on closure size. -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-07 15:59 ` Ricardo Wurmus @ 2020-08-07 16:08 ` John Soo 2020-08-24 15:44 ` Ludovic Courtès 1 sibling, 0 replies; 24+ messages in thread From: John Soo @ 2020-08-07 16:08 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: Jakub Kądziołka, guix-devel Hi Ricardo and Jakub, Ah ok. Sorry I had forgotten the point of the thread. Sounds like a plan! - John On Aug 7, 2020, at 8:59 AM, Ricardo Wurmus <rekado@elephly.net> wrote: Jakub Kądziołka <kuba@kadziolka.net> writes: >> On Fri, Aug 07, 2020 at 08:12:36AM -0700, John Soo wrote: >> I would rather wait until some evidence that the closure size would be too large. Also I’m not sure propagation is necessary for dependents to find libraries or use paths from an input. > > Ricardo already explained that this is indeed the case. Yes, and for the case of pandoc it’s significant. The closure of ghc-pandoc is >3GiB right now and with the changes it’s <200MiB. This affects lots of R packages that need Rmarkdown, and lots of bioinfo packages. There's no doubt that moving the static libs to their own output has a significant impact on closure size. -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-07 15:59 ` Ricardo Wurmus 2020-08-07 16:08 ` John Soo @ 2020-08-24 15:44 ` Ludovic Courtès 2020-08-24 17:54 ` Ricardo Wurmus 1 sibling, 1 reply; 24+ messages in thread From: Ludovic Courtès @ 2020-08-24 15:44 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: Jakub Kądziołka, guix-devel Ricardo Wurmus <rekado@elephly.net> skribis: > Yes, and for the case of pandoc it’s significant. The closure of > ghc-pandoc is >3GiB right now and with the changes it’s <200MiB. This > affects lots of R packages that need Rmarkdown, and lots of bioinfo > packages. Are those changes part of what was merged? I get: --8<---------------cut here---------------start------------->8--- $ guix describe Generacio 153 Aug 24 2020 10:53:47 (nuna) guix c687299 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: c6872990b51971922f3064cba54ab752fcdc1559 $ guix size ghc-pandoc | tail -1 total: 2033.0 MiB --8<---------------cut here---------------end--------------->8--- Or am I missing something? Ludo’. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-24 15:44 ` Ludovic Courtès @ 2020-08-24 17:54 ` Ricardo Wurmus 2020-08-28 13:53 ` Ludovic Courtès 0 siblings, 1 reply; 24+ messages in thread From: Ricardo Wurmus @ 2020-08-24 17:54 UTC (permalink / raw) To: Ludovic Courtès; +Cc: Jakub Kądziołka, guix-devel Ludovic Courtès <ludo@gnu.org> writes: > Ricardo Wurmus <rekado@elephly.net> skribis: > >> Yes, and for the case of pandoc it’s significant. The closure of >> ghc-pandoc is >3GiB right now and with the changes it’s <200MiB. This >> affects lots of R packages that need Rmarkdown, and lots of bioinfo >> packages. > > Are those changes part of what was merged? I get: > > --8<---------------cut here---------------start------------->8--- > $ guix describe > Generacio 153 Aug 24 2020 10:53:47 (nuna) > guix c687299 > repository URL: https://git.savannah.gnu.org/git/guix.git > branch: master > commit: c6872990b51971922f3064cba54ab752fcdc1559 > $ guix size ghc-pandoc | tail -1 > total: 2033.0 MiB > --8<---------------cut here---------------end--------------->8--- > > Or am I missing something? Try the “pandoc” package instead. There are now two packages: “ghc-pandoc” for the regular old shared library thing and “pandoc” for this static linking abomination. (Similarly for “ghc-pandoc-citeproc” and “pandoc-citeproc”.) I should note that it doesn’t help with one case I had in mind: “guix pack” still includes way too much, unfortunately, so there is no impact on application bundle size yet. -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-24 17:54 ` Ricardo Wurmus @ 2020-08-28 13:53 ` Ludovic Courtès 2020-08-28 14:05 ` John Soo 0 siblings, 1 reply; 24+ messages in thread From: Ludovic Courtès @ 2020-08-28 13:53 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: Jakub Kądziołka, guix-devel Hi! Ricardo Wurmus <rekado@elephly.net> skribis: > Ludovic Courtès <ludo@gnu.org> writes: > >> Ricardo Wurmus <rekado@elephly.net> skribis: >> >>> Yes, and for the case of pandoc it’s significant. The closure of >>> ghc-pandoc is >3GiB right now and with the changes it’s <200MiB. This >>> affects lots of R packages that need Rmarkdown, and lots of bioinfo >>> packages. >> >> Are those changes part of what was merged? I get: >> >> --8<---------------cut here---------------start------------->8--- >> $ guix describe >> Generacio 153 Aug 24 2020 10:53:47 (nuna) >> guix c687299 >> repository URL: https://git.savannah.gnu.org/git/guix.git >> branch: master >> commit: c6872990b51971922f3064cba54ab752fcdc1559 >> $ guix size ghc-pandoc | tail -1 >> total: 2033.0 MiB >> --8<---------------cut here---------------end--------------->8--- >> >> Or am I missing something? > > Try the “pandoc” package instead. There are now two packages: > “ghc-pandoc” for the regular old shared library thing and “pandoc” for > this static linking abomination. Oh, the abomination is much nicer, thanks! (Do we actually still need ‘ghc-pandoc’?) > (Similarly for “ghc-pandoc-citeproc” and “pandoc-citeproc”.) > > I should note that it doesn’t help with one case I had in mind: “guix > pack” still includes way too much, unfortunately, so there is no impact > on application bundle size yet. How come? It should include exactly what ‘guix size pandoc’ shows, no? Thanks! Ludo’. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-28 13:53 ` Ludovic Courtès @ 2020-08-28 14:05 ` John Soo 2020-08-28 17:29 ` Timothy Sample 0 siblings, 1 reply; 24+ messages in thread From: John Soo @ 2020-08-28 14:05 UTC (permalink / raw) To: Ludovic Courtès; +Cc: Jakub Kądziołka, guix-devel On another note: Does anyone know why idris, agda, and purescript are failing? I have only been able to do very little recently to look at them. - John ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-28 14:05 ` John Soo @ 2020-08-28 17:29 ` Timothy Sample 2020-08-29 15:34 ` Timothy Sample 0 siblings, 1 reply; 24+ messages in thread From: Timothy Sample @ 2020-08-28 17:29 UTC (permalink / raw) To: John Soo; +Cc: Jakub Kądziołka, guix-devel Hi John, John Soo <jsoo1@asu.edu> writes: > Does anyone know why idris, agda, and purescript are failing? The really short version is that we now use shared libraries for Haskell dependencies, which mostly works, except for some of the more complicated builds. I did fix Agda a little while ago, so I think it’s okay now. The issue was it couldn’t run the Agda compiler because it didn’t know where it’s own shared libraries were before installing. I fixed it by setting “LD_LIBRARY_PATH” during the build. Idris has a similar problem, but not similar enough that the same solution works. I imagine PureScript is in the same boat, but I didn’t know about it until now. This makes me wonder about Elm, too. > I have only been able to do very little recently to look at them. Me too. I‘ll take a look at PureScript and check Elm, since maybe the Agda fix will work for them. I’m not sure about Idris – it needs a bit more effort. -- Tim ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-28 17:29 ` Timothy Sample @ 2020-08-29 15:34 ` Timothy Sample 2020-08-29 15:54 ` John Soo 0 siblings, 1 reply; 24+ messages in thread From: Timothy Sample @ 2020-08-29 15:34 UTC (permalink / raw) To: John Soo; +Cc: Jakub Kądziołka, guix-devel Hi again, Timothy Sample <samplet@ngyro.com> writes: > I‘ll take a look at PureScript and check Elm, since maybe the Agda fix > will work for them. I’m not sure about Idris – it needs a bit more > effort. Good news! Elm and PureScript are fine, and I just pushed a fix for Idris: 5aaa1995922285d53a97824f2b83bdb71ec9fdd0. -- Tim ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-29 15:34 ` Timothy Sample @ 2020-08-29 15:54 ` John Soo 0 siblings, 0 replies; 24+ messages in thread From: John Soo @ 2020-08-29 15:54 UTC (permalink / raw) To: Timothy Sample; +Cc: Jakub Kądziołka, guix-devel Nice! Thanks Tim! ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-07 15:03 ` Jakub Kądziołka 2020-08-07 15:12 ` John Soo @ 2020-08-07 16:27 ` Ricardo Wurmus 2020-08-07 16:55 ` Jakub Kądziołka 1 sibling, 1 reply; 24+ messages in thread From: Ricardo Wurmus @ 2020-08-07 16:27 UTC (permalink / raw) To: Jakub Kądziołka; +Cc: guix-devel Jakub Kądziołka <kuba@kadziolka.net> writes: > On Thu, Aug 06, 2020 at 10:13:46AM +0200, Ricardo Wurmus wrote: >> Number 4 is by far the ugliest change of them all. In order to >> statically link packages we need to add all the “static” outputs of all >> Haskell inputs *and* the “static” outputs of *their* Haskell inputs. >> This is not easily accomplished, so I ended up using “package-closure” >> on all direct inputs, and then filtered the result to packages with >> names starting with “ghc-”. If there was a more appropriate tool I’d >> use it, but I don’t think it exists. > > Perhaps we should work on making propagated-inputs per-output? That way, > :static could propagate the :static output of the dependencies. > > This would also be useful in other situations. For example, a package > might contain both a binary and a library, and the library must > propagate its dependencies to make the header files work. > > I don't know what a good syntax for this would be. The binary / library problem is particularly interesting to me as we have a few Python bioinfo tools that also work as libraries. When used as a library we need propagation, but when uses as command line tools propagation is not needed (and can be harmful due to the effects on PYTHONPATH). Syntax seems less important than verifying that this actually works, though. Alas, as someone who has committed and encouraged syntactic crimes with excessive reader macros, I think my vote on syntax carries no weight :) -- Ricardo ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: merge wip-haskell? 2020-08-07 16:27 ` Ricardo Wurmus @ 2020-08-07 16:55 ` Jakub Kądziołka 0 siblings, 0 replies; 24+ messages in thread From: Jakub Kądziołka @ 2020-08-07 16:55 UTC (permalink / raw) To: Ricardo Wurmus; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 2996 bytes --] On Fri, Aug 07, 2020 at 06:27:21PM +0200, Ricardo Wurmus wrote: > > Jakub Kądziołka <kuba@kadziolka.net> writes: > > > On Thu, Aug 06, 2020 at 10:13:46AM +0200, Ricardo Wurmus wrote: > >> Number 4 is by far the ugliest change of them all. In order to > >> statically link packages we need to add all the “static” outputs of all > >> Haskell inputs *and* the “static” outputs of *their* Haskell inputs. > >> This is not easily accomplished, so I ended up using “package-closure” > >> on all direct inputs, and then filtered the result to packages with > >> names starting with “ghc-”. If there was a more appropriate tool I’d > >> use it, but I don’t think it exists. > > > > Perhaps we should work on making propagated-inputs per-output? That way, > > :static could propagate the :static output of the dependencies. > > > > This would also be useful in other situations. For example, a package > > might contain both a binary and a library, and the library must > > propagate its dependencies to make the header files work. > > > > I don't know what a good syntax for this would be. > > The binary / library problem is particularly interesting to me as we > have a few Python bioinfo tools that also work as libraries. When used > as a library we need propagation, but when uses as command line tools > propagation is not needed (and can be harmful due to the effects on > PYTHONPATH). > > Syntax seems less important than verifying that this actually works, > though. One still needs to pick a variant, even at random, to begin implementation, which is hard when no options have been suggested :) I suppose that we could add a list of outputs each propagated-input applies to, putting it after the package and its output? (this is unambiguous as we can check whether the third element of the list is a string, or a list) (propagated-inputs `(("foo-static" ,foo "static" ("static")) ("bar" ,bar ("static")))) However, this has the issue of being somewhat ad-hoc, and other extensions to the list of inputs might become troublesome. This motivates my second proposal: (propagated-inputs `(("foo-static" ,foo "static" #:propagate-to ("static")) ("bar" ,bar #:propagate-to ("static")))) On the other hand, this variant feels somewhat over-engineered. After all, we can always change the syntax if it becomes actually necessary. There's also the option of adding another field to the package record: (propagated-inputs-for-outputs ;; name up to bikeshedding :) `(("static" (("foo-static" ,foo "static") ("bar ,bar))))) > Alas, as someone who has committed and encouraged syntactic > crimes with excessive reader macros, I think my vote on syntax carries > no weight :) Well, the number of reader macro crimes encouraged seems to correlate with experience, so I would still like to ask for your opinion. Regards, Jakub Kądziołka [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2020-08-29 15:54 UTC | newest] Thread overview: 24+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-08-06 8:13 merge wip-haskell? Ricardo Wurmus 2020-08-06 13:45 ` John Soo 2020-08-07 17:46 ` Timothy Sample 2020-08-07 19:53 ` Ricardo Wurmus 2020-08-09 4:29 ` Timothy Sample 2020-08-12 12:21 ` Ricardo Wurmus 2020-08-13 15:37 ` Ricardo Wurmus 2020-08-15 8:19 ` Ricardo Wurmus 2020-08-17 15:22 ` Timothy Sample 2020-08-24 15:38 ` Ludovic Courtès 2020-08-07 15:03 ` Jakub Kądziołka 2020-08-07 15:12 ` John Soo 2020-08-07 15:46 ` Jakub Kądziołka 2020-08-07 15:59 ` Ricardo Wurmus 2020-08-07 16:08 ` John Soo 2020-08-24 15:44 ` Ludovic Courtès 2020-08-24 17:54 ` Ricardo Wurmus 2020-08-28 13:53 ` Ludovic Courtès 2020-08-28 14:05 ` John Soo 2020-08-28 17:29 ` Timothy Sample 2020-08-29 15:34 ` Timothy Sample 2020-08-29 15:54 ` John Soo 2020-08-07 16:27 ` Ricardo Wurmus 2020-08-07 16:55 ` Jakub Kądziołka
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).