bug#25449: GHC packages take a long time to install

bug#25449: GHC packages take a long time to install

[David Craven]

Hi,

time guix build ghc-attoparsec
warning: failed to install locale: Invalid argument
substitute: warning: failed to install locale: Invalid argument
/gnu/store/qb6y8294zp3la57y1hm02hyf2ck0giwp-ghc-attoparsec-0.13.0.1
$HOME/guix/pre-inst-env guix build ghc-attoparsec  252.59s user 37.52s
system 92% cpu 5:15.00 total

time guix build idris never terminates.

I expect the problem to be related to circular dependencies.

bug#25449: GHC packages take a long time to install

[Ludovic Courtès]

Hi!

David Craven <david@craven.ch> skribis:

> time guix build ghc-attoparsec
> warning: failed to install locale: Invalid argument
> substitute: warning: failed to install locale: Invalid argument
> /gnu/store/qb6y8294zp3la57y1hm02hyf2ck0giwp-ghc-attoparsec-0.13.0.1
> $HOME/guix/pre-inst-env guix build ghc-attoparsec  252.59s user 37.52s
> system 92% cpu 5:15.00 total
>
> time guix build idris never terminates.

Works for me:

--8<---------------cut here---------------start------------->8---
$ time ./pre-inst-env  guix build idris -n
The following files would be downloaded:
   /gnu/store/n89zqwsm2y85dsqln23dir2niyxwmj1d-idris-0.99
   /gnu/store/00jk2bzaxpsnkmhsahsyjkwzn49j079r-ghc-parsec-3.1.9-doc
   /gnu/store/04zvw0qx6vhmz4cbym2w5xdk04js1n9d-ghc-terminal-size-0.3.2.1
   /gnu/store/05h7mqrbc34jd2ghhg7jsn4nq9qxjwwy-ghc-tasty-smallcheck-0.8.0.1
   /gnu/store/06saavc5xiwzj3fvdw44slfvabpwbcz5-ghc-network-uri-2.6.0.3
   /gnu/store/0dwdykcwab9f4mc9ywpi5nfpfx831v6n-ghc-aeson-0.10.0.0
   /gnu/store/0m8l58890yn1yrnvi7g3fmm3qcs014wz-ghc-tasty-golden-2.3.0.2
   /gnu/store/12110cza5vfj40irql96nsd8wbqzsq88-ghc-parsec-3.1.9
   /gnu/store/14amnngak59g7dryxfs7s5vi4iqqzzx3-ghc-hashable-1.2.3.3-doc
   /gnu/store/176ch41a30zkch8xgfk1ppgmjkgyway9-ghc-blaze-builder-0.4.0.1
   /gnu/store/1zvchvw6rhd5d031j1rmgzv92zldvfp2-ghc-parallel-3.2.0.6-doc
   /gnu/store/20l19mixr786hyjr4p8csgp9y7aclcyi-ghc-base64-bytestring-1.0.0.1
   /gnu/store/2afa17zx4habpz0w9m15a3cxms79lw1x-ghc-bifunctors-5
   /gnu/store/2m4696fxka2jfl4hgi1ahv0484vzvvyf-ghc-parallel-3.2.0.6
   /gnu/store/2ryg779mx7raih297vdaxsic0mvb859f-ghc-smallcheck-1.1.1
   /gnu/store/33681jbx92yy045nvziwcx88linprz7k-ghc-digest-0.0.1.2
   /gnu/store/3b9izpzwlvdx9g3hdj714mgrzgf3zg9s-ghc-semigroups-0.18.2
   /gnu/store/3blrzsrjci39s7k6a89p7s6v6bllrlw4-hlint-1.9.21
   /gnu/store/3h8xsiks5kdkbljr8c41wz3z8kvr82aa-ghc-network-uri-2.6.0.3-doc
   /gnu/store/3lh8aajbsr73ncizrl41fj8y6r7m4dcs-ghc-tasty-0.11.0.1
   /gnu/store/444rqagdm1x62v8mg2brws10kh046bid-ghc-profunctors-5.1.1
   /gnu/store/47b0y2wj41c00f7kynj8d7jyk5bcam1n-ghc-base-orphans-0.4.4
   /gnu/store/4in45sbh6ll5fv7yilb8q7vdrmwhqkgg-ghc-trifecta-1.6
   /gnu/store/4lj6gmhg18sl2isf8yhsq7fbh8ivr49f-ghc-semigroupoids-5.0.0.4
   /gnu/store/4lwwf87mwwspzm4gpq6dhlzch62yn256-ghc-statevar-1.1.0.1
   /gnu/store/5px7v2krra76vdshq4qwy9y4q6fryvrd-ghc-xss-sanitize-0.3.5.6
   /gnu/store/63jmhlcl7qygp0k1xg8wsi8i0amw5h6d-ghc-tasty-quickcheck-0.8.4
   /gnu/store/6cvk96nn9qwnhsvy5wnm7125kjc5j4w6-ghc-split-0.2.2
   /gnu/store/6lvngqa830m598zbfvjwwyg3rggrrcyl-ghc-scientific-0.3.4.2
   /gnu/store/6xr8lmrys2w10lkwcd34np9595ffam8q-ghc-unordered-containers-0.2.5.1-doc
   /gnu/store/73r21d173n321x3s3r0nvpbarscm6nph-ghc-safe-0.3.9
   /gnu/store/75n9228c3xjab4pv0pwyyck4kc2xrhx0-ghc-blaze-html-0.8.1.1
   /gnu/store/7h6armr49n512rmhpk300h9gwp90n033-ghc-kan-extensions-4.2.3
   /gnu/store/8bs59bwrkalpfal20rmq90hrhmsha6ww-ghc-hashable-1.2.3.3
   /gnu/store/8fcydg70kz7jk942kqkic3rn2jgwm40r-ghc-data-default-instances-old-locale-0.0.1
   /gnu/store/8zfy9287syd6ksf643n13c022b6n1spp-ghc-tasty-ant-xml-1.0.2
   /gnu/store/999z7badmj850sdxk53qcy8nm9dd38k9-ghc-data-default-instances-dlist-0.0.1
   /gnu/store/9swxnkz1fsqjlwl5r7pjb5hc43z4dq9s-ghc-void-0.7.1
   /gnu/store/9wq1lmpgg3w1nk5cp4mhdznsxs2f3y26-ghc-split-0.2.2-doc
   /gnu/store/a278nipkmh5mwaibmjcqjg1n1g75glg9-ghc-doctest-0.10.1
   /gnu/store/amfbr70cm3d8wp2dp251w2la8gi1gxcd-ghc-data-default-instances-containers-0.0.1
   /gnu/store/b2l8wq4xhmank7345sakkfm65p8hbdbj-ghc-unix-compat-0.4.1.4
   /gnu/store/c81nlff3942fwa07lc9jaswhh81ysw2n-ghc-generic-deriving-1.8.0
   /gnu/store/cczd4z1yw258rri60rza49pdamsabf5k-ghc-blaze-markup-0.7.0.3
   /gnu/store/cgjymhpaw3hhzhf7v09xc0dyxiydzqax-ghc-ieee754-0.7.8
   /gnu/store/cr7s06clz3asc9fj092lillvf4yxhjr5-ghc-parsers-0.12.3
   /gnu/store/d2khrln6pd73zd7xmkk5qnkziaw0rmgj-ghc-adjunctions-4.2.2
   /gnu/store/d4mx7mw1ikrl05r7rwp10rvshv5a0i26-ghc-attoparsec-0.13.0.1
   /gnu/store/dn19jxwnbg9qgdhf6h6xcayacy9lcqhi-ghc-old-time-1.1.0.3
   /gnu/store/dv2smfhpl3k0lmi4criismja2vglzb49-ghc-cheapskate-0.1.0.4
   /gnu/store/dzpjfwc8gf0vzcwa465pipimnyqnwr99-ghc-free-4.12.1
   /gnu/store/fli45cbwpi2n3r8cqz5lxjs2ri75yr1y-ghc-vector-0.11.0.0-doc
   /gnu/store/grg3whx4s90a9agqssb82qqdyc1yflsp-ghc-zlib-0.5.4.2-doc
   /gnu/store/hcx44m0rjs74nwjaz598s87z3zn89qbr-ghc-utf8-string-1.0.1.1
   /gnu/store/hp166g3wzzksyalndy950cc1xja6p4p5-ghc-quickcheck-unicode-1.0.0.1
   /gnu/store/i9x05bj2rp484f4yg79b0lf6k7csbhfb-ghc-regex-tdfa-1.2.2
   /gnu/store/inwc9hi2x6f094lll17w92dnc2hj8qm6-ghc-http-types-0.9
   /gnu/store/iqckfbzgj9nlmdiqs5fsg1wgr3bvgjqi-ghc-distributive-0.4.4
   /gnu/store/ir7ivlmd2ch44821n6xdy4kin8dhqqbk-ghc-nats-1
   /gnu/store/jhb7x2998ba2v844rm8qa447kx80qr24-ghc-prelude-extras-0.4.0.2
   /gnu/store/jvf552k2y2hs5k7124hvpimglfrx45cn-ghc-comonad-4.2.7.2
   /gnu/store/jx94lbfaipdb9nc9p5dzgiydpx9hiwym-ghc-charset-0.3.7.1
   /gnu/store/k0grfy6q8hpqk86mf14wylqd00k66xig-ghc-reducers-3.12.1
   /gnu/store/kpl0n9cxwcfswijzfdffw0qfiflz644a-ghc-tasty-rerun-1.1.6
   /gnu/store/kwabycqnj934lc1mb3v44ggd7mzfcvzw-ghc-lens-4.13
   /gnu/store/l3xv7b9iwsa6q453bprfbgg89bqbcdsj-ghc-unordered-containers-0.2.5.1
   /gnu/store/lgbsfm17213jmhqa56v3vbdvbx7xzh1k-ghc-tagsoup-0.14
   /gnu/store/m437y7qdwk5b42qqg9whfabj45jmga8v-ghc-optparse-applicative-0.11.0.2
   /gnu/store/m5fmgjvgnl2ynp11vds2640ki6yvh02v-ghc-dlist-0.7.1.2
   /gnu/store/mpj9jjfyhhfwqw3qx5y326l6mqlssv7z-ghc-vector-0.11.0.0
   /gnu/store/mw5cdg7cza54nimf72s8ckw6bbyqpcwn-ghc-temporary-rc-1.2.0.3
   /gnu/store/n378sscfs69hln1szbgg1gcrv6m5yy0g-ghc-wai-extra-3.0.11.1
   /gnu/store/q01fjbaz3vmkm3qmiyzbqapkxjs4rl4g-ghc-fingertree-0.1.1.0
   /gnu/store/r7yii11s1asvr56f7421pn83k6m17g55-ghc-fsnotify-0.2.1
   /gnu/store/r9qvkb2cjrb50nka9375zjlqx5rb122n-ghc-reflection-2.1
   /gnu/store/rvji08dcrxjv7zbpgasmlnp9sxchdafk-ghc-zlib-0.5.4.2
   /gnu/store/sag9mrb2skzwj30qdazmg42q78icv4m9-ghc-contravariant-1.3.3
   /gnu/store/srxg0s20nk9gyhfpm36s5d6yvjj685v4-ghc-chasingbottoms-1.3.0.13
   /gnu/store/v3xrfl7xz4m7najqk94pam3ids9jxy5x-ghc-css-text-0.1.2.1
   /gnu/store/vj2dy575zzfnagsb9c5j419i34p8h3rs-ghc-vector-binary-instances-0.2.1.0
   /gnu/store/vnjlydvxx0pi530j9pgqlvpn9nbfyb6z-ghc-tagged-0.8.1
   /gnu/store/vqq7758g7c8hg66zhgaqnlvdm7c9rqmh-ghc-annotated-wl-pprint-0.7.0
   /gnu/store/w6v16q5c6f4h0wppcmm28lkwlzj62yih-ghc-hinotify-0.3.8.1
   /gnu/store/wgdjx6qr05kd9znlzwgb9w8z022yhkln-ghc-tasty-hunit-0.9.2
   /gnu/store/wkfif9rwi6dqzbkgkn888bfr46ay81kp-ghc-simple-reflect-0.3.2
   /gnu/store/wkha6i0safczax8bni5w94m35yqiiq18-ghc-uniplate-1.6.12
   /gnu/store/wqf0bn514ijil8n97kn9wvgl9hsbrl15-ghc-network-2.6.2.1
   /gnu/store/x9mjvg0chlfx0046si6wacsilmjhn9s4-ghc-data-default-instances-base-0.0.1
   /gnu/store/y3x0ap81jg78rdk8ph5mk8vilipgqf7s-ghc-test-framework-quickcheck2-0.3.0.3
   /gnu/store/y7m6pqgn2c0qg0jsn5ndrybj0sr9hvyk-ghc-zip-archive-0.2.3.7
   /gnu/store/y8pn8p7hk79mj4vzhqrwv2x43cjh7g8d-ghc-data-default-0.5.3
   /gnu/store/zfz91463zcrvmprwq1d8s8dsv836rx1a-ghc-wai-3.0.4.0
   /gnu/store/zlapxi2wrrp5af166zrv5hqj4017q1sa-ghc-network-2.6.2.1-doc

real	0m2.061s
user	0m1.880s
sys	0m0.076s
$ git describe
v0.12.0-548-g153b62957
--8<---------------cut here---------------end--------------->8---

Could you try --no-substitutes and/or --no-grafts to see if it makes a
difference?

Ludo’.

bug#25449: GHC packages take a long time to install

[David Craven]

Hi Ludo!

guix build idris -n works for me too.

guix build idris --no-substitutes does this (which may be a problem on my end):
guix build idris --no-substitutes
warning: failed to install locale: Invalid argument
@ build-started
/gnu/store/85rrbgf0y77s1qwk9lw04gcf44nhw213-libxslt-1.1.29.tar.gz.drv
- x86_64-linux /var/log/guix/drvs/85//rrbgf0y77s1qwk9lw04gcf44nhw213-libxslt-1.1.29.tar.gz.drv.bz2
error: failed to run download program
'/home/dvc/guix/nix/scripts/download': Permission denied
builder for `/gnu/store/85rrbgf0y77s1qwk9lw04gcf44nhw213-libxslt-1.1.29.tar.gz.drv'
failed with exit code 1
@ build-failed /gnu/store/85rrbgf0y77s1qwk9lw04gcf44nhw213-libxslt-1.1.29.tar.gz.drv
- 1 builder for
`/gnu/store/85rrbgf0y77s1qwk9lw04gcf44nhw213-libxslt-1.1.29.tar.gz.drv'
failed with exit code 1
@ build-started
/gnu/store/69ln3nw2m9ydkjk5c6z72rpjlsdypl0x-Python-3.5.2.tar.xz.drv -
x86_64-linux /var/log/guix/drvs/69//ln3nw2m9ydkjk5c6z72rpjlsdypl0x-Python-3.5.2.tar.xz.drv.bz2
cannot build derivation
`/gnu/store/ddn0hzpn8nc4rdy8hwns2hh49761wgqc-libxslt-1.1.29.tar.xz.drv':
1 dependencies couldn't be built
cannot build derivation
`/gnu/store/y8q83qq1ggvm0z9p0g3a8lfzbghgcrfl-libxslt-1.1.29.drv': 1
dependencies couldn't be built
guix build: error: build failed: build of
`/gnu/store/y8q83qq1ggvm0z9p0g3a8lfzbghgcrfl-libxslt-1.1.29.drv'
failed

When I do git checkout 3254e1aba9ce3925a4e0b24c647240f96b5de8d4 (the
last commit of the series that added idris)
guix build idris works fine too. This is what lead me to believe that
the problem was not on my end.

Thanks!
David

bug#25449: GHC packages take a long time to install

[Ludovic Courtès]

Hi!

David Craven <david@craven.ch> skribis:

> guix build idris -n works for me too.

OK.  And how long does it take, since this was the subject of this
report?

> guix build idris --no-substitutes does this (which may be a problem on my end):

[...]

> error: failed to run download program
> '/home/dvc/guix/nix/scripts/download': Permission denied

Problem on your end I think.  :-)

That script must be readable and executable by the guixbuild* users
(thus, its parent directories must be readable/executable as well).

HTH!

Ludo’.

bug#25449: GHC packages take a long time to install

[David Craven]

[-- Attachment #1: Type: text/plain, Size: 2370 bytes --]

> That script must be readable and executable by the guixbuild* users
> (thus, its parent directories must be readable/executable as well).

Mmh, it worked till recently... I did chmod a+x /home/dvc and it solved the
permission denied problem on guix/nix/scripts/download. Thanks!

But that doesn't fix my problem with some ghc packages taking extremely
long... Any ideas?

Anyway, I'm hitting problems all the time - need to finally switch to
guixsd :)

FYI commit 384344198dcaa97847e66d3dd82f279ede08d690 causes this on my end,
but I don't currently have time to look in to it...
Backtrace:
In unknown file:
   ?: 19 [primitive-load-path "gnu/tests/install" ...]
In ice-9/eval.scm:
 505: 18 [#<procedure 161e3c0 at ice-9/eval.scm:499:4 (exp)> (define-module
# # ...)]
In ice-9/psyntax.scm:
1106: 17 [expand-top-sequence ((define-module # # # ...)) () ((top)) ...]
 989: 16 [scan ((define-module (gnu tests install) #:use-module ...)) ()
...]
 279: 15 [scan ((#(syntax-object let # ...) (#) (# #) ...)) () ...]
In ice-9/eval.scm:
 411: 14 [eval # ()]
In ice-9/boot-9.scm:
2954: 13 [define-module* (gnu tests install) #:filename ...]
2929: 12 [resolve-imports (((gnu)) ((gnu tests)) ((gnu tests base)) ...)]
2867: 11 [resolve-interface (gnu tests base) #:select ...]
2792: 10 [#<procedure 16d6ec0 at ice-9/boot-9.scm:2780:4 (name #:optional
autoload version #:key ensure)> # ...]
3068: 9 [try-module-autoload (gnu tests base) #f]
2404: 8 [save-module-excursion #<procedure 585e4b0 at
ice-9/boot-9.scm:3069:17 ()>]
3088: 7 [#<procedure 585e4b0 at ice-9/boot-9.scm:3069:17 ()>]
In unknown file:
   ?: 6 [primitive-load-path "gnu/tests/base" ...]
In gnu/tests/base.scm:
 275: 5 [#<procedure 2c67a00 ()>]
In gnu/system/vm.scm:
 475: 4 [system-qemu-image/shared-store-script # # # ...]
In gnu/system.scm:
 635: 3 [operating-system-derivation # # #f]
 340: 2 [operating-system-services # # #f]
 291: 1 [essential-services # # #f]
 191: 0 [non-boot-file-system-service #]

gnu/system.scm:191:4: In procedure non-boot-file-system-service:
gnu/system.scm:191:4: In procedure module-lookup: Unbound variable: #{%
file-system-needed-for-boot?-procedure}#
make[2]: *** [Makefile:4878: make-go] Error 1
make[2]: Leaving directory '/home/dvc/guix'
make[1]: *** [Makefile:4021: all-recursive] Error 1
make[1]: Leaving directory '/home/dvc/guix'
make: *** [Makefile:2677: all] Error 2z

[-- Attachment #2: Type: text/html, Size: 3114 bytes --]

bug#25449: GHC packages take a long time to install

[Ricardo Wurmus]

David Craven <david@craven.ch> writes:

> FYI commit 384344198dcaa97847e66d3dd82f279ede08d690 causes this on my end,
> but I don't currently have time to look in to it...

Try “rm gnu/system.go && make”

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
http://elephly.net

bug#25449: GHC packages take a long time to install

[Ludovic Courtès]

David Craven <david@craven.ch> skribis:

>> That script must be readable and executable by the guixbuild* users
>> (thus, its parent directories must be readable/executable as well).
>
> Mmh, it worked till recently... I did chmod a+x /home/dvc and it solved the
> permission denied problem on guix/nix/scripts/download. Thanks!
>
> But that doesn't fix my problem with some ghc packages taking extremely
> long... Any ideas?

Figures please!  :-)

--8<---------------cut here---------------start------------->8---
$ time ./pre-inst-env guix build idris -n 2>/dev/null

real	0m2.067s
user	0m1.860s
sys	0m0.092s
$ git describe
v0.12.0-698-gcd294a6d9
--8<---------------cut here---------------end--------------->8---

Also, see also
<https://lists.gnu.org/archive/html/guix-devel/2017-01/msg01299.html>.

Ludo’.