Qt 5.11 tarballs anyone?

Qt 5.11 tarballs anyone?

[Ludovic Courtès]

Hi Guix!

It’s been two years since commit
0791437f972caa7e48de91ad5cb150a614f617c2 but we lost key tarballs from
that time, in particular Qt 5.11.2 tarballs, which are no longer
available at <https://download.qt.io/archive/>.

You can try for instance with:

  guix build \
  /gnu/store/sx529mnxcdy3amgyhri2w72328m8l98w-qtmultimedia-everywhere-src-5.11.2.tar.xz.drv

(This will substitute the .drv and then try to build it.)

Does anyone happen to have these tarballs?  I’d like to re-inject them
on ci.guix.

This is a reminder of how important Disarchive is!
(See <https://issues.guix.gnu.org/42162>.)

Ludo’.

Re: Qt 5.11 tarballs anyone?

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 1137 bytes --]

Ludovic Courtès 写道：
> Does anyone happen to have these tarballs?  I’d like to 
> re-inject them
> on ci.guix.

I see you(?)'ve found them and done so already:

--8<---------------cut here---------------start------------->8---
~ λ guix build 
/gnu/store/sx529mnxcdy3amgyhri2w72328m8l98w-qtmultimedia-everywhere-src-5.11.2.tar.xz.drv
...
substitute: updating substitutes from 
'https://guix.tobias.gr'... 100.0%
substitute: updating substitutes from 
'https://ci.guix.gnu.org'... 100.0%
substituting 
/gnu/store/z6flb2k839zvpajha5j179kph1hsfcrq-qtmultimedia-everywhere-src-5.11.2.tar.xz...
downloading from 
https://ci.guix.gnu.org/nar/z6flb2k839zvpajha5j179kph1hsfcrq-qtmultimedia-everywhere-src-5.11.2.tar.xz 
...
 qtmultimedia-everywhere-src-5.11.2.tar.xz  3.5MiB 
 16.3MiB/s 00:00 [##################] 100.0%
--8<---------------cut here---------------end--------------->8---

Anyway, they should be available on guix.tobias.gr as well.

There's also what looks like a comprehensive archive at 
<https://distfiles.macports.org/qt5/>, although I didn't check the 
hashes.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: Qt 5.11 tarballs anyone?

[Ludovic Courtès]

Hi Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

> There's also what looks like a comprehensive archive at 
> <https://distfiles.macports.org/qt5/>, although I didn't check the 
> hashes.

Yup, I’ve added that and sources.buildroot in commit
9d01749feaa1586b1caf449712116e7518bb2303 for posterity.

Thanks!

Ludo’.

Re: Qt 5.11 tarballs anyone?

[Ludovic Courtès]

Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

> It’s been two years since commit
> 0791437f972caa7e48de91ad5cb150a614f617c2 but we lost key tarballs from
> that time, in particular Qt 5.11.2 tarballs, which are no longer
> available at <https://download.qt.io/archive/>.

Right after sending this message, I found’em all at
<http://sources.buildroot.net/qt5tools/> and related URLs.  They’re now
back at ci.guix.

Now to actually build this code…

Ludo’.

Re: Qt 5.11 tarballs anyone?

[zimoun]

Hi Ludo,

On Sun, 24 Jan 2021 at 12:46, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> It’s been two years since commit
> 0791437f972caa7e48de91ad5cb150a614f617c2 but we lost key tarballs from

[...]

> This is a reminder of how important Disarchive is!
> (See <https://issues.guix.gnu.org/42162>.)

Yeah!  Definitively!

How could I list all the tarballs that ci.guix.gnu.org has?


Cheers,
simon

Re: Qt 5.11 tarballs anyone?

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> How could I list all the tarballs that ci.guix.gnu.org has?

I suppose you could simply run something like:

  guix build -n --sources=all $(guix package -A | cut -f1)

and see what happens.

That said, I think Chris Baines recently added something to the Guix
Data Service that periodically tries to build fixed-output derivations
(essentially source code downloads).  Am I right, Chris?

Ludo’.

Re: Qt 5.11 tarballs anyone?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]


Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Hi,
>
> zimoun <zimon.toutoune@gmail.com> skribis:
>
>> How could I list all the tarballs that ci.guix.gnu.org has?
>
> I suppose you could simply run something like:
>
>   guix build -n --sources=all $(guix package -A | cut -f1)
>
> and see what happens.
>
> That said, I think Chris Baines recently added something to the Guix
> Data Service that periodically tries to build fixed-output derivations
> (essentially source code downloads).  Am I right, Chris?

Kind of, there's this page [1] which tells you about fixed output
derivations for packages, as well as the latest build.

1: https://data.guix-patches.cbaines.net/repository/2/branch/master/latest-processed-revision/fixed-output-package-derivations?system=x86_64-linux&target=none&latest_build_status=&after_name=&limit_results=&all_results=on

It sounds like you want a page listing the derivation outputs, along
with substitute availability. That doesn't exist yet, but shouldn't be
too hard to add.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Qt 5.11 tarballs anyone?

[zimoun]

Hi Chris,

On Mon, 25 Jan 2021 at 18:49, Christopher Baines <mail@cbaines.net> wrote:

> Kind of, there's this page [1] which tells you about fixed output
> derivations for packages, as well as the latest build.
>
> 1: https://data.guix-patches.cbaines.net/repository/2/branch/master/latest-processed-revision/fixed-output-package-derivations?system=x86_64-linux&target=none&latest_build_status=&after_name=&limit_results=&all_results=on

I am not able to open this webpage.


Cheers,
simon

Re: Qt 5.11 tarballs anyone?

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]


zimoun <zimon.toutoune@gmail.com> writes:

> On Mon, 25 Jan 2021 at 18:49, Christopher Baines <mail@cbaines.net> wrote:
>
>> Kind of, there's this page [1] which tells you about fixed output
>> derivations for packages, as well as the latest build.
>>
>> 1: https://data.guix-patches.cbaines.net/repository/2/branch/master/latest-processed-revision/fixed-output-package-derivations?system=x86_64-linux&target=none&latest_build_status=&after_name=&limit_results=&all_results=on
>
> I am not able to open this webpage.

It's slow, obviously I need to look again at the performance, but it
should work eventually.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Qt 5.11 tarballs anyone?

[zimoun]

On Mon, 25 Jan 2021 at 20:12, Christopher Baines <mail@cbaines.net> wrote:

>>> 1: https://data.guix-patches.cbaines.net/repository/2/branch/master/latest-processed-revision/fixed-output-package-derivations?system=x86_64-linux&target=none&latest_build_status=&after_name=&limit_results=&all_results=on
>>
>> I am not able to open this webpage.
>
> It's slow, obviously I need to look again at the performance, but it
> should work eventually.

Maybe I have not been patient enough. :-)
I will give another try.

Cheers,
simon

Re: Qt 5.11 tarballs anyone?

[Ludovic Courtès]

Hi!

Christopher Baines <mail@cbaines.net> skribis:

> Kind of, there's this page [1] which tells you about fixed output
> derivations for packages, as well as the latest build.
>
> 1: https://data.guix-patches.cbaines.net/repository/2/branch/master/latest-processed-revision/fixed-output-package-derivations?system=x86_64-linux&target=none&latest_build_status=&after_name=&limit_results=&all_results=on

Very nice.  (And it’s all green!)

> It sounds like you want a page listing the derivation outputs, along
> with substitute availability. That doesn't exist yet, but shouldn't be
> too hard to add.

Yes, that would be nice.  But note that in effect, simply the derivation
amounts to checking substitute availability since for tarballs there’s
an automatic fallback to substitute URLs (even when substitutes are
disabled, see (guix build download-nar)).

Thanks!

Ludo’.

Re: Qt 5.11 tarballs anyone?

[zimoun]

Hi Ludo,

On Mon, 25 Jan 2021 at 17:52, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

>> How could I list all the tarballs that ci.guix.gnu.org has?
>
> I suppose you could simply run something like:
>
>   guix build -n --sources=all $(guix package -A | cut -f1)

This will download only the source at the current Guix.

Well, I would like to 1. stress disarchive to evaluate how its database
disarchive-db scales, and 2. have a number about the size of all
tarballs that Guix is able to reach (say post 1.0).


> and see what happens.
>
> That said, I think Chris Baines recently added something to the Guix
> Data Service that periodically tries to build fixed-output derivations
> (essentially source code downloads).  Am I right, Chris?

Cool!

Maybe a Berlin or Bayfront sysadmin could list all the tarballs files in
the store and publish this file.  Then I could use “guix download” to
fetch all of them.


All the best,
simon

Re: Qt 5.11 tarballs anyone?

[zimoun]

Hi,

On Mon, 25 Jan 2021 at 20:43, zimoun <zimon.toutoune@gmail.com> wrote:
> On Mon, 25 Jan 2021 at 17:52, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
>
>>> How could I list all the tarballs that ci.guix.gnu.org has?
>>
>> I suppose you could simply run something like:
>>
>>   guix build -n --sources=all $(guix package -A | cut -f1)
>
> This will download only the source at the current Guix.

After 5 unexpected failures with,

  guix build --sources=all $(guix package -A | cut -f1) --fallback

I give up.  At each try, it downloads and /gnu/ is initially at 8.6G and
ends at 77G.  Then “guix gc” is fast except the “unused links” part
which takes ages…

Well, I am going to investigate more the reasons of the failure.
However, this command is not that I want to work then with Diassemble.


All the best,
simon

Re: Qt 5.11 tarballs anyone?

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> After 5 unexpected failures with,
>
>   guix build --sources=all $(guix package -A | cut -f1) --fallback

What failures did you get?

Thanks for giving it a try,
Ludo’.

Failure guix build --sources=all $(guix package -A...

[zimoun]

Hi Ludo,

On Thu, 28 Jan 2021 at 08:47, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> What failures did you get?

With Guix 0f20b3f.

run #1

  guix build --sources=all $(guix package -A | cut -f1) --fallback

--8<---------------cut here---------------start------------->8---
download-progress
/gnu/store/960b2kch33maxjk7b6g5pp12rj2bagx6-SNPlocs.Hsapiens.dbSNP144.GRCh37_0.99.20.tar.gz
https://ci.guix.gnu.org/nar/960b2kch33maxjk7b6g5pp12rj2bagx6-SNPlocs.Hsapiens.dbSNP144.GRCh37_0.99.20.tar.gz
913353344 152961120
Backtrace:
          18 (primitive-load "/home/sitour/.config/guix/current/bin/…")
In guix/ui.scm:
  2154:12 17 (run-guix-command _ . _)
In ice-9/boot-9.scm:
  1736:10 16 (with-exception-handler _ _ #:unwind? _ # _)
  1731:15 15 (with-exception-handler #<procedure 7f51b7155fc0 at ic…> …)
In guix/status.scm:
    780:4 14 (call-with-status-report _ _)
In ice-9/boot-9.scm:
  1736:10 13 (with-exception-handler _ _ #:unwind? _ # _)
In guix/store.scm:
   636:37 12 (thunk)
   1305:8 11 (call-with-build-handler _ _)
   1305:8 10 (call-with-build-handler _ _)
   1305:8  9 (call-with-build-handler #<procedure 7f51b7155660 at g…> …)
In guix/scripts/build.scm:
   696:26  8 (_)
In guix/store.scm:
  1373:15  7 (_ #<store-connection 256.99 7f51ba6313c0> _ _)
   745:13  6 (process-stderr _ _)
In unknown file:
           5 (display "://ci.guix.gnu.org/nar/lzip/5v2xxcrh0npbzn2m…" …)
In guix/status.scm:
   703:16  4 (write! _ _ _)
   616:15  3 (_ (download-succeeded "/gnu/store/dm1fc94ais6h5q0:…" …) …)
   273:33  2 (compute-status _ #<<build-status> building: () downlo…> …)
In ice-9/boot-9.scm:
  1669:16  1 (raise-exception _ #:continuable? _)
  1669:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1669:16: In procedure raise-exception:
In procedure struct-vtable: Wrong type argument in position 1
(expecting struct): #f
--8<---------------cut here---------------end--------------->8---

re-run #2

--8<---------------cut here---------------start------------->8---
d-data-0.0.23b-alpha.tar.xz
https://ci.guix.gnu.org/nar/6x92chv2cw0wxv9yr5y6x3cvc38sx8q8-0ad-data-0.0.23b-alpha.tar.xz
694549624 298188896
Backtrace:
          17 (primitive-load "/home/sitour/.config/guix/current/bin/…")
In guix/ui.scm:
  2154:12 16 (run-guix-command _ . _)
In ice-9/boot-9.scm:
  1736:10 15 (with-exception-handler _ _ #:unwind? _ # _)
  1731:15 14 (with-exception-handler #<procedure 7f0f7150bfc0 at ic…> …)
In guix/status.scm:
    780:4 13 (call-with-status-report _ _)
In ice-9/boot-9.scm:
  1736:10 12 (with-exception-handler _ _ #:unwind? _ # _)
In guix/store.scm:
   636:37 11 (thunk)
   1305:8 10 (call-with-build-handler _ _)
   1305:8  9 (call-with-build-handler #<procedure 7f0f7150b660 at g…> …)
In guix/scripts/build.scm:
   696:26  8 (_)
In guix/store.scm:
  1373:15  7 (_ #<store-connection 256.99 7f0f749e63c0> _ _)
   745:13  6 (process-stderr _ _)
In unknown file:
           5 (display "hdjdxjx9wx1mj941b5v8-drascula-audio-2.0.zip …" …)
In guix/status.scm:
   703:16  4 (write! _ _ _)
   616:15  3 (_ (download-succeeded "/gnuhdjdxjx9wx1mj941b5v8-dr…" …) …)
   273:33  2 (compute-status _ #<<build-status> building: () downlo…> …)
In ice-9/boot-9.scm:
  1669:16  1 (raise-exception _ #:continuable? _)
  1669:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1669:16: In procedure raise-exception:
In procedure struct-vtable: Wrong type argument in position 1
(expecting struct): #f
--8<---------------cut here---------------end--------------->8---

re-run #3

--8<---------------cut here---------------start------------->8---
h76l2n7v3s5vpzm33w2l80qgg3zv-font-adobe-source-han-sans-1.004-checkout
https://ci.guix.gnu.org/nar/gzip/l0hmh76l2n7v3s5vpzm33w2l80qgg3zv-font-adobe-source-han-sans-1.004-checkout
1665898521 991264768
Backtrace:
          17 (primitive-load "/home/sitour/.config/guix/current/bin/…")
In guix/ui.scm:
  2154:12 16 (run-guix-command _ . _)
In ice-9/boot-9.scm:
  1736:10 15 (with-exception-handler _ _ #:unwind? _ # _)
  1731:15 14 (with-exception-handler #<procedure 7f774b1ecb70 at ic…> …)
In guix/status.scm:
    780:4 13 (call-with-status-report _ _)
In ice-9/boot-9.scm:
  1736:10 12 (with-exception-handler _ _ #:unwind? _ # _)
In guix/store.scm:
   636:37 11 (thunk)
   1305:8 10 (call-with-build-handler _ _)
   1305:8  9 (call-with-build-handler #<procedure 7f774b1ec210 at g…> …)
In guix/scripts/build.scm:
   696:26  8 (_)
In guix/store.scm:
  1373:15  7 (_ #<store-connection 256.99 7f774e70a3c0> _ _)
   745:13  6 (process-stderr _ _)
In unknown file:
           5 (display "heme-10.1.3-x86-64.tar.gz 69956440 65929312\…" …)
In guix/status.scm:
   703:16  4 (write! _ _ _)
   616:15  3 (_ (download-succeeded "/gnu/store/rxw5pk8c6hi7nygv…" …) …)
   273:33  2 (compute-status _ #<<build-status> building: () downlo…> …)
In ice-9/boot-9.scm:
  1669:16  1 (raise-exception _ #:continuable? _)
  1669:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1669:16: In procedure raise-exception:
--8<---------------cut here---------------end--------------->8---

re-run #4

--8<---------------cut here---------------start------------->8---
`/gnu/store/pihzkn3zdzksih46nyaynbg4kjrpcrjg-clml-0.0.0-0.95505b5-checkout/time-series/clml.time-series.asd'
-> `clml-0.0.0-0.95505b5-checkout/time-series/clml.time-series.asd'
`File som/src/lvq_pak.lisp is read-only; trying to patch anyway
patching file som/src/lvq_pak.lisp
File statistics/src/rand/rand.lisp is read-only; trying to patch anyway
patching file statistics/src/rand/rand.lisp
File time-series/src/ts-read-data.lisp is read-only; trying to patch anyway
patching file time-series/src/ts-read-data.lisp
File time-series/src/ts-state-space-model.lisp is read-only; trying to
patch anyway
patching file time-series/src/ts-state-space-model.lisp
/gnu/store/v6f44zccwh9z5zk3pjlywjyLVM2.2.03.11/lib/freeseg/

[...]

drbd-utils-9.15.1/scripts/drbd.conf.example
drbbuilder for `/gnu/store/16spw3b44rcbxxljsxxw751n9mr9if0x-adaptive-wrap-0.8.el.drv'
failed to produce output path
`/gnu/store/my6b833z32ayfvww94a268ybnfyabad3-adaptive-wrap-0.8.el'
la compilation de
/gnu/store/16spw3b44rcbxxljsxxw751n9mr9if0x-adaptive-wrap-0.8.el.drv a
échoué
Vous trouverez le journal de compilation dans «
/var/log/guix/drvs/16/spw3b44rcbxxljsxxw751n9mr9if0x-adaptive-wrap-0.8.el.drv.bz2
».
killing process 31831

[...]

killing process 6151
guix build: error: build of
`/gnu/store/16spw3b44rcbxxljsxxw751n9mr9if0x-adaptive-wrap-0.8.el.drv'
failed
--8<---------------cut here---------------end--------------->8---

then re-run always it this.  So instead

guix build --sources=all $(guix package -A | cut -f1 | tac) --fallback

downloads some and fails again with

--8<---------------cut here---------------start------------->8---
guix build: error: build of
`/gnu/store/16spw3b44rcbxxljsxxw751n9mr9if0x-adaptive-wrap-0.8.el.drv'
failed
--8<---------------cut here---------------end--------------->8---

and done; always fails.


No failure but something wrong… Several of things as:

--8<---------------cut here---------------start------------->8---
4 380502112
guix substitute: error: connect: Connexion terminée par expiration du
délai d'attente
guix substitute: error: connect: Connexion terminée par expiration du
délai d'attente
guix substitute: error: connect: Connexion terminée par expiration du
délai d'attente
--8<---------------cut here---------------end--------------->8---

Cheers,
simon

Re: Failure guix build --sources=all $(guix package -A...

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> With Guix 0f20b3f.
>
> run #1
>
>   guix build --sources=all $(guix package -A | cut -f1) --fallback
>
> download-progress
> /gnu/store/960b2kch33maxjk7b6g5pp12rj2bagx6-SNPlocs.Hsapiens.dbSNP144.GRCh37_0.99.20.tar.gz
> https://ci.guix.gnu.org/nar/960b2kch33maxjk7b6g5pp12rj2bagx6-SNPlocs.Hsapiens.dbSNP144.GRCh37_0.99.20.tar.gz
> 913353344 152961120
> Backtrace:
>           18 (primitive-load "/home/sitour/.config/guix/current/bin/…")
> In guix/ui.scm:
>   2154:12 17 (run-guix-command _ . _)
> In ice-9/boot-9.scm:
>   1736:10 16 (with-exception-handler _ _ #:unwind? _ # _)
>   1731:15 15 (with-exception-handler #<procedure 7f51b7155fc0 at ic…> …)
> In guix/status.scm:
>     780:4 14 (call-with-status-report _ _)
> In ice-9/boot-9.scm:
>   1736:10 13 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/store.scm:
>    636:37 12 (thunk)
>    1305:8 11 (call-with-build-handler _ _)
>    1305:8 10 (call-with-build-handler _ _)
>    1305:8  9 (call-with-build-handler #<procedure 7f51b7155660 at g…> …)
> In guix/scripts/build.scm:
>    696:26  8 (_)
> In guix/store.scm:
>   1373:15  7 (_ #<store-connection 256.99 7f51ba6313c0> _ _)
>    745:13  6 (process-stderr _ _)
> In unknown file:
>            5 (display "://ci.guix.gnu.org/nar/lzip/5v2xxcrh0npbzn2m…" …)
> In guix/status.scm:
>    703:16  4 (write! _ _ _)
>    616:15  3 (_ (download-succeeded "/gnu/store/dm1fc94ais6h5q0:…" …) …)
>    273:33  2 (compute-status _ #<<build-status> building: () downlo…> …)
> In ice-9/boot-9.scm:
>   1669:16  1 (raise-exception _ #:continuable? _)
>   1669:16  0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> In procedure struct-vtable: Wrong type argument in position 1
> (expecting struct): #f

That looks like a bug in (guix status).

Do you have max-jobs > 1?

Maxim reported a similar issue, gathered strace output etc., but I think
it then fell through the cracks and I can’t find it anymore.  Maxim,
does that ring a bell?

Thanks,
Ludo’.

Re: Failure guix build --sources=all $(guix package -A...

[Maxim Cournoyer]

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

[...]

>>   1373:15  7 (_ #<store-connection 256.99 7f51ba6313c0> _ _)
>>    745:13  6 (process-stderr _ _)
>> In unknown file:
>>            5 (display "://ci.guix.gnu.org/nar/lzip/5v2xxcrh0npbzn2m…" …)
>> In guix/status.scm:
>>    703:16  4 (write! _ _ _)
>>    616:15  3 (_ (download-succeeded "/gnu/store/dm1fc94ais6h5q0:…" …) …)
>>    273:33  2 (compute-status _ #<<build-status> building: () downlo…> …)
>> In ice-9/boot-9.scm:
>>   1669:16  1 (raise-exception _ #:continuable? _)
>>   1669:16  0 (raise-exception _ #:continuable? _)
>>
>> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
>> In procedure struct-vtable: Wrong type argument in position 1
>> (expecting struct): #f
>
> That looks like a bug in (guix status).
>
> Do you have max-jobs > 1?
>
> Maxim reported a similar issue, gathered strace output etc., but I think
> it then fell through the cracks and I can’t find it anymore.  Maxim,
> does that ring a bell?

That would be https://issues.guix.gnu.org/43518.  I found it by
searching for "#f" in the list of bugs ;-).

Maxim