* How long does it take to run the full rustc bootstrap chain? @ 2022-10-15 7:33 Félix Baylac Jacqué 2022-10-19 19:52 ` Efraim Flashner ` (2 more replies) 0 siblings, 3 replies; 12+ messages in thread From: Félix Baylac Jacqué @ 2022-10-15 7:33 UTC (permalink / raw) To: guix-devel Hey Guix, I'd be curious to know how long it takes to run the full rustc bootstrap chain on the Guix build farm. I'm sadly not sure how to approach this problem. Is there a way to extract this information from Cuirass or the Guix data service? Félix ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-15 7:33 How long does it take to run the full rustc bootstrap chain? Félix Baylac Jacqué @ 2022-10-19 19:52 ` Efraim Flashner 2022-10-20 9:59 ` Ludovic Courtès 2022-10-22 13:48 ` Maxim Cournoyer 2 siblings, 0 replies; 12+ messages in thread From: Efraim Flashner @ 2022-10-19 19:52 UTC (permalink / raw) To: Félix Baylac Jacqué; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1316 bytes --] On Sat, Oct 15, 2022 at 09:33:23AM +0200, Félix Baylac Jacqué wrote: > Hey Guix, > > I'd be curious to know how long it takes to run the full rustc bootstrap > chain on the Guix build farm. I'm sadly not sure how to approach this > problem. > > Is there a way to extract this information from Cuirass or the Guix data > service? If I were to go about it I would try to get the build log from each package: guix build --no-grafts -e '(@@ (gnu packages rust) rust-bootstrap)' \ --log-file and then I'd add up the build and test phases to get a rough estimate. For an approximate answer, the build farm runs 4 builds per machine, with 8 cores per build (all of this is in the guix-maintenance repo I think). When I build from mrustc to rust-1.60 on my machine with 24 cores it currently takes about 2 hours for rust-bootstrap (1.54) and then about 30-45 minutes each other version. I don't remember the bug number offhand, but there's work being done to speed up the process. The last time I ran though the test build I built from rust-1.55 to librsvg in 171 minutes. -- Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-15 7:33 How long does it take to run the full rustc bootstrap chain? Félix Baylac Jacqué 2022-10-19 19:52 ` Efraim Flashner @ 2022-10-20 9:59 ` Ludovic Courtès 2022-10-20 20:11 ` Efraim Flashner 2022-10-22 13:48 ` Maxim Cournoyer 2 siblings, 1 reply; 12+ messages in thread From: Ludovic Courtès @ 2022-10-20 9:59 UTC (permalink / raw) To: Félix Baylac Jacqué; +Cc: guix-devel Hi Félix, Félix Baylac Jacqué <felix@alternativebit.fr> skribis: > I'd be curious to know how long it takes to run the full rustc bootstrap > chain on the Guix build farm. I'm sadly not sure how to approach this > problem. I believe Efraim, Maxim, and probably a few other people have first-hand experience building the whole chain. Any estimate, people? > Is there a way to extract this information from Cuirass or the Guix data > service? Maybe not from the Guix Data Service, but most likely from Cuirass and the Guix Build Coordinator. For example, <https://ci.guix.gnu.org/build/1505621/details> shows the duration, which could can also get with: wget -qO- https://ci.guix.gnu.org/build/1505621 |jq (The (guix ci) module needs an update to let you do access that info right from Scheme.) Now the /search interface doesn’t have a JSON equivalent, but you can find the build ID using something along these lines: --8<---------------cut here---------------start------------->8--- scheme@(guile-user)> ,use(guix ci) scheme@(guile-user)> (latest-evaluations "https://ci.guix.gnu.org" 10 #:spec "master") $14 = (#<<evaluation> id: 739193 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "4716cea6256523a8ecf90a426d675bfb7620f3e4" channel: "guix">)> #<<evaluation> id: 738364 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "16d4ded6302c0650978203d0df83614896c453e8" channel: "guix">)> #<<evaluation> id: 737995 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "e61660c78f1190c578dd6f202bc5529cbdcff84e" channel: "guix">)> #<<evaluation> id: 737588 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "88746cd80bc56212ae7922c0fa1cd9a18e44c3bb" channel: "guix">)> #<<evaluation> id: 737529 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "ac553ba68e535810085dd838e48e4fa6ac553e67" channel: "guix">)> #<<evaluation> id: 736841 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "7ecd85eacbfa5b7379f563b83807d3e5258cf700" channel: "guix">)> #<<evaluation> id: 734789 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "3bb145b6e2a8c84e7739ead9ae76dc4d42bb9850" channel: "guix">)> #<<evaluation> id: 734747 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "033cbd11a837dbc7602799f15d691221653e1996" channel: "guix">)> #<<evaluation> id: 734733 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "c9eac0c3553703385c997a70741348ae5d34dc68" channel: "guix">)> #<<evaluation> id: 734718 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "2589997fab07bcebe6d87fd227852389ab5b1962" channel: "guix">)>) scheme@(guile-user)> (define jobs (evaluation-jobs "https://ci.guix.gnu.org" (evaluation-id (car $14)))) scheme@(guile-user)> (length jobs) $15 = 76649 scheme@(guile-user)> (car jobs) $16 = #<<job> build-id: 1627000 status: scheduled name: "0ad.aarch64-linux"> scheme@(guile-user)> ,use(srfi srfi-1) scheme@(guile-user)> (find (lambda (job) (string=? "rust.x86_64-linux" (job-name job))) jobs) $17 = #<<job> build-id: 1275492 status: succeeded name: "rust.x86_64-linux"> scheme@(guile-user)> (job-build "https://ci.guix.gnu.org" $17) $18 = #<<build> id: 1275492 derivation: "/gnu/store/dmzhvql66a1n31j2xpygfxpw576wpkwb-rust-1.60.0.drv" evaluation: 579940 system: "x86_64-linux" status: succeeded timestamp: 1662284654 start-time: #<date nanosecond: 0 second: 14 minute: 44 hour: 11 day: 4 month: 9 year: 2022 zone-offset: 7200> stop-time: #<date nanosecond: 0 second: 14 minute: 44 hour: 11 day: 4 month: 9 year: 2022 zone-offset: 7200> products: ()> --8<---------------cut here---------------end--------------->8--- That still doesn’t answer your question, because you’re interested in the build time of the full Rust bootstrap chain, but that’s a start! HTH, Ludo’. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-20 9:59 ` Ludovic Courtès @ 2022-10-20 20:11 ` Efraim Flashner 2022-10-22 8:57 ` Félix Baylac Jacqué 0 siblings, 1 reply; 12+ messages in thread From: Efraim Flashner @ 2022-10-20 20:11 UTC (permalink / raw) To: Ludovic Courtès; +Cc: Félix Baylac Jacqué, guix-devel [-- Attachment #1: Type: text/plain, Size: 4557 bytes --] On Thu, Oct 20, 2022 at 11:59:34AM +0200, Ludovic Courtès wrote: > Hi Félix, > > Félix Baylac Jacqué <felix@alternativebit.fr> skribis: > > > I'd be curious to know how long it takes to run the full rustc bootstrap > > chain on the Guix build farm. I'm sadly not sure how to approach this > > problem. > > I believe Efraim, Maxim, and probably a few other people have first-hand > experience building the whole chain. Any estimate, people? I have a number! I just pushed a couple of patches to staging to fix the bootstrap on aarch64, and to decrease the build time and resource usage, and the numbers are in! https://ci.guix.gnu.org/build/1632586/details 11863 seconds from downloading mrustc to building rust-1.60, for 198 minutes, or 3.3 hours. > > Is there a way to extract this information from Cuirass or the Guix data > > service? > > Maybe not from the Guix Data Service, but most likely from Cuirass and > the Guix Build Coordinator. > > For example, <https://ci.guix.gnu.org/build/1505621/details> shows the > duration, which could can also get with: > > wget -qO- https://ci.guix.gnu.org/build/1505621 |jq > > (The (guix ci) module needs an update to let you do access that info > right from Scheme.) > > Now the /search interface doesn’t have a JSON equivalent, but you can > find the build ID using something along these lines: > > --8<---------------cut here---------------start------------->8--- > scheme@(guile-user)> ,use(guix ci) > scheme@(guile-user)> (latest-evaluations "https://ci.guix.gnu.org" 10 #:spec "master") > $14 = (#<<evaluation> id: 739193 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "4716cea6256523a8ecf90a426d675bfb7620f3e4" channel: "guix">)> #<<evaluation> id: 738364 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "16d4ded6302c0650978203d0df83614896c453e8" channel: "guix">)> #<<evaluation> id: 737995 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "e61660c78f1190c578dd6f202bc5529cbdcff84e" channel: "guix">)> #<<evaluation> id: 737588 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "88746cd80bc56212ae7922c0fa1cd9a18e44c3bb" channel: "guix">)> #<<evaluation> id: 737529 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "ac553ba68e535810085dd838e48e4fa6ac553e67" channel: "guix">)> #<<evaluation> id: 736841 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "7ecd85eacbfa5b7379f563b83807d3e5258cf700" channel: "guix">)> #<<evaluation> id: 734789 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "3bb145b6e2a8c84e7739ead9ae76dc4d42bb9850" channel: "guix">)> #<<evaluation> id: 734747 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "033cbd11a837dbc7602799f15d691221653e1996" channel: "guix">)> #<<evaluation> id: 734733 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "c9eac0c3553703385c997a70741348ae5d34dc68" channel: "guix">)> #<<evaluation> id: 734718 spec: "master" complete?: #t checkouts: (#<<checkout> commit: "2589997fab07bcebe6d87fd227852389ab5b1962" channel: "guix">)>) > scheme@(guile-user)> (define jobs (evaluation-jobs "https://ci.guix.gnu.org" (evaluation-id (car $14)))) > scheme@(guile-user)> (length jobs) > $15 = 76649 > scheme@(guile-user)> (car jobs) > $16 = #<<job> build-id: 1627000 status: scheduled name: "0ad.aarch64-linux"> > scheme@(guile-user)> ,use(srfi srfi-1) > scheme@(guile-user)> (find (lambda (job) (string=? "rust.x86_64-linux" (job-name job))) jobs) > $17 = #<<job> build-id: 1275492 status: succeeded name: "rust.x86_64-linux"> > scheme@(guile-user)> (job-build "https://ci.guix.gnu.org" $17) > $18 = #<<build> id: 1275492 derivation: "/gnu/store/dmzhvql66a1n31j2xpygfxpw576wpkwb-rust-1.60.0.drv" evaluation: 579940 system: "x86_64-linux" status: succeeded timestamp: 1662284654 start-time: #<date nanosecond: 0 second: 14 minute: 44 hour: 11 day: 4 month: 9 year: 2022 zone-offset: 7200> stop-time: #<date nanosecond: 0 second: 14 minute: 44 hour: 11 day: 4 month: 9 year: 2022 zone-offset: 7200> products: ()> > --8<---------------cut here---------------end--------------->8--- > > That still doesn’t answer your question, because you’re interested in > the build time of the full Rust bootstrap chain, but that’s a start! > > HTH, > Ludo’. > -- Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-20 20:11 ` Efraim Flashner @ 2022-10-22 8:57 ` Félix Baylac Jacqué 0 siblings, 0 replies; 12+ messages in thread From: Félix Baylac Jacqué @ 2022-10-22 8:57 UTC (permalink / raw) To: Efraim Flashner, Ludovic Courtès; +Cc: guix-devel Hi Efraim, Ludovic, Ah! Great! I did not realize the full bootstrapping was happening in a single Cuirass build job :) 3.3 hours doesn't seem so bad in the grand scheme of things in the end. I was expecting things to be worse than that. Thanks a lot for the help. Félix ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-15 7:33 How long does it take to run the full rustc bootstrap chain? Félix Baylac Jacqué 2022-10-19 19:52 ` Efraim Flashner 2022-10-20 9:59 ` Ludovic Courtès @ 2022-10-22 13:48 ` Maxim Cournoyer 2022-10-26 19:37 ` bokr 2 siblings, 1 reply; 12+ messages in thread From: Maxim Cournoyer @ 2022-10-22 13:48 UTC (permalink / raw) To: Félix Baylac Jacqué; +Cc: guix-devel Hi, Félix Baylac Jacqué <felix@alternativebit.fr> writes: > Hey Guix, > > I'd be curious to know how long it takes to run the full rustc bootstrap > chain on the Guix build farm. I'm sadly not sure how to approach this > problem. > > Is there a way to extract this information from Cuirass or the Guix data > service? > > Félix It used to be 16 hours on a Ryzen 3900x machine, then it got halved to 8 hours with the work to bootstrap from 1.39, and recently we're bootstrapping from 1.54, so it must have been greatly reduced again. Looking at (gnu packages rust), the mrustc-based bootstrap starts with 1.54.0. This one is expensive, probably around 1 h 30 or more on a Ryzen 3900x CPU (24 logical CPUs). The intermediate builds are typically around 15-20 minutes on that machines, with the last one taking a bit more (30 minutes), so the current bootstrap on such a machine should take about: 1.54.0: 1h30m 1.55.0 - 1.60.0: 6 X 20 min = 1h20m 1.60.0: final build with tests and extra tools: 30 min The total should be around 3 h 20 on a fast modern x86_64 machine. I suppose the time for berlin to build it takes about this. HTH! -- Thanks, Maxim ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-22 13:48 ` Maxim Cournoyer @ 2022-10-26 19:37 ` bokr 2022-10-27 14:35 ` Maxim Cournoyer 2022-11-06 9:08 ` Efraim Flashner 0 siblings, 2 replies; 12+ messages in thread From: bokr @ 2022-10-26 19:37 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: Félix Baylac Jacqué, guix-devel Hi, On +2022-10-22 09:48:50 -0400, Maxim Cournoyer wrote: > Hi, > > Félix Baylac Jacqué <felix@alternativebit.fr> writes: > > > Hey Guix, > > > > I'd be curious to know how long it takes to run the full rustc bootstrap > > chain on the Guix build farm. I'm sadly not sure how to approach this > > problem. > > > > Is there a way to extract this information from Cuirass or the Guix data > > service? > > > > Félix > > It used to be 16 hours on a Ryzen 3900x machine, then it got halved to 8 > hours with the work to bootstrap from 1.39, and recently we're > bootstrapping from 1.54, so it must have been greatly reduced again. > > Looking at (gnu packages rust), the mrustc-based bootstrap starts with > 1.54.0. This one is expensive, probably around 1 h 30 or more on a > Ryzen 3900x CPU (24 logical CPUs). > > The intermediate builds are typically around 15-20 minutes on that > machines, with the last one taking a bit more (30 minutes), so the > current bootstrap on such a machine should take about: > > 1.54.0: 1h30m > 1.55.0 - 1.60.0: 6 X 20 min = 1h20m > 1.60.0: final build with tests and extra tools: 30 min > > The total should be around 3 h 20 on a fast modern x86_64 machine. I > suppose the time for berlin to build it takes about this. > > HTH! > > -- > Thanks, > Maxim > I'm curious what --8<---------------cut here---------------start------------->8--- $ lsblk -o size,model,type,tran,vendor,name|grep -Ei 'ssd|model';echo;lspci |grep -i nvme --8<---------------cut here---------------end--------------->8--- on your relevant machines would show. I opted for the best SSD available for my purism librem13v4 at the time, and was really happy with seems like 10x faster than the SATA SSD in my older but still i7 x86_64 previous laptop. Prob really 4-5x faster. So above combo command line now gives me --8<---------------cut here---------------start------------->8--- SIZE MODEL TYPE TRAN VENDOR NAME 465.8G Samsung SSD 970 EVO Plus 500GB disk nvme nvme0n1 01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 $ --8<---------------cut here---------------end--------------->8--- What /is/has been/ on your machines? Could your improved times be part from SSD/controller changes? There's really a huge difference between SATA and 4-lane pci (where both ends can handle it, which may require fw update or not be available) Obviously 4 lanes is also going to be faster than one. -- Regards, Bengt Richter ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-26 19:37 ` bokr @ 2022-10-27 14:35 ` Maxim Cournoyer 2022-10-28 12:00 ` Has guix system ever run a hardware survey? Joshua Branson 2022-10-31 19:02 ` How long does it take to run the full rustc bootstrap chain? Bengt Richter 2022-11-06 9:08 ` Efraim Flashner 1 sibling, 2 replies; 12+ messages in thread From: Maxim Cournoyer @ 2022-10-27 14:35 UTC (permalink / raw) To: bokr; +Cc: Félix Baylac Jacqué, guix-devel Hi, bokr@bokr.com writes: [...] > I'm curious what > > $ lsblk -o size,model,type,tran,vendor,name|grep -Ei 'ssd|model';echo;lspci |grep -i nvme > > on your relevant machines would show. > > I opted for the best SSD available for my purism librem13v4 at the time, > and was really happy with seems like 10x faster than the SATA SSD in my older > but still i7 x86_64 previous laptop. Prob really 4-5x faster. > > So above combo command line now gives me > > SIZE MODEL TYPE TRAN VENDOR NAME > 465.8G Samsung SSD 970 EVO Plus 500GB disk nvme nvme0n1 > > 01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 > $ > > What /is/has been/ on your machines? Could your improved times be part from SSD/controller changes? > > There's really a huge difference between SATA and 4-lane pci > (where both ends can handle it, which may require fw update or not be available) > Obviously 4 lanes is also going to be faster than one. --8<---------------cut here---------------start------------->8--- $ lsblk -o size,model,type,tran,vendor,name|grep -Ei 'ssd|model';echo;lspci |grep -i nvme SIZE MODEL TYPE TRAN VENDOR NAME 465.8G Samsung SSD 860 EVO 500GB disk sata ATA sda 931.5G Samsung SSD 840 EVO 1TB disk sata ATA sdc --8<---------------cut here---------------end--------------->8--- Building Rust is mostly CPU dependent; I think fast single thread performance is key as not that much happen in parallel, IIRC. The 3900X is a 12 cores (24 logical) beast. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 12+ messages in thread
* Has guix system ever run a hardware survey? 2022-10-27 14:35 ` Maxim Cournoyer @ 2022-10-28 12:00 ` Joshua Branson 2022-10-31 19:02 ` How long does it take to run the full rustc bootstrap chain? Bengt Richter 1 sibling, 0 replies; 12+ messages in thread From: Joshua Branson @ 2022-10-28 12:00 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: bokr, Félix Baylac Jacqué, guix-devel Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > Hi, > > bokr@bokr.com writes: > > [...] > >> >> I opted for the best SSD available for my purism librem13v4 at the time, >> and was really happy with seems like 10x faster than the SATA SSD in my older >> but still i7 x86_64 previous laptop. Prob really 4-5x faster. >> > > Building Rust is mostly CPU dependent; I think fast single thread > performance is key as not that much happen in parallel, IIRC. The 3900X > is a 12 cores (24 logical) beast. > This makes me wonder what sort of data we'd get if we asked guix system users what sort of machine they use guix system on. I've got an os-booted Thinkpad T400. My core 2 duo seems pretty slow compared to what ya'll have. :) ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-27 14:35 ` Maxim Cournoyer 2022-10-28 12:00 ` Has guix system ever run a hardware survey? Joshua Branson @ 2022-10-31 19:02 ` Bengt Richter 2022-11-01 12:49 ` Maxim Cournoyer 1 sibling, 1 reply; 12+ messages in thread From: Bengt Richter @ 2022-10-31 19:02 UTC (permalink / raw) To: Maxim Cournoyer; +Cc: Félix Baylac Jacqué, guix-devel Hi again, thanks for your reply... On +2022-10-27 10:35:02 -0400, Maxim Cournoyer wrote: > Hi, > (Oops, pasting back alternative I thought would be faster) > > So above combo command line now gives me > > --8<---------------cut here---------------start------------->8--- > > SIZE MODEL TYPE TRAN VENDOR NAME > > 465.8G Samsung SSD 970 EVO Plus 500GB disk nvme nvme0n1 > > > > 01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 > > $ > > --8<---------------cut here---------------end--------------->8--- [...] --8<---------------cut here---------------start------------->8--- > $ lsblk -o size,model,type,tran,vendor,name|grep -Ei 'ssd|model';echo;lspci |grep -i nvme > SIZE MODEL TYPE TRAN VENDOR NAME > 465.8G Samsung SSD 860 EVO 500GB disk sata ATA sda > 931.5G Samsung SSD 840 EVO 1TB disk sata ATA sdc > --8<---------------cut here---------------end--------------->8--- > > Building Rust is mostly CPU dependent; I think fast single thread > performance is key as not that much happen in parallel, IIRC. The 3900X > is a 12 cores (24 logical) beast. > Hm, just TRAN sata, no nvme, so it's going to be slow, but what is the effect on what you timed? Is there an easy way to get a measure of how many GB went through those SATA channels during what you timed? That would give an idea of what faster phusical disk memory access would do for you. If many people are waiting longer that they like, maybe they would chip in to fund an upgrade, to feed that 12(24)-core "beast" :-) I'd bet it is waiting a lot, if not more than computing :) -- Regards, Bengt Richter ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-31 19:02 ` How long does it take to run the full rustc bootstrap chain? Bengt Richter @ 2022-11-01 12:49 ` Maxim Cournoyer 0 siblings, 0 replies; 12+ messages in thread From: Maxim Cournoyer @ 2022-11-01 12:49 UTC (permalink / raw) To: Bengt Richter; +Cc: Félix Baylac Jacqué, guix-devel Hi Bengt, Bengt Richter <bokr@bokr.com> writes: [...] >> Building Rust is mostly CPU dependent; I think fast single thread >> performance is key as not that much happen in parallel, IIRC. The 3900X >> is a 12 cores (24 logical) beast. >> > > Hm, just TRAN sata, no nvme, so it's going to be slow, but > what is the effect on what you timed? > > Is there an easy way to get a measure of how many GB went > through those SATA channels during what you timed? That > would give an idea of what faster phusical disk memory > access would do for you. If many people are waiting longer > that they like, maybe they would chip in to fund an upgrade, > to feed that 12(24)-core "beast" :-) I really doubt it's IO bound, from experience building it multiple times. But I don't have precise data; if you are interested in learning more, I encourage you to run something like 'guix build --check --no-grafts rust' or 'guix build --check --no-grafts -e '(@@ (gnu packages rust) rust-1.55) on your machine monitor the load via atop or similar tools. -- Thanks, Maxim ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: How long does it take to run the full rustc bootstrap chain? 2022-10-26 19:37 ` bokr 2022-10-27 14:35 ` Maxim Cournoyer @ 2022-11-06 9:08 ` Efraim Flashner 1 sibling, 0 replies; 12+ messages in thread From: Efraim Flashner @ 2022-11-06 9:08 UTC (permalink / raw) To: bokr; +Cc: Maxim Cournoyer, Félix Baylac Jacqué, guix-devel [-- Attachment #1: Type: text/plain, Size: 3165 bytes --] On Wed, Oct 26, 2022 at 09:37:32PM +0200, bokr@bokr.com wrote: > Hi, > > On +2022-10-22 09:48:50 -0400, Maxim Cournoyer wrote: > > Hi, > > > > Félix Baylac Jacqué <felix@alternativebit.fr> writes: > > > > > Hey Guix, > > > > > > I'd be curious to know how long it takes to run the full rustc bootstrap > > > chain on the Guix build farm. I'm sadly not sure how to approach this > > > problem. > > > > > > Is there a way to extract this information from Cuirass or the Guix data > > > service? > > > > > > Félix > > > > It used to be 16 hours on a Ryzen 3900x machine, then it got halved to 8 > > hours with the work to bootstrap from 1.39, and recently we're > > bootstrapping from 1.54, so it must have been greatly reduced again. > > > > Looking at (gnu packages rust), the mrustc-based bootstrap starts with > > 1.54.0. This one is expensive, probably around 1 h 30 or more on a > > Ryzen 3900x CPU (24 logical CPUs). > > > > The intermediate builds are typically around 15-20 minutes on that > > machines, with the last one taking a bit more (30 minutes), so the > > current bootstrap on such a machine should take about: > > > > 1.54.0: 1h30m > > 1.55.0 - 1.60.0: 6 X 20 min = 1h20m > > 1.60.0: final build with tests and extra tools: 30 min > > > > The total should be around 3 h 20 on a fast modern x86_64 machine. I > > suppose the time for berlin to build it takes about this. > > > > HTH! > > > > -- > > Thanks, > > Maxim > > > > I'm curious what > --8<---------------cut here---------------start------------->8--- > $ lsblk -o size,model,type,tran,vendor,name|grep -Ei 'ssd|model';echo;lspci |grep -i nvme > --8<---------------cut here---------------end--------------->8--- > on your relevant machines would show. > > I opted for the best SSD available for my purism librem13v4 at the time, > and was really happy with seems like 10x faster than the SATA SSD in my older > but still i7 x86_64 previous laptop. Prob really 4-5x faster. > > So above combo command line now gives me > --8<---------------cut here---------------start------------->8--- > SIZE MODEL TYPE TRAN VENDOR NAME > 465.8G Samsung SSD 970 EVO Plus 500GB disk nvme nvme0n1 > > 01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 > $ > --8<---------------cut here---------------end--------------->8--- > > What /is/has been/ on your machines? Could your improved times be part from SSD/controller changes? > > There's really a huge difference between SATA and 4-lane pci > (where both ends can handle it, which may require fw update or not be available) > Obviously 4 lanes is also going to be faster than one. SIZE MODEL TYPE TRAN VENDOR NAME 931.5G NVME SSD 1TB disk nvme nvme0n1 01:00.0 Non-Volatile memory controller: Silicon Motion, Inc. Device 2263 (rev 03) -- Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2022-11-06 9:08 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-10-15 7:33 How long does it take to run the full rustc bootstrap chain? Félix Baylac Jacqué 2022-10-19 19:52 ` Efraim Flashner 2022-10-20 9:59 ` Ludovic Courtès 2022-10-20 20:11 ` Efraim Flashner 2022-10-22 8:57 ` Félix Baylac Jacqué 2022-10-22 13:48 ` Maxim Cournoyer 2022-10-26 19:37 ` bokr 2022-10-27 14:35 ` Maxim Cournoyer 2022-10-28 12:00 ` Has guix system ever run a hardware survey? Joshua Branson 2022-10-31 19:02 ` How long does it take to run the full rustc bootstrap chain? Bengt Richter 2022-11-01 12:49 ` Maxim Cournoyer 2022-11-06 9:08 ` Efraim Flashner
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).