* FWD: Test reader speed of Guile 3.0.6 in Lilypond [not found] ` <CAOw_e7ZxEWWU6mxJx2cNgb4nyjXnw8ZtYCa+Lp_k=-2WkSpGCg@mail.gmail.com> @ 2021-03-18 8:30 ` Dr. Arne Babenhauserheide 2021-03-18 17:21 ` Dr. Arne Babenhauserheide 2021-04-11 21:32 ` Dr. Arne Babenhauserheide 0 siblings, 2 replies; 4+ messages in thread From: Dr. Arne Babenhauserheide @ 2021-03-18 8:30 UTC (permalink / raw) To: guile-devel [-- Attachment #1: Type: text/plain, Size: 3061 bytes --] Hi, due to the reader-rewrite, I asked in the lilypond-list for testing, because they are likely the strongest power-user of the reader. Han-Wen Nienhuys checked performance in the benchmark suite: > $ cat ../bench.ly > #(define (microseconds) > (let* ((t (gettimeofday)) > (us (/ (cdr t) 1000000.0))) > (+ (car t) us))) > > #(define start (microseconds)) > > % \include "bench-largeexp.ly" > \include "bench-manysmall.ly" > > #(newline) > #(display (- (microseconds) start)) > > Parsing & evaluating '(1 2 3) 200 times. > - guile 1.8: 1.25ms > - guile 2.2: 3.2ms > - guile 3.0.6: 2.08ms > > Parsing & evaluating the giant expression in define-grobs.scm once: > > - guile 1.8: 10.6ms > - guile 2.2: 166ms > - guile 3.0.6: 71ms > > Parsing & evaluating the giant expression in define-grobs.scm once > (but quoted, ie. not real evaluation): > > - guile 1.8: 10.0ms > - guile 2.2: 13ms > - guile 3.0.6: 12.8ms > > In summary, the read speed itself for large expressions is on the same > order as 1.8, but for many small expressions (which is a much more > common use-case) there is still a 60% slowdown. >> asking here because I want to avoid surprising and avoidable changes >> that block Lilypond. I consider Lilypond to be the most important >> flagship project of Guile, and I want to do what I can to prevent >> unnecessary friction. > I appreciate the heads up you gave here today, but from our side, it > doesn't seem like the Guile project is much concerned with our needs. > The evaluation speed of GUILE 3.x is still pretty poor. Having fast, > JIT'ed code seems interesting in theory, but the way it's implemented > in Guile 3.x is a giant headache: the separate byte compilation is > extremely slow, and it is hard to manage (where should the .go files > be stored/installed, how/when are they generated etc.). It also > doesn't match our use case, because a lot of the code that we have > comes from .ly files, so it cannot be precompiled. Are there ways to improve this? For example sticking to the baseline compiler (as described in the wingolog: https://wingolog.org/archives/2020/06/03/a-baseline-compiler-for-guile ) >> The article linked above shows that setting -O1 as optimization of the >> code could help (if you’re not already doing that). > The article gives a pointer to the code, which is > module/language/tree-il/compile-bytecode.scm. I ran > > for c in $(git log module/language/tree-il/compile-bytecode.scm|grep > commit|awk '{print $2;}'); do git show $c ; done | grep -i doc > > to look for documentation, but couldn't find it. The module has one > exported symbol, which is compile-bytecode. > > Could you give some practical tips on how we'd use this? Can someone help with this? It would be pretty cool if there were a simple way to speed-up lilypond like changing optimization level that just got lost in communication (or missing docs). Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 1125 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: FWD: Test reader speed of Guile 3.0.6 in Lilypond 2021-03-18 8:30 ` FWD: Test reader speed of Guile 3.0.6 in Lilypond Dr. Arne Babenhauserheide @ 2021-03-18 17:21 ` Dr. Arne Babenhauserheide 2021-04-11 21:32 ` Dr. Arne Babenhauserheide 1 sibling, 0 replies; 4+ messages in thread From: Dr. Arne Babenhauserheide @ 2021-03-18 17:21 UTC (permalink / raw) To: guile-devel [-- Attachment #1: Type: text/plain, Size: 4335 bytes --] More tests were done by Thomas Morley, which also give a perspective on the initial very negative reaction by the lilypond developers, as it shows the abysmal speed that initial versions of Guile2-Lilypond integration had. Thomas Morley <thomasmorley65@gmail.com> writes: > Am Fr., 12. März 2021 um 10:45 Uhr schrieb Thomas Morley > <thomasmorley65@gmail.com>: >> >> Am Do., 11. März 2021 um 22:47 Uhr schrieb Dr. Arne Babenhauserheide > >> > Do you already have results? >> >> Currently my regular job eats up all my time. >> Probably I'll find some free time at upcoming weekend, not sure though... >> If so, to which guile-version should I compare, guile-1.8 (which is >> still LilyPond-default) or a certain guile-2/3-version? > > Found some time today... > > I tested a huge ly-score, once sent by Jean-Charles for testing > different guile-versions: > https://lists.gnu.org/archive/html/lilypond-user/2016-11/msg00948.html > Thanks again! > I made it compilable with recent ly-master out of > 55ca32d951dd17556df8d0332b861f0cd189379a > (though I didn't fix all warnings and programming errors) > > Tested with these guile versions: > guile-1.8.8 from tarball > guile-2-2-6 out of a69b567d97f7c9193924c775e1dd86e43a35b8bd > guile-3.0.6 from master, i.e. out of 85433fc2b122dc78342c3c83941949d1d9318399 > (although (version) says: 3.0.5.116-85433) > > There are some guile-warnings, multiple instances of: > WARNING: (#{ g478}#): `smallCaps-markup' imported from both (lily) and > (#{ g171}#) > WARNING: (scm framework-ps): imported module (lily) overrides core binding `_' > The latter one is about gettext and PITA. > > > I compiled Urtext.ly from the tarball linked above (giving a 158 pages pdf): > > %%%%%%%%%%%%%%%%%%%% > guile-1.8.8 > > real 4m37,304s > user 4m14,144s > sys 0m7,717s > > %%%%%%%%%%%%%%%%%%%% > guile-2.2.6, (version) says: 2.2.6.7-9a11b > > real 7m18,995s > user 6m41,808s > sys 0m11,960s > > eating much more memory than guile-1.8.8 > > %%%%%%%%%%%%%%%%%%%% > guile-3.0.6, (version) says: 3.0.5.116-85433 > > real 5m49,587s > user 5m11,033s > sys 0m9,488s > > eating a little more memory than with guile-1.8.8 but far less than guile-2.2.6 … > I tested _one_ huge file. > In it there's some custom-code to deal with accented letters like in > "éèêëáàâäíìîïóòôöúùûüçœæ", which guile-1 was not able to cope with in > many cases. > I think guile-2/3 does a better job in this regard. > Btw, this is my strongest argument to switch to guile-2/3 > I could eliminate that code for guile-2/3 runs and take timing values again. > Maybe it has positive impact. > Interested? Has to wait for the weekend, though. > > Otoh, adjusting small things in a small file like finding the needed value for > \override Grob.X-offset = FINDME > is a pain with guile-2/3 because of the already mentioned startup delay. > The strongest argument against switching to guile-2/3, imho. … > %%%%%%%%%%%%%%%%%%%% > %%%%%%%%%%%%%%%%%%%% > > Interestingly I found some older timing values from former tests: > > %%%%%%%%%%%%%%%%%%%% > released lilypond-2.19.52 with guile-1.8.7 > > real 8m16.191s > user 6m39.864s > sys 0m10.860s > > %%%%%%%%%%%%%%%%%%%% > with guile-versions from 2.0.14 and 2.1.7 I noted: > > Best result was: > real 25m17.082s > user 31m1.668s > sys 0m5.480s > > Worst result was: > real 77m44.348s > user 100m40.324s > sys 0m18.932s > > Though, my noted former results are a little messy: I can't say > anymore which guile-version with which lilypond-patches was > best/worst. > > Anyway, both, LilyPond and guile seem to have improved a lot. > A: Did you do these tests on the same machine? > Yes, it was all on the same machine. > Though, guile-2.0.12 was the first version with a _chance_ to compile > LilyPond, iirc > These early attempts used patches which nowadays are superseeded by > far better ones. > So while the machine is the same, LilyPond has become far better to > deal with guile-2/3. > I think those scary values are documenting where we started and are of > historic interest, nothing more. > Cheers, > Harm -- Unpolitisch sein heißt politisch sein ohne es zu merken [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 1125 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: FWD: Test reader speed of Guile 3.0.6 in Lilypond 2021-03-18 8:30 ` FWD: Test reader speed of Guile 3.0.6 in Lilypond Dr. Arne Babenhauserheide 2021-03-18 17:21 ` Dr. Arne Babenhauserheide @ 2021-04-11 21:32 ` Dr. Arne Babenhauserheide 1 sibling, 0 replies; 4+ messages in thread From: Dr. Arne Babenhauserheide @ 2021-04-11 21:32 UTC (permalink / raw) To: guile-devel [-- Attachment #1: Type: text/plain, Size: 1223 bytes --] Hi, I’d like to ask again if anyone can help the Lilypond folks with setting optimization settings for code they compile from lilypond-files: > Are there ways to improve this? For example sticking to the baseline > compiler (as described in the wingolog: > https://wingolog.org/archives/2020/06/03/a-baseline-compiler-for-guile ) > >>> The article linked above shows that setting -O1 as optimization of the >>> code could help (if you’re not already doing that). > >> The article gives a pointer to the code, which is >> module/language/tree-il/compile-bytecode.scm. I ran >> >> for c in $(git log module/language/tree-il/compile-bytecode.scm|grep >> commit|awk '{print $2;}'); do git show $c ; done | grep -i doc >> >> to look for documentation, but couldn't find it. The module has one >> exported symbol, which is compile-bytecode. >> >> Could you give some practical tips on how we'd use this? > > Can someone help with this? > > It would be pretty cool if there were a simple way to speed-up lilypond > like changing optimization level that just got lost in communication (or > missing docs). Best wishes, Arne -- Unpolitisch sein heißt politisch sein ohne es zu merken [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 1125 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <f352b8c6-6a8b-4a6d-85ac-ec32cd33fc3a@t-online.de>]
[parent not found: <e968a361-397e-8634-4521-ad1a4801b9de@gmail.com>]
[parent not found: <cab2cdfa-6c45-c07e-8c5e-be3dde767281@t-online.de>]
[parent not found: <CAGebFpKhKtQ2u6N2Le-fc1A4sp=9siHAbPkV4RCS6GOZ84YF+w@mail.gmail.com>]
[parent not found: <11662b16-c4e3-6d48-7496-6914b35e9c8e@t-online.de>]
* Fwd: Re: Test reader speed of Guile 3.0.6 [not found] ` <11662b16-c4e3-6d48-7496-6914b35e9c8e@t-online.de> @ 2021-03-21 8:21 ` Dr. Arne Babenhauserheide 0 siblings, 0 replies; 4+ messages in thread From: Dr. Arne Babenhauserheide @ 2021-03-21 8:21 UTC (permalink / raw) To: guile-devel [-- Attachment #1: Type: text/plain, Size: 481 bytes --] Knut Petersen: >> I wonder what the Carver MSDM score would look like. Searching the archives, it's seemingly become an informal standard for large-project stress tests. > > Carver MDSM compiled with guile master and lilypond master: 19.7 seconds > Carver MDSM compiled with guile 1.8 + lilypond master: 14.6 seconds > For the Carver MDSM we see a performance penalty of only about 35% for guile master. -- Unpolitisch sein heißt politisch sein ohne es zu merken [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 1125 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-04-11 21:32 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <878s6vaa7c.fsf@web.de> [not found] ` <CAOw_e7b8-AAROs4SrB+E=fG=ORDdjgG_AbcWW2ytS1tHHmAGdA@mail.gmail.com> [not found] ` <87v99wf1n0.fsf@web.de> [not found] ` <CAOw_e7ZxEWWU6mxJx2cNgb4nyjXnw8ZtYCa+Lp_k=-2WkSpGCg@mail.gmail.com> 2021-03-18 8:30 ` FWD: Test reader speed of Guile 3.0.6 in Lilypond Dr. Arne Babenhauserheide 2021-03-18 17:21 ` Dr. Arne Babenhauserheide 2021-04-11 21:32 ` Dr. Arne Babenhauserheide [not found] ` <f352b8c6-6a8b-4a6d-85ac-ec32cd33fc3a@t-online.de> [not found] ` <e968a361-397e-8634-4521-ad1a4801b9de@gmail.com> [not found] ` <cab2cdfa-6c45-c07e-8c5e-be3dde767281@t-online.de> [not found] ` <CAGebFpKhKtQ2u6N2Le-fc1A4sp=9siHAbPkV4RCS6GOZ84YF+w@mail.gmail.com> [not found] ` <11662b16-c4e3-6d48-7496-6914b35e9c8e@t-online.de> 2021-03-21 8:21 ` Fwd: Re: Test reader speed of Guile 3.0.6 Dr. Arne Babenhauserheide
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).