unofficial mirror of guile-devel@gnu.org 
 help / color / mirror / Atom feed
From: "Dr. Arne Babenhauserheide" <arne_bab@web.de>
To: "Linus Björnstam" <linus.bjornstam@veryfast.biz>
Cc: guile-user <guile-user@gnu.org>, guile-devel@gnu.org
Subject: Re: Guile Potluck 2021
Date: Sat, 27 Feb 2021 17:43:51 +0100	[thread overview]
Message-ID: <87lfb98nqw.fsf@web.de> (raw)
In-Reply-To: <87o8g58nsv.fsf@web.de>

[-- Attachment #1: Type: text/plain, Size: 2053 bytes --]

PS: Still it is important to get this code fast, because it is the
    fallback for all situations where I cannot cheat and only calculate
    a subset of the trust, and it is required at startup.

Dr. Arne Babenhauserheide <arne_bab@web.de> writes:

> Hi Linus,
>
> Linus Björnstam <linus.bjornstam@veryfast.biz> writes:
>> I had a look and there is quite a lot you can do. Just out of
>> curiosity: how much is GC time? You are generating a lot of
>> intermediate data. Instead of vector-append! maybe use something like
>> the new vectorlist srfi? If you show me a flame graph and give me some
>> test data I can have a go!
>
> Thank you!
>
> I expect GC time to be roughly half during the trust calculation (much
> more during import), because this fills up two cores.
>
> I can’t easily give you a flame graph (except if you can tell me how to
> generate it), but I now pushed a version in pure Scheme (instead of
> wisp) that runs a huge test:
>
> hg clone https://hg.sr.ht/~arnebab/wispwot
> cd wispwot
> ./run-wispwot.scm --test
>
> This imports 200k trust edges (which takes a lot of time) and then does
> three full trust calculations (which each take around 70s).
>
> The most important limitation for optimization is that the memory use
> must stay reasonable with 64k IDs which each have 1000 trust edges. This
> memory use is what the current code is optimized for (that’s why there
> are all those u8vectors and u16vectors).
>
> For the math: at 64 million trust edges, this should currently require
> (in theory) less than 200MiB of memory for the actual trust structure (2
> byte for the id, 1 byte for the trust).
>
> Going up to 300MiB for time-optimization would still be viable, but it
> shouldn’t go beyond that (there are algorithmic options to reduce the
> amount of work done per update, so the runtime of the current code is
> important but it is not a life-and-death situation).
>
> Best wishes,
> Arne


-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

  reply	other threads:[~2021-02-27 16:43 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <be71b3d5db9c43723c749f108dbd1ada7785daf2.camel.ref@yahoo.com>
2021-02-18 17:24 ` Guile Potluck 2021 Mike Gran
2021-02-20 17:38   ` Alex Sassmannshausen
2021-02-21 14:20     ` Dr. Arne Babenhauserheide
2021-02-21 17:21   ` Daniel Tornabene
2021-02-22  5:59   ` Aleix Conchillo Flaqué
2021-02-26 23:19     ` Aleix Conchillo Flaqué
2021-02-27  1:08       ` Mike Gran
2021-02-27  9:39         ` Dr. Arne Babenhauserheide
2021-02-27 10:14           ` Linus Björnstam
2021-02-27 16:42             ` Dr. Arne Babenhauserheide
2021-02-27 16:43               ` Dr. Arne Babenhauserheide [this message]
2021-03-02  7:08                 ` Dr. Arne Babenhauserheide
2021-03-01 11:34   ` pukkamustard
2021-03-01 21:14   ` Linus Björnstam
2021-03-01 21:37     ` Aleix Conchillo Flaqué
2021-03-02  0:43       ` Dr. Arne Babenhauserheide

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/guile/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87lfb98nqw.fsf@web.de \
    --to=arne_bab@web.de \
    --cc=guile-devel@gnu.org \
    --cc=guile-user@gnu.org \
    --cc=linus.bjornstam@veryfast.biz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).