unofficial mirror of guile-user@gnu.org 
 help / color / mirror / Atom feed
From: "Dr. Arne Babenhauserheide" <arne_bab@web.de>
To: "Linus Björnstam" <linus.bjornstam@veryfast.biz>
Cc: guile-user <guile-user@gnu.org>, guile-devel@gnu.org
Subject: Re: Guile Potluck 2021
Date: Sat, 27 Feb 2021 17:42:40 +0100	[thread overview]
Message-ID: <87o8g58nsv.fsf@web.de> (raw)
In-Reply-To: <10585499-c6ef-4cce-b576-60940b3aa4d9@www.fastmail.com>

[-- Attachment #1: Type: text/plain, Size: 1725 bytes --]

Hi Linus,

Linus Björnstam <linus.bjornstam@veryfast.biz> writes:
> I had a look and there is quite a lot you can do. Just out of
> curiosity: how much is GC time? You are generating a lot of
> intermediate data. Instead of vector-append! maybe use something like
> the new vectorlist srfi? If you show me a flame graph and give me some
> test data I can have a go!

Thank you!

I expect GC time to be roughly half during the trust calculation (much
more during import), because this fills up two cores.

I can’t easily give you a flame graph (except if you can tell me how to
generate it), but I now pushed a version in pure Scheme (instead of
wisp) that runs a huge test:

hg clone https://hg.sr.ht/~arnebab/wispwot
cd wispwot
./run-wispwot.scm --test

This imports 200k trust edges (which takes a lot of time) and then does
three full trust calculations (which each take around 70s).

The most important limitation for optimization is that the memory use
must stay reasonable with 64k IDs which each have 1000 trust edges. This
memory use is what the current code is optimized for (that’s why there
are all those u8vectors and u16vectors).

For the math: at 64 million trust edges, this should currently require
(in theory) less than 200MiB of memory for the actual trust structure (2
byte for the id, 1 byte for the trust).

Going up to 300MiB for time-optimization would still be viable, but it
shouldn’t go beyond that (there are algorithmic options to reduce the
amount of work done per update, so the runtime of the current code is
important but it is not a life-and-death situation).

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

  reply	other threads:[~2021-02-27 16:42 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <be71b3d5db9c43723c749f108dbd1ada7785daf2.camel.ref@yahoo.com>
2021-02-18 17:24 ` Guile Potluck 2021 Mike Gran
2021-02-20 17:38   ` Alex Sassmannshausen
2021-02-21 14:20     ` Dr. Arne Babenhauserheide
2021-02-21 17:21   ` Daniel Tornabene
2021-02-22  5:59   ` Aleix Conchillo Flaqué
2021-02-26 23:19     ` Aleix Conchillo Flaqué
2021-02-27  1:08       ` Mike Gran
2021-02-27  9:39         ` Dr. Arne Babenhauserheide
2021-02-27 10:14           ` Linus Björnstam
2021-02-27 16:42             ` Dr. Arne Babenhauserheide [this message]
2021-02-27 16:43               ` Dr. Arne Babenhauserheide
2021-03-02  7:08                 ` Dr. Arne Babenhauserheide
2021-03-01 11:34   ` pukkamustard
2021-03-01 21:14   ` Linus Björnstam
2021-03-01 21:37     ` Aleix Conchillo Flaqué
2021-03-02  0:43       ` Dr. Arne Babenhauserheide
2021-03-07  5:28   ` Extensible effects in Guile (Guile Potluck 2021) Walter Lewis
2021-03-08 15:42     ` Extensible effects in Guile Walter Lewis
2021-03-08 20:58       ` Ludovic Courtès
2021-03-08 21:54         ` Walter Lewis
2021-02-25 21:08 Guile Potluck 2021 Zelphir Kaltstahl

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/guile/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87o8g58nsv.fsf@web.de \
    --to=arne_bab@web.de \
    --cc=guile-devel@gnu.org \
    --cc=guile-user@gnu.org \
    --cc=linus.bjornstam@veryfast.biz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).