unofficial mirror of guile-user@gnu.org 
 help / color / mirror / Atom feed
* Guile Potluck 2021
       [not found] <be71b3d5db9c43723c749f108dbd1ada7785daf2.camel.ref@yahoo.com>
@ 2021-02-18 17:24 ` Mike Gran
  2021-02-20 17:38   ` Alex Sassmannshausen
                     ` (5 more replies)
  0 siblings, 6 replies; 21+ messages in thread
From: Mike Gran @ 2021-02-18 17:24 UTC (permalink / raw)
  To: guile-user, guile-devel

Hello All-

In celebration of the (slightly belated) 10-year anniversary of Guile
v2.0, we're having another Guile Potluck!  The Guile Potluck is a
randomly annual event to give people a chance to show off their Guile
projects and skills.  Think of it as a game jam, but, not constrained
to games. 

To participate, on or before Mar 6, send an email to guile-user@gnu.org
with instructions on how to find your entry, which could be anything
you like.  For example,

   - a script showing off some feature of Guile or your favorite Guile
   library
   - a blog post describing something interesting about Guile
   - an updated release of a neglected library
   - a mini-game
   - a graphical or audio demoscene-type demo
   
There probably won't be any prizes.  But there will definitely be an e-
mail and blog post about the entries.

If you think you might attempt to participate, please reply to this e-
mail so I can gauge the feasibility of some sort of participation swag.

Regards,
Mike Gran, on behalf of the Guile team





^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-18 17:24 ` Guile Potluck 2021 Mike Gran
@ 2021-02-20 17:38   ` Alex Sassmannshausen
  2021-02-21 14:20     ` Dr. Arne Babenhauserheide
  2021-02-21 17:21   ` Daniel Tornabene
                     ` (4 subsequent siblings)
  5 siblings, 1 reply; 21+ messages in thread
From: Alex Sassmannshausen @ 2021-02-20 17:38 UTC (permalink / raw)
  To: Mike Gran; +Cc: guile-user, guile-devel

[-- Attachment #1: Type: text/plain, Size: 1408 bytes --]

Hi Mike,

Great to see you organise to continue this  hallowed tradition! :-)

I would love to participate by cheating a little and submitting a new
release of Guile Hall, which is overdue and should happen between 1 and
6 March.

Best wishes,

Alex

Mike Gran <spk121@yahoo.com> writes:

> Hello All-
>
> In celebration of the (slightly belated) 10-year anniversary of Guile
> v2.0, we're having another Guile Potluck!  The Guile Potluck is a
> randomly annual event to give people a chance to show off their Guile
> projects and skills.  Think of it as a game jam, but, not constrained
> to games. 
>
> To participate, on or before Mar 6, send an email to guile-user@gnu.org
> with instructions on how to find your entry, which could be anything
> you like.  For example,
>
>    - a script showing off some feature of Guile or your favorite Guile
>    library
>    - a blog post describing something interesting about Guile
>    - an updated release of a neglected library
>    - a mini-game
>    - a graphical or audio demoscene-type demo
>    
> There probably won't be any prizes.  But there will definitely be an e-
> mail and blog post about the entries.
>
> If you think you might attempt to participate, please reply to this e-
> mail so I can gauge the feasibility of some sort of participation swag.
>
> Regards,
> Mike Gran, on behalf of the Guile team


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 272 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-20 17:38   ` Alex Sassmannshausen
@ 2021-02-21 14:20     ` Dr. Arne Babenhauserheide
  0 siblings, 0 replies; 21+ messages in thread
From: Dr. Arne Babenhauserheide @ 2021-02-21 14:20 UTC (permalink / raw)
  To: alex.sassmannshausen; +Cc: guile-user, guile-devel

[-- Attachment #1: Type: text/plain, Size: 335 bytes --]


Alex Sassmannshausen <alex.sassmannshausen@gmail.com> writes:

> I would love to participate by cheating a little and submitting a new
> release of Guile Hall, which is overdue and should happen between 1 and
> 6 March.

\o/

thank you!

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-18 17:24 ` Guile Potluck 2021 Mike Gran
  2021-02-20 17:38   ` Alex Sassmannshausen
@ 2021-02-21 17:21   ` Daniel Tornabene
  2021-02-22  5:59   ` Aleix Conchillo Flaqué
                     ` (3 subsequent siblings)
  5 siblings, 0 replies; 21+ messages in thread
From: Daniel Tornabene @ 2021-02-21 17:21 UTC (permalink / raw)
  To: Mike Gran; +Cc: guile-user, guile-devel

[-- Attachment #1: Type: text/plain, Size: 1228 bytes --]

I have something I can drop for the potluck as well if this ends up
happening

On Thu, Feb 18, 2021 at 11:43 AM Mike Gran <spk121@yahoo.com> wrote:

> Hello All-
>
> In celebration of the (slightly belated) 10-year anniversary of Guile
> v2.0, we're having another Guile Potluck!  The Guile Potluck is a
> randomly annual event to give people a chance to show off their Guile
> projects and skills.  Think of it as a game jam, but, not constrained
> to games.
>
> To participate, on or before Mar 6, send an email to guile-user@gnu.org
> with instructions on how to find your entry, which could be anything
> you like.  For example,
>
>    - a script showing off some feature of Guile or your favorite Guile
>    library
>    - a blog post describing something interesting about Guile
>    - an updated release of a neglected library
>    - a mini-game
>    - a graphical or audio demoscene-type demo
>
> There probably won't be any prizes.  But there will definitely be an e-
> mail and blog post about the entries.
>
> If you think you might attempt to participate, please reply to this e-
> mail so I can gauge the feasibility of some sort of participation swag.
>
> Regards,
> Mike Gran, on behalf of the Guile team
>
>
>
>

[-- Attachment #2: Type: text/html, Size: 1686 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-18 17:24 ` Guile Potluck 2021 Mike Gran
  2021-02-20 17:38   ` Alex Sassmannshausen
  2021-02-21 17:21   ` Daniel Tornabene
@ 2021-02-22  5:59   ` Aleix Conchillo Flaqué
  2021-02-26 23:19     ` Aleix Conchillo Flaqué
  2021-03-01 11:34   ` pukkamustard
                     ` (2 subsequent siblings)
  5 siblings, 1 reply; 21+ messages in thread
From: Aleix Conchillo Flaqué @ 2021-02-22  5:59 UTC (permalink / raw)
  To: Mike Gran; +Cc: guile-user, guile-devel

Thank you Mike!

I should have waited to release guile-oauth with the OAuth2 support :-).
But if that counts...

Also planning to release guile-redis with redis 6.2 support when they make
a release (they are in RC3), but I might do it sooner.

Best,

Aleix

On Thu, Feb 18, 2021, 9:43 AM Mike Gran <spk121@yahoo.com> wrote:

> Hello All-
>
> In celebration of the (slightly belated) 10-year anniversary of Guile
> v2.0, we're having another Guile Potluck!  The Guile Potluck is a
> randomly annual event to give people a chance to show off their Guile
> projects and skills.  Think of it as a game jam, but, not constrained
> to games.
>
> To participate, on or before Mar 6, send an email to guile-user@gnu.org
> with instructions on how to find your entry, which could be anything
> you like.  For example,
>
>    - a script showing off some feature of Guile or your favorite Guile
>    library
>    - a blog post describing something interesting about Guile
>    - an updated release of a neglected library
>    - a mini-game
>    - a graphical or audio demoscene-type demo
>
> There probably won't be any prizes.  But there will definitely be an e-
> mail and blog post about the entries.
>
> If you think you might attempt to participate, please reply to this e-
> mail so I can gauge the feasibility of some sort of participation swag.
>
> Regards,
> Mike Gran, on behalf of the Guile team
>
>
>
>


^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
@ 2021-02-25 21:08 Zelphir Kaltstahl
  0 siblings, 0 replies; 21+ messages in thread
From: Zelphir Kaltstahl @ 2021-02-25 21:08 UTC (permalink / raw)
  To: guile-user

Hello!

I want to share some projects for the Potluck as well:

- guile-fslib

  - library for working with file system locations / things (currently only supports typical locations of GNU/Linux systems, not Windows with any drive letters.

  - https://notabug.org/ZelphirKaltstahl/guile-fslib

- awesome-guile:

  - list of great guile stuff

  - https://notabug.org/ZelphirKaltstahl/awesome-guile

- fantasy-vacation-planner:

  - With restrictions due to pandemics happening all over the planet, why not play a little traveling game written in GNU Guile?
    I did cheat a little though, using Python to validate the JSON configuration files, as I do not know about a GNU Guile library for validating JSON using a JSON schema.
    However, none of the actual game code is Python, it is all pure GNU Guile.

  - https://notabug.org/ZelphirKaltstahl/fantasy-vacation-planner

- guile-user-input-output:

  - Just a small library for querying the user on command line for stuff.
    Things like "ask for an integer" or "ask for a number satisfying the condition expressed in the given lambda".

  - https://notabug.org/ZelphirKaltstahl/guile-user-input-output

- guile-examples:
 
  - The place where I keep examples of how to do many things in GNU Guile.
 
  - https://notabug.org/ZelphirKaltstahl/guile-examples

- guile-risk-calculator:

  - A program for calculating probabilities in the board game risk.
    Makes use of simulating dice roll outcomes.
    The user can specify how many time a simulation is run.
    Helped me win a few games, I think, and was tricky to get working correctly.

  - https://notabug.org/ZelphirKaltstahl/guile-risk-calculator

- guile-ml:

  - A library containing a parallelized decision tree algorithm using futures.
    I envisioned the repository to grow in number of algorithms and offer a good basis for machine learning projects.
    I guess it is a good start, but still needs lots of utilities to be simple to use, when your data is not only numbers.

  - https://notabug.org/ZelphirKaltstahl/guile-ml

Regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-22  5:59   ` Aleix Conchillo Flaqué
@ 2021-02-26 23:19     ` Aleix Conchillo Flaqué
  2021-02-27  1:08       ` Mike Gran
  0 siblings, 1 reply; 21+ messages in thread
From: Aleix Conchillo Flaqué @ 2021-02-26 23:19 UTC (permalink / raw)
  To: Mike Gran; +Cc: guile-user, guile-devel

In case it was not clear:

guile-oauth 1.0.0 with OAuth2 support: https://github.com/aconchillo/guile-oauth
guile-redis 2.1.0 with Redis 6.2.0 commands:
https://github.com/aconchillo/guile-redis

Aleix

On Sun, Feb 21, 2021 at 9:59 PM Aleix Conchillo Flaqué
<aconchillo@gmail.com> wrote:
>
> Thank you Mike!
>
> I should have waited to release guile-oauth with the OAuth2 support :-). But if that counts...
>
> Also planning to release guile-redis with redis 6.2 support when they make a release (they are in RC3), but I might do it sooner.
>
> Best,
>
> Aleix
>
> On Thu, Feb 18, 2021, 9:43 AM Mike Gran <spk121@yahoo.com> wrote:
>>
>> Hello All-
>>
>> In celebration of the (slightly belated) 10-year anniversary of Guile
>> v2.0, we're having another Guile Potluck!  The Guile Potluck is a
>> randomly annual event to give people a chance to show off their Guile
>> projects and skills.  Think of it as a game jam, but, not constrained
>> to games.
>>
>> To participate, on or before Mar 6, send an email to guile-user@gnu.org
>> with instructions on how to find your entry, which could be anything
>> you like.  For example,
>>
>>    - a script showing off some feature of Guile or your favorite Guile
>>    library
>>    - a blog post describing something interesting about Guile
>>    - an updated release of a neglected library
>>    - a mini-game
>>    - a graphical or audio demoscene-type demo
>>
>> There probably won't be any prizes.  But there will definitely be an e-
>> mail and blog post about the entries.
>>
>> If you think you might attempt to participate, please reply to this e-
>> mail so I can gauge the feasibility of some sort of participation swag.
>>
>> Regards,
>> Mike Gran, on behalf of the Guile team
>>
>>
>>



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-26 23:19     ` Aleix Conchillo Flaqué
@ 2021-02-27  1:08       ` Mike Gran
  2021-02-27  9:39         ` Dr. Arne Babenhauserheide
  0 siblings, 1 reply; 21+ messages in thread
From: Mike Gran @ 2021-02-27  1:08 UTC (permalink / raw)
  To: Aleix Conchillo Flaqué; +Cc: guile-user, guile-devel

On Fri, 2021-02-26 at 15:19 -0800, Aleix Conchillo Flaqué wrote:
> In case it was not clear:
> 
> guile-oauth 1.0.0 with OAuth2 support: 
> https://github.com/aconchillo/guile-oauth
> guile-redis 2.1.0 with Redis 6.2.0 commands:
> https://github.com/aconchillo/guile-redis
> 
> Aleix
> 

Awesome. Thanks! Look forward to checking it out.

> On Sun, Feb 21, 2021 at 9:59 PM Aleix Conchillo Flaqué
> <aconchillo@gmail.com> wrote:
> > 
> > Thank you Mike!
> > 
> > I should have waited to release guile-oauth with the OAuth2 support
> > :-). But if that counts...
> > 
> > Also planning to release guile-redis with redis 6.2 support when
> > they make a release (they are in RC3), but I might do it sooner.
> > 
> > Best,
> > 
> > Aleix
> > 
> > On Thu, Feb 18, 2021, 9:43 AM Mike Gran <spk121@yahoo.com> wrote:
> > > 
> > > Hello All-
> > > 
> > > In celebration of the (slightly belated) 10-year anniversary of
> > > Guile
> > > v2.0, we're having another Guile Potluck!  The Guile Potluck is a
> > > randomly annual event to give people a chance to show off their
> > > Guile
> > > projects and skills.  Think of it as a game jam, but, not
> > > constrained
> > > to games.
> > > 
> > > To participate, on or before Mar 6, send an email to 
> > > guile-user@gnu.org
> > > with instructions on how to find your entry, which could be
> > > anything
> > > you like.  For example,
> > > 
> > >    - a script showing off some feature of Guile or your favorite
> > > Guile
> > >    library
> > >    - a blog post describing something interesting about Guile
> > >    - an updated release of a neglected library
> > >    - a mini-game
> > >    - a graphical or audio demoscene-type demo
> > > 
> > > There probably won't be any prizes.  But there will definitely be
> > > an e-
> > > mail and blog post about the entries.
> > > 
> > > If you think you might attempt to participate, please reply to
> > > this e-
> > > mail so I can gauge the feasibility of some sort of participation
> > > swag.
> > > 
> > > Regards,
> > > Mike Gran, on behalf of the Guile team
> > > 
> > > 
> > > 





^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-27  1:08       ` Mike Gran
@ 2021-02-27  9:39         ` Dr. Arne Babenhauserheide
  2021-02-27 10:14           ` Linus Björnstam
  0 siblings, 1 reply; 21+ messages in thread
From: Dr. Arne Babenhauserheide @ 2021-02-27  9:39 UTC (permalink / raw)
  To: Mike Gran; +Cc: Aleix Conchillo Flaqué, guile-user, guile-devel

[-- Attachment #1: Type: text/plain, Size: 2745 bytes --]

I’d like to contribute a basic implementation of the Web of Trust in
Freenet that provides spam-detection without centralized control:

https://hg.sr.ht/~arnebab/wispwot/browse/wispwot/wispwot.scm

Currently this takes 70s to calculate aggregated trust scores on 300k
trust edges.

I hope to get this faster and would love if someone would want to try
to golf its runtime down :-)

Best wishes,
Arne

Mike Gran <spk121@yahoo.com> writes:

> On Fri, 2021-02-26 at 15:19 -0800, Aleix Conchillo Flaqué wrote:
>> In case it was not clear:
>> 
>> guile-oauth 1.0.0 with OAuth2 support: 
>> https://github.com/aconchillo/guile-oauth
>> guile-redis 2.1.0 with Redis 6.2.0 commands:
>> https://github.com/aconchillo/guile-redis
>> 
>> Aleix
>> 
>
> Awesome. Thanks! Look forward to checking it out.
>
>> On Sun, Feb 21, 2021 at 9:59 PM Aleix Conchillo Flaqué
>> <aconchillo@gmail.com> wrote:
>> > 
>> > Thank you Mike!
>> > 
>> > I should have waited to release guile-oauth with the OAuth2 support
>> > :-). But if that counts...
>> > 
>> > Also planning to release guile-redis with redis 6.2 support when
>> > they make a release (they are in RC3), but I might do it sooner.
>> > 
>> > Best,
>> > 
>> > Aleix
>> > 
>> > On Thu, Feb 18, 2021, 9:43 AM Mike Gran <spk121@yahoo.com> wrote:
>> > > 
>> > > Hello All-
>> > > 
>> > > In celebration of the (slightly belated) 10-year anniversary of
>> > > Guile
>> > > v2.0, we're having another Guile Potluck!  The Guile Potluck is a
>> > > randomly annual event to give people a chance to show off their
>> > > Guile
>> > > projects and skills.  Think of it as a game jam, but, not
>> > > constrained
>> > > to games.
>> > > 
>> > > To participate, on or before Mar 6, send an email to 
>> > > guile-user@gnu.org
>> > > with instructions on how to find your entry, which could be
>> > > anything
>> > > you like.  For example,
>> > > 
>> > >    - a script showing off some feature of Guile or your favorite
>> > > Guile
>> > >    library
>> > >    - a blog post describing something interesting about Guile
>> > >    - an updated release of a neglected library
>> > >    - a mini-game
>> > >    - a graphical or audio demoscene-type demo
>> > > 
>> > > There probably won't be any prizes.  But there will definitely be
>> > > an e-
>> > > mail and blog post about the entries.
>> > > 
>> > > If you think you might attempt to participate, please reply to
>> > > this e-
>> > > mail so I can gauge the feasibility of some sort of participation
>> > > swag.
>> > > 
>> > > Regards,
>> > > Mike Gran, on behalf of the Guile team
>> > > 
>> > > 
>> > > 


-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-27  9:39         ` Dr. Arne Babenhauserheide
@ 2021-02-27 10:14           ` Linus Björnstam
  2021-02-27 16:42             ` Dr. Arne Babenhauserheide
  0 siblings, 1 reply; 21+ messages in thread
From: Linus Björnstam @ 2021-02-27 10:14 UTC (permalink / raw)
  To: Dr. Arne Babenhauserheide; +Cc: guile-user, guile-devel

I had a look and there is quite a lot you can do. Just out of curiosity: how much is GC time? You are generating a lot of intermediate data. Instead of vector-append! maybe use something like the new vectorlist srfi? If you show me a flame graph and give me some test data I can have a go!

-- 
  Linus Björnstam

On Sat, 27 Feb 2021, at 10:39, Dr. Arne Babenhauserheide wrote:
> I’d like to contribute a basic implementation of the Web of Trust in
> Freenet that provides spam-detection without centralized control:
> 
> https://hg.sr.ht/~arnebab/wispwot/browse/wispwot/wispwot.scm
> 
> Currently this takes 70s to calculate aggregated trust scores on 300k
> trust edges.
> 
> I hope to get this faster and would love if someone would want to try
> to golf its runtime down :-)
> 
> Best wishes,
> Arne
> 
> Mike Gran <spk121@yahoo.com> writes:
> 
> > On Fri, 2021-02-26 at 15:19 -0800, Aleix Conchillo Flaqué wrote:
> >> In case it was not clear:
> >> 
> >> guile-oauth 1.0.0 with OAuth2 support: 
> >> https://github.com/aconchillo/guile-oauth
> >> guile-redis 2.1.0 with Redis 6.2.0 commands:
> >> https://github.com/aconchillo/guile-redis
> >> 
> >> Aleix
> >> 
> >
> > Awesome. Thanks! Look forward to checking it out.
> >
> >> On Sun, Feb 21, 2021 at 9:59 PM Aleix Conchillo Flaqué
> >> <aconchillo@gmail.com> wrote:
> >> > 
> >> > Thank you Mike!
> >> > 
> >> > I should have waited to release guile-oauth with the OAuth2 support
> >> > :-). But if that counts...
> >> > 
> >> > Also planning to release guile-redis with redis 6.2 support when
> >> > they make a release (they are in RC3), but I might do it sooner.
> >> > 
> >> > Best,
> >> > 
> >> > Aleix
> >> > 
> >> > On Thu, Feb 18, 2021, 9:43 AM Mike Gran <spk121@yahoo.com> wrote:
> >> > > 
> >> > > Hello All-
> >> > > 
> >> > > In celebration of the (slightly belated) 10-year anniversary of
> >> > > Guile
> >> > > v2.0, we're having another Guile Potluck!  The Guile Potluck is a
> >> > > randomly annual event to give people a chance to show off their
> >> > > Guile
> >> > > projects and skills.  Think of it as a game jam, but, not
> >> > > constrained
> >> > > to games.
> >> > > 
> >> > > To participate, on or before Mar 6, send an email to 
> >> > > guile-user@gnu.org
> >> > > with instructions on how to find your entry, which could be
> >> > > anything
> >> > > you like.  For example,
> >> > > 
> >> > >    - a script showing off some feature of Guile or your favorite
> >> > > Guile
> >> > >    library
> >> > >    - a blog post describing something interesting about Guile
> >> > >    - an updated release of a neglected library
> >> > >    - a mini-game
> >> > >    - a graphical or audio demoscene-type demo
> >> > > 
> >> > > There probably won't be any prizes.  But there will definitely be
> >> > > an e-
> >> > > mail and blog post about the entries.
> >> > > 
> >> > > If you think you might attempt to participate, please reply to
> >> > > this e-
> >> > > mail so I can gauge the feasibility of some sort of participation
> >> > > swag.
> >> > > 
> >> > > Regards,
> >> > > Mike Gran, on behalf of the Guile team
> >> > > 
> >> > > 
> >> > > 
> 
> 
> -- 
> Unpolitisch sein
> heißt politisch sein
> ohne es zu merken
> 
> Attachments:
> * signature.asc



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-27 10:14           ` Linus Björnstam
@ 2021-02-27 16:42             ` Dr. Arne Babenhauserheide
  2021-02-27 16:43               ` Dr. Arne Babenhauserheide
  0 siblings, 1 reply; 21+ messages in thread
From: Dr. Arne Babenhauserheide @ 2021-02-27 16:42 UTC (permalink / raw)
  To: Linus Björnstam; +Cc: guile-user, guile-devel

[-- Attachment #1: Type: text/plain, Size: 1725 bytes --]

Hi Linus,

Linus Björnstam <linus.bjornstam@veryfast.biz> writes:
> I had a look and there is quite a lot you can do. Just out of
> curiosity: how much is GC time? You are generating a lot of
> intermediate data. Instead of vector-append! maybe use something like
> the new vectorlist srfi? If you show me a flame graph and give me some
> test data I can have a go!

Thank you!

I expect GC time to be roughly half during the trust calculation (much
more during import), because this fills up two cores.

I can’t easily give you a flame graph (except if you can tell me how to
generate it), but I now pushed a version in pure Scheme (instead of
wisp) that runs a huge test:

hg clone https://hg.sr.ht/~arnebab/wispwot
cd wispwot
./run-wispwot.scm --test

This imports 200k trust edges (which takes a lot of time) and then does
three full trust calculations (which each take around 70s).

The most important limitation for optimization is that the memory use
must stay reasonable with 64k IDs which each have 1000 trust edges. This
memory use is what the current code is optimized for (that’s why there
are all those u8vectors and u16vectors).

For the math: at 64 million trust edges, this should currently require
(in theory) less than 200MiB of memory for the actual trust structure (2
byte for the id, 1 byte for the trust).

Going up to 300MiB for time-optimization would still be viable, but it
shouldn’t go beyond that (there are algorithmic options to reduce the
amount of work done per update, so the runtime of the current code is
important but it is not a life-and-death situation).

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-27 16:42             ` Dr. Arne Babenhauserheide
@ 2021-02-27 16:43               ` Dr. Arne Babenhauserheide
  2021-03-02  7:08                 ` Dr. Arne Babenhauserheide
  0 siblings, 1 reply; 21+ messages in thread
From: Dr. Arne Babenhauserheide @ 2021-02-27 16:43 UTC (permalink / raw)
  To: Linus Björnstam; +Cc: guile-user, guile-devel

[-- Attachment #1: Type: text/plain, Size: 2053 bytes --]

PS: Still it is important to get this code fast, because it is the
    fallback for all situations where I cannot cheat and only calculate
    a subset of the trust, and it is required at startup.

Dr. Arne Babenhauserheide <arne_bab@web.de> writes:

> Hi Linus,
>
> Linus Björnstam <linus.bjornstam@veryfast.biz> writes:
>> I had a look and there is quite a lot you can do. Just out of
>> curiosity: how much is GC time? You are generating a lot of
>> intermediate data. Instead of vector-append! maybe use something like
>> the new vectorlist srfi? If you show me a flame graph and give me some
>> test data I can have a go!
>
> Thank you!
>
> I expect GC time to be roughly half during the trust calculation (much
> more during import), because this fills up two cores.
>
> I can’t easily give you a flame graph (except if you can tell me how to
> generate it), but I now pushed a version in pure Scheme (instead of
> wisp) that runs a huge test:
>
> hg clone https://hg.sr.ht/~arnebab/wispwot
> cd wispwot
> ./run-wispwot.scm --test
>
> This imports 200k trust edges (which takes a lot of time) and then does
> three full trust calculations (which each take around 70s).
>
> The most important limitation for optimization is that the memory use
> must stay reasonable with 64k IDs which each have 1000 trust edges. This
> memory use is what the current code is optimized for (that’s why there
> are all those u8vectors and u16vectors).
>
> For the math: at 64 million trust edges, this should currently require
> (in theory) less than 200MiB of memory for the actual trust structure (2
> byte for the id, 1 byte for the trust).
>
> Going up to 300MiB for time-optimization would still be viable, but it
> shouldn’t go beyond that (there are algorithmic options to reduce the
> amount of work done per update, so the runtime of the current code is
> important but it is not a life-and-death situation).
>
> Best wishes,
> Arne


-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-18 17:24 ` Guile Potluck 2021 Mike Gran
                     ` (2 preceding siblings ...)
  2021-02-22  5:59   ` Aleix Conchillo Flaqué
@ 2021-03-01 11:34   ` pukkamustard
  2021-03-01 21:14   ` Linus Björnstam
  2021-03-07  5:28   ` Extensible effects in Guile (Guile Potluck 2021) Walter Lewis
  5 siblings, 0 replies; 21+ messages in thread
From: pukkamustard @ 2021-03-01 11:34 UTC (permalink / raw)
  To: Mike Gran; +Cc: guile-user, guile-devel


Hi,

I'd like to contribute an initial version of guile-cbor (v0.1.0) 
to the
potluck:

https://inqlab.net/git/guile-cbor.git/

The Concise Binary Object Representation (CBOR), as specified by 
RFC
8949, is a binary data serialization format. CBOR is similar to 
JSON but
serializes to binary which is smaller and faster to generate and 
parse.

guile-cbor provides a Guile implementation of CBOR.

Happy hacking!
pukkamustard

Mike Gran <spk121@yahoo.com> writes:

> Hello All-
>
> In celebration of the (slightly belated) 10-year anniversary of 
> Guile
> v2.0, we're having another Guile Potluck!  The Guile Potluck is 
> a
> randomly annual event to give people a chance to show off their 
> Guile
> projects and skills.  Think of it as a game jam, but, not 
> constrained
> to games.
>
> To participate, on or before Mar 6, send an email to 
> guile-user@gnu.org
> with instructions on how to find your entry, which could be 
> anything
> you like.  For example,
>
>    - a script showing off some feature of Guile or your favorite 
>    Guile
>    library
>    - a blog post describing something interesting about Guile
>    - an updated release of a neglected library
>    - a mini-game
>    - a graphical or audio demoscene-type demo
>
> There probably won't be any prizes.  But there will definitely 
> be an e-
> mail and blog post about the entries.
>
> If you think you might attempt to participate, please reply to 
> this e-
> mail so I can gauge the feasibility of some sort of 
> participation swag.
>
> Regards,
> Mike Gran, on behalf of the Guile team




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-18 17:24 ` Guile Potluck 2021 Mike Gran
                     ` (3 preceding siblings ...)
  2021-03-01 11:34   ` pukkamustard
@ 2021-03-01 21:14   ` Linus Björnstam
  2021-03-01 21:37     ` Aleix Conchillo Flaqué
  2021-03-07  5:28   ` Extensible effects in Guile (Guile Potluck 2021) Walter Lewis
  5 siblings, 1 reply; 21+ messages in thread
From: Linus Björnstam @ 2021-03-01 21:14 UTC (permalink / raw)
  To: Mike Gran, guile-user, guile-devel

Hi Guilers!

I found some code I wrote while drunk. I had completely forgotten about it, but found it a fun little exercise. I would argue that falls under the "neglected library" category.

Anyway: It implements python-style generators, with yield and, more importantly, yield-from:

https://git.sr.ht/~bjoli/awesome-coroutine-generators

I tried to extend it while sober. It didn't work, so be warned  I guess. The code is somewhat a mess, but short enough to be understandable.
Best regards
  Linus Björnstam

On Thu, 18 Feb 2021, at 18:24, Mike Gran wrote:
> Hello All-
> 
> In celebration of the (slightly belated) 10-year anniversary of Guile
> v2.0, we're having another Guile Potluck!  The Guile Potluck is a
> randomly annual event to give people a chance to show off their Guile
> projects and skills.  Think of it as a game jam, but, not constrained
> to games. 
> 
> To participate, on or before Mar 6, send an email to guile-user@gnu.org
> with instructions on how to find your entry, which could be anything
> you like.  For example,
> 
>    - a script showing off some feature of Guile or your favorite Guile
>    library
>    - a blog post describing something interesting about Guile
>    - an updated release of a neglected library
>    - a mini-game
>    - a graphical or audio demoscene-type demo
>    
> There probably won't be any prizes.  But there will definitely be an e-
> mail and blog post about the entries.
> 
> If you think you might attempt to participate, please reply to this e-
> mail so I can gauge the feasibility of some sort of participation swag.
> 
> Regards,
> Mike Gran, on behalf of the Guile team
> 
> 
> 
>



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-03-01 21:14   ` Linus Björnstam
@ 2021-03-01 21:37     ` Aleix Conchillo Flaqué
  2021-03-02  0:43       ` Dr. Arne Babenhauserheide
  0 siblings, 1 reply; 21+ messages in thread
From: Aleix Conchillo Flaqué @ 2021-03-01 21:37 UTC (permalink / raw)
  To: Linus Björnstam; +Cc: guile-user, guile-devel

On Mon, Mar 1, 2021 at 1:15 PM Linus Björnstam
<linus.internet@fastmail.se> wrote:
>
> Hi Guilers!
>
> I found some code I wrote while drunk. I had completely forgotten about it, but found it a fun little exercise. I would argue that falls under the "neglected library" category.
>
> Anyway: It implements python-style generators, with yield and, more importantly, yield-from:
>
> https://git.sr.ht/~bjoli/awesome-coroutine-generators
>
> I tried to extend it while sober. It didn't work, so be warned  I guess.

You know what to do then :-D.

> The code is somewhat a mess, but short enough to be understandable.
>

Aleix



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-03-01 21:37     ` Aleix Conchillo Flaqué
@ 2021-03-02  0:43       ` Dr. Arne Babenhauserheide
  0 siblings, 0 replies; 21+ messages in thread
From: Dr. Arne Babenhauserheide @ 2021-03-02  0:43 UTC (permalink / raw)
  To: Aleix Conchillo Flaqué; +Cc: guile-user, Linus Björnstam, guile-devel

[-- Attachment #1: Type: text/plain, Size: 246 bytes --]

Hi Guilers,

There’s one more thing for the Potluck:

I started doing some code-catas in Guile wisp:
https://www.draketo.de/software/wisp-code-katas

Liebe Grüße,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Guile Potluck 2021
  2021-02-27 16:43               ` Dr. Arne Babenhauserheide
@ 2021-03-02  7:08                 ` Dr. Arne Babenhauserheide
  0 siblings, 0 replies; 21+ messages in thread
From: Dr. Arne Babenhauserheide @ 2021-03-02  7:08 UTC (permalink / raw)
  To: Linus Björnstam; +Cc: guile-user, guile-devel

[-- Attachment #1: Type: text/plain, Size: 12793 bytes --]

Hi Linus,

I now created four profiles, one for the CSV import and three for the
trust calculation. They are generated with:

guile -L .
> (import (wispwot wispwot))
> ,profile (define changed-and-state (import-trust-csv "trust-anonymized-2020-11-01-under-attack-sorted.csv"))
> ,profile ((@@(wispwot wispwot)import-trust-value) (cdr changed-and-state) "OBSERVER" "OBSERVER" "15363" 100)
> ,profile ((@@(wispwot wispwot)import-trust-value) (cdr changed-and-state) "OBSERVER" "OBSERVER" "242" 100)
> ,profile ((@@(wispwot wispwot)import-trust-value) (cdr changed-and-state) "OBSERVER" "OBSERVER" "853" 100)

%     cumulative   self             
time   seconds     seconds  procedure
 21.92    286.68    286.68  wispwot/wispwot.scm:456:18
 21.14    276.46    276.46  wispwot/wispwot.scm:449:18
 20.86    292.23    272.82  vector->list
 13.26 2124682.83    173.39  srfi/srfi-1.scm:600:7:map2
  6.46     84.52     84.52  %after-gc-thunk
  4.04     55.03     52.83  wispwot/wispwot.scm:457:17
  3.96     54.41     51.81  wispwot/wispwot.scm:450:17
  2.27     29.69     29.63  make-srfi-4-vector
  2.25     29.43     29.43  length+
  1.01     16.43     13.24  wispwot/wispwot.scm:276:0:calculate-scores
  0.93     12.19     12.19  list-tail
  0.53      6.92      6.92  display
  0.23      3.10      3.01  write
  0.16   1295.52      2.12  wispwot/wispwot.scm:375:0:import-trust-value
  0.07      0.95      0.95  object->string
  0.06   1309.39      0.80  srfi/srfi-1.scm:452:2:fold
  0.06      0.78      0.78  srfi/srfi-4.scm:85:20:u16vector-set!
  0.06      0.98      0.75  ice-9/pretty-print.scm:273:0:pretty-print
  0.05     11.09      0.69  wispwot/wispwot.scm:513:8
  0.05      4.98      0.66  ice-9/pretty-print.scm:258:2:rev-string-append
  0.05      0.66      0.66  srfi/srfi-4.scm:85:20:s8vector-ref
  0.05      0.63      0.63  vector-move-left!
  0.03     16.75      0.45  ice-9/pretty-print.scm:37:0:generic-write
  0.03      4.38      0.45  wispwot/wispwot.scm:349:0:vector-append!
  0.03      0.45      0.45  srfi/srfi-4.scm:85:20:s8vector-set!
  0.03      0.39      0.39  srfi/srfi-4.scm:85:20:u16vector-ref
  0.03     29.90      0.33  srfi/srfi-1.scm:580:2:map
  0.03      9.09      0.33  ice-9/pretty-print.scm:62:4:loop
  0.03      0.66      0.33  make-vector
  0.02     32.59      0.27  wispwot/wispwot.scm:194:0:calculate-ranks
  0.02      0.92      0.27  wispwot/wispwot.scm:415:2:find-index-record-if-needed
  0.02      0.27      0.27  procedure?
  0.02      0.24      0.24  ice-9/boot-9.scm:1345:3
  0.02      0.24      0.24  make-string
  0.01      0.18      0.18  ice-9/pretty-print.scm:100:12
  0.01      0.18      0.18  min
  0.01      6.14      0.15  ice-9/pretty-print.scm:293:17
  0.01      0.21      0.15  ice-9/pretty-print.scm:72:22
  0.01      0.15      0.15  srfi/srfi-4.scm:85:20:u16vector->list
  0.01      0.15      0.15  remove
  0.01      0.12      0.12  length
  0.01      0.12      0.12  srfi/srfi-4.scm:76:2:make-u8vector
  0.01      0.12      0.12  random
  0.01      0.12      0.12  srfi/srfi-4.scm:85:20:s8vector-length
  0.01      0.15      0.09  srfi/srfi-4.scm:85:20:u16vector-length
  0.01      0.12      0.09  %read-line
  0.01      0.09      0.09  srfi/srfi-4.scm:85:20:list->u8vector
  0.01      0.09      0.09  srfi/srfi-4.scm:85:20:u8vector-set!
  0.01      0.09      0.09  hash-ref
  0.01      0.09      0.09  ice-9/pretty-print.scm:61:2:wr
  0.01      0.09      0.09  cdr
  0.00      1.40      0.06  ice-9/pretty-print.scm:58:2:out
  0.00      0.39      0.06  wispwot/wispwot.scm:504:4
  0.00      0.06      0.06  integer?
  0.00      0.06      0.06  ice-9/boot-9.scm:806:0:and=>
  0.00      8.08      0.03  ice-9/pretty-print.scm:95:4:pr
  0.00      0.24      0.03  srfi/srfi-4.scm:85:20:list->u8vector
  0.00      0.09      0.03  string-split
  0.00      0.03      0.03  srfi/srfi-4.scm:85:20:make-u16vector
  0.00      0.03      0.03  car
  0.00      0.03      0.03  srfi/srfi-4.scm:85:20:make-s8vector
  0.00      0.03      0.03  eof-object?
  0.00      0.03      0.03  srfi/srfi-4.scm:85:20:s8vector->list
  0.00      0.03      0.03  ice-9/pretty-print.scm:256:0:reverse-string-append
  0.00      0.03      0.03  list?
  0.00      0.03      0.03  ice-9/boot-9.scm:888:0:iota
  0.00      0.03      0.03  reverse!
  0.00      0.03      0.03  append
  0.00      0.03      0.03  list-head
  0.00      0.03      0.03  srfi/srfi-1.scm:584:5:map1
  0.00   1307.69      0.00  ice-9/ports.scm:428:0:call-with-input-file
  0.00   1307.69      0.00  <current input>:2:9
  0.00   1307.69      0.00  ice-9/ports.scm:492:3
  0.00    563.14      0.00  remove
  0.00     84.52      0.00  anon #x7a24b0
  0.00     54.83      0.00  gc
  0.00      3.10      0.00  object->string
  0.00      0.12      0.00  ice-9/rdelim.scm:193:0:read-line
  0.00      0.03      0.00  inexact->exact
---
Sample count: 43863
Total time: 1307.687229447 seconds (726.756595442 seconds in GC)


%     cumulative   self             
time   seconds     seconds  procedure
 57.35     21.72     21.56  wispwot/wispwot.scm:194:0:calculate-ranks
 11.35      4.27      4.27  wispwot/wispwot.scm:324:10:discard
  4.91      3.43      1.85  wispwot/wispwot.scm:313:10:add-to-score
  4.17      1.59      1.57  truncate/
  2.96      1.11      1.11  ice-9/boot-9.scm:888:0:iota
  2.57     15.87      0.96  wispwot/wispwot.scm:276:0:calculate-scores
  2.43    226.03      0.92  srfi/srfi-1.scm:600:7:map2
  2.13      0.90      0.80  srfi/srfi-4.scm:85:20:u16vector->list
  1.74      0.65      0.65  srfi/srfi-4.scm:85:20:u8vector-set!
  1.65      0.67      0.62  srfi/srfi-4.scm:85:20:s8vector->list
  0.96      0.85      0.36  srfi/srfi-1.scm:580:2:map
  0.91      0.34      0.34  make-srfi-4-vector
  0.87      0.33      0.33  append
  0.74      1.34      0.28  srfi/srfi-4.scm:85:20:list->u8vector
  0.70      0.31      0.26  srfi/srfi-4.scm:85:20:s8vector-length
  0.70      0.26      0.26  remove
  0.61      0.23      0.23  length+
  0.48      0.18      0.18  integer?
  0.39     21.17      0.15  srfi/srfi-1.scm:584:5:map1
  0.39      0.15      0.15  reverse!
  0.35      0.13      0.13  car
  0.30      0.11      0.11  wispwot/wispwot.scm:338:19
  0.26      0.10      0.10  procedure?
  0.26      0.10      0.10  min
  0.17      0.07      0.07  %after-gc-thunk
  0.17      0.07      0.07  list?
  0.17      0.07      0.07  length
  0.13      0.05      0.05  srfi/srfi-4.scm:85:20:list->u8vector
  0.09      0.03      0.03  delete-duplicates
  0.09      0.03      0.03  wispwot/wispwot.scm:236:10:discard
  0.00     37.59      0.00  wispwot/wispwot.scm:375:0:import-trust-value
  0.00      4.30      0.00  remove
  0.00      0.07      0.00  anon #x7a24b0
---
Sample count: 2300
Total time: 37.593051451 seconds (0.998694416 seconds in GC)

%     cumulative   self             
time   seconds     seconds  procedure
 69.60     47.03     46.98  wispwot/wispwot.scm:194:0:calculate-ranks
  9.26      6.25      6.25  wispwot/wispwot.scm:324:10:discard
  3.14      2.12      2.12  truncate/
  3.11      4.22      2.10  wispwot/wispwot.scm:313:10:add-to-score
  2.36    241.68      1.60  srfi/srfi-1.scm:600:7:map2
  1.91      1.38      1.29  ice-9/boot-9.scm:888:0:iota
  1.62     20.47      1.09  wispwot/wispwot.scm:276:0:calculate-scores
  1.30      0.93      0.88  srfi/srfi-4.scm:85:20:u16vector->list
  1.18      0.81      0.80  srfi/srfi-4.scm:85:20:s8vector->list
  1.06      0.72      0.72  srfi/srfi-4.scm:85:20:u8vector-set!
  0.63      0.54      0.42  srfi/srfi-4.scm:85:20:s8vector-length
  0.58      0.39      0.39  remove
  0.53      0.36      0.36  append
  0.46      0.31      0.31  make-srfi-4-vector
  0.46      0.31      0.31  length+
  0.41      0.28      0.28  integer?
  0.39      0.98      0.26  srfi/srfi-1.scm:580:2:map
  0.36      0.24      0.24  procedure?
  0.34      1.35      0.23  srfi/srfi-4.scm:85:20:list->u8vector
  0.31      0.21      0.21  reverse!
  0.22     16.14      0.15  srfi/srfi-1.scm:584:5:map1
  0.19      0.13      0.13  car
  0.17      0.11      0.11  wispwot/wispwot.scm:338:19
  0.14      0.10      0.10  min
  0.14      0.10      0.10  length
  0.10      0.07      0.07  list?
  0.02      0.02      0.02  srfi/srfi-4.scm:85:20:list->u8vector
  0.00     67.50      0.00  wispwot/wispwot.scm:375:0:import-trust-value
  0.00      6.25      0.00  remove
---
Sample count: 4145
Total time: 67.498343877 seconds (1.476365885 seconds in GC)


%     cumulative   self             
time   seconds     seconds  procedure
 57.04     35.87     35.81  wispwot/wispwot.scm:194:0:calculate-ranks
 12.43      7.81      7.81  wispwot/wispwot.scm:324:10:discard
  4.84      5.95      3.04  wispwot/wispwot.scm:313:10:add-to-score
  4.61      2.91      2.89  truncate/
  3.30    367.72      2.07  srfi/srfi-1.scm:600:7:map2
  2.96     26.90      1.86  wispwot/wispwot.scm:276:0:calculate-scores
  2.72      1.73      1.71  ice-9/boot-9.scm:888:0:iota
  1.81      1.13      1.13  srfi/srfi-4.scm:85:20:u8vector-set!
  1.60      1.04      1.00  srfi/srfi-4.scm:85:20:u16vector->list
  1.13      0.74      0.71  srfi/srfi-4.scm:85:20:s8vector->list
  0.99      0.62      0.62  make-srfi-4-vector
  0.84      2.35      0.53  srfi/srfi-4.scm:85:20:list->u8vector
  0.73      1.46      0.46  srfi/srfi-1.scm:580:2:map
  0.68      0.43      0.43  length+
  0.68      0.43      0.43  remove
  0.60      0.51      0.38  srfi/srfi-4.scm:85:20:s8vector-length
  0.58      0.36      0.36  append
  0.50      0.31      0.31  procedure?
  0.42      0.26      0.26  reverse!
  0.34      0.21      0.21  integer?
  0.29      0.18      0.18  list?
  0.24      0.15      0.15  wispwot/wispwot.scm:338:19
  0.18      0.12      0.12  car
  0.13      0.08      0.08  min
  0.10     22.56      0.07  srfi/srfi-1.scm:584:5:map1
  0.10      0.07      0.07  srfi/srfi-4.scm:85:20:list->u8vector
  0.10      0.07      0.07  length
  0.03      0.02      0.02  %after-gc-thunk
  0.03      0.02      0.02  wispwot/wispwot.scm:236:10:discard
  0.00     62.77      0.00  wispwot/wispwot.scm:375:0:import-trust-value
  0.00      7.82      0.00  remove
  0.00      0.02      0.00  anon #x7a24b0
---
Sample count: 3820
Total time: 62.773351245 seconds (2.013414284 seconds in GC)


(I see here that the GC time during actual trust calculation is much
lower than I had expected)


I hope they help! (you’ll have to pull again — with `hg pull -u` — to
get import-trust-csv exported)

Best wishes,
Arne


Dr. Arne Babenhauserheide <arne_bab@web.de> writes:

> PS: Still it is important to get this code fast, because it is the
>     fallback for all situations where I cannot cheat and only calculate
>     a subset of the trust, and it is required at startup.
>
> Dr. Arne Babenhauserheide <arne_bab@web.de> writes:
>
>> Hi Linus,
>>
>> Linus Björnstam <linus.bjornstam@veryfast.biz> writes:
>>> I had a look and there is quite a lot you can do. Just out of
>>> curiosity: how much is GC time? You are generating a lot of
>>> intermediate data. Instead of vector-append! maybe use something like
>>> the new vectorlist srfi? If you show me a flame graph and give me some
>>> test data I can have a go!
>>
>> Thank you!
>>
>> I expect GC time to be roughly half during the trust calculation (much
>> more during import), because this fills up two cores.
>>
>> I can’t easily give you a flame graph (except if you can tell me how to
>> generate it), but I now pushed a version in pure Scheme (instead of
>> wisp) that runs a huge test:
>>
>> hg clone https://hg.sr.ht/~arnebab/wispwot
>> cd wispwot
>> ./run-wispwot.scm --test
>>
>> This imports 200k trust edges (which takes a lot of time) and then does
>> three full trust calculations (which each take around 70s).
>>
>> The most important limitation for optimization is that the memory use
>> must stay reasonable with 64k IDs which each have 1000 trust edges. This
>> memory use is what the current code is optimized for (that’s why there
>> are all those u8vectors and u16vectors).
>>
>> For the math: at 64 million trust edges, this should currently require
>> (in theory) less than 200MiB of memory for the actual trust structure (2
>> byte for the id, 1 byte for the trust).
>>
>> Going up to 300MiB for time-optimization would still be viable, but it
>> shouldn’t go beyond that (there are algorithmic options to reduce the
>> amount of work done per update, so the runtime of the current code is
>> important but it is not a life-and-death situation).
>>
>> Best wishes,
>> Arne


-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Extensible effects in Guile (Guile Potluck 2021)
  2021-02-18 17:24 ` Guile Potluck 2021 Mike Gran
                     ` (4 preceding siblings ...)
  2021-03-01 21:14   ` Linus Björnstam
@ 2021-03-07  5:28   ` Walter Lewis
  2021-03-08 15:42     ` Extensible effects in Guile Walter Lewis
  5 siblings, 1 reply; 21+ messages in thread
From: Walter Lewis @ 2021-03-07  5:28 UTC (permalink / raw)
  To: guile-user

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]


Greetings, Guile,

Sincere apologies for the late entry.  It's like my school days 
all over again...

Attached is a literate org-mode file which implements an 
"extensible effects" system in Guile.  Extensible effects was 
first implemented in Haskell by Oleg Kiselyov and others[1], and 
as far as I know has not been attempted in Scheme.  It defines a 
metalanguage for writing programs which describe side-effects, and 
handlers which interpret them.

I'd hoped to publish it as a blog post, but technical issues got 
in the way and I didn't want to delay any longer.  I will update 
the list when the post is live; for now this file lives at 
https://git.sr.ht/~wklew/genshou

I hope some folks find it interesting!

Walter

[1] okmij.org/ftp/Haskell/extensible/


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: genshou --]
[-- Type: text/x-org, Size: 16244 bytes --]

#+title: GENSHOU: Extensible Effects in Scheme

* Introduction
:PROPERTIES:
:CUSTOM_ID: intro
:END:

This is a literate code file written for the Guile 2021 Potluck.  It describes a system for embedding effectful programs in Scheme, based on a post by Oleg Kiselyov called [[http://okmij.org/ftp/Computation/having-effect.html][Having an Effect]].  Most of the ideas described below are Oleg's; the few original ones arose in the process of translating the original Haskell code to Guile Scheme.  Like any translation, it presented many interesting problems, and the goal was to design a system which played to strengths of Scheme rather than clumsily reproduce Haskell idioms.  The idea of a "reified abort" described in [[#part2][part 2]], for example, was helpful in relating concepts to the Guile ecosystem.

All accompanying code is considered an experiment and is released into the public domain.

To install matching indentation rules in Emacs:

#+begin_src emacs-lisp
(put '%handle 'scheme-indent-function 1)
(put 'handle 'scheme-indent-function 1)
(put 'extend 'scheme-indent-function 1)
(put 'state 'scheme-indent-function 1)
(put 'maybe 'scheme-indent-function 1)
(put 'function 'scheme-indent-function 1)
#+end_src

* Part 1: Simulating Effects
:PROPERTIES:
:CUSTOM_ID: part1
:END:

Consider the following library, which uses the delimited control operators =abort-to-prompt= and =call-with-prompt= to simulate a global state:

#+begin_src scheme
(define state-prompt-tag (make-prompt-tag 'state))

(define (get-state)
  (abort-to-prompt state-prompt-tag 'get))

(define (put-state st)
  (abort-to-prompt state-prompt-tag 'put st))

(define (handle-state st thunk)
  (call-with-prompt state-prompt-tag
    thunk
    (lambda (cont . sig)
      (match sig
        (('get)
         (handle-state st (lambda () (cont st))))
        (('put st)
         (handle-state st (lambda () (cont st))))))))
#+end_src

We can use this library to perform stateful computations without incurring the penalties of actual mutation:

#+begin_src scheme :results replace
(define (fresh)
  (let ((n (get-state)))
    (put-state (1+ n))
    n))

(define (label x)
  (cons x (fresh)))

(handle-state 0 (lambda () (map label '(a b c d e))))
#+end_src

#+RESULTS:
#+begin_example
((a . 0) (b . 1) (c . 2) (d . 3) (e . 4))
#+end_example

The interface above is structured around two /effects/, which submit requests to read or write the state, and a /handler/, which receives these requests and delegates access to it.  It relies on a well-known transformation from continuation-passing style, in which the state is threaded through the program as an additional argument, to direct style using delimited control operators.

Of course, the same effect could be achieved using Guile's support for fluids and dynamic state.  Below, we describe a small system for extending Scheme with arbitrary effects without explicit use of continuations or other language-specific features.  We then see how far we can push it, and achieve some results which may be surprising.

* Part 2: Reifying Control
:PROPERTIES:
:CUSTOM_ID: part2
:END:

In the state example described in [[#part1][part 1]], invoking an effectful function like =get-state= or =put-state= immediately aborts the current continuation.  When this happens, control is transferred to a handler like =handle-state=, and the effect is "run" by plugging the continuation with a value from the context.  As we saw, this process of "trampolining" between a running computation and its context is driven by the control operators =abort-to-prompt= and =call-with-prompt=.  In [[#part2][part 2]], we describe an alternate way of simulating effects which /reifies/ this control pattern onto a data structure.

#+begin_src scheme
(use-modules (ice-9 match)
             (srfi srfi-9))
#+end_src

First, we define a record type which encapsulates the data communicated to the handler of =call-with-prompt=.  Manually destructuring it violates the protocol we establish next, so consider these functions private.

#+begin_src scheme
(define-record-type <request>
  (%make-request cont sig)
  request?
  (sig %request-signature)
  (cont %request-continuation))
#+end_src

Because we are /embedding/ a new domain of values (effects) into Scheme, we need a way to distinguish these from "pure" values. Pure values can freely move between the two domains, while "impure" requests can only be modified by the proper handlers.

#+begin_src scheme
(define (pure x)
  (%make-request #f x))

(define (lower req)
  (if (%request-continuation req)
      (error "Attempt to lower an impure request")
      (%request-signature req)))
#+end_src

Next, we establish a protocol for effects and handlers to communicate with each other.

=Request=, analogous to =abort-to-prompt=, sends a new effect request along a tagged channel.

#+begin_src scheme
(define (request tag . sig)
  (%make-request pure (cons tag sig)))
#+end_src

=Handle=, analogous to =call-with-prompt=, receives effect requests on a channel and attempts to handle them.  Unlike =call-with-prompt=, handling a request object requires routing it through one of two procedures, based on its purity.  The first procedure "lifts" Scheme values into the domain of the effect being handled.  This is simply =pure= in the standard case; sometimes, however, a handler packages up its results in a non-standard way, as we'll see later.  The second procedure is the "handler" proper, analogous to one in =call-with-prompt=: it receives the continuation and a signature describing the effect.

#+begin_src scheme
(define (handle tag req lift handler)
  (let loop ((req req))
    (let ((cont (%request-continuation req))
          (sig (%request-signature req)))
      (if (not cont)
          (lift sig)
          (if (eq? tag (car sig))
              (apply handler cont (cdr sig))
              (%make-request (compose loop cont) sig))))))
#+end_src

This establishes the interface for introducing and eliminating effect requests.  Lastly, we define an interface extending existing requests.  A request is extended by composing it with a continuation; this continuation "waits" for the request to be handled, then constructs a second request from the value returned by the handler.

#+begin_src scheme
(define (extend req cont)
  (let loop ((req req))
    (let ((cont0 (%request-continuation req))
          (sig (%request-signature req)))
      (if (not cont0)
          (cont sig)
          (%make-request (compose loop cont0) sig)))))
#+end_src

Those familiar with monads will recognize =extend= as monadic =bind=.  One way to describe what is meant by extensible effects is to say that we only need to implement one monad.  The request serves as a kind of "meta effect"; all other effects are defined in terms of =pure= and =extend=.

We also specify a dedicated request to serve as the unit of sequential composition[fn:1].  It will be used by effects which do not return a value.

#+begin_src scheme
(define (unit) (pure unit))
#+end_src

This completes the infrastructure necessary for writing effects and handlers.  [[#part3][Part 3]] constructs, piece by piece, an embedded language of effects.  By the end, we hope that the reader will be left with a sense for how it all works and why it may be useful.

* Part 3: Effectful Metalanguage
:PROPERTIES:
:CUSTOM_ID: part3
:END:

** Boolean logic
:PROPERTIES:
:CUSTOM_ID: bools
:END:

To demonstrate how effects communicate with handlers via requests, we start with a toy language which models boolean logic.  This language will then be used to test our language as we go.

Although boolean logic isn't typically considered an effect, treating it as one is a useful exercise.  Besides, we might find other benefits to relaxing our view on what constitutes an effect.

#+begin_src scheme
(define bool-tag '(bool))
#+end_src

Our boolean language has four primitive operations: the values true and false, logical negation and logical conjunction.  =True= and =false= simply send requests for operations of the same name:

#+begin_src scheme
(define (true) (request bool-tag 'true))
(define (false) (request bool-tag 'false))
#+end_src

The negation function =neg= performs a similar request, except that it is parameterized by an input request; the disjunction function =disj= is parameterized by two input requests:

#+begin_src scheme
(define (neg bool)
  (extend bool
    (lambda (p)
      (request bool-tag 'neg p))))

(define (conj bool1 bool2)
  (extend bool1
    (lambda (p)
      (extend bool2
        (lambda (q)
          (request bool-tag 'conj p q))))))
#+end_src

Using =extend=, we bind on the value returned by a request's handler /before that handler is run/, then continue the computation /as if/ it were already handled.

Taking a look at how Guile displays requests such as the one returned by =true= can be instructive:

#+begin_src scheme :results replace
(true)
#+end_src

#+RESULTS:
#+begin_example
#<<request> sig: ((bool) true) cont: #<procedure pure (x)>>
#+end_example

Unlike the effects in [[#part1][part 1]], ours can be passed around, examined and manipulated outside the presence of a handler.

The handler for booleans matches on effect signatures sent along the =bool-tag= channel.  The operation requested is then performed in Scheme, and its return value passed to the continuation.  Like =handle-state= from [[#part1][part 1]], =bool-logic= is recursive, handling further requests raised by the continuation.

#+begin_src scheme
(define (bool-logic req)
  (handle bool-tag
    req
    pure
    (lambda (cont . sig)
      (bool-logic
       (cont
        (match sig
          (('true) #t)
          (('false) #f)
          (('neg p) (not p))
          (('conj p q) (and p q))))))))
#+end_src

Having defined the primitives effects and handler of our boolean language, we can use them to derive new effects for logical disjunction and implication:

#+begin_src scheme
(define (impl bool1 bool2)
  (neg (conj bool1 (neg bool2))))

(define (disj bool1 bool2)
  (impl (neg bool1) bool2))
#+end_src

And run some tests:

#+begin_src scheme
(lower (bool-logic (neg (true))))
#+end_src

#+RESULTS:
#+begin_example
#f
#+end_example

#+begin_src scheme :results replace
(lower (bool-logic
        (disj (false)
              (neg (false)))))
#+end_src

#+RESULTS:
#+begin_example
#t
#+end_example

At any point in the computation, we can inject Scheme booleans using =pure=:

#+begin_src scheme :results replace
(lower (bool-logic
        (impl (neg (pure #f))
              (false))))
#+end_src

#+RESULTS:
#+begin_example
#f
#+end_example

** Errors
:PROPERTIES:
:CUSTOM_ID: errors
:END:

The first "proper" effect we'll implement simulates raising an error.  A program which encounters an error has to abort the current continuation and proceed from an earlier point.

#+begin_src scheme
(define error-tag '(error))

(define (fail)
  (request error-tag))
#+end_src

The =maybe= handler marks the point at which a computation should return on encountering an exception and provides an alternative, encoded lazily as a thunk:

#+begin_src scheme
(define (maybe thunk req)
  (handle error-tag
    req
    pure
    (lambda (_) (thunk))))
#+end_src

The derived effect =satisfy= returns the value returned by a request if it satisfies a predicate and fails otherwise:

#+begin_src scheme
(define (satisfy test req)
  (extend req
    (lambda (x)
      (if (test x) (pure x) (fail)))))
#+end_src

With errors implemented, we can write "safe" versions of the boolean operations defined earlier:

#+begin_src scheme
(define (safe-neg bool)
  (neg (satisfy boolean? bool)))

(define (safe-conj bool1 bool2)
  (conj (satisfy boolean? bool1) (satisfy boolean? bool2)))

(define (safe-disj bool1 bool2)
  (disj (satisfy boolean? bool1) (satisfy boolean? bool2)))

(define (safe-impl bool1 bool2)
  (impl (satisfy boolean? bool1) (satisfy boolean? bool2)))
#+end_src

A test shows how our system automatically propagates errors through an arbitrarily complex computation:

#+begin_src scheme :results replace
(maybe (lambda () "error")
  (bool-logic
   (safe-impl
    (safe-neg (pure 'oops))
    (safe-neg (true)))))
#+end_src

#+RESULTS:
#+begin_example
error
#+end_example

** Dynamic state
:PROPERTIES:
:CUSTOM_ID: state
:END:

Now we briefly return to the state example of [[#part1][part 1]], this time implemented in terms of requests.  The syntax is almost exactly the same, with two exceptions.  =Put= now has to be aware of effects, and =state= now has to decide what to do when it returns.  Usually, we want to do something with the final state, so we package it up with the return value as a pair.

#+begin_src scheme
(define state-tag '(state))

(define (get)
  (request state-tag 'get))

(define (put req)
  (extend req
    (lambda (st)
      (request state-tag 'put st))))

(define (state st req)
  (handle state-tag
    req
    (lambda (x)
      (pure (cons x st)))
    (lambda (cont . sig)
      (match sig
        (('get) (state st (cont st)))
        (('put st) (state st (cont st)))))))
#+end_src

At this point we can write and interpret pretty convoluted computations without worrying about how they compose:

#+begin_src scheme :results replace
(lower
 (maybe (lambda () (pure "error"))
   (state #f
     (bool-logic
      (conj (put (neg (get)))
            (satisfy identity
                     (disj (get)
                           (false))))))))
#+end_src

#+RESULTS:
#+begin_example
'(#t . #t)
#+end_example

** Higher-order Effects

Now that we've built a language capable of running computations involving complex interactions of effects, it may be a good idea to step back.  We've already shown that the system is extensible in one sense: introducing new effects and handlers does not require rewriting earlier ones.  On the other hand, we've lost some of the abstraction available in scheme.

For example, our example demonstrating the state implementation in [[#part1][part 1]] can no longer be written in this language.  Our language is first-order---although we have access to variables and functions by embedding it in Scheme, the effects system itself lacks these powerful means of abstraction.  The question is, can we implement /higher-order effects/?

Surprisingly, we can.  Oleg Kiselyov demonstrates how in the [[http://okmij.org/ftp/Computation/having-effect.html][Having an Effect]] post mentioned above.  We translate it to Scheme below:

#+begin_src scheme
(define higher-order-tag '(higher-order))

(define (variable sym)
  (request higher-order-tag 'var sym))

(define (function sym body)
  (request higher-order-tag 'fun sym body))

(define (higher-order req)
  (let loop ((env '()) (req req))
    (handle higher-order-tag
      req
      pure
      (lambda (cont . sig)
        (loop env
              (match sig
                (('var sym)
                 (match (assq sym env)
                   (#f (fail))
                   ((_ . x) (cont x))))
                (('fun sym body)
                 (cont (lambda (x)
                         (loop (cons (cons sym x) env)
                               body))))))))))
#+end_src

Now all we need is an effectful application function:

#+begin_src scheme
(define (ap fun val)
  (extend (satisfy procedure? fun)
    (lambda (f)
      (extend val f))))
#+end_src

And we can talk about higher-order effects and an effectful lambda calculus!

#+begin_src scheme :results replace
(define id (function 'x (variable 'x)))

(lower (higher-order (ap id (pure 0))))
#+end_src

#+RESULTS:
#+begin_example
0
#+end_example

#+begin_src scheme :results replace
(define inv
  (function 'f
    (function 'p
      (ap (variable 'f)
          (neg (variable 'p))))))
#+end_src

#+begin_src scheme :results replace
(lower
 (higher-order
  (bool-logic
   (ap (ap inv id)
       (false)))))
#+end_src

#+RESULTS:
#+begin_example
#t
#+end_example

That's it, folks.  Happy birthday, Guile! :)

* Footnotes

[fn:1] This representation of the unit type as a "self-referential set" is also taken from [[http://okmij.org/ftp/Scheme/misc.html#unitary-type][Oleg's site]].
[fn:1] This representation of the unit type as a "self-referential set" is also taken from [[http://okmij.org/ftp/Scheme/misc.html#unitary-type][Oleg's site]].

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Extensible effects in Guile
  2021-03-07  5:28   ` Extensible effects in Guile (Guile Potluck 2021) Walter Lewis
@ 2021-03-08 15:42     ` Walter Lewis
  2021-03-08 20:58       ` Ludovic Courtès
  0 siblings, 1 reply; 21+ messages in thread
From: Walter Lewis @ 2021-03-08 15:42 UTC (permalink / raw)
  To: guile-user


Walter Lewis <wklew@mailbox.org> writes:

> Attached is a literate org-mode file which implements an 
> "extensible
> effects" system in Guile.  Extensible effects was first 
> implemented in
> Haskell by Oleg Kiselyov and others[1], and as far as I know has 
> not
> been attempted in Scheme.  It defines a metalanguage for writing
> programs which describe side-effects, and handlers which 
> interpret
> them.
>
> I'd hoped to publish it as a blog post, but technical issues got 
> in
> the way and I didn't want to delay any longer.  I will update 
> the list
> when the post is live; for now this file lives at
> https://git.sr.ht/~wklew/genshou
>
> I hope some folks find it interesting!
>
> Walter
>
> [1] okmij.org/ftp/Haskell/extensible/

After some sober editing and resolution of tech issues, I 
published the blog post:

https://wklew.srht.site/posts/genshou/

The literate org-mode file is still available at:

https://git.sr.ht/~wklew/genshou

The original file sent to the list has quite a few mistakes which 
I corrected.

Walter



^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Extensible effects in Guile
  2021-03-08 15:42     ` Extensible effects in Guile Walter Lewis
@ 2021-03-08 20:58       ` Ludovic Courtès
  2021-03-08 21:54         ` Walter Lewis
  0 siblings, 1 reply; 21+ messages in thread
From: Ludovic Courtès @ 2021-03-08 20:58 UTC (permalink / raw)
  To: guile-user

Hi,

Walter Lewis <wklew@mailbox.org> skribis:

> After some sober editing and resolution of tech issues, I published
> the blog post:
>
> https://wklew.srht.site/posts/genshou/

I found it to be a super clear and inspiring read, nicely demystifying
this topic.  Thanks!

Ludo’.




^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: Extensible effects in Guile
  2021-03-08 20:58       ` Ludovic Courtès
@ 2021-03-08 21:54         ` Walter Lewis
  0 siblings, 0 replies; 21+ messages in thread
From: Walter Lewis @ 2021-03-08 21:54 UTC (permalink / raw)
  To: guile-user


Ludovic Courtès <ludo@gnu.org> writes:

> I found it to be a super clear and inspiring read, nicely 
> demystifying
> this topic.  Thanks!
>
> Ludo’.

Thank you, Ludo, it's good to hear.  I sometimes feel I am not 
explaining things correctly at all, and it's a difficult topic.

Walter



^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2021-03-08 21:54 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <be71b3d5db9c43723c749f108dbd1ada7785daf2.camel.ref@yahoo.com>
2021-02-18 17:24 ` Guile Potluck 2021 Mike Gran
2021-02-20 17:38   ` Alex Sassmannshausen
2021-02-21 14:20     ` Dr. Arne Babenhauserheide
2021-02-21 17:21   ` Daniel Tornabene
2021-02-22  5:59   ` Aleix Conchillo Flaqué
2021-02-26 23:19     ` Aleix Conchillo Flaqué
2021-02-27  1:08       ` Mike Gran
2021-02-27  9:39         ` Dr. Arne Babenhauserheide
2021-02-27 10:14           ` Linus Björnstam
2021-02-27 16:42             ` Dr. Arne Babenhauserheide
2021-02-27 16:43               ` Dr. Arne Babenhauserheide
2021-03-02  7:08                 ` Dr. Arne Babenhauserheide
2021-03-01 11:34   ` pukkamustard
2021-03-01 21:14   ` Linus Björnstam
2021-03-01 21:37     ` Aleix Conchillo Flaqué
2021-03-02  0:43       ` Dr. Arne Babenhauserheide
2021-03-07  5:28   ` Extensible effects in Guile (Guile Potluck 2021) Walter Lewis
2021-03-08 15:42     ` Extensible effects in Guile Walter Lewis
2021-03-08 20:58       ` Ludovic Courtès
2021-03-08 21:54         ` Walter Lewis
2021-02-25 21:08 Guile Potluck 2021 Zelphir Kaltstahl

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).