From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Israelsson Tampe Newsgroups: gmane.lisp.guile.devel Subject: Re: more advanced bytevector => supervectors Date: Sat, 11 Sep 2021 19:03:16 +0200 Message-ID: References: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000ec7ce805cbbb35ae" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="25918"; mail-complaints-to="usenet@ciao.gmane.io" Cc: guile-devel To: lloda Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Sat Sep 11 19:12:32 2021 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mP6YK-0006aE-Od for guile-devel@m.gmane-mx.org; Sat, 11 Sep 2021 19:12:32 +0200 Original-Received: from localhost ([::1]:48346 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mP6YI-0006sa-S6 for guile-devel@m.gmane-mx.org; Sat, 11 Sep 2021 13:12:30 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:33672) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mP6Pd-0006ne-Jz for guile-devel@gnu.org; Sat, 11 Sep 2021 13:03:34 -0400 Original-Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]:37454) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mP6Pb-0007sV-P7 for guile-devel@gnu.org; Sat, 11 Sep 2021 13:03:33 -0400 Original-Received: by mail-pl1-x62e.google.com with SMTP id f21so901105plb.4 for ; Sat, 11 Sep 2021 10:03:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=EKB6WLzctyHtmZ6Zr4PCwt4JPTZ8b4JMGw9uds5nkPM=; b=ibkJyqhZu+g6yhdUXApuQ6Yq5xBuBRDuGOV//CLk5km8EEsbl/LrZOZHFJvxISBS0C AP56CRm1aqav2Bx4Cur8WMcMEpMljEypJ8x7uqFThvcdi6FZFZG6JcYXGihjBmLuZAyK aiz+QmmJjv+edsMMTGN/L5mE+N1PYFVMzrvPgrpNu0XxBacxLDxMHxSEqg54idywRHQ4 34dgEOSQmvRncUTXZr4w+DE8xqrkB0N3ymHznKK6WyEUsMPODznuR0R2wKKmSjO7B97m g2Y902IND8tftEfNSR+YjwxOEGoRayMf8s6g+VueRSlLUnEOnhPC7FicgWmE94ubWWL7 thnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=EKB6WLzctyHtmZ6Zr4PCwt4JPTZ8b4JMGw9uds5nkPM=; b=zEEXyZRd79P9QUZdIjAsuE+lebU4V3BsMd7oUKGqAFCdfMfjBmVjVUzJ57CSGS4hik TjXSYL6sksVyhSMwGIaKKdLQSymrix0hcdJ15oaihxDGtgHD0Evu0DFlQhFq9+T2/YK5 mm7CA5z7VkRpb6ea90z4V5DaoEfsuobN/XBIx7XV1ZsqJrQrO0DteVMzFfXtFOAnpwUe RJDvk0dJcx1Fl2hhUcgV0beZS9eA2+RwdTYDpIfA1HlxCCRendGzpvXFlNW2IU3gHPrB Gi1MZtw9vrxSAxkaF6kR29IbZI2lmi6RlRZOOCeULAgN9PuHbZMRbbqq/yH0r8uGmgTX 0s3g== X-Gm-Message-State: AOAM532BaBucpHyMaaw6tISrybFL8I9v65uWm7ZNfmGTxsG6P1YQs5mq O9aYHBWm3mGMFABdgUJC5vBGJb34k7/DqtFJDQw= X-Google-Smtp-Source: ABdhPJzeMZxLZ+egSiLxsAZ+/Xknlje9LUhtbDFO6HyI4d/tF3LJ4ej2yZoOp9uAm2oZUy7y6FqMVUWfBXbOq7nTjsk= X-Received: by 2002:a17:90a:5b06:: with SMTP id o6mr3754219pji.121.1631379807565; Sat, 11 Sep 2021 10:03:27 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=stefan.itampe@gmail.com; helo=mail-pl1-x62e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.io gmane.lisp.guile.devel:20854 Archived-At: --000000000000ec7ce805cbbb35ae Content-Type: text/plain; charset="UTF-8" I did some test ands wingo's superb compiler is about equally fast for a hand made scheme loop as the automatic dispatch for getter and setter. It e.g. can copy from e.g. u8 to i16 in about 100 op's / second using native byte order. However compiling it in C lead to nasty 2 Go ops / second. So for these kind of patterns it is still better to work in C as it probaly vectorises the operation quite well. Supervectors supports pushing busy loops to C very well and I will probably enable fast C code for some simple utility ops. On Wed, Sep 8, 2021 at 9:18 AM lloda wrote: > > > On 8 Sep 2021, at 04:04, Stefan Israelsson Tampe > wrote: > > > ... > > > So using get-setter typically means > ((get-setter #f bin1 #f > (lambda (set) (set v 2 val))) > > #:is-endian 'little ;; only consider little endian setters > like I know > #:is-unsigned #t ;; only use unsigned > #:is-integer #t ;; only use integer representations > #:is-fixed #t ;; do not use the scm value vector > versions > ) > So a version where we only consider handling nonegative integers of up to > 64bit. The gain is faster compilation as this ideom will dispatch > between 4 different versions of the the loop lambda and the compiler could > inline all of them or be able to detect the one that are used and hot > compile that version > (a feature we do not have yet in guile) now whe you select between a ref > and a set you will similarly end up with 4*4 versions = 16 different loops > that. full versions > is very large and a double loop with all featurs consists of (2*2 + > 3*2*2*2 + 4 + 1)**2 = 33*33 ~ 1000 versions of the loop which is crazy if > we should expand the loop > for all cases in the compilation. Now guile would just use a functional > approach and not expand the loop everywhere. We will have parameterised > versions of > libraries so that one can select which versions to compile for. for > example the general functions that performs transform form one supervector > to another is a general > ideom that would use the full dispatc which is not practical, > > > I'm curious where you're going with this. > > I implemented something similar (iiuc) in > https://github.com/lloda/guile-newra/, specifically > https://github.com/lloda/guile-newra/blob/master/mod/newra/map.scm , > where the lookup/set methods are inlined in the loop. The compilation times > indeed grow exponentially so I'm forced to have a default 'generic' case. > > The idea for fixing this was to have some kind of run time compilation > cache so only a fixed number of type combinations that actually get used > would be compiled, instead of the tensor product of all types. But I > haven't figured out, or actually tried to do that yet. > > Regards > Daniel > > --000000000000ec7ce805cbbb35ae Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I did some test ands wingo's superb compiler is about = equally fast for a hand made=C2=A0scheme loop as the automatic dispatch for= getter and setter. It e.g. can copy from=C2=A0
e.g. u8 to i16 in about= 100 op's / second using native byte order. However compiling=C2=A0it i= n C lead to nasty 2 Go ops / second. So for these kind of patterns
it is still better to work in C as it probaly=C2=A0vectorises the operatio= n quite well. Supervectors supports pushing busy loops to C very well and I= will probably=C2=A0
enable fast C code for some simple utility o= ps.

On Wed, Sep 8, 2021 at 9:18 AM lloda <lloda@sarc.name> wrote:


On 8 Sep 2021, at 04:04, St= efan Israelsson Tampe <stefan.itampe@gmail.com> wrote:


...
<= /div>

= So using get-setter typically means
((get-setter #f bin1 #f=C2=A0=
=C2=A0 =C2=A0(lambda (set) (set v 2 val)))

<= div>=C2=A0 =C2=A0#:is-endian 'little=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = ;; only consider little endian setters like I know=C2=A0
=C2=A0 = =C2=A0#:is-unsigned=C2=A0 #t=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0;; only use u= nsigned
=C2=A0 =C2=A0#:is-integer=C2=A0 =C2=A0 =C2=A0 #t=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0;; only use integer representations
= =C2=A0 =C2=A0#:is-fixed=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 #t=C2=A0 =C2=A0 = =C2=A0 =C2=A0 ;; do not use the scm value vector versions
)
=
So a version where we only consider handling nonegative integers of up= to 64bit. The gain is faster compilation as this ideom=C2=A0will dispatch<= /div>
between 4 different versions=C2=A0of the the loop lambda and the = compiler could inline all of them or be able to detect the one that are use= d and hot compile that version
(a feature we do not have yet in g= uile) now whe you select between a ref and a set you will similarly=C2=A0en= d up with 4*4 versions =3D 16 different loops that. full versions
is very large and a double loop with all featurs=C2=A0consists of (2*2=C2= =A0+ 3*2*2*2 + 4=C2=A0+ 1)**2 =3D 33*33 ~ 1000 versions of the loop which i= s crazy if we should expand the loop
for all cases in the compila= tion. Now guile would just use a functional approach and not expand the loo= p everywhere. We will have parameterised versions of
libraries so= that one can select which versions=C2=A0to compile for. for example the ge= neral functions that performs transform form one supervector to another is = a general
ideom=C2=A0that would use the full dispatc=C2=A0which i= s not practical,=C2=A0

I'm curious where you're going with = this.

I implemented something similar (iiuc) in=C2=A0https://github.com/lloda/guile-n= ewra/, specifically=C2=A0https://github.com/lloda= /guile-newra/blob/master/mod/newra/map.scm=C2=A0, where the lookup/set = methods are inlined in the loop. The compilation times indeed grow exponent= ially so I'm forced to have a default 'generic' case.=C2=A0

The idea for fixing this was to have some kind of run= time compilation cache so only a fixed number of type combinations that ac= tually get used would be compiled, instead of the tensor product of all typ= es. But I haven't figured out, or actually tried to do that yet.
<= div>
Regards
=
Daniel
=

--000000000000ec7ce805cbbb35ae--