From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Israelsson Tampe Newsgroups: gmane.lisp.guile.devel Subject: New write function Date: Wed, 19 Oct 2022 18:47:13 +0200 Message-ID: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="00000000000097250705eb65f60f" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="25592"; mail-complaints-to="usenet@ciao.gmane.io" To: guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Wed Oct 19 18:48:19 2022 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1olCEs-0006OG-Ll for guile-devel@m.gmane-mx.org; Wed, 19 Oct 2022 18:48:18 +0200 Original-Received: from localhost ([::1]:52618 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1olCEr-0005gW-Ad for guile-devel@m.gmane-mx.org; Wed, 19 Oct 2022 12:48:17 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:38318) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1olCE7-0005fg-BH for guile-devel@gnu.org; Wed, 19 Oct 2022 12:47:31 -0400 Original-Received: from mail-ed1-x52d.google.com ([2a00:1450:4864:20::52d]:44982) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1olCE5-0000DY-7j for guile-devel@gnu.org; Wed, 19 Oct 2022 12:47:31 -0400 Original-Received: by mail-ed1-x52d.google.com with SMTP id g27so26136221edf.11 for ; Wed, 19 Oct 2022 09:47:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=w1gVheXGnm7uMJS/1RHtnJvGjybwxdCWgNkd5DO6w7o=; b=cvmM6pcWryt7mxrPBAKe4gjvCGgWXHokcLf3ZH943zdWl1NRRGRRVeRYgXTOK8PztR svaZReTkzrRK5xzt6y8kSMvMkZepB+rSRR8KgjQNY/kLgTWOaTLIEE4tjid7DdB3uMzC u/kwf5X8Uqhx2Bjn7TMdD4UEOL0UWjDjxj+eDAoJXXKqJ6W7h/wA9PnPvQgDfytQYH43 A5xTiqdnySKgNMM4ntpRTjUQoURH8NXuBQ82Pz+FmHNBg67CWd0MWBTkRfGPnAnYQxAi jIbdzYfLLnukMyTfObzUY/o+JgLl8yhxHwklhjS9OtHbETS3uPy3yck8H1LhC/LBiOAl 9t+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=w1gVheXGnm7uMJS/1RHtnJvGjybwxdCWgNkd5DO6w7o=; b=e0eMtDT3PVHoAfhTNtipqNezYjuaJtHlLMAhu/ErMn/HZdmWSM71qW02a+E+CUSy5C E3wMIOT39MQx5QtVXQJCZdajgQZXPZZKgkFziA41n3JVeluIG1Ge9KunvC+sI3tcc+kU UnDrFjVhchxNf8qD8105iE+pbsA3E16bexknye25IKnlfDeWEHma4f58jYAU90EBDOoq ULyD3lUAoonRIl2O8AK0bKH+vcP5Wqgwsw6jh1f7K28wgLU4Gle4XdNuWt4lVvTDB9S0 g8hcihcZ4opbJ21apzmb/a1A9da3KjIlR7AOhflWCqoLM8HPCyvU/PBlC5BsNQ/kjzpU YDGQ== X-Gm-Message-State: ACrzQf0xHg4byERsKoykueh2s3lOVqhJ5rgl3uJrWsJXL+8+9JSe3KVC 8E4SckpFm3bkFNITr5S32WPq3sv8AzVS2l9cBXhE+sB8YaM= X-Google-Smtp-Source: AMsMyM6A4fRYGUIVDUIwxL12wywNlQ55JOoun/95wdSIRjZKTXV0ZX/fzH6z8NI2T/P2x6GQaglEj6o3jro7BanFoGM= X-Received: by 2002:a05:6402:3588:b0:45d:7d14:baf2 with SMTP id y8-20020a056402358800b0045d7d14baf2mr8271494edc.1.1666198044876; Wed, 19 Oct 2022 09:47:24 -0700 (PDT) Received-SPF: pass client-ip=2a00:1450:4864:20::52d; envelope-from=stefan.itampe@gmail.com; helo=mail-ed1-x52d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.io gmane.lisp.guile.devel:21440 Archived-At: --00000000000097250705eb65f60f Content-Type: text/plain; charset="UTF-8" Hello guilers! I was wondering if we should make an effort to replace guile's C-based write function with a more "schemy" solution that allows us to use write/display on large scheme data structures without stalling the fibers system and be very fast and feature rich at the same time. I did a take some months ago with the repository found at: https://gitlab.com/tampe/guile-persist To import this library's write/display just make sure that it is installed and use (use-modules (ice-9 write)) Then use write and display from that module just as you usually do in guile. Now the solution I suggest we look at is to combine C routines with scheme to get the best of both worlds. In C we get quite a lot of speed from taking advantage of SIMD instructions like testing if 16 bytes are all in the range [0,127] in one instruction and the coded library tries to do good for all kinds of encodings and type of scheme strings (hence quite large). Some string operations will be in an order of 100X faster with this library than guiles own C implementation. Now C is bad in two ways. It is a less powerful language than Scheme and harder to maintain. However quite a lot of the power of scheme can be saved by separating more logic into scheme and just focusing the C code on smaller parts. Also we would like to use scheme because it interoperates well with fibers. As it is now, writing a huge scheme datastructure freezes the thread if all is in C. But this does not mean that it is impossible to use C. By instead trampolining into C serializing chunks of data we can improve the behavior in a fibers context quite dramatically. The reason to still use C is speed though and that guile cannot take advantage of SIMD instructions. This should in the end change with time but no work is currently planned for this and gcc today has a really nice infrastructure towards such constructs in C. Here are some blog posts about the writer I propose that we look into, http://itampe.com/display-strings.html http://itampe.com/printing-doubles.html This code handles doubles and floats better than guiles C based writer. First it fixes a bug when we write out a bytevector of f32 currently guile ends up writing them as doubles which creates a lot of noise in the printout. And secondly we can tell the writer to print floats in hex format which guile cannot (we also have i reader in the repository that can read hex floats) we actually can write in any of base 2,4,8,16,32 base floats/doubles The drawback is most likely tougher code to maintain, but thanks to gcc's support for SIMD instructionsm, not too bad in my view. WDYT? --00000000000097250705eb65f60f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello guilers!

I was wondering if we sh= ould make an effort to replace guile's C-based write
function= with a more "schemy" solution that allows us to use write/displa= y on
large scheme data structures without stalling the fibers sys= tem and be very fast and feature rich at the same time.

I did a take some months ago with the repository found at:

To import this library'= s write/display just make sure that it is installed and use
(use-= modules (ice-9 write))

Then use write and display = from that module just as you usually do in guile.

= Now the solution I suggest we look at is to combine C routines with scheme = to get the best of both worlds. In C we get quite a lot of speed from takin= g advantage of SIMD instructions like testing if 16 bytes are all in the ra= nge [0,127] in one instruction and the coded library tries to do good for a= ll kinds of encodings and type of scheme strings (hence quite=C2=A0large). = Some string operations will be in an order of 100X faster with this library= than guiles own C implementation. Now C is bad in two ways. It is a less p= owerful language than Scheme and harder to maintain. However quite a lot of= the power of scheme can be saved by separating more logic into scheme and = just focusing the C code on smaller parts. Also we would like to use scheme= because it interoperates well with fibers. As it is now, writing a huge sc= heme datastructure freezes the thread if all is in C. But this does not mea= n that it is impossible to use C. By instead trampolining into C serializin= g chunks of data we can improve the behavior in a fibers context quite dram= atically. The reason to still use C is speed though and that guile cannot t= ake advantage of SIMD instructions. This should in the end change with time= but no work is currently planned for this and gcc today has a really nice = infrastructure towards such constructs in C.

Here = are some blog posts about the writer I propose that we look into,
http://itampe.com/displ= ay-strings.html

This code handles doubles and floats better than guiles C based wr= iter. First it fixes a bug when we write out a bytevector of f32 currently = guile ends up writing them as doubles which creates a lot of noise in the p= rintout. And secondly we can tell the writer to print floats in hex format = which guile cannot (we also have i reader in the repository that can read h= ex floats) we actually can write in any of base 2,4,8,16,32 base floats/dou= bles

The drawback is most likely tougher code to m= aintain, but thanks to gcc's support for SIMD instructionsm, not too ba= d in my view.

WDYT?
--00000000000097250705eb65f60f--