From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.user,gmane.lisp.guile.devel Subject: [ann] fibers 0.1.0 Date: Mon, 04 Jul 2016 10:34:05 +0200 Message-ID: <87h9c57qf6.fsf@pobox.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1467621354 7242 80.91.229.3 (4 Jul 2016 08:35:54 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Mon, 4 Jul 2016 08:35:54 +0000 (UTC) Cc: guile-devel@gnu.org To: guile-user@gnu.org Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Mon Jul 04 10:35:40 2016 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bJzLA-0006fj-Sx for guile-user@m.gmane.org; Mon, 04 Jul 2016 10:34:54 +0200 Original-Received: from localhost ([::1]:45914 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJzLA-00072k-48 for guile-user@m.gmane.org; Mon, 04 Jul 2016 04:34:52 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:49025) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJzKi-00070F-21 for guile-user@gnu.org; Mon, 04 Jul 2016 04:34:25 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bJzKf-0007by-RQ for guile-user@gnu.org; Mon, 04 Jul 2016 04:34:23 -0400 Original-Received: from pb-sasl2.pobox.com ([64.147.108.67]:58131 helo=sasl.smtp.pobox.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bJzKa-0007Zz-Rn; Mon, 04 Jul 2016 04:34:16 -0400 Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl2.pobox.com (Postfix) with ESMTP id 025F922857; Mon, 4 Jul 2016 04:34:14 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:date:message-id:mime-version:content-type; s=sasl; bh=W joLGpQN1GHKXNvD4lSHuLVftKY=; b=kQIhHFIh2RsAYaVm8lz/7ZjE3payNrZpN qqWgxeft1fzJHAIjD9SDd0UVgCIyUTmpSjkG8oIGHiuiqm71JH0YfrGFsyZj9AIl ltBAMWbcm1DIaIp9DDMsefT3jIkDTGCTGuGE5+LY15fRpms16opihM+fuxdTBTKO Oa7chs4Ws8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:date:message-id:mime-version:content-type; q=dns; s= sasl; b=Z+rLR8SmKDaOyehBcQC+IIATObIqrTuhhNi4ULHo685S+bXy8Eej7KEm OqzPg9GHSftjTZLonbeZg4uVreIQdnlHlsLKyrHwy9cdkjtEWrJU+ryS0V6cajlQ 1/Zl8GcCxip99fv3kTT5topuBm4i6HfhFMPim48va+ofdwu1XLk= Original-Received: from pb-sasl2.nyi.icgroup.com (unknown [127.0.0.1]) by pb-sasl2.pobox.com (Postfix) with ESMTP id EEDA722856; Mon, 4 Jul 2016 04:34:13 -0400 (EDT) Original-Received: from clucks (unknown [88.160.190.192]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl2.pobox.com (Postfix) with ESMTPSA id 0B0F922853; Mon, 4 Jul 2016 04:34:12 -0400 (EDT) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-Pobox-Relay-ID: 1633BC90-41C2-11E6-9910-28A6F1301B6D-02397024!pb-sasl2.pobox.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 64.147.108.67 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:12722 gmane.lisp.guile.devel:18461 Archived-At: Hi all, I just released Fibers 0.1.0. Fibers is an experimental facility for Erlang-like concurrency in Guile 2.2. As an example, here is a ping server written in Fibers: (define (socket-loop socket store) (let loop () (match (accept socket) ((client . addr) (set-nonblocking! client) ;; Disable Nagle's algorithm. We buffer ourselves. (setsockopt client IPPROTO_TCP TCP_NODELAY 0) (spawn (lambda () (client-loop client addr store))) (loop))))) As you can see it's straightforward in style. For each client that comes in, we spawn a fiber to handle the client. Here's the fiber that handles the client: (define (client-loop port addr store) (let loop () (let ((line (read-line port))) (cond ((eof-object? line) (close-port port)) (else (put-string port line) (put-char port #\newline) (force-output port) (loop)))))) Nice, no? Whenever read-line, put-string, put-char, or force-output would block, a fiber suspends the the current scheduler, adds its fd to an epoll set, and the scheduler runs something else. There's a bit of boilerplate in the example when setting up the socket to make things nonblocking, but anyway here's the whole thing: https://github.com/wingo/fibers/blob/master/examples/ping-server.scm Fibers is in early stages. It used to be called "ethreads" and lived in an experimental branch of Guile, but now that we have added the suspendable ports facility to Guile, Fibers can live in an external repo. I want Guile to have a good concurrency story around servers and clients, where there can be thousands of connections at once and no blocking. I think it can be part of Guile eventually but we need to experiment as a community. Fibers is in the same space as guile-a-sync or 8sync, which have already been doing great experimentation in this regard. I suspect that in the end we will bless one facility as the default one and incorporate it into Guile, but even then we will still provide the ability for the user to choose a different set of concurrency primitives. The to-do list is somewhat long. * Fibers doesn't have any concurrency primitives besides sleep and what suspendable-ports provides. I think the next step is to implement channels, like Go. * Missing tests :/ Though there is a memcached implementation in the examples, and a web server implementation that uses fibers. * Nice REPL facilities, like ,fibers to list all fibers and their states, ,fiber to enter a backtrace using a fiber's delimited continuation as the state, ,fkill to kill a fiber, and so on. * Documentation. * Performance improvements, testing, and auditing for correctness and fairness. Currently a fiber-based ping client that opens 4000 concurrent connections and makes 25 pings each takes around a second or two on my machine, so let's say 50K pings per second per core. The web server currently runs at around 10K requests/second/core for a simple "hello world" page. I'm sure there are improvements that can be made but these numbers are already OK. Download fibers from here: https://wingolog.org/pub/fibers/fibers-0.1.0.tar.gz The sha256sum is: 6b7654d27781af2602b9e692f1d2a932a582c68e0a14efa38eaba265d4b464ab fibers-0.1.0.tar.gz Happy hacking, Andy