From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.org!not-for-mail
From: Andy Wingo <wingo@pobox.com>
Newsgroups: gmane.lisp.guile.devel
Subject: port threadsafety redux
Date: Wed, 11 Feb 2015 22:23:43 +0100
Message-ID: <87vbj816sg.fsf@pobox.com>
NNTP-Posting-Host: plane.gmane.org
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Trace: ger.gmane.org 1423689853 304 80.91.229.3 (11 Feb 2015 21:24:13 GMT)
X-Complaints-To: usenet@ger.gmane.org
NNTP-Posting-Date: Wed, 11 Feb 2015 21:24:13 +0000 (UTC)
To: guile-devel@gnu.org
Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Wed Feb 11 22:24:01 2015
Return-path: <guile-devel-bounces+guile-devel=m.gmane.org@gnu.org>
Envelope-to: guile-devel@m.gmane.org
Original-Received: from lists.gnu.org ([208.118.235.17])
	by plane.gmane.org with esmtp (Exim 4.69)
	(envelope-from <guile-devel-bounces+guile-devel=m.gmane.org@gnu.org>)
	id 1YLelK-0006Fd-21
	for guile-devel@m.gmane.org; Wed, 11 Feb 2015 22:23:58 +0100
Original-Received: from localhost ([::1]:47056 helo=lists.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <guile-devel-bounces+guile-devel=m.gmane.org@gnu.org>)
	id 1YLelJ-0007bS-DO
	for guile-devel@m.gmane.org; Wed, 11 Feb 2015 16:23:57 -0500
Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:32779)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <wingo@pobox.com>) id 1YLelE-0007bJ-Vd
	for guile-devel@gnu.org; Wed, 11 Feb 2015 16:23:54 -0500
Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <wingo@pobox.com>) id 1YLelB-0000dG-Pb
	for guile-devel@gnu.org; Wed, 11 Feb 2015 16:23:52 -0500
Original-Received: from pb-sasl1.int.icgroup.com ([208.72.237.25]:53677
	helo=sasl.smtp.pobox.com) by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <wingo@pobox.com>) id 1YLelB-0000d4-JZ
	for guile-devel@gnu.org; Wed, 11 Feb 2015 16:23:49 -0500
Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1])
	by pb-sasl1.pobox.com (Postfix) with ESMTP id A73BB35F01
	for <guile-devel@gnu.org>; Wed, 11 Feb 2015 16:23:48 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to
	:subject:date:message-id:mime-version:content-type
	:content-transfer-encoding; s=sasl; bh=bTA1fiOuHBIgCmvwDXZI9qkUg
	Eg=; b=djtGcGTih7W5l66bii59xY9wiSwp8AKyTvI6LXyNflkZ4OoBHveslYoNx
	XUp/2jSt7droOWbRLVdbtSI6brkoy5l+dHz4Z+Fdiaw6dFkfKXEY7tmea9hv3Sp7
	PpBwf2nQc2hHHG8VmUZTP2w2rI7BShgEXGNsEHlabrErZAnwjU=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:subject
	:date:message-id:mime-version:content-type
	:content-transfer-encoding; q=dns; s=sasl; b=ukwsJqyJZ/9WsvPW6+L
	IOJ04zIpXk9jOrMLo5Iwy0At8bI2OSYHuQFHAg1FGm5PKRU6/iUUBTAGG+WPs4Id
	R8i0Z3WIWWTejdqknTL9Rj/VFrqiLGv/MbDhRQifyRsy0XXNRMSp1myYozEoWMMe
	Z8vpmcEcUEW0JDYC3AJE5Wtg=
Original-Received: from pb-sasl1.int.icgroup.com (unknown [127.0.0.1])
	by pb-sasl1.pobox.com (Postfix) with ESMTP id 9D5CD35F00
	for <guile-devel@gnu.org>; Wed, 11 Feb 2015 16:23:48 -0500 (EST)
Original-Received: from badger (unknown [88.160.190.192])
	(using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by pb-sasl1.pobox.com (Postfix) with ESMTPSA id D2BFB35EFE
	for <guile-devel@gnu.org>; Wed, 11 Feb 2015 16:23:47 -0500 (EST)
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4 (gnu/linux)
X-Pobox-Relay-ID: 4434E1B0-B234-11E4-BD44-8FDD009B7A5A-02397024!pb-sasl1.pobox.com
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 208.72.237.25
X-BeenThere: guile-devel@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "Developers list for Guile,
	the GNU extensibility library" <guile-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/guile-devel>,
	<mailto:guile-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/guile-devel>
List-Post: <mailto:guile-devel@gnu.org>
List-Help: <mailto:guile-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/guile-devel>,
	<mailto:guile-devel-request@gnu.org?subject=subscribe>
Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org
Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org
Xref: news.gmane.org gmane.lisp.guile.devel:17658
Archived-At: <http://permalink.gmane.org/gmane.lisp.guile.devel/17658>

Hi!

So, threads and ports again.  We didn't really come to a resolution in
this thread:

  http://article.gmane.org/gmane.lisp.guile.devel/17023

To recap, in Guile 2.0 a port has mutable internal state that can be
corrupted when when multiple threads write to it at once.  I ran into
this when doing some multithreaded server experiments, and fixed it in
the same way that libc fixes the issue for stdio streams:

  https://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.ht=
ml#Streams-and-Threads

Namely, ports can have associated recursive mutexes.  They can be in a
mode in which every operation on a port grabs the mutex.  The interface
to set a port into unlocked mode (=C3=A0 la fsetlocking) is unimplemented,
but the machinery is there.

This change fixed the crashes I was seeing, but it slows down port
operations.  For an intel chip from a couple years ago the slowdown was
something on the order of 3x, for a tight putchar() loop; for Loongson
it could be as bad as 26x.  Mark was unhappy with this.

Mark also made the argument that locking on port operations doesn't
always make sense.  Indeed I quote from the libc documentation:

  But there are situations where this is not enough and there are also
  situations where this is not wanted. The implicit locking is not
  enough if the program requires more than one stream function call to
  happen atomically. One example would be if an output line a program
  wants to generate is created by several function calls. The functions
  by themselves would ensure only atomicity of their own operation, but
  not atomicity over all the function calls. For this it is necessary to
  perform the stream locking in the application code.

So we don't yet expose the equivalent of flockfile, but at this point
since there are still concerns out there I wanted to ask if the current
solution still makes sense.

I hope this is a fair summary of the issue.

My perspective on this is that crashes are unacceptable, and also that
it does make sense to log to stderr from multiple threads at once.  When
writing to ports under error conditions you don't always have the luxury
of being able to coordinate access in some nicer way.  I sympathize with
the desire to make put-char etc faster, as that means that more code can
be written in Scheme.

One possible alternate solution would be to expose ports more to Scheme
and so to make it easier and safer for Scheme to manipulate port data.
This would also make it possible to implement coroutines in Scheme that
yield when IO would block.

Or, we could just make stdio/stderr be locked by default, and some other
things not.  Seems squirrely to me though.

Dunno.  I would add that although there is a solution to this issue in
master, it might not make it into 2.2.  There will probably be a dozen
prereleases before 2.2.0, so even if a 2.1.1 manages to make it out the
door before we come to a solution, that doesn't mean that the choices in
such a release are the right or final ones.

Regards,

Andy
--=20
http://wingolog.org/