From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Chris Vine Newsgroups: gmane.lisp.guile.devel Subject: Re: port threadsafety redux Date: Tue, 17 Feb 2015 12:11:29 +0000 Message-ID: <20150217121129.6e0e2540@bother.homenet> References: <87vbj816sg.fsf@pobox.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1424175102 32184 80.91.229.3 (17 Feb 2015 12:11:42 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 17 Feb 2015 12:11:42 +0000 (UTC) To: guile-devel@gnu.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Tue Feb 17 13:11:34 2015 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1YNh02-0005T4-GZ for guile-devel@m.gmane.org; Tue, 17 Feb 2015 13:11:34 +0100 Original-Received: from localhost ([::1]:44905 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YNgzw-0004a2-Pa for guile-devel@m.gmane.org; Tue, 17 Feb 2015 07:11:28 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:54134) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YNgzt-0004Zx-IB for guile-devel@gnu.org; Tue, 17 Feb 2015 07:11:26 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YNgzq-0004ZN-9Y for guile-devel@gnu.org; Tue, 17 Feb 2015 07:11:25 -0500 Original-Received: from smtpout3.wanadoo.co.uk ([80.12.242.59]:35722 helo=smtpout.wanadoo.co.uk) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YNgzp-0004Z6-Vh for guile-devel@gnu.org; Tue, 17 Feb 2015 07:11:22 -0500 Original-Received: from bother.homenet ([95.146.112.1]) by mwinf5d41 with ME id tQBJ1p00D01rpef03QBKf9; Tue, 17 Feb 2015 13:11:19 +0100 X-ME-Helo: bother.homenet X-ME-Date: Tue, 17 Feb 2015 13:11:19 +0100 X-ME-IP: 95.146.112.1 Original-Received: from bother.homenet (localhost [127.0.0.1]) by bother.homenet (Postfix) with ESMTP id 1EC218C190 for ; Tue, 17 Feb 2015 12:11:29 +0000 (GMT) In-Reply-To: <87vbj816sg.fsf@pobox.com> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.25; i686-pc-linux-gnu) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 80.12.242.59 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:17663 Archived-At: On Wed, 11 Feb 2015 22:23:43 +0100 Andy Wingo wrote: > Hi! >=20 > So, threads and ports again. We didn't really come to a resolution in > this thread: >=20 > http://article.gmane.org/gmane.lisp.guile.devel/17023 >=20 > To recap, in Guile 2.0 a port has mutable internal state that can be > corrupted when when multiple threads write to it at once. I ran into > this when doing some multithreaded server experiments, and fixed it in > the same way that libc fixes the issue for stdio streams: >=20 > https://www.gnu.org/software/libc/manual/html_node/Streams-and-Threads.= html#Streams-and-Threads >=20 > Namely, ports can have associated recursive mutexes. They can be in a > mode in which every operation on a port grabs the mutex. The > interface to set a port into unlocked mode (=C3=A0 la fsetlocking) is > unimplemented, but the machinery is there. >=20 > This change fixed the crashes I was seeing, but it slows down port > operations. For an intel chip from a couple years ago the slowdown > was something on the order of 3x, for a tight putchar() loop; for > Loongson it could be as bad as 26x. Mark was unhappy with this. >=20 > Mark also made the argument that locking on port operations doesn't > always make sense. Indeed I quote from the libc documentation: >=20 > But there are situations where this is not enough and there are also > situations where this is not wanted. The implicit locking is not > enough if the program requires more than one stream function call to > happen atomically. One example would be if an output line a program > wants to generate is created by several function calls. The > functions by themselves would ensure only atomicity of their own > operation, but not atomicity over all the function calls. For this it > is necessary to perform the stream locking in the application code. >=20 > So we don't yet expose the equivalent of flockfile, but at this point > since there are still concerns out there I wanted to ask if the > current solution still makes sense. >=20 > I hope this is a fair summary of the issue. >=20 > My perspective on this is that crashes are unacceptable, and also that > it does make sense to log to stderr from multiple threads at once. > When writing to ports under error conditions you don't always have > the luxury of being able to coordinate access in some nicer way. I > sympathize with the desire to make put-char etc faster, as that means > that more code can be written in Scheme. >=20 > One possible alternate solution would be to expose ports more to > Scheme and so to make it easier and safer for Scheme to manipulate > port data. This would also make it possible to implement coroutines > in Scheme that yield when IO would block. >=20 > Or, we could just make stdio/stderr be locked by default, and some > other things not. Seems squirrely to me though. >=20 > Dunno. I would add that although there is a solution to this issue in > master, it might not make it into 2.2. There will probably be a dozen > prereleases before 2.2.0, so even if a 2.1.1 manages to make it out > the door before we come to a solution, that doesn't mean that the > choices in such a release are the right or final ones. Here is a comment from someone who is a guile user rather than a guile developer. Since guile provides native threads, a minimum requirement seems to me to be that when the guile library writes to stderr on its own account, it does so in a thread safe way (in the "doesn't crash the program" sense). Since guile (at present) writes to stderr via a global buffered port object, it means that that needs to be thread safe. An alternative is for the library to write error messages directly to the stderr file descriptor (which is intrinsically thread safe), but that would rule out character by character error printing as with put-char/write-char. Beyond that, different standards which accommodate threads within the respective standard require different things. You have referred to POSIX.1c, which requires all its functions that operate on character streams (represented by pointers to objects of type FILE) to be thread safe in a data race but not interleaving sense, and provides access for user code to the internal locks to deal with interleaving. As far as I can tell C11 is silent on the point, even though it adopts threading primitives and a memory model based on the C++11 one. C++11 does not go as far as POSIX for its own i/o streams. Instead the global objects for stdout (cout), stdin (cin) and stderr (cerr and clog) and wide stream variants must be thread safe, but whether other i/o objects are thread safe is up to the implementation - and generally they are not. Synchronization is generally left where it should be, with the user, since thread safety (in the formal data race sense) is not of itself enough. Generally you also need to write to or read from ports in a way which prevents any interleaving which would corrupt the data format which is being written or read. A compromise position is possible for guile. Ports could provide an internal lock with user access on the POSIX flockfile()/funlockfile() model, but whether the port itself uses the locks by default on reading or writing (rather than by user intervention using the above functions) could be optional on all except the default input, output and error ports. (POSIX provides a few *_unlocked() functions for reading or writing, but I think this should be a port configuration setting.) Chris