From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Amirouche Boubekki Newsgroups: gmane.lisp.guile.devel Subject: Re: (n-for-each-par-map 16 store download (reverse (iota (max-id))))) crash Date: Fri, 06 Jul 2018 07:55:09 +0200 Message-ID: References: <567d1b31558c2b51908d8a272ff61494@hypermove.net> <871schk01n.fsf@netris.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Trace: blaine.gmane.org 1530856395 14548 195.159.176.226 (6 Jul 2018 05:53:15 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 6 Jul 2018 05:53:15 +0000 (UTC) User-Agent: Roundcube Webmail/1.1.2 Cc: Guile Devel To: Mark H Weaver Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Fri Jul 06 07:53:11 2018 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fbJg7-0003gv-4o for guile-devel@m.gmane.org; Fri, 06 Jul 2018 07:53:11 +0200 Original-Received: from localhost ([::1]:55902 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fbJiE-0002HH-4B for guile-devel@m.gmane.org; Fri, 06 Jul 2018 01:55:22 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:34967) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fbJi9-0002H9-1H for guile-devel@gnu.org; Fri, 06 Jul 2018 01:55:18 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fbJi5-0003PL-VY for guile-devel@gnu.org; Fri, 06 Jul 2018 01:55:17 -0400 Original-Received: from relay9-d.mail.gandi.net ([217.70.183.199]:39731) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fbJi5-0003JS-Os for guile-devel@gnu.org; Fri, 06 Jul 2018 01:55:13 -0400 Original-Received: from webmail.gandi.net (unknown [10.200.201.1]) (Authenticated sender: amirouche@hypermove.net) by relay9-d.mail.gandi.net (Postfix) with ESMTPA id 7920DFF805; Fri, 6 Jul 2018 05:55:09 +0000 (UTC) In-Reply-To: <871schk01n.fsf@netris.org> X-Sender: amirouche@hypermove.net X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 217.70.183.199 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19586 Archived-At: Hi Mark, Thanks for your support and time. On 2018-07-05 20:30, Mark H Weaver wrote: > Hi, > > Amirouche Boubekki writes: > >> I have a program that try to download hackernews locally. >> >> What it does is simple, it fetch the max identifier and >> http-get each json value starting with the most recent >> item. I use n-for-each-par-map with 16 threads I have >> 8 cores. >> >> Here is the full program: > > [...] > >> (define (store pair) >> (if (null? pair) >> (format #t "X\n") >> (let ((port (open-file "hn.scm" "a"))) >> (format #t "~a\n" (car pair)) > > These calls to (format #t ...), which write to a port that is shared by > multiple threads, should be performed while holding a mutex to prevent > concurrent writes to the same port. Ok, but it's called by: (n-for-each-par-map n sproc pproc lst1 lst2) and the manual says about SPROC that : The sproc calls are made serially, in list element order, one at a time. In my case SPROC is 'store', so my understanding is that it should require no lock. > > I/O operations in Guile do not include built-in thread synchronization, > at least not in the fast path cases. However, an effort was made to > avoid _crashes_ on common architectures in the event of concurrent use > of the same port. Our hope was that the worst that would typically > happen is garbled I/O. Perhaps we failed to realize that hope. I saw that behavior while use fibers. >> How can I debug this? > > It would be helpful to see a GDB backtrace from a crash in this > program. I will retry without mutex and only gdb to reproduce the crash and have a trace. > Does it help to protect the shared port with a mutex? I ran the program with gdb and a lock, it downloads more suff, but it's also much slower. > Thanks, > Mark Thanks again! -- Amirouche ~ amz3 ~ http://www.hyperdev.fr