From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Amirouche Boubekki Newsgroups: gmane.lisp.guile.devel Subject: (n-for-each-par-map 16 store download (reverse (iota (max-id))))) crash Date: Thu, 05 Jul 2018 13:28:58 +0200 Message-ID: <567d1b31558c2b51908d8a272ff61494@hypermove.net> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Trace: blaine.gmane.org 1530790044 13118 195.159.176.226 (5 Jul 2018 11:27:24 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Thu, 5 Jul 2018 11:27:24 +0000 (UTC) User-Agent: Roundcube Webmail/1.1.2 To: Guile Devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Thu Jul 05 13:27:20 2018 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fb2Ps-0003Cm-Ui for guile-devel@m.gmane.org; Thu, 05 Jul 2018 13:27:17 +0200 Original-Received: from localhost ([::1]:51906 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fb2S0-00083M-4r for guile-devel@m.gmane.org; Thu, 05 Jul 2018 07:29:28 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:43108) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fb2Ri-00081o-IW for guile-devel@gnu.org; Thu, 05 Jul 2018 07:29:11 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fb2Re-0000mD-WE for guile-devel@gnu.org; Thu, 05 Jul 2018 07:29:10 -0400 Original-Received: from relay7-d.mail.gandi.net ([217.70.183.200]:56791) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fb2Re-0000OU-Iz for guile-devel@gnu.org; Thu, 05 Jul 2018 07:29:06 -0400 Original-Received: from webmail.gandi.net (unknown [10.200.201.1]) (Authenticated sender: amirouche@hypermove.net) by relay7-d.mail.gandi.net (Postfix) with ESMTPA id 6D2A820006 for ; Thu, 5 Jul 2018 11:28:58 +0000 (UTC) X-Sender: amirouche@hypermove.net X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 217.70.183.200 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19583 Archived-At: I have a program that try to download hackernews locally. What it does is simple, it fetch the max identifier and http-get each json value starting with the most recent item. I use n-for-each-par-map with 16 threads I have 8 cores. Here is the full program: (define-module (hn)) (use-modules (srfi srfi-1)) (use-modules (ice-9 receive)) (use-modules (ice-9 threads)) (use-modules (ice-9 iconv)) (use-modules (web client)) (use-modules (json)) (define (max-id) (receive (response body) (http-get "https://hacker-news.firebaseio.com/v0/maxitem.json") (string->number (bytevector->string body "utf-8")))) (define (download uid) (catch #t (lambda () (let* ((uid (1+ uid)) (url "https://hacker-news.firebaseio.com/v0/item/~a.json") (url (format #f url uid))) (cons uid (json-string->scm (call-with-values (lambda () (http-get url)) (lambda (response body) (bytevector->string body "utf-8"))))))) (lambda _ '()))) (define (store pair) (if (null? pair) (format #t "X\n") (let ((port (open-file "hn.scm" "a"))) (format #t "~a\n" (car pair)) (write (cdr pair) port) (close port)))) (define (dump) (n-for-each-par-map 16 store download (reverse (iota (max-id))))) (dump) It also requires json module from https://raw.githubusercontent.com/a-guile-mind/Culturia/master/src/json.scm How can I debug this? -- Amirouche ~ amz3 ~ http://www.hyperdev.fr