From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Chris Vine Newsgroups: gmane.lisp.guile.devel Subject: Re: Subprocess API. Date: Mon, 25 Sep 2017 19:59:39 +0100 Message-ID: <20170925195939.01638900@dell.homenet> References: <87a81llos5.fsf@gmail.com> <20170923151937.5bb7d80e@bother.homenet> <874lrqlmz5.fsf@gmail.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Trace: blaine.gmane.org 1506366004 931 195.159.176.226 (25 Sep 2017 19:00:04 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 25 Sep 2017 19:00:04 +0000 (UTC) To: guile-devel@gnu.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Mon Sep 25 20:59:54 2017 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dwYbf-0007oz-A0 for guile-devel@m.gmane.org; Mon, 25 Sep 2017 20:59:51 +0200 Original-Received: from localhost ([::1]:43860 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dwYbk-0007EI-VJ for guile-devel@m.gmane.org; Mon, 25 Sep 2017 14:59:56 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:35319) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dwYbd-00071A-3B for guile-devel@gnu.org; Mon, 25 Sep 2017 14:59:50 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dwYba-0002VD-12 for guile-devel@gnu.org; Mon, 25 Sep 2017 14:59:49 -0400 Original-Received: from mail-wm0-x230.google.com ([2a00:1450:400c:c09::230]:47192) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dwYbZ-0002Su-QE for guile-devel@gnu.org; Mon, 25 Sep 2017 14:59:45 -0400 Original-Received: by mail-wm0-x230.google.com with SMTP id r136so549262wmf.2 for ; Mon, 25 Sep 2017 11:59:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4p4g9fQn73ZWF5JQNTD6pxSSUrQYmfDh6pbatRL8af0=; b=kmwAEdw0+K2Ai5C8cl+pBQsSmrxAszl2hyOYMdo44GBKUWM9EunmB+9b/W7lsq5fqH hrZwCtgzejnEqKePJLJ2n22/KSRKn1jjSObEZbgYOYFA1kTdEt2fEuL+xTj3bv65LMgB uQEdTcosWw9XEH7QTWwRhl9wrvSODsAE7slbq769oPihMUufbrr4/RuwsZDAceKas99E 9UMGAWaMHDC75AIojdzYDA/Q6rQVDTAovw6W638KgnmvW/Y5h7oo74JaHE26AlALF5YO X34H4aZVWXSR49M5O46jZEGiFVK7CmJ3GJZ2EilaukNpLzIZ9FdTvEc4MMhoaND1NaDx 8izA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4p4g9fQn73ZWF5JQNTD6pxSSUrQYmfDh6pbatRL8af0=; b=brp9rAI/HqTRC2U+m/S5fGNBS5PCE4Icx8AIgM+EwI+I4VcQxZ7e6oAhuL778HEFHz SrfEWKBvBb00IE7/ssr2m8ivm+5KO8Mfepsdpb3OSK2QPci08mxjwluknCtC11xQrdUC Wd+5rVZfSEDf2Ci4JYbDNs1IPH3X1iGpqVPAlq/zUeCH2P5L2iTHKuAtN5HxUFz3zxxG zwKY8RiT/sX5gKjClvY+MrHFWcALB1ra53XxGTKVJ+5A9SxqL8j6ZRX2TnGuJwlpPlb4 V7WsZtR/nnx8TGW22wejYqbR2t4aofxUOF5EvsvYosyoB1tdXJJL336HePBtpKs+pynY O0aA== X-Gm-Message-State: AHPjjUjjNtD3wKnOSkXETnpyEaj+yHwsHcPyaqfX5dHJlx3pVZ/L4i92 kq/d8RX+R4klIt1Wqo6flA+pRw== X-Google-Smtp-Source: AOwi7QDV0c6iunKHkRnLKp+tvZAFU3GOKCBLLAfjRquf+ayLcQaAvTub3O48tVFhexGo6zRuxSFmPw== X-Received: by 10.28.30.84 with SMTP id e81mr1249417wme.39.1506365982696; Mon, 25 Sep 2017 11:59:42 -0700 (PDT) Original-Received: from dell.homenet (102.201.115.87.dyn.plus.net. [87.115.201.102]) by smtp.gmail.com with ESMTPSA id e34sm11557103wre.15.2017.09.25.11.59.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 25 Sep 2017 11:59:41 -0700 (PDT) Original-Received: from dell.homenet (localhost [127.0.0.1]) by dell.homenet (Postfix) with ESMTP id B1177442F54 for ; Mon, 25 Sep 2017 19:59:39 +0100 (BST) In-Reply-To: <874lrqlmz5.fsf@gmail.com> X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.31; x86_64-unknown-linux-gnu) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::230 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19314 Archived-At: On Mon, 25 Sep 2017 19:14:22 +0200 Mathieu Othacehe wrote: > Hi Chris, > > > This works exactly as you would expect from its POSIX equivalents > > and has the advantage that you can read from the pipe as the > > sub-process is proceeding rather than just collect at the end. > > Thank you ! Following your suggestion, I ended-up with : > > --8<---------------cut here---------------start------------->8--- > (let* ((err-pipe (pipe)) > (out-pipe (pipe)) > (read-out (car out-pipe)) > (write-out (cdr out-pipe)) > (read-err (car err-pipe)) > (write-err (cdr err-pipe)) > (pid (run-concurrently+ > (apply tail-call-program "...") > (write-out 1) > (write-err 2))) > (ret (status:exit-val (cdr (waitpid pid))))) > (close-port write-out) > (close-port write-err) > (let ((output (read-string read-out)) > (error (read-string read-err))) > (close-port read-out) > (close-port read-errs > (case ret > ((0) output) > (else (raise ...))))) > --8<---------------cut here---------------end--------------->8--- > > which seems to work. However, run-concurrently+ uses "primitive-fork" > which is forbiden in a multi-thread context (sadly, mine). > > Do you have any idea on how to overcome this ? Any launching of a new process requires a fork and if (as appears to be your intention) you want to replace the process image with a new one, an exec. As you appear to know, POSIX allows only async-signal-safe functions to be called in a multi-threaded program between the fork and the exec, although glibc does relax this somewhat. Since anything you do in guile between the fork and the exec has the potential to allocate memory, that appears to mean that, as you say, you cannot call primitive-fork in a guile program at a time when it is running more than one thread. If so, I do not know how to circumvent that: you could consider launching the new process in C code via the guile FFI so you can ensure that no non-async-signal-safe code is called at the wrong time; but presumably you would still have by some means to prevent the garbage collector from being able to start a memory reclaiming run in the new process after the fork and before the exec, and again I do not know how you would do that. You would also need to block system asyncs before forking (and unblock after the fork in the original process) but that is trivial to do. As regards your code, if you do not need to distinguish between stdout and stderr, you would do better to have only one pipe and use the write port of the pipe for both of descriptors 1 and 2. That means that you could read the pipe while the new process is proceeding rather than after it has finished (which risks filling up the pipe): just loop reading the read end of the pipe until an eof-object is received, and then call waitpid after that. Chris