From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id KBsXLfCM8WUtDQAAe85BDQ:P1 (envelope-from ) for ; Wed, 13 Mar 2024 12:24:32 +0100 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id KBsXLfCM8WUtDQAAe85BDQ (envelope-from ) for ; Wed, 13 Mar 2024 12:24:32 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=tugraz.at header.s=mailrelay header.b=qRP2rOd6; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=tugraz.at ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1710329072; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=No9h8oXqJvGXkYo0SSH4CVUL0Ds7xLdlJYSsGwBHfzA=; b=qfL+hzxDM+Ki0MJD5ivMTtMDI2whHHqMCyswYLKvva3yK9vi6WdWyj2YBdwxCM1Buc3QS5 i9BT66l80ksedUhkVVPlDT8DXxWMazRTmDZlzd6nBVsty3ieaNN4E+MFAemR4wWaNW5CEG jcg0l2gdo+TBBDsTxRuwX8Gw6/wrRMVKbNiBf5W971WF3QD55d7NsWCkyqoTtWKQ2s/i17 p3qKrhiZ6tDpW6teEGl1in0BSY9WyRXw35ecSzGvTLIcqtPnOTCa3zxMfkjh9A42aju+5E 8cM//FdzWUx2Rp8idNoW+toArQShi+WfDKdoLsR08ue5/30kA/Ww+akoBV0frg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=tugraz.at header.s=mailrelay header.b=qRP2rOd6; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=tugraz.at ARC-Seal: i=1; s=key1; d=yhetil.org; t=1710329072; a=rsa-sha256; cv=none; b=Jv6sw3DRe9rd666ALOiOevugfwRIQ5Je1eHXON4KlnJ/tOWhZomGJFb+ncOEPN7FrqXbnF 3yjpQu0NgoeyM3Ah9HdUPknXOH8QTak2aZBgaAyFQ+iPleLWSiduvq4Oi+EpaBAwlgFO9X oQ8w13U60oVn21tPVQY5NxzX9SS/h5N1XjYqntDMJBuP4wM73ZmN5PWQQ3j/TZbXZknIKi oiP1VnOQqpyaNyQ4QLx7yXrW+gto3YDBsCcxn0pk7cNdMOpZa0uXJXKDwWfXhS1i2UZxba NC9jefI6G0Q7Es2R2ZBDg6gP6vhHBEpJ234YV2qq0XSEFeO0KHzRRu8600bpfA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 185831D3DA for ; Wed, 13 Mar 2024 12:24:31 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rkMiI-0008Sg-2x; Wed, 13 Mar 2024 07:24:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rkMiE-0008SU-J6 for guix-devel@gnu.org; Wed, 13 Mar 2024 07:23:58 -0400 Received: from mailrelay.tugraz.at ([129.27.2.202]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rkMhx-0005t4-3j for guix-devel@gnu.org; Wed, 13 Mar 2024 07:23:56 -0400 Received: from lprikler-laptop.ist.intra (gw.ist.tugraz.at [129.27.202.101]) by mailrelay.tugraz.at (Postfix) with ESMTPSA id 4Tvp5P0lKvz1LM05 for ; Wed, 13 Mar 2024 12:22:33 +0100 (CET) DKIM-Filter: OpenDKIM Filter v2.11.0 mailrelay.tugraz.at 4Tvp5P0lKvz1LM05 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tugraz.at; s=mailrelay; t=1710328953; bh=No9h8oXqJvGXkYo0SSH4CVUL0Ds7xLdlJYSsGwBHfzA=; h=Subject:From:To:Date:From; b=qRP2rOd6iDt/bdUd2EtwxrDRRpt9BDRjYPHOSuZCxbPp8TaPp1ffi4/EHxkdlFh5o wdYGbUjvo67hg7jfxIZl2SBlTfmsQE7ekbp7MYZxGh5jYqWOvwGa3lNxyVWaceEJMJ 3EJpOAHaChH3AdjayRJdIL1PXdtU1DwfI9Mc2RP8= Message-ID: <2010bdb88116d64da3650b06e58979518b2c7277.camel@ist.tugraz.at> Subject: Processing large amounts of files From: Liliana Marie Prikler To: guix-devel@gnu.org Date: Wed, 13 Mar 2024 12:22:32 +0100 Content-Type: multipart/mixed; boundary="=-9N0t0YZl+ITTkaHFo1CC" User-Agent: Evolution 3.46.4 MIME-Version: 1.0 X-TUG-Backscatter-control: waObeELIUl4ypBWmcn/8wQ X-Scanned-By: MIMEDefang 2.74 on 129.27.10.116 Received-SPF: none client-ip=129.27.2.202; envelope-from=liliana.prikler@ist.tugraz.at; helo=mailrelay.tugraz.at X-Spam_score_int: -19 X-Spam_score: -2.0 X-Spam_bar: -- X-Spam_report: (-2.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -7.59 X-Spam-Score: -7.59 X-Migadu-Queue-Id: 185831D3DA X-TUID: NOe8fg9fWeCL --=-9N0t0YZl+ITTkaHFo1CC Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Guix, I have a somewhat unusual workflow that requires me to do a number of processes on numerous, but small input files. The original is a bit unwieldy and takes several minutes to compile, but I've managed to produce a more understandable and better performing example. Note, that after a certain number of inputs, I get the following error: info: .16 Loading workflow file `meow.gwl'... info: 2.80 Computing workflow `cat'... run: 12.96 Executing: /bin/sh -c /gnu/store/kmssbjcdcabg9fh4nxscwwpnlb4px30= h-gwl-meow.scm =E2=80=A6 error: 13.01 Wrong type argument in position 1: #f For comparison: time cat /tmp/meow/{0..7769} [=E2=80=A6] =20 real 0m0,144s user 0m0,049s sys 0m0,094s It takes GWL 6 times longer to compute the workflow=C2=A0than to create the inputs in Guile, and 600 times longer than to actually execute the shell command. I think there is room for improvement :) Cheers --=-9N0t0YZl+ITTkaHFo1CC Content-Disposition: attachment; filename="meow.gwl" Content-Type: text/plain; name="meow.gwl"; charset="UTF-8" Content-Transfer-Encoding: base64 dXNlLW1vZHVsZXMKICBndWl4IGJ1aWxkIHV0aWxzCgpkZWZpbmUgJWRpcmVjdG9yeSAiL3RtcC9t ZW93Igpta2Rpci1wICVkaXJlY3RvcnkKZGVmaW5lICVmaWxlcwogIG1hcAogICAgbGFtYmRhIDog aQogICAgICBsZXQKICAgICAgICA6IGYgOiBmaWxlICVkaXJlY3RvcnkgLyA6IG51bWJlci0+c3Ry aW5nIGkKICAgICAgICBjYWxsLXdpdGgtb3V0cHV0LWZpbGUgZgogICAgICAgICAgbGFtYmRhIDog cG9ydAogICAgICAgICAgICBkaXNwbGF5ICJtZW93XG4iIHBvcnQKICAgICAgICAuIGYKICAgIGlv dGEgNzc2OQoKcHJvY2VzcyBtZW93CiAgbmFtZSAibWVvdyIKICBpbnB1dHMgJWZpbGVzCiAgb3V0 cHV0cyAibWVvdy5jYXQiCiAgIyB7CiAgICBjYXQge3tpbnB1dHN9fSA+IHt7b3V0cHV0c319CiAg fQoKd29ya2Zsb3cgY2F0CiAgcHJvY2Vzc2VzIG1lb3c= --=-9N0t0YZl+ITTkaHFo1CC--