From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id ECvUIgu/8WErHwAAgWs5BA (envelope-from ) for ; Wed, 26 Jan 2022 22:37:15 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id 8CtBIAu/8WEmbgAA9RJhRA (envelope-from ) for ; Wed, 26 Jan 2022 22:37:15 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E6338155B6 for ; Wed, 26 Jan 2022 22:37:13 +0100 (CET) Received: from localhost ([::1]:54350 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nCpyb-0001To-2C for larch@yhetil.org; Wed, 26 Jan 2022 16:37:13 -0500 Received: from eggs.gnu.org ([209.51.188.92]:45862) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nCpyQ-0001R8-NW for bug-guix@gnu.org; Wed, 26 Jan 2022 16:37:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:60791) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nCpyQ-00019X-Do for bug-guix@gnu.org; Wed, 26 Jan 2022 16:37:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nCpyQ-00057f-AA for bug-guix@gnu.org; Wed, 26 Jan 2022 16:37:02 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#48007: computing derivations through inferior takes twice as long Resent-From: Ricardo Wurmus Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 26 Jan 2022 21:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48007 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 48007-submit@debbugs.gnu.org id=B48007.164323297619633 (code B ref 48007); Wed, 26 Jan 2022 21:37:02 +0000 Received: (at 48007) by debbugs.gnu.org; 26 Jan 2022 21:36:16 +0000 Received: from localhost ([127.0.0.1]:53694 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCpxg-00056b-2b for submit@debbugs.gnu.org; Wed, 26 Jan 2022 16:36:16 -0500 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21121) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nCpxe-00056T-DS for 48007@debbugs.gnu.org; Wed, 26 Jan 2022 16:36:14 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1643232909; cv=none; d=zohomail.com; s=zohoarc; b=UNlqA2hVzAKsJ+hoItspTwj5aJlpeyC4kpoSo/YJ6qR63ZEVrg6PxXGMXQcx5Rcc2BEUrtwkMgNWzpYmwMYksqPvE3RMrCb3vCixD11yfOuZJ007Ld6RimcTmfoFf9LcFWZnVEj9K8h3qzkJdgzEnZBErfFkS4jc02pSeP73E24= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1643232909; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=v/6Cx9nN7nP6hCEkiEFv/bhOFTawx7yu3S2uu2e5pl0=; b=i+/hAIy6X+qpVunrDfCBC89/3BvlpAdOUuIMlxFCzsXCHCpTHzzYkwt4zrvezEf6Pg7ItAbQ027VznjsV4KN6EUhSz6jTv35A6JW2sIkcYk7bfIgmkm2obHfyEtBobmqG9gk2gUbkWl/Mp1M172RIaKlZoAB/oxNih4fMEf3ytQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1643232909; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:Date:In-reply-to:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=v/6Cx9nN7nP6hCEkiEFv/bhOFTawx7yu3S2uu2e5pl0=; b=adjvWxzb8yKsqbv5254ZY0ayUrYA7kxSRBIpSrtC4bHNItQEBAv1P7QLCEOdh/BF GI/lwFnKZERT+8mrPszWo9mAJh1LKJpFJu5BR+txqwzvO+8dkcvjfSidW4yzha3c3yY EHatw/atz2fiSVrH5AlD83RZSlGTv4B9QKrREKlY= Received: from localhost (p54ad4fa6.dip0.t-ipconnect.de [84.173.79.166]) by mx.zohomail.com with SMTPS id 1643232898242782.8121641946904; Wed, 26 Jan 2022 13:34:58 -0800 (PST) References: <87wnsrpf25.fsf@elephly.net> <87r18ufcft.fsf@gnu.org> User-agent: mu4e 1.6.10; emacs 27.2 From: Ricardo Wurmus Date: Wed, 26 Jan 2022 22:32:02 +0100 In-reply-to: <87r18ufcft.fsf@gnu.org> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Message-ID: <87czkeji4h.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 48007@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1643233035; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=v/6Cx9nN7nP6hCEkiEFv/bhOFTawx7yu3S2uu2e5pl0=; b=PAE7N2S0/De0v/0NWX+Ej2HfHRxAxfXXhIQICN6uKTfsG7Rf66gJOiDKdbRDUKSrlSrf4P YPoV9sM34EZOfpT3QQhJpV4CV+coYTNYq0bcRILhW9GI1EoE0JRl9UtxgYVJXg9TgJj6Px H5Z1mMyGEicG+jL9g3TmOu0mkXSNqOb7YbrZ2X7JbJLo43DNVrlHrL7W6ADdR+dpdQIo1T fhg+9EEK2mCNEfmYLHe1yN/e7AHyVBlR8yuvK7C94gTjvT1TRyewuUFWPvS7a2maYnpAqS 1aC6/mSf4RQuWxfSU3cY5P9ILJfk6+/q3xGUmENrXSJxN/CJnlwXFxGcHNzGtA== ARC-Seal: i=2; s=key1; d=yhetil.org; t=1643233035; a=rsa-sha256; cv=fail; b=ioxBvZBoALk/fjPHHRAtBSWq2G3CApAcoOQVEFUDXig4949LCeYQpYyR3vnFvNUtqC2Yp9 2uK8f1dTDI3ifJBSu1/eUP5+AQyAWPhddP84f5DFnKvr202Nl8SfVZWLHE1AzY7ohUZCYK XDZGOoZEgQ3deJqf+DxhTSsMTMBPM1jI0mFh9eo0in494ptKfD9O1F+9rnNnPNSPxJKDt1 G7AR2GCHybyKOwIEeZyGCgpWyfTXUonvepfzp101xc4IfC9CSubkS91eNnLmmYwkFMVsnR WiM2EITL2aJobcoLAs48SYPTeunj9XB8Y/vFZhuCgMs/gGyiJc4bk/CNaZVkQQ== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=elephly.net header.s=zoho header.b=adjvWxzb; arc=reject ("signature check failed: fail, {[1] = sig:zohomail.com:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.13 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=elephly.net header.s=zoho header.b=adjvWxzb; arc=reject ("signature check failed: fail, {[1] = sig:zohomail.com:reject}"); dmarc=none; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: E6338155B6 X-Spam-Score: -1.13 X-Migadu-Scanner: scn1.migadu.com X-TUID: o29t3Ogsjx9k Ludovic Court=C3=A8s writes: > The trick is to ensure the inferior maintains its object cache across > calls. The patch needs to be cleaned up because it peeks into > internals, but we should be able to do something along these lines and > optimize a couple of other things. Yeah, this makes sense. Excellent! > If you can give it a spin on a more representative example, that=E2=80=99s > great! I tried it in the GWL with the big RNAseq workflow I adopted from PiGx and the step to generate job scripts (which reference inferior packages) became considerably faster. There is still potential for improvement, but this change is the difference between not too bad (15 seconds) and unusable (> 5 minutes). Thank you! --=20 Ricardo