From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id murzGuQDiGM0DwEAbAwnHQ (envelope-from ) for ; Thu, 01 Dec 2022 02:31:16 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id 2K8mGuQDiGMRJwAA9RJhRA (envelope-from ) for ; Thu, 01 Dec 2022 02:31:16 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 03027155DD for ; Thu, 1 Dec 2022 02:31:15 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p0YPM-0007vz-O7; Wed, 30 Nov 2022 20:30:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p0YPH-0007vP-F7 for guix-devel@gnu.org; Wed, 30 Nov 2022 20:30:33 -0500 Received: from mx1.riseup.net ([198.252.153.129]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p0YPC-0004WY-R6 for guix-devel@gnu.org; Wed, 30 Nov 2022 20:30:29 -0500 Received: from fews1.riseup.net (fews1-pn.riseup.net [10.0.1.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.riseup.net", Issuer "R3" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 4NMz5t63RtzDqP3 for ; Thu, 1 Dec 2022 01:30:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1669858221; bh=sBKS0zYw4GOJCXclWr9+ROc/td6gV377/D5pEbxZIlw=; h=From:To:Subject:Date:From; b=a/6mDW4kk4V56mxY6DkE5hEm2dlNuyst7e/W8uC3iJeybgLttNEmWHXYomVmBWh2t mR5jjOFrDigiuAdTmr3NIA9Umy22CNrfjRi3VJohGM6b36n6XtycUTnGFoZBtz2jlL mspId4piSdVJ7JJFo9kg2MLXaXA0ok1ZpGKfTVYA= X-Riseup-User-ID: 28525F37E96771123B3E910B3BAB7C0439A944D6B389C1712C72BCDF43081960 Received: from [127.0.0.1] (localhost [127.0.0.1]) by fews1.riseup.net (Postfix) with ESMTPSA id 4NMz5t13N8z5vLq for ; Thu, 1 Dec 2022 01:30:09 +0000 (UTC) From: Csepp To: guix-devel Subject: guile profiling / speeding up derivations on slow storage Date: Thu, 01 Dec 2022 02:02:16 +0100 Message-ID: <87fsdzoqqp.fsf@riseup.net> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=198.252.153.129; envelope-from=raingloom@riseup.net; helo=mx1.riseup.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1669858276; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=663UAU8Db8bjqsTYP53y3GNptMqzWfG1kVMurSDrRWQ=; b=iQbowqheQWG+6ip7lmPySeqt+aVPyOZF6ZCKWpEq6Fq3ZZPJb/J5rU8SJMBjY7H401i3zx hEnMOw0EkwKRQmPg3CcOF3Ung5GjAHFVZtp5TSHh17BUsA6OualA6xpff0Pf1ZKeMD4HlK T/YQgX513kYXihiXo3yYesPvmEp+VDGLrMlQUroEhevTlCosgoCI6yJwwGVZQhqW49Jb2D KffTvTU7IUe/TZSbEKJ7FTpH2KqVI3ZRjwmkLMBx2q3VPebQA9ykM+87C+3Yu55CPzBPdI 7EDs4BbCWQPVdVMkrrnmBTiwMI9vXn34CMp0yexv4eoYZVS0mzLaMe4fzl0cag== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1669858276; a=rsa-sha256; cv=none; b=LYKtUhP1EBXAinnCzNSkgzgTfxL+BlUTGNO11eHVSDTygJkz4po2qf6EHaCMFZvLzogGpS SDmtzivujPmnVWy7+f0//Oi8TgFMHK48YWFMcD+dl6B8Wfl0rNCZTthSAMGFXDm6EzscXy rDxmLftSK8EeRCfb732MKDnV9A+SSOiUwe9B3MAplXWuna8EmKum/BTOBb1PGDFhM4PCjP rk/Gt3ssxVMliF5jBqQv8fJVTh3KbfCehwnOx8O8jqlBAHEu9qSCIBFQliCQJrjWJWrIBA ZIqMv/AZkoZwOUIgTlZfKX1dTk/lgSojUJiJPyDtHH7nTcU+ROZK2UbjjHBzpQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=riseup.net header.s=squak header.b="a/6mDW4k"; dmarc=pass (policy=none) header.from=riseup.net; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.56 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=riseup.net header.s=squak header.b="a/6mDW4k"; dmarc=pass (policy=none) header.from=riseup.net; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 03027155DD X-Spam-Score: -2.56 X-Migadu-Scanner: scn1.migadu.com X-TUID: WMor0WSSPSN3 I'd really like to speed up guix pull on my netbook and have been thinking about ways to do so for a while. First of all, I'm curious, how do other Guile developers profile code? Could we add a profiling flag for the CLI similar to the existing debugging flags? On a related note, has anyone tried to write a causal profile for Guile, or a similar Scheme implementation? One thing that definitely needs improvements is memory usage. That should speed things up a lot too, since it would result in less swapping on machines like mine. I would love to hear some tips on doing memory profiling on Guile from someone who has done that. I also looked into porting the ideas from Scalene to Guile, but haven't gotten far since I'm busy with uni and other Guix projects. I also thought about some possible ways of caching expensive computations that are not built using the store. Since Guix is mostly purely functional in practice, I wonder if we could take some notes from languages like Yatima. Or just use a plain old database for some operations that are currently done by Scheme. I know some of this must have already been discussed, but it's pretty hard to track it down, and I think the roadmap for performance improvements really should be centralized somewhere.