From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id UGuWISswU2Z+IQAAe85BDQ:P1 (envelope-from ) for ; Sun, 26 May 2024 14:50:51 +0200 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id UGuWISswU2Z+IQAAe85BDQ (envelope-from ) for ; Sun, 26 May 2024 14:50:51 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=OUTLOOK.ES header.s=selector1 header.b=NyzCSxqU; dmarc=pass (policy=none) header.from=outlook.es; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=2; s=key1; d=yhetil.org; t=1716727851; a=rsa-sha256; cv=pass; b=aWaG1M5Hw03+A1kro5y9lFvRTN0ZJ/bvp/Oq1f5CjlL5woODG3TshKnI+/pe6mxFgE4umK muG6HLzk12a+2JYPFkRtyWRIur9T+/GHUMmXHaLGx80yBGA6bQ0Y7Y/1FN+qdupuPPHbuR Z5ugd+zaym+fStkUi2yO2M7WXB/LThMAoc8aR1bb1qy4iqiqzXTstODsmMhjB0xR6nin6n bjlcf4EbUlSteci+LE8OogxmwmwqErNzR7SunhBfY+2CHH4MmD1sEEZZPbD8gTzC8xsLS5 EmFvtpAU9VvbvyQGDB93Nv7W5pQ+atTUvwOryp89l4T1U0C39KCLC36PxAdRaA== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=OUTLOOK.ES header.s=selector1 header.b=NyzCSxqU; dmarc=pass (policy=none) header.from=outlook.es; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1716727851; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=DG53HDYhjlB80vZ1iexUVqch3G9Ss1EEHaSghdOF9Os=; b=T2HXW+H9ZiOcVOH/ovWgdC8rWlLAAQ4C1OcEr1o0RNmn8tXWBAX/XiCGaONZti/VjULDXg OruIGPHVZ4UPrXzizVENJCVT3c8HNPZIJYdRRAU1YvxfSGb7JgxYLJxDhfOOoWU2shFx3w +zH5gZ2Uf+0snsKCJh8D5JGXt05Ttv5+1gK37zMOLZpkahkcGJe6dTxFMlX6NxSL9xE2Sv SCIdb/fJ5zGkbNzZgeiOFWumNdbiDObxhHR1fPiiyg6ybdOZ9rNd5YSH/vCJqBr06b9Iip CER7tUZ3+Z6WvfVpo7Ol7baeKyIKAyk1WaLMNNSW1yZPInKmSje0gVnDm71osw== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EDD9E1C319 for ; Sun, 26 May 2024 14:50:50 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sBDKA-0003Qz-Eh; Sun, 26 May 2024 08:50:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sBDK8-0003QL-EU for guix-devel@gnu.org; Sun, 26 May 2024 08:50:04 -0400 Received: from mail-dbaeur03olkn20801.outbound.protection.outlook.com ([2a01:111:f403:2e0d::801] helo=EUR03-DBA-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sBDK4-0005w2-Mv for guix-devel@gnu.org; Sun, 26 May 2024 08:50:03 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cFh8H5ZtfnVjZsHbNTCb2yVW6tOep/P1vqN1ARRg6c1kkCo/uLJOPCH9srHNAztTRg1N4WWooNmZpqDwh9M4o6sahOFNg77QnVNOvEXuMFssoU/KEpVq260XbmVzEXF2sKUINfDCxKp4HYPZils1rflyVKHa+5MaPt0nMPB6bTgGaAr8mLsyUSs0ASYSMsFuvcZlD1gdvndAFb+MXecRSNLDOV/5/oMKoG2uRdxbT7LLCoYQ2dY2TnKepIr0Xk1iz8o1Z8DFNEuWhnEJXnsgw3cSm+gMsfunWjAMYqSPNDmamLB38j+7FacMTZMFt6tSEmzQlq1UJM4hjsczeGzAuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DG53HDYhjlB80vZ1iexUVqch3G9Ss1EEHaSghdOF9Os=; b=id0V0JBu8TWSy+uD/1pwWGuFEv5PDZW3U2UACo9FjpnvFLXY8QFlbF16mUSVlixD01BiHmFZmL6BCHYF9n/1Q/4SIK/cVhk+Hmq9A8y7geobjyANnCYM3ftENGMk1UbHurWiOe2DjQWaqAslUhUs2/KOkv1sBFBj8EpiHa9AdNfP9mow6MZf87tB3mljADSEh7bcX4z9D5ZlEx066T0UQN9d3A8z+/AKTKrJOfL+tnvjPemnD8Rm/nWJAmJ9geqx84w9O/Bpk+v5iSKWxqGpG0jyscbN8y1a3PW7wm4VWeA5DynkM0x/OzxgpfQ5PoQ+YBth42A9M4f7izjjHZh4cA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=OUTLOOK.ES; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DG53HDYhjlB80vZ1iexUVqch3G9Ss1EEHaSghdOF9Os=; b=NyzCSxqU+tgbExSHae/CK5d96z09Gvb69OycmrZ4jbHW+M6ee+cDHXnWbSOhTXuRmSVIWStXfrHMdVgUbxabG/4JTil2xn4+V8G9tzO2RVxIuJgoeRSAM7d62H1uUpNiSUkyTrg7rVyA56q9DqOP/Tz8NoFf9dyZr4qiO0VeIhBj9h84TJZGIbAvU2XuobuwhonqHRZGhm1MQNGyC+DD+oJ/b1PemfUgsiLTqdub7ox2jcSGIm6SsQFBltH8DWSCE0mjiZfFaYyJFXRHu3u42QZz3kthN7b64w6qbcK/9Hxir+nJ/Qp7nsZ8iMsAYuJ3TphO1n7WvHWAWKTddIF/Sg== Received: from AS8P251MB0337.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:34f::24) by DB4P251MB0904.EURP251.PROD.OUTLOOK.COM (2603:10a6:10:38f::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.29; Sun, 26 May 2024 12:44:54 +0000 Received: from AS8P251MB0337.EURP251.PROD.OUTLOOK.COM ([fe80::a831:f15c:60cf:27a3]) by AS8P251MB0337.EURP251.PROD.OUTLOOK.COM ([fe80::a831:f15c:60cf:27a3%6]) with mapi id 15.20.7611.016; Sun, 26 May 2024 12:44:54 +0000 From: =?utf-8?Q?Sergio_Pastor_P=C3=A9rez?= To: guix-devel@gnu.org Cc: michal_atlas+gnu@posteo.net Subject: System deployment commands hoarding all RAM Date: Sun, 26 May 2024 14:44:52 +0200 Message-ID: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-TMN: [7X25fO40uxG/qgF3Z85780x0nRkwttM+] X-ClientProxiedBy: MA2P292CA0025.ESPP292.PROD.OUTLOOK.COM (2603:10a6:250::9) To AS8P251MB0337.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:34f::24) X-Microsoft-Original-Message-ID: <84plt891yz.fsf@outlook.es> MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AS8P251MB0337:EE_|DB4P251MB0904:EE_ X-MS-Office365-Filtering-Correlation-Id: 6a3eb314-ee2a-43f2-6606-08dc7d81a499 X-Microsoft-Antispam: BCL:0;ARA:14566002|461199019|440099019|3412199016; X-Microsoft-Antispam-Message-Info: FCZVKTy/Ul7vm6XAFnT/M0vw+yYsh1b+kHkmrXDj/jOWCqeOTf4Xbv+q8AxTSCkfcB2JUZqtN4Vk2lDgMBfDx6/CjcWw/SWoQAuxcLD24Z2TmmUyLuBuDPBjGHQN4eW6r8PcFvtwppw5UNksnfQOmIePdJGF/y/MvklqFbkSI8/Vd5p1ho1Xu538MHFnlq/uJClkLMRO2cP6itHzwzVs16GNj1XusJA2nqLxg0iWZhGOfNa3naMdiSEIWNHg+b9eH5BBF//rf9GnZSLc92IUXDlw5kYn6EBnQdkO1DBJ5THVj18sUUg8kCSjxGxQbrvYOdi72IPtsCYcDbkKwuQyIK1F1CqLr+X2DsLiIho2Urtl10W686E7Qa5aOZ5AiGkW+9+QV3mo3UyM8xL4ijnn/Kp/4m9sr2uwaz3iMgd/h2r7Zx/H9DrTI+oCuUHLDfUTUJadnx0sT+COszziKwX/H/v3V2KWxRdUlbven2fKWABI7aSv1GyryPNRxa+tI1ZecsBkjlrmrVbnNY1ATxATpKDFfbRm8DoJqyOoYmm7VR6isJopyoWdz+3ZuGxYZZs8 X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?b2VHUVdYRWZ3ajZTMG9PUzZCYVBTa0VHYk5udTFoVU5RWlZMMTZGLytKWjNN?= =?utf-8?B?d1p1ZjByS2R0RHl2dVZZdUFXUWcyellOTTB0VGVPR0hBaElxM0JkYVlEc2V4?= =?utf-8?B?YlRFTG5uOXl1R1RsL1J5Z2c3NVR3c3dFTDFsRUEzdlY5UTV4eEdiaEl4TmlR?= =?utf-8?B?TFpzSHpjSXF4NFpMSVpJUExDZEhybjlkc2lEZU1tTDJQQnZQdnV6T1lJSCtu?= =?utf-8?B?dUpqWGNKOVhzQXg1dkRYci90dnU3VkJYTzF1NldHcHVsSndCYUYwMXZlQkxG?= =?utf-8?B?OFlqN05EZ083TnRtTjhkM1Y3dENsTVFlNDhsQXVZUXY1MjcrVS85Zzhncmtm?= =?utf-8?B?Wll6ZEpaQzE3WnNQT1JDVW85SitCRFBNV1JURHVWd1czZTZ2TjZzQUwySUJp?= =?utf-8?B?UXNJZk5CKzA0NHpUZjNab0dITWd2cFRsSmNCSTZLNzlJN2lRa1lsM0YxTWda?= =?utf-8?B?RnVGeGlwYXpvOU8vdDI5dEtQL1cwdWNGSGlDaVJPaVpMYlllZnBzT3VMRXpG?= =?utf-8?B?eHJpWkI0a0Vya1VpaGZMTVZWcVo2blVnb1V1dFkzc2EwU3oxeENwV3AzYkZa?= =?utf-8?B?TzBHMHVTWllnMERndHpqTDF4MEtPc0hzZnFmZlVrRis3b1RlQ2N2N0NpRFhp?= =?utf-8?B?ZWV2bU5zNW10OElrRHdzYmtOVlBZYzgwN2pTSFFtRVoyTVVLaEQ4aTZqejlr?= =?utf-8?B?MEhOT1lhVjc0d3lEeTE1blhrcHFNYlRyZWFyaGlMdlJxRGlNM2NEZExRcTJm?= =?utf-8?B?QXMwKy9JZThuSkJRWlREcFhVZk16bkxiSFJaem9CMnY1Ty9HZWZQQUk0OXNT?= =?utf-8?B?VFFCQk1HU20xdnF0NGIzdWxZb1diTjJJS2hRUUE3TGxtcHQ5N1FpeFZNYXd5?= =?utf-8?B?dHBmVWo3TlVjWGJzclJ3Y1FZcUZ0K0R1a3k3OFN5THVNbkY3cUVEYjNlNWp0?= =?utf-8?B?S1FGQ3JEdEs1VGMzelVLV1NYRDZYOHNLL0JxNFBDRXZtSFVUd240L2dqTFp6?= =?utf-8?B?Rm50UGJsbXd6Njg0QzB4OFYyNmpLU1lJTVpsT1NaRHJGcU9oQXd6RW1PM0lr?= =?utf-8?B?TUVmWDE1cWRFMlNMK0JLelN0ODk3VUl6MGFxbTBFQUhrb3AwZDUxdXd4dy9n?= =?utf-8?B?V2Z1YnBaTXJqZDcyLzArMjVPV3QwQ3RUNW9vN3ZTQklKenYzYXpzZmh2QXZP?= =?utf-8?B?WTV6NUlXaXUwTXlsY1VlQ3l1ekRTeHJEQUFaNzliVWNKOElua0pZQXpUWUZE?= =?utf-8?B?dEJLM2hPTjVSQnlVdWUxYUNTaDBGUTgyRXhTN3M3NHVHdWlabU9KVlROWkNu?= =?utf-8?B?SHdIQjlORGZwWE9QN0p6NzM3TUxnakRmamVXVWw2ZDVCY1FlNkxWVHFjMnp6?= =?utf-8?B?d1VyMm5Icm5UbTFhTFh0bVlJQXRwZzR0TTdRSlU4RSt3K080c1B5MWVqV1lk?= =?utf-8?B?cWdsVmlCRzE2bHA3ZFg5d1NDVllJWHN0aUwxenBKNU9qQjF4eWYweDlQTUNZ?= =?utf-8?B?bHBBZHMzSzBaTnhyWGlGaWg1ZGdaZ3JzaFpPRitsRHJXblRkaGVUdm5iWlJM?= =?utf-8?B?QjRHNlVicEsrTUxxOHhhd0I1N1hOWW8wckV2YXpNQjdWN1g4KzdiZzRxRVE3?= =?utf-8?B?SzkrRThiSENCTkcwcW9nK1ByT1dSUDhGYVd6L0pMQzNVcXNjS1VrU1JIV0ky?= =?utf-8?Q?lqYfqd9pL19iZ9zZR6d1?= X-OriginatorOrg: sct-15-20-4755-11-msonline-outlook-00b75.templateTenant X-MS-Exchange-CrossTenant-Network-Message-Id: 6a3eb314-ee2a-43f2-6606-08dc7d81a499 X-MS-Exchange-CrossTenant-AuthSource: AS8P251MB0337.EURP251.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 May 2024 12:44:54.7806 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4P251MB0904 Received-SPF: pass client-ip=2a01:111:f403:2e0d::801; envelope-from=sergio.pastorperez@outlook.es; helo=EUR03-DBA-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Queue-Id: EDD9E1C319 X-Migadu-Scanner: mx12.migadu.com X-Migadu-Spam-Score: -10.85 X-Spam-Score: -10.85 X-TUID: hZp9HWm/MNcM Hello all. Michal and I have been experimenting with a recursive function to apply procedures to all elements that match a certain type within a collection. With this new procedure one could write something like this: --8<---------------cut here---------------start------------->8--- (apply-to-record-type (lambda (_) (@ (gnu packages games) cowsay)) (@ (guix packages) ) my-operating-system) --8<---------------cut here---------------end--------------->8--- Which will replace every package on the system for the cowsay package. Albeit a cowsified OS will not be very useful, other kinds of transformations could greatly simplify OS definitions. It would make it very convenient to deploy systems with grafts on important libraries that are part of many packages. A more useful example would be this syntax rule: --8<---------------cut here---------------start------------->8--- (define-syntax-rule (custom-libc-operating-system exp ...) "Like 'operating-system' but graft 'libc' with the a custom 'libc' package." (apply-to-record-type replace-libc (@ (guix packages) ) (operating-system exp ...))) --8<---------------cut here---------------end--------------->8--- Which, if `replace-libc` grafts your custom libc using `package-input-rewriting`, would allow you to define a system like so: --8<---------------cut here---------------start------------->8--- (custom-libc-operating-system ...) --8<---------------cut here---------------end--------------->8--- Instead of having to manually apply the function to every package in the system which is very cumbersome to do for packages within services. The function that provides this convenience is this one: --8<---------------cut here---------------start------------->8--- (define (apply-to-record-type fn type var) "Recursing into child fields, apply FN to every element of VAR which hold= s a value of TYPE." (let ((type-predicate (record-predicate type))) (cond ((type-predicate var) (fn var)) ((procedure? var) (lambda args (apply values (map (lambda (var) (apply-to-record-type fn type var)) (call-with-values (lambda () (apply var args)) list))))) ((list? var) (map (lambda (elt) (apply-to-record-type fn type elt)) var)) ((vector? var) (vector-map (lambda (vec elt) (apply-to-record-type fn type elt)) var)) ((and (record? var) (not (eq? (@ (gnu services) ) (record-type-descriptor var)))) (let* ((record-type (record-type-descriptor var)) (record-fields (record-type-fields record-type))) (apply (record-constructor record-type) (map (lambda (field) (let* ((accessor (record-accessor record-type field)) (val (accessor var))) (apply-to-record-type fn type val))) record-fields)))) (else var)))) --8<---------------cut here---------------end--------------->8--- During our testing, we have observed that it successfully applies the function to every element of the record and it's child. Also, innocuous transformations output the same system derivation. For example: --8<---------------cut here---------------start------------->8--- (apply-to-record-type (lambda (var) var) (@ (guix packages) ) my-operating-system) --8<---------------cut here---------------end--------------->8--- Should return the same OS applied or not, since it is not doing anything. And, as expected, we see: --8<---------------cut here---------------start------------->8--- =CE=BB guix system build sheepbook.scm /gnu/store/kpk9la4h9xwp6n7vabd5lfs6kbhb2f2d-system =CE=BB guix system build transformed-sheepbook.scm /gnu/store/kpk9la4h9xwp6n7vabd5lfs6kbhb2f2d-system --8<---------------cut here---------------end--------------->8--- All this is well and good but we have noticed this issue when trying to deploy the system. Issuing the `reconfigure` or `vm` subcomands hoards all the RAM of the system until the OS kills the process. This is the output when trying to build the images: --8<---------------cut here---------------start------------->8--- =CE=BB guix system vm sheepbook.scm /gnu/store/c3pv8hwckbl01qacdpckn9yfwr74k629-run-vm.sh =CE=BB guix system vm transformed-sheepbook.scm=20 Killed --8<---------------cut here---------------end--------------->8--- Monitoring the RAM usage of the second process, one can see that it will take all the RAM available. Given that both system definitions produce the same output, I would expect the deployment process to be identical. Is this a bug on how Guix deploys the system? Or the transformation we are applying introduces a cycle somewhere that only affects the deployment commands? Thanks for your time. Have a nice evening. Sergio.