From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id buwSHpr6VGFhTQAAgWs5BA (envelope-from ) for ; Thu, 30 Sep 2021 01:45:30 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id qAQ1GZr6VGG+eQAAB5/wlQ (envelope-from ) for ; Wed, 29 Sep 2021 23:45:30 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0529AECAE for ; Thu, 30 Sep 2021 01:45:30 +0200 (CEST) Received: from localhost ([::1]:35736 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mVjGT-0002pF-5j for larch@yhetil.org; Wed, 29 Sep 2021 19:45:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56640) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mVjG2-0002oZ-KY for guix-patches@gnu.org; Wed, 29 Sep 2021 19:45:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:39431) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mVjG1-00009q-WF for guix-patches@gnu.org; Wed, 29 Sep 2021 19:45:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mVjG1-00031j-T3 for guix-patches@gnu.org; Wed, 29 Sep 2021 19:45:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50620] [PATCH 0/2] Unify 'computed-origin-method' (linux, icecat) Resent-From: Liliana Marie Prikler Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 29 Sep 2021 23:45:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50620 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: Mark H Weaver , 50620@debbugs.gnu.org, zimoun Received: via spool by 50620-submit@debbugs.gnu.org id=B50620.163295908811575 (code B ref 50620); Wed, 29 Sep 2021 23:45:01 +0000 Received: (at 50620) by debbugs.gnu.org; 29 Sep 2021 23:44:48 +0000 Received: from localhost ([127.0.0.1]:50974 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mVjFn-00030d-Py for submit@debbugs.gnu.org; Wed, 29 Sep 2021 19:44:48 -0400 Received: from mail-wm1-f65.google.com ([209.85.128.65]:55935) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mVjFk-00030O-Mm for 50620@debbugs.gnu.org; Wed, 29 Sep 2021 19:44:46 -0400 Received: by mail-wm1-f65.google.com with SMTP id v127so3108751wme.5 for <50620@debbugs.gnu.org>; Wed, 29 Sep 2021 16:44:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=p+WnMKZprEEhpm/yNCatzO1NHCnMqxtScv2TIbL5Atg=; b=UJvy4NcEImgPGVQuvrBi4851qpEl3FAOCDq7c7rwqfIyoYcuel79pMvTW80jIgVqch Guw4aBvwNITFmZJpIlcqAtwJNHK1wOBVpT2Y5dlkuG11mKwn2w5Dfl6eHJ7IWtSd/wYu Uh61S3JGCqc8QdLApsBPhJa+4h6G1xkLFeb3v8Y4r13XNgwCN2Q0Z4S0s80ITOlJgOCL QLAfCPfPV9dnry56+s94WLPzuQxBuLmbQqSlqBOmv52hjDJwuP6CD0H73oH0dzDRMcQD VCcf6b19ldeG/EO/id3A5bEueqENqdovMghoV4ZB1mDLVfLGIGUR6G8+gwzXiHbqH4xr KCjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=p+WnMKZprEEhpm/yNCatzO1NHCnMqxtScv2TIbL5Atg=; b=P3ZK5Z159e+nMs32104EUXTYBV5oUGcrDxIo77NDDhkq7yhHJwtfdO4VqtLoIgaX11 3IrIAISzUJiUex0gBWwKXgxH1CKBTCw7PVaNS38tiLfhhQOUP3TmU/r/Yr0o0W49+Wij DbhVU20A0XPQ6JtsH129qHOpYOp90qvf0UmyRS5rNE1sJCFwfZC0Qhh2GGWSI6GNEnQN rYhponjMh+2iCBsr3FRCC2BwLEzdH8kalEa5ojVzlM2XdFjb2TCsDCNunOZHJQ44H3eO tMmFTdehuibhhh/PYLYO/obyFSlZ5mppJOnZJ49Z+wuPbO+JpHT2C0ep0Pi3EwvhbDky NJyQ== X-Gm-Message-State: AOAM531LDu4hzW/FBZJEamaNi/2KhCIi8jmqHBk7cf8UqnvUR3/BUORI 1aFoIPQT37/2ebshBOREEX4= X-Google-Smtp-Source: ABdhPJwLeco4aUNb9Kno78LT9Zs4HbAbh+V4ngvmdGiPSDz3RWziO8xoEwX0yPK2sbIkb4qIm4/Z6A== X-Received: by 2002:a7b:ce0e:: with SMTP id m14mr2435878wmc.109.1632959078789; Wed, 29 Sep 2021 16:44:38 -0700 (PDT) Received: from nijino.fritz.box (85-127-52-93.dsl.dynamic.surfer.at. [85.127.52.93]) by smtp.gmail.com with ESMTPSA id l6sm1186168wrm.64.2021.09.29.16.44.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Sep 2021 16:44:38 -0700 (PDT) Message-ID: <09c009ec61b8f9c1746f98916f492b8953b16dcc.camel@gmail.com> From: Liliana Marie Prikler Date: Thu, 30 Sep 2021 01:44:37 +0200 In-Reply-To: <87pmsr5a0z.fsf_-_@gnu.org> References: <20210916114734.2686426-1-zimon.toutoune@gmail.com> <9b6ee27ff10e1042a5d61d0f93d957cf760e9ecb.camel@gmail.com> <87v930ay5y.fsf@netris.org> <87pmstghx0.fsf@netris.org> <87a6jv8qu7.fsf_-_@gnu.org> <87pmsr5a0z.fsf_-_@gnu.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.34.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1632959130; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=p+WnMKZprEEhpm/yNCatzO1NHCnMqxtScv2TIbL5Atg=; b=Tlp0bRF4Fn2c/zf6nnw5sE+5OhnnVjetbApLwIlksR1kDoo2Pk1g3WUjW7OTm/gMDYzniA BBJKRHRXP9pbq130581a9uZS5VnV2ZEL9kMiFzIUl0w9iybZreyUAB3owpPfbuX2wrK8og /hraIbOfsQmg0l7KbM9B4pmcdx5qdsI08ERu2JlgkHKm051qXgdqD2fCjAS86cd3cikr6V obZzFXbRRyZ4KP7ioDQ088o1EqufX8HPdfRjXBHIVvO+Hm0xblPdcgxQuPEZ1OPwUKhCYx TkY3Hr/zqv2rxWRemFsfPLTQYDClH/cfv2jaFIURviVsY4l8HhAIbItxU3xMDA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1632959130; a=rsa-sha256; cv=none; b=fRFDS2fCu0CA+Gn/eHNbGTdj3tHew6EjnEi9BRboYUYji0o0zdfWfmV6rU9RvRlBPCuUCd rUNj60g9eYlbKmmb3eaE3XZ0B82xGW8bd1OkxqHateeUlwff/wx2tlFGFKRIX2vetI9dDv Dy2k6Mb42zLe4Wq+NjUDgXfbvjt+zyLTcUNplCmyYPHJQhDaWPXOxRL7cERGhsKaMHafrR go0Ex5nrRUXN6QFG/2ZOOXMkgA1bj1Y9K9eizwGvo4RWPHoRgrZDLzCvpIwU+bfsDuXDs+ XLMADyOXodfaPJUHTe27vlzoeGqdnEHPMkosSJcEATL9jmWifZY1awBw2C649A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=UJvy4NcE; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -0.20 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=UJvy4NcE; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 0529AECAE X-Spam-Score: -0.20 X-Migadu-Scanner: scn1.migadu.com X-TUID: zKuIqTDiBZ05 Hi Ludo, Am Mittwoch, den 29.09.2021, 23:47 +0200 schrieb Ludovic Courtès: > [...] > > > but zimoun and I still disagree on the target. zimoun says (guix > > packages) for reasons unknown to me, whereas I say (gnu packages), > > because it's closer to where it's used and doesn't imply that this > > is > > going to be a part of the (guix) download schemes anytime soon. > > (gnu packages) is higher-level: it’s part of the distro and includes > CLI helpers such as ‘specification->package’. So I think (guix …) is > somewhat more appropriate. > > (That said, what matters more to me is how we’re going to replace it > with a proper solution.) (gnu packages) being high-level is part of the reason I want it there. Stuff that's hidden quite deep inside (guix something) will be slower to change and replace with the proper solution. When you pull on a lever, the outside moves faster :) > > > A better solution IMO would be to improve the ‘snippet’ mechanism > > > in the first place. ‘computed-origin-method’ improves on it in > > > two ways: (1) lazy evaluation of the gexp, and (2) allows the use > > > of a different base name. > > > > > > I would think #2 is addressed by the ‘file-name’ field (isn’t > > > it?). > > > > > > As for #1, it can be addressed by making the ‘snippet’ field > > > delayed or thunked. It’s a one line change; the only thing we > > > need is to measure, or attempt to measure, the impact it has on > > > module load time. > > > > > > Thoughts? > > This would work for packages, whose source are some base source > > with patches or snippets applied, as is indeed the case for linux > > and icecat. However, there are also other potential uses for > > computed origins. > > It’s hard for me to talk about potential uses in the abstract. :-) > > There might be cases where an origin simply isn’t the right tool and > one would prefer ‘computed-file’ or something else. It really > depends on the context. > > [...] > > > I think that some version of `computed-origin-method' will > > eventually need to become public API as such packages may not > > always be best described as "a base package with a snippet". If we > > had recursive origins – i.e. origins, that can take origins as > > inputs – we might be able to do some of that, but I don't think it > > would necessarily work for linux-libre or icecat, as with those you > > don't want the tainted versions to be kept around. Perhaps this > > could be worked around by not interning the intermediate origins, > > but only using their file-names inside the temporary directory in > > which the snippet is applied? > > “Recursive origins” are a bit of a stretch as a concept IMO; what you > describe is a case where I’d probably use ‘computed-file’ instead. In other words, we could/should use computed-file for linux-libre and icecat? If we reasonably can, would it make sense to use that in lieu of computed-origin-method to actually advertise the existence of computed-file to Guix users/packagers? > > Another thing is that the final act of the linux-libre promise is > > not the packing of the tarball, but the deblob-check. Guix > > currently lacks a way of modeling such checks in their origin, but > > I'd argue it would need one if we wanted to do computed origins via > > snippets. This is not required by icecat and so one > > "simplification" could be that computed-origin-method would not > > require the user to create a tarball, but instead simply provide a > > name for the tarball and a directory to create it from (via a > > promise again). > > Ah, I had overlooked that ‘deblob-check’ bit. It could be that > allowing for custom pack-and-repack procedures would be enough to > address it. I think asking users to supply their own implementation of a 200 line long function to be a bit much to only do part of the job. On the other hand, the promise for linux-libre takes 400 lines and for icecat more than 600, but I think there are some things we ought to factor out. Particularly, looking up tools like tar or gzip and even the actual packing are always the same. What we can't currently control is the top directory name and the output name. Both of that could be customized by supplying a "repack- name" field, which is used as basis for the directory name and the tarball name. Another thing we can't easily control are extraneous inputs to the patches, although the patch-inputs field *does* exist. > > A combination of the above might make computed origins obsolete for > > good, but the question remains whether that is a better > > design. What do y'all think? > > The design goal is to have clearly identified types: , > , . For each of these, we want some > flexibility: build system, origin method, etc. However, beyond some > level of stretching, it may be clearer to just use the catch-all > ‘computed-file’ or to devise a new type. After all, that’s how > came to be (we could have used instead with a > suitable build system). > > There’s a tension between “purely declarative” and “flexible”, and > it’s about striking a balance, subjectively. To be fair, I did think that "computed-tarball" might be a good abstraction in some sense, but on another hand origins are computed tarballs with a record interface. On a somewhat related note, origins have this weird situation going on where some things like git or svn checkouts need to be defined through them, whereas others may pass unhindered. I feel that this contributes to the equation of source = origin, that might have caused "computed- origin-method" to exist in the first place. What do you think? Liliana