From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id sJMQGu/D5GARzwAAgWs5BA (envelope-from ) for ; Tue, 06 Jul 2021 22:58:23 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id WFubFe/D5GBtVAAAbx9fmQ (envelope-from ) for ; Tue, 06 Jul 2021 20:58:23 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BAA009F88 for ; Tue, 6 Jul 2021 22:58:22 +0200 (CEST) Received: from localhost ([::1]:52248 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1m0s97-0000EL-0A for larch@yhetil.org; Tue, 06 Jul 2021 16:58:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53786) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1m0poV-0004d4-10 for guix-devel@gnu.org; Tue, 06 Jul 2021 14:28:55 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]:34460) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1m0poR-0000Hk-Id for guix-devel@gnu.org; Tue, 06 Jul 2021 14:28:54 -0400 Received: by mail-pf1-x42e.google.com with SMTP id f20so12378931pfa.1 for ; Tue, 06 Jul 2021 11:28:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vieiraproenca-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=SCupDkrBr6VvkIP9z2ZG/ZaVm7qkKQi+cPZHCil1P24=; b=RCzwl7tibVB+MiK65h1XAzwlI6bu1OApRFTrMRFM0rh+Obb8ykcIKuqAsOowLwXGmO qUnJtHBDoiumcEQ0alLFFwNFAURja+IHdgQI8Y9DKHdnx7H1bWc55EkOajwApc0KPC6P 7Tnalo42flnj658Gfj9A+06yLuZbcqDr6+WfjmeYabZPIk5ULJjuu+dPHtcn7aMRrLKv Yh76C95lsQnA5eTsbgG50yYowVzJ96Iq1hyGbV7N+CkXdnF6bFrPPAWJNkPCzz+LD8Gj EO7Qlutcp6fKTPqM4Q1rP4RW7GPXqsoeZ2UgrtAFn04vlfCHjk9hNLuOj+lOXC10PQBy 67Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=SCupDkrBr6VvkIP9z2ZG/ZaVm7qkKQi+cPZHCil1P24=; b=EhdPvnme5hX92lOQ2U2JRoybG2x2b+tOGFYTsrckTsdgjzaC05EF2zIQ/R34CSdbFh PAsbDauJv41KGV+ocfMpDcuCyhPYzb5DBAP28K3lhgxMmFDcyEAr2LosS+roE9vLUGQN zzN3ZLWWOzej3h9Iyj+/cUTV/7w/i9wzGPs3Ihk1NN51AZYeZi6CqqiLn0pdRdlm3xeR 23bB2If9ObgKfPoBNwMrXzKH2S1jnvvkVAd57bwsjciJU3YPkOpyOSmxuCGma9vtYANW 3AplMHQvtLfhQkvI8ijHsNUptIXbQqJnENU80LHUybFva95av7Bi1UY47hjA0JnE4tmL cTSQ== X-Gm-Message-State: AOAM531FG06FKbr1dIXSKUgvP1/muNdIxVDWSaHc6SOdIDRZAvmme1/M VYsi4MHNv4Mw/3K3e1+Aohat4l0uZEI5We/S X-Google-Smtp-Source: ABdhPJwJHmOE5GSpKvRHsaXsOqLMfO2d3l4HEhFEnGUneUfKozWJrbKPFH2Rjl1NC+H/g95NukELkA== X-Received: by 2002:a62:7dd1:0:b029:314:8a36:561 with SMTP id y200-20020a627dd10000b02903148a360561mr21555041pfc.18.1625596127890; Tue, 06 Jul 2021 11:28:47 -0700 (PDT) Received: from archlinux ([177.62.236.234]) by smtp.gmail.com with ESMTPSA id l12sm17117337pff.105.2021.07.06.11.28.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Jul 2021 11:28:47 -0700 (PDT) From: =?utf-8?Q?Nathan_Benedetto_Proen=C3=A7a?= To: Bengt Richter Subject: Re: Use of %texlive-revision and %texlive-tag in tex.scm In-Reply-To: <20210706094306.GA3432@LionPure> References: <87o8bgoo1p.fsf@archlinux.i-did-not-set--mail-host-address--so-tickle-me> <20210706094306.GA3432@LionPure> Date: Tue, 06 Jul 2021 15:28:43 -0300 Message-ID: <87o8bfi9es.fsf@archlinux.i-did-not-set--mail-host-address--so-tickle-me> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: neutral client-ip=2607:f8b0:4864:20::42e; envelope-from=nathan@vieiraproenca.com; helo=mail-pf1-x42e.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NEUTRAL=0.779 autolearn=no autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Tue, 06 Jul 2021 16:58:03 -0400 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1625605102; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=SCupDkrBr6VvkIP9z2ZG/ZaVm7qkKQi+cPZHCil1P24=; b=Hf5fQR9okBS8AhCHM+Sn9mIYVSpRLVCleT/j9wQZ2H630yMVNCgV/IXACKZ4yC45tdMxJ5 QBOX6JTtcBmtXhDSdXaGvZ/Sagq+DvrkMGYZxlf60snsCYVpN0z9TyE6dR4p0mAyeZhYvm BaOCyw+3FW+ymnjDaLavoE65XwD5fYQ7hZkk06nxsWUVPGzdbfMsBlRewUirYQw4ORDUI3 03jxLn+8id7jDVXg83t3Q8rrb3xtOfLZbzAdrAjuJpAnpuOGTSkmoWnvoO5Mk8hU3Ci5bx vuZt64+enop9KpqEmjVGgyg8aqyZTVJTFvL5ubLhM/x6XC2T93ouUXEbegPIBw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1625605102; a=rsa-sha256; cv=none; b=JFholmgXX00gJ9771rMU9Frfmcfh/9u8qaG9Jo1LhPFVgX9USw16cpf7x6mn5gJO5ghbCd esYBotKY1gWtsvb8OjMUBG3TyzjDLrigNaHEwus62TyPI+PlvQ8W/xH6Ehb/k4BXloYyAB ki+XsREdoQsoif1M2jt2/+sMQH3DZ70H0q72wzqsv+RF4aqZwOfkJyS9k33+whpW+VsjQq sIBxUn/Qc7AVlP/+ZWbyM/Vu70gCw1cvFHeuge1SWUksQ1ZrMicwKi+neHWJfJycujQKj7 1ipRyeWkuq7VszyAxDoSTkcgt81U4RHljSEMVCN0mJ3ZOw+PQALDaMtdXSuInQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=vieiraproenca-com.20150623.gappssmtp.com header.s=20150623 header.b=RCzwl7ti; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -2.61 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=vieiraproenca-com.20150623.gappssmtp.com header.s=20150623 header.b=RCzwl7ti; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: BAA009F88 X-Spam-Score: -2.61 X-Migadu-Scanner: scn0.migadu.com X-TUID: rcW+qjZLwUO/ Bengt Richter writes: > Hi Nathan, > Nice writeup! Thank you! > On +2021-07-05 11:03:46 -0300, Nathan Benedetto Proen=C3=A7a wrote: >> Hello! >>=20 >> I am trying to upgrade the package texlive, first for me, but hopefully >> for a patch, and I have a question regarding Guix policies. >>=20 >> As you can see on >>=20 >> https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build-system/tex= live.scm?id=3Df7692617b624a01615cf412773c4dad9a2deeb68 >>=20 >> the file guix/build-system/texlive.scm exposes two variables: >>=20 >> (define %texlive-tag "texlive-2019.3") >> (define %texlive-revision 51265) >>=20 >> These variables are used throughout gnu/packages/tex.scm, as you can see >> on >>=20 >> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tex.scm?= id=3Df7692617b624a01615cf412773c4dad9a2deeb68 >>=20 >> An example is the following code: >>=20 >> (define hyph-utf8-scripts >> (origin >> (method svn-fetch) >> (uri (texlive-ref "generic" "hyph-utf8")) >> (file-name (string-append "hyph-utf8-scripts-" >> (number->string %texlive-revision) >> "-checkout")) >> (sha256 >> (base32 >> "0lk7shx768sxvgr85y8bnmmnj8x4bbkgpxrz3z8jp8avi33prw83")))) >>=20 >> Grep tells me there are 290+ occurrences of `%texlive-revision`. >> What is the purpose of these variables? >>=20 >> You see, they give me the impression that Guix is really concerned about >> upgrading *all* of texlive at once. >> These variables tell me I should go to the file texlive.scm and bump the >> tag and revision, and then handle all the broken hashes. >>=20 >> Hence, it seems to me that any attempt to upgrade the texlive package >> would have to be done in a separate branch, which would only be merged >> into master when all the packages are upgraded. >>=20 >> Is this the case? >> And if so, why? >>=20 >> I have the impression that if such "monolithic" upgrade is not a goal, >> and "partial" our "per-package" upgrades are desirable, there may be >> better solutions. >>=20 >> For example, we could add keyword arguments to texlive-ref and >> texlive-origin, so the code above becomes something like this >>=20 >> (define hyph-utf8-scripts >> (origin >> (method svn-fetch) >> (uri (texlive-ref "generic" "hyph-utf8" >> #:texlive-tag "texlive-2019.3" >> #:texlive-revision 51265)) >> (file-name "hyph-utf8-scripts-51625-checkout") >> (sha256 >> (base32 >> "0lk7shx768sxvgr85y8bnmmnj8x4bbkgpxrz3z8jp8avi33prw83")))) >>=20 >> This would work right now, and we could eventually remove every use of >> %texlive-revision and %texlive-tag, so they become implementation >> details of the build-system texlive.scm; a fallback version. >> And further down the road we may even decide to remove this fallback, >> and make developers be explicit about their tags and revisions; this >> could amount to a refactor which makes the keyword arguments into >> required arguments, for example. >>=20 >> I also like the second version of the code because the hash already >> pinpoints the tag and revision: both texlive-ref and texlive-origin use >> these variables to find the correct files to download. >> This just makes this dependency explicit. >>=20 >> In any case, as this may be a choice between shipping stable and >> up-to-date packages, and as I am new to contributing to Guix, I found >> fitting to ask. >>=20 >> Thanks in advance! >> Nathan >>=20 > > I am wondering about guaranteeing generic behaviour by > guaranteeing program source and toolchain source hash > equivalences vs ignoring sources and guaranteeing end > results by testing results. It seems to me that you are talking about my email regarding using hashing in Scheme refactoring, right? This one: https://lists.gnu.org/archive/html/guix-devel/2021-07/msg00023.html I will assume this is the case, even though I we can actually simply talk about what you wrote. First thoughts: I think that what we really want to guarantee is "correctness", and that both hashes or tests are mere proxies for this. I see them as useful in distinct moments of package development. For example, lets say that I want to partially upgrade TeXLive (which I am already convinced is not a good idea, TBH, but serves as an example). To be able to do so, I may think that I want to refactor some function to pinpoint the version I will download. I can then make a single commit which updates the signature and all the call sites of the function. I can then actually upgrade the package I care about in a second commit, and do some "proper testing" in it, like producing some pdf with latex. My interest in checking hashes in the first commit is to get some confidence that this step, which may change hundred of places, did not introduce a new bug, so I can focus any debugging on the hopefully more local change made in the second commit. > I.e., if you want to print the sum of x and y passed as > strings to a program, output as a string to stdout, it > doesn't matter (other than optimization and debuggability) > what language the program was written in, so long as it was > compiled into a form that execve and co can launch and the > end result is the same. > > As part of testing, maybe strace could be used to generate > some kind of canonical kernel transaction trace that could > be used to compare behaviours for equivalency of executing > different-language programs? > > This would be a radical change in the approach to > reproducibility, maybe dynamically selecting from a > whitelist of trusted/tested substitutable executables with > hash names in /gnu but not necessarily (though not > excluding) binaries produced with guix source guarantees. > > Seems like guix is turing-complete enough to provide this > kind of substitutable foreign functions already, so might > this be a way to avoid mass recompilations? > > Or is this already available, but not so much used? > > I am not sure where to contibute thoughts like these, where > they would be of interest rather than distracting. (Pls > excuse the noise, if that's what this is to you). I am not sure I follow, although I do not regard this as noise. You are saying that one could use strace as a check that some program behaves the same as it did before, similar to the use that I suggested for hashing, right? I do not understand how this would be used, and how this avoids mass recompilations. I mean, to have binaries to inspect with strace I must build the code anyway, right? Am I missing something? > --=20 > Regards, > Bengt Richter