From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id QHPCB3eD/WJjAgEAbAwnHQ (envelope-from ) for ; Thu, 18 Aug 2022 02:10:31 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id 4OoKB3eD/WKCTgAAG6o9tA (envelope-from ) for ; Thu, 18 Aug 2022 02:10:31 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 84F3DA5C2 for ; Thu, 18 Aug 2022 02:10:30 +0200 (CEST) Received: from localhost ([::1]:41104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oOT7E-0004tF-Hn for larch@yhetil.org; Wed, 17 Aug 2022 20:10:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51588) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oOT6S-0004t6-C7 for guix-devel@gnu.org; Wed, 17 Aug 2022 20:09:40 -0400 Received: from mail-vk1-xa35.google.com ([2607:f8b0:4864:20::a35]:38557) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oOT6P-00007j-SG for guix-devel@gnu.org; Wed, 17 Aug 2022 20:09:40 -0400 Received: by mail-vk1-xa35.google.com with SMTP id bi51so65072vkb.5 for ; Wed, 17 Aug 2022 17:09:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=HogHhmRnqZ1X3AQQSw5cUF5CxdopRy0RYi5c3SngPbg=; b=hlPsddL9zow3olYXMlPIbNXVOqwcGA43K6Zzxr52eeEDAZp7B+5SgfXBD6jYkoViXH ijHM9kGiM5mtvs2tZZ5k2I85O+Xkp854M0rCEYSQJkjkH2mRQJRGL92xY23oP6txHT75 Qkcbzz4dUmIa5X02fOp/qkCyQknNCZ2KzUXC32IhH+IRzbQRdYEEasmmD7YM9KWsdiQ+ XIfO+LpwNYEmjQbwx8gnvcrodyWIEH5X1eQiNldjQxx1w+DlmkLNnIFTrVzFpHfCZ2VO DW0CuOosoE7sYUw3QuN5pRqs2p96zrfDPceCwUHmEzPTtkVIfo0YrINytpeQFFY6tdLi AxZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=HogHhmRnqZ1X3AQQSw5cUF5CxdopRy0RYi5c3SngPbg=; b=hnq5Q7RMk+dCl8nSxPDBGY0zX1BwE93PEX6Ld5Q0ZG/g13biADgkIHnv6XIKnUzxcY HD2UvzRmqmvpE0JIjtEElVJo3DIUn9AVjZDYqmxBpksgCcUum63GPunFScb6KYMdBfoF wEMCSLH2+ZQRL4INELR8CeZGnafYbu0B5re5GThfznLoq/JMijKNFApya9drk+lAQQMW snHd08axt1+b4IZW8VwFj7QHXYpiuiG435Y7OBUNZfDAFgMI0GxD8ndQnGtfmqLnqC3y yFMZIL2Cg22feI6N1n047TI0TQy4JQKsOm03Uf+jMOhVtFVD8z6gddQpSn1+zbi/K3mu /GhQ== X-Gm-Message-State: ACgBeo2gQBiQe9WLCVbBZz7wpycrwag3QNMojFvgNtYCE0Ubdk8HBoN4 fR7tUYcHhAnuI+qTDsHhGm9Ner+p+XzYftkX7AidnLX7 X-Google-Smtp-Source: AA6agR4GPVAw8ComAS1WLKXs5oTFsV3QrHrxmfCpjtqkecyDDpuPX7oGOq/+zFmOsnhAVECrUiLhgY4y03sI26WbydU= X-Received: by 2002:a1f:2b47:0:b0:380:59c5:8b8f with SMTP id r68-20020a1f2b47000000b0038059c58b8fmr235747vkr.20.1660781375938; Wed, 17 Aug 2022 17:09:35 -0700 (PDT) MIME-Version: 1.0 References: <33556713-c85b-ab0f-554e-94a40f9d418e@telenet.be> <5331d2f3-13a5-e40c-f3bb-398438a0b103@telenet.be> In-Reply-To: <5331d2f3-13a5-e40c-f3bb-398438a0b103@telenet.be> From: =?UTF-8?Q?Aleix_Conchillo_Flaqu=C3=A9?= Date: Wed, 17 Aug 2022 17:09:24 -0700 Message-ID: Subject: Re: (ice-9 base64)? To: Maxime Devos Cc: Guix-devel Content-Type: multipart/alternative; boundary="000000000000f6644805e678cb22" Received-SPF: pass client-ip=2607:f8b0:4864:20::a35; envelope-from=aconchillo@gmail.com; helo=mail-vk1-xa35.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1660781430; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=HogHhmRnqZ1X3AQQSw5cUF5CxdopRy0RYi5c3SngPbg=; b=WEiLEmraQ0CHYYwCtoxjzVforZLRzZNL6lTicOcAww1Qy5/+pBGNBmWcihNAg4MNEScXfo HELOCFYVnQmTZGLPJeRETrWDWUIJ1y3R8SoxViJ447LJBEjTiXCaOxat3ASGHdSAbZxd4X /i3ujdF0XnF77FA4u5oB0Vjr7LOp//2H5arHJf4Mic3GCnreFA8LdywWofYq94mamDz+/3 R1mcokxgEmbqY5N9u6CKuvbpPEj24cHS3rLXrhaLfs+2+be9F7qmWjIxo60rm4fsqnIOgB EOk4CPR7NBqFouemwYvwdDV3nMO9fMaE/qLoDOkIcOqmyrmlkbzcZIxQw8CgLw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1660781430; a=rsa-sha256; cv=none; b=EvDlBt8mOjGconLS9PJzF4zPpmQDBPJtR/JXZAwHbqHKt2XmNEZxh8bWb/deQrNum8vfVh YNYo/DfmczBBaYAHrvwQuRFSZ1trZPDfc5KYsgUywcH8M4uDVuFjU9ZcPvYUh0EtoZn7af GQQklbnYbQ2g0oyvdxy3xp7Hk8bJ39NiiY1XxPpGVjipkEySffhd2VUEbLTCgPcSHYKHtO PJfnVK0uytButPfZhmX0uXEaZ8ZQfcHmXazZCgMKVTASjVAkO56Z7yuj9jgsCziucYQUlf rlu/JWXFvr2o9nkTCj1Ew8psy6IZQE3Fnx/sKsUxMoY0RCNj3n9I/qoEYodKaQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=hlPsddL9; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.36 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=hlPsddL9; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 84F3DA5C2 X-Spam-Score: -1.36 X-Migadu-Scanner: scn1.migadu.com X-TUID: WtHeL8Xe83L4 --000000000000f6644805e678cb22 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thank you for the explanations Maxime! Then, if I understood correctly, IMO I would say Guile should not really care about Guix's bundling/unbundling. That is, adding (ice-9 base64) (or however we want to call it... maybe (encoding base64) following Golang and Guile's (web ....) module) should be totally independent of Guix. So, if we add (ice-9 base64) to Guile then Guix should figure out what to do with it, but it's Guix's concern not Guile's. About Guix's unbundling (maybe that's something that should go on Guix's mailing list), I don't think currently there's any unbundling for base64 modules or at least not in a package I maintain guile-jwt (guile-jwt bundles base64). And probably there's no unbundling because there's no canonical implementation? Even if there was a canonical implementation, how would that look like in Guix's guile-jwt package? What would the snippet actually do? Thanks, Aleix On Wed, Aug 17, 2022 at 12:23 PM Maxime Devos wrote: > > On 17-08-2022 18:22, Aleix Conchillo Flaqu=C3=A9 wrote: > > Hi Maxime! > > On Tue, Aug 16, 2022 at 12:04 PM Maxime Devos > wrote: > >> >> On 16-08-2022 19:21, Aleix Conchillo Flaqu=C3=A9 wrote: >> >> >> >> On Tue, Aug 16, 2022 at 9:59 AM Maxime Devos >> wrote: >> >>> >>> On 16-08-2022 18:10, Aleix Conchillo Flaqu=C3=A9 wrote: >>> >>> Hi, >>> >>> In many projects I've been copying G=C3=B6ran Weinholt's base64 >>> implementation and I've also seen it in other projects, would it make s= ense >>> to include it in Guile's standard library? [...] >>> >>> If we do this, we should contact the various other projects to make the= m >>> use (ice-9 base64). >>> >> >> I think they could switch whenever they want (i.e. whenever this was >> added to Guile) or even not switch at all. >> >> Sure, but they can't switch if they don't know about it. And if they >> don't know about it and hence don't switch, the proposal fails at its >> purpose of unbundling base64. Besides, we need them to switch (see Guix >> no-bundling policy and the reasons behind it) -- if upstream refuses to >> unbundle, then in our locally modified version for Guix. >> > > Forgive my ignorance, but what do you mean by unbundling? I'm not familia= r > with Guix at all, well, just conceptually and for trying a few commands > years ago. > > Sometimes the source code of a package contains a copy of a dependency. > This is called 'bundling'. 'Unbundling' is the act of undoing the > 'bundling', this is often done by cleaning up the source code (with what = we > call a 'snippet' in Guix: (snippet #~(delete-file-recursively > "googletest"))) and setting some configuration flags > ("-DUSE_SYSTEM_GOOGLETEST=3Dyes" or such). > > For example, in Guix we occasionally encounter a bundled "googletest" (a > test framework). > > In this case, we are kind of (un)bundling the base64 module, though it's > not _exactly_ (un)bundling because, AFAIK, there is canonical upstream > location for the base64 module to replace things with. Still seems pretty > close to me. > > Upsides of unbundling, as mentioned in '(guix)Submitting Patches': > > Sometimes, packages include copies of the source code of their > dependencies as a convenience for users. However, as a > distribution, we want to make sure that such packages end up using > the copy we already have in the distribution, if there is one. > This improves resource usage (the dependency is built and stored > only once), and allows the distribution to make transverse changes > such as applying security updates for a given software package in a > single place and have them affect the whole system=E2=80=94something= that > bundled copies prevent. > > Another benefit: reviewing for absence of malware is less work when > there's only a single copy to review, though I suppose that in this case > the module is so small the reviewing benefit is minimal. > > Whether we simply replace (guix base64) by (gcrypt base64) depends on how >>> old (gcrypt base64) is compared to the earliest 'supported' Guix for >>> pull/time-travel, but even if it is not present in the old gcrypt, we c= an >>> work-around that (we have a 'fake-gcrypt-hash' in build-aux/build-self.= scm, >>> so we can easily have a (define gcrypt-base64 [some copy])). Or simply >>> update the local guile-gcrypt in buid-aux/build-self.scm. >>> >> guile-gcrypt base64 is pretty new with the patch above (but no release >> after that), I have no idea if Guix has added anything else. >> >> base64 is available in at least 0.3.0, which is packaged in Debian >> bullseye (which is considered "stable"), so not too new, though we might >> need to change build-aux/build-self.scm if 0.1.0 doesn't have base64. G= uix >> appears to have the pre-quoted-patch version, without changes of its own >> except for a different module name. >> > > One more time, forgive me, but what is build-aux/build-self.scm? > > It's an implementation detail of Guix, it's a file (from the new version, > not the old) that is loaded by "guix pull" in the old Guix to compile the > new version of Guix. > > OTOH a similar replacement can be done for (ice-9 base64), but >>> transitioning to (ice-9 base64) would take much longer, at least until = the >>> various distributions are updated to a Guile that has (ice-9 base64), >>> whereas (gcrypt base64) could be switched to immediately. >>> >> Maybe this could be handled by each project independently. >> >> They wouldn't have to if the base64 module is put in (guile gcrypt). >> >> >> And the last forgiveness... (guile gcrypt)? > > Oops, that should have been guile-gcrypt -- it's a Guile package -- "guix > show guile-gcrypt" / > . > > Greetings, > Maxime. > --000000000000f6644805e678cb22 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thank you for the explanations Maxime!=

Then, if I understood correctly, IMO I would say Guil= e should not really care about Guix's bundling/unbundling. That is, add= ing (ice-9 base64) (or however we want to call it... maybe (encoding base64= ) following Golang and Guile's (web ....) module) should be totally ind= ependent of Guix. So, if we add (ice-9 base64) to Guile then Guix should fi= gure out what to do with it, but it's Guix's concern not Guile'= s.

About Guix's unbundling (maybe that's somet= hing that should go on Guix's mailing list), I don't think currentl= y there's any unbundling for base64 modules or at least not in a packag= e I maintain guile-jwt (guile-jwt bundles base64). And probably there's= no unbundling because there's no canonical implementation? Even if the= re was a canonical implementation,=C2=A0how would that look like in Guix= 9;s guile-jwt package? What would the snippet actually do?

Thanks,

Aleix

On Wed, Aug 17, 2022 at 12:23= PM Maxime Devos <maximedevos@= telenet.be> wrote:
=20 =20 =20


On 17-08-2022 18:22, Aleix Conchillo Flaqu=C3=A9 wrote:
=20
Hi Maxime!<= /div>

On Tue, Aug 16, 2022 at 12:04 PM Maxime Devos <maximedevos@telenet.be> wrote:


On 16-08-2022 19:21, Aleix Conchillo Flaqu=C3=A9 wrote:<= br>
<= br>

On Tue, Aug 16, 2022 at 9:59 AM Maxime Devos <maximedevos@telenet.be> wrote:


On 16-08-2022 18:10, Aleix Conchillo Flaqu=C3= =A9 wrote:
Hi,

In many projects I've been copying = G=C3=B6ran Weinholt's base64 implementation and I'= ve also seen it in other projects, would it make sense to include it in Guile's standar= d library? [...]

If we do this, we should contact the various other projects to make them use (ice-9 base64).


I think they could switch whenever they want (i.e. whenever this was added to Guile) or even not switch at all.

Sure, but they can't switch if they don't know abo= ut it. And if they don't know about it and hence don't switch, the proposal fails at its purpose of unbundling base64. Besides, we need them to switch (see Guix no-bundling policy and the reasons behind it) -- if upstream refuses to unbundle, then in our locally modified version for Guix.


Forgive my ignorance, but what do you mean by unbundling? I'm not familiar with Guix at all, well, just conceptually and for trying a few commands years ago.

Sometimes the source code of a package contains a copy of a dependency. This is called 'bundling'. 'Unbundling' i= s the act of undoing the 'bundling', this is often done by cleaning up the source code (with what we call a 'snippet' in Guix: (snippet #~(delete-file-recursively "googletest"))) and setting some configuration flags ("-DUSE_SYSTEM_GOOGLETEST=3Dyes" or suc= h).

For example, in Guix we occasionally encounter a bundled "googletest" (a test framework).

In this case, we are kind of (un)bundling the base64 module, though it's not _exactly_ (un)bundling because, AFAIK, there is canonical upstream location for the base64 module to replace things with. Still seems pretty close to me.

Upsides of unbundling, as mentioned in '(guix)Submitting Patches':

=C2=A0=C2=A0=C2=A0=C2=A0 Sometimes, pac= kages include copies of the source code of their
=C2=A0=C2=A0=C2=A0=C2=A0 dependencies as a convenience for users.= =C2=A0 However, as a
=C2=A0=C2=A0=C2=A0=C2=A0 distribution, we want to make sure that su= ch packages end up using
=C2=A0=C2=A0=C2=A0=C2=A0 the copy we already have in the distributi= on, if there is one.
=C2=A0=C2=A0=C2=A0=C2=A0 This improves resource usage (the dependen= cy is built and stored
=C2=A0=C2=A0=C2=A0=C2=A0 only once), and allows the distribution to= make transverse changes
=C2=A0=C2=A0=C2=A0=C2=A0 such as applying security updates for a gi= ven software package in a
=C2=A0=C2=A0=C2=A0=C2=A0 single place and have them affect the whol= e system=E2=80=94something that
=C2=A0=C2=A0=C2=A0=C2=A0 bundled copies prevent.
Another benefit: reviewing for absence of malware is less work when there's only a single copy to review, though I suppose that in this case the module is so small the reviewing benefit is minimal.

Whether we simply replace (guix base64) by (gcrypt base64) depends on how old (gcrypt base64) is compared to the earliest 'supported' Guix for pull/time-travel, bu= t even if it is not present in the old gcrypt, we can work-around that (we have a 'fake-gcrypt-hash' in build-aux/build-self.scm, so we can easily have a (define gcrypt-base64 [some copy])).=C2=A0 Or simply update the local guile-gcrypt in buid-aux/build-self.scm.

guile-gcrypt base64 is pretty new with the patch above (but no release after that), I have no idea if Guix has added anything else.

base64 is available in at least 0.3.0, which is packaged in Debian bullseye (which is considered "stable"), so not too new, though we might need t= o change build-aux/build-self.scm if 0.1.0 doesn't have base64.=C2=A0 Guix appears to have the pre-quoted-patch version, without changes of its own except for a different module name.


One more time, forgive me, but what is build-aux/build-self.scm?
It's an implementation detail of Guix, it's a file (from the ne= w version, not the old) that is loaded by "guix pull" in the ol= d Guix to compile the new version of Guix.

OTOH a similar replacement can be done for (ice-9 base64), but transitioning to (ice-9 base64) would take much longer, at least until the various distributions are updated to a Guile that has (ice-9 base64), whereas (gcrypt base64) could be switched to immediately.

Maybe this could be handled by each project independently.

They wouldn't have to if the base64 module is put in (guile gcrypt).


And the last forgiveness... (guile gcrypt)?

Oops, that should have been guile-gcrypt -- it's a Guile package -- "guix show guile-gcrypt" / <https://notabug.org/cwebber/guile-gcrypt>.

Greetings,
Maxime.

--000000000000f6644805e678cb22--