(ice-9 base64)?

(ice-9 base64)?

[Aleix Conchillo Flaqué]

[-- Attachment #1: Type: text/plain, Size: 608 bytes --]

Hi,

In many projects I've been copying Göran Weinholt's base64 implementation
and I've also seen it in other projects, would it make sense to include it
in Guile's standard library?

I guess it's hard to know where to draw a line and different languages do
different things. I actually like Golang's approach where they provide a
bunch of base libraries (web, crypto, etc.) and it seems that's where Guile
was going by including a web library.You could always use a different
implementation if it existed.

Anyways, I think (ice-9 base64) would be a nice addition.

Just a thought.

Aleix

[-- Attachment #2: Type: text/html, Size: 1616 bytes --]

Re: [EXT] (ice-9 base64)?

[Thompson, David]

[-- Attachment #1: Type: text/plain, Size: 819 bytes --]

+1 to that. I do the same thing in my own projects. Guix does it, too.

- Dave

On Tue, Aug 16, 2022 at 12:11 PM Aleix Conchillo Flaqué <
aconchillo@gmail.com> wrote:

> Hi,
>
> In many projects I've been copying Göran Weinholt's base64 implementation
> and I've also seen it in other projects, would it make sense to include it
> in Guile's standard library?
>
> I guess it's hard to know where to draw a line and different languages do
> different things. I actually like Golang's approach where they provide a
> bunch of base libraries (web, crypto, etc.) and it seems that's where Guile
> was going by including a web library.You could always use a different
> implementation if it existed.
>
> Anyways, I think (ice-9 base64) would be a nice addition.
>
> Just a thought.
>
> Aleix
>
>

[-- Attachment #2: Type: text/html, Size: 2107 bytes --]

Re: (ice-9 base64)?

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1493 bytes --]


On 16-08-2022 18:10, Aleix Conchillo Flaqué wrote:
> Hi,
>
> In many projects I've been copying Göran Weinholt's base64 
> implementation and I've also seen it in other projects, would it make 
> sense to include it in Guile's standard library? [...]

If we do this, we should contact the various other projects to make them 
use (ice-9 base64).

I think it would be simpler though to consider the base64 in 
guile-gcrypt to be 'canonical', it would avoid problems with old 
versions of Guile not having the base64 module and newer version having 
it, which would prevent using the proposed (ice-9 base64) in Guile 
because it would break build-aux/build-self.scm when pulling or 
time-machining from old Guix that have an old Guile.

Whether we simply replace (guix base64) by (gcrypt base64) depends on 
how old (gcrypt base64) is compared to the earliest 'supported' Guix for 
pull/time-travel, but even if it is not present in the old gcrypt, we 
can work-around that (we have a 'fake-gcrypt-hash' in 
build-aux/build-self.scm, so we can easily have a (define gcrypt-base64 
[some copy])).  Or simply update the local guile-gcrypt in 
buid-aux/build-self.scm.

OTOH a similar replacement can be done for (ice-9 base64), but 
transitioning to (ice-9 base64) would take much longer, at least until 
the various distributions are updated to a Guile that has (ice-9 
base64), whereas (gcrypt base64) could be switched to immediately.

Greetings,
Maxime.


[-- Attachment #1.1.1.2: Type: text/html, Size: 2318 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: (ice-9 base64)?

[Aleix Conchillo Flaqué]

[-- Attachment #1: Type: text/plain, Size: 2519 bytes --]

On Tue, Aug 16, 2022 at 9:59 AM Maxime Devos <maximedevos@telenet.be> wrote:

>
> On 16-08-2022 18:10, Aleix Conchillo Flaqué wrote:
>
> Hi,
>
> In many projects I've been copying Göran Weinholt's base64 implementation
> and I've also seen it in other projects, would it make sense to include it
> in Guile's standard library? [...]
>
> If we do this, we should contact the various other projects to make them
> use (ice-9 base64).
>
>
I think they could switch whenever they want (i.e. whenever this was added
to Guile) or even not switch at all.


> I think it would be simpler though to consider the base64 in guile-gcrypt
> to be 'canonical', it would avoid problems with old versions of Guile not
> having the base64 module and newer version having it, which would prevent
> using the proposed (ice-9 base64) in Guile because it would break
> build-aux/build-self.scm when pulling or time-machining from old Guix that
> have an old Guile.
>
>
I've been waiting on a guile-gcrypt release for a while now (Ludo,
Chrisitine... any help here? :-) ).  I ported guile-jwt to use guile-gcrypt
but I need a release to have latest base64 changes:

https://notabug.org/cwebber/guile-gcrypt/commit/f8934ec94df5868ee8baf1fb0f8ed0f24e7e91eb

But you are right that this would cause a backward compatible problem, but
I guess that would depend on each project. Can we do conditional module
loading? I've done this in the past with Python... if we are in Python 2
load this module, otherwise load this other one. So projects could do that.

> Whether we simply replace (guix base64) by (gcrypt base64) depends on how
> old (gcrypt base64) is compared to the earliest 'supported' Guix for
> pull/time-travel, but even if it is not present in the old gcrypt, we can
> work-around that (we have a 'fake-gcrypt-hash' in build-aux/build-self.scm,
> so we can easily have a (define gcrypt-base64 [some copy])).  Or simply
> update the local guile-gcrypt in buid-aux/build-self.scm.
>
> guile-gcrypt base64 is pretty new with the patch above (but no release
after that), I have no idea if Guix has added anything else.


> OTOH a similar replacement can be done for (ice-9 base64), but
> transitioning to (ice-9 base64) would take much longer, at least until the
> various distributions are updated to a Guile that has (ice-9 base64),
> whereas (gcrypt base64) could be switched to immediately.
>
>
> Maybe this could be handled by each project independently.

Best,

Aleix

[-- Attachment #2: Type: text/html, Size: 5367 bytes --]

Re: (ice-9 base64)?

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 2741 bytes --]

> Then, if I understood correctly, IMO I would say Guile should not 
> really care about Guix's bundling/unbundling. That is, adding (ice-9 
> base64) (or however we want to call it... maybe (encoding base64) 
> following Golang and Guile's (web ....) module) should be totally 
> independent of Guix. So, if we add (ice-9 base64) to Guile then Guix 
> should figure out what to do with it, but it's Guix's concern not Guile's.

It's not some Guix-specific quirk. It's the same for at least Debian. It 
benefits not only Guix itself but all users of the software:

> [...] allows [...] to make transverse changes
> such as applying security updates for a given software package in a
> single place and have them affect the whole system—something that
> bundled copies prevent.
... that was written with Guix in mind, but it applies to every 
distribution and everyone.

Besides, your goal appears to be to unbundle the base64 into a single 
location (as a module of Guile), if we do that I think we should go all 
the way -- just adding it to Guile increases bundling instead of 
decreasing bundling, only if the various upstreams are modified to 
unbundle and use the new location then the unbundling is completed.

On 18-08-2022 02:09, Aleix Conchillo Flaqué wrote:
> About Guix's unbundling (maybe that's something that should go on 
> Guix's mailing list),
I don't see why, there's nothing to write about except "oops some 
packages are bundling base64, let's unbundle those", and for unbundling 
those, it seems more practical to write about that here on guile-devel. 
Also I noticed I sent some messages to guix-devel instead of 
guile-devel, correcting now.
> I don't think currently there's any unbundling for base64 modules or 
> at least not in a package I maintain guile-jwt (guile-jwt bundles 
> base64). And probably there's no unbundling because there's no 
> canonical implementation? Even if there was a canonical 
> implementation, how would that look like in Guix's guile-jwt package? 
> What would the snippet actually do?
Currently, it's not done yet, presumably for that reason and maybe also 
due to nobody having noticed it yet?

How it would look like, for upstreams that refuse to unbundle or are 
unresponsive:

#~(begin
         (delete-file "local/copy/of/base64.scm")
         [also remove it from the Makefile.am]
         (substitute* (find-files "." "\\.scm$")
           (("(\\local base64 module\\)") "(gcrypt base64)")))

For responsive upstreams that do not mind these kind of improvements, 
there is a preference for submitting a patch upstream -- that way, 
everyone benefits, not only Guix.

Greetings,
Maxime.


[-- Attachment #1.1.1.2: Type: text/html, Size: 4009 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: (ice-9 base64)?

[Aleix Conchillo Flaqué]

[-- Attachment #1: Type: text/plain, Size: 3973 bytes --]

On Thu, Aug 18, 2022 at 12:56 AM Maxime Devos <maximedevos@telenet.be>
wrote:

> Then, if I understood correctly, IMO I would say Guile should not really
> care about Guix's bundling/unbundling. That is, adding (ice-9 base64) (or
> however we want to call it... maybe (encoding base64) following Golang and
> Guile's (web ....) module) should be totally independent of Guix. So, if we
> add (ice-9 base64) to Guile then Guix should figure out what to do with it,
> but it's Guix's concern not Guile's.
>
> It's not some Guix-specific quirk. It's the same for at least Debian. It
> benefits not only Guix itself but all users of the software:
>
Thanks, I understand the benefit now.

> [...] allows [...] to make transverse changes
> such as applying security updates for a given software package in a
> single place and have them affect the whole system—something that
> bundled copies prevent.
>
> ... that was written with Guix in mind, but it applies to every
> distribution and everyone.
>
> Besides, your goal appears to be to unbundle the base64 into a single
> location (as a module of Guile), if we do that I think we should go all the
> way -- just adding it to Guile increases bundling instead of decreasing
> bundling, only if the various upstreams are modified to unbundle and use
> the new location then the unbundling is completed.
>
I see... In my mind, initially, I was thinking the opposite. By adding it
to Guile, new projects will use the new base64 module and existing projects
will update whenever they want.

On 18-08-2022 02:09, Aleix Conchillo Flaqué wrote:
>
> About Guix's unbundling (maybe that's something that should go on Guix's
> mailing list),
>
> I don't see why, there's nothing to write about except "oops some packages
> are bundling base64, let's unbundle those", and for unbundling those, it
> seems more practical to write about that here on guile-devel. Also I
> noticed I sent some messages to guix-devel instead of guile-devel,
> correcting now.
>
> I don't think currently there's any unbundling for base64 modules or at
> least not in a package I maintain guile-jwt (guile-jwt bundles base64). And
> probably there's no unbundling because there's no canonical implementation?
> Even if there was a canonical implementation, how would that look like in
> Guix's guile-jwt package? What would the snippet actually do?
>
> Currently, it's not done yet, presumably for that reason and maybe also
> due to nobody having noticed it yet?
>
> How it would look like, for upstreams that refuse to unbundle or are
> unresponsive:
>
> #~(begin
>         (delete-file "local/copy/of/base64.scm")
>         [also remove it from the Makefile.am]
>         (substitute* (find-files "." "\\.scm$")
>           (("(\\local base64 module\\)") "(gcrypt base64)")))
>
OK, I was imagining something like that. In this case do we assume (gcrypt
base64) is installed? Because some projects don't have a dependency on
guile-gcrypt.

> For responsive upstreams that do not mind these kind of improvements,
> there is a preference for submitting a patch upstream -- that way, everyone
> benefits, not only Guix.
>
So, what do you think would be the way to proceed in order to include a
base64 implementation in Guile itself?

For example:

1. Add (ice-9 base64) (or (encoding base64)) to Guile and let new projects
and existing projects to update with conditional module loading to support
old versions of Guile.
2. Do unbundling in Guix packages both for projects that have not updated
upstream and for projects in (1). The unbundling would be done by pointing
to Guix's (or guile-gcrypt) base64 implementation, or is there a way they
could point to Guile's implementation?

Does that make sense or am I still missing something (I'm about to catch a
cold so my brain is not working quite well this week)? Originally, I was
thinking only in (1).

Thanks!

Aleix

[-- Attachment #2: Type: text/html, Size: 7443 bytes --]

Re: (ice-9 base64)?

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 2121 bytes --]

On 19-08-2022 02:20, Aleix Conchillo Flaqué wrote:

> So, what do you think would be the way to proceed in order to include 
> a base64 implementation in Guile itself?
>
> For example:
>
> 1. Add (ice-9 base64) (or (encoding base64)) to Guile and let new 
> projects and existing projects to update with conditional module 
> loading to support old versions of Guile.
> 2. Do unbundling in Guix packages both for projects that have not 
> updated upstream and for projects in (1). The unbundling would be done 
> by pointing to Guix's (or guile-gcrypt) base64 implementation, or is 
> there a way they could point to Guile's implementation?

If the canonical location of base64 becomes Guile itself instead of 
guile-gcrypt, then it needs to be pointed at Guile's base64. Likewise, 
Guix' base64 implementation would need to be replaced by Guile's, with a 
fallback.

I don't see why we would point to Guix' implementation, it's missing 
some bug fixes.

>
> Does that make sense or am I still missing something (I'm about to 
> catch a cold so my brain is not working quite well this week)? 
> Originally, I was thinking only in (1).

Except for the remark about (1), I think so.  I think the following list 
is a bit more clear though:

 1. Add (ice-9 base64) to Guile (or another name (encoding base64)).
 2. Inform a few upstreams that used to include a copy of base64 that it
    is now part of Guile itself -- those upstreams can then remove their
    local copy and use Guile's base64, and do conditional module loading
    if they cannot increase their minimal Guile version yet.
 3. In Guix, we will have to update Guile to a new version that has the
    base64 module and remove the local fallback copies.  And if upstream
    refuses patches to use Guile's base64 (maybe with a fallback), then
    it will need to be patched locally in Guix.

On (2): I don't think it's necessary to contact _all_ the upstreams, 
though to give a good example it would be nice to contact some of them.

(3) is a Guix concern, not really a Guile concern.

Greetings,
Maxime.


[-- Attachment #1.1.1.2: Type: text/html, Size: 3791 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]