‘computed-origin-method’ for IceCat and ungoogled-chromium

‘computed-origin-method’ for IceCat and ungoogled-chromium

[Ludovic Courtès]

Hello Mark & Marius!

Somehow I hadn’t noticed the clever trick with ‘computed-origin-method’
in IceCat and ungoogled-chromium, and it came to me as a shock today.
;-)

I stumbled upon it while traversing packages with ‘guix lint’ because
they violated the assumption that the ‘sha256’ field of <origin> is
always a bytevector.

I suspect this could be rewritten by using ‘computed-file’ instead of
<origin>.  This should be clearer, I think, notably because it would
avoid exposing a “non-conforming” <origin>.

WDYT?

Ludo’.

Re: ‘computed-origin-method’ for IceCat and ungoogled-chromium

[Mark H Weaver]

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Hello Mark & Marius!
>
> Somehow I hadn’t noticed the clever trick with ‘computed-origin-method’
> in IceCat and ungoogled-chromium, and it came to me as a shock today.
> ;-)

Yes, I probably should have brought wider attention to what I did there.
Sorry about that.  Initially I needed it on short notice for the
IceCat-60.5.0 security update, which turned out to be impractical to do
by my earlier method, where I would cherry-pick dozens of patches from
the upstream repo, because I found to my dismay that Mozilla had
reindented the entire source tree, including the ESR branch.

Anyway, the purpose of ‘computed-origin-method’ is simply to work around
some limitations in 'patch-and-repack' and snippets.  Most importantly,
I needed to produce an output directory and tarball name starting with
"icecat-" from an original source tarball named "firefox-...".

> I stumbled upon it while traversing packages with ‘guix lint’ because
> they violated the assumption that the ‘sha256’ field of <origin> is
> always a bytevector.
>
> I suspect this could be rewritten by using ‘computed-file’ instead of
> <origin>.  This should be clearer, I think, notably because it would
> avoid exposing a “non-conforming” <origin>.
>
> WDYT?

I'd be glad to switch to another mechanism, but I'm not sure
‘computed-file’ can be made to do what I need.  The problem there is
that ‘computed-file’ needs a G-expression.  I found that I needed to
delay evaluation of the G-expression used to produce 'icecat-source', to
cope with our widespread cyclic module dependencies.  If I remember
correctly, the problem is that when gnuzilla.scm is loaded, the modules
that it depends on may not have been loaded yet, due to the circular
module dependencies, so we generally run into problems unless all
references to imported packages are delayed.  That's why
‘computed-origin-method’ takes a gexp _promise_ instead of a gexp.

Also, I wasn't sure if all of the code that pattern matches on package
'source' fields handle non-origins properly.

Finally, I should mention that in addition to IceCat and
ungoogled-chromium, our linux-libre source tarballs are also now
generated using ‘computed-origin-method’.  That raises an additional
issue: in the case of our 'linux-libre' packages, we apply patches
*after* running the code that generates the pristine linux-libre
tarball.  With ‘computed-origin-method’, I can add 'patches' to that
origin type, and it will do what I need.  I'm not sure if this can be
done with ‘computed-file’ or snippets, even if the other limitations are
addressed.

What do you think?

       Mark

Re: ‘computed-origin-method’ for IceCat and ungoogled-chromium

[Ludovic Courtès]

Hi Mark,

Mark H Weaver <mhw@netris.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello Mark & Marius!
>>
>> Somehow I hadn’t noticed the clever trick with ‘computed-origin-method’
>> in IceCat and ungoogled-chromium, and it came to me as a shock today.
>> ;-)
>
> Yes, I probably should have brought wider attention to what I did there.
> Sorry about that.  Initially I needed it on short notice for the
> IceCat-60.5.0 security update, which turned out to be impractical to do
> by my earlier method, where I would cherry-pick dozens of patches from
> the upstream repo, because I found to my dismay that Mozilla had
> reindented the entire source tree, including the ESR branch.

Much appreciated!

> Anyway, the purpose of ‘computed-origin-method’ is simply to work around
> some limitations in 'patch-and-repack' and snippets.  Most importantly,
> I needed to produce an output directory and tarball name starting with
> "icecat-" from an original source tarball named "firefox-...".

OK.

>> I stumbled upon it while traversing packages with ‘guix lint’ because
>> they violated the assumption that the ‘sha256’ field of <origin> is
>> always a bytevector.
>>
>> I suspect this could be rewritten by using ‘computed-file’ instead of
>> <origin>.  This should be clearer, I think, notably because it would
>> avoid exposing a “non-conforming” <origin>.
>>
>> WDYT?
>
> I'd be glad to switch to another mechanism, but I'm not sure
> ‘computed-file’ can be made to do what I need.  The problem there is
> that ‘computed-file’ needs a G-expression.  I found that I needed to
> delay evaluation of the G-expression used to produce 'icecat-source', to
> cope with our widespread cyclic module dependencies.

Bah, true, that’s terrible.  We could fix it by making ‘source’ a
delayed or a thunked field, but that’s not great (notably because it
might have a performance impact.)

Another option (thinking out loud) would be to have a gexp compiler for
promises, such that you can splice a promise in a gexp and have it
transparently forced when needed.

Thoughts?

> Also, I wasn't sure if all of the code that pattern matches on package
> 'source' fields handle non-origins properly.

It’s supposed to (whereas sha256 = #f is not an intended use case.)

> Finally, I should mention that in addition to IceCat and
> ungoogled-chromium, our linux-libre source tarballs are also now
> generated using ‘computed-origin-method’.  That raises an additional
> issue: in the case of our 'linux-libre' packages, we apply patches
> *after* running the code that generates the pristine linux-libre
> tarball.  With ‘computed-origin-method’, I can add 'patches' to that
> origin type, and it will do what I need.  I'm not sure if this can be
> done with ‘computed-file’ or snippets, even if the other limitations are
> addressed.

Good point.

Besides, I really like it that the deblob and firefox-to-icecat scripts
are run directly in derivations; it makes more sense than downloading
pre-processed tarballs!

Thanks,
Ludo’.