Problem with texlive-default-updmap.cfg

Problem with texlive-default-updmap.cfg

[Ontje.Luensdorf]

Hi,


if I try to run this hello world workflow, I get the following error:

process hi
  # { echo "hi" }

workflow do-the-thing
  processes hi

$ guix workflow run workflow.w
info: .19 Loading workflow file `workflow.w'...
info: .21 Computing workflow `do-the-thing'...
error: 3.33 No variable named texlive-default-updmap.cfg in #<interface (gnu packages tex) 7f3ed7185d20>

Is something borked in my setup?

Thanks & best regards,
Ontje

Re: Problem with texlive-default-updmap.cfg

[Ricardo Wurmus]

Hi Ontje,

> if I try to run this hello world workflow, I get the following error:
>
> process hi
>   # { echo "hi" }
>
> workflow do-the-thing
>   processes hi
>
> $ guix workflow run workflow.w
> info: .19 Loading workflow file `workflow.w'...
> info: .21 Computing workflow `do-the-thing'...
> error: 3.33 No variable named texlive-default-updmap.cfg in #<interface (gnu packages tex) 7f3ed7185d20>
>
> Is something borked in my setup?

Maybe.  Guix commit 805af862c6f0f6c54b74125bff8d348ae8f8e6f8 added
texlive-default-updmap.cfg to the (gnu packages tex) module in question.

Could you please show me the output of “guix describe”?

-- 
Ricardo

Re: Problem with texlive-default-updmap.cfg

[Ricardo Wurmus]

Hi again,

I’m brazenly Cc-ing Ludovic, who understands the (guix inferior)
mechanism better than anyone else.

Too much information follows.

>> $ guix workflow run workflow.w
>> info: .19 Loading workflow file `workflow.w'...
>> info: .21 Computing workflow `do-the-thing'...
>> error: 3.33 No variable named texlive-default-updmap.cfg in #<interface (gnu packages tex) 7f3ed7185d20>
>>
>> Is something borked in my setup?
>
> Maybe.  Guix commit 805af862c6f0f6c54b74125bff8d348ae8f8e6f8 added
> texlive-default-updmap.cfg to the (gnu packages tex) module in question.

Looks like a genuine bug.  I haven’t quite figured out why it happens
yet, but it happens when the invoking Guix is recent enough to include
commit 805af862c6f0f6c54b74125bff8d348ae8f8e6f8.

That commit not only adds texlive-default-updmap.cfg to (gnu packages
tex), but also references it in (guix profiles).

With the GWL there really are always two versions of Guix: an older
fixed version of Guix that provides the Guile APIs that the GWL uses (=
Guix as a library) and whichever version of Guix the user used to invoke
“guix workflow” (= the invoking Guix).

We’re making an effort to capture the Guile load path at configure/build
time and then reset it to that known value at runtime when “guix
workflow” is invoked.  We do this so that the Guix library we used when
developing is the same Guix that is available at run time.

The invoking Guix on the other hand should always be what the user
expects.  It is used through (guix inferior) only.

This error indicates to me that a *new* Guix (e.g. the invoking Guix)
somehow ends up accessing an *old* Guix (e.g. the library Guix).  My
thinking is that the invoking Guix is asked to build a profile, then
builds the profile hook for TeX Live packages, and then somehow looks
*outside* of the invoking Guix for that package.

The profile hook does perform a lazy package lookup with this:

(define updmap.cfg
    (module-ref (resolve-interface '(gnu packages tex))
                'texlive-default-updmap.cfg))

I wonder if that ends up being evaluated outside of the inferior
somehow.  That would be quite a bummer.

I see one bad workaround: remove some/all profile hooks in the GWL.
This would likely circumvent this problem.

-- 
Ricardo

Re: Problem with texlive-default-updmap.cfg

[Ludovic Courtès]

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

> The invoking Guix on the other hand should always be what the user
> expects.  It is used through (guix inferior) only.
>
> This error indicates to me that a *new* Guix (e.g. the invoking Guix)
> somehow ends up accessing an *old* Guix (e.g. the library Guix).  My
> thinking is that the invoking Guix is asked to build a profile, then
> builds the profile hook for TeX Live packages, and then somehow looks
> *outside* of the invoking Guix for that package.
>
> The profile hook does perform a lazy package lookup with this:
>
> (define updmap.cfg
>     (module-ref (resolve-interface '(gnu packages tex))
>                 'texlive-default-updmap.cfg))
>
> I wonder if that ends up being evaluated outside of the inferior
> somehow.  That would be quite a bummer.

Not sure I understand the context well enough, but yes,
‘texlive-font-maps’ in (guix profiles) uses packages from the host Guix,
not from an inferior (it cannot know that inferiors are being used).

Does that lead it to build incorrect font maps or things like that?

Would it help to change ‘texlive-font-maps’ to use
‘manifest-lookup-package’ instead to find its ‘updmap.cfg’ package?
This is what several other hooks do, precisely to make sure they pick
the matching package.

HTH!

Ludo’.

Re: Problem with texlive-default-updmap.cfg

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> The invoking Guix on the other hand should always be what the user
>> expects.  It is used through (guix inferior) only.
>>
>> This error indicates to me that a *new* Guix (e.g. the invoking Guix)
>> somehow ends up accessing an *old* Guix (e.g. the library Guix).  My
>> thinking is that the invoking Guix is asked to build a profile, then
>> builds the profile hook for TeX Live packages, and then somehow looks
>> *outside* of the invoking Guix for that package.
>>
>> The profile hook does perform a lazy package lookup with this:
>>
>> (define updmap.cfg
>>     (module-ref (resolve-interface '(gnu packages tex))
>>                 'texlive-default-updmap.cfg))
>>
>> I wonder if that ends up being evaluated outside of the inferior
>> somehow.  That would be quite a bummer.
>
> Not sure I understand the context well enough, but yes,
> ‘texlive-font-maps’ in (guix profiles) uses packages from the host Guix,
> not from an inferior (it cannot know that inferiors are being used).
>
> Does that lead it to build incorrect font maps or things like that?

Worse: there’s a mismatch between what one Guix wants and what the other
offers.  Apparently the profile hook from the newer Guix is used, but
the look up of texlive-default-updmap.cfg in (gnu packages tex) happens
in the older Guix — which doesn’t *have* that package — and thus fails.

> Would it help to change ‘texlive-font-maps’ to use
> ‘manifest-lookup-package’ instead to find its ‘updmap.cfg’ package?
> This is what several other hooks do, precisely to make sure they pick
> the matching package.

Maybe.  I’ll see if wa can change this and if it makes a difference.
Thanks for the input!

-- 
Ricardo

Re: Problem with texlive-default-updmap.cfg

[zimoun]

Hi,

On Sun, 27 Feb 2022 at 18:50, Ricardo Wurmus <rekado@elephly.net> wrote:

> > Not sure I understand the context well enough, but yes,
> > ‘texlive-font-maps’ in (guix profiles) uses packages from the host Guix,
> > not from an inferior (it cannot know that inferiors are being used).
> >
> > Does that lead it to build incorrect font maps or things like that?
>
> Worse: there’s a mismatch between what one Guix wants and what the other
> offers.  Apparently the profile hook from the newer Guix is used, but
> the look up of texlive-default-updmap.cfg in (gnu packages tex) happens
> in the older Guix — which doesn’t *have* that package — and thus fails.

I do not understand all the machinery so it is probably irrelevant.

Is it not related to the recent improvements of inferiors?  And the
recent texlive-default-updmap.cfg is just what spots out the flaw.


Cheers,
simon

Re: Problem with texlive-default-updmap.cfg

[Ludovic Courtès]

Hi,

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>>> The profile hook does perform a lazy package lookup with this:
>>>
>>> (define updmap.cfg
>>>     (module-ref (resolve-interface '(gnu packages tex))
>>>                 'texlive-default-updmap.cfg))
>>>
>>> I wonder if that ends up being evaluated outside of the inferior
>>> somehow.  That would be quite a bummer.
>>
>> Not sure I understand the context well enough, but yes,
>> ‘texlive-font-maps’ in (guix profiles) uses packages from the host Guix,
>> not from an inferior (it cannot know that inferiors are being used).
>>
>> Does that lead it to build incorrect font maps or things like that?
>
> Worse: there’s a mismatch between what one Guix wants and what the other
> offers.  Apparently the profile hook from the newer Guix is used, but
> the look up of texlive-default-updmap.cfg in (gnu packages tex) happens
> in the older Guix — which doesn’t *have* that package — and thus fails.

You mean the ‘module-ref’ above happens in the inferior’s module?

That’s not possible, unless there’s some extra load path trickery going
on.

Do you have a simple reproducer?

Thanks,
Ludo’.

Re: Problem with texlive-default-updmap.cfg

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>>>> The profile hook does perform a lazy package lookup with this:
>>>>
>>>> (define updmap.cfg
>>>>     (module-ref (resolve-interface '(gnu packages tex))
>>>>                 'texlive-default-updmap.cfg))
>>>>
>>>> I wonder if that ends up being evaluated outside of the inferior
>>>> somehow.  That would be quite a bummer.
>>>
>>> Not sure I understand the context well enough, but yes,
>>> ‘texlive-font-maps’ in (guix profiles) uses packages from the host Guix,
>>> not from an inferior (it cannot know that inferiors are being used).
>>>
>>> Does that lead it to build incorrect font maps or things like that?
>>
>> Worse: there’s a mismatch between what one Guix wants and what the other
>> offers.  Apparently the profile hook from the newer Guix is used, but
>> the look up of texlive-default-updmap.cfg in (gnu packages tex) happens
>> in the older Guix — which doesn’t *have* that package — and thus fails.
>
> You mean the ‘module-ref’ above happens in the inferior’s module?

No, the other way around: the profile hook from the inferior is used
(which requires a recently added package), but the module-ref does not
happen in the context of the inferior.  It happens in the context of the
*old* Guix (the “library Guix”), which does not have that package.

> That’s not possible, unless there’s some extra load path trickery going
> on.

Oh, you bet there’s load path trickery going on!
guix/extensions/workflow.scm sets the load path such that the Guix
modules at runtime are those of the Guix library at build time.

> Do you have a simple reproducer?

Not yet, but I’ll try to build one.

-- 
Ricardo

Re: Problem with texlive-default-updmap.cfg

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Would it help to change ‘texlive-font-maps’ to use
> ‘manifest-lookup-package’ instead to find its ‘updmap.cfg’ package?
> This is what several other hooks do, precisely to make sure they pick
> the matching package.

Unfortunately, this won’t work here, because texlive-default-updmap.cfg
is never installed.  It merely provides a template file, an input to the
updmap-sys tool.

-- 
Ricardo

Re: Problem with texlive-default-updmap.cfg

[Ricardo Wurmus]

Hi again Ontje,

my apologies for taking so long to get back to you.

> if I try to run this hello world workflow, I get the following error:
>
> process hi
>   # { echo "hi" }
>
> workflow do-the-thing
>   processes hi
>
> $ guix workflow run workflow.w
> info: .19 Loading workflow file `workflow.w'...
> info: .21 Computing workflow `do-the-thing'...
> error: 3.33 No variable named texlive-default-updmap.cfg in #<interface (gnu packages tex) 7f3ed7185d20>
>
> Is something borked in my setup?

Commit 2afadeea66e01f6d78e01eee910d2868f27531b8 works around this
problem by not sharing the same Guile process that was used to launch
“guix workflow”.

The first thing we do when the GWL extension is launched is to replace
the Guile load path.  We do this to ensure that the GWL uses the very
same Guix at runtime as the Guix we used to develop it.  This avoids a
lot of potential problems on users’ machines, where arbitrary versions
of Guix would be installed.  To avoid subtle breakage we just use a
well-known version of Guix as a library — and we use the “invoking Guix”
in an inferior to look up packages, so that users get the packages that
they expect.

Unfortunately, this doesn’t actually work as intended.  By the time
“guix workflow” is evaluated the Guile process will have already loaded
some of the Guile modules of the invoking Guix.  Replacing the load path
at that point does not have the intended effect because only those
modules we load *then* will belong to the well-known version of Guix —
whatever may have been loaded between “guix” starting and the moment
the “workflow” sub-command is invoked remains.

So to avoid “contamination” we now launch a new Guile process with a
well-known load path.  It’s not the prettiest solution, but it works
around the problem effectively.  It is a matter of philosophical debate
whether the GWL can still be called an “extension” to Guix when we
aren’t even using the same process and launch an independent Guile
process, but I’m okay with that because the exploration of this space is
one of the goals of the GWL :)

-- 
Ricardo

Re: Problem with texlive-default-updmap.cfg

[Ontje.Luensdorf]

Hi Ricardo,


Ricardo Wurmus <rekado@elephly.net> writes:

> Commit 2afadeea66e01f6d78e01eee910d2868f27531b8 works around this
> problem by not sharing the same Guile process that was used to launch
> “guix workflow”.

confirmed, my hello world workflow now works, thanks!

> So to avoid “contamination” we now launch a new Guile process with a
> well-known load path.  It’s not the prettiest solution, but it works
> around the problem effectively.  It is a matter of philosophical debate
> whether the GWL can still be called an “extension” to Guix when we
> aren’t even using the same process and launch an independent Guile
> process, but I’m okay with that because the exploration of this space is
> one of the goals of the GWL :)

I see, the new approach sounds robust to me. Just one question: If I
read the commit right, lib-guix is started as a second process using
system*. Isn't it possible to exec into lib-guix? If I understand the
problem correctly, the host-guix process isn't really required anymore
and can be replaced with lib-guix?

Thanks for fixing the issue & best regards,
Ontje

Re: Problem with texlive-default-updmap.cfg

[Ricardo Wurmus]

Hi Ontje,

> If I
> read the commit right, lib-guix is started as a second process using
> system*. Isn't it possible to exec into lib-guix?

I find system* easy to use, but replacing the process with exec would
probably work just fine.

> If I understand the
> problem correctly, the host-guix process isn't really required anymore
> and can be replaced with lib-guix?

Correct.

-- 
Ricardo