importers and input package lookup

importers and input package lookup

[Attila Lendvai]

dear Guixers,

there are two, independent namespaces:
1) the scheme one, and
2) the guix package repository.

when i work on an importer (golang), it skips the packages that are already available in 2), but then it has no clue under what variable name they are stored in 1), and in which scheme module.

should the dependency lists in the package forms be emitted as (specification->package "pkgname@0.1.0") forms?

in the current codebase this emission is done in the shared guix/import/utils.scm, in package-names->package-inputs. if i change this, it'll affect every other importer (BTW, i have converted it to use the new format).

is specification->package fast enough that it all doesn't matter?

a bit of a tangent here, and a higher-level perspective, but... shouldn't the package definition DSL have support for this? then most package descriptions could be using package specifications instead of scheme variables, and 1) could be phased out. or would that be more error prone? maybe with a tool that warns for the equivalent of undefined variable warnings?

background:

the reason i'm facing this is that i'm using the machinery in an idiosyncratic way: i want to have an isolated module where a --recursive --pin-versions of the entire transitive closure of a project is captured. when some package/version is available already in guix, the importer skips it, but then refers to it through a missing variable reference. and if i want to have two of such golang packages (Swarm's bee, and geth) that share many of the dependencies/versions...

i know the arguments why it's considered a bad idea from the perspective of Guix, but then miscompiling the ethereum client can lead to losing money.

- attila
PGP: 5D5F 45C7 DFCD 0A39

Re: importers and input package lookup

[Ludovic Courtès]

Hi!

Attila Lendvai <attila@lendvai.name> skribis:

> there are two, independent namespaces:
> 1) the scheme one, and
> 2) the guix package repository.
>
> when i work on an importer (golang), it skips the packages that are already available in 2), but then it has no clue under what variable name they are stored in 1), and in which scheme module.

Does the variable name matters though?  In general what matters for the
importer is whether the package/version exists, regardless of the
variable name.

> should the dependency lists in the package forms be emitted as (specification->package "pkgname@0.1.0") forms?

No, not for packages in Guix proper.

[...]

> a bit of a tangent here, and a higher-level perspective, but... shouldn't the package definition DSL have support for this? then most package descriptions could be using package specifications instead of scheme variables, and 1) could be phased out. or would that be more error prone? maybe with a tool that warns for the equivalent of undefined variable warnings?

Package specs are ambiguous compared to variable references (they depend
on external state, on the set of chosen channels, etc.) so in general we
want to refer to variables.

HTH!

Ludo’.

Re: importers and input package lookup

[Jelle Licht]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Attila Lendvai <attila@lendvai.name> skribis:
>
>> there are two, independent namespaces:
>> 1) the scheme one, and
>> 2) the guix package repository.
>>
>> when i work on an importer (golang), it skips the packages that are already available in 2), but then it has no clue under what variable name they are stored in 1), and in which scheme module.
>
> Does the variable name matters though?  In general what matters for the
> importer is whether the package/version exists, regardless of the
> variable name.

Since these packages often end up in a `inputs' list, the variable name
seems pretty relevant.

>> should the dependency lists in the package forms be emitted as (specification->package "pkgname@0.1.0") forms?
>
> No, not for packages in Guix proper.
>
> [...]
>
>> a bit of a tangent here, and a higher-level perspective, but... shouldn't the package definition DSL have support for this? then most package descriptions could be using package specifications instead of scheme variables, and 1) could be phased out. or would that be more error prone? maybe with a tool that warns for the equivalent of undefined variable warnings?
>
> Package specs are ambiguous compared to variable references (they depend
> on external state, on the set of chosen channels, etc.) so in general we
> want to refer to variables.

They are pretty explicit at one point in time: the moment we run an
importer. So at some point in the course of running the importer, we
know we have a package definition somewhere that is pretty much
equivalent to the result of (specification->package "pkgname@0.1.0");
can we somehow serialize this package object, perhaps using a heuristic?

If someone fiddles with GUIX_PACKAGE_PATH, GUILE_LOAD_PATH or any amount
of channels this impacts the result of running an importer, but
importers already can give different results depending on these
settings.